2024-09-11T07:09:28.574 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-11T07:09:28.590 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T07:09:28.668 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901210 branch: wip-guits-quincy-2024-09-10-2008-quincy description: orch:cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} email: gabrioux@ibm.com first_in_suite: false job_id: '7901210' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-quincy-2024-09-10-2008-quincy ceph: conf: client.0: debug ms: 1 debug objecter: 20 debug rados: 20 global: ms bind msgr2: false ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug ms: 1 debug osd: 20 osd debug reject backfill probability: 0.3 osd delete sleep: 1 osd max backfills: 3 osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - but it is still running - objects unfound and apparently lost - \(POOL_APP_NOT_ENABLED\) mon_bind_msgr2: false sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: root install: ceph: flavor: default sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-guits-quincy-2024-09-10-2008-quincy sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 2282 sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-quincy-2024-09-10-2008-quincy suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 targets: smithi022.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBES5d6c/6ReRhvSS5cmsFgJT19XUDAyMKDKt58O0YdnhpcvC+TDIEQX3O0uOMeIJQQAR8R536Pg2YedXF5onmiE= smithi104.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNxbmuk/JMUkg8El1ZwNh3VTWkKpMNqe2BPvDh4Jydz+ep/AsthG+VSJNcag43LETV+HvOML6UnjGVykspHEwjs= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install nvmetcli nvme-cli -y - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - thrashosds: chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 timeout: 1200 - full_sequential: - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_06:53:43 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-11T07:09:28.668 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa; will attempt to use it 2024-09-11T07:09:28.668 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa/tasks 2024-09-11T07:09:28.669 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-11T07:09:28.670 INFO:teuthology.task.internal:Checking packages... 2024-09-11T07:09:28.691 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd0e6828a2016d48cf25ad84064e50742bb1c39b9' 2024-09-11T07:09:28.691 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-11T07:09:28.691 INFO:teuthology.packaging:ref: None 2024-09-11T07:09:28.691 INFO:teuthology.packaging:tag: None 2024-09-11T07:09:28.691 INFO:teuthology.packaging:branch: wip-guits-quincy-2024-09-10-2008-quincy 2024-09-11T07:09:28.691 INFO:teuthology.packaging:sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:09:28.691 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-quincy-2024-09-10-2008-quincy 2024-09-11T07:09:28.841 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1623.gd0e6828a 2024-09-11T07:09:28.843 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-11T07:09:28.849 INFO:teuthology.task.internal:no buildpackages task found 2024-09-11T07:09:28.849 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-11T07:09:28.889 INFO:teuthology.task.internal:Saving configuration 2024-09-11T07:09:28.902 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-11T07:09:28.909 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-11T07:09:28.965 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901210', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 07:04:32.067827', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBES5d6c/6ReRhvSS5cmsFgJT19XUDAyMKDKt58O0YdnhpcvC+TDIEQX3O0uOMeIJQQAR8R536Pg2YedXF5onmiE='} 2024-09-11T07:09:28.985 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi104.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901210', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 07:04:32.066480', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNxbmuk/JMUkg8El1ZwNh3VTWkKpMNqe2BPvDh4Jydz+ep/AsthG+VSJNcag43LETV+HvOML6UnjGVykspHEwjs='} 2024-09-11T07:09:28.986 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-11T07:09:28.997 INFO:teuthology.task.internal:roles: ubuntu@smithi022.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-11T07:09:28.997 INFO:teuthology.task.internal:roles: ubuntu@smithi104.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-11T07:09:28.997 INFO:teuthology.run_tasks:Running task console_log... 2024-09-11T07:09:29.069 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1cb0a1bac0>, signals=[15]) 2024-09-11T07:09:29.069 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-11T07:09:29.077 INFO:teuthology.task.internal:Opening connections... 2024-09-11T07:09:29.077 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi022.front.sepia.ceph.com 2024-09-11T07:09:29.078 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:09:29.156 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi104.front.sepia.ceph.com 2024-09-11T07:09:29.157 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:09:29.231 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-11T07:09:29.243 DEBUG:teuthology.orchestra.run.smithi022:> uname -m 2024-09-11T07:09:29.261 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64 2024-09-11T07:09:29.262 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release 2024-09-11T07:09:29.320 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="CentOS Stream" 2024-09-11T07:09:29.320 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="9" 2024-09-11T07:09:29.320 INFO:teuthology.orchestra.run.smithi022.stdout:ID="centos" 2024-09-11T07:09:29.320 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="rhel fedora" 2024-09-11T07:09:29.320 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="9" 2024-09-11T07:09:29.320 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM_ID="platform:el9" 2024-09-11T07:09:29.320 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T07:09:29.320 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31" 2024-09-11T07:09:29.320 INFO:teuthology.orchestra.run.smithi022.stdout:LOGO="fedora-logo-icon" 2024-09-11T07:09:29.320 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T07:09:29.320 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://centos.org/" 2024-09-11T07:09:29.320 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T07:09:29.321 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T07:09:29.321 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T07:09:29.321 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2024-09-11T07:09:29.346 DEBUG:teuthology.orchestra.run.smithi104:> uname -m 2024-09-11T07:09:29.364 INFO:teuthology.orchestra.run.smithi104.stdout:x86_64 2024-09-11T07:09:29.365 DEBUG:teuthology.orchestra.run.smithi104:> cat /etc/os-release 2024-09-11T07:09:29.422 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="CentOS Stream" 2024-09-11T07:09:29.422 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="9" 2024-09-11T07:09:29.422 INFO:teuthology.orchestra.run.smithi104.stdout:ID="centos" 2024-09-11T07:09:29.423 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE="rhel fedora" 2024-09-11T07:09:29.423 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="9" 2024-09-11T07:09:29.423 INFO:teuthology.orchestra.run.smithi104.stdout:PLATFORM_ID="platform:el9" 2024-09-11T07:09:29.423 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T07:09:29.423 INFO:teuthology.orchestra.run.smithi104.stdout:ANSI_COLOR="0;31" 2024-09-11T07:09:29.423 INFO:teuthology.orchestra.run.smithi104.stdout:LOGO="fedora-logo-icon" 2024-09-11T07:09:29.423 INFO:teuthology.orchestra.run.smithi104.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T07:09:29.423 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://centos.org/" 2024-09-11T07:09:29.423 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T07:09:29.423 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T07:09:29.423 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T07:09:29.424 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server 2024-09-11T07:09:29.446 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-11T07:09:29.457 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-11T07:09:29.470 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-11T07:09:29.470 DEBUG:teuthology.orchestra.run.smithi022:> test '!' -e /home/ubuntu/cephtest 2024-09-11T07:09:29.473 DEBUG:teuthology.orchestra.run.smithi104:> test '!' -e /home/ubuntu/cephtest 2024-09-11T07:09:29.490 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-11T07:09:29.500 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-11T07:09:29.501 DEBUG:teuthology.orchestra.run.smithi022:> test -z $(ls -A /var/lib/ceph) 2024-09-11T07:09:29.531 DEBUG:teuthology.orchestra.run.smithi104:> test -z $(ls -A /var/lib/ceph) 2024-09-11T07:09:29.566 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-11T07:09:29.627 INFO:teuthology.run_tasks:Running task kernel... 2024-09-11T07:09:29.646 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-11T07:09:29.646 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-11T07:09:29.646 DEBUG:teuthology.orchestra.run.smithi022:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T07:09:29.647 DEBUG:teuthology.orchestra.run.smithi104:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T07:09:29.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:09:29.664 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2024-09-11T07:09:29.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:09:29.665 DEBUG:teuthology.orchestra.run.smithi104:> uname -r 2024-09-11T07:09:29.721 INFO:teuthology.orchestra.run.smithi022.stdout:5.14.0-503.el9.x86_64 2024-09-11T07:09:29.721 INFO:teuthology.task.kernel:Running kernel on smithi022: 5.14.0-503.el9.x86_64 2024-09-11T07:09:29.722 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum install -y kernel 2024-09-11T07:09:29.722 INFO:teuthology.orchestra.run.smithi104.stdout:5.14.0-503.el9.x86_64 2024-09-11T07:09:29.723 INFO:teuthology.task.kernel:Running kernel on smithi104: 5.14.0-503.el9.x86_64 2024-09-11T07:09:29.723 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum install -y kernel 2024-09-11T07:09:31.979 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-09-11T07:09:32.491 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - BaseOS 9.4 MB/s | 8.2 MB 00:00 2024-09-11T07:09:33.813 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-11T07:09:34.325 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-11T07:09:38.567 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-11T07:09:39.030 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 6.5 MB 00:01 2024-09-11T07:09:40.176 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 19 kB 00:00 2024-09-11T07:09:40.731 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-11T07:09:40.997 INFO:teuthology.orchestra.run.smithi104.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-11T07:09:41.510 INFO:teuthology.orchestra.run.smithi022.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-11T07:09:46.452 INFO:teuthology.orchestra.run.smithi104.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-11T07:09:47.166 INFO:teuthology.orchestra.run.smithi022.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-11T07:09:48.417 INFO:teuthology.orchestra.run.smithi104.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T07:09:48.417 INFO:teuthology.orchestra.run.smithi104.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T07:09:48.528 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:09:48.533 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:09:48.533 INFO:teuthology.orchestra.run.smithi104.stdout: Package Architecture Version Repository Size 2024-09-11T07:09:48.533 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:09:48.533 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-09-11T07:09:48.533 INFO:teuthology.orchestra.run.smithi104.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T07:09:48.533 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-09-11T07:09:48.534 INFO:teuthology.orchestra.run.smithi104.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T07:09:48.534 INFO:teuthology.orchestra.run.smithi104.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T07:09:48.534 INFO:teuthology.orchestra.run.smithi104.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T07:09:48.534 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:09:48.534 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-09-11T07:09:48.534 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:09:48.534 INFO:teuthology.orchestra.run.smithi104.stdout:Install 4 Packages 2024-09-11T07:09:48.534 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:09:48.535 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 79 M 2024-09-11T07:09:48.535 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 126 M 2024-09-11T07:09:48.535 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-09-11T07:09:48.712 INFO:teuthology.orchestra.run.smithi104.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 174 kB/s | 23 kB 00:00 2024-09-11T07:09:49.187 INFO:teuthology.orchestra.run.smithi104.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-09-11T07:09:49.256 INFO:teuthology.orchestra.run.smithi022.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T07:09:49.256 INFO:teuthology.orchestra.run.smithi022.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T07:09:49.355 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:09:49.366 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:09:49.366 INFO:teuthology.orchestra.run.smithi022.stdout: Package Architecture Version Repository Size 2024-09-11T07:09:49.366 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:09:49.366 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-11T07:09:49.366 INFO:teuthology.orchestra.run.smithi022.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T07:09:49.366 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-09-11T07:09:49.366 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T07:09:49.366 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T07:09:49.366 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T07:09:49.366 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:09:49.366 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-11T07:09:49.366 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:09:49.367 INFO:teuthology.orchestra.run.smithi022.stdout:Install 4 Packages 2024-09-11T07:09:49.367 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:09:49.367 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 79 M 2024-09-11T07:09:49.367 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 126 M 2024-09-11T07:09:49.367 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-11T07:09:49.645 INFO:teuthology.orchestra.run.smithi104.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 32 MB/s | 35 MB 00:01 2024-09-11T07:09:49.896 INFO:teuthology.orchestra.run.smithi104.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 24 MB/s | 29 MB 00:01 2024-09-11T07:09:49.896 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:09:49.896 INFO:teuthology.orchestra.run.smithi104.stdout:Total 58 MB/s | 79 MB 00:01 2024-09-11T07:09:49.942 INFO:teuthology.orchestra.run.smithi022.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 77 kB/s | 23 kB 00:00 2024-09-11T07:09:50.575 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-09-11T07:09:50.917 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-09-11T07:09:50.917 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-09-11T07:09:51.067 INFO:teuthology.orchestra.run.smithi022.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-11T07:09:51.864 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-09-11T07:09:51.864 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-09-11T07:09:51.901 INFO:teuthology.orchestra.run.smithi022.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 15 MB/s | 35 MB 00:02 2024-09-11T07:09:52.351 INFO:teuthology.orchestra.run.smithi022.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 12 MB/s | 29 MB 00:02 2024-09-11T07:09:52.351 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:09:52.352 INFO:teuthology.orchestra.run.smithi022.stdout:Total 26 MB/s | 79 MB 00:02 2024-09-11T07:09:53.057 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-11T07:09:53.415 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-11T07:09:53.415 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-11T07:09:54.304 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-09-11T07:09:54.409 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-11T07:09:54.409 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-11T07:09:54.979 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:09:55.062 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:09:56.966 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-11T07:09:57.646 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:09:57.727 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:09:57.859 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:09:57.997 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:10:00.520 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:10:00.656 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:10:02.818 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:10:02.939 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:10:05.782 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:10:05.824 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:10:07.761 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:10:10.964 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:10:45.939 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:10:45.979 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:10:46.264 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:10:46.265 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:10:46.265 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:10:46.265 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:10:47.115 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:10:47.115 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:10:47.115 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-09-11T07:10:47.115 INFO:teuthology.orchestra.run.smithi104.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T07:10:47.115 INFO:teuthology.orchestra.run.smithi104.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T07:10:47.115 INFO:teuthology.orchestra.run.smithi104.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T07:10:47.115 INFO:teuthology.orchestra.run.smithi104.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T07:10:47.116 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:10:47.116 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:10:47.491 DEBUG:teuthology.orchestra.run.smithi104:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:10:47.572 INFO:teuthology.orchestra.run.smithi104.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:10:47.572 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:10:47.573 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T07:10:47.573 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T07:10:47.573 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T07:10:47.573 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-11T07:10:47.573 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi104.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T07:10:47.573 DEBUG:teuthology.orchestra.run.smithi104:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:10:47.632 INFO:teuthology.orchestra.run.smithi104.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:10:47.633 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:10:47.633 DEBUG:teuthology.orchestra.run.smithi104:> sudo rpm -qi grub2-tools 2024-09-11T07:10:47.678 INFO:teuthology.orchestra.run.smithi104.stdout:Name : grub2-tools 2024-09-11T07:10:47.678 INFO:teuthology.orchestra.run.smithi104.stdout:Epoch : 1 2024-09-11T07:10:47.678 INFO:teuthology.orchestra.run.smithi104.stdout:Version : 2.06 2024-09-11T07:10:47.679 INFO:teuthology.orchestra.run.smithi104.stdout:Release : 82.el9 2024-09-11T07:10:47.679 INFO:teuthology.orchestra.run.smithi104.stdout:Architecture: x86_64 2024-09-11T07:10:47.679 INFO:teuthology.orchestra.run.smithi104.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T07:10:47.679 INFO:teuthology.orchestra.run.smithi104.stdout:Group : Unspecified 2024-09-11T07:10:47.679 INFO:teuthology.orchestra.run.smithi104.stdout:Size : 8274670 2024-09-11T07:10:47.679 INFO:teuthology.orchestra.run.smithi104.stdout:License : GPLv3+ 2024-09-11T07:10:47.679 INFO:teuthology.orchestra.run.smithi104.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T07:10:47.679 INFO:teuthology.orchestra.run.smithi104.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T07:10:47.679 INFO:teuthology.orchestra.run.smithi104.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T07:10:47.679 INFO:teuthology.orchestra.run.smithi104.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T07:10:47.679 INFO:teuthology.orchestra.run.smithi104.stdout:Packager : builder@centos.org 2024-09-11T07:10:47.679 INFO:teuthology.orchestra.run.smithi104.stdout:Vendor : CentOS 2024-09-11T07:10:47.680 INFO:teuthology.orchestra.run.smithi104.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T07:10:47.680 INFO:teuthology.orchestra.run.smithi104.stdout:Summary : Support tools for GRUB. 2024-09-11T07:10:47.680 INFO:teuthology.orchestra.run.smithi104.stdout:Description : 2024-09-11T07:10:47.680 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:10:47.680 INFO:teuthology.orchestra.run.smithi104.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T07:10:47.680 INFO:teuthology.orchestra.run.smithi104.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T07:10:47.680 INFO:teuthology.orchestra.run.smithi104.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T07:10:47.680 INFO:teuthology.orchestra.run.smithi104.stdout:hardware devices. 2024-09-11T07:10:47.680 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:10:47.680 INFO:teuthology.orchestra.run.smithi104.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T07:10:47.681 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T07:10:47.681 INFO:teuthology.task.kernel:Updating grub on smithi104 to boot 5.14.0-505.el9.x86_64 2024-09-11T07:10:47.682 DEBUG:teuthology.orchestra.run.smithi104:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T07:10:48.303 INFO:teuthology.orchestra.run.smithi104.stderr:Generating grub configuration file ... 2024-09-11T07:10:49.619 INFO:teuthology.orchestra.run.smithi104.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T07:10:49.654 INFO:teuthology.orchestra.run.smithi104.stderr:done 2024-09-11T07:10:49.657 DEBUG:teuthology.orchestra.run.smithi104:> mktemp 2024-09-11T07:10:49.672 INFO:teuthology.orchestra.run.smithi104.stdout:/tmp/tmp.q1P2bTIN2F 2024-09-11T07:10:49.673 DEBUG:teuthology.orchestra.run.smithi104:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.q1P2bTIN2F 2024-09-11T07:10:49.739 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 0666 /tmp/tmp.q1P2bTIN2F 2024-09-11T07:10:49.870 DEBUG:teuthology.orchestra.remote:smithi104:/tmp/tmp.q1P2bTIN2F is 6KB 2024-09-11T07:10:49.920 DEBUG:teuthology.orchestra.run.smithi104:> rm -fr /tmp/tmp.q1P2bTIN2F 2024-09-11T07:10:49.934 DEBUG:teuthology.orchestra.run.smithi104:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T07:10:49.997 INFO:teuthology.orchestra.run.smithi104.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T07:10:49.997 INFO:teuthology.orchestra.run.smithi104.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T07:10:49.997 INFO:teuthology.orchestra.run.smithi104.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T07:10:49.997 INFO:teuthology.orchestra.run.smithi104.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T07:10:49.999 DEBUG:teuthology.orchestra.run.smithi104:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T07:10:50.009 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:10:50.050 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:10:50.152 DEBUG:teuthology.orchestra.run.smithi104:> sudo shutdown -r now 2024-09-11T07:10:50.417 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:10:50.417 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:10:50.417 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:10:50.417 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:10:51.287 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:10:51.288 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:10:51.288 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-11T07:10:51.288 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T07:10:51.288 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T07:10:51.289 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T07:10:51.289 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T07:10:51.290 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:10:51.290 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:10:51.632 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:10:51.729 INFO:teuthology.orchestra.run.smithi022.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:10:51.729 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:10:51.729 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T07:10:51.729 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T07:10:51.729 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T07:10:51.730 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-11T07:10:51.730 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi022.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T07:10:51.730 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:10:51.787 INFO:teuthology.orchestra.run.smithi022.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:10:51.788 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:10:51.788 DEBUG:teuthology.orchestra.run.smithi022:> sudo rpm -qi grub2-tools 2024-09-11T07:10:51.827 INFO:teuthology.orchestra.run.smithi022.stdout:Name : grub2-tools 2024-09-11T07:10:51.828 INFO:teuthology.orchestra.run.smithi022.stdout:Epoch : 1 2024-09-11T07:10:51.828 INFO:teuthology.orchestra.run.smithi022.stdout:Version : 2.06 2024-09-11T07:10:51.828 INFO:teuthology.orchestra.run.smithi022.stdout:Release : 82.el9 2024-09-11T07:10:51.828 INFO:teuthology.orchestra.run.smithi022.stdout:Architecture: x86_64 2024-09-11T07:10:51.828 INFO:teuthology.orchestra.run.smithi022.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T07:10:51.828 INFO:teuthology.orchestra.run.smithi022.stdout:Group : Unspecified 2024-09-11T07:10:51.828 INFO:teuthology.orchestra.run.smithi022.stdout:Size : 8274670 2024-09-11T07:10:51.828 INFO:teuthology.orchestra.run.smithi022.stdout:License : GPLv3+ 2024-09-11T07:10:51.828 INFO:teuthology.orchestra.run.smithi022.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T07:10:51.828 INFO:teuthology.orchestra.run.smithi022.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T07:10:51.829 INFO:teuthology.orchestra.run.smithi022.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T07:10:51.829 INFO:teuthology.orchestra.run.smithi022.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T07:10:51.829 INFO:teuthology.orchestra.run.smithi022.stdout:Packager : builder@centos.org 2024-09-11T07:10:51.829 INFO:teuthology.orchestra.run.smithi022.stdout:Vendor : CentOS 2024-09-11T07:10:51.829 INFO:teuthology.orchestra.run.smithi022.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T07:10:51.829 INFO:teuthology.orchestra.run.smithi022.stdout:Summary : Support tools for GRUB. 2024-09-11T07:10:51.829 INFO:teuthology.orchestra.run.smithi022.stdout:Description : 2024-09-11T07:10:51.829 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:10:51.829 INFO:teuthology.orchestra.run.smithi022.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T07:10:51.829 INFO:teuthology.orchestra.run.smithi022.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T07:10:51.829 INFO:teuthology.orchestra.run.smithi022.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T07:10:51.829 INFO:teuthology.orchestra.run.smithi022.stdout:hardware devices. 2024-09-11T07:10:51.829 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:10:51.829 INFO:teuthology.orchestra.run.smithi022.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T07:10:51.830 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T07:10:51.830 INFO:teuthology.task.kernel:Updating grub on smithi022 to boot 5.14.0-505.el9.x86_64 2024-09-11T07:10:51.830 DEBUG:teuthology.orchestra.run.smithi022:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T07:10:52.504 INFO:teuthology.orchestra.run.smithi022.stderr:Generating grub configuration file ... 2024-09-11T07:10:53.786 INFO:teuthology.orchestra.run.smithi022.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T07:10:53.826 INFO:teuthology.orchestra.run.smithi022.stderr:done 2024-09-11T07:10:53.829 DEBUG:teuthology.orchestra.run.smithi022:> mktemp 2024-09-11T07:10:53.844 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.OZPrEaP4H5 2024-09-11T07:10:53.844 DEBUG:teuthology.orchestra.run.smithi022:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.OZPrEaP4H5 2024-09-11T07:10:53.910 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.OZPrEaP4H5 2024-09-11T07:10:54.046 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.OZPrEaP4H5 is 6KB 2024-09-11T07:10:54.095 DEBUG:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.OZPrEaP4H5 2024-09-11T07:10:54.110 DEBUG:teuthology.orchestra.run.smithi022:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T07:10:54.172 INFO:teuthology.orchestra.run.smithi022.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T07:10:54.173 INFO:teuthology.orchestra.run.smithi022.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T07:10:54.173 INFO:teuthology.orchestra.run.smithi022.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T07:10:54.173 INFO:teuthology.orchestra.run.smithi022.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T07:10:54.174 DEBUG:teuthology.orchestra.run.smithi022:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T07:10:54.322 DEBUG:teuthology.orchestra.run.smithi022:> sudo shutdown -r now 2024-09-11T07:11:20.181 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-11T07:11:20.220 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-09-11T07:11:20.222 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:11:24.329 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-11T07:11:24.329 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-09-11T07:11:24.330 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:12:20.280 DEBUG:teuthology.orchestra.remote:timed out 2024-09-11T07:12:24.335 DEBUG:teuthology.orchestra.remote:timed out 2024-09-11T07:12:29.285 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-09-11T07:12:29.286 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:12:29.660 DEBUG:teuthology.orchestra.run.smithi104:> true 2024-09-11T07:12:30.193 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-09-11T07:12:30.193 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-11T07:12:30.193 DEBUG:teuthology.orchestra.run.smithi104:> uname -r 2024-09-11T07:12:30.207 INFO:teuthology.orchestra.run.smithi104.stdout:5.14.0-505.el9.x86_64 2024-09-11T07:12:30.208 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T07:12:30.208 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T07:12:30.208 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T07:12:31.208 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-11T07:12:31.209 DEBUG:teuthology.orchestra.run.smithi104:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T07:12:31.294 INFO:teuthology.orchestra.run.smithi104.stdout:ttyS1 2024-09-11T07:12:31.326 DEBUG:teuthology.parallel:result is None 2024-09-11T07:12:33.336 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-09-11T07:12:33.337 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:12:33.763 DEBUG:teuthology.orchestra.run.smithi022:> true 2024-09-11T07:12:34.110 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-09-11T07:12:34.110 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-11T07:12:34.111 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2024-09-11T07:12:34.126 INFO:teuthology.orchestra.run.smithi022.stdout:5.14.0-505.el9.x86_64 2024-09-11T07:12:34.126 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T07:12:34.126 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T07:12:34.126 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T07:12:35.127 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-11T07:12:35.127 DEBUG:teuthology.orchestra.run.smithi022:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T07:12:35.223 INFO:teuthology.orchestra.run.smithi022.stdout:ttyS1 2024-09-11T07:12:35.251 DEBUG:teuthology.parallel:result is None 2024-09-11T07:12:35.251 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T07:12:35.262 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T07:12:35.262 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T07:12:35.264 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T07:12:35.279 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T07:12:35.286 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T07:12:35.294 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T07:12:35.294 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T07:12:35.317 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T07:12:35.342 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T07:12:35.348 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T07:12:35.349 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T07:12:35.373 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T07:12:35.404 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:12:35.415 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:12:35.435 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:12:35.446 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:12:35.448 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T07:12:35.455 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T07:12:35.455 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T07:12:35.458 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T07:12:35.483 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T07:12:35.494 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T07:12:35.494 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T07:12:35.522 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T07:12:35.537 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:12:35.616 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:12:35.653 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:12:35.733 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:12:35.772 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-11T07:12:35.773 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T07:12:35.835 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:12:35.879 DEBUG:teuthology.orchestra.run.smithi104:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:12:35.947 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:12:35.988 DEBUG:teuthology.orchestra.run.smithi104:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:12:36.028 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-11T07:12:36.028 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T07:12:36.093 DEBUG:teuthology.orchestra.run.smithi022:> sudo service rsyslog restart 2024-09-11T07:12:36.096 DEBUG:teuthology.orchestra.run.smithi104:> sudo service rsyslog restart 2024-09-11T07:12:36.154 INFO:teuthology.orchestra.run.smithi022.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T07:12:36.296 INFO:teuthology.orchestra.run.smithi104.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T07:12:36.833 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T07:12:36.841 INFO:teuthology.task.internal:Starting timer... 2024-09-11T07:12:36.841 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T07:12:36.852 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T07:12:36.860 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-11T07:12:36.860 DEBUG:teuthology.orchestra.run.smithi022:> sudo service auditd rotate 2024-09-11T07:12:36.949 INFO:teuthology.orchestra.run.smithi022.stdout:Rotating logs: 2024-09-11T07:12:36.951 DEBUG:teuthology.orchestra.run.smithi104:> sudo service auditd rotate 2024-09-11T07:12:37.061 INFO:teuthology.orchestra.run.smithi104.stdout:Rotating logs: 2024-09-11T07:12:37.067 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T07:12:37.067 DEBUG:teuthology.orchestra.run.smithi022:> /usr/sbin/getenforce 2024-09-11T07:12:37.088 INFO:teuthology.orchestra.run.smithi022.stdout:Permissive 2024-09-11T07:12:37.088 DEBUG:teuthology.orchestra.run.smithi104:> /usr/sbin/getenforce 2024-09-11T07:12:37.143 INFO:teuthology.orchestra.run.smithi104.stdout:Permissive 2024-09-11T07:12:37.143 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi022.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi104.front.sepia.ceph.com': 'permissive'} 2024-09-11T07:12:37.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T07:12:37.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:12:37.169 DEBUG:teuthology.orchestra.run.smithi104:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T07:12:37.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:12:37.249 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T07:12:37.249 DEBUG:teuthology.orchestra.run.smithi022:> sudo /usr/sbin/setenforce permissive 2024-09-11T07:12:37.293 DEBUG:teuthology.orchestra.run.smithi104:> sudo /usr/sbin/setenforce permissive 2024-09-11T07:12:37.373 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T07:12:37.383 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-11T07:12:37.444 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-11T07:12:37.562 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T07:12:37.573 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T07:12:37.574 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi022.front.sepia.ceph.com,smithi104.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T07:16:20.606 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi022.front.sepia.ceph.com'), Remote(name='ubuntu@smithi104.front.sepia.ceph.com')] 2024-09-11T07:16:20.607 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-09-11T07:16:20.608 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:16:20.688 DEBUG:teuthology.orchestra.run.smithi022:> true 2024-09-11T07:16:20.769 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-09-11T07:16:20.769 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-09-11T07:16:20.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:16:20.844 DEBUG:teuthology.orchestra.run.smithi104:> true 2024-09-11T07:16:20.920 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-09-11T07:16:20.920 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T07:16:20.930 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T07:16:20.931 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T07:16:20.931 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T07:16:20.934 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T07:16:20.934 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T07:16:20.963 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T07:16:20.980 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T07:16:20.988 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T07:16:21.002 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T07:16:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: ntpd: command not found 2024-09-11T07:16:21.016 INFO:teuthology.orchestra.run.smithi022.stdout:506 Cannot talk to daemon 2024-09-11T07:16:21.026 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: ntpd: command not found 2024-09-11T07:16:21.031 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T07:16:21.037 INFO:teuthology.orchestra.run.smithi104.stdout:506 Cannot talk to daemon 2024-09-11T07:16:21.046 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T07:16:21.052 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T07:16:21.066 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T07:16:21.101 INFO:teuthology.orchestra.run.smithi022.stderr:bash: line 1: ntpq: command not found 2024-09-11T07:16:21.105 INFO:teuthology.orchestra.run.smithi022.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T07:16:21.105 INFO:teuthology.orchestra.run.smithi022.stdout:=============================================================================== 2024-09-11T07:16:21.105 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:16:21.105 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:16:21.105 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:16:21.105 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:16:21.123 INFO:teuthology.orchestra.run.smithi104.stderr:bash: line 1: ntpq: command not found 2024-09-11T07:16:21.126 INFO:teuthology.orchestra.run.smithi104.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T07:16:21.126 INFO:teuthology.orchestra.run.smithi104.stdout:=============================================================================== 2024-09-11T07:16:21.126 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:16:21.126 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:16:21.127 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:16:21.127 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:16:21.127 INFO:teuthology.run_tasks:Running task pexec... 2024-09-11T07:16:21.136 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-11T07:16:21.137 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi022.front.sepia.ceph.com 2024-09-11T07:16:21.137 DEBUG:teuthology.orchestra.run.smithi022:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-11T07:16:21.137 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi104.front.sepia.ceph.com 2024-09-11T07:16:21.137 DEBUG:teuthology.orchestra.run.smithi104:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-11T07:16:21.513 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:16:21.517 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:16:21.526 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:16:21.526 INFO:teuthology.orchestra.run.smithi022.stdout: Package Architecture Version Repository Size 2024-09-11T07:16:21.526 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:16:21.526 INFO:teuthology.orchestra.run.smithi022.stdout:Removing: 2024-09-11T07:16:21.526 INFO:teuthology.orchestra.run.smithi022.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-11T07:16:21.526 INFO:teuthology.orchestra.run.smithi022.stdout:Removing unused dependencies: 2024-09-11T07:16:21.526 INFO:teuthology.orchestra.run.smithi022.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-11T07:16:21.526 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:16:21.526 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-11T07:16:21.527 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:16:21.527 INFO:teuthology.orchestra.run.smithi022.stdout:Remove 2 Packages 2024-09-11T07:16:21.527 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:16:21.527 INFO:teuthology.orchestra.run.smithi022.stdout:Freed space: 5.5 M 2024-09-11T07:16:21.527 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-11T07:16:21.531 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:16:21.531 INFO:teuthology.orchestra.run.smithi104.stdout: Package Architecture Version Repository Size 2024-09-11T07:16:21.531 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:16:21.531 INFO:teuthology.orchestra.run.smithi104.stdout:Removing: 2024-09-11T07:16:21.531 INFO:teuthology.orchestra.run.smithi104.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-11T07:16:21.531 INFO:teuthology.orchestra.run.smithi104.stdout:Removing unused dependencies: 2024-09-11T07:16:21.531 INFO:teuthology.orchestra.run.smithi104.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-11T07:16:21.531 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:16:21.531 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-09-11T07:16:21.532 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:16:21.532 INFO:teuthology.orchestra.run.smithi104.stdout:Remove 2 Packages 2024-09-11T07:16:21.532 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:16:21.532 INFO:teuthology.orchestra.run.smithi104.stdout:Freed space: 5.5 M 2024-09-11T07:16:21.532 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-09-11T07:16:21.532 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-11T07:16:21.532 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-11T07:16:21.535 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-09-11T07:16:21.535 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-09-11T07:16:21.603 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-09-11T07:16:21.603 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-09-11T07:16:21.603 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-11T07:16:21.603 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-11T07:16:21.748 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-11T07:16:21.757 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-09-11T07:16:21.828 INFO:teuthology.orchestra.run.smithi022.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-11T07:16:21.837 INFO:teuthology.orchestra.run.smithi104.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-11T07:16:21.883 INFO:teuthology.orchestra.run.smithi022.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T07:16:21.892 INFO:teuthology.orchestra.run.smithi104.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T07:16:22.372 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T07:16:22.372 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-11T07:16:22.385 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T07:16:22.385 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-11T07:16:22.665 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-11T07:16:22.666 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:16:22.666 INFO:teuthology.orchestra.run.smithi022.stdout:Removed: 2024-09-11T07:16:22.666 INFO:teuthology.orchestra.run.smithi022.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T07:16:22.666 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:16:22.666 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:16:22.668 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-11T07:16:22.668 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:16:22.668 INFO:teuthology.orchestra.run.smithi104.stdout:Removed: 2024-09-11T07:16:22.668 INFO:teuthology.orchestra.run.smithi104.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T07:16:22.668 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:16:22.669 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:16:23.464 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:00:51 ago on Wed 11 Sep 2024 07:15:32 AM UTC. 2024-09-11T07:16:23.484 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:00:45 ago on Wed 11 Sep 2024 07:15:38 AM UTC. 2024-09-11T07:16:23.598 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:16:23.599 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:16:23.599 INFO:teuthology.orchestra.run.smithi104.stdout: Package Architecture Version Repository Size 2024-09-11T07:16:23.599 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:16:23.599 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-09-11T07:16:23.599 INFO:teuthology.orchestra.run.smithi104.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-11T07:16:23.600 INFO:teuthology.orchestra.run.smithi104.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-11T07:16:23.600 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-09-11T07:16:23.600 INFO:teuthology.orchestra.run.smithi104.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-11T07:16:23.600 INFO:teuthology.orchestra.run.smithi104.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-11T07:16:23.600 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-11T07:16:23.600 INFO:teuthology.orchestra.run.smithi104.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-11T07:16:23.600 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:16:23.600 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-09-11T07:16:23.600 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:16:23.600 INFO:teuthology.orchestra.run.smithi104.stdout:Install 6 Packages 2024-09-11T07:16:23.600 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:16:23.606 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 2.0 M 2024-09-11T07:16:23.607 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 9.2 M 2024-09-11T07:16:23.607 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-09-11T07:16:23.621 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:16:23.622 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:16:23.622 INFO:teuthology.orchestra.run.smithi022.stdout: Package Architecture Version Repository Size 2024-09-11T07:16:23.622 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:16:23.623 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-11T07:16:23.623 INFO:teuthology.orchestra.run.smithi022.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-11T07:16:23.623 INFO:teuthology.orchestra.run.smithi022.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-11T07:16:23.623 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-09-11T07:16:23.623 INFO:teuthology.orchestra.run.smithi022.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-11T07:16:23.623 INFO:teuthology.orchestra.run.smithi022.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-11T07:16:23.623 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-11T07:16:23.623 INFO:teuthology.orchestra.run.smithi022.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-11T07:16:23.623 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:16:23.623 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-11T07:16:23.623 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:16:23.623 INFO:teuthology.orchestra.run.smithi022.stdout:Install 6 Packages 2024-09-11T07:16:23.623 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:16:23.633 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 2.0 M 2024-09-11T07:16:23.633 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 9.2 M 2024-09-11T07:16:23.633 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-11T07:16:23.848 INFO:teuthology.orchestra.run.smithi022.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 408 kB/s | 44 kB 00:00 2024-09-11T07:16:23.873 INFO:teuthology.orchestra.run.smithi022.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 789 kB/s | 106 kB 00:00 2024-09-11T07:16:23.906 INFO:teuthology.orchestra.run.smithi022.stdout:(3/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 5.3 MB/s | 906 kB 00:00 2024-09-11T07:16:23.942 INFO:teuthology.orchestra.run.smithi022.stdout:(4/6): python3-configshell-1.1.30-1.el9.noarch. 758 kB/s | 72 kB 00:00 2024-09-11T07:16:23.965 INFO:teuthology.orchestra.run.smithi022.stdout:(5/6): python3-kmod-0.9-32.el9.x86_64.rpm 912 kB/s | 84 kB 00:00 2024-09-11T07:16:23.973 INFO:teuthology.orchestra.run.smithi104.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 525 kB/s | 44 kB 00:00 2024-09-11T07:16:24.006 INFO:teuthology.orchestra.run.smithi104.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 896 kB/s | 106 kB 00:00 2024-09-11T07:16:24.007 INFO:teuthology.orchestra.run.smithi022.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 8.1 MB/s | 837 kB 00:00 2024-09-11T07:16:24.007 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:16:24.007 INFO:teuthology.orchestra.run.smithi022.stdout:Total 5.4 MB/s | 2.0 MB 00:00 2024-09-11T07:16:24.040 INFO:teuthology.orchestra.run.smithi104.stdout:(3/6): python3-configshell-1.1.30-1.el9.noarch. 1.0 MB/s | 72 kB 00:00 2024-09-11T07:16:24.073 INFO:teuthology.orchestra.run.smithi104.stdout:(4/6): python3-kmod-0.9-32.el9.x86_64.rpm 1.2 MB/s | 84 kB 00:00 2024-09-11T07:16:24.084 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-11T07:16:24.099 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-11T07:16:24.099 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-11T07:16:24.149 INFO:teuthology.orchestra.run.smithi104.stdout:(5/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 7.5 MB/s | 837 kB 00:00 2024-09-11T07:16:24.199 INFO:teuthology.orchestra.run.smithi104.stdout:(6/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 2.8 MB/s | 906 kB 00:00 2024-09-11T07:16:24.199 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:16:24.199 INFO:teuthology.orchestra.run.smithi104.stdout:Total 3.4 MB/s | 2.0 MB 00:00 2024-09-11T07:16:24.273 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-09-11T07:16:24.287 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-09-11T07:16:24.287 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-09-11T07:16:24.385 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-11T07:16:24.385 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-11T07:16:24.571 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-09-11T07:16:24.571 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-09-11T07:16:24.824 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-11T07:16:24.911 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-11T07:16:24.945 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-11T07:16:24.973 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-11T07:16:24.988 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-09-11T07:16:25.076 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-11T07:16:25.116 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-11T07:16:25.148 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-11T07:16:25.196 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-11T07:16:25.238 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-11T07:16:25.353 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-11T07:16:25.387 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-11T07:16:25.716 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-11T07:16:25.752 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-11T07:16:25.841 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-11T07:16:25.876 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-11T07:16:26.718 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-11T07:16:26.718 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-11T07:16:26.718 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-11T07:16:26.718 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-11T07:16:26.718 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-11T07:16:26.718 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-11T07:16:26.838 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-11T07:16:26.839 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-11T07:16:26.839 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-11T07:16:26.839 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-11T07:16:26.839 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-11T07:16:26.839 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-11T07:16:27.066 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-11T07:16:27.066 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:16:27.066 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-11T07:16:27.066 INFO:teuthology.orchestra.run.smithi022.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T07:16:27.066 INFO:teuthology.orchestra.run.smithi022.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-11T07:16:27.066 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-11T07:16:27.066 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:16:27.066 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:16:27.231 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-11T07:16:27.231 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:16:27.231 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-09-11T07:16:27.231 INFO:teuthology.orchestra.run.smithi104.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T07:16:27.231 INFO:teuthology.orchestra.run.smithi104.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-11T07:16:27.231 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-11T07:16:27.231 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:16:27.232 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:16:27.372 DEBUG:teuthology.parallel:result is None 2024-09-11T07:16:27.546 DEBUG:teuthology.parallel:result is None 2024-09-11T07:16:27.547 INFO:teuthology.run_tasks:Running task install... 2024-09-11T07:16:27.555 DEBUG:teuthology.task.install:project ceph 2024-09-11T07:16:27.555 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'}} 2024-09-11T07:16:27.555 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'} 2024-09-11T07:16:27.556 INFO:teuthology.task.install:Using flavor: default 2024-09-11T07:16:27.561 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-11T07:16:27.561 INFO:teuthology.task.install:extra packages: [] 2024-09-11T07:16:27.561 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9', 'tag': None, 'wait_for_package': False} 2024-09-11T07:16:27.562 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:16:27.564 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9', 'tag': None, 'wait_for_package': False} 2024-09-11T07:16:27.564 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:16:27.707 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/ 2024-09-11T07:16:27.708 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1623.gd0e6828a 2024-09-11T07:16:27.726 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/ 2024-09-11T07:16:27.726 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1623.gd0e6828a 2024-09-11T07:16:27.818 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T07:16:27.819 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-11T07:16:27.819 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T07:16:27.835 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T07:16:27.835 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-11T07:16:27.835 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T07:16:27.861 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-11T07:16:27.862 DEBUG:teuthology.orchestra.run.smithi022:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d0e6828a2016d48cf25ad84064e50742bb1c39b9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T07:16:27.863 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-11T07:16:27.863 DEBUG:teuthology.orchestra.run.smithi104:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d0e6828a2016d48cf25ad84064e50742bb1c39b9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T07:16:27.928 DEBUG:teuthology.orchestra.run.smithi104:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T07:16:27.933 DEBUG:teuthology.orchestra.run.smithi022:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T07:16:28.003 DEBUG:teuthology.orchestra.run.smithi104:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T07:16:28.013 DEBUG:teuthology.orchestra.run.smithi022:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T07:16:28.069 INFO:teuthology.orchestra.run.smithi104.stdout:check_obsoletes = 1 2024-09-11T07:16:28.070 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum clean all 2024-09-11T07:16:28.081 INFO:teuthology.orchestra.run.smithi022.stdout:check_obsoletes = 1 2024-09-11T07:16:28.083 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum clean all 2024-09-11T07:16:28.297 INFO:teuthology.orchestra.run.smithi104.stdout:45 files removed 2024-09-11T07:16:28.319 INFO:teuthology.orchestra.run.smithi022.stdout:45 files removed 2024-09-11T07:16:28.325 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-radosgw 2024-09-11T07:16:28.346 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-radosgw 2024-09-11T07:16:28.864 INFO:teuthology.orchestra.run.smithi104.stdout:ceph packages for x86_64 302 kB/s | 77 kB 00:00 2024-09-11T07:16:28.929 INFO:teuthology.orchestra.run.smithi022.stdout:ceph packages for x86_64 275 kB/s | 77 kB 00:00 2024-09-11T07:16:29.139 INFO:teuthology.orchestra.run.smithi104.stdout:ceph noarch packages 46 kB/s | 11 kB 00:00 2024-09-11T07:16:29.222 INFO:teuthology.orchestra.run.smithi022.stdout:ceph noarch packages 44 kB/s | 11 kB 00:00 2024-09-11T07:16:29.407 INFO:teuthology.orchestra.run.smithi104.stdout:ceph source packages 7.7 kB/s | 1.8 kB 00:00 2024-09-11T07:16:29.497 INFO:teuthology.orchestra.run.smithi022.stdout:ceph source packages 7.5 kB/s | 1.8 kB 00:00 2024-09-11T07:16:30.164 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-11T07:16:31.081 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.2 MB 00:01 2024-09-11T07:16:32.440 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - AppStream 18 MB/s | 20 MB 00:01 2024-09-11T07:16:34.540 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - AppStream 8.6 MB/s | 20 MB 00:02 2024-09-11T07:16:38.058 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.5 MB 00:01 2024-09-11T07:16:39.007 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - CRB 9.7 MB/s | 6.5 MB 00:00 2024-09-11T07:16:39.791 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-11T07:16:40.612 INFO:teuthology.orchestra.run.smithi022.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-11T07:16:41.366 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 19 kB 00:00 2024-09-11T07:16:42.261 INFO:teuthology.orchestra.run.smithi104.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-11T07:16:46.293 INFO:teuthology.orchestra.run.smithi022.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-11T07:16:47.709 INFO:teuthology.orchestra.run.smithi104.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-09-11T07:16:48.820 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:16:48.822 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:16:48.822 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-09-11T07:16:48.822 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:16:48.822 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-11T07:16:48.823 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-radosgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 14 M 2024-09-11T07:16:48.823 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrading: 2024-09-11T07:16:48.823 INFO:teuthology.orchestra.run.smithi022.stdout: librados2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.1 M 2024-09-11T07:16:48.823 INFO:teuthology.orchestra.run.smithi022.stdout: librbd1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T07:16:48.823 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-09-11T07:16:48.823 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 5.0 M 2024-09-11T07:16:48.823 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 23 M 2024-09-11T07:16:48.823 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-selinux x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T07:16:48.823 INFO:teuthology.orchestra.run.smithi022.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T07:16:48.823 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 636 k 2024-09-11T07:16:48.823 INFO:teuthology.orchestra.run.smithi022.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T07:16:48.823 INFO:teuthology.orchestra.run.smithi022.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T07:16:48.824 INFO:teuthology.orchestra.run.smithi022.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T07:16:48.824 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 457 k 2024-09-11T07:16:48.824 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T07:16:48.824 INFO:teuthology.orchestra.run.smithi022.stdout: librgw2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 4.9 M 2024-09-11T07:16:48.824 INFO:teuthology.orchestra.run.smithi022.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T07:16:48.824 INFO:teuthology.orchestra.run.smithi022.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T07:16:48.824 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-argparse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 45 k 2024-09-11T07:16:48.824 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 120 k 2024-09-11T07:16:48.824 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cephfs x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 156 k 2024-09-11T07:16:48.824 INFO:teuthology.orchestra.run.smithi022.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T07:16:48.824 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rados x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 320 k 2024-09-11T07:16:48.824 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 294 k 2024-09-11T07:16:48.825 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 99 k 2024-09-11T07:16:48.825 INFO:teuthology.orchestra.run.smithi022.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T07:16:48.825 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:16:48.825 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-11T07:16:48.825 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:16:48.825 INFO:teuthology.orchestra.run.smithi022.stdout:Install 22 Packages 2024-09-11T07:16:48.825 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrade 2 Packages 2024-09-11T07:16:48.825 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:16:48.826 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 58 M 2024-09-11T07:16:48.826 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-11T07:16:49.861 INFO:teuthology.orchestra.run.smithi022.stdout:(1/24): ceph-base-17.2.7-1623.gd0e6828a.el9.x86 8.3 MB/s | 5.0 MB 00:00 2024-09-11T07:16:49.945 INFO:teuthology.orchestra.run.smithi022.stdout:(2/24): ceph-selinux-17.2.7-1623.gd0e6828a.el9. 302 kB/s | 25 kB 00:00 2024-09-11T07:16:50.048 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:16:50.049 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:16:50.049 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-09-11T07:16:50.049 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:16:50.049 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-09-11T07:16:50.049 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-radosgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 14 M 2024-09-11T07:16:50.049 INFO:teuthology.orchestra.run.smithi104.stdout:Upgrading: 2024-09-11T07:16:50.049 INFO:teuthology.orchestra.run.smithi104.stdout: librados2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.1 M 2024-09-11T07:16:50.049 INFO:teuthology.orchestra.run.smithi104.stdout: librbd1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T07:16:50.049 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-09-11T07:16:50.050 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 5.0 M 2024-09-11T07:16:50.050 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 23 M 2024-09-11T07:16:50.050 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-selinux x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T07:16:50.050 INFO:teuthology.orchestra.run.smithi104.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T07:16:50.050 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 636 k 2024-09-11T07:16:50.050 INFO:teuthology.orchestra.run.smithi104.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T07:16:50.050 INFO:teuthology.orchestra.run.smithi104.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T07:16:50.050 INFO:teuthology.orchestra.run.smithi104.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T07:16:50.050 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 457 k 2024-09-11T07:16:50.050 INFO:teuthology.orchestra.run.smithi104.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T07:16:50.050 INFO:teuthology.orchestra.run.smithi104.stdout: librgw2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 4.9 M 2024-09-11T07:16:50.050 INFO:teuthology.orchestra.run.smithi104.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T07:16:50.050 INFO:teuthology.orchestra.run.smithi104.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T07:16:50.050 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-argparse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 45 k 2024-09-11T07:16:50.050 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 120 k 2024-09-11T07:16:50.051 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cephfs x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 156 k 2024-09-11T07:16:50.051 INFO:teuthology.orchestra.run.smithi104.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T07:16:50.051 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rados x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 320 k 2024-09-11T07:16:50.051 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 294 k 2024-09-11T07:16:50.051 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 99 k 2024-09-11T07:16:50.051 INFO:teuthology.orchestra.run.smithi104.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T07:16:50.051 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:16:50.051 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-09-11T07:16:50.051 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:16:50.051 INFO:teuthology.orchestra.run.smithi104.stdout:Install 22 Packages 2024-09-11T07:16:50.051 INFO:teuthology.orchestra.run.smithi104.stdout:Upgrade 2 Packages 2024-09-11T07:16:50.051 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:16:50.051 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 58 M 2024-09-11T07:16:50.051 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-09-11T07:16:50.053 INFO:teuthology.orchestra.run.smithi022.stdout:(3/24): libcephfs2-17.2.7-1623.gd0e6828a.el9.x8 5.8 MB/s | 636 kB 00:00 2024-09-11T07:16:50.237 INFO:teuthology.orchestra.run.smithi022.stdout:(4/24): ceph-radosgw-17.2.7-1623.gd0e6828a.el9. 15 MB/s | 14 MB 00:00 2024-09-11T07:16:50.270 INFO:teuthology.orchestra.run.smithi022.stdout:(5/24): libradosstriper1-17.2.7-1623.gd0e6828a. 2.1 MB/s | 457 kB 00:00 2024-09-11T07:16:50.337 INFO:teuthology.orchestra.run.smithi022.stdout:(6/24): python3-ceph-argparse-17.2.7-1623.gd0e6 680 kB/s | 45 kB 00:00 2024-09-11T07:16:50.438 INFO:teuthology.orchestra.run.smithi022.stdout:(7/24): librgw2-17.2.7-1623.gd0e6828a.el9.x86_6 25 MB/s | 4.9 MB 00:00 2024-09-11T07:16:50.463 INFO:teuthology.orchestra.run.smithi022.stdout:(8/24): python3-ceph-common-17.2.7-1623.gd0e682 959 kB/s | 120 kB 00:00 2024-09-11T07:16:50.488 INFO:teuthology.orchestra.run.smithi022.stdout:(9/24): python3-cephfs-17.2.7-1623.gd0e6828a.el 3.0 MB/s | 156 kB 00:00 2024-09-11T07:16:50.513 INFO:teuthology.orchestra.run.smithi022.stdout:(10/24): python3-rados-17.2.7-1623.gd0e6828a.el 6.2 MB/s | 320 kB 00:00 2024-09-11T07:16:50.722 INFO:teuthology.orchestra.run.smithi022.stdout:(11/24): ceph-common-17.2.7-1623.gd0e6828a.el9. 15 MB/s | 23 MB 00:01 2024-09-11T07:16:50.747 INFO:teuthology.orchestra.run.smithi022.stdout:(12/24): python3-rgw-17.2.7-1623.gd0e6828a.el9. 426 kB/s | 99 kB 00:00 2024-09-11T07:16:50.759 INFO:teuthology.orchestra.run.smithi104.stdout:(1/24): ceph-base-17.2.7-1623.gd0e6828a.el9.x86 9.6 MB/s | 5.0 MB 00:00 2024-09-11T07:16:50.773 INFO:teuthology.orchestra.run.smithi022.stdout:(13/24): python3-rbd-17.2.7-1623.gd0e6828a.el9. 1.0 MB/s | 294 kB 00:00 2024-09-11T07:16:50.835 INFO:teuthology.orchestra.run.smithi104.stdout:(2/24): ceph-selinux-17.2.7-1623.gd0e6828a.el9. 335 kB/s | 25 kB 00:00 2024-09-11T07:16:50.848 INFO:teuthology.orchestra.run.smithi022.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 451 kB/s | 45 kB 00:00 2024-09-11T07:16:50.873 INFO:teuthology.orchestra.run.smithi022.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-09-11T07:16:50.915 INFO:teuthology.orchestra.run.smithi022.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-09-11T07:16:50.935 INFO:teuthology.orchestra.run.smithi104.stdout:(3/24): libcephfs2-17.2.7-1623.gd0e6828a.el9.x8 6.2 MB/s | 636 kB 00:00 2024-09-11T07:16:50.940 INFO:teuthology.orchestra.run.smithi022.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-11T07:16:50.991 INFO:teuthology.orchestra.run.smithi022.stdout:(18/24): librdkafka-1.6.1-102.el9.x86_64.rpm 3.0 MB/s | 662 kB 00:00 2024-09-11T07:16:51.019 INFO:teuthology.orchestra.run.smithi104.stdout:(4/24): libradosstriper1-17.2.7-1623.gd0e6828a. 5.4 MB/s | 457 kB 00:00 2024-09-11T07:16:51.024 INFO:teuthology.orchestra.run.smithi022.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 642 kB/s | 49 kB 00:00 2024-09-11T07:16:51.058 INFO:teuthology.orchestra.run.smithi022.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-09-11T07:16:51.083 INFO:teuthology.orchestra.run.smithi022.stdout:(21/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.8 MB/s | 309 kB 00:00 2024-09-11T07:16:51.142 INFO:teuthology.orchestra.run.smithi022.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-11T07:16:51.361 INFO:teuthology.orchestra.run.smithi104.stdout:(5/24): librgw2-17.2.7-1623.gd0e6828a.el9.x86_6 14 MB/s | 4.9 MB 00:00 2024-09-11T07:16:51.511 INFO:teuthology.orchestra.run.smithi104.stdout:(6/24): ceph-radosgw-17.2.7-1623.gd0e6828a.el9. 11 MB/s | 14 MB 00:01 2024-09-11T07:16:51.536 INFO:teuthology.orchestra.run.smithi104.stdout:(7/24): python3-ceph-argparse-17.2.7-1623.gd0e6 258 kB/s | 45 kB 00:00 2024-09-11T07:16:51.542 INFO:teuthology.orchestra.run.smithi022.stdout:(23/24): librbd1-17.2.7-1623.gd0e6828a.el9.x86_ 6.4 MB/s | 2.9 MB 00:00 2024-09-11T07:16:51.561 INFO:teuthology.orchestra.run.smithi104.stdout:(8/24): python3-ceph-common-17.2.7-1623.gd0e682 2.4 MB/s | 120 kB 00:00 2024-09-11T07:16:51.586 INFO:teuthology.orchestra.run.smithi104.stdout:(9/24): python3-cephfs-17.2.7-1623.gd0e6828a.el 3.0 MB/s | 156 kB 00:00 2024-09-11T07:16:51.601 INFO:teuthology.orchestra.run.smithi022.stdout:(24/24): librados2-17.2.7-1623.gd0e6828a.el9.x8 5.7 MB/s | 3.1 MB 00:00 2024-09-11T07:16:51.607 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:16:51.607 INFO:teuthology.orchestra.run.smithi022.stdout:Total 21 MB/s | 58 MB 00:02 2024-09-11T07:16:51.612 INFO:teuthology.orchestra.run.smithi104.stdout:(10/24): python3-rados-17.2.7-1623.gd0e6828a.el 6.2 MB/s | 320 kB 00:00 2024-09-11T07:16:51.637 INFO:teuthology.orchestra.run.smithi104.stdout:(11/24): python3-rbd-17.2.7-1623.gd0e6828a.el9. 5.7 MB/s | 294 kB 00:00 2024-09-11T07:16:51.662 INFO:teuthology.orchestra.run.smithi104.stdout:(12/24): python3-rgw-17.2.7-1623.gd0e6828a.el9. 1.9 MB/s | 99 kB 00:00 2024-09-11T07:16:51.675 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-11T07:16:51.712 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-11T07:16:51.713 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-11T07:16:51.888 INFO:teuthology.orchestra.run.smithi104.stdout:(13/24): ceph-common-17.2.7-1623.gd0e6828a.el9. 14 MB/s | 23 MB 00:01 2024-09-11T07:16:51.963 INFO:teuthology.orchestra.run.smithi104.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 492 kB/s | 160 kB 00:00 2024-09-11T07:16:52.113 INFO:teuthology.orchestra.run.smithi104.stdout:(15/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-09-11T07:16:52.147 INFO:teuthology.orchestra.run.smithi104.stdout:(16/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.5 MB/s | 662 kB 00:00 2024-09-11T07:16:52.172 INFO:teuthology.orchestra.run.smithi104.stdout:(17/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 89 kB/s | 45 kB 00:00 2024-09-11T07:16:52.197 INFO:teuthology.orchestra.run.smithi104.stdout:(18/24): python3-prettytable-0.7.2-27.el9.noarc 505 kB/s | 42 kB 00:00 2024-09-11T07:16:52.206 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-11T07:16:52.206 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-11T07:16:52.247 INFO:teuthology.orchestra.run.smithi104.stdout:(19/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 3.0 MB/s | 309 kB 00:00 2024-09-11T07:16:52.273 INFO:teuthology.orchestra.run.smithi104.stdout:(20/24): liboath-2.6.7-2.el9.x86_64.rpm 484 kB/s | 49 kB 00:00 2024-09-11T07:16:52.300 INFO:teuthology.orchestra.run.smithi104.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 779 kB/s | 67 kB 00:00 2024-09-11T07:16:52.340 INFO:teuthology.orchestra.run.smithi104.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-11T07:16:52.448 INFO:teuthology.orchestra.run.smithi104.stdout:(23/24): librados2-17.2.7-1623.gd0e6828a.el9.x8 18 MB/s | 3.1 MB 00:00 2024-09-11T07:16:52.515 INFO:teuthology.orchestra.run.smithi104.stdout:(24/24): librbd1-17.2.7-1623.gd0e6828a.el9.x86_ 14 MB/s | 2.9 MB 00:00 2024-09-11T07:16:52.519 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:16:52.520 INFO:teuthology.orchestra.run.smithi104.stdout:Total 24 MB/s | 58 MB 00:02 2024-09-11T07:16:52.580 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-09-11T07:16:52.612 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-09-11T07:16:52.612 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-09-11T07:16:53.095 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-09-11T07:16:53.095 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-09-11T07:16:53.461 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-11T07:16:53.819 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-11T07:16:53.936 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T07:16:54.030 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T07:16:54.109 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T07:16:54.138 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T07:16:54.290 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-09-11T07:16:54.352 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T07:16:54.405 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-11T07:16:54.434 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-11T07:16:54.670 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-11T07:16:54.751 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T07:16:54.843 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T07:16:54.905 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-11T07:16:54.920 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T07:16:54.941 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T07:16:54.968 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T07:16:54.987 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T07:16:55.024 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-11T07:16:55.056 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T07:16:55.068 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 10/26 2024-09-11T07:16:55.102 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-11T07:16:55.104 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T07:16:55.132 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-11T07:16:55.168 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T07:16:55.193 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T07:16:55.259 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T07:16:55.302 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-11T07:16:55.337 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-09-11T07:16:55.370 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-11T07:16:55.572 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-11T07:16:55.596 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T07:16:55.662 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T07:16:55.680 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-11T07:16:55.712 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-11T07:16:55.715 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T07:16:55.764 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 10/26 2024-09-11T07:16:55.790 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T07:16:55.800 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T07:16:55.862 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T07:16:55.866 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 19/26 2024-09-11T07:16:55.890 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T07:16:55.893 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 20/26 2024-09-11T07:16:55.956 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T07:16:56.009 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-11T07:16:56.040 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-09-11T07:16:56.073 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-11T07:16:56.367 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-11T07:16:56.411 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T07:16:56.501 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T07:16:56.574 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 19/26 2024-09-11T07:16:56.605 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 20/26 2024-09-11T07:16:58.381 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T07:16:58.415 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T07:16:58.970 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T07:16:59.001 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T07:16:59.035 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T07:16:59.075 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T07:16:59.099 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T07:16:59.099 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T07:16:59.099 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T07:16:59.099 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:16:59.136 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T07:16:59.599 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T07:16:59.628 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T07:16:59.728 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T07:16:59.729 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T07:16:59.729 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T07:16:59.729 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:16:59.762 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T07:17:09.452 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T07:17:09.452 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /sys 2024-09-11T07:17:09.452 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /proc 2024-09-11T07:17:09.452 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /mnt 2024-09-11T07:17:09.452 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /var/tmp 2024-09-11T07:17:09.452 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /home 2024-09-11T07:17:09.452 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /root 2024-09-11T07:17:09.452 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /tmp 2024-09-11T07:17:09.452 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:09.471 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T07:17:09.471 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /sys 2024-09-11T07:17:09.471 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /proc 2024-09-11T07:17:09.471 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /mnt 2024-09-11T07:17:09.471 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /var/tmp 2024-09-11T07:17:09.471 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /home 2024-09-11T07:17:09.471 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /root 2024-09-11T07:17:09.471 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /tmp 2024-09-11T07:17:09.471 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:10.777 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T07:17:10.826 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T07:17:10.848 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T07:17:10.848 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:17:10.848 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T07:17:10.848 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T07:17:10.848 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T07:17:10.848 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:10.850 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T07:17:10.906 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T07:17:10.906 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:17:10.906 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T07:17:10.906 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T07:17:10.906 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T07:17:10.906 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:10.908 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T07:17:10.938 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T07:17:10.938 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T07:17:10.996 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T07:17:10.996 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T07:17:12.198 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T07:17:12.198 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/26 2024-09-11T07:17:12.198 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T07:17:12.198 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T07:17:12.198 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T07:17:12.198 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/26 2024-09-11T07:17:12.198 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 6/26 2024-09-11T07:17:12.198 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 7/26 2024-09-11T07:17:12.199 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 8/26 2024-09-11T07:17:12.199 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 9/26 2024-09-11T07:17:12.199 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 10/26 2024-09-11T07:17:12.199 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T07:17:12.199 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T07:17:12.199 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 13/26 2024-09-11T07:17:12.199 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-11T07:17:12.199 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-11T07:17:12.199 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-11T07:17:12.199 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-11T07:17:12.199 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-11T07:17:12.199 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-09-11T07:17:12.199 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-11T07:17:12.199 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-11T07:17:12.200 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-11T07:17:12.200 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T07:17:12.200 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-11T07:17:12.200 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 25/26 2024-09-11T07:17:12.215 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T07:17:12.215 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/26 2024-09-11T07:17:12.215 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T07:17:12.215 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T07:17:12.215 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T07:17:12.215 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/26 2024-09-11T07:17:12.215 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 6/26 2024-09-11T07:17:12.215 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 7/26 2024-09-11T07:17:12.215 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 8/26 2024-09-11T07:17:12.216 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 9/26 2024-09-11T07:17:12.216 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 10/26 2024-09-11T07:17:12.216 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T07:17:12.216 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T07:17:12.216 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 13/26 2024-09-11T07:17:12.216 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-11T07:17:12.216 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-11T07:17:12.216 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-11T07:17:12.216 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-11T07:17:12.216 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-11T07:17:12.216 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-09-11T07:17:12.216 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-11T07:17:12.216 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-11T07:17:12.217 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-11T07:17:12.217 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T07:17:12.217 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-11T07:17:12.217 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 25/26 2024-09-11T07:17:13.048 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T07:17:13.048 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:13.048 INFO:teuthology.orchestra.run.smithi104.stdout:Upgraded: 2024-09-11T07:17:13.048 INFO:teuthology.orchestra.run.smithi104.stdout: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.048 INFO:teuthology.orchestra.run.smithi104.stdout: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.048 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-09-11T07:17:13.048 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.048 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.048 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.049 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.049 INFO:teuthology.orchestra.run.smithi104.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T07:17:13.049 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.049 INFO:teuthology.orchestra.run.smithi104.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T07:17:13.049 INFO:teuthology.orchestra.run.smithi104.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T07:17:13.049 INFO:teuthology.orchestra.run.smithi104.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T07:17:13.049 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.049 INFO:teuthology.orchestra.run.smithi104.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T07:17:13.049 INFO:teuthology.orchestra.run.smithi104.stdout: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.049 INFO:teuthology.orchestra.run.smithi104.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T07:17:13.049 INFO:teuthology.orchestra.run.smithi104.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T07:17:13.049 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.050 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.050 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.050 INFO:teuthology.orchestra.run.smithi104.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T07:17:13.050 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.050 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.050 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.050 INFO:teuthology.orchestra.run.smithi104.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T07:17:13.050 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:13.050 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:17:13.077 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T07:17:13.078 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:13.078 INFO:teuthology.orchestra.run.smithi022.stdout:Upgraded: 2024-09-11T07:17:13.078 INFO:teuthology.orchestra.run.smithi022.stdout: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.078 INFO:teuthology.orchestra.run.smithi022.stdout: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.078 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-11T07:17:13.078 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.078 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.078 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.078 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.078 INFO:teuthology.orchestra.run.smithi022.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T07:17:13.078 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.078 INFO:teuthology.orchestra.run.smithi022.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T07:17:13.078 INFO:teuthology.orchestra.run.smithi022.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T07:17:13.079 INFO:teuthology.orchestra.run.smithi022.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T07:17:13.079 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.079 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T07:17:13.079 INFO:teuthology.orchestra.run.smithi022.stdout: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.079 INFO:teuthology.orchestra.run.smithi022.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T07:17:13.079 INFO:teuthology.orchestra.run.smithi022.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T07:17:13.079 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.079 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.079 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.079 INFO:teuthology.orchestra.run.smithi022.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T07:17:13.079 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.079 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.079 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:13.079 INFO:teuthology.orchestra.run.smithi022.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T07:17:13.080 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:13.080 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:17:13.403 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-test 2024-09-11T07:17:13.453 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-test 2024-09-11T07:17:13.974 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:00:26 ago on Wed 11 Sep 2024 07:16:47 AM UTC. 2024-09-11T07:17:14.044 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:00:28 ago on Wed 11 Sep 2024 07:16:46 AM UTC. 2024-09-11T07:17:14.110 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:17:14.111 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:17:14.111 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-09-11T07:17:14.111 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:17:14.111 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-09-11T07:17:14.112 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-test x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 48 M 2024-09-11T07:17:14.112 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-09-11T07:17:14.112 INFO:teuthology.orchestra.run.smithi104.stdout: libcephsqlite x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 163 k 2024-09-11T07:17:14.112 INFO:teuthology.orchestra.run.smithi104.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T07:17:14.112 INFO:teuthology.orchestra.run.smithi104.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T07:17:14.112 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:14.112 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-09-11T07:17:14.112 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:17:14.112 INFO:teuthology.orchestra.run.smithi104.stdout:Install 4 Packages 2024-09-11T07:17:14.112 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:14.113 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 48 M 2024-09-11T07:17:14.113 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 193 M 2024-09-11T07:17:14.113 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-09-11T07:17:14.186 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:17:14.187 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:17:14.187 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-09-11T07:17:14.187 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:17:14.187 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-11T07:17:14.188 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-test x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 48 M 2024-09-11T07:17:14.188 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-09-11T07:17:14.188 INFO:teuthology.orchestra.run.smithi022.stdout: libcephsqlite x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 163 k 2024-09-11T07:17:14.188 INFO:teuthology.orchestra.run.smithi022.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T07:17:14.188 INFO:teuthology.orchestra.run.smithi022.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T07:17:14.188 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:14.188 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-11T07:17:14.188 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:17:14.188 INFO:teuthology.orchestra.run.smithi022.stdout:Install 4 Packages 2024-09-11T07:17:14.188 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:14.189 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 48 M 2024-09-11T07:17:14.189 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 193 M 2024-09-11T07:17:14.189 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-11T07:17:14.453 INFO:teuthology.orchestra.run.smithi022.stdout:(1/4): libcephsqlite-17.2.7-1623.gd0e6828a.el9. 765 kB/s | 163 kB 00:00 2024-09-11T07:17:14.466 INFO:teuthology.orchestra.run.smithi104.stdout:(1/4): libcephsqlite-17.2.7-1623.gd0e6828a.el9. 749 kB/s | 163 kB 00:00 2024-09-11T07:17:15.036 INFO:teuthology.orchestra.run.smithi022.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 382 kB/s | 304 kB 00:00 2024-09-11T07:17:15.041 INFO:teuthology.orchestra.run.smithi104.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 384 kB/s | 304 kB 00:00 2024-09-11T07:17:15.070 INFO:teuthology.orchestra.run.smithi022.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 104 kB/s | 64 kB 00:00 2024-09-11T07:17:15.124 INFO:teuthology.orchestra.run.smithi104.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 97 kB/s | 64 kB 00:00 2024-09-11T07:17:16.516 INFO:teuthology.orchestra.run.smithi104.stdout:(4/4): ceph-test-17.2.7-1623.gd0e6828a.el9.x86_ 21 MB/s | 48 MB 00:02 2024-09-11T07:17:16.518 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:17:16.519 INFO:teuthology.orchestra.run.smithi104.stdout:Total 20 MB/s | 48 MB 00:02 2024-09-11T07:17:16.528 INFO:teuthology.orchestra.run.smithi022.stdout:(4/4): ceph-test-17.2.7-1623.gd0e6828a.el9.x86_ 21 MB/s | 48 MB 00:02 2024-09-11T07:17:16.530 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:17:16.531 INFO:teuthology.orchestra.run.smithi022.stdout:Total 21 MB/s | 48 MB 00:02 2024-09-11T07:17:16.547 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-09-11T07:17:16.559 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-11T07:17:16.566 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-09-11T07:17:16.566 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-09-11T07:17:16.581 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-11T07:17:16.581 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-11T07:17:16.934 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-09-11T07:17:16.935 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-09-11T07:17:16.968 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-11T07:17:16.968 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-11T07:17:17.451 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-09-11T07:17:17.506 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-11T07:17:17.532 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-11T07:17:17.595 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-11T07:17:17.607 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-11T07:17:17.626 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T07:17:17.653 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-11T07:17:17.680 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T07:17:22.075 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T07:17:22.149 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T07:17:22.352 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T07:17:22.434 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T07:17:24.305 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T07:17:24.305 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/4 2024-09-11T07:17:24.305 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/4 2024-09-11T07:17:24.305 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-11T07:17:24.642 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-11T07:17:24.642 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:24.643 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-09-11T07:17:24.643 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:24.643 INFO:teuthology.orchestra.run.smithi104.stdout: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:24.643 INFO:teuthology.orchestra.run.smithi104.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T07:17:24.643 INFO:teuthology.orchestra.run.smithi104.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T07:17:24.643 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:24.643 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:17:24.838 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T07:17:24.838 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/4 2024-09-11T07:17:24.838 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/4 2024-09-11T07:17:24.838 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-11T07:17:24.957 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph 2024-09-11T07:17:25.188 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-11T07:17:25.188 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:25.189 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-11T07:17:25.189 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:25.189 INFO:teuthology.orchestra.run.smithi022.stdout: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:25.189 INFO:teuthology.orchestra.run.smithi022.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T07:17:25.189 INFO:teuthology.orchestra.run.smithi022.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T07:17:25.189 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:25.189 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:17:25.449 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph 2024-09-11T07:17:25.521 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:00:38 ago on Wed 11 Sep 2024 07:16:47 AM UTC. 2024-09-11T07:17:25.655 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:17:25.658 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:17:25.658 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-09-11T07:17:25.658 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:17:25.658 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-09-11T07:17:25.658 INFO:teuthology.orchestra.run.smithi104.stdout: ceph x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 6.5 k 2024-09-11T07:17:25.658 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-09-11T07:17:25.658 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mds x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.1 M 2024-09-11T07:17:25.658 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 1.4 M 2024-09-11T07:17:25.658 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 240 k 2024-09-11T07:17:25.658 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mon x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.9 M 2024-09-11T07:17:25.658 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-osd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 15 M 2024-09-11T07:17:25.659 INFO:teuthology.orchestra.run.smithi104.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T07:17:25.659 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T07:17:25.659 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T07:17:25.659 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T07:17:25.659 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T07:17:25.659 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T07:17:25.659 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T07:17:25.659 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-collections 2024-09-11T07:17:25.659 INFO:teuthology.orchestra.run.smithi104.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-11T07:17:25.659 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T07:17:25.659 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T07:17:25.659 INFO:teuthology.orchestra.run.smithi104.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T07:17:25.660 INFO:teuthology.orchestra.run.smithi104.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T07:17:25.660 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T07:17:25.660 INFO:teuthology.orchestra.run.smithi104.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T07:17:25.660 INFO:teuthology.orchestra.run.smithi104.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T07:17:25.660 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T07:17:25.660 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T07:17:25.660 INFO:teuthology.orchestra.run.smithi104.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T07:17:25.660 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T07:17:25.660 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T07:17:25.660 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T07:17:25.660 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T07:17:25.660 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T07:17:25.661 INFO:teuthology.orchestra.run.smithi104.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T07:17:25.661 INFO:teuthology.orchestra.run.smithi104.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T07:17:25.661 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T07:17:25.661 INFO:teuthology.orchestra.run.smithi104.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T07:17:25.661 INFO:teuthology.orchestra.run.smithi104.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T07:17:25.661 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:25.661 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-09-11T07:17:25.661 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:17:25.661 INFO:teuthology.orchestra.run.smithi104.stdout:Install 34 Packages 2024-09-11T07:17:25.661 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:25.662 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 27 M 2024-09-11T07:17:25.662 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 96 M 2024-09-11T07:17:25.662 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-09-11T07:17:26.038 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:00:40 ago on Wed 11 Sep 2024 07:16:46 AM UTC. 2024-09-11T07:17:26.179 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:17:26.181 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:17:26.181 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-09-11T07:17:26.181 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:17:26.181 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-11T07:17:26.182 INFO:teuthology.orchestra.run.smithi022.stdout: ceph x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 6.5 k 2024-09-11T07:17:26.182 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-09-11T07:17:26.182 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mds x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.1 M 2024-09-11T07:17:26.182 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 1.4 M 2024-09-11T07:17:26.182 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 240 k 2024-09-11T07:17:26.182 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mon x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.9 M 2024-09-11T07:17:26.182 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-osd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 15 M 2024-09-11T07:17:26.182 INFO:teuthology.orchestra.run.smithi022.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T07:17:26.182 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T07:17:26.182 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T07:17:26.182 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T07:17:26.182 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T07:17:26.182 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T07:17:26.182 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T07:17:26.183 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-collections 2024-09-11T07:17:26.183 INFO:teuthology.orchestra.run.smithi022.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-11T07:17:26.183 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T07:17:26.183 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T07:17:26.183 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T07:17:26.183 INFO:teuthology.orchestra.run.smithi022.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T07:17:26.183 INFO:teuthology.orchestra.run.smithi022.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T07:17:26.183 INFO:teuthology.orchestra.run.smithi022.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T07:17:26.183 INFO:teuthology.orchestra.run.smithi022.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T07:17:26.183 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T07:17:26.183 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T07:17:26.183 INFO:teuthology.orchestra.run.smithi022.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T07:17:26.183 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T07:17:26.183 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T07:17:26.184 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T07:17:26.184 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T07:17:26.184 INFO:teuthology.orchestra.run.smithi022.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T07:17:26.184 INFO:teuthology.orchestra.run.smithi022.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T07:17:26.184 INFO:teuthology.orchestra.run.smithi022.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T07:17:26.184 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T07:17:26.184 INFO:teuthology.orchestra.run.smithi022.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T07:17:26.184 INFO:teuthology.orchestra.run.smithi022.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T07:17:26.184 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:26.184 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-11T07:17:26.184 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:17:26.184 INFO:teuthology.orchestra.run.smithi022.stdout:Install 34 Packages 2024-09-11T07:17:26.184 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:26.185 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 27 M 2024-09-11T07:17:26.185 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 96 M 2024-09-11T07:17:26.185 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-11T07:17:26.477 INFO:teuthology.orchestra.run.smithi104.stdout:(1/34): ceph-17.2.7-1623.gd0e6828a.el9.x86_64.r 59 kB/s | 6.5 kB 00:00 2024-09-11T07:17:26.685 INFO:teuthology.orchestra.run.smithi104.stdout:(2/34): ceph-mds-17.2.7-1623.gd0e6828a.el9.x86_ 6.5 MB/s | 2.1 MB 00:00 2024-09-11T07:17:26.719 INFO:teuthology.orchestra.run.smithi104.stdout:(3/34): ceph-mgr-17.2.7-1623.gd0e6828a.el9.x86_ 4.0 MB/s | 1.4 MB 00:00 2024-09-11T07:17:26.786 INFO:teuthology.orchestra.run.smithi104.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1623.gd0e6 3.5 MB/s | 240 kB 00:00 2024-09-11T07:17:26.870 INFO:teuthology.orchestra.run.smithi104.stdout:(5/34): ceph-mon-17.2.7-1623.gd0e6828a.el9.x86_ 10 MB/s | 3.9 MB 00:00 2024-09-11T07:17:26.899 INFO:teuthology.orchestra.run.smithi022.stdout:(1/34): ceph-17.2.7-1623.gd0e6828a.el9.x86_64.r 50 kB/s | 6.5 kB 00:00 2024-09-11T07:17:27.020 INFO:teuthology.orchestra.run.smithi104.stdout:(6/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-11T07:17:27.070 INFO:teuthology.orchestra.run.smithi104.stdout:(7/34): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-11T07:17:27.229 INFO:teuthology.orchestra.run.smithi104.stdout:(8/34): ceph-osd-17.2.7-1623.gd0e6828a.el9.x86_ 28 MB/s | 15 MB 00:00 2024-09-11T07:17:27.262 INFO:teuthology.orchestra.run.smithi104.stdout:(9/34): python3-pycparser-2.20-6.el9.noarch.rpm 704 kB/s | 135 kB 00:00 2024-09-11T07:17:27.291 INFO:teuthology.orchestra.run.smithi022.stdout:(2/34): ceph-mgr-17.2.7-1623.gd0e6828a.el9.x86_ 2.7 MB/s | 1.4 MB 00:00 2024-09-11T07:17:27.296 INFO:teuthology.orchestra.run.smithi104.stdout:(10/34): python3-cryptography-36.0.1-4.el9.x86_ 2.9 MB/s | 1.2 MB 00:00 2024-09-11T07:17:27.338 INFO:teuthology.orchestra.run.smithi104.stdout:(11/34): python3-urllib3-1.26.5-6.el9.noarch.rp 2.8 MB/s | 215 kB 00:00 2024-09-11T07:17:27.383 INFO:teuthology.orchestra.run.smithi022.stdout:(3/34): ceph-mds-17.2.7-1623.gd0e6828a.el9.x86_ 3.4 MB/s | 2.1 MB 00:00 2024-09-11T07:17:27.396 INFO:teuthology.orchestra.run.smithi104.stdout:(12/34): python3-requests-2.25.1-8.el9.noarch.r 746 kB/s | 125 kB 00:00 2024-09-11T07:17:27.438 INFO:teuthology.orchestra.run.smithi104.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 347 kB/s | 35 kB 00:00 2024-09-11T07:17:27.458 INFO:teuthology.orchestra.run.smithi022.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1623.gd0e6 3.1 MB/s | 240 kB 00:00 2024-09-11T07:17:27.472 INFO:teuthology.orchestra.run.smithi104.stdout:(14/34): python3-mako-1.1.4-6.el9.noarch.rpm 982 kB/s | 172 kB 00:00 2024-09-11T07:17:27.514 INFO:teuthology.orchestra.run.smithi104.stdout:(15/34): python3-pytz-2021.1-5.el9.noarch.rpm 679 kB/s | 51 kB 00:00 2024-09-11T07:17:27.539 INFO:teuthology.orchestra.run.smithi104.stdout:(16/34): python3-toml-0.10.2-6.el9.noarch.rpm 621 kB/s | 42 kB 00:00 2024-09-11T07:17:27.564 INFO:teuthology.orchestra.run.smithi104.stdout:(17/34): python3-packaging-20.9-5.el9.noarch.rp 461 kB/s | 77 kB 00:00 2024-09-11T07:17:27.615 INFO:teuthology.orchestra.run.smithi104.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 433 kB/s | 43 kB 00:00 2024-09-11T07:17:27.640 INFO:teuthology.orchestra.run.smithi104.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 463 kB/s | 11 kB 00:00 2024-09-11T07:17:27.665 INFO:teuthology.orchestra.run.smithi104.stdout:(20/34): python3-jaraco-classes-3.2.1-5.el9.noa 711 kB/s | 18 kB 00:00 2024-09-11T07:17:27.690 INFO:teuthology.orchestra.run.smithi104.stdout:(21/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-11T07:17:27.709 INFO:teuthology.orchestra.run.smithi022.stdout:(5/34): ceph-mon-17.2.7-1623.gd0e6828a.el9.x86_ 4.8 MB/s | 3.9 MB 00:00 2024-09-11T07:17:27.716 INFO:teuthology.orchestra.run.smithi104.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 462 kB/s | 23 kB 00:00 2024-09-11T07:17:27.734 INFO:teuthology.orchestra.run.smithi022.stdout:(6/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 920 kB/s | 253 kB 00:00 2024-09-11T07:17:27.741 INFO:teuthology.orchestra.run.smithi104.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 389 kB/s | 19 kB 00:00 2024-09-11T07:17:27.766 INFO:teuthology.orchestra.run.smithi104.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 393 kB/s | 20 kB 00:00 2024-09-11T07:17:27.791 INFO:teuthology.orchestra.run.smithi104.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 923 kB/s | 46 kB 00:00 2024-09-11T07:17:27.792 INFO:teuthology.orchestra.run.smithi022.stdout:(7/34): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-11T07:17:27.817 INFO:teuthology.orchestra.run.smithi104.stdout:(26/34): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-11T07:17:27.842 INFO:teuthology.orchestra.run.smithi104.stdout:(27/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-11T07:17:27.851 INFO:teuthology.orchestra.run.smithi022.stdout:(8/34): python3-pycparser-2.20-6.el9.noarch.rpm 2.3 MB/s | 135 kB 00:00 2024-09-11T07:17:27.867 INFO:teuthology.orchestra.run.smithi104.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-11T07:17:27.892 INFO:teuthology.orchestra.run.smithi104.stdout:(29/34): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-11T07:17:27.910 INFO:teuthology.orchestra.run.smithi022.stdout:(9/34): python3-requests-2.25.1-8.el9.noarch.rp 2.1 MB/s | 125 kB 00:00 2024-09-11T07:17:27.918 INFO:teuthology.orchestra.run.smithi104.stdout:(30/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-11T07:17:27.943 INFO:teuthology.orchestra.run.smithi104.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-11T07:17:27.968 INFO:teuthology.orchestra.run.smithi104.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 800 kB/s | 20 kB 00:00 2024-09-11T07:17:27.969 INFO:teuthology.orchestra.run.smithi022.stdout:(10/34): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-11T07:17:27.993 INFO:teuthology.orchestra.run.smithi104.stdout:(33/34): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-11T07:17:28.012 INFO:teuthology.orchestra.run.smithi022.stdout:(11/34): python3-cryptography-36.0.1-4.el9.x86_ 4.0 MB/s | 1.2 MB 00:00 2024-09-11T07:17:28.018 INFO:teuthology.orchestra.run.smithi104.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-11T07:17:28.023 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:17:28.024 INFO:teuthology.orchestra.run.smithi104.stdout:Total 12 MB/s | 27 MB 00:02 2024-09-11T07:17:28.157 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-09-11T07:17:28.195 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-09-11T07:17:28.195 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-09-11T07:17:28.404 INFO:teuthology.orchestra.run.smithi022.stdout:(12/34): ceph-osd-17.2.7-1623.gd0e6828a.el9.x86 14 MB/s | 15 MB 00:01 2024-09-11T07:17:28.563 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-09-11T07:17:28.563 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-09-11T07:17:28.688 INFO:teuthology.orchestra.run.smithi022.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 51 kB/s | 35 kB 00:00 2024-09-11T07:17:28.821 INFO:teuthology.orchestra.run.smithi022.stdout:(14/34): python3-mako-1.1.4-6.el9.noarch.rpm 202 kB/s | 172 kB 00:00 2024-09-11T07:17:29.038 INFO:teuthology.orchestra.run.smithi022.stdout:(15/34): python3-packaging-20.9-5.el9.noarch.rp 122 kB/s | 77 kB 00:00 2024-09-11T07:17:29.080 INFO:teuthology.orchestra.run.smithi022.stdout:(16/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.3 MB/s | 43 kB 00:00 2024-09-11T07:17:29.113 INFO:teuthology.orchestra.run.smithi022.stdout:(17/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 5.1 MB/s | 172 kB 00:00 2024-09-11T07:17:29.147 INFO:teuthology.orchestra.run.smithi022.stdout:(18/34): python3-cherrypy-18.6.1-2.el9.noarch.r 10 MB/s | 358 kB 00:00 2024-09-11T07:17:29.164 INFO:teuthology.orchestra.run.smithi022.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 650 kB/s | 11 kB 00:00 2024-09-11T07:17:29.181 INFO:teuthology.orchestra.run.smithi022.stdout:(20/34): python3-jaraco-classes-3.2.1-5.el9.noa 1.0 MB/s | 18 kB 00:00 2024-09-11T07:17:29.198 INFO:teuthology.orchestra.run.smithi022.stdout:(21/34): python3-jaraco-collections-3.0.0-8.el9 1.4 MB/s | 23 kB 00:00 2024-09-11T07:17:29.223 INFO:teuthology.orchestra.run.smithi022.stdout:(22/34): python3-jaraco-functools-3.5.0-2.el9.n 781 kB/s | 19 kB 00:00 2024-09-11T07:17:29.248 INFO:teuthology.orchestra.run.smithi022.stdout:(23/34): python3-jaraco-text-3.2.0-6.el9.noarch 790 kB/s | 20 kB 00:00 2024-09-11T07:17:29.273 INFO:teuthology.orchestra.run.smithi022.stdout:(24/34): python3-logutils-0.3.5-21.el9.noarch.r 1.8 MB/s | 46 kB 00:00 2024-09-11T07:17:29.307 INFO:teuthology.orchestra.run.smithi022.stdout:(25/34): python3-more-itertools-8.12.0-2.el9.no 2.3 MB/s | 79 kB 00:00 2024-09-11T07:17:29.332 INFO:teuthology.orchestra.run.smithi022.stdout:(26/34): python3-pecan-1.4.2-3.el9.noarch.rpm 11 MB/s | 272 kB 00:00 2024-09-11T07:17:29.358 INFO:teuthology.orchestra.run.smithi022.stdout:(27/34): python3-pytz-2021.1-5.el9.noarch.rpm 76 kB/s | 51 kB 00:00 2024-09-11T07:17:29.383 INFO:teuthology.orchestra.run.smithi022.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 332 kB/s | 16 kB 00:00 2024-09-11T07:17:29.408 INFO:teuthology.orchestra.run.smithi022.stdout:(29/34): python3-toml-0.10.2-6.el9.noarch.rpm 71 kB/s | 42 kB 00:00 2024-09-11T07:17:29.426 INFO:teuthology.orchestra.run.smithi022.stdout:(30/34): python3-tempora-5.0.0-2.el9.noarch.rpm 1.0 MB/s | 36 kB 00:00 2024-09-11T07:17:29.459 INFO:teuthology.orchestra.run.smithi022.stdout:(31/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 889 kB/s | 90 kB 00:00 2024-09-11T07:17:29.476 INFO:teuthology.orchestra.run.smithi022.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 1.3 MB/s | 20 kB 00:00 2024-09-11T07:17:29.501 INFO:teuthology.orchestra.run.smithi022.stdout:(33/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 6.2 MB/s | 427 kB 00:00 2024-09-11T07:17:29.543 INFO:teuthology.orchestra.run.smithi022.stdout:(34/34): python3-webob-1.8.8-2.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-09-11T07:17:29.549 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:17:29.549 INFO:teuthology.orchestra.run.smithi022.stdout:Total 8.1 MB/s | 27 MB 00:03 2024-09-11T07:17:29.687 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-11T07:17:29.727 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-11T07:17:29.727 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-11T07:17:29.849 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-09-11T07:17:29.905 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-11T07:17:29.945 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-11T07:17:29.978 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-11T07:17:30.071 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-11T07:17:30.126 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-11T07:17:30.126 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-11T07:17:30.156 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-11T07:17:30.205 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-11T07:17:30.240 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-11T07:17:30.270 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-11T07:17:30.315 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-11T07:17:30.346 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-11T07:17:30.378 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-11T07:17:30.413 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-11T07:17:30.447 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-11T07:17:30.495 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-11T07:17:30.528 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T07:17:30.602 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-11T07:17:30.741 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-11T07:17:30.820 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-11T07:17:30.881 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-11T07:17:30.919 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-11T07:17:30.967 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-11T07:17:31.023 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-11T07:17:31.210 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-11T07:17:31.264 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-11T07:17:31.343 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-11T07:17:31.375 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-11T07:17:31.438 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-11T07:17:31.439 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-11T07:17:31.479 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-11T07:17:31.484 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-11T07:17:31.512 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-11T07:17:31.603 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-11T07:17:31.608 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-11T07:17:31.693 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-11T07:17:31.741 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-11T07:17:31.751 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 29/34 2024-09-11T07:17:31.774 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T07:17:31.778 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-11T07:17:31.800 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T07:17:31.801 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:17:31.801 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T07:17:31.801 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T07:17:31.801 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T07:17:31.801 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:31.813 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-11T07:17:31.855 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-11T07:17:31.897 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-11T07:17:31.938 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-11T07:17:31.982 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-11T07:17:32.015 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-11T07:17:32.064 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-11T07:17:32.105 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T07:17:32.181 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-11T07:17:32.332 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-11T07:17:32.416 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-11T07:17:32.467 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-11T07:17:32.505 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-11T07:17:32.563 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-11T07:17:32.635 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-11T07:17:32.827 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-11T07:17:32.875 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-11T07:17:32.956 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-11T07:17:33.045 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-11T07:17:33.102 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-11T07:17:33.158 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T07:17:33.188 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T07:17:33.188 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:17:33.188 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T07:17:33.189 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T07:17:33.189 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T07:17:33.189 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:33.230 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-11T07:17:33.386 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 29/34 2024-09-11T07:17:33.418 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T07:17:33.446 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T07:17:33.446 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:17:33.446 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T07:17:33.447 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T07:17:33.447 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T07:17:33.447 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:33.567 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T07:17:33.592 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T07:17:33.592 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:17:33.592 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T07:17:33.592 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T07:17:33.592 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T07:17:33.592 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:33.801 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T07:17:33.825 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T07:17:33.826 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:17:33.826 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T07:17:33.826 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T07:17:33.826 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T07:17:33.826 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:33.890 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T07:17:34.835 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T07:17:34.864 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T07:17:34.864 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:17:34.864 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T07:17:34.864 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T07:17:34.864 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T07:17:34.865 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:35.261 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T07:17:35.286 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T07:17:35.286 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:17:35.286 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T07:17:35.287 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T07:17:35.287 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T07:17:35.287 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:35.495 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T07:17:35.521 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T07:17:35.521 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:17:35.521 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T07:17:35.521 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T07:17:35.521 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T07:17:35.521 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:35.587 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T07:17:35.613 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T07:17:35.614 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/34 2024-09-11T07:17:35.614 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/34 2024-09-11T07:17:35.614 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/34 2024-09-11T07:17:35.614 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/34 2024-09-11T07:17:35.614 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/34 2024-09-11T07:17:35.614 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 6/34 2024-09-11T07:17:35.614 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-11T07:17:35.614 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-11T07:17:35.614 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-11T07:17:35.614 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-11T07:17:35.614 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-11T07:17:35.614 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-11T07:17:35.614 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-11T07:17:35.615 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-11T07:17:35.615 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T07:17:35.615 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-11T07:17:35.615 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-11T07:17:35.615 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-11T07:17:35.615 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-11T07:17:35.615 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-11T07:17:35.615 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-11T07:17:35.615 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-11T07:17:35.615 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-11T07:17:35.615 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-11T07:17:35.615 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-11T07:17:35.615 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-11T07:17:35.615 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-11T07:17:35.616 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-11T07:17:35.616 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-11T07:17:35.616 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-11T07:17:35.616 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-11T07:17:35.616 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-11T07:17:35.616 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-11T07:17:36.577 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-11T07:17:36.577 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:36.577 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-09-11T07:17:36.577 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:36.577 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:36.577 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:36.577 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:17:36.578 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:36.578 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:36.578 INFO:teuthology.orchestra.run.smithi104.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T07:17:36.578 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T07:17:36.578 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T07:17:36.578 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T07:17:36.578 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T07:17:36.578 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T07:17:36.578 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T07:17:36.578 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T07:17:36.578 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T07:17:36.578 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T07:17:36.578 INFO:teuthology.orchestra.run.smithi104.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T07:17:36.579 INFO:teuthology.orchestra.run.smithi104.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T07:17:36.579 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T07:17:36.579 INFO:teuthology.orchestra.run.smithi104.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T07:17:36.579 INFO:teuthology.orchestra.run.smithi104.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T07:17:36.579 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T07:17:36.579 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T07:17:36.579 INFO:teuthology.orchestra.run.smithi104.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T07:17:36.579 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T07:17:36.579 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T07:17:36.579 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T07:17:36.579 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T07:17:36.579 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T07:17:36.579 INFO:teuthology.orchestra.run.smithi104.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T07:17:36.579 INFO:teuthology.orchestra.run.smithi104.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T07:17:36.580 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T07:17:36.580 INFO:teuthology.orchestra.run.smithi104.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T07:17:36.580 INFO:teuthology.orchestra.run.smithi104.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T07:17:36.580 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:36.580 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:17:36.950 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-base 2024-09-11T07:17:37.262 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T07:17:37.262 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/34 2024-09-11T07:17:37.262 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/34 2024-09-11T07:17:37.262 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/34 2024-09-11T07:17:37.262 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/34 2024-09-11T07:17:37.262 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/34 2024-09-11T07:17:37.262 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 6/34 2024-09-11T07:17:37.262 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-11T07:17:37.262 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-11T07:17:37.262 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-11T07:17:37.262 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-11T07:17:37.263 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-11T07:17:37.263 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-11T07:17:37.263 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-11T07:17:37.263 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-11T07:17:37.263 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T07:17:37.263 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-11T07:17:37.263 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-11T07:17:37.263 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-11T07:17:37.263 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-11T07:17:37.263 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-11T07:17:37.263 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-11T07:17:37.263 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-11T07:17:37.263 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-11T07:17:37.263 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-11T07:17:37.264 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-11T07:17:37.264 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-11T07:17:37.264 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-11T07:17:37.264 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-11T07:17:37.265 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-11T07:17:37.265 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-11T07:17:37.265 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-11T07:17:37.265 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-11T07:17:37.265 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-11T07:17:37.518 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:00:50 ago on Wed 11 Sep 2024 07:16:47 AM UTC. 2024-09-11T07:17:37.610 INFO:teuthology.orchestra.run.smithi104.stdout:Package ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:17:37.653 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:17:37.654 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-09-11T07:17:37.654 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:17:37.726 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install cephadm 2024-09-11T07:17:38.280 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:00:51 ago on Wed 11 Sep 2024 07:16:47 AM UTC. 2024-09-11T07:17:38.349 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-11T07:17:38.349 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:38.349 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-11T07:17:38.349 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:38.349 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:38.349 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:38.349 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:17:38.349 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:38.349 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:38.350 INFO:teuthology.orchestra.run.smithi022.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T07:17:38.350 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T07:17:38.350 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T07:17:38.350 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T07:17:38.350 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T07:17:38.350 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T07:17:38.350 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T07:17:38.350 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T07:17:38.350 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T07:17:38.350 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T07:17:38.350 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T07:17:38.350 INFO:teuthology.orchestra.run.smithi022.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T07:17:38.351 INFO:teuthology.orchestra.run.smithi022.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T07:17:38.351 INFO:teuthology.orchestra.run.smithi022.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T07:17:38.351 INFO:teuthology.orchestra.run.smithi022.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T07:17:38.351 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T07:17:38.351 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T07:17:38.351 INFO:teuthology.orchestra.run.smithi022.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T07:17:38.351 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T07:17:38.351 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T07:17:38.351 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T07:17:38.351 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T07:17:38.351 INFO:teuthology.orchestra.run.smithi022.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T07:17:38.352 INFO:teuthology.orchestra.run.smithi022.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T07:17:38.352 INFO:teuthology.orchestra.run.smithi022.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T07:17:38.352 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T07:17:38.352 INFO:teuthology.orchestra.run.smithi022.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T07:17:38.352 INFO:teuthology.orchestra.run.smithi022.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T07:17:38.352 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:38.352 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:17:38.411 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:17:38.411 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:17:38.411 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-09-11T07:17:38.411 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:17:38.411 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-09-11T07:17:38.411 INFO:teuthology.orchestra.run.smithi104.stdout: cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 83 k 2024-09-11T07:17:38.412 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:38.412 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-09-11T07:17:38.412 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:17:38.412 INFO:teuthology.orchestra.run.smithi104.stdout:Install 1 Package 2024-09-11T07:17:38.412 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:38.412 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 83 k 2024-09-11T07:17:38.412 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 375 k 2024-09-11T07:17:38.413 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-09-11T07:17:38.608 INFO:teuthology.orchestra.run.smithi104.stdout:cephadm-17.2.7-1623.gd0e6828a.el9.noarch.rpm 424 kB/s | 83 kB 00:00 2024-09-11T07:17:38.608 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:17:38.608 INFO:teuthology.orchestra.run.smithi104.stdout:Total 421 kB/s | 83 kB 00:00 2024-09-11T07:17:38.609 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-09-11T07:17:38.613 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-09-11T07:17:38.613 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-09-11T07:17:38.615 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-09-11T07:17:38.616 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-09-11T07:17:38.687 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-base 2024-09-11T07:17:38.695 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-09-11T07:17:39.132 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:17:39.220 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:17:39.280 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:00:53 ago on Wed 11 Sep 2024 07:16:46 AM UTC. 2024-09-11T07:17:39.373 INFO:teuthology.orchestra.run.smithi022.stdout:Package ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:17:39.416 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:17:39.417 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-09-11T07:17:39.418 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:17:39.488 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install cephadm 2024-09-11T07:17:39.623 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:17:39.890 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:17:39.890 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:39.890 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-09-11T07:17:39.890 INFO:teuthology.orchestra.run.smithi104.stdout: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:17:39.890 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:39.890 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:17:40.069 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:00:54 ago on Wed 11 Sep 2024 07:16:46 AM UTC. 2024-09-11T07:17:40.094 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-immutable-object-cache 2024-09-11T07:17:40.209 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:17:40.210 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:17:40.210 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-09-11T07:17:40.210 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:17:40.210 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-11T07:17:40.210 INFO:teuthology.orchestra.run.smithi022.stdout: cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 83 k 2024-09-11T07:17:40.210 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:40.210 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-11T07:17:40.210 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:17:40.210 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2024-09-11T07:17:40.211 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:40.211 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 83 k 2024-09-11T07:17:40.211 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 375 k 2024-09-11T07:17:40.211 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-11T07:17:40.413 INFO:teuthology.orchestra.run.smithi022.stdout:cephadm-17.2.7-1623.gd0e6828a.el9.noarch.rpm 410 kB/s | 83 kB 00:00 2024-09-11T07:17:40.413 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:17:40.414 INFO:teuthology.orchestra.run.smithi022.stdout:Total 407 kB/s | 83 kB 00:00 2024-09-11T07:17:40.414 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-11T07:17:40.419 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-11T07:17:40.419 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-11T07:17:40.421 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-11T07:17:40.421 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-11T07:17:40.497 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-11T07:17:40.667 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:00:53 ago on Wed 11 Sep 2024 07:16:47 AM UTC. 2024-09-11T07:17:40.800 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:17:40.801 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:17:40.801 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repo Size 2024-09-11T07:17:40.801 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:17:40.801 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-09-11T07:17:40.801 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 137 k 2024-09-11T07:17:40.801 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:40.802 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-09-11T07:17:40.802 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:17:40.802 INFO:teuthology.orchestra.run.smithi104.stdout:Install 1 Package 2024-09-11T07:17:40.802 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:40.802 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 137 k 2024-09-11T07:17:40.802 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 379 k 2024-09-11T07:17:40.802 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-09-11T07:17:40.904 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:17:41.002 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:17:41.015 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-immutable-object-cache-17.2.7-1623.gd0e682 646 kB/s | 137 kB 00:00 2024-09-11T07:17:41.015 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:17:41.016 INFO:teuthology.orchestra.run.smithi104.stdout:Total 642 kB/s | 137 kB 00:00 2024-09-11T07:17:41.016 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-09-11T07:17:41.023 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-09-11T07:17:41.023 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-09-11T07:17:41.080 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-09-11T07:17:41.080 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-09-11T07:17:41.236 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-09-11T07:17:41.324 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T07:17:41.348 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T07:17:41.348 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:17:41.348 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T07:17:41.348 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:41.411 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:17:41.669 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:17:41.669 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:41.669 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-11T07:17:41.670 INFO:teuthology.orchestra.run.smithi022.stdout: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:17:41.670 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:41.670 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:17:41.876 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-immutable-object-cache 2024-09-11T07:17:42.273 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T07:17:42.274 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:42.274 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-09-11T07:17:42.274 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:42.274 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:42.274 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:17:42.461 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:00:56 ago on Wed 11 Sep 2024 07:16:46 AM UTC. 2024-09-11T07:17:42.492 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-mgr 2024-09-11T07:17:42.598 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:17:42.599 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:17:42.599 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repo Size 2024-09-11T07:17:42.599 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:17:42.599 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-11T07:17:42.599 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 137 k 2024-09-11T07:17:42.599 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:42.599 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-11T07:17:42.599 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:17:42.599 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2024-09-11T07:17:42.599 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:42.600 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 137 k 2024-09-11T07:17:42.600 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 379 k 2024-09-11T07:17:42.600 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-11T07:17:42.811 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-immutable-object-cache-17.2.7-1623.gd0e682 652 kB/s | 137 kB 00:00 2024-09-11T07:17:42.811 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:17:42.812 INFO:teuthology.orchestra.run.smithi022.stdout:Total 647 kB/s | 137 kB 00:00 2024-09-11T07:17:42.812 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-11T07:17:42.820 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-11T07:17:42.820 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-11T07:17:42.881 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-11T07:17:42.881 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-11T07:17:43.045 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-11T07:17:43.066 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:00:56 ago on Wed 11 Sep 2024 07:16:47 AM UTC. 2024-09-11T07:17:43.145 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T07:17:43.160 INFO:teuthology.orchestra.run.smithi104.stdout:Package ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:17:43.171 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T07:17:43.172 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:17:43.172 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T07:17:43.172 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:43.201 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:17:43.202 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-09-11T07:17:43.202 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:17:43.271 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-mgr-dashboard 2024-09-11T07:17:43.843 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:00:56 ago on Wed 11 Sep 2024 07:16:47 AM UTC. 2024-09-11T07:17:43.975 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:17:43.975 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:17:43.976 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-09-11T07:17:43.976 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:17:43.976 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-09-11T07:17:43.976 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 3.4 M 2024-09-11T07:17:43.976 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-09-11T07:17:43.976 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 24 k 2024-09-11T07:17:43.976 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 15 k 2024-09-11T07:17:43.976 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-11T07:17:43.976 INFO:teuthology.orchestra.run.smithi104.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T07:17:43.976 INFO:teuthology.orchestra.run.smithi104.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T07:17:43.976 INFO:teuthology.orchestra.run.smithi104.stdout:Installing weak dependencies: 2024-09-11T07:17:43.977 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-11T07:17:43.977 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:43.977 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-09-11T07:17:43.977 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:17:43.977 INFO:teuthology.orchestra.run.smithi104.stdout:Install 7 Packages 2024-09-11T07:17:43.977 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:43.977 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 3.7 M 2024-09-11T07:17:43.978 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 80 M 2024-09-11T07:17:43.978 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-09-11T07:17:44.144 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T07:17:44.184 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:44.184 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-11T07:17:44.184 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:17:44.184 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:44.184 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:17:44.362 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-mgr 2024-09-11T07:17:44.534 INFO:teuthology.orchestra.run.smithi104.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1623.gd0e 158 kB/s | 24 kB 00:00 2024-09-11T07:17:44.559 INFO:teuthology.orchestra.run.smithi104.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1623.gd0e6 81 kB/s | 15 kB 00:00 2024-09-11T07:17:44.601 INFO:teuthology.orchestra.run.smithi104.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 136 kB/s | 9.0 kB 00:00 2024-09-11T07:17:44.627 INFO:teuthology.orchestra.run.smithi104.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-11T07:17:44.652 INFO:teuthology.orchestra.run.smithi104.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 444 kB/s | 41 kB 00:00 2024-09-11T07:17:44.669 INFO:teuthology.orchestra.run.smithi104.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 4.4 MB/s | 188 kB 00:00 2024-09-11T07:17:44.777 INFO:teuthology.orchestra.run.smithi104.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1623.gd0e6828a 8.4 MB/s | 3.4 MB 00:00 2024-09-11T07:17:44.781 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:17:44.781 INFO:teuthology.orchestra.run.smithi104.stdout:Total 4.5 MB/s | 3.7 MB 00:00 2024-09-11T07:17:44.781 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-09-11T07:17:44.789 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-09-11T07:17:44.789 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-09-11T07:17:44.872 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-09-11T07:17:44.873 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-09-11T07:17:44.953 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:00:58 ago on Wed 11 Sep 2024 07:16:46 AM UTC. 2024-09-11T07:17:45.047 INFO:teuthology.orchestra.run.smithi022.stdout:Package ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:17:45.091 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:17:45.092 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-09-11T07:17:45.092 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:17:45.150 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-mgr-dashboard 2024-09-11T07:17:45.180 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-09-11T07:17:45.237 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-11T07:17:45.271 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-11T07:17:45.317 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-11T07:17:45.357 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-11T07:17:45.387 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 5/7 2024-09-11T07:17:45.739 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:00:59 ago on Wed 11 Sep 2024 07:16:46 AM UTC. 2024-09-11T07:17:45.877 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:17:45.878 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:17:45.878 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-09-11T07:17:45.878 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:17:45.878 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-11T07:17:45.878 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 3.4 M 2024-09-11T07:17:45.879 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-09-11T07:17:45.879 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 24 k 2024-09-11T07:17:45.879 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 15 k 2024-09-11T07:17:45.879 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-11T07:17:45.879 INFO:teuthology.orchestra.run.smithi022.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T07:17:45.879 INFO:teuthology.orchestra.run.smithi022.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T07:17:45.879 INFO:teuthology.orchestra.run.smithi022.stdout:Installing weak dependencies: 2024-09-11T07:17:45.879 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-11T07:17:45.879 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:45.879 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-11T07:17:45.879 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:17:45.879 INFO:teuthology.orchestra.run.smithi022.stdout:Install 7 Packages 2024-09-11T07:17:45.879 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:45.880 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 3.7 M 2024-09-11T07:17:45.880 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 80 M 2024-09-11T07:17:45.880 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-11T07:17:46.371 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 6/7 2024-09-11T07:17:46.407 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T07:17:46.431 INFO:teuthology.orchestra.run.smithi022.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1623.gd0e 164 kB/s | 24 kB 00:00 2024-09-11T07:17:46.457 INFO:teuthology.orchestra.run.smithi022.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1623.gd0e6 84 kB/s | 15 kB 00:00 2024-09-11T07:17:46.499 INFO:teuthology.orchestra.run.smithi022.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 135 kB/s | 9.0 kB 00:00 2024-09-11T07:17:46.532 INFO:teuthology.orchestra.run.smithi022.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 983 kB/s | 31 kB 00:00 2024-09-11T07:17:46.558 INFO:teuthology.orchestra.run.smithi022.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 407 kB/s | 41 kB 00:00 2024-09-11T07:17:46.583 INFO:teuthology.orchestra.run.smithi022.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-11T07:17:46.691 INFO:teuthology.orchestra.run.smithi022.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1623.gd0e6828a 8.2 MB/s | 3.4 MB 00:00 2024-09-11T07:17:46.695 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:17:46.696 INFO:teuthology.orchestra.run.smithi022.stdout:Total 4.5 MB/s | 3.7 MB 00:00 2024-09-11T07:17:46.696 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-11T07:17:46.705 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-11T07:17:46.705 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-11T07:17:46.792 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-11T07:17:46.793 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-11T07:17:47.108 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-11T07:17:47.176 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-11T07:17:47.211 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-11T07:17:47.267 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-11T07:17:47.306 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-11T07:17:47.335 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 5/7 2024-09-11T07:17:47.452 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T07:17:47.452 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 1/7 2024-09-11T07:17:47.452 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 2/7 2024-09-11T07:17:47.452 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 3/7 2024-09-11T07:17:47.452 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-11T07:17:47.452 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-11T07:17:47.452 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-11T07:17:47.730 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-11T07:17:47.731 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:47.731 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-09-11T07:17:47.731 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:17:47.731 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:17:47.731 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:17:47.731 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-11T07:17:47.731 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-11T07:17:47.731 INFO:teuthology.orchestra.run.smithi104.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T07:17:47.731 INFO:teuthology.orchestra.run.smithi104.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T07:17:47.731 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:47.731 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:17:47.980 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-11T07:17:48.382 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 6/7 2024-09-11T07:17:48.430 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T07:17:48.539 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:01 ago on Wed 11 Sep 2024 07:16:47 AM UTC. 2024-09-11T07:17:48.676 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:17:48.677 INFO:teuthology.orchestra.run.smithi104.stdout:======================================================================================= 2024-09-11T07:17:48.677 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-09-11T07:17:48.677 INFO:teuthology.orchestra.run.smithi104.stdout:======================================================================================= 2024-09-11T07:17:48.677 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-09-11T07:17:48.678 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 7.4 M 2024-09-11T07:17:48.678 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-09-11T07:17:48.678 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T07:17:48.678 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T07:17:48.678 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T07:17:48.678 INFO:teuthology.orchestra.run.smithi104.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T07:17:48.678 INFO:teuthology.orchestra.run.smithi104.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T07:17:48.678 INFO:teuthology.orchestra.run.smithi104.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T07:17:48.678 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T07:17:48.678 INFO:teuthology.orchestra.run.smithi104.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T07:17:48.679 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T07:17:48.679 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T07:17:48.679 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T07:17:48.679 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:48.679 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-09-11T07:17:48.679 INFO:teuthology.orchestra.run.smithi104.stdout:======================================================================================= 2024-09-11T07:17:48.679 INFO:teuthology.orchestra.run.smithi104.stdout:Install 12 Packages 2024-09-11T07:17:48.679 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:48.680 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 38 M 2024-09-11T07:17:48.680 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 212 M 2024-09-11T07:17:48.680 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-09-11T07:17:48.927 INFO:teuthology.orchestra.run.smithi104.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 934 kB/s | 191 kB 00:00 2024-09-11T07:17:49.027 INFO:teuthology.orchestra.run.smithi104.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 328 kB/s | 33 kB 00:00 2024-09-11T07:17:49.060 INFO:teuthology.orchestra.run.smithi104.stdout:(3/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.3 MB/s | 801 kB 00:00 2024-09-11T07:17:49.136 INFO:teuthology.orchestra.run.smithi104.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 243 kB/s | 18 kB 00:00 2024-09-11T07:17:49.219 INFO:teuthology.orchestra.run.smithi104.stdout:(5/12): ceph-mgr-diskprediction-local-17.2.7-16 15 MB/s | 7.4 MB 00:00 2024-09-11T07:17:49.244 INFO:teuthology.orchestra.run.smithi104.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 354 kB/s | 38 kB 00:00 2024-09-11T07:17:49.303 INFO:teuthology.orchestra.run.smithi104.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 11 MB/s | 3.0 MB 00:00 2024-09-11T07:17:49.362 INFO:teuthology.orchestra.run.smithi104.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-11T07:17:49.437 INFO:teuthology.orchestra.run.smithi104.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 5.1 MB 00:00 2024-09-11T07:17:49.462 INFO:teuthology.orchestra.run.smithi104.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.4 MB/s | 250 kB 00:00 2024-09-11T07:17:49.545 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T07:17:49.545 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 1/7 2024-09-11T07:17:49.545 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 2/7 2024-09-11T07:17:49.546 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 3/7 2024-09-11T07:17:49.546 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-11T07:17:49.546 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-11T07:17:49.546 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-11T07:17:49.579 INFO:teuthology.orchestra.run.smithi104.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 14 MB/s | 4.9 MB 00:00 2024-09-11T07:17:49.754 INFO:teuthology.orchestra.run.smithi104.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 51 MB/s | 16 MB 00:00 2024-09-11T07:17:49.755 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:17:49.755 INFO:teuthology.orchestra.run.smithi104.stdout:Total 35 MB/s | 38 MB 00:01 2024-09-11T07:17:49.836 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-11T07:17:49.837 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:49.837 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-11T07:17:49.837 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:17:49.837 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:17:49.837 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:17:49.837 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-11T07:17:49.837 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-11T07:17:49.837 INFO:teuthology.orchestra.run.smithi022.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T07:17:49.837 INFO:teuthology.orchestra.run.smithi022.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T07:17:49.837 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:49.837 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:17:50.054 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-11T07:17:50.107 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-09-11T07:17:50.133 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-09-11T07:17:50.133 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-09-11T07:17:50.638 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:04 ago on Wed 11 Sep 2024 07:16:46 AM UTC. 2024-09-11T07:17:50.641 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-09-11T07:17:50.641 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-09-11T07:17:50.777 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:17:50.778 INFO:teuthology.orchestra.run.smithi022.stdout:======================================================================================= 2024-09-11T07:17:50.778 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-09-11T07:17:50.778 INFO:teuthology.orchestra.run.smithi022.stdout:======================================================================================= 2024-09-11T07:17:50.778 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-11T07:17:50.778 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 7.4 M 2024-09-11T07:17:50.778 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-09-11T07:17:50.778 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T07:17:50.778 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T07:17:50.778 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T07:17:50.779 INFO:teuthology.orchestra.run.smithi022.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T07:17:50.779 INFO:teuthology.orchestra.run.smithi022.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T07:17:50.779 INFO:teuthology.orchestra.run.smithi022.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T07:17:50.779 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T07:17:50.779 INFO:teuthology.orchestra.run.smithi022.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T07:17:50.779 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T07:17:50.779 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T07:17:50.779 INFO:teuthology.orchestra.run.smithi022.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T07:17:50.779 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:50.779 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-11T07:17:50.779 INFO:teuthology.orchestra.run.smithi022.stdout:======================================================================================= 2024-09-11T07:17:50.779 INFO:teuthology.orchestra.run.smithi022.stdout:Install 12 Packages 2024-09-11T07:17:50.779 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:17:50.780 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 38 M 2024-09-11T07:17:50.780 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 212 M 2024-09-11T07:17:50.780 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-11T07:17:51.414 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-09-11T07:17:51.496 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-11T07:17:51.536 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-11T07:17:51.636 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-11T07:17:51.679 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-11T07:17:51.733 INFO:teuthology.orchestra.run.smithi022.stdout:(1/12): ceph-mgr-diskprediction-local-17.2.7-16 17 MB/s | 7.4 MB 00:00 2024-09-11T07:17:51.842 INFO:teuthology.orchestra.run.smithi022.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 349 kB/s | 191 kB 00:00 2024-09-11T07:17:51.863 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-11T07:17:51.894 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-11T07:17:51.971 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-11T07:17:52.109 INFO:teuthology.orchestra.run.smithi022.stdout:(3/12): libgfortran-11.5.0-2.el9.x86_64.rpm 983 kB/s | 801 kB 00:00 2024-09-11T07:17:52.293 INFO:teuthology.orchestra.run.smithi022.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 58 kB/s | 33 kB 00:00 2024-09-11T07:17:52.576 INFO:teuthology.orchestra.run.smithi022.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 39 kB/s | 18 kB 00:00 2024-09-11T07:17:52.621 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-11T07:17:52.733 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-11T07:17:52.851 INFO:teuthology.orchestra.run.smithi022.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 69 kB/s | 38 kB 00:00 2024-09-11T07:17:52.910 INFO:teuthology.orchestra.run.smithi022.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.8 MB/s | 3.0 MB 00:01 2024-09-11T07:17:53.719 INFO:teuthology.orchestra.run.smithi022.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 282 kB/s | 245 kB 00:00 2024-09-11T07:17:54.002 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T07:17:54.338 INFO:teuthology.orchestra.run.smithi022.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.6 MB/s | 5.1 MB 00:01 2024-09-11T07:17:54.419 INFO:teuthology.orchestra.run.smithi022.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 2.7 MB/s | 4.9 MB 00:01 2024-09-11T07:17:54.494 INFO:teuthology.orchestra.run.smithi022.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 323 kB/s | 250 kB 00:00 2024-09-11T07:17:55.027 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-11T07:17:55.059 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T07:17:55.744 INFO:teuthology.orchestra.run.smithi022.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-09-11T07:17:55.745 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:17:55.746 INFO:teuthology.orchestra.run.smithi022.stdout:Total 7.7 MB/s | 38 MB 00:04 2024-09-11T07:17:56.120 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-11T07:17:56.147 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-11T07:17:56.147 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-11T07:17:56.672 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-11T07:17:56.673 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-11T07:17:57.460 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-11T07:17:57.541 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T07:17:57.541 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 1/12 2024-09-11T07:17:57.541 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-11T07:17:57.541 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-11T07:17:57.541 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-11T07:17:57.541 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-11T07:17:57.541 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-11T07:17:57.541 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-11T07:17:57.541 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-11T07:17:57.541 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-11T07:17:57.541 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T07:17:57.541 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-11T07:17:57.559 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-11T07:17:57.609 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-11T07:17:57.716 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-11T07:17:57.752 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-11T07:17:57.945 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-11T07:17:57.955 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-11T07:17:57.955 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:57.955 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-09-11T07:17:57.955 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:17:57.955 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T07:17:57.956 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T07:17:57.956 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T07:17:57.956 INFO:teuthology.orchestra.run.smithi104.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T07:17:57.956 INFO:teuthology.orchestra.run.smithi104.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T07:17:57.956 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T07:17:57.956 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T07:17:57.956 INFO:teuthology.orchestra.run.smithi104.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T07:17:57.956 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T07:17:57.956 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T07:17:57.956 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T07:17:57.956 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:57.956 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:17:57.975 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-11T07:17:58.059 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-11T07:17:58.327 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-mgr-rook 2024-09-11T07:17:58.752 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-11T07:17:58.869 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-11T07:17:58.891 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:11 ago on Wed 11 Sep 2024 07:16:47 AM UTC. 2024-09-11T07:17:59.024 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:17:59.025 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:17:59.025 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-09-11T07:17:59.025 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:17:59.025 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-09-11T07:17:59.025 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-rook noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 47 k 2024-09-11T07:17:59.025 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-09-11T07:17:59.025 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T07:17:59.026 INFO:teuthology.orchestra.run.smithi104.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T07:17:59.026 INFO:teuthology.orchestra.run.smithi104.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T07:17:59.026 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T07:17:59.026 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T07:17:59.026 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T07:17:59.026 INFO:teuthology.orchestra.run.smithi104.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T07:17:59.026 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T07:17:59.026 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T07:17:59.026 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T07:17:59.026 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T07:17:59.026 INFO:teuthology.orchestra.run.smithi104.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T07:17:59.026 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:59.026 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-09-11T07:17:59.026 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:17:59.027 INFO:teuthology.orchestra.run.smithi104.stdout:Install 13 Packages 2024-09-11T07:17:59.027 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:17:59.027 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 2.2 M 2024-09-11T07:17:59.027 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 26 M 2024-09-11T07:17:59.027 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-09-11T07:17:59.499 INFO:teuthology.orchestra.run.smithi104.stdout:(1/13): ceph-mgr-rook-17.2.7-1623.gd0e6828a.el9 275 kB/s | 47 kB 00:00 2024-09-11T07:17:59.867 INFO:teuthology.orchestra.run.smithi104.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 35 kB/s | 19 kB 00:00 2024-09-11T07:17:59.892 INFO:teuthology.orchestra.run.smithi104.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 45 kB/s | 26 kB 00:00 2024-09-11T07:18:00.215 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T07:18:00.408 INFO:teuthology.orchestra.run.smithi104.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 244 kB/s | 222 kB 00:00 2024-09-11T07:18:00.600 INFO:teuthology.orchestra.run.smithi104.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 216 kB/s | 159 kB 00:00 2024-09-11T07:18:00.650 INFO:teuthology.orchestra.run.smithi104.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 648 kB/s | 32 kB 00:00 2024-09-11T07:18:00.676 INFO:teuthology.orchestra.run.smithi104.stdout:(7/13): python3-certifi-2023.05.07-4.el9.noarch 565 kB/s | 14 kB 00:00 2024-09-11T07:18:00.701 INFO:teuthology.orchestra.run.smithi104.stdout:(8/13): python3-google-auth-2.34.0-1.el9.noarch 8.7 MB/s | 220 kB 00:00 2024-09-11T07:18:00.743 INFO:teuthology.orchestra.run.smithi104.stdout:(9/13): python3-kubernetes-26.1.0-2.el9.noarch. 25 MB/s | 1.0 MB 00:00 2024-09-11T07:18:00.768 INFO:teuthology.orchestra.run.smithi104.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-11T07:18:00.785 INFO:teuthology.orchestra.run.smithi104.stdout:(11/13): python3-pyasn1-modules-0.4.8-6.el9.noa 312 kB/s | 279 kB 00:00 2024-09-11T07:18:00.810 INFO:teuthology.orchestra.run.smithi104.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 2.1 MB/s | 90 kB 00:00 2024-09-11T07:18:01.027 INFO:teuthology.orchestra.run.smithi104.stdout:(13/13): python3-requests-oauthlib-1.3.0-12.el9 87 kB/s | 54 kB 00:00 2024-09-11T07:18:01.029 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:18:01.030 INFO:teuthology.orchestra.run.smithi104.stdout:Total 1.1 MB/s | 2.2 MB 00:02 2024-09-11T07:18:01.098 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-09-11T07:18:01.114 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-09-11T07:18:01.115 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-09-11T07:18:01.204 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-09-11T07:18:01.206 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-09-11T07:18:01.297 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-11T07:18:01.325 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T07:18:01.737 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-09-11T07:18:01.906 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-11T07:18:01.961 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-11T07:18:02.016 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-11T07:18:02.054 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-11T07:18:02.095 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-11T07:18:02.197 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-11T07:18:02.307 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-11T07:18:02.354 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-11T07:18:03.060 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-11T07:18:03.111 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-11T07:18:03.144 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-11T07:18:03.190 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-11T07:18:03.206 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T07:18:03.955 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T07:18:03.955 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 1/13 2024-09-11T07:18:03.955 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-11T07:18:03.955 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-11T07:18:03.956 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-11T07:18:03.956 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-11T07:18:03.956 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-11T07:18:03.956 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-11T07:18:03.956 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-11T07:18:03.956 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-11T07:18:03.956 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-11T07:18:03.956 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-11T07:18:03.956 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-11T07:18:04.030 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T07:18:04.031 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 1/12 2024-09-11T07:18:04.031 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-11T07:18:04.031 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-11T07:18:04.031 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-11T07:18:04.031 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-11T07:18:04.031 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-11T07:18:04.031 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-11T07:18:04.031 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-11T07:18:04.031 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-11T07:18:04.031 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T07:18:04.031 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-11T07:18:04.284 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-11T07:18:04.284 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:04.285 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-09-11T07:18:04.285 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:18:04.285 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T07:18:04.285 INFO:teuthology.orchestra.run.smithi104.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T07:18:04.285 INFO:teuthology.orchestra.run.smithi104.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T07:18:04.285 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T07:18:04.285 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T07:18:04.285 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T07:18:04.285 INFO:teuthology.orchestra.run.smithi104.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T07:18:04.288 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T07:18:04.288 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T07:18:04.288 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T07:18:04.288 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T07:18:04.288 INFO:teuthology.orchestra.run.smithi104.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T07:18:04.288 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:04.288 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:18:04.460 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-11T07:18:04.460 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:04.460 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-11T07:18:04.460 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:18:04.460 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T07:18:04.460 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T07:18:04.461 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T07:18:04.461 INFO:teuthology.orchestra.run.smithi022.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T07:18:04.461 INFO:teuthology.orchestra.run.smithi022.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T07:18:04.461 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T07:18:04.461 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T07:18:04.461 INFO:teuthology.orchestra.run.smithi022.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T07:18:04.461 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T07:18:04.461 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T07:18:04.461 INFO:teuthology.orchestra.run.smithi022.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T07:18:04.461 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:04.461 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:18:04.537 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-mgr-cephadm 2024-09-11T07:18:04.811 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-mgr-rook 2024-09-11T07:18:05.132 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:18 ago on Wed 11 Sep 2024 07:16:47 AM UTC. 2024-09-11T07:18:05.294 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:18:05.295 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:18:05.295 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-09-11T07:18:05.295 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:18:05.295 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-09-11T07:18:05.295 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 124 k 2024-09-11T07:18:05.295 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-09-11T07:18:05.295 INFO:teuthology.orchestra.run.smithi104.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T07:18:05.296 INFO:teuthology.orchestra.run.smithi104.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T07:18:05.296 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T07:18:05.296 INFO:teuthology.orchestra.run.smithi104.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T07:18:05.296 INFO:teuthology.orchestra.run.smithi104.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T07:18:05.296 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:05.296 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-09-11T07:18:05.296 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:18:05.296 INFO:teuthology.orchestra.run.smithi104.stdout:Install 6 Packages 2024-09-11T07:18:05.296 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:05.297 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 7.0 M 2024-09-11T07:18:05.297 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 33 M 2024-09-11T07:18:05.297 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-09-11T07:18:05.404 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:19 ago on Wed 11 Sep 2024 07:16:46 AM UTC. 2024-09-11T07:18:05.543 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:18:05.545 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:05.545 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-09-11T07:18:05.545 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:05.545 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-11T07:18:05.545 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-rook noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 47 k 2024-09-11T07:18:05.545 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-09-11T07:18:05.545 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T07:18:05.545 INFO:teuthology.orchestra.run.smithi022.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T07:18:05.545 INFO:teuthology.orchestra.run.smithi022.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T07:18:05.545 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T07:18:05.546 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T07:18:05.546 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T07:18:05.546 INFO:teuthology.orchestra.run.smithi022.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T07:18:05.546 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T07:18:05.546 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T07:18:05.546 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T07:18:05.546 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T07:18:05.546 INFO:teuthology.orchestra.run.smithi022.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T07:18:05.546 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:05.546 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-11T07:18:05.546 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:05.546 INFO:teuthology.orchestra.run.smithi022.stdout:Install 13 Packages 2024-09-11T07:18:05.546 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:05.547 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 2.2 M 2024-09-11T07:18:05.547 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 26 M 2024-09-11T07:18:05.547 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-11T07:18:05.587 INFO:teuthology.orchestra.run.smithi104.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.3 MB/s | 248 kB 00:00 2024-09-11T07:18:05.612 INFO:teuthology.orchestra.run.smithi104.stdout:(2/6): ceph-mgr-cephadm-17.2.7-1623.gd0e6828a.e 604 kB/s | 124 kB 00:00 2024-09-11T07:18:05.662 INFO:teuthology.orchestra.run.smithi104.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.1 MB/s | 548 kB 00:00 2024-09-11T07:18:05.688 INFO:teuthology.orchestra.run.smithi104.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 767 kB/s | 58 kB 00:00 2024-09-11T07:18:05.704 INFO:teuthology.orchestra.run.smithi104.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.7 MB/s | 74 kB 00:00 2024-09-11T07:18:05.826 INFO:teuthology.orchestra.run.smithi022.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 190 kB/s | 19 kB 00:00 2024-09-11T07:18:05.852 INFO:teuthology.orchestra.run.smithi022.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 203 kB/s | 26 kB 00:00 2024-09-11T07:18:05.910 INFO:teuthology.orchestra.run.smithi022.stdout:(3/13): ceph-mgr-rook-17.2.7-1623.gd0e6828a.el9 257 kB/s | 47 kB 00:00 2024-09-11T07:18:05.952 INFO:teuthology.orchestra.run.smithi022.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.7 MB/s | 222 kB 00:00 2024-09-11T07:18:05.986 INFO:teuthology.orchestra.run.smithi022.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-11T07:18:06.011 INFO:teuthology.orchestra.run.smithi022.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 918 kB/s | 54 kB 00:00 2024-09-11T07:18:06.053 INFO:teuthology.orchestra.run.smithi022.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 481 kB/s | 32 kB 00:00 2024-09-11T07:18:06.078 INFO:teuthology.orchestra.run.smithi022.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 211 kB/s | 14 kB 00:00 2024-09-11T07:18:06.104 INFO:teuthology.orchestra.run.smithi022.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.4 MB/s | 279 kB 00:00 2024-09-11T07:18:06.129 INFO:teuthology.orchestra.run.smithi022.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.9 MB/s | 220 kB 00:00 2024-09-11T07:18:06.163 INFO:teuthology.orchestra.run.smithi022.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.4 MB/s | 90 kB 00:00 2024-09-11T07:18:06.188 INFO:teuthology.orchestra.run.smithi022.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 705 kB/s | 59 kB 00:00 2024-09-11T07:18:06.221 INFO:teuthology.orchestra.run.smithi022.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.2 MB/s | 1.0 MB 00:00 2024-09-11T07:18:06.224 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:18:06.224 INFO:teuthology.orchestra.run.smithi022.stdout:Total 3.3 MB/s | 2.2 MB 00:00 2024-09-11T07:18:06.292 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-11T07:18:06.308 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-11T07:18:06.308 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-11T07:18:06.405 INFO:teuthology.orchestra.run.smithi104.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 6.0 MB/s | 6.0 MB 00:00 2024-09-11T07:18:06.405 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-11T07:18:06.405 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-11T07:18:06.407 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:18:06.407 INFO:teuthology.orchestra.run.smithi104.stdout:Total 6.3 MB/s | 7.0 MB 00:01 2024-09-11T07:18:06.476 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-09-11T07:18:06.487 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-09-11T07:18:06.487 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-09-11T07:18:06.587 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-09-11T07:18:06.589 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-09-11T07:18:06.902 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-09-11T07:18:07.050 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-11T07:18:07.076 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-11T07:18:07.123 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-11T07:18:07.227 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-11T07:18:07.299 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-11T07:18:07.364 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-11T07:18:07.408 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-11T07:18:07.449 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-11T07:18:07.557 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-11T07:18:07.646 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-11T07:18:07.665 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-11T07:18:07.719 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-11T07:18:07.725 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-11T07:18:07.780 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-11T07:18:07.809 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T07:18:08.481 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-11T07:18:08.531 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-11T07:18:08.565 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-11T07:18:08.583 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T07:18:08.583 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/6 2024-09-11T07:18:08.583 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-11T07:18:08.583 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-11T07:18:08.583 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-11T07:18:08.584 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-11T07:18:08.612 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-11T07:18:08.635 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T07:18:08.867 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-11T07:18:08.867 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:08.867 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-09-11T07:18:08.867 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:18:08.868 INFO:teuthology.orchestra.run.smithi104.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T07:18:08.868 INFO:teuthology.orchestra.run.smithi104.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T07:18:08.868 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T07:18:08.868 INFO:teuthology.orchestra.run.smithi104.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T07:18:08.868 INFO:teuthology.orchestra.run.smithi104.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T07:18:08.868 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:08.868 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:18:09.141 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-fuse 2024-09-11T07:18:09.440 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T07:18:09.440 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 1/13 2024-09-11T07:18:09.440 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-11T07:18:09.440 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-11T07:18:09.440 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-11T07:18:09.440 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-11T07:18:09.440 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-11T07:18:09.440 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-11T07:18:09.440 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-11T07:18:09.441 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-11T07:18:09.441 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-11T07:18:09.441 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-11T07:18:09.441 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-11T07:18:09.710 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:22 ago on Wed 11 Sep 2024 07:16:47 AM UTC. 2024-09-11T07:18:09.817 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-11T07:18:09.817 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:09.817 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-11T07:18:09.817 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:18:09.817 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T07:18:09.817 INFO:teuthology.orchestra.run.smithi022.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T07:18:09.817 INFO:teuthology.orchestra.run.smithi022.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T07:18:09.817 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T07:18:09.818 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T07:18:09.818 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T07:18:09.818 INFO:teuthology.orchestra.run.smithi022.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T07:18:09.818 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T07:18:09.818 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T07:18:09.818 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T07:18:09.818 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T07:18:09.818 INFO:teuthology.orchestra.run.smithi022.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T07:18:09.818 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:09.818 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:18:09.843 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:18:09.843 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:18:09.844 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-09-11T07:18:09.844 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:18:09.844 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-09-11T07:18:09.844 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 813 k 2024-09-11T07:18:09.844 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-09-11T07:18:09.844 INFO:teuthology.orchestra.run.smithi104.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T07:18:09.844 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:09.844 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-09-11T07:18:09.844 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:18:09.844 INFO:teuthology.orchestra.run.smithi104.stdout:Install 2 Packages 2024-09-11T07:18:09.844 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:09.845 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 893 k 2024-09-11T07:18:09.845 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 2.6 M 2024-09-11T07:18:09.845 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-09-11T07:18:10.022 INFO:teuthology.orchestra.run.smithi104.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 511 kB/s | 80 kB 00:00 2024-09-11T07:18:10.054 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-mgr-cephadm 2024-09-11T07:18:10.156 INFO:teuthology.orchestra.run.smithi104.stdout:(2/2): ceph-fuse-17.2.7-1623.gd0e6828a.el9.x86_ 2.7 MB/s | 813 kB 00:00 2024-09-11T07:18:10.156 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:18:10.157 INFO:teuthology.orchestra.run.smithi104.stdout:Total 2.8 MB/s | 893 kB 00:00 2024-09-11T07:18:10.168 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-09-11T07:18:10.189 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-09-11T07:18:10.189 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-09-11T07:18:10.274 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-09-11T07:18:10.275 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-09-11T07:18:10.475 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-09-11T07:18:10.598 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-11T07:18:10.645 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:24 ago on Wed 11 Sep 2024 07:16:46 AM UTC. 2024-09-11T07:18:10.663 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T07:18:10.785 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:18:10.786 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:10.786 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-09-11T07:18:10.786 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:10.786 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-11T07:18:10.786 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 124 k 2024-09-11T07:18:10.786 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-09-11T07:18:10.786 INFO:teuthology.orchestra.run.smithi022.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T07:18:10.786 INFO:teuthology.orchestra.run.smithi022.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T07:18:10.786 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T07:18:10.786 INFO:teuthology.orchestra.run.smithi022.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T07:18:10.786 INFO:teuthology.orchestra.run.smithi022.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T07:18:10.787 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:10.787 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-11T07:18:10.787 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:10.787 INFO:teuthology.orchestra.run.smithi022.stdout:Install 6 Packages 2024-09-11T07:18:10.787 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:10.787 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 7.0 M 2024-09-11T07:18:10.787 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 33 M 2024-09-11T07:18:10.787 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-11T07:18:11.307 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T07:18:11.307 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/2 2024-09-11T07:18:11.316 INFO:teuthology.orchestra.run.smithi022.stdout:(1/6): ceph-mgr-cephadm-17.2.7-1623.gd0e6828a.e 572 kB/s | 124 kB 00:00 2024-09-11T07:18:11.383 INFO:teuthology.orchestra.run.smithi022.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 8.0 MB/s | 548 kB 00:00 2024-09-11T07:18:11.400 INFO:teuthology.orchestra.run.smithi022.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 3.4 MB/s | 58 kB 00:00 2024-09-11T07:18:11.425 INFO:teuthology.orchestra.run.smithi022.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-11T07:18:11.576 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-11T07:18:11.576 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:11.576 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-09-11T07:18:11.576 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-11T07:18:11.576 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:11.577 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:18:11.801 INFO:teuthology.orchestra.run.smithi022.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 354 kB/s | 248 kB 00:00 2024-09-11T07:18:11.857 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-volume 2024-09-11T07:18:12.435 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:25 ago on Wed 11 Sep 2024 07:16:47 AM UTC. 2024-09-11T07:18:12.569 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:18:12.570 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:18:12.570 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-09-11T07:18:12.570 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:18:12.570 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-09-11T07:18:12.570 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-volume noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 270 k 2024-09-11T07:18:12.571 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:12.571 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-09-11T07:18:12.571 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:18:12.571 INFO:teuthology.orchestra.run.smithi104.stdout:Install 1 Package 2024-09-11T07:18:12.571 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:12.571 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 270 k 2024-09-11T07:18:12.571 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 1.3 M 2024-09-11T07:18:12.571 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-09-11T07:18:12.801 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-volume-17.2.7-1623.gd0e6828a.el9.noarch.rp 1.1 MB/s | 270 kB 00:00 2024-09-11T07:18:12.802 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:18:12.802 INFO:teuthology.orchestra.run.smithi104.stdout:Total 1.1 MB/s | 270 kB 00:00 2024-09-11T07:18:12.802 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-09-11T07:18:12.810 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-09-11T07:18:12.810 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-09-11T07:18:12.841 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-09-11T07:18:12.841 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-09-11T07:18:13.054 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-09-11T07:18:13.128 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:18:13.145 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:18:13.145 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:18:13.145 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T07:18:13.145 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:14.166 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:18:14.166 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:14.166 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-09-11T07:18:14.166 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:18:14.166 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:14.166 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:18:14.466 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install librados-devel 2024-09-11T07:18:14.784 INFO:teuthology.orchestra.run.smithi022.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 1.6 MB/s | 6.0 MB 00:03 2024-09-11T07:18:14.786 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:18:14.786 INFO:teuthology.orchestra.run.smithi022.stdout:Total 1.7 MB/s | 7.0 MB 00:03 2024-09-11T07:18:14.863 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-11T07:18:14.902 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-11T07:18:14.902 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-11T07:18:14.982 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-11T07:18:14.983 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-11T07:18:15.042 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:28 ago on Wed 11 Sep 2024 07:16:47 AM UTC. 2024-09-11T07:18:15.180 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:18:15.181 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:18:15.181 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repo Size 2024-09-11T07:18:15.181 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:18:15.181 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-09-11T07:18:15.181 INFO:teuthology.orchestra.run.smithi104.stdout: librados-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 126 k 2024-09-11T07:18:15.182 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:15.182 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-09-11T07:18:15.182 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:18:15.182 INFO:teuthology.orchestra.run.smithi104.stdout:Install 1 Package 2024-09-11T07:18:15.182 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:15.182 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 126 k 2024-09-11T07:18:15.182 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 448 k 2024-09-11T07:18:15.182 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-09-11T07:18:15.312 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-11T07:18:15.389 INFO:teuthology.orchestra.run.smithi104.stdout:librados-devel-17.2.7-1623.gd0e6828a.el9.x86_64 610 kB/s | 126 kB 00:00 2024-09-11T07:18:15.390 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:18:15.390 INFO:teuthology.orchestra.run.smithi104.stdout:Total 605 kB/s | 126 kB 00:00 2024-09-11T07:18:15.390 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-09-11T07:18:15.396 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-09-11T07:18:15.396 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-09-11T07:18:15.460 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-09-11T07:18:15.461 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-09-11T07:18:15.482 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-11T07:18:15.545 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-11T07:18:15.610 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-09-11T07:18:15.703 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:16.094 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-11T07:18:16.125 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:16.182 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-11T07:18:16.254 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-11T07:18:16.281 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T07:18:16.383 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:16.383 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:16.383 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-09-11T07:18:16.383 INFO:teuthology.orchestra.run.smithi104.stdout: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:18:16.383 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:16.383 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:18:16.656 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install libcephfs2 2024-09-11T07:18:17.051 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T07:18:17.051 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/6 2024-09-11T07:18:17.051 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-11T07:18:17.051 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-11T07:18:17.051 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-11T07:18:17.051 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-11T07:18:17.229 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:30 ago on Wed 11 Sep 2024 07:16:47 AM UTC. 2024-09-11T07:18:17.322 INFO:teuthology.orchestra.run.smithi104.stdout:Package libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:18:17.365 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:18:17.366 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-09-11T07:18:17.367 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:18:17.457 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-11T07:18:17.457 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:17.457 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-11T07:18:17.458 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:18:17.458 INFO:teuthology.orchestra.run.smithi022.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T07:18:17.458 INFO:teuthology.orchestra.run.smithi022.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T07:18:17.458 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T07:18:17.458 INFO:teuthology.orchestra.run.smithi022.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T07:18:17.458 INFO:teuthology.orchestra.run.smithi022.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T07:18:17.458 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:17.458 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:18:17.459 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install libcephfs-devel 2024-09-11T07:18:17.712 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-fuse 2024-09-11T07:18:18.026 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:31 ago on Wed 11 Sep 2024 07:16:47 AM UTC. 2024-09-11T07:18:18.158 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:18:18.160 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:18:18.160 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repo Size 2024-09-11T07:18:18.160 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:18:18.160 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-09-11T07:18:18.160 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T07:18:18.160 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:18.160 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-09-11T07:18:18.160 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:18:18.160 INFO:teuthology.orchestra.run.smithi104.stdout:Install 1 Package 2024-09-11T07:18:18.160 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:18.161 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 25 k 2024-09-11T07:18:18.161 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 110 k 2024-09-11T07:18:18.161 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-09-11T07:18:18.301 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:32 ago on Wed 11 Sep 2024 07:16:46 AM UTC. 2024-09-11T07:18:18.308 INFO:teuthology.orchestra.run.smithi104.stdout:libcephfs-devel-17.2.7-1623.gd0e6828a.el9.x86_6 168 kB/s | 25 kB 00:00 2024-09-11T07:18:18.309 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:18:18.309 INFO:teuthology.orchestra.run.smithi104.stdout:Total 167 kB/s | 25 kB 00:00 2024-09-11T07:18:18.309 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-09-11T07:18:18.313 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-09-11T07:18:18.313 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-09-11T07:18:18.314 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-09-11T07:18:18.315 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-09-11T07:18:18.404 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-09-11T07:18:18.440 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:18:18.441 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:18.441 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-09-11T07:18:18.441 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:18.441 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-11T07:18:18.441 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 813 k 2024-09-11T07:18:18.441 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-09-11T07:18:18.442 INFO:teuthology.orchestra.run.smithi022.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T07:18:18.442 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:18.442 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-11T07:18:18.442 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:18.442 INFO:teuthology.orchestra.run.smithi022.stdout:Install 2 Packages 2024-09-11T07:18:18.442 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:18.442 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 893 k 2024-09-11T07:18:18.442 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 2.6 M 2024-09-11T07:18:18.443 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-11T07:18:18.505 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:18.635 INFO:teuthology.orchestra.run.smithi022.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 546 kB/s | 80 kB 00:00 2024-09-11T07:18:18.760 INFO:teuthology.orchestra.run.smithi022.stdout:(2/2): ceph-fuse-17.2.7-1623.gd0e6828a.el9.x86_ 2.9 MB/s | 813 kB 00:00 2024-09-11T07:18:18.761 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:18:18.761 INFO:teuthology.orchestra.run.smithi022.stdout:Total 2.7 MB/s | 893 kB 00:00 2024-09-11T07:18:18.774 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-11T07:18:18.796 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-11T07:18:18.797 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-11T07:18:18.859 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:18.885 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-11T07:18:18.885 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-11T07:18:19.083 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-11T07:18:19.097 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:19.097 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:19.097 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-09-11T07:18:19.097 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:18:19.097 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:19.097 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:18:19.241 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-11T07:18:19.281 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install librados2 2024-09-11T07:18:19.321 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T07:18:19.849 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:32 ago on Wed 11 Sep 2024 07:16:47 AM UTC. 2024-09-11T07:18:19.938 INFO:teuthology.orchestra.run.smithi104.stdout:Package librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:18:19.982 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:18:19.983 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-09-11T07:18:19.983 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:18:20.017 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T07:18:20.017 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/2 2024-09-11T07:18:20.058 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install librbd1 2024-09-11T07:18:20.297 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-11T07:18:20.298 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:20.298 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-11T07:18:20.298 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-11T07:18:20.298 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:20.298 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:18:20.583 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-volume 2024-09-11T07:18:20.617 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:33 ago on Wed 11 Sep 2024 07:16:47 AM UTC. 2024-09-11T07:18:20.706 INFO:teuthology.orchestra.run.smithi104.stdout:Package librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:18:20.748 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:18:20.749 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-09-11T07:18:20.749 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:18:20.812 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install python3-rados 2024-09-11T07:18:21.178 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:35 ago on Wed 11 Sep 2024 07:16:46 AM UTC. 2024-09-11T07:18:21.316 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:18:21.317 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:21.317 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-09-11T07:18:21.317 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:21.317 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-11T07:18:21.317 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-volume noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 270 k 2024-09-11T07:18:21.317 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:21.317 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-11T07:18:21.317 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:21.317 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2024-09-11T07:18:21.317 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:21.318 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 270 k 2024-09-11T07:18:21.318 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 1.3 M 2024-09-11T07:18:21.318 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-11T07:18:21.375 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:34 ago on Wed 11 Sep 2024 07:16:47 AM UTC. 2024-09-11T07:18:21.465 INFO:teuthology.orchestra.run.smithi104.stdout:Package python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:18:21.508 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:18:21.509 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-09-11T07:18:21.509 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:18:21.581 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install python3-rgw 2024-09-11T07:18:21.616 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-volume-17.2.7-1623.gd0e6828a.el9.noarch.rp 905 kB/s | 270 kB 00:00 2024-09-11T07:18:21.617 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:18:21.617 INFO:teuthology.orchestra.run.smithi022.stdout:Total 901 kB/s | 270 kB 00:00 2024-09-11T07:18:21.617 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-11T07:18:21.626 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-11T07:18:21.626 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-11T07:18:21.663 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-11T07:18:21.664 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-11T07:18:21.887 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-11T07:18:21.968 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:18:21.985 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:18:21.985 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:18:21.985 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T07:18:21.985 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:22.141 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:35 ago on Wed 11 Sep 2024 07:16:47 AM UTC. 2024-09-11T07:18:22.230 INFO:teuthology.orchestra.run.smithi104.stdout:Package python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:18:22.274 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:18:22.275 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-09-11T07:18:22.275 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:18:22.349 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install python3-cephfs 2024-09-11T07:18:22.912 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:35 ago on Wed 11 Sep 2024 07:16:47 AM UTC. 2024-09-11T07:18:23.002 INFO:teuthology.orchestra.run.smithi104.stdout:Package python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:18:23.029 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:18:23.029 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:23.030 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-11T07:18:23.030 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:18:23.030 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:23.030 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:18:23.044 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:18:23.045 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-09-11T07:18:23.045 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:18:23.120 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install python3-rbd 2024-09-11T07:18:23.324 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install librados-devel 2024-09-11T07:18:23.685 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:36 ago on Wed 11 Sep 2024 07:16:47 AM UTC. 2024-09-11T07:18:23.775 INFO:teuthology.orchestra.run.smithi104.stdout:Package python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:18:23.818 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:18:23.819 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-09-11T07:18:23.819 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:18:23.899 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install rbd-fuse 2024-09-11T07:18:23.922 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:37 ago on Wed 11 Sep 2024 07:16:46 AM UTC. 2024-09-11T07:18:24.061 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:18:24.062 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:24.062 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repo Size 2024-09-11T07:18:24.062 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:24.062 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-11T07:18:24.062 INFO:teuthology.orchestra.run.smithi022.stdout: librados-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 126 k 2024-09-11T07:18:24.062 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:24.062 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-11T07:18:24.062 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:24.062 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2024-09-11T07:18:24.062 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:24.062 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 126 k 2024-09-11T07:18:24.063 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 448 k 2024-09-11T07:18:24.063 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-11T07:18:24.280 INFO:teuthology.orchestra.run.smithi022.stdout:librados-devel-17.2.7-1623.gd0e6828a.el9.x86_64 582 kB/s | 126 kB 00:00 2024-09-11T07:18:24.280 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:18:24.281 INFO:teuthology.orchestra.run.smithi022.stdout:Total 578 kB/s | 126 kB 00:00 2024-09-11T07:18:24.281 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-11T07:18:24.287 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-11T07:18:24.287 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-11T07:18:24.355 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-11T07:18:24.355 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-11T07:18:24.457 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:37 ago on Wed 11 Sep 2024 07:16:47 AM UTC. 2024-09-11T07:18:24.511 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-11T07:18:24.590 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:18:24.591 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:18:24.591 INFO:teuthology.orchestra.run.smithi104.stdout: Package Architecture Version Repository Size 2024-09-11T07:18:24.591 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:18:24.591 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-09-11T07:18:24.592 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 82 k 2024-09-11T07:18:24.592 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:24.592 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-09-11T07:18:24.592 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:18:24.592 INFO:teuthology.orchestra.run.smithi104.stdout:Install 1 Package 2024-09-11T07:18:24.592 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:24.592 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 82 k 2024-09-11T07:18:24.592 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 222 k 2024-09-11T07:18:24.592 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-09-11T07:18:24.639 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:24.786 INFO:teuthology.orchestra.run.smithi104.stdout:rbd-fuse-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 422 kB/s | 82 kB 00:00 2024-09-11T07:18:24.787 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:18:24.787 INFO:teuthology.orchestra.run.smithi104.stdout:Total 419 kB/s | 82 kB 00:00 2024-09-11T07:18:24.787 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-09-11T07:18:24.793 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-09-11T07:18:24.794 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-09-11T07:18:24.857 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-09-11T07:18:24.857 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-09-11T07:18:25.002 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-09-11T07:18:25.092 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:25.100 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:25.372 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:25.372 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:25.372 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-11T07:18:25.372 INFO:teuthology.orchestra.run.smithi022.stdout: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:18:25.372 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:25.373 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:18:25.475 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:25.651 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install libcephfs2 2024-09-11T07:18:25.747 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:25.747 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:25.747 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-09-11T07:18:25.747 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:18:25.748 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:25.748 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:18:26.027 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install rbd-mirror 2024-09-11T07:18:26.243 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:40 ago on Wed 11 Sep 2024 07:16:46 AM UTC. 2024-09-11T07:18:26.338 INFO:teuthology.orchestra.run.smithi022.stdout:Package libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:18:26.385 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:18:26.386 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-09-11T07:18:26.386 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:18:26.444 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install libcephfs-devel 2024-09-11T07:18:26.605 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:39 ago on Wed 11 Sep 2024 07:16:47 AM UTC. 2024-09-11T07:18:26.742 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:18:26.742 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:18:26.742 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repo Size 2024-09-11T07:18:26.743 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:18:26.743 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-09-11T07:18:26.743 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-mirror x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T07:18:26.743 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:26.743 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-09-11T07:18:26.743 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:18:26.743 INFO:teuthology.orchestra.run.smithi104.stdout:Install 1 Package 2024-09-11T07:18:26.743 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:26.743 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 2.9 M 2024-09-11T07:18:26.744 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 12 M 2024-09-11T07:18:26.744 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-09-11T07:18:27.032 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:41 ago on Wed 11 Sep 2024 07:16:46 AM UTC. 2024-09-11T07:18:27.103 INFO:teuthology.orchestra.run.smithi104.stdout:rbd-mirror-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 8.2 MB/s | 2.9 MB 00:00 2024-09-11T07:18:27.104 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:18:27.104 INFO:teuthology.orchestra.run.smithi104.stdout:Total 8.1 MB/s | 2.9 MB 00:00 2024-09-11T07:18:27.104 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-09-11T07:18:27.113 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-09-11T07:18:27.113 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-09-11T07:18:27.173 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:18:27.173 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:27.174 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repo Size 2024-09-11T07:18:27.174 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:27.174 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-11T07:18:27.174 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T07:18:27.174 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:27.174 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-11T07:18:27.174 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:27.174 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2024-09-11T07:18:27.174 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:27.174 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 25 k 2024-09-11T07:18:27.174 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 110 k 2024-09-11T07:18:27.174 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-11T07:18:27.193 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-09-11T07:18:27.193 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-09-11T07:18:27.367 INFO:teuthology.orchestra.run.smithi022.stdout:libcephfs-devel-17.2.7-1623.gd0e6828a.el9.x86_6 129 kB/s | 25 kB 00:00 2024-09-11T07:18:27.368 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:18:27.368 INFO:teuthology.orchestra.run.smithi022.stdout:Total 129 kB/s | 25 kB 00:00 2024-09-11T07:18:27.368 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-11T07:18:27.372 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-11T07:18:27.372 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-11T07:18:27.373 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-11T07:18:27.374 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-11T07:18:27.459 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-11T07:18:27.575 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:27.600 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-09-11T07:18:27.673 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:27.698 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:27.698 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:18:27.698 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T07:18:27.698 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T07:18:27.698 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T07:18:27.698 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:27.951 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:28.187 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:28.188 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:28.188 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-11T07:18:28.188 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:18:28.188 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:28.188 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:18:28.358 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install librados2 2024-09-11T07:18:28.778 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:28.779 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:28.779 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-09-11T07:18:28.779 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:18:28.779 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:28.779 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:18:28.942 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:42 ago on Wed 11 Sep 2024 07:16:46 AM UTC. 2024-09-11T07:18:29.028 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install rbd-nbd 2024-09-11T07:18:29.034 INFO:teuthology.orchestra.run.smithi022.stdout:Package librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:18:29.079 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:18:29.080 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-09-11T07:18:29.081 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:18:29.145 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install librbd1 2024-09-11T07:18:29.606 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:42 ago on Wed 11 Sep 2024 07:16:47 AM UTC. 2024-09-11T07:18:29.731 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:43 ago on Wed 11 Sep 2024 07:16:46 AM UTC. 2024-09-11T07:18:29.743 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-11T07:18:29.744 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:18:29.744 INFO:teuthology.orchestra.run.smithi104.stdout: Package Architecture Version Repository Size 2024-09-11T07:18:29.744 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:18:29.744 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-09-11T07:18:29.744 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-nbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 165 k 2024-09-11T07:18:29.744 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:29.745 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-09-11T07:18:29.745 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-09-11T07:18:29.745 INFO:teuthology.orchestra.run.smithi104.stdout:Install 1 Package 2024-09-11T07:18:29.745 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:29.745 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 165 k 2024-09-11T07:18:29.745 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 477 k 2024-09-11T07:18:29.746 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-09-11T07:18:29.825 INFO:teuthology.orchestra.run.smithi022.stdout:Package librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:18:29.871 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:18:29.872 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-09-11T07:18:29.872 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:18:29.943 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install python3-rados 2024-09-11T07:18:29.958 INFO:teuthology.orchestra.run.smithi104.stdout:rbd-nbd-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 776 kB/s | 165 kB 00:00 2024-09-11T07:18:29.958 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:18:29.959 INFO:teuthology.orchestra.run.smithi104.stdout:Total 771 kB/s | 165 kB 00:00 2024-09-11T07:18:29.959 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-09-11T07:18:29.967 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-09-11T07:18:29.967 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-09-11T07:18:30.031 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-09-11T07:18:30.032 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-09-11T07:18:30.184 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-09-11T07:18:30.306 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:30.527 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:44 ago on Wed 11 Sep 2024 07:16:46 AM UTC. 2024-09-11T07:18:30.621 INFO:teuthology.orchestra.run.smithi022.stdout:Package python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:18:30.666 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:18:30.667 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-09-11T07:18:30.667 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:18:30.727 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install python3-rgw 2024-09-11T07:18:30.729 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:31.013 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:31.013 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:31.013 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-09-11T07:18:31.013 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:18:31.013 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:18:31.013 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-11T07:18:31.278 DEBUG:teuthology.parallel:result is None 2024-09-11T07:18:31.317 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:45 ago on Wed 11 Sep 2024 07:16:46 AM UTC. 2024-09-11T07:18:31.413 INFO:teuthology.orchestra.run.smithi022.stdout:Package python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:18:31.460 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:18:31.461 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-09-11T07:18:31.461 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:18:31.519 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install python3-cephfs 2024-09-11T07:18:32.123 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:46 ago on Wed 11 Sep 2024 07:16:46 AM UTC. 2024-09-11T07:18:32.217 INFO:teuthology.orchestra.run.smithi022.stdout:Package python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:18:32.262 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:18:32.264 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-09-11T07:18:32.264 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:18:32.336 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install python3-rbd 2024-09-11T07:18:32.927 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:46 ago on Wed 11 Sep 2024 07:16:46 AM UTC. 2024-09-11T07:18:33.019 INFO:teuthology.orchestra.run.smithi022.stdout:Package python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:18:33.064 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:18:33.065 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-09-11T07:18:33.065 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:18:33.124 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install rbd-fuse 2024-09-11T07:18:33.713 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:47 ago on Wed 11 Sep 2024 07:16:46 AM UTC. 2024-09-11T07:18:33.850 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:18:33.851 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:33.851 INFO:teuthology.orchestra.run.smithi022.stdout: Package Architecture Version Repository Size 2024-09-11T07:18:33.851 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:33.851 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-11T07:18:33.851 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 82 k 2024-09-11T07:18:33.851 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:33.851 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-11T07:18:33.851 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:33.851 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2024-09-11T07:18:33.851 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:33.851 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 82 k 2024-09-11T07:18:33.852 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 222 k 2024-09-11T07:18:33.852 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-11T07:18:34.046 INFO:teuthology.orchestra.run.smithi022.stdout:rbd-fuse-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 421 kB/s | 82 kB 00:00 2024-09-11T07:18:34.047 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:18:34.047 INFO:teuthology.orchestra.run.smithi022.stdout:Total 418 kB/s | 82 kB 00:00 2024-09-11T07:18:34.047 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-11T07:18:34.055 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-11T07:18:34.055 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-11T07:18:34.121 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-11T07:18:34.121 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-11T07:18:34.269 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-11T07:18:34.379 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:34.774 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:35.038 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:35.038 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:35.038 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-11T07:18:35.039 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:18:35.039 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:35.039 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:18:35.266 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install rbd-mirror 2024-09-11T07:18:35.861 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:49 ago on Wed 11 Sep 2024 07:16:46 AM UTC. 2024-09-11T07:18:36.004 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:18:36.005 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:36.005 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repo Size 2024-09-11T07:18:36.005 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:36.005 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-11T07:18:36.005 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-mirror x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T07:18:36.005 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:36.005 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-11T07:18:36.005 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:36.005 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2024-09-11T07:18:36.006 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:36.006 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 2.9 M 2024-09-11T07:18:36.006 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 12 M 2024-09-11T07:18:36.006 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-11T07:18:36.341 INFO:teuthology.orchestra.run.smithi022.stdout:rbd-mirror-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 8.8 MB/s | 2.9 MB 00:00 2024-09-11T07:18:36.342 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:18:36.342 INFO:teuthology.orchestra.run.smithi022.stdout:Total 8.7 MB/s | 2.9 MB 00:00 2024-09-11T07:18:36.342 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-11T07:18:36.352 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-11T07:18:36.352 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-11T07:18:36.436 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-11T07:18:36.437 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-11T07:18:36.872 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-11T07:18:36.933 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:36.961 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:36.961 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:18:36.961 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T07:18:36.961 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T07:18:36.961 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T07:18:36.961 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:38.178 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:38.178 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:38.178 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-11T07:18:38.178 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:18:38.178 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:38.178 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:18:38.387 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install rbd-nbd 2024-09-11T07:18:38.970 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:52 ago on Wed 11 Sep 2024 07:16:46 AM UTC. 2024-09-11T07:18:39.109 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-11T07:18:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:39.110 INFO:teuthology.orchestra.run.smithi022.stdout: Package Architecture Version Repository Size 2024-09-11T07:18:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-11T07:18:39.110 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-nbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 165 k 2024-09-11T07:18:39.110 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-11T07:18:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-11T07:18:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2024-09-11T07:18:39.110 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:39.110 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 165 k 2024-09-11T07:18:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 477 k 2024-09-11T07:18:39.111 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-11T07:18:39.320 INFO:teuthology.orchestra.run.smithi022.stdout:rbd-nbd-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 790 kB/s | 165 kB 00:00 2024-09-11T07:18:39.321 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:18:39.321 INFO:teuthology.orchestra.run.smithi022.stdout:Total 784 kB/s | 165 kB 00:00 2024-09-11T07:18:39.321 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-11T07:18:39.329 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-11T07:18:39.329 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-11T07:18:39.396 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-11T07:18:39.396 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-11T07:18:39.561 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-11T07:18:39.670 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:40.069 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:40.345 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:18:40.346 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:40.346 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-11T07:18:40.346 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:18:40.346 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:18:40.346 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-11T07:18:40.557 DEBUG:teuthology.parallel:result is None 2024-09-11T07:18:40.558 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:18:40.702 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T07:18:40.724 INFO:teuthology.orchestra.run.smithi022.stdout:17.2.7-1623.gd0e6828a.el9 2024-09-11T07:18:40.725 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1623.gd0e6828a.el9 2024-09-11T07:18:40.725 INFO:teuthology.task.install:The correct ceph version 17.2.7-1623.gd0e6828a is installed. 2024-09-11T07:18:40.727 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:18:40.877 DEBUG:teuthology.orchestra.run.smithi104:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T07:18:40.902 INFO:teuthology.orchestra.run.smithi104.stdout:17.2.7-1623.gd0e6828a.el9 2024-09-11T07:18:40.902 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1623.gd0e6828a.el9 2024-09-11T07:18:40.902 INFO:teuthology.task.install:The correct ceph version 17.2.7-1623.gd0e6828a is installed. 2024-09-11T07:18:40.904 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-11T07:18:40.905 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-11T07:18:40.905 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T07:18:40.931 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-11T07:18:40.932 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T07:18:40.972 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-11T07:18:40.972 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-11T07:18:40.973 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T07:18:40.998 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T07:18:41.064 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-11T07:18:41.064 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T07:18:41.091 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T07:18:41.159 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-11T07:18:41.160 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-11T07:18:41.160 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T07:18:41.185 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T07:18:41.251 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-11T07:18:41.251 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T07:18:41.279 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T07:18:41.346 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-11T07:18:41.346 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-11T07:18:41.346 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T07:18:41.371 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T07:18:41.437 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-11T07:18:41.437 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T07:18:41.463 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T07:18:41.529 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-11T07:18:41.626 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'ms bind msgr2': False, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)'], 'mon_bind_msgr2': False, 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9', 'cephadm_mode': 'root'} 2024-09-11T07:18:41.626 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:18:41.626 INFO:tasks.cephadm:Cluster fsid is 12a285d8-700e-11ef-bcea-c7b262605968 2024-09-11T07:18:41.627 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-11T07:18:41.627 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '[v1:172.21.15.22:6789]', 'mon.c': '[v1:172.21.15.22:6790]', 'mon.b': '[v1:172.21.15.104:6789]'} 2024-09-11T07:18:41.627 INFO:tasks.cephadm:First mon is mon.a on smithi022 2024-09-11T07:18:41.627 INFO:tasks.cephadm:First mgr is y 2024-09-11T07:18:41.627 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-11T07:18:41.627 DEBUG:teuthology.orchestra.run.smithi022:> sudo hostname $(hostname -s) 2024-09-11T07:18:41.654 DEBUG:teuthology.orchestra.run.smithi104:> sudo hostname $(hostname -s) 2024-09-11T07:18:41.679 INFO:tasks.cephadm:Downloading cephadm (repo https://git.ceph.com/ceph-ci.git ref d0e6828a2016d48cf25ad84064e50742bb1c39b9)... 2024-09-11T07:18:41.679 DEBUG:teuthology.orchestra.run.smithi022:> git clone https://git.ceph.com/ceph-ci.git testrepo && cd testrepo && git show d0e6828a2016d48cf25ad84064e50742bb1c39b9:src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T07:18:41.711 INFO:teuthology.orchestra.run.smithi022.stderr:Cloning into 'testrepo'... 2024-09-11T07:19:39.827 INFO:teuthology.orchestra.run.smithi022.stderr:Updating files: 92% (11776/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-11T07:19:39.862 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-r--r--. 1 ubuntu ubuntu 378585 Sep 11 07:19 /home/ubuntu/cephtest/cephadm 2024-09-11T07:19:39.863 DEBUG:teuthology.orchestra.run.smithi104:> git clone https://git.ceph.com/ceph-ci.git testrepo && cd testrepo && git show d0e6828a2016d48cf25ad84064e50742bb1c39b9:src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T07:19:39.882 INFO:teuthology.orchestra.run.smithi104.stderr:Cloning into 'testrepo'... 2024-09-11T07:20:37.473 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r--. 1 ubuntu ubuntu 378585 Sep 11 07:20 /home/ubuntu/cephtest/cephadm 2024-09-11T07:20:37.474 DEBUG:teuthology.orchestra.run.smithi022:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T07:20:37.497 DEBUG:teuthology.orchestra.run.smithi104:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T07:20:37.543 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 on all hosts... 2024-09-11T07:20:37.543 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 pull 2024-09-11T07:20:37.545 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 pull 2024-09-11T07:20:39.132 INFO:teuthology.orchestra.run.smithi022.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T07:20:40.992 INFO:teuthology.orchestra.run.smithi104.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T07:21:04.740 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:21:04.741 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T07:21:04.741 INFO:teuthology.orchestra.run.smithi022.stdout: "image_id": "4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b", 2024-09-11T07:21:04.741 INFO:teuthology.orchestra.run.smithi022.stdout: "repo_digests": [ 2024-09-11T07:21:04.741 INFO:teuthology.orchestra.run.smithi022.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40" 2024-09-11T07:21:04.741 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:21:04.741 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:21:06.882 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:21:06.882 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T07:21:06.882 INFO:teuthology.orchestra.run.smithi104.stdout: "image_id": "4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b", 2024-09-11T07:21:06.882 INFO:teuthology.orchestra.run.smithi104.stdout: "repo_digests": [ 2024-09-11T07:21:06.882 INFO:teuthology.orchestra.run.smithi104.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40" 2024-09-11T07:21:06.883 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:21:06.883 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:21:06.900 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /etc/ceph 2024-09-11T07:21:06.936 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /etc/ceph 2024-09-11T07:21:06.963 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 777 /etc/ceph 2024-09-11T07:21:07.005 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 777 /etc/ceph 2024-09-11T07:21:07.029 INFO:tasks.cephadm:Writing seed config... 2024-09-11T07:21:07.031 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-11T07:21:07.031 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-11T07:21:07.031 INFO:tasks.cephadm: override: [client.0] debug ms = 1 2024-09-11T07:21:07.031 INFO:tasks.cephadm: override: [client.0] debug objecter = 20 2024-09-11T07:21:07.031 INFO:tasks.cephadm: override: [client.0] debug rados = 20 2024-09-11T07:21:07.031 INFO:tasks.cephadm: override: [global] ms bind msgr2 = False 2024-09-11T07:21:07.031 INFO:tasks.cephadm: override: [global] ms type = async 2024-09-11T07:21:07.031 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-11T07:21:07.031 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-11T07:21:07.031 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-11T07:21:07.031 INFO:tasks.cephadm: override: [mon] mon min osdmap epochs = 50 2024-09-11T07:21:07.032 INFO:tasks.cephadm: override: [mon] mon osdmap full prune interval = 2 2024-09-11T07:21:07.032 INFO:tasks.cephadm: override: [mon] mon osdmap full prune min = 15 2024-09-11T07:21:07.032 INFO:tasks.cephadm: override: [mon] mon osdmap full prune txsize = 2 2024-09-11T07:21:07.032 INFO:tasks.cephadm: override: [mon] paxos service trim min = 10 2024-09-11T07:21:07.032 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-11T07:21:07.032 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-11T07:21:07.032 INFO:tasks.cephadm: override: [osd] osd debug reject backfill probability = 0.3 2024-09-11T07:21:07.032 INFO:tasks.cephadm: override: [osd] osd delete sleep = 1 2024-09-11T07:21:07.032 INFO:tasks.cephadm: override: [osd] osd max backfills = 3 2024-09-11T07:21:07.032 INFO:tasks.cephadm: override: [osd] osd scrub max interval = 120 2024-09-11T07:21:07.032 INFO:tasks.cephadm: override: [osd] osd scrub min interval = 60 2024-09-11T07:21:07.032 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-11T07:21:07.032 INFO:tasks.cephadm: override: [osd] osd snap trim sleep = 2 2024-09-11T07:21:07.033 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-11T07:21:07.033 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-11T07:21:07.063 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 12a285d8-700e-11ef-bcea-c7b262605968 ms bind msgr2 = False ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 120 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd debug reject backfill probability = 0.3 osd delete sleep = 1 osd max backfills = 3 osd scrub min interval = 60 osd snap trim sleep = 2 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon min osdmap epochs = 50 mon osdmap full prune interval = 2 mon osdmap full prune min = 15 mon osdmap full prune txsize = 2 paxos service trim min = 10 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client.0] debug ms = 1 debug objecter = 20 debug rados = 20 2024-09-11T07:21:07.064 DEBUG:teuthology.orchestra.run.smithi022:mon.a> sudo journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@mon.a.service 2024-09-11T07:21:07.106 DEBUG:teuthology.orchestra.run.smithi022:mgr.y> sudo journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@mgr.y.service 2024-09-11T07:21:07.150 INFO:tasks.cephadm:Bootstrapping... 2024-09-11T07:21:07.150 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 -v bootstrap --fsid 12a285d8-700e-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-addrv '[v1:172.21.15.22:6789]' --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:21:07.360 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:21:07.361 INFO:teuthology.orchestra.run.smithi022.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9', '-v', 'bootstrap', '--fsid', '12a285d8-700e-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-addrv', '[v1:172.21.15.22:6789]', '--skip-admin-label'] 2024-09-11T07:21:07.385 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout 5.2.2 2024-09-11T07:21:07.386 INFO:teuthology.orchestra.run.smithi022.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-11T07:21:07.386 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying podman|docker is present... 2024-09-11T07:21:07.408 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout 5.2.2 2024-09-11T07:21:07.408 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying lvm2 is present... 2024-09-11T07:21:07.408 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying time synchronization is in place... 2024-09-11T07:21:07.417 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T07:21:07.417 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T07:21:07.423 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T07:21:07.423 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-09-11T07:21:07.431 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout enabled 2024-09-11T07:21:07.438 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout active 2024-09-11T07:21:07.438 INFO:teuthology.orchestra.run.smithi022.stdout:Unit chronyd.service is enabled and running 2024-09-11T07:21:07.438 INFO:teuthology.orchestra.run.smithi022.stdout:Repeating the final host check... 2024-09-11T07:21:07.461 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout 5.2.2 2024-09-11T07:21:07.461 INFO:teuthology.orchestra.run.smithi022.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-11T07:21:07.461 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl is present 2024-09-11T07:21:07.462 INFO:teuthology.orchestra.run.smithi022.stdout:lvcreate is present 2024-09-11T07:21:07.468 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T07:21:07.469 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T07:21:07.475 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T07:21:07.476 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-09-11T07:21:07.482 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout enabled 2024-09-11T07:21:07.489 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout active 2024-09-11T07:21:07.489 INFO:teuthology.orchestra.run.smithi022.stdout:Unit chronyd.service is enabled and running 2024-09-11T07:21:07.490 INFO:teuthology.orchestra.run.smithi022.stdout:Host looks OK 2024-09-11T07:21:07.490 INFO:teuthology.orchestra.run.smithi022.stdout:Cluster fsid: 12a285d8-700e-11ef-bcea-c7b262605968 2024-09-11T07:21:07.490 INFO:teuthology.orchestra.run.smithi022.stdout:Acquiring lock 140514927694752 on /run/cephadm/12a285d8-700e-11ef-bcea-c7b262605968.lock 2024-09-11T07:21:07.490 INFO:teuthology.orchestra.run.smithi022.stdout:Lock 140514927694752 acquired on /run/cephadm/12a285d8-700e-11ef-bcea-c7b262605968.lock 2024-09-11T07:21:07.490 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying IP 172.21.15.22 port 6789 ... 2024-09-11T07:21:07.490 INFO:teuthology.orchestra.run.smithi022.stdout:Base mon IP(s) is [172.21.15.22:6789], mon addrv is [v1:172.21.15.22:6789] 2024-09-11T07:21:07.494 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.22 metric 100 2024-09-11T07:21:07.494 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.22 metric 100 2024-09-11T07:21:07.497 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-11T07:21:07.497 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-11T07:21:07.497 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1795sec hoplimit 64 pref medium 2024-09-11T07:21:07.500 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-11T07:21:07.500 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-11T07:21:07.500 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T07:21:07.500 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-11T07:21:07.500 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-09-11T07:21:07.501 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T07:21:07.501 INFO:teuthology.orchestra.run.smithi022.stdout:Mon IP `172.21.15.22` is in CIDR network `172.21.0.0/20` 2024-09-11T07:21:07.501 INFO:teuthology.orchestra.run.smithi022.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20'] 2024-09-11T07:21:07.501 INFO:teuthology.orchestra.run.smithi022.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-11T07:21:07.502 INFO:teuthology.orchestra.run.smithi022.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T07:21:07.972 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b 2024-09-11T07:21:07.973 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T07:21:07.973 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Getting image source signatures 2024-09-11T07:21:07.973 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Copying blob sha256:0d2f264d63cec2f6fb0f85eed7b2c8c5b4db3ab9edd9d71a74c04647294a553f 2024-09-11T07:21:07.973 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-11T07:21:07.973 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Copying config sha256:4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b 2024-09-11T07:21:07.973 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-11T07:21:08.885 INFO:teuthology.orchestra.run.smithi022.stdout:ceph: stdout ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable) 2024-09-11T07:21:08.885 INFO:teuthology.orchestra.run.smithi022.stdout:Ceph version: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable) 2024-09-11T07:21:08.885 INFO:teuthology.orchestra.run.smithi022.stdout:Extracting ceph user uid/gid from container image... 2024-09-11T07:21:09.833 INFO:teuthology.orchestra.run.smithi022.stdout:stat: stdout 167 167 2024-09-11T07:21:09.833 INFO:teuthology.orchestra.run.smithi022.stdout:Creating initial keys... 2024-09-11T07:21:10.930 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph-authtool: stdout AQDmROFmBEs5CxAAzQDmccPlyi/HjNyNcBLcbA== 2024-09-11T07:21:11.777 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph-authtool: stdout AQDnROFmFVb6CxAAVKY6rj/33ywvgBx7YBaZOw== 2024-09-11T07:21:12.676 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph-authtool: stdout AQDoROFmFKnoBRAAmqk04c8Wrbb+sXlOdzi5KQ== 2024-09-11T07:21:12.677 INFO:teuthology.orchestra.run.smithi022.stdout:Creating initial monmap... 2024-09-11T07:21:13.746 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T07:21:13.746 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-09-11T07:21:13.747 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 12a285d8-700e-11ef-bcea-c7b262605968 2024-09-11T07:21:13.747 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T07:21:13.747 INFO:teuthology.orchestra.run.smithi022.stdout:monmaptool for a [v1:172.21.15.22:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T07:21:13.747 INFO:teuthology.orchestra.run.smithi022.stdout:setting min_mon_release = octopus 2024-09-11T07:21:13.747 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: set fsid to 12a285d8-700e-11ef-bcea-c7b262605968 2024-09-11T07:21:13.747 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T07:21:13.747 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:21:13.747 INFO:teuthology.orchestra.run.smithi022.stdout:Creating mon... 2024-09-11T07:21:14.602 INFO:teuthology.orchestra.run.smithi022.stdout:create mon.a on 2024-09-11T07:21:14.816 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-11T07:21:14.962 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-11T07:21:15.190 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-12a285d8-700e-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-12a285d8-700e-11ef-bcea-c7b262605968.target. 2024-09-11T07:21:15.190 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-12a285d8-700e-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-12a285d8-700e-11ef-bcea-c7b262605968.target. 2024-09-11T07:21:15.429 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-12a285d8-700e-11ef-bcea-c7b262605968@mon.a 2024-09-11T07:21:15.430 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to reset failed state of unit ceph-12a285d8-700e-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-12a285d8-700e-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-11T07:21:15.697 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-12a285d8-700e-11ef-bcea-c7b262605968.target.wants/ceph-12a285d8-700e-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-12a285d8-700e-11ef-bcea-c7b262605968@.service. 2024-09-11T07:21:16.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:15 smithi022 podman[29065]: 2024-09-11 07:21:15.859624362 +0000 UTC m=+0.016095839 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:21:16.971 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:16 smithi022 podman[29065]: 2024-09-11 07:21:16.758514035 +0000 UTC m=+0.914985505 container create a466991290567a850cd569173fd0f6264046fb78e42499021e01ad093ea6e255 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-11T07:21:18.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:18 smithi022 podman[29065]: 2024-09-11 07:21:18.651177402 +0000 UTC m=+2.807648875 container init a466991290567a850cd569173fd0f6264046fb78e42499021e01ad093ea6e255 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, ceph=True) 2024-09-11T07:21:18.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:18 smithi022 podman[29065]: 2024-09-11 07:21:18.655950193 +0000 UTC m=+2.812421661 container start a466991290567a850cd569173fd0f6264046fb78e42499021e01ad093ea6e255 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-11T07:21:18.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:18 smithi022 ceph-mon[29096]: mkfs 12a285d8-700e-11ef-bcea-c7b262605968 2024-09-11T07:21:18.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:18 smithi022 ceph-mon[29096]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T07:21:19.573 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T07:21:19.573 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-09-11T07:21:19.582 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T07:21:19.582 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-09-11T07:21:19.582 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-09-11T07:21:19.582 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T07:21:19.583 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mon to start... 2024-09-11T07:21:19.583 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mon... 2024-09-11T07:21:19.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:19 smithi022 bash[29065]: a466991290567a850cd569173fd0f6264046fb78e42499021e01ad093ea6e255 2024-09-11T07:21:19.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:19 smithi022 systemd[1]: Started Ceph mon.a for 12a285d8-700e-11ef-bcea-c7b262605968. 2024-09-11T07:21:19.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:19 smithi022 ceph-mon[29096]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T07:21:19.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:19 smithi022 ceph-mon[29096]: monmap e1: 1 mons at {a=v1:172.21.15.22:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:21:19.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:19 smithi022 ceph-mon[29096]: fsmap 2024-09-11T07:21:19.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:19 smithi022 ceph-mon[29096]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T07:21:19.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:19 smithi022 ceph-mon[29096]: mgrmap e1: no daemons active 2024-09-11T07:21:23.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:22 smithi022 ceph-mon[29096]: from='client.? v1:172.21.15.22:0/76981626' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-11T07:21:26.623 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout cluster: 2024-09-11T07:21:26.623 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout id: 12a285d8-700e-11ef-bcea-c7b262605968 2024-09-11T07:21:26.623 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-11T07:21:26.623 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-09-11T07:21:26.623 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout services: 2024-09-11T07:21:26.623 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 4s) 2024-09-11T07:21:26.623 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-11T07:21:26.623 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-11T07:21:26.624 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-09-11T07:21:26.624 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout data: 2024-09-11T07:21:26.624 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-11T07:21:26.624 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-11T07:21:26.624 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-11T07:21:26.624 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout pgs: 2024-09-11T07:21:26.624 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-09-11T07:21:26.624 INFO:teuthology.orchestra.run.smithi022.stdout:mon is available 2024-09-11T07:21:26.624 INFO:teuthology.orchestra.run.smithi022.stdout:Assimilating anything we can from ceph.conf... 2024-09-11T07:21:27.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:27 smithi022 ceph-mon[29096]: from='client.? v1:172.21.15.22:0/3983831212' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T07:21:27.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:27 smithi022 ceph-mon[29096]: from='client.? v1:172.21.15.22:0/3983831212' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-11T07:21:27.670 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-09-11T07:21:27.670 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T07:21:27.670 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout fsid = 12a285d8-700e-11ef-bcea-c7b262605968 2024-09-11T07:21:27.670 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-11T07:21:27.670 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_host = [v1:172.21.15.22:6789] 2024-09-11T07:21:27.670 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T07:21:27.671 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T07:21:27.671 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T07:21:27.671 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T07:21:27.671 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-09-11T07:21:27.671 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T07:21:27.671 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T07:21:27.671 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-09-11T07:21:27.671 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T07:21:27.671 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T07:21:27.671 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T07:21:27.671 INFO:teuthology.orchestra.run.smithi022.stdout:Generating new minimal ceph.conf... 2024-09-11T07:21:28.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:28 smithi022 ceph-mon[29096]: from='client.? v1:172.21.15.22:0/2656735222' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:21:28.667 INFO:teuthology.orchestra.run.smithi022.stdout:Restarting the monitor... 2024-09-11T07:21:28.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:28 smithi022 systemd[1]: Stopping Ceph mon.a for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:21:29.401 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:28 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-a[29091]: 2024-09-11T07:21:28.948+0000 7f740c350640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:21:29.401 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:28 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-a[29091]: 2024-09-11T07:21:28.948+0000 7f740c350640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-11T07:21:29.401 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:29 smithi022 podman[29339]: 2024-09-11 07:21:29.133918143 +0000 UTC m=+0.290877266 container died a466991290567a850cd569173fd0f6264046fb78e42499021e01ad093ea6e255 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-11T07:21:29.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:29 smithi022 podman[29339]: 2024-09-11 07:21:29.490677404 +0000 UTC m=+0.647636535 container cleanup a466991290567a850cd569173fd0f6264046fb78e42499021e01ad093ea6e255 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:21:29.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:29 smithi022 bash[29339]: ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-a 2024-09-11T07:21:30.102 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:29 smithi022 podman[29351]: 2024-09-11 07:21:29.758475198 +0000 UTC m=+0.616583502 container remove a466991290567a850cd569173fd0f6264046fb78e42499021e01ad093ea6e255 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-11T07:21:30.102 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:29 smithi022 systemd[1]: ceph-12a285d8-700e-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-11T07:21:30.103 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:29 smithi022 systemd[1]: Stopped Ceph mon.a for 12a285d8-700e-11ef-bcea-c7b262605968. 2024-09-11T07:21:30.103 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:29 smithi022 systemd[1]: Starting Ceph mon.a for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:21:30.390 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 podman[29443]: 2024-09-11 07:21:30.101106645 +0000 UTC m=+0.107222534 container create e6840d670ba28df616e91626938892c2ae2ab42e6b832943fa4315a6ba29d612 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-09-11T07:21:30.391 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 podman[29443]: 2024-09-11 07:21:30.009764237 +0000 UTC m=+0.015880082 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:21:30.391 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 podman[29443]: 2024-09-11 07:21:30.303558649 +0000 UTC m=+0.309674494 container init e6840d670ba28df616e91626938892c2ae2ab42e6b832943fa4315a6ba29d612 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:21:30.391 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 podman[29443]: 2024-09-11 07:21:30.306672404 +0000 UTC m=+0.312788261 container start e6840d670ba28df616e91626938892c2ae2ab42e6b832943fa4315a6ba29d612 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:21:30.391 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T07:21:30.391 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable), process ceph-mon, pid 2 2024-09-11T07:21:30.391 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: pidfile_write: ignore empty --pid-file 2024-09-11T07:21:30.391 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: load: jerasure load: lrc 2024-09-11T07:21:30.391 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: RocksDB version: 6.15.5 2024-09-11T07:21:30.391 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-11T07:21:30.391 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Compile date Sep 10 2024 2024-09-11T07:21:30.392 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: DB SUMMARY 2024-09-11T07:21:30.392 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: DB Session ID: 856GTEYNCEBK897GJ2AZ 2024-09-11T07:21:30.392 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: CURRENT file: CURRENT 2024-09-11T07:21:30.392 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T07:21:30.392 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-09-11T07:21:30.392 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-11T07:21:30.392 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 88067 ; 2024-09-11T07:21:30.392 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.error_if_exists: 0 2024-09-11T07:21:30.392 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.create_if_missing: 0 2024-09-11T07:21:30.392 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.paranoid_checks: 1 2024-09-11T07:21:30.392 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T07:21:30.392 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.env: 0x5605f05dc440 2024-09-11T07:21:30.393 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.fs: Posix File System 2024-09-11T07:21:30.393 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.info_log: 0x5605f269ae60 2024-09-11T07:21:30.393 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T07:21:30.393 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.statistics: (nil) 2024-09-11T07:21:30.393 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.use_fsync: 0 2024-09-11T07:21:30.393 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.max_log_file_size: 0 2024-09-11T07:21:30.393 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T07:21:30.393 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T07:21:30.393 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T07:21:30.393 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T07:21:30.393 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.allow_fallocate: 1 2024-09-11T07:21:30.393 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T07:21:30.394 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T07:21:30.394 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.use_direct_reads: 0 2024-09-11T07:21:30.394 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T07:21:30.394 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T07:21:30.394 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.db_log_dir: 2024-09-11T07:21:30.394 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-09-11T07:21:30.394 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T07:21:30.394 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T07:21:30.394 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T07:21:30.394 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T07:21:30.394 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T07:21:30.394 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T07:21:30.394 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T07:21:30.394 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T07:21:30.394 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.write_buffer_manager: 0x5605f2758780 2024-09-11T07:21:30.395 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T07:21:30.395 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-11T07:21:30.395 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T07:21:30.395 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T07:21:30.395 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T07:21:30.395 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T07:21:30.396 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T07:21:30.396 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T07:21:30.396 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T07:21:30.396 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.unordered_write: 0 2024-09-11T07:21:30.397 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T07:21:30.397 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T07:21:30.397 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T07:21:30.397 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T07:21:30.397 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.row_cache: None 2024-09-11T07:21:30.397 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.wal_filter: None 2024-09-11T07:21:30.397 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T07:21:30.397 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T07:21:30.397 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.preserve_deletes: 0 2024-09-11T07:21:30.397 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.two_write_queues: 0 2024-09-11T07:21:30.397 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T07:21:30.397 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.atomic_flush: 0 2024-09-11T07:21:30.397 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T07:21:30.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T07:21:30.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T07:21:30.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.log_readahead_size: 0 2024-09-11T07:21:30.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T07:21:30.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T07:21:30.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T07:21:30.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T07:21:30.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T07:21:30.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T07:21:30.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.max_background_jobs: 2 2024-09-11T07:21:30.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.max_background_compactions: -1 2024-09-11T07:21:30.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.max_subcompactions: 1 2024-09-11T07:21:30.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T07:21:30.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T07:21:30.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T07:21:30.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T07:21:30.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T07:21:30.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T07:21:30.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T07:21:30.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T07:21:30.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.max_open_files: -1 2024-09-11T07:21:30.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T07:21:30.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T07:21:30.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T07:21:30.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T07:21:30.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.max_background_flushes: -1 2024-09-11T07:21:30.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Compression algorithms supported: 2024-09-11T07:21:30.400 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: kZSTD supported: 0 2024-09-11T07:21:30.400 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: kXpressCompression supported: 0 2024-09-11T07:21:30.400 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: kBZip2Compression supported: 0 2024-09-11T07:21:30.400 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T07:21:30.400 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: kLZ4Compression supported: 1 2024-09-11T07:21:30.400 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: kZlibCompression supported: 1 2024-09-11T07:21:30.400 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T07:21:30.400 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: kSnappyCompression supported: 1 2024-09-11T07:21:30.400 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T07:21:30.400 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2024-09-11T07:21:30.400 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-11T07:21:30.400 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T07:21:30.401 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.merge_operator: 2024-09-11T07:21:30.401 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.compaction_filter: None 2024-09-11T07:21:30.401 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T07:21:30.401 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T07:21:30.401 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T07:21:30.401 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T07:21:30.401 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5605f2641a38) 2024-09-11T07:21:30.401 INFO:journalctl@ceph.mon.a.smithi022.stdout: cache_index_and_filter_blocks: 1 2024-09-11T07:21:30.401 INFO:journalctl@ceph.mon.a.smithi022.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T07:21:30.401 INFO:journalctl@ceph.mon.a.smithi022.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T07:21:30.401 INFO:journalctl@ceph.mon.a.smithi022.stdout: pin_top_level_index_and_filter: 1 2024-09-11T07:21:30.402 INFO:journalctl@ceph.mon.a.smithi022.stdout: index_type: 0 2024-09-11T07:21:30.402 INFO:journalctl@ceph.mon.a.smithi022.stdout: data_block_index_type: 0 2024-09-11T07:21:30.402 INFO:journalctl@ceph.mon.a.smithi022.stdout: index_shortening: 1 2024-09-11T07:21:30.402 INFO:journalctl@ceph.mon.a.smithi022.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T07:21:30.402 INFO:journalctl@ceph.mon.a.smithi022.stdout: hash_index_allow_collision: 1 2024-09-11T07:21:30.402 INFO:journalctl@ceph.mon.a.smithi022.stdout: checksum: 1 2024-09-11T07:21:30.402 INFO:journalctl@ceph.mon.a.smithi022.stdout: no_block_cache: 0 2024-09-11T07:21:30.402 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache: 0x5605f26771f0 2024-09-11T07:21:30.402 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache_name: BinnedLRUCache 2024-09-11T07:21:30.402 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache_options: 2024-09-11T07:21:30.402 INFO:journalctl@ceph.mon.a.smithi022.stdout: capacity : 536870912 2024-09-11T07:21:30.402 INFO:journalctl@ceph.mon.a.smithi022.stdout: num_shard_bits : 4 2024-09-11T07:21:30.402 INFO:journalctl@ceph.mon.a.smithi022.stdout: strict_capacity_limit : 0 2024-09-11T07:21:30.402 INFO:journalctl@ceph.mon.a.smithi022.stdout: high_pri_pool_ratio: 0.000 2024-09-11T07:21:30.403 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache_compressed: (nil) 2024-09-11T07:21:30.403 INFO:journalctl@ceph.mon.a.smithi022.stdout: persistent_cache: (nil) 2024-09-11T07:21:30.403 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_size: 4096 2024-09-11T07:21:30.403 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_size_deviation: 10 2024-09-11T07:21:30.403 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_restart_interval: 16 2024-09-11T07:21:30.403 INFO:journalctl@ceph.mon.a.smithi022.stdout: index_block_restart_interval: 1 2024-09-11T07:21:30.403 INFO:journalctl@ceph.mon.a.smithi022.stdout: metadata_block_size: 4096 2024-09-11T07:21:30.403 INFO:journalctl@ceph.mon.a.smithi022.stdout: partition_filters: 0 2024-09-11T07:21:30.403 INFO:journalctl@ceph.mon.a.smithi022.stdout: use_delta_encoding: 1 2024-09-11T07:21:30.403 INFO:journalctl@ceph.mon.a.smithi022.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-11T07:21:30.403 INFO:journalctl@ceph.mon.a.smithi022.stdout: whole_key_filtering: 1 2024-09-11T07:21:30.403 INFO:journalctl@ceph.mon.a.smithi022.stdout: verify_compression: 0 2024-09-11T07:21:30.403 INFO:journalctl@ceph.mon.a.smithi022.stdout: read_amp_bytes_per_bit: 0 2024-09-11T07:21:30.403 INFO:journalctl@ceph.mon.a.smithi022.stdout: format_version: 4 2024-09-11T07:21:30.404 INFO:journalctl@ceph.mon.a.smithi022.stdout: enable_index_compression: 1 2024-09-11T07:21:30.404 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_align: 0 2024-09-11T07:21:30.404 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T07:21:30.404 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T07:21:30.404 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.compression: NoCompression 2024-09-11T07:21:30.404 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T07:21:30.404 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T07:21:30.404 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T07:21:30.404 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.num_levels: 7 2024-09-11T07:21:30.405 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T07:21:30.405 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T07:21:30.405 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T07:21:30.405 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T07:21:30.405 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T07:21:30.405 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T07:21:30.405 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T07:21:30.405 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T07:21:30.405 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T07:21:30.405 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T07:21:30.405 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T07:21:30.405 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T07:21:30.405 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T07:21:30.405 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T07:21:30.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T07:21:30.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T07:21:30.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T07:21:30.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T07:21:30.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T07:21:30.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T07:21:30.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T07:21:30.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T07:21:30.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T07:21:30.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T07:21:30.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T07:21:30.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T07:21:30.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T07:21:30.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T07:21:30.406 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T07:21:30.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T07:21:30.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T07:21:30.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T07:21:30.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T07:21:30.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T07:21:30.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.arena_block_size: 4194304 2024-09-11T07:21:30.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T07:21:30.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T07:21:30.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-11T07:21:30.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T07:21:30.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T07:21:30.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T07:21:30.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T07:21:30.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T07:21:30.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T07:21:30.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T07:21:30.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T07:21:30.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T07:21:30.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T07:21:30.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T07:21:30.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T07:21:30.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.inplace_update_support: 0 2024-09-11T07:21:30.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T07:21:30.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T07:21:30.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T07:21:30.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T07:21:30.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.bloom_locality: 0 2024-09-11T07:21:30.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.max_successive_merges: 0 2024-09-11T07:21:30.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T07:21:30.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T07:21:30.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T07:21:30.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T07:21:30.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.ttl: 2592000 2024-09-11T07:21:30.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T07:21:30.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.enable_blob_files: false 2024-09-11T07:21:30.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.min_blob_size: 0 2024-09-11T07:21:30.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T07:21:30.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T07:21:30.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T07:21:30.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T07:21:30.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-11T07:21:30.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-09-11T07:21:30.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-09-11T07:21:30.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726039290330459, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-09-11T07:21:30.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-09-11T07:21:30.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-11T07:21:30.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726039290331361, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 83989, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 82078, "index_size": 245, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 11453, "raw_average_key_size": 48, "raw_value_size": 75742, "raw_average_value_size": 319, "num_data_blocks": 11, "num_entries": 237, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1726039290, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "b5264608-20ae-4f3e-aaff-a9de986e45f5", "db_session_id": "856GTEYNCEBK897GJ2AZ"}} 2024-09-11T07:21:30.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-09-11T07:21:30.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726039290331709, "job": 1, "event": "recovery_finished"} 2024-09-11T07:21:30.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T07:21:30.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x5605f26aa700 2024-09-11T07:21:30.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: DB pointer 0x5605f267a000 2024-09-11T07:21:30.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-11T07:21:30.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-11T07:21:30.410 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** DB Stats ** 2024-09-11T07:21:30.410 INFO:journalctl@ceph.mon.a.smithi022.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:21:30.410 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T07:21:30.410 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T07:21:30.411 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T07:21:30.411 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T07:21:30.411 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-11T07:21:30.411 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T07:21:30.411 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-09-11T07:21:30.411 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** Compaction Stats [default] ** 2024-09-11T07:21:30.411 INFO:journalctl@ceph.mon.a.smithi022.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:21:30.411 INFO:journalctl@ceph.mon.a.smithi022.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:21:30.411 INFO:journalctl@ceph.mon.a.smithi022.stdout: L0 2/0 83.77 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 131.1 0.00 0.00 1 0.001 0 0 2024-09-11T07:21:30.411 INFO:journalctl@ceph.mon.a.smithi022.stdout: Sum 2/0 83.77 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 131.1 0.00 0.00 1 0.001 0 0 2024-09-11T07:21:30.411 INFO:journalctl@ceph.mon.a.smithi022.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 131.1 0.00 0.00 1 0.001 0 0 2024-09-11T07:21:30.411 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-09-11T07:21:30.411 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** Compaction Stats [default] ** 2024-09-11T07:21:30.411 INFO:journalctl@ceph.mon.a.smithi022.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:21:30.411 INFO:journalctl@ceph.mon.a.smithi022.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:21:30.412 INFO:journalctl@ceph.mon.a.smithi022.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 131.1 0.00 0.00 1 0.001 0 0 2024-09-11T07:21:30.412 INFO:journalctl@ceph.mon.a.smithi022.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:21:30.412 INFO:journalctl@ceph.mon.a.smithi022.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T07:21:30.412 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T07:21:30.412 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T07:21:30.412 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T07:21:30.412 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T07:21:30.412 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative compaction: 0.00 GB write, 24.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:21:30.412 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval compaction: 0.00 GB write, 24.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:21:30.412 INFO:journalctl@ceph.mon.a.smithi022.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T07:21:30.412 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-09-11T07:21:30.412 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T07:21:30.412 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-09-11T07:21:30.412 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** Compaction Stats [default] ** 2024-09-11T07:21:30.412 INFO:journalctl@ceph.mon.a.smithi022.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:21:30.413 INFO:journalctl@ceph.mon.a.smithi022.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:21:30.413 INFO:journalctl@ceph.mon.a.smithi022.stdout: L0 2/0 83.77 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 131.1 0.00 0.00 1 0.001 0 0 2024-09-11T07:21:30.413 INFO:journalctl@ceph.mon.a.smithi022.stdout: Sum 2/0 83.77 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 131.1 0.00 0.00 1 0.001 0 0 2024-09-11T07:21:30.413 INFO:journalctl@ceph.mon.a.smithi022.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-11T07:21:30.413 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-09-11T07:21:30.413 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** Compaction Stats [default] ** 2024-09-11T07:21:30.413 INFO:journalctl@ceph.mon.a.smithi022.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:21:30.413 INFO:journalctl@ceph.mon.a.smithi022.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:21:30.413 INFO:journalctl@ceph.mon.a.smithi022.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 131.1 0.00 0.00 1 0.001 0 0 2024-09-11T07:21:30.413 INFO:journalctl@ceph.mon.a.smithi022.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:21:30.413 INFO:journalctl@ceph.mon.a.smithi022.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T07:21:30.413 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T07:21:30.413 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T07:21:30.413 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T07:21:30.414 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T07:21:30.414 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative compaction: 0.00 GB write, 23.77 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:21:30.414 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:21:30.414 INFO:journalctl@ceph.mon.a.smithi022.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T07:21:30.414 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-09-11T07:21:30.414 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T07:21:30.414 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: starting mon.a rank 0 at public addrs v1:172.21.15.22:6789/0 at bind addrs v1:172.21.15.22:6789/0 mon_data /var/lib/ceph/mon/ceph-a fsid 12a285d8-700e-11ef-bcea-c7b262605968 2024-09-11T07:21:30.414 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: mon.a@-1(???) e1 preinit fsid 12a285d8-700e-11ef-bcea-c7b262605968 2024-09-11T07:21:30.414 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: mon.a@-1(???).mds e1 new map 2024-09-11T07:21:30.414 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: mon.a@-1(???).mds e1 print_map 2024-09-11T07:21:30.414 INFO:journalctl@ceph.mon.a.smithi022.stdout: e1 2024-09-11T07:21:30.414 INFO:journalctl@ceph.mon.a.smithi022.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T07:21:30.414 INFO:journalctl@ceph.mon.a.smithi022.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-11T07:21:30.414 INFO:journalctl@ceph.mon.a.smithi022.stdout: legacy client fscid: -1 2024-09-11T07:21:30.415 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-09-11T07:21:30.415 INFO:journalctl@ceph.mon.a.smithi022.stdout: No filesystems configured 2024-09-11T07:21:30.415 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T07:21:30.415 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:21:30.415 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:21:30.415 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:21:30.415 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-11T07:21:30.415 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T07:21:30.415 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: monmap e1: 1 mons at {a=v1:172.21.15.22:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:21:30.415 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: fsmap 2024-09-11T07:21:30.415 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T07:21:30.415 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 ceph-mon[29462]: mgrmap e1: no daemons active 2024-09-11T07:21:30.416 INFO:teuthology.orchestra.run.smithi022.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-11T07:21:30.643 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 bash[29443]: e6840d670ba28df616e91626938892c2ae2ab42e6b832943fa4315a6ba29d612 2024-09-11T07:21:30.643 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:30 smithi022 systemd[1]: Started Ceph mon.a for 12a285d8-700e-11ef-bcea-c7b262605968. 2024-09-11T07:21:31.744 INFO:teuthology.orchestra.run.smithi022.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-11T07:21:31.744 INFO:teuthology.orchestra.run.smithi022.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:21:31.744 INFO:teuthology.orchestra.run.smithi022.stdout:Creating mgr... 2024-09-11T07:21:31.745 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-11T07:21:31.745 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-11T07:21:32.071 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-12a285d8-700e-11ef-bcea-c7b262605968@mgr.y 2024-09-11T07:21:32.071 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to reset failed state of unit ceph-12a285d8-700e-11ef-bcea-c7b262605968@mgr.y.service: Unit ceph-12a285d8-700e-11ef-bcea-c7b262605968@mgr.y.service not loaded. 2024-09-11T07:21:32.079 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:31 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3485607143' entity='client.admin' 2024-09-11T07:21:32.295 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-12a285d8-700e-11ef-bcea-c7b262605968.target.wants/ceph-12a285d8-700e-11ef-bcea-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-12a285d8-700e-11ef-bcea-c7b262605968@.service. 2024-09-11T07:21:32.417 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:32 smithi022 systemd[1]: Starting Ceph mgr.y for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:21:32.917 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:32 smithi022 podman[29672]: 2024-09-11 07:21:32.432518847 +0000 UTC m=+0.016439135 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:21:34.155 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:33 smithi022 podman[29672]: 2024-09-11 07:21:33.692287305 +0000 UTC m=+1.276207582 container create 022b1767529aa35be84838926d54af37199fc6839648ee9a70ef725f8231d76e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-11T07:21:34.378 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T07:21:34.378 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-09-11T07:21:34.386 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T07:21:34.386 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-09-11T07:21:34.387 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-09-11T07:21:34.387 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T07:21:34.396 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T07:21:34.397 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-09-11T07:21:34.406 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T07:21:34.406 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-09-11T07:21:34.406 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-09-11T07:21:34.406 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-11T07:21:34.406 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr to start... 2024-09-11T07:21:34.406 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr... 2024-09-11T07:21:34.417 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:34 smithi022 podman[29672]: 2024-09-11 07:21:34.269933311 +0000 UTC m=+1.853853596 container init 022b1767529aa35be84838926d54af37199fc6839648ee9a70ef725f8231d76e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-11T07:21:34.417 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:34 smithi022 podman[29672]: 2024-09-11 07:21:34.273176304 +0000 UTC m=+1.857096590 container start 022b1767529aa35be84838926d54af37199fc6839648ee9a70ef725f8231d76e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-11T07:21:34.417 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:34 smithi022 bash[29672]: 022b1767529aa35be84838926d54af37199fc6839648ee9a70ef725f8231d76e 2024-09-11T07:21:34.417 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:34 smithi022 systemd[1]: Started Ceph mgr.y for 12a285d8-700e-11ef-bcea-c7b262605968. 2024-09-11T07:21:34.417 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:34 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:34.392+0000 7f882d65fec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:21:34.678 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:34 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:34.462+0000 7f882d65fec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:21:34.678 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:34 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:34.530+0000 7f882d65fec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:21:34.678 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:34 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:34.666+0000 7f882d65fec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:21:35.109 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:35 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2344608329' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T07:21:35.110 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:34 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:34.815+0000 7f882d65fec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:21:35.110 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:35 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:35.106+0000 7f882d65fec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:21:35.417 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:35 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:35.170+0000 7f882d65fec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:21:35.417 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:35 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:35.341+0000 7f882d65fec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:21:35.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-09-11T07:21:35.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:21:35.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "12a285d8-700e-11ef-bcea-c7b262605968", 2024-09-11T07:21:35.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T07:21:35.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T07:21:35.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T07:21:35.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T07:21:35.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:21:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T07:21:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T07:21:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2024-09-11T07:21:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:21:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T07:21:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T07:21:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:21:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-09-11T07:21:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T07:21:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:21:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T07:21:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T07:21:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:21:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T07:21:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:21:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T07:21:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T07:21:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T07:21:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T07:21:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T07:21:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T07:21:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:21:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T07:21:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T07:21:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T07:21:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T07:21:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T07:21:35.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T07:21:35.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T07:21:35.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T07:21:35.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T07:21:35.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:21:35.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T07:21:35.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:21:35.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T07:21:35.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T07:21:35.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:21:35.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T07:21:35.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T07:21:35.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T07:21:35.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T07:21:35.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T07:21:35.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T07:21:35.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T07:21:35.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:21:35.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:21:35.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:21:35.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T07:21:35.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:21:35.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T07:21:18.695840+0000", 2024-09-11T07:21:35.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:21:35.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:21:35.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T07:21:35.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:21:35.608 INFO:teuthology.orchestra.run.smithi022.stdout:mgr not available, waiting (1/15)... 2024-09-11T07:21:36.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:36 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:36.222+0000 7f882d65fec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:21:36.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:36 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:36.289+0000 7f882d65fec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:21:37.335 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:37 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:37.031+0000 7f882d65fec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:21:37.335 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:37 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:37.146+0000 7f882d65fec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:21:37.335 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:37 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:37.271+0000 7f882d65fec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:21:37.335 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:37 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:37.332+0000 7f882d65fec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:21:37.594 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:37 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:37.462+0000 7f882d65fec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:21:37.594 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:37 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:37.531+0000 7f882d65fec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:21:37.594 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:37 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:37.591+0000 7f882d65fec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:21:37.851 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:37 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:37.736+0000 7f882d65fec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:21:37.851 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:37 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:37.804+0000 7f882d65fec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:21:38.116 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:37 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:37.874+0000 7f882d65fec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:21:38.116 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:37 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:37.957+0000 7f882d65fec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:21:38.116 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:38 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:38.019+0000 7f882d65fec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:21:38.116 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:38 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:38.079+0000 7f882d65fec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:21:38.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:38 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2132617460' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T07:21:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-09-11T07:21:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:21:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "12a285d8-700e-11ef-bcea-c7b262605968", 2024-09-11T07:21:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T07:21:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T07:21:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T07:21:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T07:21:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:21:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T07:21:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T07:21:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2024-09-11T07:21:38.682 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:21:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T07:21:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T07:21:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:21:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-09-11T07:21:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T07:21:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:21:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T07:21:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T07:21:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:21:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T07:21:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:21:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T07:21:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T07:21:38.683 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T07:21:38.684 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T07:21:38.684 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T07:21:38.684 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T07:21:38.690 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:21:38.690 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T07:21:38.690 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T07:21:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T07:21:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T07:21:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T07:21:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T07:21:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T07:21:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T07:21:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T07:21:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:21:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T07:21:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:21:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T07:21:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T07:21:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:21:38.691 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T07:21:38.692 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T07:21:38.692 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T07:21:38.692 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T07:21:38.692 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T07:21:38.692 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T07:21:38.692 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T07:21:38.692 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:21:38.692 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:21:38.692 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:21:38.692 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T07:21:38.692 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:21:38.692 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T07:21:18.695840+0000", 2024-09-11T07:21:38.692 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:21:38.692 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:21:38.693 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T07:21:38.693 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:21:38.693 INFO:teuthology.orchestra.run.smithi022.stdout:mgr not available, waiting (2/15)... 2024-09-11T07:21:39.167 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:38 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:38.866+0000 7f882d65fec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:21:39.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:39 smithi022 ceph-mon[29462]: Activating manager daemon y 2024-09-11T07:21:39.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:39 smithi022 ceph-mon[29462]: mgrmap e2: y(active, starting, since 0.00309592s) 2024-09-11T07:21:39.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:39 smithi022 ceph-mon[29462]: from='mgr.14100 v1:172.21.15.22:0/459778843' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:21:39.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:39 smithi022 ceph-mon[29462]: from='mgr.14100 v1:172.21.15.22:0/459778843' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:21:39.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:39 smithi022 ceph-mon[29462]: from='mgr.14100 v1:172.21.15.22:0/459778843' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:21:39.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:39 smithi022 ceph-mon[29462]: from='mgr.14100 v1:172.21.15.22:0/459778843' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:21:39.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:39 smithi022 ceph-mon[29462]: from='mgr.14100 v1:172.21.15.22:0/459778843' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:21:39.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:39 smithi022 ceph-mon[29462]: from='mgr.14100 v1:172.21.15.22:0/459778843' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:21:39.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:39 smithi022 ceph-mon[29462]: from='mgr.14100 v1:172.21.15.22:0/459778843' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:21:39.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:39 smithi022 ceph-mon[29462]: from='mgr.14100 v1:172.21.15.22:0/459778843' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:21:39.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:39 smithi022 ceph-mon[29462]: Manager daemon y is now available 2024-09-11T07:21:39.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:39 smithi022 ceph-mon[29462]: from='mgr.14100 v1:172.21.15.22:0/459778843' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:21:39.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:39 smithi022 ceph-mon[29462]: from='mgr.14100 v1:172.21.15.22:0/459778843' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:21:39.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:39 smithi022 ceph-mon[29462]: from='mgr.14100 v1:172.21.15.22:0/459778843' entity='mgr.y' 2024-09-11T07:21:39.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:39 smithi022 ceph-mon[29462]: from='mgr.14100 v1:172.21.15.22:0/459778843' entity='mgr.y' 2024-09-11T07:21:39.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:39 smithi022 ceph-mon[29462]: from='mgr.14100 v1:172.21.15.22:0/459778843' entity='mgr.y' 2024-09-11T07:21:41.104 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:40 smithi022 ceph-mon[29462]: mgrmap e3: y(active, since 1.00694s) 2024-09-11T07:21:41.937 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-09-11T07:21:41.938 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:21:41.938 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "12a285d8-700e-11ef-bcea-c7b262605968", 2024-09-11T07:21:41.938 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T07:21:41.938 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T07:21:41.938 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T07:21:41.938 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T07:21:41.938 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:21:41.938 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T07:21:41.938 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T07:21:41.938 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2024-09-11T07:21:41.938 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:21:41.938 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T07:21:41.939 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T07:21:41.939 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:21:41.939 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-09-11T07:21:41.939 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T07:21:41.939 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:21:41.939 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T07:21:41.939 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T07:21:41.939 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:21:41.939 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T07:21:41.939 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:21:41.939 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T07:21:41.939 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T07:21:41.939 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T07:21:41.939 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T07:21:41.940 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T07:21:41.940 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T07:21:41.943 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:21:41.943 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T07:21:41.943 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T07:21:41.943 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T07:21:41.944 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T07:21:41.944 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T07:21:41.944 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T07:21:41.944 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T07:21:41.944 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T07:21:41.944 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T07:21:41.944 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:21:41.944 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T07:21:41.944 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:21:41.944 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T07:21:41.944 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T07:21:41.945 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:21:41.945 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T07:21:41.945 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T07:21:41.945 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T07:21:41.945 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T07:21:41.945 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T07:21:41.945 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T07:21:41.945 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T07:21:41.945 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:21:41.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:21:41.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:21:41.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T07:21:41.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:21:41.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T07:21:18.695840+0000", 2024-09-11T07:21:41.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:21:41.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:21:41.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T07:21:41.946 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:21:41.946 INFO:teuthology.orchestra.run.smithi022.stdout:mgr is available 2024-09-11T07:21:42.149 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:41 smithi022 ceph-mon[29462]: mgrmap e4: y(active, since 2s) 2024-09-11T07:21:42.150 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:41 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1824293473' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T07:21:43.072 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:42 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2530836269' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T07:21:43.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-09-11T07:21:43.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T07:21:43.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout fsid = 12a285d8-700e-11ef-bcea-c7b262605968 2024-09-11T07:21:43.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-11T07:21:43.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T07:21:43.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T07:21:43.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T07:21:43.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T07:21:43.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-09-11T07:21:43.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T07:21:43.077 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T07:21:43.078 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-09-11T07:21:43.078 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T07:21:43.078 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T07:21:43.078 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T07:21:43.078 INFO:teuthology.orchestra.run.smithi022.stdout:Enabling cephadm module... 2024-09-11T07:21:44.136 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:43 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3371471979' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-11T07:21:44.136 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:43 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ignoring --setuser ceph since I am not root 2024-09-11T07:21:44.136 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:43 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ignoring --setgroup ceph since I am not root 2024-09-11T07:21:44.136 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:44 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:44.009+0000 7fe34fa75ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:21:44.136 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:44 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:44.066+0000 7fe34fa75ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:21:44.137 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:44 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:44.133+0000 7fe34fa75ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:21:44.407 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:44 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:44.262+0000 7fe34fa75ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:21:44.407 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:44 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:44.404+0000 7fe34fa75ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:21:44.948 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:44 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3371471979' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-11T07:21:44.948 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:44 smithi022 ceph-mon[29462]: mgrmap e5: y(active, since 5s) 2024-09-11T07:21:44.949 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:44 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:44.677+0000 7fe34fa75ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:21:44.949 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:44 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:44.739+0000 7fe34fa75ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:21:44.949 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:44 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:44.912+0000 7fe34fa75ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:21:45.823 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:21:45.823 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-11T07:21:45.823 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T07:21:45.823 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-11T07:21:45.823 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T07:21:45.823 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:21:45.823 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for the mgr to restart... 2024-09-11T07:21:45.823 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr epoch 5... 2024-09-11T07:21:46.060 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:45 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1271454774' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T07:21:46.060 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:45 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:45.803+0000 7fe34fa75ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:21:46.060 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:45 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:45.875+0000 7fe34fa75ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:21:46.917 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:46 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:46.620+0000 7fe34fa75ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:21:46.917 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:46 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:46.736+0000 7fe34fa75ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:21:46.917 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:46 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:46.862+0000 7fe34fa75ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:21:47.186 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:46 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:46.923+0000 7fe34fa75ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:21:47.186 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:47 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:47.054+0000 7fe34fa75ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:21:47.186 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:47 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:47.123+0000 7fe34fa75ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:21:47.186 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:47 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:47.183+0000 7fe34fa75ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:21:47.459 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:47 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:47.322+0000 7fe34fa75ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:21:47.459 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:47 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:47.388+0000 7fe34fa75ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:21:47.459 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:47 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:47.455+0000 7fe34fa75ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:21:47.917 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:47 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:47.539+0000 7fe34fa75ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:21:47.917 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:47 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:47.600+0000 7fe34fa75ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:21:47.917 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:47 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:47.659+0000 7fe34fa75ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:21:48.917 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:48 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:48.436+0000 7fe34fa75ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:21:48.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:48 smithi022 ceph-mon[29462]: Active manager daemon y restarted 2024-09-11T07:21:48.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:48 smithi022 ceph-mon[29462]: Activating manager daemon y 2024-09-11T07:21:48.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:48 smithi022 ceph-mon[29462]: osdmap e2: 0 total, 0 up, 0 in 2024-09-11T07:21:48.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:48 smithi022 ceph-mon[29462]: mgrmap e6: y(active, starting, since 0.00321286s) 2024-09-11T07:21:48.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:48 smithi022 ceph-mon[29462]: from='mgr.14118 v1:172.21.15.22:0/3876348789' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:21:48.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:48 smithi022 ceph-mon[29462]: from='mgr.14118 v1:172.21.15.22:0/3876348789' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:21:48.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:48 smithi022 ceph-mon[29462]: from='mgr.14118 v1:172.21.15.22:0/3876348789' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:21:48.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:48 smithi022 ceph-mon[29462]: from='mgr.14118 v1:172.21.15.22:0/3876348789' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:21:48.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:48 smithi022 ceph-mon[29462]: from='mgr.14118 v1:172.21.15.22:0/3876348789' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:21:48.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:48 smithi022 ceph-mon[29462]: Manager daemon y is now available 2024-09-11T07:21:48.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:48 smithi022 ceph-mon[29462]: from='mgr.14118 v1:172.21.15.22:0/3876348789' entity='mgr.y' 2024-09-11T07:21:48.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:48 smithi022 ceph-mon[29462]: from='mgr.14118 v1:172.21.15.22:0/3876348789' entity='mgr.y' 2024-09-11T07:21:48.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:48 smithi022 ceph-mon[29462]: from='mgr.14118 v1:172.21.15.22:0/3876348789' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:21:48.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:48 smithi022 ceph-mon[29462]: from='mgr.14118 v1:172.21.15.22:0/3876348789' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:21:48.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:48 smithi022 ceph-mon[29462]: from='mgr.14118 v1:172.21.15.22:0/3876348789' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:21:48.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:48 smithi022 ceph-mon[29462]: from='mgr.14118 v1:172.21.15.22:0/3876348789' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:21:49.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:49 smithi022 ceph-mon[29462]: Found migration_current of "None". Setting to last migration. 2024-09-11T07:21:49.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:49 smithi022 ceph-mon[29462]: mgrmap e7: y(active, since 1.00512s) 2024-09-11T07:21:50.027 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:21:50.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-11T07:21:50.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T07:21:50.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:21:50.028 INFO:teuthology.orchestra.run.smithi022.stdout:mgr epoch 5 is available 2024-09-11T07:21:50.028 INFO:teuthology.orchestra.run.smithi022.stdout:Setting orchestrator backend to cephadm... 2024-09-11T07:21:50.632 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:50 smithi022 ceph-mon[29462]: from='client.14122 v1:172.21.15.22:0/4276244700' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T07:21:50.632 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:50 smithi022 ceph-mon[29462]: from='client.14122 v1:172.21.15.22:0/4276244700' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T07:21:51.617 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:51 smithi022 ceph-mon[29462]: mgrmap e8: y(active, since 2s) 2024-09-11T07:21:51.617 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:51 smithi022 ceph-mon[29462]: from='client.14130 v1:172.21.15.22:0/473022085' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:21:51.617 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:51 smithi022 ceph-mon[29462]: from='mgr.14118 v1:172.21.15.22:0/3876348789' entity='mgr.y' 2024-09-11T07:21:51.617 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:51 smithi022 ceph-mon[29462]: from='mgr.14118 v1:172.21.15.22:0/3876348789' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:21:51.617 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:51 smithi022 ceph-mon[29462]: from='mgr.14118 v1:172.21.15.22:0/3876348789' entity='mgr.y' 2024-09-11T07:21:51.617 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:51 smithi022 ceph-mon[29462]: from='mgr.14118 v1:172.21.15.22:0/3876348789' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:21:51.617 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:51.339+0000 7fe301e23640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:07:21:51] ENGINE Error in HTTPServer.serve 2024-09-11T07:21:51.618 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: Traceback (most recent call last): 2024-09-11T07:21:51.618 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:21:51.618 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: self._connections.run(self.expiration_interval) 2024-09-11T07:21:51.618 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:21:51.618 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: self._run(expiration_interval) 2024-09-11T07:21:51.618 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:21:51.618 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:21:51.618 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:21:51.618 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:21:51.618 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:21:51.618 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: s = self.context.wrap_socket( 2024-09-11T07:21:51.618 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:21:51.618 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: return self.sslsocket_class._create( 2024-09-11T07:21:51.619 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:21:51.619 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: self.do_handshake() 2024-09-11T07:21:51.619 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:21:51.619 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: self._sslobj.do_handshake() 2024-09-11T07:21:51.619 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:21:51.619 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:21:52.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-11T07:21:52.607 INFO:teuthology.orchestra.run.smithi022.stdout:Generating ssh key... 2024-09-11T07:21:52.832 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:52 smithi022 ceph-mon[29462]: [11/Sep/2024:07:21:51] ENGINE Bus STARTING 2024-09-11T07:21:52.832 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:52 smithi022 ceph-mon[29462]: [11/Sep/2024:07:21:51] ENGINE Serving on https://172.21.15.22:7150 2024-09-11T07:21:52.832 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:52 smithi022 ceph-mon[29462]: [11/Sep/2024:07:21:51] ENGINE Bus STARTED 2024-09-11T07:21:52.832 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:52 smithi022 ceph-mon[29462]: [11/Sep/2024:07:21:51] ENGINE Error in HTTPServer.serve 2024-09-11T07:21:52.832 INFO:journalctl@ceph.mon.a.smithi022.stdout: Traceback (most recent call last): 2024-09-11T07:21:52.832 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:21:52.832 INFO:journalctl@ceph.mon.a.smithi022.stdout: self._connections.run(self.expiration_interval) 2024-09-11T07:21:52.832 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:21:52.832 INFO:journalctl@ceph.mon.a.smithi022.stdout: self._run(expiration_interval) 2024-09-11T07:21:52.833 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:21:52.833 INFO:journalctl@ceph.mon.a.smithi022.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:21:52.833 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:21:52.833 INFO:journalctl@ceph.mon.a.smithi022.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:21:52.833 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:21:52.833 INFO:journalctl@ceph.mon.a.smithi022.stdout: s = self.context.wrap_socket( 2024-09-11T07:21:52.833 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:21:52.833 INFO:journalctl@ceph.mon.a.smithi022.stdout: return self.sslsocket_class._create( 2024-09-11T07:21:52.833 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:21:52.833 INFO:journalctl@ceph.mon.a.smithi022.stdout: self.do_handshake() 2024-09-11T07:21:52.833 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:21:52.833 INFO:journalctl@ceph.mon.a.smithi022.stdout: self._sslobj.do_handshake() 2024-09-11T07:21:52.833 INFO:journalctl@ceph.mon.a.smithi022.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:21:52.833 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:52 smithi022 ceph-mon[29462]: from='client.14132 v1:172.21.15.22:0/1620613746' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:21:53.741 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:53 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: Generating public/private rsa key pair. 2024-09-11T07:21:53.741 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:53 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: Your identification has been saved in /tmp/tmpkcgf_dkr/key 2024-09-11T07:21:53.741 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:53 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: Your public key has been saved in /tmp/tmpkcgf_dkr/key.pub 2024-09-11T07:21:53.741 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:53 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: The key fingerprint is: 2024-09-11T07:21:53.741 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:53 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: SHA256:Rls3ROAXMuv9qfGuGML5ZUIDlvlp1aIDBCzG+oEHcgQ ceph-12a285d8-700e-11ef-bcea-c7b262605968 2024-09-11T07:21:53.741 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:53 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: The key's randomart image is: 2024-09-11T07:21:53.741 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:53 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: +---[RSA 3072]----+ 2024-09-11T07:21:53.741 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:53 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: |Eo.. .... +o+ | 2024-09-11T07:21:53.741 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:53 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: |. o + .. + = o | 2024-09-11T07:21:53.741 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:53 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: | o = . B + * . | 2024-09-11T07:21:53.742 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:53 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: | o o o B B o | 2024-09-11T07:21:53.742 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:53 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: | o . S X . | 2024-09-11T07:21:53.742 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:53 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: | . o + o . . | 2024-09-11T07:21:53.742 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:53 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: | + o + o | 2024-09-11T07:21:53.742 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:53 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: | o * + | 2024-09-11T07:21:53.742 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:53 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: | o ooo | 2024-09-11T07:21:53.742 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:21:53 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: +----[SHA256]-----+ 2024-09-11T07:21:54.578 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:54 smithi022 ceph-mon[29462]: from='client.14134 v1:172.21.15.22:0/342875254' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:21:54.578 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:54 smithi022 ceph-mon[29462]: Generating ssh key... 2024-09-11T07:21:54.578 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:54 smithi022 ceph-mon[29462]: from='mgr.14118 v1:172.21.15.22:0/3876348789' entity='mgr.y' 2024-09-11T07:21:54.579 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:54 smithi022 ceph-mon[29462]: from='mgr.14118 v1:172.21.15.22:0/3876348789' entity='mgr.y' 2024-09-11T07:21:55.119 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp+dqnFByBg3VJGT1fz/plizfxFVApJqRh2KQU/DsnSVb2rnn38Vsk4kD3TzNtINertIbnxaLu1X5K+yx+aARcvfagb6Jgp5lep+4r0EpohPPshb30z516oY1BUerwIli6Jr/E+JJCDj9RwErerOBT8YD7/VIBJF7EZW0n9bNHpgVa33CCyDpVNp1oT4lZpn3CU19G4cRJmCSVuP2w9f38rLSwt2aD9VnJl2IkEoAzkU+DR+gTm4Vw9hJ3uaWPsHixP9LAEp2R4f7rlFpUqIBgvzFlBDGGiV/vQRLOiKs3ylDoibygmDfOmcCqZAz4fldORS8xYy6HCx3qG3F9MrVfbxDoRsOYniU0RmSZYluyTPeeTCDCKCSUBYpgyqkCAAUuGL0bGvNmtNKztj1uc7cFD1fs0ugu4/xQseMXBpGefrvX9KDfJ+EnYW6nHcrt5Dxgwl2NcLS9lzlQT/2IzyHMAOqw9ToMkk6GJuxUuvwHq2hVnGCwXw7ZaEzV8sIHioU= ceph-12a285d8-700e-11ef-bcea-c7b262605968 2024-09-11T07:21:55.120 INFO:teuthology.orchestra.run.smithi022.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-11T07:21:55.120 INFO:teuthology.orchestra.run.smithi022.stdout:Adding key to root@localhost authorized_keys... 2024-09-11T07:21:55.122 INFO:teuthology.orchestra.run.smithi022.stdout:Adding host smithi022... 2024-09-11T07:21:55.898 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:55 smithi022 ceph-mon[29462]: from='client.14136 v1:172.21.15.22:0/4127015035' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:21:56.743 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:56 smithi022 ceph-mon[29462]: from='client.14138 v1:172.21.15.22:0/305088124' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi022", "addr": "172.21.15.22", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:21:57.633 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:21:57 smithi022 ceph-mon[29462]: Deploying cephadm binary to smithi022 2024-09-11T07:21:59.710 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Added host 'smithi022' with addr '172.21.15.22' 2024-09-11T07:21:59.710 INFO:teuthology.orchestra.run.smithi022.stdout:Deploying unmanaged mon service... 2024-09-11T07:22:00.325 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:00 smithi022 ceph-mon[29462]: from='mgr.14118 v1:172.21.15.22:0/3876348789' entity='mgr.y' 2024-09-11T07:22:00.325 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:00 smithi022 ceph-mon[29462]: Added host smithi022 2024-09-11T07:22:00.325 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:00 smithi022 ceph-mon[29462]: from='mgr.14118 v1:172.21.15.22:0/3876348789' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:22:01.043 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-11T07:22:01.043 INFO:teuthology.orchestra.run.smithi022.stdout:Deploying unmanaged mgr service... 2024-09-11T07:22:01.654 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:01 smithi022 ceph-mon[29462]: from='client.14140 v1:172.21.15.22:0/4008752437' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:22:01.654 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:01 smithi022 ceph-mon[29462]: Saving service mon spec with placement count:5 2024-09-11T07:22:01.654 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:01 smithi022 ceph-mon[29462]: from='mgr.14118 v1:172.21.15.22:0/3876348789' entity='mgr.y' 2024-09-11T07:22:02.285 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-11T07:22:02.954 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:02 smithi022 ceph-mon[29462]: from='client.14142 v1:172.21.15.22:0/1723170823' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:22:02.954 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:02 smithi022 ceph-mon[29462]: Saving service mgr spec with placement count:2 2024-09-11T07:22:02.954 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:02 smithi022 ceph-mon[29462]: from='mgr.14118 v1:172.21.15.22:0/3876348789' entity='mgr.y' 2024-09-11T07:22:04.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:04 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4242889141' entity='client.admin' 2024-09-11T07:22:04.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:04 smithi022 ceph-mon[29462]: from='mgr.14118 v1:172.21.15.22:0/3876348789' entity='mgr.y' 2024-09-11T07:22:05.294 INFO:teuthology.orchestra.run.smithi022.stdout:Enabling the dashboard module... 2024-09-11T07:22:05.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:05 smithi022 ceph-mon[29462]: from='mgr.14118 v1:172.21.15.22:0/3876348789' entity='mgr.y' 2024-09-11T07:22:05.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:05 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3801661236' entity='client.admin' 2024-09-11T07:22:06.910 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:06 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1599962760' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-11T07:22:06.910 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:06 smithi022 ceph-mon[29462]: from='mgr.14118 v1:172.21.15.22:0/3876348789' entity='mgr.y' 2024-09-11T07:22:06.910 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:06 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ignoring --setuser ceph since I am not root 2024-09-11T07:22:06.911 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:06 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ignoring --setgroup ceph since I am not root 2024-09-11T07:22:06.911 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:06 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:06.775+0000 7f29cd8f5ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:22:06.911 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:06 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:06.835+0000 7f29cd8f5ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:22:06.911 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:06 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:06.908+0000 7f29cd8f5ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:22:07.167 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:07 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:07.042+0000 7f29cd8f5ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:22:07.462 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:07 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:07.184+0000 7f29cd8f5ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:22:07.462 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:07 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:07.459+0000 7f29cd8f5ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:22:07.877 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:07 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1599962760' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-11T07:22:07.877 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:07 smithi022 ceph-mon[29462]: mgrmap e9: y(active, since 18s) 2024-09-11T07:22:07.877 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:07 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:07.524+0000 7f29cd8f5ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:22:07.877 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:07 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:07.699+0000 7f29cd8f5ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:22:08.486 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:22:08.487 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-11T07:22:08.487 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T07:22:08.487 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-11T07:22:08.487 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T07:22:08.487 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:22:08.487 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for the mgr to restart... 2024-09-11T07:22:08.487 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr epoch 9... 2024-09-11T07:22:08.811 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:08 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1231737726' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T07:22:08.811 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:08 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:08.598+0000 7f29cd8f5ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:22:08.811 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:08 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:08.667+0000 7f29cd8f5ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:22:09.721 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:09.416+0000 7f29cd8f5ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:22:09.721 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:09.531+0000 7f29cd8f5ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:22:09.721 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:09.657+0000 7f29cd8f5ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:22:09.721 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:09.719+0000 7f29cd8f5ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:22:09.981 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:09.849+0000 7f29cd8f5ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:22:09.981 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:09.919+0000 7f29cd8f5ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:22:09.981 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:09.979+0000 7f29cd8f5ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:22:10.254 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:10.119+0000 7f29cd8f5ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:22:10.255 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:10.185+0000 7f29cd8f5ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:22:10.255 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:10.252+0000 7f29cd8f5ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:22:10.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:10.337+0000 7f29cd8f5ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:22:10.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:10.396+0000 7f29cd8f5ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:22:10.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:10.455+0000 7f29cd8f5ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:22:11.503 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:11 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:11.236+0000 7f29cd8f5ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:22:11.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:11 smithi022 ceph-mon[29462]: Active manager daemon y restarted 2024-09-11T07:22:11.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:11 smithi022 ceph-mon[29462]: Activating manager daemon y 2024-09-11T07:22:11.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:11 smithi022 ceph-mon[29462]: osdmap e3: 0 total, 0 up, 0 in 2024-09-11T07:22:11.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:11 smithi022 ceph-mon[29462]: mgrmap e10: y(active, starting, since 0.0031317s) 2024-09-11T07:22:11.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:11 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:22:11.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:11 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:22:11.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:11 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:22:11.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:11 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:22:11.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:11 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:22:11.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:11 smithi022 ceph-mon[29462]: Manager daemon y is now available 2024-09-11T07:22:11.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:11 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:22:11.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:11 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:22:11.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:11 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:22:13.177 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:13 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:13.090+0000 7f297e452640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:07:22:13] ENGINE Error in HTTPServer.serve 2024-09-11T07:22:13.177 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:13 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: Traceback (most recent call last): 2024-09-11T07:22:13.177 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:13 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:22:13.177 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:13 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: self._connections.run(self.expiration_interval) 2024-09-11T07:22:13.177 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:13 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:22:13.177 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:13 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: self._run(expiration_interval) 2024-09-11T07:22:13.177 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:13 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:22:13.177 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:13 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:22:13.178 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:13 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:22:13.178 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:13 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:22:13.178 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:13 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:22:13.178 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:13 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: s = self.context.wrap_socket( 2024-09-11T07:22:13.178 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:13 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:22:13.178 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:13 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: return self.sslsocket_class._create( 2024-09-11T07:22:13.178 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:13 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:22:13.178 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:13 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: self.do_handshake() 2024-09-11T07:22:13.178 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:13 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:22:13.178 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:13 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: self._sslobj.do_handshake() 2024-09-11T07:22:13.178 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:13 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:22:13.178 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:13 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:13.179 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:22:13.179 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-11T07:22:13.179 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T07:22:13.179 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:22:13.179 INFO:teuthology.orchestra.run.smithi022.stdout:mgr epoch 9 is available 2024-09-11T07:22:13.180 INFO:teuthology.orchestra.run.smithi022.stdout:Generating a dashboard self-signed certificate... 2024-09-11T07:22:13.429 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:13 smithi022 ceph-mon[29462]: from='client.14154 v1:172.21.15.22:0/2103366994' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T07:22:13.429 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:13 smithi022 ceph-mon[29462]: mgrmap e11: y(active, since 1.00542s) 2024-09-11T07:22:13.429 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:13 smithi022 ceph-mon[29462]: from='client.14154 v1:172.21.15.22:0/2103366994' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T07:22:13.429 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:13 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:13.429 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:13 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:14.469 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-11T07:22:14.469 INFO:teuthology.orchestra.run.smithi022.stdout:Creating initial admin user... 2024-09-11T07:22:14.593 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:14 smithi022 ceph-mon[29462]: [11/Sep/2024:07:22:12] ENGINE Bus STARTING 2024-09-11T07:22:14.593 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:14 smithi022 ceph-mon[29462]: [11/Sep/2024:07:22:13] ENGINE Serving on https://172.21.15.22:7150 2024-09-11T07:22:14.593 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:14 smithi022 ceph-mon[29462]: [11/Sep/2024:07:22:13] ENGINE Bus STARTED 2024-09-11T07:22:14.593 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:14 smithi022 ceph-mon[29462]: [11/Sep/2024:07:22:13] ENGINE Error in HTTPServer.serve 2024-09-11T07:22:14.593 INFO:journalctl@ceph.mon.a.smithi022.stdout: Traceback (most recent call last): 2024-09-11T07:22:14.593 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:22:14.593 INFO:journalctl@ceph.mon.a.smithi022.stdout: self._connections.run(self.expiration_interval) 2024-09-11T07:22:14.593 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:22:14.593 INFO:journalctl@ceph.mon.a.smithi022.stdout: self._run(expiration_interval) 2024-09-11T07:22:14.594 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:22:14.594 INFO:journalctl@ceph.mon.a.smithi022.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:22:14.594 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:22:14.594 INFO:journalctl@ceph.mon.a.smithi022.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:22:14.594 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:22:14.594 INFO:journalctl@ceph.mon.a.smithi022.stdout: s = self.context.wrap_socket( 2024-09-11T07:22:14.594 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:22:14.594 INFO:journalctl@ceph.mon.a.smithi022.stdout: return self.sslsocket_class._create( 2024-09-11T07:22:14.594 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:22:14.594 INFO:journalctl@ceph.mon.a.smithi022.stdout: self.do_handshake() 2024-09-11T07:22:14.594 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:22:14.594 INFO:journalctl@ceph.mon.a.smithi022.stdout: self._sslobj.do_handshake() 2024-09-11T07:22:14.594 INFO:journalctl@ceph.mon.a.smithi022.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:22:14.595 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:14 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:14.595 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:14 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:14.595 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:14 smithi022 ceph-mon[29462]: mgrmap e12: y(active, since 2s) 2024-09-11T07:22:14.595 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:14 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:15.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:15 smithi022 ceph-mon[29462]: from='client.14162 v1:172.21.15.22:0/996658380' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:22:16.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:16 smithi022 ceph-mon[29462]: from='client.14164 v1:172.21.15.22:0/987544120' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:22:16.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:16 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:16.632 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$3539Det4ONb890rB36pmnev8onJbW8rwNlvvJ1Kex.cZ5PIGngSm.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726039335, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-11T07:22:16.632 INFO:teuthology.orchestra.run.smithi022.stdout:Fetching dashboard port number... 2024-09-11T07:22:17.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:17 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4258566801' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-11T07:22:17.805 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 8443 2024-09-11T07:22:17.816 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T07:22:17.816 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-09-11T07:22:17.824 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T07:22:17.825 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-09-11T07:22:17.825 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-09-11T07:22:17.825 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-11T07:22:17.826 INFO:teuthology.orchestra.run.smithi022.stdout:Ceph Dashboard is now available at: 2024-09-11T07:22:17.827 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:22:17.827 INFO:teuthology.orchestra.run.smithi022.stdout: URL: https://smithi022.front.sepia.ceph.com:8443/ 2024-09-11T07:22:17.827 INFO:teuthology.orchestra.run.smithi022.stdout: User: admin 2024-09-11T07:22:17.827 INFO:teuthology.orchestra.run.smithi022.stdout: Password: 2m7xm9av4h 2024-09-11T07:22:17.827 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:22:17.828 INFO:teuthology.orchestra.run.smithi022.stdout:Saving cluster configuration to /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/config directory 2024-09-11T07:22:17.828 INFO:teuthology.orchestra.run.smithi022.stdout:Enabling autotune for osd_memory_target 2024-09-11T07:22:18.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:18 smithi022 ceph-mon[29462]: mgrmap e13: y(active, since 6s) 2024-09-11T07:22:20.259 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-11T07:22:20.260 INFO:teuthology.orchestra.run.smithi022.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-11T07:22:20.260 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:22:20.260 INFO:teuthology.orchestra.run.smithi022.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:22:20.260 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:22:20.260 INFO:teuthology.orchestra.run.smithi022.stdout:Or, if you are only running a single cluster on this host: 2024-09-11T07:22:20.260 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:22:20.260 INFO:teuthology.orchestra.run.smithi022.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-11T07:22:20.260 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:22:20.260 INFO:teuthology.orchestra.run.smithi022.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-11T07:22:20.260 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:22:20.260 INFO:teuthology.orchestra.run.smithi022.stdout: ceph telemetry on 2024-09-11T07:22:20.260 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:22:20.261 INFO:teuthology.orchestra.run.smithi022.stdout:For more information see: 2024-09-11T07:22:20.261 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:22:20.261 INFO:teuthology.orchestra.run.smithi022.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-09-11T07:22:20.261 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:22:20.261 INFO:teuthology.orchestra.run.smithi022.stdout:Bootstrap complete. 2024-09-11T07:22:20.298 INFO:tasks.cephadm:Fetching config... 2024-09-11T07:22:20.299 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-11T07:22:20.299 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-11T07:22:20.316 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-11T07:22:20.317 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-11T07:22:20.317 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-11T07:22:20.373 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-11T07:22:20.373 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-11T07:22:20.373 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-11T07:22:20.438 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-11T07:22:20.439 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-11T07:22:20.439 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-11T07:22:20.495 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-11T07:22:20.496 DEBUG:teuthology.orchestra.run.smithi022:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp+dqnFByBg3VJGT1fz/plizfxFVApJqRh2KQU/DsnSVb2rnn38Vsk4kD3TzNtINertIbnxaLu1X5K+yx+aARcvfagb6Jgp5lep+4r0EpohPPshb30z516oY1BUerwIli6Jr/E+JJCDj9RwErerOBT8YD7/VIBJF7EZW0n9bNHpgVa33CCyDpVNp1oT4lZpn3CU19G4cRJmCSVuP2w9f38rLSwt2aD9VnJl2IkEoAzkU+DR+gTm4Vw9hJ3uaWPsHixP9LAEp2R4f7rlFpUqIBgvzFlBDGGiV/vQRLOiKs3ylDoibygmDfOmcCqZAz4fldORS8xYy6HCx3qG3F9MrVfbxDoRsOYniU0RmSZYluyTPeeTCDCKCSUBYpgyqkCAAUuGL0bGvNmtNKztj1uc7cFD1fs0ugu4/xQseMXBpGefrvX9KDfJ+EnYW6nHcrt5Dxgwl2NcLS9lzlQT/2IzyHMAOqw9ToMkk6GJuxUuvwHq2hVnGCwXw7ZaEzV8sIHioU= ceph-12a285d8-700e-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T07:22:20.572 INFO:teuthology.orchestra.run.smithi022.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp+dqnFByBg3VJGT1fz/plizfxFVApJqRh2KQU/DsnSVb2rnn38Vsk4kD3TzNtINertIbnxaLu1X5K+yx+aARcvfagb6Jgp5lep+4r0EpohPPshb30z516oY1BUerwIli6Jr/E+JJCDj9RwErerOBT8YD7/VIBJF7EZW0n9bNHpgVa33CCyDpVNp1oT4lZpn3CU19G4cRJmCSVuP2w9f38rLSwt2aD9VnJl2IkEoAzkU+DR+gTm4Vw9hJ3uaWPsHixP9LAEp2R4f7rlFpUqIBgvzFlBDGGiV/vQRLOiKs3ylDoibygmDfOmcCqZAz4fldORS8xYy6HCx3qG3F9MrVfbxDoRsOYniU0RmSZYluyTPeeTCDCKCSUBYpgyqkCAAUuGL0bGvNmtNKztj1uc7cFD1fs0ugu4/xQseMXBpGefrvX9KDfJ+EnYW6nHcrt5Dxgwl2NcLS9lzlQT/2IzyHMAOqw9ToMkk6GJuxUuvwHq2hVnGCwXw7ZaEzV8sIHioU= ceph-12a285d8-700e-11ef-bcea-c7b262605968 2024-09-11T07:22:20.584 DEBUG:teuthology.orchestra.run.smithi104:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp+dqnFByBg3VJGT1fz/plizfxFVApJqRh2KQU/DsnSVb2rnn38Vsk4kD3TzNtINertIbnxaLu1X5K+yx+aARcvfagb6Jgp5lep+4r0EpohPPshb30z516oY1BUerwIli6Jr/E+JJCDj9RwErerOBT8YD7/VIBJF7EZW0n9bNHpgVa33CCyDpVNp1oT4lZpn3CU19G4cRJmCSVuP2w9f38rLSwt2aD9VnJl2IkEoAzkU+DR+gTm4Vw9hJ3uaWPsHixP9LAEp2R4f7rlFpUqIBgvzFlBDGGiV/vQRLOiKs3ylDoibygmDfOmcCqZAz4fldORS8xYy6HCx3qG3F9MrVfbxDoRsOYniU0RmSZYluyTPeeTCDCKCSUBYpgyqkCAAUuGL0bGvNmtNKztj1uc7cFD1fs0ugu4/xQseMXBpGefrvX9KDfJ+EnYW6nHcrt5Dxgwl2NcLS9lzlQT/2IzyHMAOqw9ToMkk6GJuxUuvwHq2hVnGCwXw7ZaEzV8sIHioU= ceph-12a285d8-700e-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T07:22:20.713 INFO:teuthology.orchestra.run.smithi104.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp+dqnFByBg3VJGT1fz/plizfxFVApJqRh2KQU/DsnSVb2rnn38Vsk4kD3TzNtINertIbnxaLu1X5K+yx+aARcvfagb6Jgp5lep+4r0EpohPPshb30z516oY1BUerwIli6Jr/E+JJCDj9RwErerOBT8YD7/VIBJF7EZW0n9bNHpgVa33CCyDpVNp1oT4lZpn3CU19G4cRJmCSVuP2w9f38rLSwt2aD9VnJl2IkEoAzkU+DR+gTm4Vw9hJ3uaWPsHixP9LAEp2R4f7rlFpUqIBgvzFlBDGGiV/vQRLOiKs3ylDoibygmDfOmcCqZAz4fldORS8xYy6HCx3qG3F9MrVfbxDoRsOYniU0RmSZYluyTPeeTCDCKCSUBYpgyqkCAAUuGL0bGvNmtNKztj1uc7cFD1fs0ugu4/xQseMXBpGefrvX9KDfJ+EnYW6nHcrt5Dxgwl2NcLS9lzlQT/2IzyHMAOqw9ToMkk6GJuxUuvwHq2hVnGCwXw7ZaEzV8sIHioU= ceph-12a285d8-700e-11ef-bcea-c7b262605968 2024-09-11T07:22:20.727 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-11T07:22:20.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:20 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/854319797' entity='client.admin' 2024-09-11T07:22:20.980 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:22:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:23 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:23 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:23 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi022", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:22:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:23 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:23 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:22:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:23 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:23.969 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-11T07:22:23.970 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-11T07:22:24.219 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:22:24.629 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:24 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/552371049' entity='client.admin' 2024-09-11T07:22:24.629 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:24 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:22:24.629 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:24 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:26.642 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi104 2024-09-11T07:22:26.642 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-11T07:22:26.643 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/etc/ceph/ceph.conf 2024-09-11T07:22:26.663 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-11T07:22:26.663 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:22:26.721 INFO:tasks.cephadm:Adding host smithi104 to orchestrator... 2024-09-11T07:22:26.721 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph orch host add smithi104 2024-09-11T07:22:26.976 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:22:27.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:26 smithi022 ceph-mon[29462]: from='client.14174 v1:172.21.15.22:0/2875975194' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:22:27.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:26 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:27.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:26 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:22:27.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:26 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:22:27.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:26 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:22:27.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:26 smithi022 ceph-mon[29462]: Updating smithi022:/etc/ceph/ceph.conf 2024-09-11T07:22:27.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:26 smithi022 ceph-mon[29462]: Updating smithi022:/var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:22:27.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:26 smithi022 ceph-mon[29462]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:22:27.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:26 smithi022 ceph-mon[29462]: Updating smithi022:/var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:22:27.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:26 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:27.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:26 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:27.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:26 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:29.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:28 smithi022 ceph-mon[29462]: from='client.14176 v1:172.21.15.22:0/321624772' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi104", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:22:31.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:30 smithi022 ceph-mon[29462]: Deploying cephadm binary to smithi104 2024-09-11T07:22:31.920 INFO:teuthology.orchestra.run.smithi022.stdout:Added host 'smithi104' with addr '172.21.15.104' 2024-09-11T07:22:32.514 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:32 smithi022 ceph-mon[29462]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:22:32.514 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:32 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:32.514 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:32 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:22:32.514 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:32 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:32.542 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T07:22:32.791 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:22:33.853 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:33 smithi022 ceph-mon[29462]: Added host smithi104 2024-09-11T07:22:33.853 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:33 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:33.853 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:33 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:34.498 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:22:34.499 INFO:teuthology.orchestra.run.smithi022.stdout:[{"addr": "172.21.15.22", "hostname": "smithi022", "labels": [], "status": ""}, {"addr": "172.21.15.104", "hostname": "smithi104", "labels": [], "status": ""}] 2024-09-11T07:22:34.809 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:34 smithi022 ceph-mon[29462]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:22:35.110 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-11T07:22:35.111 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-11T07:22:35.364 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:22:35.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:35 smithi022 ceph-mon[29462]: from='client.14178 v1:172.21.15.22:0/2430832932' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:22:36.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:36 smithi022 ceph-mon[29462]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:22:36.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:36 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:36.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:36 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:36.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:36 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:36.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:36 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:36.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:36 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi104", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:22:36.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:36 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:22:36.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:36 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:22:37.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:37 smithi022 ceph-mon[29462]: Updating smithi104:/etc/ceph/ceph.conf 2024-09-11T07:22:37.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:37 smithi022 ceph-mon[29462]: Updating smithi104:/var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:22:37.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:37 smithi022 ceph-mon[29462]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:22:37.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:37 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:37.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:37 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:37.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:37 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:37.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:37 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3430495946' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-11T07:22:38.135 INFO:teuthology.orchestra.run.smithi022.stderr:adjusted tunables profile to default 2024-09-11T07:22:38.705 INFO:tasks.cephadm:Adding mon.a on smithi022 2024-09-11T07:22:38.705 INFO:tasks.cephadm:Adding mon.c on smithi022 2024-09-11T07:22:38.705 INFO:tasks.cephadm:Adding mon.b on smithi104 2024-09-11T07:22:38.705 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi022:[v1:172.21.15.22:6789]=a;smithi022:[v1:172.21.15.22:6790]=c;smithi104:[v1:172.21.15.104:6789]=b' 2024-09-11T07:22:38.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:38 smithi022 ceph-mon[29462]: Updating smithi104:/var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:22:38.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:38 smithi022 ceph-mon[29462]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:22:38.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:38 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3430495946' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T07:22:38.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:38 smithi022 ceph-mon[29462]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:22:38.948 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:22:40.726 INFO:teuthology.orchestra.run.smithi104.stdout:Scheduled mon update... 2024-09-11T07:22:40.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:40 smithi022 ceph-mon[29462]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:22:41.308 DEBUG:teuthology.orchestra.run.smithi022:mon.c> sudo journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@mon.c.service 2024-09-11T07:22:41.311 DEBUG:teuthology.orchestra.run.smithi104:mon.b> sudo journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@mon.b.service 2024-09-11T07:22:41.313 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T07:22:41.313 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T07:22:42.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:41 smithi022 ceph-mon[29462]: from='client.14182 v1:172.21.15.104:0/3317283533' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi022:[v1:172.21.15.22:6789]=a;smithi022:[v1:172.21.15.22:6790]=c;smithi104:[v1:172.21.15.104:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:22:42.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:41 smithi022 ceph-mon[29462]: Saving service mon spec with placement smithi022:[v1:172.21.15.22:6789]=a;smithi022:[v1:172.21.15.22:6790]=c;smithi104:[v1:172.21.15.104:6789]=b;count:3 2024-09-11T07:22:42.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:41 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:42.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:41 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:22:42.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:41 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:22:42.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:41 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:22:42.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:41 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:42.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:41 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:22:42.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:41 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:22:42.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:41 smithi022 ceph-mon[29462]: Deploying daemon mon.b on smithi104 2024-09-11T07:22:42.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:41 smithi022 ceph-mon[29462]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:22:43.453 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:22:44.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:44 smithi104 systemd[1]: Starting Ceph mon.b for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:22:44.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:44 smithi022 ceph-mon[29462]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:22:45.257 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 podman[33455]: 2024-09-11 07:22:45.059352132 +0000 UTC m=+0.096361013 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:22:45.257 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 podman[33455]: 2024-09-11 07:22:45.255354185 +0000 UTC m=+0.292363051 container create a00a636a58c32139f1377c348694d6be406705ca60f2450007b611ee1ad485de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-11T07:22:45.362 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:22:45.362 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":1,"fsid":"12a285d8-700e-11ef-bcea-c7b262605968","modified":"2024-09-11T07:21:13.000870Z","created":"2024-09-11T07:21:13.000870Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-11T07:22:45.363 INFO:teuthology.orchestra.run.smithi104.stderr:dumped monmap epoch 1 2024-09-11T07:22:45.511 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 podman[33455]: 2024-09-11 07:22:45.466512684 +0000 UTC m=+0.503521556 container init a00a636a58c32139f1377c348694d6be406705ca60f2450007b611ee1ad485de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-b, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-11T07:22:45.511 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 podman[33455]: 2024-09-11 07:22:45.470187434 +0000 UTC m=+0.507196300 container start a00a636a58c32139f1377c348694d6be406705ca60f2450007b611ee1ad485de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-b, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-11T07:22:45.511 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T07:22:45.511 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable), process ceph-mon, pid 2 2024-09-11T07:22:45.511 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: pidfile_write: ignore empty --pid-file 2024-09-11T07:22:45.511 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: load: jerasure load: lrc 2024-09-11T07:22:45.511 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: RocksDB version: 6.15.5 2024-09-11T07:22:45.512 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-11T07:22:45.512 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Compile date Sep 10 2024 2024-09-11T07:22:45.512 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: DB SUMMARY 2024-09-11T07:22:45.512 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: DB Session ID: 0RS9YIYH0ZVK0X2C0GYJ 2024-09-11T07:22:45.512 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: CURRENT file: CURRENT 2024-09-11T07:22:45.512 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T07:22:45.512 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-11T07:22:45.512 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-11T07:22:45.512 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 476 ; 2024-09-11T07:22:45.512 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.error_if_exists: 0 2024-09-11T07:22:45.512 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.create_if_missing: 0 2024-09-11T07:22:45.512 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.paranoid_checks: 1 2024-09-11T07:22:45.512 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T07:22:45.513 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.env: 0x55b699760440 2024-09-11T07:22:45.513 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.fs: Posix File System 2024-09-11T07:22:45.513 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.info_log: 0x55b69af48e20 2024-09-11T07:22:45.513 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T07:22:45.513 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.statistics: (nil) 2024-09-11T07:22:45.515 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.use_fsync: 0 2024-09-11T07:22:45.515 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.max_log_file_size: 0 2024-09-11T07:22:45.515 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T07:22:45.515 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T07:22:45.515 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T07:22:45.515 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T07:22:45.515 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.allow_fallocate: 1 2024-09-11T07:22:45.515 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T07:22:45.515 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T07:22:45.516 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.use_direct_reads: 0 2024-09-11T07:22:45.516 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T07:22:45.516 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T07:22:45.516 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.db_log_dir: 2024-09-11T07:22:45.516 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2024-09-11T07:22:45.516 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T07:22:45.516 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T07:22:45.516 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T07:22:45.516 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T07:22:45.516 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T07:22:45.517 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T07:22:45.517 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T07:22:45.517 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T07:22:45.517 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.write_buffer_manager: 0x55b69b0067e0 2024-09-11T07:22:45.517 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T07:22:45.517 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-11T07:22:45.517 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T07:22:45.517 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T07:22:45.517 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T07:22:45.517 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T07:22:45.517 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T07:22:45.518 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T07:22:45.518 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T07:22:45.518 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.unordered_write: 0 2024-09-11T07:22:45.518 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T07:22:45.518 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T07:22:45.518 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T07:22:45.518 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T07:22:45.518 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.row_cache: None 2024-09-11T07:22:45.518 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.wal_filter: None 2024-09-11T07:22:45.518 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T07:22:45.518 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T07:22:45.518 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.preserve_deletes: 0 2024-09-11T07:22:45.519 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.two_write_queues: 0 2024-09-11T07:22:45.519 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T07:22:45.519 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.atomic_flush: 0 2024-09-11T07:22:45.519 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T07:22:45.519 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T07:22:45.519 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T07:22:45.519 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.log_readahead_size: 0 2024-09-11T07:22:45.519 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T07:22:45.519 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T07:22:45.519 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T07:22:45.519 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T07:22:45.519 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T07:22:45.519 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T07:22:45.520 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.max_background_jobs: 2 2024-09-11T07:22:45.520 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.max_background_compactions: -1 2024-09-11T07:22:45.520 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.max_subcompactions: 1 2024-09-11T07:22:45.520 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T07:22:45.520 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T07:22:45.520 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T07:22:45.520 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T07:22:45.520 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T07:22:45.520 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T07:22:45.521 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T07:22:45.521 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T07:22:45.521 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.max_open_files: -1 2024-09-11T07:22:45.521 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T07:22:45.521 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T07:22:45.521 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T07:22:45.521 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T07:22:45.521 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.max_background_flushes: -1 2024-09-11T07:22:45.521 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Compression algorithms supported: 2024-09-11T07:22:45.521 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: kZSTD supported: 0 2024-09-11T07:22:45.521 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: kXpressCompression supported: 0 2024-09-11T07:22:45.521 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: kBZip2Compression supported: 0 2024-09-11T07:22:45.521 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T07:22:45.521 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: kLZ4Compression supported: 1 2024-09-11T07:22:45.522 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: kZlibCompression supported: 1 2024-09-11T07:22:45.522 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T07:22:45.522 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: kSnappyCompression supported: 1 2024-09-11T07:22:45.522 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T07:22:45.522 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 2024-09-11T07:22:45.522 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-11T07:22:45.522 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T07:22:45.522 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.merge_operator: 2024-09-11T07:22:45.522 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.compaction_filter: None 2024-09-11T07:22:45.522 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T07:22:45.522 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T07:22:45.522 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T07:22:45.522 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T07:22:45.523 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b69aeefa38) 2024-09-11T07:22:45.523 INFO:journalctl@ceph.mon.b.smithi104.stdout: cache_index_and_filter_blocks: 1 2024-09-11T07:22:45.523 INFO:journalctl@ceph.mon.b.smithi104.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T07:22:45.523 INFO:journalctl@ceph.mon.b.smithi104.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T07:22:45.523 INFO:journalctl@ceph.mon.b.smithi104.stdout: pin_top_level_index_and_filter: 1 2024-09-11T07:22:45.523 INFO:journalctl@ceph.mon.b.smithi104.stdout: index_type: 0 2024-09-11T07:22:45.523 INFO:journalctl@ceph.mon.b.smithi104.stdout: data_block_index_type: 0 2024-09-11T07:22:45.523 INFO:journalctl@ceph.mon.b.smithi104.stdout: index_shortening: 1 2024-09-11T07:22:45.523 INFO:journalctl@ceph.mon.b.smithi104.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T07:22:45.523 INFO:journalctl@ceph.mon.b.smithi104.stdout: hash_index_allow_collision: 1 2024-09-11T07:22:45.523 INFO:journalctl@ceph.mon.b.smithi104.stdout: checksum: 1 2024-09-11T07:22:45.523 INFO:journalctl@ceph.mon.b.smithi104.stdout: no_block_cache: 0 2024-09-11T07:22:45.523 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_cache: 0x55b69af251f0 2024-09-11T07:22:45.523 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_cache_name: BinnedLRUCache 2024-09-11T07:22:45.524 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_cache_options: 2024-09-11T07:22:45.524 INFO:journalctl@ceph.mon.b.smithi104.stdout: capacity : 536870912 2024-09-11T07:22:45.524 INFO:journalctl@ceph.mon.b.smithi104.stdout: num_shard_bits : 4 2024-09-11T07:22:45.524 INFO:journalctl@ceph.mon.b.smithi104.stdout: strict_capacity_limit : 0 2024-09-11T07:22:45.524 INFO:journalctl@ceph.mon.b.smithi104.stdout: high_pri_pool_ratio: 0.000 2024-09-11T07:22:45.524 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_cache_compressed: (nil) 2024-09-11T07:22:45.524 INFO:journalctl@ceph.mon.b.smithi104.stdout: persistent_cache: (nil) 2024-09-11T07:22:45.524 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_size: 4096 2024-09-11T07:22:45.524 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_size_deviation: 10 2024-09-11T07:22:45.524 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_restart_interval: 16 2024-09-11T07:22:45.524 INFO:journalctl@ceph.mon.b.smithi104.stdout: index_block_restart_interval: 1 2024-09-11T07:22:45.524 INFO:journalctl@ceph.mon.b.smithi104.stdout: metadata_block_size: 4096 2024-09-11T07:22:45.524 INFO:journalctl@ceph.mon.b.smithi104.stdout: partition_filters: 0 2024-09-11T07:22:45.525 INFO:journalctl@ceph.mon.b.smithi104.stdout: use_delta_encoding: 1 2024-09-11T07:22:45.525 INFO:journalctl@ceph.mon.b.smithi104.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-11T07:22:45.525 INFO:journalctl@ceph.mon.b.smithi104.stdout: whole_key_filtering: 1 2024-09-11T07:22:45.525 INFO:journalctl@ceph.mon.b.smithi104.stdout: verify_compression: 0 2024-09-11T07:22:45.525 INFO:journalctl@ceph.mon.b.smithi104.stdout: read_amp_bytes_per_bit: 0 2024-09-11T07:22:45.525 INFO:journalctl@ceph.mon.b.smithi104.stdout: format_version: 4 2024-09-11T07:22:45.525 INFO:journalctl@ceph.mon.b.smithi104.stdout: enable_index_compression: 1 2024-09-11T07:22:45.525 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_align: 0 2024-09-11T07:22:45.525 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T07:22:45.525 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T07:22:45.525 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.compression: NoCompression 2024-09-11T07:22:45.525 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T07:22:45.525 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T07:22:45.525 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T07:22:45.526 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.num_levels: 7 2024-09-11T07:22:45.526 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T07:22:45.526 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T07:22:45.526 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T07:22:45.526 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T07:22:45.526 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T07:22:45.526 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T07:22:45.526 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T07:22:45.526 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T07:22:45.526 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T07:22:45.526 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T07:22:45.526 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T07:22:45.526 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T07:22:45.526 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T07:22:45.527 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T07:22:45.527 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T07:22:45.527 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T07:22:45.527 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T07:22:45.527 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T07:22:45.527 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T07:22:45.527 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T07:22:45.527 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T07:22:45.527 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T07:22:45.527 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T07:22:45.527 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T07:22:45.527 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T07:22:45.527 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T07:22:45.527 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T07:22:45.528 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T07:22:45.528 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T07:22:45.528 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T07:22:45.528 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T07:22:45.528 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T07:22:45.528 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T07:22:45.528 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T07:22:45.528 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.arena_block_size: 4194304 2024-09-11T07:22:45.528 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T07:22:45.528 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T07:22:45.528 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-11T07:22:45.528 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T07:22:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T07:22:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T07:22:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T07:22:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T07:22:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T07:22:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T07:22:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T07:22:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T07:22:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T07:22:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T07:22:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T07:22:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.inplace_update_support: 0 2024-09-11T07:22:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T07:22:45.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T07:22:45.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T07:22:45.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T07:22:45.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.bloom_locality: 0 2024-09-11T07:22:45.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.max_successive_merges: 0 2024-09-11T07:22:45.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T07:22:45.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T07:22:45.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T07:22:45.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T07:22:45.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.ttl: 2592000 2024-09-11T07:22:45.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T07:22:45.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.enable_blob_files: false 2024-09-11T07:22:45.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.min_blob_size: 0 2024-09-11T07:22:45.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T07:22:45.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T07:22:45.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T07:22:45.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T07:22:45.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-11T07:22:45.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-11T07:22:45.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-11T07:22:45.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726039365493760, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-11T07:22:45.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-11T07:22:45.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-11T07:22:45.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726039365494067, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1536, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 488, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 366, "raw_average_value_size": 73, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1726039365, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "fd39d09b-0862-4aa9-a1ef-f77a480d9230", "db_session_id": "0RS9YIYH0ZVK0X2C0GYJ"}} 2024-09-11T07:22:45.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-11T07:22:45.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726039365494385, "job": 1, "event": "recovery_finished"} 2024-09-11T07:22:45.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T07:22:45.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55b69af58700 2024-09-11T07:22:45.532 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: DB pointer 0x55b69af28000 2024-09-11T07:22:45.532 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-11T07:22:45.532 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-11T07:22:45.532 INFO:journalctl@ceph.mon.b.smithi104.stdout: ** DB Stats ** 2024-09-11T07:22:45.532 INFO:journalctl@ceph.mon.b.smithi104.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:22:45.532 INFO:journalctl@ceph.mon.b.smithi104.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T07:22:45.532 INFO:journalctl@ceph.mon.b.smithi104.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T07:22:45.532 INFO:journalctl@ceph.mon.b.smithi104.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T07:22:45.532 INFO:journalctl@ceph.mon.b.smithi104.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T07:22:45.532 INFO:journalctl@ceph.mon.b.smithi104.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-11T07:22:45.532 INFO:journalctl@ceph.mon.b.smithi104.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T07:22:45.532 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2024-09-11T07:22:45.532 INFO:journalctl@ceph.mon.b.smithi104.stdout: ** Compaction Stats [default] ** 2024-09-11T07:22:45.532 INFO:journalctl@ceph.mon.b.smithi104.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:22:45.533 INFO:journalctl@ceph.mon.b.smithi104.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:22:45.533 INFO:journalctl@ceph.mon.b.smithi104.stdout: L0 1/0 1.50 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-11T07:22:45.533 INFO:journalctl@ceph.mon.b.smithi104.stdout: Sum 1/0 1.50 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-11T07:22:45.533 INFO:journalctl@ceph.mon.b.smithi104.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-11T07:22:45.533 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2024-09-11T07:22:45.533 INFO:journalctl@ceph.mon.b.smithi104.stdout: ** Compaction Stats [default] ** 2024-09-11T07:22:45.533 INFO:journalctl@ceph.mon.b.smithi104.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:22:45.533 INFO:journalctl@ceph.mon.b.smithi104.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:22:45.533 INFO:journalctl@ceph.mon.b.smithi104.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-11T07:22:45.533 INFO:journalctl@ceph.mon.b.smithi104.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:22:45.533 INFO:journalctl@ceph.mon.b.smithi104.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T07:22:45.533 INFO:journalctl@ceph.mon.b.smithi104.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T07:22:45.533 INFO:journalctl@ceph.mon.b.smithi104.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T07:22:45.533 INFO:journalctl@ceph.mon.b.smithi104.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T07:22:45.534 INFO:journalctl@ceph.mon.b.smithi104.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T07:22:45.534 INFO:journalctl@ceph.mon.b.smithi104.stdout: Cumulative compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:22:45.534 INFO:journalctl@ceph.mon.b.smithi104.stdout: Interval compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:22:45.534 INFO:journalctl@ceph.mon.b.smithi104.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T07:22:45.534 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2024-09-11T07:22:45.534 INFO:journalctl@ceph.mon.b.smithi104.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T07:22:45.534 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2024-09-11T07:22:45.534 INFO:journalctl@ceph.mon.b.smithi104.stdout: ** Compaction Stats [default] ** 2024-09-11T07:22:45.534 INFO:journalctl@ceph.mon.b.smithi104.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:22:45.534 INFO:journalctl@ceph.mon.b.smithi104.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:22:45.534 INFO:journalctl@ceph.mon.b.smithi104.stdout: L0 1/0 1.50 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-11T07:22:45.534 INFO:journalctl@ceph.mon.b.smithi104.stdout: Sum 1/0 1.50 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-11T07:22:45.534 INFO:journalctl@ceph.mon.b.smithi104.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-11T07:22:45.535 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2024-09-11T07:22:45.535 INFO:journalctl@ceph.mon.b.smithi104.stdout: ** Compaction Stats [default] ** 2024-09-11T07:22:45.535 INFO:journalctl@ceph.mon.b.smithi104.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:22:45.535 INFO:journalctl@ceph.mon.b.smithi104.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:22:45.535 INFO:journalctl@ceph.mon.b.smithi104.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.0 0.00 0.00 1 0.000 0 0 2024-09-11T07:22:45.536 INFO:journalctl@ceph.mon.b.smithi104.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:22:45.536 INFO:journalctl@ceph.mon.b.smithi104.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T07:22:45.536 INFO:journalctl@ceph.mon.b.smithi104.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T07:22:45.536 INFO:journalctl@ceph.mon.b.smithi104.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T07:22:45.536 INFO:journalctl@ceph.mon.b.smithi104.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T07:22:45.536 INFO:journalctl@ceph.mon.b.smithi104.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T07:22:45.536 INFO:journalctl@ceph.mon.b.smithi104.stdout: Cumulative compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:22:45.537 INFO:journalctl@ceph.mon.b.smithi104.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:22:45.537 INFO:journalctl@ceph.mon.b.smithi104.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T07:22:45.537 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2024-09-11T07:22:45.537 INFO:journalctl@ceph.mon.b.smithi104.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T07:22:45.537 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-11T07:22:45.537 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: using public_addrv v1:172.21.15.104:6789/0 2024-09-11T07:22:45.537 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: starting mon.b rank -1 at public addrs v1:172.21.15.104:6789/0 at bind addrs v1:172.21.15.104:6789/0 mon_data /var/lib/ceph/mon/ceph-b fsid 12a285d8-700e-11ef-bcea-c7b262605968 2024-09-11T07:22:45.537 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: mon.b@-1(???) e0 preinit fsid 12a285d8-700e-11ef-bcea-c7b262605968 2024-09-11T07:22:45.537 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: mon.b@-1(synchronizing).mds e1 new map 2024-09-11T07:22:45.537 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-11T07:22:45.537 INFO:journalctl@ceph.mon.b.smithi104.stdout: e1 2024-09-11T07:22:45.537 INFO:journalctl@ceph.mon.b.smithi104.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T07:22:45.537 INFO:journalctl@ceph.mon.b.smithi104.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-11T07:22:45.537 INFO:journalctl@ceph.mon.b.smithi104.stdout: legacy client fscid: -1 2024-09-11T07:22:45.538 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2024-09-11T07:22:45.538 INFO:journalctl@ceph.mon.b.smithi104.stdout: No filesystems configured 2024-09-11T07:22:45.538 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-11T07:22:45.538 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-11T07:22:45.538 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-11T07:22:45.538 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-11T07:22:45.538 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-11T07:22:45.538 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-11T07:22:45.538 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T07:22:45.538 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:22:45.538 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:22:45.538 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:22:45.539 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:22:45.539 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:45.539 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:45.539 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:45.539 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:45.539 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi104", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:22:45.539 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:22:45.539 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:22:45.540 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: Updating smithi104:/etc/ceph/ceph.conf 2024-09-11T07:22:45.540 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: Updating smithi104:/var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:22:45.540 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:22:45.540 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:45.540 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:45.540 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:45.540 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3430495946' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-11T07:22:45.540 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: Updating smithi104:/var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:22:45.540 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:22:45.540 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3430495946' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T07:22:45.540 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:22:45.540 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:22:45.541 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: from='client.14182 v1:172.21.15.104:0/3317283533' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi022:[v1:172.21.15.22:6789]=a;smithi022:[v1:172.21.15.22:6790]=c;smithi104:[v1:172.21.15.104:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:22:45.541 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: Saving service mon spec with placement smithi022:[v1:172.21.15.22:6789]=a;smithi022:[v1:172.21.15.22:6790]=c;smithi104:[v1:172.21.15.104:6789]=b;count:3 2024-09-11T07:22:45.541 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:45.541 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:22:45.541 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:22:45.541 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:22:45.541 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:45.541 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:22:45.541 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:22:45.541 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: Deploying daemon mon.b on smithi104 2024-09-11T07:22:45.541 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:22:45.541 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:22:45.541 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 ceph-mon[33507]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-11T07:22:45.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 bash[33455]: a00a636a58c32139f1377c348694d6be406705ca60f2450007b611ee1ad485de 2024-09-11T07:22:45.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:45 smithi104 systemd[1]: Started Ceph mon.b for 12a285d8-700e-11ef-bcea-c7b262605968. 2024-09-11T07:22:47.121 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T07:22:47.121 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T07:22:47.375 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:22:50.761 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:22:50.761 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":2,"fsid":"12a285d8-700e-11ef-bcea-c7b262605968","modified":"2024-09-11T07:22:45.514913Z","created":"2024-09-11T07:21:13.000870Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6789","nonce":0}]},"addr":"172.21.15.104:6789/0","public_addr":"172.21.15.104:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T07:22:50.761 INFO:teuthology.orchestra.run.smithi104.stderr:dumped monmap epoch 2 2024-09-11T07:22:50.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:50 smithi022 ceph-mon[29462]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:22:50.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:50 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:22:50.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:50 smithi022 ceph-mon[29462]: mon.a calling monitor election 2024-09-11T07:22:50.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:50 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:22:50.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:50 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:22:50.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:50 smithi022 ceph-mon[29462]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:22:50.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:50 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:22:50.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:50 smithi022 ceph-mon[29462]: mon.b calling monitor election 2024-09-11T07:22:50.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:50 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:22:50.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:50 smithi022 ceph-mon[29462]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:22:50.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:50 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:22:50.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:50 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:22:50.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:50 smithi022 ceph-mon[29462]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T07:22:50.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:50 smithi022 ceph-mon[29462]: monmap e2: 2 mons at {a=v1:172.21.15.22:6789/0,b=v1:172.21.15.104:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:22:50.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:50 smithi022 ceph-mon[29462]: fsmap 2024-09-11T07:22:50.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:50 smithi022 ceph-mon[29462]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:22:50.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:50 smithi022 ceph-mon[29462]: mgrmap e13: y(active, since 39s) 2024-09-11T07:22:50.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:50 smithi022 ceph-mon[29462]: overall HEALTH_OK 2024-09-11T07:22:50.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:50 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:50.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:50 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:50.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:50 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:50.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:50 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:22:50.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:50 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:22:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:50 smithi104 ceph-mon[33507]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:22:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:50 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:22:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:50 smithi104 ceph-mon[33507]: mon.a calling monitor election 2024-09-11T07:22:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:50 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:22:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:50 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:22:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:50 smithi104 ceph-mon[33507]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:22:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:50 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:22:51.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:50 smithi104 ceph-mon[33507]: mon.b calling monitor election 2024-09-11T07:22:51.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:50 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:22:51.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:50 smithi104 ceph-mon[33507]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:22:51.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:50 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:22:51.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:50 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:22:51.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:50 smithi104 ceph-mon[33507]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T07:22:51.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:50 smithi104 ceph-mon[33507]: monmap e2: 2 mons at {a=v1:172.21.15.22:6789/0,b=v1:172.21.15.104:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:22:51.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:50 smithi104 ceph-mon[33507]: fsmap 2024-09-11T07:22:51.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:50 smithi104 ceph-mon[33507]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:22:51.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:50 smithi104 ceph-mon[33507]: mgrmap e13: y(active, since 39s) 2024-09-11T07:22:51.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:50 smithi104 ceph-mon[33507]: overall HEALTH_OK 2024-09-11T07:22:51.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:50 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:51.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:50 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:51.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:50 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:51.032 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:50 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:22:51.032 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:50 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:22:51.886 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:51 smithi022 ceph-mon[29462]: Deploying daemon mon.c on smithi022 2024-09-11T07:22:51.886 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:51 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.104:0/3743069462' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:22:51.887 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:51 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:22:52.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:51 smithi104 ceph-mon[33507]: Deploying daemon mon.c on smithi022 2024-09-11T07:22:52.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:51 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.104:0/3743069462' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:22:52.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:51 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:22:52.306 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T07:22:52.307 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T07:22:52.554 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:22:52.876 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:52 smithi104 ceph-mon[33507]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:22:52.893 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:52 smithi022 ceph-mon[29462]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:22:52.893 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:22:52 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:22:52.513+0000 7f29b1e87640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-11T07:22:53.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:22:53 smithi022 systemd[1]: Started Ceph mon.c for 12a285d8-700e-11ef-bcea-c7b262605968. 2024-09-11T07:22:58.984 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-11T07:22:58.984 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":3,"fsid":"12a285d8-700e-11ef-bcea-c7b262605968","modified":"2024-09-11T07:22:53.726749Z","created":"2024-09-11T07:21:13.000870Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6789","nonce":0}]},"addr":"172.21.15.104:6789/0","public_addr":"172.21.15.104:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6790","nonce":0}]},"addr":"172.21.15.22:6790/0","public_addr":"172.21.15.22:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T07:22:58.985 INFO:teuthology.orchestra.run.smithi104.stderr:dumped monmap epoch 3 2024-09-11T07:22:59.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:58 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:22:59.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:58 smithi022 ceph-mon[29462]: mon.a calling monitor election 2024-09-11T07:22:59.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:58 smithi022 ceph-mon[29462]: mon.b calling monitor election 2024-09-11T07:22:59.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:58 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:22:59.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:58 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:22:59.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:58 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:22:59.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:58 smithi022 ceph-mon[29462]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:22:59.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:58 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:22:59.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:58 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:22:59.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:58 smithi022 ceph-mon[29462]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:22:59.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:58 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:22:59.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:58 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:22:59.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:58 smithi022 ceph-mon[29462]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T07:22:59.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:58 smithi022 ceph-mon[29462]: monmap e3: 3 mons at {a=v1:172.21.15.22:6789/0,b=v1:172.21.15.104:6789/0,c=v1:172.21.15.22:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:22:59.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:58 smithi022 ceph-mon[29462]: fsmap 2024-09-11T07:22:59.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:58 smithi022 ceph-mon[29462]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:22:59.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:58 smithi022 ceph-mon[29462]: mgrmap e13: y(active, since 47s) 2024-09-11T07:22:59.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:58 smithi022 ceph-mon[29462]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-11T07:22:59.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:58 smithi022 ceph-mon[29462]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-11T07:22:59.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:58 smithi022 ceph-mon[29462]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-11T07:22:59.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:58 smithi022 ceph-mon[29462]: mon.c (rank 2) addr v1:172.21.15.22:6790/0 is down (out of quorum) 2024-09-11T07:22:59.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:58 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:59.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:58 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:59.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:58 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:59.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:58 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:59.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:58 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:22:59.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:58 smithi104 ceph-mon[33507]: mon.a calling monitor election 2024-09-11T07:22:59.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:58 smithi104 ceph-mon[33507]: mon.b calling monitor election 2024-09-11T07:22:59.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:58 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:22:59.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:58 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:22:59.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:58 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:22:59.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:58 smithi104 ceph-mon[33507]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:22:59.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:58 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:22:59.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:58 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:22:59.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:58 smithi104 ceph-mon[33507]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:22:59.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:58 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:22:59.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:58 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:22:59.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:58 smithi104 ceph-mon[33507]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T07:22:59.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:58 smithi104 ceph-mon[33507]: monmap e3: 3 mons at {a=v1:172.21.15.22:6789/0,b=v1:172.21.15.104:6789/0,c=v1:172.21.15.22:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:22:59.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:58 smithi104 ceph-mon[33507]: fsmap 2024-09-11T07:22:59.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:58 smithi104 ceph-mon[33507]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:22:59.282 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:58 smithi104 ceph-mon[33507]: mgrmap e13: y(active, since 47s) 2024-09-11T07:22:59.282 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:58 smithi104 ceph-mon[33507]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-11T07:22:59.282 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:58 smithi104 ceph-mon[33507]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-11T07:22:59.282 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:58 smithi104 ceph-mon[33507]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-11T07:22:59.282 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:58 smithi104 ceph-mon[33507]: mon.c (rank 2) addr v1:172.21.15.22:6790/0 is down (out of quorum) 2024-09-11T07:22:59.282 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:58 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:59.282 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:58 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:59.282 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:58 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:59.282 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:58 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:22:59.534 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-11T07:22:59.534 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-11T07:22:59.918 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:23:00.054 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:59 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:00.054 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:59 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.104:0/7978677' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:23:00.054 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:59 smithi104 ceph-mon[33507]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:00.054 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:22:59 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:23:00.067 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:59 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:00.067 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:59 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.104:0/7978677' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:23:00.067 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:59 smithi022 ceph-mon[29462]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:00.068 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:22:59 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:23:01.092 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:23:01.092 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: mon.a calling monitor election 2024-09-11T07:23:01.093 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: mon.b calling monitor election 2024-09-11T07:23:01.093 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:23:01.093 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:23:01.093 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:23:01.093 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:01.093 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:23:01.093 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:23:01.093 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:01.093 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:23:01.093 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:23:01.093 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T07:23:01.093 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: monmap e3: 3 mons at {a=v1:172.21.15.22:6789/0,b=v1:172.21.15.104:6789/0,c=v1:172.21.15.22:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:23:01.093 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: fsmap 2024-09-11T07:23:01.094 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:23:01.094 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: mgrmap e13: y(active, since 47s) 2024-09-11T07:23:01.094 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-11T07:23:01.094 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-11T07:23:01.094 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-11T07:23:01.094 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: mon.c (rank 2) addr v1:172.21.15.22:6790/0 is down (out of quorum) 2024-09-11T07:23:01.094 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:01.094 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:01.094 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:01.094 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:01.094 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:01.094 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.104:0/7978677' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:23:01.094 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:01.094 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:00 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:23:02.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:01 smithi104 ceph-mon[33507]: mon.c calling monitor election 2024-09-11T07:23:02.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:01 smithi104 ceph-mon[33507]: mon.c calling monitor election 2024-09-11T07:23:02.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:01 smithi104 ceph-mon[33507]: mon.b calling monitor election 2024-09-11T07:23:02.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:01 smithi104 ceph-mon[33507]: mon.a calling monitor election 2024-09-11T07:23:02.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:01 smithi104 ceph-mon[33507]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T07:23:02.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:01 smithi104 ceph-mon[33507]: monmap e3: 3 mons at {a=v1:172.21.15.22:6789/0,b=v1:172.21.15.104:6789/0,c=v1:172.21.15.22:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:23:02.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:01 smithi104 ceph-mon[33507]: fsmap 2024-09-11T07:23:02.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:01 smithi104 ceph-mon[33507]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:23:02.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:01 smithi104 ceph-mon[33507]: mgrmap e13: y(active, since 49s) 2024-09-11T07:23:02.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:01 smithi104 ceph-mon[33507]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-11T07:23:02.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:01 smithi104 ceph-mon[33507]: Cluster is now healthy 2024-09-11T07:23:02.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:01 smithi104 ceph-mon[33507]: overall HEALTH_OK 2024-09-11T07:23:02.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:01 smithi104 ceph-mon[33507]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T07:23:02.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:01 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:23:02.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:01 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:23:02.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:01 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:02.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:01 smithi104 ceph-mon[33507]: Reconfiguring daemon mon.a on smithi022 2024-09-11T07:23:02.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:01 smithi104 ceph-mon[33507]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:02.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:01 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:02.041 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[29462]: mon.c calling monitor election 2024-09-11T07:23:02.041 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[29462]: mon.c calling monitor election 2024-09-11T07:23:02.041 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[29462]: mon.b calling monitor election 2024-09-11T07:23:02.041 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[29462]: mon.a calling monitor election 2024-09-11T07:23:02.041 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[29462]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T07:23:02.041 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[29462]: monmap e3: 3 mons at {a=v1:172.21.15.22:6789/0,b=v1:172.21.15.104:6789/0,c=v1:172.21.15.22:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:23:02.041 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[29462]: fsmap 2024-09-11T07:23:02.041 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[29462]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:23:02.042 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[29462]: mgrmap e13: y(active, since 49s) 2024-09-11T07:23:02.042 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[29462]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-11T07:23:02.042 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[29462]: Cluster is now healthy 2024-09-11T07:23:02.042 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[29462]: overall HEALTH_OK 2024-09-11T07:23:02.042 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[29462]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T07:23:02.042 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:23:02.042 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:23:02.042 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:02.042 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[29462]: Reconfiguring daemon mon.a on smithi022 2024-09-11T07:23:02.042 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[29462]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:02.042 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:02.042 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[37330]: mon.c calling monitor election 2024-09-11T07:23:02.043 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[37330]: mon.c calling monitor election 2024-09-11T07:23:02.043 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[37330]: mon.b calling monitor election 2024-09-11T07:23:02.043 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[37330]: mon.a calling monitor election 2024-09-11T07:23:02.043 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[37330]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T07:23:02.043 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[37330]: monmap e3: 3 mons at {a=v1:172.21.15.22:6789/0,b=v1:172.21.15.104:6789/0,c=v1:172.21.15.22:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:23:02.043 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[37330]: fsmap 2024-09-11T07:23:02.043 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[37330]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:23:02.043 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[37330]: mgrmap e13: y(active, since 49s) 2024-09-11T07:23:02.043 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[37330]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-11T07:23:02.043 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[37330]: Cluster is now healthy 2024-09-11T07:23:02.043 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[37330]: overall HEALTH_OK 2024-09-11T07:23:02.043 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[37330]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T07:23:02.043 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:23:02.044 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:23:02.044 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:02.044 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[37330]: Reconfiguring daemon mon.a on smithi022 2024-09-11T07:23:02.044 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[37330]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:02.044 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:01 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:02.074 INFO:teuthology.orchestra.run.smithi022.stdout:# minimal ceph.conf for 12a285d8-700e-11ef-bcea-c7b262605968 2024-09-11T07:23:02.075 INFO:teuthology.orchestra.run.smithi022.stdout:[global] 2024-09-11T07:23:02.075 INFO:teuthology.orchestra.run.smithi022.stdout: fsid = 12a285d8-700e-11ef-bcea-c7b262605968 2024-09-11T07:23:02.075 INFO:teuthology.orchestra.run.smithi022.stdout: mon_host = 172.21.15.22:6789/0 172.21.15.104:6789/0 v1:172.21.15.22:6790/0 2024-09-11T07:23:02.997 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:02 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:23:02.997 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:02 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/720102992' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:02.998 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:02 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:02.998 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:02 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:02.998 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:02 smithi104 ceph-mon[33507]: Reconfiguring mon.c (monmap changed)... 2024-09-11T07:23:02.998 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:02 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:23:02.998 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:02 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:23:02.998 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:02 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:03.005 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:02 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:23:03.005 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:02 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/720102992' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:03.005 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:02 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:03.005 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:02 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:03.005 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:02 smithi022 ceph-mon[37330]: Reconfiguring mon.c (monmap changed)... 2024-09-11T07:23:03.006 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:02 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:23:03.006 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:02 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:23:03.006 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:02 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:03.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:02 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:23:03.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:02 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/720102992' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:03.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:02 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:03.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:02 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:03.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:02 smithi022 ceph-mon[29462]: Reconfiguring mon.c (monmap changed)... 2024-09-11T07:23:03.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:02 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:23:03.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:02 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:23:03.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:02 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:03.321 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-11T07:23:03.321 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-11T07:23:03.321 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T07:23:03.349 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-11T07:23:03.350 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:23:03.418 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-11T07:23:03.418 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T07:23:03.451 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-11T07:23:03.451 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:23:03.520 INFO:tasks.cephadm:Adding mgr.y on smithi022 2024-09-11T07:23:03.521 INFO:tasks.cephadm:Adding mgr.x on smithi104 2024-09-11T07:23:03.521 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph orch apply mgr '2;smithi022=y;smithi104=x' 2024-09-11T07:23:03.798 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:23:03.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:03 smithi022 ceph-mon[29462]: Reconfiguring daemon mon.c on smithi022 2024-09-11T07:23:03.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:03 smithi022 ceph-mon[29462]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:03.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:03 smithi022 ceph-mon[37330]: Reconfiguring daemon mon.c on smithi022 2024-09-11T07:23:03.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:03 smithi022 ceph-mon[37330]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:04.005 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:03 smithi104 ceph-mon[33507]: Reconfiguring daemon mon.c on smithi022 2024-09-11T07:23:04.005 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:03 smithi104 ceph-mon[33507]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:05.836 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:05 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:05.836 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:05 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:05.837 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:05 smithi104 ceph-mon[33507]: Reconfiguring mon.b (monmap changed)... 2024-09-11T07:23:05.837 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:05 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:23:05.837 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:05 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:23:05.837 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:05 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:05.837 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:05 smithi104 ceph-mon[33507]: Reconfiguring daemon mon.b on smithi104 2024-09-11T07:23:05.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:05 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:05.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:05 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:05.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:05 smithi022 ceph-mon[29462]: Reconfiguring mon.b (monmap changed)... 2024-09-11T07:23:05.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:05 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:23:05.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:05 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:23:05.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:05 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:05.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:05 smithi022 ceph-mon[29462]: Reconfiguring daemon mon.b on smithi104 2024-09-11T07:23:05.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:05 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:05.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:05 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:05.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:05 smithi022 ceph-mon[37330]: Reconfiguring mon.b (monmap changed)... 2024-09-11T07:23:05.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:05 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:23:05.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:05 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:23:05.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:05 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:05.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:05 smithi022 ceph-mon[37330]: Reconfiguring daemon mon.b on smithi104 2024-09-11T07:23:06.096 INFO:teuthology.orchestra.run.smithi104.stdout:Scheduled mgr update... 2024-09-11T07:23:06.859 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:06 smithi104 ceph-mon[33507]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:06.859 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:06 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:06.859 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:06 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:06.859 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:06 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:06.859 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:06 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:06.859 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:06 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:06.859 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:06 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:23:06.859 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:06 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:06.859 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:06 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:23:06.859 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:06 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T07:23:06.859 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:06 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:23:06.860 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:06 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:06.881 DEBUG:teuthology.orchestra.run.smithi104:mgr.x> sudo journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@mgr.x.service 2024-09-11T07:23:06.884 INFO:tasks.cephadm:Deploying OSDs... 2024-09-11T07:23:06.884 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-11T07:23:06.884 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T07:23:06.904 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T07:23:06.904 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_1 2024-09-11T07:23:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:06 smithi022 ceph-mon[29462]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:06 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:06 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:06 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:06 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:06 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:06 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:23:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:06 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:06 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:23:06.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:06 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T07:23:06.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:06 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:23:06.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:06 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:06 smithi022 ceph-mon[37330]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:06 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:06 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:06 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:06 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:06 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:06.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:06 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:23:06.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:06 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:06.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:06 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:23:06.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:06 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T07:23:06.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:06 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:23:06.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:06 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:06.937 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T07:23:06.938 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:23:06.938 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 963 Links: 1 2024-09-11T07:23:06.938 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:23:06.938 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:23:06.938 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-09-11 07:22:53.205812798 +0000 2024-09-11T07:23:06.938 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-09-11 07:16:25.837009658 +0000 2024-09-11T07:23:06.938 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-09-11 07:16:25.837009658 +0000 2024-09-11T07:23:06.938 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-09-11 07:16:25.837009658 +0000 2024-09-11T07:23:06.938 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T07:23:07.007 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-09-11T07:23:07.007 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-09-11T07:23:07.007 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000131585 s, 3.9 MB/s 2024-09-11T07:23:07.009 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T07:23:07.068 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_2 2024-09-11T07:23:07.126 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T07:23:07.127 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:23:07.127 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 973 Links: 1 2024-09-11T07:23:07.127 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:23:07.127 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:23:07.127 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-09-11 07:22:53.205812798 +0000 2024-09-11T07:23:07.127 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-09-11 07:16:25.837009658 +0000 2024-09-11T07:23:07.127 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-09-11 07:16:25.837009658 +0000 2024-09-11T07:23:07.127 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-09-11 07:16:25.837009658 +0000 2024-09-11T07:23:07.127 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T07:23:07.194 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-09-11T07:23:07.195 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-09-11T07:23:07.195 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000107214 s, 4.8 MB/s 2024-09-11T07:23:07.196 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T07:23:07.256 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_3 2024-09-11T07:23:07.315 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T07:23:07.315 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:23:07.315 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 982 Links: 1 2024-09-11T07:23:07.316 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:23:07.316 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:23:07.316 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-09-11 07:22:53.205812798 +0000 2024-09-11T07:23:07.316 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-09-11 07:16:25.839009630 +0000 2024-09-11T07:23:07.316 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-09-11 07:16:25.839009630 +0000 2024-09-11T07:23:07.316 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-09-11 07:16:25.839009630 +0000 2024-09-11T07:23:07.316 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T07:23:07.384 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-09-11T07:23:07.384 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-09-11T07:23:07.384 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000119344 s, 4.3 MB/s 2024-09-11T07:23:07.385 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T07:23:07.445 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_4 2024-09-11T07:23:07.504 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T07:23:07.504 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:23:07.504 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 968 Links: 1 2024-09-11T07:23:07.504 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:23:07.504 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:23:07.505 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-09-11 07:22:53.206812783 +0000 2024-09-11T07:23:07.505 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-09-11 07:16:25.837009658 +0000 2024-09-11T07:23:07.505 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-09-11 07:16:25.837009658 +0000 2024-09-11T07:23:07.505 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-09-11 07:16:25.837009658 +0000 2024-09-11T07:23:07.505 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T07:23:07.573 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-09-11T07:23:07.573 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-09-11T07:23:07.573 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000127357 s, 4.0 MB/s 2024-09-11T07:23:07.575 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T07:23:07.634 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-11T07:23:07.634 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T07:23:07.651 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T07:23:07.651 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_1 2024-09-11T07:23:07.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:07 smithi022 ceph-mon[29462]: from='client.24103 v1:172.21.15.104:0/1155163496' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi022=y;smithi104=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:23:07.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:07 smithi022 ceph-mon[29462]: Saving service mgr spec with placement smithi022=y;smithi104=x;count:2 2024-09-11T07:23:07.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:07 smithi022 ceph-mon[29462]: Deploying daemon mgr.x on smithi104 2024-09-11T07:23:07.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:07 smithi022 ceph-mon[37330]: from='client.24103 v1:172.21.15.104:0/1155163496' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi022=y;smithi104=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:23:07.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:07 smithi022 ceph-mon[37330]: Saving service mgr spec with placement smithi022=y;smithi104=x;count:2 2024-09-11T07:23:07.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:07 smithi022 ceph-mon[37330]: Deploying daemon mgr.x on smithi104 2024-09-11T07:23:07.706 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T07:23:07.706 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:23:07.706 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 976 Links: 1 2024-09-11T07:23:07.706 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:23:07.707 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:23:07.707 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-09-11 07:22:44.219493577 +0000 2024-09-11T07:23:07.707 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-09-11 07:16:25.944718611 +0000 2024-09-11T07:23:07.707 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-09-11 07:16:25.944718611 +0000 2024-09-11T07:23:07.707 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-09-11 07:16:25.944718611 +0000 2024-09-11T07:23:07.707 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T07:23:07.767 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:07 smithi104 ceph-mon[33507]: from='client.24103 v1:172.21.15.104:0/1155163496' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi022=y;smithi104=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:23:07.768 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:07 smithi104 ceph-mon[33507]: Saving service mgr spec with placement smithi022=y;smithi104=x;count:2 2024-09-11T07:23:07.768 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:07 smithi104 ceph-mon[33507]: Deploying daemon mgr.x on smithi104 2024-09-11T07:23:07.770 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-09-11T07:23:07.770 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-09-11T07:23:07.770 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000213967 s, 2.4 MB/s 2024-09-11T07:23:07.771 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T07:23:07.827 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_2 2024-09-11T07:23:07.883 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T07:23:07.883 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:23:07.883 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 974 Links: 1 2024-09-11T07:23:07.883 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:23:07.883 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:23:07.883 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-09-11 07:22:44.219493577 +0000 2024-09-11T07:23:07.883 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-09-11 07:16:25.944718611 +0000 2024-09-11T07:23:07.883 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-09-11 07:16:25.944718611 +0000 2024-09-11T07:23:07.884 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-09-11 07:16:25.944718611 +0000 2024-09-11T07:23:07.884 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T07:23:07.946 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-09-11T07:23:07.947 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-09-11T07:23:07.947 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000186387 s, 2.7 MB/s 2024-09-11T07:23:07.948 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T07:23:08.004 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_3 2024-09-11T07:23:08.062 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T07:23:08.062 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:23:08.062 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 967 Links: 1 2024-09-11T07:23:08.062 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:23:08.062 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:23:08.063 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-09-11 07:22:44.219493577 +0000 2024-09-11T07:23:08.063 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-09-11 07:16:25.943718419 +0000 2024-09-11T07:23:08.063 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-09-11 07:16:25.943718419 +0000 2024-09-11T07:23:08.063 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-09-11 07:16:25.943718419 +0000 2024-09-11T07:23:08.063 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T07:23:08.177 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-09-11T07:23:08.177 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-09-11T07:23:08.177 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000255479 s, 2.0 MB/s 2024-09-11T07:23:08.178 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T07:23:08.194 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_4 2024-09-11T07:23:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T07:23:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:23:08.252 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 980 Links: 1 2024-09-11T07:23:08.253 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:23:08.253 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:23:08.253 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-09-11 07:23:08.130215195 +0000 2024-09-11T07:23:08.253 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-09-11 07:16:25.944718611 +0000 2024-09-11T07:23:08.253 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-09-11 07:16:25.944718611 +0000 2024-09-11T07:23:08.253 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-09-11 07:16:25.944718611 +0000 2024-09-11T07:23:08.253 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T07:23:08.352 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-09-11T07:23:08.353 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-09-11T07:23:08.353 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000230848 s, 2.2 MB/s 2024-09-11T07:23:08.354 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T07:23:08.369 INFO:tasks.cephadm:Deploying osd.0 on smithi022 with /dev/vg_nvme/lv_4... 2024-09-11T07:23:08.370 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-11T07:23:08.583 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:08 smithi104 ceph-mon[33507]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:08.631 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:23:08.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:08 smithi022 ceph-mon[29462]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:08.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:08 smithi022 ceph-mon[37330]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:08.885 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:08 smithi104 podman[35326]: 2024-09-11 07:23:08.819676554 +0000 UTC m=+0.349958470 container start 5d978c819564a188fc0c090d77ec6ce442946e7b689637082f046e4e5986fb08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:23:09.199 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:08 smithi104 bash[35326]: 5d978c819564a188fc0c090d77ec6ce442946e7b689637082f046e4e5986fb08 2024-09-11T07:23:09.199 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:08 smithi104 systemd[1]: Started Ceph mgr.x for 12a285d8-700e-11ef-bcea-c7b262605968. 2024-09-11T07:23:09.199 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:08 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:23:08.941+0000 7f01865a5ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:23:09.199 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:08 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:23:08.998+0000 7f01865a5ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:23:09.199 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:09 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:23:09.066+0000 7f01865a5ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:23:09.199 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:09 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:23:09.196+0000 7f01865a5ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:23:09.530 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:09 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:23:09.348+0000 7f01865a5ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:23:09.954 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:09 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:23:09.618+0000 7f01865a5ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:23:09.955 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:09 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:23:09.682+0000 7f01865a5ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:23:09.955 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:09 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:23:09.849+0000 7f01865a5ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:23:10.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:09 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:10.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:09 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:10.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:09 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:10.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:09 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:10.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:09 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:10.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:09 smithi104 ceph-mon[33507]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:10.333 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:09 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:10.333 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:09 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:10.333 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:09 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:10.333 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:09 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:10.333 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:09 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:10.333 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:09 smithi022 ceph-mon[37330]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:10.334 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:09 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:10.334 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:09 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:10.334 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:09 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:10.334 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:09 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:10.334 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:09 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:10.334 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:09 smithi022 ceph-mon[29462]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:10.334 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-11T07:23:10.417 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-11T07:23:10.428 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2024-09-11T07:23:10.428 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2024-09-11T07:23:10.428 INFO:teuthology.orchestra.run.smithi022.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00886602 s, 1.2 GB/s 2024-09-11T07:23:10.428 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2024-09-11T07:23:11.029 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:10 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:23:10.717+0000 7f01865a5ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:23:11.030 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:10 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:23:10.782+0000 7f01865a5ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:23:11.070 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi022:vg_nvme/lv_4 2024-09-11T07:23:11.321 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:11 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:11.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:11 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:11.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:11 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:11.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:11 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:23:11.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:11 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:11.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:11 smithi022 ceph-mon[29462]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T07:23:11.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:11 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:23:11.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:11 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:23:11.323 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:11 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:11.323 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:11 smithi022 ceph-mon[29462]: Reconfiguring daemon mgr.y on smithi022 2024-09-11T07:23:11.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:11 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:11.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:11 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:11.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:11 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:11.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:11 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:23:11.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:11 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:11.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:11 smithi022 ceph-mon[37330]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T07:23:11.324 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:11 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:23:11.324 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:11 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:23:11.324 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:11 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:11.324 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:11 smithi022 ceph-mon[37330]: Reconfiguring daemon mgr.y on smithi022 2024-09-11T07:23:11.496 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:11 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:11.496 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:11 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:11.496 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:11 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:11.496 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:11 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:23:11.496 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:11 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:11.496 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:11 smithi104 ceph-mon[33507]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T07:23:11.496 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:11 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:23:11.496 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:11 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:23:11.497 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:11 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:11.497 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:11 smithi104 ceph-mon[33507]: Reconfiguring daemon mgr.y on smithi022 2024-09-11T07:23:11.780 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:11 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:23:11.494+0000 7f01865a5ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:23:11.780 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:11 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:23:11.603+0000 7f01865a5ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:23:11.780 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:11 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:23:11.722+0000 7f01865a5ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:23:12.171 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:11 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:23:11.781+0000 7f01865a5ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:23:12.171 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:11 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:23:11.908+0000 7f01865a5ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:23:12.171 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:11 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:23:11.974+0000 7f01865a5ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:23:12.171 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:12 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:23:12.031+0000 7f01865a5ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:23:12.171 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:12 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:23:12.165+0000 7f01865a5ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:23:12.249 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:23:12.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:12 smithi022 ceph-mon[29462]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:12.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:12 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:12.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:12 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:12.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:12 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:12.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:12 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:12.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:12 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:12.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:12 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:23:12.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:12 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:12.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:12 smithi022 ceph-mon[37330]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:12.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:12 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:12.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:12 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:12.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:12 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:12.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:12 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:12.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:12 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:12.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:12 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:23:12.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:12 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:12.437 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:12 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:23:12.229+0000 7f01865a5ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:23:12.437 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:12 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:23:12.296+0000 7f01865a5ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:23:12.437 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:12 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:23:12.377+0000 7f01865a5ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:23:12.437 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:12 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:23:12.435+0000 7f01865a5ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:23:12.438 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:12 smithi104 ceph-mon[33507]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:12.438 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:12 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:12.438 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:12 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:12.438 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:12 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:12.438 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:12 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:12.438 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:12 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:12.438 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:12 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:23:12.438 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:12 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:12.779 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:12 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:23:12.492+0000 7f01865a5ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:23:13.338 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:13 smithi022 ceph-mon[37330]: Standby manager daemon x started 2024-09-11T07:23:13.338 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:13 smithi022 ceph-mon[37330]: from='mgr.? v1:172.21.15.104:0/2183796261' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:23:13.339 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:13 smithi022 ceph-mon[37330]: from='mgr.? v1:172.21.15.104:0/2183796261' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:23:13.339 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:13 smithi022 ceph-mon[37330]: from='mgr.? v1:172.21.15.104:0/2183796261' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:23:13.339 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:13 smithi022 ceph-mon[37330]: from='mgr.? v1:172.21.15.104:0/2183796261' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:23:13.339 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:13 smithi022 ceph-mon[29462]: Standby manager daemon x started 2024-09-11T07:23:13.339 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:13 smithi022 ceph-mon[29462]: from='mgr.? v1:172.21.15.104:0/2183796261' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:23:13.339 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:13 smithi022 ceph-mon[29462]: from='mgr.? v1:172.21.15.104:0/2183796261' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:23:13.339 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:13 smithi022 ceph-mon[29462]: from='mgr.? v1:172.21.15.104:0/2183796261' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:23:13.339 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:13 smithi022 ceph-mon[29462]: from='mgr.? v1:172.21.15.104:0/2183796261' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:23:13.530 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:23:13 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:23:13.234+0000 7f01865a5ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:23:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:13 smithi104 ceph-mon[33507]: Standby manager daemon x started 2024-09-11T07:23:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:13 smithi104 ceph-mon[33507]: from='mgr.? v1:172.21.15.104:0/2183796261' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:23:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:13 smithi104 ceph-mon[33507]: from='mgr.? v1:172.21.15.104:0/2183796261' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:23:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:13 smithi104 ceph-mon[33507]: from='mgr.? v1:172.21.15.104:0/2183796261' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:23:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:13 smithi104 ceph-mon[33507]: from='mgr.? v1:172.21.15.104:0/2183796261' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:23:14.558 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:14 smithi022 ceph-mon[29462]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:14.558 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:14 smithi022 ceph-mon[29462]: mgrmap e14: y(active, since 62s), standbys: x 2024-09-11T07:23:14.558 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:14 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:23:14.558 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:14 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:23:14.558 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:14 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:23:14.558 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:14 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:14.559 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:14 smithi022 ceph-mon[37330]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:14.559 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:14 smithi022 ceph-mon[37330]: mgrmap e14: y(active, since 62s), standbys: x 2024-09-11T07:23:14.559 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:14 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:23:14.559 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:14 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:23:14.559 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:14 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:23:14.559 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:14 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:14.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:14 smithi104 ceph-mon[33507]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:14.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:14 smithi104 ceph-mon[33507]: mgrmap e14: y(active, since 62s), standbys: x 2024-09-11T07:23:14.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:14 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:23:14.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:14 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:23:14.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:14 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:23:14.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:14 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:15.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:15 smithi022 ceph-mon[29462]: from='client.24116 v1:172.21.15.22:0/55778282' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:23:15.398 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:15 smithi022 ceph-mon[37330]: from='client.24116 v1:172.21.15.22:0/55778282' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:23:15.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:15 smithi104 ceph-mon[33507]: from='client.24116 v1:172.21.15.22:0/55778282' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:23:16.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:16 smithi022 ceph-mon[29462]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:16.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:16 smithi022 ceph-mon[37330]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:16.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:16 smithi104 ceph-mon[33507]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:17.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:17 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2876201263' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09075f16-32f4-4f88-a116-c3d7f977a0e0"}]: dispatch 2024-09-11T07:23:17.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:17 smithi022 ceph-mon[29462]: from='client.24122 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09075f16-32f4-4f88-a116-c3d7f977a0e0"}]: dispatch 2024-09-11T07:23:17.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:17 smithi022 ceph-mon[29462]: from='client.24122 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09075f16-32f4-4f88-a116-c3d7f977a0e0"}]': finished 2024-09-11T07:23:17.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:17 smithi022 ceph-mon[29462]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T07:23:17.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:17 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:23:17.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:17 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1931027635' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:23:17.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:17 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2876201263' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09075f16-32f4-4f88-a116-c3d7f977a0e0"}]: dispatch 2024-09-11T07:23:17.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:17 smithi022 ceph-mon[37330]: from='client.24122 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09075f16-32f4-4f88-a116-c3d7f977a0e0"}]: dispatch 2024-09-11T07:23:17.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:17 smithi022 ceph-mon[37330]: from='client.24122 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09075f16-32f4-4f88-a116-c3d7f977a0e0"}]': finished 2024-09-11T07:23:17.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:17 smithi022 ceph-mon[37330]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T07:23:17.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:17 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:23:17.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:17 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1931027635' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:23:17.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:17 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2876201263' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09075f16-32f4-4f88-a116-c3d7f977a0e0"}]: dispatch 2024-09-11T07:23:17.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:17 smithi104 ceph-mon[33507]: from='client.24122 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09075f16-32f4-4f88-a116-c3d7f977a0e0"}]: dispatch 2024-09-11T07:23:17.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:17 smithi104 ceph-mon[33507]: from='client.24122 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09075f16-32f4-4f88-a116-c3d7f977a0e0"}]': finished 2024-09-11T07:23:17.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:17 smithi104 ceph-mon[33507]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T07:23:17.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:17 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:23:17.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:17 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1931027635' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:23:18.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:18 smithi022 ceph-mon[29462]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:18.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:18 smithi022 ceph-mon[37330]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:18 smithi104 ceph-mon[33507]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:20.440 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:20 smithi022 ceph-mon[29462]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:20.441 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:20 smithi022 ceph-mon[37330]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:20.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:20 smithi104 ceph-mon[33507]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:22.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:22 smithi022 ceph-mon[29462]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:22.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:22 smithi022 ceph-mon[37330]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:22.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:22 smithi104 ceph-mon[33507]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:23.567 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:23 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T07:23:23.568 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:23 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:23.568 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:23 smithi022 ceph-mon[29462]: Deploying daemon osd.0 on smithi022 2024-09-11T07:23:23.568 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:23 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T07:23:23.568 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:23 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:23.568 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:23 smithi022 ceph-mon[37330]: Deploying daemon osd.0 on smithi022 2024-09-11T07:23:23.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:23 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T07:23:23.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:23 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:23.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:23 smithi104 ceph-mon[33507]: Deploying daemon osd.0 on smithi022 2024-09-11T07:23:24.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:24 smithi022 ceph-mon[29462]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:24.446 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:24 smithi022 ceph-mon[37330]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:24.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:24 smithi104 ceph-mon[33507]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:26.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:26 smithi022 ceph-mon[29462]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:26.657 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:26 smithi022 ceph-mon[37330]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:26.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:26 smithi104 ceph-mon[33507]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:28.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:28 smithi022 ceph-mon[29462]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:28.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:28 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:28.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:28 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:28.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:28 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:28.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:28 smithi022 ceph-mon[37330]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:28.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:28 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:28.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:28 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:28.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:28 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:28.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:28 smithi104 ceph-mon[33507]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:28.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:28 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:28.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:28 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:28.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:28 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:30.617 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:30 smithi022 ceph-mon[29462]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:30.617 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:30 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:30.618 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:30 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:30.618 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:30 smithi022 ceph-mon[37330]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:30.618 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:30 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:30.618 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:30 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:30.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:30 smithi104 ceph-mon[33507]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:30.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:30 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:30.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:30 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:30.933 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 0 on host 'smithi022' 2024-09-11T07:23:31.538 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:31 smithi022 ceph-mon[29462]: from='osd.0 v1:172.21.15.22:6801/2821493591' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:23:31.538 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:31 smithi022 ceph-mon[29462]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:23:31.538 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:31 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:31.538 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:31 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:31.539 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:31 smithi022 ceph-mon[37330]: from='osd.0 v1:172.21.15.22:6801/2821493591' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:23:31.539 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:31 smithi022 ceph-mon[37330]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:23:31.539 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:31 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:31.539 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:31 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:31.562 DEBUG:teuthology.orchestra.run.smithi022:osd.0> sudo journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.0.service 2024-09-11T07:23:31.565 INFO:tasks.cephadm:Deploying osd.1 on smithi022 with /dev/vg_nvme/lv_3... 2024-09-11T07:23:31.565 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-11T07:23:31.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:31 smithi104 ceph-mon[33507]: from='osd.0 v1:172.21.15.22:6801/2821493591' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:23:31.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:31 smithi104 ceph-mon[33507]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:23:31.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:31 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:31.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:31 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:31.862 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:23:32.648 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:32 smithi022 ceph-mon[29462]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:32.648 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:32 smithi022 ceph-mon[29462]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T07:23:32.648 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:32 smithi022 ceph-mon[29462]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T07:23:32.648 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:32 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:23:32.649 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:32 smithi022 ceph-mon[29462]: from='osd.0 v1:172.21.15.22:6801/2821493591' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-11T07:23:32.649 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:32 smithi022 ceph-mon[29462]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-11T07:23:32.649 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:32 smithi022 ceph-mon[37330]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:32.650 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:32 smithi022 ceph-mon[37330]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T07:23:32.650 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:32 smithi022 ceph-mon[37330]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T07:23:32.650 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:32 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:23:32.650 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:32 smithi022 ceph-mon[37330]: from='osd.0 v1:172.21.15.22:6801/2821493591' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-11T07:23:32.650 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:32 smithi022 ceph-mon[37330]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-11T07:23:32.651 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 11 07:23:32 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-0[41188]: 2024-09-11T07:23:32.387+0000 7f948aa4a640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T07:23:32.651 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 11 07:23:32 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-0[41188]: 2024-09-11T07:23:32.389+0000 7f9486883640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:23:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:32 smithi104 ceph-mon[33507]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:32 smithi104 ceph-mon[33507]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T07:23:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:32 smithi104 ceph-mon[33507]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T07:23:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:32 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:23:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:32 smithi104 ceph-mon[33507]: from='osd.0 v1:172.21.15.22:6801/2821493591' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-11T07:23:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:32 smithi104 ceph-mon[33507]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-11T07:23:33.616 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:33 smithi022 ceph-mon[29462]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-09-11T07:23:33.616 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:33 smithi022 ceph-mon[29462]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T07:23:33.616 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:33 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:23:33.616 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:33 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:23:33.617 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:33 smithi022 ceph-mon[37330]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-09-11T07:23:33.617 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:33 smithi022 ceph-mon[37330]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T07:23:33.617 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:33 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:23:33.617 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:33 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:23:33.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:33 smithi104 ceph-mon[33507]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-09-11T07:23:33.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:33 smithi104 ceph-mon[33507]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T07:23:33.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:33 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:23:33.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:33 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:23:34.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:34 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:23:34.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:34 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:23:34.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:34 smithi022 ceph-mon[29462]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:34.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:34 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:23:34.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:34 smithi022 ceph-mon[29462]: osd.0 v1:172.21.15.22:6801/2821493591 boot 2024-09-11T07:23:34.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:34 smithi022 ceph-mon[29462]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T07:23:34.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:34 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:23:34.490 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:34 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:23:34.490 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:34 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:23:34.490 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:34 smithi022 ceph-mon[37330]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:34.490 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:34 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:23:34.490 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:34 smithi022 ceph-mon[37330]: osd.0 v1:172.21.15.22:6801/2821493591 boot 2024-09-11T07:23:34.490 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:34 smithi022 ceph-mon[37330]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T07:23:34.490 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:34 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:23:34.727 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-11T07:23:34.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:34 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:23:34.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:34 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:23:34.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:34 smithi104 ceph-mon[33507]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:23:34.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:34 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:23:34.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:34 smithi104 ceph-mon[33507]: osd.0 v1:172.21.15.22:6801/2821493591 boot 2024-09-11T07:23:34.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:34 smithi104 ceph-mon[33507]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T07:23:34.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:34 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:23:34.820 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-11T07:23:34.831 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2024-09-11T07:23:34.832 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2024-09-11T07:23:34.832 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.009325 s, 1.1 GB/s 2024-09-11T07:23:34.832 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2024-09-11T07:23:35.468 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:35 smithi022 ceph-mon[29462]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T07:23:35.468 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:35 smithi022 ceph-mon[37330]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T07:23:35.470 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi022:vg_nvme/lv_3 2024-09-11T07:23:35.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:35 smithi104 ceph-mon[33507]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T07:23:35.801 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:23:36.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:36 smithi022 ceph-mon[37330]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:36.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:36 smithi022 ceph-mon[29462]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:36.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:36 smithi104 ceph-mon[33507]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:38.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[29462]: Detected new or changed devices on smithi022 2024-09-11T07:23:38.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:38.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:38.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:23:38.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[29462]: Adjusting osd_memory_target on smithi022 to 16068M 2024-09-11T07:23:38.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:38.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:38.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:23:38.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:38.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:38.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[29462]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:38.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:38.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:23:38.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:38.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[37330]: Detected new or changed devices on smithi022 2024-09-11T07:23:38.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:38.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:38.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:23:38.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[37330]: Adjusting osd_memory_target on smithi022 to 16068M 2024-09-11T07:23:38.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:38.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:38.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:23:38.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:38.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:38.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[37330]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:38.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:38.420 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:23:38.420 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:38 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:38 smithi104 ceph-mon[33507]: Detected new or changed devices on smithi022 2024-09-11T07:23:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:38 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:38 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:38 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:23:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:38 smithi104 ceph-mon[33507]: Adjusting osd_memory_target on smithi022 to 16068M 2024-09-11T07:23:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:38 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:38 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:38 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:23:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:38 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:38 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:38.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:38 smithi104 ceph-mon[33507]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:38.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:38 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:38.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:38 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:23:38.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:38 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:39.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:39 smithi022 ceph-mon[29462]: from='client.24143 v1:172.21.15.22:0/424258941' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:23:39.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:39 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:23:39.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:39 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:23:39.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:39 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:39.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:39 smithi022 ceph-mon[37330]: from='client.24143 v1:172.21.15.22:0/424258941' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:23:39.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:39 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:23:39.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:39 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:23:39.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:39 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:39.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:39 smithi104 ceph-mon[33507]: from='client.24143 v1:172.21.15.22:0/424258941' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:23:39.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:39 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:23:39.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:39 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:23:39.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:39 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:40.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:40 smithi022 ceph-mon[29462]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:40.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:40 smithi022 ceph-mon[37330]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:40.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:40 smithi104 ceph-mon[33507]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:41.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:41 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/487488807' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52cc1e06-9506-48c0-9f7b-d4270dff4b9f"}]: dispatch 2024-09-11T07:23:41.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:41 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/487488807' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52cc1e06-9506-48c0-9f7b-d4270dff4b9f"}]': finished 2024-09-11T07:23:41.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:41 smithi022 ceph-mon[29462]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T07:23:41.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:41 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:23:41.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:41 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/487488807' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52cc1e06-9506-48c0-9f7b-d4270dff4b9f"}]: dispatch 2024-09-11T07:23:41.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:41 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/487488807' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52cc1e06-9506-48c0-9f7b-d4270dff4b9f"}]': finished 2024-09-11T07:23:41.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:41 smithi022 ceph-mon[37330]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T07:23:41.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:41 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:23:41.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:41 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/487488807' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52cc1e06-9506-48c0-9f7b-d4270dff4b9f"}]: dispatch 2024-09-11T07:23:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:41 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/487488807' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52cc1e06-9506-48c0-9f7b-d4270dff4b9f"}]': finished 2024-09-11T07:23:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:41 smithi104 ceph-mon[33507]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T07:23:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:41 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:23:42.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:42 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4132020373' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:23:42.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:42 smithi022 ceph-mon[29462]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:42.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:42 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4132020373' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:23:42.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:42 smithi022 ceph-mon[37330]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:42.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:42 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4132020373' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:23:42.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:42 smithi104 ceph-mon[33507]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:44.592 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:44 smithi022 ceph-mon[29462]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:44.592 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:44 smithi022 ceph-mon[37330]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:44.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:44 smithi104 ceph-mon[33507]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:46.654 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:46 smithi022 ceph-mon[37330]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:46.655 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:46 smithi022 ceph-mon[29462]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:46.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:46 smithi104 ceph-mon[33507]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:47.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:47 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T07:23:47.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:47 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:47.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:47 smithi022 ceph-mon[29462]: Deploying daemon osd.1 on smithi022 2024-09-11T07:23:47.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:47 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T07:23:47.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:47 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:47.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:47 smithi022 ceph-mon[37330]: Deploying daemon osd.1 on smithi022 2024-09-11T07:23:47.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:47 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T07:23:47.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:47 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:47.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:47 smithi104 ceph-mon[33507]: Deploying daemon osd.1 on smithi022 2024-09-11T07:23:48.547 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:48 smithi022 ceph-mon[29462]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:48.548 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:48 smithi022 ceph-mon[37330]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:48.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:48 smithi104 ceph-mon[33507]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:50.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:50 smithi022 ceph-mon[29462]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:50.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:50 smithi022 ceph-mon[37330]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:50.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:50 smithi104 ceph-mon[33507]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:52.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:52 smithi022 ceph-mon[37330]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:52.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:52 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:52.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:52 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:52.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:52 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:52.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:52 smithi022 ceph-mon[29462]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:52.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:52 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:52.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:52 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:52.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:52 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:52.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:52 smithi104 ceph-mon[33507]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:52.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:52 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:52.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:52 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:52.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:52 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:54.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:54 smithi022 ceph-mon[29462]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:54.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:54 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:54.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:54 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:54.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:54 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:54.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:54 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:23:54.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:54 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:54.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:54 smithi022 ceph-mon[37330]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:54.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:54 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:54.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:54 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:54.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:54 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:54.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:54 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:23:54.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:54 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:54.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:54 smithi104 ceph-mon[33507]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:54.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:54 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:54.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:54 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:54.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:54 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:54.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:54 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:23:54.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:54 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:54.995 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 1 on host 'smithi022' 2024-09-11T07:23:55.621 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:55 smithi022 ceph-mon[29462]: from='osd.1 v1:172.21.15.22:6805/4037911317' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T07:23:55.622 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:55 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:55.622 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:55 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:55.622 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:55 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:55.622 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:55 smithi022 ceph-mon[37330]: from='osd.1 v1:172.21.15.22:6805/4037911317' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T07:23:55.622 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:55 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:55.622 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:55 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:55.622 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:55 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:55.646 DEBUG:teuthology.orchestra.run.smithi022:osd.1> sudo journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.1.service 2024-09-11T07:23:55.649 INFO:tasks.cephadm:Deploying osd.2 on smithi022 with /dev/vg_nvme/lv_2... 2024-09-11T07:23:55.649 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-11T07:23:55.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:55 smithi104 ceph-mon[33507]: from='osd.1 v1:172.21.15.22:6805/4037911317' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T07:23:55.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:55 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:55.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:55 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:55.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:55 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:23:56.325 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:23:56.653 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:56 smithi022 ceph-mon[29462]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:56.653 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:56 smithi022 ceph-mon[29462]: from='osd.1 v1:172.21.15.22:6805/4037911317' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T07:23:56.653 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:56 smithi022 ceph-mon[29462]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T07:23:56.653 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:56 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:23:56.653 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:56 smithi022 ceph-mon[29462]: from='osd.1 v1:172.21.15.22:6805/4037911317' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-11T07:23:56.654 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:56 smithi022 ceph-mon[37330]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:56.654 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:56 smithi022 ceph-mon[37330]: from='osd.1 v1:172.21.15.22:6805/4037911317' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T07:23:56.654 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:56 smithi022 ceph-mon[37330]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T07:23:56.654 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:56 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:23:56.654 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:56 smithi022 ceph-mon[37330]: from='osd.1 v1:172.21.15.22:6805/4037911317' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-11T07:23:56.654 INFO:journalctl@ceph.osd.1.smithi022.stdout:Sep 11 07:23:56 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-1[47211]: 2024-09-11T07:23:56.388+0000 7fe3d9958640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T07:23:56.654 INFO:journalctl@ceph.osd.1.smithi022.stdout:Sep 11 07:23:56 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-1[47211]: 2024-09-11T07:23:56.392+0000 7fe3d378d640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:23:56.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:56 smithi104 ceph-mon[33507]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:56.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:56 smithi104 ceph-mon[33507]: from='osd.1 v1:172.21.15.22:6805/4037911317' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T07:23:56.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:56 smithi104 ceph-mon[33507]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T07:23:56.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:56 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:23:56.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:56 smithi104 ceph-mon[33507]: from='osd.1 v1:172.21.15.22:6805/4037911317' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-11T07:23:57.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:57 smithi022 ceph-mon[29462]: from='osd.1 v1:172.21.15.22:6805/4037911317' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-09-11T07:23:57.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:57 smithi022 ceph-mon[29462]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T07:23:57.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:57 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:23:57.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:57 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:23:57.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:57 smithi022 ceph-mon[37330]: from='osd.1 v1:172.21.15.22:6805/4037911317' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-09-11T07:23:57.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:57 smithi022 ceph-mon[37330]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T07:23:57.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:57 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:23:57.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:57 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:23:57.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:57 smithi104 ceph-mon[33507]: from='osd.1 v1:172.21.15.22:6805/4037911317' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-09-11T07:23:57.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:57 smithi104 ceph-mon[33507]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T07:23:57.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:57 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:23:57.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:57 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:23:58.664 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:58 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:23:58.665 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:58 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:23:58.665 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:58 smithi022 ceph-mon[29462]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:58.665 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:58 smithi022 ceph-mon[29462]: osd.1 v1:172.21.15.22:6805/4037911317 boot 2024-09-11T07:23:58.665 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:58 smithi022 ceph-mon[29462]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T07:23:58.665 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:58 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:23:58.665 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:58 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:23:58.665 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:58 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:23:58.665 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:58 smithi022 ceph-mon[37330]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:58.665 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:58 smithi022 ceph-mon[37330]: osd.1 v1:172.21.15.22:6805/4037911317 boot 2024-09-11T07:23:58.665 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:58 smithi022 ceph-mon[37330]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T07:23:58.665 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:58 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:23:58.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:58 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:23:58.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:58 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:23:58.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:58 smithi104 ceph-mon[33507]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:23:58.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:58 smithi104 ceph-mon[33507]: osd.1 v1:172.21.15.22:6805/4037911317 boot 2024-09-11T07:23:58.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:58 smithi104 ceph-mon[33507]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T07:23:58.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:58 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:23:58.806 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-11T07:23:58.886 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-11T07:23:58.901 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2024-09-11T07:23:58.901 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2024-09-11T07:23:58.901 INFO:teuthology.orchestra.run.smithi022.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0136435 s, 769 MB/s 2024-09-11T07:23:58.901 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 2024-09-11T07:23:58.901 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2024-09-11T07:23:59.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:23:59 smithi022 ceph-mon[29462]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T07:23:59.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:23:59 smithi022 ceph-mon[37330]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T07:23:59.774 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi022:vg_nvme/lv_2 2024-09-11T07:23:59.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:23:59 smithi104 ceph-mon[33507]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T07:24:00.267 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:24:00.496 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:00 smithi022 ceph-mon[29462]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:00.496 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:00 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:00.496 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:00 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:00.496 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:00 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:00.496 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:00 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:00.496 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:00 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:00.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:00 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:00.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:00 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:24:00.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:00 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:00.497 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:00 smithi022 ceph-mon[37330]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:00.497 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:00 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:00.497 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:00 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:00.497 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:00 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:00.497 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:00 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:00.497 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:00 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:00.497 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:00 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:00.497 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:00 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:24:00.498 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:00 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:00.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:00 smithi104 ceph-mon[33507]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:00.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:00 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:00.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:00 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:00.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:00 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:00.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:00 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:00.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:00 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:00.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:00 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:00.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:00 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:24:00.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:00 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:01.600 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:01 smithi022 ceph-mon[29462]: Detected new or changed devices on smithi022 2024-09-11T07:24:01.600 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:01 smithi022 ceph-mon[29462]: Adjusting osd_memory_target on smithi022 to 8034M 2024-09-11T07:24:01.601 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:01 smithi022 ceph-mon[37330]: Detected new or changed devices on smithi022 2024-09-11T07:24:01.601 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:01 smithi022 ceph-mon[37330]: Adjusting osd_memory_target on smithi022 to 8034M 2024-09-11T07:24:01.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:01 smithi104 ceph-mon[33507]: Detected new or changed devices on smithi022 2024-09-11T07:24:01.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:01 smithi104 ceph-mon[33507]: Adjusting osd_memory_target on smithi022 to 8034M 2024-09-11T07:24:02.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:02 smithi022 ceph-mon[37330]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:02.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:02 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:24:02.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:02 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:24:02.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:02 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:02.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:02 smithi022 ceph-mon[29462]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:02.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:02 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:24:02.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:02 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:24:02.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:02 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:02.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:02 smithi104 ceph-mon[33507]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:02.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:02 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:24:02.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:02 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:24:02.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:02 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:03.644 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:03 smithi022 ceph-mon[29462]: from='client.14271 v1:172.21.15.22:0/1522443011' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:24:03.644 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:03 smithi022 ceph-mon[37330]: from='client.14271 v1:172.21.15.22:0/1522443011' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:24:03.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:03 smithi104 ceph-mon[33507]: from='client.14271 v1:172.21.15.22:0/1522443011' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:24:04.679 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:04 smithi022 ceph-mon[37330]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:04.679 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:04 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3282482398' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa2b2730-2c1d-45d0-be8a-494a60dc6b9b"}]: dispatch 2024-09-11T07:24:04.679 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:04 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3282482398' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa2b2730-2c1d-45d0-be8a-494a60dc6b9b"}]': finished 2024-09-11T07:24:04.679 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:04 smithi022 ceph-mon[37330]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T07:24:04.679 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:04 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:24:04.679 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:04 smithi022 ceph-mon[29462]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:04.680 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:04 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3282482398' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa2b2730-2c1d-45d0-be8a-494a60dc6b9b"}]: dispatch 2024-09-11T07:24:04.680 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:04 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3282482398' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa2b2730-2c1d-45d0-be8a-494a60dc6b9b"}]': finished 2024-09-11T07:24:04.680 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:04 smithi022 ceph-mon[29462]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T07:24:04.680 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:04 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:24:04.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:04 smithi104 ceph-mon[33507]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:04.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:04 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3282482398' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa2b2730-2c1d-45d0-be8a-494a60dc6b9b"}]: dispatch 2024-09-11T07:24:04.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:04 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3282482398' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa2b2730-2c1d-45d0-be8a-494a60dc6b9b"}]': finished 2024-09-11T07:24:04.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:04 smithi104 ceph-mon[33507]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T07:24:04.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:04 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:24:05.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:05 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3775911905' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:24:05.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:05 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3775911905' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:24:05.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:05 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3775911905' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:24:06.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:06 smithi104 ceph-mon[33507]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:06 smithi022 ceph-mon[29462]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:06.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:06 smithi022 ceph-mon[37330]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:08.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:08 smithi104 ceph-mon[33507]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:08.825 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:08 smithi022 ceph-mon[29462]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:08.825 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:08 smithi022 ceph-mon[37330]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:10.515 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:10 smithi022 ceph-mon[29462]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:10.515 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:10 smithi022 ceph-mon[37330]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:10.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:10 smithi104 ceph-mon[33507]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:11.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:11 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T07:24:11.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:11 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:11.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:11 smithi022 ceph-mon[29462]: Deploying daemon osd.2 on smithi022 2024-09-11T07:24:11.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:11 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T07:24:11.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:11 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:11.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:11 smithi022 ceph-mon[37330]: Deploying daemon osd.2 on smithi022 2024-09-11T07:24:11.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:11 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T07:24:11.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:11 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:11.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:11 smithi104 ceph-mon[33507]: Deploying daemon osd.2 on smithi022 2024-09-11T07:24:12.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:12 smithi022 ceph-mon[29462]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:12.483 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:12 smithi022 ceph-mon[37330]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:12.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:12 smithi104 ceph-mon[33507]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:14.609 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:14 smithi022 ceph-mon[29462]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:14.609 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:14 smithi022 ceph-mon[37330]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:14.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:14 smithi104 ceph-mon[33507]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:15.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:15 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:24:15.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:15 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:15.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:15 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:15.826 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:15 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:24:15.827 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:15 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:15.827 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:15 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:15.827 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:15 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:24:15.827 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:15 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:15.827 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:15 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:16.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:16 smithi104 ceph-mon[33507]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:16.834 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:16 smithi022 ceph-mon[29462]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:16.834 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:16 smithi022 ceph-mon[37330]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:18.527 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:18 smithi022 ceph-mon[29462]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:18.528 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:18 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:18.528 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:18 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:18.528 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:18 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:18.528 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:18 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:24:18.528 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:18 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:18.528 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:18 smithi022 ceph-mon[29462]: from='osd.2 v1:172.21.15.22:6809/1675799666' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:24:18.528 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:18 smithi022 ceph-mon[29462]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:24:18.528 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:18 smithi022 ceph-mon[37330]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:18.528 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:18 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:18.529 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:18 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:18.529 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:18 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:18.529 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:18 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:24:18.529 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:18 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:18.529 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:18 smithi022 ceph-mon[37330]: from='osd.2 v1:172.21.15.22:6809/1675799666' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:24:18.529 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:18 smithi022 ceph-mon[37330]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:24:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:18 smithi104 ceph-mon[33507]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:18 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:18 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:18 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:18 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:24:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:18 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:18 smithi104 ceph-mon[33507]: from='osd.2 v1:172.21.15.22:6809/1675799666' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:24:18.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:18 smithi104 ceph-mon[33507]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:24:18.862 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 2 on host 'smithi022' 2024-09-11T07:24:19.624 DEBUG:teuthology.orchestra.run.smithi022:osd.2> sudo journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.2.service 2024-09-11T07:24:19.626 INFO:tasks.cephadm:Deploying osd.3 on smithi022 with /dev/vg_nvme/lv_1... 2024-09-11T07:24:19.627 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-11T07:24:19.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:19 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:24:19.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:19 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:19.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:19 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:19.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:19 smithi104 ceph-mon[33507]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T07:24:19.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:19 smithi104 ceph-mon[33507]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T07:24:19.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:19 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:24:19.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:19 smithi104 ceph-mon[33507]: from='osd.2 v1:172.21.15.22:6809/1675799666' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-11T07:24:19.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:19 smithi104 ceph-mon[33507]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-11T07:24:19.796 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:19 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:24:19.796 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:19 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:19.797 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:19 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:19.797 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:19 smithi022 ceph-mon[29462]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T07:24:19.797 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:19 smithi022 ceph-mon[29462]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T07:24:19.797 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:19 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:24:19.797 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:19 smithi022 ceph-mon[29462]: from='osd.2 v1:172.21.15.22:6809/1675799666' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-11T07:24:19.797 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:19 smithi022 ceph-mon[29462]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-11T07:24:19.797 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:19 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:24:19.797 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:19 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:19.797 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:19 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:19.797 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:19 smithi022 ceph-mon[37330]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T07:24:19.797 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:19 smithi022 ceph-mon[37330]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T07:24:19.798 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:19 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:24:19.798 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:19 smithi022 ceph-mon[37330]: from='osd.2 v1:172.21.15.22:6809/1675799666' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-11T07:24:19.798 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:19 smithi022 ceph-mon[37330]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-11T07:24:20.417 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:24:20 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2[53240]: 2024-09-11T07:24:20.202+0000 7fe7a7a9b640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T07:24:20.417 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:24:20 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2[53240]: 2024-09-11T07:24:20.205+0000 7fe7a30d3640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:24:20.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:20 smithi104 ceph-mon[33507]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:20.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:20 smithi104 ceph-mon[33507]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-09-11T07:24:20.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:20 smithi104 ceph-mon[33507]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T07:24:20.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:20 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:24:20.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:20 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:24:20.853 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:20 smithi022 ceph-mon[37330]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:20.854 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:20 smithi022 ceph-mon[37330]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-09-11T07:24:20.854 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:20 smithi022 ceph-mon[37330]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T07:24:20.854 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:20 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:24:20.854 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:20 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:24:20.854 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:20 smithi022 ceph-mon[29462]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:20.854 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:20 smithi022 ceph-mon[29462]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-09-11T07:24:20.854 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:20 smithi022 ceph-mon[29462]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T07:24:20.854 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:20 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:24:20.854 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:20 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:24:20.860 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:24:22.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:22 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:24:22.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:22 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:24:22.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:22 smithi022 ceph-mon[29462]: osd.2 v1:172.21.15.22:6809/1675799666 boot 2024-09-11T07:24:22.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:22 smithi022 ceph-mon[29462]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T07:24:22.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:22 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:24:22.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:22 smithi022 ceph-mon[29462]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:22.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:22 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T07:24:22.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:22 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:24:22.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:22 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:24:22.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:22 smithi022 ceph-mon[37330]: osd.2 v1:172.21.15.22:6809/1675799666 boot 2024-09-11T07:24:22.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:22 smithi022 ceph-mon[37330]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T07:24:22.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:22 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:24:22.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:22 smithi022 ceph-mon[37330]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:22.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:22 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T07:24:22.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:22 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:24:22.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:22 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:24:22.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:22 smithi104 ceph-mon[33507]: osd.2 v1:172.21.15.22:6809/1675799666 boot 2024-09-11T07:24:22.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:22 smithi104 ceph-mon[33507]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T07:24:22.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:22 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:24:22.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:22 smithi104 ceph-mon[33507]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:24:22.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:22 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T07:24:23.328 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:23 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T07:24:23.328 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:23 smithi022 ceph-mon[29462]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T07:24:23.328 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:23 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:24:23.328 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:23 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T07:24:23.328 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:23 smithi022 ceph-mon[37330]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T07:24:23.328 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:23 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:24:23.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:23 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T07:24:23.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:23 smithi104 ceph-mon[33507]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T07:24:23.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:23 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:24:23.572 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-11T07:24:23.672 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-11T07:24:23.684 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2024-09-11T07:24:23.684 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2024-09-11T07:24:23.684 INFO:teuthology.orchestra.run.smithi022.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0100065 s, 1.0 GB/s 2024-09-11T07:24:23.685 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2024-09-11T07:24:24.247 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 11 07:24:24 smithi022 sudo[56879]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:24:24.247 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 11 07:24:24 smithi022 sudo[56879]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:24:24.247 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 11 07:24:24 smithi022 sudo[56879]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:24:24.247 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 11 07:24:24 smithi022 sudo[56879]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:24:24.247 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:24:24.248 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[37330]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T07:24:24.248 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[37330]: Detected new or changed devices on smithi022 2024-09-11T07:24:24.248 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:24.248 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:24.248 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:24.248 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:24.248 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:24.248 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[37330]: Adjusting osd_memory_target on smithi022 to 5356M 2024-09-11T07:24:24.248 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:24.248 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:24.248 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:24:24.248 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:24.249 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[37330]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:24:24.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:24:24.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[29462]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T07:24:24.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[29462]: Detected new or changed devices on smithi022 2024-09-11T07:24:24.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:24.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:24.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:24.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:24.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:24.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[29462]: Adjusting osd_memory_target on smithi022 to 5356M 2024-09-11T07:24:24.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:24.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:24.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:24:24.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:24.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:24 smithi022 ceph-mon[29462]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:24:24.251 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi022:vg_nvme/lv_1 2024-09-11T07:24:24.514 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:24:24.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:24 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:24:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:24 smithi104 ceph-mon[33507]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T07:24:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:24 smithi104 ceph-mon[33507]: Detected new or changed devices on smithi022 2024-09-11T07:24:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:24 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:24 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:24 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:24 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:24 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:24 smithi104 ceph-mon[33507]: Adjusting osd_memory_target on smithi022 to 5356M 2024-09-11T07:24:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:24 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:24 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:24 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:24:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:24 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:24.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:24 smithi104 ceph-mon[33507]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:24:24.851 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 11 07:24:24 smithi022 sudo[56957]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:24:24.851 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 11 07:24:24 smithi022 sudo[56957]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:24:24.851 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 11 07:24:24 smithi022 sudo[56957]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:24:24.851 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 11 07:24:24 smithi022 sudo[56957]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:24:25.167 INFO:journalctl@ceph.osd.1.smithi022.stdout:Sep 11 07:24:25 smithi022 sudo[56976]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:24:25.167 INFO:journalctl@ceph.osd.1.smithi022.stdout:Sep 11 07:24:25 smithi022 sudo[56976]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:24:25.167 INFO:journalctl@ceph.osd.1.smithi022.stdout:Sep 11 07:24:25 smithi022 sudo[56976]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:24:25.167 INFO:journalctl@ceph.osd.1.smithi022.stdout:Sep 11 07:24:25 smithi022 sudo[56976]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:24:25.511 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:25 smithi022 ceph-mon[29462]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T07:24:25.511 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:25 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:25.512 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:25 smithi022 ceph-mon[37330]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T07:24:25.512 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:25 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:25.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:25 smithi104 ceph-mon[33507]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T07:24:25.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:25 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:25.773 INFO:journalctl@ceph.osd.1.smithi022.stdout:Sep 11 07:24:25 smithi022 sudo[56985]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:24:25.773 INFO:journalctl@ceph.osd.1.smithi022.stdout:Sep 11 07:24:25 smithi022 sudo[56985]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:24:25.773 INFO:journalctl@ceph.osd.1.smithi022.stdout:Sep 11 07:24:25 smithi022 sudo[56985]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:24:25.773 INFO:journalctl@ceph.osd.1.smithi022.stdout:Sep 11 07:24:25 smithi022 sudo[56985]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:24:26.058 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:24:25 smithi022 sudo[57013]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:24:26.058 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:24:25 smithi022 sudo[57013]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:24:26.058 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:24:25 smithi022 sudo[57013]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:24:26.058 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:24:25 smithi022 sudo[57013]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:24:26.340 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:26 smithi022 ceph-mon[29462]: pgmap v78: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:24:26.340 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:26 smithi022 ceph-mon[37330]: pgmap v78: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:24:26.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:26 smithi104 ceph-mon[33507]: pgmap v78: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:24:26.605 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:24:26 smithi022 sudo[57105]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:24:26.606 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:24:26 smithi022 sudo[57105]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:24:26.606 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:24:26 smithi022 sudo[57105]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:24:26.606 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:24:26 smithi022 sudo[57105]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:24:27.110 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:26 smithi022 sudo[57173]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:24:27.110 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:26 smithi022 sudo[57173]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:24:27.111 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:26 smithi022 sudo[57173]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:24:27.111 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:26 smithi022 sudo[57173]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:24:27.643 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:27 smithi022 sudo[57191]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:24:27.643 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:27 smithi022 sudo[57191]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:24:27.643 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:27 smithi022 sudo[57191]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:24:27.643 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:27 smithi022 sudo[57191]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:24:27.643 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:27 smithi022 ceph-mon[29462]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T07:24:27.643 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:27 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:24:27.643 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:27 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:24:27.643 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:27 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:27.644 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:27 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:24:27.644 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:27 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:24:27.644 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:27 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:24:27.644 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:27 smithi022 ceph-mon[37330]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T07:24:27.644 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:27 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:24:27.644 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:27 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:24:27.644 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:27 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:27.644 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:27 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:24:27.644 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:27 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:24:27.644 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:27 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:24:27.653 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:27 smithi104 ceph-mon[33507]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T07:24:27.653 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:27 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:24:27.653 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:27 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:24:27.653 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:27 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:27.653 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:27 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:24:27.653 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:27 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:24:27.653 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:27 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:24:27.653 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:27 smithi104 sudo[35997]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:24:27.653 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:27 smithi104 sudo[35997]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:24:27.654 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:27 smithi104 sudo[35997]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:24:27.654 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:27 smithi104 sudo[35997]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:24:28.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:27 smithi104 sudo[36000]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:24:28.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:27 smithi104 sudo[36000]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:24:28.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:27 smithi104 sudo[36000]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:24:28.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:27 smithi104 sudo[36000]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:24:28.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:28 smithi022 sudo[57236]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:24:28.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:28 smithi022 sudo[57236]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:24:28.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:28 smithi022 sudo[57236]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:24:28.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:28 smithi022 sudo[57236]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:24:29.064 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:28 smithi022 sudo[57269]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:24:29.064 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:28 smithi022 sudo[57269]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:24:29.064 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:28 smithi022 sudo[57269]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:24:29.064 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:28 smithi022 sudo[57269]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:24:29.064 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:28 smithi022 ceph-mon[37330]: from='client.14304 v1:172.21.15.22:0/2646228868' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:24:29.064 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:28 smithi022 ceph-mon[37330]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:24:29.064 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:28 smithi022 ceph-mon[37330]: pgmap v79: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:24:29.064 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:28 smithi022 ceph-mon[37330]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:24:29.064 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:28 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:24:29.065 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:28 smithi022 ceph-mon[37330]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:24:29.065 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:28 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:24:29.065 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:28 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:24:29.065 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:28 smithi022 ceph-mon[37330]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:24:29.065 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:28 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:29.065 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:28 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:24:29.065 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:28 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:24:29.065 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:28 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:24:29.065 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:28 smithi022 ceph-mon[29462]: from='client.14304 v1:172.21.15.22:0/2646228868' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:24:29.065 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:28 smithi022 ceph-mon[29462]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:24:29.065 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:28 smithi022 ceph-mon[29462]: pgmap v79: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:24:29.066 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:28 smithi022 ceph-mon[29462]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:24:29.066 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:28 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:24:29.066 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:28 smithi022 ceph-mon[29462]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:24:29.066 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:28 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:24:29.066 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:28 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:24:29.066 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:28 smithi022 ceph-mon[29462]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:24:29.066 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:28 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:29.066 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:28 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:24:29.066 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:28 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:24:29.066 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:28 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:24:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:28 smithi104 ceph-mon[33507]: from='client.14304 v1:172.21.15.22:0/2646228868' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:24:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:28 smithi104 ceph-mon[33507]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:24:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:28 smithi104 ceph-mon[33507]: pgmap v79: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:24:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:28 smithi104 ceph-mon[33507]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:24:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:28 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:24:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:28 smithi104 ceph-mon[33507]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:24:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:28 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:24:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:28 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:24:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:28 smithi104 ceph-mon[33507]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:24:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:28 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:29.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:28 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:24:29.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:28 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:24:29.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:28 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:24:30.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:29 smithi022 ceph-mon[29462]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:24:30.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:29 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4033578144' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d53d3fc-e401-47df-b70b-4b4b1f4539df"}]: dispatch 2024-09-11T07:24:30.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:29 smithi022 ceph-mon[29462]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:24:30.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:29 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4033578144' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d53d3fc-e401-47df-b70b-4b4b1f4539df"}]': finished 2024-09-11T07:24:30.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:29 smithi022 ceph-mon[29462]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T07:24:30.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:29 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:24:30.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:29 smithi022 ceph-mon[29462]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:24:30.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:29 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2631413093' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:24:30.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:29 smithi022 ceph-mon[37330]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:24:30.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:29 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4033578144' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d53d3fc-e401-47df-b70b-4b4b1f4539df"}]: dispatch 2024-09-11T07:24:30.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:29 smithi022 ceph-mon[37330]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:24:30.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:29 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4033578144' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d53d3fc-e401-47df-b70b-4b4b1f4539df"}]': finished 2024-09-11T07:24:30.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:29 smithi022 ceph-mon[37330]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T07:24:30.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:29 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:24:30.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:29 smithi022 ceph-mon[37330]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:24:30.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:29 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2631413093' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:24:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:29 smithi104 ceph-mon[33507]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:24:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:29 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4033578144' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d53d3fc-e401-47df-b70b-4b4b1f4539df"}]: dispatch 2024-09-11T07:24:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:29 smithi104 ceph-mon[33507]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:24:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:29 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4033578144' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d53d3fc-e401-47df-b70b-4b4b1f4539df"}]': finished 2024-09-11T07:24:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:29 smithi104 ceph-mon[33507]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T07:24:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:29 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:24:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:29 smithi104 ceph-mon[33507]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:24:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:29 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2631413093' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:24:32.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:32 smithi022 ceph-mon[29462]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:24:32.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:32 smithi022 ceph-mon[37330]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:24:32.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:32 smithi104 ceph-mon[33507]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:24:34.650 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:34 smithi022 ceph-mon[29462]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:24:34.650 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:34 smithi022 ceph-mon[37330]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:24:34.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:34 smithi104 ceph-mon[33507]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:24:35.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:35 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T07:24:35.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:35 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:35.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:35 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T07:24:35.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:35 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:35.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:35 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T07:24:35.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:35 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:36.566 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:36 smithi022 ceph-mon[29462]: Deploying daemon osd.3 on smithi022 2024-09-11T07:24:36.566 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:36 smithi022 ceph-mon[29462]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.4 KiB/s rd, 41 KiB/s wr, 3 op/s 2024-09-11T07:24:36.566 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:36 smithi022 ceph-mon[37330]: Deploying daemon osd.3 on smithi022 2024-09-11T07:24:36.567 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:36 smithi022 ceph-mon[37330]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.4 KiB/s rd, 41 KiB/s wr, 3 op/s 2024-09-11T07:24:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:36 smithi104 ceph-mon[33507]: Deploying daemon osd.3 on smithi022 2024-09-11T07:24:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:36 smithi104 ceph-mon[33507]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.4 KiB/s rd, 41 KiB/s wr, 3 op/s 2024-09-11T07:24:38.482 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:38 smithi022 ceph-mon[29462]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.1 KiB/s rd, 31 KiB/s wr, 2 op/s 2024-09-11T07:24:38.482 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:38 smithi022 ceph-mon[37330]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.1 KiB/s rd, 31 KiB/s wr, 2 op/s 2024-09-11T07:24:38.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:38 smithi104 ceph-mon[33507]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.1 KiB/s rd, 31 KiB/s wr, 2 op/s 2024-09-11T07:24:40.602 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:40 smithi022 ceph-mon[37330]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 882 B/s rd, 25 KiB/s wr, 2 op/s 2024-09-11T07:24:40.602 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:40 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:24:40.602 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:40 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:40.602 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:40 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:40.603 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:40 smithi022 ceph-mon[29462]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 882 B/s rd, 25 KiB/s wr, 2 op/s 2024-09-11T07:24:40.603 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:40 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:24:40.603 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:40 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:40.603 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:40 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:40.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:40 smithi104 ceph-mon[33507]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 882 B/s rd, 25 KiB/s wr, 2 op/s 2024-09-11T07:24:40.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:40 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:24:40.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:40 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:40.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:40 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:42.493 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:42 smithi022 ceph-mon[37330]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-11T07:24:42.493 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:42 smithi022 ceph-mon[29462]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-11T07:24:42.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:42 smithi104 ceph-mon[33507]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-11T07:24:43.116 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 3 on host 'smithi022' 2024-09-11T07:24:43.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:43 smithi022 ceph-mon[29462]: from='osd.3 v1:172.21.15.22:6813/1501962752' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T07:24:43.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:43 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:43.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:43 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:43.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:43 smithi022 ceph-mon[37330]: from='osd.3 v1:172.21.15.22:6813/1501962752' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T07:24:43.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:43 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:43.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:43 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:43.747 DEBUG:teuthology.orchestra.run.smithi022:osd.3> sudo journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.3.service 2024-09-11T07:24:43.751 INFO:tasks.cephadm:Deploying osd.4 on smithi104 with /dev/vg_nvme/lv_4... 2024-09-11T07:24:43.751 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-11T07:24:43.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:43 smithi104 ceph-mon[33507]: from='osd.3 v1:172.21.15.22:6813/1501962752' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T07:24:43.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:43 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:43.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:43 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:44.001 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:24:44.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:44 smithi022 ceph-mon[29462]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-11T07:24:44.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:44 smithi022 ceph-mon[29462]: from='osd.3 v1:172.21.15.22:6813/1501962752' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T07:24:44.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:44 smithi022 ceph-mon[29462]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T07:24:44.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:44 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:24:44.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:44 smithi022 ceph-mon[29462]: from='osd.3 v1:172.21.15.22:6813/1501962752' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-11T07:24:44.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:44 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:44.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:44 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:44.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:44 smithi022 ceph-mon[37330]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-11T07:24:44.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:44 smithi022 ceph-mon[37330]: from='osd.3 v1:172.21.15.22:6813/1501962752' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T07:24:44.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:44 smithi022 ceph-mon[37330]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T07:24:44.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:44 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:24:44.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:44 smithi022 ceph-mon[37330]: from='osd.3 v1:172.21.15.22:6813/1501962752' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-11T07:24:44.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:44 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:44.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:44 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:44.669 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 11 07:24:44 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-3[58935]: 2024-09-11T07:24:44.393+0000 7eff35916640 -1 osd.3 0 waiting for initial osdmap 2024-09-11T07:24:44.669 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 11 07:24:44 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-3[58935]: 2024-09-11T07:24:44.399+0000 7eff2d725640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:24:44.738 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:44 smithi104 ceph-mon[33507]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-11T07:24:44.738 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:44 smithi104 ceph-mon[33507]: from='osd.3 v1:172.21.15.22:6813/1501962752' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T07:24:44.738 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:44 smithi104 ceph-mon[33507]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T07:24:44.738 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:44 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:24:44.739 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:44 smithi104 ceph-mon[33507]: from='osd.3 v1:172.21.15.22:6813/1501962752' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-11T07:24:44.739 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:44 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:44.739 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:44 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:45.629 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-11T07:24:45.630 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:45 smithi104 ceph-mon[33507]: from='osd.3 v1:172.21.15.22:6813/1501962752' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-09-11T07:24:45.630 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:45 smithi104 ceph-mon[33507]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T07:24:45.630 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:45 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:24:45.630 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:45 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:24:45.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:45 smithi022 ceph-mon[29462]: from='osd.3 v1:172.21.15.22:6813/1501962752' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-09-11T07:24:45.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:45 smithi022 ceph-mon[29462]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T07:24:45.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:45 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:24:45.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:45 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:24:45.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:45 smithi022 ceph-mon[37330]: from='osd.3 v1:172.21.15.22:6813/1501962752' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-09-11T07:24:45.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:45 smithi022 ceph-mon[37330]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T07:24:45.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:45 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:24:45.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:45 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:24:45.716 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-11T07:24:45.729 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-09-11T07:24:45.730 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-09-11T07:24:45.730 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114783 s, 914 MB/s 2024-09-11T07:24:45.730 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-09-11T07:24:46.311 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_4 2024-09-11T07:24:46.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:46 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:24:46.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:46 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:24:46.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:46 smithi104 ceph-mon[33507]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:24:46.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:46 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:24:46.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:46 smithi104 ceph-mon[33507]: osd.3 v1:172.21.15.22:6813/1501962752 boot 2024-09-11T07:24:46.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:46 smithi104 ceph-mon[33507]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T07:24:46.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:46 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:24:46.597 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:24:46.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:46 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:24:46.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:46 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:24:46.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:46 smithi022 ceph-mon[37330]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:24:46.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:46 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:24:46.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:46 smithi022 ceph-mon[37330]: osd.3 v1:172.21.15.22:6813/1501962752 boot 2024-09-11T07:24:46.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:46 smithi022 ceph-mon[37330]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T07:24:46.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:46 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:24:46.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:46 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:24:46.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:46 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:24:46.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:46 smithi022 ceph-mon[29462]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:24:46.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:46 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:24:46.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:46 smithi022 ceph-mon[29462]: osd.3 v1:172.21.15.22:6813/1501962752 boot 2024-09-11T07:24:46.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:46 smithi022 ceph-mon[29462]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T07:24:46.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:46 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:24:47.492 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:47 smithi104 ceph-mon[33507]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T07:24:47.493 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:47 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:47.493 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:47 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:47.493 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:47 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:47.493 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:47 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:47.493 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:47 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:47.493 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:47 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:47.493 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:47 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:47.493 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:47 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:47.493 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:47 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:24:47.493 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:47 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:47.493 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:47 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:24:47.493 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:47 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:47.494 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:47 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:24:47.494 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:47 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:47.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[29462]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T07:24:47.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:47.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:47.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:47.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:47.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:47.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:47.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:47.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:47.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:24:47.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:47.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:24:47.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:47.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:24:47.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:47.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[37330]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T07:24:47.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:47.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:47.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:47.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:47.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:47.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:24:47.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:47.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:47.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:24:47.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:47.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:24:47.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:47.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:24:47.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:47 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:24:48.658 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:48 smithi104 ceph-mon[33507]: Detected new or changed devices on smithi022 2024-09-11T07:24:48.658 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:48 smithi104 ceph-mon[33507]: Adjusting osd_memory_target on smithi022 to 4017M 2024-09-11T07:24:48.658 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:48 smithi104 ceph-mon[33507]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:24:48.658 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:48 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:24:48.658 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:48 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:24:48.658 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:48 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:48.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:48 smithi022 ceph-mon[29462]: Detected new or changed devices on smithi022 2024-09-11T07:24:48.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:48 smithi022 ceph-mon[29462]: Adjusting osd_memory_target on smithi022 to 4017M 2024-09-11T07:24:48.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:48 smithi022 ceph-mon[29462]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:24:48.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:48 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:24:48.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:48 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:24:48.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:48 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:48.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:48 smithi022 ceph-mon[37330]: Detected new or changed devices on smithi022 2024-09-11T07:24:48.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:48 smithi022 ceph-mon[37330]: Adjusting osd_memory_target on smithi022 to 4017M 2024-09-11T07:24:48.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:48 smithi022 ceph-mon[37330]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:24:48.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:48 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:24:48.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:48 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:24:48.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:48 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:49.521 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:49 smithi104 ceph-mon[33507]: from='client.24226 v1:172.21.15.104:0/1438441293' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:24:49.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:49 smithi022 ceph-mon[29462]: from='client.24226 v1:172.21.15.104:0/1438441293' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:24:49.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:49 smithi022 ceph-mon[37330]: from='client.24226 v1:172.21.15.104:0/1438441293' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:24:50.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:50 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:24:50.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:50 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:24:50.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:50 smithi022 ceph-mon[29462]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:24:50.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:50 smithi022 ceph-mon[29462]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T07:24:50.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:50 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.104:0/3776268085' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3207fe0-8b97-4067-a755-ba6d449c53c7"}]: dispatch 2024-09-11T07:24:50.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:50 smithi022 ceph-mon[29462]: from='client.24232 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3207fe0-8b97-4067-a755-ba6d449c53c7"}]: dispatch 2024-09-11T07:24:50.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:50 smithi022 ceph-mon[29462]: from='client.24232 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3207fe0-8b97-4067-a755-ba6d449c53c7"}]': finished 2024-09-11T07:24:50.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:50 smithi022 ceph-mon[29462]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T07:24:50.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:50 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:24:50.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:50 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:24:50.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:50 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:24:50.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:50 smithi022 ceph-mon[37330]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:24:50.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:50 smithi022 ceph-mon[37330]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T07:24:50.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:50 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.104:0/3776268085' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3207fe0-8b97-4067-a755-ba6d449c53c7"}]: dispatch 2024-09-11T07:24:50.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:50 smithi022 ceph-mon[37330]: from='client.24232 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3207fe0-8b97-4067-a755-ba6d449c53c7"}]: dispatch 2024-09-11T07:24:50.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:50 smithi022 ceph-mon[37330]: from='client.24232 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3207fe0-8b97-4067-a755-ba6d449c53c7"}]': finished 2024-09-11T07:24:50.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:50 smithi022 ceph-mon[37330]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T07:24:50.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:50 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:24:50.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:50 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:24:50.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:50 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:24:50.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:50 smithi104 ceph-mon[33507]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:24:50.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:50 smithi104 ceph-mon[33507]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T07:24:50.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:50 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.104:0/3776268085' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3207fe0-8b97-4067-a755-ba6d449c53c7"}]: dispatch 2024-09-11T07:24:50.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:50 smithi104 ceph-mon[33507]: from='client.24232 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3207fe0-8b97-4067-a755-ba6d449c53c7"}]: dispatch 2024-09-11T07:24:50.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:50 smithi104 ceph-mon[33507]: from='client.24232 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3207fe0-8b97-4067-a755-ba6d449c53c7"}]': finished 2024-09-11T07:24:50.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:50 smithi104 ceph-mon[33507]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T07:24:50.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:50 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:24:51.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:51 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.104:0/3364502940' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:24:51.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:51 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.104:0/3364502940' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:24:51.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:51 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.104:0/3364502940' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:24:52.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:52 smithi022 ceph-mon[29462]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:24:52.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:52 smithi022 ceph-mon[37330]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:24:52.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:52 smithi104 ceph-mon[33507]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:24:54.639 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:54 smithi104 ceph-mon[33507]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:24:54.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:54 smithi022 ceph-mon[29462]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:24:54.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:54 smithi022 ceph-mon[37330]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:24:56.731 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:56 smithi104 ceph-mon[33507]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:24:56.731 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:56 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T07:24:56.731 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:56 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:56.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:56 smithi022 ceph-mon[29462]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:24:56.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:56 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T07:24:56.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:56 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:56.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:56 smithi022 ceph-mon[37330]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:24:56.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:56 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T07:24:56.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:56 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:24:57.490 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:57 smithi104 ceph-mon[33507]: Deploying daemon osd.4 on smithi104 2024-09-11T07:24:57.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:57 smithi022 ceph-mon[29462]: Deploying daemon osd.4 on smithi104 2024-09-11T07:24:57.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:57 smithi022 ceph-mon[37330]: Deploying daemon osd.4 on smithi104 2024-09-11T07:24:58.646 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:24:58 smithi104 ceph-mon[33507]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:24:58.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:24:58 smithi022 ceph-mon[29462]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:24:58.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:24:58 smithi022 ceph-mon[37330]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:25:00.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:00 smithi104 ceph-mon[33507]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:25:00.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:00 smithi022 ceph-mon[29462]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:25:00.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:00 smithi022 ceph-mon[37330]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:25:01.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:01 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:01.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:01 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:01.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:01 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:01.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:01 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:01.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:01 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:01.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:01 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:01.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:01 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:01.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:01 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:01.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:01 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:02.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:02 smithi022 ceph-mon[29462]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:25:02.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:02 smithi022 ceph-mon[37330]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:25:03.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:02 smithi104 ceph-mon[33507]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:25:03.380 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 4 on host 'smithi104' 2024-09-11T07:25:03.982 DEBUG:teuthology.orchestra.run.smithi104:osd.4> sudo journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.4.service 2024-09-11T07:25:03.984 INFO:tasks.cephadm:Deploying osd.5 on smithi104 with /dev/vg_nvme/lv_3... 2024-09-11T07:25:03.985 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-11T07:25:04.447 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:25:04.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:04 smithi022 ceph-mon[29462]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:25:04.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:04 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:04.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:04 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:04.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:04 smithi022 ceph-mon[29462]: from='osd.4 v1:172.21.15.104:6800/1210229443' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:25:04.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:04 smithi022 ceph-mon[29462]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:25:04.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:04 smithi022 ceph-mon[37330]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:25:04.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:04 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:04.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:04 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:04.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:04 smithi022 ceph-mon[37330]: from='osd.4 v1:172.21.15.104:6800/1210229443' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:25:04.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:04 smithi022 ceph-mon[37330]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:25:04.718 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:04 smithi104 ceph-mon[33507]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:25:04.718 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:04 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:04.718 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:04 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:04.718 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:04 smithi104 ceph-mon[33507]: from='osd.4 v1:172.21.15.104:6800/1210229443' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:25:04.718 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:04 smithi104 ceph-mon[33507]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:25:05.628 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:05 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:05.628 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:05 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:05.628 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:05 smithi104 ceph-mon[33507]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T07:25:05.628 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:05 smithi104 ceph-mon[33507]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T07:25:05.628 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:05 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:25:05.628 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:05 smithi104 ceph-mon[33507]: from='osd.4 v1:172.21.15.104:6800/1210229443' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:25:05.628 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:05 smithi104 ceph-mon[33507]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:25:05.903 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 11 07:25:05 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-4[38402]: 2024-09-11T07:25:05.638+0000 7fadc7acc640 -1 osd.4 0 waiting for initial osdmap 2024-09-11T07:25:05.903 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 11 07:25:05 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-4[38402]: 2024-09-11T07:25:05.642+0000 7fadc1901640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:25:05.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:05 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:05.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:05 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:05.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:05 smithi022 ceph-mon[29462]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T07:25:05.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:05 smithi022 ceph-mon[29462]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T07:25:05.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:05 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:25:05.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:05 smithi022 ceph-mon[29462]: from='osd.4 v1:172.21.15.104:6800/1210229443' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:25:05.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:05 smithi022 ceph-mon[29462]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:25:05.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:05 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:05.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:05 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:05.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:05 smithi022 ceph-mon[37330]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T07:25:05.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:05 smithi022 ceph-mon[37330]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T07:25:05.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:05 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:25:05.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:05 smithi022 ceph-mon[37330]: from='osd.4 v1:172.21.15.104:6800/1210229443' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:25:05.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:05 smithi022 ceph-mon[37330]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:25:06.034 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-11T07:25:06.132 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-11T07:25:06.147 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-09-11T07:25:06.147 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-09-11T07:25:06.147 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128111 s, 818 MB/s 2024-09-11T07:25:06.147 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-09-11T07:25:06.764 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_3 2024-09-11T07:25:06.883 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:06 smithi104 ceph-mon[33507]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:25:06.883 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:06 smithi104 ceph-mon[33507]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-11T07:25:06.883 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:06 smithi104 ceph-mon[33507]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T07:25:06.883 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:06 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:25:06.883 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:06 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:25:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:06 smithi022 ceph-mon[29462]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:25:06.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:06 smithi022 ceph-mon[29462]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-11T07:25:06.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:06 smithi022 ceph-mon[29462]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T07:25:06.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:06 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:25:06.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:06 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:25:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:06 smithi022 ceph-mon[37330]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:25:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:06 smithi022 ceph-mon[37330]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-11T07:25:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:06 smithi022 ceph-mon[37330]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T07:25:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:06 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:25:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:06 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:25:07.238 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:25:07.894 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:07 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:25:07.894 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:07 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:25:07.894 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:07 smithi104 ceph-mon[33507]: osd.4 v1:172.21.15.104:6800/1210229443 boot 2024-09-11T07:25:07.895 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:07 smithi104 ceph-mon[33507]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T07:25:07.895 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:07 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:25:07.895 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:07 smithi104 ceph-mon[33507]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T07:25:07.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:07 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:25:07.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:07 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:25:07.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:07 smithi022 ceph-mon[29462]: osd.4 v1:172.21.15.104:6800/1210229443 boot 2024-09-11T07:25:07.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:07 smithi022 ceph-mon[29462]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T07:25:07.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:07 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:25:07.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:07 smithi022 ceph-mon[29462]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T07:25:07.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:07 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:25:07.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:07 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:25:07.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:07 smithi022 ceph-mon[37330]: osd.4 v1:172.21.15.104:6800/1210229443 boot 2024-09-11T07:25:07.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:07 smithi022 ceph-mon[37330]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T07:25:07.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:07 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:25:07.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:07 smithi022 ceph-mon[37330]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T07:25:08.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:08 smithi022 ceph-mon[29462]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:25:08.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:08 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:25:08.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:08 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:25:08.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:08 smithi022 ceph-mon[29462]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T07:25:08.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:08 smithi022 ceph-mon[37330]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:25:08.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:08 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:25:08.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:08 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:25:08.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:08 smithi022 ceph-mon[37330]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T07:25:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:08 smithi104 ceph-mon[33507]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:25:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:08 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:25:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:08 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:25:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:08 smithi104 ceph-mon[33507]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T07:25:10.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[29462]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:25:10.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[29462]: Detected new or changed devices on smithi104 2024-09-11T07:25:10.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:10.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:10.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:25:10.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[29462]: Adjusting osd_memory_target on smithi104 to 17088M 2024-09-11T07:25:10.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:10.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:10.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:25:10.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:10.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:10.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:10.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:25:10.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:10.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[37330]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:25:10.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[37330]: Detected new or changed devices on smithi104 2024-09-11T07:25:10.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:10.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:10.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:25:10.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[37330]: Adjusting osd_memory_target on smithi104 to 17088M 2024-09-11T07:25:10.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:10.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:10.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:25:10.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:10.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:10.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:10.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:25:10.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:10 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:10.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:10 smithi104 ceph-mon[33507]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:25:10.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:10 smithi104 ceph-mon[33507]: Detected new or changed devices on smithi104 2024-09-11T07:25:10.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:10 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:10.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:10 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:10.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:10 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:25:10.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:10 smithi104 ceph-mon[33507]: Adjusting osd_memory_target on smithi104 to 17088M 2024-09-11T07:25:10.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:10 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:10.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:10 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:10.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:10 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:25:10.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:10 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:10.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:10 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:10.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:10 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:10.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:10 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:25:10.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:10 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:11.507 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:11 smithi104 ceph-mon[33507]: from='client.24253 v1:172.21.15.104:0/3532696122' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:25:11.507 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:11 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:25:11.507 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:11 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:25:11.507 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:11 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:11.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:11 smithi022 ceph-mon[29462]: from='client.24253 v1:172.21.15.104:0/3532696122' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:25:11.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:11 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:25:11.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:11 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:25:11.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:11 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:11.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:11 smithi022 ceph-mon[37330]: from='client.24253 v1:172.21.15.104:0/3532696122' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:25:11.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:11 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:25:11.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:11 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:25:11.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:11 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:12.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:12 smithi022 ceph-mon[29462]: pgmap v113: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:25:12.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:12 smithi022 ceph-mon[37330]: pgmap v113: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:25:12.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:12 smithi104 ceph-mon[33507]: pgmap v113: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:25:13.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:13 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.104:0/1717747870' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89c9352b-30b2-48c6-869e-047d07a23a43"}]: dispatch 2024-09-11T07:25:13.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:13 smithi022 ceph-mon[29462]: from='client.24259 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89c9352b-30b2-48c6-869e-047d07a23a43"}]: dispatch 2024-09-11T07:25:13.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:13 smithi022 ceph-mon[29462]: from='client.24259 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89c9352b-30b2-48c6-869e-047d07a23a43"}]': finished 2024-09-11T07:25:13.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:13 smithi022 ceph-mon[29462]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T07:25:13.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:13 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:25:13.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:13 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.104:0/1416535637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:25:13.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:13 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.104:0/1717747870' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89c9352b-30b2-48c6-869e-047d07a23a43"}]: dispatch 2024-09-11T07:25:13.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:13 smithi022 ceph-mon[37330]: from='client.24259 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89c9352b-30b2-48c6-869e-047d07a23a43"}]: dispatch 2024-09-11T07:25:13.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:13 smithi022 ceph-mon[37330]: from='client.24259 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89c9352b-30b2-48c6-869e-047d07a23a43"}]': finished 2024-09-11T07:25:13.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:13 smithi022 ceph-mon[37330]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T07:25:13.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:13 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:25:13.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:13 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.104:0/1416535637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:25:13.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:13 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.104:0/1717747870' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89c9352b-30b2-48c6-869e-047d07a23a43"}]: dispatch 2024-09-11T07:25:13.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:13 smithi104 ceph-mon[33507]: from='client.24259 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89c9352b-30b2-48c6-869e-047d07a23a43"}]: dispatch 2024-09-11T07:25:13.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:13 smithi104 ceph-mon[33507]: from='client.24259 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89c9352b-30b2-48c6-869e-047d07a23a43"}]': finished 2024-09-11T07:25:13.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:13 smithi104 ceph-mon[33507]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T07:25:13.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:13 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:25:13.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:13 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.104:0/1416535637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:25:14.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:14 smithi022 ceph-mon[29462]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:25:14.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:14 smithi022 ceph-mon[37330]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:25:14.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:14 smithi104 ceph-mon[33507]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:25:16.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:16 smithi104 ceph-mon[33507]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-11T07:25:16.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:16 smithi022 ceph-mon[29462]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-11T07:25:16.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:16 smithi022 ceph-mon[37330]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-11T07:25:18.594 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:18 smithi104 ceph-mon[33507]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-11T07:25:18.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:18 smithi022 ceph-mon[29462]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-11T07:25:18.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:18 smithi022 ceph-mon[37330]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-11T07:25:19.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:19 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T07:25:19.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:19 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:19.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:19 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T07:25:19.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:19 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:19.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:19 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T07:25:19.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:19 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:20.518 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:20 smithi104 ceph-mon[33507]: Deploying daemon osd.5 on smithi104 2024-09-11T07:25:20.518 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:20 smithi104 ceph-mon[33507]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:25:20.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:20 smithi022 ceph-mon[29462]: Deploying daemon osd.5 on smithi104 2024-09-11T07:25:20.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:20 smithi022 ceph-mon[29462]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:25:20.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:20 smithi022 ceph-mon[37330]: Deploying daemon osd.5 on smithi104 2024-09-11T07:25:20.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:20 smithi022 ceph-mon[37330]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:25:22.472 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:22 smithi104 ceph-mon[33507]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:25:22.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:22 smithi022 ceph-mon[29462]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:25:22.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:22 smithi022 ceph-mon[37330]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:25:24.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:24 smithi022 ceph-mon[29462]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-11T07:25:24.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:24 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:24.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:24 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:24.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:24 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:24.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:24 smithi022 ceph-mon[37330]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-11T07:25:24.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:24 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:24.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:24 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:24.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:24 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:24.745 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:24 smithi104 ceph-mon[33507]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-11T07:25:24.746 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:24 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:24.746 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:24 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:24.746 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:24 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:26.629 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:26 smithi104 ceph-mon[33507]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T07:25:26.664 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 5 on host 'smithi104' 2024-09-11T07:25:26.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:26 smithi022 ceph-mon[29462]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T07:25:26.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:26 smithi022 ceph-mon[37330]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T07:25:27.282 DEBUG:teuthology.orchestra.run.smithi104:osd.5> sudo journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.5.service 2024-09-11T07:25:27.285 INFO:tasks.cephadm:Deploying osd.6 on smithi104 with /dev/vg_nvme/lv_2... 2024-09-11T07:25:27.285 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-11T07:25:27.575 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:25:27.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:27 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:27.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:27 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:27.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:27 smithi022 ceph-mon[29462]: from='osd.5 v1:172.21.15.104:6804/2735328732' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:25:27.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:27 smithi022 ceph-mon[29462]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:25:27.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:27 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:27.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:27 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:27.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:27 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:27.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:27 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:27.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:27 smithi022 ceph-mon[37330]: from='osd.5 v1:172.21.15.104:6804/2735328732' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:25:27.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:27 smithi022 ceph-mon[37330]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:25:27.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:27 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:27.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:27 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:27.953 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:27 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:27.953 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:27 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:27.953 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:27 smithi104 ceph-mon[33507]: from='osd.5 v1:172.21.15.104:6804/2735328732' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:25:27.953 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:27 smithi104 ceph-mon[33507]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:25:27.953 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:27 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:27.953 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:27 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:28.882 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:28 smithi104 ceph-mon[33507]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:25:28.882 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:28 smithi104 ceph-mon[33507]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T07:25:28.882 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:28 smithi104 ceph-mon[33507]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T07:25:28.882 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:28 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:25:28.883 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:28 smithi104 ceph-mon[33507]: from='osd.5 v1:172.21.15.104:6804/2735328732' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:25:28.883 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:28 smithi104 ceph-mon[33507]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:25:28.883 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:25:28 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:25:28.680+0000 7fafea996640 -1 osd.5 0 waiting for initial osdmap 2024-09-11T07:25:28.883 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:25:28 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:25:28.685+0000 7fafe2fa6640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:25:29.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:28 smithi022 ceph-mon[29462]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:25:29.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:28 smithi022 ceph-mon[29462]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T07:25:29.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:28 smithi022 ceph-mon[29462]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T07:25:29.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:28 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:25:29.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:28 smithi022 ceph-mon[29462]: from='osd.5 v1:172.21.15.104:6804/2735328732' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:25:29.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:28 smithi022 ceph-mon[29462]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:25:29.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:28 smithi022 ceph-mon[37330]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:25:29.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:28 smithi022 ceph-mon[37330]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T07:25:29.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:28 smithi022 ceph-mon[37330]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T07:25:29.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:28 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:25:29.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:28 smithi022 ceph-mon[37330]: from='osd.5 v1:172.21.15.104:6804/2735328732' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:25:29.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:28 smithi022 ceph-mon[37330]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:25:29.225 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-11T07:25:29.320 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-11T07:25:29.338 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-09-11T07:25:29.338 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-09-11T07:25:29.338 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0156072 s, 672 MB/s 2024-09-11T07:25:29.338 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-09-11T07:25:29.953 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:29 smithi104 ceph-mon[33507]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-11T07:25:29.953 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:29 smithi104 ceph-mon[33507]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T07:25:29.953 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:29 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:25:29.953 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:29 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:25:29.954 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_2 2024-09-11T07:25:30.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:29 smithi022 ceph-mon[29462]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-11T07:25:30.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:29 smithi022 ceph-mon[29462]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T07:25:30.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:29 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:25:30.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:29 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:25:30.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:29 smithi022 ceph-mon[37330]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-11T07:25:30.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:29 smithi022 ceph-mon[37330]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T07:25:30.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:29 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:25:30.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:29 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:25:30.810 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:30 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:25:30.810 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:30 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:25:30.810 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:30 smithi104 ceph-mon[33507]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:25:30.810 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:30 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:25:30.810 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:30 smithi104 ceph-mon[33507]: osd.5 v1:172.21.15.104:6804/2735328732 boot 2024-09-11T07:25:30.810 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:30 smithi104 ceph-mon[33507]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T07:25:30.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:30 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:25:30.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:30 smithi104 ceph-mon[33507]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T07:25:31.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:30 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:25:31.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:30 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:25:31.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:30 smithi022 ceph-mon[29462]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:25:31.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:30 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:25:31.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:30 smithi022 ceph-mon[29462]: osd.5 v1:172.21.15.104:6804/2735328732 boot 2024-09-11T07:25:31.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:30 smithi022 ceph-mon[29462]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T07:25:31.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:30 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:25:31.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:30 smithi022 ceph-mon[29462]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T07:25:31.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:30 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:25:31.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:30 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:25:31.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:30 smithi022 ceph-mon[37330]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:25:31.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:30 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:25:31.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:30 smithi022 ceph-mon[37330]: osd.5 v1:172.21.15.104:6804/2735328732 boot 2024-09-11T07:25:31.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:30 smithi022 ceph-mon[37330]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T07:25:31.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:30 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:25:31.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:30 smithi022 ceph-mon[37330]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T07:25:31.460 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:25:32.628 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:32 smithi104 ceph-mon[33507]: pgmap v128: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:32.628 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:32 smithi104 ceph-mon[33507]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T07:25:32.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:32 smithi022 ceph-mon[29462]: pgmap v128: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:32.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:32 smithi022 ceph-mon[29462]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T07:25:32.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:32 smithi022 ceph-mon[37330]: pgmap v128: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:32.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:32 smithi022 ceph-mon[37330]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T07:25:33.465 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:33 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:25:33.465 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:33 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:25:33.465 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:33 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:33.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:33 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:25:33.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:33 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:25:33.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:33 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:33.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:33 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:25:33.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:33 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:25:33.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:33 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:34.516 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:34 smithi104 ceph-mon[33507]: from='client.24280 v1:172.21.15.104:0/3724782631' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:25:34.517 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:34 smithi104 ceph-mon[33507]: pgmap v130: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:34.517 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:34 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:25:34.517 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:34 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:25:34.517 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:34 smithi104 ceph-mon[33507]: osdmap e40: 6 total, 6 up, 6 in 2024-09-11T07:25:34.517 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:34 smithi104 ceph-mon[33507]: Detected new or changed devices on smithi104 2024-09-11T07:25:34.517 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:34 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:34.517 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:34 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:34.517 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:34 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:25:34.517 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:34 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:25:34.518 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:34 smithi104 ceph-mon[33507]: Adjusting osd_memory_target on smithi104 to 8544M 2024-09-11T07:25:34.518 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:34 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:34.518 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:34 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:34.518 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:34 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:25:34.518 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:34 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:34.518 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:34 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:34.518 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:34 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:34.518 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:34 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:25:34.518 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:34 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:34.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[37330]: from='client.24280 v1:172.21.15.104:0/3724782631' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:25:34.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[37330]: pgmap v130: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:34.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:25:34.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:25:34.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[37330]: osdmap e40: 6 total, 6 up, 6 in 2024-09-11T07:25:34.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[37330]: Detected new or changed devices on smithi104 2024-09-11T07:25:34.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:34.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:34.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:25:34.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:25:34.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[37330]: Adjusting osd_memory_target on smithi104 to 8544M 2024-09-11T07:25:34.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:34.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:34.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:25:34.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:34.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:34.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:34.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:25:34.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:34.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[29462]: from='client.24280 v1:172.21.15.104:0/3724782631' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:25:34.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[29462]: pgmap v130: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:34.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:25:34.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:25:34.670 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[29462]: osdmap e40: 6 total, 6 up, 6 in 2024-09-11T07:25:34.670 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[29462]: Detected new or changed devices on smithi104 2024-09-11T07:25:34.670 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:34.670 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:34.670 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:25:34.670 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:25:34.670 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[29462]: Adjusting osd_memory_target on smithi104 to 8544M 2024-09-11T07:25:34.670 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:34.670 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:34.670 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:25:34.670 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:34.670 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:34.671 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:34.671 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:25:34.671 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:34 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:36.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:36 smithi022 ceph-mon[37330]: pgmap v132: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:36.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:36 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.104:0/1722519520' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "704563de-d88f-480a-b469-720135704d42"}]: dispatch 2024-09-11T07:25:36.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:36 smithi022 ceph-mon[37330]: from='client.24286 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "704563de-d88f-480a-b469-720135704d42"}]: dispatch 2024-09-11T07:25:36.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:36 smithi022 ceph-mon[37330]: from='client.24286 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "704563de-d88f-480a-b469-720135704d42"}]': finished 2024-09-11T07:25:36.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:36 smithi022 ceph-mon[37330]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T07:25:36.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:36 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:25:36.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:36 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.104:0/857721039' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:25:36.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:36 smithi022 ceph-mon[29462]: pgmap v132: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:36.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:36 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.104:0/1722519520' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "704563de-d88f-480a-b469-720135704d42"}]: dispatch 2024-09-11T07:25:36.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:36 smithi022 ceph-mon[29462]: from='client.24286 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "704563de-d88f-480a-b469-720135704d42"}]: dispatch 2024-09-11T07:25:36.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:36 smithi022 ceph-mon[29462]: from='client.24286 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "704563de-d88f-480a-b469-720135704d42"}]': finished 2024-09-11T07:25:36.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:36 smithi022 ceph-mon[29462]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T07:25:36.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:36 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:25:36.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:36 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.104:0/857721039' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:25:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:36 smithi104 ceph-mon[33507]: pgmap v132: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:36 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.104:0/1722519520' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "704563de-d88f-480a-b469-720135704d42"}]: dispatch 2024-09-11T07:25:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:36 smithi104 ceph-mon[33507]: from='client.24286 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "704563de-d88f-480a-b469-720135704d42"}]: dispatch 2024-09-11T07:25:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:36 smithi104 ceph-mon[33507]: from='client.24286 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "704563de-d88f-480a-b469-720135704d42"}]': finished 2024-09-11T07:25:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:36 smithi104 ceph-mon[33507]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T07:25:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:36 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:25:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:36 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.104:0/857721039' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:25:38.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:38 smithi104 ceph-mon[33507]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:38 smithi104 ceph-mon[33507]: 1.0 scrub starts 2024-09-11T07:25:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:38 smithi104 ceph-mon[33507]: 1.0 scrub ok 2024-09-11T07:25:38.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:38 smithi022 ceph-mon[29462]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:38.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:38 smithi022 ceph-mon[29462]: 1.0 scrub starts 2024-09-11T07:25:38.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:38 smithi022 ceph-mon[29462]: 1.0 scrub ok 2024-09-11T07:25:38.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:38 smithi022 ceph-mon[37330]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:38.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:38 smithi022 ceph-mon[37330]: 1.0 scrub starts 2024-09-11T07:25:38.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:38 smithi022 ceph-mon[37330]: 1.0 scrub ok 2024-09-11T07:25:40.514 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:40 smithi104 ceph-mon[33507]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:40.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:40 smithi022 ceph-mon[29462]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:40.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:40 smithi022 ceph-mon[37330]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:42.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:42 smithi022 ceph-mon[29462]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:42.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:42 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T07:25:42.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:42 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:42.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:42 smithi022 ceph-mon[29462]: Deploying daemon osd.6 on smithi104 2024-09-11T07:25:42.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:42 smithi022 ceph-mon[37330]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:42.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:42 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T07:25:42.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:42 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:42.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:42 smithi022 ceph-mon[37330]: Deploying daemon osd.6 on smithi104 2024-09-11T07:25:42.683 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:42 smithi104 ceph-mon[33507]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:42.684 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:42 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T07:25:42.684 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:42 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:42.684 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:42 smithi104 ceph-mon[33507]: Deploying daemon osd.6 on smithi104 2024-09-11T07:25:44.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:44 smithi022 ceph-mon[29462]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:44.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:44 smithi022 ceph-mon[37330]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:44.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:44 smithi104 ceph-mon[33507]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:46.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:46 smithi022 ceph-mon[29462]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:46.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:46 smithi022 ceph-mon[37330]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:46.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:46 smithi104 ceph-mon[33507]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:47.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:47 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:47.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:47 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:47.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:47 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:47.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:47 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:47.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:47 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:47.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:47 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:47.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:47 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:47.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:47 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:47.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:47 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:48.828 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:48 smithi104 ceph-mon[33507]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:48.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:48 smithi022 ceph-mon[29462]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:48.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:48 smithi022 ceph-mon[37330]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:49.651 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 6 on host 'smithi104' 2024-09-11T07:25:50.318 DEBUG:teuthology.orchestra.run.smithi104:osd.6> sudo journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.6.service 2024-09-11T07:25:50.320 INFO:tasks.cephadm:Deploying osd.7 on smithi104 with /dev/vg_nvme/lv_1... 2024-09-11T07:25:50.321 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-11T07:25:50.638 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:25:50.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:50 smithi022 ceph-mon[29462]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:50.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:50 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:50.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:50 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:50.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:50 smithi022 ceph-mon[29462]: from='osd.6 v1:172.21.15.104:6808/721493010' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:25:50.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:50 smithi022 ceph-mon[29462]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:25:50.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:50 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:50.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:50 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:50.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:50 smithi022 ceph-mon[37330]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:50.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:50 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:50.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:50 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:50.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:50 smithi022 ceph-mon[37330]: from='osd.6 v1:172.21.15.104:6808/721493010' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:25:50.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:50 smithi022 ceph-mon[37330]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:25:50.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:50 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:50.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:50 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:50.919 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:50 smithi104 ceph-mon[33507]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:50.919 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:50 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:50.919 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:50 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:50.919 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:50 smithi104 ceph-mon[33507]: from='osd.6 v1:172.21.15.104:6808/721493010' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:25:50.919 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:50 smithi104 ceph-mon[33507]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:25:50.919 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:50 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:50.919 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:50 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:51.670 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:51 smithi104 ceph-mon[33507]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T07:25:51.671 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:51 smithi104 ceph-mon[33507]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T07:25:51.671 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:51 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:25:51.671 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:51 smithi104 ceph-mon[33507]: from='osd.6 v1:172.21.15.104:6808/721493010' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:25:51.671 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:51 smithi104 ceph-mon[33507]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:25:51.671 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:25:51 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:25:51.661+0000 7f6563c2d640 -1 osd.6 0 waiting for initial osdmap 2024-09-11T07:25:51.671 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:25:51 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:25:51.668+0000 7f655ca60640 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:25:51.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:51 smithi022 ceph-mon[29462]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T07:25:51.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:51 smithi022 ceph-mon[29462]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T07:25:51.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:51 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:25:51.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:51 smithi022 ceph-mon[29462]: from='osd.6 v1:172.21.15.104:6808/721493010' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:25:51.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:51 smithi022 ceph-mon[29462]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:25:51.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:51 smithi022 ceph-mon[37330]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T07:25:51.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:51 smithi022 ceph-mon[37330]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T07:25:51.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:51 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:25:51.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:51 smithi022 ceph-mon[37330]: from='osd.6 v1:172.21.15.104:6808/721493010' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:25:51.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:51 smithi022 ceph-mon[37330]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:25:52.316 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-11T07:25:52.399 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-11T07:25:52.417 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-09-11T07:25:52.417 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-09-11T07:25:52.417 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0159869 s, 656 MB/s 2024-09-11T07:25:52.417 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-09-11T07:25:52.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:52 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:25:52.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:52 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:25:52.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:52 smithi022 ceph-mon[29462]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:52.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:52 smithi022 ceph-mon[29462]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-11T07:25:52.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:52 smithi022 ceph-mon[29462]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T07:25:52.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:52 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:25:52.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:52 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:25:52.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:52 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:25:52.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:52 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:25:52.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:52 smithi022 ceph-mon[37330]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:52.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:52 smithi022 ceph-mon[37330]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-11T07:25:52.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:52 smithi022 ceph-mon[37330]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T07:25:52.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:52 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:25:52.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:52 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:25:53.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:52 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:25:53.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:52 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:25:53.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:52 smithi104 ceph-mon[33507]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:53.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:52 smithi104 ceph-mon[33507]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-11T07:25:53.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:52 smithi104 ceph-mon[33507]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T07:25:53.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:52 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:25:53.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:52 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:25:53.772 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:53 smithi104 ceph-mon[33507]: osd.6 v1:172.21.15.104:6808/721493010 boot 2024-09-11T07:25:53.772 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:53 smithi104 ceph-mon[33507]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T07:25:53.772 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:53 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:25:53.796 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_1 2024-09-11T07:25:54.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:53 smithi022 ceph-mon[29462]: osd.6 v1:172.21.15.104:6808/721493010 boot 2024-09-11T07:25:54.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:53 smithi022 ceph-mon[29462]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T07:25:54.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:53 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:25:54.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:53 smithi022 ceph-mon[37330]: osd.6 v1:172.21.15.104:6808/721493010 boot 2024-09-11T07:25:54.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:53 smithi022 ceph-mon[37330]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T07:25:54.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:53 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:25:54.305 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:25:55.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:54 smithi104 ceph-mon[33507]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:55.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:54 smithi104 ceph-mon[33507]: osdmap e45: 7 total, 7 up, 7 in 2024-09-11T07:25:55.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:54 smithi022 ceph-mon[29462]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:55.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:54 smithi022 ceph-mon[29462]: osdmap e45: 7 total, 7 up, 7 in 2024-09-11T07:25:55.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:54 smithi022 ceph-mon[37330]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:25:55.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:54 smithi022 ceph-mon[37330]: osdmap e45: 7 total, 7 up, 7 in 2024-09-11T07:25:55.965 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:55 smithi104 ceph-mon[33507]: osdmap e46: 7 total, 7 up, 7 in 2024-09-11T07:25:56.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:55 smithi022 ceph-mon[29462]: osdmap e46: 7 total, 7 up, 7 in 2024-09-11T07:25:56.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:55 smithi022 ceph-mon[37330]: osdmap e46: 7 total, 7 up, 7 in 2024-09-11T07:25:56.890 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:56 smithi104 ceph-mon[33507]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:25:56.891 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:56 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:25:56.891 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:56 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:25:56.891 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:56 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:56.891 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:56 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:56.891 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:56 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:56.891 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:56 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:25:56.891 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:56 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:25:56.891 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:56 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:25:56.891 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:56 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:56.891 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:56 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:56.891 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:56 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:25:56.891 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:56 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:56.892 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:56 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:56.892 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:56 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:56.892 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:56 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:25:56.892 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:56 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:57.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[29462]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:25:57.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:25:57.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:25:57.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:57.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:57.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:57.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:25:57.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:25:57.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:25:57.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:57.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:57.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:25:57.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:57.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:57.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:57.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:25:57.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:57.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[37330]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:25:57.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:25:57.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:25:57.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:57.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:57.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:57.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:25:57.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:25:57.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:25:57.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:57.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:57.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:25:57.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:57.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:25:57.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:25:57.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:25:57.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:56 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:25:58.021 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:57 smithi104 ceph-mon[33507]: from='client.24307 v1:172.21.15.104:0/2609345260' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:25:58.022 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:57 smithi104 ceph-mon[33507]: Detected new or changed devices on smithi104 2024-09-11T07:25:58.022 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:57 smithi104 ceph-mon[33507]: Adjusting osd_memory_target on smithi104 to 5696M 2024-09-11T07:25:58.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:57 smithi022 ceph-mon[29462]: from='client.24307 v1:172.21.15.104:0/2609345260' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:25:58.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:57 smithi022 ceph-mon[29462]: Detected new or changed devices on smithi104 2024-09-11T07:25:58.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:57 smithi022 ceph-mon[29462]: Adjusting osd_memory_target on smithi104 to 5696M 2024-09-11T07:25:58.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:57 smithi022 ceph-mon[37330]: from='client.24307 v1:172.21.15.104:0/2609345260' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:25:58.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:57 smithi022 ceph-mon[37330]: Detected new or changed devices on smithi104 2024-09-11T07:25:58.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:57 smithi022 ceph-mon[37330]: Adjusting osd_memory_target on smithi104 to 5696M 2024-09-11T07:25:59.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:58 smithi104 ceph-mon[33507]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:25:59.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:58 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:25:59.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:58 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:25:59.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:58 smithi104 ceph-mon[33507]: osdmap e47: 7 total, 7 up, 7 in 2024-09-11T07:25:59.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:58 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.104:0/3665912874' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18199e47-0052-49f0-b45f-0fb201e0d0f4"}]: dispatch 2024-09-11T07:25:59.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:58 smithi104 ceph-mon[33507]: from='client.24313 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18199e47-0052-49f0-b45f-0fb201e0d0f4"}]: dispatch 2024-09-11T07:25:59.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:58 smithi104 ceph-mon[33507]: from='client.24313 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18199e47-0052-49f0-b45f-0fb201e0d0f4"}]': finished 2024-09-11T07:25:59.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:58 smithi104 ceph-mon[33507]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T07:25:59.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:58 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:25:59.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:58 smithi022 ceph-mon[29462]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:25:59.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:58 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:25:59.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:58 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:25:59.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:58 smithi022 ceph-mon[29462]: osdmap e47: 7 total, 7 up, 7 in 2024-09-11T07:25:59.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:58 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.104:0/3665912874' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18199e47-0052-49f0-b45f-0fb201e0d0f4"}]: dispatch 2024-09-11T07:25:59.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:58 smithi022 ceph-mon[29462]: from='client.24313 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18199e47-0052-49f0-b45f-0fb201e0d0f4"}]: dispatch 2024-09-11T07:25:59.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:58 smithi022 ceph-mon[29462]: from='client.24313 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18199e47-0052-49f0-b45f-0fb201e0d0f4"}]': finished 2024-09-11T07:25:59.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:58 smithi022 ceph-mon[29462]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T07:25:59.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:58 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:25:59.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:58 smithi022 ceph-mon[37330]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:25:59.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:58 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:25:59.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:58 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:25:59.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:58 smithi022 ceph-mon[37330]: osdmap e47: 7 total, 7 up, 7 in 2024-09-11T07:25:59.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:58 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.104:0/3665912874' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18199e47-0052-49f0-b45f-0fb201e0d0f4"}]: dispatch 2024-09-11T07:25:59.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:58 smithi022 ceph-mon[37330]: from='client.24313 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18199e47-0052-49f0-b45f-0fb201e0d0f4"}]: dispatch 2024-09-11T07:25:59.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:58 smithi022 ceph-mon[37330]: from='client.24313 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18199e47-0052-49f0-b45f-0fb201e0d0f4"}]': finished 2024-09-11T07:25:59.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:58 smithi022 ceph-mon[37330]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T07:25:59.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:58 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:26:00.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:25:59 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.104:0/3763915930' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:26:00.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:25:59 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.104:0/3763915930' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:26:00.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:25:59 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.104:0/3763915930' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:26:01.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:00 smithi104 ceph-mon[33507]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:01.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:00 smithi022 ceph-mon[29462]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:01.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:00 smithi022 ceph-mon[37330]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:03.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:02 smithi104 ceph-mon[33507]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:03.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:02 smithi022 ceph-mon[29462]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:03.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:02 smithi022 ceph-mon[37330]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:04.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:04 smithi104 ceph-mon[33507]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:04.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:04 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T07:26:04.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:04 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:04.969 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:04 smithi022 ceph-mon[29462]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:04.970 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:04 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T07:26:04.970 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:04 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:04.970 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:04 smithi022 ceph-mon[37330]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:04.970 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:04 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T07:26:04.970 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:04 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:06.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:05 smithi104 ceph-mon[33507]: Deploying daemon osd.7 on smithi104 2024-09-11T07:26:06.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:05 smithi022 ceph-mon[29462]: Deploying daemon osd.7 on smithi104 2024-09-11T07:26:06.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:05 smithi022 ceph-mon[37330]: Deploying daemon osd.7 on smithi104 2024-09-11T07:26:07.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:06 smithi104 ceph-mon[33507]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:07.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:06 smithi022 ceph-mon[29462]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:07.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:06 smithi022 ceph-mon[37330]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:08.923 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:08 smithi104 ceph-mon[33507]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:09.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:08 smithi022 ceph-mon[29462]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:09.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:08 smithi022 ceph-mon[37330]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:10.024 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:09 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:10.024 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:09 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:10.024 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:09 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:10.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:09 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:10.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:09 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:10.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:09 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:10.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:09 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:10.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:09 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:10.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:09 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:10.866 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:10 smithi104 ceph-mon[33507]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:11.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:10 smithi022 ceph-mon[29462]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:11.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:10 smithi022 ceph-mon[37330]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:11.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:11 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:26:11.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:11 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:26:12.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:11 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:26:12.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:11 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:26:12.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:11 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:26:12.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:11 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:26:12.417 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 7 on host 'smithi104' 2024-09-11T07:26:13.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:12 smithi104 ceph-mon[33507]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:13.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:12 smithi104 ceph-mon[33507]: osdmap e49: 8 total, 7 up, 8 in 2024-09-11T07:26:13.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:12 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:26:13.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:12 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:13.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:12 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:13.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:12 smithi104 ceph-mon[33507]: from='osd.7 v1:172.21.15.104:6812/3018423954' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:26:13.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:12 smithi104 ceph-mon[33507]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:26:13.060 DEBUG:teuthology.orchestra.run.smithi104:osd.7> sudo journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.7.service 2024-09-11T07:26:13.062 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-11T07:26:13.063 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T07:26:13.092 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:12 smithi022 ceph-mon[37330]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:13.092 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:12 smithi022 ceph-mon[37330]: osdmap e49: 8 total, 7 up, 8 in 2024-09-11T07:26:13.092 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:12 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:26:13.092 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:12 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:13.093 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:12 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:13.093 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:12 smithi022 ceph-mon[37330]: from='osd.7 v1:172.21.15.104:6812/3018423954' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:26:13.093 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:12 smithi022 ceph-mon[37330]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:26:13.093 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:12 smithi022 ceph-mon[29462]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:13.093 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:12 smithi022 ceph-mon[29462]: osdmap e49: 8 total, 7 up, 8 in 2024-09-11T07:26:13.093 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:12 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:26:13.093 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:12 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:13.093 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:12 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:13.093 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:12 smithi022 ceph-mon[29462]: from='osd.7 v1:172.21.15.104:6812/3018423954' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:26:13.093 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:12 smithi022 ceph-mon[29462]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:26:13.329 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:26:14.682 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:14 smithi022 ceph-mon[37330]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:14.683 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:14 smithi022 ceph-mon[37330]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T07:26:14.683 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:14 smithi022 ceph-mon[37330]: osdmap e50: 8 total, 7 up, 8 in 2024-09-11T07:26:14.683 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:14 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:26:14.683 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:14 smithi022 ceph-mon[37330]: from='osd.7 v1:172.21.15.104:6812/3018423954' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:26:14.683 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:14 smithi022 ceph-mon[37330]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:26:14.683 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:14 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:14.683 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:14 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:14.683 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:14 smithi022 ceph-mon[29462]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:14.683 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:14 smithi022 ceph-mon[29462]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T07:26:14.683 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:14 smithi022 ceph-mon[29462]: osdmap e50: 8 total, 7 up, 8 in 2024-09-11T07:26:14.683 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:14 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:26:14.684 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:14 smithi022 ceph-mon[29462]: from='osd.7 v1:172.21.15.104:6812/3018423954' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:26:14.684 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:14 smithi022 ceph-mon[29462]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:26:14.684 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:14 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:14.684 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:14 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:14.769 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:14 smithi104 ceph-mon[33507]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:14.769 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:14 smithi104 ceph-mon[33507]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T07:26:14.769 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:14 smithi104 ceph-mon[33507]: osdmap e50: 8 total, 7 up, 8 in 2024-09-11T07:26:14.769 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:14 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:26:14.769 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:14 smithi104 ceph-mon[33507]: from='osd.7 v1:172.21.15.104:6812/3018423954' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:26:14.769 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:14 smithi104 ceph-mon[33507]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:26:14.769 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:14 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:14.770 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:14 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:14.770 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:26:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[55581]: 2024-09-11T07:26:14.422+0000 7f19b122f640 -1 osd.7 0 waiting for initial osdmap 2024-09-11T07:26:14.770 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:26:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[55581]: 2024-09-11T07:26:14.428+0000 7f19ad068640 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:26:15.173 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:26:15.656 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:15 smithi104 ceph-mon[33507]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-11T07:26:15.656 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:15 smithi104 ceph-mon[33507]: osdmap e51: 8 total, 7 up, 8 in 2024-09-11T07:26:15.656 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:15 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:26:15.656 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:15 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:26:15.656 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:15 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4974554' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T07:26:15.656 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:15 smithi104 ceph-mon[33507]: osd.7 v1:172.21.15.104:6812/3018423954 boot 2024-09-11T07:26:15.656 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:15 smithi104 ceph-mon[33507]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T07:26:15.657 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:15 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:26:15.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:15 smithi022 ceph-mon[37330]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-11T07:26:15.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:15 smithi022 ceph-mon[37330]: osdmap e51: 8 total, 7 up, 8 in 2024-09-11T07:26:15.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:15 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:26:15.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:15 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:26:15.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:15 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4974554' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T07:26:15.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:15 smithi022 ceph-mon[37330]: osd.7 v1:172.21.15.104:6812/3018423954 boot 2024-09-11T07:26:15.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:15 smithi022 ceph-mon[37330]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T07:26:15.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:15 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:26:15.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:15 smithi022 ceph-mon[29462]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-11T07:26:15.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:15 smithi022 ceph-mon[29462]: osdmap e51: 8 total, 7 up, 8 in 2024-09-11T07:26:15.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:15 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:26:15.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:15 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:26:15.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:15 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4974554' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T07:26:15.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:15 smithi022 ceph-mon[29462]: osd.7 v1:172.21.15.104:6812/3018423954 boot 2024-09-11T07:26:15.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:15 smithi022 ceph-mon[29462]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T07:26:15.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:15 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:26:15.856 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":51,"num_osds":8,"num_up_osds":7,"osd_up_since":1726039552,"num_in_osds":8,"osd_in_since":1726039558,"num_remapped_pgs":0} 2024-09-11T07:26:16.660 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:16 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:26:16.660 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:16 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:26:16.660 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:16 smithi104 ceph-mon[33507]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:16.660 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:16 smithi104 ceph-mon[33507]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T07:26:16.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:16 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:26:16.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:16 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:26:16.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:16 smithi022 ceph-mon[37330]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:16.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:16 smithi022 ceph-mon[37330]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T07:26:16.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:16 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:26:16.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:16 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:26:16.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:16 smithi022 ceph-mon[29462]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:26:16.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:16 smithi022 ceph-mon[29462]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T07:26:16.858 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T07:26:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:26:17.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:26:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:26:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[37330]: Detected new or changed devices on smithi104 2024-09-11T07:26:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:26:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:26:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:26:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:26:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[37330]: Adjusting osd_memory_target on smithi104 to 4272M 2024-09-11T07:26:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:17.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:17.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:17.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:17.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:17.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:17.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[37330]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T07:26:17.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:26:17.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:26:17.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[29462]: Detected new or changed devices on smithi104 2024-09-11T07:26:17.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:17.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:17.920 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:26:17.920 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:26:17.920 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:26:17.920 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:26:17.920 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[29462]: Adjusting osd_memory_target on smithi104 to 4272M 2024-09-11T07:26:17.920 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:17.920 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:17.920 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:17.920 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:17.920 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:17.920 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:17.920 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:17.921 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:17.921 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:17 smithi022 ceph-mon[29462]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T07:26:18.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:17 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:26:18.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:17 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:26:18.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:17 smithi104 ceph-mon[33507]: Detected new or changed devices on smithi104 2024-09-11T07:26:18.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:17 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:18.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:17 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:18.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:17 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:26:18.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:17 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:26:18.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:17 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:26:18.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:17 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:26:18.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:17 smithi104 ceph-mon[33507]: Adjusting osd_memory_target on smithi104 to 4272M 2024-09-11T07:26:18.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:17 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:18.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:17 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:18.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:17 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:18.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:17 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:18.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:17 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:18.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:17 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:18.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:17 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:18.032 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:17 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:18.032 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:17 smithi104 ceph-mon[33507]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T07:26:18.816 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:18 smithi022 ceph-mon[29462]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:26:18.817 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:18 smithi022 ceph-mon[37330]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:26:19.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:18 smithi104 ceph-mon[33507]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:26:19.075 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:26:19.723 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":54,"num_osds":8,"num_up_osds":8,"osd_up_since":1726039575,"num_in_osds":8,"osd_in_since":1726039558,"num_remapped_pgs":0} 2024-09-11T07:26:19.724 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T07:26:19.980 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:26:20.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:19 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3373577808' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T07:26:20.101 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:19 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3373577808' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T07:26:20.102 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:19 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3373577808' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T07:26:20.951 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:20 smithi022 ceph-mon[29462]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:26:20.951 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:20 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:26:20.951 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:20 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:26:20.951 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:20 smithi022 ceph-mon[29462]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T07:26:20.952 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:20 smithi022 ceph-mon[37330]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:26:20.952 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:20 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:26:20.952 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:20 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:26:20.952 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:20 smithi022 ceph-mon[37330]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T07:26:21.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:20 smithi104 ceph-mon[33507]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:26:21.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:20 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:26:21.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:20 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:26:21.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:20 smithi104 ceph-mon[33507]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T07:26:21.580 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:26:21.581 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":55,"fsid":"12a285d8-700e-11ef-bcea-c7b262605968","created":"2024-09-11T07:21:18.695332+0000","modified":"2024-09-11T07:26:19.689806+0000","last_up_change":"2024-09-11T07:26:15.385688+0000","last_in_change":"2024-09-11T07:25:58.551813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T07:24:21.772426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"09075f16-32f4-4f88-a116-c3d7f977a0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6801","nonce":2821493591}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6802","nonce":2821493591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6804","nonce":2821493591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6803","nonce":2821493591}]},"public_addr":"172.21.15.22:6801/2821493591","cluster_addr":"172.21.15.22:6802/2821493591","heartbeat_back_addr":"172.21.15.22:6804/2821493591","heartbeat_front_addr":"172.21.15.22:6803/2821493591","state":["exists","up"]},{"osd":1,"uuid":"52cc1e06-9506-48c0-9f7b-d4270dff4b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6805","nonce":4037911317}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6806","nonce":4037911317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6808","nonce":4037911317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6807","nonce":4037911317}]},"public_addr":"172.21.15.22:6805/4037911317","cluster_addr":"172.21.15.22:6806/4037911317","heartbeat_back_addr":"172.21.15.22:6808/4037911317","heartbeat_front_addr":"172.21.15.22:6807/4037911317","state":["exists","up"]},{"osd":2,"uuid":"fa2b2730-2c1d-45d0-be8a-494a60dc6b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6809","nonce":1675799666}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6810","nonce":1675799666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6812","nonce":1675799666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6811","nonce":1675799666}]},"public_addr":"172.21.15.22:6809/1675799666","cluster_addr":"172.21.15.22:6810/1675799666","heartbeat_back_addr":"172.21.15.22:6812/1675799666","heartbeat_front_addr":"172.21.15.22:6811/1675799666","state":["exists","up"]},{"osd":3,"uuid":"8d53d3fc-e401-47df-b70b-4b4b1f4539df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6813","nonce":1501962752}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6814","nonce":1501962752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6816","nonce":1501962752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6815","nonce":1501962752}]},"public_addr":"172.21.15.22:6813/1501962752","cluster_addr":"172.21.15.22:6814/1501962752","heartbeat_back_addr":"172.21.15.22:6816/1501962752","heartbeat_front_addr":"172.21.15.22:6815/1501962752","state":["exists","up"]},{"osd":4,"uuid":"d3207fe0-8b97-4067-a755-ba6d449c53c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6800","nonce":1210229443}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6801","nonce":1210229443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6803","nonce":1210229443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6802","nonce":1210229443}]},"public_addr":"172.21.15.104:6800/1210229443","cluster_addr":"172.21.15.104:6801/1210229443","heartbeat_back_addr":"172.21.15.104:6803/1210229443","heartbeat_front_addr":"172.21.15.104:6802/1210229443","state":["exists","up"]},{"osd":5,"uuid":"89c9352b-30b2-48c6-869e-047d07a23a43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6804","nonce":2735328732}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6805","nonce":2735328732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6807","nonce":2735328732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6806","nonce":2735328732}]},"public_addr":"172.21.15.104:6804/2735328732","cluster_addr":"172.21.15.104:6805/2735328732","heartbeat_back_addr":"172.21.15.104:6807/2735328732","heartbeat_front_addr":"172.21.15.104:6806/2735328732","state":["exists","up"]},{"osd":6,"uuid":"704563de-d88f-480a-b469-720135704d42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6808","nonce":721493010}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6809","nonce":721493010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6811","nonce":721493010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6810","nonce":721493010}]},"public_addr":"172.21.15.104:6808/721493010","cluster_addr":"172.21.15.104:6809/721493010","heartbeat_back_addr":"172.21.15.104:6811/721493010","heartbeat_front_addr":"172.21.15.104:6810/721493010","state":["exists","up"]},{"osd":7,"uuid":"18199e47-0052-49f0-b45f-0fb201e0d0f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6812","nonce":3018423954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6813","nonce":3018423954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6815","nonce":3018423954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6814","nonce":3018423954}]},"public_addr":"172.21.15.104:6812/3018423954","cluster_addr":"172.21.15.104:6813/3018423954","heartbeat_back_addr":"172.21.15.104:6815/3018423954","heartbeat_front_addr":"172.21.15.104:6814/3018423954","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:26:11.045272+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:26:19.640995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:25:33.365033+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:25:57.303359+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:26:16.517155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:25:27.725438+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:25:50.638323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:26:13.561670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/2968034970":"2024-09-12T07:22:11.238618+0000","172.21.15.22:0/2197488530":"2024-09-12T07:22:11.238618+0000","172.21.15.22:6800/3573193776":"2024-09-12T07:22:11.238618+0000","172.21.15.22:0/4236450731":"2024-09-12T07:21:48.438371+0000","172.21.15.22:0/639336081":"2024-09-12T07:21:48.438371+0000","172.21.15.22:0/2661764422":"2024-09-12T07:21:48.438371+0000","172.21.15.22:0/585929238":"2024-09-12T07:22:11.238618+0000","172.21.15.22:6800/4016369460":"2024-09-12T07:21:48.438371+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T07:26:21.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:21 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/484026424' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:26:21.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:21 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/484026424' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:26:22.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:21 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/484026424' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:26:22.200 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T07:24:21.772426+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-09-11T07:26:22.200 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-11T07:26:22.453 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:26:22.894 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:22 smithi022 ceph-mon[29462]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:26:22.895 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:22 smithi022 ceph-mon[37330]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:26:23.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:22 smithi104 ceph-mon[33507]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:26:24.280 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-09-11T07:26:24.913 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:24 smithi022 ceph-mon[29462]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:26:24.913 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:24 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/571136099' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T07:26:24.914 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:24 smithi022 ceph-mon[37330]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:26:24.914 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:24 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/571136099' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T07:26:24.939 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi022 2024-09-11T07:26:24.939 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi022=foo.a' 2024-09-11T07:26:24.966 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:24 smithi104 ceph-mon[33507]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:26:24.966 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:24 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/571136099' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T07:26:25.186 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:26:26.970 INFO:teuthology.orchestra.run.smithi104.stdout:Scheduled rgw.foo.a update... 2024-09-11T07:26:26.971 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:26 smithi104 ceph-mon[33507]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:26:27.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:26 smithi022 ceph-mon[29462]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:26:27.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:26 smithi022 ceph-mon[37330]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:26:27.623 DEBUG:teuthology.orchestra.run.smithi022:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@rgw.foo.a.service 2024-09-11T07:26:27.625 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi104 2024-09-11T07:26:27.625 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-11T07:26:27.908 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:26:28.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:27 smithi022 ceph-mon[37330]: from='client.24358 v1:172.21.15.104:0/4281185956' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi022=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:26:28.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:27 smithi022 ceph-mon[37330]: Saving service rgw.foo.a spec with placement smithi022=foo.a;count:1 2024-09-11T07:26:28.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:27 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:28.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:27 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:28.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:27 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:28.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:27 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:28.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:27 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:28.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:27 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T07:26:28.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:27 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T07:26:28.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:27 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:28.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:27 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:28.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:27 smithi022 ceph-mon[37330]: Deploying daemon rgw.foo.a on smithi022 2024-09-11T07:26:28.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:27 smithi022 ceph-mon[37330]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:26:28.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:27 smithi022 ceph-mon[29462]: from='client.24358 v1:172.21.15.104:0/4281185956' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi022=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:26:28.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:27 smithi022 ceph-mon[29462]: Saving service rgw.foo.a spec with placement smithi022=foo.a;count:1 2024-09-11T07:26:28.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:27 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:28.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:27 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:28.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:27 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:28.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:27 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:28.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:27 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:28.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:27 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T07:26:28.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:27 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T07:26:28.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:27 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:28.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:27 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:28.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:27 smithi022 ceph-mon[29462]: Deploying daemon rgw.foo.a on smithi022 2024-09-11T07:26:28.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:27 smithi022 ceph-mon[29462]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:26:28.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:27 smithi104 ceph-mon[33507]: from='client.24358 v1:172.21.15.104:0/4281185956' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi022=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:26:28.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:27 smithi104 ceph-mon[33507]: Saving service rgw.foo.a spec with placement smithi022=foo.a;count:1 2024-09-11T07:26:28.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:27 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:28.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:27 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:28.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:27 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:28.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:27 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:28.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:27 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:28.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:27 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T07:26:28.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:27 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T07:26:28.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:27 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:28.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:27 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:28.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:27 smithi104 ceph-mon[33507]: Deploying daemon rgw.foo.a on smithi022 2024-09-11T07:26:28.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:27 smithi104 ceph-mon[33507]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:26:29.917 INFO:journalctl@ceph.rgw.foo.a.smithi022.stdout:Sep 11 07:26:29 smithi022 systemd[1]: Started Ceph rgw.foo.a for 12a285d8-700e-11ef-bcea-c7b262605968. 2024-09-11T07:26:30.411 INFO:teuthology.orchestra.run.smithi104.stderr:pool 'datapool' created 2024-09-11T07:26:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:30 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.104:0/3187375111' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T07:26:30.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:30 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.104:0/3187375111' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T07:26:30.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:30 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.104:0/3187375111' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T07:26:31.100 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- rbd pool init datapool 2024-09-11T07:26:31.338 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:26:31.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[37330]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:26:31.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[37330]: from='client.24364 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T07:26:31.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:31.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[37330]: from='client.24364 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T07:26:31.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[37330]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T07:26:31.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1860969011' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T07:26:31.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:31.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[37330]: from='client.24353 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T07:26:31.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:31.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[37330]: Saving service rgw.foo.a spec with placement smithi022=foo.a;count:1 2024-09-11T07:26:31.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:31.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:31.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:31.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:31.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[29462]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:26:31.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[29462]: from='client.24364 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T07:26:31.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:31.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[29462]: from='client.24364 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T07:26:31.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[29462]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T07:26:31.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1860969011' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T07:26:31.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:31.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[29462]: from='client.24353 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T07:26:31.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:31.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[29462]: Saving service rgw.foo.a spec with placement smithi022=foo.a;count:1 2024-09-11T07:26:31.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:31.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:31.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:31.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:31 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:31.670 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:31 smithi104 ceph-mon[33507]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:26:31.670 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:31 smithi104 ceph-mon[33507]: from='client.24364 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T07:26:31.670 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:31 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:31.670 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:31 smithi104 ceph-mon[33507]: from='client.24364 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T07:26:31.670 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:31 smithi104 ceph-mon[33507]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T07:26:31.670 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:31 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1860969011' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T07:26:31.670 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:31 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:31.670 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:31 smithi104 ceph-mon[33507]: from='client.24353 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T07:26:31.670 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:31 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:31.670 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:31 smithi104 ceph-mon[33507]: Saving service rgw.foo.a spec with placement smithi022=foo.a;count:1 2024-09-11T07:26:31.670 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:31 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:31.670 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:31 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:31.671 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:31 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:31.671 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:31 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:32.750 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:32 smithi104 ceph-mon[33507]: pgmap v176: 36 pgs: 11 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:26:32.750 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:32 smithi104 ceph-mon[33507]: from='client.24353 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T07:26:32.750 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:32 smithi104 ceph-mon[33507]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T07:26:32.750 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:32 smithi104 ceph-mon[33507]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:26:32.751 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:32 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:32.751 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:32 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:32.751 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:32 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:32.751 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:32 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:32.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:32 smithi022 ceph-mon[29462]: pgmap v176: 36 pgs: 11 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:26:32.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:32 smithi022 ceph-mon[29462]: from='client.24353 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T07:26:32.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:32 smithi022 ceph-mon[29462]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T07:26:32.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:32 smithi022 ceph-mon[29462]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:26:32.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:32 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:32.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:32 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:32.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:32 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:32.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:32 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:32.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:32 smithi022 ceph-mon[37330]: pgmap v176: 36 pgs: 11 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:26:32.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:32 smithi022 ceph-mon[37330]: from='client.24353 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T07:26:32.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:32 smithi022 ceph-mon[37330]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T07:26:32.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:32 smithi022 ceph-mon[37330]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:26:32.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:32 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:32.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:32 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:32.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:32 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:32.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:32 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:33.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:33 smithi022 ceph-mon[29462]: Checking dashboard <-> RGW credentials 2024-09-11T07:26:33.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:33 smithi022 ceph-mon[29462]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T07:26:33.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:33 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1860969011' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:26:33.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:33 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1888422395' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:26:33.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:33 smithi022 ceph-mon[29462]: from='client.24353 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:26:33.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:33 smithi022 ceph-mon[29462]: from='client.24377 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:26:33.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:33 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.104:0/3053503268' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T07:26:33.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:33 smithi022 ceph-mon[29462]: from='client.24403 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T07:26:33.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:33 smithi022 ceph-mon[37330]: Checking dashboard <-> RGW credentials 2024-09-11T07:26:33.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:33 smithi022 ceph-mon[37330]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T07:26:33.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:33 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1860969011' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:26:33.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:33 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1888422395' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:26:33.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:33 smithi022 ceph-mon[37330]: from='client.24353 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:26:33.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:33 smithi022 ceph-mon[37330]: from='client.24377 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:26:33.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:33 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.104:0/3053503268' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T07:26:33.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:33 smithi022 ceph-mon[37330]: from='client.24403 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T07:26:34.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:33 smithi104 ceph-mon[33507]: Checking dashboard <-> RGW credentials 2024-09-11T07:26:34.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:33 smithi104 ceph-mon[33507]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T07:26:34.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:33 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1860969011' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:26:34.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:33 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1888422395' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:26:34.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:33 smithi104 ceph-mon[33507]: from='client.24353 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:26:34.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:33 smithi104 ceph-mon[33507]: from='client.24377 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:26:34.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:33 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.104:0/3053503268' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T07:26:34.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:33 smithi104 ceph-mon[33507]: from='client.24403 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T07:26:34.818 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:34 smithi022 ceph-mon[29462]: pgmap v179: 68 pgs: 15 creating+peering, 6 active+clean, 47 unknown; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 341 B/s wr, 0 op/s 2024-09-11T07:26:34.818 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:34 smithi022 ceph-mon[29462]: from='client.24353 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T07:26:34.819 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:34 smithi022 ceph-mon[29462]: from='client.24377 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T07:26:34.819 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:34 smithi022 ceph-mon[29462]: from='client.24403 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T07:26:34.819 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:34 smithi022 ceph-mon[29462]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T07:26:34.819 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:34 smithi022 ceph-mon[37330]: pgmap v179: 68 pgs: 15 creating+peering, 6 active+clean, 47 unknown; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 341 B/s wr, 0 op/s 2024-09-11T07:26:34.819 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:34 smithi022 ceph-mon[37330]: from='client.24353 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T07:26:34.819 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:34 smithi022 ceph-mon[37330]: from='client.24377 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T07:26:34.819 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:34 smithi022 ceph-mon[37330]: from='client.24403 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T07:26:34.819 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:34 smithi022 ceph-mon[37330]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T07:26:35.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:34 smithi104 ceph-mon[33507]: pgmap v179: 68 pgs: 15 creating+peering, 6 active+clean, 47 unknown; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 341 B/s wr, 0 op/s 2024-09-11T07:26:35.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:34 smithi104 ceph-mon[33507]: from='client.24353 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T07:26:35.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:34 smithi104 ceph-mon[33507]: from='client.24377 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T07:26:35.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:34 smithi104 ceph-mon[33507]: from='client.24403 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T07:26:35.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:34 smithi104 ceph-mon[33507]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T07:26:35.862 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:35 smithi104 ceph-mon[33507]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T07:26:35.863 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:35 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1888422395' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:26:35.863 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:35 smithi104 ceph-mon[33507]: from='client.24377 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:26:35.863 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:35 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1860969011' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:26:35.863 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:35 smithi104 ceph-mon[33507]: from='client.24353 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:26:35.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:35 smithi022 ceph-mon[29462]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T07:26:35.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:35 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1888422395' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:26:35.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:35 smithi022 ceph-mon[29462]: from='client.24377 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:26:35.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:35 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1860969011' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:26:35.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:35 smithi022 ceph-mon[29462]: from='client.24353 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:26:35.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:35 smithi022 ceph-mon[37330]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T07:26:35.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:35 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1888422395' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:26:35.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:35 smithi022 ceph-mon[37330]: from='client.24377 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:26:35.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:35 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1860969011' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:26:35.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:35 smithi022 ceph-mon[37330]: from='client.24353 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:26:36.226 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.104 --placement '1;smithi104=iscsi.a' 2024-09-11T07:26:36.472 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:26:36.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:36 smithi022 ceph-mon[29462]: pgmap v182: 100 pgs: 20 creating+peering, 29 active+clean, 51 unknown; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T07:26:36.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:36 smithi022 ceph-mon[29462]: from='client.24377 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T07:26:36.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:36 smithi022 ceph-mon[29462]: from='client.24353 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T07:26:36.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:36 smithi022 ceph-mon[29462]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T07:26:36.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:36 smithi022 ceph-mon[37330]: pgmap v182: 100 pgs: 20 creating+peering, 29 active+clean, 51 unknown; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T07:26:36.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:36 smithi022 ceph-mon[37330]: from='client.24377 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T07:26:36.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:36 smithi022 ceph-mon[37330]: from='client.24353 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T07:26:36.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:36 smithi022 ceph-mon[37330]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T07:26:36.926 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:36 smithi104 ceph-mon[33507]: pgmap v182: 100 pgs: 20 creating+peering, 29 active+clean, 51 unknown; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T07:26:36.926 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:36 smithi104 ceph-mon[33507]: from='client.24377 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T07:26:36.926 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:36 smithi104 ceph-mon[33507]: from='client.24353 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T07:26:36.926 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:36 smithi104 ceph-mon[33507]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T07:26:37.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:37 smithi104 ceph-mon[33507]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T07:26:37.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:37 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4007432209' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:26:37.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:37 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3531306186' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:26:37.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:37 smithi104 ceph-mon[33507]: from='client.24415 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:26:37.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:37 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4007432209' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T07:26:37.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:37 smithi104 ceph-mon[33507]: from='client.24415 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T07:26:37.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:37 smithi104 ceph-mon[33507]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T07:26:37.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:37 smithi022 ceph-mon[29462]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T07:26:37.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:37 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4007432209' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:26:37.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:37 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3531306186' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:26:37.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:37 smithi022 ceph-mon[29462]: from='client.24415 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:26:37.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:37 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4007432209' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T07:26:37.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:37 smithi022 ceph-mon[29462]: from='client.24415 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T07:26:37.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:37 smithi022 ceph-mon[29462]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T07:26:37.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:37 smithi022 ceph-mon[37330]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T07:26:37.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:37 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4007432209' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:26:37.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:37 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3531306186' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:26:37.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:37 smithi022 ceph-mon[37330]: from='client.24415 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:26:37.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:37 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4007432209' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T07:26:37.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:37 smithi022 ceph-mon[37330]: from='client.24415 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T07:26:37.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:37 smithi022 ceph-mon[37330]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T07:26:38.295 INFO:teuthology.orchestra.run.smithi104.stdout:Scheduled iscsi.datapool update... 2024-09-11T07:26:38.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:38 smithi022 ceph-mon[29462]: pgmap v185: 132 pgs: 16 creating+peering, 81 active+clean, 35 unknown; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-11T07:26:38.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:38 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4007432209' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:26:38.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:38 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3531306186' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:26:38.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:38 smithi022 ceph-mon[29462]: from='client.24415 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:26:38.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:38 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:38.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:38 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4007432209' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T07:26:38.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:38 smithi022 ceph-mon[29462]: from='client.24415 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T07:26:38.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:38 smithi022 ceph-mon[29462]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T07:26:38.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:38 smithi022 ceph-mon[37330]: pgmap v185: 132 pgs: 16 creating+peering, 81 active+clean, 35 unknown; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-11T07:26:38.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:38 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4007432209' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:26:38.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:38 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3531306186' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:26:38.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:38 smithi022 ceph-mon[37330]: from='client.24415 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:26:38.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:38 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:38.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:38 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4007432209' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T07:26:38.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:38 smithi022 ceph-mon[37330]: from='client.24415 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T07:26:38.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:38 smithi022 ceph-mon[37330]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T07:26:38.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:38 smithi104 ceph-mon[33507]: pgmap v185: 132 pgs: 16 creating+peering, 81 active+clean, 35 unknown; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-11T07:26:38.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:38 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4007432209' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:26:38.963 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:38 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3531306186' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:26:38.963 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:38 smithi104 ceph-mon[33507]: from='client.24415 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:26:38.963 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:38 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:38.963 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:38 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4007432209' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T07:26:38.963 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:38 smithi104 ceph-mon[33507]: from='client.24415 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T07:26:38.963 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:38 smithi104 ceph-mon[33507]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T07:26:38.986 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-11T07:26:38.986 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-11T07:26:38.986 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-11T07:26:39.016 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-11T07:26:39.017 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-11T07:26:39.044 DEBUG:teuthology.orchestra.run.smithi104:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@iscsi.iscsi.a.service 2024-09-11T07:26:39.086 INFO:tasks.cephadm:Adding prometheus.a on smithi104 2024-09-11T07:26:39.086 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph orch apply prometheus '1;smithi104=a' 2024-09-11T07:26:39.756 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:26:40.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:39 smithi104 ceph-mon[33507]: from='client.24421 v1:172.21.15.104:0/217158150' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.104", "placement": "1;smithi104=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:26:40.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:39 smithi104 ceph-mon[33507]: Saving service iscsi.datapool spec with placement smithi104=iscsi.a;count:1 2024-09-11T07:26:40.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:39 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:40.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:39 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:40.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:39 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:40.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:39 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T07:26:40.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:39 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T07:26:40.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:39 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:40.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:39 smithi022 ceph-mon[29462]: from='client.24421 v1:172.21.15.104:0/217158150' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.104", "placement": "1;smithi104=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:26:40.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:39 smithi022 ceph-mon[29462]: Saving service iscsi.datapool spec with placement smithi104=iscsi.a;count:1 2024-09-11T07:26:40.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:39 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:40.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:39 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:40.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:39 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:40.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:39 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T07:26:40.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:39 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T07:26:40.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:39 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:40.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:39 smithi022 ceph-mon[37330]: from='client.24421 v1:172.21.15.104:0/217158150' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.104", "placement": "1;smithi104=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:26:40.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:39 smithi022 ceph-mon[37330]: Saving service iscsi.datapool spec with placement smithi104=iscsi.a;count:1 2024-09-11T07:26:40.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:39 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:40.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:39 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:40.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:39 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:40.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:39 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T07:26:40.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:39 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T07:26:40.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:39 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:40.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:40 smithi104 ceph-mon[33507]: Deploying daemon iscsi.iscsi.a on smithi104 2024-09-11T07:26:40.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:40 smithi104 ceph-mon[33507]: pgmap v188: 132 pgs: 11 creating+peering, 96 active+clean, 25 unknown; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-09-11T07:26:40.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:40 smithi104 ceph-mon[33507]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:26:40.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:40 smithi104 ceph-mon[33507]: Cluster is now healthy 2024-09-11T07:26:41.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:40 smithi022 ceph-mon[29462]: Deploying daemon iscsi.iscsi.a on smithi104 2024-09-11T07:26:41.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:40 smithi022 ceph-mon[29462]: pgmap v188: 132 pgs: 11 creating+peering, 96 active+clean, 25 unknown; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-09-11T07:26:41.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:40 smithi022 ceph-mon[29462]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:26:41.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:40 smithi022 ceph-mon[29462]: Cluster is now healthy 2024-09-11T07:26:41.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:40 smithi022 ceph-mon[37330]: Deploying daemon iscsi.iscsi.a on smithi104 2024-09-11T07:26:41.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:40 smithi022 ceph-mon[37330]: pgmap v188: 132 pgs: 11 creating+peering, 96 active+clean, 25 unknown; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-09-11T07:26:41.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:40 smithi022 ceph-mon[37330]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:26:41.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:40 smithi022 ceph-mon[37330]: Cluster is now healthy 2024-09-11T07:26:42.255 INFO:teuthology.orchestra.run.smithi104.stdout:Scheduled prometheus update... 2024-09-11T07:26:42.905 DEBUG:teuthology.orchestra.run.smithi104:prometheus.a> sudo journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@prometheus.a.service 2024-09-11T07:26:42.907 INFO:tasks.cephadm:Adding node-exporter.a on smithi022 2024-09-11T07:26:42.907 INFO:tasks.cephadm:Adding node-exporter.b on smithi104 2024-09-11T07:26:42.908 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph orch apply node-exporter '2;smithi022=a;smithi104=b' 2024-09-11T07:26:43.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:42 smithi104 ceph-mon[33507]: pgmap v189: 132 pgs: 7 creating+peering, 118 active+clean, 7 unknown; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 22 KiB/s rd, 2.6 KiB/s wr, 52 op/s 2024-09-11T07:26:43.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:42 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:26:43.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:42 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:26:43.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:42 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:43.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:42 smithi022 ceph-mon[29462]: pgmap v189: 132 pgs: 7 creating+peering, 118 active+clean, 7 unknown; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 22 KiB/s rd, 2.6 KiB/s wr, 52 op/s 2024-09-11T07:26:43.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:42 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:26:43.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:42 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:26:43.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:42 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:43.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:42 smithi022 ceph-mon[37330]: pgmap v189: 132 pgs: 7 creating+peering, 118 active+clean, 7 unknown; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 22 KiB/s rd, 2.6 KiB/s wr, 52 op/s 2024-09-11T07:26:43.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:42 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:26:43.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:42 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:26:43.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:42 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:43.440 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:26:43.562 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:26:43 smithi104 systemd[1]: Started Ceph iscsi.iscsi.a for 12a285d8-700e-11ef-bcea-c7b262605968. 2024-09-11T07:26:43.834 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:26:43 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug Started the configuration object watcher 2024-09-11T07:26:43.834 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:26:43 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug Checking for config object changes every 1s 2024-09-11T07:26:43.834 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:26:43 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug Processing osd blocklist entries for this node 2024-09-11T07:26:43.834 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:43 smithi104 ceph-mon[33507]: from='client.24442 v1:172.21.15.104:0/4236462472' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi104=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:26:43.834 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:43 smithi104 ceph-mon[33507]: Saving service prometheus spec with placement smithi104=a;count:1 2024-09-11T07:26:43.834 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:43 smithi104 ceph-mon[33507]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T07:26:43.834 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:43 smithi104 ceph-mon[33507]: pgmap v191: 132 pgs: 132 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 56 KiB/s rd, 4.7 KiB/s wr, 138 op/s 2024-09-11T07:26:43.834 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:43 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:43.835 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:43 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:43.835 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:43 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:43.835 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:43 smithi104 ceph-mon[33507]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T07:26:43.835 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:43 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:43.835 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:43 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:44.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:43 smithi022 ceph-mon[29462]: from='client.24442 v1:172.21.15.104:0/4236462472' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi104=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:26:44.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:43 smithi022 ceph-mon[29462]: Saving service prometheus spec with placement smithi104=a;count:1 2024-09-11T07:26:44.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:43 smithi022 ceph-mon[29462]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T07:26:44.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:43 smithi022 ceph-mon[29462]: pgmap v191: 132 pgs: 132 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 56 KiB/s rd, 4.7 KiB/s wr, 138 op/s 2024-09-11T07:26:44.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:43 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:44.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:43 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:44.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:43 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:44.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:43 smithi022 ceph-mon[29462]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T07:26:44.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:43 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:44.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:43 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:44.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:43 smithi022 ceph-mon[37330]: from='client.24442 v1:172.21.15.104:0/4236462472' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi104=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:26:44.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:43 smithi022 ceph-mon[37330]: Saving service prometheus spec with placement smithi104=a;count:1 2024-09-11T07:26:44.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:43 smithi022 ceph-mon[37330]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T07:26:44.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:43 smithi022 ceph-mon[37330]: pgmap v191: 132 pgs: 132 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 56 KiB/s rd, 4.7 KiB/s wr, 138 op/s 2024-09-11T07:26:44.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:43 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:44.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:43 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:44.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:43 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:44.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:43 smithi022 ceph-mon[37330]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T07:26:44.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:43 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:44.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:43 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:26:44.280 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:26:43 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug Reading the configuration object to update local LIO configuration 2024-09-11T07:26:44.280 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:26:43 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug Configuration does not have an entry for this host(smithi104.front.sepia.ceph.com) - nothing to define to LIO 2024-09-11T07:26:44.280 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:26:43 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-11T07:26:44.281 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:26:43 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: * Environment: production 2024-09-11T07:26:44.281 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:26:43 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T07:26:44.281 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:26:43 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: Use a production WSGI server instead. 2024-09-11T07:26:44.281 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:26:43 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: * Debug mode: off 2024-09-11T07:26:44.281 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:26:43 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug * Running on all addresses. 2024-09-11T07:26:44.281 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:26:43 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T07:26:44.281 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:26:43 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: * Running on all addresses. 2024-09-11T07:26:44.281 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:26:43 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T07:26:44.281 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:26:43 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug * Running on http://[fe80::ec4:7aff:fe8f:ce89]:5000/ (Press CTRL+C to quit) 2024-09-11T07:26:44.281 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:26:43 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: * Running on http://[fe80::ec4:7aff:fe8f:ce89]:5000/ (Press CTRL+C to quit) 2024-09-11T07:26:44.851 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:44 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.104:0/3449125076' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T07:26:45.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:44 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.104:0/3449125076' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T07:26:45.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:44 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.104:0/3449125076' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T07:26:45.227 INFO:teuthology.orchestra.run.smithi104.stdout:Scheduled node-exporter update... 2024-09-11T07:26:45.919 DEBUG:teuthology.orchestra.run.smithi022:node-exporter.a> sudo journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@node-exporter.a.service 2024-09-11T07:26:45.922 DEBUG:teuthology.orchestra.run.smithi104:node-exporter.b> sudo journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@node-exporter.b.service 2024-09-11T07:26:45.924 INFO:tasks.cephadm:Adding alertmanager.a on smithi022 2024-09-11T07:26:45.924 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph orch apply alertmanager '1;smithi022=a' 2024-09-11T07:26:46.292 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:26:46.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:46 smithi104 ceph-mon[33507]: from='client.24475 v1:172.21.15.104:0/1071541609' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi022=a;smithi104=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:26:46.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:46 smithi104 ceph-mon[33507]: Saving service node-exporter spec with placement smithi022=a;smithi104=b;count:2 2024-09-11T07:26:46.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:46 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:46.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:46 smithi104 ceph-mon[33507]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 60 KiB/s rd, 4.6 KiB/s wr, 145 op/s 2024-09-11T07:26:46.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:46 smithi104 ceph-mon[33507]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T07:26:46.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:46 smithi022 ceph-mon[29462]: from='client.24475 v1:172.21.15.104:0/1071541609' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi022=a;smithi104=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:26:46.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:46 smithi022 ceph-mon[29462]: Saving service node-exporter spec with placement smithi022=a;smithi104=b;count:2 2024-09-11T07:26:46.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:46 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:46.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:46 smithi022 ceph-mon[29462]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 60 KiB/s rd, 4.6 KiB/s wr, 145 op/s 2024-09-11T07:26:46.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:46 smithi022 ceph-mon[29462]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T07:26:46.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:46 smithi022 ceph-mon[37330]: from='client.24475 v1:172.21.15.104:0/1071541609' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi022=a;smithi104=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:26:46.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:46 smithi022 ceph-mon[37330]: Saving service node-exporter spec with placement smithi022=a;smithi104=b;count:2 2024-09-11T07:26:46.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:46 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:46.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:46 smithi022 ceph-mon[37330]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 60 KiB/s rd, 4.6 KiB/s wr, 145 op/s 2024-09-11T07:26:46.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:46 smithi022 ceph-mon[37330]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T07:26:47.396 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:47 smithi104 ceph-mon[33507]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T07:26:47.396 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:47 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:47.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:47 smithi022 ceph-mon[29462]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T07:26:47.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:47 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:47.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:47 smithi022 ceph-mon[37330]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T07:26:47.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:47 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:48.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:48 smithi104 ceph-mon[33507]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 56 KiB/s rd, 4.0 KiB/s wr, 135 op/s 2024-09-11T07:26:48.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:48 smithi022 ceph-mon[29462]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 56 KiB/s rd, 4.0 KiB/s wr, 135 op/s 2024-09-11T07:26:48.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:48 smithi022 ceph-mon[37330]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 56 KiB/s rd, 4.0 KiB/s wr, 135 op/s 2024-09-11T07:26:48.944 INFO:teuthology.orchestra.run.smithi104.stdout:Scheduled alertmanager update... 2024-09-11T07:26:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:49 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:26:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:49 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:26:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:49 smithi104 ceph-mon[33507]: osdmap e67: 8 total, 8 up, 8 in 2024-09-11T07:26:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:49 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:49 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:49.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:49 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:49.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:49 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:49.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:49 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:49.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:49 smithi104 ceph-mon[33507]: Deploying daemon prometheus.a on smithi104 2024-09-11T07:26:49.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:49 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:49.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:49 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:26:49.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:49 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:26:49.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:49 smithi022 ceph-mon[37330]: osdmap e67: 8 total, 8 up, 8 in 2024-09-11T07:26:49.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:49 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:49.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:49 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:49.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:49 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:49.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:49 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:49.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:49 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:49.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:49 smithi022 ceph-mon[37330]: Deploying daemon prometheus.a on smithi104 2024-09-11T07:26:49.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:49 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:49.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:49 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:26:49.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:49 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:26:49.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:49 smithi022 ceph-mon[29462]: osdmap e67: 8 total, 8 up, 8 in 2024-09-11T07:26:49.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:49 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:49.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:49 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:49.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:49 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:26:49.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:49 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:26:49.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:49 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:49.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:49 smithi022 ceph-mon[29462]: Deploying daemon prometheus.a on smithi104 2024-09-11T07:26:49.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:49 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:49.696 DEBUG:teuthology.orchestra.run.smithi022:alertmanager.a> sudo journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@alertmanager.a.service 2024-09-11T07:26:49.699 INFO:tasks.cephadm:Adding grafana.a on smithi104 2024-09-11T07:26:49.700 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph orch apply grafana '1;smithi104=a' 2024-09-11T07:26:50.115 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:26:50.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:50 smithi104 ceph-mon[33507]: from='client.24481 v1:172.21.15.104:0/1284336335' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi022=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:26:50.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:50 smithi104 ceph-mon[33507]: Saving service alertmanager spec with placement smithi022=a;count:1 2024-09-11T07:26:50.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:50 smithi104 ceph-mon[33507]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 1.4 KiB/s wr, 47 op/s 2024-09-11T07:26:50.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:50 smithi022 ceph-mon[29462]: from='client.24481 v1:172.21.15.104:0/1284336335' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi022=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:26:50.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:50 smithi022 ceph-mon[29462]: Saving service alertmanager spec with placement smithi022=a;count:1 2024-09-11T07:26:50.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:50 smithi022 ceph-mon[29462]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 1.4 KiB/s wr, 47 op/s 2024-09-11T07:26:50.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:50 smithi022 ceph-mon[37330]: from='client.24481 v1:172.21.15.104:0/1284336335' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi022=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:26:50.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:50 smithi022 ceph-mon[37330]: Saving service alertmanager spec with placement smithi022=a;count:1 2024-09-11T07:26:50.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:50 smithi022 ceph-mon[37330]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 1.4 KiB/s wr, 47 op/s 2024-09-11T07:26:51.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:51 smithi022 ceph-mon[29462]: osdmap e68: 8 total, 8 up, 8 in 2024-09-11T07:26:51.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:51 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:51.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:51 smithi022 ceph-mon[37330]: osdmap e68: 8 total, 8 up, 8 in 2024-09-11T07:26:51.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:51 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:51.690 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:51 smithi104 ceph-mon[33507]: osdmap e68: 8 total, 8 up, 8 in 2024-09-11T07:26:51.690 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:51 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:52.531 INFO:teuthology.orchestra.run.smithi104.stdout:Scheduled grafana update... 2024-09-11T07:26:52.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:52 smithi022 ceph-mon[29462]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 170 B/s wr, 2 op/s 2024-09-11T07:26:52.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:52 smithi022 ceph-mon[37330]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 170 B/s wr, 2 op/s 2024-09-11T07:26:52.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:52 smithi104 ceph-mon[33507]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 170 B/s wr, 2 op/s 2024-09-11T07:26:53.510 DEBUG:teuthology.orchestra.run.smithi104:grafana.a> sudo journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@grafana.a.service 2024-09-11T07:26:53.512 INFO:tasks.cephadm:Setting up client nodes... 2024-09-11T07:26:53.512 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T07:26:53.611 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:53 smithi104 ceph-mon[33507]: from='client.24487 v1:172.21.15.104:0/3675654826' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi104=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:26:53.612 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:53 smithi104 ceph-mon[33507]: Saving service grafana spec with placement smithi104=a;count:1 2024-09-11T07:26:53.612 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:53 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:53.781 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:26:53.877 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:53 smithi022 ceph-mon[37330]: from='client.24487 v1:172.21.15.104:0/3675654826' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi104=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:26:53.877 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:53 smithi022 ceph-mon[37330]: Saving service grafana spec with placement smithi104=a;count:1 2024-09-11T07:26:53.877 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:53 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:53.878 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:53 smithi022 ceph-mon[29462]: from='client.24487 v1:172.21.15.104:0/3675654826' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi104=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:26:53.878 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:53 smithi022 ceph-mon[29462]: Saving service grafana spec with placement smithi104=a;count:1 2024-09-11T07:26:53.878 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:53 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:26:54.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:26:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:26:54.881 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:54 smithi022 ceph-mon[29462]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 648 B/s rd, 129 B/s wr, 1 op/s 2024-09-11T07:26:54.881 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:54 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:26:54.881 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:54 smithi022 ceph-mon[37330]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 648 B/s rd, 129 B/s wr, 1 op/s 2024-09-11T07:26:54.881 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:54 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:26:55.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:54 smithi104 ceph-mon[33507]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 648 B/s rd, 129 B/s wr, 1 op/s 2024-09-11T07:26:55.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:54 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:26:55.731 INFO:teuthology.orchestra.run.smithi022.stdout:[client.0] 2024-09-11T07:26:55.732 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQA/RuFmu4BLKxAARD/+G0QkFDWJPH+6MnipRQ== 2024-09-11T07:26:56.308 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-11T07:26:56.308 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-11T07:26:56.309 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-11T07:26:56.351 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T07:26:56.582 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:56 smithi022 ceph-mon[29462]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 127 B/s wr, 2 op/s 2024-09-11T07:26:56.582 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:56 smithi022 ceph-mon[29462]: osdmap e69: 8 total, 8 up, 8 in 2024-09-11T07:26:56.582 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:56 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3476214830' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:26:56.582 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:56 smithi022 ceph-mon[29462]: from='client.24476 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:26:56.582 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:56 smithi022 ceph-mon[29462]: from='client.24476 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T07:26:56.582 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:56 smithi022 ceph-mon[37330]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 127 B/s wr, 2 op/s 2024-09-11T07:26:56.582 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:56 smithi022 ceph-mon[37330]: osdmap e69: 8 total, 8 up, 8 in 2024-09-11T07:26:56.583 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:56 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3476214830' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:26:56.583 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:56 smithi022 ceph-mon[37330]: from='client.24476 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:26:56.583 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:56 smithi022 ceph-mon[37330]: from='client.24476 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T07:26:56.771 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:56 smithi104 ceph-mon[33507]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 127 B/s wr, 2 op/s 2024-09-11T07:26:56.771 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:56 smithi104 ceph-mon[33507]: osdmap e69: 8 total, 8 up, 8 in 2024-09-11T07:26:56.771 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:56 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3476214830' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:26:56.771 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:56 smithi104 ceph-mon[33507]: from='client.24476 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:26:56.771 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:56 smithi104 ceph-mon[33507]: from='client.24476 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T07:26:58.470 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:58 smithi104 ceph-mon[33507]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:26:58.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:58 smithi022 ceph-mon[29462]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:26:58.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:58 smithi022 ceph-mon[37330]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:26:59.277 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:26:59.475 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:59 smithi104 ceph-mon[33507]: 1.0 scrub starts 2024-09-11T07:26:59.475 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:26:59 smithi104 ceph-mon[33507]: 1.0 scrub ok 2024-09-11T07:26:59.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:59 smithi022 ceph-mon[29462]: 1.0 scrub starts 2024-09-11T07:26:59.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:26:59 smithi022 ceph-mon[29462]: 1.0 scrub ok 2024-09-11T07:26:59.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:59 smithi022 ceph-mon[37330]: 1.0 scrub starts 2024-09-11T07:26:59.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:26:59 smithi022 ceph-mon[37330]: 1.0 scrub ok 2024-09-11T07:27:00.507 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:00 smithi104 ceph-mon[33507]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:27:00.507 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:00 smithi104 ceph-mon[33507]: osdmap e70: 8 total, 8 up, 8 in 2024-09-11T07:27:00.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:00 smithi022 ceph-mon[29462]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:27:00.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:00 smithi022 ceph-mon[29462]: osdmap e70: 8 total, 8 up, 8 in 2024-09-11T07:27:00.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:00 smithi022 ceph-mon[37330]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:27:00.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:00 smithi022 ceph-mon[37330]: osdmap e70: 8 total, 8 up, 8 in 2024-09-11T07:27:01.246 INFO:teuthology.orchestra.run.smithi104.stdout:[client.1] 2024-09-11T07:27:01.246 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQBFRuFm0gVTDhAADaIdSViAMC8UAXa+LzZY3w== 2024-09-11T07:27:01.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:01 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.104:0/1705767864' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:27:01.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:01 smithi104 ceph-mon[33507]: from='client.24499 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:27:01.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:01 smithi104 ceph-mon[33507]: from='client.24499 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T07:27:01.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:01 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.104:0/1705767864' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:27:01.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:01 smithi022 ceph-mon[37330]: from='client.24499 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:27:01.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:01 smithi022 ceph-mon[37330]: from='client.24499 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T07:27:01.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:01 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.104:0/1705767864' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:27:01.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:01 smithi022 ceph-mon[29462]: from='client.24499 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:27:01.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:01 smithi022 ceph-mon[29462]: from='client.24499 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T07:27:01.951 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-11T07:27:01.951 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-11T07:27:01.951 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-11T07:27:01.991 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T07:27:01.991 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-11T07:27:01.991 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-11T07:27:02.259 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:02.280 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:27:02 smithi104 systemd[1]: Started Ceph prometheus.a for 12a285d8-700e-11ef-bcea-c7b262605968. 2024-09-11T07:27:02.639 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:02 smithi022 ceph-mon[29462]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:02.640 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:02 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:27:02.640 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:02 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:27:02.640 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:02 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:27:02.640 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:02 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T07:27:02.640 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:02 smithi022 ceph-mon[37330]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:02.640 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:02 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:27:02.640 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:02 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:27:02.640 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:02 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:27:02.640 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:02 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T07:27:02.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:02 smithi104 ceph-mon[33507]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:02.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:02 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:27:02.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:02 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:27:02.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:02 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' 2024-09-11T07:27:02.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:02 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T07:27:03.532 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:03 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ignoring --setuser ceph since I am not root 2024-09-11T07:27:03.533 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:03 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ignoring --setgroup ceph since I am not root 2024-09-11T07:27:03.533 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:03 smithi022 ceph-mgr[29693]: -- 172.21.15.22:0/1368406406 <== mon.2 v1:172.21.15.22:6790/0 6 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (unknown 1357652691 0 0) 0x556c5fec49c0 con 0x556c5fed0c00 2024-09-11T07:27:03.533 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:03 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:03.395+0000 7f0b991a0ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:27:03.533 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:03 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:03.456+0000 7f0b991a0ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:27:03.533 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:03 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:03.527+0000 7f0b991a0ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:27:03.633 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:03 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ignoring --setuser ceph since I am not root 2024-09-11T07:27:03.633 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:03 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ignoring --setgroup ceph since I am not root 2024-09-11T07:27:03.633 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:03 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:27:03.389+0000 7fc492f88ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:27:03.633 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:03 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:27:03.444+0000 7fc492f88ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:27:03.633 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:03 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:27:03.508+0000 7fc492f88ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:27:03.633 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:03 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:27:03.630+0000 7fc492f88ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:27:03.798 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:03 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:03.657+0000 7f0b991a0ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:27:04.019 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:03 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:27:03.764+0000 7fc492f88ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:27:04.090 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:03 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:03.806+0000 7f0b991a0ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:27:04.224 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:27:04.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:04 smithi104 ceph-mon[33507]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T07:27:04.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:04 smithi104 ceph-mon[33507]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T07:27:04.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:04 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4059610664' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T07:27:04.280 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:27:04.017+0000 7fc492f88ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:27:04.280 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:27:04.076+0000 7fc492f88ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:27:04.281 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:27:04.234+0000 7fc492f88ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:27:04.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:04 smithi022 ceph-mon[29462]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T07:27:04.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:04 smithi022 ceph-mon[29462]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T07:27:04.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:04 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4059610664' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T07:27:04.418 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:04 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:04.088+0000 7f0b991a0ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:27:04.418 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:04 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:04.151+0000 7f0b991a0ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:27:04.418 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:04 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:04.325+0000 7f0b991a0ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:27:04.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:04 smithi022 ceph-mon[37330]: from='mgr.14150 v1:172.21.15.22:0/865287710' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T07:27:04.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:04 smithi022 ceph-mon[37330]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T07:27:04.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:04 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4059610664' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T07:27:05.013 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6800","nonce":2504196132}]},"active_addr":"172.21.15.22:6800/2504196132","active_change":"2024-09-11T07:22:11.238734+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":24109,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.22:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":544031024}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":1451388349}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":599318175}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":240999923}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":2104069730}]}]} 2024-09-11T07:27:05.017 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-11T07:27:05.018 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-11T07:27:05.018 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T07:27:05.276 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:05.302 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:05 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:05.224+0000 7f0b991a0ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:27:05.302 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:05 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:05.300+0000 7f0b991a0ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:27:05.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:05 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:27:05.077+0000 7fc492f88ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:27:05.530 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:05 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:27:05.140+0000 7fc492f88ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:27:06.127 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:05 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:27:05.841+0000 7fc492f88ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:27:06.127 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:05 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:27:05.950+0000 7fc492f88ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:27:06.127 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:06 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:27:06.068+0000 7fc492f88ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:27:06.370 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:06 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:06.058+0000 7f0b991a0ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:27:06.370 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:06 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:06.178+0000 7f0b991a0ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:27:06.370 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:06 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:06.306+0000 7f0b991a0ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:27:06.370 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:06 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:06.368+0000 7f0b991a0ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:27:06.410 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:06 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:27:06.125+0000 7fc492f88ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:27:06.410 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:06 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:27:06.248+0000 7fc492f88ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:27:06.410 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:06 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:27:06.315+0000 7fc492f88ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:27:06.410 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:06 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:27:06.371+0000 7fc492f88ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:27:06.641 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:06 smithi022 ceph-mon[29462]: osdmap e71: 8 total, 8 up, 8 in 2024-09-11T07:27:06.642 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:06 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:06.502+0000 7f0b991a0ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:27:06.642 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:06 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:06.577+0000 7f0b991a0ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:27:06.642 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:06 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:06.639+0000 7f0b991a0ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:27:06.642 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:06 smithi022 ceph-mon[37330]: osdmap e71: 8 total, 8 up, 8 in 2024-09-11T07:27:06.710 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:06 smithi104 ceph-mon[33507]: osdmap e71: 8 total, 8 up, 8 in 2024-09-11T07:27:06.711 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:06 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:27:06.503+0000 7fc492f88ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:27:06.711 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:06 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:27:06.565+0000 7fc492f88ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:27:06.711 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:06 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:27:06.629+0000 7fc492f88ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:27:06.711 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:06 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:27:06.708+0000 7fc492f88ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:27:06.917 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:06 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:06.781+0000 7f0b991a0ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:27:06.917 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:06 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:06.849+0000 7f0b991a0ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:27:07.030 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:06 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:27:06.765+0000 7fc492f88ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:27:07.030 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:06 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:27:06.822+0000 7fc492f88ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:27:07.206 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:06 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:06.917+0000 7f0b991a0ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:27:07.207 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:07 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:07.001+0000 7f0b991a0ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:27:07.207 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:07 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:07.061+0000 7f0b991a0ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:27:07.207 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:07 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:07.121+0000 7f0b991a0ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:27:07.365 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:27:07.365 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":71,"fsid":"12a285d8-700e-11ef-bcea-c7b262605968","created":"2024-09-11T07:21:18.695332+0000","modified":"2024-09-11T07:27:05.406527+0000","last_up_change":"2024-09-11T07:26:15.385688+0000","last_in_change":"2024-09-11T07:25:58.551813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T07:24:21.772426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-11T07:26:29.941354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-11T07:26:30.396859+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-11T07:26:31.922926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-11T07:26:34.007746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-11T07:26:36.077215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"09075f16-32f4-4f88-a116-c3d7f977a0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6801","nonce":2821493591}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6802","nonce":2821493591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6804","nonce":2821493591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6803","nonce":2821493591}]},"public_addr":"172.21.15.22:6801/2821493591","cluster_addr":"172.21.15.22:6802/2821493591","heartbeat_back_addr":"172.21.15.22:6804/2821493591","heartbeat_front_addr":"172.21.15.22:6803/2821493591","state":["exists","up"]},{"osd":1,"uuid":"52cc1e06-9506-48c0-9f7b-d4270dff4b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6805","nonce":4037911317}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6806","nonce":4037911317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6808","nonce":4037911317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6807","nonce":4037911317}]},"public_addr":"172.21.15.22:6805/4037911317","cluster_addr":"172.21.15.22:6806/4037911317","heartbeat_back_addr":"172.21.15.22:6808/4037911317","heartbeat_front_addr":"172.21.15.22:6807/4037911317","state":["exists","up"]},{"osd":2,"uuid":"fa2b2730-2c1d-45d0-be8a-494a60dc6b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6809","nonce":1675799666}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6810","nonce":1675799666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6812","nonce":1675799666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6811","nonce":1675799666}]},"public_addr":"172.21.15.22:6809/1675799666","cluster_addr":"172.21.15.22:6810/1675799666","heartbeat_back_addr":"172.21.15.22:6812/1675799666","heartbeat_front_addr":"172.21.15.22:6811/1675799666","state":["exists","up"]},{"osd":3,"uuid":"8d53d3fc-e401-47df-b70b-4b4b1f4539df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6813","nonce":1501962752}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6814","nonce":1501962752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6816","nonce":1501962752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6815","nonce":1501962752}]},"public_addr":"172.21.15.22:6813/1501962752","cluster_addr":"172.21.15.22:6814/1501962752","heartbeat_back_addr":"172.21.15.22:6816/1501962752","heartbeat_front_addr":"172.21.15.22:6815/1501962752","state":["exists","up"]},{"osd":4,"uuid":"d3207fe0-8b97-4067-a755-ba6d449c53c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6800","nonce":1210229443}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6801","nonce":1210229443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6803","nonce":1210229443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6802","nonce":1210229443}]},"public_addr":"172.21.15.104:6800/1210229443","cluster_addr":"172.21.15.104:6801/1210229443","heartbeat_back_addr":"172.21.15.104:6803/1210229443","heartbeat_front_addr":"172.21.15.104:6802/1210229443","state":["exists","up"]},{"osd":5,"uuid":"89c9352b-30b2-48c6-869e-047d07a23a43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6804","nonce":2735328732}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6805","nonce":2735328732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6807","nonce":2735328732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6806","nonce":2735328732}]},"public_addr":"172.21.15.104:6804/2735328732","cluster_addr":"172.21.15.104:6805/2735328732","heartbeat_back_addr":"172.21.15.104:6807/2735328732","heartbeat_front_addr":"172.21.15.104:6806/2735328732","state":["exists","up"]},{"osd":6,"uuid":"704563de-d88f-480a-b469-720135704d42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6808","nonce":721493010}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6809","nonce":721493010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6811","nonce":721493010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6810","nonce":721493010}]},"public_addr":"172.21.15.104:6808/721493010","cluster_addr":"172.21.15.104:6809/721493010","heartbeat_back_addr":"172.21.15.104:6811/721493010","heartbeat_front_addr":"172.21.15.104:6810/721493010","state":["exists","up"]},{"osd":7,"uuid":"18199e47-0052-49f0-b45f-0fb201e0d0f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6812","nonce":3018423954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6813","nonce":3018423954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6815","nonce":3018423954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6814","nonce":3018423954}]},"public_addr":"172.21.15.104:6812/3018423954","cluster_addr":"172.21.15.104:6813/3018423954","heartbeat_back_addr":"172.21.15.104:6815/3018423954","heartbeat_front_addr":"172.21.15.104:6814/3018423954","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:26:11.045272+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:26:19.640995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:26:47.772714+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:25:57.303359+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:26:16.517155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:26:42.236944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:25:50.638323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:26:13.561670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/2968034970":"2024-09-12T07:22:11.238618+0000","172.21.15.22:0/2197488530":"2024-09-12T07:22:11.238618+0000","172.21.15.22:6800/3573193776":"2024-09-12T07:22:11.238618+0000","172.21.15.22:0/4236450731":"2024-09-12T07:21:48.438371+0000","172.21.15.22:0/639336081":"2024-09-12T07:21:48.438371+0000","172.21.15.22:0/2661764422":"2024-09-12T07:21:48.438371+0000","172.21.15.22:0/585929238":"2024-09-12T07:22:11.238618+0000","172.21.15.22:6800/4016369460":"2024-09-12T07:21:48.438371+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T07:27:07.661 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:07 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2058912380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:27:07.661 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:07 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:27:07.551+0000 7fc492f88ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:27:07.661 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:07 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: [11/Sep/2024:07:27:07] ENGINE Bus STARTING 2024-09-11T07:27:07.661 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:07 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: CherryPy Checker: 2024-09-11T07:27:07.661 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:07 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: The Application mounted at '' has an empty config. 2024-09-11T07:27:07.661 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:07 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: 2024-09-11T07:27:07.661 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:07 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: [11/Sep/2024:07:27:07] ENGINE Serving on http://:::9283 2024-09-11T07:27:07.662 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:07 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: [11/Sep/2024:07:27:07] ENGINE Bus STARTED 2024-09-11T07:27:07.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:07 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2058912380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:27:07.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:07 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2058912380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:27:08.211 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:07 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:07.917+0000 7f0b991a0ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:27:08.211 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:07 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:27:07] ENGINE Bus STARTING 2024-09-11T07:27:08.212 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:07 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: CherryPy Checker: 2024-09-11T07:27:08.212 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:07 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: The Application mounted at '' has an empty config. 2024-09-11T07:27:08.212 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:07 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:08.212 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:27:08] ENGINE Serving on http://:::9283 2024-09-11T07:27:08.212 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:27:08] ENGINE Bus STARTED 2024-09-11T07:27:08.236 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-11T07:27:08.237 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T07:27:08.510 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:08.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: Standby manager daemon x restarted 2024-09-11T07:27:08.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: Standby manager daemon x started 2024-09-11T07:27:08.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: from='mgr.? v1:172.21.15.104:0/2487578648' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:27:08.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: from='mgr.? v1:172.21.15.104:0/2487578648' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:27:08.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: from='mgr.? v1:172.21.15.104:0/2487578648' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:27:08.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: from='mgr.? v1:172.21.15.104:0/2487578648' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:27:08.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: Active manager daemon y restarted 2024-09-11T07:27:08.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: Activating manager daemon y 2024-09-11T07:27:08.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: osdmap e72: 8 total, 8 up, 8 in 2024-09-11T07:27:08.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: mgrmap e18: y(active, starting, since 0.00769847s), standbys: x 2024-09-11T07:27:08.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:27:08.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:27:08.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:27:08.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:27:08.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:27:08.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:27:08.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:27:08.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:27:08.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:27:08.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:27:08.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:27:08.532 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:27:08.532 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:27:08.532 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:27:08.532 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:27:08.532 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:27:08.532 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: Manager daemon y is now available 2024-09-11T07:27:08.532 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:27:08.532 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:08.532 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:27:08.532 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:27:08.532 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:27:08.635 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: Standby manager daemon x restarted 2024-09-11T07:27:08.635 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: Standby manager daemon x started 2024-09-11T07:27:08.635 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: from='mgr.? v1:172.21.15.104:0/2487578648' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:27:08.635 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: from='mgr.? v1:172.21.15.104:0/2487578648' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:27:08.635 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: from='mgr.? v1:172.21.15.104:0/2487578648' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:27:08.635 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: from='mgr.? v1:172.21.15.104:0/2487578648' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:27:08.635 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: Active manager daemon y restarted 2024-09-11T07:27:08.635 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: Activating manager daemon y 2024-09-11T07:27:08.635 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: osdmap e72: 8 total, 8 up, 8 in 2024-09-11T07:27:08.635 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: mgrmap e18: y(active, starting, since 0.00769847s), standbys: x 2024-09-11T07:27:08.635 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:27:08.635 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:27:08.636 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:27:08.636 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:27:08.636 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:27:08.636 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:27:08.636 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:27:08.636 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:27:08.636 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:27:08.636 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:27:08.636 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:27:08.636 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:27:08.636 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:27:08.636 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:27:08.636 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:27:08.637 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:27:08.637 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: Manager daemon y is now available 2024-09-11T07:27:08.637 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:27:08.637 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:08.637 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:27:08.637 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:27:08.637 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:27:08.637 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: Standby manager daemon x restarted 2024-09-11T07:27:08.637 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: Standby manager daemon x started 2024-09-11T07:27:08.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: from='mgr.? v1:172.21.15.104:0/2487578648' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:27:08.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: from='mgr.? v1:172.21.15.104:0/2487578648' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:27:08.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: from='mgr.? v1:172.21.15.104:0/2487578648' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:27:08.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: from='mgr.? v1:172.21.15.104:0/2487578648' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:27:08.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: Active manager daemon y restarted 2024-09-11T07:27:08.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: Activating manager daemon y 2024-09-11T07:27:08.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: osdmap e72: 8 total, 8 up, 8 in 2024-09-11T07:27:08.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: mgrmap e18: y(active, starting, since 0.00769847s), standbys: x 2024-09-11T07:27:08.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:27:08.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:27:08.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:27:08.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:27:08.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:27:08.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:27:08.639 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:27:08.639 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:27:08.639 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:27:08.639 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:27:08.639 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:27:08.639 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:27:08.639 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:27:08.639 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:27:08.639 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:27:08.639 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:27:08.639 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: Manager daemon y is now available 2024-09-11T07:27:08.639 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:27:08.639 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:08.639 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:27:08.639 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:27:08.640 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:27:09.195 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:27:08 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:27:10.047 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-mon[29462]: mgrmap e19: y(active, since 1.01054s), standbys: x 2024-09-11T07:27:10.047 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:27:10.047 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-mon[29462]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:27:10.048 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:10.048 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-mon[37330]: mgrmap e19: y(active, since 1.01054s), standbys: x 2024-09-11T07:27:10.048 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:27:10.048 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-mon[37330]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:27:10.048 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:10.048 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:09.773+0000 7f0b3bf62640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:07:27:09] ENGINE Error in HTTPServer.serve 2024-09-11T07:27:10.048 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: Traceback (most recent call last): 2024-09-11T07:27:10.049 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:27:10.049 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: self._connections.run(self.expiration_interval) 2024-09-11T07:27:10.049 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:27:10.049 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: self._run(expiration_interval) 2024-09-11T07:27:10.049 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:27:10.049 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:27:10.049 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:27:10.049 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:27:10.049 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:27:10.049 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: s = self.context.wrap_socket( 2024-09-11T07:27:10.049 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:27:10.049 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: return self.sslsocket_class._create( 2024-09-11T07:27:10.049 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:27:10.049 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: self.do_handshake() 2024-09-11T07:27:10.050 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:27:10.050 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: self._sslobj.do_handshake() 2024-09-11T07:27:10.050 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:27:10.050 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:09 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:27:10.220 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:27:10.220 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":72,"fsid":"12a285d8-700e-11ef-bcea-c7b262605968","created":"2024-09-11T07:21:18.695332+0000","modified":"2024-09-11T07:27:07.920119+0000","last_up_change":"2024-09-11T07:26:15.385688+0000","last_in_change":"2024-09-11T07:25:58.551813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T07:24:21.772426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-11T07:26:29.941354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-11T07:26:30.396859+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-11T07:26:31.922926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-11T07:26:34.007746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-11T07:26:36.077215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"09075f16-32f4-4f88-a116-c3d7f977a0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6801","nonce":2821493591}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6802","nonce":2821493591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6804","nonce":2821493591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6803","nonce":2821493591}]},"public_addr":"172.21.15.22:6801/2821493591","cluster_addr":"172.21.15.22:6802/2821493591","heartbeat_back_addr":"172.21.15.22:6804/2821493591","heartbeat_front_addr":"172.21.15.22:6803/2821493591","state":["exists","up"]},{"osd":1,"uuid":"52cc1e06-9506-48c0-9f7b-d4270dff4b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6805","nonce":4037911317}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6806","nonce":4037911317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6808","nonce":4037911317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6807","nonce":4037911317}]},"public_addr":"172.21.15.22:6805/4037911317","cluster_addr":"172.21.15.22:6806/4037911317","heartbeat_back_addr":"172.21.15.22:6808/4037911317","heartbeat_front_addr":"172.21.15.22:6807/4037911317","state":["exists","up"]},{"osd":2,"uuid":"fa2b2730-2c1d-45d0-be8a-494a60dc6b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6809","nonce":1675799666}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6810","nonce":1675799666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6812","nonce":1675799666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6811","nonce":1675799666}]},"public_addr":"172.21.15.22:6809/1675799666","cluster_addr":"172.21.15.22:6810/1675799666","heartbeat_back_addr":"172.21.15.22:6812/1675799666","heartbeat_front_addr":"172.21.15.22:6811/1675799666","state":["exists","up"]},{"osd":3,"uuid":"8d53d3fc-e401-47df-b70b-4b4b1f4539df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6813","nonce":1501962752}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6814","nonce":1501962752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6816","nonce":1501962752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6815","nonce":1501962752}]},"public_addr":"172.21.15.22:6813/1501962752","cluster_addr":"172.21.15.22:6814/1501962752","heartbeat_back_addr":"172.21.15.22:6816/1501962752","heartbeat_front_addr":"172.21.15.22:6815/1501962752","state":["exists","up"]},{"osd":4,"uuid":"d3207fe0-8b97-4067-a755-ba6d449c53c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6800","nonce":1210229443}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6801","nonce":1210229443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6803","nonce":1210229443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6802","nonce":1210229443}]},"public_addr":"172.21.15.104:6800/1210229443","cluster_addr":"172.21.15.104:6801/1210229443","heartbeat_back_addr":"172.21.15.104:6803/1210229443","heartbeat_front_addr":"172.21.15.104:6802/1210229443","state":["exists","up"]},{"osd":5,"uuid":"89c9352b-30b2-48c6-869e-047d07a23a43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6804","nonce":2735328732}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6805","nonce":2735328732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6807","nonce":2735328732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6806","nonce":2735328732}]},"public_addr":"172.21.15.104:6804/2735328732","cluster_addr":"172.21.15.104:6805/2735328732","heartbeat_back_addr":"172.21.15.104:6807/2735328732","heartbeat_front_addr":"172.21.15.104:6806/2735328732","state":["exists","up"]},{"osd":6,"uuid":"704563de-d88f-480a-b469-720135704d42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6808","nonce":721493010}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6809","nonce":721493010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6811","nonce":721493010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6810","nonce":721493010}]},"public_addr":"172.21.15.104:6808/721493010","cluster_addr":"172.21.15.104:6809/721493010","heartbeat_back_addr":"172.21.15.104:6811/721493010","heartbeat_front_addr":"172.21.15.104:6810/721493010","state":["exists","up"]},{"osd":7,"uuid":"18199e47-0052-49f0-b45f-0fb201e0d0f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6812","nonce":3018423954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6813","nonce":3018423954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6815","nonce":3018423954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6814","nonce":3018423954}]},"public_addr":"172.21.15.104:6812/3018423954","cluster_addr":"172.21.15.104:6813/3018423954","heartbeat_back_addr":"172.21.15.104:6815/3018423954","heartbeat_front_addr":"172.21.15.104:6814/3018423954","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:26:11.045272+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:26:19.640995+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:26:47.772714+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:25:57.303359+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:26:16.517155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:26:42.236944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:25:50.638323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:26:13.561670+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/240999923":"2024-09-12T07:27:07.919998+0000","172.21.15.22:6800/4016369460":"2024-09-12T07:21:48.438371+0000","172.21.15.22:0/585929238":"2024-09-12T07:22:11.238618+0000","172.21.15.22:0/544031024":"2024-09-12T07:27:07.919998+0000","172.21.15.22:6800/2504196132":"2024-09-12T07:27:07.919998+0000","172.21.15.22:0/1451388349":"2024-09-12T07:27:07.919998+0000","172.21.15.22:0/639336081":"2024-09-12T07:21:48.438371+0000","172.21.15.22:0/2104069730":"2024-09-12T07:27:07.919998+0000","172.21.15.22:0/4236450731":"2024-09-12T07:21:48.438371+0000","172.21.15.22:6800/3573193776":"2024-09-12T07:22:11.238618+0000","172.21.15.22:0/2661764422":"2024-09-12T07:21:48.438371+0000","172.21.15.22:0/2197488530":"2024-09-12T07:22:11.238618+0000","172.21.15.22:0/2968034970":"2024-09-12T07:22:11.238618+0000","172.21.15.22:0/599318175":"2024-09-12T07:27:07.919998+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T07:27:10.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:09 smithi104 ceph-mon[33507]: mgrmap e19: y(active, since 1.01054s), standbys: x 2024-09-11T07:27:10.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:09 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:27:10.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:09 smithi104 ceph-mon[33507]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:27:10.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:09 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:11.062 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:10 smithi022 ceph-mon[29462]: [11/Sep/2024:07:27:09] ENGINE Bus STARTING 2024-09-11T07:27:11.062 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:10 smithi022 ceph-mon[29462]: [11/Sep/2024:07:27:09] ENGINE Serving on https://172.21.15.22:7150 2024-09-11T07:27:11.062 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:10 smithi022 ceph-mon[29462]: [11/Sep/2024:07:27:09] ENGINE Bus STARTED 2024-09-11T07:27:11.062 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:10 smithi022 ceph-mon[29462]: [11/Sep/2024:07:27:09] ENGINE Error in HTTPServer.serve 2024-09-11T07:27:11.062 INFO:journalctl@ceph.mon.a.smithi022.stdout: Traceback (most recent call last): 2024-09-11T07:27:11.062 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:27:11.062 INFO:journalctl@ceph.mon.a.smithi022.stdout: self._connections.run(self.expiration_interval) 2024-09-11T07:27:11.062 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:27:11.062 INFO:journalctl@ceph.mon.a.smithi022.stdout: self._run(expiration_interval) 2024-09-11T07:27:11.063 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:27:11.063 INFO:journalctl@ceph.mon.a.smithi022.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:27:11.063 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:27:11.063 INFO:journalctl@ceph.mon.a.smithi022.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:27:11.063 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:27:11.063 INFO:journalctl@ceph.mon.a.smithi022.stdout: s = self.context.wrap_socket( 2024-09-11T07:27:11.063 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:27:11.063 INFO:journalctl@ceph.mon.a.smithi022.stdout: return self.sslsocket_class._create( 2024-09-11T07:27:11.064 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:27:11.064 INFO:journalctl@ceph.mon.a.smithi022.stdout: self.do_handshake() 2024-09-11T07:27:11.064 INFO:journalctl@ceph.mon.a.smithi022.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:27:11.064 INFO:journalctl@ceph.mon.a.smithi022.stdout: self._sslobj.do_handshake() 2024-09-11T07:27:11.064 INFO:journalctl@ceph.mon.a.smithi022.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:27:11.064 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:10 smithi022 ceph-mon[29462]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:27:11.064 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:10 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/50993811' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:27:11.064 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:10 smithi022 ceph-mon[29462]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-11T07:27:11.064 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:10 smithi022 ceph-mon[37330]: [11/Sep/2024:07:27:09] ENGINE Bus STARTING 2024-09-11T07:27:11.064 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:10 smithi022 ceph-mon[37330]: [11/Sep/2024:07:27:09] ENGINE Serving on https://172.21.15.22:7150 2024-09-11T07:27:11.065 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:10 smithi022 ceph-mon[37330]: [11/Sep/2024:07:27:09] ENGINE Bus STARTED 2024-09-11T07:27:11.065 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:10 smithi022 ceph-mon[37330]: [11/Sep/2024:07:27:09] ENGINE Error in HTTPServer.serve 2024-09-11T07:27:11.065 INFO:journalctl@ceph.mon.c.smithi022.stdout: Traceback (most recent call last): 2024-09-11T07:27:11.065 INFO:journalctl@ceph.mon.c.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:27:11.065 INFO:journalctl@ceph.mon.c.smithi022.stdout: self._connections.run(self.expiration_interval) 2024-09-11T07:27:11.065 INFO:journalctl@ceph.mon.c.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:27:11.065 INFO:journalctl@ceph.mon.c.smithi022.stdout: self._run(expiration_interval) 2024-09-11T07:27:11.065 INFO:journalctl@ceph.mon.c.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:27:11.065 INFO:journalctl@ceph.mon.c.smithi022.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:27:11.065 INFO:journalctl@ceph.mon.c.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:27:11.065 INFO:journalctl@ceph.mon.c.smithi022.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:27:11.065 INFO:journalctl@ceph.mon.c.smithi022.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:27:11.066 INFO:journalctl@ceph.mon.c.smithi022.stdout: s = self.context.wrap_socket( 2024-09-11T07:27:11.066 INFO:journalctl@ceph.mon.c.smithi022.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:27:11.066 INFO:journalctl@ceph.mon.c.smithi022.stdout: return self.sslsocket_class._create( 2024-09-11T07:27:11.066 INFO:journalctl@ceph.mon.c.smithi022.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:27:11.066 INFO:journalctl@ceph.mon.c.smithi022.stdout: self.do_handshake() 2024-09-11T07:27:11.066 INFO:journalctl@ceph.mon.c.smithi022.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:27:11.066 INFO:journalctl@ceph.mon.c.smithi022.stdout: self._sslobj.do_handshake() 2024-09-11T07:27:11.066 INFO:journalctl@ceph.mon.c.smithi022.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:27:11.066 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:10 smithi022 ceph-mon[37330]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:27:11.066 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:10 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/50993811' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:27:11.067 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:10 smithi022 ceph-mon[37330]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-11T07:27:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:10 smithi104 ceph-mon[33507]: [11/Sep/2024:07:27:09] ENGINE Bus STARTING 2024-09-11T07:27:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:10 smithi104 ceph-mon[33507]: [11/Sep/2024:07:27:09] ENGINE Serving on https://172.21.15.22:7150 2024-09-11T07:27:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:10 smithi104 ceph-mon[33507]: [11/Sep/2024:07:27:09] ENGINE Bus STARTED 2024-09-11T07:27:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:10 smithi104 ceph-mon[33507]: [11/Sep/2024:07:27:09] ENGINE Error in HTTPServer.serve 2024-09-11T07:27:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout: Traceback (most recent call last): 2024-09-11T07:27:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:27:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout: self._connections.run(self.expiration_interval) 2024-09-11T07:27:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:27:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout: self._run(expiration_interval) 2024-09-11T07:27:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:27:11.281 INFO:journalctl@ceph.mon.b.smithi104.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:27:11.281 INFO:journalctl@ceph.mon.b.smithi104.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:27:11.281 INFO:journalctl@ceph.mon.b.smithi104.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:27:11.281 INFO:journalctl@ceph.mon.b.smithi104.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:27:11.281 INFO:journalctl@ceph.mon.b.smithi104.stdout: s = self.context.wrap_socket( 2024-09-11T07:27:11.281 INFO:journalctl@ceph.mon.b.smithi104.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:27:11.281 INFO:journalctl@ceph.mon.b.smithi104.stdout: return self.sslsocket_class._create( 2024-09-11T07:27:11.281 INFO:journalctl@ceph.mon.b.smithi104.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:27:11.281 INFO:journalctl@ceph.mon.b.smithi104.stdout: self.do_handshake() 2024-09-11T07:27:11.281 INFO:journalctl@ceph.mon.b.smithi104.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:27:11.281 INFO:journalctl@ceph.mon.b.smithi104.stdout: self._sslobj.do_handshake() 2024-09-11T07:27:11.281 INFO:journalctl@ceph.mon.b.smithi104.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:27:11.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:10 smithi104 ceph-mon[33507]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:27:11.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:10 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/50993811' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:27:11.282 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:10 smithi104 ceph-mon[33507]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-11T07:27:11.376 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-11T07:27:11.376 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-11T07:27:11.377 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-11T07:27:11.377 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-11T07:27:11.378 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-11T07:27:11.378 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-11T07:27:11.378 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-11T07:27:11.379 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-11T07:27:11.932 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:11.934 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:11.941 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:11.945 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:11.945 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:11.951 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:11.987 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:12.023 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:12.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:12 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:27:12.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:12 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:27:12.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:12 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:12.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:12 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:12.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:12 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:12.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:12 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:12.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:12 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:12.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:12 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:12.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:12 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:12.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:12 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:12.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:12 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:12.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:12 smithi022 ceph-mon[29462]: osdmap e73: 8 total, 8 up, 8 in 2024-09-11T07:27:12.409 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:12 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:27:12.409 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:12 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:27:12.409 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:12 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:12.410 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:12 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:12.410 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:12 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:12.410 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:12 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:12.410 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:12 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:12.410 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:12 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:12.410 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:12 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:12.410 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:12 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:12.410 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:12 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:12.410 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:12 smithi022 ceph-mon[37330]: osdmap e73: 8 total, 8 up, 8 in 2024-09-11T07:27:12.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:12 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:27:12.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:12 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:27:12.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:12 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:12.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:12 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:12.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:12 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:12.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:12 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:12.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:12 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:12.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:12 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:12.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:12 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:12.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:12 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:12.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:12 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:12.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:12 smithi104 ceph-mon[33507]: osdmap e73: 8 total, 8 up, 8 in 2024-09-11T07:27:13.382 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:13 smithi022 ceph-mon[29462]: Adjusting osd_memory_target on smithi104 to 3760M 2024-09-11T07:27:13.382 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:13 smithi022 ceph-mon[29462]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:27:13.382 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:13 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:13.382 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:13 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:13.382 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:13 smithi022 ceph-mon[29462]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-11T07:27:13.382 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:13 smithi022 ceph-mon[37330]: Adjusting osd_memory_target on smithi104 to 3760M 2024-09-11T07:27:13.382 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:13 smithi022 ceph-mon[37330]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:27:13.383 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:13 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:13.383 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:13 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:13.383 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:13 smithi022 ceph-mon[37330]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-11T07:27:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:13 smithi104 ceph-mon[33507]: Adjusting osd_memory_target on smithi104 to 3760M 2024-09-11T07:27:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:13 smithi104 ceph-mon[33507]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:27:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:13 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:13 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:13 smithi104 ceph-mon[33507]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-11T07:27:14.530 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:27:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:27:15.125 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:15 smithi022 ceph-mon[37330]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-11T07:27:15.125 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:15 smithi022 ceph-mon[29462]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-11T07:27:15.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:15 smithi104 ceph-mon[33507]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-11T07:27:16.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:16 smithi022 ceph-mon[37330]: osdmap e74: 8 total, 8 up, 8 in 2024-09-11T07:27:16.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:16 smithi022 ceph-mon[37330]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T07:27:16.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:16 smithi022 ceph-mon[29462]: osdmap e74: 8 total, 8 up, 8 in 2024-09-11T07:27:16.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:16 smithi022 ceph-mon[29462]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T07:27:16.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:16 smithi104 ceph-mon[33507]: osdmap e74: 8 total, 8 up, 8 in 2024-09-11T07:27:16.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:16 smithi104 ceph-mon[33507]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T07:27:17.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:17 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:27:17.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:17 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:27:17.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:17 smithi022 ceph-mon[29462]: osdmap e75: 8 total, 8 up, 8 in 2024-09-11T07:27:17.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:17 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:27:17.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:17 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:27:17.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:17 smithi022 ceph-mon[37330]: osdmap e75: 8 total, 8 up, 8 in 2024-09-11T07:27:17.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:17 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:27:17.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:17 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:27:17.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:17 smithi104 ceph-mon[33507]: osdmap e75: 8 total, 8 up, 8 in 2024-09-11T07:27:18.681 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:18 smithi022 ceph-mon[37330]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-11T07:27:18.681 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:18 smithi022 ceph-mon[29462]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-11T07:27:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:18 smithi104 ceph-mon[33507]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-11T07:27:19.280 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:27:18 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:27:19.580 INFO:teuthology.orchestra.run.smithi022.stdout:107374182432 2024-09-11T07:27:19.581 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-11T07:27:19.681 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:19 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:27:19.681 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:19 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:27:19.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:19 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:27:21.434 INFO:teuthology.orchestra.run.smithi022.stdout:34359738415 2024-09-11T07:27:21.435 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-11T07:27:21.507 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[37330]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T07:27:21.508 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:21.508 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:21.508 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:21.508 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:21.508 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:21.508 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:21.508 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[37330]: Adjusting osd_memory_target on smithi022 to 3761M 2024-09-11T07:27:21.508 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:21.508 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:21.508 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:27:21.508 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[37330]: Updating smithi022:/etc/ceph/ceph.conf 2024-09-11T07:27:21.508 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[37330]: Updating smithi104:/etc/ceph/ceph.conf 2024-09-11T07:27:21.508 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[37330]: osdmap e76: 8 total, 8 up, 8 in 2024-09-11T07:27:21.509 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:21.509 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:21.509 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:21.509 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:21.509 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:21.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[29462]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T07:27:21.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:21.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:21.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:21.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:21.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:21.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:21.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[29462]: Adjusting osd_memory_target on smithi022 to 3761M 2024-09-11T07:27:21.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:21.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:21.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:27:21.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[29462]: Updating smithi022:/etc/ceph/ceph.conf 2024-09-11T07:27:21.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[29462]: Updating smithi104:/etc/ceph/ceph.conf 2024-09-11T07:27:21.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[29462]: osdmap e76: 8 total, 8 up, 8 in 2024-09-11T07:27:21.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:21.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:21.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:21.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:21.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:21 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:21 smithi104 ceph-mon[33507]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T07:27:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:21 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:21 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:21 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:21 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:21 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:21 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:27:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:21 smithi104 ceph-mon[33507]: Adjusting osd_memory_target on smithi022 to 3761M 2024-09-11T07:27:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:21 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:21 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:27:21.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:21 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:27:21.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:21 smithi104 ceph-mon[33507]: Updating smithi022:/etc/ceph/ceph.conf 2024-09-11T07:27:21.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:21 smithi104 ceph-mon[33507]: Updating smithi104:/etc/ceph/ceph.conf 2024-09-11T07:27:21.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:21 smithi104 ceph-mon[33507]: osdmap e76: 8 total, 8 up, 8 in 2024-09-11T07:27:21.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:21 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:21.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:21 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:21.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:21 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:21.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:21 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:21.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:21 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:22.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:22 smithi104 ceph-mon[33507]: Updating smithi104:/var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:27:22.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:22 smithi104 ceph-mon[33507]: Updating smithi022:/var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:27:22.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:22 smithi104 ceph-mon[33507]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:27:22.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:22 smithi104 ceph-mon[33507]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:27:22.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:22 smithi104 ceph-mon[33507]: Updating smithi104:/var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:27:22.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:22 smithi104 ceph-mon[33507]: Updating smithi022:/var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:27:22.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:22 smithi104 ceph-mon[33507]: Deploying daemon alertmanager.a on smithi022 2024-09-11T07:27:22.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:22 smithi022 ceph-mon[29462]: Updating smithi104:/var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:27:22.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:22 smithi022 ceph-mon[29462]: Updating smithi022:/var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:27:22.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:22 smithi022 ceph-mon[29462]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:27:22.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:22 smithi022 ceph-mon[29462]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:27:22.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:22 smithi022 ceph-mon[29462]: Updating smithi104:/var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:27:22.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:22 smithi022 ceph-mon[29462]: Updating smithi022:/var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:27:22.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:22 smithi022 ceph-mon[29462]: Deploying daemon alertmanager.a on smithi022 2024-09-11T07:27:22.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:22 smithi022 ceph-mon[37330]: Updating smithi104:/var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:27:22.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:22 smithi022 ceph-mon[37330]: Updating smithi022:/var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:27:22.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:22 smithi022 ceph-mon[37330]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:27:22.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:22 smithi022 ceph-mon[37330]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:27:22.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:22 smithi022 ceph-mon[37330]: Updating smithi104:/var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:27:22.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:22 smithi022 ceph-mon[37330]: Updating smithi022:/var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:27:22.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:22 smithi022 ceph-mon[37330]: Deploying daemon alertmanager.a on smithi022 2024-09-11T07:27:23.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:23 smithi022 ceph-mon[29462]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 784 B/s rd, 0 op/s 2024-09-11T07:27:23.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:27:23.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:23 smithi022 ceph-mon[37330]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 784 B/s rd, 0 op/s 2024-09-11T07:27:23.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:27:23.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:23 smithi104 ceph-mon[33507]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 784 B/s rd, 0 op/s 2024-09-11T07:27:23.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:27:24.530 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:24 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:27:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:27:25.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:25 smithi104 ceph-mon[33507]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:25.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:25 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:27:25.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:25 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:27:25.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:25 smithi022 ceph-mon[29462]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:25.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:25 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:27:25.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:25 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:27:25.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:25 smithi022 ceph-mon[37330]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:25.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:25 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:27:25.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:25 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:27:26.387 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:26 smithi022 ceph-mon[37330]: osdmap e77: 8 total, 8 up, 8 in 2024-09-11T07:27:26.387 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:26 smithi022 ceph-mon[37330]: osdmap e78: 8 total, 8 up, 8 in 2024-09-11T07:27:26.387 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:26 smithi022 ceph-mon[29462]: osdmap e77: 8 total, 8 up, 8 in 2024-09-11T07:27:26.387 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:26 smithi022 ceph-mon[29462]: osdmap e78: 8 total, 8 up, 8 in 2024-09-11T07:27:26.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:26 smithi104 ceph-mon[33507]: osdmap e77: 8 total, 8 up, 8 in 2024-09-11T07:27:26.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:26 smithi104 ceph-mon[33507]: osdmap e78: 8 total, 8 up, 8 in 2024-09-11T07:27:27.155 INFO:teuthology.orchestra.run.smithi022.stdout:158913789976 2024-09-11T07:27:27.155 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-11T07:27:27.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:27 smithi022 ceph-mon[29462]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:27:27.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:27 smithi022 ceph-mon[37330]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:27:27.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:27 smithi104 ceph-mon[33507]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:27:29.092 INFO:teuthology.orchestra.run.smithi022.stdout:55834574889 2024-09-11T07:27:29.092 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-11T07:27:29.269 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:27:28 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:27:29.374 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:29 smithi022 ceph-mon[29462]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 680 B/s rd, 0 op/s 2024-09-11T07:27:29.374 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:29 smithi022 ceph-mon[37330]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 680 B/s rd, 0 op/s 2024-09-11T07:27:29.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:29 smithi104 ceph-mon[33507]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 680 B/s rd, 0 op/s 2024-09-11T07:27:30.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:30 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:27:30.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:30 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:27:30.486 INFO:teuthology.orchestra.run.smithi022.stdout:188978561043 2024-09-11T07:27:30.486 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-11T07:27:30.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:30 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:27:31.347 INFO:teuthology.orchestra.run.smithi022.stdout:223338299407 2024-09-11T07:27:31.347 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-11T07:27:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:31 smithi104 ceph-mon[33507]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:31.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:31 smithi022 ceph-mon[29462]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:31.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:31 smithi022 ceph-mon[37330]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:31.867 INFO:teuthology.orchestra.run.smithi022.stdout:133143986203 2024-09-11T07:27:31.867 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-11T07:27:32.293 INFO:teuthology.orchestra.run.smithi022.stdout:77309411368 2024-09-11T07:27:32.294 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-11T07:27:32.368 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:32.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:32 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:27:32.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:32 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:27:32.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:32 smithi022 ceph-mon[29462]: osdmap e79: 8 total, 8 up, 8 in 2024-09-11T07:27:32.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:32 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:27:32.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:32 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:27:32.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:32 smithi022 ceph-mon[37330]: osdmap e79: 8 total, 8 up, 8 in 2024-09-11T07:27:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:32 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:27:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:32 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:27:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:32 smithi104 ceph-mon[33507]: osdmap e79: 8 total, 8 up, 8 in 2024-09-11T07:27:32.899 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:33.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:33 smithi022 ceph-mon[29462]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 766 B/s rd, 0 op/s 2024-09-11T07:27:33.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:33 smithi022 ceph-mon[37330]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 766 B/s rd, 0 op/s 2024-09-11T07:27:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:33 smithi104 ceph-mon[33507]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 766 B/s rd, 0 op/s 2024-09-11T07:27:33.589 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:33.589 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:34.297 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:27:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:27:34.407 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:34.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:34 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:27:34.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:34 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:27:34.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:34 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:27:34.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:34 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:27:34.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:34 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:27:34.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:34 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:27:35.359 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:35 smithi022 ceph-mon[29462]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:35.360 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:35 smithi022 ceph-mon[29462]: 2.1a scrub starts 2024-09-11T07:27:35.360 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:35 smithi022 ceph-mon[29462]: 2.1a scrub ok 2024-09-11T07:27:35.360 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:35 smithi022 ceph-mon[29462]: 4.6 scrub starts 2024-09-11T07:27:35.360 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:35 smithi022 ceph-mon[29462]: 4.6 scrub ok 2024-09-11T07:27:35.360 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:35 smithi022 ceph-mon[29462]: osdmap e80: 8 total, 8 up, 8 in 2024-09-11T07:27:35.360 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:35 smithi022 ceph-mon[37330]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:35.360 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:35 smithi022 ceph-mon[37330]: 2.1a scrub starts 2024-09-11T07:27:35.360 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:35 smithi022 ceph-mon[37330]: 2.1a scrub ok 2024-09-11T07:27:35.360 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:35 smithi022 ceph-mon[37330]: 4.6 scrub starts 2024-09-11T07:27:35.360 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:35 smithi022 ceph-mon[37330]: 4.6 scrub ok 2024-09-11T07:27:35.361 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:35 smithi022 ceph-mon[37330]: osdmap e80: 8 total, 8 up, 8 in 2024-09-11T07:27:35.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:35 smithi104 ceph-mon[33507]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:35.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:35 smithi104 ceph-mon[33507]: 2.1a scrub starts 2024-09-11T07:27:35.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:35 smithi104 ceph-mon[33507]: 2.1a scrub ok 2024-09-11T07:27:35.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:35 smithi104 ceph-mon[33507]: 4.6 scrub starts 2024-09-11T07:27:35.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:35 smithi104 ceph-mon[33507]: 4.6 scrub ok 2024-09-11T07:27:35.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:35 smithi104 ceph-mon[33507]: osdmap e80: 8 total, 8 up, 8 in 2024-09-11T07:27:36.561 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:36.561 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:36.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:36 smithi022 ceph-mon[37330]: 2.8 scrub starts 2024-09-11T07:27:36.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:36 smithi022 ceph-mon[37330]: 2.8 scrub ok 2024-09-11T07:27:36.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:36 smithi022 ceph-mon[37330]: 2.9 scrub starts 2024-09-11T07:27:36.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:36 smithi022 ceph-mon[37330]: 2.9 scrub ok 2024-09-11T07:27:36.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:36 smithi022 ceph-mon[37330]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:36.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:36 smithi022 ceph-mon[29462]: 2.8 scrub starts 2024-09-11T07:27:36.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:36 smithi022 ceph-mon[29462]: 2.8 scrub ok 2024-09-11T07:27:36.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:36 smithi022 ceph-mon[29462]: 2.9 scrub starts 2024-09-11T07:27:36.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:36 smithi022 ceph-mon[29462]: 2.9 scrub ok 2024-09-11T07:27:36.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:36 smithi022 ceph-mon[29462]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:36 smithi104 ceph-mon[33507]: 2.8 scrub starts 2024-09-11T07:27:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:36 smithi104 ceph-mon[33507]: 2.8 scrub ok 2024-09-11T07:27:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:36 smithi104 ceph-mon[33507]: 2.9 scrub starts 2024-09-11T07:27:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:36 smithi104 ceph-mon[33507]: 2.9 scrub ok 2024-09-11T07:27:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:36 smithi104 ceph-mon[33507]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:37.746 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:37 smithi022 ceph-mon[29462]: 2.1e scrub starts 2024-09-11T07:27:37.746 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:37 smithi022 ceph-mon[29462]: 2.1e scrub ok 2024-09-11T07:27:37.746 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:37 smithi022 ceph-mon[37330]: 2.1e scrub starts 2024-09-11T07:27:37.746 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:37 smithi022 ceph-mon[37330]: 2.1e scrub ok 2024-09-11T07:27:37.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:37 smithi104 ceph-mon[33507]: 2.1e scrub starts 2024-09-11T07:27:37.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:37 smithi104 ceph-mon[33507]: 2.1e scrub ok 2024-09-11T07:27:38.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:38 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:27:38.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:38 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:27:38.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:38 smithi104 ceph-mon[33507]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:27:38.782 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:27:38.902 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:38 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:27:38.902 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:38 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:27:38.902 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:38 smithi022 ceph-mon[29462]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:27:38.903 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:27:38.903 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:38 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:27:38.903 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:38 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:27:38.903 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:38 smithi022 ceph-mon[37330]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:27:38.903 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:27:39.279 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:27:38 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:27:39.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:39 smithi022 ceph-mon[29462]: osdmap e81: 8 total, 8 up, 8 in 2024-09-11T07:27:39.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:39 smithi022 ceph-mon[29462]: 2.19 scrub starts 2024-09-11T07:27:39.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:39 smithi022 ceph-mon[29462]: 2.19 scrub ok 2024-09-11T07:27:39.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:39 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:27:39.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:39 smithi022 ceph-mon[37330]: osdmap e81: 8 total, 8 up, 8 in 2024-09-11T07:27:39.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:39 smithi022 ceph-mon[37330]: 2.19 scrub starts 2024-09-11T07:27:39.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:39 smithi022 ceph-mon[37330]: 2.19 scrub ok 2024-09-11T07:27:39.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:39 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:27:39.750 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:39.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:39 smithi104 ceph-mon[33507]: osdmap e81: 8 total, 8 up, 8 in 2024-09-11T07:27:39.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:39 smithi104 ceph-mon[33507]: 2.19 scrub starts 2024-09-11T07:27:39.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:39 smithi104 ceph-mon[33507]: 2.19 scrub ok 2024-09-11T07:27:39.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:39 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:27:40.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:40 smithi104 ceph-mon[33507]: 2.7 scrub starts 2024-09-11T07:27:40.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:40 smithi104 ceph-mon[33507]: 2.7 scrub ok 2024-09-11T07:27:40.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:40 smithi104 ceph-mon[33507]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:40.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:40 smithi104 ceph-mon[33507]: osdmap e82: 8 total, 8 up, 8 in 2024-09-11T07:27:40.826 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:40 smithi022 ceph-mon[29462]: 2.7 scrub starts 2024-09-11T07:27:40.826 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:40 smithi022 ceph-mon[29462]: 2.7 scrub ok 2024-09-11T07:27:40.826 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:40 smithi022 ceph-mon[29462]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:40.826 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:40 smithi022 ceph-mon[29462]: osdmap e82: 8 total, 8 up, 8 in 2024-09-11T07:27:40.827 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:40 smithi022 ceph-mon[37330]: 2.7 scrub starts 2024-09-11T07:27:40.827 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:40 smithi022 ceph-mon[37330]: 2.7 scrub ok 2024-09-11T07:27:40.827 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:40 smithi022 ceph-mon[37330]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:40.827 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:40 smithi022 ceph-mon[37330]: osdmap e82: 8 total, 8 up, 8 in 2024-09-11T07:27:41.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:41 smithi022 ceph-mon[29462]: 4.16 scrub starts 2024-09-11T07:27:41.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:41 smithi022 ceph-mon[29462]: 4.16 scrub ok 2024-09-11T07:27:41.501 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:41 smithi022 ceph-mon[37330]: 4.16 scrub starts 2024-09-11T07:27:41.501 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:41 smithi022 ceph-mon[37330]: 4.16 scrub ok 2024-09-11T07:27:41.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:41 smithi104 ceph-mon[33507]: 4.16 scrub starts 2024-09-11T07:27:41.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:41 smithi104 ceph-mon[33507]: 4.16 scrub ok 2024-09-11T07:27:42.120 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:27:42] ENGINE Bus STOPPING 2024-09-11T07:27:42.282 INFO:teuthology.orchestra.run.smithi022.stdout:107374182437 2024-09-11T07:27:42.417 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:27:42] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T07:27:42.417 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:27:42] ENGINE Bus STOPPED 2024-09-11T07:27:42.418 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:27:42] ENGINE Bus STARTING 2024-09-11T07:27:42.418 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:27:42] ENGINE Serving on http://:::9283 2024-09-11T07:27:42.418 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:27:42] ENGINE Bus STARTED 2024-09-11T07:27:42.734 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[37330]: 5.2 scrub starts 2024-09-11T07:27:42.734 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[37330]: 5.2 scrub ok 2024-09-11T07:27:42.734 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[37330]: 2.4 scrub starts 2024-09-11T07:27:42.734 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[37330]: 2.4 scrub ok 2024-09-11T07:27:42.734 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[37330]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 669 B/s rd, 0 op/s 2024-09-11T07:27:42.734 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:42.734 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:42.734 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:42.734 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:42.734 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:42.735 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:42.735 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T07:27:42.735 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[37330]: from='mon.0 v1:172.21.15.22:0/3782157251' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T07:27:42.735 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:42.735 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[37330]: Deploying daemon grafana.a on smithi104 2024-09-11T07:27:42.735 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[37330]: 5.6 scrub starts 2024-09-11T07:27:42.735 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[37330]: 5.6 scrub ok 2024-09-11T07:27:42.735 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2326609687' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T07:27:42.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[29462]: 5.2 scrub starts 2024-09-11T07:27:42.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[29462]: 5.2 scrub ok 2024-09-11T07:27:42.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[29462]: 2.4 scrub starts 2024-09-11T07:27:42.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[29462]: 2.4 scrub ok 2024-09-11T07:27:42.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[29462]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 669 B/s rd, 0 op/s 2024-09-11T07:27:42.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:42.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:42.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:42.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:42.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:42.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:42.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T07:27:42.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[29462]: from='mon.0 v1:172.21.15.22:0/3782157251' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T07:27:42.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:42.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[29462]: Deploying daemon grafana.a on smithi104 2024-09-11T07:27:42.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[29462]: 5.6 scrub starts 2024-09-11T07:27:42.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[29462]: 5.6 scrub ok 2024-09-11T07:27:42.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:42 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2326609687' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T07:27:42.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:42 smithi104 ceph-mon[33507]: 5.2 scrub starts 2024-09-11T07:27:42.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:42 smithi104 ceph-mon[33507]: 5.2 scrub ok 2024-09-11T07:27:42.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:42 smithi104 ceph-mon[33507]: 2.4 scrub starts 2024-09-11T07:27:42.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:42 smithi104 ceph-mon[33507]: 2.4 scrub ok 2024-09-11T07:27:42.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:42 smithi104 ceph-mon[33507]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 669 B/s rd, 0 op/s 2024-09-11T07:27:42.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:42 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:42.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:42 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:42.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:42 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:42.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:42 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:42.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:42 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:42.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:42 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:42.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:42 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T07:27:42.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:42 smithi104 ceph-mon[33507]: from='mon.0 v1:172.21.15.22:0/3782157251' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T07:27:42.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:42 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:42.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:42 smithi104 ceph-mon[33507]: Deploying daemon grafana.a on smithi104 2024-09-11T07:27:42.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:42 smithi104 ceph-mon[33507]: 5.6 scrub starts 2024-09-11T07:27:42.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:42 smithi104 ceph-mon[33507]: 5.6 scrub ok 2024-09-11T07:27:42.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:42 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2326609687' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T07:27:42.909 INFO:teuthology.orchestra.run.smithi022.stdout:55834574893 2024-09-11T07:27:43.111 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182432 got 107374182437 for osd.3 2024-09-11T07:27:43.111 DEBUG:teuthology.parallel:result is None 2024-09-11T07:27:43.774 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:43 smithi022 ceph-mon[29462]: 6.12 scrub starts 2024-09-11T07:27:43.774 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:43 smithi022 ceph-mon[29462]: 6.12 scrub ok 2024-09-11T07:27:43.774 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:43 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/645615634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T07:27:43.774 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:43 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:43.775 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:43 smithi022 ceph-mon[37330]: 6.12 scrub starts 2024-09-11T07:27:43.775 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:43 smithi022 ceph-mon[37330]: 6.12 scrub ok 2024-09-11T07:27:43.775 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:43 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/645615634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T07:27:43.775 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:43 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:43.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:43 smithi104 ceph-mon[33507]: 6.12 scrub starts 2024-09-11T07:27:43.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:43 smithi104 ceph-mon[33507]: 6.12 scrub ok 2024-09-11T07:27:43.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:43 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/645615634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T07:27:43.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:43 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:44.167 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:27:43 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[69934]: ts=2024-09-11T07:27:43.829Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000536762s 2024-09-11T07:27:44.447 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:27:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:27:44.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:44 smithi022 ceph-mon[29462]: 5.16 deep-scrub starts 2024-09-11T07:27:44.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:44 smithi022 ceph-mon[29462]: 5.16 deep-scrub ok 2024-09-11T07:27:44.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:44 smithi022 ceph-mon[29462]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:44.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:44 smithi022 ceph-mon[37330]: 5.16 deep-scrub starts 2024-09-11T07:27:44.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:44 smithi022 ceph-mon[37330]: 5.16 deep-scrub ok 2024-09-11T07:27:44.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:44 smithi022 ceph-mon[37330]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:44.720 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574889 got 55834574893 for osd.1 2024-09-11T07:27:44.721 DEBUG:teuthology.parallel:result is None 2024-09-11T07:27:44.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:44 smithi104 ceph-mon[33507]: 5.16 deep-scrub starts 2024-09-11T07:27:44.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:44 smithi104 ceph-mon[33507]: 5.16 deep-scrub ok 2024-09-11T07:27:44.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:44 smithi104 ceph-mon[33507]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:45.356 INFO:teuthology.orchestra.run.smithi022.stdout:188978561047 2024-09-11T07:27:45.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:45 smithi022 ceph-mon[37330]: 5.4 scrub starts 2024-09-11T07:27:45.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:45 smithi022 ceph-mon[37330]: 5.4 scrub ok 2024-09-11T07:27:45.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:45 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/323066094' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T07:27:45.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:45 smithi022 ceph-mon[29462]: 5.4 scrub starts 2024-09-11T07:27:45.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:45 smithi022 ceph-mon[29462]: 5.4 scrub ok 2024-09-11T07:27:45.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:45 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/323066094' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T07:27:45.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:45 smithi104 ceph-mon[33507]: 5.4 scrub starts 2024-09-11T07:27:45.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:45 smithi104 ceph-mon[33507]: 5.4 scrub ok 2024-09-11T07:27:45.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:45 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/323066094' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T07:27:46.662 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:46 smithi022 ceph-mon[37330]: 2.1b scrub starts 2024-09-11T07:27:46.662 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:46 smithi022 ceph-mon[37330]: 2.1b scrub ok 2024-09-11T07:27:46.662 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:46 smithi022 ceph-mon[37330]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:46.662 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:46 smithi022 ceph-mon[29462]: 2.1b scrub starts 2024-09-11T07:27:46.662 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:46 smithi022 ceph-mon[29462]: 2.1b scrub ok 2024-09-11T07:27:46.662 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:46 smithi022 ceph-mon[29462]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:46.724 INFO:teuthology.orchestra.run.smithi022.stdout:34359738420 2024-09-11T07:27:46.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:46 smithi104 ceph-mon[33507]: 2.1b scrub starts 2024-09-11T07:27:46.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:46 smithi104 ceph-mon[33507]: 2.1b scrub ok 2024-09-11T07:27:46.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:46 smithi104 ceph-mon[33507]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:47.033 INFO:teuthology.orchestra.run.smithi022.stdout:158913789981 2024-09-11T07:27:47.455 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:47 smithi022 ceph-mon[29462]: 2.11 scrub starts 2024-09-11T07:27:47.455 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:47 smithi022 ceph-mon[29462]: 2.11 scrub ok 2024-09-11T07:27:47.455 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:47 smithi022 ceph-mon[29462]: 2.f scrub starts 2024-09-11T07:27:47.455 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:47 smithi022 ceph-mon[29462]: 2.f scrub ok 2024-09-11T07:27:47.455 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:47 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1925033877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T07:27:47.455 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:47 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1073143549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T07:27:47.455 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:47 smithi022 ceph-mon[29462]: 6.c scrub starts 2024-09-11T07:27:47.566 INFO:teuthology.orchestra.run.smithi022.stdout:223338299411 2024-09-11T07:27:47.566 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:47 smithi022 ceph-mon[29462]: 6.c scrub ok 2024-09-11T07:27:47.566 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:47 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3673830856' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T07:27:47.566 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:47 smithi022 ceph-mon[29462]: 5.d scrub starts 2024-09-11T07:27:47.566 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:47 smithi022 ceph-mon[29462]: 5.d scrub ok 2024-09-11T07:27:47.566 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:47 smithi022 ceph-mon[37330]: 2.11 scrub starts 2024-09-11T07:27:47.567 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:47 smithi022 ceph-mon[37330]: 2.11 scrub ok 2024-09-11T07:27:47.567 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:47 smithi022 ceph-mon[37330]: 2.f scrub starts 2024-09-11T07:27:47.567 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:47 smithi022 ceph-mon[37330]: 2.f scrub ok 2024-09-11T07:27:47.567 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:47 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1925033877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T07:27:47.567 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:47 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1073143549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T07:27:47.567 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:47 smithi022 ceph-mon[37330]: 6.c scrub starts 2024-09-11T07:27:47.567 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:47 smithi022 ceph-mon[37330]: 6.c scrub ok 2024-09-11T07:27:47.567 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:47 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3673830856' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T07:27:47.567 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:47 smithi022 ceph-mon[37330]: 5.d scrub starts 2024-09-11T07:27:47.567 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:47 smithi022 ceph-mon[37330]: 5.d scrub ok 2024-09-11T07:27:47.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:47 smithi104 ceph-mon[33507]: 2.11 scrub starts 2024-09-11T07:27:47.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:47 smithi104 ceph-mon[33507]: 2.11 scrub ok 2024-09-11T07:27:47.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:47 smithi104 ceph-mon[33507]: 2.f scrub starts 2024-09-11T07:27:47.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:47 smithi104 ceph-mon[33507]: 2.f scrub ok 2024-09-11T07:27:47.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:47 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1925033877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T07:27:47.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:47 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1073143549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T07:27:47.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:47 smithi104 ceph-mon[33507]: 6.c scrub starts 2024-09-11T07:27:47.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:47 smithi104 ceph-mon[33507]: 6.c scrub ok 2024-09-11T07:27:47.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:47 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3673830856' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T07:27:47.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:47 smithi104 ceph-mon[33507]: 5.d scrub starts 2024-09-11T07:27:47.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:47 smithi104 ceph-mon[33507]: 5.d scrub ok 2024-09-11T07:27:47.862 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738415 got 34359738420 for osd.0 2024-09-11T07:27:47.862 DEBUG:teuthology.parallel:result is None 2024-09-11T07:27:47.988 INFO:teuthology.orchestra.run.smithi022.stdout:77309411371 2024-09-11T07:27:48.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:48 smithi022 ceph-mon[29462]: 6.15 scrub starts 2024-09-11T07:27:48.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:48 smithi022 ceph-mon[29462]: 6.15 scrub ok 2024-09-11T07:27:48.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:48 smithi022 ceph-mon[29462]: 4.1a scrub starts 2024-09-11T07:27:48.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:48 smithi022 ceph-mon[29462]: 4.1a scrub ok 2024-09-11T07:27:48.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:48 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1207608302' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T07:27:48.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:48 smithi022 ceph-mon[29462]: 4.10 deep-scrub starts 2024-09-11T07:27:48.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:48 smithi022 ceph-mon[29462]: 4.10 deep-scrub ok 2024-09-11T07:27:48.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:48 smithi022 ceph-mon[29462]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:27:48.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:48 smithi022 ceph-mon[37330]: 6.15 scrub starts 2024-09-11T07:27:48.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:48 smithi022 ceph-mon[37330]: 6.15 scrub ok 2024-09-11T07:27:48.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:48 smithi022 ceph-mon[37330]: 4.1a scrub starts 2024-09-11T07:27:48.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:48 smithi022 ceph-mon[37330]: 4.1a scrub ok 2024-09-11T07:27:48.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:48 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1207608302' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T07:27:48.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:48 smithi022 ceph-mon[37330]: 4.10 deep-scrub starts 2024-09-11T07:27:48.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:48 smithi022 ceph-mon[37330]: 4.10 deep-scrub ok 2024-09-11T07:27:48.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:48 smithi022 ceph-mon[37330]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:27:48.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:48 smithi104 ceph-mon[33507]: 6.15 scrub starts 2024-09-11T07:27:48.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:48 smithi104 ceph-mon[33507]: 6.15 scrub ok 2024-09-11T07:27:48.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:48 smithi104 ceph-mon[33507]: 4.1a scrub starts 2024-09-11T07:27:48.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:48 smithi104 ceph-mon[33507]: 4.1a scrub ok 2024-09-11T07:27:48.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:48 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1207608302' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T07:27:48.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:48 smithi104 ceph-mon[33507]: 4.10 deep-scrub starts 2024-09-11T07:27:48.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:48 smithi104 ceph-mon[33507]: 4.10 deep-scrub ok 2024-09-11T07:27:48.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:48 smithi104 ceph-mon[33507]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:27:49.287 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:27:48 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:27:49.624 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:49 smithi022 ceph-mon[29462]: 2.3 scrub starts 2024-09-11T07:27:49.624 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:49 smithi022 ceph-mon[29462]: 2.3 scrub ok 2024-09-11T07:27:49.624 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:49 smithi022 ceph-mon[29462]: 4.13 scrub starts 2024-09-11T07:27:49.624 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:49 smithi022 ceph-mon[29462]: 4.13 scrub ok 2024-09-11T07:27:49.624 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:49 smithi022 ceph-mon[29462]: 4.17 deep-scrub starts 2024-09-11T07:27:49.624 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:49 smithi022 ceph-mon[29462]: 4.17 deep-scrub ok 2024-09-11T07:27:49.624 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:49 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:27:49.624 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:49 smithi022 ceph-mon[37330]: 2.3 scrub starts 2024-09-11T07:27:49.624 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:49 smithi022 ceph-mon[37330]: 2.3 scrub ok 2024-09-11T07:27:49.625 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:49 smithi022 ceph-mon[37330]: 4.13 scrub starts 2024-09-11T07:27:49.625 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:49 smithi022 ceph-mon[37330]: 4.13 scrub ok 2024-09-11T07:27:49.625 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:49 smithi022 ceph-mon[37330]: 4.17 deep-scrub starts 2024-09-11T07:27:49.625 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:49 smithi022 ceph-mon[37330]: 4.17 deep-scrub ok 2024-09-11T07:27:49.625 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:49 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:27:49.652 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561043 got 188978561047 for osd.6 2024-09-11T07:27:49.652 DEBUG:teuthology.parallel:result is None 2024-09-11T07:27:49.770 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:49 smithi104 ceph-mon[33507]: 2.3 scrub starts 2024-09-11T07:27:49.770 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:49 smithi104 ceph-mon[33507]: 2.3 scrub ok 2024-09-11T07:27:49.770 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:49 smithi104 ceph-mon[33507]: 4.13 scrub starts 2024-09-11T07:27:49.770 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:49 smithi104 ceph-mon[33507]: 4.13 scrub ok 2024-09-11T07:27:49.770 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:49 smithi104 ceph-mon[33507]: 4.17 deep-scrub starts 2024-09-11T07:27:49.770 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:49 smithi104 ceph-mon[33507]: 4.17 deep-scrub ok 2024-09-11T07:27:49.770 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:49 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:27:49.778 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411368 got 77309411371 for osd.2 2024-09-11T07:27:49.778 DEBUG:teuthology.parallel:result is None 2024-09-11T07:27:50.192 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789976 got 158913789981 for osd.5 2024-09-11T07:27:50.192 DEBUG:teuthology.parallel:result is None 2024-09-11T07:27:50.760 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:50 smithi104 ceph-mon[33507]: 4.5 scrub starts 2024-09-11T07:27:50.760 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:50 smithi104 ceph-mon[33507]: 4.5 scrub ok 2024-09-11T07:27:50.761 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:50 smithi104 ceph-mon[33507]: 6.16 scrub starts 2024-09-11T07:27:50.761 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:50 smithi104 ceph-mon[33507]: 6.16 scrub ok 2024-09-11T07:27:50.761 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:50 smithi104 ceph-mon[33507]: 2.15 deep-scrub starts 2024-09-11T07:27:50.761 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:50 smithi104 ceph-mon[33507]: 2.15 deep-scrub ok 2024-09-11T07:27:50.761 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:50 smithi104 ceph-mon[33507]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:27:50.761 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:50 smithi104 ceph-mon[33507]: 2.e deep-scrub starts 2024-09-11T07:27:50.761 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:50 smithi104 ceph-mon[33507]: 2.e deep-scrub ok 2024-09-11T07:27:50.792 INFO:teuthology.orchestra.run.smithi022.stdout:133143986210 2024-09-11T07:27:50.793 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:50 smithi022 ceph-mon[29462]: 4.5 scrub starts 2024-09-11T07:27:50.793 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:50 smithi022 ceph-mon[29462]: 4.5 scrub ok 2024-09-11T07:27:50.793 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:50 smithi022 ceph-mon[29462]: 6.16 scrub starts 2024-09-11T07:27:50.793 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:50 smithi022 ceph-mon[29462]: 6.16 scrub ok 2024-09-11T07:27:50.793 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:50 smithi022 ceph-mon[29462]: 2.15 deep-scrub starts 2024-09-11T07:27:50.793 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:50 smithi022 ceph-mon[29462]: 2.15 deep-scrub ok 2024-09-11T07:27:50.793 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:50 smithi022 ceph-mon[29462]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:27:50.793 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:50 smithi022 ceph-mon[29462]: 2.e deep-scrub starts 2024-09-11T07:27:50.793 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:50 smithi022 ceph-mon[29462]: 2.e deep-scrub ok 2024-09-11T07:27:50.793 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:50 smithi022 ceph-mon[37330]: 4.5 scrub starts 2024-09-11T07:27:50.794 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:50 smithi022 ceph-mon[37330]: 4.5 scrub ok 2024-09-11T07:27:50.794 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:50 smithi022 ceph-mon[37330]: 6.16 scrub starts 2024-09-11T07:27:50.794 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:50 smithi022 ceph-mon[37330]: 6.16 scrub ok 2024-09-11T07:27:50.794 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:50 smithi022 ceph-mon[37330]: 2.15 deep-scrub starts 2024-09-11T07:27:50.794 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:50 smithi022 ceph-mon[37330]: 2.15 deep-scrub ok 2024-09-11T07:27:50.794 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:50 smithi022 ceph-mon[37330]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:27:50.794 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:50 smithi022 ceph-mon[37330]: 2.e deep-scrub starts 2024-09-11T07:27:50.794 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:50 smithi022 ceph-mon[37330]: 2.e deep-scrub ok 2024-09-11T07:27:51.116 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299407 got 223338299411 for osd.7 2024-09-11T07:27:51.117 DEBUG:teuthology.parallel:result is None 2024-09-11T07:27:51.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:51 smithi104 ceph-mon[33507]: 5.19 deep-scrub starts 2024-09-11T07:27:51.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:51 smithi104 ceph-mon[33507]: 5.19 deep-scrub ok 2024-09-11T07:27:51.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:51 smithi104 ceph-mon[33507]: 4.19 deep-scrub starts 2024-09-11T07:27:51.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:51 smithi104 ceph-mon[33507]: 4.19 deep-scrub ok 2024-09-11T07:27:51.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:51 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/176931120' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T07:27:51.812 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:51 smithi022 ceph-mon[29462]: 5.19 deep-scrub starts 2024-09-11T07:27:51.812 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:51 smithi022 ceph-mon[29462]: 5.19 deep-scrub ok 2024-09-11T07:27:51.812 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:51 smithi022 ceph-mon[29462]: 4.19 deep-scrub starts 2024-09-11T07:27:51.813 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:51 smithi022 ceph-mon[29462]: 4.19 deep-scrub ok 2024-09-11T07:27:51.813 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:51 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/176931120' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T07:27:51.813 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:51 smithi022 ceph-mon[37330]: 5.19 deep-scrub starts 2024-09-11T07:27:51.813 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:51 smithi022 ceph-mon[37330]: 5.19 deep-scrub ok 2024-09-11T07:27:51.813 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:51 smithi022 ceph-mon[37330]: 4.19 deep-scrub starts 2024-09-11T07:27:51.813 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:51 smithi022 ceph-mon[37330]: 4.19 deep-scrub ok 2024-09-11T07:27:51.813 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:51 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/176931120' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T07:27:51.842 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986203 got 133143986210 for osd.4 2024-09-11T07:27:51.842 DEBUG:teuthology.parallel:result is None 2024-09-11T07:27:51.842 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T07:27:51.842 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:27:52.107 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:52.167 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:27:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[69934]: ts=2024-09-11T07:27:51.833Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.004727759s 2024-09-11T07:27:52.471 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:52 smithi104 podman[63626]: 2024-09-11 07:27:52.239577175 +0000 UTC m=+0.014870867 image pull 954c08fa618887fe9a6fe572ae541197c77e625b05f4193df7898f5ac649ada9 quay.io/ceph/ceph-grafana:9.4.7 2024-09-11T07:27:52.541 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:52 smithi022 ceph-mon[29462]: 2.1f scrub starts 2024-09-11T07:27:52.541 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:52 smithi022 ceph-mon[29462]: 2.1f scrub ok 2024-09-11T07:27:52.541 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:52 smithi022 ceph-mon[29462]: 4.2 scrub starts 2024-09-11T07:27:52.541 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:52 smithi022 ceph-mon[29462]: 4.2 scrub ok 2024-09-11T07:27:52.541 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:52 smithi022 ceph-mon[29462]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 888 B/s rd, 0 op/s 2024-09-11T07:27:52.541 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:52 smithi022 ceph-mon[37330]: 2.1f scrub starts 2024-09-11T07:27:52.541 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:52 smithi022 ceph-mon[37330]: 2.1f scrub ok 2024-09-11T07:27:52.541 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:52 smithi022 ceph-mon[37330]: 4.2 scrub starts 2024-09-11T07:27:52.541 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:52 smithi022 ceph-mon[37330]: 4.2 scrub ok 2024-09-11T07:27:52.541 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:52 smithi022 ceph-mon[37330]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 888 B/s rd, 0 op/s 2024-09-11T07:27:52.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:52 smithi104 ceph-mon[33507]: 2.1f scrub starts 2024-09-11T07:27:52.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:52 smithi104 ceph-mon[33507]: 2.1f scrub ok 2024-09-11T07:27:52.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:52 smithi104 ceph-mon[33507]: 4.2 scrub starts 2024-09-11T07:27:52.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:52 smithi104 ceph-mon[33507]: 4.2 scrub ok 2024-09-11T07:27:52.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:52 smithi104 ceph-mon[33507]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 888 B/s rd, 0 op/s 2024-09-11T07:27:53.261 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:52 smithi104 podman[63626]: 2024-09-11 07:27:52.941068732 +0000 UTC m=+0.716362415 container create 5ed6f8ca9d7d4edc988ea0ee7d1bff9e1b8d521317573da52ef56357c7e75cb0 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a, release=239.1651231664, distribution-scope=public, summary=Grafana Container configured for Ceph mgr/dashboard integration, name=ubi8, vcs-type=git, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.tags=base rhel8, io.buildah.version=1.32.0, build-date=2022-04-29T11:48:11.055897, io.k8s.display-name=Red Hat Universal Base Image 8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, description=Ceph Grafana Container, maintainer=Paul Cuzner , vendor=Red Hat, Inc., vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.openshift.expose-services=, version=8.5, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., architecture=x86_64, com.redhat.component=ubi8-container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI) 2024-09-11T07:27:53.261 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 podman[63626]: 2024-09-11 07:27:53.184760715 +0000 UTC m=+0.960054412 container init 5ed6f8ca9d7d4edc988ea0ee7d1bff9e1b8d521317573da52ef56357c7e75cb0 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a, io.k8s.display-name=Red Hat Universal Base Image 8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, build-date=2022-04-29T11:48:11.055897, io.buildah.version=1.32.0, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, summary=Grafana Container configured for Ceph mgr/dashboard integration, description=Ceph Grafana Container, io.openshift.expose-services=, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, architecture=x86_64, io.openshift.tags=base rhel8, name=ubi8, vcs-type=git, vendor=Red Hat, Inc., version=8.5, com.redhat.component=ubi8-container, release=239.1651231664, distribution-scope=public, maintainer=Paul Cuzner ) 2024-09-11T07:27:53.261 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 podman[63626]: 2024-09-11 07:27:53.187676967 +0000 UTC m=+0.962970658 container start 5ed6f8ca9d7d4edc988ea0ee7d1bff9e1b8d521317573da52ef56357c7e75cb0 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a, description=Ceph Grafana Container, name=ubi8, distribution-scope=public, io.openshift.tags=base rhel8, vendor=Red Hat, Inc., summary=Grafana Container configured for Ceph mgr/dashboard integration, io.openshift.expose-services=, build-date=2022-04-29T11:48:11.055897, release=239.1651231664, architecture=x86_64, com.redhat.component=ubi8-container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, maintainer=Paul Cuzner , io.buildah.version=1.32.0, vcs-type=git, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, version=8.5, io.k8s.display-name=Red Hat Universal Base Image 8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.) 2024-09-11T07:27:53.514 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-mon[33507]: 2.1c deep-scrub starts 2024-09-11T07:27:53.514 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-mon[33507]: 2.1c deep-scrub ok 2024-09-11T07:27:53.514 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-mon[33507]: 4.1b deep-scrub starts 2024-09-11T07:27:53.514 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-mon[33507]: 4.1b deep-scrub ok 2024-09-11T07:27:53.514 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:27:53.514 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-mon[33507]: 5.8 scrub starts 2024-09-11T07:27:53.514 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-mon[33507]: 5.8 scrub ok 2024-09-11T07:27:53.514 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:53.514 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:53.515 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:53.515 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:53.515 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 bash[63626]: 5ed6f8ca9d7d4edc988ea0ee7d1bff9e1b8d521317573da52ef56357c7e75cb0 2024-09-11T07:27:53.515 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 systemd[1]: Started Ceph grafana.a for 12a285d8-700e-11ef-bcea-c7b262605968. 2024-09-11T07:27:53.516 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=settings t=2024-09-11T07:27:53.346745257Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-11T07:27:53.516 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=settings t=2024-09-11T07:27:53.346964174Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-11T07:27:53.516 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=settings t=2024-09-11T07:27:53.346990405Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-11T07:27:53.516 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=settings t=2024-09-11T07:27:53.347000095Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-11T07:27:53.516 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=settings t=2024-09-11T07:27:53.347006248Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-11T07:27:53.516 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=settings t=2024-09-11T07:27:53.347012317Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-11T07:27:53.516 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=settings t=2024-09-11T07:27:53.347018178Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-11T07:27:53.516 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=settings t=2024-09-11T07:27:53.347026143Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-11T07:27:53.516 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=settings t=2024-09-11T07:27:53.347032111Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-11T07:27:53.516 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=settings t=2024-09-11T07:27:53.347038045Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-11T07:27:53.516 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=settings t=2024-09-11T07:27:53.34704359Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-11T07:27:53.517 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=settings t=2024-09-11T07:27:53.347049086Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-11T07:27:53.517 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=settings t=2024-09-11T07:27:53.34705491Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-11T07:27:53.517 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=settings t=2024-09-11T07:27:53.347060766Z level=info msg="App mode production" 2024-09-11T07:27:53.517 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=sqlstore t=2024-09-11T07:27:53.347106212Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-11T07:27:53.517 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=sqlstore t=2024-09-11T07:27:53.347121922Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-11T07:27:53.517 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.347947164Z level=info msg="Starting DB migrations" 2024-09-11T07:27:53.517 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.348015946Z level=info msg="Executing migration" id="create migration_log table" 2024-09-11T07:27:53.517 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.348727926Z level=info msg="Executing migration" id="create user table" 2024-09-11T07:27:53.517 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.349177821Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-11T07:27:53.517 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.349609116Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-11T07:27:53.517 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.350034896Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-11T07:27:53.517 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.350469899Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-11T07:27:53.517 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.350840144Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-11T07:27:53.518 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.35155356Z level=info msg="Executing migration" id="create user table v2" 2024-09-11T07:27:53.518 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.351908081Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-11T07:27:53.518 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.352296727Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-11T07:27:53.518 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.352694324Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-11T07:27:53.518 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.353035631Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-11T07:27:53.518 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.353378174Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-11T07:27:53.518 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.353759141Z level=info msg="Executing migration" id="Update user table charset" 2024-09-11T07:27:53.518 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.354061257Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-11T07:27:53.518 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.354463136Z level=info msg="Executing migration" id="Add missing user data" 2024-09-11T07:27:53.518 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.354777167Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-11T07:27:53.518 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.355216992Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-11T07:27:53.518 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.355585091Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-11T07:27:53.518 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.35599966Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-11T07:27:53.518 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.357126408Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-11T07:27:53.519 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.357543814Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-11T07:27:53.519 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.357941259Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-11T07:27:53.519 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.358317076Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-11T07:27:53.519 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.360802823Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-11T07:27:53.519 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.36378612Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-11T07:27:53.519 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.366390274Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-11T07:27:53.519 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.369284772Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-11T07:27:53.519 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.37234145Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-11T07:27:53.519 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.375402681Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-11T07:27:53.519 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.37838681Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-11T07:27:53.519 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.381178569Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-11T07:27:53.519 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.384115564Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-11T07:27:53.519 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.387034126Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-11T07:27:53.519 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.389838108Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-11T07:27:53.520 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.392611416Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-11T07:27:53.520 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.395279324Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-11T07:27:53.520 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.397963355Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-11T07:27:53.520 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.39835143Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-11T07:27:53.520 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.401097096Z level=info msg="Executing migration" id="create star table" 2024-09-11T07:27:53.520 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.401529369Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-11T07:27:53.520 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.403898788Z level=info msg="Executing migration" id="create org table v1" 2024-09-11T07:27:53.520 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.406462818Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-11T07:27:53.520 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.409160415Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-11T07:27:53.520 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.411797926Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-11T07:27:53.520 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.414430431Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-11T07:27:53.520 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.414997436Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-11T07:27:53.520 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.415437928Z level=info msg="Executing migration" id="Update org table charset" 2024-09-11T07:27:53.520 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.418075429Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-11T07:27:53.521 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.418401906Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-11T07:27:53.521 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.42089093Z level=info msg="Executing migration" id="create dashboard table" 2024-09-11T07:27:53.521 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.421532493Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-11T07:27:53.521 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.421955909Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-11T07:27:53.521 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.424835229Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-11T07:27:53.521 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.427540741Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-11T07:27:53.521 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.428266114Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-11T07:27:53.521 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.42870525Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-11T07:27:53.521 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.429579255Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-11T07:27:53.521 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.430414252Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-11T07:27:53.521 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.431399121Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-11T07:27:53.521 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.432002957Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-11T07:27:53.521 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.432946475Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-11T07:27:53.521 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.433619341Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-11T07:27:53.522 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.434440223Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-11T07:27:53.522 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.435008295Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-11T07:27:53.522 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.435875017Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-11T07:27:53.522 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.43687329Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-11T07:27:53.522 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.437597793Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-11T07:27:53.522 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.438184685Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-11T07:27:53.522 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.439028597Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-11T07:27:53.522 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.43990101Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-11T07:27:53.522 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.440715203Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-11T07:27:53.522 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.441079954Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-11T07:27:53.522 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.441872641Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-11T07:27:53.522 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.442412029Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-11T07:27:53.522 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.443231217Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-11T07:27:53.522 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.443798105Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-11T07:27:53.522 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.444154953Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-11T07:27:53.523 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.445003415Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-11T07:27:53.523 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.445849209Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-11T07:27:53.523 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.446683422Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-11T07:27:53.523 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.447171661Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-11T07:27:53.523 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.448020621Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-11T07:27:53.523 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.448985834Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-11T07:27:53.523 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.449837379Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-11T07:27:53.523 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.450309779Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-11T07:27:53.524 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.451288439Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-11T07:27:53.524 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.452015644Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-11T07:27:53.524 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.452871376Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-11T07:27:53.524 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.453438745Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-11T07:27:53.524 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.454265902Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-11T07:27:53.524 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.454629319Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-11T07:27:53.524 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.457343003Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-11T07:27:53.524 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.460166262Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-11T07:27:53.524 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.46077024Z level=info msg="Executing migration" id="create data_source table" 2024-09-11T07:27:53.524 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.463625386Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-11T07:27:53.524 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.46646695Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-11T07:27:53.524 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.46692848Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-11T07:27:53.524 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.467378059Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-11T07:27:53.524 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.470153837Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-11T07:27:53.524 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.473259804Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-11T07:27:53.525 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.475930259Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-11T07:27:53.525 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.47885762Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-11T07:27:53.525 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.481664321Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-11T07:27:53.525 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.484497149Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-11T07:27:53.525 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.485117935Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-11T07:27:53.525 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.48797403Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-11T07:27:53.525 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.490926914Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-11T07:27:53.525 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.493792073Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-11T07:27:53.525 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.494506516Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-11T07:27:53.525 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.497440025Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-11T07:27:53.525 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.497784821Z level=info msg="Executing migration" id="Add uid column" 2024-09-11T07:27:53.525 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.500368841Z level=info msg="Executing migration" id="Update uid value" 2024-09-11T07:27:53.525 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.503366653Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-11T07:27:53.525 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.503934197Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-11T07:27:53.526 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.50673202Z level=info msg="Executing migration" id="create api_key table" 2024-09-11T07:27:53.526 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.50971096Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-11T07:27:53.526 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.512616499Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-11T07:27:53.526 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.515283941Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-11T07:27:53.606 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:53 smithi022 ceph-mon[29462]: 2.1c deep-scrub starts 2024-09-11T07:27:53.606 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:53 smithi022 ceph-mon[29462]: 2.1c deep-scrub ok 2024-09-11T07:27:53.606 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:53 smithi022 ceph-mon[29462]: 4.1b deep-scrub starts 2024-09-11T07:27:53.606 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:53 smithi022 ceph-mon[29462]: 4.1b deep-scrub ok 2024-09-11T07:27:53.606 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:27:53.606 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:53 smithi022 ceph-mon[29462]: 5.8 scrub starts 2024-09-11T07:27:53.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:53 smithi022 ceph-mon[29462]: 5.8 scrub ok 2024-09-11T07:27:53.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:53.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:53.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:53.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:53.607 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:53 smithi022 ceph-mon[37330]: 2.1c deep-scrub starts 2024-09-11T07:27:53.607 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:53 smithi022 ceph-mon[37330]: 2.1c deep-scrub ok 2024-09-11T07:27:53.607 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:53 smithi022 ceph-mon[37330]: 4.1b deep-scrub starts 2024-09-11T07:27:53.607 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:53 smithi022 ceph-mon[37330]: 4.1b deep-scrub ok 2024-09-11T07:27:53.607 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:27:53.607 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:53 smithi022 ceph-mon[37330]: 5.8 scrub starts 2024-09-11T07:27:53.607 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:53 smithi022 ceph-mon[37330]: 5.8 scrub ok 2024-09-11T07:27:53.607 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:53.608 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:53.608 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:53.608 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:53.764 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.51798096Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-11T07:27:53.765 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.520574383Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-11T07:27:53.765 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.521077416Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-11T07:27:53.765 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.523775393Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-11T07:27:53.765 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.526638671Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-11T07:27:53.765 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.529353045Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-11T07:27:53.765 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.529929568Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-11T07:27:53.765 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.532660795Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-11T07:27:53.765 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.535522657Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-11T07:27:53.765 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.53807505Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-11T07:27:53.765 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.538473958Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-11T07:27:53.765 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.541190401Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-11T07:27:53.765 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.541837522Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-11T07:27:53.765 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.543895748Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-11T07:27:53.766 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.546658904Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-11T07:27:53.766 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.547219467Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-11T07:27:53.766 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.550077378Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-11T07:27:53.766 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.552861909Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-11T07:27:53.766 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.553861223Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-11T07:27:53.766 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.554340383Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-11T07:27:53.766 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.55535886Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-11T07:27:53.766 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.556041595Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-11T07:27:53.766 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.55704299Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-11T07:27:53.766 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.557442336Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-11T07:27:53.766 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.558253468Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-11T07:27:53.766 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.558797692Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-11T07:27:53.766 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.559385151Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-11T07:27:53.766 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.560189254Z level=info msg="Executing migration" id="create quota table v1" 2024-09-11T07:27:53.766 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.560593362Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-11T07:27:53.767 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.561137957Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-11T07:27:53.767 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.561863149Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-11T07:27:53.767 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.562711245Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-11T07:27:53.767 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.563647617Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-11T07:27:53.767 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.564255349Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-11T07:27:53.767 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.565210801Z level=info msg="Executing migration" id="create session table" 2024-09-11T07:27:53.767 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.565945273Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-11T07:27:53.767 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.566904129Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-11T07:27:53.767 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.567215244Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-11T07:27:53.769 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.568081807Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-11T07:27:53.769 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.568929261Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-11T07:27:53.769 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.569930828Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-11T07:27:53.769 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.570331073Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-11T07:27:53.769 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.571360219Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-11T07:27:53.769 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.571689791Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-11T07:27:53.769 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.572081825Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-11T07:27:53.769 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.573050056Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-11T07:27:53.769 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.573767044Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-11T07:27:53.769 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.574730377Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-11T07:27:53.769 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.57560406Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-11T07:27:53.769 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.576427547Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-11T07:27:53.770 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.576715604Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-11T07:27:53.770 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.577678256Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-11T07:27:53.770 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.578633304Z level=info msg="Executing migration" id="create alert table v1" 2024-09-11T07:27:53.770 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.581553377Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-11T07:27:53.770 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.582015722Z level=info msg="Executing migration" id="add index alert state" 2024-09-11T07:27:53.770 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.58490545Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-11T07:27:53.770 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.587744145Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-11T07:27:53.770 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.590715152Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-11T07:27:53.770 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.593641204Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-11T07:27:53.770 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.596402533Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-11T07:27:53.770 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.599457547Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-11T07:27:53.770 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.602135591Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-11T07:27:53.770 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.60261627Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-11T07:27:53.770 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.605601464Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-11T07:27:53.770 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.606014429Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-11T07:27:53.771 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.608631312Z level=info msg="Executing migration" id="Add column is_default" 2024-09-11T07:27:53.771 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.611467784Z level=info msg="Executing migration" id="Add column frequency" 2024-09-11T07:27:53.771 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.612102786Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-11T07:27:53.771 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.615100259Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-11T07:27:53.771 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.617988474Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-11T07:27:53.771 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.620664255Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-11T07:27:53.771 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.621105655Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-11T07:27:53.771 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.623996781Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-11T07:27:53.771 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.624483942Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-11T07:27:53.771 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.626634106Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-11T07:27:53.771 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.629212509Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-11T07:27:53.771 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.629630624Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-11T07:27:53.771 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.632467616Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-11T07:27:53.771 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.635172905Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-11T07:27:53.771 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.63790326Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-11T07:27:53.772 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.638263406Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-11T07:27:53.772 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.641080626Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-11T07:27:53.772 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.643734976Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-11T07:27:53.772 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.646559568Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-11T07:27:53.772 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.646917568Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-11T07:27:53.772 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.64986511Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-11T07:27:53.772 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.652397325Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-11T07:27:53.772 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.655096029Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-11T07:27:53.772 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.655601251Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-11T07:27:53.772 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.656402019Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-11T07:27:53.772 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.657391749Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-11T07:27:53.772 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.658098536Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-11T07:27:53.772 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.65857206Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-11T07:27:53.772 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.659416451Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-11T07:27:53.772 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.660229435Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-11T07:27:53.773 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.661115363Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-11T07:27:53.773 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.661564752Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-11T07:27:53.773 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.662405355Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-11T07:27:53.773 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.6632827Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-11T07:27:53.773 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.664129688Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-11T07:27:53.773 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.664631199Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-11T07:27:53.773 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.66618572Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-11T07:27:53.773 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.667063195Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-11T07:27:53.773 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.667908036Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-11T07:27:53.773 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.668269515Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-11T07:27:53.773 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.669069505Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-11T07:27:53.773 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.669393302Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-11T07:27:53.773 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.67011791Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-11T07:27:53.773 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.670977127Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-11T07:27:53.773 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.671456348Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-11T07:27:53.774 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.672289283Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-11T07:27:53.774 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.673294823Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-11T07:27:53.774 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.674263569Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-11T07:27:53.774 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.674743627Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-11T07:27:53.774 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.675570329Z level=info msg="Executing migration" id="Move region to single row" 2024-09-11T07:27:53.774 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.676416949Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-11T07:27:53.774 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.677273331Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-11T07:27:53.774 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.67775958Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-11T07:27:53.774 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.678600436Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-11T07:27:53.774 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.679478085Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-11T07:27:53.774 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.680354828Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-11T07:27:53.774 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.680853658Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-11T07:27:53.774 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.683561751Z level=info msg="Executing migration" id="create test_data table" 2024-09-11T07:27:53.774 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.68636768Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-11T07:27:53.774 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.687027175Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-11T07:27:53.775 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.687492818Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-11T07:27:53.775 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.690543749Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-11T07:27:53.775 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.693314091Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-11T07:27:53.775 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.696078131Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-11T07:27:53.775 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.696389186Z level=info msg="Executing migration" id="create team table" 2024-09-11T07:27:53.775 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.699349734Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-11T07:27:53.775 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.702178613Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-11T07:27:53.775 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.705057611Z level=info msg="Executing migration" id="create team member table" 2024-09-11T07:27:53.775 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.705529832Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-11T07:27:53.775 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.708245982Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-11T07:27:53.775 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.710918857Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-11T07:27:53.775 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.713676185Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-11T07:27:53.775 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.71442312Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-11T07:27:53.775 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.717119338Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-11T07:27:53.775 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.719901642Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-11T07:27:53.775 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.722640037Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-11T07:27:53.776 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.723146179Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-11T07:27:53.776 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.725624817Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-11T07:27:53.776 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.726105871Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-11T07:27:53.776 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.72878728Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-11T07:27:53.776 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.729357767Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-11T07:27:53.776 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.729808483Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-11T07:27:53.776 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.730273906Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-11T07:27:53.776 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.730655042Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-11T07:27:53.776 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.730971398Z level=info msg="Executing migration" id="create tag table" 2024-09-11T07:27:53.776 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.73140038Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-11T07:27:53.777 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.731881501Z level=info msg="Executing migration" id="create login attempt table" 2024-09-11T07:27:53.777 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.732322697Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-11T07:27:53.777 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.732766837Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-11T07:27:53.777 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.733217256Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-11T07:27:53.777 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.735085576Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-11T07:27:53.777 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.735497851Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-11T07:27:53.777 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.735929453Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-11T07:27:53.777 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.736267371Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-11T07:27:53.777 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.736600106Z level=info msg="Executing migration" id="create user auth table" 2024-09-11T07:27:53.777 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.736987796Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-11T07:27:53.777 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.737439923Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-11T07:27:53.778 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.737712815Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-11T07:27:53.778 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.738498022Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-11T07:27:53.778 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.739267594Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-11T07:27:53.778 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.740007434Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-11T07:27:53.778 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.740773543Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-11T07:27:53.778 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.7412153Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-11T07:27:53.778 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.741920101Z level=info msg="Executing migration" id="create server_lock table" 2024-09-11T07:27:53.778 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.74232098Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-11T07:27:53.778 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.742760221Z level=info msg="Executing migration" id="create user auth token table" 2024-09-11T07:27:53.778 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.74319534Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-11T07:27:53.778 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.743642808Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-11T07:27:53.778 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.744111197Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-11T07:27:53.778 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.744552967Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-11T07:27:53.779 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.745344202Z level=info msg="Executing migration" id="create cache_data table" 2024-09-11T07:27:53.779 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.745774311Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-11T07:27:53.779 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.746232317Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-11T07:27:53.779 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.746668524Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-11T07:27:53.779 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.749355383Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-11T07:27:53.779 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.749675903Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-11T07:27:53.779 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.750002791Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-11T07:27:53.779 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.750433197Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-11T07:27:53.779 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.753098486Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-11T07:27:53.779 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.755993256Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-11T07:27:53.779 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.758498347Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-11T07:27:53.779 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.759186562Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-11T07:27:53.779 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.762041286Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-11T07:27:53.779 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.762894427Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-11T07:27:53.780 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.763591567Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-11T07:27:53.780 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.764430269Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-11T07:27:53.780 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.765283103Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-11T07:27:53.780 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.766131333Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-11T07:27:53.780 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.766987621Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-11T07:27:54.015 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.767849003Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-11T07:27:54.015 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.768687592Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-11T07:27:54.015 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.769528855Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-11T07:27:54.015 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.770419367Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-11T07:27:54.015 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.771241847Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-11T07:27:54.015 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.772124061Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-11T07:27:54.015 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.773024548Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-11T07:27:54.016 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.774277913Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-11T07:27:54.016 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.775147546Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-11T07:27:54.016 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.775990506Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-11T07:27:54.016 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.779855345Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-11T07:27:54.016 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.783149694Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-11T07:27:54.016 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.783997937Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-11T07:27:54.016 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.784972361Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-11T07:27:54.016 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.785873791Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-11T07:27:54.016 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.786331427Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-11T07:27:54.016 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.787153784Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-11T07:27:54.016 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.788110941Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-11T07:27:54.017 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.788842966Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-11T07:27:54.017 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.789781467Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-11T07:27:54.017 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.790759362Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-11T07:27:54.017 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.791616941Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-11T07:27:54.017 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.792499197Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-11T07:27:54.017 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.793368564Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-11T07:27:54.017 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.79422704Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-11T07:27:54.017 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.79505994Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-11T07:27:54.017 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.795942779Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-11T07:27:54.017 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.796804125Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-11T07:27:54.017 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.797685998Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-11T07:27:54.017 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.798547511Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-11T07:27:54.017 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.801514274Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-11T07:27:54.017 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.804368378Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-11T07:27:54.018 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.807125183Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-11T07:27:54.018 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.809898341Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-11T07:27:54.018 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.810560007Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-11T07:27:54.019 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.813625234Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-11T07:27:54.019 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.816366019Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-11T07:27:54.019 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.81913174Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-11T07:27:54.019 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.82205541Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-11T07:27:54.019 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.824806054Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-11T07:27:54.019 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.827877959Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-11T07:27:54.019 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.830540147Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-11T07:27:54.019 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.833531717Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-11T07:27:54.019 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.836341918Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-11T07:27:54.019 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.839264056Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-11T07:27:54.019 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.842202165Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-11T07:27:54.019 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.844882901Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-11T07:27:54.019 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.847640282Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-11T07:27:54.020 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.850384312Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-11T07:27:54.020 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.85311568Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-11T07:27:54.020 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.855969666Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-11T07:27:54.020 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.858576729Z level=info msg="Executing migration" id="create alert_image table" 2024-09-11T07:27:54.020 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.861145116Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-11T07:27:54.020 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.863845362Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-11T07:27:54.020 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.866623204Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-11T07:27:54.020 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.869311613Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-11T07:27:54.020 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.869745069Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-11T07:27:54.020 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.869817262Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-11T07:27:54.020 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.872584326Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-11T07:27:54.020 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.875024909Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-11T07:27:54.020 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.875509621Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-11T07:27:54.020 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.875622878Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-11T07:27:54.020 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.875738605Z level=info msg="alerts found to migrate" alerts=0 2024-09-11T07:27:54.021 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.876504909Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-11T07:27:54.021 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.87725196Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-11T07:27:54.021 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.878073335Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-11T07:27:54.021 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.878922822Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-11T07:27:54.021 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.879799966Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-11T07:27:54.021 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.880652878Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-11T07:27:54.021 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.881527499Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-11T07:27:54.021 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.882510519Z level=info msg="Executing migration" id="create data_keys table" 2024-09-11T07:27:54.021 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.88326437Z level=info msg="Executing migration" id="create secrets table" 2024-09-11T07:27:54.021 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.88612138Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-11T07:27:54.021 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.8904304Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-11T07:27:54.021 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.89335635Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-11T07:27:54.021 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.896333029Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-11T07:27:54.021 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.900885806Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-11T07:27:54.022 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.905178309Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-11T07:27:54.022 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.90807354Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-11T07:27:54.022 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.911060698Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-11T07:27:54.022 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.913755087Z level=info msg="Executing migration" id="create permission table" 2024-09-11T07:27:54.022 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.91666361Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-11T07:27:54.022 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.919643785Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-11T07:27:54.022 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.922797746Z level=info msg="Executing migration" id="create role table" 2024-09-11T07:27:54.022 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.925543646Z level=info msg="Executing migration" id="add column display_name" 2024-09-11T07:27:54.022 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.928481393Z level=info msg="Executing migration" id="add column group_name" 2024-09-11T07:27:54.022 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.931313944Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-11T07:27:54.022 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.934248633Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-11T07:27:54.022 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.936979796Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-11T07:27:54.022 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.939674449Z level=info msg="Executing migration" id="create team role table" 2024-09-11T07:27:54.022 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.942351745Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-11T07:27:54.022 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.945128015Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-11T07:27:54.023 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.947884508Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-11T07:27:54.023 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.950718526Z level=info msg="Executing migration" id="create user role table" 2024-09-11T07:27:54.023 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.953261791Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-11T07:27:54.023 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.955819001Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-11T07:27:54.023 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.958544597Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-11T07:27:54.023 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.961270523Z level=info msg="Executing migration" id="create builtin role table" 2024-09-11T07:27:54.023 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.963887216Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-11T07:27:54.023 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.966552767Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-11T07:27:54.023 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.969498731Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-11T07:27:54.023 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.97202179Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-11T07:27:54.023 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.974949048Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-11T07:27:54.023 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.977572781Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-11T07:27:54.023 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.980270844Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-11T07:27:54.023 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.983162546Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-11T07:27:54.023 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.985896664Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-11T07:27:54.023 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.986648411Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-11T07:27:54.024 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.987660857Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-11T07:27:54.024 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.988500078Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-11T07:27:54.024 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.989391884Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-11T07:27:54.024 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.990243549Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-11T07:27:54.024 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.991176705Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-11T07:27:54.024 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.991907026Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-11T07:27:54.024 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.99296793Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-11T07:27:54.024 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.993603949Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-11T07:27:54.024 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.994600103Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-11T07:27:54.024 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.995322462Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-11T07:27:54.024 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.996310168Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-11T07:27:54.024 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.997472229Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-11T07:27:54.024 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.998330924Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-11T07:27:54.024 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:53.999197833Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-11T07:27:54.025 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.00000886Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-11T07:27:54.025 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.000886333Z level=info msg="Executing migration" id="add correlation config column" 2024-09-11T07:27:54.025 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.001859278Z level=info msg="Executing migration" id="create entity_events table" 2024-09-11T07:27:54.025 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.002739568Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-11T07:27:54.025 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.003697804Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-11T07:27:54.025 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.003847949Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-11T07:27:54.025 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.004545899Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T07:27:54.025 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.004775103Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T07:27:54.025 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.005290811Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-11T07:27:54.025 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.006315587Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-11T07:27:54.025 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.007057944Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-11T07:27:54.025 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.007902288Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T07:27:54.025 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.0089145Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-11T07:27:54.025 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.009638152Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-11T07:27:54.025 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.010493892Z level=info msg="Executing migration" id="Drop public config table" 2024-09-11T07:27:54.026 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.011418424Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-11T07:27:54.026 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.012318801Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-11T07:27:54.026 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.013301378Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-11T07:27:54.026 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.01405619Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-11T07:27:54.026 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.014936681Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-11T07:27:54.031 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:27:54.033 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:27:54.279 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:27:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:27:54.280 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.020004412Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-11T07:27:54.280 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.021593381Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-11T07:27:54.280 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.02459409Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-11T07:27:54.281 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.027529441Z level=info msg="Executing migration" id="add share column" 2024-09-11T07:27:54.281 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.030490045Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-11T07:27:54.281 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.033373044Z level=info msg="Executing migration" id="create file table" 2024-09-11T07:27:54.281 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.036241819Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-11T07:27:54.281 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.039099833Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-11T07:27:54.281 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.041968035Z level=info msg="Executing migration" id="create file_meta table" 2024-09-11T07:27:54.281 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.044933602Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-11T07:27:54.281 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.047692588Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-11T07:27:54.281 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.050682724Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-11T07:27:54.281 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.053374931Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-11T07:27:54.281 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.053722831Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-11T07:27:54.282 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.056603302Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-11T07:27:54.282 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.059297791Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-11T07:27:54.282 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.06220106Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-11T07:27:54.282 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.062689304Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-11T07:27:54.282 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.065669334Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-11T07:27:54.282 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.068439039Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-11T07:27:54.282 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.071210762Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-11T07:27:54.282 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.072340012Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-11T07:27:54.282 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.075159202Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-11T07:27:54.282 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.077939086Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-11T07:27:54.282 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.087813582Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-11T07:27:54.282 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.088295337Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-11T07:27:54.282 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.090989562Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-11T07:27:54.282 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.09451172Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-11T07:27:54.283 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.097208392Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-11T07:27:54.283 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.097549258Z level=info msg="Executing migration" id="create folder table" 2024-09-11T07:27:54.283 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.100234902Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-11T07:27:54.283 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.100717925Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-11T07:27:54.283 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.103025546Z level=info msg="Executing migration" id="Update folder title length" 2024-09-11T07:27:54.283 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.105743458Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-11T07:27:54.283 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=migrator t=2024-09-11T07:27:54.106193708Z level=info msg="migrations completed" performed=485 skipped=0 duration=758.196011ms 2024-09-11T07:27:54.283 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=sqlstore t=2024-09-11T07:27:54.106568086Z level=info msg="Created default organization" 2024-09-11T07:27:54.283 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=plugin.loader t=2024-09-11T07:27:54.165219841Z level=info msg="Plugin registered" pluginID=input 2024-09-11T07:27:54.283 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=plugin.loader t=2024-09-11T07:27:54.19234973Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-11T07:27:54.283 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=plugin.loader t=2024-09-11T07:27:54.1923684Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-11T07:27:54.283 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=secrets t=2024-09-11T07:27:54.19240027Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-11T07:27:54.283 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=query_data t=2024-09-11T07:27:54.193984526Z level=info msg="Query Service initialization" 2024-09-11T07:27:54.283 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=live.push_http t=2024-09-11T07:27:54.19635407Z level=info msg="Live Push Gateway initialization" 2024-09-11T07:27:54.592 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-mon[33507]: Deploying daemon node-exporter.a on smithi022 2024-09-11T07:27:54.592 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-mon[33507]: 2.d scrub starts 2024-09-11T07:27:54.592 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-mon[33507]: 2.d scrub ok 2024-09-11T07:27:54.592 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-mon[33507]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:54.593 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-mon[33507]: from='client.14742 v1:172.21.15.22:0/3758027991' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:27:54.593 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-mon[33507]: 2.c scrub starts 2024-09-11T07:27:54.593 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-mon[33507]: 2.c scrub ok 2024-09-11T07:27:54.744 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:54 smithi022 ceph-mon[29462]: Deploying daemon node-exporter.a on smithi022 2024-09-11T07:27:54.744 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:54 smithi022 ceph-mon[29462]: 2.d scrub starts 2024-09-11T07:27:54.744 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:54 smithi022 ceph-mon[29462]: 2.d scrub ok 2024-09-11T07:27:54.744 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:54 smithi022 ceph-mon[29462]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:54.744 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:54 smithi022 ceph-mon[29462]: from='client.14742 v1:172.21.15.22:0/3758027991' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:27:54.744 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:54 smithi022 ceph-mon[29462]: 2.c scrub starts 2024-09-11T07:27:54.744 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:54 smithi022 ceph-mon[29462]: 2.c scrub ok 2024-09-11T07:27:54.745 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:54 smithi022 ceph-mon[37330]: Deploying daemon node-exporter.a on smithi022 2024-09-11T07:27:54.745 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:54 smithi022 ceph-mon[37330]: 2.d scrub starts 2024-09-11T07:27:54.745 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:54 smithi022 ceph-mon[37330]: 2.d scrub ok 2024-09-11T07:27:54.745 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:54 smithi022 ceph-mon[37330]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:27:54.745 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:54 smithi022 ceph-mon[37330]: from='client.14742 v1:172.21.15.22:0/3758027991' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:27:54.745 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:54 smithi022 ceph-mon[37330]: 2.c scrub starts 2024-09-11T07:27:54.745 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:54 smithi022 ceph-mon[37330]: 2.c scrub ok 2024-09-11T07:27:54.746 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-09-11T07:27:53.943757+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":929,"num_read_kb":786,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":562,"ondisk_log_size":562,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2387212,"kb_used_data":8396,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747574004,"statfs":{"total":767960285184,"available":765515780096,"internally_reserved":0,"allocated":8597504,"data_stored":4591153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003071"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.731875+0000","last_change":"2024-09-11T07:26:37.863505+0000","last_active":"2024-09-11T07:27:38.731875+0000","last_peered":"2024-09-11T07:27:38.731875+0000","last_clean":"2024-09-11T07:27:38.731875+0000","last_became_active":"2024-09-11T07:26:37.863165+0000","last_became_peered":"2024-09-11T07:26:37.863165+0000","last_unstale":"2024-09-11T07:27:38.731875+0000","last_undegraded":"2024-09-11T07:27:38.731875+0000","last_fullsized":"2024-09-11T07:27:38.731875+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:05.169084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:51.198008+0000","last_change":"2024-09-11T07:27:51.197966+0000","last_active":"2024-09-11T07:27:51.198008+0000","last_peered":"2024-09-11T07:27:51.198008+0000","last_clean":"2024-09-11T07:27:51.198008+0000","last_became_active":"2024-09-11T07:26:31.421005+0000","last_became_peered":"2024-09-11T07:26:31.421005+0000","last_unstale":"2024-09-11T07:27:51.198008+0000","last_undegraded":"2024-09-11T07:27:51.198008+0000","last_fullsized":"2024-09-11T07:27:51.198008+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:51.197903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:51.197903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:00.681709+0000","scrub_duration":0.001600113,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"82'22","reported_seq":74,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:50.756811+0000","last_change":"2024-09-11T07:27:50.756777+0000","last_active":"2024-09-11T07:27:50.756811+0000","last_peered":"2024-09-11T07:27:50.756811+0000","last_clean":"2024-09-11T07:27:50.756811+0000","last_became_active":"2024-09-11T07:26:33.544627+0000","last_became_peered":"2024-09-11T07:26:33.544627+0000","last_unstale":"2024-09-11T07:27:50.756811+0000","last_undegraded":"2024-09-11T07:27:50.756811+0000","last_fullsized":"2024-09-11T07:27:50.756811+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.756707+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.257406+0000","scrub_duration":0.0049180669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431123+0000","last_change":"2024-09-11T07:26:35.557117+0000","last_active":"2024-09-11T07:27:38.431123+0000","last_peered":"2024-09-11T07:27:38.431123+0000","last_clean":"2024-09-11T07:27:38.431123+0000","last_became_active":"2024-09-11T07:26:35.556296+0000","last_became_peered":"2024-09-11T07:26:35.556296+0000","last_unstale":"2024-09-11T07:27:38.431123+0000","last_undegraded":"2024-09-11T07:27:38.431123+0000","last_fullsized":"2024-09-11T07:27:38.431123+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:56.174508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.732645+0000","last_change":"2024-09-11T07:27:36.776948+0000","last_active":"2024-09-11T07:27:38.732645+0000","last_peered":"2024-09-11T07:27:38.732645+0000","last_clean":"2024-09-11T07:27:38.732645+0000","last_became_active":"2024-09-11T07:26:31.420178+0000","last_became_peered":"2024-09-11T07:26:31.420178+0000","last_unstale":"2024-09-11T07:27:38.732645+0000","last_undegraded":"2024-09-11T07:27:38.732645+0000","last_fullsized":"2024-09-11T07:27:38.732645+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:36.776883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:36.776883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:49.961787+0000","scrub_duration":0.0017449290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"64'9","reported_seq":48,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431116+0000","last_change":"2024-09-11T07:26:33.541403+0000","last_active":"2024-09-11T07:27:38.431116+0000","last_peered":"2024-09-11T07:27:38.431116+0000","last_clean":"2024-09-11T07:27:38.431116+0000","last_became_active":"2024-09-11T07:26:33.540732+0000","last_became_peered":"2024-09-11T07:26:33.540732+0000","last_unstale":"2024-09-11T07:27:38.431116+0000","last_undegraded":"2024-09-11T07:27:38.431116+0000","last_fullsized":"2024-09-11T07:27:38.431116+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:53.423645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:50.648499+0000","last_change":"2024-09-11T07:27:50.648473+0000","last_active":"2024-09-11T07:27:50.648499+0000","last_peered":"2024-09-11T07:27:50.648499+0000","last_clean":"2024-09-11T07:27:50.648499+0000","last_became_active":"2024-09-11T07:26:35.561096+0000","last_became_peered":"2024-09-11T07:26:35.561096+0000","last_unstale":"2024-09-11T07:27:50.648499+0000","last_undegraded":"2024-09-11T07:27:50.648499+0000","last_fullsized":"2024-09-11T07:27:50.648499+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.648420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:05.663592+0000","scrub_duration":0.001391446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431209+0000","last_change":"2024-09-11T07:26:37.567195+0000","last_active":"2024-09-11T07:27:38.431209+0000","last_peered":"2024-09-11T07:27:38.431209+0000","last_clean":"2024-09-11T07:27:38.431209+0000","last_became_active":"2024-09-11T07:26:37.566910+0000","last_became_peered":"2024-09-11T07:26:37.566910+0000","last_unstale":"2024-09-11T07:27:38.431209+0000","last_undegraded":"2024-09-11T07:27:38.431209+0000","last_fullsized":"2024-09-11T07:27:38.431209+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:57.605592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.013548+0000","last_change":"2024-09-11T07:26:31.423041+0000","last_active":"2024-09-11T07:27:39.013548+0000","last_peered":"2024-09-11T07:27:39.013548+0000","last_clean":"2024-09-11T07:27:39.013548+0000","last_became_active":"2024-09-11T07:26:31.422772+0000","last_became_peered":"2024-09-11T07:26:31.422772+0000","last_unstale":"2024-09-11T07:27:39.013548+0000","last_undegraded":"2024-09-11T07:27:39.013548+0000","last_fullsized":"2024-09-11T07:27:39.013548+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:00.083055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"82'7","reported_seq":54,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:52.097212+0000","last_change":"2024-09-11T07:27:52.097170+0000","last_active":"2024-09-11T07:27:52.097212+0000","last_peered":"2024-09-11T07:27:52.097212+0000","last_clean":"2024-09-11T07:27:52.097212+0000","last_became_active":"2024-09-11T07:26:33.540351+0000","last_became_peered":"2024-09-11T07:26:33.540351+0000","last_unstale":"2024-09-11T07:27:52.097212+0000","last_undegraded":"2024-09-11T07:27:52.097212+0000","last_fullsized":"2024-09-11T07:27:52.097212+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'7","last_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_deep_scrub":"82'7","last_deep_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_clean_scrub_stamp":"2024-09-11T07:27:52.097071+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:53.905741+0000","scrub_duration":0.0025786670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.012856+0000","last_change":"2024-09-11T07:26:35.563858+0000","last_active":"2024-09-11T07:27:39.012856+0000","last_peered":"2024-09-11T07:27:39.012856+0000","last_clean":"2024-09-11T07:27:39.012856+0000","last_became_active":"2024-09-11T07:26:35.563536+0000","last_became_peered":"2024-09-11T07:26:35.563536+0000","last_unstale":"2024-09-11T07:27:39.012856+0000","last_undegraded":"2024-09-11T07:27:39.012856+0000","last_fullsized":"2024-09-11T07:27:39.012856+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:01.379998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.603094+0000","last_change":"2024-09-11T07:26:37.562008+0000","last_active":"2024-09-11T07:27:39.603094+0000","last_peered":"2024-09-11T07:27:39.603094+0000","last_clean":"2024-09-11T07:27:39.603094+0000","last_became_active":"2024-09-11T07:26:37.561758+0000","last_became_peered":"2024-09-11T07:26:37.561758+0000","last_unstale":"2024-09-11T07:27:39.603094+0000","last_undegraded":"2024-09-11T07:27:39.603094+0000","last_fullsized":"2024-09-11T07:27:39.603094+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:04.521146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:52.058044+0000","last_change":"2024-09-11T07:27:52.058013+0000","last_active":"2024-09-11T07:27:52.058044+0000","last_peered":"2024-09-11T07:27:52.058044+0000","last_clean":"2024-09-11T07:27:52.058044+0000","last_became_active":"2024-09-11T07:26:31.420969+0000","last_became_peered":"2024-09-11T07:26:31.420969+0000","last_unstale":"2024-09-11T07:27:52.058044+0000","last_undegraded":"2024-09-11T07:27:52.058044+0000","last_fullsized":"2024-09-11T07:27:52.058044+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_clean_scrub_stamp":"2024-09-11T07:27:52.057959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:12.902027+0000","scrub_duration":0.0014644840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"64'9","reported_seq":58,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:47.108747+0000","last_change":"2024-09-11T07:27:47.108704+0000","last_active":"2024-09-11T07:27:47.108747+0000","last_peered":"2024-09-11T07:27:47.108747+0000","last_clean":"2024-09-11T07:27:47.108747+0000","last_became_active":"2024-09-11T07:26:33.544386+0000","last_became_peered":"2024-09-11T07:26:33.544386+0000","last_unstale":"2024-09-11T07:27:47.108747+0000","last_undegraded":"2024-09-11T07:27:47.108747+0000","last_fullsized":"2024-09-11T07:27:47.108747+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:27:47.108645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.108645+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:08.302050+0000","scrub_duration":0.0024650560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.602477+0000","last_change":"2024-09-11T07:26:35.569943+0000","last_active":"2024-09-11T07:27:39.602477+0000","last_peered":"2024-09-11T07:27:39.602477+0000","last_clean":"2024-09-11T07:27:39.602477+0000","last_became_active":"2024-09-11T07:26:35.569630+0000","last_became_peered":"2024-09-11T07:26:35.569630+0000","last_unstale":"2024-09-11T07:27:39.602477+0000","last_undegraded":"2024-09-11T07:27:39.602477+0000","last_fullsized":"2024-09-11T07:27:39.602477+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:42.449548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428069+0000","last_change":"2024-09-11T07:26:37.565826+0000","last_active":"2024-09-11T07:27:38.428069+0000","last_peered":"2024-09-11T07:27:38.428069+0000","last_clean":"2024-09-11T07:27:38.428069+0000","last_became_active":"2024-09-11T07:26:37.565475+0000","last_became_peered":"2024-09-11T07:26:37.565475+0000","last_unstale":"2024-09-11T07:27:38.428069+0000","last_undegraded":"2024-09-11T07:27:38.428069+0000","last_fullsized":"2024-09-11T07:27:38.428069+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:50.954255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":60,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431907+0000","last_change":"2024-09-11T07:26:33.541535+0000","last_active":"2024-09-11T07:27:38.431907+0000","last_peered":"2024-09-11T07:27:38.431907+0000","last_clean":"2024-09-11T07:27:38.431907+0000","last_became_active":"2024-09-11T07:26:33.540575+0000","last_became_peered":"2024-09-11T07:26:33.540575+0000","last_unstale":"2024-09-11T07:27:38.431907+0000","last_undegraded":"2024-09-11T07:27:38.431907+0000","last_fullsized":"2024-09-11T07:27:38.431907+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:50.680900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.013458+0000","last_change":"2024-09-11T07:26:31.420109+0000","last_active":"2024-09-11T07:27:39.013458+0000","last_peered":"2024-09-11T07:27:39.013458+0000","last_clean":"2024-09-11T07:27:39.013458+0000","last_became_active":"2024-09-11T07:26:31.419488+0000","last_became_peered":"2024-09-11T07:26:31.419488+0000","last_unstale":"2024-09-11T07:27:39.013458+0000","last_undegraded":"2024-09-11T07:27:39.013458+0000","last_fullsized":"2024-09-11T07:27:39.013458+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:50.392677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428913+0000","last_change":"2024-09-11T07:26:35.559684+0000","last_active":"2024-09-11T07:27:38.428913+0000","last_peered":"2024-09-11T07:27:38.428913+0000","last_clean":"2024-09-11T07:27:38.428913+0000","last_became_active":"2024-09-11T07:26:35.557989+0000","last_became_peered":"2024-09-11T07:26:35.557989+0000","last_unstale":"2024-09-11T07:27:38.428913+0000","last_undegraded":"2024-09-11T07:27:38.428913+0000","last_fullsized":"2024-09-11T07:27:38.428913+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:54.992514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.013197+0000","last_change":"2024-09-11T07:26:37.565013+0000","last_active":"2024-09-11T07:27:39.013197+0000","last_peered":"2024-09-11T07:27:39.013197+0000","last_clean":"2024-09-11T07:27:39.013197+0000","last_became_active":"2024-09-11T07:26:37.564554+0000","last_became_peered":"2024-09-11T07:26:37.564554+0000","last_unstale":"2024-09-11T07:27:39.013197+0000","last_undegraded":"2024-09-11T07:27:39.013197+0000","last_fullsized":"2024-09-11T07:27:39.013197+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:46.952581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431592+0000","last_change":"2024-09-11T07:26:33.536333+0000","last_active":"2024-09-11T07:27:38.431592+0000","last_peered":"2024-09-11T07:27:38.431592+0000","last_clean":"2024-09-11T07:27:38.431592+0000","last_became_active":"2024-09-11T07:26:33.535411+0000","last_became_peered":"2024-09-11T07:26:33.535411+0000","last_unstale":"2024-09-11T07:27:38.431592+0000","last_undegraded":"2024-09-11T07:27:38.431592+0000","last_fullsized":"2024-09-11T07:27:38.431592+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:00.222835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428884+0000","last_change":"2024-09-11T07:26:31.419259+0000","last_active":"2024-09-11T07:27:38.428884+0000","last_peered":"2024-09-11T07:27:38.428884+0000","last_clean":"2024-09-11T07:27:38.428884+0000","last_became_active":"2024-09-11T07:26:31.418178+0000","last_became_peered":"2024-09-11T07:26:31.418178+0000","last_unstale":"2024-09-11T07:27:38.428884+0000","last_undegraded":"2024-09-11T07:27:38.428884+0000","last_fullsized":"2024-09-11T07:27:38.428884+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:53.539520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"64'8","reported_seq":64,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:47.127543+0000","last_change":"2024-09-11T07:27:47.127485+0000","last_active":"2024-09-11T07:27:47.127543+0000","last_peered":"2024-09-11T07:27:47.127543+0000","last_clean":"2024-09-11T07:27:47.127543+0000","last_became_active":"2024-09-11T07:26:35.564531+0000","last_became_peered":"2024-09-11T07:26:35.564531+0000","last_unstale":"2024-09-11T07:27:47.127543+0000","last_undegraded":"2024-09-11T07:27:47.127543+0000","last_fullsized":"2024-09-11T07:27:47.127543+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:27:47.127428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.127428+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:58.343731+0000","scrub_duration":0.0014091329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431540+0000","last_change":"2024-09-11T07:26:37.567193+0000","last_active":"2024-09-11T07:27:38.431540+0000","last_peered":"2024-09-11T07:27:38.431540+0000","last_clean":"2024-09-11T07:27:38.431540+0000","last_became_active":"2024-09-11T07:26:37.566919+0000","last_became_peered":"2024-09-11T07:26:37.566919+0000","last_unstale":"2024-09-11T07:27:38.431540+0000","last_undegraded":"2024-09-11T07:27:38.431540+0000","last_fullsized":"2024-09-11T07:27:38.431540+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:05.045246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"64'15","reported_seq":57,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.429231+0000","last_change":"2024-09-11T07:26:33.551090+0000","last_active":"2024-09-11T07:27:38.429231+0000","last_peered":"2024-09-11T07:27:38.429231+0000","last_clean":"2024-09-11T07:27:38.429231+0000","last_became_active":"2024-09-11T07:26:33.549733+0000","last_became_peered":"2024-09-11T07:26:33.549733+0000","last_unstale":"2024-09-11T07:27:38.429231+0000","last_undegraded":"2024-09-11T07:27:38.429231+0000","last_fullsized":"2024-09-11T07:27:38.429231+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:45.951848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428362+0000","last_change":"2024-09-11T07:27:35.589317+0000","last_active":"2024-09-11T07:27:38.428362+0000","last_peered":"2024-09-11T07:27:38.428362+0000","last_clean":"2024-09-11T07:27:38.428362+0000","last_became_active":"2024-09-11T07:26:31.418478+0000","last_became_peered":"2024-09-11T07:26:31.418478+0000","last_unstale":"2024-09-11T07:27:38.428362+0000","last_undegraded":"2024-09-11T07:27:38.428362+0000","last_fullsized":"2024-09-11T07:27:38.428362+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:35.589251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:35.589251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:01.839680+0000","scrub_duration":0.0014929380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:44.497246+0000","last_change":"2024-09-11T07:26:35.560767+0000","last_active":"2024-09-11T07:27:44.497246+0000","last_peered":"2024-09-11T07:27:44.497246+0000","last_clean":"2024-09-11T07:27:44.497246+0000","last_became_active":"2024-09-11T07:26:35.560406+0000","last_became_peered":"2024-09-11T07:26:35.560406+0000","last_unstale":"2024-09-11T07:27:44.497246+0000","last_undegraded":"2024-09-11T07:27:44.497246+0000","last_fullsized":"2024-09-11T07:27:44.497246+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.603454+0000","last_change":"2024-09-11T07:26:37.568289+0000","last_active":"2024-09-11T07:27:39.603454+0000","last_peered":"2024-09-11T07:27:39.603454+0000","last_clean":"2024-09-11T07:27:39.603454+0000","last_became_active":"2024-09-11T07:26:37.568005+0000","last_became_peered":"2024-09-11T07:26:37.568005+0000","last_unstale":"2024-09-11T07:27:39.603454+0000","last_undegraded":"2024-09-11T07:27:39.603454+0000","last_fullsized":"2024-09-11T07:27:39.603454+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:00.612703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"64'11","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431717+0000","last_change":"2024-09-11T07:26:33.541083+0000","last_active":"2024-09-11T07:27:38.431717+0000","last_peered":"2024-09-11T07:27:38.431717+0000","last_clean":"2024-09-11T07:27:38.431717+0000","last_became_active":"2024-09-11T07:26:33.540777+0000","last_became_peered":"2024-09-11T07:26:33.540777+0000","last_unstale":"2024-09-11T07:27:38.431717+0000","last_undegraded":"2024-09-11T07:27:38.431717+0000","last_fullsized":"2024-09-11T07:27:38.431717+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:47.691831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.012800+0000","last_change":"2024-09-11T07:27:34.932151+0000","last_active":"2024-09-11T07:27:39.012800+0000","last_peered":"2024-09-11T07:27:39.012800+0000","last_clean":"2024-09-11T07:27:39.012800+0000","last_became_active":"2024-09-11T07:26:31.418238+0000","last_became_peered":"2024-09-11T07:26:31.418238+0000","last_unstale":"2024-09-11T07:27:39.012800+0000","last_undegraded":"2024-09-11T07:27:39.012800+0000","last_fullsized":"2024-09-11T07:27:39.012800+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:34.932092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:34.932092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:46.904196+0000","scrub_duration":0.002710259,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.602649+0000","last_change":"2024-09-11T07:26:35.569433+0000","last_active":"2024-09-11T07:27:39.602649+0000","last_peered":"2024-09-11T07:27:39.602649+0000","last_clean":"2024-09-11T07:27:39.602649+0000","last_became_active":"2024-09-11T07:26:35.569081+0000","last_became_peered":"2024-09-11T07:26:35.569081+0000","last_unstale":"2024-09-11T07:27:39.602649+0000","last_undegraded":"2024-09-11T07:27:39.602649+0000","last_fullsized":"2024-09-11T07:27:39.602649+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:44.797975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:46.752315+0000","last_change":"2024-09-11T07:27:46.752272+0000","last_active":"2024-09-11T07:27:46.752315+0000","last_peered":"2024-09-11T07:27:46.752315+0000","last_clean":"2024-09-11T07:27:46.752315+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:27:46.752315+0000","last_undegraded":"2024-09-11T07:27:46.752315+0000","last_fullsized":"2024-09-11T07:27:46.752315+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:46.752181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:46.752181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.072860+0000","scrub_duration":0.001508782,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"64'14","reported_seq":53,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431834+0000","last_change":"2024-09-11T07:26:33.538152+0000","last_active":"2024-09-11T07:27:38.431834+0000","last_peered":"2024-09-11T07:27:38.431834+0000","last_clean":"2024-09-11T07:27:38.431834+0000","last_became_active":"2024-09-11T07:26:33.537850+0000","last_became_peered":"2024-09-11T07:26:33.537850+0000","last_unstale":"2024-09-11T07:27:38.431834+0000","last_undegraded":"2024-09-11T07:27:38.431834+0000","last_fullsized":"2024-09-11T07:27:38.431834+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:57.850522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.014986+0000","last_change":"2024-09-11T07:27:39.014916+0000","last_active":"2024-09-11T07:27:39.014986+0000","last_peered":"2024-09-11T07:27:39.014986+0000","last_clean":"2024-09-11T07:27:39.014986+0000","last_became_active":"2024-09-11T07:26:31.423905+0000","last_became_peered":"2024-09-11T07:26:31.423905+0000","last_unstale":"2024-09-11T07:27:39.014986+0000","last_undegraded":"2024-09-11T07:27:39.014986+0000","last_fullsized":"2024-09-11T07:27:39.014986+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:39.014840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:39.014840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:40.097304+0000","scrub_duration":0.0032789239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"64'8","reported_seq":54,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:44.497264+0000","last_change":"2024-09-11T07:26:35.563242+0000","last_active":"2024-09-11T07:27:44.497264+0000","last_peered":"2024-09-11T07:27:44.497264+0000","last_clean":"2024-09-11T07:27:44.497264+0000","last_became_active":"2024-09-11T07:26:35.562997+0000","last_became_peered":"2024-09-11T07:26:35.562997+0000","last_unstale":"2024-09-11T07:27:44.497264+0000","last_undegraded":"2024-09-11T07:27:44.497264+0000","last_fullsized":"2024-09-11T07:27:44.497264+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:50.078049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.013053+0000","last_change":"2024-09-11T07:26:37.862564+0000","last_active":"2024-09-11T07:27:39.013053+0000","last_peered":"2024-09-11T07:27:39.013053+0000","last_clean":"2024-09-11T07:27:39.013053+0000","last_became_active":"2024-09-11T07:26:37.862246+0000","last_became_peered":"2024-09-11T07:26:37.862246+0000","last_unstale":"2024-09-11T07:27:39.013053+0000","last_undegraded":"2024-09-11T07:27:39.013053+0000","last_fullsized":"2024-09-11T07:27:39.013053+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:04.346579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"64'18","reported_seq":64,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.731805+0000","last_change":"2024-09-11T07:26:33.546711+0000","last_active":"2024-09-11T07:27:38.731805+0000","last_peered":"2024-09-11T07:27:38.731805+0000","last_clean":"2024-09-11T07:27:38.731805+0000","last_became_active":"2024-09-11T07:26:33.546188+0000","last_became_peered":"2024-09-11T07:26:33.546188+0000","last_unstale":"2024-09-11T07:27:38.731805+0000","last_undegraded":"2024-09-11T07:27:38.731805+0000","last_fullsized":"2024-09-11T07:27:38.731805+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:02.137900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.429518+0000","last_change":"2024-09-11T07:26:31.421802+0000","last_active":"2024-09-11T07:27:38.429518+0000","last_peered":"2024-09-11T07:27:38.429518+0000","last_clean":"2024-09-11T07:27:38.429518+0000","last_became_active":"2024-09-11T07:26:31.421500+0000","last_became_peered":"2024-09-11T07:26:31.421500+0000","last_unstale":"2024-09-11T07:27:38.429518+0000","last_undegraded":"2024-09-11T07:27:38.429518+0000","last_fullsized":"2024-09-11T07:27:38.429518+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:56.390924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431190+0000","last_change":"2024-09-11T07:26:35.562816+0000","last_active":"2024-09-11T07:27:38.431190+0000","last_peered":"2024-09-11T07:27:38.431190+0000","last_clean":"2024-09-11T07:27:38.431190+0000","last_became_active":"2024-09-11T07:26:35.561284+0000","last_became_peered":"2024-09-11T07:26:35.561284+0000","last_unstale":"2024-09-11T07:27:38.431190+0000","last_undegraded":"2024-09-11T07:27:38.431190+0000","last_fullsized":"2024-09-11T07:27:38.431190+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:01.396434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431748+0000","last_change":"2024-09-11T07:26:37.565014+0000","last_active":"2024-09-11T07:27:38.431748+0000","last_peered":"2024-09-11T07:27:38.431748+0000","last_clean":"2024-09-11T07:27:38.431748+0000","last_became_active":"2024-09-11T07:26:37.564616+0000","last_became_peered":"2024-09-11T07:26:37.564616+0000","last_unstale":"2024-09-11T07:27:38.431748+0000","last_undegraded":"2024-09-11T07:27:38.431748+0000","last_fullsized":"2024-09-11T07:27:38.431748+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:04.390896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":68,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.429080+0000","last_change":"2024-09-11T07:26:33.541012+0000","last_active":"2024-09-11T07:27:38.429080+0000","last_peered":"2024-09-11T07:27:38.429080+0000","last_clean":"2024-09-11T07:27:38.429080+0000","last_became_active":"2024-09-11T07:26:33.540639+0000","last_became_peered":"2024-09-11T07:26:33.540639+0000","last_unstale":"2024-09-11T07:27:38.429080+0000","last_undegraded":"2024-09-11T07:27:38.429080+0000","last_fullsized":"2024-09-11T07:27:38.429080+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:51.550418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.013736+0000","last_change":"2024-09-11T07:26:31.419925+0000","last_active":"2024-09-11T07:27:39.013736+0000","last_peered":"2024-09-11T07:27:39.013736+0000","last_clean":"2024-09-11T07:27:39.013736+0000","last_became_active":"2024-09-11T07:26:31.419185+0000","last_became_peered":"2024-09-11T07:26:31.419185+0000","last_unstale":"2024-09-11T07:27:39.013736+0000","last_undegraded":"2024-09-11T07:27:39.013736+0000","last_fullsized":"2024-09-11T07:27:39.013736+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:52.580179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:41.000506+0000","last_change":"2024-09-11T07:27:41.000469+0000","last_active":"2024-09-11T07:27:41.000506+0000","last_peered":"2024-09-11T07:27:41.000506+0000","last_clean":"2024-09-11T07:27:41.000506+0000","last_became_active":"2024-09-11T07:26:35.565359+0000","last_became_peered":"2024-09-11T07:26:35.565359+0000","last_unstale":"2024-09-11T07:27:41.000506+0000","last_undegraded":"2024-09-11T07:27:41.000506+0000","last_fullsized":"2024-09-11T07:27:41.000506+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:41.000401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:41.000401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.049825+0000","scrub_duration":0.001655164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428073+0000","last_change":"2024-09-11T07:26:37.861736+0000","last_active":"2024-09-11T07:27:38.428073+0000","last_peered":"2024-09-11T07:27:38.428073+0000","last_clean":"2024-09-11T07:27:38.428073+0000","last_became_active":"2024-09-11T07:26:37.861279+0000","last_became_peered":"2024-09-11T07:26:37.861279+0000","last_unstale":"2024-09-11T07:27:38.428073+0000","last_undegraded":"2024-09-11T07:27:38.428073+0000","last_fullsized":"2024-09-11T07:27:38.428073+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:58.793313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:51.642564+0000","last_change":"2024-09-11T07:27:51.642529+0000","last_active":"2024-09-11T07:27:51.642564+0000","last_peered":"2024-09-11T07:27:51.642564+0000","last_clean":"2024-09-11T07:27:51.642564+0000","last_became_active":"2024-09-11T07:26:33.543063+0000","last_became_peered":"2024-09-11T07:26:33.543063+0000","last_unstale":"2024-09-11T07:27:51.642564+0000","last_undegraded":"2024-09-11T07:27:51.642564+0000","last_fullsized":"2024-09-11T07:27:51.642564+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:27:51.642464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:51.642464+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:13.146284+0000","scrub_duration":0.0020211439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":51,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:41.618402+0000","last_change":"2024-09-11T07:27:41.618362+0000","last_active":"2024-09-11T07:27:41.618402+0000","last_peered":"2024-09-11T07:27:41.618402+0000","last_clean":"2024-09-11T07:27:41.618402+0000","last_became_active":"2024-09-11T07:26:31.422110+0000","last_became_peered":"2024-09-11T07:26:31.422110+0000","last_unstale":"2024-09-11T07:27:41.618402+0000","last_undegraded":"2024-09-11T07:27:41.618402+0000","last_fullsized":"2024-09-11T07:27:41.618402+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:27:41.618296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:41.618296+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:57.404591+0000","scrub_duration":0.0020650450000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":54,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:44.497569+0000","last_change":"2024-09-11T07:26:35.560735+0000","last_active":"2024-09-11T07:27:44.497569+0000","last_peered":"2024-09-11T07:27:44.497569+0000","last_clean":"2024-09-11T07:27:44.497569+0000","last_became_active":"2024-09-11T07:26:35.560422+0000","last_became_peered":"2024-09-11T07:26:35.560422+0000","last_unstale":"2024-09-11T07:27:44.497569+0000","last_undegraded":"2024-09-11T07:27:44.497569+0000","last_fullsized":"2024-09-11T07:27:44.497569+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.427894+0000","last_change":"2024-09-11T07:26:37.568087+0000","last_active":"2024-09-11T07:27:38.427894+0000","last_peered":"2024-09-11T07:27:38.427894+0000","last_clean":"2024-09-11T07:27:38.427894+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:27:38.427894+0000","last_undegraded":"2024-09-11T07:27:38.427894+0000","last_fullsized":"2024-09-11T07:27:38.427894+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"64'28","reported_seq":96,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:44.497299+0000","last_change":"2024-09-11T07:26:33.549518+0000","last_active":"2024-09-11T07:27:44.497299+0000","last_peered":"2024-09-11T07:27:44.497299+0000","last_clean":"2024-09-11T07:27:44.497299+0000","last_became_active":"2024-09-11T07:26:33.549234+0000","last_became_peered":"2024-09-11T07:26:33.549234+0000","last_unstale":"2024-09-11T07:27:44.497299+0000","last_undegraded":"2024-09-11T07:27:44.497299+0000","last_fullsized":"2024-09-11T07:27:44.497299+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:50.484684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.603309+0000","last_change":"2024-09-11T07:26:31.424898+0000","last_active":"2024-09-11T07:27:39.603309+0000","last_peered":"2024-09-11T07:27:39.603309+0000","last_clean":"2024-09-11T07:27:39.603309+0000","last_became_active":"2024-09-11T07:26:31.424618+0000","last_became_peered":"2024-09-11T07:26:31.424618+0000","last_unstale":"2024-09-11T07:27:39.603309+0000","last_undegraded":"2024-09-11T07:27:39.603309+0000","last_fullsized":"2024-09-11T07:27:39.603309+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:55.128959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.429016+0000","last_change":"2024-09-11T07:26:35.560677+0000","last_active":"2024-09-11T07:27:38.429016+0000","last_peered":"2024-09-11T07:27:38.429016+0000","last_clean":"2024-09-11T07:27:38.429016+0000","last_became_active":"2024-09-11T07:26:35.560363+0000","last_became_peered":"2024-09-11T07:26:35.560363+0000","last_unstale":"2024-09-11T07:27:38.429016+0000","last_undegraded":"2024-09-11T07:27:38.429016+0000","last_fullsized":"2024-09-11T07:27:38.429016+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:53.196351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.732485+0000","last_change":"2024-09-11T07:26:37.563366+0000","last_active":"2024-09-11T07:27:38.732485+0000","last_peered":"2024-09-11T07:27:38.732485+0000","last_clean":"2024-09-11T07:27:38.732485+0000","last_became_active":"2024-09-11T07:26:37.562452+0000","last_became_peered":"2024-09-11T07:26:37.562452+0000","last_unstale":"2024-09-11T07:27:38.732485+0000","last_undegraded":"2024-09-11T07:27:38.732485+0000","last_fullsized":"2024-09-11T07:27:38.732485+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:44.615521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"64'13","reported_seq":59,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.429346+0000","last_change":"2024-09-11T07:26:33.547483+0000","last_active":"2024-09-11T07:27:38.429346+0000","last_peered":"2024-09-11T07:27:38.429346+0000","last_clean":"2024-09-11T07:27:38.429346+0000","last_became_active":"2024-09-11T07:26:33.546126+0000","last_became_peered":"2024-09-11T07:26:33.546126+0000","last_unstale":"2024-09-11T07:27:38.429346+0000","last_undegraded":"2024-09-11T07:27:38.429346+0000","last_fullsized":"2024-09-11T07:27:38.429346+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:56.391970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":118,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:52.716275+0000","last_change":"2024-09-11T07:26:35.548541+0000","last_active":"2024-09-11T07:27:52.716275+0000","last_peered":"2024-09-11T07:27:52.716275+0000","last_clean":"2024-09-11T07:27:52.716275+0000","last_became_active":"2024-09-11T07:26:31.421276+0000","last_became_peered":"2024-09-11T07:26:31.421276+0000","last_unstale":"2024-09-11T07:27:52.716275+0000","last_undegraded":"2024-09-11T07:27:52.716275+0000","last_fullsized":"2024-09-11T07:27:52.716275+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.012923+0000","last_change":"2024-09-11T07:26:31.413634+0000","last_active":"2024-09-11T07:27:39.012923+0000","last_peered":"2024-09-11T07:27:39.012923+0000","last_clean":"2024-09-11T07:27:39.012923+0000","last_became_active":"2024-09-11T07:26:31.413354+0000","last_became_peered":"2024-09-11T07:26:31.413354+0000","last_unstale":"2024-09-11T07:27:39.012923+0000","last_undegraded":"2024-09-11T07:27:39.012923+0000","last_fullsized":"2024-09-11T07:27:39.012923+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:55.160768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:42.141181+0000","last_change":"2024-09-11T07:27:42.141156+0000","last_active":"2024-09-11T07:27:42.141181+0000","last_peered":"2024-09-11T07:27:42.141181+0000","last_clean":"2024-09-11T07:27:42.141181+0000","last_became_active":"2024-09-11T07:26:35.565849+0000","last_became_peered":"2024-09-11T07:26:35.565849+0000","last_unstale":"2024-09-11T07:27:42.141181+0000","last_undegraded":"2024-09-11T07:27:42.141181+0000","last_fullsized":"2024-09-11T07:27:42.141181+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.141100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.141100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.707826+0000","scrub_duration":0.001495672,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.013249+0000","last_change":"2024-09-11T07:26:37.861648+0000","last_active":"2024-09-11T07:27:39.013249+0000","last_peered":"2024-09-11T07:27:39.013249+0000","last_clean":"2024-09-11T07:27:39.013249+0000","last_became_active":"2024-09-11T07:26:37.861292+0000","last_became_peered":"2024-09-11T07:26:37.861292+0000","last_unstale":"2024-09-11T07:27:39.013249+0000","last_undegraded":"2024-09-11T07:27:39.013249+0000","last_fullsized":"2024-09-11T07:27:39.013249+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:50.030830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":60,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428689+0000","last_change":"2024-09-11T07:27:34.213946+0000","last_active":"2024-09-11T07:27:38.428689+0000","last_peered":"2024-09-11T07:27:38.428689+0000","last_clean":"2024-09-11T07:27:38.428689+0000","last_became_active":"2024-09-11T07:26:33.548903+0000","last_became_peered":"2024-09-11T07:26:33.548903+0000","last_unstale":"2024-09-11T07:27:38.428689+0000","last_undegraded":"2024-09-11T07:27:38.428689+0000","last_fullsized":"2024-09-11T07:27:38.428689+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-09-11T07:27:34.213870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:34.213870+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:50.149624+0000","scrub_duration":0.0026667679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428424+0000","last_change":"2024-09-11T07:26:35.551577+0000","last_active":"2024-09-11T07:27:38.428424+0000","last_peered":"2024-09-11T07:27:38.428424+0000","last_clean":"2024-09-11T07:27:38.428424+0000","last_became_active":"2024-09-11T07:26:31.420320+0000","last_became_peered":"2024-09-11T07:26:31.420320+0000","last_unstale":"2024-09-11T07:27:38.428424+0000","last_undegraded":"2024-09-11T07:27:38.428424+0000","last_fullsized":"2024-09-11T07:27:38.428424+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:49.887263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.012844+0000","last_change":"2024-09-11T07:26:31.418190+0000","last_active":"2024-09-11T07:27:39.012844+0000","last_peered":"2024-09-11T07:27:39.012844+0000","last_clean":"2024-09-11T07:27:39.012844+0000","last_became_active":"2024-09-11T07:26:31.417455+0000","last_became_peered":"2024-09-11T07:26:31.417455+0000","last_unstale":"2024-09-11T07:27:39.012844+0000","last_undegraded":"2024-09-11T07:27:39.012844+0000","last_fullsized":"2024-09-11T07:27:39.012844+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:54.951321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.602486+0000","last_change":"2024-09-11T07:26:35.564511+0000","last_active":"2024-09-11T07:27:39.602486+0000","last_peered":"2024-09-11T07:27:39.602486+0000","last_clean":"2024-09-11T07:27:39.602486+0000","last_became_active":"2024-09-11T07:26:35.564178+0000","last_became_peered":"2024-09-11T07:26:35.564178+0000","last_unstale":"2024-09-11T07:27:39.602486+0000","last_undegraded":"2024-09-11T07:27:39.602486+0000","last_fullsized":"2024-09-11T07:27:39.602486+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:01.503548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428838+0000","last_change":"2024-09-11T07:26:37.561674+0000","last_active":"2024-09-11T07:27:38.428838+0000","last_peered":"2024-09-11T07:27:38.428838+0000","last_clean":"2024-09-11T07:27:38.428838+0000","last_became_active":"2024-09-11T07:26:37.561378+0000","last_became_peered":"2024-09-11T07:26:37.561378+0000","last_unstale":"2024-09-11T07:27:38.428838+0000","last_undegraded":"2024-09-11T07:27:38.428838+0000","last_fullsized":"2024-09-11T07:27:38.428838+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:44.852330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":64,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:49.061760+0000","last_change":"2024-09-11T07:27:49.061728+0000","last_active":"2024-09-11T07:27:49.061760+0000","last_peered":"2024-09-11T07:27:49.061760+0000","last_clean":"2024-09-11T07:27:49.061760+0000","last_became_active":"2024-09-11T07:26:33.541359+0000","last_became_peered":"2024-09-11T07:26:33.541359+0000","last_unstale":"2024-09-11T07:27:49.061760+0000","last_undegraded":"2024-09-11T07:27:49.061760+0000","last_fullsized":"2024-09-11T07:27:49.061760+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-09-11T07:27:49.061670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.061670+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:51.283390+0000","scrub_duration":0.001809801,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"61'2","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.732096+0000","last_change":"2024-09-11T07:26:35.556632+0000","last_active":"2024-09-11T07:27:38.732096+0000","last_peered":"2024-09-11T07:27:38.732096+0000","last_clean":"2024-09-11T07:27:38.732096+0000","last_became_active":"2024-09-11T07:26:31.419501+0000","last_became_peered":"2024-09-11T07:26:31.419501+0000","last_unstale":"2024-09-11T07:27:38.732096+0000","last_undegraded":"2024-09-11T07:27:38.732096+0000","last_fullsized":"2024-09-11T07:27:38.732096+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:45.065345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:47.903234+0000","last_change":"2024-09-11T07:27:47.903204+0000","last_active":"2024-09-11T07:27:47.903234+0000","last_peered":"2024-09-11T07:27:47.903234+0000","last_clean":"2024-09-11T07:27:47.903234+0000","last_became_active":"2024-09-11T07:26:31.423304+0000","last_became_peered":"2024-09-11T07:26:31.423304+0000","last_unstale":"2024-09-11T07:27:47.903234+0000","last_undegraded":"2024-09-11T07:27:47.903234+0000","last_fullsized":"2024-09-11T07:27:47.903234+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:47.903135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.903135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:01.867608+0000","scrub_duration":0.001284908,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"72'79","reported_seq":79,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.013039+0000","last_change":"2024-09-11T07:26:57.872633+0000","last_active":"2024-09-11T07:27:39.013039+0000","last_peered":"2024-09-11T07:27:39.013039+0000","last_clean":"2024-09-11T07:27:39.013039+0000","last_became_active":"2024-09-11T07:26:17.397381+0000","last_became_peered":"2024-09-11T07:26:17.397381+0000","last_unstale":"2024-09-11T07:27:39.013039+0000","last_undegraded":"2024-09-11T07:27:39.013039+0000","last_fullsized":"2024-09-11T07:27:39.013039+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'72","last_scrub_stamp":"2024-09-11T07:26:57.872570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:24:22.202219+0000","last_clean_scrub_stamp":"2024-09-11T07:26:57.872570+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:06.077652+0000","scrub_duration":0.0015639180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:44.036179+0000","last_change":"2024-09-11T07:27:44.036138+0000","last_active":"2024-09-11T07:27:44.036179+0000","last_peered":"2024-09-11T07:27:44.036179+0000","last_clean":"2024-09-11T07:27:44.036179+0000","last_became_active":"2024-09-11T07:26:35.564480+0000","last_became_peered":"2024-09-11T07:26:35.564480+0000","last_unstale":"2024-09-11T07:27:44.036179+0000","last_undegraded":"2024-09-11T07:27:44.036179+0000","last_fullsized":"2024-09-11T07:27:44.036179+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:44.036058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:44.036058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:48.598709+0000","scrub_duration":0.001488931,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.603710+0000","last_change":"2024-09-11T07:26:37.560202+0000","last_active":"2024-09-11T07:27:39.603710+0000","last_peered":"2024-09-11T07:27:39.603710+0000","last_clean":"2024-09-11T07:27:39.603710+0000","last_became_active":"2024-09-11T07:26:37.559920+0000","last_became_peered":"2024-09-11T07:26:37.559920+0000","last_unstale":"2024-09-11T07:27:39.603710+0000","last_undegraded":"2024-09-11T07:27:39.603710+0000","last_fullsized":"2024-09-11T07:27:39.603710+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:55.457939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":63,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.672378+0000","last_change":"2024-09-11T07:26:33.544014+0000","last_active":"2024-09-11T07:27:38.672378+0000","last_peered":"2024-09-11T07:27:38.672378+0000","last_clean":"2024-09-11T07:27:38.672378+0000","last_became_active":"2024-09-11T07:26:33.543687+0000","last_became_peered":"2024-09-11T07:26:33.543687+0000","last_unstale":"2024-09-11T07:27:38.672378+0000","last_undegraded":"2024-09-11T07:27:38.672378+0000","last_fullsized":"2024-09-11T07:27:38.672378+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:55.743551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.013246+0000","last_change":"2024-09-11T07:26:31.419837+0000","last_active":"2024-09-11T07:27:39.013246+0000","last_peered":"2024-09-11T07:27:39.013246+0000","last_clean":"2024-09-11T07:27:39.013246+0000","last_became_active":"2024-09-11T07:26:31.419560+0000","last_became_peered":"2024-09-11T07:26:31.419560+0000","last_unstale":"2024-09-11T07:27:39.013246+0000","last_undegraded":"2024-09-11T07:27:39.013246+0000","last_fullsized":"2024-09-11T07:27:39.013246+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:46.444292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.012835+0000","last_change":"2024-09-11T07:26:35.564320+0000","last_active":"2024-09-11T07:27:39.012835+0000","last_peered":"2024-09-11T07:27:39.012835+0000","last_clean":"2024-09-11T07:27:39.012835+0000","last_became_active":"2024-09-11T07:26:35.564044+0000","last_became_peered":"2024-09-11T07:26:35.564044+0000","last_unstale":"2024-09-11T07:27:39.012835+0000","last_undegraded":"2024-09-11T07:27:39.012835+0000","last_fullsized":"2024-09-11T07:27:39.012835+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:01.424414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.013152+0000","last_change":"2024-09-11T07:26:37.565084+0000","last_active":"2024-09-11T07:27:39.013152+0000","last_peered":"2024-09-11T07:27:39.013152+0000","last_clean":"2024-09-11T07:27:39.013152+0000","last_became_active":"2024-09-11T07:26:37.564801+0000","last_became_peered":"2024-09-11T07:26:37.564801+0000","last_unstale":"2024-09-11T07:27:39.013152+0000","last_undegraded":"2024-09-11T07:27:39.013152+0000","last_fullsized":"2024-09-11T07:27:39.013152+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:53.493323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":48,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428999+0000","last_change":"2024-09-11T07:26:33.543461+0000","last_active":"2024-09-11T07:27:38.428999+0000","last_peered":"2024-09-11T07:27:38.428999+0000","last_clean":"2024-09-11T07:27:38.428999+0000","last_became_active":"2024-09-11T07:26:33.543148+0000","last_became_peered":"2024-09-11T07:26:33.543148+0000","last_unstale":"2024-09-11T07:27:38.428999+0000","last_undegraded":"2024-09-11T07:27:38.428999+0000","last_fullsized":"2024-09-11T07:27:38.428999+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:40.632848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:53.660005+0000","last_change":"2024-09-11T07:27:53.659961+0000","last_active":"2024-09-11T07:27:53.660005+0000","last_peered":"2024-09-11T07:27:53.660005+0000","last_clean":"2024-09-11T07:27:53.660005+0000","last_became_active":"2024-09-11T07:26:31.423056+0000","last_became_peered":"2024-09-11T07:26:31.423056+0000","last_unstale":"2024-09-11T07:27:53.660005+0000","last_undegraded":"2024-09-11T07:27:53.660005+0000","last_fullsized":"2024-09-11T07:27:53.660005+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:53.659902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:53.659902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.781453+0000","scrub_duration":0.0011613719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.012953+0000","last_change":"2024-09-11T07:26:35.563452+0000","last_active":"2024-09-11T07:27:39.012953+0000","last_peered":"2024-09-11T07:27:39.012953+0000","last_clean":"2024-09-11T07:27:39.012953+0000","last_became_active":"2024-09-11T07:26:35.563182+0000","last_became_peered":"2024-09-11T07:26:35.563182+0000","last_unstale":"2024-09-11T07:27:39.012953+0000","last_undegraded":"2024-09-11T07:27:39.012953+0000","last_fullsized":"2024-09-11T07:27:39.012953+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:00.634009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428385+0000","last_change":"2024-09-11T07:26:37.565396+0000","last_active":"2024-09-11T07:27:38.428385+0000","last_peered":"2024-09-11T07:27:38.428385+0000","last_clean":"2024-09-11T07:27:38.428385+0000","last_became_active":"2024-09-11T07:26:37.565029+0000","last_became_peered":"2024-09-11T07:26:37.565029+0000","last_unstale":"2024-09-11T07:27:38.428385+0000","last_undegraded":"2024-09-11T07:27:38.428385+0000","last_fullsized":"2024-09-11T07:27:38.428385+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:03.261465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":57,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.602970+0000","last_change":"2024-09-11T07:26:33.538495+0000","last_active":"2024-09-11T07:27:39.602970+0000","last_peered":"2024-09-11T07:27:39.602970+0000","last_clean":"2024-09-11T07:27:39.602970+0000","last_became_active":"2024-09-11T07:26:33.538216+0000","last_became_peered":"2024-09-11T07:26:33.538216+0000","last_unstale":"2024-09-11T07:27:39.602970+0000","last_undegraded":"2024-09-11T07:27:39.602970+0000","last_fullsized":"2024-09-11T07:27:39.602970+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:52.847742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:50.132667+0000","last_change":"2024-09-11T07:27:50.132629+0000","last_active":"2024-09-11T07:27:50.132667+0000","last_peered":"2024-09-11T07:27:50.132667+0000","last_clean":"2024-09-11T07:27:50.132667+0000","last_became_active":"2024-09-11T07:26:31.418490+0000","last_became_peered":"2024-09-11T07:26:31.418490+0000","last_unstale":"2024-09-11T07:27:50.132667+0000","last_undegraded":"2024-09-11T07:27:50.132667+0000","last_fullsized":"2024-09-11T07:27:50.132667+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.132552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.048681+0000","scrub_duration":0.001622807,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":54,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:44.497171+0000","last_change":"2024-09-11T07:26:35.567147+0000","last_active":"2024-09-11T07:27:44.497171+0000","last_peered":"2024-09-11T07:27:44.497171+0000","last_clean":"2024-09-11T07:27:44.497171+0000","last_became_active":"2024-09-11T07:26:35.566888+0000","last_became_peered":"2024-09-11T07:26:35.566888+0000","last_unstale":"2024-09-11T07:27:44.497171+0000","last_undegraded":"2024-09-11T07:27:44.497171+0000","last_fullsized":"2024-09-11T07:27:44.497171+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:03.574568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.603130+0000","last_change":"2024-09-11T07:26:37.861501+0000","last_active":"2024-09-11T07:27:39.603130+0000","last_peered":"2024-09-11T07:27:39.603130+0000","last_clean":"2024-09-11T07:27:39.603130+0000","last_became_active":"2024-09-11T07:26:37.861134+0000","last_became_peered":"2024-09-11T07:26:37.861134+0000","last_unstale":"2024-09-11T07:27:39.603130+0000","last_undegraded":"2024-09-11T07:27:39.603130+0000","last_fullsized":"2024-09-11T07:27:39.603130+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:50.897367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"64'12","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431292+0000","last_change":"2024-09-11T07:26:33.540837+0000","last_active":"2024-09-11T07:27:38.431292+0000","last_peered":"2024-09-11T07:27:38.431292+0000","last_clean":"2024-09-11T07:27:38.431292+0000","last_became_active":"2024-09-11T07:26:33.540363+0000","last_became_peered":"2024-09-11T07:26:33.540363+0000","last_unstale":"2024-09-11T07:27:38.431292+0000","last_undegraded":"2024-09-11T07:27:38.431292+0000","last_fullsized":"2024-09-11T07:27:38.431292+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:49.841213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":61,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:46.077895+0000","last_change":"2024-09-11T07:27:46.077857+0000","last_active":"2024-09-11T07:27:46.077895+0000","last_peered":"2024-09-11T07:27:46.077895+0000","last_clean":"2024-09-11T07:27:46.077895+0000","last_became_active":"2024-09-11T07:26:31.419718+0000","last_became_peered":"2024-09-11T07:26:31.419718+0000","last_unstale":"2024-09-11T07:27:46.077895+0000","last_undegraded":"2024-09-11T07:27:46.077895+0000","last_fullsized":"2024-09-11T07:27:46.077895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:27:46.077786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:46.077786+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:07.735381+0000","scrub_duration":0.0019376720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:53.209546+0000","last_change":"2024-09-11T07:27:53.209518+0000","last_active":"2024-09-11T07:27:53.209546+0000","last_peered":"2024-09-11T07:27:53.209546+0000","last_clean":"2024-09-11T07:27:53.209546+0000","last_became_active":"2024-09-11T07:26:35.561967+0000","last_became_peered":"2024-09-11T07:26:35.561967+0000","last_unstale":"2024-09-11T07:27:53.209546+0000","last_undegraded":"2024-09-11T07:27:53.209546+0000","last_fullsized":"2024-09-11T07:27:53.209546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:53.209452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:53.209452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:23.125281+0000","scrub_duration":0.0014563029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.731999+0000","last_change":"2024-09-11T07:26:37.568046+0000","last_active":"2024-09-11T07:27:38.731999+0000","last_peered":"2024-09-11T07:27:38.731999+0000","last_clean":"2024-09-11T07:27:38.731999+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:27:38.731999+0000","last_undegraded":"2024-09-11T07:27:38.731999+0000","last_fullsized":"2024-09-11T07:27:38.731999+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:48.487836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":58,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.351623+0000","last_change":"2024-09-11T07:27:40.351572+0000","last_active":"2024-09-11T07:27:40.351623+0000","last_peered":"2024-09-11T07:27:40.351623+0000","last_clean":"2024-09-11T07:27:40.351623+0000","last_became_active":"2024-09-11T07:26:33.547344+0000","last_became_peered":"2024-09-11T07:26:33.547344+0000","last_unstale":"2024-09-11T07:27:40.351623+0000","last_undegraded":"2024-09-11T07:27:40.351623+0000","last_fullsized":"2024-09-11T07:27:40.351623+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:27:40.351507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:40.351507+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.098394+0000","scrub_duration":0.0021803980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.012802+0000","last_change":"2024-09-11T07:26:31.416001+0000","last_active":"2024-09-11T07:27:39.012802+0000","last_peered":"2024-09-11T07:27:39.012802+0000","last_clean":"2024-09-11T07:27:39.012802+0000","last_became_active":"2024-09-11T07:26:31.415701+0000","last_became_peered":"2024-09-11T07:26:31.415701+0000","last_unstale":"2024-09-11T07:27:39.012802+0000","last_undegraded":"2024-09-11T07:27:39.012802+0000","last_fullsized":"2024-09-11T07:27:39.012802+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:48.358757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.732551+0000","last_change":"2024-09-11T07:26:35.559563+0000","last_active":"2024-09-11T07:27:38.732551+0000","last_peered":"2024-09-11T07:27:38.732551+0000","last_clean":"2024-09-11T07:27:38.732551+0000","last_became_active":"2024-09-11T07:26:35.559262+0000","last_became_peered":"2024-09-11T07:26:35.559262+0000","last_unstale":"2024-09-11T07:27:38.732551+0000","last_undegraded":"2024-09-11T07:27:38.732551+0000","last_fullsized":"2024-09-11T07:27:38.732551+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:52.116965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.012842+0000","last_change":"2024-09-11T07:26:37.564431+0000","last_active":"2024-09-11T07:27:39.012842+0000","last_peered":"2024-09-11T07:27:39.012842+0000","last_clean":"2024-09-11T07:27:39.012842+0000","last_became_active":"2024-09-11T07:26:37.564084+0000","last_became_peered":"2024-09-11T07:26:37.564084+0000","last_unstale":"2024-09-11T07:27:39.012842+0000","last_undegraded":"2024-09-11T07:27:39.012842+0000","last_fullsized":"2024-09-11T07:27:39.012842+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:52.135522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:45.990660+0000","last_change":"2024-09-11T07:27:45.990617+0000","last_active":"2024-09-11T07:27:45.990660+0000","last_peered":"2024-09-11T07:27:45.990660+0000","last_clean":"2024-09-11T07:27:45.990660+0000","last_became_active":"2024-09-11T07:26:31.423229+0000","last_became_peered":"2024-09-11T07:26:31.423229+0000","last_unstale":"2024-09-11T07:27:45.990660+0000","last_undegraded":"2024-09-11T07:27:45.990660+0000","last_fullsized":"2024-09-11T07:27:45.990660+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:45.990548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:45.990548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:51.641867+0000","scrub_duration":0.0014585150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"82'9","reported_seq":54,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:48.761507+0000","last_change":"2024-09-11T07:27:48.761470+0000","last_active":"2024-09-11T07:27:48.761507+0000","last_peered":"2024-09-11T07:27:48.761507+0000","last_clean":"2024-09-11T07:27:48.761507+0000","last_became_active":"2024-09-11T07:26:33.543628+0000","last_became_peered":"2024-09-11T07:26:33.543628+0000","last_unstale":"2024-09-11T07:27:48.761507+0000","last_undegraded":"2024-09-11T07:27:48.761507+0000","last_fullsized":"2024-09-11T07:27:48.761507+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'9","last_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:27:48.761397+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.913254+0000","scrub_duration":0.0037047439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:42.917767+0000","last_change":"2024-09-11T07:27:42.917739+0000","last_active":"2024-09-11T07:27:42.917767+0000","last_peered":"2024-09-11T07:27:42.917767+0000","last_clean":"2024-09-11T07:27:42.917767+0000","last_became_active":"2024-09-11T07:26:35.566894+0000","last_became_peered":"2024-09-11T07:26:35.566894+0000","last_unstale":"2024-09-11T07:27:42.917767+0000","last_undegraded":"2024-09-11T07:27:42.917767+0000","last_fullsized":"2024-09-11T07:27:42.917767+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.917680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:06.763663+0000","scrub_duration":0.0013496649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:47.031926+0000","last_change":"2024-09-11T07:27:47.031883+0000","last_active":"2024-09-11T07:27:47.031926+0000","last_peered":"2024-09-11T07:27:47.031926+0000","last_clean":"2024-09-11T07:27:47.031926+0000","last_became_active":"2024-09-11T07:26:37.863100+0000","last_became_peered":"2024-09-11T07:26:37.863100+0000","last_unstale":"2024-09-11T07:27:47.031926+0000","last_undegraded":"2024-09-11T07:27:47.031926+0000","last_fullsized":"2024-09-11T07:27:47.031926+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:47.031807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.031807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:54.469643+0000","scrub_duration":0.001556195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"64'10","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.732611+0000","last_change":"2024-09-11T07:26:33.545346+0000","last_active":"2024-09-11T07:27:38.732611+0000","last_peered":"2024-09-11T07:27:38.732611+0000","last_clean":"2024-09-11T07:27:38.732611+0000","last_became_active":"2024-09-11T07:26:33.545078+0000","last_became_peered":"2024-09-11T07:26:33.545078+0000","last_unstale":"2024-09-11T07:27:38.732611+0000","last_undegraded":"2024-09-11T07:27:38.732611+0000","last_fullsized":"2024-09-11T07:27:38.732611+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:56.398062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.603236+0000","last_change":"2024-09-11T07:26:31.424634+0000","last_active":"2024-09-11T07:27:39.603236+0000","last_peered":"2024-09-11T07:27:39.603236+0000","last_clean":"2024-09-11T07:27:39.603236+0000","last_became_active":"2024-09-11T07:26:31.423789+0000","last_became_peered":"2024-09-11T07:26:31.423789+0000","last_unstale":"2024-09-11T07:27:39.603236+0000","last_undegraded":"2024-09-11T07:27:39.603236+0000","last_fullsized":"2024-09-11T07:27:39.603236+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:57.863276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":54,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:44.497412+0000","last_change":"2024-09-11T07:26:35.562509+0000","last_active":"2024-09-11T07:27:44.497412+0000","last_peered":"2024-09-11T07:27:44.497412+0000","last_clean":"2024-09-11T07:27:44.497412+0000","last_became_active":"2024-09-11T07:26:35.562245+0000","last_became_peered":"2024-09-11T07:26:35.562245+0000","last_unstale":"2024-09-11T07:27:44.497412+0000","last_undegraded":"2024-09-11T07:27:44.497412+0000","last_fullsized":"2024-09-11T07:27:44.497412+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:58.799807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:49.245150+0000","last_change":"2024-09-11T07:27:49.245119+0000","last_active":"2024-09-11T07:27:49.245150+0000","last_peered":"2024-09-11T07:27:49.245150+0000","last_clean":"2024-09-11T07:27:49.245150+0000","last_became_active":"2024-09-11T07:26:37.566749+0000","last_became_peered":"2024-09-11T07:26:37.566749+0000","last_unstale":"2024-09-11T07:27:49.245150+0000","last_undegraded":"2024-09-11T07:27:49.245150+0000","last_fullsized":"2024-09-11T07:27:49.245150+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:49.245047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.245047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:49.614854+0000","scrub_duration":0.001867966,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"64'9","reported_seq":48,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.603530+0000","last_change":"2024-09-11T07:26:33.537700+0000","last_active":"2024-09-11T07:27:39.603530+0000","last_peered":"2024-09-11T07:27:39.603530+0000","last_clean":"2024-09-11T07:27:39.603530+0000","last_became_active":"2024-09-11T07:26:33.537425+0000","last_became_peered":"2024-09-11T07:26:33.537425+0000","last_unstale":"2024-09-11T07:27:39.603530+0000","last_undegraded":"2024-09-11T07:27:39.603530+0000","last_fullsized":"2024-09-11T07:27:39.603530+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:46.145102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.429201+0000","last_change":"2024-09-11T07:26:31.419682+0000","last_active":"2024-09-11T07:27:38.429201+0000","last_peered":"2024-09-11T07:27:38.429201+0000","last_clean":"2024-09-11T07:27:38.429201+0000","last_became_active":"2024-09-11T07:26:31.419409+0000","last_became_peered":"2024-09-11T07:26:31.419409+0000","last_unstale":"2024-09-11T07:27:38.429201+0000","last_undegraded":"2024-09-11T07:27:38.429201+0000","last_fullsized":"2024-09-11T07:27:38.429201+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":54,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:44.497272+0000","last_change":"2024-09-11T07:26:35.560917+0000","last_active":"2024-09-11T07:27:44.497272+0000","last_peered":"2024-09-11T07:27:44.497272+0000","last_clean":"2024-09-11T07:27:44.497272+0000","last_became_active":"2024-09-11T07:26:35.560610+0000","last_became_peered":"2024-09-11T07:26:35.560610+0000","last_unstale":"2024-09-11T07:27:44.497272+0000","last_undegraded":"2024-09-11T07:27:44.497272+0000","last_fullsized":"2024-09-11T07:27:44.497272+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:57.052975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431782+0000","last_change":"2024-09-11T07:26:37.566136+0000","last_active":"2024-09-11T07:27:38.431782+0000","last_peered":"2024-09-11T07:27:38.431782+0000","last_clean":"2024-09-11T07:27:38.431782+0000","last_became_active":"2024-09-11T07:26:37.565723+0000","last_became_peered":"2024-09-11T07:26:37.565723+0000","last_unstale":"2024-09-11T07:27:38.431782+0000","last_undegraded":"2024-09-11T07:27:38.431782+0000","last_fullsized":"2024-09-11T07:27:38.431782+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:00.803451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":48,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428656+0000","last_change":"2024-09-11T07:26:33.541085+0000","last_active":"2024-09-11T07:27:38.428656+0000","last_peered":"2024-09-11T07:27:38.428656+0000","last_clean":"2024-09-11T07:27:38.428656+0000","last_became_active":"2024-09-11T07:26:33.540688+0000","last_became_peered":"2024-09-11T07:26:33.540688+0000","last_unstale":"2024-09-11T07:27:38.428656+0000","last_undegraded":"2024-09-11T07:27:38.428656+0000","last_fullsized":"2024-09-11T07:27:38.428656+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:44.875878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.672532+0000","last_change":"2024-09-11T07:26:31.425981+0000","last_active":"2024-09-11T07:27:38.672532+0000","last_peered":"2024-09-11T07:27:38.672532+0000","last_clean":"2024-09-11T07:27:38.672532+0000","last_became_active":"2024-09-11T07:26:31.425587+0000","last_became_peered":"2024-09-11T07:26:31.425587+0000","last_unstale":"2024-09-11T07:27:38.672532+0000","last_undegraded":"2024-09-11T07:27:38.672532+0000","last_fullsized":"2024-09-11T07:27:38.672532+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:46.405065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.732072+0000","last_change":"2024-09-11T07:26:35.562603+0000","last_active":"2024-09-11T07:27:38.732072+0000","last_peered":"2024-09-11T07:27:38.732072+0000","last_clean":"2024-09-11T07:27:38.732072+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:27:38.732072+0000","last_undegraded":"2024-09-11T07:27:38.732072+0000","last_fullsized":"2024-09-11T07:27:38.732072+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:01.555693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428292+0000","last_change":"2024-09-11T07:26:37.569191+0000","last_active":"2024-09-11T07:27:38.428292+0000","last_peered":"2024-09-11T07:27:38.428292+0000","last_clean":"2024-09-11T07:27:38.428292+0000","last_became_active":"2024-09-11T07:26:37.568886+0000","last_became_peered":"2024-09-11T07:26:37.568886+0000","last_unstale":"2024-09-11T07:27:38.428292+0000","last_undegraded":"2024-09-11T07:27:38.428292+0000","last_fullsized":"2024-09-11T07:27:38.428292+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":61,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:48.081485+0000","last_change":"2024-09-11T07:27:48.081460+0000","last_active":"2024-09-11T07:27:48.081485+0000","last_peered":"2024-09-11T07:27:48.081485+0000","last_clean":"2024-09-11T07:27:48.081485+0000","last_became_active":"2024-09-11T07:26:33.540208+0000","last_became_peered":"2024-09-11T07:26:33.540208+0000","last_unstale":"2024-09-11T07:27:48.081485+0000","last_undegraded":"2024-09-11T07:27:48.081485+0000","last_fullsized":"2024-09-11T07:27:48.081485+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:27:48.081392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:48.081392+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:04.524601+0000","scrub_duration":0.0022394590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:49.677089+0000","last_change":"2024-09-11T07:27:49.677063+0000","last_active":"2024-09-11T07:27:49.677089+0000","last_peered":"2024-09-11T07:27:49.677089+0000","last_clean":"2024-09-11T07:27:49.677089+0000","last_became_active":"2024-09-11T07:26:31.422671+0000","last_became_peered":"2024-09-11T07:26:31.422671+0000","last_unstale":"2024-09-11T07:27:49.677089+0000","last_undegraded":"2024-09-11T07:27:49.677089+0000","last_fullsized":"2024-09-11T07:27:49.677089+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.677003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:54.836177+0000","scrub_duration":0.0013962969999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428504+0000","last_change":"2024-09-11T07:26:35.567257+0000","last_active":"2024-09-11T07:27:38.428504+0000","last_peered":"2024-09-11T07:27:38.428504+0000","last_clean":"2024-09-11T07:27:38.428504+0000","last_became_active":"2024-09-11T07:26:35.566931+0000","last_became_peered":"2024-09-11T07:26:35.566931+0000","last_unstale":"2024-09-11T07:27:38.428504+0000","last_undegraded":"2024-09-11T07:27:38.428504+0000","last_fullsized":"2024-09-11T07:27:38.428504+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:55.331264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.731668+0000","last_change":"2024-09-11T07:26:37.561886+0000","last_active":"2024-09-11T07:27:38.731668+0000","last_peered":"2024-09-11T07:27:38.731668+0000","last_clean":"2024-09-11T07:27:38.731668+0000","last_became_active":"2024-09-11T07:26:37.561573+0000","last_became_peered":"2024-09-11T07:26:37.561573+0000","last_unstale":"2024-09-11T07:27:38.731668+0000","last_undegraded":"2024-09-11T07:27:38.731668+0000","last_fullsized":"2024-09-11T07:27:38.731668+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:55.357802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"82'6","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:47.789547+0000","last_change":"2024-09-11T07:27:47.789519+0000","last_active":"2024-09-11T07:27:47.789547+0000","last_peered":"2024-09-11T07:27:47.789547+0000","last_clean":"2024-09-11T07:27:47.789547+0000","last_became_active":"2024-09-11T07:26:33.545830+0000","last_became_peered":"2024-09-11T07:26:33.545830+0000","last_unstale":"2024-09-11T07:27:47.789547+0000","last_undegraded":"2024-09-11T07:27:47.789547+0000","last_fullsized":"2024-09-11T07:27:47.789547+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'6","last_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_deep_scrub":"82'6","last_deep_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.789465+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:59.162056+0000","scrub_duration":0.002624679,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.603611+0000","last_change":"2024-09-11T07:26:31.420799+0000","last_active":"2024-09-11T07:27:39.603611+0000","last_peered":"2024-09-11T07:27:39.603611+0000","last_clean":"2024-09-11T07:27:39.603611+0000","last_became_active":"2024-09-11T07:26:31.420518+0000","last_became_peered":"2024-09-11T07:26:31.420518+0000","last_unstale":"2024-09-11T07:27:39.603611+0000","last_undegraded":"2024-09-11T07:27:39.603611+0000","last_fullsized":"2024-09-11T07:27:39.603611+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:43.691430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.672144+0000","last_change":"2024-09-11T07:26:35.560782+0000","last_active":"2024-09-11T07:27:38.672144+0000","last_peered":"2024-09-11T07:27:38.672144+0000","last_clean":"2024-09-11T07:27:38.672144+0000","last_became_active":"2024-09-11T07:26:35.560506+0000","last_became_peered":"2024-09-11T07:26:35.560506+0000","last_unstale":"2024-09-11T07:27:38.672144+0000","last_undegraded":"2024-09-11T07:27:38.672144+0000","last_fullsized":"2024-09-11T07:27:38.672144+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:55.619984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:42.059097+0000","last_change":"2024-09-11T07:27:42.059066+0000","last_active":"2024-09-11T07:27:42.059097+0000","last_peered":"2024-09-11T07:27:42.059097+0000","last_clean":"2024-09-11T07:27:42.059097+0000","last_became_active":"2024-09-11T07:26:37.564208+0000","last_became_peered":"2024-09-11T07:26:37.564208+0000","last_unstale":"2024-09-11T07:27:42.059097+0000","last_undegraded":"2024-09-11T07:27:42.059097+0000","last_fullsized":"2024-09-11T07:27:42.059097+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.059001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.059001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:51.267826+0000","scrub_duration":0.001418094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.731791+0000","last_change":"2024-09-11T07:26:33.542913+0000","last_active":"2024-09-11T07:27:38.731791+0000","last_peered":"2024-09-11T07:27:38.731791+0000","last_clean":"2024-09-11T07:27:38.731791+0000","last_became_active":"2024-09-11T07:26:33.542625+0000","last_became_peered":"2024-09-11T07:26:33.542625+0000","last_unstale":"2024-09-11T07:27:38.731791+0000","last_undegraded":"2024-09-11T07:27:38.731791+0000","last_fullsized":"2024-09-11T07:27:38.731791+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:58.649007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.672873+0000","last_change":"2024-09-11T07:26:31.426196+0000","last_active":"2024-09-11T07:27:38.672873+0000","last_peered":"2024-09-11T07:27:38.672873+0000","last_clean":"2024-09-11T07:27:38.672873+0000","last_became_active":"2024-09-11T07:26:31.425876+0000","last_became_peered":"2024-09-11T07:26:31.425876+0000","last_unstale":"2024-09-11T07:27:38.672873+0000","last_undegraded":"2024-09-11T07:27:38.672873+0000","last_fullsized":"2024-09-11T07:27:38.672873+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:59.116397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.012605+0000","last_change":"2024-09-11T07:26:35.565591+0000","last_active":"2024-09-11T07:27:39.012605+0000","last_peered":"2024-09-11T07:27:39.012605+0000","last_clean":"2024-09-11T07:27:39.012605+0000","last_became_active":"2024-09-11T07:26:35.565339+0000","last_became_peered":"2024-09-11T07:26:35.565339+0000","last_unstale":"2024-09-11T07:27:39.012605+0000","last_undegraded":"2024-09-11T07:27:39.012605+0000","last_fullsized":"2024-09-11T07:27:39.012605+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:01.008070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.731969+0000","last_change":"2024-09-11T07:26:37.862599+0000","last_active":"2024-09-11T07:27:38.731969+0000","last_peered":"2024-09-11T07:27:38.731969+0000","last_clean":"2024-09-11T07:27:38.731969+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:27:38.731969+0000","last_undegraded":"2024-09-11T07:27:38.731969+0000","last_fullsized":"2024-09-11T07:27:38.731969+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:03.134547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.013085+0000","last_change":"2024-09-11T07:26:37.564199+0000","last_active":"2024-09-11T07:27:39.013085+0000","last_peered":"2024-09-11T07:27:39.013085+0000","last_clean":"2024-09-11T07:27:39.013085+0000","last_became_active":"2024-09-11T07:26:37.563942+0000","last_became_peered":"2024-09-11T07:26:37.563942+0000","last_unstale":"2024-09-11T07:27:39.013085+0000","last_undegraded":"2024-09-11T07:27:39.013085+0000","last_fullsized":"2024-09-11T07:27:39.013085+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:01.320334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.602998+0000","last_change":"2024-09-11T07:26:31.424413+0000","last_active":"2024-09-11T07:27:39.602998+0000","last_peered":"2024-09-11T07:27:39.602998+0000","last_clean":"2024-09-11T07:27:39.602998+0000","last_became_active":"2024-09-11T07:26:31.423932+0000","last_became_peered":"2024-09-11T07:26:31.423932+0000","last_unstale":"2024-09-11T07:27:39.602998+0000","last_undegraded":"2024-09-11T07:27:39.602998+0000","last_fullsized":"2024-09-11T07:27:39.602998+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:48.169462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428600+0000","last_change":"2024-09-11T07:26:33.547101+0000","last_active":"2024-09-11T07:27:38.428600+0000","last_peered":"2024-09-11T07:27:38.428600+0000","last_clean":"2024-09-11T07:27:38.428600+0000","last_became_active":"2024-09-11T07:26:33.546797+0000","last_became_peered":"2024-09-11T07:26:33.546797+0000","last_unstale":"2024-09-11T07:27:38.428600+0000","last_undegraded":"2024-09-11T07:27:38.428600+0000","last_fullsized":"2024-09-11T07:27:38.428600+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"64'8","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:44.497171+0000","last_change":"2024-09-11T07:26:35.561379+0000","last_active":"2024-09-11T07:27:44.497171+0000","last_peered":"2024-09-11T07:27:44.497171+0000","last_clean":"2024-09-11T07:27:44.497171+0000","last_became_active":"2024-09-11T07:26:35.559304+0000","last_became_peered":"2024-09-11T07:26:35.559304+0000","last_unstale":"2024-09-11T07:27:44.497171+0000","last_undegraded":"2024-09-11T07:27:44.497171+0000","last_fullsized":"2024-09-11T07:27:44.497171+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:59.074755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.429308+0000","last_change":"2024-09-11T07:26:37.563308+0000","last_active":"2024-09-11T07:27:38.429308+0000","last_peered":"2024-09-11T07:27:38.429308+0000","last_clean":"2024-09-11T07:27:38.429308+0000","last_became_active":"2024-09-11T07:26:37.562501+0000","last_became_peered":"2024-09-11T07:26:37.562501+0000","last_unstale":"2024-09-11T07:27:38.429308+0000","last_undegraded":"2024-09-11T07:27:38.429308+0000","last_fullsized":"2024-09-11T07:27:38.429308+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:52.287238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.732915+0000","last_change":"2024-09-11T07:27:38.732867+0000","last_active":"2024-09-11T07:27:38.732915+0000","last_peered":"2024-09-11T07:27:38.732915+0000","last_clean":"2024-09-11T07:27:38.732915+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:27:38.732915+0000","last_undegraded":"2024-09-11T07:27:38.732915+0000","last_fullsized":"2024-09-11T07:27:38.732915+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:38.732795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:38.732795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:56.730699+0000","scrub_duration":0.0025091950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.672775+0000","last_change":"2024-09-11T07:26:33.541438+0000","last_active":"2024-09-11T07:27:38.672775+0000","last_peered":"2024-09-11T07:27:38.672775+0000","last_clean":"2024-09-11T07:27:38.672775+0000","last_became_active":"2024-09-11T07:26:33.541154+0000","last_became_peered":"2024-09-11T07:26:33.541154+0000","last_unstale":"2024-09-11T07:27:38.672775+0000","last_undegraded":"2024-09-11T07:27:38.672775+0000","last_fullsized":"2024-09-11T07:27:38.672775+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:02.386606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428544+0000","last_change":"2024-09-11T07:26:35.559093+0000","last_active":"2024-09-11T07:27:38.428544+0000","last_peered":"2024-09-11T07:27:38.428544+0000","last_clean":"2024-09-11T07:27:38.428544+0000","last_became_active":"2024-09-11T07:26:35.558809+0000","last_became_peered":"2024-09-11T07:26:35.558809+0000","last_unstale":"2024-09-11T07:27:38.428544+0000","last_undegraded":"2024-09-11T07:27:38.428544+0000","last_fullsized":"2024-09-11T07:27:38.428544+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:00.864666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431627+0000","last_change":"2024-09-11T07:26:37.862660+0000","last_active":"2024-09-11T07:27:38.431627+0000","last_peered":"2024-09-11T07:27:38.431627+0000","last_clean":"2024-09-11T07:27:38.431627+0000","last_became_active":"2024-09-11T07:26:37.862373+0000","last_became_peered":"2024-09-11T07:26:37.862373+0000","last_unstale":"2024-09-11T07:27:38.431627+0000","last_undegraded":"2024-09-11T07:27:38.431627+0000","last_fullsized":"2024-09-11T07:27:38.431627+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:49.412299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.672490+0000","last_change":"2024-09-11T07:27:33.983889+0000","last_active":"2024-09-11T07:27:38.672490+0000","last_peered":"2024-09-11T07:27:38.672490+0000","last_clean":"2024-09-11T07:27:38.672490+0000","last_became_active":"2024-09-11T07:26:31.424651+0000","last_became_peered":"2024-09-11T07:26:31.424651+0000","last_unstale":"2024-09-11T07:27:38.672490+0000","last_undegraded":"2024-09-11T07:27:38.672490+0000","last_fullsized":"2024-09-11T07:27:38.672490+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:33.983813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:33.983813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:55.264233+0000","scrub_duration":0.001290759,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":57,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.013092+0000","last_change":"2024-09-11T07:26:33.542921+0000","last_active":"2024-09-11T07:27:39.013092+0000","last_peered":"2024-09-11T07:27:39.013092+0000","last_clean":"2024-09-11T07:27:39.013092+0000","last_became_active":"2024-09-11T07:26:33.542544+0000","last_became_peered":"2024-09-11T07:26:33.542544+0000","last_unstale":"2024-09-11T07:27:39.013092+0000","last_undegraded":"2024-09-11T07:27:39.013092+0000","last_fullsized":"2024-09-11T07:27:39.013092+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:52.019544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.429601+0000","last_change":"2024-09-11T07:26:35.556136+0000","last_active":"2024-09-11T07:27:38.429601+0000","last_peered":"2024-09-11T07:27:38.429601+0000","last_clean":"2024-09-11T07:27:38.429601+0000","last_became_active":"2024-09-11T07:26:35.555862+0000","last_became_peered":"2024-09-11T07:26:35.555862+0000","last_unstale":"2024-09-11T07:27:38.429601+0000","last_undegraded":"2024-09-11T07:27:38.429601+0000","last_fullsized":"2024-09-11T07:27:38.429601+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:48.624824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.732535+0000","last_change":"2024-09-11T07:26:37.862060+0000","last_active":"2024-09-11T07:27:38.732535+0000","last_peered":"2024-09-11T07:27:38.732535+0000","last_clean":"2024-09-11T07:27:38.732535+0000","last_became_active":"2024-09-11T07:26:37.861647+0000","last_became_peered":"2024-09-11T07:26:37.861647+0000","last_unstale":"2024-09-11T07:27:38.732535+0000","last_undegraded":"2024-09-11T07:27:38.732535+0000","last_fullsized":"2024-09-11T07:27:38.732535+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:52.108042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:45.732651+0000","last_change":"2024-09-11T07:27:45.732604+0000","last_active":"2024-09-11T07:27:45.732651+0000","last_peered":"2024-09-11T07:27:45.732651+0000","last_clean":"2024-09-11T07:27:45.732651+0000","last_became_active":"2024-09-11T07:26:31.420598+0000","last_became_peered":"2024-09-11T07:26:31.420598+0000","last_unstale":"2024-09-11T07:27:45.732651+0000","last_undegraded":"2024-09-11T07:27:45.732651+0000","last_fullsized":"2024-09-11T07:27:45.732651+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:45.732528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:45.732528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:48.605075+0000","scrub_duration":0.0015466659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"64'12","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.732469+0000","last_change":"2024-09-11T07:26:33.543016+0000","last_active":"2024-09-11T07:27:38.732469+0000","last_peered":"2024-09-11T07:27:38.732469+0000","last_clean":"2024-09-11T07:27:38.732469+0000","last_became_active":"2024-09-11T07:26:33.542711+0000","last_became_peered":"2024-09-11T07:26:33.542711+0000","last_unstale":"2024-09-11T07:27:38.732469+0000","last_undegraded":"2024-09-11T07:27:38.732469+0000","last_fullsized":"2024-09-11T07:27:38.732469+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:44.518917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431617+0000","last_change":"2024-09-11T07:26:35.562350+0000","last_active":"2024-09-11T07:27:38.431617+0000","last_peered":"2024-09-11T07:27:38.431617+0000","last_clean":"2024-09-11T07:27:38.431617+0000","last_became_active":"2024-09-11T07:26:35.562037+0000","last_became_peered":"2024-09-11T07:26:35.562037+0000","last_unstale":"2024-09-11T07:27:38.431617+0000","last_undegraded":"2024-09-11T07:27:38.431617+0000","last_fullsized":"2024-09-11T07:27:38.431617+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:42.573835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":405,"ondisk_log_size":405,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299413,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298768,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446384,"statfs":{"total":95995035648,"available":95689097216,"internally_reserved":0,"allocated":1449984,"data_stored":943568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.52600000000000002}]},{"osd":1,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.377},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.34899999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.46500000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":12.625999999999999,"5min":12.625999999999999,"15min":12.625999999999999},"last":0.432},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.41699999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.50900000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.49199999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54200000000000004}]}]},{"osd":6,"up_from":44,"seq":188978561049,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298744,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446408,"statfs":{"total":95995035648,"available":95689121792,"internally_reserved":0,"allocated":1425408,"data_stored":942046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.374,"5min":0.247,"15min":0.247},"max":{"1min":0.65000000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.374},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.60499999999999998,"15min":0.624},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.754,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.49299999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.82899999999999996,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.49299999999999999,"15min":0.495},"min":{"1min":0.35799999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.48199999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.61499999999999999,"15min":0.63500000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.70299999999999996,"5min":3.234,"15min":3.234},"last":0.317},{"interface":"front","average":{"1min":0.496,"5min":0.59099999999999997,"15min":0.60699999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.80200000000000005,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.436}]},{"osd":3,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51500000000000001,"15min":0.51800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.63500000000000001,"15min":0.65500000000000003},"min":{"1min":0.41199999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.66900000000000004,"5min":3.1739999999999999,"15min":3.1739999999999999},"last":0.46000000000000002}]},{"osd":4,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.58299999999999996,"15min":0.60899999999999999},"min":{"1min":0.32700000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.65200000000000002,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.28399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.42199999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.65600000000000003,"15min":0.69699999999999995},"min":{"1min":0.32400000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64000000000000001,"5min":5.7329999999999997,"15min":5.7329999999999997},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.52000000000000002,"15min":0.52700000000000002},"min":{"1min":0.34000000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70899999999999996,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.38700000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:27:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.6349999999999998,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.47299999999999998}]}]},{"osd":5,"up_from":37,"seq":158913789983,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298168,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446984,"statfs":{"total":95995035648,"available":95689711616,"internally_reserved":0,"allocated":835584,"data_stored":352559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.35499999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.50600000000000001,"15min":0.502},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.36299999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.32800000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.373},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.45100000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.44},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.42099999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.375,"15min":0.37},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.42599999999999999,"15min":0.41999999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.40699999999999997}]},{"osd":6,"last update":"Wed Sep 11 07:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.34999999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:27:21 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.33500000000000002}]}]},{"osd":4,"up_from":31,"seq":133143986211,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298192,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446960,"statfs":{"total":95995035648,"available":95689687040,"internally_reserved":0,"allocated":860160,"data_stored":353004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51600000000000001,"15min":0.51300000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68999999999999995,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.502,"15min":0.498},"min":{"1min":0.36099999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.55800000000000005}]},{"osd":1,"last update":"Wed Sep 11 07:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.5,"15min":0.49399999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.71299999999999997,"15min":0.73699999999999999},"min":{"1min":0.372,"5min":0.318,"15min":0.318},"max":{"1min":1.1200000000000001,"5min":4.5510000000000002,"15min":4.5510000000000002},"last":0.68600000000000005}]},{"osd":2,"last update":"Wed Sep 11 07:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.73099999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77400000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.57099999999999995}]},{"osd":5,"last update":"Wed Sep 11 07:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.34499999999999997,"15min":0.33800000000000002},"min":{"1min":0.27600000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.378,"15min":0.372},"min":{"1min":0.29399999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.38400000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.42499999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:27:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.36499999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182440,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298200,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446952,"statfs":{"total":95995035648,"available":95689678848,"internally_reserved":0,"allocated":868352,"data_stored":352468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:26:52 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.39100000000000001,"15min":0.39500000000000002},"min":{"1min":0.21099999999999999,"5min":0.185,"15min":0.185},"max":{"1min":0.55000000000000004,"5min":0.621,"15min":0.621},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.435,"15min":0.44400000000000001},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.56299999999999994,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.38400000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:26:52 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.441,"15min":0.44},"min":{"1min":0.28999999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.42499999999999999,"15min":0.434},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.372}]},{"osd":2,"last update":"Wed Sep 11 07:26:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.46300000000000002,"15min":0.46899999999999997},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.46600000000000003,"15min":0.47299999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.67700000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.32300000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:27:12 2024","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":0.746,"15min":0.62},"min":{"1min":0.47299999999999998,"5min":0.376,"15min":0.376},"max":{"1min":21.856000000000002,"5min":21.856000000000002,"15min":21.856000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":1.345,"5min":0.72199999999999998,"15min":0.61799999999999999},"min":{"1min":0.432,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":19.146000000000001,"5min":19.146000000000001,"15min":19.146000000000001},"last":0.55500000000000005}]},{"osd":5,"last update":"Wed Sep 11 07:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":0.71999999999999997,"15min":0.622},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":18.416,"5min":18.416,"15min":18.416},"last":0.59299999999999997}]},{"osd":6,"last update":"Wed Sep 11 07:26:53 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":20.219999999999999,"5min":20.219999999999999,"15min":20.219999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":1.51,"5min":1.51,"15min":1.51},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":19.609000000000002,"5min":19.609000000000002,"15min":19.609000000000002},"last":0.46899999999999997}]},{"osd":7,"last update":"Wed Sep 11 07:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.61199999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298164,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446988,"statfs":{"total":95995035648,"available":95689715712,"internally_reserved":0,"allocated":831488,"data_stored":351805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:26 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.375,"15min":0.36299999999999999},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.435},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.38200000000000001,"15min":0.374},"min":{"1min":0.26600000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.55900000000000005,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.33000000000000002}]},{"osd":1,"last update":"Wed Sep 11 07:27:26 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.40300000000000002,"15min":0.38600000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.40200000000000002,"15min":0.38300000000000001},"min":{"1min":0.218,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.38300000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.439,"15min":0.42299999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64500000000000002,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.45900000000000002,"15min":0.45400000000000001},"min":{"1min":0.26600000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65000000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.308}]},{"osd":4,"last update":"Wed Sep 11 07:27:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55800000000000005,"15min":0.55300000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57699999999999996,"15min":0.57599999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80700000000000005,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.55500000000000005}]},{"osd":5,"last update":"Wed Sep 11 07:27:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.68799999999999994,"15min":0.69699999999999995},"min":{"1min":0.45500000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85899999999999999,"5min":2.931,"15min":2.931},"last":0.56799999999999995}]},{"osd":6,"last update":"Wed Sep 11 07:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.622},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.66500000000000004}]},{"osd":7,"last update":"Wed Sep 11 07:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.58999999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298216,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446936,"statfs":{"total":95995035648,"available":95689662464,"internally_reserved":0,"allocated":884736,"data_stored":353325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:02 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.34100000000000003,"15min":0.33100000000000002},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.45800000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.36399999999999999,"15min":0.35499999999999998},"min":{"1min":0.219,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.54700000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.27300000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:27:26 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.41899999999999998,"15min":0.41599999999999998},"min":{"1min":0.317,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.41199999999999998,"15min":0.40799999999999997},"min":{"1min":0.28699999999999998,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.33000000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:27:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.44500000000000001,"15min":0.44700000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.60699999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.41399999999999998},"min":{"1min":0.307,"5min":0.253,"15min":0.253},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.307}]},{"osd":4,"last update":"Wed Sep 11 07:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.64600000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:27:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.44,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.71699999999999997,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.625}]},{"osd":6,"last update":"Wed Sep 11 07:26:54 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.69499999999999995}]},{"osd":7,"last update":"Wed Sep 11 07:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.65800000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298760,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446392,"statfs":{"total":95995035648,"available":95689105408,"internally_reserved":0,"allocated":1441792,"data_stored":942378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:26:59 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.35899999999999999,"15min":0.34300000000000003},"min":{"1min":0.26600000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.248},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.35199999999999998,"15min":0.34000000000000002},"min":{"1min":0.24299999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.33100000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:27:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.40400000000000003,"15min":0.40699999999999997},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.434},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.59299999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.307}]},{"osd":3,"last update":"Wed Sep 11 07:26:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.46899999999999997,"15min":0.47699999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.56599999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.376},{"interface":"front","average":{"1min":0.44,"5min":0.47999999999999998,"15min":0.48599999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59299999999999997,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.36299999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56299999999999994,"15min":0.56899999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69499999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.745},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59799999999999998,"15min":0.60099999999999998},"min":{"1min":0.42499999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.66300000000000003}]},{"osd":5,"last update":"Wed Sep 11 07:27:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.40000000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.79100000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.505},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.41699999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78100000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.63800000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:26:52 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.72499999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.69599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:27:54.754 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:27:55.030 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:55.152 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:54 smithi022 bash[70777]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-11T07:27:55.280 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=infra.usagestats.collector t=2024-09-11T07:27:54.941711657Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-11T07:27:55.281 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=provisioning.datasources t=2024-09-11T07:27:54.942392738Z level=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-09-11T07:27:55.281 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=provisioning.datasources t=2024-09-11T07:27:54.975387809Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-11T07:27:55.281 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:55 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=provisioning.datasources t=2024-09-11T07:27:55.016988874Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-11T07:27:55.281 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:55 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=provisioning.alerting t=2024-09-11T07:27:55.046120084Z level=info msg="starting to provision alerting" 2024-09-11T07:27:55.281 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:55 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=provisioning.alerting t=2024-09-11T07:27:55.046154311Z level=info msg="finished to provision alerting" 2024-09-11T07:27:55.281 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:55 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=grafanaStorageLogger t=2024-09-11T07:27:55.046737622Z level=info msg="storage starting" 2024-09-11T07:27:55.281 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:55 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=ngalert.state.manager t=2024-09-11T07:27:55.047269758Z level=info msg="Warming state cache for startup" 2024-09-11T07:27:55.281 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:55 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=http.server t=2024-09-11T07:27:55.052635581Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-11T07:27:55.281 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:55 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=ngalert.state.manager t=2024-09-11T07:27:55.057932607Z level=info msg="State cache has been initialized" states=0 duration=10.662172ms 2024-09-11T07:27:55.281 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:55 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=ticker t=2024-09-11T07:27:55.057993881Z level=info msg=starting first_tick=2024-09-11T07:28:00Z 2024-09-11T07:27:55.281 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:55 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=ngalert.multiorg.alertmanager t=2024-09-11T07:27:55.057990358Z level=info msg="starting MultiOrg Alertmanager" 2024-09-11T07:27:55.282 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:55 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=sqlstore.transactions t=2024-09-11T07:27:55.104432156Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T07:27:55.282 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:55 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=sqlstore.transactions t=2024-09-11T07:27:55.126963118Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T07:27:55.282 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:27:55 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=sqlstore.transactions t=2024-09-11T07:27:55.137456189Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-11T07:27:55.417 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:55 smithi022 bash[70777]: Getting image source signatures 2024-09-11T07:27:55.417 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:55 smithi022 bash[70777]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-11T07:27:55.417 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:55 smithi022 bash[70777]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-11T07:27:55.417 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:55 smithi022 bash[70777]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-11T07:27:55.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:55 smithi104 ceph-mon[33507]: 2.14 deep-scrub starts 2024-09-11T07:27:55.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:55 smithi104 ceph-mon[33507]: 2.14 deep-scrub ok 2024-09-11T07:27:55.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:55 smithi104 ceph-mon[33507]: 6.0 deep-scrub starts 2024-09-11T07:27:55.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:55 smithi104 ceph-mon[33507]: 6.0 deep-scrub ok 2024-09-11T07:27:55.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:55 smithi104 ceph-mon[33507]: 6.4 scrub starts 2024-09-11T07:27:55.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:55 smithi104 ceph-mon[33507]: 6.4 scrub ok 2024-09-11T07:27:55.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:55 smithi022 ceph-mon[29462]: 2.14 deep-scrub starts 2024-09-11T07:27:55.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:55 smithi022 ceph-mon[29462]: 2.14 deep-scrub ok 2024-09-11T07:27:55.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:55 smithi022 ceph-mon[29462]: 6.0 deep-scrub starts 2024-09-11T07:27:55.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:55 smithi022 ceph-mon[29462]: 6.0 deep-scrub ok 2024-09-11T07:27:55.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:55 smithi022 ceph-mon[29462]: 6.4 scrub starts 2024-09-11T07:27:55.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:55 smithi022 ceph-mon[29462]: 6.4 scrub ok 2024-09-11T07:27:55.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:55 smithi022 ceph-mon[37330]: 2.14 deep-scrub starts 2024-09-11T07:27:55.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:55 smithi022 ceph-mon[37330]: 2.14 deep-scrub ok 2024-09-11T07:27:55.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:55 smithi022 ceph-mon[37330]: 6.0 deep-scrub starts 2024-09-11T07:27:55.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:55 smithi022 ceph-mon[37330]: 6.0 deep-scrub ok 2024-09-11T07:27:55.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:55 smithi022 ceph-mon[37330]: 6.4 scrub starts 2024-09-11T07:27:55.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:55 smithi022 ceph-mon[37330]: 6.4 scrub ok 2024-09-11T07:27:56.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:56 smithi022 ceph-mon[29462]: 6.5 scrub starts 2024-09-11T07:27:56.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:56 smithi022 ceph-mon[29462]: 6.5 scrub ok 2024-09-11T07:27:56.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:56 smithi022 ceph-mon[29462]: 4.1e scrub starts 2024-09-11T07:27:56.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:56 smithi022 ceph-mon[29462]: 4.1e scrub ok 2024-09-11T07:27:56.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:56 smithi022 ceph-mon[29462]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:27:56.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:56 smithi022 ceph-mon[37330]: 6.5 scrub starts 2024-09-11T07:27:56.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:56 smithi022 ceph-mon[37330]: 6.5 scrub ok 2024-09-11T07:27:56.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:56 smithi022 ceph-mon[37330]: 4.1e scrub starts 2024-09-11T07:27:56.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:56 smithi022 ceph-mon[37330]: 4.1e scrub ok 2024-09-11T07:27:56.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:56 smithi022 ceph-mon[37330]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:27:56.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:56 smithi104 ceph-mon[33507]: 6.5 scrub starts 2024-09-11T07:27:56.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:56 smithi104 ceph-mon[33507]: 6.5 scrub ok 2024-09-11T07:27:56.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:56 smithi104 ceph-mon[33507]: 4.1e scrub starts 2024-09-11T07:27:56.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:56 smithi104 ceph-mon[33507]: 4.1e scrub ok 2024-09-11T07:27:56.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:56 smithi104 ceph-mon[33507]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:27:57.251 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:57 smithi022 bash[70777]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-11T07:27:57.667 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:57 smithi022 bash[70777]: Writing manifest to image destination 2024-09-11T07:27:57.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:57 smithi022 ceph-mon[29462]: 5.11 scrub starts 2024-09-11T07:27:57.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:57 smithi022 ceph-mon[29462]: 5.11 scrub ok 2024-09-11T07:27:57.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:57 smithi022 ceph-mon[29462]: 5.e scrub starts 2024-09-11T07:27:57.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:57 smithi022 ceph-mon[29462]: 5.e scrub ok 2024-09-11T07:27:57.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:57 smithi022 ceph-mon[29462]: 6.f scrub starts 2024-09-11T07:27:57.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:57 smithi022 ceph-mon[29462]: 6.f scrub ok 2024-09-11T07:27:57.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:57 smithi022 ceph-mon[37330]: 5.11 scrub starts 2024-09-11T07:27:57.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:57 smithi022 ceph-mon[37330]: 5.11 scrub ok 2024-09-11T07:27:57.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:57 smithi022 ceph-mon[37330]: 5.e scrub starts 2024-09-11T07:27:57.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:57 smithi022 ceph-mon[37330]: 5.e scrub ok 2024-09-11T07:27:57.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:57 smithi022 ceph-mon[37330]: 6.f scrub starts 2024-09-11T07:27:57.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:57 smithi022 ceph-mon[37330]: 6.f scrub ok 2024-09-11T07:27:57.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:57 smithi104 ceph-mon[33507]: 5.11 scrub starts 2024-09-11T07:27:57.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:57 smithi104 ceph-mon[33507]: 5.11 scrub ok 2024-09-11T07:27:57.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:57 smithi104 ceph-mon[33507]: 5.e scrub starts 2024-09-11T07:27:57.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:57 smithi104 ceph-mon[33507]: 5.e scrub ok 2024-09-11T07:27:57.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:57 smithi104 ceph-mon[33507]: 6.f scrub starts 2024-09-11T07:27:57.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:57 smithi104 ceph-mon[33507]: 6.f scrub ok 2024-09-11T07:27:57.838 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:27:57.840 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:27:58.264 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 podman[70777]: 2024-09-11 07:27:58.106546831 +0000 UTC m=+3.277991267 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-11T07:27:58.642 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-mon[29462]: 5.1b scrub starts 2024-09-11T07:27:58.642 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-mon[29462]: 5.1b scrub ok 2024-09-11T07:27:58.642 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-mon[29462]: 5.1c scrub starts 2024-09-11T07:27:58.642 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-mon[29462]: 5.1c scrub ok 2024-09-11T07:27:58.642 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-mon[29462]: 4.12 scrub starts 2024-09-11T07:27:58.642 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-mon[29462]: 4.12 scrub ok 2024-09-11T07:27:58.642 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-mon[29462]: from='client.24644 v1:172.21.15.22:0/3616785996' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:27:58.642 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-mon[29462]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:27:58.642 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:58.643 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-mon[37330]: 5.1b scrub starts 2024-09-11T07:27:58.643 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-mon[37330]: 5.1b scrub ok 2024-09-11T07:27:58.643 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-mon[37330]: 5.1c scrub starts 2024-09-11T07:27:58.643 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-mon[37330]: 5.1c scrub ok 2024-09-11T07:27:58.643 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-mon[37330]: 4.12 scrub starts 2024-09-11T07:27:58.643 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-mon[37330]: 4.12 scrub ok 2024-09-11T07:27:58.643 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-mon[37330]: from='client.24644 v1:172.21.15.22:0/3616785996' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:27:58.643 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-mon[37330]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:27:58.643 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:58.644 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 podman[70777]: 2024-09-11 07:27:58.26331142 +0000 UTC m=+3.434755868 container create 494ea4af76a7526556334b0baa2251f388c5ae977eda9bb11f3793b0cd33b5b5 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T07:27:58.644 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 podman[70777]: 2024-09-11 07:27:58.465634035 +0000 UTC m=+3.637078487 container init 494ea4af76a7526556334b0baa2251f388c5ae977eda9bb11f3793b0cd33b5b5 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T07:27:58.644 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 podman[70777]: 2024-09-11 07:27:58.468660292 +0000 UTC m=+3.640104739 container start 494ea4af76a7526556334b0baa2251f388c5ae977eda9bb11f3793b0cd33b5b5 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T07:27:58.644 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.472Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-11T07:27:58.644 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.472Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-11T07:27:58.644 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.472Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-11T07:27:58.644 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.472Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-11T07:27:58.644 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.473Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-11T07:27:58.644 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.473Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-11T07:27:58.644 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-11T07:27:58.645 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=arp 2024-09-11T07:27:58.645 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-11T07:27:58.645 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-11T07:27:58.645 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-11T07:27:58.645 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-11T07:27:58.645 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-11T07:27:58.645 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-11T07:27:58.645 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-11T07:27:58.645 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-11T07:27:58.645 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=edac 2024-09-11T07:27:58.645 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-11T07:27:58.645 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-11T07:27:58.645 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-11T07:27:58.645 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-11T07:27:58.645 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-11T07:27:58.646 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-11T07:27:58.646 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-11T07:27:58.646 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-11T07:27:58.646 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-11T07:27:58.646 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-11T07:27:58.646 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-11T07:27:58.646 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-11T07:27:58.646 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-11T07:27:58.646 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-11T07:27:58.646 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-11T07:27:58.646 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-11T07:27:58.646 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=os 2024-09-11T07:27:58.646 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-11T07:27:58.646 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-11T07:27:58.646 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-11T07:27:58.646 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-11T07:27:58.647 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-11T07:27:58.647 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-11T07:27:58.647 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-11T07:27:58.647 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=stat 2024-09-11T07:27:58.647 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-11T07:27:58.647 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-11T07:27:58.647 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-11T07:27:58.647 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=time 2024-09-11T07:27:58.647 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-11T07:27:58.647 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=uname 2024-09-11T07:27:58.647 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-11T07:27:58.647 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-11T07:27:58.647 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-11T07:27:58.647 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-11T07:27:58.648 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a[70995]: ts=2024-09-11T07:27:58.474Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-11T07:27:58.648 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 bash[70777]: 494ea4af76a7526556334b0baa2251f388c5ae977eda9bb11f3793b0cd33b5b5 2024-09-11T07:27:58.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:58 smithi104 ceph-mon[33507]: 5.1b scrub starts 2024-09-11T07:27:58.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:58 smithi104 ceph-mon[33507]: 5.1b scrub ok 2024-09-11T07:27:58.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:58 smithi104 ceph-mon[33507]: 5.1c scrub starts 2024-09-11T07:27:58.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:58 smithi104 ceph-mon[33507]: 5.1c scrub ok 2024-09-11T07:27:58.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:58 smithi104 ceph-mon[33507]: 4.12 scrub starts 2024-09-11T07:27:58.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:58 smithi104 ceph-mon[33507]: 4.12 scrub ok 2024-09-11T07:27:58.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:58 smithi104 ceph-mon[33507]: from='client.24644 v1:172.21.15.22:0/3616785996' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:27:58.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:58 smithi104 ceph-mon[33507]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:27:58.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:58 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:58.917 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:27:58 smithi022 systemd[1]: Started Ceph node-exporter.a for 12a285d8-700e-11ef-bcea-c7b262605968. 2024-09-11T07:27:59.136 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-09-11T07:27:55.944437+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":929,"num_read_kb":786,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":562,"ondisk_log_size":562,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2387212,"kb_used_data":8396,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747574004,"statfs":{"total":767960285184,"available":765515780096,"internally_reserved":0,"allocated":8597504,"data_stored":4591153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003357"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.731875+0000","last_change":"2024-09-11T07:26:37.863505+0000","last_active":"2024-09-11T07:27:38.731875+0000","last_peered":"2024-09-11T07:27:38.731875+0000","last_clean":"2024-09-11T07:27:38.731875+0000","last_became_active":"2024-09-11T07:26:37.863165+0000","last_became_peered":"2024-09-11T07:26:37.863165+0000","last_unstale":"2024-09-11T07:27:38.731875+0000","last_undegraded":"2024-09-11T07:27:38.731875+0000","last_fullsized":"2024-09-11T07:27:38.731875+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:05.169084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:51.198008+0000","last_change":"2024-09-11T07:27:51.197966+0000","last_active":"2024-09-11T07:27:51.198008+0000","last_peered":"2024-09-11T07:27:51.198008+0000","last_clean":"2024-09-11T07:27:51.198008+0000","last_became_active":"2024-09-11T07:26:31.421005+0000","last_became_peered":"2024-09-11T07:26:31.421005+0000","last_unstale":"2024-09-11T07:27:51.198008+0000","last_undegraded":"2024-09-11T07:27:51.198008+0000","last_fullsized":"2024-09-11T07:27:51.198008+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:51.197903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:51.197903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:00.681709+0000","scrub_duration":0.001600113,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"82'22","reported_seq":74,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:50.756811+0000","last_change":"2024-09-11T07:27:50.756777+0000","last_active":"2024-09-11T07:27:50.756811+0000","last_peered":"2024-09-11T07:27:50.756811+0000","last_clean":"2024-09-11T07:27:50.756811+0000","last_became_active":"2024-09-11T07:26:33.544627+0000","last_became_peered":"2024-09-11T07:26:33.544627+0000","last_unstale":"2024-09-11T07:27:50.756811+0000","last_undegraded":"2024-09-11T07:27:50.756811+0000","last_fullsized":"2024-09-11T07:27:50.756811+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.756707+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.257406+0000","scrub_duration":0.0049180669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431123+0000","last_change":"2024-09-11T07:26:35.557117+0000","last_active":"2024-09-11T07:27:38.431123+0000","last_peered":"2024-09-11T07:27:38.431123+0000","last_clean":"2024-09-11T07:27:38.431123+0000","last_became_active":"2024-09-11T07:26:35.556296+0000","last_became_peered":"2024-09-11T07:26:35.556296+0000","last_unstale":"2024-09-11T07:27:38.431123+0000","last_undegraded":"2024-09-11T07:27:38.431123+0000","last_fullsized":"2024-09-11T07:27:38.431123+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:56.174508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.732645+0000","last_change":"2024-09-11T07:27:36.776948+0000","last_active":"2024-09-11T07:27:38.732645+0000","last_peered":"2024-09-11T07:27:38.732645+0000","last_clean":"2024-09-11T07:27:38.732645+0000","last_became_active":"2024-09-11T07:26:31.420178+0000","last_became_peered":"2024-09-11T07:26:31.420178+0000","last_unstale":"2024-09-11T07:27:38.732645+0000","last_undegraded":"2024-09-11T07:27:38.732645+0000","last_fullsized":"2024-09-11T07:27:38.732645+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:36.776883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:36.776883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:49.961787+0000","scrub_duration":0.0017449290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"64'9","reported_seq":48,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431116+0000","last_change":"2024-09-11T07:26:33.541403+0000","last_active":"2024-09-11T07:27:38.431116+0000","last_peered":"2024-09-11T07:27:38.431116+0000","last_clean":"2024-09-11T07:27:38.431116+0000","last_became_active":"2024-09-11T07:26:33.540732+0000","last_became_peered":"2024-09-11T07:26:33.540732+0000","last_unstale":"2024-09-11T07:27:38.431116+0000","last_undegraded":"2024-09-11T07:27:38.431116+0000","last_fullsized":"2024-09-11T07:27:38.431116+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:53.423645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:50.648499+0000","last_change":"2024-09-11T07:27:50.648473+0000","last_active":"2024-09-11T07:27:50.648499+0000","last_peered":"2024-09-11T07:27:50.648499+0000","last_clean":"2024-09-11T07:27:50.648499+0000","last_became_active":"2024-09-11T07:26:35.561096+0000","last_became_peered":"2024-09-11T07:26:35.561096+0000","last_unstale":"2024-09-11T07:27:50.648499+0000","last_undegraded":"2024-09-11T07:27:50.648499+0000","last_fullsized":"2024-09-11T07:27:50.648499+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.648420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:05.663592+0000","scrub_duration":0.001391446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431209+0000","last_change":"2024-09-11T07:26:37.567195+0000","last_active":"2024-09-11T07:27:38.431209+0000","last_peered":"2024-09-11T07:27:38.431209+0000","last_clean":"2024-09-11T07:27:38.431209+0000","last_became_active":"2024-09-11T07:26:37.566910+0000","last_became_peered":"2024-09-11T07:26:37.566910+0000","last_unstale":"2024-09-11T07:27:38.431209+0000","last_undegraded":"2024-09-11T07:27:38.431209+0000","last_fullsized":"2024-09-11T07:27:38.431209+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:57.605592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.013548+0000","last_change":"2024-09-11T07:26:31.423041+0000","last_active":"2024-09-11T07:27:39.013548+0000","last_peered":"2024-09-11T07:27:39.013548+0000","last_clean":"2024-09-11T07:27:39.013548+0000","last_became_active":"2024-09-11T07:26:31.422772+0000","last_became_peered":"2024-09-11T07:26:31.422772+0000","last_unstale":"2024-09-11T07:27:39.013548+0000","last_undegraded":"2024-09-11T07:27:39.013548+0000","last_fullsized":"2024-09-11T07:27:39.013548+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:00.083055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"82'7","reported_seq":54,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:52.097212+0000","last_change":"2024-09-11T07:27:52.097170+0000","last_active":"2024-09-11T07:27:52.097212+0000","last_peered":"2024-09-11T07:27:52.097212+0000","last_clean":"2024-09-11T07:27:52.097212+0000","last_became_active":"2024-09-11T07:26:33.540351+0000","last_became_peered":"2024-09-11T07:26:33.540351+0000","last_unstale":"2024-09-11T07:27:52.097212+0000","last_undegraded":"2024-09-11T07:27:52.097212+0000","last_fullsized":"2024-09-11T07:27:52.097212+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'7","last_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_deep_scrub":"82'7","last_deep_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_clean_scrub_stamp":"2024-09-11T07:27:52.097071+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:53.905741+0000","scrub_duration":0.0025786670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.012856+0000","last_change":"2024-09-11T07:26:35.563858+0000","last_active":"2024-09-11T07:27:39.012856+0000","last_peered":"2024-09-11T07:27:39.012856+0000","last_clean":"2024-09-11T07:27:39.012856+0000","last_became_active":"2024-09-11T07:26:35.563536+0000","last_became_peered":"2024-09-11T07:26:35.563536+0000","last_unstale":"2024-09-11T07:27:39.012856+0000","last_undegraded":"2024-09-11T07:27:39.012856+0000","last_fullsized":"2024-09-11T07:27:39.012856+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:01.379998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.603094+0000","last_change":"2024-09-11T07:26:37.562008+0000","last_active":"2024-09-11T07:27:39.603094+0000","last_peered":"2024-09-11T07:27:39.603094+0000","last_clean":"2024-09-11T07:27:39.603094+0000","last_became_active":"2024-09-11T07:26:37.561758+0000","last_became_peered":"2024-09-11T07:26:37.561758+0000","last_unstale":"2024-09-11T07:27:39.603094+0000","last_undegraded":"2024-09-11T07:27:39.603094+0000","last_fullsized":"2024-09-11T07:27:39.603094+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:04.521146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:52.058044+0000","last_change":"2024-09-11T07:27:52.058013+0000","last_active":"2024-09-11T07:27:52.058044+0000","last_peered":"2024-09-11T07:27:52.058044+0000","last_clean":"2024-09-11T07:27:52.058044+0000","last_became_active":"2024-09-11T07:26:31.420969+0000","last_became_peered":"2024-09-11T07:26:31.420969+0000","last_unstale":"2024-09-11T07:27:52.058044+0000","last_undegraded":"2024-09-11T07:27:52.058044+0000","last_fullsized":"2024-09-11T07:27:52.058044+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_clean_scrub_stamp":"2024-09-11T07:27:52.057959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:12.902027+0000","scrub_duration":0.0014644840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"64'9","reported_seq":58,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:47.108747+0000","last_change":"2024-09-11T07:27:47.108704+0000","last_active":"2024-09-11T07:27:47.108747+0000","last_peered":"2024-09-11T07:27:47.108747+0000","last_clean":"2024-09-11T07:27:47.108747+0000","last_became_active":"2024-09-11T07:26:33.544386+0000","last_became_peered":"2024-09-11T07:26:33.544386+0000","last_unstale":"2024-09-11T07:27:47.108747+0000","last_undegraded":"2024-09-11T07:27:47.108747+0000","last_fullsized":"2024-09-11T07:27:47.108747+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:27:47.108645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.108645+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:08.302050+0000","scrub_duration":0.0024650560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.602477+0000","last_change":"2024-09-11T07:26:35.569943+0000","last_active":"2024-09-11T07:27:39.602477+0000","last_peered":"2024-09-11T07:27:39.602477+0000","last_clean":"2024-09-11T07:27:39.602477+0000","last_became_active":"2024-09-11T07:26:35.569630+0000","last_became_peered":"2024-09-11T07:26:35.569630+0000","last_unstale":"2024-09-11T07:27:39.602477+0000","last_undegraded":"2024-09-11T07:27:39.602477+0000","last_fullsized":"2024-09-11T07:27:39.602477+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:42.449548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428069+0000","last_change":"2024-09-11T07:26:37.565826+0000","last_active":"2024-09-11T07:27:38.428069+0000","last_peered":"2024-09-11T07:27:38.428069+0000","last_clean":"2024-09-11T07:27:38.428069+0000","last_became_active":"2024-09-11T07:26:37.565475+0000","last_became_peered":"2024-09-11T07:26:37.565475+0000","last_unstale":"2024-09-11T07:27:38.428069+0000","last_undegraded":"2024-09-11T07:27:38.428069+0000","last_fullsized":"2024-09-11T07:27:38.428069+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:50.954255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":60,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431907+0000","last_change":"2024-09-11T07:26:33.541535+0000","last_active":"2024-09-11T07:27:38.431907+0000","last_peered":"2024-09-11T07:27:38.431907+0000","last_clean":"2024-09-11T07:27:38.431907+0000","last_became_active":"2024-09-11T07:26:33.540575+0000","last_became_peered":"2024-09-11T07:26:33.540575+0000","last_unstale":"2024-09-11T07:27:38.431907+0000","last_undegraded":"2024-09-11T07:27:38.431907+0000","last_fullsized":"2024-09-11T07:27:38.431907+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:50.680900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.013458+0000","last_change":"2024-09-11T07:26:31.420109+0000","last_active":"2024-09-11T07:27:39.013458+0000","last_peered":"2024-09-11T07:27:39.013458+0000","last_clean":"2024-09-11T07:27:39.013458+0000","last_became_active":"2024-09-11T07:26:31.419488+0000","last_became_peered":"2024-09-11T07:26:31.419488+0000","last_unstale":"2024-09-11T07:27:39.013458+0000","last_undegraded":"2024-09-11T07:27:39.013458+0000","last_fullsized":"2024-09-11T07:27:39.013458+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:50.392677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428913+0000","last_change":"2024-09-11T07:26:35.559684+0000","last_active":"2024-09-11T07:27:38.428913+0000","last_peered":"2024-09-11T07:27:38.428913+0000","last_clean":"2024-09-11T07:27:38.428913+0000","last_became_active":"2024-09-11T07:26:35.557989+0000","last_became_peered":"2024-09-11T07:26:35.557989+0000","last_unstale":"2024-09-11T07:27:38.428913+0000","last_undegraded":"2024-09-11T07:27:38.428913+0000","last_fullsized":"2024-09-11T07:27:38.428913+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:54.992514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.013197+0000","last_change":"2024-09-11T07:26:37.565013+0000","last_active":"2024-09-11T07:27:39.013197+0000","last_peered":"2024-09-11T07:27:39.013197+0000","last_clean":"2024-09-11T07:27:39.013197+0000","last_became_active":"2024-09-11T07:26:37.564554+0000","last_became_peered":"2024-09-11T07:26:37.564554+0000","last_unstale":"2024-09-11T07:27:39.013197+0000","last_undegraded":"2024-09-11T07:27:39.013197+0000","last_fullsized":"2024-09-11T07:27:39.013197+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:46.952581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431592+0000","last_change":"2024-09-11T07:26:33.536333+0000","last_active":"2024-09-11T07:27:38.431592+0000","last_peered":"2024-09-11T07:27:38.431592+0000","last_clean":"2024-09-11T07:27:38.431592+0000","last_became_active":"2024-09-11T07:26:33.535411+0000","last_became_peered":"2024-09-11T07:26:33.535411+0000","last_unstale":"2024-09-11T07:27:38.431592+0000","last_undegraded":"2024-09-11T07:27:38.431592+0000","last_fullsized":"2024-09-11T07:27:38.431592+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:00.222835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428884+0000","last_change":"2024-09-11T07:26:31.419259+0000","last_active":"2024-09-11T07:27:38.428884+0000","last_peered":"2024-09-11T07:27:38.428884+0000","last_clean":"2024-09-11T07:27:38.428884+0000","last_became_active":"2024-09-11T07:26:31.418178+0000","last_became_peered":"2024-09-11T07:26:31.418178+0000","last_unstale":"2024-09-11T07:27:38.428884+0000","last_undegraded":"2024-09-11T07:27:38.428884+0000","last_fullsized":"2024-09-11T07:27:38.428884+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:53.539520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"64'8","reported_seq":64,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:47.127543+0000","last_change":"2024-09-11T07:27:47.127485+0000","last_active":"2024-09-11T07:27:47.127543+0000","last_peered":"2024-09-11T07:27:47.127543+0000","last_clean":"2024-09-11T07:27:47.127543+0000","last_became_active":"2024-09-11T07:26:35.564531+0000","last_became_peered":"2024-09-11T07:26:35.564531+0000","last_unstale":"2024-09-11T07:27:47.127543+0000","last_undegraded":"2024-09-11T07:27:47.127543+0000","last_fullsized":"2024-09-11T07:27:47.127543+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:27:47.127428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.127428+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:58.343731+0000","scrub_duration":0.0014091329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431540+0000","last_change":"2024-09-11T07:26:37.567193+0000","last_active":"2024-09-11T07:27:38.431540+0000","last_peered":"2024-09-11T07:27:38.431540+0000","last_clean":"2024-09-11T07:27:38.431540+0000","last_became_active":"2024-09-11T07:26:37.566919+0000","last_became_peered":"2024-09-11T07:26:37.566919+0000","last_unstale":"2024-09-11T07:27:38.431540+0000","last_undegraded":"2024-09-11T07:27:38.431540+0000","last_fullsized":"2024-09-11T07:27:38.431540+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:05.045246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"64'15","reported_seq":57,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.429231+0000","last_change":"2024-09-11T07:26:33.551090+0000","last_active":"2024-09-11T07:27:38.429231+0000","last_peered":"2024-09-11T07:27:38.429231+0000","last_clean":"2024-09-11T07:27:38.429231+0000","last_became_active":"2024-09-11T07:26:33.549733+0000","last_became_peered":"2024-09-11T07:26:33.549733+0000","last_unstale":"2024-09-11T07:27:38.429231+0000","last_undegraded":"2024-09-11T07:27:38.429231+0000","last_fullsized":"2024-09-11T07:27:38.429231+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:45.951848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428362+0000","last_change":"2024-09-11T07:27:35.589317+0000","last_active":"2024-09-11T07:27:38.428362+0000","last_peered":"2024-09-11T07:27:38.428362+0000","last_clean":"2024-09-11T07:27:38.428362+0000","last_became_active":"2024-09-11T07:26:31.418478+0000","last_became_peered":"2024-09-11T07:26:31.418478+0000","last_unstale":"2024-09-11T07:27:38.428362+0000","last_undegraded":"2024-09-11T07:27:38.428362+0000","last_fullsized":"2024-09-11T07:27:38.428362+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:35.589251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:35.589251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:01.839680+0000","scrub_duration":0.0014929380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:44.497246+0000","last_change":"2024-09-11T07:26:35.560767+0000","last_active":"2024-09-11T07:27:44.497246+0000","last_peered":"2024-09-11T07:27:44.497246+0000","last_clean":"2024-09-11T07:27:44.497246+0000","last_became_active":"2024-09-11T07:26:35.560406+0000","last_became_peered":"2024-09-11T07:26:35.560406+0000","last_unstale":"2024-09-11T07:27:44.497246+0000","last_undegraded":"2024-09-11T07:27:44.497246+0000","last_fullsized":"2024-09-11T07:27:44.497246+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.603454+0000","last_change":"2024-09-11T07:26:37.568289+0000","last_active":"2024-09-11T07:27:39.603454+0000","last_peered":"2024-09-11T07:27:39.603454+0000","last_clean":"2024-09-11T07:27:39.603454+0000","last_became_active":"2024-09-11T07:26:37.568005+0000","last_became_peered":"2024-09-11T07:26:37.568005+0000","last_unstale":"2024-09-11T07:27:39.603454+0000","last_undegraded":"2024-09-11T07:27:39.603454+0000","last_fullsized":"2024-09-11T07:27:39.603454+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:00.612703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"64'11","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431717+0000","last_change":"2024-09-11T07:26:33.541083+0000","last_active":"2024-09-11T07:27:38.431717+0000","last_peered":"2024-09-11T07:27:38.431717+0000","last_clean":"2024-09-11T07:27:38.431717+0000","last_became_active":"2024-09-11T07:26:33.540777+0000","last_became_peered":"2024-09-11T07:26:33.540777+0000","last_unstale":"2024-09-11T07:27:38.431717+0000","last_undegraded":"2024-09-11T07:27:38.431717+0000","last_fullsized":"2024-09-11T07:27:38.431717+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:47.691831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.012800+0000","last_change":"2024-09-11T07:27:34.932151+0000","last_active":"2024-09-11T07:27:39.012800+0000","last_peered":"2024-09-11T07:27:39.012800+0000","last_clean":"2024-09-11T07:27:39.012800+0000","last_became_active":"2024-09-11T07:26:31.418238+0000","last_became_peered":"2024-09-11T07:26:31.418238+0000","last_unstale":"2024-09-11T07:27:39.012800+0000","last_undegraded":"2024-09-11T07:27:39.012800+0000","last_fullsized":"2024-09-11T07:27:39.012800+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:34.932092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:34.932092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:46.904196+0000","scrub_duration":0.002710259,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.602649+0000","last_change":"2024-09-11T07:26:35.569433+0000","last_active":"2024-09-11T07:27:39.602649+0000","last_peered":"2024-09-11T07:27:39.602649+0000","last_clean":"2024-09-11T07:27:39.602649+0000","last_became_active":"2024-09-11T07:26:35.569081+0000","last_became_peered":"2024-09-11T07:26:35.569081+0000","last_unstale":"2024-09-11T07:27:39.602649+0000","last_undegraded":"2024-09-11T07:27:39.602649+0000","last_fullsized":"2024-09-11T07:27:39.602649+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:44.797975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:46.752315+0000","last_change":"2024-09-11T07:27:46.752272+0000","last_active":"2024-09-11T07:27:46.752315+0000","last_peered":"2024-09-11T07:27:46.752315+0000","last_clean":"2024-09-11T07:27:46.752315+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:27:46.752315+0000","last_undegraded":"2024-09-11T07:27:46.752315+0000","last_fullsized":"2024-09-11T07:27:46.752315+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:46.752181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:46.752181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.072860+0000","scrub_duration":0.001508782,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"64'14","reported_seq":53,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431834+0000","last_change":"2024-09-11T07:26:33.538152+0000","last_active":"2024-09-11T07:27:38.431834+0000","last_peered":"2024-09-11T07:27:38.431834+0000","last_clean":"2024-09-11T07:27:38.431834+0000","last_became_active":"2024-09-11T07:26:33.537850+0000","last_became_peered":"2024-09-11T07:26:33.537850+0000","last_unstale":"2024-09-11T07:27:38.431834+0000","last_undegraded":"2024-09-11T07:27:38.431834+0000","last_fullsized":"2024-09-11T07:27:38.431834+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:57.850522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.014986+0000","last_change":"2024-09-11T07:27:39.014916+0000","last_active":"2024-09-11T07:27:39.014986+0000","last_peered":"2024-09-11T07:27:39.014986+0000","last_clean":"2024-09-11T07:27:39.014986+0000","last_became_active":"2024-09-11T07:26:31.423905+0000","last_became_peered":"2024-09-11T07:26:31.423905+0000","last_unstale":"2024-09-11T07:27:39.014986+0000","last_undegraded":"2024-09-11T07:27:39.014986+0000","last_fullsized":"2024-09-11T07:27:39.014986+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:39.014840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:39.014840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:40.097304+0000","scrub_duration":0.0032789239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"64'8","reported_seq":54,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:44.497264+0000","last_change":"2024-09-11T07:26:35.563242+0000","last_active":"2024-09-11T07:27:44.497264+0000","last_peered":"2024-09-11T07:27:44.497264+0000","last_clean":"2024-09-11T07:27:44.497264+0000","last_became_active":"2024-09-11T07:26:35.562997+0000","last_became_peered":"2024-09-11T07:26:35.562997+0000","last_unstale":"2024-09-11T07:27:44.497264+0000","last_undegraded":"2024-09-11T07:27:44.497264+0000","last_fullsized":"2024-09-11T07:27:44.497264+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:50.078049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.013053+0000","last_change":"2024-09-11T07:26:37.862564+0000","last_active":"2024-09-11T07:27:39.013053+0000","last_peered":"2024-09-11T07:27:39.013053+0000","last_clean":"2024-09-11T07:27:39.013053+0000","last_became_active":"2024-09-11T07:26:37.862246+0000","last_became_peered":"2024-09-11T07:26:37.862246+0000","last_unstale":"2024-09-11T07:27:39.013053+0000","last_undegraded":"2024-09-11T07:27:39.013053+0000","last_fullsized":"2024-09-11T07:27:39.013053+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:04.346579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"64'18","reported_seq":64,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.731805+0000","last_change":"2024-09-11T07:26:33.546711+0000","last_active":"2024-09-11T07:27:38.731805+0000","last_peered":"2024-09-11T07:27:38.731805+0000","last_clean":"2024-09-11T07:27:38.731805+0000","last_became_active":"2024-09-11T07:26:33.546188+0000","last_became_peered":"2024-09-11T07:26:33.546188+0000","last_unstale":"2024-09-11T07:27:38.731805+0000","last_undegraded":"2024-09-11T07:27:38.731805+0000","last_fullsized":"2024-09-11T07:27:38.731805+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:02.137900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.429518+0000","last_change":"2024-09-11T07:26:31.421802+0000","last_active":"2024-09-11T07:27:38.429518+0000","last_peered":"2024-09-11T07:27:38.429518+0000","last_clean":"2024-09-11T07:27:38.429518+0000","last_became_active":"2024-09-11T07:26:31.421500+0000","last_became_peered":"2024-09-11T07:26:31.421500+0000","last_unstale":"2024-09-11T07:27:38.429518+0000","last_undegraded":"2024-09-11T07:27:38.429518+0000","last_fullsized":"2024-09-11T07:27:38.429518+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:56.390924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431190+0000","last_change":"2024-09-11T07:26:35.562816+0000","last_active":"2024-09-11T07:27:38.431190+0000","last_peered":"2024-09-11T07:27:38.431190+0000","last_clean":"2024-09-11T07:27:38.431190+0000","last_became_active":"2024-09-11T07:26:35.561284+0000","last_became_peered":"2024-09-11T07:26:35.561284+0000","last_unstale":"2024-09-11T07:27:38.431190+0000","last_undegraded":"2024-09-11T07:27:38.431190+0000","last_fullsized":"2024-09-11T07:27:38.431190+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:01.396434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431748+0000","last_change":"2024-09-11T07:26:37.565014+0000","last_active":"2024-09-11T07:27:38.431748+0000","last_peered":"2024-09-11T07:27:38.431748+0000","last_clean":"2024-09-11T07:27:38.431748+0000","last_became_active":"2024-09-11T07:26:37.564616+0000","last_became_peered":"2024-09-11T07:26:37.564616+0000","last_unstale":"2024-09-11T07:27:38.431748+0000","last_undegraded":"2024-09-11T07:27:38.431748+0000","last_fullsized":"2024-09-11T07:27:38.431748+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:04.390896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":68,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.429080+0000","last_change":"2024-09-11T07:26:33.541012+0000","last_active":"2024-09-11T07:27:38.429080+0000","last_peered":"2024-09-11T07:27:38.429080+0000","last_clean":"2024-09-11T07:27:38.429080+0000","last_became_active":"2024-09-11T07:26:33.540639+0000","last_became_peered":"2024-09-11T07:26:33.540639+0000","last_unstale":"2024-09-11T07:27:38.429080+0000","last_undegraded":"2024-09-11T07:27:38.429080+0000","last_fullsized":"2024-09-11T07:27:38.429080+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:51.550418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.013736+0000","last_change":"2024-09-11T07:26:31.419925+0000","last_active":"2024-09-11T07:27:39.013736+0000","last_peered":"2024-09-11T07:27:39.013736+0000","last_clean":"2024-09-11T07:27:39.013736+0000","last_became_active":"2024-09-11T07:26:31.419185+0000","last_became_peered":"2024-09-11T07:26:31.419185+0000","last_unstale":"2024-09-11T07:27:39.013736+0000","last_undegraded":"2024-09-11T07:27:39.013736+0000","last_fullsized":"2024-09-11T07:27:39.013736+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:52.580179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:41.000506+0000","last_change":"2024-09-11T07:27:41.000469+0000","last_active":"2024-09-11T07:27:41.000506+0000","last_peered":"2024-09-11T07:27:41.000506+0000","last_clean":"2024-09-11T07:27:41.000506+0000","last_became_active":"2024-09-11T07:26:35.565359+0000","last_became_peered":"2024-09-11T07:26:35.565359+0000","last_unstale":"2024-09-11T07:27:41.000506+0000","last_undegraded":"2024-09-11T07:27:41.000506+0000","last_fullsized":"2024-09-11T07:27:41.000506+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:41.000401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:41.000401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.049825+0000","scrub_duration":0.001655164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428073+0000","last_change":"2024-09-11T07:26:37.861736+0000","last_active":"2024-09-11T07:27:38.428073+0000","last_peered":"2024-09-11T07:27:38.428073+0000","last_clean":"2024-09-11T07:27:38.428073+0000","last_became_active":"2024-09-11T07:26:37.861279+0000","last_became_peered":"2024-09-11T07:26:37.861279+0000","last_unstale":"2024-09-11T07:27:38.428073+0000","last_undegraded":"2024-09-11T07:27:38.428073+0000","last_fullsized":"2024-09-11T07:27:38.428073+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:58.793313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:51.642564+0000","last_change":"2024-09-11T07:27:51.642529+0000","last_active":"2024-09-11T07:27:51.642564+0000","last_peered":"2024-09-11T07:27:51.642564+0000","last_clean":"2024-09-11T07:27:51.642564+0000","last_became_active":"2024-09-11T07:26:33.543063+0000","last_became_peered":"2024-09-11T07:26:33.543063+0000","last_unstale":"2024-09-11T07:27:51.642564+0000","last_undegraded":"2024-09-11T07:27:51.642564+0000","last_fullsized":"2024-09-11T07:27:51.642564+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:27:51.642464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:51.642464+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:13.146284+0000","scrub_duration":0.0020211439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":51,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:41.618402+0000","last_change":"2024-09-11T07:27:41.618362+0000","last_active":"2024-09-11T07:27:41.618402+0000","last_peered":"2024-09-11T07:27:41.618402+0000","last_clean":"2024-09-11T07:27:41.618402+0000","last_became_active":"2024-09-11T07:26:31.422110+0000","last_became_peered":"2024-09-11T07:26:31.422110+0000","last_unstale":"2024-09-11T07:27:41.618402+0000","last_undegraded":"2024-09-11T07:27:41.618402+0000","last_fullsized":"2024-09-11T07:27:41.618402+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:27:41.618296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:41.618296+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:57.404591+0000","scrub_duration":0.0020650450000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":54,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:44.497569+0000","last_change":"2024-09-11T07:26:35.560735+0000","last_active":"2024-09-11T07:27:44.497569+0000","last_peered":"2024-09-11T07:27:44.497569+0000","last_clean":"2024-09-11T07:27:44.497569+0000","last_became_active":"2024-09-11T07:26:35.560422+0000","last_became_peered":"2024-09-11T07:26:35.560422+0000","last_unstale":"2024-09-11T07:27:44.497569+0000","last_undegraded":"2024-09-11T07:27:44.497569+0000","last_fullsized":"2024-09-11T07:27:44.497569+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.427894+0000","last_change":"2024-09-11T07:26:37.568087+0000","last_active":"2024-09-11T07:27:38.427894+0000","last_peered":"2024-09-11T07:27:38.427894+0000","last_clean":"2024-09-11T07:27:38.427894+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:27:38.427894+0000","last_undegraded":"2024-09-11T07:27:38.427894+0000","last_fullsized":"2024-09-11T07:27:38.427894+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"64'28","reported_seq":96,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:44.497299+0000","last_change":"2024-09-11T07:26:33.549518+0000","last_active":"2024-09-11T07:27:44.497299+0000","last_peered":"2024-09-11T07:27:44.497299+0000","last_clean":"2024-09-11T07:27:44.497299+0000","last_became_active":"2024-09-11T07:26:33.549234+0000","last_became_peered":"2024-09-11T07:26:33.549234+0000","last_unstale":"2024-09-11T07:27:44.497299+0000","last_undegraded":"2024-09-11T07:27:44.497299+0000","last_fullsized":"2024-09-11T07:27:44.497299+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:50.484684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.603309+0000","last_change":"2024-09-11T07:26:31.424898+0000","last_active":"2024-09-11T07:27:39.603309+0000","last_peered":"2024-09-11T07:27:39.603309+0000","last_clean":"2024-09-11T07:27:39.603309+0000","last_became_active":"2024-09-11T07:26:31.424618+0000","last_became_peered":"2024-09-11T07:26:31.424618+0000","last_unstale":"2024-09-11T07:27:39.603309+0000","last_undegraded":"2024-09-11T07:27:39.603309+0000","last_fullsized":"2024-09-11T07:27:39.603309+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:55.128959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.429016+0000","last_change":"2024-09-11T07:26:35.560677+0000","last_active":"2024-09-11T07:27:38.429016+0000","last_peered":"2024-09-11T07:27:38.429016+0000","last_clean":"2024-09-11T07:27:38.429016+0000","last_became_active":"2024-09-11T07:26:35.560363+0000","last_became_peered":"2024-09-11T07:26:35.560363+0000","last_unstale":"2024-09-11T07:27:38.429016+0000","last_undegraded":"2024-09-11T07:27:38.429016+0000","last_fullsized":"2024-09-11T07:27:38.429016+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:53.196351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.732485+0000","last_change":"2024-09-11T07:26:37.563366+0000","last_active":"2024-09-11T07:27:38.732485+0000","last_peered":"2024-09-11T07:27:38.732485+0000","last_clean":"2024-09-11T07:27:38.732485+0000","last_became_active":"2024-09-11T07:26:37.562452+0000","last_became_peered":"2024-09-11T07:26:37.562452+0000","last_unstale":"2024-09-11T07:27:38.732485+0000","last_undegraded":"2024-09-11T07:27:38.732485+0000","last_fullsized":"2024-09-11T07:27:38.732485+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:44.615521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"64'13","reported_seq":59,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.429346+0000","last_change":"2024-09-11T07:26:33.547483+0000","last_active":"2024-09-11T07:27:38.429346+0000","last_peered":"2024-09-11T07:27:38.429346+0000","last_clean":"2024-09-11T07:27:38.429346+0000","last_became_active":"2024-09-11T07:26:33.546126+0000","last_became_peered":"2024-09-11T07:26:33.546126+0000","last_unstale":"2024-09-11T07:27:38.429346+0000","last_undegraded":"2024-09-11T07:27:38.429346+0000","last_fullsized":"2024-09-11T07:27:38.429346+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:56.391970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":118,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:52.716275+0000","last_change":"2024-09-11T07:26:35.548541+0000","last_active":"2024-09-11T07:27:52.716275+0000","last_peered":"2024-09-11T07:27:52.716275+0000","last_clean":"2024-09-11T07:27:52.716275+0000","last_became_active":"2024-09-11T07:26:31.421276+0000","last_became_peered":"2024-09-11T07:26:31.421276+0000","last_unstale":"2024-09-11T07:27:52.716275+0000","last_undegraded":"2024-09-11T07:27:52.716275+0000","last_fullsized":"2024-09-11T07:27:52.716275+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.012923+0000","last_change":"2024-09-11T07:26:31.413634+0000","last_active":"2024-09-11T07:27:39.012923+0000","last_peered":"2024-09-11T07:27:39.012923+0000","last_clean":"2024-09-11T07:27:39.012923+0000","last_became_active":"2024-09-11T07:26:31.413354+0000","last_became_peered":"2024-09-11T07:26:31.413354+0000","last_unstale":"2024-09-11T07:27:39.012923+0000","last_undegraded":"2024-09-11T07:27:39.012923+0000","last_fullsized":"2024-09-11T07:27:39.012923+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:55.160768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:42.141181+0000","last_change":"2024-09-11T07:27:42.141156+0000","last_active":"2024-09-11T07:27:42.141181+0000","last_peered":"2024-09-11T07:27:42.141181+0000","last_clean":"2024-09-11T07:27:42.141181+0000","last_became_active":"2024-09-11T07:26:35.565849+0000","last_became_peered":"2024-09-11T07:26:35.565849+0000","last_unstale":"2024-09-11T07:27:42.141181+0000","last_undegraded":"2024-09-11T07:27:42.141181+0000","last_fullsized":"2024-09-11T07:27:42.141181+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.141100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.141100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.707826+0000","scrub_duration":0.001495672,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.013249+0000","last_change":"2024-09-11T07:26:37.861648+0000","last_active":"2024-09-11T07:27:39.013249+0000","last_peered":"2024-09-11T07:27:39.013249+0000","last_clean":"2024-09-11T07:27:39.013249+0000","last_became_active":"2024-09-11T07:26:37.861292+0000","last_became_peered":"2024-09-11T07:26:37.861292+0000","last_unstale":"2024-09-11T07:27:39.013249+0000","last_undegraded":"2024-09-11T07:27:39.013249+0000","last_fullsized":"2024-09-11T07:27:39.013249+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:50.030830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":60,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428689+0000","last_change":"2024-09-11T07:27:34.213946+0000","last_active":"2024-09-11T07:27:38.428689+0000","last_peered":"2024-09-11T07:27:38.428689+0000","last_clean":"2024-09-11T07:27:38.428689+0000","last_became_active":"2024-09-11T07:26:33.548903+0000","last_became_peered":"2024-09-11T07:26:33.548903+0000","last_unstale":"2024-09-11T07:27:38.428689+0000","last_undegraded":"2024-09-11T07:27:38.428689+0000","last_fullsized":"2024-09-11T07:27:38.428689+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-09-11T07:27:34.213870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:34.213870+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:50.149624+0000","scrub_duration":0.0026667679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428424+0000","last_change":"2024-09-11T07:26:35.551577+0000","last_active":"2024-09-11T07:27:38.428424+0000","last_peered":"2024-09-11T07:27:38.428424+0000","last_clean":"2024-09-11T07:27:38.428424+0000","last_became_active":"2024-09-11T07:26:31.420320+0000","last_became_peered":"2024-09-11T07:26:31.420320+0000","last_unstale":"2024-09-11T07:27:38.428424+0000","last_undegraded":"2024-09-11T07:27:38.428424+0000","last_fullsized":"2024-09-11T07:27:38.428424+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:49.887263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.012844+0000","last_change":"2024-09-11T07:26:31.418190+0000","last_active":"2024-09-11T07:27:39.012844+0000","last_peered":"2024-09-11T07:27:39.012844+0000","last_clean":"2024-09-11T07:27:39.012844+0000","last_became_active":"2024-09-11T07:26:31.417455+0000","last_became_peered":"2024-09-11T07:26:31.417455+0000","last_unstale":"2024-09-11T07:27:39.012844+0000","last_undegraded":"2024-09-11T07:27:39.012844+0000","last_fullsized":"2024-09-11T07:27:39.012844+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:54.951321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.602486+0000","last_change":"2024-09-11T07:26:35.564511+0000","last_active":"2024-09-11T07:27:39.602486+0000","last_peered":"2024-09-11T07:27:39.602486+0000","last_clean":"2024-09-11T07:27:39.602486+0000","last_became_active":"2024-09-11T07:26:35.564178+0000","last_became_peered":"2024-09-11T07:26:35.564178+0000","last_unstale":"2024-09-11T07:27:39.602486+0000","last_undegraded":"2024-09-11T07:27:39.602486+0000","last_fullsized":"2024-09-11T07:27:39.602486+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:01.503548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428838+0000","last_change":"2024-09-11T07:26:37.561674+0000","last_active":"2024-09-11T07:27:38.428838+0000","last_peered":"2024-09-11T07:27:38.428838+0000","last_clean":"2024-09-11T07:27:38.428838+0000","last_became_active":"2024-09-11T07:26:37.561378+0000","last_became_peered":"2024-09-11T07:26:37.561378+0000","last_unstale":"2024-09-11T07:27:38.428838+0000","last_undegraded":"2024-09-11T07:27:38.428838+0000","last_fullsized":"2024-09-11T07:27:38.428838+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:44.852330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":64,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:49.061760+0000","last_change":"2024-09-11T07:27:49.061728+0000","last_active":"2024-09-11T07:27:49.061760+0000","last_peered":"2024-09-11T07:27:49.061760+0000","last_clean":"2024-09-11T07:27:49.061760+0000","last_became_active":"2024-09-11T07:26:33.541359+0000","last_became_peered":"2024-09-11T07:26:33.541359+0000","last_unstale":"2024-09-11T07:27:49.061760+0000","last_undegraded":"2024-09-11T07:27:49.061760+0000","last_fullsized":"2024-09-11T07:27:49.061760+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-09-11T07:27:49.061670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.061670+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:51.283390+0000","scrub_duration":0.001809801,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"61'2","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.732096+0000","last_change":"2024-09-11T07:26:35.556632+0000","last_active":"2024-09-11T07:27:38.732096+0000","last_peered":"2024-09-11T07:27:38.732096+0000","last_clean":"2024-09-11T07:27:38.732096+0000","last_became_active":"2024-09-11T07:26:31.419501+0000","last_became_peered":"2024-09-11T07:26:31.419501+0000","last_unstale":"2024-09-11T07:27:38.732096+0000","last_undegraded":"2024-09-11T07:27:38.732096+0000","last_fullsized":"2024-09-11T07:27:38.732096+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:45.065345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:47.903234+0000","last_change":"2024-09-11T07:27:47.903204+0000","last_active":"2024-09-11T07:27:47.903234+0000","last_peered":"2024-09-11T07:27:47.903234+0000","last_clean":"2024-09-11T07:27:47.903234+0000","last_became_active":"2024-09-11T07:26:31.423304+0000","last_became_peered":"2024-09-11T07:26:31.423304+0000","last_unstale":"2024-09-11T07:27:47.903234+0000","last_undegraded":"2024-09-11T07:27:47.903234+0000","last_fullsized":"2024-09-11T07:27:47.903234+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:47.903135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.903135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:01.867608+0000","scrub_duration":0.001284908,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"72'79","reported_seq":79,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.013039+0000","last_change":"2024-09-11T07:26:57.872633+0000","last_active":"2024-09-11T07:27:39.013039+0000","last_peered":"2024-09-11T07:27:39.013039+0000","last_clean":"2024-09-11T07:27:39.013039+0000","last_became_active":"2024-09-11T07:26:17.397381+0000","last_became_peered":"2024-09-11T07:26:17.397381+0000","last_unstale":"2024-09-11T07:27:39.013039+0000","last_undegraded":"2024-09-11T07:27:39.013039+0000","last_fullsized":"2024-09-11T07:27:39.013039+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'72","last_scrub_stamp":"2024-09-11T07:26:57.872570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:24:22.202219+0000","last_clean_scrub_stamp":"2024-09-11T07:26:57.872570+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:06.077652+0000","scrub_duration":0.0015639180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:44.036179+0000","last_change":"2024-09-11T07:27:44.036138+0000","last_active":"2024-09-11T07:27:44.036179+0000","last_peered":"2024-09-11T07:27:44.036179+0000","last_clean":"2024-09-11T07:27:44.036179+0000","last_became_active":"2024-09-11T07:26:35.564480+0000","last_became_peered":"2024-09-11T07:26:35.564480+0000","last_unstale":"2024-09-11T07:27:44.036179+0000","last_undegraded":"2024-09-11T07:27:44.036179+0000","last_fullsized":"2024-09-11T07:27:44.036179+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:44.036058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:44.036058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:48.598709+0000","scrub_duration":0.001488931,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.603710+0000","last_change":"2024-09-11T07:26:37.560202+0000","last_active":"2024-09-11T07:27:39.603710+0000","last_peered":"2024-09-11T07:27:39.603710+0000","last_clean":"2024-09-11T07:27:39.603710+0000","last_became_active":"2024-09-11T07:26:37.559920+0000","last_became_peered":"2024-09-11T07:26:37.559920+0000","last_unstale":"2024-09-11T07:27:39.603710+0000","last_undegraded":"2024-09-11T07:27:39.603710+0000","last_fullsized":"2024-09-11T07:27:39.603710+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:55.457939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":63,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.672378+0000","last_change":"2024-09-11T07:26:33.544014+0000","last_active":"2024-09-11T07:27:38.672378+0000","last_peered":"2024-09-11T07:27:38.672378+0000","last_clean":"2024-09-11T07:27:38.672378+0000","last_became_active":"2024-09-11T07:26:33.543687+0000","last_became_peered":"2024-09-11T07:26:33.543687+0000","last_unstale":"2024-09-11T07:27:38.672378+0000","last_undegraded":"2024-09-11T07:27:38.672378+0000","last_fullsized":"2024-09-11T07:27:38.672378+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:55.743551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.013246+0000","last_change":"2024-09-11T07:26:31.419837+0000","last_active":"2024-09-11T07:27:39.013246+0000","last_peered":"2024-09-11T07:27:39.013246+0000","last_clean":"2024-09-11T07:27:39.013246+0000","last_became_active":"2024-09-11T07:26:31.419560+0000","last_became_peered":"2024-09-11T07:26:31.419560+0000","last_unstale":"2024-09-11T07:27:39.013246+0000","last_undegraded":"2024-09-11T07:27:39.013246+0000","last_fullsized":"2024-09-11T07:27:39.013246+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:46.444292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.012835+0000","last_change":"2024-09-11T07:26:35.564320+0000","last_active":"2024-09-11T07:27:39.012835+0000","last_peered":"2024-09-11T07:27:39.012835+0000","last_clean":"2024-09-11T07:27:39.012835+0000","last_became_active":"2024-09-11T07:26:35.564044+0000","last_became_peered":"2024-09-11T07:26:35.564044+0000","last_unstale":"2024-09-11T07:27:39.012835+0000","last_undegraded":"2024-09-11T07:27:39.012835+0000","last_fullsized":"2024-09-11T07:27:39.012835+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:01.424414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.013152+0000","last_change":"2024-09-11T07:26:37.565084+0000","last_active":"2024-09-11T07:27:39.013152+0000","last_peered":"2024-09-11T07:27:39.013152+0000","last_clean":"2024-09-11T07:27:39.013152+0000","last_became_active":"2024-09-11T07:26:37.564801+0000","last_became_peered":"2024-09-11T07:26:37.564801+0000","last_unstale":"2024-09-11T07:27:39.013152+0000","last_undegraded":"2024-09-11T07:27:39.013152+0000","last_fullsized":"2024-09-11T07:27:39.013152+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:53.493323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":48,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428999+0000","last_change":"2024-09-11T07:26:33.543461+0000","last_active":"2024-09-11T07:27:38.428999+0000","last_peered":"2024-09-11T07:27:38.428999+0000","last_clean":"2024-09-11T07:27:38.428999+0000","last_became_active":"2024-09-11T07:26:33.543148+0000","last_became_peered":"2024-09-11T07:26:33.543148+0000","last_unstale":"2024-09-11T07:27:38.428999+0000","last_undegraded":"2024-09-11T07:27:38.428999+0000","last_fullsized":"2024-09-11T07:27:38.428999+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:40.632848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:53.660005+0000","last_change":"2024-09-11T07:27:53.659961+0000","last_active":"2024-09-11T07:27:53.660005+0000","last_peered":"2024-09-11T07:27:53.660005+0000","last_clean":"2024-09-11T07:27:53.660005+0000","last_became_active":"2024-09-11T07:26:31.423056+0000","last_became_peered":"2024-09-11T07:26:31.423056+0000","last_unstale":"2024-09-11T07:27:53.660005+0000","last_undegraded":"2024-09-11T07:27:53.660005+0000","last_fullsized":"2024-09-11T07:27:53.660005+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:53.659902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:53.659902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.781453+0000","scrub_duration":0.0011613719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.012953+0000","last_change":"2024-09-11T07:26:35.563452+0000","last_active":"2024-09-11T07:27:39.012953+0000","last_peered":"2024-09-11T07:27:39.012953+0000","last_clean":"2024-09-11T07:27:39.012953+0000","last_became_active":"2024-09-11T07:26:35.563182+0000","last_became_peered":"2024-09-11T07:26:35.563182+0000","last_unstale":"2024-09-11T07:27:39.012953+0000","last_undegraded":"2024-09-11T07:27:39.012953+0000","last_fullsized":"2024-09-11T07:27:39.012953+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:00.634009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428385+0000","last_change":"2024-09-11T07:26:37.565396+0000","last_active":"2024-09-11T07:27:38.428385+0000","last_peered":"2024-09-11T07:27:38.428385+0000","last_clean":"2024-09-11T07:27:38.428385+0000","last_became_active":"2024-09-11T07:26:37.565029+0000","last_became_peered":"2024-09-11T07:26:37.565029+0000","last_unstale":"2024-09-11T07:27:38.428385+0000","last_undegraded":"2024-09-11T07:27:38.428385+0000","last_fullsized":"2024-09-11T07:27:38.428385+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:03.261465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":57,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.602970+0000","last_change":"2024-09-11T07:26:33.538495+0000","last_active":"2024-09-11T07:27:39.602970+0000","last_peered":"2024-09-11T07:27:39.602970+0000","last_clean":"2024-09-11T07:27:39.602970+0000","last_became_active":"2024-09-11T07:26:33.538216+0000","last_became_peered":"2024-09-11T07:26:33.538216+0000","last_unstale":"2024-09-11T07:27:39.602970+0000","last_undegraded":"2024-09-11T07:27:39.602970+0000","last_fullsized":"2024-09-11T07:27:39.602970+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:52.847742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:50.132667+0000","last_change":"2024-09-11T07:27:50.132629+0000","last_active":"2024-09-11T07:27:50.132667+0000","last_peered":"2024-09-11T07:27:50.132667+0000","last_clean":"2024-09-11T07:27:50.132667+0000","last_became_active":"2024-09-11T07:26:31.418490+0000","last_became_peered":"2024-09-11T07:26:31.418490+0000","last_unstale":"2024-09-11T07:27:50.132667+0000","last_undegraded":"2024-09-11T07:27:50.132667+0000","last_fullsized":"2024-09-11T07:27:50.132667+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.132552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.048681+0000","scrub_duration":0.001622807,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":54,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:44.497171+0000","last_change":"2024-09-11T07:26:35.567147+0000","last_active":"2024-09-11T07:27:44.497171+0000","last_peered":"2024-09-11T07:27:44.497171+0000","last_clean":"2024-09-11T07:27:44.497171+0000","last_became_active":"2024-09-11T07:26:35.566888+0000","last_became_peered":"2024-09-11T07:26:35.566888+0000","last_unstale":"2024-09-11T07:27:44.497171+0000","last_undegraded":"2024-09-11T07:27:44.497171+0000","last_fullsized":"2024-09-11T07:27:44.497171+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:03.574568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.603130+0000","last_change":"2024-09-11T07:26:37.861501+0000","last_active":"2024-09-11T07:27:39.603130+0000","last_peered":"2024-09-11T07:27:39.603130+0000","last_clean":"2024-09-11T07:27:39.603130+0000","last_became_active":"2024-09-11T07:26:37.861134+0000","last_became_peered":"2024-09-11T07:26:37.861134+0000","last_unstale":"2024-09-11T07:27:39.603130+0000","last_undegraded":"2024-09-11T07:27:39.603130+0000","last_fullsized":"2024-09-11T07:27:39.603130+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:50.897367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"64'12","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431292+0000","last_change":"2024-09-11T07:26:33.540837+0000","last_active":"2024-09-11T07:27:38.431292+0000","last_peered":"2024-09-11T07:27:38.431292+0000","last_clean":"2024-09-11T07:27:38.431292+0000","last_became_active":"2024-09-11T07:26:33.540363+0000","last_became_peered":"2024-09-11T07:26:33.540363+0000","last_unstale":"2024-09-11T07:27:38.431292+0000","last_undegraded":"2024-09-11T07:27:38.431292+0000","last_fullsized":"2024-09-11T07:27:38.431292+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:49.841213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":61,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:46.077895+0000","last_change":"2024-09-11T07:27:46.077857+0000","last_active":"2024-09-11T07:27:46.077895+0000","last_peered":"2024-09-11T07:27:46.077895+0000","last_clean":"2024-09-11T07:27:46.077895+0000","last_became_active":"2024-09-11T07:26:31.419718+0000","last_became_peered":"2024-09-11T07:26:31.419718+0000","last_unstale":"2024-09-11T07:27:46.077895+0000","last_undegraded":"2024-09-11T07:27:46.077895+0000","last_fullsized":"2024-09-11T07:27:46.077895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:27:46.077786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:46.077786+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:07.735381+0000","scrub_duration":0.0019376720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:53.209546+0000","last_change":"2024-09-11T07:27:53.209518+0000","last_active":"2024-09-11T07:27:53.209546+0000","last_peered":"2024-09-11T07:27:53.209546+0000","last_clean":"2024-09-11T07:27:53.209546+0000","last_became_active":"2024-09-11T07:26:35.561967+0000","last_became_peered":"2024-09-11T07:26:35.561967+0000","last_unstale":"2024-09-11T07:27:53.209546+0000","last_undegraded":"2024-09-11T07:27:53.209546+0000","last_fullsized":"2024-09-11T07:27:53.209546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:53.209452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:53.209452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:23.125281+0000","scrub_duration":0.0014563029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.731999+0000","last_change":"2024-09-11T07:26:37.568046+0000","last_active":"2024-09-11T07:27:38.731999+0000","last_peered":"2024-09-11T07:27:38.731999+0000","last_clean":"2024-09-11T07:27:38.731999+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:27:38.731999+0000","last_undegraded":"2024-09-11T07:27:38.731999+0000","last_fullsized":"2024-09-11T07:27:38.731999+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:48.487836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":58,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.351623+0000","last_change":"2024-09-11T07:27:40.351572+0000","last_active":"2024-09-11T07:27:40.351623+0000","last_peered":"2024-09-11T07:27:40.351623+0000","last_clean":"2024-09-11T07:27:40.351623+0000","last_became_active":"2024-09-11T07:26:33.547344+0000","last_became_peered":"2024-09-11T07:26:33.547344+0000","last_unstale":"2024-09-11T07:27:40.351623+0000","last_undegraded":"2024-09-11T07:27:40.351623+0000","last_fullsized":"2024-09-11T07:27:40.351623+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:27:40.351507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:40.351507+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.098394+0000","scrub_duration":0.0021803980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.012802+0000","last_change":"2024-09-11T07:26:31.416001+0000","last_active":"2024-09-11T07:27:39.012802+0000","last_peered":"2024-09-11T07:27:39.012802+0000","last_clean":"2024-09-11T07:27:39.012802+0000","last_became_active":"2024-09-11T07:26:31.415701+0000","last_became_peered":"2024-09-11T07:26:31.415701+0000","last_unstale":"2024-09-11T07:27:39.012802+0000","last_undegraded":"2024-09-11T07:27:39.012802+0000","last_fullsized":"2024-09-11T07:27:39.012802+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:48.358757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.732551+0000","last_change":"2024-09-11T07:26:35.559563+0000","last_active":"2024-09-11T07:27:38.732551+0000","last_peered":"2024-09-11T07:27:38.732551+0000","last_clean":"2024-09-11T07:27:38.732551+0000","last_became_active":"2024-09-11T07:26:35.559262+0000","last_became_peered":"2024-09-11T07:26:35.559262+0000","last_unstale":"2024-09-11T07:27:38.732551+0000","last_undegraded":"2024-09-11T07:27:38.732551+0000","last_fullsized":"2024-09-11T07:27:38.732551+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:52.116965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.012842+0000","last_change":"2024-09-11T07:26:37.564431+0000","last_active":"2024-09-11T07:27:39.012842+0000","last_peered":"2024-09-11T07:27:39.012842+0000","last_clean":"2024-09-11T07:27:39.012842+0000","last_became_active":"2024-09-11T07:26:37.564084+0000","last_became_peered":"2024-09-11T07:26:37.564084+0000","last_unstale":"2024-09-11T07:27:39.012842+0000","last_undegraded":"2024-09-11T07:27:39.012842+0000","last_fullsized":"2024-09-11T07:27:39.012842+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:52.135522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:45.990660+0000","last_change":"2024-09-11T07:27:45.990617+0000","last_active":"2024-09-11T07:27:45.990660+0000","last_peered":"2024-09-11T07:27:45.990660+0000","last_clean":"2024-09-11T07:27:45.990660+0000","last_became_active":"2024-09-11T07:26:31.423229+0000","last_became_peered":"2024-09-11T07:26:31.423229+0000","last_unstale":"2024-09-11T07:27:45.990660+0000","last_undegraded":"2024-09-11T07:27:45.990660+0000","last_fullsized":"2024-09-11T07:27:45.990660+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:45.990548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:45.990548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:51.641867+0000","scrub_duration":0.0014585150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"82'9","reported_seq":54,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:48.761507+0000","last_change":"2024-09-11T07:27:48.761470+0000","last_active":"2024-09-11T07:27:48.761507+0000","last_peered":"2024-09-11T07:27:48.761507+0000","last_clean":"2024-09-11T07:27:48.761507+0000","last_became_active":"2024-09-11T07:26:33.543628+0000","last_became_peered":"2024-09-11T07:26:33.543628+0000","last_unstale":"2024-09-11T07:27:48.761507+0000","last_undegraded":"2024-09-11T07:27:48.761507+0000","last_fullsized":"2024-09-11T07:27:48.761507+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'9","last_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:27:48.761397+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.913254+0000","scrub_duration":0.0037047439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:42.917767+0000","last_change":"2024-09-11T07:27:42.917739+0000","last_active":"2024-09-11T07:27:42.917767+0000","last_peered":"2024-09-11T07:27:42.917767+0000","last_clean":"2024-09-11T07:27:42.917767+0000","last_became_active":"2024-09-11T07:26:35.566894+0000","last_became_peered":"2024-09-11T07:26:35.566894+0000","last_unstale":"2024-09-11T07:27:42.917767+0000","last_undegraded":"2024-09-11T07:27:42.917767+0000","last_fullsized":"2024-09-11T07:27:42.917767+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.917680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:06.763663+0000","scrub_duration":0.0013496649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:47.031926+0000","last_change":"2024-09-11T07:27:47.031883+0000","last_active":"2024-09-11T07:27:47.031926+0000","last_peered":"2024-09-11T07:27:47.031926+0000","last_clean":"2024-09-11T07:27:47.031926+0000","last_became_active":"2024-09-11T07:26:37.863100+0000","last_became_peered":"2024-09-11T07:26:37.863100+0000","last_unstale":"2024-09-11T07:27:47.031926+0000","last_undegraded":"2024-09-11T07:27:47.031926+0000","last_fullsized":"2024-09-11T07:27:47.031926+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:47.031807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.031807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:54.469643+0000","scrub_duration":0.001556195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"64'10","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.732611+0000","last_change":"2024-09-11T07:26:33.545346+0000","last_active":"2024-09-11T07:27:38.732611+0000","last_peered":"2024-09-11T07:27:38.732611+0000","last_clean":"2024-09-11T07:27:38.732611+0000","last_became_active":"2024-09-11T07:26:33.545078+0000","last_became_peered":"2024-09-11T07:26:33.545078+0000","last_unstale":"2024-09-11T07:27:38.732611+0000","last_undegraded":"2024-09-11T07:27:38.732611+0000","last_fullsized":"2024-09-11T07:27:38.732611+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:56.398062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.603236+0000","last_change":"2024-09-11T07:26:31.424634+0000","last_active":"2024-09-11T07:27:39.603236+0000","last_peered":"2024-09-11T07:27:39.603236+0000","last_clean":"2024-09-11T07:27:39.603236+0000","last_became_active":"2024-09-11T07:26:31.423789+0000","last_became_peered":"2024-09-11T07:26:31.423789+0000","last_unstale":"2024-09-11T07:27:39.603236+0000","last_undegraded":"2024-09-11T07:27:39.603236+0000","last_fullsized":"2024-09-11T07:27:39.603236+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:57.863276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":54,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:44.497412+0000","last_change":"2024-09-11T07:26:35.562509+0000","last_active":"2024-09-11T07:27:44.497412+0000","last_peered":"2024-09-11T07:27:44.497412+0000","last_clean":"2024-09-11T07:27:44.497412+0000","last_became_active":"2024-09-11T07:26:35.562245+0000","last_became_peered":"2024-09-11T07:26:35.562245+0000","last_unstale":"2024-09-11T07:27:44.497412+0000","last_undegraded":"2024-09-11T07:27:44.497412+0000","last_fullsized":"2024-09-11T07:27:44.497412+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:58.799807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:49.245150+0000","last_change":"2024-09-11T07:27:49.245119+0000","last_active":"2024-09-11T07:27:49.245150+0000","last_peered":"2024-09-11T07:27:49.245150+0000","last_clean":"2024-09-11T07:27:49.245150+0000","last_became_active":"2024-09-11T07:26:37.566749+0000","last_became_peered":"2024-09-11T07:26:37.566749+0000","last_unstale":"2024-09-11T07:27:49.245150+0000","last_undegraded":"2024-09-11T07:27:49.245150+0000","last_fullsized":"2024-09-11T07:27:49.245150+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:49.245047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.245047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:49.614854+0000","scrub_duration":0.001867966,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"64'9","reported_seq":48,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.603530+0000","last_change":"2024-09-11T07:26:33.537700+0000","last_active":"2024-09-11T07:27:39.603530+0000","last_peered":"2024-09-11T07:27:39.603530+0000","last_clean":"2024-09-11T07:27:39.603530+0000","last_became_active":"2024-09-11T07:26:33.537425+0000","last_became_peered":"2024-09-11T07:26:33.537425+0000","last_unstale":"2024-09-11T07:27:39.603530+0000","last_undegraded":"2024-09-11T07:27:39.603530+0000","last_fullsized":"2024-09-11T07:27:39.603530+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:46.145102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.429201+0000","last_change":"2024-09-11T07:26:31.419682+0000","last_active":"2024-09-11T07:27:38.429201+0000","last_peered":"2024-09-11T07:27:38.429201+0000","last_clean":"2024-09-11T07:27:38.429201+0000","last_became_active":"2024-09-11T07:26:31.419409+0000","last_became_peered":"2024-09-11T07:26:31.419409+0000","last_unstale":"2024-09-11T07:27:38.429201+0000","last_undegraded":"2024-09-11T07:27:38.429201+0000","last_fullsized":"2024-09-11T07:27:38.429201+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":54,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:44.497272+0000","last_change":"2024-09-11T07:26:35.560917+0000","last_active":"2024-09-11T07:27:44.497272+0000","last_peered":"2024-09-11T07:27:44.497272+0000","last_clean":"2024-09-11T07:27:44.497272+0000","last_became_active":"2024-09-11T07:26:35.560610+0000","last_became_peered":"2024-09-11T07:26:35.560610+0000","last_unstale":"2024-09-11T07:27:44.497272+0000","last_undegraded":"2024-09-11T07:27:44.497272+0000","last_fullsized":"2024-09-11T07:27:44.497272+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:57.052975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431782+0000","last_change":"2024-09-11T07:26:37.566136+0000","last_active":"2024-09-11T07:27:38.431782+0000","last_peered":"2024-09-11T07:27:38.431782+0000","last_clean":"2024-09-11T07:27:38.431782+0000","last_became_active":"2024-09-11T07:26:37.565723+0000","last_became_peered":"2024-09-11T07:26:37.565723+0000","last_unstale":"2024-09-11T07:27:38.431782+0000","last_undegraded":"2024-09-11T07:27:38.431782+0000","last_fullsized":"2024-09-11T07:27:38.431782+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:00.803451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":48,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428656+0000","last_change":"2024-09-11T07:26:33.541085+0000","last_active":"2024-09-11T07:27:38.428656+0000","last_peered":"2024-09-11T07:27:38.428656+0000","last_clean":"2024-09-11T07:27:38.428656+0000","last_became_active":"2024-09-11T07:26:33.540688+0000","last_became_peered":"2024-09-11T07:26:33.540688+0000","last_unstale":"2024-09-11T07:27:38.428656+0000","last_undegraded":"2024-09-11T07:27:38.428656+0000","last_fullsized":"2024-09-11T07:27:38.428656+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:44.875878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.672532+0000","last_change":"2024-09-11T07:26:31.425981+0000","last_active":"2024-09-11T07:27:38.672532+0000","last_peered":"2024-09-11T07:27:38.672532+0000","last_clean":"2024-09-11T07:27:38.672532+0000","last_became_active":"2024-09-11T07:26:31.425587+0000","last_became_peered":"2024-09-11T07:26:31.425587+0000","last_unstale":"2024-09-11T07:27:38.672532+0000","last_undegraded":"2024-09-11T07:27:38.672532+0000","last_fullsized":"2024-09-11T07:27:38.672532+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:46.405065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.732072+0000","last_change":"2024-09-11T07:26:35.562603+0000","last_active":"2024-09-11T07:27:38.732072+0000","last_peered":"2024-09-11T07:27:38.732072+0000","last_clean":"2024-09-11T07:27:38.732072+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:27:38.732072+0000","last_undegraded":"2024-09-11T07:27:38.732072+0000","last_fullsized":"2024-09-11T07:27:38.732072+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:01.555693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428292+0000","last_change":"2024-09-11T07:26:37.569191+0000","last_active":"2024-09-11T07:27:38.428292+0000","last_peered":"2024-09-11T07:27:38.428292+0000","last_clean":"2024-09-11T07:27:38.428292+0000","last_became_active":"2024-09-11T07:26:37.568886+0000","last_became_peered":"2024-09-11T07:26:37.568886+0000","last_unstale":"2024-09-11T07:27:38.428292+0000","last_undegraded":"2024-09-11T07:27:38.428292+0000","last_fullsized":"2024-09-11T07:27:38.428292+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":61,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:48.081485+0000","last_change":"2024-09-11T07:27:48.081460+0000","last_active":"2024-09-11T07:27:48.081485+0000","last_peered":"2024-09-11T07:27:48.081485+0000","last_clean":"2024-09-11T07:27:48.081485+0000","last_became_active":"2024-09-11T07:26:33.540208+0000","last_became_peered":"2024-09-11T07:26:33.540208+0000","last_unstale":"2024-09-11T07:27:48.081485+0000","last_undegraded":"2024-09-11T07:27:48.081485+0000","last_fullsized":"2024-09-11T07:27:48.081485+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:27:48.081392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:48.081392+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:04.524601+0000","scrub_duration":0.0022394590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:49.677089+0000","last_change":"2024-09-11T07:27:49.677063+0000","last_active":"2024-09-11T07:27:49.677089+0000","last_peered":"2024-09-11T07:27:49.677089+0000","last_clean":"2024-09-11T07:27:49.677089+0000","last_became_active":"2024-09-11T07:26:31.422671+0000","last_became_peered":"2024-09-11T07:26:31.422671+0000","last_unstale":"2024-09-11T07:27:49.677089+0000","last_undegraded":"2024-09-11T07:27:49.677089+0000","last_fullsized":"2024-09-11T07:27:49.677089+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.677003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:54.836177+0000","scrub_duration":0.0013962969999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428504+0000","last_change":"2024-09-11T07:26:35.567257+0000","last_active":"2024-09-11T07:27:38.428504+0000","last_peered":"2024-09-11T07:27:38.428504+0000","last_clean":"2024-09-11T07:27:38.428504+0000","last_became_active":"2024-09-11T07:26:35.566931+0000","last_became_peered":"2024-09-11T07:26:35.566931+0000","last_unstale":"2024-09-11T07:27:38.428504+0000","last_undegraded":"2024-09-11T07:27:38.428504+0000","last_fullsized":"2024-09-11T07:27:38.428504+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:55.331264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.731668+0000","last_change":"2024-09-11T07:26:37.561886+0000","last_active":"2024-09-11T07:27:38.731668+0000","last_peered":"2024-09-11T07:27:38.731668+0000","last_clean":"2024-09-11T07:27:38.731668+0000","last_became_active":"2024-09-11T07:26:37.561573+0000","last_became_peered":"2024-09-11T07:26:37.561573+0000","last_unstale":"2024-09-11T07:27:38.731668+0000","last_undegraded":"2024-09-11T07:27:38.731668+0000","last_fullsized":"2024-09-11T07:27:38.731668+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:55.357802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"82'6","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:47.789547+0000","last_change":"2024-09-11T07:27:47.789519+0000","last_active":"2024-09-11T07:27:47.789547+0000","last_peered":"2024-09-11T07:27:47.789547+0000","last_clean":"2024-09-11T07:27:47.789547+0000","last_became_active":"2024-09-11T07:26:33.545830+0000","last_became_peered":"2024-09-11T07:26:33.545830+0000","last_unstale":"2024-09-11T07:27:47.789547+0000","last_undegraded":"2024-09-11T07:27:47.789547+0000","last_fullsized":"2024-09-11T07:27:47.789547+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'6","last_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_deep_scrub":"82'6","last_deep_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.789465+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:59.162056+0000","scrub_duration":0.002624679,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.603611+0000","last_change":"2024-09-11T07:26:31.420799+0000","last_active":"2024-09-11T07:27:39.603611+0000","last_peered":"2024-09-11T07:27:39.603611+0000","last_clean":"2024-09-11T07:27:39.603611+0000","last_became_active":"2024-09-11T07:26:31.420518+0000","last_became_peered":"2024-09-11T07:26:31.420518+0000","last_unstale":"2024-09-11T07:27:39.603611+0000","last_undegraded":"2024-09-11T07:27:39.603611+0000","last_fullsized":"2024-09-11T07:27:39.603611+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:43.691430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.672144+0000","last_change":"2024-09-11T07:26:35.560782+0000","last_active":"2024-09-11T07:27:38.672144+0000","last_peered":"2024-09-11T07:27:38.672144+0000","last_clean":"2024-09-11T07:27:38.672144+0000","last_became_active":"2024-09-11T07:26:35.560506+0000","last_became_peered":"2024-09-11T07:26:35.560506+0000","last_unstale":"2024-09-11T07:27:38.672144+0000","last_undegraded":"2024-09-11T07:27:38.672144+0000","last_fullsized":"2024-09-11T07:27:38.672144+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:55.619984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:42.059097+0000","last_change":"2024-09-11T07:27:42.059066+0000","last_active":"2024-09-11T07:27:42.059097+0000","last_peered":"2024-09-11T07:27:42.059097+0000","last_clean":"2024-09-11T07:27:42.059097+0000","last_became_active":"2024-09-11T07:26:37.564208+0000","last_became_peered":"2024-09-11T07:26:37.564208+0000","last_unstale":"2024-09-11T07:27:42.059097+0000","last_undegraded":"2024-09-11T07:27:42.059097+0000","last_fullsized":"2024-09-11T07:27:42.059097+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.059001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.059001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:51.267826+0000","scrub_duration":0.001418094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.731791+0000","last_change":"2024-09-11T07:26:33.542913+0000","last_active":"2024-09-11T07:27:38.731791+0000","last_peered":"2024-09-11T07:27:38.731791+0000","last_clean":"2024-09-11T07:27:38.731791+0000","last_became_active":"2024-09-11T07:26:33.542625+0000","last_became_peered":"2024-09-11T07:26:33.542625+0000","last_unstale":"2024-09-11T07:27:38.731791+0000","last_undegraded":"2024-09-11T07:27:38.731791+0000","last_fullsized":"2024-09-11T07:27:38.731791+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:58.649007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.672873+0000","last_change":"2024-09-11T07:26:31.426196+0000","last_active":"2024-09-11T07:27:38.672873+0000","last_peered":"2024-09-11T07:27:38.672873+0000","last_clean":"2024-09-11T07:27:38.672873+0000","last_became_active":"2024-09-11T07:26:31.425876+0000","last_became_peered":"2024-09-11T07:26:31.425876+0000","last_unstale":"2024-09-11T07:27:38.672873+0000","last_undegraded":"2024-09-11T07:27:38.672873+0000","last_fullsized":"2024-09-11T07:27:38.672873+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:59.116397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.012605+0000","last_change":"2024-09-11T07:26:35.565591+0000","last_active":"2024-09-11T07:27:39.012605+0000","last_peered":"2024-09-11T07:27:39.012605+0000","last_clean":"2024-09-11T07:27:39.012605+0000","last_became_active":"2024-09-11T07:26:35.565339+0000","last_became_peered":"2024-09-11T07:26:35.565339+0000","last_unstale":"2024-09-11T07:27:39.012605+0000","last_undegraded":"2024-09-11T07:27:39.012605+0000","last_fullsized":"2024-09-11T07:27:39.012605+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:01.008070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.731969+0000","last_change":"2024-09-11T07:26:37.862599+0000","last_active":"2024-09-11T07:27:38.731969+0000","last_peered":"2024-09-11T07:27:38.731969+0000","last_clean":"2024-09-11T07:27:38.731969+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:27:38.731969+0000","last_undegraded":"2024-09-11T07:27:38.731969+0000","last_fullsized":"2024-09-11T07:27:38.731969+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:03.134547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.013085+0000","last_change":"2024-09-11T07:26:37.564199+0000","last_active":"2024-09-11T07:27:39.013085+0000","last_peered":"2024-09-11T07:27:39.013085+0000","last_clean":"2024-09-11T07:27:39.013085+0000","last_became_active":"2024-09-11T07:26:37.563942+0000","last_became_peered":"2024-09-11T07:26:37.563942+0000","last_unstale":"2024-09-11T07:27:39.013085+0000","last_undegraded":"2024-09-11T07:27:39.013085+0000","last_fullsized":"2024-09-11T07:27:39.013085+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:01.320334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.602998+0000","last_change":"2024-09-11T07:26:31.424413+0000","last_active":"2024-09-11T07:27:39.602998+0000","last_peered":"2024-09-11T07:27:39.602998+0000","last_clean":"2024-09-11T07:27:39.602998+0000","last_became_active":"2024-09-11T07:26:31.423932+0000","last_became_peered":"2024-09-11T07:26:31.423932+0000","last_unstale":"2024-09-11T07:27:39.602998+0000","last_undegraded":"2024-09-11T07:27:39.602998+0000","last_fullsized":"2024-09-11T07:27:39.602998+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:48.169462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428600+0000","last_change":"2024-09-11T07:26:33.547101+0000","last_active":"2024-09-11T07:27:38.428600+0000","last_peered":"2024-09-11T07:27:38.428600+0000","last_clean":"2024-09-11T07:27:38.428600+0000","last_became_active":"2024-09-11T07:26:33.546797+0000","last_became_peered":"2024-09-11T07:26:33.546797+0000","last_unstale":"2024-09-11T07:27:38.428600+0000","last_undegraded":"2024-09-11T07:27:38.428600+0000","last_fullsized":"2024-09-11T07:27:38.428600+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"64'8","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:44.497171+0000","last_change":"2024-09-11T07:26:35.561379+0000","last_active":"2024-09-11T07:27:44.497171+0000","last_peered":"2024-09-11T07:27:44.497171+0000","last_clean":"2024-09-11T07:27:44.497171+0000","last_became_active":"2024-09-11T07:26:35.559304+0000","last_became_peered":"2024-09-11T07:26:35.559304+0000","last_unstale":"2024-09-11T07:27:44.497171+0000","last_undegraded":"2024-09-11T07:27:44.497171+0000","last_fullsized":"2024-09-11T07:27:44.497171+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:59.074755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.429308+0000","last_change":"2024-09-11T07:26:37.563308+0000","last_active":"2024-09-11T07:27:38.429308+0000","last_peered":"2024-09-11T07:27:38.429308+0000","last_clean":"2024-09-11T07:27:38.429308+0000","last_became_active":"2024-09-11T07:26:37.562501+0000","last_became_peered":"2024-09-11T07:26:37.562501+0000","last_unstale":"2024-09-11T07:27:38.429308+0000","last_undegraded":"2024-09-11T07:27:38.429308+0000","last_fullsized":"2024-09-11T07:27:38.429308+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:52.287238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.732915+0000","last_change":"2024-09-11T07:27:38.732867+0000","last_active":"2024-09-11T07:27:38.732915+0000","last_peered":"2024-09-11T07:27:38.732915+0000","last_clean":"2024-09-11T07:27:38.732915+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:27:38.732915+0000","last_undegraded":"2024-09-11T07:27:38.732915+0000","last_fullsized":"2024-09-11T07:27:38.732915+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:38.732795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:38.732795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:56.730699+0000","scrub_duration":0.0025091950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.672775+0000","last_change":"2024-09-11T07:26:33.541438+0000","last_active":"2024-09-11T07:27:38.672775+0000","last_peered":"2024-09-11T07:27:38.672775+0000","last_clean":"2024-09-11T07:27:38.672775+0000","last_became_active":"2024-09-11T07:26:33.541154+0000","last_became_peered":"2024-09-11T07:26:33.541154+0000","last_unstale":"2024-09-11T07:27:38.672775+0000","last_undegraded":"2024-09-11T07:27:38.672775+0000","last_fullsized":"2024-09-11T07:27:38.672775+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:02.386606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.428544+0000","last_change":"2024-09-11T07:26:35.559093+0000","last_active":"2024-09-11T07:27:38.428544+0000","last_peered":"2024-09-11T07:27:38.428544+0000","last_clean":"2024-09-11T07:27:38.428544+0000","last_became_active":"2024-09-11T07:26:35.558809+0000","last_became_peered":"2024-09-11T07:26:35.558809+0000","last_unstale":"2024-09-11T07:27:38.428544+0000","last_undegraded":"2024-09-11T07:27:38.428544+0000","last_fullsized":"2024-09-11T07:27:38.428544+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:00.864666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431627+0000","last_change":"2024-09-11T07:26:37.862660+0000","last_active":"2024-09-11T07:27:38.431627+0000","last_peered":"2024-09-11T07:27:38.431627+0000","last_clean":"2024-09-11T07:27:38.431627+0000","last_became_active":"2024-09-11T07:26:37.862373+0000","last_became_peered":"2024-09-11T07:26:37.862373+0000","last_unstale":"2024-09-11T07:27:38.431627+0000","last_undegraded":"2024-09-11T07:27:38.431627+0000","last_fullsized":"2024-09-11T07:27:38.431627+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:49.412299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.672490+0000","last_change":"2024-09-11T07:27:33.983889+0000","last_active":"2024-09-11T07:27:38.672490+0000","last_peered":"2024-09-11T07:27:38.672490+0000","last_clean":"2024-09-11T07:27:38.672490+0000","last_became_active":"2024-09-11T07:26:31.424651+0000","last_became_peered":"2024-09-11T07:26:31.424651+0000","last_unstale":"2024-09-11T07:27:38.672490+0000","last_undegraded":"2024-09-11T07:27:38.672490+0000","last_fullsized":"2024-09-11T07:27:38.672490+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:33.983813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:33.983813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:55.264233+0000","scrub_duration":0.001290759,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":57,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:39.013092+0000","last_change":"2024-09-11T07:26:33.542921+0000","last_active":"2024-09-11T07:27:39.013092+0000","last_peered":"2024-09-11T07:27:39.013092+0000","last_clean":"2024-09-11T07:27:39.013092+0000","last_became_active":"2024-09-11T07:26:33.542544+0000","last_became_peered":"2024-09-11T07:26:33.542544+0000","last_unstale":"2024-09-11T07:27:39.013092+0000","last_undegraded":"2024-09-11T07:27:39.013092+0000","last_fullsized":"2024-09-11T07:27:39.013092+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:52.019544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.429601+0000","last_change":"2024-09-11T07:26:35.556136+0000","last_active":"2024-09-11T07:27:38.429601+0000","last_peered":"2024-09-11T07:27:38.429601+0000","last_clean":"2024-09-11T07:27:38.429601+0000","last_became_active":"2024-09-11T07:26:35.555862+0000","last_became_peered":"2024-09-11T07:26:35.555862+0000","last_unstale":"2024-09-11T07:27:38.429601+0000","last_undegraded":"2024-09-11T07:27:38.429601+0000","last_fullsized":"2024-09-11T07:27:38.429601+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:48.624824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.732535+0000","last_change":"2024-09-11T07:26:37.862060+0000","last_active":"2024-09-11T07:27:38.732535+0000","last_peered":"2024-09-11T07:27:38.732535+0000","last_clean":"2024-09-11T07:27:38.732535+0000","last_became_active":"2024-09-11T07:26:37.861647+0000","last_became_peered":"2024-09-11T07:26:37.861647+0000","last_unstale":"2024-09-11T07:27:38.732535+0000","last_undegraded":"2024-09-11T07:27:38.732535+0000","last_fullsized":"2024-09-11T07:27:38.732535+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:52.108042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:45.732651+0000","last_change":"2024-09-11T07:27:45.732604+0000","last_active":"2024-09-11T07:27:45.732651+0000","last_peered":"2024-09-11T07:27:45.732651+0000","last_clean":"2024-09-11T07:27:45.732651+0000","last_became_active":"2024-09-11T07:26:31.420598+0000","last_became_peered":"2024-09-11T07:26:31.420598+0000","last_unstale":"2024-09-11T07:27:45.732651+0000","last_undegraded":"2024-09-11T07:27:45.732651+0000","last_fullsized":"2024-09-11T07:27:45.732651+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:45.732528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:45.732528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:48.605075+0000","scrub_duration":0.0015466659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"64'12","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.732469+0000","last_change":"2024-09-11T07:26:33.543016+0000","last_active":"2024-09-11T07:27:38.732469+0000","last_peered":"2024-09-11T07:27:38.732469+0000","last_clean":"2024-09-11T07:27:38.732469+0000","last_became_active":"2024-09-11T07:26:33.542711+0000","last_became_peered":"2024-09-11T07:26:33.542711+0000","last_unstale":"2024-09-11T07:27:38.732469+0000","last_undegraded":"2024-09-11T07:27:38.732469+0000","last_fullsized":"2024-09-11T07:27:38.732469+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:44.518917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T07:27:38.431617+0000","last_change":"2024-09-11T07:26:35.562350+0000","last_active":"2024-09-11T07:27:38.431617+0000","last_peered":"2024-09-11T07:27:38.431617+0000","last_clean":"2024-09-11T07:27:38.431617+0000","last_became_active":"2024-09-11T07:26:35.562037+0000","last_became_peered":"2024-09-11T07:26:35.562037+0000","last_unstale":"2024-09-11T07:27:38.431617+0000","last_undegraded":"2024-09-11T07:27:38.431617+0000","last_fullsized":"2024-09-11T07:27:38.431617+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:27:42.573835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":405,"ondisk_log_size":405,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299413,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298768,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446384,"statfs":{"total":95995035648,"available":95689097216,"internally_reserved":0,"allocated":1449984,"data_stored":943568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.52600000000000002}]},{"osd":1,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.377},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.34899999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.46500000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":12.625999999999999,"5min":12.625999999999999,"15min":12.625999999999999},"last":0.432},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.41699999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.50900000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.49199999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54200000000000004}]}]},{"osd":6,"up_from":44,"seq":188978561049,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298744,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446408,"statfs":{"total":95995035648,"available":95689121792,"internally_reserved":0,"allocated":1425408,"data_stored":942046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.374,"5min":0.247,"15min":0.247},"max":{"1min":0.65000000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.374},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.60499999999999998,"15min":0.624},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.754,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.49299999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.82899999999999996,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.49299999999999999,"15min":0.495},"min":{"1min":0.35799999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.48199999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.61499999999999999,"15min":0.63500000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.70299999999999996,"5min":3.234,"15min":3.234},"last":0.317},{"interface":"front","average":{"1min":0.496,"5min":0.59099999999999997,"15min":0.60699999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.80200000000000005,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.436}]},{"osd":3,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51500000000000001,"15min":0.51800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.63500000000000001,"15min":0.65500000000000003},"min":{"1min":0.41199999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.66900000000000004,"5min":3.1739999999999999,"15min":3.1739999999999999},"last":0.46000000000000002}]},{"osd":4,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.58299999999999996,"15min":0.60899999999999999},"min":{"1min":0.32700000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.65200000000000002,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.28399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.42199999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.65600000000000003,"15min":0.69699999999999995},"min":{"1min":0.32400000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64000000000000001,"5min":5.7329999999999997,"15min":5.7329999999999997},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.52000000000000002,"15min":0.52700000000000002},"min":{"1min":0.34000000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70899999999999996,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.38700000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:27:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.6349999999999998,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.47299999999999998}]}]},{"osd":5,"up_from":37,"seq":158913789983,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298168,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446984,"statfs":{"total":95995035648,"available":95689711616,"internally_reserved":0,"allocated":835584,"data_stored":352559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.35499999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.50600000000000001,"15min":0.502},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.36299999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.32800000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.373},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.45100000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.44},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.42099999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.375,"15min":0.37},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.42599999999999999,"15min":0.41999999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.40699999999999997}]},{"osd":6,"last update":"Wed Sep 11 07:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.34999999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:27:21 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.33500000000000002}]}]},{"osd":4,"up_from":31,"seq":133143986211,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298192,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446960,"statfs":{"total":95995035648,"available":95689687040,"internally_reserved":0,"allocated":860160,"data_stored":353004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51600000000000001,"15min":0.51300000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68999999999999995,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.502,"15min":0.498},"min":{"1min":0.36099999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.55800000000000005}]},{"osd":1,"last update":"Wed Sep 11 07:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.5,"15min":0.49399999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.71299999999999997,"15min":0.73699999999999999},"min":{"1min":0.372,"5min":0.318,"15min":0.318},"max":{"1min":1.1200000000000001,"5min":4.5510000000000002,"15min":4.5510000000000002},"last":0.68600000000000005}]},{"osd":2,"last update":"Wed Sep 11 07:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.73099999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77400000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.57099999999999995}]},{"osd":5,"last update":"Wed Sep 11 07:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.34499999999999997,"15min":0.33800000000000002},"min":{"1min":0.27600000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.378,"15min":0.372},"min":{"1min":0.29399999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.38400000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:26:56 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.42499999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:27:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.36499999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182440,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298200,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446952,"statfs":{"total":95995035648,"available":95689678848,"internally_reserved":0,"allocated":868352,"data_stored":352468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:26:52 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.39100000000000001,"15min":0.39500000000000002},"min":{"1min":0.21099999999999999,"5min":0.185,"15min":0.185},"max":{"1min":0.55000000000000004,"5min":0.621,"15min":0.621},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.435,"15min":0.44400000000000001},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.56299999999999994,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.38400000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:26:52 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.441,"15min":0.44},"min":{"1min":0.28999999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.42499999999999999,"15min":0.434},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.372}]},{"osd":2,"last update":"Wed Sep 11 07:26:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.46300000000000002,"15min":0.46899999999999997},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.46600000000000003,"15min":0.47299999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.67700000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.32300000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:27:12 2024","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":0.746,"15min":0.62},"min":{"1min":0.47299999999999998,"5min":0.376,"15min":0.376},"max":{"1min":21.856000000000002,"5min":21.856000000000002,"15min":21.856000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":1.345,"5min":0.72199999999999998,"15min":0.61799999999999999},"min":{"1min":0.432,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":19.146000000000001,"5min":19.146000000000001,"15min":19.146000000000001},"last":0.55500000000000005}]},{"osd":5,"last update":"Wed Sep 11 07:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":0.71999999999999997,"15min":0.622},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":18.416,"5min":18.416,"15min":18.416},"last":0.59299999999999997}]},{"osd":6,"last update":"Wed Sep 11 07:26:53 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":20.219999999999999,"5min":20.219999999999999,"15min":20.219999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":1.51,"5min":1.51,"15min":1.51},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":19.609000000000002,"5min":19.609000000000002,"15min":19.609000000000002},"last":0.46899999999999997}]},{"osd":7,"last update":"Wed Sep 11 07:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.61199999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298164,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446988,"statfs":{"total":95995035648,"available":95689715712,"internally_reserved":0,"allocated":831488,"data_stored":351805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:26 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.375,"15min":0.36299999999999999},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.435},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.38200000000000001,"15min":0.374},"min":{"1min":0.26600000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.55900000000000005,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.33000000000000002}]},{"osd":1,"last update":"Wed Sep 11 07:27:26 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.40300000000000002,"15min":0.38600000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.40200000000000002,"15min":0.38300000000000001},"min":{"1min":0.218,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.38300000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.439,"15min":0.42299999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64500000000000002,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.45900000000000002,"15min":0.45400000000000001},"min":{"1min":0.26600000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65000000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.308}]},{"osd":4,"last update":"Wed Sep 11 07:27:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55800000000000005,"15min":0.55300000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57699999999999996,"15min":0.57599999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80700000000000005,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.55500000000000005}]},{"osd":5,"last update":"Wed Sep 11 07:27:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.68799999999999994,"15min":0.69699999999999995},"min":{"1min":0.45500000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85899999999999999,"5min":2.931,"15min":2.931},"last":0.56799999999999995}]},{"osd":6,"last update":"Wed Sep 11 07:26:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.622},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.66500000000000004}]},{"osd":7,"last update":"Wed Sep 11 07:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.58999999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298216,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446936,"statfs":{"total":95995035648,"available":95689662464,"internally_reserved":0,"allocated":884736,"data_stored":353325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:02 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.34100000000000003,"15min":0.33100000000000002},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.45800000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.36399999999999999,"15min":0.35499999999999998},"min":{"1min":0.219,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.54700000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.27300000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:27:26 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.41899999999999998,"15min":0.41599999999999998},"min":{"1min":0.317,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.41199999999999998,"15min":0.40799999999999997},"min":{"1min":0.28699999999999998,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.33000000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:27:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.44500000000000001,"15min":0.44700000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.60699999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.41399999999999998},"min":{"1min":0.307,"5min":0.253,"15min":0.253},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.307}]},{"osd":4,"last update":"Wed Sep 11 07:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.64600000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:27:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.44,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.71699999999999997,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.625}]},{"osd":6,"last update":"Wed Sep 11 07:26:54 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.69499999999999995}]},{"osd":7,"last update":"Wed Sep 11 07:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.65800000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298760,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446392,"statfs":{"total":95995035648,"available":95689105408,"internally_reserved":0,"allocated":1441792,"data_stored":942378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:26:59 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.35899999999999999,"15min":0.34300000000000003},"min":{"1min":0.26600000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.248},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.35199999999999998,"15min":0.34000000000000002},"min":{"1min":0.24299999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.33100000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:27:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.40400000000000003,"15min":0.40699999999999997},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.434},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.59299999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.307}]},{"osd":3,"last update":"Wed Sep 11 07:26:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.46899999999999997,"15min":0.47699999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.56599999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.376},{"interface":"front","average":{"1min":0.44,"5min":0.47999999999999998,"15min":0.48599999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59299999999999997,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.36299999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56299999999999994,"15min":0.56899999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69499999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.745},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59799999999999998,"15min":0.60099999999999998},"min":{"1min":0.42499999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.66300000000000003}]},{"osd":5,"last update":"Wed Sep 11 07:27:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.40000000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.79100000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.505},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.41699999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78100000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.63800000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:26:52 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.72499999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.69599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:27:59.142 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T07:27:59.142 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T07:27:59.142 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-11T07:27:59.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-11T07:27:59.279 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:27:58 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:27:59.401 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:27:59.780 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:27:59 smithi104 bash[63907]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-11T07:27:59.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:59 smithi104 ceph-mon[33507]: 2.16 deep-scrub starts 2024-09-11T07:27:59.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:59 smithi104 ceph-mon[33507]: 2.16 deep-scrub ok 2024-09-11T07:27:59.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:59 smithi104 ceph-mon[33507]: 4.e scrub starts 2024-09-11T07:27:59.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:59 smithi104 ceph-mon[33507]: 4.e scrub ok 2024-09-11T07:27:59.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:59 smithi104 ceph-mon[33507]: 6.10 scrub starts 2024-09-11T07:27:59.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:59 smithi104 ceph-mon[33507]: 6.10 scrub ok 2024-09-11T07:27:59.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:59 smithi104 ceph-mon[33507]: 2.10 scrub starts 2024-09-11T07:27:59.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:59 smithi104 ceph-mon[33507]: 2.10 scrub ok 2024-09-11T07:27:59.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:59 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:59.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:59 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:59.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:59 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:59.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:59 smithi104 ceph-mon[33507]: Deploying daemon node-exporter.b on smithi104 2024-09-11T07:27:59.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:59 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:27:59.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:59 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:27:59.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:27:59 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:27:59.798 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[29462]: 2.16 deep-scrub starts 2024-09-11T07:27:59.798 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[29462]: 2.16 deep-scrub ok 2024-09-11T07:27:59.798 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[29462]: 4.e scrub starts 2024-09-11T07:27:59.798 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[29462]: 4.e scrub ok 2024-09-11T07:27:59.798 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[29462]: 6.10 scrub starts 2024-09-11T07:27:59.798 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[29462]: 6.10 scrub ok 2024-09-11T07:27:59.798 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[29462]: 2.10 scrub starts 2024-09-11T07:27:59.798 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[29462]: 2.10 scrub ok 2024-09-11T07:27:59.798 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:59.798 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:59.798 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:59.798 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[29462]: Deploying daemon node-exporter.b on smithi104 2024-09-11T07:27:59.798 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:27:59.799 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:27:59.799 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:27:59.799 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[37330]: 2.16 deep-scrub starts 2024-09-11T07:27:59.799 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[37330]: 2.16 deep-scrub ok 2024-09-11T07:27:59.799 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[37330]: 4.e scrub starts 2024-09-11T07:27:59.799 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[37330]: 4.e scrub ok 2024-09-11T07:27:59.799 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[37330]: 6.10 scrub starts 2024-09-11T07:27:59.799 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[37330]: 6.10 scrub ok 2024-09-11T07:27:59.799 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[37330]: 2.10 scrub starts 2024-09-11T07:27:59.799 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[37330]: 2.10 scrub ok 2024-09-11T07:27:59.799 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:59.799 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:59.800 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:27:59.800 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[37330]: Deploying daemon node-exporter.b on smithi104 2024-09-11T07:27:59.800 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:27:59.800 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:27:59.800 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:27:59 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:28:00.280 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:27:59 smithi104 bash[63907]: Getting image source signatures 2024-09-11T07:28:00.280 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:27:59 smithi104 bash[63907]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-11T07:28:00.280 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:27:59 smithi104 bash[63907]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-11T07:28:00.280 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:27:59 smithi104 bash[63907]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-11T07:28:00.620 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:00 smithi022 ceph-mon[29462]: 4.a scrub starts 2024-09-11T07:28:00.620 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:00 smithi022 ceph-mon[29462]: 4.a scrub ok 2024-09-11T07:28:00.620 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:00 smithi022 ceph-mon[29462]: 3.0 scrub starts 2024-09-11T07:28:00.620 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:00 smithi022 ceph-mon[29462]: 3.0 scrub ok 2024-09-11T07:28:00.620 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:00 smithi022 ceph-mon[29462]: osdmap e83: 8 total, 8 up, 8 in 2024-09-11T07:28:00.620 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:00 smithi022 ceph-mon[29462]: 4.1d scrub starts 2024-09-11T07:28:00.621 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:00 smithi022 ceph-mon[29462]: 4.1d scrub ok 2024-09-11T07:28:00.621 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:00 smithi022 ceph-mon[29462]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:28:00.621 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:00 smithi022 ceph-mon[29462]: 4.1c scrub starts 2024-09-11T07:28:00.621 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:00 smithi022 ceph-mon[29462]: 4.1c scrub ok 2024-09-11T07:28:00.621 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:00 smithi022 ceph-mon[37330]: 4.a scrub starts 2024-09-11T07:28:00.621 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:00 smithi022 ceph-mon[37330]: 4.a scrub ok 2024-09-11T07:28:00.621 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:00 smithi022 ceph-mon[37330]: 3.0 scrub starts 2024-09-11T07:28:00.621 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:00 smithi022 ceph-mon[37330]: 3.0 scrub ok 2024-09-11T07:28:00.621 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:00 smithi022 ceph-mon[37330]: osdmap e83: 8 total, 8 up, 8 in 2024-09-11T07:28:00.621 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:00 smithi022 ceph-mon[37330]: 4.1d scrub starts 2024-09-11T07:28:00.621 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:00 smithi022 ceph-mon[37330]: 4.1d scrub ok 2024-09-11T07:28:00.622 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:00 smithi022 ceph-mon[37330]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:28:00.622 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:00 smithi022 ceph-mon[37330]: 4.1c scrub starts 2024-09-11T07:28:00.622 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:00 smithi022 ceph-mon[37330]: 4.1c scrub ok 2024-09-11T07:28:00.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:00 smithi104 ceph-mon[33507]: 4.a scrub starts 2024-09-11T07:28:00.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:00 smithi104 ceph-mon[33507]: 4.a scrub ok 2024-09-11T07:28:00.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:00 smithi104 ceph-mon[33507]: 3.0 scrub starts 2024-09-11T07:28:00.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:00 smithi104 ceph-mon[33507]: 3.0 scrub ok 2024-09-11T07:28:00.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:00 smithi104 ceph-mon[33507]: osdmap e83: 8 total, 8 up, 8 in 2024-09-11T07:28:00.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:00 smithi104 ceph-mon[33507]: 4.1d scrub starts 2024-09-11T07:28:00.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:00 smithi104 ceph-mon[33507]: 4.1d scrub ok 2024-09-11T07:28:00.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:00 smithi104 ceph-mon[33507]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:28:00.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:00 smithi104 ceph-mon[33507]: 4.1c scrub starts 2024-09-11T07:28:00.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:00 smithi104 ceph-mon[33507]: 4.1c scrub ok 2024-09-11T07:28:01.310 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:28:01.311 INFO:teuthology.orchestra.run.smithi022.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T07:28:01.726 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:01 smithi104 bash[63907]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-11T07:28:01.726 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:01 smithi104 bash[63907]: Writing manifest to image destination 2024-09-11T07:28:01.822 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-11T07:28:01.822 INFO:tasks.cephadm:Setup complete, yielding 2024-09-11T07:28:01.822 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-11T07:28:01.834 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-11T07:28:01.834 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-guits-quincy-2024-09-10-2008-quincy'}, 'ceph': {'conf': {'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'ms bind msgr2': False, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)'], 'mon_bind_msgr2': False, 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'cephadm': {'cephadm_mode': 'root'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'workunit': {'branch': 'wip-guits-quincy-2024-09-10-2008-quincy', 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'}} 2024-09-11T07:28:01.834 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-11T07:28:01.835 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-11T07:28:02.030 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:01 smithi104 podman[63907]: 2024-09-11 07:28:01.754514314 +0000 UTC m=+2.262640470 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-11T07:28:02.030 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:01 smithi104 podman[63907]: 2024-09-11 07:28:01.870047977 +0000 UTC m=+2.378174133 container create 5b33e3e5e5c0d6312f21e2248fdfa6d6280818a7e77e0488574cd75147e9bed3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T07:28:02.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:01 smithi104 ceph-mon[33507]: 5.f scrub starts 2024-09-11T07:28:02.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:01 smithi104 ceph-mon[33507]: 5.f scrub ok 2024-09-11T07:28:02.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:01 smithi104 ceph-mon[33507]: 6.1e scrub starts 2024-09-11T07:28:02.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:01 smithi104 ceph-mon[33507]: 6.1e scrub ok 2024-09-11T07:28:02.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:01 smithi104 ceph-mon[33507]: 6.6 scrub starts 2024-09-11T07:28:02.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:01 smithi104 ceph-mon[33507]: 6.6 scrub ok 2024-09-11T07:28:02.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:01 smithi104 ceph-mon[33507]: 5.1f scrub starts 2024-09-11T07:28:02.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:01 smithi104 ceph-mon[33507]: 5.1f scrub ok 2024-09-11T07:28:02.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:01 smithi104 ceph-mon[33507]: 2.13 scrub starts 2024-09-11T07:28:02.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:01 smithi104 ceph-mon[33507]: 2.13 scrub ok 2024-09-11T07:28:02.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:01 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1260966995' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T07:28:02.093 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:28:02.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:01 smithi022 ceph-mon[29462]: 5.f scrub starts 2024-09-11T07:28:02.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:01 smithi022 ceph-mon[29462]: 5.f scrub ok 2024-09-11T07:28:02.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:01 smithi022 ceph-mon[29462]: 6.1e scrub starts 2024-09-11T07:28:02.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:01 smithi022 ceph-mon[29462]: 6.1e scrub ok 2024-09-11T07:28:02.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:01 smithi022 ceph-mon[29462]: 6.6 scrub starts 2024-09-11T07:28:02.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:01 smithi022 ceph-mon[29462]: 6.6 scrub ok 2024-09-11T07:28:02.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:01 smithi022 ceph-mon[29462]: 5.1f scrub starts 2024-09-11T07:28:02.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:01 smithi022 ceph-mon[29462]: 5.1f scrub ok 2024-09-11T07:28:02.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:01 smithi022 ceph-mon[29462]: 2.13 scrub starts 2024-09-11T07:28:02.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:01 smithi022 ceph-mon[29462]: 2.13 scrub ok 2024-09-11T07:28:02.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:01 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1260966995' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T07:28:02.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:01 smithi022 ceph-mon[37330]: 5.f scrub starts 2024-09-11T07:28:02.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:01 smithi022 ceph-mon[37330]: 5.f scrub ok 2024-09-11T07:28:02.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:01 smithi022 ceph-mon[37330]: 6.1e scrub starts 2024-09-11T07:28:02.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:01 smithi022 ceph-mon[37330]: 6.1e scrub ok 2024-09-11T07:28:02.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:01 smithi022 ceph-mon[37330]: 6.6 scrub starts 2024-09-11T07:28:02.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:01 smithi022 ceph-mon[37330]: 6.6 scrub ok 2024-09-11T07:28:02.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:01 smithi022 ceph-mon[37330]: 5.1f scrub starts 2024-09-11T07:28:02.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:01 smithi022 ceph-mon[37330]: 5.1f scrub ok 2024-09-11T07:28:02.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:01 smithi022 ceph-mon[37330]: 2.13 scrub starts 2024-09-11T07:28:02.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:01 smithi022 ceph-mon[37330]: 2.13 scrub ok 2024-09-11T07:28:02.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:01 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1260966995' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T07:28:02.530 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 podman[63907]: 2024-09-11 07:28:02.055377919 +0000 UTC m=+2.563504076 container init 5b33e3e5e5c0d6312f21e2248fdfa6d6280818a7e77e0488574cd75147e9bed3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T07:28:02.530 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 podman[63907]: 2024-09-11 07:28:02.058136115 +0000 UTC m=+2.566262274 container start 5b33e3e5e5c0d6312f21e2248fdfa6d6280818a7e77e0488574cd75147e9bed3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T07:28:02.531 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.061Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-11T07:28:02.531 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.061Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-11T07:28:02.531 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.062Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-11T07:28:02.531 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.062Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-11T07:28:02.531 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.062Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-11T07:28:02.531 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.062Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-11T07:28:02.531 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-11T07:28:02.531 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=arp 2024-09-11T07:28:02.531 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-11T07:28:02.531 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-11T07:28:02.532 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-11T07:28:02.532 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-11T07:28:02.532 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-11T07:28:02.532 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-11T07:28:02.532 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-11T07:28:02.532 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-11T07:28:02.532 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=edac 2024-09-11T07:28:02.532 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-11T07:28:02.532 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-11T07:28:02.532 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-11T07:28:02.532 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-11T07:28:02.532 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-11T07:28:02.533 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-11T07:28:02.533 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-11T07:28:02.533 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-11T07:28:02.533 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-11T07:28:02.533 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-11T07:28:02.533 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-11T07:28:02.533 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-11T07:28:02.533 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-11T07:28:02.533 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-11T07:28:02.533 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-11T07:28:02.534 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-11T07:28:02.534 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=os 2024-09-11T07:28:02.534 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-11T07:28:02.534 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-11T07:28:02.534 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-11T07:28:02.534 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-11T07:28:02.534 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-11T07:28:02.534 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-11T07:28:02.534 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-11T07:28:02.535 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=stat 2024-09-11T07:28:02.535 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-11T07:28:02.535 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-11T07:28:02.535 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-11T07:28:02.535 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=time 2024-09-11T07:28:02.535 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-11T07:28:02.535 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=uname 2024-09-11T07:28:02.535 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-11T07:28:02.535 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-11T07:28:02.535 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-11T07:28:02.535 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-11T07:28:02.535 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b[63980]: ts=2024-09-11T07:28:02.063Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-11T07:28:02.535 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 bash[63907]: 5b33e3e5e5c0d6312f21e2248fdfa6d6280818a7e77e0488574cd75147e9bed3 2024-09-11T07:28:02.535 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:28:02 smithi104 systemd[1]: Started Ceph node-exporter.b for 12a285d8-700e-11ef-bcea-c7b262605968. 2024-09-11T07:28:03.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-mon[33507]: 4.15 deep-scrub starts 2024-09-11T07:28:03.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-mon[33507]: 4.15 deep-scrub ok 2024-09-11T07:28:03.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-mon[33507]: 4.9 deep-scrub starts 2024-09-11T07:28:03.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-mon[33507]: 4.9 deep-scrub ok 2024-09-11T07:28:03.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-mon[33507]: 3.2 deep-scrub starts 2024-09-11T07:28:03.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-mon[33507]: 3.2 deep-scrub ok 2024-09-11T07:28:03.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-mon[33507]: 4.f deep-scrub starts 2024-09-11T07:28:03.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-mon[33507]: 4.f deep-scrub ok 2024-09-11T07:28:03.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-mon[33507]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:28:03.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-mon[33507]: 2.17 scrub starts 2024-09-11T07:28:03.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-mon[33507]: 2.17 scrub ok 2024-09-11T07:28:03.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:03.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:03.032 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:03.032 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:03.032 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:03.032 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-mon[33507]: 4.b scrub starts 2024-09-11T07:28:03.032 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-mon[33507]: 4.b scrub ok 2024-09-11T07:28:03.032 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:02 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:03.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[29462]: 4.15 deep-scrub starts 2024-09-11T07:28:03.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[29462]: 4.15 deep-scrub ok 2024-09-11T07:28:03.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[29462]: 4.9 deep-scrub starts 2024-09-11T07:28:03.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[29462]: 4.9 deep-scrub ok 2024-09-11T07:28:03.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[29462]: 3.2 deep-scrub starts 2024-09-11T07:28:03.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[29462]: 3.2 deep-scrub ok 2024-09-11T07:28:03.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[29462]: 4.f deep-scrub starts 2024-09-11T07:28:03.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[29462]: 4.f deep-scrub ok 2024-09-11T07:28:03.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[29462]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:28:03.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[29462]: 2.17 scrub starts 2024-09-11T07:28:03.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[29462]: 2.17 scrub ok 2024-09-11T07:28:03.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:03.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:03.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:03.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:03.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:03.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[29462]: 4.b scrub starts 2024-09-11T07:28:03.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[29462]: 4.b scrub ok 2024-09-11T07:28:03.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:03.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[37330]: 4.15 deep-scrub starts 2024-09-11T07:28:03.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[37330]: 4.15 deep-scrub ok 2024-09-11T07:28:03.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[37330]: 4.9 deep-scrub starts 2024-09-11T07:28:03.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[37330]: 4.9 deep-scrub ok 2024-09-11T07:28:03.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[37330]: 3.2 deep-scrub starts 2024-09-11T07:28:03.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[37330]: 3.2 deep-scrub ok 2024-09-11T07:28:03.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[37330]: 4.f deep-scrub starts 2024-09-11T07:28:03.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[37330]: 4.f deep-scrub ok 2024-09-11T07:28:03.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[37330]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:28:03.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[37330]: 2.17 scrub starts 2024-09-11T07:28:03.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[37330]: 2.17 scrub ok 2024-09-11T07:28:03.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:03.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:03.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:03.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:03.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:03.171 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[37330]: 4.b scrub starts 2024-09-11T07:28:03.171 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[37330]: 4.b scrub ok 2024-09-11T07:28:03.171 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:02 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:04.280 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:28:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:28:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:28:04.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:03 smithi104 ceph-mon[33507]: 6.b scrub starts 2024-09-11T07:28:04.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:03 smithi104 ceph-mon[33507]: 6.b scrub ok 2024-09-11T07:28:04.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:03 smithi104 ceph-mon[33507]: 2.18 scrub starts 2024-09-11T07:28:04.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:03 smithi104 ceph-mon[33507]: 2.18 scrub ok 2024-09-11T07:28:04.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:03 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:04.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:03 smithi104 ceph-mon[33507]: 4.1f scrub starts 2024-09-11T07:28:04.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:03 smithi104 ceph-mon[33507]: 4.1f scrub ok 2024-09-11T07:28:04.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:03 smithi104 ceph-mon[33507]: 5.3 scrub starts 2024-09-11T07:28:04.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:03 smithi104 ceph-mon[33507]: 5.3 scrub ok 2024-09-11T07:28:04.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:03 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1184504209' entity='client.admin' 2024-09-11T07:28:04.281 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:03 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:28:03] ENGINE Bus STOPPING 2024-09-11T07:28:04.282 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:03 smithi022 ceph-mon[37330]: 6.b scrub starts 2024-09-11T07:28:04.282 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:03 smithi022 ceph-mon[37330]: 6.b scrub ok 2024-09-11T07:28:04.282 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:03 smithi022 ceph-mon[37330]: 2.18 scrub starts 2024-09-11T07:28:04.282 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:03 smithi022 ceph-mon[37330]: 2.18 scrub ok 2024-09-11T07:28:04.282 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:03 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:04.282 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:03 smithi022 ceph-mon[37330]: 4.1f scrub starts 2024-09-11T07:28:04.282 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:03 smithi022 ceph-mon[37330]: 4.1f scrub ok 2024-09-11T07:28:04.282 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:03 smithi022 ceph-mon[37330]: 5.3 scrub starts 2024-09-11T07:28:04.282 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:03 smithi022 ceph-mon[37330]: 5.3 scrub ok 2024-09-11T07:28:04.282 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:03 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1184504209' entity='client.admin' 2024-09-11T07:28:04.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:03 smithi022 ceph-mon[29462]: 6.b scrub starts 2024-09-11T07:28:04.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:03 smithi022 ceph-mon[29462]: 6.b scrub ok 2024-09-11T07:28:04.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:03 smithi022 ceph-mon[29462]: 2.18 scrub starts 2024-09-11T07:28:04.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:03 smithi022 ceph-mon[29462]: 2.18 scrub ok 2024-09-11T07:28:04.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:03 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:04.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:03 smithi022 ceph-mon[29462]: 4.1f scrub starts 2024-09-11T07:28:04.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:03 smithi022 ceph-mon[29462]: 4.1f scrub ok 2024-09-11T07:28:04.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:03 smithi022 ceph-mon[29462]: 5.3 scrub starts 2024-09-11T07:28:04.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:03 smithi022 ceph-mon[29462]: 5.3 scrub ok 2024-09-11T07:28:04.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:03 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1184504209' entity='client.admin' 2024-09-11T07:28:04.623 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:04 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:28:04] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T07:28:04.623 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:04 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:28:04] ENGINE Bus STOPPED 2024-09-11T07:28:04.623 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:04 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:28:04] ENGINE Bus STARTING 2024-09-11T07:28:04.623 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:04 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:28:04] ENGINE Serving on http://:::9283 2024-09-11T07:28:04.623 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:04 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:28:04] ENGINE Bus STARTED 2024-09-11T07:28:05.147 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:04 smithi022 ceph-mon[29462]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:28:05.147 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:04 smithi022 ceph-mon[29462]: 6.a scrub starts 2024-09-11T07:28:05.148 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:04 smithi022 ceph-mon[29462]: 6.a scrub ok 2024-09-11T07:28:05.148 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:04 smithi022 ceph-mon[29462]: 4.d scrub starts 2024-09-11T07:28:05.148 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:04 smithi022 ceph-mon[29462]: 4.d scrub ok 2024-09-11T07:28:05.148 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:04 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:28:05.148 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:04 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:28:05.148 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:04 smithi022 ceph-mon[37330]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:28:05.148 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:04 smithi022 ceph-mon[37330]: 6.a scrub starts 2024-09-11T07:28:05.148 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:04 smithi022 ceph-mon[37330]: 6.a scrub ok 2024-09-11T07:28:05.148 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:04 smithi022 ceph-mon[37330]: 4.d scrub starts 2024-09-11T07:28:05.148 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:04 smithi022 ceph-mon[37330]: 4.d scrub ok 2024-09-11T07:28:05.148 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:04 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:28:05.149 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:04 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:28:05.178 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-11T07:28:05.178 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T07:28:05.178 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:28:05.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:04 smithi104 ceph-mon[33507]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:28:05.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:04 smithi104 ceph-mon[33507]: 6.a scrub starts 2024-09-11T07:28:05.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:04 smithi104 ceph-mon[33507]: 6.a scrub ok 2024-09-11T07:28:05.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:04 smithi104 ceph-mon[33507]: 4.d scrub starts 2024-09-11T07:28:05.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:04 smithi104 ceph-mon[33507]: 4.d scrub ok 2024-09-11T07:28:05.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:04 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:28:05.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:04 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:28:05.469 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:28:06.271 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:06 smithi022 ceph-mon[29462]: osdmap e84: 8 total, 8 up, 8 in 2024-09-11T07:28:06.271 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:06 smithi022 ceph-mon[37330]: osdmap e84: 8 total, 8 up, 8 in 2024-09-11T07:28:06.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:06 smithi104 ceph-mon[33507]: osdmap e84: 8 total, 8 up, 8 in 2024-09-11T07:28:07.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:07 smithi022 ceph-mon[37330]: 5.0 deep-scrub starts 2024-09-11T07:28:07.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:07 smithi022 ceph-mon[37330]: 5.0 deep-scrub ok 2024-09-11T07:28:07.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:07 smithi022 ceph-mon[37330]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:28:07.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:07 smithi022 ceph-mon[37330]: 6.14 scrub starts 2024-09-11T07:28:07.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:07 smithi022 ceph-mon[37330]: 6.14 scrub ok 2024-09-11T07:28:07.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:07 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:07.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:07 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:07.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:07 smithi022 ceph-mon[29462]: 5.0 deep-scrub starts 2024-09-11T07:28:07.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:07 smithi022 ceph-mon[29462]: 5.0 deep-scrub ok 2024-09-11T07:28:07.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:07 smithi022 ceph-mon[29462]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:28:07.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:07 smithi022 ceph-mon[29462]: 6.14 scrub starts 2024-09-11T07:28:07.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:07 smithi022 ceph-mon[29462]: 6.14 scrub ok 2024-09-11T07:28:07.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:07 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:07.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:07 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:07.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:07 smithi104 ceph-mon[33507]: 5.0 deep-scrub starts 2024-09-11T07:28:07.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:07 smithi104 ceph-mon[33507]: 5.0 deep-scrub ok 2024-09-11T07:28:07.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:07 smithi104 ceph-mon[33507]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:28:07.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:07 smithi104 ceph-mon[33507]: 6.14 scrub starts 2024-09-11T07:28:07.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:07 smithi104 ceph-mon[33507]: 6.14 scrub ok 2024-09-11T07:28:07.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:07 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:07.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:07 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:08.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:08 smithi104 ceph-mon[33507]: 6.1f scrub starts 2024-09-11T07:28:08.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:08 smithi104 ceph-mon[33507]: 6.1f scrub ok 2024-09-11T07:28:08.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:08 smithi104 ceph-mon[33507]: 5.1d scrub starts 2024-09-11T07:28:08.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:08 smithi104 ceph-mon[33507]: 5.1d scrub ok 2024-09-11T07:28:08.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2024-09-11T07:28:08.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2024-09-11T07:28:08.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-09-11T07:28:08.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2024-09-11T07:28:08.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:28:08.316 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:08 smithi022 ceph-mon[29462]: 6.1f scrub starts 2024-09-11T07:28:08.316 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:08 smithi022 ceph-mon[29462]: 6.1f scrub ok 2024-09-11T07:28:08.316 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:08 smithi022 ceph-mon[29462]: 5.1d scrub starts 2024-09-11T07:28:08.316 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:08 smithi022 ceph-mon[29462]: 5.1d scrub ok 2024-09-11T07:28:08.316 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2024-09-11T07:28:08.316 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2024-09-11T07:28:08.316 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-09-11T07:28:08.317 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2024-09-11T07:28:08.317 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:28:08.317 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:08 smithi022 ceph-mon[37330]: 6.1f scrub starts 2024-09-11T07:28:08.317 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:08 smithi022 ceph-mon[37330]: 6.1f scrub ok 2024-09-11T07:28:08.317 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:08 smithi022 ceph-mon[37330]: 5.1d scrub starts 2024-09-11T07:28:08.317 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:08 smithi022 ceph-mon[37330]: 5.1d scrub ok 2024-09-11T07:28:08.317 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2024-09-11T07:28:08.317 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2024-09-11T07:28:08.317 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-09-11T07:28:08.317 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2024-09-11T07:28:08.317 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:28:08.490 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:28:08.492 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:28:09.280 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:28:08 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:28:09.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:09 smithi104 ceph-mon[33507]: 2.1 deep-scrub starts 2024-09-11T07:28:09.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:09 smithi104 ceph-mon[33507]: 2.1 deep-scrub ok 2024-09-11T07:28:09.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:09 smithi104 ceph-mon[33507]: 4.8 scrub starts 2024-09-11T07:28:09.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:09 smithi104 ceph-mon[33507]: 4.8 scrub ok 2024-09-11T07:28:09.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:09 smithi104 ceph-mon[33507]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 622 B/s rd, 0 op/s 2024-09-11T07:28:09.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:09 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]': finished 2024-09-11T07:28:09.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:09 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]': finished 2024-09-11T07:28:09.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:09 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-09-11T07:28:09.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:09 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]': finished 2024-09-11T07:28:09.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:09 smithi104 ceph-mon[33507]: osdmap e85: 8 total, 8 up, 8 in 2024-09-11T07:28:09.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:09 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:09.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:09 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:09.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:09 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:09.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:09 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:09.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:09 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:09.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:09 smithi104 ceph-mon[33507]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T07:28:09.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:09 smithi104 ceph-mon[33507]: Reconfiguring daemon alertmanager.a on smithi022 2024-09-11T07:28:09.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:09 smithi104 ceph-mon[33507]: 3.1 scrub starts 2024-09-11T07:28:09.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:09 smithi104 ceph-mon[33507]: 3.1 scrub ok 2024-09-11T07:28:09.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[37330]: 2.1 deep-scrub starts 2024-09-11T07:28:09.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[37330]: 2.1 deep-scrub ok 2024-09-11T07:28:09.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[37330]: 4.8 scrub starts 2024-09-11T07:28:09.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[37330]: 4.8 scrub ok 2024-09-11T07:28:09.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[37330]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 622 B/s rd, 0 op/s 2024-09-11T07:28:09.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]': finished 2024-09-11T07:28:09.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]': finished 2024-09-11T07:28:09.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-09-11T07:28:09.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]': finished 2024-09-11T07:28:09.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[37330]: osdmap e85: 8 total, 8 up, 8 in 2024-09-11T07:28:09.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:09.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:09.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:09.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:09.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:09.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[37330]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T07:28:09.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[37330]: Reconfiguring daemon alertmanager.a on smithi022 2024-09-11T07:28:09.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[37330]: 3.1 scrub starts 2024-09-11T07:28:09.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[37330]: 3.1 scrub ok 2024-09-11T07:28:09.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[29462]: 2.1 deep-scrub starts 2024-09-11T07:28:09.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[29462]: 2.1 deep-scrub ok 2024-09-11T07:28:09.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[29462]: 4.8 scrub starts 2024-09-11T07:28:09.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[29462]: 4.8 scrub ok 2024-09-11T07:28:09.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[29462]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 622 B/s rd, 0 op/s 2024-09-11T07:28:09.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]': finished 2024-09-11T07:28:09.420 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]': finished 2024-09-11T07:28:09.420 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-09-11T07:28:09.420 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]': finished 2024-09-11T07:28:09.420 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[29462]: osdmap e85: 8 total, 8 up, 8 in 2024-09-11T07:28:09.420 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:09.420 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:09.420 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:09.420 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:09.420 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:09.420 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[29462]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T07:28:09.420 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[29462]: Reconfiguring daemon alertmanager.a on smithi022 2024-09-11T07:28:09.420 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[29462]: 3.1 scrub starts 2024-09-11T07:28:09.420 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:09 smithi022 ceph-mon[29462]: 3.1 scrub ok 2024-09-11T07:28:10.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:10 smithi104 ceph-mon[33507]: from='client.24662 v1:172.21.15.22:0/1380837853' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:28:10.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:10 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:28:10.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:10 smithi104 ceph-mon[33507]: osdmap e86: 8 total, 8 up, 8 in 2024-09-11T07:28:10.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:10 smithi104 ceph-mon[33507]: 6.18 scrub starts 2024-09-11T07:28:10.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:10 smithi104 ceph-mon[33507]: 6.18 scrub ok 2024-09-11T07:28:10.320 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:10 smithi022 ceph-mon[29462]: from='client.24662 v1:172.21.15.22:0/1380837853' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:28:10.320 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:10 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:28:10.320 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:10 smithi022 ceph-mon[29462]: osdmap e86: 8 total, 8 up, 8 in 2024-09-11T07:28:10.320 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:10 smithi022 ceph-mon[29462]: 6.18 scrub starts 2024-09-11T07:28:10.321 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:10 smithi022 ceph-mon[29462]: 6.18 scrub ok 2024-09-11T07:28:10.321 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:10 smithi022 ceph-mon[37330]: from='client.24662 v1:172.21.15.22:0/1380837853' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:28:10.321 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:10 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:28:10.321 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:10 smithi022 ceph-mon[37330]: osdmap e86: 8 total, 8 up, 8 in 2024-09-11T07:28:10.321 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:10 smithi022 ceph-mon[37330]: 6.18 scrub starts 2024-09-11T07:28:10.321 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:10 smithi022 ceph-mon[37330]: 6.18 scrub ok 2024-09-11T07:28:10.353 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":46,"stamp":"2024-09-11T07:28:08.006800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":939,"num_read_kb":796,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":579,"ondisk_log_size":579,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385388,"kb_used_data":6572,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575828,"statfs":{"total":767960285184,"available":765517647872,"internally_reserved":0,"allocated":6729728,"data_stored":4033769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.061733"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.418797+0000","last_change":"2024-09-11T07:26:37.863505+0000","last_active":"2024-09-11T07:27:40.418797+0000","last_peered":"2024-09-11T07:27:40.418797+0000","last_clean":"2024-09-11T07:27:40.418797+0000","last_became_active":"2024-09-11T07:26:37.863165+0000","last_became_peered":"2024-09-11T07:26:37.863165+0000","last_unstale":"2024-09-11T07:27:40.418797+0000","last_undegraded":"2024-09-11T07:27:40.418797+0000","last_fullsized":"2024-09-11T07:27:40.418797+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:05.169084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:51.198029+0000","last_change":"2024-09-11T07:27:51.197966+0000","last_active":"2024-09-11T07:27:51.198029+0000","last_peered":"2024-09-11T07:27:51.198029+0000","last_clean":"2024-09-11T07:27:51.198029+0000","last_became_active":"2024-09-11T07:26:31.421005+0000","last_became_peered":"2024-09-11T07:26:31.421005+0000","last_unstale":"2024-09-11T07:27:51.198029+0000","last_undegraded":"2024-09-11T07:27:51.198029+0000","last_fullsized":"2024-09-11T07:27:51.198029+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:51.197903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:51.197903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:00.681709+0000","scrub_duration":0.001600113,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"82'22","reported_seq":75,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:50.756846+0000","last_change":"2024-09-11T07:27:50.756777+0000","last_active":"2024-09-11T07:27:50.756846+0000","last_peered":"2024-09-11T07:27:50.756846+0000","last_clean":"2024-09-11T07:27:50.756846+0000","last_became_active":"2024-09-11T07:26:33.544627+0000","last_became_peered":"2024-09-11T07:26:33.544627+0000","last_unstale":"2024-09-11T07:27:50.756846+0000","last_undegraded":"2024-09-11T07:27:50.756846+0000","last_fullsized":"2024-09-11T07:27:50.756846+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.756707+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.257406+0000","scrub_duration":0.0049180669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.421070+0000","last_change":"2024-09-11T07:26:35.557117+0000","last_active":"2024-09-11T07:27:40.421070+0000","last_peered":"2024-09-11T07:27:40.421070+0000","last_clean":"2024-09-11T07:27:40.421070+0000","last_became_active":"2024-09-11T07:26:35.556296+0000","last_became_peered":"2024-09-11T07:26:35.556296+0000","last_unstale":"2024-09-11T07:27:40.421070+0000","last_undegraded":"2024-09-11T07:27:40.421070+0000","last_fullsized":"2024-09-11T07:27:40.421070+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.420339+0000","last_change":"2024-09-11T07:27:36.776948+0000","last_active":"2024-09-11T07:27:40.420339+0000","last_peered":"2024-09-11T07:27:40.420339+0000","last_clean":"2024-09-11T07:27:40.420339+0000","last_became_active":"2024-09-11T07:26:31.420178+0000","last_became_peered":"2024-09-11T07:26:31.420178+0000","last_unstale":"2024-09-11T07:27:40.420339+0000","last_undegraded":"2024-09-11T07:27:40.420339+0000","last_fullsized":"2024-09-11T07:27:40.420339+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:36.776883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:36.776883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:49.961787+0000","scrub_duration":0.0017449290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"64'9","reported_seq":49,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.421133+0000","last_change":"2024-09-11T07:26:33.541403+0000","last_active":"2024-09-11T07:27:40.421133+0000","last_peered":"2024-09-11T07:27:40.421133+0000","last_clean":"2024-09-11T07:27:40.421133+0000","last_became_active":"2024-09-11T07:26:33.540732+0000","last_became_peered":"2024-09-11T07:26:33.540732+0000","last_unstale":"2024-09-11T07:27:40.421133+0000","last_undegraded":"2024-09-11T07:27:40.421133+0000","last_fullsized":"2024-09-11T07:27:40.421133+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:50.648509+0000","last_change":"2024-09-11T07:27:50.648473+0000","last_active":"2024-09-11T07:27:50.648509+0000","last_peered":"2024-09-11T07:27:50.648509+0000","last_clean":"2024-09-11T07:27:50.648509+0000","last_became_active":"2024-09-11T07:26:35.561096+0000","last_became_peered":"2024-09-11T07:26:35.561096+0000","last_unstale":"2024-09-11T07:27:50.648509+0000","last_undegraded":"2024-09-11T07:27:50.648509+0000","last_fullsized":"2024-09-11T07:27:50.648509+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.648420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:05.663592+0000","scrub_duration":0.001391446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.421248+0000","last_change":"2024-09-11T07:26:37.567195+0000","last_active":"2024-09-11T07:27:40.421248+0000","last_peered":"2024-09-11T07:27:40.421248+0000","last_clean":"2024-09-11T07:27:40.421248+0000","last_became_active":"2024-09-11T07:26:37.566910+0000","last_became_peered":"2024-09-11T07:26:37.566910+0000","last_unstale":"2024-09-11T07:27:40.421248+0000","last_undegraded":"2024-09-11T07:27:40.421248+0000","last_fullsized":"2024-09-11T07:27:40.421248+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.442378+0000","last_change":"2024-09-11T07:26:31.423041+0000","last_active":"2024-09-11T07:27:40.442378+0000","last_peered":"2024-09-11T07:27:40.442378+0000","last_clean":"2024-09-11T07:27:40.442378+0000","last_became_active":"2024-09-11T07:26:31.422772+0000","last_became_peered":"2024-09-11T07:26:31.422772+0000","last_unstale":"2024-09-11T07:27:40.442378+0000","last_undegraded":"2024-09-11T07:27:40.442378+0000","last_fullsized":"2024-09-11T07:27:40.442378+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"82'7","reported_seq":55,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:52.097243+0000","last_change":"2024-09-11T07:27:52.097170+0000","last_active":"2024-09-11T07:27:52.097243+0000","last_peered":"2024-09-11T07:27:52.097243+0000","last_clean":"2024-09-11T07:27:52.097243+0000","last_became_active":"2024-09-11T07:26:33.540351+0000","last_became_peered":"2024-09-11T07:26:33.540351+0000","last_unstale":"2024-09-11T07:27:52.097243+0000","last_undegraded":"2024-09-11T07:27:52.097243+0000","last_fullsized":"2024-09-11T07:27:52.097243+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'7","last_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_deep_scrub":"82'7","last_deep_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_clean_scrub_stamp":"2024-09-11T07:27:52.097071+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:53.905741+0000","scrub_duration":0.0025786670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.441364+0000","last_change":"2024-09-11T07:26:35.563858+0000","last_active":"2024-09-11T07:27:40.441364+0000","last_peered":"2024-09-11T07:27:40.441364+0000","last_clean":"2024-09-11T07:27:40.441364+0000","last_became_active":"2024-09-11T07:26:35.563536+0000","last_became_peered":"2024-09-11T07:26:35.563536+0000","last_unstale":"2024-09-11T07:27:40.441364+0000","last_undegraded":"2024-09-11T07:27:40.441364+0000","last_fullsized":"2024-09-11T07:27:40.441364+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:01.379998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.441211+0000","last_change":"2024-09-11T07:26:37.562008+0000","last_active":"2024-09-11T07:27:40.441211+0000","last_peered":"2024-09-11T07:27:40.441211+0000","last_clean":"2024-09-11T07:27:40.441211+0000","last_became_active":"2024-09-11T07:26:37.561758+0000","last_became_peered":"2024-09-11T07:26:37.561758+0000","last_unstale":"2024-09-11T07:27:40.441211+0000","last_undegraded":"2024-09-11T07:27:40.441211+0000","last_fullsized":"2024-09-11T07:27:40.441211+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:04.521146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:52.058062+0000","last_change":"2024-09-11T07:27:52.058013+0000","last_active":"2024-09-11T07:27:52.058062+0000","last_peered":"2024-09-11T07:27:52.058062+0000","last_clean":"2024-09-11T07:27:52.058062+0000","last_became_active":"2024-09-11T07:26:31.420969+0000","last_became_peered":"2024-09-11T07:26:31.420969+0000","last_unstale":"2024-09-11T07:27:52.058062+0000","last_undegraded":"2024-09-11T07:27:52.058062+0000","last_fullsized":"2024-09-11T07:27:52.058062+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_clean_scrub_stamp":"2024-09-11T07:27:52.057959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:12.902027+0000","scrub_duration":0.0014644840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"64'9","reported_seq":59,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:47.108765+0000","last_change":"2024-09-11T07:27:47.108704+0000","last_active":"2024-09-11T07:27:47.108765+0000","last_peered":"2024-09-11T07:27:47.108765+0000","last_clean":"2024-09-11T07:27:47.108765+0000","last_became_active":"2024-09-11T07:26:33.544386+0000","last_became_peered":"2024-09-11T07:26:33.544386+0000","last_unstale":"2024-09-11T07:27:47.108765+0000","last_undegraded":"2024-09-11T07:27:47.108765+0000","last_fullsized":"2024-09-11T07:27:47.108765+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:27:47.108645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.108645+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:08.302050+0000","scrub_duration":0.0024650560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:57.092083+0000","last_change":"2024-09-11T07:27:57.092029+0000","last_active":"2024-09-11T07:27:57.092083+0000","last_peered":"2024-09-11T07:27:57.092083+0000","last_clean":"2024-09-11T07:27:57.092083+0000","last_became_active":"2024-09-11T07:26:35.569630+0000","last_became_peered":"2024-09-11T07:26:35.569630+0000","last_unstale":"2024-09-11T07:27:57.092083+0000","last_undegraded":"2024-09-11T07:27:57.092083+0000","last_fullsized":"2024-09-11T07:27:57.092083+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:57.091958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.091958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:57.303120+0000","scrub_duration":0.001785872,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.441419+0000","last_change":"2024-09-11T07:26:37.565826+0000","last_active":"2024-09-11T07:27:40.441419+0000","last_peered":"2024-09-11T07:27:40.441419+0000","last_clean":"2024-09-11T07:27:40.441419+0000","last_became_active":"2024-09-11T07:26:37.565475+0000","last_became_peered":"2024-09-11T07:26:37.565475+0000","last_unstale":"2024-09-11T07:27:40.441419+0000","last_undegraded":"2024-09-11T07:27:40.441419+0000","last_fullsized":"2024-09-11T07:27:40.441419+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":61,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.421878+0000","last_change":"2024-09-11T07:26:33.541535+0000","last_active":"2024-09-11T07:27:40.421878+0000","last_peered":"2024-09-11T07:27:40.421878+0000","last_clean":"2024-09-11T07:27:40.421878+0000","last_became_active":"2024-09-11T07:26:33.540575+0000","last_became_peered":"2024-09-11T07:26:33.540575+0000","last_unstale":"2024-09-11T07:27:40.421878+0000","last_undegraded":"2024-09-11T07:27:40.421878+0000","last_fullsized":"2024-09-11T07:27:40.421878+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.442813+0000","last_change":"2024-09-11T07:26:31.420109+0000","last_active":"2024-09-11T07:27:40.442813+0000","last_peered":"2024-09-11T07:27:40.442813+0000","last_clean":"2024-09-11T07:27:40.442813+0000","last_became_active":"2024-09-11T07:26:31.419488+0000","last_became_peered":"2024-09-11T07:26:31.419488+0000","last_unstale":"2024-09-11T07:27:40.442813+0000","last_undegraded":"2024-09-11T07:27:40.442813+0000","last_fullsized":"2024-09-11T07:27:40.442813+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.441982+0000","last_change":"2024-09-11T07:26:35.559684+0000","last_active":"2024-09-11T07:27:40.441982+0000","last_peered":"2024-09-11T07:27:40.441982+0000","last_clean":"2024-09-11T07:27:40.441982+0000","last_became_active":"2024-09-11T07:26:35.557989+0000","last_became_peered":"2024-09-11T07:26:35.557989+0000","last_unstale":"2024-09-11T07:27:40.441982+0000","last_undegraded":"2024-09-11T07:27:40.441982+0000","last_fullsized":"2024-09-11T07:27:40.441982+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:57.274664+0000","last_change":"2024-09-11T07:27:57.274611+0000","last_active":"2024-09-11T07:27:57.274664+0000","last_peered":"2024-09-11T07:27:57.274664+0000","last_clean":"2024-09-11T07:27:57.274664+0000","last_became_active":"2024-09-11T07:26:37.564554+0000","last_became_peered":"2024-09-11T07:26:37.564554+0000","last_unstale":"2024-09-11T07:27:57.274664+0000","last_undegraded":"2024-09-11T07:27:57.274664+0000","last_fullsized":"2024-09-11T07:27:57.274664+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:57.274554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.274554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.058396+0000","scrub_duration":0.001419584,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.421639+0000","last_change":"2024-09-11T07:26:33.536333+0000","last_active":"2024-09-11T07:27:40.421639+0000","last_peered":"2024-09-11T07:27:40.421639+0000","last_clean":"2024-09-11T07:27:40.421639+0000","last_became_active":"2024-09-11T07:26:33.535411+0000","last_became_peered":"2024-09-11T07:26:33.535411+0000","last_unstale":"2024-09-11T07:27:40.421639+0000","last_undegraded":"2024-09-11T07:27:40.421639+0000","last_fullsized":"2024-09-11T07:27:40.421639+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:00.222835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.441814+0000","last_change":"2024-09-11T07:26:31.419259+0000","last_active":"2024-09-11T07:27:40.441814+0000","last_peered":"2024-09-11T07:27:40.441814+0000","last_clean":"2024-09-11T07:27:40.441814+0000","last_became_active":"2024-09-11T07:26:31.418178+0000","last_became_peered":"2024-09-11T07:26:31.418178+0000","last_unstale":"2024-09-11T07:27:40.441814+0000","last_undegraded":"2024-09-11T07:27:40.441814+0000","last_fullsized":"2024-09-11T07:27:40.441814+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"64'8","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:59.499212+0000","last_change":"2024-09-11T07:27:47.127485+0000","last_active":"2024-09-11T07:27:59.499212+0000","last_peered":"2024-09-11T07:27:59.499212+0000","last_clean":"2024-09-11T07:27:59.499212+0000","last_became_active":"2024-09-11T07:26:35.564531+0000","last_became_peered":"2024-09-11T07:26:35.564531+0000","last_unstale":"2024-09-11T07:27:59.499212+0000","last_undegraded":"2024-09-11T07:27:59.499212+0000","last_fullsized":"2024-09-11T07:27:59.499212+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:27:47.127428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.127428+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:58.343731+0000","scrub_duration":0.0014091329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.421771+0000","last_change":"2024-09-11T07:26:37.567193+0000","last_active":"2024-09-11T07:27:40.421771+0000","last_peered":"2024-09-11T07:27:40.421771+0000","last_clean":"2024-09-11T07:27:40.421771+0000","last_became_active":"2024-09-11T07:26:37.566919+0000","last_became_peered":"2024-09-11T07:26:37.566919+0000","last_unstale":"2024-09-11T07:27:40.421771+0000","last_undegraded":"2024-09-11T07:27:40.421771+0000","last_fullsized":"2024-09-11T07:27:40.421771+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:05.045246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"83'22","reported_seq":75,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T07:28:01.719219+0000","last_change":"2024-09-11T07:28:01.719185+0000","last_active":"2024-09-11T07:28:01.719219+0000","last_peered":"2024-09-11T07:28:01.719219+0000","last_clean":"2024-09-11T07:28:01.719219+0000","last_became_active":"2024-09-11T07:26:33.549733+0000","last_became_peered":"2024-09-11T07:26:33.549733+0000","last_unstale":"2024-09-11T07:28:01.719219+0000","last_undegraded":"2024-09-11T07:28:01.719219+0000","last_fullsized":"2024-09-11T07:28:01.719219+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'22","last_scrub_stamp":"2024-09-11T07:28:01.719128+0000","last_deep_scrub":"83'22","last_deep_scrub_stamp":"2024-09-11T07:28:01.719128+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.719128+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:27.431086+0000","scrub_duration":0.0047817240000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.441252+0000","last_change":"2024-09-11T07:27:35.589317+0000","last_active":"2024-09-11T07:27:40.441252+0000","last_peered":"2024-09-11T07:27:40.441252+0000","last_clean":"2024-09-11T07:27:40.441252+0000","last_became_active":"2024-09-11T07:26:31.418478+0000","last_became_peered":"2024-09-11T07:26:31.418478+0000","last_unstale":"2024-09-11T07:27:40.441252+0000","last_undegraded":"2024-09-11T07:27:40.441252+0000","last_fullsized":"2024-09-11T07:27:40.441252+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:35.589251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:35.589251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:01.839680+0000","scrub_duration":0.0014929380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:59.499189+0000","last_change":"2024-09-11T07:27:56.218355+0000","last_active":"2024-09-11T07:27:59.499189+0000","last_peered":"2024-09-11T07:27:59.499189+0000","last_clean":"2024-09-11T07:27:59.499189+0000","last_became_active":"2024-09-11T07:26:35.560406+0000","last_became_peered":"2024-09-11T07:26:35.560406+0000","last_unstale":"2024-09-11T07:27:59.499189+0000","last_undegraded":"2024-09-11T07:27:59.499189+0000","last_fullsized":"2024-09-11T07:27:59.499189+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:27:56.218301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:56.218301+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:21.374837+0000","scrub_duration":0.001687413,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.442025+0000","last_change":"2024-09-11T07:26:37.568289+0000","last_active":"2024-09-11T07:27:40.442025+0000","last_peered":"2024-09-11T07:27:40.442025+0000","last_clean":"2024-09-11T07:27:40.442025+0000","last_became_active":"2024-09-11T07:26:37.568005+0000","last_became_peered":"2024-09-11T07:26:37.568005+0000","last_unstale":"2024-09-11T07:27:40.442025+0000","last_undegraded":"2024-09-11T07:27:40.442025+0000","last_fullsized":"2024-09-11T07:27:40.442025+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:00.612703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"64'11","reported_seq":62,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:58.158538+0000","last_change":"2024-09-11T07:27:58.158485+0000","last_active":"2024-09-11T07:27:58.158538+0000","last_peered":"2024-09-11T07:27:58.158538+0000","last_clean":"2024-09-11T07:27:58.158538+0000","last_became_active":"2024-09-11T07:26:33.540777+0000","last_became_peered":"2024-09-11T07:26:33.540777+0000","last_unstale":"2024-09-11T07:27:58.158538+0000","last_undegraded":"2024-09-11T07:27:58.158538+0000","last_fullsized":"2024-09-11T07:27:58.158538+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:27:58.158420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.158420+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.832014+0000","scrub_duration":0.0021737459999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.441625+0000","last_change":"2024-09-11T07:27:34.932151+0000","last_active":"2024-09-11T07:27:40.441625+0000","last_peered":"2024-09-11T07:27:40.441625+0000","last_clean":"2024-09-11T07:27:40.441625+0000","last_became_active":"2024-09-11T07:26:31.418238+0000","last_became_peered":"2024-09-11T07:26:31.418238+0000","last_unstale":"2024-09-11T07:27:40.441625+0000","last_undegraded":"2024-09-11T07:27:40.441625+0000","last_fullsized":"2024-09-11T07:27:40.441625+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:34.932092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:34.932092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:46.904196+0000","scrub_duration":0.002710259,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:28:00.005216+0000","last_change":"2024-09-11T07:28:00.005119+0000","last_active":"2024-09-11T07:28:00.005216+0000","last_peered":"2024-09-11T07:28:00.005216+0000","last_clean":"2024-09-11T07:28:00.005216+0000","last_became_active":"2024-09-11T07:26:35.569081+0000","last_became_peered":"2024-09-11T07:26:35.569081+0000","last_unstale":"2024-09-11T07:28:00.005216+0000","last_undegraded":"2024-09-11T07:28:00.005216+0000","last_fullsized":"2024-09-11T07:28:00.005216+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:00.005031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.005031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:10.748420+0000","scrub_duration":0.001403533,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:46.752337+0000","last_change":"2024-09-11T07:27:46.752272+0000","last_active":"2024-09-11T07:27:46.752337+0000","last_peered":"2024-09-11T07:27:46.752337+0000","last_clean":"2024-09-11T07:27:46.752337+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:27:46.752337+0000","last_undegraded":"2024-09-11T07:27:46.752337+0000","last_fullsized":"2024-09-11T07:27:46.752337+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:46.752181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:46.752181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.072860+0000","scrub_duration":0.001508782,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"64'14","reported_seq":54,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.421327+0000","last_change":"2024-09-11T07:26:33.538152+0000","last_active":"2024-09-11T07:27:40.421327+0000","last_peered":"2024-09-11T07:27:40.421327+0000","last_clean":"2024-09-11T07:27:40.421327+0000","last_became_active":"2024-09-11T07:26:33.537850+0000","last_became_peered":"2024-09-11T07:26:33.537850+0000","last_unstale":"2024-09-11T07:27:40.421327+0000","last_undegraded":"2024-09-11T07:27:40.421327+0000","last_fullsized":"2024-09-11T07:27:40.421327+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.442543+0000","last_change":"2024-09-11T07:27:39.014916+0000","last_active":"2024-09-11T07:27:40.442543+0000","last_peered":"2024-09-11T07:27:40.442543+0000","last_clean":"2024-09-11T07:27:40.442543+0000","last_became_active":"2024-09-11T07:26:31.423905+0000","last_became_peered":"2024-09-11T07:26:31.423905+0000","last_unstale":"2024-09-11T07:27:40.442543+0000","last_undegraded":"2024-09-11T07:27:40.442543+0000","last_fullsized":"2024-09-11T07:27:40.442543+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:39.014840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:39.014840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:40.097304+0000","scrub_duration":0.0032789239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"64'8","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:59.499219+0000","last_change":"2024-09-11T07:26:35.563242+0000","last_active":"2024-09-11T07:27:59.499219+0000","last_peered":"2024-09-11T07:27:59.499219+0000","last_clean":"2024-09-11T07:27:59.499219+0000","last_became_active":"2024-09-11T07:26:35.562997+0000","last_became_peered":"2024-09-11T07:26:35.562997+0000","last_unstale":"2024-09-11T07:27:59.499219+0000","last_undegraded":"2024-09-11T07:27:59.499219+0000","last_fullsized":"2024-09-11T07:27:59.499219+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.442411+0000","last_change":"2024-09-11T07:26:37.862564+0000","last_active":"2024-09-11T07:27:40.442411+0000","last_peered":"2024-09-11T07:27:40.442411+0000","last_clean":"2024-09-11T07:27:40.442411+0000","last_became_active":"2024-09-11T07:26:37.862246+0000","last_became_peered":"2024-09-11T07:26:37.862246+0000","last_unstale":"2024-09-11T07:27:40.442411+0000","last_undegraded":"2024-09-11T07:27:40.442411+0000","last_fullsized":"2024-09-11T07:27:40.442411+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:04.346579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"64'18","reported_seq":65,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.418437+0000","last_change":"2024-09-11T07:26:33.546711+0000","last_active":"2024-09-11T07:27:40.418437+0000","last_peered":"2024-09-11T07:27:40.418437+0000","last_clean":"2024-09-11T07:27:40.418437+0000","last_became_active":"2024-09-11T07:26:33.546188+0000","last_became_peered":"2024-09-11T07:26:33.546188+0000","last_unstale":"2024-09-11T07:27:40.418437+0000","last_undegraded":"2024-09-11T07:27:40.418437+0000","last_fullsized":"2024-09-11T07:27:40.418437+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:02.137900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.443151+0000","last_change":"2024-09-11T07:26:31.421802+0000","last_active":"2024-09-11T07:27:40.443151+0000","last_peered":"2024-09-11T07:27:40.443151+0000","last_clean":"2024-09-11T07:27:40.443151+0000","last_became_active":"2024-09-11T07:26:31.421500+0000","last_became_peered":"2024-09-11T07:26:31.421500+0000","last_unstale":"2024-09-11T07:27:40.443151+0000","last_undegraded":"2024-09-11T07:27:40.443151+0000","last_fullsized":"2024-09-11T07:27:40.443151+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.421216+0000","last_change":"2024-09-11T07:26:35.562816+0000","last_active":"2024-09-11T07:27:40.421216+0000","last_peered":"2024-09-11T07:27:40.421216+0000","last_clean":"2024-09-11T07:27:40.421216+0000","last_became_active":"2024-09-11T07:26:35.561284+0000","last_became_peered":"2024-09-11T07:26:35.561284+0000","last_unstale":"2024-09-11T07:27:40.421216+0000","last_undegraded":"2024-09-11T07:27:40.421216+0000","last_fullsized":"2024-09-11T07:27:40.421216+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:01.396434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.421441+0000","last_change":"2024-09-11T07:26:37.565014+0000","last_active":"2024-09-11T07:27:40.421441+0000","last_peered":"2024-09-11T07:27:40.421441+0000","last_clean":"2024-09-11T07:27:40.421441+0000","last_became_active":"2024-09-11T07:26:37.564616+0000","last_became_peered":"2024-09-11T07:26:37.564616+0000","last_unstale":"2024-09-11T07:27:40.421441+0000","last_undegraded":"2024-09-11T07:27:40.421441+0000","last_fullsized":"2024-09-11T07:27:40.421441+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:04.390896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":69,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.442249+0000","last_change":"2024-09-11T07:26:33.541012+0000","last_active":"2024-09-11T07:27:40.442249+0000","last_peered":"2024-09-11T07:27:40.442249+0000","last_clean":"2024-09-11T07:27:40.442249+0000","last_became_active":"2024-09-11T07:26:33.540639+0000","last_became_peered":"2024-09-11T07:26:33.540639+0000","last_unstale":"2024-09-11T07:27:40.442249+0000","last_undegraded":"2024-09-11T07:27:40.442249+0000","last_fullsized":"2024-09-11T07:27:40.442249+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.442789+0000","last_change":"2024-09-11T07:26:31.419925+0000","last_active":"2024-09-11T07:27:40.442789+0000","last_peered":"2024-09-11T07:27:40.442789+0000","last_clean":"2024-09-11T07:27:40.442789+0000","last_became_active":"2024-09-11T07:26:31.419185+0000","last_became_peered":"2024-09-11T07:26:31.419185+0000","last_unstale":"2024-09-11T07:27:40.442789+0000","last_undegraded":"2024-09-11T07:27:40.442789+0000","last_fullsized":"2024-09-11T07:27:40.442789+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:41.000519+0000","last_change":"2024-09-11T07:27:41.000469+0000","last_active":"2024-09-11T07:27:41.000519+0000","last_peered":"2024-09-11T07:27:41.000519+0000","last_clean":"2024-09-11T07:27:41.000519+0000","last_became_active":"2024-09-11T07:26:35.565359+0000","last_became_peered":"2024-09-11T07:26:35.565359+0000","last_unstale":"2024-09-11T07:27:41.000519+0000","last_undegraded":"2024-09-11T07:27:41.000519+0000","last_fullsized":"2024-09-11T07:27:41.000519+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:41.000401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:41.000401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.049825+0000","scrub_duration":0.001655164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.441098+0000","last_change":"2024-09-11T07:26:37.861736+0000","last_active":"2024-09-11T07:27:40.441098+0000","last_peered":"2024-09-11T07:27:40.441098+0000","last_clean":"2024-09-11T07:27:40.441098+0000","last_became_active":"2024-09-11T07:26:37.861279+0000","last_became_peered":"2024-09-11T07:26:37.861279+0000","last_unstale":"2024-09-11T07:27:40.441098+0000","last_undegraded":"2024-09-11T07:27:40.441098+0000","last_fullsized":"2024-09-11T07:27:40.441098+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":58,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:51.642581+0000","last_change":"2024-09-11T07:27:51.642529+0000","last_active":"2024-09-11T07:27:51.642581+0000","last_peered":"2024-09-11T07:27:51.642581+0000","last_clean":"2024-09-11T07:27:51.642581+0000","last_became_active":"2024-09-11T07:26:33.543063+0000","last_became_peered":"2024-09-11T07:26:33.543063+0000","last_unstale":"2024-09-11T07:27:51.642581+0000","last_undegraded":"2024-09-11T07:27:51.642581+0000","last_fullsized":"2024-09-11T07:27:51.642581+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:27:51.642464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:51.642464+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:13.146284+0000","scrub_duration":0.0020211439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":52,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:41.618426+0000","last_change":"2024-09-11T07:27:41.618362+0000","last_active":"2024-09-11T07:27:41.618426+0000","last_peered":"2024-09-11T07:27:41.618426+0000","last_clean":"2024-09-11T07:27:41.618426+0000","last_became_active":"2024-09-11T07:26:31.422110+0000","last_became_peered":"2024-09-11T07:26:31.422110+0000","last_unstale":"2024-09-11T07:27:41.618426+0000","last_undegraded":"2024-09-11T07:27:41.618426+0000","last_fullsized":"2024-09-11T07:27:41.618426+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:27:41.618296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:41.618296+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:57.404591+0000","scrub_duration":0.0020650450000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":67,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T07:28:03.183095+0000","last_change":"2024-09-11T07:28:03.183056+0000","last_active":"2024-09-11T07:28:03.183095+0000","last_peered":"2024-09-11T07:28:03.183095+0000","last_clean":"2024-09-11T07:28:03.183095+0000","last_became_active":"2024-09-11T07:26:35.560422+0000","last_became_peered":"2024-09-11T07:26:35.560422+0000","last_unstale":"2024-09-11T07:28:03.183095+0000","last_undegraded":"2024-09-11T07:28:03.183095+0000","last_fullsized":"2024-09-11T07:28:03.183095+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:28:03.182993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:03.182993+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:26.896461+0000","scrub_duration":0.0018949889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:54.148647+0000","last_change":"2024-09-11T07:27:54.148594+0000","last_active":"2024-09-11T07:27:54.148647+0000","last_peered":"2024-09-11T07:27:54.148647+0000","last_clean":"2024-09-11T07:27:54.148647+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:27:54.148647+0000","last_undegraded":"2024-09-11T07:27:54.148647+0000","last_fullsized":"2024-09-11T07:27:54.148647+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.148534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:02.854448+0000","scrub_duration":0.0013536620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"64'28","reported_seq":99,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:59.499212+0000","last_change":"2024-09-11T07:26:33.549518+0000","last_active":"2024-09-11T07:27:59.499212+0000","last_peered":"2024-09-11T07:27:59.499212+0000","last_clean":"2024-09-11T07:27:59.499212+0000","last_became_active":"2024-09-11T07:26:33.549234+0000","last_became_peered":"2024-09-11T07:26:33.549234+0000","last_unstale":"2024-09-11T07:27:59.499212+0000","last_undegraded":"2024-09-11T07:27:59.499212+0000","last_fullsized":"2024-09-11T07:27:59.499212+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.442198+0000","last_change":"2024-09-11T07:26:31.424898+0000","last_active":"2024-09-11T07:27:40.442198+0000","last_peered":"2024-09-11T07:27:40.442198+0000","last_clean":"2024-09-11T07:27:40.442198+0000","last_became_active":"2024-09-11T07:26:31.424618+0000","last_became_peered":"2024-09-11T07:26:31.424618+0000","last_unstale":"2024-09-11T07:27:40.442198+0000","last_undegraded":"2024-09-11T07:27:40.442198+0000","last_fullsized":"2024-09-11T07:27:40.442198+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.441594+0000","last_change":"2024-09-11T07:26:35.560677+0000","last_active":"2024-09-11T07:27:40.441594+0000","last_peered":"2024-09-11T07:27:40.441594+0000","last_clean":"2024-09-11T07:27:40.441594+0000","last_became_active":"2024-09-11T07:26:35.560363+0000","last_became_peered":"2024-09-11T07:26:35.560363+0000","last_unstale":"2024-09-11T07:27:40.441594+0000","last_undegraded":"2024-09-11T07:27:40.441594+0000","last_fullsized":"2024-09-11T07:27:40.441594+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T07:28:00.686666+0000","last_change":"2024-09-11T07:28:00.686634+0000","last_active":"2024-09-11T07:28:00.686666+0000","last_peered":"2024-09-11T07:28:00.686666+0000","last_clean":"2024-09-11T07:28:00.686666+0000","last_became_active":"2024-09-11T07:26:37.562452+0000","last_became_peered":"2024-09-11T07:26:37.562452+0000","last_unstale":"2024-09-11T07:28:00.686666+0000","last_undegraded":"2024-09-11T07:28:00.686666+0000","last_fullsized":"2024-09-11T07:28:00.686666+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:00.686571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.686571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.798861+0000","scrub_duration":0.0016714309999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"64'13","reported_seq":60,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.443189+0000","last_change":"2024-09-11T07:26:33.547483+0000","last_active":"2024-09-11T07:27:40.443189+0000","last_peered":"2024-09-11T07:27:40.443189+0000","last_clean":"2024-09-11T07:27:40.443189+0000","last_became_active":"2024-09-11T07:26:33.546126+0000","last_became_peered":"2024-09-11T07:26:33.546126+0000","last_unstale":"2024-09-11T07:27:40.443189+0000","last_undegraded":"2024-09-11T07:27:40.443189+0000","last_fullsized":"2024-09-11T07:27:40.443189+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":139,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T07:28:02.729962+0000","last_change":"2024-09-11T07:27:59.678930+0000","last_active":"2024-09-11T07:28:02.729962+0000","last_peered":"2024-09-11T07:28:02.729962+0000","last_clean":"2024-09-11T07:28:02.729962+0000","last_became_active":"2024-09-11T07:26:31.421276+0000","last_became_peered":"2024-09-11T07:26:31.421276+0000","last_unstale":"2024-09-11T07:28:02.729962+0000","last_undegraded":"2024-09-11T07:28:02.729962+0000","last_fullsized":"2024-09-11T07:28:02.729962+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'5","last_scrub_stamp":"2024-09-11T07:27:59.678865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:59.678865+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.114929+0000","scrub_duration":0.001792486,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":86,"num_read_kb":81,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.418853+0000","last_change":"2024-09-11T07:26:31.413634+0000","last_active":"2024-09-11T07:27:40.418853+0000","last_peered":"2024-09-11T07:27:40.418853+0000","last_clean":"2024-09-11T07:27:40.418853+0000","last_became_active":"2024-09-11T07:26:31.413354+0000","last_became_peered":"2024-09-11T07:26:31.413354+0000","last_unstale":"2024-09-11T07:27:40.418853+0000","last_undegraded":"2024-09-11T07:27:40.418853+0000","last_fullsized":"2024-09-11T07:27:40.418853+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:42.141195+0000","last_change":"2024-09-11T07:27:42.141156+0000","last_active":"2024-09-11T07:27:42.141195+0000","last_peered":"2024-09-11T07:27:42.141195+0000","last_clean":"2024-09-11T07:27:42.141195+0000","last_became_active":"2024-09-11T07:26:35.565849+0000","last_became_peered":"2024-09-11T07:26:35.565849+0000","last_unstale":"2024-09-11T07:27:42.141195+0000","last_undegraded":"2024-09-11T07:27:42.141195+0000","last_fullsized":"2024-09-11T07:27:42.141195+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.141100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.141100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.707826+0000","scrub_duration":0.001495672,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:55.018213+0000","last_change":"2024-09-11T07:27:55.018180+0000","last_active":"2024-09-11T07:27:55.018213+0000","last_peered":"2024-09-11T07:27:55.018213+0000","last_clean":"2024-09-11T07:27:55.018213+0000","last_became_active":"2024-09-11T07:26:37.861292+0000","last_became_peered":"2024-09-11T07:26:37.861292+0000","last_unstale":"2024-09-11T07:27:55.018213+0000","last_undegraded":"2024-09-11T07:27:55.018213+0000","last_fullsized":"2024-09-11T07:27:55.018213+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:55.018125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:55.018125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.234213+0000","scrub_duration":0.0013534339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":61,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.442515+0000","last_change":"2024-09-11T07:27:34.213946+0000","last_active":"2024-09-11T07:27:40.442515+0000","last_peered":"2024-09-11T07:27:40.442515+0000","last_clean":"2024-09-11T07:27:40.442515+0000","last_became_active":"2024-09-11T07:26:33.548903+0000","last_became_peered":"2024-09-11T07:26:33.548903+0000","last_unstale":"2024-09-11T07:27:40.442515+0000","last_undegraded":"2024-09-11T07:27:40.442515+0000","last_fullsized":"2024-09-11T07:27:40.442515+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-09-11T07:27:34.213870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:34.213870+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:50.149624+0000","scrub_duration":0.0026667679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.441889+0000","last_change":"2024-09-11T07:26:35.551577+0000","last_active":"2024-09-11T07:27:40.441889+0000","last_peered":"2024-09-11T07:27:40.441889+0000","last_clean":"2024-09-11T07:27:40.441889+0000","last_became_active":"2024-09-11T07:26:31.420320+0000","last_became_peered":"2024-09-11T07:26:31.420320+0000","last_unstale":"2024-09-11T07:27:40.441889+0000","last_undegraded":"2024-09-11T07:27:40.441889+0000","last_fullsized":"2024-09-11T07:27:40.441889+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.441696+0000","last_change":"2024-09-11T07:26:31.418190+0000","last_active":"2024-09-11T07:27:40.441696+0000","last_peered":"2024-09-11T07:27:40.441696+0000","last_clean":"2024-09-11T07:27:40.441696+0000","last_became_active":"2024-09-11T07:26:31.417455+0000","last_became_peered":"2024-09-11T07:26:31.417455+0000","last_unstale":"2024-09-11T07:27:40.441696+0000","last_undegraded":"2024-09-11T07:27:40.441696+0000","last_fullsized":"2024-09-11T07:27:40.441696+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.443005+0000","last_change":"2024-09-11T07:26:35.564511+0000","last_active":"2024-09-11T07:27:40.443005+0000","last_peered":"2024-09-11T07:27:40.443005+0000","last_clean":"2024-09-11T07:27:40.443005+0000","last_became_active":"2024-09-11T07:26:35.564178+0000","last_became_peered":"2024-09-11T07:26:35.564178+0000","last_unstale":"2024-09-11T07:27:40.443005+0000","last_undegraded":"2024-09-11T07:27:40.443005+0000","last_fullsized":"2024-09-11T07:27:40.443005+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:01.503548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:54.621982+0000","last_change":"2024-09-11T07:27:54.621934+0000","last_active":"2024-09-11T07:27:54.621982+0000","last_peered":"2024-09-11T07:27:54.621982+0000","last_clean":"2024-09-11T07:27:54.621982+0000","last_became_active":"2024-09-11T07:26:37.561378+0000","last_became_peered":"2024-09-11T07:26:37.561378+0000","last_unstale":"2024-09-11T07:27:54.621982+0000","last_undegraded":"2024-09-11T07:27:54.621982+0000","last_fullsized":"2024-09-11T07:27:54.621982+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.621874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.621874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:07.014230+0000","scrub_duration":0.0013604750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":65,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:49.061777+0000","last_change":"2024-09-11T07:27:49.061728+0000","last_active":"2024-09-11T07:27:49.061777+0000","last_peered":"2024-09-11T07:27:49.061777+0000","last_clean":"2024-09-11T07:27:49.061777+0000","last_became_active":"2024-09-11T07:26:33.541359+0000","last_became_peered":"2024-09-11T07:26:33.541359+0000","last_unstale":"2024-09-11T07:27:49.061777+0000","last_undegraded":"2024-09-11T07:27:49.061777+0000","last_fullsized":"2024-09-11T07:27:49.061777+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-09-11T07:27:49.061670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.061670+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:51.283390+0000","scrub_duration":0.001809801,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'3","reported_seq":53,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T07:28:01.711339+0000","last_change":"2024-09-11T07:28:01.711301+0000","last_active":"2024-09-11T07:28:01.711339+0000","last_peered":"2024-09-11T07:28:01.711339+0000","last_clean":"2024-09-11T07:28:01.711339+0000","last_became_active":"2024-09-11T07:26:31.419501+0000","last_became_peered":"2024-09-11T07:26:31.419501+0000","last_unstale":"2024-09-11T07:28:01.711339+0000","last_undegraded":"2024-09-11T07:28:01.711339+0000","last_fullsized":"2024-09-11T07:28:01.711339+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.711241+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:02.735271+0000","scrub_duration":0.0034627350000000002,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:47.903253+0000","last_change":"2024-09-11T07:27:47.903204+0000","last_active":"2024-09-11T07:27:47.903253+0000","last_peered":"2024-09-11T07:27:47.903253+0000","last_clean":"2024-09-11T07:27:47.903253+0000","last_became_active":"2024-09-11T07:26:31.423304+0000","last_became_peered":"2024-09-11T07:26:31.423304+0000","last_unstale":"2024-09-11T07:27:47.903253+0000","last_undegraded":"2024-09-11T07:27:47.903253+0000","last_fullsized":"2024-09-11T07:27:47.903253+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:47.903135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.903135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:01.867608+0000","scrub_duration":0.001284908,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"72'79","reported_seq":80,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.441891+0000","last_change":"2024-09-11T07:26:57.872633+0000","last_active":"2024-09-11T07:27:40.441891+0000","last_peered":"2024-09-11T07:27:40.441891+0000","last_clean":"2024-09-11T07:27:40.441891+0000","last_became_active":"2024-09-11T07:26:17.397381+0000","last_became_peered":"2024-09-11T07:26:17.397381+0000","last_unstale":"2024-09-11T07:27:40.441891+0000","last_undegraded":"2024-09-11T07:27:40.441891+0000","last_fullsized":"2024-09-11T07:27:40.441891+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'72","last_scrub_stamp":"2024-09-11T07:26:57.872570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:24:22.202219+0000","last_clean_scrub_stamp":"2024-09-11T07:26:57.872570+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:06.077652+0000","scrub_duration":0.0015639180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:44.036194+0000","last_change":"2024-09-11T07:27:44.036138+0000","last_active":"2024-09-11T07:27:44.036194+0000","last_peered":"2024-09-11T07:27:44.036194+0000","last_clean":"2024-09-11T07:27:44.036194+0000","last_became_active":"2024-09-11T07:26:35.564480+0000","last_became_peered":"2024-09-11T07:26:35.564480+0000","last_unstale":"2024-09-11T07:27:44.036194+0000","last_undegraded":"2024-09-11T07:27:44.036194+0000","last_fullsized":"2024-09-11T07:27:44.036194+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:44.036058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:44.036058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:48.598709+0000","scrub_duration":0.001488931,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.443071+0000","last_change":"2024-09-11T07:26:37.560202+0000","last_active":"2024-09-11T07:27:40.443071+0000","last_peered":"2024-09-11T07:27:40.443071+0000","last_clean":"2024-09-11T07:27:40.443071+0000","last_became_active":"2024-09-11T07:26:37.559920+0000","last_became_peered":"2024-09-11T07:26:37.559920+0000","last_unstale":"2024-09-11T07:27:40.443071+0000","last_undegraded":"2024-09-11T07:27:40.443071+0000","last_fullsized":"2024-09-11T07:27:40.443071+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":74,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:59.128346+0000","last_change":"2024-09-11T07:27:59.128292+0000","last_active":"2024-09-11T07:27:59.128346+0000","last_peered":"2024-09-11T07:27:59.128346+0000","last_clean":"2024-09-11T07:27:59.128346+0000","last_became_active":"2024-09-11T07:26:33.543687+0000","last_became_peered":"2024-09-11T07:26:33.543687+0000","last_unstale":"2024-09-11T07:27:59.128346+0000","last_undegraded":"2024-09-11T07:27:59.128346+0000","last_fullsized":"2024-09-11T07:27:59.128346+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-11T07:27:59.128237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:59.128237+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:16.835564+0000","scrub_duration":0.00199329,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:54.182193+0000","last_change":"2024-09-11T07:27:54.182157+0000","last_active":"2024-09-11T07:27:54.182193+0000","last_peered":"2024-09-11T07:27:54.182193+0000","last_clean":"2024-09-11T07:27:54.182193+0000","last_became_active":"2024-09-11T07:26:31.419560+0000","last_became_peered":"2024-09-11T07:26:31.419560+0000","last_unstale":"2024-09-11T07:27:54.182193+0000","last_undegraded":"2024-09-11T07:27:54.182193+0000","last_fullsized":"2024-09-11T07:27:54.182193+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.182106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.182106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:04.075506+0000","scrub_duration":0.001213254,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.419068+0000","last_change":"2024-09-11T07:26:35.564320+0000","last_active":"2024-09-11T07:27:40.419068+0000","last_peered":"2024-09-11T07:27:40.419068+0000","last_clean":"2024-09-11T07:27:40.419068+0000","last_became_active":"2024-09-11T07:26:35.564044+0000","last_became_peered":"2024-09-11T07:26:35.564044+0000","last_unstale":"2024-09-11T07:27:40.419068+0000","last_undegraded":"2024-09-11T07:27:40.419068+0000","last_fullsized":"2024-09-11T07:27:40.419068+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:01.424414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.442209+0000","last_change":"2024-09-11T07:26:37.565084+0000","last_active":"2024-09-11T07:27:40.442209+0000","last_peered":"2024-09-11T07:27:40.442209+0000","last_clean":"2024-09-11T07:27:40.442209+0000","last_became_active":"2024-09-11T07:26:37.564801+0000","last_became_peered":"2024-09-11T07:26:37.564801+0000","last_unstale":"2024-09-11T07:27:40.442209+0000","last_undegraded":"2024-09-11T07:27:40.442209+0000","last_fullsized":"2024-09-11T07:27:40.442209+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":59,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T07:28:02.231010+0000","last_change":"2024-09-11T07:28:02.230985+0000","last_active":"2024-09-11T07:28:02.231010+0000","last_peered":"2024-09-11T07:28:02.231010+0000","last_clean":"2024-09-11T07:28:02.231010+0000","last_became_active":"2024-09-11T07:26:33.543148+0000","last_became_peered":"2024-09-11T07:26:33.543148+0000","last_unstale":"2024-09-11T07:28:02.231010+0000","last_undegraded":"2024-09-11T07:28:02.231010+0000","last_fullsized":"2024-09-11T07:28:02.231010+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:28:02.230928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.230928+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:27.244330+0000","scrub_duration":0.001888926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:53.660023+0000","last_change":"2024-09-11T07:27:53.659961+0000","last_active":"2024-09-11T07:27:53.660023+0000","last_peered":"2024-09-11T07:27:53.660023+0000","last_clean":"2024-09-11T07:27:53.660023+0000","last_became_active":"2024-09-11T07:26:31.423056+0000","last_became_peered":"2024-09-11T07:26:31.423056+0000","last_unstale":"2024-09-11T07:27:53.660023+0000","last_undegraded":"2024-09-11T07:27:53.660023+0000","last_fullsized":"2024-09-11T07:27:53.660023+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:53.659902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:53.659902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.781453+0000","scrub_duration":0.0011613719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.419072+0000","last_change":"2024-09-11T07:26:35.563452+0000","last_active":"2024-09-11T07:27:40.419072+0000","last_peered":"2024-09-11T07:27:40.419072+0000","last_clean":"2024-09-11T07:27:40.419072+0000","last_became_active":"2024-09-11T07:26:35.563182+0000","last_became_peered":"2024-09-11T07:26:35.563182+0000","last_unstale":"2024-09-11T07:27:40.419072+0000","last_undegraded":"2024-09-11T07:27:40.419072+0000","last_fullsized":"2024-09-11T07:27:40.419072+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:00.634009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.441850+0000","last_change":"2024-09-11T07:26:37.565396+0000","last_active":"2024-09-11T07:27:40.441850+0000","last_peered":"2024-09-11T07:27:40.441850+0000","last_clean":"2024-09-11T07:27:40.441850+0000","last_became_active":"2024-09-11T07:26:37.565029+0000","last_became_peered":"2024-09-11T07:26:37.565029+0000","last_unstale":"2024-09-11T07:27:40.441850+0000","last_undegraded":"2024-09-11T07:27:40.441850+0000","last_fullsized":"2024-09-11T07:27:40.441850+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:03.261465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":58,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.441440+0000","last_change":"2024-09-11T07:26:33.538495+0000","last_active":"2024-09-11T07:27:40.441440+0000","last_peered":"2024-09-11T07:27:40.441440+0000","last_clean":"2024-09-11T07:27:40.441440+0000","last_became_active":"2024-09-11T07:26:33.538216+0000","last_became_peered":"2024-09-11T07:26:33.538216+0000","last_unstale":"2024-09-11T07:27:40.441440+0000","last_undegraded":"2024-09-11T07:27:40.441440+0000","last_fullsized":"2024-09-11T07:27:40.441440+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:50.132681+0000","last_change":"2024-09-11T07:27:50.132629+0000","last_active":"2024-09-11T07:27:50.132681+0000","last_peered":"2024-09-11T07:27:50.132681+0000","last_clean":"2024-09-11T07:27:50.132681+0000","last_became_active":"2024-09-11T07:26:31.418490+0000","last_became_peered":"2024-09-11T07:26:31.418490+0000","last_unstale":"2024-09-11T07:27:50.132681+0000","last_undegraded":"2024-09-11T07:27:50.132681+0000","last_fullsized":"2024-09-11T07:27:50.132681+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.132552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.048681+0000","scrub_duration":0.001622807,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:59.499159+0000","last_change":"2024-09-11T07:26:35.567147+0000","last_active":"2024-09-11T07:27:59.499159+0000","last_peered":"2024-09-11T07:27:59.499159+0000","last_clean":"2024-09-11T07:27:59.499159+0000","last_became_active":"2024-09-11T07:26:35.566888+0000","last_became_peered":"2024-09-11T07:26:35.566888+0000","last_unstale":"2024-09-11T07:27:59.499159+0000","last_undegraded":"2024-09-11T07:27:59.499159+0000","last_fullsized":"2024-09-11T07:27:59.499159+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:03.574568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.441523+0000","last_change":"2024-09-11T07:26:37.861501+0000","last_active":"2024-09-11T07:27:40.441523+0000","last_peered":"2024-09-11T07:27:40.441523+0000","last_clean":"2024-09-11T07:27:40.441523+0000","last_became_active":"2024-09-11T07:26:37.861134+0000","last_became_peered":"2024-09-11T07:26:37.861134+0000","last_unstale":"2024-09-11T07:27:40.441523+0000","last_undegraded":"2024-09-11T07:27:40.441523+0000","last_fullsized":"2024-09-11T07:27:40.441523+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"83'17","reported_seq":71,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T07:28:01.098109+0000","last_change":"2024-09-11T07:28:01.098068+0000","last_active":"2024-09-11T07:28:01.098109+0000","last_peered":"2024-09-11T07:28:01.098109+0000","last_clean":"2024-09-11T07:28:01.098109+0000","last_became_active":"2024-09-11T07:26:33.540363+0000","last_became_peered":"2024-09-11T07:26:33.540363+0000","last_unstale":"2024-09-11T07:28:01.098109+0000","last_undegraded":"2024-09-11T07:28:01.098109+0000","last_fullsized":"2024-09-11T07:28:01.098109+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'17","last_scrub_stamp":"2024-09-11T07:28:01.097981+0000","last_deep_scrub":"83'17","last_deep_scrub_stamp":"2024-09-11T07:28:01.097981+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.097981+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:26.179592+0000","scrub_duration":0.0042048720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":62,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:46.077920+0000","last_change":"2024-09-11T07:27:46.077857+0000","last_active":"2024-09-11T07:27:46.077920+0000","last_peered":"2024-09-11T07:27:46.077920+0000","last_clean":"2024-09-11T07:27:46.077920+0000","last_became_active":"2024-09-11T07:26:31.419718+0000","last_became_peered":"2024-09-11T07:26:31.419718+0000","last_unstale":"2024-09-11T07:27:46.077920+0000","last_undegraded":"2024-09-11T07:27:46.077920+0000","last_fullsized":"2024-09-11T07:27:46.077920+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:27:46.077786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:46.077786+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:07.735381+0000","scrub_duration":0.0019376720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:53.209557+0000","last_change":"2024-09-11T07:27:53.209518+0000","last_active":"2024-09-11T07:27:53.209557+0000","last_peered":"2024-09-11T07:27:53.209557+0000","last_clean":"2024-09-11T07:27:53.209557+0000","last_became_active":"2024-09-11T07:26:35.561967+0000","last_became_peered":"2024-09-11T07:26:35.561967+0000","last_unstale":"2024-09-11T07:27:53.209557+0000","last_undegraded":"2024-09-11T07:27:53.209557+0000","last_fullsized":"2024-09-11T07:27:53.209557+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:53.209452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:53.209452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:23.125281+0000","scrub_duration":0.0014563029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T07:28:02.677244+0000","last_change":"2024-09-11T07:28:02.677186+0000","last_active":"2024-09-11T07:28:02.677244+0000","last_peered":"2024-09-11T07:28:02.677244+0000","last_clean":"2024-09-11T07:28:02.677244+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:28:02.677244+0000","last_undegraded":"2024-09-11T07:28:02.677244+0000","last_fullsized":"2024-09-11T07:28:02.677244+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:02.677095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.677095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.575070+0000","scrub_duration":0.001519269,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":59,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.442435+0000","last_change":"2024-09-11T07:27:40.351572+0000","last_active":"2024-09-11T07:27:40.442435+0000","last_peered":"2024-09-11T07:27:40.442435+0000","last_clean":"2024-09-11T07:27:40.442435+0000","last_became_active":"2024-09-11T07:26:33.547344+0000","last_became_peered":"2024-09-11T07:26:33.547344+0000","last_unstale":"2024-09-11T07:27:40.442435+0000","last_undegraded":"2024-09-11T07:27:40.442435+0000","last_fullsized":"2024-09-11T07:27:40.442435+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:27:40.351507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:40.351507+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.098394+0000","scrub_duration":0.0021803980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:58.324346+0000","last_change":"2024-09-11T07:27:58.324311+0000","last_active":"2024-09-11T07:27:58.324346+0000","last_peered":"2024-09-11T07:27:58.324346+0000","last_clean":"2024-09-11T07:27:58.324346+0000","last_became_active":"2024-09-11T07:26:31.415701+0000","last_became_peered":"2024-09-11T07:26:31.415701+0000","last_unstale":"2024-09-11T07:27:58.324346+0000","last_undegraded":"2024-09-11T07:27:58.324346+0000","last_fullsized":"2024-09-11T07:27:58.324346+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:58.324255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.324255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.617500+0000","scrub_duration":0.0013365549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.419865+0000","last_change":"2024-09-11T07:26:35.559563+0000","last_active":"2024-09-11T07:27:40.419865+0000","last_peered":"2024-09-11T07:27:40.419865+0000","last_clean":"2024-09-11T07:27:40.419865+0000","last_became_active":"2024-09-11T07:26:35.559262+0000","last_became_peered":"2024-09-11T07:26:35.559262+0000","last_unstale":"2024-09-11T07:27:40.419865+0000","last_undegraded":"2024-09-11T07:27:40.419865+0000","last_fullsized":"2024-09-11T07:27:40.419865+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.419333+0000","last_change":"2024-09-11T07:26:37.564431+0000","last_active":"2024-09-11T07:27:40.419333+0000","last_peered":"2024-09-11T07:27:40.419333+0000","last_clean":"2024-09-11T07:27:40.419333+0000","last_became_active":"2024-09-11T07:26:37.564084+0000","last_became_peered":"2024-09-11T07:26:37.564084+0000","last_unstale":"2024-09-11T07:27:40.419333+0000","last_undegraded":"2024-09-11T07:27:40.419333+0000","last_fullsized":"2024-09-11T07:27:40.419333+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:45.990678+0000","last_change":"2024-09-11T07:27:45.990617+0000","last_active":"2024-09-11T07:27:45.990678+0000","last_peered":"2024-09-11T07:27:45.990678+0000","last_clean":"2024-09-11T07:27:45.990678+0000","last_became_active":"2024-09-11T07:26:31.423229+0000","last_became_peered":"2024-09-11T07:26:31.423229+0000","last_unstale":"2024-09-11T07:27:45.990678+0000","last_undegraded":"2024-09-11T07:27:45.990678+0000","last_fullsized":"2024-09-11T07:27:45.990678+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:45.990548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:45.990548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:51.641867+0000","scrub_duration":0.0014585150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"82'9","reported_seq":55,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:48.761532+0000","last_change":"2024-09-11T07:27:48.761470+0000","last_active":"2024-09-11T07:27:48.761532+0000","last_peered":"2024-09-11T07:27:48.761532+0000","last_clean":"2024-09-11T07:27:48.761532+0000","last_became_active":"2024-09-11T07:26:33.543628+0000","last_became_peered":"2024-09-11T07:26:33.543628+0000","last_unstale":"2024-09-11T07:27:48.761532+0000","last_undegraded":"2024-09-11T07:27:48.761532+0000","last_fullsized":"2024-09-11T07:27:48.761532+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'9","last_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:27:48.761397+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.913254+0000","scrub_duration":0.0037047439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:42.917777+0000","last_change":"2024-09-11T07:27:42.917739+0000","last_active":"2024-09-11T07:27:42.917777+0000","last_peered":"2024-09-11T07:27:42.917777+0000","last_clean":"2024-09-11T07:27:42.917777+0000","last_became_active":"2024-09-11T07:26:35.566894+0000","last_became_peered":"2024-09-11T07:26:35.566894+0000","last_unstale":"2024-09-11T07:27:42.917777+0000","last_undegraded":"2024-09-11T07:27:42.917777+0000","last_fullsized":"2024-09-11T07:27:42.917777+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.917680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:06.763663+0000","scrub_duration":0.0013496649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:47.031943+0000","last_change":"2024-09-11T07:27:47.031883+0000","last_active":"2024-09-11T07:27:47.031943+0000","last_peered":"2024-09-11T07:27:47.031943+0000","last_clean":"2024-09-11T07:27:47.031943+0000","last_became_active":"2024-09-11T07:26:37.863100+0000","last_became_peered":"2024-09-11T07:26:37.863100+0000","last_unstale":"2024-09-11T07:27:47.031943+0000","last_undegraded":"2024-09-11T07:27:47.031943+0000","last_fullsized":"2024-09-11T07:27:47.031943+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:47.031807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.031807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:54.469643+0000","scrub_duration":0.001556195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"64'10","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.420281+0000","last_change":"2024-09-11T07:26:33.545346+0000","last_active":"2024-09-11T07:27:40.420281+0000","last_peered":"2024-09-11T07:27:40.420281+0000","last_clean":"2024-09-11T07:27:40.420281+0000","last_became_active":"2024-09-11T07:26:33.545078+0000","last_became_peered":"2024-09-11T07:26:33.545078+0000","last_unstale":"2024-09-11T07:27:40.420281+0000","last_undegraded":"2024-09-11T07:27:40.420281+0000","last_fullsized":"2024-09-11T07:27:40.420281+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.442094+0000","last_change":"2024-09-11T07:26:31.424634+0000","last_active":"2024-09-11T07:27:40.442094+0000","last_peered":"2024-09-11T07:27:40.442094+0000","last_clean":"2024-09-11T07:27:40.442094+0000","last_became_active":"2024-09-11T07:26:31.423789+0000","last_became_peered":"2024-09-11T07:26:31.423789+0000","last_unstale":"2024-09-11T07:27:40.442094+0000","last_undegraded":"2024-09-11T07:27:40.442094+0000","last_fullsized":"2024-09-11T07:27:40.442094+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:59.499216+0000","last_change":"2024-09-11T07:26:35.562509+0000","last_active":"2024-09-11T07:27:59.499216+0000","last_peered":"2024-09-11T07:27:59.499216+0000","last_clean":"2024-09-11T07:27:59.499216+0000","last_became_active":"2024-09-11T07:26:35.562245+0000","last_became_peered":"2024-09-11T07:26:35.562245+0000","last_unstale":"2024-09-11T07:27:59.499216+0000","last_undegraded":"2024-09-11T07:27:59.499216+0000","last_fullsized":"2024-09-11T07:27:59.499216+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:49.245161+0000","last_change":"2024-09-11T07:27:49.245119+0000","last_active":"2024-09-11T07:27:49.245161+0000","last_peered":"2024-09-11T07:27:49.245161+0000","last_clean":"2024-09-11T07:27:49.245161+0000","last_became_active":"2024-09-11T07:26:37.566749+0000","last_became_peered":"2024-09-11T07:26:37.566749+0000","last_unstale":"2024-09-11T07:27:49.245161+0000","last_undegraded":"2024-09-11T07:27:49.245161+0000","last_fullsized":"2024-09-11T07:27:49.245161+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:49.245047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.245047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:49.614854+0000","scrub_duration":0.001867966,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"83'13","reported_seq":63,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T07:28:00.959710+0000","last_change":"2024-09-11T07:28:00.959669+0000","last_active":"2024-09-11T07:28:00.959710+0000","last_peered":"2024-09-11T07:28:00.959710+0000","last_clean":"2024-09-11T07:28:00.959710+0000","last_became_active":"2024-09-11T07:26:33.537425+0000","last_became_peered":"2024-09-11T07:26:33.537425+0000","last_unstale":"2024-09-11T07:28:00.959710+0000","last_undegraded":"2024-09-11T07:28:00.959710+0000","last_fullsized":"2024-09-11T07:28:00.959710+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-11T07:28:00.959583+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-11T07:28:00.959583+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.959583+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:06.630980+0000","scrub_duration":0.0032987350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T07:28:01.237372+0000","last_change":"2024-09-11T07:28:01.237329+0000","last_active":"2024-09-11T07:28:01.237372+0000","last_peered":"2024-09-11T07:28:01.237372+0000","last_clean":"2024-09-11T07:28:01.237372+0000","last_became_active":"2024-09-11T07:26:31.419409+0000","last_became_peered":"2024-09-11T07:26:31.419409+0000","last_unstale":"2024-09-11T07:28:01.237372+0000","last_undegraded":"2024-09-11T07:28:01.237372+0000","last_fullsized":"2024-09-11T07:28:01.237372+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:01.237267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.237267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.266034+0000","scrub_duration":0.0012797399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:59.499243+0000","last_change":"2024-09-11T07:26:35.560917+0000","last_active":"2024-09-11T07:27:59.499243+0000","last_peered":"2024-09-11T07:27:59.499243+0000","last_clean":"2024-09-11T07:27:59.499243+0000","last_became_active":"2024-09-11T07:26:35.560610+0000","last_became_peered":"2024-09-11T07:26:35.560610+0000","last_unstale":"2024-09-11T07:27:59.499243+0000","last_undegraded":"2024-09-11T07:27:59.499243+0000","last_fullsized":"2024-09-11T07:27:59.499243+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.421955+0000","last_change":"2024-09-11T07:26:37.566136+0000","last_active":"2024-09-11T07:27:40.421955+0000","last_peered":"2024-09-11T07:27:40.421955+0000","last_clean":"2024-09-11T07:27:40.421955+0000","last_became_active":"2024-09-11T07:26:37.565723+0000","last_became_peered":"2024-09-11T07:26:37.565723+0000","last_unstale":"2024-09-11T07:27:40.421955+0000","last_undegraded":"2024-09-11T07:27:40.421955+0000","last_fullsized":"2024-09-11T07:27:40.421955+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:00.803451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":59,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:57.601371+0000","last_change":"2024-09-11T07:27:57.601328+0000","last_active":"2024-09-11T07:27:57.601371+0000","last_peered":"2024-09-11T07:27:57.601371+0000","last_clean":"2024-09-11T07:27:57.601371+0000","last_became_active":"2024-09-11T07:26:33.540688+0000","last_became_peered":"2024-09-11T07:26:33.540688+0000","last_unstale":"2024-09-11T07:27:57.601371+0000","last_undegraded":"2024-09-11T07:27:57.601371+0000","last_fullsized":"2024-09-11T07:27:57.601371+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:27:57.601261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.601261+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:58.826830+0000","scrub_duration":0.002247724,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:54.102627+0000","last_change":"2024-09-11T07:27:54.102580+0000","last_active":"2024-09-11T07:27:54.102627+0000","last_peered":"2024-09-11T07:27:54.102627+0000","last_clean":"2024-09-11T07:27:54.102627+0000","last_became_active":"2024-09-11T07:26:31.425587+0000","last_became_peered":"2024-09-11T07:26:31.425587+0000","last_unstale":"2024-09-11T07:27:54.102627+0000","last_undegraded":"2024-09-11T07:27:54.102627+0000","last_fullsized":"2024-09-11T07:27:54.102627+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.102528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.102528+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.102528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:06.835183+0000","scrub_duration":0.0012736080000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.419020+0000","last_change":"2024-09-11T07:26:35.562603+0000","last_active":"2024-09-11T07:27:40.419020+0000","last_peered":"2024-09-11T07:27:40.419020+0000","last_clean":"2024-09-11T07:27:40.419020+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:27:40.419020+0000","last_undegraded":"2024-09-11T07:27:40.419020+0000","last_fullsized":"2024-09-11T07:27:40.419020+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:01.555693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:58.192620+0000","last_change":"2024-09-11T07:27:58.192585+0000","last_active":"2024-09-11T07:27:58.192620+0000","last_peered":"2024-09-11T07:27:58.192620+0000","last_clean":"2024-09-11T07:27:58.192620+0000","last_became_active":"2024-09-11T07:26:37.568886+0000","last_became_peered":"2024-09-11T07:26:37.568886+0000","last_unstale":"2024-09-11T07:27:58.192620+0000","last_undegraded":"2024-09-11T07:27:58.192620+0000","last_fullsized":"2024-09-11T07:27:58.192620+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:58.192528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.192528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:18.757569+0000","scrub_duration":0.001884698,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":62,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:48.081507+0000","last_change":"2024-09-11T07:27:48.081460+0000","last_active":"2024-09-11T07:27:48.081507+0000","last_peered":"2024-09-11T07:27:48.081507+0000","last_clean":"2024-09-11T07:27:48.081507+0000","last_became_active":"2024-09-11T07:26:33.540208+0000","last_became_peered":"2024-09-11T07:26:33.540208+0000","last_unstale":"2024-09-11T07:27:48.081507+0000","last_undegraded":"2024-09-11T07:27:48.081507+0000","last_fullsized":"2024-09-11T07:27:48.081507+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:27:48.081392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:48.081392+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:04.524601+0000","scrub_duration":0.0022394590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:49.677100+0000","last_change":"2024-09-11T07:27:49.677063+0000","last_active":"2024-09-11T07:27:49.677100+0000","last_peered":"2024-09-11T07:27:49.677100+0000","last_clean":"2024-09-11T07:27:49.677100+0000","last_became_active":"2024-09-11T07:26:31.422671+0000","last_became_peered":"2024-09-11T07:26:31.422671+0000","last_unstale":"2024-09-11T07:27:49.677100+0000","last_undegraded":"2024-09-11T07:27:49.677100+0000","last_fullsized":"2024-09-11T07:27:49.677100+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.677003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:54.836177+0000","scrub_duration":0.0013962969999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.441462+0000","last_change":"2024-09-11T07:26:35.567257+0000","last_active":"2024-09-11T07:27:40.441462+0000","last_peered":"2024-09-11T07:27:40.441462+0000","last_clean":"2024-09-11T07:27:40.441462+0000","last_became_active":"2024-09-11T07:26:35.566931+0000","last_became_peered":"2024-09-11T07:26:35.566931+0000","last_unstale":"2024-09-11T07:27:40.441462+0000","last_undegraded":"2024-09-11T07:27:40.441462+0000","last_fullsized":"2024-09-11T07:27:40.441462+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.418334+0000","last_change":"2024-09-11T07:26:37.561886+0000","last_active":"2024-09-11T07:27:40.418334+0000","last_peered":"2024-09-11T07:27:40.418334+0000","last_clean":"2024-09-11T07:27:40.418334+0000","last_became_active":"2024-09-11T07:26:37.561573+0000","last_became_peered":"2024-09-11T07:26:37.561573+0000","last_unstale":"2024-09-11T07:27:40.418334+0000","last_undegraded":"2024-09-11T07:27:40.418334+0000","last_fullsized":"2024-09-11T07:27:40.418334+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"82'6","reported_seq":51,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:47.789563+0000","last_change":"2024-09-11T07:27:47.789519+0000","last_active":"2024-09-11T07:27:47.789563+0000","last_peered":"2024-09-11T07:27:47.789563+0000","last_clean":"2024-09-11T07:27:47.789563+0000","last_became_active":"2024-09-11T07:26:33.545830+0000","last_became_peered":"2024-09-11T07:26:33.545830+0000","last_unstale":"2024-09-11T07:27:47.789563+0000","last_undegraded":"2024-09-11T07:27:47.789563+0000","last_fullsized":"2024-09-11T07:27:47.789563+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'6","last_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_deep_scrub":"82'6","last_deep_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.789465+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:59.162056+0000","scrub_duration":0.002624679,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:58.043133+0000","last_change":"2024-09-11T07:27:58.043076+0000","last_active":"2024-09-11T07:27:58.043133+0000","last_peered":"2024-09-11T07:27:58.043133+0000","last_clean":"2024-09-11T07:27:58.043133+0000","last_became_active":"2024-09-11T07:26:31.420518+0000","last_became_peered":"2024-09-11T07:26:31.420518+0000","last_unstale":"2024-09-11T07:27:58.043133+0000","last_undegraded":"2024-09-11T07:27:58.043133+0000","last_fullsized":"2024-09-11T07:27:58.043133+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.043009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:24.326885+0000","scrub_duration":0.0015014900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:56.124636+0000","last_change":"2024-09-11T07:27:56.124601+0000","last_active":"2024-09-11T07:27:56.124636+0000","last_peered":"2024-09-11T07:27:56.124636+0000","last_clean":"2024-09-11T07:27:56.124636+0000","last_became_active":"2024-09-11T07:26:35.560506+0000","last_became_peered":"2024-09-11T07:26:35.560506+0000","last_unstale":"2024-09-11T07:27:56.124636+0000","last_undegraded":"2024-09-11T07:27:56.124636+0000","last_fullsized":"2024-09-11T07:27:56.124636+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:56.124542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:56.124542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.847074+0000","scrub_duration":0.0011688619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:42.059109+0000","last_change":"2024-09-11T07:27:42.059066+0000","last_active":"2024-09-11T07:27:42.059109+0000","last_peered":"2024-09-11T07:27:42.059109+0000","last_clean":"2024-09-11T07:27:42.059109+0000","last_became_active":"2024-09-11T07:26:37.564208+0000","last_became_peered":"2024-09-11T07:26:37.564208+0000","last_unstale":"2024-09-11T07:27:42.059109+0000","last_undegraded":"2024-09-11T07:27:42.059109+0000","last_fullsized":"2024-09-11T07:27:42.059109+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.059001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.059001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:51.267826+0000","scrub_duration":0.001418094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":52,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.418534+0000","last_change":"2024-09-11T07:26:33.542913+0000","last_active":"2024-09-11T07:27:40.418534+0000","last_peered":"2024-09-11T07:27:40.418534+0000","last_clean":"2024-09-11T07:27:40.418534+0000","last_became_active":"2024-09-11T07:26:33.542625+0000","last_became_peered":"2024-09-11T07:26:33.542625+0000","last_unstale":"2024-09-11T07:27:40.418534+0000","last_undegraded":"2024-09-11T07:27:40.418534+0000","last_fullsized":"2024-09-11T07:27:40.418534+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T07:28:02.157550+0000","last_change":"2024-09-11T07:28:02.157520+0000","last_active":"2024-09-11T07:28:02.157550+0000","last_peered":"2024-09-11T07:28:02.157550+0000","last_clean":"2024-09-11T07:28:02.157550+0000","last_became_active":"2024-09-11T07:26:31.425876+0000","last_became_peered":"2024-09-11T07:26:31.425876+0000","last_unstale":"2024-09-11T07:28:02.157550+0000","last_undegraded":"2024-09-11T07:28:02.157550+0000","last_fullsized":"2024-09-11T07:28:02.157550+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:02.157455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.157455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:12.096002+0000","scrub_duration":0.0016990040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.441343+0000","last_change":"2024-09-11T07:26:35.565591+0000","last_active":"2024-09-11T07:27:40.441343+0000","last_peered":"2024-09-11T07:27:40.441343+0000","last_clean":"2024-09-11T07:27:40.441343+0000","last_became_active":"2024-09-11T07:26:35.565339+0000","last_became_peered":"2024-09-11T07:26:35.565339+0000","last_unstale":"2024-09-11T07:27:40.441343+0000","last_undegraded":"2024-09-11T07:27:40.441343+0000","last_fullsized":"2024-09-11T07:27:40.441343+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:01.008070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.418879+0000","last_change":"2024-09-11T07:26:37.862599+0000","last_active":"2024-09-11T07:27:40.418879+0000","last_peered":"2024-09-11T07:27:40.418879+0000","last_clean":"2024-09-11T07:27:40.418879+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:27:40.418879+0000","last_undegraded":"2024-09-11T07:27:40.418879+0000","last_fullsized":"2024-09-11T07:27:40.418879+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:03.134547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.441703+0000","last_change":"2024-09-11T07:26:37.564199+0000","last_active":"2024-09-11T07:27:40.441703+0000","last_peered":"2024-09-11T07:27:40.441703+0000","last_clean":"2024-09-11T07:27:40.441703+0000","last_became_active":"2024-09-11T07:26:37.563942+0000","last_became_peered":"2024-09-11T07:26:37.563942+0000","last_unstale":"2024-09-11T07:27:40.441703+0000","last_undegraded":"2024-09-11T07:27:40.441703+0000","last_fullsized":"2024-09-11T07:27:40.441703+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:01.320334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":54,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T07:28:02.969526+0000","last_change":"2024-09-11T07:28:02.969498+0000","last_active":"2024-09-11T07:28:02.969526+0000","last_peered":"2024-09-11T07:28:02.969526+0000","last_clean":"2024-09-11T07:28:02.969526+0000","last_became_active":"2024-09-11T07:26:31.423932+0000","last_became_peered":"2024-09-11T07:26:31.423932+0000","last_unstale":"2024-09-11T07:28:02.969526+0000","last_undegraded":"2024-09-11T07:28:02.969526+0000","last_fullsized":"2024-09-11T07:28:02.969526+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:28:02.969442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.969442+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:22.652879+0000","scrub_duration":0.0014604450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":58,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:55.150674+0000","last_change":"2024-09-11T07:27:55.150630+0000","last_active":"2024-09-11T07:27:55.150674+0000","last_peered":"2024-09-11T07:27:55.150674+0000","last_clean":"2024-09-11T07:27:55.150674+0000","last_became_active":"2024-09-11T07:26:33.546797+0000","last_became_peered":"2024-09-11T07:26:33.546797+0000","last_unstale":"2024-09-11T07:27:55.150674+0000","last_undegraded":"2024-09-11T07:27:55.150674+0000","last_fullsized":"2024-09-11T07:27:55.150674+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:27:55.150567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:55.150567+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:10.187014+0000","scrub_duration":0.0016612689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"64'8","reported_seq":70,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T07:28:01.123494+0000","last_change":"2024-09-11T07:28:01.123469+0000","last_active":"2024-09-11T07:28:01.123494+0000","last_peered":"2024-09-11T07:28:01.123494+0000","last_clean":"2024-09-11T07:28:01.123494+0000","last_became_active":"2024-09-11T07:26:35.559304+0000","last_became_peered":"2024-09-11T07:26:35.559304+0000","last_unstale":"2024-09-11T07:28:01.123494+0000","last_undegraded":"2024-09-11T07:28:01.123494+0000","last_fullsized":"2024-09-11T07:28:01.123494+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:28:01.123401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.123401+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:18.715539+0000","scrub_duration":0.00167504,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.441108+0000","last_change":"2024-09-11T07:26:37.563308+0000","last_active":"2024-09-11T07:27:40.441108+0000","last_peered":"2024-09-11T07:27:40.441108+0000","last_clean":"2024-09-11T07:27:40.441108+0000","last_became_active":"2024-09-11T07:26:37.562501+0000","last_became_peered":"2024-09-11T07:26:37.562501+0000","last_unstale":"2024-09-11T07:27:40.441108+0000","last_undegraded":"2024-09-11T07:27:40.441108+0000","last_fullsized":"2024-09-11T07:27:40.441108+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.418716+0000","last_change":"2024-09-11T07:27:38.732867+0000","last_active":"2024-09-11T07:27:40.418716+0000","last_peered":"2024-09-11T07:27:40.418716+0000","last_clean":"2024-09-11T07:27:40.418716+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:27:40.418716+0000","last_undegraded":"2024-09-11T07:27:40.418716+0000","last_fullsized":"2024-09-11T07:27:40.418716+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:38.732795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:38.732795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:56.730699+0000","scrub_duration":0.0025091950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":62,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T07:28:03.164319+0000","last_change":"2024-09-11T07:28:03.164281+0000","last_active":"2024-09-11T07:28:03.164319+0000","last_peered":"2024-09-11T07:28:03.164319+0000","last_clean":"2024-09-11T07:28:03.164319+0000","last_became_active":"2024-09-11T07:26:33.541154+0000","last_became_peered":"2024-09-11T07:26:33.541154+0000","last_unstale":"2024-09-11T07:28:03.164319+0000","last_undegraded":"2024-09-11T07:28:03.164319+0000","last_fullsized":"2024-09-11T07:28:03.164319+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:28:03.164220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:03.164220+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:20.207460+0000","scrub_duration":0.0020677629999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.442280+0000","last_change":"2024-09-11T07:26:35.559093+0000","last_active":"2024-09-11T07:27:40.442280+0000","last_peered":"2024-09-11T07:27:40.442280+0000","last_clean":"2024-09-11T07:27:40.442280+0000","last_became_active":"2024-09-11T07:26:35.558809+0000","last_became_peered":"2024-09-11T07:26:35.558809+0000","last_unstale":"2024-09-11T07:27:40.442280+0000","last_undegraded":"2024-09-11T07:27:40.442280+0000","last_fullsized":"2024-09-11T07:27:40.442280+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:00.864666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T07:28:00.126783+0000","last_change":"2024-09-11T07:28:00.126756+0000","last_active":"2024-09-11T07:28:00.126783+0000","last_peered":"2024-09-11T07:28:00.126783+0000","last_clean":"2024-09-11T07:28:00.126783+0000","last_became_active":"2024-09-11T07:26:37.862373+0000","last_became_peered":"2024-09-11T07:26:37.862373+0000","last_unstale":"2024-09-11T07:28:00.126783+0000","last_undegraded":"2024-09-11T07:28:00.126783+0000","last_fullsized":"2024-09-11T07:28:00.126783+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:00.126702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.126702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:18.534592+0000","scrub_duration":0.001184903,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.441400+0000","last_change":"2024-09-11T07:27:33.983889+0000","last_active":"2024-09-11T07:27:40.441400+0000","last_peered":"2024-09-11T07:27:40.441400+0000","last_clean":"2024-09-11T07:27:40.441400+0000","last_became_active":"2024-09-11T07:26:31.424651+0000","last_became_peered":"2024-09-11T07:26:31.424651+0000","last_unstale":"2024-09-11T07:27:40.441400+0000","last_undegraded":"2024-09-11T07:27:40.441400+0000","last_fullsized":"2024-09-11T07:27:40.441400+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:33.983813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:33.983813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:55.264233+0000","scrub_duration":0.001290759,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":68,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T07:28:00.272747+0000","last_change":"2024-09-11T07:28:00.272702+0000","last_active":"2024-09-11T07:28:00.272747+0000","last_peered":"2024-09-11T07:28:00.272747+0000","last_clean":"2024-09-11T07:28:00.272747+0000","last_became_active":"2024-09-11T07:26:33.542544+0000","last_became_peered":"2024-09-11T07:26:33.542544+0000","last_unstale":"2024-09-11T07:28:00.272747+0000","last_undegraded":"2024-09-11T07:28:00.272747+0000","last_fullsized":"2024-09-11T07:28:00.272747+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-09-11T07:28:00.272628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.272628+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:05.374102+0000","scrub_duration":0.0021830019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.442958+0000","last_change":"2024-09-11T07:26:35.556136+0000","last_active":"2024-09-11T07:27:40.442958+0000","last_peered":"2024-09-11T07:27:40.442958+0000","last_clean":"2024-09-11T07:27:40.442958+0000","last_became_active":"2024-09-11T07:26:35.555862+0000","last_became_peered":"2024-09-11T07:26:35.555862+0000","last_unstale":"2024-09-11T07:27:40.442958+0000","last_undegraded":"2024-09-11T07:27:40.442958+0000","last_fullsized":"2024-09-11T07:27:40.442958+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:40.419803+0000","last_change":"2024-09-11T07:26:37.862060+0000","last_active":"2024-09-11T07:27:40.419803+0000","last_peered":"2024-09-11T07:27:40.419803+0000","last_clean":"2024-09-11T07:27:40.419803+0000","last_became_active":"2024-09-11T07:26:37.861647+0000","last_became_peered":"2024-09-11T07:26:37.861647+0000","last_unstale":"2024-09-11T07:27:40.419803+0000","last_undegraded":"2024-09-11T07:27:40.419803+0000","last_fullsized":"2024-09-11T07:27:40.419803+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:45.732670+0000","last_change":"2024-09-11T07:27:45.732604+0000","last_active":"2024-09-11T07:27:45.732670+0000","last_peered":"2024-09-11T07:27:45.732670+0000","last_clean":"2024-09-11T07:27:45.732670+0000","last_became_active":"2024-09-11T07:26:31.420598+0000","last_became_peered":"2024-09-11T07:26:31.420598+0000","last_unstale":"2024-09-11T07:27:45.732670+0000","last_undegraded":"2024-09-11T07:27:45.732670+0000","last_fullsized":"2024-09-11T07:27:45.732670+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:45.732528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:45.732528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:48.605075+0000","scrub_duration":0.0015466659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"64'12","reported_seq":66,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T07:27:59.737887+0000","last_change":"2024-09-11T07:27:59.737848+0000","last_active":"2024-09-11T07:27:59.737887+0000","last_peered":"2024-09-11T07:27:59.737887+0000","last_clean":"2024-09-11T07:27:59.737887+0000","last_became_active":"2024-09-11T07:26:33.542711+0000","last_became_peered":"2024-09-11T07:26:33.542711+0000","last_unstale":"2024-09-11T07:27:59.737887+0000","last_undegraded":"2024-09-11T07:27:59.737887+0000","last_fullsized":"2024-09-11T07:27:59.737887+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-09-11T07:27:59.737791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:59.737791+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:11.558595+0000","scrub_duration":0.00367732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T07:27:57.199298+0000","last_change":"2024-09-11T07:27:57.199239+0000","last_active":"2024-09-11T07:27:57.199298+0000","last_peered":"2024-09-11T07:27:57.199298+0000","last_clean":"2024-09-11T07:27:57.199298+0000","last_became_active":"2024-09-11T07:26:35.562037+0000","last_became_peered":"2024-09-11T07:26:35.562037+0000","last_unstale":"2024-09-11T07:27:57.199298+0000","last_undegraded":"2024-09-11T07:27:57.199298+0000","last_fullsized":"2024-09-11T07:27:57.199298+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:57.199173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.199173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.093487+0000","scrub_duration":0.0015950770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":421,"ondisk_log_size":421,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":81,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299415,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298540,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446612,"statfs":{"total":95995035648,"available":95689330688,"internally_reserved":0,"allocated":1216512,"data_stored":873895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.498}]},{"osd":1,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.47999999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.623}]},{"osd":3,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":12.625999999999999,"5min":12.625999999999999,"15min":12.625999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.60799999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.375},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.40999999999999998}]},{"osd":5,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.35199999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.38400000000000001}]}]},{"osd":6,"up_from":44,"seq":188978561051,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298516,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446636,"statfs":{"total":95995035648,"available":95689355264,"internally_reserved":0,"allocated":1191936,"data_stored":872373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.374,"5min":0.247,"15min":0.247},"max":{"1min":0.65000000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.60499999999999998,"15min":0.624},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.754,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.56799999999999995}]},{"osd":1,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.82899999999999996,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.49299999999999999,"15min":0.495},"min":{"1min":0.35799999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.60499999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.61499999999999999,"15min":0.63500000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.70299999999999996,"5min":3.234,"15min":3.234},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.496,"5min":0.59099999999999997,"15min":0.60699999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.80200000000000005,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.73399999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51500000000000001,"15min":0.51800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.63500000000000001,"15min":0.65500000000000003},"min":{"1min":0.41199999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.66900000000000004,"5min":3.1739999999999999,"15min":3.1739999999999999},"last":0.54000000000000004}]},{"osd":4,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.58299999999999996,"15min":0.60899999999999999},"min":{"1min":0.32700000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.65200000000000002,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.28399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.45500000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.65600000000000003,"15min":0.69699999999999995},"min":{"1min":0.32400000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64000000000000001,"5min":5.7329999999999997,"15min":5.7329999999999997},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.52000000000000002,"15min":0.52700000000000002},"min":{"1min":0.34000000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70899999999999996,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.48399999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:27:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.6349999999999998,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":0.44},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.51800000000000002}]}]},{"osd":5,"up_from":37,"seq":158913789985,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297940,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447212,"statfs":{"total":95995035648,"available":95689945088,"internally_reserved":0,"allocated":602112,"data_stored":282886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.35499999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.50600000000000001,"15min":0.502},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.53100000000000003}]},{"osd":1,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.57799999999999996}]},{"osd":2,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.56899999999999995}]},{"osd":3,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.55100000000000005}]},{"osd":4,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.375,"15min":0.37},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.42599999999999999,"15min":0.41999999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.40799999999999997}]},{"osd":6,"last update":"Wed Sep 11 07:28:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.41599999999999998,"15min":0.40600000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.376},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.42699999999999999,"15min":0.42199999999999999},"min":{"1min":0.32100000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.436}]},{"osd":7,"last update":"Wed Sep 11 07:27:21 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.45100000000000001}]}]},{"osd":4,"up_from":31,"seq":133143986213,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297964,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447188,"statfs":{"total":95995035648,"available":95689920512,"internally_reserved":0,"allocated":626688,"data_stored":283331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51600000000000001,"15min":0.51300000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68999999999999995,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.504},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.502,"15min":0.498},"min":{"1min":0.36099999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.60399999999999998}]},{"osd":1,"last update":"Wed Sep 11 07:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.5,"15min":0.49399999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.71299999999999997,"15min":0.73699999999999999},"min":{"1min":0.372,"5min":0.318,"15min":0.318},"max":{"1min":1.1200000000000001,"5min":4.5510000000000002,"15min":4.5510000000000002},"last":0.53500000000000003}]},{"osd":2,"last update":"Wed Sep 11 07:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.59399999999999997}]},{"osd":3,"last update":"Wed Sep 11 07:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77400000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.58199999999999996}]},{"osd":5,"last update":"Wed Sep 11 07:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.34499999999999997,"15min":0.33800000000000002},"min":{"1min":0.27600000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.378,"15min":0.372},"min":{"1min":0.29399999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.36099999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:27:59 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42199999999999999,"15min":0.41799999999999998},"min":{"1min":0.315,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.44,"15min":0.43099999999999999},"min":{"1min":0.33500000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.436}]},{"osd":7,"last update":"Wed Sep 11 07:27:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.38800000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182442,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297972,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447180,"statfs":{"total":95995035648,"available":95689912320,"internally_reserved":0,"allocated":634880,"data_stored":282795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:54 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38600000000000001,"15min":0.39300000000000002},"min":{"1min":0.20799999999999999,"5min":0.185,"15min":0.185},"max":{"1min":0.54300000000000004,"5min":0.621,"15min":0.621},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.42499999999999999,"15min":0.441},"min":{"1min":0.26200000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.317}]},{"osd":1,"last update":"Wed Sep 11 07:27:54 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.43099999999999999,"15min":0.437},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.625,"5min":0.752,"15min":0.752},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.41599999999999998,"15min":0.43099999999999999},"min":{"1min":0.23599999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.374}]},{"osd":2,"last update":"Wed Sep 11 07:27:54 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.45100000000000001,"15min":0.46500000000000002},"min":{"1min":0.26900000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.45400000000000001,"15min":0.46899999999999997},"min":{"1min":0.28899999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.59799999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:27:12 2024","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":0.746,"15min":0.62},"min":{"1min":0.47299999999999998,"5min":0.376,"15min":0.376},"max":{"1min":21.856000000000002,"5min":21.856000000000002,"15min":21.856000000000002},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.345,"5min":0.72199999999999998,"15min":0.61799999999999999},"min":{"1min":0.432,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":19.146000000000001,"5min":19.146000000000001,"15min":19.146000000000001},"last":0.68999999999999995}]},{"osd":5,"last update":"Wed Sep 11 07:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.3069999999999999,"5min":0.71999999999999997,"15min":0.622},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":18.416,"5min":18.416,"15min":18.416},"last":0.66500000000000004}]},{"osd":6,"last update":"Wed Sep 11 07:27:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":1.306,"15min":1.4330000000000001},"min":{"1min":0.439,"5min":0.437,"15min":0.437},"max":{"1min":0.72899999999999998,"5min":20.219999999999999,"15min":20.219999999999999},"last":0.745},{"interface":"front","average":{"1min":0.55500000000000005,"5min":1.319,"15min":1.446},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72699999999999998,"5min":19.609000000000002,"15min":19.609000000000002},"last":0.65400000000000003}]},{"osd":7,"last update":"Wed Sep 11 07:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.71899999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411375,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297936,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447216,"statfs":{"total":95995035648,"available":95689949184,"internally_reserved":0,"allocated":598016,"data_stored":282132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:26 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.375,"15min":0.36299999999999999},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.38200000000000001,"15min":0.374},"min":{"1min":0.26600000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.55900000000000005,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.53100000000000003}]},{"osd":1,"last update":"Wed Sep 11 07:27:26 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.40300000000000002,"15min":0.38600000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.40200000000000002,"15min":0.38300000000000001},"min":{"1min":0.218,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.44}]},{"osd":3,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.439,"15min":0.42299999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64500000000000002,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.45900000000000002,"15min":0.45400000000000001},"min":{"1min":0.26600000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65000000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.49099999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:27:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55800000000000005,"15min":0.55300000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57699999999999996,"15min":0.57599999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80700000000000005,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.622}]},{"osd":5,"last update":"Wed Sep 11 07:27:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.68799999999999994,"15min":0.69699999999999995},"min":{"1min":0.45500000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85899999999999999,"5min":2.931,"15min":2.931},"last":0.57099999999999995}]},{"osd":6,"last update":"Wed Sep 11 07:27:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.85299999999999998,"5min":1.494,"15min":1.494},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.63100000000000001,"15min":0.63300000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.875,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.60799999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.68200000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297988,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447164,"statfs":{"total":95995035648,"available":95689895936,"internally_reserved":0,"allocated":651264,"data_stored":283652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:02 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.34100000000000003,"15min":0.33100000000000002},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.45800000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.36399999999999999,"15min":0.35499999999999998},"min":{"1min":0.219,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.54700000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.29899999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:27:26 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.41899999999999998,"15min":0.41599999999999998},"min":{"1min":0.317,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.41199999999999998,"15min":0.40799999999999997},"min":{"1min":0.28699999999999998,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.373}]},{"osd":3,"last update":"Wed Sep 11 07:27:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.44500000000000001,"15min":0.44700000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.60699999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.41399999999999998},"min":{"1min":0.307,"5min":0.253,"15min":0.253},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.45900000000000002}]},{"osd":4,"last update":"Wed Sep 11 07:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.442}]},{"osd":5,"last update":"Wed Sep 11 07:27:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.44,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.71699999999999997,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.50800000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:27:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57199999999999995,"15min":0.57499999999999996},"min":{"1min":0.41199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.754,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.40999999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.48999999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.40699999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298532,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446620,"statfs":{"total":95995035648,"available":95689338880,"internally_reserved":0,"allocated":1208320,"data_stored":872705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:28:02 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.36699999999999999,"15min":0.34499999999999997},"min":{"1min":0.246,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.35899999999999999,"15min":0.34200000000000003},"min":{"1min":0.23799999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.38900000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:27:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.40400000000000003,"15min":0.40699999999999997},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.59299999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.442}]},{"osd":3,"last update":"Wed Sep 11 07:27:55 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45800000000000002,"15min":0.47399999999999998},"min":{"1min":0.317,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.61199999999999999,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.375},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.46300000000000002,"15min":0.48099999999999998},"min":{"1min":0.28299999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.56200000000000006,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.33000000000000002}]},{"osd":4,"last update":"Wed Sep 11 07:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56299999999999994,"15min":0.56899999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69499999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59799999999999998,"15min":0.60099999999999998},"min":{"1min":0.42499999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.42899999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:27:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.40000000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.79100000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.41699999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78100000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.46100000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:27:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59399999999999997,"15min":0.59699999999999998},"min":{"1min":0.34799999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60099999999999998,"15min":0.60399999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.76400000000000001,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.496}]},{"osd":7,"last update":"Wed Sep 11 07:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.627},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.50800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:28:10.363 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:28:11.278 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:11 smithi022 ceph-mon[37330]: 4.4 scrub starts 2024-09-11T07:28:11.278 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:11 smithi022 ceph-mon[37330]: 4.4 scrub ok 2024-09-11T07:28:11.278 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:11 smithi022 ceph-mon[37330]: pgmap v48: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:28:11.278 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:11 smithi022 ceph-mon[29462]: 4.4 scrub starts 2024-09-11T07:28:11.278 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:11 smithi022 ceph-mon[29462]: 4.4 scrub ok 2024-09-11T07:28:11.279 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:11 smithi022 ceph-mon[29462]: pgmap v48: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:28:11.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:11 smithi104 ceph-mon[33507]: 4.4 scrub starts 2024-09-11T07:28:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:11 smithi104 ceph-mon[33507]: 4.4 scrub ok 2024-09-11T07:28:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:11 smithi104 ceph-mon[33507]: pgmap v48: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:28:11.897 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:28:12.020 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:11 smithi022 systemd[1]: Stopping Ceph alertmanager.a for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:28:12.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:12 smithi104 ceph-mon[33507]: 5.a scrub starts 2024-09-11T07:28:12.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:12 smithi104 ceph-mon[33507]: 5.a scrub ok 2024-09-11T07:28:12.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:12 smithi104 ceph-mon[33507]: 6.1d scrub starts 2024-09-11T07:28:12.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:12 smithi104 ceph-mon[33507]: 6.1d scrub ok 2024-09-11T07:28:12.316 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:12 smithi022 ceph-mon[37330]: 5.a scrub starts 2024-09-11T07:28:12.316 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:12 smithi022 ceph-mon[37330]: 5.a scrub ok 2024-09-11T07:28:12.316 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:12 smithi022 ceph-mon[37330]: 6.1d scrub starts 2024-09-11T07:28:12.316 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:12 smithi022 ceph-mon[37330]: 6.1d scrub ok 2024-09-11T07:28:12.316 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:12 smithi022 ceph-mon[29462]: 5.a scrub starts 2024-09-11T07:28:12.316 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:12 smithi022 ceph-mon[29462]: 5.a scrub ok 2024-09-11T07:28:12.317 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:12 smithi022 ceph-mon[29462]: 6.1d scrub starts 2024-09-11T07:28:12.317 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:12 smithi022 ceph-mon[29462]: 6.1d scrub ok 2024-09-11T07:28:13.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:13 smithi104 ceph-mon[33507]: 2.b scrub starts 2024-09-11T07:28:13.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:13 smithi104 ceph-mon[33507]: 2.b scrub ok 2024-09-11T07:28:13.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:13 smithi104 ceph-mon[33507]: pgmap v49: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 736 B/s rd, 0 op/s 2024-09-11T07:28:13.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:13 smithi104 ceph-mon[33507]: 4.3 scrub starts 2024-09-11T07:28:13.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:13 smithi104 ceph-mon[33507]: 4.3 scrub ok 2024-09-11T07:28:13.337 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:13 smithi022 ceph-mon[29462]: 2.b scrub starts 2024-09-11T07:28:13.337 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:13 smithi022 ceph-mon[29462]: 2.b scrub ok 2024-09-11T07:28:13.337 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:13 smithi022 ceph-mon[29462]: pgmap v49: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 736 B/s rd, 0 op/s 2024-09-11T07:28:13.337 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:13 smithi022 ceph-mon[29462]: 4.3 scrub starts 2024-09-11T07:28:13.337 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:13 smithi022 ceph-mon[29462]: 4.3 scrub ok 2024-09-11T07:28:13.338 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:13 smithi022 ceph-mon[37330]: 2.b scrub starts 2024-09-11T07:28:13.338 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:13 smithi022 ceph-mon[37330]: 2.b scrub ok 2024-09-11T07:28:13.338 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:13 smithi022 ceph-mon[37330]: pgmap v49: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 736 B/s rd, 0 op/s 2024-09-11T07:28:13.338 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:13 smithi022 ceph-mon[37330]: 4.3 scrub starts 2024-09-11T07:28:13.338 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:13 smithi022 ceph-mon[37330]: 4.3 scrub ok 2024-09-11T07:28:13.610 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:13 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[69934]: ts=2024-09-11T07:28:13.493Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-11T07:28:13.869 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:28:13.870 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:13 smithi022 podman[72295]: 2024-09-11 07:28:13.702557404 +0000 UTC m=+0.552003429 container died 3ef59f810f3a7aae7e632e0cbc80e5c5a17f0cde0d9e3d4a9b9ae1089ca3ae06 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:28:13.875 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:28:14.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:14 smithi022 ceph-mon[37330]: 2.2 deep-scrub starts 2024-09-11T07:28:14.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:14 smithi022 ceph-mon[37330]: 2.2 deep-scrub ok 2024-09-11T07:28:14.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:14 smithi022 ceph-mon[37330]: 5.17 scrub starts 2024-09-11T07:28:14.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:14 smithi022 ceph-mon[37330]: 5.17 scrub ok 2024-09-11T07:28:14.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:14 smithi022 ceph-mon[37330]: 2.5 scrub starts 2024-09-11T07:28:14.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:14 smithi022 ceph-mon[37330]: 2.5 scrub ok 2024-09-11T07:28:14.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:14 smithi022 ceph-mon[29462]: 2.2 deep-scrub starts 2024-09-11T07:28:14.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:14 smithi022 ceph-mon[29462]: 2.2 deep-scrub ok 2024-09-11T07:28:14.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:14 smithi022 ceph-mon[29462]: 5.17 scrub starts 2024-09-11T07:28:14.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:14 smithi022 ceph-mon[29462]: 5.17 scrub ok 2024-09-11T07:28:14.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:14 smithi022 ceph-mon[29462]: 2.5 scrub starts 2024-09-11T07:28:14.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:14 smithi022 ceph-mon[29462]: 2.5 scrub ok 2024-09-11T07:28:14.480 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:14 smithi022 podman[72295]: 2024-09-11 07:28:14.219327918 +0000 UTC m=+1.068773948 container cleanup 3ef59f810f3a7aae7e632e0cbc80e5c5a17f0cde0d9e3d4a9b9ae1089ca3ae06 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:28:14.480 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:14 smithi022 bash[72295]: ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a 2024-09-11T07:28:14.530 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:28:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:28:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:28:14.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:14 smithi104 ceph-mon[33507]: 2.2 deep-scrub starts 2024-09-11T07:28:14.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:14 smithi104 ceph-mon[33507]: 2.2 deep-scrub ok 2024-09-11T07:28:14.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:14 smithi104 ceph-mon[33507]: 5.17 scrub starts 2024-09-11T07:28:14.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:14 smithi104 ceph-mon[33507]: 5.17 scrub ok 2024-09-11T07:28:14.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:14 smithi104 ceph-mon[33507]: 2.5 scrub starts 2024-09-11T07:28:14.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:14 smithi104 ceph-mon[33507]: 2.5 scrub ok 2024-09-11T07:28:14.917 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:14 smithi022 podman[72313]: 2024-09-11 07:28:14.479096372 +0000 UTC m=+0.950234135 container remove 3ef59f810f3a7aae7e632e0cbc80e5c5a17f0cde0d9e3d4a9b9ae1089ca3ae06 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:28:14.917 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:14 smithi022 podman[72313]: 2024-09-11 07:28:14.562994351 +0000 UTC m=+1.034132077 volume remove 7f2f608811f3abc1f31f341b0570d1ce5757d7472a9e05a3dd3b33b7fd402f50 2024-09-11T07:28:15.202 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:15 smithi022 ceph-mon[29462]: 5.b scrub starts 2024-09-11T07:28:15.202 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:15 smithi022 ceph-mon[29462]: 5.b scrub ok 2024-09-11T07:28:15.202 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:15 smithi022 ceph-mon[29462]: 6.7 scrub starts 2024-09-11T07:28:15.202 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:15 smithi022 ceph-mon[29462]: 6.7 scrub ok 2024-09-11T07:28:15.202 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:15 smithi022 ceph-mon[29462]: from='client.14772 v1:172.21.15.22:0/4156975087' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:28:15.203 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:15 smithi022 ceph-mon[29462]: 6.8 scrub starts 2024-09-11T07:28:15.203 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:15 smithi022 ceph-mon[29462]: 6.8 scrub ok 2024-09-11T07:28:15.203 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:15 smithi022 ceph-mon[29462]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 2 objects/s recovering 2024-09-11T07:28:15.203 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:15 smithi022 ceph-mon[37330]: 5.b scrub starts 2024-09-11T07:28:15.203 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:15 smithi022 ceph-mon[37330]: 5.b scrub ok 2024-09-11T07:28:15.203 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:15 smithi022 ceph-mon[37330]: 6.7 scrub starts 2024-09-11T07:28:15.203 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:15 smithi022 ceph-mon[37330]: 6.7 scrub ok 2024-09-11T07:28:15.203 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:15 smithi022 ceph-mon[37330]: from='client.14772 v1:172.21.15.22:0/4156975087' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:28:15.203 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:15 smithi022 ceph-mon[37330]: 6.8 scrub starts 2024-09-11T07:28:15.203 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:15 smithi022 ceph-mon[37330]: 6.8 scrub ok 2024-09-11T07:28:15.203 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:15 smithi022 ceph-mon[37330]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 2 objects/s recovering 2024-09-11T07:28:15.232 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":49,"stamp":"2024-09-11T07:28:11.947327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":944,"num_read_kb":801,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":580,"ondisk_log_size":580,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385556,"kb_used_data":6688,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575660,"statfs":{"total":767960285184,"available":765517475840,"internally_reserved":0,"allocated":6848512,"data_stored":4101585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.947775"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.652094+0000","last_change":"2024-09-11T07:26:37.863505+0000","last_active":"2024-09-11T07:28:05.652094+0000","last_peered":"2024-09-11T07:28:05.652094+0000","last_clean":"2024-09-11T07:28:05.652094+0000","last_became_active":"2024-09-11T07:26:37.863165+0000","last_became_peered":"2024-09-11T07:26:37.863165+0000","last_unstale":"2024-09-11T07:28:05.652094+0000","last_undegraded":"2024-09-11T07:28:05.652094+0000","last_fullsized":"2024-09-11T07:28:05.652094+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.002992+0000","last_change":"2024-09-11T07:27:51.197966+0000","last_active":"2024-09-11T07:28:05.002992+0000","last_peered":"2024-09-11T07:28:05.002992+0000","last_clean":"2024-09-11T07:28:05.002992+0000","last_became_active":"2024-09-11T07:26:31.421005+0000","last_became_peered":"2024-09-11T07:26:31.421005+0000","last_unstale":"2024-09-11T07:28:05.002992+0000","last_undegraded":"2024-09-11T07:28:05.002992+0000","last_fullsized":"2024-09-11T07:28:05.002992+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:51.197903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:51.197903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:00.681709+0000","scrub_duration":0.001600113,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"82'22","reported_seq":77,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.651979+0000","last_change":"2024-09-11T07:27:50.756777+0000","last_active":"2024-09-11T07:28:05.651979+0000","last_peered":"2024-09-11T07:28:05.651979+0000","last_clean":"2024-09-11T07:28:05.651979+0000","last_became_active":"2024-09-11T07:26:33.544627+0000","last_became_peered":"2024-09-11T07:26:33.544627+0000","last_unstale":"2024-09-11T07:28:05.651979+0000","last_undegraded":"2024-09-11T07:28:05.651979+0000","last_fullsized":"2024-09-11T07:28:05.651979+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.756707+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.257406+0000","scrub_duration":0.0049180669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.652763+0000","last_change":"2024-09-11T07:26:35.557117+0000","last_active":"2024-09-11T07:28:05.652763+0000","last_peered":"2024-09-11T07:28:05.652763+0000","last_clean":"2024-09-11T07:28:05.652763+0000","last_became_active":"2024-09-11T07:26:35.556296+0000","last_became_peered":"2024-09-11T07:26:35.556296+0000","last_unstale":"2024-09-11T07:28:05.652763+0000","last_undegraded":"2024-09-11T07:28:05.652763+0000","last_fullsized":"2024-09-11T07:28:05.652763+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.652883+0000","last_change":"2024-09-11T07:27:36.776948+0000","last_active":"2024-09-11T07:28:05.652883+0000","last_peered":"2024-09-11T07:28:05.652883+0000","last_clean":"2024-09-11T07:28:05.652883+0000","last_became_active":"2024-09-11T07:26:31.420178+0000","last_became_peered":"2024-09-11T07:26:31.420178+0000","last_unstale":"2024-09-11T07:28:05.652883+0000","last_undegraded":"2024-09-11T07:28:05.652883+0000","last_fullsized":"2024-09-11T07:28:05.652883+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:36.776883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:36.776883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:49.961787+0000","scrub_duration":0.0017449290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"64'9","reported_seq":51,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.652851+0000","last_change":"2024-09-11T07:26:33.541403+0000","last_active":"2024-09-11T07:28:05.652851+0000","last_peered":"2024-09-11T07:28:05.652851+0000","last_clean":"2024-09-11T07:28:05.652851+0000","last_became_active":"2024-09-11T07:26:33.540732+0000","last_became_peered":"2024-09-11T07:26:33.540732+0000","last_unstale":"2024-09-11T07:28:05.652851+0000","last_undegraded":"2024-09-11T07:28:05.652851+0000","last_fullsized":"2024-09-11T07:28:05.652851+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.001402+0000","last_change":"2024-09-11T07:27:50.648473+0000","last_active":"2024-09-11T07:28:05.001402+0000","last_peered":"2024-09-11T07:28:05.001402+0000","last_clean":"2024-09-11T07:28:05.001402+0000","last_became_active":"2024-09-11T07:26:35.561096+0000","last_became_peered":"2024-09-11T07:26:35.561096+0000","last_unstale":"2024-09-11T07:28:05.001402+0000","last_undegraded":"2024-09-11T07:28:05.001402+0000","last_fullsized":"2024-09-11T07:28:05.001402+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.648420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:05.663592+0000","scrub_duration":0.001391446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.652338+0000","last_change":"2024-09-11T07:26:37.567195+0000","last_active":"2024-09-11T07:28:05.652338+0000","last_peered":"2024-09-11T07:28:05.652338+0000","last_clean":"2024-09-11T07:28:05.652338+0000","last_became_active":"2024-09-11T07:26:37.566910+0000","last_became_peered":"2024-09-11T07:26:37.566910+0000","last_unstale":"2024-09-11T07:28:05.652338+0000","last_undegraded":"2024-09-11T07:28:05.652338+0000","last_fullsized":"2024-09-11T07:28:05.652338+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.800346+0000","last_change":"2024-09-11T07:26:31.423041+0000","last_active":"2024-09-11T07:28:05.800346+0000","last_peered":"2024-09-11T07:28:05.800346+0000","last_clean":"2024-09-11T07:28:05.800346+0000","last_became_active":"2024-09-11T07:26:31.422772+0000","last_became_peered":"2024-09-11T07:26:31.422772+0000","last_unstale":"2024-09-11T07:28:05.800346+0000","last_undegraded":"2024-09-11T07:28:05.800346+0000","last_fullsized":"2024-09-11T07:28:05.800346+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"82'7","reported_seq":57,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.652614+0000","last_change":"2024-09-11T07:27:52.097170+0000","last_active":"2024-09-11T07:28:05.652614+0000","last_peered":"2024-09-11T07:28:05.652614+0000","last_clean":"2024-09-11T07:28:05.652614+0000","last_became_active":"2024-09-11T07:26:33.540351+0000","last_became_peered":"2024-09-11T07:26:33.540351+0000","last_unstale":"2024-09-11T07:28:05.652614+0000","last_undegraded":"2024-09-11T07:28:05.652614+0000","last_fullsized":"2024-09-11T07:28:05.652614+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'7","last_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_deep_scrub":"82'7","last_deep_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_clean_scrub_stamp":"2024-09-11T07:27:52.097071+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:53.905741+0000","scrub_duration":0.0025786670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.799974+0000","last_change":"2024-09-11T07:26:35.563858+0000","last_active":"2024-09-11T07:28:05.799974+0000","last_peered":"2024-09-11T07:28:05.799974+0000","last_clean":"2024-09-11T07:28:05.799974+0000","last_became_active":"2024-09-11T07:26:35.563536+0000","last_became_peered":"2024-09-11T07:26:35.563536+0000","last_unstale":"2024-09-11T07:28:05.799974+0000","last_undegraded":"2024-09-11T07:28:05.799974+0000","last_fullsized":"2024-09-11T07:28:05.799974+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.895052+0000","last_change":"2024-09-11T07:26:37.562008+0000","last_active":"2024-09-11T07:28:05.895052+0000","last_peered":"2024-09-11T07:28:05.895052+0000","last_clean":"2024-09-11T07:28:05.895052+0000","last_became_active":"2024-09-11T07:26:37.561758+0000","last_became_peered":"2024-09-11T07:26:37.561758+0000","last_unstale":"2024-09-11T07:28:05.895052+0000","last_undegraded":"2024-09-11T07:28:05.895052+0000","last_fullsized":"2024-09-11T07:28:05.895052+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.800365+0000","last_change":"2024-09-11T07:27:52.058013+0000","last_active":"2024-09-11T07:28:05.800365+0000","last_peered":"2024-09-11T07:28:05.800365+0000","last_clean":"2024-09-11T07:28:05.800365+0000","last_became_active":"2024-09-11T07:26:31.420969+0000","last_became_peered":"2024-09-11T07:26:31.420969+0000","last_unstale":"2024-09-11T07:28:05.800365+0000","last_undegraded":"2024-09-11T07:28:05.800365+0000","last_fullsized":"2024-09-11T07:28:05.800365+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_clean_scrub_stamp":"2024-09-11T07:27:52.057959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:12.902027+0000","scrub_duration":0.0014644840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"64'9","reported_seq":61,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.652609+0000","last_change":"2024-09-11T07:27:47.108704+0000","last_active":"2024-09-11T07:28:05.652609+0000","last_peered":"2024-09-11T07:28:05.652609+0000","last_clean":"2024-09-11T07:28:05.652609+0000","last_became_active":"2024-09-11T07:26:33.544386+0000","last_became_peered":"2024-09-11T07:26:33.544386+0000","last_unstale":"2024-09-11T07:28:05.652609+0000","last_undegraded":"2024-09-11T07:28:05.652609+0000","last_fullsized":"2024-09-11T07:28:05.652609+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:27:47.108645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.108645+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:08.302050+0000","scrub_duration":0.0024650560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.895588+0000","last_change":"2024-09-11T07:27:57.092029+0000","last_active":"2024-09-11T07:28:05.895588+0000","last_peered":"2024-09-11T07:28:05.895588+0000","last_clean":"2024-09-11T07:28:05.895588+0000","last_became_active":"2024-09-11T07:26:35.569630+0000","last_became_peered":"2024-09-11T07:26:35.569630+0000","last_unstale":"2024-09-11T07:28:05.895588+0000","last_undegraded":"2024-09-11T07:28:05.895588+0000","last_fullsized":"2024-09-11T07:28:05.895588+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:57.091958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.091958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:57.303120+0000","scrub_duration":0.001785872,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.001369+0000","last_change":"2024-09-11T07:26:37.565826+0000","last_active":"2024-09-11T07:28:05.001369+0000","last_peered":"2024-09-11T07:28:05.001369+0000","last_clean":"2024-09-11T07:28:05.001369+0000","last_became_active":"2024-09-11T07:26:37.565475+0000","last_became_peered":"2024-09-11T07:26:37.565475+0000","last_unstale":"2024-09-11T07:28:05.001369+0000","last_undegraded":"2024-09-11T07:28:05.001369+0000","last_fullsized":"2024-09-11T07:28:05.001369+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":75,"reported_epoch":85,"state":"peering","last_fresh":"2024-09-11T07:28:08.009327+0000","last_change":"2024-09-11T07:28:08.009499+0000","last_active":"2024-09-11T07:28:08.009306+0000","last_peered":"2024-09-11T07:28:05.653064+0000","last_clean":"2024-09-11T07:28:05.653064+0000","last_became_active":"2024-09-11T07:26:33.540575+0000","last_became_peered":"2024-09-11T07:26:33.540575+0000","last_unstale":"2024-09-11T07:28:08.009327+0000","last_undegraded":"2024-09-11T07:28:08.009327+0000","last_fullsized":"2024-09-11T07:28:08.009327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-09-11T07:28:04.137453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:04.137453+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:05.411033+0000","scrub_duration":0.001914083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2,5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.800280+0000","last_change":"2024-09-11T07:26:31.420109+0000","last_active":"2024-09-11T07:28:05.800280+0000","last_peered":"2024-09-11T07:28:05.800280+0000","last_clean":"2024-09-11T07:28:05.800280+0000","last_became_active":"2024-09-11T07:26:31.419488+0000","last_became_peered":"2024-09-11T07:26:31.419488+0000","last_unstale":"2024-09-11T07:28:05.800280+0000","last_undegraded":"2024-09-11T07:28:05.800280+0000","last_fullsized":"2024-09-11T07:28:05.800280+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.002186+0000","last_change":"2024-09-11T07:26:35.559684+0000","last_active":"2024-09-11T07:28:05.002186+0000","last_peered":"2024-09-11T07:28:05.002186+0000","last_clean":"2024-09-11T07:28:05.002186+0000","last_became_active":"2024-09-11T07:26:35.557989+0000","last_became_peered":"2024-09-11T07:26:35.557989+0000","last_unstale":"2024-09-11T07:28:05.002186+0000","last_undegraded":"2024-09-11T07:28:05.002186+0000","last_fullsized":"2024-09-11T07:28:05.002186+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:06.304906+0000","last_change":"2024-09-11T07:27:57.274611+0000","last_active":"2024-09-11T07:28:06.304906+0000","last_peered":"2024-09-11T07:28:06.304906+0000","last_clean":"2024-09-11T07:28:06.304906+0000","last_became_active":"2024-09-11T07:26:37.564554+0000","last_became_peered":"2024-09-11T07:26:37.564554+0000","last_unstale":"2024-09-11T07:28:06.304906+0000","last_undegraded":"2024-09-11T07:28:06.304906+0000","last_fullsized":"2024-09-11T07:28:06.304906+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:57.274554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.274554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.058396+0000","scrub_duration":0.001419584,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":50,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.652807+0000","last_change":"2024-09-11T07:26:33.536333+0000","last_active":"2024-09-11T07:28:05.652807+0000","last_peered":"2024-09-11T07:28:05.652807+0000","last_clean":"2024-09-11T07:28:05.652807+0000","last_became_active":"2024-09-11T07:26:33.535411+0000","last_became_peered":"2024-09-11T07:26:33.535411+0000","last_unstale":"2024-09-11T07:28:05.652807+0000","last_undegraded":"2024-09-11T07:28:05.652807+0000","last_fullsized":"2024-09-11T07:28:05.652807+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.002320+0000","last_change":"2024-09-11T07:26:31.419259+0000","last_active":"2024-09-11T07:28:05.002320+0000","last_peered":"2024-09-11T07:28:05.002320+0000","last_clean":"2024-09-11T07:28:05.002320+0000","last_became_active":"2024-09-11T07:26:31.418178+0000","last_became_peered":"2024-09-11T07:26:31.418178+0000","last_unstale":"2024-09-11T07:28:05.002320+0000","last_undegraded":"2024-09-11T07:28:05.002320+0000","last_fullsized":"2024-09-11T07:28:05.002320+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"64'8","reported_seq":70,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:06.304778+0000","last_change":"2024-09-11T07:27:47.127485+0000","last_active":"2024-09-11T07:28:06.304778+0000","last_peered":"2024-09-11T07:28:06.304778+0000","last_clean":"2024-09-11T07:28:06.304778+0000","last_became_active":"2024-09-11T07:26:35.564531+0000","last_became_peered":"2024-09-11T07:26:35.564531+0000","last_unstale":"2024-09-11T07:28:06.304778+0000","last_undegraded":"2024-09-11T07:28:06.304778+0000","last_fullsized":"2024-09-11T07:28:06.304778+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:27:47.127428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.127428+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:58.343731+0000","scrub_duration":0.0014091329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.652341+0000","last_change":"2024-09-11T07:26:37.567193+0000","last_active":"2024-09-11T07:28:05.652341+0000","last_peered":"2024-09-11T07:28:05.652341+0000","last_clean":"2024-09-11T07:28:05.652341+0000","last_became_active":"2024-09-11T07:26:37.566919+0000","last_became_peered":"2024-09-11T07:26:37.566919+0000","last_unstale":"2024-09-11T07:28:05.652341+0000","last_undegraded":"2024-09-11T07:28:05.652341+0000","last_fullsized":"2024-09-11T07:28:05.652341+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"83'22","reported_seq":77,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.003196+0000","last_change":"2024-09-11T07:28:01.719185+0000","last_active":"2024-09-11T07:28:05.003196+0000","last_peered":"2024-09-11T07:28:05.003196+0000","last_clean":"2024-09-11T07:28:05.003196+0000","last_became_active":"2024-09-11T07:26:33.549733+0000","last_became_peered":"2024-09-11T07:26:33.549733+0000","last_unstale":"2024-09-11T07:28:05.003196+0000","last_undegraded":"2024-09-11T07:28:05.003196+0000","last_fullsized":"2024-09-11T07:28:05.003196+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'22","last_scrub_stamp":"2024-09-11T07:28:01.719128+0000","last_deep_scrub":"83'22","last_deep_scrub_stamp":"2024-09-11T07:28:01.719128+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.719128+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:27.431086+0000","scrub_duration":0.0047817240000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.001604+0000","last_change":"2024-09-11T07:27:35.589317+0000","last_active":"2024-09-11T07:28:05.001604+0000","last_peered":"2024-09-11T07:28:05.001604+0000","last_clean":"2024-09-11T07:28:05.001604+0000","last_became_active":"2024-09-11T07:26:31.418478+0000","last_became_peered":"2024-09-11T07:26:31.418478+0000","last_unstale":"2024-09-11T07:28:05.001604+0000","last_undegraded":"2024-09-11T07:28:05.001604+0000","last_fullsized":"2024-09-11T07:28:05.001604+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:35.589251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:35.589251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:01.839680+0000","scrub_duration":0.0014929380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":73,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.653101+0000","last_change":"2024-09-11T07:27:56.218355+0000","last_active":"2024-09-11T07:28:05.653101+0000","last_peered":"2024-09-11T07:28:05.653101+0000","last_clean":"2024-09-11T07:28:05.653101+0000","last_became_active":"2024-09-11T07:26:35.560406+0000","last_became_peered":"2024-09-11T07:26:35.560406+0000","last_unstale":"2024-09-11T07:28:05.653101+0000","last_undegraded":"2024-09-11T07:28:05.653101+0000","last_fullsized":"2024-09-11T07:28:05.653101+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:27:56.218301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:56.218301+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:21.374837+0000","scrub_duration":0.001687413,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.895146+0000","last_change":"2024-09-11T07:26:37.568289+0000","last_active":"2024-09-11T07:28:05.895146+0000","last_peered":"2024-09-11T07:28:05.895146+0000","last_clean":"2024-09-11T07:28:05.895146+0000","last_became_active":"2024-09-11T07:26:37.568005+0000","last_became_peered":"2024-09-11T07:26:37.568005+0000","last_unstale":"2024-09-11T07:28:05.895146+0000","last_undegraded":"2024-09-11T07:28:05.895146+0000","last_fullsized":"2024-09-11T07:28:05.895146+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"64'11","reported_seq":64,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.653126+0000","last_change":"2024-09-11T07:27:58.158485+0000","last_active":"2024-09-11T07:28:05.653126+0000","last_peered":"2024-09-11T07:28:05.653126+0000","last_clean":"2024-09-11T07:28:05.653126+0000","last_became_active":"2024-09-11T07:26:33.540777+0000","last_became_peered":"2024-09-11T07:26:33.540777+0000","last_unstale":"2024-09-11T07:28:05.653126+0000","last_undegraded":"2024-09-11T07:28:05.653126+0000","last_fullsized":"2024-09-11T07:28:05.653126+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:27:58.158420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.158420+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.832014+0000","scrub_duration":0.0021737459999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.799914+0000","last_change":"2024-09-11T07:27:34.932151+0000","last_active":"2024-09-11T07:28:05.799914+0000","last_peered":"2024-09-11T07:28:05.799914+0000","last_clean":"2024-09-11T07:28:05.799914+0000","last_became_active":"2024-09-11T07:26:31.418238+0000","last_became_peered":"2024-09-11T07:26:31.418238+0000","last_unstale":"2024-09-11T07:28:05.799914+0000","last_undegraded":"2024-09-11T07:28:05.799914+0000","last_fullsized":"2024-09-11T07:28:05.799914+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:34.932092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:34.932092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:46.904196+0000","scrub_duration":0.002710259,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.895970+0000","last_change":"2024-09-11T07:28:00.005119+0000","last_active":"2024-09-11T07:28:05.895970+0000","last_peered":"2024-09-11T07:28:05.895970+0000","last_clean":"2024-09-11T07:28:05.895970+0000","last_became_active":"2024-09-11T07:26:35.569081+0000","last_became_peered":"2024-09-11T07:26:35.569081+0000","last_unstale":"2024-09-11T07:28:05.895970+0000","last_undegraded":"2024-09-11T07:28:05.895970+0000","last_fullsized":"2024-09-11T07:28:05.895970+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:00.005031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.005031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:10.748420+0000","scrub_duration":0.001403533,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.652242+0000","last_change":"2024-09-11T07:27:46.752272+0000","last_active":"2024-09-11T07:28:05.652242+0000","last_peered":"2024-09-11T07:28:05.652242+0000","last_clean":"2024-09-11T07:28:05.652242+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:28:05.652242+0000","last_undegraded":"2024-09-11T07:28:05.652242+0000","last_fullsized":"2024-09-11T07:28:05.652242+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:46.752181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:46.752181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.072860+0000","scrub_duration":0.001508782,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"64'14","reported_seq":56,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.652693+0000","last_change":"2024-09-11T07:26:33.538152+0000","last_active":"2024-09-11T07:28:05.652693+0000","last_peered":"2024-09-11T07:28:05.652693+0000","last_clean":"2024-09-11T07:28:05.652693+0000","last_became_active":"2024-09-11T07:26:33.537850+0000","last_became_peered":"2024-09-11T07:26:33.537850+0000","last_unstale":"2024-09-11T07:28:05.652693+0000","last_undegraded":"2024-09-11T07:28:05.652693+0000","last_fullsized":"2024-09-11T07:28:05.652693+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":49,"reported_epoch":85,"state":"peering","last_fresh":"2024-09-11T07:28:08.011738+0000","last_change":"2024-09-11T07:28:08.012000+0000","last_active":"2024-09-11T07:28:08.011708+0000","last_peered":"2024-09-11T07:28:05.001461+0000","last_clean":"2024-09-11T07:28:05.001461+0000","last_became_active":"2024-09-11T07:26:31.423905+0000","last_became_peered":"2024-09-11T07:26:31.423905+0000","last_unstale":"2024-09-11T07:28:08.011738+0000","last_undegraded":"2024-09-11T07:28:08.011738+0000","last_fullsized":"2024-09-11T07:28:08.011738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:39.014840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:39.014840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:45.078792+0000","scrub_duration":0.0032789239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"84'9","reported_seq":71,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.653329+0000","last_change":"2024-09-11T07:28:05.653289+0000","last_active":"2024-09-11T07:28:05.653329+0000","last_peered":"2024-09-11T07:28:05.653329+0000","last_clean":"2024-09-11T07:28:05.653329+0000","last_became_active":"2024-09-11T07:26:35.562997+0000","last_became_peered":"2024-09-11T07:26:35.562997+0000","last_unstale":"2024-09-11T07:28:05.653329+0000","last_undegraded":"2024-09-11T07:28:05.653329+0000","last_fullsized":"2024-09-11T07:28:05.653329+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-11T07:28:05.653237+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-11T07:28:05.653237+0000","last_clean_scrub_stamp":"2024-09-11T07:28:05.653237+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:22.713095+0000","scrub_duration":0.0027957030000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.800062+0000","last_change":"2024-09-11T07:26:37.862564+0000","last_active":"2024-09-11T07:28:05.800062+0000","last_peered":"2024-09-11T07:28:05.800062+0000","last_clean":"2024-09-11T07:28:05.800062+0000","last_became_active":"2024-09-11T07:26:37.862246+0000","last_became_peered":"2024-09-11T07:26:37.862246+0000","last_unstale":"2024-09-11T07:28:05.800062+0000","last_undegraded":"2024-09-11T07:28:05.800062+0000","last_fullsized":"2024-09-11T07:28:05.800062+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"64'18","reported_seq":67,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.651747+0000","last_change":"2024-09-11T07:26:33.546711+0000","last_active":"2024-09-11T07:28:05.651747+0000","last_peered":"2024-09-11T07:28:05.651747+0000","last_clean":"2024-09-11T07:28:05.651747+0000","last_became_active":"2024-09-11T07:26:33.546188+0000","last_became_peered":"2024-09-11T07:26:33.546188+0000","last_unstale":"2024-09-11T07:28:05.651747+0000","last_undegraded":"2024-09-11T07:28:05.651747+0000","last_fullsized":"2024-09-11T07:28:05.651747+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.002637+0000","last_change":"2024-09-11T07:26:31.421802+0000","last_active":"2024-09-11T07:28:05.002637+0000","last_peered":"2024-09-11T07:28:05.002637+0000","last_clean":"2024-09-11T07:28:05.002637+0000","last_became_active":"2024-09-11T07:26:31.421500+0000","last_became_peered":"2024-09-11T07:26:31.421500+0000","last_unstale":"2024-09-11T07:28:05.002637+0000","last_undegraded":"2024-09-11T07:28:05.002637+0000","last_fullsized":"2024-09-11T07:28:05.002637+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.652535+0000","last_change":"2024-09-11T07:26:35.562816+0000","last_active":"2024-09-11T07:28:05.652535+0000","last_peered":"2024-09-11T07:28:05.652535+0000","last_clean":"2024-09-11T07:28:05.652535+0000","last_became_active":"2024-09-11T07:26:35.561284+0000","last_became_peered":"2024-09-11T07:26:35.561284+0000","last_unstale":"2024-09-11T07:28:05.652535+0000","last_undegraded":"2024-09-11T07:28:05.652535+0000","last_fullsized":"2024-09-11T07:28:05.652535+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.652586+0000","last_change":"2024-09-11T07:26:37.565014+0000","last_active":"2024-09-11T07:28:05.652586+0000","last_peered":"2024-09-11T07:28:05.652586+0000","last_clean":"2024-09-11T07:28:05.652586+0000","last_became_active":"2024-09-11T07:26:37.564616+0000","last_became_peered":"2024-09-11T07:26:37.564616+0000","last_unstale":"2024-09-11T07:28:05.652586+0000","last_undegraded":"2024-09-11T07:28:05.652586+0000","last_fullsized":"2024-09-11T07:28:05.652586+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":71,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.003329+0000","last_change":"2024-09-11T07:26:33.541012+0000","last_active":"2024-09-11T07:28:05.003329+0000","last_peered":"2024-09-11T07:28:05.003329+0000","last_clean":"2024-09-11T07:28:05.003329+0000","last_became_active":"2024-09-11T07:26:33.540639+0000","last_became_peered":"2024-09-11T07:26:33.540639+0000","last_unstale":"2024-09-11T07:28:05.003329+0000","last_undegraded":"2024-09-11T07:28:05.003329+0000","last_fullsized":"2024-09-11T07:28:05.003329+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.800387+0000","last_change":"2024-09-11T07:26:31.419925+0000","last_active":"2024-09-11T07:28:05.800387+0000","last_peered":"2024-09-11T07:28:05.800387+0000","last_clean":"2024-09-11T07:28:05.800387+0000","last_became_active":"2024-09-11T07:26:31.419185+0000","last_became_peered":"2024-09-11T07:26:31.419185+0000","last_unstale":"2024-09-11T07:28:05.800387+0000","last_undegraded":"2024-09-11T07:28:05.800387+0000","last_fullsized":"2024-09-11T07:28:05.800387+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.001146+0000","last_change":"2024-09-11T07:27:41.000469+0000","last_active":"2024-09-11T07:28:05.001146+0000","last_peered":"2024-09-11T07:28:05.001146+0000","last_clean":"2024-09-11T07:28:05.001146+0000","last_became_active":"2024-09-11T07:26:35.565359+0000","last_became_peered":"2024-09-11T07:26:35.565359+0000","last_unstale":"2024-09-11T07:28:05.001146+0000","last_undegraded":"2024-09-11T07:28:05.001146+0000","last_fullsized":"2024-09-11T07:28:05.001146+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:41.000401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:41.000401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.049825+0000","scrub_duration":0.001655164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.001317+0000","last_change":"2024-09-11T07:26:37.861736+0000","last_active":"2024-09-11T07:28:05.001317+0000","last_peered":"2024-09-11T07:28:05.001317+0000","last_clean":"2024-09-11T07:28:05.001317+0000","last_became_active":"2024-09-11T07:26:37.861279+0000","last_became_peered":"2024-09-11T07:26:37.861279+0000","last_unstale":"2024-09-11T07:28:05.001317+0000","last_undegraded":"2024-09-11T07:28:05.001317+0000","last_fullsized":"2024-09-11T07:28:05.001317+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.002098+0000","last_change":"2024-09-11T07:27:51.642529+0000","last_active":"2024-09-11T07:28:05.002098+0000","last_peered":"2024-09-11T07:28:05.002098+0000","last_clean":"2024-09-11T07:28:05.002098+0000","last_became_active":"2024-09-11T07:26:33.543063+0000","last_became_peered":"2024-09-11T07:26:33.543063+0000","last_unstale":"2024-09-11T07:28:05.002098+0000","last_undegraded":"2024-09-11T07:28:05.002098+0000","last_fullsized":"2024-09-11T07:28:05.002098+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:27:51.642464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:51.642464+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:13.146284+0000","scrub_duration":0.0020211439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":54,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.002485+0000","last_change":"2024-09-11T07:27:41.618362+0000","last_active":"2024-09-11T07:28:05.002485+0000","last_peered":"2024-09-11T07:28:05.002485+0000","last_clean":"2024-09-11T07:28:05.002485+0000","last_became_active":"2024-09-11T07:26:31.422110+0000","last_became_peered":"2024-09-11T07:26:31.422110+0000","last_unstale":"2024-09-11T07:28:05.002485+0000","last_undegraded":"2024-09-11T07:28:05.002485+0000","last_fullsized":"2024-09-11T07:28:05.002485+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:27:41.618296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:41.618296+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:57.404591+0000","scrub_duration":0.0020650450000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":70,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.003009+0000","last_change":"2024-09-11T07:28:03.183056+0000","last_active":"2024-09-11T07:28:05.003009+0000","last_peered":"2024-09-11T07:28:05.003009+0000","last_clean":"2024-09-11T07:28:05.003009+0000","last_became_active":"2024-09-11T07:26:35.560422+0000","last_became_peered":"2024-09-11T07:26:35.560422+0000","last_unstale":"2024-09-11T07:28:05.003009+0000","last_undegraded":"2024-09-11T07:28:05.003009+0000","last_fullsized":"2024-09-11T07:28:05.003009+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:28:03.182993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:03.182993+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:26.896461+0000","scrub_duration":0.0018949889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.001180+0000","last_change":"2024-09-11T07:27:54.148594+0000","last_active":"2024-09-11T07:28:05.001180+0000","last_peered":"2024-09-11T07:28:05.001180+0000","last_clean":"2024-09-11T07:28:05.001180+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:28:05.001180+0000","last_undegraded":"2024-09-11T07:28:05.001180+0000","last_fullsized":"2024-09-11T07:28:05.001180+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.148534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:02.854448+0000","scrub_duration":0.0013536620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"64'28","reported_seq":102,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.002346+0000","last_change":"2024-09-11T07:26:33.549518+0000","last_active":"2024-09-11T07:28:05.002346+0000","last_peered":"2024-09-11T07:28:05.002346+0000","last_clean":"2024-09-11T07:28:05.002346+0000","last_became_active":"2024-09-11T07:26:33.549234+0000","last_became_peered":"2024-09-11T07:26:33.549234+0000","last_unstale":"2024-09-11T07:28:05.002346+0000","last_undegraded":"2024-09-11T07:28:05.002346+0000","last_fullsized":"2024-09-11T07:28:05.002346+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.895642+0000","last_change":"2024-09-11T07:26:31.424898+0000","last_active":"2024-09-11T07:28:05.895642+0000","last_peered":"2024-09-11T07:28:05.895642+0000","last_clean":"2024-09-11T07:28:05.895642+0000","last_became_active":"2024-09-11T07:26:31.424618+0000","last_became_peered":"2024-09-11T07:26:31.424618+0000","last_unstale":"2024-09-11T07:28:05.895642+0000","last_undegraded":"2024-09-11T07:28:05.895642+0000","last_fullsized":"2024-09-11T07:28:05.895642+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.002489+0000","last_change":"2024-09-11T07:26:35.560677+0000","last_active":"2024-09-11T07:28:05.002489+0000","last_peered":"2024-09-11T07:28:05.002489+0000","last_clean":"2024-09-11T07:28:05.002489+0000","last_became_active":"2024-09-11T07:26:35.560363+0000","last_became_peered":"2024-09-11T07:26:35.560363+0000","last_unstale":"2024-09-11T07:28:05.002489+0000","last_undegraded":"2024-09-11T07:28:05.002489+0000","last_fullsized":"2024-09-11T07:28:05.002489+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.652802+0000","last_change":"2024-09-11T07:28:00.686634+0000","last_active":"2024-09-11T07:28:05.652802+0000","last_peered":"2024-09-11T07:28:05.652802+0000","last_clean":"2024-09-11T07:28:05.652802+0000","last_became_active":"2024-09-11T07:26:37.562452+0000","last_became_peered":"2024-09-11T07:26:37.562452+0000","last_unstale":"2024-09-11T07:28:05.652802+0000","last_undegraded":"2024-09-11T07:28:05.652802+0000","last_fullsized":"2024-09-11T07:28:05.652802+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:00.686571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.686571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.798861+0000","scrub_duration":0.0016714309999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"64'13","reported_seq":62,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.003281+0000","last_change":"2024-09-11T07:26:33.547483+0000","last_active":"2024-09-11T07:28:05.003281+0000","last_peered":"2024-09-11T07:28:05.003281+0000","last_clean":"2024-09-11T07:28:05.003281+0000","last_became_active":"2024-09-11T07:26:33.546126+0000","last_became_peered":"2024-09-11T07:26:33.546126+0000","last_unstale":"2024-09-11T07:28:05.003281+0000","last_undegraded":"2024-09-11T07:28:05.003281+0000","last_fullsized":"2024-09-11T07:28:05.003281+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":146,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012125+0000","last_change":"2024-09-11T07:27:59.678930+0000","last_active":"2024-09-11T07:28:08.012125+0000","last_peered":"2024-09-11T07:28:08.012125+0000","last_clean":"2024-09-11T07:28:08.012125+0000","last_became_active":"2024-09-11T07:26:31.421276+0000","last_became_peered":"2024-09-11T07:26:31.421276+0000","last_unstale":"2024-09-11T07:28:08.012125+0000","last_undegraded":"2024-09-11T07:28:08.012125+0000","last_fullsized":"2024-09-11T07:28:08.012125+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'5","last_scrub_stamp":"2024-09-11T07:27:59.678865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:59.678865+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.114929+0000","scrub_duration":0.001792486,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":91,"num_read_kb":86,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:07.327759+0000","last_change":"2024-09-11T07:28:07.327716+0000","last_active":"2024-09-11T07:28:07.327759+0000","last_peered":"2024-09-11T07:28:07.327759+0000","last_clean":"2024-09-11T07:28:07.327759+0000","last_became_active":"2024-09-11T07:26:31.413354+0000","last_became_peered":"2024-09-11T07:26:31.413354+0000","last_unstale":"2024-09-11T07:28:07.327759+0000","last_undegraded":"2024-09-11T07:28:07.327759+0000","last_fullsized":"2024-09-11T07:28:07.327759+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:07.327648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:07.327648+0000","last_clean_scrub_stamp":"2024-09-11T07:28:07.327648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:35.306829+0000","scrub_duration":0.0011895899999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:06.304821+0000","last_change":"2024-09-11T07:27:42.141156+0000","last_active":"2024-09-11T07:28:06.304821+0000","last_peered":"2024-09-11T07:28:06.304821+0000","last_clean":"2024-09-11T07:28:06.304821+0000","last_became_active":"2024-09-11T07:26:35.565849+0000","last_became_peered":"2024-09-11T07:26:35.565849+0000","last_unstale":"2024-09-11T07:28:06.304821+0000","last_undegraded":"2024-09-11T07:28:06.304821+0000","last_fullsized":"2024-09-11T07:28:06.304821+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.141100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.141100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.707826+0000","scrub_duration":0.001495672,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.800191+0000","last_change":"2024-09-11T07:27:55.018180+0000","last_active":"2024-09-11T07:28:05.800191+0000","last_peered":"2024-09-11T07:28:05.800191+0000","last_clean":"2024-09-11T07:28:05.800191+0000","last_became_active":"2024-09-11T07:26:37.861292+0000","last_became_peered":"2024-09-11T07:26:37.861292+0000","last_unstale":"2024-09-11T07:28:05.800191+0000","last_undegraded":"2024-09-11T07:28:05.800191+0000","last_fullsized":"2024-09-11T07:28:05.800191+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:55.018125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:55.018125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.234213+0000","scrub_duration":0.0013534339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":65,"reported_epoch":85,"state":"peering","last_fresh":"2024-09-11T07:28:08.014002+0000","last_change":"2024-09-11T07:28:08.014255+0000","last_active":"2024-09-11T07:28:08.013969+0000","last_peered":"2024-09-11T07:28:05.003501+0000","last_clean":"2024-09-11T07:28:05.003501+0000","last_became_active":"2024-09-11T07:26:33.548903+0000","last_became_peered":"2024-09-11T07:26:33.548903+0000","last_unstale":"2024-09-11T07:28:08.014002+0000","last_undegraded":"2024-09-11T07:28:08.014002+0000","last_fullsized":"2024-09-11T07:28:08.014002+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-09-11T07:27:34.213870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:34.213870+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:40.529565+0000","scrub_duration":0.0026667679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2,5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.286875+0000","last_change":"2024-09-11T07:28:08.286847+0000","last_active":"2024-09-11T07:28:08.286875+0000","last_peered":"2024-09-11T07:28:08.286875+0000","last_clean":"2024-09-11T07:28:08.286875+0000","last_became_active":"2024-09-11T07:26:31.420320+0000","last_became_peered":"2024-09-11T07:26:31.420320+0000","last_unstale":"2024-09-11T07:28:08.286875+0000","last_undegraded":"2024-09-11T07:28:08.286875+0000","last_fullsized":"2024-09-11T07:28:08.286875+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-11T07:28:08.286788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:08.286788+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.046248+0000","scrub_duration":0.001825976,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.799930+0000","last_change":"2024-09-11T07:26:31.418190+0000","last_active":"2024-09-11T07:28:05.799930+0000","last_peered":"2024-09-11T07:28:05.799930+0000","last_clean":"2024-09-11T07:28:05.799930+0000","last_became_active":"2024-09-11T07:26:31.417455+0000","last_became_peered":"2024-09-11T07:26:31.417455+0000","last_unstale":"2024-09-11T07:28:05.799930+0000","last_undegraded":"2024-09-11T07:28:05.799930+0000","last_fullsized":"2024-09-11T07:28:05.799930+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.895992+0000","last_change":"2024-09-11T07:26:35.564511+0000","last_active":"2024-09-11T07:28:05.895992+0000","last_peered":"2024-09-11T07:28:05.895992+0000","last_clean":"2024-09-11T07:28:05.895992+0000","last_became_active":"2024-09-11T07:26:35.564178+0000","last_became_peered":"2024-09-11T07:26:35.564178+0000","last_unstale":"2024-09-11T07:28:05.895992+0000","last_undegraded":"2024-09-11T07:28:05.895992+0000","last_fullsized":"2024-09-11T07:28:05.895992+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.002049+0000","last_change":"2024-09-11T07:27:54.621934+0000","last_active":"2024-09-11T07:28:05.002049+0000","last_peered":"2024-09-11T07:28:05.002049+0000","last_clean":"2024-09-11T07:28:05.002049+0000","last_became_active":"2024-09-11T07:26:37.561378+0000","last_became_peered":"2024-09-11T07:26:37.561378+0000","last_unstale":"2024-09-11T07:28:05.002049+0000","last_undegraded":"2024-09-11T07:28:05.002049+0000","last_fullsized":"2024-09-11T07:28:05.002049+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.621874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.621874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:07.014230+0000","scrub_duration":0.0013604750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":67,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.001217+0000","last_change":"2024-09-11T07:27:49.061728+0000","last_active":"2024-09-11T07:28:05.001217+0000","last_peered":"2024-09-11T07:28:05.001217+0000","last_clean":"2024-09-11T07:28:05.001217+0000","last_became_active":"2024-09-11T07:26:33.541359+0000","last_became_peered":"2024-09-11T07:26:33.541359+0000","last_unstale":"2024-09-11T07:28:05.001217+0000","last_undegraded":"2024-09-11T07:28:05.001217+0000","last_fullsized":"2024-09-11T07:28:05.001217+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-09-11T07:27:49.061670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.061670+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:51.283390+0000","scrub_duration":0.001809801,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'3","reported_seq":55,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.652070+0000","last_change":"2024-09-11T07:28:01.711301+0000","last_active":"2024-09-11T07:28:05.652070+0000","last_peered":"2024-09-11T07:28:05.652070+0000","last_clean":"2024-09-11T07:28:05.652070+0000","last_became_active":"2024-09-11T07:26:31.419501+0000","last_became_peered":"2024-09-11T07:26:31.419501+0000","last_unstale":"2024-09-11T07:28:05.652070+0000","last_undegraded":"2024-09-11T07:28:05.652070+0000","last_fullsized":"2024-09-11T07:28:05.652070+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.711241+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:02.735271+0000","scrub_duration":0.0034627350000000002,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.895730+0000","last_change":"2024-09-11T07:27:47.903204+0000","last_active":"2024-09-11T07:28:05.895730+0000","last_peered":"2024-09-11T07:28:05.895730+0000","last_clean":"2024-09-11T07:28:05.895730+0000","last_became_active":"2024-09-11T07:26:31.423304+0000","last_became_peered":"2024-09-11T07:26:31.423304+0000","last_unstale":"2024-09-11T07:28:05.895730+0000","last_undegraded":"2024-09-11T07:28:05.895730+0000","last_fullsized":"2024-09-11T07:28:05.895730+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:47.903135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.903135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:01.867608+0000","scrub_duration":0.001284908,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"72'79","reported_seq":82,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.800051+0000","last_change":"2024-09-11T07:26:57.872633+0000","last_active":"2024-09-11T07:28:05.800051+0000","last_peered":"2024-09-11T07:28:05.800051+0000","last_clean":"2024-09-11T07:28:05.800051+0000","last_became_active":"2024-09-11T07:26:17.397381+0000","last_became_peered":"2024-09-11T07:26:17.397381+0000","last_unstale":"2024-09-11T07:28:05.800051+0000","last_undegraded":"2024-09-11T07:28:05.800051+0000","last_fullsized":"2024-09-11T07:28:05.800051+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'72","last_scrub_stamp":"2024-09-11T07:26:57.872570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:24:22.202219+0000","last_clean_scrub_stamp":"2024-09-11T07:26:57.872570+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015639180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.800170+0000","last_change":"2024-09-11T07:27:44.036138+0000","last_active":"2024-09-11T07:28:05.800170+0000","last_peered":"2024-09-11T07:28:05.800170+0000","last_clean":"2024-09-11T07:28:05.800170+0000","last_became_active":"2024-09-11T07:26:35.564480+0000","last_became_peered":"2024-09-11T07:26:35.564480+0000","last_unstale":"2024-09-11T07:28:05.800170+0000","last_undegraded":"2024-09-11T07:28:05.800170+0000","last_fullsized":"2024-09-11T07:28:05.800170+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:44.036058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:44.036058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:48.598709+0000","scrub_duration":0.001488931,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.896049+0000","last_change":"2024-09-11T07:26:37.560202+0000","last_active":"2024-09-11T07:28:05.896049+0000","last_peered":"2024-09-11T07:28:05.896049+0000","last_clean":"2024-09-11T07:28:05.896049+0000","last_became_active":"2024-09-11T07:26:37.559920+0000","last_became_peered":"2024-09-11T07:26:37.559920+0000","last_unstale":"2024-09-11T07:28:05.896049+0000","last_undegraded":"2024-09-11T07:28:05.896049+0000","last_fullsized":"2024-09-11T07:28:05.896049+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":76,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.001125+0000","last_change":"2024-09-11T07:27:59.128292+0000","last_active":"2024-09-11T07:28:05.001125+0000","last_peered":"2024-09-11T07:28:05.001125+0000","last_clean":"2024-09-11T07:28:05.001125+0000","last_became_active":"2024-09-11T07:26:33.543687+0000","last_became_peered":"2024-09-11T07:26:33.543687+0000","last_unstale":"2024-09-11T07:28:05.001125+0000","last_undegraded":"2024-09-11T07:28:05.001125+0000","last_fullsized":"2024-09-11T07:28:05.001125+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-11T07:27:59.128237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:59.128237+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:16.835564+0000","scrub_duration":0.00199329,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:06.305125+0000","last_change":"2024-09-11T07:27:54.182157+0000","last_active":"2024-09-11T07:28:06.305125+0000","last_peered":"2024-09-11T07:28:06.305125+0000","last_clean":"2024-09-11T07:28:06.305125+0000","last_became_active":"2024-09-11T07:26:31.419560+0000","last_became_peered":"2024-09-11T07:26:31.419560+0000","last_unstale":"2024-09-11T07:28:06.305125+0000","last_undegraded":"2024-09-11T07:28:06.305125+0000","last_fullsized":"2024-09-11T07:28:06.305125+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.182106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.182106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:04.075506+0000","scrub_duration":0.001213254,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:06.304619+0000","last_change":"2024-09-11T07:26:35.564320+0000","last_active":"2024-09-11T07:28:06.304619+0000","last_peered":"2024-09-11T07:28:06.304619+0000","last_clean":"2024-09-11T07:28:06.304619+0000","last_became_active":"2024-09-11T07:26:35.564044+0000","last_became_peered":"2024-09-11T07:26:35.564044+0000","last_unstale":"2024-09-11T07:28:06.304619+0000","last_undegraded":"2024-09-11T07:28:06.304619+0000","last_fullsized":"2024-09-11T07:28:06.304619+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.800001+0000","last_change":"2024-09-11T07:26:37.565084+0000","last_active":"2024-09-11T07:28:05.800001+0000","last_peered":"2024-09-11T07:28:05.800001+0000","last_clean":"2024-09-11T07:28:05.800001+0000","last_became_active":"2024-09-11T07:26:37.564801+0000","last_became_peered":"2024-09-11T07:26:37.564801+0000","last_unstale":"2024-09-11T07:28:05.800001+0000","last_undegraded":"2024-09-11T07:28:05.800001+0000","last_fullsized":"2024-09-11T07:28:05.800001+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":61,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.003102+0000","last_change":"2024-09-11T07:28:02.230985+0000","last_active":"2024-09-11T07:28:05.003102+0000","last_peered":"2024-09-11T07:28:05.003102+0000","last_clean":"2024-09-11T07:28:05.003102+0000","last_became_active":"2024-09-11T07:26:33.543148+0000","last_became_peered":"2024-09-11T07:26:33.543148+0000","last_unstale":"2024-09-11T07:28:05.003102+0000","last_undegraded":"2024-09-11T07:28:05.003102+0000","last_fullsized":"2024-09-11T07:28:05.003102+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:28:02.230928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.230928+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:27.244330+0000","scrub_duration":0.001888926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.003104+0000","last_change":"2024-09-11T07:27:53.659961+0000","last_active":"2024-09-11T07:28:05.003104+0000","last_peered":"2024-09-11T07:28:05.003104+0000","last_clean":"2024-09-11T07:28:05.003104+0000","last_became_active":"2024-09-11T07:26:31.423056+0000","last_became_peered":"2024-09-11T07:26:31.423056+0000","last_unstale":"2024-09-11T07:28:05.003104+0000","last_undegraded":"2024-09-11T07:28:05.003104+0000","last_fullsized":"2024-09-11T07:28:05.003104+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:53.659902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:53.659902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.781453+0000","scrub_duration":0.0011613719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:06.304482+0000","last_change":"2024-09-11T07:26:35.563452+0000","last_active":"2024-09-11T07:28:06.304482+0000","last_peered":"2024-09-11T07:28:06.304482+0000","last_clean":"2024-09-11T07:28:06.304482+0000","last_became_active":"2024-09-11T07:26:35.563182+0000","last_became_peered":"2024-09-11T07:26:35.563182+0000","last_unstale":"2024-09-11T07:28:06.304482+0000","last_undegraded":"2024-09-11T07:28:06.304482+0000","last_fullsized":"2024-09-11T07:28:06.304482+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.001532+0000","last_change":"2024-09-11T07:26:37.565396+0000","last_active":"2024-09-11T07:28:05.001532+0000","last_peered":"2024-09-11T07:28:05.001532+0000","last_clean":"2024-09-11T07:28:05.001532+0000","last_became_active":"2024-09-11T07:26:37.565029+0000","last_became_peered":"2024-09-11T07:26:37.565029+0000","last_unstale":"2024-09-11T07:28:05.001532+0000","last_undegraded":"2024-09-11T07:28:05.001532+0000","last_fullsized":"2024-09-11T07:28:05.001532+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":70,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:07.811186+0000","last_change":"2024-09-11T07:28:07.811136+0000","last_active":"2024-09-11T07:28:07.811186+0000","last_peered":"2024-09-11T07:28:07.811186+0000","last_clean":"2024-09-11T07:28:07.811186+0000","last_became_active":"2024-09-11T07:26:33.538216+0000","last_became_peered":"2024-09-11T07:26:33.538216+0000","last_unstale":"2024-09-11T07:28:07.811186+0000","last_undegraded":"2024-09-11T07:28:07.811186+0000","last_fullsized":"2024-09-11T07:28:07.811186+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-09-11T07:28:07.811071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:07.811071+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:23.495093+0000","scrub_duration":0.0019627080000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:06.304945+0000","last_change":"2024-09-11T07:27:50.132629+0000","last_active":"2024-09-11T07:28:06.304945+0000","last_peered":"2024-09-11T07:28:06.304945+0000","last_clean":"2024-09-11T07:28:06.304945+0000","last_became_active":"2024-09-11T07:26:31.418490+0000","last_became_peered":"2024-09-11T07:26:31.418490+0000","last_unstale":"2024-09-11T07:28:06.304945+0000","last_undegraded":"2024-09-11T07:28:06.304945+0000","last_fullsized":"2024-09-11T07:28:06.304945+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.132552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.048681+0000","scrub_duration":0.001622807,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.799916+0000","last_change":"2024-09-11T07:26:35.567147+0000","last_active":"2024-09-11T07:28:05.799916+0000","last_peered":"2024-09-11T07:28:05.799916+0000","last_clean":"2024-09-11T07:28:05.799916+0000","last_became_active":"2024-09-11T07:26:35.566888+0000","last_became_peered":"2024-09-11T07:26:35.566888+0000","last_unstale":"2024-09-11T07:28:05.799916+0000","last_undegraded":"2024-09-11T07:28:05.799916+0000","last_fullsized":"2024-09-11T07:28:05.799916+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.895057+0000","last_change":"2024-09-11T07:28:03.961280+0000","last_active":"2024-09-11T07:28:05.895057+0000","last_peered":"2024-09-11T07:28:05.895057+0000","last_clean":"2024-09-11T07:28:05.895057+0000","last_became_active":"2024-09-11T07:26:37.861134+0000","last_became_peered":"2024-09-11T07:26:37.861134+0000","last_unstale":"2024-09-11T07:28:05.895057+0000","last_undegraded":"2024-09-11T07:28:05.895057+0000","last_fullsized":"2024-09-11T07:28:05.895057+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:03.961220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:03.961220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:29.223995+0000","scrub_duration":0.0011943730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"83'17","reported_seq":73,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.652613+0000","last_change":"2024-09-11T07:28:01.098068+0000","last_active":"2024-09-11T07:28:05.652613+0000","last_peered":"2024-09-11T07:28:05.652613+0000","last_clean":"2024-09-11T07:28:05.652613+0000","last_became_active":"2024-09-11T07:26:33.540363+0000","last_became_peered":"2024-09-11T07:26:33.540363+0000","last_unstale":"2024-09-11T07:28:05.652613+0000","last_undegraded":"2024-09-11T07:28:05.652613+0000","last_fullsized":"2024-09-11T07:28:05.652613+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'17","last_scrub_stamp":"2024-09-11T07:28:01.097981+0000","last_deep_scrub":"83'17","last_deep_scrub_stamp":"2024-09-11T07:28:01.097981+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.097981+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:26.179592+0000","scrub_duration":0.0042048720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":64,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.653342+0000","last_change":"2024-09-11T07:27:46.077857+0000","last_active":"2024-09-11T07:28:05.653342+0000","last_peered":"2024-09-11T07:28:05.653342+0000","last_clean":"2024-09-11T07:28:05.653342+0000","last_became_active":"2024-09-11T07:26:31.419718+0000","last_became_peered":"2024-09-11T07:26:31.419718+0000","last_unstale":"2024-09-11T07:28:05.653342+0000","last_undegraded":"2024-09-11T07:28:05.653342+0000","last_fullsized":"2024-09-11T07:28:05.653342+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:27:46.077786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:46.077786+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:07.735381+0000","scrub_duration":0.0019376720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:06.304465+0000","last_change":"2024-09-11T07:27:53.209518+0000","last_active":"2024-09-11T07:28:06.304465+0000","last_peered":"2024-09-11T07:28:06.304465+0000","last_clean":"2024-09-11T07:28:06.304465+0000","last_became_active":"2024-09-11T07:26:35.561967+0000","last_became_peered":"2024-09-11T07:26:35.561967+0000","last_unstale":"2024-09-11T07:28:06.304465+0000","last_undegraded":"2024-09-11T07:28:06.304465+0000","last_fullsized":"2024-09-11T07:28:06.304465+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:53.209452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:53.209452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:23.125281+0000","scrub_duration":0.0014563029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.651418+0000","last_change":"2024-09-11T07:28:02.677186+0000","last_active":"2024-09-11T07:28:05.651418+0000","last_peered":"2024-09-11T07:28:05.651418+0000","last_clean":"2024-09-11T07:28:05.651418+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:28:05.651418+0000","last_undegraded":"2024-09-11T07:28:05.651418+0000","last_fullsized":"2024-09-11T07:28:05.651418+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:02.677095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.677095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.575070+0000","scrub_duration":0.001519269,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":61,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.003403+0000","last_change":"2024-09-11T07:27:40.351572+0000","last_active":"2024-09-11T07:28:05.003403+0000","last_peered":"2024-09-11T07:28:05.003403+0000","last_clean":"2024-09-11T07:28:05.003403+0000","last_became_active":"2024-09-11T07:26:33.547344+0000","last_became_peered":"2024-09-11T07:26:33.547344+0000","last_unstale":"2024-09-11T07:28:05.003403+0000","last_undegraded":"2024-09-11T07:28:05.003403+0000","last_fullsized":"2024-09-11T07:28:05.003403+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:27:40.351507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:40.351507+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.098394+0000","scrub_duration":0.0021803980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:06.304553+0000","last_change":"2024-09-11T07:27:58.324311+0000","last_active":"2024-09-11T07:28:06.304553+0000","last_peered":"2024-09-11T07:28:06.304553+0000","last_clean":"2024-09-11T07:28:06.304553+0000","last_became_active":"2024-09-11T07:26:31.415701+0000","last_became_peered":"2024-09-11T07:26:31.415701+0000","last_unstale":"2024-09-11T07:28:06.304553+0000","last_undegraded":"2024-09-11T07:28:06.304553+0000","last_fullsized":"2024-09-11T07:28:06.304553+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:58.324255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.324255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.617500+0000","scrub_duration":0.0013365549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.652913+0000","last_change":"2024-09-11T07:26:35.559563+0000","last_active":"2024-09-11T07:28:05.652913+0000","last_peered":"2024-09-11T07:28:05.652913+0000","last_clean":"2024-09-11T07:28:05.652913+0000","last_became_active":"2024-09-11T07:26:35.559262+0000","last_became_peered":"2024-09-11T07:26:35.559262+0000","last_unstale":"2024-09-11T07:28:05.652913+0000","last_undegraded":"2024-09-11T07:28:05.652913+0000","last_fullsized":"2024-09-11T07:28:05.652913+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:06.305606+0000","last_change":"2024-09-11T07:28:06.305563+0000","last_active":"2024-09-11T07:28:06.305606+0000","last_peered":"2024-09-11T07:28:06.305606+0000","last_clean":"2024-09-11T07:28:06.305606+0000","last_became_active":"2024-09-11T07:26:37.564084+0000","last_became_peered":"2024-09-11T07:26:37.564084+0000","last_unstale":"2024-09-11T07:28:06.305606+0000","last_undegraded":"2024-09-11T07:28:06.305606+0000","last_fullsized":"2024-09-11T07:28:06.305606+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:28:06.305502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:06.305502+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.659083+0000","scrub_duration":0.0021934649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.001083+0000","last_change":"2024-09-11T07:27:45.990617+0000","last_active":"2024-09-11T07:28:05.001083+0000","last_peered":"2024-09-11T07:28:05.001083+0000","last_clean":"2024-09-11T07:28:05.001083+0000","last_became_active":"2024-09-11T07:26:31.423229+0000","last_became_peered":"2024-09-11T07:26:31.423229+0000","last_unstale":"2024-09-11T07:28:05.001083+0000","last_undegraded":"2024-09-11T07:28:05.001083+0000","last_fullsized":"2024-09-11T07:28:05.001083+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:45.990548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:45.990548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:51.641867+0000","scrub_duration":0.0014585150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"82'9","reported_seq":57,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.653000+0000","last_change":"2024-09-11T07:27:48.761470+0000","last_active":"2024-09-11T07:28:05.653000+0000","last_peered":"2024-09-11T07:28:05.653000+0000","last_clean":"2024-09-11T07:28:05.653000+0000","last_became_active":"2024-09-11T07:26:33.543628+0000","last_became_peered":"2024-09-11T07:26:33.543628+0000","last_unstale":"2024-09-11T07:28:05.653000+0000","last_undegraded":"2024-09-11T07:28:05.653000+0000","last_fullsized":"2024-09-11T07:28:05.653000+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'9","last_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:27:48.761397+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.913254+0000","scrub_duration":0.0037047439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.895925+0000","last_change":"2024-09-11T07:27:42.917739+0000","last_active":"2024-09-11T07:28:05.895925+0000","last_peered":"2024-09-11T07:28:05.895925+0000","last_clean":"2024-09-11T07:28:05.895925+0000","last_became_active":"2024-09-11T07:26:35.566894+0000","last_became_peered":"2024-09-11T07:26:35.566894+0000","last_unstale":"2024-09-11T07:28:05.895925+0000","last_undegraded":"2024-09-11T07:28:05.895925+0000","last_fullsized":"2024-09-11T07:28:05.895925+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.917680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:06.763663+0000","scrub_duration":0.0013496649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.800188+0000","last_change":"2024-09-11T07:27:47.031883+0000","last_active":"2024-09-11T07:28:05.800188+0000","last_peered":"2024-09-11T07:28:05.800188+0000","last_clean":"2024-09-11T07:28:05.800188+0000","last_became_active":"2024-09-11T07:26:37.863100+0000","last_became_peered":"2024-09-11T07:26:37.863100+0000","last_unstale":"2024-09-11T07:28:05.800188+0000","last_undegraded":"2024-09-11T07:28:05.800188+0000","last_fullsized":"2024-09-11T07:28:05.800188+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:47.031807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.031807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:54.469643+0000","scrub_duration":0.001556195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"64'10","reported_seq":50,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.652468+0000","last_change":"2024-09-11T07:26:33.545346+0000","last_active":"2024-09-11T07:28:05.652468+0000","last_peered":"2024-09-11T07:28:05.652468+0000","last_clean":"2024-09-11T07:28:05.652468+0000","last_became_active":"2024-09-11T07:26:33.545078+0000","last_became_peered":"2024-09-11T07:26:33.545078+0000","last_unstale":"2024-09-11T07:28:05.652468+0000","last_undegraded":"2024-09-11T07:28:05.652468+0000","last_fullsized":"2024-09-11T07:28:05.652468+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.895592+0000","last_change":"2024-09-11T07:26:31.424634+0000","last_active":"2024-09-11T07:28:05.895592+0000","last_peered":"2024-09-11T07:28:05.895592+0000","last_clean":"2024-09-11T07:28:05.895592+0000","last_became_active":"2024-09-11T07:26:31.423789+0000","last_became_peered":"2024-09-11T07:26:31.423789+0000","last_unstale":"2024-09-11T07:28:05.895592+0000","last_undegraded":"2024-09-11T07:28:05.895592+0000","last_fullsized":"2024-09-11T07:28:05.895592+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.895758+0000","last_change":"2024-09-11T07:26:35.562509+0000","last_active":"2024-09-11T07:28:05.895758+0000","last_peered":"2024-09-11T07:28:05.895758+0000","last_clean":"2024-09-11T07:28:05.895758+0000","last_became_active":"2024-09-11T07:26:35.562245+0000","last_became_peered":"2024-09-11T07:26:35.562245+0000","last_unstale":"2024-09-11T07:28:05.895758+0000","last_undegraded":"2024-09-11T07:28:05.895758+0000","last_fullsized":"2024-09-11T07:28:05.895758+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.003111+0000","last_change":"2024-09-11T07:27:49.245119+0000","last_active":"2024-09-11T07:28:05.003111+0000","last_peered":"2024-09-11T07:28:05.003111+0000","last_clean":"2024-09-11T07:28:05.003111+0000","last_became_active":"2024-09-11T07:26:37.566749+0000","last_became_peered":"2024-09-11T07:26:37.566749+0000","last_unstale":"2024-09-11T07:28:05.003111+0000","last_undegraded":"2024-09-11T07:28:05.003111+0000","last_fullsized":"2024-09-11T07:28:05.003111+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:49.245047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.245047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:49.614854+0000","scrub_duration":0.001867966,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"83'13","reported_seq":65,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.895811+0000","last_change":"2024-09-11T07:28:00.959669+0000","last_active":"2024-09-11T07:28:05.895811+0000","last_peered":"2024-09-11T07:28:05.895811+0000","last_clean":"2024-09-11T07:28:05.895811+0000","last_became_active":"2024-09-11T07:26:33.537425+0000","last_became_peered":"2024-09-11T07:26:33.537425+0000","last_unstale":"2024-09-11T07:28:05.895811+0000","last_undegraded":"2024-09-11T07:28:05.895811+0000","last_fullsized":"2024-09-11T07:28:05.895811+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-11T07:28:00.959583+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-11T07:28:00.959583+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.959583+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:06.630980+0000","scrub_duration":0.0032987350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.003264+0000","last_change":"2024-09-11T07:28:01.237329+0000","last_active":"2024-09-11T07:28:05.003264+0000","last_peered":"2024-09-11T07:28:05.003264+0000","last_clean":"2024-09-11T07:28:05.003264+0000","last_became_active":"2024-09-11T07:26:31.419409+0000","last_became_peered":"2024-09-11T07:26:31.419409+0000","last_unstale":"2024-09-11T07:28:05.003264+0000","last_undegraded":"2024-09-11T07:28:05.003264+0000","last_fullsized":"2024-09-11T07:28:05.003264+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:01.237267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.237267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.266034+0000","scrub_duration":0.0012797399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.652286+0000","last_change":"2024-09-11T07:26:35.560917+0000","last_active":"2024-09-11T07:28:05.652286+0000","last_peered":"2024-09-11T07:28:05.652286+0000","last_clean":"2024-09-11T07:28:05.652286+0000","last_became_active":"2024-09-11T07:26:35.560610+0000","last_became_peered":"2024-09-11T07:26:35.560610+0000","last_unstale":"2024-09-11T07:28:05.652286+0000","last_undegraded":"2024-09-11T07:28:05.652286+0000","last_fullsized":"2024-09-11T07:28:05.652286+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.653202+0000","last_change":"2024-09-11T07:26:37.566136+0000","last_active":"2024-09-11T07:28:05.653202+0000","last_peered":"2024-09-11T07:28:05.653202+0000","last_clean":"2024-09-11T07:28:05.653202+0000","last_became_active":"2024-09-11T07:26:37.565723+0000","last_became_peered":"2024-09-11T07:26:37.565723+0000","last_unstale":"2024-09-11T07:28:05.653202+0000","last_undegraded":"2024-09-11T07:28:05.653202+0000","last_fullsized":"2024-09-11T07:28:05.653202+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":61,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.002187+0000","last_change":"2024-09-11T07:27:57.601328+0000","last_active":"2024-09-11T07:28:05.002187+0000","last_peered":"2024-09-11T07:28:05.002187+0000","last_clean":"2024-09-11T07:28:05.002187+0000","last_became_active":"2024-09-11T07:26:33.540688+0000","last_became_peered":"2024-09-11T07:26:33.540688+0000","last_unstale":"2024-09-11T07:28:05.002187+0000","last_undegraded":"2024-09-11T07:28:05.002187+0000","last_fullsized":"2024-09-11T07:28:05.002187+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:27:57.601261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.601261+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:58.826830+0000","scrub_duration":0.002247724,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.001200+0000","last_change":"2024-09-11T07:27:54.102580+0000","last_active":"2024-09-11T07:28:05.001200+0000","last_peered":"2024-09-11T07:28:05.001200+0000","last_clean":"2024-09-11T07:28:05.001200+0000","last_became_active":"2024-09-11T07:26:31.425587+0000","last_became_peered":"2024-09-11T07:26:31.425587+0000","last_unstale":"2024-09-11T07:28:05.001200+0000","last_undegraded":"2024-09-11T07:28:05.001200+0000","last_fullsized":"2024-09-11T07:28:05.001200+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.102528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.102528+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.102528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:06.835183+0000","scrub_duration":0.0012736080000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.652170+0000","last_change":"2024-09-11T07:26:35.562603+0000","last_active":"2024-09-11T07:28:05.652170+0000","last_peered":"2024-09-11T07:28:05.652170+0000","last_clean":"2024-09-11T07:28:05.652170+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:28:05.652170+0000","last_undegraded":"2024-09-11T07:28:05.652170+0000","last_fullsized":"2024-09-11T07:28:05.652170+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.001739+0000","last_change":"2024-09-11T07:27:58.192585+0000","last_active":"2024-09-11T07:28:05.001739+0000","last_peered":"2024-09-11T07:28:05.001739+0000","last_clean":"2024-09-11T07:28:05.001739+0000","last_became_active":"2024-09-11T07:26:37.568886+0000","last_became_peered":"2024-09-11T07:26:37.568886+0000","last_unstale":"2024-09-11T07:28:05.001739+0000","last_undegraded":"2024-09-11T07:28:05.001739+0000","last_fullsized":"2024-09-11T07:28:05.001739+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:58.192528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.192528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:18.757569+0000","scrub_duration":0.001884698,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":64,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.652677+0000","last_change":"2024-09-11T07:27:48.081460+0000","last_active":"2024-09-11T07:28:05.652677+0000","last_peered":"2024-09-11T07:28:05.652677+0000","last_clean":"2024-09-11T07:28:05.652677+0000","last_became_active":"2024-09-11T07:26:33.540208+0000","last_became_peered":"2024-09-11T07:26:33.540208+0000","last_unstale":"2024-09-11T07:28:05.652677+0000","last_undegraded":"2024-09-11T07:28:05.652677+0000","last_fullsized":"2024-09-11T07:28:05.652677+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:27:48.081392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:48.081392+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:04.524601+0000","scrub_duration":0.0022394590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.003007+0000","last_change":"2024-09-11T07:27:49.677063+0000","last_active":"2024-09-11T07:28:05.003007+0000","last_peered":"2024-09-11T07:28:05.003007+0000","last_clean":"2024-09-11T07:28:05.003007+0000","last_became_active":"2024-09-11T07:26:31.422671+0000","last_became_peered":"2024-09-11T07:26:31.422671+0000","last_unstale":"2024-09-11T07:28:05.003007+0000","last_undegraded":"2024-09-11T07:28:05.003007+0000","last_fullsized":"2024-09-11T07:28:05.003007+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.677003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:54.836177+0000","scrub_duration":0.0013962969999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.001967+0000","last_change":"2024-09-11T07:26:35.567257+0000","last_active":"2024-09-11T07:28:05.001967+0000","last_peered":"2024-09-11T07:28:05.001967+0000","last_clean":"2024-09-11T07:28:05.001967+0000","last_became_active":"2024-09-11T07:26:35.566931+0000","last_became_peered":"2024-09-11T07:26:35.566931+0000","last_unstale":"2024-09-11T07:28:05.001967+0000","last_undegraded":"2024-09-11T07:28:05.001967+0000","last_fullsized":"2024-09-11T07:28:05.001967+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.651459+0000","last_change":"2024-09-11T07:26:37.561886+0000","last_active":"2024-09-11T07:28:05.651459+0000","last_peered":"2024-09-11T07:28:05.651459+0000","last_clean":"2024-09-11T07:28:05.651459+0000","last_became_active":"2024-09-11T07:26:37.561573+0000","last_became_peered":"2024-09-11T07:26:37.561573+0000","last_unstale":"2024-09-11T07:28:05.651459+0000","last_undegraded":"2024-09-11T07:28:05.651459+0000","last_fullsized":"2024-09-11T07:28:05.651459+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"82'6","reported_seq":53,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.651667+0000","last_change":"2024-09-11T07:27:47.789519+0000","last_active":"2024-09-11T07:28:05.651667+0000","last_peered":"2024-09-11T07:28:05.651667+0000","last_clean":"2024-09-11T07:28:05.651667+0000","last_became_active":"2024-09-11T07:26:33.545830+0000","last_became_peered":"2024-09-11T07:26:33.545830+0000","last_unstale":"2024-09-11T07:28:05.651667+0000","last_undegraded":"2024-09-11T07:28:05.651667+0000","last_fullsized":"2024-09-11T07:28:05.651667+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'6","last_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_deep_scrub":"82'6","last_deep_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.789465+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:59.162056+0000","scrub_duration":0.002624679,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.896009+0000","last_change":"2024-09-11T07:27:58.043076+0000","last_active":"2024-09-11T07:28:05.896009+0000","last_peered":"2024-09-11T07:28:05.896009+0000","last_clean":"2024-09-11T07:28:05.896009+0000","last_became_active":"2024-09-11T07:26:31.420518+0000","last_became_peered":"2024-09-11T07:26:31.420518+0000","last_unstale":"2024-09-11T07:28:05.896009+0000","last_undegraded":"2024-09-11T07:28:05.896009+0000","last_fullsized":"2024-09-11T07:28:05.896009+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.043009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:24.326885+0000","scrub_duration":0.0015014900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.000915+0000","last_change":"2024-09-11T07:27:56.124601+0000","last_active":"2024-09-11T07:28:05.000915+0000","last_peered":"2024-09-11T07:28:05.000915+0000","last_clean":"2024-09-11T07:28:05.000915+0000","last_became_active":"2024-09-11T07:26:35.560506+0000","last_became_peered":"2024-09-11T07:26:35.560506+0000","last_unstale":"2024-09-11T07:28:05.000915+0000","last_undegraded":"2024-09-11T07:28:05.000915+0000","last_fullsized":"2024-09-11T07:28:05.000915+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:56.124542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:56.124542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.847074+0000","scrub_duration":0.0011688619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.799878+0000","last_change":"2024-09-11T07:27:42.059066+0000","last_active":"2024-09-11T07:28:05.799878+0000","last_peered":"2024-09-11T07:28:05.799878+0000","last_clean":"2024-09-11T07:28:05.799878+0000","last_became_active":"2024-09-11T07:26:37.564208+0000","last_became_peered":"2024-09-11T07:26:37.564208+0000","last_unstale":"2024-09-11T07:28:05.799878+0000","last_undegraded":"2024-09-11T07:28:05.799878+0000","last_fullsized":"2024-09-11T07:28:05.799878+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.059001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.059001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:51.267826+0000","scrub_duration":0.001418094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":54,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.651856+0000","last_change":"2024-09-11T07:26:33.542913+0000","last_active":"2024-09-11T07:28:05.651856+0000","last_peered":"2024-09-11T07:28:05.651856+0000","last_clean":"2024-09-11T07:28:05.651856+0000","last_became_active":"2024-09-11T07:26:33.542625+0000","last_became_peered":"2024-09-11T07:26:33.542625+0000","last_unstale":"2024-09-11T07:28:05.651856+0000","last_undegraded":"2024-09-11T07:28:05.651856+0000","last_fullsized":"2024-09-11T07:28:05.651856+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.001446+0000","last_change":"2024-09-11T07:28:02.157520+0000","last_active":"2024-09-11T07:28:05.001446+0000","last_peered":"2024-09-11T07:28:05.001446+0000","last_clean":"2024-09-11T07:28:05.001446+0000","last_became_active":"2024-09-11T07:26:31.425876+0000","last_became_peered":"2024-09-11T07:26:31.425876+0000","last_unstale":"2024-09-11T07:28:05.001446+0000","last_undegraded":"2024-09-11T07:28:05.001446+0000","last_fullsized":"2024-09-11T07:28:05.001446+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:02.157455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.157455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:12.096002+0000","scrub_duration":0.0016990040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.799753+0000","last_change":"2024-09-11T07:26:35.565591+0000","last_active":"2024-09-11T07:28:05.799753+0000","last_peered":"2024-09-11T07:28:05.799753+0000","last_clean":"2024-09-11T07:28:05.799753+0000","last_became_active":"2024-09-11T07:26:35.565339+0000","last_became_peered":"2024-09-11T07:26:35.565339+0000","last_unstale":"2024-09-11T07:28:05.799753+0000","last_undegraded":"2024-09-11T07:28:05.799753+0000","last_fullsized":"2024-09-11T07:28:05.799753+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.652137+0000","last_change":"2024-09-11T07:26:37.862599+0000","last_active":"2024-09-11T07:28:05.652137+0000","last_peered":"2024-09-11T07:28:05.652137+0000","last_clean":"2024-09-11T07:28:05.652137+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:28:05.652137+0000","last_undegraded":"2024-09-11T07:28:05.652137+0000","last_fullsized":"2024-09-11T07:28:05.652137+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.800158+0000","last_change":"2024-09-11T07:26:37.564199+0000","last_active":"2024-09-11T07:28:05.800158+0000","last_peered":"2024-09-11T07:28:05.800158+0000","last_clean":"2024-09-11T07:28:05.800158+0000","last_became_active":"2024-09-11T07:26:37.563942+0000","last_became_peered":"2024-09-11T07:26:37.563942+0000","last_unstale":"2024-09-11T07:28:05.800158+0000","last_undegraded":"2024-09-11T07:28:05.800158+0000","last_fullsized":"2024-09-11T07:28:05.800158+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":56,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.895359+0000","last_change":"2024-09-11T07:28:02.969498+0000","last_active":"2024-09-11T07:28:05.895359+0000","last_peered":"2024-09-11T07:28:05.895359+0000","last_clean":"2024-09-11T07:28:05.895359+0000","last_became_active":"2024-09-11T07:26:31.423932+0000","last_became_peered":"2024-09-11T07:26:31.423932+0000","last_unstale":"2024-09-11T07:28:05.895359+0000","last_undegraded":"2024-09-11T07:28:05.895359+0000","last_fullsized":"2024-09-11T07:28:05.895359+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:28:02.969442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.969442+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:22.652879+0000","scrub_duration":0.0014604450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.003392+0000","last_change":"2024-09-11T07:27:55.150630+0000","last_active":"2024-09-11T07:28:05.003392+0000","last_peered":"2024-09-11T07:28:05.003392+0000","last_clean":"2024-09-11T07:28:05.003392+0000","last_became_active":"2024-09-11T07:26:33.546797+0000","last_became_peered":"2024-09-11T07:26:33.546797+0000","last_unstale":"2024-09-11T07:28:05.003392+0000","last_undegraded":"2024-09-11T07:28:05.003392+0000","last_fullsized":"2024-09-11T07:28:05.003392+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:27:55.150567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:55.150567+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:10.187014+0000","scrub_duration":0.0016612689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"64'8","reported_seq":73,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.001382+0000","last_change":"2024-09-11T07:28:01.123469+0000","last_active":"2024-09-11T07:28:05.001382+0000","last_peered":"2024-09-11T07:28:05.001382+0000","last_clean":"2024-09-11T07:28:05.001382+0000","last_became_active":"2024-09-11T07:26:35.559304+0000","last_became_peered":"2024-09-11T07:26:35.559304+0000","last_unstale":"2024-09-11T07:28:05.001382+0000","last_undegraded":"2024-09-11T07:28:05.001382+0000","last_fullsized":"2024-09-11T07:28:05.001382+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:28:01.123401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.123401+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:18.715539+0000","scrub_duration":0.00167504,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.002113+0000","last_change":"2024-09-11T07:26:37.563308+0000","last_active":"2024-09-11T07:28:05.002113+0000","last_peered":"2024-09-11T07:28:05.002113+0000","last_clean":"2024-09-11T07:28:05.002113+0000","last_became_active":"2024-09-11T07:26:37.562501+0000","last_became_peered":"2024-09-11T07:26:37.562501+0000","last_unstale":"2024-09-11T07:28:05.002113+0000","last_undegraded":"2024-09-11T07:28:05.002113+0000","last_fullsized":"2024-09-11T07:28:05.002113+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.652030+0000","last_change":"2024-09-11T07:27:38.732867+0000","last_active":"2024-09-11T07:28:05.652030+0000","last_peered":"2024-09-11T07:28:05.652030+0000","last_clean":"2024-09-11T07:28:05.652030+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:28:05.652030+0000","last_undegraded":"2024-09-11T07:28:05.652030+0000","last_fullsized":"2024-09-11T07:28:05.652030+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:38.732795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:38.732795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:56.730699+0000","scrub_duration":0.0025091950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":66,"reported_epoch":85,"state":"peering","last_fresh":"2024-09-11T07:28:08.012634+0000","last_change":"2024-09-11T07:28:08.012914+0000","last_active":"2024-09-11T07:28:08.012606+0000","last_peered":"2024-09-11T07:28:05.001388+0000","last_clean":"2024-09-11T07:28:05.001388+0000","last_became_active":"2024-09-11T07:26:33.541154+0000","last_became_peered":"2024-09-11T07:26:33.541154+0000","last_unstale":"2024-09-11T07:28:08.012634+0000","last_undegraded":"2024-09-11T07:28:08.012634+0000","last_fullsized":"2024-09-11T07:28:08.012634+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:28:03.164220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:03.164220+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:27.958211+0000","scrub_duration":0.0020677629999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2,5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.003303+0000","last_change":"2024-09-11T07:26:35.559093+0000","last_active":"2024-09-11T07:28:05.003303+0000","last_peered":"2024-09-11T07:28:05.003303+0000","last_clean":"2024-09-11T07:28:05.003303+0000","last_became_active":"2024-09-11T07:26:35.558809+0000","last_became_peered":"2024-09-11T07:26:35.558809+0000","last_unstale":"2024-09-11T07:28:05.003303+0000","last_undegraded":"2024-09-11T07:28:05.003303+0000","last_fullsized":"2024-09-11T07:28:05.003303+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.653065+0000","last_change":"2024-09-11T07:28:00.126756+0000","last_active":"2024-09-11T07:28:05.653065+0000","last_peered":"2024-09-11T07:28:05.653065+0000","last_clean":"2024-09-11T07:28:05.653065+0000","last_became_active":"2024-09-11T07:26:37.862373+0000","last_became_peered":"2024-09-11T07:26:37.862373+0000","last_unstale":"2024-09-11T07:28:05.653065+0000","last_undegraded":"2024-09-11T07:28:05.653065+0000","last_fullsized":"2024-09-11T07:28:05.653065+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:00.126702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.126702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:18.534592+0000","scrub_duration":0.001184903,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.001235+0000","last_change":"2024-09-11T07:27:33.983889+0000","last_active":"2024-09-11T07:28:05.001235+0000","last_peered":"2024-09-11T07:28:05.001235+0000","last_clean":"2024-09-11T07:28:05.001235+0000","last_became_active":"2024-09-11T07:26:31.424651+0000","last_became_peered":"2024-09-11T07:26:31.424651+0000","last_unstale":"2024-09-11T07:28:05.001235+0000","last_undegraded":"2024-09-11T07:28:05.001235+0000","last_fullsized":"2024-09-11T07:28:05.001235+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:33.983813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:33.983813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:55.264233+0000","scrub_duration":0.001290759,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":70,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:06.304991+0000","last_change":"2024-09-11T07:28:00.272702+0000","last_active":"2024-09-11T07:28:06.304991+0000","last_peered":"2024-09-11T07:28:06.304991+0000","last_clean":"2024-09-11T07:28:06.304991+0000","last_became_active":"2024-09-11T07:26:33.542544+0000","last_became_peered":"2024-09-11T07:26:33.542544+0000","last_unstale":"2024-09-11T07:28:06.304991+0000","last_undegraded":"2024-09-11T07:28:06.304991+0000","last_fullsized":"2024-09-11T07:28:06.304991+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-09-11T07:28:00.272628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.272628+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:05.374102+0000","scrub_duration":0.0021830019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:07.721242+0000","last_change":"2024-09-11T07:28:07.721176+0000","last_active":"2024-09-11T07:28:07.721242+0000","last_peered":"2024-09-11T07:28:07.721242+0000","last_clean":"2024-09-11T07:28:07.721242+0000","last_became_active":"2024-09-11T07:26:35.555862+0000","last_became_peered":"2024-09-11T07:26:35.555862+0000","last_unstale":"2024-09-11T07:28:07.721242+0000","last_undegraded":"2024-09-11T07:28:07.721242+0000","last_fullsized":"2024-09-11T07:28:07.721242+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:07.721112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:07.721112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:31.345841+0000","scrub_duration":0.0020675020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:06.674667+0000","last_change":"2024-09-11T07:28:06.674616+0000","last_active":"2024-09-11T07:28:06.674667+0000","last_peered":"2024-09-11T07:28:06.674667+0000","last_clean":"2024-09-11T07:28:06.674667+0000","last_became_active":"2024-09-11T07:26:37.861647+0000","last_became_peered":"2024-09-11T07:26:37.861647+0000","last_unstale":"2024-09-11T07:28:06.674667+0000","last_undegraded":"2024-09-11T07:28:06.674667+0000","last_fullsized":"2024-09-11T07:28:06.674667+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:06.674558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:06.674558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:17.029204+0000","scrub_duration":0.001489488,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.652303+0000","last_change":"2024-09-11T07:27:45.732604+0000","last_active":"2024-09-11T07:28:05.652303+0000","last_peered":"2024-09-11T07:28:05.652303+0000","last_clean":"2024-09-11T07:28:05.652303+0000","last_became_active":"2024-09-11T07:26:31.420598+0000","last_became_peered":"2024-09-11T07:26:31.420598+0000","last_unstale":"2024-09-11T07:28:05.652303+0000","last_undegraded":"2024-09-11T07:28:05.652303+0000","last_fullsized":"2024-09-11T07:28:05.652303+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:45.732528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:45.732528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:48.605075+0000","scrub_duration":0.0015466659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"64'12","reported_seq":68,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.652486+0000","last_change":"2024-09-11T07:27:59.737848+0000","last_active":"2024-09-11T07:28:05.652486+0000","last_peered":"2024-09-11T07:28:05.652486+0000","last_clean":"2024-09-11T07:28:05.652486+0000","last_became_active":"2024-09-11T07:26:33.542711+0000","last_became_peered":"2024-09-11T07:26:33.542711+0000","last_unstale":"2024-09-11T07:28:05.652486+0000","last_undegraded":"2024-09-11T07:28:05.652486+0000","last_fullsized":"2024-09-11T07:28:05.652486+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-09-11T07:27:59.737791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:59.737791+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:11.558595+0000","scrub_duration":0.00367732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T07:28:05.652763+0000","last_change":"2024-09-11T07:27:57.199239+0000","last_active":"2024-09-11T07:28:05.652763+0000","last_peered":"2024-09-11T07:28:05.652763+0000","last_clean":"2024-09-11T07:28:05.652763+0000","last_became_active":"2024-09-11T07:26:35.562037+0000","last_became_peered":"2024-09-11T07:26:35.562037+0000","last_unstale":"2024-09-11T07:28:05.652763+0000","last_undegraded":"2024-09-11T07:28:05.652763+0000","last_fullsized":"2024-09-11T07:28:05.652763+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:57.199173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.199173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.093487+0000","scrub_duration":0.0015950770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":421,"ondisk_log_size":421,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":86,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299416,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298556,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446596,"statfs":{"total":95995035648,"available":95689314304,"internally_reserved":0,"allocated":1220608,"data_stored":876604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.54100000000000004}]},{"osd":1,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.49299999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.39800000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":12.625999999999999,"5min":12.625999999999999,"15min":12.625999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.51200000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.43099999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.378}]},{"osd":6,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.47799999999999998}]}]},{"osd":6,"up_from":44,"seq":188978561052,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298536,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446616,"statfs":{"total":95995035648,"available":95689334784,"internally_reserved":0,"allocated":1208320,"data_stored":881674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.374,"5min":0.247,"15min":0.247},"max":{"1min":0.65000000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.60499999999999998,"15min":0.624},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.754,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.39700000000000002}]},{"osd":1,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.82899999999999996,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.377},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.49299999999999999,"15min":0.495},"min":{"1min":0.35799999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.38900000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.61499999999999999,"15min":0.63500000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.70299999999999996,"5min":3.234,"15min":3.234},"last":0.37},{"interface":"front","average":{"1min":0.496,"5min":0.59099999999999997,"15min":0.60699999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.80200000000000005,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.41399999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51500000000000001,"15min":0.51800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.63500000000000001,"15min":0.65500000000000003},"min":{"1min":0.41199999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.66900000000000004,"5min":3.1739999999999999,"15min":3.1739999999999999},"last":0.42399999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.58299999999999996,"15min":0.60899999999999999},"min":{"1min":0.32700000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.65200000000000002,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.251},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.28399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.35999999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.65600000000000003,"15min":0.69699999999999995},"min":{"1min":0.32400000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64000000000000001,"5min":5.7329999999999997,"15min":5.7329999999999997},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.52000000000000002,"15min":0.52700000000000002},"min":{"1min":0.34000000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70899999999999996,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.315}]},{"osd":7,"last update":"Wed Sep 11 07:27:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.6349999999999998,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.34300000000000003}]}]},{"osd":5,"up_from":37,"seq":158913789986,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297960,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447192,"statfs":{"total":95995035648,"available":95689924608,"internally_reserved":0,"allocated":618496,"data_stored":292187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.35499999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.50600000000000001,"15min":0.502},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.39500000000000002}]},{"osd":1,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.49199999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.56299999999999994}]},{"osd":3,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.52500000000000002}]},{"osd":4,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.375,"15min":0.37},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.436},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.42599999999999999,"15min":0.41999999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.51200000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:28:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.41599999999999998,"15min":0.40600000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.42699999999999999,"15min":0.42199999999999999},"min":{"1min":0.32100000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.58799999999999997}]},{"osd":7,"last update":"Wed Sep 11 07:27:21 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.60499999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986214,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297984,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447168,"statfs":{"total":95995035648,"available":95689900032,"internally_reserved":0,"allocated":643072,"data_stored":292632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51600000000000001,"15min":0.51300000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68999999999999995,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.502,"15min":0.498},"min":{"1min":0.36099999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.442}]},{"osd":1,"last update":"Wed Sep 11 07:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.5,"15min":0.49399999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.71299999999999997,"15min":0.73699999999999999},"min":{"1min":0.372,"5min":0.318,"15min":0.318},"max":{"1min":1.1200000000000001,"5min":4.5510000000000002,"15min":4.5510000000000002},"last":0.39000000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.36899999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77400000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.49399999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.34499999999999997,"15min":0.33800000000000002},"min":{"1min":0.27600000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.378,"15min":0.372},"min":{"1min":0.29399999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.53000000000000003}]},{"osd":6,"last update":"Wed Sep 11 07:27:59 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42199999999999999,"15min":0.41799999999999998},"min":{"1min":0.315,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.44,"15min":0.43099999999999999},"min":{"1min":0.33500000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.50900000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:27:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.46400000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182443,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297992,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447160,"statfs":{"total":95995035648,"available":95689891840,"internally_reserved":0,"allocated":651264,"data_stored":292096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:54 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38600000000000001,"15min":0.39300000000000002},"min":{"1min":0.20799999999999999,"5min":0.185,"15min":0.185},"max":{"1min":0.54300000000000004,"5min":0.621,"15min":0.621},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.42499999999999999,"15min":0.441},"min":{"1min":0.26200000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.30599999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:27:54 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.43099999999999999,"15min":0.437},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.625,"5min":0.752,"15min":0.752},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.41599999999999998,"15min":0.43099999999999999},"min":{"1min":0.23599999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.37}]},{"osd":2,"last update":"Wed Sep 11 07:27:54 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.45100000000000001,"15min":0.46500000000000002},"min":{"1min":0.26900000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.45400000000000001,"15min":0.46899999999999997},"min":{"1min":0.28899999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.32400000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:27:12 2024","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":0.746,"15min":0.62},"min":{"1min":0.47299999999999998,"5min":0.376,"15min":0.376},"max":{"1min":21.856000000000002,"5min":21.856000000000002,"15min":21.856000000000002},"last":0.498},{"interface":"front","average":{"1min":1.345,"5min":0.72199999999999998,"15min":0.61799999999999999},"min":{"1min":0.432,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":19.146000000000001,"5min":19.146000000000001,"15min":19.146000000000001},"last":0.55200000000000005}]},{"osd":5,"last update":"Wed Sep 11 07:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.3069999999999999,"5min":0.71999999999999997,"15min":0.622},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":18.416,"5min":18.416,"15min":18.416},"last":0.46899999999999997}]},{"osd":6,"last update":"Wed Sep 11 07:27:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":1.306,"15min":1.4330000000000001},"min":{"1min":0.439,"5min":0.437,"15min":0.437},"max":{"1min":0.72899999999999998,"5min":20.219999999999999,"15min":20.219999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.55500000000000005,"5min":1.319,"15min":1.446},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72699999999999998,"5min":19.609000000000002,"15min":19.609000000000002},"last":0.48099999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.51300000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411376,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297968,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447184,"statfs":{"total":95995035648,"available":95689916416,"internally_reserved":0,"allocated":614400,"data_stored":291433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:26 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.375,"15min":0.36299999999999999},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.38200000000000001,"15min":0.374},"min":{"1min":0.26600000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.55900000000000005,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.311}]},{"osd":1,"last update":"Wed Sep 11 07:27:26 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.40300000000000002,"15min":0.38600000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.40200000000000002,"15min":0.38300000000000001},"min":{"1min":0.218,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.32600000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.439,"15min":0.42299999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64500000000000002,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.44,"5min":0.45900000000000002,"15min":0.45400000000000001},"min":{"1min":0.26600000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65000000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.35399999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:27:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55800000000000005,"15min":0.55300000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57699999999999996,"15min":0.57599999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80700000000000005,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.52300000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:27:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.68799999999999994,"15min":0.69699999999999995},"min":{"1min":0.45500000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85899999999999999,"5min":2.931,"15min":2.931},"last":0.48799999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:27:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.85299999999999998,"5min":1.494,"15min":1.494},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.63100000000000001,"15min":0.63300000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.875,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.42899999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54200000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298008,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447144,"statfs":{"total":95995035648,"available":95689875456,"internally_reserved":0,"allocated":667648,"data_stored":292953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:28:04 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.34000000000000002,"15min":0.33100000000000002},"min":{"1min":0.22800000000000001,"5min":0.186,"15min":0.186},"max":{"1min":0.49299999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.36499999999999999,"15min":0.35499999999999998},"min":{"1min":0.22700000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.57899999999999996}]},{"osd":2,"last update":"Wed Sep 11 07:27:26 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.41899999999999998,"15min":0.41599999999999998},"min":{"1min":0.317,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.41199999999999998,"15min":0.40799999999999997},"min":{"1min":0.28699999999999998,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.56699999999999995}]},{"osd":3,"last update":"Wed Sep 11 07:27:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.44500000000000001,"15min":0.44700000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.60699999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.41399999999999998},"min":{"1min":0.307,"5min":0.253,"15min":0.253},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.54800000000000004}]},{"osd":4,"last update":"Wed Sep 11 07:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.751},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.71999999999999997}]},{"osd":5,"last update":"Wed Sep 11 07:27:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.44,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.71699999999999997,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.66600000000000004}]},{"osd":6,"last update":"Wed Sep 11 07:27:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57199999999999995,"15min":0.57499999999999996},"min":{"1min":0.41199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.754,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.40999999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.73999999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.70499999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738425,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298552,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446600,"statfs":{"total":95995035648,"available":95689318400,"internally_reserved":0,"allocated":1224704,"data_stored":882006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:28:02 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.36699999999999999,"15min":0.34499999999999997},"min":{"1min":0.246,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.35899999999999999,"15min":0.34200000000000003},"min":{"1min":0.23799999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.38500000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:27:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.40400000000000003,"15min":0.40699999999999997},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.371},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.59299999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.32400000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:27:55 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45800000000000002,"15min":0.47399999999999998},"min":{"1min":0.317,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.61199999999999999,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.46300000000000002,"15min":0.48099999999999998},"min":{"1min":0.28299999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.56200000000000006,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.41499999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:27:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56299999999999994,"15min":0.56899999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69499999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59799999999999998,"15min":0.60099999999999998},"min":{"1min":0.42499999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.58899999999999997}]},{"osd":5,"last update":"Wed Sep 11 07:27:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.40000000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.79100000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.41699999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78100000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51300000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:27:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59399999999999997,"15min":0.59699999999999998},"min":{"1min":0.34799999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60099999999999998,"15min":0.60399999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.76400000000000001,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.61599999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.501},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.56899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:28:15.238 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:28:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:15 smithi104 ceph-mon[33507]: 5.b scrub starts 2024-09-11T07:28:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:15 smithi104 ceph-mon[33507]: 5.b scrub ok 2024-09-11T07:28:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:15 smithi104 ceph-mon[33507]: 6.7 scrub starts 2024-09-11T07:28:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:15 smithi104 ceph-mon[33507]: 6.7 scrub ok 2024-09-11T07:28:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:15 smithi104 ceph-mon[33507]: from='client.14772 v1:172.21.15.22:0/4156975087' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:28:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:15 smithi104 ceph-mon[33507]: 6.8 scrub starts 2024-09-11T07:28:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:15 smithi104 ceph-mon[33507]: 6.8 scrub ok 2024-09-11T07:28:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:15 smithi104 ceph-mon[33507]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 2 objects/s recovering 2024-09-11T07:28:15.537 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:28:15.633 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:15 smithi022 systemd[1]: ceph-12a285d8-700e-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-11T07:28:15.633 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:15 smithi022 systemd[1]: Stopped Ceph alertmanager.a for 12a285d8-700e-11ef-bcea-c7b262605968. 2024-09-11T07:28:15.634 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:15 smithi022 systemd[1]: Starting Ceph alertmanager.a for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:28:15.917 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:15 smithi022 podman[72487]: 2024-09-11 07:28:15.535667367 +0000 UTC m=+0.019889798 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-11T07:28:15.918 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:15 smithi022 podman[72487]: 2024-09-11 07:28:15.670046726 +0000 UTC m=+0.154269182 volume create b51e41168b703fb8c54a4e0ff2710f1fd1b4124dafdffe7fe8acd2b0e7c40df9 2024-09-11T07:28:15.918 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:15 smithi022 podman[72487]: 2024-09-11 07:28:15.779096251 +0000 UTC m=+0.263318666 container create 18e7fd2d2738f439ee3ee99ca3783c563e95bc06fe17cff00ce3afeea03a502d (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:28:16.390 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:16 smithi022 podman[72487]: 2024-09-11 07:28:16.108498569 +0000 UTC m=+0.592720986 container init 18e7fd2d2738f439ee3ee99ca3783c563e95bc06fe17cff00ce3afeea03a502d (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:28:16.390 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:16 smithi022 podman[72487]: 2024-09-11 07:28:16.111580939 +0000 UTC m=+0.595803364 container start 18e7fd2d2738f439ee3ee99ca3783c563e95bc06fe17cff00ce3afeea03a502d (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:28:16.390 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:28:16.124Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-11T07:28:16.390 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:28:16.125Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-11T07:28:16.390 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:28:16.125Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.22 port=9094 2024-09-11T07:28:16.390 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:28:16.126Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-11T07:28:16.391 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:28:16.151Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T07:28:16.391 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:28:16.151Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T07:28:16.391 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:28:16.154Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-11T07:28:16.391 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:28:16.154Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-11T07:28:16.391 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:16 smithi022 bash[72487]: 18e7fd2d2738f439ee3ee99ca3783c563e95bc06fe17cff00ce3afeea03a502d 2024-09-11T07:28:16.391 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:16 smithi022 systemd[1]: Started Ceph alertmanager.a for 12a285d8-700e-11ef-bcea-c7b262605968. 2024-09-11T07:28:16.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-mon[29462]: 2.12 scrub starts 2024-09-11T07:28:16.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-mon[29462]: 2.12 scrub ok 2024-09-11T07:28:16.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-mon[29462]: 4.18 deep-scrub starts 2024-09-11T07:28:16.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-mon[29462]: 4.18 deep-scrub ok 2024-09-11T07:28:16.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-mon[29462]: 5.5 scrub starts 2024-09-11T07:28:16.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-mon[29462]: 5.5 scrub ok 2024-09-11T07:28:16.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-mon[29462]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 2 objects/s recovering 2024-09-11T07:28:16.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:16.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:16.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-mon[29462]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T07:28:16.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-mon[29462]: Reconfiguring daemon prometheus.a on smithi104 2024-09-11T07:28:16.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-mon[37330]: 2.12 scrub starts 2024-09-11T07:28:16.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-mon[37330]: 2.12 scrub ok 2024-09-11T07:28:16.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-mon[37330]: 4.18 deep-scrub starts 2024-09-11T07:28:16.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-mon[37330]: 4.18 deep-scrub ok 2024-09-11T07:28:16.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-mon[37330]: 5.5 scrub starts 2024-09-11T07:28:16.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-mon[37330]: 5.5 scrub ok 2024-09-11T07:28:16.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-mon[37330]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 2 objects/s recovering 2024-09-11T07:28:16.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:16.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:16.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-mon[37330]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T07:28:16.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:16 smithi022 ceph-mon[37330]: Reconfiguring daemon prometheus.a on smithi104 2024-09-11T07:28:16.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:16 smithi104 ceph-mon[33507]: 2.12 scrub starts 2024-09-11T07:28:16.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:16 smithi104 ceph-mon[33507]: 2.12 scrub ok 2024-09-11T07:28:16.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:16 smithi104 ceph-mon[33507]: 4.18 deep-scrub starts 2024-09-11T07:28:16.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:16 smithi104 ceph-mon[33507]: 4.18 deep-scrub ok 2024-09-11T07:28:16.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:16 smithi104 ceph-mon[33507]: 5.5 scrub starts 2024-09-11T07:28:16.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:16 smithi104 ceph-mon[33507]: 5.5 scrub ok 2024-09-11T07:28:16.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:16 smithi104 ceph-mon[33507]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 2 objects/s recovering 2024-09-11T07:28:16.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:16 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:16.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:16 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:16.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:16 smithi104 ceph-mon[33507]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T07:28:16.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:16 smithi104 ceph-mon[33507]: Reconfiguring daemon prometheus.a on smithi104 2024-09-11T07:28:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:17 smithi022 ceph-mon[29462]: 2.0 deep-scrub starts 2024-09-11T07:28:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:17 smithi022 ceph-mon[29462]: 2.0 deep-scrub ok 2024-09-11T07:28:17.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:17 smithi022 ceph-mon[37330]: 2.0 deep-scrub starts 2024-09-11T07:28:17.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:17 smithi022 ceph-mon[37330]: 2.0 deep-scrub ok 2024-09-11T07:28:18.025 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:28:18.026 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:28:18.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:17 smithi104 ceph-mon[33507]: 2.0 deep-scrub starts 2024-09-11T07:28:18.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:17 smithi104 ceph-mon[33507]: 2.0 deep-scrub ok 2024-09-11T07:28:18.314 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:18 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:28:18.127Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.001074894s 2024-09-11T07:28:18.633 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2024-09-11T07:28:17.948545+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":949,"num_read_kb":806,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":590588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":580,"ondisk_log_size":580,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":399,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385624,"kb_used_data":6796,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575592,"statfs":{"total":767960285184,"available":765517406208,"internally_reserved":0,"allocated":6959104,"data_stored":4160177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.941745"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008184+0000","last_change":"2024-09-11T07:26:37.863505+0000","last_active":"2024-09-11T07:28:08.008184+0000","last_peered":"2024-09-11T07:28:08.008184+0000","last_clean":"2024-09-11T07:28:08.008184+0000","last_became_active":"2024-09-11T07:26:37.863165+0000","last_became_peered":"2024-09-11T07:26:37.863165+0000","last_unstale":"2024-09-11T07:28:08.008184+0000","last_undegraded":"2024-09-11T07:28:08.008184+0000","last_fullsized":"2024-09-11T07:28:08.008184+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014603+0000","last_change":"2024-09-11T07:27:51.197966+0000","last_active":"2024-09-11T07:28:08.014603+0000","last_peered":"2024-09-11T07:28:08.014603+0000","last_clean":"2024-09-11T07:28:08.014603+0000","last_became_active":"2024-09-11T07:26:31.421005+0000","last_became_peered":"2024-09-11T07:26:31.421005+0000","last_unstale":"2024-09-11T07:28:08.014603+0000","last_undegraded":"2024-09-11T07:28:08.014603+0000","last_fullsized":"2024-09-11T07:28:08.014603+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:51.197903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:51.197903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:00.681709+0000","scrub_duration":0.001600113,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"82'22","reported_seq":78,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008850+0000","last_change":"2024-09-11T07:27:50.756777+0000","last_active":"2024-09-11T07:28:08.008850+0000","last_peered":"2024-09-11T07:28:08.008850+0000","last_clean":"2024-09-11T07:28:08.008850+0000","last_became_active":"2024-09-11T07:26:33.544627+0000","last_became_peered":"2024-09-11T07:26:33.544627+0000","last_unstale":"2024-09-11T07:28:08.008850+0000","last_undegraded":"2024-09-11T07:28:08.008850+0000","last_fullsized":"2024-09-11T07:28:08.008850+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.756707+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.257406+0000","scrub_duration":0.0049180669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008137+0000","last_change":"2024-09-11T07:26:35.557117+0000","last_active":"2024-09-11T07:28:08.008137+0000","last_peered":"2024-09-11T07:28:08.008137+0000","last_clean":"2024-09-11T07:28:08.008137+0000","last_became_active":"2024-09-11T07:26:35.556296+0000","last_became_peered":"2024-09-11T07:26:35.556296+0000","last_unstale":"2024-09-11T07:28:08.008137+0000","last_undegraded":"2024-09-11T07:28:08.008137+0000","last_fullsized":"2024-09-11T07:28:08.008137+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.009266+0000","last_change":"2024-09-11T07:27:36.776948+0000","last_active":"2024-09-11T07:28:08.009266+0000","last_peered":"2024-09-11T07:28:08.009266+0000","last_clean":"2024-09-11T07:28:08.009266+0000","last_became_active":"2024-09-11T07:26:31.420178+0000","last_became_peered":"2024-09-11T07:26:31.420178+0000","last_unstale":"2024-09-11T07:28:08.009266+0000","last_undegraded":"2024-09-11T07:28:08.009266+0000","last_fullsized":"2024-09-11T07:28:08.009266+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:36.776883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:36.776883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:49.961787+0000","scrub_duration":0.0017449290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"64'9","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008221+0000","last_change":"2024-09-11T07:26:33.541403+0000","last_active":"2024-09-11T07:28:08.008221+0000","last_peered":"2024-09-11T07:28:08.008221+0000","last_clean":"2024-09-11T07:28:08.008221+0000","last_became_active":"2024-09-11T07:26:33.540732+0000","last_became_peered":"2024-09-11T07:26:33.540732+0000","last_unstale":"2024-09-11T07:28:08.008221+0000","last_undegraded":"2024-09-11T07:28:08.008221+0000","last_fullsized":"2024-09-11T07:28:08.008221+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012014+0000","last_change":"2024-09-11T07:27:50.648473+0000","last_active":"2024-09-11T07:28:08.012014+0000","last_peered":"2024-09-11T07:28:08.012014+0000","last_clean":"2024-09-11T07:28:08.012014+0000","last_became_active":"2024-09-11T07:26:35.561096+0000","last_became_peered":"2024-09-11T07:26:35.561096+0000","last_unstale":"2024-09-11T07:28:08.012014+0000","last_undegraded":"2024-09-11T07:28:08.012014+0000","last_fullsized":"2024-09-11T07:28:08.012014+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.648420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:05.663592+0000","scrub_duration":0.001391446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008289+0000","last_change":"2024-09-11T07:26:37.567195+0000","last_active":"2024-09-11T07:28:08.008289+0000","last_peered":"2024-09-11T07:28:08.008289+0000","last_clean":"2024-09-11T07:28:08.008289+0000","last_became_active":"2024-09-11T07:26:37.566910+0000","last_became_peered":"2024-09-11T07:26:37.566910+0000","last_unstale":"2024-09-11T07:28:08.008289+0000","last_undegraded":"2024-09-11T07:28:08.008289+0000","last_fullsized":"2024-09-11T07:28:08.008289+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.015214+0000","last_change":"2024-09-11T07:26:31.423041+0000","last_active":"2024-09-11T07:28:08.015214+0000","last_peered":"2024-09-11T07:28:08.015214+0000","last_clean":"2024-09-11T07:28:08.015214+0000","last_became_active":"2024-09-11T07:26:31.422772+0000","last_became_peered":"2024-09-11T07:26:31.422772+0000","last_unstale":"2024-09-11T07:28:08.015214+0000","last_undegraded":"2024-09-11T07:28:08.015214+0000","last_fullsized":"2024-09-11T07:28:08.015214+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"82'7","reported_seq":58,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008514+0000","last_change":"2024-09-11T07:27:52.097170+0000","last_active":"2024-09-11T07:28:08.008514+0000","last_peered":"2024-09-11T07:28:08.008514+0000","last_clean":"2024-09-11T07:28:08.008514+0000","last_became_active":"2024-09-11T07:26:33.540351+0000","last_became_peered":"2024-09-11T07:26:33.540351+0000","last_unstale":"2024-09-11T07:28:08.008514+0000","last_undegraded":"2024-09-11T07:28:08.008514+0000","last_fullsized":"2024-09-11T07:28:08.008514+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'7","last_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_deep_scrub":"82'7","last_deep_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_clean_scrub_stamp":"2024-09-11T07:27:52.097071+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:53.905741+0000","scrub_duration":0.0025786670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014667+0000","last_change":"2024-09-11T07:26:35.563858+0000","last_active":"2024-09-11T07:28:08.014667+0000","last_peered":"2024-09-11T07:28:08.014667+0000","last_clean":"2024-09-11T07:28:08.014667+0000","last_became_active":"2024-09-11T07:26:35.563536+0000","last_became_peered":"2024-09-11T07:26:35.563536+0000","last_unstale":"2024-09-11T07:28:08.014667+0000","last_undegraded":"2024-09-11T07:28:08.014667+0000","last_fullsized":"2024-09-11T07:28:08.014667+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011217+0000","last_change":"2024-09-11T07:26:37.562008+0000","last_active":"2024-09-11T07:28:08.011217+0000","last_peered":"2024-09-11T07:28:08.011217+0000","last_clean":"2024-09-11T07:28:08.011217+0000","last_became_active":"2024-09-11T07:26:37.561758+0000","last_became_peered":"2024-09-11T07:26:37.561758+0000","last_unstale":"2024-09-11T07:28:08.011217+0000","last_undegraded":"2024-09-11T07:28:08.011217+0000","last_fullsized":"2024-09-11T07:28:08.011217+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.015134+0000","last_change":"2024-09-11T07:27:52.058013+0000","last_active":"2024-09-11T07:28:08.015134+0000","last_peered":"2024-09-11T07:28:08.015134+0000","last_clean":"2024-09-11T07:28:08.015134+0000","last_became_active":"2024-09-11T07:26:31.420969+0000","last_became_peered":"2024-09-11T07:26:31.420969+0000","last_unstale":"2024-09-11T07:28:08.015134+0000","last_undegraded":"2024-09-11T07:28:08.015134+0000","last_fullsized":"2024-09-11T07:28:08.015134+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_clean_scrub_stamp":"2024-09-11T07:27:52.057959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:12.902027+0000","scrub_duration":0.0014644840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"64'9","reported_seq":62,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008543+0000","last_change":"2024-09-11T07:27:47.108704+0000","last_active":"2024-09-11T07:28:08.008543+0000","last_peered":"2024-09-11T07:28:08.008543+0000","last_clean":"2024-09-11T07:28:08.008543+0000","last_became_active":"2024-09-11T07:26:33.544386+0000","last_became_peered":"2024-09-11T07:26:33.544386+0000","last_unstale":"2024-09-11T07:28:08.008543+0000","last_undegraded":"2024-09-11T07:28:08.008543+0000","last_fullsized":"2024-09-11T07:28:08.008543+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:27:47.108645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.108645+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:08.302050+0000","scrub_duration":0.0024650560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011669+0000","last_change":"2024-09-11T07:27:57.092029+0000","last_active":"2024-09-11T07:28:08.011669+0000","last_peered":"2024-09-11T07:28:08.011669+0000","last_clean":"2024-09-11T07:28:08.011669+0000","last_became_active":"2024-09-11T07:26:35.569630+0000","last_became_peered":"2024-09-11T07:26:35.569630+0000","last_unstale":"2024-09-11T07:28:08.011669+0000","last_undegraded":"2024-09-11T07:28:08.011669+0000","last_fullsized":"2024-09-11T07:28:08.011669+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:57.091958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.091958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:57.303120+0000","scrub_duration":0.001785872,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.333896+0000","last_change":"2024-09-11T07:28:09.333866+0000","last_active":"2024-09-11T07:28:09.333896+0000","last_peered":"2024-09-11T07:28:09.333896+0000","last_clean":"2024-09-11T07:28:09.333896+0000","last_became_active":"2024-09-11T07:26:37.565475+0000","last_became_peered":"2024-09-11T07:26:37.565475+0000","last_unstale":"2024-09-11T07:28:09.333896+0000","last_undegraded":"2024-09-11T07:28:09.333896+0000","last_fullsized":"2024-09-11T07:28:09.333896+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:09.333804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:09.333804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:11.385070+0000","scrub_duration":0.003498438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":102,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.027694+0000","last_change":"2024-09-11T07:28:09.027694+0000","last_active":"2024-09-11T07:28:09.027694+0000","last_peered":"2024-09-11T07:28:09.027694+0000","last_clean":"2024-09-11T07:28:09.027694+0000","last_became_active":"2024-09-11T07:28:09.018213+0000","last_became_peered":"2024-09-11T07:28:09.018213+0000","last_unstale":"2024-09-11T07:28:09.027694+0000","last_undegraded":"2024-09-11T07:28:09.027694+0000","last_fullsized":"2024-09-11T07:28:09.027694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-09-11T07:28:04.137453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:04.137453+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:05.572290+0000","scrub_duration":0.001914083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:11.897419+0000","last_change":"2024-09-11T07:28:11.897385+0000","last_active":"2024-09-11T07:28:11.897419+0000","last_peered":"2024-09-11T07:28:11.897419+0000","last_clean":"2024-09-11T07:28:11.897419+0000","last_became_active":"2024-09-11T07:26:31.419488+0000","last_became_peered":"2024-09-11T07:26:31.419488+0000","last_unstale":"2024-09-11T07:28:11.897419+0000","last_undegraded":"2024-09-11T07:28:11.897419+0000","last_fullsized":"2024-09-11T07:28:11.897419+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:11.897319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:11.897319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:18.864403+0000","scrub_duration":0.0013939480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012787+0000","last_change":"2024-09-11T07:26:35.559684+0000","last_active":"2024-09-11T07:28:08.012787+0000","last_peered":"2024-09-11T07:28:08.012787+0000","last_clean":"2024-09-11T07:28:08.012787+0000","last_became_active":"2024-09-11T07:26:35.557989+0000","last_became_peered":"2024-09-11T07:26:35.557989+0000","last_unstale":"2024-09-11T07:28:08.012787+0000","last_undegraded":"2024-09-11T07:28:08.012787+0000","last_fullsized":"2024-09-11T07:28:08.012787+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008505+0000","last_change":"2024-09-11T07:27:57.274611+0000","last_active":"2024-09-11T07:28:08.008505+0000","last_peered":"2024-09-11T07:28:08.008505+0000","last_clean":"2024-09-11T07:28:08.008505+0000","last_became_active":"2024-09-11T07:26:37.564554+0000","last_became_peered":"2024-09-11T07:26:37.564554+0000","last_unstale":"2024-09-11T07:28:08.008505+0000","last_undegraded":"2024-09-11T07:28:08.008505+0000","last_fullsized":"2024-09-11T07:28:08.008505+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:57.274554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.274554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.058396+0000","scrub_duration":0.001419584,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":51,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008687+0000","last_change":"2024-09-11T07:26:33.536333+0000","last_active":"2024-09-11T07:28:08.008687+0000","last_peered":"2024-09-11T07:28:08.008687+0000","last_clean":"2024-09-11T07:28:08.008687+0000","last_became_active":"2024-09-11T07:26:33.535411+0000","last_became_peered":"2024-09-11T07:26:33.535411+0000","last_unstale":"2024-09-11T07:28:08.008687+0000","last_undegraded":"2024-09-11T07:28:08.008687+0000","last_fullsized":"2024-09-11T07:28:08.008687+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012635+0000","last_change":"2024-09-11T07:26:31.419259+0000","last_active":"2024-09-11T07:28:08.012635+0000","last_peered":"2024-09-11T07:28:08.012635+0000","last_clean":"2024-09-11T07:28:08.012635+0000","last_became_active":"2024-09-11T07:26:31.418178+0000","last_became_peered":"2024-09-11T07:26:31.418178+0000","last_unstale":"2024-09-11T07:28:08.012635+0000","last_undegraded":"2024-09-11T07:28:08.012635+0000","last_fullsized":"2024-09-11T07:28:08.012635+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"64'8","reported_seq":72,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.009801+0000","last_change":"2024-09-11T07:27:47.127485+0000","last_active":"2024-09-11T07:28:09.009801+0000","last_peered":"2024-09-11T07:28:09.009801+0000","last_clean":"2024-09-11T07:28:09.009801+0000","last_became_active":"2024-09-11T07:26:35.564531+0000","last_became_peered":"2024-09-11T07:26:35.564531+0000","last_unstale":"2024-09-11T07:28:09.009801+0000","last_undegraded":"2024-09-11T07:28:09.009801+0000","last_fullsized":"2024-09-11T07:28:09.009801+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:27:47.127428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.127428+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:58.343731+0000","scrub_duration":0.0014091329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008839+0000","last_change":"2024-09-11T07:26:37.567193+0000","last_active":"2024-09-11T07:28:08.008839+0000","last_peered":"2024-09-11T07:28:08.008839+0000","last_clean":"2024-09-11T07:28:08.008839+0000","last_became_active":"2024-09-11T07:26:37.566919+0000","last_became_peered":"2024-09-11T07:26:37.566919+0000","last_unstale":"2024-09-11T07:28:08.008839+0000","last_undegraded":"2024-09-11T07:28:08.008839+0000","last_fullsized":"2024-09-11T07:28:08.008839+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"83'22","reported_seq":78,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.013407+0000","last_change":"2024-09-11T07:28:01.719185+0000","last_active":"2024-09-11T07:28:08.013407+0000","last_peered":"2024-09-11T07:28:08.013407+0000","last_clean":"2024-09-11T07:28:08.013407+0000","last_became_active":"2024-09-11T07:26:33.549733+0000","last_became_peered":"2024-09-11T07:26:33.549733+0000","last_unstale":"2024-09-11T07:28:08.013407+0000","last_undegraded":"2024-09-11T07:28:08.013407+0000","last_fullsized":"2024-09-11T07:28:08.013407+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'22","last_scrub_stamp":"2024-09-11T07:28:01.719128+0000","last_deep_scrub":"83'22","last_deep_scrub_stamp":"2024-09-11T07:28:01.719128+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.719128+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:27.431086+0000","scrub_duration":0.0047817240000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012176+0000","last_change":"2024-09-11T07:27:35.589317+0000","last_active":"2024-09-11T07:28:08.012176+0000","last_peered":"2024-09-11T07:28:08.012176+0000","last_clean":"2024-09-11T07:28:08.012176+0000","last_became_active":"2024-09-11T07:26:31.418478+0000","last_became_peered":"2024-09-11T07:26:31.418478+0000","last_unstale":"2024-09-11T07:28:08.012176+0000","last_undegraded":"2024-09-11T07:28:08.012176+0000","last_fullsized":"2024-09-11T07:28:08.012176+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:35.589251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:35.589251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:01.839680+0000","scrub_duration":0.0014929380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":75,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.010315+0000","last_change":"2024-09-11T07:27:56.218355+0000","last_active":"2024-09-11T07:28:09.010315+0000","last_peered":"2024-09-11T07:28:09.010315+0000","last_clean":"2024-09-11T07:28:09.010315+0000","last_became_active":"2024-09-11T07:26:35.560406+0000","last_became_peered":"2024-09-11T07:26:35.560406+0000","last_unstale":"2024-09-11T07:28:09.010315+0000","last_undegraded":"2024-09-11T07:28:09.010315+0000","last_fullsized":"2024-09-11T07:28:09.010315+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:27:56.218301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:56.218301+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:21.374837+0000","scrub_duration":0.001687413,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012888+0000","last_change":"2024-09-11T07:26:37.568289+0000","last_active":"2024-09-11T07:28:08.012888+0000","last_peered":"2024-09-11T07:28:08.012888+0000","last_clean":"2024-09-11T07:28:08.012888+0000","last_became_active":"2024-09-11T07:26:37.568005+0000","last_became_peered":"2024-09-11T07:26:37.568005+0000","last_unstale":"2024-09-11T07:28:08.012888+0000","last_undegraded":"2024-09-11T07:28:08.012888+0000","last_fullsized":"2024-09-11T07:28:08.012888+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"64'11","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.009008+0000","last_change":"2024-09-11T07:27:58.158485+0000","last_active":"2024-09-11T07:28:08.009008+0000","last_peered":"2024-09-11T07:28:08.009008+0000","last_clean":"2024-09-11T07:28:08.009008+0000","last_became_active":"2024-09-11T07:26:33.540777+0000","last_became_peered":"2024-09-11T07:26:33.540777+0000","last_unstale":"2024-09-11T07:28:08.009008+0000","last_undegraded":"2024-09-11T07:28:08.009008+0000","last_fullsized":"2024-09-11T07:28:08.009008+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:27:58.158420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.158420+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.832014+0000","scrub_duration":0.0021737459999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014438+0000","last_change":"2024-09-11T07:27:34.932151+0000","last_active":"2024-09-11T07:28:08.014438+0000","last_peered":"2024-09-11T07:28:08.014438+0000","last_clean":"2024-09-11T07:28:08.014438+0000","last_became_active":"2024-09-11T07:26:31.418238+0000","last_became_peered":"2024-09-11T07:26:31.418238+0000","last_unstale":"2024-09-11T07:28:08.014438+0000","last_undegraded":"2024-09-11T07:28:08.014438+0000","last_fullsized":"2024-09-11T07:28:08.014438+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:34.932092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:34.932092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:46.904196+0000","scrub_duration":0.002710259,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012589+0000","last_change":"2024-09-11T07:28:00.005119+0000","last_active":"2024-09-11T07:28:08.012589+0000","last_peered":"2024-09-11T07:28:08.012589+0000","last_clean":"2024-09-11T07:28:08.012589+0000","last_became_active":"2024-09-11T07:26:35.569081+0000","last_became_peered":"2024-09-11T07:26:35.569081+0000","last_unstale":"2024-09-11T07:28:08.012589+0000","last_undegraded":"2024-09-11T07:28:08.012589+0000","last_fullsized":"2024-09-11T07:28:08.012589+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:00.005031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.005031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:10.748420+0000","scrub_duration":0.001403533,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.009787+0000","last_change":"2024-09-11T07:27:46.752272+0000","last_active":"2024-09-11T07:28:08.009787+0000","last_peered":"2024-09-11T07:28:08.009787+0000","last_clean":"2024-09-11T07:28:08.009787+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:28:08.009787+0000","last_undegraded":"2024-09-11T07:28:08.009787+0000","last_fullsized":"2024-09-11T07:28:08.009787+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:46.752181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:46.752181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.072860+0000","scrub_duration":0.001508782,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"64'14","reported_seq":57,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008369+0000","last_change":"2024-09-11T07:26:33.538152+0000","last_active":"2024-09-11T07:28:08.008369+0000","last_peered":"2024-09-11T07:28:08.008369+0000","last_clean":"2024-09-11T07:28:08.008369+0000","last_became_active":"2024-09-11T07:26:33.537850+0000","last_became_peered":"2024-09-11T07:26:33.537850+0000","last_unstale":"2024-09-11T07:28:08.008369+0000","last_undegraded":"2024-09-11T07:28:08.008369+0000","last_fullsized":"2024-09-11T07:28:08.008369+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":58,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.015415+0000","last_change":"2024-09-11T07:28:09.015415+0000","last_active":"2024-09-11T07:28:09.015415+0000","last_peered":"2024-09-11T07:28:09.015415+0000","last_clean":"2024-09-11T07:28:09.015415+0000","last_became_active":"2024-09-11T07:28:09.014995+0000","last_became_peered":"2024-09-11T07:28:09.014995+0000","last_unstale":"2024-09-11T07:28:09.015415+0000","last_undegraded":"2024-09-11T07:28:09.015415+0000","last_fullsized":"2024-09-11T07:28:09.015415+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:39.014840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:39.014840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:56.130768+0000","scrub_duration":0.0032789239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"84'9","reported_seq":73,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.009195+0000","last_change":"2024-09-11T07:28:05.653289+0000","last_active":"2024-09-11T07:28:09.009195+0000","last_peered":"2024-09-11T07:28:09.009195+0000","last_clean":"2024-09-11T07:28:09.009195+0000","last_became_active":"2024-09-11T07:26:35.562997+0000","last_became_peered":"2024-09-11T07:26:35.562997+0000","last_unstale":"2024-09-11T07:28:09.009195+0000","last_undegraded":"2024-09-11T07:28:09.009195+0000","last_fullsized":"2024-09-11T07:28:09.009195+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-11T07:28:05.653237+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-11T07:28:05.653237+0000","last_clean_scrub_stamp":"2024-09-11T07:28:05.653237+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:22.713095+0000","scrub_duration":0.0027957030000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014528+0000","last_change":"2024-09-11T07:26:37.862564+0000","last_active":"2024-09-11T07:28:08.014528+0000","last_peered":"2024-09-11T07:28:08.014528+0000","last_clean":"2024-09-11T07:28:08.014528+0000","last_became_active":"2024-09-11T07:26:37.862246+0000","last_became_peered":"2024-09-11T07:26:37.862246+0000","last_unstale":"2024-09-11T07:28:08.014528+0000","last_undegraded":"2024-09-11T07:28:08.014528+0000","last_fullsized":"2024-09-11T07:28:08.014528+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"64'18","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008309+0000","last_change":"2024-09-11T07:26:33.546711+0000","last_active":"2024-09-11T07:28:08.008309+0000","last_peered":"2024-09-11T07:28:08.008309+0000","last_clean":"2024-09-11T07:28:08.008309+0000","last_became_active":"2024-09-11T07:26:33.546188+0000","last_became_peered":"2024-09-11T07:26:33.546188+0000","last_unstale":"2024-09-11T07:28:08.008309+0000","last_undegraded":"2024-09-11T07:28:08.008309+0000","last_fullsized":"2024-09-11T07:28:08.008309+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.013334+0000","last_change":"2024-09-11T07:26:31.421802+0000","last_active":"2024-09-11T07:28:08.013334+0000","last_peered":"2024-09-11T07:28:08.013334+0000","last_clean":"2024-09-11T07:28:08.013334+0000","last_became_active":"2024-09-11T07:26:31.421500+0000","last_became_peered":"2024-09-11T07:26:31.421500+0000","last_unstale":"2024-09-11T07:28:08.013334+0000","last_undegraded":"2024-09-11T07:28:08.013334+0000","last_fullsized":"2024-09-11T07:28:08.013334+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008249+0000","last_change":"2024-09-11T07:26:35.562816+0000","last_active":"2024-09-11T07:28:08.008249+0000","last_peered":"2024-09-11T07:28:08.008249+0000","last_clean":"2024-09-11T07:28:08.008249+0000","last_became_active":"2024-09-11T07:26:35.561284+0000","last_became_peered":"2024-09-11T07:26:35.561284+0000","last_unstale":"2024-09-11T07:28:08.008249+0000","last_undegraded":"2024-09-11T07:28:08.008249+0000","last_fullsized":"2024-09-11T07:28:08.008249+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008495+0000","last_change":"2024-09-11T07:26:37.565014+0000","last_active":"2024-09-11T07:28:08.008495+0000","last_peered":"2024-09-11T07:28:08.008495+0000","last_clean":"2024-09-11T07:28:08.008495+0000","last_became_active":"2024-09-11T07:26:37.564616+0000","last_became_peered":"2024-09-11T07:26:37.564616+0000","last_unstale":"2024-09-11T07:28:08.008495+0000","last_undegraded":"2024-09-11T07:28:08.008495+0000","last_fullsized":"2024-09-11T07:28:08.008495+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":82,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:12.466273+0000","last_change":"2024-09-11T07:28:12.466246+0000","last_active":"2024-09-11T07:28:12.466273+0000","last_peered":"2024-09-11T07:28:12.466273+0000","last_clean":"2024-09-11T07:28:12.466273+0000","last_became_active":"2024-09-11T07:26:33.540639+0000","last_became_peered":"2024-09-11T07:26:33.540639+0000","last_unstale":"2024-09-11T07:28:12.466273+0000","last_undegraded":"2024-09-11T07:28:12.466273+0000","last_fullsized":"2024-09-11T07:28:12.466273+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-11T07:28:12.466179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:12.466179+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:12.523827+0000","scrub_duration":0.0027522670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:12.896635+0000","last_change":"2024-09-11T07:28:12.896603+0000","last_active":"2024-09-11T07:28:12.896635+0000","last_peered":"2024-09-11T07:28:12.896635+0000","last_clean":"2024-09-11T07:28:12.896635+0000","last_became_active":"2024-09-11T07:26:31.419185+0000","last_became_peered":"2024-09-11T07:26:31.419185+0000","last_unstale":"2024-09-11T07:28:12.896635+0000","last_undegraded":"2024-09-11T07:28:12.896635+0000","last_fullsized":"2024-09-11T07:28:12.896635+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:12.896538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:12.896538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.577764+0000","scrub_duration":0.0017083070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011053+0000","last_change":"2024-09-11T07:27:41.000469+0000","last_active":"2024-09-11T07:28:08.011053+0000","last_peered":"2024-09-11T07:28:08.011053+0000","last_clean":"2024-09-11T07:28:08.011053+0000","last_became_active":"2024-09-11T07:26:35.565359+0000","last_became_peered":"2024-09-11T07:26:35.565359+0000","last_unstale":"2024-09-11T07:28:08.011053+0000","last_undegraded":"2024-09-11T07:28:08.011053+0000","last_fullsized":"2024-09-11T07:28:08.011053+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:41.000401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:41.000401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.049825+0000","scrub_duration":0.001655164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011937+0000","last_change":"2024-09-11T07:26:37.861736+0000","last_active":"2024-09-11T07:28:08.011937+0000","last_peered":"2024-09-11T07:28:08.011937+0000","last_clean":"2024-09-11T07:28:08.011937+0000","last_became_active":"2024-09-11T07:26:37.861279+0000","last_became_peered":"2024-09-11T07:26:37.861279+0000","last_unstale":"2024-09-11T07:28:08.011937+0000","last_undegraded":"2024-09-11T07:28:08.011937+0000","last_fullsized":"2024-09-11T07:28:08.011937+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":61,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012533+0000","last_change":"2024-09-11T07:27:51.642529+0000","last_active":"2024-09-11T07:28:08.012533+0000","last_peered":"2024-09-11T07:28:08.012533+0000","last_clean":"2024-09-11T07:28:08.012533+0000","last_became_active":"2024-09-11T07:26:33.543063+0000","last_became_peered":"2024-09-11T07:26:33.543063+0000","last_unstale":"2024-09-11T07:28:08.012533+0000","last_undegraded":"2024-09-11T07:28:08.012533+0000","last_fullsized":"2024-09-11T07:28:08.012533+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:27:51.642464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:51.642464+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:13.146284+0000","scrub_duration":0.0020211439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":55,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012985+0000","last_change":"2024-09-11T07:27:41.618362+0000","last_active":"2024-09-11T07:28:08.012985+0000","last_peered":"2024-09-11T07:28:08.012985+0000","last_clean":"2024-09-11T07:28:08.012985+0000","last_became_active":"2024-09-11T07:26:31.422110+0000","last_became_peered":"2024-09-11T07:26:31.422110+0000","last_unstale":"2024-09-11T07:28:08.012985+0000","last_undegraded":"2024-09-11T07:28:08.012985+0000","last_fullsized":"2024-09-11T07:28:08.012985+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:27:41.618296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:41.618296+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:57.404591+0000","scrub_duration":0.0020650450000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":72,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.014037+0000","last_change":"2024-09-11T07:28:03.183056+0000","last_active":"2024-09-11T07:28:09.014037+0000","last_peered":"2024-09-11T07:28:09.014037+0000","last_clean":"2024-09-11T07:28:09.014037+0000","last_became_active":"2024-09-11T07:26:35.560422+0000","last_became_peered":"2024-09-11T07:26:35.560422+0000","last_unstale":"2024-09-11T07:28:09.014037+0000","last_undegraded":"2024-09-11T07:28:09.014037+0000","last_fullsized":"2024-09-11T07:28:09.014037+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:28:03.182993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:03.182993+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:26.896461+0000","scrub_duration":0.0018949889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.013563+0000","last_change":"2024-09-11T07:27:54.148594+0000","last_active":"2024-09-11T07:28:08.013563+0000","last_peered":"2024-09-11T07:28:08.013563+0000","last_clean":"2024-09-11T07:28:08.013563+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:28:08.013563+0000","last_undegraded":"2024-09-11T07:28:08.013563+0000","last_fullsized":"2024-09-11T07:28:08.013563+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.148534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:02.854448+0000","scrub_duration":0.0013536620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"64'28","reported_seq":114,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.770096+0000","last_change":"2024-09-11T07:28:09.770058+0000","last_active":"2024-09-11T07:28:09.770096+0000","last_peered":"2024-09-11T07:28:09.770096+0000","last_clean":"2024-09-11T07:28:09.770096+0000","last_became_active":"2024-09-11T07:26:33.549234+0000","last_became_peered":"2024-09-11T07:26:33.549234+0000","last_unstale":"2024-09-11T07:28:09.770096+0000","last_undegraded":"2024-09-11T07:28:09.770096+0000","last_fullsized":"2024-09-11T07:28:09.770096+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'28","last_scrub_stamp":"2024-09-11T07:28:09.769992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:09.769992+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:37.423717+0000","scrub_duration":0.002388463,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:12.775168+0000","last_change":"2024-09-11T07:28:12.775128+0000","last_active":"2024-09-11T07:28:12.775168+0000","last_peered":"2024-09-11T07:28:12.775168+0000","last_clean":"2024-09-11T07:28:12.775168+0000","last_became_active":"2024-09-11T07:26:31.424618+0000","last_became_peered":"2024-09-11T07:26:31.424618+0000","last_unstale":"2024-09-11T07:28:12.775168+0000","last_undegraded":"2024-09-11T07:28:12.775168+0000","last_fullsized":"2024-09-11T07:28:12.775168+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:12.775067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:12.775067+0000","last_clean_scrub_stamp":"2024-09-11T07:28:12.775067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:21.184139+0000","scrub_duration":0.0015206589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014438+0000","last_change":"2024-09-11T07:26:35.560677+0000","last_active":"2024-09-11T07:28:08.014438+0000","last_peered":"2024-09-11T07:28:08.014438+0000","last_clean":"2024-09-11T07:28:08.014438+0000","last_became_active":"2024-09-11T07:26:35.560363+0000","last_became_peered":"2024-09-11T07:26:35.560363+0000","last_unstale":"2024-09-11T07:28:08.014438+0000","last_undegraded":"2024-09-11T07:28:08.014438+0000","last_fullsized":"2024-09-11T07:28:08.014438+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.009101+0000","last_change":"2024-09-11T07:28:00.686634+0000","last_active":"2024-09-11T07:28:08.009101+0000","last_peered":"2024-09-11T07:28:08.009101+0000","last_clean":"2024-09-11T07:28:08.009101+0000","last_became_active":"2024-09-11T07:26:37.562452+0000","last_became_peered":"2024-09-11T07:26:37.562452+0000","last_unstale":"2024-09-11T07:28:08.009101+0000","last_undegraded":"2024-09-11T07:28:08.009101+0000","last_fullsized":"2024-09-11T07:28:08.009101+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:00.686571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.686571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.798861+0000","scrub_duration":0.0016714309999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"64'13","reported_seq":63,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.013413+0000","last_change":"2024-09-11T07:26:33.547483+0000","last_active":"2024-09-11T07:28:08.013413+0000","last_peered":"2024-09-11T07:28:08.013413+0000","last_clean":"2024-09-11T07:28:08.013413+0000","last_became_active":"2024-09-11T07:26:33.546126+0000","last_became_peered":"2024-09-11T07:26:33.546126+0000","last_unstale":"2024-09-11T07:28:08.013413+0000","last_undegraded":"2024-09-11T07:28:08.013413+0000","last_fullsized":"2024-09-11T07:28:08.013413+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":152,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:12.745992+0000","last_change":"2024-09-11T07:27:59.678930+0000","last_active":"2024-09-11T07:28:12.745992+0000","last_peered":"2024-09-11T07:28:12.745992+0000","last_clean":"2024-09-11T07:28:12.745992+0000","last_became_active":"2024-09-11T07:26:31.421276+0000","last_became_peered":"2024-09-11T07:26:31.421276+0000","last_unstale":"2024-09-11T07:28:12.745992+0000","last_undegraded":"2024-09-11T07:28:12.745992+0000","last_fullsized":"2024-09-11T07:28:12.745992+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'5","last_scrub_stamp":"2024-09-11T07:27:59.678865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:59.678865+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.114929+0000","scrub_duration":0.001792486,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":96,"num_read_kb":91,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.009362+0000","last_change":"2024-09-11T07:28:07.327716+0000","last_active":"2024-09-11T07:28:08.009362+0000","last_peered":"2024-09-11T07:28:08.009362+0000","last_clean":"2024-09-11T07:28:08.009362+0000","last_became_active":"2024-09-11T07:26:31.413354+0000","last_became_peered":"2024-09-11T07:26:31.413354+0000","last_unstale":"2024-09-11T07:28:08.009362+0000","last_undegraded":"2024-09-11T07:28:08.009362+0000","last_fullsized":"2024-09-11T07:28:08.009362+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:07.327648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:07.327648+0000","last_clean_scrub_stamp":"2024-09-11T07:28:07.327648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:35.306829+0000","scrub_duration":0.0011895899999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008446+0000","last_change":"2024-09-11T07:27:42.141156+0000","last_active":"2024-09-11T07:28:08.008446+0000","last_peered":"2024-09-11T07:28:08.008446+0000","last_clean":"2024-09-11T07:28:08.008446+0000","last_became_active":"2024-09-11T07:26:35.565849+0000","last_became_peered":"2024-09-11T07:26:35.565849+0000","last_unstale":"2024-09-11T07:28:08.008446+0000","last_undegraded":"2024-09-11T07:28:08.008446+0000","last_fullsized":"2024-09-11T07:28:08.008446+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.141100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.141100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.707826+0000","scrub_duration":0.001495672,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014975+0000","last_change":"2024-09-11T07:27:55.018180+0000","last_active":"2024-09-11T07:28:08.014975+0000","last_peered":"2024-09-11T07:28:08.014975+0000","last_clean":"2024-09-11T07:28:08.014975+0000","last_became_active":"2024-09-11T07:26:37.861292+0000","last_became_peered":"2024-09-11T07:26:37.861292+0000","last_unstale":"2024-09-11T07:28:08.014975+0000","last_undegraded":"2024-09-11T07:28:08.014975+0000","last_fullsized":"2024-09-11T07:28:08.014975+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:55.018125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:55.018125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.234213+0000","scrub_duration":0.0013534339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":88,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.025399+0000","last_change":"2024-09-11T07:28:09.025399+0000","last_active":"2024-09-11T07:28:09.025399+0000","last_peered":"2024-09-11T07:28:09.025399+0000","last_clean":"2024-09-11T07:28:09.025399+0000","last_became_active":"2024-09-11T07:28:09.018134+0000","last_became_peered":"2024-09-11T07:28:09.018134+0000","last_unstale":"2024-09-11T07:28:09.025399+0000","last_undegraded":"2024-09-11T07:28:09.025399+0000","last_fullsized":"2024-09-11T07:28:09.025399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-09-11T07:27:34.213870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:34.213870+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:46.533561+0000","scrub_duration":0.0026667679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":54,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.286887+0000","last_change":"2024-09-11T07:28:08.286847+0000","last_active":"2024-09-11T07:28:08.286887+0000","last_peered":"2024-09-11T07:28:08.286887+0000","last_clean":"2024-09-11T07:28:08.286887+0000","last_became_active":"2024-09-11T07:26:31.420320+0000","last_became_peered":"2024-09-11T07:26:31.420320+0000","last_unstale":"2024-09-11T07:28:08.286887+0000","last_undegraded":"2024-09-11T07:28:08.286887+0000","last_fullsized":"2024-09-11T07:28:08.286887+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-11T07:28:08.286788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:08.286788+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.046248+0000","scrub_duration":0.001825976,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014527+0000","last_change":"2024-09-11T07:26:31.418190+0000","last_active":"2024-09-11T07:28:08.014527+0000","last_peered":"2024-09-11T07:28:08.014527+0000","last_clean":"2024-09-11T07:28:08.014527+0000","last_became_active":"2024-09-11T07:26:31.417455+0000","last_became_peered":"2024-09-11T07:26:31.417455+0000","last_unstale":"2024-09-11T07:28:08.014527+0000","last_undegraded":"2024-09-11T07:28:08.014527+0000","last_fullsized":"2024-09-11T07:28:08.014527+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012666+0000","last_change":"2024-09-11T07:26:35.564511+0000","last_active":"2024-09-11T07:28:08.012666+0000","last_peered":"2024-09-11T07:28:08.012666+0000","last_clean":"2024-09-11T07:28:08.012666+0000","last_became_active":"2024-09-11T07:26:35.564178+0000","last_became_peered":"2024-09-11T07:26:35.564178+0000","last_unstale":"2024-09-11T07:28:08.012666+0000","last_undegraded":"2024-09-11T07:28:08.012666+0000","last_fullsized":"2024-09-11T07:28:08.012666+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012691+0000","last_change":"2024-09-11T07:27:54.621934+0000","last_active":"2024-09-11T07:28:08.012691+0000","last_peered":"2024-09-11T07:28:08.012691+0000","last_clean":"2024-09-11T07:28:08.012691+0000","last_became_active":"2024-09-11T07:26:37.561378+0000","last_became_peered":"2024-09-11T07:26:37.561378+0000","last_unstale":"2024-09-11T07:28:08.012691+0000","last_undegraded":"2024-09-11T07:28:08.012691+0000","last_fullsized":"2024-09-11T07:28:08.012691+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.621874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.621874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:07.014230+0000","scrub_duration":0.0013604750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011749+0000","last_change":"2024-09-11T07:27:49.061728+0000","last_active":"2024-09-11T07:28:08.011749+0000","last_peered":"2024-09-11T07:28:08.011749+0000","last_clean":"2024-09-11T07:28:08.011749+0000","last_became_active":"2024-09-11T07:26:33.541359+0000","last_became_peered":"2024-09-11T07:26:33.541359+0000","last_unstale":"2024-09-11T07:28:08.011749+0000","last_undegraded":"2024-09-11T07:28:08.011749+0000","last_fullsized":"2024-09-11T07:28:08.011749+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-09-11T07:27:49.061670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.061670+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:51.283390+0000","scrub_duration":0.001809801,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'3","reported_seq":56,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.009267+0000","last_change":"2024-09-11T07:28:01.711301+0000","last_active":"2024-09-11T07:28:08.009267+0000","last_peered":"2024-09-11T07:28:08.009267+0000","last_clean":"2024-09-11T07:28:08.009267+0000","last_became_active":"2024-09-11T07:26:31.419501+0000","last_became_peered":"2024-09-11T07:26:31.419501+0000","last_unstale":"2024-09-11T07:28:08.009267+0000","last_undegraded":"2024-09-11T07:28:08.009267+0000","last_fullsized":"2024-09-11T07:28:08.009267+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.711241+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:02.735271+0000","scrub_duration":0.0034627350000000002,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011920+0000","last_change":"2024-09-11T07:27:47.903204+0000","last_active":"2024-09-11T07:28:08.011920+0000","last_peered":"2024-09-11T07:28:08.011920+0000","last_clean":"2024-09-11T07:28:08.011920+0000","last_became_active":"2024-09-11T07:26:31.423304+0000","last_became_peered":"2024-09-11T07:26:31.423304+0000","last_unstale":"2024-09-11T07:28:08.011920+0000","last_undegraded":"2024-09-11T07:28:08.011920+0000","last_fullsized":"2024-09-11T07:28:08.011920+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:47.903135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.903135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:01.867608+0000","scrub_duration":0.001284908,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"72'79","reported_seq":83,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014624+0000","last_change":"2024-09-11T07:26:57.872633+0000","last_active":"2024-09-11T07:28:08.014624+0000","last_peered":"2024-09-11T07:28:08.014624+0000","last_clean":"2024-09-11T07:28:08.014624+0000","last_became_active":"2024-09-11T07:26:17.397381+0000","last_became_peered":"2024-09-11T07:26:17.397381+0000","last_unstale":"2024-09-11T07:28:08.014624+0000","last_undegraded":"2024-09-11T07:28:08.014624+0000","last_fullsized":"2024-09-11T07:28:08.014624+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'72","last_scrub_stamp":"2024-09-11T07:26:57.872570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:24:22.202219+0000","last_clean_scrub_stamp":"2024-09-11T07:26:57.872570+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015639180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014924+0000","last_change":"2024-09-11T07:27:44.036138+0000","last_active":"2024-09-11T07:28:08.014924+0000","last_peered":"2024-09-11T07:28:08.014924+0000","last_clean":"2024-09-11T07:28:08.014924+0000","last_became_active":"2024-09-11T07:26:35.564480+0000","last_became_peered":"2024-09-11T07:26:35.564480+0000","last_unstale":"2024-09-11T07:28:08.014924+0000","last_undegraded":"2024-09-11T07:28:08.014924+0000","last_fullsized":"2024-09-11T07:28:08.014924+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:44.036058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:44.036058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:48.598709+0000","scrub_duration":0.001488931,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:13.796782+0000","last_change":"2024-09-11T07:28:13.796731+0000","last_active":"2024-09-11T07:28:13.796782+0000","last_peered":"2024-09-11T07:28:13.796782+0000","last_clean":"2024-09-11T07:28:13.796782+0000","last_became_active":"2024-09-11T07:26:37.559920+0000","last_became_peered":"2024-09-11T07:26:37.559920+0000","last_unstale":"2024-09-11T07:28:13.796782+0000","last_undegraded":"2024-09-11T07:28:13.796782+0000","last_fullsized":"2024-09-11T07:28:13.796782+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:13.796657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:13.796657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:31.783767+0000","scrub_duration":0.001458984,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":77,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011287+0000","last_change":"2024-09-11T07:27:59.128292+0000","last_active":"2024-09-11T07:28:08.011287+0000","last_peered":"2024-09-11T07:28:08.011287+0000","last_clean":"2024-09-11T07:28:08.011287+0000","last_became_active":"2024-09-11T07:26:33.543687+0000","last_became_peered":"2024-09-11T07:26:33.543687+0000","last_unstale":"2024-09-11T07:28:08.011287+0000","last_undegraded":"2024-09-11T07:28:08.011287+0000","last_fullsized":"2024-09-11T07:28:08.011287+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-11T07:27:59.128237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:59.128237+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:16.835564+0000","scrub_duration":0.00199329,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011343+0000","last_change":"2024-09-11T07:27:54.182157+0000","last_active":"2024-09-11T07:28:08.011343+0000","last_peered":"2024-09-11T07:28:08.011343+0000","last_clean":"2024-09-11T07:28:08.011343+0000","last_became_active":"2024-09-11T07:26:31.419560+0000","last_became_peered":"2024-09-11T07:26:31.419560+0000","last_unstale":"2024-09-11T07:28:08.011343+0000","last_undegraded":"2024-09-11T07:28:08.011343+0000","last_fullsized":"2024-09-11T07:28:08.011343+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.182106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.182106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:04.075506+0000","scrub_duration":0.001213254,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:13.357708+0000","last_change":"2024-09-11T07:28:13.357659+0000","last_active":"2024-09-11T07:28:13.357708+0000","last_peered":"2024-09-11T07:28:13.357708+0000","last_clean":"2024-09-11T07:28:13.357708+0000","last_became_active":"2024-09-11T07:26:35.564044+0000","last_became_peered":"2024-09-11T07:26:35.564044+0000","last_unstale":"2024-09-11T07:28:13.357708+0000","last_undegraded":"2024-09-11T07:28:13.357708+0000","last_fullsized":"2024-09-11T07:28:13.357708+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:13.357544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:13.357544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:42.978875+0000","scrub_duration":0.0017127900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:13.918182+0000","last_change":"2024-09-11T07:28:13.918148+0000","last_active":"2024-09-11T07:28:13.918182+0000","last_peered":"2024-09-11T07:28:13.918182+0000","last_clean":"2024-09-11T07:28:13.918182+0000","last_became_active":"2024-09-11T07:26:37.564801+0000","last_became_peered":"2024-09-11T07:26:37.564801+0000","last_unstale":"2024-09-11T07:28:13.918182+0000","last_undegraded":"2024-09-11T07:28:13.918182+0000","last_fullsized":"2024-09-11T07:28:13.918182+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:13.918082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:13.918082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:31.000135+0000","scrub_duration":0.0014064769999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":62,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014507+0000","last_change":"2024-09-11T07:28:02.230985+0000","last_active":"2024-09-11T07:28:08.014507+0000","last_peered":"2024-09-11T07:28:08.014507+0000","last_clean":"2024-09-11T07:28:08.014507+0000","last_became_active":"2024-09-11T07:26:33.543148+0000","last_became_peered":"2024-09-11T07:26:33.543148+0000","last_unstale":"2024-09-11T07:28:08.014507+0000","last_undegraded":"2024-09-11T07:28:08.014507+0000","last_fullsized":"2024-09-11T07:28:08.014507+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:28:02.230928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.230928+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:27.244330+0000","scrub_duration":0.001888926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.013300+0000","last_change":"2024-09-11T07:27:53.659961+0000","last_active":"2024-09-11T07:28:08.013300+0000","last_peered":"2024-09-11T07:28:08.013300+0000","last_clean":"2024-09-11T07:28:08.013300+0000","last_became_active":"2024-09-11T07:26:31.423056+0000","last_became_peered":"2024-09-11T07:26:31.423056+0000","last_unstale":"2024-09-11T07:28:08.013300+0000","last_undegraded":"2024-09-11T07:28:08.013300+0000","last_fullsized":"2024-09-11T07:28:08.013300+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:53.659902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:53.659902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.781453+0000","scrub_duration":0.0011613719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:10.337633+0000","last_change":"2024-09-11T07:28:10.337599+0000","last_active":"2024-09-11T07:28:10.337633+0000","last_peered":"2024-09-11T07:28:10.337633+0000","last_clean":"2024-09-11T07:28:10.337633+0000","last_became_active":"2024-09-11T07:26:35.563182+0000","last_became_peered":"2024-09-11T07:26:35.563182+0000","last_unstale":"2024-09-11T07:28:10.337633+0000","last_undegraded":"2024-09-11T07:28:10.337633+0000","last_fullsized":"2024-09-11T07:28:10.337633+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:10.337527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:10.337527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:24.013574+0000","scrub_duration":0.001985653,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014034+0000","last_change":"2024-09-11T07:26:37.565396+0000","last_active":"2024-09-11T07:28:08.014034+0000","last_peered":"2024-09-11T07:28:08.014034+0000","last_clean":"2024-09-11T07:28:08.014034+0000","last_became_active":"2024-09-11T07:26:37.565029+0000","last_became_peered":"2024-09-11T07:26:37.565029+0000","last_unstale":"2024-09-11T07:28:08.014034+0000","last_undegraded":"2024-09-11T07:28:08.014034+0000","last_fullsized":"2024-09-11T07:28:08.014034+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":71,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011549+0000","last_change":"2024-09-11T07:28:07.811136+0000","last_active":"2024-09-11T07:28:08.011549+0000","last_peered":"2024-09-11T07:28:08.011549+0000","last_clean":"2024-09-11T07:28:08.011549+0000","last_became_active":"2024-09-11T07:26:33.538216+0000","last_became_peered":"2024-09-11T07:26:33.538216+0000","last_unstale":"2024-09-11T07:28:08.011549+0000","last_undegraded":"2024-09-11T07:28:08.011549+0000","last_fullsized":"2024-09-11T07:28:08.011549+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-09-11T07:28:07.811071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:07.811071+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:23.495093+0000","scrub_duration":0.0019627080000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.009267+0000","last_change":"2024-09-11T07:27:50.132629+0000","last_active":"2024-09-11T07:28:08.009267+0000","last_peered":"2024-09-11T07:28:08.009267+0000","last_clean":"2024-09-11T07:28:08.009267+0000","last_became_active":"2024-09-11T07:26:31.418490+0000","last_became_peered":"2024-09-11T07:26:31.418490+0000","last_unstale":"2024-09-11T07:28:08.009267+0000","last_undegraded":"2024-09-11T07:28:08.009267+0000","last_fullsized":"2024-09-11T07:28:08.009267+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.132552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.048681+0000","scrub_duration":0.001622807,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":62,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.332047+0000","last_change":"2024-09-11T07:26:35.567147+0000","last_active":"2024-09-11T07:28:09.332047+0000","last_peered":"2024-09-11T07:28:09.332047+0000","last_clean":"2024-09-11T07:28:09.332047+0000","last_became_active":"2024-09-11T07:26:35.566888+0000","last_became_peered":"2024-09-11T07:26:35.566888+0000","last_unstale":"2024-09-11T07:28:09.332047+0000","last_undegraded":"2024-09-11T07:28:09.332047+0000","last_fullsized":"2024-09-11T07:28:09.332047+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011429+0000","last_change":"2024-09-11T07:28:03.961280+0000","last_active":"2024-09-11T07:28:08.011429+0000","last_peered":"2024-09-11T07:28:08.011429+0000","last_clean":"2024-09-11T07:28:08.011429+0000","last_became_active":"2024-09-11T07:26:37.861134+0000","last_became_peered":"2024-09-11T07:26:37.861134+0000","last_unstale":"2024-09-11T07:28:08.011429+0000","last_undegraded":"2024-09-11T07:28:08.011429+0000","last_fullsized":"2024-09-11T07:28:08.011429+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:03.961220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:03.961220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:29.223995+0000","scrub_duration":0.0011943730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"83'17","reported_seq":74,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008268+0000","last_change":"2024-09-11T07:28:01.098068+0000","last_active":"2024-09-11T07:28:08.008268+0000","last_peered":"2024-09-11T07:28:08.008268+0000","last_clean":"2024-09-11T07:28:08.008268+0000","last_became_active":"2024-09-11T07:26:33.540363+0000","last_became_peered":"2024-09-11T07:26:33.540363+0000","last_unstale":"2024-09-11T07:28:08.008268+0000","last_undegraded":"2024-09-11T07:28:08.008268+0000","last_fullsized":"2024-09-11T07:28:08.008268+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'17","last_scrub_stamp":"2024-09-11T07:28:01.097981+0000","last_deep_scrub":"83'17","last_deep_scrub_stamp":"2024-09-11T07:28:01.097981+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.097981+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:26.179592+0000","scrub_duration":0.0042048720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.009183+0000","last_change":"2024-09-11T07:27:46.077857+0000","last_active":"2024-09-11T07:28:08.009183+0000","last_peered":"2024-09-11T07:28:08.009183+0000","last_clean":"2024-09-11T07:28:08.009183+0000","last_became_active":"2024-09-11T07:26:31.419718+0000","last_became_peered":"2024-09-11T07:26:31.419718+0000","last_unstale":"2024-09-11T07:28:08.009183+0000","last_undegraded":"2024-09-11T07:28:08.009183+0000","last_fullsized":"2024-09-11T07:28:08.009183+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:27:46.077786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:46.077786+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:07.735381+0000","scrub_duration":0.0019376720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008226+0000","last_change":"2024-09-11T07:27:53.209518+0000","last_active":"2024-09-11T07:28:08.008226+0000","last_peered":"2024-09-11T07:28:08.008226+0000","last_clean":"2024-09-11T07:28:08.008226+0000","last_became_active":"2024-09-11T07:26:35.561967+0000","last_became_peered":"2024-09-11T07:26:35.561967+0000","last_unstale":"2024-09-11T07:28:08.008226+0000","last_undegraded":"2024-09-11T07:28:08.008226+0000","last_fullsized":"2024-09-11T07:28:08.008226+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:53.209452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:53.209452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:23.125281+0000","scrub_duration":0.0014563029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008343+0000","last_change":"2024-09-11T07:28:02.677186+0000","last_active":"2024-09-11T07:28:08.008343+0000","last_peered":"2024-09-11T07:28:08.008343+0000","last_clean":"2024-09-11T07:28:08.008343+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:28:08.008343+0000","last_undegraded":"2024-09-11T07:28:08.008343+0000","last_fullsized":"2024-09-11T07:28:08.008343+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:02.677095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.677095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.575070+0000","scrub_duration":0.001519269,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":62,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014937+0000","last_change":"2024-09-11T07:27:40.351572+0000","last_active":"2024-09-11T07:28:08.014937+0000","last_peered":"2024-09-11T07:28:08.014937+0000","last_clean":"2024-09-11T07:28:08.014937+0000","last_became_active":"2024-09-11T07:26:33.547344+0000","last_became_peered":"2024-09-11T07:26:33.547344+0000","last_unstale":"2024-09-11T07:28:08.014937+0000","last_undegraded":"2024-09-11T07:28:08.014937+0000","last_fullsized":"2024-09-11T07:28:08.014937+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:27:40.351507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:40.351507+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.098394+0000","scrub_duration":0.0021803980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008361+0000","last_change":"2024-09-11T07:27:58.324311+0000","last_active":"2024-09-11T07:28:08.008361+0000","last_peered":"2024-09-11T07:28:08.008361+0000","last_clean":"2024-09-11T07:28:08.008361+0000","last_became_active":"2024-09-11T07:26:31.415701+0000","last_became_peered":"2024-09-11T07:26:31.415701+0000","last_unstale":"2024-09-11T07:28:08.008361+0000","last_undegraded":"2024-09-11T07:28:08.008361+0000","last_fullsized":"2024-09-11T07:28:08.008361+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:58.324255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.324255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.617500+0000","scrub_duration":0.0013365549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:12.778737+0000","last_change":"2024-09-11T07:28:12.778700+0000","last_active":"2024-09-11T07:28:12.778737+0000","last_peered":"2024-09-11T07:28:12.778737+0000","last_clean":"2024-09-11T07:28:12.778737+0000","last_became_active":"2024-09-11T07:26:35.559262+0000","last_became_peered":"2024-09-11T07:26:35.559262+0000","last_unstale":"2024-09-11T07:28:12.778737+0000","last_undegraded":"2024-09-11T07:28:12.778737+0000","last_fullsized":"2024-09-11T07:28:12.778737+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:12.778645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:12.778645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.979009+0000","scrub_duration":0.0013084870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.009416+0000","last_change":"2024-09-11T07:28:06.305563+0000","last_active":"2024-09-11T07:28:08.009416+0000","last_peered":"2024-09-11T07:28:08.009416+0000","last_clean":"2024-09-11T07:28:08.009416+0000","last_became_active":"2024-09-11T07:26:37.564084+0000","last_became_peered":"2024-09-11T07:26:37.564084+0000","last_unstale":"2024-09-11T07:28:08.009416+0000","last_undegraded":"2024-09-11T07:28:08.009416+0000","last_fullsized":"2024-09-11T07:28:08.009416+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:28:06.305502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:06.305502+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.659083+0000","scrub_duration":0.0021934649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011410+0000","last_change":"2024-09-11T07:27:45.990617+0000","last_active":"2024-09-11T07:28:08.011410+0000","last_peered":"2024-09-11T07:28:08.011410+0000","last_clean":"2024-09-11T07:28:08.011410+0000","last_became_active":"2024-09-11T07:26:31.423229+0000","last_became_peered":"2024-09-11T07:26:31.423229+0000","last_unstale":"2024-09-11T07:28:08.011410+0000","last_undegraded":"2024-09-11T07:28:08.011410+0000","last_fullsized":"2024-09-11T07:28:08.011410+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:45.990548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:45.990548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:51.641867+0000","scrub_duration":0.0014585150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"82'9","reported_seq":58,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.009614+0000","last_change":"2024-09-11T07:27:48.761470+0000","last_active":"2024-09-11T07:28:08.009614+0000","last_peered":"2024-09-11T07:28:08.009614+0000","last_clean":"2024-09-11T07:28:08.009614+0000","last_became_active":"2024-09-11T07:26:33.543628+0000","last_became_peered":"2024-09-11T07:26:33.543628+0000","last_unstale":"2024-09-11T07:28:08.009614+0000","last_undegraded":"2024-09-11T07:28:08.009614+0000","last_fullsized":"2024-09-11T07:28:08.009614+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'9","last_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:27:48.761397+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.913254+0000","scrub_duration":0.0037047439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012854+0000","last_change":"2024-09-11T07:27:42.917739+0000","last_active":"2024-09-11T07:28:08.012854+0000","last_peered":"2024-09-11T07:28:08.012854+0000","last_clean":"2024-09-11T07:28:08.012854+0000","last_became_active":"2024-09-11T07:26:35.566894+0000","last_became_peered":"2024-09-11T07:26:35.566894+0000","last_unstale":"2024-09-11T07:28:08.012854+0000","last_undegraded":"2024-09-11T07:28:08.012854+0000","last_fullsized":"2024-09-11T07:28:08.012854+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.917680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:06.763663+0000","scrub_duration":0.0013496649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014956+0000","last_change":"2024-09-11T07:27:47.031883+0000","last_active":"2024-09-11T07:28:08.014956+0000","last_peered":"2024-09-11T07:28:08.014956+0000","last_clean":"2024-09-11T07:28:08.014956+0000","last_became_active":"2024-09-11T07:26:37.863100+0000","last_became_peered":"2024-09-11T07:26:37.863100+0000","last_unstale":"2024-09-11T07:28:08.014956+0000","last_undegraded":"2024-09-11T07:28:08.014956+0000","last_fullsized":"2024-09-11T07:28:08.014956+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:47.031807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.031807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:54.469643+0000","scrub_duration":0.001556195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"64'10","reported_seq":51,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.010087+0000","last_change":"2024-09-11T07:26:33.545346+0000","last_active":"2024-09-11T07:28:08.010087+0000","last_peered":"2024-09-11T07:28:08.010087+0000","last_clean":"2024-09-11T07:28:08.010087+0000","last_became_active":"2024-09-11T07:26:33.545078+0000","last_became_peered":"2024-09-11T07:26:33.545078+0000","last_unstale":"2024-09-11T07:28:08.010087+0000","last_undegraded":"2024-09-11T07:28:08.010087+0000","last_fullsized":"2024-09-11T07:28:08.010087+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011726+0000","last_change":"2024-09-11T07:26:31.424634+0000","last_active":"2024-09-11T07:28:08.011726+0000","last_peered":"2024-09-11T07:28:08.011726+0000","last_clean":"2024-09-11T07:28:08.011726+0000","last_became_active":"2024-09-11T07:26:31.423789+0000","last_became_peered":"2024-09-11T07:26:31.423789+0000","last_unstale":"2024-09-11T07:28:08.011726+0000","last_undegraded":"2024-09-11T07:28:08.011726+0000","last_fullsized":"2024-09-11T07:28:08.011726+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":62,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.016882+0000","last_change":"2024-09-11T07:26:35.562509+0000","last_active":"2024-09-11T07:28:09.016882+0000","last_peered":"2024-09-11T07:28:09.016882+0000","last_clean":"2024-09-11T07:28:09.016882+0000","last_became_active":"2024-09-11T07:26:35.562245+0000","last_became_peered":"2024-09-11T07:26:35.562245+0000","last_unstale":"2024-09-11T07:28:09.016882+0000","last_undegraded":"2024-09-11T07:28:09.016882+0000","last_fullsized":"2024-09-11T07:28:09.016882+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014618+0000","last_change":"2024-09-11T07:27:49.245119+0000","last_active":"2024-09-11T07:28:08.014618+0000","last_peered":"2024-09-11T07:28:08.014618+0000","last_clean":"2024-09-11T07:28:08.014618+0000","last_became_active":"2024-09-11T07:26:37.566749+0000","last_became_peered":"2024-09-11T07:26:37.566749+0000","last_unstale":"2024-09-11T07:28:08.014618+0000","last_undegraded":"2024-09-11T07:28:08.014618+0000","last_fullsized":"2024-09-11T07:28:08.014618+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:49.245047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.245047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:49.614854+0000","scrub_duration":0.001867966,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"83'13","reported_seq":66,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012745+0000","last_change":"2024-09-11T07:28:00.959669+0000","last_active":"2024-09-11T07:28:08.012745+0000","last_peered":"2024-09-11T07:28:08.012745+0000","last_clean":"2024-09-11T07:28:08.012745+0000","last_became_active":"2024-09-11T07:26:33.537425+0000","last_became_peered":"2024-09-11T07:26:33.537425+0000","last_unstale":"2024-09-11T07:28:08.012745+0000","last_undegraded":"2024-09-11T07:28:08.012745+0000","last_fullsized":"2024-09-11T07:28:08.012745+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-11T07:28:00.959583+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-11T07:28:00.959583+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.959583+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:06.630980+0000","scrub_duration":0.0032987350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014648+0000","last_change":"2024-09-11T07:28:01.237329+0000","last_active":"2024-09-11T07:28:08.014648+0000","last_peered":"2024-09-11T07:28:08.014648+0000","last_clean":"2024-09-11T07:28:08.014648+0000","last_became_active":"2024-09-11T07:26:31.419409+0000","last_became_peered":"2024-09-11T07:26:31.419409+0000","last_unstale":"2024-09-11T07:28:08.014648+0000","last_undegraded":"2024-09-11T07:28:08.014648+0000","last_fullsized":"2024-09-11T07:28:08.014648+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:01.237267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.237267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.266034+0000","scrub_duration":0.0012797399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":62,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.012233+0000","last_change":"2024-09-11T07:26:35.560917+0000","last_active":"2024-09-11T07:28:09.012233+0000","last_peered":"2024-09-11T07:28:09.012233+0000","last_clean":"2024-09-11T07:28:09.012233+0000","last_became_active":"2024-09-11T07:26:35.560610+0000","last_became_peered":"2024-09-11T07:26:35.560610+0000","last_unstale":"2024-09-11T07:28:09.012233+0000","last_undegraded":"2024-09-11T07:28:09.012233+0000","last_fullsized":"2024-09-11T07:28:09.012233+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008988+0000","last_change":"2024-09-11T07:26:37.566136+0000","last_active":"2024-09-11T07:28:08.008988+0000","last_peered":"2024-09-11T07:28:08.008988+0000","last_clean":"2024-09-11T07:28:08.008988+0000","last_became_active":"2024-09-11T07:26:37.565723+0000","last_became_peered":"2024-09-11T07:26:37.565723+0000","last_unstale":"2024-09-11T07:28:08.008988+0000","last_undegraded":"2024-09-11T07:28:08.008988+0000","last_fullsized":"2024-09-11T07:28:08.008988+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":62,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012553+0000","last_change":"2024-09-11T07:27:57.601328+0000","last_active":"2024-09-11T07:28:08.012553+0000","last_peered":"2024-09-11T07:28:08.012553+0000","last_clean":"2024-09-11T07:28:08.012553+0000","last_became_active":"2024-09-11T07:26:33.540688+0000","last_became_peered":"2024-09-11T07:26:33.540688+0000","last_unstale":"2024-09-11T07:28:08.012553+0000","last_undegraded":"2024-09-11T07:28:08.012553+0000","last_fullsized":"2024-09-11T07:28:08.012553+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:27:57.601261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.601261+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:58.826830+0000","scrub_duration":0.002247724,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011550+0000","last_change":"2024-09-11T07:27:54.102580+0000","last_active":"2024-09-11T07:28:08.011550+0000","last_peered":"2024-09-11T07:28:08.011550+0000","last_clean":"2024-09-11T07:28:08.011550+0000","last_became_active":"2024-09-11T07:26:31.425587+0000","last_became_peered":"2024-09-11T07:26:31.425587+0000","last_unstale":"2024-09-11T07:28:08.011550+0000","last_undegraded":"2024-09-11T07:28:08.011550+0000","last_fullsized":"2024-09-11T07:28:08.011550+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.102528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.102528+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.102528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:06.835183+0000","scrub_duration":0.0012736080000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008412+0000","last_change":"2024-09-11T07:26:35.562603+0000","last_active":"2024-09-11T07:28:08.008412+0000","last_peered":"2024-09-11T07:28:08.008412+0000","last_clean":"2024-09-11T07:28:08.008412+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:28:08.008412+0000","last_undegraded":"2024-09-11T07:28:08.008412+0000","last_fullsized":"2024-09-11T07:28:08.008412+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.013558+0000","last_change":"2024-09-11T07:27:58.192585+0000","last_active":"2024-09-11T07:28:08.013558+0000","last_peered":"2024-09-11T07:28:08.013558+0000","last_clean":"2024-09-11T07:28:08.013558+0000","last_became_active":"2024-09-11T07:26:37.568886+0000","last_became_peered":"2024-09-11T07:26:37.568886+0000","last_unstale":"2024-09-11T07:28:08.013558+0000","last_undegraded":"2024-09-11T07:28:08.013558+0000","last_fullsized":"2024-09-11T07:28:08.013558+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:58.192528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.192528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:18.757569+0000","scrub_duration":0.001884698,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008607+0000","last_change":"2024-09-11T07:27:48.081460+0000","last_active":"2024-09-11T07:28:08.008607+0000","last_peered":"2024-09-11T07:28:08.008607+0000","last_clean":"2024-09-11T07:28:08.008607+0000","last_became_active":"2024-09-11T07:26:33.540208+0000","last_became_peered":"2024-09-11T07:26:33.540208+0000","last_unstale":"2024-09-11T07:28:08.008607+0000","last_undegraded":"2024-09-11T07:28:08.008607+0000","last_fullsized":"2024-09-11T07:28:08.008607+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:27:48.081392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:48.081392+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:04.524601+0000","scrub_duration":0.0022394590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.013373+0000","last_change":"2024-09-11T07:27:49.677063+0000","last_active":"2024-09-11T07:28:08.013373+0000","last_peered":"2024-09-11T07:28:08.013373+0000","last_clean":"2024-09-11T07:28:08.013373+0000","last_became_active":"2024-09-11T07:26:31.422671+0000","last_became_peered":"2024-09-11T07:26:31.422671+0000","last_unstale":"2024-09-11T07:28:08.013373+0000","last_undegraded":"2024-09-11T07:28:08.013373+0000","last_fullsized":"2024-09-11T07:28:08.013373+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.677003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:54.836177+0000","scrub_duration":0.0013962969999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012435+0000","last_change":"2024-09-11T07:26:35.567257+0000","last_active":"2024-09-11T07:28:08.012435+0000","last_peered":"2024-09-11T07:28:08.012435+0000","last_clean":"2024-09-11T07:28:08.012435+0000","last_became_active":"2024-09-11T07:26:35.566931+0000","last_became_peered":"2024-09-11T07:26:35.566931+0000","last_unstale":"2024-09-11T07:28:08.012435+0000","last_undegraded":"2024-09-11T07:28:08.012435+0000","last_fullsized":"2024-09-11T07:28:08.012435+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008472+0000","last_change":"2024-09-11T07:26:37.561886+0000","last_active":"2024-09-11T07:28:08.008472+0000","last_peered":"2024-09-11T07:28:08.008472+0000","last_clean":"2024-09-11T07:28:08.008472+0000","last_became_active":"2024-09-11T07:26:37.561573+0000","last_became_peered":"2024-09-11T07:26:37.561573+0000","last_unstale":"2024-09-11T07:28:08.008472+0000","last_undegraded":"2024-09-11T07:28:08.008472+0000","last_fullsized":"2024-09-11T07:28:08.008472+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"82'6","reported_seq":54,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008222+0000","last_change":"2024-09-11T07:27:47.789519+0000","last_active":"2024-09-11T07:28:08.008222+0000","last_peered":"2024-09-11T07:28:08.008222+0000","last_clean":"2024-09-11T07:28:08.008222+0000","last_became_active":"2024-09-11T07:26:33.545830+0000","last_became_peered":"2024-09-11T07:26:33.545830+0000","last_unstale":"2024-09-11T07:28:08.008222+0000","last_undegraded":"2024-09-11T07:28:08.008222+0000","last_fullsized":"2024-09-11T07:28:08.008222+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'6","last_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_deep_scrub":"82'6","last_deep_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.789465+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:59.162056+0000","scrub_duration":0.002624679,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012450+0000","last_change":"2024-09-11T07:27:58.043076+0000","last_active":"2024-09-11T07:28:08.012450+0000","last_peered":"2024-09-11T07:28:08.012450+0000","last_clean":"2024-09-11T07:28:08.012450+0000","last_became_active":"2024-09-11T07:26:31.420518+0000","last_became_peered":"2024-09-11T07:26:31.420518+0000","last_unstale":"2024-09-11T07:28:08.012450+0000","last_undegraded":"2024-09-11T07:28:08.012450+0000","last_fullsized":"2024-09-11T07:28:08.012450+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.043009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:24.326885+0000","scrub_duration":0.0015014900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011108+0000","last_change":"2024-09-11T07:27:56.124601+0000","last_active":"2024-09-11T07:28:08.011108+0000","last_peered":"2024-09-11T07:28:08.011108+0000","last_clean":"2024-09-11T07:28:08.011108+0000","last_became_active":"2024-09-11T07:26:35.560506+0000","last_became_peered":"2024-09-11T07:26:35.560506+0000","last_unstale":"2024-09-11T07:28:08.011108+0000","last_undegraded":"2024-09-11T07:28:08.011108+0000","last_fullsized":"2024-09-11T07:28:08.011108+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:56.124542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:56.124542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.847074+0000","scrub_duration":0.0011688619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014648+0000","last_change":"2024-09-11T07:27:42.059066+0000","last_active":"2024-09-11T07:28:08.014648+0000","last_peered":"2024-09-11T07:28:08.014648+0000","last_clean":"2024-09-11T07:28:08.014648+0000","last_became_active":"2024-09-11T07:26:37.564208+0000","last_became_peered":"2024-09-11T07:26:37.564208+0000","last_unstale":"2024-09-11T07:28:08.014648+0000","last_undegraded":"2024-09-11T07:28:08.014648+0000","last_fullsized":"2024-09-11T07:28:08.014648+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.059001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.059001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:51.267826+0000","scrub_duration":0.001418094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":55,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008651+0000","last_change":"2024-09-11T07:26:33.542913+0000","last_active":"2024-09-11T07:28:08.008651+0000","last_peered":"2024-09-11T07:28:08.008651+0000","last_clean":"2024-09-11T07:28:08.008651+0000","last_became_active":"2024-09-11T07:26:33.542625+0000","last_became_peered":"2024-09-11T07:26:33.542625+0000","last_unstale":"2024-09-11T07:28:08.008651+0000","last_undegraded":"2024-09-11T07:28:08.008651+0000","last_fullsized":"2024-09-11T07:28:08.008651+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012369+0000","last_change":"2024-09-11T07:28:02.157520+0000","last_active":"2024-09-11T07:28:08.012369+0000","last_peered":"2024-09-11T07:28:08.012369+0000","last_clean":"2024-09-11T07:28:08.012369+0000","last_became_active":"2024-09-11T07:26:31.425876+0000","last_became_peered":"2024-09-11T07:26:31.425876+0000","last_unstale":"2024-09-11T07:28:08.012369+0000","last_undegraded":"2024-09-11T07:28:08.012369+0000","last_fullsized":"2024-09-11T07:28:08.012369+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:02.157455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.157455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:12.096002+0000","scrub_duration":0.0016990040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014258+0000","last_change":"2024-09-11T07:26:35.565591+0000","last_active":"2024-09-11T07:28:08.014258+0000","last_peered":"2024-09-11T07:28:08.014258+0000","last_clean":"2024-09-11T07:28:08.014258+0000","last_became_active":"2024-09-11T07:26:35.565339+0000","last_became_peered":"2024-09-11T07:26:35.565339+0000","last_unstale":"2024-09-11T07:28:08.014258+0000","last_undegraded":"2024-09-11T07:28:08.014258+0000","last_fullsized":"2024-09-11T07:28:08.014258+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008264+0000","last_change":"2024-09-11T07:26:37.862599+0000","last_active":"2024-09-11T07:28:08.008264+0000","last_peered":"2024-09-11T07:28:08.008264+0000","last_clean":"2024-09-11T07:28:08.008264+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:28:08.008264+0000","last_undegraded":"2024-09-11T07:28:08.008264+0000","last_fullsized":"2024-09-11T07:28:08.008264+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014794+0000","last_change":"2024-09-11T07:26:37.564199+0000","last_active":"2024-09-11T07:28:08.014794+0000","last_peered":"2024-09-11T07:28:08.014794+0000","last_clean":"2024-09-11T07:28:08.014794+0000","last_became_active":"2024-09-11T07:26:37.563942+0000","last_became_peered":"2024-09-11T07:26:37.563942+0000","last_unstale":"2024-09-11T07:28:08.014794+0000","last_undegraded":"2024-09-11T07:28:08.014794+0000","last_fullsized":"2024-09-11T07:28:08.014794+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":57,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011597+0000","last_change":"2024-09-11T07:28:02.969498+0000","last_active":"2024-09-11T07:28:08.011597+0000","last_peered":"2024-09-11T07:28:08.011597+0000","last_clean":"2024-09-11T07:28:08.011597+0000","last_became_active":"2024-09-11T07:26:31.423932+0000","last_became_peered":"2024-09-11T07:26:31.423932+0000","last_unstale":"2024-09-11T07:28:08.011597+0000","last_undegraded":"2024-09-11T07:28:08.011597+0000","last_fullsized":"2024-09-11T07:28:08.011597+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:28:02.969442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.969442+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:22.652879+0000","scrub_duration":0.0014604450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":61,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014848+0000","last_change":"2024-09-11T07:27:55.150630+0000","last_active":"2024-09-11T07:28:08.014848+0000","last_peered":"2024-09-11T07:28:08.014848+0000","last_clean":"2024-09-11T07:28:08.014848+0000","last_became_active":"2024-09-11T07:26:33.546797+0000","last_became_peered":"2024-09-11T07:26:33.546797+0000","last_unstale":"2024-09-11T07:28:08.014848+0000","last_undegraded":"2024-09-11T07:28:08.014848+0000","last_fullsized":"2024-09-11T07:28:08.014848+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:27:55.150567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:55.150567+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:10.187014+0000","scrub_duration":0.0016612689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"64'8","reported_seq":75,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.012439+0000","last_change":"2024-09-11T07:28:01.123469+0000","last_active":"2024-09-11T07:28:09.012439+0000","last_peered":"2024-09-11T07:28:09.012439+0000","last_clean":"2024-09-11T07:28:09.012439+0000","last_became_active":"2024-09-11T07:26:35.559304+0000","last_became_peered":"2024-09-11T07:26:35.559304+0000","last_unstale":"2024-09-11T07:28:09.012439+0000","last_undegraded":"2024-09-11T07:28:09.012439+0000","last_fullsized":"2024-09-11T07:28:09.012439+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:28:01.123401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.123401+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:18.715539+0000","scrub_duration":0.00167504,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:11.735074+0000","last_change":"2024-09-11T07:28:11.735040+0000","last_active":"2024-09-11T07:28:11.735074+0000","last_peered":"2024-09-11T07:28:11.735074+0000","last_clean":"2024-09-11T07:28:11.735074+0000","last_became_active":"2024-09-11T07:26:37.562501+0000","last_became_peered":"2024-09-11T07:26:37.562501+0000","last_unstale":"2024-09-11T07:28:11.735074+0000","last_undegraded":"2024-09-11T07:28:11.735074+0000","last_fullsized":"2024-09-11T07:28:11.735074+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:11.734968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:11.734968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:13.880946+0000","scrub_duration":0.0021993350000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.010282+0000","last_change":"2024-09-11T07:27:38.732867+0000","last_active":"2024-09-11T07:28:08.010282+0000","last_peered":"2024-09-11T07:28:08.010282+0000","last_clean":"2024-09-11T07:28:08.010282+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:28:08.010282+0000","last_undegraded":"2024-09-11T07:28:08.010282+0000","last_fullsized":"2024-09-11T07:28:08.010282+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:38.732795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:38.732795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:56.730699+0000","scrub_duration":0.0025091950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":87,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.024756+0000","last_change":"2024-09-11T07:28:09.024756+0000","last_active":"2024-09-11T07:28:09.024756+0000","last_peered":"2024-09-11T07:28:09.024756+0000","last_clean":"2024-09-11T07:28:09.024756+0000","last_became_active":"2024-09-11T07:28:09.017997+0000","last_became_peered":"2024-09-11T07:28:09.017997+0000","last_unstale":"2024-09-11T07:28:09.024756+0000","last_undegraded":"2024-09-11T07:28:09.024756+0000","last_fullsized":"2024-09-11T07:28:09.024756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:28:03.164220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:03.164220+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:06.470841+0000","scrub_duration":0.0020677629999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014766+0000","last_change":"2024-09-11T07:26:35.559093+0000","last_active":"2024-09-11T07:28:08.014766+0000","last_peered":"2024-09-11T07:28:08.014766+0000","last_clean":"2024-09-11T07:28:08.014766+0000","last_became_active":"2024-09-11T07:26:35.558809+0000","last_became_peered":"2024-09-11T07:26:35.558809+0000","last_unstale":"2024-09-11T07:28:08.014766+0000","last_undegraded":"2024-09-11T07:28:08.014766+0000","last_fullsized":"2024-09-11T07:28:08.014766+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008901+0000","last_change":"2024-09-11T07:28:00.126756+0000","last_active":"2024-09-11T07:28:08.008901+0000","last_peered":"2024-09-11T07:28:08.008901+0000","last_clean":"2024-09-11T07:28:08.008901+0000","last_became_active":"2024-09-11T07:26:37.862373+0000","last_became_peered":"2024-09-11T07:26:37.862373+0000","last_unstale":"2024-09-11T07:28:08.008901+0000","last_undegraded":"2024-09-11T07:28:08.008901+0000","last_fullsized":"2024-09-11T07:28:08.008901+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:00.126702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.126702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:18.534592+0000","scrub_duration":0.001184903,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011441+0000","last_change":"2024-09-11T07:27:33.983889+0000","last_active":"2024-09-11T07:28:08.011441+0000","last_peered":"2024-09-11T07:28:08.011441+0000","last_clean":"2024-09-11T07:28:08.011441+0000","last_became_active":"2024-09-11T07:26:31.424651+0000","last_became_peered":"2024-09-11T07:26:31.424651+0000","last_unstale":"2024-09-11T07:28:08.011441+0000","last_undegraded":"2024-09-11T07:28:08.011441+0000","last_fullsized":"2024-09-11T07:28:08.011441+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:33.983813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:33.983813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:55.264233+0000","scrub_duration":0.001290759,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":71,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011285+0000","last_change":"2024-09-11T07:28:00.272702+0000","last_active":"2024-09-11T07:28:08.011285+0000","last_peered":"2024-09-11T07:28:08.011285+0000","last_clean":"2024-09-11T07:28:08.011285+0000","last_became_active":"2024-09-11T07:26:33.542544+0000","last_became_peered":"2024-09-11T07:26:33.542544+0000","last_unstale":"2024-09-11T07:28:08.011285+0000","last_undegraded":"2024-09-11T07:28:08.011285+0000","last_fullsized":"2024-09-11T07:28:08.011285+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-09-11T07:28:00.272628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.272628+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:05.374102+0000","scrub_duration":0.0021830019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.013186+0000","last_change":"2024-09-11T07:28:07.721176+0000","last_active":"2024-09-11T07:28:08.013186+0000","last_peered":"2024-09-11T07:28:08.013186+0000","last_clean":"2024-09-11T07:28:08.013186+0000","last_became_active":"2024-09-11T07:26:35.555862+0000","last_became_peered":"2024-09-11T07:26:35.555862+0000","last_unstale":"2024-09-11T07:28:08.013186+0000","last_undegraded":"2024-09-11T07:28:08.013186+0000","last_fullsized":"2024-09-11T07:28:08.013186+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:07.721112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:07.721112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:31.345841+0000","scrub_duration":0.0020675020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.009434+0000","last_change":"2024-09-11T07:28:06.674616+0000","last_active":"2024-09-11T07:28:08.009434+0000","last_peered":"2024-09-11T07:28:08.009434+0000","last_clean":"2024-09-11T07:28:08.009434+0000","last_became_active":"2024-09-11T07:26:37.861647+0000","last_became_peered":"2024-09-11T07:26:37.861647+0000","last_unstale":"2024-09-11T07:28:08.009434+0000","last_undegraded":"2024-09-11T07:28:08.009434+0000","last_fullsized":"2024-09-11T07:28:08.009434+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:06.674558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:06.674558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:17.029204+0000","scrub_duration":0.001489488,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008581+0000","last_change":"2024-09-11T07:27:45.732604+0000","last_active":"2024-09-11T07:28:08.008581+0000","last_peered":"2024-09-11T07:28:08.008581+0000","last_clean":"2024-09-11T07:28:08.008581+0000","last_became_active":"2024-09-11T07:26:31.420598+0000","last_became_peered":"2024-09-11T07:26:31.420598+0000","last_unstale":"2024-09-11T07:28:08.008581+0000","last_undegraded":"2024-09-11T07:28:08.008581+0000","last_fullsized":"2024-09-11T07:28:08.008581+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:45.732528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:45.732528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:48.605075+0000","scrub_duration":0.0015466659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"64'12","reported_seq":69,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008797+0000","last_change":"2024-09-11T07:27:59.737848+0000","last_active":"2024-09-11T07:28:08.008797+0000","last_peered":"2024-09-11T07:28:08.008797+0000","last_clean":"2024-09-11T07:28:08.008797+0000","last_became_active":"2024-09-11T07:26:33.542711+0000","last_became_peered":"2024-09-11T07:26:33.542711+0000","last_unstale":"2024-09-11T07:28:08.008797+0000","last_undegraded":"2024-09-11T07:28:08.008797+0000","last_fullsized":"2024-09-11T07:28:08.008797+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-09-11T07:27:59.737791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:59.737791+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:11.558595+0000","scrub_duration":0.00367732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008620+0000","last_change":"2024-09-11T07:27:57.199239+0000","last_active":"2024-09-11T07:28:08.008620+0000","last_peered":"2024-09-11T07:28:08.008620+0000","last_clean":"2024-09-11T07:28:08.008620+0000","last_became_active":"2024-09-11T07:26:35.562037+0000","last_became_peered":"2024-09-11T07:26:35.562037+0000","last_unstale":"2024-09-11T07:28:08.008620+0000","last_undegraded":"2024-09-11T07:28:08.008620+0000","last_fullsized":"2024-09-11T07:28:08.008620+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:57.199173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.199173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.093487+0000","scrub_duration":0.0015950770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":421,"ondisk_log_size":421,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":96,"num_read_kb":91,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299417,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298568,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446584,"statfs":{"total":95995035648,"available":95689302016,"internally_reserved":0,"allocated":1245184,"data_stored":889696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.65900000000000003}]},{"osd":1,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.59799999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.57399999999999995}]},{"osd":3,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":12.625999999999999,"5min":12.625999999999999,"15min":12.625999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.67700000000000005}]},{"osd":4,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.498}]},{"osd":5,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.70599999999999996}]},{"osd":6,"last update":"Wed Sep 11 07:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54400000000000004}]}]},{"osd":6,"up_from":44,"seq":188978561053,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298552,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446600,"statfs":{"total":95995035648,"available":95689318400,"internally_reserved":0,"allocated":1220608,"data_stored":888174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.374,"5min":0.247,"15min":0.247},"max":{"1min":0.65000000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.314},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.60499999999999998,"15min":0.624},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.754,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.34499999999999997}]},{"osd":1,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.82899999999999996,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.49299999999999999,"15min":0.495},"min":{"1min":0.35799999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.51200000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.61499999999999999,"15min":0.63500000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.70299999999999996,"5min":3.234,"15min":3.234},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.59099999999999997,"15min":0.60699999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.80200000000000005,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.41899999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51500000000000001,"15min":0.51800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.63500000000000001,"15min":0.65500000000000003},"min":{"1min":0.41199999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.66900000000000004,"5min":3.1739999999999999,"15min":3.1739999999999999},"last":0.53600000000000003}]},{"osd":4,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.58299999999999996,"15min":0.60899999999999999},"min":{"1min":0.32700000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.65200000000000002,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.28399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.57499999999999996}]},{"osd":5,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.65600000000000003,"15min":0.69699999999999995},"min":{"1min":0.32400000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64000000000000001,"5min":5.7329999999999997,"15min":5.7329999999999997},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.52000000000000002,"15min":0.52700000000000002},"min":{"1min":0.34000000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70899999999999996,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.56000000000000005}]},{"osd":7,"last update":"Wed Sep 11 07:27:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.6349999999999998,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.495}]}]},{"osd":5,"up_from":37,"seq":158913789987,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297972,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447180,"statfs":{"total":95995035648,"available":95689912320,"internally_reserved":0,"allocated":634880,"data_stored":298797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.35499999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.317},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.50600000000000001,"15min":0.502},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.38200000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.29599999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.42799999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.48299999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.375,"15min":0.37},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.42599999999999999,"15min":0.41999999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.34200000000000003}]},{"osd":6,"last update":"Wed Sep 11 07:28:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.41599999999999998,"15min":0.40600000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.42699999999999999,"15min":0.42199999999999999},"min":{"1min":0.32100000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.46000000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:27:21 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.54000000000000004}]}]},{"osd":4,"up_from":31,"seq":133143986215,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297992,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447160,"statfs":{"total":95995035648,"available":95689891840,"internally_reserved":0,"allocated":655360,"data_stored":299132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51600000000000001,"15min":0.51300000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68999999999999995,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.626},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.502,"15min":0.498},"min":{"1min":0.36099999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.63900000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.5,"15min":0.49399999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.71299999999999997,"15min":0.73699999999999999},"min":{"1min":0.372,"5min":0.318,"15min":0.318},"max":{"1min":1.1200000000000001,"5min":4.5510000000000002,"15min":4.5510000000000002},"last":0.65800000000000003}]},{"osd":2,"last update":"Wed Sep 11 07:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.51600000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77400000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.66900000000000004}]},{"osd":5,"last update":"Wed Sep 11 07:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.34499999999999997,"15min":0.33800000000000002},"min":{"1min":0.27600000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.378,"15min":0.372},"min":{"1min":0.29399999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.438}]},{"osd":6,"last update":"Wed Sep 11 07:27:59 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42199999999999999,"15min":0.41799999999999998},"min":{"1min":0.315,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.44,"15min":0.43099999999999999},"min":{"1min":0.33500000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.498}]},{"osd":7,"last update":"Wed Sep 11 07:27:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.59599999999999997}]}]},{"osd":3,"up_from":25,"seq":107374182444,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298000,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447152,"statfs":{"total":95995035648,"available":95689883648,"internally_reserved":0,"allocated":663552,"data_stored":298596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:54 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38600000000000001,"15min":0.39300000000000002},"min":{"1min":0.20799999999999999,"5min":0.185,"15min":0.185},"max":{"1min":0.54300000000000004,"5min":0.621,"15min":0.621},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.42499999999999999,"15min":0.441},"min":{"1min":0.26200000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.39900000000000002}]},{"osd":1,"last update":"Wed Sep 11 07:27:54 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.43099999999999999,"15min":0.437},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.625,"5min":0.752,"15min":0.752},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.41599999999999998,"15min":0.43099999999999999},"min":{"1min":0.23599999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.34899999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:27:54 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.45100000000000001,"15min":0.46500000000000002},"min":{"1min":0.26900000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.498},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.45400000000000001,"15min":0.46899999999999997},"min":{"1min":0.28899999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.432}]},{"osd":4,"last update":"Wed Sep 11 07:27:12 2024","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":0.746,"15min":0.62},"min":{"1min":0.47299999999999998,"5min":0.376,"15min":0.376},"max":{"1min":21.856000000000002,"5min":21.856000000000002,"15min":21.856000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":1.345,"5min":0.72199999999999998,"15min":0.61799999999999999},"min":{"1min":0.432,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":19.146000000000001,"5min":19.146000000000001,"15min":19.146000000000001},"last":0.45300000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":1.3069999999999999,"5min":0.71999999999999997,"15min":0.622},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":18.416,"5min":18.416,"15min":18.416},"last":0.60199999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:27:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":1.306,"15min":1.4330000000000001},"min":{"1min":0.439,"5min":0.437,"15min":0.437},"max":{"1min":0.72899999999999998,"5min":20.219999999999999,"15min":20.219999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":1.319,"15min":1.446},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72699999999999998,"5min":19.609000000000002,"15min":19.609000000000002},"last":0.58899999999999997}]},{"osd":7,"last update":"Wed Sep 11 07:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.628},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.57099999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411377,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297972,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447180,"statfs":{"total":95995035648,"available":95689912320,"internally_reserved":0,"allocated":630784,"data_stored":298043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:26 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.375,"15min":0.36299999999999999},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.38200000000000001,"15min":0.374},"min":{"1min":0.26600000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.55900000000000005,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.32200000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:27:26 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.40300000000000002,"15min":0.38600000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.254},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.40200000000000002,"15min":0.38300000000000001},"min":{"1min":0.218,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.34100000000000003}]},{"osd":3,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.439,"15min":0.42299999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64500000000000002,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":0.307},{"interface":"front","average":{"1min":0.44,"5min":0.45900000000000002,"15min":0.45400000000000001},"min":{"1min":0.26600000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65000000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.374}]},{"osd":4,"last update":"Wed Sep 11 07:27:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55800000000000005,"15min":0.55300000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57699999999999996,"15min":0.57599999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80700000000000005,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.50600000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:27:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.68799999999999994,"15min":0.69699999999999995},"min":{"1min":0.45500000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85899999999999999,"5min":2.931,"15min":2.931},"last":0.59099999999999997}]},{"osd":6,"last update":"Wed Sep 11 07:27:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.85299999999999998,"5min":1.494,"15min":1.494},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.63100000000000001,"15min":0.63300000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.875,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.46999999999999997}]},{"osd":7,"last update":"Wed Sep 11 07:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.499},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.622}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298008,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447144,"statfs":{"total":95995035648,"available":95689875456,"internally_reserved":0,"allocated":671744,"data_stored":299233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:28:04 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.34000000000000002,"15min":0.33100000000000002},"min":{"1min":0.22800000000000001,"5min":0.186,"15min":0.186},"max":{"1min":0.49299999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.36499999999999999,"15min":0.35499999999999998},"min":{"1min":0.22700000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.38200000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:27:26 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.41899999999999998,"15min":0.41599999999999998},"min":{"1min":0.317,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.41199999999999998,"15min":0.40799999999999997},"min":{"1min":0.28699999999999998,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.50900000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:27:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.44500000000000001,"15min":0.44700000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.60699999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.41399999999999998},"min":{"1min":0.307,"5min":0.253,"15min":0.253},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.42399999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:27:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.56200000000000006}]},{"osd":5,"last update":"Wed Sep 11 07:27:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.44,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.71699999999999997,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.61799999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:27:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57199999999999995,"15min":0.57499999999999996},"min":{"1min":0.41199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.754,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.40999999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.53600000000000003}]},{"osd":7,"last update":"Wed Sep 11 07:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.495}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298560,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446592,"statfs":{"total":95995035648,"available":95689310208,"internally_reserved":0,"allocated":1236992,"data_stored":888506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:28:02 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.36699999999999999,"15min":0.34499999999999997},"min":{"1min":0.246,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.35899999999999999,"15min":0.34200000000000003},"min":{"1min":0.23799999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.32500000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:27:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.40400000000000003,"15min":0.40699999999999997},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.59299999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.308}]},{"osd":3,"last update":"Wed Sep 11 07:27:55 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45800000000000002,"15min":0.47399999999999998},"min":{"1min":0.317,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.61199999999999999,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.46300000000000002,"15min":0.48099999999999998},"min":{"1min":0.28299999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.56200000000000006,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.35499999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:28:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56000000000000005,"15min":0.56799999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.77600000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.52900000000000003}]},{"osd":5,"last update":"Wed Sep 11 07:27:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.40000000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.79100000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.41699999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78100000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57599999999999996}]},{"osd":6,"last update":"Wed Sep 11 07:27:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59399999999999997,"15min":0.59699999999999998},"min":{"1min":0.34799999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60099999999999998,"15min":0.60399999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.76400000000000001,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.54600000000000004}]},{"osd":7,"last update":"Wed Sep 11 07:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.48499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:28:18.639 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph status --format=json 2024-09-11T07:28:18.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:18 smithi104 ceph-mon[33507]: 5.15 scrub starts 2024-09-11T07:28:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:18 smithi104 ceph-mon[33507]: 5.15 scrub ok 2024-09-11T07:28:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:18 smithi104 ceph-mon[33507]: 5.18 scrub starts 2024-09-11T07:28:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:18 smithi104 ceph-mon[33507]: 5.18 scrub ok 2024-09-11T07:28:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:18 smithi104 ceph-mon[33507]: 5.1e scrub starts 2024-09-11T07:28:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:18 smithi104 ceph-mon[33507]: 5.1e scrub ok 2024-09-11T07:28:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:18 smithi104 ceph-mon[33507]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 22 B/s, 1 objects/s recovering 2024-09-11T07:28:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:18 smithi104 ceph-mon[33507]: from='client.24674 v1:172.21.15.22:0/972981605' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:28:18.900 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:28:18.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:18 smithi022 ceph-mon[29462]: 5.15 scrub starts 2024-09-11T07:28:18.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:18 smithi022 ceph-mon[29462]: 5.15 scrub ok 2024-09-11T07:28:18.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:18 smithi022 ceph-mon[29462]: 5.18 scrub starts 2024-09-11T07:28:18.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:18 smithi022 ceph-mon[29462]: 5.18 scrub ok 2024-09-11T07:28:18.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:18 smithi022 ceph-mon[29462]: 5.1e scrub starts 2024-09-11T07:28:18.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:18 smithi022 ceph-mon[29462]: 5.1e scrub ok 2024-09-11T07:28:18.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:18 smithi022 ceph-mon[29462]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 22 B/s, 1 objects/s recovering 2024-09-11T07:28:18.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:18 smithi022 ceph-mon[29462]: from='client.24674 v1:172.21.15.22:0/972981605' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:28:18.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:18 smithi022 ceph-mon[37330]: 5.15 scrub starts 2024-09-11T07:28:18.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:18 smithi022 ceph-mon[37330]: 5.15 scrub ok 2024-09-11T07:28:18.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:18 smithi022 ceph-mon[37330]: 5.18 scrub starts 2024-09-11T07:28:18.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:18 smithi022 ceph-mon[37330]: 5.18 scrub ok 2024-09-11T07:28:18.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:18 smithi022 ceph-mon[37330]: 5.1e scrub starts 2024-09-11T07:28:18.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:18 smithi022 ceph-mon[37330]: 5.1e scrub ok 2024-09-11T07:28:18.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:18 smithi022 ceph-mon[37330]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 22 B/s, 1 objects/s recovering 2024-09-11T07:28:18.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:18 smithi022 ceph-mon[37330]: from='client.24674 v1:172.21.15.22:0/972981605' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:28:19.062 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:28:18 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:28:19.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:19 smithi022 ceph-mon[29462]: 6.9 scrub starts 2024-09-11T07:28:19.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:19 smithi022 ceph-mon[29462]: 6.9 scrub ok 2024-09-11T07:28:19.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:19 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:28:19.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:19 smithi022 ceph-mon[37330]: 6.9 scrub starts 2024-09-11T07:28:19.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:19 smithi022 ceph-mon[37330]: 6.9 scrub ok 2024-09-11T07:28:19.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:19 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:28:19.961 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:19 smithi104 ceph-mon[33507]: 6.9 scrub starts 2024-09-11T07:28:19.961 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:19 smithi104 ceph-mon[33507]: 6.9 scrub ok 2024-09-11T07:28:19.961 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:19 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:28:20.280 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:20 smithi104 systemd[1]: Stopping Ceph prometheus.a for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:28:20.660 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:20 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[61777]: ts=2024-09-11T07:28:20.363Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-11T07:28:20.660 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:20 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[61777]: ts=2024-09-11T07:28:20.364Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-11T07:28:20.660 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:20 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[61777]: ts=2024-09-11T07:28:20.364Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-11T07:28:20.660 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:20 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[61777]: ts=2024-09-11T07:28:20.364Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-11T07:28:20.660 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:20 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[61777]: ts=2024-09-11T07:28:20.364Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-11T07:28:20.661 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:20 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[61777]: ts=2024-09-11T07:28:20.364Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-11T07:28:20.661 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:20 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[61777]: ts=2024-09-11T07:28:20.364Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-11T07:28:20.661 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:20 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[61777]: ts=2024-09-11T07:28:20.364Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-11T07:28:20.661 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:20 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[61777]: ts=2024-09-11T07:28:20.364Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-11T07:28:20.661 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:20 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[61777]: ts=2024-09-11T07:28:20.364Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-11T07:28:20.661 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:20 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[61777]: ts=2024-09-11T07:28:20.364Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-11T07:28:20.661 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:20 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[61777]: ts=2024-09-11T07:28:20.364Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-11T07:28:20.661 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:20 smithi104 podman[64784]: 2024-09-11 07:28:20.374179806 +0000 UTC m=+0.118289602 container died a18d9e629c34f88d0fd71d2314dd449f3ab931a8192b2351020f0dcd697a6e05 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:28:20.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:20 smithi022 ceph-mon[37330]: 6.d scrub starts 2024-09-11T07:28:20.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:20 smithi022 ceph-mon[37330]: 6.d scrub ok 2024-09-11T07:28:20.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:20 smithi022 ceph-mon[37330]: 2.a scrub starts 2024-09-11T07:28:20.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:20 smithi022 ceph-mon[37330]: 2.a scrub ok 2024-09-11T07:28:20.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:20 smithi022 ceph-mon[37330]: 6.1a scrub starts 2024-09-11T07:28:20.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:20 smithi022 ceph-mon[37330]: 6.1a scrub ok 2024-09-11T07:28:20.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:20 smithi022 ceph-mon[37330]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 935 B/s rd, 0 op/s; 20 B/s, 1 objects/s recovering 2024-09-11T07:28:20.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:20 smithi022 ceph-mon[29462]: 6.d scrub starts 2024-09-11T07:28:20.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:20 smithi022 ceph-mon[29462]: 6.d scrub ok 2024-09-11T07:28:20.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:20 smithi022 ceph-mon[29462]: 2.a scrub starts 2024-09-11T07:28:20.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:20 smithi022 ceph-mon[29462]: 2.a scrub ok 2024-09-11T07:28:20.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:20 smithi022 ceph-mon[29462]: 6.1a scrub starts 2024-09-11T07:28:20.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:20 smithi022 ceph-mon[29462]: 6.1a scrub ok 2024-09-11T07:28:20.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:20 smithi022 ceph-mon[29462]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 935 B/s rd, 0 op/s; 20 B/s, 1 objects/s recovering 2024-09-11T07:28:20.932 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:20 smithi104 podman[64784]: 2024-09-11 07:28:20.679549234 +0000 UTC m=+0.423659030 container cleanup a18d9e629c34f88d0fd71d2314dd449f3ab931a8192b2351020f0dcd697a6e05 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:28:20.932 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:20 smithi104 bash[64784]: ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a 2024-09-11T07:28:20.932 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:20 smithi104 ceph-mon[33507]: 6.d scrub starts 2024-09-11T07:28:20.932 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:20 smithi104 ceph-mon[33507]: 6.d scrub ok 2024-09-11T07:28:20.932 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:20 smithi104 ceph-mon[33507]: 2.a scrub starts 2024-09-11T07:28:20.933 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:20 smithi104 ceph-mon[33507]: 2.a scrub ok 2024-09-11T07:28:20.933 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:20 smithi104 ceph-mon[33507]: 6.1a scrub starts 2024-09-11T07:28:20.933 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:20 smithi104 ceph-mon[33507]: 6.1a scrub ok 2024-09-11T07:28:20.933 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:20 smithi104 ceph-mon[33507]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 935 B/s rd, 0 op/s; 20 B/s, 1 objects/s recovering 2024-09-11T07:28:20.941 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:28:20.941 INFO:teuthology.orchestra.run.smithi022.stdout:{"fsid":"12a285d8-700e-11ef-bcea-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":320,"monmap":{"epoch":3,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":86,"num_osds":8,"num_up_osds":8,"osd_up_since":1726039575,"num_in_osds":8,"osd_in_since":1726039558,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":132}],"num_pgs":132,"num_pools":6,"num_objects":196,"data_bytes":596188,"bytes_used":2442866688,"bytes_avail":765517418496,"bytes_total":767960285184,"recovering_objects_per_sec":1,"recovering_bytes_per_sec":20,"recovering_keys_per_sec":0,"num_objects_recovered":19,"num_bytes_recovered":220,"num_keys_recovered":0,"read_bytes_sec":935,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.22:8443/","prometheus":"http://172.21.15.22:9283/"}},"servicemap":{"epoch":14,"modified":"2024-09-11T07:28:09.946897+0000","services":{"rgw":{"daemons":{"summary":"","14496":{"start_epoch":12,"start_stamp":"2024-09-11T07:26:38.680156+0000","gid":14496,"addr":"172.21.15.22:0/4007432209","metadata":{"arch":"x86_64","ceph_release":"quincy","ceph_version":"ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)","ceph_version_short":"17.2.7-1623-gd0e6828a","container_hostname":"smithi022","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40","cpu":"Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 9","distro_version":"9","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi022","id":"foo.a","kernel_description":"#1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024","kernel_version":"5.14.0-505.el9.x86_64","mem_swap_kb":"0","mem_total_kb":"32493488","num_handles":"1","os":"Linux","pid":"2","realm_id":"","realm_name":"","zone_id":"0b3b8b5f-128e-4275-9a53-2f5d469a5fa0","zone_name":"default","zonegroup_id":"ad4c13d5-72d7-4cd3-8b1b-9e61229fc6b2","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T07:28:21.280 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:20 smithi104 podman[64796]: 2024-09-11 07:28:20.931017198 +0000 UTC m=+0.558338445 container remove a18d9e629c34f88d0fd71d2314dd449f3ab931a8192b2351020f0dcd697a6e05 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:28:21.280 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 systemd[1]: ceph-12a285d8-700e-11ef-bcea-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-11T07:28:21.280 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 systemd[1]: Stopped Ceph prometheus.a for 12a285d8-700e-11ef-bcea-c7b262605968. 2024-09-11T07:28:21.280 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 systemd[1]: ceph-12a285d8-700e-11ef-bcea-c7b262605968@prometheus.a.service: Consumed 1.005s CPU time. 2024-09-11T07:28:21.280 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 systemd[1]: Starting Ceph prometheus.a for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:28:21.558 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 podman[64889]: 2024-09-11 07:28:21.209279823 +0000 UTC m=+0.016628305 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-11T07:28:21.558 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 podman[64889]: 2024-09-11 07:28:21.306855032 +0000 UTC m=+0.114203551 container create 79e4627df0f1e0410710e4a310cd7df609aff1aefe0ab327c43d6651fd5c8107 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:28:21.558 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 podman[64889]: 2024-09-11 07:28:21.508639648 +0000 UTC m=+0.315988124 container init 79e4627df0f1e0410710e4a310cd7df609aff1aefe0ab327c43d6651fd5c8107 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:28:21.558 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 podman[64889]: 2024-09-11 07:28:21.511589984 +0000 UTC m=+0.318938461 container start 79e4627df0f1e0410710e4a310cd7df609aff1aefe0ab327c43d6651fd5c8107 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:28:21.558 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:28:21.534Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-11T07:28:21.558 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:28:21.534Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-11T07:28:21.558 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:28:21.534Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi104 (none))" 2024-09-11T07:28:21.559 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:28:21.535Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-11T07:28:21.559 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:28:21.535Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-11T07:28:21.559 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:28:21.537Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-11T07:28:21.559 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:28:21.537Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-11T07:28:21.559 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:28:21.538Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-11T07:28:21.559 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:28:21.538Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-11T07:28:21.559 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:28:21.539Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-11T07:28:21.559 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:28:21.539Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.105µs 2024-09-11T07:28:21.559 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:28:21.539Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-11T07:28:21.559 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:28:21.540Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-11T07:28:21.559 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:28:21.541Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-11T07:28:21.559 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:28:21.541Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=26.124µs wal_replay_duration=1.37428ms wbl_replay_duration=131ns total_replay_duration=1.429191ms 2024-09-11T07:28:21.559 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:28:21.542Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-11T07:28:21.560 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:28:21.542Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-11T07:28:21.560 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:28:21.542Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-11T07:28:21.560 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:28:21.556Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=14.291628ms db_storage=704ns remote_storage=984ns web_handler=357ns query_engine=578ns scrape=388.698µs scrape_sd=24.879µs notify=25.669µs notify_sd=7.043µs rules=13.547314ms tracing=4.452µs 2024-09-11T07:28:21.560 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:28:21.556Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-11T07:28:21.560 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:28:21.556Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-11T07:28:21.691 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-09-11T07:28:21.691 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:28:21.915 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:21 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:28:21] ENGINE Bus STOPPING 2024-09-11T07:28:21.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:21 smithi022 ceph-mon[29462]: 4.1 scrub starts 2024-09-11T07:28:21.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:21 smithi022 ceph-mon[29462]: 4.1 scrub ok 2024-09-11T07:28:21.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:21 smithi022 ceph-mon[29462]: 6.11 scrub starts 2024-09-11T07:28:21.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:21 smithi022 ceph-mon[29462]: 6.11 scrub ok 2024-09-11T07:28:21.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:21 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2757845688' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-11T07:28:21.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:21 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:21.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:21 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:21.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:21 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T07:28:21.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:21 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi022.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T07:28:21.916 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:21 smithi022 ceph-mon[37330]: 4.1 scrub starts 2024-09-11T07:28:21.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:21 smithi022 ceph-mon[37330]: 4.1 scrub ok 2024-09-11T07:28:21.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:21 smithi022 ceph-mon[37330]: 6.11 scrub starts 2024-09-11T07:28:21.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:21 smithi022 ceph-mon[37330]: 6.11 scrub ok 2024-09-11T07:28:21.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:21 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2757845688' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-11T07:28:21.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:21 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:21.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:21 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:21.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:21 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T07:28:21.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:21 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi022.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T07:28:21.980 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:28:22.030 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 bash[64889]: 79e4627df0f1e0410710e4a310cd7df609aff1aefe0ab327c43d6651fd5c8107 2024-09-11T07:28:22.030 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:28:21 smithi104 systemd[1]: Started Ceph prometheus.a for 12a285d8-700e-11ef-bcea-c7b262605968. 2024-09-11T07:28:22.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-mon[33507]: 4.1 scrub starts 2024-09-11T07:28:22.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-mon[33507]: 4.1 scrub ok 2024-09-11T07:28:22.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-mon[33507]: 6.11 scrub starts 2024-09-11T07:28:22.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-mon[33507]: 6.11 scrub ok 2024-09-11T07:28:22.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2757845688' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-11T07:28:22.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:22.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:22.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T07:28:22.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:21 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi022.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T07:28:22.167 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:21 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:28:21] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T07:28:22.167 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:21 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:28:21] ENGINE Bus STOPPED 2024-09-11T07:28:22.167 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:21 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:28:21] ENGINE Bus STARTING 2024-09-11T07:28:22.167 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:28:22] ENGINE Serving on http://:::9283 2024-09-11T07:28:22.167 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:28:22] ENGINE Bus STARTED 2024-09-11T07:28:22.167 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:28:22] ENGINE Bus STOPPING 2024-09-11T07:28:22.917 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:28:22] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T07:28:22.917 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:28:22] ENGINE Bus STOPPED 2024-09-11T07:28:22.917 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:28:22] ENGINE Bus STARTING 2024-09-11T07:28:22.918 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:28:22] ENGINE Serving on http://:::9283 2024-09-11T07:28:22.918 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:28:22] ENGINE Bus STARTED 2024-09-11T07:28:22.918 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:28:22] ENGINE Bus STOPPING 2024-09-11T07:28:22.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[29462]: 4.c scrub starts 2024-09-11T07:28:22.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[29462]: 4.c scrub ok 2024-09-11T07:28:22.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[29462]: from='mon.0 v1:172.21.15.22:0/3782157251' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T07:28:22.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[29462]: from='mon.0 v1:172.21.15.22:0/3782157251' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi022.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T07:28:22.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:22.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T07:28:22.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[29462]: from='mon.0 v1:172.21.15.22:0/3782157251' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T07:28:22.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi104.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T07:28:22.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[29462]: from='mon.0 v1:172.21.15.22:0/3782157251' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi104.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T07:28:22.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:22.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T07:28:22.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[29462]: from='mon.0 v1:172.21.15.22:0/3782157251' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T07:28:22.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi104.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T07:28:22.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[29462]: from='mon.0 v1:172.21.15.22:0/3782157251' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi104.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T07:28:22.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:22.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:22.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[29462]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 18 B/s, 1 objects/s recovering 2024-09-11T07:28:22.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[37330]: 4.c scrub starts 2024-09-11T07:28:22.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[37330]: 4.c scrub ok 2024-09-11T07:28:22.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[37330]: from='mon.0 v1:172.21.15.22:0/3782157251' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T07:28:22.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[37330]: from='mon.0 v1:172.21.15.22:0/3782157251' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi022.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T07:28:22.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:22.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T07:28:22.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[37330]: from='mon.0 v1:172.21.15.22:0/3782157251' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T07:28:22.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi104.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T07:28:22.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[37330]: from='mon.0 v1:172.21.15.22:0/3782157251' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi104.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T07:28:22.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:22.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T07:28:22.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[37330]: from='mon.0 v1:172.21.15.22:0/3782157251' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T07:28:22.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi104.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T07:28:22.921 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[37330]: from='mon.0 v1:172.21.15.22:0/3782157251' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi104.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T07:28:22.921 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:22.921 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:22.921 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:22 smithi022 ceph-mon[37330]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 18 B/s, 1 objects/s recovering 2024-09-11T07:28:23.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:22 smithi104 ceph-mon[33507]: 4.c scrub starts 2024-09-11T07:28:23.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:22 smithi104 ceph-mon[33507]: 4.c scrub ok 2024-09-11T07:28:23.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:22 smithi104 ceph-mon[33507]: from='mon.0 v1:172.21.15.22:0/3782157251' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T07:28:23.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:22 smithi104 ceph-mon[33507]: from='mon.0 v1:172.21.15.22:0/3782157251' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi022.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T07:28:23.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:22 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:23.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:22 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T07:28:23.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:22 smithi104 ceph-mon[33507]: from='mon.0 v1:172.21.15.22:0/3782157251' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T07:28:23.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:22 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi104.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T07:28:23.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:22 smithi104 ceph-mon[33507]: from='mon.0 v1:172.21.15.22:0/3782157251' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi104.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T07:28:23.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:22 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:23.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:22 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T07:28:23.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:22 smithi104 ceph-mon[33507]: from='mon.0 v1:172.21.15.22:0/3782157251' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T07:28:23.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:22 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi104.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T07:28:23.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:22 smithi104 ceph-mon[33507]: from='mon.0 v1:172.21.15.22:0/3782157251' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi104.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T07:28:23.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:22 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:23.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:22 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:23.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:22 smithi104 ceph-mon[33507]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 18 B/s, 1 objects/s recovering 2024-09-11T07:28:23.417 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:23 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:28:23] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T07:28:23.417 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:23 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:28:23] ENGINE Bus STOPPED 2024-09-11T07:28:23.417 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:23 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:28:23] ENGINE Bus STARTING 2024-09-11T07:28:23.417 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:23 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:28:23] ENGINE Serving on http://:::9283 2024-09-11T07:28:23.417 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:23 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:28:23] ENGINE Bus STARTED 2024-09-11T07:28:24.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:23 smithi104 ceph-mon[33507]: 5.7 scrub starts 2024-09-11T07:28:24.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:23 smithi104 ceph-mon[33507]: 5.7 scrub ok 2024-09-11T07:28:24.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:23 smithi104 ceph-mon[33507]: 6.17 scrub starts 2024-09-11T07:28:24.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:23 smithi104 ceph-mon[33507]: 6.17 scrub ok 2024-09-11T07:28:24.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:23 smithi104 ceph-mon[33507]: 4.14 deep-scrub starts 2024-09-11T07:28:24.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:23 smithi104 ceph-mon[33507]: 4.14 deep-scrub ok 2024-09-11T07:28:24.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:28:24.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:23 smithi104 ceph-mon[33507]: 2.1d scrub starts 2024-09-11T07:28:24.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:23 smithi104 ceph-mon[33507]: 2.1d scrub ok 2024-09-11T07:28:24.054 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:23 smithi022 ceph-mon[29462]: 5.7 scrub starts 2024-09-11T07:28:24.054 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:23 smithi022 ceph-mon[29462]: 5.7 scrub ok 2024-09-11T07:28:24.054 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:23 smithi022 ceph-mon[29462]: 6.17 scrub starts 2024-09-11T07:28:24.054 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:23 smithi022 ceph-mon[29462]: 6.17 scrub ok 2024-09-11T07:28:24.054 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:23 smithi022 ceph-mon[29462]: 4.14 deep-scrub starts 2024-09-11T07:28:24.054 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:23 smithi022 ceph-mon[29462]: 4.14 deep-scrub ok 2024-09-11T07:28:24.054 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:28:24.054 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:23 smithi022 ceph-mon[29462]: 2.1d scrub starts 2024-09-11T07:28:24.054 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:23 smithi022 ceph-mon[29462]: 2.1d scrub ok 2024-09-11T07:28:24.055 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:23 smithi022 ceph-mon[37330]: 5.7 scrub starts 2024-09-11T07:28:24.055 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:23 smithi022 ceph-mon[37330]: 5.7 scrub ok 2024-09-11T07:28:24.055 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:23 smithi022 ceph-mon[37330]: 6.17 scrub starts 2024-09-11T07:28:24.055 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:23 smithi022 ceph-mon[37330]: 6.17 scrub ok 2024-09-11T07:28:24.055 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:23 smithi022 ceph-mon[37330]: 4.14 deep-scrub starts 2024-09-11T07:28:24.055 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:23 smithi022 ceph-mon[37330]: 4.14 deep-scrub ok 2024-09-11T07:28:24.055 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:28:24.055 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:23 smithi022 ceph-mon[37330]: 2.1d scrub starts 2024-09-11T07:28:24.055 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:23 smithi022 ceph-mon[37330]: 2.1d scrub ok 2024-09-11T07:28:24.075 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:28:24.077 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:28:24.881 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-09-11T07:28:23.950018+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":959,"num_read_kb":816,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":590588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":589,"ondisk_log_size":589,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385612,"kb_used_data":6796,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575604,"statfs":{"total":767960285184,"available":765517418496,"internally_reserved":0,"allocated":6959104,"data_stored":4160177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002690"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008184+0000","last_change":"2024-09-11T07:26:37.863505+0000","last_active":"2024-09-11T07:28:08.008184+0000","last_peered":"2024-09-11T07:28:08.008184+0000","last_clean":"2024-09-11T07:28:08.008184+0000","last_became_active":"2024-09-11T07:26:37.863165+0000","last_became_peered":"2024-09-11T07:26:37.863165+0000","last_unstale":"2024-09-11T07:28:08.008184+0000","last_undegraded":"2024-09-11T07:28:08.008184+0000","last_fullsized":"2024-09-11T07:28:08.008184+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014603+0000","last_change":"2024-09-11T07:27:51.197966+0000","last_active":"2024-09-11T07:28:08.014603+0000","last_peered":"2024-09-11T07:28:08.014603+0000","last_clean":"2024-09-11T07:28:08.014603+0000","last_became_active":"2024-09-11T07:26:31.421005+0000","last_became_peered":"2024-09-11T07:26:31.421005+0000","last_unstale":"2024-09-11T07:28:08.014603+0000","last_undegraded":"2024-09-11T07:28:08.014603+0000","last_fullsized":"2024-09-11T07:28:08.014603+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:51.197903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:51.197903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:00.681709+0000","scrub_duration":0.001600113,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"82'22","reported_seq":78,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008850+0000","last_change":"2024-09-11T07:27:50.756777+0000","last_active":"2024-09-11T07:28:08.008850+0000","last_peered":"2024-09-11T07:28:08.008850+0000","last_clean":"2024-09-11T07:28:08.008850+0000","last_became_active":"2024-09-11T07:26:33.544627+0000","last_became_peered":"2024-09-11T07:26:33.544627+0000","last_unstale":"2024-09-11T07:28:08.008850+0000","last_undegraded":"2024-09-11T07:28:08.008850+0000","last_fullsized":"2024-09-11T07:28:08.008850+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.756707+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.257406+0000","scrub_duration":0.0049180669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:17.046791+0000","last_change":"2024-09-11T07:28:17.046764+0000","last_active":"2024-09-11T07:28:17.046791+0000","last_peered":"2024-09-11T07:28:17.046791+0000","last_clean":"2024-09-11T07:28:17.046791+0000","last_became_active":"2024-09-11T07:26:35.556296+0000","last_became_peered":"2024-09-11T07:26:35.556296+0000","last_unstale":"2024-09-11T07:28:17.046791+0000","last_undegraded":"2024-09-11T07:28:17.046791+0000","last_fullsized":"2024-09-11T07:28:17.046791+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:17.046711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:17.046711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:21.028176+0000","scrub_duration":0.0011666510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.009266+0000","last_change":"2024-09-11T07:27:36.776948+0000","last_active":"2024-09-11T07:28:08.009266+0000","last_peered":"2024-09-11T07:28:08.009266+0000","last_clean":"2024-09-11T07:28:08.009266+0000","last_became_active":"2024-09-11T07:26:31.420178+0000","last_became_peered":"2024-09-11T07:26:31.420178+0000","last_unstale":"2024-09-11T07:28:08.009266+0000","last_undegraded":"2024-09-11T07:28:08.009266+0000","last_fullsized":"2024-09-11T07:28:08.009266+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:36.776883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:36.776883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:49.961787+0000","scrub_duration":0.0017449290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"86'13","reported_seq":66,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:15.025895+0000","last_change":"2024-09-11T07:28:15.025864+0000","last_active":"2024-09-11T07:28:15.025895+0000","last_peered":"2024-09-11T07:28:15.025895+0000","last_clean":"2024-09-11T07:28:15.025895+0000","last_became_active":"2024-09-11T07:26:33.540732+0000","last_became_peered":"2024-09-11T07:26:33.540732+0000","last_unstale":"2024-09-11T07:28:15.025895+0000","last_undegraded":"2024-09-11T07:28:15.025895+0000","last_fullsized":"2024-09-11T07:28:15.025895+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'13","last_scrub_stamp":"2024-09-11T07:28:15.025757+0000","last_deep_scrub":"86'13","last_deep_scrub_stamp":"2024-09-11T07:28:15.025757+0000","last_clean_scrub_stamp":"2024-09-11T07:28:15.025757+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:26.818521+0000","scrub_duration":0.0039188890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012014+0000","last_change":"2024-09-11T07:27:50.648473+0000","last_active":"2024-09-11T07:28:08.012014+0000","last_peered":"2024-09-11T07:28:08.012014+0000","last_clean":"2024-09-11T07:28:08.012014+0000","last_became_active":"2024-09-11T07:26:35.561096+0000","last_became_peered":"2024-09-11T07:26:35.561096+0000","last_unstale":"2024-09-11T07:28:08.012014+0000","last_undegraded":"2024-09-11T07:28:08.012014+0000","last_fullsized":"2024-09-11T07:28:08.012014+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.648420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:05.663592+0000","scrub_duration":0.001391446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:19.063784+0000","last_change":"2024-09-11T07:28:19.063739+0000","last_active":"2024-09-11T07:28:19.063784+0000","last_peered":"2024-09-11T07:28:19.063784+0000","last_clean":"2024-09-11T07:28:19.063784+0000","last_became_active":"2024-09-11T07:26:37.566910+0000","last_became_peered":"2024-09-11T07:26:37.566910+0000","last_unstale":"2024-09-11T07:28:19.063784+0000","last_undegraded":"2024-09-11T07:28:19.063784+0000","last_fullsized":"2024-09-11T07:28:19.063784+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:19.063651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:19.063651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:42.984286+0000","scrub_duration":0.0019504990000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:23.003116+0000","last_change":"2024-09-11T07:28:23.003084+0000","last_active":"2024-09-11T07:28:23.003116+0000","last_peered":"2024-09-11T07:28:23.003116+0000","last_clean":"2024-09-11T07:28:23.003116+0000","last_became_active":"2024-09-11T07:26:31.422772+0000","last_became_peered":"2024-09-11T07:26:31.422772+0000","last_unstale":"2024-09-11T07:28:23.003116+0000","last_undegraded":"2024-09-11T07:28:23.003116+0000","last_fullsized":"2024-09-11T07:28:23.003116+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:23.003028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:23.003028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.008955+0000","scrub_duration":0.0013267520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"82'7","reported_seq":58,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008514+0000","last_change":"2024-09-11T07:27:52.097170+0000","last_active":"2024-09-11T07:28:08.008514+0000","last_peered":"2024-09-11T07:28:08.008514+0000","last_clean":"2024-09-11T07:28:08.008514+0000","last_became_active":"2024-09-11T07:26:33.540351+0000","last_became_peered":"2024-09-11T07:26:33.540351+0000","last_unstale":"2024-09-11T07:28:08.008514+0000","last_undegraded":"2024-09-11T07:28:08.008514+0000","last_fullsized":"2024-09-11T07:28:08.008514+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'7","last_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_deep_scrub":"82'7","last_deep_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_clean_scrub_stamp":"2024-09-11T07:27:52.097071+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:53.905741+0000","scrub_duration":0.0025786670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014667+0000","last_change":"2024-09-11T07:26:35.563858+0000","last_active":"2024-09-11T07:28:08.014667+0000","last_peered":"2024-09-11T07:28:08.014667+0000","last_clean":"2024-09-11T07:28:08.014667+0000","last_became_active":"2024-09-11T07:26:35.563536+0000","last_became_peered":"2024-09-11T07:26:35.563536+0000","last_unstale":"2024-09-11T07:28:08.014667+0000","last_undegraded":"2024-09-11T07:28:08.014667+0000","last_fullsized":"2024-09-11T07:28:08.014667+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011217+0000","last_change":"2024-09-11T07:26:37.562008+0000","last_active":"2024-09-11T07:28:08.011217+0000","last_peered":"2024-09-11T07:28:08.011217+0000","last_clean":"2024-09-11T07:28:08.011217+0000","last_became_active":"2024-09-11T07:26:37.561758+0000","last_became_peered":"2024-09-11T07:26:37.561758+0000","last_unstale":"2024-09-11T07:28:08.011217+0000","last_undegraded":"2024-09-11T07:28:08.011217+0000","last_fullsized":"2024-09-11T07:28:08.011217+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.015134+0000","last_change":"2024-09-11T07:27:52.058013+0000","last_active":"2024-09-11T07:28:08.015134+0000","last_peered":"2024-09-11T07:28:08.015134+0000","last_clean":"2024-09-11T07:28:08.015134+0000","last_became_active":"2024-09-11T07:26:31.420969+0000","last_became_peered":"2024-09-11T07:26:31.420969+0000","last_unstale":"2024-09-11T07:28:08.015134+0000","last_undegraded":"2024-09-11T07:28:08.015134+0000","last_fullsized":"2024-09-11T07:28:08.015134+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_clean_scrub_stamp":"2024-09-11T07:27:52.057959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:12.902027+0000","scrub_duration":0.0014644840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"64'9","reported_seq":62,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008543+0000","last_change":"2024-09-11T07:27:47.108704+0000","last_active":"2024-09-11T07:28:08.008543+0000","last_peered":"2024-09-11T07:28:08.008543+0000","last_clean":"2024-09-11T07:28:08.008543+0000","last_became_active":"2024-09-11T07:26:33.544386+0000","last_became_peered":"2024-09-11T07:26:33.544386+0000","last_unstale":"2024-09-11T07:28:08.008543+0000","last_undegraded":"2024-09-11T07:28:08.008543+0000","last_fullsized":"2024-09-11T07:28:08.008543+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:27:47.108645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.108645+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:08.302050+0000","scrub_duration":0.0024650560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011669+0000","last_change":"2024-09-11T07:27:57.092029+0000","last_active":"2024-09-11T07:28:08.011669+0000","last_peered":"2024-09-11T07:28:08.011669+0000","last_clean":"2024-09-11T07:28:08.011669+0000","last_became_active":"2024-09-11T07:26:35.569630+0000","last_became_peered":"2024-09-11T07:26:35.569630+0000","last_unstale":"2024-09-11T07:28:08.011669+0000","last_undegraded":"2024-09-11T07:28:08.011669+0000","last_fullsized":"2024-09-11T07:28:08.011669+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:57.091958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.091958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:57.303120+0000","scrub_duration":0.001785872,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.333896+0000","last_change":"2024-09-11T07:28:09.333866+0000","last_active":"2024-09-11T07:28:09.333896+0000","last_peered":"2024-09-11T07:28:09.333896+0000","last_clean":"2024-09-11T07:28:09.333896+0000","last_became_active":"2024-09-11T07:26:37.565475+0000","last_became_peered":"2024-09-11T07:26:37.565475+0000","last_unstale":"2024-09-11T07:28:09.333896+0000","last_undegraded":"2024-09-11T07:28:09.333896+0000","last_fullsized":"2024-09-11T07:28:09.333896+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:09.333804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:09.333804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:11.385070+0000","scrub_duration":0.003498438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":102,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.027694+0000","last_change":"2024-09-11T07:28:09.027694+0000","last_active":"2024-09-11T07:28:09.027694+0000","last_peered":"2024-09-11T07:28:09.027694+0000","last_clean":"2024-09-11T07:28:09.027694+0000","last_became_active":"2024-09-11T07:28:09.018213+0000","last_became_peered":"2024-09-11T07:28:09.018213+0000","last_unstale":"2024-09-11T07:28:09.027694+0000","last_undegraded":"2024-09-11T07:28:09.027694+0000","last_fullsized":"2024-09-11T07:28:09.027694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-09-11T07:28:04.137453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:04.137453+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:05.572290+0000","scrub_duration":0.001914083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:11.897419+0000","last_change":"2024-09-11T07:28:11.897385+0000","last_active":"2024-09-11T07:28:11.897419+0000","last_peered":"2024-09-11T07:28:11.897419+0000","last_clean":"2024-09-11T07:28:11.897419+0000","last_became_active":"2024-09-11T07:26:31.419488+0000","last_became_peered":"2024-09-11T07:26:31.419488+0000","last_unstale":"2024-09-11T07:28:11.897419+0000","last_undegraded":"2024-09-11T07:28:11.897419+0000","last_fullsized":"2024-09-11T07:28:11.897419+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:11.897319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:11.897319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:18.864403+0000","scrub_duration":0.0013939480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012787+0000","last_change":"2024-09-11T07:26:35.559684+0000","last_active":"2024-09-11T07:28:08.012787+0000","last_peered":"2024-09-11T07:28:08.012787+0000","last_clean":"2024-09-11T07:28:08.012787+0000","last_became_active":"2024-09-11T07:26:35.557989+0000","last_became_peered":"2024-09-11T07:26:35.557989+0000","last_unstale":"2024-09-11T07:28:08.012787+0000","last_undegraded":"2024-09-11T07:28:08.012787+0000","last_fullsized":"2024-09-11T07:28:08.012787+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008505+0000","last_change":"2024-09-11T07:27:57.274611+0000","last_active":"2024-09-11T07:28:08.008505+0000","last_peered":"2024-09-11T07:28:08.008505+0000","last_clean":"2024-09-11T07:28:08.008505+0000","last_became_active":"2024-09-11T07:26:37.564554+0000","last_became_peered":"2024-09-11T07:26:37.564554+0000","last_unstale":"2024-09-11T07:28:08.008505+0000","last_undegraded":"2024-09-11T07:28:08.008505+0000","last_fullsized":"2024-09-11T07:28:08.008505+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:57.274554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.274554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.058396+0000","scrub_duration":0.001419584,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":61,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:21.039970+0000","last_change":"2024-09-11T07:28:21.039929+0000","last_active":"2024-09-11T07:28:21.039970+0000","last_peered":"2024-09-11T07:28:21.039970+0000","last_clean":"2024-09-11T07:28:21.039970+0000","last_became_active":"2024-09-11T07:26:33.535411+0000","last_became_peered":"2024-09-11T07:26:33.535411+0000","last_unstale":"2024-09-11T07:28:21.039970+0000","last_undegraded":"2024-09-11T07:28:21.039970+0000","last_fullsized":"2024-09-11T07:28:21.039970+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:28:21.039828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:21.039828+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:24.128795+0000","scrub_duration":0.0021618829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:18.878037+0000","last_change":"2024-09-11T07:28:18.878004+0000","last_active":"2024-09-11T07:28:18.878037+0000","last_peered":"2024-09-11T07:28:18.878037+0000","last_clean":"2024-09-11T07:28:18.878037+0000","last_became_active":"2024-09-11T07:26:31.418178+0000","last_became_peered":"2024-09-11T07:26:31.418178+0000","last_unstale":"2024-09-11T07:28:18.878037+0000","last_undegraded":"2024-09-11T07:28:18.878037+0000","last_fullsized":"2024-09-11T07:28:18.878037+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:18.877940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:18.877940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:21.852832+0000","scrub_duration":0.001611924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"64'8","reported_seq":74,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:14.502184+0000","last_change":"2024-09-11T07:27:47.127485+0000","last_active":"2024-09-11T07:28:14.502184+0000","last_peered":"2024-09-11T07:28:14.502184+0000","last_clean":"2024-09-11T07:28:14.502184+0000","last_became_active":"2024-09-11T07:26:35.564531+0000","last_became_peered":"2024-09-11T07:26:35.564531+0000","last_unstale":"2024-09-11T07:28:14.502184+0000","last_undegraded":"2024-09-11T07:28:14.502184+0000","last_fullsized":"2024-09-11T07:28:14.502184+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:27:47.127428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.127428+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:58.343731+0000","scrub_duration":0.0014091329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008839+0000","last_change":"2024-09-11T07:26:37.567193+0000","last_active":"2024-09-11T07:28:08.008839+0000","last_peered":"2024-09-11T07:28:08.008839+0000","last_clean":"2024-09-11T07:28:08.008839+0000","last_became_active":"2024-09-11T07:26:37.566919+0000","last_became_peered":"2024-09-11T07:26:37.566919+0000","last_unstale":"2024-09-11T07:28:08.008839+0000","last_undegraded":"2024-09-11T07:28:08.008839+0000","last_fullsized":"2024-09-11T07:28:08.008839+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"83'22","reported_seq":78,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.013407+0000","last_change":"2024-09-11T07:28:01.719185+0000","last_active":"2024-09-11T07:28:08.013407+0000","last_peered":"2024-09-11T07:28:08.013407+0000","last_clean":"2024-09-11T07:28:08.013407+0000","last_became_active":"2024-09-11T07:26:33.549733+0000","last_became_peered":"2024-09-11T07:26:33.549733+0000","last_unstale":"2024-09-11T07:28:08.013407+0000","last_undegraded":"2024-09-11T07:28:08.013407+0000","last_fullsized":"2024-09-11T07:28:08.013407+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'22","last_scrub_stamp":"2024-09-11T07:28:01.719128+0000","last_deep_scrub":"83'22","last_deep_scrub_stamp":"2024-09-11T07:28:01.719128+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.719128+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:27.431086+0000","scrub_duration":0.0047817240000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012176+0000","last_change":"2024-09-11T07:27:35.589317+0000","last_active":"2024-09-11T07:28:08.012176+0000","last_peered":"2024-09-11T07:28:08.012176+0000","last_clean":"2024-09-11T07:28:08.012176+0000","last_became_active":"2024-09-11T07:26:31.418478+0000","last_became_peered":"2024-09-11T07:26:31.418478+0000","last_unstale":"2024-09-11T07:28:08.012176+0000","last_undegraded":"2024-09-11T07:28:08.012176+0000","last_fullsized":"2024-09-11T07:28:08.012176+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:35.589251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:35.589251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:01.839680+0000","scrub_duration":0.0014929380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":77,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:14.502317+0000","last_change":"2024-09-11T07:27:56.218355+0000","last_active":"2024-09-11T07:28:14.502317+0000","last_peered":"2024-09-11T07:28:14.502317+0000","last_clean":"2024-09-11T07:28:14.502317+0000","last_became_active":"2024-09-11T07:26:35.560406+0000","last_became_peered":"2024-09-11T07:26:35.560406+0000","last_unstale":"2024-09-11T07:28:14.502317+0000","last_undegraded":"2024-09-11T07:28:14.502317+0000","last_fullsized":"2024-09-11T07:28:14.502317+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:27:56.218301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:56.218301+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:21.374837+0000","scrub_duration":0.001687413,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:18.785767+0000","last_change":"2024-09-11T07:28:18.785729+0000","last_active":"2024-09-11T07:28:18.785767+0000","last_peered":"2024-09-11T07:28:18.785767+0000","last_clean":"2024-09-11T07:28:18.785767+0000","last_became_active":"2024-09-11T07:26:37.568005+0000","last_became_peered":"2024-09-11T07:26:37.568005+0000","last_unstale":"2024-09-11T07:28:18.785767+0000","last_undegraded":"2024-09-11T07:28:18.785767+0000","last_fullsized":"2024-09-11T07:28:18.785767+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:18.785667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:18.785667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:39.054254+0000","scrub_duration":0.0013564829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"64'11","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.009008+0000","last_change":"2024-09-11T07:27:58.158485+0000","last_active":"2024-09-11T07:28:08.009008+0000","last_peered":"2024-09-11T07:28:08.009008+0000","last_clean":"2024-09-11T07:28:08.009008+0000","last_became_active":"2024-09-11T07:26:33.540777+0000","last_became_peered":"2024-09-11T07:26:33.540777+0000","last_unstale":"2024-09-11T07:28:08.009008+0000","last_undegraded":"2024-09-11T07:28:08.009008+0000","last_fullsized":"2024-09-11T07:28:08.009008+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:27:58.158420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.158420+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.832014+0000","scrub_duration":0.0021737459999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014438+0000","last_change":"2024-09-11T07:27:34.932151+0000","last_active":"2024-09-11T07:28:08.014438+0000","last_peered":"2024-09-11T07:28:08.014438+0000","last_clean":"2024-09-11T07:28:08.014438+0000","last_became_active":"2024-09-11T07:26:31.418238+0000","last_became_peered":"2024-09-11T07:26:31.418238+0000","last_unstale":"2024-09-11T07:28:08.014438+0000","last_undegraded":"2024-09-11T07:28:08.014438+0000","last_fullsized":"2024-09-11T07:28:08.014438+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:34.932092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:34.932092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:46.904196+0000","scrub_duration":0.002710259,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012589+0000","last_change":"2024-09-11T07:28:00.005119+0000","last_active":"2024-09-11T07:28:08.012589+0000","last_peered":"2024-09-11T07:28:08.012589+0000","last_clean":"2024-09-11T07:28:08.012589+0000","last_became_active":"2024-09-11T07:26:35.569081+0000","last_became_peered":"2024-09-11T07:26:35.569081+0000","last_unstale":"2024-09-11T07:28:08.012589+0000","last_undegraded":"2024-09-11T07:28:08.012589+0000","last_fullsized":"2024-09-11T07:28:08.012589+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:00.005031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.005031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:10.748420+0000","scrub_duration":0.001403533,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.009787+0000","last_change":"2024-09-11T07:27:46.752272+0000","last_active":"2024-09-11T07:28:08.009787+0000","last_peered":"2024-09-11T07:28:08.009787+0000","last_clean":"2024-09-11T07:28:08.009787+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:28:08.009787+0000","last_undegraded":"2024-09-11T07:28:08.009787+0000","last_fullsized":"2024-09-11T07:28:08.009787+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:46.752181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:46.752181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.072860+0000","scrub_duration":0.001508782,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"64'14","reported_seq":67,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:20.085437+0000","last_change":"2024-09-11T07:28:20.085403+0000","last_active":"2024-09-11T07:28:20.085437+0000","last_peered":"2024-09-11T07:28:20.085437+0000","last_clean":"2024-09-11T07:28:20.085437+0000","last_became_active":"2024-09-11T07:26:33.537850+0000","last_became_peered":"2024-09-11T07:26:33.537850+0000","last_unstale":"2024-09-11T07:28:20.085437+0000","last_undegraded":"2024-09-11T07:28:20.085437+0000","last_fullsized":"2024-09-11T07:28:20.085437+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-09-11T07:28:20.085339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:20.085339+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:27.624554+0000","scrub_duration":0.0019854640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":58,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.015415+0000","last_change":"2024-09-11T07:28:09.015415+0000","last_active":"2024-09-11T07:28:09.015415+0000","last_peered":"2024-09-11T07:28:09.015415+0000","last_clean":"2024-09-11T07:28:09.015415+0000","last_became_active":"2024-09-11T07:28:09.014995+0000","last_became_peered":"2024-09-11T07:28:09.014995+0000","last_unstale":"2024-09-11T07:28:09.015415+0000","last_undegraded":"2024-09-11T07:28:09.015415+0000","last_fullsized":"2024-09-11T07:28:09.015415+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:39.014840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:39.014840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:56.130768+0000","scrub_duration":0.0032789239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"84'9","reported_seq":75,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:14.502186+0000","last_change":"2024-09-11T07:28:05.653289+0000","last_active":"2024-09-11T07:28:14.502186+0000","last_peered":"2024-09-11T07:28:14.502186+0000","last_clean":"2024-09-11T07:28:14.502186+0000","last_became_active":"2024-09-11T07:26:35.562997+0000","last_became_peered":"2024-09-11T07:26:35.562997+0000","last_unstale":"2024-09-11T07:28:14.502186+0000","last_undegraded":"2024-09-11T07:28:14.502186+0000","last_fullsized":"2024-09-11T07:28:14.502186+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-11T07:28:05.653237+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-11T07:28:05.653237+0000","last_clean_scrub_stamp":"2024-09-11T07:28:05.653237+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:22.713095+0000","scrub_duration":0.0027957030000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014528+0000","last_change":"2024-09-11T07:26:37.862564+0000","last_active":"2024-09-11T07:28:08.014528+0000","last_peered":"2024-09-11T07:28:08.014528+0000","last_clean":"2024-09-11T07:28:08.014528+0000","last_became_active":"2024-09-11T07:26:37.862246+0000","last_became_peered":"2024-09-11T07:26:37.862246+0000","last_unstale":"2024-09-11T07:28:08.014528+0000","last_undegraded":"2024-09-11T07:28:08.014528+0000","last_fullsized":"2024-09-11T07:28:08.014528+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"64'18","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008309+0000","last_change":"2024-09-11T07:26:33.546711+0000","last_active":"2024-09-11T07:28:08.008309+0000","last_peered":"2024-09-11T07:28:08.008309+0000","last_clean":"2024-09-11T07:28:08.008309+0000","last_became_active":"2024-09-11T07:26:33.546188+0000","last_became_peered":"2024-09-11T07:26:33.546188+0000","last_unstale":"2024-09-11T07:28:08.008309+0000","last_undegraded":"2024-09-11T07:28:08.008309+0000","last_fullsized":"2024-09-11T07:28:08.008309+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.013334+0000","last_change":"2024-09-11T07:26:31.421802+0000","last_active":"2024-09-11T07:28:08.013334+0000","last_peered":"2024-09-11T07:28:08.013334+0000","last_clean":"2024-09-11T07:28:08.013334+0000","last_became_active":"2024-09-11T07:26:31.421500+0000","last_became_peered":"2024-09-11T07:26:31.421500+0000","last_unstale":"2024-09-11T07:28:08.013334+0000","last_undegraded":"2024-09-11T07:28:08.013334+0000","last_fullsized":"2024-09-11T07:28:08.013334+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008249+0000","last_change":"2024-09-11T07:26:35.562816+0000","last_active":"2024-09-11T07:28:08.008249+0000","last_peered":"2024-09-11T07:28:08.008249+0000","last_clean":"2024-09-11T07:28:08.008249+0000","last_became_active":"2024-09-11T07:26:35.561284+0000","last_became_peered":"2024-09-11T07:26:35.561284+0000","last_unstale":"2024-09-11T07:28:08.008249+0000","last_undegraded":"2024-09-11T07:28:08.008249+0000","last_fullsized":"2024-09-11T07:28:08.008249+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008495+0000","last_change":"2024-09-11T07:26:37.565014+0000","last_active":"2024-09-11T07:28:08.008495+0000","last_peered":"2024-09-11T07:28:08.008495+0000","last_clean":"2024-09-11T07:28:08.008495+0000","last_became_active":"2024-09-11T07:26:37.564616+0000","last_became_peered":"2024-09-11T07:26:37.564616+0000","last_unstale":"2024-09-11T07:28:08.008495+0000","last_undegraded":"2024-09-11T07:28:08.008495+0000","last_fullsized":"2024-09-11T07:28:08.008495+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":82,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:12.466273+0000","last_change":"2024-09-11T07:28:12.466246+0000","last_active":"2024-09-11T07:28:12.466273+0000","last_peered":"2024-09-11T07:28:12.466273+0000","last_clean":"2024-09-11T07:28:12.466273+0000","last_became_active":"2024-09-11T07:26:33.540639+0000","last_became_peered":"2024-09-11T07:26:33.540639+0000","last_unstale":"2024-09-11T07:28:12.466273+0000","last_undegraded":"2024-09-11T07:28:12.466273+0000","last_fullsized":"2024-09-11T07:28:12.466273+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-11T07:28:12.466179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:12.466179+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:12.523827+0000","scrub_duration":0.0027522670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:12.896635+0000","last_change":"2024-09-11T07:28:12.896603+0000","last_active":"2024-09-11T07:28:12.896635+0000","last_peered":"2024-09-11T07:28:12.896635+0000","last_clean":"2024-09-11T07:28:12.896635+0000","last_became_active":"2024-09-11T07:26:31.419185+0000","last_became_peered":"2024-09-11T07:26:31.419185+0000","last_unstale":"2024-09-11T07:28:12.896635+0000","last_undegraded":"2024-09-11T07:28:12.896635+0000","last_fullsized":"2024-09-11T07:28:12.896635+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:12.896538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:12.896538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.577764+0000","scrub_duration":0.0017083070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011053+0000","last_change":"2024-09-11T07:27:41.000469+0000","last_active":"2024-09-11T07:28:08.011053+0000","last_peered":"2024-09-11T07:28:08.011053+0000","last_clean":"2024-09-11T07:28:08.011053+0000","last_became_active":"2024-09-11T07:26:35.565359+0000","last_became_peered":"2024-09-11T07:26:35.565359+0000","last_unstale":"2024-09-11T07:28:08.011053+0000","last_undegraded":"2024-09-11T07:28:08.011053+0000","last_fullsized":"2024-09-11T07:28:08.011053+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:41.000401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:41.000401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.049825+0000","scrub_duration":0.001655164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011937+0000","last_change":"2024-09-11T07:26:37.861736+0000","last_active":"2024-09-11T07:28:08.011937+0000","last_peered":"2024-09-11T07:28:08.011937+0000","last_clean":"2024-09-11T07:28:08.011937+0000","last_became_active":"2024-09-11T07:26:37.861279+0000","last_became_peered":"2024-09-11T07:26:37.861279+0000","last_unstale":"2024-09-11T07:28:08.011937+0000","last_undegraded":"2024-09-11T07:28:08.011937+0000","last_fullsized":"2024-09-11T07:28:08.011937+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":61,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012533+0000","last_change":"2024-09-11T07:27:51.642529+0000","last_active":"2024-09-11T07:28:08.012533+0000","last_peered":"2024-09-11T07:28:08.012533+0000","last_clean":"2024-09-11T07:28:08.012533+0000","last_became_active":"2024-09-11T07:26:33.543063+0000","last_became_peered":"2024-09-11T07:26:33.543063+0000","last_unstale":"2024-09-11T07:28:08.012533+0000","last_undegraded":"2024-09-11T07:28:08.012533+0000","last_fullsized":"2024-09-11T07:28:08.012533+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:27:51.642464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:51.642464+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:13.146284+0000","scrub_duration":0.0020211439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":55,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012985+0000","last_change":"2024-09-11T07:27:41.618362+0000","last_active":"2024-09-11T07:28:08.012985+0000","last_peered":"2024-09-11T07:28:08.012985+0000","last_clean":"2024-09-11T07:28:08.012985+0000","last_became_active":"2024-09-11T07:26:31.422110+0000","last_became_peered":"2024-09-11T07:26:31.422110+0000","last_unstale":"2024-09-11T07:28:08.012985+0000","last_undegraded":"2024-09-11T07:28:08.012985+0000","last_fullsized":"2024-09-11T07:28:08.012985+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:27:41.618296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:41.618296+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:57.404591+0000","scrub_duration":0.0020650450000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":74,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:14.502184+0000","last_change":"2024-09-11T07:28:03.183056+0000","last_active":"2024-09-11T07:28:14.502184+0000","last_peered":"2024-09-11T07:28:14.502184+0000","last_clean":"2024-09-11T07:28:14.502184+0000","last_became_active":"2024-09-11T07:26:35.560422+0000","last_became_peered":"2024-09-11T07:26:35.560422+0000","last_unstale":"2024-09-11T07:28:14.502184+0000","last_undegraded":"2024-09-11T07:28:14.502184+0000","last_fullsized":"2024-09-11T07:28:14.502184+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:28:03.182993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:03.182993+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:26.896461+0000","scrub_duration":0.0018949889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.013563+0000","last_change":"2024-09-11T07:27:54.148594+0000","last_active":"2024-09-11T07:28:08.013563+0000","last_peered":"2024-09-11T07:28:08.013563+0000","last_clean":"2024-09-11T07:28:08.013563+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:28:08.013563+0000","last_undegraded":"2024-09-11T07:28:08.013563+0000","last_fullsized":"2024-09-11T07:28:08.013563+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.148534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:02.854448+0000","scrub_duration":0.0013536620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"64'28","reported_seq":116,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:14.502205+0000","last_change":"2024-09-11T07:28:09.770058+0000","last_active":"2024-09-11T07:28:14.502205+0000","last_peered":"2024-09-11T07:28:14.502205+0000","last_clean":"2024-09-11T07:28:14.502205+0000","last_became_active":"2024-09-11T07:26:33.549234+0000","last_became_peered":"2024-09-11T07:26:33.549234+0000","last_unstale":"2024-09-11T07:28:14.502205+0000","last_undegraded":"2024-09-11T07:28:14.502205+0000","last_fullsized":"2024-09-11T07:28:14.502205+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'28","last_scrub_stamp":"2024-09-11T07:28:09.769992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:09.769992+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:37.423717+0000","scrub_duration":0.002388463,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:12.775168+0000","last_change":"2024-09-11T07:28:12.775128+0000","last_active":"2024-09-11T07:28:12.775168+0000","last_peered":"2024-09-11T07:28:12.775168+0000","last_clean":"2024-09-11T07:28:12.775168+0000","last_became_active":"2024-09-11T07:26:31.424618+0000","last_became_peered":"2024-09-11T07:26:31.424618+0000","last_unstale":"2024-09-11T07:28:12.775168+0000","last_undegraded":"2024-09-11T07:28:12.775168+0000","last_fullsized":"2024-09-11T07:28:12.775168+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:12.775067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:12.775067+0000","last_clean_scrub_stamp":"2024-09-11T07:28:12.775067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:21.184139+0000","scrub_duration":0.0015206589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:15.439156+0000","last_change":"2024-09-11T07:28:15.439112+0000","last_active":"2024-09-11T07:28:15.439156+0000","last_peered":"2024-09-11T07:28:15.439156+0000","last_clean":"2024-09-11T07:28:15.439156+0000","last_became_active":"2024-09-11T07:26:35.560363+0000","last_became_peered":"2024-09-11T07:26:35.560363+0000","last_unstale":"2024-09-11T07:28:15.439156+0000","last_undegraded":"2024-09-11T07:28:15.439156+0000","last_fullsized":"2024-09-11T07:28:15.439156+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:15.439020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:15.439020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:29.665525+0000","scrub_duration":0.001542844,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.009101+0000","last_change":"2024-09-11T07:28:00.686634+0000","last_active":"2024-09-11T07:28:08.009101+0000","last_peered":"2024-09-11T07:28:08.009101+0000","last_clean":"2024-09-11T07:28:08.009101+0000","last_became_active":"2024-09-11T07:26:37.562452+0000","last_became_peered":"2024-09-11T07:26:37.562452+0000","last_unstale":"2024-09-11T07:28:08.009101+0000","last_undegraded":"2024-09-11T07:28:08.009101+0000","last_fullsized":"2024-09-11T07:28:08.009101+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:00.686571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.686571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.798861+0000","scrub_duration":0.0016714309999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"64'13","reported_seq":63,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.013413+0000","last_change":"2024-09-11T07:26:33.547483+0000","last_active":"2024-09-11T07:28:08.013413+0000","last_peered":"2024-09-11T07:28:08.013413+0000","last_clean":"2024-09-11T07:28:08.013413+0000","last_became_active":"2024-09-11T07:26:33.546126+0000","last_became_peered":"2024-09-11T07:26:33.546126+0000","last_unstale":"2024-09-11T07:28:08.013413+0000","last_undegraded":"2024-09-11T07:28:08.013413+0000","last_fullsized":"2024-09-11T07:28:08.013413+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":162,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:22.759588+0000","last_change":"2024-09-11T07:27:59.678930+0000","last_active":"2024-09-11T07:28:22.759588+0000","last_peered":"2024-09-11T07:28:22.759588+0000","last_clean":"2024-09-11T07:28:22.759588+0000","last_became_active":"2024-09-11T07:26:31.421276+0000","last_became_peered":"2024-09-11T07:26:31.421276+0000","last_unstale":"2024-09-11T07:28:22.759588+0000","last_undegraded":"2024-09-11T07:28:22.759588+0000","last_fullsized":"2024-09-11T07:28:22.759588+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'5","last_scrub_stamp":"2024-09-11T07:27:59.678865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:59.678865+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.114929+0000","scrub_duration":0.001792486,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":106,"num_read_kb":101,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.009362+0000","last_change":"2024-09-11T07:28:07.327716+0000","last_active":"2024-09-11T07:28:08.009362+0000","last_peered":"2024-09-11T07:28:08.009362+0000","last_clean":"2024-09-11T07:28:08.009362+0000","last_became_active":"2024-09-11T07:26:31.413354+0000","last_became_peered":"2024-09-11T07:26:31.413354+0000","last_unstale":"2024-09-11T07:28:08.009362+0000","last_undegraded":"2024-09-11T07:28:08.009362+0000","last_fullsized":"2024-09-11T07:28:08.009362+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:07.327648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:07.327648+0000","last_clean_scrub_stamp":"2024-09-11T07:28:07.327648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:35.306829+0000","scrub_duration":0.0011895899999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008446+0000","last_change":"2024-09-11T07:27:42.141156+0000","last_active":"2024-09-11T07:28:08.008446+0000","last_peered":"2024-09-11T07:28:08.008446+0000","last_clean":"2024-09-11T07:28:08.008446+0000","last_became_active":"2024-09-11T07:26:35.565849+0000","last_became_peered":"2024-09-11T07:26:35.565849+0000","last_unstale":"2024-09-11T07:28:08.008446+0000","last_undegraded":"2024-09-11T07:28:08.008446+0000","last_fullsized":"2024-09-11T07:28:08.008446+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.141100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.141100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.707826+0000","scrub_duration":0.001495672,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014975+0000","last_change":"2024-09-11T07:27:55.018180+0000","last_active":"2024-09-11T07:28:08.014975+0000","last_peered":"2024-09-11T07:28:08.014975+0000","last_clean":"2024-09-11T07:28:08.014975+0000","last_became_active":"2024-09-11T07:26:37.861292+0000","last_became_peered":"2024-09-11T07:26:37.861292+0000","last_unstale":"2024-09-11T07:28:08.014975+0000","last_undegraded":"2024-09-11T07:28:08.014975+0000","last_fullsized":"2024-09-11T07:28:08.014975+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:55.018125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:55.018125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.234213+0000","scrub_duration":0.0013534339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":88,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.025399+0000","last_change":"2024-09-11T07:28:09.025399+0000","last_active":"2024-09-11T07:28:09.025399+0000","last_peered":"2024-09-11T07:28:09.025399+0000","last_clean":"2024-09-11T07:28:09.025399+0000","last_became_active":"2024-09-11T07:28:09.018134+0000","last_became_peered":"2024-09-11T07:28:09.018134+0000","last_unstale":"2024-09-11T07:28:09.025399+0000","last_undegraded":"2024-09-11T07:28:09.025399+0000","last_fullsized":"2024-09-11T07:28:09.025399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-09-11T07:27:34.213870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:34.213870+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:46.533561+0000","scrub_duration":0.0026667679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":54,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.286887+0000","last_change":"2024-09-11T07:28:08.286847+0000","last_active":"2024-09-11T07:28:08.286887+0000","last_peered":"2024-09-11T07:28:08.286887+0000","last_clean":"2024-09-11T07:28:08.286887+0000","last_became_active":"2024-09-11T07:26:31.420320+0000","last_became_peered":"2024-09-11T07:26:31.420320+0000","last_unstale":"2024-09-11T07:28:08.286887+0000","last_undegraded":"2024-09-11T07:28:08.286887+0000","last_fullsized":"2024-09-11T07:28:08.286887+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-11T07:28:08.286788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:08.286788+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.046248+0000","scrub_duration":0.001825976,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:16.967571+0000","last_change":"2024-09-11T07:28:16.967543+0000","last_active":"2024-09-11T07:28:16.967571+0000","last_peered":"2024-09-11T07:28:16.967571+0000","last_clean":"2024-09-11T07:28:16.967571+0000","last_became_active":"2024-09-11T07:26:31.417455+0000","last_became_peered":"2024-09-11T07:26:31.417455+0000","last_unstale":"2024-09-11T07:28:16.967571+0000","last_undegraded":"2024-09-11T07:28:16.967571+0000","last_fullsized":"2024-09-11T07:28:16.967571+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:16.967492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:16.967492+0000","last_clean_scrub_stamp":"2024-09-11T07:28:16.967492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:29.710608+0000","scrub_duration":0.0012319799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:21.868098+0000","last_change":"2024-09-11T07:28:21.868074+0000","last_active":"2024-09-11T07:28:21.868098+0000","last_peered":"2024-09-11T07:28:21.868098+0000","last_clean":"2024-09-11T07:28:21.868098+0000","last_became_active":"2024-09-11T07:26:35.564178+0000","last_became_peered":"2024-09-11T07:26:35.564178+0000","last_unstale":"2024-09-11T07:28:21.868098+0000","last_undegraded":"2024-09-11T07:28:21.868098+0000","last_fullsized":"2024-09-11T07:28:21.868098+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:21.868020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:21.868020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:40.076469+0000","scrub_duration":0.0013323009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012691+0000","last_change":"2024-09-11T07:27:54.621934+0000","last_active":"2024-09-11T07:28:08.012691+0000","last_peered":"2024-09-11T07:28:08.012691+0000","last_clean":"2024-09-11T07:28:08.012691+0000","last_became_active":"2024-09-11T07:26:37.561378+0000","last_became_peered":"2024-09-11T07:26:37.561378+0000","last_unstale":"2024-09-11T07:28:08.012691+0000","last_undegraded":"2024-09-11T07:28:08.012691+0000","last_fullsized":"2024-09-11T07:28:08.012691+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.621874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.621874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:07.014230+0000","scrub_duration":0.0013604750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011749+0000","last_change":"2024-09-11T07:27:49.061728+0000","last_active":"2024-09-11T07:28:08.011749+0000","last_peered":"2024-09-11T07:28:08.011749+0000","last_clean":"2024-09-11T07:28:08.011749+0000","last_became_active":"2024-09-11T07:26:33.541359+0000","last_became_peered":"2024-09-11T07:26:33.541359+0000","last_unstale":"2024-09-11T07:28:08.011749+0000","last_undegraded":"2024-09-11T07:28:08.011749+0000","last_fullsized":"2024-09-11T07:28:08.011749+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-09-11T07:27:49.061670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.061670+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:51.283390+0000","scrub_duration":0.001809801,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'3","reported_seq":56,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.009267+0000","last_change":"2024-09-11T07:28:01.711301+0000","last_active":"2024-09-11T07:28:08.009267+0000","last_peered":"2024-09-11T07:28:08.009267+0000","last_clean":"2024-09-11T07:28:08.009267+0000","last_became_active":"2024-09-11T07:26:31.419501+0000","last_became_peered":"2024-09-11T07:26:31.419501+0000","last_unstale":"2024-09-11T07:28:08.009267+0000","last_undegraded":"2024-09-11T07:28:08.009267+0000","last_fullsized":"2024-09-11T07:28:08.009267+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.711241+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:02.735271+0000","scrub_duration":0.0034627350000000002,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011920+0000","last_change":"2024-09-11T07:27:47.903204+0000","last_active":"2024-09-11T07:28:08.011920+0000","last_peered":"2024-09-11T07:28:08.011920+0000","last_clean":"2024-09-11T07:28:08.011920+0000","last_became_active":"2024-09-11T07:26:31.423304+0000","last_became_peered":"2024-09-11T07:26:31.423304+0000","last_unstale":"2024-09-11T07:28:08.011920+0000","last_undegraded":"2024-09-11T07:28:08.011920+0000","last_fullsized":"2024-09-11T07:28:08.011920+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:47.903135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.903135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:01.867608+0000","scrub_duration":0.001284908,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"72'79","reported_seq":83,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014624+0000","last_change":"2024-09-11T07:26:57.872633+0000","last_active":"2024-09-11T07:28:08.014624+0000","last_peered":"2024-09-11T07:28:08.014624+0000","last_clean":"2024-09-11T07:28:08.014624+0000","last_became_active":"2024-09-11T07:26:17.397381+0000","last_became_peered":"2024-09-11T07:26:17.397381+0000","last_unstale":"2024-09-11T07:28:08.014624+0000","last_undegraded":"2024-09-11T07:28:08.014624+0000","last_fullsized":"2024-09-11T07:28:08.014624+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'72","last_scrub_stamp":"2024-09-11T07:26:57.872570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:24:22.202219+0000","last_clean_scrub_stamp":"2024-09-11T07:26:57.872570+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015639180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014924+0000","last_change":"2024-09-11T07:27:44.036138+0000","last_active":"2024-09-11T07:28:08.014924+0000","last_peered":"2024-09-11T07:28:08.014924+0000","last_clean":"2024-09-11T07:28:08.014924+0000","last_became_active":"2024-09-11T07:26:35.564480+0000","last_became_peered":"2024-09-11T07:26:35.564480+0000","last_unstale":"2024-09-11T07:28:08.014924+0000","last_undegraded":"2024-09-11T07:28:08.014924+0000","last_fullsized":"2024-09-11T07:28:08.014924+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:44.036058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:44.036058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:48.598709+0000","scrub_duration":0.001488931,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:13.796782+0000","last_change":"2024-09-11T07:28:13.796731+0000","last_active":"2024-09-11T07:28:13.796782+0000","last_peered":"2024-09-11T07:28:13.796782+0000","last_clean":"2024-09-11T07:28:13.796782+0000","last_became_active":"2024-09-11T07:26:37.559920+0000","last_became_peered":"2024-09-11T07:26:37.559920+0000","last_unstale":"2024-09-11T07:28:13.796782+0000","last_undegraded":"2024-09-11T07:28:13.796782+0000","last_fullsized":"2024-09-11T07:28:13.796782+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:13.796657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:13.796657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:31.783767+0000","scrub_duration":0.001458984,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":77,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011287+0000","last_change":"2024-09-11T07:27:59.128292+0000","last_active":"2024-09-11T07:28:08.011287+0000","last_peered":"2024-09-11T07:28:08.011287+0000","last_clean":"2024-09-11T07:28:08.011287+0000","last_became_active":"2024-09-11T07:26:33.543687+0000","last_became_peered":"2024-09-11T07:26:33.543687+0000","last_unstale":"2024-09-11T07:28:08.011287+0000","last_undegraded":"2024-09-11T07:28:08.011287+0000","last_fullsized":"2024-09-11T07:28:08.011287+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-11T07:27:59.128237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:59.128237+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:16.835564+0000","scrub_duration":0.00199329,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011343+0000","last_change":"2024-09-11T07:27:54.182157+0000","last_active":"2024-09-11T07:28:08.011343+0000","last_peered":"2024-09-11T07:28:08.011343+0000","last_clean":"2024-09-11T07:28:08.011343+0000","last_became_active":"2024-09-11T07:26:31.419560+0000","last_became_peered":"2024-09-11T07:26:31.419560+0000","last_unstale":"2024-09-11T07:28:08.011343+0000","last_undegraded":"2024-09-11T07:28:08.011343+0000","last_fullsized":"2024-09-11T07:28:08.011343+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.182106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.182106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:04.075506+0000","scrub_duration":0.001213254,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:13.357708+0000","last_change":"2024-09-11T07:28:13.357659+0000","last_active":"2024-09-11T07:28:13.357708+0000","last_peered":"2024-09-11T07:28:13.357708+0000","last_clean":"2024-09-11T07:28:13.357708+0000","last_became_active":"2024-09-11T07:26:35.564044+0000","last_became_peered":"2024-09-11T07:26:35.564044+0000","last_unstale":"2024-09-11T07:28:13.357708+0000","last_undegraded":"2024-09-11T07:28:13.357708+0000","last_fullsized":"2024-09-11T07:28:13.357708+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:13.357544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:13.357544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:42.978875+0000","scrub_duration":0.0017127900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:13.918182+0000","last_change":"2024-09-11T07:28:13.918148+0000","last_active":"2024-09-11T07:28:13.918182+0000","last_peered":"2024-09-11T07:28:13.918182+0000","last_clean":"2024-09-11T07:28:13.918182+0000","last_became_active":"2024-09-11T07:26:37.564801+0000","last_became_peered":"2024-09-11T07:26:37.564801+0000","last_unstale":"2024-09-11T07:28:13.918182+0000","last_undegraded":"2024-09-11T07:28:13.918182+0000","last_fullsized":"2024-09-11T07:28:13.918182+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:13.918082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:13.918082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:31.000135+0000","scrub_duration":0.0014064769999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":62,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014507+0000","last_change":"2024-09-11T07:28:02.230985+0000","last_active":"2024-09-11T07:28:08.014507+0000","last_peered":"2024-09-11T07:28:08.014507+0000","last_clean":"2024-09-11T07:28:08.014507+0000","last_became_active":"2024-09-11T07:26:33.543148+0000","last_became_peered":"2024-09-11T07:26:33.543148+0000","last_unstale":"2024-09-11T07:28:08.014507+0000","last_undegraded":"2024-09-11T07:28:08.014507+0000","last_fullsized":"2024-09-11T07:28:08.014507+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:28:02.230928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.230928+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:27.244330+0000","scrub_duration":0.001888926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.013300+0000","last_change":"2024-09-11T07:27:53.659961+0000","last_active":"2024-09-11T07:28:08.013300+0000","last_peered":"2024-09-11T07:28:08.013300+0000","last_clean":"2024-09-11T07:28:08.013300+0000","last_became_active":"2024-09-11T07:26:31.423056+0000","last_became_peered":"2024-09-11T07:26:31.423056+0000","last_unstale":"2024-09-11T07:28:08.013300+0000","last_undegraded":"2024-09-11T07:28:08.013300+0000","last_fullsized":"2024-09-11T07:28:08.013300+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:53.659902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:53.659902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.781453+0000","scrub_duration":0.0011613719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:10.337633+0000","last_change":"2024-09-11T07:28:10.337599+0000","last_active":"2024-09-11T07:28:10.337633+0000","last_peered":"2024-09-11T07:28:10.337633+0000","last_clean":"2024-09-11T07:28:10.337633+0000","last_became_active":"2024-09-11T07:26:35.563182+0000","last_became_peered":"2024-09-11T07:26:35.563182+0000","last_unstale":"2024-09-11T07:28:10.337633+0000","last_undegraded":"2024-09-11T07:28:10.337633+0000","last_fullsized":"2024-09-11T07:28:10.337633+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:10.337527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:10.337527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:24.013574+0000","scrub_duration":0.001985653,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:18.504023+0000","last_change":"2024-09-11T07:28:18.503995+0000","last_active":"2024-09-11T07:28:18.504023+0000","last_peered":"2024-09-11T07:28:18.504023+0000","last_clean":"2024-09-11T07:28:18.504023+0000","last_became_active":"2024-09-11T07:26:37.565029+0000","last_became_peered":"2024-09-11T07:26:37.565029+0000","last_unstale":"2024-09-11T07:28:18.504023+0000","last_undegraded":"2024-09-11T07:28:18.504023+0000","last_fullsized":"2024-09-11T07:28:18.504023+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:18.503930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:18.503930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:36.076642+0000","scrub_duration":0.001490297,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":71,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011549+0000","last_change":"2024-09-11T07:28:07.811136+0000","last_active":"2024-09-11T07:28:08.011549+0000","last_peered":"2024-09-11T07:28:08.011549+0000","last_clean":"2024-09-11T07:28:08.011549+0000","last_became_active":"2024-09-11T07:26:33.538216+0000","last_became_peered":"2024-09-11T07:26:33.538216+0000","last_unstale":"2024-09-11T07:28:08.011549+0000","last_undegraded":"2024-09-11T07:28:08.011549+0000","last_fullsized":"2024-09-11T07:28:08.011549+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-09-11T07:28:07.811071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:07.811071+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:23.495093+0000","scrub_duration":0.0019627080000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.009267+0000","last_change":"2024-09-11T07:27:50.132629+0000","last_active":"2024-09-11T07:28:08.009267+0000","last_peered":"2024-09-11T07:28:08.009267+0000","last_clean":"2024-09-11T07:28:08.009267+0000","last_became_active":"2024-09-11T07:26:31.418490+0000","last_became_peered":"2024-09-11T07:26:31.418490+0000","last_unstale":"2024-09-11T07:28:08.009267+0000","last_undegraded":"2024-09-11T07:28:08.009267+0000","last_fullsized":"2024-09-11T07:28:08.009267+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.132552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.048681+0000","scrub_duration":0.001622807,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":64,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:14.502311+0000","last_change":"2024-09-11T07:26:35.567147+0000","last_active":"2024-09-11T07:28:14.502311+0000","last_peered":"2024-09-11T07:28:14.502311+0000","last_clean":"2024-09-11T07:28:14.502311+0000","last_became_active":"2024-09-11T07:26:35.566888+0000","last_became_peered":"2024-09-11T07:26:35.566888+0000","last_unstale":"2024-09-11T07:28:14.502311+0000","last_undegraded":"2024-09-11T07:28:14.502311+0000","last_fullsized":"2024-09-11T07:28:14.502311+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011429+0000","last_change":"2024-09-11T07:28:03.961280+0000","last_active":"2024-09-11T07:28:08.011429+0000","last_peered":"2024-09-11T07:28:08.011429+0000","last_clean":"2024-09-11T07:28:08.011429+0000","last_became_active":"2024-09-11T07:26:37.861134+0000","last_became_peered":"2024-09-11T07:26:37.861134+0000","last_unstale":"2024-09-11T07:28:08.011429+0000","last_undegraded":"2024-09-11T07:28:08.011429+0000","last_fullsized":"2024-09-11T07:28:08.011429+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:03.961220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:03.961220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:29.223995+0000","scrub_duration":0.0011943730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"83'17","reported_seq":74,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008268+0000","last_change":"2024-09-11T07:28:01.098068+0000","last_active":"2024-09-11T07:28:08.008268+0000","last_peered":"2024-09-11T07:28:08.008268+0000","last_clean":"2024-09-11T07:28:08.008268+0000","last_became_active":"2024-09-11T07:26:33.540363+0000","last_became_peered":"2024-09-11T07:26:33.540363+0000","last_unstale":"2024-09-11T07:28:08.008268+0000","last_undegraded":"2024-09-11T07:28:08.008268+0000","last_fullsized":"2024-09-11T07:28:08.008268+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'17","last_scrub_stamp":"2024-09-11T07:28:01.097981+0000","last_deep_scrub":"83'17","last_deep_scrub_stamp":"2024-09-11T07:28:01.097981+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.097981+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:26.179592+0000","scrub_duration":0.0042048720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.009183+0000","last_change":"2024-09-11T07:27:46.077857+0000","last_active":"2024-09-11T07:28:08.009183+0000","last_peered":"2024-09-11T07:28:08.009183+0000","last_clean":"2024-09-11T07:28:08.009183+0000","last_became_active":"2024-09-11T07:26:31.419718+0000","last_became_peered":"2024-09-11T07:26:31.419718+0000","last_unstale":"2024-09-11T07:28:08.009183+0000","last_undegraded":"2024-09-11T07:28:08.009183+0000","last_fullsized":"2024-09-11T07:28:08.009183+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:27:46.077786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:46.077786+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:07.735381+0000","scrub_duration":0.0019376720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008226+0000","last_change":"2024-09-11T07:27:53.209518+0000","last_active":"2024-09-11T07:28:08.008226+0000","last_peered":"2024-09-11T07:28:08.008226+0000","last_clean":"2024-09-11T07:28:08.008226+0000","last_became_active":"2024-09-11T07:26:35.561967+0000","last_became_peered":"2024-09-11T07:26:35.561967+0000","last_unstale":"2024-09-11T07:28:08.008226+0000","last_undegraded":"2024-09-11T07:28:08.008226+0000","last_fullsized":"2024-09-11T07:28:08.008226+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:53.209452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:53.209452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:23.125281+0000","scrub_duration":0.0014563029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008343+0000","last_change":"2024-09-11T07:28:02.677186+0000","last_active":"2024-09-11T07:28:08.008343+0000","last_peered":"2024-09-11T07:28:08.008343+0000","last_clean":"2024-09-11T07:28:08.008343+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:28:08.008343+0000","last_undegraded":"2024-09-11T07:28:08.008343+0000","last_fullsized":"2024-09-11T07:28:08.008343+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:02.677095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.677095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.575070+0000","scrub_duration":0.001519269,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":62,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014937+0000","last_change":"2024-09-11T07:27:40.351572+0000","last_active":"2024-09-11T07:28:08.014937+0000","last_peered":"2024-09-11T07:28:08.014937+0000","last_clean":"2024-09-11T07:28:08.014937+0000","last_became_active":"2024-09-11T07:26:33.547344+0000","last_became_peered":"2024-09-11T07:26:33.547344+0000","last_unstale":"2024-09-11T07:28:08.014937+0000","last_undegraded":"2024-09-11T07:28:08.014937+0000","last_fullsized":"2024-09-11T07:28:08.014937+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:27:40.351507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:40.351507+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.098394+0000","scrub_duration":0.0021803980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008361+0000","last_change":"2024-09-11T07:27:58.324311+0000","last_active":"2024-09-11T07:28:08.008361+0000","last_peered":"2024-09-11T07:28:08.008361+0000","last_clean":"2024-09-11T07:28:08.008361+0000","last_became_active":"2024-09-11T07:26:31.415701+0000","last_became_peered":"2024-09-11T07:26:31.415701+0000","last_unstale":"2024-09-11T07:28:08.008361+0000","last_undegraded":"2024-09-11T07:28:08.008361+0000","last_fullsized":"2024-09-11T07:28:08.008361+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:58.324255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.324255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.617500+0000","scrub_duration":0.0013365549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:12.778737+0000","last_change":"2024-09-11T07:28:12.778700+0000","last_active":"2024-09-11T07:28:12.778737+0000","last_peered":"2024-09-11T07:28:12.778737+0000","last_clean":"2024-09-11T07:28:12.778737+0000","last_became_active":"2024-09-11T07:26:35.559262+0000","last_became_peered":"2024-09-11T07:26:35.559262+0000","last_unstale":"2024-09-11T07:28:12.778737+0000","last_undegraded":"2024-09-11T07:28:12.778737+0000","last_fullsized":"2024-09-11T07:28:12.778737+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:12.778645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:12.778645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.979009+0000","scrub_duration":0.0013084870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.009416+0000","last_change":"2024-09-11T07:28:06.305563+0000","last_active":"2024-09-11T07:28:08.009416+0000","last_peered":"2024-09-11T07:28:08.009416+0000","last_clean":"2024-09-11T07:28:08.009416+0000","last_became_active":"2024-09-11T07:26:37.564084+0000","last_became_peered":"2024-09-11T07:26:37.564084+0000","last_unstale":"2024-09-11T07:28:08.009416+0000","last_undegraded":"2024-09-11T07:28:08.009416+0000","last_fullsized":"2024-09-11T07:28:08.009416+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:28:06.305502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:06.305502+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.659083+0000","scrub_duration":0.0021934649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011410+0000","last_change":"2024-09-11T07:27:45.990617+0000","last_active":"2024-09-11T07:28:08.011410+0000","last_peered":"2024-09-11T07:28:08.011410+0000","last_clean":"2024-09-11T07:28:08.011410+0000","last_became_active":"2024-09-11T07:26:31.423229+0000","last_became_peered":"2024-09-11T07:26:31.423229+0000","last_unstale":"2024-09-11T07:28:08.011410+0000","last_undegraded":"2024-09-11T07:28:08.011410+0000","last_fullsized":"2024-09-11T07:28:08.011410+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:45.990548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:45.990548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:51.641867+0000","scrub_duration":0.0014585150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"82'9","reported_seq":58,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.009614+0000","last_change":"2024-09-11T07:27:48.761470+0000","last_active":"2024-09-11T07:28:08.009614+0000","last_peered":"2024-09-11T07:28:08.009614+0000","last_clean":"2024-09-11T07:28:08.009614+0000","last_became_active":"2024-09-11T07:26:33.543628+0000","last_became_peered":"2024-09-11T07:26:33.543628+0000","last_unstale":"2024-09-11T07:28:08.009614+0000","last_undegraded":"2024-09-11T07:28:08.009614+0000","last_fullsized":"2024-09-11T07:28:08.009614+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'9","last_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:27:48.761397+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.913254+0000","scrub_duration":0.0037047439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012854+0000","last_change":"2024-09-11T07:27:42.917739+0000","last_active":"2024-09-11T07:28:08.012854+0000","last_peered":"2024-09-11T07:28:08.012854+0000","last_clean":"2024-09-11T07:28:08.012854+0000","last_became_active":"2024-09-11T07:26:35.566894+0000","last_became_peered":"2024-09-11T07:26:35.566894+0000","last_unstale":"2024-09-11T07:28:08.012854+0000","last_undegraded":"2024-09-11T07:28:08.012854+0000","last_fullsized":"2024-09-11T07:28:08.012854+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.917680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:06.763663+0000","scrub_duration":0.0013496649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014956+0000","last_change":"2024-09-11T07:27:47.031883+0000","last_active":"2024-09-11T07:28:08.014956+0000","last_peered":"2024-09-11T07:28:08.014956+0000","last_clean":"2024-09-11T07:28:08.014956+0000","last_became_active":"2024-09-11T07:26:37.863100+0000","last_became_peered":"2024-09-11T07:26:37.863100+0000","last_unstale":"2024-09-11T07:28:08.014956+0000","last_undegraded":"2024-09-11T07:28:08.014956+0000","last_fullsized":"2024-09-11T07:28:08.014956+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:47.031807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.031807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:54.469643+0000","scrub_duration":0.001556195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"86'15","reported_seq":66,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:22.858220+0000","last_change":"2024-09-11T07:28:22.858192+0000","last_active":"2024-09-11T07:28:22.858220+0000","last_peered":"2024-09-11T07:28:22.858220+0000","last_clean":"2024-09-11T07:28:22.858220+0000","last_became_active":"2024-09-11T07:26:33.545078+0000","last_became_peered":"2024-09-11T07:26:33.545078+0000","last_unstale":"2024-09-11T07:28:22.858220+0000","last_undegraded":"2024-09-11T07:28:22.858220+0000","last_fullsized":"2024-09-11T07:28:22.858220+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'15","last_scrub_stamp":"2024-09-11T07:28:22.858137+0000","last_deep_scrub":"86'15","last_deep_scrub_stamp":"2024-09-11T07:28:22.858137+0000","last_clean_scrub_stamp":"2024-09-11T07:28:22.858137+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:22.911147+0000","scrub_duration":0.003508101,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:14.819117+0000","last_change":"2024-09-11T07:28:14.819081+0000","last_active":"2024-09-11T07:28:14.819117+0000","last_peered":"2024-09-11T07:28:14.819117+0000","last_clean":"2024-09-11T07:28:14.819117+0000","last_became_active":"2024-09-11T07:26:31.423789+0000","last_became_peered":"2024-09-11T07:26:31.423789+0000","last_unstale":"2024-09-11T07:28:14.819117+0000","last_undegraded":"2024-09-11T07:28:14.819117+0000","last_fullsized":"2024-09-11T07:28:14.819117+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:14.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:14.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:17.757008+0000","scrub_duration":0.001929339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":74,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:16.782555+0000","last_change":"2024-09-11T07:28:16.782518+0000","last_active":"2024-09-11T07:28:16.782555+0000","last_peered":"2024-09-11T07:28:16.782555+0000","last_clean":"2024-09-11T07:28:16.782555+0000","last_became_active":"2024-09-11T07:26:35.562245+0000","last_became_peered":"2024-09-11T07:26:35.562245+0000","last_unstale":"2024-09-11T07:28:16.782555+0000","last_undegraded":"2024-09-11T07:28:16.782555+0000","last_fullsized":"2024-09-11T07:28:16.782555+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:28:16.782462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:16.782462+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:30.251276+0000","scrub_duration":0.001462283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014618+0000","last_change":"2024-09-11T07:27:49.245119+0000","last_active":"2024-09-11T07:28:08.014618+0000","last_peered":"2024-09-11T07:28:08.014618+0000","last_clean":"2024-09-11T07:28:08.014618+0000","last_became_active":"2024-09-11T07:26:37.566749+0000","last_became_peered":"2024-09-11T07:26:37.566749+0000","last_unstale":"2024-09-11T07:28:08.014618+0000","last_undegraded":"2024-09-11T07:28:08.014618+0000","last_fullsized":"2024-09-11T07:28:08.014618+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:49.245047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.245047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:49.614854+0000","scrub_duration":0.001867966,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"83'13","reported_seq":66,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012745+0000","last_change":"2024-09-11T07:28:00.959669+0000","last_active":"2024-09-11T07:28:08.012745+0000","last_peered":"2024-09-11T07:28:08.012745+0000","last_clean":"2024-09-11T07:28:08.012745+0000","last_became_active":"2024-09-11T07:26:33.537425+0000","last_became_peered":"2024-09-11T07:26:33.537425+0000","last_unstale":"2024-09-11T07:28:08.012745+0000","last_undegraded":"2024-09-11T07:28:08.012745+0000","last_fullsized":"2024-09-11T07:28:08.012745+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-11T07:28:00.959583+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-11T07:28:00.959583+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.959583+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:06.630980+0000","scrub_duration":0.0032987350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014648+0000","last_change":"2024-09-11T07:28:01.237329+0000","last_active":"2024-09-11T07:28:08.014648+0000","last_peered":"2024-09-11T07:28:08.014648+0000","last_clean":"2024-09-11T07:28:08.014648+0000","last_became_active":"2024-09-11T07:26:31.419409+0000","last_became_peered":"2024-09-11T07:26:31.419409+0000","last_unstale":"2024-09-11T07:28:08.014648+0000","last_undegraded":"2024-09-11T07:28:08.014648+0000","last_fullsized":"2024-09-11T07:28:08.014648+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:01.237267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.237267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.266034+0000","scrub_duration":0.0012797399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":74,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:23.883345+0000","last_change":"2024-09-11T07:28:23.883321+0000","last_active":"2024-09-11T07:28:23.883345+0000","last_peered":"2024-09-11T07:28:23.883345+0000","last_clean":"2024-09-11T07:28:23.883345+0000","last_became_active":"2024-09-11T07:26:35.560610+0000","last_became_peered":"2024-09-11T07:26:35.560610+0000","last_unstale":"2024-09-11T07:28:23.883345+0000","last_undegraded":"2024-09-11T07:28:23.883345+0000","last_fullsized":"2024-09-11T07:28:23.883345+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:28:23.883265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:23.883265+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:49.732087+0000","scrub_duration":0.0020633370000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:22.053387+0000","last_change":"2024-09-11T07:28:22.053350+0000","last_active":"2024-09-11T07:28:22.053387+0000","last_peered":"2024-09-11T07:28:22.053387+0000","last_clean":"2024-09-11T07:28:22.053387+0000","last_became_active":"2024-09-11T07:26:37.565723+0000","last_became_peered":"2024-09-11T07:26:37.565723+0000","last_unstale":"2024-09-11T07:28:22.053387+0000","last_undegraded":"2024-09-11T07:28:22.053387+0000","last_fullsized":"2024-09-11T07:28:22.053387+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:22.053291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:22.053291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:40.893276+0000","scrub_duration":0.001270356,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":62,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012553+0000","last_change":"2024-09-11T07:27:57.601328+0000","last_active":"2024-09-11T07:28:08.012553+0000","last_peered":"2024-09-11T07:28:08.012553+0000","last_clean":"2024-09-11T07:28:08.012553+0000","last_became_active":"2024-09-11T07:26:33.540688+0000","last_became_peered":"2024-09-11T07:26:33.540688+0000","last_unstale":"2024-09-11T07:28:08.012553+0000","last_undegraded":"2024-09-11T07:28:08.012553+0000","last_fullsized":"2024-09-11T07:28:08.012553+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:27:57.601261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.601261+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:58.826830+0000","scrub_duration":0.002247724,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011550+0000","last_change":"2024-09-11T07:27:54.102580+0000","last_active":"2024-09-11T07:28:08.011550+0000","last_peered":"2024-09-11T07:28:08.011550+0000","last_clean":"2024-09-11T07:28:08.011550+0000","last_became_active":"2024-09-11T07:26:31.425587+0000","last_became_peered":"2024-09-11T07:26:31.425587+0000","last_unstale":"2024-09-11T07:28:08.011550+0000","last_undegraded":"2024-09-11T07:28:08.011550+0000","last_fullsized":"2024-09-11T07:28:08.011550+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.102528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.102528+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.102528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:06.835183+0000","scrub_duration":0.0012736080000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008412+0000","last_change":"2024-09-11T07:26:35.562603+0000","last_active":"2024-09-11T07:28:08.008412+0000","last_peered":"2024-09-11T07:28:08.008412+0000","last_clean":"2024-09-11T07:28:08.008412+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:28:08.008412+0000","last_undegraded":"2024-09-11T07:28:08.008412+0000","last_fullsized":"2024-09-11T07:28:08.008412+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.013558+0000","last_change":"2024-09-11T07:27:58.192585+0000","last_active":"2024-09-11T07:28:08.013558+0000","last_peered":"2024-09-11T07:28:08.013558+0000","last_clean":"2024-09-11T07:28:08.013558+0000","last_became_active":"2024-09-11T07:26:37.568886+0000","last_became_peered":"2024-09-11T07:26:37.568886+0000","last_unstale":"2024-09-11T07:28:08.013558+0000","last_undegraded":"2024-09-11T07:28:08.013558+0000","last_fullsized":"2024-09-11T07:28:08.013558+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:58.192528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.192528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:18.757569+0000","scrub_duration":0.001884698,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008607+0000","last_change":"2024-09-11T07:27:48.081460+0000","last_active":"2024-09-11T07:28:08.008607+0000","last_peered":"2024-09-11T07:28:08.008607+0000","last_clean":"2024-09-11T07:28:08.008607+0000","last_became_active":"2024-09-11T07:26:33.540208+0000","last_became_peered":"2024-09-11T07:26:33.540208+0000","last_unstale":"2024-09-11T07:28:08.008607+0000","last_undegraded":"2024-09-11T07:28:08.008607+0000","last_fullsized":"2024-09-11T07:28:08.008607+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:27:48.081392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:48.081392+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:04.524601+0000","scrub_duration":0.0022394590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.013373+0000","last_change":"2024-09-11T07:27:49.677063+0000","last_active":"2024-09-11T07:28:08.013373+0000","last_peered":"2024-09-11T07:28:08.013373+0000","last_clean":"2024-09-11T07:28:08.013373+0000","last_became_active":"2024-09-11T07:26:31.422671+0000","last_became_peered":"2024-09-11T07:26:31.422671+0000","last_unstale":"2024-09-11T07:28:08.013373+0000","last_undegraded":"2024-09-11T07:28:08.013373+0000","last_fullsized":"2024-09-11T07:28:08.013373+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.677003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:54.836177+0000","scrub_duration":0.0013962969999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012435+0000","last_change":"2024-09-11T07:26:35.567257+0000","last_active":"2024-09-11T07:28:08.012435+0000","last_peered":"2024-09-11T07:28:08.012435+0000","last_clean":"2024-09-11T07:28:08.012435+0000","last_became_active":"2024-09-11T07:26:35.566931+0000","last_became_peered":"2024-09-11T07:26:35.566931+0000","last_unstale":"2024-09-11T07:28:08.012435+0000","last_undegraded":"2024-09-11T07:28:08.012435+0000","last_fullsized":"2024-09-11T07:28:08.012435+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:20.766593+0000","last_change":"2024-09-11T07:28:20.766566+0000","last_active":"2024-09-11T07:28:20.766593+0000","last_peered":"2024-09-11T07:28:20.766593+0000","last_clean":"2024-09-11T07:28:20.766593+0000","last_became_active":"2024-09-11T07:26:37.561573+0000","last_became_peered":"2024-09-11T07:26:37.561573+0000","last_unstale":"2024-09-11T07:28:20.766593+0000","last_undegraded":"2024-09-11T07:28:20.766593+0000","last_fullsized":"2024-09-11T07:28:20.766593+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:20.766508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:20.766508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:42.240723+0000","scrub_duration":0.0012453239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"82'6","reported_seq":54,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008222+0000","last_change":"2024-09-11T07:27:47.789519+0000","last_active":"2024-09-11T07:28:08.008222+0000","last_peered":"2024-09-11T07:28:08.008222+0000","last_clean":"2024-09-11T07:28:08.008222+0000","last_became_active":"2024-09-11T07:26:33.545830+0000","last_became_peered":"2024-09-11T07:26:33.545830+0000","last_unstale":"2024-09-11T07:28:08.008222+0000","last_undegraded":"2024-09-11T07:28:08.008222+0000","last_fullsized":"2024-09-11T07:28:08.008222+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'6","last_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_deep_scrub":"82'6","last_deep_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.789465+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:59.162056+0000","scrub_duration":0.002624679,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012450+0000","last_change":"2024-09-11T07:27:58.043076+0000","last_active":"2024-09-11T07:28:08.012450+0000","last_peered":"2024-09-11T07:28:08.012450+0000","last_clean":"2024-09-11T07:28:08.012450+0000","last_became_active":"2024-09-11T07:26:31.420518+0000","last_became_peered":"2024-09-11T07:26:31.420518+0000","last_unstale":"2024-09-11T07:28:08.012450+0000","last_undegraded":"2024-09-11T07:28:08.012450+0000","last_fullsized":"2024-09-11T07:28:08.012450+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.043009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:24.326885+0000","scrub_duration":0.0015014900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011108+0000","last_change":"2024-09-11T07:27:56.124601+0000","last_active":"2024-09-11T07:28:08.011108+0000","last_peered":"2024-09-11T07:28:08.011108+0000","last_clean":"2024-09-11T07:28:08.011108+0000","last_became_active":"2024-09-11T07:26:35.560506+0000","last_became_peered":"2024-09-11T07:26:35.560506+0000","last_unstale":"2024-09-11T07:28:08.011108+0000","last_undegraded":"2024-09-11T07:28:08.011108+0000","last_fullsized":"2024-09-11T07:28:08.011108+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:56.124542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:56.124542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.847074+0000","scrub_duration":0.0011688619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014648+0000","last_change":"2024-09-11T07:27:42.059066+0000","last_active":"2024-09-11T07:28:08.014648+0000","last_peered":"2024-09-11T07:28:08.014648+0000","last_clean":"2024-09-11T07:28:08.014648+0000","last_became_active":"2024-09-11T07:26:37.564208+0000","last_became_peered":"2024-09-11T07:26:37.564208+0000","last_unstale":"2024-09-11T07:28:08.014648+0000","last_undegraded":"2024-09-11T07:28:08.014648+0000","last_fullsized":"2024-09-11T07:28:08.014648+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.059001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.059001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:51.267826+0000","scrub_duration":0.001418094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":55,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008651+0000","last_change":"2024-09-11T07:26:33.542913+0000","last_active":"2024-09-11T07:28:08.008651+0000","last_peered":"2024-09-11T07:28:08.008651+0000","last_clean":"2024-09-11T07:28:08.008651+0000","last_became_active":"2024-09-11T07:26:33.542625+0000","last_became_peered":"2024-09-11T07:26:33.542625+0000","last_unstale":"2024-09-11T07:28:08.008651+0000","last_undegraded":"2024-09-11T07:28:08.008651+0000","last_fullsized":"2024-09-11T07:28:08.008651+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.012369+0000","last_change":"2024-09-11T07:28:02.157520+0000","last_active":"2024-09-11T07:28:08.012369+0000","last_peered":"2024-09-11T07:28:08.012369+0000","last_clean":"2024-09-11T07:28:08.012369+0000","last_became_active":"2024-09-11T07:26:31.425876+0000","last_became_peered":"2024-09-11T07:26:31.425876+0000","last_unstale":"2024-09-11T07:28:08.012369+0000","last_undegraded":"2024-09-11T07:28:08.012369+0000","last_fullsized":"2024-09-11T07:28:08.012369+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:02.157455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.157455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:12.096002+0000","scrub_duration":0.0016990040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014258+0000","last_change":"2024-09-11T07:26:35.565591+0000","last_active":"2024-09-11T07:28:08.014258+0000","last_peered":"2024-09-11T07:28:08.014258+0000","last_clean":"2024-09-11T07:28:08.014258+0000","last_became_active":"2024-09-11T07:26:35.565339+0000","last_became_peered":"2024-09-11T07:26:35.565339+0000","last_unstale":"2024-09-11T07:28:08.014258+0000","last_undegraded":"2024-09-11T07:28:08.014258+0000","last_fullsized":"2024-09-11T07:28:08.014258+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008264+0000","last_change":"2024-09-11T07:26:37.862599+0000","last_active":"2024-09-11T07:28:08.008264+0000","last_peered":"2024-09-11T07:28:08.008264+0000","last_clean":"2024-09-11T07:28:08.008264+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:28:08.008264+0000","last_undegraded":"2024-09-11T07:28:08.008264+0000","last_fullsized":"2024-09-11T07:28:08.008264+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014794+0000","last_change":"2024-09-11T07:26:37.564199+0000","last_active":"2024-09-11T07:28:08.014794+0000","last_peered":"2024-09-11T07:28:08.014794+0000","last_clean":"2024-09-11T07:28:08.014794+0000","last_became_active":"2024-09-11T07:26:37.563942+0000","last_became_peered":"2024-09-11T07:26:37.563942+0000","last_unstale":"2024-09-11T07:28:08.014794+0000","last_undegraded":"2024-09-11T07:28:08.014794+0000","last_fullsized":"2024-09-11T07:28:08.014794+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":57,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011597+0000","last_change":"2024-09-11T07:28:02.969498+0000","last_active":"2024-09-11T07:28:08.011597+0000","last_peered":"2024-09-11T07:28:08.011597+0000","last_clean":"2024-09-11T07:28:08.011597+0000","last_became_active":"2024-09-11T07:26:31.423932+0000","last_became_peered":"2024-09-11T07:26:31.423932+0000","last_unstale":"2024-09-11T07:28:08.011597+0000","last_undegraded":"2024-09-11T07:28:08.011597+0000","last_fullsized":"2024-09-11T07:28:08.011597+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:28:02.969442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.969442+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:22.652879+0000","scrub_duration":0.0014604450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":61,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.014848+0000","last_change":"2024-09-11T07:27:55.150630+0000","last_active":"2024-09-11T07:28:08.014848+0000","last_peered":"2024-09-11T07:28:08.014848+0000","last_clean":"2024-09-11T07:28:08.014848+0000","last_became_active":"2024-09-11T07:26:33.546797+0000","last_became_peered":"2024-09-11T07:26:33.546797+0000","last_unstale":"2024-09-11T07:28:08.014848+0000","last_undegraded":"2024-09-11T07:28:08.014848+0000","last_fullsized":"2024-09-11T07:28:08.014848+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:27:55.150567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:55.150567+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:10.187014+0000","scrub_duration":0.0016612689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"64'8","reported_seq":77,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:14.502374+0000","last_change":"2024-09-11T07:28:01.123469+0000","last_active":"2024-09-11T07:28:14.502374+0000","last_peered":"2024-09-11T07:28:14.502374+0000","last_clean":"2024-09-11T07:28:14.502374+0000","last_became_active":"2024-09-11T07:26:35.559304+0000","last_became_peered":"2024-09-11T07:26:35.559304+0000","last_unstale":"2024-09-11T07:28:14.502374+0000","last_undegraded":"2024-09-11T07:28:14.502374+0000","last_fullsized":"2024-09-11T07:28:14.502374+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:28:01.123401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.123401+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:18.715539+0000","scrub_duration":0.00167504,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:11.735074+0000","last_change":"2024-09-11T07:28:11.735040+0000","last_active":"2024-09-11T07:28:11.735074+0000","last_peered":"2024-09-11T07:28:11.735074+0000","last_clean":"2024-09-11T07:28:11.735074+0000","last_became_active":"2024-09-11T07:26:37.562501+0000","last_became_peered":"2024-09-11T07:26:37.562501+0000","last_unstale":"2024-09-11T07:28:11.735074+0000","last_undegraded":"2024-09-11T07:28:11.735074+0000","last_fullsized":"2024-09-11T07:28:11.735074+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:11.734968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:11.734968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:13.880946+0000","scrub_duration":0.0021993350000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.010282+0000","last_change":"2024-09-11T07:27:38.732867+0000","last_active":"2024-09-11T07:28:08.010282+0000","last_peered":"2024-09-11T07:28:08.010282+0000","last_clean":"2024-09-11T07:28:08.010282+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:28:08.010282+0000","last_undegraded":"2024-09-11T07:28:08.010282+0000","last_fullsized":"2024-09-11T07:28:08.010282+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:38.732795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:38.732795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:56.730699+0000","scrub_duration":0.0025091950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":87,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.024756+0000","last_change":"2024-09-11T07:28:09.024756+0000","last_active":"2024-09-11T07:28:09.024756+0000","last_peered":"2024-09-11T07:28:09.024756+0000","last_clean":"2024-09-11T07:28:09.024756+0000","last_became_active":"2024-09-11T07:28:09.017997+0000","last_became_peered":"2024-09-11T07:28:09.017997+0000","last_unstale":"2024-09-11T07:28:09.024756+0000","last_undegraded":"2024-09-11T07:28:09.024756+0000","last_fullsized":"2024-09-11T07:28:09.024756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:28:03.164220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:03.164220+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:06.470841+0000","scrub_duration":0.0020677629999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:17.484587+0000","last_change":"2024-09-11T07:28:17.484549+0000","last_active":"2024-09-11T07:28:17.484587+0000","last_peered":"2024-09-11T07:28:17.484587+0000","last_clean":"2024-09-11T07:28:17.484587+0000","last_became_active":"2024-09-11T07:26:35.558809+0000","last_became_peered":"2024-09-11T07:26:35.558809+0000","last_unstale":"2024-09-11T07:28:17.484587+0000","last_undegraded":"2024-09-11T07:28:17.484587+0000","last_fullsized":"2024-09-11T07:28:17.484587+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:17.484488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:17.484488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.968070+0000","scrub_duration":0.001695118,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008901+0000","last_change":"2024-09-11T07:28:00.126756+0000","last_active":"2024-09-11T07:28:08.008901+0000","last_peered":"2024-09-11T07:28:08.008901+0000","last_clean":"2024-09-11T07:28:08.008901+0000","last_became_active":"2024-09-11T07:26:37.862373+0000","last_became_peered":"2024-09-11T07:26:37.862373+0000","last_unstale":"2024-09-11T07:28:08.008901+0000","last_undegraded":"2024-09-11T07:28:08.008901+0000","last_fullsized":"2024-09-11T07:28:08.008901+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:00.126702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.126702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:18.534592+0000","scrub_duration":0.001184903,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011441+0000","last_change":"2024-09-11T07:27:33.983889+0000","last_active":"2024-09-11T07:28:08.011441+0000","last_peered":"2024-09-11T07:28:08.011441+0000","last_clean":"2024-09-11T07:28:08.011441+0000","last_became_active":"2024-09-11T07:26:31.424651+0000","last_became_peered":"2024-09-11T07:26:31.424651+0000","last_unstale":"2024-09-11T07:28:08.011441+0000","last_undegraded":"2024-09-11T07:28:08.011441+0000","last_fullsized":"2024-09-11T07:28:08.011441+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:33.983813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:33.983813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:55.264233+0000","scrub_duration":0.001290759,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":71,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.011285+0000","last_change":"2024-09-11T07:28:00.272702+0000","last_active":"2024-09-11T07:28:08.011285+0000","last_peered":"2024-09-11T07:28:08.011285+0000","last_clean":"2024-09-11T07:28:08.011285+0000","last_became_active":"2024-09-11T07:26:33.542544+0000","last_became_peered":"2024-09-11T07:26:33.542544+0000","last_unstale":"2024-09-11T07:28:08.011285+0000","last_undegraded":"2024-09-11T07:28:08.011285+0000","last_fullsized":"2024-09-11T07:28:08.011285+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-09-11T07:28:00.272628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.272628+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:05.374102+0000","scrub_duration":0.0021830019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.013186+0000","last_change":"2024-09-11T07:28:07.721176+0000","last_active":"2024-09-11T07:28:08.013186+0000","last_peered":"2024-09-11T07:28:08.013186+0000","last_clean":"2024-09-11T07:28:08.013186+0000","last_became_active":"2024-09-11T07:26:35.555862+0000","last_became_peered":"2024-09-11T07:26:35.555862+0000","last_unstale":"2024-09-11T07:28:08.013186+0000","last_undegraded":"2024-09-11T07:28:08.013186+0000","last_fullsized":"2024-09-11T07:28:08.013186+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:07.721112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:07.721112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:31.345841+0000","scrub_duration":0.0020675020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.009434+0000","last_change":"2024-09-11T07:28:06.674616+0000","last_active":"2024-09-11T07:28:08.009434+0000","last_peered":"2024-09-11T07:28:08.009434+0000","last_clean":"2024-09-11T07:28:08.009434+0000","last_became_active":"2024-09-11T07:26:37.861647+0000","last_became_peered":"2024-09-11T07:26:37.861647+0000","last_unstale":"2024-09-11T07:28:08.009434+0000","last_undegraded":"2024-09-11T07:28:08.009434+0000","last_fullsized":"2024-09-11T07:28:08.009434+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:06.674558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:06.674558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:17.029204+0000","scrub_duration":0.001489488,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008581+0000","last_change":"2024-09-11T07:27:45.732604+0000","last_active":"2024-09-11T07:28:08.008581+0000","last_peered":"2024-09-11T07:28:08.008581+0000","last_clean":"2024-09-11T07:28:08.008581+0000","last_became_active":"2024-09-11T07:26:31.420598+0000","last_became_peered":"2024-09-11T07:26:31.420598+0000","last_unstale":"2024-09-11T07:28:08.008581+0000","last_undegraded":"2024-09-11T07:28:08.008581+0000","last_fullsized":"2024-09-11T07:28:08.008581+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:45.732528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:45.732528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:48.605075+0000","scrub_duration":0.0015466659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"64'12","reported_seq":69,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008797+0000","last_change":"2024-09-11T07:27:59.737848+0000","last_active":"2024-09-11T07:28:08.008797+0000","last_peered":"2024-09-11T07:28:08.008797+0000","last_clean":"2024-09-11T07:28:08.008797+0000","last_became_active":"2024-09-11T07:26:33.542711+0000","last_became_peered":"2024-09-11T07:26:33.542711+0000","last_unstale":"2024-09-11T07:28:08.008797+0000","last_undegraded":"2024-09-11T07:28:08.008797+0000","last_fullsized":"2024-09-11T07:28:08.008797+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-09-11T07:27:59.737791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:59.737791+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:11.558595+0000","scrub_duration":0.00367732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T07:28:08.008620+0000","last_change":"2024-09-11T07:27:57.199239+0000","last_active":"2024-09-11T07:28:08.008620+0000","last_peered":"2024-09-11T07:28:08.008620+0000","last_clean":"2024-09-11T07:28:08.008620+0000","last_became_active":"2024-09-11T07:26:35.562037+0000","last_became_peered":"2024-09-11T07:26:35.562037+0000","last_unstale":"2024-09-11T07:28:08.008620+0000","last_undegraded":"2024-09-11T07:28:08.008620+0000","last_fullsized":"2024-09-11T07:28:08.008620+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:57.199173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.199173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.093487+0000","scrub_duration":0.0015950770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":430,"ondisk_log_size":430,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":101,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299419,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298568,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446584,"statfs":{"total":95995035648,"available":95689302016,"internally_reserved":0,"allocated":1245184,"data_stored":889696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.50900000000000001,"15min":0.50800000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52700000000000002,"15min":0.52800000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69399999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.53100000000000003}]},{"osd":1,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.503},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.34899999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.66900000000000004,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.56200000000000006}]},{"osd":2,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52400000000000002,"15min":0.52300000000000002},"min":{"1min":0.34599999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69199999999999995,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53000000000000003,"15min":0.52800000000000002},"min":{"1min":0.38700000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73999999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.45100000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":1.0249999999999999,"15min":1.1080000000000001},"min":{"1min":0.34000000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70499999999999996,"5min":12.625999999999999,"15min":12.625999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.35899999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.72899999999999998,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.54900000000000004}]},{"osd":4,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.44,"15min":0.442},"min":{"1min":0.29799999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.63100000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.442,"15min":0.44400000000000001},"min":{"1min":0.28999999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.434}]},{"osd":5,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.436,"15min":0.44},"min":{"1min":0.33000000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.56200000000000006,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.33100000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70599999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.49199999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.34200000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67400000000000004,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.47899999999999998,"15min":0.47799999999999998},"min":{"1min":0.31,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70899999999999996,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54100000000000004}]}]},{"osd":6,"up_from":44,"seq":188978561055,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298544,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446608,"statfs":{"total":95995035648,"available":95689326592,"internally_reserved":0,"allocated":1220608,"data_stored":888174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.374,"5min":0.247,"15min":0.247},"max":{"1min":0.65000000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.311},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.60499999999999998,"15min":0.624},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.754,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.41399999999999998}]},{"osd":1,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.82899999999999996,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.49299999999999999,"15min":0.495},"min":{"1min":0.35799999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.35599999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.61499999999999999,"15min":0.63500000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.70299999999999996,"5min":3.234,"15min":3.234},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.59099999999999997,"15min":0.60699999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.80200000000000005,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.34300000000000003}]},{"osd":3,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51500000000000001,"15min":0.51800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.63500000000000001,"15min":0.65500000000000003},"min":{"1min":0.41199999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.66900000000000004,"5min":3.1739999999999999,"15min":3.1739999999999999},"last":0.433}]},{"osd":4,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.58299999999999996,"15min":0.60899999999999999},"min":{"1min":0.32700000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.65200000000000002,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.28399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.38700000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.65600000000000003,"15min":0.69699999999999995},"min":{"1min":0.32400000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64000000000000001,"5min":5.7329999999999997,"15min":5.7329999999999997},"last":0.372},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.52000000000000002,"15min":0.52700000000000002},"min":{"1min":0.34000000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70899999999999996,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.38100000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:28:23 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.58499999999999996,"15min":0.60499999999999998},"min":{"1min":0.34999999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.66300000000000003,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.34300000000000003,"5min":0.313,"15min":0.313},"max":{"1min":0.67300000000000004,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.42299999999999999}]}]},{"osd":5,"up_from":37,"seq":158913789989,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297972,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447180,"statfs":{"total":95995035648,"available":95689912320,"internally_reserved":0,"allocated":634880,"data_stored":298797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.35499999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.50600000000000001,"15min":0.502},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.46999999999999997}]},{"osd":1,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.316},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.437}]},{"osd":2,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.45000000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.47799999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.375,"15min":0.37},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.42599999999999999,"15min":0.41999999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.42599999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:28:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.41599999999999998,"15min":0.40600000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.42699999999999999,"15min":0.42199999999999999},"min":{"1min":0.32100000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.40999999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:28:21 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.48099999999999998,"15min":0.48899999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53800000000000003,"5min":1.202,"15min":1.202},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.50700000000000001,"15min":0.51300000000000001},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.61799999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.48699999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986217,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297992,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447160,"statfs":{"total":95995035648,"available":95689891840,"internally_reserved":0,"allocated":655360,"data_stored":299132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:28:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52700000000000002,"15min":0.51700000000000002},"min":{"1min":0.42699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.498},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.499},"min":{"1min":0.34999999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.46100000000000002}]},{"osd":1,"last update":"Wed Sep 11 07:28:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.50700000000000001,"15min":0.496},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72199999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.67100000000000004,"15min":0.72299999999999998},"min":{"1min":0.38100000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.81499999999999995,"5min":4.5510000000000002,"15min":4.5510000000000002},"last":0.438}]},{"osd":2,"last update":"Wed Sep 11 07:28:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53600000000000003,"15min":0.53000000000000003},"min":{"1min":0.40200000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.73499999999999999,"5min":1.21,"15min":1.21},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.36899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76800000000000002,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.48599999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:28:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.38,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.70099999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.46500000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.34499999999999997,"15min":0.33800000000000002},"min":{"1min":0.27600000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.378,"15min":0.372},"min":{"1min":0.29399999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.311}]},{"osd":6,"last update":"Wed Sep 11 07:27:59 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42199999999999999,"15min":0.41799999999999998},"min":{"1min":0.315,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.44,"15min":0.43099999999999999},"min":{"1min":0.33500000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.34999999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48999999999999999,"15min":0.48699999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48099999999999998,"15min":0.48299999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.371}]}]},{"osd":3,"up_from":25,"seq":107374182446,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298000,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447152,"statfs":{"total":95995035648,"available":95689883648,"internally_reserved":0,"allocated":663552,"data_stored":298596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:54 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38600000000000001,"15min":0.39300000000000002},"min":{"1min":0.20799999999999999,"5min":0.185,"15min":0.185},"max":{"1min":0.54300000000000004,"5min":0.621,"15min":0.621},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.42499999999999999,"15min":0.441},"min":{"1min":0.26200000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.42499999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:27:54 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.43099999999999999,"15min":0.437},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.625,"5min":0.752,"15min":0.752},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.41599999999999998,"15min":0.43099999999999999},"min":{"1min":0.23599999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.48199999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:27:54 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.45100000000000001,"15min":0.46500000000000002},"min":{"1min":0.26900000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.45400000000000001,"15min":0.46899999999999997},"min":{"1min":0.28899999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.56399999999999995}]},{"osd":4,"last update":"Wed Sep 11 07:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.747,"15min":0.621},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77200000000000002,"5min":21.856000000000002,"15min":21.856000000000002},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.4399999999999999,"5min":0.89700000000000002,"15min":0.67700000000000005},"min":{"1min":0.436,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":18.370000000000001,"5min":19.146000000000001,"15min":19.146000000000001},"last":0.54900000000000004}]},{"osd":5,"last update":"Wed Sep 11 07:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":0.71999999999999997,"15min":0.622},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":18.416,"5min":18.416,"15min":18.416},"last":0.501}]},{"osd":6,"last update":"Wed Sep 11 07:27:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":1.306,"15min":1.4330000000000001},"min":{"1min":0.439,"5min":0.437,"15min":0.437},"max":{"1min":0.72899999999999998,"5min":20.219999999999999,"15min":20.219999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":1.319,"15min":1.446},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72699999999999998,"5min":19.609000000000002,"15min":19.609000000000002},"last":0.77700000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.45900000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.80700000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59799999999999998,"15min":0.60399999999999998},"min":{"1min":0.39700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71899999999999997,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.59399999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411379,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297968,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447184,"statfs":{"total":95995035648,"available":95689916416,"internally_reserved":0,"allocated":630784,"data_stored":298043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:26 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.375,"15min":0.36299999999999999},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.38200000000000001,"15min":0.374},"min":{"1min":0.26600000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.55900000000000005,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.47199999999999998}]},{"osd":1,"last update":"Wed Sep 11 07:27:26 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.40300000000000002,"15min":0.38600000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.495},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.40200000000000002,"15min":0.38300000000000001},"min":{"1min":0.218,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.55400000000000005}]},{"osd":3,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.439,"15min":0.42299999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64500000000000002,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.44,"5min":0.45900000000000002,"15min":0.45400000000000001},"min":{"1min":0.26600000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65000000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.51600000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.94799999999999995,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.86299999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:27:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.68799999999999994,"15min":0.69699999999999995},"min":{"1min":0.45500000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85899999999999999,"5min":2.931,"15min":2.931},"last":0.72999999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:27:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.85299999999999998,"5min":1.494,"15min":1.494},"last":0.747},{"interface":"front","average":{"1min":0.623,"5min":0.63100000000000001,"15min":0.63300000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.875,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.88600000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.64700000000000002,"15min":0.65700000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.92700000000000005,"5min":1.514,"15min":1.514},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.90700000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298008,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447144,"statfs":{"total":95995035648,"available":95689875456,"internally_reserved":0,"allocated":671744,"data_stored":299233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:28:04 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.34000000000000002,"15min":0.33100000000000002},"min":{"1min":0.22800000000000001,"5min":0.186,"15min":0.186},"max":{"1min":0.49299999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.36499999999999999,"15min":0.35499999999999998},"min":{"1min":0.22700000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.45700000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:27:26 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.41899999999999998,"15min":0.41599999999999998},"min":{"1min":0.317,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.41199999999999998,"15min":0.40799999999999997},"min":{"1min":0.28699999999999998,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.36899999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:27:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.44500000000000001,"15min":0.44700000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.60699999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.41399999999999998},"min":{"1min":0.307,"5min":0.253,"15min":0.253},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.33400000000000002}]},{"osd":4,"last update":"Wed Sep 11 07:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54200000000000004,"15min":0.54700000000000004},"min":{"1min":0.34999999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71999999999999997,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.443}]},{"osd":5,"last update":"Wed Sep 11 07:27:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.435},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.44,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.71699999999999997,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.52200000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:27:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57199999999999995,"15min":0.57499999999999996},"min":{"1min":0.41199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.754,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.40999999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.47899999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.51100000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738428,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298560,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446592,"statfs":{"total":95995035648,"available":95689310208,"internally_reserved":0,"allocated":1236992,"data_stored":888506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:28:02 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.36699999999999999,"15min":0.34499999999999997},"min":{"1min":0.246,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.318},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.35899999999999999,"15min":0.34200000000000003},"min":{"1min":0.23799999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.33000000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:27:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.40400000000000003,"15min":0.40699999999999997},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.59299999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.255}]},{"osd":3,"last update":"Wed Sep 11 07:27:55 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45800000000000002,"15min":0.47399999999999998},"min":{"1min":0.317,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.61199999999999999,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.311},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.46300000000000002,"15min":0.48099999999999998},"min":{"1min":0.28299999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.56200000000000006,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.28599999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:28:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56000000000000005,"15min":0.56799999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.77600000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.51300000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:27:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.40000000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.79100000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.442},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.41699999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78100000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.379}]},{"osd":6,"last update":"Wed Sep 11 07:27:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59399999999999997,"15min":0.59699999999999998},"min":{"1min":0.34799999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.433},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60099999999999998,"15min":0.60399999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.76400000000000001,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45100000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:28:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58499999999999996,"15min":0.59299999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.748,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.42799999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.49199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:28:25.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:24 smithi104 ceph-mon[33507]: 5.14 scrub starts 2024-09-11T07:28:25.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:24 smithi104 ceph-mon[33507]: 5.14 scrub ok 2024-09-11T07:28:25.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:24 smithi104 ceph-mon[33507]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 18 B/s, 1 objects/s recovering 2024-09-11T07:28:25.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:24 smithi104 ceph-mon[33507]: from='client.24686 v1:172.21.15.22:0/1986715097' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:28:25.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:24 smithi022 ceph-mon[37330]: 5.14 scrub starts 2024-09-11T07:28:25.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:24 smithi022 ceph-mon[37330]: 5.14 scrub ok 2024-09-11T07:28:25.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:24 smithi022 ceph-mon[37330]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 18 B/s, 1 objects/s recovering 2024-09-11T07:28:25.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:24 smithi022 ceph-mon[37330]: from='client.24686 v1:172.21.15.22:0/1986715097' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:28:25.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:24 smithi022 ceph-mon[29462]: 5.14 scrub starts 2024-09-11T07:28:25.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:24 smithi022 ceph-mon[29462]: 5.14 scrub ok 2024-09-11T07:28:25.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:24 smithi022 ceph-mon[29462]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 18 B/s, 1 objects/s recovering 2024-09-11T07:28:25.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:24 smithi022 ceph-mon[29462]: from='client.24686 v1:172.21.15.22:0/1986715097' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:28:26.417 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:28:26 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:28:26.130Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003580209s 2024-09-11T07:28:26.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:26 smithi022 ceph-mon[29462]: 6.19 scrub starts 2024-09-11T07:28:26.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:26 smithi022 ceph-mon[29462]: 6.19 scrub ok 2024-09-11T07:28:26.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:26 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:26.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:26 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:26.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:26 smithi022 ceph-mon[29462]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:28:26.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:26 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:26.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:26 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:26.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:26 smithi022 ceph-mon[37330]: 6.19 scrub starts 2024-09-11T07:28:26.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:26 smithi022 ceph-mon[37330]: 6.19 scrub ok 2024-09-11T07:28:26.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:26 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:26.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:26 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:26.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:26 smithi022 ceph-mon[37330]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:28:26.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:26 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:26.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:26 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:27.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:26 smithi104 ceph-mon[33507]: 6.19 scrub starts 2024-09-11T07:28:27.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:26 smithi104 ceph-mon[33507]: 6.19 scrub ok 2024-09-11T07:28:27.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:26 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:27.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:26 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:27.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:26 smithi104 ceph-mon[33507]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:28:27.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:26 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:27.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:26 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:27.888 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:28:27.912 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:27 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:27.912 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:27 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:27.912 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:27 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:27.912 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:27 smithi022 ceph-mon[37330]: 5.10 deep-scrub starts 2024-09-11T07:28:27.913 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:27 smithi022 ceph-mon[37330]: 5.10 deep-scrub ok 2024-09-11T07:28:27.913 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:27 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:27.913 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:27 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:27.913 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:27 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:27.913 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:27 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:27.913 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:27 smithi022 ceph-mon[29462]: 5.10 deep-scrub starts 2024-09-11T07:28:27.913 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:27 smithi022 ceph-mon[29462]: 5.10 deep-scrub ok 2024-09-11T07:28:27.913 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:27 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:27.993 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:27 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:27.993 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:27 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:27.993 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:27 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:27.993 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:27 smithi104 ceph-mon[33507]: 5.10 deep-scrub starts 2024-09-11T07:28:27.993 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:27 smithi104 ceph-mon[33507]: 5.10 deep-scrub ok 2024-09-11T07:28:27.993 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:27 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:28:28.150 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:28:28.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:28 smithi022 ceph-mon[29462]: 5.c scrub starts 2024-09-11T07:28:28.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:28 smithi022 ceph-mon[29462]: 5.c scrub ok 2024-09-11T07:28:28.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:28 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:28:28.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:28 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:28:28.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:28 smithi022 ceph-mon[29462]: osdmap e87: 8 total, 8 up, 8 in 2024-09-11T07:28:28.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:28 smithi022 ceph-mon[29462]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:28:28.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:28 smithi022 ceph-mon[37330]: 5.c scrub starts 2024-09-11T07:28:28.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:28 smithi022 ceph-mon[37330]: 5.c scrub ok 2024-09-11T07:28:28.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:28 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:28:28.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:28 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:28:28.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:28 smithi022 ceph-mon[37330]: osdmap e87: 8 total, 8 up, 8 in 2024-09-11T07:28:28.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:28 smithi022 ceph-mon[37330]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:28:29.008 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:28 smithi104 ceph-mon[33507]: 5.c scrub starts 2024-09-11T07:28:29.008 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:28 smithi104 ceph-mon[33507]: 5.c scrub ok 2024-09-11T07:28:29.008 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:28 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:28:29.008 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:28 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:28:29.008 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:28 smithi104 ceph-mon[33507]: osdmap e87: 8 total, 8 up, 8 in 2024-09-11T07:28:29.008 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:28 smithi104 ceph-mon[33507]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:28:29.279 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:28:29 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:28:29.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:29 smithi104 ceph-mon[33507]: 5.1 deep-scrub starts 2024-09-11T07:28:29.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:29 smithi104 ceph-mon[33507]: 5.1 deep-scrub ok 2024-09-11T07:28:29.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:29 smithi104 ceph-mon[33507]: 4.11 scrub starts 2024-09-11T07:28:29.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:29 smithi104 ceph-mon[33507]: 4.11 scrub ok 2024-09-11T07:28:29.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:29 smithi104 ceph-mon[33507]: 6.1c scrub starts 2024-09-11T07:28:29.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:29 smithi104 ceph-mon[33507]: 6.1c scrub ok 2024-09-11T07:28:29.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:29 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:28:29.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:29 smithi022 ceph-mon[29462]: 5.1 deep-scrub starts 2024-09-11T07:28:29.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:29 smithi022 ceph-mon[29462]: 5.1 deep-scrub ok 2024-09-11T07:28:29.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:29 smithi022 ceph-mon[29462]: 4.11 scrub starts 2024-09-11T07:28:29.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:29 smithi022 ceph-mon[29462]: 4.11 scrub ok 2024-09-11T07:28:29.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:29 smithi022 ceph-mon[29462]: 6.1c scrub starts 2024-09-11T07:28:29.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:29 smithi022 ceph-mon[29462]: 6.1c scrub ok 2024-09-11T07:28:29.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:29 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:28:29.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:29 smithi022 ceph-mon[37330]: 5.1 deep-scrub starts 2024-09-11T07:28:29.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:29 smithi022 ceph-mon[37330]: 5.1 deep-scrub ok 2024-09-11T07:28:29.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:29 smithi022 ceph-mon[37330]: 4.11 scrub starts 2024-09-11T07:28:29.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:29 smithi022 ceph-mon[37330]: 4.11 scrub ok 2024-09-11T07:28:29.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:29 smithi022 ceph-mon[37330]: 6.1c scrub starts 2024-09-11T07:28:29.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:29 smithi022 ceph-mon[37330]: 6.1c scrub ok 2024-09-11T07:28:29.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:29 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:28:30.022 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:28:30.024 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:28:30.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:30 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:28:30] "GET /metrics HTTP/1.1" 200 220045 "" "Prometheus/2.43.0" 2024-09-11T07:28:30.732 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2024-09-11T07:28:29.951373+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":964,"num_read_kb":821,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":590588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":589,"ondisk_log_size":589,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385740,"kb_used_data":6892,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575476,"statfs":{"total":767960285184,"available":765517287424,"internally_reserved":0,"allocated":7057408,"data_stored":4212385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002468"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.009314+0000","last_change":"2024-09-11T07:26:37.863505+0000","last_active":"2024-09-11T07:28:09.009314+0000","last_peered":"2024-09-11T07:28:09.009314+0000","last_clean":"2024-09-11T07:28:09.009314+0000","last_became_active":"2024-09-11T07:26:37.863165+0000","last_became_peered":"2024-09-11T07:26:37.863165+0000","last_unstale":"2024-09-11T07:28:09.009314+0000","last_undegraded":"2024-09-11T07:28:09.009314+0000","last_fullsized":"2024-09-11T07:28:09.009314+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.014491+0000","last_change":"2024-09-11T07:27:51.197966+0000","last_active":"2024-09-11T07:28:09.014491+0000","last_peered":"2024-09-11T07:28:09.014491+0000","last_clean":"2024-09-11T07:28:09.014491+0000","last_became_active":"2024-09-11T07:26:31.421005+0000","last_became_peered":"2024-09-11T07:26:31.421005+0000","last_unstale":"2024-09-11T07:28:09.014491+0000","last_undegraded":"2024-09-11T07:28:09.014491+0000","last_fullsized":"2024-09-11T07:28:09.014491+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:51.197903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:51.197903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:00.681709+0000","scrub_duration":0.001600113,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"82'22","reported_seq":79,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.009499+0000","last_change":"2024-09-11T07:27:50.756777+0000","last_active":"2024-09-11T07:28:09.009499+0000","last_peered":"2024-09-11T07:28:09.009499+0000","last_clean":"2024-09-11T07:28:09.009499+0000","last_became_active":"2024-09-11T07:26:33.544627+0000","last_became_peered":"2024-09-11T07:26:33.544627+0000","last_unstale":"2024-09-11T07:28:09.009499+0000","last_undegraded":"2024-09-11T07:28:09.009499+0000","last_fullsized":"2024-09-11T07:28:09.009499+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.756707+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.257406+0000","scrub_duration":0.0049180669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:17.046808+0000","last_change":"2024-09-11T07:28:17.046764+0000","last_active":"2024-09-11T07:28:17.046808+0000","last_peered":"2024-09-11T07:28:17.046808+0000","last_clean":"2024-09-11T07:28:17.046808+0000","last_became_active":"2024-09-11T07:26:35.556296+0000","last_became_peered":"2024-09-11T07:26:35.556296+0000","last_unstale":"2024-09-11T07:28:17.046808+0000","last_undegraded":"2024-09-11T07:28:17.046808+0000","last_fullsized":"2024-09-11T07:28:17.046808+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:17.046711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:17.046711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:21.028176+0000","scrub_duration":0.0011666510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.012470+0000","last_change":"2024-09-11T07:27:36.776948+0000","last_active":"2024-09-11T07:28:09.012470+0000","last_peered":"2024-09-11T07:28:09.012470+0000","last_clean":"2024-09-11T07:28:09.012470+0000","last_became_active":"2024-09-11T07:26:31.420178+0000","last_became_peered":"2024-09-11T07:26:31.420178+0000","last_unstale":"2024-09-11T07:28:09.012470+0000","last_undegraded":"2024-09-11T07:28:09.012470+0000","last_fullsized":"2024-09-11T07:28:09.012470+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:36.776883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:36.776883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:49.961787+0000","scrub_duration":0.0017449290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"86'13","reported_seq":67,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:15.025918+0000","last_change":"2024-09-11T07:28:15.025864+0000","last_active":"2024-09-11T07:28:15.025918+0000","last_peered":"2024-09-11T07:28:15.025918+0000","last_clean":"2024-09-11T07:28:15.025918+0000","last_became_active":"2024-09-11T07:26:33.540732+0000","last_became_peered":"2024-09-11T07:26:33.540732+0000","last_unstale":"2024-09-11T07:28:15.025918+0000","last_undegraded":"2024-09-11T07:28:15.025918+0000","last_fullsized":"2024-09-11T07:28:15.025918+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'13","last_scrub_stamp":"2024-09-11T07:28:15.025757+0000","last_deep_scrub":"86'13","last_deep_scrub_stamp":"2024-09-11T07:28:15.025757+0000","last_clean_scrub_stamp":"2024-09-11T07:28:15.025757+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:26.818521+0000","scrub_duration":0.0039188890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.012298+0000","last_change":"2024-09-11T07:27:50.648473+0000","last_active":"2024-09-11T07:28:09.012298+0000","last_peered":"2024-09-11T07:28:09.012298+0000","last_clean":"2024-09-11T07:28:09.012298+0000","last_became_active":"2024-09-11T07:26:35.561096+0000","last_became_peered":"2024-09-11T07:26:35.561096+0000","last_unstale":"2024-09-11T07:28:09.012298+0000","last_undegraded":"2024-09-11T07:28:09.012298+0000","last_fullsized":"2024-09-11T07:28:09.012298+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.648420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:05.663592+0000","scrub_duration":0.001391446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:19.063805+0000","last_change":"2024-09-11T07:28:19.063739+0000","last_active":"2024-09-11T07:28:19.063805+0000","last_peered":"2024-09-11T07:28:19.063805+0000","last_clean":"2024-09-11T07:28:19.063805+0000","last_became_active":"2024-09-11T07:26:37.566910+0000","last_became_peered":"2024-09-11T07:26:37.566910+0000","last_unstale":"2024-09-11T07:28:19.063805+0000","last_undegraded":"2024-09-11T07:28:19.063805+0000","last_fullsized":"2024-09-11T07:28:19.063805+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:19.063651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:19.063651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:42.984286+0000","scrub_duration":0.0019504990000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:23.003134+0000","last_change":"2024-09-11T07:28:23.003084+0000","last_active":"2024-09-11T07:28:23.003134+0000","last_peered":"2024-09-11T07:28:23.003134+0000","last_clean":"2024-09-11T07:28:23.003134+0000","last_became_active":"2024-09-11T07:26:31.422772+0000","last_became_peered":"2024-09-11T07:26:31.422772+0000","last_unstale":"2024-09-11T07:28:23.003134+0000","last_undegraded":"2024-09-11T07:28:23.003134+0000","last_fullsized":"2024-09-11T07:28:23.003134+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:23.003028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:23.003028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.008955+0000","scrub_duration":0.0013267520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"82'7","reported_seq":59,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.009729+0000","last_change":"2024-09-11T07:27:52.097170+0000","last_active":"2024-09-11T07:28:09.009729+0000","last_peered":"2024-09-11T07:28:09.009729+0000","last_clean":"2024-09-11T07:28:09.009729+0000","last_became_active":"2024-09-11T07:26:33.540351+0000","last_became_peered":"2024-09-11T07:26:33.540351+0000","last_unstale":"2024-09-11T07:28:09.009729+0000","last_undegraded":"2024-09-11T07:28:09.009729+0000","last_fullsized":"2024-09-11T07:28:09.009729+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'7","last_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_deep_scrub":"82'7","last_deep_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_clean_scrub_stamp":"2024-09-11T07:27:52.097071+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:53.905741+0000","scrub_duration":0.0025786670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.332004+0000","last_change":"2024-09-11T07:26:35.563858+0000","last_active":"2024-09-11T07:28:09.332004+0000","last_peered":"2024-09-11T07:28:09.332004+0000","last_clean":"2024-09-11T07:28:09.332004+0000","last_became_active":"2024-09-11T07:26:35.563536+0000","last_became_peered":"2024-09-11T07:26:35.563536+0000","last_unstale":"2024-09-11T07:28:09.332004+0000","last_undegraded":"2024-09-11T07:28:09.332004+0000","last_fullsized":"2024-09-11T07:28:09.332004+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:24.869400+0000","last_change":"2024-09-11T07:28:24.869360+0000","last_active":"2024-09-11T07:28:24.869400+0000","last_peered":"2024-09-11T07:28:24.869400+0000","last_clean":"2024-09-11T07:28:24.869400+0000","last_became_active":"2024-09-11T07:26:37.561758+0000","last_became_peered":"2024-09-11T07:26:37.561758+0000","last_unstale":"2024-09-11T07:28:24.869400+0000","last_undegraded":"2024-09-11T07:28:24.869400+0000","last_fullsized":"2024-09-11T07:28:24.869400+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:24.869307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:24.869307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:43.695642+0000","scrub_duration":0.001382999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.332717+0000","last_change":"2024-09-11T07:27:52.058013+0000","last_active":"2024-09-11T07:28:09.332717+0000","last_peered":"2024-09-11T07:28:09.332717+0000","last_clean":"2024-09-11T07:28:09.332717+0000","last_became_active":"2024-09-11T07:26:31.420969+0000","last_became_peered":"2024-09-11T07:26:31.420969+0000","last_unstale":"2024-09-11T07:28:09.332717+0000","last_undegraded":"2024-09-11T07:28:09.332717+0000","last_fullsized":"2024-09-11T07:28:09.332717+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_clean_scrub_stamp":"2024-09-11T07:27:52.057959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:12.902027+0000","scrub_duration":0.0014644840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"64'9","reported_seq":63,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.009939+0000","last_change":"2024-09-11T07:27:47.108704+0000","last_active":"2024-09-11T07:28:09.009939+0000","last_peered":"2024-09-11T07:28:09.009939+0000","last_clean":"2024-09-11T07:28:09.009939+0000","last_became_active":"2024-09-11T07:26:33.544386+0000","last_became_peered":"2024-09-11T07:26:33.544386+0000","last_unstale":"2024-09-11T07:28:09.009939+0000","last_undegraded":"2024-09-11T07:28:09.009939+0000","last_fullsized":"2024-09-11T07:28:09.009939+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:27:47.108645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.108645+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:08.302050+0000","scrub_duration":0.0024650560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.015333+0000","last_change":"2024-09-11T07:27:57.092029+0000","last_active":"2024-09-11T07:28:09.015333+0000","last_peered":"2024-09-11T07:28:09.015333+0000","last_clean":"2024-09-11T07:28:09.015333+0000","last_became_active":"2024-09-11T07:26:35.569630+0000","last_became_peered":"2024-09-11T07:26:35.569630+0000","last_unstale":"2024-09-11T07:28:09.015333+0000","last_undegraded":"2024-09-11T07:28:09.015333+0000","last_fullsized":"2024-09-11T07:28:09.015333+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:57.091958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.091958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:57.303120+0000","scrub_duration":0.001785872,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.333906+0000","last_change":"2024-09-11T07:28:09.333866+0000","last_active":"2024-09-11T07:28:09.333906+0000","last_peered":"2024-09-11T07:28:09.333906+0000","last_clean":"2024-09-11T07:28:09.333906+0000","last_became_active":"2024-09-11T07:26:37.565475+0000","last_became_peered":"2024-09-11T07:26:37.565475+0000","last_unstale":"2024-09-11T07:28:09.333906+0000","last_undegraded":"2024-09-11T07:28:09.333906+0000","last_fullsized":"2024-09-11T07:28:09.333906+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:09.333804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:09.333804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:11.385070+0000","scrub_duration":0.003498438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":103,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.027961+0000","last_change":"2024-09-11T07:28:09.027694+0000","last_active":"2024-09-11T07:28:09.027961+0000","last_peered":"2024-09-11T07:28:09.027961+0000","last_clean":"2024-09-11T07:28:09.027961+0000","last_became_active":"2024-09-11T07:28:09.018213+0000","last_became_peered":"2024-09-11T07:28:09.018213+0000","last_unstale":"2024-09-11T07:28:09.027961+0000","last_undegraded":"2024-09-11T07:28:09.027961+0000","last_fullsized":"2024-09-11T07:28:09.027961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-09-11T07:28:04.137453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:04.137453+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:05.572290+0000","scrub_duration":0.001914083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:11.897431+0000","last_change":"2024-09-11T07:28:11.897385+0000","last_active":"2024-09-11T07:28:11.897431+0000","last_peered":"2024-09-11T07:28:11.897431+0000","last_clean":"2024-09-11T07:28:11.897431+0000","last_became_active":"2024-09-11T07:26:31.419488+0000","last_became_peered":"2024-09-11T07:26:31.419488+0000","last_unstale":"2024-09-11T07:28:11.897431+0000","last_undegraded":"2024-09-11T07:28:11.897431+0000","last_fullsized":"2024-09-11T07:28:11.897431+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:11.897319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:11.897319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:18.864403+0000","scrub_duration":0.0013939480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:26.841415+0000","last_change":"2024-09-11T07:28:26.841381+0000","last_active":"2024-09-11T07:28:26.841415+0000","last_peered":"2024-09-11T07:28:26.841415+0000","last_clean":"2024-09-11T07:28:26.841415+0000","last_became_active":"2024-09-11T07:26:35.557989+0000","last_became_peered":"2024-09-11T07:26:35.557989+0000","last_unstale":"2024-09-11T07:28:26.841415+0000","last_undegraded":"2024-09-11T07:28:26.841415+0000","last_fullsized":"2024-09-11T07:28:26.841415+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:26.841326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:26.841326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:42.433747+0000","scrub_duration":0.0015001350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.009761+0000","last_change":"2024-09-11T07:27:57.274611+0000","last_active":"2024-09-11T07:28:09.009761+0000","last_peered":"2024-09-11T07:28:09.009761+0000","last_clean":"2024-09-11T07:28:09.009761+0000","last_became_active":"2024-09-11T07:26:37.564554+0000","last_became_peered":"2024-09-11T07:26:37.564554+0000","last_unstale":"2024-09-11T07:28:09.009761+0000","last_undegraded":"2024-09-11T07:28:09.009761+0000","last_fullsized":"2024-09-11T07:28:09.009761+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:57.274554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.274554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.058396+0000","scrub_duration":0.001419584,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":62,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:21.039997+0000","last_change":"2024-09-11T07:28:21.039929+0000","last_active":"2024-09-11T07:28:21.039997+0000","last_peered":"2024-09-11T07:28:21.039997+0000","last_clean":"2024-09-11T07:28:21.039997+0000","last_became_active":"2024-09-11T07:26:33.535411+0000","last_became_peered":"2024-09-11T07:26:33.535411+0000","last_unstale":"2024-09-11T07:28:21.039997+0000","last_undegraded":"2024-09-11T07:28:21.039997+0000","last_fullsized":"2024-09-11T07:28:21.039997+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:28:21.039828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:21.039828+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:24.128795+0000","scrub_duration":0.0021618829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:18.878047+0000","last_change":"2024-09-11T07:28:18.878004+0000","last_active":"2024-09-11T07:28:18.878047+0000","last_peered":"2024-09-11T07:28:18.878047+0000","last_clean":"2024-09-11T07:28:18.878047+0000","last_became_active":"2024-09-11T07:26:31.418178+0000","last_became_peered":"2024-09-11T07:26:31.418178+0000","last_unstale":"2024-09-11T07:28:18.878047+0000","last_undegraded":"2024-09-11T07:28:18.878047+0000","last_fullsized":"2024-09-11T07:28:18.878047+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:18.877940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:18.877940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:21.852832+0000","scrub_duration":0.001611924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"64'8","reported_seq":76,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:24.504249+0000","last_change":"2024-09-11T07:27:47.127485+0000","last_active":"2024-09-11T07:28:24.504249+0000","last_peered":"2024-09-11T07:28:24.504249+0000","last_clean":"2024-09-11T07:28:24.504249+0000","last_became_active":"2024-09-11T07:26:35.564531+0000","last_became_peered":"2024-09-11T07:26:35.564531+0000","last_unstale":"2024-09-11T07:28:24.504249+0000","last_undegraded":"2024-09-11T07:28:24.504249+0000","last_fullsized":"2024-09-11T07:28:24.504249+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:27:47.127428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.127428+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:58.343731+0000","scrub_duration":0.0014091329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.010190+0000","last_change":"2024-09-11T07:26:37.567193+0000","last_active":"2024-09-11T07:28:09.010190+0000","last_peered":"2024-09-11T07:28:09.010190+0000","last_clean":"2024-09-11T07:28:09.010190+0000","last_became_active":"2024-09-11T07:26:37.566919+0000","last_became_peered":"2024-09-11T07:26:37.566919+0000","last_unstale":"2024-09-11T07:28:09.010190+0000","last_undegraded":"2024-09-11T07:28:09.010190+0000","last_fullsized":"2024-09-11T07:28:09.010190+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"83'22","reported_seq":79,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.014467+0000","last_change":"2024-09-11T07:28:01.719185+0000","last_active":"2024-09-11T07:28:09.014467+0000","last_peered":"2024-09-11T07:28:09.014467+0000","last_clean":"2024-09-11T07:28:09.014467+0000","last_became_active":"2024-09-11T07:26:33.549733+0000","last_became_peered":"2024-09-11T07:26:33.549733+0000","last_unstale":"2024-09-11T07:28:09.014467+0000","last_undegraded":"2024-09-11T07:28:09.014467+0000","last_fullsized":"2024-09-11T07:28:09.014467+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'22","last_scrub_stamp":"2024-09-11T07:28:01.719128+0000","last_deep_scrub":"83'22","last_deep_scrub_stamp":"2024-09-11T07:28:01.719128+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.719128+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:27.431086+0000","scrub_duration":0.0047817240000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.012481+0000","last_change":"2024-09-11T07:27:35.589317+0000","last_active":"2024-09-11T07:28:09.012481+0000","last_peered":"2024-09-11T07:28:09.012481+0000","last_clean":"2024-09-11T07:28:09.012481+0000","last_became_active":"2024-09-11T07:26:31.418478+0000","last_became_peered":"2024-09-11T07:26:31.418478+0000","last_unstale":"2024-09-11T07:28:09.012481+0000","last_undegraded":"2024-09-11T07:28:09.012481+0000","last_fullsized":"2024-09-11T07:28:09.012481+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:35.589251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:35.589251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:01.839680+0000","scrub_duration":0.0014929380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":79,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:24.504327+0000","last_change":"2024-09-11T07:27:56.218355+0000","last_active":"2024-09-11T07:28:24.504327+0000","last_peered":"2024-09-11T07:28:24.504327+0000","last_clean":"2024-09-11T07:28:24.504327+0000","last_became_active":"2024-09-11T07:26:35.560406+0000","last_became_peered":"2024-09-11T07:26:35.560406+0000","last_unstale":"2024-09-11T07:28:24.504327+0000","last_undegraded":"2024-09-11T07:28:24.504327+0000","last_fullsized":"2024-09-11T07:28:24.504327+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:27:56.218301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:56.218301+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:21.374837+0000","scrub_duration":0.001687413,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:18.785789+0000","last_change":"2024-09-11T07:28:18.785729+0000","last_active":"2024-09-11T07:28:18.785789+0000","last_peered":"2024-09-11T07:28:18.785789+0000","last_clean":"2024-09-11T07:28:18.785789+0000","last_became_active":"2024-09-11T07:26:37.568005+0000","last_became_peered":"2024-09-11T07:26:37.568005+0000","last_unstale":"2024-09-11T07:28:18.785789+0000","last_undegraded":"2024-09-11T07:28:18.785789+0000","last_fullsized":"2024-09-11T07:28:18.785789+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:18.785667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:18.785667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:39.054254+0000","scrub_duration":0.0013564829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"64'11","reported_seq":66,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.010324+0000","last_change":"2024-09-11T07:27:58.158485+0000","last_active":"2024-09-11T07:28:09.010324+0000","last_peered":"2024-09-11T07:28:09.010324+0000","last_clean":"2024-09-11T07:28:09.010324+0000","last_became_active":"2024-09-11T07:26:33.540777+0000","last_became_peered":"2024-09-11T07:26:33.540777+0000","last_unstale":"2024-09-11T07:28:09.010324+0000","last_undegraded":"2024-09-11T07:28:09.010324+0000","last_fullsized":"2024-09-11T07:28:09.010324+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:27:58.158420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.158420+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.832014+0000","scrub_duration":0.0021737459999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.331893+0000","last_change":"2024-09-11T07:27:34.932151+0000","last_active":"2024-09-11T07:28:09.331893+0000","last_peered":"2024-09-11T07:28:09.331893+0000","last_clean":"2024-09-11T07:28:09.331893+0000","last_became_active":"2024-09-11T07:26:31.418238+0000","last_became_peered":"2024-09-11T07:26:31.418238+0000","last_unstale":"2024-09-11T07:28:09.331893+0000","last_undegraded":"2024-09-11T07:28:09.331893+0000","last_fullsized":"2024-09-11T07:28:09.331893+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:34.932092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:34.932092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:46.904196+0000","scrub_duration":0.002710259,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.016898+0000","last_change":"2024-09-11T07:28:00.005119+0000","last_active":"2024-09-11T07:28:09.016898+0000","last_peered":"2024-09-11T07:28:09.016898+0000","last_clean":"2024-09-11T07:28:09.016898+0000","last_became_active":"2024-09-11T07:26:35.569081+0000","last_became_peered":"2024-09-11T07:26:35.569081+0000","last_unstale":"2024-09-11T07:28:09.016898+0000","last_undegraded":"2024-09-11T07:28:09.016898+0000","last_fullsized":"2024-09-11T07:28:09.016898+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:00.005031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.005031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:10.748420+0000","scrub_duration":0.001403533,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.009833+0000","last_change":"2024-09-11T07:27:46.752272+0000","last_active":"2024-09-11T07:28:09.009833+0000","last_peered":"2024-09-11T07:28:09.009833+0000","last_clean":"2024-09-11T07:28:09.009833+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:28:09.009833+0000","last_undegraded":"2024-09-11T07:28:09.009833+0000","last_fullsized":"2024-09-11T07:28:09.009833+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:46.752181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:46.752181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.072860+0000","scrub_duration":0.001508782,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"64'14","reported_seq":68,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:20.085468+0000","last_change":"2024-09-11T07:28:20.085403+0000","last_active":"2024-09-11T07:28:20.085468+0000","last_peered":"2024-09-11T07:28:20.085468+0000","last_clean":"2024-09-11T07:28:20.085468+0000","last_became_active":"2024-09-11T07:26:33.537850+0000","last_became_peered":"2024-09-11T07:26:33.537850+0000","last_unstale":"2024-09-11T07:28:20.085468+0000","last_undegraded":"2024-09-11T07:28:20.085468+0000","last_fullsized":"2024-09-11T07:28:20.085468+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-09-11T07:28:20.085339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:20.085339+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:27.624554+0000","scrub_duration":0.0019854640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":59,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.017672+0000","last_change":"2024-09-11T07:28:09.015415+0000","last_active":"2024-09-11T07:28:09.017672+0000","last_peered":"2024-09-11T07:28:09.017672+0000","last_clean":"2024-09-11T07:28:09.017672+0000","last_became_active":"2024-09-11T07:28:09.014995+0000","last_became_peered":"2024-09-11T07:28:09.014995+0000","last_unstale":"2024-09-11T07:28:09.017672+0000","last_undegraded":"2024-09-11T07:28:09.017672+0000","last_fullsized":"2024-09-11T07:28:09.017672+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:39.014840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:39.014840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:56.130768+0000","scrub_duration":0.0032789239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"84'9","reported_seq":77,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:24.504207+0000","last_change":"2024-09-11T07:28:05.653289+0000","last_active":"2024-09-11T07:28:24.504207+0000","last_peered":"2024-09-11T07:28:24.504207+0000","last_clean":"2024-09-11T07:28:24.504207+0000","last_became_active":"2024-09-11T07:26:35.562997+0000","last_became_peered":"2024-09-11T07:26:35.562997+0000","last_unstale":"2024-09-11T07:28:24.504207+0000","last_undegraded":"2024-09-11T07:28:24.504207+0000","last_fullsized":"2024-09-11T07:28:24.504207+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-11T07:28:05.653237+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-11T07:28:05.653237+0000","last_clean_scrub_stamp":"2024-09-11T07:28:05.653237+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:22.713095+0000","scrub_duration":0.0027957030000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.332155+0000","last_change":"2024-09-11T07:26:37.862564+0000","last_active":"2024-09-11T07:28:09.332155+0000","last_peered":"2024-09-11T07:28:09.332155+0000","last_clean":"2024-09-11T07:28:09.332155+0000","last_became_active":"2024-09-11T07:26:37.862246+0000","last_became_peered":"2024-09-11T07:26:37.862246+0000","last_unstale":"2024-09-11T07:28:09.332155+0000","last_undegraded":"2024-09-11T07:28:09.332155+0000","last_fullsized":"2024-09-11T07:28:09.332155+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"64'18","reported_seq":69,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.009321+0000","last_change":"2024-09-11T07:26:33.546711+0000","last_active":"2024-09-11T07:28:09.009321+0000","last_peered":"2024-09-11T07:28:09.009321+0000","last_clean":"2024-09-11T07:28:09.009321+0000","last_became_active":"2024-09-11T07:26:33.546188+0000","last_became_peered":"2024-09-11T07:26:33.546188+0000","last_unstale":"2024-09-11T07:28:09.009321+0000","last_undegraded":"2024-09-11T07:28:09.009321+0000","last_fullsized":"2024-09-11T07:28:09.009321+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.014880+0000","last_change":"2024-09-11T07:26:31.421802+0000","last_active":"2024-09-11T07:28:09.014880+0000","last_peered":"2024-09-11T07:28:09.014880+0000","last_clean":"2024-09-11T07:28:09.014880+0000","last_became_active":"2024-09-11T07:26:31.421500+0000","last_became_peered":"2024-09-11T07:26:31.421500+0000","last_unstale":"2024-09-11T07:28:09.014880+0000","last_undegraded":"2024-09-11T07:28:09.014880+0000","last_fullsized":"2024-09-11T07:28:09.014880+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:26:30.397002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T07:28:27.968655+0000","last_change":"2024-09-11T07:28:27.968629+0000","last_active":"2024-09-11T07:28:27.968655+0000","last_peered":"2024-09-11T07:28:27.968655+0000","last_clean":"2024-09-11T07:28:27.968655+0000","last_became_active":"2024-09-11T07:26:35.561284+0000","last_became_peered":"2024-09-11T07:26:35.561284+0000","last_unstale":"2024-09-11T07:28:27.968655+0000","last_undegraded":"2024-09-11T07:28:27.968655+0000","last_fullsized":"2024-09-11T07:28:27.968655+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:27.968579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:27.968579+0000","last_clean_scrub_stamp":"2024-09-11T07:28:27.968579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:53.162017+0000","scrub_duration":0.0026385499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.009737+0000","last_change":"2024-09-11T07:26:37.565014+0000","last_active":"2024-09-11T07:28:09.009737+0000","last_peered":"2024-09-11T07:28:09.009737+0000","last_clean":"2024-09-11T07:28:09.009737+0000","last_became_active":"2024-09-11T07:26:37.564616+0000","last_became_peered":"2024-09-11T07:26:37.564616+0000","last_unstale":"2024-09-11T07:28:09.009737+0000","last_undegraded":"2024-09-11T07:28:09.009737+0000","last_fullsized":"2024-09-11T07:28:09.009737+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":83,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:12.466297+0000","last_change":"2024-09-11T07:28:12.466246+0000","last_active":"2024-09-11T07:28:12.466297+0000","last_peered":"2024-09-11T07:28:12.466297+0000","last_clean":"2024-09-11T07:28:12.466297+0000","last_became_active":"2024-09-11T07:26:33.540639+0000","last_became_peered":"2024-09-11T07:26:33.540639+0000","last_unstale":"2024-09-11T07:28:12.466297+0000","last_undegraded":"2024-09-11T07:28:12.466297+0000","last_fullsized":"2024-09-11T07:28:12.466297+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-11T07:28:12.466179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:12.466179+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:12.523827+0000","scrub_duration":0.0027522670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:12.896650+0000","last_change":"2024-09-11T07:28:12.896603+0000","last_active":"2024-09-11T07:28:12.896650+0000","last_peered":"2024-09-11T07:28:12.896650+0000","last_clean":"2024-09-11T07:28:12.896650+0000","last_became_active":"2024-09-11T07:26:31.419185+0000","last_became_peered":"2024-09-11T07:26:31.419185+0000","last_unstale":"2024-09-11T07:28:12.896650+0000","last_undegraded":"2024-09-11T07:28:12.896650+0000","last_fullsized":"2024-09-11T07:28:12.896650+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:12.896538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:12.896538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.577764+0000","scrub_duration":0.0017083070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.011765+0000","last_change":"2024-09-11T07:27:41.000469+0000","last_active":"2024-09-11T07:28:09.011765+0000","last_peered":"2024-09-11T07:28:09.011765+0000","last_clean":"2024-09-11T07:28:09.011765+0000","last_became_active":"2024-09-11T07:26:35.565359+0000","last_became_peered":"2024-09-11T07:26:35.565359+0000","last_unstale":"2024-09-11T07:28:09.011765+0000","last_undegraded":"2024-09-11T07:28:09.011765+0000","last_fullsized":"2024-09-11T07:28:09.011765+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:41.000401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:41.000401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.049825+0000","scrub_duration":0.001655164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.012222+0000","last_change":"2024-09-11T07:26:37.861736+0000","last_active":"2024-09-11T07:28:09.012222+0000","last_peered":"2024-09-11T07:28:09.012222+0000","last_clean":"2024-09-11T07:28:09.012222+0000","last_became_active":"2024-09-11T07:26:37.861279+0000","last_became_peered":"2024-09-11T07:26:37.861279+0000","last_unstale":"2024-09-11T07:28:09.012222+0000","last_undegraded":"2024-09-11T07:28:09.012222+0000","last_fullsized":"2024-09-11T07:28:09.012222+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":62,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.013704+0000","last_change":"2024-09-11T07:27:51.642529+0000","last_active":"2024-09-11T07:28:09.013704+0000","last_peered":"2024-09-11T07:28:09.013704+0000","last_clean":"2024-09-11T07:28:09.013704+0000","last_became_active":"2024-09-11T07:26:33.543063+0000","last_became_peered":"2024-09-11T07:26:33.543063+0000","last_unstale":"2024-09-11T07:28:09.013704+0000","last_undegraded":"2024-09-11T07:28:09.013704+0000","last_fullsized":"2024-09-11T07:28:09.013704+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:27:51.642464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:51.642464+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:13.146284+0000","scrub_duration":0.0020211439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":56,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.013415+0000","last_change":"2024-09-11T07:27:41.618362+0000","last_active":"2024-09-11T07:28:09.013415+0000","last_peered":"2024-09-11T07:28:09.013415+0000","last_clean":"2024-09-11T07:28:09.013415+0000","last_became_active":"2024-09-11T07:26:31.422110+0000","last_became_peered":"2024-09-11T07:26:31.422110+0000","last_unstale":"2024-09-11T07:28:09.013415+0000","last_undegraded":"2024-09-11T07:28:09.013415+0000","last_fullsized":"2024-09-11T07:28:09.013415+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:27:41.618296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:41.618296+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:57.404591+0000","scrub_duration":0.0020650450000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":76,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:24.504237+0000","last_change":"2024-09-11T07:28:03.183056+0000","last_active":"2024-09-11T07:28:24.504237+0000","last_peered":"2024-09-11T07:28:24.504237+0000","last_clean":"2024-09-11T07:28:24.504237+0000","last_became_active":"2024-09-11T07:26:35.560422+0000","last_became_peered":"2024-09-11T07:26:35.560422+0000","last_unstale":"2024-09-11T07:28:24.504237+0000","last_undegraded":"2024-09-11T07:28:24.504237+0000","last_fullsized":"2024-09-11T07:28:24.504237+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:28:03.182993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:03.182993+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:26.896461+0000","scrub_duration":0.0018949889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.012775+0000","last_change":"2024-09-11T07:27:54.148594+0000","last_active":"2024-09-11T07:28:09.012775+0000","last_peered":"2024-09-11T07:28:09.012775+0000","last_clean":"2024-09-11T07:28:09.012775+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:28:09.012775+0000","last_undegraded":"2024-09-11T07:28:09.012775+0000","last_fullsized":"2024-09-11T07:28:09.012775+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.148534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:02.854448+0000","scrub_duration":0.0013536620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"64'28","reported_seq":118,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:24.504250+0000","last_change":"2024-09-11T07:28:09.770058+0000","last_active":"2024-09-11T07:28:24.504250+0000","last_peered":"2024-09-11T07:28:24.504250+0000","last_clean":"2024-09-11T07:28:24.504250+0000","last_became_active":"2024-09-11T07:26:33.549234+0000","last_became_peered":"2024-09-11T07:26:33.549234+0000","last_unstale":"2024-09-11T07:28:24.504250+0000","last_undegraded":"2024-09-11T07:28:24.504250+0000","last_fullsized":"2024-09-11T07:28:24.504250+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'28","last_scrub_stamp":"2024-09-11T07:28:09.769992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:09.769992+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:37.423717+0000","scrub_duration":0.002388463,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:12.775189+0000","last_change":"2024-09-11T07:28:12.775128+0000","last_active":"2024-09-11T07:28:12.775189+0000","last_peered":"2024-09-11T07:28:12.775189+0000","last_clean":"2024-09-11T07:28:12.775189+0000","last_became_active":"2024-09-11T07:26:31.424618+0000","last_became_peered":"2024-09-11T07:26:31.424618+0000","last_unstale":"2024-09-11T07:28:12.775189+0000","last_undegraded":"2024-09-11T07:28:12.775189+0000","last_fullsized":"2024-09-11T07:28:12.775189+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:12.775067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:12.775067+0000","last_clean_scrub_stamp":"2024-09-11T07:28:12.775067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:21.184139+0000","scrub_duration":0.0015206589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:15.439172+0000","last_change":"2024-09-11T07:28:15.439112+0000","last_active":"2024-09-11T07:28:15.439172+0000","last_peered":"2024-09-11T07:28:15.439172+0000","last_clean":"2024-09-11T07:28:15.439172+0000","last_became_active":"2024-09-11T07:26:35.560363+0000","last_became_peered":"2024-09-11T07:26:35.560363+0000","last_unstale":"2024-09-11T07:28:15.439172+0000","last_undegraded":"2024-09-11T07:28:15.439172+0000","last_fullsized":"2024-09-11T07:28:15.439172+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:15.439020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:15.439020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:29.665525+0000","scrub_duration":0.001542844,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.012988+0000","last_change":"2024-09-11T07:28:00.686634+0000","last_active":"2024-09-11T07:28:09.012988+0000","last_peered":"2024-09-11T07:28:09.012988+0000","last_clean":"2024-09-11T07:28:09.012988+0000","last_became_active":"2024-09-11T07:26:37.562452+0000","last_became_peered":"2024-09-11T07:26:37.562452+0000","last_unstale":"2024-09-11T07:28:09.012988+0000","last_undegraded":"2024-09-11T07:28:09.012988+0000","last_fullsized":"2024-09-11T07:28:09.012988+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:00.686571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.686571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.798861+0000","scrub_duration":0.0016714309999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"64'13","reported_seq":64,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.014543+0000","last_change":"2024-09-11T07:26:33.547483+0000","last_active":"2024-09-11T07:28:09.014543+0000","last_peered":"2024-09-11T07:28:09.014543+0000","last_clean":"2024-09-11T07:28:09.014543+0000","last_became_active":"2024-09-11T07:26:33.546126+0000","last_became_peered":"2024-09-11T07:26:33.546126+0000","last_unstale":"2024-09-11T07:28:09.014543+0000","last_undegraded":"2024-09-11T07:28:09.014543+0000","last_fullsized":"2024-09-11T07:28:09.014543+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:26:32.521323+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":168,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T07:28:28.594299+0000","last_change":"2024-09-11T07:27:59.678930+0000","last_active":"2024-09-11T07:28:28.594299+0000","last_peered":"2024-09-11T07:28:28.594299+0000","last_clean":"2024-09-11T07:28:28.594299+0000","last_became_active":"2024-09-11T07:26:31.421276+0000","last_became_peered":"2024-09-11T07:26:31.421276+0000","last_unstale":"2024-09-11T07:28:28.594299+0000","last_undegraded":"2024-09-11T07:28:28.594299+0000","last_fullsized":"2024-09-11T07:28:28.594299+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'5","last_scrub_stamp":"2024-09-11T07:27:59.678865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:59.678865+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.114929+0000","scrub_duration":0.001792486,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":111,"num_read_kb":106,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.011239+0000","last_change":"2024-09-11T07:28:07.327716+0000","last_active":"2024-09-11T07:28:09.011239+0000","last_peered":"2024-09-11T07:28:09.011239+0000","last_clean":"2024-09-11T07:28:09.011239+0000","last_became_active":"2024-09-11T07:26:31.413354+0000","last_became_peered":"2024-09-11T07:26:31.413354+0000","last_unstale":"2024-09-11T07:28:09.011239+0000","last_undegraded":"2024-09-11T07:28:09.011239+0000","last_fullsized":"2024-09-11T07:28:09.011239+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:07.327648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:07.327648+0000","last_clean_scrub_stamp":"2024-09-11T07:28:07.327648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:35.306829+0000","scrub_duration":0.0011895899999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.010305+0000","last_change":"2024-09-11T07:27:42.141156+0000","last_active":"2024-09-11T07:28:09.010305+0000","last_peered":"2024-09-11T07:28:09.010305+0000","last_clean":"2024-09-11T07:28:09.010305+0000","last_became_active":"2024-09-11T07:26:35.565849+0000","last_became_peered":"2024-09-11T07:26:35.565849+0000","last_unstale":"2024-09-11T07:28:09.010305+0000","last_undegraded":"2024-09-11T07:28:09.010305+0000","last_fullsized":"2024-09-11T07:28:09.010305+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.141100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.141100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.707826+0000","scrub_duration":0.001495672,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.332520+0000","last_change":"2024-09-11T07:27:55.018180+0000","last_active":"2024-09-11T07:28:09.332520+0000","last_peered":"2024-09-11T07:28:09.332520+0000","last_clean":"2024-09-11T07:28:09.332520+0000","last_became_active":"2024-09-11T07:26:37.861292+0000","last_became_peered":"2024-09-11T07:26:37.861292+0000","last_unstale":"2024-09-11T07:28:09.332520+0000","last_undegraded":"2024-09-11T07:28:09.332520+0000","last_fullsized":"2024-09-11T07:28:09.332520+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:55.018125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:55.018125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.234213+0000","scrub_duration":0.0013534339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":89,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.025628+0000","last_change":"2024-09-11T07:28:09.025399+0000","last_active":"2024-09-11T07:28:09.025628+0000","last_peered":"2024-09-11T07:28:09.025628+0000","last_clean":"2024-09-11T07:28:09.025628+0000","last_became_active":"2024-09-11T07:28:09.018134+0000","last_became_peered":"2024-09-11T07:28:09.018134+0000","last_unstale":"2024-09-11T07:28:09.025628+0000","last_undegraded":"2024-09-11T07:28:09.025628+0000","last_fullsized":"2024-09-11T07:28:09.025628+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-09-11T07:27:34.213870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:34.213870+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:46.533561+0000","scrub_duration":0.0026667679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.014123+0000","last_change":"2024-09-11T07:28:08.286847+0000","last_active":"2024-09-11T07:28:09.014123+0000","last_peered":"2024-09-11T07:28:09.014123+0000","last_clean":"2024-09-11T07:28:09.014123+0000","last_became_active":"2024-09-11T07:26:31.420320+0000","last_became_peered":"2024-09-11T07:26:31.420320+0000","last_unstale":"2024-09-11T07:28:09.014123+0000","last_undegraded":"2024-09-11T07:28:09.014123+0000","last_fullsized":"2024-09-11T07:28:09.014123+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-11T07:28:08.286788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:08.286788+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.046248+0000","scrub_duration":0.001825976,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:16.967590+0000","last_change":"2024-09-11T07:28:16.967543+0000","last_active":"2024-09-11T07:28:16.967590+0000","last_peered":"2024-09-11T07:28:16.967590+0000","last_clean":"2024-09-11T07:28:16.967590+0000","last_became_active":"2024-09-11T07:26:31.417455+0000","last_became_peered":"2024-09-11T07:26:31.417455+0000","last_unstale":"2024-09-11T07:28:16.967590+0000","last_undegraded":"2024-09-11T07:28:16.967590+0000","last_fullsized":"2024-09-11T07:28:16.967590+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:16.967492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:16.967492+0000","last_clean_scrub_stamp":"2024-09-11T07:28:16.967492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:29.710608+0000","scrub_duration":0.0012319799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:21.868107+0000","last_change":"2024-09-11T07:28:21.868074+0000","last_active":"2024-09-11T07:28:21.868107+0000","last_peered":"2024-09-11T07:28:21.868107+0000","last_clean":"2024-09-11T07:28:21.868107+0000","last_became_active":"2024-09-11T07:26:35.564178+0000","last_became_peered":"2024-09-11T07:26:35.564178+0000","last_unstale":"2024-09-11T07:28:21.868107+0000","last_undegraded":"2024-09-11T07:28:21.868107+0000","last_fullsized":"2024-09-11T07:28:21.868107+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:21.868020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:21.868020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:40.076469+0000","scrub_duration":0.0013323009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.013076+0000","last_change":"2024-09-11T07:27:54.621934+0000","last_active":"2024-09-11T07:28:09.013076+0000","last_peered":"2024-09-11T07:28:09.013076+0000","last_clean":"2024-09-11T07:28:09.013076+0000","last_became_active":"2024-09-11T07:26:37.561378+0000","last_became_peered":"2024-09-11T07:26:37.561378+0000","last_unstale":"2024-09-11T07:28:09.013076+0000","last_undegraded":"2024-09-11T07:28:09.013076+0000","last_fullsized":"2024-09-11T07:28:09.013076+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.621874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.621874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:07.014230+0000","scrub_duration":0.0013604750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":69,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.012517+0000","last_change":"2024-09-11T07:27:49.061728+0000","last_active":"2024-09-11T07:28:09.012517+0000","last_peered":"2024-09-11T07:28:09.012517+0000","last_clean":"2024-09-11T07:28:09.012517+0000","last_became_active":"2024-09-11T07:26:33.541359+0000","last_became_peered":"2024-09-11T07:26:33.541359+0000","last_unstale":"2024-09-11T07:28:09.012517+0000","last_undegraded":"2024-09-11T07:28:09.012517+0000","last_fullsized":"2024-09-11T07:28:09.012517+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-09-11T07:27:49.061670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.061670+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:51.283390+0000","scrub_duration":0.001809801,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'3","reported_seq":57,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.009496+0000","last_change":"2024-09-11T07:28:01.711301+0000","last_active":"2024-09-11T07:28:09.009496+0000","last_peered":"2024-09-11T07:28:09.009496+0000","last_clean":"2024-09-11T07:28:09.009496+0000","last_became_active":"2024-09-11T07:26:31.419501+0000","last_became_peered":"2024-09-11T07:26:31.419501+0000","last_unstale":"2024-09-11T07:28:09.009496+0000","last_undegraded":"2024-09-11T07:28:09.009496+0000","last_fullsized":"2024-09-11T07:28:09.009496+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.711241+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:02.735271+0000","scrub_duration":0.0034627350000000002,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.016373+0000","last_change":"2024-09-11T07:27:47.903204+0000","last_active":"2024-09-11T07:28:09.016373+0000","last_peered":"2024-09-11T07:28:09.016373+0000","last_clean":"2024-09-11T07:28:09.016373+0000","last_became_active":"2024-09-11T07:26:31.423304+0000","last_became_peered":"2024-09-11T07:26:31.423304+0000","last_unstale":"2024-09-11T07:28:09.016373+0000","last_undegraded":"2024-09-11T07:28:09.016373+0000","last_fullsized":"2024-09-11T07:28:09.016373+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:47.903135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.903135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:01.867608+0000","scrub_duration":0.001284908,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"72'79","reported_seq":84,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.332280+0000","last_change":"2024-09-11T07:26:57.872633+0000","last_active":"2024-09-11T07:28:09.332280+0000","last_peered":"2024-09-11T07:28:09.332280+0000","last_clean":"2024-09-11T07:28:09.332280+0000","last_became_active":"2024-09-11T07:26:17.397381+0000","last_became_peered":"2024-09-11T07:26:17.397381+0000","last_unstale":"2024-09-11T07:28:09.332280+0000","last_undegraded":"2024-09-11T07:28:09.332280+0000","last_fullsized":"2024-09-11T07:28:09.332280+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'72","last_scrub_stamp":"2024-09-11T07:26:57.872570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:24:22.202219+0000","last_clean_scrub_stamp":"2024-09-11T07:26:57.872570+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015639180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.332391+0000","last_change":"2024-09-11T07:27:44.036138+0000","last_active":"2024-09-11T07:28:09.332391+0000","last_peered":"2024-09-11T07:28:09.332391+0000","last_clean":"2024-09-11T07:28:09.332391+0000","last_became_active":"2024-09-11T07:26:35.564480+0000","last_became_peered":"2024-09-11T07:26:35.564480+0000","last_unstale":"2024-09-11T07:28:09.332391+0000","last_undegraded":"2024-09-11T07:28:09.332391+0000","last_fullsized":"2024-09-11T07:28:09.332391+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:44.036058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:44.036058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:48.598709+0000","scrub_duration":0.001488931,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:13.796805+0000","last_change":"2024-09-11T07:28:13.796731+0000","last_active":"2024-09-11T07:28:13.796805+0000","last_peered":"2024-09-11T07:28:13.796805+0000","last_clean":"2024-09-11T07:28:13.796805+0000","last_became_active":"2024-09-11T07:26:37.559920+0000","last_became_peered":"2024-09-11T07:26:37.559920+0000","last_unstale":"2024-09-11T07:28:13.796805+0000","last_undegraded":"2024-09-11T07:28:13.796805+0000","last_fullsized":"2024-09-11T07:28:13.796805+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:13.796657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:13.796657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:31.783767+0000","scrub_duration":0.001458984,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":78,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.012054+0000","last_change":"2024-09-11T07:27:59.128292+0000","last_active":"2024-09-11T07:28:09.012054+0000","last_peered":"2024-09-11T07:28:09.012054+0000","last_clean":"2024-09-11T07:28:09.012054+0000","last_became_active":"2024-09-11T07:26:33.543687+0000","last_became_peered":"2024-09-11T07:26:33.543687+0000","last_unstale":"2024-09-11T07:28:09.012054+0000","last_undegraded":"2024-09-11T07:28:09.012054+0000","last_fullsized":"2024-09-11T07:28:09.012054+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-11T07:27:59.128237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:59.128237+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:16.835564+0000","scrub_duration":0.00199329,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.010106+0000","last_change":"2024-09-11T07:27:54.182157+0000","last_active":"2024-09-11T07:28:09.010106+0000","last_peered":"2024-09-11T07:28:09.010106+0000","last_clean":"2024-09-11T07:28:09.010106+0000","last_became_active":"2024-09-11T07:26:31.419560+0000","last_became_peered":"2024-09-11T07:26:31.419560+0000","last_unstale":"2024-09-11T07:28:09.010106+0000","last_undegraded":"2024-09-11T07:28:09.010106+0000","last_fullsized":"2024-09-11T07:28:09.010106+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.182106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.182106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:04.075506+0000","scrub_duration":0.001213254,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:13.357732+0000","last_change":"2024-09-11T07:28:13.357659+0000","last_active":"2024-09-11T07:28:13.357732+0000","last_peered":"2024-09-11T07:28:13.357732+0000","last_clean":"2024-09-11T07:28:13.357732+0000","last_became_active":"2024-09-11T07:26:35.564044+0000","last_became_peered":"2024-09-11T07:26:35.564044+0000","last_unstale":"2024-09-11T07:28:13.357732+0000","last_undegraded":"2024-09-11T07:28:13.357732+0000","last_fullsized":"2024-09-11T07:28:13.357732+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:13.357544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:13.357544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:42.978875+0000","scrub_duration":0.0017127900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:13.918196+0000","last_change":"2024-09-11T07:28:13.918148+0000","last_active":"2024-09-11T07:28:13.918196+0000","last_peered":"2024-09-11T07:28:13.918196+0000","last_clean":"2024-09-11T07:28:13.918196+0000","last_became_active":"2024-09-11T07:26:37.564801+0000","last_became_peered":"2024-09-11T07:26:37.564801+0000","last_unstale":"2024-09-11T07:28:13.918196+0000","last_undegraded":"2024-09-11T07:28:13.918196+0000","last_fullsized":"2024-09-11T07:28:13.918196+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:13.918082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:13.918082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:31.000135+0000","scrub_duration":0.0014064769999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":63,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.014119+0000","last_change":"2024-09-11T07:28:02.230985+0000","last_active":"2024-09-11T07:28:09.014119+0000","last_peered":"2024-09-11T07:28:09.014119+0000","last_clean":"2024-09-11T07:28:09.014119+0000","last_became_active":"2024-09-11T07:26:33.543148+0000","last_became_peered":"2024-09-11T07:26:33.543148+0000","last_unstale":"2024-09-11T07:28:09.014119+0000","last_undegraded":"2024-09-11T07:28:09.014119+0000","last_fullsized":"2024-09-11T07:28:09.014119+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:28:02.230928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.230928+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:27.244330+0000","scrub_duration":0.001888926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.015045+0000","last_change":"2024-09-11T07:27:53.659961+0000","last_active":"2024-09-11T07:28:09.015045+0000","last_peered":"2024-09-11T07:28:09.015045+0000","last_clean":"2024-09-11T07:28:09.015045+0000","last_became_active":"2024-09-11T07:26:31.423056+0000","last_became_peered":"2024-09-11T07:26:31.423056+0000","last_unstale":"2024-09-11T07:28:09.015045+0000","last_undegraded":"2024-09-11T07:28:09.015045+0000","last_fullsized":"2024-09-11T07:28:09.015045+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:53.659902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:53.659902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.781453+0000","scrub_duration":0.0011613719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:10.337646+0000","last_change":"2024-09-11T07:28:10.337599+0000","last_active":"2024-09-11T07:28:10.337646+0000","last_peered":"2024-09-11T07:28:10.337646+0000","last_clean":"2024-09-11T07:28:10.337646+0000","last_became_active":"2024-09-11T07:26:35.563182+0000","last_became_peered":"2024-09-11T07:26:35.563182+0000","last_unstale":"2024-09-11T07:28:10.337646+0000","last_undegraded":"2024-09-11T07:28:10.337646+0000","last_fullsized":"2024-09-11T07:28:10.337646+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:10.337527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:10.337527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:24.013574+0000","scrub_duration":0.001985653,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:18.504034+0000","last_change":"2024-09-11T07:28:18.503995+0000","last_active":"2024-09-11T07:28:18.504034+0000","last_peered":"2024-09-11T07:28:18.504034+0000","last_clean":"2024-09-11T07:28:18.504034+0000","last_became_active":"2024-09-11T07:26:37.565029+0000","last_became_peered":"2024-09-11T07:26:37.565029+0000","last_unstale":"2024-09-11T07:28:18.504034+0000","last_undegraded":"2024-09-11T07:28:18.504034+0000","last_fullsized":"2024-09-11T07:28:18.504034+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:18.503930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:18.503930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:36.076642+0000","scrub_duration":0.001490297,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":72,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.015977+0000","last_change":"2024-09-11T07:28:07.811136+0000","last_active":"2024-09-11T07:28:09.015977+0000","last_peered":"2024-09-11T07:28:09.015977+0000","last_clean":"2024-09-11T07:28:09.015977+0000","last_became_active":"2024-09-11T07:26:33.538216+0000","last_became_peered":"2024-09-11T07:26:33.538216+0000","last_unstale":"2024-09-11T07:28:09.015977+0000","last_undegraded":"2024-09-11T07:28:09.015977+0000","last_fullsized":"2024-09-11T07:28:09.015977+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-09-11T07:28:07.811071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:07.811071+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:23.495093+0000","scrub_duration":0.0019627080000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.010515+0000","last_change":"2024-09-11T07:27:50.132629+0000","last_active":"2024-09-11T07:28:09.010515+0000","last_peered":"2024-09-11T07:28:09.010515+0000","last_clean":"2024-09-11T07:28:09.010515+0000","last_became_active":"2024-09-11T07:26:31.418490+0000","last_became_peered":"2024-09-11T07:26:31.418490+0000","last_unstale":"2024-09-11T07:28:09.010515+0000","last_undegraded":"2024-09-11T07:28:09.010515+0000","last_fullsized":"2024-09-11T07:28:09.010515+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.132552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.048681+0000","scrub_duration":0.001622807,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":66,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:24.504271+0000","last_change":"2024-09-11T07:26:35.567147+0000","last_active":"2024-09-11T07:28:24.504271+0000","last_peered":"2024-09-11T07:28:24.504271+0000","last_clean":"2024-09-11T07:28:24.504271+0000","last_became_active":"2024-09-11T07:26:35.566888+0000","last_became_peered":"2024-09-11T07:26:35.566888+0000","last_unstale":"2024-09-11T07:28:24.504271+0000","last_undegraded":"2024-09-11T07:28:24.504271+0000","last_fullsized":"2024-09-11T07:28:24.504271+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.016264+0000","last_change":"2024-09-11T07:28:03.961280+0000","last_active":"2024-09-11T07:28:09.016264+0000","last_peered":"2024-09-11T07:28:09.016264+0000","last_clean":"2024-09-11T07:28:09.016264+0000","last_became_active":"2024-09-11T07:26:37.861134+0000","last_became_peered":"2024-09-11T07:26:37.861134+0000","last_unstale":"2024-09-11T07:28:09.016264+0000","last_undegraded":"2024-09-11T07:28:09.016264+0000","last_fullsized":"2024-09-11T07:28:09.016264+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:03.961220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:03.961220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:29.223995+0000","scrub_duration":0.0011943730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"83'17","reported_seq":75,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.009545+0000","last_change":"2024-09-11T07:28:01.098068+0000","last_active":"2024-09-11T07:28:09.009545+0000","last_peered":"2024-09-11T07:28:09.009545+0000","last_clean":"2024-09-11T07:28:09.009545+0000","last_became_active":"2024-09-11T07:26:33.540363+0000","last_became_peered":"2024-09-11T07:26:33.540363+0000","last_unstale":"2024-09-11T07:28:09.009545+0000","last_undegraded":"2024-09-11T07:28:09.009545+0000","last_fullsized":"2024-09-11T07:28:09.009545+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'17","last_scrub_stamp":"2024-09-11T07:28:01.097981+0000","last_deep_scrub":"83'17","last_deep_scrub_stamp":"2024-09-11T07:28:01.097981+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.097981+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:26.179592+0000","scrub_duration":0.0042048720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":66,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.010537+0000","last_change":"2024-09-11T07:27:46.077857+0000","last_active":"2024-09-11T07:28:09.010537+0000","last_peered":"2024-09-11T07:28:09.010537+0000","last_clean":"2024-09-11T07:28:09.010537+0000","last_became_active":"2024-09-11T07:26:31.419718+0000","last_became_peered":"2024-09-11T07:26:31.419718+0000","last_unstale":"2024-09-11T07:28:09.010537+0000","last_undegraded":"2024-09-11T07:28:09.010537+0000","last_fullsized":"2024-09-11T07:28:09.010537+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:27:46.077786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:46.077786+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:07.735381+0000","scrub_duration":0.0019376720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.011615+0000","last_change":"2024-09-11T07:27:53.209518+0000","last_active":"2024-09-11T07:28:09.011615+0000","last_peered":"2024-09-11T07:28:09.011615+0000","last_clean":"2024-09-11T07:28:09.011615+0000","last_became_active":"2024-09-11T07:26:35.561967+0000","last_became_peered":"2024-09-11T07:26:35.561967+0000","last_unstale":"2024-09-11T07:28:09.011615+0000","last_undegraded":"2024-09-11T07:28:09.011615+0000","last_fullsized":"2024-09-11T07:28:09.011615+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:53.209452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:53.209452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:23.125281+0000","scrub_duration":0.0014563029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.009469+0000","last_change":"2024-09-11T07:28:02.677186+0000","last_active":"2024-09-11T07:28:09.009469+0000","last_peered":"2024-09-11T07:28:09.009469+0000","last_clean":"2024-09-11T07:28:09.009469+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:28:09.009469+0000","last_undegraded":"2024-09-11T07:28:09.009469+0000","last_fullsized":"2024-09-11T07:28:09.009469+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:02.677095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.677095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.575070+0000","scrub_duration":0.001519269,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":63,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.014910+0000","last_change":"2024-09-11T07:27:40.351572+0000","last_active":"2024-09-11T07:28:09.014910+0000","last_peered":"2024-09-11T07:28:09.014910+0000","last_clean":"2024-09-11T07:28:09.014910+0000","last_became_active":"2024-09-11T07:26:33.547344+0000","last_became_peered":"2024-09-11T07:26:33.547344+0000","last_unstale":"2024-09-11T07:28:09.014910+0000","last_undegraded":"2024-09-11T07:28:09.014910+0000","last_fullsized":"2024-09-11T07:28:09.014910+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:27:40.351507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:40.351507+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.098394+0000","scrub_duration":0.0021803980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.011693+0000","last_change":"2024-09-11T07:27:58.324311+0000","last_active":"2024-09-11T07:28:09.011693+0000","last_peered":"2024-09-11T07:28:09.011693+0000","last_clean":"2024-09-11T07:28:09.011693+0000","last_became_active":"2024-09-11T07:26:31.415701+0000","last_became_peered":"2024-09-11T07:26:31.415701+0000","last_unstale":"2024-09-11T07:28:09.011693+0000","last_undegraded":"2024-09-11T07:28:09.011693+0000","last_fullsized":"2024-09-11T07:28:09.011693+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:58.324255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.324255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.617500+0000","scrub_duration":0.0013365549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:12.778749+0000","last_change":"2024-09-11T07:28:12.778700+0000","last_active":"2024-09-11T07:28:12.778749+0000","last_peered":"2024-09-11T07:28:12.778749+0000","last_clean":"2024-09-11T07:28:12.778749+0000","last_became_active":"2024-09-11T07:26:35.559262+0000","last_became_peered":"2024-09-11T07:26:35.559262+0000","last_unstale":"2024-09-11T07:28:12.778749+0000","last_undegraded":"2024-09-11T07:28:12.778749+0000","last_fullsized":"2024-09-11T07:28:12.778749+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:12.778645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:12.778645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.979009+0000","scrub_duration":0.0013084870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.009651+0000","last_change":"2024-09-11T07:28:06.305563+0000","last_active":"2024-09-11T07:28:09.009651+0000","last_peered":"2024-09-11T07:28:09.009651+0000","last_clean":"2024-09-11T07:28:09.009651+0000","last_became_active":"2024-09-11T07:26:37.564084+0000","last_became_peered":"2024-09-11T07:26:37.564084+0000","last_unstale":"2024-09-11T07:28:09.009651+0000","last_undegraded":"2024-09-11T07:28:09.009651+0000","last_fullsized":"2024-09-11T07:28:09.009651+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:28:06.305502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:06.305502+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.659083+0000","scrub_duration":0.0021934649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.012373+0000","last_change":"2024-09-11T07:27:45.990617+0000","last_active":"2024-09-11T07:28:09.012373+0000","last_peered":"2024-09-11T07:28:09.012373+0000","last_clean":"2024-09-11T07:28:09.012373+0000","last_became_active":"2024-09-11T07:26:31.423229+0000","last_became_peered":"2024-09-11T07:26:31.423229+0000","last_unstale":"2024-09-11T07:28:09.012373+0000","last_undegraded":"2024-09-11T07:28:09.012373+0000","last_fullsized":"2024-09-11T07:28:09.012373+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:45.990548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:45.990548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:51.641867+0000","scrub_duration":0.0014585150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"82'9","reported_seq":59,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.012142+0000","last_change":"2024-09-11T07:27:48.761470+0000","last_active":"2024-09-11T07:28:09.012142+0000","last_peered":"2024-09-11T07:28:09.012142+0000","last_clean":"2024-09-11T07:28:09.012142+0000","last_became_active":"2024-09-11T07:26:33.543628+0000","last_became_peered":"2024-09-11T07:26:33.543628+0000","last_unstale":"2024-09-11T07:28:09.012142+0000","last_undegraded":"2024-09-11T07:28:09.012142+0000","last_fullsized":"2024-09-11T07:28:09.012142+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'9","last_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:27:48.761397+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.913254+0000","scrub_duration":0.0037047439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.016852+0000","last_change":"2024-09-11T07:27:42.917739+0000","last_active":"2024-09-11T07:28:09.016852+0000","last_peered":"2024-09-11T07:28:09.016852+0000","last_clean":"2024-09-11T07:28:09.016852+0000","last_became_active":"2024-09-11T07:26:35.566894+0000","last_became_peered":"2024-09-11T07:26:35.566894+0000","last_unstale":"2024-09-11T07:28:09.016852+0000","last_undegraded":"2024-09-11T07:28:09.016852+0000","last_fullsized":"2024-09-11T07:28:09.016852+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.917680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:06.763663+0000","scrub_duration":0.0013496649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.332489+0000","last_change":"2024-09-11T07:27:47.031883+0000","last_active":"2024-09-11T07:28:09.332489+0000","last_peered":"2024-09-11T07:28:09.332489+0000","last_clean":"2024-09-11T07:28:09.332489+0000","last_became_active":"2024-09-11T07:26:37.863100+0000","last_became_peered":"2024-09-11T07:26:37.863100+0000","last_unstale":"2024-09-11T07:28:09.332489+0000","last_undegraded":"2024-09-11T07:28:09.332489+0000","last_fullsized":"2024-09-11T07:28:09.332489+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:47.031807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.031807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:54.469643+0000","scrub_duration":0.001556195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"86'15","reported_seq":67,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:22.858248+0000","last_change":"2024-09-11T07:28:22.858192+0000","last_active":"2024-09-11T07:28:22.858248+0000","last_peered":"2024-09-11T07:28:22.858248+0000","last_clean":"2024-09-11T07:28:22.858248+0000","last_became_active":"2024-09-11T07:26:33.545078+0000","last_became_peered":"2024-09-11T07:26:33.545078+0000","last_unstale":"2024-09-11T07:28:22.858248+0000","last_undegraded":"2024-09-11T07:28:22.858248+0000","last_fullsized":"2024-09-11T07:28:22.858248+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'15","last_scrub_stamp":"2024-09-11T07:28:22.858137+0000","last_deep_scrub":"86'15","last_deep_scrub_stamp":"2024-09-11T07:28:22.858137+0000","last_clean_scrub_stamp":"2024-09-11T07:28:22.858137+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:22.911147+0000","scrub_duration":0.003508101,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:14.819129+0000","last_change":"2024-09-11T07:28:14.819081+0000","last_active":"2024-09-11T07:28:14.819129+0000","last_peered":"2024-09-11T07:28:14.819129+0000","last_clean":"2024-09-11T07:28:14.819129+0000","last_became_active":"2024-09-11T07:26:31.423789+0000","last_became_peered":"2024-09-11T07:26:31.423789+0000","last_unstale":"2024-09-11T07:28:14.819129+0000","last_undegraded":"2024-09-11T07:28:14.819129+0000","last_fullsized":"2024-09-11T07:28:14.819129+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:14.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:14.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:17.757008+0000","scrub_duration":0.001929339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":76,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:24.504309+0000","last_change":"2024-09-11T07:28:16.782518+0000","last_active":"2024-09-11T07:28:24.504309+0000","last_peered":"2024-09-11T07:28:24.504309+0000","last_clean":"2024-09-11T07:28:24.504309+0000","last_became_active":"2024-09-11T07:26:35.562245+0000","last_became_peered":"2024-09-11T07:26:35.562245+0000","last_unstale":"2024-09-11T07:28:24.504309+0000","last_undegraded":"2024-09-11T07:28:24.504309+0000","last_fullsized":"2024-09-11T07:28:24.504309+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:28:16.782462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:16.782462+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:30.251276+0000","scrub_duration":0.001462283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.014625+0000","last_change":"2024-09-11T07:27:49.245119+0000","last_active":"2024-09-11T07:28:09.014625+0000","last_peered":"2024-09-11T07:28:09.014625+0000","last_clean":"2024-09-11T07:28:09.014625+0000","last_became_active":"2024-09-11T07:26:37.566749+0000","last_became_peered":"2024-09-11T07:26:37.566749+0000","last_unstale":"2024-09-11T07:28:09.014625+0000","last_undegraded":"2024-09-11T07:28:09.014625+0000","last_fullsized":"2024-09-11T07:28:09.014625+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:49.245047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.245047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:49.614854+0000","scrub_duration":0.001867966,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"83'13","reported_seq":67,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.016895+0000","last_change":"2024-09-11T07:28:00.959669+0000","last_active":"2024-09-11T07:28:09.016895+0000","last_peered":"2024-09-11T07:28:09.016895+0000","last_clean":"2024-09-11T07:28:09.016895+0000","last_became_active":"2024-09-11T07:26:33.537425+0000","last_became_peered":"2024-09-11T07:26:33.537425+0000","last_unstale":"2024-09-11T07:28:09.016895+0000","last_undegraded":"2024-09-11T07:28:09.016895+0000","last_fullsized":"2024-09-11T07:28:09.016895+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-11T07:28:00.959583+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-11T07:28:00.959583+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.959583+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:06.630980+0000","scrub_duration":0.0032987350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.014315+0000","last_change":"2024-09-11T07:28:01.237329+0000","last_active":"2024-09-11T07:28:09.014315+0000","last_peered":"2024-09-11T07:28:09.014315+0000","last_clean":"2024-09-11T07:28:09.014315+0000","last_became_active":"2024-09-11T07:26:31.419409+0000","last_became_peered":"2024-09-11T07:26:31.419409+0000","last_unstale":"2024-09-11T07:28:09.014315+0000","last_undegraded":"2024-09-11T07:28:09.014315+0000","last_fullsized":"2024-09-11T07:28:09.014315+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:01.237267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.237267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.266034+0000","scrub_duration":0.0012797399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":76,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:24.504279+0000","last_change":"2024-09-11T07:28:23.883321+0000","last_active":"2024-09-11T07:28:24.504279+0000","last_peered":"2024-09-11T07:28:24.504279+0000","last_clean":"2024-09-11T07:28:24.504279+0000","last_became_active":"2024-09-11T07:26:35.560610+0000","last_became_peered":"2024-09-11T07:26:35.560610+0000","last_unstale":"2024-09-11T07:28:24.504279+0000","last_undegraded":"2024-09-11T07:28:24.504279+0000","last_fullsized":"2024-09-11T07:28:24.504279+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:28:23.883265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:23.883265+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:49.732087+0000","scrub_duration":0.0020633370000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:22.053403+0000","last_change":"2024-09-11T07:28:22.053350+0000","last_active":"2024-09-11T07:28:22.053403+0000","last_peered":"2024-09-11T07:28:22.053403+0000","last_clean":"2024-09-11T07:28:22.053403+0000","last_became_active":"2024-09-11T07:26:37.565723+0000","last_became_peered":"2024-09-11T07:26:37.565723+0000","last_unstale":"2024-09-11T07:28:22.053403+0000","last_undegraded":"2024-09-11T07:28:22.053403+0000","last_fullsized":"2024-09-11T07:28:22.053403+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:22.053291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:22.053291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:40.893276+0000","scrub_duration":0.001270356,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":63,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.013624+0000","last_change":"2024-09-11T07:27:57.601328+0000","last_active":"2024-09-11T07:28:09.013624+0000","last_peered":"2024-09-11T07:28:09.013624+0000","last_clean":"2024-09-11T07:28:09.013624+0000","last_became_active":"2024-09-11T07:26:33.540688+0000","last_became_peered":"2024-09-11T07:26:33.540688+0000","last_unstale":"2024-09-11T07:28:09.013624+0000","last_undegraded":"2024-09-11T07:28:09.013624+0000","last_fullsized":"2024-09-11T07:28:09.013624+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:27:57.601261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.601261+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:58.826830+0000","scrub_duration":0.002247724,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.012049+0000","last_change":"2024-09-11T07:27:54.102580+0000","last_active":"2024-09-11T07:28:09.012049+0000","last_peered":"2024-09-11T07:28:09.012049+0000","last_clean":"2024-09-11T07:28:09.012049+0000","last_became_active":"2024-09-11T07:26:31.425587+0000","last_became_peered":"2024-09-11T07:26:31.425587+0000","last_unstale":"2024-09-11T07:28:09.012049+0000","last_undegraded":"2024-09-11T07:28:09.012049+0000","last_fullsized":"2024-09-11T07:28:09.012049+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.102528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.102528+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.102528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:06.835183+0000","scrub_duration":0.0012736080000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.009539+0000","last_change":"2024-09-11T07:26:35.562603+0000","last_active":"2024-09-11T07:28:09.009539+0000","last_peered":"2024-09-11T07:28:09.009539+0000","last_clean":"2024-09-11T07:28:09.009539+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:28:09.009539+0000","last_undegraded":"2024-09-11T07:28:09.009539+0000","last_fullsized":"2024-09-11T07:28:09.009539+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.012689+0000","last_change":"2024-09-11T07:27:58.192585+0000","last_active":"2024-09-11T07:28:09.012689+0000","last_peered":"2024-09-11T07:28:09.012689+0000","last_clean":"2024-09-11T07:28:09.012689+0000","last_became_active":"2024-09-11T07:26:37.568886+0000","last_became_peered":"2024-09-11T07:26:37.568886+0000","last_unstale":"2024-09-11T07:28:09.012689+0000","last_undegraded":"2024-09-11T07:28:09.012689+0000","last_fullsized":"2024-09-11T07:28:09.012689+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:58.192528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.192528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:18.757569+0000","scrub_duration":0.001884698,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":66,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.009806+0000","last_change":"2024-09-11T07:27:48.081460+0000","last_active":"2024-09-11T07:28:09.009806+0000","last_peered":"2024-09-11T07:28:09.009806+0000","last_clean":"2024-09-11T07:28:09.009806+0000","last_became_active":"2024-09-11T07:26:33.540208+0000","last_became_peered":"2024-09-11T07:26:33.540208+0000","last_unstale":"2024-09-11T07:28:09.009806+0000","last_undegraded":"2024-09-11T07:28:09.009806+0000","last_fullsized":"2024-09-11T07:28:09.009806+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:27:48.081392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:48.081392+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:04.524601+0000","scrub_duration":0.0022394590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.015041+0000","last_change":"2024-09-11T07:27:49.677063+0000","last_active":"2024-09-11T07:28:09.015041+0000","last_peered":"2024-09-11T07:28:09.015041+0000","last_clean":"2024-09-11T07:28:09.015041+0000","last_became_active":"2024-09-11T07:26:31.422671+0000","last_became_peered":"2024-09-11T07:26:31.422671+0000","last_unstale":"2024-09-11T07:28:09.015041+0000","last_undegraded":"2024-09-11T07:28:09.015041+0000","last_fullsized":"2024-09-11T07:28:09.015041+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.677003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:54.836177+0000","scrub_duration":0.0013962969999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.013496+0000","last_change":"2024-09-11T07:26:35.567257+0000","last_active":"2024-09-11T07:28:09.013496+0000","last_peered":"2024-09-11T07:28:09.013496+0000","last_clean":"2024-09-11T07:28:09.013496+0000","last_became_active":"2024-09-11T07:26:35.566931+0000","last_became_peered":"2024-09-11T07:26:35.566931+0000","last_unstale":"2024-09-11T07:28:09.013496+0000","last_undegraded":"2024-09-11T07:28:09.013496+0000","last_fullsized":"2024-09-11T07:28:09.013496+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:26:34.535883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:20.766612+0000","last_change":"2024-09-11T07:28:20.766566+0000","last_active":"2024-09-11T07:28:20.766612+0000","last_peered":"2024-09-11T07:28:20.766612+0000","last_clean":"2024-09-11T07:28:20.766612+0000","last_became_active":"2024-09-11T07:26:37.561573+0000","last_became_peered":"2024-09-11T07:26:37.561573+0000","last_unstale":"2024-09-11T07:28:20.766612+0000","last_undegraded":"2024-09-11T07:28:20.766612+0000","last_fullsized":"2024-09-11T07:28:20.766612+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:20.766508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:20.766508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:42.240723+0000","scrub_duration":0.0012453239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"82'6","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.009199+0000","last_change":"2024-09-11T07:27:47.789519+0000","last_active":"2024-09-11T07:28:09.009199+0000","last_peered":"2024-09-11T07:28:09.009199+0000","last_clean":"2024-09-11T07:28:09.009199+0000","last_became_active":"2024-09-11T07:26:33.545830+0000","last_became_peered":"2024-09-11T07:26:33.545830+0000","last_unstale":"2024-09-11T07:28:09.009199+0000","last_undegraded":"2024-09-11T07:28:09.009199+0000","last_fullsized":"2024-09-11T07:28:09.009199+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'6","last_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_deep_scrub":"82'6","last_deep_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.789465+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:59.162056+0000","scrub_duration":0.002624679,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.016963+0000","last_change":"2024-09-11T07:27:58.043076+0000","last_active":"2024-09-11T07:28:09.016963+0000","last_peered":"2024-09-11T07:28:09.016963+0000","last_clean":"2024-09-11T07:28:09.016963+0000","last_became_active":"2024-09-11T07:26:31.420518+0000","last_became_peered":"2024-09-11T07:26:31.420518+0000","last_unstale":"2024-09-11T07:28:09.016963+0000","last_undegraded":"2024-09-11T07:28:09.016963+0000","last_fullsized":"2024-09-11T07:28:09.016963+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.043009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:24.326885+0000","scrub_duration":0.0015014900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.011798+0000","last_change":"2024-09-11T07:27:56.124601+0000","last_active":"2024-09-11T07:28:09.011798+0000","last_peered":"2024-09-11T07:28:09.011798+0000","last_clean":"2024-09-11T07:28:09.011798+0000","last_became_active":"2024-09-11T07:26:35.560506+0000","last_became_peered":"2024-09-11T07:26:35.560506+0000","last_unstale":"2024-09-11T07:28:09.011798+0000","last_undegraded":"2024-09-11T07:28:09.011798+0000","last_fullsized":"2024-09-11T07:28:09.011798+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:56.124542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:56.124542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.847074+0000","scrub_duration":0.0011688619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.331866+0000","last_change":"2024-09-11T07:27:42.059066+0000","last_active":"2024-09-11T07:28:09.331866+0000","last_peered":"2024-09-11T07:28:09.331866+0000","last_clean":"2024-09-11T07:28:09.331866+0000","last_became_active":"2024-09-11T07:26:37.564208+0000","last_became_peered":"2024-09-11T07:26:37.564208+0000","last_unstale":"2024-09-11T07:28:09.331866+0000","last_undegraded":"2024-09-11T07:28:09.331866+0000","last_fullsized":"2024-09-11T07:28:09.331866+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.059001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.059001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:51.267826+0000","scrub_duration":0.001418094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":66,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T07:28:28.771191+0000","last_change":"2024-09-11T07:28:28.771165+0000","last_active":"2024-09-11T07:28:28.771191+0000","last_peered":"2024-09-11T07:28:28.771191+0000","last_clean":"2024-09-11T07:28:28.771191+0000","last_became_active":"2024-09-11T07:26:33.542625+0000","last_became_peered":"2024-09-11T07:26:33.542625+0000","last_unstale":"2024-09-11T07:28:28.771191+0000","last_undegraded":"2024-09-11T07:28:28.771191+0000","last_fullsized":"2024-09-11T07:28:28.771191+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:28:28.771107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:28.771107+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:44.367423+0000","scrub_duration":0.001700355,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.012518+0000","last_change":"2024-09-11T07:28:02.157520+0000","last_active":"2024-09-11T07:28:09.012518+0000","last_peered":"2024-09-11T07:28:09.012518+0000","last_clean":"2024-09-11T07:28:09.012518+0000","last_became_active":"2024-09-11T07:26:31.425876+0000","last_became_peered":"2024-09-11T07:26:31.425876+0000","last_unstale":"2024-09-11T07:28:09.012518+0000","last_undegraded":"2024-09-11T07:28:09.012518+0000","last_fullsized":"2024-09-11T07:28:09.012518+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:02.157455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.157455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:12.096002+0000","scrub_duration":0.0016990040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:27.019079+0000","last_change":"2024-09-11T07:28:27.019028+0000","last_active":"2024-09-11T07:28:27.019079+0000","last_peered":"2024-09-11T07:28:27.019079+0000","last_clean":"2024-09-11T07:28:27.019079+0000","last_became_active":"2024-09-11T07:26:35.565339+0000","last_became_peered":"2024-09-11T07:26:35.565339+0000","last_unstale":"2024-09-11T07:28:27.019079+0000","last_undegraded":"2024-09-11T07:28:27.019079+0000","last_fullsized":"2024-09-11T07:28:27.019079+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:27.018961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:27.018961+0000","last_clean_scrub_stamp":"2024-09-11T07:28:27.018961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:27.125944+0000","scrub_duration":0.001482106,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.009396+0000","last_change":"2024-09-11T07:26:37.862599+0000","last_active":"2024-09-11T07:28:09.009396+0000","last_peered":"2024-09-11T07:28:09.009396+0000","last_clean":"2024-09-11T07:28:09.009396+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:28:09.009396+0000","last_undegraded":"2024-09-11T07:28:09.009396+0000","last_fullsized":"2024-09-11T07:28:09.009396+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.332332+0000","last_change":"2024-09-11T07:26:37.564199+0000","last_active":"2024-09-11T07:28:09.332332+0000","last_peered":"2024-09-11T07:28:09.332332+0000","last_clean":"2024-09-11T07:28:09.332332+0000","last_became_active":"2024-09-11T07:26:37.563942+0000","last_became_peered":"2024-09-11T07:26:37.563942+0000","last_unstale":"2024-09-11T07:28:09.332332+0000","last_undegraded":"2024-09-11T07:28:09.332332+0000","last_fullsized":"2024-09-11T07:28:09.332332+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:26:36.546273+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":58,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.016070+0000","last_change":"2024-09-11T07:28:02.969498+0000","last_active":"2024-09-11T07:28:09.016070+0000","last_peered":"2024-09-11T07:28:09.016070+0000","last_clean":"2024-09-11T07:28:09.016070+0000","last_became_active":"2024-09-11T07:26:31.423932+0000","last_became_peered":"2024-09-11T07:26:31.423932+0000","last_unstale":"2024-09-11T07:28:09.016070+0000","last_undegraded":"2024-09-11T07:28:09.016070+0000","last_fullsized":"2024-09-11T07:28:09.016070+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:28:02.969442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.969442+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:22.652879+0000","scrub_duration":0.0014604450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":62,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.014835+0000","last_change":"2024-09-11T07:27:55.150630+0000","last_active":"2024-09-11T07:28:09.014835+0000","last_peered":"2024-09-11T07:28:09.014835+0000","last_clean":"2024-09-11T07:28:09.014835+0000","last_became_active":"2024-09-11T07:26:33.546797+0000","last_became_peered":"2024-09-11T07:26:33.546797+0000","last_unstale":"2024-09-11T07:28:09.014835+0000","last_undegraded":"2024-09-11T07:28:09.014835+0000","last_fullsized":"2024-09-11T07:28:09.014835+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:27:55.150567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:55.150567+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:10.187014+0000","scrub_duration":0.0016612689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"64'8","reported_seq":79,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:24.504307+0000","last_change":"2024-09-11T07:28:01.123469+0000","last_active":"2024-09-11T07:28:24.504307+0000","last_peered":"2024-09-11T07:28:24.504307+0000","last_clean":"2024-09-11T07:28:24.504307+0000","last_became_active":"2024-09-11T07:26:35.559304+0000","last_became_peered":"2024-09-11T07:26:35.559304+0000","last_unstale":"2024-09-11T07:28:24.504307+0000","last_undegraded":"2024-09-11T07:28:24.504307+0000","last_fullsized":"2024-09-11T07:28:24.504307+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:28:01.123401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.123401+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:18.715539+0000","scrub_duration":0.00167504,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:11.735087+0000","last_change":"2024-09-11T07:28:11.735040+0000","last_active":"2024-09-11T07:28:11.735087+0000","last_peered":"2024-09-11T07:28:11.735087+0000","last_clean":"2024-09-11T07:28:11.735087+0000","last_became_active":"2024-09-11T07:26:37.562501+0000","last_became_peered":"2024-09-11T07:26:37.562501+0000","last_unstale":"2024-09-11T07:28:11.735087+0000","last_undegraded":"2024-09-11T07:28:11.735087+0000","last_fullsized":"2024-09-11T07:28:11.735087+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:11.734968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:11.734968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:13.880946+0000","scrub_duration":0.0021993350000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.009583+0000","last_change":"2024-09-11T07:27:38.732867+0000","last_active":"2024-09-11T07:28:09.009583+0000","last_peered":"2024-09-11T07:28:09.009583+0000","last_clean":"2024-09-11T07:28:09.009583+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:28:09.009583+0000","last_undegraded":"2024-09-11T07:28:09.009583+0000","last_fullsized":"2024-09-11T07:28:09.009583+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:38.732795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:38.732795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:56.730699+0000","scrub_duration":0.0025091950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":88,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.025137+0000","last_change":"2024-09-11T07:28:09.024756+0000","last_active":"2024-09-11T07:28:09.025137+0000","last_peered":"2024-09-11T07:28:09.025137+0000","last_clean":"2024-09-11T07:28:09.025137+0000","last_became_active":"2024-09-11T07:28:09.017997+0000","last_became_peered":"2024-09-11T07:28:09.017997+0000","last_unstale":"2024-09-11T07:28:09.025137+0000","last_undegraded":"2024-09-11T07:28:09.025137+0000","last_fullsized":"2024-09-11T07:28:09.025137+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:28:03.164220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:03.164220+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:06.470841+0000","scrub_duration":0.0020677629999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:17.484604+0000","last_change":"2024-09-11T07:28:17.484549+0000","last_active":"2024-09-11T07:28:17.484604+0000","last_peered":"2024-09-11T07:28:17.484604+0000","last_clean":"2024-09-11T07:28:17.484604+0000","last_became_active":"2024-09-11T07:26:35.558809+0000","last_became_peered":"2024-09-11T07:26:35.558809+0000","last_unstale":"2024-09-11T07:28:17.484604+0000","last_undegraded":"2024-09-11T07:28:17.484604+0000","last_fullsized":"2024-09-11T07:28:17.484604+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:17.484488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:17.484488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.968070+0000","scrub_duration":0.001695118,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.010220+0000","last_change":"2024-09-11T07:28:00.126756+0000","last_active":"2024-09-11T07:28:09.010220+0000","last_peered":"2024-09-11T07:28:09.010220+0000","last_clean":"2024-09-11T07:28:09.010220+0000","last_became_active":"2024-09-11T07:26:37.862373+0000","last_became_peered":"2024-09-11T07:26:37.862373+0000","last_unstale":"2024-09-11T07:28:09.010220+0000","last_undegraded":"2024-09-11T07:28:09.010220+0000","last_fullsized":"2024-09-11T07:28:09.010220+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:00.126702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.126702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:18.534592+0000","scrub_duration":0.001184903,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.012206+0000","last_change":"2024-09-11T07:27:33.983889+0000","last_active":"2024-09-11T07:28:09.012206+0000","last_peered":"2024-09-11T07:28:09.012206+0000","last_clean":"2024-09-11T07:28:09.012206+0000","last_became_active":"2024-09-11T07:26:31.424651+0000","last_became_peered":"2024-09-11T07:26:31.424651+0000","last_unstale":"2024-09-11T07:28:09.012206+0000","last_undegraded":"2024-09-11T07:28:09.012206+0000","last_fullsized":"2024-09-11T07:28:09.012206+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:33.983813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:33.983813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:55.264233+0000","scrub_duration":0.001290759,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":72,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.009904+0000","last_change":"2024-09-11T07:28:00.272702+0000","last_active":"2024-09-11T07:28:09.009904+0000","last_peered":"2024-09-11T07:28:09.009904+0000","last_clean":"2024-09-11T07:28:09.009904+0000","last_became_active":"2024-09-11T07:26:33.542544+0000","last_became_peered":"2024-09-11T07:26:33.542544+0000","last_unstale":"2024-09-11T07:28:09.009904+0000","last_undegraded":"2024-09-11T07:28:09.009904+0000","last_fullsized":"2024-09-11T07:28:09.009904+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-09-11T07:28:00.272628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.272628+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:05.374102+0000","scrub_duration":0.0021830019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.014912+0000","last_change":"2024-09-11T07:28:07.721176+0000","last_active":"2024-09-11T07:28:09.014912+0000","last_peered":"2024-09-11T07:28:09.014912+0000","last_clean":"2024-09-11T07:28:09.014912+0000","last_became_active":"2024-09-11T07:26:35.555862+0000","last_became_peered":"2024-09-11T07:26:35.555862+0000","last_unstale":"2024-09-11T07:28:09.014912+0000","last_undegraded":"2024-09-11T07:28:09.014912+0000","last_fullsized":"2024-09-11T07:28:09.014912+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:07.721112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:07.721112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:31.345841+0000","scrub_duration":0.0020675020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.010825+0000","last_change":"2024-09-11T07:28:06.674616+0000","last_active":"2024-09-11T07:28:09.010825+0000","last_peered":"2024-09-11T07:28:09.010825+0000","last_clean":"2024-09-11T07:28:09.010825+0000","last_became_active":"2024-09-11T07:26:37.861647+0000","last_became_peered":"2024-09-11T07:26:37.861647+0000","last_unstale":"2024-09-11T07:28:09.010825+0000","last_undegraded":"2024-09-11T07:28:09.010825+0000","last_fullsized":"2024-09-11T07:28:09.010825+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:06.674558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:06.674558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:17.029204+0000","scrub_duration":0.001489488,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.009692+0000","last_change":"2024-09-11T07:27:45.732604+0000","last_active":"2024-09-11T07:28:09.009692+0000","last_peered":"2024-09-11T07:28:09.009692+0000","last_clean":"2024-09-11T07:28:09.009692+0000","last_became_active":"2024-09-11T07:26:31.420598+0000","last_became_peered":"2024-09-11T07:26:31.420598+0000","last_unstale":"2024-09-11T07:28:09.009692+0000","last_undegraded":"2024-09-11T07:28:09.009692+0000","last_fullsized":"2024-09-11T07:28:09.009692+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:45.732528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:45.732528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:48.605075+0000","scrub_duration":0.0015466659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"64'12","reported_seq":70,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.010377+0000","last_change":"2024-09-11T07:27:59.737848+0000","last_active":"2024-09-11T07:28:09.010377+0000","last_peered":"2024-09-11T07:28:09.010377+0000","last_clean":"2024-09-11T07:28:09.010377+0000","last_became_active":"2024-09-11T07:26:33.542711+0000","last_became_peered":"2024-09-11T07:26:33.542711+0000","last_unstale":"2024-09-11T07:28:09.010377+0000","last_undegraded":"2024-09-11T07:28:09.010377+0000","last_fullsized":"2024-09-11T07:28:09.010377+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-09-11T07:27:59.737791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:59.737791+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:11.558595+0000","scrub_duration":0.00367732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T07:28:09.010221+0000","last_change":"2024-09-11T07:27:57.199239+0000","last_active":"2024-09-11T07:28:09.010221+0000","last_peered":"2024-09-11T07:28:09.010221+0000","last_clean":"2024-09-11T07:28:09.010221+0000","last_became_active":"2024-09-11T07:26:35.562037+0000","last_became_peered":"2024-09-11T07:26:35.562037+0000","last_unstale":"2024-09-11T07:28:09.010221+0000","last_undegraded":"2024-09-11T07:28:09.010221+0000","last_fullsized":"2024-09-11T07:28:09.010221+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:57.199173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.199173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.093487+0000","scrub_duration":0.0015950770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":430,"ondisk_log_size":430,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":106,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299420,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298584,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446568,"statfs":{"total":95995035648,"available":95689285632,"internally_reserved":0,"allocated":1257472,"data_stored":896222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.50900000000000001,"15min":0.50800000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52700000000000002,"15min":0.52800000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69399999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.53100000000000003}]},{"osd":1,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.503},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.34899999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.66900000000000004,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.56200000000000006}]},{"osd":2,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52400000000000002,"15min":0.52300000000000002},"min":{"1min":0.34599999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69199999999999995,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53000000000000003,"15min":0.52800000000000002},"min":{"1min":0.38700000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73999999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.45100000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":1.0249999999999999,"15min":1.1080000000000001},"min":{"1min":0.34000000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70499999999999996,"5min":12.625999999999999,"15min":12.625999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.35899999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.72899999999999998,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.54900000000000004}]},{"osd":4,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.44,"15min":0.442},"min":{"1min":0.29799999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.63100000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.442,"15min":0.44400000000000001},"min":{"1min":0.28999999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.434}]},{"osd":5,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.436,"15min":0.44},"min":{"1min":0.33000000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.56200000000000006,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.33100000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70599999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.49199999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.34200000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67400000000000004,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.47899999999999998,"15min":0.47799999999999998},"min":{"1min":0.31,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70899999999999996,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54100000000000004}]}]},{"osd":6,"up_from":44,"seq":188978561056,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298560,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446592,"statfs":{"total":95995035648,"available":95689310208,"internally_reserved":0,"allocated":1232896,"data_stored":894700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.374,"5min":0.247,"15min":0.247},"max":{"1min":0.65000000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.60499999999999998,"15min":0.624},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.754,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.57899999999999996}]},{"osd":1,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.82899999999999996,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.49299999999999999,"15min":0.495},"min":{"1min":0.35799999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.54600000000000004}]},{"osd":2,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.61499999999999999,"15min":0.63500000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.70299999999999996,"5min":3.234,"15min":3.234},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.59099999999999997,"15min":0.60699999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.80200000000000005,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.59299999999999997}]},{"osd":3,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51500000000000001,"15min":0.51800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.63500000000000001,"15min":0.65500000000000003},"min":{"1min":0.41199999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.66900000000000004,"5min":3.1739999999999999,"15min":3.1739999999999999},"last":0.61299999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.58299999999999996,"15min":0.60899999999999999},"min":{"1min":0.32700000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.65200000000000002,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.28399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.501}]},{"osd":5,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.65600000000000003,"15min":0.69699999999999995},"min":{"1min":0.32400000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64000000000000001,"5min":5.7329999999999997,"15min":5.7329999999999997},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.52000000000000002,"15min":0.52700000000000002},"min":{"1min":0.34000000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70899999999999996,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.45600000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:28:23 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.58499999999999996,"15min":0.60499999999999998},"min":{"1min":0.34999999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.66300000000000003,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.34300000000000003,"5min":0.313,"15min":0.313},"max":{"1min":0.67300000000000004,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.56000000000000005}]}]},{"osd":5,"up_from":37,"seq":158913789990,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297988,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447164,"statfs":{"total":95995035648,"available":95689895936,"internally_reserved":0,"allocated":647168,"data_stored":305323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.35499999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.50600000000000001,"15min":0.502},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.51500000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.442},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.47699999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.53500000000000003}]},{"osd":3,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.435},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.55800000000000005}]},{"osd":4,"last update":"Wed Sep 11 07:27:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.375,"15min":0.37},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.42599999999999999,"15min":0.41999999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.39000000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:28:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.41599999999999998,"15min":0.40600000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.42699999999999999,"15min":0.42199999999999999},"min":{"1min":0.32100000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.34899999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:28:21 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.48099999999999998,"15min":0.48899999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53800000000000003,"5min":1.202,"15min":1.202},"last":0.376},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.50700000000000001,"15min":0.51300000000000001},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.61799999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.32100000000000001}]}]},{"osd":4,"up_from":31,"seq":133143986218,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298008,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447144,"statfs":{"total":95995035648,"available":95689875456,"internally_reserved":0,"allocated":667648,"data_stored":305658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:28:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52700000000000002,"15min":0.51700000000000002},"min":{"1min":0.42699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.499},"min":{"1min":0.34999999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.45200000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:28:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.50700000000000001,"15min":0.496},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72199999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.67100000000000004,"15min":0.72299999999999998},"min":{"1min":0.38100000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.81499999999999995,"5min":4.5510000000000002,"15min":4.5510000000000002},"last":0.47099999999999997}]},{"osd":2,"last update":"Wed Sep 11 07:28:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53600000000000003,"15min":0.53000000000000003},"min":{"1min":0.40200000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.73499999999999999,"5min":1.21,"15min":1.21},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.36899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76800000000000002,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.40999999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:28:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.38,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.70099999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.433},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.441}]},{"osd":5,"last update":"Wed Sep 11 07:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.34499999999999997,"15min":0.33800000000000002},"min":{"1min":0.27600000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.378,"15min":0.372},"min":{"1min":0.29399999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.30499999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:27:59 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42199999999999999,"15min":0.41799999999999998},"min":{"1min":0.315,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.44,"15min":0.43099999999999999},"min":{"1min":0.33500000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.41999999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48999999999999999,"15min":0.48699999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48099999999999998,"15min":0.48299999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.371}]}]},{"osd":3,"up_from":25,"seq":107374182447,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298016,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447136,"statfs":{"total":95995035648,"available":95689867264,"internally_reserved":0,"allocated":675840,"data_stored":305122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:54 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38600000000000001,"15min":0.39300000000000002},"min":{"1min":0.20799999999999999,"5min":0.185,"15min":0.185},"max":{"1min":0.54300000000000004,"5min":0.621,"15min":0.621},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.42499999999999999,"15min":0.441},"min":{"1min":0.26200000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.56200000000000006}]},{"osd":1,"last update":"Wed Sep 11 07:27:54 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.43099999999999999,"15min":0.437},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.625,"5min":0.752,"15min":0.752},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.41599999999999998,"15min":0.43099999999999999},"min":{"1min":0.23599999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.51400000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:27:54 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.45100000000000001,"15min":0.46500000000000002},"min":{"1min":0.26900000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.45400000000000001,"15min":0.46899999999999997},"min":{"1min":0.28899999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.54000000000000004}]},{"osd":4,"last update":"Wed Sep 11 07:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.747,"15min":0.621},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77200000000000002,"5min":21.856000000000002,"15min":21.856000000000002},"last":0.64900000000000002},{"interface":"front","average":{"1min":1.4399999999999999,"5min":0.89700000000000002,"15min":0.67700000000000005},"min":{"1min":0.436,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":18.370000000000001,"5min":19.146000000000001,"15min":19.146000000000001},"last":0.746}]},{"osd":5,"last update":"Wed Sep 11 07:27:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":1.3069999999999999,"5min":0.71999999999999997,"15min":0.622},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":18.416,"5min":18.416,"15min":18.416},"last":0.76800000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:27:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":1.306,"15min":1.4330000000000001},"min":{"1min":0.439,"5min":0.437,"15min":0.437},"max":{"1min":0.72899999999999998,"5min":20.219999999999999,"15min":20.219999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":1.319,"15min":1.446},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72699999999999998,"5min":19.609000000000002,"15min":19.609000000000002},"last":0.77700000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.45900000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.80700000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59799999999999998,"15min":0.60399999999999998},"min":{"1min":0.39700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71899999999999997,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.68700000000000006}]}]},{"osd":2,"up_from":18,"seq":77309411380,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297984,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447168,"statfs":{"total":95995035648,"available":95689900032,"internally_reserved":0,"allocated":643072,"data_stored":304569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:28:28 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.375,"15min":0.36299999999999999},"min":{"1min":0.20799999999999999,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.56299999999999994,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.371,"5min":0.38200000000000001,"15min":0.374},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57599999999999996,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.36199999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:28:28 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.39800000000000002,"15min":0.38400000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.499,"5min":1.179,"15min":1.179},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.41099999999999998,"15min":0.38600000000000001},"min":{"1min":0.255,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.66600000000000004,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.48599999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.439,"15min":0.42299999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64500000000000002,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":0.443},{"interface":"front","average":{"1min":0.44,"5min":0.45900000000000002,"15min":0.45400000000000001},"min":{"1min":0.26600000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65000000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.51100000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.94799999999999995,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.497}]},{"osd":5,"last update":"Wed Sep 11 07:27:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.68799999999999994,"15min":0.69699999999999995},"min":{"1min":0.45500000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85899999999999999,"5min":2.931,"15min":2.931},"last":0.52900000000000003}]},{"osd":6,"last update":"Wed Sep 11 07:27:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.85299999999999998,"5min":1.494,"15min":1.494},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.63100000000000001,"15min":0.63300000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.875,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.57499999999999996}]},{"osd":7,"last update":"Wed Sep 11 07:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.64700000000000002,"15min":0.65700000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.92700000000000005,"5min":1.514,"15min":1.514},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.63700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298024,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447128,"statfs":{"total":95995035648,"available":95689859072,"internally_reserved":0,"allocated":684032,"data_stored":305759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:28:04 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.34000000000000002,"15min":0.33100000000000002},"min":{"1min":0.22800000000000001,"5min":0.186,"15min":0.186},"max":{"1min":0.49299999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.36499999999999999,"15min":0.35499999999999998},"min":{"1min":0.22700000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.34499999999999997}]},{"osd":2,"last update":"Wed Sep 11 07:27:26 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.41899999999999998,"15min":0.41599999999999998},"min":{"1min":0.317,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.41199999999999998,"15min":0.40799999999999997},"min":{"1min":0.28699999999999998,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.28100000000000003}]},{"osd":3,"last update":"Wed Sep 11 07:27:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.44500000000000001,"15min":0.44700000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.60699999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.41399999999999998},"min":{"1min":0.307,"5min":0.253,"15min":0.253},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.30399999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54200000000000004,"15min":0.54700000000000004},"min":{"1min":0.34999999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.433},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71999999999999997,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.46800000000000003}]},{"osd":5,"last update":"Wed Sep 11 07:27:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.502},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.44,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.71699999999999997,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.44500000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:27:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57199999999999995,"15min":0.57499999999999996},"min":{"1min":0.41199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.754,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.40999999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.54800000000000004}]},{"osd":7,"last update":"Wed Sep 11 07:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.57399999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738429,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298576,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446576,"statfs":{"total":95995035648,"available":95689293824,"internally_reserved":0,"allocated":1249280,"data_stored":895032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:28:02 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.36699999999999999,"15min":0.34499999999999997},"min":{"1min":0.246,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.254},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.35899999999999999,"15min":0.34200000000000003},"min":{"1min":0.23799999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.27100000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:27:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.40400000000000003,"15min":0.40699999999999997},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.59299999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.30399999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:27:55 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45800000000000002,"15min":0.47399999999999998},"min":{"1min":0.317,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.61199999999999999,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.46300000000000002,"15min":0.48099999999999998},"min":{"1min":0.28299999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.56200000000000006,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.34499999999999997}]},{"osd":4,"last update":"Wed Sep 11 07:28:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56000000000000005,"15min":0.56799999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.77600000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.55700000000000005}]},{"osd":5,"last update":"Wed Sep 11 07:27:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.40000000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.79100000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.41699999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78100000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.58399999999999996}]},{"osd":6,"last update":"Wed Sep 11 07:27:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59399999999999997,"15min":0.59699999999999998},"min":{"1min":0.34799999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60099999999999998,"15min":0.60399999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.76400000000000001,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.48099999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:28:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58499999999999996,"15min":0.59299999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.748,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.498},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.42799999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.61199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:28:30.739 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T07:28:30.739 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd dump 2024-09-11T07:28:30.959 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:30 smithi104 ceph-mon[33507]: 5.13 scrub starts 2024-09-11T07:28:30.959 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:30 smithi104 ceph-mon[33507]: 5.13 scrub ok 2024-09-11T07:28:30.959 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:30 smithi104 ceph-mon[33507]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:28:30.959 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:30 smithi104 ceph-mon[33507]: from='client.24709 v1:172.21.15.22:0/1404111803' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:28:31.045 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:30 smithi022 ceph-mon[37330]: 5.13 scrub starts 2024-09-11T07:28:31.046 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:30 smithi022 ceph-mon[37330]: 5.13 scrub ok 2024-09-11T07:28:31.046 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:30 smithi022 ceph-mon[37330]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:28:31.046 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:30 smithi022 ceph-mon[37330]: from='client.24709 v1:172.21.15.22:0/1404111803' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:28:31.046 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:30 smithi022 ceph-mon[29462]: 5.13 scrub starts 2024-09-11T07:28:31.046 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:30 smithi022 ceph-mon[29462]: 5.13 scrub ok 2024-09-11T07:28:31.046 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:30 smithi022 ceph-mon[29462]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:28:31.046 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:30 smithi022 ceph-mon[29462]: from='client.24709 v1:172.21.15.22:0/1404111803' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:28:31.572 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:28:31.974 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:31 smithi022 ceph-mon[29462]: 6.2 deep-scrub starts 2024-09-11T07:28:31.974 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:31 smithi022 ceph-mon[29462]: 6.2 deep-scrub ok 2024-09-11T07:28:31.974 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:31 smithi022 ceph-mon[29462]: 4.0 deep-scrub starts 2024-09-11T07:28:31.974 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:31 smithi022 ceph-mon[29462]: 4.0 deep-scrub ok 2024-09-11T07:28:31.974 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:31 smithi022 ceph-mon[29462]: 5.1a deep-scrub starts 2024-09-11T07:28:31.974 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:31 smithi022 ceph-mon[29462]: 5.1a deep-scrub ok 2024-09-11T07:28:31.974 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:31 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:31.974 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:31 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:31.975 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:31 smithi022 ceph-mon[37330]: 6.2 deep-scrub starts 2024-09-11T07:28:31.975 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:31 smithi022 ceph-mon[37330]: 6.2 deep-scrub ok 2024-09-11T07:28:31.975 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:31 smithi022 ceph-mon[37330]: 4.0 deep-scrub starts 2024-09-11T07:28:31.975 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:31 smithi022 ceph-mon[37330]: 4.0 deep-scrub ok 2024-09-11T07:28:31.975 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:31 smithi022 ceph-mon[37330]: 5.1a deep-scrub starts 2024-09-11T07:28:31.975 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:31 smithi022 ceph-mon[37330]: 5.1a deep-scrub ok 2024-09-11T07:28:31.975 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:31 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:31.975 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:31 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:32.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:31 smithi104 ceph-mon[33507]: 6.2 deep-scrub starts 2024-09-11T07:28:32.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:31 smithi104 ceph-mon[33507]: 6.2 deep-scrub ok 2024-09-11T07:28:32.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:31 smithi104 ceph-mon[33507]: 4.0 deep-scrub starts 2024-09-11T07:28:32.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:31 smithi104 ceph-mon[33507]: 4.0 deep-scrub ok 2024-09-11T07:28:32.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:31 smithi104 ceph-mon[33507]: 5.1a deep-scrub starts 2024-09-11T07:28:32.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:31 smithi104 ceph-mon[33507]: 5.1a deep-scrub ok 2024-09-11T07:28:32.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:31 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:32.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:31 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:32.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[29462]: 5.12 deep-scrub starts 2024-09-11T07:28:32.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[29462]: 5.12 deep-scrub ok 2024-09-11T07:28:32.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[29462]: 6.e scrub starts 2024-09-11T07:28:32.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[29462]: 6.e scrub ok 2024-09-11T07:28:32.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[29462]: 6.13 scrub starts 2024-09-11T07:28:32.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[29462]: 6.13 scrub ok 2024-09-11T07:28:32.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[29462]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:28:32.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[29462]: 5.9 scrub starts 2024-09-11T07:28:32.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[29462]: 5.9 scrub ok 2024-09-11T07:28:32.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:32.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:32.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:32.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:32.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:32.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[37330]: 5.12 deep-scrub starts 2024-09-11T07:28:32.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[37330]: 5.12 deep-scrub ok 2024-09-11T07:28:32.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[37330]: 6.e scrub starts 2024-09-11T07:28:32.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[37330]: 6.e scrub ok 2024-09-11T07:28:32.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[37330]: 6.13 scrub starts 2024-09-11T07:28:32.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[37330]: 6.13 scrub ok 2024-09-11T07:28:32.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[37330]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:28:32.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[37330]: 5.9 scrub starts 2024-09-11T07:28:32.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[37330]: 5.9 scrub ok 2024-09-11T07:28:32.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:32.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:32.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:32.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:32.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:32 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:33.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:32 smithi104 ceph-mon[33507]: 5.12 deep-scrub starts 2024-09-11T07:28:33.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:32 smithi104 ceph-mon[33507]: 5.12 deep-scrub ok 2024-09-11T07:28:33.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:32 smithi104 ceph-mon[33507]: 6.e scrub starts 2024-09-11T07:28:33.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:32 smithi104 ceph-mon[33507]: 6.e scrub ok 2024-09-11T07:28:33.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:32 smithi104 ceph-mon[33507]: 6.13 scrub starts 2024-09-11T07:28:33.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:32 smithi104 ceph-mon[33507]: 6.13 scrub ok 2024-09-11T07:28:33.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:32 smithi104 ceph-mon[33507]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:28:33.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:32 smithi104 ceph-mon[33507]: 5.9 scrub starts 2024-09-11T07:28:33.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:32 smithi104 ceph-mon[33507]: 5.9 scrub ok 2024-09-11T07:28:33.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:32 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:33.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:32 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:33.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:32 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:28:33.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:32 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:28:33.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:32 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:33.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:33 smithi022 ceph-mon[29462]: 6.1b scrub starts 2024-09-11T07:28:33.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:33 smithi022 ceph-mon[29462]: 6.1b scrub ok 2024-09-11T07:28:33.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:33 smithi022 ceph-mon[37330]: 6.1b scrub starts 2024-09-11T07:28:33.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:33 smithi022 ceph-mon[37330]: 6.1b scrub ok 2024-09-11T07:28:34.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:33 smithi104 ceph-mon[33507]: 6.1b scrub starts 2024-09-11T07:28:34.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:33 smithi104 ceph-mon[33507]: 6.1b scrub ok 2024-09-11T07:28:34.041 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 87 2024-09-11T07:28:34.041 INFO:teuthology.orchestra.run.smithi022.stdout:fsid 12a285d8-700e-11ef-bcea-c7b262605968 2024-09-11T07:28:34.041 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-09-11T07:21:18.695332+0000 2024-09-11T07:28:34.041 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2024-09-11T07:28:27.659108+0000 2024-09-11T07:28:34.041 INFO:teuthology.orchestra.run.smithi022.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-11T07:28:34.041 INFO:teuthology.orchestra.run.smithi022.stdout:crush_version 18 2024-09-11T07:28:34.041 INFO:teuthology.orchestra.run.smithi022.stdout:full_ratio 0.95 2024-09-11T07:28:34.041 INFO:teuthology.orchestra.run.smithi022.stdout:backfillfull_ratio 0.9 2024-09-11T07:28:34.042 INFO:teuthology.orchestra.run.smithi022.stdout:nearfull_ratio 0.85 2024-09-11T07:28:34.042 INFO:teuthology.orchestra.run.smithi022.stdout:require_min_compat_client luminous 2024-09-11T07:28:34.042 INFO:teuthology.orchestra.run.smithi022.stdout:min_compat_client luminous 2024-09-11T07:28:34.042 INFO:teuthology.orchestra.run.smithi022.stdout:require_osd_release quincy 2024-09-11T07:28:34.042 INFO:teuthology.orchestra.run.smithi022.stdout:stretch_mode_enabled false 2024-09-11T07:28:34.042 INFO:teuthology.orchestra.run.smithi022.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr 2024-09-11T07:28:34.042 INFO:teuthology.orchestra.run.smithi022.stdout:pool 2 '.rgw.root' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 58 flags hashpspool stripe_width 0 application rgw 2024-09-11T07:28:34.042 INFO:teuthology.orchestra.run.smithi022.stdout:pool 3 'datapool' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 3 pgp_num 3 autoscale_mode off last_change 61 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd 2024-09-11T07:28:34.042 INFO:teuthology.orchestra.run.smithi022.stdout:pool 4 'default.rgw.log' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 60 flags hashpspool stripe_width 0 application rgw 2024-09-11T07:28:34.042 INFO:teuthology.orchestra.run.smithi022.stdout:pool 5 'default.rgw.control' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 62 flags hashpspool stripe_width 0 application rgw 2024-09-11T07:28:34.042 INFO:teuthology.orchestra.run.smithi022.stdout:pool 6 'default.rgw.meta' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 64 flags hashpspool stripe_width 0 pg_autoscale_bias 4 application rgw 2024-09-11T07:28:34.042 INFO:teuthology.orchestra.run.smithi022.stdout:max_osd 8 2024-09-11T07:28:34.042 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 up in weight 1 up_from 8 up_thru 85 down_at 0 last_clean_interval [0,0) v1:172.21.15.22:6801/2821493591 v1:172.21.15.22:6802/2821493591 exists,up 09075f16-32f4-4f88-a116-c3d7f977a0e0 2024-09-11T07:28:34.042 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 up in weight 1 up_from 13 up_thru 62 down_at 0 last_clean_interval [0,0) v1:172.21.15.22:6805/4037911317 v1:172.21.15.22:6806/4037911317 exists,up 52cc1e06-9506-48c0-9f7b-d4270dff4b9f 2024-09-11T07:28:34.042 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 up in weight 1 up_from 18 up_thru 62 down_at 0 last_clean_interval [0,0) v1:172.21.15.22:6809/1675799666 v1:172.21.15.22:6810/1675799666 exists,up fa2b2730-2c1d-45d0-be8a-494a60dc6b9b 2024-09-11T07:28:34.043 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 up in weight 1 up_from 25 up_thru 62 down_at 0 last_clean_interval [0,0) v1:172.21.15.22:6813/1501962752 v1:172.21.15.22:6814/1501962752 exists,up 8d53d3fc-e401-47df-b70b-4b4b1f4539df 2024-09-11T07:28:34.043 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 up in weight 1 up_from 31 up_thru 85 down_at 0 last_clean_interval [0,0) v1:172.21.15.104:6800/1210229443 v1:172.21.15.104:6801/1210229443 exists,up d3207fe0-8b97-4067-a755-ba6d449c53c7 2024-09-11T07:28:34.043 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 up in weight 1 up_from 37 up_thru 62 down_at 0 last_clean_interval [0,0) v1:172.21.15.104:6804/2735328732 v1:172.21.15.104:6805/2735328732 exists,up 89c9352b-30b2-48c6-869e-047d07a23a43 2024-09-11T07:28:34.043 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6 up in weight 1 up_from 44 up_thru 85 down_at 0 last_clean_interval [0,0) v1:172.21.15.104:6808/721493010 v1:172.21.15.104:6809/721493010 exists,up 704563de-d88f-480a-b469-720135704d42 2024-09-11T07:28:34.043 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7 up in weight 1 up_from 52 up_thru 62 down_at 0 last_clean_interval [0,0) v1:172.21.15.104:6812/3018423954 v1:172.21.15.104:6813/3018423954 exists,up 18199e47-0052-49f0-b45f-0fb201e0d0f4 2024-09-11T07:28:34.043 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 2.7 [7,4] 2024-09-11T07:28:34.043 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 4.6 [1,5] 2024-09-11T07:28:34.043 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 4.d [1,5] 2024-09-11T07:28:34.043 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 4.1f [1,2] 2024-09-11T07:28:34.043 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/240999923 expires 2024-09-12T07:27:07.919998+0000 2024-09-11T07:28:34.043 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:6800/4016369460 expires 2024-09-12T07:21:48.438371+0000 2024-09-11T07:28:34.043 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/585929238 expires 2024-09-12T07:22:11.238618+0000 2024-09-11T07:28:34.043 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/544031024 expires 2024-09-12T07:27:07.919998+0000 2024-09-11T07:28:34.043 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:6800/2504196132 expires 2024-09-12T07:27:07.919998+0000 2024-09-11T07:28:34.043 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/1451388349 expires 2024-09-12T07:27:07.919998+0000 2024-09-11T07:28:34.044 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/639336081 expires 2024-09-12T07:21:48.438371+0000 2024-09-11T07:28:34.044 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/2104069730 expires 2024-09-12T07:27:07.919998+0000 2024-09-11T07:28:34.044 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/4236450731 expires 2024-09-12T07:21:48.438371+0000 2024-09-11T07:28:34.044 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:6800/3573193776 expires 2024-09-12T07:22:11.238618+0000 2024-09-11T07:28:34.044 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/2661764422 expires 2024-09-12T07:21:48.438371+0000 2024-09-11T07:28:34.044 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/2197488530 expires 2024-09-12T07:22:11.238618+0000 2024-09-11T07:28:34.044 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/2968034970 expires 2024-09-12T07:22:11.238618+0000 2024-09-11T07:28:34.044 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/599318175 expires 2024-09-12T07:27:07.919998+0000 2024-09-11T07:28:34.530 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:28:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:28:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:28:34.750 INFO:tasks.cephadm.ceph_manager.ceph:['osd.0 up in weight 1 up_from 8 up_thru 85 down_at 0 last_clean_interval [0,0) v1:172.21.15.22:6801/2821493591 v1:172.21.15.22:6802/2821493591 exists,up 09075f16-32f4-4f88-a116-c3d7f977a0e0', 'osd.1 up in weight 1 up_from 13 up_thru 62 down_at 0 last_clean_interval [0,0) v1:172.21.15.22:6805/4037911317 v1:172.21.15.22:6806/4037911317 exists,up 52cc1e06-9506-48c0-9f7b-d4270dff4b9f', 'osd.2 up in weight 1 up_from 18 up_thru 62 down_at 0 last_clean_interval [0,0) v1:172.21.15.22:6809/1675799666 v1:172.21.15.22:6810/1675799666 exists,up fa2b2730-2c1d-45d0-be8a-494a60dc6b9b', 'osd.3 up in weight 1 up_from 25 up_thru 62 down_at 0 last_clean_interval [0,0) v1:172.21.15.22:6813/1501962752 v1:172.21.15.22:6814/1501962752 exists,up 8d53d3fc-e401-47df-b70b-4b4b1f4539df', 'osd.4 up in weight 1 up_from 31 up_thru 85 down_at 0 last_clean_interval [0,0) v1:172.21.15.104:6800/1210229443 v1:172.21.15.104:6801/1210229443 exists,up d3207fe0-8b97-4067-a755-ba6d449c53c7', 'osd.5 up in weight 1 up_from 37 up_thru 62 down_at 0 last_clean_interval [0,0) v1:172.21.15.104:6804/2735328732 v1:172.21.15.104:6805/2735328732 exists,up 89c9352b-30b2-48c6-869e-047d07a23a43', 'osd.6 up in weight 1 up_from 44 up_thru 85 down_at 0 last_clean_interval [0,0) v1:172.21.15.104:6808/721493010 v1:172.21.15.104:6809/721493010 exists,up 704563de-d88f-480a-b469-720135704d42', 'osd.7 up in weight 1 up_from 52 up_thru 62 down_at 0 last_clean_interval [0,0) v1:172.21.15.104:6812/3018423954 v1:172.21.15.104:6813/3018423954 exists,up 18199e47-0052-49f0-b45f-0fb201e0d0f4'] 2024-09-11T07:28:34.751 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon mon.a config show 2024-09-11T07:28:35.010 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:28:35.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:34 smithi104 ceph-mon[33507]: 2.6 scrub starts 2024-09-11T07:28:35.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:34 smithi104 ceph-mon[33507]: 2.6 scrub ok 2024-09-11T07:28:35.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:34 smithi104 ceph-mon[33507]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:28:35.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:34 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3799443237' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T07:28:35.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:34 smithi104 ceph-mon[33507]: 6.3 scrub starts 2024-09-11T07:28:35.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:34 smithi104 ceph-mon[33507]: 6.3 scrub ok 2024-09-11T07:28:35.131 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:34 smithi022 ceph-mon[29462]: 2.6 scrub starts 2024-09-11T07:28:35.131 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:34 smithi022 ceph-mon[29462]: 2.6 scrub ok 2024-09-11T07:28:35.131 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:34 smithi022 ceph-mon[29462]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:28:35.131 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:34 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3799443237' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T07:28:35.131 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:34 smithi022 ceph-mon[29462]: 6.3 scrub starts 2024-09-11T07:28:35.131 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:34 smithi022 ceph-mon[29462]: 6.3 scrub ok 2024-09-11T07:28:35.132 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:34 smithi022 ceph-mon[37330]: 2.6 scrub starts 2024-09-11T07:28:35.132 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:34 smithi022 ceph-mon[37330]: 2.6 scrub ok 2024-09-11T07:28:35.132 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:34 smithi022 ceph-mon[37330]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:28:35.132 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:34 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3799443237' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T07:28:35.132 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:34 smithi022 ceph-mon[37330]: 6.3 scrub starts 2024-09-11T07:28:35.132 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:34 smithi022 ceph-mon[37330]: 6.3 scrub ok 2024-09-11T07:28:36.641 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:28:36.641 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mon.a", 2024-09-11T07:28:36.641 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster": "ceph", 2024-09-11T07:28:36.641 INFO:teuthology.orchestra.run.smithi022.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-11T07:28:36.642 INFO:teuthology.orchestra.run.smithi022.stdout: "admin_socket_mode": "", 2024-09-11T07:28:36.642 INFO:teuthology.orchestra.run.smithi022.stdout: "auth_allow_insecure_global_id_reclaim": "false", 2024-09-11T07:28:36.642 INFO:teuthology.orchestra.run.smithi022.stdout: "auth_client_required": "cephx, none", 2024-09-11T07:28:36.642 INFO:teuthology.orchestra.run.smithi022.stdout: "auth_cluster_required": "cephx", 2024-09-11T07:28:36.642 INFO:teuthology.orchestra.run.smithi022.stdout: "auth_debug": "true", 2024-09-11T07:28:36.642 INFO:teuthology.orchestra.run.smithi022.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-11T07:28:36.642 INFO:teuthology.orchestra.run.smithi022.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-11T07:28:36.642 INFO:teuthology.orchestra.run.smithi022.stdout: "auth_service_required": "cephx", 2024-09-11T07:28:36.642 INFO:teuthology.orchestra.run.smithi022.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-11T07:28:36.642 INFO:teuthology.orchestra.run.smithi022.stdout: "auth_supported": "", 2024-09-11T07:28:36.642 INFO:teuthology.orchestra.run.smithi022.stdout: "bdev_aio": "true", 2024-09-11T07:28:36.642 INFO:teuthology.orchestra.run.smithi022.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-11T07:28:36.642 INFO:teuthology.orchestra.run.smithi022.stdout: "bdev_aio_poll_ms": "250", 2024-09-11T07:28:36.643 INFO:teuthology.orchestra.run.smithi022.stdout: "bdev_aio_reap_max": "16", 2024-09-11T07:28:36.643 INFO:teuthology.orchestra.run.smithi022.stdout: "bdev_async_discard": "false", 2024-09-11T07:28:36.643 INFO:teuthology.orchestra.run.smithi022.stdout: "bdev_block_size": "4096", 2024-09-11T07:28:36.643 INFO:teuthology.orchestra.run.smithi022.stdout: "bdev_debug_aio": "false", 2024-09-11T07:28:36.643 INFO:teuthology.orchestra.run.smithi022.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-11T07:28:36.643 INFO:teuthology.orchestra.run.smithi022.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-11T07:28:36.643 INFO:teuthology.orchestra.run.smithi022.stdout: "bdev_debug_inflight_ios": "false", 2024-09-11T07:28:36.643 INFO:teuthology.orchestra.run.smithi022.stdout: "bdev_enable_discard": "false", 2024-09-11T07:28:36.644 INFO:teuthology.orchestra.run.smithi022.stdout: "bdev_flock_retry": "3", 2024-09-11T07:28:36.644 INFO:teuthology.orchestra.run.smithi022.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-11T07:28:36.644 INFO:teuthology.orchestra.run.smithi022.stdout: "bdev_inject_crash": "0", 2024-09-11T07:28:36.644 INFO:teuthology.orchestra.run.smithi022.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-11T07:28:36.644 INFO:teuthology.orchestra.run.smithi022.stdout: "bdev_ioring": "false", 2024-09-11T07:28:36.644 INFO:teuthology.orchestra.run.smithi022.stdout: "bdev_ioring_hipri": "false", 2024-09-11T07:28:36.644 INFO:teuthology.orchestra.run.smithi022.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-11T07:28:36.644 INFO:teuthology.orchestra.run.smithi022.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-11T07:28:36.644 INFO:teuthology.orchestra.run.smithi022.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-11T07:28:36.644 INFO:teuthology.orchestra.run.smithi022.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-11T07:28:36.644 INFO:teuthology.orchestra.run.smithi022.stdout: "bdev_type": "", 2024-09-11T07:28:36.644 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_alloc_size": "1048576", 2024-09-11T07:28:36.644 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_allocator": "hybrid", 2024-09-11T07:28:36.645 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_buffered_io": "true", 2024-09-11T07:28:36.645 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_check_for_zeros": "false", 2024-09-11T07:28:36.645 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-11T07:28:36.645 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-11T07:28:36.645 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_compact_log_sync": "false", 2024-09-11T07:28:36.645 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-11T07:28:36.645 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-11T07:28:36.645 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-11T07:28:36.645 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-11T07:28:36.645 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_max_log_runway": "4194304", 2024-09-11T07:28:36.646 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_max_prefetch": "1048576", 2024-09-11T07:28:36.646 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_min_flush_size": "524288", 2024-09-11T07:28:36.646 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_min_log_runway": "1048576", 2024-09-11T07:28:36.646 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_replay_recovery": "false", 2024-09-11T07:28:36.646 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-11T07:28:36.647 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-11T07:28:36.647 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_sync_write": "false", 2024-09-11T07:28:36.647 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-11T07:28:36.647 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-11T07:28:36.647 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-11T07:28:36.647 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_allocation_from_file": "true", 2024-09-11T07:28:36.647 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_allocator": "hybrid", 2024-09-11T07:28:36.648 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_async_db_compaction": "true", 2024-09-11T07:28:36.648 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-11T07:28:36.648 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-11T07:28:36.648 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-11T07:28:36.648 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-11T07:28:36.648 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-11T07:28:36.648 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-11T07:28:36.649 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-11T07:28:36.649 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_block_create": "true", 2024-09-11T07:28:36.649 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_block_db_create": "false", 2024-09-11T07:28:36.649 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_block_db_path": "", 2024-09-11T07:28:36.649 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_block_db_size": "0", 2024-09-11T07:28:36.649 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_block_path": "", 2024-09-11T07:28:36.649 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_block_preallocate_file": "false", 2024-09-11T07:28:36.649 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_block_size": "107374182400", 2024-09-11T07:28:36.649 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_block_wal_create": "false", 2024-09-11T07:28:36.649 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_block_wal_path": "", 2024-09-11T07:28:36.649 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_block_wal_size": "100663296", 2024-09-11T07:28:36.649 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_bluefs": "true", 2024-09-11T07:28:36.649 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-11T07:28:36.650 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-11T07:28:36.650 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-11T07:28:36.650 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-11T07:28:36.650 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-11T07:28:36.650 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-11T07:28:36.650 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-11T07:28:36.650 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-11T07:28:36.650 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_autotune": "true", 2024-09-11T07:28:36.650 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-11T07:28:36.650 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-11T07:28:36.650 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-11T07:28:36.650 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-11T07:28:36.650 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_size": "0", 2024-09-11T07:28:36.650 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-11T07:28:36.650 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-11T07:28:36.651 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-11T07:28:36.651 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-11T07:28:36.651 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_type": "2q", 2024-09-11T07:28:36.651 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-11T07:28:36.651 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_clone_cow": "true", 2024-09-11T07:28:36.651 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-11T07:28:36.651 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-11T07:28:36.651 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-11T07:28:36.651 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-11T07:28:36.651 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-11T07:28:36.651 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-11T07:28:36.651 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-11T07:28:36.651 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_compression_mode": "none", 2024-09-11T07:28:36.651 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-11T07:28:36.652 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_csum_type": "crc32c", 2024-09-11T07:28:36.652 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-11T07:28:36.652 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_debug_freelist": "false", 2024-09-11T07:28:36.652 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-11T07:28:36.652 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-11T07:28:36.652 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-11T07:28:36.652 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-11T07:28:36.652 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-11T07:28:36.652 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-11T07:28:36.652 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-11T07:28:36.652 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-11T07:28:36.652 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-11T07:28:36.653 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-11T07:28:36.653 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-11T07:28:36.653 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-11T07:28:36.653 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-11T07:28:36.653 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-11T07:28:36.653 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_debug_small_allocations": "0", 2024-09-11T07:28:36.653 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-11T07:28:36.653 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_default_buffered_read": "true", 2024-09-11T07:28:36.653 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_default_buffered_write": "false", 2024-09-11T07:28:36.653 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-11T07:28:36.653 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-11T07:28:36.653 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-11T07:28:36.653 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-11T07:28:36.653 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-11T07:28:36.654 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-11T07:28:36.654 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-11T07:28:36.654 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-11T07:28:36.654 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fail_eio": "false", 2024-09-11T07:28:36.654 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-11T07:28:36.654 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-11T07:28:36.654 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-11T07:28:36.654 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-11T07:28:36.654 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-11T07:28:36.654 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-11T07:28:36.654 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_on_mount": "false", 2024-09-11T07:28:36.654 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-11T07:28:36.654 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_on_umount": "false", 2024-09-11T07:28:36.654 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-11T07:28:36.654 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-11T07:28:36.655 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-11T07:28:36.655 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-11T07:28:36.655 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-11T07:28:36.655 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-11T07:28:36.655 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-11T07:28:36.655 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-11T07:28:36.655 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_ignore_data_csum": "false", 2024-09-11T07:28:36.655 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-11T07:28:36.655 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-11T07:28:36.655 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-11T07:28:36.655 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-11T07:28:36.655 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_log_op_age": "5.000000", 2024-09-11T07:28:36.656 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_max_alloc_size": "0", 2024-09-11T07:28:36.656 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_max_blob_size": "0", 2024-09-11T07:28:36.656 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-11T07:28:36.656 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-11T07:28:36.656 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-11T07:28:36.656 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_max_deferred_txc": "32", 2024-09-11T07:28:36.656 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_min_alloc_size": "0", 2024-09-11T07:28:36.656 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-11T07:28:36.656 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-11T07:28:36.656 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_nid_prealloc": "1024", 2024-09-11T07:28:36.656 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-11T07:28:36.657 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-11T07:28:36.657 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-11T07:28:36.657 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-11T07:28:36.657 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_retry_disk_reads": "3", 2024-09-11T07:28:36.657 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_rocksdb_cf": "true", 2024-09-11T07:28:36.657 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P", 2024-09-11T07:28:36.657 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824", 2024-09-11T07:28:36.657 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-11T07:28:36.657 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-11T07:28:36.657 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-11T07:28:36.657 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-11T07:28:36.657 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_spdk_mem": "512", 2024-09-11T07:28:36.657 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-11T07:28:36.658 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-11T07:28:36.658 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-11T07:28:36.658 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-11T07:28:36.658 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-11T07:28:36.658 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-11T07:28:36.658 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-11T07:28:36.658 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_tracing": "false", 2024-09-11T07:28:36.658 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-11T07:28:36.658 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-11T07:28:36.658 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-11T07:28:36.658 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-11T07:28:36.658 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-11T07:28:36.658 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-11T07:28:36.658 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-11T07:28:36.659 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-11T07:28:36.659 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-11T07:28:36.659 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_zero_block_detection": "false", 2024-09-11T07:28:36.659 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-11T07:28:36.659 INFO:teuthology.orchestra.run.smithi022.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-11T07:28:36.659 INFO:teuthology.orchestra.run.smithi022.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-11T07:28:36.659 INFO:teuthology.orchestra.run.smithi022.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-11T07:28:36.659 INFO:teuthology.orchestra.run.smithi022.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-11T07:28:36.659 INFO:teuthology.orchestra.run.smithi022.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-11T07:28:36.659 INFO:teuthology.orchestra.run.smithi022.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-11T07:28:36.659 INFO:teuthology.orchestra.run.smithi022.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-11T07:28:36.659 INFO:teuthology.orchestra.run.smithi022.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-11T07:28:36.659 INFO:teuthology.orchestra.run.smithi022.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-11T07:28:36.660 INFO:teuthology.orchestra.run.smithi022.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-11T07:28:36.660 INFO:teuthology.orchestra.run.smithi022.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-11T07:28:36.660 INFO:teuthology.orchestra.run.smithi022.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-11T07:28:36.660 INFO:teuthology.orchestra.run.smithi022.stdout: "cephx_cluster_require_signatures": "false", 2024-09-11T07:28:36.660 INFO:teuthology.orchestra.run.smithi022.stdout: "cephx_cluster_require_version": "2", 2024-09-11T07:28:36.660 INFO:teuthology.orchestra.run.smithi022.stdout: "cephx_require_signatures": "false", 2024-09-11T07:28:36.660 INFO:teuthology.orchestra.run.smithi022.stdout: "cephx_require_version": "2", 2024-09-11T07:28:36.660 INFO:teuthology.orchestra.run.smithi022.stdout: "cephx_service_require_signatures": "false", 2024-09-11T07:28:36.660 INFO:teuthology.orchestra.run.smithi022.stdout: "cephx_service_require_version": "2", 2024-09-11T07:28:36.660 INFO:teuthology.orchestra.run.smithi022.stdout: "cephx_sign_messages": "true", 2024-09-11T07:28:36.660 INFO:teuthology.orchestra.run.smithi022.stdout: "chdir": "", 2024-09-11T07:28:36.661 INFO:teuthology.orchestra.run.smithi022.stdout: "client_acl_type": "", 2024-09-11T07:28:36.661 INFO:teuthology.orchestra.run.smithi022.stdout: "client_asio_thread_count": "2", 2024-09-11T07:28:36.661 INFO:teuthology.orchestra.run.smithi022.stdout: "client_cache_mid": "0.750000", 2024-09-11T07:28:36.661 INFO:teuthology.orchestra.run.smithi022.stdout: "client_cache_size": "16384", 2024-09-11T07:28:36.661 INFO:teuthology.orchestra.run.smithi022.stdout: "client_caps_release_delay": "5", 2024-09-11T07:28:36.661 INFO:teuthology.orchestra.run.smithi022.stdout: "client_check_pool_perm": "true", 2024-09-11T07:28:36.661 INFO:teuthology.orchestra.run.smithi022.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-11T07:28:36.661 INFO:teuthology.orchestra.run.smithi022.stdout: "client_debug_force_sync_read": "false", 2024-09-11T07:28:36.661 INFO:teuthology.orchestra.run.smithi022.stdout: "client_debug_getattr_caps": "false", 2024-09-11T07:28:36.661 INFO:teuthology.orchestra.run.smithi022.stdout: "client_debug_inject_features": "", 2024-09-11T07:28:36.661 INFO:teuthology.orchestra.run.smithi022.stdout: "client_debug_inject_tick_delay": "0", 2024-09-11T07:28:36.662 INFO:teuthology.orchestra.run.smithi022.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-11T07:28:36.662 INFO:teuthology.orchestra.run.smithi022.stdout: "client_die_on_failed_remount": "false", 2024-09-11T07:28:36.662 INFO:teuthology.orchestra.run.smithi022.stdout: "client_dirsize_rbytes": "true", 2024-09-11T07:28:36.662 INFO:teuthology.orchestra.run.smithi022.stdout: "client_force_lazyio": "false", 2024-09-11T07:28:36.662 INFO:teuthology.orchestra.run.smithi022.stdout: "client_fs": "", 2024-09-11T07:28:36.662 INFO:teuthology.orchestra.run.smithi022.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-11T07:28:36.662 INFO:teuthology.orchestra.run.smithi022.stdout: "client_inject_release_failure": "false", 2024-09-11T07:28:36.662 INFO:teuthology.orchestra.run.smithi022.stdout: "client_max_inline_size": "4096", 2024-09-11T07:28:36.662 INFO:teuthology.orchestra.run.smithi022.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-11T07:28:36.662 INFO:teuthology.orchestra.run.smithi022.stdout: "client_mds_namespace": "", 2024-09-11T07:28:36.662 INFO:teuthology.orchestra.run.smithi022.stdout: "client_metadata": "", 2024-09-11T07:28:36.662 INFO:teuthology.orchestra.run.smithi022.stdout: "client_mount_gid": "-1", 2024-09-11T07:28:36.662 INFO:teuthology.orchestra.run.smithi022.stdout: "client_mount_timeout": "300", 2024-09-11T07:28:36.662 INFO:teuthology.orchestra.run.smithi022.stdout: "client_mount_uid": "-1", 2024-09-11T07:28:36.663 INFO:teuthology.orchestra.run.smithi022.stdout: "client_mountpoint": "/", 2024-09-11T07:28:36.663 INFO:teuthology.orchestra.run.smithi022.stdout: "client_notify_timeout": "10", 2024-09-11T07:28:36.663 INFO:teuthology.orchestra.run.smithi022.stdout: "client_oc": "true", 2024-09-11T07:28:36.663 INFO:teuthology.orchestra.run.smithi022.stdout: "client_oc_max_dirty": "104857600", 2024-09-11T07:28:36.663 INFO:teuthology.orchestra.run.smithi022.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-11T07:28:36.663 INFO:teuthology.orchestra.run.smithi022.stdout: "client_oc_max_objects": "1000", 2024-09-11T07:28:36.663 INFO:teuthology.orchestra.run.smithi022.stdout: "client_oc_size": "209715200", 2024-09-11T07:28:36.663 INFO:teuthology.orchestra.run.smithi022.stdout: "client_oc_target_dirty": "8388608", 2024-09-11T07:28:36.663 INFO:teuthology.orchestra.run.smithi022.stdout: "client_permissions": "true", 2024-09-11T07:28:36.663 INFO:teuthology.orchestra.run.smithi022.stdout: "client_quota": "true", 2024-09-11T07:28:36.663 INFO:teuthology.orchestra.run.smithi022.stdout: "client_quota_df": "true", 2024-09-11T07:28:36.663 INFO:teuthology.orchestra.run.smithi022.stdout: "client_readahead_max_bytes": "0", 2024-09-11T07:28:36.663 INFO:teuthology.orchestra.run.smithi022.stdout: "client_readahead_max_periods": "4", 2024-09-11T07:28:36.663 INFO:teuthology.orchestra.run.smithi022.stdout: "client_readahead_min": "131072", 2024-09-11T07:28:36.664 INFO:teuthology.orchestra.run.smithi022.stdout: "client_reconnect_stale": "false", 2024-09-11T07:28:36.664 INFO:teuthology.orchestra.run.smithi022.stdout: "client_shutdown_timeout": "30", 2024-09-11T07:28:36.664 INFO:teuthology.orchestra.run.smithi022.stdout: "client_snapdir": ".snap", 2024-09-11T07:28:36.664 INFO:teuthology.orchestra.run.smithi022.stdout: "client_tick_interval": "1", 2024-09-11T07:28:36.664 INFO:teuthology.orchestra.run.smithi022.stdout: "client_trace": "", 2024-09-11T07:28:36.664 INFO:teuthology.orchestra.run.smithi022.stdout: "client_try_dentry_invalidate": "false", 2024-09-11T07:28:36.664 INFO:teuthology.orchestra.run.smithi022.stdout: "client_use_faked_inos": "false", 2024-09-11T07:28:36.664 INFO:teuthology.orchestra.run.smithi022.stdout: "client_use_random_mds": "false", 2024-09-11T07:28:36.664 INFO:teuthology.orchestra.run.smithi022.stdout: "clog_to_graylog": "false", 2024-09-11T07:28:36.664 INFO:teuthology.orchestra.run.smithi022.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-11T07:28:36.665 INFO:teuthology.orchestra.run.smithi022.stdout: "clog_to_graylog_port": "12201", 2024-09-11T07:28:36.665 INFO:teuthology.orchestra.run.smithi022.stdout: "clog_to_monitors": "default=true", 2024-09-11T07:28:36.665 INFO:teuthology.orchestra.run.smithi022.stdout: "clog_to_syslog": "false", 2024-09-11T07:28:36.665 INFO:teuthology.orchestra.run.smithi022.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-11T07:28:36.665 INFO:teuthology.orchestra.run.smithi022.stdout: "clog_to_syslog_level": "info", 2024-09-11T07:28:36.665 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "-", 2024-09-11T07:28:36.665 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_network": "", 2024-09-11T07:28:36.665 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_network_interface": "", 2024-09-11T07:28:36.665 INFO:teuthology.orchestra.run.smithi022.stdout: "compressor_zlib_isal": "false", 2024-09-11T07:28:36.665 INFO:teuthology.orchestra.run.smithi022.stdout: "compressor_zlib_level": "5", 2024-09-11T07:28:36.665 INFO:teuthology.orchestra.run.smithi022.stdout: "compressor_zlib_winsize": "-15", 2024-09-11T07:28:36.665 INFO:teuthology.orchestra.run.smithi022.stdout: "compressor_zstd_level": "1", 2024-09-11T07:28:36.665 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40", 2024-09-11T07:28:36.665 INFO:teuthology.orchestra.run.smithi022.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-11T07:28:36.666 INFO:teuthology.orchestra.run.smithi022.stdout: "crimson_alien_op_num_threads": "6", 2024-09-11T07:28:36.666 INFO:teuthology.orchestra.run.smithi022.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-11T07:28:36.666 INFO:teuthology.orchestra.run.smithi022.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-11T07:28:36.666 INFO:teuthology.orchestra.run.smithi022.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-11T07:28:36.666 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_location": "", 2024-09-11T07:28:36.666 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_location_hook": "", 2024-09-11T07:28:36.666 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_location_hook_timeout": "10", 2024-09-11T07:28:36.666 INFO:teuthology.orchestra.run.smithi022.stdout: "daemonize": "false", 2024-09-11T07:28:36.666 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_alienstore": "0/5", 2024-09-11T07:28:36.666 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_allow_any_pool_priority": "false", 2024-09-11T07:28:36.666 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_asok": "1/5", 2024-09-11T07:28:36.666 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_asok_assert_abort": "false", 2024-09-11T07:28:36.666 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_asserts_on_shutdown": "true", 2024-09-11T07:28:36.666 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_auth": "1/5", 2024-09-11T07:28:36.666 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_bdev": "1/3", 2024-09-11T07:28:36.667 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_bluefs": "1/5", 2024-09-11T07:28:36.667 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_bluestore": "1/5", 2024-09-11T07:28:36.667 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_buffer": "0/1", 2024-09-11T07:28:36.667 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_ceph_exporter": "1/5", 2024-09-11T07:28:36.667 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_cephfs_mirror": "0/5", 2024-09-11T07:28:36.667 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_cephsqlite": "0/5", 2024-09-11T07:28:36.667 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_civetweb": "1/10", 2024-09-11T07:28:36.667 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_client": "0/5", 2024-09-11T07:28:36.667 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_compressor": "1/5", 2024-09-11T07:28:36.667 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_context": "0/1", 2024-09-11T07:28:36.667 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_crush": "1/1", 2024-09-11T07:28:36.667 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_crypto": "1/5", 2024-09-11T07:28:36.667 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_deliberately_leak_memory": "false", 2024-09-11T07:28:36.667 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_disable_randomized_ping": "false", 2024-09-11T07:28:36.667 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_dpdk": "1/5", 2024-09-11T07:28:36.668 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_eventtrace": "1/5", 2024-09-11T07:28:36.668 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_filer": "0/1", 2024-09-11T07:28:36.668 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_filestore": "1/3", 2024-09-11T07:28:36.668 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_finisher": "1/1", 2024-09-11T07:28:36.668 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_fuse": "1/5", 2024-09-11T07:28:36.668 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_heartbeat_testing_span": "0", 2024-09-11T07:28:36.668 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_heartbeatmap": "1/5", 2024-09-11T07:28:36.668 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-11T07:28:36.668 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_javaclient": "1/5", 2024-09-11T07:28:36.668 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_journal": "1/3", 2024-09-11T07:28:36.668 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_journaler": "0/5", 2024-09-11T07:28:36.668 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_kstore": "1/5", 2024-09-11T07:28:36.669 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_leveldb": "4/5", 2024-09-11T07:28:36.669 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_lockdep": "0/1", 2024-09-11T07:28:36.669 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_mclock": "1/5", 2024-09-11T07:28:36.669 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_mds": "1/5", 2024-09-11T07:28:36.669 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_mds_balancer": "1/5", 2024-09-11T07:28:36.669 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_mds_locker": "1/5", 2024-09-11T07:28:36.669 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_mds_log": "1/5", 2024-09-11T07:28:36.669 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_mds_log_expire": "1/5", 2024-09-11T07:28:36.669 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_mds_migrator": "1/5", 2024-09-11T07:28:36.669 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_memdb": "4/5", 2024-09-11T07:28:36.669 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_mgr": "2/5", 2024-09-11T07:28:36.669 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_mgrc": "1/5", 2024-09-11T07:28:36.670 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_mon": "20/20", 2024-09-11T07:28:36.670 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_monc": "0/10", 2024-09-11T07:28:36.670 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_ms": "1/1", 2024-09-11T07:28:36.670 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_none": "0/5", 2024-09-11T07:28:36.670 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_objclass": "0/5", 2024-09-11T07:28:36.670 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_objectcacher": "0/5", 2024-09-11T07:28:36.670 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_objecter": "0/1", 2024-09-11T07:28:36.670 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_optracker": "0/5", 2024-09-11T07:28:36.670 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_osd": "1/5", 2024-09-11T07:28:36.670 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_paxos": "20/20", 2024-09-11T07:28:36.670 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_perfcounter": "1/5", 2024-09-11T07:28:36.670 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_prioritycache": "1/5", 2024-09-11T07:28:36.670 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_rados": "0/5", 2024-09-11T07:28:36.670 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_rbd": "0/5", 2024-09-11T07:28:36.670 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_rbd_mirror": "0/5", 2024-09-11T07:28:36.671 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_rbd_pwl": "0/5", 2024-09-11T07:28:36.671 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_rbd_replay": "0/5", 2024-09-11T07:28:36.671 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_refs": "0/0", 2024-09-11T07:28:36.671 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_reserver": "1/1", 2024-09-11T07:28:36.671 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_rgw": "1/5", 2024-09-11T07:28:36.671 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_rgw_datacache": "1/5", 2024-09-11T07:28:36.671 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_rgw_sync": "1/5", 2024-09-11T07:28:36.671 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_rocksdb": "4/5", 2024-09-11T07:28:36.671 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_seastore": "0/5", 2024-09-11T07:28:36.671 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_seastore_cache": "0/5", 2024-09-11T07:28:36.671 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_seastore_cleaner": "0/5", 2024-09-11T07:28:36.671 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_seastore_device": "0/5", 2024-09-11T07:28:36.671 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_seastore_journal": "0/5", 2024-09-11T07:28:36.671 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_seastore_lba": "0/5", 2024-09-11T07:28:36.671 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_seastore_odata": "0/5", 2024-09-11T07:28:36.672 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_seastore_omap": "0/5", 2024-09-11T07:28:36.672 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_seastore_onode": "0/5", 2024-09-11T07:28:36.672 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_seastore_tm": "0/5", 2024-09-11T07:28:36.672 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_striper": "0/1", 2024-09-11T07:28:36.672 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_test": "0/5", 2024-09-11T07:28:36.672 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_throttle": "1/1", 2024-09-11T07:28:36.672 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_timer": "0/1", 2024-09-11T07:28:36.672 INFO:teuthology.orchestra.run.smithi022.stdout: "debug_tp": "0/5", 2024-09-11T07:28:36.672 INFO:teuthology.orchestra.run.smithi022.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-11T07:28:36.672 INFO:teuthology.orchestra.run.smithi022.stdout: "device_failure_prediction_mode": "none", 2024-09-11T07:28:36.672 INFO:teuthology.orchestra.run.smithi022.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-11T07:28:36.672 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-11T07:28:36.672 INFO:teuthology.orchestra.run.smithi022.stdout: "err_to_graylog": "false", 2024-09-11T07:28:36.673 INFO:teuthology.orchestra.run.smithi022.stdout: "err_to_journald": "false", 2024-09-11T07:28:36.673 INFO:teuthology.orchestra.run.smithi022.stdout: "err_to_stderr": "true", 2024-09-11T07:28:36.673 INFO:teuthology.orchestra.run.smithi022.stdout: "err_to_syslog": "false", 2024-09-11T07:28:36.673 INFO:teuthology.orchestra.run.smithi022.stdout: "event_tracing": "false", 2024-09-11T07:28:36.673 INFO:teuthology.orchestra.run.smithi022.stdout: "exporter_addr": "0.0.0.0", 2024-09-11T07:28:36.673 INFO:teuthology.orchestra.run.smithi022.stdout: "exporter_http_port": "9926", 2024-09-11T07:28:36.673 INFO:teuthology.orchestra.run.smithi022.stdout: "exporter_prio_limit": "5", 2024-09-11T07:28:36.673 INFO:teuthology.orchestra.run.smithi022.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-11T07:28:36.673 INFO:teuthology.orchestra.run.smithi022.stdout: "exporter_sort_metrics": "true", 2024-09-11T07:28:36.673 INFO:teuthology.orchestra.run.smithi022.stdout: "exporter_stats_period": "5", 2024-09-11T07:28:36.673 INFO:teuthology.orchestra.run.smithi022.stdout: "fake_statfs_for_testing": "0", 2024-09-11T07:28:36.673 INFO:teuthology.orchestra.run.smithi022.stdout: "fatal_signal_handlers": "true", 2024-09-11T07:28:36.673 INFO:teuthology.orchestra.run.smithi022.stdout: "filer_max_purge_ops": "10", 2024-09-11T07:28:36.674 INFO:teuthology.orchestra.run.smithi022.stdout: "filer_max_truncate_ops": "128", 2024-09-11T07:28:36.674 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_apply_finisher_threads": "1", 2024-09-11T07:28:36.674 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_blackhole": "false", 2024-09-11T07:28:36.674 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_btrfs_clone_range": "true", 2024-09-11T07:28:36.674 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_btrfs_snap": "true", 2024-09-11T07:28:36.674 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_caller_concurrency": "10", 2024-09-11T07:28:36.674 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_collect_device_partition_information": "true", 2024-09-11T07:28:36.674 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_commit_timeout": "600.000000", 2024-09-11T07:28:36.674 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_debug_inject_read_err": "false", 2024-09-11T07:28:36.674 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_debug_omap_check": "false", 2024-09-11T07:28:36.674 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-11T07:28:36.674 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_debug_verify_split": "false", 2024-09-11T07:28:36.674 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_dump_file": "", 2024-09-11T07:28:36.674 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-11T07:28:36.675 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-11T07:28:36.675 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_fadvise": "true", 2024-09-11T07:28:36.675 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_fail_eio": "true", 2024-09-11T07:28:36.675 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_fd_cache_shards": "16", 2024-09-11T07:28:36.675 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_fd_cache_size": "128", 2024-09-11T07:28:36.675 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_fiemap": "false", 2024-09-11T07:28:36.675 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_fiemap_threshold": "4096", 2024-09-11T07:28:36.675 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-11T07:28:36.675 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-11T07:28:36.675 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_inject_stall": "0", 2024-09-11T07:28:36.675 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_journal_parallel": "false", 2024-09-11T07:28:36.675 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_journal_trailing": "false", 2024-09-11T07:28:36.675 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_journal_writeahead": "false", 2024-09-11T07:28:36.675 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_kill_at": "0", 2024-09-11T07:28:36.676 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-11T07:28:36.676 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-11T07:28:36.676 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-11T07:28:36.676 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-11T07:28:36.676 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-11T07:28:36.676 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_max_inline_xattrs": "0", 2024-09-11T07:28:36.676 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-11T07:28:36.676 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-11T07:28:36.676 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-11T07:28:36.676 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-11T07:28:36.676 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_max_xattr_value_size": "0", 2024-09-11T07:28:36.676 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-11T07:28:36.676 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-11T07:28:36.677 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-11T07:28:36.677 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_merge_threshold": "-10", 2024-09-11T07:28:36.677 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-11T07:28:36.677 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_odsync_write": "false", 2024-09-11T07:28:36.677 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_omap_backend": "rocksdb", 2024-09-11T07:28:36.677 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_omap_backend_path": "", 2024-09-11T07:28:36.677 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-11T07:28:36.677 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-11T07:28:36.677 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-11T07:28:36.677 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_op_thread_timeout": "60", 2024-09-11T07:28:36.677 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_op_threads": "2", 2024-09-11T07:28:36.677 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_punch_hole": "false", 2024-09-11T07:28:36.677 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-11T07:28:36.677 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-11T07:28:36.678 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-11T07:28:36.678 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-11T07:28:36.678 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-11T07:28:36.678 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-11T07:28:36.678 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-11T07:28:36.678 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-11T07:28:36.678 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-11T07:28:36.678 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_queue_max_ops": "50", 2024-09-11T07:28:36.678 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-11T07:28:36.678 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_seek_data_hole": "false", 2024-09-11T07:28:36.678 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_sloppy_crc": "false", 2024-09-11T07:28:36.678 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-11T07:28:36.678 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_splice": "false", 2024-09-11T07:28:36.678 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_split_multiple": "2", 2024-09-11T07:28:36.678 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_split_rand_factor": "20", 2024-09-11T07:28:36.679 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_update_to": "1000", 2024-09-11T07:28:36.679 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-11T07:28:36.679 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-11T07:28:36.679 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-11T07:28:36.679 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-11T07:28:36.679 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-11T07:28:36.679 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-11T07:28:36.679 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_wbthrottle_enable": "true", 2024-09-11T07:28:36.679 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-11T07:28:36.679 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-11T07:28:36.679 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-11T07:28:36.679 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-11T07:28:36.679 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-11T07:28:36.679 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-11T07:28:36.680 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_xfs_extsize": "false", 2024-09-11T07:28:36.680 INFO:teuthology.orchestra.run.smithi022.stdout: "filestore_zfs_snap": "false", 2024-09-11T07:28:36.680 INFO:teuthology.orchestra.run.smithi022.stdout: "fio_dir": "/tmp/fio", 2024-09-11T07:28:36.680 INFO:teuthology.orchestra.run.smithi022.stdout: "fsid": "12a285d8-700e-11ef-bcea-c7b262605968", 2024-09-11T07:28:36.680 INFO:teuthology.orchestra.run.smithi022.stdout: "fuse_allow_other": "true", 2024-09-11T07:28:36.680 INFO:teuthology.orchestra.run.smithi022.stdout: "fuse_atomic_o_trunc": "true", 2024-09-11T07:28:36.680 INFO:teuthology.orchestra.run.smithi022.stdout: "fuse_big_writes": "true", 2024-09-11T07:28:36.680 INFO:teuthology.orchestra.run.smithi022.stdout: "fuse_debug": "false", 2024-09-11T07:28:36.681 INFO:teuthology.orchestra.run.smithi022.stdout: "fuse_default_permissions": "false", 2024-09-11T07:28:36.681 INFO:teuthology.orchestra.run.smithi022.stdout: "fuse_disable_pagecache": "false", 2024-09-11T07:28:36.681 INFO:teuthology.orchestra.run.smithi022.stdout: "fuse_max_write": "0", 2024-09-11T07:28:36.681 INFO:teuthology.orchestra.run.smithi022.stdout: "fuse_multithreaded": "true", 2024-09-11T07:28:36.681 INFO:teuthology.orchestra.run.smithi022.stdout: "fuse_require_active_mds": "true", 2024-09-11T07:28:36.681 INFO:teuthology.orchestra.run.smithi022.stdout: "fuse_set_user_groups": "true", 2024-09-11T07:28:36.681 INFO:teuthology.orchestra.run.smithi022.stdout: "fuse_splice_move": "true", 2024-09-11T07:28:36.681 INFO:teuthology.orchestra.run.smithi022.stdout: "fuse_splice_read": "true", 2024-09-11T07:28:36.681 INFO:teuthology.orchestra.run.smithi022.stdout: "fuse_splice_write": "true", 2024-09-11T07:28:36.681 INFO:teuthology.orchestra.run.smithi022.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-11T07:28:36.681 INFO:teuthology.orchestra.run.smithi022.stdout: "fuse_use_invalidate_cb": "true", 2024-09-11T07:28:36.681 INFO:teuthology.orchestra.run.smithi022.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-11T07:28:36.682 INFO:teuthology.orchestra.run.smithi022.stdout: "gss_target_name": "ceph", 2024-09-11T07:28:36.682 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_file": "", 2024-09-11T07:28:36.682 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_inject_failure": "0", 2024-09-11T07:28:36.682 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_interval": "5", 2024-09-11T07:28:36.682 INFO:teuthology.orchestra.run.smithi022.stdout: "host": "smithi022", 2024-09-11T07:28:36.682 INFO:teuthology.orchestra.run.smithi022.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-11T07:28:36.682 INFO:teuthology.orchestra.run.smithi022.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-11T07:28:36.682 INFO:teuthology.orchestra.run.smithi022.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-11T07:28:36.682 INFO:teuthology.orchestra.run.smithi022.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-11T07:28:36.682 INFO:teuthology.orchestra.run.smithi022.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-11T07:28:36.682 INFO:teuthology.orchestra.run.smithi022.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-11T07:28:36.682 INFO:teuthology.orchestra.run.smithi022.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-11T07:28:36.682 INFO:teuthology.orchestra.run.smithi022.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-11T07:28:36.682 INFO:teuthology.orchestra.run.smithi022.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-11T07:28:36.683 INFO:teuthology.orchestra.run.smithi022.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-11T07:28:36.683 INFO:teuthology.orchestra.run.smithi022.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-11T07:28:36.683 INFO:teuthology.orchestra.run.smithi022.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-11T07:28:36.683 INFO:teuthology.orchestra.run.smithi022.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-11T07:28:36.683 INFO:teuthology.orchestra.run.smithi022.stdout: "inject_early_sigterm": "false", 2024-09-11T07:28:36.683 INFO:teuthology.orchestra.run.smithi022.stdout: "jaeger_tracing_enable": "false", 2024-09-11T07:28:36.683 INFO:teuthology.orchestra.run.smithi022.stdout: "journal_aio": "true", 2024-09-11T07:28:36.683 INFO:teuthology.orchestra.run.smithi022.stdout: "journal_align_min_size": "65536", 2024-09-11T07:28:36.683 INFO:teuthology.orchestra.run.smithi022.stdout: "journal_block_align": "true", 2024-09-11T07:28:36.683 INFO:teuthology.orchestra.run.smithi022.stdout: "journal_block_size": "4096", 2024-09-11T07:28:36.683 INFO:teuthology.orchestra.run.smithi022.stdout: "journal_dio": "true", 2024-09-11T07:28:36.683 INFO:teuthology.orchestra.run.smithi022.stdout: "journal_discard": "false", 2024-09-11T07:28:36.683 INFO:teuthology.orchestra.run.smithi022.stdout: "journal_force_aio": "false", 2024-09-11T07:28:36.683 INFO:teuthology.orchestra.run.smithi022.stdout: "journal_ignore_corruption": "false", 2024-09-11T07:28:36.683 INFO:teuthology.orchestra.run.smithi022.stdout: "journal_max_write_bytes": "10485760", 2024-09-11T07:28:36.684 INFO:teuthology.orchestra.run.smithi022.stdout: "journal_max_write_entries": "100", 2024-09-11T07:28:36.684 INFO:teuthology.orchestra.run.smithi022.stdout: "journal_replay_from": "0", 2024-09-11T07:28:36.684 INFO:teuthology.orchestra.run.smithi022.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-11T07:28:36.684 INFO:teuthology.orchestra.run.smithi022.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-11T07:28:36.684 INFO:teuthology.orchestra.run.smithi022.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-11T07:28:36.684 INFO:teuthology.orchestra.run.smithi022.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-11T07:28:36.684 INFO:teuthology.orchestra.run.smithi022.stdout: "journal_write_header_frequency": "0", 2024-09-11T07:28:36.684 INFO:teuthology.orchestra.run.smithi022.stdout: "journal_zero_on_create": "false", 2024-09-11T07:28:36.684 INFO:teuthology.orchestra.run.smithi022.stdout: "journaler_prefetch_periods": "10", 2024-09-11T07:28:36.684 INFO:teuthology.orchestra.run.smithi022.stdout: "journaler_prezero_periods": "5", 2024-09-11T07:28:36.684 INFO:teuthology.orchestra.run.smithi022.stdout: "journaler_write_head_interval": "15", 2024-09-11T07:28:36.684 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "", 2024-09-11T07:28:36.684 INFO:teuthology.orchestra.run.smithi022.stdout: "keyfile": "", 2024-09-11T07:28:36.684 INFO:teuthology.orchestra.run.smithi022.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-11T07:28:36.685 INFO:teuthology.orchestra.run.smithi022.stdout: "kstore_backend": "rocksdb", 2024-09-11T07:28:36.685 INFO:teuthology.orchestra.run.smithi022.stdout: "kstore_default_stripe_size": "65536", 2024-09-11T07:28:36.685 INFO:teuthology.orchestra.run.smithi022.stdout: "kstore_fsck_on_mount": "false", 2024-09-11T07:28:36.685 INFO:teuthology.orchestra.run.smithi022.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-11T07:28:36.685 INFO:teuthology.orchestra.run.smithi022.stdout: "kstore_max_bytes": "67108864", 2024-09-11T07:28:36.685 INFO:teuthology.orchestra.run.smithi022.stdout: "kstore_max_ops": "512", 2024-09-11T07:28:36.685 INFO:teuthology.orchestra.run.smithi022.stdout: "kstore_nid_prealloc": "1024", 2024-09-11T07:28:36.685 INFO:teuthology.orchestra.run.smithi022.stdout: "kstore_onode_map_size": "1024", 2024-09-11T07:28:36.685 INFO:teuthology.orchestra.run.smithi022.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-11T07:28:36.685 INFO:teuthology.orchestra.run.smithi022.stdout: "kstore_sync_submit_transaction": "false", 2024-09-11T07:28:36.685 INFO:teuthology.orchestra.run.smithi022.stdout: "kstore_sync_transaction": "false", 2024-09-11T07:28:36.685 INFO:teuthology.orchestra.run.smithi022.stdout: "leveldb_block_size": "65536", 2024-09-11T07:28:36.685 INFO:teuthology.orchestra.run.smithi022.stdout: "leveldb_bloom_size": "0", 2024-09-11T07:28:36.685 INFO:teuthology.orchestra.run.smithi022.stdout: "leveldb_cache_size": "536870912", 2024-09-11T07:28:36.686 INFO:teuthology.orchestra.run.smithi022.stdout: "leveldb_compact_on_mount": "false", 2024-09-11T07:28:36.686 INFO:teuthology.orchestra.run.smithi022.stdout: "leveldb_compression": "false", 2024-09-11T07:28:36.686 INFO:teuthology.orchestra.run.smithi022.stdout: "leveldb_log": "", 2024-09-11T07:28:36.686 INFO:teuthology.orchestra.run.smithi022.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-11T07:28:36.686 INFO:teuthology.orchestra.run.smithi022.stdout: "leveldb_max_open_files": "0", 2024-09-11T07:28:36.686 INFO:teuthology.orchestra.run.smithi022.stdout: "leveldb_paranoid": "false", 2024-09-11T07:28:36.686 INFO:teuthology.orchestra.run.smithi022.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-11T07:28:36.686 INFO:teuthology.orchestra.run.smithi022.stdout: "librados_thread_count": "2", 2024-09-11T07:28:36.686 INFO:teuthology.orchestra.run.smithi022.stdout: "lockdep": "false", 2024-09-11T07:28:36.686 INFO:teuthology.orchestra.run.smithi022.stdout: "lockdep_force_backtrace": "false", 2024-09-11T07:28:36.686 INFO:teuthology.orchestra.run.smithi022.stdout: "log_coarse_timestamps": "true", 2024-09-11T07:28:36.686 INFO:teuthology.orchestra.run.smithi022.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-11T07:28:36.686 INFO:teuthology.orchestra.run.smithi022.stdout: "log_flush_on_exit": "false", 2024-09-11T07:28:36.686 INFO:teuthology.orchestra.run.smithi022.stdout: "log_graylog_host": "127.0.0.1", 2024-09-11T07:28:36.687 INFO:teuthology.orchestra.run.smithi022.stdout: "log_graylog_port": "12201", 2024-09-11T07:28:36.687 INFO:teuthology.orchestra.run.smithi022.stdout: "log_max_new": "1000", 2024-09-11T07:28:36.687 INFO:teuthology.orchestra.run.smithi022.stdout: "log_max_recent": "10000", 2024-09-11T07:28:36.687 INFO:teuthology.orchestra.run.smithi022.stdout: "log_stderr_prefix": "", 2024-09-11T07:28:36.687 INFO:teuthology.orchestra.run.smithi022.stdout: "log_stop_at_utilization": "0.970000", 2024-09-11T07:28:36.687 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": "true", 2024-09-11T07:28:36.687 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_graylog": "false", 2024-09-11T07:28:36.687 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_journald": "false", 2024-09-11T07:28:36.687 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_stderr": "false", 2024-09-11T07:28:36.687 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_syslog": "false", 2024-09-11T07:28:36.687 INFO:teuthology.orchestra.run.smithi022.stdout: "max_rotating_auth_attempts": "10", 2024-09-11T07:28:36.687 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-11T07:28:36.687 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_action_on_write_error": "1", 2024-09-11T07:28:36.687 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_alternate_name_max": "8192", 2024-09-11T07:28:36.688 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_asio_thread_count": "2", 2024-09-11T07:28:36.688 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_bal_export_pin": "true", 2024-09-11T07:28:36.688 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_bal_fragment_dirs": "true", 2024-09-11T07:28:36.688 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-11T07:28:36.688 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_bal_fragment_interval": "5", 2024-09-11T07:28:36.688 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-11T07:28:36.688 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-11T07:28:36.688 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_bal_interval": "10", 2024-09-11T07:28:36.688 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_bal_max": "-1", 2024-09-11T07:28:36.688 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_bal_max_until": "-1", 2024-09-11T07:28:36.688 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_bal_merge_size": "50", 2024-09-11T07:28:36.688 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_bal_midchunk": "0.300000", 2024-09-11T07:28:36.688 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-11T07:28:36.689 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_bal_min_start": "0.200000", 2024-09-11T07:28:36.689 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_bal_minchunk": "0.001000", 2024-09-11T07:28:36.689 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_bal_mode": "0", 2024-09-11T07:28:36.689 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_bal_need_max": "1.200000", 2024-09-11T07:28:36.689 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_bal_need_min": "0.800000", 2024-09-11T07:28:36.689 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-11T07:28:36.689 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-11T07:28:36.689 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_bal_split_bits": "3", 2024-09-11T07:28:36.689 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-11T07:28:36.689 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_bal_split_size": "10000", 2024-09-11T07:28:36.689 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-11T07:28:36.689 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_bal_target_decay": "10.000000", 2024-09-11T07:28:36.689 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-11T07:28:36.690 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_beacon_grace": "15.000000", 2024-09-11T07:28:36.690 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_beacon_interval": "4.000000", 2024-09-11T07:28:36.690 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-11T07:28:36.690 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-11T07:28:36.690 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_cache_mid": "0.700000", 2024-09-11T07:28:36.690 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_cache_release_free_interval": "10", 2024-09-11T07:28:36.690 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_cache_reservation": "0.050000", 2024-09-11T07:28:36.690 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-11T07:28:36.690 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_cache_trim_interval": "1", 2024-09-11T07:28:36.690 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_cache_trim_threshold": "262144", 2024-09-11T07:28:36.690 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-11T07:28:36.690 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-11T07:28:36.690 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-11T07:28:36.690 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_client_prealloc_inos": "1000", 2024-09-11T07:28:36.691 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-11T07:28:36.691 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_connect_bootstrapping": "false", 2024-09-11T07:28:36.691 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_damage_table_max_entries": "10000", 2024-09-11T07:28:36.691 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-11T07:28:36.691 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_debug_auth_pins": "false", 2024-09-11T07:28:36.691 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_debug_frag": "false", 2024-09-11T07:28:36.691 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_debug_scatterstat": "false", 2024-09-11T07:28:36.691 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_debug_subtrees": "false", 2024-09-11T07:28:36.691 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_decay_halflife": "5.000000", 2024-09-11T07:28:36.692 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_default_dir_hash": "2", 2024-09-11T07:28:36.692 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_defer_session_stale": "true", 2024-09-11T07:28:36.692 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_deny_all_reconnect": "false", 2024-09-11T07:28:36.692 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_dir_keys_per_op": "16384", 2024-09-11T07:28:36.692 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_dir_max_commit_size": "10", 2024-09-11T07:28:36.692 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_dir_max_entries": "0", 2024-09-11T07:28:36.692 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-11T07:28:36.692 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-11T07:28:36.692 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_dump_cache_on_map": "false", 2024-09-11T07:28:36.693 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-11T07:28:36.693 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-11T07:28:36.693 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_early_reply": "true", 2024-09-11T07:28:36.693 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_enable_op_tracker": "true", 2024-09-11T07:28:36.693 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_enforce_unique_name": "true", 2024-09-11T07:28:36.693 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-11T07:28:36.693 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-11T07:28:36.693 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_export_ephemeral_random": "true", 2024-09-11T07:28:36.693 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-11T07:28:36.693 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-11T07:28:36.694 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-11T07:28:36.694 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-11T07:28:36.694 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-11T07:28:36.694 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-11T07:28:36.694 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_health_summarize_threshold": "10", 2024-09-11T07:28:36.694 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-11T07:28:36.694 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-11T07:28:36.694 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_inject_health_dummy": "false", 2024-09-11T07:28:36.694 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-11T07:28:36.694 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_inject_migrator_session_race": "false", 2024-09-11T07:28:36.694 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-11T07:28:36.694 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-11T07:28:36.695 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-11T07:28:36.695 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_join_fs": "", 2024-09-11T07:28:36.695 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_journal_format": "1", 2024-09-11T07:28:36.695 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_kill_create_at": "0", 2024-09-11T07:28:36.695 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_kill_export_at": "0", 2024-09-11T07:28:36.695 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_kill_import_at": "0", 2024-09-11T07:28:36.695 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_kill_journal_at": "0", 2024-09-11T07:28:36.695 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_kill_journal_expire_at": "0", 2024-09-11T07:28:36.695 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_kill_journal_replay_at": "0", 2024-09-11T07:28:36.695 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_kill_link_at": "0", 2024-09-11T07:28:36.695 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_kill_mdstable_at": "0", 2024-09-11T07:28:36.695 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_kill_openc_at": "0", 2024-09-11T07:28:36.695 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_kill_rename_at": "0", 2024-09-11T07:28:36.695 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-09-11T07:28:36.696 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_log_events_per_segment": "1024", 2024-09-11T07:28:36.696 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_log_max_events": "-1", 2024-09-11T07:28:36.696 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_log_max_segments": "128", 2024-09-11T07:28:36.696 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_log_pause": "false", 2024-09-11T07:28:36.696 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_log_segment_size": "0", 2024-09-11T07:28:36.696 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-11T07:28:36.696 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_log_warn_factor": "2.000000", 2024-09-11T07:28:36.696 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_max_caps_per_client": "1048576", 2024-09-11T07:28:36.696 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_max_completed_flushes": "100000", 2024-09-11T07:28:36.696 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_max_completed_requests": "100000", 2024-09-11T07:28:36.696 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_max_export_size": "20971520", 2024-09-11T07:28:36.696 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_max_file_recover": "32", 2024-09-11T07:28:36.696 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_max_purge_files": "64", 2024-09-11T07:28:36.696 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_max_purge_ops": "8192", 2024-09-11T07:28:36.696 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-11T07:28:36.697 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-11T07:28:36.697 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_max_snaps_per_dir": "100", 2024-09-11T07:28:36.697 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_max_xattr_pairs_size": "65536", 2024-09-11T07:28:36.697 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_metrics_update_interval": "2", 2024-09-11T07:28:36.697 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_min_caps_per_client": "100", 2024-09-11T07:28:36.697 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_min_caps_working_set": "10000", 2024-09-11T07:28:36.697 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-11T07:28:36.697 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_numa_node": "-1", 2024-09-11T07:28:36.697 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-11T07:28:36.697 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_op_complaint_time": "30.000000", 2024-09-11T07:28:36.697 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_op_history_duration": "600", 2024-09-11T07:28:36.697 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_op_history_size": "20", 2024-09-11T07:28:36.697 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_op_history_slow_op_size": "20", 2024-09-11T07:28:36.698 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-11T07:28:36.698 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_op_log_threshold": "5", 2024-09-11T07:28:36.698 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_ping_grace": "15", 2024-09-11T07:28:36.698 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_ping_interval": "5", 2024-09-11T07:28:36.698 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-11T07:28:36.698 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-11T07:28:36.698 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_recall_max_caps": "30000", 2024-09-11T07:28:36.698 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-11T07:28:36.698 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-11T07:28:36.698 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-11T07:28:36.698 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_recall_warning_threshold": "262144", 2024-09-11T07:28:36.698 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-11T07:28:36.698 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_replay_interval": "1.000000", 2024-09-11T07:28:36.698 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-11T07:28:36.698 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-11T07:28:36.699 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_root_ino_gid": "0", 2024-09-11T07:28:36.699 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_root_ino_uid": "0", 2024-09-11T07:28:36.699 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-11T07:28:36.699 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-11T07:28:36.699 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-11T07:28:36.699 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-11T07:28:36.699 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-11T07:28:36.699 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-11T07:28:36.699 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-11T07:28:36.699 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-11T07:28:36.699 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-11T07:28:36.699 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-11T07:28:36.699 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_shutdown_check": "0", 2024-09-11T07:28:36.699 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_skip_ino": "0", 2024-09-11T07:28:36.700 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-11T07:28:36.700 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_snap_max_uid": "4294967294", 2024-09-11T07:28:36.700 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_snap_min_uid": "0", 2024-09-11T07:28:36.700 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_snap_rstat": "false", 2024-09-11T07:28:36.700 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_standby_replay_damaged": "false", 2024-09-11T07:28:36.700 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_symlink_recovery": "true", 2024-09-11T07:28:36.700 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-11T07:28:36.700 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_thrash_exports": "0", 2024-09-11T07:28:36.700 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_thrash_fragments": "0", 2024-09-11T07:28:36.700 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_tick_interval": "5.000000", 2024-09-11T07:28:36.700 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_valgrind_exit": "false", 2024-09-11T07:28:36.700 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_verify_backtrace": "1", 2024-09-11T07:28:36.700 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_verify_scatter": "false", 2024-09-11T07:28:36.700 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-11T07:28:36.700 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_wipe_sessions": "false", 2024-09-11T07:28:36.701 INFO:teuthology.orchestra.run.smithi022.stdout: "mempool_debug": "false", 2024-09-11T07:28:36.701 INFO:teuthology.orchestra.run.smithi022.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-11T07:28:36.701 INFO:teuthology.orchestra.run.smithi022.stdout: "memstore_device_bytes": "1073741824", 2024-09-11T07:28:36.701 INFO:teuthology.orchestra.run.smithi022.stdout: "memstore_page_set": "false", 2024-09-11T07:28:36.701 INFO:teuthology.orchestra.run.smithi022.stdout: "memstore_page_size": "65536", 2024-09-11T07:28:36.701 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_client_bytes": "134217728", 2024-09-11T07:28:36.701 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_client_messages": "512", 2024-09-11T07:28:36.701 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-11T07:28:36.701 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-11T07:28:36.701 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-11T07:28:36.701 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-11T07:28:36.701 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_disabled_modules": "diskprediction_local", 2024-09-11T07:28:36.701 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-11T07:28:36.701 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_max_pg_num_change": "128", 2024-09-11T07:28:36.702 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_mds_bytes": "134217728", 2024-09-11T07:28:36.702 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_mds_messages": "128", 2024-09-11T07:28:36.702 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-11T07:28:36.702 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_mon_bytes": "134217728", 2024-09-11T07:28:36.702 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_mon_messages": "128", 2024-09-11T07:28:36.702 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_osd_bytes": "536870912", 2024-09-11T07:28:36.702 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_osd_messages": "8192", 2024-09-11T07:28:36.702 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_pool": "true", 2024-09-11T07:28:36.702 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-11T07:28:36.702 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_standby_modules": "true", 2024-09-11T07:28:36.702 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_stats_period": "5", 2024-09-11T07:28:36.702 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_stats_threshold": "5", 2024-09-11T07:28:36.702 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_tick_period": "2", 2024-09-11T07:28:36.702 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-11T07:28:36.702 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-11T07:28:36.702 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_allow_pool_delete": "true", 2024-09-11T07:28:36.703 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_allow_pool_size_one": "false", 2024-09-11T07:28:36.703 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_auth_validate_all_caps": "true", 2024-09-11T07:28:36.703 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-11T07:28:36.703 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-11T07:28:36.703 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_client_bytes": "104857600", 2024-09-11T07:28:36.703 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_client_directed_command_retry": "2", 2024-09-11T07:28:36.703 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-11T07:28:36.704 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-11T07:28:36.704 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-11T07:28:36.704 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-11T07:28:36.704 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_client_hunt_parallel": "3", 2024-09-11T07:28:36.704 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_client_log_interval": "1.000000", 2024-09-11T07:28:36.704 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-11T07:28:36.704 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_client_ping_interval": "10.000000", 2024-09-11T07:28:36.704 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-11T07:28:36.704 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-11T07:28:36.705 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-11T07:28:36.705 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-11T07:28:36.705 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_cluster_log_level": "debug", 2024-09-11T07:28:36.705 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_cluster_log_to_file": "false", 2024-09-11T07:28:36.705 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-11T07:28:36.705 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-11T07:28:36.705 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-11T07:28:36.705 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_cluster_log_to_journald": "true", 2024-09-11T07:28:36.705 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-11T07:28:36.705 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-11T07:28:36.705 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-11T07:28:36.705 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_compact_on_bootstrap": "false", 2024-09-11T07:28:36.705 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_compact_on_start": "false", 2024-09-11T07:28:36.706 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_compact_on_trim": "true", 2024-09-11T07:28:36.706 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-11T07:28:36.706 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-11T07:28:36.706 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-11T07:28:36.706 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_cpu_threads": "4", 2024-09-11T07:28:36.706 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_crush_min_required_version": "hammer", 2024-09-11T07:28:36.706 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_daemon_bytes": "419430400", 2024-09-11T07:28:36.706 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-11T07:28:36.706 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_data_avail_crit": "5", 2024-09-11T07:28:36.706 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_data_avail_warn": "5", 2024-09-11T07:28:36.706 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_data_size_warn": "16106127360", 2024-09-11T07:28:36.706 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-11T07:28:36.706 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-11T07:28:36.706 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_debug_dump_json": "false", 2024-09-11T07:28:36.706 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-11T07:28:36.707 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_debug_dump_transactions": "false", 2024-09-11T07:28:36.707 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_debug_extra_checks": "false", 2024-09-11T07:28:36.707 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-11T07:28:36.707 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-11T07:28:36.707 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_debug_no_require_pacific": "false", 2024-09-11T07:28:36.707 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_debug_no_require_quincy": "false", 2024-09-11T07:28:36.707 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-11T07:28:36.707 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-11T07:28:36.707 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-11T07:28:36.707 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_down_mkfs_grace": "60", 2024-09-11T07:28:36.707 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_election_default_strategy": "1", 2024-09-11T07:28:36.707 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_election_timeout": "5.000000", 2024-09-11T07:28:36.707 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-11T07:28:36.707 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_elector_ping_divisor": "2", 2024-09-11T07:28:36.707 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-11T07:28:36.708 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_enable_op_tracker": "true", 2024-09-11T07:28:36.708 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_fake_pool_delete": "false", 2024-09-11T07:28:36.708 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_force_quorum_join": "false", 2024-09-11T07:28:36.708 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_globalid_prealloc": "10000", 2024-09-11T07:28:36.708 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_health_detail_to_clog": "true", 2024-09-11T07:28:36.708 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_health_log_update_period": "5", 2024-09-11T07:28:36.708 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_health_max_detail": "50", 2024-09-11T07:28:36.708 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_health_to_clog": "true", 2024-09-11T07:28:36.708 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_health_to_clog_interval": "600", 2024-09-11T07:28:36.708 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-11T07:28:36.708 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_host": "172.21.15.22:6789/0", 2024-09-11T07:28:36.708 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_host_override": "", 2024-09-11T07:28:36.708 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_initial_members": "", 2024-09-11T07:28:36.708 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-11T07:28:36.709 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-11T07:28:36.709 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-11T07:28:36.709 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-11T07:28:36.709 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-11T07:28:36.709 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_lease": "5.000000", 2024-09-11T07:28:36.709 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-11T07:28:36.709 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-11T07:28:36.709 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_log_full_interval": "50", 2024-09-11T07:28:36.709 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_log_max": "10000", 2024-09-11T07:28:36.709 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_log_max_summary": "50", 2024-09-11T07:28:36.709 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-11T07:28:36.709 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_max_log_epochs": "500", 2024-09-11T07:28:36.709 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-11T07:28:36.709 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-11T07:28:36.710 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_max_osd": "10000", 2024-09-11T07:28:36.710 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_max_pg_per_osd": "10000", 2024-09-11T07:28:36.710 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_max_pool_pg_num": "65536", 2024-09-11T07:28:36.710 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-11T07:28:36.710 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-11T07:28:36.710 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_mds_force_trim_to": "0", 2024-09-11T07:28:36.710 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_mds_skip_sanity": "false", 2024-09-11T07:28:36.710 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_memory_autotune": "true", 2024-09-11T07:28:36.710 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_memory_target": "2147483648", 2024-09-11T07:28:36.710 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_mgr_beacon_grace": "30", 2024-09-11T07:28:36.710 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-11T07:28:36.710 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_mgr_digest_period": "5", 2024-09-11T07:28:36.711 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_mgr_inactive_grace": "60", 2024-09-11T07:28:36.711 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-11T07:28:36.711 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-11T07:28:36.711 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_min_osdmap_epochs": "50", 2024-09-11T07:28:36.711 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_op_complaint_time": "30", 2024-09-11T07:28:36.711 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_op_history_duration": "600", 2024-09-11T07:28:36.711 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_op_history_size": "20", 2024-09-11T07:28:36.711 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_op_history_slow_op_size": "20", 2024-09-11T07:28:36.711 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-11T07:28:36.711 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_op_log_threshold": "5", 2024-09-11T07:28:36.711 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-11T07:28:36.711 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-11T07:28:36.711 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-11T07:28:36.712 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_auto_mark_in": "false", 2024-09-11T07:28:36.712 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-11T07:28:36.712 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-11T07:28:36.712 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-11T07:28:36.712 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_cache_size": "500", 2024-09-11T07:28:36.712 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-11T07:28:36.712 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-11T07:28:36.712 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-11T07:28:36.712 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_down_out_interval": "600", 2024-09-11T07:28:36.712 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-11T07:28:36.712 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-11T07:28:36.712 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_force_trim_to": "0", 2024-09-11T07:28:36.712 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-11T07:28:36.713 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-11T07:28:36.713 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-11T07:28:36.713 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-11T07:28:36.713 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-11T07:28:36.713 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-11T07:28:36.713 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-11T07:28:36.713 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-11T07:28:36.713 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_min_down_reporters": "2", 2024-09-11T07:28:36.713 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-11T07:28:36.713 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-11T07:28:36.714 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-11T07:28:36.714 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-11T07:28:36.714 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-11T07:28:36.714 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-11T07:28:36.714 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_report_timeout": "900", 2024-09-11T07:28:36.714 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-11T07:28:36.714 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-11T07:28:36.714 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-11T07:28:36.714 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-11T07:28:36.714 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-11T07:28:36.714 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osdmap_full_prune_interval": "2", 2024-09-11T07:28:36.714 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osdmap_full_prune_min": "15", 2024-09-11T07:28:36.714 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_osdmap_full_prune_txsize": "2", 2024-09-11T07:28:36.714 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-11T07:28:36.715 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_pg_stuck_threshold": "60", 2024-09-11T07:28:36.715 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-11T07:28:36.715 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-11T07:28:36.715 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-11T07:28:36.715 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-11T07:28:36.715 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-11T07:28:36.715 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-11T07:28:36.715 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_probe_timeout": "2.000000", 2024-09-11T07:28:36.715 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_reweight_max_change": "0.050000", 2024-09-11T07:28:36.715 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_reweight_max_osds": "4", 2024-09-11T07:28:36.715 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-11T07:28:36.715 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-11T07:28:36.715 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-11T07:28:36.715 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-11T07:28:36.715 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-11T07:28:36.716 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_scrub_interval": "86400", 2024-09-11T07:28:36.716 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_scrub_max_keys": "100", 2024-09-11T07:28:36.716 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_scrub_timeout": "300", 2024-09-11T07:28:36.716 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_session_timeout": "300", 2024-09-11T07:28:36.716 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_smart_report_timeout": "5", 2024-09-11T07:28:36.716 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_stat_smooth_intervals": "6", 2024-09-11T07:28:36.716 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-11T07:28:36.716 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_stretch_pool_min_size": "2", 2024-09-11T07:28:36.716 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_stretch_pool_size": "4", 2024-09-11T07:28:36.716 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-11T07:28:36.716 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-11T07:28:36.716 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_sync_debug": "false", 2024-09-11T07:28:36.717 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-11T07:28:36.717 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-11T07:28:36.717 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_sync_provider_kill_at": "0", 2024-09-11T07:28:36.717 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_sync_requester_kill_at": "0", 2024-09-11T07:28:36.717 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_sync_timeout": "60.000000", 2024-09-11T07:28:36.717 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_target_pg_per_osd": "100", 2024-09-11T07:28:36.717 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_tick_interval": "5", 2024-09-11T07:28:36.717 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_timecheck_interval": "300.000000", 2024-09-11T07:28:36.717 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-11T07:28:36.717 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_warn_older_version_delay": "604800", 2024-09-11T07:28:36.717 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-11T07:28:36.717 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-11T07:28:36.717 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-11T07:28:36.718 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_warn_on_filestore_osds": "true", 2024-09-11T07:28:36.718 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-11T07:28:36.718 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-11T07:28:36.718 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-11T07:28:36.718 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_warn_on_misplaced": "false", 2024-09-11T07:28:36.718 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-11T07:28:36.718 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_warn_on_older_version": "true", 2024-09-11T07:28:36.718 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-11T07:28:36.718 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_warn_on_pool_no_app": "true", 2024-09-11T07:28:36.718 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_warn_on_pool_no_redundancy": "true", 2024-09-11T07:28:36.718 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-11T07:28:36.718 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-11T07:28:36.718 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-11T07:28:36.719 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-11T07:28:36.719 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-11T07:28:36.719 INFO:teuthology.orchestra.run.smithi022.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-11T07:28:36.719 INFO:teuthology.orchestra.run.smithi022.stdout: "monmap": "", 2024-09-11T07:28:36.719 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_async_op_threads": "3", 2024-09-11T07:28:36.719 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-11T07:28:36.719 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_async_rdma_cm": "false", 2024-09-11T07:28:36.719 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_async_rdma_device_name": "", 2024-09-11T07:28:36.719 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_async_rdma_dscp": "96", 2024-09-11T07:28:36.719 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-11T07:28:36.719 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-11T07:28:36.719 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_async_rdma_local_gid": "", 2024-09-11T07:28:36.719 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-11T07:28:36.719 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_async_rdma_port_num": "1", 2024-09-11T07:28:36.719 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-11T07:28:36.720 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-11T07:28:36.720 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-11T07:28:36.720 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-11T07:28:36.720 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_async_rdma_sl": "3", 2024-09-11T07:28:36.720 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_async_rdma_support_srq": "true", 2024-09-11T07:28:36.720 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_async_rdma_type": "ib", 2024-09-11T07:28:36.720 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_async_reap_threshold": "5", 2024-09-11T07:28:36.720 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_bind_before_connect": "false", 2024-09-11T07:28:36.720 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_bind_ipv4": "true", 2024-09-11T07:28:36.720 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_bind_ipv6": "false", 2024-09-11T07:28:36.720 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_bind_msgr1": "true", 2024-09-11T07:28:36.720 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_bind_msgr2": "false", 2024-09-11T07:28:36.720 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_bind_port_max": "7568", 2024-09-11T07:28:36.720 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_bind_port_min": "6800", 2024-09-11T07:28:36.721 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-11T07:28:36.721 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_bind_retry_count": "3", 2024-09-11T07:28:36.721 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_bind_retry_delay": "5", 2024-09-11T07:28:36.721 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_blackhole_client": "false", 2024-09-11T07:28:36.721 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_blackhole_mds": "false", 2024-09-11T07:28:36.721 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_blackhole_mgr": "false", 2024-09-11T07:28:36.721 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_blackhole_mon": "false", 2024-09-11T07:28:36.721 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_blackhole_osd": "false", 2024-09-11T07:28:36.721 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_client_mode": "crc secure", 2024-09-11T07:28:36.721 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_cluster_mode": "crc secure", 2024-09-11T07:28:36.721 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_cluster_type": "", 2024-09-11T07:28:36.721 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_compress_secure": "false", 2024-09-11T07:28:36.721 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_connection_idle_timeout": "900", 2024-09-11T07:28:36.721 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_connection_ready_timeout": "10", 2024-09-11T07:28:36.722 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_crc_data": "true", 2024-09-11T07:28:36.722 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_crc_header": "true", 2024-09-11T07:28:36.722 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_die_on_bad_msg": "false", 2024-09-11T07:28:36.722 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_die_on_bug": "true", 2024-09-11T07:28:36.722 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_die_on_old_message": "true", 2024-09-11T07:28:36.722 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_die_on_skipped_message": "false", 2024-09-11T07:28:36.722 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-11T07:28:36.722 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-11T07:28:36.722 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_dpdk_coremask": "0xF", 2024-09-11T07:28:36.722 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-11T07:28:36.722 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-11T07:28:36.722 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_dpdk_enable_tso": "true", 2024-09-11T07:28:36.722 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-11T07:28:36.722 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-11T07:28:36.723 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_dpdk_hugepages": "", 2024-09-11T07:28:36.723 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-11T07:28:36.723 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-11T07:28:36.723 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_dpdk_lro": "true", 2024-09-11T07:28:36.723 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_dpdk_memory_channel": "4", 2024-09-11T07:28:36.723 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-11T07:28:36.723 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_dpdk_pmd": "", 2024-09-11T07:28:36.723 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_dpdk_port_id": "0", 2024-09-11T07:28:36.723 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-11T07:28:36.723 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-11T07:28:36.723 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_dump_on_send": "false", 2024-09-11T07:28:36.723 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_initial_backoff": "0.200000", 2024-09-11T07:28:36.723 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_inject_delay_max": "1.000000", 2024-09-11T07:28:36.723 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-11T07:28:36.723 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_inject_delay_type": "", 2024-09-11T07:28:36.724 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-11T07:28:36.724 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_inject_socket_failures": "0", 2024-09-11T07:28:36.724 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_learn_addr_from_peer": "true", 2024-09-11T07:28:36.724 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_max_accept_failures": "4", 2024-09-11T07:28:36.724 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_max_backoff": "15.000000", 2024-09-11T07:28:36.724 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_mon_client_mode": "secure crc", 2024-09-11T07:28:36.724 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-11T07:28:36.724 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_mon_service_mode": "secure crc", 2024-09-11T07:28:36.724 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_osd_compress_min_size": "1024", 2024-09-11T07:28:36.724 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_osd_compress_mode": "none", 2024-09-11T07:28:36.724 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_osd_compression_algorithm": "snappy zlib zstd lz4", 2024-09-11T07:28:36.724 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-11T07:28:36.724 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_pq_min_cost": "65536", 2024-09-11T07:28:36.724 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_public_type": "", 2024-09-11T07:28:36.725 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_service_mode": "crc secure", 2024-09-11T07:28:36.725 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_tcp_listen_backlog": "512", 2024-09-11T07:28:36.725 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_tcp_nodelay": "true", 2024-09-11T07:28:36.725 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-11T07:28:36.725 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_tcp_rcvbuf": "0", 2024-09-11T07:28:36.725 INFO:teuthology.orchestra.run.smithi022.stdout: "ms_type": "async+posix", 2024-09-11T07:28:36.725 INFO:teuthology.orchestra.run.smithi022.stdout: "no_config_file": "false", 2024-09-11T07:28:36.725 INFO:teuthology.orchestra.run.smithi022.stdout: "objecter_completion_locks_per_session": "32", 2024-09-11T07:28:36.725 INFO:teuthology.orchestra.run.smithi022.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-11T07:28:36.725 INFO:teuthology.orchestra.run.smithi022.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-11T07:28:36.725 INFO:teuthology.orchestra.run.smithi022.stdout: "objecter_inflight_ops": "1024", 2024-09-11T07:28:36.725 INFO:teuthology.orchestra.run.smithi022.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-11T07:28:36.725 INFO:teuthology.orchestra.run.smithi022.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-11T07:28:36.725 INFO:teuthology.orchestra.run.smithi022.stdout: "objecter_tick_interval": "5.000000", 2024-09-11T07:28:36.726 INFO:teuthology.orchestra.run.smithi022.stdout: "objecter_timeout": "10.000000", 2024-09-11T07:28:36.726 INFO:teuthology.orchestra.run.smithi022.stdout: "objectstore_blackhole": "false", 2024-09-11T07:28:36.726 INFO:teuthology.orchestra.run.smithi022.stdout: "openssl_engine_opts": "", 2024-09-11T07:28:36.726 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_agent_delay_time": "5.000000", 2024-09-11T07:28:36.726 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_agent_hist_halflife": "1000", 2024-09-11T07:28:36.726 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_agent_max_low_ops": "2", 2024-09-11T07:28:36.726 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_agent_max_ops": "4", 2024-09-11T07:28:36.726 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-11T07:28:36.726 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-11T07:28:36.726 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_agent_slop": "0.020000", 2024-09-11T07:28:36.726 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-11T07:28:36.726 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-11T07:28:36.726 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_asio_thread_count": "2", 2024-09-11T07:28:36.726 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_async_recovery_min_cost": "100", 2024-09-11T07:28:36.726 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-11T07:28:36.727 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-11T07:28:36.727 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_backfill_scan_max": "512", 2024-09-11T07:28:36.727 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_backfill_scan_min": "64", 2024-09-11T07:28:36.727 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_backoff_on_degraded": "false", 2024-09-11T07:28:36.727 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_backoff_on_peering": "false", 2024-09-11T07:28:36.727 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_backoff_on_unfound": "true", 2024-09-11T07:28:36.727 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_beacon_report_interval": "300", 2024-09-11T07:28:36.727 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_bench_duration": "30", 2024-09-11T07:28:36.727 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-11T07:28:36.727 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_bench_max_block_size": "67108864", 2024-09-11T07:28:36.727 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-11T07:28:36.727 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_blkin_trace_all": "false", 2024-09-11T07:28:36.727 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-11T07:28:36.728 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-11T07:28:36.728 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_check_for_log_corruption": "false", 2024-09-11T07:28:36.728 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-11T07:28:36.728 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-11T07:28:36.728 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-11T07:28:36.728 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-11T07:28:36.728 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_class_update_on_start": "true", 2024-09-11T07:28:36.728 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_client_message_cap": "256", 2024-09-11T07:28:36.728 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_client_message_size_cap": "524288000", 2024-09-11T07:28:36.728 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_client_op_priority": "63", 2024-09-11T07:28:36.728 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_client_watch_timeout": "30", 2024-09-11T07:28:36.728 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_command_max_records": "256", 2024-09-11T07:28:36.728 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_compact_on_start": "false", 2024-09-11T07:28:36.729 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-11T07:28:36.729 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_crush_chooseleaf_type": "1", 2024-09-11T07:28:36.729 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-11T07:28:36.729 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_crush_update_on_start": "true", 2024-09-11T07:28:36.729 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_crush_update_weight_set": "true", 2024-09-11T07:28:36.729 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-11T07:28:36.729 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-11T07:28:36.729 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-11T07:28:36.729 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-11T07:28:36.729 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-11T07:28:36.729 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_debug_feed_pullee": "-1", 2024-09-11T07:28:36.729 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-11T07:28:36.730 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-11T07:28:36.730 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-11T07:28:36.730 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_debug_misdirected_ops": "false", 2024-09-11T07:28:36.730 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_debug_no_acting_change": "false", 2024-09-11T07:28:36.730 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_debug_no_purge_strays": "false", 2024-09-11T07:28:36.730 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_debug_op_order": "false", 2024-09-11T07:28:36.730 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-11T07:28:36.730 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-11T07:28:36.730 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-11T07:28:36.730 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-11T07:28:36.730 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_debug_shutdown": "false", 2024-09-11T07:28:36.730 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-11T07:28:36.730 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-11T07:28:36.730 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-11T07:28:36.730 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-11T07:28:36.731 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_debug_verify_snaps": "false", 2024-09-11T07:28:36.731 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-11T07:28:36.731 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-11T07:28:36.731 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_deep_scrub_keys": "1024", 2024-09-11T07:28:36.731 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-11T07:28:36.731 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-11T07:28:36.731 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-11T07:28:36.731 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_deep_scrub_stride": "524288", 2024-09-11T07:28:36.731 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-11T07:28:36.731 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_default_data_pool_replay_window": "45", 2024-09-11T07:28:36.731 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_default_notify_timeout": "30", 2024-09-11T07:28:36.731 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_delete_sleep": "0.000000", 2024-09-11T07:28:36.731 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-11T07:28:36.731 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-11T07:28:36.731 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-11T07:28:36.732 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_discard_disconnected_ops": "true", 2024-09-11T07:28:36.732 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_enable_op_tracker": "true", 2024-09-11T07:28:36.732 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-11T07:28:36.732 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-11T07:28:36.732 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-11T07:28:36.732 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_fast_info": "true", 2024-09-11T07:28:36.732 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_fast_shutdown": "true", 2024-09-11T07:28:36.732 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-11T07:28:36.732 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-11T07:28:36.732 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-11T07:28:36.732 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-11T07:28:36.732 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-11T07:28:36.733 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_function_tracing": "false", 2024-09-11T07:28:36.733 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_heartbeat_grace": "20", 2024-09-11T07:28:36.733 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_heartbeat_interval": "6", 2024-09-11T07:28:36.733 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-11T07:28:36.733 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_heartbeat_min_peers": "10", 2024-09-11T07:28:36.733 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_heartbeat_min_size": "2000", 2024-09-11T07:28:36.733 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_heartbeat_stale": "600", 2024-09-11T07:28:36.733 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-11T07:28:36.733 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_hit_set_max_size": "100000", 2024-09-11T07:28:36.733 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_hit_set_min_size": "1000", 2024-09-11T07:28:36.733 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-11T07:28:36.733 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-11T07:28:36.733 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-11T07:28:36.733 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-11T07:28:36.734 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-11T07:28:36.734 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-11T07:28:36.734 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_journal_size": "5120", 2024-09-11T07:28:36.734 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_kill_backfill_at": "0", 2024-09-11T07:28:36.734 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-11T07:28:36.734 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_map_cache_size": "50", 2024-09-11T07:28:36.734 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_map_dedup": "true", 2024-09-11T07:28:36.734 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_map_message_max": "40", 2024-09-11T07:28:36.734 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-11T07:28:36.734 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_map_share_max_epochs": "40", 2024-09-11T07:28:36.734 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_max_attr_name_len": "100", 2024-09-11T07:28:36.734 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_max_attr_size": "0", 2024-09-11T07:28:36.734 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_max_backfills": "1", 2024-09-11T07:28:36.734 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_max_markdown_count": "5", 2024-09-11T07:28:36.734 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_max_markdown_period": "600", 2024-09-11T07:28:36.735 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_max_object_name_len": "2048", 2024-09-11T07:28:36.735 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_max_object_namespace_len": "256", 2024-09-11T07:28:36.735 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_max_object_size": "134217728", 2024-09-11T07:28:36.735 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-11T07:28:36.735 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-11T07:28:36.735 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_max_pg_blocked_by": "16", 2024-09-11T07:28:36.735 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_max_pg_log_entries": "10000", 2024-09-11T07:28:36.735 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-11T07:28:36.735 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_max_pgls": "1024", 2024-09-11T07:28:36.735 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_max_push_cost": "8388608", 2024-09-11T07:28:36.735 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_max_push_objects": "10", 2024-09-11T07:28:36.736 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_max_scrubs": "3", 2024-09-11T07:28:36.736 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-11T07:28:36.736 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_max_trimming_pgs": "2", 2024-09-11T07:28:36.736 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_max_write_op_reply_len": "64", 2024-09-11T07:28:36.736 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_max_write_size": "90", 2024-09-11T07:28:36.736 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-11T07:28:36.736 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-11T07:28:36.736 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-11T07:28:36.736 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-11T07:28:36.736 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-11T07:28:36.736 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-11T07:28:36.736 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-11T07:28:36.736 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-11T07:28:36.736 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mclock_profile": "balanced", 2024-09-11T07:28:36.737 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-11T07:28:36.737 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-11T07:28:36.737 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-11T07:28:36.737 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-11T07:28:36.737 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-11T07:28:36.737 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-11T07:28:36.737 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-11T07:28:36.737 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-11T07:28:36.737 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-11T07:28:36.737 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-11T07:28:36.737 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-11T07:28:36.737 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_memory_base": "805306368", 2024-09-11T07:28:36.737 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_memory_cache_min": "134217728", 2024-09-11T07:28:36.737 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-11T07:28:36.738 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-11T07:28:36.738 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_memory_target": "4294967296", 2024-09-11T07:28:36.738 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_memory_target_autotune": "false", 2024-09-11T07:28:36.738 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-11T07:28:36.738 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_min_pg_log_entries": "250", 2024-09-11T07:28:36.738 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_min_recovery_priority": "0", 2024-09-11T07:28:36.738 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-11T07:28:36.738 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-11T07:28:36.738 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mon_report_interval": "5", 2024-09-11T07:28:36.738 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-11T07:28:36.738 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-11T07:28:36.738 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_num_cache_shards": "32", 2024-09-11T07:28:36.738 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_num_op_tracker_shard": "32", 2024-09-11T07:28:36.738 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_numa_auto_affinity": "true", 2024-09-11T07:28:36.738 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_numa_node": "-1", 2024-09-11T07:28:36.739 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_numa_prefer_iface": "true", 2024-09-11T07:28:36.739 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-11T07:28:36.739 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_objecter_finishers": "1", 2024-09-11T07:28:36.739 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_objectstore": "bluestore", 2024-09-11T07:28:36.739 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_objectstore_fuse": "false", 2024-09-11T07:28:36.739 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_objectstore_tracing": "false", 2024-09-11T07:28:36.739 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_op_complaint_time": "30.000000", 2024-09-11T07:28:36.739 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_op_history_duration": "600", 2024-09-11T07:28:36.739 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_op_history_size": "20", 2024-09-11T07:28:36.739 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_op_history_slow_op_size": "20", 2024-09-11T07:28:36.740 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-11T07:28:36.740 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_op_log_threshold": "5", 2024-09-11T07:28:36.740 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_op_num_shards": "0", 2024-09-11T07:28:36.740 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_op_num_shards_hdd": "5", 2024-09-11T07:28:36.740 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_op_num_shards_ssd": "8", 2024-09-11T07:28:36.740 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-11T07:28:36.740 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-11T07:28:36.740 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-11T07:28:36.740 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-11T07:28:36.740 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_op_pq_min_cost": "65536", 2024-09-11T07:28:36.740 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-11T07:28:36.741 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_op_queue_cut_off": "high", 2024-09-11T07:28:36.741 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-11T07:28:36.741 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_op_thread_timeout": "15", 2024-09-11T07:28:36.741 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_open_classes_on_start": "true", 2024-09-11T07:28:36.741 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_os_flags": "0", 2024-09-11T07:28:36.741 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_peering_op_priority": "255", 2024-09-11T07:28:36.741 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pg_delete_cost": "1048576", 2024-09-11T07:28:36.741 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pg_delete_priority": "5", 2024-09-11T07:28:36.741 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-11T07:28:36.741 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-11T07:28:36.741 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-11T07:28:36.741 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pg_log_trim_max": "10000", 2024-09-11T07:28:36.741 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pg_log_trim_min": "100", 2024-09-11T07:28:36.741 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-11T07:28:36.741 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-11T07:28:36.742 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-11T07:28:36.742 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-11T07:28:36.742 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-11T07:28:36.742 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-11T07:28:36.742 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-11T07:28:36.742 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-11T07:28:36.742 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-11T07:28:36.742 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-11T07:28:36.742 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-11T07:28:36.742 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-11T07:28:36.742 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-11T07:28:36.742 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-11T07:28:36.742 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-11T07:28:36.742 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-11T07:28:36.742 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-11T07:28:36.743 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pool_default_flags": "0", 2024-09-11T07:28:36.743 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-11T07:28:36.743 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pool_default_min_size": "0", 2024-09-11T07:28:36.743 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-11T07:28:36.743 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pool_default_pg_num": "32", 2024-09-11T07:28:36.743 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pool_default_pgp_num": "0", 2024-09-11T07:28:36.743 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-11T07:28:36.743 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pool_default_size": "3", 2024-09-11T07:28:36.743 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pool_default_type": "replicated", 2024-09-11T07:28:36.743 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-11T07:28:36.743 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-11T07:28:36.743 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_push_per_object_cost": "1000", 2024-09-11T07:28:36.743 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-11T07:28:36.743 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_recover_clone_overlap": "true", 2024-09-11T07:28:36.743 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-11T07:28:36.744 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_recovery_cost": "20971520", 2024-09-11T07:28:36.744 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-11T07:28:36.744 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_recovery_max_active": "0", 2024-09-11T07:28:36.744 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-11T07:28:36.744 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-11T07:28:36.744 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-11T07:28:36.744 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-11T07:28:36.744 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_recovery_max_single_start": "1", 2024-09-11T07:28:36.744 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_recovery_op_priority": "3", 2024-09-11T07:28:36.744 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-11T07:28:36.744 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_recovery_priority": "5", 2024-09-11T07:28:36.744 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-11T07:28:36.744 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_recovery_sleep": "0.000000", 2024-09-11T07:28:36.744 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-11T07:28:36.745 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-11T07:28:36.745 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-11T07:28:36.745 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_repair_during_recovery": "false", 2024-09-11T07:28:36.745 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_requested_scrub_priority": "120", 2024-09-11T07:28:36.745 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-11T07:28:36.745 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-11T07:28:36.745 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_scrub_auto_repair": "false", 2024-09-11T07:28:36.745 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-11T07:28:36.745 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-11T07:28:36.745 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_scrub_begin_hour": "0", 2024-09-11T07:28:36.745 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_scrub_begin_week_day": "0", 2024-09-11T07:28:36.745 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_scrub_chunk_max": "25", 2024-09-11T07:28:36.745 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_scrub_chunk_min": "5", 2024-09-11T07:28:36.745 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_scrub_cost": "52428800", 2024-09-11T07:28:36.745 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_scrub_during_recovery": "false", 2024-09-11T07:28:36.746 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_scrub_end_hour": "0", 2024-09-11T07:28:36.746 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_scrub_end_week_day": "0", 2024-09-11T07:28:36.746 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_scrub_event_cost": "4096", 2024-09-11T07:28:36.746 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-11T07:28:36.746 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-11T07:28:36.746 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_scrub_invalid_stats": "true", 2024-09-11T07:28:36.746 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-11T07:28:36.746 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-11T07:28:36.746 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_scrub_max_preemptions": "5", 2024-09-11T07:28:36.746 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-11T07:28:36.746 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_scrub_priority": "5", 2024-09-11T07:28:36.746 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_scrub_sleep": "0.000000", 2024-09-11T07:28:36.746 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-11T07:28:36.746 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_skip_data_digest": "false", 2024-09-11T07:28:36.746 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_smart_report_timeout": "5", 2024-09-11T07:28:36.747 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_snap_trim_cost": "1048576", 2024-09-11T07:28:36.747 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_snap_trim_priority": "5", 2024-09-11T07:28:36.747 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-11T07:28:36.747 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-11T07:28:36.747 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-11T07:28:36.747 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-11T07:28:36.747 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-11T07:28:36.747 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_target_transaction_size": "30", 2024-09-11T07:28:36.747 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-11T07:28:36.747 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-11T07:28:36.747 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-11T07:28:36.747 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-11T07:28:36.747 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-11T07:28:36.747 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-11T07:28:36.747 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-11T07:28:36.748 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-11T07:28:36.748 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-11T07:28:36.748 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-11T07:28:36.748 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_tracing": "false", 2024-09-11T07:28:36.748 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_use_stale_snap": "false", 2024-09-11T07:28:36.748 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-11T07:28:36.748 INFO:teuthology.orchestra.run.smithi022.stdout: "osdc_blkin_trace_all": "false", 2024-09-11T07:28:36.748 INFO:teuthology.orchestra.run.smithi022.stdout: "paxos_kill_at": "0", 2024-09-11T07:28:36.748 INFO:teuthology.orchestra.run.smithi022.stdout: "paxos_max_join_drift": "10", 2024-09-11T07:28:36.748 INFO:teuthology.orchestra.run.smithi022.stdout: "paxos_min": "500", 2024-09-11T07:28:36.748 INFO:teuthology.orchestra.run.smithi022.stdout: "paxos_min_wait": "0.050000", 2024-09-11T07:28:36.748 INFO:teuthology.orchestra.run.smithi022.stdout: "paxos_propose_interval": "1.000000", 2024-09-11T07:28:36.748 INFO:teuthology.orchestra.run.smithi022.stdout: "paxos_service_trim_max": "500", 2024-09-11T07:28:36.749 INFO:teuthology.orchestra.run.smithi022.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-11T07:28:36.749 INFO:teuthology.orchestra.run.smithi022.stdout: "paxos_service_trim_min": "10", 2024-09-11T07:28:36.749 INFO:teuthology.orchestra.run.smithi022.stdout: "paxos_stash_full_interval": "25", 2024-09-11T07:28:36.749 INFO:teuthology.orchestra.run.smithi022.stdout: "paxos_trim_max": "500", 2024-09-11T07:28:36.749 INFO:teuthology.orchestra.run.smithi022.stdout: "paxos_trim_min": "250", 2024-09-11T07:28:36.749 INFO:teuthology.orchestra.run.smithi022.stdout: "perf": "true", 2024-09-11T07:28:36.749 INFO:teuthology.orchestra.run.smithi022.stdout: "pid_file": "", 2024-09-11T07:28:36.749 INFO:teuthology.orchestra.run.smithi022.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-11T07:28:36.749 INFO:teuthology.orchestra.run.smithi022.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-11T07:28:36.749 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "-", 2024-09-11T07:28:36.749 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrv": "", 2024-09-11T07:28:36.749 INFO:teuthology.orchestra.run.smithi022.stdout: "public_bind_addr": "-", 2024-09-11T07:28:36.749 INFO:teuthology.orchestra.run.smithi022.stdout: "public_network": "", 2024-09-11T07:28:36.749 INFO:teuthology.orchestra.run.smithi022.stdout: "public_network_interface": "", 2024-09-11T07:28:36.749 INFO:teuthology.orchestra.run.smithi022.stdout: "qat_compressor_enabled": "false", 2024-09-11T07:28:36.750 INFO:teuthology.orchestra.run.smithi022.stdout: "rados_mon_op_timeout": "0", 2024-09-11T07:28:36.750 INFO:teuthology.orchestra.run.smithi022.stdout: "rados_osd_op_timeout": "0", 2024-09-11T07:28:36.750 INFO:teuthology.orchestra.run.smithi022.stdout: "rados_tracing": "false", 2024-09-11T07:28:36.750 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_atime_update_interval": "60", 2024-09-11T07:28:36.750 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-11T07:28:36.750 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_balance_parent_reads": "false", 2024-09-11T07:28:36.750 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_balance_snap_reads": "false", 2024-09-11T07:28:36.750 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_blkin_trace_all": "false", 2024-09-11T07:28:36.750 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-11T07:28:36.750 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-11T07:28:36.750 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_cache": "true", 2024-09-11T07:28:36.750 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-11T07:28:36.750 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-11T07:28:36.750 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-11T07:28:36.750 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-11T07:28:36.750 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_cache_policy": "writearound", 2024-09-11T07:28:36.751 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_cache_size": "33554432", 2024-09-11T07:28:36.751 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-11T07:28:36.751 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-11T07:28:36.751 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_clone_copy_on_read": "false", 2024-09-11T07:28:36.751 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_compression_hint": "none", 2024-09-11T07:28:36.751 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_concurrent_management_ops": "10", 2024-09-11T07:28:36.751 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-11T07:28:36.751 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_default_clone_format": "auto", 2024-09-11T07:28:36.751 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_default_data_pool": "", 2024-09-11T07:28:36.751 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_default_features": "61", 2024-09-11T07:28:36.751 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_default_format": "2", 2024-09-11T07:28:36.751 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_default_map_options": "", 2024-09-11T07:28:36.751 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_default_order": "22", 2024-09-11T07:28:36.751 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_default_pool": "rbd", 2024-09-11T07:28:36.751 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-11T07:28:36.752 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_default_stripe_count": "0", 2024-09-11T07:28:36.752 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_default_stripe_unit": "0", 2024-09-11T07:28:36.752 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-11T07:28:36.752 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-11T07:28:36.752 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-11T07:28:36.752 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_enable_alloc_hint": "true", 2024-09-11T07:28:36.752 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-11T07:28:36.752 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_io_scheduler": "simple", 2024-09-11T07:28:36.752 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-11T07:28:36.752 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-11T07:28:36.752 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-11T07:28:36.752 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-11T07:28:36.752 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-11T07:28:36.752 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-11T07:28:36.753 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-11T07:28:36.753 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-11T07:28:36.753 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-11T07:28:36.753 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_journal_order": "24", 2024-09-11T07:28:36.753 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_journal_pool": "", 2024-09-11T07:28:36.753 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_journal_splay_width": "4", 2024-09-11T07:28:36.753 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_localize_parent_reads": "false", 2024-09-11T07:28:36.753 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_localize_snap_reads": "false", 2024-09-11T07:28:36.753 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-11T07:28:36.753 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-11T07:28:36.753 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-11T07:28:36.753 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-11T07:28:36.753 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-11T07:28:36.754 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-11T07:28:36.754 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-11T07:28:36.754 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-11T07:28:36.754 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-11T07:28:36.754 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-11T07:28:36.754 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-11T07:28:36.754 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-11T07:28:36.754 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-11T07:28:36.754 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-11T07:28:36.754 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-11T07:28:36.754 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-11T07:28:36.754 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-11T07:28:36.754 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-11T07:28:36.754 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-11T07:28:36.755 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-11T07:28:36.755 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-11T07:28:36.755 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-11T07:28:36.755 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-11T07:28:36.755 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-11T07:28:36.755 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-11T07:28:36.755 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-11T07:28:36.755 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-11T07:28:36.755 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-11T07:28:36.755 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-11T07:28:36.755 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-11T07:28:36.755 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-11T07:28:36.755 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_mtime_update_interval": "60", 2024-09-11T07:28:36.755 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_non_blocking_aio": "true", 2024-09-11T07:28:36.755 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_op_thread_timeout": "60", 2024-09-11T07:28:36.755 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_op_threads": "1", 2024-09-11T07:28:36.756 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_parent_cache_enabled": "false", 2024-09-11T07:28:36.756 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-11T07:28:36.756 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-11T07:28:36.756 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-11T07:28:36.756 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_plugins": "", 2024-09-11T07:28:36.756 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_qos_bps_burst": "0", 2024-09-11T07:28:36.756 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-11T07:28:36.756 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_qos_bps_limit": "0", 2024-09-11T07:28:36.756 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_qos_exclude_ops": "0", 2024-09-11T07:28:36.756 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_qos_iops_burst": "0", 2024-09-11T07:28:36.756 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-11T07:28:36.756 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_qos_iops_limit": "0", 2024-09-11T07:28:36.757 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-11T07:28:36.757 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-11T07:28:36.757 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-11T07:28:36.757 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-11T07:28:36.757 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-11T07:28:36.757 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-11T07:28:36.757 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-11T07:28:36.757 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-11T07:28:36.757 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-11T07:28:36.757 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-11T07:28:36.757 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-11T07:28:36.757 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-11T07:28:36.757 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-11T07:28:36.757 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-11T07:28:36.757 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_read_from_replica_policy": "default", 2024-09-11T07:28:36.758 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-11T07:28:36.758 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-11T07:28:36.758 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-11T07:28:36.758 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-11T07:28:36.758 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_skip_partial_discard": "true", 2024-09-11T07:28:36.758 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-11T07:28:36.758 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_tracing": "false", 2024-09-11T07:28:36.758 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_validate_names": "true", 2024-09-11T07:28:36.758 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_validate_pool": "true", 2024-09-11T07:28:36.758 INFO:teuthology.orchestra.run.smithi022.stdout: "restapi_base_url": "", 2024-09-11T07:28:36.758 INFO:teuthology.orchestra.run.smithi022.stdout: "restapi_log_level": "", 2024-09-11T07:28:36.758 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_acl_grants_max_num": "100", 2024-09-11T07:28:36.758 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_admin_entry": "admin", 2024-09-11T07:28:36.758 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_backend_store": "rados", 2024-09-11T07:28:36.758 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_barbican_url": "", 2024-09-11T07:28:36.758 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_beast_enable_async": "true", 2024-09-11T07:28:36.759 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-11T07:28:36.759 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-11T07:28:36.759 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-11T07:28:36.759 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-11T07:28:36.759 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-11T07:28:36.759 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_cache_enabled": "true", 2024-09-11T07:28:36.759 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_cache_expiry_interval": "900", 2024-09-11T07:28:36.759 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_cache_lru_size": "10000", 2024-09-11T07:28:36.759 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_content_length_compat": "false", 2024-09-11T07:28:36.759 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_copy_obj_progress": "true", 2024-09-11T07:28:36.759 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-11T07:28:36.759 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_cors_rules_max_num": "100", 2024-09-11T07:28:36.759 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_cross_domain_policy": "", 2024-09-11T07:28:36.759 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-11T07:28:36.759 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_kmip_addr": "", 2024-09-11T07:28:36.760 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-11T07:28:36.760 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-11T07:28:36.760 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-11T07:28:36.760 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:36 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:28:36.760 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:36 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:28:36.760 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:36 smithi022 ceph-mon[29462]: 1.0 deep-scrub starts 2024-09-11T07:28:36.760 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:36 smithi022 ceph-mon[29462]: 1.0 deep-scrub ok 2024-09-11T07:28:36.760 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:36 smithi022 ceph-mon[29462]: osdmap e88: 8 total, 8 up, 8 in 2024-09-11T07:28:36.760 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:36 smithi022 ceph-mon[29462]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:28:36.761 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:36 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:28:36.761 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:36 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:28:36.761 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:36 smithi022 ceph-mon[37330]: 1.0 deep-scrub starts 2024-09-11T07:28:36.761 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:36 smithi022 ceph-mon[37330]: 1.0 deep-scrub ok 2024-09-11T07:28:36.761 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:36 smithi022 ceph-mon[37330]: osdmap e88: 8 total, 8 up, 8 in 2024-09-11T07:28:36.761 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:36 smithi022 ceph-mon[37330]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:28:36.761 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-11T07:28:36.761 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_kmip_password": "", 2024-09-11T07:28:36.762 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-11T07:28:36.762 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_kmip_username": "", 2024-09-11T07:28:36.762 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_require_ssl": "true", 2024-09-11T07:28:36.762 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-11T07:28:36.762 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-11T07:28:36.762 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-11T07:28:36.762 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-11T07:28:36.762 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-11T07:28:36.762 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-11T07:28:36.762 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-11T07:28:36.762 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-11T07:28:36.762 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-11T07:28:36.762 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-11T07:28:36.763 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-11T07:28:36.763 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-11T07:28:36.763 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-11T07:28:36.763 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-11T07:28:36.763 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-11T07:28:36.763 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_vault_addr": "", 2024-09-11T07:28:36.763 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_vault_auth": "token", 2024-09-11T07:28:36.763 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_vault_namespace": "", 2024-09-11T07:28:36.763 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_vault_prefix": "", 2024-09-11T07:28:36.763 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-11T07:28:36.763 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-11T07:28:36.763 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-11T07:28:36.763 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-11T07:28:36.763 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_vault_token_file": "", 2024-09-11T07:28:36.763 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-11T07:28:36.764 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_curl_buffersize": "524288", 2024-09-11T07:28:36.764 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-11T07:28:36.764 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_curl_low_speed_time": "300", 2024-09-11T07:28:36.764 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-11T07:28:36.764 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-11T07:28:36.764 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-11T07:28:36.764 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-11T07:28:36.764 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-11T07:28:36.764 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-11T07:28:36.764 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-11T07:28:36.765 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-11T07:28:36.765 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-11T07:28:36.765 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-11T07:28:36.765 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_data_log_changes_size": "1000", 2024-09-11T07:28:36.765 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_data_log_num_shards": "128", 2024-09-11T07:28:36.765 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-11T07:28:36.765 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_data_log_window": "30", 2024-09-11T07:28:36.765 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_data_notify_interval_msec": "200", 2024-09-11T07:28:36.765 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-11T07:28:36.765 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-11T07:28:36.765 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-11T07:28:36.765 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-11T07:28:36.766 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-11T07:28:36.766 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-11T07:28:36.766 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-11T07:28:36.766 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-11T07:28:36.766 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-11T07:28:36.766 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-11T07:28:36.766 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-11T07:28:36.766 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-11T07:28:36.766 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-11T07:28:36.766 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-11T07:28:36.766 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-11T07:28:36.766 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-11T07:28:36.766 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-11T07:28:36.766 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-11T07:28:36.766 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-11T07:28:36.767 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-11T07:28:36.767 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-11T07:28:36.767 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-11T07:28:36.767 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_dns_name": "", 2024-09-11T07:28:36.767 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_dns_s3website_name": "", 2024-09-11T07:28:36.767 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_dynamic_resharding": "true", 2024-09-11T07:28:36.767 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-11T07:28:36.767 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_enable_gc_threads": "true", 2024-09-11T07:28:36.767 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_enable_lc_threads": "true", 2024-09-11T07:28:36.767 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_enable_ops_log": "false", 2024-09-11T07:28:36.767 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_enable_quota_threads": "true", 2024-09-11T07:28:36.767 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_enable_static_website": "false", 2024-09-11T07:28:36.767 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_enable_usage_log": "false", 2024-09-11T07:28:36.767 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_enforce_swift_acls": "true", 2024-09-11T07:28:36.767 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_exit_timeout_secs": "120", 2024-09-11T07:28:36.768 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_expose_bucket": "false", 2024-09-11T07:28:36.768 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_extended_http_attrs": "", 2024-09-11T07:28:36.768 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-11T07:28:36.768 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_frontends": "beast port=7480", 2024-09-11T07:28:36.768 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-11T07:28:36.768 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_gc_max_deferred": "50", 2024-09-11T07:28:36.768 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-11T07:28:36.768 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_gc_max_objs": "32", 2024-09-11T07:28:36.768 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-11T07:28:36.768 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-11T07:28:36.768 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-11T07:28:36.768 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-11T07:28:36.768 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_gc_processor_period": "3600", 2024-09-11T07:28:36.768 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-11T07:28:36.769 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-11T07:28:36.769 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-11T07:28:36.769 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-11T07:28:36.769 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_init_timeout": "300", 2024-09-11T07:28:36.769 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-11T07:28:36.769 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_kafka_connection_idle": "300", 2024-09-11T07:28:36.769 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-11T07:28:36.769 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-11T07:28:36.769 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-11T07:28:36.769 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_keystone_admin_domain": "", 2024-09-11T07:28:36.769 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_keystone_admin_password": "", 2024-09-11T07:28:36.770 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_keystone_admin_password_path": "", 2024-09-11T07:28:36.770 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_keystone_admin_project": "", 2024-09-11T07:28:36.770 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_keystone_admin_tenant": "", 2024-09-11T07:28:36.770 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_keystone_admin_token": "", 2024-09-11T07:28:36.770 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_keystone_admin_token_path": "", 2024-09-11T07:28:36.770 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_keystone_admin_user": "", 2024-09-11T07:28:36.770 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_keystone_api_version": "2", 2024-09-11T07:28:36.770 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_keystone_barbican_domain": "", 2024-09-11T07:28:36.770 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_keystone_barbican_password": "", 2024-09-11T07:28:36.770 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_keystone_barbican_project": "", 2024-09-11T07:28:36.770 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-11T07:28:36.770 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_keystone_barbican_user": "", 2024-09-11T07:28:36.770 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-11T07:28:36.771 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-11T07:28:36.771 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-11T07:28:36.771 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-11T07:28:36.771 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-11T07:28:36.771 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_keystone_url": "", 2024-09-11T07:28:36.771 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-11T07:28:36.771 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_lc_debug_interval": "-1", 2024-09-11T07:28:36.771 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_lc_lock_max_time": "90", 2024-09-11T07:28:36.771 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_lc_max_objs": "32", 2024-09-11T07:28:36.771 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_lc_max_rules": "1000", 2024-09-11T07:28:36.771 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_lc_max_worker": "3", 2024-09-11T07:28:36.771 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-11T07:28:36.771 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_lc_thread_delay": "0", 2024-09-11T07:28:36.771 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-11T07:28:36.772 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_ldap_dnattr": "uid", 2024-09-11T07:28:36.772 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-11T07:28:36.772 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_ldap_searchfilter": "", 2024-09-11T07:28:36.772 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-11T07:28:36.772 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_ldap_uri": "", 2024-09-11T07:28:36.772 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-11T07:28:36.772 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-11T07:28:36.772 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-11T07:28:36.772 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_log_http_headers": "", 2024-09-11T07:28:36.772 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-11T07:28:36.772 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-11T07:28:36.772 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_log_object_name_utc": "false", 2024-09-11T07:28:36.772 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-11T07:28:36.773 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_max_attr_name_len": "0", 2024-09-11T07:28:36.773 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_max_attr_size": "0", 2024-09-11T07:28:36.773 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-11T07:28:36.773 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_max_chunk_size": "4194304", 2024-09-11T07:28:36.773 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-11T07:28:36.773 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-11T07:28:36.773 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_max_listing_results": "1000", 2024-09-11T07:28:36.773 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_max_notify_retries": "10", 2024-09-11T07:28:36.773 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-11T07:28:36.773 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_max_put_param_size": "1048576", 2024-09-11T07:28:36.773 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_max_put_size": "5368709120", 2024-09-11T07:28:36.773 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_max_slo_entries": "1000", 2024-09-11T07:28:36.773 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_md_log_max_shards": "64", 2024-09-11T07:28:36.773 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-11T07:28:36.773 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-11T07:28:36.774 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_mp_lock_max_time": "600", 2024-09-11T07:28:36.774 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-11T07:28:36.774 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-11T07:28:36.774 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-11T07:28:36.774 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-11T07:28:36.774 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-11T07:28:36.774 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-11T07:28:36.774 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-11T07:28:36.774 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_nfs_max_gc": "300", 2024-09-11T07:28:36.774 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-11T07:28:36.774 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-11T07:28:36.774 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-11T07:28:36.774 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-11T07:28:36.774 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-11T07:28:36.775 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-11T07:28:36.775 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-11T07:28:36.775 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_num_async_rados_threads": "32", 2024-09-11T07:28:36.775 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_num_control_oids": "8", 2024-09-11T07:28:36.775 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_numa_node": "-1", 2024-09-11T07:28:36.775 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-11T07:28:36.775 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-11T07:28:36.775 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_objexp_chunk_size": "100", 2024-09-11T07:28:36.775 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_objexp_gc_interval": "600", 2024-09-11T07:28:36.775 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-11T07:28:36.775 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-11T07:28:36.775 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-11T07:28:36.775 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_op_thread_timeout": "600", 2024-09-11T07:28:36.775 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_op_tracing": "false", 2024-09-11T07:28:36.775 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_opa_token": "", 2024-09-11T07:28:36.776 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_opa_url": "", 2024-09-11T07:28:36.776 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_opa_verify_ssl": "true", 2024-09-11T07:28:36.776 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-11T07:28:36.776 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_ops_log_file_path": "", 2024-09-11T07:28:36.776 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_ops_log_rados": "true", 2024-09-11T07:28:36.776 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_ops_log_socket_path": "", 2024-09-11T07:28:36.776 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-11T07:28:36.776 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-11T07:28:36.776 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_period_push_interval": "2.000000", 2024-09-11T07:28:36.776 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-11T07:28:36.776 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-11T07:28:36.776 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_print_continue": "true", 2024-09-11T07:28:36.776 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-11T07:28:36.777 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-11T07:28:36.777 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-11T07:28:36.777 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-11T07:28:36.777 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-11T07:28:36.777 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_rados_tracing": "false", 2024-09-11T07:28:36.777 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_realm": "", 2024-09-11T07:28:36.777 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_realm_id": "", 2024-09-11T07:28:36.777 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-11T07:28:36.777 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_region": "", 2024-09-11T07:28:36.777 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-11T07:28:36.777 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-11T07:28:36.778 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-11T07:28:36.778 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-11T07:28:36.778 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_request_uri": "", 2024-09-11T07:28:36.778 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_reshard_batch_size": "64", 2024-09-11T07:28:36.778 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-11T07:28:36.778 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_reshard_max_aio": "128", 2024-09-11T07:28:36.778 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_reshard_num_logs": "16", 2024-09-11T07:28:36.778 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_reshard_thread_interval": "600", 2024-09-11T07:28:36.778 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_resolve_cname": "false", 2024-09-11T07:28:36.778 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-11T07:28:36.778 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_run_sync_thread": "true", 2024-09-11T07:28:36.778 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-11T07:28:36.778 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-11T07:28:36.778 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-11T07:28:36.779 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-11T07:28:36.779 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-11T07:28:36.779 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-11T07:28:36.779 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-11T07:28:36.779 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-11T07:28:36.779 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-11T07:28:36.779 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_scheduler_type": "throttler", 2024-09-11T07:28:36.779 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_script_uri": "", 2024-09-11T07:28:36.779 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_service_provider_name": "", 2024-09-11T07:28:36.779 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-11T07:28:36.779 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_sts_client_id": "", 2024-09-11T07:28:36.779 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_sts_client_secret": "", 2024-09-11T07:28:36.779 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_sts_entry": "sts", 2024-09-11T07:28:36.779 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_sts_key": "sts", 2024-09-11T07:28:36.779 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-11T07:28:36.780 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_sts_min_session_duration": "900", 2024-09-11T07:28:36.780 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_sts_token_introspection_url": "", 2024-09-11T07:28:36.780 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_swift_account_in_url": "false", 2024-09-11T07:28:36.780 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_swift_auth_entry": "auth", 2024-09-11T07:28:36.780 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_swift_auth_url": "", 2024-09-11T07:28:36.780 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_swift_custom_header": "", 2024-09-11T07:28:36.780 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-11T07:28:36.780 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_swift_need_stats": "true", 2024-09-11T07:28:36.780 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_swift_tenant_name": "", 2024-09-11T07:28:36.780 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_swift_token_expiration": "86400", 2024-09-11T07:28:36.780 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_swift_url": "", 2024-09-11T07:28:36.780 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_swift_url_prefix": "swift", 2024-09-11T07:28:36.780 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-11T07:28:36.781 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-11T07:28:36.781 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_sync_lease_period": "120", 2024-09-11T07:28:36.781 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-11T07:28:36.781 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-11T07:28:36.781 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-11T07:28:36.781 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-11T07:28:36.781 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-11T07:28:36.781 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-11T07:28:36.781 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-11T07:28:36.781 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-11T07:28:36.781 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-11T07:28:36.781 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_thread_pool_size": "512", 2024-09-11T07:28:36.781 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_torrent_comment": "", 2024-09-11T07:28:36.781 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_torrent_createby": "", 2024-09-11T07:28:36.782 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_torrent_encoding": "", 2024-09-11T07:28:36.782 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_torrent_flag": "false", 2024-09-11T07:28:36.782 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_torrent_origin": "", 2024-09-11T07:28:36.782 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-11T07:28:36.782 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_torrent_tracker": "", 2024-09-11T07:28:36.782 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_trust_forwarded_https": "false", 2024-09-11T07:28:36.782 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-11T07:28:36.782 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-11T07:28:36.782 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_usage_max_shards": "32", 2024-09-11T07:28:36.782 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_usage_max_user_shards": "1", 2024-09-11T07:28:36.782 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_use_opa_authz": "false", 2024-09-11T07:28:36.782 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-11T07:28:36.782 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-11T07:28:36.782 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_user_max_buckets": "1000", 2024-09-11T07:28:36.783 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-11T07:28:36.783 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-11T07:28:36.783 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-11T07:28:36.783 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-11T07:28:36.783 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_user_unique_email": "true", 2024-09-11T07:28:36.783 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_verify_ssl": "true", 2024-09-11T07:28:36.783 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-11T07:28:36.783 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_zone": "", 2024-09-11T07:28:36.783 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_zone_id": "", 2024-09-11T07:28:36.783 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-11T07:28:36.783 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_zonegroup": "", 2024-09-11T07:28:36.783 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_zonegroup_id": "", 2024-09-11T07:28:36.783 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-11T07:28:36.783 INFO:teuthology.orchestra.run.smithi022.stdout: "rocksdb_block_size": "4096", 2024-09-11T07:28:36.783 INFO:teuthology.orchestra.run.smithi022.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-11T07:28:36.784 INFO:teuthology.orchestra.run.smithi022.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-11T07:28:36.784 INFO:teuthology.orchestra.run.smithi022.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-11T07:28:36.784 INFO:teuthology.orchestra.run.smithi022.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-11T07:28:36.784 INFO:teuthology.orchestra.run.smithi022.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-11T07:28:36.784 INFO:teuthology.orchestra.run.smithi022.stdout: "rocksdb_cache_size": "536870912", 2024-09-11T07:28:36.784 INFO:teuthology.orchestra.run.smithi022.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-11T07:28:36.784 INFO:teuthology.orchestra.run.smithi022.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-11T07:28:36.784 INFO:teuthology.orchestra.run.smithi022.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-11T07:28:36.784 INFO:teuthology.orchestra.run.smithi022.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-11T07:28:36.784 INFO:teuthology.orchestra.run.smithi022.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-11T07:28:36.784 INFO:teuthology.orchestra.run.smithi022.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-11T07:28:36.784 INFO:teuthology.orchestra.run.smithi022.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-11T07:28:36.784 INFO:teuthology.orchestra.run.smithi022.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-11T07:28:36.784 INFO:teuthology.orchestra.run.smithi022.stdout: "rocksdb_index_type": "binary_search", 2024-09-11T07:28:36.785 INFO:teuthology.orchestra.run.smithi022.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-11T07:28:36.785 INFO:teuthology.orchestra.run.smithi022.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-11T07:28:36.785 INFO:teuthology.orchestra.run.smithi022.stdout: "rocksdb_partition_filters": "false", 2024-09-11T07:28:36.785 INFO:teuthology.orchestra.run.smithi022.stdout: "rocksdb_perf": "false", 2024-09-11T07:28:36.785 INFO:teuthology.orchestra.run.smithi022.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-11T07:28:36.785 INFO:teuthology.orchestra.run.smithi022.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-11T07:28:36.785 INFO:teuthology.orchestra.run.smithi022.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-11T07:28:36.785 INFO:teuthology.orchestra.run.smithi022.stdout: "run_dir": "/var/run/ceph", 2024-09-11T07:28:36.785 INFO:teuthology.orchestra.run.smithi022.stdout: "seastore_block_create": "true", 2024-09-11T07:28:36.785 INFO:teuthology.orchestra.run.smithi022.stdout: "seastore_cache_lru_size": "67108864", 2024-09-11T07:28:36.785 INFO:teuthology.orchestra.run.smithi022.stdout: "seastore_default_max_object_size": "16777216", 2024-09-11T07:28:36.785 INFO:teuthology.orchestra.run.smithi022.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-11T07:28:36.785 INFO:teuthology.orchestra.run.smithi022.stdout: "seastore_device_size": "107374182400", 2024-09-11T07:28:36.785 INFO:teuthology.orchestra.run.smithi022.stdout: "seastore_init_rewrite_segments_num_per_device": "6", 2024-09-11T07:28:36.786 INFO:teuthology.orchestra.run.smithi022.stdout: "seastore_journal_batch_capacity": "16", 2024-09-11T07:28:36.786 INFO:teuthology.orchestra.run.smithi022.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-11T07:28:36.786 INFO:teuthology.orchestra.run.smithi022.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-11T07:28:36.786 INFO:teuthology.orchestra.run.smithi022.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-11T07:28:36.786 INFO:teuthology.orchestra.run.smithi022.stdout: "seastore_segment_size": "67108864", 2024-09-11T07:28:36.786 INFO:teuthology.orchestra.run.smithi022.stdout: "setgroup": "ceph", 2024-09-11T07:28:36.786 INFO:teuthology.orchestra.run.smithi022.stdout: "setuser": "ceph", 2024-09-11T07:28:36.786 INFO:teuthology.orchestra.run.smithi022.stdout: "setuser_match_path": "", 2024-09-11T07:28:36.786 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-11T07:28:36.786 INFO:teuthology.orchestra.run.smithi022.stdout: "thp": "false", 2024-09-11T07:28:36.786 INFO:teuthology.orchestra.run.smithi022.stdout: "threadpool_default_timeout": "60", 2024-09-11T07:28:36.786 INFO:teuthology.orchestra.run.smithi022.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-11T07:28:36.786 INFO:teuthology.orchestra.run.smithi022.stdout: "throttler_perf_counter": "true" 2024-09-11T07:28:36.786 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:28:36.787 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:36 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:28:36.787 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:36 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:28:36.787 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:36 smithi104 ceph-mon[33507]: 1.0 deep-scrub starts 2024-09-11T07:28:36.787 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:36 smithi104 ceph-mon[33507]: 1.0 deep-scrub ok 2024-09-11T07:28:36.787 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:36 smithi104 ceph-mon[33507]: osdmap e88: 8 total, 8 up, 8 in 2024-09-11T07:28:36.787 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:36 smithi104 ceph-mon[33507]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:28:37.861 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-11T07:28:38.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:37 smithi104 ceph-mon[33507]: 4.7 scrub starts 2024-09-11T07:28:38.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:37 smithi104 ceph-mon[33507]: 4.7 scrub ok 2024-09-11T07:28:38.117 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:28:38.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:37 smithi022 ceph-mon[29462]: 4.7 scrub starts 2024-09-11T07:28:38.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:37 smithi022 ceph-mon[29462]: 4.7 scrub ok 2024-09-11T07:28:38.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:37 smithi022 ceph-mon[37330]: 4.7 scrub starts 2024-09-11T07:28:38.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:37 smithi022 ceph-mon[37330]: 4.7 scrub ok 2024-09-11T07:28:38.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:38 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:28:38.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:38 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:28:38.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:38 smithi022 ceph-mon[29462]: osdmap e89: 8 total, 8 up, 8 in 2024-09-11T07:28:38.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:38 smithi022 ceph-mon[29462]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:28:38.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:28:38.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:38 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:28:38.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:38 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:28:38.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:38 smithi022 ceph-mon[37330]: osdmap e89: 8 total, 8 up, 8 in 2024-09-11T07:28:38.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:38 smithi022 ceph-mon[37330]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:28:38.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:28:39.015 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:38 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:28:39.015 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:38 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:28:39.015 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:38 smithi104 ceph-mon[33507]: osdmap e89: 8 total, 8 up, 8 in 2024-09-11T07:28:39.015 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:38 smithi104 ceph-mon[33507]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:28:39.015 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:28:39.280 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:28:39 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:28:39.789 INFO:teuthology.orchestra.run.smithi022.stdout:mon.a: {} 2024-09-11T07:28:39.789 INFO:teuthology.orchestra.run.smithi022.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T07:28:39.798 INFO:teuthology.orchestra.run.smithi022.stdout:mon.b: {} 2024-09-11T07:28:39.798 INFO:teuthology.orchestra.run.smithi022.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T07:28:39.806 INFO:teuthology.orchestra.run.smithi022.stdout:mon.c: {} 2024-09-11T07:28:39.806 INFO:teuthology.orchestra.run.smithi022.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T07:28:39.815 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:39 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:28:39.815 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:39 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:28:40.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:39 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:28:40.334 INFO:teuthology.run_tasks:Running task full_sequential... 2024-09-11T07:28:40.355 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-11T07:28:40.357 INFO:tasks.radosbench:Beginning radosbench... 2024-09-11T07:28:40.357 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_0 2024-09-11T07:28:40.357 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pool create unique_pool_0 16 2024-09-11T07:28:40.358 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-11T07:28:40.358 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:28:40.358 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:28:40.358 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-11T07:28:40.358 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-11T07:28:40.359 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-11T07:28:40.359 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:28:40.359 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-11T07:28:40.360 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:28:40.360 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-11T07:28:40.360 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-11T07:28:40.361 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-11T07:28:40.361 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T07:28:40.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:40.389 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:40.389 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:28:40.490 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:28:40.515 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:40.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:40.517 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:28:40.617 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:28:40.647 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:40.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:40.648 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:28:40.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:28:40] "GET /metrics HTTP/1.1" 200 220053 "" "Prometheus/2.43.0" 2024-09-11T07:28:40.689 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:28:40.689 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:28:40.693 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:28:40.750 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:28:40.773 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:40.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:40.773 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:28:40.874 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:28:40.896 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:40.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:40.897 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:28:40.979 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:40 smithi022 ceph-mon[29462]: 6.1 deep-scrub starts 2024-09-11T07:28:40.979 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:40 smithi022 ceph-mon[29462]: 6.1 deep-scrub ok 2024-09-11T07:28:40.979 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:40 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4129021682' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:28:40.979 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:40 smithi022 ceph-mon[29462]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:28:40.980 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:40 smithi022 ceph-mon[37330]: 6.1 deep-scrub starts 2024-09-11T07:28:40.980 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:40 smithi022 ceph-mon[37330]: 6.1 deep-scrub ok 2024-09-11T07:28:40.980 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:40 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4129021682' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:28:40.980 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:40 smithi022 ceph-mon[37330]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:28:40.998 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:28:41.021 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:41.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:41.022 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:28:41.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:40 smithi104 ceph-mon[33507]: 6.1 deep-scrub starts 2024-09-11T07:28:41.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:40 smithi104 ceph-mon[33507]: 6.1 deep-scrub ok 2024-09-11T07:28:41.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:40 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4129021682' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:28:41.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:40 smithi104 ceph-mon[33507]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:28:41.123 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:28:41.145 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:41.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:41.146 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:28:41.247 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:28:41.269 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:41.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:41.270 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:28:41.371 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:28:41.393 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:41.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:41.394 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:28:41.494 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:28:41.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:41.517 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:41.518 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:28:41.618 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:28:41.644 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:41.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:41.645 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:28:41.746 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:28:41.771 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:41.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:41.772 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:28:41.873 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:28:41.896 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:41.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:41.897 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:28:41.998 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:28:42.021 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:42.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:42.021 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:28:42.122 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:28:42.144 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:42.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:42.145 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:28:42.246 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:28:42.272 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:42.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:42.273 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:28:42.374 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:28:42.398 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:42.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:42.399 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:28:42.500 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:28:42.522 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:42.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:42.523 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:28:42.624 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:28:42.650 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:42.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:42.651 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:28:42.752 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:28:42.774 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:42.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:42.775 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:28:42.876 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:28:42.901 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:42.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:42.902 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:28:43.003 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:28:43.025 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:43.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:43.026 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:28:43.099 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:42 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:28:43.099 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:42 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:28:43.099 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:42 smithi022 ceph-mon[29462]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:28:43.099 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:42 smithi022 ceph-mon[29462]: osdmap e90: 8 total, 8 up, 8 in 2024-09-11T07:28:43.100 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:42 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:28:43.100 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:42 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:28:43.100 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:42 smithi022 ceph-mon[37330]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:28:43.100 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:42 smithi022 ceph-mon[37330]: osdmap e90: 8 total, 8 up, 8 in 2024-09-11T07:28:43.127 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:28:43.149 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:43.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:43.151 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:28:43.251 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:28:43.274 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:43.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:43.275 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:28:43.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:42 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:28:43.282 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:42 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:28:43.282 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:42 smithi104 ceph-mon[33507]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:28:43.282 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:42 smithi104 ceph-mon[33507]: osdmap e90: 8 total, 8 up, 8 in 2024-09-11T07:28:43.375 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:28:43.401 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:43.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:43.402 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:28:43.503 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:28:43.528 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:43.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:43.529 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:28:43.629 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:28:43.653 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:43.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:43.654 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:28:43.677 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:28:43.678 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:28:43.755 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:28:43.778 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:43.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:43.779 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:28:43.879 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:28:43.901 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:43.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:43.902 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:28:44.003 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:28:44.029 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:44.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:44.030 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:28:44.131 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:28:44.156 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:44.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:44.157 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:28:44.258 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:28:44.283 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:44.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:44.284 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:28:44.385 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:28:44.408 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:44.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:44.409 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:28:44.509 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:28:44.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:28:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:28:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:28:44.534 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:44.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:44.535 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:28:44.636 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:28:44.658 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:44.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:44.659 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:28:44.760 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:28:44.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:44.786 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:44.786 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:28:44.887 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:28:44.909 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:44.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:44.910 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:28:45.011 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:28:45.037 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:45.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:45.038 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:28:45.139 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:28:45.165 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:45.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:45.166 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:28:45.266 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:28:45.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:44 smithi104 ceph-mon[33507]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:28:45.290 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:45.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:45.291 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:28:45.391 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:28:45.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:45.414 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:45.414 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:28:45.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:44 smithi022 ceph-mon[29462]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:28:45.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:44 smithi022 ceph-mon[37330]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:28:45.516 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:28:45.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:45.540 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:45.540 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:28:45.640 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:28:45.666 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:45.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:45.667 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:28:45.763 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:28:45.763 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:28:45.763 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 90, 2024-09-11T07:28:45.763 INFO:teuthology.orchestra.run.smithi022.stdout: "fsid": "12a285d8-700e-11ef-bcea-c7b262605968", 2024-09-11T07:28:45.763 INFO:teuthology.orchestra.run.smithi022.stdout: "created": "2024-09-11T07:21:18.695332+0000", 2024-09-11T07:28:45.763 INFO:teuthology.orchestra.run.smithi022.stdout: "modified": "2024-09-11T07:28:41.959172+0000", 2024-09-11T07:28:45.764 INFO:teuthology.orchestra.run.smithi022.stdout: "last_up_change": "2024-09-11T07:26:15.385688+0000", 2024-09-11T07:28:45.764 INFO:teuthology.orchestra.run.smithi022.stdout: "last_in_change": "2024-09-11T07:25:58.551813+0000", 2024-09-11T07:28:45.764 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-11T07:28:45.764 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_num": 5799936, 2024-09-11T07:28:45.764 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_set": [ 2024-09-11T07:28:45.764 INFO:teuthology.orchestra.run.smithi022.stdout: "pglog_hardlimit", 2024-09-11T07:28:45.764 INFO:teuthology.orchestra.run.smithi022.stdout: "purged_snapdirs", 2024-09-11T07:28:45.764 INFO:teuthology.orchestra.run.smithi022.stdout: "recovery_deletes", 2024-09-11T07:28:45.764 INFO:teuthology.orchestra.run.smithi022.stdout: "sortbitwise" 2024-09-11T07:28:45.764 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:28:45.764 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_version": 18, 2024-09-11T07:28:45.765 INFO:teuthology.orchestra.run.smithi022.stdout: "full_ratio": 0.94999998807907104, 2024-09-11T07:28:45.765 INFO:teuthology.orchestra.run.smithi022.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-11T07:28:45.765 INFO:teuthology.orchestra.run.smithi022.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-11T07:28:45.765 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_snapshot": "", 2024-09-11T07:28:45.765 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_max": 6, 2024-09-11T07:28:45.765 INFO:teuthology.orchestra.run.smithi022.stdout: "max_osd": 8, 2024-09-11T07:28:45.765 INFO:teuthology.orchestra.run.smithi022.stdout: "require_min_compat_client": "luminous", 2024-09-11T07:28:45.765 INFO:teuthology.orchestra.run.smithi022.stdout: "min_compat_client": "luminous", 2024-09-11T07:28:45.765 INFO:teuthology.orchestra.run.smithi022.stdout: "require_osd_release": "quincy", 2024-09-11T07:28:45.765 INFO:teuthology.orchestra.run.smithi022.stdout: "pools": [ 2024-09-11T07:28:45.765 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.765 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 1, 2024-09-11T07:28:45.766 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": ".mgr", 2024-09-11T07:28:45.766 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:24:21.772426+0000", 2024-09-11T07:28:45.766 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:28:45.766 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:28:45.766 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:28:45.766 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:28:45.766 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:28:45.766 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:28:45.766 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:28:45.766 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:28:45.766 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:28:45.766 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:28:45.766 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:28:45.766 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:28:45.767 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 1, 2024-09-11T07:28:45.767 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 1, 2024-09-11T07:28:45.767 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 1, 2024-09-11T07:28:45.767 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 1, 2024-09-11T07:28:45.767 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 1, 2024-09-11T07:28:45.767 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:28:45.767 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:28:45.767 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:28:45.767 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:28:45.767 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:28:45.767 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:28:45.767 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:28:45.767 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.767 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "21", 2024-09-11T07:28:45.767 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:28:45.768 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:28:45.768 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:28:45.768 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:28:45.768 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:28:45.768 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:28:45.768 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:28:45.768 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:28:45.768 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:28:45.769 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:28:45.769 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:28:45.769 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:28:45.769 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:28:45.769 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:28:45.769 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:28:45.769 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:28:45.769 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:28:45.769 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:28:45.769 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:28:45.769 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:28:45.769 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:28:45.770 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:28:45.770 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:28:45.770 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:28:45.770 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:28:45.770 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:28:45.770 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.770 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:28:45.770 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:28:45.770 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:28:45.770 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:28:45.770 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:28:45.770 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:28:45.771 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:28:45.771 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:28:45.771 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:28:45.771 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:28:45.771 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:28:45.771 INFO:teuthology.orchestra.run.smithi022.stdout: "options": { 2024-09-11T07:28:45.771 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_max": 32, 2024-09-11T07:28:45.771 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_min": 1 2024-09-11T07:28:45.771 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.771 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:28:45.771 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": {} 2024-09-11T07:28:45.771 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.771 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.771 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.772 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 2, 2024-09-11T07:28:45.772 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": ".rgw.root", 2024-09-11T07:28:45.772 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:29.941354+0000", 2024-09-11T07:28:45.772 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:28:45.772 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:28:45.772 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:28:45.772 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:28:45.772 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:28:45.772 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:28:45.772 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:28:45.772 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:28:45.772 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:28:45.772 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:28:45.773 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:28:45.773 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:28:45.773 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:28:45.773 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:28:45.773 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:28:45.773 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:28:45.773 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:28:45.773 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:28:45.773 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:28:45.773 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:28:45.773 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:28:45.773 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:28:45.773 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:28:45.773 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:28:45.773 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.774 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "58", 2024-09-11T07:28:45.774 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:28:45.774 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:28:45.774 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:28:45.774 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:28:45.774 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:28:45.774 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:28:45.774 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:28:45.774 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:28:45.774 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:28:45.774 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:28:45.774 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:28:45.774 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:28:45.774 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:28:45.774 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:28:45.775 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:28:45.775 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:28:45.775 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:28:45.775 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:28:45.775 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:28:45.775 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:28:45.775 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:28:45.775 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:28:45.775 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:28:45.775 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:28:45.775 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:28:45.775 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:28:45.775 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.775 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:28:45.775 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:28:45.775 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:28:45.776 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:28:45.776 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:28:45.776 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:28:45.776 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:28:45.776 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:28:45.776 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:28:45.776 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:28:45.776 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:28:45.776 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:28:45.776 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:28:45.776 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:28:45.776 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.776 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.776 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.777 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 3, 2024-09-11T07:28:45.777 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "datapool", 2024-09-11T07:28:45.777 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:30.396859+0000", 2024-09-11T07:28:45.777 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 8193, 2024-09-11T07:28:45.777 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T07:28:45.777 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:28:45.777 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:28:45.777 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:28:45.777 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:28:45.777 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:28:45.777 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:28:45.777 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:28:45.777 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:28:45.777 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:28:45.778 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:28:45.778 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 3, 2024-09-11T07:28:45.778 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 3, 2024-09-11T07:28:45.778 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 3, 2024-09-11T07:28:45.778 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 3, 2024-09-11T07:28:45.778 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 3, 2024-09-11T07:28:45.778 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:28:45.778 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:28:45.778 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:28:45.778 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:28:45.778 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:28:45.778 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:28:45.778 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:28:45.778 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.779 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "61", 2024-09-11T07:28:45.779 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:28:45.779 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:28:45.779 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:28:45.779 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:28:45.779 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:28:45.779 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 3, 2024-09-11T07:28:45.779 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 61, 2024-09-11T07:28:45.779 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:28:45.779 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:28:45.779 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:28:45.779 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:28:45.779 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:28:45.779 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:28:45.780 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:28:45.780 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:28:45.780 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:28:45.780 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:28:45.780 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:28:45.780 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:28:45.780 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:28:45.780 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:28:45.780 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:28:45.780 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:28:45.780 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:28:45.780 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:28:45.780 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:28:45.780 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.780 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:28:45.781 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:28:45.782 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:28:45.782 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:28:45.782 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:28:45.782 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:28:45.782 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:28:45.782 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:28:45.782 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:28:45.782 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:28:45.782 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:28:45.782 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:28:45.783 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:28:45.783 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:28:45.783 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd": {} 2024-09-11T07:28:45.783 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.783 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.783 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.783 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 4, 2024-09-11T07:28:45.783 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.log", 2024-09-11T07:28:45.783 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:31.922926+0000", 2024-09-11T07:28:45.783 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:28:45.783 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:28:45.783 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:28:45.783 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:28:45.784 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:28:45.784 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:28:45.784 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:28:45.784 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:28:45.784 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:28:45.784 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:28:45.784 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:28:45.784 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:28:45.784 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:28:45.784 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:28:45.784 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:28:45.784 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:28:45.784 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:28:45.784 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:28:45.785 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:28:45.785 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:28:45.785 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:28:45.785 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:28:45.785 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:28:45.785 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:28:45.785 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.785 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "60", 2024-09-11T07:28:45.785 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:28:45.785 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:28:45.785 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:28:45.785 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:28:45.785 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:28:45.785 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:28:45.785 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:28:45.786 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:28:45.786 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:28:45.786 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:28:45.786 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:28:45.786 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:28:45.786 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:28:45.786 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:28:45.786 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:28:45.786 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:28:45.786 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:28:45.786 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:28:45.786 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:28:45.786 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:28:45.786 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:28:45.786 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:28:45.787 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:28:45.787 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:28:45.787 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:28:45.787 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:28:45.787 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.787 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:28:45.787 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:28:45.787 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:28:45.787 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:28:45.787 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:28:45.787 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:28:45.787 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:28:45.787 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:28:45.788 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:28:45.788 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:28:45.788 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:28:45.788 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:28:45.788 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:28:45.788 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:28:45.788 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.788 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.788 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.788 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 5, 2024-09-11T07:28:45.788 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.control", 2024-09-11T07:28:45.788 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:34.007746+0000", 2024-09-11T07:28:45.789 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:28:45.789 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:28:45.789 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:28:45.789 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:28:45.789 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:28:45.789 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:28:45.789 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:28:45.789 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:28:45.789 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:28:45.789 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:28:45.789 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:28:45.789 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:28:45.789 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:28:45.789 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:28:45.789 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:28:45.790 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:28:45.790 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:28:45.790 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:28:45.790 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:28:45.790 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:28:45.790 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:28:45.790 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:28:45.790 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:28:45.790 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:28:45.790 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.790 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "62", 2024-09-11T07:28:45.790 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:28:45.790 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:28:45.791 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:28:45.791 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:28:45.791 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:28:45.791 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:28:45.791 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:28:45.791 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:28:45.791 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:28:45.791 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:28:45.791 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:28:45.791 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:28:45.791 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:28:45.791 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:28:45.791 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:28:45.791 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:28:45.792 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:28:45.792 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:28:45.792 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:28:45.792 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:28:45.792 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:28:45.792 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:28:45.792 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:28:45.792 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:28:45.792 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:28:45.792 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:28:45.792 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.792 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:28:45.792 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:28:45.792 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:28:45.793 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:28:45.793 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:28:45.793 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:28:45.793 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:28:45.793 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:28:45.793 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:28:45.793 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:28:45.793 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:28:45.793 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:28:45.793 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:28:45.793 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:28:45.793 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.793 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.793 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.794 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 6, 2024-09-11T07:28:45.794 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.meta", 2024-09-11T07:28:45.794 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:36.077215+0000", 2024-09-11T07:28:45.794 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:28:45.794 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:28:45.794 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:28:45.794 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:28:45.794 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:28:45.794 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:28:45.794 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:28:45.794 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:28:45.794 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:28:45.794 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:28:45.794 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:28:45.794 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:28:45.794 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:28:45.795 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:28:45.795 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:28:45.795 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:28:45.795 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:28:45.795 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:28:45.795 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:28:45.795 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:28:45.795 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:28:45.795 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:28:45.795 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:28:45.795 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:28:45.795 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.795 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "64", 2024-09-11T07:28:45.795 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:28:45.795 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:28:45.796 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:28:45.796 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:28:45.796 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:28:45.796 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:28:45.796 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:28:45.796 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:28:45.796 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:28:45.796 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:28:45.796 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:28:45.796 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:28:45.796 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:28:45.796 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:28:45.796 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:28:45.796 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:28:45.797 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:28:45.797 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:28:45.797 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:28:45.797 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:28:45.797 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:28:45.797 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:28:45.797 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:28:45.797 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:28:45.797 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:28:45.797 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:28:45.797 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.797 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:28:45.797 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:28:45.797 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:28:45.797 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:28:45.798 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:28:45.798 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:28:45.798 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:28:45.798 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:28:45.798 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:28:45.798 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:28:45.798 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:28:45.798 INFO:teuthology.orchestra.run.smithi022.stdout: "options": { 2024-09-11T07:28:45.798 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_bias": 4 2024-09-11T07:28:45.798 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.798 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:28:45.798 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:28:45.798 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.799 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.799 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:28:45.799 INFO:teuthology.orchestra.run.smithi022.stdout: "osds": [ 2024-09-11T07:28:45.799 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.799 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 0, 2024-09-11T07:28:45.799 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "09075f16-32f4-4f88-a116-c3d7f977a0e0", 2024-09-11T07:28:45.799 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:28:45.799 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:28:45.799 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:28:45.799 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:28:45.799 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:28:45.799 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:28:45.799 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 8, 2024-09-11T07:28:45.799 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 85, 2024-09-11T07:28:45.799 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:28:45.800 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:28:45.800 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:28:45.800 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.800 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.800 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.800 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6801", 2024-09-11T07:28:45.800 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:28:45.800 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.800 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.800 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.800 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:28:45.800 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.800 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.800 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.800 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6802", 2024-09-11T07:28:45.801 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:28:45.801 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.801 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.801 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.801 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:28:45.801 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.801 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.801 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.801 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6804", 2024-09-11T07:28:45.801 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:28:45.801 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.801 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.801 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.801 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:28:45.801 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.802 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.802 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.802 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6803", 2024-09-11T07:28:45.802 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:28:45.802 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.802 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.802 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.802 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6801/2821493591", 2024-09-11T07:28:45.802 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6802/2821493591", 2024-09-11T07:28:45.802 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6804/2821493591", 2024-09-11T07:28:45.802 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6803/2821493591", 2024-09-11T07:28:45.802 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:28:45.802 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:28:45.802 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:28:45.803 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.803 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.803 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.803 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 1, 2024-09-11T07:28:45.803 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "52cc1e06-9506-48c0-9f7b-d4270dff4b9f", 2024-09-11T07:28:45.803 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:28:45.803 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:28:45.803 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:28:45.803 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:28:45.803 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:28:45.803 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:28:45.803 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 13, 2024-09-11T07:28:45.803 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 62, 2024-09-11T07:28:45.803 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:28:45.803 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:28:45.804 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:28:45.804 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.804 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.804 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.804 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6805", 2024-09-11T07:28:45.804 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:28:45.804 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.804 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.804 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.804 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:28:45.804 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.804 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.804 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.804 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6806", 2024-09-11T07:28:45.804 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:28:45.805 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.805 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.805 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.805 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:28:45.805 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.805 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.805 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.805 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6808", 2024-09-11T07:28:45.805 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:28:45.805 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.805 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.805 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.805 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:28:45.805 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.806 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.806 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.806 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6807", 2024-09-11T07:28:45.806 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:28:45.806 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.806 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.806 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.806 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6805/4037911317", 2024-09-11T07:28:45.806 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6806/4037911317", 2024-09-11T07:28:45.806 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6808/4037911317", 2024-09-11T07:28:45.806 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6807/4037911317", 2024-09-11T07:28:45.806 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:28:45.806 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:28:45.806 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:28:45.806 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.807 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.807 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.807 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 2, 2024-09-11T07:28:45.807 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "fa2b2730-2c1d-45d0-be8a-494a60dc6b9b", 2024-09-11T07:28:45.807 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:28:45.807 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:28:45.807 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:28:45.807 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:28:45.807 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:28:45.807 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:28:45.807 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 18, 2024-09-11T07:28:45.807 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 62, 2024-09-11T07:28:45.807 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:28:45.807 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:28:45.807 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:28:45.808 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.808 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.808 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.808 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6809", 2024-09-11T07:28:45.808 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:28:45.808 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.808 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.808 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.808 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:28:45.808 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.808 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.808 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.808 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6810", 2024-09-11T07:28:45.808 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:28:45.808 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.809 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.809 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.809 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:28:45.809 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.809 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.809 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.809 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6812", 2024-09-11T07:28:45.809 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:28:45.809 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.809 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.809 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.809 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:28:45.809 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.809 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.810 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.810 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6811", 2024-09-11T07:28:45.810 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:28:45.810 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.810 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.810 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.810 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6809/1675799666", 2024-09-11T07:28:45.810 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6810/1675799666", 2024-09-11T07:28:45.810 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6812/1675799666", 2024-09-11T07:28:45.810 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6811/1675799666", 2024-09-11T07:28:45.811 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:28:45.811 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:28:45.811 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:28:45.811 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.811 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.811 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.811 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 3, 2024-09-11T07:28:45.811 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "8d53d3fc-e401-47df-b70b-4b4b1f4539df", 2024-09-11T07:28:45.811 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:28:45.811 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:28:45.811 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:28:45.811 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:28:45.812 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:28:45.812 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:28:45.812 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 25, 2024-09-11T07:28:45.812 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 62, 2024-09-11T07:28:45.812 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:28:45.812 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:28:45.812 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:28:45.812 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.812 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.812 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.812 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6813", 2024-09-11T07:28:45.812 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:28:45.812 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.813 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.813 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.813 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:28:45.813 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.813 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.813 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.813 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6814", 2024-09-11T07:28:45.813 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:28:45.813 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.813 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.813 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.813 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:28:45.814 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.814 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.814 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.814 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6816", 2024-09-11T07:28:45.814 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:28:45.814 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.814 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.814 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.814 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:28:45.814 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.815 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.815 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.815 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6815", 2024-09-11T07:28:45.815 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:28:45.815 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.815 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.815 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.815 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6813/1501962752", 2024-09-11T07:28:45.815 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6814/1501962752", 2024-09-11T07:28:45.815 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6816/1501962752", 2024-09-11T07:28:45.815 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6815/1501962752", 2024-09-11T07:28:45.815 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:28:45.815 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:28:45.815 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:28:45.816 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.816 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.816 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.816 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 4, 2024-09-11T07:28:45.816 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "d3207fe0-8b97-4067-a755-ba6d449c53c7", 2024-09-11T07:28:45.816 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:28:45.816 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:28:45.816 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:28:45.816 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:28:45.816 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:28:45.816 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:28:45.816 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 31, 2024-09-11T07:28:45.816 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 85, 2024-09-11T07:28:45.816 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:28:45.816 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:28:45.817 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:28:45.817 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.817 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.817 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.817 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6800", 2024-09-11T07:28:45.817 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:28:45.817 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.817 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.817 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.817 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:28:45.817 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.817 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.817 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.818 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6801", 2024-09-11T07:28:45.818 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:28:45.818 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.818 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.818 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.818 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:28:45.818 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.818 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.818 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.818 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6803", 2024-09-11T07:28:45.818 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:28:45.818 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.818 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.819 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.819 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:28:45.819 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.819 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.819 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.819 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6802", 2024-09-11T07:28:45.819 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:28:45.819 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.819 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.819 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.819 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6800/1210229443", 2024-09-11T07:28:45.819 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6801/1210229443", 2024-09-11T07:28:45.819 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6803/1210229443", 2024-09-11T07:28:45.819 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6802/1210229443", 2024-09-11T07:28:45.819 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:28:45.820 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:28:45.820 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:28:45.820 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.820 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.820 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.820 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 5, 2024-09-11T07:28:45.820 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "89c9352b-30b2-48c6-869e-047d07a23a43", 2024-09-11T07:28:45.820 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:28:45.820 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:28:45.820 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:28:45.820 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:28:45.820 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:28:45.820 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:28:45.820 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 37, 2024-09-11T07:28:45.820 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 62, 2024-09-11T07:28:45.821 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:28:45.821 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:28:45.821 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:28:45.821 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.821 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.821 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.821 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6804", 2024-09-11T07:28:45.821 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:28:45.821 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.821 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.821 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.821 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:28:45.821 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.822 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.822 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.822 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6805", 2024-09-11T07:28:45.822 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:28:45.822 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.822 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.822 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.822 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:28:45.822 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.822 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.822 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.822 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6807", 2024-09-11T07:28:45.822 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:28:45.822 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.822 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.823 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.823 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:28:45.823 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.823 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.823 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.823 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6806", 2024-09-11T07:28:45.823 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:28:45.823 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.823 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.823 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.824 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6804/2735328732", 2024-09-11T07:28:45.824 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6805/2735328732", 2024-09-11T07:28:45.824 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6807/2735328732", 2024-09-11T07:28:45.824 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6806/2735328732", 2024-09-11T07:28:45.824 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:28:45.824 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:28:45.824 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:28:45.824 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.824 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.824 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.824 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 6, 2024-09-11T07:28:45.824 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "704563de-d88f-480a-b469-720135704d42", 2024-09-11T07:28:45.824 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:28:45.824 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:28:45.824 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:28:45.825 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:28:45.825 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:28:45.825 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:28:45.825 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 44, 2024-09-11T07:28:45.825 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 85, 2024-09-11T07:28:45.825 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:28:45.825 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:28:45.825 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:28:45.825 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.825 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.825 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.825 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6808", 2024-09-11T07:28:45.825 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:28:45.826 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.826 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.826 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.826 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:28:45.826 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.826 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.826 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.826 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6809", 2024-09-11T07:28:45.826 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:28:45.826 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.826 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.826 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.826 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:28:45.826 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.826 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.827 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.827 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6811", 2024-09-11T07:28:45.827 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:28:45.827 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.827 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.827 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.827 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:28:45.827 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.827 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.827 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.827 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6810", 2024-09-11T07:28:45.827 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:28:45.828 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.828 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.828 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.828 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6808/721493010", 2024-09-11T07:28:45.828 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6809/721493010", 2024-09-11T07:28:45.828 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6811/721493010", 2024-09-11T07:28:45.828 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6810/721493010", 2024-09-11T07:28:45.828 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:28:45.828 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:28:45.828 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:28:45.828 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.828 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.828 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.828 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 7, 2024-09-11T07:28:45.828 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "18199e47-0052-49f0-b45f-0fb201e0d0f4", 2024-09-11T07:28:45.829 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:28:45.829 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:28:45.829 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:28:45.829 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:28:45.829 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:28:45.829 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:28:45.829 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 52, 2024-09-11T07:28:45.829 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 62, 2024-09-11T07:28:45.829 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:28:45.829 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:28:45.829 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:28:45.829 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.829 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.829 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.829 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6812", 2024-09-11T07:28:45.830 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3018423954 2024-09-11T07:28:45.830 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.830 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.830 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.830 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:28:45.830 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.830 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.830 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.830 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6813", 2024-09-11T07:28:45.830 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3018423954 2024-09-11T07:28:45.830 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.830 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.830 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.830 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:28:45.830 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.830 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.831 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.831 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6815", 2024-09-11T07:28:45.831 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3018423954 2024-09-11T07:28:45.831 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.831 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.831 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.831 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:28:45.831 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:28:45.831 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.831 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:28:45.831 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6814", 2024-09-11T07:28:45.831 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3018423954 2024-09-11T07:28:45.831 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.831 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.831 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.832 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6812/3018423954", 2024-09-11T07:28:45.832 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6813/3018423954", 2024-09-11T07:28:45.832 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6815/3018423954", 2024-09-11T07:28:45.832 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6814/3018423954", 2024-09-11T07:28:45.832 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:28:45.832 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:28:45.832 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:28:45.832 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.832 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.832 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:28:45.832 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_xinfo": [ 2024-09-11T07:28:45.832 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.832 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 0, 2024-09-11T07:28:45.832 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:28:45.833 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:28:45.833 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:28:45.833 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:28:45.833 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:28:45.833 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:27:24.407137+0000", 2024-09-11T07:28:45.833 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:28:45.833 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.833 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.833 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 1, 2024-09-11T07:28:45.833 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:28:45.833 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:28:45.833 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:28:45.833 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:28:45.833 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:28:45.834 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:27:37.722267+0000", 2024-09-11T07:28:45.834 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:28:45.834 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.834 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.834 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 2, 2024-09-11T07:28:45.834 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:28:45.834 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:28:45.834 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:28:45.834 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:28:45.834 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:28:45.834 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:27:58.992759+0000", 2024-09-11T07:28:45.834 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:28:45.834 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.834 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.835 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 3, 2024-09-11T07:28:45.835 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:28:45.835 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:28:45.835 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:28:45.835 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:28:45.835 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:28:45.835 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:28:41.908744+0000", 2024-09-11T07:28:45.835 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:28:45.835 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.835 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.835 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 4, 2024-09-11T07:28:45.835 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:28:45.835 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:28:45.835 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:28:45.835 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:28:45.836 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:28:45.836 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:28:37.656398+0000", 2024-09-11T07:28:45.836 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:28:45.836 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.836 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.836 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 5, 2024-09-11T07:28:45.836 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:28:45.836 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:28:45.837 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:28:45.837 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:28:45.837 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:28:45.837 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:28:04.150344+0000", 2024-09-11T07:28:45.837 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:28:45.837 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.837 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.837 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 6, 2024-09-11T07:28:45.837 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:28:45.837 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:28:45.837 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:28:45.837 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:28:45.837 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:28:45.837 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:28:26.989477+0000", 2024-09-11T07:28:45.837 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:28:45.838 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.838 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.838 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 7, 2024-09-11T07:28:45.838 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:28:45.838 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:28:45.838 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:28:45.838 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:28:45.838 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:28:45.838 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:28:34.821339+0000", 2024-09-11T07:28:45.838 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:28:45.838 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.838 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:28:45.838 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_upmap": [], 2024-09-11T07:28:45.839 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_upmap_items": [ 2024-09-11T07:28:45.839 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.839 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "2.7", 2024-09-11T07:28:45.839 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:28:45.839 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.839 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 7, 2024-09-11T07:28:45.839 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 4 2024-09-11T07:28:45.839 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.839 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.839 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.839 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.839 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.6", 2024-09-11T07:28:45.839 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:28:45.839 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.840 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 1, 2024-09-11T07:28:45.840 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 5 2024-09-11T07:28:45.840 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.840 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.840 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.840 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.840 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.d", 2024-09-11T07:28:45.840 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:28:45.840 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.840 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 1, 2024-09-11T07:28:45.840 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 5 2024-09-11T07:28:45.840 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.840 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.840 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.840 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.841 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.1f", 2024-09-11T07:28:45.841 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:28:45.841 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:28:45.841 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 1, 2024-09-11T07:28:45.841 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 2 2024-09-11T07:28:45.841 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.841 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:28:45.841 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.841 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:28:45.841 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_temp": [], 2024-09-11T07:28:45.841 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_temp": [], 2024-09-11T07:28:45.841 INFO:teuthology.orchestra.run.smithi022.stdout: "blocklist": { 2024-09-11T07:28:45.841 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/240999923": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:28:45.841 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/4016369460": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:28:45.842 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/585929238": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:28:45.842 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/544031024": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:28:45.842 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/2504196132": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:28:45.842 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/1451388349": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:28:45.842 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/639336081": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:28:45.842 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2104069730": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:28:45.842 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/4236450731": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:28:45.842 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/3573193776": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:28:45.842 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2661764422": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:28:45.842 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2197488530": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:28:45.842 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2968034970": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:28:45.842 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/599318175": "2024-09-12T07:27:07.919998+0000" 2024-09-11T07:28:45.842 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.842 INFO:teuthology.orchestra.run.smithi022.stdout: "range_blocklist": {}, 2024-09-11T07:28:45.842 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profiles": { 2024-09-11T07:28:45.842 INFO:teuthology.orchestra.run.smithi022.stdout: "default": { 2024-09-11T07:28:45.843 INFO:teuthology.orchestra.run.smithi022.stdout: "crush-failure-domain": "osd", 2024-09-11T07:28:45.843 INFO:teuthology.orchestra.run.smithi022.stdout: "k": "2", 2024-09-11T07:28:45.843 INFO:teuthology.orchestra.run.smithi022.stdout: "m": "1", 2024-09-11T07:28:45.843 INFO:teuthology.orchestra.run.smithi022.stdout: "plugin": "jerasure", 2024-09-11T07:28:45.843 INFO:teuthology.orchestra.run.smithi022.stdout: "technique": "reed_sol_van" 2024-09-11T07:28:45.843 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.843 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:28:45.843 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps_queue": [], 2024-09-11T07:28:45.843 INFO:teuthology.orchestra.run.smithi022.stdout: "new_removed_snaps": [], 2024-09-11T07:28:45.843 INFO:teuthology.orchestra.run.smithi022.stdout: "new_purged_snaps": [], 2024-09-11T07:28:45.843 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_node_flags": {}, 2024-09-11T07:28:45.843 INFO:teuthology.orchestra.run.smithi022.stdout: "device_class_flags": {}, 2024-09-11T07:28:45.844 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode": { 2024-09-11T07:28:45.844 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode_enabled": false, 2024-09-11T07:28:45.844 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_bucket_count": 0, 2024-09-11T07:28:45.844 INFO:teuthology.orchestra.run.smithi022.stdout: "degraded_stretch_mode": 0, 2024-09-11T07:28:45.844 INFO:teuthology.orchestra.run.smithi022.stdout: "recovering_stretch_mode": 0, 2024-09-11T07:28:45.844 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode_bucket": 0 2024-09-11T07:28:45.844 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:28:45.844 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:28:45.858 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:45.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:45.859 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:28:45.959 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:28:45.983 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:45.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:45.984 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:28:46.084 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:28:46.110 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:46.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:46.111 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:28:46.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:45 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/748124872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:28:46.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:45 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/748124872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:28:46.212 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:28:46.236 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:46.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:46.237 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:28:46.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:45 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/748124872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:28:46.338 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:28:46.362 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:46.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:46.363 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:28:46.464 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:28:46.487 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:46.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:46.488 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:28:46.589 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:28:46.615 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:46.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:46.616 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:28:46.717 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:28:46.741 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:46.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:46.742 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:28:46.842 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:28:46.865 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:46.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:46.866 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:28:46.967 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:28:46.992 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:46.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:46.993 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:28:47.094 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:28:47.119 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:47.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:47.120 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:28:47.221 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:28:47.243 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:47.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:47.244 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:28:47.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:46 smithi104 ceph-mon[33507]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:28:47.345 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:28:47.371 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:47.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:47.372 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:28:47.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:46 smithi022 ceph-mon[29462]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:28:47.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:46 smithi022 ceph-mon[37330]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:28:47.473 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:28:47.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:47.498 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:47.498 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:28:47.599 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:28:47.624 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:47.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:47.625 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:28:47.726 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:28:47.751 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:47.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:47.752 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:28:47.839 INFO:tasks.thrashosds.thrasher:j is {'epoch': 90, 'fsid': '12a285d8-700e-11ef-bcea-c7b262605968', 'created': '2024-09-11T07:21:18.695332+0000', 'modified': '2024-09-11T07:28:41.959172+0000', 'last_up_change': '2024-09-11T07:26:15.385688+0000', 'last_in_change': '2024-09-11T07:25:58.551813+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 6, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T07:24:21.772426+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': '.rgw.root', 'create_time': '2024-09-11T07:26:29.941354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 3, 'pool_name': 'datapool', 'create_time': '2024-09-11T07:26:30.396859+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-11T07:26:31.922926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-11T07:26:34.007746+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-11T07:26:36.077215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}], 'osds': [{'osd': 0, 'uuid': '09075f16-32f4-4f88-a116-c3d7f977a0e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 85, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6801', 'nonce': 2821493591}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6802', 'nonce': 2821493591}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6804', 'nonce': 2821493591}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6803', 'nonce': 2821493591}]}, 'public_addr': '172.21.15.22:6801/2821493591', 'cluster_addr': '172.21.15.22:6802/2821493591', 'heartbeat_back_addr': '172.21.15.22:6804/2821493591', 'heartbeat_front_addr': '172.21.15.22:6803/2821493591', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '52cc1e06-9506-48c0-9f7b-d4270dff4b9f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 62, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6805', 'nonce': 4037911317}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6806', 'nonce': 4037911317}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6808', 'nonce': 4037911317}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6807', 'nonce': 4037911317}]}, 'public_addr': '172.21.15.22:6805/4037911317', 'cluster_addr': '172.21.15.22:6806/4037911317', 'heartbeat_back_addr': '172.21.15.22:6808/4037911317', 'heartbeat_front_addr': '172.21.15.22:6807/4037911317', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'fa2b2730-2c1d-45d0-be8a-494a60dc6b9b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 62, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6809', 'nonce': 1675799666}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6810', 'nonce': 1675799666}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6812', 'nonce': 1675799666}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6811', 'nonce': 1675799666}]}, 'public_addr': '172.21.15.22:6809/1675799666', 'cluster_addr': '172.21.15.22:6810/1675799666', 'heartbeat_back_addr': '172.21.15.22:6812/1675799666', 'heartbeat_front_addr': '172.21.15.22:6811/1675799666', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '8d53d3fc-e401-47df-b70b-4b4b1f4539df', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 25, 'up_thru': 62, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6813', 'nonce': 1501962752}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6814', 'nonce': 1501962752}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6816', 'nonce': 1501962752}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6815', 'nonce': 1501962752}]}, 'public_addr': '172.21.15.22:6813/1501962752', 'cluster_addr': '172.21.15.22:6814/1501962752', 'heartbeat_back_addr': '172.21.15.22:6816/1501962752', 'heartbeat_front_addr': '172.21.15.22:6815/1501962752', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'd3207fe0-8b97-4067-a755-ba6d449c53c7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 85, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6800', 'nonce': 1210229443}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6801', 'nonce': 1210229443}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6803', 'nonce': 1210229443}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6802', 'nonce': 1210229443}]}, 'public_addr': '172.21.15.104:6800/1210229443', 'cluster_addr': '172.21.15.104:6801/1210229443', 'heartbeat_back_addr': '172.21.15.104:6803/1210229443', 'heartbeat_front_addr': '172.21.15.104:6802/1210229443', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '89c9352b-30b2-48c6-869e-047d07a23a43', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 37, 'up_thru': 62, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6804', 'nonce': 2735328732}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6805', 'nonce': 2735328732}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6807', 'nonce': 2735328732}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6806', 'nonce': 2735328732}]}, 'public_addr': '172.21.15.104:6804/2735328732', 'cluster_addr': '172.21.15.104:6805/2735328732', 'heartbeat_back_addr': '172.21.15.104:6807/2735328732', 'heartbeat_front_addr': '172.21.15.104:6806/2735328732', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '704563de-d88f-480a-b469-720135704d42', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 44, 'up_thru': 85, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6808', 'nonce': 721493010}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6809', 'nonce': 721493010}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6811', 'nonce': 721493010}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6810', 'nonce': 721493010}]}, 'public_addr': '172.21.15.104:6808/721493010', 'cluster_addr': '172.21.15.104:6809/721493010', 'heartbeat_back_addr': '172.21.15.104:6811/721493010', 'heartbeat_front_addr': '172.21.15.104:6810/721493010', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '18199e47-0052-49f0-b45f-0fb201e0d0f4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 52, 'up_thru': 62, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6812', 'nonce': 3018423954}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6813', 'nonce': 3018423954}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6815', 'nonce': 3018423954}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6814', 'nonce': 3018423954}]}, 'public_addr': '172.21.15.104:6812/3018423954', 'cluster_addr': '172.21.15.104:6813/3018423954', 'heartbeat_back_addr': '172.21.15.104:6815/3018423954', 'heartbeat_front_addr': '172.21.15.104:6814/3018423954', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:27:24.407137+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:27:37.722267+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:27:58.992759+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:28:41.908744+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:28:37.656398+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:28:04.150344+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:28:26.989477+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:28:34.821339+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '2.7', 'mappings': [{'from': 7, 'to': 4}]}, {'pgid': '4.6', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.d', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.1f', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.22:0/240999923': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:6800/4016369460': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/585929238': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/544031024': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:6800/2504196132': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/1451388349': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/639336081': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/2104069730': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/4236450731': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:6800/3573193776': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/2661764422': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/2197488530': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/2968034970': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/599318175': '2024-09-12T07:27:07.919998+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-11T07:28:47.839 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:28:47.853 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:28:47.906 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:47.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:47.907 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:28:48.008 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:28:48.034 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:48.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:48.035 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:28:48.071 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:47 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:28:48.071 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:47 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:28:48.071 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:47 smithi022 ceph-mon[29462]: osdmap e91: 8 total, 8 up, 8 in 2024-09-11T07:28:48.071 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:47 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:28:48.071 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:47 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:28:48.071 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:47 smithi022 ceph-mon[37330]: osdmap e91: 8 total, 8 up, 8 in 2024-09-11T07:28:48.136 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:28:48.158 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:48.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:48.158 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:28:48.259 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:28:48.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:47 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:28:48.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:47 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:28:48.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:47 smithi104 ceph-mon[33507]: osdmap e91: 8 total, 8 up, 8 in 2024-09-11T07:28:48.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:48.286 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:48.286 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:28:48.386 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:28:48.412 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:48.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:48.413 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:28:48.434 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:28:48.434 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:28:48.434 INFO:teuthology.orchestra.run.smithi022.stdout: "num_ops": 0 2024-09-11T07:28:48.434 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:28:48.514 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:28:48.538 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:48.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:48.539 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:28:48.640 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:28:48.666 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:48.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:48.667 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:28:48.768 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:28:48.794 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:48.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:48.794 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:28:48.895 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:28:48.917 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:48.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:48.918 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:28:48.985 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'unique_pool_0' created 2024-09-11T07:28:49.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:48 smithi022 ceph-mon[29462]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:28:49.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:48 smithi022 ceph-mon[29462]: 2.8 scrub starts 2024-09-11T07:28:49.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:48 smithi022 ceph-mon[29462]: 2.8 scrub ok 2024-09-11T07:28:49.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:48 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2834888159' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2024-09-11T07:28:49.001 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:48 smithi022 ceph-mon[37330]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:28:49.001 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:48 smithi022 ceph-mon[37330]: 2.8 scrub starts 2024-09-11T07:28:49.001 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:48 smithi022 ceph-mon[37330]: 2.8 scrub ok 2024-09-11T07:28:49.001 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:48 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2834888159' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2024-09-11T07:28:49.020 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:28:49.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:49.045 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:49.046 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:28:49.133 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-11T07:28:49.146 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:28:49.170 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:49.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:49.170 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:28:49.271 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:28:49.279 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:28:49 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:28:49.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:48 smithi104 ceph-mon[33507]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:28:49.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:48 smithi104 ceph-mon[33507]: 2.8 scrub starts 2024-09-11T07:28:49.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:48 smithi104 ceph-mon[33507]: 2.8 scrub ok 2024-09-11T07:28:49.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:48 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2834888159' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2024-09-11T07:28:49.296 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:49.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:49.297 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:28:49.398 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:28:49.423 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:49.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:49.424 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:28:49.524 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:28:49.548 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:49.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:49.549 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:28:49.649 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:28:49.674 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:49.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:49.675 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:28:49.679 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-09-11T07:28:49.776 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:28:49.798 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:49.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:49.799 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:28:49.900 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:28:49.925 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:49.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:49.926 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:28:50.027 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:28:50.050 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:50.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:50.051 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:28:50.152 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:28:50.174 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:50.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:50.175 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:28:50.276 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:28:50.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:49 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2834888159' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-09-11T07:28:50.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:49 smithi104 ceph-mon[33507]: osdmap e92: 8 total, 8 up, 8 in 2024-09-11T07:28:50.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:49 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:28:50.301 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:50.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:50.302 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:28:50.303 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:49 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2834888159' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-09-11T07:28:50.303 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:49 smithi022 ceph-mon[37330]: osdmap e92: 8 total, 8 up, 8 in 2024-09-11T07:28:50.303 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:49 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:28:50.303 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:49 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2834888159' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-09-11T07:28:50.303 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:49 smithi022 ceph-mon[29462]: osdmap e92: 8 total, 8 up, 8 in 2024-09-11T07:28:50.303 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:49 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:28:50.316 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:28:50.316 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:28:50.332 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:28:50.332 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:28:50.347 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:28:50.347 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:28:50.365 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:28:50.365 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:28:50.382 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:28:50.382 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:28:50.397 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:28:50.398 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:28:50.403 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:28:50.413 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:28:50.414 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:28:50.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:50.428 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:50.428 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:28:50.429 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:28:50.429 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:28:50.529 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:28:50.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:50.552 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:50.552 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:28:50.571 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:28:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:28:50] "GET /metrics HTTP/1.1" 200 220053 "" "Prometheus/2.43.0" 2024-09-11T07:28:50.653 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:28:50.676 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:50.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:50.677 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:28:50.778 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:28:50.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:50.801 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:50.801 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:28:50.902 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:28:50.928 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:50.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:50.929 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:28:51.000 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is set 2024-09-11T07:28:51.030 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:28:51.056 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:51.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:51.057 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:28:51.158 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:28:51.182 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:51.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:51.183 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:28:51.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:50 smithi104 ceph-mon[33507]: pgmap v74: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:28:51.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:50 smithi104 ceph-mon[33507]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:28:51.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:50 smithi104 ceph-mon[33507]: osdmap e93: 8 total, 8 up, 8 in 2024-09-11T07:28:51.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:50 smithi104 ceph-mon[33507]: 5.4 scrub starts 2024-09-11T07:28:51.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:50 smithi104 ceph-mon[33507]: 5.4 scrub ok 2024-09-11T07:28:51.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:50 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1686669226' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:28:51.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:50 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4059586775' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:28:51.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:50 smithi104 ceph-mon[33507]: from='client.24734 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:28:51.285 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:28:51.308 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:51.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:51.309 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:28:51.317 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:50 smithi022 ceph-mon[29462]: pgmap v74: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:28:51.317 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:50 smithi022 ceph-mon[29462]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:28:51.317 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:50 smithi022 ceph-mon[29462]: osdmap e93: 8 total, 8 up, 8 in 2024-09-11T07:28:51.317 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:50 smithi022 ceph-mon[29462]: 5.4 scrub starts 2024-09-11T07:28:51.317 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:50 smithi022 ceph-mon[29462]: 5.4 scrub ok 2024-09-11T07:28:51.317 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:50 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1686669226' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:28:51.317 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:50 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4059586775' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:28:51.317 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:50 smithi022 ceph-mon[29462]: from='client.24734 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:28:51.318 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:50 smithi022 ceph-mon[37330]: pgmap v74: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:28:51.318 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:50 smithi022 ceph-mon[37330]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:28:51.318 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:50 smithi022 ceph-mon[37330]: osdmap e93: 8 total, 8 up, 8 in 2024-09-11T07:28:51.318 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:50 smithi022 ceph-mon[37330]: 5.4 scrub starts 2024-09-11T07:28:51.318 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:50 smithi022 ceph-mon[37330]: 5.4 scrub ok 2024-09-11T07:28:51.318 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:50 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1686669226' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:28:51.318 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:50 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4059586775' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:28:51.318 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:50 smithi022 ceph-mon[37330]: from='client.24734 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:28:51.410 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:28:51.437 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:51.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:51.438 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:28:51.538 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:28:51.564 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:51.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:51.565 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:28:51.666 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:28:51.689 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:51.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:51.690 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:28:51.791 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:28:51.817 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:51.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:51.818 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:28:51.919 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:28:51.944 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:51.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:51.945 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:28:51.957 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:28:52.000 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:28:52.045 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:28:52.071 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:52.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:52.072 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:28:52.173 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:28:52.198 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:52.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:52.199 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:28:52.226 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:51 smithi022 ceph-mon[29462]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:28:52.226 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:51 smithi022 ceph-mon[29462]: from='client.24734 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:28:52.226 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:51 smithi022 ceph-mon[29462]: osdmap e94: 8 total, 8 up, 8 in 2024-09-11T07:28:52.226 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:51 smithi022 ceph-mon[29462]: 4.6 deep-scrub starts 2024-09-11T07:28:52.226 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:51 smithi022 ceph-mon[29462]: 4.6 deep-scrub ok 2024-09-11T07:28:52.226 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:51 smithi022 ceph-mon[37330]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:28:52.226 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:51 smithi022 ceph-mon[37330]: from='client.24734 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:28:52.227 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:51 smithi022 ceph-mon[37330]: osdmap e94: 8 total, 8 up, 8 in 2024-09-11T07:28:52.227 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:51 smithi022 ceph-mon[37330]: 4.6 deep-scrub starts 2024-09-11T07:28:52.227 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:51 smithi022 ceph-mon[37330]: 4.6 deep-scrub ok 2024-09-11T07:28:52.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:51 smithi104 ceph-mon[33507]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:28:52.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:51 smithi104 ceph-mon[33507]: from='client.24734 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:28:52.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:51 smithi104 ceph-mon[33507]: osdmap e94: 8 total, 8 up, 8 in 2024-09-11T07:28:52.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:51 smithi104 ceph-mon[33507]: 4.6 deep-scrub starts 2024-09-11T07:28:52.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:51 smithi104 ceph-mon[33507]: 4.6 deep-scrub ok 2024-09-11T07:28:52.300 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:28:52.323 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:52.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:52.324 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:28:52.424 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:28:52.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:52.449 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:52.449 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:28:52.550 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:28:52.574 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:52.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:52.575 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:28:52.676 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:28:52.698 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:52.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:52.700 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:28:52.800 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:28:52.814 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:28:52.822 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:52.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:52.823 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:28:52.924 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:28:52.947 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:52.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:52.948 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:28:53.049 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:28:53.072 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:53.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:53.073 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:28:53.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:52 smithi022 ceph-mon[29462]: pgmap v77: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T07:28:53.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:52 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:53.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:52 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:28:53.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:52 smithi022 ceph-mon[37330]: pgmap v77: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T07:28:53.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:52 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:53.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:52 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:28:53.174 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:28:53.195 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:53.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:53.196 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:28:53.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:53 smithi104 ceph-mon[33507]: pgmap v77: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T07:28:53.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:28:53.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:28:53.297 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:28:53.317 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:53.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:53.318 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:28:53.342 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:28:53.419 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:28:53.441 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:53.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:53.442 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:28:53.542 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:28:53.563 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:53.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:53.564 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:28:53.665 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:28:53.686 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:53.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:53.687 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:28:53.788 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:28:53.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:53.810 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:53.810 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:28:53.911 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:28:53.926 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T07:28:53.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:53.934 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:53.934 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:28:54.034 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:28:54.057 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:54.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:54.058 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:28:54.158 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:28:54.180 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:54.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:54.181 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:28:54.282 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:28:54.304 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:54.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:54.305 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:28:54.405 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:28:54.428 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:54.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:54.429 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:28:54.530 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:28:54.531 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:28:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:28:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:28:54.552 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:54.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:54.553 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:28:54.654 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:28:54.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:54.676 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:54.677 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:28:54.777 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:28:54.799 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:54.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:54.800 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:28:54.901 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:28:54.923 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:54.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:54.924 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:28:55.025 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:28:55.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:55.051 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:55.051 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:28:55.152 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:28:55.174 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:55.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:55.175 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:28:55.228 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:55 smithi022 ceph-mon[29462]: pgmap v78: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T07:28:55.228 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:55 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:28:55.228 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:55 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:28:55.228 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:55 smithi022 ceph-mon[37330]: pgmap v78: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T07:28:55.228 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:55 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:28:55.228 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:55 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:28:55.276 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:28:55.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:55 smithi104 ceph-mon[33507]: pgmap v78: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T07:28:55.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:55 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:28:55.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:55 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:28:55.302 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:55.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:55.303 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:28:55.404 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:28:55.429 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:55.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:55.430 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:28:55.502 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:28:55.504 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:28:55.531 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:28:55.555 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:55.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:55.556 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:28:55.656 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:28:55.679 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:55.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:55.680 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:28:55.781 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:28:55.795 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:28:55.795 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:28:55.805 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:55.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:55.806 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:28:55.906 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:28:55.932 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:55.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:55.933 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:28:56.034 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:28:56.058 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:56.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:56.059 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:28:56.159 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:28:56.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:56 smithi022 ceph-mon[29462]: osdmap e95: 8 total, 8 up, 8 in 2024-09-11T07:28:56.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:56 smithi022 ceph-mon[37330]: osdmap e95: 8 total, 8 up, 8 in 2024-09-11T07:28:56.183 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:56.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:56.184 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:28:56.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:56 smithi104 ceph-mon[33507]: osdmap e95: 8 total, 8 up, 8 in 2024-09-11T07:28:56.285 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:28:56.308 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:56.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:56.309 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:28:56.402 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:28:56.409 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:28:56.436 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:56.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:56.437 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:28:56.538 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:28:56.562 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:56.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:56.563 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:28:56.663 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:28:56.687 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:56.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:56.688 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:28:56.789 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:28:56.815 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:56.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:56.816 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:28:56.917 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:28:56.943 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:56.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:56.944 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:28:57.044 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:28:57.052 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-11T07:28:57.070 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:57.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:57.071 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:28:57.172 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:28:57.196 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:57.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:57.197 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:28:57.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:57 smithi104 ceph-mon[33507]: from='client.24740 v1:172.21.15.22:0/3790172509' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:28:57.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:57 smithi104 ceph-mon[33507]: pgmap v80: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:28:57.298 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:28:57.320 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:57.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:57.321 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:28:57.344 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:57 smithi022 ceph-mon[29462]: from='client.24740 v1:172.21.15.22:0/3790172509' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:28:57.344 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:57 smithi022 ceph-mon[29462]: pgmap v80: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:28:57.344 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:57 smithi022 ceph-mon[37330]: from='client.24740 v1:172.21.15.22:0/3790172509' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:28:57.345 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:57 smithi022 ceph-mon[37330]: pgmap v80: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:28:57.422 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:28:57.444 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:57.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:57.445 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:28:57.546 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:28:57.568 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:57.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:57.569 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:28:57.668 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2024-09-11T07:28:55.005155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":989,"num_read_kb":846,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":590588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":605,"ondisk_log_size":605,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2386444,"kb_used_data":7564,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747574772,"statfs":{"total":767960285184,"available":765516566528,"internally_reserved":0,"allocated":7745536,"data_stored":4591193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.025855"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.996643+0000","last_change":"2024-09-11T07:28:32.778166+0000","last_active":"2024-09-11T07:28:49.996643+0000","last_peered":"2024-09-11T07:28:49.996643+0000","last_clean":"2024-09-11T07:28:49.996643+0000","last_became_active":"2024-09-11T07:26:37.863165+0000","last_became_peered":"2024-09-11T07:26:37.863165+0000","last_unstale":"2024-09-11T07:28:49.996643+0000","last_undegraded":"2024-09-11T07:28:49.996643+0000","last_fullsized":"2024-09-11T07:28:49.996643+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:32.778109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:32.778109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:47.483048+0000","scrub_duration":0.001472253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.997186+0000","last_change":"2024-09-11T07:27:51.197966+0000","last_active":"2024-09-11T07:28:49.997186+0000","last_peered":"2024-09-11T07:28:49.997186+0000","last_clean":"2024-09-11T07:28:49.997186+0000","last_became_active":"2024-09-11T07:26:31.421005+0000","last_became_peered":"2024-09-11T07:26:31.421005+0000","last_unstale":"2024-09-11T07:28:49.997186+0000","last_undegraded":"2024-09-11T07:28:49.997186+0000","last_fullsized":"2024-09-11T07:28:49.997186+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:51.197903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:51.197903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:00.681709+0000","scrub_duration":0.001600113,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"82'22","reported_seq":86,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.998118+0000","last_change":"2024-09-11T07:27:50.756777+0000","last_active":"2024-09-11T07:28:49.998118+0000","last_peered":"2024-09-11T07:28:49.998118+0000","last_clean":"2024-09-11T07:28:49.998118+0000","last_became_active":"2024-09-11T07:26:33.544627+0000","last_became_peered":"2024-09-11T07:26:33.544627+0000","last_unstale":"2024-09-11T07:28:49.998118+0000","last_undegraded":"2024-09-11T07:28:49.998118+0000","last_fullsized":"2024-09-11T07:28:49.998118+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.756707+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.257406+0000","scrub_duration":0.0049180669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.992859+0000","last_change":"2024-09-11T07:28:17.046764+0000","last_active":"2024-09-11T07:28:49.992859+0000","last_peered":"2024-09-11T07:28:49.992859+0000","last_clean":"2024-09-11T07:28:49.992859+0000","last_became_active":"2024-09-11T07:26:35.556296+0000","last_became_peered":"2024-09-11T07:26:35.556296+0000","last_unstale":"2024-09-11T07:28:49.992859+0000","last_undegraded":"2024-09-11T07:28:49.992859+0000","last_fullsized":"2024-09-11T07:28:49.992859+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:17.046711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:17.046711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:21.028176+0000","scrub_duration":0.0011666510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.996709+0000","last_change":"2024-09-11T07:27:36.776948+0000","last_active":"2024-09-11T07:28:49.996709+0000","last_peered":"2024-09-11T07:28:49.996709+0000","last_clean":"2024-09-11T07:28:49.996709+0000","last_became_active":"2024-09-11T07:26:31.420178+0000","last_became_peered":"2024-09-11T07:26:31.420178+0000","last_unstale":"2024-09-11T07:28:49.996709+0000","last_undegraded":"2024-09-11T07:28:49.996709+0000","last_fullsized":"2024-09-11T07:28:49.996709+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:36.776883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:36.776883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017449290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"86'13","reported_seq":74,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.992816+0000","last_change":"2024-09-11T07:28:15.025864+0000","last_active":"2024-09-11T07:28:49.992816+0000","last_peered":"2024-09-11T07:28:49.992816+0000","last_clean":"2024-09-11T07:28:49.992816+0000","last_became_active":"2024-09-11T07:26:33.540732+0000","last_became_peered":"2024-09-11T07:26:33.540732+0000","last_unstale":"2024-09-11T07:28:49.992816+0000","last_undegraded":"2024-09-11T07:28:49.992816+0000","last_fullsized":"2024-09-11T07:28:49.992816+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'13","last_scrub_stamp":"2024-09-11T07:28:15.025757+0000","last_deep_scrub":"86'13","last_deep_scrub_stamp":"2024-09-11T07:28:15.025757+0000","last_clean_scrub_stamp":"2024-09-11T07:28:15.025757+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:26.818521+0000","scrub_duration":0.0039188890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.992296+0000","last_change":"2024-09-11T07:27:50.648473+0000","last_active":"2024-09-11T07:28:49.992296+0000","last_peered":"2024-09-11T07:28:49.992296+0000","last_clean":"2024-09-11T07:28:49.992296+0000","last_became_active":"2024-09-11T07:26:35.561096+0000","last_became_peered":"2024-09-11T07:26:35.561096+0000","last_unstale":"2024-09-11T07:28:49.992296+0000","last_undegraded":"2024-09-11T07:28:49.992296+0000","last_fullsized":"2024-09-11T07:28:49.992296+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.648420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:05.663592+0000","scrub_duration":0.001391446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.992923+0000","last_change":"2024-09-11T07:28:19.063739+0000","last_active":"2024-09-11T07:28:49.992923+0000","last_peered":"2024-09-11T07:28:49.992923+0000","last_clean":"2024-09-11T07:28:49.992923+0000","last_became_active":"2024-09-11T07:26:37.566910+0000","last_became_peered":"2024-09-11T07:26:37.566910+0000","last_unstale":"2024-09-11T07:28:49.992923+0000","last_undegraded":"2024-09-11T07:28:49.992923+0000","last_fullsized":"2024-09-11T07:28:49.992923+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:19.063651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:19.063651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:42.984286+0000","scrub_duration":0.0019504990000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.998079+0000","last_change":"2024-09-11T07:28:23.003084+0000","last_active":"2024-09-11T07:28:49.998079+0000","last_peered":"2024-09-11T07:28:49.998079+0000","last_clean":"2024-09-11T07:28:49.998079+0000","last_became_active":"2024-09-11T07:26:31.422772+0000","last_became_peered":"2024-09-11T07:26:31.422772+0000","last_unstale":"2024-09-11T07:28:49.998079+0000","last_undegraded":"2024-09-11T07:28:49.998079+0000","last_fullsized":"2024-09-11T07:28:49.998079+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:23.003028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:23.003028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.008955+0000","scrub_duration":0.0013267520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"82'7","reported_seq":66,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993343+0000","last_change":"2024-09-11T07:27:52.097170+0000","last_active":"2024-09-11T07:28:49.993343+0000","last_peered":"2024-09-11T07:28:49.993343+0000","last_clean":"2024-09-11T07:28:49.993343+0000","last_became_active":"2024-09-11T07:26:33.540351+0000","last_became_peered":"2024-09-11T07:26:33.540351+0000","last_unstale":"2024-09-11T07:28:49.993343+0000","last_undegraded":"2024-09-11T07:28:49.993343+0000","last_fullsized":"2024-09-11T07:28:49.993343+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'7","last_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_deep_scrub":"82'7","last_deep_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_clean_scrub_stamp":"2024-09-11T07:27:52.097071+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:53.905741+0000","scrub_duration":0.0025786670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.996640+0000","last_change":"2024-09-11T07:28:30.973423+0000","last_active":"2024-09-11T07:28:49.996640+0000","last_peered":"2024-09-11T07:28:49.996640+0000","last_clean":"2024-09-11T07:28:49.996640+0000","last_became_active":"2024-09-11T07:26:35.563536+0000","last_became_peered":"2024-09-11T07:26:35.563536+0000","last_unstale":"2024-09-11T07:28:49.996640+0000","last_undegraded":"2024-09-11T07:28:49.996640+0000","last_fullsized":"2024-09-11T07:28:49.996640+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:30.973323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.973323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:30.973323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:40.899348+0000","scrub_duration":0.0014944509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.998138+0000","last_change":"2024-09-11T07:28:24.869360+0000","last_active":"2024-09-11T07:28:49.998138+0000","last_peered":"2024-09-11T07:28:49.998138+0000","last_clean":"2024-09-11T07:28:49.998138+0000","last_became_active":"2024-09-11T07:26:37.561758+0000","last_became_peered":"2024-09-11T07:26:37.561758+0000","last_unstale":"2024-09-11T07:28:49.998138+0000","last_undegraded":"2024-09-11T07:28:49.998138+0000","last_fullsized":"2024-09-11T07:28:49.998138+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:24.869307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:24.869307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:43.695642+0000","scrub_duration":0.001382999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.998036+0000","last_change":"2024-09-11T07:27:52.058013+0000","last_active":"2024-09-11T07:28:49.998036+0000","last_peered":"2024-09-11T07:28:49.998036+0000","last_clean":"2024-09-11T07:28:49.998036+0000","last_became_active":"2024-09-11T07:26:31.420969+0000","last_became_peered":"2024-09-11T07:26:31.420969+0000","last_unstale":"2024-09-11T07:28:49.998036+0000","last_undegraded":"2024-09-11T07:28:49.998036+0000","last_fullsized":"2024-09-11T07:28:49.998036+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_clean_scrub_stamp":"2024-09-11T07:27:52.057959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:12.902027+0000","scrub_duration":0.0014644840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"64'9","reported_seq":70,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993264+0000","last_change":"2024-09-11T07:27:47.108704+0000","last_active":"2024-09-11T07:28:49.993264+0000","last_peered":"2024-09-11T07:28:49.993264+0000","last_clean":"2024-09-11T07:28:49.993264+0000","last_became_active":"2024-09-11T07:26:33.544386+0000","last_became_peered":"2024-09-11T07:26:33.544386+0000","last_unstale":"2024-09-11T07:28:49.993264+0000","last_undegraded":"2024-09-11T07:28:49.993264+0000","last_fullsized":"2024-09-11T07:28:49.993264+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:27:47.108645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.108645+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:08.302050+0000","scrub_duration":0.0024650560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.999165+0000","last_change":"2024-09-11T07:27:57.092029+0000","last_active":"2024-09-11T07:28:49.999165+0000","last_peered":"2024-09-11T07:28:49.999165+0000","last_clean":"2024-09-11T07:28:49.999165+0000","last_became_active":"2024-09-11T07:26:35.569630+0000","last_became_peered":"2024-09-11T07:26:35.569630+0000","last_unstale":"2024-09-11T07:28:49.999165+0000","last_undegraded":"2024-09-11T07:28:49.999165+0000","last_fullsized":"2024-09-11T07:28:49.999165+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:57.091958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.091958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:57.303120+0000","scrub_duration":0.001785872,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.995994+0000","last_change":"2024-09-11T07:28:09.333866+0000","last_active":"2024-09-11T07:28:49.995994+0000","last_peered":"2024-09-11T07:28:49.995994+0000","last_clean":"2024-09-11T07:28:49.995994+0000","last_became_active":"2024-09-11T07:26:37.565475+0000","last_became_peered":"2024-09-11T07:26:37.565475+0000","last_unstale":"2024-09-11T07:28:49.995994+0000","last_undegraded":"2024-09-11T07:28:49.995994+0000","last_fullsized":"2024-09-11T07:28:49.995994+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:09.333804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:09.333804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:11.385070+0000","scrub_duration":0.003498438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.003073+0000","last_change":"2024-09-11T07:28:50.002862+0000","last_active":"2024-09-11T07:28:50.003073+0000","last_peered":"2024-09-11T07:28:50.003073+0000","last_clean":"2024-09-11T07:28:50.003073+0000","last_became_active":"2024-09-11T07:28:50.002582+0000","last_became_peered":"2024-09-11T07:28:50.002582+0000","last_unstale":"2024-09-11T07:28:50.003073+0000","last_undegraded":"2024-09-11T07:28:50.003073+0000","last_fullsized":"2024-09-11T07:28:50.003073+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:30:12.931996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":110,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993781+0000","last_change":"2024-09-11T07:28:09.027694+0000","last_active":"2024-09-11T07:28:49.993781+0000","last_peered":"2024-09-11T07:28:49.993781+0000","last_clean":"2024-09-11T07:28:49.993781+0000","last_became_active":"2024-09-11T07:28:09.018213+0000","last_became_peered":"2024-09-11T07:28:09.018213+0000","last_unstale":"2024-09-11T07:28:49.993781+0000","last_undegraded":"2024-09-11T07:28:49.993781+0000","last_fullsized":"2024-09-11T07:28:49.993781+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-09-11T07:28:04.137453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:04.137453+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:05.572290+0000","scrub_duration":0.001914083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.997341+0000","last_change":"2024-09-11T07:28:11.897385+0000","last_active":"2024-09-11T07:28:49.997341+0000","last_peered":"2024-09-11T07:28:49.997341+0000","last_clean":"2024-09-11T07:28:49.997341+0000","last_became_active":"2024-09-11T07:26:31.419488+0000","last_became_peered":"2024-09-11T07:26:31.419488+0000","last_unstale":"2024-09-11T07:28:49.997341+0000","last_undegraded":"2024-09-11T07:28:49.997341+0000","last_fullsized":"2024-09-11T07:28:49.997341+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:11.897319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:11.897319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:18.864403+0000","scrub_duration":0.0013939480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.992989+0000","last_change":"2024-09-11T07:28:26.841381+0000","last_active":"2024-09-11T07:28:49.992989+0000","last_peered":"2024-09-11T07:28:49.992989+0000","last_clean":"2024-09-11T07:28:49.992989+0000","last_became_active":"2024-09-11T07:26:35.557989+0000","last_became_peered":"2024-09-11T07:26:35.557989+0000","last_unstale":"2024-09-11T07:28:49.992989+0000","last_undegraded":"2024-09-11T07:28:49.992989+0000","last_fullsized":"2024-09-11T07:28:49.992989+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:26.841326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:26.841326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:42.433747+0000","scrub_duration":0.0015001350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993456+0000","last_change":"2024-09-11T07:27:57.274611+0000","last_active":"2024-09-11T07:28:49.993456+0000","last_peered":"2024-09-11T07:28:49.993456+0000","last_clean":"2024-09-11T07:28:49.993456+0000","last_became_active":"2024-09-11T07:26:37.564554+0000","last_became_peered":"2024-09-11T07:26:37.564554+0000","last_unstale":"2024-09-11T07:28:49.993456+0000","last_undegraded":"2024-09-11T07:28:49.993456+0000","last_fullsized":"2024-09-11T07:28:49.993456+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:57.274554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.274554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.058396+0000","scrub_duration":0.001419584,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.001107+0000","last_change":"2024-09-11T07:28:50.000249+0000","last_active":"2024-09-11T07:28:50.001107+0000","last_peered":"2024-09-11T07:28:50.001107+0000","last_clean":"2024-09-11T07:28:50.001107+0000","last_became_active":"2024-09-11T07:28:49.999894+0000","last_became_peered":"2024-09-11T07:28:49.999894+0000","last_unstale":"2024-09-11T07:28:50.001107+0000","last_undegraded":"2024-09-11T07:28:50.001107+0000","last_fullsized":"2024-09-11T07:28:50.001107+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:51.279633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":69,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993520+0000","last_change":"2024-09-11T07:28:21.039929+0000","last_active":"2024-09-11T07:28:49.993520+0000","last_peered":"2024-09-11T07:28:49.993520+0000","last_clean":"2024-09-11T07:28:49.993520+0000","last_became_active":"2024-09-11T07:26:33.535411+0000","last_became_peered":"2024-09-11T07:26:33.535411+0000","last_unstale":"2024-09-11T07:28:49.993520+0000","last_undegraded":"2024-09-11T07:28:49.993520+0000","last_fullsized":"2024-09-11T07:28:49.993520+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:28:21.039828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:21.039828+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:24.128795+0000","scrub_duration":0.0021618829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.992826+0000","last_change":"2024-09-11T07:28:18.878004+0000","last_active":"2024-09-11T07:28:49.992826+0000","last_peered":"2024-09-11T07:28:49.992826+0000","last_clean":"2024-09-11T07:28:49.992826+0000","last_became_active":"2024-09-11T07:26:31.418178+0000","last_became_peered":"2024-09-11T07:26:31.418178+0000","last_unstale":"2024-09-11T07:28:49.992826+0000","last_undegraded":"2024-09-11T07:28:49.992826+0000","last_fullsized":"2024-09-11T07:28:49.992826+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:18.877940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:18.877940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:21.852832+0000","scrub_duration":0.001611924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"64'8","reported_seq":88,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993314+0000","last_change":"2024-09-11T07:27:47.127485+0000","last_active":"2024-09-11T07:28:49.993314+0000","last_peered":"2024-09-11T07:28:49.993314+0000","last_clean":"2024-09-11T07:28:49.993314+0000","last_became_active":"2024-09-11T07:26:35.564531+0000","last_became_peered":"2024-09-11T07:26:35.564531+0000","last_unstale":"2024-09-11T07:28:49.993314+0000","last_undegraded":"2024-09-11T07:28:49.993314+0000","last_fullsized":"2024-09-11T07:28:49.993314+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:27:47.127428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.127428+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:58.343731+0000","scrub_duration":0.0014091329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993735+0000","last_change":"2024-09-11T07:28:31.007700+0000","last_active":"2024-09-11T07:28:49.993735+0000","last_peered":"2024-09-11T07:28:49.993735+0000","last_clean":"2024-09-11T07:28:49.993735+0000","last_became_active":"2024-09-11T07:26:37.566919+0000","last_became_peered":"2024-09-11T07:26:37.566919+0000","last_unstale":"2024-09-11T07:28:49.993735+0000","last_undegraded":"2024-09-11T07:28:49.993735+0000","last_fullsized":"2024-09-11T07:28:49.993735+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:31.007651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:31.007651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:49.292980+0000","scrub_duration":0.001233963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.003588+0000","last_change":"2024-09-11T07:28:50.003247+0000","last_active":"2024-09-11T07:28:50.003588+0000","last_peered":"2024-09-11T07:28:50.003588+0000","last_clean":"2024-09-11T07:28:50.003588+0000","last_became_active":"2024-09-11T07:28:50.002911+0000","last_became_peered":"2024-09-11T07:28:50.002911+0000","last_unstale":"2024-09-11T07:28:50.003588+0000","last_undegraded":"2024-09-11T07:28:50.003588+0000","last_fullsized":"2024-09-11T07:28:50.003588+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:52.607485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"83'22","reported_seq":86,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993660+0000","last_change":"2024-09-11T07:28:01.719185+0000","last_active":"2024-09-11T07:28:49.993660+0000","last_peered":"2024-09-11T07:28:49.993660+0000","last_clean":"2024-09-11T07:28:49.993660+0000","last_became_active":"2024-09-11T07:26:33.549733+0000","last_became_peered":"2024-09-11T07:26:33.549733+0000","last_unstale":"2024-09-11T07:28:49.993660+0000","last_undegraded":"2024-09-11T07:28:49.993660+0000","last_fullsized":"2024-09-11T07:28:49.993660+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'22","last_scrub_stamp":"2024-09-11T07:28:01.719128+0000","last_deep_scrub":"83'22","last_deep_scrub_stamp":"2024-09-11T07:28:01.719128+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.719128+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:27.431086+0000","scrub_duration":0.0047817240000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.992483+0000","last_change":"2024-09-11T07:27:35.589317+0000","last_active":"2024-09-11T07:28:49.992483+0000","last_peered":"2024-09-11T07:28:49.992483+0000","last_clean":"2024-09-11T07:28:49.992483+0000","last_became_active":"2024-09-11T07:26:31.418478+0000","last_became_peered":"2024-09-11T07:26:31.418478+0000","last_unstale":"2024-09-11T07:28:49.992483+0000","last_undegraded":"2024-09-11T07:28:49.992483+0000","last_fullsized":"2024-09-11T07:28:49.992483+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:35.589251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:35.589251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:01.839680+0000","scrub_duration":0.0014929380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":91,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993774+0000","last_change":"2024-09-11T07:27:56.218355+0000","last_active":"2024-09-11T07:28:49.993774+0000","last_peered":"2024-09-11T07:28:49.993774+0000","last_clean":"2024-09-11T07:28:49.993774+0000","last_became_active":"2024-09-11T07:26:35.560406+0000","last_became_peered":"2024-09-11T07:26:35.560406+0000","last_unstale":"2024-09-11T07:28:49.993774+0000","last_undegraded":"2024-09-11T07:28:49.993774+0000","last_fullsized":"2024-09-11T07:28:49.993774+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:27:56.218301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:56.218301+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:21.374837+0000","scrub_duration":0.001687413,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.999408+0000","last_change":"2024-09-11T07:28:18.785729+0000","last_active":"2024-09-11T07:28:49.999408+0000","last_peered":"2024-09-11T07:28:49.999408+0000","last_clean":"2024-09-11T07:28:49.999408+0000","last_became_active":"2024-09-11T07:26:37.568005+0000","last_became_peered":"2024-09-11T07:26:37.568005+0000","last_unstale":"2024-09-11T07:28:49.999408+0000","last_undegraded":"2024-09-11T07:28:49.999408+0000","last_fullsized":"2024-09-11T07:28:49.999408+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:18.785667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:18.785667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:39.054254+0000","scrub_duration":0.0013564829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.005403+0000","last_change":"2024-09-11T07:28:50.004962+0000","last_active":"2024-09-11T07:28:50.005403+0000","last_peered":"2024-09-11T07:28:50.005403+0000","last_clean":"2024-09-11T07:28:50.005403+0000","last_became_active":"2024-09-11T07:28:50.004515+0000","last_became_peered":"2024-09-11T07:28:50.004515+0000","last_unstale":"2024-09-11T07:28:50.005403+0000","last_undegraded":"2024-09-11T07:28:50.005403+0000","last_fullsized":"2024-09-11T07:28:50.005403+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:50.097337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"64'11","reported_seq":73,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993849+0000","last_change":"2024-09-11T07:27:58.158485+0000","last_active":"2024-09-11T07:28:49.993849+0000","last_peered":"2024-09-11T07:28:49.993849+0000","last_clean":"2024-09-11T07:28:49.993849+0000","last_became_active":"2024-09-11T07:26:33.540777+0000","last_became_peered":"2024-09-11T07:26:33.540777+0000","last_unstale":"2024-09-11T07:28:49.993849+0000","last_undegraded":"2024-09-11T07:28:49.993849+0000","last_fullsized":"2024-09-11T07:28:49.993849+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:27:58.158420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.158420+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.832014+0000","scrub_duration":0.0021737459999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":66,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.997236+0000","last_change":"2024-09-11T07:28:48.208248+0000","last_active":"2024-09-11T07:28:49.997236+0000","last_peered":"2024-09-11T07:28:49.997236+0000","last_clean":"2024-09-11T07:28:49.997236+0000","last_became_active":"2024-09-11T07:26:31.418238+0000","last_became_peered":"2024-09-11T07:26:31.418238+0000","last_unstale":"2024-09-11T07:28:49.997236+0000","last_undegraded":"2024-09-11T07:28:49.997236+0000","last_fullsized":"2024-09-11T07:28:49.997236+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.208184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.208184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:30:00.127894+0000","scrub_duration":0.001606994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.002031+0000","last_change":"2024-09-11T07:28:00.005119+0000","last_active":"2024-09-11T07:28:50.002031+0000","last_peered":"2024-09-11T07:28:50.002031+0000","last_clean":"2024-09-11T07:28:50.002031+0000","last_became_active":"2024-09-11T07:26:35.569081+0000","last_became_peered":"2024-09-11T07:26:35.569081+0000","last_unstale":"2024-09-11T07:28:50.002031+0000","last_undegraded":"2024-09-11T07:28:50.002031+0000","last_fullsized":"2024-09-11T07:28:50.002031+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:00.005031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.005031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:10.748420+0000","scrub_duration":0.001403533,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.995962+0000","last_change":"2024-09-11T07:27:46.752272+0000","last_active":"2024-09-11T07:28:49.995962+0000","last_peered":"2024-09-11T07:28:49.995962+0000","last_clean":"2024-09-11T07:28:49.995962+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:28:49.995962+0000","last_undegraded":"2024-09-11T07:28:49.995962+0000","last_fullsized":"2024-09-11T07:28:49.995962+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:46.752181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:46.752181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.072860+0000","scrub_duration":0.001508782,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.002750+0000","last_change":"2024-09-11T07:28:50.000121+0000","last_active":"2024-09-11T07:28:50.002750+0000","last_peered":"2024-09-11T07:28:50.002750+0000","last_clean":"2024-09-11T07:28:50.002750+0000","last_became_active":"2024-09-11T07:28:49.999841+0000","last_became_peered":"2024-09-11T07:28:49.999841+0000","last_unstale":"2024-09-11T07:28:50.002750+0000","last_undegraded":"2024-09-11T07:28:50.002750+0000","last_fullsized":"2024-09-11T07:28:50.002750+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:30:12.057468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"64'14","reported_seq":75,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993049+0000","last_change":"2024-09-11T07:28:20.085403+0000","last_active":"2024-09-11T07:28:49.993049+0000","last_peered":"2024-09-11T07:28:49.993049+0000","last_clean":"2024-09-11T07:28:49.993049+0000","last_became_active":"2024-09-11T07:26:33.537850+0000","last_became_peered":"2024-09-11T07:26:33.537850+0000","last_unstale":"2024-09-11T07:28:49.993049+0000","last_undegraded":"2024-09-11T07:28:49.993049+0000","last_fullsized":"2024-09-11T07:28:49.993049+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-09-11T07:28:20.085339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:20.085339+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:27.624554+0000","scrub_duration":0.0019854640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":66,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.002792+0000","last_change":"2024-09-11T07:28:09.015415+0000","last_active":"2024-09-11T07:28:50.002792+0000","last_peered":"2024-09-11T07:28:50.002792+0000","last_clean":"2024-09-11T07:28:50.002792+0000","last_became_active":"2024-09-11T07:28:09.014995+0000","last_became_peered":"2024-09-11T07:28:09.014995+0000","last_unstale":"2024-09-11T07:28:50.002792+0000","last_undegraded":"2024-09-11T07:28:50.002792+0000","last_fullsized":"2024-09-11T07:28:50.002792+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:39.014840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:39.014840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:56.130768+0000","scrub_duration":0.0032789239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"84'9","reported_seq":89,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.996013+0000","last_change":"2024-09-11T07:28:05.653289+0000","last_active":"2024-09-11T07:28:49.996013+0000","last_peered":"2024-09-11T07:28:49.996013+0000","last_clean":"2024-09-11T07:28:49.996013+0000","last_became_active":"2024-09-11T07:26:35.562997+0000","last_became_peered":"2024-09-11T07:26:35.562997+0000","last_unstale":"2024-09-11T07:28:49.996013+0000","last_undegraded":"2024-09-11T07:28:49.996013+0000","last_fullsized":"2024-09-11T07:28:49.996013+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-11T07:28:05.653237+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-11T07:28:05.653237+0000","last_clean_scrub_stamp":"2024-09-11T07:28:05.653237+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:22.713095+0000","scrub_duration":0.0027957030000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.996329+0000","last_change":"2024-09-11T07:28:34.041835+0000","last_active":"2024-09-11T07:28:49.996329+0000","last_peered":"2024-09-11T07:28:49.996329+0000","last_clean":"2024-09-11T07:28:49.996329+0000","last_became_active":"2024-09-11T07:26:37.862246+0000","last_became_peered":"2024-09-11T07:26:37.862246+0000","last_unstale":"2024-09-11T07:28:49.996329+0000","last_undegraded":"2024-09-11T07:28:49.996329+0000","last_fullsized":"2024-09-11T07:28:49.996329+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:34.041748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:34.041748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:39.192356+0000","scrub_duration":0.0015228100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.002696+0000","last_change":"2024-09-11T07:28:50.000140+0000","last_active":"2024-09-11T07:28:50.002696+0000","last_peered":"2024-09-11T07:28:50.002696+0000","last_clean":"2024-09-11T07:28:50.002696+0000","last_became_active":"2024-09-11T07:28:49.999860+0000","last_became_peered":"2024-09-11T07:28:49.999860+0000","last_unstale":"2024-09-11T07:28:50.002696+0000","last_undegraded":"2024-09-11T07:28:50.002696+0000","last_fullsized":"2024-09-11T07:28:50.002696+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:30:17.881059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"87'26","reported_seq":94,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.996182+0000","last_change":"2024-09-11T07:28:30.788748+0000","last_active":"2024-09-11T07:28:49.996182+0000","last_peered":"2024-09-11T07:28:49.996182+0000","last_clean":"2024-09-11T07:28:49.996182+0000","last_became_active":"2024-09-11T07:26:33.546188+0000","last_became_peered":"2024-09-11T07:26:33.546188+0000","last_unstale":"2024-09-11T07:28:49.996182+0000","last_undegraded":"2024-09-11T07:28:49.996182+0000","last_fullsized":"2024-09-11T07:28:49.996182+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'26","last_scrub_stamp":"2024-09-11T07:28:30.788647+0000","last_deep_scrub":"87'26","last_deep_scrub_stamp":"2024-09-11T07:28:30.788647+0000","last_clean_scrub_stamp":"2024-09-11T07:28:30.788647+0000","objects_scrubbed":8,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:35.415117+0000","scrub_duration":0.0060027819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993443+0000","last_change":"2024-09-11T07:28:32.839800+0000","last_active":"2024-09-11T07:28:49.993443+0000","last_peered":"2024-09-11T07:28:49.993443+0000","last_clean":"2024-09-11T07:28:49.993443+0000","last_became_active":"2024-09-11T07:26:31.421500+0000","last_became_peered":"2024-09-11T07:26:31.421500+0000","last_unstale":"2024-09-11T07:28:49.993443+0000","last_undegraded":"2024-09-11T07:28:49.993443+0000","last_fullsized":"2024-09-11T07:28:49.993443+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:32.839737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:32.839737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:35.859113+0000","scrub_duration":0.0014376460000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.992915+0000","last_change":"2024-09-11T07:28:27.968629+0000","last_active":"2024-09-11T07:28:49.992915+0000","last_peered":"2024-09-11T07:28:49.992915+0000","last_clean":"2024-09-11T07:28:49.992915+0000","last_became_active":"2024-09-11T07:26:35.561284+0000","last_became_peered":"2024-09-11T07:26:35.561284+0000","last_unstale":"2024-09-11T07:28:49.992915+0000","last_undegraded":"2024-09-11T07:28:49.992915+0000","last_fullsized":"2024-09-11T07:28:49.992915+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:27.968579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:27.968579+0000","last_clean_scrub_stamp":"2024-09-11T07:28:27.968579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:53.162017+0000","scrub_duration":0.0026385499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993157+0000","last_change":"2024-09-11T07:28:29.978861+0000","last_active":"2024-09-11T07:28:49.993157+0000","last_peered":"2024-09-11T07:28:49.993157+0000","last_clean":"2024-09-11T07:28:49.993157+0000","last_became_active":"2024-09-11T07:26:37.564616+0000","last_became_peered":"2024-09-11T07:26:37.564616+0000","last_unstale":"2024-09-11T07:28:49.993157+0000","last_undegraded":"2024-09-11T07:28:49.993157+0000","last_fullsized":"2024-09-11T07:28:49.993157+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:29.978798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:29.978798+0000","last_clean_scrub_stamp":"2024-09-11T07:28:29.978798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:48.118680+0000","scrub_duration":0.001285797,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.003504+0000","last_change":"2024-09-11T07:28:50.002675+0000","last_active":"2024-09-11T07:28:50.003504+0000","last_peered":"2024-09-11T07:28:50.003504+0000","last_clean":"2024-09-11T07:28:50.003504+0000","last_became_active":"2024-09-11T07:28:50.002344+0000","last_became_peered":"2024-09-11T07:28:50.002344+0000","last_unstale":"2024-09-11T07:28:50.003504+0000","last_undegraded":"2024-09-11T07:28:50.003504+0000","last_fullsized":"2024-09-11T07:28:50.003504+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:30:09.362930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":90,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.996594+0000","last_change":"2024-09-11T07:28:12.466246+0000","last_active":"2024-09-11T07:28:49.996594+0000","last_peered":"2024-09-11T07:28:49.996594+0000","last_clean":"2024-09-11T07:28:49.996594+0000","last_became_active":"2024-09-11T07:26:33.540639+0000","last_became_peered":"2024-09-11T07:26:33.540639+0000","last_unstale":"2024-09-11T07:28:49.996594+0000","last_undegraded":"2024-09-11T07:28:49.996594+0000","last_fullsized":"2024-09-11T07:28:49.996594+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-11T07:28:12.466179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:12.466179+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:12.523827+0000","scrub_duration":0.0027522670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.998151+0000","last_change":"2024-09-11T07:28:12.896603+0000","last_active":"2024-09-11T07:28:49.998151+0000","last_peered":"2024-09-11T07:28:49.998151+0000","last_clean":"2024-09-11T07:28:49.998151+0000","last_became_active":"2024-09-11T07:26:31.419185+0000","last_became_peered":"2024-09-11T07:26:31.419185+0000","last_unstale":"2024-09-11T07:28:49.998151+0000","last_undegraded":"2024-09-11T07:28:49.998151+0000","last_fullsized":"2024-09-11T07:28:49.998151+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:12.896538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:12.896538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.577764+0000","scrub_duration":0.0017083070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.997735+0000","last_change":"2024-09-11T07:27:41.000469+0000","last_active":"2024-09-11T07:28:49.997735+0000","last_peered":"2024-09-11T07:28:49.997735+0000","last_clean":"2024-09-11T07:28:49.997735+0000","last_became_active":"2024-09-11T07:26:35.565359+0000","last_became_peered":"2024-09-11T07:26:35.565359+0000","last_unstale":"2024-09-11T07:28:49.997735+0000","last_undegraded":"2024-09-11T07:28:49.997735+0000","last_fullsized":"2024-09-11T07:28:49.997735+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:41.000401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:41.000401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.049825+0000","scrub_duration":0.001655164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.992213+0000","last_change":"2024-09-11T07:28:39.723663+0000","last_active":"2024-09-11T07:28:49.992213+0000","last_peered":"2024-09-11T07:28:49.992213+0000","last_clean":"2024-09-11T07:28:49.992213+0000","last_became_active":"2024-09-11T07:26:37.861279+0000","last_became_peered":"2024-09-11T07:26:37.861279+0000","last_unstale":"2024-09-11T07:28:49.992213+0000","last_undegraded":"2024-09-11T07:28:49.992213+0000","last_fullsized":"2024-09-11T07:28:49.992213+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:39.723598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:39.723598+0000","last_clean_scrub_stamp":"2024-09-11T07:28:39.723598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:48.587552+0000","scrub_duration":0.001623202,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.001015+0000","last_change":"2024-09-11T07:28:50.000309+0000","last_active":"2024-09-11T07:28:50.001015+0000","last_peered":"2024-09-11T07:28:50.001015+0000","last_clean":"2024-09-11T07:28:50.001015+0000","last_became_active":"2024-09-11T07:28:49.999983+0000","last_became_peered":"2024-09-11T07:28:49.999983+0000","last_unstale":"2024-09-11T07:28:50.001015+0000","last_undegraded":"2024-09-11T07:28:50.001015+0000","last_fullsized":"2024-09-11T07:28:50.001015+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:30:14.061143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":69,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.992768+0000","last_change":"2024-09-11T07:27:51.642529+0000","last_active":"2024-09-11T07:28:49.992768+0000","last_peered":"2024-09-11T07:28:49.992768+0000","last_clean":"2024-09-11T07:28:49.992768+0000","last_became_active":"2024-09-11T07:26:33.543063+0000","last_became_peered":"2024-09-11T07:26:33.543063+0000","last_unstale":"2024-09-11T07:28:49.992768+0000","last_undegraded":"2024-09-11T07:28:49.992768+0000","last_fullsized":"2024-09-11T07:28:49.992768+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:27:51.642464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:51.642464+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:13.146284+0000","scrub_duration":0.0020211439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":63,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993246+0000","last_change":"2024-09-11T07:27:41.618362+0000","last_active":"2024-09-11T07:28:49.993246+0000","last_peered":"2024-09-11T07:28:49.993246+0000","last_clean":"2024-09-11T07:28:49.993246+0000","last_became_active":"2024-09-11T07:26:31.422110+0000","last_became_peered":"2024-09-11T07:26:31.422110+0000","last_unstale":"2024-09-11T07:28:49.993246+0000","last_undegraded":"2024-09-11T07:28:49.993246+0000","last_fullsized":"2024-09-11T07:28:49.993246+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:27:41.618296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:41.618296+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:57.404591+0000","scrub_duration":0.0020650450000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":88,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.996427+0000","last_change":"2024-09-11T07:28:03.183056+0000","last_active":"2024-09-11T07:28:49.996427+0000","last_peered":"2024-09-11T07:28:49.996427+0000","last_clean":"2024-09-11T07:28:49.996427+0000","last_became_active":"2024-09-11T07:26:35.560422+0000","last_became_peered":"2024-09-11T07:26:35.560422+0000","last_unstale":"2024-09-11T07:28:49.996427+0000","last_undegraded":"2024-09-11T07:28:49.996427+0000","last_fullsized":"2024-09-11T07:28:49.996427+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:28:03.182993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:03.182993+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:26.896461+0000","scrub_duration":0.0018949889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.995864+0000","last_change":"2024-09-11T07:27:54.148594+0000","last_active":"2024-09-11T07:28:49.995864+0000","last_peered":"2024-09-11T07:28:49.995864+0000","last_clean":"2024-09-11T07:28:49.995864+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:28:49.995864+0000","last_undegraded":"2024-09-11T07:28:49.995864+0000","last_fullsized":"2024-09-11T07:28:49.995864+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.148534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:02.854448+0000","scrub_duration":0.0013536620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.003883+0000","last_change":"2024-09-11T07:28:50.003505+0000","last_active":"2024-09-11T07:28:50.003883+0000","last_peered":"2024-09-11T07:28:50.003883+0000","last_clean":"2024-09-11T07:28:50.003883+0000","last_became_active":"2024-09-11T07:28:50.003113+0000","last_became_peered":"2024-09-11T07:28:50.003113+0000","last_unstale":"2024-09-11T07:28:50.003883+0000","last_undegraded":"2024-09-11T07:28:50.003883+0000","last_fullsized":"2024-09-11T07:28:50.003883+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:30:10.365940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"64'28","reported_seq":130,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993117+0000","last_change":"2024-09-11T07:28:09.770058+0000","last_active":"2024-09-11T07:28:49.993117+0000","last_peered":"2024-09-11T07:28:49.993117+0000","last_clean":"2024-09-11T07:28:49.993117+0000","last_became_active":"2024-09-11T07:26:33.549234+0000","last_became_peered":"2024-09-11T07:26:33.549234+0000","last_unstale":"2024-09-11T07:28:49.993117+0000","last_undegraded":"2024-09-11T07:28:49.993117+0000","last_fullsized":"2024-09-11T07:28:49.993117+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'28","last_scrub_stamp":"2024-09-11T07:28:09.769992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:09.769992+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:37.423717+0000","scrub_duration":0.002388463,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.998820+0000","last_change":"2024-09-11T07:28:12.775128+0000","last_active":"2024-09-11T07:28:49.998820+0000","last_peered":"2024-09-11T07:28:49.998820+0000","last_clean":"2024-09-11T07:28:49.998820+0000","last_became_active":"2024-09-11T07:26:31.424618+0000","last_became_peered":"2024-09-11T07:26:31.424618+0000","last_unstale":"2024-09-11T07:28:49.998820+0000","last_undegraded":"2024-09-11T07:28:49.998820+0000","last_fullsized":"2024-09-11T07:28:49.998820+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:12.775067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:12.775067+0000","last_clean_scrub_stamp":"2024-09-11T07:28:12.775067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:21.184139+0000","scrub_duration":0.0015206589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.996030+0000","last_change":"2024-09-11T07:28:15.439112+0000","last_active":"2024-09-11T07:28:49.996030+0000","last_peered":"2024-09-11T07:28:49.996030+0000","last_clean":"2024-09-11T07:28:49.996030+0000","last_became_active":"2024-09-11T07:26:35.560363+0000","last_became_peered":"2024-09-11T07:26:35.560363+0000","last_unstale":"2024-09-11T07:28:49.996030+0000","last_undegraded":"2024-09-11T07:28:49.996030+0000","last_fullsized":"2024-09-11T07:28:49.996030+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:15.439020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:15.439020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:29.665525+0000","scrub_duration":0.001542844,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.996525+0000","last_change":"2024-09-11T07:28:00.686634+0000","last_active":"2024-09-11T07:28:49.996525+0000","last_peered":"2024-09-11T07:28:49.996525+0000","last_clean":"2024-09-11T07:28:49.996525+0000","last_became_active":"2024-09-11T07:26:37.562452+0000","last_became_peered":"2024-09-11T07:26:37.562452+0000","last_unstale":"2024-09-11T07:28:49.996525+0000","last_undegraded":"2024-09-11T07:28:49.996525+0000","last_fullsized":"2024-09-11T07:28:49.996525+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:00.686571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.686571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.798861+0000","scrub_duration":0.0016714309999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.005648+0000","last_change":"2024-09-11T07:28:50.005359+0000","last_active":"2024-09-11T07:28:50.005648+0000","last_peered":"2024-09-11T07:28:50.005648+0000","last_clean":"2024-09-11T07:28:50.005648+0000","last_became_active":"2024-09-11T07:28:50.004676+0000","last_became_peered":"2024-09-11T07:28:50.004676+0000","last_unstale":"2024-09-11T07:28:50.005648+0000","last_undegraded":"2024-09-11T07:28:50.005648+0000","last_fullsized":"2024-09-11T07:28:50.005648+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:52.244321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"64'13","reported_seq":81,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993663+0000","last_change":"2024-09-11T07:28:36.723421+0000","last_active":"2024-09-11T07:28:49.993663+0000","last_peered":"2024-09-11T07:28:49.993663+0000","last_clean":"2024-09-11T07:28:49.993663+0000","last_became_active":"2024-09-11T07:26:33.546126+0000","last_became_peered":"2024-09-11T07:26:33.546126+0000","last_unstale":"2024-09-11T07:28:49.993663+0000","last_undegraded":"2024-09-11T07:28:49.993663+0000","last_fullsized":"2024-09-11T07:28:49.993663+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'13","last_scrub_stamp":"2024-09-11T07:28:36.723359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:36.723359+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:51.245537+0000","scrub_duration":0.0020419800000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":200,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-11T07:28:52.808252+0000","last_change":"2024-09-11T07:27:59.678930+0000","last_active":"2024-09-11T07:28:52.808252+0000","last_peered":"2024-09-11T07:28:52.808252+0000","last_clean":"2024-09-11T07:28:52.808252+0000","last_became_active":"2024-09-11T07:26:31.421276+0000","last_became_peered":"2024-09-11T07:26:31.421276+0000","last_unstale":"2024-09-11T07:28:52.808252+0000","last_undegraded":"2024-09-11T07:28:52.808252+0000","last_fullsized":"2024-09-11T07:28:52.808252+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'5","last_scrub_stamp":"2024-09-11T07:27:59.678865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:59.678865+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.114929+0000","scrub_duration":0.001792486,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":136,"num_read_kb":131,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.994470+0000","last_change":"2024-09-11T07:28:07.327716+0000","last_active":"2024-09-11T07:28:49.994470+0000","last_peered":"2024-09-11T07:28:49.994470+0000","last_clean":"2024-09-11T07:28:49.994470+0000","last_became_active":"2024-09-11T07:26:31.413354+0000","last_became_peered":"2024-09-11T07:26:31.413354+0000","last_unstale":"2024-09-11T07:28:49.994470+0000","last_undegraded":"2024-09-11T07:28:49.994470+0000","last_fullsized":"2024-09-11T07:28:49.994470+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:07.327648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:07.327648+0000","last_clean_scrub_stamp":"2024-09-11T07:28:07.327648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:35.306829+0000","scrub_duration":0.0011895899999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.994333+0000","last_change":"2024-09-11T07:27:42.141156+0000","last_active":"2024-09-11T07:28:49.994333+0000","last_peered":"2024-09-11T07:28:49.994333+0000","last_clean":"2024-09-11T07:28:49.994333+0000","last_became_active":"2024-09-11T07:26:35.565849+0000","last_became_peered":"2024-09-11T07:26:35.565849+0000","last_unstale":"2024-09-11T07:28:49.994333+0000","last_undegraded":"2024-09-11T07:28:49.994333+0000","last_fullsized":"2024-09-11T07:28:49.994333+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.141100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.141100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.707826+0000","scrub_duration":0.001495672,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.998361+0000","last_change":"2024-09-11T07:27:55.018180+0000","last_active":"2024-09-11T07:28:49.998361+0000","last_peered":"2024-09-11T07:28:49.998361+0000","last_clean":"2024-09-11T07:28:49.998361+0000","last_became_active":"2024-09-11T07:26:37.861292+0000","last_became_peered":"2024-09-11T07:26:37.861292+0000","last_unstale":"2024-09-11T07:28:49.998361+0000","last_undegraded":"2024-09-11T07:28:49.998361+0000","last_fullsized":"2024-09-11T07:28:49.998361+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:55.018125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:55.018125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.234213+0000","scrub_duration":0.0013534339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.001634+0000","last_change":"2024-09-11T07:28:50.001302+0000","last_active":"2024-09-11T07:28:50.001634+0000","last_peered":"2024-09-11T07:28:50.001634+0000","last_clean":"2024-09-11T07:28:50.001634+0000","last_became_active":"2024-09-11T07:28:50.001027+0000","last_became_peered":"2024-09-11T07:28:50.001027+0000","last_unstale":"2024-09-11T07:28:50.001634+0000","last_undegraded":"2024-09-11T07:28:50.001634+0000","last_fullsized":"2024-09-11T07:28:50.001634+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:30:16.655233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"94'18","reported_seq":112,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-11T07:28:51.437499+0000","last_change":"2024-09-11T07:28:51.437460+0000","last_active":"2024-09-11T07:28:51.437499+0000","last_peered":"2024-09-11T07:28:51.437499+0000","last_clean":"2024-09-11T07:28:51.437499+0000","last_became_active":"2024-09-11T07:28:09.018134+0000","last_became_peered":"2024-09-11T07:28:09.018134+0000","last_unstale":"2024-09-11T07:28:51.437499+0000","last_undegraded":"2024-09-11T07:28:51.437499+0000","last_fullsized":"2024-09-11T07:28:51.437499+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'18","last_scrub_stamp":"2024-09-11T07:28:51.437397+0000","last_deep_scrub":"94'18","last_deep_scrub_stamp":"2024-09-11T07:28:51.437397+0000","last_clean_scrub_stamp":"2024-09-11T07:28:51.437397+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:59.214386+0000","scrub_duration":0.0052534629999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":62,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.997428+0000","last_change":"2024-09-11T07:28:08.286847+0000","last_active":"2024-09-11T07:28:49.997428+0000","last_peered":"2024-09-11T07:28:49.997428+0000","last_clean":"2024-09-11T07:28:49.997428+0000","last_became_active":"2024-09-11T07:26:31.420320+0000","last_became_peered":"2024-09-11T07:26:31.420320+0000","last_unstale":"2024-09-11T07:28:49.997428+0000","last_undegraded":"2024-09-11T07:28:49.997428+0000","last_fullsized":"2024-09-11T07:28:49.997428+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-11T07:28:08.286788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:08.286788+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.046248+0000","scrub_duration":0.001825976,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.997332+0000","last_change":"2024-09-11T07:28:16.967543+0000","last_active":"2024-09-11T07:28:49.997332+0000","last_peered":"2024-09-11T07:28:49.997332+0000","last_clean":"2024-09-11T07:28:49.997332+0000","last_became_active":"2024-09-11T07:26:31.417455+0000","last_became_peered":"2024-09-11T07:26:31.417455+0000","last_unstale":"2024-09-11T07:28:49.997332+0000","last_undegraded":"2024-09-11T07:28:49.997332+0000","last_fullsized":"2024-09-11T07:28:49.997332+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:16.967492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:16.967492+0000","last_clean_scrub_stamp":"2024-09-11T07:28:16.967492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:29.710608+0000","scrub_duration":0.0012319799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.002076+0000","last_change":"2024-09-11T07:28:21.868074+0000","last_active":"2024-09-11T07:28:50.002076+0000","last_peered":"2024-09-11T07:28:50.002076+0000","last_clean":"2024-09-11T07:28:50.002076+0000","last_became_active":"2024-09-11T07:26:35.564178+0000","last_became_peered":"2024-09-11T07:26:35.564178+0000","last_unstale":"2024-09-11T07:28:50.002076+0000","last_undegraded":"2024-09-11T07:28:50.002076+0000","last_fullsized":"2024-09-11T07:28:50.002076+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:21.868020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:21.868020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:40.076469+0000","scrub_duration":0.0013323009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.992919+0000","last_change":"2024-09-11T07:27:54.621934+0000","last_active":"2024-09-11T07:28:49.992919+0000","last_peered":"2024-09-11T07:28:49.992919+0000","last_clean":"2024-09-11T07:28:49.992919+0000","last_became_active":"2024-09-11T07:26:37.561378+0000","last_became_peered":"2024-09-11T07:26:37.561378+0000","last_unstale":"2024-09-11T07:28:49.992919+0000","last_undegraded":"2024-09-11T07:28:49.992919+0000","last_fullsized":"2024-09-11T07:28:49.992919+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.621874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.621874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:07.014230+0000","scrub_duration":0.0013604750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.001596+0000","last_change":"2024-09-11T07:28:50.001374+0000","last_active":"2024-09-11T07:28:50.001596+0000","last_peered":"2024-09-11T07:28:50.001596+0000","last_clean":"2024-09-11T07:28:50.001596+0000","last_became_active":"2024-09-11T07:28:50.000671+0000","last_became_peered":"2024-09-11T07:28:50.000671+0000","last_unstale":"2024-09-11T07:28:50.001596+0000","last_undegraded":"2024-09-11T07:28:50.001596+0000","last_fullsized":"2024-09-11T07:28:50.001596+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:30:06.554606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":76,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.001905+0000","last_change":"2024-09-11T07:27:49.061728+0000","last_active":"2024-09-11T07:28:50.001905+0000","last_peered":"2024-09-11T07:28:50.001905+0000","last_clean":"2024-09-11T07:28:50.001905+0000","last_became_active":"2024-09-11T07:26:33.541359+0000","last_became_peered":"2024-09-11T07:26:33.541359+0000","last_unstale":"2024-09-11T07:28:50.001905+0000","last_undegraded":"2024-09-11T07:28:50.001905+0000","last_fullsized":"2024-09-11T07:28:50.001905+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-09-11T07:27:49.061670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.061670+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:51.283390+0000","scrub_duration":0.001809801,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'3","reported_seq":64,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.998462+0000","last_change":"2024-09-11T07:28:01.711301+0000","last_active":"2024-09-11T07:28:49.998462+0000","last_peered":"2024-09-11T07:28:49.998462+0000","last_clean":"2024-09-11T07:28:49.998462+0000","last_became_active":"2024-09-11T07:26:31.419501+0000","last_became_peered":"2024-09-11T07:26:31.419501+0000","last_unstale":"2024-09-11T07:28:49.998462+0000","last_undegraded":"2024-09-11T07:28:49.998462+0000","last_fullsized":"2024-09-11T07:28:49.998462+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.711241+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:02.735271+0000","scrub_duration":0.0034627350000000002,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.000150+0000","last_change":"2024-09-11T07:27:47.903204+0000","last_active":"2024-09-11T07:28:50.000150+0000","last_peered":"2024-09-11T07:28:50.000150+0000","last_clean":"2024-09-11T07:28:50.000150+0000","last_became_active":"2024-09-11T07:26:31.423304+0000","last_became_peered":"2024-09-11T07:26:31.423304+0000","last_unstale":"2024-09-11T07:28:50.000150+0000","last_undegraded":"2024-09-11T07:28:50.000150+0000","last_fullsized":"2024-09-11T07:28:50.000150+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:47.903135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.903135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:01.867608+0000","scrub_duration":0.001284908,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"87'81","reported_seq":103,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.997530+0000","last_change":"2024-09-11T07:28:35.050762+0000","last_active":"2024-09-11T07:28:49.997530+0000","last_peered":"2024-09-11T07:28:49.997530+0000","last_clean":"2024-09-11T07:28:49.997530+0000","last_became_active":"2024-09-11T07:26:17.397381+0000","last_became_peered":"2024-09-11T07:26:17.397381+0000","last_unstale":"2024-09-11T07:28:49.997530+0000","last_undegraded":"2024-09-11T07:28:49.997530+0000","last_fullsized":"2024-09-11T07:28:49.997530+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:28:35.050689+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:28:35.050689+0000","last_clean_scrub_stamp":"2024-09-11T07:28:35.050689+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:39.908801+0000","scrub_duration":0.0039160200000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":62,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.190629+0000","last_change":"2024-09-11T07:28:50.190566+0000","last_active":"2024-09-11T07:28:50.190629+0000","last_peered":"2024-09-11T07:28:50.190629+0000","last_clean":"2024-09-11T07:28:50.190629+0000","last_became_active":"2024-09-11T07:26:35.564480+0000","last_became_peered":"2024-09-11T07:26:35.564480+0000","last_unstale":"2024-09-11T07:28:50.190629+0000","last_undegraded":"2024-09-11T07:28:50.190629+0000","last_fullsized":"2024-09-11T07:28:50.190629+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:50.190484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:50.190484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:30:13.399771+0000","scrub_duration":0.021062551999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.002139+0000","last_change":"2024-09-11T07:28:13.796731+0000","last_active":"2024-09-11T07:28:50.002139+0000","last_peered":"2024-09-11T07:28:50.002139+0000","last_clean":"2024-09-11T07:28:50.002139+0000","last_became_active":"2024-09-11T07:26:37.559920+0000","last_became_peered":"2024-09-11T07:26:37.559920+0000","last_unstale":"2024-09-11T07:28:50.002139+0000","last_undegraded":"2024-09-11T07:28:50.002139+0000","last_fullsized":"2024-09-11T07:28:50.002139+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:13.796657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:13.796657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:31.783767+0000","scrub_duration":0.001458984,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.003581+0000","last_change":"2024-09-11T07:28:50.003338+0000","last_active":"2024-09-11T07:28:50.003581+0000","last_peered":"2024-09-11T07:28:50.003581+0000","last_clean":"2024-09-11T07:28:50.003581+0000","last_became_active":"2024-09-11T07:28:50.003088+0000","last_became_peered":"2024-09-11T07:28:50.003088+0000","last_unstale":"2024-09-11T07:28:50.003581+0000","last_undegraded":"2024-09-11T07:28:50.003581+0000","last_fullsized":"2024-09-11T07:28:50.003581+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:53.711463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":85,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.997920+0000","last_change":"2024-09-11T07:27:59.128292+0000","last_active":"2024-09-11T07:28:49.997920+0000","last_peered":"2024-09-11T07:28:49.997920+0000","last_clean":"2024-09-11T07:28:49.997920+0000","last_became_active":"2024-09-11T07:26:33.543687+0000","last_became_peered":"2024-09-11T07:26:33.543687+0000","last_unstale":"2024-09-11T07:28:49.997920+0000","last_undegraded":"2024-09-11T07:28:49.997920+0000","last_fullsized":"2024-09-11T07:28:49.997920+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-11T07:27:59.128237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:59.128237+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:16.835564+0000","scrub_duration":0.00199329,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.994335+0000","last_change":"2024-09-11T07:27:54.182157+0000","last_active":"2024-09-11T07:28:49.994335+0000","last_peered":"2024-09-11T07:28:49.994335+0000","last_clean":"2024-09-11T07:28:49.994335+0000","last_became_active":"2024-09-11T07:26:31.419560+0000","last_became_peered":"2024-09-11T07:26:31.419560+0000","last_unstale":"2024-09-11T07:28:49.994335+0000","last_undegraded":"2024-09-11T07:28:49.994335+0000","last_fullsized":"2024-09-11T07:28:49.994335+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.182106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.182106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:04.075506+0000","scrub_duration":0.001213254,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993500+0000","last_change":"2024-09-11T07:28:13.357659+0000","last_active":"2024-09-11T07:28:49.993500+0000","last_peered":"2024-09-11T07:28:49.993500+0000","last_clean":"2024-09-11T07:28:49.993500+0000","last_became_active":"2024-09-11T07:26:35.564044+0000","last_became_peered":"2024-09-11T07:26:35.564044+0000","last_unstale":"2024-09-11T07:28:49.993500+0000","last_undegraded":"2024-09-11T07:28:49.993500+0000","last_fullsized":"2024-09-11T07:28:49.993500+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:13.357544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:13.357544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:42.978875+0000","scrub_duration":0.0017127900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.997425+0000","last_change":"2024-09-11T07:28:13.918148+0000","last_active":"2024-09-11T07:28:49.997425+0000","last_peered":"2024-09-11T07:28:49.997425+0000","last_clean":"2024-09-11T07:28:49.997425+0000","last_became_active":"2024-09-11T07:26:37.564801+0000","last_became_peered":"2024-09-11T07:26:37.564801+0000","last_unstale":"2024-09-11T07:28:49.997425+0000","last_undegraded":"2024-09-11T07:28:49.997425+0000","last_fullsized":"2024-09-11T07:28:49.997425+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:13.918082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:13.918082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:31.000135+0000","scrub_duration":0.0014064769999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.000087+0000","last_change":"2024-09-11T07:28:49.999416+0000","last_active":"2024-09-11T07:28:50.000087+0000","last_peered":"2024-09-11T07:28:50.000087+0000","last_clean":"2024-09-11T07:28:50.000087+0000","last_became_active":"2024-09-11T07:28:49.999151+0000","last_became_peered":"2024-09-11T07:28:49.999151+0000","last_unstale":"2024-09-11T07:28:50.000087+0000","last_undegraded":"2024-09-11T07:28:50.000087+0000","last_fullsized":"2024-09-11T07:28:50.000087+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:51.009131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":70,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.996511+0000","last_change":"2024-09-11T07:28:02.230985+0000","last_active":"2024-09-11T07:28:49.996511+0000","last_peered":"2024-09-11T07:28:49.996511+0000","last_clean":"2024-09-11T07:28:49.996511+0000","last_became_active":"2024-09-11T07:26:33.543148+0000","last_became_peered":"2024-09-11T07:26:33.543148+0000","last_unstale":"2024-09-11T07:28:49.996511+0000","last_undegraded":"2024-09-11T07:28:49.996511+0000","last_fullsized":"2024-09-11T07:28:49.996511+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:28:02.230928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.230928+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:27.244330+0000","scrub_duration":0.001888926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993285+0000","last_change":"2024-09-11T07:27:53.659961+0000","last_active":"2024-09-11T07:28:49.993285+0000","last_peered":"2024-09-11T07:28:49.993285+0000","last_clean":"2024-09-11T07:28:49.993285+0000","last_became_active":"2024-09-11T07:26:31.423056+0000","last_became_peered":"2024-09-11T07:26:31.423056+0000","last_unstale":"2024-09-11T07:28:49.993285+0000","last_undegraded":"2024-09-11T07:28:49.993285+0000","last_fullsized":"2024-09-11T07:28:49.993285+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:53.659902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:53.659902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.781453+0000","scrub_duration":0.0011613719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993696+0000","last_change":"2024-09-11T07:28:10.337599+0000","last_active":"2024-09-11T07:28:49.993696+0000","last_peered":"2024-09-11T07:28:49.993696+0000","last_clean":"2024-09-11T07:28:49.993696+0000","last_became_active":"2024-09-11T07:26:35.563182+0000","last_became_peered":"2024-09-11T07:26:35.563182+0000","last_unstale":"2024-09-11T07:28:49.993696+0000","last_undegraded":"2024-09-11T07:28:49.993696+0000","last_fullsized":"2024-09-11T07:28:49.993696+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:10.337527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:10.337527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:24.013574+0000","scrub_duration":0.001985653,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.997346+0000","last_change":"2024-09-11T07:28:18.503995+0000","last_active":"2024-09-11T07:28:49.997346+0000","last_peered":"2024-09-11T07:28:49.997346+0000","last_clean":"2024-09-11T07:28:49.997346+0000","last_became_active":"2024-09-11T07:26:37.565029+0000","last_became_peered":"2024-09-11T07:26:37.565029+0000","last_unstale":"2024-09-11T07:28:49.997346+0000","last_undegraded":"2024-09-11T07:28:49.997346+0000","last_fullsized":"2024-09-11T07:28:49.997346+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:18.503930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:18.503930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:36.076642+0000","scrub_duration":0.001490297,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.004303+0000","last_change":"2024-09-11T07:28:50.003935+0000","last_active":"2024-09-11T07:28:50.004303+0000","last_peered":"2024-09-11T07:28:50.004303+0000","last_clean":"2024-09-11T07:28:50.004303+0000","last_became_active":"2024-09-11T07:28:50.003627+0000","last_became_peered":"2024-09-11T07:28:50.003627+0000","last_unstale":"2024-09-11T07:28:50.004303+0000","last_undegraded":"2024-09-11T07:28:50.004303+0000","last_fullsized":"2024-09-11T07:28:50.004303+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:30:04.839897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":79,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.999054+0000","last_change":"2024-09-11T07:28:07.811136+0000","last_active":"2024-09-11T07:28:49.999054+0000","last_peered":"2024-09-11T07:28:49.999054+0000","last_clean":"2024-09-11T07:28:49.999054+0000","last_became_active":"2024-09-11T07:26:33.538216+0000","last_became_peered":"2024-09-11T07:26:33.538216+0000","last_unstale":"2024-09-11T07:28:49.999054+0000","last_undegraded":"2024-09-11T07:28:49.999054+0000","last_fullsized":"2024-09-11T07:28:49.999054+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-09-11T07:28:07.811071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:07.811071+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:23.495093+0000","scrub_duration":0.0019627080000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.994477+0000","last_change":"2024-09-11T07:27:50.132629+0000","last_active":"2024-09-11T07:28:49.994477+0000","last_peered":"2024-09-11T07:28:49.994477+0000","last_clean":"2024-09-11T07:28:49.994477+0000","last_became_active":"2024-09-11T07:26:31.418490+0000","last_became_peered":"2024-09-11T07:26:31.418490+0000","last_unstale":"2024-09-11T07:28:49.994477+0000","last_undegraded":"2024-09-11T07:28:49.994477+0000","last_fullsized":"2024-09-11T07:28:49.994477+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.132552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.048681+0000","scrub_duration":0.001622807,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":88,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.996407+0000","last_change":"2024-09-11T07:28:32.017483+0000","last_active":"2024-09-11T07:28:49.996407+0000","last_peered":"2024-09-11T07:28:49.996407+0000","last_clean":"2024-09-11T07:28:49.996407+0000","last_became_active":"2024-09-11T07:26:35.566888+0000","last_became_peered":"2024-09-11T07:26:35.566888+0000","last_unstale":"2024-09-11T07:28:49.996407+0000","last_undegraded":"2024-09-11T07:28:49.996407+0000","last_fullsized":"2024-09-11T07:28:49.996407+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:28:32.017418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:32.017418+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:55.321090+0000","scrub_duration":0.0016851839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.998394+0000","last_change":"2024-09-11T07:28:03.961280+0000","last_active":"2024-09-11T07:28:49.998394+0000","last_peered":"2024-09-11T07:28:49.998394+0000","last_clean":"2024-09-11T07:28:49.998394+0000","last_became_active":"2024-09-11T07:26:37.861134+0000","last_became_peered":"2024-09-11T07:26:37.861134+0000","last_unstale":"2024-09-11T07:28:49.998394+0000","last_undegraded":"2024-09-11T07:28:49.998394+0000","last_fullsized":"2024-09-11T07:28:49.998394+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:03.961220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:03.961220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:29.223995+0000","scrub_duration":0.0011943730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.005445+0000","last_change":"2024-09-11T07:28:50.004530+0000","last_active":"2024-09-11T07:28:50.005445+0000","last_peered":"2024-09-11T07:28:50.005445+0000","last_clean":"2024-09-11T07:28:50.005445+0000","last_became_active":"2024-09-11T07:28:50.004253+0000","last_became_peered":"2024-09-11T07:28:50.004253+0000","last_unstale":"2024-09-11T07:28:50.005445+0000","last_undegraded":"2024-09-11T07:28:50.005445+0000","last_fullsized":"2024-09-11T07:28:50.005445+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:30:16.438942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"83'17","reported_seq":82,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993003+0000","last_change":"2024-09-11T07:28:01.098068+0000","last_active":"2024-09-11T07:28:49.993003+0000","last_peered":"2024-09-11T07:28:49.993003+0000","last_clean":"2024-09-11T07:28:49.993003+0000","last_became_active":"2024-09-11T07:26:33.540363+0000","last_became_peered":"2024-09-11T07:26:33.540363+0000","last_unstale":"2024-09-11T07:28:49.993003+0000","last_undegraded":"2024-09-11T07:28:49.993003+0000","last_fullsized":"2024-09-11T07:28:49.993003+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'17","last_scrub_stamp":"2024-09-11T07:28:01.097981+0000","last_deep_scrub":"83'17","last_deep_scrub_stamp":"2024-09-11T07:28:01.097981+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.097981+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:26.179592+0000","scrub_duration":0.0042048720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":73,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993984+0000","last_change":"2024-09-11T07:27:46.077857+0000","last_active":"2024-09-11T07:28:49.993984+0000","last_peered":"2024-09-11T07:28:49.993984+0000","last_clean":"2024-09-11T07:28:49.993984+0000","last_became_active":"2024-09-11T07:26:31.419718+0000","last_became_peered":"2024-09-11T07:26:31.419718+0000","last_unstale":"2024-09-11T07:28:49.993984+0000","last_undegraded":"2024-09-11T07:28:49.993984+0000","last_fullsized":"2024-09-11T07:28:49.993984+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:27:46.077786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:46.077786+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:07.735381+0000","scrub_duration":0.0019376720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.994374+0000","last_change":"2024-09-11T07:27:53.209518+0000","last_active":"2024-09-11T07:28:49.994374+0000","last_peered":"2024-09-11T07:28:49.994374+0000","last_clean":"2024-09-11T07:28:49.994374+0000","last_became_active":"2024-09-11T07:26:35.561967+0000","last_became_peered":"2024-09-11T07:26:35.561967+0000","last_unstale":"2024-09-11T07:28:49.994374+0000","last_undegraded":"2024-09-11T07:28:49.994374+0000","last_fullsized":"2024-09-11T07:28:49.994374+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:53.209452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:53.209452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:23.125281+0000","scrub_duration":0.0014563029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.996798+0000","last_change":"2024-09-11T07:28:02.677186+0000","last_active":"2024-09-11T07:28:49.996798+0000","last_peered":"2024-09-11T07:28:49.996798+0000","last_clean":"2024-09-11T07:28:49.996798+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:28:49.996798+0000","last_undegraded":"2024-09-11T07:28:49.996798+0000","last_fullsized":"2024-09-11T07:28:49.996798+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:02.677095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.677095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:03.575070+0000","scrub_duration":0.001519269,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":70,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.997467+0000","last_change":"2024-09-11T07:27:40.351572+0000","last_active":"2024-09-11T07:28:49.997467+0000","last_peered":"2024-09-11T07:28:49.997467+0000","last_clean":"2024-09-11T07:28:49.997467+0000","last_became_active":"2024-09-11T07:26:33.547344+0000","last_became_peered":"2024-09-11T07:26:33.547344+0000","last_unstale":"2024-09-11T07:28:49.997467+0000","last_undegraded":"2024-09-11T07:28:49.997467+0000","last_fullsized":"2024-09-11T07:28:49.997467+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:27:40.351507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:40.351507+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.098394+0000","scrub_duration":0.0021803980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.994506+0000","last_change":"2024-09-11T07:27:58.324311+0000","last_active":"2024-09-11T07:28:49.994506+0000","last_peered":"2024-09-11T07:28:49.994506+0000","last_clean":"2024-09-11T07:28:49.994506+0000","last_became_active":"2024-09-11T07:26:31.415701+0000","last_became_peered":"2024-09-11T07:26:31.415701+0000","last_unstale":"2024-09-11T07:28:49.994506+0000","last_undegraded":"2024-09-11T07:28:49.994506+0000","last_fullsized":"2024-09-11T07:28:49.994506+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:58.324255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.324255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.617500+0000","scrub_duration":0.0013365549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.998195+0000","last_change":"2024-09-11T07:28:12.778700+0000","last_active":"2024-09-11T07:28:49.998195+0000","last_peered":"2024-09-11T07:28:49.998195+0000","last_clean":"2024-09-11T07:28:49.998195+0000","last_became_active":"2024-09-11T07:26:35.559262+0000","last_became_peered":"2024-09-11T07:26:35.559262+0000","last_unstale":"2024-09-11T07:28:49.998195+0000","last_undegraded":"2024-09-11T07:28:49.998195+0000","last_fullsized":"2024-09-11T07:28:49.998195+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:12.778645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:12.778645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.979009+0000","scrub_duration":0.0013084870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993843+0000","last_change":"2024-09-11T07:28:06.305563+0000","last_active":"2024-09-11T07:28:49.993843+0000","last_peered":"2024-09-11T07:28:49.993843+0000","last_clean":"2024-09-11T07:28:49.993843+0000","last_became_active":"2024-09-11T07:26:37.564084+0000","last_became_peered":"2024-09-11T07:26:37.564084+0000","last_unstale":"2024-09-11T07:28:49.993843+0000","last_undegraded":"2024-09-11T07:28:49.993843+0000","last_fullsized":"2024-09-11T07:28:49.993843+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:28:06.305502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:06.305502+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.659083+0000","scrub_duration":0.0021934649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.997773+0000","last_change":"2024-09-11T07:27:45.990617+0000","last_active":"2024-09-11T07:28:49.997773+0000","last_peered":"2024-09-11T07:28:49.997773+0000","last_clean":"2024-09-11T07:28:49.997773+0000","last_became_active":"2024-09-11T07:26:31.423229+0000","last_became_peered":"2024-09-11T07:26:31.423229+0000","last_unstale":"2024-09-11T07:28:49.997773+0000","last_undegraded":"2024-09-11T07:28:49.997773+0000","last_fullsized":"2024-09-11T07:28:49.997773+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:45.990548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:45.990548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:51.641867+0000","scrub_duration":0.0014585150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"82'9","reported_seq":66,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.998171+0000","last_change":"2024-09-11T07:27:48.761470+0000","last_active":"2024-09-11T07:28:49.998171+0000","last_peered":"2024-09-11T07:28:49.998171+0000","last_clean":"2024-09-11T07:28:49.998171+0000","last_became_active":"2024-09-11T07:26:33.543628+0000","last_became_peered":"2024-09-11T07:26:33.543628+0000","last_unstale":"2024-09-11T07:28:49.998171+0000","last_undegraded":"2024-09-11T07:28:49.998171+0000","last_fullsized":"2024-09-11T07:28:49.998171+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'9","last_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:27:48.761397+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.913254+0000","scrub_duration":0.0037047439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.000518+0000","last_change":"2024-09-11T07:27:42.917739+0000","last_active":"2024-09-11T07:28:50.000518+0000","last_peered":"2024-09-11T07:28:50.000518+0000","last_clean":"2024-09-11T07:28:50.000518+0000","last_became_active":"2024-09-11T07:26:35.566894+0000","last_became_peered":"2024-09-11T07:26:35.566894+0000","last_unstale":"2024-09-11T07:28:50.000518+0000","last_undegraded":"2024-09-11T07:28:50.000518+0000","last_fullsized":"2024-09-11T07:28:50.000518+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.917680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:06.763663+0000","scrub_duration":0.0013496649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.997750+0000","last_change":"2024-09-11T07:27:47.031883+0000","last_active":"2024-09-11T07:28:49.997750+0000","last_peered":"2024-09-11T07:28:49.997750+0000","last_clean":"2024-09-11T07:28:49.997750+0000","last_became_active":"2024-09-11T07:26:37.863100+0000","last_became_peered":"2024-09-11T07:26:37.863100+0000","last_unstale":"2024-09-11T07:28:49.997750+0000","last_undegraded":"2024-09-11T07:28:49.997750+0000","last_fullsized":"2024-09-11T07:28:49.997750+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:47.031807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.031807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:54.469643+0000","scrub_duration":0.001556195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"86'15","reported_seq":74,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.996463+0000","last_change":"2024-09-11T07:28:22.858192+0000","last_active":"2024-09-11T07:28:49.996463+0000","last_peered":"2024-09-11T07:28:49.996463+0000","last_clean":"2024-09-11T07:28:49.996463+0000","last_became_active":"2024-09-11T07:26:33.545078+0000","last_became_peered":"2024-09-11T07:26:33.545078+0000","last_unstale":"2024-09-11T07:28:49.996463+0000","last_undegraded":"2024-09-11T07:28:49.996463+0000","last_fullsized":"2024-09-11T07:28:49.996463+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'15","last_scrub_stamp":"2024-09-11T07:28:22.858137+0000","last_deep_scrub":"86'15","last_deep_scrub_stamp":"2024-09-11T07:28:22.858137+0000","last_clean_scrub_stamp":"2024-09-11T07:28:22.858137+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:22.911147+0000","scrub_duration":0.003508101,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.998702+0000","last_change":"2024-09-11T07:28:14.819081+0000","last_active":"2024-09-11T07:28:49.998702+0000","last_peered":"2024-09-11T07:28:49.998702+0000","last_clean":"2024-09-11T07:28:49.998702+0000","last_became_active":"2024-09-11T07:26:31.423789+0000","last_became_peered":"2024-09-11T07:26:31.423789+0000","last_unstale":"2024-09-11T07:28:49.998702+0000","last_undegraded":"2024-09-11T07:28:49.998702+0000","last_fullsized":"2024-09-11T07:28:49.998702+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:14.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:14.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:17.757008+0000","scrub_duration":0.001929339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":88,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.999470+0000","last_change":"2024-09-11T07:28:16.782518+0000","last_active":"2024-09-11T07:28:49.999470+0000","last_peered":"2024-09-11T07:28:49.999470+0000","last_clean":"2024-09-11T07:28:49.999470+0000","last_became_active":"2024-09-11T07:26:35.562245+0000","last_became_peered":"2024-09-11T07:26:35.562245+0000","last_unstale":"2024-09-11T07:28:49.999470+0000","last_undegraded":"2024-09-11T07:28:49.999470+0000","last_fullsized":"2024-09-11T07:28:49.999470+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:28:16.782462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:16.782462+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:30.251276+0000","scrub_duration":0.001462283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.997062+0000","last_change":"2024-09-11T07:27:49.245119+0000","last_active":"2024-09-11T07:28:49.997062+0000","last_peered":"2024-09-11T07:28:49.997062+0000","last_clean":"2024-09-11T07:28:49.997062+0000","last_became_active":"2024-09-11T07:26:37.566749+0000","last_became_peered":"2024-09-11T07:26:37.566749+0000","last_unstale":"2024-09-11T07:28:49.997062+0000","last_undegraded":"2024-09-11T07:28:49.997062+0000","last_fullsized":"2024-09-11T07:28:49.997062+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:49.245047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.245047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001867966,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"83'13","reported_seq":74,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.999485+0000","last_change":"2024-09-11T07:28:00.959669+0000","last_active":"2024-09-11T07:28:49.999485+0000","last_peered":"2024-09-11T07:28:49.999485+0000","last_clean":"2024-09-11T07:28:49.999485+0000","last_became_active":"2024-09-11T07:26:33.537425+0000","last_became_peered":"2024-09-11T07:26:33.537425+0000","last_unstale":"2024-09-11T07:28:49.999485+0000","last_undegraded":"2024-09-11T07:28:49.999485+0000","last_fullsized":"2024-09-11T07:28:49.999485+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-11T07:28:00.959583+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-11T07:28:00.959583+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.959583+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:06.630980+0000","scrub_duration":0.0032987350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.996858+0000","last_change":"2024-09-11T07:28:01.237329+0000","last_active":"2024-09-11T07:28:49.996858+0000","last_peered":"2024-09-11T07:28:49.996858+0000","last_clean":"2024-09-11T07:28:49.996858+0000","last_became_active":"2024-09-11T07:26:31.419409+0000","last_became_peered":"2024-09-11T07:26:31.419409+0000","last_unstale":"2024-09-11T07:28:49.996858+0000","last_undegraded":"2024-09-11T07:28:49.996858+0000","last_fullsized":"2024-09-11T07:28:49.996858+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:01.237267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.237267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.266034+0000","scrub_duration":0.0012797399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":88,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.996176+0000","last_change":"2024-09-11T07:28:23.883321+0000","last_active":"2024-09-11T07:28:49.996176+0000","last_peered":"2024-09-11T07:28:49.996176+0000","last_clean":"2024-09-11T07:28:49.996176+0000","last_became_active":"2024-09-11T07:26:35.560610+0000","last_became_peered":"2024-09-11T07:26:35.560610+0000","last_unstale":"2024-09-11T07:28:49.996176+0000","last_undegraded":"2024-09-11T07:28:49.996176+0000","last_fullsized":"2024-09-11T07:28:49.996176+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:28:23.883265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:23.883265+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:49.732087+0000","scrub_duration":0.0020633370000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993719+0000","last_change":"2024-09-11T07:28:22.053350+0000","last_active":"2024-09-11T07:28:49.993719+0000","last_peered":"2024-09-11T07:28:49.993719+0000","last_clean":"2024-09-11T07:28:49.993719+0000","last_became_active":"2024-09-11T07:26:37.565723+0000","last_became_peered":"2024-09-11T07:26:37.565723+0000","last_unstale":"2024-09-11T07:28:49.993719+0000","last_undegraded":"2024-09-11T07:28:49.993719+0000","last_fullsized":"2024-09-11T07:28:49.993719+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:22.053291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:22.053291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:40.893276+0000","scrub_duration":0.001270356,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":70,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.992684+0000","last_change":"2024-09-11T07:27:57.601328+0000","last_active":"2024-09-11T07:28:49.992684+0000","last_peered":"2024-09-11T07:28:49.992684+0000","last_clean":"2024-09-11T07:28:49.992684+0000","last_became_active":"2024-09-11T07:26:33.540688+0000","last_became_peered":"2024-09-11T07:26:33.540688+0000","last_unstale":"2024-09-11T07:28:49.992684+0000","last_undegraded":"2024-09-11T07:28:49.992684+0000","last_fullsized":"2024-09-11T07:28:49.992684+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:27:57.601261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.601261+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:58.826830+0000","scrub_duration":0.002247724,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.001161+0000","last_change":"2024-09-11T07:27:54.102580+0000","last_active":"2024-09-11T07:28:50.001161+0000","last_peered":"2024-09-11T07:28:50.001161+0000","last_clean":"2024-09-11T07:28:50.001161+0000","last_became_active":"2024-09-11T07:26:31.425587+0000","last_became_peered":"2024-09-11T07:26:31.425587+0000","last_unstale":"2024-09-11T07:28:50.001161+0000","last_undegraded":"2024-09-11T07:28:50.001161+0000","last_fullsized":"2024-09-11T07:28:50.001161+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.102528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.102528+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.102528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:06.835183+0000","scrub_duration":0.0012736080000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.996866+0000","last_change":"2024-09-11T07:28:29.789817+0000","last_active":"2024-09-11T07:28:49.996866+0000","last_peered":"2024-09-11T07:28:49.996866+0000","last_clean":"2024-09-11T07:28:49.996866+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:28:49.996866+0000","last_undegraded":"2024-09-11T07:28:49.996866+0000","last_fullsized":"2024-09-11T07:28:49.996866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:29.789752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:29.789752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:42.508949+0000","scrub_duration":0.001301548,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.995780+0000","last_change":"2024-09-11T07:27:58.192585+0000","last_active":"2024-09-11T07:28:49.995780+0000","last_peered":"2024-09-11T07:28:49.995780+0000","last_clean":"2024-09-11T07:28:49.995780+0000","last_became_active":"2024-09-11T07:26:37.568886+0000","last_became_peered":"2024-09-11T07:26:37.568886+0000","last_unstale":"2024-09-11T07:28:49.995780+0000","last_undegraded":"2024-09-11T07:28:49.995780+0000","last_fullsized":"2024-09-11T07:28:49.995780+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:58.192528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.192528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:18.757569+0000","scrub_duration":0.001884698,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":73,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993448+0000","last_change":"2024-09-11T07:27:48.081460+0000","last_active":"2024-09-11T07:28:49.993448+0000","last_peered":"2024-09-11T07:28:49.993448+0000","last_clean":"2024-09-11T07:28:49.993448+0000","last_became_active":"2024-09-11T07:26:33.540208+0000","last_became_peered":"2024-09-11T07:26:33.540208+0000","last_unstale":"2024-09-11T07:28:49.993448+0000","last_undegraded":"2024-09-11T07:28:49.993448+0000","last_fullsized":"2024-09-11T07:28:49.993448+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:27:48.081392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:48.081392+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:04.524601+0000","scrub_duration":0.0022394590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993214+0000","last_change":"2024-09-11T07:27:49.677063+0000","last_active":"2024-09-11T07:28:49.993214+0000","last_peered":"2024-09-11T07:28:49.993214+0000","last_clean":"2024-09-11T07:28:49.993214+0000","last_became_active":"2024-09-11T07:26:31.422671+0000","last_became_peered":"2024-09-11T07:26:31.422671+0000","last_unstale":"2024-09-11T07:28:49.993214+0000","last_undegraded":"2024-09-11T07:28:49.993214+0000","last_fullsized":"2024-09-11T07:28:49.993214+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.677003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:54.836177+0000","scrub_duration":0.0013962969999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.992605+0000","last_change":"2024-09-11T07:28:30.847535+0000","last_active":"2024-09-11T07:28:49.992605+0000","last_peered":"2024-09-11T07:28:49.992605+0000","last_clean":"2024-09-11T07:28:49.992605+0000","last_became_active":"2024-09-11T07:26:35.566931+0000","last_became_peered":"2024-09-11T07:26:35.566931+0000","last_unstale":"2024-09-11T07:28:49.992605+0000","last_undegraded":"2024-09-11T07:28:49.992605+0000","last_fullsized":"2024-09-11T07:28:49.992605+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:30.847474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.847474+0000","last_clean_scrub_stamp":"2024-09-11T07:28:30.847474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:48.388338+0000","scrub_duration":0.001576682,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.997747+0000","last_change":"2024-09-11T07:28:20.766566+0000","last_active":"2024-09-11T07:28:49.997747+0000","last_peered":"2024-09-11T07:28:49.997747+0000","last_clean":"2024-09-11T07:28:49.997747+0000","last_became_active":"2024-09-11T07:26:37.561573+0000","last_became_peered":"2024-09-11T07:26:37.561573+0000","last_unstale":"2024-09-11T07:28:49.997747+0000","last_undegraded":"2024-09-11T07:28:49.997747+0000","last_fullsized":"2024-09-11T07:28:49.997747+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:20.766508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:20.766508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:42.240723+0000","scrub_duration":0.0012453239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"82'6","reported_seq":62,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.996099+0000","last_change":"2024-09-11T07:27:47.789519+0000","last_active":"2024-09-11T07:28:49.996099+0000","last_peered":"2024-09-11T07:28:49.996099+0000","last_clean":"2024-09-11T07:28:49.996099+0000","last_became_active":"2024-09-11T07:26:33.545830+0000","last_became_peered":"2024-09-11T07:26:33.545830+0000","last_unstale":"2024-09-11T07:28:49.996099+0000","last_undegraded":"2024-09-11T07:28:49.996099+0000","last_fullsized":"2024-09-11T07:28:49.996099+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'6","last_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_deep_scrub":"82'6","last_deep_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.789465+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:59.162056+0000","scrub_duration":0.002624679,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.001735+0000","last_change":"2024-09-11T07:27:58.043076+0000","last_active":"2024-09-11T07:28:50.001735+0000","last_peered":"2024-09-11T07:28:50.001735+0000","last_clean":"2024-09-11T07:28:50.001735+0000","last_became_active":"2024-09-11T07:26:31.420518+0000","last_became_peered":"2024-09-11T07:26:31.420518+0000","last_unstale":"2024-09-11T07:28:50.001735+0000","last_undegraded":"2024-09-11T07:28:50.001735+0000","last_fullsized":"2024-09-11T07:28:50.001735+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.043009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:24.326885+0000","scrub_duration":0.0015014900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.997489+0000","last_change":"2024-09-11T07:27:56.124601+0000","last_active":"2024-09-11T07:28:49.997489+0000","last_peered":"2024-09-11T07:28:49.997489+0000","last_clean":"2024-09-11T07:28:49.997489+0000","last_became_active":"2024-09-11T07:26:35.560506+0000","last_became_peered":"2024-09-11T07:26:35.560506+0000","last_unstale":"2024-09-11T07:28:49.997489+0000","last_undegraded":"2024-09-11T07:28:49.997489+0000","last_fullsized":"2024-09-11T07:28:49.997489+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:56.124542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:56.124542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:09.847074+0000","scrub_duration":0.0011688619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.996578+0000","last_change":"2024-09-11T07:27:42.059066+0000","last_active":"2024-09-11T07:28:49.996578+0000","last_peered":"2024-09-11T07:28:49.996578+0000","last_clean":"2024-09-11T07:28:49.996578+0000","last_became_active":"2024-09-11T07:26:37.564208+0000","last_became_peered":"2024-09-11T07:26:37.564208+0000","last_unstale":"2024-09-11T07:28:49.996578+0000","last_undegraded":"2024-09-11T07:28:49.996578+0000","last_fullsized":"2024-09-11T07:28:49.996578+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.059001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.059001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001418094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":73,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.997931+0000","last_change":"2024-09-11T07:28:28.771165+0000","last_active":"2024-09-11T07:28:49.997931+0000","last_peered":"2024-09-11T07:28:49.997931+0000","last_clean":"2024-09-11T07:28:49.997931+0000","last_became_active":"2024-09-11T07:26:33.542625+0000","last_became_peered":"2024-09-11T07:26:33.542625+0000","last_unstale":"2024-09-11T07:28:49.997931+0000","last_undegraded":"2024-09-11T07:28:49.997931+0000","last_fullsized":"2024-09-11T07:28:49.997931+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:28:28.771107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:28.771107+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:44.367423+0000","scrub_duration":0.001700355,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.002762+0000","last_change":"2024-09-11T07:28:02.157520+0000","last_active":"2024-09-11T07:28:50.002762+0000","last_peered":"2024-09-11T07:28:50.002762+0000","last_clean":"2024-09-11T07:28:50.002762+0000","last_became_active":"2024-09-11T07:26:31.425876+0000","last_became_peered":"2024-09-11T07:26:31.425876+0000","last_unstale":"2024-09-11T07:28:50.002762+0000","last_undegraded":"2024-09-11T07:28:50.002762+0000","last_fullsized":"2024-09-11T07:28:50.002762+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:02.157455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.157455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:12.096002+0000","scrub_duration":0.0016990040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.999142+0000","last_change":"2024-09-11T07:28:27.019028+0000","last_active":"2024-09-11T07:28:49.999142+0000","last_peered":"2024-09-11T07:28:49.999142+0000","last_clean":"2024-09-11T07:28:49.999142+0000","last_became_active":"2024-09-11T07:26:35.565339+0000","last_became_peered":"2024-09-11T07:26:35.565339+0000","last_unstale":"2024-09-11T07:28:49.999142+0000","last_undegraded":"2024-09-11T07:28:49.999142+0000","last_fullsized":"2024-09-11T07:28:49.999142+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:27.018961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:27.018961+0000","last_clean_scrub_stamp":"2024-09-11T07:28:27.018961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:27.125944+0000","scrub_duration":0.001482106,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.996728+0000","last_change":"2024-09-11T07:28:31.783418+0000","last_active":"2024-09-11T07:28:49.996728+0000","last_peered":"2024-09-11T07:28:49.996728+0000","last_clean":"2024-09-11T07:28:49.996728+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:28:49.996728+0000","last_undegraded":"2024-09-11T07:28:49.996728+0000","last_fullsized":"2024-09-11T07:28:49.996728+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:31.783359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:31.783359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:58.426334+0000","scrub_duration":0.0016047909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.997142+0000","last_change":"2024-09-11T07:28:28.990232+0000","last_active":"2024-09-11T07:28:49.997142+0000","last_peered":"2024-09-11T07:28:49.997142+0000","last_clean":"2024-09-11T07:28:49.997142+0000","last_became_active":"2024-09-11T07:26:37.563942+0000","last_became_peered":"2024-09-11T07:26:37.563942+0000","last_unstale":"2024-09-11T07:28:49.997142+0000","last_undegraded":"2024-09-11T07:28:49.997142+0000","last_fullsized":"2024-09-11T07:28:49.997142+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:28:28.990175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:28.990175+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:45.813007+0000","scrub_duration":0.001863689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":65,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.999158+0000","last_change":"2024-09-11T07:28:02.969498+0000","last_active":"2024-09-11T07:28:49.999158+0000","last_peered":"2024-09-11T07:28:49.999158+0000","last_clean":"2024-09-11T07:28:49.999158+0000","last_became_active":"2024-09-11T07:26:31.423932+0000","last_became_peered":"2024-09-11T07:26:31.423932+0000","last_unstale":"2024-09-11T07:28:49.999158+0000","last_undegraded":"2024-09-11T07:28:49.999158+0000","last_fullsized":"2024-09-11T07:28:49.999158+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:28:02.969442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.969442+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:22.652879+0000","scrub_duration":0.0014604450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":69,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.997372+0000","last_change":"2024-09-11T07:27:55.150630+0000","last_active":"2024-09-11T07:28:49.997372+0000","last_peered":"2024-09-11T07:28:49.997372+0000","last_clean":"2024-09-11T07:28:49.997372+0000","last_became_active":"2024-09-11T07:26:33.546797+0000","last_became_peered":"2024-09-11T07:26:33.546797+0000","last_unstale":"2024-09-11T07:28:49.997372+0000","last_undegraded":"2024-09-11T07:28:49.997372+0000","last_fullsized":"2024-09-11T07:28:49.997372+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:27:55.150567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:55.150567+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:10.187014+0000","scrub_duration":0.0016612689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"64'8","reported_seq":91,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.002572+0000","last_change":"2024-09-11T07:28:01.123469+0000","last_active":"2024-09-11T07:28:50.002572+0000","last_peered":"2024-09-11T07:28:50.002572+0000","last_clean":"2024-09-11T07:28:50.002572+0000","last_became_active":"2024-09-11T07:26:35.559304+0000","last_became_peered":"2024-09-11T07:26:35.559304+0000","last_unstale":"2024-09-11T07:28:50.002572+0000","last_undegraded":"2024-09-11T07:28:50.002572+0000","last_fullsized":"2024-09-11T07:28:50.002572+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:28:01.123401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.123401+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:18.715539+0000","scrub_duration":0.00167504,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.992846+0000","last_change":"2024-09-11T07:28:11.735040+0000","last_active":"2024-09-11T07:28:49.992846+0000","last_peered":"2024-09-11T07:28:49.992846+0000","last_clean":"2024-09-11T07:28:49.992846+0000","last_became_active":"2024-09-11T07:26:37.562501+0000","last_became_peered":"2024-09-11T07:26:37.562501+0000","last_unstale":"2024-09-11T07:28:49.992846+0000","last_undegraded":"2024-09-11T07:28:49.992846+0000","last_fullsized":"2024-09-11T07:28:49.992846+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:11.734968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:11.734968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:13.880946+0000","scrub_duration":0.0021993350000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.998252+0000","last_change":"2024-09-11T07:27:38.732867+0000","last_active":"2024-09-11T07:28:49.998252+0000","last_peered":"2024-09-11T07:28:49.998252+0000","last_clean":"2024-09-11T07:28:49.998252+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:28:49.998252+0000","last_undegraded":"2024-09-11T07:28:49.998252+0000","last_fullsized":"2024-09-11T07:28:49.998252+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:38.732795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:38.732795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:56.730699+0000","scrub_duration":0.0025091950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":95,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:50.002670+0000","last_change":"2024-09-11T07:28:09.024756+0000","last_active":"2024-09-11T07:28:50.002670+0000","last_peered":"2024-09-11T07:28:50.002670+0000","last_clean":"2024-09-11T07:28:50.002670+0000","last_became_active":"2024-09-11T07:28:09.017997+0000","last_became_peered":"2024-09-11T07:28:09.017997+0000","last_unstale":"2024-09-11T07:28:50.002670+0000","last_undegraded":"2024-09-11T07:28:50.002670+0000","last_fullsized":"2024-09-11T07:28:50.002670+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:28:03.164220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:03.164220+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:06.470841+0000","scrub_duration":0.0020677629999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.997295+0000","last_change":"2024-09-11T07:28:17.484549+0000","last_active":"2024-09-11T07:28:49.997295+0000","last_peered":"2024-09-11T07:28:49.997295+0000","last_clean":"2024-09-11T07:28:49.997295+0000","last_became_active":"2024-09-11T07:26:35.558809+0000","last_became_peered":"2024-09-11T07:26:35.558809+0000","last_unstale":"2024-09-11T07:28:49.997295+0000","last_undegraded":"2024-09-11T07:28:49.997295+0000","last_fullsized":"2024-09-11T07:28:49.997295+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:17.484488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:17.484488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.968070+0000","scrub_duration":0.001695118,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993666+0000","last_change":"2024-09-11T07:28:00.126756+0000","last_active":"2024-09-11T07:28:49.993666+0000","last_peered":"2024-09-11T07:28:49.993666+0000","last_clean":"2024-09-11T07:28:49.993666+0000","last_became_active":"2024-09-11T07:26:37.862373+0000","last_became_peered":"2024-09-11T07:26:37.862373+0000","last_unstale":"2024-09-11T07:28:49.993666+0000","last_undegraded":"2024-09-11T07:28:49.993666+0000","last_fullsized":"2024-09-11T07:28:49.993666+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:00.126702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.126702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:18.534592+0000","scrub_duration":0.001184903,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.998071+0000","last_change":"2024-09-11T07:27:33.983889+0000","last_active":"2024-09-11T07:28:49.998071+0000","last_peered":"2024-09-11T07:28:49.998071+0000","last_clean":"2024-09-11T07:28:49.998071+0000","last_became_active":"2024-09-11T07:26:31.424651+0000","last_became_peered":"2024-09-11T07:26:31.424651+0000","last_unstale":"2024-09-11T07:28:49.998071+0000","last_undegraded":"2024-09-11T07:28:49.998071+0000","last_fullsized":"2024-09-11T07:28:49.998071+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:33.983813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:33.983813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:28:55.264233+0000","scrub_duration":0.001290759,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":79,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.994070+0000","last_change":"2024-09-11T07:28:00.272702+0000","last_active":"2024-09-11T07:28:49.994070+0000","last_peered":"2024-09-11T07:28:49.994070+0000","last_clean":"2024-09-11T07:28:49.994070+0000","last_became_active":"2024-09-11T07:26:33.542544+0000","last_became_peered":"2024-09-11T07:26:33.542544+0000","last_unstale":"2024-09-11T07:28:49.994070+0000","last_undegraded":"2024-09-11T07:28:49.994070+0000","last_fullsized":"2024-09-11T07:28:49.994070+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-09-11T07:28:00.272628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.272628+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:05.374102+0000","scrub_duration":0.0021830019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993086+0000","last_change":"2024-09-11T07:28:07.721176+0000","last_active":"2024-09-11T07:28:49.993086+0000","last_peered":"2024-09-11T07:28:49.993086+0000","last_clean":"2024-09-11T07:28:49.993086+0000","last_became_active":"2024-09-11T07:26:35.555862+0000","last_became_peered":"2024-09-11T07:26:35.555862+0000","last_unstale":"2024-09-11T07:28:49.993086+0000","last_undegraded":"2024-09-11T07:28:49.993086+0000","last_fullsized":"2024-09-11T07:28:49.993086+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:07.721112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:07.721112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:31.345841+0000","scrub_duration":0.0020675020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.997187+0000","last_change":"2024-09-11T07:28:06.674616+0000","last_active":"2024-09-11T07:28:49.997187+0000","last_peered":"2024-09-11T07:28:49.997187+0000","last_clean":"2024-09-11T07:28:49.997187+0000","last_became_active":"2024-09-11T07:26:37.861647+0000","last_became_peered":"2024-09-11T07:26:37.861647+0000","last_unstale":"2024-09-11T07:28:49.997187+0000","last_undegraded":"2024-09-11T07:28:49.997187+0000","last_fullsized":"2024-09-11T07:28:49.997187+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:06.674558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:06.674558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:17.029204+0000","scrub_duration":0.001489488,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.997051+0000","last_change":"2024-09-11T07:27:45.732604+0000","last_active":"2024-09-11T07:28:49.997051+0000","last_peered":"2024-09-11T07:28:49.997051+0000","last_clean":"2024-09-11T07:28:49.997051+0000","last_became_active":"2024-09-11T07:26:31.420598+0000","last_became_peered":"2024-09-11T07:26:31.420598+0000","last_unstale":"2024-09-11T07:28:49.997051+0000","last_undegraded":"2024-09-11T07:28:49.997051+0000","last_fullsized":"2024-09-11T07:28:49.997051+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:45.732528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:45.732528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015466659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"64'12","reported_seq":77,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.996238+0000","last_change":"2024-09-11T07:27:59.737848+0000","last_active":"2024-09-11T07:28:49.996238+0000","last_peered":"2024-09-11T07:28:49.996238+0000","last_clean":"2024-09-11T07:28:49.996238+0000","last_became_active":"2024-09-11T07:26:33.542711+0000","last_became_peered":"2024-09-11T07:26:33.542711+0000","last_unstale":"2024-09-11T07:28:49.996238+0000","last_undegraded":"2024-09-11T07:28:49.996238+0000","last_fullsized":"2024-09-11T07:28:49.996238+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-09-11T07:27:59.737791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:59.737791+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:11.558595+0000","scrub_duration":0.00367732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-11T07:28:49.993580+0000","last_change":"2024-09-11T07:27:57.199239+0000","last_active":"2024-09-11T07:28:49.993580+0000","last_peered":"2024-09-11T07:28:49.993580+0000","last_clean":"2024-09-11T07:28:49.993580+0000","last_became_active":"2024-09-11T07:26:35.562037+0000","last_became_peered":"2024-09-11T07:26:35.562037+0000","last_unstale":"2024-09-11T07:28:49.993580+0000","last_undegraded":"2024-09-11T07:28:49.993580+0000","last_fullsized":"2024-09-11T07:28:49.993580+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:57.199173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.199173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.093487+0000","scrub_duration":0.0015950770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":444,"ondisk_log_size":444,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":136,"num_read_kb":131,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299425,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298676,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446476,"statfs":{"total":95995035648,"available":95689191424,"internally_reserved":0,"allocated":1343488,"data_stored":943573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.50900000000000001,"15min":0.50800000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52700000000000002,"15min":0.52800000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69399999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.65000000000000002}]},{"osd":1,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.628},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.34899999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.66900000000000004,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.58299999999999996}]},{"osd":2,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52400000000000002,"15min":0.52300000000000002},"min":{"1min":0.34599999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69199999999999995,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53000000000000003,"15min":0.52800000000000002},"min":{"1min":0.38700000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73999999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.56100000000000005}]},{"osd":3,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":1.0249999999999999,"15min":1.1080000000000001},"min":{"1min":0.34000000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70499999999999996,"5min":12.625999999999999,"15min":12.625999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.35899999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.72899999999999998,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.59699999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.44,"15min":0.442},"min":{"1min":0.29799999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.63100000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.442,"15min":0.44400000000000001},"min":{"1min":0.28999999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.42799999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.436,"15min":0.44},"min":{"1min":0.33000000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.56200000000000006,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.33100000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70599999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.54300000000000004}]},{"osd":6,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.34200000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67400000000000004,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.47899999999999998,"15min":0.47799999999999998},"min":{"1min":0.31,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70899999999999996,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.47399999999999998}]}]},{"osd":6,"up_from":44,"seq":188978561061,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298644,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446508,"statfs":{"total":95995035648,"available":95689224192,"internally_reserved":0,"allocated":1318912,"data_stored":942051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.374,"5min":0.247,"15min":0.247},"max":{"1min":0.65000000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.60499999999999998,"15min":0.624},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.754,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.621}]},{"osd":1,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.82899999999999996,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.49299999999999999,"15min":0.495},"min":{"1min":0.35799999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.54200000000000004}]},{"osd":2,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.61499999999999999,"15min":0.63500000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.70299999999999996,"5min":3.234,"15min":3.234},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.496,"5min":0.59099999999999997,"15min":0.60699999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.80200000000000005,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.60399999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51500000000000001,"15min":0.51800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.503},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.63500000000000001,"15min":0.65500000000000003},"min":{"1min":0.41199999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.66900000000000004,"5min":3.1739999999999999,"15min":3.1739999999999999},"last":0.70699999999999996}]},{"osd":4,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.58299999999999996,"15min":0.60899999999999999},"min":{"1min":0.32700000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.65200000000000002,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.28399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.63600000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.65600000000000003,"15min":0.69699999999999995},"min":{"1min":0.32400000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64000000000000001,"5min":5.7329999999999997,"15min":5.7329999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.52000000000000002,"15min":0.52700000000000002},"min":{"1min":0.34000000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70899999999999996,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.69399999999999995}]},{"osd":7,"last update":"Wed Sep 11 07:28:23 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.58499999999999996,"15min":0.60499999999999998},"min":{"1min":0.34999999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.66300000000000003,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.495,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.34300000000000003,"5min":0.313,"15min":0.313},"max":{"1min":0.67300000000000004,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.71999999999999997}]}]},{"osd":5,"up_from":37,"seq":158913789995,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298072,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447080,"statfs":{"total":95995035648,"available":95689809920,"internally_reserved":0,"allocated":733184,"data_stored":352674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:28:35 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.505,"15min":0.51100000000000001},"min":{"1min":0.317,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70599999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.5,"15min":0.5},"min":{"1min":0.34300000000000003,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.65700000000000003,"5min":1.107,"15min":1.107},"last":0.50800000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:28:35 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.52100000000000002,"15min":0.53000000000000003},"min":{"1min":0.316,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67000000000000004,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.378},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.49399999999999999,"15min":0.49299999999999999},"min":{"1min":0.29599999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.64800000000000002,"5min":1.073,"15min":1.073},"last":0.41699999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:28:35 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.373,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.67600000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.53200000000000003,"15min":0.53700000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64700000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.52500000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:28:35 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.33000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.69899999999999995,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.498},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.377,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73699999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.41099999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:28:35 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.378,"15min":0.371},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.495,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.42099999999999999,"15min":0.41799999999999998},"min":{"1min":0.26900000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51200000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.39500000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:28:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.41599999999999998,"15min":0.40600000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.438},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.42699999999999999,"15min":0.42199999999999999},"min":{"1min":0.32100000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.48399999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:28:21 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.48099999999999998,"15min":0.48899999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53800000000000003,"5min":1.202,"15min":1.202},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.50700000000000001,"15min":0.51300000000000001},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.61799999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.47099999999999997}]}]},{"osd":4,"up_from":31,"seq":133143986223,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298096,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447056,"statfs":{"total":95995035648,"available":95689785344,"internally_reserved":0,"allocated":753664,"data_stored":353009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:28:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52700000000000002,"15min":0.51700000000000002},"min":{"1min":0.42699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.499},"min":{"1min":0.34999999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.56000000000000005}]},{"osd":1,"last update":"Wed Sep 11 07:28:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.50700000000000001,"15min":0.496},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72199999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.442},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.67100000000000004,"15min":0.72299999999999998},"min":{"1min":0.38100000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.81499999999999995,"5min":4.5510000000000002,"15min":4.5510000000000002},"last":0.57999999999999996}]},{"osd":2,"last update":"Wed Sep 11 07:28:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53600000000000003,"15min":0.53000000000000003},"min":{"1min":0.40200000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.73499999999999999,"5min":1.21,"15min":1.21},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.36899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76800000000000002,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.40200000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:28:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.38,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.70099999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.499},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.55100000000000005}]},{"osd":5,"last update":"Wed Sep 11 07:28:32 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.36399999999999999,"15min":0.34399999999999997},"min":{"1min":0.28699999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.377},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.39500000000000002,"15min":0.377},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.42799999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:27:59 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42199999999999999,"15min":0.41799999999999998},"min":{"1min":0.315,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.44,"15min":0.43099999999999999},"min":{"1min":0.33500000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.47299999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48999999999999999,"15min":0.48699999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48099999999999998,"15min":0.48299999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.45700000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182452,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298104,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447048,"statfs":{"total":95995035648,"available":95689777152,"internally_reserved":0,"allocated":761856,"data_stored":352473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:27:54 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38600000000000001,"15min":0.39300000000000002},"min":{"1min":0.20799999999999999,"5min":0.185,"15min":0.185},"max":{"1min":0.54300000000000004,"5min":0.621,"15min":0.621},"last":0.245},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.42499999999999999,"15min":0.441},"min":{"1min":0.26200000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.33100000000000002}]},{"osd":1,"last update":"Wed Sep 11 07:27:54 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.43099999999999999,"15min":0.437},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.625,"5min":0.752,"15min":0.752},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.41599999999999998,"15min":0.43099999999999999},"min":{"1min":0.23599999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.28299999999999997}]},{"osd":2,"last update":"Wed Sep 11 07:27:54 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.45100000000000001,"15min":0.46500000000000002},"min":{"1min":0.26900000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.45400000000000001,"15min":0.46899999999999997},"min":{"1min":0.28899999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.38}]},{"osd":4,"last update":"Wed Sep 11 07:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.747,"15min":0.621},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77200000000000002,"5min":21.856000000000002,"15min":21.856000000000002},"last":0.49399999999999999},{"interface":"front","average":{"1min":1.4399999999999999,"5min":0.89700000000000002,"15min":0.67700000000000005},"min":{"1min":0.436,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":18.370000000000001,"5min":19.146000000000001,"15min":19.146000000000001},"last":0.42699999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:28:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.42599999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.71799999999999997,"15min":0.622},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76800000000000002,"5min":18.416,"15min":18.416},"last":0.47699999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:27:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":1.306,"15min":1.4330000000000001},"min":{"1min":0.439,"5min":0.437,"15min":0.437},"max":{"1min":0.72899999999999998,"5min":20.219999999999999,"15min":20.219999999999999},"last":0.437},{"interface":"front","average":{"1min":0.55500000000000005,"5min":1.319,"15min":1.446},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72699999999999998,"5min":19.609000000000002,"15min":19.609000000000002},"last":0.51700000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.45900000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.80700000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59799999999999998,"15min":0.60399999999999998},"min":{"1min":0.39700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71899999999999997,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.45600000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411385,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298068,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447084,"statfs":{"total":95995035648,"available":95689814016,"internally_reserved":0,"allocated":729088,"data_stored":351920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:28:28 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.375,"15min":0.36299999999999999},"min":{"1min":0.20799999999999999,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.56299999999999994,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.371,"5min":0.38200000000000001,"15min":0.374},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57599999999999996,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.35699999999999998}]},{"osd":1,"last update":"Wed Sep 11 07:28:28 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.39800000000000002,"15min":0.38400000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.499,"5min":1.179,"15min":1.179},"last":0.433},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.41099999999999998,"15min":0.38600000000000001},"min":{"1min":0.255,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.66600000000000004,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.41599999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:27:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.439,"15min":0.42299999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64500000000000002,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.45900000000000002,"15min":0.45400000000000001},"min":{"1min":0.26600000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65000000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.49099999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.94799999999999995,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.622},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.59299999999999997}]},{"osd":5,"last update":"Wed Sep 11 07:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.66600000000000004,"15min":0.68999999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90400000000000003,"5min":2.931,"15min":2.931},"last":0.71999999999999997}]},{"osd":6,"last update":"Wed Sep 11 07:27:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.85299999999999998,"5min":1.494,"15min":1.494},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.63100000000000001,"15min":0.63300000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.875,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.69499999999999995}]},{"osd":7,"last update":"Wed Sep 11 07:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.64700000000000002,"15min":0.65700000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.92700000000000005,"5min":1.514,"15min":1.514},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.78300000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298116,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447036,"statfs":{"total":95995035648,"available":95689764864,"internally_reserved":0,"allocated":770048,"data_stored":353110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:28:04 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.34000000000000002,"15min":0.33100000000000002},"min":{"1min":0.22800000000000001,"5min":0.186,"15min":0.186},"max":{"1min":0.49299999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.36499999999999999,"15min":0.35499999999999998},"min":{"1min":0.22700000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.44400000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:28:31 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.41099999999999998,"15min":0.41299999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.58799999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.26300000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.56699999999999995,"5min":1.075,"15min":1.075},"last":0.49299999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:27:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.44500000000000001,"15min":0.44700000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.60699999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.41399999999999998},"min":{"1min":0.307,"5min":0.253,"15min":0.253},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.55700000000000005}]},{"osd":4,"last update":"Wed Sep 11 07:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54200000000000004,"15min":0.54700000000000004},"min":{"1min":0.34999999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71999999999999997,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.63100000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:28:36 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72399999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.621},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.68899999999999995,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.66600000000000004}]},{"osd":6,"last update":"Wed Sep 11 07:27:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57199999999999995,"15min":0.57499999999999996},"min":{"1min":0.41199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.754,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.40999999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.70899999999999996}]},{"osd":7,"last update":"Wed Sep 11 07:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.746},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.75800000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738434,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298668,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446484,"statfs":{"total":95995035648,"available":95689199616,"internally_reserved":0,"allocated":1335296,"data_stored":942383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:28:02 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.36699999999999999,"15min":0.34499999999999997},"min":{"1min":0.246,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.35899999999999999,"15min":0.34200000000000003},"min":{"1min":0.23799999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.40999999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:28:29 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.39400000000000002,"15min":0.40400000000000003},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.56799999999999995,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.316},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.39900000000000002,"15min":0.40300000000000002},"min":{"1min":0.255,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.54400000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.39700000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:27:55 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45800000000000002,"15min":0.47399999999999998},"min":{"1min":0.317,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.61199999999999999,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.46300000000000002,"15min":0.48099999999999998},"min":{"1min":0.28299999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.56200000000000006,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.42699999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:28:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56000000000000005,"15min":0.56799999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.77600000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.67200000000000004}]},{"osd":5,"last update":"Wed Sep 11 07:28:31 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.71099999999999997,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59999999999999998,"15min":0.61199999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.63700000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:27:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59399999999999997,"15min":0.59699999999999998},"min":{"1min":0.34799999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60099999999999998,"15min":0.60399999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.76400000000000001,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.56200000000000006}]},{"osd":7,"last update":"Wed Sep 11 07:28:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58499999999999996,"15min":0.59299999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.748,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.625},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.42799999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.52400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:28:57.675 INFO:tasks.thrashosds.thrasher:Setting 2.1c to [5, 2, 7] 2024-09-11T07:28:57.675 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.1c', '5', '2', '7'] 2024-09-11T07:28:57.675 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pg-upmap 2.1c 5 2 7 2024-09-11T07:28:57.676 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:28:57.698 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:57.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:57.698 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:28:57.799 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:28:57.825 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:57.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:57.826 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:28:57.927 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:28:57.951 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:57.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:57.952 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:28:58.053 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:28:58.074 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:58.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:58.075 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:28:58.176 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:28:58.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:58.199 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:58.199 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:28:58.300 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:28:58.326 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:58.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:58.327 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:28:58.427 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:28:58.453 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:58.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:58.454 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:28:58.555 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:28:58.577 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:58.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:58.578 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:28:58.679 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:28:58.701 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:58.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:58.702 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:28:58.803 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:28:58.825 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:58.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:58.826 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:28:58.927 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:28:58.949 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:58.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:58.950 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:28:58.985 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:28:58.986 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:28:59.010 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:28:59 smithi022 ceph-mon[37330]: pgmap v81: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 642 B/s rd, 0 op/s 2024-09-11T07:28:59.011 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:28:59 smithi022 ceph-mon[29462]: pgmap v81: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 642 B/s rd, 0 op/s 2024-09-11T07:28:59.051 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:28:59.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:59.075 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:59.075 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:28:59.176 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:28:59.202 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:59.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:59.203 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:28:59.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:28:59 smithi104 ceph-mon[33507]: pgmap v81: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 642 B/s rd, 0 op/s 2024-09-11T07:28:59.280 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:28:59 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:28:59.303 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:28:59.329 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:59.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:59.330 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:28:59.430 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:28:59.456 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:59.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:59.457 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:28:59.558 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:28:59.569 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:28:59.569 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:28:59.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:59.586 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:59.586 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:28:59.587 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:28:59.587 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:28:59.601 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:28:59.601 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:28:59.616 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:28:59.617 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:28:59.633 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:28:59.633 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:28:59.648 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:28:59.648 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:28:59.664 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:28:59.664 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:28:59.681 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:28:59.681 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:28:59.688 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:28:59.712 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:59.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:59.713 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:28:59.814 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:28:59.836 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:28:59.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:59.837 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:28:59.938 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:28:59.963 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:28:59.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:28:59.964 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:00.022 INFO:teuthology.orchestra.run.smithi022.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-09-11T07:29:00.065 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:00.090 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:00.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:00.091 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:00.192 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:00.217 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:00.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:00.219 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:00.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:00 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:29:00.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:00 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3715475624' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:29:00.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:00 smithi104 ceph-mon[33507]: from='client.24746 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:29:00.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:00 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2452097986' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:29:00.319 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:00.340 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:00 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:29:00.341 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:00 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3715475624' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:29:00.341 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:00 smithi022 ceph-mon[29462]: from='client.24746 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:29:00.341 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:00 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2452097986' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:29:00.341 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:00 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:29:00.341 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:00 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3715475624' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:29:00.341 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:00 smithi022 ceph-mon[37330]: from='client.24746 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:29:00.341 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:00 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2452097986' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:29:00.343 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:00.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:00.344 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:00.445 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:00.468 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:00.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:00.469 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:00.570 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:00.591 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:00.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:00.592 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:00.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:29:00 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:29:00] "GET /metrics HTTP/1.1" 200 222524 "" "Prometheus/2.43.0" 2024-09-11T07:29:00.693 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:00.720 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:00.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:00.721 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:00.822 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:00.847 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:00.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:00.848 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:00.949 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:00.972 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:00.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:00.973 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:01.059 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:29:01.074 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:01.098 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:01.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:01.099 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:01.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:01 smithi022 ceph-mon[29462]: pgmap v82: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:29:01.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:01 smithi022 ceph-mon[29462]: from='client.24746 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:29:01.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:01 smithi022 ceph-mon[29462]: osdmap e96: 8 total, 8 up, 8 in 2024-09-11T07:29:01.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:01 smithi022 ceph-mon[37330]: pgmap v82: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:29:01.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:01 smithi022 ceph-mon[37330]: from='client.24746 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:29:01.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:01 smithi022 ceph-mon[37330]: osdmap e96: 8 total, 8 up, 8 in 2024-09-11T07:29:01.200 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:01.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:01.227 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:01.227 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:01.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:01 smithi104 ceph-mon[33507]: pgmap v82: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:29:01.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:01 smithi104 ceph-mon[33507]: from='client.24746 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:29:01.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:01 smithi104 ceph-mon[33507]: osdmap e96: 8 total, 8 up, 8 in 2024-09-11T07:29:01.328 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:01.351 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:01.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:01.352 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:01.452 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:01.478 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:01.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:01.479 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:01.579 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:01.605 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:01.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:01.606 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:01.707 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:01.730 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:01.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:01.731 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:01.832 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:01.857 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:01.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:01.858 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:01.959 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:01.983 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:01.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:01.984 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:02.085 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:02.108 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:02.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:02.109 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:02.143 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:02 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:29:02.143 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:02 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:29:02.143 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:02 smithi022 ceph-mon[29462]: osdmap e97: 8 total, 8 up, 8 in 2024-09-11T07:29:02.143 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:02 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:29:02.143 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:02 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:29:02.143 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:02 smithi022 ceph-mon[37330]: osdmap e97: 8 total, 8 up, 8 in 2024-09-11T07:29:02.210 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:02.232 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:02.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:02.233 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:02.334 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:02.356 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:02.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:02.357 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:02.457 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:02.479 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:02.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:02.480 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:02.501 DEBUG:teuthology.orchestra.run.smithi022:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_0 bench 90 write' 2024-09-11T07:29:02.503 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-11T07:29:02.503 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-11T07:29:02.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:02 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:29:02.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:02 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:29:02.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:02 smithi104 ceph-mon[33507]: osdmap e97: 8 total, 8 up, 8 in 2024-09-11T07:29:02.580 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:hints = 1 2024-09-11T07:29:02.580 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-11T07:29:02.580 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Object prefix: benchmark_data_smithi022_77720 2024-09-11T07:29:02.580 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T07:29:02.581 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 0 0 0 0 0 0 - 0 2024-09-11T07:29:02.581 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:02.615 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:02.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:02.616 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:02.717 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:02.757 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:02.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:02.758 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:02.859 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:02.860 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:29:02.889 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:02.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:02.890 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:02.991 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:03.020 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:03.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:03.021 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:03.122 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:03.152 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:03.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:03.153 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:03.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:03 smithi022 ceph-mon[29462]: pgmap v85: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 736 B/s rd, 0 op/s 2024-09-11T07:29:03.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:03 smithi022 ceph-mon[29462]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:29:03.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:03 smithi022 ceph-mon[37330]: pgmap v85: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 736 B/s rd, 0 op/s 2024-09-11T07:29:03.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:03 smithi022 ceph-mon[37330]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:29:03.254 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:03.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:03 smithi104 ceph-mon[33507]: pgmap v85: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 736 B/s rd, 0 op/s 2024-09-11T07:29:03.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:03 smithi104 ceph-mon[33507]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:29:03.281 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:03.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:03.282 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:03.383 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:03.423 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:03.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:03.425 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:03.525 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:03.568 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:03.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:03.569 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:03.580 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 1 15 3491 3476 217.238 217.25 0.00417568 0.00458758 2024-09-11T07:29:03.670 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:03.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:03.712 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:03.813 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:03.858 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:03.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:03.859 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:03.960 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:04.001 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:04.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:04.002 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:04.103 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:04.125 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:04.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:04.126 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:04.227 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:04.255 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:04.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:04.256 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:04.357 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:04.384 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:04.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:04.385 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:04.486 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:04.514 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:04.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:04.515 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:04.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:29:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:29:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:29:04.580 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 2 15 6652 6637 207.388 197.562 0.00459273 0.00481276 2024-09-11T07:29:04.616 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:04.645 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:04.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:04.647 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:04.748 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:04.790 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:29:04.803 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:04.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:04.805 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:04.829 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:29:04.829 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 20, 2024-09-11T07:29:04.829 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 600, 2024-09-11T07:29:04.830 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:29:04.830 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.830 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:1264 7.6 7:6bf1c116:::benchmark_data_smithi022_77720_object1263:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-11T07:29:04.830 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:02.930437+0000", 2024-09-11T07:29:04.830 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 1.892484139, 2024-09-11T07:29:04.830 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.011331859, 2024-09-11T07:29:04.830 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:29:04.830 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:29:04.830 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:29:04.830 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:29:04.831 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:29:04.831 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 1264 2024-09-11T07:29:04.831 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.831 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:29:04.831 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.831 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:29:04.831 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.930437+0000", 2024-09-11T07:29:04.831 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.832 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.832 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.832 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:29:04.832 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.930437+0000", 2024-09-11T07:29:04.832 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.832 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.832 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.832 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:29:04.832 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.930435+0000", 2024-09-11T07:29:04.832 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:29:04.833 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.833 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.833 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:29:04.833 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.930482+0000", 2024-09-11T07:29:04.833 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6612e-05 2024-09-11T07:29:04.833 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.833 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.833 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:29:04.834 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.930494+0000", 2024-09-11T07:29:04.834 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.24e-05 2024-09-11T07:29:04.834 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.834 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.834 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:29:04.834 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.930501+0000", 2024-09-11T07:29:04.835 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.8539999999999999e-06 2024-09-11T07:29:04.835 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.835 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.835 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:29:04.835 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.930840+0000", 2024-09-11T07:29:04.835 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000338835 2024-09-11T07:29:04.835 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.835 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.835 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:29:04.835 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.931262+0000", 2024-09-11T07:29:04.836 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00042225500000000002 2024-09-11T07:29:04.836 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.836 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.836 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:29:04.836 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.931417+0000", 2024-09-11T07:29:04.836 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015505299999999999 2024-09-11T07:29:04.836 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.836 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.836 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.837 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.933425+0000", 2024-09-11T07:29:04.837 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0020082049999999999 2024-09-11T07:29:04.837 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.837 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.837 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:29:04.837 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.933451+0000", 2024-09-11T07:29:04.837 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5718e-05 2024-09-11T07:29:04.837 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.837 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.838 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.838 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.941677+0000", 2024-09-11T07:29:04.838 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0082258539999999995 2024-09-11T07:29:04.838 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.838 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.838 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:29:04.838 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.941732+0000", 2024-09-11T07:29:04.838 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.5770000000000003e-05 2024-09-11T07:29:04.838 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.839 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.839 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:29:04.839 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.941768+0000", 2024-09-11T07:29:04.839 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5958999999999999e-05 2024-09-11T07:29:04.839 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.839 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:04.839 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.839 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.839 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.839 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:1265 7.6 7:68053e8b:::benchmark_data_smithi022_77720_object1264:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-11T07:29:04.840 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:02.930515+0000", 2024-09-11T07:29:04.840 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 1.8924058239999999, 2024-09-11T07:29:04.840 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.011423996, 2024-09-11T07:29:04.840 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:29:04.840 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:29:04.840 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:29:04.840 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:29:04.840 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:29:04.840 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 1265 2024-09-11T07:29:04.840 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.841 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:29:04.841 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.841 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:29:04.841 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.930515+0000", 2024-09-11T07:29:04.841 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.841 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.841 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.842 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:29:04.842 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.930515+0000", 2024-09-11T07:29:04.842 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.842 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.842 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.842 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:29:04.842 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.930514+0000", 2024-09-11T07:29:04.843 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:29:04.843 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.843 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.843 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:29:04.843 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.930544+0000", 2024-09-11T07:29:04.843 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0247999999999999e-05 2024-09-11T07:29:04.843 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.844 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.844 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:29:04.844 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.930553+0000", 2024-09-11T07:29:04.844 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.2800000000000003e-06 2024-09-11T07:29:04.844 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.844 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.845 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:29:04.845 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.930557+0000", 2024-09-11T07:29:04.845 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8210000000000001e-06 2024-09-11T07:29:04.845 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.845 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.846 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:29:04.846 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.932042+0000", 2024-09-11T07:29:04.846 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0014843040000000001 2024-09-11T07:29:04.846 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.846 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.847 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:29:04.847 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.932137+0000", 2024-09-11T07:29:04.847 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.5600000000000006e-05 2024-09-11T07:29:04.847 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.847 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.847 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:29:04.848 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.932261+0000", 2024-09-11T07:29:04.848 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000124017 2024-09-11T07:29:04.848 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.848 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.848 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.848 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.933708+0000", 2024-09-11T07:29:04.849 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001447018 2024-09-11T07:29:04.849 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.849 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.849 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:29:04.849 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.933820+0000", 2024-09-11T07:29:04.850 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000111803 2024-09-11T07:29:04.850 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.850 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.850 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.850 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.941843+0000", 2024-09-11T07:29:04.850 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0080227839999999998 2024-09-11T07:29:04.850 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.851 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.851 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:29:04.851 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.941892+0000", 2024-09-11T07:29:04.851 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9064000000000003e-05 2024-09-11T07:29:04.851 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.851 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.852 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:29:04.852 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:02.941939+0000", 2024-09-11T07:29:04.852 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6828e-05 2024-09-11T07:29:04.852 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.852 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:04.852 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.853 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.853 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.853 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:2203 7.6 7:6721f2c7:::benchmark_data_smithi022_77720_object2202:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-11T07:29:04.853 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:03.204562+0000", 2024-09-11T07:29:04.853 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 1.6183582300000001, 2024-09-11T07:29:04.853 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.010682646000000001, 2024-09-11T07:29:04.853 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:29:04.854 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:29:04.854 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:29:04.854 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:29:04.854 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:29:04.854 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 2203 2024-09-11T07:29:04.854 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.855 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:29:04.855 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.855 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:29:04.855 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:03.204562+0000", 2024-09-11T07:29:04.855 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.855 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.855 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.855 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:29:04.856 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:03.204562+0000", 2024-09-11T07:29:04.856 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.856 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.856 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.856 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:29:04.856 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:03.204561+0000", 2024-09-11T07:29:04.856 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:29:04.857 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.857 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.857 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:29:04.857 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:03.204586+0000", 2024-09-11T07:29:04.857 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.421e-05 2024-09-11T07:29:04.857 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.858 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.858 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:29:04.858 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:03.204600+0000", 2024-09-11T07:29:04.858 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3900000000000001e-05 2024-09-11T07:29:04.858 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.858 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.858 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:29:04.859 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:03.204605+0000", 2024-09-11T07:29:04.859 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9089999999999997e-06 2024-09-11T07:29:04.859 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.859 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.859 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:29:04.859 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:03.206533+0000", 2024-09-11T07:29:04.860 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0019270520000000001 2024-09-11T07:29:04.860 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.860 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.860 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:29:04.860 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:03.206969+0000", 2024-09-11T07:29:04.860 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00043637299999999999 2024-09-11T07:29:04.861 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.861 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.861 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:29:04.861 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:03.207101+0000", 2024-09-11T07:29:04.861 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013212 2024-09-11T07:29:04.861 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.862 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.862 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.862 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:03.208561+0000", 2024-09-11T07:29:04.862 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001459583 2024-09-11T07:29:04.862 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.862 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.863 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.863 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:03.211329+0000", 2024-09-11T07:29:04.863 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0027679689999999999 2024-09-11T07:29:04.863 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.863 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.864 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:29:04.864 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:03.215141+0000", 2024-09-11T07:29:04.864 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0038126739999999998 2024-09-11T07:29:04.864 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.864 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.864 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:29:04.865 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:03.215209+0000", 2024-09-11T07:29:04.865 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7780999999999994e-05 2024-09-11T07:29:04.865 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.865 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.865 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:29:04.865 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:03.215245+0000", 2024-09-11T07:29:04.866 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6115999999999998e-05 2024-09-11T07:29:04.866 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.866 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:04.866 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.866 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.866 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.866 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:5124 7.9 7:9151c680:::benchmark_data_smithi022_77720_object5123:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-11T07:29:04.867 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:04.078295+0000", 2024-09-11T07:29:04.867 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.74462602499999997, 2024-09-11T07:29:04.867 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.022151715999999998, 2024-09-11T07:29:04.867 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:29:04.867 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:29:04.867 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:29:04.867 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:29:04.868 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:29:04.868 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 5124 2024-09-11T07:29:04.868 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.868 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:29:04.868 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.868 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:29:04.868 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.078295+0000", 2024-09-11T07:29:04.869 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.869 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.869 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.869 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:29:04.869 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.078295+0000", 2024-09-11T07:29:04.869 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.869 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.869 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.869 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:29:04.869 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.078294+0000", 2024-09-11T07:29:04.869 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:29:04.869 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.869 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.870 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:29:04.870 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.078325+0000", 2024-09-11T07:29:04.870 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.1609000000000002e-05 2024-09-11T07:29:04.870 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.870 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.870 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:29:04.870 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.078335+0000", 2024-09-11T07:29:04.870 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.1949999999999992e-06 2024-09-11T07:29:04.870 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.870 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.870 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:29:04.870 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.078339+0000", 2024-09-11T07:29:04.870 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5879999999999999e-06 2024-09-11T07:29:04.871 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.871 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.871 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:29:04.871 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.079921+0000", 2024-09-11T07:29:04.871 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001581341 2024-09-11T07:29:04.871 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.871 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.871 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:29:04.871 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.080032+0000", 2024-09-11T07:29:04.871 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000111706 2024-09-11T07:29:04.872 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.872 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.872 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,4", 2024-09-11T07:29:04.872 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.080172+0000", 2024-09-11T07:29:04.872 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013991000000000001 2024-09-11T07:29:04.872 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.872 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.872 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:29:04.872 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.081271+0000", 2024-09-11T07:29:04.872 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0010987010000000001 2024-09-11T07:29:04.873 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.873 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.873 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.873 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.083293+0000", 2024-09-11T07:29:04.873 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0020217049999999999 2024-09-11T07:29:04.873 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.873 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.873 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.873 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.100198+0000", 2024-09-11T07:29:04.873 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.016905818999999999 2024-09-11T07:29:04.873 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.873 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.873 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:29:04.874 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.100319+0000", 2024-09-11T07:29:04.874 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012050299999999999 2024-09-11T07:29:04.874 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.874 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.874 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:29:04.874 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.100446+0000", 2024-09-11T07:29:04.874 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012747400000000001 2024-09-11T07:29:04.874 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.874 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:04.874 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.875 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.875 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.875 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:5130 7.9 7:98394baf:::benchmark_data_smithi022_77720_object5129:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-11T07:29:04.875 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:04.079929+0000", 2024-09-11T07:29:04.875 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.74299126699999996, 2024-09-11T07:29:04.875 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.020744879000000001, 2024-09-11T07:29:04.875 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:29:04.875 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:29:04.875 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:29:04.875 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:29:04.875 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:29:04.876 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 5130 2024-09-11T07:29:04.876 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.876 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:29:04.876 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.876 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:29:04.876 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.079929+0000", 2024-09-11T07:29:04.876 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.876 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.876 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.876 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:29:04.876 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.079929+0000", 2024-09-11T07:29:04.876 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.877 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.877 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.877 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:29:04.877 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.079928+0000", 2024-09-11T07:29:04.877 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:29:04.878 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.878 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.878 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:29:04.878 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.079955+0000", 2024-09-11T07:29:04.878 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.6373e-05 2024-09-11T07:29:04.878 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.878 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.879 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:29:04.879 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.079965+0000", 2024-09-11T07:29:04.879 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0254e-05 2024-09-11T07:29:04.879 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.879 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.879 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:29:04.879 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.079980+0000", 2024-09-11T07:29:04.879 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4810000000000001e-05 2024-09-11T07:29:04.879 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.879 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.879 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:29:04.879 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.080532+0000", 2024-09-11T07:29:04.880 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00055281500000000001 2024-09-11T07:29:04.880 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.880 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.880 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:29:04.880 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.080649+0000", 2024-09-11T07:29:04.880 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000116334 2024-09-11T07:29:04.880 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.880 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.880 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,4", 2024-09-11T07:29:04.880 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.080804+0000", 2024-09-11T07:29:04.880 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015494700000000001 2024-09-11T07:29:04.880 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.881 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.881 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:29:04.881 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.081638+0000", 2024-09-11T07:29:04.881 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00083429800000000003 2024-09-11T07:29:04.881 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.881 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.881 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.881 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.083612+0000", 2024-09-11T07:29:04.881 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0019742700000000002 2024-09-11T07:29:04.881 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.881 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.881 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.882 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.100598+0000", 2024-09-11T07:29:04.882 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.016985495999999999 2024-09-11T07:29:04.882 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.882 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.882 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:29:04.882 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.100642+0000", 2024-09-11T07:29:04.882 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4004000000000001e-05 2024-09-11T07:29:04.882 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.882 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.882 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:29:04.882 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.100674+0000", 2024-09-11T07:29:04.882 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2579000000000001e-05 2024-09-11T07:29:04.883 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.883 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:04.883 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.883 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.883 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.883 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:5146 7.4 7:2220a679:::benchmark_data_smithi022_77720_object5145:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-11T07:29:04.883 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:04.086972+0000", 2024-09-11T07:29:04.883 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.73594827799999996, 2024-09-11T07:29:04.883 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.013624261, 2024-09-11T07:29:04.883 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:29:04.883 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:29:04.883 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:29:04.884 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:29:04.884 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:29:04.884 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 5146 2024-09-11T07:29:04.884 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.884 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:29:04.884 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.884 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:29:04.884 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.086972+0000", 2024-09-11T07:29:04.884 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.884 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.884 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.884 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:29:04.884 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.086972+0000", 2024-09-11T07:29:04.885 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.885 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.885 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.885 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:29:04.885 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.086971+0000", 2024-09-11T07:29:04.885 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:29:04.885 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.885 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.885 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:29:04.885 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.087010+0000", 2024-09-11T07:29:04.885 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8931999999999999e-05 2024-09-11T07:29:04.885 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.886 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.886 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:29:04.886 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.087024+0000", 2024-09-11T07:29:04.886 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4129e-05 2024-09-11T07:29:04.886 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.886 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.886 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:29:04.886 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.087030+0000", 2024-09-11T07:29:04.886 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3380000000000001e-06 2024-09-11T07:29:04.886 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.886 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.886 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:29:04.887 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.087082+0000", 2024-09-11T07:29:04.887 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1320999999999999e-05 2024-09-11T07:29:04.887 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.887 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.887 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:29:04.887 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.087148+0000", 2024-09-11T07:29:04.887 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.6377000000000005e-05 2024-09-11T07:29:04.887 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.887 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.887 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,4", 2024-09-11T07:29:04.887 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.087222+0000", 2024-09-11T07:29:04.888 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.4132000000000001e-05 2024-09-11T07:29:04.888 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.888 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.888 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:29:04.888 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.087950+0000", 2024-09-11T07:29:04.888 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00072793100000000002 2024-09-11T07:29:04.888 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.888 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.888 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.888 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.088083+0000", 2024-09-11T07:29:04.888 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000133282 2024-09-11T07:29:04.889 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.889 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.889 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.889 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.100460+0000", 2024-09-11T07:29:04.889 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.012376254 2024-09-11T07:29:04.889 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.889 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.889 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:29:04.889 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.100511+0000", 2024-09-11T07:29:04.889 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1297e-05 2024-09-11T07:29:04.889 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.889 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.890 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:29:04.890 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.100597+0000", 2024-09-11T07:29:04.890 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.5779999999999995e-05 2024-09-11T07:29:04.890 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.890 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:04.890 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.890 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.890 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.890 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:5158 7.f 7:fd9de218:::benchmark_data_smithi022_77720_object5157:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-11T07:29:04.890 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:04.101897+0000", 2024-09-11T07:29:04.890 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.72102350900000001, 2024-09-11T07:29:04.890 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.019700453999999999, 2024-09-11T07:29:04.891 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:29:04.891 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:29:04.891 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:29:04.891 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:29:04.891 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:29:04.891 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 5158 2024-09-11T07:29:04.891 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.891 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:29:04.891 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.891 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:29:04.891 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.101897+0000", 2024-09-11T07:29:04.892 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.892 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.892 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.892 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:29:04.892 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.101897+0000", 2024-09-11T07:29:04.892 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.892 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.892 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.892 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:29:04.892 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.101895+0000", 2024-09-11T07:29:04.892 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:29:04.893 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.893 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.893 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:29:04.893 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.101919+0000", 2024-09-11T07:29:04.893 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4090000000000001e-05 2024-09-11T07:29:04.893 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.893 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.893 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:29:04.893 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.102826+0000", 2024-09-11T07:29:04.893 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00090702299999999997 2024-09-11T07:29:04.893 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.893 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.894 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:29:04.894 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.102836+0000", 2024-09-11T07:29:04.894 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.3870000000000001e-06 2024-09-11T07:29:04.894 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.894 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.894 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:29:04.894 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.102882+0000", 2024-09-11T07:29:04.894 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6625000000000002e-05 2024-09-11T07:29:04.894 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.894 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.894 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:29:04.894 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.103001+0000", 2024-09-11T07:29:04.895 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000118532 2024-09-11T07:29:04.895 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.895 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.895 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:29:04.895 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.103155+0000", 2024-09-11T07:29:04.895 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015387300000000001 2024-09-11T07:29:04.895 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.895 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.895 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:29:04.895 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.103808+0000", 2024-09-11T07:29:04.895 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00065371500000000003 2024-09-11T07:29:04.895 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.895 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.896 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.896 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.105335+0000", 2024-09-11T07:29:04.896 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0015271690000000001 2024-09-11T07:29:04.896 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.896 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.896 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.896 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.121510+0000", 2024-09-11T07:29:04.896 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.016175017 2024-09-11T07:29:04.896 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.896 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.896 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:29:04.896 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.121562+0000", 2024-09-11T07:29:04.896 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1094999999999998e-05 2024-09-11T07:29:04.897 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.897 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.897 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:29:04.897 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.121598+0000", 2024-09-11T07:29:04.897 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6069000000000003e-05 2024-09-11T07:29:04.897 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.897 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:04.897 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.897 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.897 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.897 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:5165 7.f 7:fe9ae08e:::benchmark_data_smithi022_77720_object5164:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-11T07:29:04.897 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:04.104608+0000", 2024-09-11T07:29:04.897 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.71831246100000001, 2024-09-11T07:29:04.898 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.017235292999999999, 2024-09-11T07:29:04.898 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:29:04.898 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:29:04.898 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:29:04.898 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:29:04.898 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:29:04.898 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 5165 2024-09-11T07:29:04.898 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.898 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:29:04.898 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.898 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:29:04.898 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.104608+0000", 2024-09-11T07:29:04.898 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.899 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.899 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.899 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:29:04.899 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.104608+0000", 2024-09-11T07:29:04.899 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.899 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.899 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.899 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:29:04.899 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.104606+0000", 2024-09-11T07:29:04.899 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:29:04.899 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.899 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.899 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:29:04.900 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.104636+0000", 2024-09-11T07:29:04.900 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0326000000000001e-05 2024-09-11T07:29:04.900 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.900 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.900 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:29:04.900 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.104647+0000", 2024-09-11T07:29:04.900 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0379e-05 2024-09-11T07:29:04.900 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.900 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.900 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:29:04.900 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.104654+0000", 2024-09-11T07:29:04.900 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9079999999999998e-06 2024-09-11T07:29:04.900 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.901 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.901 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:29:04.901 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.104700+0000", 2024-09-11T07:29:04.901 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6279999999999997e-05 2024-09-11T07:29:04.901 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.901 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.901 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:29:04.901 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.104830+0000", 2024-09-11T07:29:04.901 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012977799999999999 2024-09-11T07:29:04.901 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.901 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.901 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:29:04.902 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.104953+0000", 2024-09-11T07:29:04.902 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012361499999999999 2024-09-11T07:29:04.902 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.902 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.902 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:29:04.902 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.105508+0000", 2024-09-11T07:29:04.902 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00055419200000000001 2024-09-11T07:29:04.902 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.902 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.902 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.902 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.105843+0000", 2024-09-11T07:29:04.903 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00033575599999999999 2024-09-11T07:29:04.903 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.903 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.903 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.903 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.121768+0000", 2024-09-11T07:29:04.903 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.015924421000000001 2024-09-11T07:29:04.903 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.903 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.903 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:29:04.903 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.121809+0000", 2024-09-11T07:29:04.903 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0720000000000003e-05 2024-09-11T07:29:04.903 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.903 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.904 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:29:04.904 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.121844+0000", 2024-09-11T07:29:04.904 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5008000000000001e-05 2024-09-11T07:29:04.904 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.904 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:04.904 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.904 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.904 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.904 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:5171 7.4 7:2046b0c7:::benchmark_data_smithi022_77720_object5170:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-11T07:29:04.904 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:04.121208+0000", 2024-09-11T07:29:04.904 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.70171240999999995, 2024-09-11T07:29:04.904 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.017893342999999999, 2024-09-11T07:29:04.905 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:29:04.905 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:29:04.905 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:29:04.905 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:29:04.905 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:29:04.905 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 5171 2024-09-11T07:29:04.905 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.905 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:29:04.905 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.905 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:29:04.905 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.121208+0000", 2024-09-11T07:29:04.905 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.906 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.906 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.906 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:29:04.906 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.121208+0000", 2024-09-11T07:29:04.906 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.906 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.906 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.906 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:29:04.906 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.121206+0000", 2024-09-11T07:29:04.906 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:29:04.906 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.906 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.906 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:29:04.907 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.121231+0000", 2024-09-11T07:29:04.907 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5159000000000001e-05 2024-09-11T07:29:04.907 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.907 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.907 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:29:04.907 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.121245+0000", 2024-09-11T07:29:04.907 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3954999999999999e-05 2024-09-11T07:29:04.907 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.907 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.907 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:29:04.907 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.121253+0000", 2024-09-11T07:29:04.907 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.7069999999999992e-06 2024-09-11T07:29:04.908 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.908 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.908 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:29:04.908 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.121299+0000", 2024-09-11T07:29:04.908 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5676000000000001e-05 2024-09-11T07:29:04.908 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.908 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.908 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:29:04.908 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.121407+0000", 2024-09-11T07:29:04.908 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000108015 2024-09-11T07:29:04.908 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.908 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.909 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,4", 2024-09-11T07:29:04.909 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.121524+0000", 2024-09-11T07:29:04.909 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000117306 2024-09-11T07:29:04.909 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.909 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.909 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:29:04.909 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.122235+0000", 2024-09-11T07:29:04.909 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00071084200000000005 2024-09-11T07:29:04.909 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.909 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.909 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.909 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.123465+0000", 2024-09-11T07:29:04.909 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001230037 2024-09-11T07:29:04.910 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.910 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.910 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.910 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.139010+0000", 2024-09-11T07:29:04.910 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.01554492 2024-09-11T07:29:04.910 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.910 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.910 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:29:04.910 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.139059+0000", 2024-09-11T07:29:04.910 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9486e-05 2024-09-11T07:29:04.910 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.910 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.911 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:29:04.911 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.139102+0000", 2024-09-11T07:29:04.911 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2459000000000001e-05 2024-09-11T07:29:04.911 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.911 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:04.911 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.911 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.911 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.911 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:5177 7.f 7:f5de1d1c:::benchmark_data_smithi022_77720_object5176:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-11T07:29:04.911 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:04.122503+0000", 2024-09-11T07:29:04.911 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.70041724999999999, 2024-09-11T07:29:04.911 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.018969356, 2024-09-11T07:29:04.912 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:29:04.912 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:29:04.912 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:29:04.912 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:29:04.912 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:29:04.912 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 5177 2024-09-11T07:29:04.912 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.912 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:29:04.912 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.912 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:29:04.912 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.122503+0000", 2024-09-11T07:29:04.912 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.912 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.913 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.913 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:29:04.913 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.122503+0000", 2024-09-11T07:29:04.913 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.913 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.913 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.913 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:29:04.913 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.122502+0000", 2024-09-11T07:29:04.913 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:29:04.913 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.914 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.914 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:29:04.914 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.122527+0000", 2024-09-11T07:29:04.914 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4567e-05 2024-09-11T07:29:04.914 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.914 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.914 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:29:04.914 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.122603+0000", 2024-09-11T07:29:04.914 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.6168999999999995e-05 2024-09-11T07:29:04.914 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.914 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.914 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:29:04.914 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.122610+0000", 2024-09-11T07:29:04.915 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.8410000000000002e-06 2024-09-11T07:29:04.915 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.915 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.915 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:29:04.915 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.122685+0000", 2024-09-11T07:29:04.915 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5814000000000003e-05 2024-09-11T07:29:04.915 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.915 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.915 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:29:04.915 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.122826+0000", 2024-09-11T07:29:04.916 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014034799999999999 2024-09-11T07:29:04.916 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.916 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.916 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:29:04.916 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.122951+0000", 2024-09-11T07:29:04.916 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012524199999999999 2024-09-11T07:29:04.916 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.916 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.916 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:29:04.916 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.123955+0000", 2024-09-11T07:29:04.916 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00100372 2024-09-11T07:29:04.916 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.916 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.917 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.917 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.124466+0000", 2024-09-11T07:29:04.917 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00051135999999999998 2024-09-11T07:29:04.917 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.917 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.917 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.917 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.141396+0000", 2024-09-11T07:29:04.917 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.016929752999999999 2024-09-11T07:29:04.917 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.917 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.917 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:29:04.918 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.141438+0000", 2024-09-11T07:29:04.918 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2061000000000003e-05 2024-09-11T07:29:04.918 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.918 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.918 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:29:04.918 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.141473+0000", 2024-09-11T07:29:04.918 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4910000000000003e-05 2024-09-11T07:29:04.918 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.918 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:04.918 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.918 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.918 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.918 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:5179 7.f 7:f251c1d0:::benchmark_data_smithi022_77720_object5178:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-11T07:29:04.919 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:04.122624+0000", 2024-09-11T07:29:04.919 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.70029650200000004, 2024-09-11T07:29:04.919 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.018964897000000001, 2024-09-11T07:29:04.919 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:29:04.919 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:29:04.919 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:29:04.919 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:29:04.919 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:29:04.919 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 5179 2024-09-11T07:29:04.919 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.919 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:29:04.919 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.919 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:29:04.920 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.122624+0000", 2024-09-11T07:29:04.920 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.920 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.920 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.920 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:29:04.920 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.122624+0000", 2024-09-11T07:29:04.920 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.920 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.920 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.920 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:29:04.920 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.122623+0000", 2024-09-11T07:29:04.920 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:29:04.920 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.921 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.921 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:29:04.921 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.122646+0000", 2024-09-11T07:29:04.921 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3252999999999999e-05 2024-09-11T07:29:04.921 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.921 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.921 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:29:04.921 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.122653+0000", 2024-09-11T07:29:04.921 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.8460000000000002e-06 2024-09-11T07:29:04.921 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.921 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.922 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:29:04.922 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.122658+0000", 2024-09-11T07:29:04.922 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1220000000000003e-06 2024-09-11T07:29:04.922 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.922 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.922 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:29:04.922 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.123366+0000", 2024-09-11T07:29:04.922 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00070739100000000003 2024-09-11T07:29:04.922 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.922 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.922 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:29:04.923 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.123484+0000", 2024-09-11T07:29:04.923 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000118752 2024-09-11T07:29:04.923 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.923 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.923 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:29:04.923 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.123636+0000", 2024-09-11T07:29:04.923 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000151903 2024-09-11T07:29:04.923 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.923 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.923 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:29:04.923 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.124184+0000", 2024-09-11T07:29:04.923 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00054712599999999999 2024-09-11T07:29:04.924 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.924 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.924 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.924 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.124940+0000", 2024-09-11T07:29:04.924 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00075649400000000002 2024-09-11T07:29:04.924 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.924 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.924 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.924 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.141507+0000", 2024-09-11T07:29:04.924 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.016566562999999999 2024-09-11T07:29:04.924 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.924 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.925 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:04.926 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:29:04.926 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.141555+0000", 2024-09-11T07:29:04.926 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8267999999999999e-05 2024-09-11T07:29:04.926 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.926 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.926 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:29:04.926 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.141589+0000", 2024-09-11T07:29:04.927 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4272e-05 2024-09-11T07:29:04.927 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.927 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:04.927 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.927 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.927 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.927 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:5180 7.f 7:f39ba262:::benchmark_data_smithi022_77720_object5179:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-11T07:29:04.927 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:04.124454+0000", 2024-09-11T07:29:04.927 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.69846696699999999, 2024-09-11T07:29:04.927 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.017232434000000001, 2024-09-11T07:29:04.927 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:29:04.928 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:29:04.928 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:29:04.928 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:29:04.928 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:29:04.928 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 5180 2024-09-11T07:29:04.928 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.928 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:29:04.928 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.928 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:29:04.928 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.124454+0000", 2024-09-11T07:29:04.928 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.928 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.928 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.929 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:29:04.929 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.124454+0000", 2024-09-11T07:29:04.929 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.929 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.929 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.929 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:29:04.929 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.124452+0000", 2024-09-11T07:29:04.929 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:29:04.929 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.929 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.929 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:29:04.929 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.124478+0000", 2024-09-11T07:29:04.929 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5341999999999999e-05 2024-09-11T07:29:04.930 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.930 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.930 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:29:04.930 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.124485+0000", 2024-09-11T07:29:04.930 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.4209999999999998e-06 2024-09-11T07:29:04.930 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.930 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.930 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:29:04.930 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.124489+0000", 2024-09-11T07:29:04.930 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8079999999999998e-06 2024-09-11T07:29:04.930 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.930 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.930 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:29:04.931 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.124530+0000", 2024-09-11T07:29:04.931 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1331000000000001e-05 2024-09-11T07:29:04.931 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.931 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.931 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:29:04.931 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.124602+0000", 2024-09-11T07:29:04.931 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1790000000000002e-05 2024-09-11T07:29:04.931 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.931 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.931 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:29:04.931 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.124677+0000", 2024-09-11T07:29:04.931 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.4516000000000003e-05 2024-09-11T07:29:04.931 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.932 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.932 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:29:04.932 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.125071+0000", 2024-09-11T07:29:04.932 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000394108 2024-09-11T07:29:04.932 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.932 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.932 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.932 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.125217+0000", 2024-09-11T07:29:04.932 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000146317 2024-09-11T07:29:04.932 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.932 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.933 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.933 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.141624+0000", 2024-09-11T07:29:04.933 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.016406667999999999 2024-09-11T07:29:04.933 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.933 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.933 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:29:04.933 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.141655+0000", 2024-09-11T07:29:04.933 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.1742999999999997e-05 2024-09-11T07:29:04.934 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.934 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.934 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:29:04.934 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.141686+0000", 2024-09-11T07:29:04.934 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0812000000000002e-05 2024-09-11T07:29:04.934 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.934 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:04.934 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.934 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.934 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.934 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:6371 7.9 7:9ca95789:::benchmark_data_smithi022_77720_object6370:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-11T07:29:04.935 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:04.484988+0000", 2024-09-11T07:29:04.935 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.33793304600000001, 2024-09-11T07:29:04.935 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.011859954000000001, 2024-09-11T07:29:04.935 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:29:04.935 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:29:04.935 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:29:04.935 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:29:04.935 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:29:04.935 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 6371 2024-09-11T07:29:04.935 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.935 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:29:04.935 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.936 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:29:04.936 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.484988+0000", 2024-09-11T07:29:04.936 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.936 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.936 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.936 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:29:04.936 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.484988+0000", 2024-09-11T07:29:04.936 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.936 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.936 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.936 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:29:04.937 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.484986+0000", 2024-09-11T07:29:04.937 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:29:04.937 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.937 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.937 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:29:04.937 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.485012+0000", 2024-09-11T07:29:04.937 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5795e-05 2024-09-11T07:29:04.937 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.937 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.937 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:29:04.937 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.485021+0000", 2024-09-11T07:29:04.938 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.2019999999999993e-06 2024-09-11T07:29:04.938 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.938 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.938 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:29:04.938 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.485027+0000", 2024-09-11T07:29:04.938 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.463e-06 2024-09-11T07:29:04.938 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.938 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.938 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:29:04.938 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.485060+0000", 2024-09-11T07:29:04.938 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3009000000000002e-05 2024-09-11T07:29:04.939 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.939 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.939 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:29:04.939 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.485175+0000", 2024-09-11T07:29:04.939 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000115127 2024-09-11T07:29:04.939 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.939 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.939 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,4", 2024-09-11T07:29:04.939 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.485334+0000", 2024-09-11T07:29:04.939 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015879699999999999 2024-09-11T07:29:04.939 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.939 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.940 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:29:04.940 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.487071+0000", 2024-09-11T07:29:04.940 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001737541 2024-09-11T07:29:04.940 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.940 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.940 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.940 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.489395+0000", 2024-09-11T07:29:04.940 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002323472 2024-09-11T07:29:04.940 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.940 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.940 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.940 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.496773+0000", 2024-09-11T07:29:04.941 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0073786440000000002 2024-09-11T07:29:04.941 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.941 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.941 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:29:04.941 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.496815+0000", 2024-09-11T07:29:04.941 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1471000000000003e-05 2024-09-11T07:29:04.941 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.941 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.941 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:29:04.942 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.496848+0000", 2024-09-11T07:29:04.942 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2971e-05 2024-09-11T07:29:04.942 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.942 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:04.942 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.942 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.942 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.942 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:6373 7.4 7:273b956b:::benchmark_data_smithi022_77720_object6372:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-11T07:29:04.942 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:04.485368+0000", 2024-09-11T07:29:04.942 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.33755291300000001, 2024-09-11T07:29:04.943 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.011476991000000001, 2024-09-11T07:29:04.943 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:29:04.943 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:29:04.943 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:29:04.943 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:29:04.943 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:29:04.943 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 6373 2024-09-11T07:29:04.943 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.943 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:29:04.943 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.943 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:29:04.943 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.485368+0000", 2024-09-11T07:29:04.944 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.944 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.944 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.944 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:29:04.944 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.485368+0000", 2024-09-11T07:29:04.944 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.944 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.944 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.944 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:29:04.944 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.485366+0000", 2024-09-11T07:29:04.944 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:29:04.944 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.945 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.945 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:29:04.945 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.485395+0000", 2024-09-11T07:29:04.945 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8282000000000001e-05 2024-09-11T07:29:04.945 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.945 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.945 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:29:04.945 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.485412+0000", 2024-09-11T07:29:04.945 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.6969999999999998e-05 2024-09-11T07:29:04.945 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.945 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.945 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:29:04.946 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.485418+0000", 2024-09-11T07:29:04.946 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.02e-06 2024-09-11T07:29:04.946 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.946 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.946 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:29:04.946 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.485466+0000", 2024-09-11T07:29:04.946 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8553000000000001e-05 2024-09-11T07:29:04.946 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.946 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.946 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:29:04.946 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.485643+0000", 2024-09-11T07:29:04.946 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00017682599999999999 2024-09-11T07:29:04.947 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.947 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.947 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,4", 2024-09-11T07:29:04.947 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.485818+0000", 2024-09-11T07:29:04.947 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00017451899999999999 2024-09-11T07:29:04.947 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.947 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.947 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:29:04.947 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.487151+0000", 2024-09-11T07:29:04.947 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001333692 2024-09-11T07:29:04.947 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.947 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.948 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.948 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.489663+0000", 2024-09-11T07:29:04.948 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0025118509999999998 2024-09-11T07:29:04.948 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.948 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.948 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.948 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.496756+0000", 2024-09-11T07:29:04.948 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.007092831 2024-09-11T07:29:04.948 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.948 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.948 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:29:04.948 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.496808+0000", 2024-09-11T07:29:04.949 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1891000000000002e-05 2024-09-11T07:29:04.949 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.949 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.949 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:29:04.949 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.496845+0000", 2024-09-11T07:29:04.949 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6968999999999998e-05 2024-09-11T07:29:04.949 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.949 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:04.949 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.949 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.949 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.949 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:7182 7.6 7:691e4e7a:::benchmark_data_smithi022_77720_object7181:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-11T07:29:04.950 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:04.770467+0000", 2024-09-11T07:29:04.950 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.052453991999999998, 2024-09-11T07:29:04.950 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.016686207000000002, 2024-09-11T07:29:04.950 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:29:04.950 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:29:04.950 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:29:04.950 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:29:04.950 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:29:04.950 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 7182 2024-09-11T07:29:04.950 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.950 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:29:04.950 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.951 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:29:04.951 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.770467+0000", 2024-09-11T07:29:04.951 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.951 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.951 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.951 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:29:04.951 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.770467+0000", 2024-09-11T07:29:04.951 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.951 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.951 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.951 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:29:04.951 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.770464+0000", 2024-09-11T07:29:04.952 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:29:04.952 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.952 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.952 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:29:04.952 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.770501+0000", 2024-09-11T07:29:04.952 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7038e-05 2024-09-11T07:29:04.952 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.952 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.952 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:29:04.952 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.770518+0000", 2024-09-11T07:29:04.953 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.6432000000000001e-05 2024-09-11T07:29:04.953 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.953 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.953 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:29:04.953 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.770525+0000", 2024-09-11T07:29:04.953 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.6120000000000001e-06 2024-09-11T07:29:04.953 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.953 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.953 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:29:04.953 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.773845+0000", 2024-09-11T07:29:04.953 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0033198189999999999 2024-09-11T07:29:04.954 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.954 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.954 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:29:04.954 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.773983+0000", 2024-09-11T07:29:04.954 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000137489 2024-09-11T07:29:04.954 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.954 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.954 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:29:04.954 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.774126+0000", 2024-09-11T07:29:04.954 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014325500000000001 2024-09-11T07:29:04.954 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.954 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.955 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.955 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.787017+0000", 2024-09-11T07:29:04.955 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.012891023999999999 2024-09-11T07:29:04.955 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.955 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.955 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:29:04.955 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.787023+0000", 2024-09-11T07:29:04.955 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.224e-06 2024-09-11T07:29:04.955 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.955 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.955 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.955 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.787084+0000", 2024-09-11T07:29:04.955 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0633e-05 2024-09-11T07:29:04.956 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.956 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.956 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:29:04.956 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.787123+0000", 2024-09-11T07:29:04.956 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9486000000000001e-05 2024-09-11T07:29:04.956 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.956 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.956 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:29:04.956 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.787153+0000", 2024-09-11T07:29:04.956 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.9743e-05 2024-09-11T07:29:04.956 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.956 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:04.956 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.957 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.957 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.957 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:7183 7.6 7:6666d5ba:::benchmark_data_smithi022_77720_object7182:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-11T07:29:04.957 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:04.770539+0000", 2024-09-11T07:29:04.957 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.052381669999999998, 2024-09-11T07:29:04.957 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.016712682, 2024-09-11T07:29:04.957 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:29:04.957 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:29:04.957 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:29:04.957 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:29:04.957 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:29:04.957 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 7183 2024-09-11T07:29:04.958 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.958 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:29:04.958 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.958 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:29:04.958 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.770539+0000", 2024-09-11T07:29:04.958 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.958 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.958 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.958 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:29:04.958 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.770539+0000", 2024-09-11T07:29:04.958 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.958 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.958 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.959 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:29:04.959 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.770538+0000", 2024-09-11T07:29:04.959 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:29:04.959 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.959 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.959 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:29:04.959 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.770622+0000", 2024-09-11T07:29:04.959 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3616999999999996e-05 2024-09-11T07:29:04.959 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.959 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.959 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:29:04.959 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.770635+0000", 2024-09-11T07:29:04.959 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3495e-05 2024-09-11T07:29:04.960 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.960 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.960 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:29:04.960 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.770641+0000", 2024-09-11T07:29:04.960 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3669999999999996e-06 2024-09-11T07:29:04.960 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.960 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.960 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:29:04.960 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.775334+0000", 2024-09-11T07:29:04.960 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0046937050000000003 2024-09-11T07:29:04.960 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.960 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.961 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:29:04.961 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.775450+0000", 2024-09-11T07:29:04.961 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000115692 2024-09-11T07:29:04.961 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.961 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.961 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:29:04.961 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.775590+0000", 2024-09-11T07:29:04.961 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000139617 2024-09-11T07:29:04.961 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.961 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.961 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:29:04.961 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.787028+0000", 2024-09-11T07:29:04.962 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.011438428 2024-09-11T07:29:04.962 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.962 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.962 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.962 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.787057+0000", 2024-09-11T07:29:04.962 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.9034999999999999e-05 2024-09-11T07:29:04.962 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.962 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.962 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.962 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.787193+0000", 2024-09-11T07:29:04.962 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013604900000000001 2024-09-11T07:29:04.962 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.962 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.963 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:29:04.963 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.787225+0000", 2024-09-11T07:29:04.963 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.1436999999999997e-05 2024-09-11T07:29:04.963 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.963 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.963 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:29:04.963 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.787252+0000", 2024-09-11T07:29:04.963 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7121999999999999e-05 2024-09-11T07:29:04.963 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.963 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:04.963 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.963 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.963 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.964 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:7184 7.6 7:6e2b4569:::benchmark_data_smithi022_77720_object7183:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-11T07:29:04.964 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:04.770646+0000", 2024-09-11T07:29:04.964 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.052274356000000001, 2024-09-11T07:29:04.964 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.021543343999999999, 2024-09-11T07:29:04.964 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:29:04.964 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:29:04.964 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:29:04.964 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:29:04.964 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:29:04.964 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 7184 2024-09-11T07:29:04.964 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.964 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:29:04.964 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.965 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:29:04.965 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.770646+0000", 2024-09-11T07:29:04.965 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.965 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.965 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.965 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:29:04.965 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.770646+0000", 2024-09-11T07:29:04.965 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.965 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.965 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.965 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:29:04.965 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.770645+0000", 2024-09-11T07:29:04.965 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:29:04.966 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.966 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.966 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:29:04.966 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.770673+0000", 2024-09-11T07:29:04.966 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7155e-05 2024-09-11T07:29:04.966 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.966 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.966 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:29:04.966 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.770680+0000", 2024-09-11T07:29:04.966 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5310000000000004e-06 2024-09-11T07:29:04.966 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.966 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.966 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:29:04.966 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.770684+0000", 2024-09-11T07:29:04.967 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2030000000000002e-06 2024-09-11T07:29:04.967 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.967 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.967 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:29:04.967 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.775958+0000", 2024-09-11T07:29:04.967 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0052734799999999997 2024-09-11T07:29:04.967 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.967 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.967 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:29:04.967 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.786334+0000", 2024-09-11T07:29:04.967 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.010375759999999999 2024-09-11T07:29:04.967 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.968 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.968 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:29:04.968 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.786504+0000", 2024-09-11T07:29:04.968 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00017041699999999999 2024-09-11T07:29:04.968 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.968 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.968 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.968 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.790570+0000", 2024-09-11T07:29:04.968 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0040661289999999999 2024-09-11T07:29:04.968 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.968 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.968 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:29:04.969 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.791583+0000", 2024-09-11T07:29:04.969 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0010132679999999999 2024-09-11T07:29:04.969 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.969 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.969 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.969 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.792102+0000", 2024-09-11T07:29:04.969 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000518963 2024-09-11T07:29:04.969 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.969 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.969 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:29:04.969 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.792153+0000", 2024-09-11T07:29:04.969 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.0321999999999997e-05 2024-09-11T07:29:04.970 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.970 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.970 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:29:04.970 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.792190+0000", 2024-09-11T07:29:04.970 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6981999999999999e-05 2024-09-11T07:29:04.970 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.970 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:04.970 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.970 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.970 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.970 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:7186 7.f 7:ff777217:::benchmark_data_smithi022_77720_object7185:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-11T07:29:04.971 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:04.770690+0000", 2024-09-11T07:29:04.971 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.05223117, 2024-09-11T07:29:04.971 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.016169236999999999, 2024-09-11T07:29:04.971 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:29:04.971 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:29:04.971 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:29:04.971 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:29:04.971 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:29:04.971 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 7186 2024-09-11T07:29:04.971 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.971 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:29:04.971 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.972 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:29:04.972 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.770690+0000", 2024-09-11T07:29:04.972 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.972 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.972 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.972 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:29:04.972 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.770690+0000", 2024-09-11T07:29:04.972 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.972 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.972 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.972 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:29:04.972 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.770689+0000", 2024-09-11T07:29:04.973 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:29:04.973 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.973 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.973 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:29:04.973 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.770711+0000", 2024-09-11T07:29:04.973 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2405e-05 2024-09-11T07:29:04.973 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.973 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.973 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:29:04.973 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.770717+0000", 2024-09-11T07:29:04.973 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9240000000000004e-06 2024-09-11T07:29:04.973 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.974 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.974 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:29:04.974 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.770726+0000", 2024-09-11T07:29:04.974 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.3819999999999993e-06 2024-09-11T07:29:04.974 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.974 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.974 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:29:04.974 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.771792+0000", 2024-09-11T07:29:04.974 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001065898 2024-09-11T07:29:04.974 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.974 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.974 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:29:04.974 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.771908+0000", 2024-09-11T07:29:04.974 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000115878 2024-09-11T07:29:04.975 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.975 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.975 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:29:04.975 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.772049+0000", 2024-09-11T07:29:04.975 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014051600000000001 2024-09-11T07:29:04.975 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.975 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.975 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.975 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.774538+0000", 2024-09-11T07:29:04.975 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002489607 2024-09-11T07:29:04.975 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.975 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.976 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:29:04.976 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.775821+0000", 2024-09-11T07:29:04.976 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001282163 2024-09-11T07:29:04.976 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.976 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.976 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.976 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.786773+0000", 2024-09-11T07:29:04.976 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.010952584 2024-09-11T07:29:04.976 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.976 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.976 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:29:04.976 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.786819+0000", 2024-09-11T07:29:04.976 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5417000000000002e-05 2024-09-11T07:29:04.977 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.977 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.977 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:29:04.977 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.786859+0000", 2024-09-11T07:29:04.977 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0219000000000002e-05 2024-09-11T07:29:04.977 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.977 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:04.977 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.977 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.977 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.977 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop_reply(client.24778.0:7182 7.6 e97/92 ondisk, result = 0)", 2024-09-11T07:29:04.977 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:04.775944+0000", 2024-09-11T07:29:04.977 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.046977016000000003, 2024-09-11T07:29:04.978 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.011085154999999999, 2024-09-11T07:29:04.978 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:29:04.978 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:29:04.978 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:29:04.978 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.978 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:29:04.978 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.775944+0000", 2024-09-11T07:29:04.978 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.978 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.978 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.978 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:29:04.978 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.775944+0000", 2024-09-11T07:29:04.979 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.979 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.979 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.979 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:29:04.979 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.775942+0000", 2024-09-11T07:29:04.979 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:29:04.979 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.979 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.979 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:29:04.979 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.775948+0000", 2024-09-11T07:29:04.979 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7540000000000003e-06 2024-09-11T07:29:04.979 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.980 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.980 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:29:04.980 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.775957+0000", 2024-09-11T07:29:04.980 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.5029999999999997e-06 2024-09-11T07:29:04.980 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.980 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.980 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:29:04.980 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.775961+0000", 2024-09-11T07:29:04.980 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7110000000000003e-06 2024-09-11T07:29:04.980 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.981 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.981 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:29:04.981 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.787006+0000", 2024-09-11T07:29:04.981 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.011044823 2024-09-11T07:29:04.981 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.981 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.981 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:29:04.981 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.787012+0000", 2024-09-11T07:29:04.981 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6280000000000002e-06 2024-09-11T07:29:04.981 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.981 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.981 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:29:04.982 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.787029+0000", 2024-09-11T07:29:04.982 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.7116e-05 2024-09-11T07:29:04.982 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.982 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:04.982 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.982 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.982 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.982 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:7198 7.6 7:62c8b4ea:::benchmark_data_smithi022_77720_object7197:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-11T07:29:04.982 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:04.776126+0000", 2024-09-11T07:29:04.982 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.046794394000000003, 2024-09-11T07:29:04.982 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.016251473999999998, 2024-09-11T07:29:04.982 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:29:04.982 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:29:04.983 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:29:04.983 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:29:04.983 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:29:04.983 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 7198 2024-09-11T07:29:04.983 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.983 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:29:04.983 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.983 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:29:04.983 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.776126+0000", 2024-09-11T07:29:04.983 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.983 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.984 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.984 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:29:04.984 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.776126+0000", 2024-09-11T07:29:04.984 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:04.984 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.984 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.984 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:29:04.984 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.776118+0000", 2024-09-11T07:29:04.984 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999914 2024-09-11T07:29:04.984 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.984 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.984 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:29:04.985 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.776156+0000", 2024-09-11T07:29:04.985 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8665000000000003e-05 2024-09-11T07:29:04.985 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.985 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.985 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:29:04.985 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.776168+0000", 2024-09-11T07:29:04.985 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1396999999999999e-05 2024-09-11T07:29:04.985 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.985 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.985 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:29:04.985 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.776174+0000", 2024-09-11T07:29:04.986 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3010000000000003e-06 2024-09-11T07:29:04.986 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.986 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.986 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:29:04.986 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.789877+0000", 2024-09-11T07:29:04.986 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.013702628 2024-09-11T07:29:04.986 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.986 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.986 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:29:04.986 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.789990+0000", 2024-09-11T07:29:04.986 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000113108 2024-09-11T07:29:04.986 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.986 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.987 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:29:04.987 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.790122+0000", 2024-09-11T07:29:04.987 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013203800000000001 2024-09-11T07:29:04.987 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.987 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.987 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:29:04.987 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.791588+0000", 2024-09-11T07:29:04.987 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001466188 2024-09-11T07:29:04.987 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.987 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.987 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.987 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.791696+0000", 2024-09-11T07:29:04.987 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000108297 2024-09-11T07:29:04.988 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.988 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.988 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:29:04.988 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.792240+0000", 2024-09-11T07:29:04.988 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00054352600000000001 2024-09-11T07:29:04.988 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.988 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.988 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:29:04.988 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.792286+0000", 2024-09-11T07:29:04.988 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6323999999999999e-05 2024-09-11T07:29:04.988 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:04.988 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:04.988 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:29:04.989 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:04.792378+0000", 2024-09-11T07:29:04.989 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.1588000000000005e-05 2024-09-11T07:29:04.989 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.989 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:04.989 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.989 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:04.989 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:04.989 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:29:05.030 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:05.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:05.031 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:05.105 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:05 smithi022 ceph-mon[29462]: pgmap v86: 148 pgs: 148 active+clean; 293 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 37 MiB/s wr, 586 op/s 2024-09-11T07:29:05.106 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:05 smithi022 ceph-mon[37330]: pgmap v86: 148 pgs: 148 active+clean; 293 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 37 MiB/s wr, 586 op/s 2024-09-11T07:29:05.132 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:05.173 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:05.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:05.174 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:05.275 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:05.315 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:05.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:05.316 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:05.416 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:05.459 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:05.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:05.461 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:05.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:05 smithi104 ceph-mon[33507]: pgmap v86: 148 pgs: 148 active+clean; 293 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 37 MiB/s wr, 586 op/s 2024-09-11T07:29:05.561 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:05.582 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 3 15 9476 9461 196.984 176.5 0.00326545 0.00506762 2024-09-11T07:29:05.601 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:05.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:05.602 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:05.703 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:05.744 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:05.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:05.745 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:05.846 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:05.874 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:05.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:05.875 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:05.976 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:06.015 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:06.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:06.016 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:06.117 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:06.161 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:06.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:06.163 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:06.264 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:06.304 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:06.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:06.306 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:06.406 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:06.435 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:06.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:06.436 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:06.536 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:06.561 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:06.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:06.562 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:06.582 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 4 16 12286 12270 191.626 175.562 0.0049188 0.00516298 2024-09-11T07:29:06.664 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:06.705 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:06.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:06.706 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:06.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:06 smithi104 ceph-mon[33507]: osdmap e98: 8 total, 8 up, 8 in 2024-09-11T07:29:06.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:06 smithi104 ceph-mon[33507]: pgmap v88: 148 pgs: 148 active+clean; 293 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 852 B/s rd, 49 MiB/s wr, 781 op/s 2024-09-11T07:29:06.807 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:06.831 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:06.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:06.832 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:06 smithi022 ceph-mon[29462]: osdmap e98: 8 total, 8 up, 8 in 2024-09-11T07:29:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:06 smithi022 ceph-mon[29462]: pgmap v88: 148 pgs: 148 active+clean; 293 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 852 B/s rd, 49 MiB/s wr, 781 op/s 2024-09-11T07:29:06.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:06 smithi022 ceph-mon[37330]: osdmap e98: 8 total, 8 up, 8 in 2024-09-11T07:29:06.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:06 smithi022 ceph-mon[37330]: pgmap v88: 148 pgs: 148 active+clean; 293 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 852 B/s rd, 49 MiB/s wr, 781 op/s 2024-09-11T07:29:06.933 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:06.972 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:06.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:06.973 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:07.074 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:07.099 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:07.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:07.100 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:07.201 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:07.242 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:07.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:07.243 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:07.343 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:07.381 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:07.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:07.382 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:07.483 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:07.509 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:07.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:07.510 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:07.582 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 5 15 14715 14700 183.676 151.875 0.00380495 0.0054369 2024-09-11T07:29:07.611 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:07.615 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is set 2024-09-11T07:29:07.636 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:07.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:07.637 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:07.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:07 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1525423842' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:29:07.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:07 smithi022 ceph-mon[29462]: from='client.24784 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:29:07.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:07 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1525423842' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:29:07.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:07 smithi022 ceph-mon[37330]: from='client.24784 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:29:07.737 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:07.768 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:07.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:07.769 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:07.869 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:07.911 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:07.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:07.912 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:08.013 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:08.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:07 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1525423842' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:29:08.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:07 smithi104 ceph-mon[33507]: from='client.24784 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:29:08.058 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:08.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:08.060 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:08.160 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:08.191 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:08.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:08.192 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:08.292 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:08.336 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:08.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:08.337 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:08.438 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:08.493 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:08.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:08.494 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:08.582 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 6 16 17261 17245 179.572 159.062 0.00426905 0.00556226 2024-09-11T07:29:08.595 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:08.622 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:08.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:08.623 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:08.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:08 smithi022 ceph-mon[29462]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:29:08.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:08 smithi022 ceph-mon[29462]: from='client.24784 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:29:08.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:08 smithi022 ceph-mon[29462]: osdmap e99: 8 total, 8 up, 8 in 2024-09-11T07:29:08.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:08 smithi022 ceph-mon[29462]: pgmap v90: 148 pgs: 148 active+clean; 293 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 737 B/s rd, 42 MiB/s wr, 675 op/s 2024-09-11T07:29:08.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:29:08.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:29:08.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:08 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/159045854' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "2.1c", "id": [5, 2, 7]}]: dispatch 2024-09-11T07:29:08.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:08 smithi022 ceph-mon[29462]: from='client.24770 ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "2.1c", "id": [5, 2, 7]}]: dispatch 2024-09-11T07:29:08.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:08 smithi022 ceph-mon[37330]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:29:08.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:08 smithi022 ceph-mon[37330]: from='client.24784 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:29:08.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:08 smithi022 ceph-mon[37330]: osdmap e99: 8 total, 8 up, 8 in 2024-09-11T07:29:08.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:08 smithi022 ceph-mon[37330]: pgmap v90: 148 pgs: 148 active+clean; 293 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 737 B/s rd, 42 MiB/s wr, 675 op/s 2024-09-11T07:29:08.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:29:08.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:29:08.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:08 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/159045854' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "2.1c", "id": [5, 2, 7]}]: dispatch 2024-09-11T07:29:08.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:08 smithi022 ceph-mon[37330]: from='client.24770 ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "2.1c", "id": [5, 2, 7]}]: dispatch 2024-09-11T07:29:08.724 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:08.753 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:08.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:08.754 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:08.855 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:08.897 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:08.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:08.898 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:08.934 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-11T07:29:08.999 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:09.005 INFO:teuthology.orchestra.run.smithi022.stderr:set 2.1c pg_upmap mapping to [5,2,7] 2024-09-11T07:29:09.026 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:09.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:09.027 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:09.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:08 smithi104 ceph-mon[33507]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:29:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:08 smithi104 ceph-mon[33507]: from='client.24784 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:29:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:08 smithi104 ceph-mon[33507]: osdmap e99: 8 total, 8 up, 8 in 2024-09-11T07:29:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:08 smithi104 ceph-mon[33507]: pgmap v90: 148 pgs: 148 active+clean; 293 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 737 B/s rd, 42 MiB/s wr, 675 op/s 2024-09-11T07:29:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:29:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:29:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:08 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/159045854' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "2.1c", "id": [5, 2, 7]}]: dispatch 2024-09-11T07:29:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:08 smithi104 ceph-mon[33507]: from='client.24770 ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "2.1c", "id": [5, 2, 7]}]: dispatch 2024-09-11T07:29:09.128 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:09.156 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:09.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:09.157 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:09.257 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:09.307 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:09.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:09.309 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:09.409 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:09.454 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:09.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:09.455 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:09.530 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:29:09 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:29:09.556 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:09.582 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 7 16 19742 19726 176.07 155.062 0.00326665 0.00567273 2024-09-11T07:29:09.597 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:09.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:09.599 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:09.700 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:09.729 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:09.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:09.729 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:09.830 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:09.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:09.860 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:09.860 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:09.961 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:09.989 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:09.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:09.990 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:10.091 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:10.130 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:10.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:10.131 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:10.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:10 smithi022 ceph-mon[29462]: from='client.24770 ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "2.1c", "id": [5, 2, 7]}]': finished 2024-09-11T07:29:10.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:10 smithi022 ceph-mon[29462]: osdmap e100: 8 total, 8 up, 8 in 2024-09-11T07:29:10.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:10 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:29:10.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:10 smithi022 ceph-mon[37330]: from='client.24770 ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "2.1c", "id": [5, 2, 7]}]': finished 2024-09-11T07:29:10.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:10 smithi022 ceph-mon[37330]: osdmap e100: 8 total, 8 up, 8 in 2024-09-11T07:29:10.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:10 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:29:10.232 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:10.260 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:10.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:10.261 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:10.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:10 smithi104 ceph-mon[33507]: from='client.24770 ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "2.1c", "id": [5, 2, 7]}]': finished 2024-09-11T07:29:10.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:10 smithi104 ceph-mon[33507]: osdmap e100: 8 total, 8 up, 8 in 2024-09-11T07:29:10.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:10 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:29:10.362 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:10.402 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:10.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:10.403 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:10.481 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T07:29:10.504 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:10.525 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:29:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:29:10] "GET /metrics HTTP/1.1" 200 222927 "" "Prometheus/2.43.0" 2024-09-11T07:29:10.567 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:10.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:10.568 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:10.582 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 8 16 22185 22169 173.145 152.688 0.00174084 0.0057579 2024-09-11T07:29:10.669 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:10.696 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:10.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:10.697 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:10.798 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:10.824 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:10.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:10.826 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:10.926 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:10.955 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:10.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:10.956 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:11.057 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:11.102 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:11.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:11.103 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:11.204 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:11.246 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:11.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:11.263 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:11.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:11 smithi104 ceph-mon[33507]: pgmap v92: 148 pgs: 148 active+clean; 1.1 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 852 B/s rd, 141 MiB/s wr, 2.25k op/s 2024-09-11T07:29:11.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:11 smithi104 ceph-mon[33507]: osdmap e101: 8 total, 8 up, 8 in 2024-09-11T07:29:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:11 smithi104 ceph-mon[33507]: osdmap e102: 8 total, 8 up, 8 in 2024-09-11T07:29:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:11 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:29:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:11 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:29:11.364 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:11.405 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:11.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:11.406 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:11.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:11 smithi022 ceph-mon[29462]: pgmap v92: 148 pgs: 148 active+clean; 1.1 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 852 B/s rd, 141 MiB/s wr, 2.25k op/s 2024-09-11T07:29:11.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:11 smithi022 ceph-mon[29462]: osdmap e101: 8 total, 8 up, 8 in 2024-09-11T07:29:11.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:11 smithi022 ceph-mon[29462]: osdmap e102: 8 total, 8 up, 8 in 2024-09-11T07:29:11.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:11 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:29:11.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:11 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:29:11.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:11 smithi022 ceph-mon[37330]: pgmap v92: 148 pgs: 148 active+clean; 1.1 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 852 B/s rd, 141 MiB/s wr, 2.25k op/s 2024-09-11T07:29:11.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:11 smithi022 ceph-mon[37330]: osdmap e101: 8 total, 8 up, 8 in 2024-09-11T07:29:11.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:11 smithi022 ceph-mon[37330]: osdmap e102: 8 total, 8 up, 8 in 2024-09-11T07:29:11.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:11 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:29:11.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:11 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:29:11.507 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:11.538 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:11.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:11.539 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:11.583 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 9 16 24331 24315 168.809 134.125 0.00388819 0.00591788 2024-09-11T07:29:11.640 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:11.670 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:11.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:11.671 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:11.772 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:11.809 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:11.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:11.810 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:11.911 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:11.940 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:11.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:11.941 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:12.042 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:12.071 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:12.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:12.072 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:12.173 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:12.215 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:12.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:12.216 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:12.316 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:12.361 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:12.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:12.362 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:12.463 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:12.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:12.494 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:12.494 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:12.583 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 10 16 26521 26505 165.615 136.875 0.00383941 0.00603197 2024-09-11T07:29:12.594 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:12.615 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:12.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:12.616 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:12.717 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:12.746 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:12.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:12.747 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:12.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:12 smithi104 ceph-mon[33507]: osdmap e103: 8 total, 8 up, 8 in 2024-09-11T07:29:12.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:12 smithi104 ceph-mon[33507]: pgmap v96: 148 pgs: 148 active+clean; 1.1 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 211 MiB/s wr, 3.38k op/s 2024-09-11T07:29:12.848 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:12.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:12.871 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:12.871 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:12.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:12 smithi022 ceph-mon[29462]: osdmap e103: 8 total, 8 up, 8 in 2024-09-11T07:29:12.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:12 smithi022 ceph-mon[29462]: pgmap v96: 148 pgs: 148 active+clean; 1.1 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 211 MiB/s wr, 3.38k op/s 2024-09-11T07:29:12.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:12 smithi022 ceph-mon[37330]: osdmap e103: 8 total, 8 up, 8 in 2024-09-11T07:29:12.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:12 smithi022 ceph-mon[37330]: pgmap v96: 148 pgs: 148 active+clean; 1.1 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 211 MiB/s wr, 3.38k op/s 2024-09-11T07:29:12.972 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:13.012 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:13.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:13.014 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:13.114 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:13.154 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:13.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:13.155 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:13.256 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:13.295 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:13.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:13.296 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:13.397 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:13.437 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:13.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:13.438 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:13.539 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:13.583 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 11 16 28676 28660 162.803 134.688 0.0151608 0.00613674 2024-09-11T07:29:13.585 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:13.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:13.586 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:13.687 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:13.716 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:13.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:13.717 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:13.818 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:13.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:13.858 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:13.858 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:13.959 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:13.999 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:13.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:14.000 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:14.101 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:14.103 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:29:14.168 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:14.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:14.170 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:14.271 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:14.318 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:14.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:14.319 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:14.420 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:14.446 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:14.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:14.447 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:14.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:29:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:29:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:29:14.548 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:14.583 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 12 16 30717 30701 159.864 127.562 0.00996616 0.00624946 2024-09-11T07:29:14.595 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:14.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:14.596 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:14.697 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:14.725 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:14.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:14.726 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:14.827 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:14.854 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:14.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:14.855 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:14.911 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:29:14.955 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:14.984 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:14.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:14.986 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:15.086 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:15.116 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:15.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:15.117 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:15.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:15 smithi022 ceph-mon[29462]: pgmap v97: 148 pgs: 148 active+clean; 1.8 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.0 KiB/s rd, 141 MiB/s wr, 2.26k op/s 2024-09-11T07:29:15.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:15 smithi022 ceph-mon[37330]: pgmap v97: 148 pgs: 148 active+clean; 1.8 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.0 KiB/s rd, 141 MiB/s wr, 2.26k op/s 2024-09-11T07:29:15.218 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:15.243 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:15.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:15.244 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:15.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:15 smithi104 ceph-mon[33507]: pgmap v97: 148 pgs: 148 active+clean; 1.8 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.0 KiB/s rd, 141 MiB/s wr, 2.26k op/s 2024-09-11T07:29:15.299 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:29:15.299 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:29:15.299 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_0 2024-09-11T07:29:15.300 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool unique_pool_0 by 10 min 8 2024-09-11T07:29:15.300 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:29:15.345 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:15.375 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:15.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:15.376 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:15.476 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:15.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:15.536 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:15.536 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:15.583 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 13 16 32597 32581 156.605 117.5 0.0110142 0.00637937 2024-09-11T07:29:15.620 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:29:15.621 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:29:15.637 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:15.654 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:29:15.654 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:29:15.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:15.682 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:15.682 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:15.690 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:29:15.691 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:29:15.727 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:29:15.727 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:29:15.771 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:29:15.771 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:29:15.783 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:15.797 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:29:15.798 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:29:15.825 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:15.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:15.826 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:15.833 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:29:15.834 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:29:15.865 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:29:15.865 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:29:15.927 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:15.965 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:15.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:15.967 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:16.067 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:16.106 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:16.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:16.107 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:16.208 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:16.236 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:16.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:16.237 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:16.338 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:16.377 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:16.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:16.378 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:16.478 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:16.517 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:16.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:16.518 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:16.583 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 14 15 34567 34552 154.217 123.188 0.0123283 0.00647946 2024-09-11T07:29:16.620 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:16.659 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:16.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:16.661 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:16.762 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:16.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:16 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3618078789' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:29:16.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:16 smithi104 ceph-mon[33507]: pgmap v98: 148 pgs: 148 active+clean; 1.8 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 852 B/s rd, 117 MiB/s wr, 1.87k op/s 2024-09-11T07:29:16.791 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:16.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:16.792 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:16.812 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:16 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3618078789' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:29:16.812 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:16 smithi022 ceph-mon[29462]: pgmap v98: 148 pgs: 148 active+clean; 1.8 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 852 B/s rd, 117 MiB/s wr, 1.87k op/s 2024-09-11T07:29:16.812 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:16 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3618078789' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:29:16.812 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:16 smithi022 ceph-mon[37330]: pgmap v98: 148 pgs: 148 active+clean; 1.8 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 852 B/s rd, 117 MiB/s wr, 1.87k op/s 2024-09-11T07:29:16.893 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:16.936 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:16.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:16.937 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:17.038 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:17.066 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:17.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:17.067 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:17.168 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:17.209 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:17.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:17.210 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:17.310 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:17.336 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:17.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:17.337 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:17.437 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:17.465 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:17.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:17.466 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:17.566 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:17.584 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 15 15 36416 36401 151.637 115.562 0.00668506 0.00658899 2024-09-11T07:29:17.603 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:17.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:17.604 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:17.704 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:17.745 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:17.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:17.746 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:17.847 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:17.888 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:17.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:17.889 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:17.988 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:29:17.990 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:18.017 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:18.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:18.018 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:18.119 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:18.147 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:18.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:18.148 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:18.249 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:18.274 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:18.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:18.275 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:18.375 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:18.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:18.419 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:18.419 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:18.520 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:18.548 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:18.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:18.549 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:18.584 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 16 16 38286 38270 149.46 116.812 0.00686359 0.00668514 2024-09-11T07:29:18.649 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:18.697 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:18.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:18.698 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:18.799 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:18.840 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:18.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:18.841 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:18.942 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:18.966 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:18.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:18.967 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:19.068 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:19.093 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:19.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:19.094 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:19.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:19 smithi022 ceph-mon[29462]: pgmap v99: 148 pgs: 148 active+clean; 1.8 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 642 B/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-11T07:29:19.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:19 smithi022 ceph-mon[37330]: pgmap v99: 148 pgs: 148 active+clean; 1.8 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 642 B/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-11T07:29:19.195 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:19.224 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:19.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:19.225 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:19.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:19 smithi104 ceph-mon[33507]: pgmap v99: 148 pgs: 148 active+clean; 1.8 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 642 B/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-11T07:29:19.280 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:29:19 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:29:19.319 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:29:19.326 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:19.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:19.405 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:19.405 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:19.506 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:19.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:19.549 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:19.549 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:19.584 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 17 16 40004 39988 146.983 107.375 0.0123593 0.00679798 2024-09-11T07:29:19.650 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:19.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:19.693 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:19.693 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:19.794 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:19.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:19.830 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:19.830 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:19.931 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:19.963 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:19.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:19.964 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:20.047 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is unset 2024-09-11T07:29:20.066 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:20.092 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:20.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:20.093 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:20.194 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:20.236 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:20.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:20.238 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:20.300 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:29:20.300 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:29:20.301 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:20 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:29:20.301 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:20 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/260260974' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:29:20.301 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:20 smithi022 ceph-mon[29462]: from='client.24779 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:29:20.301 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:20 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:29:20.301 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:20 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/260260974' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:29:20.301 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:20 smithi022 ceph-mon[37330]: from='client.24779 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:29:20.338 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:20.367 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:20.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:20.368 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:20.469 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:20.509 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:20.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:20.510 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:20.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:20 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:29:20.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:20 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/260260974' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:29:20.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:20 smithi104 ceph-mon[33507]: from='client.24779 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:29:20.584 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 18 16 41692 41676 144.678 105.5 0.0133577 0.00690601 2024-09-11T07:29:20.611 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:20.652 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:20.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:20.653 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:20.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:29:20 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:29:20] "GET /metrics HTTP/1.1" 200 222927 "" "Prometheus/2.43.0" 2024-09-11T07:29:20.754 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:20.799 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:20.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:20.800 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:20.901 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:20.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:20.939 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:20.939 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:21.040 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:21.067 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:21.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:21.068 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:21.169 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:21.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:21.216 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:21.216 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:21.317 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:21.345 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:21.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:21.346 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:21.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:21 smithi022 ceph-mon[29462]: pgmap v100: 148 pgs: 148 active+clean; 2.4 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1.0 KiB/s rd, 136 MiB/s wr, 2.17k op/s 2024-09-11T07:29:21.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:21 smithi022 ceph-mon[29462]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:29:21.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:21 smithi022 ceph-mon[29462]: from='client.24779 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:29:21.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:21 smithi022 ceph-mon[29462]: osdmap e104: 8 total, 8 up, 8 in 2024-09-11T07:29:21.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:21 smithi022 ceph-mon[29462]: osdmap e105: 8 total, 8 up, 8 in 2024-09-11T07:29:21.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:21 smithi022 ceph-mon[29462]: 6.16 scrub starts 2024-09-11T07:29:21.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:21 smithi022 ceph-mon[29462]: 6.16 scrub ok 2024-09-11T07:29:21.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:21 smithi022 ceph-mon[29462]: 4.5 scrub starts 2024-09-11T07:29:21.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:21 smithi022 ceph-mon[29462]: 4.5 scrub ok 2024-09-11T07:29:21.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:21 smithi022 ceph-mon[37330]: pgmap v100: 148 pgs: 148 active+clean; 2.4 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1.0 KiB/s rd, 136 MiB/s wr, 2.17k op/s 2024-09-11T07:29:21.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:21 smithi022 ceph-mon[37330]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:29:21.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:21 smithi022 ceph-mon[37330]: from='client.24779 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:29:21.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:21 smithi022 ceph-mon[37330]: osdmap e104: 8 total, 8 up, 8 in 2024-09-11T07:29:21.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:21 smithi022 ceph-mon[37330]: osdmap e105: 8 total, 8 up, 8 in 2024-09-11T07:29:21.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:21 smithi022 ceph-mon[37330]: 6.16 scrub starts 2024-09-11T07:29:21.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:21 smithi022 ceph-mon[37330]: 6.16 scrub ok 2024-09-11T07:29:21.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:21 smithi022 ceph-mon[37330]: 4.5 scrub starts 2024-09-11T07:29:21.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:21 smithi022 ceph-mon[37330]: 4.5 scrub ok 2024-09-11T07:29:21.446 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:21.476 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:21.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:21.477 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:21 smithi104 ceph-mon[33507]: pgmap v100: 148 pgs: 148 active+clean; 2.4 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1.0 KiB/s rd, 136 MiB/s wr, 2.17k op/s 2024-09-11T07:29:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:21 smithi104 ceph-mon[33507]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:29:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:21 smithi104 ceph-mon[33507]: from='client.24779 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:29:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:21 smithi104 ceph-mon[33507]: osdmap e104: 8 total, 8 up, 8 in 2024-09-11T07:29:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:21 smithi104 ceph-mon[33507]: osdmap e105: 8 total, 8 up, 8 in 2024-09-11T07:29:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:21 smithi104 ceph-mon[33507]: 6.16 scrub starts 2024-09-11T07:29:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:21 smithi104 ceph-mon[33507]: 6.16 scrub ok 2024-09-11T07:29:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:21 smithi104 ceph-mon[33507]: 4.5 scrub starts 2024-09-11T07:29:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:21 smithi104 ceph-mon[33507]: 4.5 scrub ok 2024-09-11T07:29:21.579 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:21.584 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 19 16 43528 43512 143.102 114.75 0.00924309 0.00698237 2024-09-11T07:29:21.618 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:21.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:21.619 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:21.720 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:21.744 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:21.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:21.745 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:21.846 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:21.875 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:21.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:21.876 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:21.977 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:22.022 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:22.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:22.023 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:22.123 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:22.152 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:22.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:22.153 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:22.209 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-11T07:29:22.254 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:22.296 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:22.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:22.297 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:22.398 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:22.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:22 smithi022 ceph-mon[29462]: 2.11 scrub starts 2024-09-11T07:29:22.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:22 smithi022 ceph-mon[29462]: 2.11 scrub ok 2024-09-11T07:29:22.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:22 smithi022 ceph-mon[37330]: 2.11 scrub starts 2024-09-11T07:29:22.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:22 smithi022 ceph-mon[37330]: 2.11 scrub ok 2024-09-11T07:29:22.423 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:22.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:22.424 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:22.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:22 smithi104 ceph-mon[33507]: 2.11 scrub starts 2024-09-11T07:29:22.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:22 smithi104 ceph-mon[33507]: 2.11 scrub ok 2024-09-11T07:29:22.525 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:22.551 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:22.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:22.552 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:22.585 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:2024-09-11T07:29:22.585074+0000 min lat: 0.00117769 max lat: 0.0879694 avg lat: 0.00707918 2024-09-11T07:29:22.586 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T07:29:22.586 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 20 16 45195 45179 141.147 104.188 0.00639977 0.00707918 2024-09-11T07:29:22.653 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:22.683 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:29:22.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:22.713 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:22.713 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:22.814 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:22.842 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:22.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:22.843 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:22.944 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:22.967 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:22.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:22.968 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:23.068 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:23.085 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:29:23.086 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:23 smithi022 ceph-mon[29462]: 2.1b scrub starts 2024-09-11T07:29:23.086 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:23 smithi022 ceph-mon[29462]: 2.1b scrub ok 2024-09-11T07:29:23.086 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:23 smithi022 ceph-mon[29462]: 5.1b scrub starts 2024-09-11T07:29:23.086 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:23 smithi022 ceph-mon[29462]: 5.1b scrub ok 2024-09-11T07:29:23.086 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:23 smithi022 ceph-mon[29462]: pgmap v103: 148 pgs: 148 active+clean; 2.4 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 639 B/s rd, 74 MiB/s wr, 1.19k op/s 2024-09-11T07:29:23.086 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:23 smithi022 ceph-mon[29462]: 6.12 scrub starts 2024-09-11T07:29:23.086 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:23 smithi022 ceph-mon[29462]: 6.12 scrub ok 2024-09-11T07:29:23.086 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:23 smithi022 ceph-mon[29462]: 4.1b scrub starts 2024-09-11T07:29:23.086 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:23 smithi022 ceph-mon[29462]: 4.1b scrub ok 2024-09-11T07:29:23.087 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:23 smithi022 ceph-mon[29462]: 2.1a scrub starts 2024-09-11T07:29:23.087 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:23 smithi022 ceph-mon[29462]: 2.1a scrub ok 2024-09-11T07:29:23.087 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:29:23.087 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:23 smithi022 ceph-mon[37330]: 2.1b scrub starts 2024-09-11T07:29:23.087 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:23 smithi022 ceph-mon[37330]: 2.1b scrub ok 2024-09-11T07:29:23.087 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:23 smithi022 ceph-mon[37330]: 5.1b scrub starts 2024-09-11T07:29:23.087 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:23 smithi022 ceph-mon[37330]: 5.1b scrub ok 2024-09-11T07:29:23.087 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:23 smithi022 ceph-mon[37330]: pgmap v103: 148 pgs: 148 active+clean; 2.4 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 639 B/s rd, 74 MiB/s wr, 1.19k op/s 2024-09-11T07:29:23.087 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:23 smithi022 ceph-mon[37330]: 6.12 scrub starts 2024-09-11T07:29:23.087 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:23 smithi022 ceph-mon[37330]: 6.12 scrub ok 2024-09-11T07:29:23.088 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:23 smithi022 ceph-mon[37330]: 4.1b scrub starts 2024-09-11T07:29:23.088 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:23 smithi022 ceph-mon[37330]: 4.1b scrub ok 2024-09-11T07:29:23.088 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:23 smithi022 ceph-mon[37330]: 2.1a scrub starts 2024-09-11T07:29:23.088 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:23 smithi022 ceph-mon[37330]: 2.1a scrub ok 2024-09-11T07:29:23.088 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:29:23.094 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:29:23.095 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:23.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:23.096 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:23.197 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:23.238 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:23.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:23.240 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:23.340 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:23.366 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:23.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:23.367 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:23.468 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:23.496 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:23.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:23.497 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:23.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:23 smithi104 ceph-mon[33507]: 2.1b scrub starts 2024-09-11T07:29:23.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:23 smithi104 ceph-mon[33507]: 2.1b scrub ok 2024-09-11T07:29:23.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:23 smithi104 ceph-mon[33507]: 5.1b scrub starts 2024-09-11T07:29:23.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:23 smithi104 ceph-mon[33507]: 5.1b scrub ok 2024-09-11T07:29:23.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:23 smithi104 ceph-mon[33507]: pgmap v103: 148 pgs: 148 active+clean; 2.4 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 639 B/s rd, 74 MiB/s wr, 1.19k op/s 2024-09-11T07:29:23.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:23 smithi104 ceph-mon[33507]: 6.12 scrub starts 2024-09-11T07:29:23.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:23 smithi104 ceph-mon[33507]: 6.12 scrub ok 2024-09-11T07:29:23.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:23 smithi104 ceph-mon[33507]: 4.1b scrub starts 2024-09-11T07:29:23.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:23 smithi104 ceph-mon[33507]: 4.1b scrub ok 2024-09-11T07:29:23.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:23 smithi104 ceph-mon[33507]: 2.1a scrub starts 2024-09-11T07:29:23.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:23 smithi104 ceph-mon[33507]: 2.1a scrub ok 2024-09-11T07:29:23.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:29:23.585 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 21 16 46684 46668 138.858 93.0625 0.00496621 0.00719557 2024-09-11T07:29:23.598 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:23.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:23.620 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:23.620 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:23.721 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:23.758 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T07:29:23.759 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:23.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:23.760 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:23.861 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:23.887 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:23.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:23.888 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:23.988 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:24.014 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:24.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:24.015 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:24.116 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:24.172 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:24.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:24.173 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:24.274 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:24.303 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:24.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:24.304 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:24.405 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:24.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:24 smithi022 ceph-mon[29462]: 2.1e scrub starts 2024-09-11T07:29:24.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:24 smithi022 ceph-mon[29462]: 2.1e scrub ok 2024-09-11T07:29:24.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:24 smithi022 ceph-mon[29462]: 6.15 scrub starts 2024-09-11T07:29:24.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:24 smithi022 ceph-mon[29462]: 6.15 scrub ok 2024-09-11T07:29:24.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:24 smithi022 ceph-mon[29462]: from='client.24808 v1:172.21.15.22:0/486736823' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:29:24.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:24 smithi022 ceph-mon[37330]: 2.1e scrub starts 2024-09-11T07:29:24.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:24 smithi022 ceph-mon[37330]: 2.1e scrub ok 2024-09-11T07:29:24.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:24 smithi022 ceph-mon[37330]: 6.15 scrub starts 2024-09-11T07:29:24.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:24 smithi022 ceph-mon[37330]: 6.15 scrub ok 2024-09-11T07:29:24.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:24 smithi022 ceph-mon[37330]: from='client.24808 v1:172.21.15.22:0/486736823' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:29:24.426 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:24.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:24.427 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:24.528 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:24.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:29:24 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:29:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:29:24.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:24 smithi104 ceph-mon[33507]: 2.1e scrub starts 2024-09-11T07:29:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:24 smithi104 ceph-mon[33507]: 2.1e scrub ok 2024-09-11T07:29:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:24 smithi104 ceph-mon[33507]: 6.15 scrub starts 2024-09-11T07:29:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:24 smithi104 ceph-mon[33507]: 6.15 scrub ok 2024-09-11T07:29:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:24 smithi104 ceph-mon[33507]: from='client.24808 v1:172.21.15.22:0/486736823' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:29:24.554 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:24.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:24.555 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:24.586 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 22 16 48049 48033 136.424 85.3125 0.00546828 0.00732529 2024-09-11T07:29:24.656 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:24.700 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:24.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:24.701 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:24.802 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:24.836 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:24.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:24.837 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:24.938 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:24.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:24.971 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:24.971 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:25.072 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:25.099 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:25.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:25.100 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:25.201 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:25.227 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:25.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:25.228 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:25.329 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:25.355 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:25.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:25.356 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:25.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:25 smithi022 ceph-mon[29462]: 5.d scrub starts 2024-09-11T07:29:25.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:25 smithi022 ceph-mon[29462]: 5.d scrub ok 2024-09-11T07:29:25.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:25 smithi022 ceph-mon[29462]: pgmap v104: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 138 MiB/s wr, 2.21k op/s 2024-09-11T07:29:25.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:25 smithi022 ceph-mon[29462]: 2.15 scrub starts 2024-09-11T07:29:25.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:25 smithi022 ceph-mon[29462]: 2.15 scrub ok 2024-09-11T07:29:25.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:25 smithi022 ceph-mon[37330]: 5.d scrub starts 2024-09-11T07:29:25.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:25 smithi022 ceph-mon[37330]: 5.d scrub ok 2024-09-11T07:29:25.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:25 smithi022 ceph-mon[37330]: pgmap v104: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 138 MiB/s wr, 2.21k op/s 2024-09-11T07:29:25.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:25 smithi022 ceph-mon[37330]: 2.15 scrub starts 2024-09-11T07:29:25.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:25 smithi022 ceph-mon[37330]: 2.15 scrub ok 2024-09-11T07:29:25.457 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:25.485 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:25.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:25.486 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:25.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:25 smithi104 ceph-mon[33507]: 5.d scrub starts 2024-09-11T07:29:25.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:25 smithi104 ceph-mon[33507]: 5.d scrub ok 2024-09-11T07:29:25.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:25 smithi104 ceph-mon[33507]: pgmap v104: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 138 MiB/s wr, 2.21k op/s 2024-09-11T07:29:25.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:25 smithi104 ceph-mon[33507]: 2.15 scrub starts 2024-09-11T07:29:25.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:25 smithi104 ceph-mon[33507]: 2.15 scrub ok 2024-09-11T07:29:25.586 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 23 16 49608 49592 134.729 97.4375 0.00847595 0.00741707 2024-09-11T07:29:25.587 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:25.627 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:25.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:25.628 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:25.729 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:25.767 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:25.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:25.768 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:25.822 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:29:25.869 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:25.893 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:25.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:25.894 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:25.995 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:26.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:26.044 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:26.044 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:26.145 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:26.171 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:26.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:26.172 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:26.273 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:26.317 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:26.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:26.318 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:26.419 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:26.446 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:26.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:26.447 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:26.548 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:26.575 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:26.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:26.576 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:26.586 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 24 16 51235 51219 133.352 101.688 0.0089694 0.00749363 2024-09-11T07:29:26.677 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:26.717 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:26.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:26.718 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:26.819 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:26.846 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:26.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:26.847 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:26.948 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:26.989 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:26.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:26.990 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:27.091 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:27.119 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:27.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:27.120 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:27.166 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-09-11T07:29:21.965928+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2549684444,"num_objects":39092,"num_object_clones":0,"num_object_copies":117276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39092,"num_whiteouts":0,"num_read":1014,"num_read_kb":871,"num_write":39439,"num_write_kb":2491015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":590588,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39501,"ondisk_log_size":39501,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9776876,"kb_used_data":7323180,"kb_used_omap":13,"kb_used_meta":2453618,"kb_avail":740184340,"statfs":{"total":767960285184,"available":757948764160,"internally_reserved":0,"allocated":7498936320,"data_stored":7495394993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2512505536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,11,4,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":621281280,"num_objects":9480,"num_object_clones":0,"num_object_copies":28440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9480,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":9480,"num_write_kb":606720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002524"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.448321+0000","last_change":"2024-09-11T07:28:32.778166+0000","last_active":"2024-09-11T07:29:10.448321+0000","last_peered":"2024-09-11T07:29:10.448321+0000","last_clean":"2024-09-11T07:29:10.448321+0000","last_became_active":"2024-09-11T07:26:37.863165+0000","last_became_peered":"2024-09-11T07:26:37.863165+0000","last_unstale":"2024-09-11T07:29:10.448321+0000","last_undegraded":"2024-09-11T07:29:10.448321+0000","last_fullsized":"2024-09-11T07:29:10.448321+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:32.778109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:32.778109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:47.483048+0000","scrub_duration":0.001472253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":65,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.448073+0000","last_change":"2024-09-11T07:27:51.197966+0000","last_active":"2024-09-11T07:29:10.448073+0000","last_peered":"2024-09-11T07:29:10.448073+0000","last_clean":"2024-09-11T07:29:10.448073+0000","last_became_active":"2024-09-11T07:26:31.421005+0000","last_became_peered":"2024-09-11T07:26:31.421005+0000","last_unstale":"2024-09-11T07:29:10.448073+0000","last_undegraded":"2024-09-11T07:29:10.448073+0000","last_fullsized":"2024-09-11T07:29:10.448073+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:51.197903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:51.197903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001600113,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"82'22","reported_seq":95,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446633+0000","last_change":"2024-09-11T07:27:50.756777+0000","last_active":"2024-09-11T07:29:10.446633+0000","last_peered":"2024-09-11T07:29:10.446633+0000","last_clean":"2024-09-11T07:29:10.446633+0000","last_became_active":"2024-09-11T07:26:33.544627+0000","last_became_peered":"2024-09-11T07:26:33.544627+0000","last_unstale":"2024-09-11T07:29:10.446633+0000","last_undegraded":"2024-09-11T07:29:10.446633+0000","last_fullsized":"2024-09-11T07:29:10.446633+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.756707+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.257406+0000","scrub_duration":0.0049180669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":61,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.441351+0000","last_change":"2024-09-11T07:28:17.046764+0000","last_active":"2024-09-11T07:29:10.441351+0000","last_peered":"2024-09-11T07:29:10.441351+0000","last_clean":"2024-09-11T07:29:10.441351+0000","last_became_active":"2024-09-11T07:26:35.556296+0000","last_became_peered":"2024-09-11T07:26:35.556296+0000","last_unstale":"2024-09-11T07:29:10.441351+0000","last_undegraded":"2024-09-11T07:29:10.441351+0000","last_fullsized":"2024-09-11T07:29:10.441351+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:17.046711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:17.046711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:21.028176+0000","scrub_duration":0.0011666510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":65,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.449190+0000","last_change":"2024-09-11T07:27:36.776948+0000","last_active":"2024-09-11T07:29:10.449190+0000","last_peered":"2024-09-11T07:29:10.449190+0000","last_clean":"2024-09-11T07:29:10.449190+0000","last_became_active":"2024-09-11T07:26:31.420178+0000","last_became_peered":"2024-09-11T07:26:31.420178+0000","last_unstale":"2024-09-11T07:29:10.449190+0000","last_undegraded":"2024-09-11T07:29:10.449190+0000","last_fullsized":"2024-09-11T07:29:10.449190+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:36.776883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:36.776883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017449290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"86'13","reported_seq":83,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.441406+0000","last_change":"2024-09-11T07:28:15.025864+0000","last_active":"2024-09-11T07:29:10.441406+0000","last_peered":"2024-09-11T07:29:10.441406+0000","last_clean":"2024-09-11T07:29:10.441406+0000","last_became_active":"2024-09-11T07:26:33.540732+0000","last_became_peered":"2024-09-11T07:26:33.540732+0000","last_unstale":"2024-09-11T07:29:10.441406+0000","last_undegraded":"2024-09-11T07:29:10.441406+0000","last_fullsized":"2024-09-11T07:29:10.441406+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'13","last_scrub_stamp":"2024-09-11T07:28:15.025757+0000","last_deep_scrub":"86'13","last_deep_scrub_stamp":"2024-09-11T07:28:15.025757+0000","last_clean_scrub_stamp":"2024-09-11T07:28:15.025757+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:26.818521+0000","scrub_duration":0.0039188890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":61,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446165+0000","last_change":"2024-09-11T07:27:50.648473+0000","last_active":"2024-09-11T07:29:10.446165+0000","last_peered":"2024-09-11T07:29:10.446165+0000","last_clean":"2024-09-11T07:29:10.446165+0000","last_became_active":"2024-09-11T07:26:35.561096+0000","last_became_peered":"2024-09-11T07:26:35.561096+0000","last_unstale":"2024-09-11T07:29:10.446165+0000","last_undegraded":"2024-09-11T07:29:10.446165+0000","last_fullsized":"2024-09-11T07:29:10.446165+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.648420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001391446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.441471+0000","last_change":"2024-09-11T07:28:19.063739+0000","last_active":"2024-09-11T07:29:10.441471+0000","last_peered":"2024-09-11T07:29:10.441471+0000","last_clean":"2024-09-11T07:29:10.441471+0000","last_became_active":"2024-09-11T07:26:37.566910+0000","last_became_peered":"2024-09-11T07:26:37.566910+0000","last_unstale":"2024-09-11T07:29:10.441471+0000","last_undegraded":"2024-09-11T07:29:10.441471+0000","last_fullsized":"2024-09-11T07:29:10.441471+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:19.063651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:19.063651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:42.984286+0000","scrub_duration":0.0019504990000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":65,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.444732+0000","last_change":"2024-09-11T07:28:23.003084+0000","last_active":"2024-09-11T07:29:10.444732+0000","last_peered":"2024-09-11T07:29:10.444732+0000","last_clean":"2024-09-11T07:29:10.444732+0000","last_became_active":"2024-09-11T07:26:31.422772+0000","last_became_peered":"2024-09-11T07:26:31.422772+0000","last_unstale":"2024-09-11T07:29:10.444732+0000","last_undegraded":"2024-09-11T07:29:10.444732+0000","last_fullsized":"2024-09-11T07:29:10.444732+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:23.003028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:23.003028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.008955+0000","scrub_duration":0.0013267520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"82'7","reported_seq":75,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.441877+0000","last_change":"2024-09-11T07:27:52.097170+0000","last_active":"2024-09-11T07:29:10.441877+0000","last_peered":"2024-09-11T07:29:10.441877+0000","last_clean":"2024-09-11T07:29:10.441877+0000","last_became_active":"2024-09-11T07:26:33.540351+0000","last_became_peered":"2024-09-11T07:26:33.540351+0000","last_unstale":"2024-09-11T07:29:10.441877+0000","last_undegraded":"2024-09-11T07:29:10.441877+0000","last_fullsized":"2024-09-11T07:29:10.441877+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'7","last_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_deep_scrub":"82'7","last_deep_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_clean_scrub_stamp":"2024-09-11T07:27:52.097071+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025786670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":61,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.443024+0000","last_change":"2024-09-11T07:28:30.973423+0000","last_active":"2024-09-11T07:29:10.443024+0000","last_peered":"2024-09-11T07:29:10.443024+0000","last_clean":"2024-09-11T07:29:10.443024+0000","last_became_active":"2024-09-11T07:26:35.563536+0000","last_became_peered":"2024-09-11T07:26:35.563536+0000","last_unstale":"2024-09-11T07:29:10.443024+0000","last_undegraded":"2024-09-11T07:29:10.443024+0000","last_fullsized":"2024-09-11T07:29:10.443024+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:30.973323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.973323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:30.973323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:40.899348+0000","scrub_duration":0.0014944509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446479+0000","last_change":"2024-09-11T07:28:24.869360+0000","last_active":"2024-09-11T07:29:10.446479+0000","last_peered":"2024-09-11T07:29:10.446479+0000","last_clean":"2024-09-11T07:29:10.446479+0000","last_became_active":"2024-09-11T07:26:37.561758+0000","last_became_peered":"2024-09-11T07:26:37.561758+0000","last_unstale":"2024-09-11T07:29:10.446479+0000","last_undegraded":"2024-09-11T07:29:10.446479+0000","last_fullsized":"2024-09-11T07:29:10.446479+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:24.869307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:24.869307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:43.695642+0000","scrub_duration":0.001382999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446366+0000","last_change":"2024-09-11T07:29:10.007726+0000","last_active":"2024-09-11T07:29:10.446366+0000","last_peered":"2024-09-11T07:29:10.446366+0000","last_clean":"2024-09-11T07:29:10.446366+0000","last_became_active":"2024-09-11T07:29:10.007439+0000","last_became_peered":"2024-09-11T07:29:10.007439+0000","last_unstale":"2024-09-11T07:29:10.446366+0000","last_undegraded":"2024-09-11T07:29:10.446366+0000","last_fullsized":"2024-09-11T07:29:10.446366+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_clean_scrub_stamp":"2024-09-11T07:27:52.057959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.972332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"64'9","reported_seq":79,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.441697+0000","last_change":"2024-09-11T07:27:47.108704+0000","last_active":"2024-09-11T07:29:10.441697+0000","last_peered":"2024-09-11T07:29:10.441697+0000","last_clean":"2024-09-11T07:29:10.441697+0000","last_became_active":"2024-09-11T07:26:33.544386+0000","last_became_peered":"2024-09-11T07:26:33.544386+0000","last_unstale":"2024-09-11T07:29:10.441697+0000","last_undegraded":"2024-09-11T07:29:10.441697+0000","last_fullsized":"2024-09-11T07:29:10.441697+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:27:47.108645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.108645+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024650560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":61,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446577+0000","last_change":"2024-09-11T07:27:57.092029+0000","last_active":"2024-09-11T07:29:10.446577+0000","last_peered":"2024-09-11T07:29:10.446577+0000","last_clean":"2024-09-11T07:29:10.446577+0000","last_became_active":"2024-09-11T07:26:35.569630+0000","last_became_peered":"2024-09-11T07:26:35.569630+0000","last_unstale":"2024-09-11T07:29:10.446577+0000","last_undegraded":"2024-09-11T07:29:10.446577+0000","last_fullsized":"2024-09-11T07:29:10.446577+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:57.091958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.091958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001785872,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.442371+0000","last_change":"2024-09-11T07:28:09.333866+0000","last_active":"2024-09-11T07:29:10.442371+0000","last_peered":"2024-09-11T07:29:10.442371+0000","last_clean":"2024-09-11T07:29:10.442371+0000","last_became_active":"2024-09-11T07:26:37.565475+0000","last_became_peered":"2024-09-11T07:26:37.565475+0000","last_unstale":"2024-09-11T07:29:10.442371+0000","last_undegraded":"2024-09-11T07:29:10.442371+0000","last_fullsized":"2024-09-11T07:29:10.442371+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:09.333804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:09.333804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003498438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"103'2423","reported_seq":2442,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-11T07:29:18.910208+0000","last_change":"2024-09-11T07:28:50.002862+0000","last_active":"2024-09-11T07:29:18.910208+0000","last_peered":"2024-09-11T07:29:18.910208+0000","last_clean":"2024-09-11T07:29:18.910208+0000","last_became_active":"2024-09-11T07:28:50.002582+0000","last_became_peered":"2024-09-11T07:28:50.002582+0000","last_unstale":"2024-09-11T07:29:18.910208+0000","last_undegraded":"2024-09-11T07:29:18.910208+0000","last_fullsized":"2024-09-11T07:29:18.910208+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":2423,"log_dups_size":0,"ondisk_log_size":2423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:30:12.931996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":158793728,"num_objects":2423,"num_object_clones":0,"num_object_copies":7269,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2423,"num_write_kb":155072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":119,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.442154+0000","last_change":"2024-09-11T07:28:09.027694+0000","last_active":"2024-09-11T07:29:10.442154+0000","last_peered":"2024-09-11T07:29:10.442154+0000","last_clean":"2024-09-11T07:29:10.442154+0000","last_became_active":"2024-09-11T07:28:09.018213+0000","last_became_peered":"2024-09-11T07:28:09.018213+0000","last_unstale":"2024-09-11T07:29:10.442154+0000","last_undegraded":"2024-09-11T07:29:10.442154+0000","last_fullsized":"2024-09-11T07:29:10.442154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-09-11T07:28:04.137453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:04.137453+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001914083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":65,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.443682+0000","last_change":"2024-09-11T07:28:11.897385+0000","last_active":"2024-09-11T07:29:10.443682+0000","last_peered":"2024-09-11T07:29:10.443682+0000","last_clean":"2024-09-11T07:29:10.443682+0000","last_became_active":"2024-09-11T07:26:31.419488+0000","last_became_peered":"2024-09-11T07:26:31.419488+0000","last_unstale":"2024-09-11T07:29:10.443682+0000","last_undegraded":"2024-09-11T07:29:10.443682+0000","last_fullsized":"2024-09-11T07:29:10.443682+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:11.897319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:11.897319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:18.864403+0000","scrub_duration":0.0013939480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":61,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.449905+0000","last_change":"2024-09-11T07:28:26.841381+0000","last_active":"2024-09-11T07:29:10.449905+0000","last_peered":"2024-09-11T07:29:10.449905+0000","last_clean":"2024-09-11T07:29:10.449905+0000","last_became_active":"2024-09-11T07:26:35.557989+0000","last_became_peered":"2024-09-11T07:26:35.557989+0000","last_unstale":"2024-09-11T07:29:10.449905+0000","last_undegraded":"2024-09-11T07:29:10.449905+0000","last_fullsized":"2024-09-11T07:29:10.449905+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:26.841326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:26.841326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:42.433747+0000","scrub_duration":0.0015001350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.441065+0000","last_change":"2024-09-11T07:27:57.274611+0000","last_active":"2024-09-11T07:29:10.441065+0000","last_peered":"2024-09-11T07:29:10.441065+0000","last_clean":"2024-09-11T07:29:10.441065+0000","last_became_active":"2024-09-11T07:26:37.564554+0000","last_became_peered":"2024-09-11T07:26:37.564554+0000","last_unstale":"2024-09-11T07:29:10.441065+0000","last_undegraded":"2024-09-11T07:29:10.441065+0000","last_fullsized":"2024-09-11T07:29:10.441065+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:57.274554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.274554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001419584,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"103'2439","reported_seq":2458,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-11T07:29:18.937124+0000","last_change":"2024-09-11T07:28:50.000249+0000","last_active":"2024-09-11T07:29:18.937124+0000","last_peered":"2024-09-11T07:29:18.937124+0000","last_clean":"2024-09-11T07:29:18.937124+0000","last_became_active":"2024-09-11T07:28:49.999894+0000","last_became_peered":"2024-09-11T07:28:49.999894+0000","last_unstale":"2024-09-11T07:29:18.937124+0000","last_undegraded":"2024-09-11T07:29:18.937124+0000","last_fullsized":"2024-09-11T07:29:18.937124+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":2439,"log_dups_size":0,"ondisk_log_size":2439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:51.279633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":159842304,"num_objects":2439,"num_object_clones":0,"num_object_copies":7317,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2439,"num_write_kb":156096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":78,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.441916+0000","last_change":"2024-09-11T07:28:21.039929+0000","last_active":"2024-09-11T07:29:10.441916+0000","last_peered":"2024-09-11T07:29:10.441916+0000","last_clean":"2024-09-11T07:29:10.441916+0000","last_became_active":"2024-09-11T07:26:33.535411+0000","last_became_peered":"2024-09-11T07:26:33.535411+0000","last_unstale":"2024-09-11T07:29:10.441916+0000","last_undegraded":"2024-09-11T07:29:10.441916+0000","last_fullsized":"2024-09-11T07:29:10.441916+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:28:21.039828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:21.039828+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:24.128795+0000","scrub_duration":0.0021618829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":65,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446440+0000","last_change":"2024-09-11T07:28:18.878004+0000","last_active":"2024-09-11T07:29:10.446440+0000","last_peered":"2024-09-11T07:29:10.446440+0000","last_clean":"2024-09-11T07:29:10.446440+0000","last_became_active":"2024-09-11T07:26:31.418178+0000","last_became_peered":"2024-09-11T07:26:31.418178+0000","last_unstale":"2024-09-11T07:29:10.446440+0000","last_undegraded":"2024-09-11T07:29:10.446440+0000","last_fullsized":"2024-09-11T07:29:10.446440+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:18.877940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:18.877940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:21.852832+0000","scrub_duration":0.001611924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"64'8","reported_seq":102,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-11T07:29:11.445534+0000","last_change":"2024-09-11T07:27:47.127485+0000","last_active":"2024-09-11T07:29:11.445534+0000","last_peered":"2024-09-11T07:29:11.445534+0000","last_clean":"2024-09-11T07:29:11.445534+0000","last_became_active":"2024-09-11T07:26:35.564531+0000","last_became_peered":"2024-09-11T07:26:35.564531+0000","last_unstale":"2024-09-11T07:29:11.445534+0000","last_undegraded":"2024-09-11T07:29:11.445534+0000","last_fullsized":"2024-09-11T07:29:11.445534+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:27:47.127428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.127428+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014091329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.442551+0000","last_change":"2024-09-11T07:28:31.007700+0000","last_active":"2024-09-11T07:29:10.442551+0000","last_peered":"2024-09-11T07:29:10.442551+0000","last_clean":"2024-09-11T07:29:10.442551+0000","last_became_active":"2024-09-11T07:26:37.566919+0000","last_became_peered":"2024-09-11T07:26:37.566919+0000","last_unstale":"2024-09-11T07:29:10.442551+0000","last_undegraded":"2024-09-11T07:29:10.442551+0000","last_fullsized":"2024-09-11T07:29:10.442551+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:31.007651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:31.007651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:49.292980+0000","scrub_duration":0.001233963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"103'2435","reported_seq":2454,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-11T07:29:18.912723+0000","last_change":"2024-09-11T07:28:50.003247+0000","last_active":"2024-09-11T07:29:18.912723+0000","last_peered":"2024-09-11T07:29:18.912723+0000","last_clean":"2024-09-11T07:29:18.912723+0000","last_became_active":"2024-09-11T07:28:50.002911+0000","last_became_peered":"2024-09-11T07:28:50.002911+0000","last_unstale":"2024-09-11T07:29:18.912723+0000","last_undegraded":"2024-09-11T07:29:18.912723+0000","last_fullsized":"2024-09-11T07:29:18.912723+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":2435,"log_dups_size":0,"ondisk_log_size":2435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:52.607485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":159580160,"num_objects":2435,"num_object_clones":0,"num_object_copies":7305,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2435,"num_write_kb":155840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"83'22","reported_seq":95,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446146+0000","last_change":"2024-09-11T07:28:01.719185+0000","last_active":"2024-09-11T07:29:10.446146+0000","last_peered":"2024-09-11T07:29:10.446146+0000","last_clean":"2024-09-11T07:29:10.446146+0000","last_became_active":"2024-09-11T07:26:33.549733+0000","last_became_peered":"2024-09-11T07:26:33.549733+0000","last_unstale":"2024-09-11T07:29:10.446146+0000","last_undegraded":"2024-09-11T07:29:10.446146+0000","last_fullsized":"2024-09-11T07:29:10.446146+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'22","last_scrub_stamp":"2024-09-11T07:28:01.719128+0000","last_deep_scrub":"83'22","last_deep_scrub_stamp":"2024-09-11T07:28:01.719128+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.719128+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:27.431086+0000","scrub_duration":0.0047817240000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":65,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446349+0000","last_change":"2024-09-11T07:27:35.589317+0000","last_active":"2024-09-11T07:29:10.446349+0000","last_peered":"2024-09-11T07:29:10.446349+0000","last_clean":"2024-09-11T07:29:10.446349+0000","last_became_active":"2024-09-11T07:26:31.418478+0000","last_became_peered":"2024-09-11T07:26:31.418478+0000","last_unstale":"2024-09-11T07:29:10.446349+0000","last_undegraded":"2024-09-11T07:29:10.446349+0000","last_fullsized":"2024-09-11T07:29:10.446349+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:35.589251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:35.589251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014929380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":105,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-11T07:29:11.496750+0000","last_change":"2024-09-11T07:27:56.218355+0000","last_active":"2024-09-11T07:29:11.496750+0000","last_peered":"2024-09-11T07:29:11.496750+0000","last_clean":"2024-09-11T07:29:11.496750+0000","last_became_active":"2024-09-11T07:26:35.560406+0000","last_became_peered":"2024-09-11T07:26:35.560406+0000","last_unstale":"2024-09-11T07:29:11.496750+0000","last_undegraded":"2024-09-11T07:29:11.496750+0000","last_fullsized":"2024-09-11T07:29:11.496750+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:27:56.218301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:56.218301+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:21.374837+0000","scrub_duration":0.001687413,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446543+0000","last_change":"2024-09-11T07:28:18.785729+0000","last_active":"2024-09-11T07:29:10.446543+0000","last_peered":"2024-09-11T07:29:10.446543+0000","last_clean":"2024-09-11T07:29:10.446543+0000","last_became_active":"2024-09-11T07:26:37.568005+0000","last_became_peered":"2024-09-11T07:26:37.568005+0000","last_unstale":"2024-09-11T07:29:10.446543+0000","last_undegraded":"2024-09-11T07:29:10.446543+0000","last_fullsized":"2024-09-11T07:29:10.446543+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:18.785667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:18.785667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:39.054254+0000","scrub_duration":0.0013564829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"103'2496","reported_seq":2515,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-11T07:29:18.937747+0000","last_change":"2024-09-11T07:28:50.004962+0000","last_active":"2024-09-11T07:29:18.937747+0000","last_peered":"2024-09-11T07:29:18.937747+0000","last_clean":"2024-09-11T07:29:18.937747+0000","last_became_active":"2024-09-11T07:28:50.004515+0000","last_became_peered":"2024-09-11T07:28:50.004515+0000","last_unstale":"2024-09-11T07:29:18.937747+0000","last_undegraded":"2024-09-11T07:29:18.937747+0000","last_fullsized":"2024-09-11T07:29:18.937747+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":2496,"log_dups_size":0,"ondisk_log_size":2496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:50.097337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163577856,"num_objects":2496,"num_object_clones":0,"num_object_copies":7488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2496,"num_write_kb":159744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"64'11","reported_seq":82,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.442745+0000","last_change":"2024-09-11T07:27:58.158485+0000","last_active":"2024-09-11T07:29:10.442745+0000","last_peered":"2024-09-11T07:29:10.442745+0000","last_clean":"2024-09-11T07:29:10.442745+0000","last_became_active":"2024-09-11T07:26:33.540777+0000","last_became_peered":"2024-09-11T07:26:33.540777+0000","last_unstale":"2024-09-11T07:29:10.442745+0000","last_undegraded":"2024-09-11T07:29:10.442745+0000","last_fullsized":"2024-09-11T07:29:10.442745+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:27:58.158420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.158420+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.832014+0000","scrub_duration":0.0021737459999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":75,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.442728+0000","last_change":"2024-09-11T07:28:48.208248+0000","last_active":"2024-09-11T07:29:10.442728+0000","last_peered":"2024-09-11T07:29:10.442728+0000","last_clean":"2024-09-11T07:29:10.442728+0000","last_became_active":"2024-09-11T07:26:31.418238+0000","last_became_peered":"2024-09-11T07:26:31.418238+0000","last_unstale":"2024-09-11T07:29:10.442728+0000","last_undegraded":"2024-09-11T07:29:10.442728+0000","last_fullsized":"2024-09-11T07:29:10.442728+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.208184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.208184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:30:00.127894+0000","scrub_duration":0.001606994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":61,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446964+0000","last_change":"2024-09-11T07:28:00.005119+0000","last_active":"2024-09-11T07:29:10.446964+0000","last_peered":"2024-09-11T07:29:10.446964+0000","last_clean":"2024-09-11T07:29:10.446964+0000","last_became_active":"2024-09-11T07:26:35.569081+0000","last_became_peered":"2024-09-11T07:26:35.569081+0000","last_unstale":"2024-09-11T07:29:10.446964+0000","last_undegraded":"2024-09-11T07:29:10.446964+0000","last_fullsized":"2024-09-11T07:29:10.446964+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:00.005031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.005031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001403533,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.448882+0000","last_change":"2024-09-11T07:27:46.752272+0000","last_active":"2024-09-11T07:29:10.448882+0000","last_peered":"2024-09-11T07:29:10.448882+0000","last_clean":"2024-09-11T07:29:10.448882+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:29:10.448882+0000","last_undegraded":"2024-09-11T07:29:10.448882+0000","last_fullsized":"2024-09-11T07:29:10.448882+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:46.752181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:46.752181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001508782,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2","version":"103'2479","reported_seq":2496,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-11T07:29:18.941711+0000","last_change":"2024-09-11T07:28:50.000121+0000","last_active":"2024-09-11T07:29:18.941711+0000","last_peered":"2024-09-11T07:29:18.941711+0000","last_clean":"2024-09-11T07:29:18.941711+0000","last_became_active":"2024-09-11T07:28:49.999841+0000","last_became_peered":"2024-09-11T07:28:49.999841+0000","last_unstale":"2024-09-11T07:29:18.941711+0000","last_undegraded":"2024-09-11T07:29:18.941711+0000","last_fullsized":"2024-09-11T07:29:18.941711+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":2479,"log_dups_size":0,"ondisk_log_size":2479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:30:12.057468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":162463744,"num_objects":2479,"num_object_clones":0,"num_object_copies":7437,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2479,"num_write_kb":158656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"64'14","reported_seq":84,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.441578+0000","last_change":"2024-09-11T07:28:20.085403+0000","last_active":"2024-09-11T07:29:10.441578+0000","last_peered":"2024-09-11T07:29:10.441578+0000","last_clean":"2024-09-11T07:29:10.441578+0000","last_became_active":"2024-09-11T07:26:33.537850+0000","last_became_peered":"2024-09-11T07:26:33.537850+0000","last_unstale":"2024-09-11T07:29:10.441578+0000","last_undegraded":"2024-09-11T07:29:10.441578+0000","last_fullsized":"2024-09-11T07:29:10.441578+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-09-11T07:28:20.085339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:20.085339+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:27.624554+0000","scrub_duration":0.0019854640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":75,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.439513+0000","last_change":"2024-09-11T07:28:09.015415+0000","last_active":"2024-09-11T07:29:10.439513+0000","last_peered":"2024-09-11T07:29:10.439513+0000","last_clean":"2024-09-11T07:29:10.439513+0000","last_became_active":"2024-09-11T07:28:09.014995+0000","last_became_peered":"2024-09-11T07:28:09.014995+0000","last_unstale":"2024-09-11T07:29:10.439513+0000","last_undegraded":"2024-09-11T07:29:10.439513+0000","last_fullsized":"2024-09-11T07:29:10.439513+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:39.014840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:39.014840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032789239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"84'9","reported_seq":103,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-11T07:29:11.444314+0000","last_change":"2024-09-11T07:28:05.653289+0000","last_active":"2024-09-11T07:29:11.444314+0000","last_peered":"2024-09-11T07:29:11.444314+0000","last_clean":"2024-09-11T07:29:11.444314+0000","last_became_active":"2024-09-11T07:26:35.562997+0000","last_became_peered":"2024-09-11T07:26:35.562997+0000","last_unstale":"2024-09-11T07:29:11.444314+0000","last_undegraded":"2024-09-11T07:29:11.444314+0000","last_fullsized":"2024-09-11T07:29:11.444314+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-11T07:28:05.653237+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-11T07:28:05.653237+0000","last_clean_scrub_stamp":"2024-09-11T07:28:05.653237+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:22.713095+0000","scrub_duration":0.0027957030000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.443402+0000","last_change":"2024-09-11T07:28:34.041835+0000","last_active":"2024-09-11T07:29:10.443402+0000","last_peered":"2024-09-11T07:29:10.443402+0000","last_clean":"2024-09-11T07:29:10.443402+0000","last_became_active":"2024-09-11T07:26:37.862246+0000","last_became_peered":"2024-09-11T07:26:37.862246+0000","last_unstale":"2024-09-11T07:29:10.443402+0000","last_undegraded":"2024-09-11T07:29:10.443402+0000","last_fullsized":"2024-09-11T07:29:10.443402+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:34.041748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:34.041748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:39.192356+0000","scrub_duration":0.0015228100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"103'2415","reported_seq":2434,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-11T07:29:18.945530+0000","last_change":"2024-09-11T07:28:50.000140+0000","last_active":"2024-09-11T07:29:18.945530+0000","last_peered":"2024-09-11T07:29:18.945530+0000","last_clean":"2024-09-11T07:29:18.945530+0000","last_became_active":"2024-09-11T07:28:49.999860+0000","last_became_peered":"2024-09-11T07:28:49.999860+0000","last_unstale":"2024-09-11T07:29:18.945530+0000","last_undegraded":"2024-09-11T07:29:18.945530+0000","last_fullsized":"2024-09-11T07:29:18.945530+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":2415,"log_dups_size":0,"ondisk_log_size":2415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:30:17.881059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":158269440,"num_objects":2415,"num_object_clones":0,"num_object_copies":7245,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2415,"num_write_kb":154560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"87'26","reported_seq":103,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.447133+0000","last_change":"2024-09-11T07:28:30.788748+0000","last_active":"2024-09-11T07:29:10.447133+0000","last_peered":"2024-09-11T07:29:10.447133+0000","last_clean":"2024-09-11T07:29:10.447133+0000","last_became_active":"2024-09-11T07:26:33.546188+0000","last_became_peered":"2024-09-11T07:26:33.546188+0000","last_unstale":"2024-09-11T07:29:10.447133+0000","last_undegraded":"2024-09-11T07:29:10.447133+0000","last_fullsized":"2024-09-11T07:29:10.447133+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'26","last_scrub_stamp":"2024-09-11T07:28:30.788647+0000","last_deep_scrub":"87'26","last_deep_scrub_stamp":"2024-09-11T07:28:30.788647+0000","last_clean_scrub_stamp":"2024-09-11T07:28:30.788647+0000","objects_scrubbed":8,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:35.415117+0000","scrub_duration":0.0060027819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":65,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.448674+0000","last_change":"2024-09-11T07:28:32.839800+0000","last_active":"2024-09-11T07:29:10.448674+0000","last_peered":"2024-09-11T07:29:10.448674+0000","last_clean":"2024-09-11T07:29:10.448674+0000","last_became_active":"2024-09-11T07:26:31.421500+0000","last_became_peered":"2024-09-11T07:26:31.421500+0000","last_unstale":"2024-09-11T07:29:10.448674+0000","last_undegraded":"2024-09-11T07:29:10.448674+0000","last_fullsized":"2024-09-11T07:29:10.448674+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:32.839737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:32.839737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:35.859113+0000","scrub_duration":0.0014376460000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":61,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.441476+0000","last_change":"2024-09-11T07:28:27.968629+0000","last_active":"2024-09-11T07:29:10.441476+0000","last_peered":"2024-09-11T07:29:10.441476+0000","last_clean":"2024-09-11T07:29:10.441476+0000","last_became_active":"2024-09-11T07:26:35.561284+0000","last_became_peered":"2024-09-11T07:26:35.561284+0000","last_unstale":"2024-09-11T07:29:10.441476+0000","last_undegraded":"2024-09-11T07:29:10.441476+0000","last_fullsized":"2024-09-11T07:29:10.441476+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:27.968579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:27.968579+0000","last_clean_scrub_stamp":"2024-09-11T07:28:27.968579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:53.162017+0000","scrub_duration":0.0026385499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.441680+0000","last_change":"2024-09-11T07:28:29.978861+0000","last_active":"2024-09-11T07:29:10.441680+0000","last_peered":"2024-09-11T07:29:10.441680+0000","last_clean":"2024-09-11T07:29:10.441680+0000","last_became_active":"2024-09-11T07:26:37.564616+0000","last_became_peered":"2024-09-11T07:26:37.564616+0000","last_unstale":"2024-09-11T07:29:10.441680+0000","last_undegraded":"2024-09-11T07:29:10.441680+0000","last_fullsized":"2024-09-11T07:29:10.441680+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:29.978798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:29.978798+0000","last_clean_scrub_stamp":"2024-09-11T07:28:29.978798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:48.118680+0000","scrub_duration":0.001285797,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"103'2379","reported_seq":2397,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-11T07:29:18.940054+0000","last_change":"2024-09-11T07:28:50.002675+0000","last_active":"2024-09-11T07:29:18.940054+0000","last_peered":"2024-09-11T07:29:18.940054+0000","last_clean":"2024-09-11T07:29:18.940054+0000","last_became_active":"2024-09-11T07:28:50.002344+0000","last_became_peered":"2024-09-11T07:28:50.002344+0000","last_unstale":"2024-09-11T07:29:18.940054+0000","last_undegraded":"2024-09-11T07:29:18.940054+0000","last_fullsized":"2024-09-11T07:29:18.940054+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":2379,"log_dups_size":0,"ondisk_log_size":2379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:30:09.362930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":155910144,"num_objects":2379,"num_object_clones":0,"num_object_copies":7137,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2379,"num_write_kb":152256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":99,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.448336+0000","last_change":"2024-09-11T07:28:12.466246+0000","last_active":"2024-09-11T07:29:10.448336+0000","last_peered":"2024-09-11T07:29:10.448336+0000","last_clean":"2024-09-11T07:29:10.448336+0000","last_became_active":"2024-09-11T07:26:33.540639+0000","last_became_peered":"2024-09-11T07:26:33.540639+0000","last_unstale":"2024-09-11T07:29:10.448336+0000","last_undegraded":"2024-09-11T07:29:10.448336+0000","last_fullsized":"2024-09-11T07:29:10.448336+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-11T07:28:12.466179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:12.466179+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:12.523827+0000","scrub_duration":0.0027522670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":65,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.444779+0000","last_change":"2024-09-11T07:28:12.896603+0000","last_active":"2024-09-11T07:29:10.444779+0000","last_peered":"2024-09-11T07:29:10.444779+0000","last_clean":"2024-09-11T07:29:10.444779+0000","last_became_active":"2024-09-11T07:26:31.419185+0000","last_became_peered":"2024-09-11T07:26:31.419185+0000","last_unstale":"2024-09-11T07:29:10.444779+0000","last_undegraded":"2024-09-11T07:29:10.444779+0000","last_fullsized":"2024-09-11T07:29:10.444779+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:12.896538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:12.896538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.577764+0000","scrub_duration":0.0017083070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":61,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.438510+0000","last_change":"2024-09-11T07:27:41.000469+0000","last_active":"2024-09-11T07:29:10.438510+0000","last_peered":"2024-09-11T07:29:10.438510+0000","last_clean":"2024-09-11T07:29:10.438510+0000","last_became_active":"2024-09-11T07:26:35.565359+0000","last_became_peered":"2024-09-11T07:26:35.565359+0000","last_unstale":"2024-09-11T07:29:10.438510+0000","last_undegraded":"2024-09-11T07:29:10.438510+0000","last_fullsized":"2024-09-11T07:29:10.438510+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:41.000401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:41.000401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001655164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446076+0000","last_change":"2024-09-11T07:28:39.723663+0000","last_active":"2024-09-11T07:29:10.446076+0000","last_peered":"2024-09-11T07:29:10.446076+0000","last_clean":"2024-09-11T07:29:10.446076+0000","last_became_active":"2024-09-11T07:26:37.861279+0000","last_became_peered":"2024-09-11T07:26:37.861279+0000","last_unstale":"2024-09-11T07:29:10.446076+0000","last_undegraded":"2024-09-11T07:29:10.446076+0000","last_fullsized":"2024-09-11T07:29:10.446076+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:39.723598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:39.723598+0000","last_clean_scrub_stamp":"2024-09-11T07:28:39.723598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:48.587552+0000","scrub_duration":0.001623202,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"103'2418","reported_seq":2437,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-11T07:29:18.909635+0000","last_change":"2024-09-11T07:28:50.000309+0000","last_active":"2024-09-11T07:29:18.909635+0000","last_peered":"2024-09-11T07:29:18.909635+0000","last_clean":"2024-09-11T07:29:18.909635+0000","last_became_active":"2024-09-11T07:28:49.999983+0000","last_became_peered":"2024-09-11T07:28:49.999983+0000","last_unstale":"2024-09-11T07:29:18.909635+0000","last_undegraded":"2024-09-11T07:29:18.909635+0000","last_fullsized":"2024-09-11T07:29:18.909635+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":2418,"log_dups_size":0,"ondisk_log_size":2418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:30:14.061143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":158466048,"num_objects":2418,"num_object_clones":0,"num_object_copies":7254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2418,"num_write_kb":154752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":78,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446311+0000","last_change":"2024-09-11T07:27:51.642529+0000","last_active":"2024-09-11T07:29:10.446311+0000","last_peered":"2024-09-11T07:29:10.446311+0000","last_clean":"2024-09-11T07:29:10.446311+0000","last_became_active":"2024-09-11T07:26:33.543063+0000","last_became_peered":"2024-09-11T07:26:33.543063+0000","last_unstale":"2024-09-11T07:29:10.446311+0000","last_undegraded":"2024-09-11T07:29:10.446311+0000","last_fullsized":"2024-09-11T07:29:10.446311+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:27:51.642464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:51.642464+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:13.146284+0000","scrub_duration":0.0020211439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":72,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.449623+0000","last_change":"2024-09-11T07:27:41.618362+0000","last_active":"2024-09-11T07:29:10.449623+0000","last_peered":"2024-09-11T07:29:10.449623+0000","last_clean":"2024-09-11T07:29:10.449623+0000","last_became_active":"2024-09-11T07:26:31.422110+0000","last_became_peered":"2024-09-11T07:26:31.422110+0000","last_unstale":"2024-09-11T07:29:10.449623+0000","last_undegraded":"2024-09-11T07:29:10.449623+0000","last_fullsized":"2024-09-11T07:29:10.449623+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:27:41.618296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:41.618296+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020650450000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":102,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-11T07:29:11.498781+0000","last_change":"2024-09-11T07:28:03.183056+0000","last_active":"2024-09-11T07:29:11.498781+0000","last_peered":"2024-09-11T07:29:11.498781+0000","last_clean":"2024-09-11T07:29:11.498781+0000","last_became_active":"2024-09-11T07:26:35.560422+0000","last_became_peered":"2024-09-11T07:26:35.560422+0000","last_unstale":"2024-09-11T07:29:11.498781+0000","last_undegraded":"2024-09-11T07:29:11.498781+0000","last_fullsized":"2024-09-11T07:29:11.498781+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:28:03.182993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:03.182993+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:26.896461+0000","scrub_duration":0.0018949889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.442665+0000","last_change":"2024-09-11T07:27:54.148594+0000","last_active":"2024-09-11T07:29:10.442665+0000","last_peered":"2024-09-11T07:29:10.442665+0000","last_clean":"2024-09-11T07:29:10.442665+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:29:10.442665+0000","last_undegraded":"2024-09-11T07:29:10.442665+0000","last_fullsized":"2024-09-11T07:29:10.442665+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.148534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013536620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"103'2459","reported_seq":2478,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-11T07:29:18.934039+0000","last_change":"2024-09-11T07:28:50.003505+0000","last_active":"2024-09-11T07:29:18.934039+0000","last_peered":"2024-09-11T07:29:18.934039+0000","last_clean":"2024-09-11T07:29:18.934039+0000","last_became_active":"2024-09-11T07:28:50.003113+0000","last_became_peered":"2024-09-11T07:28:50.003113+0000","last_unstale":"2024-09-11T07:29:18.934039+0000","last_undegraded":"2024-09-11T07:29:18.934039+0000","last_fullsized":"2024-09-11T07:29:18.934039+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":2459,"log_dups_size":0,"ondisk_log_size":2459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:30:10.365940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":161153024,"num_objects":2459,"num_object_clones":0,"num_object_copies":7377,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2459,"num_write_kb":157376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"64'28","reported_seq":144,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-11T07:29:11.498439+0000","last_change":"2024-09-11T07:28:09.770058+0000","last_active":"2024-09-11T07:29:11.498439+0000","last_peered":"2024-09-11T07:29:11.498439+0000","last_clean":"2024-09-11T07:29:11.498439+0000","last_became_active":"2024-09-11T07:26:33.549234+0000","last_became_peered":"2024-09-11T07:26:33.549234+0000","last_unstale":"2024-09-11T07:29:11.498439+0000","last_undegraded":"2024-09-11T07:29:11.498439+0000","last_fullsized":"2024-09-11T07:29:11.498439+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'28","last_scrub_stamp":"2024-09-11T07:28:09.769992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:09.769992+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:37.423717+0000","scrub_duration":0.002388463,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":65,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446515+0000","last_change":"2024-09-11T07:28:12.775128+0000","last_active":"2024-09-11T07:29:10.446515+0000","last_peered":"2024-09-11T07:29:10.446515+0000","last_clean":"2024-09-11T07:29:10.446515+0000","last_became_active":"2024-09-11T07:26:31.424618+0000","last_became_peered":"2024-09-11T07:26:31.424618+0000","last_unstale":"2024-09-11T07:29:10.446515+0000","last_undegraded":"2024-09-11T07:29:10.446515+0000","last_fullsized":"2024-09-11T07:29:10.446515+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:12.775067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:12.775067+0000","last_clean_scrub_stamp":"2024-09-11T07:28:12.775067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:21.184139+0000","scrub_duration":0.0015206589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":61,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.445779+0000","last_change":"2024-09-11T07:28:15.439112+0000","last_active":"2024-09-11T07:29:10.445779+0000","last_peered":"2024-09-11T07:29:10.445779+0000","last_clean":"2024-09-11T07:29:10.445779+0000","last_became_active":"2024-09-11T07:26:35.560363+0000","last_became_peered":"2024-09-11T07:26:35.560363+0000","last_unstale":"2024-09-11T07:29:10.445779+0000","last_undegraded":"2024-09-11T07:29:10.445779+0000","last_fullsized":"2024-09-11T07:29:10.445779+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:15.439020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:15.439020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:29.665525+0000","scrub_duration":0.001542844,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.448990+0000","last_change":"2024-09-11T07:28:00.686634+0000","last_active":"2024-09-11T07:29:10.448990+0000","last_peered":"2024-09-11T07:29:10.448990+0000","last_clean":"2024-09-11T07:29:10.448990+0000","last_became_active":"2024-09-11T07:26:37.562452+0000","last_became_peered":"2024-09-11T07:26:37.562452+0000","last_unstale":"2024-09-11T07:29:10.448990+0000","last_undegraded":"2024-09-11T07:29:10.448990+0000","last_fullsized":"2024-09-11T07:29:10.448990+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:00.686571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.686571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016714309999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"103'2432","reported_seq":2451,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-11T07:29:18.920717+0000","last_change":"2024-09-11T07:28:50.005359+0000","last_active":"2024-09-11T07:29:18.920717+0000","last_peered":"2024-09-11T07:29:18.920717+0000","last_clean":"2024-09-11T07:29:18.920717+0000","last_became_active":"2024-09-11T07:28:50.004676+0000","last_became_peered":"2024-09-11T07:28:50.004676+0000","last_unstale":"2024-09-11T07:29:18.920717+0000","last_undegraded":"2024-09-11T07:29:18.920717+0000","last_fullsized":"2024-09-11T07:29:18.920717+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":2432,"log_dups_size":0,"ondisk_log_size":2432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:52.244321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":159383552,"num_objects":2432,"num_object_clones":0,"num_object_copies":7296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2432,"num_write_kb":155648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"64'13","reported_seq":90,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446219+0000","last_change":"2024-09-11T07:28:36.723421+0000","last_active":"2024-09-11T07:29:10.446219+0000","last_peered":"2024-09-11T07:29:10.446219+0000","last_clean":"2024-09-11T07:29:10.446219+0000","last_became_active":"2024-09-11T07:26:33.546126+0000","last_became_peered":"2024-09-11T07:26:33.546126+0000","last_unstale":"2024-09-11T07:29:10.446219+0000","last_undegraded":"2024-09-11T07:29:10.446219+0000","last_fullsized":"2024-09-11T07:29:10.446219+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'13","last_scrub_stamp":"2024-09-11T07:28:36.723359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:36.723359+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:51.245537+0000","scrub_duration":0.0020419800000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":234,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-11T07:29:17.845409+0000","last_change":"2024-09-11T07:27:59.678930+0000","last_active":"2024-09-11T07:29:17.845409+0000","last_peered":"2024-09-11T07:29:17.845409+0000","last_clean":"2024-09-11T07:29:17.845409+0000","last_became_active":"2024-09-11T07:26:31.421276+0000","last_became_peered":"2024-09-11T07:26:31.421276+0000","last_unstale":"2024-09-11T07:29:17.845409+0000","last_undegraded":"2024-09-11T07:29:17.845409+0000","last_fullsized":"2024-09-11T07:29:17.845409+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'5","last_scrub_stamp":"2024-09-11T07:27:59.678865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:59.678865+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.114929+0000","scrub_duration":0.001792486,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":161,"num_read_kb":156,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":65,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.440250+0000","last_change":"2024-09-11T07:28:07.327716+0000","last_active":"2024-09-11T07:29:10.440250+0000","last_peered":"2024-09-11T07:29:10.440250+0000","last_clean":"2024-09-11T07:29:10.440250+0000","last_became_active":"2024-09-11T07:26:31.413354+0000","last_became_peered":"2024-09-11T07:26:31.413354+0000","last_unstale":"2024-09-11T07:29:10.440250+0000","last_undegraded":"2024-09-11T07:29:10.440250+0000","last_fullsized":"2024-09-11T07:29:10.440250+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:07.327648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:07.327648+0000","last_clean_scrub_stamp":"2024-09-11T07:28:07.327648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:35.306829+0000","scrub_duration":0.0011895899999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":61,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.443262+0000","last_change":"2024-09-11T07:27:42.141156+0000","last_active":"2024-09-11T07:29:10.443262+0000","last_peered":"2024-09-11T07:29:10.443262+0000","last_clean":"2024-09-11T07:29:10.443262+0000","last_became_active":"2024-09-11T07:26:35.565849+0000","last_became_peered":"2024-09-11T07:26:35.565849+0000","last_unstale":"2024-09-11T07:29:10.443262+0000","last_undegraded":"2024-09-11T07:29:10.443262+0000","last_fullsized":"2024-09-11T07:29:10.443262+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.141100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.141100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001495672,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.444506+0000","last_change":"2024-09-11T07:27:55.018180+0000","last_active":"2024-09-11T07:29:10.444506+0000","last_peered":"2024-09-11T07:29:10.444506+0000","last_clean":"2024-09-11T07:29:10.444506+0000","last_became_active":"2024-09-11T07:26:37.861292+0000","last_became_peered":"2024-09-11T07:26:37.861292+0000","last_unstale":"2024-09-11T07:29:10.444506+0000","last_undegraded":"2024-09-11T07:29:10.444506+0000","last_fullsized":"2024-09-11T07:29:10.444506+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:55.018125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:55.018125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013534339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"103'2474","reported_seq":2493,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-11T07:29:18.915352+0000","last_change":"2024-09-11T07:28:50.001302+0000","last_active":"2024-09-11T07:29:18.915352+0000","last_peered":"2024-09-11T07:29:18.915352+0000","last_clean":"2024-09-11T07:29:18.915352+0000","last_became_active":"2024-09-11T07:28:50.001027+0000","last_became_peered":"2024-09-11T07:28:50.001027+0000","last_unstale":"2024-09-11T07:29:18.915352+0000","last_undegraded":"2024-09-11T07:29:18.915352+0000","last_fullsized":"2024-09-11T07:29:18.915352+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":2474,"log_dups_size":0,"ondisk_log_size":2474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:30:16.655233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":162136064,"num_objects":2474,"num_object_clones":0,"num_object_copies":7422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2474,"num_write_kb":158336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"94'18","reported_seq":121,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446534+0000","last_change":"2024-09-11T07:28:51.437460+0000","last_active":"2024-09-11T07:29:10.446534+0000","last_peered":"2024-09-11T07:29:10.446534+0000","last_clean":"2024-09-11T07:29:10.446534+0000","last_became_active":"2024-09-11T07:28:09.018134+0000","last_became_peered":"2024-09-11T07:28:09.018134+0000","last_unstale":"2024-09-11T07:29:10.446534+0000","last_undegraded":"2024-09-11T07:29:10.446534+0000","last_fullsized":"2024-09-11T07:29:10.446534+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'18","last_scrub_stamp":"2024-09-11T07:28:51.437397+0000","last_deep_scrub":"94'18","last_deep_scrub_stamp":"2024-09-11T07:28:51.437397+0000","last_clean_scrub_stamp":"2024-09-11T07:28:51.437397+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:59.214386+0000","scrub_duration":0.0052534629999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":71,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.444007+0000","last_change":"2024-09-11T07:28:08.286847+0000","last_active":"2024-09-11T07:29:10.444007+0000","last_peered":"2024-09-11T07:29:10.444007+0000","last_clean":"2024-09-11T07:29:10.444007+0000","last_became_active":"2024-09-11T07:26:31.420320+0000","last_became_peered":"2024-09-11T07:26:31.420320+0000","last_unstale":"2024-09-11T07:29:10.444007+0000","last_undegraded":"2024-09-11T07:29:10.444007+0000","last_fullsized":"2024-09-11T07:29:10.444007+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-11T07:28:08.286788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:08.286788+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.046248+0000","scrub_duration":0.001825976,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":65,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.442865+0000","last_change":"2024-09-11T07:28:16.967543+0000","last_active":"2024-09-11T07:29:10.442865+0000","last_peered":"2024-09-11T07:29:10.442865+0000","last_clean":"2024-09-11T07:29:10.442865+0000","last_became_active":"2024-09-11T07:26:31.417455+0000","last_became_peered":"2024-09-11T07:26:31.417455+0000","last_unstale":"2024-09-11T07:29:10.442865+0000","last_undegraded":"2024-09-11T07:29:10.442865+0000","last_fullsized":"2024-09-11T07:29:10.442865+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:16.967492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:16.967492+0000","last_clean_scrub_stamp":"2024-09-11T07:28:16.967492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:29.710608+0000","scrub_duration":0.0012319799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":61,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446979+0000","last_change":"2024-09-11T07:28:21.868074+0000","last_active":"2024-09-11T07:29:10.446979+0000","last_peered":"2024-09-11T07:29:10.446979+0000","last_clean":"2024-09-11T07:29:10.446979+0000","last_became_active":"2024-09-11T07:26:35.564178+0000","last_became_peered":"2024-09-11T07:26:35.564178+0000","last_unstale":"2024-09-11T07:29:10.446979+0000","last_undegraded":"2024-09-11T07:29:10.446979+0000","last_fullsized":"2024-09-11T07:29:10.446979+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:21.868020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:21.868020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:40.076469+0000","scrub_duration":0.0013323009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.448163+0000","last_change":"2024-09-11T07:27:54.621934+0000","last_active":"2024-09-11T07:29:10.448163+0000","last_peered":"2024-09-11T07:29:10.448163+0000","last_clean":"2024-09-11T07:29:10.448163+0000","last_became_active":"2024-09-11T07:26:37.561378+0000","last_became_peered":"2024-09-11T07:26:37.561378+0000","last_unstale":"2024-09-11T07:29:10.448163+0000","last_undegraded":"2024-09-11T07:29:10.448163+0000","last_fullsized":"2024-09-11T07:29:10.448163+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.621874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.621874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013604750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"103'2458","reported_seq":2477,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-11T07:29:18.936560+0000","last_change":"2024-09-11T07:28:50.001374+0000","last_active":"2024-09-11T07:29:18.936560+0000","last_peered":"2024-09-11T07:29:18.936560+0000","last_clean":"2024-09-11T07:29:18.936560+0000","last_became_active":"2024-09-11T07:28:50.000671+0000","last_became_peered":"2024-09-11T07:28:50.000671+0000","last_unstale":"2024-09-11T07:29:18.936560+0000","last_undegraded":"2024-09-11T07:29:18.936560+0000","last_fullsized":"2024-09-11T07:29:18.936560+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":2458,"log_dups_size":0,"ondisk_log_size":2458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:30:06.554606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":161087488,"num_objects":2458,"num_object_clones":0,"num_object_copies":7374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2458,"num_write_kb":157312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":85,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.438802+0000","last_change":"2024-09-11T07:27:49.061728+0000","last_active":"2024-09-11T07:29:10.438802+0000","last_peered":"2024-09-11T07:29:10.438802+0000","last_clean":"2024-09-11T07:29:10.438802+0000","last_became_active":"2024-09-11T07:26:33.541359+0000","last_became_peered":"2024-09-11T07:26:33.541359+0000","last_unstale":"2024-09-11T07:29:10.438802+0000","last_undegraded":"2024-09-11T07:29:10.438802+0000","last_fullsized":"2024-09-11T07:29:10.438802+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-09-11T07:27:49.061670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.061670+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001809801,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'3","reported_seq":73,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.448057+0000","last_change":"2024-09-11T07:28:01.711301+0000","last_active":"2024-09-11T07:29:10.448057+0000","last_peered":"2024-09-11T07:29:10.448057+0000","last_clean":"2024-09-11T07:29:10.448057+0000","last_became_active":"2024-09-11T07:26:31.419501+0000","last_became_peered":"2024-09-11T07:26:31.419501+0000","last_unstale":"2024-09-11T07:29:10.448057+0000","last_undegraded":"2024-09-11T07:29:10.448057+0000","last_fullsized":"2024-09-11T07:29:10.448057+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.711241+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034627350000000002,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":65,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446837+0000","last_change":"2024-09-11T07:27:47.903204+0000","last_active":"2024-09-11T07:29:10.446837+0000","last_peered":"2024-09-11T07:29:10.446837+0000","last_clean":"2024-09-11T07:29:10.446837+0000","last_became_active":"2024-09-11T07:26:31.423304+0000","last_became_peered":"2024-09-11T07:26:31.423304+0000","last_unstale":"2024-09-11T07:29:10.446837+0000","last_undegraded":"2024-09-11T07:29:10.446837+0000","last_fullsized":"2024-09-11T07:29:10.446837+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:47.903135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.903135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001284908,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"87'81","reported_seq":112,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.443011+0000","last_change":"2024-09-11T07:28:35.050762+0000","last_active":"2024-09-11T07:29:10.443011+0000","last_peered":"2024-09-11T07:29:10.443011+0000","last_clean":"2024-09-11T07:29:10.443011+0000","last_became_active":"2024-09-11T07:26:17.397381+0000","last_became_peered":"2024-09-11T07:26:17.397381+0000","last_unstale":"2024-09-11T07:29:10.443011+0000","last_undegraded":"2024-09-11T07:29:10.443011+0000","last_fullsized":"2024-09-11T07:29:10.443011+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:28:35.050689+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:28:35.050689+0000","last_clean_scrub_stamp":"2024-09-11T07:28:35.050689+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:39.908801+0000","scrub_duration":0.0039160200000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":71,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.442837+0000","last_change":"2024-09-11T07:28:50.190566+0000","last_active":"2024-09-11T07:29:10.442837+0000","last_peered":"2024-09-11T07:29:10.442837+0000","last_clean":"2024-09-11T07:29:10.442837+0000","last_became_active":"2024-09-11T07:26:35.564480+0000","last_became_peered":"2024-09-11T07:26:35.564480+0000","last_unstale":"2024-09-11T07:29:10.442837+0000","last_undegraded":"2024-09-11T07:29:10.442837+0000","last_fullsized":"2024-09-11T07:29:10.442837+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:50.190484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:50.190484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:30:13.399771+0000","scrub_duration":0.021062551999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.447053+0000","last_change":"2024-09-11T07:28:13.796731+0000","last_active":"2024-09-11T07:29:10.447053+0000","last_peered":"2024-09-11T07:29:10.447053+0000","last_clean":"2024-09-11T07:29:10.447053+0000","last_became_active":"2024-09-11T07:26:37.559920+0000","last_became_peered":"2024-09-11T07:26:37.559920+0000","last_unstale":"2024-09-11T07:29:10.447053+0000","last_undegraded":"2024-09-11T07:29:10.447053+0000","last_fullsized":"2024-09-11T07:29:10.447053+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:13.796657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:13.796657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:31.783767+0000","scrub_duration":0.001458984,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.9","version":"103'2443","reported_seq":2462,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-11T07:29:18.918490+0000","last_change":"2024-09-11T07:28:50.003338+0000","last_active":"2024-09-11T07:29:18.918490+0000","last_peered":"2024-09-11T07:29:18.918490+0000","last_clean":"2024-09-11T07:29:18.918490+0000","last_became_active":"2024-09-11T07:28:50.003088+0000","last_became_peered":"2024-09-11T07:28:50.003088+0000","last_unstale":"2024-09-11T07:29:18.918490+0000","last_undegraded":"2024-09-11T07:29:18.918490+0000","last_fullsized":"2024-09-11T07:29:18.918490+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":2443,"log_dups_size":0,"ondisk_log_size":2443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:53.711463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":160104448,"num_objects":2443,"num_object_clones":0,"num_object_copies":7329,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2443,"num_write_kb":156352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":94,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.438793+0000","last_change":"2024-09-11T07:27:59.128292+0000","last_active":"2024-09-11T07:29:10.438793+0000","last_peered":"2024-09-11T07:29:10.438793+0000","last_clean":"2024-09-11T07:29:10.438793+0000","last_became_active":"2024-09-11T07:26:33.543687+0000","last_became_peered":"2024-09-11T07:26:33.543687+0000","last_unstale":"2024-09-11T07:29:10.438793+0000","last_undegraded":"2024-09-11T07:29:10.438793+0000","last_fullsized":"2024-09-11T07:29:10.438793+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-11T07:27:59.128237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:59.128237+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:16.835564+0000","scrub_duration":0.00199329,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":65,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.441217+0000","last_change":"2024-09-11T07:27:54.182157+0000","last_active":"2024-09-11T07:29:10.441217+0000","last_peered":"2024-09-11T07:29:10.441217+0000","last_clean":"2024-09-11T07:29:10.441217+0000","last_became_active":"2024-09-11T07:26:31.419560+0000","last_became_peered":"2024-09-11T07:26:31.419560+0000","last_unstale":"2024-09-11T07:29:10.441217+0000","last_undegraded":"2024-09-11T07:29:10.441217+0000","last_fullsized":"2024-09-11T07:29:10.441217+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.182106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.182106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001213254,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":61,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.440732+0000","last_change":"2024-09-11T07:28:13.357659+0000","last_active":"2024-09-11T07:29:10.440732+0000","last_peered":"2024-09-11T07:29:10.440732+0000","last_clean":"2024-09-11T07:29:10.440732+0000","last_became_active":"2024-09-11T07:26:35.564044+0000","last_became_peered":"2024-09-11T07:26:35.564044+0000","last_unstale":"2024-09-11T07:29:10.440732+0000","last_undegraded":"2024-09-11T07:29:10.440732+0000","last_fullsized":"2024-09-11T07:29:10.440732+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:13.357544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:13.357544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:42.978875+0000","scrub_duration":0.0017127900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.442935+0000","last_change":"2024-09-11T07:28:13.918148+0000","last_active":"2024-09-11T07:29:10.442935+0000","last_peered":"2024-09-11T07:29:10.442935+0000","last_clean":"2024-09-11T07:29:10.442935+0000","last_became_active":"2024-09-11T07:26:37.564801+0000","last_became_peered":"2024-09-11T07:26:37.564801+0000","last_unstale":"2024-09-11T07:29:10.442935+0000","last_undegraded":"2024-09-11T07:29:10.442935+0000","last_fullsized":"2024-09-11T07:29:10.442935+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:13.918082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:13.918082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:31.000135+0000","scrub_duration":0.0014064769999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"103'2423","reported_seq":2442,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-11T07:29:18.936076+0000","last_change":"2024-09-11T07:28:49.999416+0000","last_active":"2024-09-11T07:29:18.936076+0000","last_peered":"2024-09-11T07:29:18.936076+0000","last_clean":"2024-09-11T07:29:18.936076+0000","last_became_active":"2024-09-11T07:28:49.999151+0000","last_became_peered":"2024-09-11T07:28:49.999151+0000","last_unstale":"2024-09-11T07:29:18.936076+0000","last_undegraded":"2024-09-11T07:29:18.936076+0000","last_fullsized":"2024-09-11T07:29:18.936076+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":2423,"log_dups_size":0,"ondisk_log_size":2423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:51.009131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":158793728,"num_objects":2423,"num_object_clones":0,"num_object_copies":7269,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2423,"num_write_kb":155072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":79,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446978+0000","last_change":"2024-09-11T07:28:02.230985+0000","last_active":"2024-09-11T07:29:10.446978+0000","last_peered":"2024-09-11T07:29:10.446978+0000","last_clean":"2024-09-11T07:29:10.446978+0000","last_became_active":"2024-09-11T07:26:33.543148+0000","last_became_peered":"2024-09-11T07:26:33.543148+0000","last_unstale":"2024-09-11T07:29:10.446978+0000","last_undegraded":"2024-09-11T07:29:10.446978+0000","last_fullsized":"2024-09-11T07:29:10.446978+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:28:02.230928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.230928+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:27.244330+0000","scrub_duration":0.001888926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":65,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446938+0000","last_change":"2024-09-11T07:27:53.659961+0000","last_active":"2024-09-11T07:29:10.446938+0000","last_peered":"2024-09-11T07:29:10.446938+0000","last_clean":"2024-09-11T07:29:10.446938+0000","last_became_active":"2024-09-11T07:26:31.423056+0000","last_became_peered":"2024-09-11T07:26:31.423056+0000","last_unstale":"2024-09-11T07:29:10.446938+0000","last_undegraded":"2024-09-11T07:29:10.446938+0000","last_fullsized":"2024-09-11T07:29:10.446938+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:53.659902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:53.659902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011613719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":61,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.440960+0000","last_change":"2024-09-11T07:28:10.337599+0000","last_active":"2024-09-11T07:29:10.440960+0000","last_peered":"2024-09-11T07:29:10.440960+0000","last_clean":"2024-09-11T07:29:10.440960+0000","last_became_active":"2024-09-11T07:26:35.563182+0000","last_became_peered":"2024-09-11T07:26:35.563182+0000","last_unstale":"2024-09-11T07:29:10.440960+0000","last_undegraded":"2024-09-11T07:29:10.440960+0000","last_fullsized":"2024-09-11T07:29:10.440960+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:10.337527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:10.337527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:24.013574+0000","scrub_duration":0.001985653,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.443908+0000","last_change":"2024-09-11T07:28:18.503995+0000","last_active":"2024-09-11T07:29:10.443908+0000","last_peered":"2024-09-11T07:29:10.443908+0000","last_clean":"2024-09-11T07:29:10.443908+0000","last_became_active":"2024-09-11T07:26:37.565029+0000","last_became_peered":"2024-09-11T07:26:37.565029+0000","last_unstale":"2024-09-11T07:29:10.443908+0000","last_undegraded":"2024-09-11T07:29:10.443908+0000","last_fullsized":"2024-09-11T07:29:10.443908+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:18.503930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:18.503930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:36.076642+0000","scrub_duration":0.001490297,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.b","version":"103'2362","reported_seq":2381,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-11T07:29:18.939693+0000","last_change":"2024-09-11T07:28:50.003935+0000","last_active":"2024-09-11T07:29:18.939693+0000","last_peered":"2024-09-11T07:29:18.939693+0000","last_clean":"2024-09-11T07:29:18.939693+0000","last_became_active":"2024-09-11T07:28:50.003627+0000","last_became_peered":"2024-09-11T07:28:50.003627+0000","last_unstale":"2024-09-11T07:29:18.939693+0000","last_undegraded":"2024-09-11T07:29:18.939693+0000","last_fullsized":"2024-09-11T07:29:18.939693+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":2362,"log_dups_size":0,"ondisk_log_size":2362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:30:04.839897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154796032,"num_objects":2362,"num_object_clones":0,"num_object_copies":7086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2362,"num_write_kb":151168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":88,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446159+0000","last_change":"2024-09-11T07:28:07.811136+0000","last_active":"2024-09-11T07:29:10.446159+0000","last_peered":"2024-09-11T07:29:10.446159+0000","last_clean":"2024-09-11T07:29:10.446159+0000","last_became_active":"2024-09-11T07:26:33.538216+0000","last_became_peered":"2024-09-11T07:26:33.538216+0000","last_unstale":"2024-09-11T07:29:10.446159+0000","last_undegraded":"2024-09-11T07:29:10.446159+0000","last_fullsized":"2024-09-11T07:29:10.446159+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-09-11T07:28:07.811071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:07.811071+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:23.495093+0000","scrub_duration":0.0019627080000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":65,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.444346+0000","last_change":"2024-09-11T07:27:50.132629+0000","last_active":"2024-09-11T07:29:10.444346+0000","last_peered":"2024-09-11T07:29:10.444346+0000","last_clean":"2024-09-11T07:29:10.444346+0000","last_became_active":"2024-09-11T07:26:31.418490+0000","last_became_peered":"2024-09-11T07:26:31.418490+0000","last_unstale":"2024-09-11T07:29:10.444346+0000","last_undegraded":"2024-09-11T07:29:10.444346+0000","last_fullsized":"2024-09-11T07:29:10.444346+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_clean_scrub_stamp":"2024-09-11T07:27:50.132552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.048681+0000","scrub_duration":0.001622807,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":102,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-11T07:29:11.441017+0000","last_change":"2024-09-11T07:28:32.017483+0000","last_active":"2024-09-11T07:29:11.441017+0000","last_peered":"2024-09-11T07:29:11.441017+0000","last_clean":"2024-09-11T07:29:11.441017+0000","last_became_active":"2024-09-11T07:26:35.566888+0000","last_became_peered":"2024-09-11T07:26:35.566888+0000","last_unstale":"2024-09-11T07:29:11.441017+0000","last_undegraded":"2024-09-11T07:29:11.441017+0000","last_fullsized":"2024-09-11T07:29:11.441017+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:28:32.017418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:32.017418+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:55.321090+0000","scrub_duration":0.0016851839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446145+0000","last_change":"2024-09-11T07:28:03.961280+0000","last_active":"2024-09-11T07:29:10.446145+0000","last_peered":"2024-09-11T07:29:10.446145+0000","last_clean":"2024-09-11T07:29:10.446145+0000","last_became_active":"2024-09-11T07:26:37.861134+0000","last_became_peered":"2024-09-11T07:26:37.861134+0000","last_unstale":"2024-09-11T07:29:10.446145+0000","last_undegraded":"2024-09-11T07:29:10.446145+0000","last_fullsized":"2024-09-11T07:29:10.446145+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:03.961220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:03.961220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:29.223995+0000","scrub_duration":0.0011943730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.a","version":"103'2361","reported_seq":2380,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-11T07:29:18.922554+0000","last_change":"2024-09-11T07:28:50.004530+0000","last_active":"2024-09-11T07:29:18.922554+0000","last_peered":"2024-09-11T07:29:18.922554+0000","last_clean":"2024-09-11T07:29:18.922554+0000","last_became_active":"2024-09-11T07:28:50.004253+0000","last_became_peered":"2024-09-11T07:28:50.004253+0000","last_unstale":"2024-09-11T07:29:18.922554+0000","last_undegraded":"2024-09-11T07:29:18.922554+0000","last_fullsized":"2024-09-11T07:29:18.922554+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:48.977159+0000","last_clean_scrub_stamp":"2024-09-11T07:28:48.977159+0000","objects_scrubbed":0,"log_size":2361,"log_dups_size":0,"ondisk_log_size":2361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:30:16.438942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154730496,"num_objects":2361,"num_object_clones":0,"num_object_copies":7083,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2361,"num_write_kb":151104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"83'17","reported_seq":91,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.441556+0000","last_change":"2024-09-11T07:28:01.098068+0000","last_active":"2024-09-11T07:29:10.441556+0000","last_peered":"2024-09-11T07:29:10.441556+0000","last_clean":"2024-09-11T07:29:10.441556+0000","last_became_active":"2024-09-11T07:26:33.540363+0000","last_became_peered":"2024-09-11T07:26:33.540363+0000","last_unstale":"2024-09-11T07:29:10.441556+0000","last_undegraded":"2024-09-11T07:29:10.441556+0000","last_fullsized":"2024-09-11T07:29:10.441556+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'17","last_scrub_stamp":"2024-09-11T07:28:01.097981+0000","last_deep_scrub":"83'17","last_deep_scrub_stamp":"2024-09-11T07:28:01.097981+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.097981+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:26.179592+0000","scrub_duration":0.0042048720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":82,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.442385+0000","last_change":"2024-09-11T07:27:46.077857+0000","last_active":"2024-09-11T07:29:10.442385+0000","last_peered":"2024-09-11T07:29:10.442385+0000","last_clean":"2024-09-11T07:29:10.442385+0000","last_became_active":"2024-09-11T07:26:31.419718+0000","last_became_peered":"2024-09-11T07:26:31.419718+0000","last_unstale":"2024-09-11T07:29:10.442385+0000","last_undegraded":"2024-09-11T07:29:10.442385+0000","last_fullsized":"2024-09-11T07:29:10.442385+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:27:46.077786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:46.077786+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019376720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":61,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.442430+0000","last_change":"2024-09-11T07:27:53.209518+0000","last_active":"2024-09-11T07:29:10.442430+0000","last_peered":"2024-09-11T07:29:10.442430+0000","last_clean":"2024-09-11T07:29:10.442430+0000","last_became_active":"2024-09-11T07:26:35.561967+0000","last_became_peered":"2024-09-11T07:26:35.561967+0000","last_unstale":"2024-09-11T07:29:10.442430+0000","last_undegraded":"2024-09-11T07:29:10.442430+0000","last_fullsized":"2024-09-11T07:29:10.442430+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:53.209452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:53.209452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:23.125281+0000","scrub_duration":0.0014563029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.448475+0000","last_change":"2024-09-11T07:28:02.677186+0000","last_active":"2024-09-11T07:29:10.448475+0000","last_peered":"2024-09-11T07:29:10.448475+0000","last_clean":"2024-09-11T07:29:10.448475+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:29:10.448475+0000","last_undegraded":"2024-09-11T07:29:10.448475+0000","last_fullsized":"2024-09-11T07:29:10.448475+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:02.677095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.677095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001519269,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":79,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446457+0000","last_change":"2024-09-11T07:27:40.351572+0000","last_active":"2024-09-11T07:29:10.446457+0000","last_peered":"2024-09-11T07:29:10.446457+0000","last_clean":"2024-09-11T07:29:10.446457+0000","last_became_active":"2024-09-11T07:26:33.547344+0000","last_became_peered":"2024-09-11T07:26:33.547344+0000","last_unstale":"2024-09-11T07:29:10.446457+0000","last_undegraded":"2024-09-11T07:29:10.446457+0000","last_fullsized":"2024-09-11T07:29:10.446457+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:27:40.351507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:40.351507+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021803980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":65,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.442584+0000","last_change":"2024-09-11T07:27:58.324311+0000","last_active":"2024-09-11T07:29:10.442584+0000","last_peered":"2024-09-11T07:29:10.442584+0000","last_clean":"2024-09-11T07:29:10.442584+0000","last_became_active":"2024-09-11T07:26:31.415701+0000","last_became_peered":"2024-09-11T07:26:31.415701+0000","last_unstale":"2024-09-11T07:29:10.442584+0000","last_undegraded":"2024-09-11T07:29:10.442584+0000","last_fullsized":"2024-09-11T07:29:10.442584+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:58.324255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.324255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.617500+0000","scrub_duration":0.0013365549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":61,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.449202+0000","last_change":"2024-09-11T07:28:12.778700+0000","last_active":"2024-09-11T07:29:10.449202+0000","last_peered":"2024-09-11T07:29:10.449202+0000","last_clean":"2024-09-11T07:29:10.449202+0000","last_became_active":"2024-09-11T07:26:35.559262+0000","last_became_peered":"2024-09-11T07:26:35.559262+0000","last_unstale":"2024-09-11T07:29:10.449202+0000","last_undegraded":"2024-09-11T07:29:10.449202+0000","last_fullsized":"2024-09-11T07:29:10.449202+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:12.778645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:12.778645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.979009+0000","scrub_duration":0.0013084870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":60,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.440746+0000","last_change":"2024-09-11T07:28:06.305563+0000","last_active":"2024-09-11T07:29:10.440746+0000","last_peered":"2024-09-11T07:29:10.440746+0000","last_clean":"2024-09-11T07:29:10.440746+0000","last_became_active":"2024-09-11T07:26:37.564084+0000","last_became_peered":"2024-09-11T07:26:37.564084+0000","last_unstale":"2024-09-11T07:29:10.440746+0000","last_undegraded":"2024-09-11T07:29:10.440746+0000","last_fullsized":"2024-09-11T07:29:10.440746+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:28:06.305502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:06.305502+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:19.659083+0000","scrub_duration":0.0021934649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":65,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.439479+0000","last_change":"2024-09-11T07:27:45.990617+0000","last_active":"2024-09-11T07:29:10.439479+0000","last_peered":"2024-09-11T07:29:10.439479+0000","last_clean":"2024-09-11T07:29:10.439479+0000","last_became_active":"2024-09-11T07:26:31.423229+0000","last_became_peered":"2024-09-11T07:26:31.423229+0000","last_unstale":"2024-09-11T07:29:10.439479+0000","last_undegraded":"2024-09-11T07:29:10.439479+0000","last_fullsized":"2024-09-11T07:29:10.439479+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:45.990548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:45.990548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014585150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"82'9","reported_seq":75,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.449482+0000","last_change":"2024-09-11T07:27:48.761470+0000","last_active":"2024-09-11T07:29:10.449482+0000","last_peered":"2024-09-11T07:29:10.449482+0000","last_clean":"2024-09-11T07:29:10.449482+0000","last_became_active":"2024-09-11T07:26:33.543628+0000","last_became_peered":"2024-09-11T07:26:33.543628+0000","last_unstale":"2024-09-11T07:29:10.449482+0000","last_undegraded":"2024-09-11T07:29:10.449482+0000","last_fullsized":"2024-09-11T07:29:10.449482+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'9","last_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:27:48.761397+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037047439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":61,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446939+0000","last_change":"2024-09-11T07:27:42.917739+0000","last_active":"2024-09-11T07:29:10.446939+0000","last_peered":"2024-09-11T07:29:10.446939+0000","last_clean":"2024-09-11T07:29:10.446939+0000","last_became_active":"2024-09-11T07:26:35.566894+0000","last_became_peered":"2024-09-11T07:26:35.566894+0000","last_unstale":"2024-09-11T07:29:10.446939+0000","last_undegraded":"2024-09-11T07:29:10.446939+0000","last_fullsized":"2024-09-11T07:29:10.446939+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.917680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013496649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.443950+0000","last_change":"2024-09-11T07:27:47.031883+0000","last_active":"2024-09-11T07:29:10.443950+0000","last_peered":"2024-09-11T07:29:10.443950+0000","last_clean":"2024-09-11T07:29:10.443950+0000","last_became_active":"2024-09-11T07:26:37.863100+0000","last_became_peered":"2024-09-11T07:26:37.863100+0000","last_unstale":"2024-09-11T07:29:10.443950+0000","last_undegraded":"2024-09-11T07:29:10.443950+0000","last_fullsized":"2024-09-11T07:29:10.443950+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:47.031807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.031807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001556195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"86'15","reported_seq":83,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.449148+0000","last_change":"2024-09-11T07:28:22.858192+0000","last_active":"2024-09-11T07:29:10.449148+0000","last_peered":"2024-09-11T07:29:10.449148+0000","last_clean":"2024-09-11T07:29:10.449148+0000","last_became_active":"2024-09-11T07:26:33.545078+0000","last_became_peered":"2024-09-11T07:26:33.545078+0000","last_unstale":"2024-09-11T07:29:10.449148+0000","last_undegraded":"2024-09-11T07:29:10.449148+0000","last_fullsized":"2024-09-11T07:29:10.449148+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'15","last_scrub_stamp":"2024-09-11T07:28:22.858137+0000","last_deep_scrub":"86'15","last_deep_scrub_stamp":"2024-09-11T07:28:22.858137+0000","last_clean_scrub_stamp":"2024-09-11T07:28:22.858137+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:22.911147+0000","scrub_duration":0.003508101,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":65,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446393+0000","last_change":"2024-09-11T07:28:14.819081+0000","last_active":"2024-09-11T07:29:10.446393+0000","last_peered":"2024-09-11T07:29:10.446393+0000","last_clean":"2024-09-11T07:29:10.446393+0000","last_became_active":"2024-09-11T07:26:31.423789+0000","last_became_peered":"2024-09-11T07:26:31.423789+0000","last_unstale":"2024-09-11T07:29:10.446393+0000","last_undegraded":"2024-09-11T07:29:10.446393+0000","last_fullsized":"2024-09-11T07:29:10.446393+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:14.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:14.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:17.757008+0000","scrub_duration":0.001929339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":102,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-11T07:29:11.495821+0000","last_change":"2024-09-11T07:28:16.782518+0000","last_active":"2024-09-11T07:29:11.495821+0000","last_peered":"2024-09-11T07:29:11.495821+0000","last_clean":"2024-09-11T07:29:11.495821+0000","last_became_active":"2024-09-11T07:26:35.562245+0000","last_became_peered":"2024-09-11T07:26:35.562245+0000","last_unstale":"2024-09-11T07:29:11.495821+0000","last_undegraded":"2024-09-11T07:29:11.495821+0000","last_fullsized":"2024-09-11T07:29:11.495821+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:28:16.782462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:16.782462+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:30.251276+0000","scrub_duration":0.001462283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.447127+0000","last_change":"2024-09-11T07:27:49.245119+0000","last_active":"2024-09-11T07:29:10.447127+0000","last_peered":"2024-09-11T07:29:10.447127+0000","last_clean":"2024-09-11T07:29:10.447127+0000","last_became_active":"2024-09-11T07:26:37.566749+0000","last_became_peered":"2024-09-11T07:26:37.566749+0000","last_unstale":"2024-09-11T07:29:10.447127+0000","last_undegraded":"2024-09-11T07:29:10.447127+0000","last_fullsized":"2024-09-11T07:29:10.447127+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:49.245047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.245047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001867966,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"83'13","reported_seq":83,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446638+0000","last_change":"2024-09-11T07:28:00.959669+0000","last_active":"2024-09-11T07:29:10.446638+0000","last_peered":"2024-09-11T07:29:10.446638+0000","last_clean":"2024-09-11T07:29:10.446638+0000","last_became_active":"2024-09-11T07:26:33.537425+0000","last_became_peered":"2024-09-11T07:26:33.537425+0000","last_unstale":"2024-09-11T07:29:10.446638+0000","last_undegraded":"2024-09-11T07:29:10.446638+0000","last_fullsized":"2024-09-11T07:29:10.446638+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-11T07:28:00.959583+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-11T07:28:00.959583+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.959583+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032987350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":65,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.447216+0000","last_change":"2024-09-11T07:28:01.237329+0000","last_active":"2024-09-11T07:29:10.447216+0000","last_peered":"2024-09-11T07:29:10.447216+0000","last_clean":"2024-09-11T07:29:10.447216+0000","last_became_active":"2024-09-11T07:26:31.419409+0000","last_became_peered":"2024-09-11T07:26:31.419409+0000","last_unstale":"2024-09-11T07:29:10.447216+0000","last_undegraded":"2024-09-11T07:29:10.447216+0000","last_fullsized":"2024-09-11T07:29:10.447216+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:01.237267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.237267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.266034+0000","scrub_duration":0.0012797399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":102,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-11T07:29:11.493868+0000","last_change":"2024-09-11T07:28:23.883321+0000","last_active":"2024-09-11T07:29:11.493868+0000","last_peered":"2024-09-11T07:29:11.493868+0000","last_clean":"2024-09-11T07:29:11.493868+0000","last_became_active":"2024-09-11T07:26:35.560610+0000","last_became_peered":"2024-09-11T07:26:35.560610+0000","last_unstale":"2024-09-11T07:29:11.493868+0000","last_undegraded":"2024-09-11T07:29:11.493868+0000","last_fullsized":"2024-09-11T07:29:11.493868+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:28:23.883265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:23.883265+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:49.732087+0000","scrub_duration":0.0020633370000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.442055+0000","last_change":"2024-09-11T07:28:22.053350+0000","last_active":"2024-09-11T07:29:10.442055+0000","last_peered":"2024-09-11T07:29:10.442055+0000","last_clean":"2024-09-11T07:29:10.442055+0000","last_became_active":"2024-09-11T07:26:37.565723+0000","last_became_peered":"2024-09-11T07:26:37.565723+0000","last_unstale":"2024-09-11T07:29:10.442055+0000","last_undegraded":"2024-09-11T07:29:10.442055+0000","last_fullsized":"2024-09-11T07:29:10.442055+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:22.053291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:22.053291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:40.893276+0000","scrub_duration":0.001270356,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":79,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446225+0000","last_change":"2024-09-11T07:27:57.601328+0000","last_active":"2024-09-11T07:29:10.446225+0000","last_peered":"2024-09-11T07:29:10.446225+0000","last_clean":"2024-09-11T07:29:10.446225+0000","last_became_active":"2024-09-11T07:26:33.540688+0000","last_became_peered":"2024-09-11T07:26:33.540688+0000","last_unstale":"2024-09-11T07:29:10.446225+0000","last_undegraded":"2024-09-11T07:29:10.446225+0000","last_fullsized":"2024-09-11T07:29:10.446225+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:27:57.601261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.601261+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002247724,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":65,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.438964+0000","last_change":"2024-09-11T07:27:54.102580+0000","last_active":"2024-09-11T07:29:10.438964+0000","last_peered":"2024-09-11T07:29:10.438964+0000","last_clean":"2024-09-11T07:29:10.438964+0000","last_became_active":"2024-09-11T07:26:31.425587+0000","last_became_peered":"2024-09-11T07:26:31.425587+0000","last_unstale":"2024-09-11T07:29:10.438964+0000","last_undegraded":"2024-09-11T07:29:10.438964+0000","last_fullsized":"2024-09-11T07:29:10.438964+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:54.102528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.102528+0000","last_clean_scrub_stamp":"2024-09-11T07:27:54.102528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012736080000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":61,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.448542+0000","last_change":"2024-09-11T07:28:29.789817+0000","last_active":"2024-09-11T07:29:10.448542+0000","last_peered":"2024-09-11T07:29:10.448542+0000","last_clean":"2024-09-11T07:29:10.448542+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:29:10.448542+0000","last_undegraded":"2024-09-11T07:29:10.448542+0000","last_fullsized":"2024-09-11T07:29:10.448542+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:29.789752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:29.789752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:42.508949+0000","scrub_duration":0.001301548,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.441956+0000","last_change":"2024-09-11T07:27:58.192585+0000","last_active":"2024-09-11T07:29:10.441956+0000","last_peered":"2024-09-11T07:29:10.441956+0000","last_clean":"2024-09-11T07:29:10.441956+0000","last_became_active":"2024-09-11T07:26:37.568886+0000","last_became_peered":"2024-09-11T07:26:37.568886+0000","last_unstale":"2024-09-11T07:29:10.441956+0000","last_undegraded":"2024-09-11T07:29:10.441956+0000","last_fullsized":"2024-09-11T07:29:10.441956+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:58.192528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.192528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:18.757569+0000","scrub_duration":0.001884698,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":82,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.441936+0000","last_change":"2024-09-11T07:27:48.081460+0000","last_active":"2024-09-11T07:29:10.441936+0000","last_peered":"2024-09-11T07:29:10.441936+0000","last_clean":"2024-09-11T07:29:10.441936+0000","last_became_active":"2024-09-11T07:26:33.540208+0000","last_became_peered":"2024-09-11T07:26:33.540208+0000","last_unstale":"2024-09-11T07:29:10.441936+0000","last_undegraded":"2024-09-11T07:29:10.441936+0000","last_fullsized":"2024-09-11T07:29:10.441936+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:27:48.081392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:48.081392+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022394590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":65,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446871+0000","last_change":"2024-09-11T07:27:49.677063+0000","last_active":"2024-09-11T07:29:10.446871+0000","last_peered":"2024-09-11T07:29:10.446871+0000","last_clean":"2024-09-11T07:29:10.446871+0000","last_became_active":"2024-09-11T07:26:31.422671+0000","last_became_peered":"2024-09-11T07:26:31.422671+0000","last_unstale":"2024-09-11T07:29:10.446871+0000","last_undegraded":"2024-09-11T07:29:10.446871+0000","last_fullsized":"2024-09-11T07:29:10.446871+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:27:49.677003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013962969999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":61,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446076+0000","last_change":"2024-09-11T07:28:30.847535+0000","last_active":"2024-09-11T07:29:10.446076+0000","last_peered":"2024-09-11T07:29:10.446076+0000","last_clean":"2024-09-11T07:29:10.446076+0000","last_became_active":"2024-09-11T07:26:35.566931+0000","last_became_peered":"2024-09-11T07:26:35.566931+0000","last_unstale":"2024-09-11T07:29:10.446076+0000","last_undegraded":"2024-09-11T07:29:10.446076+0000","last_fullsized":"2024-09-11T07:29:10.446076+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:30.847474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.847474+0000","last_clean_scrub_stamp":"2024-09-11T07:28:30.847474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:48.388338+0000","scrub_duration":0.001576682,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446270+0000","last_change":"2024-09-11T07:28:20.766566+0000","last_active":"2024-09-11T07:29:10.446270+0000","last_peered":"2024-09-11T07:29:10.446270+0000","last_clean":"2024-09-11T07:29:10.446270+0000","last_became_active":"2024-09-11T07:26:37.561573+0000","last_became_peered":"2024-09-11T07:26:37.561573+0000","last_unstale":"2024-09-11T07:29:10.446270+0000","last_undegraded":"2024-09-11T07:29:10.446270+0000","last_fullsized":"2024-09-11T07:29:10.446270+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:20.766508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:20.766508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:42.240723+0000","scrub_duration":0.0012453239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"82'6","reported_seq":71,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.447052+0000","last_change":"2024-09-11T07:27:47.789519+0000","last_active":"2024-09-11T07:29:10.447052+0000","last_peered":"2024-09-11T07:29:10.447052+0000","last_clean":"2024-09-11T07:29:10.447052+0000","last_became_active":"2024-09-11T07:26:33.545830+0000","last_became_peered":"2024-09-11T07:26:33.545830+0000","last_unstale":"2024-09-11T07:29:10.447052+0000","last_undegraded":"2024-09-11T07:29:10.447052+0000","last_fullsized":"2024-09-11T07:29:10.447052+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'6","last_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_deep_scrub":"82'6","last_deep_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_clean_scrub_stamp":"2024-09-11T07:27:47.789465+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002624679,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":65,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.447055+0000","last_change":"2024-09-11T07:27:58.043076+0000","last_active":"2024-09-11T07:29:10.447055+0000","last_peered":"2024-09-11T07:29:10.447055+0000","last_clean":"2024-09-11T07:29:10.447055+0000","last_became_active":"2024-09-11T07:26:31.420518+0000","last_became_peered":"2024-09-11T07:26:31.420518+0000","last_unstale":"2024-09-11T07:29:10.447055+0000","last_undegraded":"2024-09-11T07:29:10.447055+0000","last_fullsized":"2024-09-11T07:29:10.447055+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_clean_scrub_stamp":"2024-09-11T07:27:58.043009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:24.326885+0000","scrub_duration":0.0015014900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":61,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.438400+0000","last_change":"2024-09-11T07:27:56.124601+0000","last_active":"2024-09-11T07:29:10.438400+0000","last_peered":"2024-09-11T07:29:10.438400+0000","last_clean":"2024-09-11T07:29:10.438400+0000","last_became_active":"2024-09-11T07:26:35.560506+0000","last_became_peered":"2024-09-11T07:26:35.560506+0000","last_unstale":"2024-09-11T07:29:10.438400+0000","last_undegraded":"2024-09-11T07:29:10.438400+0000","last_fullsized":"2024-09-11T07:29:10.438400+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:56.124542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:56.124542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011688619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.442954+0000","last_change":"2024-09-11T07:27:42.059066+0000","last_active":"2024-09-11T07:29:10.442954+0000","last_peered":"2024-09-11T07:29:10.442954+0000","last_clean":"2024-09-11T07:29:10.442954+0000","last_became_active":"2024-09-11T07:26:37.564208+0000","last_became_peered":"2024-09-11T07:26:37.564208+0000","last_unstale":"2024-09-11T07:29:10.442954+0000","last_undegraded":"2024-09-11T07:29:10.442954+0000","last_fullsized":"2024-09-11T07:29:10.442954+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:42.059001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:27:42.059001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001418094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":82,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446455+0000","last_change":"2024-09-11T07:28:28.771165+0000","last_active":"2024-09-11T07:29:10.446455+0000","last_peered":"2024-09-11T07:29:10.446455+0000","last_clean":"2024-09-11T07:29:10.446455+0000","last_became_active":"2024-09-11T07:26:33.542625+0000","last_became_peered":"2024-09-11T07:26:33.542625+0000","last_unstale":"2024-09-11T07:29:10.446455+0000","last_undegraded":"2024-09-11T07:29:10.446455+0000","last_fullsized":"2024-09-11T07:29:10.446455+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:28:28.771107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:28.771107+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:44.367423+0000","scrub_duration":0.001700355,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":65,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.439405+0000","last_change":"2024-09-11T07:28:02.157520+0000","last_active":"2024-09-11T07:29:10.439405+0000","last_peered":"2024-09-11T07:29:10.439405+0000","last_clean":"2024-09-11T07:29:10.439405+0000","last_became_active":"2024-09-11T07:26:31.425876+0000","last_became_peered":"2024-09-11T07:26:31.425876+0000","last_unstale":"2024-09-11T07:29:10.439405+0000","last_undegraded":"2024-09-11T07:29:10.439405+0000","last_fullsized":"2024-09-11T07:29:10.439405+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:02.157455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.157455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:12.096002+0000","scrub_duration":0.0016990040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":61,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.442473+0000","last_change":"2024-09-11T07:28:27.019028+0000","last_active":"2024-09-11T07:29:10.442473+0000","last_peered":"2024-09-11T07:29:10.442473+0000","last_clean":"2024-09-11T07:29:10.442473+0000","last_became_active":"2024-09-11T07:26:35.565339+0000","last_became_peered":"2024-09-11T07:26:35.565339+0000","last_unstale":"2024-09-11T07:29:10.442473+0000","last_undegraded":"2024-09-11T07:29:10.442473+0000","last_fullsized":"2024-09-11T07:29:10.442473+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:27.018961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:27.018961+0000","last_clean_scrub_stamp":"2024-09-11T07:28:27.018961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:27.125944+0000","scrub_duration":0.001482106,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.448401+0000","last_change":"2024-09-11T07:28:31.783418+0000","last_active":"2024-09-11T07:29:10.448401+0000","last_peered":"2024-09-11T07:29:10.448401+0000","last_clean":"2024-09-11T07:29:10.448401+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:29:10.448401+0000","last_undegraded":"2024-09-11T07:29:10.448401+0000","last_fullsized":"2024-09-11T07:29:10.448401+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:31.783359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:31.783359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:58.426334+0000","scrub_duration":0.0016047909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":60,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.442757+0000","last_change":"2024-09-11T07:28:28.990232+0000","last_active":"2024-09-11T07:29:10.442757+0000","last_peered":"2024-09-11T07:29:10.442757+0000","last_clean":"2024-09-11T07:29:10.442757+0000","last_became_active":"2024-09-11T07:26:37.563942+0000","last_became_peered":"2024-09-11T07:26:37.563942+0000","last_unstale":"2024-09-11T07:29:10.442757+0000","last_undegraded":"2024-09-11T07:29:10.442757+0000","last_fullsized":"2024-09-11T07:29:10.442757+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:28:28.990175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:28.990175+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:45.813007+0000","scrub_duration":0.001863689,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":74,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446230+0000","last_change":"2024-09-11T07:28:02.969498+0000","last_active":"2024-09-11T07:29:10.446230+0000","last_peered":"2024-09-11T07:29:10.446230+0000","last_clean":"2024-09-11T07:29:10.446230+0000","last_became_active":"2024-09-11T07:26:31.423932+0000","last_became_peered":"2024-09-11T07:26:31.423932+0000","last_unstale":"2024-09-11T07:29:10.446230+0000","last_undegraded":"2024-09-11T07:29:10.446230+0000","last_fullsized":"2024-09-11T07:29:10.446230+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:28:02.969442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:28:02.969442+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:22.652879+0000","scrub_duration":0.0014604450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":78,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446365+0000","last_change":"2024-09-11T07:27:55.150630+0000","last_active":"2024-09-11T07:29:10.446365+0000","last_peered":"2024-09-11T07:29:10.446365+0000","last_clean":"2024-09-11T07:29:10.446365+0000","last_became_active":"2024-09-11T07:26:33.546797+0000","last_became_peered":"2024-09-11T07:26:33.546797+0000","last_unstale":"2024-09-11T07:29:10.446365+0000","last_undegraded":"2024-09-11T07:29:10.446365+0000","last_fullsized":"2024-09-11T07:29:10.446365+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:27:55.150567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:55.150567+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016612689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"64'8","reported_seq":105,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-11T07:29:11.499091+0000","last_change":"2024-09-11T07:28:01.123469+0000","last_active":"2024-09-11T07:29:11.499091+0000","last_peered":"2024-09-11T07:29:11.499091+0000","last_clean":"2024-09-11T07:29:11.499091+0000","last_became_active":"2024-09-11T07:26:35.559304+0000","last_became_peered":"2024-09-11T07:26:35.559304+0000","last_unstale":"2024-09-11T07:29:11.499091+0000","last_undegraded":"2024-09-11T07:29:11.499091+0000","last_fullsized":"2024-09-11T07:29:11.499091+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:28:01.123401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:01.123401+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:18.715539+0000","scrub_duration":0.00167504,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446443+0000","last_change":"2024-09-11T07:28:11.735040+0000","last_active":"2024-09-11T07:29:10.446443+0000","last_peered":"2024-09-11T07:29:10.446443+0000","last_clean":"2024-09-11T07:29:10.446443+0000","last_became_active":"2024-09-11T07:26:37.562501+0000","last_became_peered":"2024-09-11T07:26:37.562501+0000","last_unstale":"2024-09-11T07:29:10.446443+0000","last_undegraded":"2024-09-11T07:29:10.446443+0000","last_fullsized":"2024-09-11T07:29:10.446443+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:11.734968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:11.734968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:13.880946+0000","scrub_duration":0.0021993350000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":65,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446756+0000","last_change":"2024-09-11T07:27:38.732867+0000","last_active":"2024-09-11T07:29:10.446756+0000","last_peered":"2024-09-11T07:29:10.446756+0000","last_clean":"2024-09-11T07:29:10.446756+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:29:10.446756+0000","last_undegraded":"2024-09-11T07:29:10.446756+0000","last_fullsized":"2024-09-11T07:29:10.446756+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:38.732795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:38.732795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025091950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":104,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.439361+0000","last_change":"2024-09-11T07:28:09.024756+0000","last_active":"2024-09-11T07:29:10.439361+0000","last_peered":"2024-09-11T07:29:10.439361+0000","last_clean":"2024-09-11T07:29:10.439361+0000","last_became_active":"2024-09-11T07:28:09.017997+0000","last_became_peered":"2024-09-11T07:28:09.017997+0000","last_unstale":"2024-09-11T07:29:10.439361+0000","last_undegraded":"2024-09-11T07:29:10.439361+0000","last_fullsized":"2024-09-11T07:29:10.439361+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:28:03.164220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:03.164220+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020677629999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":61,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446290+0000","last_change":"2024-09-11T07:28:17.484549+0000","last_active":"2024-09-11T07:29:10.446290+0000","last_peered":"2024-09-11T07:29:10.446290+0000","last_clean":"2024-09-11T07:29:10.446290+0000","last_became_active":"2024-09-11T07:26:35.558809+0000","last_became_peered":"2024-09-11T07:26:35.558809+0000","last_unstale":"2024-09-11T07:29:10.446290+0000","last_undegraded":"2024-09-11T07:29:10.446290+0000","last_fullsized":"2024-09-11T07:29:10.446290+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:17.484488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:17.484488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:25.968070+0000","scrub_duration":0.001695118,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.442613+0000","last_change":"2024-09-11T07:28:00.126756+0000","last_active":"2024-09-11T07:29:10.442613+0000","last_peered":"2024-09-11T07:29:10.442613+0000","last_clean":"2024-09-11T07:29:10.442613+0000","last_became_active":"2024-09-11T07:26:37.862373+0000","last_became_peered":"2024-09-11T07:26:37.862373+0000","last_unstale":"2024-09-11T07:29:10.442613+0000","last_undegraded":"2024-09-11T07:29:10.442613+0000","last_fullsized":"2024-09-11T07:29:10.442613+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:00.126702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.126702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:18.534592+0000","scrub_duration":0.001184903,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":65,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.438837+0000","last_change":"2024-09-11T07:27:33.983889+0000","last_active":"2024-09-11T07:29:10.438837+0000","last_peered":"2024-09-11T07:29:10.438837+0000","last_clean":"2024-09-11T07:29:10.438837+0000","last_became_active":"2024-09-11T07:26:31.424651+0000","last_became_peered":"2024-09-11T07:26:31.424651+0000","last_unstale":"2024-09-11T07:29:10.438837+0000","last_undegraded":"2024-09-11T07:29:10.438837+0000","last_fullsized":"2024-09-11T07:29:10.438837+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:33.983813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:33.983813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001290759,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":88,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.440986+0000","last_change":"2024-09-11T07:28:00.272702+0000","last_active":"2024-09-11T07:29:10.440986+0000","last_peered":"2024-09-11T07:29:10.440986+0000","last_clean":"2024-09-11T07:29:10.440986+0000","last_became_active":"2024-09-11T07:26:33.542544+0000","last_became_peered":"2024-09-11T07:26:33.542544+0000","last_unstale":"2024-09-11T07:29:10.440986+0000","last_undegraded":"2024-09-11T07:29:10.440986+0000","last_fullsized":"2024-09-11T07:29:10.440986+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-09-11T07:28:00.272628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:28:00.272628+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021830019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":61,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.446679+0000","last_change":"2024-09-11T07:28:07.721176+0000","last_active":"2024-09-11T07:29:10.446679+0000","last_peered":"2024-09-11T07:29:10.446679+0000","last_clean":"2024-09-11T07:29:10.446679+0000","last_became_active":"2024-09-11T07:26:35.555862+0000","last_became_peered":"2024-09-11T07:26:35.555862+0000","last_unstale":"2024-09-11T07:29:10.446679+0000","last_undegraded":"2024-09-11T07:29:10.446679+0000","last_fullsized":"2024-09-11T07:29:10.446679+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:07.721112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:28:07.721112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:31.345841+0000","scrub_duration":0.0020675020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":59,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.449177+0000","last_change":"2024-09-11T07:28:06.674616+0000","last_active":"2024-09-11T07:29:10.449177+0000","last_peered":"2024-09-11T07:29:10.449177+0000","last_clean":"2024-09-11T07:29:10.449177+0000","last_became_active":"2024-09-11T07:26:37.861647+0000","last_became_peered":"2024-09-11T07:26:37.861647+0000","last_unstale":"2024-09-11T07:29:10.449177+0000","last_undegraded":"2024-09-11T07:29:10.449177+0000","last_fullsized":"2024-09-11T07:29:10.449177+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:28:06.674558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:28:06.674558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:17.029204+0000","scrub_duration":0.001489488,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":65,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.449253+0000","last_change":"2024-09-11T07:27:45.732604+0000","last_active":"2024-09-11T07:29:10.449253+0000","last_peered":"2024-09-11T07:29:10.449253+0000","last_clean":"2024-09-11T07:29:10.449253+0000","last_became_active":"2024-09-11T07:26:31.420598+0000","last_became_peered":"2024-09-11T07:26:31.420598+0000","last_unstale":"2024-09-11T07:29:10.449253+0000","last_undegraded":"2024-09-11T07:29:10.449253+0000","last_fullsized":"2024-09-11T07:29:10.449253+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:45.732528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:27:45.732528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015466659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"64'12","reported_seq":86,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.448845+0000","last_change":"2024-09-11T07:27:59.737848+0000","last_active":"2024-09-11T07:29:10.448845+0000","last_peered":"2024-09-11T07:29:10.448845+0000","last_clean":"2024-09-11T07:29:10.448845+0000","last_became_active":"2024-09-11T07:26:33.542711+0000","last_became_peered":"2024-09-11T07:26:33.542711+0000","last_unstale":"2024-09-11T07:29:10.448845+0000","last_undegraded":"2024-09-11T07:29:10.448845+0000","last_fullsized":"2024-09-11T07:29:10.448845+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-09-11T07:27:59.737791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:27:59.737791+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:11.558595+0000","scrub_duration":0.00367732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":61,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-11T07:29:10.441966+0000","last_change":"2024-09-11T07:27:57.199239+0000","last_active":"2024-09-11T07:29:10.441966+0000","last_peered":"2024-09-11T07:29:10.441966+0000","last_clean":"2024-09-11T07:29:10.441966+0000","last_became_active":"2024-09-11T07:26:35.562037+0000","last_became_peered":"2024-09-11T07:26:35.562037+0000","last_unstale":"2024-09-11T07:29:10.441966+0000","last_undegraded":"2024-09-11T07:29:10.441966+0000","last_fullsized":"2024-09-11T07:29:10.441966+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:27:57.199173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:27:57.199173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:29:14.093487+0000","scrub_duration":0.0015950770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":2549088256,"num_objects":38896,"num_object_clones":0,"num_object_copies":116688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38896,"num_write_kb":2489344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":7649034240,"data_stored":7649034240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38896,"ondisk_log_size":38896,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":444,"ondisk_log_size":444,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":161,"num_read_kb":156,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299430,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1068052,"kb_used_data":754060,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":92677100,"statfs":{"total":95995035648,"available":94901350400,"internally_reserved":0,"allocated":772157440,"data_stored":771709152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.50900000000000001,"15min":0.50800000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":1.397},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52700000000000002,"15min":0.52800000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69399999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":1.6419999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":1.6579999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.34899999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.66900000000000004,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":1.099}]},{"osd":2,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52400000000000002,"15min":0.52300000000000002},"min":{"1min":0.34599999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69199999999999995,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":3.3250000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53000000000000003,"15min":0.52800000000000002},"min":{"1min":0.38700000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73999999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":2.0099999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":1.0249999999999999,"15min":1.1080000000000001},"min":{"1min":0.34000000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70499999999999996,"5min":12.625999999999999,"15min":12.625999999999999},"last":4.0289999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.35899999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.72899999999999998,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":1.7629999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.44,"15min":0.442},"min":{"1min":0.29799999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.63100000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.442,"15min":0.44400000000000001},"min":{"1min":0.28999999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.47099999999999997}]},{"osd":5,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.436,"15min":0.44},"min":{"1min":0.33000000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.56200000000000006,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.499},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.33100000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70599999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.40000000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.34200000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67400000000000004,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.47899999999999998,"15min":0.47799999999999998},"min":{"1min":0.31,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70899999999999996,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.50700000000000001}]}]},{"osd":6,"up_from":44,"seq":188978561066,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1079228,"kb_used_data":765236,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":92665924,"statfs":{"total":95995035648,"available":94889906176,"internally_reserved":0,"allocated":783601664,"data_stored":783176430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:28:55 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48499999999999999,"15min":0.47899999999999998},"min":{"1min":0.28199999999999997,"5min":0.247,"15min":0.247},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.252},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58599999999999997,"15min":0.61799999999999999},"min":{"1min":0.316,"5min":0.31,"15min":0.31},"max":{"1min":0.75800000000000001,"5min":2.8210000000000002,"15min":2.8210000000000002},"last":0.35599999999999998}]},{"osd":1,"last update":"Wed Sep 11 07:28:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.57499999999999996,"15min":0.59999999999999998},"min":{"1min":0.33000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.80100000000000005,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.499,"15min":0.497},"min":{"1min":0.35499999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.86899999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:28:55 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.58899999999999997,"15min":0.627},"min":{"1min":0.32900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.79200000000000004,"5min":3.234,"15min":3.234},"last":1.052},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57999999999999996,"15min":0.60299999999999998},"min":{"1min":0.34300000000000003,"5min":0.313,"15min":0.313},"max":{"1min":0.78200000000000003,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.316}]},{"osd":3,"last update":"Wed Sep 11 07:28:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76300000000000001,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.61199999999999999,"15min":0.64800000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72099999999999997,"5min":3.1739999999999999,"15min":3.1739999999999999},"last":0.751}]},{"osd":4,"last update":"Wed Sep 11 07:28:55 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.55000000000000004,"15min":0.59799999999999998},"min":{"1min":0.251,"5min":0.249,"15min":0.249},"max":{"1min":0.76100000000000001,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.42699999999999999,"15min":0.42299999999999999},"min":{"1min":0.28699999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.65900000000000003,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.28899999999999998}]},{"osd":5,"last update":"Wed Sep 11 07:28:55 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.60299999999999998,"15min":0.67900000000000005},"min":{"1min":0.28999999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.65800000000000003,"5min":5.7329999999999997,"15min":5.7329999999999997},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.51200000000000001,"15min":0.52400000000000002},"min":{"1min":0.315,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72399999999999998,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":0.34300000000000003}]},{"osd":7,"last update":"Wed Sep 11 07:28:23 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.58499999999999996,"15min":0.60499999999999998},"min":{"1min":0.34999999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.66300000000000003,"5min":3.6349999999999998,"15min":3.6349999999999998},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.34300000000000003,"5min":0.313,"15min":0.313},"max":{"1min":0.67300000000000004,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.36599999999999999}]}]},{"osd":5,"up_from":37,"seq":158913790000,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":910336,"kb_used_data":608824,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":92834816,"statfs":{"total":95995035648,"available":95062851584,"internally_reserved":0,"allocated":623435776,"data_stored":623006893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:28:35 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.505,"15min":0.51100000000000001},"min":{"1min":0.317,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70599999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.5,"15min":0.5},"min":{"1min":0.34300000000000003,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.65700000000000003,"5min":1.107,"15min":1.107},"last":1.2210000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:28:35 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.52100000000000002,"15min":0.53000000000000003},"min":{"1min":0.316,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67000000000000004,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.443},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.49399999999999999,"15min":0.49299999999999999},"min":{"1min":0.29599999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.64800000000000002,"5min":1.073,"15min":1.073},"last":0.32300000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:28:35 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.373,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.67600000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.53200000000000003,"15min":0.53700000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64700000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.40699999999999997}]},{"osd":3,"last update":"Wed Sep 11 07:28:35 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.33000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.69899999999999995,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.377,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73699999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.34000000000000002}]},{"osd":4,"last update":"Wed Sep 11 07:28:35 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.378,"15min":0.371},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.495,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.42099999999999999,"15min":0.41799999999999998},"min":{"1min":0.26900000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51200000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.36299999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:29:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.42799999999999999,"15min":0.40999999999999998},"min":{"1min":0.35999999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70899999999999996,"5min":1.175,"15min":1.175},"last":1.3149999999999999},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.439,"15min":0.42599999999999999},"min":{"1min":0.34899999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66100000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.58699999999999997}]},{"osd":7,"last update":"Wed Sep 11 07:28:21 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.48099999999999998,"15min":0.48899999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53800000000000003,"5min":1.202,"15min":1.202},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.50700000000000001,"15min":0.51300000000000001},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.61799999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.39400000000000002}]}]},{"osd":4,"up_from":31,"seq":133143986228,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1057876,"kb_used_data":748044,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":92687276,"statfs":{"total":95995035648,"available":94911770624,"internally_reserved":0,"allocated":765997056,"data_stored":765548028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:28:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52700000000000002,"15min":0.51700000000000002},"min":{"1min":0.42699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.499},"min":{"1min":0.34999999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.40799999999999997}]},{"osd":1,"last update":"Wed Sep 11 07:28:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.50700000000000001,"15min":0.496},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72199999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.67100000000000004,"15min":0.72299999999999998},"min":{"1min":0.38100000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.81499999999999995,"5min":4.5510000000000002,"15min":4.5510000000000002},"last":0.248}]},{"osd":2,"last update":"Wed Sep 11 07:28:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53600000000000003,"15min":0.53000000000000003},"min":{"1min":0.40200000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.73499999999999999,"5min":1.21,"15min":1.21},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.36899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76800000000000002,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":1.6180000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:28:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.38,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.70099999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.753},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.28699999999999998}]},{"osd":5,"last update":"Wed Sep 11 07:28:32 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.36399999999999999,"15min":0.34399999999999997},"min":{"1min":0.28699999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.39500000000000002,"15min":0.377},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.27800000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:29:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.434,"15min":0.42199999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.45700000000000002,"15min":0.437},"min":{"1min":0.28699999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.68300000000000005,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.33300000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48999999999999999,"15min":0.48699999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.31},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48099999999999998,"15min":0.48299999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.34300000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1694492,"kb_used_data":1384660,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":92050660,"statfs":{"total":95995035648,"available":94259875840,"internally_reserved":0,"allocated":1417891840,"data_stored":1417434084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,7,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:28:55 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.376,"15min":0.39000000000000001},"min":{"1min":0.245,"5min":0.185,"15min":0.185},"max":{"1min":0.55400000000000005,"5min":0.621,"15min":0.621},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40600000000000003,"15min":0.434},"min":{"1min":0.26400000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.56200000000000006,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.66500000000000004}]},{"osd":1,"last update":"Wed Sep 11 07:28:55 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.41899999999999998,"15min":0.433},"min":{"1min":0.29199999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.61499999999999999,"5min":0.752,"15min":0.752},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.39800000000000002,"15min":0.42499999999999999},"min":{"1min":0.24199999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.51400000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.77500000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:28:55 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.434,"15min":0.45900000000000002},"min":{"1min":0.28000000000000003,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.59899999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.438,"15min":0.46400000000000002},"min":{"1min":0.27500000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.59799999999999998,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.52300000000000002}]},{"osd":4,"last update":"Wed Sep 11 07:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.747,"15min":0.621},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77200000000000002,"5min":21.856000000000002,"15min":21.856000000000002},"last":0.41299999999999998},{"interface":"front","average":{"1min":1.4399999999999999,"5min":0.89700000000000002,"15min":0.67700000000000005},"min":{"1min":0.436,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":18.370000000000001,"5min":19.146000000000001,"15min":19.146000000000001},"last":0.32400000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:28:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.42599999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.44},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.71799999999999997,"15min":0.622},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76800000000000002,"5min":18.416,"15min":18.416},"last":0.36299999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:28:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":1.123,"15min":1.3720000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.76700000000000002,"5min":20.219999999999999,"15min":20.219999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":1.1319999999999999,"15min":1.3839999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.77700000000000002,"5min":19.609000000000002,"15min":19.609000000000002},"last":0.47599999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.45900000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.80700000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.502},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59799999999999998,"15min":0.60399999999999998},"min":{"1min":0.39700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71899999999999997,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.38400000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1220348,"kb_used_data":922996,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":92524804,"statfs":{"total":95995035648,"available":94745399296,"internally_reserved":0,"allocated":945147904,"data_stored":944722363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:28:28 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.375,"15min":0.36299999999999999},"min":{"1min":0.20799999999999999,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.56299999999999994,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":1.4410000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.38200000000000001,"15min":0.374},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57599999999999996,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.76600000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:28:28 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.39800000000000002,"15min":0.38400000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.499,"5min":1.179,"15min":1.179},"last":0.438},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.41099999999999998,"15min":0.38600000000000001},"min":{"1min":0.255,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.66600000000000004,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":1.75}]},{"osd":3,"last update":"Wed Sep 11 07:28:54 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.435,"15min":0.42099999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.53500000000000003,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.316,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.51600000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.84699999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.94799999999999995,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.316},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.502}]},{"osd":5,"last update":"Wed Sep 11 07:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.66600000000000004,"15min":0.68999999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90400000000000003,"5min":2.931,"15min":2.931},"last":0.36899999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:29:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.61499999999999999,"15min":0.63500000000000001},"min":{"1min":0.38700000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.75800000000000001,"5min":1.494,"15min":1.494},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.623,"15min":0.63},"min":{"1min":0.41799999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.88600000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.40000000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.64700000000000002,"15min":0.65700000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.92700000000000005,"5min":1.514,"15min":1.514},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.53500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1199524,"kb_used_data":902172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":92545628,"statfs":{"total":95995035648,"available":94766723072,"internally_reserved":0,"allocated":923824128,"data_stored":923358817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:29:09 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.35199999999999998,"15min":0.33500000000000002},"min":{"1min":0.23000000000000001,"5min":0.186,"15min":0.186},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":1.4510000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.373,"15min":0.35799999999999998},"min":{"1min":0.25700000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.56000000000000005,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.218}]},{"osd":2,"last update":"Wed Sep 11 07:28:31 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.41099999999999998,"15min":0.41299999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.58799999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.377,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.26300000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.56699999999999995,"5min":1.075,"15min":1.075},"last":0.58899999999999997}]},{"osd":3,"last update":"Wed Sep 11 07:28:54 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.43099999999999999,"15min":0.443},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.57699999999999996,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.40799999999999997,"15min":0.41199999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.55700000000000005,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":1.881}]},{"osd":4,"last update":"Wed Sep 11 07:29:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53800000000000003,"15min":0.54600000000000004},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.74199999999999999,"5min":0.751,"15min":0.751},"last":0.499},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85499999999999998,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.53400000000000003}]},{"osd":5,"last update":"Wed Sep 11 07:28:36 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72399999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.68899999999999995,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.61199999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56799999999999995,"15min":0.57399999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76600000000000001,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55900000000000005,"15min":0.55200000000000005},"min":{"1min":0.40999999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.73999999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":1.8049999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":1.845}]}]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1547020,"kb_used_data":1237188,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":92198132,"statfs":{"total":95995035648,"available":94410887168,"internally_reserved":0,"allocated":1266880512,"data_stored":1266439226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:29:03 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.38300000000000001,"15min":0.35099999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":2.2320000000000002,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.37,"15min":0.34599999999999997},"min":{"1min":0.23300000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.1499999999999999,"5min":1.167,"15min":1.167},"last":1.2629999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:28:29 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.39400000000000002,"15min":0.40400000000000003},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.56799999999999995,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.39900000000000002,"15min":0.40300000000000002},"min":{"1min":0.255,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.54400000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.82599999999999996}]},{"osd":3,"last update":"Wed Sep 11 07:29:00 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.438,"15min":0.46700000000000003},"min":{"1min":0.27900000000000003,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.64300000000000002,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":1.3160000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.44,"15min":0.47299999999999998},"min":{"1min":0.28599999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.47299999999999998,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.85699999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:29:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.32400000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.61299999999999999,"15min":0.60599999999999998},"min":{"1min":0.39700000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":2.5510000000000002,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":0.63900000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:28:31 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.71099999999999997,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59999999999999998,"15min":0.61199999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.60499999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:29:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58099999999999996,"15min":0.59299999999999997},"min":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68999999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.42099999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.71399999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.88900000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:28:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58499999999999996,"15min":0.59299999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.748,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.755},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.42799999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.72299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1286668288,"data_stored":1286668288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":956235776,"data_stored":956235776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":955580416,"data_stored":955580416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1431961600,"data_stored":1431961600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":794886144,"data_stored":794886144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":624492544,"data_stored":624492544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":797638656,"data_stored":797638656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":801570816,"data_stored":801570816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:29:27.174 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj 6, currently 16 2024-09-11T07:29:27.221 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:27.264 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:27.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:27.265 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:27.366 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:27.395 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:27.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:27.396 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:27.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:27 smithi022 ceph-mon[29462]: 6.f scrub starts 2024-09-11T07:29:27.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:27 smithi022 ceph-mon[29462]: 6.f scrub ok 2024-09-11T07:29:27.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:27 smithi022 ceph-mon[29462]: pgmap v105: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 138 MiB/s wr, 2.21k op/s 2024-09-11T07:29:27.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:27 smithi022 ceph-mon[29462]: 4.13 scrub starts 2024-09-11T07:29:27.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:27 smithi022 ceph-mon[29462]: 4.13 scrub ok 2024-09-11T07:29:27.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:27 smithi022 ceph-mon[29462]: 2.7 scrub starts 2024-09-11T07:29:27.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:27 smithi022 ceph-mon[29462]: 2.7 scrub ok 2024-09-11T07:29:27.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:27 smithi022 ceph-mon[37330]: 6.f scrub starts 2024-09-11T07:29:27.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:27 smithi022 ceph-mon[37330]: 6.f scrub ok 2024-09-11T07:29:27.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:27 smithi022 ceph-mon[37330]: pgmap v105: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 138 MiB/s wr, 2.21k op/s 2024-09-11T07:29:27.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:27 smithi022 ceph-mon[37330]: 4.13 scrub starts 2024-09-11T07:29:27.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:27 smithi022 ceph-mon[37330]: 4.13 scrub ok 2024-09-11T07:29:27.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:27 smithi022 ceph-mon[37330]: 2.7 scrub starts 2024-09-11T07:29:27.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:27 smithi022 ceph-mon[37330]: 2.7 scrub ok 2024-09-11T07:29:27.497 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:27.525 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:27.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:27.526 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:27.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:27 smithi104 ceph-mon[33507]: 6.f scrub starts 2024-09-11T07:29:27.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:27 smithi104 ceph-mon[33507]: 6.f scrub ok 2024-09-11T07:29:27.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:27 smithi104 ceph-mon[33507]: pgmap v105: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 138 MiB/s wr, 2.21k op/s 2024-09-11T07:29:27.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:27 smithi104 ceph-mon[33507]: 4.13 scrub starts 2024-09-11T07:29:27.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:27 smithi104 ceph-mon[33507]: 4.13 scrub ok 2024-09-11T07:29:27.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:27 smithi104 ceph-mon[33507]: 2.7 scrub starts 2024-09-11T07:29:27.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:27 smithi104 ceph-mon[33507]: 2.7 scrub ok 2024-09-11T07:29:27.586 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 25 16 52754 52738 131.815 94.9375 0.00692157 0.00758153 2024-09-11T07:29:27.626 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:27.653 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:27.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:27.654 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:27.755 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:27.781 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:27.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:27.782 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:27.800 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:29:27.883 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:27.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:27.930 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:27.930 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:28.030 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:28.077 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:28.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:28.078 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:28.179 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:28.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:28.220 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:28.220 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:28.321 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:28.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:28.359 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:28.359 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:28.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:28 smithi022 ceph-mon[29462]: 6.5 scrub starts 2024-09-11T07:29:28.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:28 smithi022 ceph-mon[29462]: 6.5 scrub ok 2024-09-11T07:29:28.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:28 smithi022 ceph-mon[37330]: 6.5 scrub starts 2024-09-11T07:29:28.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:28 smithi022 ceph-mon[37330]: 6.5 scrub ok 2024-09-11T07:29:28.460 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:28.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:28.499 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:28.499 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:28.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:28 smithi104 ceph-mon[33507]: 6.5 scrub starts 2024-09-11T07:29:28.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:28 smithi104 ceph-mon[33507]: 6.5 scrub ok 2024-09-11T07:29:28.586 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 26 16 54293 54277 130.445 96.1875 0.0064144 0.00766028 2024-09-11T07:29:28.599 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:28.626 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:28.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:28.627 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:28.728 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:28.756 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:28.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:28.757 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:28.858 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:28.905 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:28.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:28.906 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:29.007 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:29.034 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:29.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:29.035 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:29.112 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:29:29.112 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:29:29.136 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:29.150 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:29:29.150 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:29:29.160 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:29.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:29.161 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:29.189 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:29:29.189 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:29:29.224 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:29:29.224 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:29:29.257 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:29:29.257 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:29:29.262 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:29.266 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:29 smithi022 ceph-mon[29462]: pgmap v106: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-11T07:29:29.266 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:29 smithi022 ceph-mon[29462]: 2.3 scrub starts 2024-09-11T07:29:29.266 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:29 smithi022 ceph-mon[29462]: 2.3 scrub ok 2024-09-11T07:29:29.266 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:29 smithi022 ceph-mon[29462]: 5.2 scrub starts 2024-09-11T07:29:29.266 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:29 smithi022 ceph-mon[29462]: 5.2 scrub ok 2024-09-11T07:29:29.267 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:29 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2305006219' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:29:29.267 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:29 smithi022 ceph-mon[37330]: pgmap v106: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-11T07:29:29.267 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:29 smithi022 ceph-mon[37330]: 2.3 scrub starts 2024-09-11T07:29:29.267 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:29 smithi022 ceph-mon[37330]: 2.3 scrub ok 2024-09-11T07:29:29.267 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:29 smithi022 ceph-mon[37330]: 5.2 scrub starts 2024-09-11T07:29:29.267 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:29 smithi022 ceph-mon[37330]: 5.2 scrub ok 2024-09-11T07:29:29.267 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:29 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2305006219' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:29:29.300 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:29:29.301 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:29:29.312 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:29.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:29.313 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:29.333 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:29:29.334 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:29:29.368 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:29:29.368 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:29:29.414 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:29.442 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:29.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:29.443 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:29.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:29:29 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:29:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:29 smithi104 ceph-mon[33507]: pgmap v106: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-11T07:29:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:29 smithi104 ceph-mon[33507]: 2.3 scrub starts 2024-09-11T07:29:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:29 smithi104 ceph-mon[33507]: 2.3 scrub ok 2024-09-11T07:29:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:29 smithi104 ceph-mon[33507]: 5.2 scrub starts 2024-09-11T07:29:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:29 smithi104 ceph-mon[33507]: 5.2 scrub ok 2024-09-11T07:29:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:29 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2305006219' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:29:29.544 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:29.570 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:29.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:29.571 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:29.586 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 27 16 55761 55745 129.011 91.75 0.0147422 0.00774604 2024-09-11T07:29:29.673 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:29.713 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:29.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:29.714 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:29.814 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:29.842 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:29.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:29.843 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:29.944 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:29.986 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:29.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:29.987 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:30.088 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:30.129 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:30.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:30.130 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:30.232 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:30.259 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:30.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:30.260 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:30.361 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:30.374 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:30 smithi022 ceph-mon[29462]: 2.19 scrub starts 2024-09-11T07:29:30.374 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:30 smithi022 ceph-mon[29462]: 2.19 scrub ok 2024-09-11T07:29:30.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:30 smithi022 ceph-mon[29462]: 5.6 scrub starts 2024-09-11T07:29:30.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:30 smithi022 ceph-mon[29462]: 5.6 scrub ok 2024-09-11T07:29:30.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:30 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:29:30.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:30 smithi022 ceph-mon[29462]: 4.d scrub starts 2024-09-11T07:29:30.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:30 smithi022 ceph-mon[29462]: 4.d scrub ok 2024-09-11T07:29:30.376 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:30 smithi022 ceph-mon[37330]: 2.19 scrub starts 2024-09-11T07:29:30.377 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:30 smithi022 ceph-mon[37330]: 2.19 scrub ok 2024-09-11T07:29:30.377 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:30 smithi022 ceph-mon[37330]: 5.6 scrub starts 2024-09-11T07:29:30.377 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:30 smithi022 ceph-mon[37330]: 5.6 scrub ok 2024-09-11T07:29:30.377 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:30 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:29:30.377 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:30 smithi022 ceph-mon[37330]: 4.d scrub starts 2024-09-11T07:29:30.377 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:30 smithi022 ceph-mon[37330]: 4.d scrub ok 2024-09-11T07:29:30.402 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:30.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:30.404 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:30.504 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:30.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:30 smithi104 ceph-mon[33507]: 2.19 scrub starts 2024-09-11T07:29:30.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:30 smithi104 ceph-mon[33507]: 2.19 scrub ok 2024-09-11T07:29:30.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:30 smithi104 ceph-mon[33507]: 5.6 scrub starts 2024-09-11T07:29:30.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:30 smithi104 ceph-mon[33507]: 5.6 scrub ok 2024-09-11T07:29:30.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:30 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:29:30.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:30 smithi104 ceph-mon[33507]: 4.d scrub starts 2024-09-11T07:29:30.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:30 smithi104 ceph-mon[33507]: 4.d scrub ok 2024-09-11T07:29:30.549 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:30.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:30.550 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:30.586 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 28 16 57260 57244 127.75 93.6875 0.0159293 0.00782214 2024-09-11T07:29:30.651 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:30.668 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:29:30 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:29:30] "GET /metrics HTTP/1.1" 200 223456 "" "Prometheus/2.43.0" 2024-09-11T07:29:30.693 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:30.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:30.694 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:30.794 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:30.821 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:30.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:30.822 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:30.923 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:30.949 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:30.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:30.950 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:31.050 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:31.093 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:31.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:31.094 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:31.195 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:31.224 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:31.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:31.225 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:31.326 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:31.356 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:31.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:31.357 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:31.359 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:31 smithi022 ceph-mon[29462]: pgmap v107: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.0 KiB/s rd, 100 MiB/s wr, 1.61k op/s 2024-09-11T07:29:31.359 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:31 smithi022 ceph-mon[29462]: 2.b scrub starts 2024-09-11T07:29:31.359 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:31 smithi022 ceph-mon[29462]: 2.b scrub ok 2024-09-11T07:29:31.359 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:31 smithi022 ceph-mon[29462]: 4.15 scrub starts 2024-09-11T07:29:31.359 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:31 smithi022 ceph-mon[29462]: 4.15 scrub ok 2024-09-11T07:29:31.359 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:31 smithi022 ceph-mon[29462]: 2.f scrub starts 2024-09-11T07:29:31.359 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:31 smithi022 ceph-mon[29462]: 2.f scrub ok 2024-09-11T07:29:31.360 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:31 smithi022 ceph-mon[29462]: 2.1f scrub starts 2024-09-11T07:29:31.360 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:31 smithi022 ceph-mon[29462]: 2.1f scrub ok 2024-09-11T07:29:31.360 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:31 smithi022 ceph-mon[37330]: pgmap v107: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.0 KiB/s rd, 100 MiB/s wr, 1.61k op/s 2024-09-11T07:29:31.360 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:31 smithi022 ceph-mon[37330]: 2.b scrub starts 2024-09-11T07:29:31.360 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:31 smithi022 ceph-mon[37330]: 2.b scrub ok 2024-09-11T07:29:31.360 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:31 smithi022 ceph-mon[37330]: 4.15 scrub starts 2024-09-11T07:29:31.360 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:31 smithi022 ceph-mon[37330]: 4.15 scrub ok 2024-09-11T07:29:31.360 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:31 smithi022 ceph-mon[37330]: 2.f scrub starts 2024-09-11T07:29:31.360 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:31 smithi022 ceph-mon[37330]: 2.f scrub ok 2024-09-11T07:29:31.360 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:31 smithi022 ceph-mon[37330]: 2.1f scrub starts 2024-09-11T07:29:31.360 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:31 smithi022 ceph-mon[37330]: 2.1f scrub ok 2024-09-11T07:29:31.458 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:31.483 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:31.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:31.484 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:31 smithi104 ceph-mon[33507]: pgmap v107: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.0 KiB/s rd, 100 MiB/s wr, 1.61k op/s 2024-09-11T07:29:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:31 smithi104 ceph-mon[33507]: 2.b scrub starts 2024-09-11T07:29:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:31 smithi104 ceph-mon[33507]: 2.b scrub ok 2024-09-11T07:29:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:31 smithi104 ceph-mon[33507]: 4.15 scrub starts 2024-09-11T07:29:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:31 smithi104 ceph-mon[33507]: 4.15 scrub ok 2024-09-11T07:29:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:31 smithi104 ceph-mon[33507]: 2.f scrub starts 2024-09-11T07:29:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:31 smithi104 ceph-mon[33507]: 2.f scrub ok 2024-09-11T07:29:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:31 smithi104 ceph-mon[33507]: 2.1f scrub starts 2024-09-11T07:29:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:31 smithi104 ceph-mon[33507]: 2.1f scrub ok 2024-09-11T07:29:31.585 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:31.586 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 29 16 58694 58678 126.435 89.625 0.006839 0.0079018 2024-09-11T07:29:31.609 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:31.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:31.610 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:31.711 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:31.737 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:29:31.738 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:29:31.738 INFO:teuthology.orchestra.run.smithi022.stdout: "complaint_time": 30, 2024-09-11T07:29:31.738 INFO:teuthology.orchestra.run.smithi022.stdout: "num_blocked_ops": 0 2024-09-11T07:29:31.738 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:29:31.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:31.749 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:31.749 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:31.850 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:31.876 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:31.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:31.877 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:31.978 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:32.016 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:32.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:32.019 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:32.120 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:32.144 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:32.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:32.145 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:32.175 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:29:32.175 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:29:32.175 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-11T07:29:32.175 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-11T07:29:32.247 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:32.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:32.272 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:32.272 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:32.373 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:32.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:32 smithi022 ceph-mon[29462]: 2.c scrub starts 2024-09-11T07:29:32.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:32 smithi022 ceph-mon[29462]: 2.c scrub ok 2024-09-11T07:29:32.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:32 smithi022 ceph-mon[29462]: 4.10 scrub starts 2024-09-11T07:29:32.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:32 smithi022 ceph-mon[29462]: 4.10 scrub ok 2024-09-11T07:29:32.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:32 smithi022 ceph-mon[29462]: 5.16 scrub starts 2024-09-11T07:29:32.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:32 smithi022 ceph-mon[29462]: 5.16 scrub ok 2024-09-11T07:29:32.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:32 smithi022 ceph-mon[29462]: 2.4 scrub starts 2024-09-11T07:29:32.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:32 smithi022 ceph-mon[29462]: 2.4 scrub ok 2024-09-11T07:29:32.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:32 smithi022 ceph-mon[37330]: 2.c scrub starts 2024-09-11T07:29:32.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:32 smithi022 ceph-mon[37330]: 2.c scrub ok 2024-09-11T07:29:32.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:32 smithi022 ceph-mon[37330]: 4.10 scrub starts 2024-09-11T07:29:32.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:32 smithi022 ceph-mon[37330]: 4.10 scrub ok 2024-09-11T07:29:32.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:32 smithi022 ceph-mon[37330]: 5.16 scrub starts 2024-09-11T07:29:32.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:32 smithi022 ceph-mon[37330]: 5.16 scrub ok 2024-09-11T07:29:32.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:32 smithi022 ceph-mon[37330]: 2.4 scrub starts 2024-09-11T07:29:32.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:32 smithi022 ceph-mon[37330]: 2.4 scrub ok 2024-09-11T07:29:32.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:32.426 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:32.426 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:32.527 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:32.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:32 smithi104 ceph-mon[33507]: 2.c scrub starts 2024-09-11T07:29:32.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:32 smithi104 ceph-mon[33507]: 2.c scrub ok 2024-09-11T07:29:32.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:32 smithi104 ceph-mon[33507]: 4.10 scrub starts 2024-09-11T07:29:32.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:32 smithi104 ceph-mon[33507]: 4.10 scrub ok 2024-09-11T07:29:32.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:32 smithi104 ceph-mon[33507]: 5.16 scrub starts 2024-09-11T07:29:32.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:32 smithi104 ceph-mon[33507]: 5.16 scrub ok 2024-09-11T07:29:32.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:32 smithi104 ceph-mon[33507]: 2.4 scrub starts 2024-09-11T07:29:32.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:32 smithi104 ceph-mon[33507]: 2.4 scrub ok 2024-09-11T07:29:32.568 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:32.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:32.569 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:32.586 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 30 16 59784 59768 124.491 68.125 0.00899347 0.00802655 2024-09-11T07:29:32.670 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:32.692 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:32.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:32.693 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:32.794 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:32.833 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:32.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:32.834 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:32.935 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:32.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:32.985 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:32.985 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:33.086 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:33.113 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:33.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:33.114 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:33.122 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:29:33.214 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:33.241 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:33.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:33.241 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:33.342 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:33.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:33.388 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:33.388 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:33.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:33 smithi022 ceph-mon[29462]: 4.1c scrub starts 2024-09-11T07:29:33.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:33 smithi022 ceph-mon[29462]: 4.1c scrub ok 2024-09-11T07:29:33.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:33 smithi022 ceph-mon[29462]: pgmap v108: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 889 B/s rd, 87 MiB/s wr, 1.39k op/s 2024-09-11T07:29:33.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:33 smithi022 ceph-mon[29462]: 4.1a scrub starts 2024-09-11T07:29:33.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:33 smithi022 ceph-mon[29462]: 4.12 scrub starts 2024-09-11T07:29:33.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:33 smithi022 ceph-mon[29462]: 4.1a scrub ok 2024-09-11T07:29:33.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:33 smithi022 ceph-mon[29462]: 4.12 scrub ok 2024-09-11T07:29:33.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:33 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:29:33.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:33 smithi022 ceph-mon[29462]: 3.2 scrub starts 2024-09-11T07:29:33.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:33 smithi022 ceph-mon[29462]: 3.2 scrub ok 2024-09-11T07:29:33.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:33 smithi022 ceph-mon[37330]: 4.1c scrub starts 2024-09-11T07:29:33.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:33 smithi022 ceph-mon[37330]: 4.1c scrub ok 2024-09-11T07:29:33.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:33 smithi022 ceph-mon[37330]: pgmap v108: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 889 B/s rd, 87 MiB/s wr, 1.39k op/s 2024-09-11T07:29:33.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:33 smithi022 ceph-mon[37330]: 4.1a scrub starts 2024-09-11T07:29:33.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:33 smithi022 ceph-mon[37330]: 4.12 scrub starts 2024-09-11T07:29:33.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:33 smithi022 ceph-mon[37330]: 4.1a scrub ok 2024-09-11T07:29:33.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:33 smithi022 ceph-mon[37330]: 4.12 scrub ok 2024-09-11T07:29:33.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:33 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:29:33.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:33 smithi022 ceph-mon[37330]: 3.2 scrub starts 2024-09-11T07:29:33.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:33 smithi022 ceph-mon[37330]: 3.2 scrub ok 2024-09-11T07:29:33.489 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:33.518 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:33.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:33.519 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:33.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:33 smithi104 ceph-mon[33507]: 4.1c scrub starts 2024-09-11T07:29:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:33 smithi104 ceph-mon[33507]: 4.1c scrub ok 2024-09-11T07:29:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:33 smithi104 ceph-mon[33507]: pgmap v108: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 889 B/s rd, 87 MiB/s wr, 1.39k op/s 2024-09-11T07:29:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:33 smithi104 ceph-mon[33507]: 4.1a scrub starts 2024-09-11T07:29:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:33 smithi104 ceph-mon[33507]: 4.12 scrub starts 2024-09-11T07:29:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:33 smithi104 ceph-mon[33507]: 4.1a scrub ok 2024-09-11T07:29:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:33 smithi104 ceph-mon[33507]: 4.12 scrub ok 2024-09-11T07:29:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:33 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:29:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:33 smithi104 ceph-mon[33507]: 3.2 scrub starts 2024-09-11T07:29:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:33 smithi104 ceph-mon[33507]: 3.2 scrub ok 2024-09-11T07:29:33.587 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 31 16 60924 60908 122.771 71.25 0.0155718 0.00813987 2024-09-11T07:29:33.619 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:33.666 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-11T07:29:33.670 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:33.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:33.672 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:33.772 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:33.818 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:33.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:33.819 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:33.920 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:33.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:33.944 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:33.944 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:34.045 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:34.078 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:29:34.102 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:29:34.106 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:34.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:34.107 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:34.149 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:34 smithi022 ceph-mon[29462]: 2.5 scrub starts 2024-09-11T07:29:34.208 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:34.231 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:34.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:34.232 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:34.333 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:34.362 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:34.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:34.363 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:34.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:34 smithi022 ceph-mon[29462]: 2.5 scrub ok 2024-09-11T07:29:34.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:34 smithi022 ceph-mon[29462]: 5.1c scrub starts 2024-09-11T07:29:34.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:34 smithi022 ceph-mon[29462]: 5.1c scrub ok 2024-09-11T07:29:34.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:34 smithi022 ceph-mon[29462]: 4.1f scrub starts 2024-09-11T07:29:34.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:34 smithi022 ceph-mon[29462]: 4.1f scrub ok 2024-09-11T07:29:34.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:34 smithi022 ceph-mon[29462]: 6.c scrub starts 2024-09-11T07:29:34.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:34 smithi022 ceph-mon[29462]: 6.c scrub ok 2024-09-11T07:29:34.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:34 smithi022 ceph-mon[37330]: 2.5 scrub starts 2024-09-11T07:29:34.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:34 smithi022 ceph-mon[37330]: 2.5 scrub ok 2024-09-11T07:29:34.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:34 smithi022 ceph-mon[37330]: 5.1c scrub starts 2024-09-11T07:29:34.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:34 smithi022 ceph-mon[37330]: 5.1c scrub ok 2024-09-11T07:29:34.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:34 smithi022 ceph-mon[37330]: 4.1f scrub starts 2024-09-11T07:29:34.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:34 smithi022 ceph-mon[37330]: 4.1f scrub ok 2024-09-11T07:29:34.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:34 smithi022 ceph-mon[37330]: 6.c scrub starts 2024-09-11T07:29:34.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:34 smithi022 ceph-mon[37330]: 6.c scrub ok 2024-09-11T07:29:34.464 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:34.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:34.509 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:34.510 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:34.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:29:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:29:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:29:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:34 smithi104 ceph-mon[33507]: 2.5 scrub starts 2024-09-11T07:29:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:34 smithi104 ceph-mon[33507]: 2.5 scrub ok 2024-09-11T07:29:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:34 smithi104 ceph-mon[33507]: 5.1c scrub starts 2024-09-11T07:29:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:34 smithi104 ceph-mon[33507]: 5.1c scrub ok 2024-09-11T07:29:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:34 smithi104 ceph-mon[33507]: 4.1f scrub starts 2024-09-11T07:29:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:34 smithi104 ceph-mon[33507]: 4.1f scrub ok 2024-09-11T07:29:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:34 smithi104 ceph-mon[33507]: 6.c scrub starts 2024-09-11T07:29:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:34 smithi104 ceph-mon[33507]: 6.c scrub ok 2024-09-11T07:29:34.587 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 32 16 62000 61984 121.036 67.25 0.00519806 0.00825688 2024-09-11T07:29:34.611 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:34.637 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:34.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:34.639 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:34.739 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:34.791 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:34.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:34.793 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:34.893 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:34.921 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:34.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:34.922 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:35.022 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:35.049 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:35.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:35.050 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:35.150 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:35.178 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:35.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:35.179 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:35.254 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is unset 2024-09-11T07:29:35.280 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:35.323 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:35.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:35.325 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:35.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:35 smithi022 ceph-mon[29462]: 2.e scrub starts 2024-09-11T07:29:35.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:35 smithi022 ceph-mon[29462]: 2.e scrub ok 2024-09-11T07:29:35.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:35 smithi022 ceph-mon[29462]: pgmap v109: 148 pgs: 148 active+clean; 3.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 117 MiB/s wr, 1.87k op/s 2024-09-11T07:29:35.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:35 smithi022 ceph-mon[29462]: 2.1d scrub starts 2024-09-11T07:29:35.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:35 smithi022 ceph-mon[29462]: 2.1d scrub ok 2024-09-11T07:29:35.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:35 smithi022 ceph-mon[29462]: 2.9 scrub starts 2024-09-11T07:29:35.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:35 smithi022 ceph-mon[29462]: 2.9 scrub ok 2024-09-11T07:29:35.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:35 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:35.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:35 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:29:35.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:35 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:29:35.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:35 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3552007819' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:29:35.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:35 smithi022 ceph-mon[37330]: 2.e scrub starts 2024-09-11T07:29:35.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:35 smithi022 ceph-mon[37330]: 2.e scrub ok 2024-09-11T07:29:35.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:35 smithi022 ceph-mon[37330]: pgmap v109: 148 pgs: 148 active+clean; 3.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 117 MiB/s wr, 1.87k op/s 2024-09-11T07:29:35.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:35 smithi022 ceph-mon[37330]: 2.1d scrub starts 2024-09-11T07:29:35.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:35 smithi022 ceph-mon[37330]: 2.1d scrub ok 2024-09-11T07:29:35.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:35 smithi022 ceph-mon[37330]: 2.9 scrub starts 2024-09-11T07:29:35.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:35 smithi022 ceph-mon[37330]: 2.9 scrub ok 2024-09-11T07:29:35.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:35 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:35.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:35 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:29:35.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:35 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:29:35.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:35 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3552007819' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:29:35.425 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:35.455 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:35.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:35.456 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:35.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:35 smithi104 ceph-mon[33507]: 2.e scrub starts 2024-09-11T07:29:35.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:35 smithi104 ceph-mon[33507]: 2.e scrub ok 2024-09-11T07:29:35.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:35 smithi104 ceph-mon[33507]: pgmap v109: 148 pgs: 148 active+clean; 3.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 117 MiB/s wr, 1.87k op/s 2024-09-11T07:29:35.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:35 smithi104 ceph-mon[33507]: 2.1d scrub starts 2024-09-11T07:29:35.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:35 smithi104 ceph-mon[33507]: 2.1d scrub ok 2024-09-11T07:29:35.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:35 smithi104 ceph-mon[33507]: 2.9 scrub starts 2024-09-11T07:29:35.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:35 smithi104 ceph-mon[33507]: 2.9 scrub ok 2024-09-11T07:29:35.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:35 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:29:35.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:35 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:29:35.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:35 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:29:35.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:35 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3552007819' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:29:35.557 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:35.587 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 33 16 63392 63376 120.005 87 0.00751995 0.00832797 2024-09-11T07:29:35.594 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:35.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:35.595 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:35.696 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:35.726 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:35.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:35.727 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:35.828 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:35.854 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:35.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:35.855 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:35.956 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:36.000 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:36.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:36.001 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:36.102 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:36.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:36.148 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:36.148 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:36.249 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:36.275 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:36.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:36.276 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:36.376 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:36.425 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:36.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:36.427 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:36.528 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:36.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:36 smithi104 ceph-mon[33507]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:29:36.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:36 smithi104 ceph-mon[33507]: Cluster is now healthy 2024-09-11T07:29:36.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:36 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3552007819' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:29:36.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:36 smithi104 ceph-mon[33507]: osdmap e106: 8 total, 8 up, 8 in 2024-09-11T07:29:36.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:36 smithi104 ceph-mon[33507]: 2.14 scrub starts 2024-09-11T07:29:36.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:36 smithi104 ceph-mon[33507]: 2.14 scrub ok 2024-09-11T07:29:36.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:36 smithi104 ceph-mon[33507]: osdmap e107: 8 total, 8 up, 8 in 2024-09-11T07:29:36.554 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:36.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:36.555 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:36.587 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 34 16 64788 64772 119.041 87.25 0.00385264 0.00839526 2024-09-11T07:29:36.604 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:29:36.655 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:36.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:36 smithi022 ceph-mon[29462]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:29:36.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:36 smithi022 ceph-mon[29462]: Cluster is now healthy 2024-09-11T07:29:36.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:36 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3552007819' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:29:36.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:36 smithi022 ceph-mon[29462]: osdmap e106: 8 total, 8 up, 8 in 2024-09-11T07:29:36.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:36 smithi022 ceph-mon[29462]: 2.14 scrub starts 2024-09-11T07:29:36.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:36 smithi022 ceph-mon[29462]: 2.14 scrub ok 2024-09-11T07:29:36.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:36 smithi022 ceph-mon[29462]: osdmap e107: 8 total, 8 up, 8 in 2024-09-11T07:29:36.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:36 smithi022 ceph-mon[37330]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:29:36.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:36 smithi022 ceph-mon[37330]: Cluster is now healthy 2024-09-11T07:29:36.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:36 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3552007819' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:29:36.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:36 smithi022 ceph-mon[37330]: osdmap e106: 8 total, 8 up, 8 in 2024-09-11T07:29:36.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:36 smithi022 ceph-mon[37330]: 2.14 scrub starts 2024-09-11T07:29:36.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:36 smithi022 ceph-mon[37330]: 2.14 scrub ok 2024-09-11T07:29:36.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:36 smithi022 ceph-mon[37330]: osdmap e107: 8 total, 8 up, 8 in 2024-09-11T07:29:36.707 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:36.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:36.709 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:36.810 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:36.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:36.850 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:36.850 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:36.951 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:36.979 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:36.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:36.980 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:37.081 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:37.121 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:37.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:37.122 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:37.223 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:37.270 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:37.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:37.271 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:37.372 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:37.401 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:37.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:37.402 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:37.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:37 smithi022 ceph-mon[29462]: pgmap v112: 148 pgs: 148 active+clean; 3.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 111 MiB/s wr, 1.78k op/s 2024-09-11T07:29:37.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:37 smithi022 ceph-mon[29462]: 5.f scrub starts 2024-09-11T07:29:37.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:37 smithi022 ceph-mon[29462]: 5.f scrub ok 2024-09-11T07:29:37.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:37 smithi022 ceph-mon[29462]: 5.11 scrub starts 2024-09-11T07:29:37.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:37 smithi022 ceph-mon[29462]: 5.11 scrub ok 2024-09-11T07:29:37.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:37 smithi022 ceph-mon[37330]: pgmap v112: 148 pgs: 148 active+clean; 3.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 111 MiB/s wr, 1.78k op/s 2024-09-11T07:29:37.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:37 smithi022 ceph-mon[37330]: 5.f scrub starts 2024-09-11T07:29:37.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:37 smithi022 ceph-mon[37330]: 5.f scrub ok 2024-09-11T07:29:37.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:37 smithi022 ceph-mon[37330]: 5.11 scrub starts 2024-09-11T07:29:37.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:37 smithi022 ceph-mon[37330]: 5.11 scrub ok 2024-09-11T07:29:37.503 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:37.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:37 smithi104 ceph-mon[33507]: pgmap v112: 148 pgs: 148 active+clean; 3.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 111 MiB/s wr, 1.78k op/s 2024-09-11T07:29:37.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:37 smithi104 ceph-mon[33507]: 5.f scrub starts 2024-09-11T07:29:37.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:37 smithi104 ceph-mon[33507]: 5.f scrub ok 2024-09-11T07:29:37.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:37 smithi104 ceph-mon[33507]: 5.11 scrub starts 2024-09-11T07:29:37.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:37 smithi104 ceph-mon[33507]: 5.11 scrub ok 2024-09-11T07:29:37.530 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:37.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:37.531 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:37.587 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 35 16 66173 66157 118.113 86.5625 0.00799561 0.00846101 2024-09-11T07:29:37.632 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:37.703 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:37.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:37.704 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:37.805 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:37.857 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:37.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:37.858 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:37.959 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:37.984 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:37.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:37.985 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:38.086 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:38.114 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:38.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:38.115 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:38.215 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:38.256 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:38.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:38.259 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:38.360 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:38.404 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:38.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:38.405 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:38.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:38 smithi022 ceph-mon[29462]: 5.19 scrub starts 2024-09-11T07:29:38.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:38 smithi022 ceph-mon[29462]: 5.19 scrub ok 2024-09-11T07:29:38.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:38 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:29:38.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:38 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:29:38.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:29:38.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:29:38.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:38 smithi022 ceph-mon[37330]: 5.19 scrub starts 2024-09-11T07:29:38.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:38 smithi022 ceph-mon[37330]: 5.19 scrub ok 2024-09-11T07:29:38.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:38 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:29:38.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:38 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:29:38.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:29:38.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:29:38.507 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:38.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:38 smithi104 ceph-mon[33507]: 5.19 scrub starts 2024-09-11T07:29:38.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:38 smithi104 ceph-mon[33507]: 5.19 scrub ok 2024-09-11T07:29:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:38 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:29:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:38 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:29:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:29:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:29:38.553 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:38.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:38.554 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:38.587 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 36 16 67394 67378 116.952 76.3125 0.0308041 0.00854496 2024-09-11T07:29:38.655 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:38.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:38.696 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:38.696 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:38.755 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:29:38.757 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:29:38.758 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 108, 2024-09-11T07:29:38.758 INFO:teuthology.orchestra.run.smithi022.stdout: "fsid": "12a285d8-700e-11ef-bcea-c7b262605968", 2024-09-11T07:29:38.758 INFO:teuthology.orchestra.run.smithi022.stdout: "created": "2024-09-11T07:21:18.695332+0000", 2024-09-11T07:29:38.758 INFO:teuthology.orchestra.run.smithi022.stdout: "modified": "2024-09-11T07:29:38.167828+0000", 2024-09-11T07:29:38.758 INFO:teuthology.orchestra.run.smithi022.stdout: "last_up_change": "2024-09-11T07:26:15.385688+0000", 2024-09-11T07:29:38.758 INFO:teuthology.orchestra.run.smithi022.stdout: "last_in_change": "2024-09-11T07:25:58.551813+0000", 2024-09-11T07:29:38.758 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-11T07:29:38.758 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_num": 5799936, 2024-09-11T07:29:38.758 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_set": [ 2024-09-11T07:29:38.758 INFO:teuthology.orchestra.run.smithi022.stdout: "pglog_hardlimit", 2024-09-11T07:29:38.758 INFO:teuthology.orchestra.run.smithi022.stdout: "purged_snapdirs", 2024-09-11T07:29:38.758 INFO:teuthology.orchestra.run.smithi022.stdout: "recovery_deletes", 2024-09-11T07:29:38.758 INFO:teuthology.orchestra.run.smithi022.stdout: "sortbitwise" 2024-09-11T07:29:38.758 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:29:38.759 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_version": 18, 2024-09-11T07:29:38.759 INFO:teuthology.orchestra.run.smithi022.stdout: "full_ratio": 0.94999998807907104, 2024-09-11T07:29:38.759 INFO:teuthology.orchestra.run.smithi022.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-11T07:29:38.759 INFO:teuthology.orchestra.run.smithi022.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-11T07:29:38.759 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_snapshot": "", 2024-09-11T07:29:38.759 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_max": 7, 2024-09-11T07:29:38.759 INFO:teuthology.orchestra.run.smithi022.stdout: "max_osd": 8, 2024-09-11T07:29:38.759 INFO:teuthology.orchestra.run.smithi022.stdout: "require_min_compat_client": "luminous", 2024-09-11T07:29:38.759 INFO:teuthology.orchestra.run.smithi022.stdout: "min_compat_client": "luminous", 2024-09-11T07:29:38.759 INFO:teuthology.orchestra.run.smithi022.stdout: "require_osd_release": "quincy", 2024-09-11T07:29:38.759 INFO:teuthology.orchestra.run.smithi022.stdout: "pools": [ 2024-09-11T07:29:38.759 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.759 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 1, 2024-09-11T07:29:38.759 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": ".mgr", 2024-09-11T07:29:38.759 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:24:21.772426+0000", 2024-09-11T07:29:38.760 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:29:38.760 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:29:38.760 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:29:38.760 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:29:38.760 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:29:38.760 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:29:38.760 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:29:38.760 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:29:38.760 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:29:38.760 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:29:38.760 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:29:38.760 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:29:38.760 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 1, 2024-09-11T07:29:38.760 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 1, 2024-09-11T07:29:38.761 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 1, 2024-09-11T07:29:38.761 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 1, 2024-09-11T07:29:38.761 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 1, 2024-09-11T07:29:38.761 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:29:38.761 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:29:38.761 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:29:38.761 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:29:38.761 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:29:38.761 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:29:38.761 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:29:38.761 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.761 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "21", 2024-09-11T07:29:38.761 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:29:38.762 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:29:38.762 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:29:38.762 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:29:38.762 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:29:38.762 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:29:38.762 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:29:38.762 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:29:38.762 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:29:38.762 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:29:38.762 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:29:38.762 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:29:38.762 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:29:38.762 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:29:38.762 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:29:38.762 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:29:38.763 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:29:38.763 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:29:38.763 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:29:38.763 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:29:38.763 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:29:38.763 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:29:38.763 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:29:38.763 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:29:38.763 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:29:38.763 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:29:38.763 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.763 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:29:38.763 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:29:38.763 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:29:38.764 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:29:38.764 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:29:38.764 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:29:38.764 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:29:38.764 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:29:38.764 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:29:38.764 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:29:38.764 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:29:38.764 INFO:teuthology.orchestra.run.smithi022.stdout: "options": { 2024-09-11T07:29:38.764 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_max": 32, 2024-09-11T07:29:38.764 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_min": 1 2024-09-11T07:29:38.764 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.764 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:29:38.764 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": {} 2024-09-11T07:29:38.764 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.765 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.765 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.765 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 2, 2024-09-11T07:29:38.765 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": ".rgw.root", 2024-09-11T07:29:38.765 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:29.941354+0000", 2024-09-11T07:29:38.765 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:29:38.765 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:29:38.765 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:29:38.765 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:29:38.765 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:29:38.765 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:29:38.765 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:29:38.765 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:29:38.765 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:29:38.765 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:29:38.766 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:29:38.766 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:29:38.766 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:29:38.766 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:29:38.766 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:29:38.766 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:29:38.766 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:29:38.766 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:29:38.766 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:29:38.766 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:29:38.766 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:29:38.766 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:29:38.766 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:29:38.766 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:29:38.766 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.767 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "58", 2024-09-11T07:29:38.767 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:29:38.767 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:29:38.767 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:29:38.767 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:29:38.767 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:29:38.767 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:29:38.767 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:29:38.767 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:29:38.767 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:29:38.767 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:29:38.767 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:29:38.767 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:29:38.767 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:29:38.767 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:29:38.768 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:29:38.768 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:29:38.768 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:29:38.768 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:29:38.768 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:29:38.768 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:29:38.768 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:29:38.768 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:29:38.768 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:29:38.768 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:29:38.768 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:29:38.768 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:29:38.768 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.768 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:29:38.769 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:29:38.769 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:29:38.769 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:29:38.769 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:29:38.769 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:29:38.769 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:29:38.769 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:29:38.769 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:29:38.769 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:29:38.769 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:29:38.769 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:29:38.769 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:29:38.769 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:29:38.769 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.770 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.770 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.770 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 3, 2024-09-11T07:29:38.770 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "datapool", 2024-09-11T07:29:38.770 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:30.396859+0000", 2024-09-11T07:29:38.770 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 8193, 2024-09-11T07:29:38.770 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T07:29:38.770 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:29:38.770 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:29:38.770 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:29:38.770 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:29:38.770 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:29:38.770 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:29:38.770 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:29:38.770 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:29:38.771 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:29:38.771 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:29:38.771 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 3, 2024-09-11T07:29:38.771 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 3, 2024-09-11T07:29:38.771 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 3, 2024-09-11T07:29:38.771 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 3, 2024-09-11T07:29:38.771 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 3, 2024-09-11T07:29:38.771 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:29:38.771 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:29:38.771 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:29:38.771 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:29:38.771 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:29:38.771 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:29:38.771 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:29:38.771 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.772 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "61", 2024-09-11T07:29:38.772 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:29:38.772 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:29:38.772 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:29:38.772 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:29:38.772 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:29:38.772 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 3, 2024-09-11T07:29:38.772 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 61, 2024-09-11T07:29:38.772 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:29:38.772 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:29:38.772 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:29:38.772 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:29:38.772 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:29:38.772 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:29:38.772 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:29:38.773 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:29:38.773 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:29:38.773 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:29:38.773 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:29:38.773 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:29:38.773 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:29:38.773 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:29:38.773 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:29:38.773 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:29:38.773 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:29:38.773 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:29:38.773 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:29:38.773 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.773 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:29:38.774 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:29:38.774 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:29:38.774 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:29:38.774 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:29:38.774 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:29:38.774 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:29:38.774 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:29:38.774 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:29:38.774 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:29:38.774 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:29:38.774 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:29:38.774 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:29:38.774 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd": {} 2024-09-11T07:29:38.774 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.774 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.775 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.775 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 4, 2024-09-11T07:29:38.775 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.log", 2024-09-11T07:29:38.775 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:31.922926+0000", 2024-09-11T07:29:38.775 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:29:38.775 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:29:38.775 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:29:38.775 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:29:38.775 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:29:38.775 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:29:38.775 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:29:38.775 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:29:38.775 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:29:38.775 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:29:38.775 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:29:38.776 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:29:38.776 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:29:38.776 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:29:38.776 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:29:38.776 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:29:38.776 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:29:38.776 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:29:38.776 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:29:38.776 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:29:38.776 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:29:38.776 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:29:38.776 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:29:38.776 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:29:38.776 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.777 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "60", 2024-09-11T07:29:38.777 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:29:38.777 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:29:38.777 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:29:38.777 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:29:38.777 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:29:38.777 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:29:38.777 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:29:38.777 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:29:38.777 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:29:38.777 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:29:38.777 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:29:38.777 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:29:38.777 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:29:38.777 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:29:38.777 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:29:38.778 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:29:38.778 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:29:38.778 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:29:38.778 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:29:38.778 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:29:38.778 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:29:38.778 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:29:38.778 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:29:38.778 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:29:38.778 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:29:38.778 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:29:38.778 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.778 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:29:38.778 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:29:38.778 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:29:38.779 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:29:38.779 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:29:38.779 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:29:38.779 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:29:38.779 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:29:38.779 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:29:38.779 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:29:38.779 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:29:38.779 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:29:38.779 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:29:38.779 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:29:38.779 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.779 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.779 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.779 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 5, 2024-09-11T07:29:38.780 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.control", 2024-09-11T07:29:38.780 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:34.007746+0000", 2024-09-11T07:29:38.780 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:29:38.780 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:29:38.780 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:29:38.780 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:29:38.780 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:29:38.780 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:29:38.780 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:29:38.780 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:29:38.780 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:29:38.780 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:29:38.780 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:29:38.780 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:29:38.780 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:29:38.781 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:29:38.781 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:29:38.781 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:29:38.781 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:29:38.781 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:29:38.781 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:29:38.781 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:29:38.781 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:29:38.781 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:29:38.781 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:29:38.781 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:29:38.781 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.781 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "62", 2024-09-11T07:29:38.781 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:29:38.782 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:29:38.782 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:29:38.782 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:29:38.782 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:29:38.782 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:29:38.782 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:29:38.782 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:29:38.782 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:29:38.782 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:29:38.782 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:29:38.782 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:29:38.782 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:29:38.782 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:29:38.782 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:29:38.782 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:29:38.783 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:29:38.783 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:29:38.783 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:29:38.783 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:29:38.783 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:29:38.783 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:29:38.783 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:29:38.783 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:29:38.783 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:29:38.783 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:29:38.783 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.783 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:29:38.783 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:29:38.783 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:29:38.783 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:29:38.784 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:29:38.784 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:29:38.784 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:29:38.784 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:29:38.784 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:29:38.784 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:29:38.784 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:29:38.784 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:29:38.784 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:29:38.784 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:29:38.784 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.784 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.784 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.784 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 6, 2024-09-11T07:29:38.784 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.meta", 2024-09-11T07:29:38.785 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:36.077215+0000", 2024-09-11T07:29:38.785 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:29:38.785 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:29:38.785 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:29:38.785 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:29:38.785 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:29:38.785 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:29:38.785 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:29:38.785 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:29:38.785 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:29:38.785 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:29:38.785 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:29:38.785 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:29:38.785 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:29:38.785 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:29:38.786 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:29:38.786 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:29:38.786 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:29:38.786 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:29:38.786 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:29:38.786 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:29:38.786 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:29:38.786 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:29:38.786 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:29:38.786 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:29:38.786 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.786 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "64", 2024-09-11T07:29:38.786 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:29:38.786 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:29:38.786 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:29:38.787 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:29:38.787 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:29:38.787 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:29:38.787 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:29:38.787 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:29:38.787 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:29:38.787 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:29:38.787 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:29:38.787 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:29:38.787 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:29:38.787 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:29:38.787 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:29:38.787 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:29:38.787 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:29:38.788 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:29:38.788 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:29:38.788 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:29:38.788 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:29:38.788 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:29:38.788 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:29:38.788 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:29:38.788 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:29:38.788 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:29:38.788 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.788 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:29:38.788 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:29:38.788 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:29:38.788 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:29:38.788 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:29:38.789 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:29:38.789 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:29:38.789 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:29:38.789 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:29:38.789 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:29:38.789 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:29:38.789 INFO:teuthology.orchestra.run.smithi022.stdout: "options": { 2024-09-11T07:29:38.789 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_bias": 4 2024-09-11T07:29:38.789 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.790 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:29:38.790 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:29:38.790 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.790 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.790 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.790 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 7, 2024-09-11T07:29:38.790 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "unique_pool_0", 2024-09-11T07:29:38.790 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:28:48.880975+0000", 2024-09-11T07:29:38.790 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:29:38.790 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:29:38.790 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:29:38.790 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:29:38.790 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:29:38.790 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:29:38.791 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:29:38.791 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:29:38.791 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:29:38.791 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:29:38.791 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:29:38.791 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:29:38.791 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 16, 2024-09-11T07:29:38.791 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 16, 2024-09-11T07:29:38.791 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 16, 2024-09-11T07:29:38.791 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 16, 2024-09-11T07:29:38.791 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 16, 2024-09-11T07:29:38.791 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:29:38.791 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:29:38.791 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:29:38.791 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:29:38.791 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:29:38.792 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:29:38.792 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:29:38.792 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.792 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "96", 2024-09-11T07:29:38.792 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:29:38.792 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:29:38.792 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:29:38.792 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:29:38.792 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:29:38.792 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:29:38.792 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:29:38.792 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:29:38.792 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:29:38.792 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:29:38.792 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:29:38.793 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:29:38.793 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:29:38.793 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:29:38.793 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:29:38.793 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:29:38.793 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:29:38.793 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:29:38.793 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:29:38.793 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:29:38.793 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:29:38.793 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:29:38.793 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:29:38.793 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:29:38.793 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:29:38.793 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:29:38.794 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.794 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:29:38.794 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:29:38.794 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:29:38.794 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:29:38.794 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:29:38.794 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:29:38.794 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:29:38.794 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:29:38.794 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:29:38.794 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:29:38.794 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:29:38.794 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:29:38.794 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:29:38.794 INFO:teuthology.orchestra.run.smithi022.stdout: "rados": {} 2024-09-11T07:29:38.795 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.795 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.795 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:29:38.795 INFO:teuthology.orchestra.run.smithi022.stdout: "osds": [ 2024-09-11T07:29:38.795 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.795 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 0, 2024-09-11T07:29:38.795 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "09075f16-32f4-4f88-a116-c3d7f977a0e0", 2024-09-11T07:29:38.795 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:29:38.795 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:29:38.795 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:29:38.795 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:29:38.795 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:29:38.795 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:29:38.795 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 8, 2024-09-11T07:29:38.795 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 92, 2024-09-11T07:29:38.796 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:29:38.796 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:29:38.796 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:29:38.796 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.796 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.796 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.796 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6801", 2024-09-11T07:29:38.796 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:29:38.796 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.796 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.796 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.796 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:29:38.796 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.796 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.796 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.797 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6802", 2024-09-11T07:29:38.797 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:29:38.797 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.797 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.797 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.797 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:29:38.797 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.797 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.797 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.797 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6804", 2024-09-11T07:29:38.797 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:29:38.797 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.797 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.797 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.798 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:29:38.798 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.798 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.798 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.798 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6803", 2024-09-11T07:29:38.798 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:29:38.798 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.798 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.798 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.798 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6801/2821493591", 2024-09-11T07:29:38.799 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6802/2821493591", 2024-09-11T07:29:38.799 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6804/2821493591", 2024-09-11T07:29:38.799 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6803/2821493591", 2024-09-11T07:29:38.799 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:29:38.799 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:29:38.799 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:29:38.799 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.799 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.799 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.799 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 1, 2024-09-11T07:29:38.799 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "52cc1e06-9506-48c0-9f7b-d4270dff4b9f", 2024-09-11T07:29:38.799 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:29:38.799 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:29:38.799 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:29:38.800 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:29:38.800 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:29:38.800 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:29:38.800 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 13, 2024-09-11T07:29:38.800 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 92, 2024-09-11T07:29:38.800 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:29:38.800 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:29:38.800 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:29:38.800 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.800 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.800 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.800 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6805", 2024-09-11T07:29:38.800 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:29:38.800 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.801 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.801 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.801 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:29:38.801 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.801 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.801 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.801 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6806", 2024-09-11T07:29:38.801 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:29:38.801 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.801 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.801 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.801 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:29:38.801 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.801 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.801 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.802 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6808", 2024-09-11T07:29:38.802 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:29:38.802 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.802 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.802 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.802 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:29:38.802 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.802 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.802 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.802 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6807", 2024-09-11T07:29:38.802 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:29:38.802 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.802 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.802 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.802 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6805/4037911317", 2024-09-11T07:29:38.803 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6806/4037911317", 2024-09-11T07:29:38.803 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6808/4037911317", 2024-09-11T07:29:38.803 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6807/4037911317", 2024-09-11T07:29:38.803 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:29:38.803 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:29:38.803 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:29:38.803 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.803 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.803 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.803 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 2, 2024-09-11T07:29:38.803 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "fa2b2730-2c1d-45d0-be8a-494a60dc6b9b", 2024-09-11T07:29:38.803 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:29:38.803 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:29:38.803 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:29:38.803 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:29:38.804 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:29:38.804 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:29:38.804 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 18, 2024-09-11T07:29:38.804 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 92, 2024-09-11T07:29:38.804 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:29:38.804 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:29:38.804 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:29:38.804 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.804 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.804 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.804 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6809", 2024-09-11T07:29:38.804 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:29:38.804 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.804 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.805 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.805 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:29:38.805 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.805 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.805 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.805 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6810", 2024-09-11T07:29:38.805 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:29:38.805 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.805 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.805 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.805 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:29:38.805 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.805 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.805 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.805 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6812", 2024-09-11T07:29:38.805 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:29:38.806 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.806 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.806 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.806 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:29:38.806 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.806 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.806 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.806 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6811", 2024-09-11T07:29:38.806 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:29:38.806 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.806 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.806 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.806 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6809/1675799666", 2024-09-11T07:29:38.806 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6810/1675799666", 2024-09-11T07:29:38.806 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6812/1675799666", 2024-09-11T07:29:38.807 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6811/1675799666", 2024-09-11T07:29:38.807 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:29:38.807 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:29:38.807 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:29:38.807 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.807 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.807 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.807 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 3, 2024-09-11T07:29:38.807 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "8d53d3fc-e401-47df-b70b-4b4b1f4539df", 2024-09-11T07:29:38.807 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:29:38.807 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:29:38.807 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:29:38.807 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:29:38.807 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:29:38.807 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:29:38.808 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 25, 2024-09-11T07:29:38.808 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 92, 2024-09-11T07:29:38.808 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:29:38.808 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:29:38.808 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:29:38.808 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.808 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.808 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.808 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6813", 2024-09-11T07:29:38.808 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:29:38.808 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.808 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.808 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.809 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:29:38.809 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.809 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.809 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.809 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6814", 2024-09-11T07:29:38.809 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:29:38.809 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.809 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.809 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.809 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:29:38.809 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.809 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.809 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.810 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6816", 2024-09-11T07:29:38.810 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:29:38.810 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.810 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.810 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.810 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:29:38.810 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.810 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.810 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.810 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6815", 2024-09-11T07:29:38.810 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:29:38.810 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.810 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.810 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.810 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6813/1501962752", 2024-09-11T07:29:38.811 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6814/1501962752", 2024-09-11T07:29:38.811 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6816/1501962752", 2024-09-11T07:29:38.811 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6815/1501962752", 2024-09-11T07:29:38.811 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:29:38.811 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:29:38.811 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:29:38.811 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.811 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.811 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.811 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 4, 2024-09-11T07:29:38.811 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "d3207fe0-8b97-4067-a755-ba6d449c53c7", 2024-09-11T07:29:38.811 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:29:38.811 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:29:38.811 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:29:38.812 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:29:38.812 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:29:38.812 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:29:38.812 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 31, 2024-09-11T07:29:38.812 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 92, 2024-09-11T07:29:38.812 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:29:38.812 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:29:38.812 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:29:38.812 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.812 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.812 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.812 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6800", 2024-09-11T07:29:38.812 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:29:38.813 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.813 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.813 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.813 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:29:38.813 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.813 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.813 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.813 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6801", 2024-09-11T07:29:38.813 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:29:38.813 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.813 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.813 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.813 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:29:38.813 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.814 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.814 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.814 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6803", 2024-09-11T07:29:38.814 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:29:38.814 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.814 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.814 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.814 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:29:38.814 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.814 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.814 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.814 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6802", 2024-09-11T07:29:38.814 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:29:38.814 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.815 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.815 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.815 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6800/1210229443", 2024-09-11T07:29:38.815 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6801/1210229443", 2024-09-11T07:29:38.815 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6803/1210229443", 2024-09-11T07:29:38.815 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6802/1210229443", 2024-09-11T07:29:38.815 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:29:38.815 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:29:38.815 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:29:38.815 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.815 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.815 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.815 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 5, 2024-09-11T07:29:38.815 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "89c9352b-30b2-48c6-869e-047d07a23a43", 2024-09-11T07:29:38.816 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:29:38.816 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:29:38.816 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:29:38.816 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:29:38.816 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:29:38.816 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:29:38.816 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 37, 2024-09-11T07:29:38.816 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 100, 2024-09-11T07:29:38.816 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:29:38.816 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:29:38.816 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:29:38.816 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.816 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.817 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.817 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6804", 2024-09-11T07:29:38.817 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:29:38.817 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.817 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.817 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.817 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:29:38.817 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.817 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.817 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.817 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6805", 2024-09-11T07:29:38.817 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:29:38.817 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.817 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.818 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.818 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:29:38.818 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.818 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.818 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.818 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6807", 2024-09-11T07:29:38.818 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:29:38.818 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.818 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.818 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.818 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:29:38.818 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.818 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.818 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.819 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6806", 2024-09-11T07:29:38.819 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:29:38.819 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.819 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.819 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.819 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6804/2735328732", 2024-09-11T07:29:38.819 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6805/2735328732", 2024-09-11T07:29:38.819 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6807/2735328732", 2024-09-11T07:29:38.819 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6806/2735328732", 2024-09-11T07:29:38.819 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:29:38.819 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:29:38.819 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:29:38.819 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.819 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.819 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.820 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 6, 2024-09-11T07:29:38.820 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "704563de-d88f-480a-b469-720135704d42", 2024-09-11T07:29:38.820 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:29:38.820 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:29:38.820 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:29:38.820 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:29:38.820 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:29:38.820 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:29:38.820 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 44, 2024-09-11T07:29:38.820 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 92, 2024-09-11T07:29:38.820 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:29:38.820 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:29:38.820 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:29:38.820 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.821 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.821 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.821 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6808", 2024-09-11T07:29:38.821 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:29:38.821 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.821 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.821 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.821 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:29:38.821 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.821 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.821 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.821 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6809", 2024-09-11T07:29:38.821 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:29:38.821 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.821 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.821 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.822 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:29:38.822 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.822 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.822 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.822 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6811", 2024-09-11T07:29:38.822 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:29:38.822 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.822 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.822 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.822 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:29:38.822 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.822 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.822 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.822 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6810", 2024-09-11T07:29:38.823 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:29:38.823 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.823 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.823 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.823 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6808/721493010", 2024-09-11T07:29:38.823 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6809/721493010", 2024-09-11T07:29:38.823 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6811/721493010", 2024-09-11T07:29:38.823 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6810/721493010", 2024-09-11T07:29:38.823 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:29:38.823 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:29:38.823 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:29:38.823 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.823 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.823 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.824 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 7, 2024-09-11T07:29:38.824 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "18199e47-0052-49f0-b45f-0fb201e0d0f4", 2024-09-11T07:29:38.824 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:29:38.824 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:29:38.824 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:29:38.824 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:29:38.824 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:29:38.824 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:29:38.824 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 52, 2024-09-11T07:29:38.824 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 92, 2024-09-11T07:29:38.824 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:29:38.824 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:29:38.824 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:29:38.824 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.825 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.825 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.825 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6812", 2024-09-11T07:29:38.825 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3018423954 2024-09-11T07:29:38.825 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.825 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.825 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.825 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:29:38.825 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.825 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.825 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.825 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6813", 2024-09-11T07:29:38.825 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3018423954 2024-09-11T07:29:38.825 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.825 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.826 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.826 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:29:38.826 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.826 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:38.827 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.827 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.827 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6815", 2024-09-11T07:29:38.827 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3018423954 2024-09-11T07:29:38.827 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.827 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.827 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.827 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:29:38.827 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:29:38.827 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.827 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:29:38.827 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6814", 2024-09-11T07:29:38.827 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3018423954 2024-09-11T07:29:38.828 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.828 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.828 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.828 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6812/3018423954", 2024-09-11T07:29:38.828 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6813/3018423954", 2024-09-11T07:29:38.828 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6815/3018423954", 2024-09-11T07:29:38.828 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6814/3018423954", 2024-09-11T07:29:38.828 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:29:38.828 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:29:38.828 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:29:38.828 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.828 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.828 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:29:38.829 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_xinfo": [ 2024-09-11T07:29:38.829 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.829 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 0, 2024-09-11T07:29:38.829 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:29:38.829 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:29:38.829 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:29:38.829 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:29:38.829 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:29:38.829 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:28:46.781105+0000", 2024-09-11T07:29:38.829 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:29:38.829 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.829 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.829 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 1, 2024-09-11T07:29:38.829 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:29:38.829 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:29:38.830 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:29:38.830 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:29:38.830 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:29:38.830 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:28:54.400218+0000", 2024-09-11T07:29:38.830 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:29:38.830 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.830 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.830 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 2, 2024-09-11T07:29:38.830 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:29:38.830 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:29:38.830 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:29:38.830 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:29:38.830 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:29:38.830 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:29:00.368036+0000", 2024-09-11T07:29:38.830 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:29:38.831 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.831 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.831 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 3, 2024-09-11T07:29:38.831 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:29:38.831 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:29:38.831 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:29:38.831 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:29:38.831 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:29:38.831 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:28:41.908744+0000", 2024-09-11T07:29:38.831 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:29:38.831 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.831 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.831 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 4, 2024-09-11T07:29:38.831 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:29:38.831 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:29:38.832 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:29:38.832 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:29:38.832 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:29:38.832 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:28:37.656398+0000", 2024-09-11T07:29:38.832 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:29:38.832 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.832 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.832 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 5, 2024-09-11T07:29:38.832 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:29:38.832 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:29:38.832 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:29:38.832 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:29:38.832 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:29:38.832 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:29:10.678249+0000", 2024-09-11T07:29:38.832 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:29:38.833 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.833 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.833 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 6, 2024-09-11T07:29:38.833 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:29:38.833 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:29:38.833 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:29:38.833 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:29:38.833 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:29:38.833 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:28:26.989477+0000", 2024-09-11T07:29:38.833 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:29:38.833 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.833 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.833 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 7, 2024-09-11T07:29:38.833 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:29:38.833 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:29:38.834 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:29:38.834 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:29:38.834 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:29:38.834 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:29:37.516403+0000", 2024-09-11T07:29:38.834 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:29:38.834 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.834 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:29:38.834 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_upmap": [ 2024-09-11T07:29:38.834 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.834 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "2.1c", 2024-09-11T07:29:38.834 INFO:teuthology.orchestra.run.smithi022.stdout: "osds": [ 2024-09-11T07:29:38.834 INFO:teuthology.orchestra.run.smithi022.stdout: 5, 2024-09-11T07:29:38.834 INFO:teuthology.orchestra.run.smithi022.stdout: 2, 2024-09-11T07:29:38.834 INFO:teuthology.orchestra.run.smithi022.stdout: 7 2024-09-11T07:29:38.834 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.835 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.835 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:29:38.835 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_upmap_items": [ 2024-09-11T07:29:38.835 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.835 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "2.7", 2024-09-11T07:29:38.835 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:29:38.835 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.835 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 7, 2024-09-11T07:29:38.835 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 4 2024-09-11T07:29:38.835 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.835 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.835 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.835 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.835 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.6", 2024-09-11T07:29:38.835 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:29:38.835 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.836 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 1, 2024-09-11T07:29:38.836 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 5 2024-09-11T07:29:38.836 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.836 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.836 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.836 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.836 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.d", 2024-09-11T07:29:38.836 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:29:38.836 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.836 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 1, 2024-09-11T07:29:38.836 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 5 2024-09-11T07:29:38.836 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.836 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.836 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.836 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.837 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.1f", 2024-09-11T07:29:38.837 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:29:38.837 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:38.837 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 1, 2024-09-11T07:29:38.837 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 2 2024-09-11T07:29:38.837 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.837 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:38.837 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.837 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:29:38.837 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_temp": [], 2024-09-11T07:29:38.837 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_temp": [], 2024-09-11T07:29:38.837 INFO:teuthology.orchestra.run.smithi022.stdout: "blocklist": { 2024-09-11T07:29:38.837 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/240999923": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:29:38.837 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/4016369460": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:29:38.837 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/585929238": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:29:38.838 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/544031024": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:29:38.838 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/2504196132": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:29:38.838 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/1451388349": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:29:38.838 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/639336081": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:29:38.838 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2104069730": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:29:38.838 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/4236450731": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:29:38.838 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/3573193776": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:29:38.838 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2661764422": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:29:38.838 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2197488530": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:29:38.838 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2968034970": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:29:38.838 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/599318175": "2024-09-12T07:27:07.919998+0000" 2024-09-11T07:29:38.838 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.838 INFO:teuthology.orchestra.run.smithi022.stdout: "range_blocklist": {}, 2024-09-11T07:29:38.838 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profiles": { 2024-09-11T07:29:38.838 INFO:teuthology.orchestra.run.smithi022.stdout: "default": { 2024-09-11T07:29:38.839 INFO:teuthology.orchestra.run.smithi022.stdout: "crush-failure-domain": "osd", 2024-09-11T07:29:38.839 INFO:teuthology.orchestra.run.smithi022.stdout: "k": "2", 2024-09-11T07:29:38.839 INFO:teuthology.orchestra.run.smithi022.stdout: "m": "1", 2024-09-11T07:29:38.839 INFO:teuthology.orchestra.run.smithi022.stdout: "plugin": "jerasure", 2024-09-11T07:29:38.839 INFO:teuthology.orchestra.run.smithi022.stdout: "technique": "reed_sol_van" 2024-09-11T07:29:38.839 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.839 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:38.839 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps_queue": [], 2024-09-11T07:29:38.839 INFO:teuthology.orchestra.run.smithi022.stdout: "new_removed_snaps": [], 2024-09-11T07:29:38.839 INFO:teuthology.orchestra.run.smithi022.stdout: "new_purged_snaps": [], 2024-09-11T07:29:38.839 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_node_flags": {}, 2024-09-11T07:29:38.839 INFO:teuthology.orchestra.run.smithi022.stdout: "device_class_flags": {}, 2024-09-11T07:29:38.839 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode": { 2024-09-11T07:29:38.839 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode_enabled": false, 2024-09-11T07:29:38.839 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_bucket_count": 0, 2024-09-11T07:29:38.839 INFO:teuthology.orchestra.run.smithi022.stdout: "degraded_stretch_mode": 0, 2024-09-11T07:29:38.840 INFO:teuthology.orchestra.run.smithi022.stdout: "recovering_stretch_mode": 0, 2024-09-11T07:29:38.840 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode_bucket": 0 2024-09-11T07:29:38.840 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:38.840 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:29:38.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:38.878 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:38.878 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:38.979 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:39.004 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:39.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:39.005 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:39.106 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:39.131 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:39.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:39.132 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:39.145 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T07:29:39.232 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:39.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:39.279 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:39.279 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:39.380 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:39.408 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:39.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:39.409 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:39.510 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:39.530 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:29:39 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:29:39.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:39 smithi104 ceph-mon[33507]: pgmap v113: 148 pgs: 148 active+clean; 3.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 51 MiB/s wr, 809 op/s 2024-09-11T07:29:39.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:39 smithi104 ceph-mon[33507]: 4.e deep-scrub starts 2024-09-11T07:29:39.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:39 smithi104 ceph-mon[33507]: 4.e deep-scrub ok 2024-09-11T07:29:39.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:39 smithi104 ceph-mon[33507]: osdmap e108: 8 total, 8 up, 8 in 2024-09-11T07:29:39.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:39 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1881231071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:29:39.562 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:39.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:39.565 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:39.588 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 37 16 68621 68605 115.861 76.6875 0.0118916 0.00862639 2024-09-11T07:29:39.666 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:39.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:39 smithi022 ceph-mon[29462]: pgmap v113: 148 pgs: 148 active+clean; 3.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 51 MiB/s wr, 809 op/s 2024-09-11T07:29:39.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:39 smithi022 ceph-mon[29462]: 4.e deep-scrub starts 2024-09-11T07:29:39.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:39 smithi022 ceph-mon[29462]: 4.e deep-scrub ok 2024-09-11T07:29:39.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:39 smithi022 ceph-mon[29462]: osdmap e108: 8 total, 8 up, 8 in 2024-09-11T07:29:39.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:39 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1881231071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:29:39.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:39 smithi022 ceph-mon[37330]: pgmap v113: 148 pgs: 148 active+clean; 3.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 51 MiB/s wr, 809 op/s 2024-09-11T07:29:39.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:39 smithi022 ceph-mon[37330]: 4.e deep-scrub starts 2024-09-11T07:29:39.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:39 smithi022 ceph-mon[37330]: 4.e deep-scrub ok 2024-09-11T07:29:39.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:39 smithi022 ceph-mon[37330]: osdmap e108: 8 total, 8 up, 8 in 2024-09-11T07:29:39.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:39 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1881231071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:29:39.712 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:39.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:39.713 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:39.814 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:39.824 INFO:tasks.thrashosds.thrasher:j is {'epoch': 108, 'fsid': '12a285d8-700e-11ef-bcea-c7b262605968', 'created': '2024-09-11T07:21:18.695332+0000', 'modified': '2024-09-11T07:29:38.167828+0000', 'last_up_change': '2024-09-11T07:26:15.385688+0000', 'last_in_change': '2024-09-11T07:25:58.551813+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 7, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T07:24:21.772426+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': '.rgw.root', 'create_time': '2024-09-11T07:26:29.941354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 3, 'pool_name': 'datapool', 'create_time': '2024-09-11T07:26:30.396859+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-11T07:26:31.922926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-11T07:26:34.007746+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-11T07:26:36.077215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}, {'pool': 7, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-11T07:28:48.880975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '09075f16-32f4-4f88-a116-c3d7f977a0e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 92, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6801', 'nonce': 2821493591}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6802', 'nonce': 2821493591}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6804', 'nonce': 2821493591}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6803', 'nonce': 2821493591}]}, 'public_addr': '172.21.15.22:6801/2821493591', 'cluster_addr': '172.21.15.22:6802/2821493591', 'heartbeat_back_addr': '172.21.15.22:6804/2821493591', 'heartbeat_front_addr': '172.21.15.22:6803/2821493591', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '52cc1e06-9506-48c0-9f7b-d4270dff4b9f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 92, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6805', 'nonce': 4037911317}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6806', 'nonce': 4037911317}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6808', 'nonce': 4037911317}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6807', 'nonce': 4037911317}]}, 'public_addr': '172.21.15.22:6805/4037911317', 'cluster_addr': '172.21.15.22:6806/4037911317', 'heartbeat_back_addr': '172.21.15.22:6808/4037911317', 'heartbeat_front_addr': '172.21.15.22:6807/4037911317', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'fa2b2730-2c1d-45d0-be8a-494a60dc6b9b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 92, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6809', 'nonce': 1675799666}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6810', 'nonce': 1675799666}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6812', 'nonce': 1675799666}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6811', 'nonce': 1675799666}]}, 'public_addr': '172.21.15.22:6809/1675799666', 'cluster_addr': '172.21.15.22:6810/1675799666', 'heartbeat_back_addr': '172.21.15.22:6812/1675799666', 'heartbeat_front_addr': '172.21.15.22:6811/1675799666', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '8d53d3fc-e401-47df-b70b-4b4b1f4539df', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 25, 'up_thru': 92, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6813', 'nonce': 1501962752}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6814', 'nonce': 1501962752}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6816', 'nonce': 1501962752}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6815', 'nonce': 1501962752}]}, 'public_addr': '172.21.15.22:6813/1501962752', 'cluster_addr': '172.21.15.22:6814/1501962752', 'heartbeat_back_addr': '172.21.15.22:6816/1501962752', 'heartbeat_front_addr': '172.21.15.22:6815/1501962752', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'd3207fe0-8b97-4067-a755-ba6d449c53c7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 92, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6800', 'nonce': 1210229443}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6801', 'nonce': 1210229443}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6803', 'nonce': 1210229443}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6802', 'nonce': 1210229443}]}, 'public_addr': '172.21.15.104:6800/1210229443', 'cluster_addr': '172.21.15.104:6801/1210229443', 'heartbeat_back_addr': '172.21.15.104:6803/1210229443', 'heartbeat_front_addr': '172.21.15.104:6802/1210229443', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '89c9352b-30b2-48c6-869e-047d07a23a43', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 37, 'up_thru': 100, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6804', 'nonce': 2735328732}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6805', 'nonce': 2735328732}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6807', 'nonce': 2735328732}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6806', 'nonce': 2735328732}]}, 'public_addr': '172.21.15.104:6804/2735328732', 'cluster_addr': '172.21.15.104:6805/2735328732', 'heartbeat_back_addr': '172.21.15.104:6807/2735328732', 'heartbeat_front_addr': '172.21.15.104:6806/2735328732', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '704563de-d88f-480a-b469-720135704d42', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 44, 'up_thru': 92, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6808', 'nonce': 721493010}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6809', 'nonce': 721493010}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6811', 'nonce': 721493010}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6810', 'nonce': 721493010}]}, 'public_addr': '172.21.15.104:6808/721493010', 'cluster_addr': '172.21.15.104:6809/721493010', 'heartbeat_back_addr': '172.21.15.104:6811/721493010', 'heartbeat_front_addr': '172.21.15.104:6810/721493010', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '18199e47-0052-49f0-b45f-0fb201e0d0f4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 52, 'up_thru': 92, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6812', 'nonce': 3018423954}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6813', 'nonce': 3018423954}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6815', 'nonce': 3018423954}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6814', 'nonce': 3018423954}]}, 'public_addr': '172.21.15.104:6812/3018423954', 'cluster_addr': '172.21.15.104:6813/3018423954', 'heartbeat_back_addr': '172.21.15.104:6815/3018423954', 'heartbeat_front_addr': '172.21.15.104:6814/3018423954', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:28:46.781105+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:28:54.400218+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:29:00.368036+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:28:41.908744+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:28:37.656398+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:29:10.678249+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:28:26.989477+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:29:37.516403+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.1c', 'osds': [5, 2, 7]}], 'pg_upmap_items': [{'pgid': '2.7', 'mappings': [{'from': 7, 'to': 4}]}, {'pgid': '4.6', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.d', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.1f', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.22:0/240999923': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:6800/4016369460': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/585929238': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/544031024': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:6800/2504196132': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/1451388349': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/639336081': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/2104069730': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/4236450731': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:6800/3573193776': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/2661764422': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/2197488530': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/2968034970': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/599318175': '2024-09-12T07:27:07.919998+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-11T07:29:39.824 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 4.d 2024-09-11T07:29:39.824 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd rm-pg-upmap-items 4.d 2024-09-11T07:29:39.853 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:39.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:39.855 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:39.956 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:39.983 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:39.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:39.984 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:40.084 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:40.133 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:40.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:40.134 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:40.235 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:40.262 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:40.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:40.263 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:40.364 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:40.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:40 smithi022 ceph-mon[29462]: 6.14 scrub starts 2024-09-11T07:29:40.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:40 smithi022 ceph-mon[29462]: 6.14 scrub ok 2024-09-11T07:29:40.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:40 smithi022 ceph-mon[29462]: 5.10 deep-scrub starts 2024-09-11T07:29:40.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:40 smithi022 ceph-mon[29462]: 5.10 deep-scrub ok 2024-09-11T07:29:40.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:40 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:29:40.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:40 smithi022 ceph-mon[29462]: 2.1c scrub starts 2024-09-11T07:29:40.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:40 smithi022 ceph-mon[29462]: 2.1c scrub ok 2024-09-11T07:29:40.375 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:40 smithi022 ceph-mon[37330]: 6.14 scrub starts 2024-09-11T07:29:40.375 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:40 smithi022 ceph-mon[37330]: 6.14 scrub ok 2024-09-11T07:29:40.375 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:40 smithi022 ceph-mon[37330]: 5.10 deep-scrub starts 2024-09-11T07:29:40.375 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:40 smithi022 ceph-mon[37330]: 5.10 deep-scrub ok 2024-09-11T07:29:40.376 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:40 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:29:40.376 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:40 smithi022 ceph-mon[37330]: 2.1c scrub starts 2024-09-11T07:29:40.376 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:40 smithi022 ceph-mon[37330]: 2.1c scrub ok 2024-09-11T07:29:40.390 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:40.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:40.391 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:40.492 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:40.521 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:40.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:40.522 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:40.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:40 smithi104 ceph-mon[33507]: 6.14 scrub starts 2024-09-11T07:29:40.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:40 smithi104 ceph-mon[33507]: 6.14 scrub ok 2024-09-11T07:29:40.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:40 smithi104 ceph-mon[33507]: 5.10 deep-scrub starts 2024-09-11T07:29:40.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:40 smithi104 ceph-mon[33507]: 5.10 deep-scrub ok 2024-09-11T07:29:40.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:40 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:29:40.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:40 smithi104 ceph-mon[33507]: 2.1c scrub starts 2024-09-11T07:29:40.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:40 smithi104 ceph-mon[33507]: 2.1c scrub ok 2024-09-11T07:29:40.589 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 38 15 69869 69854 114.866 78.0625 0.0162871 0.00869984 2024-09-11T07:29:40.623 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:40.648 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:40.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:40.649 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:40.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:29:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:29:40] "GET /metrics HTTP/1.1" 200 223558 "" "Prometheus/2.43.0" 2024-09-11T07:29:40.750 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:40.794 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:40.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:40.796 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:40.896 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:40.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:40.929 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:40.929 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:41.030 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:41.078 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:41.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:41.080 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:41.180 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:41.207 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:41.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:41.208 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:41.309 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:41.337 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:41.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:41.338 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:41.439 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:41.467 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:41.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:41.468 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:41.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:41 smithi104 ceph-mon[33507]: pgmap v115: 148 pgs: 148 active+clean; 4.1 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 68 MiB/s wr, 1.09k op/s 2024-09-11T07:29:41.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:41 smithi104 ceph-mon[33507]: 2.0 scrub starts 2024-09-11T07:29:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:41 smithi104 ceph-mon[33507]: 2.0 scrub ok 2024-09-11T07:29:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:41 smithi104 ceph-mon[33507]: osdmap e109: 8 total, 8 up, 8 in 2024-09-11T07:29:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:41 smithi104 ceph-mon[33507]: 6.b scrub starts 2024-09-11T07:29:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:41 smithi104 ceph-mon[33507]: 6.b scrub ok 2024-09-11T07:29:41.568 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:41.589 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 39 16 71044 71028 113.802 73.375 0.0192451 0.00878121 2024-09-11T07:29:41.593 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:41.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:41.595 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:41 smithi022 ceph-mon[29462]: pgmap v115: 148 pgs: 148 active+clean; 4.1 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 68 MiB/s wr, 1.09k op/s 2024-09-11T07:29:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:41 smithi022 ceph-mon[29462]: 2.0 scrub starts 2024-09-11T07:29:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:41 smithi022 ceph-mon[29462]: 2.0 scrub ok 2024-09-11T07:29:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:41 smithi022 ceph-mon[29462]: osdmap e109: 8 total, 8 up, 8 in 2024-09-11T07:29:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:41 smithi022 ceph-mon[29462]: 6.b scrub starts 2024-09-11T07:29:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:41 smithi022 ceph-mon[29462]: 6.b scrub ok 2024-09-11T07:29:41.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:41 smithi022 ceph-mon[37330]: pgmap v115: 148 pgs: 148 active+clean; 4.1 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 68 MiB/s wr, 1.09k op/s 2024-09-11T07:29:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:41 smithi022 ceph-mon[37330]: 2.0 scrub starts 2024-09-11T07:29:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:41 smithi022 ceph-mon[37330]: 2.0 scrub ok 2024-09-11T07:29:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:41 smithi022 ceph-mon[37330]: osdmap e109: 8 total, 8 up, 8 in 2024-09-11T07:29:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:41 smithi022 ceph-mon[37330]: 6.b scrub starts 2024-09-11T07:29:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:41 smithi022 ceph-mon[37330]: 6.b scrub ok 2024-09-11T07:29:41.695 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:41.722 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:41.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:41.723 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:41.824 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:41.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:41.868 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:41.869 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:41.969 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:42.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:42.023 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:42.024 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:42.124 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:42.146 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:42.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:42.147 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:42.248 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:42.276 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:42.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:42.277 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:42.378 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:42.414 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:42.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:42.416 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:42.516 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:42.560 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:42.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:42.561 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:42.589 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:2024-09-11T07:29:42.588428+0000 min lat: 0.00117769 max lat: 0.112583 avg lat: 0.00886353 2024-09-11T07:29:42.589 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T07:29:42.589 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 40 16 72167 72151 112.712 70.1875 0.00612993 0.00886353 2024-09-11T07:29:42.662 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:42.714 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:42.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:42.715 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:42.755 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:42 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:29:42.755 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:42 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:29:42.755 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:42 smithi022 ceph-mon[29462]: 2.12 scrub starts 2024-09-11T07:29:42.755 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:42 smithi022 ceph-mon[29462]: 2.12 scrub ok 2024-09-11T07:29:42.755 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:42 smithi022 ceph-mon[29462]: osdmap e110: 8 total, 8 up, 8 in 2024-09-11T07:29:42.755 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:42 smithi022 ceph-mon[29462]: pgmap v118: 148 pgs: 148 active+clean; 4.1 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 68 MiB/s wr, 1.09k op/s 2024-09-11T07:29:42.756 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:42 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:29:42.756 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:42 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:29:42.756 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:42 smithi022 ceph-mon[37330]: 2.12 scrub starts 2024-09-11T07:29:42.756 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:42 smithi022 ceph-mon[37330]: 2.12 scrub ok 2024-09-11T07:29:42.756 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:42 smithi022 ceph-mon[37330]: osdmap e110: 8 total, 8 up, 8 in 2024-09-11T07:29:42.756 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:42 smithi022 ceph-mon[37330]: pgmap v118: 148 pgs: 148 active+clean; 4.1 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 68 MiB/s wr, 1.09k op/s 2024-09-11T07:29:42.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:42 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:29:42.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:42 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:29:42.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:42 smithi104 ceph-mon[33507]: 2.12 scrub starts 2024-09-11T07:29:42.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:42 smithi104 ceph-mon[33507]: 2.12 scrub ok 2024-09-11T07:29:42.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:42 smithi104 ceph-mon[33507]: osdmap e110: 8 total, 8 up, 8 in 2024-09-11T07:29:42.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:42 smithi104 ceph-mon[33507]: pgmap v118: 148 pgs: 148 active+clean; 4.1 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 68 MiB/s wr, 1.09k op/s 2024-09-11T07:29:42.815 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:42.838 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:42.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:42.839 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:42.940 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:42.969 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:42.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:42.970 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:43.071 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:43.096 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:43.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:43.097 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:43.198 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:43.236 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:43.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:43.237 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:43.338 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:43.359 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:43.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:43.360 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:43.460 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:43.480 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:29:43.481 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:29:43.481 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:29:43.481 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:29:43.481 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:29:43.481 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:29:43.481 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:29:43.481 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:29:43.481 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:29:43.481 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:29:43.481 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:29:43.482 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:29:43.482 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:29:43.482 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:29:43.482 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:29:43.482 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:29:43.502 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:43.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:43.503 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:43.589 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 41 16 73233 73217 111.587 66.625 0.00606867 0.0089514 2024-09-11T07:29:43.604 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:43.626 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:43.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:43.627 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:43.728 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:43.757 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:43.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:43.758 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:43.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:43 smithi104 ceph-mon[33507]: 6.8 scrub starts 2024-09-11T07:29:43.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:43 smithi104 ceph-mon[33507]: 6.1e scrub starts 2024-09-11T07:29:43.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:43 smithi104 ceph-mon[33507]: 6.1e scrub ok 2024-09-11T07:29:43.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:43 smithi104 ceph-mon[33507]: 6.8 scrub ok 2024-09-11T07:29:43.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:43 smithi104 ceph-mon[33507]: 2.17 scrub starts 2024-09-11T07:29:43.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:43 smithi104 ceph-mon[33507]: 2.17 scrub ok 2024-09-11T07:29:43.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:43 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2802453172' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:29:43.859 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:43.893 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:43.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:43.894 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:43.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:43 smithi022 ceph-mon[29462]: 6.8 scrub starts 2024-09-11T07:29:43.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:43 smithi022 ceph-mon[29462]: 6.1e scrub starts 2024-09-11T07:29:43.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:43 smithi022 ceph-mon[29462]: 6.1e scrub ok 2024-09-11T07:29:43.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:43 smithi022 ceph-mon[29462]: 6.8 scrub ok 2024-09-11T07:29:43.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:43 smithi022 ceph-mon[29462]: 2.17 scrub starts 2024-09-11T07:29:43.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:43 smithi022 ceph-mon[29462]: 2.17 scrub ok 2024-09-11T07:29:43.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:43 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2802453172' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:29:43.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:43 smithi022 ceph-mon[37330]: 6.8 scrub starts 2024-09-11T07:29:43.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:43 smithi022 ceph-mon[37330]: 6.1e scrub starts 2024-09-11T07:29:43.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:43 smithi022 ceph-mon[37330]: 6.1e scrub ok 2024-09-11T07:29:43.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:43 smithi022 ceph-mon[37330]: 6.8 scrub ok 2024-09-11T07:29:43.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:43 smithi022 ceph-mon[37330]: 2.17 scrub starts 2024-09-11T07:29:43.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:43 smithi022 ceph-mon[37330]: 2.17 scrub ok 2024-09-11T07:29:43.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:43 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2802453172' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:29:43.995 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:44.025 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:44.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:44.026 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:44.127 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:44.155 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:44.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:44.156 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:44.257 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:44.319 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:44.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:44.320 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:44.421 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:44.447 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:44.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:44.448 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:44.476 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:29:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:29:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:29:44.549 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:44.589 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 42 16 74347 74331 110.588 69.625 0.0114298 0.00903673 2024-09-11T07:29:44.590 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:44.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:44.590 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:44.691 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:44.728 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:44.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:44.729 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:44.753 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:44 smithi022 ceph-mon[37330]: 6.4 scrub starts 2024-09-11T07:29:44.753 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:44 smithi022 ceph-mon[37330]: 6.4 scrub ok 2024-09-11T07:29:44.753 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:44 smithi022 ceph-mon[37330]: 6.0 scrub starts 2024-09-11T07:29:44.753 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:44 smithi022 ceph-mon[37330]: 6.0 scrub ok 2024-09-11T07:29:44.753 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:44 smithi022 ceph-mon[37330]: pgmap v119: 148 pgs: 148 active+clean; 4.3 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.7 KiB/s rd, 94 MiB/s wr, 1.50k op/s 2024-09-11T07:29:44.754 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:44 smithi022 ceph-mon[29462]: 6.4 scrub starts 2024-09-11T07:29:44.755 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:44 smithi022 ceph-mon[29462]: 6.4 scrub ok 2024-09-11T07:29:44.755 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:44 smithi022 ceph-mon[29462]: 6.0 scrub starts 2024-09-11T07:29:44.755 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:44 smithi022 ceph-mon[29462]: 6.0 scrub ok 2024-09-11T07:29:44.755 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:44 smithi022 ceph-mon[29462]: pgmap v119: 148 pgs: 148 active+clean; 4.3 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.7 KiB/s rd, 94 MiB/s wr, 1.50k op/s 2024-09-11T07:29:44.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:44 smithi104 ceph-mon[33507]: 6.4 scrub starts 2024-09-11T07:29:44.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:44 smithi104 ceph-mon[33507]: 6.4 scrub ok 2024-09-11T07:29:44.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:44 smithi104 ceph-mon[33507]: 6.0 scrub starts 2024-09-11T07:29:44.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:44 smithi104 ceph-mon[33507]: 6.0 scrub ok 2024-09-11T07:29:44.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:44 smithi104 ceph-mon[33507]: pgmap v119: 148 pgs: 148 active+clean; 4.3 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.7 KiB/s rd, 94 MiB/s wr, 1.50k op/s 2024-09-11T07:29:44.830 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:44.858 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:44.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:44.859 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:44.960 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:45.037 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:29:45.040 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:45.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:45.041 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:45.048 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:29:45.142 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:45.196 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:45.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:45.197 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:45.298 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:45.326 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:45.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:45.327 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:45.428 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:45.456 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:45.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:45.457 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:45.558 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:45.589 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 43 16 75521 75505 109.723 73.375 0.00677719 0.0091082 2024-09-11T07:29:45.599 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:45.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:45.600 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:45.701 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:45.726 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:45.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:45.727 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:45.780 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:45 smithi022 ceph-mon[29462]: 5.8 deep-scrub starts 2024-09-11T07:29:45.780 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:45 smithi022 ceph-mon[29462]: 5.8 deep-scrub ok 2024-09-11T07:29:45.780 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:45 smithi022 ceph-mon[29462]: osdmap e111: 8 total, 8 up, 8 in 2024-09-11T07:29:45.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:45 smithi104 ceph-mon[33507]: 5.8 deep-scrub starts 2024-09-11T07:29:45.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:45 smithi104 ceph-mon[33507]: 5.8 deep-scrub ok 2024-09-11T07:29:45.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:45 smithi104 ceph-mon[33507]: osdmap e111: 8 total, 8 up, 8 in 2024-09-11T07:29:45.781 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:45 smithi022 ceph-mon[37330]: 5.8 deep-scrub starts 2024-09-11T07:29:45.781 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:45 smithi022 ceph-mon[37330]: 5.8 deep-scrub ok 2024-09-11T07:29:45.781 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:45 smithi022 ceph-mon[37330]: osdmap e111: 8 total, 8 up, 8 in 2024-09-11T07:29:45.828 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:45.849 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:29:45.849 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:29:45.879 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:45.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:45.881 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:45.981 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:46.008 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:46.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:46.009 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:46.110 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:46.153 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:46.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:46.154 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:46.255 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:46.298 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:46.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:46.299 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:46.400 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:46.439 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:46.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:46.440 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:46.541 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:46.567 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:46.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:46.568 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:46.589 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 44 16 76723 76707 108.936 75.125 0.0207061 0.00917429 2024-09-11T07:29:46.669 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:46.707 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:46.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:46.708 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:46.755 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:29:46.768 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:46 smithi022 ceph-mon[29462]: 5.18 scrub starts 2024-09-11T07:29:46.768 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:46 smithi022 ceph-mon[29462]: 5.18 scrub ok 2024-09-11T07:29:46.768 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:46 smithi022 ceph-mon[29462]: 2.d scrub starts 2024-09-11T07:29:46.769 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:46 smithi022 ceph-mon[29462]: 2.2 scrub starts 2024-09-11T07:29:46.769 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:46 smithi022 ceph-mon[29462]: 2.2 scrub ok 2024-09-11T07:29:46.769 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:46 smithi022 ceph-mon[29462]: 2.d scrub ok 2024-09-11T07:29:46.769 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:46 smithi022 ceph-mon[29462]: 5.a scrub starts 2024-09-11T07:29:46.769 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:46 smithi022 ceph-mon[29462]: 5.a scrub ok 2024-09-11T07:29:46.769 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:46 smithi022 ceph-mon[29462]: pgmap v121: 148 pgs: 148 active+clean; 4.5 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 60 MiB/s wr, 959 op/s 2024-09-11T07:29:46.769 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:46 smithi022 ceph-mon[37330]: 5.18 scrub starts 2024-09-11T07:29:46.769 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:46 smithi022 ceph-mon[37330]: 5.18 scrub ok 2024-09-11T07:29:46.769 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:46 smithi022 ceph-mon[37330]: 2.d scrub starts 2024-09-11T07:29:46.769 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:46 smithi022 ceph-mon[37330]: 2.2 scrub starts 2024-09-11T07:29:46.770 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:46 smithi022 ceph-mon[37330]: 2.2 scrub ok 2024-09-11T07:29:46.770 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:46 smithi022 ceph-mon[37330]: 2.d scrub ok 2024-09-11T07:29:46.770 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:46 smithi022 ceph-mon[37330]: 5.a scrub starts 2024-09-11T07:29:46.770 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:46 smithi022 ceph-mon[37330]: 5.a scrub ok 2024-09-11T07:29:46.770 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:46 smithi022 ceph-mon[37330]: pgmap v121: 148 pgs: 148 active+clean; 4.5 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 60 MiB/s wr, 959 op/s 2024-09-11T07:29:46.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:46 smithi104 ceph-mon[33507]: 5.18 scrub starts 2024-09-11T07:29:46.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:46 smithi104 ceph-mon[33507]: 5.18 scrub ok 2024-09-11T07:29:46.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:46 smithi104 ceph-mon[33507]: 2.d scrub starts 2024-09-11T07:29:46.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:46 smithi104 ceph-mon[33507]: 2.2 scrub starts 2024-09-11T07:29:46.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:46 smithi104 ceph-mon[33507]: 2.2 scrub ok 2024-09-11T07:29:46.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:46 smithi104 ceph-mon[33507]: 2.d scrub ok 2024-09-11T07:29:46.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:46 smithi104 ceph-mon[33507]: 5.a scrub starts 2024-09-11T07:29:46.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:46 smithi104 ceph-mon[33507]: 5.a scrub ok 2024-09-11T07:29:46.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:46 smithi104 ceph-mon[33507]: pgmap v121: 148 pgs: 148 active+clean; 4.5 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 60 MiB/s wr, 959 op/s 2024-09-11T07:29:46.810 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:46.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:46.862 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:46.862 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:46.963 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:46.987 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:46.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:46.989 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:47.089 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:47.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:47.116 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:47.116 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:47.217 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:47.267 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:47.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:47.268 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:47.369 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:47.423 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-11T07:29:47.425 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:47.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:47.427 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:47.528 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:47.554 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:47.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:47.555 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:47.589 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 45 16 77858 77842 108.092 70.9375 0.0209853 0.00924599 2024-09-11T07:29:47.656 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:47.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:47.705 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:47.706 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:47.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:47 smithi104 ceph-mon[33507]: 6.3 scrub starts 2024-09-11T07:29:47.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:47 smithi104 ceph-mon[33507]: 6.3 scrub ok 2024-09-11T07:29:47.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:47 smithi104 ceph-mon[33507]: 4.a scrub starts 2024-09-11T07:29:47.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:47 smithi104 ceph-mon[33507]: 4.a scrub ok 2024-09-11T07:29:47.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:47 smithi104 ceph-mon[33507]: 6.6 scrub starts 2024-09-11T07:29:47.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:47 smithi104 ceph-mon[33507]: 6.6 scrub ok 2024-09-11T07:29:47.808 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:47.838 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:47.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:47.839 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:47.849 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:47 smithi022 ceph-mon[29462]: 6.3 scrub starts 2024-09-11T07:29:47.849 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:47 smithi022 ceph-mon[29462]: 6.3 scrub ok 2024-09-11T07:29:47.849 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:47 smithi022 ceph-mon[29462]: 4.a scrub starts 2024-09-11T07:29:47.849 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:47 smithi022 ceph-mon[29462]: 4.a scrub ok 2024-09-11T07:29:47.849 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:47 smithi022 ceph-mon[29462]: 6.6 scrub starts 2024-09-11T07:29:47.849 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:47 smithi022 ceph-mon[29462]: 6.6 scrub ok 2024-09-11T07:29:47.849 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:47 smithi022 ceph-mon[37330]: 6.3 scrub starts 2024-09-11T07:29:47.850 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:47 smithi022 ceph-mon[37330]: 6.3 scrub ok 2024-09-11T07:29:47.850 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:47 smithi022 ceph-mon[37330]: 4.a scrub starts 2024-09-11T07:29:47.850 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:47 smithi022 ceph-mon[37330]: 4.a scrub ok 2024-09-11T07:29:47.850 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:47 smithi022 ceph-mon[37330]: 6.6 scrub starts 2024-09-11T07:29:47.850 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:47 smithi022 ceph-mon[37330]: 6.6 scrub ok 2024-09-11T07:29:47.939 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:47.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:47.998 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:47.998 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:48.099 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:48.125 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:48.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:48.126 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:48.227 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:48.252 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:48.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:48.253 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:48.354 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:48.402 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:48.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:48.403 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:48.503 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:48.530 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:48.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:48.531 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:48.590 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 46 16 79032 79016 107.337 73.375 0.018656 0.0093115 2024-09-11T07:29:48.632 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:48.661 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:48.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:48.661 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:48.762 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:48.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:48 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:29:48.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:48 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:29:48.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:48 smithi104 ceph-mon[33507]: 1.0 scrub starts 2024-09-11T07:29:48.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:48 smithi104 ceph-mon[33507]: 1.0 scrub ok 2024-09-11T07:29:48.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:48 smithi104 ceph-mon[33507]: 2.18 scrub starts 2024-09-11T07:29:48.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:48 smithi104 ceph-mon[33507]: 2.18 scrub ok 2024-09-11T07:29:48.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:48 smithi104 ceph-mon[33507]: 4.2 scrub starts 2024-09-11T07:29:48.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:48 smithi104 ceph-mon[33507]: 4.2 scrub ok 2024-09-11T07:29:48.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:48 smithi104 ceph-mon[33507]: 4.16 scrub starts 2024-09-11T07:29:48.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:48 smithi104 ceph-mon[33507]: osdmap e112: 8 total, 8 up, 8 in 2024-09-11T07:29:48.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:48 smithi104 ceph-mon[33507]: 4.16 scrub ok 2024-09-11T07:29:48.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:48 smithi104 ceph-mon[33507]: pgmap v123: 148 pgs: 148 active+clean; 4.5 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 785 B/s rd, 55 MiB/s wr, 883 op/s 2024-09-11T07:29:48.790 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:48.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:48.791 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:48.891 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:48.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:48 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:29:48.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:48 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:29:48.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:48 smithi022 ceph-mon[29462]: 1.0 scrub starts 2024-09-11T07:29:48.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:48 smithi022 ceph-mon[29462]: 1.0 scrub ok 2024-09-11T07:29:48.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:48 smithi022 ceph-mon[29462]: 2.18 scrub starts 2024-09-11T07:29:48.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:48 smithi022 ceph-mon[29462]: 2.18 scrub ok 2024-09-11T07:29:48.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:48 smithi022 ceph-mon[29462]: 4.2 scrub starts 2024-09-11T07:29:48.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:48 smithi022 ceph-mon[29462]: 4.2 scrub ok 2024-09-11T07:29:48.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:48 smithi022 ceph-mon[29462]: 4.16 scrub starts 2024-09-11T07:29:48.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:48 smithi022 ceph-mon[29462]: osdmap e112: 8 total, 8 up, 8 in 2024-09-11T07:29:48.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:48 smithi022 ceph-mon[29462]: 4.16 scrub ok 2024-09-11T07:29:48.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:48 smithi022 ceph-mon[29462]: pgmap v123: 148 pgs: 148 active+clean; 4.5 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 785 B/s rd, 55 MiB/s wr, 883 op/s 2024-09-11T07:29:48.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:48 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:29:48.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:48 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:29:48.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:48 smithi022 ceph-mon[37330]: 1.0 scrub starts 2024-09-11T07:29:48.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:48 smithi022 ceph-mon[37330]: 1.0 scrub ok 2024-09-11T07:29:48.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:48 smithi022 ceph-mon[37330]: 2.18 scrub starts 2024-09-11T07:29:48.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:48 smithi022 ceph-mon[37330]: 2.18 scrub ok 2024-09-11T07:29:48.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:48 smithi022 ceph-mon[37330]: 4.2 scrub starts 2024-09-11T07:29:48.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:48 smithi022 ceph-mon[37330]: 4.2 scrub ok 2024-09-11T07:29:48.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:48 smithi022 ceph-mon[37330]: 4.16 scrub starts 2024-09-11T07:29:48.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:48 smithi022 ceph-mon[37330]: osdmap e112: 8 total, 8 up, 8 in 2024-09-11T07:29:48.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:48 smithi022 ceph-mon[37330]: 4.16 scrub ok 2024-09-11T07:29:48.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:48 smithi022 ceph-mon[37330]: pgmap v123: 148 pgs: 148 active+clean; 4.5 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 785 B/s rd, 55 MiB/s wr, 883 op/s 2024-09-11T07:29:48.919 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:48.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:48.920 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:49.021 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:49.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:49.084 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:49.084 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:49.185 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:49.211 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:49.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:49.212 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:49.313 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:49.336 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:49.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:49.337 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:49.437 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:49.479 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:49.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:49.480 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:49.521 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:29:49 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:29:49.581 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:49.590 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 47 16 80137 80121 106.522 69.0625 0.0214256 0.00938279 2024-09-11T07:29:49.627 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:49.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:49.629 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:49.730 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:49.771 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:49.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:49 smithi104 ceph-mon[33507]: 5.e scrub starts 2024-09-11T07:29:49.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:49 smithi104 ceph-mon[33507]: 5.e scrub ok 2024-09-11T07:29:49.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:49 smithi104 ceph-mon[33507]: 5.1a scrub starts 2024-09-11T07:29:49.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:49 smithi104 ceph-mon[33507]: 5.1a scrub ok 2024-09-11T07:29:49.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:49 smithi104 ceph-mon[33507]: 2.10 deep-scrub starts 2024-09-11T07:29:49.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:49 smithi104 ceph-mon[33507]: 2.10 deep-scrub ok 2024-09-11T07:29:49.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:49 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:29:49.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:49.781 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:49.833 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:29:49.882 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:49.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:49 smithi022 ceph-mon[37330]: 5.e scrub starts 2024-09-11T07:29:49.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:49 smithi022 ceph-mon[37330]: 5.e scrub ok 2024-09-11T07:29:49.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:49 smithi022 ceph-mon[37330]: 5.1a scrub starts 2024-09-11T07:29:49.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:49 smithi022 ceph-mon[37330]: 5.1a scrub ok 2024-09-11T07:29:49.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:49 smithi022 ceph-mon[37330]: 2.10 deep-scrub starts 2024-09-11T07:29:49.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:49 smithi022 ceph-mon[37330]: 2.10 deep-scrub ok 2024-09-11T07:29:49.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:49 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:29:49.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:49 smithi022 ceph-mon[29462]: 5.e scrub starts 2024-09-11T07:29:49.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:49 smithi022 ceph-mon[29462]: 5.e scrub ok 2024-09-11T07:29:49.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:49 smithi022 ceph-mon[29462]: 5.1a scrub starts 2024-09-11T07:29:49.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:49 smithi022 ceph-mon[29462]: 5.1a scrub ok 2024-09-11T07:29:49.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:49 smithi022 ceph-mon[29462]: 2.10 deep-scrub starts 2024-09-11T07:29:49.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:49 smithi022 ceph-mon[29462]: 2.10 deep-scrub ok 2024-09-11T07:29:49.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:49 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:29:49.943 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:49.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:49.944 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:50.044 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:50.070 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:50.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:50.071 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:50.172 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:50.198 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:50.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:50.199 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:50.300 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:50.349 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:50.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:50.350 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:50.450 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:50.464 INFO:teuthology.orchestra.run.smithi022.stderr:clear 4.d pg_upmap_items mapping 2024-09-11T07:29:50.466 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:29:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:29:50] "GET /metrics HTTP/1.1" 200 223558 "" "Prometheus/2.43.0" 2024-09-11T07:29:50.478 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:50.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:50.479 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:50.580 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:50.590 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 48 16 81175 81159 105.654 64.875 0.0154873 0.0094598 2024-09-11T07:29:50.608 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:50.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:50.609 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:50.709 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:50.737 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:50.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:50.738 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:50.795 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:50 smithi022 ceph-mon[29462]: 4.c deep-scrub starts 2024-09-11T07:29:50.795 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:50 smithi022 ceph-mon[29462]: 4.c deep-scrub ok 2024-09-11T07:29:50.796 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:50 smithi022 ceph-mon[29462]: 4.17 scrub starts 2024-09-11T07:29:50.796 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:50 smithi022 ceph-mon[29462]: 4.17 scrub ok 2024-09-11T07:29:50.796 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:50 smithi022 ceph-mon[29462]: 2.1 scrub starts 2024-09-11T07:29:50.796 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:50 smithi022 ceph-mon[29462]: 2.1 scrub ok 2024-09-11T07:29:50.796 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:50 smithi022 ceph-mon[29462]: pgmap v124: 148 pgs: 148 active+clean; 4.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 91 MiB/s wr, 1.46k op/s 2024-09-11T07:29:50.796 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:50 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3314592347' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.d"}]: dispatch 2024-09-11T07:29:50.796 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:50 smithi022 ceph-mon[29462]: from='client.24809 ' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.d"}]: dispatch 2024-09-11T07:29:50.796 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:50 smithi022 ceph-mon[29462]: from='client.24809 ' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.d"}]': finished 2024-09-11T07:29:50.796 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:50 smithi022 ceph-mon[29462]: osdmap e113: 8 total, 8 up, 8 in 2024-09-11T07:29:50.796 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:50 smithi022 ceph-mon[37330]: 4.c deep-scrub starts 2024-09-11T07:29:50.796 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:50 smithi022 ceph-mon[37330]: 4.c deep-scrub ok 2024-09-11T07:29:50.797 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:50 smithi022 ceph-mon[37330]: 4.17 scrub starts 2024-09-11T07:29:50.797 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:50 smithi022 ceph-mon[37330]: 4.17 scrub ok 2024-09-11T07:29:50.797 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:50 smithi022 ceph-mon[37330]: 2.1 scrub starts 2024-09-11T07:29:50.797 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:50 smithi022 ceph-mon[37330]: 2.1 scrub ok 2024-09-11T07:29:50.797 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:50 smithi022 ceph-mon[37330]: pgmap v124: 148 pgs: 148 active+clean; 4.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 91 MiB/s wr, 1.46k op/s 2024-09-11T07:29:50.797 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:50 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3314592347' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.d"}]: dispatch 2024-09-11T07:29:50.797 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:50 smithi022 ceph-mon[37330]: from='client.24809 ' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.d"}]: dispatch 2024-09-11T07:29:50.797 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:50 smithi022 ceph-mon[37330]: from='client.24809 ' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.d"}]': finished 2024-09-11T07:29:50.797 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:50 smithi022 ceph-mon[37330]: osdmap e113: 8 total, 8 up, 8 in 2024-09-11T07:29:50.839 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:50.882 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:50.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:50.883 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:50.984 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:51.013 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:51.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:51.014 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:50 smithi104 ceph-mon[33507]: 4.c deep-scrub starts 2024-09-11T07:29:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:50 smithi104 ceph-mon[33507]: 4.c deep-scrub ok 2024-09-11T07:29:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:50 smithi104 ceph-mon[33507]: 4.17 scrub starts 2024-09-11T07:29:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:50 smithi104 ceph-mon[33507]: 4.17 scrub ok 2024-09-11T07:29:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:50 smithi104 ceph-mon[33507]: 2.1 scrub starts 2024-09-11T07:29:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:50 smithi104 ceph-mon[33507]: 2.1 scrub ok 2024-09-11T07:29:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:50 smithi104 ceph-mon[33507]: pgmap v124: 148 pgs: 148 active+clean; 4.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 91 MiB/s wr, 1.46k op/s 2024-09-11T07:29:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:50 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3314592347' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.d"}]: dispatch 2024-09-11T07:29:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:50 smithi104 ceph-mon[33507]: from='client.24809 ' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.d"}]: dispatch 2024-09-11T07:29:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:50 smithi104 ceph-mon[33507]: from='client.24809 ' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.d"}]': finished 2024-09-11T07:29:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:50 smithi104 ceph-mon[33507]: osdmap e113: 8 total, 8 up, 8 in 2024-09-11T07:29:51.115 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:51.154 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:51.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:51.155 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:51.256 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:51.281 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:51.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:51.282 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:51.383 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:51.423 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:51.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:51.424 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:51.472 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is set 2024-09-11T07:29:51.525 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:51.552 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:51.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:51.553 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:51.590 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 49 16 82309 82293 104.945 70.875 0.0116021 0.00952403 2024-09-11T07:29:51.653 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:51.689 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:51.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:51.691 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:51.791 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:51.832 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:51.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:51.833 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:51.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:51 smithi022 ceph-mon[29462]: 4.8 scrub starts 2024-09-11T07:29:51.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:51 smithi022 ceph-mon[29462]: 4.8 scrub ok 2024-09-11T07:29:51.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:51 smithi022 ceph-mon[29462]: 6.1d scrub starts 2024-09-11T07:29:51.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:51 smithi022 ceph-mon[29462]: 6.1d scrub ok 2024-09-11T07:29:51.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:51 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2198991776' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:29:51.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:51 smithi022 ceph-mon[29462]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:29:51.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:51 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2198991776' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:29:51.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:51 smithi022 ceph-mon[29462]: osdmap e114: 8 total, 8 up, 8 in 2024-09-11T07:29:51.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:51 smithi022 ceph-mon[37330]: 4.8 scrub starts 2024-09-11T07:29:51.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:51 smithi022 ceph-mon[37330]: 4.8 scrub ok 2024-09-11T07:29:51.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:51 smithi022 ceph-mon[37330]: 6.1d scrub starts 2024-09-11T07:29:51.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:51 smithi022 ceph-mon[37330]: 6.1d scrub ok 2024-09-11T07:29:51.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:51 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2198991776' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:29:51.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:51 smithi022 ceph-mon[37330]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:29:51.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:51 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2198991776' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:29:51.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:51 smithi022 ceph-mon[37330]: osdmap e114: 8 total, 8 up, 8 in 2024-09-11T07:29:51.934 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:51.959 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:51.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:51.960 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:52.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:51 smithi104 ceph-mon[33507]: 4.8 scrub starts 2024-09-11T07:29:52.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:51 smithi104 ceph-mon[33507]: 4.8 scrub ok 2024-09-11T07:29:52.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:51 smithi104 ceph-mon[33507]: 6.1d scrub starts 2024-09-11T07:29:52.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:51 smithi104 ceph-mon[33507]: 6.1d scrub ok 2024-09-11T07:29:52.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:51 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2198991776' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:29:52.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:51 smithi104 ceph-mon[33507]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:29:52.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:51 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2198991776' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:29:52.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:51 smithi104 ceph-mon[33507]: osdmap e114: 8 total, 8 up, 8 in 2024-09-11T07:29:52.061 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:52.100 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:52.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:52.101 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:52.202 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:52.242 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:52.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:52.243 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:52.344 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:52.370 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:52.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:52.371 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:52.472 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:52.513 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:52.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:52.514 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:52.590 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 50 16 83490 83474 104.322 73.8125 0.0109174 0.00957997 2024-09-11T07:29:52.614 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:52.642 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:52.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:52.643 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:52.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:52 smithi022 ceph-mon[29462]: 4.9 deep-scrub starts 2024-09-11T07:29:52.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:52 smithi022 ceph-mon[29462]: 4.9 deep-scrub ok 2024-09-11T07:29:52.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:52 smithi022 ceph-mon[29462]: 6.1c scrub starts 2024-09-11T07:29:52.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:52 smithi022 ceph-mon[29462]: 6.1c scrub ok 2024-09-11T07:29:52.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:52 smithi022 ceph-mon[29462]: 2.16 scrub starts 2024-09-11T07:29:52.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:52 smithi022 ceph-mon[29462]: 2.16 scrub ok 2024-09-11T07:29:52.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:52 smithi022 ceph-mon[29462]: 5.1f scrub starts 2024-09-11T07:29:52.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:52 smithi022 ceph-mon[29462]: 5.1f scrub ok 2024-09-11T07:29:52.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:52 smithi022 ceph-mon[29462]: pgmap v127: 148 pgs: 148 active+clean; 4.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 61 MiB/s wr, 984 op/s 2024-09-11T07:29:52.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:52 smithi022 ceph-mon[37330]: 4.9 deep-scrub starts 2024-09-11T07:29:52.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:52 smithi022 ceph-mon[37330]: 4.9 deep-scrub ok 2024-09-11T07:29:52.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:52 smithi022 ceph-mon[37330]: 6.1c scrub starts 2024-09-11T07:29:52.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:52 smithi022 ceph-mon[37330]: 6.1c scrub ok 2024-09-11T07:29:52.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:52 smithi022 ceph-mon[37330]: 2.16 scrub starts 2024-09-11T07:29:52.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:52 smithi022 ceph-mon[37330]: 2.16 scrub ok 2024-09-11T07:29:52.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:52 smithi022 ceph-mon[37330]: 5.1f scrub starts 2024-09-11T07:29:52.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:52 smithi022 ceph-mon[37330]: 5.1f scrub ok 2024-09-11T07:29:52.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:52 smithi022 ceph-mon[37330]: pgmap v127: 148 pgs: 148 active+clean; 4.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 61 MiB/s wr, 984 op/s 2024-09-11T07:29:52.743 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:52.767 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:52.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:52.768 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:52.869 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:52.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:52.911 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:52.931 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:52 smithi104 ceph-mon[33507]: 4.9 deep-scrub starts 2024-09-11T07:29:52.931 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:52 smithi104 ceph-mon[33507]: 4.9 deep-scrub ok 2024-09-11T07:29:52.931 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:52 smithi104 ceph-mon[33507]: 6.1c scrub starts 2024-09-11T07:29:52.931 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:52 smithi104 ceph-mon[33507]: 6.1c scrub ok 2024-09-11T07:29:52.931 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:52 smithi104 ceph-mon[33507]: 2.16 scrub starts 2024-09-11T07:29:52.931 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:52 smithi104 ceph-mon[33507]: 2.16 scrub ok 2024-09-11T07:29:52.931 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:52 smithi104 ceph-mon[33507]: 5.1f scrub starts 2024-09-11T07:29:52.931 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:52 smithi104 ceph-mon[33507]: 5.1f scrub ok 2024-09-11T07:29:52.931 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:52 smithi104 ceph-mon[33507]: pgmap v127: 148 pgs: 148 active+clean; 4.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 61 MiB/s wr, 984 op/s 2024-09-11T07:29:53.008 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:29:53.012 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:53.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:53.036 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:53.036 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:53.137 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:53.164 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:53.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:53.165 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:53.265 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:53.306 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:53.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:53.307 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:53.408 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:53.447 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:53.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:53.448 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:53.549 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:53.590 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 51 16 84634 84618 103.678 71.5 0.0119199 0.00964037 2024-09-11T07:29:53.592 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:53.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:53.593 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:53.694 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:53.735 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:53.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:53.736 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:53.837 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:53.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:53.876 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:53.876 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:53.977 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:54.007 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:54.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:54.008 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:54.108 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:54.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:54.151 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:54.151 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:54.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:54 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:29:54.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:54 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:29:54.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:54 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:29:54.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:54 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:29:54.252 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:54.278 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:54.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:54.279 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:54.280 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:29:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:29:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:29:54.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:54 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:29:54.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:54 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:29:54.381 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:54.407 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:54.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:54.408 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:54.509 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:54.531 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:54.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:54.532 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:54.590 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 52 16 85670 85654 102.93 64.75 0.0184441 0.00970985 2024-09-11T07:29:54.633 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:29:54.673 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:54.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:54.674 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:29:54.775 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:54.804 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:54.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:54.805 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:54.905 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:54.942 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:54.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:54.943 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:55.044 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:55.070 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:55.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:55.071 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:55.172 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:55.198 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:55.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:55.199 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:55.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:55 smithi104 ceph-mon[33507]: pgmap v128: 148 pgs: 148 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.5 KiB/s rd, 80 MiB/s wr, 1.28k op/s; 16 B/s, 1 objects/s recovering 2024-09-11T07:29:55.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:55 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/79591215' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:29:55.300 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:55.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:55.345 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:55.345 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:55.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:55 smithi022 ceph-mon[37330]: pgmap v128: 148 pgs: 148 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.5 KiB/s rd, 80 MiB/s wr, 1.28k op/s; 16 B/s, 1 objects/s recovering 2024-09-11T07:29:55.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:55 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/79591215' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:29:55.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:55 smithi022 ceph-mon[29462]: pgmap v128: 148 pgs: 148 active+clean; 5.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.5 KiB/s rd, 80 MiB/s wr, 1.28k op/s; 16 B/s, 1 objects/s recovering 2024-09-11T07:29:55.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:55 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/79591215' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:29:55.435 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:29:55.435 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:29:55.435 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:29:55.435 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:29:55.435 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:29:55.435 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:29:55.435 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:29:55.435 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:29:55.435 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:29:55.435 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:29:55.436 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:29:55.436 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:29:55.436 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:29:55.436 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:29:55.436 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:29:55.436 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:29:55.446 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:55.473 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:55.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:55.474 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:55.510 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T07:29:55.575 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:55.590 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 53 16 86846 86830 102.374 73.5 0.0101704 0.00976255 2024-09-11T07:29:55.602 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:55.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:55.603 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:55.704 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:55.765 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:55.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:55.766 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:55.866 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:55.913 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:55.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:55.914 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:56.014 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:56.042 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:56.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:56.043 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:56.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:56.201 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:56.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:56.202 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:56.252 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:29:56.253 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:29:56.253 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-11T07:29:56.253 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T07:29:56.253 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T07:29:56.253 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.5 config set filestore_inject_stall 3 2024-09-11T07:29:56.302 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:56.335 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:56.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:56.336 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:56.437 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:56.466 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:56.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:56.467 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:56.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:56 smithi104 ceph-mon[33507]: osdmap e115: 8 total, 8 up, 8 in 2024-09-11T07:29:56.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:56 smithi104 ceph-mon[33507]: pgmap v130: 148 pgs: 148 active+clean; 5.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 58 MiB/s wr, 927 op/s; 18 B/s, 1 objects/s recovering 2024-09-11T07:29:56.568 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:56.591 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 54 16 87958 87942 101.765 69.5 0.00855247 0.00982041 2024-09-11T07:29:56.591 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:29:56.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:56.611 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:56.612 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:56.631 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:29:56.713 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:56.739 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:56.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:56.740 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:56.769 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:56 smithi022 ceph-mon[37330]: osdmap e115: 8 total, 8 up, 8 in 2024-09-11T07:29:56.769 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:56 smithi022 ceph-mon[37330]: pgmap v130: 148 pgs: 148 active+clean; 5.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 58 MiB/s wr, 927 op/s; 18 B/s, 1 objects/s recovering 2024-09-11T07:29:56.769 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:56 smithi022 ceph-mon[29462]: osdmap e115: 8 total, 8 up, 8 in 2024-09-11T07:29:56.769 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:56 smithi022 ceph-mon[29462]: pgmap v130: 148 pgs: 148 active+clean; 5.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 58 MiB/s wr, 927 op/s; 18 B/s, 1 objects/s recovering 2024-09-11T07:29:56.841 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:56.884 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:56.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:56.885 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:56.986 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:57.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:57.043 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:57.043 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:57.144 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:57.173 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:57.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:57.173 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:57.274 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:57.320 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:57.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:57.321 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:57.422 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:57.459 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:57.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:57.460 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:57.561 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:57.589 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:57.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:57.590 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:57.591 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 55 16 89076 89060 101.185 69.875 0.0123678 0.00987727 2024-09-11T07:29:57.690 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:57.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:57.727 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:57.727 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:57.828 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:57.869 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:57.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:57.870 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:57.971 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:58.010 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:58.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:58.011 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:58.099 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:29:58.111 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:29:58.137 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:58.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:58.138 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:29:58.239 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:58.290 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:58.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:58.292 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:58.392 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:58.421 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:29:58.421 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:29:58.421 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:89908 7.8 7:1d704200:::benchmark_data_smithi022_77720_object89907:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-11T07:29:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:58.413976+0000", 2024-09-11T07:29:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0048500899999999996, 2024-09-11T07:29:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.004904009, 2024-09-11T07:29:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:29:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "waiting for sub ops", 2024-09-11T07:29:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:29:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:29:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:29:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 89908 2024-09-11T07:29:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:29:58.422 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:29:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.413976+0000", 2024-09-11T07:29:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:29:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.413976+0000", 2024-09-11T07:29:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:29:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.413973+0000", 2024-09-11T07:29:58.423 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:29:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:29:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.414001+0000", 2024-09-11T07:29:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8047000000000001e-05 2024-09-11T07:29:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:29:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.414021+0000", 2024-09-11T07:29:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0210000000000001e-05 2024-09-11T07:29:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:29:58.424 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.414028+0000", 2024-09-11T07:29:58.425 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.6100000000000002e-06 2024-09-11T07:29:58.425 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.425 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.425 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:29:58.425 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.414176+0000", 2024-09-11T07:29:58.428 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000148407 2024-09-11T07:29:58.428 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.428 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.428 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:29:58.428 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.414509+0000", 2024-09-11T07:29:58.428 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00033236799999999999 2024-09-11T07:29:58.428 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.428 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.428 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 4,7", 2024-09-11T07:29:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.414921+0000", 2024-09-11T07:29:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00041222200000000001 2024-09-11T07:29:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:29:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.418488+0000", 2024-09-11T07:29:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.003566643 2024-09-11T07:29:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:89918 7.8 7:190c877c:::benchmark_data_smithi022_77720_object89917:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-11T07:29:58.429 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:58.418432+0000", 2024-09-11T07:29:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.00039345299999999999, 2024-09-11T07:29:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00054458600000000001, 2024-09-11T07:29:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:29:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "waiting for sub ops", 2024-09-11T07:29:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:29:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:29:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:29:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 89918 2024-09-11T07:29:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:29:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:29:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.418432+0000", 2024-09-11T07:29:58.430 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:29:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.418432+0000", 2024-09-11T07:29:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:29:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.418430+0000", 2024-09-11T07:29:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:29:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:29:58.431 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.418456+0000", 2024-09-11T07:29:58.432 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.6712000000000001e-05 2024-09-11T07:29:58.432 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.432 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.432 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:29:58.432 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.418470+0000", 2024-09-11T07:29:58.432 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4173e-05 2024-09-11T07:29:58.432 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.432 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.432 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:29:58.432 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.418477+0000", 2024-09-11T07:29:58.432 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9179999999999997e-06 2024-09-11T07:29:58.432 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.433 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.433 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:29:58.433 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.418510+0000", 2024-09-11T07:29:58.433 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2381e-05 2024-09-11T07:29:58.433 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.433 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.433 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:29:58.433 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.418735+0000", 2024-09-11T07:29:58.433 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000225744 2024-09-11T07:29:58.433 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.433 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.433 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 4,7", 2024-09-11T07:29:58.433 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.418899+0000", 2024-09-11T07:29:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00016395900000000001 2024-09-11T07:29:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop_reply(client.24778.0:89908 7.8 e116/92)", 2024-09-11T07:29:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:58.418634+0000", 2024-09-11T07:29:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.00019181999999999999, 2024-09-11T07:29:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00044919900000000001, 2024-09-11T07:29:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:29:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "queued for pg", 2024-09-11T07:29:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:29:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.434 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:29:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.418634+0000", 2024-09-11T07:29:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:29:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.418634+0000", 2024-09-11T07:29:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:29:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.418632+0000", 2024-09-11T07:29:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:29:58.435 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:29:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.418639+0000", 2024-09-11T07:29:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0509999999999999e-06 2024-09-11T07:29:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:29:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.418651+0000", 2024-09-11T07:29:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.201e-05 2024-09-11T07:29:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:29:58.436 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.418656+0000", 2024-09-11T07:29:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2070000000000003e-06 2024-09-11T07:29:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop_reply(client.24778.0:89908 7.8 e116/92)", 2024-09-11T07:29:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:58.418672+0000", 2024-09-11T07:29:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.00015367000000000001, 2024-09-11T07:29:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000456377, 2024-09-11T07:29:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:29:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "queued for pg", 2024-09-11T07:29:58.437 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:29:58.438 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.438 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:29:58.438 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.418672+0000", 2024-09-11T07:29:58.438 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:58.438 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.438 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.438 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:29:58.438 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.418672+0000", 2024-09-11T07:29:58.438 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:29:58.438 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.438 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.438 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:29:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.418671+0000", 2024-09-11T07:29:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:29:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:29:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.418675+0000", 2024-09-11T07:29:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0069999999999999e-06 2024-09-11T07:29:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:29:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.418681+0000", 2024-09-11T07:29:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6799999999999998e-06 2024-09-11T07:29:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:29:58.439 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:29:58.440 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:29:58.440 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:58.418685+0000", 2024-09-11T07:29:58.440 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8010000000000001e-06 2024-09-11T07:29:58.440 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:58.440 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:29:58.440 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:58.440 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:29:58.440 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:29:58.440 INFO:teuthology.orchestra.run.smithi022.stdout: "num_ops": 4 2024-09-11T07:29:58.440 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:29:58.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:58.477 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:58.477 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:58.578 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:29:58.591 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 56 16 90104 90088 100.526 64.25 0.0152253 0.00994316 2024-09-11T07:29:58.606 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:58.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:58.607 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:29:58.707 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:29:58.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:58.750 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:58.751 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:29:58.831 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:58 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:29:58.831 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:58 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:29:58.831 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:58 smithi104 ceph-mon[33507]: osdmap e116: 8 total, 8 up, 8 in 2024-09-11T07:29:58.831 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:58 smithi104 ceph-mon[33507]: pgmap v132: 148 pgs: 148 active+clean; 5.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 785 B/s rd, 53 MiB/s wr, 854 op/s; 16 B/s, 1 objects/s recovering 2024-09-11T07:29:58.851 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:58.890 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:58.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:58.891 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:58.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:58 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:29:58.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:58 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:29:58.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:58 smithi022 ceph-mon[29462]: osdmap e116: 8 total, 8 up, 8 in 2024-09-11T07:29:58.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:58 smithi022 ceph-mon[29462]: pgmap v132: 148 pgs: 148 active+clean; 5.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 785 B/s rd, 53 MiB/s wr, 854 op/s; 16 B/s, 1 objects/s recovering 2024-09-11T07:29:58.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:58 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:29:58.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:58 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:29:58.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:58 smithi022 ceph-mon[37330]: osdmap e116: 8 total, 8 up, 8 in 2024-09-11T07:29:58.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:58 smithi022 ceph-mon[37330]: pgmap v132: 148 pgs: 148 active+clean; 5.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 785 B/s rd, 53 MiB/s wr, 854 op/s; 16 B/s, 1 objects/s recovering 2024-09-11T07:29:58.960 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:29:58.961 INFO:teuthology.orchestra.run.smithi104.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-11T07:29:58.961 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:29:58.992 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:29:59.043 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:59.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:59.044 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:29:59.145 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:59.172 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:59.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:59.173 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:59.274 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:59.315 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:59.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:59.316 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:59.416 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:59.446 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:59.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:59.447 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:59.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:29:59 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:29:59.548 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:29:59.589 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-11T07:29:59.591 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:29:59.591 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 57 16 91207 91191 99.9715 68.9375 0.00751035 0.00999785 2024-09-11T07:29:59.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:59.592 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:29:59.693 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:29:59.721 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:59.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:59.722 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:29:59.802 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:29:59 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:29:59.803 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:29:59 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:29:59.823 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:29:59.846 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:29:59.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:29:59.847 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:29:59.899 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:29:59.949 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:00.025 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:00.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:00.026 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:00.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:29:59 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:00.127 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:00.153 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:00.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:00.154 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:00.254 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:00.283 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:00.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:00.285 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:00.385 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:00.425 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:00.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:00.426 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:00.475 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is set 2024-09-11T07:30:00.527 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:00.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:00.566 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:00.566 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:00.591 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 58 16 92184 92168 99.3006 61.0625 0.00679163 0.0100651 2024-09-11T07:30:00.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:30:00 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:30:00] "GET /metrics HTTP/1.1" 200 223629 "" "Prometheus/2.43.0" 2024-09-11T07:30:00.667 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:00.691 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:00.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:00.692 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:00.793 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:00.820 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:00.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:00.821 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:00.922 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:00.971 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:00.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:00.972 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:00.980 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:00 smithi022 ceph-mon[29462]: pgmap v133: 148 pgs: 148 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 86 MiB/s wr, 1.38k op/s; 13 B/s, 0 objects/s recovering 2024-09-11T07:30:00.980 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:00 smithi022 ceph-mon[29462]: Health detail: HEALTH_WARN noscrub flag(s) set 2024-09-11T07:30:00.980 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:00 smithi022 ceph-mon[29462]: [WRN] OSDMAP_FLAGS: noscrub flag(s) set 2024-09-11T07:30:00.980 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:00 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2804049567' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:30:00.980 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:00 smithi022 ceph-mon[29462]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:30:00.980 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:00 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2804049567' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:30:00.980 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:00 smithi022 ceph-mon[29462]: osdmap e117: 8 total, 8 up, 8 in 2024-09-11T07:30:00.980 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:00 smithi022 ceph-mon[37330]: pgmap v133: 148 pgs: 148 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 86 MiB/s wr, 1.38k op/s; 13 B/s, 0 objects/s recovering 2024-09-11T07:30:00.981 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:00 smithi022 ceph-mon[37330]: Health detail: HEALTH_WARN noscrub flag(s) set 2024-09-11T07:30:00.981 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:00 smithi022 ceph-mon[37330]: [WRN] OSDMAP_FLAGS: noscrub flag(s) set 2024-09-11T07:30:00.981 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:00 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2804049567' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:30:00.981 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:00 smithi022 ceph-mon[37330]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:30:00.981 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:00 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2804049567' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:30:00.981 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:00 smithi022 ceph-mon[37330]: osdmap e117: 8 total, 8 up, 8 in 2024-09-11T07:30:01.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:00 smithi104 ceph-mon[33507]: pgmap v133: 148 pgs: 148 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 86 MiB/s wr, 1.38k op/s; 13 B/s, 0 objects/s recovering 2024-09-11T07:30:01.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:00 smithi104 ceph-mon[33507]: Health detail: HEALTH_WARN noscrub flag(s) set 2024-09-11T07:30:01.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:00 smithi104 ceph-mon[33507]: [WRN] OSDMAP_FLAGS: noscrub flag(s) set 2024-09-11T07:30:01.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:00 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2804049567' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:30:01.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:00 smithi104 ceph-mon[33507]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:30:01.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:00 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2804049567' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:30:01.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:00 smithi104 ceph-mon[33507]: osdmap e117: 8 total, 8 up, 8 in 2024-09-11T07:30:01.073 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:01.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:01.103 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:01.103 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:01.204 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:01.235 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:01.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:01.236 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:01.336 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:01.412 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:01.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:01.413 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:01.514 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:01.540 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:01.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:01.541 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:01.591 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 59 16 93328 93312 98.8293 71.5 0.0095508 0.0101135 2024-09-11T07:30:01.642 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:01.686 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:01.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:01.687 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:01.788 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:01.804 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:01.838 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:01.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:01.839 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:01.940 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:02.002 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:02.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:02.003 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:02.104 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:02.129 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:02.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:02.130 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:02.231 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:02.277 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:02.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:02.278 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:02.379 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:02.421 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:02.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:02.422 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:02.523 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:02.568 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:02.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:02.570 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:02.591 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:2024-09-11T07:30:02.590737+0000 min lat: 0.00117769 max lat: 0.136258 avg lat: 0.0101652 2024-09-11T07:30:02.591 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T07:30:02.591 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 60 16 94430 94414 98.3301 68.875 0.022661 0.0101652 2024-09-11T07:30:02.671 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:02.706 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:02.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:02.708 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:02.809 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:02.846 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:02.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:02.847 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:02.948 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:02.990 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:02.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:02.991 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:03.092 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:03.117 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:03.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:03.118 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:03.137 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:30:03.137 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:30:03.137 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:03 smithi022 ceph-mon[29462]: pgmap v135: 148 pgs: 148 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 785 B/s rd, 52 MiB/s wr, 836 op/s 2024-09-11T07:30:03.137 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:03 smithi022 ceph-mon[37330]: pgmap v135: 148 pgs: 148 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 785 B/s rd, 52 MiB/s wr, 836 op/s 2024-09-11T07:30:03.181 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:30:03.181 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:30:03.219 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:03.224 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:30:03.225 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:30:03.257 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:30:03.257 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:30:03.260 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:03.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:03.261 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:03.289 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:30:03.289 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:30:03.322 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:30:03.322 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:30:03.324 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T07:30:03.356 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:30:03.356 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:30:03.362 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:03.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:03.387 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:03.387 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:03.389 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:30:03.389 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:30:03.488 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:03.512 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:03.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:03.514 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:03.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:03 smithi104 ceph-mon[33507]: pgmap v135: 148 pgs: 148 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 785 B/s rd, 52 MiB/s wr, 836 op/s 2024-09-11T07:30:03.591 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 61 16 95431 95415 97.7436 62.5625 0.011048 0.0102256 2024-09-11T07:30:03.615 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:03.679 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:03.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:03.680 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:03.780 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:03.807 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:03.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:03.809 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:03.909 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:03.938 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:03.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:03.939 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:04.040 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:04.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:04.086 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:04.086 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:04.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:04 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3834887993' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:30:04.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:04 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3834887993' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:30:04.187 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:04.213 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:04.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:04.214 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:04.315 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:04.337 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:04.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:04.339 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:04.339 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:30:04.339 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:30:04.440 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:04.487 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:04.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:04.489 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:04.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:04 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3834887993' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:30:04.530 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:30:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:30:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:30:04.589 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:04.591 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 62 16 96398 96382 97.1419 60.4375 0.0121331 0.0102891 2024-09-11T07:30:04.614 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:04.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:04.615 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:04.716 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:04.749 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:30:04.749 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:30:04.749 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-11T07:30:04.749 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-11T07:30:04.758 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:04.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:04.760 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:04.860 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:04.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:04.926 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:04.926 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:05.028 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:05.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:05.095 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:05.095 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:05.196 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:05.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:05.243 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:05.243 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:05.324 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:05.345 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:05.370 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:05.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:05.371 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:05.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:05 smithi022 ceph-mon[29462]: pgmap v136: 148 pgs: 148 active+clean; 5.7 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.1 KiB/s rd, 61 MiB/s wr, 983 op/s 2024-09-11T07:30:05.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:05 smithi022 ceph-mon[37330]: pgmap v136: 148 pgs: 148 active+clean; 5.7 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.1 KiB/s rd, 61 MiB/s wr, 983 op/s 2024-09-11T07:30:05.472 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:05.526 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:05.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:05.527 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:05.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:05 smithi104 ceph-mon[33507]: pgmap v136: 148 pgs: 148 active+clean; 5.7 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.1 KiB/s rd, 61 MiB/s wr, 983 op/s 2024-09-11T07:30:05.592 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 63 16 97311 97295 96.5057 57.0625 0.00975837 0.0103574 2024-09-11T07:30:05.628 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:05.669 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:05.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:05.670 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:05.771 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:05.806 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:05.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:05.808 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:05.908 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:05.932 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:05.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:05.933 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:06.034 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:06.074 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:06.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:06.075 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:06.175 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:06.216 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:06.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:06.217 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:06.318 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:06.340 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:06.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:06.341 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:06.441 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:06.464 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:06.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:06.465 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:06.566 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:06.586 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:06.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:06.587 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:06.591 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 64 16 98205 98189 95.8708 55.875 0.0340086 0.0104202 2024-09-11T07:30:06.688 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:06.728 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:06.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:06.729 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:06.778 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:30:06.829 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:06.856 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:06.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:06.858 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:06.958 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:06.984 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:06.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:06.986 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:07.086 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:07.112 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:07.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:07.113 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:07.213 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:07.253 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:07.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:07.254 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:07.335 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:07 smithi022 ceph-mon[29462]: pgmap v137: 148 pgs: 148 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.1 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-11T07:30:07.336 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:07 smithi022 ceph-mon[37330]: pgmap v137: 148 pgs: 148 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.1 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-11T07:30:07.354 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:07.376 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-11T07:30:07.379 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:07.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:07.380 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:07.481 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:07.507 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:07.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:07.508 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:07.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:07 smithi104 ceph-mon[33507]: pgmap v137: 148 pgs: 148 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.1 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-11T07:30:07.591 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 65 16 99253 99237 95.4035 65.5 0.0171857 0.0104773 2024-09-11T07:30:07.609 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:07.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:07.637 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:07.637 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:07.738 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:07.762 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:07.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:07.763 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:07.864 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:07.889 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:07.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:07.890 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:07.990 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:08.017 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:08.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:08.018 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:08.119 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:08.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:08.156 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:08.157 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:08.257 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:08.319 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:08.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:08.321 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:08.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:08 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:30:08.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:08 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:30:08.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:30:08.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-11T07:30:08.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:08 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:30:08.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:08 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:30:08.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:30:08.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-11T07:30:08.422 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:08.450 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:08.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:08.452 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:08.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:08 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:30:08.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:08 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:30:08.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:30:08.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-11T07:30:08.552 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:08.579 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:08.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:08.580 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:08.592 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 66 16 100297 100281 94.9466 65.25 0.0180827 0.0105272 2024-09-11T07:30:08.681 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:08.718 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:08.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:08.719 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:08.741 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:08.786 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:08.794 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:08.819 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:08.844 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:08.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:08.845 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:08.946 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:09.001 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:09.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:09.005 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:09.106 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:09.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:09.165 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:09.166 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:09.266 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:09.311 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:09.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:09.312 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:09.412 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:09.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:09 smithi022 ceph-mon[29462]: pgmap v138: 148 pgs: 148 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 921 B/s rd, 67 MiB/s wr, 1.08k op/s 2024-09-11T07:30:09.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:09 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-09-11T07:30:09.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:09 smithi022 ceph-mon[29462]: osdmap e118: 8 total, 8 up, 8 in 2024-09-11T07:30:09.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:09 smithi022 ceph-mon[37330]: pgmap v138: 148 pgs: 148 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 921 B/s rd, 67 MiB/s wr, 1.08k op/s 2024-09-11T07:30:09.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:09 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-09-11T07:30:09.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:09 smithi022 ceph-mon[37330]: osdmap e118: 8 total, 8 up, 8 in 2024-09-11T07:30:09.451 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:09.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:09.452 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:09.530 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:30:09 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:30:09.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:09 smithi104 ceph-mon[33507]: pgmap v138: 148 pgs: 148 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 921 B/s rd, 67 MiB/s wr, 1.08k op/s 2024-09-11T07:30:09.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:09 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-09-11T07:30:09.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:09 smithi104 ceph-mon[33507]: osdmap e118: 8 total, 8 up, 8 in 2024-09-11T07:30:09.553 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:09.592 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 67 16 101336 101320 94.4987 64.9375 0.0213693 0.0105774 2024-09-11T07:30:09.596 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:09.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:09.597 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:09.698 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:09.743 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:09.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:09.744 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:09.845 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:09.890 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:09.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:09.891 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:09.992 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:10.042 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:10.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:10.043 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:10.144 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:10.171 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:10.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:10.172 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:10.273 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:10.314 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:10.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:10.315 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:10.374 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:10 smithi022 ceph-mon[29462]: osdmap e119: 8 total, 8 up, 8 in 2024-09-11T07:30:10.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:10 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:10.375 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:10 smithi022 ceph-mon[37330]: osdmap e119: 8 total, 8 up, 8 in 2024-09-11T07:30:10.375 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:10 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:10.415 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:10.461 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:10.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:10.462 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:10.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:10 smithi104 ceph-mon[33507]: osdmap e119: 8 total, 8 up, 8 in 2024-09-11T07:30:10.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:10 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:10.563 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:10.592 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 68 16 102428 102412 94.1126 68.25 0.0143131 0.0106205 2024-09-11T07:30:10.601 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:10.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:10.602 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:10.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:30:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:30:10] "GET /metrics HTTP/1.1" 200 223621 "" "Prometheus/2.43.0" 2024-09-11T07:30:10.702 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:10.747 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:10.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:10.748 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:10.849 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:10.891 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:10.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:10.892 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:10.993 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:11.019 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:11.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:11.021 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:11.121 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:11.146 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:11.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:11.147 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:11.248 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:11.287 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:11.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:11.289 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:11.389 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:11.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:11 smithi022 ceph-mon[29462]: pgmap v141: 148 pgs: 1 peering, 147 active+clean; 6.1 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.1 KiB/s rd, 80 MiB/s wr, 1.27k op/s 2024-09-11T07:30:11.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:11 smithi022 ceph-mon[37330]: pgmap v141: 148 pgs: 1 peering, 147 active+clean; 6.1 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.1 KiB/s rd, 80 MiB/s wr, 1.27k op/s 2024-09-11T07:30:11.429 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:11.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:11.430 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:11.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:11 smithi104 ceph-mon[33507]: pgmap v141: 148 pgs: 1 peering, 147 active+clean; 6.1 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.1 KiB/s rd, 80 MiB/s wr, 1.27k op/s 2024-09-11T07:30:11.531 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:11.570 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:11.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:11.571 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:11.592 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 69 16 103526 103510 93.7432 68.625 0.0228305 0.0106623 2024-09-11T07:30:11.672 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:11.736 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:11.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:11.737 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:11.838 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:11.889 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:11.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:11.890 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:11.991 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:12.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:12.020 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:12.020 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:12.121 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:12.149 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:12.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:12.150 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:12.251 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:12.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:12.295 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:12.295 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:12.396 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:12.422 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:12.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:12.423 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:12.524 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:12.573 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:30:12.573 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:30:12.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:12.585 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:12.586 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:12.592 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 70 16 104524 104508 93.295 62.375 0.0260337 0.0107133 2024-09-11T07:30:12.622 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:30:12.622 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:30:12.670 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:30:12.678 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:30:12.686 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:12.715 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:30:12.715 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:30:12.720 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:12.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:12.721 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:12.731 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:30:12.732 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:30:12.747 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:30:12.747 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:30:12.765 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:30:12.765 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:30:12.800 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:30:12.801 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:30:12.822 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:12.861 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:12.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:12.862 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:12.963 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:12.990 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:12.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:12.991 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:13.092 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:13.123 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is unset 2024-09-11T07:30:13.137 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:13.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:13.138 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:13.239 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:13.265 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:13.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:13.266 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:13.367 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:13.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:13 smithi022 ceph-mon[29462]: pgmap v142: 148 pgs: 1 peering, 147 active+clean; 6.1 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 61 MiB/s wr, 971 op/s 2024-09-11T07:30:13.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:13 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1873613415' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:30:13.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:13 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3245393814' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:30:13.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:13 smithi022 ceph-mon[37330]: pgmap v142: 148 pgs: 1 peering, 147 active+clean; 6.1 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 61 MiB/s wr, 971 op/s 2024-09-11T07:30:13.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:13 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1873613415' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:30:13.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:13 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3245393814' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:30:13.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:13.420 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:13.420 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:13.521 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:13.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:13 smithi104 ceph-mon[33507]: pgmap v142: 148 pgs: 1 peering, 147 active+clean; 6.1 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 61 MiB/s wr, 971 op/s 2024-09-11T07:30:13.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:13 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1873613415' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:30:13.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:13 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3245393814' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:30:13.546 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:13.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:13.547 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:13.594 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 71 16 105531 105515 92.8654 62.9375 0.0115499 0.010763 2024-09-11T07:30:13.648 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:13.697 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:13.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:13.699 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:13.800 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:13.826 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:13.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:13.827 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:13.928 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:13.973 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:13.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:13.975 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:14.076 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:14.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:14.116 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:14.117 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:14.218 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:14.259 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:14.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:14.260 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:14.361 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:14.412 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:14.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:14.414 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:14.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:14 smithi022 ceph-mon[29462]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:30:14.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:14 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3245393814' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:30:14.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:14 smithi022 ceph-mon[29462]: osdmap e120: 8 total, 8 up, 8 in 2024-09-11T07:30:14.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:14 smithi022 ceph-mon[37330]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:30:14.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:14 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3245393814' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:30:14.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:14 smithi022 ceph-mon[37330]: osdmap e120: 8 total, 8 up, 8 in 2024-09-11T07:30:14.515 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:14.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:30:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:30:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:30:14.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:14 smithi104 ceph-mon[33507]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:30:14.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:14 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3245393814' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:30:14.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:14 smithi104 ceph-mon[33507]: osdmap e120: 8 total, 8 up, 8 in 2024-09-11T07:30:14.540 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:14.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:14.541 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:14.594 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 72 16 106608 106592 92.5104 67.3125 0.0140495 0.0108046 2024-09-11T07:30:14.642 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:14.671 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:14.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:14.671 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:14.772 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:14.802 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:14.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:14.803 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:14.904 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:14.931 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:14.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:14.933 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:15.033 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:15.058 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:15.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:15.059 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:15.159 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:15.207 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:15.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:15.208 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:15.216 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:15 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:30:15.216 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:15 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:30:15.216 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:15 smithi022 ceph-mon[29462]: pgmap v144: 148 pgs: 1 peering, 147 active+clean; 6.3 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-11T07:30:15.216 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:15 smithi022 ceph-mon[29462]: 5.9 scrub starts 2024-09-11T07:30:15.216 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:15 smithi022 ceph-mon[29462]: 5.9 scrub ok 2024-09-11T07:30:15.216 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:15 smithi022 ceph-mon[29462]: osdmap e121: 8 total, 8 up, 8 in 2024-09-11T07:30:15.216 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:15 smithi022 ceph-mon[29462]: 3.0 scrub starts 2024-09-11T07:30:15.216 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:15 smithi022 ceph-mon[29462]: 3.0 scrub ok 2024-09-11T07:30:15.216 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:15 smithi022 ceph-mon[29462]: 7.d scrub starts 2024-09-11T07:30:15.217 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:15 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:30:15.217 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:15 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:30:15.217 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:15 smithi022 ceph-mon[37330]: pgmap v144: 148 pgs: 1 peering, 147 active+clean; 6.3 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-11T07:30:15.217 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:15 smithi022 ceph-mon[37330]: 5.9 scrub starts 2024-09-11T07:30:15.217 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:15 smithi022 ceph-mon[37330]: 5.9 scrub ok 2024-09-11T07:30:15.217 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:15 smithi022 ceph-mon[37330]: osdmap e121: 8 total, 8 up, 8 in 2024-09-11T07:30:15.217 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:15 smithi022 ceph-mon[37330]: 3.0 scrub starts 2024-09-11T07:30:15.217 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:15 smithi022 ceph-mon[37330]: 3.0 scrub ok 2024-09-11T07:30:15.217 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:15 smithi022 ceph-mon[37330]: 7.d scrub starts 2024-09-11T07:30:15.218 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:30:15.218 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 20, 2024-09-11T07:30:15.218 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 600, 2024-09-11T07:30:15.218 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:30:15.218 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.218 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:23747 7.e 7:7a39a578:::benchmark_data_smithi022_77720_object23746:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-11T07:30:15.218 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:11.256089+0000", 2024-09-11T07:30:15.218 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 63.955011493999997, 2024-09-11T07:30:15.218 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.044460710000000001, 2024-09-11T07:30:15.218 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:15.219 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:15.219 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:15.219 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:15.219 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:15.219 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 23747 2024-09-11T07:30:15.219 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.219 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:15.219 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.219 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:15.219 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.256089+0000", 2024-09-11T07:30:15.219 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.219 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.219 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.220 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:15.220 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.256089+0000", 2024-09-11T07:30:15.220 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.220 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.220 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.220 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:15.220 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.256087+0000", 2024-09-11T07:30:15.220 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:30:15.220 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.220 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.220 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:15.220 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.256110+0000", 2024-09-11T07:30:15.221 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2708e-05 2024-09-11T07:30:15.221 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.221 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.221 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:15.221 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.256121+0000", 2024-09-11T07:30:15.221 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0924000000000001e-05 2024-09-11T07:30:15.221 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.221 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.221 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:15.221 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.256126+0000", 2024-09-11T07:30:15.221 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6389999999999997e-06 2024-09-11T07:30:15.221 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.221 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.222 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:15.222 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.256796+0000", 2024-09-11T07:30:15.222 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00067015800000000002 2024-09-11T07:30:15.222 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.222 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.222 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:15.222 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.256907+0000", 2024-09-11T07:30:15.222 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011093499999999999 2024-09-11T07:30:15.222 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.222 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.222 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:30:15.222 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.257026+0000", 2024-09-11T07:30:15.222 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000118225 2024-09-11T07:30:15.222 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.223 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.223 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.223 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.258849+0000", 2024-09-11T07:30:15.223 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0018237570000000001 2024-09-11T07:30:15.223 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.223 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.223 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:15.223 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.260081+0000", 2024-09-11T07:30:15.223 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0012314579999999999 2024-09-11T07:30:15.223 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.223 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.223 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.223 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.279679+0000", 2024-09-11T07:30:15.223 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.019598603999999999 2024-09-11T07:30:15.224 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.224 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.224 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:15.224 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.280218+0000", 2024-09-11T07:30:15.224 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00053848100000000005 2024-09-11T07:30:15.224 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.224 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.224 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:15.224 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.300549+0000", 2024-09-11T07:30:15.224 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.020331592999999999 2024-09-11T07:30:15.224 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.224 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:15.224 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.224 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.225 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.225 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:23753 7.e 7:77f3d0d9:::benchmark_data_smithi022_77720_object23752:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-11T07:30:15.225 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:11.258708+0000", 2024-09-11T07:30:15.225 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 63.952392404999998, 2024-09-11T07:30:15.225 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.043380032999999998, 2024-09-11T07:30:15.225 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:15.225 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:15.225 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:15.225 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:15.225 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:15.225 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 23753 2024-09-11T07:30:15.225 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.225 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:15.225 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.225 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:15.226 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.258708+0000", 2024-09-11T07:30:15.226 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.226 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.226 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.226 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:15.226 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.258708+0000", 2024-09-11T07:30:15.226 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.226 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.226 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.226 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:15.226 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.258706+0000", 2024-09-11T07:30:15.226 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:30:15.226 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.226 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.227 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:15.227 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.258743+0000", 2024-09-11T07:30:15.227 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6554000000000003e-05 2024-09-11T07:30:15.227 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.227 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.227 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:15.227 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.258754+0000", 2024-09-11T07:30:15.227 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1195999999999999e-05 2024-09-11T07:30:15.227 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.227 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.227 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:15.227 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.258760+0000", 2024-09-11T07:30:15.227 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6720000000000001e-06 2024-09-11T07:30:15.228 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.228 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.228 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:15.228 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.258873+0000", 2024-09-11T07:30:15.228 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000113955 2024-09-11T07:30:15.228 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.228 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.228 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:15.228 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.258994+0000", 2024-09-11T07:30:15.228 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000120377 2024-09-11T07:30:15.228 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.228 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.228 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:30:15.228 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.259152+0000", 2024-09-11T07:30:15.229 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000158529 2024-09-11T07:30:15.229 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.229 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.229 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.229 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.260248+0000", 2024-09-11T07:30:15.229 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00109545 2024-09-11T07:30:15.229 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.229 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.229 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:15.229 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.260277+0000", 2024-09-11T07:30:15.229 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8887e-05 2024-09-11T07:30:15.229 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.229 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.230 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.230 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.302000+0000", 2024-09-11T07:30:15.230 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.041723199000000002 2024-09-11T07:30:15.230 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.230 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.230 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:15.230 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.302049+0000", 2024-09-11T07:30:15.230 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9227000000000001e-05 2024-09-11T07:30:15.230 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.230 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.231 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:15.231 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.302088+0000", 2024-09-11T07:30:15.231 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8708999999999998e-05 2024-09-11T07:30:15.231 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.231 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:15.231 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.231 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.231 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.231 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:24102 7.2 e102/92 7:413b5fb6:::benchmark_data_smithi022_77720_object24101:head v 102'1563, mlcod=102'1560)", 2024-09-11T07:30:15.231 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:11.442572+0000", 2024-09-11T07:30:15.231 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 63.768528285999999, 2024-09-11T07:30:15.231 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.051290802000000003, 2024-09-11T07:30:15.231 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:15.232 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:15.232 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:15.232 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.232 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:15.232 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.442572+0000", 2024-09-11T07:30:15.232 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.232 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.232 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.232 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:15.233 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.442572+0000", 2024-09-11T07:30:15.233 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.233 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.233 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.233 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:15.233 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.442571+0000", 2024-09-11T07:30:15.233 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:30:15.233 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.233 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.233 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:15.233 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.442603+0000", 2024-09-11T07:30:15.233 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2178999999999998e-05 2024-09-11T07:30:15.234 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.234 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.234 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:15.234 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.442613+0000", 2024-09-11T07:30:15.234 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0176999999999999e-05 2024-09-11T07:30:15.234 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.234 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.234 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:15.234 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.442618+0000", 2024-09-11T07:30:15.234 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.0429999999999998e-06 2024-09-11T07:30:15.234 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.234 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.234 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:15.234 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.442792+0000", 2024-09-11T07:30:15.235 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00017448100000000001 2024-09-11T07:30:15.235 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.235 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.235 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:15.235 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.442932+0000", 2024-09-11T07:30:15.235 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013960000000000001 2024-09-11T07:30:15.235 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.235 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.235 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:15.235 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.493829+0000", 2024-09-11T07:30:15.235 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.050896798 2024-09-11T07:30:15.235 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.235 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.236 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:15.236 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.493863+0000", 2024-09-11T07:30:15.236 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3868999999999997e-05 2024-09-11T07:30:15.236 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.236 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:15.236 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.236 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.236 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.236 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:24107 7.a e102/92 7:58272ad4:::benchmark_data_smithi022_77720_object24106:head v 102'1483, mlcod=102'1480)", 2024-09-11T07:30:15.236 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:11.443542+0000", 2024-09-11T07:30:15.236 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 63.767558027, 2024-09-11T07:30:15.236 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.050414107, 2024-09-11T07:30:15.236 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:15.237 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:15.237 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:15.237 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.237 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:15.237 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.443542+0000", 2024-09-11T07:30:15.237 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.237 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.237 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.237 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:15.237 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.443542+0000", 2024-09-11T07:30:15.237 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.237 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.237 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.237 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:15.238 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.443541+0000", 2024-09-11T07:30:15.238 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:30:15.238 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.238 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.238 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:15.238 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.443571+0000", 2024-09-11T07:30:15.238 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0258999999999999e-05 2024-09-11T07:30:15.238 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.238 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.238 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:15.238 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.443582+0000", 2024-09-11T07:30:15.238 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1107000000000001e-05 2024-09-11T07:30:15.238 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.238 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.239 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:15.239 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.443589+0000", 2024-09-11T07:30:15.239 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5089999999999999e-06 2024-09-11T07:30:15.239 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.239 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.239 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:15.239 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.443907+0000", 2024-09-11T07:30:15.239 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00031792799999999998 2024-09-11T07:30:15.239 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.239 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.239 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:15.239 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.443925+0000", 2024-09-11T07:30:15.239 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8598e-05 2024-09-11T07:30:15.239 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.240 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.240 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:15.240 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.493928+0000", 2024-09-11T07:30:15.240 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.050002682 2024-09-11T07:30:15.240 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.240 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.240 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:15.240 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.493956+0000", 2024-09-11T07:30:15.240 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8334e-05 2024-09-11T07:30:15.240 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.240 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:15.240 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.240 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.240 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.241 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:32058 7.8 7:14d971a6:::benchmark_data_smithi022_77720_object32057:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-11T07:30:15.241 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:15.232354+0000", 2024-09-11T07:30:15.241 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 59.978745947999997, 2024-09-11T07:30:15.241 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.056887986000000001, 2024-09-11T07:30:15.241 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:15.241 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:15.241 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:15.241 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:15.241 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:15.241 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 32058 2024-09-11T07:30:15.241 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.241 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:15.241 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.242 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:15.242 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.232354+0000", 2024-09-11T07:30:15.242 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.242 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.242 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.242 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:15.242 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.232354+0000", 2024-09-11T07:30:15.242 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.242 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.242 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.242 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:15.242 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.232352+0000", 2024-09-11T07:30:15.242 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:30:15.242 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.243 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.243 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:15.243 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.232375+0000", 2024-09-11T07:30:15.243 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2469000000000001e-05 2024-09-11T07:30:15.243 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.243 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.243 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:15.243 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.232397+0000", 2024-09-11T07:30:15.243 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2336000000000001e-05 2024-09-11T07:30:15.243 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.243 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.243 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:15.243 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.232403+0000", 2024-09-11T07:30:15.244 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8320000000000002e-06 2024-09-11T07:30:15.244 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.244 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.244 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:15.244 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.232760+0000", 2024-09-11T07:30:15.244 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000356483 2024-09-11T07:30:15.244 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.244 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.244 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:15.244 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.235543+0000", 2024-09-11T07:30:15.244 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0027835070000000002 2024-09-11T07:30:15.244 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.244 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.245 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 4,7", 2024-09-11T07:30:15.245 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.235728+0000", 2024-09-11T07:30:15.245 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00018458299999999999 2024-09-11T07:30:15.245 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.245 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.245 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:15.245 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.237350+0000", 2024-09-11T07:30:15.245 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0016221810000000001 2024-09-11T07:30:15.245 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.245 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.245 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.245 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.273583+0000", 2024-09-11T07:30:15.246 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.036232762000000002 2024-09-11T07:30:15.246 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.246 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.246 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.246 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.289175+0000", 2024-09-11T07:30:15.246 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.015592488999999999 2024-09-11T07:30:15.246 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.246 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.246 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:15.246 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.289219+0000", 2024-09-11T07:30:15.246 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4184999999999998e-05 2024-09-11T07:30:15.246 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.246 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.247 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:15.247 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.289242+0000", 2024-09-11T07:30:15.247 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3003999999999999e-05 2024-09-11T07:30:15.248 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.248 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:15.248 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.248 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.248 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.248 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:32067 7.8 7:1a22d9d6:::benchmark_data_smithi022_77720_object32066:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-11T07:30:15.248 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:15.237030+0000", 2024-09-11T07:30:15.248 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 59.974070480000002, 2024-09-11T07:30:15.248 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.052279648999999997, 2024-09-11T07:30:15.248 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:15.248 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:15.248 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:15.249 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:15.249 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:15.249 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 32067 2024-09-11T07:30:15.249 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.249 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:15.249 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.249 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:15.249 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.237030+0000", 2024-09-11T07:30:15.249 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.249 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.249 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.249 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:15.249 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.237030+0000", 2024-09-11T07:30:15.250 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.250 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.250 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.250 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:15.250 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.237027+0000", 2024-09-11T07:30:15.250 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:30:15.250 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.250 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.250 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:15.250 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.237057+0000", 2024-09-11T07:30:15.250 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.9862e-05 2024-09-11T07:30:15.250 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.251 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.251 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:15.251 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.237069+0000", 2024-09-11T07:30:15.251 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2308e-05 2024-09-11T07:30:15.251 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.251 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.251 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:15.251 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.237076+0000", 2024-09-11T07:30:15.251 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4230000000000002e-06 2024-09-11T07:30:15.251 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.251 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.251 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:15.252 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.237370+0000", 2024-09-11T07:30:15.252 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00029456600000000001 2024-09-11T07:30:15.252 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.252 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.252 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:15.252 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.237475+0000", 2024-09-11T07:30:15.252 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000105115 2024-09-11T07:30:15.252 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.252 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.252 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 4,7", 2024-09-11T07:30:15.252 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.237608+0000", 2024-09-11T07:30:15.252 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013272699999999999 2024-09-11T07:30:15.253 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.253 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.253 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:15.253 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.240360+0000", 2024-09-11T07:30:15.253 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002752361 2024-09-11T07:30:15.253 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.253 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.253 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.253 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.273615+0000", 2024-09-11T07:30:15.253 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.033254696 2024-09-11T07:30:15.253 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.253 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.253 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.254 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.289269+0000", 2024-09-11T07:30:15.254 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.015653530999999998 2024-09-11T07:30:15.254 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.254 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.254 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:15.254 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.289292+0000", 2024-09-11T07:30:15.254 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3003999999999999e-05 2024-09-11T07:30:15.254 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.254 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.254 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:15.254 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.289309+0000", 2024-09-11T07:30:15.254 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.7808e-05 2024-09-11T07:30:15.254 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.254 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:15.254 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.255 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.255 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.255 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:32084 7.8 7:1e6c3fa4:::benchmark_data_smithi022_77720_object32083:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-11T07:30:15.255 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:15.245355+0000", 2024-09-11T07:30:15.255 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 59.965745431999999, 2024-09-11T07:30:15.255 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.044071859999999997, 2024-09-11T07:30:15.255 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:15.255 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:15.255 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:15.255 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:15.255 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:15.255 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 32084 2024-09-11T07:30:15.255 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.256 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:15.256 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.256 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:15.256 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.245355+0000", 2024-09-11T07:30:15.256 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.256 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.256 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.256 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:15.256 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.245355+0000", 2024-09-11T07:30:15.256 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.256 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.256 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.256 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:15.256 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.245352+0000", 2024-09-11T07:30:15.257 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:30:15.257 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.257 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.257 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:15.257 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.245382+0000", 2024-09-11T07:30:15.257 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.9838000000000001e-05 2024-09-11T07:30:15.257 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.257 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.257 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:15.257 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.245394+0000", 2024-09-11T07:30:15.257 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2065000000000001e-05 2024-09-11T07:30:15.257 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.257 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.257 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:15.258 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.245399+0000", 2024-09-11T07:30:15.258 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3560000000000003e-06 2024-09-11T07:30:15.258 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.258 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.258 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:15.258 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.245428+0000", 2024-09-11T07:30:15.258 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.9255999999999999e-05 2024-09-11T07:30:15.258 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.258 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.258 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:15.258 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.245500+0000", 2024-09-11T07:30:15.258 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1972999999999997e-05 2024-09-11T07:30:15.258 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.258 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.259 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 4,7", 2024-09-11T07:30:15.259 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.245579+0000", 2024-09-11T07:30:15.259 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.8745999999999994e-05 2024-09-11T07:30:15.259 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.259 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.259 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:15.259 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.246676+0000", 2024-09-11T07:30:15.259 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0010971970000000001 2024-09-11T07:30:15.259 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.259 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.259 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.259 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.273640+0000", 2024-09-11T07:30:15.259 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.026964715 2024-09-11T07:30:15.259 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.260 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.260 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.260 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.289342+0000", 2024-09-11T07:30:15.260 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.015701919000000002 2024-09-11T07:30:15.260 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.260 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.260 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:15.260 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.289384+0000", 2024-09-11T07:30:15.260 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1310999999999998e-05 2024-09-11T07:30:15.260 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.260 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.260 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:15.260 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:15.289427+0000", 2024-09-11T07:30:15.261 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2957000000000002e-05 2024-09-11T07:30:15.261 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.261 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:15.261 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.261 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.261 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.261 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:59519 7.e 7:783627a1:::benchmark_data_smithi022_77720_object59518:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-11T07:30:15.261 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:32.272700+0000", 2024-09-11T07:30:15.261 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 42.938399836999999, 2024-09-11T07:30:15.261 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.053010038000000002, 2024-09-11T07:30:15.261 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:15.261 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:15.261 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:15.262 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:15.262 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:15.262 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 59519 2024-09-11T07:30:15.262 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.262 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:15.262 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.262 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:15.262 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.272700+0000", 2024-09-11T07:30:15.262 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.262 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.262 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.262 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:15.262 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.272700+0000", 2024-09-11T07:30:15.262 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.263 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.263 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.263 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:15.263 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.272698+0000", 2024-09-11T07:30:15.263 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:30:15.263 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.263 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.263 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:15.263 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.272736+0000", 2024-09-11T07:30:15.263 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7694000000000002e-05 2024-09-11T07:30:15.263 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.263 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.263 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:15.263 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.272748+0000", 2024-09-11T07:30:15.263 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1921e-05 2024-09-11T07:30:15.264 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.264 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.264 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:15.264 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.272754+0000", 2024-09-11T07:30:15.264 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7669999999999998e-06 2024-09-11T07:30:15.264 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.264 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.264 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:15.264 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.272799+0000", 2024-09-11T07:30:15.264 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4740000000000002e-05 2024-09-11T07:30:15.264 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.264 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.264 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:15.265 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.272931+0000", 2024-09-11T07:30:15.265 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000131421 2024-09-11T07:30:15.265 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.265 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.265 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:30:15.265 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.273065+0000", 2024-09-11T07:30:15.265 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013405700000000001 2024-09-11T07:30:15.265 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.265 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.265 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.265 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.275213+0000", 2024-09-11T07:30:15.265 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002148535 2024-09-11T07:30:15.265 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.266 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.266 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:15.266 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.275438+0000", 2024-09-11T07:30:15.266 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000224997 2024-09-11T07:30:15.266 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.266 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.266 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.266 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.325425+0000", 2024-09-11T07:30:15.266 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.049986532 2024-09-11T07:30:15.266 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.266 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.266 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:15.266 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.325665+0000", 2024-09-11T07:30:15.266 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000239922 2024-09-11T07:30:15.267 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.267 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.267 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:15.267 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.325710+0000", 2024-09-11T07:30:15.267 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5837999999999997e-05 2024-09-11T07:30:15.267 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.267 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:15.267 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.267 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.267 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.267 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:59525 7.e 7:766763fb:::benchmark_data_smithi022_77720_object59524:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-11T07:30:15.267 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:32.278588+0000", 2024-09-11T07:30:15.267 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 42.932511857000001, 2024-09-11T07:30:15.267 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.055494289000000002, 2024-09-11T07:30:15.268 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:15.268 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:15.268 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:15.268 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:15.268 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:15.268 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 59525 2024-09-11T07:30:15.268 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.268 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:15.268 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.268 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:15.268 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.278588+0000", 2024-09-11T07:30:15.268 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.268 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.268 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.269 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:15.269 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.278588+0000", 2024-09-11T07:30:15.269 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.269 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.269 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.269 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:15.269 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.278586+0000", 2024-09-11T07:30:15.269 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:30:15.269 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.269 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.269 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:15.269 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.278609+0000", 2024-09-11T07:30:15.269 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2690000000000001e-05 2024-09-11T07:30:15.269 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.270 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.270 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:15.270 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.278617+0000", 2024-09-11T07:30:15.270 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3580000000000003e-06 2024-09-11T07:30:15.270 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.270 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.270 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:15.270 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.278622+0000", 2024-09-11T07:30:15.270 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4109999999999998e-06 2024-09-11T07:30:15.270 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.270 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.270 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:15.270 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.278677+0000", 2024-09-11T07:30:15.270 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.4790999999999998e-05 2024-09-11T07:30:15.271 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.271 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.271 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:15.271 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.278813+0000", 2024-09-11T07:30:15.271 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000136568 2024-09-11T07:30:15.271 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.271 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.271 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:30:15.271 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.278987+0000", 2024-09-11T07:30:15.271 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000174136 2024-09-11T07:30:15.271 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.271 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.271 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.271 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.313389+0000", 2024-09-11T07:30:15.272 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.034402184000000002 2024-09-11T07:30:15.272 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.272 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.272 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:15.272 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.313396+0000", 2024-09-11T07:30:15.272 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.6449999999999999e-06 2024-09-11T07:30:15.272 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.272 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.272 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.272 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.333797+0000", 2024-09-11T07:30:15.272 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.020400425999999999 2024-09-11T07:30:15.272 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.272 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.272 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:15.273 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.334044+0000", 2024-09-11T07:30:15.273 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0002477 2024-09-11T07:30:15.273 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.273 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.273 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:15.273 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.334083+0000", 2024-09-11T07:30:15.273 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8414000000000002e-05 2024-09-11T07:30:15.273 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.273 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:15.273 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.273 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.273 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.273 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:59527 7.e 7:73c5a0c4:::benchmark_data_smithi022_77720_object59526:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-11T07:30:15.273 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:32.279060+0000", 2024-09-11T07:30:15.274 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 42.932040043999997, 2024-09-11T07:30:15.274 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.057152702999999999, 2024-09-11T07:30:15.274 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:15.274 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:15.274 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:15.274 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:15.274 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:15.274 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 59527 2024-09-11T07:30:15.274 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.274 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:15.274 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.274 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:15.274 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.279060+0000", 2024-09-11T07:30:15.274 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.275 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.275 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.275 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:15.275 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.279060+0000", 2024-09-11T07:30:15.275 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.275 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.275 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.275 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:15.275 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.279059+0000", 2024-09-11T07:30:15.275 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:30:15.275 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.275 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.275 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:15.275 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.279078+0000", 2024-09-11T07:30:15.275 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9145000000000001e-05 2024-09-11T07:30:15.276 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.276 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.276 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:15.276 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.279089+0000", 2024-09-11T07:30:15.276 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0743e-05 2024-09-11T07:30:15.276 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.276 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.276 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:15.276 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.279093+0000", 2024-09-11T07:30:15.276 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5970000000000002e-06 2024-09-11T07:30:15.276 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.276 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.276 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:15.276 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.280975+0000", 2024-09-11T07:30:15.277 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0018813059999999999 2024-09-11T07:30:15.277 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.277 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.277 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:15.277 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.281050+0000", 2024-09-11T07:30:15.277 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5374e-05 2024-09-11T07:30:15.277 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.277 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.277 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:30:15.277 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.281178+0000", 2024-09-11T07:30:15.277 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000127649 2024-09-11T07:30:15.277 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.277 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.278 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.278 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.313460+0000", 2024-09-11T07:30:15.278 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.032282725999999998 2024-09-11T07:30:15.278 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.278 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.278 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:15.278 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.325362+0000", 2024-09-11T07:30:15.278 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.011902121999999999 2024-09-11T07:30:15.278 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.278 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.278 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.278 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.334150+0000", 2024-09-11T07:30:15.278 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0087872230000000003 2024-09-11T07:30:15.278 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.278 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.278 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:15.279 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.336155+0000", 2024-09-11T07:30:15.279 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0020053190000000002 2024-09-11T07:30:15.279 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.279 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.279 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:15.279 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.336213+0000", 2024-09-11T07:30:15.279 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7917999999999997e-05 2024-09-11T07:30:15.279 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.279 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:15.279 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.279 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.279 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.279 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:59528 7.8 7:168caf2d:::benchmark_data_smithi022_77720_object59527:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-11T07:30:15.279 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:32.281494+0000", 2024-09-11T07:30:15.280 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 42.929605926999997, 2024-09-11T07:30:15.280 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.044454841000000002, 2024-09-11T07:30:15.280 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:15.280 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:15.280 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:15.280 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:15.280 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:15.280 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 59528 2024-09-11T07:30:15.280 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.280 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:15.280 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.280 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:15.280 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.281494+0000", 2024-09-11T07:30:15.281 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.281 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.281 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.281 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:15.281 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.281494+0000", 2024-09-11T07:30:15.281 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.281 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.281 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.281 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:15.281 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.281493+0000", 2024-09-11T07:30:15.281 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:30:15.282 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.282 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.282 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:15.282 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.281511+0000", 2024-09-11T07:30:15.282 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8253e-05 2024-09-11T07:30:15.282 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.282 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.282 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:15.282 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.310616+0000", 2024-09-11T07:30:15.282 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.029104747 2024-09-11T07:30:15.282 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.282 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.282 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:15.282 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.310634+0000", 2024-09-11T07:30:15.283 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8379000000000001e-05 2024-09-11T07:30:15.283 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.283 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.283 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:15.283 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.311194+0000", 2024-09-11T07:30:15.283 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00055948499999999997 2024-09-11T07:30:15.283 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.283 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.283 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:15.283 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.311571+0000", 2024-09-11T07:30:15.283 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00037683999999999998 2024-09-11T07:30:15.283 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.283 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.284 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 4,7", 2024-09-11T07:30:15.284 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.311735+0000", 2024-09-11T07:30:15.284 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00016438499999999999 2024-09-11T07:30:15.284 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.284 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.284 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.284 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.323446+0000", 2024-09-11T07:30:15.284 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.011710952 2024-09-11T07:30:15.284 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.284 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.284 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.284 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.323484+0000", 2024-09-11T07:30:15.284 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8089000000000001e-05 2024-09-11T07:30:15.284 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.285 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.285 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:15.285 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.325862+0000", 2024-09-11T07:30:15.285 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0023781359999999999 2024-09-11T07:30:15.285 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.285 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.285 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:15.285 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.325910+0000", 2024-09-11T07:30:15.285 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7469999999999998e-05 2024-09-11T07:30:15.285 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.285 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.285 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:15.285 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:32.325949+0000", 2024-09-11T07:30:15.285 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9518999999999999e-05 2024-09-11T07:30:15.285 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.285 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:15.286 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.286 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.286 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.286 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:70031 7.e 7:7295b444:::benchmark_data_smithi022_77720_object70030:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-11T07:30:15.286 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:40.706437+0000", 2024-09-11T07:30:15.286 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 34.504663086999997, 2024-09-11T07:30:15.286 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.043519612999999999, 2024-09-11T07:30:15.286 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:15.286 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:15.286 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:15.286 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:15.286 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:15.286 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 70031 2024-09-11T07:30:15.286 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.287 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:15.287 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.287 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:15.287 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:40.706437+0000", 2024-09-11T07:30:15.287 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.287 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.287 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.287 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:15.287 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:40.706437+0000", 2024-09-11T07:30:15.287 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.287 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.287 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.287 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:15.287 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:40.706435+0000", 2024-09-11T07:30:15.287 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:30:15.288 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.288 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.288 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:15.288 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:40.706458+0000", 2024-09-11T07:30:15.288 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2631999999999999e-05 2024-09-11T07:30:15.288 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.288 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.288 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:15.288 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:40.706469+0000", 2024-09-11T07:30:15.288 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1389999999999999e-05 2024-09-11T07:30:15.288 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.288 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.288 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:15.288 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:40.706475+0000", 2024-09-11T07:30:15.289 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8250000000000001e-06 2024-09-11T07:30:15.289 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.289 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.289 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:15.289 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:40.706565+0000", 2024-09-11T07:30:15.289 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.0593999999999994e-05 2024-09-11T07:30:15.290 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.290 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.290 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:15.290 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:40.706714+0000", 2024-09-11T07:30:15.290 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014888600000000001 2024-09-11T07:30:15.290 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.290 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.290 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:30:15.290 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:40.706800+0000", 2024-09-11T07:30:15.290 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.5406000000000002e-05 2024-09-11T07:30:15.290 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.290 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.290 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:15.290 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:40.708789+0000", 2024-09-11T07:30:15.291 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0019892880000000001 2024-09-11T07:30:15.291 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.291 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.291 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.291 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:40.709085+0000", 2024-09-11T07:30:15.291 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000296321 2024-09-11T07:30:15.291 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.291 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.291 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.291 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:40.749686+0000", 2024-09-11T07:30:15.291 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.040600110000000002 2024-09-11T07:30:15.291 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.291 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.291 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:15.291 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:40.749832+0000", 2024-09-11T07:30:15.292 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014675199999999999 2024-09-11T07:30:15.292 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.292 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.292 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:15.292 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:40.749957+0000", 2024-09-11T07:30:15.292 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012448 2024-09-11T07:30:15.292 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.292 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:15.292 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.292 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.292 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.292 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:70607 7.e 7:79b70bbb:::benchmark_data_smithi022_77720_object70606:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-11T07:30:15.292 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:41.195175+0000", 2024-09-11T07:30:15.292 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 34.015925465000002, 2024-09-11T07:30:15.293 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.057048935000000002, 2024-09-11T07:30:15.293 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:15.293 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:15.293 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:15.293 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:15.293 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:15.293 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 70607 2024-09-11T07:30:15.293 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.293 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:15.293 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.293 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:15.293 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.195175+0000", 2024-09-11T07:30:15.293 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.294 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.294 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.294 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:15.294 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.195175+0000", 2024-09-11T07:30:15.294 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.294 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.294 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.294 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:15.294 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.195172+0000", 2024-09-11T07:30:15.294 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:30:15.294 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.294 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.294 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:15.294 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.195205+0000", 2024-09-11T07:30:15.294 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2699e-05 2024-09-11T07:30:15.295 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.295 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.295 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:15.295 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.195217+0000", 2024-09-11T07:30:15.295 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2411e-05 2024-09-11T07:30:15.295 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.295 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.295 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:15.295 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.195223+0000", 2024-09-11T07:30:15.295 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1129999999999997e-06 2024-09-11T07:30:15.295 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.295 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.295 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:15.295 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.195696+0000", 2024-09-11T07:30:15.296 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00047209200000000002 2024-09-11T07:30:15.296 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.296 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.296 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:15.296 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.195837+0000", 2024-09-11T07:30:15.296 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000141333 2024-09-11T07:30:15.296 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.296 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.296 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:30:15.296 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.195957+0000", 2024-09-11T07:30:15.296 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000119788 2024-09-11T07:30:15.296 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.296 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.296 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:15.297 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.199287+0000", 2024-09-11T07:30:15.297 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0033298780000000001 2024-09-11T07:30:15.297 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.297 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.297 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.297 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.199829+0000", 2024-09-11T07:30:15.297 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00054247699999999998 2024-09-11T07:30:15.297 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.297 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.297 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.297 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.251916+0000", 2024-09-11T07:30:15.297 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.052086562000000003 2024-09-11T07:30:15.297 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.298 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.298 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:15.298 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.252094+0000", 2024-09-11T07:30:15.298 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00017838199999999999 2024-09-11T07:30:15.298 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.298 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.298 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:15.298 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.252224+0000", 2024-09-11T07:30:15.298 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012975199999999999 2024-09-11T07:30:15.298 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.298 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:15.298 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.298 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.298 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.298 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:70627 7.e 7:75605726:::benchmark_data_smithi022_77720_object70626:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-11T07:30:15.299 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:41.248173+0000", 2024-09-11T07:30:15.299 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 33.962927286999999, 2024-09-11T07:30:15.299 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.046705443999999999, 2024-09-11T07:30:15.299 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:15.299 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:15.299 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:15.299 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:15.299 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:15.299 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 70627 2024-09-11T07:30:15.299 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.299 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:15.299 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.299 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:15.299 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.248173+0000", 2024-09-11T07:30:15.299 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.300 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.300 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.300 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:15.300 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.248173+0000", 2024-09-11T07:30:15.300 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.300 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.300 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.300 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:15.300 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.248171+0000", 2024-09-11T07:30:15.300 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:30:15.300 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.300 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.300 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:15.301 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.248197+0000", 2024-09-11T07:30:15.301 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.6400000000000001e-05 2024-09-11T07:30:15.301 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.301 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.301 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:15.301 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.248209+0000", 2024-09-11T07:30:15.301 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2063e-05 2024-09-11T07:30:15.301 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.301 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.301 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:15.301 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.248216+0000", 2024-09-11T07:30:15.301 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.6150000000000002e-06 2024-09-11T07:30:15.301 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.301 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.301 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:15.302 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.248594+0000", 2024-09-11T07:30:15.302 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000378322 2024-09-11T07:30:15.302 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.302 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.302 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:15.302 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.248707+0000", 2024-09-11T07:30:15.302 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011257800000000001 2024-09-11T07:30:15.302 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.302 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.302 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:30:15.302 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.248839+0000", 2024-09-11T07:30:15.302 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013226500000000001 2024-09-11T07:30:15.302 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.303 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.303 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.303 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.252395+0000", 2024-09-11T07:30:15.303 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0035563389999999999 2024-09-11T07:30:15.303 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.303 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.303 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:15.303 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.256040+0000", 2024-09-11T07:30:15.303 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0036451309999999998 2024-09-11T07:30:15.303 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.303 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.303 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.303 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.294784+0000", 2024-09-11T07:30:15.303 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.038743741999999998 2024-09-11T07:30:15.303 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.304 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.304 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:15.304 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.294833+0000", 2024-09-11T07:30:15.304 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9166000000000003e-05 2024-09-11T07:30:15.304 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.304 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.304 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:15.304 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.294878+0000", 2024-09-11T07:30:15.304 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5126999999999997e-05 2024-09-11T07:30:15.304 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.304 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:15.304 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.304 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.304 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.305 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:70629 7.e 7:78549458:::benchmark_data_smithi022_77720_object70628:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-11T07:30:15.305 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:41.249385+0000", 2024-09-11T07:30:15.305 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 33.961715046000002, 2024-09-11T07:30:15.305 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.050318332, 2024-09-11T07:30:15.305 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:15.305 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:15.305 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:15.305 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:15.305 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:15.305 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 70629 2024-09-11T07:30:15.305 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.305 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:15.305 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.305 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:15.306 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.249385+0000", 2024-09-11T07:30:15.306 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.306 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.306 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.306 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:15.306 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.249385+0000", 2024-09-11T07:30:15.306 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.306 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.306 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.306 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:15.306 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.249383+0000", 2024-09-11T07:30:15.306 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:30:15.306 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.306 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.306 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:15.307 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.249408+0000", 2024-09-11T07:30:15.307 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4516e-05 2024-09-11T07:30:15.307 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.307 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.307 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:15.307 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.249419+0000", 2024-09-11T07:30:15.307 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0974e-05 2024-09-11T07:30:15.307 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.307 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.307 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:15.307 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.249425+0000", 2024-09-11T07:30:15.307 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6910000000000002e-06 2024-09-11T07:30:15.307 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.307 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.307 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:15.308 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.252490+0000", 2024-09-11T07:30:15.308 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.003065365 2024-09-11T07:30:15.308 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.308 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.308 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:15.308 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.252868+0000", 2024-09-11T07:30:15.308 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000377792 2024-09-11T07:30:15.308 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.308 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.308 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:30:15.308 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.253599+0000", 2024-09-11T07:30:15.308 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00073117899999999999 2024-09-11T07:30:15.308 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.308 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.308 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.309 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.258025+0000", 2024-09-11T07:30:15.309 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0044259479999999999 2024-09-11T07:30:15.309 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.309 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.309 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:15.309 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.258032+0000", 2024-09-11T07:30:15.309 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.3050000000000004e-06 2024-09-11T07:30:15.309 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.309 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.309 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.309 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.299632+0000", 2024-09-11T07:30:15.309 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.041600093999999997 2024-09-11T07:30:15.309 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.309 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.310 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:15.310 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.299671+0000", 2024-09-11T07:30:15.310 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8562000000000001e-05 2024-09-11T07:30:15.310 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.310 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.310 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:15.310 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.299703+0000", 2024-09-11T07:30:15.310 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2731999999999998e-05 2024-09-11T07:30:15.310 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.310 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:15.310 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.310 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.310 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.311 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:70635 7.e 7:74e87fc3:::benchmark_data_smithi022_77720_object70634:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-11T07:30:15.311 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:41.254841+0000", 2024-09-11T07:30:15.311 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 33.956259297000003, 2024-09-11T07:30:15.311 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.047023294, 2024-09-11T07:30:15.311 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:15.311 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:15.311 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:15.311 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:15.311 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:15.311 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 70635 2024-09-11T07:30:15.311 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.311 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:15.311 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.312 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:15.312 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.254841+0000", 2024-09-11T07:30:15.312 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.312 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.312 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.312 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:15.312 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.254841+0000", 2024-09-11T07:30:15.312 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.312 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.312 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.312 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:15.312 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.254839+0000", 2024-09-11T07:30:15.313 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:30:15.313 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.313 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.313 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:15.313 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.254868+0000", 2024-09-11T07:30:15.313 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.9034999999999999e-05 2024-09-11T07:30:15.313 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.313 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.313 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:15.313 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.254886+0000", 2024-09-11T07:30:15.313 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8400999999999999e-05 2024-09-11T07:30:15.313 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.313 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.314 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:15.314 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.254894+0000", 2024-09-11T07:30:15.314 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.729e-06 2024-09-11T07:30:15.314 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.314 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.314 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:15.314 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.256066+0000", 2024-09-11T07:30:15.314 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0011725570000000001 2024-09-11T07:30:15.314 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.314 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.314 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:15.314 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.256217+0000", 2024-09-11T07:30:15.314 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015086900000000001 2024-09-11T07:30:15.314 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.314 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.315 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:30:15.315 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.256395+0000", 2024-09-11T07:30:15.315 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000177932 2024-09-11T07:30:15.315 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.315 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.315 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:15.315 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.259349+0000", 2024-09-11T07:30:15.315 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0029538490000000001 2024-09-11T07:30:15.315 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.315 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.315 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.315 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.259384+0000", 2024-09-11T07:30:15.315 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.54e-05 2024-09-11T07:30:15.315 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.315 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.315 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.316 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.301780+0000", 2024-09-11T07:30:15.316 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.042395668999999997 2024-09-11T07:30:15.316 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.316 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.316 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:15.316 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.301827+0000", 2024-09-11T07:30:15.316 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7354000000000001e-05 2024-09-11T07:30:15.316 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.316 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.316 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:15.316 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.301864+0000", 2024-09-11T07:30:15.316 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6798000000000002e-05 2024-09-11T07:30:15.316 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.316 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:15.317 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.317 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.317 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.317 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:70639 7.e 7:7a2679ef:::benchmark_data_smithi022_77720_object70638:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-11T07:30:15.317 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:41.257890+0000", 2024-09-11T07:30:15.317 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 33.953210669999997, 2024-09-11T07:30:15.317 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.046066099999999999, 2024-09-11T07:30:15.317 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:15.317 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:15.317 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:15.317 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:15.317 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:15.317 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 70639 2024-09-11T07:30:15.317 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.317 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:15.317 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.318 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:15.318 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.257890+0000", 2024-09-11T07:30:15.318 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.318 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.318 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.318 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:15.318 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.257890+0000", 2024-09-11T07:30:15.318 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.318 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.318 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.318 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:15.318 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.257887+0000", 2024-09-11T07:30:15.318 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:30:15.318 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.318 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.319 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:15.319 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.257916+0000", 2024-09-11T07:30:15.319 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8229999999999999e-05 2024-09-11T07:30:15.319 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.319 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.319 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:15.319 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.257929+0000", 2024-09-11T07:30:15.319 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3662000000000001e-05 2024-09-11T07:30:15.319 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.319 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.319 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:15.319 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.257936+0000", 2024-09-11T07:30:15.319 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3770000000000001e-06 2024-09-11T07:30:15.319 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.319 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.320 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:15.320 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.258051+0000", 2024-09-11T07:30:15.320 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011555799999999999 2024-09-11T07:30:15.320 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.320 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.320 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:15.320 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.258135+0000", 2024-09-11T07:30:15.320 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.4172000000000007e-05 2024-09-11T07:30:15.320 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.320 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.320 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:30:15.320 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.258223+0000", 2024-09-11T07:30:15.320 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.7226999999999997e-05 2024-09-11T07:30:15.320 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.320 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.321 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:15.321 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.259454+0000", 2024-09-11T07:30:15.321 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0012313560000000001 2024-09-11T07:30:15.321 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.321 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.321 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.321 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.259636+0000", 2024-09-11T07:30:15.321 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00018206 2024-09-11T07:30:15.321 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.321 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.321 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.321 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.303879+0000", 2024-09-11T07:30:15.321 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.044242663000000002 2024-09-11T07:30:15.321 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.322 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.322 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:15.322 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.303922+0000", 2024-09-11T07:30:15.322 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3328999999999998e-05 2024-09-11T07:30:15.322 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.322 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.322 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:15.322 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.303956+0000", 2024-09-11T07:30:15.322 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3609999999999998e-05 2024-09-11T07:30:15.322 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.322 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:15.322 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.322 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.322 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.322 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:71230 7.e 7:7116ddd7:::benchmark_data_smithi022_77720_object71229:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-11T07:30:15.322 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:41.726367+0000", 2024-09-11T07:30:15.323 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 33.484732907999998, 2024-09-11T07:30:15.323 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.055176661000000002, 2024-09-11T07:30:15.323 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:15.323 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:15.323 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:15.323 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:15.323 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:15.323 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 71230 2024-09-11T07:30:15.323 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.323 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:15.323 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.323 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:15.323 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.726367+0000", 2024-09-11T07:30:15.323 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.323 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.324 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.324 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:15.324 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.726367+0000", 2024-09-11T07:30:15.324 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.324 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.324 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.324 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:15.324 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.726365+0000", 2024-09-11T07:30:15.324 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:30:15.324 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.324 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.324 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:15.324 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.726391+0000", 2024-09-11T07:30:15.324 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.6740000000000001e-05 2024-09-11T07:30:15.324 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.325 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.325 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:15.325 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.726404+0000", 2024-09-11T07:30:15.325 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2542e-05 2024-09-11T07:30:15.325 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.325 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.325 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:15.325 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.726410+0000", 2024-09-11T07:30:15.325 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7420000000000003e-06 2024-09-11T07:30:15.325 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.325 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.325 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:15.325 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.726458+0000", 2024-09-11T07:30:15.325 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7914999999999998e-05 2024-09-11T07:30:15.325 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.326 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.326 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:15.326 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.726574+0000", 2024-09-11T07:30:15.326 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000116234 2024-09-11T07:30:15.326 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.326 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.326 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:30:15.326 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.726846+0000", 2024-09-11T07:30:15.326 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00027205399999999998 2024-09-11T07:30:15.326 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.326 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.326 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.326 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.730111+0000", 2024-09-11T07:30:15.326 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0032648899999999999 2024-09-11T07:30:15.327 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.327 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.327 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:15.327 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.730122+0000", 2024-09-11T07:30:15.327 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1269e-05 2024-09-11T07:30:15.327 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.327 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.327 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.327 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.781442+0000", 2024-09-11T07:30:15.327 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.051319707999999999 2024-09-11T07:30:15.327 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.327 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.327 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:15.328 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.781491+0000", 2024-09-11T07:30:15.328 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9595999999999997e-05 2024-09-11T07:30:15.328 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.328 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.328 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:15.328 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:41.781544+0000", 2024-09-11T07:30:15.328 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2645000000000002e-05 2024-09-11T07:30:15.328 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.328 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:15.328 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.328 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.328 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.329 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:15.329 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:85610 7.e 7:745ba9ac:::benchmark_data_smithi022_77720_object85609:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-11T07:30:15.329 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:54.504708+0000", 2024-09-11T07:30:15.329 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 20.706391921000002, 2024-09-11T07:30:15.329 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.058539039000000001, 2024-09-11T07:30:15.329 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:15.330 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:15.330 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:15.330 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:15.330 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:15.330 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 85610 2024-09-11T07:30:15.330 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.330 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:15.330 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.330 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:15.330 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:54.504708+0000", 2024-09-11T07:30:15.330 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.330 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.331 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.331 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:15.331 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:54.504708+0000", 2024-09-11T07:30:15.331 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.331 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.331 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.331 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:15.331 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:54.504706+0000", 2024-09-11T07:30:15.331 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:30:15.331 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.331 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.331 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:15.331 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:54.504741+0000", 2024-09-11T07:30:15.332 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4748e-05 2024-09-11T07:30:15.332 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.332 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.332 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:15.332 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:54.504754+0000", 2024-09-11T07:30:15.332 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3499000000000001e-05 2024-09-11T07:30:15.332 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.332 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.332 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:15.332 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:54.504760+0000", 2024-09-11T07:30:15.332 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8459999999999996e-06 2024-09-11T07:30:15.332 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.332 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.332 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:15.333 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:54.505900+0000", 2024-09-11T07:30:15.333 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001139873 2024-09-11T07:30:15.333 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.333 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.333 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:15.333 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:54.506208+0000", 2024-09-11T07:30:15.333 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00030791800000000002 2024-09-11T07:30:15.333 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.333 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.333 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:30:15.333 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:54.506400+0000", 2024-09-11T07:30:15.333 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00019178399999999999 2024-09-11T07:30:15.333 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.333 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.333 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.334 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:54.509533+0000", 2024-09-11T07:30:15.334 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0031330170000000001 2024-09-11T07:30:15.334 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.334 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.334 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:15.334 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:54.509552+0000", 2024-09-11T07:30:15.334 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9758e-05 2024-09-11T07:30:15.334 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.334 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.334 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.334 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:54.562659+0000", 2024-09-11T07:30:15.334 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.053106484000000002 2024-09-11T07:30:15.334 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.334 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.335 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:15.335 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:54.563039+0000", 2024-09-11T07:30:15.335 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00038045499999999998 2024-09-11T07:30:15.335 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.335 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.335 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:15.335 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:54.563247+0000", 2024-09-11T07:30:15.335 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00020797400000000001 2024-09-11T07:30:15.335 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.335 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:15.335 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.335 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.335 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.335 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:105458 7.e 7:73affca5:::benchmark_data_smithi022_77720_object105457:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-11T07:30:15.336 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:13.514305+0000", 2024-09-11T07:30:15.336 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 1.696794916, 2024-09-11T07:30:15.336 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.043983746999999997, 2024-09-11T07:30:15.336 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:15.336 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:15.336 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:15.336 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:15.336 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:15.336 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 105458 2024-09-11T07:30:15.336 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.336 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:15.336 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.336 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:15.337 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:13.514305+0000", 2024-09-11T07:30:15.337 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.337 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.337 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.337 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:15.337 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:13.514305+0000", 2024-09-11T07:30:15.337 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:15.337 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.337 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.337 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:15.337 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:13.514304+0000", 2024-09-11T07:30:15.337 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:30:15.337 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.337 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.338 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:15.338 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:13.514329+0000", 2024-09-11T07:30:15.338 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5074000000000001e-05 2024-09-11T07:30:15.338 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.338 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.338 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:15.338 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:13.514340+0000", 2024-09-11T07:30:15.338 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1586000000000001e-05 2024-09-11T07:30:15.338 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.338 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.338 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:15.338 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:13.514345+0000", 2024-09-11T07:30:15.338 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6009999999999997e-06 2024-09-11T07:30:15.338 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.339 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.339 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:15.339 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:13.526110+0000", 2024-09-11T07:30:15.339 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.011765441 2024-09-11T07:30:15.339 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.339 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.339 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:15.339 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:13.526245+0000", 2024-09-11T07:30:15.339 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000134919 2024-09-11T07:30:15.339 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.339 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.339 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:30:15.339 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:13.526949+0000", 2024-09-11T07:30:15.339 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00070323599999999999 2024-09-11T07:30:15.339 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.340 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.340 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.340 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:13.536686+0000", 2024-09-11T07:30:15.340 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0097374019999999992 2024-09-11T07:30:15.340 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.340 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.340 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:15.340 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:13.536697+0000", 2024-09-11T07:30:15.340 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0652000000000001e-05 2024-09-11T07:30:15.340 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.340 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.340 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:15.340 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:13.558196+0000", 2024-09-11T07:30:15.340 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.021499626000000001 2024-09-11T07:30:15.340 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.341 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.341 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:15.341 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:13.558251+0000", 2024-09-11T07:30:15.341 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.4836999999999997e-05 2024-09-11T07:30:15.341 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:15.341 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:15.341 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:15.341 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:13.558289+0000", 2024-09-11T07:30:15.341 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7932999999999997e-05 2024-09-11T07:30:15.341 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.341 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:15.341 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.341 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:15.342 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:15.342 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:30:15.368 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:15.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:15.370 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:15.470 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:15.509 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:15.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:15.510 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:15.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:15 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:30:15.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:15 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:30:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:15 smithi104 ceph-mon[33507]: pgmap v144: 148 pgs: 1 peering, 147 active+clean; 6.3 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-11T07:30:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:15 smithi104 ceph-mon[33507]: 5.9 scrub starts 2024-09-11T07:30:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:15 smithi104 ceph-mon[33507]: 5.9 scrub ok 2024-09-11T07:30:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:15 smithi104 ceph-mon[33507]: osdmap e121: 8 total, 8 up, 8 in 2024-09-11T07:30:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:15 smithi104 ceph-mon[33507]: 3.0 scrub starts 2024-09-11T07:30:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:15 smithi104 ceph-mon[33507]: 3.0 scrub ok 2024-09-11T07:30:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:15 smithi104 ceph-mon[33507]: 7.d scrub starts 2024-09-11T07:30:15.594 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 73 16 107625 107609 92.1138 63.5625 0.017484 0.010851 2024-09-11T07:30:15.610 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:15.637 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:15.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:15.638 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:15.739 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:15.766 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:15.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:15.767 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:15.868 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:15.904 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:15.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:15.905 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:16.006 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:16.048 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:16.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:16.049 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:16.150 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:16.176 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:16.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:16.177 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:16.278 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:16.324 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:16.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:16.325 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:16.426 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:16.469 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:16.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:16.470 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:16.571 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:16.594 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 74 16 108616 108600 91.7055 61.9375 0.0104069 0.0108995 2024-09-11T07:30:16.612 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:16.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:16.613 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:16.714 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:16.738 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:16.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:16.739 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:16.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:16 smithi104 ceph-mon[33507]: 4.1e scrub starts 2024-09-11T07:30:16.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:16 smithi104 ceph-mon[33507]: 4.1e scrub ok 2024-09-11T07:30:16.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:16 smithi104 ceph-mon[33507]: pgmap v146: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 741 B/s rd, 48 MiB/s wr, 763 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:30:16.840 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:16.877 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:16.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:16.878 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:16.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:16 smithi022 ceph-mon[29462]: 4.1e scrub starts 2024-09-11T07:30:16.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:16 smithi022 ceph-mon[29462]: 4.1e scrub ok 2024-09-11T07:30:16.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:16 smithi022 ceph-mon[29462]: pgmap v146: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 741 B/s rd, 48 MiB/s wr, 763 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:30:16.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:16 smithi022 ceph-mon[37330]: 4.1e scrub starts 2024-09-11T07:30:16.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:16 smithi022 ceph-mon[37330]: 4.1e scrub ok 2024-09-11T07:30:16.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:16 smithi022 ceph-mon[37330]: pgmap v146: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 741 B/s rd, 48 MiB/s wr, 763 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:30:16.979 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:17.002 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T07:30:17.007 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:17.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:17.007 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:17.108 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:17.151 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:17.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:17.152 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:17.253 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:17.277 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:17.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:17.278 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:17.379 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:17.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:17.403 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:17.403 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:17.504 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:17.527 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:17.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:17.528 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:17.594 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 75 16 109561 109545 91.2702 59.0625 0.024028 0.0109517 2024-09-11T07:30:17.629 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:17.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:17.680 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:17.680 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:17.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:17 smithi104 ceph-mon[33507]: 4.18 scrub starts 2024-09-11T07:30:17.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:17 smithi104 ceph-mon[33507]: 4.18 scrub ok 2024-09-11T07:30:17.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:17 smithi104 ceph-mon[33507]: 2.8 scrub starts 2024-09-11T07:30:17.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:17 smithi104 ceph-mon[33507]: 2.8 scrub ok 2024-09-11T07:30:17.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:17 smithi104 ceph-mon[33507]: 2.a scrub starts 2024-09-11T07:30:17.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:17 smithi104 ceph-mon[33507]: 2.a scrub ok 2024-09-11T07:30:17.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:17 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:30:17.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:17 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:30:17.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:17 smithi104 ceph-mon[33507]: 6.18 scrub starts 2024-09-11T07:30:17.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:17 smithi104 ceph-mon[33507]: 6.18 scrub ok 2024-09-11T07:30:17.782 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:17.828 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:17.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:17.829 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:17 smithi022 ceph-mon[29462]: 4.18 scrub starts 2024-09-11T07:30:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:17 smithi022 ceph-mon[29462]: 4.18 scrub ok 2024-09-11T07:30:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:17 smithi022 ceph-mon[29462]: 2.8 scrub starts 2024-09-11T07:30:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:17 smithi022 ceph-mon[29462]: 2.8 scrub ok 2024-09-11T07:30:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:17 smithi022 ceph-mon[29462]: 2.a scrub starts 2024-09-11T07:30:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:17 smithi022 ceph-mon[29462]: 2.a scrub ok 2024-09-11T07:30:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:17 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:30:17.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:17 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:30:17.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:17 smithi022 ceph-mon[29462]: 6.18 scrub starts 2024-09-11T07:30:17.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:17 smithi022 ceph-mon[29462]: 6.18 scrub ok 2024-09-11T07:30:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:17 smithi022 ceph-mon[37330]: 4.18 scrub starts 2024-09-11T07:30:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:17 smithi022 ceph-mon[37330]: 4.18 scrub ok 2024-09-11T07:30:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:17 smithi022 ceph-mon[37330]: 2.8 scrub starts 2024-09-11T07:30:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:17 smithi022 ceph-mon[37330]: 2.8 scrub ok 2024-09-11T07:30:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:17 smithi022 ceph-mon[37330]: 2.a scrub starts 2024-09-11T07:30:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:17 smithi022 ceph-mon[37330]: 2.a scrub ok 2024-09-11T07:30:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:17 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:30:17.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:17 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:30:17.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:17 smithi022 ceph-mon[37330]: 6.18 scrub starts 2024-09-11T07:30:17.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:17 smithi022 ceph-mon[37330]: 6.18 scrub ok 2024-09-11T07:30:17.930 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:17.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:17.970 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:17.970 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:18.048 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-11T07:30:18.072 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:18.140 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:18.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:18.141 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:18.242 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:18.291 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:18.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:18.292 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:18.393 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:18.419 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:18.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:18.420 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:18.521 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:18.548 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:18.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:18.549 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:18.594 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 76 16 110591 110575 90.9163 64.375 0.0110523 0.0109935 2024-09-11T07:30:18.627 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:18 smithi022 ceph-mon[29462]: 4.1 scrub starts 2024-09-11T07:30:18.627 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:18 smithi022 ceph-mon[29462]: 4.1 scrub ok 2024-09-11T07:30:18.627 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:18 smithi022 ceph-mon[29462]: 7.d scrub ok 2024-09-11T07:30:18.627 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:18 smithi022 ceph-mon[29462]: 4.f scrub starts 2024-09-11T07:30:18.627 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:18 smithi022 ceph-mon[29462]: 4.f scrub ok 2024-09-11T07:30:18.627 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:18 smithi022 ceph-mon[29462]: osdmap e122: 8 total, 8 up, 8 in 2024-09-11T07:30:18.627 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:18 smithi022 ceph-mon[29462]: 5.b scrub starts 2024-09-11T07:30:18.627 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:18 smithi022 ceph-mon[29462]: 5.b scrub ok 2024-09-11T07:30:18.628 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:18 smithi022 ceph-mon[29462]: pgmap v148: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 55 MiB/s wr, 878 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:30:18.628 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:18 smithi022 ceph-mon[37330]: 4.1 scrub starts 2024-09-11T07:30:18.628 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:18 smithi022 ceph-mon[37330]: 4.1 scrub ok 2024-09-11T07:30:18.628 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:18 smithi022 ceph-mon[37330]: 7.d scrub ok 2024-09-11T07:30:18.628 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:18 smithi022 ceph-mon[37330]: 4.f scrub starts 2024-09-11T07:30:18.628 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:18 smithi022 ceph-mon[37330]: 4.f scrub ok 2024-09-11T07:30:18.628 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:18 smithi022 ceph-mon[37330]: osdmap e122: 8 total, 8 up, 8 in 2024-09-11T07:30:18.628 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:18 smithi022 ceph-mon[37330]: 5.b scrub starts 2024-09-11T07:30:18.628 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:18 smithi022 ceph-mon[37330]: 5.b scrub ok 2024-09-11T07:30:18.628 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:18 smithi022 ceph-mon[37330]: pgmap v148: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 55 MiB/s wr, 878 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:30:18.636 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:18.649 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:18.715 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:18.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:18.716 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:18 smithi104 ceph-mon[33507]: 4.1 scrub starts 2024-09-11T07:30:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:18 smithi104 ceph-mon[33507]: 4.1 scrub ok 2024-09-11T07:30:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:18 smithi104 ceph-mon[33507]: 7.d scrub ok 2024-09-11T07:30:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:18 smithi104 ceph-mon[33507]: 4.f scrub starts 2024-09-11T07:30:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:18 smithi104 ceph-mon[33507]: 4.f scrub ok 2024-09-11T07:30:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:18 smithi104 ceph-mon[33507]: osdmap e122: 8 total, 8 up, 8 in 2024-09-11T07:30:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:18 smithi104 ceph-mon[33507]: 5.b scrub starts 2024-09-11T07:30:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:18 smithi104 ceph-mon[33507]: 5.b scrub ok 2024-09-11T07:30:18.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:18 smithi104 ceph-mon[33507]: pgmap v148: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 55 MiB/s wr, 878 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:30:18.817 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:18.876 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:18.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:18.879 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:18.979 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:19.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:19.030 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:19.030 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:19.044 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:30:19.044 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:30:19.044 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 122, 2024-09-11T07:30:19.044 INFO:teuthology.orchestra.run.smithi022.stdout: "fsid": "12a285d8-700e-11ef-bcea-c7b262605968", 2024-09-11T07:30:19.044 INFO:teuthology.orchestra.run.smithi022.stdout: "created": "2024-09-11T07:21:18.695332+0000", 2024-09-11T07:30:19.044 INFO:teuthology.orchestra.run.smithi022.stdout: "modified": "2024-09-11T07:30:17.478738+0000", 2024-09-11T07:30:19.044 INFO:teuthology.orchestra.run.smithi022.stdout: "last_up_change": "2024-09-11T07:26:15.385688+0000", 2024-09-11T07:30:19.044 INFO:teuthology.orchestra.run.smithi022.stdout: "last_in_change": "2024-09-11T07:25:58.551813+0000", 2024-09-11T07:30:19.044 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-11T07:30:19.045 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_num": 5804032, 2024-09-11T07:30:19.045 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_set": [ 2024-09-11T07:30:19.045 INFO:teuthology.orchestra.run.smithi022.stdout: "nodeep-scrub", 2024-09-11T07:30:19.045 INFO:teuthology.orchestra.run.smithi022.stdout: "pglog_hardlimit", 2024-09-11T07:30:19.045 INFO:teuthology.orchestra.run.smithi022.stdout: "purged_snapdirs", 2024-09-11T07:30:19.045 INFO:teuthology.orchestra.run.smithi022.stdout: "recovery_deletes", 2024-09-11T07:30:19.045 INFO:teuthology.orchestra.run.smithi022.stdout: "sortbitwise" 2024-09-11T07:30:19.045 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:30:19.045 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_version": 18, 2024-09-11T07:30:19.045 INFO:teuthology.orchestra.run.smithi022.stdout: "full_ratio": 0.94999998807907104, 2024-09-11T07:30:19.045 INFO:teuthology.orchestra.run.smithi022.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-11T07:30:19.045 INFO:teuthology.orchestra.run.smithi022.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-11T07:30:19.045 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_snapshot": "", 2024-09-11T07:30:19.046 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_max": 7, 2024-09-11T07:30:19.046 INFO:teuthology.orchestra.run.smithi022.stdout: "max_osd": 8, 2024-09-11T07:30:19.046 INFO:teuthology.orchestra.run.smithi022.stdout: "require_min_compat_client": "luminous", 2024-09-11T07:30:19.046 INFO:teuthology.orchestra.run.smithi022.stdout: "min_compat_client": "luminous", 2024-09-11T07:30:19.046 INFO:teuthology.orchestra.run.smithi022.stdout: "require_osd_release": "quincy", 2024-09-11T07:30:19.046 INFO:teuthology.orchestra.run.smithi022.stdout: "pools": [ 2024-09-11T07:30:19.046 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.046 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 1, 2024-09-11T07:30:19.046 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": ".mgr", 2024-09-11T07:30:19.046 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:24:21.772426+0000", 2024-09-11T07:30:19.046 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:30:19.046 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:30:19.046 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:30:19.046 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:30:19.047 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:30:19.047 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:30:19.047 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:30:19.047 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:30:19.047 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:30:19.047 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:30:19.047 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:30:19.047 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:30:19.047 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 1, 2024-09-11T07:30:19.047 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 1, 2024-09-11T07:30:19.047 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 1, 2024-09-11T07:30:19.047 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 1, 2024-09-11T07:30:19.047 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 1, 2024-09-11T07:30:19.048 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:30:19.048 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:30:19.048 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:30:19.048 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:30:19.048 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:30:19.048 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:30:19.048 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:30:19.048 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.048 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "21", 2024-09-11T07:30:19.048 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:30:19.048 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:30:19.048 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:30:19.048 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:30:19.049 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:30:19.049 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:30:19.049 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:30:19.049 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:30:19.049 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:30:19.049 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:30:19.049 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:30:19.049 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:30:19.049 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:30:19.049 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:30:19.049 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:30:19.049 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:30:19.049 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:30:19.050 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:30:19.050 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:30:19.050 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:30:19.050 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:30:19.050 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:30:19.050 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:30:19.050 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:30:19.050 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:30:19.050 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:30:19.050 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.050 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:30:19.050 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:30:19.050 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:30:19.051 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:30:19.051 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:30:19.051 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:30:19.051 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:30:19.051 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:30:19.051 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:30:19.051 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:30:19.051 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:30:19.051 INFO:teuthology.orchestra.run.smithi022.stdout: "options": { 2024-09-11T07:30:19.051 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_max": 32, 2024-09-11T07:30:19.051 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_min": 1 2024-09-11T07:30:19.051 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.051 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:30:19.051 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": {} 2024-09-11T07:30:19.052 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.052 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.052 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.052 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 2, 2024-09-11T07:30:19.052 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": ".rgw.root", 2024-09-11T07:30:19.052 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:29.941354+0000", 2024-09-11T07:30:19.052 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:30:19.052 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:30:19.052 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:30:19.052 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:30:19.052 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:30:19.052 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:30:19.052 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:30:19.053 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:30:19.053 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:30:19.053 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:30:19.053 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:30:19.053 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:30:19.053 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:30:19.053 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:30:19.053 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:30:19.053 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:30:19.053 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:30:19.053 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:30:19.053 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:30:19.053 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:30:19.054 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:30:19.054 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:30:19.054 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:30:19.054 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:30:19.054 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.054 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "58", 2024-09-11T07:30:19.054 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:30:19.054 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:30:19.054 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:30:19.054 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:30:19.054 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:30:19.054 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:30:19.054 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:30:19.055 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:30:19.055 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:30:19.055 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:30:19.055 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:30:19.055 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:30:19.055 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:30:19.055 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:30:19.055 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:30:19.055 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:30:19.055 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:30:19.055 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:30:19.055 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:30:19.055 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:30:19.056 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:30:19.056 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:30:19.056 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:30:19.056 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:30:19.056 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:30:19.056 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:30:19.056 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.056 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:30:19.056 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:30:19.056 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:30:19.056 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:30:19.056 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:30:19.056 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:30:19.057 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:30:19.057 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:30:19.057 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:30:19.057 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:30:19.057 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:30:19.057 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:30:19.057 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:30:19.057 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:30:19.057 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.057 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.057 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.057 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 3, 2024-09-11T07:30:19.057 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "datapool", 2024-09-11T07:30:19.058 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:30.396859+0000", 2024-09-11T07:30:19.058 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 8193, 2024-09-11T07:30:19.058 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T07:30:19.058 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:30:19.058 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:30:19.058 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:30:19.058 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:30:19.058 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:30:19.058 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:30:19.058 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:30:19.058 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:30:19.058 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:30:19.059 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:30:19.059 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 3, 2024-09-11T07:30:19.059 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 3, 2024-09-11T07:30:19.059 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 3, 2024-09-11T07:30:19.059 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 3, 2024-09-11T07:30:19.059 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 3, 2024-09-11T07:30:19.059 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:30:19.059 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:30:19.059 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:30:19.060 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:30:19.060 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:30:19.060 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:30:19.060 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:30:19.060 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.060 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "61", 2024-09-11T07:30:19.060 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:30:19.060 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:30:19.060 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:30:19.060 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:30:19.060 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:30:19.060 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 3, 2024-09-11T07:30:19.060 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 61, 2024-09-11T07:30:19.061 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:30:19.061 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:30:19.061 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:30:19.061 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:30:19.061 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:30:19.061 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:30:19.061 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:30:19.061 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:30:19.061 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:30:19.061 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:30:19.061 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:30:19.061 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:30:19.061 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:30:19.062 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:30:19.062 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:30:19.062 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:30:19.062 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:30:19.062 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:30:19.062 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:30:19.062 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.062 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:30:19.062 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:30:19.062 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:30:19.062 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:30:19.062 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:30:19.062 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:30:19.063 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:30:19.063 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:30:19.063 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:30:19.063 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:30:19.063 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:30:19.063 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:30:19.063 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:30:19.063 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd": {} 2024-09-11T07:30:19.063 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.063 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.063 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.063 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 4, 2024-09-11T07:30:19.063 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.log", 2024-09-11T07:30:19.064 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:31.922926+0000", 2024-09-11T07:30:19.064 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:30:19.064 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:30:19.064 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:30:19.064 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:30:19.064 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:30:19.064 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:30:19.064 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:30:19.064 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:30:19.064 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:30:19.064 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:30:19.064 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:30:19.064 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:30:19.065 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:30:19.065 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:30:19.065 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:30:19.065 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:30:19.065 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:30:19.065 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:30:19.065 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:30:19.065 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:30:19.065 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:30:19.065 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:30:19.065 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:30:19.065 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:30:19.065 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.066 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "60", 2024-09-11T07:30:19.066 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:30:19.066 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:30:19.066 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:30:19.066 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:30:19.066 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:30:19.066 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:30:19.066 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:30:19.066 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:30:19.066 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:30:19.066 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:30:19.066 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:30:19.066 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:30:19.067 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:30:19.067 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:30:19.067 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:30:19.067 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:30:19.067 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:30:19.067 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:30:19.067 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:30:19.067 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:30:19.067 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:30:19.067 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:30:19.067 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:30:19.067 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:30:19.067 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:30:19.068 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:30:19.068 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.068 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:30:19.068 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:30:19.068 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:30:19.068 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:30:19.068 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:30:19.068 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:30:19.068 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:30:19.068 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:30:19.068 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:30:19.068 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:30:19.068 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:30:19.068 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:30:19.069 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:30:19.069 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:30:19.069 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.069 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.069 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.069 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 5, 2024-09-11T07:30:19.069 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.control", 2024-09-11T07:30:19.069 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:34.007746+0000", 2024-09-11T07:30:19.069 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:30:19.069 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:30:19.069 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:30:19.069 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:30:19.070 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:30:19.070 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:30:19.070 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:30:19.070 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:30:19.070 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:30:19.070 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:30:19.070 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:30:19.070 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:30:19.070 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:30:19.070 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:30:19.070 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:30:19.070 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:30:19.070 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:30:19.071 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:30:19.071 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:30:19.071 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:30:19.071 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:30:19.071 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:30:19.071 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:30:19.071 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:30:19.071 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.071 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "62", 2024-09-11T07:30:19.071 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:30:19.071 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:30:19.071 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:30:19.071 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:30:19.071 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:30:19.072 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:30:19.072 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:30:19.072 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:30:19.072 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:30:19.072 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:30:19.072 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:30:19.072 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:30:19.072 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:30:19.072 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:30:19.072 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:30:19.072 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:30:19.072 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:30:19.072 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:30:19.073 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:30:19.073 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:30:19.073 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:30:19.073 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:30:19.073 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:30:19.073 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:30:19.073 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:30:19.073 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:30:19.073 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.073 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:30:19.073 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:30:19.073 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:30:19.073 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:30:19.074 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:30:19.074 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:30:19.074 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:30:19.074 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:30:19.074 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:30:19.074 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:30:19.074 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:30:19.074 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:30:19.074 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:30:19.074 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:30:19.074 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.074 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.074 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.074 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 6, 2024-09-11T07:30:19.075 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.meta", 2024-09-11T07:30:19.075 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:36.077215+0000", 2024-09-11T07:30:19.075 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:30:19.075 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:30:19.075 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:30:19.075 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:30:19.075 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:30:19.075 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:30:19.075 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:30:19.075 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:30:19.075 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:30:19.075 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:30:19.075 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:30:19.076 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:30:19.076 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:30:19.076 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:30:19.076 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:30:19.076 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:30:19.076 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:30:19.076 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:30:19.076 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:30:19.076 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:30:19.076 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:30:19.076 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:30:19.076 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:30:19.076 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:30:19.076 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.077 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "64", 2024-09-11T07:30:19.077 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:30:19.077 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:30:19.077 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:30:19.077 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:30:19.077 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:30:19.077 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:30:19.077 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:30:19.077 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:30:19.077 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:30:19.077 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:30:19.077 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:30:19.077 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:30:19.078 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:30:19.078 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:30:19.078 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:30:19.078 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:30:19.078 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:30:19.078 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:30:19.078 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:30:19.078 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:30:19.079 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:30:19.079 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:30:19.079 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:30:19.079 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:30:19.079 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:30:19.079 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:30:19.079 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.079 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:30:19.079 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:30:19.079 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:30:19.079 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:30:19.079 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:30:19.080 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:30:19.080 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:30:19.080 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:30:19.080 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:30:19.080 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:30:19.080 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:30:19.080 INFO:teuthology.orchestra.run.smithi022.stdout: "options": { 2024-09-11T07:30:19.080 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_bias": 4 2024-09-11T07:30:19.081 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.081 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:30:19.081 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:30:19.081 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.081 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.081 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.081 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 7, 2024-09-11T07:30:19.082 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "unique_pool_0", 2024-09-11T07:30:19.082 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:28:48.880975+0000", 2024-09-11T07:30:19.082 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:30:19.082 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:30:19.082 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:30:19.082 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:30:19.082 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:30:19.082 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:30:19.082 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:30:19.082 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:30:19.082 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:30:19.083 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:30:19.083 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:30:19.083 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:30:19.083 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 16, 2024-09-11T07:30:19.083 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 16, 2024-09-11T07:30:19.083 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 16, 2024-09-11T07:30:19.083 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 16, 2024-09-11T07:30:19.083 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 16, 2024-09-11T07:30:19.083 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:30:19.083 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:30:19.083 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:30:19.084 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:30:19.084 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:30:19.084 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:30:19.084 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:30:19.084 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.084 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "96", 2024-09-11T07:30:19.084 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:30:19.084 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:30:19.084 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:30:19.084 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:30:19.084 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:30:19.085 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:30:19.085 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:30:19.085 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:30:19.085 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:30:19.085 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:30:19.085 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:30:19.085 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:30:19.085 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:30:19.085 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:30:19.085 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:30:19.086 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:30:19.086 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:30:19.086 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:30:19.086 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:30:19.086 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:30:19.086 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:30:19.086 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:30:19.086 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:30:19.086 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:30:19.086 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:30:19.086 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:30:19.087 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.087 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:30:19.087 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:30:19.087 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:30:19.087 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:30:19.087 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:30:19.087 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:30:19.087 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:30:19.087 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:30:19.087 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:30:19.087 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:30:19.088 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:30:19.088 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:30:19.088 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:30:19.088 INFO:teuthology.orchestra.run.smithi022.stdout: "rados": {} 2024-09-11T07:30:19.088 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.088 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.088 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:30:19.088 INFO:teuthology.orchestra.run.smithi022.stdout: "osds": [ 2024-09-11T07:30:19.088 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.088 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 0, 2024-09-11T07:30:19.088 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "09075f16-32f4-4f88-a116-c3d7f977a0e0", 2024-09-11T07:30:19.089 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:30:19.089 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:30:19.089 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:30:19.089 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:30:19.089 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:30:19.089 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:30:19.089 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 8, 2024-09-11T07:30:19.089 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 92, 2024-09-11T07:30:19.089 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:30:19.089 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:30:19.089 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:30:19.090 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.090 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.090 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.090 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6801", 2024-09-11T07:30:19.090 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:30:19.090 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.090 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.090 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.090 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:30:19.090 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.090 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.090 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.091 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6802", 2024-09-11T07:30:19.091 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:30:19.091 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.091 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.091 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.091 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:30:19.091 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.091 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.091 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.091 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6804", 2024-09-11T07:30:19.091 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:30:19.092 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.092 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.092 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.092 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:30:19.092 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.092 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.092 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.092 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6803", 2024-09-11T07:30:19.092 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:30:19.092 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.093 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.093 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.093 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6801/2821493591", 2024-09-11T07:30:19.093 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6802/2821493591", 2024-09-11T07:30:19.093 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6804/2821493591", 2024-09-11T07:30:19.093 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6803/2821493591", 2024-09-11T07:30:19.093 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:30:19.093 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:30:19.093 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:30:19.093 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.093 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.094 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.094 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 1, 2024-09-11T07:30:19.094 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "52cc1e06-9506-48c0-9f7b-d4270dff4b9f", 2024-09-11T07:30:19.094 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:30:19.094 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:30:19.094 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:30:19.094 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:30:19.094 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:30:19.094 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:30:19.095 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 13, 2024-09-11T07:30:19.095 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 92, 2024-09-11T07:30:19.095 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:30:19.095 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:30:19.095 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:30:19.095 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.095 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.095 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.095 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6805", 2024-09-11T07:30:19.095 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:30:19.095 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.096 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.096 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.096 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:30:19.096 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.096 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.096 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.096 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6806", 2024-09-11T07:30:19.096 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:30:19.096 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.096 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.096 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.096 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:30:19.097 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.097 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.097 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.097 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6808", 2024-09-11T07:30:19.097 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:30:19.097 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.097 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.097 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.097 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:30:19.097 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.097 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.098 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.098 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6807", 2024-09-11T07:30:19.098 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:30:19.098 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.098 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.098 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.098 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6805/4037911317", 2024-09-11T07:30:19.098 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6806/4037911317", 2024-09-11T07:30:19.098 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6808/4037911317", 2024-09-11T07:30:19.098 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6807/4037911317", 2024-09-11T07:30:19.099 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:30:19.099 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:30:19.099 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:30:19.099 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.099 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.099 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.099 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 2, 2024-09-11T07:30:19.099 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "fa2b2730-2c1d-45d0-be8a-494a60dc6b9b", 2024-09-11T07:30:19.099 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:30:19.099 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:30:19.099 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:30:19.099 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:30:19.100 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:30:19.100 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:30:19.100 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 18, 2024-09-11T07:30:19.100 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 92, 2024-09-11T07:30:19.100 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:30:19.100 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:30:19.100 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:30:19.100 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.100 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.100 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.101 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6809", 2024-09-11T07:30:19.101 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:30:19.101 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.101 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.101 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.101 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:30:19.101 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.101 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.101 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.101 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6810", 2024-09-11T07:30:19.101 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:30:19.101 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.102 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.102 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.102 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:30:19.102 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.102 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.102 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.102 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6812", 2024-09-11T07:30:19.102 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:30:19.102 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.102 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.102 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.103 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:30:19.103 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.103 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.103 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.103 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6811", 2024-09-11T07:30:19.103 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:30:19.103 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.103 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.103 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.103 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6809/1675799666", 2024-09-11T07:30:19.103 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6810/1675799666", 2024-09-11T07:30:19.103 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6812/1675799666", 2024-09-11T07:30:19.103 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6811/1675799666", 2024-09-11T07:30:19.103 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:30:19.104 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:30:19.104 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:30:19.104 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.104 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.104 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.104 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 3, 2024-09-11T07:30:19.104 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "8d53d3fc-e401-47df-b70b-4b4b1f4539df", 2024-09-11T07:30:19.104 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:30:19.104 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:30:19.104 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:30:19.104 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:30:19.105 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:30:19.105 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:30:19.105 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 25, 2024-09-11T07:30:19.105 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 118, 2024-09-11T07:30:19.105 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:30:19.105 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:30:19.105 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:30:19.105 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.105 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.105 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.105 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6813", 2024-09-11T07:30:19.105 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:30:19.106 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.106 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.106 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.106 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:30:19.106 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.106 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.106 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.106 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6814", 2024-09-11T07:30:19.106 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:30:19.106 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.106 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.106 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.107 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:30:19.107 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.107 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.107 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.107 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6816", 2024-09-11T07:30:19.107 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:30:19.107 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.107 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.108 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.108 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:30:19.108 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.108 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.108 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.108 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6815", 2024-09-11T07:30:19.108 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:30:19.108 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.108 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.108 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.108 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6813/1501962752", 2024-09-11T07:30:19.109 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6814/1501962752", 2024-09-11T07:30:19.109 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6816/1501962752", 2024-09-11T07:30:19.109 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6815/1501962752", 2024-09-11T07:30:19.109 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:30:19.109 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:30:19.109 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:30:19.109 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.109 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.109 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.109 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 4, 2024-09-11T07:30:19.110 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "d3207fe0-8b97-4067-a755-ba6d449c53c7", 2024-09-11T07:30:19.110 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:30:19.110 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:30:19.110 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:30:19.110 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:30:19.110 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:30:19.110 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:30:19.110 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 31, 2024-09-11T07:30:19.110 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 113, 2024-09-11T07:30:19.110 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:30:19.110 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:30:19.110 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:30:19.111 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.111 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.111 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.111 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6800", 2024-09-11T07:30:19.111 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:30:19.111 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.111 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.111 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.111 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:30:19.111 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.111 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.112 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.112 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6801", 2024-09-11T07:30:19.112 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:30:19.112 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.112 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.112 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.112 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:30:19.112 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.112 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.112 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.112 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6803", 2024-09-11T07:30:19.113 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:30:19.113 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.113 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.113 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.113 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:30:19.113 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.113 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.113 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.113 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6802", 2024-09-11T07:30:19.113 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:30:19.113 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.114 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.114 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.114 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6800/1210229443", 2024-09-11T07:30:19.114 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6801/1210229443", 2024-09-11T07:30:19.114 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6803/1210229443", 2024-09-11T07:30:19.114 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6802/1210229443", 2024-09-11T07:30:19.114 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:30:19.114 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:30:19.114 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:30:19.114 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.114 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.114 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.115 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 5, 2024-09-11T07:30:19.115 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "89c9352b-30b2-48c6-869e-047d07a23a43", 2024-09-11T07:30:19.115 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:30:19.115 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:30:19.115 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:30:19.115 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:30:19.115 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:30:19.115 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:30:19.115 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 37, 2024-09-11T07:30:19.115 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 100, 2024-09-11T07:30:19.115 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:30:19.115 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:30:19.116 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:30:19.116 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.116 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.116 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.116 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6804", 2024-09-11T07:30:19.116 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:30:19.116 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.116 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.116 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.116 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:30:19.116 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.116 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.117 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.117 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6805", 2024-09-11T07:30:19.117 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:30:19.117 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.117 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.117 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.117 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:30:19.117 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.117 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.117 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.117 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6807", 2024-09-11T07:30:19.118 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:30:19.118 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.118 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.118 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.118 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:30:19.118 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.118 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.118 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.118 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6806", 2024-09-11T07:30:19.118 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:30:19.118 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.118 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.119 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.119 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6804/2735328732", 2024-09-11T07:30:19.119 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6805/2735328732", 2024-09-11T07:30:19.119 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6807/2735328732", 2024-09-11T07:30:19.119 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6806/2735328732", 2024-09-11T07:30:19.119 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:30:19.119 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:30:19.119 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:30:19.119 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.119 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.119 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.119 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 6, 2024-09-11T07:30:19.119 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "704563de-d88f-480a-b469-720135704d42", 2024-09-11T07:30:19.120 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:30:19.120 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:30:19.120 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:30:19.120 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:30:19.120 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:30:19.120 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:30:19.120 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 44, 2024-09-11T07:30:19.120 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 92, 2024-09-11T07:30:19.120 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:30:19.120 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:30:19.120 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:30:19.120 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.121 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.121 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.121 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6808", 2024-09-11T07:30:19.121 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:30:19.121 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.121 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.121 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.121 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:30:19.121 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.121 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.122 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.122 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6809", 2024-09-11T07:30:19.122 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:30:19.122 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.122 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.122 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.122 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:30:19.122 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.122 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.122 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.122 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6811", 2024-09-11T07:30:19.122 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:30:19.123 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.123 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.123 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.123 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:30:19.123 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.123 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.123 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.123 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6810", 2024-09-11T07:30:19.123 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:30:19.123 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.123 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.123 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.124 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6808/721493010", 2024-09-11T07:30:19.124 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6809/721493010", 2024-09-11T07:30:19.124 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6811/721493010", 2024-09-11T07:30:19.124 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6810/721493010", 2024-09-11T07:30:19.124 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:30:19.124 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:30:19.124 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:30:19.124 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.124 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.124 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.124 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 7, 2024-09-11T07:30:19.124 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "18199e47-0052-49f0-b45f-0fb201e0d0f4", 2024-09-11T07:30:19.125 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:30:19.125 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:30:19.125 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:30:19.125 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:30:19.125 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:30:19.125 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:30:19.125 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 52, 2024-09-11T07:30:19.125 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 92, 2024-09-11T07:30:19.125 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:30:19.125 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:30:19.125 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:30:19.126 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.126 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.126 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.126 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6812", 2024-09-11T07:30:19.126 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3018423954 2024-09-11T07:30:19.126 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.126 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.126 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.126 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:30:19.126 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.126 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.126 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.126 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6813", 2024-09-11T07:30:19.127 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3018423954 2024-09-11T07:30:19.127 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.127 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.127 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.127 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:30:19.128 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.128 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.128 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.128 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6815", 2024-09-11T07:30:19.128 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3018423954 2024-09-11T07:30:19.128 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.128 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.128 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.128 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:30:19.128 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:30:19.128 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.129 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:30:19.129 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6814", 2024-09-11T07:30:19.129 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3018423954 2024-09-11T07:30:19.129 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.129 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.129 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.129 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6812/3018423954", 2024-09-11T07:30:19.129 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6813/3018423954", 2024-09-11T07:30:19.129 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6815/3018423954", 2024-09-11T07:30:19.129 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6814/3018423954", 2024-09-11T07:30:19.129 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:30:19.129 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:30:19.130 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:30:19.130 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.130 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.130 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:30:19.130 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_xinfo": [ 2024-09-11T07:30:19.130 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.130 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 0, 2024-09-11T07:30:19.130 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:30:19.130 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:30:19.130 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:30:19.130 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:30:19.131 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:30:19.131 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:30:16.531847+0000", 2024-09-11T07:30:19.131 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:30:19.131 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.131 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.131 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 1, 2024-09-11T07:30:19.131 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:30:19.131 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:30:19.131 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:30:19.131 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:30:19.131 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:30:19.132 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:30:07.299072+0000", 2024-09-11T07:30:19.132 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:30:19.132 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.132 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.132 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 2, 2024-09-11T07:30:19.132 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:30:19.132 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:30:19.132 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:30:19.132 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:30:19.132 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:30:19.132 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:30:13.516466+0000", 2024-09-11T07:30:19.133 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:30:19.133 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.133 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.133 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 3, 2024-09-11T07:30:19.133 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:30:19.133 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:30:19.133 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:30:19.133 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:30:19.133 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:30:19.133 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:29:57.588987+0000", 2024-09-11T07:30:19.134 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:30:19.134 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.134 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.134 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 4, 2024-09-11T07:30:19.134 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:30:19.134 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:30:19.134 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:30:19.134 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:30:19.134 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:30:19.134 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:29:40.758972+0000", 2024-09-11T07:30:19.134 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:30:19.134 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.135 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.135 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 5, 2024-09-11T07:30:19.135 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:30:19.135 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:30:19.135 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:30:19.135 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:30:19.135 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:30:19.135 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:29:10.678249+0000", 2024-09-11T07:30:19.135 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:30:19.135 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.135 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.135 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 6, 2024-09-11T07:30:19.136 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:30:19.136 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:30:19.136 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:30:19.136 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:30:19.136 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:30:19.136 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:29:46.707244+0000", 2024-09-11T07:30:19.136 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:30:19.136 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.136 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.136 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 7, 2024-09-11T07:30:19.136 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:30:19.137 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:30:19.137 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:30:19.137 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:30:19.137 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:30:19.137 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:29:37.516403+0000", 2024-09-11T07:30:19.137 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:30:19.137 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.137 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:30:19.137 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_upmap": [ 2024-09-11T07:30:19.137 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "2.1c", 2024-09-11T07:30:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: "osds": [ 2024-09-11T07:30:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: 5, 2024-09-11T07:30:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: 2, 2024-09-11T07:30:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: 7 2024-09-11T07:30:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:30:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_upmap_items": [ 2024-09-11T07:30:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "2.7", 2024-09-11T07:30:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:30:19.139 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.139 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 7, 2024-09-11T07:30:19.139 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 4 2024-09-11T07:30:19.139 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.139 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.139 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.139 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.139 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.6", 2024-09-11T07:30:19.139 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:30:19.139 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.139 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 1, 2024-09-11T07:30:19.139 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 5 2024-09-11T07:30:19.140 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.140 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.140 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.140 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.140 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.14", 2024-09-11T07:30:19.140 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:30:19.140 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.140 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 1, 2024-09-11T07:30:19.140 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 2 2024-09-11T07:30:19.140 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.140 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.1f", 2024-09-11T07:30:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:30:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 1, 2024-09-11T07:30:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 2 2024-09-11T07:30:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:30:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_temp": [], 2024-09-11T07:30:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_temp": [], 2024-09-11T07:30:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: "blocklist": { 2024-09-11T07:30:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/240999923": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:30:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/4016369460": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:30:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/585929238": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:30:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/544031024": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:30:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/2504196132": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:30:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/1451388349": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:30:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/639336081": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:30:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2104069730": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:30:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/4236450731": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:30:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/3573193776": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:30:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2661764422": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:30:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2197488530": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:30:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2968034970": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:30:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/599318175": "2024-09-12T07:27:07.919998+0000" 2024-09-11T07:30:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: "range_blocklist": {}, 2024-09-11T07:30:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profiles": { 2024-09-11T07:30:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: "default": { 2024-09-11T07:30:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: "crush-failure-domain": "osd", 2024-09-11T07:30:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: "k": "2", 2024-09-11T07:30:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: "m": "1", 2024-09-11T07:30:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: "plugin": "jerasure", 2024-09-11T07:30:19.144 INFO:teuthology.orchestra.run.smithi022.stdout: "technique": "reed_sol_van" 2024-09-11T07:30:19.144 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.144 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:19.144 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps_queue": [], 2024-09-11T07:30:19.144 INFO:teuthology.orchestra.run.smithi022.stdout: "new_removed_snaps": [], 2024-09-11T07:30:19.144 INFO:teuthology.orchestra.run.smithi022.stdout: "new_purged_snaps": [], 2024-09-11T07:30:19.144 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_node_flags": {}, 2024-09-11T07:30:19.144 INFO:teuthology.orchestra.run.smithi022.stdout: "device_class_flags": {}, 2024-09-11T07:30:19.144 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode": { 2024-09-11T07:30:19.144 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode_enabled": false, 2024-09-11T07:30:19.144 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_bucket_count": 0, 2024-09-11T07:30:19.144 INFO:teuthology.orchestra.run.smithi022.stdout: "degraded_stretch_mode": 0, 2024-09-11T07:30:19.145 INFO:teuthology.orchestra.run.smithi022.stdout: "recovering_stretch_mode": 0, 2024-09-11T07:30:19.145 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode_bucket": 0 2024-09-11T07:30:19.145 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:19.145 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:30:19.145 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:19.172 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:19.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:19.173 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:19.274 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:19.316 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:19.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:19.317 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:19.418 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:19.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:19.464 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:19.465 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:19.502 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:30:19 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:30:19.565 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:19.568 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:30:19.595 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 77 16 111552 111536 90.5156 60.0625 0.00542174 0.0110424 2024-09-11T07:30:19.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:19.616 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:19.616 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:19.717 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:19.742 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:19.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:19.743 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:19.749 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:19.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:19 smithi104 ceph-mon[33507]: 6.a scrub starts 2024-09-11T07:30:19.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:19 smithi104 ceph-mon[33507]: 6.a scrub ok 2024-09-11T07:30:19.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:19 smithi104 ceph-mon[33507]: 5.1d scrub starts 2024-09-11T07:30:19.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:19 smithi104 ceph-mon[33507]: 5.1d scrub ok 2024-09-11T07:30:19.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:19 smithi104 ceph-mon[33507]: 4.3 scrub starts 2024-09-11T07:30:19.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:19 smithi104 ceph-mon[33507]: 4.3 scrub ok 2024-09-11T07:30:19.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:19 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1253880498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:30:19.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:19 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:19.798 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:19 smithi022 ceph-mon[29462]: 6.a scrub starts 2024-09-11T07:30:19.799 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:19 smithi022 ceph-mon[29462]: 6.a scrub ok 2024-09-11T07:30:19.799 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:19 smithi022 ceph-mon[29462]: 5.1d scrub starts 2024-09-11T07:30:19.799 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:19 smithi022 ceph-mon[29462]: 5.1d scrub ok 2024-09-11T07:30:19.799 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:19 smithi022 ceph-mon[29462]: 4.3 scrub starts 2024-09-11T07:30:19.799 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:19 smithi022 ceph-mon[29462]: 4.3 scrub ok 2024-09-11T07:30:19.799 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:19 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1253880498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:30:19.799 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:19 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:19.799 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:19 smithi022 ceph-mon[37330]: 6.a scrub starts 2024-09-11T07:30:19.799 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:19 smithi022 ceph-mon[37330]: 6.a scrub ok 2024-09-11T07:30:19.800 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:19 smithi022 ceph-mon[37330]: 5.1d scrub starts 2024-09-11T07:30:19.800 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:19 smithi022 ceph-mon[37330]: 5.1d scrub ok 2024-09-11T07:30:19.800 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:19 smithi022 ceph-mon[37330]: 4.3 scrub starts 2024-09-11T07:30:19.800 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:19 smithi022 ceph-mon[37330]: 4.3 scrub ok 2024-09-11T07:30:19.800 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:19 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1253880498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:30:19.800 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:19 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:19.844 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:19.868 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:19.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:19.869 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:19.970 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:20.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:20.014 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:20.015 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:20.115 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:20.141 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:20.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:20.142 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:20.243 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:20.288 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:20.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:20.290 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:20.368 INFO:tasks.thrashosds.thrasher:j is {'epoch': 122, 'fsid': '12a285d8-700e-11ef-bcea-c7b262605968', 'created': '2024-09-11T07:21:18.695332+0000', 'modified': '2024-09-11T07:30:17.478738+0000', 'last_up_change': '2024-09-11T07:26:15.385688+0000', 'last_in_change': '2024-09-11T07:25:58.551813+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 7, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T07:24:21.772426+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': '.rgw.root', 'create_time': '2024-09-11T07:26:29.941354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 3, 'pool_name': 'datapool', 'create_time': '2024-09-11T07:26:30.396859+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-11T07:26:31.922926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-11T07:26:34.007746+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-11T07:26:36.077215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}, {'pool': 7, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-11T07:28:48.880975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '09075f16-32f4-4f88-a116-c3d7f977a0e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 92, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6801', 'nonce': 2821493591}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6802', 'nonce': 2821493591}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6804', 'nonce': 2821493591}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6803', 'nonce': 2821493591}]}, 'public_addr': '172.21.15.22:6801/2821493591', 'cluster_addr': '172.21.15.22:6802/2821493591', 'heartbeat_back_addr': '172.21.15.22:6804/2821493591', 'heartbeat_front_addr': '172.21.15.22:6803/2821493591', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '52cc1e06-9506-48c0-9f7b-d4270dff4b9f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 92, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6805', 'nonce': 4037911317}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6806', 'nonce': 4037911317}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6808', 'nonce': 4037911317}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6807', 'nonce': 4037911317}]}, 'public_addr': '172.21.15.22:6805/4037911317', 'cluster_addr': '172.21.15.22:6806/4037911317', 'heartbeat_back_addr': '172.21.15.22:6808/4037911317', 'heartbeat_front_addr': '172.21.15.22:6807/4037911317', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'fa2b2730-2c1d-45d0-be8a-494a60dc6b9b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 92, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6809', 'nonce': 1675799666}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6810', 'nonce': 1675799666}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6812', 'nonce': 1675799666}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6811', 'nonce': 1675799666}]}, 'public_addr': '172.21.15.22:6809/1675799666', 'cluster_addr': '172.21.15.22:6810/1675799666', 'heartbeat_back_addr': '172.21.15.22:6812/1675799666', 'heartbeat_front_addr': '172.21.15.22:6811/1675799666', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '8d53d3fc-e401-47df-b70b-4b4b1f4539df', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 25, 'up_thru': 118, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6813', 'nonce': 1501962752}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6814', 'nonce': 1501962752}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6816', 'nonce': 1501962752}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6815', 'nonce': 1501962752}]}, 'public_addr': '172.21.15.22:6813/1501962752', 'cluster_addr': '172.21.15.22:6814/1501962752', 'heartbeat_back_addr': '172.21.15.22:6816/1501962752', 'heartbeat_front_addr': '172.21.15.22:6815/1501962752', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'd3207fe0-8b97-4067-a755-ba6d449c53c7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 113, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6800', 'nonce': 1210229443}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6801', 'nonce': 1210229443}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6803', 'nonce': 1210229443}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6802', 'nonce': 1210229443}]}, 'public_addr': '172.21.15.104:6800/1210229443', 'cluster_addr': '172.21.15.104:6801/1210229443', 'heartbeat_back_addr': '172.21.15.104:6803/1210229443', 'heartbeat_front_addr': '172.21.15.104:6802/1210229443', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '89c9352b-30b2-48c6-869e-047d07a23a43', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 37, 'up_thru': 100, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6804', 'nonce': 2735328732}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6805', 'nonce': 2735328732}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6807', 'nonce': 2735328732}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6806', 'nonce': 2735328732}]}, 'public_addr': '172.21.15.104:6804/2735328732', 'cluster_addr': '172.21.15.104:6805/2735328732', 'heartbeat_back_addr': '172.21.15.104:6807/2735328732', 'heartbeat_front_addr': '172.21.15.104:6806/2735328732', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '704563de-d88f-480a-b469-720135704d42', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 44, 'up_thru': 92, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6808', 'nonce': 721493010}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6809', 'nonce': 721493010}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6811', 'nonce': 721493010}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6810', 'nonce': 721493010}]}, 'public_addr': '172.21.15.104:6808/721493010', 'cluster_addr': '172.21.15.104:6809/721493010', 'heartbeat_back_addr': '172.21.15.104:6811/721493010', 'heartbeat_front_addr': '172.21.15.104:6810/721493010', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '18199e47-0052-49f0-b45f-0fb201e0d0f4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 52, 'up_thru': 92, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6812', 'nonce': 3018423954}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6813', 'nonce': 3018423954}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6815', 'nonce': 3018423954}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6814', 'nonce': 3018423954}]}, 'public_addr': '172.21.15.104:6812/3018423954', 'cluster_addr': '172.21.15.104:6813/3018423954', 'heartbeat_back_addr': '172.21.15.104:6815/3018423954', 'heartbeat_front_addr': '172.21.15.104:6814/3018423954', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:30:16.531847+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:30:07.299072+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:30:13.516466+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:29:57.588987+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:29:40.758972+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:29:10.678249+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:29:46.707244+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:29:37.516403+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.1c', 'osds': [5, 2, 7]}], 'pg_upmap_items': [{'pgid': '2.7', 'mappings': [{'from': 7, 'to': 4}]}, {'pgid': '4.6', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.14', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.1f', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.22:0/240999923': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:6800/4016369460': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/585929238': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/544031024': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:6800/2504196132': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/1451388349': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/639336081': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/2104069730': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/4236450731': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:6800/3573193776': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/2661764422': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/2197488530': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/2968034970': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/599318175': '2024-09-12T07:27:07.919998+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-11T07:30:20.368 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 4.1f 2024-09-11T07:30:20.369 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd rm-pg-upmap-items 4.1f 2024-09-11T07:30:20.391 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:20.469 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:20.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:20.470 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:20.570 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:20.595 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 78 16 112516 112500 90.1275 60.25 0.0156448 0.0110905 2024-09-11T07:30:20.598 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:20.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:20.599 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:20.616 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:30:20 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:30:20] "GET /metrics HTTP/1.1" 200 223621 "" "Prometheus/2.43.0" 2024-09-11T07:30:20.616 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:20 smithi022 ceph-mon[29462]: 6.17 scrub starts 2024-09-11T07:30:20.616 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:20 smithi022 ceph-mon[29462]: 6.17 scrub ok 2024-09-11T07:30:20.617 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:20 smithi022 ceph-mon[29462]: 5.4 scrub starts 2024-09-11T07:30:20.617 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:20 smithi022 ceph-mon[29462]: 5.4 scrub ok 2024-09-11T07:30:20.617 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:20 smithi022 ceph-mon[29462]: 3.1 scrub starts 2024-09-11T07:30:20.617 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:20 smithi022 ceph-mon[29462]: 3.1 scrub ok 2024-09-11T07:30:20.617 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:20 smithi022 ceph-mon[29462]: pgmap v149: 148 pgs: 148 active+clean; 6.8 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.5 KiB/s rd, 93 MiB/s wr, 1.49k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:30:20.617 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:20 smithi022 ceph-mon[29462]: osdmap e123: 8 total, 8 up, 8 in 2024-09-11T07:30:20.617 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:20 smithi022 ceph-mon[37330]: 6.17 scrub starts 2024-09-11T07:30:20.617 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:20 smithi022 ceph-mon[37330]: 6.17 scrub ok 2024-09-11T07:30:20.617 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:20 smithi022 ceph-mon[37330]: 5.4 scrub starts 2024-09-11T07:30:20.617 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:20 smithi022 ceph-mon[37330]: 5.4 scrub ok 2024-09-11T07:30:20.617 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:20 smithi022 ceph-mon[37330]: 3.1 scrub starts 2024-09-11T07:30:20.618 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:20 smithi022 ceph-mon[37330]: 3.1 scrub ok 2024-09-11T07:30:20.618 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:20 smithi022 ceph-mon[37330]: pgmap v149: 148 pgs: 148 active+clean; 6.8 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.5 KiB/s rd, 93 MiB/s wr, 1.49k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:30:20.618 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:20 smithi022 ceph-mon[37330]: osdmap e123: 8 total, 8 up, 8 in 2024-09-11T07:30:20.699 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:20.743 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:20.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:20.744 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:20.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:20 smithi104 ceph-mon[33507]: 6.17 scrub starts 2024-09-11T07:30:20.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:20 smithi104 ceph-mon[33507]: 6.17 scrub ok 2024-09-11T07:30:20.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:20 smithi104 ceph-mon[33507]: 5.4 scrub starts 2024-09-11T07:30:20.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:20 smithi104 ceph-mon[33507]: 5.4 scrub ok 2024-09-11T07:30:20.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:20 smithi104 ceph-mon[33507]: 3.1 scrub starts 2024-09-11T07:30:20.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:20 smithi104 ceph-mon[33507]: 3.1 scrub ok 2024-09-11T07:30:20.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:20 smithi104 ceph-mon[33507]: pgmap v149: 148 pgs: 148 active+clean; 6.8 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.5 KiB/s rd, 93 MiB/s wr, 1.49k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:30:20.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:20 smithi104 ceph-mon[33507]: osdmap e123: 8 total, 8 up, 8 in 2024-09-11T07:30:20.845 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:20.869 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:20.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:20.870 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:20.971 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:21.018 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:21.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:21.019 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:21.120 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:21.165 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:21.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:21.166 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:21.267 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:21.295 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:21.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:21.296 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:21.396 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:21.421 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:21.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:21.422 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:21.522 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:21.549 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:21.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:21.549 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:21.595 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 79 16 113520 113504 89.7809 62.75 0.0080568 0.011133 2024-09-11T07:30:21.650 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:21.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:21.680 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:21.680 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:21.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:21 smithi104 ceph-mon[33507]: 6.1a scrub starts 2024-09-11T07:30:21.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:21 smithi104 ceph-mon[33507]: 6.1a scrub ok 2024-09-11T07:30:21.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:21 smithi104 ceph-mon[33507]: 7.5 scrub starts 2024-09-11T07:30:21.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:21 smithi104 ceph-mon[33507]: 2.6 scrub starts 2024-09-11T07:30:21.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:21 smithi104 ceph-mon[33507]: 2.6 scrub ok 2024-09-11T07:30:21.781 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:21.782 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:21.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:21.846 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:21.847 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:21.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:21 smithi022 ceph-mon[29462]: 6.1a scrub starts 2024-09-11T07:30:21.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:21 smithi022 ceph-mon[29462]: 6.1a scrub ok 2024-09-11T07:30:21.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:21 smithi022 ceph-mon[29462]: 7.5 scrub starts 2024-09-11T07:30:21.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:21 smithi022 ceph-mon[29462]: 2.6 scrub starts 2024-09-11T07:30:21.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:21 smithi022 ceph-mon[29462]: 2.6 scrub ok 2024-09-11T07:30:21.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:21 smithi022 ceph-mon[37330]: 6.1a scrub starts 2024-09-11T07:30:21.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:21 smithi022 ceph-mon[37330]: 6.1a scrub ok 2024-09-11T07:30:21.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:21 smithi022 ceph-mon[37330]: 7.5 scrub starts 2024-09-11T07:30:21.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:21 smithi022 ceph-mon[37330]: 2.6 scrub starts 2024-09-11T07:30:21.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:21 smithi022 ceph-mon[37330]: 2.6 scrub ok 2024-09-11T07:30:21.948 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:21.975 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:21.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:21.976 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:22.077 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:22.117 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:22.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:22.118 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:22.219 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:22.233 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:22.274 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:22.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:22.275 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:22.376 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:22.422 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:22.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:22.423 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:22.524 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:22.568 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:22.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:22.569 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:22.595 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:2024-09-11T07:30:22.594487+0000 min lat: 0.00117769 max lat: 0.136258 avg lat: 0.0111758 2024-09-11T07:30:22.595 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T07:30:22.595 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 80 16 114514 114498 89.4352 62.125 0.0253443 0.0111758 2024-09-11T07:30:22.670 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:22.712 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:22.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:22.713 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:22.814 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:22.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:22.846 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:22.846 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:22.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:22 smithi022 ceph-mon[29462]: 6.10 scrub starts 2024-09-11T07:30:22.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:22 smithi022 ceph-mon[29462]: 6.10 scrub ok 2024-09-11T07:30:22.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:22 smithi022 ceph-mon[29462]: pgmap v151: 148 pgs: 148 active+clean; 6.8 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 652 B/s rd, 40 MiB/s wr, 633 op/s 2024-09-11T07:30:22.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:22 smithi022 ceph-mon[37330]: 6.10 scrub starts 2024-09-11T07:30:22.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:22 smithi022 ceph-mon[37330]: 6.10 scrub ok 2024-09-11T07:30:22.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:22 smithi022 ceph-mon[37330]: pgmap v151: 148 pgs: 148 active+clean; 6.8 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 652 B/s rd, 40 MiB/s wr, 633 op/s 2024-09-11T07:30:22.946 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:22.989 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:22.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:22.990 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:23.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:22 smithi104 ceph-mon[33507]: 6.10 scrub starts 2024-09-11T07:30:23.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:22 smithi104 ceph-mon[33507]: 6.10 scrub ok 2024-09-11T07:30:23.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:22 smithi104 ceph-mon[33507]: pgmap v151: 148 pgs: 148 active+clean; 6.8 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 652 B/s rd, 40 MiB/s wr, 633 op/s 2024-09-11T07:30:23.091 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:23.119 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:23.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:23.120 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:23.220 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:23.267 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:23.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:23.269 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:23.369 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:23.399 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:23.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:23.400 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:23.501 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:23.556 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is unset 2024-09-11T07:30:23.571 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:23.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:23.572 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:23.595 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 81 16 115505 115489 89.0957 61.9375 0.0100261 0.0112192 2024-09-11T07:30:23.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:23 smithi022 ceph-mon[37330]: 7.c scrub starts 2024-09-11T07:30:23.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:23 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/235919016' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:30:23.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:23 smithi022 ceph-mon[37330]: 7.8 scrub starts 2024-09-11T07:30:23.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:30:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:23 smithi022 ceph-mon[29462]: 7.c scrub starts 2024-09-11T07:30:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:23 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/235919016' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:30:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:23 smithi022 ceph-mon[29462]: 7.8 scrub starts 2024-09-11T07:30:23.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:30:23.673 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:23.701 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:23.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:23.702 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:23.803 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:23.834 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:23.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:23.835 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:23.935 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:23.979 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:23.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:23.980 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:24.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:23 smithi104 ceph-mon[33507]: 7.c scrub starts 2024-09-11T07:30:24.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:23 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/235919016' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:30:24.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:23 smithi104 ceph-mon[33507]: 7.8 scrub starts 2024-09-11T07:30:24.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:30:24.081 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:24.122 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:24.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:24.123 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:24.224 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:24.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:24.252 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:24.252 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:24.353 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:24.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:24.397 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:24.398 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:24.499 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:24.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:24.529 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:24.530 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:24.530 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:30:24 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:30:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:30:24.595 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 82 16 116525 116509 88.7865 63.75 0.0123282 0.0112582 2024-09-11T07:30:24.630 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:24.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:24 smithi022 ceph-mon[29462]: 7.5 scrub ok 2024-09-11T07:30:24.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:24 smithi022 ceph-mon[29462]: 6.2 scrub starts 2024-09-11T07:30:24.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:24 smithi022 ceph-mon[29462]: 6.2 scrub ok 2024-09-11T07:30:24.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:24 smithi022 ceph-mon[29462]: 5.15 scrub starts 2024-09-11T07:30:24.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:24 smithi022 ceph-mon[29462]: 5.15 scrub ok 2024-09-11T07:30:24.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:24 smithi022 ceph-mon[29462]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:30:24.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:24 smithi022 ceph-mon[29462]: Cluster is now healthy 2024-09-11T07:30:24.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:24 smithi022 ceph-mon[29462]: 2.13 scrub starts 2024-09-11T07:30:24.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:24 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/235919016' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:30:24.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:24 smithi022 ceph-mon[29462]: osdmap e124: 8 total, 8 up, 8 in 2024-09-11T07:30:24.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:24 smithi022 ceph-mon[29462]: 2.13 scrub ok 2024-09-11T07:30:24.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:24 smithi022 ceph-mon[29462]: pgmap v153: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 786 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-11T07:30:24.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:24 smithi022 ceph-mon[37330]: 7.5 scrub ok 2024-09-11T07:30:24.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:24 smithi022 ceph-mon[37330]: 6.2 scrub starts 2024-09-11T07:30:24.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:24 smithi022 ceph-mon[37330]: 6.2 scrub ok 2024-09-11T07:30:24.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:24 smithi022 ceph-mon[37330]: 5.15 scrub starts 2024-09-11T07:30:24.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:24 smithi022 ceph-mon[37330]: 5.15 scrub ok 2024-09-11T07:30:24.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:24 smithi022 ceph-mon[37330]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:30:24.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:24 smithi022 ceph-mon[37330]: Cluster is now healthy 2024-09-11T07:30:24.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:24 smithi022 ceph-mon[37330]: 2.13 scrub starts 2024-09-11T07:30:24.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:24 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/235919016' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:30:24.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:24 smithi022 ceph-mon[37330]: osdmap e124: 8 total, 8 up, 8 in 2024-09-11T07:30:24.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:24 smithi022 ceph-mon[37330]: 2.13 scrub ok 2024-09-11T07:30:24.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:24 smithi022 ceph-mon[37330]: pgmap v153: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 786 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-11T07:30:24.675 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:24.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:24.676 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:24.777 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:24.819 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:24.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:24.820 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:24.921 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:24.963 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:24.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:24.964 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:25.032 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:24 smithi104 ceph-mon[33507]: 7.5 scrub ok 2024-09-11T07:30:25.032 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:24 smithi104 ceph-mon[33507]: 6.2 scrub starts 2024-09-11T07:30:25.032 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:24 smithi104 ceph-mon[33507]: 6.2 scrub ok 2024-09-11T07:30:25.032 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:24 smithi104 ceph-mon[33507]: 5.15 scrub starts 2024-09-11T07:30:25.032 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:24 smithi104 ceph-mon[33507]: 5.15 scrub ok 2024-09-11T07:30:25.032 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:24 smithi104 ceph-mon[33507]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:30:25.032 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:24 smithi104 ceph-mon[33507]: Cluster is now healthy 2024-09-11T07:30:25.033 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:24 smithi104 ceph-mon[33507]: 2.13 scrub starts 2024-09-11T07:30:25.033 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:24 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/235919016' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:30:25.033 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:24 smithi104 ceph-mon[33507]: osdmap e124: 8 total, 8 up, 8 in 2024-09-11T07:30:25.033 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:24 smithi104 ceph-mon[33507]: 2.13 scrub ok 2024-09-11T07:30:25.033 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:24 smithi104 ceph-mon[33507]: pgmap v153: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 786 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-11T07:30:25.065 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:25.089 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:25.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:25.090 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:25.191 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:25.218 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:25.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:25.219 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:25.320 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:25.349 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:25.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:25.350 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:25.450 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:25.475 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:25.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:25.476 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:25.577 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:25.595 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 83 16 117500 117484 88.451 60.9375 0.00847617 0.0113 2024-09-11T07:30:25.603 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:25.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:25.604 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:25.676 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:30:25.676 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:30:25.676 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.676 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:117565 7.0 e125/92 7:00b2b346:::benchmark_data_smithi022_77720_object117564:head v 125'7273, mlcod=125'7270)", 2024-09-11T07:30:25.676 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:25.659863+0000", 2024-09-11T07:30:25.677 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.013282659, 2024-09-11T07:30:25.677 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.013311659, 2024-09-11T07:30:25.677 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:25.677 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:30:25.677 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:25.677 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.677 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:25.677 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.659863+0000", 2024-09-11T07:30:25.677 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:25.677 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.677 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.677 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:25.677 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.659863+0000", 2024-09-11T07:30:25.678 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:25.678 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.678 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.678 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:25.678 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.659863+0000", 2024-09-11T07:30:25.678 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:30:25.678 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.678 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.678 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:25.678 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.659896+0000", 2024-09-11T07:30:25.678 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3540000000000001e-05 2024-09-11T07:30:25.678 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.678 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.678 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:25.678 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.659906+0000", 2024-09-11T07:30:25.679 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.5969999999999998e-06 2024-09-11T07:30:25.679 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.679 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.679 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:25.679 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.659910+0000", 2024-09-11T07:30:25.679 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8979999999999998e-06 2024-09-11T07:30:25.679 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.679 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.679 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:25.679 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.665724+0000", 2024-09-11T07:30:25.679 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0058142059999999997 2024-09-11T07:30:25.680 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.680 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.680 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:25.680 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.665746+0000", 2024-09-11T07:30:25.680 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1940999999999999e-05 2024-09-11T07:30:25.680 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:25.680 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:25.680 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:25.680 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.680 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.680 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:117566 7.0 e125/92 7:095b1979:::benchmark_data_smithi022_77720_object117565:head v 125'7274, mlcod=125'7270)", 2024-09-11T07:30:25.680 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:25.661933+0000", 2024-09-11T07:30:25.680 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.011213235, 2024-09-11T07:30:25.681 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.011314568000000001, 2024-09-11T07:30:25.681 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:25.681 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:30:25.681 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:25.681 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.681 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:25.681 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.661933+0000", 2024-09-11T07:30:25.681 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:25.681 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.681 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.681 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:25.681 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.661933+0000", 2024-09-11T07:30:25.681 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:25.682 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.682 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.682 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:25.682 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.661932+0000", 2024-09-11T07:30:25.682 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:30:25.682 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.682 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.682 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:25.682 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.661964+0000", 2024-09-11T07:30:25.682 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2709999999999997e-05 2024-09-11T07:30:25.682 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.682 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.682 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:25.683 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.661977+0000", 2024-09-11T07:30:25.683 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2376e-05 2024-09-11T07:30:25.683 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.683 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.683 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:25.683 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.661983+0000", 2024-09-11T07:30:25.683 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4189999999999999e-06 2024-09-11T07:30:25.683 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.683 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.683 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:25.683 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.672208+0000", 2024-09-11T07:30:25.683 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.010224982000000001 2024-09-11T07:30:25.683 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.684 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.684 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:25.684 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.672227+0000", 2024-09-11T07:30:25.684 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8604e-05 2024-09-11T07:30:25.684 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:25.684 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:25.684 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:25.684 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.684 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.684 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:117573 7.2 7:48edb364:::benchmark_data_smithi022_77720_object117572:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-11T07:30:25.684 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:25.666457+0000", 2024-09-11T07:30:25.684 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0066890300000000003, 2024-09-11T07:30:25.684 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0068619880000000003, 2024-09-11T07:30:25.684 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:25.685 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "waiting for sub ops", 2024-09-11T07:30:25.685 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:25.685 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:25.685 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:25.685 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 117573 2024-09-11T07:30:25.685 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.685 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:25.685 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.685 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:25.685 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.666457+0000", 2024-09-11T07:30:25.685 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:25.685 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.685 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.685 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:25.686 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.666457+0000", 2024-09-11T07:30:25.686 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:25.686 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.686 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.686 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:25.686 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.666455+0000", 2024-09-11T07:30:25.686 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:30:25.686 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.686 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.686 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:25.686 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.666479+0000", 2024-09-11T07:30:25.686 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4029999999999999e-05 2024-09-11T07:30:25.686 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.686 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.687 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:25.687 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.666489+0000", 2024-09-11T07:30:25.687 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0437e-05 2024-09-11T07:30:25.687 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.687 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.687 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:25.687 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.666495+0000", 2024-09-11T07:30:25.687 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7049999999999996e-06 2024-09-11T07:30:25.687 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.687 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.687 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:25.687 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.666537+0000", 2024-09-11T07:30:25.687 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2357000000000001e-05 2024-09-11T07:30:25.687 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.688 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.688 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:25.688 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.666651+0000", 2024-09-11T07:30:25.688 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000113992 2024-09-11T07:30:25.688 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.688 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.688 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:30:25.688 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.666800+0000", 2024-09-11T07:30:25.688 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014883799999999999 2024-09-11T07:30:25.688 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:25.688 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:25.688 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:25.688 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.688 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.689 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:117574 7.1 e125/92 7:8ad1fdbf:::benchmark_data_smithi022_77720_object117573:head v 125'7271, mlcod=125'7269)", 2024-09-11T07:30:25.689 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:25.668588+0000", 2024-09-11T07:30:25.689 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0045582080000000002, 2024-09-11T07:30:25.689 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0048082979999999999, 2024-09-11T07:30:25.689 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:25.689 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:30:25.689 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:25.689 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.689 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:25.689 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.668588+0000", 2024-09-11T07:30:25.689 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:25.689 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.689 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.689 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:25.690 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.668588+0000", 2024-09-11T07:30:25.690 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:25.690 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.690 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.690 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:25.690 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.668586+0000", 2024-09-11T07:30:25.690 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:30:25.690 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.690 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.690 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:25.690 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.668620+0000", 2024-09-11T07:30:25.690 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3800999999999997e-05 2024-09-11T07:30:25.691 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.691 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.691 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:25.691 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.668631+0000", 2024-09-11T07:30:25.691 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1029999999999999e-05 2024-09-11T07:30:25.692 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.692 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.692 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:25.692 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.668637+0000", 2024-09-11T07:30:25.692 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0889999999999996e-06 2024-09-11T07:30:25.692 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.692 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.692 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:25.692 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.669175+0000", 2024-09-11T07:30:25.692 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00053720199999999997 2024-09-11T07:30:25.692 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.692 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.692 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:25.693 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.669200+0000", 2024-09-11T07:30:25.693 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5677000000000002e-05 2024-09-11T07:30:25.693 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:25.693 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:25.693 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:25.693 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.693 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.693 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:117577 7.a 7:5c7b6c6d:::benchmark_data_smithi022_77720_object117576:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-11T07:30:25.693 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:25.670385+0000", 2024-09-11T07:30:25.693 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0027613440000000002, 2024-09-11T07:30:25.693 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0030963079999999999, 2024-09-11T07:30:25.693 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:25.693 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "waiting for sub ops", 2024-09-11T07:30:25.693 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:25.694 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:25.694 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:25.694 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 117577 2024-09-11T07:30:25.694 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.694 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:25.694 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.694 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:25.694 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.670385+0000", 2024-09-11T07:30:25.694 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:25.694 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.694 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.694 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:25.694 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.670385+0000", 2024-09-11T07:30:25.694 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:25.694 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.695 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.695 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:25.695 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.670382+0000", 2024-09-11T07:30:25.695 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:30:25.695 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.695 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.695 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:25.695 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.670407+0000", 2024-09-11T07:30:25.695 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5001e-05 2024-09-11T07:30:25.695 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.695 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.695 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:25.696 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.670419+0000", 2024-09-11T07:30:25.696 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2069000000000001e-05 2024-09-11T07:30:25.696 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.696 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.696 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:25.696 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.670425+0000", 2024-09-11T07:30:25.696 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.998e-06 2024-09-11T07:30:25.696 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.696 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.696 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:25.696 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.670470+0000", 2024-09-11T07:30:25.696 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4518999999999998e-05 2024-09-11T07:30:25.696 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.697 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.697 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:25.697 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.670590+0000", 2024-09-11T07:30:25.697 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000120798 2024-09-11T07:30:25.697 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.697 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.697 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,5", 2024-09-11T07:30:25.697 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.670741+0000", 2024-09-11T07:30:25.697 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000150409 2024-09-11T07:30:25.697 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:25.697 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:25.697 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:25.697 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.698 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.698 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:117576 7.1 e125/92 7:8921dbe0:::benchmark_data_smithi022_77720_object117575:head v 125'7272, mlcod=125'7269)", 2024-09-11T07:30:25.698 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:25.671239+0000", 2024-09-11T07:30:25.698 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.001906554, 2024-09-11T07:30:25.698 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0023204079999999999, 2024-09-11T07:30:25.698 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:25.698 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:30:25.698 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:25.698 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.698 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:25.698 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.671239+0000", 2024-09-11T07:30:25.698 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:25.698 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.698 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.699 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:25.699 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.671239+0000", 2024-09-11T07:30:25.699 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:25.699 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.699 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.699 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:25.699 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.671238+0000", 2024-09-11T07:30:25.699 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:30:25.699 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.699 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.699 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:25.699 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.671269+0000", 2024-09-11T07:30:25.699 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.1306000000000001e-05 2024-09-11T07:30:25.699 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.699 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.700 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:25.700 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.671280+0000", 2024-09-11T07:30:25.700 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0927000000000001e-05 2024-09-11T07:30:25.700 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.700 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.700 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:25.700 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.671286+0000", 2024-09-11T07:30:25.700 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6110000000000001e-06 2024-09-11T07:30:25.700 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.700 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.700 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:25.700 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.672304+0000", 2024-09-11T07:30:25.700 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0010181439999999999 2024-09-11T07:30:25.700 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.701 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.701 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:25.701 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.672329+0000", 2024-09-11T07:30:25.701 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5083999999999999e-05 2024-09-11T07:30:25.701 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:25.701 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:25.701 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:25.701 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.701 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.701 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:117579 7.3 7:cd99c06b:::benchmark_data_smithi022_77720_object117578:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-11T07:30:25.701 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:25.671389+0000", 2024-09-11T07:30:25.701 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0017571349999999999, 2024-09-11T07:30:25.701 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0022324369999999999, 2024-09-11T07:30:25.701 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:25.702 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "waiting for sub ops", 2024-09-11T07:30:25.702 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:25.702 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:25.702 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:25.702 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 117579 2024-09-11T07:30:25.702 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.702 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:25.702 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.702 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:25.702 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.671389+0000", 2024-09-11T07:30:25.702 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:25.702 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.702 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.702 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:25.702 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.671389+0000", 2024-09-11T07:30:25.702 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:25.703 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.703 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.703 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:25.703 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.671387+0000", 2024-09-11T07:30:25.703 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:30:25.703 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.703 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.703 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:25.703 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.671423+0000", 2024-09-11T07:30:25.703 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5688000000000001e-05 2024-09-11T07:30:25.703 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.703 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.703 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:25.703 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.671433+0000", 2024-09-11T07:30:25.703 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.041e-05 2024-09-11T07:30:25.703 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.704 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.704 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:25.704 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.671439+0000", 2024-09-11T07:30:25.704 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3070000000000002e-06 2024-09-11T07:30:25.704 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.704 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.704 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:25.704 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.672222+0000", 2024-09-11T07:30:25.704 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00078312100000000001 2024-09-11T07:30:25.704 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.704 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.704 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:25.704 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.672351+0000", 2024-09-11T07:30:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000128892 2024-09-11T07:30:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,4", 2024-09-11T07:30:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.672512+0000", 2024-09-11T07:30:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000161192 2024-09-11T07:30:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:117580 7.3 7.1f70123 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-11T07:30:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:25.671451+0000", 2024-09-11T07:30:25.705 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.001694711, 2024-09-11T07:30:25.706 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002242417, 2024-09-11T07:30:25.706 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:25.706 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "queued for pg", 2024-09-11T07:30:25.706 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:25.706 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:25.706 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:25.706 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 117580 2024-09-11T07:30:25.706 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.706 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:25.706 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.706 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:25.706 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.671451+0000", 2024-09-11T07:30:25.706 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:25.706 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.707 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.707 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:25.707 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.671451+0000", 2024-09-11T07:30:25.707 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:25.707 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.707 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.707 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:25.707 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.671451+0000", 2024-09-11T07:30:25.707 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:30:25.707 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.707 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.707 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:25.707 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.671473+0000", 2024-09-11T07:30:25.707 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2200000000000001e-05 2024-09-11T07:30:25.707 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.708 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.708 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:25.708 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.671479+0000", 2024-09-11T07:30:25.708 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1659999999999999e-06 2024-09-11T07:30:25.708 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.708 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.708 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:25.708 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.671483+0000", 2024-09-11T07:30:25.708 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0380000000000001e-06 2024-09-11T07:30:25.708 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:25.708 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:25.708 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:25.708 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.708 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.709 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:117581 7.1 e125/92)", 2024-09-11T07:30:25.709 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:25.673424+0000", 2024-09-11T07:30:25.709 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 4294967295.9997225, 2024-09-11T07:30:25.709 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00033021599999999998, 2024-09-11T07:30:25.709 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:25.709 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "queued for pg", 2024-09-11T07:30:25.709 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:25.709 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.709 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:25.709 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.673424+0000", 2024-09-11T07:30:25.709 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:25.709 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.709 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.709 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:25.709 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.673424+0000", 2024-09-11T07:30:25.709 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:25.710 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.710 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.710 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:25.710 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.673422+0000", 2024-09-11T07:30:25.710 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:30:25.710 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.710 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.710 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:25.710 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.673476+0000", 2024-09-11T07:30:25.710 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.38e-05 2024-09-11T07:30:25.710 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.710 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.710 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:25.710 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.673485+0000", 2024-09-11T07:30:25.710 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.6809999999999993e-06 2024-09-11T07:30:25.711 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.711 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.711 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:25.711 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.673491+0000", 2024-09-11T07:30:25.711 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.5740000000000003e-06 2024-09-11T07:30:25.711 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:25.711 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:25.711 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:25.711 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.711 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.711 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop_reply(client.24778.0:117577 7.a e125/92)", 2024-09-11T07:30:25.711 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:25.673703+0000", 2024-09-11T07:30:25.711 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 4294967295.9994426, 2024-09-11T07:30:25.711 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.6811999999999994e-05, 2024-09-11T07:30:25.711 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:25.712 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "queued for pg", 2024-09-11T07:30:25.712 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:25.712 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.712 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:25.712 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.673703+0000", 2024-09-11T07:30:25.712 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:25.712 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.712 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.712 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:25.712 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.673703+0000", 2024-09-11T07:30:25.712 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:25.712 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.712 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.712 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:25.713 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.673702+0000", 2024-09-11T07:30:25.713 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:30:25.713 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.713 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.713 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:25.713 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.673708+0000", 2024-09-11T07:30:25.713 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8440000000000003e-06 2024-09-11T07:30:25.713 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.713 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.713 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:25.713 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.673727+0000", 2024-09-11T07:30:25.713 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8929000000000001e-05 2024-09-11T07:30:25.713 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.713 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.714 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:25.714 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.673732+0000", 2024-09-11T07:30:25.714 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.711e-06 2024-09-11T07:30:25.714 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:25.714 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:25.714 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:25.714 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.714 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.714 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:117562 7.2 7:4b3db51f:::benchmark_data_smithi022_77720_object117561:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-11T07:30:25.714 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:25.653867+0000", 2024-09-11T07:30:25.714 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.019278970999999999, 2024-09-11T07:30:25.714 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.019987332999999999, 2024-09-11T07:30:25.714 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:25.714 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "waiting for sub ops", 2024-09-11T07:30:25.714 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:25.714 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:25.715 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:25.715 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 117562 2024-09-11T07:30:25.715 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.715 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:25.715 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.715 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:25.715 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.653867+0000", 2024-09-11T07:30:25.715 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:25.715 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.715 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.715 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:25.715 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.653867+0000", 2024-09-11T07:30:25.715 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:25.715 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.715 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.715 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:25.716 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.653866+0000", 2024-09-11T07:30:25.716 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:30:25.716 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.716 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.716 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:25.716 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.653888+0000", 2024-09-11T07:30:25.716 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1330000000000001e-05 2024-09-11T07:30:25.716 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.716 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.716 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:25.716 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.653894+0000", 2024-09-11T07:30:25.716 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3559999999999998e-06 2024-09-11T07:30:25.716 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.716 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.716 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:25.717 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.653898+0000", 2024-09-11T07:30:25.717 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7689999999999998e-06 2024-09-11T07:30:25.717 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.717 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.717 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:25.717 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.662308+0000", 2024-09-11T07:30:25.717 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0084102759999999995 2024-09-11T07:30:25.717 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.717 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.717 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:25.717 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.662440+0000", 2024-09-11T07:30:25.717 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013239400000000001 2024-09-11T07:30:25.717 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.717 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.717 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:30:25.718 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.662590+0000", 2024-09-11T07:30:25.718 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000149727 2024-09-11T07:30:25.718 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:25.718 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:25.718 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:25.718 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.669937+0000", 2024-09-11T07:30:25.718 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0073465919999999999 2024-09-11T07:30:25.718 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:25.718 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:25.718 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:25.718 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:25.718 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:30:25.718 INFO:teuthology.orchestra.run.smithi022.stdout: "num_ops": 11 2024-09-11T07:30:25.719 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:30:25.719 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:25.747 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:25.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:25.748 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:25.822 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:25 smithi022 ceph-mon[29462]: 5.1e deep-scrub starts 2024-09-11T07:30:25.822 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:25 smithi022 ceph-mon[29462]: 5.1e deep-scrub ok 2024-09-11T07:30:25.822 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:25 smithi022 ceph-mon[29462]: 7.e scrub starts 2024-09-11T07:30:25.822 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:25 smithi022 ceph-mon[29462]: osdmap e125: 8 total, 8 up, 8 in 2024-09-11T07:30:25.822 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:25 smithi022 ceph-mon[37330]: 5.1e deep-scrub starts 2024-09-11T07:30:25.822 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:25 smithi022 ceph-mon[37330]: 5.1e deep-scrub ok 2024-09-11T07:30:25.822 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:25 smithi022 ceph-mon[37330]: 7.e scrub starts 2024-09-11T07:30:25.822 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:25 smithi022 ceph-mon[37330]: osdmap e125: 8 total, 8 up, 8 in 2024-09-11T07:30:25.849 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:25.874 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:25.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:25.875 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:25.976 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:26.002 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:26.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:26.003 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:26.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:25 smithi104 ceph-mon[33507]: 5.1e deep-scrub starts 2024-09-11T07:30:26.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:25 smithi104 ceph-mon[33507]: 5.1e deep-scrub ok 2024-09-11T07:30:26.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:25 smithi104 ceph-mon[33507]: 7.e scrub starts 2024-09-11T07:30:26.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:25 smithi104 ceph-mon[33507]: osdmap e125: 8 total, 8 up, 8 in 2024-09-11T07:30:26.104 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:26.132 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:26.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:26.133 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:26.234 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:26.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:26.290 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:26.290 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:26.391 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:26.414 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:26.415 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T07:30:26.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:26.416 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:26.516 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:26.557 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:26 smithi022 ceph-mon[37330]: 7.c scrub ok 2024-09-11T07:30:26.583 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:26.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:26.584 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:26.595 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 84 16 118456 118440 88.1089 59.75 0.023417 0.0113441 2024-09-11T07:30:26.685 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:26.710 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:26.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:26.711 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:26.811 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:26.854 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:26.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:26.855 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:26.885 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:26 smithi022 ceph-mon[29462]: 7.c scrub ok 2024-09-11T07:30:26.885 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:26 smithi022 ceph-mon[29462]: 7.8 scrub ok 2024-09-11T07:30:26.885 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:26 smithi022 ceph-mon[29462]: 5.3 scrub starts 2024-09-11T07:30:26.885 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:26 smithi022 ceph-mon[29462]: 5.3 scrub ok 2024-09-11T07:30:26.885 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:26 smithi022 ceph-mon[29462]: pgmap v155: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 7.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 52 MiB/s wr, 831 op/s 2024-09-11T07:30:26.886 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:26 smithi022 ceph-mon[37330]: 7.8 scrub ok 2024-09-11T07:30:26.886 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:26 smithi022 ceph-mon[37330]: 5.3 scrub starts 2024-09-11T07:30:26.886 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:26 smithi022 ceph-mon[37330]: 5.3 scrub ok 2024-09-11T07:30:26.886 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:26 smithi022 ceph-mon[37330]: pgmap v155: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 7.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 52 MiB/s wr, 831 op/s 2024-09-11T07:30:26.957 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:26.981 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:26.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:26.982 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:27.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:26 smithi104 ceph-mon[33507]: 7.c scrub ok 2024-09-11T07:30:27.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:26 smithi104 ceph-mon[33507]: 7.8 scrub ok 2024-09-11T07:30:27.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:26 smithi104 ceph-mon[33507]: 5.3 scrub starts 2024-09-11T07:30:27.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:26 smithi104 ceph-mon[33507]: 5.3 scrub ok 2024-09-11T07:30:27.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:26 smithi104 ceph-mon[33507]: pgmap v155: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 7.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 52 MiB/s wr, 831 op/s 2024-09-11T07:30:27.083 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:27.126 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:27.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:27.127 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:27.228 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:27.298 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:27.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:27.299 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:27.400 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:27.428 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:27.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:27.429 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:27.529 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:27.553 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:27.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:27.554 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:27.596 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 85 16 119434 119418 87.7914 61.125 0.0322598 0.0113854 2024-09-11T07:30:27.598 INFO:teuthology.orchestra.run.smithi022.stderr:clear 4.1f pg_upmap_items mapping 2024-09-11T07:30:27.655 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:27.684 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:27.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:27.685 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:27.786 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:27.827 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:27.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:27.828 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:27.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:27 smithi022 ceph-mon[37330]: 6.e scrub starts 2024-09-11T07:30:27.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:27 smithi022 ceph-mon[37330]: 6.e scrub ok 2024-09-11T07:30:27.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:27 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3113594440' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.1f"}]: dispatch 2024-09-11T07:30:27.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:27 smithi022 ceph-mon[29462]: 6.e scrub starts 2024-09-11T07:30:27.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:27 smithi022 ceph-mon[29462]: 6.e scrub ok 2024-09-11T07:30:27.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:27 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3113594440' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.1f"}]: dispatch 2024-09-11T07:30:27.929 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:27.954 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:27.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:27.955 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:28.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:27 smithi104 ceph-mon[33507]: 6.e scrub starts 2024-09-11T07:30:28.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:27 smithi104 ceph-mon[33507]: 6.e scrub ok 2024-09-11T07:30:28.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:27 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3113594440' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.1f"}]: dispatch 2024-09-11T07:30:28.056 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:28.081 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:28.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:28.082 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:28.183 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:28.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:28.227 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:28.227 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:28.328 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:28.354 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:28.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:28.355 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:28.456 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:28.496 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:28.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:28.497 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:28.596 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 86 16 120315 120299 87.4108 55.0625 0.0132847 0.0114344 2024-09-11T07:30:28.598 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:28.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:28.638 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:28.638 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:28.739 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:28.761 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:28.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:28.762 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:28.863 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:28.901 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:28.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:28.902 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:28.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:28 smithi022 ceph-mon[37330]: 6.15 scrub starts 2024-09-11T07:30:28.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:28 smithi022 ceph-mon[37330]: 6.15 scrub ok 2024-09-11T07:30:28.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:28 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3113594440' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.1f"}]': finished 2024-09-11T07:30:28.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:28 smithi022 ceph-mon[37330]: osdmap e126: 8 total, 8 up, 8 in 2024-09-11T07:30:28.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:28 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:30:28.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:28 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:30:28.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:28 smithi022 ceph-mon[37330]: pgmap v157: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 7.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 52 MiB/s wr, 831 op/s 2024-09-11T07:30:28.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:28 smithi022 ceph-mon[37330]: osdmap e127: 8 total, 8 up, 8 in 2024-09-11T07:30:28.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:28 smithi022 ceph-mon[29462]: 6.15 scrub starts 2024-09-11T07:30:28.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:28 smithi022 ceph-mon[29462]: 6.15 scrub ok 2024-09-11T07:30:28.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:28 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3113594440' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.1f"}]': finished 2024-09-11T07:30:28.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:28 smithi022 ceph-mon[29462]: osdmap e126: 8 total, 8 up, 8 in 2024-09-11T07:30:28.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:28 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:30:28.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:28 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:30:28.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:28 smithi022 ceph-mon[29462]: pgmap v157: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 7.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 52 MiB/s wr, 831 op/s 2024-09-11T07:30:28.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:28 smithi022 ceph-mon[29462]: osdmap e127: 8 total, 8 up, 8 in 2024-09-11T07:30:29.004 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:29.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:28 smithi104 ceph-mon[33507]: 6.15 scrub starts 2024-09-11T07:30:29.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:28 smithi104 ceph-mon[33507]: 6.15 scrub ok 2024-09-11T07:30:29.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:28 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3113594440' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.1f"}]': finished 2024-09-11T07:30:29.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:28 smithi104 ceph-mon[33507]: osdmap e126: 8 total, 8 up, 8 in 2024-09-11T07:30:29.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:28 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:30:29.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:28 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:30:29.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:28 smithi104 ceph-mon[33507]: pgmap v157: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 7.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 52 MiB/s wr, 831 op/s 2024-09-11T07:30:29.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:28 smithi104 ceph-mon[33507]: osdmap e127: 8 total, 8 up, 8 in 2024-09-11T07:30:29.051 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:29.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:29.052 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:29.152 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:29.174 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:29.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:29.175 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:29.275 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:29.301 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:29.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:29.303 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:29.403 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:29.433 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:29.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:29.434 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:29.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:30:29 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:30:29.535 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:29.557 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:29.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:29.558 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:29.596 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 87 16 120979 120963 86.8831 41.5 0.0736277 0.0115002 2024-09-11T07:30:29.658 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:29.699 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:29.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:29.701 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:29.802 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:29.823 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:29.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:29.824 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:29.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:29 smithi022 ceph-mon[29462]: 5.1 scrub starts 2024-09-11T07:30:29.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:29 smithi022 ceph-mon[29462]: 5.1 scrub ok 2024-09-11T07:30:29.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:29 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:29.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:29 smithi022 ceph-mon[37330]: 5.1 scrub starts 2024-09-11T07:30:29.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:29 smithi022 ceph-mon[37330]: 5.1 scrub ok 2024-09-11T07:30:29.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:29 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:29.925 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:29.969 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:29.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:29.970 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:30.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:29 smithi104 ceph-mon[33507]: 5.1 scrub starts 2024-09-11T07:30:30.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:29 smithi104 ceph-mon[33507]: 5.1 scrub ok 2024-09-11T07:30:30.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:29 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:30.070 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:30.096 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:30.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:30.097 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:30.198 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:30.225 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:30.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:30.226 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:30.327 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:30.354 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:30.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:30.355 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:30.456 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:30.481 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:30.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:30.482 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:30.583 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:30.596 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 88 16 121838 121822 86.5057 53.6875 0.014177 0.0115548 2024-09-11T07:30:30.626 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:30.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:30.628 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:30.635 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:30:30 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:30:30] "GET /metrics HTTP/1.1" 200 223675 "" "Prometheus/2.43.0" 2024-09-11T07:30:30.729 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:30.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:30.754 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:30.754 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:30.855 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:30.881 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:30.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:30.882 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:30.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:30 smithi022 ceph-mon[29462]: 4.4 scrub starts 2024-09-11T07:30:30.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:30 smithi022 ceph-mon[29462]: 4.4 scrub ok 2024-09-11T07:30:30.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:30 smithi022 ceph-mon[29462]: 7.7 scrub starts 2024-09-11T07:30:30.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:30 smithi022 ceph-mon[29462]: 4.b scrub starts 2024-09-11T07:30:30.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:30 smithi022 ceph-mon[29462]: 4.b scrub ok 2024-09-11T07:30:30.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:30 smithi022 ceph-mon[29462]: 7.e scrub ok 2024-09-11T07:30:30.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:30 smithi022 ceph-mon[29462]: pgmap v159: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.7 KiB/s rd, 81 MiB/s wr, 1.29k op/s; 18 B/s, 0 objects/s recovering 2024-09-11T07:30:30.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:30 smithi022 ceph-mon[37330]: 4.4 scrub starts 2024-09-11T07:30:30.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:30 smithi022 ceph-mon[37330]: 4.4 scrub ok 2024-09-11T07:30:30.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:30 smithi022 ceph-mon[37330]: 7.7 scrub starts 2024-09-11T07:30:30.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:30 smithi022 ceph-mon[37330]: 4.b scrub starts 2024-09-11T07:30:30.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:30 smithi022 ceph-mon[37330]: 4.b scrub ok 2024-09-11T07:30:30.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:30 smithi022 ceph-mon[37330]: 7.e scrub ok 2024-09-11T07:30:30.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:30 smithi022 ceph-mon[37330]: pgmap v159: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.7 KiB/s rd, 81 MiB/s wr, 1.29k op/s; 18 B/s, 0 objects/s recovering 2024-09-11T07:30:30.983 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:31.024 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:31.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:31.025 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:31.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:30 smithi104 ceph-mon[33507]: 4.4 scrub starts 2024-09-11T07:30:31.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:30 smithi104 ceph-mon[33507]: 4.4 scrub ok 2024-09-11T07:30:31.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:30 smithi104 ceph-mon[33507]: 7.7 scrub starts 2024-09-11T07:30:31.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:30 smithi104 ceph-mon[33507]: 4.b scrub starts 2024-09-11T07:30:31.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:30 smithi104 ceph-mon[33507]: 4.b scrub ok 2024-09-11T07:30:31.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:30 smithi104 ceph-mon[33507]: 7.e scrub ok 2024-09-11T07:30:31.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:30 smithi104 ceph-mon[33507]: pgmap v159: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.7 KiB/s rd, 81 MiB/s wr, 1.29k op/s; 18 B/s, 0 objects/s recovering 2024-09-11T07:30:31.125 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:31.156 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:31.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:31.157 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:31.258 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:31.287 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:31.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:31.288 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:31.389 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:31.420 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:31.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:31.421 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:31.522 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:31.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:31.570 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:31.570 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:31.596 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 89 16 122851 122835 86.245 63.3125 0.0270503 0.0115899 2024-09-11T07:30:31.671 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:31.692 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:31 smithi022 ceph-mon[29462]: 6.7 scrub starts 2024-09-11T07:30:31.692 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:31 smithi022 ceph-mon[29462]: 6.7 scrub ok 2024-09-11T07:30:31.692 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:31 smithi022 ceph-mon[29462]: 4.1d deep-scrub starts 2024-09-11T07:30:31.692 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:31 smithi022 ceph-mon[29462]: 4.1d deep-scrub ok 2024-09-11T07:30:31.692 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:31 smithi022 ceph-mon[29462]: 5.c scrub starts 2024-09-11T07:30:31.693 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:31 smithi022 ceph-mon[29462]: 5.c scrub ok 2024-09-11T07:30:31.693 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:31 smithi022 ceph-mon[37330]: 6.7 scrub starts 2024-09-11T07:30:31.693 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:31 smithi022 ceph-mon[37330]: 6.7 scrub ok 2024-09-11T07:30:31.693 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:31 smithi022 ceph-mon[37330]: 4.1d deep-scrub starts 2024-09-11T07:30:31.693 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:31 smithi022 ceph-mon[37330]: 4.1d deep-scrub ok 2024-09-11T07:30:31.693 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:31 smithi022 ceph-mon[37330]: 5.c scrub starts 2024-09-11T07:30:31.693 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:31 smithi022 ceph-mon[37330]: 5.c scrub ok 2024-09-11T07:30:31.714 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:31.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:31.715 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:31.816 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:31.859 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:31.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:31.860 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:31.914 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:30:31.915 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:30:31.946 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:30:31.947 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:30:31.961 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:31.971 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-11T07:30:31.983 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:30:31.984 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:30:31.987 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:31.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:31.988 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:32.021 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:30:32.021 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:30:32.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:31 smithi104 ceph-mon[33507]: 6.7 scrub starts 2024-09-11T07:30:32.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:31 smithi104 ceph-mon[33507]: 6.7 scrub ok 2024-09-11T07:30:32.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:31 smithi104 ceph-mon[33507]: 4.1d deep-scrub starts 2024-09-11T07:30:32.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:31 smithi104 ceph-mon[33507]: 4.1d deep-scrub ok 2024-09-11T07:30:32.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:31 smithi104 ceph-mon[33507]: 5.c scrub starts 2024-09-11T07:30:32.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:31 smithi104 ceph-mon[33507]: 5.c scrub ok 2024-09-11T07:30:32.050 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:30:32.051 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:30:32.089 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:32.090 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:30:32.090 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:30:32.123 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:30:32.123 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:30:32.152 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:30:32.152 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:30:32.153 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:32.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:32.154 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:32.255 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:32.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:32.304 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:32.304 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:32.405 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:32.433 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:32.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:32.434 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:32.535 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:32.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:32.566 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:32.566 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:32.596 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Total time run: 90.0051 2024-09-11T07:30:32.596 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Total writes made: 123752 2024-09-11T07:30:32.596 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Write size: 65536 2024-09-11T07:30:32.597 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Object size: 65536 2024-09-11T07:30:32.597 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Bandwidth (MB/sec): 85.934 2024-09-11T07:30:32.597 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Stddev Bandwidth: 34.7742 2024-09-11T07:30:32.597 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Max bandwidth (MB/sec): 217.25 2024-09-11T07:30:32.597 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Min bandwidth (MB/sec): 41.5 2024-09-11T07:30:32.597 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Average IOPS: 1374 2024-09-11T07:30:32.597 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Stddev IOPS: 556.386 2024-09-11T07:30:32.597 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Max IOPS: 3476 2024-09-11T07:30:32.597 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Min IOPS: 664 2024-09-11T07:30:32.597 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Average Latency(s): 0.0116329 2024-09-11T07:30:32.597 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Stddev Latency(s): 0.00859828 2024-09-11T07:30:32.597 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Max latency(s): 0.229521 2024-09-11T07:30:32.597 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Min latency(s): 0.00117769 2024-09-11T07:30:32.601 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Cleaning up (deleting benchmark objects) 2024-09-11T07:30:32.666 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:32.697 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:32.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:32.697 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:32.711 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:32.798 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:32.843 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:32.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:32.844 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:32.855 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:32 smithi022 ceph-mon[29462]: 7.1 scrub starts 2024-09-11T07:30:32.855 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:32 smithi022 ceph-mon[29462]: 6.d deep-scrub starts 2024-09-11T07:30:32.855 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:32 smithi022 ceph-mon[29462]: 6.d deep-scrub ok 2024-09-11T07:30:32.855 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:32 smithi022 ceph-mon[29462]: 6.1f scrub starts 2024-09-11T07:30:32.855 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:32 smithi022 ceph-mon[29462]: 6.1f scrub ok 2024-09-11T07:30:32.855 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:32 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3471207549' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:30:32.855 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:32 smithi022 ceph-mon[29462]: pgmap v160: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 784 B/s rd, 45 MiB/s wr, 719 op/s; 16 B/s, 0 objects/s recovering 2024-09-11T07:30:32.855 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:32 smithi022 ceph-mon[29462]: 2.b scrub starts 2024-09-11T07:30:32.855 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:32 smithi022 ceph-mon[29462]: 2.b scrub ok 2024-09-11T07:30:32.856 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:32 smithi022 ceph-mon[37330]: 7.1 scrub starts 2024-09-11T07:30:32.856 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:32 smithi022 ceph-mon[37330]: 6.d deep-scrub starts 2024-09-11T07:30:32.856 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:32 smithi022 ceph-mon[37330]: 6.d deep-scrub ok 2024-09-11T07:30:32.856 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:32 smithi022 ceph-mon[37330]: 6.1f scrub starts 2024-09-11T07:30:32.856 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:32 smithi022 ceph-mon[37330]: 6.1f scrub ok 2024-09-11T07:30:32.856 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:32 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3471207549' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:30:32.856 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:32 smithi022 ceph-mon[37330]: pgmap v160: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 784 B/s rd, 45 MiB/s wr, 719 op/s; 16 B/s, 0 objects/s recovering 2024-09-11T07:30:32.856 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:32 smithi022 ceph-mon[37330]: 2.b scrub starts 2024-09-11T07:30:32.856 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:32 smithi022 ceph-mon[37330]: 2.b scrub ok 2024-09-11T07:30:32.945 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:32.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:32.997 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:32.997 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:33.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:32 smithi104 ceph-mon[33507]: 7.1 scrub starts 2024-09-11T07:30:33.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:32 smithi104 ceph-mon[33507]: 6.d deep-scrub starts 2024-09-11T07:30:33.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:32 smithi104 ceph-mon[33507]: 6.d deep-scrub ok 2024-09-11T07:30:33.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:32 smithi104 ceph-mon[33507]: 6.1f scrub starts 2024-09-11T07:30:33.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:32 smithi104 ceph-mon[33507]: 6.1f scrub ok 2024-09-11T07:30:33.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:32 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3471207549' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:30:33.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:32 smithi104 ceph-mon[33507]: pgmap v160: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 784 B/s rd, 45 MiB/s wr, 719 op/s; 16 B/s, 0 objects/s recovering 2024-09-11T07:30:33.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:32 smithi104 ceph-mon[33507]: 2.b scrub starts 2024-09-11T07:30:33.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:32 smithi104 ceph-mon[33507]: 2.b scrub ok 2024-09-11T07:30:33.098 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:33.123 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:33.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:33.124 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:33.225 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:33.254 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:33.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:33.255 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:33.355 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:33.381 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:33.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:33.382 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:33.482 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:33.510 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:33.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:33.511 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:33.611 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:33.636 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:33.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:33.637 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:33.738 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:33.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:33.780 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:33.823 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:33 smithi022 ceph-mon[29462]: 4.1b scrub starts 2024-09-11T07:30:33.823 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:33 smithi022 ceph-mon[29462]: 4.1b scrub ok 2024-09-11T07:30:33.823 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:33 smithi022 ceph-mon[29462]: 5.12 scrub starts 2024-09-11T07:30:33.823 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:33 smithi022 ceph-mon[29462]: 5.12 scrub ok 2024-09-11T07:30:33.823 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:33 smithi022 ceph-mon[29462]: 5.5 scrub starts 2024-09-11T07:30:33.823 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:33 smithi022 ceph-mon[29462]: 5.5 scrub ok 2024-09-11T07:30:33.823 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:33 smithi022 ceph-mon[37330]: 4.1b scrub starts 2024-09-11T07:30:33.823 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:33 smithi022 ceph-mon[37330]: 4.1b scrub ok 2024-09-11T07:30:33.824 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:33 smithi022 ceph-mon[37330]: 5.12 scrub starts 2024-09-11T07:30:33.824 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:33 smithi022 ceph-mon[37330]: 5.12 scrub ok 2024-09-11T07:30:33.824 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:33 smithi022 ceph-mon[37330]: 5.5 scrub starts 2024-09-11T07:30:33.824 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:33 smithi022 ceph-mon[37330]: 5.5 scrub ok 2024-09-11T07:30:33.880 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:33.904 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:33.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:33.905 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:34.006 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:34.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:33 smithi104 ceph-mon[33507]: 4.1b scrub starts 2024-09-11T07:30:34.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:33 smithi104 ceph-mon[33507]: 4.1b scrub ok 2024-09-11T07:30:34.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:33 smithi104 ceph-mon[33507]: 5.12 scrub starts 2024-09-11T07:30:34.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:33 smithi104 ceph-mon[33507]: 5.12 scrub ok 2024-09-11T07:30:34.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:33 smithi104 ceph-mon[33507]: 5.5 scrub starts 2024-09-11T07:30:34.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:33 smithi104 ceph-mon[33507]: 5.5 scrub ok 2024-09-11T07:30:34.046 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:34.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:34.047 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:34.148 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:34.183 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:34.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:34.184 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:34.285 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:34.329 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:34.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:34.330 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:34.350 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:34.431 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:34.478 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:34.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:34.479 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:34.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:30:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:30:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:30:34.580 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:34.607 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:34.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:34.608 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:34.709 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:34.780 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:34.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:34.781 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:34.882 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:34.908 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:34.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:34.909 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:34.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:34 smithi022 ceph-mon[29462]: 7.7 scrub ok 2024-09-11T07:30:34.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:34 smithi022 ceph-mon[29462]: 7.1 scrub ok 2024-09-11T07:30:34.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:34 smithi022 ceph-mon[29462]: 6.1 scrub starts 2024-09-11T07:30:34.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:34 smithi022 ceph-mon[29462]: 6.1 scrub ok 2024-09-11T07:30:34.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:34 smithi022 ceph-mon[29462]: 4.19 scrub starts 2024-09-11T07:30:34.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:34 smithi022 ceph-mon[29462]: 4.19 scrub ok 2024-09-11T07:30:34.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:34 smithi022 ceph-mon[29462]: pgmap v161: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.4 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 766 B/s rd, 46 MiB/s wr, 812 op/s; 13 B/s, 0 objects/s recovering 2024-09-11T07:30:34.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:34 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:30:34.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:34 smithi022 ceph-mon[37330]: 7.7 scrub ok 2024-09-11T07:30:34.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:34 smithi022 ceph-mon[37330]: 7.1 scrub ok 2024-09-11T07:30:34.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:34 smithi022 ceph-mon[37330]: 6.1 scrub starts 2024-09-11T07:30:34.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:34 smithi022 ceph-mon[37330]: 6.1 scrub ok 2024-09-11T07:30:34.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:34 smithi022 ceph-mon[37330]: 4.19 scrub starts 2024-09-11T07:30:34.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:34 smithi022 ceph-mon[37330]: 4.19 scrub ok 2024-09-11T07:30:34.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:34 smithi022 ceph-mon[37330]: pgmap v161: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.4 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 766 B/s rd, 46 MiB/s wr, 812 op/s; 13 B/s, 0 objects/s recovering 2024-09-11T07:30:34.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:34 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:30:35.010 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:35.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:34 smithi104 ceph-mon[33507]: 7.7 scrub ok 2024-09-11T07:30:35.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:34 smithi104 ceph-mon[33507]: 7.1 scrub ok 2024-09-11T07:30:35.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:34 smithi104 ceph-mon[33507]: 6.1 scrub starts 2024-09-11T07:30:35.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:34 smithi104 ceph-mon[33507]: 6.1 scrub ok 2024-09-11T07:30:35.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:34 smithi104 ceph-mon[33507]: 4.19 scrub starts 2024-09-11T07:30:35.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:34 smithi104 ceph-mon[33507]: 4.19 scrub ok 2024-09-11T07:30:35.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:34 smithi104 ceph-mon[33507]: pgmap v161: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.4 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 766 B/s rd, 46 MiB/s wr, 812 op/s; 13 B/s, 0 objects/s recovering 2024-09-11T07:30:35.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:34 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:30:35.033 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:35.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:35.034 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:35.135 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:35.176 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:35.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:35.177 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:35.278 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:35.322 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:35.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:35.323 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:35.424 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:35.449 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:35.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:35.450 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:35.551 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:35.578 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:35.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:35.579 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:35.680 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:35.706 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:35.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:35.707 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:35.808 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:35.832 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:35.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:35.833 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:35.892 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:35 smithi022 ceph-mon[29462]: 4.7 scrub starts 2024-09-11T07:30:35.892 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:35 smithi022 ceph-mon[29462]: 4.7 scrub ok 2024-09-11T07:30:35.892 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:35 smithi022 ceph-mon[29462]: 5.0 scrub starts 2024-09-11T07:30:35.892 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:35 smithi022 ceph-mon[29462]: 5.0 scrub ok 2024-09-11T07:30:35.892 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:35 smithi022 ceph-mon[29462]: 6.9 scrub starts 2024-09-11T07:30:35.892 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:35 smithi022 ceph-mon[29462]: 6.9 scrub ok 2024-09-11T07:30:35.892 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:35 smithi022 ceph-mon[29462]: 6.12 scrub starts 2024-09-11T07:30:35.892 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:35 smithi022 ceph-mon[29462]: 6.12 scrub ok 2024-09-11T07:30:35.893 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:35 smithi022 ceph-mon[37330]: 4.7 scrub starts 2024-09-11T07:30:35.893 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:35 smithi022 ceph-mon[37330]: 4.7 scrub ok 2024-09-11T07:30:35.893 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:35 smithi022 ceph-mon[37330]: 5.0 scrub starts 2024-09-11T07:30:35.893 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:35 smithi022 ceph-mon[37330]: 5.0 scrub ok 2024-09-11T07:30:35.893 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:35 smithi022 ceph-mon[37330]: 6.9 scrub starts 2024-09-11T07:30:35.893 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:35 smithi022 ceph-mon[37330]: 6.9 scrub ok 2024-09-11T07:30:35.893 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:35 smithi022 ceph-mon[37330]: 6.12 scrub starts 2024-09-11T07:30:35.893 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:35 smithi022 ceph-mon[37330]: 6.12 scrub ok 2024-09-11T07:30:35.934 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:35.970 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:35.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:35.971 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:36.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:35 smithi104 ceph-mon[33507]: 4.7 scrub starts 2024-09-11T07:30:36.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:35 smithi104 ceph-mon[33507]: 4.7 scrub ok 2024-09-11T07:30:36.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:35 smithi104 ceph-mon[33507]: 5.0 scrub starts 2024-09-11T07:30:36.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:35 smithi104 ceph-mon[33507]: 5.0 scrub ok 2024-09-11T07:30:36.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:35 smithi104 ceph-mon[33507]: 6.9 scrub starts 2024-09-11T07:30:36.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:35 smithi104 ceph-mon[33507]: 6.9 scrub ok 2024-09-11T07:30:36.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:35 smithi104 ceph-mon[33507]: 6.12 scrub starts 2024-09-11T07:30:36.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:35 smithi104 ceph-mon[33507]: 6.12 scrub ok 2024-09-11T07:30:36.071 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:36.111 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:36.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:36.112 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:36.213 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:36.238 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:36.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:36.239 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:36.339 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:36.364 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:36.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:36.365 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:36.381 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:30:36.466 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:36.490 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:36.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:36.491 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:36.592 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:36.616 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:36.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:36.617 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:36.700 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is set 2024-09-11T07:30:36.701 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:36 smithi022 ceph-mon[29462]: 5.17 deep-scrub starts 2024-09-11T07:30:36.701 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:36 smithi022 ceph-mon[29462]: 5.17 deep-scrub ok 2024-09-11T07:30:36.701 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:36 smithi022 ceph-mon[29462]: 4.5 scrub starts 2024-09-11T07:30:36.701 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:36 smithi022 ceph-mon[29462]: 4.5 scrub ok 2024-09-11T07:30:36.702 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:36 smithi022 ceph-mon[29462]: pgmap v162: 148 pgs: 148 active+clean; 7.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.3 KiB/s rd, 58 MiB/s wr, 1.13k op/s; 13 B/s, 0 objects/s recovering 2024-09-11T07:30:36.702 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:36 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1804146159' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:30:36.702 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:36 smithi022 ceph-mon[29462]: 6.5 scrub starts 2024-09-11T07:30:36.702 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:36 smithi022 ceph-mon[29462]: 6.5 scrub ok 2024-09-11T07:30:36.702 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:36 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:30:36.702 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:36 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:30:36.702 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:36 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:30:36.702 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:36 smithi022 ceph-mon[37330]: 5.17 deep-scrub starts 2024-09-11T07:30:36.702 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:36 smithi022 ceph-mon[37330]: 5.17 deep-scrub ok 2024-09-11T07:30:36.702 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:36 smithi022 ceph-mon[37330]: 4.5 scrub starts 2024-09-11T07:30:36.702 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:36 smithi022 ceph-mon[37330]: 4.5 scrub ok 2024-09-11T07:30:36.703 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:36 smithi022 ceph-mon[37330]: pgmap v162: 148 pgs: 148 active+clean; 7.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.3 KiB/s rd, 58 MiB/s wr, 1.13k op/s; 13 B/s, 0 objects/s recovering 2024-09-11T07:30:36.703 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:36 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1804146159' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:30:36.703 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:36 smithi022 ceph-mon[37330]: 6.5 scrub starts 2024-09-11T07:30:36.703 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:36 smithi022 ceph-mon[37330]: 6.5 scrub ok 2024-09-11T07:30:36.703 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:36 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:30:36.703 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:36 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:30:36.703 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:36 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:30:36.718 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:36.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:36.771 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:36.771 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:36.872 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:36.919 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:36.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:36.920 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:37.021 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:36 smithi104 ceph-mon[33507]: 5.17 deep-scrub starts 2024-09-11T07:30:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:36 smithi104 ceph-mon[33507]: 5.17 deep-scrub ok 2024-09-11T07:30:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:36 smithi104 ceph-mon[33507]: 4.5 scrub starts 2024-09-11T07:30:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:36 smithi104 ceph-mon[33507]: 4.5 scrub ok 2024-09-11T07:30:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:36 smithi104 ceph-mon[33507]: pgmap v162: 148 pgs: 148 active+clean; 7.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.3 KiB/s rd, 58 MiB/s wr, 1.13k op/s; 13 B/s, 0 objects/s recovering 2024-09-11T07:30:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:36 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1804146159' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:30:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:36 smithi104 ceph-mon[33507]: 6.5 scrub starts 2024-09-11T07:30:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:36 smithi104 ceph-mon[33507]: 6.5 scrub ok 2024-09-11T07:30:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:36 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:30:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:36 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:30:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:36 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:30:37.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:37.069 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:37.069 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:37.170 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:37.195 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:37.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:37.196 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:37.297 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:37.337 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:37.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:37.338 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:37.439 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:37.481 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:37.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:37.482 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:37.496 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:30:37.496 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:30:37.496 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-11T07:30:37.496 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T07:30:37.496 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T07:30:37.497 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.4 config set filestore_inject_stall 3 2024-09-11T07:30:37.583 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:37.613 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:37.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:37.614 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:37.714 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:37.755 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:37.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:37.756 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:37.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:37 smithi104 ceph-mon[33507]: 2.1a scrub starts 2024-09-11T07:30:37.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:37 smithi104 ceph-mon[33507]: 2.1a scrub ok 2024-09-11T07:30:37.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:37 smithi104 ceph-mon[33507]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:30:37.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:37 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1804146159' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:30:37.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:37 smithi104 ceph-mon[33507]: osdmap e128: 8 total, 8 up, 8 in 2024-09-11T07:30:37.837 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:30:37.856 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:37.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:37.898 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:37.898 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:37.991 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:37 smithi022 ceph-mon[29462]: 2.1a scrub starts 2024-09-11T07:30:37.991 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:37 smithi022 ceph-mon[29462]: 2.1a scrub ok 2024-09-11T07:30:37.991 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:37 smithi022 ceph-mon[29462]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:30:37.991 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:37 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1804146159' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:30:37.991 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:37 smithi022 ceph-mon[29462]: osdmap e128: 8 total, 8 up, 8 in 2024-09-11T07:30:37.993 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:37 smithi022 ceph-mon[37330]: 2.1a scrub starts 2024-09-11T07:30:37.993 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:37 smithi022 ceph-mon[37330]: 2.1a scrub ok 2024-09-11T07:30:37.993 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:37 smithi022 ceph-mon[37330]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:30:37.993 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:37 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1804146159' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:30:37.993 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:37 smithi022 ceph-mon[37330]: osdmap e128: 8 total, 8 up, 8 in 2024-09-11T07:30:37.999 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:38.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:38.058 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:38.058 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:38.155 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:38.158 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:38.209 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:38.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:38.210 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:38.311 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:38.355 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:38.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:38.356 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:38.457 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:38.496 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:38.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:38.497 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:38.598 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:38.626 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:38.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:38.627 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:38.719 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:30:38.719 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:30:38.728 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:38.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:38.770 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:38.770 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:38.871 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:38.909 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:30:38.910 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:38 smithi022 ceph-mon[29462]: pgmap v164: 148 pgs: 148 active+clean; 7.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 52 MiB/s wr, 1.01k op/s; 11 B/s, 0 objects/s recovering 2024-09-11T07:30:38.911 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:30:38.911 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:38 smithi022 ceph-mon[37330]: pgmap v164: 148 pgs: 148 active+clean; 7.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 52 MiB/s wr, 1.01k op/s; 11 B/s, 0 objects/s recovering 2024-09-11T07:30:38.923 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:38.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:38.924 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:39.022 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:30:39.022 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:38 smithi104 ceph-mon[33507]: pgmap v164: 148 pgs: 148 active+clean; 7.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 52 MiB/s wr, 1.01k op/s; 11 B/s, 0 objects/s recovering 2024-09-11T07:30:39.024 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:39.067 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:39.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:39.068 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:39.169 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:39.193 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:39.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:39.194 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:39.279 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:30:39 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:30:39.295 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:39.339 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:39.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:39.340 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:39.396 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T07:30:39.441 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:39.509 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:39.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:39.510 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:39.567 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-11T07:30:39.610 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:39.634 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:39.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:39.635 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:39.695 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:39 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:39.736 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:39.788 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:39.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:39.789 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:39.890 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:39.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:39.919 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:39.919 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:39.965 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:39 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:39.991 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:30:39.992 INFO:teuthology.orchestra.run.smithi104.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-11T07:30:39.992 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:30:40.020 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:40.075 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:40.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:40.076 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:40.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:39 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:40.176 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:40.208 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:40.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:40.209 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:40.310 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:40.349 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:40.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:40.350 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:40.451 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:40.478 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:30:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:30:40] "GET /metrics HTTP/1.1" 200 223717 "" "Prometheus/2.43.0" 2024-09-11T07:30:40.498 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:40.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:40.500 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:40.587 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:40.600 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:40.600 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:40.628 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:40.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:40.629 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:40.729 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:40.731 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:40 smithi022 ceph-mon[29462]: pgmap v165: 148 pgs: 148 active+clean; 7.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 20 MiB/s wr, 1.01k op/s 2024-09-11T07:30:40.732 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:40 smithi022 ceph-mon[37330]: pgmap v165: 148 pgs: 148 active+clean; 7.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 20 MiB/s wr, 1.01k op/s 2024-09-11T07:30:40.772 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:40.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:40.773 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:40.874 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:40.917 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:40.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:40.918 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:41.019 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:41.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:40 smithi104 ceph-mon[33507]: pgmap v165: 148 pgs: 148 active+clean; 7.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 20 MiB/s wr, 1.01k op/s 2024-09-11T07:30:41.047 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:41.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:41.047 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:41.148 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:41.177 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:41.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:41.178 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:41.279 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:41.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:41.314 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:41.314 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:41.415 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:41.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:41.444 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:41.444 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:41.545 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:41.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:41.585 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:41.585 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:41.686 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:41.725 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:41.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:41.726 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:41.827 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:41.866 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:41.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:41.867 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:41.968 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:42.009 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:42.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:42.010 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:42.111 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:42.139 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:42.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:42.140 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:42.240 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:42.281 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:42.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:42.282 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:42.383 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:42.421 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:42.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:42.422 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:42.523 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:42.550 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:42.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:42.551 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:42.652 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:42.696 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:42.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:42.697 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:42.798 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:42.823 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:42.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:42.824 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:42.925 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:42.948 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:42.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:42.949 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:42.962 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:30:42.962 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:30:42.995 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:30:42.995 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:30:43.034 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:30:43.034 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:30:43.050 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:43.069 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:30:43.069 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:30:43.074 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:43.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:43.076 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:43.100 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:30:43.100 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:30:43.133 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:30:43.134 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:30:43.166 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:30:43.167 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:30:43.176 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:43.196 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:30:43.196 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:30:43.202 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:43.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:43.203 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:43.304 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:43.342 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:43 smithi022 ceph-mon[29462]: pgmap v166: 148 pgs: 148 active+clean; 7.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 20 MiB/s wr, 1.01k op/s 2024-09-11T07:30:43.342 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:43 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/342550575' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:30:43.342 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:43 smithi022 ceph-mon[37330]: pgmap v166: 148 pgs: 148 active+clean; 7.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 20 MiB/s wr, 1.01k op/s 2024-09-11T07:30:43.342 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:43 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/342550575' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:30:43.351 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:43.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:43.352 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:43.453 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:43.507 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:43.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:43.508 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:43.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:43 smithi104 ceph-mon[33507]: pgmap v166: 148 pgs: 148 active+clean; 7.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 20 MiB/s wr, 1.01k op/s 2024-09-11T07:30:43.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:43 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/342550575' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:30:43.609 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:43.635 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:43.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:43.636 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:43.737 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:43.783 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:43.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:43.784 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:43.884 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:43.925 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:43.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:43.926 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:44.027 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:44.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:44.070 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:44.070 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:44.171 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:44.211 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:44.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:44.212 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:44.313 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:44.339 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:44.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:44.340 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:44.441 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:44.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:44.477 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:44.477 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:44.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:30:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:30:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:30:44.578 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:44.621 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:44.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:44.622 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:44.723 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:44.748 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:44.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:44.749 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:44.850 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:44.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:44.879 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:44.879 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:44.980 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:45.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:45.023 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:45.023 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:45.094 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:30:45.094 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 20, 2024-09-11T07:30:45.094 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 600, 2024-09-11T07:30:45.095 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:30:45.095 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.095 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:119855 7.a 7:5e2f0bbe:::benchmark_data_smithi022_77720_object119854:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-11T07:30:45.095 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:28.041084+0000", 2024-09-11T07:30:45.095 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 17.048952092, 2024-09-11T07:30:45.095 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.134808968, 2024-09-11T07:30:45.095 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:45.095 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:45.095 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:45.095 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:45.095 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:45.095 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 119855 2024-09-11T07:30:45.096 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.096 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:45.096 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.096 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:45.096 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.041084+0000", 2024-09-11T07:30:45.096 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.096 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.096 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.096 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:45.097 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.041084+0000", 2024-09-11T07:30:45.097 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.097 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.097 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.097 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:45.097 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.041082+0000", 2024-09-11T07:30:45.097 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:30:45.097 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.097 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.097 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:45.097 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.041114+0000", 2024-09-11T07:30:45.097 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2254e-05 2024-09-11T07:30:45.098 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.098 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.098 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:45.098 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.043456+0000", 2024-09-11T07:30:45.098 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002342078 2024-09-11T07:30:45.098 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.098 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.098 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:45.098 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.043468+0000", 2024-09-11T07:30:45.098 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1489e-05 2024-09-11T07:30:45.098 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.098 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.098 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:45.099 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.051188+0000", 2024-09-11T07:30:45.099 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0077202800000000004 2024-09-11T07:30:45.099 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.099 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.099 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:45.099 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.051309+0000", 2024-09-11T07:30:45.099 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012084899999999999 2024-09-11T07:30:45.099 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.099 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.099 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,5", 2024-09-11T07:30:45.099 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.051446+0000", 2024-09-11T07:30:45.099 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013662799999999999 2024-09-11T07:30:45.099 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.100 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.100 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:45.100 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.057558+0000", 2024-09-11T07:30:45.100 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0061127999999999998 2024-09-11T07:30:45.100 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.100 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.100 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:45.100 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.062873+0000", 2024-09-11T07:30:45.100 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0053149429999999999 2024-09-11T07:30:45.100 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.100 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.100 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:45.100 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.175703+0000", 2024-09-11T07:30:45.100 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.112829236 2024-09-11T07:30:45.101 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.101 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.101 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:45.101 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.175853+0000", 2024-09-11T07:30:45.101 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015002799999999999 2024-09-11T07:30:45.101 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.101 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.101 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:45.101 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.175893+0000", 2024-09-11T07:30:45.101 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0182999999999998e-05 2024-09-11T07:30:45.102 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.102 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:45.102 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.102 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.102 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.102 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:119856 7.a 7:5613c109:::benchmark_data_smithi022_77720_object119855:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-11T07:30:45.102 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:28.043477+0000", 2024-09-11T07:30:45.102 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 17.046558769000001, 2024-09-11T07:30:45.102 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13270822299999999, 2024-09-11T07:30:45.102 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:45.102 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:45.102 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:45.102 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:45.102 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:45.103 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 119856 2024-09-11T07:30:45.103 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.103 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:45.103 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.103 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:45.103 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.043477+0000", 2024-09-11T07:30:45.103 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.103 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.103 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.103 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:45.103 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.043477+0000", 2024-09-11T07:30:45.103 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.103 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.103 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.104 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:45.104 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.043476+0000", 2024-09-11T07:30:45.104 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:30:45.104 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.104 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.104 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:45.104 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.043506+0000", 2024-09-11T07:30:45.104 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.9864000000000001e-05 2024-09-11T07:30:45.104 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.104 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.104 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:45.104 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.043515+0000", 2024-09-11T07:30:45.104 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.1640000000000007e-06 2024-09-11T07:30:45.105 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.105 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.105 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:45.105 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.043519+0000", 2024-09-11T07:30:45.105 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1350000000000002e-06 2024-09-11T07:30:45.105 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.105 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.105 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:45.105 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.054190+0000", 2024-09-11T07:30:45.105 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.010671531 2024-09-11T07:30:45.105 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.105 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.105 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:45.106 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.054321+0000", 2024-09-11T07:30:45.106 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013081200000000001 2024-09-11T07:30:45.106 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.106 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.106 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,5", 2024-09-11T07:30:45.106 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.054463+0000", 2024-09-11T07:30:45.106 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000141614 2024-09-11T07:30:45.106 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.106 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.106 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:45.106 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.057590+0000", 2024-09-11T07:30:45.106 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.003127356 2024-09-11T07:30:45.106 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.107 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.107 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:45.107 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.175901+0000", 2024-09-11T07:30:45.107 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.118310726 2024-09-11T07:30:45.107 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.107 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.107 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:45.107 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.176105+0000", 2024-09-11T07:30:45.107 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00020366399999999999 2024-09-11T07:30:45.107 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.107 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.107 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:45.107 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.176150+0000", 2024-09-11T07:30:45.107 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5927999999999998e-05 2024-09-11T07:30:45.107 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.108 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.108 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:45.108 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.176185+0000", 2024-09-11T07:30:45.108 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4944000000000003e-05 2024-09-11T07:30:45.108 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.108 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:45.108 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.108 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.108 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.108 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:119858 7.a 7:5e79791e:::benchmark_data_smithi022_77720_object119857:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-11T07:30:45.108 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:28.043846+0000", 2024-09-11T07:30:45.108 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 17.046190126999999, 2024-09-11T07:30:45.108 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13634125, 2024-09-11T07:30:45.108 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:45.109 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:45.109 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:45.109 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:45.109 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:45.109 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 119858 2024-09-11T07:30:45.109 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.110 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:45.110 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.110 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:45.110 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.043846+0000", 2024-09-11T07:30:45.110 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.110 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.110 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.110 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:45.110 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.043846+0000", 2024-09-11T07:30:45.110 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.110 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.110 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.110 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:45.110 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.043844+0000", 2024-09-11T07:30:45.111 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:30:45.111 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.111 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.111 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:45.111 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.043870+0000", 2024-09-11T07:30:45.111 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5998000000000001e-05 2024-09-11T07:30:45.111 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.111 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.111 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:45.111 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.043883+0000", 2024-09-11T07:30:45.111 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2702e-05 2024-09-11T07:30:45.111 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.111 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.111 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:45.111 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.043889+0000", 2024-09-11T07:30:45.112 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5270000000000004e-06 2024-09-11T07:30:45.112 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.112 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.112 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:45.112 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.059653+0000", 2024-09-11T07:30:45.112 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.015763513999999999 2024-09-11T07:30:45.112 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.112 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.112 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:45.112 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.059784+0000", 2024-09-11T07:30:45.112 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013114600000000001 2024-09-11T07:30:45.112 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.113 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.113 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,5", 2024-09-11T07:30:45.113 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.059925+0000", 2024-09-11T07:30:45.113 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014092600000000001 2024-09-11T07:30:45.113 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.113 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.113 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:45.113 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.062906+0000", 2024-09-11T07:30:45.113 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0029813940000000001 2024-09-11T07:30:45.113 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.113 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.113 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:45.113 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.175945+0000", 2024-09-11T07:30:45.113 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.11303914299999999 2024-09-11T07:30:45.114 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.114 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.114 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:45.114 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.180100+0000", 2024-09-11T07:30:45.114 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0041547889999999999 2024-09-11T07:30:45.114 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.114 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.114 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:45.114 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.180149+0000", 2024-09-11T07:30:45.114 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8874e-05 2024-09-11T07:30:45.114 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.114 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.115 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:45.115 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.180187+0000", 2024-09-11T07:30:45.115 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8084000000000003e-05 2024-09-11T07:30:45.115 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.115 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:45.115 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.115 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.115 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.115 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:120403 7.a 7:53556683:::benchmark_data_smithi022_77720_object120402:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:30:45.115 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:28.687289+0000", 2024-09-11T07:30:45.115 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 16.402747385000001, 2024-09-11T07:30:45.115 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.15092119800000001, 2024-09-11T07:30:45.115 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:45.115 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:45.116 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:45.116 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:45.116 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:45.116 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 120403 2024-09-11T07:30:45.116 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.116 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:45.116 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.116 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:45.116 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.687289+0000", 2024-09-11T07:30:45.116 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.116 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.116 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.116 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:45.116 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.687289+0000", 2024-09-11T07:30:45.117 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.117 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.117 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.117 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:45.117 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.687288+0000", 2024-09-11T07:30:45.117 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:30:45.117 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.117 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.117 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:45.117 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.687396+0000", 2024-09-11T07:30:45.117 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010856 2024-09-11T07:30:45.117 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.117 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.117 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:45.118 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.687420+0000", 2024-09-11T07:30:45.118 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4114e-05 2024-09-11T07:30:45.118 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.118 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.118 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:45.118 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.687446+0000", 2024-09-11T07:30:45.118 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5367e-05 2024-09-11T07:30:45.118 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.118 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.118 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:45.118 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.691757+0000", 2024-09-11T07:30:45.118 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0043116700000000001 2024-09-11T07:30:45.118 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.118 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.118 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:45.119 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.691876+0000", 2024-09-11T07:30:45.119 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000118707 2024-09-11T07:30:45.119 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.119 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.119 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,5", 2024-09-11T07:30:45.119 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.692007+0000", 2024-09-11T07:30:45.119 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013065599999999999 2024-09-11T07:30:45.119 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.119 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.119 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:45.119 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695100+0000", 2024-09-11T07:30:45.119 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0030930789999999999 2024-09-11T07:30:45.119 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.119 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.120 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:45.120 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.700863+0000", 2024-09-11T07:30:45.120 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0057636079999999999 2024-09-11T07:30:45.120 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.120 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.120 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:45.120 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838144+0000", 2024-09-11T07:30:45.120 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13728073099999999 2024-09-11T07:30:45.120 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.120 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.121 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:45.121 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838179+0000", 2024-09-11T07:30:45.121 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5129000000000002e-05 2024-09-11T07:30:45.121 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.121 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.121 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:45.121 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838210+0000", 2024-09-11T07:30:45.121 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0633999999999998e-05 2024-09-11T07:30:45.121 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.121 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:45.121 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.121 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.121 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.121 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:120408 7.2 7:4165bb05:::benchmark_data_smithi022_77720_object120407:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:30:45.122 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:28.688203+0000", 2024-09-11T07:30:45.122 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 16.401833121999999, 2024-09-11T07:30:45.122 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.149908602, 2024-09-11T07:30:45.122 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:45.122 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:45.122 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:45.122 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:45.122 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:45.122 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 120408 2024-09-11T07:30:45.122 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.122 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:45.122 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.122 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:45.123 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.688203+0000", 2024-09-11T07:30:45.123 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.123 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.123 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.123 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:45.123 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.688203+0000", 2024-09-11T07:30:45.123 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.123 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.123 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.123 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:45.123 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.688202+0000", 2024-09-11T07:30:45.123 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:30:45.123 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.123 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.124 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:45.124 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.688225+0000", 2024-09-11T07:30:45.124 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3796999999999999e-05 2024-09-11T07:30:45.124 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.124 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.124 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:45.124 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.688235+0000", 2024-09-11T07:30:45.124 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.7599999999999997e-06 2024-09-11T07:30:45.124 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.124 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.124 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:45.124 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.688241+0000", 2024-09-11T07:30:45.124 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6890000000000001e-06 2024-09-11T07:30:45.124 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.125 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.125 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:45.125 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.691642+0000", 2024-09-11T07:30:45.125 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.003401669 2024-09-11T07:30:45.125 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.125 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.125 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:45.125 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.691766+0000", 2024-09-11T07:30:45.125 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012310700000000001 2024-09-11T07:30:45.125 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.125 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.125 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:30:45.126 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.691893+0000", 2024-09-11T07:30:45.126 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012741099999999999 2024-09-11T07:30:45.126 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.126 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.126 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:45.126 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.700857+0000", 2024-09-11T07:30:45.126 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0089635749999999997 2024-09-11T07:30:45.126 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.126 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.126 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:45.126 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.701785+0000", 2024-09-11T07:30:45.126 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00092816999999999997 2024-09-11T07:30:45.127 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.127 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.127 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:45.127 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838024+0000", 2024-09-11T07:30:45.127 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13623940600000001 2024-09-11T07:30:45.127 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.127 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.127 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:45.127 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838072+0000", 2024-09-11T07:30:45.127 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8340999999999997e-05 2024-09-11T07:30:45.127 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.127 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.127 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:45.128 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838111+0000", 2024-09-11T07:30:45.128 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8998000000000002e-05 2024-09-11T07:30:45.128 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.128 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:45.128 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.128 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.128 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.128 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:120413 7.a 7:570dbbb8:::benchmark_data_smithi022_77720_object120412:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:30:45.128 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:28.694903+0000", 2024-09-11T07:30:45.128 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 16.395132786000001, 2024-09-11T07:30:45.128 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.14339426699999999, 2024-09-11T07:30:45.128 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:45.129 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:45.129 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:45.129 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:45.129 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:45.129 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 120413 2024-09-11T07:30:45.129 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.129 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:45.129 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.129 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:45.129 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.694903+0000", 2024-09-11T07:30:45.129 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.129 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.129 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.129 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:45.130 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.694903+0000", 2024-09-11T07:30:45.130 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.130 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.130 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.130 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:45.130 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.694901+0000", 2024-09-11T07:30:45.130 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:30:45.130 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.130 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.130 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:45.130 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.694926+0000", 2024-09-11T07:30:45.130 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4505999999999999e-05 2024-09-11T07:30:45.131 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.131 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.131 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:45.131 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.694938+0000", 2024-09-11T07:30:45.131 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2326e-05 2024-09-11T07:30:45.131 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.131 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.131 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:45.131 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.694944+0000", 2024-09-11T07:30:45.131 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2299999999999996e-06 2024-09-11T07:30:45.131 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.131 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.132 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:45.132 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695119+0000", 2024-09-11T07:30:45.132 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00017516499999999999 2024-09-11T07:30:45.132 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.132 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.132 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:45.132 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695231+0000", 2024-09-11T07:30:45.132 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011134800000000001 2024-09-11T07:30:45.132 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.132 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.132 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,5", 2024-09-11T07:30:45.132 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695360+0000", 2024-09-11T07:30:45.132 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012901899999999999 2024-09-11T07:30:45.132 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.133 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.133 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:45.133 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.698418+0000", 2024-09-11T07:30:45.133 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0030584520000000001 2024-09-11T07:30:45.133 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.133 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.133 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:45.133 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.700986+0000", 2024-09-11T07:30:45.133 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0025678440000000001 2024-09-11T07:30:45.133 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.133 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.134 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:45.134 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838237+0000", 2024-09-11T07:30:45.134 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13725120900000001 2024-09-11T07:30:45.134 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.134 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.134 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:45.134 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838270+0000", 2024-09-11T07:30:45.134 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2852000000000003e-05 2024-09-11T07:30:45.134 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.134 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.134 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:45.134 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838297+0000", 2024-09-11T07:30:45.135 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7477000000000001e-05 2024-09-11T07:30:45.135 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.135 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:45.135 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.135 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.135 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.135 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:120415 7.2 7:4b247b8a:::benchmark_data_smithi022_77720_object120414:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:30:45.135 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:28.695503+0000", 2024-09-11T07:30:45.135 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 16.394532856000001, 2024-09-11T07:30:45.135 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.14288304399999999, 2024-09-11T07:30:45.135 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:45.135 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:45.136 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:45.136 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:45.136 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:45.136 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 120415 2024-09-11T07:30:45.136 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.136 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:45.136 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.136 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:45.136 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695503+0000", 2024-09-11T07:30:45.136 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.136 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.136 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.136 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:45.137 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695503+0000", 2024-09-11T07:30:45.137 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.137 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.137 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.137 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:45.137 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695502+0000", 2024-09-11T07:30:45.137 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:30:45.137 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.137 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.137 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:45.137 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695525+0000", 2024-09-11T07:30:45.137 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3680999999999999e-05 2024-09-11T07:30:45.137 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.138 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.138 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:45.138 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695537+0000", 2024-09-11T07:30:45.138 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.129e-05 2024-09-11T07:30:45.138 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.138 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.138 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:45.138 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695542+0000", 2024-09-11T07:30:45.138 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:45.140 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7899999999999996e-06 2024-09-11T07:30:45.140 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.140 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.140 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:45.140 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.697877+0000", 2024-09-11T07:30:45.140 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0023348319999999998 2024-09-11T07:30:45.140 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.140 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.140 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:45.140 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.698008+0000", 2024-09-11T07:30:45.141 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000131144 2024-09-11T07:30:45.141 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.141 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.141 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:30:45.141 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.698162+0000", 2024-09-11T07:30:45.141 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000153788 2024-09-11T07:30:45.141 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.141 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.141 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:45.141 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.701572+0000", 2024-09-11T07:30:45.142 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.003409568 2024-09-11T07:30:45.142 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.142 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.142 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:45.142 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.707469+0000", 2024-09-11T07:30:45.142 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0058970669999999998 2024-09-11T07:30:45.142 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.142 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.142 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:45.142 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838325+0000", 2024-09-11T07:30:45.142 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13085648699999999 2024-09-11T07:30:45.143 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.143 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.143 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:45.143 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838358+0000", 2024-09-11T07:30:45.143 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2539999999999997e-05 2024-09-11T07:30:45.143 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.143 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.143 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:45.143 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838386+0000", 2024-09-11T07:30:45.144 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.828e-05 2024-09-11T07:30:45.144 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.144 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:45.144 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.144 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.144 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.145 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:120425 7.2 7:47ef4c79:::benchmark_data_smithi022_77720_object120424:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:30:45.145 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:28.703929+0000", 2024-09-11T07:30:45.145 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 16.386106577, 2024-09-11T07:30:45.145 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13808211300000001, 2024-09-11T07:30:45.145 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:45.145 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:45.146 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:45.146 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:45.146 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:45.146 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 120425 2024-09-11T07:30:45.146 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.146 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:45.146 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.146 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:45.146 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.703929+0000", 2024-09-11T07:30:45.146 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.147 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.147 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.147 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:45.147 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.703929+0000", 2024-09-11T07:30:45.147 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.147 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.147 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.147 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:45.147 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.703926+0000", 2024-09-11T07:30:45.147 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999967 2024-09-11T07:30:45.147 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.147 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.147 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:45.147 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.703961+0000", 2024-09-11T07:30:45.148 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4691999999999999e-05 2024-09-11T07:30:45.148 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.148 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.148 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:45.148 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.703974+0000", 2024-09-11T07:30:45.148 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3237999999999999e-05 2024-09-11T07:30:45.148 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.148 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.148 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:45.148 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.703981+0000", 2024-09-11T07:30:45.148 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.776e-06 2024-09-11T07:30:45.148 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.149 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.149 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:45.149 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.704039+0000", 2024-09-11T07:30:45.149 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8721000000000002e-05 2024-09-11T07:30:45.149 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.149 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.149 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:45.149 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.704175+0000", 2024-09-11T07:30:45.149 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013562000000000001 2024-09-11T07:30:45.149 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.149 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.149 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:30:45.149 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.704408+0000", 2024-09-11T07:30:45.150 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00023285999999999999 2024-09-11T07:30:45.150 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.150 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.150 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:45.150 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.707315+0000", 2024-09-11T07:30:45.150 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0029075049999999999 2024-09-11T07:30:45.150 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.150 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.150 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:45.150 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.707573+0000", 2024-09-11T07:30:45.150 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00025795400000000002 2024-09-11T07:30:45.150 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.150 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.150 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:45.151 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.841933+0000", 2024-09-11T07:30:45.151 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13435992799999999 2024-09-11T07:30:45.151 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.151 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.151 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:45.151 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.841979+0000", 2024-09-11T07:30:45.151 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5269000000000003e-05 2024-09-11T07:30:45.151 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.151 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.151 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:45.151 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.842011+0000", 2024-09-11T07:30:45.151 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2977e-05 2024-09-11T07:30:45.152 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.152 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:45.152 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.152 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.152 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.152 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:120637 7.a 7:5b8c9ff4:::benchmark_data_smithi022_77720_object120636:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:30:45.152 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.029397+0000", 2024-09-11T07:30:45.152 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 16.060639428000002, 2024-09-11T07:30:45.152 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.22035210899999999, 2024-09-11T07:30:45.153 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:45.153 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:45.153 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:45.153 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:45.153 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:45.153 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 120637 2024-09-11T07:30:45.153 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.153 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:45.153 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.153 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:45.153 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.029397+0000", 2024-09-11T07:30:45.153 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.154 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.154 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.154 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:45.154 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.029397+0000", 2024-09-11T07:30:45.154 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.154 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.154 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.154 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:45.154 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.029394+0000", 2024-09-11T07:30:45.154 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:30:45.155 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.155 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.155 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:45.155 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.029420+0000", 2024-09-11T07:30:45.155 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5621000000000001e-05 2024-09-11T07:30:45.155 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.155 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.155 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:45.155 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.029433+0000", 2024-09-11T07:30:45.155 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3437e-05 2024-09-11T07:30:45.155 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.155 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.155 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:45.155 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.029440+0000", 2024-09-11T07:30:45.156 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7290000000000003e-06 2024-09-11T07:30:45.156 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.156 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.156 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:45.156 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.031329+0000", 2024-09-11T07:30:45.156 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0018888970000000001 2024-09-11T07:30:45.156 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.156 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.156 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:45.156 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.031434+0000", 2024-09-11T07:30:45.156 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000104985 2024-09-11T07:30:45.156 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.157 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.157 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,5", 2024-09-11T07:30:45.157 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.031563+0000", 2024-09-11T07:30:45.157 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000128794 2024-09-11T07:30:45.157 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.157 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.157 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:45.157 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.040366+0000", 2024-09-11T07:30:45.157 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0088039620000000002 2024-09-11T07:30:45.157 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.157 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.157 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:45.158 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.041770+0000", 2024-09-11T07:30:45.158 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001403257 2024-09-11T07:30:45.158 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.158 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.158 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:45.158 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.249644+0000", 2024-09-11T07:30:45.158 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20787381199999999 2024-09-11T07:30:45.158 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.158 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.158 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:45.158 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.249698+0000", 2024-09-11T07:30:45.158 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.4273e-05 2024-09-11T07:30:45.159 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.159 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.159 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:45.159 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.249749+0000", 2024-09-11T07:30:45.159 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.0803000000000001e-05 2024-09-11T07:30:45.159 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.159 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:45.159 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.159 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.159 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.159 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:120649 7.2 7:4a3799e9:::benchmark_data_smithi022_77720_object120648:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:30:45.160 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.034409+0000", 2024-09-11T07:30:45.160 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 16.055626649000001, 2024-09-11T07:30:45.160 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.215573452, 2024-09-11T07:30:45.160 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:45.160 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:45.160 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:45.160 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:45.160 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:45.160 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 120649 2024-09-11T07:30:45.161 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.161 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:45.161 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.161 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:45.161 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.034409+0000", 2024-09-11T07:30:45.161 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.161 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.161 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.161 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:45.161 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.034409+0000", 2024-09-11T07:30:45.161 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.161 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.162 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.162 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:45.162 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.034407+0000", 2024-09-11T07:30:45.162 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:30:45.162 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.162 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.162 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:45.162 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.034433+0000", 2024-09-11T07:30:45.162 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.6526999999999999e-05 2024-09-11T07:30:45.162 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.162 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.162 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:45.162 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.034448+0000", 2024-09-11T07:30:45.163 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4080000000000001e-05 2024-09-11T07:30:45.163 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.163 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.163 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:45.163 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.034454+0000", 2024-09-11T07:30:45.163 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5429999999999999e-06 2024-09-11T07:30:45.163 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.163 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.163 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:45.163 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.036213+0000", 2024-09-11T07:30:45.163 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001758857 2024-09-11T07:30:45.163 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.164 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.164 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:45.164 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.036318+0000", 2024-09-11T07:30:45.164 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000105245 2024-09-11T07:30:45.164 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.164 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.164 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:30:45.164 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.036447+0000", 2024-09-11T07:30:45.164 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012836999999999999 2024-09-11T07:30:45.164 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.164 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.164 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:45.165 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.042776+0000", 2024-09-11T07:30:45.165 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0063298790000000001 2024-09-11T07:30:45.165 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.165 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.165 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:45.165 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.043761+0000", 2024-09-11T07:30:45.165 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00098433900000000009 2024-09-11T07:30:45.165 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.165 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.165 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:45.165 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.249911+0000", 2024-09-11T07:30:45.165 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20615025200000001 2024-09-11T07:30:45.165 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.166 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.166 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:45.166 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.249951+0000", 2024-09-11T07:30:45.166 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9817000000000002e-05 2024-09-11T07:30:45.166 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.166 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.166 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:45.166 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.249983+0000", 2024-09-11T07:30:45.166 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.188e-05 2024-09-11T07:30:45.166 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.166 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:45.166 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.166 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.167 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.167 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120650 7.c e127/92 7:3551ad5b:::benchmark_data_smithi022_77720_object120649:head v 127'7564, mlcod=127'7562)", 2024-09-11T07:30:45.167 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.038754+0000", 2024-09-11T07:30:45.167 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 16.051281490000001, 2024-09-11T07:30:45.167 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.21640670200000001, 2024-09-11T07:30:45.167 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:45.167 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:45.167 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:45.167 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.167 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:45.167 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.038754+0000", 2024-09-11T07:30:45.167 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.167 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.167 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.168 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:45.168 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.038754+0000", 2024-09-11T07:30:45.168 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.168 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.168 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.168 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:45.168 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.038752+0000", 2024-09-11T07:30:45.168 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:30:45.168 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.168 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.168 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:45.168 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.038797+0000", 2024-09-11T07:30:45.169 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4357000000000002e-05 2024-09-11T07:30:45.169 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.169 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.169 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:45.169 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.038809+0000", 2024-09-11T07:30:45.169 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1829e-05 2024-09-11T07:30:45.169 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.169 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.169 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:45.169 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.038815+0000", 2024-09-11T07:30:45.169 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3579999999999999e-06 2024-09-11T07:30:45.169 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.169 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.170 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:45.170 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.038871+0000", 2024-09-11T07:30:45.170 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6193999999999998e-05 2024-09-11T07:30:45.170 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.170 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.170 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:45.170 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.038889+0000", 2024-09-11T07:30:45.170 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.7915999999999999e-05 2024-09-11T07:30:45.170 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.170 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.170 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:45.170 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.255019+0000", 2024-09-11T07:30:45.170 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.21613015499999999 2024-09-11T07:30:45.170 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.171 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.171 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:45.171 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.255161+0000", 2024-09-11T07:30:45.171 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014201900000000001 2024-09-11T07:30:45.171 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.171 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:45.171 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.171 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.171 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.171 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120651 7.c e127/92 7:32aa7de5:::benchmark_data_smithi022_77720_object120650:head v 127'7565, mlcod=127'7562)", 2024-09-11T07:30:45.171 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.041033+0000", 2024-09-11T07:30:45.171 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 16.049002763000001, 2024-09-11T07:30:45.171 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.214179168, 2024-09-11T07:30:45.172 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:45.172 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:45.172 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:45.172 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.172 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:45.172 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.041033+0000", 2024-09-11T07:30:45.172 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.172 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.172 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.172 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:45.172 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.041033+0000", 2024-09-11T07:30:45.172 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.172 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.172 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.173 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:45.173 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.041032+0000", 2024-09-11T07:30:45.173 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:30:45.173 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.173 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.173 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:45.173 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.041062+0000", 2024-09-11T07:30:45.173 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0488999999999999e-05 2024-09-11T07:30:45.173 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.173 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.173 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:45.173 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.041072+0000", 2024-09-11T07:30:45.173 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.5629999999999998e-06 2024-09-11T07:30:45.174 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.174 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.174 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:45.174 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.041077+0000", 2024-09-11T07:30:45.174 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.0579999999999997e-06 2024-09-11T07:30:45.174 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.174 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.174 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:45.174 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.045254+0000", 2024-09-11T07:30:45.174 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0041766110000000002 2024-09-11T07:30:45.174 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.174 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.174 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:45.174 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.045273+0000", 2024-09-11T07:30:45.175 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9636e-05 2024-09-11T07:30:45.175 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.175 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.175 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:45.175 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.255164+0000", 2024-09-11T07:30:45.175 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20989069099999999 2024-09-11T07:30:45.175 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.175 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.175 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:45.175 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.255212+0000", 2024-09-11T07:30:45.175 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8322000000000003e-05 2024-09-11T07:30:45.175 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.175 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:45.175 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.176 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.176 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.176 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120654 7.c e127/92 7:3089e4e7:::benchmark_data_smithi022_77720_object120653:head v 127'7566, mlcod=127'7562)", 2024-09-11T07:30:45.176 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.044199+0000", 2024-09-11T07:30:45.176 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 16.045837389999999, 2024-09-11T07:30:45.176 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.211072754, 2024-09-11T07:30:45.176 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:45.176 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:45.176 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:45.176 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.176 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:45.177 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.044199+0000", 2024-09-11T07:30:45.177 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.177 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.177 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.177 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:45.177 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.044199+0000", 2024-09-11T07:30:45.177 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.177 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.177 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.177 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:45.177 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.044197+0000", 2024-09-11T07:30:45.177 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:30:45.178 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.178 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.178 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:45.178 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.044244+0000", 2024-09-11T07:30:45.178 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6838000000000001e-05 2024-09-11T07:30:45.178 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.178 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.178 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:45.178 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.044256+0000", 2024-09-11T07:30:45.178 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2041e-05 2024-09-11T07:30:45.178 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.179 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.179 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:45.179 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.044262+0000", 2024-09-11T07:30:45.179 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2110000000000003e-06 2024-09-11T07:30:45.179 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.179 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.179 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:45.179 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.048642+0000", 2024-09-11T07:30:45.179 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0043793469999999996 2024-09-11T07:30:45.179 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.179 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.180 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:45.180 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.048660+0000", 2024-09-11T07:30:45.180 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8363999999999999e-05 2024-09-11T07:30:45.180 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.180 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.180 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:45.180 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.255215+0000", 2024-09-11T07:30:45.180 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20655538200000001 2024-09-11T07:30:45.180 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.180 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.180 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:45.180 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.255271+0000", 2024-09-11T07:30:45.181 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6033999999999999e-05 2024-09-11T07:30:45.181 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.181 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:45.181 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.181 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.181 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.181 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:120659 7.3 7:c2975cc8:::benchmark_data_smithi022_77720_object120658:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:30:45.181 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.046561+0000", 2024-09-11T07:30:45.181 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 16.043474743000001, 2024-09-11T07:30:45.181 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20440429299999999, 2024-09-11T07:30:45.181 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:45.181 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:45.181 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:45.182 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:45.182 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:45.182 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 120659 2024-09-11T07:30:45.182 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.182 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:45.182 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.182 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:45.182 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046561+0000", 2024-09-11T07:30:45.182 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.182 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.182 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.182 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:45.182 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046561+0000", 2024-09-11T07:30:45.182 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.183 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.183 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.183 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:45.183 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046559+0000", 2024-09-11T07:30:45.183 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:30:45.183 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.183 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.183 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:45.183 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046587+0000", 2024-09-11T07:30:45.183 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8705e-05 2024-09-11T07:30:45.183 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.183 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.183 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:45.184 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046600+0000", 2024-09-11T07:30:45.184 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2499e-05 2024-09-11T07:30:45.184 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.184 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.184 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:45.184 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046606+0000", 2024-09-11T07:30:45.184 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0159999999999997e-06 2024-09-11T07:30:45.184 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.184 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.184 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:45.184 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046693+0000", 2024-09-11T07:30:45.184 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.6792999999999994e-05 2024-09-11T07:30:45.184 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.185 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.185 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:45.185 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046953+0000", 2024-09-11T07:30:45.185 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00026056300000000001 2024-09-11T07:30:45.185 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.185 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.185 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,4", 2024-09-11T07:30:45.185 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.047062+0000", 2024-09-11T07:30:45.185 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010925999999999999 2024-09-11T07:30:45.185 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.185 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.185 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:45.185 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.049336+0000", 2024-09-11T07:30:45.186 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002273691 2024-09-11T07:30:45.186 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.186 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.186 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:45.186 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050203+0000", 2024-09-11T07:30:45.186 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000866591 2024-09-11T07:30:45.186 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.186 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.186 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:45.186 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.250805+0000", 2024-09-11T07:30:45.186 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20060213800000001 2024-09-11T07:30:45.186 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.186 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.186 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:45.187 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.250854+0000", 2024-09-11T07:30:45.187 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9335000000000001e-05 2024-09-11T07:30:45.187 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.187 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.187 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:45.187 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.250965+0000", 2024-09-11T07:30:45.187 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000111242 2024-09-11T07:30:45.187 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.187 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:45.187 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.187 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.187 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.189 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120656 7.c e127/92 7:35364cdf:::benchmark_data_smithi022_77720_object120655:head v 127'7567, mlcod=127'7562)", 2024-09-11T07:30:45.189 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.046959+0000", 2024-09-11T07:30:45.189 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 16.043076728999999, 2024-09-11T07:30:45.189 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.21188748199999999, 2024-09-11T07:30:45.189 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:45.189 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:45.189 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:45.189 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.189 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:45.189 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046959+0000", 2024-09-11T07:30:45.189 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.189 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.190 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.190 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:45.190 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046959+0000", 2024-09-11T07:30:45.190 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.190 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.190 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.190 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:45.190 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046958+0000", 2024-09-11T07:30:45.190 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:30:45.190 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.190 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.190 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:45.190 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.047009+0000", 2024-09-11T07:30:45.191 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1489000000000001e-05 2024-09-11T07:30:45.191 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.191 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.191 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:45.191 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.047020+0000", 2024-09-11T07:30:45.191 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.064e-05 2024-09-11T07:30:45.191 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.191 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.191 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:45.191 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.047025+0000", 2024-09-11T07:30:45.191 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1279999999999999e-06 2024-09-11T07:30:45.191 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.191 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.192 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:45.192 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246150+0000", 2024-09-11T07:30:45.192 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.199124994 2024-09-11T07:30:45.192 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.192 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.192 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:45.192 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246173+0000", 2024-09-11T07:30:45.192 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3091999999999998e-05 2024-09-11T07:30:45.192 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.192 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.192 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:45.192 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.258588+0000", 2024-09-11T07:30:45.192 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.012415433 2024-09-11T07:30:45.193 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.193 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.193 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:45.193 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.258847+0000", 2024-09-11T07:30:45.193 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00025835000000000002 2024-09-11T07:30:45.193 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.193 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:45.193 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.193 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.193 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.193 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120657 7.c e127/92 7:333acda7:::benchmark_data_smithi022_77720_object120656:head v 127'7568, mlcod=127'7562)", 2024-09-11T07:30:45.193 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.050053+0000", 2024-09-11T07:30:45.193 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 16.039982542000001, 2024-09-11T07:30:45.194 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.22377028800000001, 2024-09-11T07:30:45.194 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:45.194 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:45.194 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:45.194 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.194 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:45.194 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050053+0000", 2024-09-11T07:30:45.194 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.194 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.194 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.194 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:45.194 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050053+0000", 2024-09-11T07:30:45.194 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.194 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.194 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.195 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:45.195 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050052+0000", 2024-09-11T07:30:45.195 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:30:45.195 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.195 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.195 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:45.195 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050092+0000", 2024-09-11T07:30:45.195 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0565000000000002e-05 2024-09-11T07:30:45.195 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.195 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.195 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:45.195 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050102+0000", 2024-09-11T07:30:45.195 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.6150000000000003e-06 2024-09-11T07:30:45.195 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.195 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.196 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:45.196 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050106+0000", 2024-09-11T07:30:45.196 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9400000000000004e-06 2024-09-11T07:30:45.196 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.196 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.196 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:45.196 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.255309+0000", 2024-09-11T07:30:45.196 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.205202789 2024-09-11T07:30:45.196 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.196 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.196 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:45.196 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.255342+0000", 2024-09-11T07:30:45.196 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2854000000000001e-05 2024-09-11T07:30:45.196 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.196 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.197 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:45.197 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.273773+0000", 2024-09-11T07:30:45.197 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.018431050000000001 2024-09-11T07:30:45.197 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.197 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.197 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:45.197 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.273824+0000", 2024-09-11T07:30:45.197 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1072000000000002e-05 2024-09-11T07:30:45.197 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.197 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:45.197 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.197 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.197 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.198 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120662 7.7 e127/92 7:edc0624b:::benchmark_data_smithi022_77720_object120661:head v 127'7599, mlcod=127'7596)", 2024-09-11T07:30:45.198 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.050439+0000", 2024-09-11T07:30:45.198 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 16.039596703000001, 2024-09-11T07:30:45.198 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20149834799999999, 2024-09-11T07:30:45.198 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:45.198 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:45.198 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:45.198 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.198 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:45.198 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050439+0000", 2024-09-11T07:30:45.198 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.198 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.198 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.198 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:45.198 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050439+0000", 2024-09-11T07:30:45.199 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.199 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.199 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.199 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:45.199 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050438+0000", 2024-09-11T07:30:45.199 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:30:45.199 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.199 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.199 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:45.199 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050526+0000", 2024-09-11T07:30:45.199 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.8176000000000004e-05 2024-09-11T07:30:45.199 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.199 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.200 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:45.200 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050535+0000", 2024-09-11T07:30:45.200 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.1050000000000001e-06 2024-09-11T07:30:45.200 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.200 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.200 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:45.200 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050540+0000", 2024-09-11T07:30:45.200 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8740000000000003e-06 2024-09-11T07:30:45.200 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.200 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.200 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:45.200 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.248240+0000", 2024-09-11T07:30:45.200 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.19770011900000001 2024-09-11T07:30:45.200 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.201 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.201 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:45.201 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.248260+0000", 2024-09-11T07:30:45.201 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9468000000000001e-05 2024-09-11T07:30:45.201 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.201 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.201 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:45.201 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.251902+0000", 2024-09-11T07:30:45.201 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0036426689999999999 2024-09-11T07:30:45.201 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.201 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.202 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:45.202 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.251938+0000", 2024-09-11T07:30:45.202 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5219999999999998e-05 2024-09-11T07:30:45.202 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.202 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:45.202 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.202 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.202 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.202 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120661 7.1 e127/92 7:8ef41966:::benchmark_data_smithi022_77720_object120660:head v 127'7487, mlcod=127'7484)", 2024-09-11T07:30:45.202 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.050540+0000", 2024-09-11T07:30:45.202 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 16.039496416999999, 2024-09-11T07:30:45.202 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20305398899999999, 2024-09-11T07:30:45.202 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:45.202 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:45.202 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:45.203 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.203 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:45.203 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050540+0000", 2024-09-11T07:30:45.203 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.203 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.203 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.203 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:45.203 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050540+0000", 2024-09-11T07:30:45.203 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.203 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.203 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.203 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:45.203 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050539+0000", 2024-09-11T07:30:45.204 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:30:45.204 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.204 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.204 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:45.204 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050602+0000", 2024-09-11T07:30:45.204 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3145000000000006e-05 2024-09-11T07:30:45.204 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.204 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.204 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:45.204 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246370+0000", 2024-09-11T07:30:45.204 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.19576812599999999 2024-09-11T07:30:45.204 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.204 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.205 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:45.205 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246378+0000", 2024-09-11T07:30:45.205 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.8860000000000004e-06 2024-09-11T07:30:45.205 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.205 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.205 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:45.205 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246487+0000", 2024-09-11T07:30:45.205 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000109171 2024-09-11T07:30:45.205 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.205 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.205 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:45.205 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246507+0000", 2024-09-11T07:30:45.205 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0064999999999998e-05 2024-09-11T07:30:45.206 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.206 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.206 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:45.206 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.253510+0000", 2024-09-11T07:30:45.206 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0070029630000000001 2024-09-11T07:30:45.206 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.206 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.206 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:45.206 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.253594+0000", 2024-09-11T07:30:45.206 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3560000000000006e-05 2024-09-11T07:30:45.206 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.206 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:45.206 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.206 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.207 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.207 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:121076 7.2 7:449556c8:::benchmark_data_smithi022_77720_object121075:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:30:45.207 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.689105+0000", 2024-09-11T07:30:45.207 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 15.400930603000001, 2024-09-11T07:30:45.207 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.132796636, 2024-09-11T07:30:45.207 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:45.207 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:45.207 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:30:45.207 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:30:45.207 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:30:45.207 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 121076 2024-09-11T07:30:45.207 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.207 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:45.207 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.208 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:45.208 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.689105+0000", 2024-09-11T07:30:45.208 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.208 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.208 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.208 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:45.208 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.689105+0000", 2024-09-11T07:30:45.208 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.208 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.208 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.208 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:45.208 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.689104+0000", 2024-09-11T07:30:45.209 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:30:45.209 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.209 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.209 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:45.209 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.689129+0000", 2024-09-11T07:30:45.209 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4859999999999999e-05 2024-09-11T07:30:45.209 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.209 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.209 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:45.209 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.689137+0000", 2024-09-11T07:30:45.209 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.7940000000000003e-06 2024-09-11T07:30:45.209 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.209 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.210 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:45.210 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.689142+0000", 2024-09-11T07:30:45.210 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2059999999999998e-06 2024-09-11T07:30:45.210 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.210 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.210 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:45.210 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.692376+0000", 2024-09-11T07:30:45.210 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0032336869999999998 2024-09-11T07:30:45.210 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.210 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.210 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:45.210 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.692483+0000", 2024-09-11T07:30:45.210 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010708 2024-09-11T07:30:45.210 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.210 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.211 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:30:45.211 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.692968+0000", 2024-09-11T07:30:45.211 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00048511600000000001 2024-09-11T07:30:45.211 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.211 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.211 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:30:45.211 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.699162+0000", 2024-09-11T07:30:45.211 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.006193998 2024-09-11T07:30:45.211 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.211 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.211 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:45.211 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.821791+0000", 2024-09-11T07:30:45.211 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.122628429 2024-09-11T07:30:45.211 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.212 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.212 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:30:45.212 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.821817+0000", 2024-09-11T07:30:45.212 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.6571999999999999e-05 2024-09-11T07:30:45.212 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.212 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.212 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:45.212 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.821863+0000", 2024-09-11T07:30:45.212 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6127000000000001e-05 2024-09-11T07:30:45.212 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.212 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.212 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:45.213 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.821902+0000", 2024-09-11T07:30:45.213 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8744e-05 2024-09-11T07:30:45.213 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.213 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:45.213 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.213 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.213 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.213 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:121096 7.f e127/92 7:f02b271f:::benchmark_data_smithi022_77720_object121095:head v 127'7560, mlcod=127'7558)", 2024-09-11T07:30:45.213 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.707790+0000", 2024-09-11T07:30:45.213 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 15.382245878999999, 2024-09-11T07:30:45.213 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13013040300000001, 2024-09-11T07:30:45.213 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:30:45.213 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:30:45.214 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:30:45.214 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.214 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:30:45.214 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.707790+0000", 2024-09-11T07:30:45.214 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.214 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.214 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.214 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:30:45.214 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.707790+0000", 2024-09-11T07:30:45.214 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:30:45.214 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.214 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.214 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:30:45.214 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.707789+0000", 2024-09-11T07:30:45.214 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:30:45.215 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.215 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.215 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:30:45.215 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.707816+0000", 2024-09-11T07:30:45.215 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.6818999999999999e-05 2024-09-11T07:30:45.215 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.215 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.215 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:30:45.215 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.707823+0000", 2024-09-11T07:30:45.215 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0149999999999998e-06 2024-09-11T07:30:45.216 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.216 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.216 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:30:45.216 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.707828+0000", 2024-09-11T07:30:45.216 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4780000000000002e-06 2024-09-11T07:30:45.216 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.216 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.216 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:30:45.216 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.707867+0000", 2024-09-11T07:30:45.216 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9731000000000003e-05 2024-09-11T07:30:45.216 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.216 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.216 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:30:45.216 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.707892+0000", 2024-09-11T07:30:45.217 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4739000000000001e-05 2024-09-11T07:30:45.217 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.217 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.217 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:30:45.217 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.837896+0000", 2024-09-11T07:30:45.217 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13000419899999999 2024-09-11T07:30:45.217 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:30:45.217 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:30:45.217 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:30:45.217 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.837920+0000", 2024-09-11T07:30:45.217 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4066999999999998e-05 2024-09-11T07:30:45.217 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.217 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:45.217 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.218 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:30:45.218 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:30:45.218 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:30:45.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:45.219 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:45.219 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:45.320 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:45.347 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:45.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:45.349 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:45.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:45 smithi022 ceph-mon[29462]: pgmap v167: 148 pgs: 148 active+clean; 7.1 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 12 MiB/s wr, 934 op/s 2024-09-11T07:30:45.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:45 smithi022 ceph-mon[37330]: pgmap v167: 148 pgs: 148 active+clean; 7.1 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 12 MiB/s wr, 934 op/s 2024-09-11T07:30:45.450 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:45.476 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:45.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:45.477 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:45 smithi104 ceph-mon[33507]: pgmap v167: 148 pgs: 148 active+clean; 7.1 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 12 MiB/s wr, 934 op/s 2024-09-11T07:30:45.579 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:45.634 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:45.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:45.636 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:45.736 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:45.754 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:30:45.754 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:30:45.755 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-11T07:30:45.755 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T07:30:45.755 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T07:30:45.755 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 config set filestore_inject_stall 3 2024-09-11T07:30:45.759 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:45.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:45.760 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:45.861 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:45.885 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:45.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:45.886 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:45.986 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:46.027 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:46.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:46.028 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:46.097 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is set 2024-09-11T07:30:46.129 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:46.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:46.178 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:46.178 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:46.279 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:46.319 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:46.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:46.320 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:46.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:46 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/756768822' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:30:46.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:46 smithi022 ceph-mon[29462]: from='client.24886 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:30:46.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:46 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/756768822' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:30:46.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:46 smithi022 ceph-mon[37330]: from='client.24886 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:30:46.421 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:46.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:46.465 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:46.465 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:46.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:46 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/756768822' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:30:46.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:46 smithi104 ceph-mon[33507]: from='client.24886 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:30:46.566 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:46.593 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:46.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:46.594 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:46.631 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-11T07:30:46.695 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:46.746 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:46.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:46.747 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:46.833 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:30:46.848 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:46.941 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:46.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:46.942 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:46.965 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:30:47.042 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:47.081 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:47.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:47.083 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:47.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:47 smithi022 ceph-mon[29462]: pgmap v168: 148 pgs: 148 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-11T07:30:47.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:47 smithi022 ceph-mon[29462]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:30:47.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:47 smithi022 ceph-mon[29462]: from='client.24886 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:30:47.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:47 smithi022 ceph-mon[29462]: osdmap e129: 8 total, 8 up, 8 in 2024-09-11T07:30:47.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:47 smithi022 ceph-mon[37330]: pgmap v168: 148 pgs: 148 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-11T07:30:47.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:47 smithi022 ceph-mon[37330]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:30:47.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:47 smithi022 ceph-mon[37330]: from='client.24886 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:30:47.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:47 smithi022 ceph-mon[37330]: osdmap e129: 8 total, 8 up, 8 in 2024-09-11T07:30:47.183 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:47.210 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:47.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:47.211 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:47.312 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:47.359 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:47.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:47.359 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:47.375 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:47 smithi104 ceph-mon[33507]: pgmap v168: 148 pgs: 148 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-11T07:30:47.375 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:47 smithi104 ceph-mon[33507]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:30:47.375 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:47 smithi104 ceph-mon[33507]: from='client.24886 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:30:47.376 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:47 smithi104 ceph-mon[33507]: osdmap e129: 8 total, 8 up, 8 in 2024-09-11T07:30:47.460 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:47.495 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:47.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:47.496 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:47.597 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:47.619 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:47.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:47.620 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:47.631 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:47.650 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:47.721 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:47.747 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:47.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:47.748 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:47.849 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:47.898 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:47.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:47.899 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:48.000 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:48.046 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:48.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:48.047 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:48.148 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:48.173 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:48.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:48.175 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:48.275 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:48.314 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:48.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:48.315 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:48.416 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:48.447 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:48.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:48.448 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:48.548 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:48.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:48.584 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:48.585 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:48.685 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:48.725 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:48.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:48.726 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:48.827 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:48.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:48.860 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:48.860 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:48.961 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:48.965 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:30:48.965 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:30:48.965 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 0 2024-09-11T07:30:48.965 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:30:48.998 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:48.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:48.999 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:49.100 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:49.144 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:49.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:49.145 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:49.246 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:49.259 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:49 smithi022 ceph-mon[29462]: pgmap v170: 148 pgs: 148 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-11T07:30:49.260 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:49 smithi022 ceph-mon[37330]: pgmap v170: 148 pgs: 148 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-11T07:30:49.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:49.302 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:49.302 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:49.403 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:49.443 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:49.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:49.444 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:49 smithi104 ceph-mon[33507]: pgmap v170: 148 pgs: 148 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-11T07:30:49.530 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:30:49 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:30:49.544 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:49.548 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T07:30:49.591 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:49.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:49.593 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:49.694 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:49.718 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:49.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:49.719 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:49.764 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-11T07:30:49.820 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:49.855 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:49.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:49.856 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:49.956 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:50.001 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:50.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:50.002 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:50.072 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:30:50.073 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:30:50.087 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:30:50.103 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:30:50.103 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:30:50.103 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:50.135 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:50 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:50.135 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:50 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/684553217' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:30:50.136 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:30:50.136 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:30:50.136 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:50 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:50.137 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:50 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/684553217' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:30:50.146 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:50.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:50.147 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:50.174 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:30:50.175 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:30:50.204 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:30:50.204 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:30:50.235 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:30:50.235 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:30:50.247 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:50.268 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:30:50.268 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:30:50.286 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:50.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:50.287 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:50.302 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:30:50.303 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:30:50.366 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:50 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:30:50.366 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:50 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/684553217' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:30:50.388 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:50.417 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:30:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:30:50] "GET /metrics HTTP/1.1" 200 223717 "" "Prometheus/2.43.0" 2024-09-11T07:30:50.429 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:50.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:50.430 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:50.531 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:50.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:50.578 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:50.578 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:50.679 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:50.719 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:50.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:50.720 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:50.821 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:50.849 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:50.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:50.851 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:50.951 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:50.983 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:50.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:50.984 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:51.085 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:51.127 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:51.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:51.128 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:51.229 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:51.268 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:51.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:51.269 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:51.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:51 smithi104 ceph-mon[33507]: pgmap v171: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.09k op/s 2024-09-11T07:30:51.370 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:51.402 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:51.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:51.403 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:51.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:51 smithi022 ceph-mon[29462]: pgmap v171: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.09k op/s 2024-09-11T07:30:51.409 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:51 smithi022 ceph-mon[37330]: pgmap v171: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.09k op/s 2024-09-11T07:30:51.504 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:51.531 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:51.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:51.532 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:51.633 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:51.678 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:51.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:51.679 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:51.780 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:51.805 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:51.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:51.806 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:51.834 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:51.877 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:30:51.878 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-11T07:30:51.878 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:30:51.907 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:51.933 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:51.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:51.934 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:51.950 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:30:51.950 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:30:52.035 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:52.076 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:52.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:52.077 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:52.177 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:52.205 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:52.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:52.206 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:52.306 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:52.334 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:52.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:52.334 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:52.435 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:52.463 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:52.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:52.464 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:52.565 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:52.589 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:52.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:52.590 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:52.691 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:52.713 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:52.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:52.714 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:52.766 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-11T07:30:52.814 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:52.850 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:52.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:52.851 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:52.952 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:52.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:52.975 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:52.975 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:53.024 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:30:53.074 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:30:53.075 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:53.139 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:53.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:53.140 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:53.240 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:53.280 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:53.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:53.281 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:53.381 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:53.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:53.406 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:53.407 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:53.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:53 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:30:53.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:53 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:30:53.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:53 smithi022 ceph-mon[29462]: pgmap v172: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.09k op/s 2024-09-11T07:30:53.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:53 smithi022 ceph-mon[29462]: osdmap e130: 8 total, 8 up, 8 in 2024-09-11T07:30:53.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:30:53.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:53 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:30:53.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:53 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:30:53.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:53 smithi022 ceph-mon[37330]: pgmap v172: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.09k op/s 2024-09-11T07:30:53.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:53 smithi022 ceph-mon[37330]: osdmap e130: 8 total, 8 up, 8 in 2024-09-11T07:30:53.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:30:53.454 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:53 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:30:53.454 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:53 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:30:53.454 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:53 smithi104 ceph-mon[33507]: pgmap v172: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.09k op/s 2024-09-11T07:30:53.454 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:53 smithi104 ceph-mon[33507]: osdmap e130: 8 total, 8 up, 8 in 2024-09-11T07:30:53.455 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:30:53.508 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:53.543 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:53.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:53.544 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:53.645 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:53.675 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:53.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:53.676 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:53.777 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:53.815 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:53.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:53.816 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:53.917 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:53.950 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:53.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:53.951 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:54.052 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:54.081 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:54.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:54.082 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:54.183 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:54.227 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:54.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:54.228 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:54.279 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:30:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:30:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:30:54.329 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:54.369 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:54.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:54.370 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:54.470 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:54.529 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:54.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:54.530 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:54.631 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:54.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:54.669 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:54.669 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:54.770 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:54.799 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:54.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:54.800 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:54.901 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:54.946 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:54.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:54.948 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:54.956 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:30:54.957 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:30:55.049 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:55.075 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:55.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:55.077 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:55.178 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:55.229 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:55.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:55.230 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:55.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:55 smithi022 ceph-mon[29462]: pgmap v174: 148 pgs: 148 active+clean; 6.4 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 793 op/s 2024-09-11T07:30:55.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:55 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:30:55.235 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:55 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:30:55.236 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:55 smithi022 ceph-mon[37330]: pgmap v174: 148 pgs: 148 active+clean; 6.4 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 793 op/s 2024-09-11T07:30:55.236 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:55 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:30:55.236 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:55 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:30:55.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:55 smithi104 ceph-mon[33507]: pgmap v174: 148 pgs: 148 active+clean; 6.4 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 793 op/s 2024-09-11T07:30:55.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:55 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:30:55.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:55 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:30:55.331 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:55.368 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:55.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:55.369 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:55.470 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:55.495 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:55.496 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:30:55.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:55.497 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:55.597 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:55.622 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:55.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:55.623 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:55.723 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:55.751 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:55.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:55.752 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:55.777 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-11T07:30:55.853 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:55.897 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:55.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:55.898 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:55.998 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:56.037 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:56.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:56.038 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:56.139 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:30:56.146 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:30:56.170 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:56.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:56.171 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:30:56.272 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:56.318 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:56.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:56.319 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:56.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:56 smithi022 ceph-mon[29462]: osdmap e131: 8 total, 8 up, 8 in 2024-09-11T07:30:56.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:56 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2978416482' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:30:56.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:56 smithi022 ceph-mon[37330]: osdmap e131: 8 total, 8 up, 8 in 2024-09-11T07:30:56.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:56 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2978416482' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:30:56.420 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:30:56.420 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:56 smithi104 ceph-mon[33507]: osdmap e131: 8 total, 8 up, 8 in 2024-09-11T07:30:56.420 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:56 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2978416482' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:30:56.446 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:56.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:56.447 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:30:56.494 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is unset 2024-09-11T07:30:56.547 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:56.589 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:56.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:56.590 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:56.691 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:56.730 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:56.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:56.731 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:56.832 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:56.865 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:56.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:56.866 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:56.967 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:57.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:57.012 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:57.112 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:57.136 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:57.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:57.137 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:57.238 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:57.264 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:57.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:57.265 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:57.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:57 smithi104 ceph-mon[33507]: pgmap v176: 148 pgs: 148 active+clean; 6.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T07:30:57.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:57 smithi104 ceph-mon[33507]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:30:57.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:57 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2978416482' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:30:57.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:57 smithi104 ceph-mon[33507]: osdmap e132: 8 total, 8 up, 8 in 2024-09-11T07:30:57.366 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:57.393 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:57.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:57.394 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:57.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:57 smithi022 ceph-mon[29462]: pgmap v176: 148 pgs: 148 active+clean; 6.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T07:30:57.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:57 smithi022 ceph-mon[29462]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:30:57.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:57 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2978416482' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:30:57.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:57 smithi022 ceph-mon[29462]: osdmap e132: 8 total, 8 up, 8 in 2024-09-11T07:30:57.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:57 smithi022 ceph-mon[37330]: pgmap v176: 148 pgs: 148 active+clean; 6.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T07:30:57.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:57 smithi022 ceph-mon[37330]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:30:57.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:57 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2978416482' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:30:57.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:57 smithi022 ceph-mon[37330]: osdmap e132: 8 total, 8 up, 8 in 2024-09-11T07:30:57.495 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:57.531 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:57.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:57.532 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:57.632 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:57.672 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:57.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:57.673 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:57.773 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:57.813 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:57.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:57.814 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:57.915 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:57.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:57.948 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:57.948 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:58.017 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:30:58.017 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:30:58.049 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:58.078 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:58.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:58.079 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:58.179 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:58.226 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:58.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:58.227 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:58.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:58 smithi104 ceph-mon[33507]: 7.f scrub starts 2024-09-11T07:30:58.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:58 smithi104 ceph-mon[33507]: 2.0 scrub starts 2024-09-11T07:30:58.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:58 smithi104 ceph-mon[33507]: 2.0 scrub ok 2024-09-11T07:30:58.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:58 smithi104 ceph-mon[33507]: 7.b scrub starts 2024-09-11T07:30:58.328 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:30:58.376 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:58.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:58.377 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:30:58.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:58 smithi022 ceph-mon[29462]: 7.f scrub starts 2024-09-11T07:30:58.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:58 smithi022 ceph-mon[29462]: 2.0 scrub starts 2024-09-11T07:30:58.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:58 smithi022 ceph-mon[29462]: 2.0 scrub ok 2024-09-11T07:30:58.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:58 smithi022 ceph-mon[29462]: 7.b scrub starts 2024-09-11T07:30:58.410 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:58 smithi022 ceph-mon[37330]: 7.f scrub starts 2024-09-11T07:30:58.410 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:58 smithi022 ceph-mon[37330]: 2.0 scrub starts 2024-09-11T07:30:58.410 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:58 smithi022 ceph-mon[37330]: 2.0 scrub ok 2024-09-11T07:30:58.410 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:58 smithi022 ceph-mon[37330]: 7.b scrub starts 2024-09-11T07:30:58.478 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:58.503 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:58.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:58.504 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:58.605 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:58.644 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:58.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:58.645 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:58.658 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-11T07:30:58.746 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:58.771 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:58.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:58.772 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:58.873 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:58.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:58.918 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:58.918 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:58.995 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:30:59.019 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:30:59.060 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:59.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:59.061 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:30:59.162 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:59.188 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:59.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:59.189 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:59.291 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:30:59.303 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:59 smithi022 ceph-mon[37330]: 2.e scrub starts 2024-09-11T07:30:59.303 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:59 smithi022 ceph-mon[37330]: 2.e scrub ok 2024-09-11T07:30:59.303 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:59 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:30:59.303 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:59 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:30:59.303 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:59 smithi022 ceph-mon[37330]: pgmap v178: 148 pgs: 148 active+clean; 6.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 862 op/s 2024-09-11T07:30:59.303 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:59 smithi022 ceph-mon[37330]: osdmap e133: 8 total, 8 up, 8 in 2024-09-11T07:30:59.303 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:59 smithi022 ceph-mon[37330]: 2.9 scrub starts 2024-09-11T07:30:59.303 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:30:59 smithi022 ceph-mon[37330]: 2.9 scrub ok 2024-09-11T07:30:59.306 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:59 smithi022 ceph-mon[29462]: 2.e scrub starts 2024-09-11T07:30:59.306 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:59 smithi022 ceph-mon[29462]: 2.e scrub ok 2024-09-11T07:30:59.306 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:59 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:30:59.306 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:59 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:30:59.306 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:59 smithi022 ceph-mon[29462]: pgmap v178: 148 pgs: 148 active+clean; 6.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 862 op/s 2024-09-11T07:30:59.307 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:59 smithi022 ceph-mon[29462]: osdmap e133: 8 total, 8 up, 8 in 2024-09-11T07:30:59.307 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:59 smithi022 ceph-mon[29462]: 2.9 scrub starts 2024-09-11T07:30:59.307 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:30:59 smithi022 ceph-mon[29462]: 2.9 scrub ok 2024-09-11T07:30:59.330 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:59.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:59.331 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:30:59.376 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:30:59.376 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:30:59.383 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:30:59 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:30:59.384 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:59 smithi104 ceph-mon[33507]: 2.e scrub starts 2024-09-11T07:30:59.384 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:59 smithi104 ceph-mon[33507]: 2.e scrub ok 2024-09-11T07:30:59.384 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:59 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:30:59.384 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:59 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:30:59.384 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:59 smithi104 ceph-mon[33507]: pgmap v178: 148 pgs: 148 active+clean; 6.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 862 op/s 2024-09-11T07:30:59.384 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:59 smithi104 ceph-mon[33507]: osdmap e133: 8 total, 8 up, 8 in 2024-09-11T07:30:59.384 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:59 smithi104 ceph-mon[33507]: 2.9 scrub starts 2024-09-11T07:30:59.384 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:30:59 smithi104 ceph-mon[33507]: 2.9 scrub ok 2024-09-11T07:30:59.416 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:30:59.417 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:30:59.432 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:59.448 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:30:59.448 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:30:59.459 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:59.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:59.460 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:59.486 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:30:59.487 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:30:59.520 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:30:59.520 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:30:59.552 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:30:59.552 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:30:59.561 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:59.582 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:30:59.582 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:30:59.588 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:59.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:59.589 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:30:59.615 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:30:59.616 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:30:59.689 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:30:59.732 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:30:59.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:59.733 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:30:59.834 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:30:59.859 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:59.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:59.860 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:30:59.960 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:30:59.976 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:30:59.976 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:30:59.976 INFO:tasks.thrashosds.thrasher:Killing osd 7, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:30:59.976 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-11T07:30:59.976 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.7 2024-09-11T07:30:59.985 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:30:59.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:30:59.986 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:00.086 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:00.114 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:00.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:00.115 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:00.165 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:00 smithi104 systemd[1]: Stopping Ceph osd.7 for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:31:00.216 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:00.256 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:00.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:00.257 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:00.267 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:00 smithi022 ceph-mon[29462]: 4.0 scrub starts 2024-09-11T07:31:00.267 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:00 smithi022 ceph-mon[29462]: 4.0 scrub ok 2024-09-11T07:31:00.267 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:00 smithi022 ceph-mon[29462]: 2.c scrub starts 2024-09-11T07:31:00.267 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:00 smithi022 ceph-mon[29462]: 2.c scrub ok 2024-09-11T07:31:00.267 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:00 smithi022 ceph-mon[29462]: 7.4 scrub starts 2024-09-11T07:31:00.267 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:00 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:00.267 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:00 smithi022 ceph-mon[29462]: 7.b scrub ok 2024-09-11T07:31:00.267 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:00 smithi022 ceph-mon[29462]: 6.8 scrub starts 2024-09-11T07:31:00.268 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:00 smithi022 ceph-mon[29462]: 6.8 scrub ok 2024-09-11T07:31:00.268 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:00 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/870313248' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:31:00.268 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:00 smithi022 ceph-mon[37330]: 4.0 scrub starts 2024-09-11T07:31:00.268 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:00 smithi022 ceph-mon[37330]: 4.0 scrub ok 2024-09-11T07:31:00.268 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:00 smithi022 ceph-mon[37330]: 2.c scrub starts 2024-09-11T07:31:00.268 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:00 smithi022 ceph-mon[37330]: 2.c scrub ok 2024-09-11T07:31:00.268 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:00 smithi022 ceph-mon[37330]: 7.4 scrub starts 2024-09-11T07:31:00.268 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:00 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:00.268 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:00 smithi022 ceph-mon[37330]: 7.b scrub ok 2024-09-11T07:31:00.268 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:00 smithi022 ceph-mon[37330]: 6.8 scrub starts 2024-09-11T07:31:00.268 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:00 smithi022 ceph-mon[37330]: 6.8 scrub ok 2024-09-11T07:31:00.269 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:00 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/870313248' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:31:00.357 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:00.382 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:00.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:00.383 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:00.484 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:00.505 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:00 smithi104 ceph-mon[33507]: 4.0 scrub starts 2024-09-11T07:31:00.505 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:00 smithi104 ceph-mon[33507]: 4.0 scrub ok 2024-09-11T07:31:00.505 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:00 smithi104 ceph-mon[33507]: 2.c scrub starts 2024-09-11T07:31:00.505 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:00 smithi104 ceph-mon[33507]: 2.c scrub ok 2024-09-11T07:31:00.505 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:00 smithi104 ceph-mon[33507]: 7.4 scrub starts 2024-09-11T07:31:00.506 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:00 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:00.506 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:00 smithi104 ceph-mon[33507]: 7.b scrub ok 2024-09-11T07:31:00.506 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:00 smithi104 ceph-mon[33507]: 6.8 scrub starts 2024-09-11T07:31:00.506 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:00 smithi104 ceph-mon[33507]: 6.8 scrub ok 2024-09-11T07:31:00.506 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:00 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/870313248' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:31:00.515 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:00.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:00.516 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:00.617 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:00.647 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:00.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:00.648 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:00.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:31:00 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:31:00] "GET /metrics HTTP/1.1" 200 223712 "" "Prometheus/2.43.0" 2024-09-11T07:31:00.748 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:00.752 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T07:31:00.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:00.775 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:00.776 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:00.802 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:31:00.802 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:31:00.802 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2024-09-11T07:31:00.802 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2024-09-11T07:31:00.802 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:31:00.877 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:00.903 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:00.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:00.904 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:01.005 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:01.033 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:01.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:01.034 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:01.135 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:01.159 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:01.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:01.160 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:01.250 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:31:01.261 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:01.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:01.324 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:01.324 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:01.421 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:01 smithi022 ceph-mon[29462]: pgmap v180: 148 pgs: 3 active+clean+scrubbing, 145 active+clean; 5.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.58k op/s 2024-09-11T07:31:01.422 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:01 smithi022 ceph-mon[29462]: 2.1d scrub starts 2024-09-11T07:31:01.422 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:01 smithi022 ceph-mon[29462]: 2.1d scrub ok 2024-09-11T07:31:01.422 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:01 smithi022 ceph-mon[29462]: 7.f scrub ok 2024-09-11T07:31:01.422 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:01 smithi022 ceph-mon[37330]: pgmap v180: 148 pgs: 3 active+clean+scrubbing, 145 active+clean; 5.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.58k op/s 2024-09-11T07:31:01.422 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:01 smithi022 ceph-mon[37330]: 2.1d scrub starts 2024-09-11T07:31:01.422 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:01 smithi022 ceph-mon[37330]: 2.1d scrub ok 2024-09-11T07:31:01.422 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:01 smithi022 ceph-mon[37330]: 7.f scrub ok 2024-09-11T07:31:01.425 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:01.464 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:01.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:01.465 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:01.502 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:01 smithi104 ceph-mon[33507]: pgmap v180: 148 pgs: 3 active+clean+scrubbing, 145 active+clean; 5.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.58k op/s 2024-09-11T07:31:01.502 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:01 smithi104 ceph-mon[33507]: 2.1d scrub starts 2024-09-11T07:31:01.502 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:01 smithi104 ceph-mon[33507]: 2.1d scrub ok 2024-09-11T07:31:01.502 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:01 smithi104 ceph-mon[33507]: 7.f scrub ok 2024-09-11T07:31:01.540 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-11T07:31:01.567 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:01.612 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:01.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:01.613 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:01.713 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:01.755 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:01.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:01.756 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:01.779 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:01 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[55581]: 2024-09-11T07:31:01.705+0000 7f19b2253640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:31:01.779 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:01 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[55581]: 2024-09-11T07:31:01.705+0000 7f19b2253640 -1 osd.7 133 *** Got signal Terminated *** 2024-09-11T07:31:01.780 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:01 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[55581]: 2024-09-11T07:31:01.705+0000 7f19b2253640 -1 osd.7 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:31:01.857 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:01.865 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:31:01.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:01.885 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:01.885 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:01.986 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:02.030 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:02.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:02.032 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:02.133 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:02.174 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:02.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:02.175 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:02.265 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:02 smithi104 ceph-mon[33507]: 6.11 scrub starts 2024-09-11T07:31:02.265 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:02 smithi104 ceph-mon[33507]: 6.11 scrub ok 2024-09-11T07:31:02.265 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:02 smithi104 ceph-mon[33507]: 4.e scrub starts 2024-09-11T07:31:02.265 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:02 smithi104 ceph-mon[33507]: 4.e scrub ok 2024-09-11T07:31:02.266 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:02 smithi104 ceph-mon[33507]: 5.7 scrub starts 2024-09-11T07:31:02.266 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:02 smithi104 ceph-mon[33507]: 5.7 scrub ok 2024-09-11T07:31:02.266 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:02 smithi104 ceph-mon[33507]: 5.2 scrub starts 2024-09-11T07:31:02.266 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:02 smithi104 ceph-mon[33507]: 5.2 scrub ok 2024-09-11T07:31:02.266 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:02 smithi104 ceph-mon[33507]: 2.4 scrub starts 2024-09-11T07:31:02.266 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:02 smithi104 ceph-mon[33507]: 2.4 scrub ok 2024-09-11T07:31:02.266 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:02 smithi104 ceph-mon[33507]: osd.7 marked itself down and dead 2024-09-11T07:31:02.276 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:02.298 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:02.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:02.299 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:02.307 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:31:02.399 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:02.421 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:02.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:02.422 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:02.425 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:02 smithi022 ceph-mon[29462]: 6.11 scrub starts 2024-09-11T07:31:02.425 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:02 smithi022 ceph-mon[29462]: 6.11 scrub ok 2024-09-11T07:31:02.425 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:02 smithi022 ceph-mon[29462]: 4.e scrub starts 2024-09-11T07:31:02.425 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:02 smithi022 ceph-mon[29462]: 4.e scrub ok 2024-09-11T07:31:02.425 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:02 smithi022 ceph-mon[29462]: 5.7 scrub starts 2024-09-11T07:31:02.426 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:02 smithi022 ceph-mon[29462]: 5.7 scrub ok 2024-09-11T07:31:02.426 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:02 smithi022 ceph-mon[29462]: 5.2 scrub starts 2024-09-11T07:31:02.426 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:02 smithi022 ceph-mon[29462]: 5.2 scrub ok 2024-09-11T07:31:02.426 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:02 smithi022 ceph-mon[29462]: 2.4 scrub starts 2024-09-11T07:31:02.426 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:02 smithi022 ceph-mon[29462]: 2.4 scrub ok 2024-09-11T07:31:02.426 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:02 smithi022 ceph-mon[29462]: osd.7 marked itself down and dead 2024-09-11T07:31:02.426 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:02 smithi022 ceph-mon[37330]: 6.11 scrub starts 2024-09-11T07:31:02.426 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:02 smithi022 ceph-mon[37330]: 6.11 scrub ok 2024-09-11T07:31:02.426 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:02 smithi022 ceph-mon[37330]: 4.e scrub starts 2024-09-11T07:31:02.426 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:02 smithi022 ceph-mon[37330]: 4.e scrub ok 2024-09-11T07:31:02.426 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:02 smithi022 ceph-mon[37330]: 5.7 scrub starts 2024-09-11T07:31:02.426 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:02 smithi022 ceph-mon[37330]: 5.7 scrub ok 2024-09-11T07:31:02.427 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:02 smithi022 ceph-mon[37330]: 5.2 scrub starts 2024-09-11T07:31:02.427 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:02 smithi022 ceph-mon[37330]: 5.2 scrub ok 2024-09-11T07:31:02.427 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:02 smithi022 ceph-mon[37330]: 2.4 scrub starts 2024-09-11T07:31:02.427 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:02 smithi022 ceph-mon[37330]: 2.4 scrub ok 2024-09-11T07:31:02.427 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:02 smithi022 ceph-mon[37330]: osd.7 marked itself down and dead 2024-09-11T07:31:02.523 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:02.544 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:02.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:02.545 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:02.646 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:02.668 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:02.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:02.669 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:02.770 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:02.792 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:02.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:02.793 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:02.894 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:02.915 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:02.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:02.916 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:03.017 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:03.042 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:03.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:03.043 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:03.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:03.168 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:03.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:03.169 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:03.270 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:03.279 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:03 smithi104 podman[75917]: 2024-09-11 07:31:03.011430622 +0000 UTC m=+1.436012930 container died 8d5070e0b1a61abb80aa000227b3bf2c4f8478257e16098f87629a9594f52e03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-11T07:31:03.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:03 smithi104 ceph-mon[33507]: 6.f scrub starts 2024-09-11T07:31:03.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:03 smithi104 ceph-mon[33507]: 6.f scrub ok 2024-09-11T07:31:03.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:03 smithi104 ceph-mon[33507]: pgmap v181: 148 pgs: 3 active+clean+scrubbing, 145 active+clean; 5.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 745 B/s rd, 0 B/s wr, 763 op/s 2024-09-11T07:31:03.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:03 smithi104 ceph-mon[33507]: 7.4 scrub ok 2024-09-11T07:31:03.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:03 smithi104 ceph-mon[33507]: 6.19 scrub starts 2024-09-11T07:31:03.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:03 smithi104 ceph-mon[33507]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-11T07:31:03.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:03 smithi104 ceph-mon[33507]: 6.19 scrub ok 2024-09-11T07:31:03.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:03 smithi104 ceph-mon[33507]: osdmap e134: 8 total, 7 up, 8 in 2024-09-11T07:31:03.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:03.310 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:03.311 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:03.411 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:03.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:03.465 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:03.465 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:03.492 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:03 smithi022 ceph-mon[29462]: 6.f scrub starts 2024-09-11T07:31:03.493 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:03 smithi022 ceph-mon[29462]: 6.f scrub ok 2024-09-11T07:31:03.493 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:03 smithi022 ceph-mon[29462]: pgmap v181: 148 pgs: 3 active+clean+scrubbing, 145 active+clean; 5.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 745 B/s rd, 0 B/s wr, 763 op/s 2024-09-11T07:31:03.493 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:03 smithi022 ceph-mon[29462]: 7.4 scrub ok 2024-09-11T07:31:03.493 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:03 smithi022 ceph-mon[29462]: 6.19 scrub starts 2024-09-11T07:31:03.493 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:03 smithi022 ceph-mon[29462]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-11T07:31:03.493 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:03 smithi022 ceph-mon[29462]: 6.19 scrub ok 2024-09-11T07:31:03.493 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:03 smithi022 ceph-mon[29462]: osdmap e134: 8 total, 7 up, 8 in 2024-09-11T07:31:03.494 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:03 smithi022 ceph-mon[37330]: 6.f scrub starts 2024-09-11T07:31:03.494 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:03 smithi022 ceph-mon[37330]: 6.f scrub ok 2024-09-11T07:31:03.494 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:03 smithi022 ceph-mon[37330]: pgmap v181: 148 pgs: 3 active+clean+scrubbing, 145 active+clean; 5.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 745 B/s rd, 0 B/s wr, 763 op/s 2024-09-11T07:31:03.494 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:03 smithi022 ceph-mon[37330]: 7.4 scrub ok 2024-09-11T07:31:03.494 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:03 smithi022 ceph-mon[37330]: 6.19 scrub starts 2024-09-11T07:31:03.494 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:03 smithi022 ceph-mon[37330]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-11T07:31:03.495 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:03 smithi022 ceph-mon[37330]: 6.19 scrub ok 2024-09-11T07:31:03.495 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:03 smithi022 ceph-mon[37330]: osdmap e134: 8 total, 7 up, 8 in 2024-09-11T07:31:03.566 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:03.615 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:03.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:03.616 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:03.717 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:03.744 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:03.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:03.745 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:03.779 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:03 smithi104 podman[75917]: 2024-09-11 07:31:03.556783828 +0000 UTC m=+1.981366134 container cleanup 8d5070e0b1a61abb80aa000227b3bf2c4f8478257e16098f87629a9594f52e03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-11T07:31:03.780 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:03 smithi104 bash[75917]: ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7 2024-09-11T07:31:03.845 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:03.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:03.896 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:03.896 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:03.959 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:31:03.998 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:04.043 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:03 smithi104 podman[76053]: 2024-09-11 07:31:03.783765384 +0000 UTC m=+1.195660911 container remove 8d5070e0b1a61abb80aa000227b3bf2c4f8478257e16098f87629a9594f52e03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-11T07:31:04.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:04.047 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:04.047 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:04.147 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:04.198 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:04.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:04.199 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:04.300 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:04.325 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:04.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:04.326 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:04.385 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:31:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:31:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:31:04.386 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:04 smithi104 ceph-mon[33507]: 5.1c scrub starts 2024-09-11T07:31:04.386 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:04 smithi104 ceph-mon[33507]: 5.1c scrub ok 2024-09-11T07:31:04.386 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:04 smithi104 ceph-mon[33507]: 7.9 scrub starts 2024-09-11T07:31:04.386 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:04 smithi104 ceph-mon[33507]: osdmap e135: 8 total, 7 up, 8 in 2024-09-11T07:31:04.386 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:04 smithi104 ceph-mon[33507]: 2.15 scrub starts 2024-09-11T07:31:04.386 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:04 smithi104 ceph-mon[33507]: 2.15 scrub ok 2024-09-11T07:31:04.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:04 smithi022 ceph-mon[29462]: 5.1c scrub starts 2024-09-11T07:31:04.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:04 smithi022 ceph-mon[29462]: 5.1c scrub ok 2024-09-11T07:31:04.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:04 smithi022 ceph-mon[29462]: 7.9 scrub starts 2024-09-11T07:31:04.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:04 smithi022 ceph-mon[29462]: osdmap e135: 8 total, 7 up, 8 in 2024-09-11T07:31:04.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:04 smithi022 ceph-mon[29462]: 2.15 scrub starts 2024-09-11T07:31:04.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:04 smithi022 ceph-mon[29462]: 2.15 scrub ok 2024-09-11T07:31:04.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:04 smithi022 ceph-mon[37330]: 5.1c scrub starts 2024-09-11T07:31:04.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:04 smithi022 ceph-mon[37330]: 5.1c scrub ok 2024-09-11T07:31:04.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:04 smithi022 ceph-mon[37330]: 7.9 scrub starts 2024-09-11T07:31:04.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:04 smithi022 ceph-mon[37330]: osdmap e135: 8 total, 7 up, 8 in 2024-09-11T07:31:04.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:04 smithi022 ceph-mon[37330]: 2.15 scrub starts 2024-09-11T07:31:04.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:04 smithi022 ceph-mon[37330]: 2.15 scrub ok 2024-09-11T07:31:04.427 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:04.452 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:04.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:04.453 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:04.514 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:31:04.515 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2024-09-11T07:31:04.515 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2024-09-11T07:31:04.515 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2024-09-11T07:31:04.515 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.515 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:12277 7.0 7:009e457c:::benchmark_data_smithi022_77720_object12276:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-11T07:31:04.515 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:29:06.542620+0000", 2024-09-11T07:31:04.515 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 117.966855037, 2024-09-11T07:31:04.515 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.053338785999999999, 2024-09-11T07:31:04.515 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:31:04.515 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:31:04.515 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:31:04.515 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:31:04.515 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:31:04.516 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 12277 2024-09-11T07:31:04.516 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.516 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:31:04.516 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.516 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:31:04.516 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.542620+0000", 2024-09-11T07:31:04.516 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.516 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.516 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.516 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:31:04.516 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.542620+0000", 2024-09-11T07:31:04.516 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.516 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.517 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.517 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:31:04.517 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.542619+0000", 2024-09-11T07:31:04.517 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:31:04.517 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.517 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.517 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:31:04.517 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.542677+0000", 2024-09-11T07:31:04.517 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8553e-05 2024-09-11T07:31:04.517 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.517 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.517 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:31:04.517 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.542688+0000", 2024-09-11T07:31:04.517 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0321e-05 2024-09-11T07:31:04.518 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.518 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.518 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:31:04.518 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.542692+0000", 2024-09-11T07:31:04.518 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7740000000000004e-06 2024-09-11T07:31:04.518 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.518 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.518 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:31:04.518 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.542723+0000", 2024-09-11T07:31:04.518 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.0929999999999997e-05 2024-09-11T07:31:04.518 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.518 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.518 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:31:04.518 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.542797+0000", 2024-09-11T07:31:04.518 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3886000000000004e-05 2024-09-11T07:31:04.519 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.519 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.519 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:31:04.519 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.542891+0000", 2024-09-11T07:31:04.519 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3490000000000001e-05 2024-09-11T07:31:04.519 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.519 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.519 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:31:04.519 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.543415+0000", 2024-09-11T07:31:04.519 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00052410800000000002 2024-09-11T07:31:04.519 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.519 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.519 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.520 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.544176+0000", 2024-09-11T07:31:04.520 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000761478 2024-09-11T07:31:04.520 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.520 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.520 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.520 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.595875+0000", 2024-09-11T07:31:04.520 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.051698979999999999 2024-09-11T07:31:04.520 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.520 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.520 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:31:04.520 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.595922+0000", 2024-09-11T07:31:04.520 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6297000000000001e-05 2024-09-11T07:31:04.520 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.521 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.521 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:31:04.521 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.595959+0000", 2024-09-11T07:31:04.521 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7481999999999998e-05 2024-09-11T07:31:04.521 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.521 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:31:04.521 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.521 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.521 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.521 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:12299 7.0 7:06e1769f:::benchmark_data_smithi022_77720_object12298:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-11T07:31:04.521 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:29:06.598512+0000", 2024-09-11T07:31:04.521 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 117.910963143, 2024-09-11T07:31:04.521 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.042126969, 2024-09-11T07:31:04.521 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:31:04.522 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:31:04.522 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:31:04.522 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:31:04.522 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:31:04.522 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 12299 2024-09-11T07:31:04.522 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.522 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:31:04.522 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.522 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:31:04.522 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.598512+0000", 2024-09-11T07:31:04.522 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.522 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.522 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.522 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:31:04.523 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.598512+0000", 2024-09-11T07:31:04.523 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.523 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.523 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.523 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:31:04.523 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.598510+0000", 2024-09-11T07:31:04.523 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:31:04.523 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.523 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.523 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:31:04.523 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.598599+0000", 2024-09-11T07:31:04.523 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8523000000000006e-05 2024-09-11T07:31:04.523 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.523 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.523 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:31:04.523 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.598611+0000", 2024-09-11T07:31:04.524 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2293000000000001e-05 2024-09-11T07:31:04.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.524 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.524 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:31:04.524 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.598617+0000", 2024-09-11T07:31:04.524 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0499999999999997e-06 2024-09-11T07:31:04.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.524 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.524 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:31:04.524 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.598660+0000", 2024-09-11T07:31:04.524 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3034999999999997e-05 2024-09-11T07:31:04.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.524 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.524 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:31:04.525 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.598736+0000", 2024-09-11T07:31:04.525 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6029e-05 2024-09-11T07:31:04.525 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.525 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.525 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:31:04.525 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.598862+0000", 2024-09-11T07:31:04.525 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012608500000000001 2024-09-11T07:31:04.525 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.525 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.525 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:31:04.525 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.599455+0000", 2024-09-11T07:31:04.525 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00059235199999999998 2024-09-11T07:31:04.525 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.525 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.526 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.526 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.600766+0000", 2024-09-11T07:31:04.526 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0013110120000000001 2024-09-11T07:31:04.526 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.526 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.526 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.526 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.640558+0000", 2024-09-11T07:31:04.526 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.039791950999999999 2024-09-11T07:31:04.526 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.526 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.526 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:31:04.526 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.640602+0000", 2024-09-11T07:31:04.526 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4314999999999998e-05 2024-09-11T07:31:04.526 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.526 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.527 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:31:04.527 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:06.640639+0000", 2024-09-11T07:31:04.527 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7030999999999998e-05 2024-09-11T07:31:04.527 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.527 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:31:04.527 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.527 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.527 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.527 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:22183 7.0 7:049033f5:::benchmark_data_smithi022_77720_object22182:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-11T07:31:04.527 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:29:10.565289+0000", 2024-09-11T07:31:04.527 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 113.944186704, 2024-09-11T07:31:04.527 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.050790452, 2024-09-11T07:31:04.527 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:31:04.527 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:31:04.527 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:31:04.527 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:31:04.528 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:31:04.528 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 22183 2024-09-11T07:31:04.528 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.528 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:31:04.528 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.528 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:31:04.528 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:10.565289+0000", 2024-09-11T07:31:04.528 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.528 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.528 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.528 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:31:04.528 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:10.565289+0000", 2024-09-11T07:31:04.528 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.528 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.528 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.529 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:31:04.529 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:10.565287+0000", 2024-09-11T07:31:04.529 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:31:04.529 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.529 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.529 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:31:04.529 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:10.565352+0000", 2024-09-11T07:31:04.529 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5067000000000006e-05 2024-09-11T07:31:04.529 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.529 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.529 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:31:04.529 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:10.565362+0000", 2024-09-11T07:31:04.529 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0253e-05 2024-09-11T07:31:04.529 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.530 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.530 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:31:04.530 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:10.565367+0000", 2024-09-11T07:31:04.530 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7439999999999998e-06 2024-09-11T07:31:04.530 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.530 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.530 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:31:04.530 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:10.565408+0000", 2024-09-11T07:31:04.530 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1493999999999999e-05 2024-09-11T07:31:04.530 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.530 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.530 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:31:04.530 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:10.565535+0000", 2024-09-11T07:31:04.530 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012666399999999999 2024-09-11T07:31:04.530 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.531 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.531 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:31:04.531 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:10.565689+0000", 2024-09-11T07:31:04.531 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015435000000000001 2024-09-11T07:31:04.531 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.531 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.531 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:31:04.531 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:10.566451+0000", 2024-09-11T07:31:04.531 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00076186799999999996 2024-09-11T07:31:04.531 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.531 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.531 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.531 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:10.566971+0000", 2024-09-11T07:31:04.531 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00051990900000000004 2024-09-11T07:31:04.532 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.532 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.532 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.532 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:10.615996+0000", 2024-09-11T07:31:04.532 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.049025177000000003 2024-09-11T07:31:04.532 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.532 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.532 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:31:04.532 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:10.616045+0000", 2024-09-11T07:31:04.532 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8749999999999999e-05 2024-09-11T07:31:04.532 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.532 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.532 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:31:04.532 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:10.616079+0000", 2024-09-11T07:31:04.533 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4066000000000002e-05 2024-09-11T07:31:04.533 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.533 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:31:04.533 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.533 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.533 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.533 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:24108 7.0 7:07baab64:::benchmark_data_smithi022_77720_object24107:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-11T07:31:04.533 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:29:11.441644+0000", 2024-09-11T07:31:04.533 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 113.067830792, 2024-09-11T07:31:04.533 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.060108473000000003, 2024-09-11T07:31:04.533 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:31:04.533 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:31:04.533 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:31:04.534 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:31:04.534 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:31:04.534 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 24108 2024-09-11T07:31:04.534 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.534 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:31:04.534 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.534 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:31:04.534 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:11.441644+0000", 2024-09-11T07:31:04.534 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.534 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.534 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.534 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:31:04.534 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:11.441644+0000", 2024-09-11T07:31:04.534 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.535 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.535 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.535 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:31:04.535 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:11.441642+0000", 2024-09-11T07:31:04.535 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999971 2024-09-11T07:31:04.535 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.535 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.535 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:31:04.535 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:11.441686+0000", 2024-09-11T07:31:04.535 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3807999999999998e-05 2024-09-11T07:31:04.535 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.535 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.535 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:31:04.536 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:11.441702+0000", 2024-09-11T07:31:04.536 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6509000000000001e-05 2024-09-11T07:31:04.536 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.536 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.536 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:31:04.536 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:11.441709+0000", 2024-09-11T07:31:04.536 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4640000000000003e-06 2024-09-11T07:31:04.536 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.536 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.536 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:31:04.536 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:11.441753+0000", 2024-09-11T07:31:04.536 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4895999999999998e-05 2024-09-11T07:31:04.536 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.536 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.536 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:31:04.537 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:11.441889+0000", 2024-09-11T07:31:04.537 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000135852 2024-09-11T07:31:04.537 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.537 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.537 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:31:04.537 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:11.442026+0000", 2024-09-11T07:31:04.537 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000136644 2024-09-11T07:31:04.537 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.537 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.537 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:31:04.537 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:11.442907+0000", 2024-09-11T07:31:04.537 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00088060299999999998 2024-09-11T07:31:04.537 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.537 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.537 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.538 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:11.495475+0000", 2024-09-11T07:31:04.538 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.052568888000000001 2024-09-11T07:31:04.538 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.538 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.538 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.538 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:11.501699+0000", 2024-09-11T07:31:04.538 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0062236349999999999 2024-09-11T07:31:04.538 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.538 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.538 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:31:04.538 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:11.501731+0000", 2024-09-11T07:31:04.538 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2203999999999999e-05 2024-09-11T07:31:04.538 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.538 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.538 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:31:04.539 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:11.501753+0000", 2024-09-11T07:31:04.539 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1631e-05 2024-09-11T07:31:04.539 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.539 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:31:04.539 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.539 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.539 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.539 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:27104 7.0 7:01142e94:::benchmark_data_smithi022_77720_object27103:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-11T07:31:04.539 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:29:12.842499+0000", 2024-09-11T07:31:04.539 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 111.666976236, 2024-09-11T07:31:04.539 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.043722165, 2024-09-11T07:31:04.539 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:31:04.539 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:31:04.539 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:31:04.539 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:31:04.539 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:31:04.540 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 27104 2024-09-11T07:31:04.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.540 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:31:04.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.540 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:31:04.540 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.842499+0000", 2024-09-11T07:31:04.540 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.540 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:31:04.540 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.842499+0000", 2024-09-11T07:31:04.540 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.541 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:31:04.541 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.842497+0000", 2024-09-11T07:31:04.541 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:31:04.541 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.542 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.542 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:31:04.542 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.842543+0000", 2024-09-11T07:31:04.542 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6548000000000002e-05 2024-09-11T07:31:04.542 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.542 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.542 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:31:04.542 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.842557+0000", 2024-09-11T07:31:04.542 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3772e-05 2024-09-11T07:31:04.542 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.542 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.542 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:31:04.542 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.842564+0000", 2024-09-11T07:31:04.542 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5490000000000003e-06 2024-09-11T07:31:04.543 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.543 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.543 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:31:04.543 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.842621+0000", 2024-09-11T07:31:04.543 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7720000000000003e-05 2024-09-11T07:31:04.543 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.543 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.543 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:31:04.543 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.842728+0000", 2024-09-11T07:31:04.543 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000106759 2024-09-11T07:31:04.543 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.543 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.543 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:31:04.543 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.842873+0000", 2024-09-11T07:31:04.543 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014471800000000001 2024-09-11T07:31:04.544 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.544 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.544 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:31:04.544 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.843961+0000", 2024-09-11T07:31:04.544 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001087729 2024-09-11T07:31:04.544 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.544 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.544 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.544 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.845042+0000", 2024-09-11T07:31:04.544 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0010811709999999999 2024-09-11T07:31:04.544 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.544 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.544 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.544 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.886167+0000", 2024-09-11T07:31:04.544 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.041125545999999999 2024-09-11T07:31:04.545 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.545 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:31:04.545 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.886197+0000", 2024-09-11T07:31:04.545 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9788999999999999e-05 2024-09-11T07:31:04.545 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.545 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:31:04.545 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.886221+0000", 2024-09-11T07:31:04.545 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3988000000000001e-05 2024-09-11T07:31:04.545 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.545 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:31:04.545 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.545 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.546 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:27113 7.0 7:034aaca4:::benchmark_data_smithi022_77720_object27112:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-11T07:31:04.546 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:29:12.845924+0000", 2024-09-11T07:31:04.546 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 111.66355129199999, 2024-09-11T07:31:04.546 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.042771165, 2024-09-11T07:31:04.546 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:31:04.546 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:31:04.546 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:31:04.546 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:31:04.546 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:31:04.546 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 27113 2024-09-11T07:31:04.546 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.546 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:31:04.546 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.546 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:31:04.547 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.845924+0000", 2024-09-11T07:31:04.547 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.547 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.547 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.547 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:31:04.547 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.845924+0000", 2024-09-11T07:31:04.547 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.547 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.547 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.547 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:31:04.547 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.845922+0000", 2024-09-11T07:31:04.547 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:31:04.547 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.547 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.548 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:31:04.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.845981+0000", 2024-09-11T07:31:04.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8928000000000002e-05 2024-09-11T07:31:04.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.548 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.548 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:31:04.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.845993+0000", 2024-09-11T07:31:04.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2432000000000001e-05 2024-09-11T07:31:04.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.548 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.548 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:31:04.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.846000+0000", 2024-09-11T07:31:04.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9970000000000001e-06 2024-09-11T07:31:04.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.549 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:31:04.549 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.846032+0000", 2024-09-11T07:31:04.549 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1733000000000003e-05 2024-09-11T07:31:04.549 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.549 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:31:04.549 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.846116+0000", 2024-09-11T07:31:04.549 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3814999999999996e-05 2024-09-11T07:31:04.549 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.549 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:31:04.549 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.846209+0000", 2024-09-11T07:31:04.549 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2724000000000002e-05 2024-09-11T07:31:04.549 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.550 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.550 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:31:04.550 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.847019+0000", 2024-09-11T07:31:04.550 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00081048600000000002 2024-09-11T07:31:04.550 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.550 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.550 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.550 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.847370+0000", 2024-09-11T07:31:04.550 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00035055200000000001 2024-09-11T07:31:04.550 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.550 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.550 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.550 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.888645+0000", 2024-09-11T07:31:04.550 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.041275312000000001 2024-09-11T07:31:04.550 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.551 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.551 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:31:04.551 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.888674+0000", 2024-09-11T07:31:04.551 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9204e-05 2024-09-11T07:31:04.551 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.551 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.551 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:31:04.551 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:12.888695+0000", 2024-09-11T07:31:04.551 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1052000000000001e-05 2024-09-11T07:31:04.551 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.551 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:31:04.551 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.551 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.551 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.551 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:59834 7.0 7:0fc368a8:::benchmark_data_smithi022_77720_object59833:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-11T07:31:04.551 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:29:32.630993+0000", 2024-09-11T07:31:04.552 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 91.878482336999994, 2024-09-11T07:31:04.552 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.066140649999999995, 2024-09-11T07:31:04.552 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:31:04.552 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:31:04.552 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:31:04.552 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:31:04.552 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:31:04.552 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 59834 2024-09-11T07:31:04.552 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.552 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:31:04.552 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.552 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:31:04.552 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.630993+0000", 2024-09-11T07:31:04.552 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.553 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.553 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.553 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:31:04.553 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.630993+0000", 2024-09-11T07:31:04.553 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.553 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.553 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.553 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:31:04.553 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.630991+0000", 2024-09-11T07:31:04.553 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:31:04.553 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.553 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.553 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:31:04.553 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.631043+0000", 2024-09-11T07:31:04.553 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2442999999999999e-05 2024-09-11T07:31:04.554 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.554 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.554 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:31:04.554 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.631056+0000", 2024-09-11T07:31:04.554 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2286999999999999e-05 2024-09-11T07:31:04.554 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.554 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.554 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:31:04.554 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.631061+0000", 2024-09-11T07:31:04.555 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8520000000000003e-06 2024-09-11T07:31:04.555 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.555 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.555 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:31:04.555 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.631107+0000", 2024-09-11T07:31:04.555 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6319000000000002e-05 2024-09-11T07:31:04.555 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.555 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.555 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:31:04.555 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.631226+0000", 2024-09-11T07:31:04.555 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011901299999999999 2024-09-11T07:31:04.556 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.556 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.556 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:31:04.556 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.631372+0000", 2024-09-11T07:31:04.556 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000146312 2024-09-11T07:31:04.556 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.556 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.556 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:31:04.556 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.635131+0000", 2024-09-11T07:31:04.556 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003758974 2024-09-11T07:31:04.557 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.557 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.557 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.557 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.636521+0000", 2024-09-11T07:31:04.557 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001389805 2024-09-11T07:31:04.557 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.557 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.557 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.557 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.697028+0000", 2024-09-11T07:31:04.557 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.060506521000000001 2024-09-11T07:31:04.557 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.558 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.558 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:31:04.558 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.697083+0000", 2024-09-11T07:31:04.558 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5442000000000002e-05 2024-09-11T07:31:04.558 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.558 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.558 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:31:04.558 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.697134+0000", 2024-09-11T07:31:04.558 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0640000000000003e-05 2024-09-11T07:31:04.558 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.558 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:31:04.559 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.559 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.559 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.559 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:59835 7.0 7:06238b80:::benchmark_data_smithi022_77720_object59834:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-11T07:31:04.559 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:29:32.631067+0000", 2024-09-11T07:31:04.559 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 91.878408179000004, 2024-09-11T07:31:04.559 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.066172445999999996, 2024-09-11T07:31:04.559 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:31:04.559 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:31:04.559 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:31:04.559 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:31:04.560 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:31:04.560 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 59835 2024-09-11T07:31:04.560 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.560 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:31:04.560 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.560 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:31:04.560 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.631067+0000", 2024-09-11T07:31:04.560 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.560 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.560 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.560 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:31:04.561 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.631067+0000", 2024-09-11T07:31:04.561 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.561 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.561 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.561 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:31:04.561 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.631067+0000", 2024-09-11T07:31:04.561 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:31:04.561 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.561 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.561 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:31:04.561 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.631091+0000", 2024-09-11T07:31:04.562 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.4664e-05 2024-09-11T07:31:04.562 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.562 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.562 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:31:04.562 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.631096+0000", 2024-09-11T07:31:04.562 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.459e-06 2024-09-11T07:31:04.562 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.562 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.562 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:31:04.562 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.631098+0000", 2024-09-11T07:31:04.562 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.6189999999999998e-06 2024-09-11T07:31:04.563 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.563 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.563 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:31:04.563 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.633239+0000", 2024-09-11T07:31:04.563 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0021406490000000001 2024-09-11T07:31:04.563 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.563 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.563 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:31:04.563 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.633359+0000", 2024-09-11T07:31:04.563 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000119894 2024-09-11T07:31:04.563 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.563 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.564 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:31:04.564 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.633489+0000", 2024-09-11T07:31:04.564 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013036699999999999 2024-09-11T07:31:04.564 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.564 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.564 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:31:04.564 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.635306+0000", 2024-09-11T07:31:04.564 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0018172939999999999 2024-09-11T07:31:04.564 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.564 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.565 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.565 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.636568+0000", 2024-09-11T07:31:04.565 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0012617699999999999 2024-09-11T07:31:04.565 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.565 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.565 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.565 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.697179+0000", 2024-09-11T07:31:04.565 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.060610576999999999 2024-09-11T07:31:04.565 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.565 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.565 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:31:04.566 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.697210+0000", 2024-09-11T07:31:04.566 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.133e-05 2024-09-11T07:31:04.566 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.566 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.566 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:31:04.566 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.697240+0000", 2024-09-11T07:31:04.566 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9383999999999999e-05 2024-09-11T07:31:04.566 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.566 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:31:04.566 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.566 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.566 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.567 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:59844 7.0 7:0aa14a53:::benchmark_data_smithi022_77720_object59843:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-11T07:31:04.567 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:29:32.637235+0000", 2024-09-11T07:31:04.567 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 91.872240046000002, 2024-09-11T07:31:04.567 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.060129231999999998, 2024-09-11T07:31:04.567 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:31:04.567 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:31:04.567 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:31:04.567 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:31:04.567 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:31:04.567 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 59844 2024-09-11T07:31:04.567 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.567 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:31:04.568 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.568 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:31:04.568 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.637235+0000", 2024-09-11T07:31:04.568 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.568 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.568 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.568 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:31:04.568 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.637235+0000", 2024-09-11T07:31:04.568 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.568 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.568 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.568 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:31:04.569 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.637231+0000", 2024-09-11T07:31:04.569 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999957 2024-09-11T07:31:04.569 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.569 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.569 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:31:04.569 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.637316+0000", 2024-09-11T07:31:04.569 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4895999999999995e-05 2024-09-11T07:31:04.569 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.569 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.569 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:31:04.569 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.637332+0000", 2024-09-11T07:31:04.569 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6138e-05 2024-09-11T07:31:04.569 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.570 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.570 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:31:04.570 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.637339+0000", 2024-09-11T07:31:04.570 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0489999999999998e-06 2024-09-11T07:31:04.570 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.570 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.570 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:31:04.570 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.637394+0000", 2024-09-11T07:31:04.570 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5065000000000002e-05 2024-09-11T07:31:04.570 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.570 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.570 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:31:04.571 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.637519+0000", 2024-09-11T07:31:04.571 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012501799999999999 2024-09-11T07:31:04.571 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.571 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.571 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:31:04.571 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.637657+0000", 2024-09-11T07:31:04.571 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000138095 2024-09-11T07:31:04.571 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.571 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.571 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:31:04.571 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.639559+0000", 2024-09-11T07:31:04.571 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0019018520000000001 2024-09-11T07:31:04.572 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.572 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.572 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.572 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.642280+0000", 2024-09-11T07:31:04.572 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0027212640000000001 2024-09-11T07:31:04.572 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.572 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.572 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.572 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.697278+0000", 2024-09-11T07:31:04.572 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.054997348000000001 2024-09-11T07:31:04.572 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.572 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.573 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:31:04.573 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.697321+0000", 2024-09-11T07:31:04.573 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3764000000000003e-05 2024-09-11T07:31:04.573 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.573 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.573 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:31:04.573 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.697364+0000", 2024-09-11T07:31:04.573 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3170999999999997e-05 2024-09-11T07:31:04.573 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.573 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:31:04.573 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.574 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.574 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.574 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:59853 7.0 7:0119d386:::benchmark_data_smithi022_77720_object59852:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-11T07:31:04.574 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:29:32.643179+0000", 2024-09-11T07:31:04.574 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 91.866296198000001, 2024-09-11T07:31:04.574 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.062574705999999994, 2024-09-11T07:31:04.574 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:31:04.574 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:31:04.574 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:31:04.574 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:31:04.574 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:31:04.575 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 59853 2024-09-11T07:31:04.575 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.575 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:31:04.575 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.575 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:31:04.575 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.643179+0000", 2024-09-11T07:31:04.575 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.575 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.575 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.575 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:31:04.575 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.643179+0000", 2024-09-11T07:31:04.576 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.576 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.576 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.576 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:31:04.576 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.643174+0000", 2024-09-11T07:31:04.576 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999952 2024-09-11T07:31:04.576 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.576 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.576 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:31:04.576 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.643223+0000", 2024-09-11T07:31:04.576 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8937999999999998e-05 2024-09-11T07:31:04.576 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.577 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.577 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:31:04.577 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.643240+0000", 2024-09-11T07:31:04.577 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6603e-05 2024-09-11T07:31:04.577 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.577 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.577 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:31:04.577 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.643246+0000", 2024-09-11T07:31:04.577 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8170000000000004e-06 2024-09-11T07:31:04.577 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.577 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.578 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:31:04.578 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.643286+0000", 2024-09-11T07:31:04.578 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0293999999999997e-05 2024-09-11T07:31:04.578 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.578 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.578 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:31:04.578 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.643414+0000", 2024-09-11T07:31:04.578 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000128274 2024-09-11T07:31:04.578 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.578 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.578 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:31:04.578 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.643554+0000", 2024-09-11T07:31:04.579 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013956299999999999 2024-09-11T07:31:04.579 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.579 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.579 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:31:04.579 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.645801+0000", 2024-09-11T07:31:04.579 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0022468919999999999 2024-09-11T07:31:04.579 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.579 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.579 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.579 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.647268+0000", 2024-09-11T07:31:04.579 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0014673659999999999 2024-09-11T07:31:04.579 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.579 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.580 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.580 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.705680+0000", 2024-09-11T07:31:04.580 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.058411905 2024-09-11T07:31:04.580 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.580 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.580 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:31:04.580 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.705727+0000", 2024-09-11T07:31:04.580 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6987999999999998e-05 2024-09-11T07:31:04.580 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.580 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.580 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:31:04.581 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:32.705754+0000", 2024-09-11T07:31:04.581 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.6894e-05 2024-09-11T07:31:04.581 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.581 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:31:04.581 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.581 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.581 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.581 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:70021 7.0 7:06816f25:::benchmark_data_smithi022_77720_object70020:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-11T07:31:04.581 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:29:40.698394+0000", 2024-09-11T07:31:04.581 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 83.811080916999998, 2024-09-11T07:31:04.582 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.053477324, 2024-09-11T07:31:04.582 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:31:04.582 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:31:04.582 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:31:04.582 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:31:04.582 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:31:04.582 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 70021 2024-09-11T07:31:04.582 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.582 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:31:04.582 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.582 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:31:04.582 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:40.698394+0000", 2024-09-11T07:31:04.583 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.583 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.583 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.583 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:31:04.583 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:40.698394+0000", 2024-09-11T07:31:04.583 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.583 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.583 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.583 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:31:04.583 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:40.698392+0000", 2024-09-11T07:31:04.583 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:31:04.583 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.583 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.583 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:31:04.584 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:40.698453+0000", 2024-09-11T07:31:04.584 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1081999999999996e-05 2024-09-11T07:31:04.584 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.584 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.584 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:31:04.584 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:40.698466+0000", 2024-09-11T07:31:04.584 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2604e-05 2024-09-11T07:31:04.584 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.584 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.584 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:31:04.584 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:40.698471+0000", 2024-09-11T07:31:04.585 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8810000000000001e-06 2024-09-11T07:31:04.585 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.585 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.585 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:31:04.585 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:40.698522+0000", 2024-09-11T07:31:04.585 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0346000000000002e-05 2024-09-11T07:31:04.585 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.585 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.585 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:31:04.585 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:40.698649+0000", 2024-09-11T07:31:04.585 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000126952 2024-09-11T07:31:04.586 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.586 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.586 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:31:04.586 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:40.698776+0000", 2024-09-11T07:31:04.586 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012743499999999999 2024-09-11T07:31:04.586 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.586 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.586 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:31:04.586 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:40.700554+0000", 2024-09-11T07:31:04.586 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0017778659999999999 2024-09-11T07:31:04.586 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.586 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.586 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.587 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:40.701925+0000", 2024-09-11T07:31:04.587 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001370614 2024-09-11T07:31:04.587 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.587 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.587 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.587 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:40.751813+0000", 2024-09-11T07:31:04.587 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.049888710000000003 2024-09-11T07:31:04.587 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:04.589 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.589 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.589 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:31:04.589 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:40.751851+0000", 2024-09-11T07:31:04.589 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7478000000000003e-05 2024-09-11T07:31:04.589 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.589 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.589 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:31:04.589 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:40.751872+0000", 2024-09-11T07:31:04.590 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0846e-05 2024-09-11T07:31:04.590 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.590 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:31:04.590 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.590 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.590 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.590 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:72167 7.0 7:02a60d75:::benchmark_data_smithi022_77720_object72166:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-11T07:31:04.590 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:29:42.580489+0000", 2024-09-11T07:31:04.590 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 81.928986308999995, 2024-09-11T07:31:04.590 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.047585573999999999, 2024-09-11T07:31:04.590 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:31:04.590 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:31:04.591 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:31:04.591 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:31:04.591 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:31:04.591 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 72167 2024-09-11T07:31:04.591 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.591 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:31:04.591 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.591 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:31:04.591 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:42.580489+0000", 2024-09-11T07:31:04.591 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.591 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.592 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.592 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:31:04.592 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:42.580489+0000", 2024-09-11T07:31:04.592 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.592 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.592 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.592 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:31:04.592 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:42.580481+0000", 2024-09-11T07:31:04.592 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999919 2024-09-11T07:31:04.592 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.592 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.593 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:31:04.593 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:42.580538+0000", 2024-09-11T07:31:04.593 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6641999999999997e-05 2024-09-11T07:31:04.593 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.593 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.593 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:31:04.593 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:42.580552+0000", 2024-09-11T07:31:04.593 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3901e-05 2024-09-11T07:31:04.593 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.593 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.593 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:31:04.594 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:42.580558+0000", 2024-09-11T07:31:04.594 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2720000000000003e-06 2024-09-11T07:31:04.594 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.594 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.594 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:31:04.594 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:42.580598+0000", 2024-09-11T07:31:04.594 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9904000000000003e-05 2024-09-11T07:31:04.594 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.594 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.594 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:31:04.594 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:42.580685+0000", 2024-09-11T07:31:04.594 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7422000000000004e-05 2024-09-11T07:31:04.595 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.595 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.595 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:31:04.595 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:42.580769+0000", 2024-09-11T07:31:04.595 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4019999999999999e-05 2024-09-11T07:31:04.595 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.595 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.595 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:31:04.595 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:42.582541+0000", 2024-09-11T07:31:04.595 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0017721989999999999 2024-09-11T07:31:04.595 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.596 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.596 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.596 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:42.582866+0000", 2024-09-11T07:31:04.596 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00032479200000000002 2024-09-11T07:31:04.596 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.596 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.596 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.596 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:42.628020+0000", 2024-09-11T07:31:04.596 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.045153894 2024-09-11T07:31:04.596 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.596 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.596 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:31:04.597 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:42.628052+0000", 2024-09-11T07:31:04.597 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1643999999999997e-05 2024-09-11T07:31:04.597 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.597 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.597 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:31:04.597 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:42.628075+0000", 2024-09-11T07:31:04.597 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.2804000000000001e-05 2024-09-11T07:31:04.597 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.597 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:31:04.597 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.597 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.597 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.597 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:89190 7.0 7:060868bf:::benchmark_data_smithi022_77720_object89189:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-11T07:31:04.598 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:29:57.699936+0000", 2024-09-11T07:31:04.598 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 66.809539056999995, 2024-09-11T07:31:04.598 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.069647379999999995, 2024-09-11T07:31:04.598 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:31:04.598 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:31:04.598 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:31:04.598 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:31:04.598 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:31:04.598 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 89190 2024-09-11T07:31:04.598 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.598 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:31:04.598 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.599 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:31:04.599 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:57.699936+0000", 2024-09-11T07:31:04.599 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.599 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.599 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.599 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:31:04.599 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:57.699936+0000", 2024-09-11T07:31:04.599 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.599 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.599 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.599 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:31:04.600 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:57.699934+0000", 2024-09-11T07:31:04.600 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:31:04.600 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.600 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.600 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:31:04.600 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:57.752284+0000", 2024-09-11T07:31:04.600 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.052349875999999997 2024-09-11T07:31:04.600 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.600 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.600 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:31:04.600 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:57.752295+0000", 2024-09-11T07:31:04.600 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0857e-05 2024-09-11T07:31:04.601 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.601 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.601 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:31:04.601 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:57.752300+0000", 2024-09-11T07:31:04.601 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1189999999999996e-06 2024-09-11T07:31:04.601 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.601 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.601 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:31:04.601 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:57.752346+0000", 2024-09-11T07:31:04.601 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6137999999999998e-05 2024-09-11T07:31:04.601 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.602 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.602 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:31:04.602 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:57.752468+0000", 2024-09-11T07:31:04.602 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000121903 2024-09-11T07:31:04.602 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.602 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.602 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:31:04.602 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:57.752607+0000", 2024-09-11T07:31:04.602 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013911300000000001 2024-09-11T07:31:04.602 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.602 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.602 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:31:04.603 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:57.757402+0000", 2024-09-11T07:31:04.603 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0047950029999999999 2024-09-11T07:31:04.603 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.603 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.603 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.603 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:57.767193+0000", 2024-09-11T07:31:04.603 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0097910089999999998 2024-09-11T07:31:04.603 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.603 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.603 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.603 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:57.769482+0000", 2024-09-11T07:31:04.603 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0022886730000000002 2024-09-11T07:31:04.603 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.604 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.604 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:31:04.604 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:57.769542+0000", 2024-09-11T07:31:04.604 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9892000000000002e-05 2024-09-11T07:31:04.604 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.604 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.604 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:31:04.604 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:29:57.769584+0000", 2024-09-11T07:31:04.604 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1638999999999999e-05 2024-09-11T07:31:04.604 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.604 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:31:04.604 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.604 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.605 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.605 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:108760 7.0 7:0b776351:::benchmark_data_smithi022_77720_object108759:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-11T07:31:04.605 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:30:16.727258+0000", 2024-09-11T07:31:04.605 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 47.782217142, 2024-09-11T07:31:04.605 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.046370045999999998, 2024-09-11T07:31:04.605 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:31:04.605 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:31:04.605 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:31:04.605 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:31:04.605 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:31:04.605 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 108760 2024-09-11T07:31:04.605 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.606 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:31:04.606 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.606 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:31:04.606 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:16.727258+0000", 2024-09-11T07:31:04.606 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.606 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.606 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.606 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:31:04.606 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:16.727258+0000", 2024-09-11T07:31:04.606 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.606 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.606 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.607 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:31:04.607 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:16.727256+0000", 2024-09-11T07:31:04.607 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:31:04.607 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.607 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.607 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:31:04.607 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:16.727298+0000", 2024-09-11T07:31:04.607 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2391000000000001e-05 2024-09-11T07:31:04.607 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.607 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.607 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:31:04.608 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:16.727313+0000", 2024-09-11T07:31:04.608 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5007e-05 2024-09-11T07:31:04.608 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.608 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.608 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:31:04.608 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:16.727320+0000", 2024-09-11T07:31:04.608 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4170000000000003e-06 2024-09-11T07:31:04.608 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.608 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.608 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:31:04.608 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:16.727366+0000", 2024-09-11T07:31:04.608 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5621000000000002e-05 2024-09-11T07:31:04.609 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.609 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.609 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:31:04.609 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:16.727463+0000", 2024-09-11T07:31:04.609 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7413999999999999e-05 2024-09-11T07:31:04.609 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.609 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.609 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:31:04.609 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:16.727620+0000", 2024-09-11T07:31:04.609 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015647600000000001 2024-09-11T07:31:04.609 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.610 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.610 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:31:04.610 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:16.729788+0000", 2024-09-11T07:31:04.610 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002167675 2024-09-11T07:31:04.610 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.610 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.610 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.610 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:16.750890+0000", 2024-09-11T07:31:04.610 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.021102045999999999 2024-09-11T07:31:04.610 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.610 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.611 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.611 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:16.773539+0000", 2024-09-11T07:31:04.611 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.022649314 2024-09-11T07:31:04.611 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.611 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.611 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:31:04.611 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:16.773590+0000", 2024-09-11T07:31:04.611 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0924999999999998e-05 2024-09-11T07:31:04.611 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.611 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.611 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:31:04.611 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:16.773628+0000", 2024-09-11T07:31:04.612 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.824e-05 2024-09-11T07:31:04.612 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.612 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:31:04.612 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.612 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.612 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.612 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:109203 7.0 7:02a36061:::benchmark_data_smithi022_77720_object109202:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-11T07:31:04.612 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:30:17.195556+0000", 2024-09-11T07:31:04.612 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 47.313919675000001, 2024-09-11T07:31:04.612 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.063577474999999994, 2024-09-11T07:31:04.612 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:31:04.613 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:31:04.613 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:31:04.613 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:31:04.613 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:31:04.613 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 109203 2024-09-11T07:31:04.613 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.613 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:31:04.613 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.613 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:31:04.613 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.195556+0000", 2024-09-11T07:31:04.613 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.614 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.614 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.614 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:31:04.614 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.195556+0000", 2024-09-11T07:31:04.614 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.614 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.614 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.614 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:31:04.614 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.195553+0000", 2024-09-11T07:31:04.614 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:31:04.614 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.614 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.615 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:31:04.615 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.195699+0000", 2024-09-11T07:31:04.615 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014614900000000001 2024-09-11T07:31:04.615 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.615 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.615 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:31:04.615 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.195730+0000", 2024-09-11T07:31:04.615 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1065999999999997e-05 2024-09-11T07:31:04.615 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.615 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.615 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:31:04.615 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.195738+0000", 2024-09-11T07:31:04.615 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3679999999999996e-06 2024-09-11T07:31:04.616 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.616 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.616 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:31:04.616 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.195784+0000", 2024-09-11T07:31:04.616 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6394999999999999e-05 2024-09-11T07:31:04.616 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.616 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.616 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:31:04.616 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.195883+0000", 2024-09-11T07:31:04.616 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.9086999999999995e-05 2024-09-11T07:31:04.616 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.616 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.616 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:31:04.617 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.196055+0000", 2024-09-11T07:31:04.617 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000171915 2024-09-11T07:31:04.617 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.617 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.617 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:31:04.617 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.198879+0000", 2024-09-11T07:31:04.617 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0028234419999999998 2024-09-11T07:31:04.617 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.617 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.617 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.617 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.203733+0000", 2024-09-11T07:31:04.617 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0048547640000000001 2024-09-11T07:31:04.618 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.618 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.618 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.618 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.259066+0000", 2024-09-11T07:31:04.618 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.055332915000000003 2024-09-11T07:31:04.618 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.618 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.618 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:31:04.618 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.259099+0000", 2024-09-11T07:31:04.618 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2471999999999997e-05 2024-09-11T07:31:04.618 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.619 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.619 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:31:04.619 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.259133+0000", 2024-09-11T07:31:04.619 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4363000000000003e-05 2024-09-11T07:31:04.619 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.619 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:31:04.619 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.619 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.619 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.619 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:118447 7.0 7:0bc48b04:::benchmark_data_smithi022_77720_object118446:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-11T07:31:04.619 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:30:26.581081+0000", 2024-09-11T07:31:04.619 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 37.928393982000003, 2024-09-11T07:31:04.620 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.045989546999999999, 2024-09-11T07:31:04.620 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:31:04.620 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:31:04.620 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:31:04.620 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:31:04.620 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:31:04.620 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 118447 2024-09-11T07:31:04.620 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.620 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:31:04.620 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.620 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:31:04.621 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:26.581081+0000", 2024-09-11T07:31:04.621 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.621 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.621 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.621 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:31:04.621 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:26.581081+0000", 2024-09-11T07:31:04.621 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.621 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.621 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.621 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:31:04.622 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:26.581081+0000", 2024-09-11T07:31:04.622 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:31:04.622 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.622 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.622 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:31:04.622 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:26.581106+0000", 2024-09-11T07:31:04.622 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.5554999999999998e-05 2024-09-11T07:31:04.622 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.622 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.622 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:31:04.622 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:26.581112+0000", 2024-09-11T07:31:04.622 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3229999999999997e-06 2024-09-11T07:31:04.623 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.623 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.623 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:31:04.623 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:26.581114+0000", 2024-09-11T07:31:04.623 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.7429999999999998e-06 2024-09-11T07:31:04.623 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.623 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.623 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:31:04.623 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:26.583434+0000", 2024-09-11T07:31:04.623 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0023196359999999999 2024-09-11T07:31:04.623 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.624 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.624 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:31:04.624 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:26.583514+0000", 2024-09-11T07:31:04.624 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9868000000000002e-05 2024-09-11T07:31:04.624 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.624 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.624 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:31:04.624 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:26.583601+0000", 2024-09-11T07:31:04.624 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7496000000000004e-05 2024-09-11T07:31:04.624 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.624 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.625 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:31:04.625 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:26.585129+0000", 2024-09-11T07:31:04.625 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001527163 2024-09-11T07:31:04.625 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.625 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.625 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.625 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:26.588774+0000", 2024-09-11T07:31:04.625 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0036454349999999998 2024-09-11T07:31:04.625 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.625 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.625 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.625 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:26.627006+0000", 2024-09-11T07:31:04.626 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.038232150999999999 2024-09-11T07:31:04.626 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.626 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.626 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:31:04.626 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:26.627035+0000", 2024-09-11T07:31:04.626 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.8428999999999998e-05 2024-09-11T07:31:04.626 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.626 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.626 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:31:04.626 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:26.627071+0000", 2024-09-11T07:31:04.626 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6238000000000001e-05 2024-09-11T07:31:04.626 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.627 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:31:04.627 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.627 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.627 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.627 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:119878 7.0 7:0d5a1409:::benchmark_data_smithi022_77720_object119877:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-11T07:31:04.627 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:30:28.063301+0000", 2024-09-11T07:31:04.627 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 36.44617444, 2024-09-11T07:31:04.627 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.12935533599999999, 2024-09-11T07:31:04.627 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:31:04.627 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:31:04.627 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:31:04.628 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:31:04.628 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:31:04.628 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 119878 2024-09-11T07:31:04.628 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.628 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:31:04.628 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.628 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:31:04.628 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063301+0000", 2024-09-11T07:31:04.628 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.628 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.629 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.629 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:31:04.629 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063301+0000", 2024-09-11T07:31:04.629 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.629 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.629 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.629 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:31:04.629 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063298+0000", 2024-09-11T07:31:04.629 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:31:04.629 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.629 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.629 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:31:04.630 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063402+0000", 2024-09-11T07:31:04.630 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010310999999999999 2024-09-11T07:31:04.630 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.630 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.630 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:31:04.630 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063414+0000", 2024-09-11T07:31:04.630 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2237e-05 2024-09-11T07:31:04.630 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.630 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.630 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:31:04.630 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063419+0000", 2024-09-11T07:31:04.630 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5489999999999997e-06 2024-09-11T07:31:04.631 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.631 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.631 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:31:04.631 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063454+0000", 2024-09-11T07:31:04.631 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5132000000000002e-05 2024-09-11T07:31:04.631 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.631 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.631 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:31:04.631 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063565+0000", 2024-09-11T07:31:04.631 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000110246 2024-09-11T07:31:04.631 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.632 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.632 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:31:04.632 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063651+0000", 2024-09-11T07:31:04.632 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6661000000000003e-05 2024-09-11T07:31:04.632 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.632 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.632 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:31:04.632 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.066483+0000", 2024-09-11T07:31:04.632 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0028320490000000001 2024-09-11T07:31:04.632 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.633 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.633 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.633 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.066815+0000", 2024-09-11T07:31:04.633 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00033145700000000001 2024-09-11T07:31:04.633 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.633 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.633 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.633 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.192594+0000", 2024-09-11T07:31:04.633 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.12577914400000001 2024-09-11T07:31:04.633 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.633 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.633 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:31:04.634 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.192633+0000", 2024-09-11T07:31:04.634 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9042999999999999e-05 2024-09-11T07:31:04.634 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.634 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.634 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:31:04.634 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.192656+0000", 2024-09-11T07:31:04.634 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3125999999999998e-05 2024-09-11T07:31:04.634 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.634 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:31:04.634 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.634 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.635 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.635 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:120961 7.0 7:03c80417:::benchmark_data_smithi022_77720_object120960:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:31:04.635 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:30:29.524767+0000", 2024-09-11T07:31:04.635 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 34.984708642999998, 2024-09-11T07:31:04.635 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.052123584000000001, 2024-09-11T07:31:04.635 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:31:04.635 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:31:04.635 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:31:04.635 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:31:04.635 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:31:04.636 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 120961 2024-09-11T07:31:04.636 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.636 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:31:04.636 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.636 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:31:04.636 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.524767+0000", 2024-09-11T07:31:04.636 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.636 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.636 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.636 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:31:04.637 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.524767+0000", 2024-09-11T07:31:04.637 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.637 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.637 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.637 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:31:04.637 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.524757+0000", 2024-09-11T07:31:04.637 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999905 2024-09-11T07:31:04.637 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.637 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.637 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:31:04.637 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.524799+0000", 2024-09-11T07:31:04.637 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1718000000000003e-05 2024-09-11T07:31:04.638 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.638 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.638 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:31:04.638 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.524812+0000", 2024-09-11T07:31:04.638 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3159000000000001e-05 2024-09-11T07:31:04.638 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.638 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.638 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:31:04.638 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.524818+0000", 2024-09-11T07:31:04.638 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8109999999999999e-06 2024-09-11T07:31:04.638 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.638 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.639 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:31:04.639 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.524872+0000", 2024-09-11T07:31:04.639 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4419999999999997e-05 2024-09-11T07:31:04.639 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.639 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.639 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:31:04.639 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.525003+0000", 2024-09-11T07:31:04.639 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013052800000000001 2024-09-11T07:31:04.639 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.639 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.639 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:31:04.639 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.525150+0000", 2024-09-11T07:31:04.639 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000147143 2024-09-11T07:31:04.640 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.640 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.640 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:31:04.640 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.528076+0000", 2024-09-11T07:31:04.640 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002925633 2024-09-11T07:31:04.640 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.640 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.640 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.640 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.528706+0000", 2024-09-11T07:31:04.640 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00063073700000000003 2024-09-11T07:31:04.640 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.640 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.641 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.641 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.576784+0000", 2024-09-11T07:31:04.641 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.048077519999999999 2024-09-11T07:31:04.641 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.641 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.641 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:31:04.641 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.576846+0000", 2024-09-11T07:31:04.641 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2147e-05 2024-09-11T07:31:04.641 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.642 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.642 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:31:04.642 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.576890+0000", 2024-09-11T07:31:04.642 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4121e-05 2024-09-11T07:31:04.642 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.642 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:31:04.642 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.642 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.642 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.642 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:121102 7.0 7:09266ef0:::benchmark_data_smithi022_77720_object121101:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:31:04.642 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:30:29.710752+0000", 2024-09-11T07:31:04.642 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 34.798723295000002, 2024-09-11T07:31:04.643 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.121706142, 2024-09-11T07:31:04.643 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:31:04.643 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:31:04.643 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:31:04.643 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:31:04.643 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:31:04.643 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 121102 2024-09-11T07:31:04.643 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.643 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:31:04.643 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.643 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:31:04.643 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.710752+0000", 2024-09-11T07:31:04.644 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.644 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.644 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.644 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:31:04.644 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.710752+0000", 2024-09-11T07:31:04.644 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.644 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.644 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.644 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:31:04.644 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.710749+0000", 2024-09-11T07:31:04.644 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:31:04.645 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.645 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.645 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:31:04.645 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.710809+0000", 2024-09-11T07:31:04.645 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9255000000000001e-05 2024-09-11T07:31:04.645 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.645 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.645 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:31:04.645 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.710832+0000", 2024-09-11T07:31:04.645 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3031e-05 2024-09-11T07:31:04.645 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.645 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.646 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:31:04.646 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.710839+0000", 2024-09-11T07:31:04.646 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3889999999999999e-06 2024-09-11T07:31:04.646 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.646 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.646 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:31:04.646 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.710883+0000", 2024-09-11T07:31:04.646 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3770999999999998e-05 2024-09-11T07:31:04.646 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.646 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.646 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:31:04.646 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.711002+0000", 2024-09-11T07:31:04.646 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000119518 2024-09-11T07:31:04.647 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.647 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.647 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:31:04.647 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.711141+0000", 2024-09-11T07:31:04.647 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013841 2024-09-11T07:31:04.647 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.647 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.647 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:31:04.647 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.713588+0000", 2024-09-11T07:31:04.647 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0024468179999999999 2024-09-11T07:31:04.647 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.648 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.648 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.648 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.713875+0000", 2024-09-11T07:31:04.648 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00028783500000000001 2024-09-11T07:31:04.648 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.648 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.648 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.648 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.832376+0000", 2024-09-11T07:31:04.648 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.11850076800000001 2024-09-11T07:31:04.648 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.648 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.648 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:31:04.649 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.832424+0000", 2024-09-11T07:31:04.649 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8047999999999998e-05 2024-09-11T07:31:04.649 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.649 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.649 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:31:04.649 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.832458+0000", 2024-09-11T07:31:04.649 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3849000000000001e-05 2024-09-11T07:31:04.649 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.649 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:31:04.649 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.649 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.649 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.650 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:122115 7.0 7:05084030:::benchmark_data_smithi022_77720_object122114:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:31:04.650 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:30:30.870658+0000", 2024-09-11T07:31:04.650 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 33.638817615000001, 2024-09-11T07:31:04.650 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.041707514000000001, 2024-09-11T07:31:04.650 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:31:04.650 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:31:04.650 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:31:04.650 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:31:04.650 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:31:04.650 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 122115 2024-09-11T07:31:04.650 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.650 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:31:04.651 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.651 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:31:04.651 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:30.870658+0000", 2024-09-11T07:31:04.651 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.651 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.651 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.651 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:31:04.651 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:30.870658+0000", 2024-09-11T07:31:04.651 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:31:04.651 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.651 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.652 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:31:04.652 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:30.870655+0000", 2024-09-11T07:31:04.652 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999971 2024-09-11T07:31:04.652 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.652 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.652 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:31:04.652 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:30.870697+0000", 2024-09-11T07:31:04.652 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2435000000000003e-05 2024-09-11T07:31:04.652 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.652 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.652 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:31:04.652 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:30.870717+0000", 2024-09-11T07:31:04.653 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9391000000000001e-05 2024-09-11T07:31:04.653 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.653 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.653 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:31:04.653 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:30.870724+0000", 2024-09-11T07:31:04.653 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.2540000000000003e-06 2024-09-11T07:31:04.653 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.653 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.653 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:31:04.653 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:30.870774+0000", 2024-09-11T07:31:04.653 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9534999999999999e-05 2024-09-11T07:31:04.653 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.654 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.654 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:31:04.654 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:30.870907+0000", 2024-09-11T07:31:04.654 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013376300000000001 2024-09-11T07:31:04.654 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.654 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.654 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:31:04.654 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:30.871043+0000", 2024-09-11T07:31:04.654 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000136107 2024-09-11T07:31:04.654 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.654 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.654 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.654 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:30.873689+0000", 2024-09-11T07:31:04.655 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00264511 2024-09-11T07:31:04.655 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.655 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.655 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:31:04.655 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:30.873801+0000", 2024-09-11T07:31:04.655 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000112074 2024-09-11T07:31:04.655 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.655 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.655 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:31:04.655 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:30.912277+0000", 2024-09-11T07:31:04.655 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.038476726000000003 2024-09-11T07:31:04.656 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.656 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.656 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:31:04.656 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:30.912328+0000", 2024-09-11T07:31:04.656 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0133000000000003e-05 2024-09-11T07:31:04.656 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:31:04.656 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:31:04.656 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:31:04.656 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:30.912365+0000", 2024-09-11T07:31:04.656 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7642999999999999e-05 2024-09-11T07:31:04.656 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.657 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:31:04.657 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.657 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:31:04.657 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:31:04.657 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:31:04.710 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:04.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:04.711 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:04.812 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:04.853 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:04.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:04.854 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:04.955 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:05.000 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:05.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:05.001 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:05.102 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:05.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:05.136 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:05.136 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:05.212 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is unset 2024-09-11T07:31:05.237 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:05.260 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:05.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:05.261 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:05.362 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:05.400 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:05.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:05.401 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:05.502 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:05.529 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:05.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:05.530 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:05.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:05 smithi104 ceph-mon[33507]: 5.13 scrub starts 2024-09-11T07:31:05.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:05 smithi104 ceph-mon[33507]: 5.13 scrub ok 2024-09-11T07:31:05.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:05 smithi104 ceph-mon[33507]: 5.a scrub starts 2024-09-11T07:31:05.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:05 smithi104 ceph-mon[33507]: 5.a scrub ok 2024-09-11T07:31:05.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:05 smithi104 ceph-mon[33507]: pgmap v184: 148 pgs: 8 active+undersized, 13 stale+active+clean, 3 active+clean+scrubbing, 5 active+undersized+degraded, 119 active+clean; 5.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 952 op/s; 15/285483 objects degraded (0.005%) 2024-09-11T07:31:05.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:05 smithi104 ceph-mon[33507]: 7.0 scrub starts 2024-09-11T07:31:05.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:05 smithi104 ceph-mon[33507]: Health check failed: Degraded data redundancy: 15/285483 objects degraded (0.005%), 5 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:05.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:05 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1731740661' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:31:05.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:05 smithi104 ceph-mon[33507]: from='client.24902 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:31:05.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:05 smithi104 ceph-mon[33507]: 4.12 scrub starts 2024-09-11T07:31:05.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:05 smithi104 ceph-mon[33507]: 4.12 scrub ok 2024-09-11T07:31:05.630 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:05.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:05 smithi022 ceph-mon[29462]: 5.13 scrub starts 2024-09-11T07:31:05.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:05 smithi022 ceph-mon[29462]: 5.13 scrub ok 2024-09-11T07:31:05.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:05 smithi022 ceph-mon[29462]: 5.a scrub starts 2024-09-11T07:31:05.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:05 smithi022 ceph-mon[29462]: 5.a scrub ok 2024-09-11T07:31:05.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:05 smithi022 ceph-mon[29462]: pgmap v184: 148 pgs: 8 active+undersized, 13 stale+active+clean, 3 active+clean+scrubbing, 5 active+undersized+degraded, 119 active+clean; 5.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 952 op/s; 15/285483 objects degraded (0.005%) 2024-09-11T07:31:05.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:05 smithi022 ceph-mon[29462]: 7.0 scrub starts 2024-09-11T07:31:05.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:05 smithi022 ceph-mon[29462]: Health check failed: Degraded data redundancy: 15/285483 objects degraded (0.005%), 5 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:05.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:05 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1731740661' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:31:05.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:05 smithi022 ceph-mon[29462]: from='client.24902 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:31:05.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:05 smithi022 ceph-mon[29462]: 4.12 scrub starts 2024-09-11T07:31:05.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:05 smithi022 ceph-mon[29462]: 4.12 scrub ok 2024-09-11T07:31:05.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:05 smithi022 ceph-mon[37330]: 5.13 scrub starts 2024-09-11T07:31:05.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:05 smithi022 ceph-mon[37330]: 5.13 scrub ok 2024-09-11T07:31:05.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:05 smithi022 ceph-mon[37330]: 5.a scrub starts 2024-09-11T07:31:05.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:05 smithi022 ceph-mon[37330]: 5.a scrub ok 2024-09-11T07:31:05.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:05 smithi022 ceph-mon[37330]: pgmap v184: 148 pgs: 8 active+undersized, 13 stale+active+clean, 3 active+clean+scrubbing, 5 active+undersized+degraded, 119 active+clean; 5.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 952 op/s; 15/285483 objects degraded (0.005%) 2024-09-11T07:31:05.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:05 smithi022 ceph-mon[37330]: 7.0 scrub starts 2024-09-11T07:31:05.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:05 smithi022 ceph-mon[37330]: Health check failed: Degraded data redundancy: 15/285483 objects degraded (0.005%), 5 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:05.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:05 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1731740661' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:31:05.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:05 smithi022 ceph-mon[37330]: from='client.24902 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:31:05.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:05 smithi022 ceph-mon[37330]: 4.12 scrub starts 2024-09-11T07:31:05.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:05 smithi022 ceph-mon[37330]: 4.12 scrub ok 2024-09-11T07:31:05.676 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:05.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:05.678 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:05.778 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:05.819 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:05.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:05.820 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:05.921 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:05.956 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:05.957 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-11T07:31:05.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:05.958 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:06.059 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:06.098 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:06.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:06.100 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:06.201 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:06.223 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:06.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:06.224 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:06.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[29462]: 5.8 scrub starts 2024-09-11T07:31:06.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[29462]: 5.8 scrub ok 2024-09-11T07:31:06.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[29462]: 6.1e scrub starts 2024-09-11T07:31:06.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[29462]: 6.1e scrub ok 2024-09-11T07:31:06.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[29462]: 4.6 scrub starts 2024-09-11T07:31:06.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[29462]: 4.6 scrub ok 2024-09-11T07:31:06.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[29462]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:31:06.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[29462]: from='client.24902 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:31:06.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[29462]: osdmap e136: 8 total, 7 up, 8 in 2024-09-11T07:31:06.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[29462]: 7.9 scrub ok 2024-09-11T07:31:06.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[29462]: 4.1f scrub starts 2024-09-11T07:31:06.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[29462]: 4.1f scrub ok 2024-09-11T07:31:06.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[29462]: 6.4 scrub starts 2024-09-11T07:31:06.230 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[29462]: 6.4 scrub ok 2024-09-11T07:31:06.230 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[37330]: 5.8 scrub starts 2024-09-11T07:31:06.230 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[37330]: 5.8 scrub ok 2024-09-11T07:31:06.230 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[37330]: 6.1e scrub starts 2024-09-11T07:31:06.230 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[37330]: 6.1e scrub ok 2024-09-11T07:31:06.230 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[37330]: 4.6 scrub starts 2024-09-11T07:31:06.230 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[37330]: 4.6 scrub ok 2024-09-11T07:31:06.230 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[37330]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:31:06.230 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[37330]: from='client.24902 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:31:06.230 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[37330]: osdmap e136: 8 total, 7 up, 8 in 2024-09-11T07:31:06.231 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[37330]: 7.9 scrub ok 2024-09-11T07:31:06.231 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[37330]: 4.1f scrub starts 2024-09-11T07:31:06.231 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[37330]: 4.1f scrub ok 2024-09-11T07:31:06.231 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[37330]: 6.4 scrub starts 2024-09-11T07:31:06.231 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:06 smithi022 ceph-mon[37330]: 6.4 scrub ok 2024-09-11T07:31:06.326 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:06.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:06.366 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:06.366 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:06.467 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:06.492 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:06.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:06.493 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:06.529 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:06 smithi104 podman[76296]: 2024-09-11 07:31:06.126502 +0000 UTC m=+0.020072970 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:31:06.530 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:06 smithi104 podman[76296]: 2024-09-11 07:31:06.241678915 +0000 UTC m=+0.135249863 container create 43c94b7e5b1c4a9243369a302edce1708f671d1d742b096bded674f3ac0e00d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T07:31:06.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:06 smithi104 ceph-mon[33507]: 5.8 scrub starts 2024-09-11T07:31:06.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:06 smithi104 ceph-mon[33507]: 5.8 scrub ok 2024-09-11T07:31:06.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:06 smithi104 ceph-mon[33507]: 6.1e scrub starts 2024-09-11T07:31:06.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:06 smithi104 ceph-mon[33507]: 6.1e scrub ok 2024-09-11T07:31:06.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:06 smithi104 ceph-mon[33507]: 4.6 scrub starts 2024-09-11T07:31:06.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:06 smithi104 ceph-mon[33507]: 4.6 scrub ok 2024-09-11T07:31:06.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:06 smithi104 ceph-mon[33507]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:31:06.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:06 smithi104 ceph-mon[33507]: from='client.24902 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:31:06.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:06 smithi104 ceph-mon[33507]: osdmap e136: 8 total, 7 up, 8 in 2024-09-11T07:31:06.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:06 smithi104 ceph-mon[33507]: 7.9 scrub ok 2024-09-11T07:31:06.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:06 smithi104 ceph-mon[33507]: 4.1f scrub starts 2024-09-11T07:31:06.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:06 smithi104 ceph-mon[33507]: 4.1f scrub ok 2024-09-11T07:31:06.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:06 smithi104 ceph-mon[33507]: 6.4 scrub starts 2024-09-11T07:31:06.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:06 smithi104 ceph-mon[33507]: 6.4 scrub ok 2024-09-11T07:31:06.593 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:06.637 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:06.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:06.638 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:06.739 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:06.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:06.765 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:06.765 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:06.866 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:06.906 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:06.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:06.907 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:07.007 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:07.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:06 smithi104 podman[76296]: 2024-09-11 07:31:06.644854519 +0000 UTC m=+0.538425477 container init 43c94b7e5b1c4a9243369a302edce1708f671d1d742b096bded674f3ac0e00d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-11T07:31:07.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:06 smithi104 podman[76296]: 2024-09-11 07:31:06.651100658 +0000 UTC m=+0.544671605 container start 43c94b7e5b1c4a9243369a302edce1708f671d1d742b096bded674f3ac0e00d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-11T07:31:07.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:06 smithi104 podman[76296]: 2024-09-11 07:31:06.75357463 +0000 UTC m=+0.647145583 container attach 43c94b7e5b1c4a9243369a302edce1708f671d1d742b096bded674f3ac0e00d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-11T07:31:07.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:06 smithi104 podman[76296]: 2024-09-11 07:31:06.868452993 +0000 UTC m=+0.762023943 container died 43c94b7e5b1c4a9243369a302edce1708f671d1d742b096bded674f3ac0e00d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:31:07.045 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:07.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:07.046 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:07.107 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:31:07.146 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:07.187 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:07.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:07.188 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:07.289 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:07.326 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:07.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:07.327 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:07.428 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:07.468 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:07.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:07.469 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:07.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:07 smithi104 ceph-mon[33507]: 6.13 scrub starts 2024-09-11T07:31:07.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:07 smithi104 ceph-mon[33507]: 6.13 scrub ok 2024-09-11T07:31:07.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:07 smithi104 ceph-mon[33507]: 4.1c scrub starts 2024-09-11T07:31:07.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:07 smithi104 ceph-mon[33507]: 4.1c scrub ok 2024-09-11T07:31:07.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:07 smithi104 ceph-mon[33507]: pgmap v186: 148 pgs: 35 active+undersized, 1 active+clean+scrubbing, 26 active+undersized+degraded, 86 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 670 op/s; 28601/274842 objects degraded (10.406%); 0 B/s, 0 objects/s recovering 2024-09-11T07:31:07.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:07 smithi104 ceph-mon[33507]: 7.0 scrub ok 2024-09-11T07:31:07.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:07 smithi104 ceph-mon[33507]: 4.13 deep-scrub starts 2024-09-11T07:31:07.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:07 smithi104 ceph-mon[33507]: 4.13 deep-scrub ok 2024-09-11T07:31:07.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:07 smithi104 ceph-mon[33507]: 5.16 scrub starts 2024-09-11T07:31:07.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:07 smithi104 ceph-mon[33507]: 5.16 scrub ok 2024-09-11T07:31:07.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:07 smithi104 ceph-mon[33507]: 2.14 scrub starts 2024-09-11T07:31:07.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:07 smithi104 ceph-mon[33507]: 2.14 scrub ok 2024-09-11T07:31:07.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:07 smithi104 ceph-mon[33507]: 4.2 scrub starts 2024-09-11T07:31:07.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:07 smithi104 ceph-mon[33507]: 4.2 scrub ok 2024-09-11T07:31:07.570 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:07.591 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:07.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:07.592 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:07.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[29462]: 6.13 scrub starts 2024-09-11T07:31:07.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[29462]: 6.13 scrub ok 2024-09-11T07:31:07.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[29462]: 4.1c scrub starts 2024-09-11T07:31:07.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[29462]: 4.1c scrub ok 2024-09-11T07:31:07.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[29462]: pgmap v186: 148 pgs: 35 active+undersized, 1 active+clean+scrubbing, 26 active+undersized+degraded, 86 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 670 op/s; 28601/274842 objects degraded (10.406%); 0 B/s, 0 objects/s recovering 2024-09-11T07:31:07.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[29462]: 7.0 scrub ok 2024-09-11T07:31:07.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[29462]: 4.13 deep-scrub starts 2024-09-11T07:31:07.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[29462]: 4.13 deep-scrub ok 2024-09-11T07:31:07.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[29462]: 5.16 scrub starts 2024-09-11T07:31:07.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[29462]: 5.16 scrub ok 2024-09-11T07:31:07.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[29462]: 2.14 scrub starts 2024-09-11T07:31:07.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[29462]: 2.14 scrub ok 2024-09-11T07:31:07.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[29462]: 4.2 scrub starts 2024-09-11T07:31:07.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[29462]: 4.2 scrub ok 2024-09-11T07:31:07.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[37330]: 6.13 scrub starts 2024-09-11T07:31:07.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[37330]: 6.13 scrub ok 2024-09-11T07:31:07.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[37330]: 4.1c scrub starts 2024-09-11T07:31:07.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[37330]: 4.1c scrub ok 2024-09-11T07:31:07.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[37330]: pgmap v186: 148 pgs: 35 active+undersized, 1 active+clean+scrubbing, 26 active+undersized+degraded, 86 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 670 op/s; 28601/274842 objects degraded (10.406%); 0 B/s, 0 objects/s recovering 2024-09-11T07:31:07.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[37330]: 7.0 scrub ok 2024-09-11T07:31:07.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[37330]: 4.13 deep-scrub starts 2024-09-11T07:31:07.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[37330]: 4.13 deep-scrub ok 2024-09-11T07:31:07.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[37330]: 5.16 scrub starts 2024-09-11T07:31:07.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[37330]: 5.16 scrub ok 2024-09-11T07:31:07.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[37330]: 2.14 scrub starts 2024-09-11T07:31:07.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[37330]: 2.14 scrub ok 2024-09-11T07:31:07.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[37330]: 4.2 scrub starts 2024-09-11T07:31:07.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:07 smithi022 ceph-mon[37330]: 4.2 scrub ok 2024-09-11T07:31:07.693 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:07.736 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:07.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:07.737 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:07.838 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:07.863 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:07.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:07.864 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:07.965 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:08.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:08.020 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:08.021 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:08.039 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.7.service' 2024-09-11T07:31:08.121 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:08.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:08.179 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:08.179 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:08.187 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:31:08.187 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-11T07:31:08.187 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.7 2024-09-11T07:31:08.187 INFO:tasks.thrashosds.thrasher:remote for osd 7 is ubuntu@smithi104.front.sepia.ceph.com 2024-09-11T07:31:08.188 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -n osd.7 --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal --op list-pgs 2024-09-11T07:31:08.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:08 smithi104 ceph-mon[33507]: 2.1 scrub starts 2024-09-11T07:31:08.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:08 smithi104 ceph-mon[33507]: 2.1 scrub ok 2024-09-11T07:31:08.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:08 smithi104 ceph-mon[33507]: 4.d scrub starts 2024-09-11T07:31:08.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:08 smithi104 ceph-mon[33507]: 4.d scrub ok 2024-09-11T07:31:08.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:31:08.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:31:08.280 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:08.344 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:08.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:08.345 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:08.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:08 smithi022 ceph-mon[29462]: 2.1 scrub starts 2024-09-11T07:31:08.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:08 smithi022 ceph-mon[29462]: 2.1 scrub ok 2024-09-11T07:31:08.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:08 smithi022 ceph-mon[29462]: 4.d scrub starts 2024-09-11T07:31:08.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:08 smithi022 ceph-mon[29462]: 4.d scrub ok 2024-09-11T07:31:08.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:31:08.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:31:08.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:08 smithi022 ceph-mon[37330]: 2.1 scrub starts 2024-09-11T07:31:08.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:08 smithi022 ceph-mon[37330]: 2.1 scrub ok 2024-09-11T07:31:08.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:08 smithi022 ceph-mon[37330]: 4.d scrub starts 2024-09-11T07:31:08.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:08 smithi022 ceph-mon[37330]: 4.d scrub ok 2024-09-11T07:31:08.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:31:08.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:31:08.447 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:08.471 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:08.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:08.472 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:08.573 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:08.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:08.597 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:08.598 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:08.692 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:31:08.692 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:31:08.698 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:08.701 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T07:31:08.711 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:31:08.711 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:31:08.737 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:08.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:08.738 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:08.752 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:31:08.752 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:31:08.788 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:31:08.788 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:31:08.823 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:31:08.823 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:31:08.839 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:08.857 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/osd.7/config 2024-09-11T07:31:08.860 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:31:08.860 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:31:08.880 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:08.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:08.881 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:08.893 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:31:08.893 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:31:08.896 INFO:teuthology.orchestra.run.smithi022.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-11T07:31:08.982 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:09.024 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:09.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:09.025 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:09.126 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:09.149 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:09.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:09.150 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:09.250 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:09.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:09.294 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:09.294 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:09.395 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:09.437 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:09.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:09.439 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:09.510 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:31:09 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:31:09.510 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:09 smithi104 ceph-mon[33507]: pgmap v187: 148 pgs: 35 active+undersized, 1 active+clean+scrubbing, 26 active+undersized+degraded, 86 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 851 B/s rd, 0 B/s wr, 669 op/s; 28601/274842 objects degraded (10.406%); 0 B/s, 0 objects/s recovering 2024-09-11T07:31:09.511 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:09 smithi104 ceph-mon[33507]: 4.1a deep-scrub starts 2024-09-11T07:31:09.511 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:09 smithi104 ceph-mon[33507]: 4.1a deep-scrub ok 2024-09-11T07:31:09.511 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:09 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/385978945' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:31:09.515 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:09 smithi022 ceph-mon[29462]: pgmap v187: 148 pgs: 35 active+undersized, 1 active+clean+scrubbing, 26 active+undersized+degraded, 86 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 851 B/s rd, 0 B/s wr, 669 op/s; 28601/274842 objects degraded (10.406%); 0 B/s, 0 objects/s recovering 2024-09-11T07:31:09.515 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:09 smithi022 ceph-mon[29462]: 4.1a deep-scrub starts 2024-09-11T07:31:09.515 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:09 smithi022 ceph-mon[29462]: 4.1a deep-scrub ok 2024-09-11T07:31:09.515 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:09 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/385978945' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:31:09.515 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:09 smithi022 ceph-mon[37330]: pgmap v187: 148 pgs: 35 active+undersized, 1 active+clean+scrubbing, 26 active+undersized+degraded, 86 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 851 B/s rd, 0 B/s wr, 669 op/s; 28601/274842 objects degraded (10.406%); 0 B/s, 0 objects/s recovering 2024-09-11T07:31:09.515 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:09 smithi022 ceph-mon[37330]: 4.1a deep-scrub starts 2024-09-11T07:31:09.515 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:09 smithi022 ceph-mon[37330]: 4.1a deep-scrub ok 2024-09-11T07:31:09.516 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:09 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/385978945' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:31:09.539 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:09.559 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T07:31:09.559 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T07:31:09.590 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:09.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:09.591 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:09.669 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:31:09.692 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:09.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:09.744 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:09.744 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:09.845 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:09.890 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:09.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:09.891 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:09.992 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:10.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:10.031 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:10.031 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:10.132 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:10.154 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:10.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:10.155 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:10.256 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:10.274 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:10 smithi104 ceph-mon[33507]: 5.f scrub starts 2024-09-11T07:31:10.274 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:10 smithi104 ceph-mon[33507]: 5.f scrub ok 2024-09-11T07:31:10.274 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:10 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:10.293 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:10.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:10.295 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:10.374 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:10 smithi022 ceph-mon[29462]: 5.f scrub starts 2024-09-11T07:31:10.374 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:10 smithi022 ceph-mon[29462]: 5.f scrub ok 2024-09-11T07:31:10.374 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:10 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:10.375 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:10 smithi022 ceph-mon[37330]: 5.f scrub starts 2024-09-11T07:31:10.375 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:10 smithi022 ceph-mon[37330]: 5.f scrub ok 2024-09-11T07:31:10.375 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:10 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:10.395 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:10.436 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:10.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:10.437 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:10.537 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:10.570 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:10.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:10.571 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:10.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:31:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:31:10] "GET /metrics HTTP/1.1" 200 223913 "" "Prometheus/2.43.0" 2024-09-11T07:31:10.672 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:10.715 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:10.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:10.716 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:10.817 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:10.857 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:10.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:10.858 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:10.959 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:10.987 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:10.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:10.988 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:11.089 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:11.139 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:11.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:11.141 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:11.242 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:11.269 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:11.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:11.270 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:11.370 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:11.411 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:11.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:11.412 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:11.472 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:11 smithi022 ceph-mon[29462]: 4.c deep-scrub starts 2024-09-11T07:31:11.472 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:11 smithi022 ceph-mon[29462]: 4.c deep-scrub ok 2024-09-11T07:31:11.472 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:11 smithi022 ceph-mon[29462]: pgmap v188: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.18k op/s; 26946/259095 objects degraded (10.400%); 0 B/s, 0 objects/s recovering 2024-09-11T07:31:11.472 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:11 smithi022 ceph-mon[29462]: 6.0 scrub starts 2024-09-11T07:31:11.472 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:11 smithi022 ceph-mon[29462]: 6.0 scrub ok 2024-09-11T07:31:11.472 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:11 smithi022 ceph-mon[29462]: Health check update: Degraded data redundancy: 26946/259095 objects degraded (10.400%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:11.472 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:11 smithi022 ceph-mon[37330]: 4.c deep-scrub starts 2024-09-11T07:31:11.473 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:11 smithi022 ceph-mon[37330]: 4.c deep-scrub ok 2024-09-11T07:31:11.473 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:11 smithi022 ceph-mon[37330]: pgmap v188: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.18k op/s; 26946/259095 objects degraded (10.400%); 0 B/s, 0 objects/s recovering 2024-09-11T07:31:11.473 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:11 smithi022 ceph-mon[37330]: 6.0 scrub starts 2024-09-11T07:31:11.473 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:11 smithi022 ceph-mon[37330]: 6.0 scrub ok 2024-09-11T07:31:11.473 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:11 smithi022 ceph-mon[37330]: Health check update: Degraded data redundancy: 26946/259095 objects degraded (10.400%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:11.513 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:11.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:11 smithi104 ceph-mon[33507]: 4.c deep-scrub starts 2024-09-11T07:31:11.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:11 smithi104 ceph-mon[33507]: 4.c deep-scrub ok 2024-09-11T07:31:11.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:11 smithi104 ceph-mon[33507]: pgmap v188: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.18k op/s; 26946/259095 objects degraded (10.400%); 0 B/s, 0 objects/s recovering 2024-09-11T07:31:11.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:11 smithi104 ceph-mon[33507]: 6.0 scrub starts 2024-09-11T07:31:11.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:11 smithi104 ceph-mon[33507]: 6.0 scrub ok 2024-09-11T07:31:11.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:11 smithi104 ceph-mon[33507]: Health check update: Degraded data redundancy: 26946/259095 objects degraded (10.400%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:11.560 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:31:11.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:11.561 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:11.562 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:11.662 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:11.687 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:11.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:11.688 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:11.789 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:11.813 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:11.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:11.814 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:11.915 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:11.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:11.969 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:11.969 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:12.033 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:31:12.070 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:12.096 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:12.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:12.098 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:12.198 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:12.241 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:12.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:12.242 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:12.246 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is set 2024-09-11T07:31:12.343 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:12.385 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:12.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:12.386 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:12.455 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:31:12.455 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:31:12.486 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:12.522 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:12.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:12.523 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:12.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:12 smithi104 ceph-mon[33507]: 5.18 scrub starts 2024-09-11T07:31:12.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:12 smithi104 ceph-mon[33507]: 5.18 scrub ok 2024-09-11T07:31:12.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:12 smithi104 ceph-mon[33507]: 2.2 scrub starts 2024-09-11T07:31:12.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:12 smithi104 ceph-mon[33507]: 2.2 scrub ok 2024-09-11T07:31:12.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:12 smithi104 ceph-mon[33507]: 2.11 scrub starts 2024-09-11T07:31:12.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:12 smithi104 ceph-mon[33507]: 2.11 scrub ok 2024-09-11T07:31:12.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:12 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1242870083' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:31:12.624 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:12.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:12 smithi022 ceph-mon[29462]: 5.18 scrub starts 2024-09-11T07:31:12.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:12 smithi022 ceph-mon[29462]: 5.18 scrub ok 2024-09-11T07:31:12.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:12 smithi022 ceph-mon[29462]: 2.2 scrub starts 2024-09-11T07:31:12.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:12 smithi022 ceph-mon[29462]: 2.2 scrub ok 2024-09-11T07:31:12.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:12 smithi022 ceph-mon[29462]: 2.11 scrub starts 2024-09-11T07:31:12.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:12 smithi022 ceph-mon[29462]: 2.11 scrub ok 2024-09-11T07:31:12.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:12 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1242870083' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:31:12.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:12 smithi022 ceph-mon[37330]: 5.18 scrub starts 2024-09-11T07:31:12.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:12 smithi022 ceph-mon[37330]: 5.18 scrub ok 2024-09-11T07:31:12.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:12 smithi022 ceph-mon[37330]: 2.2 scrub starts 2024-09-11T07:31:12.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:12 smithi022 ceph-mon[37330]: 2.2 scrub ok 2024-09-11T07:31:12.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:12 smithi022 ceph-mon[37330]: 2.11 scrub starts 2024-09-11T07:31:12.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:12 smithi022 ceph-mon[37330]: 2.11 scrub ok 2024-09-11T07:31:12.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:12 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1242870083' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:31:12.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:12.673 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:12.673 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:12.774 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:12.797 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:12.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:12.798 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:12.899 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:12.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:12.939 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:12.939 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:13.040 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:13.064 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:13.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:13.065 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:13.166 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:13.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:13.204 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:13.205 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:13.305 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:13.343 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:13.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:13.344 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:13.445 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:13.487 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:13.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:13.488 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:13.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:13 smithi104 ceph-mon[33507]: pgmap v189: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.05k op/s; 26946/259095 objects degraded (10.400%); 0 B/s, 0 objects/s recovering 2024-09-11T07:31:13.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:13 smithi104 ceph-mon[33507]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:31:13.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:13 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1242870083' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:31:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:13 smithi104 ceph-mon[33507]: osdmap e137: 8 total, 7 up, 8 in 2024-09-11T07:31:13.589 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:13.613 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:13.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:13.614 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:13.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:13 smithi022 ceph-mon[29462]: pgmap v189: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.05k op/s; 26946/259095 objects degraded (10.400%); 0 B/s, 0 objects/s recovering 2024-09-11T07:31:13.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:13 smithi022 ceph-mon[29462]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:31:13.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:13 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1242870083' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:31:13.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:13 smithi022 ceph-mon[29462]: osdmap e137: 8 total, 7 up, 8 in 2024-09-11T07:31:13.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:13 smithi022 ceph-mon[37330]: pgmap v189: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.05k op/s; 26946/259095 objects degraded (10.400%); 0 B/s, 0 objects/s recovering 2024-09-11T07:31:13.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:13 smithi022 ceph-mon[37330]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:31:13.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:13 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1242870083' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:31:13.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:13 smithi022 ceph-mon[37330]: osdmap e137: 8 total, 7 up, 8 in 2024-09-11T07:31:13.715 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:13.740 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:13.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:13.741 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:13.842 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:13.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:13.877 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:13.877 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:13.978 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:13.999 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:13.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:14.000 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:14.100 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:14.141 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:14.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:14.152 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:14.252 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:14.284 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:14.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:14.285 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:14.386 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:14.410 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:14.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:14.411 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:14.511 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:14.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:31:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:31:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:31:14.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:14.562 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:14.563 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:14.663 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:14.689 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:14.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:14.690 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:14.791 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:14.838 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:14.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:14.839 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:14.940 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:14.964 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:14.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:14.965 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:15.007 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T07:31:15.065 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:15.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:15.113 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:15.113 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:15.117 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:31:15.118 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:31:15.150 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:31:15.151 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:31:15.181 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:31:15.181 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:31:15.192 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-11T07:31:15.214 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:15.222 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:31:15.222 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:31:15.255 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:31:15.255 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:31:15.266 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:15.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:15.267 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:15.286 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:31:15.286 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:31:15.335 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:15 smithi022 ceph-mon[29462]: pgmap v191: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 5.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 580 B/s rd, 0 B/s wr, 675 op/s; 26946/256983 objects degraded (10.486%) 2024-09-11T07:31:15.335 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:15 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2255119638' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:31:15.335 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:31:15.335 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:31:15.336 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:15 smithi022 ceph-mon[37330]: pgmap v191: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 5.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 580 B/s rd, 0 B/s wr, 675 op/s; 26946/256983 objects degraded (10.486%) 2024-09-11T07:31:15.336 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:15 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2255119638' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:31:15.336 INFO:teuthology.orchestra.run.smithi022.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-11T07:31:15.368 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:15.390 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:15.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:15.392 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:15.440 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:15 smithi104 ceph-mon[33507]: pgmap v191: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 5.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 580 B/s rd, 0 B/s wr, 675 op/s; 26946/256983 objects degraded (10.486%) 2024-09-11T07:31:15.440 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:15 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2255119638' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:31:15.492 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:15.519 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:15.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:15.520 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:15.621 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:15.656 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:15.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:15.657 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:15.758 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:15.765 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:31:15.793 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:15.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:15.794 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:15.895 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:15.936 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:15.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:15.937 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:16.037 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:16.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:16.085 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:16.085 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:16.122 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T07:31:16.123 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T07:31:16.162 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:31:16.185 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:16.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:16.243 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:16.244 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:16.327 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:16 smithi022 ceph-mon[29462]: Health check update: Degraded data redundancy: 26946/256983 objects degraded (10.486%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:16.327 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:16 smithi022 ceph-mon[37330]: Health check update: Degraded data redundancy: 26946/256983 objects degraded (10.486%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:16.344 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:16.385 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:16.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:16.386 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:16.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:16 smithi104 ceph-mon[33507]: Health check update: Degraded data redundancy: 26946/256983 objects degraded (10.486%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:16.432 INFO:teuthology.orchestra.run.smithi104.stdout:4.5 2024-09-11T07:31:16.432 INFO:teuthology.orchestra.run.smithi104.stdout:7.c 2024-09-11T07:31:16.432 INFO:teuthology.orchestra.run.smithi104.stdout:4.3 2024-09-11T07:31:16.432 INFO:teuthology.orchestra.run.smithi104.stdout:5.1f 2024-09-11T07:31:16.432 INFO:teuthology.orchestra.run.smithi104.stdout:2.1a 2024-09-11T07:31:16.432 INFO:teuthology.orchestra.run.smithi104.stdout:4.0 2024-09-11T07:31:16.432 INFO:teuthology.orchestra.run.smithi104.stdout:2.f 2024-09-11T07:31:16.432 INFO:teuthology.orchestra.run.smithi104.stdout:2.0 2024-09-11T07:31:16.432 INFO:teuthology.orchestra.run.smithi104.stdout:2.e 2024-09-11T07:31:16.433 INFO:teuthology.orchestra.run.smithi104.stdout:2.a 2024-09-11T07:31:16.433 INFO:teuthology.orchestra.run.smithi104.stdout:4.17 2024-09-11T07:31:16.433 INFO:teuthology.orchestra.run.smithi104.stdout:1.0 2024-09-11T07:31:16.433 INFO:teuthology.orchestra.run.smithi104.stdout:2.4 2024-09-11T07:31:16.433 INFO:teuthology.orchestra.run.smithi104.stdout:5.19 2024-09-11T07:31:16.433 INFO:teuthology.orchestra.run.smithi104.stdout:4.14 2024-09-11T07:31:16.433 INFO:teuthology.orchestra.run.smithi104.stdout:4.1e 2024-09-11T07:31:16.433 INFO:teuthology.orchestra.run.smithi104.stdout:2.12 2024-09-11T07:31:16.433 INFO:teuthology.orchestra.run.smithi104.stdout:2.18 2024-09-11T07:31:16.433 INFO:teuthology.orchestra.run.smithi104.stdout:6.9 2024-09-11T07:31:16.433 INFO:teuthology.orchestra.run.smithi104.stdout:2.b 2024-09-11T07:31:16.433 INFO:teuthology.orchestra.run.smithi104.stdout:4.8 2024-09-11T07:31:16.434 INFO:teuthology.orchestra.run.smithi104.stdout:6.15 2024-09-11T07:31:16.434 INFO:teuthology.orchestra.run.smithi104.stdout:4.11 2024-09-11T07:31:16.434 INFO:teuthology.orchestra.run.smithi104.stdout:2.1d 2024-09-11T07:31:16.434 INFO:teuthology.orchestra.run.smithi104.stdout:2.1f 2024-09-11T07:31:16.434 INFO:teuthology.orchestra.run.smithi104.stdout:4.a 2024-09-11T07:31:16.434 INFO:teuthology.orchestra.run.smithi104.stdout:6.12 2024-09-11T07:31:16.434 INFO:teuthology.orchestra.run.smithi104.stdout:5.4 2024-09-11T07:31:16.434 INFO:teuthology.orchestra.run.smithi104.stdout:2.5 2024-09-11T07:31:16.434 INFO:teuthology.orchestra.run.smithi104.stdout:2.1c 2024-09-11T07:31:16.434 INFO:teuthology.orchestra.run.smithi104.stdout:2.3 2024-09-11T07:31:16.434 INFO:teuthology.orchestra.run.smithi104.stdout:5.1 2024-09-11T07:31:16.434 INFO:teuthology.orchestra.run.smithi104.stdout:5.10 2024-09-11T07:31:16.435 INFO:teuthology.orchestra.run.smithi104.stdout:5.11 2024-09-11T07:31:16.435 INFO:teuthology.orchestra.run.smithi104.stdout:5.1e 2024-09-11T07:31:16.435 INFO:teuthology.orchestra.run.smithi104.stdout:5.14 2024-09-11T07:31:16.435 INFO:teuthology.orchestra.run.smithi104.stdout:5.17 2024-09-11T07:31:16.435 INFO:teuthology.orchestra.run.smithi104.stdout:4.15 2024-09-11T07:31:16.435 INFO:teuthology.orchestra.run.smithi104.stdout:5.1b 2024-09-11T07:31:16.435 INFO:teuthology.orchestra.run.smithi104.stdout:5.9 2024-09-11T07:31:16.435 INFO:teuthology.orchestra.run.smithi104.stdout:5.d 2024-09-11T07:31:16.435 INFO:teuthology.orchestra.run.smithi104.stdout:6.14 2024-09-11T07:31:16.435 INFO:teuthology.orchestra.run.smithi104.stdout:6.6 2024-09-11T07:31:16.435 INFO:teuthology.orchestra.run.smithi104.stdout:6.16 2024-09-11T07:31:16.436 INFO:teuthology.orchestra.run.smithi104.stdout:6.18 2024-09-11T07:31:16.436 INFO:teuthology.orchestra.run.smithi104.stdout:6.1b 2024-09-11T07:31:16.436 INFO:teuthology.orchestra.run.smithi104.stdout:5.6 2024-09-11T07:31:16.436 INFO:teuthology.orchestra.run.smithi104.stdout:6.3 2024-09-11T07:31:16.436 INFO:teuthology.orchestra.run.smithi104.stdout:7.6 2024-09-11T07:31:16.436 INFO:teuthology.orchestra.run.smithi104.stdout:2.1b 2024-09-11T07:31:16.436 INFO:teuthology.orchestra.run.smithi104.stdout:2.9 2024-09-11T07:31:16.436 INFO:teuthology.orchestra.run.smithi104.stdout:6.5 2024-09-11T07:31:16.436 INFO:teuthology.orchestra.run.smithi104.stdout:5.1a 2024-09-11T07:31:16.436 INFO:teuthology.orchestra.run.smithi104.stdout:6.8 2024-09-11T07:31:16.436 INFO:teuthology.orchestra.run.smithi104.stdout:7.f 2024-09-11T07:31:16.437 INFO:teuthology.orchestra.run.smithi104.stdout:6.b 2024-09-11T07:31:16.437 INFO:teuthology.orchestra.run.smithi104.stdout:2.8 2024-09-11T07:31:16.437 INFO:teuthology.orchestra.run.smithi104.stdout:7.5 2024-09-11T07:31:16.437 INFO:teuthology.orchestra.run.smithi104.stdout:4.16 2024-09-11T07:31:16.437 INFO:teuthology.orchestra.run.smithi104.stdout:6.1c 2024-09-11T07:31:16.437 INFO:teuthology.orchestra.run.smithi104.stdout:7.8 2024-09-11T07:31:16.487 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:16.530 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:16.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:16.531 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:16.632 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:16.657 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:16.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:16.658 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:16.758 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:16.783 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:16.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:16.784 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:16.885 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:16.925 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:16.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:16.926 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:17.027 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:17.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:17.051 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:17.051 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:17.152 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:17.194 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:17.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:17.195 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:17.296 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:17.332 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:17.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:17.333 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:17.434 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:17.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:17.473 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:17.473 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:17.514 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:17 smithi022 ceph-mon[29462]: pgmap v192: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.9 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1022 B/s rd, 0 B/s wr, 1.06k op/s; 25285/242865 objects degraded (10.411%) 2024-09-11T07:31:17.514 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:17 smithi022 ceph-mon[37330]: pgmap v192: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.9 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1022 B/s rd, 0 B/s wr, 1.06k op/s; 25285/242865 objects degraded (10.411%) 2024-09-11T07:31:17.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:17 smithi104 ceph-mon[33507]: pgmap v192: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.9 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1022 B/s rd, 0 B/s wr, 1.06k op/s; 25285/242865 objects degraded (10.411%) 2024-09-11T07:31:17.575 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:17.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:17.624 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:17.624 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:17.725 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:17.747 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:17.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:17.748 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:17.848 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:17.895 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:17.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:17.896 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:17.996 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:18.020 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:18.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:18.021 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:18.122 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:18.124 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:31:18.150 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:18.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:18.150 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:18.156 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -n osd.7 --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal --op export --pgid 5.10 --file /var/log/ceph/exp.5.10.7 2024-09-11T07:31:18.252 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:18.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:18.278 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:18.278 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:18.379 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:18.412 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:18.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:18.413 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:18.426 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/osd.7/config 2024-09-11T07:31:18.514 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:18.571 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:18.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:18.572 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:18.598 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:31:18.632 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:31:18.633 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:31:18.633 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2024-09-11T07:31:18.633 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2024-09-11T07:31:18.633 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:31:18.673 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:18.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:18.724 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:18.724 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:18.825 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:18.851 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:18.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:18.852 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:18.953 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:18.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:18.977 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:18.977 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:19.078 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:19.114 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:19.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:19.115 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:19.216 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:19.253 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:19.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:19.254 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:19.270 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is set 2024-09-11T07:31:19.279 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-11T07:31:19.355 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:19.380 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:19.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:19.381 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:19.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:19 smithi022 ceph-mon[29462]: pgmap v193: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.9 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.07k op/s; 25285/242865 objects degraded (10.411%) 2024-09-11T07:31:19.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:19 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/430239707' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:31:19.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:19 smithi022 ceph-mon[29462]: from='client.24916 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:31:19.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:19 smithi022 ceph-mon[37330]: pgmap v193: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.9 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.07k op/s; 25285/242865 objects degraded (10.411%) 2024-09-11T07:31:19.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:19 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/430239707' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:31:19.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:19 smithi022 ceph-mon[37330]: from='client.24916 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:31:19.482 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:19.525 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:19.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:19.526 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:19.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:19 smithi104 ceph-mon[33507]: pgmap v193: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.9 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.07k op/s; 25285/242865 objects degraded (10.411%) 2024-09-11T07:31:19.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:19 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/430239707' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:31:19.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:19 smithi104 ceph-mon[33507]: from='client.24916 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:31:19.530 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:31:19 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:31:19.626 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:19.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:19.663 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:19.664 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:19.765 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:19.788 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:19.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:19.789 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:19.890 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:19.913 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:19.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:19.914 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:20.015 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:20.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:20.054 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:20.055 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:20.155 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:20.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:20.181 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:20.181 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:20.282 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:20.309 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:20.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:20.310 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:20.411 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:20.439 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:20.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:20.440 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:20.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:20 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:20.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:20 smithi104 ceph-mon[33507]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:31:20.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:20 smithi104 ceph-mon[33507]: from='client.24916 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:31:20.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:20 smithi104 ceph-mon[33507]: osdmap e138: 8 total, 7 up, 8 in 2024-09-11T07:31:20.541 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:20.564 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:20.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:20.565 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:20.661 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:31:20.666 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:20.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:31:20 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:31:20] "GET /metrics HTTP/1.1" 200 223913 "" "Prometheus/2.43.0" 2024-09-11T07:31:20.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:20 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:20.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:20 smithi022 ceph-mon[29462]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:31:20.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:20 smithi022 ceph-mon[29462]: from='client.24916 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:31:20.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:20 smithi022 ceph-mon[29462]: osdmap e138: 8 total, 7 up, 8 in 2024-09-11T07:31:20.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:20 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:20.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:20 smithi022 ceph-mon[37330]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:31:20.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:20 smithi022 ceph-mon[37330]: from='client.24916 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:31:20.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:20 smithi022 ceph-mon[37330]: osdmap e138: 8 total, 7 up, 8 in 2024-09-11T07:31:20.713 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:20.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:20.714 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:20.814 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:20.840 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:20.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:20.841 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:20.942 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:20.980 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:20.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:20.981 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:21.082 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:21.120 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:21.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:21.121 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:21.222 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:21.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:21.269 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:21.269 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:21.370 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:21.415 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:21.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:21.416 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:21.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:21 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:31:21.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:21 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:31:21.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:21 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:31:21.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:21 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:31:21.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:21 smithi022 ceph-mon[29462]: pgmap v195: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s; 23652/227028 objects degraded (10.418%) 2024-09-11T07:31:21.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:21 smithi022 ceph-mon[29462]: osdmap e139: 8 total, 7 up, 8 in 2024-09-11T07:31:21.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:21 smithi022 ceph-mon[29462]: Health check update: Degraded data redundancy: 23652/227028 objects degraded (10.418%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:21.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:21 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:31:21.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:21 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:31:21.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:21 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:31:21.420 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:21 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:31:21.420 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:21 smithi022 ceph-mon[37330]: pgmap v195: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s; 23652/227028 objects degraded (10.418%) 2024-09-11T07:31:21.420 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:21 smithi022 ceph-mon[37330]: osdmap e139: 8 total, 7 up, 8 in 2024-09-11T07:31:21.420 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:21 smithi022 ceph-mon[37330]: Health check update: Degraded data redundancy: 23652/227028 objects degraded (10.418%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:21.517 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:21 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:31:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:21 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:31:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:21 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:31:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:21 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:31:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:21 smithi104 ceph-mon[33507]: pgmap v195: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s; 23652/227028 objects degraded (10.418%) 2024-09-11T07:31:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:21 smithi104 ceph-mon[33507]: osdmap e139: 8 total, 7 up, 8 in 2024-09-11T07:31:21.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:21 smithi104 ceph-mon[33507]: Health check update: Degraded data redundancy: 23652/227028 objects degraded (10.418%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:21.533 INFO:teuthology.orchestra.run.smithi104.stderr:Exporting 5.10 info 5.10( empty local-lis/les=60/61 n=0 ec=60/60 lis/c=60/60 les/c/f=61/61/0 sis=60) 2024-09-11T07:31:21.534 INFO:teuthology.orchestra.run.smithi104.stderr:Export successful 2024-09-11T07:31:21.554 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:21.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:21.555 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:21.655 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:21.683 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:21.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:21.684 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:21.785 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:21.811 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:21.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:21.812 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:21.913 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:21.950 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:21.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:21.951 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:22.052 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:22.079 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:22.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:22.080 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:22.180 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:22.221 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:22.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:22.222 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:22.323 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:22.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:22.376 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:22.376 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:22.477 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:22.518 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:22.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:22.519 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:22.620 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:22.665 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:22.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:22.666 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:22.766 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:22.819 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:22.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:22.820 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:22.921 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:22.940 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -n osd.7 --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal --force --op remove --pgid 5.10 2024-09-11T07:31:22.945 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:22.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:22.946 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:23.047 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:23.059 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:31:23.060 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:31:23.081 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:23.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:23.082 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:23.097 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:31:23.097 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:31:23.131 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:31:23.131 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:31:23.169 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:31:23.169 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:31:23.178 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/osd.7/config 2024-09-11T07:31:23.182 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:23.208 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:31:23.209 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:31:23.216 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:23.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:23.217 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:23.243 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:31:23.243 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:31:23.274 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:31:23.275 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:31:23.277 INFO:teuthology.orchestra.run.smithi022.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-11T07:31:23.318 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:23.342 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:23.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:23.343 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:23.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:23 smithi022 ceph-mon[29462]: pgmap v197: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.25k op/s; 23652/227028 objects degraded (10.418%) 2024-09-11T07:31:23.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:31:23.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:31:23.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:23 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/402617138' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:31:23.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:23 smithi022 ceph-mon[37330]: pgmap v197: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.25k op/s; 23652/227028 objects degraded (10.418%) 2024-09-11T07:31:23.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:31:23.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:31:23.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:23 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/402617138' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:31:23.443 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:23.479 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:23.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:23.480 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:23.522 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:23 smithi104 ceph-mon[33507]: pgmap v197: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.25k op/s; 23652/227028 objects degraded (10.418%) 2024-09-11T07:31:23.522 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:31:23.522 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:31:23.522 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:23 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/402617138' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:31:23.540 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:31:23.540 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:31:23.581 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:23.621 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:23.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:23.625 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:23.725 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:23.753 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:23.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:23.755 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:23.821 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T07:31:23.855 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:23.879 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:23.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:23.880 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:23.981 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:24.003 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:24.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:24.005 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:24.105 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:24.129 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:24.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:24.130 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:24.186 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:31:24 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:31:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:31:24.223 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-11T07:31:24.231 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:24.255 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:24.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:24.256 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:24.257 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T07:31:24.257 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T07:31:24.357 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:24.366 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:31:24.382 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:24.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:24.383 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:24.484 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:24.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:24.510 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:24.511 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:24.611 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:24.635 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:24.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:24.636 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:24.736 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:24.774 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:24.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:24.775 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:24.875 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:24.897 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:24.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:24.898 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:24.999 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:25.023 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:25.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:25.025 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:25.126 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:25.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:25.168 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:25.168 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:25.268 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:25.309 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:25.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:25.310 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:25.410 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:25.457 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:25.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:25.458 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:25.559 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:25.591 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:25 smithi104 ceph-mon[33507]: pgmap v198: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 873 op/s; 23340/221907 objects degraded (10.518%) 2024-09-11T07:31:25.615 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:25.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:25.616 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:25.641 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:25 smithi022 ceph-mon[29462]: pgmap v198: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 873 op/s; 23340/221907 objects degraded (10.518%) 2024-09-11T07:31:25.642 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:25 smithi022 ceph-mon[37330]: pgmap v198: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 873 op/s; 23340/221907 objects degraded (10.518%) 2024-09-11T07:31:25.717 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:25.748 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:25.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:25.750 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:25.850 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:25.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:25.899 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:25.900 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:26.000 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:26.056 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:26.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:26.057 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:26.158 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:26.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:26.199 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:26.200 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:26.201 INFO:teuthology.orchestra.run.smithi104.stdout: marking collection for removal 2024-09-11T07:31:26.201 INFO:teuthology.orchestra.run.smithi104.stdout:setting '_remove' omap key 2024-09-11T07:31:26.201 INFO:teuthology.orchestra.run.smithi104.stdout:finish_remove_pgs 5.10_head removing 5.10 2024-09-11T07:31:26.201 INFO:teuthology.orchestra.run.smithi104.stdout:Remove successful 2024-09-11T07:31:26.258 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:31:26.300 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:26.325 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:26.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:26.326 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:26.426 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:26.469 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:26.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:26.470 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:26.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:26 smithi104 ceph-mon[33507]: Health check update: Degraded data redundancy: 23340/221907 objects degraded (10.518%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:26.571 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:26.584 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:31:26.644 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:26.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:26.645 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:26.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:26 smithi022 ceph-mon[29462]: Health check update: Degraded data redundancy: 23340/221907 objects degraded (10.518%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:26.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:26 smithi022 ceph-mon[37330]: Health check update: Degraded data redundancy: 23340/221907 objects degraded (10.518%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:26.746 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:26.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:26.794 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:26.795 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:26.895 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:26.929 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:26.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:26.930 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:27.031 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:27.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:27.071 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:27.071 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:27.171 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:27.209 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:27.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:27.210 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:27.311 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:27.351 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:27.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:27.353 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:27.453 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:27.479 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:27 smithi104 ceph-mon[33507]: pgmap v199: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s; 22058/210927 objects degraded (10.458%) 2024-09-11T07:31:27.489 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:27.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:27.490 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:27.509 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -n osd.7 --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.5.10.7 2024-09-11T07:31:27.590 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:27.615 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:27.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:27.616 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:27.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:27 smithi022 ceph-mon[29462]: pgmap v199: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s; 22058/210927 objects degraded (10.458%) 2024-09-11T07:31:27.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:27 smithi022 ceph-mon[37330]: pgmap v199: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s; 22058/210927 objects degraded (10.458%) 2024-09-11T07:31:27.716 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:27.733 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/osd.7/config 2024-09-11T07:31:27.745 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:27.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:27.746 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:27.847 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:27.869 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:27.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:27.870 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:27.971 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:28.008 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:28.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:28.009 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:28.110 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:28.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:28.149 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:28.149 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:28.250 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:28.289 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:28.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:28.290 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:28.391 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:28.429 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:28.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:28.430 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:28.530 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:28.565 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:28.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:28.566 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:28.667 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:28.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:28.708 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:28.708 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:28.809 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:28.833 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:28.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:28.834 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:28.935 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:28.957 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:28.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:28.958 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:29.059 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:29.103 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:29.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:29.104 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:29.205 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:29.246 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:29.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:29.247 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:29.279 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:31:29 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:31:29.348 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:29.385 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:29.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:29.386 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:29.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:29 smithi022 ceph-mon[29462]: pgmap v200: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 583 B/s rd, 0 B/s wr, 874 op/s; 22058/210927 objects degraded (10.458%) 2024-09-11T07:31:29.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:29 smithi022 ceph-mon[37330]: pgmap v200: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 583 B/s rd, 0 B/s wr, 874 op/s; 22058/210927 objects degraded (10.458%) 2024-09-11T07:31:29.487 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:29.516 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:29.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:29.517 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:29.618 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:29.670 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:29.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:29.671 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:29.765 INFO:teuthology.orchestra.run.smithi104.stdout:get_pg_num_history pg_num_history pg_num_history(e133 pg_nums {1={19=1},2={56=32},3={56=3},4={58=32},5={60=32},6={62=32},7={92=16}} deleted_pools ) 2024-09-11T07:31:29.765 INFO:teuthology.orchestra.run.smithi104.stdout:Importing pgid 5.10 2024-09-11T07:31:29.766 INFO:teuthology.orchestra.run.smithi104.stdout:write_pg epoch 108 info 5.10( empty local-lis/les=60/61 n=0 ec=60/60 lis/c=60/60 les/c/f=61/61/0 sis=60) 2024-09-11T07:31:29.766 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:29 smithi104 ceph-mon[33507]: pgmap v200: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 583 B/s rd, 0 B/s wr, 874 op/s; 22058/210927 objects degraded (10.458%) 2024-09-11T07:31:29.766 INFO:teuthology.orchestra.run.smithi104.stdout:Import successful 2024-09-11T07:31:29.771 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:29.797 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:29.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:29.798 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:29.899 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:29.922 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:29.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:29.923 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:30.024 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:30.049 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:30.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:30.050 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:30.151 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:30.190 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:30.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:30.191 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:30.292 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:30.357 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:30.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:30.358 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:30.459 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:30.481 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:30.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:30.482 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:30.504 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:31:30.583 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:30.604 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:31:30 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:31:30] "GET /metrics HTTP/1.1" 200 223925 "" "Prometheus/2.43.0" 2024-09-11T07:31:30.606 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:30 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:30.606 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:30 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:30.616 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:30 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:30.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:30.642 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:30.642 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:30.743 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:30.765 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:30.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:30.766 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:30.768 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:31:30.768 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:31:30.867 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:30.889 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:30.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:30.890 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:30.991 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:31.036 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:31.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:31.037 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:31.065 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/exp.5.10.7 2024-09-11T07:31:31.138 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:31.176 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:31.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:31.177 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:31.278 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:31.300 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:31.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:31.301 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:31.336 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is unset 2024-09-11T07:31:31.402 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:31.427 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:31.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:31.428 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:31.529 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:31.569 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:31.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:31.570 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:31.616 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:31 smithi022 ceph-mon[29462]: pgmap v201: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.05k op/s; 20513/195516 objects degraded (10.492%) 2024-09-11T07:31:31.616 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:31 smithi022 ceph-mon[29462]: Health check update: Degraded data redundancy: 20513/195516 objects degraded (10.492%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:31.616 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:31 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/558207363' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:31:31.616 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:31 smithi022 ceph-mon[29462]: from='client.24931 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:31:31.616 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:31 smithi022 ceph-mon[37330]: pgmap v201: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.05k op/s; 20513/195516 objects degraded (10.492%) 2024-09-11T07:31:31.616 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:31 smithi022 ceph-mon[37330]: Health check update: Degraded data redundancy: 20513/195516 objects degraded (10.492%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:31.616 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:31 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/558207363' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:31:31.617 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:31 smithi022 ceph-mon[37330]: from='client.24931 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:31:31.671 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:31.711 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:31.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:31.713 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:31.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:31 smithi104 ceph-mon[33507]: pgmap v201: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.05k op/s; 20513/195516 objects degraded (10.492%) 2024-09-11T07:31:31.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:31 smithi104 ceph-mon[33507]: Health check update: Degraded data redundancy: 20513/195516 objects degraded (10.492%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:31.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:31 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/558207363' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:31:31.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:31 smithi104 ceph-mon[33507]: from='client.24931 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:31:31.814 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:31.866 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:31.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:31.867 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:31.968 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:32.006 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:32.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:32.011 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:32.111 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:32.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:32.153 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:32.153 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:32.254 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:32.277 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:32.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:32.278 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:32.291 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-11T07:31:32.379 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:32.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:32.429 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:32.429 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:32.530 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:32.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:32.598 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:32.598 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:32.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:32 smithi022 ceph-mon[29462]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:31:32.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:32 smithi022 ceph-mon[29462]: from='client.24931 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:31:32.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:32 smithi022 ceph-mon[29462]: osdmap e140: 8 total, 7 up, 8 in 2024-09-11T07:31:32.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:32 smithi022 ceph-mon[29462]: 2.7 scrub starts 2024-09-11T07:31:32.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:32 smithi022 ceph-mon[29462]: 2.7 scrub ok 2024-09-11T07:31:32.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:32 smithi022 ceph-mon[29462]: 7.2 scrub starts 2024-09-11T07:31:32.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:32 smithi022 ceph-mon[37330]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:31:32.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:32 smithi022 ceph-mon[37330]: from='client.24931 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:31:32.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:32 smithi022 ceph-mon[37330]: osdmap e140: 8 total, 7 up, 8 in 2024-09-11T07:31:32.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:32 smithi022 ceph-mon[37330]: 2.7 scrub starts 2024-09-11T07:31:32.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:32 smithi022 ceph-mon[37330]: 2.7 scrub ok 2024-09-11T07:31:32.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:32 smithi022 ceph-mon[37330]: 7.2 scrub starts 2024-09-11T07:31:32.699 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:32.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:32.757 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:32.757 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:32.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:32 smithi104 ceph-mon[33507]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:31:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:32 smithi104 ceph-mon[33507]: from='client.24931 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:31:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:32 smithi104 ceph-mon[33507]: osdmap e140: 8 total, 7 up, 8 in 2024-09-11T07:31:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:32 smithi104 ceph-mon[33507]: 2.7 scrub starts 2024-09-11T07:31:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:32 smithi104 ceph-mon[33507]: 2.7 scrub ok 2024-09-11T07:31:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:32 smithi104 ceph-mon[33507]: 7.2 scrub starts 2024-09-11T07:31:32.858 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:32.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:32.895 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:32.895 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:32.996 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:33.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:33.021 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:33.021 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:33.122 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:33.162 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:33.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:33.163 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:33.264 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:33.305 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:33.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:33.306 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:33.407 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:33.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:33 smithi022 ceph-mon[29462]: 4.9 scrub starts 2024-09-11T07:31:33.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:33 smithi022 ceph-mon[29462]: 4.9 scrub ok 2024-09-11T07:31:33.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:33 smithi022 ceph-mon[29462]: pgmap v203: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.05k op/s; 20513/195516 objects degraded (10.492%) 2024-09-11T07:31:33.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:33 smithi022 ceph-mon[29462]: 2.17 scrub starts 2024-09-11T07:31:33.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:33 smithi022 ceph-mon[29462]: 2.17 scrub ok 2024-09-11T07:31:33.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:33 smithi022 ceph-mon[37330]: 4.9 scrub starts 2024-09-11T07:31:33.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:33 smithi022 ceph-mon[37330]: 4.9 scrub ok 2024-09-11T07:31:33.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:33 smithi022 ceph-mon[37330]: pgmap v203: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.05k op/s; 20513/195516 objects degraded (10.492%) 2024-09-11T07:31:33.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:33 smithi022 ceph-mon[37330]: 2.17 scrub starts 2024-09-11T07:31:33.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:33 smithi022 ceph-mon[37330]: 2.17 scrub ok 2024-09-11T07:31:33.477 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:33.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:33.478 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:33.579 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:33.624 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:33.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:33.625 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:33.725 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:33.769 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:31:33.773 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:31:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:31:33.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:33.777 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:33.778 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:33.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:33 smithi104 ceph-mon[33507]: 4.9 scrub starts 2024-09-11T07:31:33.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:33 smithi104 ceph-mon[33507]: 4.9 scrub ok 2024-09-11T07:31:33.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:33 smithi104 ceph-mon[33507]: pgmap v203: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 4.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.05k op/s; 20513/195516 objects degraded (10.492%) 2024-09-11T07:31:33.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:33 smithi104 ceph-mon[33507]: 2.17 scrub starts 2024-09-11T07:31:33.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:33 smithi104 ceph-mon[33507]: 2.17 scrub ok 2024-09-11T07:31:33.812 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:31:33.812 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:31:33.846 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:31:33.846 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:31:33.879 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:33.880 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:31:33.880 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:31:33.911 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:31:33.911 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:31:33.929 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:33.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:33.930 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:33.942 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:31:33.943 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:31:33.975 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:31:33.975 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:31:33.978 INFO:teuthology.orchestra.run.smithi022.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-11T07:31:34.031 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:34.080 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:34.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:34.081 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:34.182 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:34.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:34.224 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:34.224 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:34.325 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:34.347 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:34.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:34.348 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:34.364 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:31:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:31:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:31:34.449 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:34.472 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:34.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:34.473 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:34.574 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:34.622 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:34.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:34.623 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:34.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:34 smithi022 ceph-mon[29462]: 5.e scrub starts 2024-09-11T07:31:34.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:34 smithi022 ceph-mon[29462]: 5.e scrub ok 2024-09-11T07:31:34.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:34 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3493522886' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:31:34.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:34 smithi022 ceph-mon[37330]: 5.e scrub starts 2024-09-11T07:31:34.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:34 smithi022 ceph-mon[37330]: 5.e scrub ok 2024-09-11T07:31:34.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:34 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3493522886' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:31:34.724 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:34.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:34.768 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:34.768 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:34.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:34 smithi104 ceph-mon[33507]: 5.e scrub starts 2024-09-11T07:31:34.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:34 smithi104 ceph-mon[33507]: 5.e scrub ok 2024-09-11T07:31:34.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:34 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3493522886' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:31:34.808 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T07:31:34.809 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T07:31:34.869 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:34.911 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:34.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:34.912 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:35.013 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:35.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:35.057 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:35.057 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:35.158 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:35.199 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:35.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:35.200 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:35.204 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T07:31:35.302 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:35.324 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:35.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:35.325 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:35.426 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:35.449 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:35.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:35.450 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:35.550 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:35.573 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:35 smithi022 ceph-mon[29462]: 4.18 scrub starts 2024-09-11T07:31:35.573 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:35 smithi022 ceph-mon[29462]: 4.18 scrub ok 2024-09-11T07:31:35.573 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:35 smithi022 ceph-mon[29462]: pgmap v204: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 3.9 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.04k op/s; 20175/190788 objects degraded (10.575%) 2024-09-11T07:31:35.573 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:35 smithi022 ceph-mon[29462]: 7.2 scrub ok 2024-09-11T07:31:35.573 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:35 smithi022 ceph-mon[29462]: 7.a scrub starts 2024-09-11T07:31:35.573 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:35 smithi022 ceph-mon[37330]: 4.18 scrub starts 2024-09-11T07:31:35.574 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:35 smithi022 ceph-mon[37330]: 4.18 scrub ok 2024-09-11T07:31:35.574 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:35 smithi022 ceph-mon[37330]: pgmap v204: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 3.9 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.04k op/s; 20175/190788 objects degraded (10.575%) 2024-09-11T07:31:35.574 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:35 smithi022 ceph-mon[37330]: 7.2 scrub ok 2024-09-11T07:31:35.574 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:35 smithi022 ceph-mon[37330]: 7.a scrub starts 2024-09-11T07:31:35.575 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:35.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:35.576 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:35.677 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:35.700 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:35.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:35.701 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:35.746 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:31:35.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:35 smithi104 ceph-mon[33507]: 4.18 scrub starts 2024-09-11T07:31:35.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:35 smithi104 ceph-mon[33507]: 4.18 scrub ok 2024-09-11T07:31:35.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:35 smithi104 ceph-mon[33507]: pgmap v204: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 3.9 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.04k op/s; 20175/190788 objects degraded (10.575%) 2024-09-11T07:31:35.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:35 smithi104 ceph-mon[33507]: 7.2 scrub ok 2024-09-11T07:31:35.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:35 smithi104 ceph-mon[33507]: 7.a scrub starts 2024-09-11T07:31:35.802 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:35.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:35.849 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:35.850 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:35.950 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:35.976 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:35.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:35.977 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:36.078 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:36.096 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [7] live_osds: [0, 1, 2, 3, 4, 5, 6] 2024-09-11T07:31:36.096 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:31:36.096 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-11T07:31:36.096 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T07:31:36.096 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T07:31:36.097 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.1 config set filestore_inject_stall 3 2024-09-11T07:31:36.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:36.127 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:36.127 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:36.228 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:36.265 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:36.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:36.267 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:36.367 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:36.391 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:36.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:36.392 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:36.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:36 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:31:36.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:36 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:31:36.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:36 smithi022 ceph-mon[29462]: osdmap e141: 8 total, 7 up, 8 in 2024-09-11T07:31:36.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:36 smithi022 ceph-mon[29462]: Health check update: Degraded data redundancy: 20175/190788 objects degraded (10.575%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:36.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:36 smithi022 ceph-mon[29462]: 6.1d scrub starts 2024-09-11T07:31:36.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:36 smithi022 ceph-mon[29462]: 6.1d scrub ok 2024-09-11T07:31:36.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:36 smithi022 ceph-mon[29462]: pgmap v206: 148 pgs: 1 active+clean+scrubbing, 35 active+undersized, 26 active+undersized+degraded, 86 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.28k op/s; 18894/180135 objects degraded (10.489%) 2024-09-11T07:31:36.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:36 smithi022 ceph-mon[29462]: 7.3 scrub starts 2024-09-11T07:31:36.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:36 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:31:36.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:36 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:31:36.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:36 smithi022 ceph-mon[37330]: osdmap e141: 8 total, 7 up, 8 in 2024-09-11T07:31:36.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:36 smithi022 ceph-mon[37330]: Health check update: Degraded data redundancy: 20175/190788 objects degraded (10.575%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:36.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:36 smithi022 ceph-mon[37330]: 6.1d scrub starts 2024-09-11T07:31:36.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:36 smithi022 ceph-mon[37330]: 6.1d scrub ok 2024-09-11T07:31:36.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:36 smithi022 ceph-mon[37330]: pgmap v206: 148 pgs: 1 active+clean+scrubbing, 35 active+undersized, 26 active+undersized+degraded, 86 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.28k op/s; 18894/180135 objects degraded (10.489%) 2024-09-11T07:31:36.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:36 smithi022 ceph-mon[37330]: 7.3 scrub starts 2024-09-11T07:31:36.492 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:36.548 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:36.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:36.549 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:36.650 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:36.697 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:36.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:36.698 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:36.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:36 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:31:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:36 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:31:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:36 smithi104 ceph-mon[33507]: osdmap e141: 8 total, 7 up, 8 in 2024-09-11T07:31:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:36 smithi104 ceph-mon[33507]: Health check update: Degraded data redundancy: 20175/190788 objects degraded (10.575%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:36 smithi104 ceph-mon[33507]: 6.1d scrub starts 2024-09-11T07:31:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:36 smithi104 ceph-mon[33507]: 6.1d scrub ok 2024-09-11T07:31:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:36 smithi104 ceph-mon[33507]: pgmap v206: 148 pgs: 1 active+clean+scrubbing, 35 active+undersized, 26 active+undersized+degraded, 86 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.28k op/s; 18894/180135 objects degraded (10.489%) 2024-09-11T07:31:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:36 smithi104 ceph-mon[33507]: 7.3 scrub starts 2024-09-11T07:31:36.799 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:36.809 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:31:36.853 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:36.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:36.854 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:36.955 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:37.066 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:37.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:37.067 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:37.167 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:37.168 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:31:37.168 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:31:37.169 INFO:teuthology.orchestra.run.smithi022.stdout: "complaint_time": 30, 2024-09-11T07:31:37.169 INFO:teuthology.orchestra.run.smithi022.stdout: "num_blocked_ops": 0 2024-09-11T07:31:37.169 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:31:37.191 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:37.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:37.192 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:37.293 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:37.315 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:37.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:37.316 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:37.417 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:37.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:37.483 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:37.483 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:37.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:37 smithi022 ceph-mon[29462]: 2.16 scrub starts 2024-09-11T07:31:37.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:37 smithi022 ceph-mon[29462]: 2.16 scrub ok 2024-09-11T07:31:37.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:37 smithi022 ceph-mon[29462]: 2.d scrub starts 2024-09-11T07:31:37.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:37 smithi022 ceph-mon[29462]: 2.d scrub ok 2024-09-11T07:31:37.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:37 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:31:37.584 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:37.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:37.648 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:37.648 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:37.749 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:37.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:37 smithi104 ceph-mon[33507]: 2.16 scrub starts 2024-09-11T07:31:37.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:37 smithi104 ceph-mon[33507]: 2.16 scrub ok 2024-09-11T07:31:37.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:37 smithi104 ceph-mon[33507]: 2.d scrub starts 2024-09-11T07:31:37.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:37 smithi104 ceph-mon[33507]: 2.d scrub ok 2024-09-11T07:31:37.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:37 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:31:37.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:37.794 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:37.794 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:37.852 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:37 smithi022 ceph-mon[37330]: 2.16 scrub starts 2024-09-11T07:31:37.852 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:37 smithi022 ceph-mon[37330]: 2.16 scrub ok 2024-09-11T07:31:37.852 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:37 smithi022 ceph-mon[37330]: 2.d scrub starts 2024-09-11T07:31:37.852 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:37 smithi022 ceph-mon[37330]: 2.d scrub ok 2024-09-11T07:31:37.852 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:37 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:31:37.895 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:37.899 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-11T07:31:37.919 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:37.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:37.920 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:38.021 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:38.088 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:38.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:38.089 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:38.190 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:38.212 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:38.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:38.213 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:38.314 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:38.335 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:38.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:38.337 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:38.430 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:31:38.437 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:38.442 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:31:38.459 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:31:38.521 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:38.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:38.523 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:38.563 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[29462]: 6.17 scrub starts 2024-09-11T07:31:38.563 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[29462]: 6.17 scrub ok 2024-09-11T07:31:38.563 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[29462]: 2.10 scrub starts 2024-09-11T07:31:38.563 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[29462]: 2.10 scrub ok 2024-09-11T07:31:38.563 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[29462]: 5.3 scrub starts 2024-09-11T07:31:38.563 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[29462]: 5.3 scrub ok 2024-09-11T07:31:38.563 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:31:38.563 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:31:38.563 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[29462]: osdmap e142: 8 total, 7 up, 8 in 2024-09-11T07:31:38.563 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[29462]: 2.6 scrub starts 2024-09-11T07:31:38.564 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[29462]: 2.6 scrub ok 2024-09-11T07:31:38.564 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:31:38.564 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:31:38.564 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:31:38.564 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[29462]: 7.a scrub ok 2024-09-11T07:31:38.564 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[29462]: 7.3 scrub ok 2024-09-11T07:31:38.564 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:31:38.564 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[29462]: 5.b scrub starts 2024-09-11T07:31:38.564 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[29462]: 5.b scrub ok 2024-09-11T07:31:38.564 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[29462]: pgmap v208: 148 pgs: 1 active+clean+scrubbing, 35 active+undersized, 26 active+undersized+degraded, 86 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 763 B/s rd, 0 B/s wr, 764 op/s; 18894/180135 objects degraded (10.489%) 2024-09-11T07:31:38.564 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[29462]: 4.10 scrub starts 2024-09-11T07:31:38.564 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[29462]: 4.10 scrub ok 2024-09-11T07:31:38.565 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[37330]: 6.17 scrub starts 2024-09-11T07:31:38.565 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[37330]: 6.17 scrub ok 2024-09-11T07:31:38.565 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[37330]: 2.10 scrub starts 2024-09-11T07:31:38.565 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[37330]: 2.10 scrub ok 2024-09-11T07:31:38.565 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[37330]: 5.3 scrub starts 2024-09-11T07:31:38.565 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[37330]: 5.3 scrub ok 2024-09-11T07:31:38.565 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:31:38.565 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:31:38.565 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[37330]: osdmap e142: 8 total, 7 up, 8 in 2024-09-11T07:31:38.565 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[37330]: 2.6 scrub starts 2024-09-11T07:31:38.566 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[37330]: 2.6 scrub ok 2024-09-11T07:31:38.566 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:31:38.566 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:31:38.566 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:31:38.566 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[37330]: 7.a scrub ok 2024-09-11T07:31:38.566 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[37330]: 7.3 scrub ok 2024-09-11T07:31:38.566 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:31:38.566 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[37330]: 5.b scrub starts 2024-09-11T07:31:38.566 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[37330]: 5.b scrub ok 2024-09-11T07:31:38.566 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[37330]: pgmap v208: 148 pgs: 1 active+clean+scrubbing, 35 active+undersized, 26 active+undersized+degraded, 86 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 763 B/s rd, 0 B/s wr, 764 op/s; 18894/180135 objects degraded (10.489%) 2024-09-11T07:31:38.566 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[37330]: 4.10 scrub starts 2024-09-11T07:31:38.566 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:38 smithi022 ceph-mon[37330]: 4.10 scrub ok 2024-09-11T07:31:38.624 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:38.669 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:38.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:38.670 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:38.771 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:38.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:38 smithi104 ceph-mon[33507]: 6.17 scrub starts 2024-09-11T07:31:38.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:38 smithi104 ceph-mon[33507]: 6.17 scrub ok 2024-09-11T07:31:38.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:38 smithi104 ceph-mon[33507]: 2.10 scrub starts 2024-09-11T07:31:38.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:38 smithi104 ceph-mon[33507]: 2.10 scrub ok 2024-09-11T07:31:38.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:38 smithi104 ceph-mon[33507]: 5.3 scrub starts 2024-09-11T07:31:38.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:38 smithi104 ceph-mon[33507]: 5.3 scrub ok 2024-09-11T07:31:38.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:38 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:31:38.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:38 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:31:38.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:38 smithi104 ceph-mon[33507]: osdmap e142: 8 total, 7 up, 8 in 2024-09-11T07:31:38.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:38 smithi104 ceph-mon[33507]: 2.6 scrub starts 2024-09-11T07:31:38.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:38 smithi104 ceph-mon[33507]: 2.6 scrub ok 2024-09-11T07:31:38.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:31:38.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:31:38.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:31:38.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:38 smithi104 ceph-mon[33507]: 7.a scrub ok 2024-09-11T07:31:38.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:38 smithi104 ceph-mon[33507]: 7.3 scrub ok 2024-09-11T07:31:38.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:31:38.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:38 smithi104 ceph-mon[33507]: 5.b scrub starts 2024-09-11T07:31:38.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:38 smithi104 ceph-mon[33507]: 5.b scrub ok 2024-09-11T07:31:38.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:38 smithi104 ceph-mon[33507]: pgmap v208: 148 pgs: 1 active+clean+scrubbing, 35 active+undersized, 26 active+undersized+degraded, 86 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 763 B/s rd, 0 B/s wr, 764 op/s; 18894/180135 objects degraded (10.489%) 2024-09-11T07:31:38.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:38 smithi104 ceph-mon[33507]: 4.10 scrub starts 2024-09-11T07:31:38.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:38 smithi104 ceph-mon[33507]: 4.10 scrub ok 2024-09-11T07:31:38.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:38.829 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:38.829 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:38.930 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:38.952 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:38.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:38.953 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:39.054 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:39.078 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:39.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:39.079 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:39.180 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:39.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:39.224 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:39.325 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:39.369 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:39.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:39.370 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:39.471 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:39.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:39.512 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:39.512 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:39.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:31:39 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:31:39.613 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:39.652 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:39.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:39.653 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:39.754 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:39.805 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:39.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:39.806 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:39.824 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is unset 2024-09-11T07:31:39.907 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:39.933 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:39.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:39.934 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:40.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:39 smithi104 ceph-mon[33507]: 6.a scrub starts 2024-09-11T07:31:40.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:39 smithi104 ceph-mon[33507]: 6.a scrub ok 2024-09-11T07:31:40.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:39 smithi104 ceph-mon[33507]: 3.1 scrub starts 2024-09-11T07:31:40.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:39 smithi104 ceph-mon[33507]: 3.1 scrub ok 2024-09-11T07:31:40.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:39 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:40.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:39 smithi104 ceph-mon[33507]: 2.1e scrub starts 2024-09-11T07:31:40.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:39 smithi104 ceph-mon[33507]: 2.1e scrub ok 2024-09-11T07:31:40.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:39 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2625858701' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:31:40.035 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:40.082 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:40.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:40.083 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:40.146 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:39 smithi022 ceph-mon[29462]: 6.a scrub starts 2024-09-11T07:31:40.146 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:39 smithi022 ceph-mon[29462]: 6.a scrub ok 2024-09-11T07:31:40.147 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:39 smithi022 ceph-mon[29462]: 3.1 scrub starts 2024-09-11T07:31:40.147 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:39 smithi022 ceph-mon[29462]: 3.1 scrub ok 2024-09-11T07:31:40.147 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:39 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:40.147 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:39 smithi022 ceph-mon[29462]: 2.1e scrub starts 2024-09-11T07:31:40.147 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:39 smithi022 ceph-mon[29462]: 2.1e scrub ok 2024-09-11T07:31:40.147 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:39 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2625858701' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:31:40.147 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:39 smithi022 ceph-mon[37330]: 6.a scrub starts 2024-09-11T07:31:40.147 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:39 smithi022 ceph-mon[37330]: 6.a scrub ok 2024-09-11T07:31:40.147 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:39 smithi022 ceph-mon[37330]: 3.1 scrub starts 2024-09-11T07:31:40.147 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:39 smithi022 ceph-mon[37330]: 3.1 scrub ok 2024-09-11T07:31:40.147 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:39 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:40.148 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:39 smithi022 ceph-mon[37330]: 2.1e scrub starts 2024-09-11T07:31:40.148 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:39 smithi022 ceph-mon[37330]: 2.1e scrub ok 2024-09-11T07:31:40.148 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:39 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2625858701' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:31:40.184 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:40.232 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:40.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:40.233 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:40.333 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:40.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:40.375 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:40.376 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:40.417 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:31:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:31:40] "GET /metrics HTTP/1.1" 200 223939 "" "Prometheus/2.43.0" 2024-09-11T07:31:40.476 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:40.500 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:40.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:40.501 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:40.601 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:40.638 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:40.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:40.639 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:40.740 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:40.763 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:40.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:40.764 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:40.865 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:40.892 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:40.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:40.893 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:40.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:40 smithi022 ceph-mon[29462]: 7.d scrub starts 2024-09-11T07:31:40.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:40 smithi022 ceph-mon[29462]: 3.0 scrub starts 2024-09-11T07:31:40.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:40 smithi022 ceph-mon[29462]: 3.0 scrub ok 2024-09-11T07:31:40.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:40 smithi022 ceph-mon[29462]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:31:40.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:40 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2625858701' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:31:40.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:40 smithi022 ceph-mon[29462]: osdmap e143: 8 total, 7 up, 8 in 2024-09-11T07:31:40.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:40 smithi022 ceph-mon[29462]: pgmap v210: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.36k op/s; 17424/166290 objects degraded (10.478%) 2024-09-11T07:31:40.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:40 smithi022 ceph-mon[29462]: Health check update: Degraded data redundancy: 18894/180135 objects degraded (10.489%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:40.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:40 smithi022 ceph-mon[37330]: 7.d scrub starts 2024-09-11T07:31:40.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:40 smithi022 ceph-mon[37330]: 3.0 scrub starts 2024-09-11T07:31:40.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:40 smithi022 ceph-mon[37330]: 3.0 scrub ok 2024-09-11T07:31:40.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:40 smithi022 ceph-mon[37330]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:31:40.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:40 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2625858701' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:31:40.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:40 smithi022 ceph-mon[37330]: osdmap e143: 8 total, 7 up, 8 in 2024-09-11T07:31:40.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:40 smithi022 ceph-mon[37330]: pgmap v210: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.36k op/s; 17424/166290 objects degraded (10.478%) 2024-09-11T07:31:40.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:40 smithi022 ceph-mon[37330]: Health check update: Degraded data redundancy: 18894/180135 objects degraded (10.489%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:40.994 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:41.018 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:41.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:41.019 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:41.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:40 smithi104 ceph-mon[33507]: 7.d scrub starts 2024-09-11T07:31:41.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:40 smithi104 ceph-mon[33507]: 3.0 scrub starts 2024-09-11T07:31:41.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:40 smithi104 ceph-mon[33507]: 3.0 scrub ok 2024-09-11T07:31:41.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:40 smithi104 ceph-mon[33507]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:31:41.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:40 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2625858701' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:31:41.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:40 smithi104 ceph-mon[33507]: osdmap e143: 8 total, 7 up, 8 in 2024-09-11T07:31:41.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:40 smithi104 ceph-mon[33507]: pgmap v210: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.36k op/s; 17424/166290 objects degraded (10.478%) 2024-09-11T07:31:41.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:40 smithi104 ceph-mon[33507]: Health check update: Degraded data redundancy: 18894/180135 objects degraded (10.489%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:41.120 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:41.144 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:41.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:41.145 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:41.246 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:41.284 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:41.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:41.285 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:41.386 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:41.431 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:41.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:41.432 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:41.533 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:41.568 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:41.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:41.569 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:41.670 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:41.692 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:41.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:41.693 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:41.794 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:41.817 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:41.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:41.818 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:41.919 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:41.959 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:41.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:41.960 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:42.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:41 smithi104 ceph-mon[33507]: 7.d scrub ok 2024-09-11T07:31:42.061 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:42.105 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:42.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:42.106 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:42.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:41 smithi022 ceph-mon[29462]: 7.d scrub ok 2024-09-11T07:31:42.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:41 smithi022 ceph-mon[37330]: 7.d scrub ok 2024-09-11T07:31:42.207 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:42.247 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:42.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:42.248 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:42.349 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:42.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:42.394 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:42.394 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:42.495 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:42.532 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:42.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:42.533 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:42.634 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:42.677 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:42.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:42.678 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:42.778 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:42.800 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:42.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:42.801 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:42.902 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:42.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:42 smithi022 ceph-mon[29462]: 6.2 scrub starts 2024-09-11T07:31:42.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:42 smithi022 ceph-mon[29462]: 6.2 scrub ok 2024-09-11T07:31:42.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:42 smithi022 ceph-mon[29462]: pgmap v211: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 767 B/s rd, 0 B/s wr, 692 op/s; 17424/166290 objects degraded (10.478%) 2024-09-11T07:31:42.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:42 smithi022 ceph-mon[37330]: 6.2 scrub starts 2024-09-11T07:31:42.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:42 smithi022 ceph-mon[37330]: 6.2 scrub ok 2024-09-11T07:31:42.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:42 smithi022 ceph-mon[37330]: pgmap v211: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 767 B/s rd, 0 B/s wr, 692 op/s; 17424/166290 objects degraded (10.478%) 2024-09-11T07:31:42.926 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:42.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:42.927 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:43.028 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:43.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:42 smithi104 ceph-mon[33507]: 6.2 scrub starts 2024-09-11T07:31:43.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:42 smithi104 ceph-mon[33507]: 6.2 scrub ok 2024-09-11T07:31:43.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:42 smithi104 ceph-mon[33507]: pgmap v211: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 767 B/s rd, 0 B/s wr, 692 op/s; 17424/166290 objects degraded (10.478%) 2024-09-11T07:31:43.068 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:43.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:43.070 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:43.170 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:43.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:43.196 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:43.197 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:43.298 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:43.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:43.323 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:43.323 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:43.380 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T07:31:43.424 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:43.448 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:43.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:43.450 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:43.551 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:43.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:43.600 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:43.600 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:43.701 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:43.726 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:43.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:43.727 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:43.828 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:43.852 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:43.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:43.853 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:43.954 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:43.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:43.978 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:43.978 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:44.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:43 smithi104 ceph-mon[33507]: 2.13 scrub starts 2024-09-11T07:31:44.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:43 smithi104 ceph-mon[33507]: 2.13 scrub ok 2024-09-11T07:31:44.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:43 smithi104 ceph-mon[33507]: 7.7 scrub starts 2024-09-11T07:31:44.079 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:44.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:44.129 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:44.129 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:44.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:43 smithi022 ceph-mon[29462]: 2.13 scrub starts 2024-09-11T07:31:44.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:43 smithi022 ceph-mon[29462]: 2.13 scrub ok 2024-09-11T07:31:44.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:43 smithi022 ceph-mon[29462]: 7.7 scrub starts 2024-09-11T07:31:44.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:43 smithi022 ceph-mon[37330]: 2.13 scrub starts 2024-09-11T07:31:44.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:43 smithi022 ceph-mon[37330]: 2.13 scrub ok 2024-09-11T07:31:44.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:43 smithi022 ceph-mon[37330]: 7.7 scrub starts 2024-09-11T07:31:44.231 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:44.273 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:44.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:44.274 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:44.375 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:44.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:44.397 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:44.397 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:44.498 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:44.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:44.523 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:44.523 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:44.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:31:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:31:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:31:44.624 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:44.646 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:44.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:44.647 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:44.747 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:44.792 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:44.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:44.793 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:44.837 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:44 smithi022 ceph-mon[29462]: 5.15 scrub starts 2024-09-11T07:31:44.855 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:44 smithi022 ceph-mon[29462]: 5.15 scrub ok 2024-09-11T07:31:44.855 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:44 smithi022 ceph-mon[29462]: 4.1b scrub starts 2024-09-11T07:31:44.855 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:44 smithi022 ceph-mon[29462]: 4.1b scrub ok 2024-09-11T07:31:44.855 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:44 smithi022 ceph-mon[29462]: 5.5 scrub starts 2024-09-11T07:31:44.855 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:44 smithi022 ceph-mon[29462]: 5.5 scrub ok 2024-09-11T07:31:44.855 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:44 smithi022 ceph-mon[29462]: 5.1d scrub starts 2024-09-11T07:31:44.855 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:44 smithi022 ceph-mon[29462]: 5.1d scrub ok 2024-09-11T07:31:44.856 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:44 smithi022 ceph-mon[29462]: 7.e scrub starts 2024-09-11T07:31:44.856 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:44 smithi022 ceph-mon[29462]: pgmap v212: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 653 op/s; 17424/164457 objects degraded (10.595%) 2024-09-11T07:31:44.856 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:44 smithi022 ceph-mon[29462]: 7.7 scrub ok 2024-09-11T07:31:44.856 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:44 smithi022 ceph-mon[37330]: 5.15 scrub starts 2024-09-11T07:31:44.856 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:44 smithi022 ceph-mon[37330]: 5.15 scrub ok 2024-09-11T07:31:44.856 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:44 smithi022 ceph-mon[37330]: 4.1b scrub starts 2024-09-11T07:31:44.856 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:44 smithi022 ceph-mon[37330]: 4.1b scrub ok 2024-09-11T07:31:44.856 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:44 smithi022 ceph-mon[37330]: 5.5 scrub starts 2024-09-11T07:31:44.857 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:44 smithi022 ceph-mon[37330]: 5.5 scrub ok 2024-09-11T07:31:44.857 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:44 smithi022 ceph-mon[37330]: 5.1d scrub starts 2024-09-11T07:31:44.857 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:44 smithi022 ceph-mon[37330]: 5.1d scrub ok 2024-09-11T07:31:44.857 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:44 smithi022 ceph-mon[37330]: 7.e scrub starts 2024-09-11T07:31:44.857 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:44 smithi022 ceph-mon[37330]: pgmap v212: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 653 op/s; 17424/164457 objects degraded (10.595%) 2024-09-11T07:31:44.857 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:44 smithi022 ceph-mon[37330]: 7.7 scrub ok 2024-09-11T07:31:44.893 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:31:44.893 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-11T07:31:44.893 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:31:44.893 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:44.932 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:44.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:44.933 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:45.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:44 smithi104 ceph-mon[33507]: 5.15 scrub starts 2024-09-11T07:31:45.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:44 smithi104 ceph-mon[33507]: 5.15 scrub ok 2024-09-11T07:31:45.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:44 smithi104 ceph-mon[33507]: 4.1b scrub starts 2024-09-11T07:31:45.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:44 smithi104 ceph-mon[33507]: 4.1b scrub ok 2024-09-11T07:31:45.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:44 smithi104 ceph-mon[33507]: 5.5 scrub starts 2024-09-11T07:31:45.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:44 smithi104 ceph-mon[33507]: 5.5 scrub ok 2024-09-11T07:31:45.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:44 smithi104 ceph-mon[33507]: 5.1d scrub starts 2024-09-11T07:31:45.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:44 smithi104 ceph-mon[33507]: 5.1d scrub ok 2024-09-11T07:31:45.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:44 smithi104 ceph-mon[33507]: 7.e scrub starts 2024-09-11T07:31:45.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:44 smithi104 ceph-mon[33507]: pgmap v212: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 653 op/s; 17424/164457 objects degraded (10.595%) 2024-09-11T07:31:45.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:44 smithi104 ceph-mon[33507]: 7.7 scrub ok 2024-09-11T07:31:45.034 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:45.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:45.077 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:45.077 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:45.178 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:45.217 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:45.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:45.219 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:45.319 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:45.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:45.382 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:45.382 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:45.483 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:45.507 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:45.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:45.508 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:45.608 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:45.655 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:45.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:45.656 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:45.757 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:45.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:45.781 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:45.781 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:45.882 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:45.900 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:45 smithi022 ceph-mon[29462]: 6.e scrub starts 2024-09-11T07:31:45.900 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:45 smithi022 ceph-mon[29462]: 6.e scrub ok 2024-09-11T07:31:45.900 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:45 smithi022 ceph-mon[29462]: 4.f scrub starts 2024-09-11T07:31:45.901 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:45 smithi022 ceph-mon[29462]: 4.f scrub ok 2024-09-11T07:31:45.901 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:45 smithi022 ceph-mon[29462]: 6.c scrub starts 2024-09-11T07:31:45.901 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:45 smithi022 ceph-mon[29462]: 6.c scrub ok 2024-09-11T07:31:45.901 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:45 smithi022 ceph-mon[29462]: Health check update: Degraded data redundancy: 17424/164457 objects degraded (10.595%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:45.901 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:45 smithi022 ceph-mon[37330]: 6.e scrub starts 2024-09-11T07:31:45.901 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:45 smithi022 ceph-mon[37330]: 6.e scrub ok 2024-09-11T07:31:45.901 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:45 smithi022 ceph-mon[37330]: 4.f scrub starts 2024-09-11T07:31:45.901 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:45 smithi022 ceph-mon[37330]: 4.f scrub ok 2024-09-11T07:31:45.901 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:45 smithi022 ceph-mon[37330]: 6.c scrub starts 2024-09-11T07:31:45.902 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:45 smithi022 ceph-mon[37330]: 6.c scrub ok 2024-09-11T07:31:45.902 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:45 smithi022 ceph-mon[37330]: Health check update: Degraded data redundancy: 17424/164457 objects degraded (10.595%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:45.928 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:45.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:45.929 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:46.029 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:46.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:45 smithi104 ceph-mon[33507]: 6.e scrub starts 2024-09-11T07:31:46.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:45 smithi104 ceph-mon[33507]: 6.e scrub ok 2024-09-11T07:31:46.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:45 smithi104 ceph-mon[33507]: 4.f scrub starts 2024-09-11T07:31:46.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:45 smithi104 ceph-mon[33507]: 4.f scrub ok 2024-09-11T07:31:46.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:45 smithi104 ceph-mon[33507]: 6.c scrub starts 2024-09-11T07:31:46.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:45 smithi104 ceph-mon[33507]: 6.c scrub ok 2024-09-11T07:31:46.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:45 smithi104 ceph-mon[33507]: Health check update: Degraded data redundancy: 17424/164457 objects degraded (10.595%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:46.055 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:46.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:46.056 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:46.157 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:46.205 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:46.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:46.206 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:46.307 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:46.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:46.367 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:46.367 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:46.468 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:46.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:46.508 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:46.508 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:46.609 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:46.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:46.651 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:46.651 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:46.752 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:46.796 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:46.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:46.797 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:46.898 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:46.920 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:46.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:46.921 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:46.949 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:31:46.949 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:31:46.997 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:31:46.997 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:31:47.022 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:47.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:46 smithi104 ceph-mon[33507]: 6.1 scrub starts 2024-09-11T07:31:47.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:46 smithi104 ceph-mon[33507]: 6.1 scrub ok 2024-09-11T07:31:47.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:46 smithi104 ceph-mon[33507]: 7.e scrub ok 2024-09-11T07:31:47.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:46 smithi104 ceph-mon[33507]: pgmap v213: 148 pgs: 2 active+clean+scrubbing, 35 active+undersized, 26 active+undersized+degraded, 85 active+clean; 3.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.13k op/s; 15802/151113 objects degraded (10.457%) 2024-09-11T07:31:47.045 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:31:47.045 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:31:47.046 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:46 smithi022 ceph-mon[29462]: 6.1 scrub starts 2024-09-11T07:31:47.046 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:46 smithi022 ceph-mon[29462]: 6.1 scrub ok 2024-09-11T07:31:47.046 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:46 smithi022 ceph-mon[29462]: 7.e scrub ok 2024-09-11T07:31:47.046 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:46 smithi022 ceph-mon[29462]: pgmap v213: 148 pgs: 2 active+clean+scrubbing, 35 active+undersized, 26 active+undersized+degraded, 85 active+clean; 3.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.13k op/s; 15802/151113 objects degraded (10.457%) 2024-09-11T07:31:47.047 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:46 smithi022 ceph-mon[37330]: 6.1 scrub starts 2024-09-11T07:31:47.047 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:46 smithi022 ceph-mon[37330]: 6.1 scrub ok 2024-09-11T07:31:47.047 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:46 smithi022 ceph-mon[37330]: 7.e scrub ok 2024-09-11T07:31:47.047 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:46 smithi022 ceph-mon[37330]: pgmap v213: 148 pgs: 2 active+clean+scrubbing, 35 active+undersized, 26 active+undersized+degraded, 85 active+clean; 3.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.13k op/s; 15802/151113 objects degraded (10.457%) 2024-09-11T07:31:47.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:47.047 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:47.048 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:47.095 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:31:47.095 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:31:47.108 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:31:47.109 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:31:47.119 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:31:47.140 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:31:47.141 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:31:47.148 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:47.171 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:31:47.171 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:31:47.172 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:47.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:47.173 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:47.205 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:31:47.205 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:31:47.206 INFO:teuthology.orchestra.run.smithi022.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-11T07:31:47.274 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:47.315 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:47.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:47.316 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:47.417 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:47.454 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:47.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:47.455 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:47.556 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:47.597 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:47.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:47.598 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:47.699 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:47.739 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:47.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:47.740 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:47.840 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:47.891 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:47.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:47.892 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:47.981 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:47 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:31:47.981 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:47 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:31:47.981 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:47 smithi022 ceph-mon[29462]: 4.1 scrub starts 2024-09-11T07:31:47.981 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:47 smithi022 ceph-mon[29462]: 4.1 scrub ok 2024-09-11T07:31:47.981 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:47 smithi022 ceph-mon[29462]: 5.c scrub starts 2024-09-11T07:31:47.981 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:47 smithi022 ceph-mon[29462]: 5.c scrub ok 2024-09-11T07:31:47.981 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:47 smithi022 ceph-mon[29462]: osdmap e144: 8 total, 7 up, 8 in 2024-09-11T07:31:47.981 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:47 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2783597381' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:31:47.982 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:47 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:31:47.982 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:47 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:31:47.982 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:47 smithi022 ceph-mon[37330]: 4.1 scrub starts 2024-09-11T07:31:47.982 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:47 smithi022 ceph-mon[37330]: 4.1 scrub ok 2024-09-11T07:31:47.982 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:47 smithi022 ceph-mon[37330]: 5.c scrub starts 2024-09-11T07:31:47.982 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:47 smithi022 ceph-mon[37330]: 5.c scrub ok 2024-09-11T07:31:47.982 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:47 smithi022 ceph-mon[37330]: osdmap e144: 8 total, 7 up, 8 in 2024-09-11T07:31:47.982 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:47 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2783597381' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:31:47.994 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:48.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:47 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:31:48.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:47 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:31:48.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:47 smithi104 ceph-mon[33507]: 4.1 scrub starts 2024-09-11T07:31:48.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:47 smithi104 ceph-mon[33507]: 4.1 scrub ok 2024-09-11T07:31:48.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:47 smithi104 ceph-mon[33507]: 5.c scrub starts 2024-09-11T07:31:48.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:47 smithi104 ceph-mon[33507]: 5.c scrub ok 2024-09-11T07:31:48.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:47 smithi104 ceph-mon[33507]: osdmap e144: 8 total, 7 up, 8 in 2024-09-11T07:31:48.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:47 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2783597381' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:31:48.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:48.031 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:48.031 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:48.132 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:48.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:48.178 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:48.178 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:48.279 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:48.317 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:48.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:48.318 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:48.419 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:48.442 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:48.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:48.443 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:48.543 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:48.582 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:48.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:48.583 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:48.684 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:48.709 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:48.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:48.710 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:48.777 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-11T07:31:48.811 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:48.834 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:48.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:48.835 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:48.936 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:48.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:48.983 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:48.983 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:49.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:48 smithi104 ceph-mon[33507]: 4.4 scrub starts 2024-09-11T07:31:49.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:48 smithi104 ceph-mon[33507]: 4.4 scrub ok 2024-09-11T07:31:49.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:48 smithi104 ceph-mon[33507]: pgmap v215: 148 pgs: 2 active+clean+scrubbing, 35 active+undersized, 26 active+undersized+degraded, 85 active+clean; 3.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 613 B/s rd, 0 B/s wr, 606 op/s; 15802/151113 objects degraded (10.457%) 2024-09-11T07:31:49.084 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:49.106 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:49.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:49.107 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:49.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:48 smithi022 ceph-mon[29462]: 4.4 scrub starts 2024-09-11T07:31:49.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:48 smithi022 ceph-mon[29462]: 4.4 scrub ok 2024-09-11T07:31:49.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:48 smithi022 ceph-mon[29462]: pgmap v215: 148 pgs: 2 active+clean+scrubbing, 35 active+undersized, 26 active+undersized+degraded, 85 active+clean; 3.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 613 B/s rd, 0 B/s wr, 606 op/s; 15802/151113 objects degraded (10.457%) 2024-09-11T07:31:49.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:48 smithi022 ceph-mon[37330]: 4.4 scrub starts 2024-09-11T07:31:49.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:48 smithi022 ceph-mon[37330]: 4.4 scrub ok 2024-09-11T07:31:49.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:48 smithi022 ceph-mon[37330]: pgmap v215: 148 pgs: 2 active+clean+scrubbing, 35 active+undersized, 26 active+undersized+degraded, 85 active+clean; 3.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 613 B/s rd, 0 B/s wr, 606 op/s; 15802/151113 objects degraded (10.457%) 2024-09-11T07:31:49.208 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:49.233 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:49.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:49.234 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:49.334 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:49.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:49.392 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:49.392 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:49.493 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:49.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:49.516 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:49.517 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:49.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:31:49 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:31:49.617 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:49.641 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:49.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:49.643 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:49.743 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:49.789 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:49.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:49.791 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:49.891 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:49.935 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:49.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:49.936 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:50.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:49 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:50.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:49 smithi104 ceph-mon[33507]: 3.2 scrub starts 2024-09-11T07:31:50.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:49 smithi104 ceph-mon[33507]: 3.2 scrub ok 2024-09-11T07:31:50.037 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:50.061 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:50.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:50.061 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:50.162 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:50.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:49 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:50.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:49 smithi022 ceph-mon[29462]: 3.2 scrub starts 2024-09-11T07:31:50.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:49 smithi022 ceph-mon[29462]: 3.2 scrub ok 2024-09-11T07:31:50.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:49 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:31:50.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:49 smithi022 ceph-mon[37330]: 3.2 scrub starts 2024-09-11T07:31:50.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:49 smithi022 ceph-mon[37330]: 3.2 scrub ok 2024-09-11T07:31:50.199 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:50.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:50.200 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:50.300 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:50.322 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:50.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:50.323 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:50.424 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:50.447 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:50.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:50.448 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:50.548 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:50.570 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:50.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:50.571 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:50.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:31:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:31:50] "GET /metrics HTTP/1.1" 200 223939 "" "Prometheus/2.43.0" 2024-09-11T07:31:50.672 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:50.714 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:50.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:50.715 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:50.816 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:50.841 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:50.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:50.842 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:50.943 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:50.976 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:50.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:50.977 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:51.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:50 smithi104 ceph-mon[33507]: 6.1a scrub starts 2024-09-11T07:31:51.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:50 smithi104 ceph-mon[33507]: 6.1a scrub ok 2024-09-11T07:31:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:50 smithi104 ceph-mon[33507]: pgmap v216: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 2.7 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.02k op/s; 14226/135708 objects degraded (10.483%) 2024-09-11T07:31:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:50 smithi104 ceph-mon[33507]: Health check update: Degraded data redundancy: 15802/151113 objects degraded (10.457%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:51.077 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:51.120 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:51.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:51.121 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:51.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:50 smithi022 ceph-mon[29462]: 6.1a scrub starts 2024-09-11T07:31:51.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:50 smithi022 ceph-mon[29462]: 6.1a scrub ok 2024-09-11T07:31:51.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:50 smithi022 ceph-mon[29462]: pgmap v216: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 2.7 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.02k op/s; 14226/135708 objects degraded (10.483%) 2024-09-11T07:31:51.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:50 smithi022 ceph-mon[29462]: Health check update: Degraded data redundancy: 15802/151113 objects degraded (10.457%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:51.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:50 smithi022 ceph-mon[37330]: 6.1a scrub starts 2024-09-11T07:31:51.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:50 smithi022 ceph-mon[37330]: 6.1a scrub ok 2024-09-11T07:31:51.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:50 smithi022 ceph-mon[37330]: pgmap v216: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 2.7 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.02k op/s; 14226/135708 objects degraded (10.483%) 2024-09-11T07:31:51.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:50 smithi022 ceph-mon[37330]: Health check update: Degraded data redundancy: 15802/151113 objects degraded (10.457%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:51.222 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:51.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:51.246 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:51.246 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:51.347 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:51.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:51.385 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:51.385 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:51.486 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:51.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:51.529 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:51.529 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:51.630 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:51.662 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [7] live_osds: [0, 1, 2, 3, 4, 5, 6] 2024-09-11T07:31:51.663 INFO:tasks.thrashosds.thrasher:Reviving osd 7 2024-09-11T07:31:51.663 INFO:tasks.cephadm.osd.7:Restarting osd.7 (starting--it wasn't running)... 2024-09-11T07:31:51.663 DEBUG:teuthology.orchestra.run.smithi104:osd.7> sudo journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.7.service 2024-09-11T07:31:51.665 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl start ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.7 2024-09-11T07:31:51.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:51.669 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:51.670 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:51.770 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:51.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:51.808 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:51.809 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:51.909 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:51.937 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:51.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:51.938 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:51.982 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:51 smithi104 ceph-mon[33507]: 6.10 scrub starts 2024-09-11T07:31:51.982 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:51 smithi104 ceph-mon[33507]: 6.10 scrub ok 2024-09-11T07:31:51.982 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:51 smithi104 systemd[1]: Starting Ceph osd.7 for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:31:52.038 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:52.061 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:52.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:52.062 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:52.163 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:52.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:51 smithi022 ceph-mon[29462]: 6.10 scrub starts 2024-09-11T07:31:52.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:51 smithi022 ceph-mon[29462]: 6.10 scrub ok 2024-09-11T07:31:52.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:51 smithi022 ceph-mon[37330]: 6.10 scrub starts 2024-09-11T07:31:52.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:51 smithi022 ceph-mon[37330]: 6.10 scrub ok 2024-09-11T07:31:52.187 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:52.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:52.188 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:52.246 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:51 smithi104 podman[81114]: 2024-09-11 07:31:51.884550067 +0000 UTC m=+0.018990493 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:31:52.246 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:52 smithi104 podman[81114]: 2024-09-11 07:31:52.075889907 +0000 UTC m=+0.210330381 container create da7ee37684b54eb158701669b589feedb2e7f836813555eba896c3a5a997efbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7-activate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-11T07:31:52.289 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:52.316 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:52.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:52.317 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:52.418 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:52.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:52.459 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:52.460 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:52.530 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:52 smithi104 podman[81114]: 2024-09-11 07:31:52.372447269 +0000 UTC m=+0.506887706 container init da7ee37684b54eb158701669b589feedb2e7f836813555eba896c3a5a997efbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7-activate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-11T07:31:52.530 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:52 smithi104 podman[81114]: 2024-09-11 07:31:52.377097724 +0000 UTC m=+0.511538152 container start da7ee37684b54eb158701669b589feedb2e7f836813555eba896c3a5a997efbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7-activate, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-11T07:31:52.530 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:52 smithi104 podman[81114]: 2024-09-11 07:31:52.487480617 +0000 UTC m=+0.621921052 container attach da7ee37684b54eb158701669b589feedb2e7f836813555eba896c3a5a997efbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7-activate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-11T07:31:52.561 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:52.591 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:52.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:52.592 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:52.693 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:52.703 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T07:31:52.703 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T07:31:52.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:52.733 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:52.733 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:52.763 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:31:52.834 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:52.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:52 smithi104 ceph-mon[33507]: 4.7 deep-scrub starts 2024-09-11T07:31:52.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:52 smithi104 ceph-mon[33507]: 4.7 deep-scrub ok 2024-09-11T07:31:52.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:52 smithi104 ceph-mon[33507]: pgmap v217: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 2.7 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.02k op/s; 14226/135708 objects degraded (10.483%) 2024-09-11T07:31:52.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:52 smithi104 ceph-mon[33507]: 2.19 scrub starts 2024-09-11T07:31:52.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:52 smithi104 ceph-mon[33507]: 2.19 scrub ok 2024-09-11T07:31:52.870 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:52 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7-activate[81162]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-11T07:31:52.870 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:52 smithi104 bash[81114]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-11T07:31:52.870 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:52 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7-activate[81162]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-7 --no-mon-config --dev /dev/mapper/vg_nvme-lv_1 2024-09-11T07:31:52.870 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:52 smithi104 bash[81114]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-7 --no-mon-config --dev /dev/mapper/vg_nvme-lv_1 2024-09-11T07:31:52.870 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:52 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7-activate[81162]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_1 2024-09-11T07:31:52.870 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:52 smithi104 bash[81114]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_1 2024-09-11T07:31:52.870 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:52 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7-activate[81162]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2024-09-11T07:31:52.871 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:52 smithi104 bash[81114]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2024-09-11T07:31:52.871 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:52 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7-activate[81162]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-7/block 2024-09-11T07:31:52.871 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:52 smithi104 bash[81114]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-7/block 2024-09-11T07:31:52.871 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:52 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7-activate[81162]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-11T07:31:52.871 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:52 smithi104 bash[81114]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-11T07:31:52.871 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:52 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7-activate[81162]: --> ceph-volume raw activate successful for osd ID: 7 2024-09-11T07:31:52.871 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:52 smithi104 bash[81114]: --> ceph-volume raw activate successful for osd ID: 7 2024-09-11T07:31:52.871 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:52 smithi104 conmon[81162]: conmon da7ee37684b54eb15870 : Failed to open cgroups file: /sys/fs/cgroup/machine.slice/libpod-da7ee37684b54eb158701669b589feedb2e7f836813555eba896c3a5a997efbd.scope/container/memory.events 2024-09-11T07:31:52.871 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:52 smithi104 podman[81114]: 2024-09-11 07:31:52.772701035 +0000 UTC m=+0.907141468 container died da7ee37684b54eb158701669b589feedb2e7f836813555eba896c3a5a997efbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7-activate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:31:52.883 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:52.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:52.884 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:52.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:52 smithi022 ceph-mon[29462]: 4.7 deep-scrub starts 2024-09-11T07:31:52.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:52 smithi022 ceph-mon[29462]: 4.7 deep-scrub ok 2024-09-11T07:31:52.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:52 smithi022 ceph-mon[29462]: pgmap v217: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 2.7 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.02k op/s; 14226/135708 objects degraded (10.483%) 2024-09-11T07:31:52.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:52 smithi022 ceph-mon[29462]: 2.19 scrub starts 2024-09-11T07:31:52.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:52 smithi022 ceph-mon[29462]: 2.19 scrub ok 2024-09-11T07:31:52.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:52 smithi022 ceph-mon[37330]: 4.7 deep-scrub starts 2024-09-11T07:31:52.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:52 smithi022 ceph-mon[37330]: 4.7 deep-scrub ok 2024-09-11T07:31:52.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:52 smithi022 ceph-mon[37330]: pgmap v217: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 2.7 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.02k op/s; 14226/135708 objects degraded (10.483%) 2024-09-11T07:31:52.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:52 smithi022 ceph-mon[37330]: 2.19 scrub starts 2024-09-11T07:31:52.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:52 smithi022 ceph-mon[37330]: 2.19 scrub ok 2024-09-11T07:31:52.985 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:53.009 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:53.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:53.010 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:53.111 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:53.150 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:53.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:53.151 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:53.252 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:53.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:53.296 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:53.296 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:53.397 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:53.438 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:53.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:53.439 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:53.540 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:53.568 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:53.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:53.569 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:53.670 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:53.695 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:53.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:53.696 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:53.748 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:53 smithi104 podman[81314]: 2024-09-11 07:31:53.477700029 +0000 UTC m=+0.696077521 container remove da7ee37684b54eb158701669b589feedb2e7f836813555eba896c3a5a997efbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7-activate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T07:31:53.748 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:53 smithi104 podman[81413]: 2024-09-11 07:31:53.624281974 +0000 UTC m=+0.019651310 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:31:53.796 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:53.837 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:53.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:53.838 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:53.939 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:53.972 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:53.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:53.973 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:54.003 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:53 smithi104 podman[81413]: 2024-09-11 07:31:53.746892932 +0000 UTC m=+0.142262256 container create 1bfaac5696d838d343422ce2b9c4763abacc2bb3c2f3033e67ad5eb285348862 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:31:54.016 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is set 2024-09-11T07:31:54.074 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:54.108 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-11T07:31:54.116 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:54.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:54.117 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:54.217 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:54.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:54.259 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:54.259 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:54.279 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:31:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:31:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:31:54.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:54 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:31:54.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:54 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:31:54.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:54 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/715967561' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:31:54.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:54 smithi104 ceph-mon[33507]: from='client.24952 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:31:54.280 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:54 smithi104 podman[81413]: 2024-09-11 07:31:54.002589826 +0000 UTC m=+0.397959156 container init 1bfaac5696d838d343422ce2b9c4763abacc2bb3c2f3033e67ad5eb285348862 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-11T07:31:54.280 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:54 smithi104 podman[81413]: 2024-09-11 07:31:54.006408594 +0000 UTC m=+0.401777919 container start 1bfaac5696d838d343422ce2b9c4763abacc2bb3c2f3033e67ad5eb285348862 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD) 2024-09-11T07:31:54.280 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:54 smithi104 bash[81413]: 1bfaac5696d838d343422ce2b9c4763abacc2bb3c2f3033e67ad5eb285348862 2024-09-11T07:31:54.280 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:54 smithi104 systemd[1]: Started Ceph osd.7 for 12a285d8-700e-11ef-bcea-c7b262605968. 2024-09-11T07:31:54.360 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:54.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:54.406 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:54.406 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:54.411 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:31:54.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:54 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:31:54.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:54 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:31:54.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:54 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/715967561' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:31:54.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:54 smithi022 ceph-mon[29462]: from='client.24952 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:31:54.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:54 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:31:54.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:54 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:31:54.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:54 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/715967561' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:31:54.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:54 smithi022 ceph-mon[37330]: from='client.24952 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:31:54.507 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:54.552 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:54.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:54.553 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:54.654 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:54.704 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:31:54.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:54.705 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:54.705 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:54.806 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:54.819 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:31:54.548+0000 7f18b6681440 -1 Falling back to public interface 2024-09-11T07:31:54.847 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:54.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:54.848 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:54.949 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:54.974 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:54.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:54.975 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:55.076 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:55.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:55.128 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:55.128 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:55.159 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:55 smithi022 ceph-mon[37330]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:31:55.159 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:55 smithi022 ceph-mon[37330]: from='client.24952 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:31:55.159 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:55 smithi022 ceph-mon[37330]: osdmap e145: 8 total, 7 up, 8 in 2024-09-11T07:31:55.159 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:55 smithi022 ceph-mon[37330]: pgmap v219: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 2.7 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 830 op/s; 13918/131193 objects degraded (10.609%) 2024-09-11T07:31:55.160 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:55 smithi022 ceph-mon[29462]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:31:55.160 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:55 smithi022 ceph-mon[29462]: from='client.24952 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:31:55.160 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:55 smithi022 ceph-mon[29462]: osdmap e145: 8 total, 7 up, 8 in 2024-09-11T07:31:55.160 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:55 smithi022 ceph-mon[29462]: pgmap v219: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 2.7 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 830 op/s; 13918/131193 objects degraded (10.609%) 2024-09-11T07:31:55.229 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:55.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:55 smithi104 ceph-mon[33507]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:31:55.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:55 smithi104 ceph-mon[33507]: from='client.24952 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:31:55.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:55 smithi104 ceph-mon[33507]: osdmap e145: 8 total, 7 up, 8 in 2024-09-11T07:31:55.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:55 smithi104 ceph-mon[33507]: pgmap v219: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 2.7 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 830 op/s; 13918/131193 objects degraded (10.609%) 2024-09-11T07:31:55.288 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:55.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:55.289 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:55.390 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:55.433 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:55.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:55.434 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:55.535 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:55.558 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:55.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:55.559 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:55.659 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:55.697 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:55.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:55.699 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:55.715 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:31:55.786 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:31:55.787 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:31:55.800 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:55.848 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:55.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:55.849 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:55.950 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:55.976 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:55.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:55.977 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:56.078 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:56.118 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:56.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:56.119 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:56.215 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:56 smithi104 ceph-mon[33507]: Health check update: Degraded data redundancy: 13918/131193 objects degraded (10.609%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:56.216 INFO:teuthology.orchestra.run.smithi104.stderr:no valid command found; 10 closest matches: 2024-09-11T07:31:56.216 INFO:teuthology.orchestra.run.smithi104.stderr:0 2024-09-11T07:31:56.216 INFO:teuthology.orchestra.run.smithi104.stderr:1 2024-09-11T07:31:56.216 INFO:teuthology.orchestra.run.smithi104.stderr:2 2024-09-11T07:31:56.216 INFO:teuthology.orchestra.run.smithi104.stderr:abort 2024-09-11T07:31:56.217 INFO:teuthology.orchestra.run.smithi104.stderr:assert 2024-09-11T07:31:56.217 INFO:teuthology.orchestra.run.smithi104.stderr:config diff 2024-09-11T07:31:56.217 INFO:teuthology.orchestra.run.smithi104.stderr:config diff get 2024-09-11T07:31:56.217 INFO:teuthology.orchestra.run.smithi104.stderr:config get 2024-09-11T07:31:56.217 INFO:teuthology.orchestra.run.smithi104.stderr:config help [] 2024-09-11T07:31:56.217 INFO:teuthology.orchestra.run.smithi104.stderr:config set ... 2024-09-11T07:31:56.217 INFO:teuthology.orchestra.run.smithi104.stderr:admin_socket: invalid command 2024-09-11T07:31:56.219 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:56.256 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:56.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:56.257 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:56.358 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:56.399 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:56.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:56.400 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:56.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:56 smithi022 ceph-mon[29462]: Health check update: Degraded data redundancy: 13918/131193 objects degraded (10.609%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:56.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:56 smithi022 ceph-mon[37330]: Health check update: Degraded data redundancy: 13918/131193 objects degraded (10.609%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:31:56.501 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:56.525 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:56.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:56.526 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:56.626 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:56.652 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:56.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:56.653 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:56.753 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:31:56.794 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:56.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:56.795 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:31:56.895 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:56.921 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-11T07:31:56.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:56.922 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:56.922 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:57.023 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:57.032 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T07:31:57.033 INFO:tasks.cephadm.ceph_manager.ceph:waiting on admin_socket for osd-7, ['dump_ops_in_flight'] 2024-09-11T07:31:57.047 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:57.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:57.048 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:57.149 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:57.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:57 smithi022 ceph-mon[29462]: pgmap v220: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 2.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.09k op/s; 12685/120801 objects degraded (10.501%) 2024-09-11T07:31:57.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:57 smithi022 ceph-mon[37330]: pgmap v220: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 2.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.09k op/s; 12685/120801 objects degraded (10.501%) 2024-09-11T07:31:57.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:57.193 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:57.193 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:57.203 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T07:31:57.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:57 smithi104 ceph-mon[33507]: pgmap v220: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 2.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.09k op/s; 12685/120801 objects degraded (10.501%) 2024-09-11T07:31:57.294 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:57.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:57.339 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:57.339 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:57.440 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:57.486 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:57.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:57.487 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:57.587 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:57.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:57.637 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:57.638 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:57.738 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:57.774 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:57.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:57.775 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:57.876 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:57.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:57.932 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:57.932 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:58.033 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:58.043 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:31:58.088 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:58.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:58.089 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:58.190 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:58.216 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:58.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:58.217 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:58.317 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:58.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:58.357 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:58.357 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:58.458 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:58.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:58.503 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:58.503 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:58.529 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:58 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:31:58.207+0000 7f18b6681440 -1 osd.7 133 log_to_monitors true 2024-09-11T07:31:58.569 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:31:58.603 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:58.625 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:58.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:58.626 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:58.650 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:31:58.651 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:31:58.686 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:31:58.686 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:31:58.723 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:31:58.723 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:31:58.727 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:58.748 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:58.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:58.749 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:58.755 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:31:58.755 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:31:58.782 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:31:58.782 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:31:58.810 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:31:58.811 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:31:58.843 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:31:58.844 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:31:58.844 INFO:teuthology.orchestra.run.smithi022.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-11T07:31:58.850 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:31:58.871 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:58.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:58.872 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:31:58.973 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:31:58.997 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:58.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:58.998 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:31:59.099 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:59.141 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:59.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:59.142 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:59.243 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:31:59.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:59 smithi022 ceph-mon[29462]: pgmap v221: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 2.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.01k op/s; 12685/120801 objects degraded (10.501%) 2024-09-11T07:31:59.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:59 smithi022 ceph-mon[29462]: from='osd.7 v1:172.21.15.104:6812/2218863853' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:31:59.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:59 smithi022 ceph-mon[29462]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:31:59.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:31:59 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1028085188' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:31:59.247 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:59 smithi022 ceph-mon[37330]: pgmap v221: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 2.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.01k op/s; 12685/120801 objects degraded (10.501%) 2024-09-11T07:31:59.247 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:59 smithi022 ceph-mon[37330]: from='osd.7 v1:172.21.15.104:6812/2218863853' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:31:59.247 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:59 smithi022 ceph-mon[37330]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:31:59.248 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:31:59 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1028085188' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:31:59.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:59.268 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:31:59.268 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:31:59.369 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:31:59.407 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:59.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:59.409 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:31:59.510 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:59.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:31:59 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:31:59.530 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:31:59 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:31:59.086+0000 7f18acc2a640 -1 osd.7 133 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:31:59.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:59 smithi104 ceph-mon[33507]: pgmap v221: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 2.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.01k op/s; 12685/120801 objects degraded (10.501%) 2024-09-11T07:31:59.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:59 smithi104 ceph-mon[33507]: from='osd.7 v1:172.21.15.104:6812/2218863853' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:31:59.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:59 smithi104 ceph-mon[33507]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:31:59.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:31:59 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1028085188' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:31:59.546 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:59.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:59.547 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:59.648 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:31:59.686 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:59.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:59.688 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:31:59.789 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:59.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:59.829 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:59.830 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:31:59.930 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:31:59.971 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:31:59.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:31:59.972 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:00.072 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:00.110 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:00.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:00.111 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:00.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:00 smithi022 ceph-mon[29462]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T07:32:00.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:00 smithi022 ceph-mon[29462]: osdmap e146: 8 total, 7 up, 8 in 2024-09-11T07:32:00.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:00 smithi022 ceph-mon[29462]: from='osd.7 v1:172.21.15.104:6812/2218863853' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:32:00.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:00 smithi022 ceph-mon[29462]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:32:00.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:00 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:00.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:00 smithi022 ceph-mon[37330]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T07:32:00.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:00 smithi022 ceph-mon[37330]: osdmap e146: 8 total, 7 up, 8 in 2024-09-11T07:32:00.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:00 smithi022 ceph-mon[37330]: from='osd.7 v1:172.21.15.104:6812/2218863853' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:32:00.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:00 smithi022 ceph-mon[37330]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:32:00.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:00 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:00.211 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:00.222 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T07:32:00.222 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T07:32:00.232 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:00.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:00.233 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:00.334 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:00.355 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:00.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:00.356 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:00.450 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:32:00 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:32:00] "GET /metrics HTTP/1.1" 200 223974 "" "Prometheus/2.43.0" 2024-09-11T07:32:00.458 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:00.479 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:00.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:00.480 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:00.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:00 smithi104 ceph-mon[33507]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T07:32:00.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:00 smithi104 ceph-mon[33507]: osdmap e146: 8 total, 7 up, 8 in 2024-09-11T07:32:00.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:00 smithi104 ceph-mon[33507]: from='osd.7 v1:172.21.15.104:6812/2218863853' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:32:00.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:00 smithi104 ceph-mon[33507]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-09-11T07:32:00.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:00 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:00.581 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:00.603 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:00.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:00.604 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:00.705 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:00.726 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:00.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:00.727 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:00.828 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:00.849 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:00.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:00.850 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:00.951 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:00.972 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:00.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:00.973 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:01.074 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:01.097 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:01.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:01.098 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:01.199 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:01.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:01.222 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:01.222 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:01.323 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:01.345 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:01.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:01.346 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:01.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:01 smithi022 ceph-mon[29462]: pgmap v223: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 2.1 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.25k op/s; 11084/105606 objects degraded (10.496%) 2024-09-11T07:32:01.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:01 smithi022 ceph-mon[29462]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-11T07:32:01.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:01 smithi022 ceph-mon[29462]: osd.7 v1:172.21.15.104:6812/2218863853 boot 2024-09-11T07:32:01.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:01 smithi022 ceph-mon[29462]: osdmap e147: 8 total, 8 up, 8 in 2024-09-11T07:32:01.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:01 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:32:01.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:01 smithi022 ceph-mon[29462]: Health check update: Degraded data redundancy: 11084/105606 objects degraded (10.496%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:32:01.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:01 smithi022 ceph-mon[29462]: osdmap e148: 8 total, 8 up, 8 in 2024-09-11T07:32:01.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:01 smithi022 ceph-mon[37330]: pgmap v223: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 2.1 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.25k op/s; 11084/105606 objects degraded (10.496%) 2024-09-11T07:32:01.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:01 smithi022 ceph-mon[37330]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-11T07:32:01.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:01 smithi022 ceph-mon[37330]: osd.7 v1:172.21.15.104:6812/2218863853 boot 2024-09-11T07:32:01.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:01 smithi022 ceph-mon[37330]: osdmap e147: 8 total, 8 up, 8 in 2024-09-11T07:32:01.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:01 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:32:01.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:01 smithi022 ceph-mon[37330]: Health check update: Degraded data redundancy: 11084/105606 objects degraded (10.496%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:32:01.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:01 smithi022 ceph-mon[37330]: osdmap e148: 8 total, 8 up, 8 in 2024-09-11T07:32:01.446 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:01.468 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:01.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:01.470 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:01.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:01 smithi104 ceph-mon[33507]: pgmap v223: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 2.1 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.25k op/s; 11084/105606 objects degraded (10.496%) 2024-09-11T07:32:01.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:01 smithi104 ceph-mon[33507]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-11T07:32:01.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:01 smithi104 ceph-mon[33507]: osd.7 v1:172.21.15.104:6812/2218863853 boot 2024-09-11T07:32:01.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:01 smithi104 ceph-mon[33507]: osdmap e147: 8 total, 8 up, 8 in 2024-09-11T07:32:01.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:01 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:32:01.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:01 smithi104 ceph-mon[33507]: Health check update: Degraded data redundancy: 11084/105606 objects degraded (10.496%), 26 pgs degraded (PG_DEGRADED) 2024-09-11T07:32:01.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:01 smithi104 ceph-mon[33507]: osdmap e148: 8 total, 8 up, 8 in 2024-09-11T07:32:01.571 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:01.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:01.593 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:01.593 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:01.694 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:01.716 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:01.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:01.717 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:01.818 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:01.841 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:01.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:01.842 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:01.942 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:01.965 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:01.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:01.966 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:02.034 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-11T07:32:02.067 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:02.116 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:02.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:02.117 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:02.217 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:02.219 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:32:02.219 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:32:02.219 INFO:teuthology.orchestra.run.smithi022.stdout: "complaint_time": 30, 2024-09-11T07:32:02.219 INFO:teuthology.orchestra.run.smithi022.stdout: "num_blocked_ops": 0 2024-09-11T07:32:02.219 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:32:02.223 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:32:02.249 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:02.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:02.250 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:02.351 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:02.391 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:02.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:02.392 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:02.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:02 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:32:02.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:02 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:32:02.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:02 smithi022 ceph-mon[29462]: osdmap e149: 8 total, 8 up, 8 in 2024-09-11T07:32:02.417 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:32:02.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:02 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:32:02.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:02 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:32:02.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:02 smithi022 ceph-mon[37330]: osdmap e149: 8 total, 8 up, 8 in 2024-09-11T07:32:02.493 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:02.520 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:02.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:02.521 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:02.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:02 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:32:02.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:02 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:32:02.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:02 smithi104 ceph-mon[33507]: osdmap e149: 8 total, 8 up, 8 in 2024-09-11T07:32:02.621 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:02.664 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:02.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:02.665 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:02.766 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:02.800 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:02.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:02.801 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:02.902 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:02.940 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:02.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:02.941 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:03.042 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:03.073 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:03.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:03.075 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:03.119 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-11T07:32:03.175 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:03.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:03.230 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:03.231 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:03.331 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:03.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:03.389 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:03.389 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:03.396 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:03 smithi022 ceph-mon[29462]: pgmap v227: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 2.1 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s; 11084/105606 objects degraded (10.496%) 2024-09-11T07:32:03.396 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:03 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:32:03.396 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:03 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:32:03.396 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:03 smithi022 ceph-mon[37330]: pgmap v227: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 2.1 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s; 11084/105606 objects degraded (10.496%) 2024-09-11T07:32:03.396 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:03 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:32:03.396 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:03 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:32:03.490 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:03.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:03.521 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:03.522 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:03.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:03 smithi104 ceph-mon[33507]: pgmap v227: 148 pgs: 35 active+undersized, 26 active+undersized+degraded, 87 active+clean; 2.1 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s; 11084/105606 objects degraded (10.496%) 2024-09-11T07:32:03.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:03 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:32:03.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:03 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:32:03.596 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:32:03.623 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:03.649 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:03.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:03.650 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:03.750 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:03.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:03.807 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:03.807 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:03.908 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:03.931 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:32:03.970 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:03.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:03.971 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:04.072 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:04.110 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:04.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:04.111 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:04.160 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:32:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:32:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:32:04.165 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is set 2024-09-11T07:32:04.212 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:04.253 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:04.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:04.254 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:04.355 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:04.363 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:32:04.363 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2024-09-11T07:32:04.363 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:32:04.363 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.24778.0:215381 7.5 e151/148 7:a9dc2670:::benchmark_data_smithi022_77720_object91626:head v 151'13708, mlcod=135'9932)", 2024-09-11T07:32:04.364 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:32:04.354192+0000", 2024-09-11T07:32:04.364 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.0078067989999999997, 2024-09-11T07:32:04.364 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0078282979999999992, 2024-09-11T07:32:04.364 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:32:04.364 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2024-09-11T07:32:04.364 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:32:04.364 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:32:04.364 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:32:04.364 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:04.354192+0000", 2024-09-11T07:32:04.364 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:32:04.364 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:32:04.364 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:32:04.364 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:32:04.365 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:04.354192+0000", 2024-09-11T07:32:04.365 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:32:04.365 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:32:04.365 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:32:04.365 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:32:04.365 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:04.354192+0000", 2024-09-11T07:32:04.365 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:32:04.365 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:32:04.365 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:32:04.365 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:32:04.365 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:04.354197+0000", 2024-09-11T07:32:04.365 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8149999999999996e-06 2024-09-11T07:32:04.365 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:32:04.366 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:32:04.366 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:32:04.366 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:04.354202+0000", 2024-09-11T07:32:04.366 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.699e-06 2024-09-11T07:32:04.366 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:32:04.366 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:32:04.366 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:32:04.366 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:04.354206+0000", 2024-09-11T07:32:04.366 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0659999999999997e-06 2024-09-11T07:32:04.366 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:32:04.366 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:32:04.366 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:32:04.366 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:04.354253+0000", 2024-09-11T07:32:04.366 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6451e-05 2024-09-11T07:32:04.367 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:32:04.367 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:32:04.367 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:32:04.367 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:04.354270+0000", 2024-09-11T07:32:04.367 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7354999999999999e-05 2024-09-11T07:32:04.367 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:32:04.367 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:32:04.367 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:32:04.367 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:32:04.367 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:32:04.367 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.24778.0:215386 7.5 e151/148 7:a5984da2:::benchmark_data_smithi022_77720_object91631:head v 151'13709, mlcod=135'9932)", 2024-09-11T07:32:04.367 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:32:04.359980+0000", 2024-09-11T07:32:04.367 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.0020196400000000001, 2024-09-11T07:32:04.367 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0020801299999999999, 2024-09-11T07:32:04.368 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:32:04.368 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2024-09-11T07:32:04.368 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:32:04.368 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:32:04.368 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:32:04.368 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:04.359980+0000", 2024-09-11T07:32:04.368 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:32:04.368 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:32:04.368 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:32:04.368 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:32:04.368 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:04.359980+0000", 2024-09-11T07:32:04.368 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:32:04.368 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:32:04.368 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:32:04.369 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:32:04.369 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:04.359978+0000", 2024-09-11T07:32:04.369 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:32:04.369 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:32:04.369 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:32:04.369 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:32:04.369 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:04.359988+0000", 2024-09-11T07:32:04.369 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0172e-05 2024-09-11T07:32:04.369 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:32:04.369 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:32:04.369 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:32:04.369 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:04.360000+0000", 2024-09-11T07:32:04.369 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.17e-05 2024-09-11T07:32:04.369 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:32:04.370 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:32:04.370 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:32:04.370 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:04.360006+0000", 2024-09-11T07:32:04.370 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7620000000000001e-06 2024-09-11T07:32:04.370 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:32:04.370 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:32:04.370 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:32:04.370 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:04.360281+0000", 2024-09-11T07:32:04.370 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000275574 2024-09-11T07:32:04.370 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:32:04.370 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:32:04.370 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:32:04.370 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:04.360310+0000", 2024-09-11T07:32:04.371 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.8972e-05 2024-09-11T07:32:04.371 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:32:04.371 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:32:04.371 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:32:04.371 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:32:04.371 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2024-09-11T07:32:04.371 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 2 2024-09-11T07:32:04.371 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:32:04.396 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:04.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:04.397 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:04.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:04 smithi022 ceph-mon[29462]: osdmap e150: 8 total, 8 up, 8 in 2024-09-11T07:32:04.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:04 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1446373556' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:32:04.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:04 smithi022 ceph-mon[29462]: from='client.24967 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:32:04.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:04 smithi022 ceph-mon[37330]: osdmap e150: 8 total, 8 up, 8 in 2024-09-11T07:32:04.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:04 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1446373556' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:32:04.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:04 smithi022 ceph-mon[37330]: from='client.24967 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:32:04.498 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:04.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:04 smithi104 ceph-mon[33507]: osdmap e150: 8 total, 8 up, 8 in 2024-09-11T07:32:04.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:04 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1446373556' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:32:04.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:04 smithi104 ceph-mon[33507]: from='client.24967 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:32:04.540 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:04.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:04.541 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:04.642 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:04.675 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:04.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:04.677 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:04.777 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:04.815 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:04.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:04.816 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:04.917 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:04.954 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:04.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:04.955 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:05.055 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:05.066 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight 0 1 2024-09-11T07:32:05.087 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:05.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:05.088 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:05.189 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:05.217 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:05.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:05.218 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:05.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:05 smithi104 ceph-mon[33507]: pgmap v229: 148 pgs: 1 active+recovering+undersized+remapped, 28 active+undersized, 18 active+undersized+degraded, 101 active+clean; 2.1 GiB data, 9.7 GiB used, 705 GiB / 715 GiB avail; 0 B/s wr, 219 op/s; 8817/102972 objects degraded (8.563%); 0 B/s, 40 objects/s recovering 2024-09-11T07:32:05.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:05 smithi104 ceph-mon[33507]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:32:05.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:05 smithi104 ceph-mon[33507]: from='client.24967 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:32:05.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:05 smithi104 ceph-mon[33507]: osdmap e151: 8 total, 8 up, 8 in 2024-09-11T07:32:05.319 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:05.344 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:05.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:05.345 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:05.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:05 smithi022 ceph-mon[29462]: pgmap v229: 148 pgs: 1 active+recovering+undersized+remapped, 28 active+undersized, 18 active+undersized+degraded, 101 active+clean; 2.1 GiB data, 9.7 GiB used, 705 GiB / 715 GiB avail; 0 B/s wr, 219 op/s; 8817/102972 objects degraded (8.563%); 0 B/s, 40 objects/s recovering 2024-09-11T07:32:05.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:05 smithi022 ceph-mon[29462]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:32:05.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:05 smithi022 ceph-mon[29462]: from='client.24967 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:32:05.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:05 smithi022 ceph-mon[29462]: osdmap e151: 8 total, 8 up, 8 in 2024-09-11T07:32:05.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:05 smithi022 ceph-mon[37330]: pgmap v229: 148 pgs: 1 active+recovering+undersized+remapped, 28 active+undersized, 18 active+undersized+degraded, 101 active+clean; 2.1 GiB data, 9.7 GiB used, 705 GiB / 715 GiB avail; 0 B/s wr, 219 op/s; 8817/102972 objects degraded (8.563%); 0 B/s, 40 objects/s recovering 2024-09-11T07:32:05.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:05 smithi022 ceph-mon[37330]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:32:05.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:05 smithi022 ceph-mon[37330]: from='client.24967 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:32:05.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:05 smithi022 ceph-mon[37330]: osdmap e151: 8 total, 8 up, 8 in 2024-09-11T07:32:05.446 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:05.492 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:05.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:05.493 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:05.594 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:05.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:05.641 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:05.641 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:05.742 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:05.787 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:05.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:05.789 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:05.889 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:05.936 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:05.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:05.937 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:06.038 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:06.066 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:06.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:06.067 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:06.168 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:06.207 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:06.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:06.208 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:06.309 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:06.335 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:06.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:06.336 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:06.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:06 smithi022 ceph-mon[29462]: Health check update: Degraded data redundancy: 8817/102972 objects degraded (8.563%), 18 pgs degraded, 46 pgs undersized (PG_DEGRADED) 2024-09-11T07:32:06.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:06 smithi022 ceph-mon[37330]: Health check update: Degraded data redundancy: 8817/102972 objects degraded (8.563%), 18 pgs degraded, 46 pgs undersized (PG_DEGRADED) 2024-09-11T07:32:06.436 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:06.475 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:06.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:06.476 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:06.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:06 smithi104 ceph-mon[33507]: Health check update: Degraded data redundancy: 8817/102972 objects degraded (8.563%), 18 pgs degraded, 46 pgs undersized (PG_DEGRADED) 2024-09-11T07:32:06.577 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:06.607 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:06.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:06.608 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:06.708 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:06.757 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:06.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:06.761 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:06.862 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:06.889 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:06.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:06.890 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:06.991 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:07.034 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:07.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:07.035 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:07.135 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:07.168 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:07.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:07.169 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:07.270 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:07.291 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:07.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:07.292 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:07.393 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:07.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:07 smithi022 ceph-mon[29462]: pgmap v231: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 2.0 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 920 B/s rd, 0 B/s wr, 507 op/s; 7265/97152 objects degraded (7.478%); 0 B/s, 60 objects/s recovering 2024-09-11T07:32:07.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:07 smithi022 ceph-mon[37330]: pgmap v231: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 2.0 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 920 B/s rd, 0 B/s wr, 507 op/s; 7265/97152 objects degraded (7.478%); 0 B/s, 60 objects/s recovering 2024-09-11T07:32:07.421 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:07.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:07.422 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:07.523 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:07.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:07 smithi104 ceph-mon[33507]: pgmap v231: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 2.0 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 920 B/s rd, 0 B/s wr, 507 op/s; 7265/97152 objects degraded (7.478%); 0 B/s, 60 objects/s recovering 2024-09-11T07:32:07.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:07.575 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:07.575 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:07.676 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:07.714 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:07.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:07.715 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:07.816 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:07.858 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:07.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:07.859 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:07.960 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:08.004 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:08.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:08.005 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:08.105 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:08.140 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:08.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:08.142 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:08.243 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:08.282 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:08.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:08.283 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:08.384 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:08.414 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:08.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:08.415 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:08.516 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:08.543 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:08.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:08.544 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:08.645 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:08.684 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:08.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:08.685 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:08.786 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:08.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:08.845 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:08.845 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:08.946 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:08.977 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:08.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:08.979 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:09.015 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:09 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:32:09.016 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:09 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:32:09.016 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:09 smithi022 ceph-mon[29462]: pgmap v232: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 2.0 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 780 B/s rd, 0 B/s wr, 430 op/s; 7265/97152 objects degraded (7.478%); 0 B/s, 51 objects/s recovering 2024-09-11T07:32:09.079 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:09.121 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:09.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:09.122 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:09.223 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:09.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:09.249 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:09.249 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:09.279 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:32:09 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:32:09.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:09 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:32:09.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:09 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:32:09.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:09 smithi104 ceph-mon[33507]: pgmap v232: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 2.0 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 780 B/s rd, 0 B/s wr, 430 op/s; 7265/97152 objects degraded (7.478%); 0 B/s, 51 objects/s recovering 2024-09-11T07:32:09.321 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:09 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:32:09.321 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:09 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:32:09.321 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:09 smithi022 ceph-mon[37330]: pgmap v232: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 2.0 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 780 B/s rd, 0 B/s wr, 430 op/s; 7265/97152 objects degraded (7.478%); 0 B/s, 51 objects/s recovering 2024-09-11T07:32:09.351 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:09.377 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:09.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:09.379 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:09.479 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:09.522 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:09.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:09.524 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:09.527 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:32:09.590 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:32:09.591 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 20, 2024-09-11T07:32:09.591 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 600, 2024-09-11T07:32:09.591 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:32:09.591 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.591 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.14156.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-11T07:32:09.591 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:24:23.216116+0000", 2024-09-11T07:32:09.591 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 466.37045198300001, 2024-09-11T07:32:09.591 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.432417987, 2024-09-11T07:32:09.591 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:32:09.591 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:32:09.591 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:32:09.591 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.14156", 2024-09-11T07:32:09.592 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/599318175", 2024-09-11T07:32:09.592 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 1 2024-09-11T07:32:09.592 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.592 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:32:09.592 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.592 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:32:09.592 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:24:23.216116+0000", 2024-09-11T07:32:09.592 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.592 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.592 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.592 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:32:09.592 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:24:23.216116+0000", 2024-09-11T07:32:09.592 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.593 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.593 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.593 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:32:09.593 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:24:23.216113+0000", 2024-09-11T07:32:09.593 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999967 2024-09-11T07:32:09.593 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.593 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.593 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:32:09.593 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:24:23.216127+0000", 2024-09-11T07:32:09.593 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3463e-05 2024-09-11T07:32:09.593 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.593 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.593 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:32:09.593 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:24:23.216150+0000", 2024-09-11T07:32:09.594 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2935999999999999e-05 2024-09-11T07:32:09.594 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.594 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.594 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:32:09.594 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:24:23.216157+0000", 2024-09-11T07:32:09.594 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.418e-06 2024-09-11T07:32:09.594 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.594 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.594 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:32:09.594 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:24:23.216217+0000", 2024-09-11T07:32:09.594 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0472e-05 2024-09-11T07:32:09.594 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.594 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.595 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:32:09.595 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:24:23.216224+0000", 2024-09-11T07:32:09.595 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7340000000000002e-06 2024-09-11T07:32:09.595 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.595 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.595 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:32:09.595 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:24:23.647370+0000", 2024-09-11T07:32:09.595 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.43114617 2024-09-11T07:32:09.595 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.595 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.595 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:32:09.595 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:24:23.647522+0000", 2024-09-11T07:32:09.595 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000152051 2024-09-11T07:32:09.595 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.595 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.596 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2", 2024-09-11T07:32:09.596 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:24:23.647710+0000", 2024-09-11T07:32:09.596 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00018746400000000001 2024-09-11T07:32:09.596 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.596 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.596 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:32:09.596 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:24:23.648402+0000", 2024-09-11T07:32:09.596 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00069246100000000001 2024-09-11T07:32:09.596 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.596 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.596 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:32:09.596 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:24:23.648436+0000", 2024-09-11T07:32:09.596 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3788000000000002e-05 2024-09-11T07:32:09.596 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.597 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.597 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:32:09.597 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:24:23.648468+0000", 2024-09-11T07:32:09.597 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.1412999999999999e-05 2024-09-11T07:32:09.597 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.597 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.597 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:32:09.597 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:24:23.648507+0000", 2024-09-11T07:32:09.597 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9740999999999998e-05 2024-09-11T07:32:09.597 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.597 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.597 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:32:09.597 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:24:23.648534+0000", 2024-09-11T07:32:09.598 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7010000000000001e-05 2024-09-11T07:32:09.598 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.598 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:09.598 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.598 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.598 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.598 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:8273 7.b 7:d6886959:::benchmark_data_smithi022_77720_object8272:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-11T07:32:09.598 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:05.163602+0000", 2024-09-11T07:32:09.598 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 184.422966261, 2024-09-11T07:32:09.598 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.037142240999999999, 2024-09-11T07:32:09.598 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:32:09.598 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:32:09.598 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:32:09.598 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:32:09.599 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:32:09.599 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 8273 2024-09-11T07:32:09.599 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.599 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:32:09.599 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.599 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:32:09.599 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.163602+0000", 2024-09-11T07:32:09.599 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.599 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.599 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.599 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:32:09.599 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.163602+0000", 2024-09-11T07:32:09.599 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.599 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.600 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.600 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:32:09.600 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.163600+0000", 2024-09-11T07:32:09.600 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:32:09.600 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.600 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.600 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:32:09.600 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.163627+0000", 2024-09-11T07:32:09.600 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7518999999999999e-05 2024-09-11T07:32:09.600 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.600 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.600 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:32:09.600 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.163643+0000", 2024-09-11T07:32:09.600 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5624000000000002e-05 2024-09-11T07:32:09.600 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.601 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.601 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:32:09.601 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.163649+0000", 2024-09-11T07:32:09.601 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5359999999999998e-06 2024-09-11T07:32:09.601 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.601 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.601 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:32:09.601 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.163696+0000", 2024-09-11T07:32:09.601 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6295999999999999e-05 2024-09-11T07:32:09.601 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.601 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.601 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:32:09.601 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.163831+0000", 2024-09-11T07:32:09.601 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013494700000000001 2024-09-11T07:32:09.602 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.602 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.602 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:32:09.602 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.163954+0000", 2024-09-11T07:32:09.602 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000122924 2024-09-11T07:32:09.602 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.602 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.602 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:32:09.602 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.164556+0000", 2024-09-11T07:32:09.602 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00060287000000000003 2024-09-11T07:32:09.602 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.602 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.602 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:32:09.602 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.182930+0000", 2024-09-11T07:32:09.603 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.018373295000000001 2024-09-11T07:32:09.603 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.603 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.603 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:32:09.603 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.200642+0000", 2024-09-11T07:32:09.603 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.017711764000000001 2024-09-11T07:32:09.603 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.603 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.603 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:32:09.603 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.200695+0000", 2024-09-11T07:32:09.603 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3541000000000001e-05 2024-09-11T07:32:09.603 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.603 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.603 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:32:09.603 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.200744+0000", 2024-09-11T07:32:09.604 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9181000000000002e-05 2024-09-11T07:32:09.604 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.604 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:09.604 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.604 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.604 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.604 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:8281 7.b 7:d9384bb1:::benchmark_data_smithi022_77720_object8280:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-11T07:32:09.605 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:05.165115+0000", 2024-09-11T07:32:09.605 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 184.42145356500001, 2024-09-11T07:32:09.605 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.035891356999999999, 2024-09-11T07:32:09.605 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:32:09.605 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:32:09.605 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:32:09.605 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:32:09.605 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:32:09.605 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 8281 2024-09-11T07:32:09.605 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.605 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:32:09.605 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.605 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:32:09.605 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.165115+0000", 2024-09-11T07:32:09.606 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.606 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.606 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.606 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:32:09.606 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.165115+0000", 2024-09-11T07:32:09.606 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.606 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.606 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.606 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:32:09.606 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.165113+0000", 2024-09-11T07:32:09.606 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:32:09.606 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.606 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.606 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:32:09.606 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.165143+0000", 2024-09-11T07:32:09.606 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0391000000000001e-05 2024-09-11T07:32:09.607 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.607 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.607 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:32:09.607 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.165158+0000", 2024-09-11T07:32:09.607 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4961e-05 2024-09-11T07:32:09.607 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.607 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.607 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:32:09.607 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.165165+0000", 2024-09-11T07:32:09.607 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0690000000000004e-06 2024-09-11T07:32:09.607 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.607 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.607 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:32:09.607 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.165216+0000", 2024-09-11T07:32:09.607 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1075999999999997e-05 2024-09-11T07:32:09.607 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.608 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.608 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:32:09.608 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.165354+0000", 2024-09-11T07:32:09.608 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013769100000000001 2024-09-11T07:32:09.608 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.608 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.608 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:32:09.608 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.165520+0000", 2024-09-11T07:32:09.608 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00016609799999999999 2024-09-11T07:32:09.608 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.608 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.608 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:32:09.608 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.166748+0000", 2024-09-11T07:32:09.608 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0012278549999999999 2024-09-11T07:32:09.609 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.609 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.609 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:32:09.609 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.182957+0000", 2024-09-11T07:32:09.609 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.016208977999999999 2024-09-11T07:32:09.609 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.609 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.609 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:32:09.609 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.200790+0000", 2024-09-11T07:32:09.609 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.017833112000000002 2024-09-11T07:32:09.609 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.609 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.609 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:32:09.609 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.200966+0000", 2024-09-11T07:32:09.609 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000175797 2024-09-11T07:32:09.610 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.610 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.610 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:32:09.610 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:05.201006+0000", 2024-09-11T07:32:09.610 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0521e-05 2024-09-11T07:32:09.610 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.610 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:09.610 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.610 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.610 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.610 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:24102 7.2 e102/92 7:413b5fb6:::benchmark_data_smithi022_77720_object24101:head v 102'1563, mlcod=102'1560)", 2024-09-11T07:32:09.610 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:11.441977+0000", 2024-09-11T07:32:09.610 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 178.144591566, 2024-09-11T07:32:09.610 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.060123865999999998, 2024-09-11T07:32:09.610 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:32:09.610 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:32:09.611 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:32:09.611 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.611 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:32:09.611 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.441977+0000", 2024-09-11T07:32:09.611 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.611 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.611 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.611 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:32:09.611 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.441977+0000", 2024-09-11T07:32:09.611 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.611 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.611 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.611 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:32:09.611 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.441975+0000", 2024-09-11T07:32:09.612 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:32:09.612 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.612 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.612 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:32:09.612 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.441999+0000", 2024-09-11T07:32:09.612 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.353e-05 2024-09-11T07:32:09.612 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.612 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.612 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:32:09.612 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.498051+0000", 2024-09-11T07:32:09.612 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.056052356999999997 2024-09-11T07:32:09.612 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.612 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.612 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:32:09.613 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.498063+0000", 2024-09-11T07:32:09.613 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2065000000000001e-05 2024-09-11T07:32:09.613 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.613 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.613 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:32:09.613 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.499419+0000", 2024-09-11T07:32:09.613 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0013562610000000001 2024-09-11T07:32:09.613 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.613 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.613 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:32:09.613 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.499438+0000", 2024-09-11T07:32:09.613 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8417e-05 2024-09-11T07:32:09.613 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.613 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.613 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:32:09.614 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.501884+0000", 2024-09-11T07:32:09.614 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0024465530000000002 2024-09-11T07:32:09.614 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.614 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.614 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:32:09.614 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.502101+0000", 2024-09-11T07:32:09.614 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000216219 2024-09-11T07:32:09.614 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.614 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:09.614 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.614 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.614 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.614 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:24108 7.0 e102/92 7:07baab64:::benchmark_data_smithi022_77720_object24107:head v 102'1494, mlcod=102'1492)", 2024-09-11T07:32:09.614 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:11.442340+0000", 2024-09-11T07:32:09.614 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 178.14422827300001, 2024-09-11T07:32:09.615 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.057753658999999999, 2024-09-11T07:32:09.615 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:32:09.615 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:32:09.615 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:32:09.615 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.615 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:32:09.615 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.442340+0000", 2024-09-11T07:32:09.615 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.615 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.615 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.615 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:32:09.615 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.442340+0000", 2024-09-11T07:32:09.615 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.615 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.616 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.616 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:32:09.616 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.442339+0000", 2024-09-11T07:32:09.616 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:32:09.616 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.616 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.616 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:32:09.616 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.442369+0000", 2024-09-11T07:32:09.616 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0125999999999999e-05 2024-09-11T07:32:09.616 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.616 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.616 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:32:09.616 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.498044+0000", 2024-09-11T07:32:09.616 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.055675692999999998 2024-09-11T07:32:09.616 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.617 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.617 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:32:09.617 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.498065+0000", 2024-09-11T07:32:09.617 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0227000000000001e-05 2024-09-11T07:32:09.617 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.617 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.617 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:32:09.617 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.498371+0000", 2024-09-11T07:32:09.617 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00030612800000000002 2024-09-11T07:32:09.617 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.617 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.617 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:32:09.617 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.498393+0000", 2024-09-11T07:32:09.617 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1903e-05 2024-09-11T07:32:09.617 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.618 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.618 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:32:09.618 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.500050+0000", 2024-09-11T07:32:09.618 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0016569009999999999 2024-09-11T07:32:09.618 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.618 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.618 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:32:09.618 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.500094+0000", 2024-09-11T07:32:09.618 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4020000000000002e-05 2024-09-11T07:32:09.618 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.618 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:09.618 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.618 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.618 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.618 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:24110 7.b 7:d3b0d0d7:::benchmark_data_smithi022_77720_object24109:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-11T07:32:09.619 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:11.442658+0000", 2024-09-11T07:32:09.619 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 178.143910654, 2024-09-11T07:32:09.619 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.061872910000000003, 2024-09-11T07:32:09.619 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:32:09.619 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:32:09.619 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:32:09.619 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:32:09.619 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:32:09.619 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 24110 2024-09-11T07:32:09.619 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.619 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:32:09.619 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.619 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:32:09.619 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.442658+0000", 2024-09-11T07:32:09.619 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.620 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.620 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.620 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:32:09.620 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.442658+0000", 2024-09-11T07:32:09.620 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.620 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.620 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.620 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:32:09.620 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.442655+0000", 2024-09-11T07:32:09.620 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:32:09.620 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.620 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.620 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:32:09.620 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.442680+0000", 2024-09-11T07:32:09.621 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5068000000000001e-05 2024-09-11T07:32:09.621 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.621 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.621 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:32:09.621 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.498059+0000", 2024-09-11T07:32:09.621 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.055379123000000002 2024-09-11T07:32:09.621 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.621 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.621 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:32:09.621 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.498080+0000", 2024-09-11T07:32:09.621 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0271e-05 2024-09-11T07:32:09.621 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.621 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.622 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:32:09.622 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.499990+0000", 2024-09-11T07:32:09.622 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001910471 2024-09-11T07:32:09.622 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.622 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.622 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:32:09.622 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.500168+0000", 2024-09-11T07:32:09.622 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00017778800000000001 2024-09-11T07:32:09.622 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.622 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.622 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:32:09.622 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.500340+0000", 2024-09-11T07:32:09.622 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00017171100000000001 2024-09-11T07:32:09.622 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.622 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.623 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:32:09.623 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.502646+0000", 2024-09-11T07:32:09.623 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0023060540000000001 2024-09-11T07:32:09.623 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.623 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.623 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:32:09.623 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.504289+0000", 2024-09-11T07:32:09.623 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0016436560000000001 2024-09-11T07:32:09.623 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.623 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.623 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:32:09.623 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.504317+0000", 2024-09-11T07:32:09.623 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7284000000000002e-05 2024-09-11T07:32:09.623 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.623 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.624 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:32:09.624 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.504492+0000", 2024-09-11T07:32:09.624 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00017533800000000001 2024-09-11T07:32:09.624 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.624 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.624 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:32:09.624 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:11.504531+0000", 2024-09-11T07:32:09.624 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8625999999999999e-05 2024-09-11T07:32:09.624 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.624 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:09.624 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.624 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.624 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.625 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:91154 7.b 7:d8dcfa0e:::benchmark_data_smithi022_77720_object91153:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-11T07:32:09.625 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:59.536681+0000", 2024-09-11T07:32:09.625 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 130.04988679799999, 2024-09-11T07:32:09.625 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.039245069, 2024-09-11T07:32:09.625 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:32:09.625 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:32:09.625 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:32:09.625 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:32:09.625 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:32:09.625 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 91154 2024-09-11T07:32:09.625 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.625 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:32:09.625 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.625 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:32:09.625 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.536681+0000", 2024-09-11T07:32:09.626 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.626 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.626 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.626 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:32:09.626 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.536681+0000", 2024-09-11T07:32:09.626 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.626 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.626 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.626 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:32:09.626 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.536679+0000", 2024-09-11T07:32:09.626 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:32:09.626 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.626 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.626 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:32:09.626 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.536702+0000", 2024-09-11T07:32:09.626 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3000999999999999e-05 2024-09-11T07:32:09.627 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.627 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.627 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:32:09.627 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.537660+0000", 2024-09-11T07:32:09.627 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00095715700000000004 2024-09-11T07:32:09.627 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.627 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.627 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:32:09.627 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.537669+0000", 2024-09-11T07:32:09.627 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.3819999999999993e-06 2024-09-11T07:32:09.627 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.627 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.627 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:32:09.627 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.537878+0000", 2024-09-11T07:32:09.628 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00020911599999999999 2024-09-11T07:32:09.628 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.628 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.628 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:32:09.628 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.538197+0000", 2024-09-11T07:32:09.628 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00031905699999999998 2024-09-11T07:32:09.628 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.628 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.628 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:32:09.628 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.538590+0000", 2024-09-11T07:32:09.628 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00039287000000000002 2024-09-11T07:32:09.628 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.628 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.628 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:32:09.628 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.555670+0000", 2024-09-11T07:32:09.628 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.017080254999999999 2024-09-11T07:32:09.629 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.629 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.629 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:32:09.629 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.555713+0000", 2024-09-11T07:32:09.629 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3007999999999999e-05 2024-09-11T07:32:09.629 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.629 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.629 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:32:09.629 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.575845+0000", 2024-09-11T07:32:09.629 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.020131713999999998 2024-09-11T07:32:09.629 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.629 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.629 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:32:09.629 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.575890+0000", 2024-09-11T07:32:09.629 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4722000000000003e-05 2024-09-11T07:32:09.630 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.630 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.630 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:32:09.630 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.575927+0000", 2024-09-11T07:32:09.630 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:09.631 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6885999999999999e-05 2024-09-11T07:32:09.631 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.631 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:09.631 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.631 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.631 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.631 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:91155 7.b 7:db4e00da:::benchmark_data_smithi022_77720_object91154:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-11T07:32:09.631 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:29:59.537686+0000", 2024-09-11T07:32:09.631 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 130.04888218799999, 2024-09-11T07:32:09.632 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.038309079000000003, 2024-09-11T07:32:09.632 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:32:09.632 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:32:09.632 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:32:09.632 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:32:09.632 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:32:09.632 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 91155 2024-09-11T07:32:09.632 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.632 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:32:09.632 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.632 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:32:09.632 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.537686+0000", 2024-09-11T07:32:09.632 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.632 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.632 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.633 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:32:09.633 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.537686+0000", 2024-09-11T07:32:09.633 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.633 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.633 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.633 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:32:09.633 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.537685+0000", 2024-09-11T07:32:09.633 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:32:09.633 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.633 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.633 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:32:09.633 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.537708+0000", 2024-09-11T07:32:09.633 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2807999999999999e-05 2024-09-11T07:32:09.633 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.634 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.634 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:32:09.634 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.537716+0000", 2024-09-11T07:32:09.634 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.0860000000000002e-06 2024-09-11T07:32:09.634 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.634 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.634 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:32:09.634 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.537726+0000", 2024-09-11T07:32:09.634 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0332999999999999e-05 2024-09-11T07:32:09.634 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.634 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.634 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:32:09.634 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.542750+0000", 2024-09-11T07:32:09.634 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0050237939999999998 2024-09-11T07:32:09.634 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.634 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.635 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:32:09.635 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.542879+0000", 2024-09-11T07:32:09.635 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012960900000000001 2024-09-11T07:32:09.635 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.635 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.635 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:32:09.635 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.543019+0000", 2024-09-11T07:32:09.635 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013990000000000001 2024-09-11T07:32:09.635 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.635 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.635 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:32:09.635 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.555753+0000", 2024-09-11T07:32:09.635 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.012733314000000001 2024-09-11T07:32:09.635 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.635 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.636 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:32:09.636 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.555800+0000", 2024-09-11T07:32:09.636 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7630999999999998e-05 2024-09-11T07:32:09.636 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.636 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.636 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:32:09.636 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.575935+0000", 2024-09-11T07:32:09.636 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.020135093 2024-09-11T07:32:09.636 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.636 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.636 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:32:09.636 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.575966+0000", 2024-09-11T07:32:09.636 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0375e-05 2024-09-11T07:32:09.636 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.637 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.637 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:32:09.637 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:29:59.575995+0000", 2024-09-11T07:32:09.637 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.9456000000000001e-05 2024-09-11T07:32:09.637 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.637 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:09.637 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.637 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.637 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.637 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:92046 7.b 7:d1b42d24:::benchmark_data_smithi022_77720_object92045:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-11T07:32:09.637 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:00.457974+0000", 2024-09-11T07:32:09.637 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 129.128594252, 2024-09-11T07:32:09.637 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.035864472000000001, 2024-09-11T07:32:09.637 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:32:09.637 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:32:09.638 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:32:09.638 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:32:09.638 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:32:09.638 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 92046 2024-09-11T07:32:09.638 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.638 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:32:09.638 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.638 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:32:09.638 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:00.457974+0000", 2024-09-11T07:32:09.638 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.638 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.638 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.638 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:32:09.638 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:00.457974+0000", 2024-09-11T07:32:09.638 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.638 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.639 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.639 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:32:09.639 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:00.457972+0000", 2024-09-11T07:32:09.639 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:32:09.639 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.639 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.639 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:32:09.639 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:00.458000+0000", 2024-09-11T07:32:09.639 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8705e-05 2024-09-11T07:32:09.639 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.639 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.639 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:32:09.639 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:00.458017+0000", 2024-09-11T07:32:09.639 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.7013000000000001e-05 2024-09-11T07:32:09.639 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.640 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.640 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:32:09.640 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:00.458024+0000", 2024-09-11T07:32:09.640 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.917e-06 2024-09-11T07:32:09.640 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.640 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.640 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:32:09.640 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:00.463354+0000", 2024-09-11T07:32:09.640 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0053295779999999997 2024-09-11T07:32:09.640 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.640 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.640 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:32:09.640 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:00.463487+0000", 2024-09-11T07:32:09.640 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000132977 2024-09-11T07:32:09.640 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.641 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.641 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:32:09.641 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:00.463642+0000", 2024-09-11T07:32:09.641 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015546000000000001 2024-09-11T07:32:09.641 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.641 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.641 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:32:09.641 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:00.470367+0000", 2024-09-11T07:32:09.641 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0067244180000000002 2024-09-11T07:32:09.641 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.641 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.641 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:32:09.641 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:00.470394+0000", 2024-09-11T07:32:09.642 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7560999999999999e-05 2024-09-11T07:32:09.642 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.642 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.642 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:32:09.642 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:00.493682+0000", 2024-09-11T07:32:09.642 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.023287569000000001 2024-09-11T07:32:09.642 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.642 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.642 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:32:09.642 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:00.493794+0000", 2024-09-11T07:32:09.642 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011212 2024-09-11T07:32:09.642 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.642 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.642 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:32:09.643 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:00.493838+0000", 2024-09-11T07:32:09.643 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4422999999999998e-05 2024-09-11T07:32:09.643 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.643 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:09.643 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.643 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.643 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.643 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:107851 7.b 7:d4fca45d:::benchmark_data_smithi022_77720_object107850:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-11T07:32:09.643 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:15.794792+0000", 2024-09-11T07:32:09.643 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 113.79177614300001, 2024-09-11T07:32:09.643 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.039892566999999997, 2024-09-11T07:32:09.643 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:32:09.643 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:32:09.643 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:32:09.643 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:32:09.643 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:32:09.644 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 107851 2024-09-11T07:32:09.644 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.644 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:32:09.644 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.644 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:32:09.644 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:15.794792+0000", 2024-09-11T07:32:09.644 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.644 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.644 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.644 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:32:09.644 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:15.794792+0000", 2024-09-11T07:32:09.644 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.644 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.644 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.644 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:32:09.645 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:15.794791+0000", 2024-09-11T07:32:09.645 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:32:09.645 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.645 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.645 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:32:09.645 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:15.794811+0000", 2024-09-11T07:32:09.645 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0194e-05 2024-09-11T07:32:09.645 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.645 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.645 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:32:09.645 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:15.794820+0000", 2024-09-11T07:32:09.645 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.8200000000000003e-06 2024-09-11T07:32:09.645 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.645 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.645 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:32:09.646 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:15.794824+0000", 2024-09-11T07:32:09.646 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.155e-06 2024-09-11T07:32:09.646 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.646 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.646 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:32:09.646 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:15.794850+0000", 2024-09-11T07:32:09.646 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.6191e-05 2024-09-11T07:32:09.646 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.646 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.646 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:32:09.646 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:15.794913+0000", 2024-09-11T07:32:09.646 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2638000000000006e-05 2024-09-11T07:32:09.646 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.646 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.647 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:32:09.647 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:15.826410+0000", 2024-09-11T07:32:09.647 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.031496903 2024-09-11T07:32:09.647 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.647 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.647 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:32:09.647 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:15.829338+0000", 2024-09-11T07:32:09.647 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0029282219999999999 2024-09-11T07:32:09.647 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.647 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.647 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:32:09.647 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:15.829362+0000", 2024-09-11T07:32:09.647 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3883000000000002e-05 2024-09-11T07:32:09.647 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.647 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.647 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:32:09.648 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:15.834598+0000", 2024-09-11T07:32:09.648 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00523575 2024-09-11T07:32:09.648 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.648 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.648 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:32:09.648 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:15.834647+0000", 2024-09-11T07:32:09.648 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9604000000000001e-05 2024-09-11T07:32:09.648 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.648 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.648 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:32:09.648 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:15.834685+0000", 2024-09-11T07:32:09.648 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7570000000000001e-05 2024-09-11T07:32:09.648 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.648 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:09.648 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.649 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.649 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.649 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:112557 7.0 e123/92 7:08b0295c:::benchmark_data_smithi022_77720_object112556:head v 123'6961, mlcod=123'6957)", 2024-09-11T07:32:09.649 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:20.636773+0000", 2024-09-11T07:32:09.649 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 108.949795552, 2024-09-11T07:32:09.649 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.036417937999999997, 2024-09-11T07:32:09.649 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:32:09.649 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:32:09.649 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:32:09.649 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.649 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:32:09.649 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:20.636773+0000", 2024-09-11T07:32:09.649 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.649 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.650 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.650 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:32:09.650 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:20.636773+0000", 2024-09-11T07:32:09.650 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.650 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.650 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.650 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:32:09.650 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:20.636771+0000", 2024-09-11T07:32:09.650 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:32:09.650 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.650 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.650 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:32:09.650 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:20.636879+0000", 2024-09-11T07:32:09.650 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000107991 2024-09-11T07:32:09.650 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.650 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.651 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:32:09.651 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:20.636903+0000", 2024-09-11T07:32:09.651 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4304e-05 2024-09-11T07:32:09.651 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.651 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.651 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:32:09.651 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:20.636909+0000", 2024-09-11T07:32:09.651 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.5140000000000001e-06 2024-09-11T07:32:09.651 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.651 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.651 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:32:09.651 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:20.650261+0000", 2024-09-11T07:32:09.651 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.013352538000000001 2024-09-11T07:32:09.651 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.651 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.651 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:32:09.652 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:20.650321+0000", 2024-09-11T07:32:09.652 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9823e-05 2024-09-11T07:32:09.652 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.652 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.652 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:32:09.652 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:20.673147+0000", 2024-09-11T07:32:09.652 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.022825532999999999 2024-09-11T07:32:09.652 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.652 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.652 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:32:09.652 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:20.673191+0000", 2024-09-11T07:32:09.652 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3992000000000002e-05 2024-09-11T07:32:09.652 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.652 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:09.653 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.653 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.653 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.653 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:117291 7.d e124/92 7:bf42c454:::benchmark_data_smithi022_77720_object117290:head v 124'7393, mlcod=124'7389)", 2024-09-11T07:32:09.653 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:25.342961+0000", 2024-09-11T07:32:09.653 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 104.24360728800001, 2024-09-11T07:32:09.653 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.038293710000000002, 2024-09-11T07:32:09.653 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:32:09.653 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:32:09.653 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:32:09.653 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.653 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:32:09.653 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.342961+0000", 2024-09-11T07:32:09.654 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.654 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.654 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.654 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:32:09.654 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.342961+0000", 2024-09-11T07:32:09.654 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.654 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.654 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.654 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:32:09.654 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.342961+0000", 2024-09-11T07:32:09.654 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:32:09.654 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.654 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.654 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:32:09.654 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.342991+0000", 2024-09-11T07:32:09.654 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0830999999999997e-05 2024-09-11T07:32:09.655 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.655 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.655 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:32:09.655 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.342997+0000", 2024-09-11T07:32:09.655 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1109999999999999e-06 2024-09-11T07:32:09.655 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.655 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.655 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:32:09.655 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.343000+0000", 2024-09-11T07:32:09.655 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8360000000000002e-06 2024-09-11T07:32:09.655 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.655 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.655 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:32:09.655 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.351247+0000", 2024-09-11T07:32:09.656 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0082463220000000004 2024-09-11T07:32:09.656 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.656 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.656 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:32:09.656 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.351621+0000", 2024-09-11T07:32:09.656 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00037435099999999998 2024-09-11T07:32:09.656 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.656 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.656 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:32:09.656 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.381157+0000", 2024-09-11T07:32:09.656 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.029536474 2024-09-11T07:32:09.656 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.656 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.657 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:32:09.657 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:25.381255+0000", 2024-09-11T07:32:09.657 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.7202999999999998e-05 2024-09-11T07:32:09.657 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.657 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:09.658 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.658 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.658 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.658 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:118447 7.0 e125/92 7:0bc48b04:::benchmark_data_smithi022_77720_object118446:head v 125'7321, mlcod=125'7317)", 2024-09-11T07:32:09.658 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:26.584762+0000", 2024-09-11T07:32:09.658 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 103.00180593899999, 2024-09-11T07:32:09.658 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.040984977999999998, 2024-09-11T07:32:09.658 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:32:09.658 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:32:09.658 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:32:09.658 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.658 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:32:09.658 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:26.584762+0000", 2024-09-11T07:32:09.658 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.659 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.659 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.659 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:32:09.659 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:26.584762+0000", 2024-09-11T07:32:09.659 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.659 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.659 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.659 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:32:09.659 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:26.584761+0000", 2024-09-11T07:32:09.659 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:32:09.659 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.659 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.659 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:32:09.660 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:26.584797+0000", 2024-09-11T07:32:09.660 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5555000000000001e-05 2024-09-11T07:32:09.660 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.660 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.660 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:32:09.660 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:26.584804+0000", 2024-09-11T07:32:09.660 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.029e-06 2024-09-11T07:32:09.660 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.660 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.660 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:32:09.660 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:26.584808+0000", 2024-09-11T07:32:09.660 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9570000000000004e-06 2024-09-11T07:32:09.660 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.661 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.661 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:32:09.661 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:26.596479+0000", 2024-09-11T07:32:09.661 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.011670699 2024-09-11T07:32:09.661 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.661 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.661 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:32:09.661 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:26.596528+0000", 2024-09-11T07:32:09.661 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9356e-05 2024-09-11T07:32:09.661 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.661 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.661 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:32:09.661 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:26.625436+0000", 2024-09-11T07:32:09.662 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.028907836999999999 2024-09-11T07:32:09.662 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.662 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.662 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:32:09.662 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:26.625747+0000", 2024-09-11T07:32:09.662 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00031142300000000002 2024-09-11T07:32:09.662 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.662 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:09.662 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.662 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.662 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.662 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:138167 7.4 e128/92 7:255cbe1f:::benchmark_data_smithi022_77720_object14412:head v 128'8718, mlcod=128'8715)", 2024-09-11T07:32:09.662 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:45.922248+0000", 2024-09-11T07:32:09.662 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 83.664320240999999, 2024-09-11T07:32:09.662 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.040840432000000003, 2024-09-11T07:32:09.663 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:32:09.663 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:32:09.663 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:32:09.663 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.663 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:32:09.663 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:45.922248+0000", 2024-09-11T07:32:09.663 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.663 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.663 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.663 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:32:09.663 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:45.922248+0000", 2024-09-11T07:32:09.663 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.663 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.663 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.664 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:32:09.664 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:45.922247+0000", 2024-09-11T07:32:09.664 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:32:09.664 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.664 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.664 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:32:09.664 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:45.922253+0000", 2024-09-11T07:32:09.664 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6570000000000002e-06 2024-09-11T07:32:09.664 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.664 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.664 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:32:09.664 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:45.922260+0000", 2024-09-11T07:32:09.664 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.6159999999999998e-06 2024-09-11T07:32:09.664 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.664 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.664 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:32:09.665 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:45.922265+0000", 2024-09-11T07:32:09.665 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.113e-06 2024-09-11T07:32:09.665 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.665 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.665 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:32:09.665 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:45.936840+0000", 2024-09-11T07:32:09.665 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.014575723 2024-09-11T07:32:09.665 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.665 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.665 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:32:09.665 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:45.936899+0000", 2024-09-11T07:32:09.665 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8276999999999998e-05 2024-09-11T07:32:09.665 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.666 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.666 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:32:09.666 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:45.962943+0000", 2024-09-11T07:32:09.666 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.026044406999999999 2024-09-11T07:32:09.666 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.666 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.666 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:32:09.666 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:45.963088+0000", 2024-09-11T07:32:09.666 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014534699999999999 2024-09-11T07:32:09.666 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.666 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:09.666 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.666 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.666 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.667 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:160152 7.b 7:dbc43886:::benchmark_data_smithi022_77720_object36397:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-11T07:32:09.667 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:31:07.989636+0000", 2024-09-11T07:32:09.667 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 61.596931843999997, 2024-09-11T07:32:09.667 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.039369657000000002, 2024-09-11T07:32:09.667 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:32:09.667 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:32:09.667 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:32:09.667 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:32:09.667 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:32:09.667 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 160152 2024-09-11T07:32:09.667 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.667 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:32:09.667 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.667 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:32:09.667 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:07.989636+0000", 2024-09-11T07:32:09.668 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.668 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.668 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.668 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:32:09.668 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:07.989636+0000", 2024-09-11T07:32:09.668 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.668 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.668 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.668 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:32:09.668 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:07.989634+0000", 2024-09-11T07:32:09.668 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:32:09.668 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.668 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.668 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:32:09.668 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:07.989643+0000", 2024-09-11T07:32:09.669 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.0259999999999996e-06 2024-09-11T07:32:09.669 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.669 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.669 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:32:09.669 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:07.989661+0000", 2024-09-11T07:32:09.669 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.7839999999999999e-05 2024-09-11T07:32:09.669 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.669 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.669 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:32:09.669 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:07.989668+0000", 2024-09-11T07:32:09.669 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.4209999999999998e-06 2024-09-11T07:32:09.669 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.669 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.669 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:32:09.670 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:07.989762+0000", 2024-09-11T07:32:09.670 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.3602000000000003e-05 2024-09-11T07:32:09.670 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.670 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.670 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:32:09.670 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:07.989883+0000", 2024-09-11T07:32:09.670 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000121537 2024-09-11T07:32:09.670 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.670 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.670 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:32:09.670 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:07.990246+0000", 2024-09-11T07:32:09.670 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00036239600000000002 2024-09-11T07:32:09.670 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.670 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.671 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:32:09.671 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:07.996179+0000", 2024-09-11T07:32:09.671 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0059331970000000003 2024-09-11T07:32:09.671 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.671 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.671 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:32:09.671 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:07.996203+0000", 2024-09-11T07:32:09.671 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4454e-05 2024-09-11T07:32:09.671 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.671 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.671 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:32:09.671 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:08.028303+0000", 2024-09-11T07:32:09.671 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.032099654999999998 2024-09-11T07:32:09.671 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.672 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.672 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:32:09.672 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:08.028482+0000", 2024-09-11T07:32:09.672 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00017900500000000001 2024-09-11T07:32:09.672 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.672 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.672 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:32:09.672 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:08.029006+0000", 2024-09-11T07:32:09.672 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00052393399999999999 2024-09-11T07:32:09.672 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.672 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:09.672 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.672 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.672 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.673 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:172739 7.d e138/92 7:bb785fd0:::benchmark_data_smithi022_77720_object48984:head v 138'10943, mlcod=138'10938)", 2024-09-11T07:32:09.673 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:31:19.814435+0000", 2024-09-11T07:32:09.673 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 49.772132796000001, 2024-09-11T07:32:09.673 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.039398298999999998, 2024-09-11T07:32:09.673 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:32:09.673 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:32:09.673 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:32:09.673 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.673 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:32:09.673 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:19.814435+0000", 2024-09-11T07:32:09.673 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.673 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.674 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.674 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:32:09.674 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:19.814435+0000", 2024-09-11T07:32:09.674 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.674 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.674 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.674 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:32:09.674 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:19.814434+0000", 2024-09-11T07:32:09.674 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:32:09.674 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.674 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.674 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:32:09.674 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:19.814441+0000", 2024-09-11T07:32:09.674 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.8369999999999999e-06 2024-09-11T07:32:09.675 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.675 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.675 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:32:09.675 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:19.814451+0000", 2024-09-11T07:32:09.675 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0087e-05 2024-09-11T07:32:09.675 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.675 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.675 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:32:09.675 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:19.814457+0000", 2024-09-11T07:32:09.675 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8409999999999996e-06 2024-09-11T07:32:09.675 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.675 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.675 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:32:09.675 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:19.829612+0000", 2024-09-11T07:32:09.676 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.015155462999999999 2024-09-11T07:32:09.676 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.676 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.676 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:32:09.676 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:19.829631+0000", 2024-09-11T07:32:09.676 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8674000000000001e-05 2024-09-11T07:32:09.676 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.677 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.677 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:32:09.677 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:19.853795+0000", 2024-09-11T07:32:09.677 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.024163724000000001 2024-09-11T07:32:09.677 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.677 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.677 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:32:09.677 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:19.853834+0000", 2024-09-11T07:32:09.677 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8927999999999997e-05 2024-09-11T07:32:09.677 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.677 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:09.678 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.678 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.678 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.678 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:172744 7.d e138/92 7:bab47dd3:::benchmark_data_smithi022_77720_object48989:head v 138'10944, mlcod=138'10938)", 2024-09-11T07:32:09.678 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:31:19.817085+0000", 2024-09-11T07:32:09.678 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 49.769483557000001, 2024-09-11T07:32:09.678 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.036773608999999999, 2024-09-11T07:32:09.678 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:32:09.678 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:32:09.678 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:32:09.678 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.678 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:32:09.678 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:19.817085+0000", 2024-09-11T07:32:09.678 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.679 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.679 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.679 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:32:09.679 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:19.817085+0000", 2024-09-11T07:32:09.679 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.679 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.679 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.679 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:32:09.679 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:19.817084+0000", 2024-09-11T07:32:09.679 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:32:09.679 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.679 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.679 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:32:09.679 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:19.817090+0000", 2024-09-11T07:32:09.680 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5549999999999999e-06 2024-09-11T07:32:09.680 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.680 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.680 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:32:09.680 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:19.817101+0000", 2024-09-11T07:32:09.680 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0535e-05 2024-09-11T07:32:09.680 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.680 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.680 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:32:09.680 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:19.817105+0000", 2024-09-11T07:32:09.680 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4950000000000002e-06 2024-09-11T07:32:09.680 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.680 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.681 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:32:09.681 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:19.841265+0000", 2024-09-11T07:32:09.681 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.024159217 2024-09-11T07:32:09.681 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.681 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.681 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:32:09.681 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:19.841283+0000", 2024-09-11T07:32:09.681 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8014999999999999e-05 2024-09-11T07:32:09.681 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.681 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.681 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:32:09.681 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:19.853837+0000", 2024-09-11T07:32:09.681 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.012554223 2024-09-11T07:32:09.681 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.682 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.682 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:32:09.682 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:19.853858+0000", 2024-09-11T07:32:09.682 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1579000000000001e-05 2024-09-11T07:32:09.682 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.682 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:09.682 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.682 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.682 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.682 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:189097 7.d e141/92 7:bbc5905a:::benchmark_data_smithi022_77720_object65342:head v 141'11999, mlcod=141'11996)", 2024-09-11T07:32:09.682 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:31:35.610667+0000", 2024-09-11T07:32:09.682 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 33.975901581999999, 2024-09-11T07:32:09.682 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.035686172000000002, 2024-09-11T07:32:09.682 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:32:09.682 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:32:09.683 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:32:09.683 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.683 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:32:09.683 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:35.610667+0000", 2024-09-11T07:32:09.683 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.683 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.683 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.683 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:32:09.683 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:35.610667+0000", 2024-09-11T07:32:09.683 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.683 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.683 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.683 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:32:09.683 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:35.610666+0000", 2024-09-11T07:32:09.684 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:32:09.684 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.684 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.684 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:32:09.684 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:35.610671+0000", 2024-09-11T07:32:09.684 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2510000000000002e-06 2024-09-11T07:32:09.684 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.684 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.684 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:32:09.684 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:35.610682+0000", 2024-09-11T07:32:09.684 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0530000000000001e-05 2024-09-11T07:32:09.684 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.684 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.684 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:32:09.685 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:35.610687+0000", 2024-09-11T07:32:09.685 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3199999999999999e-06 2024-09-11T07:32:09.685 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.685 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.685 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:32:09.685 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:35.621715+0000", 2024-09-11T07:32:09.685 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.011028513 2024-09-11T07:32:09.685 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.685 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.685 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:32:09.685 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:35.621773+0000", 2024-09-11T07:32:09.685 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7499e-05 2024-09-11T07:32:09.685 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.685 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.686 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:32:09.686 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:35.646240+0000", 2024-09-11T07:32:09.687 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.024466964000000001 2024-09-11T07:32:09.687 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.687 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.687 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:32:09.687 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:35.646353+0000", 2024-09-11T07:32:09.687 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000112952 2024-09-11T07:32:09.687 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.687 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:09.687 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.687 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.687 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.687 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:198342 7.b 7:d6268fd3:::benchmark_data_smithi022_77720_object74587:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e143)", 2024-09-11T07:32:09.687 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:31:45.227094+0000", 2024-09-11T07:32:09.688 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 24.359474064, 2024-09-11T07:32:09.688 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.035733823999999997, 2024-09-11T07:32:09.688 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:32:09.688 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:32:09.688 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:32:09.688 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:32:09.688 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:32:09.688 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 198342 2024-09-11T07:32:09.688 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.688 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:32:09.688 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.688 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:32:09.688 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:45.227094+0000", 2024-09-11T07:32:09.689 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.689 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.689 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.689 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:32:09.689 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:45.227094+0000", 2024-09-11T07:32:09.689 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.689 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.689 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.689 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:32:09.689 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:45.227092+0000", 2024-09-11T07:32:09.689 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:32:09.689 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.689 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.689 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:32:09.690 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:45.227099+0000", 2024-09-11T07:32:09.690 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4829999999999996e-06 2024-09-11T07:32:09.690 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.690 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.690 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:32:09.690 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:45.227113+0000", 2024-09-11T07:32:09.690 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.384e-05 2024-09-11T07:32:09.690 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.690 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.690 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:32:09.690 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:45.227119+0000", 2024-09-11T07:32:09.690 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.6950000000000003e-06 2024-09-11T07:32:09.690 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.690 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.691 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:32:09.691 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:45.238043+0000", 2024-09-11T07:32:09.691 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.010924067000000001 2024-09-11T07:32:09.691 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.691 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.691 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:32:09.691 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:45.238142+0000", 2024-09-11T07:32:09.691 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.9170000000000001e-05 2024-09-11T07:32:09.691 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.691 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.691 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:32:09.691 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:45.238246+0000", 2024-09-11T07:32:09.691 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000103397 2024-09-11T07:32:09.691 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.691 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.692 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:32:09.692 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:45.256868+0000", 2024-09-11T07:32:09.692 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.018621841 2024-09-11T07:32:09.692 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.692 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.692 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:32:09.692 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:45.256935+0000", 2024-09-11T07:32:09.692 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7036e-05 2024-09-11T07:32:09.692 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.692 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.692 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:32:09.692 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:45.262665+0000", 2024-09-11T07:32:09.692 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0057305350000000001 2024-09-11T07:32:09.692 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.693 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.693 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:32:09.693 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:45.262780+0000", 2024-09-11T07:32:09.693 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000115081 2024-09-11T07:32:09.693 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.693 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.693 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:32:09.693 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:45.262828+0000", 2024-09-11T07:32:09.693 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7685999999999997e-05 2024-09-11T07:32:09.693 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.693 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:09.693 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.693 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.694 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.694 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:218748 7.b 7:dbb6ec79:::benchmark_data_smithi022_77720_object94993:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-11T07:32:09.694 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:08.100930+0000", 2024-09-11T07:32:09.694 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 1.485637809, 2024-09-11T07:32:09.694 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.040674676999999999, 2024-09-11T07:32:09.694 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:32:09.694 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:32:09.694 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:32:09.694 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:32:09.694 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:32:09.694 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 218748 2024-09-11T07:32:09.694 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.694 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:32:09.694 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.694 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:32:09.695 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:08.100930+0000", 2024-09-11T07:32:09.695 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.695 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.695 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.695 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:32:09.695 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:08.100930+0000", 2024-09-11T07:32:09.695 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:09.695 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.695 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.695 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:32:09.695 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:08.100930+0000", 2024-09-11T07:32:09.695 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:32:09.695 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.695 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.696 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:32:09.696 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:08.100933+0000", 2024-09-11T07:32:09.696 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0529999999999998e-06 2024-09-11T07:32:09.696 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.696 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.696 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:32:09.696 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:08.100938+0000", 2024-09-11T07:32:09.696 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9529999999999998e-06 2024-09-11T07:32:09.696 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.696 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.696 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:32:09.696 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:08.100942+0000", 2024-09-11T07:32:09.696 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7689999999999998e-06 2024-09-11T07:32:09.696 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.697 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.697 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:32:09.697 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:08.107620+0000", 2024-09-11T07:32:09.697 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0066783329999999998 2024-09-11T07:32:09.697 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.697 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.697 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:32:09.697 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:08.107756+0000", 2024-09-11T07:32:09.697 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013651200000000001 2024-09-11T07:32:09.697 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.697 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.697 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:32:09.697 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:08.107884+0000", 2024-09-11T07:32:09.698 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000127432 2024-09-11T07:32:09.698 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.698 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.698 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:32:09.698 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:08.111960+0000", 2024-09-11T07:32:09.698 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.004076049 2024-09-11T07:32:09.698 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.698 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.698 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:32:09.698 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:08.111990+0000", 2024-09-11T07:32:09.698 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.9652e-05 2024-09-11T07:32:09.698 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.698 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.698 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:32:09.698 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:08.141472+0000", 2024-09-11T07:32:09.699 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.029482745000000001 2024-09-11T07:32:09.699 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.699 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.699 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:32:09.699 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:08.141564+0000", 2024-09-11T07:32:09.699 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.1494000000000002e-05 2024-09-11T07:32:09.699 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:09.699 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:09.699 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:32:09.699 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:08.141605+0000", 2024-09-11T07:32:09.699 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1300000000000001e-05 2024-09-11T07:32:09.699 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.699 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:09.699 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.699 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:09.700 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:09.700 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:32:09.710 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:09.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:09.711 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:09.811 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:09.837 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:09.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:09.839 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:09.940 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:09.944 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T07:32:09.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:09.979 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:09.979 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:09.987 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:32:09.987 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:32:09.987 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:32:09.987 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:32:09.987 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:32:09.988 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:32:09.988 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:32:09.988 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:32:09.988 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:32:09.988 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:32:09.998 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:32:09.998 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:32:10.038 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:32:10.038 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:32:10.066 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:32:10.066 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:32:10.079 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:10.106 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:10.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:10.107 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:10.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:10 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:10.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:10 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4182974961' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:32:10.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:10 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:10.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:10 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4182974961' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:32:10.208 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:10.233 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:10.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:10.234 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:10.334 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:10.367 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:10.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:10.368 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:10.469 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:10.502 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:10.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:10.503 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:10.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:10 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:10.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:10 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4182974961' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:32:10.603 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:10.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:10.653 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:10.653 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:10.671 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:32:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:32:10] "GET /metrics HTTP/1.1" 200 223782 "" "Prometheus/2.43.0" 2024-09-11T07:32:10.755 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:10.797 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:10.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:10.798 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:10.899 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:10.932 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:10.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:10.933 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:11.033 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:11.064 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:11.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:11.065 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:11.166 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:11.201 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:11.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:11.202 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:11.303 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:11.335 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:11.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:11.336 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:11.382 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:11 smithi022 ceph-mon[29462]: pgmap v233: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 1.7 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 912 op/s; 7265/83718 objects degraded (8.678%); 0 B/s, 152 objects/s recovering 2024-09-11T07:32:11.382 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:11 smithi022 ceph-mon[29462]: Health check update: Degraded data redundancy: 7265/97152 objects degraded (7.478%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:32:11.382 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:11 smithi022 ceph-mon[37330]: pgmap v233: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 1.7 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 912 op/s; 7265/83718 objects degraded (8.678%); 0 B/s, 152 objects/s recovering 2024-09-11T07:32:11.382 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:11 smithi022 ceph-mon[37330]: Health check update: Degraded data redundancy: 7265/97152 objects degraded (7.478%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:32:11.436 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:11.477 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:11.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:11.478 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:11.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:11 smithi104 ceph-mon[33507]: pgmap v233: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 1.7 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 912 op/s; 7265/83718 objects degraded (8.678%); 0 B/s, 152 objects/s recovering 2024-09-11T07:32:11.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:11 smithi104 ceph-mon[33507]: Health check update: Degraded data redundancy: 7265/97152 objects degraded (7.478%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:32:11.579 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:11.609 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:11.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:11.610 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:11.701 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-11T07:32:11.711 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:11.785 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:11.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:11.786 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:11.887 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:11.916 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:11.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:11.918 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:12.018 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:12.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:12.080 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:12.080 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:12.181 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:12.211 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:12.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:12.212 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:12.313 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:12.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:12.345 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:12.346 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:12.446 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:12.478 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:12.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:12.479 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:12.579 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:12.619 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:12.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:12.620 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:12.721 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:12.751 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:12.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:12.752 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:12.853 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:12.882 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:12.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:12.883 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:12.984 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:13.003 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:32:13.069 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:13.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:13.070 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:13.170 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:13.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:13.241 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:13.241 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:13.263 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:13 smithi022 ceph-mon[29462]: pgmap v234: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 1.7 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 815 op/s; 7265/83718 objects degraded (8.678%); 0 B/s, 136 objects/s recovering 2024-09-11T07:32:13.263 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:13 smithi022 ceph-mon[37330]: pgmap v234: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 1.7 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 815 op/s; 7265/83718 objects degraded (8.678%); 0 B/s, 136 objects/s recovering 2024-09-11T07:32:13.343 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:13.364 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:13.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:13.365 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:13.426 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:32:13.466 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:13.513 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:13.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:13.514 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:13.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:13 smithi104 ceph-mon[33507]: pgmap v234: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 1.7 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 815 op/s; 7265/83718 objects degraded (8.678%); 0 B/s, 136 objects/s recovering 2024-09-11T07:32:13.615 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:13.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:13.678 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:13.678 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:13.779 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:13.822 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:13.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:13.823 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:13.924 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:13.953 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:13.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:13.954 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:14.055 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:14.099 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:14.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:14.100 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:14.201 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:14.233 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:14.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:14.234 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:14.335 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:14.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:14.386 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:14.386 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:14.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:14 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:32:14.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:14 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:32:14.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:14 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:32:14.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:14 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:32:14.487 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:14.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:32:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:32:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:32:14.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:14 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:32:14.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:14 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:32:14.532 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:14.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:14.533 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:14.634 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:14.676 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:14.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:14.677 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:14.777 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:14.809 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:14.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:14.810 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:14.911 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:14.955 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:14.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:14.956 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:15.057 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:15.091 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:15.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:15.092 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:15.193 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:15.233 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:15.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:15.234 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:15.335 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:15.361 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:15.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:15.362 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:15.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:15 smithi022 ceph-mon[29462]: pgmap v235: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 1.6 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1022 B/s rd, 0 B/s wr, 780 op/s; 7265/79563 objects degraded (9.131%); 0 B/s, 151 objects/s recovering 2024-09-11T07:32:15.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:15 smithi022 ceph-mon[29462]: osdmap e152: 8 total, 8 up, 8 in 2024-09-11T07:32:15.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:15 smithi022 ceph-mon[37330]: pgmap v235: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 1.6 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1022 B/s rd, 0 B/s wr, 780 op/s; 7265/79563 objects degraded (9.131%); 0 B/s, 151 objects/s recovering 2024-09-11T07:32:15.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:15 smithi022 ceph-mon[37330]: osdmap e152: 8 total, 8 up, 8 in 2024-09-11T07:32:15.463 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:15.493 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:15.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:15.494 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:15.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:15 smithi104 ceph-mon[33507]: pgmap v235: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 1.6 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1022 B/s rd, 0 B/s wr, 780 op/s; 7265/79563 objects degraded (9.131%); 0 B/s, 151 objects/s recovering 2024-09-11T07:32:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:15 smithi104 ceph-mon[33507]: osdmap e152: 8 total, 8 up, 8 in 2024-09-11T07:32:15.595 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:15.738 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:15.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:15.739 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:15.840 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:15.872 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:15.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:15.873 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:15.974 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:16.007 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:16.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:16.008 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:16.109 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:16.138 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:16.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:16.139 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:16.240 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:16.265 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:16.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:16.266 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:16.367 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:16.395 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:16.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:16.397 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:16.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:16 smithi022 ceph-mon[29462]: Health check update: Degraded data redundancy: 7265/79563 objects degraded (9.131%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:32:16.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:16 smithi022 ceph-mon[37330]: Health check update: Degraded data redundancy: 7265/79563 objects degraded (9.131%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:32:16.498 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:16.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:16 smithi104 ceph-mon[33507]: Health check update: Degraded data redundancy: 7265/79563 objects degraded (9.131%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:32:16.537 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:16.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:16.539 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:16.640 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:16.662 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:16.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:16.663 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:16.764 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:16.803 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:16.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:16.804 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:16.905 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:16.933 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:16.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:16.934 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:17.035 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:17.056 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:17.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:17.057 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:17.158 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:17.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:17.198 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:17.198 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:17.299 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:17.320 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:17.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:17.321 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:17.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:17 smithi022 ceph-mon[29462]: pgmap v237: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 1.4 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 894 op/s; 7265/70320 objects degraded (10.331%); 0 B/s, 180 objects/s recovering 2024-09-11T07:32:17.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:17 smithi022 ceph-mon[37330]: pgmap v237: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 1.4 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 894 op/s; 7265/70320 objects degraded (10.331%); 0 B/s, 180 objects/s recovering 2024-09-11T07:32:17.421 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:17.461 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:17.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:17.462 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:17.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:17 smithi104 ceph-mon[33507]: pgmap v237: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 1.4 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 894 op/s; 7265/70320 objects degraded (10.331%); 0 B/s, 180 objects/s recovering 2024-09-11T07:32:17.563 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:17.584 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:17.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:17.585 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:17.686 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:17.713 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:17.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:17.715 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:17.815 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:17.858 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:17.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:17.859 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:17.960 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:17.984 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:17.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:17.985 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:18.086 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:18.127 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:18.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:18.128 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:18.213 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:32:18.229 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:18.282 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:18.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:18.283 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:18.383 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:18.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:18.405 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:18.406 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:18.507 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:18.550 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:18.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:18.551 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:18.652 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:18.677 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:18.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:18.678 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:18.779 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:18.820 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:18.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:18.821 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:18.922 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:18.955 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:18.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:18.956 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:19.057 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:19.085 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:19.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:19.087 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:19.187 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:19.223 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:19.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:19.224 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:19.324 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:19.358 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:19.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:19.360 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:19.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:19 smithi022 ceph-mon[29462]: pgmap v238: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 1.4 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 894 op/s; 7265/70320 objects degraded (10.331%); 0 B/s, 180 objects/s recovering 2024-09-11T07:32:19.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:19 smithi022 ceph-mon[37330]: pgmap v238: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 1.4 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 894 op/s; 7265/70320 objects degraded (10.331%); 0 B/s, 180 objects/s recovering 2024-09-11T07:32:19.461 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:19.505 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:19.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:19.508 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:19.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:32:19 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:32:19.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:19 smithi104 ceph-mon[33507]: pgmap v238: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 1.4 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 894 op/s; 7265/70320 objects degraded (10.331%); 0 B/s, 180 objects/s recovering 2024-09-11T07:32:19.609 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:19.636 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:19.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:19.637 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:19.738 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:19.764 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:19.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:19.765 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:19.866 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:19.889 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:19.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:19.890 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:19.991 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:20.020 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:20.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:20.021 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:20.122 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:20.154 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:20.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:20.156 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:20.257 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:20.303 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:20.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:20.304 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:20.404 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:20.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:20 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:20.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:20 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/745287607' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T07:32:20.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:20 smithi022 ceph-mon[29462]: from='client.24962 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T07:32:20.417 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:32:20 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:32:20] "GET /metrics HTTP/1.1" 200 223782 "" "Prometheus/2.43.0" 2024-09-11T07:32:20.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:20 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:20.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:20 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/745287607' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T07:32:20.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:20 smithi022 ceph-mon[37330]: from='client.24962 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T07:32:20.440 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:20.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:20.442 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:20.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:20 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:20.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:20 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/745287607' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T07:32:20.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:20 smithi104 ceph-mon[33507]: from='client.24962 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T07:32:20.542 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:20.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:20.575 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:20.575 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:20.632 INFO:teuthology.orchestra.run.smithi022.stderr:reweighted osd.0 to 1 (10000) 2024-09-11T07:32:20.676 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:20.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:20.706 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:20.706 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:20.807 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:20.835 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:20.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:20.836 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:20.937 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:20.973 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:20.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:20.974 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:21.075 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:21.104 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:21.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:21.105 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:21.206 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:21.237 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:21.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:21.238 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:21.339 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:21.377 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:21.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:21.378 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:21.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:21 smithi022 ceph-mon[29462]: pgmap v239: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 1.2 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 850 op/s; 7265/58221 objects degraded (12.478%); 0 B/s, 198 objects/s recovering 2024-09-11T07:32:21.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:21 smithi022 ceph-mon[29462]: from='client.24962 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-11T07:32:21.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:21 smithi022 ceph-mon[29462]: osdmap e153: 8 total, 8 up, 8 in 2024-09-11T07:32:21.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:21 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:32:21.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:21 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:32:21.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:21 smithi022 ceph-mon[29462]: Health check update: Degraded data redundancy: 7265/58221 objects degraded (12.478%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:32:21.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:21 smithi022 ceph-mon[37330]: pgmap v239: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 1.2 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 850 op/s; 7265/58221 objects degraded (12.478%); 0 B/s, 198 objects/s recovering 2024-09-11T07:32:21.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:21 smithi022 ceph-mon[37330]: from='client.24962 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-11T07:32:21.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:21 smithi022 ceph-mon[37330]: osdmap e153: 8 total, 8 up, 8 in 2024-09-11T07:32:21.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:21 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:32:21.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:21 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:32:21.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:21 smithi022 ceph-mon[37330]: Health check update: Degraded data redundancy: 7265/58221 objects degraded (12.478%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:32:21.479 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:21.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:21.508 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:21.508 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:21.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:21 smithi104 ceph-mon[33507]: pgmap v239: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 1.2 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 850 op/s; 7265/58221 objects degraded (12.478%); 0 B/s, 198 objects/s recovering 2024-09-11T07:32:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:21 smithi104 ceph-mon[33507]: from='client.24962 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-11T07:32:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:21 smithi104 ceph-mon[33507]: osdmap e153: 8 total, 8 up, 8 in 2024-09-11T07:32:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:21 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:32:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:21 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:32:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:21 smithi104 ceph-mon[33507]: Health check update: Degraded data redundancy: 7265/58221 objects degraded (12.478%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:32:21.609 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:21.654 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:21.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:21.655 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:21.756 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:21.795 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:21.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:21.796 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:21.897 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:21.941 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:21.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:21.942 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:22.042 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:22.070 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:22.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:22.071 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:22.172 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:22.219 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:22.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:22.220 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:22.321 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:22.362 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:22.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:22.363 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:22.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:22 smithi104 ceph-mon[33507]: osdmap e154: 8 total, 8 up, 8 in 2024-09-11T07:32:22.463 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:22.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:22.502 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:22.503 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:22.603 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:22.640 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:22.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:22.641 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:22.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:22 smithi022 ceph-mon[29462]: osdmap e154: 8 total, 8 up, 8 in 2024-09-11T07:32:22.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:22 smithi022 ceph-mon[37330]: osdmap e154: 8 total, 8 up, 8 in 2024-09-11T07:32:22.742 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:22.765 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:22.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:22.766 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:22.867 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:22.909 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:22.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:22.910 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:23.011 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:23.040 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:23.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:23.042 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:23.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:23.177 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:23.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:23.179 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:23.279 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:23.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:23 smithi104 ceph-mon[33507]: pgmap v242: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 1.2 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 897 op/s; 7265/58221 objects degraded (12.478%); 0 B/s, 192 objects/s recovering 2024-09-11T07:32:23.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:32:23.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:23.317 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:23.317 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:23.418 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:23.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:23.456 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:23.456 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:23.557 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:23.591 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:23.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:23.592 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:23 smithi022 ceph-mon[29462]: pgmap v242: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 1.2 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 897 op/s; 7265/58221 objects degraded (12.478%); 0 B/s, 192 objects/s recovering 2024-09-11T07:32:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:32:23.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:23 smithi022 ceph-mon[37330]: pgmap v242: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 1.2 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 897 op/s; 7265/58221 objects degraded (12.478%); 0 B/s, 192 objects/s recovering 2024-09-11T07:32:23.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:32:23.693 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:23.716 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:23.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:23.717 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:23.818 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:23.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:23.843 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:23.843 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:23.944 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:23.986 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:23.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:23.987 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:24.088 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:24.126 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:24.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:24.127 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:24.227 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:24.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:24.253 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:24.253 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:24.354 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:24.357 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:32:24 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:32:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:32:24.390 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:24.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:24.392 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:24.492 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:24.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:24.529 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:24.530 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:24.630 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:24.673 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:24.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:24.675 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:24.775 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:24.806 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:24.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:24.807 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:24.907 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:24.939 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:24.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:24.940 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:25.040 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:25.062 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:25.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:25.063 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:25.164 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:25.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:25.204 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:25.204 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:25.305 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:25.333 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:25.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:25.334 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:25.435 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:25.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:25.466 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:25.466 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:25.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:25 smithi104 ceph-mon[33507]: pgmap v243: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 1.1 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 572 op/s; 7265/56601 objects degraded (12.835%); 0 B/s, 133 objects/s recovering 2024-09-11T07:32:25.567 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:25.589 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:25.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:25.590 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:25.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:25 smithi022 ceph-mon[29462]: pgmap v243: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 1.1 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 572 op/s; 7265/56601 objects degraded (12.835%); 0 B/s, 133 objects/s recovering 2024-09-11T07:32:25.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:25 smithi022 ceph-mon[37330]: pgmap v243: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 1.1 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 572 op/s; 7265/56601 objects degraded (12.835%); 0 B/s, 133 objects/s recovering 2024-09-11T07:32:25.691 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:25.731 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:25.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:25.732 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:25.833 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:25.855 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:25.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:25.856 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:25.957 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:25.985 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:25.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:25.986 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:26.087 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:26.124 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:26.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:26.125 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:26.226 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:26.269 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:26.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:26.270 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:26.371 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:26.404 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:26.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:26.406 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:26.446 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight 1 1 2024-09-11T07:32:26.507 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:26.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:26 smithi104 ceph-mon[33507]: Health check update: Degraded data redundancy: 7265/56601 objects degraded (12.835%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:32:26.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:26 smithi022 ceph-mon[29462]: Health check update: Degraded data redundancy: 7265/56601 objects degraded (12.835%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:32:26.532 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:26 smithi022 ceph-mon[37330]: Health check update: Degraded data redundancy: 7265/56601 objects degraded (12.835%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:32:26.536 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:26.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:26.537 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:26.638 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:26.690 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:26.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:26.691 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:26.792 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:26.818 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:26.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:26.819 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:26.920 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:26.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:26.975 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:26.976 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:27.076 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:27.124 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:27.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:27.125 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:27.226 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:27.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:27.266 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:27.266 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:27.367 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:27.405 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:27.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:27.406 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:27.506 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:27.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:27 smithi104 ceph-mon[33507]: pgmap v244: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 928 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.05k op/s; 7265/45114 objects degraded (16.104%); 0 B/s, 278 objects/s recovering 2024-09-11T07:32:27.537 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:27.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:27.538 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:27.539 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:27 smithi022 ceph-mon[29462]: pgmap v244: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 928 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.05k op/s; 7265/45114 objects degraded (16.104%); 0 B/s, 278 objects/s recovering 2024-09-11T07:32:27.539 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:27 smithi022 ceph-mon[37330]: pgmap v244: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 928 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.05k op/s; 7265/45114 objects degraded (16.104%); 0 B/s, 278 objects/s recovering 2024-09-11T07:32:27.639 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:27.680 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:27.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:27.681 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:27.782 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:27.808 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:27.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:27.809 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:27.910 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:27.935 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:27.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:27.936 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:28.037 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:28.066 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:28.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:28.067 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:28.168 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:28.207 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:28.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:28.209 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:28.309 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:28.339 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:28.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:28.340 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:28.441 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:28.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:28.487 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:28.488 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:28.588 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:28.633 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:28.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:28.634 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:28.700 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:32:28.735 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:28.761 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:32:28.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:28.762 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:28.762 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:28.863 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:28.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:28.892 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:28.893 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:28.994 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:29.023 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:29.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:29.024 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:29.125 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:29.174 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:29.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:29.175 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:29.276 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:29.312 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:29.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:29.313 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:29.414 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:29.459 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:29.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:29.460 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:29.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:32:29 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:32:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:29 smithi104 ceph-mon[33507]: pgmap v245: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 928 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 546 op/s; 7265/45114 objects degraded (16.104%); 0 B/s, 144 objects/s recovering 2024-09-11T07:32:29.561 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:29.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:29.606 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:29.607 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:29.649 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:29 smithi022 ceph-mon[29462]: pgmap v245: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 928 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 546 op/s; 7265/45114 objects degraded (16.104%); 0 B/s, 144 objects/s recovering 2024-09-11T07:32:29.650 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:29 smithi022 ceph-mon[37330]: pgmap v245: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 928 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 546 op/s; 7265/45114 objects degraded (16.104%); 0 B/s, 144 objects/s recovering 2024-09-11T07:32:29.708 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:29.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:29.741 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:29.741 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:29.842 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:29.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:29.889 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:29.889 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:29.990 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:30.027 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:30.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:30.028 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:30.129 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:30.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:30.176 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:30.176 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:30.277 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:30.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:30.313 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:30.313 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:30.414 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:30.417 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:32:30 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:32:30] "GET /metrics HTTP/1.1" 200 223877 "" "Prometheus/2.43.0" 2024-09-11T07:32:30.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:30 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:30.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:30 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:30.455 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:30.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:30.456 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:30.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:30 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:30.557 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:30.589 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:30.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:30.590 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:30.690 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:30.718 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:30.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:30.719 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:30.820 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:30.863 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:30.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:30.864 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:30.965 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:30.996 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:30.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:30.997 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:31.098 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:31.138 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:31.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:31.139 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:31.240 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:31.266 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:31.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:31.267 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:31.368 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:31.409 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:31.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:31.410 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:31.511 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:31.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:31 smithi104 ceph-mon[33507]: pgmap v246: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 651 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 887 op/s; 7265/31815 objects degraded (22.835%); 0 B/s, 264 objects/s recovering 2024-09-11T07:32:31.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:31 smithi104 ceph-mon[33507]: Health check update: Degraded data redundancy: 7265/31815 objects degraded (22.835%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:32:31.545 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:31.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:31.546 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:31.647 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:31.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:31 smithi022 ceph-mon[29462]: pgmap v246: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 651 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 887 op/s; 7265/31815 objects degraded (22.835%); 0 B/s, 264 objects/s recovering 2024-09-11T07:32:31.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:31 smithi022 ceph-mon[29462]: Health check update: Degraded data redundancy: 7265/31815 objects degraded (22.835%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:32:31.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:31 smithi022 ceph-mon[37330]: pgmap v246: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 651 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 887 op/s; 7265/31815 objects degraded (22.835%); 0 B/s, 264 objects/s recovering 2024-09-11T07:32:31.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:31 smithi022 ceph-mon[37330]: Health check update: Degraded data redundancy: 7265/31815 objects degraded (22.835%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:32:31.674 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:31.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:31.676 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:31.776 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:31.798 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:31.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:31.799 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:31.900 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:31.922 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:31.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:31.923 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:32.023 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:32.049 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:32.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:32.053 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:32.153 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:32.175 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:32.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:32.176 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:32.277 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:32.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:32.303 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:32.303 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:32.404 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:32.432 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:32.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:32.433 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:32.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:32 smithi104 ceph-mon[33507]: osdmap e155: 8 total, 8 up, 8 in 2024-09-11T07:32:32.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:32 smithi104 ceph-mon[33507]: osdmap e156: 8 total, 8 up, 8 in 2024-09-11T07:32:32.534 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:32.560 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:32.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:32.561 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:32.662 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:32.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:32 smithi022 ceph-mon[29462]: osdmap e155: 8 total, 8 up, 8 in 2024-09-11T07:32:32.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:32 smithi022 ceph-mon[29462]: osdmap e156: 8 total, 8 up, 8 in 2024-09-11T07:32:32.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:32 smithi022 ceph-mon[37330]: osdmap e155: 8 total, 8 up, 8 in 2024-09-11T07:32:32.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:32 smithi022 ceph-mon[37330]: osdmap e156: 8 total, 8 up, 8 in 2024-09-11T07:32:32.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:32.700 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:32.700 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:32.801 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:32.851 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:32.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:32.852 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:32.953 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:32.998 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:32.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:33.000 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:33.101 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:33.126 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:33.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:33.128 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:33.229 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:33.253 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:33.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:33.254 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:33.355 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:33.384 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:33.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:33.385 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:33.394 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:33 smithi022 ceph-mon[29462]: pgmap v248: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 651 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 880 op/s; 7265/31815 objects degraded (22.835%); 0 B/s, 262 objects/s recovering 2024-09-11T07:32:33.394 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:33 smithi022 ceph-mon[37330]: pgmap v248: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 651 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 880 op/s; 7265/31815 objects degraded (22.835%); 0 B/s, 262 objects/s recovering 2024-09-11T07:32:33.486 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:33.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:33 smithi104 ceph-mon[33507]: pgmap v248: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 651 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 880 op/s; 7265/31815 objects degraded (22.835%); 0 B/s, 262 objects/s recovering 2024-09-11T07:32:33.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:33.532 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:33.532 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:33.599 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:32:33.599 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:32:33.633 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:33.634 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:32:33.634 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:32:33.663 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:33.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:33.664 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:33.672 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:32:33.673 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:32:33.713 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:32:33.713 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:32:33.757 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:32:33.757 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:32:33.765 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:33.790 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:33.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:33.791 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:33.793 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:32:33.793 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:32:33.806 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:32:33.806 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:32:33.806 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.806 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:240480 7.5 e156/148 7:a3389738:::benchmark_data_smithi022_77720_object116725:head v 156'15319, mlcod=135'9932)", 2024-09-11T07:32:33.806 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:33.801053+0000", 2024-09-11T07:32:33.806 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0025862289999999998, 2024-09-11T07:32:33.806 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002617556, 2024-09-11T07:32:33.806 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:32:33.806 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:32:33.807 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:32:33.807 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.807 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:32:33.807 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.801053+0000", 2024-09-11T07:32:33.807 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:33.807 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.807 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.807 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:32:33.807 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.801053+0000", 2024-09-11T07:32:33.807 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:33.807 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.807 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.807 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:32:33.808 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.801052+0000", 2024-09-11T07:32:33.808 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:32:33.808 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.808 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.808 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:32:33.808 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.801059+0000", 2024-09-11T07:32:33.808 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5420000000000002e-06 2024-09-11T07:32:33.808 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.808 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.808 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:32:33.808 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.801067+0000", 2024-09-11T07:32:33.808 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.49e-06 2024-09-11T07:32:33.808 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.809 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.809 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:32:33.809 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.801072+0000", 2024-09-11T07:32:33.809 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.78e-06 2024-09-11T07:32:33.809 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.809 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.809 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:32:33.809 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.801148+0000", 2024-09-11T07:32:33.809 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.6379999999999997e-05 2024-09-11T07:32:33.809 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.809 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.809 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:32:33.809 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.801168+0000", 2024-09-11T07:32:33.810 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9922e-05 2024-09-11T07:32:33.810 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:33.810 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:33.810 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:33.810 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.810 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.810 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:240475 7.8 7:1ec2e7c3:::benchmark_data_smithi022_77720_object116720:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-11T07:32:33.810 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:33.795305+0000", 2024-09-11T07:32:33.810 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0083346370000000006, 2024-09-11T07:32:33.810 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0084472809999999992, 2024-09-11T07:32:33.810 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:32:33.811 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "waiting for sub ops", 2024-09-11T07:32:33.811 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:32:33.811 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:32:33.811 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:32:33.811 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 240475 2024-09-11T07:32:33.811 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.811 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:32:33.811 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.811 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:32:33.811 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.795305+0000", 2024-09-11T07:32:33.811 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:33.812 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.812 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.812 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:32:33.812 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.795305+0000", 2024-09-11T07:32:33.812 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:33.812 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.812 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.812 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:32:33.812 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.795303+0000", 2024-09-11T07:32:33.812 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:32:33.812 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.812 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.812 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:32:33.813 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.795311+0000", 2024-09-11T07:32:33.813 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.5830000000000007e-06 2024-09-11T07:32:33.813 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.813 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.813 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:32:33.813 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.795326+0000", 2024-09-11T07:32:33.813 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5107e-05 2024-09-11T07:32:33.813 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.813 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.813 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:32:33.813 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.795334+0000", 2024-09-11T07:32:33.813 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.3579999999999997e-06 2024-09-11T07:32:33.813 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.813 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.814 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:32:33.814 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.799487+0000", 2024-09-11T07:32:33.814 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0041534659999999998 2024-09-11T07:32:33.814 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.814 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.814 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:32:33.814 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.799605+0000", 2024-09-11T07:32:33.814 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011833900000000001 2024-09-11T07:32:33.814 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.814 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.814 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 4,7", 2024-09-11T07:32:33.814 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.799737+0000", 2024-09-11T07:32:33.814 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013145799999999999 2024-09-11T07:32:33.814 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:33.815 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:33.815 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:33.815 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.815 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.815 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:240477 7.8 7.ddb555b8 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-11T07:32:33.815 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:33.796174+0000", 2024-09-11T07:32:33.815 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.007465162, 2024-09-11T07:32:33.815 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0076492590000000003, 2024-09-11T07:32:33.815 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:32:33.815 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "queued for pg", 2024-09-11T07:32:33.815 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:32:33.815 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:32:33.815 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:32:33.815 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 240477 2024-09-11T07:32:33.815 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.816 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:32:33.816 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.816 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:32:33.816 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.796174+0000", 2024-09-11T07:32:33.816 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:33.816 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.816 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.816 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:32:33.816 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.796174+0000", 2024-09-11T07:32:33.816 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:33.816 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.816 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.816 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:32:33.817 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.796173+0000", 2024-09-11T07:32:33.817 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:32:33.817 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.817 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.817 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:32:33.817 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.796179+0000", 2024-09-11T07:32:33.817 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.404e-06 2024-09-11T07:32:33.817 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.817 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.817 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:32:33.817 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.796189+0000", 2024-09-11T07:32:33.817 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0066e-05 2024-09-11T07:32:33.817 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.817 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.818 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:32:33.818 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.796195+0000", 2024-09-11T07:32:33.818 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7259999999999999e-06 2024-09-11T07:32:33.818 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:33.818 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:33.818 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:33.818 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.818 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.818 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:240478 7.e 7:7a6a99d3:::benchmark_data_smithi022_77720_object116723:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-11T07:32:33.818 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:33.796202+0000", 2024-09-11T07:32:33.818 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0074370670000000003, 2024-09-11T07:32:33.818 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0076751809999999997, 2024-09-11T07:32:33.818 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:32:33.818 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "waiting for sub ops", 2024-09-11T07:32:33.819 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:32:33.819 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:32:33.819 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:32:33.819 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 240478 2024-09-11T07:32:33.819 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.819 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:32:33.819 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.819 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:32:33.819 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.796202+0000", 2024-09-11T07:32:33.819 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:33.819 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.819 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.819 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:32:33.819 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.796202+0000", 2024-09-11T07:32:33.820 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:33.820 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.820 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.820 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:32:33.820 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.796202+0000", 2024-09-11T07:32:33.820 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:32:33.820 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.820 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.820 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:32:33.820 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.796205+0000", 2024-09-11T07:32:33.820 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.1700000000000001e-06 2024-09-11T07:32:33.820 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.820 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.821 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:32:33.821 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.796210+0000", 2024-09-11T07:32:33.821 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.729e-06 2024-09-11T07:32:33.821 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.821 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.821 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:32:33.821 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.796213+0000", 2024-09-11T07:32:33.821 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5599999999999998e-06 2024-09-11T07:32:33.821 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.821 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.822 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:32:33.822 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.796360+0000", 2024-09-11T07:32:33.822 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000146277 2024-09-11T07:32:33.822 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.822 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.822 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:32:33.822 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.796456+0000", 2024-09-11T07:32:33.822 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.6067e-05 2024-09-11T07:32:33.822 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.822 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.822 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:32:33.822 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.796561+0000", 2024-09-11T07:32:33.822 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010495199999999999 2024-09-11T07:32:33.822 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:33.822 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:33.823 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:33.823 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.823 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.823 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:240474 7.3 e156/92 7:c23d7769:::benchmark_data_smithi022_77720_object116719:head v 156'14942, mlcod=156'14940)", 2024-09-11T07:32:33.823 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:33.796312+0000", 2024-09-11T07:32:33.823 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0073278680000000004, 2024-09-11T07:32:33.823 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0076349110000000003, 2024-09-11T07:32:33.823 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:32:33.823 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:32:33.823 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:32:33.823 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.823 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:32:33.823 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.796312+0000", 2024-09-11T07:32:33.823 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:33.823 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.824 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.824 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:32:33.824 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.796312+0000", 2024-09-11T07:32:33.824 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:33.824 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.824 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.824 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:32:33.824 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.796311+0000", 2024-09-11T07:32:33.824 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:32:33.824 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.824 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.824 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:32:33.824 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.796318+0000", 2024-09-11T07:32:33.824 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.2359999999999998e-06 2024-09-11T07:32:33.825 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.825 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.825 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:32:33.825 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.796329+0000", 2024-09-11T07:32:33.825 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.076e-05 2024-09-11T07:32:33.825 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.825 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.825 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:32:33.825 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.796333+0000", 2024-09-11T07:32:33.825 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6020000000000002e-06 2024-09-11T07:32:33.825 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.825 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.825 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:32:33.826 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.796510+0000", 2024-09-11T07:32:33.826 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00017650599999999999 2024-09-11T07:32:33.826 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.826 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.826 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:32:33.826 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.796530+0000", 2024-09-11T07:32:33.826 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9956999999999999e-05 2024-09-11T07:32:33.826 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:33.826 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:33.826 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:33.826 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.826 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.826 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:240479 7.3 e156/92 7:c3655086:::benchmark_data_smithi022_77720_object116724:head v 156'14943, mlcod=156'14940)", 2024-09-11T07:32:33.826 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:33.799791+0000", 2024-09-11T07:32:33.827 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0038482479999999999, 2024-09-11T07:32:33.827 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0042106920000000003, 2024-09-11T07:32:33.827 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:32:33.827 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:32:33.827 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:32:33.827 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.827 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:32:33.827 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.799791+0000", 2024-09-11T07:32:33.827 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:33.827 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.827 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.827 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:32:33.827 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.799791+0000", 2024-09-11T07:32:33.827 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:32:33.827 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.827 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.828 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:32:33.828 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.799790+0000", 2024-09-11T07:32:33.828 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:32:33.828 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.828 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.828 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:32:33.828 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.799799+0000", 2024-09-11T07:32:33.828 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.0010000000000007e-06 2024-09-11T07:32:33.828 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.828 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.828 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:32:33.828 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.799810+0000", 2024-09-11T07:32:33.828 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.152e-05 2024-09-11T07:32:33.828 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.829 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.829 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:32:33.829 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.799817+0000", 2024-09-11T07:32:33.829 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1380000000000004e-06 2024-09-11T07:32:33.829 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.829 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.829 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:32:33.829 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.800907+0000", 2024-09-11T07:32:33.829 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0010908199999999999 2024-09-11T07:32:33.829 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:32:33.829 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:32:33.829 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:32:33.829 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:33.800926+0000", 2024-09-11T07:32:33.829 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8674000000000001e-05 2024-09-11T07:32:33.829 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:33.830 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:32:33.830 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:33.830 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:32:33.830 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:32:33.830 INFO:teuthology.orchestra.run.smithi022.stdout: "num_ops": 6 2024-09-11T07:32:33.830 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:32:33.830 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:32:33.831 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:32:33.864 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:32:33.865 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:32:33.891 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:33.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:33.944 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:33.944 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:34.045 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:34.069 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:34.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:34.071 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:34.171 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:34.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:34.212 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:34.212 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:34.251 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is unset 2024-09-11T07:32:34.311 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:34 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2982499089' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:32:34.311 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:34 smithi022 ceph-mon[29462]: from='client.24982 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:32:34.311 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:34 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2857326390' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:32:34.312 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:34 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2982499089' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:32:34.312 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:34 smithi022 ceph-mon[37330]: from='client.24982 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:32:34.312 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:34 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2857326390' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:32:34.312 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:34.352 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:34.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:34.353 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:34.452 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:32:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:32:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:32:34.453 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:34 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2982499089' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:32:34.453 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:34 smithi104 ceph-mon[33507]: from='client.24982 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:32:34.453 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:34 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2857326390' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:32:34.454 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:34.501 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:34.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:34.502 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:34.603 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:34.630 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:34.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:34.631 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:34.732 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:34.758 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:34.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:34.760 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:34.861 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:34.901 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:34.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:34.902 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:35.003 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:35.036 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:35.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:35.037 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:35.138 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:35.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:35.169 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:35.169 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:35.270 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:35.302 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:35.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:35.303 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:35.404 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:35.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:35.432 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:35.432 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:35.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:35 smithi104 ceph-mon[33507]: pgmap v250: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 621 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 615 op/s; 7265/30348 objects degraded (23.939%); 0 B/s, 182 objects/s recovering 2024-09-11T07:32:35.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:35 smithi104 ceph-mon[33507]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:32:35.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:35 smithi104 ceph-mon[33507]: from='client.24982 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:32:35.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:35 smithi104 ceph-mon[33507]: osdmap e157: 8 total, 8 up, 8 in 2024-09-11T07:32:35.533 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:35.573 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:35.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:35.574 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:35.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:35 smithi022 ceph-mon[29462]: pgmap v250: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 621 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 615 op/s; 7265/30348 objects degraded (23.939%); 0 B/s, 182 objects/s recovering 2024-09-11T07:32:35.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:35 smithi022 ceph-mon[29462]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:32:35.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:35 smithi022 ceph-mon[29462]: from='client.24982 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:32:35.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:35 smithi022 ceph-mon[29462]: osdmap e157: 8 total, 8 up, 8 in 2024-09-11T07:32:35.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:35 smithi022 ceph-mon[37330]: pgmap v250: 148 pgs: 2 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 143 active+clean; 621 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 615 op/s; 7265/30348 objects degraded (23.939%); 0 B/s, 182 objects/s recovering 2024-09-11T07:32:35.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:35 smithi022 ceph-mon[37330]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:32:35.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:35 smithi022 ceph-mon[37330]: from='client.24982 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:32:35.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:35 smithi022 ceph-mon[37330]: osdmap e157: 8 total, 8 up, 8 in 2024-09-11T07:32:35.674 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:35.700 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:35.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:35.701 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:35.802 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:35.846 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:35.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:35.847 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:35.948 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:35.975 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:35.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:35.976 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:36.077 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:36.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:36.131 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:36.132 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:36.232 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:36.279 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:36.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:36.280 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:36.381 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:36.422 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:36 smithi022 ceph-mon[29462]: Health check update: Degraded data redundancy: 7265/30348 objects degraded (23.939%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:32:36.423 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:36 smithi022 ceph-mon[37330]: Health check update: Degraded data redundancy: 7265/30348 objects degraded (23.939%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:32:36.426 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:36.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:36.427 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:36.528 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:36.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:36 smithi104 ceph-mon[33507]: Health check update: Degraded data redundancy: 7265/30348 objects degraded (23.939%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:32:36.549 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:36.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:36.550 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:36.650 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:36.697 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:36.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:36.699 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:36.799 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:36.826 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:36.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:36.827 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:36.928 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:36.967 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:36.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:36.969 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:37.069 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:37.095 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:37.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:37.096 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:37.197 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:37.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:37.228 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:37.228 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:37.283 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-11T07:32:37.328 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:37.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:37.392 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:37.392 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:37.394 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:37 smithi022 ceph-mon[37330]: pgmap v252: 148 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 144 active+clean; 414 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 633 op/s; 3633/20421 objects degraded (17.791%); 0 B/s, 223 objects/s recovering 2024-09-11T07:32:37.394 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:37 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:32:37.394 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:37 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:32:37.394 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:37 smithi022 ceph-mon[37330]: osdmap e158: 8 total, 8 up, 8 in 2024-09-11T07:32:37.396 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:37 smithi022 ceph-mon[29462]: pgmap v252: 148 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 144 active+clean; 414 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 633 op/s; 3633/20421 objects degraded (17.791%); 0 B/s, 223 objects/s recovering 2024-09-11T07:32:37.397 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:37 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:32:37.397 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:37 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:32:37.397 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:37 smithi022 ceph-mon[29462]: osdmap e158: 8 total, 8 up, 8 in 2024-09-11T07:32:37.493 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:37.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:37 smithi104 ceph-mon[33507]: pgmap v252: 148 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 144 active+clean; 414 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 633 op/s; 3633/20421 objects degraded (17.791%); 0 B/s, 223 objects/s recovering 2024-09-11T07:32:37.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:37 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:32:37.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:37 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:32:37.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:37 smithi104 ceph-mon[33507]: osdmap e158: 8 total, 8 up, 8 in 2024-09-11T07:32:37.552 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:37.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:37.554 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:37.654 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:37.703 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:37.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:37.705 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:37.740 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T07:32:37.805 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:37.828 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:37.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:37.829 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:37.930 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:37.954 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:37.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:37.955 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:38.056 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:38.081 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:38.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:38.082 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:38.182 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:38.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:38.208 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:38.208 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:38.309 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:38.332 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:38.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:38.337 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:38.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:32:38.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:32:38.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:32:38.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:32:38.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:32:38.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:32:38.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:32:38.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:32:38.437 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:38.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:38.499 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:38.499 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:32:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:32:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:32:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:32:38.600 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:38.625 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:38.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:38.626 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:38.727 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:38.748 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:38.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:38.749 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:38.850 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:38.876 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:38.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:38.877 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:38.978 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:39.002 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:39.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:39.003 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:39.104 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:39.133 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:32:39.144 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:39.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:39.149 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:39.250 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:39.272 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:39.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:39.272 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:39.373 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:39.398 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:39.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:39.399 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:39.465 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:39 smithi022 ceph-mon[29462]: 5.1b scrub starts 2024-09-11T07:32:39.465 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:39 smithi022 ceph-mon[29462]: pgmap v254: 148 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 144 active+clean; 414 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 633 op/s; 3633/20421 objects degraded (17.791%); 0 B/s, 223 objects/s recovering 2024-09-11T07:32:39.466 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:39 smithi022 ceph-mon[29462]: 5.11 scrub starts 2024-09-11T07:32:39.466 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:39 smithi022 ceph-mon[29462]: 5.19 scrub starts 2024-09-11T07:32:39.466 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:39 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:32:39.466 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:39 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:32:39.466 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:39 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:32:39.466 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:39 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:32:39.466 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:39 smithi022 ceph-mon[37330]: 5.1b scrub starts 2024-09-11T07:32:39.466 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:39 smithi022 ceph-mon[37330]: pgmap v254: 148 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 144 active+clean; 414 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 633 op/s; 3633/20421 objects degraded (17.791%); 0 B/s, 223 objects/s recovering 2024-09-11T07:32:39.466 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:39 smithi022 ceph-mon[37330]: 5.11 scrub starts 2024-09-11T07:32:39.466 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:39 smithi022 ceph-mon[37330]: 5.19 scrub starts 2024-09-11T07:32:39.467 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:39 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:32:39.467 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:39 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:32:39.467 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:39 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:32:39.467 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:39 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:32:39.500 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:39.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:32:39 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:32:39.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:39 smithi104 ceph-mon[33507]: 5.1b scrub starts 2024-09-11T07:32:39.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:39 smithi104 ceph-mon[33507]: pgmap v254: 148 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 144 active+clean; 414 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 633 op/s; 3633/20421 objects degraded (17.791%); 0 B/s, 223 objects/s recovering 2024-09-11T07:32:39.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:39 smithi104 ceph-mon[33507]: 5.11 scrub starts 2024-09-11T07:32:39.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:39 smithi104 ceph-mon[33507]: 5.19 scrub starts 2024-09-11T07:32:39.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:39 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:32:39.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:39 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:32:39.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:39 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:32:39.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:39 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:32:39.561 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:39.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:39.562 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:39.663 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:39.689 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:39.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:39.690 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:39.791 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:39.815 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:39.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:39.816 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:39.917 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:39.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:39.967 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:39.967 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:40.068 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:40.095 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:40.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:40.096 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:40.197 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:40.241 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:40.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:40.242 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:40.343 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:40.369 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:40.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:40.370 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:40.470 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:40.496 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:40.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:40.497 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:40.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:40 smithi104 ceph-mon[33507]: 2.3 scrub starts 2024-09-11T07:32:40.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:40 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:40.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:40 smithi104 ceph-mon[33507]: osdmap e159: 8 total, 8 up, 8 in 2024-09-11T07:32:40.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:40 smithi104 ceph-mon[33507]: 2.a scrub starts 2024-09-11T07:32:40.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:40 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:32:40.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:40 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:32:40.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:40 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:32:40.598 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:40.645 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:40.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:40.646 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:40.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:32:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:32:40] "GET /metrics HTTP/1.1" 200 223885 "" "Prometheus/2.43.0" 2024-09-11T07:32:40.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:40 smithi022 ceph-mon[37330]: 2.3 scrub starts 2024-09-11T07:32:40.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:40 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:40.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:40 smithi022 ceph-mon[37330]: osdmap e159: 8 total, 8 up, 8 in 2024-09-11T07:32:40.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:40 smithi022 ceph-mon[37330]: 2.a scrub starts 2024-09-11T07:32:40.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:40 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:32:40.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:40 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:32:40.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:40 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:32:40.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:40 smithi022 ceph-mon[29462]: 2.3 scrub starts 2024-09-11T07:32:40.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:40 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:40.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:40 smithi022 ceph-mon[29462]: osdmap e159: 8 total, 8 up, 8 in 2024-09-11T07:32:40.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:40 smithi022 ceph-mon[29462]: 2.a scrub starts 2024-09-11T07:32:40.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:40 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:32:40.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:40 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:32:40.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:40 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:32:40.747 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:40.770 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:40.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:40.772 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:40.872 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:40.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:40.921 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:40.921 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:41.022 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:41.051 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:41.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:41.052 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:41.153 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:41.179 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:41.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:41.180 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:41.281 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:41.330 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:41.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:41.331 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:41.420 INFO:teuthology.orchestra.run.smithi022.stderr:reweighted osd.1 to 1 (10000) 2024-09-11T07:32:41.431 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:41.458 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:41.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:41.459 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:41.560 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:41.585 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:41.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:41.586 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:41.636 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:32:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:41 smithi022 ceph-mon[29462]: pgmap v256: 148 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 144 active+clean; 164 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.22k op/s; 3633/8457 objects degraded (42.958%); 0 B/s, 363 objects/s recovering 2024-09-11T07:32:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:41 smithi022 ceph-mon[29462]: 5.12 scrub starts 2024-09-11T07:32:41.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:41 smithi022 ceph-mon[29462]: Health check update: Degraded data redundancy: 3633/20421 objects degraded (17.791%), 1 pg degraded (PG_DEGRADED) 2024-09-11T07:32:41.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:41 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3133365027' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T07:32:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:41 smithi022 ceph-mon[37330]: pgmap v256: 148 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 144 active+clean; 164 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.22k op/s; 3633/8457 objects degraded (42.958%); 0 B/s, 363 objects/s recovering 2024-09-11T07:32:41.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:41 smithi022 ceph-mon[37330]: 5.12 scrub starts 2024-09-11T07:32:41.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:41 smithi022 ceph-mon[37330]: Health check update: Degraded data redundancy: 3633/20421 objects degraded (17.791%), 1 pg degraded (PG_DEGRADED) 2024-09-11T07:32:41.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:41 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3133365027' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T07:32:41.687 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:41.709 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:41.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:41.710 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:41.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:41 smithi104 ceph-mon[33507]: pgmap v256: 148 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 144 active+clean; 164 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.22k op/s; 3633/8457 objects degraded (42.958%); 0 B/s, 363 objects/s recovering 2024-09-11T07:32:41.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:41 smithi104 ceph-mon[33507]: 5.12 scrub starts 2024-09-11T07:32:41.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:41 smithi104 ceph-mon[33507]: Health check update: Degraded data redundancy: 3633/20421 objects degraded (17.791%), 1 pg degraded (PG_DEGRADED) 2024-09-11T07:32:41.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:41 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3133365027' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T07:32:41.811 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:41.840 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:41.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:41.841 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:41.942 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:41.982 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:41.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:41.983 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:42.084 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:42.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:42.136 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:42.136 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:42.237 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:42.259 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:42.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:42.260 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:42.361 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:42.402 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:42.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:42.403 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:42.504 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:42.532 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:42.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:42.533 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:42.545 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Removed 123752 objects 2024-09-11T07:32:42.545 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Clean up completed and total clean up time :129.944 2024-09-11T07:32:42.634 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:42.658 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:42.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:42.659 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:42.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:42 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:32:42.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:42 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:32:42.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:42 smithi022 ceph-mon[29462]: 4.15 scrub starts 2024-09-11T07:32:42.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:42 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3133365027' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-11T07:32:42.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:42 smithi022 ceph-mon[29462]: osdmap e160: 8 total, 8 up, 8 in 2024-09-11T07:32:42.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:42 smithi022 ceph-mon[29462]: 4.2 scrub starts 2024-09-11T07:32:42.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:42 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:32:42.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:42 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:32:42.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:42 smithi022 ceph-mon[37330]: 4.15 scrub starts 2024-09-11T07:32:42.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:42 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3133365027' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-11T07:32:42.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:42 smithi022 ceph-mon[37330]: osdmap e160: 8 total, 8 up, 8 in 2024-09-11T07:32:42.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:42 smithi022 ceph-mon[37330]: 4.2 scrub starts 2024-09-11T07:32:42.760 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:42.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:42 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:32:42.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:42 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:32:42.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:42 smithi104 ceph-mon[33507]: 4.15 scrub starts 2024-09-11T07:32:42.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:42 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3133365027' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-11T07:32:42.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:42 smithi104 ceph-mon[33507]: osdmap e160: 8 total, 8 up, 8 in 2024-09-11T07:32:42.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:42 smithi104 ceph-mon[33507]: 4.2 scrub starts 2024-09-11T07:32:42.786 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:42.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:42.787 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:42.888 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:42.912 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:42.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:42.913 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:43.014 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:43.038 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:43.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:43.039 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:43.084 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:32:43.099 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:32:43.120 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight 2 1 2024-09-11T07:32:43.140 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:43.187 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:43.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:43.188 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:43.289 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:43.317 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:43.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:43.319 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:43.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:43 smithi022 ceph-mon[29462]: pgmap v258: 148 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 144 active+clean; 164 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 664 op/s; 3633/8457 objects degraded (42.958%); 0 B/s, 140 objects/s recovering 2024-09-11T07:32:43.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:43 smithi022 ceph-mon[29462]: 4.a scrub starts 2024-09-11T07:32:43.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:43 smithi022 ceph-mon[37330]: pgmap v258: 148 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 144 active+clean; 164 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 664 op/s; 3633/8457 objects degraded (42.958%); 0 B/s, 140 objects/s recovering 2024-09-11T07:32:43.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:43 smithi022 ceph-mon[37330]: 4.a scrub starts 2024-09-11T07:32:43.420 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:43.442 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:43.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:43.443 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:43.544 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:43.566 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:43.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:43.567 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:43.667 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:43.693 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:43.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:43.694 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:43.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:43 smithi104 ceph-mon[33507]: pgmap v258: 148 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 144 active+clean; 164 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 664 op/s; 3633/8457 objects degraded (42.958%); 0 B/s, 140 objects/s recovering 2024-09-11T07:32:43.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:43 smithi104 ceph-mon[33507]: 4.a scrub starts 2024-09-11T07:32:43.795 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:43.820 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:43.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:43.821 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:43.921 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:43.943 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:43.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:43.944 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:44.045 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:44.070 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:44.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:44.071 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:44.171 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:44.195 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:44.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:44.196 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:44.297 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:44.322 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:44.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:44.323 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:44.424 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:44.446 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:44.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:44.447 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:44.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:32:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:32:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:32:44.548 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:44.572 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:44.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:44.573 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:44.673 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:44.698 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:44.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:44.699 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:44.799 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:44.821 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:44.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:44.822 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:44.923 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:44.944 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:44.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:44.945 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:45.046 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:45.068 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:45.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:45.069 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:45.169 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:45.191 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:45.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:45.192 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:45.292 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:45.314 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:45.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:45.315 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:45.415 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:45.440 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:45.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:45.441 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:45.542 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:45.566 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:45.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:45.567 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:45.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:45 smithi022 ceph-mon[37330]: pgmap v259: 148 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 144 active+clean; 107 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 677 B/s rd, 0 B/s wr, 649 op/s; 3633/5715 objects degraded (63.570%); 0 B/s, 111 objects/s recovering 2024-09-11T07:32:45.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:45 smithi022 ceph-mon[37330]: 5.1f scrub starts 2024-09-11T07:32:45.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:45 smithi022 ceph-mon[37330]: 2.4 scrub starts 2024-09-11T07:32:45.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:45 smithi022 ceph-mon[29462]: pgmap v259: 148 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 144 active+clean; 107 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 677 B/s rd, 0 B/s wr, 649 op/s; 3633/5715 objects degraded (63.570%); 0 B/s, 111 objects/s recovering 2024-09-11T07:32:45.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:45 smithi022 ceph-mon[29462]: 5.1f scrub starts 2024-09-11T07:32:45.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:45 smithi022 ceph-mon[29462]: 2.4 scrub starts 2024-09-11T07:32:45.668 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:45.690 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:45.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:45.691 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:45.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:45 smithi104 ceph-mon[33507]: pgmap v259: 148 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 144 active+clean; 107 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 677 B/s rd, 0 B/s wr, 649 op/s; 3633/5715 objects degraded (63.570%); 0 B/s, 111 objects/s recovering 2024-09-11T07:32:45.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:45 smithi104 ceph-mon[33507]: 5.1f scrub starts 2024-09-11T07:32:45.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:45 smithi104 ceph-mon[33507]: 2.4 scrub starts 2024-09-11T07:32:45.792 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:45.817 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:45.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:45.818 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:45.919 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:45.943 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:45.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:45.944 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:46.045 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:46.069 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:46.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:46.070 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:46.171 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:46.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:46.197 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:46.197 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:46.298 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:46.467 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:46.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:46.468 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:46.569 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:46.594 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:46.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:46.595 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:46.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:46 smithi022 ceph-mon[29462]: 2.15 scrub starts 2024-09-11T07:32:46.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:46 smithi022 ceph-mon[29462]: Health check update: Degraded data redundancy: 3633/5715 objects degraded (63.570%), 1 pg degraded (PG_DEGRADED) 2024-09-11T07:32:46.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:46 smithi022 ceph-mon[37330]: 2.15 scrub starts 2024-09-11T07:32:46.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:46 smithi022 ceph-mon[37330]: Health check update: Degraded data redundancy: 3633/5715 objects degraded (63.570%), 1 pg degraded (PG_DEGRADED) 2024-09-11T07:32:46.696 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:46.721 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:46.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:46.722 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:46.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:46 smithi104 ceph-mon[33507]: 2.15 scrub starts 2024-09-11T07:32:46.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:46 smithi104 ceph-mon[33507]: Health check update: Degraded data redundancy: 3633/5715 objects degraded (63.570%), 1 pg degraded (PG_DEGRADED) 2024-09-11T07:32:46.823 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:46.845 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:46.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:46.846 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:46.946 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:46.972 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:46.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:46.973 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:47.074 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:47.096 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:47.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:47.098 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:47.198 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:47.223 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:47.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:47.224 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:47.325 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:47.352 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:47.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:47.353 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:47.454 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:47.482 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:47.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:47.483 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:47.583 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:47.605 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:47.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:47.606 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:47.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:47 smithi022 ceph-mon[29462]: pgmap v260: 148 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 144 active+clean; 582 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 827 op/s; 3633/588 objects degraded (617.857%); 0 B/s, 330 objects/s recovering 2024-09-11T07:32:47.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:47 smithi022 ceph-mon[29462]: osdmap e161: 8 total, 8 up, 8 in 2024-09-11T07:32:47.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:47 smithi022 ceph-mon[29462]: 7.c scrub starts 2024-09-11T07:32:47.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:47 smithi022 ceph-mon[29462]: 6.4 scrub starts 2024-09-11T07:32:47.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:47 smithi022 ceph-mon[37330]: pgmap v260: 148 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 144 active+clean; 582 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 827 op/s; 3633/588 objects degraded (617.857%); 0 B/s, 330 objects/s recovering 2024-09-11T07:32:47.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:47 smithi022 ceph-mon[37330]: osdmap e161: 8 total, 8 up, 8 in 2024-09-11T07:32:47.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:47 smithi022 ceph-mon[37330]: 7.c scrub starts 2024-09-11T07:32:47.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:47 smithi022 ceph-mon[37330]: 6.4 scrub starts 2024-09-11T07:32:47.707 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:47.729 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:47.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:47.730 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:47.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:47 smithi104 ceph-mon[33507]: pgmap v260: 148 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 144 active+clean; 582 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 827 op/s; 3633/588 objects degraded (617.857%); 0 B/s, 330 objects/s recovering 2024-09-11T07:32:47.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:47 smithi104 ceph-mon[33507]: osdmap e161: 8 total, 8 up, 8 in 2024-09-11T07:32:47.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:47 smithi104 ceph-mon[33507]: 7.c scrub starts 2024-09-11T07:32:47.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:47 smithi104 ceph-mon[33507]: 6.4 scrub starts 2024-09-11T07:32:47.831 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:47.853 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:47.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:47.854 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:47.955 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:47.977 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:47.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:47.978 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:48.079 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:48.102 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:48.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:48.103 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:48.203 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:48.227 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:48.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:48.228 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:48.277 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_0 2024-09-11T07:32:48.277 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-09-11T07:32:48.329 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:48.352 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:48.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:48.353 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:48.454 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:48.475 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:48.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:48.476 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:48.577 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:48.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:48.599 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:48.599 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:48.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:48 smithi022 ceph-mon[29462]: 2.1a scrub starts 2024-09-11T07:32:48.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:48 smithi022 ceph-mon[29462]: osdmap e162: 8 total, 8 up, 8 in 2024-09-11T07:32:48.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:48 smithi022 ceph-mon[37330]: 2.1a scrub starts 2024-09-11T07:32:48.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:48 smithi022 ceph-mon[37330]: osdmap e162: 8 total, 8 up, 8 in 2024-09-11T07:32:48.700 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:48.724 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:48.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:48.725 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:48.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:48 smithi104 ceph-mon[33507]: 2.1a scrub starts 2024-09-11T07:32:48.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:48 smithi104 ceph-mon[33507]: osdmap e162: 8 total, 8 up, 8 in 2024-09-11T07:32:48.826 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:48.847 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:48.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:48.848 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:48.949 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:48.970 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:48.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:48.971 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:49.071 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:49.094 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:49.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:49.095 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:49.196 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:49.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:49.218 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:49.219 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:49.320 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:49.345 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:49.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:49.346 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:49.447 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:49.470 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:49.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:49.471 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:49.572 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:49.596 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:49.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:49.597 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:49.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:49 smithi022 ceph-mon[29462]: pgmap v263: 148 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 144 active+clean; 582 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 754 B/s rd, 0 B/s wr, 387 op/s; 3633/588 objects degraded (617.857%); 0 B/s, 265 objects/s recovering 2024-09-11T07:32:49.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:49 smithi022 ceph-mon[29462]: 4.5 scrub starts 2024-09-11T07:32:49.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:49 smithi022 ceph-mon[29462]: osdmap e163: 8 total, 8 up, 8 in 2024-09-11T07:32:49.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:49 smithi022 ceph-mon[37330]: pgmap v263: 148 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 144 active+clean; 582 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 754 B/s rd, 0 B/s wr, 387 op/s; 3633/588 objects degraded (617.857%); 0 B/s, 265 objects/s recovering 2024-09-11T07:32:49.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:49 smithi022 ceph-mon[37330]: 4.5 scrub starts 2024-09-11T07:32:49.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:49 smithi022 ceph-mon[37330]: osdmap e163: 8 total, 8 up, 8 in 2024-09-11T07:32:49.698 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:49.719 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:49.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:49.720 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:49.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:32:49 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:32:49.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:49 smithi104 ceph-mon[33507]: pgmap v263: 148 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 144 active+clean; 582 KiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 754 B/s rd, 0 B/s wr, 387 op/s; 3633/588 objects degraded (617.857%); 0 B/s, 265 objects/s recovering 2024-09-11T07:32:49.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:49 smithi104 ceph-mon[33507]: 4.5 scrub starts 2024-09-11T07:32:49.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:49 smithi104 ceph-mon[33507]: osdmap e163: 8 total, 8 up, 8 in 2024-09-11T07:32:49.820 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:49.844 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:49.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:49.845 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:49.946 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:49.968 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:49.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:49.969 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:50.070 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:50.092 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:50.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:50.094 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:50.194 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:50.219 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:50.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:50.220 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:50.320 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:50.342 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:50.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:50.343 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:50.444 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:50.465 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:50.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:50.466 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:50.567 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:50.591 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:50.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:50.592 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:50.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:32:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:32:50] "GET /metrics HTTP/1.1" 200 223885 "" "Prometheus/2.43.0" 2024-09-11T07:32:50.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:50 smithi022 ceph-mon[29462]: 2.18 scrub starts 2024-09-11T07:32:50.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:50 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:50.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:50 smithi022 ceph-mon[29462]: osdmap e164: 8 total, 8 up, 8 in 2024-09-11T07:32:50.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:50 smithi022 ceph-mon[29462]: 2.9 scrub starts 2024-09-11T07:32:50.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:50 smithi022 ceph-mon[37330]: 2.18 scrub starts 2024-09-11T07:32:50.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:50 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:50.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:50 smithi022 ceph-mon[37330]: osdmap e164: 8 total, 8 up, 8 in 2024-09-11T07:32:50.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:50 smithi022 ceph-mon[37330]: 2.9 scrub starts 2024-09-11T07:32:50.692 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:50.717 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:50.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:50.718 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:50.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:50 smithi104 ceph-mon[33507]: 2.18 scrub starts 2024-09-11T07:32:50.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:50 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:32:50.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:50 smithi104 ceph-mon[33507]: osdmap e164: 8 total, 8 up, 8 in 2024-09-11T07:32:50.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:50 smithi104 ceph-mon[33507]: 2.9 scrub starts 2024-09-11T07:32:50.819 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:50.841 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:50.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:50.842 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:50.943 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:50.965 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:50.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:50.966 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:51.067 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:51.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:51.090 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:51.090 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:51.190 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:51.213 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:51.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:51.214 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:51.315 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:51.338 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:51.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:51.340 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:51.440 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:51.463 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:51.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:51.464 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:51.565 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:51.586 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:51.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:51.587 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:51.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:51 smithi022 ceph-mon[29462]: pgmap v266: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 582 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 847 objects/s recovering 2024-09-11T07:32:51.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:51 smithi022 ceph-mon[29462]: 4.8 scrub starts 2024-09-11T07:32:51.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:51 smithi022 ceph-mon[29462]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3633/588 objects degraded (617.857%), 1 pg degraded) 2024-09-11T07:32:51.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:51 smithi022 ceph-mon[37330]: pgmap v266: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 582 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 847 objects/s recovering 2024-09-11T07:32:51.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:51 smithi022 ceph-mon[37330]: 4.8 scrub starts 2024-09-11T07:32:51.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:51 smithi022 ceph-mon[37330]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3633/588 objects degraded (617.857%), 1 pg degraded) 2024-09-11T07:32:51.688 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:51.711 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:51.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:51.712 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:51.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:51 smithi104 ceph-mon[33507]: pgmap v266: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 582 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 847 objects/s recovering 2024-09-11T07:32:51.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:51 smithi104 ceph-mon[33507]: 4.8 scrub starts 2024-09-11T07:32:51.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:51 smithi104 ceph-mon[33507]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3633/588 objects degraded (617.857%), 1 pg degraded) 2024-09-11T07:32:51.812 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:51.837 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:51.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:51.838 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:51.939 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:51.961 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:51.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:51.962 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:52.062 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:52.084 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:52.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:52.085 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:52.186 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:52.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:52.212 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:52.212 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:52.313 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:52.337 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:52.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:52.338 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:52.439 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:52.463 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:52.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:52.464 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:52.565 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:52.586 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:52.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:52.587 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:52.688 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:52.710 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:52.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:52.711 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:52.812 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:52.837 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:52.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:52.838 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:52.939 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:52.962 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:52.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:52.963 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:53.064 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:53.087 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:53.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:53.089 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:53.189 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:53.213 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:53.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:53.214 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:53.315 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:53.339 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:53.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:53.340 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:53.441 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:53.463 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:53.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:53.464 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:53.565 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:53.589 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:53.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:53.590 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:53.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:53 smithi022 ceph-mon[29462]: pgmap v267: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 582 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 889 B/s rd, 0 op/s; 0 B/s, 588 objects/s recovering 2024-09-11T07:32:53.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:53 smithi022 ceph-mon[29462]: 2.1c scrub starts 2024-09-11T07:32:53.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:32:53.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:32:53.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:53 smithi022 ceph-mon[37330]: pgmap v267: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 582 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 889 B/s rd, 0 op/s; 0 B/s, 588 objects/s recovering 2024-09-11T07:32:53.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:53 smithi022 ceph-mon[37330]: 2.1c scrub starts 2024-09-11T07:32:53.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:32:53.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:32:53.691 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:53.715 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:53.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:53.716 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:53.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:53 smithi104 ceph-mon[33507]: pgmap v267: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 582 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 889 B/s rd, 0 op/s; 0 B/s, 588 objects/s recovering 2024-09-11T07:32:53.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:53 smithi104 ceph-mon[33507]: 2.1c scrub starts 2024-09-11T07:32:53.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:32:53.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:32:53.816 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:53.839 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:53.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:53.840 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:53.940 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:53.964 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:53.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:53.965 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:54.066 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:54.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:54.090 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:54.091 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:54.191 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:54.215 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:54.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:54.216 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:54.279 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:32:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:32:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:32:54.316 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:54.341 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:54.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:54.342 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:54.443 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:54.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:54.467 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:54.568 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:54.591 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:54.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:54.592 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:54.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:54 smithi022 ceph-mon[29462]: 4.1f scrub starts 2024-09-11T07:32:54.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:54 smithi022 ceph-mon[29462]: 4.1f scrub ok 2024-09-11T07:32:54.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:54 smithi022 ceph-mon[29462]: 2.f scrub starts 2024-09-11T07:32:54.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:54 smithi022 ceph-mon[37330]: 4.1f scrub starts 2024-09-11T07:32:54.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:54 smithi022 ceph-mon[37330]: 4.1f scrub ok 2024-09-11T07:32:54.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:54 smithi022 ceph-mon[37330]: 2.f scrub starts 2024-09-11T07:32:54.693 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:54.713 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:54.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:54.714 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:54.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:54 smithi104 ceph-mon[33507]: 4.1f scrub starts 2024-09-11T07:32:54.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:54 smithi104 ceph-mon[33507]: 4.1f scrub ok 2024-09-11T07:32:54.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:54 smithi104 ceph-mon[33507]: 2.f scrub starts 2024-09-11T07:32:54.815 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:54.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:54.839 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:54.839 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:54.940 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:54.965 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:54.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:54.966 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:55.067 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:55.089 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:55.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:55.090 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:55.191 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:55.213 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:55.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:55.214 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:55.315 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:55.336 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:55.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:55.337 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:55.438 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:55.459 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:55.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:55.460 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:55.561 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:55.582 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:55.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:55.583 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:55.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:55 smithi022 ceph-mon[29462]: pgmap v268: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 582 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 757 B/s rd, 0 op/s; 0 B/s, 501 objects/s recovering 2024-09-11T07:32:55.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:55 smithi022 ceph-mon[29462]: 2.12 scrub starts 2024-09-11T07:32:55.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:55 smithi022 ceph-mon[29462]: 2.14 scrub starts 2024-09-11T07:32:55.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:55 smithi022 ceph-mon[37330]: pgmap v268: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 582 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 757 B/s rd, 0 op/s; 0 B/s, 501 objects/s recovering 2024-09-11T07:32:55.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:55 smithi022 ceph-mon[37330]: 2.12 scrub starts 2024-09-11T07:32:55.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:55 smithi022 ceph-mon[37330]: 2.14 scrub starts 2024-09-11T07:32:55.683 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:55.704 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:55.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:55.705 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:55.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:55 smithi104 ceph-mon[33507]: pgmap v268: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 582 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 757 B/s rd, 0 op/s; 0 B/s, 501 objects/s recovering 2024-09-11T07:32:55.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:55 smithi104 ceph-mon[33507]: 2.12 scrub starts 2024-09-11T07:32:55.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:55 smithi104 ceph-mon[33507]: 2.14 scrub starts 2024-09-11T07:32:55.806 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:55.827 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:55.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:55.828 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:55.929 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:55.950 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:55.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:55.951 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:56.052 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:56.074 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:56.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:56.075 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:56.175 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:56.198 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:56.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:56.199 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:56.300 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:56.322 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:56.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:56.323 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:56.424 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:56.446 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:56.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:56.447 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:56.548 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:56.571 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:56.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:56.572 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:56.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:56 smithi022 ceph-mon[29462]: osdmap e165: 8 total, 8 up, 8 in 2024-09-11T07:32:56.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:56 smithi022 ceph-mon[37330]: osdmap e165: 8 total, 8 up, 8 in 2024-09-11T07:32:56.672 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:56.695 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:56.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:56.696 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:56.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:56 smithi104 ceph-mon[33507]: osdmap e165: 8 total, 8 up, 8 in 2024-09-11T07:32:56.797 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:32:56.818 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:56.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:56.819 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:32:56.920 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:56.946 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:56.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:56.947 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:57.048 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:57.071 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:57.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:57.072 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:57.173 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:57.197 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:57.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:57.198 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:57.299 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:57.321 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:57.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:57.322 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:57.422 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:57.445 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:57.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:57.446 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:57.546 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:57.568 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:57.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:57.569 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:57.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:57 smithi022 ceph-mon[29462]: pgmap v270: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 582 KiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 0 B/s, 848 objects/s recovering 2024-09-11T07:32:57.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:57 smithi022 ceph-mon[29462]: 5.14 scrub starts 2024-09-11T07:32:57.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:57 smithi022 ceph-mon[29462]: osdmap e166: 8 total, 8 up, 8 in 2024-09-11T07:32:57.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:57 smithi022 ceph-mon[29462]: 5.1 scrub starts 2024-09-11T07:32:57.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:57 smithi022 ceph-mon[37330]: pgmap v270: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 582 KiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 0 B/s, 848 objects/s recovering 2024-09-11T07:32:57.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:57 smithi022 ceph-mon[37330]: 5.14 scrub starts 2024-09-11T07:32:57.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:57 smithi022 ceph-mon[37330]: osdmap e166: 8 total, 8 up, 8 in 2024-09-11T07:32:57.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:57 smithi022 ceph-mon[37330]: 5.1 scrub starts 2024-09-11T07:32:57.669 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:57.691 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:57.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:57.692 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:57.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:57 smithi104 ceph-mon[33507]: pgmap v270: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 582 KiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 0 B/s, 848 objects/s recovering 2024-09-11T07:32:57.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:57 smithi104 ceph-mon[33507]: 5.14 scrub starts 2024-09-11T07:32:57.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:57 smithi104 ceph-mon[33507]: osdmap e166: 8 total, 8 up, 8 in 2024-09-11T07:32:57.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:57 smithi104 ceph-mon[33507]: 5.1 scrub starts 2024-09-11T07:32:57.793 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:57.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:57.816 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:57.816 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:57.917 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:57.939 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:57.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:57.940 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:58.040 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:58.063 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:58.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:58.064 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:58.165 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:58.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:58.187 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:58.187 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:58.288 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:58.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:58.311 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:58.311 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:58.372 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is unset 2024-09-11T07:32:58.412 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:58.434 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:58.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:58.435 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:58.536 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:32:58.562 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:58.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:58.563 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:32:58.663 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:58.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:58 smithi022 ceph-mon[37330]: 6.7 scrub starts 2024-09-11T07:32:58.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:58 smithi022 ceph-mon[37330]: 6.7 scrub ok 2024-09-11T07:32:58.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:58 smithi022 ceph-mon[37330]: 7.6 scrub starts 2024-09-11T07:32:58.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:58 smithi022 ceph-mon[37330]: 7.6 scrub ok 2024-09-11T07:32:58.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:58 smithi022 ceph-mon[37330]: 6.3 scrub starts 2024-09-11T07:32:58.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:58 smithi022 ceph-mon[37330]: 6.3 scrub ok 2024-09-11T07:32:58.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:58 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/811680077' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:32:58.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:58 smithi022 ceph-mon[29462]: 6.7 scrub starts 2024-09-11T07:32:58.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:58 smithi022 ceph-mon[29462]: 6.7 scrub ok 2024-09-11T07:32:58.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:58 smithi022 ceph-mon[29462]: 7.6 scrub starts 2024-09-11T07:32:58.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:58 smithi022 ceph-mon[29462]: 7.6 scrub ok 2024-09-11T07:32:58.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:58 smithi022 ceph-mon[29462]: 6.3 scrub starts 2024-09-11T07:32:58.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:58 smithi022 ceph-mon[29462]: 6.3 scrub ok 2024-09-11T07:32:58.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:58 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/811680077' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:32:58.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:58.687 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:58.687 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:58.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:58 smithi104 ceph-mon[33507]: 6.7 scrub starts 2024-09-11T07:32:58.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:58 smithi104 ceph-mon[33507]: 6.7 scrub ok 2024-09-11T07:32:58.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:58 smithi104 ceph-mon[33507]: 7.6 scrub starts 2024-09-11T07:32:58.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:58 smithi104 ceph-mon[33507]: 7.6 scrub ok 2024-09-11T07:32:58.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:58 smithi104 ceph-mon[33507]: 6.3 scrub starts 2024-09-11T07:32:58.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:58 smithi104 ceph-mon[33507]: 6.3 scrub ok 2024-09-11T07:32:58.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:58 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/811680077' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:32:58.788 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:32:58.810 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:58.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:58.812 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:32:58.870 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:32:58.871 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:32:58.871 INFO:teuthology.orchestra.run.smithi022.stdout: "complaint_time": 30, 2024-09-11T07:32:58.871 INFO:teuthology.orchestra.run.smithi022.stdout: "num_blocked_ops": 0 2024-09-11T07:32:58.871 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:32:58.913 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:58.935 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:58.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:58.936 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:59.037 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:59.063 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:59.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:59.064 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:59.165 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:59.189 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:59.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:59.190 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:59.291 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:32:59.314 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:59.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:59.315 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:32:59.344 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:32:59.416 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:32:59.439 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:59.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:59.440 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:32:59.541 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:59.565 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:59.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:59.566 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:32:59.667 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:32:59.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:59 smithi022 ceph-mon[37330]: pgmap v272: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 582 KiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 0 B/s, 397 objects/s recovering 2024-09-11T07:32:59.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:59 smithi022 ceph-mon[37330]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:32:59.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:59 smithi022 ceph-mon[37330]: Cluster is now healthy 2024-09-11T07:32:59.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:59 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/811680077' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:32:59.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:59 smithi022 ceph-mon[37330]: osdmap e167: 8 total, 8 up, 8 in 2024-09-11T07:32:59.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:59 smithi022 ceph-mon[37330]: 7.b scrub starts 2024-09-11T07:32:59.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:59 smithi022 ceph-mon[37330]: 7.b scrub ok 2024-09-11T07:32:59.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:59 smithi022 ceph-mon[37330]: 6.16 scrub starts 2024-09-11T07:32:59.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:32:59 smithi022 ceph-mon[37330]: 6.16 scrub ok 2024-09-11T07:32:59.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:59 smithi022 ceph-mon[29462]: pgmap v272: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 582 KiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 0 B/s, 397 objects/s recovering 2024-09-11T07:32:59.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:59 smithi022 ceph-mon[29462]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:32:59.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:59 smithi022 ceph-mon[29462]: Cluster is now healthy 2024-09-11T07:32:59.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:59 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/811680077' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:32:59.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:59 smithi022 ceph-mon[29462]: osdmap e167: 8 total, 8 up, 8 in 2024-09-11T07:32:59.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:59 smithi022 ceph-mon[29462]: 7.b scrub starts 2024-09-11T07:32:59.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:59 smithi022 ceph-mon[29462]: 7.b scrub ok 2024-09-11T07:32:59.670 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:59 smithi022 ceph-mon[29462]: 6.16 scrub starts 2024-09-11T07:32:59.670 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:32:59 smithi022 ceph-mon[29462]: 6.16 scrub ok 2024-09-11T07:32:59.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:59.693 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:32:59.693 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:32:59.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:32:59 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:32:59.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:59 smithi104 ceph-mon[33507]: pgmap v272: 148 pgs: 2 active+recovering+undersized+remapped, 146 active+clean; 582 KiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 0 B/s, 397 objects/s recovering 2024-09-11T07:32:59.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:59 smithi104 ceph-mon[33507]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:32:59.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:59 smithi104 ceph-mon[33507]: Cluster is now healthy 2024-09-11T07:32:59.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:59 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/811680077' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:32:59.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:59 smithi104 ceph-mon[33507]: osdmap e167: 8 total, 8 up, 8 in 2024-09-11T07:32:59.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:59 smithi104 ceph-mon[33507]: 7.b scrub starts 2024-09-11T07:32:59.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:59 smithi104 ceph-mon[33507]: 7.b scrub ok 2024-09-11T07:32:59.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:59 smithi104 ceph-mon[33507]: 6.16 scrub starts 2024-09-11T07:32:59.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:32:59 smithi104 ceph-mon[33507]: 6.16 scrub ok 2024-09-11T07:32:59.794 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:32:59.798 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-11T07:32:59.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:59.818 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:59.818 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:32:59.920 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:32:59.942 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:32:59.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:32:59.943 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:00.044 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:00.070 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:00.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:00.071 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:00.172 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:00.196 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:00.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:00.197 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:00.298 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:00.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:00.325 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:00.325 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:00.326 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:33:00.326 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:33:00.327 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:33:00.352 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:33:00.352 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:33:00.369 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:33:00.369 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:33:00.388 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:33:00.389 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:33:00.404 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:33:00.404 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:33:00.420 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:33:00.420 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:33:00.426 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:00.435 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:33:00.436 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:33:00.449 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:00.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:00.450 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:00.451 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:33:00.451 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:33:00.551 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:00.574 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:00.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:00.575 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:00.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:33:00 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:33:00] "GET /metrics HTTP/1.1" 200 223798 "" "Prometheus/2.43.0" 2024-09-11T07:33:00.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:00 smithi022 ceph-mon[29462]: 4.14 scrub starts 2024-09-11T07:33:00.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:00 smithi022 ceph-mon[29462]: 4.14 scrub ok 2024-09-11T07:33:00.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:00 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:00.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:00 smithi022 ceph-mon[29462]: osdmap e168: 8 total, 8 up, 8 in 2024-09-11T07:33:00.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:00 smithi022 ceph-mon[29462]: 2.1f scrub starts 2024-09-11T07:33:00.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:00 smithi022 ceph-mon[29462]: 2.1f scrub ok 2024-09-11T07:33:00.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:00 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/443571760' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:33:00.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:00 smithi022 ceph-mon[37330]: 4.14 scrub starts 2024-09-11T07:33:00.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:00 smithi022 ceph-mon[37330]: 4.14 scrub ok 2024-09-11T07:33:00.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:00 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:00.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:00 smithi022 ceph-mon[37330]: osdmap e168: 8 total, 8 up, 8 in 2024-09-11T07:33:00.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:00 smithi022 ceph-mon[37330]: 2.1f scrub starts 2024-09-11T07:33:00.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:00 smithi022 ceph-mon[37330]: 2.1f scrub ok 2024-09-11T07:33:00.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:00 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/443571760' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:33:00.676 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:00.699 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:00.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:00.700 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:00.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:00 smithi104 ceph-mon[33507]: 4.14 scrub starts 2024-09-11T07:33:00.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:00 smithi104 ceph-mon[33507]: 4.14 scrub ok 2024-09-11T07:33:00.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:00 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:00.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:00 smithi104 ceph-mon[33507]: osdmap e168: 8 total, 8 up, 8 in 2024-09-11T07:33:00.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:00 smithi104 ceph-mon[33507]: 2.1f scrub starts 2024-09-11T07:33:00.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:00 smithi104 ceph-mon[33507]: 2.1f scrub ok 2024-09-11T07:33:00.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:00 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/443571760' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:33:00.800 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:00.822 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:00.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:00.823 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:00.924 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:00.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:00.950 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:00.951 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:01.051 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:01.074 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:01.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:01.075 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:01.175 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:01.199 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:01.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:01.200 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:01.301 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:01.327 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:01.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:01.328 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:01.428 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:01.453 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:01.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:01.454 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:01.556 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:01.583 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:01.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:01.584 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:01.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:01 smithi022 ceph-mon[29462]: pgmap v275: 148 pgs: 1 peering, 147 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:33:01.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:01 smithi022 ceph-mon[29462]: 5.6 scrub starts 2024-09-11T07:33:01.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:01 smithi022 ceph-mon[29462]: 5.6 scrub ok 2024-09-11T07:33:01.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:01 smithi022 ceph-mon[37330]: pgmap v275: 148 pgs: 1 peering, 147 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:33:01.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:01 smithi022 ceph-mon[37330]: 5.6 scrub starts 2024-09-11T07:33:01.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:01 smithi022 ceph-mon[37330]: 5.6 scrub ok 2024-09-11T07:33:01.685 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:01.710 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:01.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:01.711 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:01.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:01 smithi104 ceph-mon[33507]: pgmap v275: 148 pgs: 1 peering, 147 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:33:01.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:01 smithi104 ceph-mon[33507]: 5.6 scrub starts 2024-09-11T07:33:01.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:01 smithi104 ceph-mon[33507]: 5.6 scrub ok 2024-09-11T07:33:01.812 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:01.836 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:01.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:01.837 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:01.938 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:01.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:01.965 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:01.965 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:02.066 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:02.091 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:02.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:02.092 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:02.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T07:33:02.194 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:02.219 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:02.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:02.220 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:02.320 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:02.344 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:02.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:02.345 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:02.446 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:02.471 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:02.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:02.473 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:02.574 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:02.597 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:02.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:02.598 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:02.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:02 smithi022 ceph-mon[37330]: 5.d scrub starts 2024-09-11T07:33:02.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:02 smithi022 ceph-mon[37330]: 5.d scrub ok 2024-09-11T07:33:02.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:02 smithi022 ceph-mon[37330]: 5.2 scrub starts 2024-09-11T07:33:02.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:02 smithi022 ceph-mon[37330]: 5.2 scrub ok 2024-09-11T07:33:02.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:02 smithi022 ceph-mon[29462]: 5.d scrub starts 2024-09-11T07:33:02.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:02 smithi022 ceph-mon[29462]: 5.d scrub ok 2024-09-11T07:33:02.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:02 smithi022 ceph-mon[29462]: 5.2 scrub starts 2024-09-11T07:33:02.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:02 smithi022 ceph-mon[29462]: 5.2 scrub ok 2024-09-11T07:33:02.699 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:02.724 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:02.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:02.725 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:02.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:02 smithi104 ceph-mon[33507]: 5.d scrub starts 2024-09-11T07:33:02.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:02 smithi104 ceph-mon[33507]: 5.d scrub ok 2024-09-11T07:33:02.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:02 smithi104 ceph-mon[33507]: 5.2 scrub starts 2024-09-11T07:33:02.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:02 smithi104 ceph-mon[33507]: 5.2 scrub ok 2024-09-11T07:33:02.827 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:02.853 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:02.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:02.854 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:02.955 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:02.981 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:02.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:02.984 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:03.085 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:03.111 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:03.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:03.112 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:03.212 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:03.238 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:03.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:03.239 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:03.340 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:03.368 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:03.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:03.369 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:03.469 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:03.496 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:03.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:03.498 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:03.598 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:03.624 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:03.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:03.625 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:03.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:03 smithi022 ceph-mon[37330]: pgmap v276: 148 pgs: 1 peering, 147 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:03.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:03 smithi022 ceph-mon[37330]: 4.12 scrub starts 2024-09-11T07:33:03.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:03 smithi022 ceph-mon[37330]: 4.12 scrub ok 2024-09-11T07:33:03.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:03 smithi022 ceph-mon[29462]: pgmap v276: 148 pgs: 1 peering, 147 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:03.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:03 smithi022 ceph-mon[29462]: 4.12 scrub starts 2024-09-11T07:33:03.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:03 smithi022 ceph-mon[29462]: 4.12 scrub ok 2024-09-11T07:33:03.726 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:03.753 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:03.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:03.754 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:03.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:03 smithi104 ceph-mon[33507]: pgmap v276: 148 pgs: 1 peering, 147 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:03.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:03 smithi104 ceph-mon[33507]: 4.12 scrub starts 2024-09-11T07:33:03.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:03 smithi104 ceph-mon[33507]: 4.12 scrub ok 2024-09-11T07:33:03.855 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:03.881 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:03.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:03.882 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:03.983 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:04.010 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:04.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:04.011 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:04.112 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:04.138 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:04.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:04.138 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:04.239 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:04.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:04.267 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:04.268 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:04.368 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:04.391 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:33:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:33:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:33:04.393 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:04.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:04.394 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:04.495 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:04.520 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:04.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:04.521 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:04.622 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:04.645 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:04.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:04.646 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:04.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:04 smithi022 ceph-mon[29462]: 6.14 scrub starts 2024-09-11T07:33:04.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:04 smithi022 ceph-mon[29462]: 6.14 scrub ok 2024-09-11T07:33:04.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:04 smithi022 ceph-mon[29462]: 4.11 scrub starts 2024-09-11T07:33:04.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:04 smithi022 ceph-mon[29462]: 4.11 scrub ok 2024-09-11T07:33:04.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:04 smithi022 ceph-mon[29462]: 2.11 scrub starts 2024-09-11T07:33:04.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:04 smithi022 ceph-mon[29462]: 2.11 scrub ok 2024-09-11T07:33:04.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:04 smithi022 ceph-mon[37330]: 6.14 scrub starts 2024-09-11T07:33:04.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:04 smithi022 ceph-mon[37330]: 6.14 scrub ok 2024-09-11T07:33:04.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:04 smithi022 ceph-mon[37330]: 4.11 scrub starts 2024-09-11T07:33:04.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:04 smithi022 ceph-mon[37330]: 4.11 scrub ok 2024-09-11T07:33:04.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:04 smithi022 ceph-mon[37330]: 2.11 scrub starts 2024-09-11T07:33:04.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:04 smithi022 ceph-mon[37330]: 2.11 scrub ok 2024-09-11T07:33:04.747 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:04.769 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:04.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:04.770 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:04.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:04 smithi104 ceph-mon[33507]: 6.14 scrub starts 2024-09-11T07:33:04.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:04 smithi104 ceph-mon[33507]: 6.14 scrub ok 2024-09-11T07:33:04.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:04 smithi104 ceph-mon[33507]: 4.11 scrub starts 2024-09-11T07:33:04.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:04 smithi104 ceph-mon[33507]: 4.11 scrub ok 2024-09-11T07:33:04.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:04 smithi104 ceph-mon[33507]: 2.11 scrub starts 2024-09-11T07:33:04.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:04 smithi104 ceph-mon[33507]: 2.11 scrub ok 2024-09-11T07:33:04.871 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:04.898 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:04.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:04.899 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:05.000 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:05.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:05.029 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:05.029 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:05.130 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:05.157 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:05.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:05.158 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:05.259 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:05.282 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:05.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:05.283 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:05.384 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:05.407 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:05.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:05.408 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:05.509 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:05.531 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:05.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:05.532 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:05.634 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:05.657 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:05.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:05.658 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:05.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:05 smithi022 ceph-mon[29462]: 7.8 scrub starts 2024-09-11T07:33:05.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:05 smithi022 ceph-mon[29462]: 7.8 scrub ok 2024-09-11T07:33:05.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:05 smithi022 ceph-mon[29462]: pgmap v277: 148 pgs: 1 peering, 147 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 663 B/s rd, 0 op/s 2024-09-11T07:33:05.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:05 smithi022 ceph-mon[29462]: 2.7 scrub starts 2024-09-11T07:33:05.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:05 smithi022 ceph-mon[29462]: 2.7 scrub ok 2024-09-11T07:33:05.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:05 smithi022 ceph-mon[37330]: 7.8 scrub starts 2024-09-11T07:33:05.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:05 smithi022 ceph-mon[37330]: 7.8 scrub ok 2024-09-11T07:33:05.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:05 smithi022 ceph-mon[37330]: pgmap v277: 148 pgs: 1 peering, 147 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 663 B/s rd, 0 op/s 2024-09-11T07:33:05.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:05 smithi022 ceph-mon[37330]: 2.7 scrub starts 2024-09-11T07:33:05.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:05 smithi022 ceph-mon[37330]: 2.7 scrub ok 2024-09-11T07:33:05.759 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:05.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:05 smithi104 ceph-mon[33507]: 7.8 scrub starts 2024-09-11T07:33:05.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:05 smithi104 ceph-mon[33507]: 7.8 scrub ok 2024-09-11T07:33:05.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:05 smithi104 ceph-mon[33507]: pgmap v277: 148 pgs: 1 peering, 147 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 663 B/s rd, 0 op/s 2024-09-11T07:33:05.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:05 smithi104 ceph-mon[33507]: 2.7 scrub starts 2024-09-11T07:33:05.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:05 smithi104 ceph-mon[33507]: 2.7 scrub ok 2024-09-11T07:33:05.787 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:05.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:05.788 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:05.888 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:05.912 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:05.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:05.913 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:06.014 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:06.038 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:06.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:06.039 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:06.140 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:06.168 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:06.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:06.170 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:06.270 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:06.296 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:06.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:06.297 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:06.398 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:06.424 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:06.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:06.425 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:06.526 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:06.552 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:06.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:06.553 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:06.654 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:06.677 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:06.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:06.679 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:06.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:06 smithi104 ceph-mon[33507]: 2.1 scrub starts 2024-09-11T07:33:06.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:06 smithi104 ceph-mon[33507]: 2.1 scrub ok 2024-09-11T07:33:06.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:06 smithi104 ceph-mon[33507]: 4.16 scrub starts 2024-09-11T07:33:06.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:06 smithi104 ceph-mon[33507]: 4.16 scrub ok 2024-09-11T07:33:06.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:06 smithi104 ceph-mon[33507]: pgmap v278: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 220 objects/s recovering 2024-09-11T07:33:06.780 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:06.807 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:06.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:06.808 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:06.909 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:06 smithi022 ceph-mon[29462]: 2.1 scrub starts 2024-09-11T07:33:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:06 smithi022 ceph-mon[29462]: 2.1 scrub ok 2024-09-11T07:33:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:06 smithi022 ceph-mon[29462]: 4.16 scrub starts 2024-09-11T07:33:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:06 smithi022 ceph-mon[29462]: 4.16 scrub ok 2024-09-11T07:33:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:06 smithi022 ceph-mon[29462]: pgmap v278: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 220 objects/s recovering 2024-09-11T07:33:06.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:06 smithi022 ceph-mon[37330]: 2.1 scrub starts 2024-09-11T07:33:06.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:06 smithi022 ceph-mon[37330]: 2.1 scrub ok 2024-09-11T07:33:06.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:06 smithi022 ceph-mon[37330]: 4.16 scrub starts 2024-09-11T07:33:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:06 smithi022 ceph-mon[37330]: 4.16 scrub ok 2024-09-11T07:33:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:06 smithi022 ceph-mon[37330]: pgmap v278: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 220 objects/s recovering 2024-09-11T07:33:06.935 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:06.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:06.936 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:07.037 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:07.062 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:07.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:07.063 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:07.164 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:07.191 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:07.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:07.192 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:07.293 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:07.319 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:07.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:07.320 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:07.421 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:07.448 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:07.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:07.449 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:07.550 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:07.578 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:07.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:07.579 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:07.680 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:07.705 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:07.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:07.706 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:07.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:07 smithi104 ceph-mon[33507]: 6.18 scrub starts 2024-09-11T07:33:07.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:07 smithi104 ceph-mon[33507]: 6.18 scrub ok 2024-09-11T07:33:07.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:07 smithi104 ceph-mon[33507]: 5.1a scrub starts 2024-09-11T07:33:07.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:07 smithi104 ceph-mon[33507]: 5.1a scrub ok 2024-09-11T07:33:07.807 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:07.834 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:07.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:07.835 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:07.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:07 smithi022 ceph-mon[29462]: 6.18 scrub starts 2024-09-11T07:33:07.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:07 smithi022 ceph-mon[29462]: 6.18 scrub ok 2024-09-11T07:33:07.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:07 smithi022 ceph-mon[29462]: 5.1a scrub starts 2024-09-11T07:33:07.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:07 smithi022 ceph-mon[29462]: 5.1a scrub ok 2024-09-11T07:33:07.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:07 smithi022 ceph-mon[37330]: 6.18 scrub starts 2024-09-11T07:33:07.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:07 smithi022 ceph-mon[37330]: 6.18 scrub ok 2024-09-11T07:33:07.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:07 smithi022 ceph-mon[37330]: 5.1a scrub starts 2024-09-11T07:33:07.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:07 smithi022 ceph-mon[37330]: 5.1a scrub ok 2024-09-11T07:33:07.936 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:07.962 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:07.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:07.963 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:08.064 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:08.090 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:08.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:08.091 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:08.192 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:08.217 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:08.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:08.218 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:08.319 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:08.344 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:08.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:08.345 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:08.446 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:08.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:08.476 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:08.476 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:08.577 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:08.603 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:08.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:08.604 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:08.705 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:08.732 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:08.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:08.733 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:08.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:08 smithi104 ceph-mon[33507]: 6.d scrub starts 2024-09-11T07:33:08.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:08 smithi104 ceph-mon[33507]: 6.d scrub ok 2024-09-11T07:33:08.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:08 smithi104 ceph-mon[33507]: 2.6 scrub starts 2024-09-11T07:33:08.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:08 smithi104 ceph-mon[33507]: 2.6 scrub ok 2024-09-11T07:33:08.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:08 smithi104 ceph-mon[33507]: 4.1e scrub starts 2024-09-11T07:33:08.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:08 smithi104 ceph-mon[33507]: 4.1e scrub ok 2024-09-11T07:33:08.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:33:08.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:33:08.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:08 smithi104 ceph-mon[33507]: pgmap v279: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 0 B/s, 181 objects/s recovering 2024-09-11T07:33:08.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-11T07:33:08.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:08 smithi104 ceph-mon[33507]: 6.1b scrub starts 2024-09-11T07:33:08.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:08 smithi104 ceph-mon[33507]: 6.1b scrub ok 2024-09-11T07:33:08.834 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:08.860 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:08.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:08.861 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:08.901 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:08 smithi022 ceph-mon[37330]: 6.d scrub starts 2024-09-11T07:33:08.901 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:08 smithi022 ceph-mon[37330]: 6.d scrub ok 2024-09-11T07:33:08.901 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:08 smithi022 ceph-mon[37330]: 2.6 scrub starts 2024-09-11T07:33:08.901 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:08 smithi022 ceph-mon[37330]: 2.6 scrub ok 2024-09-11T07:33:08.901 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:08 smithi022 ceph-mon[37330]: 4.1e scrub starts 2024-09-11T07:33:08.901 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:08 smithi022 ceph-mon[37330]: 4.1e scrub ok 2024-09-11T07:33:08.901 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:33:08.901 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:33:08.901 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:08 smithi022 ceph-mon[37330]: pgmap v279: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 0 B/s, 181 objects/s recovering 2024-09-11T07:33:08.901 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-11T07:33:08.902 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:08 smithi022 ceph-mon[37330]: 6.1b scrub starts 2024-09-11T07:33:08.902 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:08 smithi022 ceph-mon[37330]: 6.1b scrub ok 2024-09-11T07:33:08.902 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:08 smithi022 ceph-mon[29462]: 6.d scrub starts 2024-09-11T07:33:08.902 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:08 smithi022 ceph-mon[29462]: 6.d scrub ok 2024-09-11T07:33:08.902 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:08 smithi022 ceph-mon[29462]: 2.6 scrub starts 2024-09-11T07:33:08.902 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:08 smithi022 ceph-mon[29462]: 2.6 scrub ok 2024-09-11T07:33:08.902 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:08 smithi022 ceph-mon[29462]: 4.1e scrub starts 2024-09-11T07:33:08.902 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:08 smithi022 ceph-mon[29462]: 4.1e scrub ok 2024-09-11T07:33:08.902 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:33:08.902 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:33:08.902 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:08 smithi022 ceph-mon[29462]: pgmap v279: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 0 B/s, 181 objects/s recovering 2024-09-11T07:33:08.903 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-11T07:33:08.903 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:08 smithi022 ceph-mon[29462]: 6.1b scrub starts 2024-09-11T07:33:08.903 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:08 smithi022 ceph-mon[29462]: 6.1b scrub ok 2024-09-11T07:33:08.962 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:08.989 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:08.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:08.990 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:09.091 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:09.114 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:09.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:09.115 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:09.216 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:09.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:09.246 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:09.246 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:09.347 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:09.370 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:09.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:09.372 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:09.472 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:09.498 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:09.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:09.499 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:09.600 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:09.624 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:09.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:09.625 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:09.726 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:09.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:09.749 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:09.749 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:09.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:33:09 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:33:09.850 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:09.872 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:09.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:09.874 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:09.974 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:09.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:09.999 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:09.999 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:10.100 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:10.126 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:10.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:10.126 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:10.227 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:10.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:10 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-11T07:33:10.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:10 smithi022 ceph-mon[29462]: osdmap e169: 8 total, 8 up, 8 in 2024-09-11T07:33:10.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:10 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:10.252 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:10 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-11T07:33:10.252 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:10 smithi022 ceph-mon[37330]: osdmap e169: 8 total, 8 up, 8 in 2024-09-11T07:33:10.252 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:10 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:10.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:10.253 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:10.253 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:10.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:10 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-11T07:33:10.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:10 smithi104 ceph-mon[33507]: osdmap e169: 8 total, 8 up, 8 in 2024-09-11T07:33:10.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:10 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:10.354 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:10.382 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:10.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:10.383 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:10.484 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:10.508 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:10.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:10.509 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:10.517 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:33:10.610 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:10.628 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:33:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:33:10] "GET /metrics HTTP/1.1" 200 223793 "" "Prometheus/2.43.0" 2024-09-11T07:33:10.633 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:10.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:10.634 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:10.735 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:10.759 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:10.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:10.761 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:10.861 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:10.886 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:10.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:10.887 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:10.988 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:11.010 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:11.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:11.011 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:11.022 INFO:teuthology.orchestra.run.smithi022.stderr:reweighted osd.2 to 1 (10000) 2024-09-11T07:33:11.112 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:11.135 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:11.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:11.136 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:11.212 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:11 smithi022 ceph-mon[29462]: 2.1b scrub starts 2024-09-11T07:33:11.212 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:11 smithi022 ceph-mon[29462]: 2.1b scrub ok 2024-09-11T07:33:11.212 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:11 smithi022 ceph-mon[29462]: osdmap e170: 8 total, 8 up, 8 in 2024-09-11T07:33:11.213 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:11 smithi022 ceph-mon[29462]: 5.a scrub starts 2024-09-11T07:33:11.213 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:11 smithi022 ceph-mon[29462]: 5.a scrub ok 2024-09-11T07:33:11.213 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:11 smithi022 ceph-mon[29462]: pgmap v282: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 454 objects/s recovering 2024-09-11T07:33:11.213 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:11 smithi022 ceph-mon[29462]: 5.f scrub starts 2024-09-11T07:33:11.213 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:11 smithi022 ceph-mon[29462]: 5.f scrub ok 2024-09-11T07:33:11.213 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:11 smithi022 ceph-mon[29462]: 1.0 deep-scrub starts 2024-09-11T07:33:11.213 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:11 smithi022 ceph-mon[29462]: 1.0 deep-scrub ok 2024-09-11T07:33:11.213 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:11 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/567541540' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T07:33:11.213 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:11 smithi022 ceph-mon[37330]: 2.1b scrub starts 2024-09-11T07:33:11.213 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:11 smithi022 ceph-mon[37330]: 2.1b scrub ok 2024-09-11T07:33:11.213 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:11 smithi022 ceph-mon[37330]: osdmap e170: 8 total, 8 up, 8 in 2024-09-11T07:33:11.214 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:11 smithi022 ceph-mon[37330]: 5.a scrub starts 2024-09-11T07:33:11.214 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:11 smithi022 ceph-mon[37330]: 5.a scrub ok 2024-09-11T07:33:11.214 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:11 smithi022 ceph-mon[37330]: pgmap v282: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 454 objects/s recovering 2024-09-11T07:33:11.214 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:11 smithi022 ceph-mon[37330]: 5.f scrub starts 2024-09-11T07:33:11.214 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:11 smithi022 ceph-mon[37330]: 5.f scrub ok 2024-09-11T07:33:11.214 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:11 smithi022 ceph-mon[37330]: 1.0 deep-scrub starts 2024-09-11T07:33:11.214 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:11 smithi022 ceph-mon[37330]: 1.0 deep-scrub ok 2024-09-11T07:33:11.214 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:11 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/567541540' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T07:33:11.236 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:11.248 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:33:11.262 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:11.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:11.263 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:11.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:11 smithi104 ceph-mon[33507]: 2.1b scrub starts 2024-09-11T07:33:11.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:11 smithi104 ceph-mon[33507]: 2.1b scrub ok 2024-09-11T07:33:11.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:11 smithi104 ceph-mon[33507]: osdmap e170: 8 total, 8 up, 8 in 2024-09-11T07:33:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:11 smithi104 ceph-mon[33507]: 5.a scrub starts 2024-09-11T07:33:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:11 smithi104 ceph-mon[33507]: 5.a scrub ok 2024-09-11T07:33:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:11 smithi104 ceph-mon[33507]: pgmap v282: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 454 objects/s recovering 2024-09-11T07:33:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:11 smithi104 ceph-mon[33507]: 5.f scrub starts 2024-09-11T07:33:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:11 smithi104 ceph-mon[33507]: 5.f scrub ok 2024-09-11T07:33:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:11 smithi104 ceph-mon[33507]: 1.0 deep-scrub starts 2024-09-11T07:33:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:11 smithi104 ceph-mon[33507]: 1.0 deep-scrub ok 2024-09-11T07:33:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:11 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/567541540' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T07:33:11.364 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:11.387 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:11.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:11.388 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:11.489 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:11.515 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:11.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:11.516 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:11.616 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:11.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:11.640 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:11.640 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:11.741 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:11.767 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:11.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:11.768 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:11.869 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:11.894 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:11.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:11.895 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:11.996 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:12.018 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:12.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:12.019 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:12.120 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:12.143 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:12.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:12.144 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:12.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:12 smithi022 ceph-mon[37330]: 6.b scrub starts 2024-09-11T07:33:12.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:12 smithi022 ceph-mon[37330]: 6.b scrub ok 2024-09-11T07:33:12.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:12 smithi022 ceph-mon[37330]: 4.3 scrub starts 2024-09-11T07:33:12.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:12 smithi022 ceph-mon[37330]: 4.3 scrub ok 2024-09-11T07:33:12.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:12 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/567541540' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-11T07:33:12.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:12 smithi022 ceph-mon[37330]: osdmap e171: 8 total, 8 up, 8 in 2024-09-11T07:33:12.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:12 smithi022 ceph-mon[37330]: 7.0 scrub starts 2024-09-11T07:33:12.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:12 smithi022 ceph-mon[37330]: 7.0 scrub ok 2024-09-11T07:33:12.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:12 smithi022 ceph-mon[29462]: 6.b scrub starts 2024-09-11T07:33:12.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:12 smithi022 ceph-mon[29462]: 6.b scrub ok 2024-09-11T07:33:12.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:12 smithi022 ceph-mon[29462]: 4.3 scrub starts 2024-09-11T07:33:12.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:12 smithi022 ceph-mon[29462]: 4.3 scrub ok 2024-09-11T07:33:12.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:12 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/567541540' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-11T07:33:12.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:12 smithi022 ceph-mon[29462]: osdmap e171: 8 total, 8 up, 8 in 2024-09-11T07:33:12.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:12 smithi022 ceph-mon[29462]: 7.0 scrub starts 2024-09-11T07:33:12.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:12 smithi022 ceph-mon[29462]: 7.0 scrub ok 2024-09-11T07:33:12.245 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:12.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:12.268 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:12.268 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:12.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:12 smithi104 ceph-mon[33507]: 6.b scrub starts 2024-09-11T07:33:12.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:12 smithi104 ceph-mon[33507]: 6.b scrub ok 2024-09-11T07:33:12.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:12 smithi104 ceph-mon[33507]: 4.3 scrub starts 2024-09-11T07:33:12.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:12 smithi104 ceph-mon[33507]: 4.3 scrub ok 2024-09-11T07:33:12.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:12 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/567541540' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-11T07:33:12.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:12 smithi104 ceph-mon[33507]: osdmap e171: 8 total, 8 up, 8 in 2024-09-11T07:33:12.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:12 smithi104 ceph-mon[33507]: 7.0 scrub starts 2024-09-11T07:33:12.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:12 smithi104 ceph-mon[33507]: 7.0 scrub ok 2024-09-11T07:33:12.302 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight 3 1 2024-09-11T07:33:12.334 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:33:12.369 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:12.397 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:12.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:12.398 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:12.499 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:12.527 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:12.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:12.528 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:12.629 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:12.653 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:12.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:12.654 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:12.755 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:12.780 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:12.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:12.781 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:12.882 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:12.904 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:12.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:12.905 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:13.009 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:13.035 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:13.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:13.036 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:13.137 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:13.162 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:13 smithi022 ceph-mon[29462]: 5.10 scrub starts 2024-09-11T07:33:13.162 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:13 smithi022 ceph-mon[29462]: 5.10 scrub ok 2024-09-11T07:33:13.162 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:13 smithi022 ceph-mon[29462]: 4.17 scrub starts 2024-09-11T07:33:13.162 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:13 smithi022 ceph-mon[29462]: 4.17 scrub ok 2024-09-11T07:33:13.162 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:13 smithi022 ceph-mon[29462]: 6.9 scrub starts 2024-09-11T07:33:13.162 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:13 smithi022 ceph-mon[29462]: 6.9 scrub ok 2024-09-11T07:33:13.162 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:13 smithi022 ceph-mon[29462]: 2.e scrub starts 2024-09-11T07:33:13.163 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:13 smithi022 ceph-mon[29462]: 2.e scrub ok 2024-09-11T07:33:13.163 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:13 smithi022 ceph-mon[29462]: pgmap v284: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:13.163 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:13 smithi022 ceph-mon[29462]: 2.2 scrub starts 2024-09-11T07:33:13.163 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:13 smithi022 ceph-mon[29462]: 2.2 scrub ok 2024-09-11T07:33:13.163 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:13 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:33:13.163 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:13 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:33:13.163 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:13 smithi022 ceph-mon[37330]: 5.10 scrub starts 2024-09-11T07:33:13.163 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:13 smithi022 ceph-mon[37330]: 5.10 scrub ok 2024-09-11T07:33:13.163 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:13 smithi022 ceph-mon[37330]: 4.17 scrub starts 2024-09-11T07:33:13.163 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:13 smithi022 ceph-mon[37330]: 4.17 scrub ok 2024-09-11T07:33:13.163 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:13 smithi022 ceph-mon[37330]: 6.9 scrub starts 2024-09-11T07:33:13.164 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:13 smithi022 ceph-mon[37330]: 6.9 scrub ok 2024-09-11T07:33:13.164 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:13 smithi022 ceph-mon[37330]: 2.e scrub starts 2024-09-11T07:33:13.164 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:13 smithi022 ceph-mon[37330]: 2.e scrub ok 2024-09-11T07:33:13.164 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:13 smithi022 ceph-mon[37330]: pgmap v284: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:13.164 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:13 smithi022 ceph-mon[37330]: 2.2 scrub starts 2024-09-11T07:33:13.164 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:13 smithi022 ceph-mon[37330]: 2.2 scrub ok 2024-09-11T07:33:13.164 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:13 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:33:13.164 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:13 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:33:13.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:13.165 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:13.165 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:13.266 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:13.292 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:13.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:13.294 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:13.394 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:13.417 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:13.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:13.418 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:13.519 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:13.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:13 smithi104 ceph-mon[33507]: 5.10 scrub starts 2024-09-11T07:33:13.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:13 smithi104 ceph-mon[33507]: 5.10 scrub ok 2024-09-11T07:33:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:13 smithi104 ceph-mon[33507]: 4.17 scrub starts 2024-09-11T07:33:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:13 smithi104 ceph-mon[33507]: 4.17 scrub ok 2024-09-11T07:33:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:13 smithi104 ceph-mon[33507]: 6.9 scrub starts 2024-09-11T07:33:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:13 smithi104 ceph-mon[33507]: 6.9 scrub ok 2024-09-11T07:33:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:13 smithi104 ceph-mon[33507]: 2.e scrub starts 2024-09-11T07:33:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:13 smithi104 ceph-mon[33507]: 2.e scrub ok 2024-09-11T07:33:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:13 smithi104 ceph-mon[33507]: pgmap v284: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:13 smithi104 ceph-mon[33507]: 2.2 scrub starts 2024-09-11T07:33:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:13 smithi104 ceph-mon[33507]: 2.2 scrub ok 2024-09-11T07:33:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:13 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:33:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:13 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:33:13.542 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:13.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:13.543 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:13.644 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:13.670 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:13.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:13.671 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:13.772 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:13.798 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:13.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:13.799 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:13.899 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:13.922 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:13.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:13.923 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:13.958 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:33:14.025 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:14.041 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'unique_pool_0' removed 2024-09-11T07:33:14.051 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:14.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:14.052 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:14.152 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:14.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:14 smithi022 ceph-mon[29462]: osdmap e172: 8 total, 8 up, 8 in 2024-09-11T07:33:14.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:14 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4155131232' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:33:14.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:14 smithi022 ceph-mon[29462]: from='client.24998 ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:33:14.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:14 smithi022 ceph-mon[37330]: osdmap e172: 8 total, 8 up, 8 in 2024-09-11T07:33:14.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:14 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4155131232' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:33:14.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:14 smithi022 ceph-mon[37330]: from='client.24998 ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:33:14.175 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:14.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:14.176 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:14.277 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:14.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:14.300 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:14.300 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:14.401 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:14.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:14.424 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:14.424 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:14.525 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:14.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:33:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:33:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:33:14.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:14 smithi104 ceph-mon[33507]: osdmap e172: 8 total, 8 up, 8 in 2024-09-11T07:33:14.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:14 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4155131232' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:33:14.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:14 smithi104 ceph-mon[33507]: from='client.24998 ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:33:14.548 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:14.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:14.549 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:14.650 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:14.672 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:14.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:14.673 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:14.774 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:14.797 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:14.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:14.798 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:14.898 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:14.920 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:14.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:14.921 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:15.022 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:15.045 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:15.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:15.046 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:15.147 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:15.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:15 smithi022 ceph-mon[29462]: 2.5 scrub starts 2024-09-11T07:33:15.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:15 smithi022 ceph-mon[29462]: 2.5 scrub ok 2024-09-11T07:33:15.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:15 smithi022 ceph-mon[29462]: 6.6 scrub starts 2024-09-11T07:33:15.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:15 smithi022 ceph-mon[29462]: 6.6 scrub ok 2024-09-11T07:33:15.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:15 smithi022 ceph-mon[29462]: 5.1e scrub starts 2024-09-11T07:33:15.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:15 smithi022 ceph-mon[29462]: 5.1e scrub ok 2024-09-11T07:33:15.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:15 smithi022 ceph-mon[29462]: from='client.24998 ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T07:33:15.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:15 smithi022 ceph-mon[29462]: osdmap e173: 8 total, 8 up, 8 in 2024-09-11T07:33:15.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:15 smithi022 ceph-mon[29462]: pgmap v287: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 87 B/s, 0 objects/s recovering 2024-09-11T07:33:15.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:15 smithi022 ceph-mon[29462]: 3.0 scrub starts 2024-09-11T07:33:15.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:15 smithi022 ceph-mon[29462]: 3.0 scrub ok 2024-09-11T07:33:15.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:15 smithi022 ceph-mon[37330]: 2.5 scrub starts 2024-09-11T07:33:15.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:15 smithi022 ceph-mon[37330]: 2.5 scrub ok 2024-09-11T07:33:15.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:15 smithi022 ceph-mon[37330]: 6.6 scrub starts 2024-09-11T07:33:15.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:15 smithi022 ceph-mon[37330]: 6.6 scrub ok 2024-09-11T07:33:15.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:15 smithi022 ceph-mon[37330]: 5.1e scrub starts 2024-09-11T07:33:15.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:15 smithi022 ceph-mon[37330]: 5.1e scrub ok 2024-09-11T07:33:15.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:15 smithi022 ceph-mon[37330]: from='client.24998 ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T07:33:15.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:15 smithi022 ceph-mon[37330]: osdmap e173: 8 total, 8 up, 8 in 2024-09-11T07:33:15.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:15 smithi022 ceph-mon[37330]: pgmap v287: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 87 B/s, 0 objects/s recovering 2024-09-11T07:33:15.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:15 smithi022 ceph-mon[37330]: 3.0 scrub starts 2024-09-11T07:33:15.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:15 smithi022 ceph-mon[37330]: 3.0 scrub ok 2024-09-11T07:33:15.170 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:15.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:15.171 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:15.272 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:15.296 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:15.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:15.297 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:15.397 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:15.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:15.420 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:15.420 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:15.474 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:33:15.520 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:15.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:15 smithi104 ceph-mon[33507]: 2.5 scrub starts 2024-09-11T07:33:15.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:15 smithi104 ceph-mon[33507]: 2.5 scrub ok 2024-09-11T07:33:15.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:15 smithi104 ceph-mon[33507]: 6.6 scrub starts 2024-09-11T07:33:15.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:15 smithi104 ceph-mon[33507]: 6.6 scrub ok 2024-09-11T07:33:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:15 smithi104 ceph-mon[33507]: 5.1e scrub starts 2024-09-11T07:33:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:15 smithi104 ceph-mon[33507]: 5.1e scrub ok 2024-09-11T07:33:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:15 smithi104 ceph-mon[33507]: from='client.24998 ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T07:33:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:15 smithi104 ceph-mon[33507]: osdmap e173: 8 total, 8 up, 8 in 2024-09-11T07:33:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:15 smithi104 ceph-mon[33507]: pgmap v287: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 87 B/s, 0 objects/s recovering 2024-09-11T07:33:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:15 smithi104 ceph-mon[33507]: 3.0 scrub starts 2024-09-11T07:33:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:15 smithi104 ceph-mon[33507]: 3.0 scrub ok 2024-09-11T07:33:15.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:15.544 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:15.544 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:15.645 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:15.667 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:15.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:15.668 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:15.769 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:15.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:15.792 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:15.792 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:15.893 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:15.916 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:15.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:15.917 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:16.017 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:16.039 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:16.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:16.040 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:16.076 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:33:16.076 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:33:16.141 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:16.164 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:16.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:16.165 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:16.265 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:16.287 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:16.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:16.288 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:16.349 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:16 smithi022 ceph-mon[29462]: 4.b scrub starts 2024-09-11T07:33:16.349 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:16 smithi022 ceph-mon[29462]: 4.b scrub ok 2024-09-11T07:33:16.349 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:16 smithi022 ceph-mon[29462]: 5.16 scrub starts 2024-09-11T07:33:16.349 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:16 smithi022 ceph-mon[29462]: 5.16 scrub ok 2024-09-11T07:33:16.349 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:16 smithi022 ceph-mon[29462]: 4.e scrub starts 2024-09-11T07:33:16.349 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:16 smithi022 ceph-mon[29462]: 4.e scrub ok 2024-09-11T07:33:16.349 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:16 smithi022 ceph-mon[29462]: 2.17 scrub starts 2024-09-11T07:33:16.349 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:16 smithi022 ceph-mon[29462]: 2.17 scrub ok 2024-09-11T07:33:16.349 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:16 smithi022 ceph-mon[37330]: 4.b scrub starts 2024-09-11T07:33:16.350 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:16 smithi022 ceph-mon[37330]: 4.b scrub ok 2024-09-11T07:33:16.350 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:16 smithi022 ceph-mon[37330]: 5.16 scrub starts 2024-09-11T07:33:16.350 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:16 smithi022 ceph-mon[37330]: 5.16 scrub ok 2024-09-11T07:33:16.350 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:16 smithi022 ceph-mon[37330]: 4.e scrub starts 2024-09-11T07:33:16.350 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:16 smithi022 ceph-mon[37330]: 4.e scrub ok 2024-09-11T07:33:16.350 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:16 smithi022 ceph-mon[37330]: 2.17 scrub starts 2024-09-11T07:33:16.350 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:16 smithi022 ceph-mon[37330]: 2.17 scrub ok 2024-09-11T07:33:16.389 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:16.411 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:16.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:16.412 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:16.513 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:16.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:16 smithi104 ceph-mon[33507]: 4.b scrub starts 2024-09-11T07:33:16.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:16 smithi104 ceph-mon[33507]: 4.b scrub ok 2024-09-11T07:33:16.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:16 smithi104 ceph-mon[33507]: 5.16 scrub starts 2024-09-11T07:33:16.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:16 smithi104 ceph-mon[33507]: 5.16 scrub ok 2024-09-11T07:33:16.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:16 smithi104 ceph-mon[33507]: 4.e scrub starts 2024-09-11T07:33:16.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:16 smithi104 ceph-mon[33507]: 4.e scrub ok 2024-09-11T07:33:16.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:16 smithi104 ceph-mon[33507]: 2.17 scrub starts 2024-09-11T07:33:16.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:16 smithi104 ceph-mon[33507]: 2.17 scrub ok 2024-09-11T07:33:16.535 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:16.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:16.535 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:16.636 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:16.659 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:16.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:16.660 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:16.755 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-11T07:33:16.761 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:16.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:16.784 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:16.784 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:16.885 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:16.908 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:16.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:16.909 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:17.009 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:17.096 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:17.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:17.097 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:17.198 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:17.220 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:17.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:17.221 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:17.322 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:17.344 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:17.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:17.345 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:17.396 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:17 smithi022 ceph-mon[37330]: 6.1c scrub starts 2024-09-11T07:33:17.396 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:17 smithi022 ceph-mon[37330]: 6.1c scrub ok 2024-09-11T07:33:17.396 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:17 smithi022 ceph-mon[37330]: pgmap v288: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s; 59 B/s, 0 objects/s recovering 2024-09-11T07:33:17.396 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:17 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:33:17.396 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:17 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:33:17.396 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:17 smithi022 ceph-mon[37330]: 5.11 scrub starts 2024-09-11T07:33:17.396 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:17 smithi022 ceph-mon[37330]: 5.11 scrub ok 2024-09-11T07:33:17.396 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:17 smithi022 ceph-mon[29462]: 6.1c scrub starts 2024-09-11T07:33:17.396 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:17 smithi022 ceph-mon[29462]: 6.1c scrub ok 2024-09-11T07:33:17.397 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:17 smithi022 ceph-mon[29462]: pgmap v288: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s; 59 B/s, 0 objects/s recovering 2024-09-11T07:33:17.397 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:17 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:33:17.397 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:17 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:33:17.397 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:17 smithi022 ceph-mon[29462]: 5.11 scrub starts 2024-09-11T07:33:17.397 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:17 smithi022 ceph-mon[29462]: 5.11 scrub ok 2024-09-11T07:33:17.446 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:17.469 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:17.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:17.470 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:17.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:17 smithi104 ceph-mon[33507]: 6.1c scrub starts 2024-09-11T07:33:17.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:17 smithi104 ceph-mon[33507]: 6.1c scrub ok 2024-09-11T07:33:17.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:17 smithi104 ceph-mon[33507]: pgmap v288: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s; 59 B/s, 0 objects/s recovering 2024-09-11T07:33:17.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:17 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:33:17.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:17 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:33:17.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:17 smithi104 ceph-mon[33507]: 5.11 scrub starts 2024-09-11T07:33:17.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:17 smithi104 ceph-mon[33507]: 5.11 scrub ok 2024-09-11T07:33:17.571 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:17.631 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:17.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:17.632 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:17.733 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:17.757 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:17.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:17.759 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:17.860 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:17.882 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:17.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:17.883 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:17.984 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:18.006 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:18.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:18.007 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:18.074 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:33:18.074 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:33:18.090 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:33:18.090 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:33:18.105 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:33:18.105 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:33:18.109 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:18.120 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:33:18.120 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:33:18.131 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:18.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:18.133 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:18.136 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:33:18.136 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:33:18.152 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:33:18.152 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:33:18.168 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:33:18.168 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:33:18.183 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:33:18.184 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:33:18.233 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:18.255 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:18.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:18.256 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:18.357 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:18.383 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:18.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:18.384 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:18.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:18 smithi022 ceph-mon[29462]: 6.0 scrub starts 2024-09-11T07:33:18.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:18 smithi022 ceph-mon[29462]: 6.0 scrub ok 2024-09-11T07:33:18.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:18 smithi022 ceph-mon[29462]: osdmap e174: 8 total, 8 up, 8 in 2024-09-11T07:33:18.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:18 smithi022 ceph-mon[29462]: 2.d scrub starts 2024-09-11T07:33:18.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:18 smithi022 ceph-mon[29462]: 2.d scrub ok 2024-09-11T07:33:18.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:18 smithi022 ceph-mon[37330]: 6.0 scrub starts 2024-09-11T07:33:18.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:18 smithi022 ceph-mon[37330]: 6.0 scrub ok 2024-09-11T07:33:18.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:18 smithi022 ceph-mon[37330]: osdmap e174: 8 total, 8 up, 8 in 2024-09-11T07:33:18.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:18 smithi022 ceph-mon[37330]: 2.d scrub starts 2024-09-11T07:33:18.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:18 smithi022 ceph-mon[37330]: 2.d scrub ok 2024-09-11T07:33:18.485 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:18.508 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:18.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:18.509 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:18.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:18 smithi104 ceph-mon[33507]: 6.0 scrub starts 2024-09-11T07:33:18.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:18 smithi104 ceph-mon[33507]: 6.0 scrub ok 2024-09-11T07:33:18.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:18 smithi104 ceph-mon[33507]: osdmap e174: 8 total, 8 up, 8 in 2024-09-11T07:33:18.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:18 smithi104 ceph-mon[33507]: 2.d scrub starts 2024-09-11T07:33:18.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:18 smithi104 ceph-mon[33507]: 2.d scrub ok 2024-09-11T07:33:18.570 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-11T07:33:18.571 INFO:tasks.radosbench:Beginning radosbench... 2024-09-11T07:33:18.572 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_1 2024-09-11T07:33:18.572 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pool create unique_pool_1 16 2024-09-11T07:33:18.610 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:18.637 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:18.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:18.638 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:18.739 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:18.761 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:18.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:18.762 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:18.863 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:18.889 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:18.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:18.890 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:18.991 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:19.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:19.016 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:19.016 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:19.054 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is set 2024-09-11T07:33:19.117 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:19.139 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:19.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:19.140 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:19.241 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:19.263 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:19.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:19.264 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:19.309 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:19 smithi104 ceph-mon[33507]: 2.8 scrub starts 2024-09-11T07:33:19.309 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:19 smithi104 ceph-mon[33507]: 2.8 scrub ok 2024-09-11T07:33:19.309 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:19 smithi104 ceph-mon[33507]: 6.1f scrub starts 2024-09-11T07:33:19.309 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:19 smithi104 ceph-mon[33507]: 6.1f scrub ok 2024-09-11T07:33:19.309 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:19 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3250465326' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:33:19.309 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:19 smithi104 ceph-mon[33507]: pgmap v290: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s; 59 B/s, 0 objects/s recovering 2024-09-11T07:33:19.309 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:19 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/207756787' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:33:19.309 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:19 smithi104 ceph-mon[33507]: from='client.25004 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:33:19.309 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:19 smithi104 ceph-mon[33507]: 5.18 scrub starts 2024-09-11T07:33:19.310 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:19 smithi104 ceph-mon[33507]: 5.18 scrub ok 2024-09-11T07:33:19.365 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:19.378 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:19 smithi022 ceph-mon[29462]: 2.8 scrub starts 2024-09-11T07:33:19.379 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:19 smithi022 ceph-mon[29462]: 2.8 scrub ok 2024-09-11T07:33:19.379 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:19 smithi022 ceph-mon[29462]: 6.1f scrub starts 2024-09-11T07:33:19.379 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:19 smithi022 ceph-mon[29462]: 6.1f scrub ok 2024-09-11T07:33:19.379 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:19 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3250465326' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:33:19.379 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:19 smithi022 ceph-mon[29462]: pgmap v290: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s; 59 B/s, 0 objects/s recovering 2024-09-11T07:33:19.379 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:19 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/207756787' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:33:19.379 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:19 smithi022 ceph-mon[29462]: from='client.25004 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:33:19.379 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:19 smithi022 ceph-mon[29462]: 5.18 scrub starts 2024-09-11T07:33:19.379 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:19 smithi022 ceph-mon[29462]: 5.18 scrub ok 2024-09-11T07:33:19.379 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:19 smithi022 ceph-mon[37330]: 2.8 scrub starts 2024-09-11T07:33:19.380 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:19 smithi022 ceph-mon[37330]: 2.8 scrub ok 2024-09-11T07:33:19.380 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:19 smithi022 ceph-mon[37330]: 6.1f scrub starts 2024-09-11T07:33:19.380 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:19 smithi022 ceph-mon[37330]: 6.1f scrub ok 2024-09-11T07:33:19.380 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:19 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3250465326' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:33:19.380 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:19 smithi022 ceph-mon[37330]: pgmap v290: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s; 59 B/s, 0 objects/s recovering 2024-09-11T07:33:19.380 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:19 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/207756787' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:33:19.380 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:19 smithi022 ceph-mon[37330]: from='client.25004 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:33:19.380 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:19 smithi022 ceph-mon[37330]: 5.18 scrub starts 2024-09-11T07:33:19.380 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:19 smithi022 ceph-mon[37330]: 5.18 scrub ok 2024-09-11T07:33:19.391 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:19.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:19.392 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:19.493 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:19.518 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:19.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:19.519 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:19.620 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:19.644 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:19.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:19.645 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:19.746 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:19.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:19.771 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:19.771 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:19.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:33:19 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:33:19.872 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:19.895 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:19.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:19.896 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:19.997 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:20.020 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:20.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:20.021 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:20.121 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:20.144 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:20.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:20.145 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:20.236 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:20 smithi022 ceph-mon[37330]: 4.6 scrub starts 2024-09-11T07:33:20.237 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:20 smithi022 ceph-mon[37330]: 4.6 scrub ok 2024-09-11T07:33:20.237 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:20 smithi022 ceph-mon[37330]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:33:20.237 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:20 smithi022 ceph-mon[37330]: from='client.25004 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:33:20.237 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:20 smithi022 ceph-mon[37330]: osdmap e175: 8 total, 8 up, 8 in 2024-09-11T07:33:20.237 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:20 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:20.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:20 smithi022 ceph-mon[29462]: 4.6 scrub starts 2024-09-11T07:33:20.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:20 smithi022 ceph-mon[29462]: 4.6 scrub ok 2024-09-11T07:33:20.238 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:20 smithi022 ceph-mon[29462]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:33:20.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:20 smithi022 ceph-mon[29462]: from='client.25004 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:33:20.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:20 smithi022 ceph-mon[29462]: osdmap e175: 8 total, 8 up, 8 in 2024-09-11T07:33:20.239 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:20 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:20.245 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:20.272 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:20.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:20.274 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:20.375 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:20.400 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:20.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:20.401 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:20.496 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:33:20 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:33:20] "GET /metrics HTTP/1.1" 200 223793 "" "Prometheus/2.43.0" 2024-09-11T07:33:20.502 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:20.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:20 smithi104 ceph-mon[33507]: 4.6 scrub starts 2024-09-11T07:33:20.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:20 smithi104 ceph-mon[33507]: 4.6 scrub ok 2024-09-11T07:33:20.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:20 smithi104 ceph-mon[33507]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:33:20.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:20 smithi104 ceph-mon[33507]: from='client.25004 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:33:20.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:20 smithi104 ceph-mon[33507]: osdmap e175: 8 total, 8 up, 8 in 2024-09-11T07:33:20.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:20 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:20.533 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:20.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:20.534 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:20.635 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:20.660 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:20.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:20.661 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:20.762 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:20.766 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:33:20.786 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:20.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:20.787 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:20.887 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:20.909 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:20.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:20.910 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:21.011 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:21.035 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:21.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:21.036 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:21.137 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:21.159 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:21.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:21.160 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:21.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:21 smithi022 ceph-mon[29462]: pgmap v292: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T07:33:21.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:21 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1328688852' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T07:33:21.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:21 smithi022 ceph-mon[29462]: from='client.25016 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T07:33:21.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:21 smithi022 ceph-mon[37330]: pgmap v292: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T07:33:21.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:21 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1328688852' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T07:33:21.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:21 smithi022 ceph-mon[37330]: from='client.25016 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T07:33:21.261 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:21.287 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:21.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:21.288 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:21.389 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:21.412 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:21.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:21.413 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:21.514 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:21.523 INFO:teuthology.orchestra.run.smithi022.stderr:reweighted osd.3 to 1 (10000) 2024-09-11T07:33:21.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:21 smithi104 ceph-mon[33507]: pgmap v292: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T07:33:21.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:21 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1328688852' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T07:33:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:21 smithi104 ceph-mon[33507]: from='client.25016 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T07:33:21.540 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:21.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:21.541 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:21.642 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:21.664 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:21.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:21.665 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:21.766 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:21.789 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:21.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:21.790 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:21.891 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:21.916 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:21.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:21.917 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:21.983 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:33:22.018 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:22.043 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:22.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:22.044 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:22.144 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:22.167 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:22.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:22.168 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:22.269 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:22.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:22.296 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:22.297 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:22.397 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:22.420 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:22.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:22.421 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:22.522 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:22.548 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:22.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:22.549 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:22.649 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:22.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:22 smithi022 ceph-mon[37330]: from='client.25016 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-11T07:33:22.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:22 smithi022 ceph-mon[37330]: osdmap e176: 8 total, 8 up, 8 in 2024-09-11T07:33:22.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:22 smithi022 ceph-mon[37330]: pgmap v294: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:22.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:22 smithi022 ceph-mon[29462]: from='client.25016 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-11T07:33:22.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:22 smithi022 ceph-mon[29462]: osdmap e176: 8 total, 8 up, 8 in 2024-09-11T07:33:22.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:22 smithi022 ceph-mon[29462]: pgmap v294: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:22.679 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:22.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:22.681 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:22.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:22 smithi104 ceph-mon[33507]: from='client.25016 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-11T07:33:22.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:22 smithi104 ceph-mon[33507]: osdmap e176: 8 total, 8 up, 8 in 2024-09-11T07:33:22.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:22 smithi104 ceph-mon[33507]: pgmap v294: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:22.782 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:22.807 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:22.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:22.808 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:22.811 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight 4 1 2024-09-11T07:33:22.909 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:22.932 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:22.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:22.933 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:23.034 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:23.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:23.057 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:23.058 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:23.159 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:23.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:23.182 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:23.282 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:23.308 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:23.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:23.309 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:23.338 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:33:23.339 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:33:23.410 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:23.436 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:23.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:23.437 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:23.538 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:23.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:23.561 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:23.561 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:23.662 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:23.684 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:23.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:23.685 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:23.786 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:23.812 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:23.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:23.813 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:23.912 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T07:33:23.914 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:23.939 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:23.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:23.940 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:24.041 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:24.067 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:24.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:24.068 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:24.169 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:24.192 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:24.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:24.193 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:24.280 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:33:24 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:33:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:33:24.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:24 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:33:24.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:24 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:33:24.293 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:24.315 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:24.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:24.316 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:24.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:24 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:33:24.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:24 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:33:24.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:24 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:33:24.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:24 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:33:24.418 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:24.440 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:24.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:24.441 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:24.542 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:24.565 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:24.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:24.566 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:24.666 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:24.688 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:24.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:24.689 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:24.790 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:24.800 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:33:24.817 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:24.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:24.818 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:24.919 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:24.941 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:24.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:24.942 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:25.042 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:25.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:25.070 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:25.070 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:25.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:25 smithi022 ceph-mon[37330]: pgmap v295: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 727 B/s rd, 0 op/s 2024-09-11T07:33:25.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:25 smithi022 ceph-mon[29462]: pgmap v295: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 727 B/s rd, 0 op/s 2024-09-11T07:33:25.171 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:25.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:25.195 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:25.195 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:25.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:25 smithi104 ceph-mon[33507]: pgmap v295: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 727 B/s rd, 0 op/s 2024-09-11T07:33:25.296 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:25.319 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:25.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:25.320 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:25.421 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:25.444 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:25.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:25.445 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:25.545 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:25.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:25.570 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:25.570 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:25.671 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:25.696 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:25.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:25.698 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:25.799 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:25.824 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:25.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:25.825 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:25.926 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:25.948 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:25.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:25.949 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:25.953 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:33:25.953 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:33:25.953 INFO:teuthology.orchestra.run.smithi022.stdout: "num_ops": 0 2024-09-11T07:33:25.953 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:33:26.050 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:26.072 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:26.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:26.073 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:26.174 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:26.196 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:26.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:26.197 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:26.298 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:26.324 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:26.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:26.325 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:26.426 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:26.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:26.450 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:26.450 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:26.550 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:26.573 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:26.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:26.574 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:26.577 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-11T07:33:26.675 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:26.696 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:26.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:26.697 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:26.798 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:26.822 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:26.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:26.824 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:26.924 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:26.948 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:26.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:26.949 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:27.050 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:27.072 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:27.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:27.073 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:27.174 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:27.198 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:27.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:27.199 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:27.300 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:27.324 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:27.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:27.325 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:27.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:27 smithi022 ceph-mon[29462]: pgmap v296: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:27.407 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:27 smithi022 ceph-mon[37330]: pgmap v296: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:27.414 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:33:27.426 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:27.453 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:27.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:27.454 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:27.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:27 smithi104 ceph-mon[33507]: pgmap v296: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:27.555 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:27.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:27.582 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:27.582 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:27.683 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:27.708 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:27.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:27.709 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:27.810 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:27.832 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:27.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:27.833 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:27.934 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:27.956 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:27.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:27.957 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:28.058 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:28.084 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:28.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:28.085 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:28.148 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'unique_pool_1' created 2024-09-11T07:33:28.185 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:28.207 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:28.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:28.208 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:28.309 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:28.332 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:28.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:28.333 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:28.343 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:28 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/530923436' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2024-09-11T07:33:28.344 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:28 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/530923436' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2024-09-11T07:33:28.434 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:28.460 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:28.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:28.461 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:28.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:28 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/530923436' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2024-09-11T07:33:28.562 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:28.585 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:28.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:28.586 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:28.687 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:28.709 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:28.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:28.710 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:28.811 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:28.833 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:28.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:28.834 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:28.935 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:28.961 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:28.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:28.962 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:29.062 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:29.087 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:29.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:29.088 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:29.152 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:29 smithi022 ceph-mon[29462]: pgmap v297: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:33:29.153 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:29 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/530923436' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-09-11T07:33:29.153 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:29 smithi022 ceph-mon[29462]: osdmap e177: 8 total, 8 up, 8 in 2024-09-11T07:33:29.153 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:29 smithi022 ceph-mon[37330]: pgmap v297: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:33:29.153 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:29 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/530923436' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-09-11T07:33:29.153 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:29 smithi022 ceph-mon[37330]: osdmap e177: 8 total, 8 up, 8 in 2024-09-11T07:33:29.189 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:29.214 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:29.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:29.215 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:29.315 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:29.339 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:29.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:29.340 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:29.441 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:29.464 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:29.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:29.465 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:29.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:33:29 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:33:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:29 smithi104 ceph-mon[33507]: pgmap v297: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:33:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:29 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/530923436' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-09-11T07:33:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:29 smithi104 ceph-mon[33507]: osdmap e177: 8 total, 8 up, 8 in 2024-09-11T07:33:29.565 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:29.591 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:29.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:29.592 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:29.693 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:29.717 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:29.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:29.718 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:29.819 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:29.842 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:29.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:29.843 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:29.944 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:29.966 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:29.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:29.967 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:30.068 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:30.094 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:30.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:30.095 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:30.195 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:30.218 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:30.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:30.220 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:30.227 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:33:30.228 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:33:30.244 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:33:30.244 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:33:30.259 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:33:30.259 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:33:30.274 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:33:30.274 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:33:30.290 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:33:30.290 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:33:30.305 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:33:30.305 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:33:30.321 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:30.321 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:33:30.322 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:33:30.337 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:33:30.337 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:33:30.347 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:30.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:30.348 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:30.417 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:33:30 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:33:30] "GET /metrics HTTP/1.1" 200 221466 "" "Prometheus/2.43.0" 2024-09-11T07:33:30.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:30 smithi022 ceph-mon[29462]: osdmap e178: 8 total, 8 up, 8 in 2024-09-11T07:33:30.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:30 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:30.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:30 smithi022 ceph-mon[37330]: osdmap e178: 8 total, 8 up, 8 in 2024-09-11T07:33:30.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:30 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:30.448 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:30.472 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:30.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:30.473 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:30.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:30 smithi104 ceph-mon[33507]: osdmap e178: 8 total, 8 up, 8 in 2024-09-11T07:33:30.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:30 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:30.574 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:30.598 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:30.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:30.599 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:30.700 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:30.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:30.725 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:30.725 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:30.826 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:30.848 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:30.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:30.849 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:30.950 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:30.976 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:30.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:30.977 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:31.078 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:31.100 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:31.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:31.101 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:31.186 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pool application enable unique_pool_1 rados --yes-i-really-mean-it || true 2024-09-11T07:33:31.202 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:31.228 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:31.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:31.229 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:31.330 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:31.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:31.353 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:31.353 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:31.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:31 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:33:31.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:31 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:33:31.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:31 smithi022 ceph-mon[29462]: pgmap v300: 148 pgs: 6 creating+peering, 142 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:31.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:31 smithi022 ceph-mon[29462]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:33:31.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:31 smithi022 ceph-mon[29462]: osdmap e179: 8 total, 8 up, 8 in 2024-09-11T07:33:31.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:31 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/572247118' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:33:31.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:31 smithi022 ceph-mon[29462]: osdmap e180: 8 total, 8 up, 8 in 2024-09-11T07:33:31.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:31 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:33:31.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:31 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:33:31.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:31 smithi022 ceph-mon[37330]: pgmap v300: 148 pgs: 6 creating+peering, 142 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:31.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:31 smithi022 ceph-mon[37330]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:33:31.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:31 smithi022 ceph-mon[37330]: osdmap e179: 8 total, 8 up, 8 in 2024-09-11T07:33:31.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:31 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/572247118' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:33:31.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:31 smithi022 ceph-mon[37330]: osdmap e180: 8 total, 8 up, 8 in 2024-09-11T07:33:31.454 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:31.479 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:31.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:31.480 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:31.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:31 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:33:31.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:31 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:33:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:31 smithi104 ceph-mon[33507]: pgmap v300: 148 pgs: 6 creating+peering, 142 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:31 smithi104 ceph-mon[33507]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:33:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:31 smithi104 ceph-mon[33507]: osdmap e179: 8 total, 8 up, 8 in 2024-09-11T07:33:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:31 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/572247118' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:33:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:31 smithi104 ceph-mon[33507]: osdmap e180: 8 total, 8 up, 8 in 2024-09-11T07:33:31.581 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:31.605 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:31.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:31.606 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:31.706 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:31.728 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:31.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:31.729 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:31.831 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:31.857 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:31.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:31.858 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:31.958 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:31.984 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:31.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:31.985 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:32.086 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:32.111 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:32.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:32.112 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:32.166 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is set 2024-09-11T07:33:32.213 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:32.238 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:32.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:32.239 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:32.339 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:32.363 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:32.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:32.364 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:32.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:32 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/154727491' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:33:32.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:32 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/154727491' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:33:32.465 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:32.488 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:32.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:32.489 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:32.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:32 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/154727491' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:33:32.580 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:33:32.590 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:32.595 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:33:32.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:32.615 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:32.615 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:32.716 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:32.740 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:32.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:32.741 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:32.842 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:32.867 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:32.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:32.868 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:32.969 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:32.994 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:32.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:32.995 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:33.096 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:33.122 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:33.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:33.124 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:33.167 INFO:teuthology.orchestra.run.smithi022.stderr:reweighted osd.4 to 1 (10000) 2024-09-11T07:33:33.225 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:33.248 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:33.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:33.249 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:33.350 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:33.376 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:33.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:33.377 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:33.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:33 smithi022 ceph-mon[29462]: pgmap v303: 148 pgs: 6 creating+peering, 142 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:33.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:33 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:33:33.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:33 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:33:33.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:33 smithi022 ceph-mon[29462]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:33:33.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:33 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/154727491' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:33:33.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:33 smithi022 ceph-mon[29462]: osdmap e181: 8 total, 8 up, 8 in 2024-09-11T07:33:33.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:33 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4056459505' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T07:33:33.409 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:33 smithi022 ceph-mon[37330]: pgmap v303: 148 pgs: 6 creating+peering, 142 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:33.410 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:33 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:33:33.410 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:33 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:33:33.410 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:33 smithi022 ceph-mon[37330]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:33:33.410 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:33 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/154727491' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:33:33.410 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:33 smithi022 ceph-mon[37330]: osdmap e181: 8 total, 8 up, 8 in 2024-09-11T07:33:33.410 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:33 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4056459505' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T07:33:33.477 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:33.502 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:33.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:33.503 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:33.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:33 smithi104 ceph-mon[33507]: pgmap v303: 148 pgs: 6 creating+peering, 142 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:33:33.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:33 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:33:33.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:33 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:33:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:33 smithi104 ceph-mon[33507]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:33:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:33 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/154727491' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:33:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:33 smithi104 ceph-mon[33507]: osdmap e181: 8 total, 8 up, 8 in 2024-09-11T07:33:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:33 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4056459505' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T07:33:33.603 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:33.627 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:33.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:33.628 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:33.728 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:33.751 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:33.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:33.752 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:33.790 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:33:33.853 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:33.879 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:33.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:33.880 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:33.981 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:34.003 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:34.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:34.004 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:34.105 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:34.116 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight 5 1 2024-09-11T07:33:34.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:34.128 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:34.128 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:34.230 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:34.252 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:34.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:34.253 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:34.354 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:34.376 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:34.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:34.377 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:34.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:34 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4056459505' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-11T07:33:34.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:34 smithi022 ceph-mon[29462]: osdmap e182: 8 total, 8 up, 8 in 2024-09-11T07:33:34.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:34 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4056459505' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-11T07:33:34.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:34 smithi022 ceph-mon[37330]: osdmap e182: 8 total, 8 up, 8 in 2024-09-11T07:33:34.479 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:34.505 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:34.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:34.506 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:34.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:33:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:33:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:33:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:34 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4056459505' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-11T07:33:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:34 smithi104 ceph-mon[33507]: osdmap e182: 8 total, 8 up, 8 in 2024-09-11T07:33:34.606 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:34.629 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:34.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:34.630 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:34.731 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:34.754 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:34.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:34.756 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:34.857 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:34.883 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:34.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:34.884 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:34.985 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:35.011 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:35.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:35.012 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:35.113 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:35.136 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:35.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:35.137 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:35.238 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:35.264 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:35.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:35.265 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:35.366 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:35.388 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:35.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:35.389 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:35.490 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:35.515 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:35.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:35.516 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:35.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:35 smithi104 ceph-mon[33507]: pgmap v306: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:33:35.617 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:35.641 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:35 smithi022 ceph-mon[29462]: pgmap v306: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:33:35.641 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:35 smithi022 ceph-mon[37330]: pgmap v306: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:33:35.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:35.643 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:35.644 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:35.744 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:35.767 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:35.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:35.768 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:35.869 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:35.891 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:35.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:35.892 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:35.993 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:36.016 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:36.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:36.018 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:36.118 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:36.126 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:33:36.144 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:36.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:36.145 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:36.246 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:36.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:36.270 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:36.270 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:36.371 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:36.385 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:33:36.396 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:36.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:36.397 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:36.498 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:36.527 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:36.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:36.529 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:36.630 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:36.653 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:36.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:36.654 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:36.684 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T07:33:36.755 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:36.779 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:36.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:36 smithi104 ceph-mon[33507]: osdmap e183: 8 total, 8 up, 8 in 2024-09-11T07:33:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:36 smithi104 ceph-mon[33507]: pgmap v308: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 922 B/s rd, 0 op/s 2024-09-11T07:33:36.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:36.780 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:36.809 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:36 smithi022 ceph-mon[29462]: osdmap e183: 8 total, 8 up, 8 in 2024-09-11T07:33:36.809 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:36 smithi022 ceph-mon[29462]: pgmap v308: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 922 B/s rd, 0 op/s 2024-09-11T07:33:36.809 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:36 smithi022 ceph-mon[37330]: osdmap e183: 8 total, 8 up, 8 in 2024-09-11T07:33:36.809 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:36 smithi022 ceph-mon[37330]: pgmap v308: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 922 B/s rd, 0 op/s 2024-09-11T07:33:36.881 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:36.905 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:36.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:36.906 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:37.007 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:37.031 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:37.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:37.031 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:37.132 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:37.155 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:37.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:37.156 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:37.256 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:37.278 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:37.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:37.280 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:37.380 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:37.406 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:37.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:37.407 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:37.508 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:37.535 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:37.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:37.536 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:37.546 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:33:37.546 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:33:37.604 INFO:teuthology.orchestra.run.smithi022.stderr:enabled application 'rados' on pool 'unique_pool_1' 2024-09-11T07:33:37.637 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:37.663 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:37.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:37.664 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:37.765 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:37.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:37.791 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:37.791 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:37.892 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:37.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:37.915 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:37.915 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:37.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:37 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/245445373' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:33:37.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:37 smithi022 ceph-mon[29462]: from='client.25046 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:33:37.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:37 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/245445373' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:33:37.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:37 smithi022 ceph-mon[37330]: from='client.25046 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:33:38.016 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:38.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:37 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/245445373' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:33:38.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:37 smithi104 ceph-mon[33507]: from='client.25046 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:33:38.041 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:38.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:38.042 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:38.142 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:38.165 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:38.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:38.166 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:38.266 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:38.292 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:38.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:38.293 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:38.394 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:38.419 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:38.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:38.420 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:38.521 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:38.546 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:38.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:38.547 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:38.647 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:38.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:38.672 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:38.673 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:38.773 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:38.798 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:38.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:38.799 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:38.899 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:38.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:38 smithi022 ceph-mon[29462]: from='client.25046 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:33:38.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:38 smithi022 ceph-mon[29462]: osdmap e184: 8 total, 8 up, 8 in 2024-09-11T07:33:38.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:33:38.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:33:38.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:38 smithi022 ceph-mon[29462]: pgmap v310: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 863 B/s rd, 0 op/s 2024-09-11T07:33:38.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:38 smithi022 ceph-mon[37330]: from='client.25046 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:33:38.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:38 smithi022 ceph-mon[37330]: osdmap e184: 8 total, 8 up, 8 in 2024-09-11T07:33:38.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:33:38.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:33:38.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:38 smithi022 ceph-mon[37330]: pgmap v310: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 863 B/s rd, 0 op/s 2024-09-11T07:33:38.925 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:38.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:38.926 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:39.026 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:39.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:38 smithi104 ceph-mon[33507]: from='client.25046 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:33:39.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:38 smithi104 ceph-mon[33507]: osdmap e184: 8 total, 8 up, 8 in 2024-09-11T07:33:39.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:33:39.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:33:39.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:38 smithi104 ceph-mon[33507]: pgmap v310: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 863 B/s rd, 0 op/s 2024-09-11T07:33:39.051 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:39.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:39.052 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:39.153 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:39.177 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:39.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:39.178 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:39.279 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:39.303 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:39.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:39.304 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:39.405 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:39.432 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:39.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:39.433 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:39.534 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:39.560 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:39.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:39.561 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:39.608 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:33:39 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:33:39.661 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:39.687 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:39.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:39.688 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:39.789 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:39.813 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:39.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:39.814 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:39.915 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:39.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:39 smithi022 ceph-mon[37330]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:33:39.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:39 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:39.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:39 smithi022 ceph-mon[29462]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:33:39.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:39 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:39.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:39.938 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:39.938 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:40.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:39 smithi104 ceph-mon[33507]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:33:40.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:39 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:40.039 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:40.061 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:40.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:40.062 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:40.138 DEBUG:teuthology.orchestra.run.smithi022:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_1 bench 90 write' 2024-09-11T07:33:40.140 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-11T07:33:40.140 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-11T07:33:40.163 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:40.184 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:40.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:40.185 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:40.198 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:hints = 1 2024-09-11T07:33:40.198 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-11T07:33:40.198 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Object prefix: benchmark_data_smithi022_108406 2024-09-11T07:33:40.198 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T07:33:40.198 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 0 0 0 0 0 0 - 0 2024-09-11T07:33:40.209 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-11T07:33:40.286 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:40.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:40.352 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:40.352 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:40.454 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:40.525 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:40.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:40.527 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:40.627 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:40.637 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:33:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:33:40] "GET /metrics HTTP/1.1" 200 223727 "" "Prometheus/2.43.0" 2024-09-11T07:33:40.637 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:40 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:33:40.637 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:40 smithi022 ceph-mon[37330]: pgmap v311: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T07:33:40.637 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:40 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:33:40.637 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:40 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:33:40.637 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:40 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:33:40.637 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:40 smithi022 ceph-mon[37330]: osdmap e185: 8 total, 8 up, 8 in 2024-09-11T07:33:40.638 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:40 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:33:40.638 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:40 smithi022 ceph-mon[29462]: pgmap v311: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T07:33:40.638 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:40 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:33:40.638 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:40 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:33:40.638 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:40 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:33:40.638 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:40 smithi022 ceph-mon[29462]: osdmap e185: 8 total, 8 up, 8 in 2024-09-11T07:33:40.670 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:40.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:40.671 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:40.772 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:40.813 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:40.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:40.815 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:40.915 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:40.944 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:40.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:40.945 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:41.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:40 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:33:41.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:40 smithi104 ceph-mon[33507]: pgmap v311: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T07:33:41.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:40 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:33:41.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:40 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:33:41.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:40 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:33:41.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:40 smithi104 ceph-mon[33507]: osdmap e185: 8 total, 8 up, 8 in 2024-09-11T07:33:41.046 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:41.074 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:41.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:41.075 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:41.176 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:41.199 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 1 16 2813 2797 174.712 174.812 0.00328496 0.00569482 2024-09-11T07:33:41.217 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:41.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:41.218 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:41.232 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:33:41.233 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:33:41.268 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:33:41.268 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:33:41.302 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:33:41.303 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:33:41.319 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:41.341 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:33:41.342 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:33:41.362 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:41.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:41.363 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:41.379 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:33:41.379 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:33:41.386 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:33:41.411 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:33:41.411 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:33:41.445 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:33:41.445 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:33:41.464 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:41.481 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:33:41.482 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:33:41.491 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:41.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:41.492 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:41.592 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:41.622 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:41.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:41.624 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:41.725 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:41.768 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:41.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:41.769 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:41.870 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:41.885 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:41 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3893430645' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:33:41.888 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:41 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3893430645' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:33:41.912 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:41.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:41.913 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:42.014 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:42.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:41 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3893430645' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:33:42.041 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:42.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:42.042 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:42.142 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:42.182 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:42.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:42.183 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:42.199 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 2 16 5542 5526 172.631 170.562 0.00419832 0.00577937 2024-09-11T07:33:42.284 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:42.321 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:42.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:42.322 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:42.423 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:42.465 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:42.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:42.467 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:42.568 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:42.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:42.610 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:42.610 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:42.650 INFO:teuthology.orchestra.run.smithi022.stderr:reweighted osd.5 to 1 (10000) 2024-09-11T07:33:42.711 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:42.746 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:42.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:42.747 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:42.848 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:42.880 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:42.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:42.881 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:42.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:42 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:33:42.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:42 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:33:42.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:42 smithi022 ceph-mon[29462]: osdmap e186: 8 total, 8 up, 8 in 2024-09-11T07:33:42.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:42 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3089490867' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T07:33:42.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:42 smithi022 ceph-mon[29462]: from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T07:33:42.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:42 smithi022 ceph-mon[29462]: pgmap v314: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:42.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:42 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:33:42.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:42 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:33:42.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:42 smithi022 ceph-mon[37330]: osdmap e186: 8 total, 8 up, 8 in 2024-09-11T07:33:42.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:42 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3089490867' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T07:33:42.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:42 smithi022 ceph-mon[37330]: from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T07:33:42.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:42 smithi022 ceph-mon[37330]: pgmap v314: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:42.982 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:43.011 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:43.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:43.012 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:43.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:42 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:33:43.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:42 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:33:43.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:42 smithi104 ceph-mon[33507]: osdmap e186: 8 total, 8 up, 8 in 2024-09-11T07:33:43.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:42 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3089490867' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T07:33:43.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:42 smithi104 ceph-mon[33507]: from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T07:33:43.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:42 smithi104 ceph-mon[33507]: pgmap v314: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:33:43.113 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:43.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:43.156 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:43.156 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:43.199 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 3 16 8910 8894 185.246 210.5 0.004235 0.00538809 2024-09-11T07:33:43.257 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:43.289 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:43.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:43.291 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:43.391 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:43.422 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:43.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:43.423 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:43.523 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:43.564 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:43.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:43.565 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:43.666 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:43.710 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:43.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:43.712 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:43.812 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:43.842 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:43.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:43.844 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:43.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:43 smithi022 ceph-mon[29462]: from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-11T07:33:43.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:43 smithi022 ceph-mon[29462]: osdmap e187: 8 total, 8 up, 8 in 2024-09-11T07:33:43.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:43 smithi022 ceph-mon[37330]: from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-11T07:33:43.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:43 smithi022 ceph-mon[37330]: osdmap e187: 8 total, 8 up, 8 in 2024-09-11T07:33:43.944 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:43.974 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:43.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:43.975 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:44.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:43 smithi104 ceph-mon[33507]: from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-11T07:33:44.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:43 smithi104 ceph-mon[33507]: osdmap e187: 8 total, 8 up, 8 in 2024-09-11T07:33:44.037 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight 6 1 2024-09-11T07:33:44.076 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:44.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:44.129 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:44.129 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:44.199 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 4 16 12112 12096 188.96 200.125 0.00341372 0.00528373 2024-09-11T07:33:44.230 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:44.258 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:44.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:44.259 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:44.360 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:44.390 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:44.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:44.391 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:44.492 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:44.509 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:33:44.519 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:44.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:44.520 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:44.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:33:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:33:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:33:44.621 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:44.652 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:44.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:44.653 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:44.673 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:33:44.743 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:44 smithi022 ceph-mon[29462]: pgmap v316: 148 pgs: 148 active+clean; 273 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 852 B/s rd, 45 MiB/s wr, 727 op/s 2024-09-11T07:33:44.743 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:44 smithi022 ceph-mon[37330]: pgmap v316: 148 pgs: 148 active+clean; 273 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 852 B/s rd, 45 MiB/s wr, 727 op/s 2024-09-11T07:33:44.754 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:44.779 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:44.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:44.780 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:44.881 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:44.919 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:44.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:44.921 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:45.021 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:45.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:44 smithi104 ceph-mon[33507]: pgmap v316: 148 pgs: 148 active+clean; 273 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 852 B/s rd, 45 MiB/s wr, 727 op/s 2024-09-11T07:33:45.048 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:45.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:45.049 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:45.150 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:45.171 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:45.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:45.172 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:45.200 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 5 16 14434 14418 180.191 145.125 0.00147772 0.00544863 2024-09-11T07:33:45.272 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:45.303 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:45.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:45.304 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:45.360 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:33:45.405 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:45.449 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:45.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:45.450 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:45.551 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:45.580 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:45.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:45.581 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:45.682 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:45.711 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:45.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:45.712 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:45.813 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:45.845 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:45.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:45.847 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:45.947 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:45.978 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:45.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:45.980 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:46.081 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:46.128 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:46.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:46.129 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:46.200 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 6 16 16965 16949 176.521 158.188 0.00911238 0.00565709 2024-09-11T07:33:46.230 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:46.268 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:46.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:46.270 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:46.371 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:46.409 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:46.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:46.410 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:46.511 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:46.540 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:46.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:46.541 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:46.642 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:46.680 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:46.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:46.682 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:46.783 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:46.811 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:46.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:46.812 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:46.913 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:46.943 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:46.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:46.944 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:47.045 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:47.086 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:47.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:47.087 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:47.170 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is unset 2024-09-11T07:33:47.188 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:47.200 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 7 16 19572 19556 174.579 162.938 0.00588399 0.00572122 2024-09-11T07:33:47.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:47.219 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:47.219 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:47.320 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:47.351 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:47.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:47.352 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:47.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:47 smithi022 ceph-mon[29462]: pgmap v317: 148 pgs: 148 active+clean; 770 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 852 B/s rd, 128 MiB/s wr, 2.05k op/s 2024-09-11T07:33:47.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:47 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/849308517' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:33:47.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:47 smithi022 ceph-mon[29462]: from='client.25087 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:33:47.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:47 smithi022 ceph-mon[37330]: pgmap v317: 148 pgs: 148 active+clean; 770 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 852 B/s rd, 128 MiB/s wr, 2.05k op/s 2024-09-11T07:33:47.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:47 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/849308517' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:33:47.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:47 smithi022 ceph-mon[37330]: from='client.25087 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:33:47.453 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:47.480 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:47.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:47.481 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:47.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:47 smithi104 ceph-mon[33507]: pgmap v317: 148 pgs: 148 active+clean; 770 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 852 B/s rd, 128 MiB/s wr, 2.05k op/s 2024-09-11T07:33:47.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:47 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/849308517' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:33:47.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:47 smithi104 ceph-mon[33507]: from='client.25087 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:33:47.582 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:47.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:47.617 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:47.617 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:47.718 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:47.755 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:47.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:47.756 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:47.857 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:47.896 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:47.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:47.897 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:47.998 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:48.030 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:48.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:48.031 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:48.132 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:48.170 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:48.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:48.171 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:48.200 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 8 16 22295 22279 174.028 170.188 0.00684045 0.00574055 2024-09-11T07:33:48.273 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:48.301 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:48.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:48.302 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:48.332 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:48 smithi022 ceph-mon[29462]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:33:48.332 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:48 smithi022 ceph-mon[29462]: from='client.25087 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:33:48.332 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:48 smithi022 ceph-mon[29462]: osdmap e188: 8 total, 8 up, 8 in 2024-09-11T07:33:48.332 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:48 smithi022 ceph-mon[37330]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:33:48.332 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:48 smithi022 ceph-mon[37330]: from='client.25087 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:33:48.332 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:48 smithi022 ceph-mon[37330]: osdmap e188: 8 total, 8 up, 8 in 2024-09-11T07:33:48.403 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:48.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:48.434 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:48.434 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:48.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:48 smithi104 ceph-mon[33507]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:33:48.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:48 smithi104 ceph-mon[33507]: from='client.25087 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:33:48.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:48 smithi104 ceph-mon[33507]: osdmap e188: 8 total, 8 up, 8 in 2024-09-11T07:33:48.535 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:48.566 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:48.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:48.567 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:48.610 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:33:48.667 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:48.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:48.713 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:48.813 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:48.845 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:48.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:48.846 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:48.947 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:48.985 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:48.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:48.986 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:49.086 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:49.125 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:49.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:49.126 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:49.200 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 9 16 24840 24824 172.364 159.062 0.0154778 0.00579584 2024-09-11T07:33:49.228 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:49.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:49.280 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:49.280 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:49.381 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:49.411 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:49.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:49.413 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:49.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:49 smithi022 ceph-mon[29462]: 5.14 scrub starts 2024-09-11T07:33:49.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:49 smithi022 ceph-mon[29462]: 5.14 scrub ok 2024-09-11T07:33:49.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:49 smithi022 ceph-mon[29462]: pgmap v319: 148 pgs: 148 active+clean; 770 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 793 B/s rd, 119 MiB/s wr, 1.91k op/s 2024-09-11T07:33:49.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:49 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:33:49.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:49 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:33:49.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:49 smithi022 ceph-mon[29462]: 2.f scrub starts 2024-09-11T07:33:49.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:49 smithi022 ceph-mon[29462]: 2.f scrub ok 2024-09-11T07:33:49.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:49 smithi022 ceph-mon[37330]: 5.14 scrub starts 2024-09-11T07:33:49.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:49 smithi022 ceph-mon[37330]: 5.14 scrub ok 2024-09-11T07:33:49.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:49 smithi022 ceph-mon[37330]: pgmap v319: 148 pgs: 148 active+clean; 770 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 793 B/s rd, 119 MiB/s wr, 1.91k op/s 2024-09-11T07:33:49.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:49 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:33:49.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:49 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:33:49.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:49 smithi022 ceph-mon[37330]: 2.f scrub starts 2024-09-11T07:33:49.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:49 smithi022 ceph-mon[37330]: 2.f scrub ok 2024-09-11T07:33:49.514 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:49.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:33:49 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:33:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:49 smithi104 ceph-mon[33507]: 5.14 scrub starts 2024-09-11T07:33:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:49 smithi104 ceph-mon[33507]: 5.14 scrub ok 2024-09-11T07:33:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:49 smithi104 ceph-mon[33507]: pgmap v319: 148 pgs: 148 active+clean; 770 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 793 B/s rd, 119 MiB/s wr, 1.91k op/s 2024-09-11T07:33:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:49 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:33:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:49 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:33:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:49 smithi104 ceph-mon[33507]: 2.f scrub starts 2024-09-11T07:33:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:49 smithi104 ceph-mon[33507]: 2.f scrub ok 2024-09-11T07:33:49.567 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:49.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:49.568 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:49.669 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:49.698 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:49.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:49.699 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:49.800 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:49.830 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:49.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:49.831 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:49.931 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:33:49.931 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 20, 2024-09-11T07:33:49.932 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 600, 2024-09-11T07:33:49.932 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:33:49.932 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.932 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:120403 7.a 7:53556683:::benchmark_data_smithi022_77720_object120402:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:33:49.932 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:28.687289+0000", 2024-09-11T07:33:49.932 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 201.239878852, 2024-09-11T07:33:49.932 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.15092119800000001, 2024-09-11T07:33:49.932 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:33:49.932 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:33:49.932 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:33:49.932 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:33:49.933 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:33:49.933 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 120403 2024-09-11T07:33:49.933 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.933 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:33:49.933 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.933 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:33:49.933 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.687289+0000", 2024-09-11T07:33:49.933 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:49.933 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.933 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.933 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:33:49.933 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.687289+0000", 2024-09-11T07:33:49.934 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:49.934 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.934 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.934 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:33:49.934 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.687288+0000", 2024-09-11T07:33:49.934 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:33:49.934 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.934 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.934 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:33:49.934 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.687396+0000", 2024-09-11T07:33:49.934 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010856 2024-09-11T07:33:49.934 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.935 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.935 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:33:49.935 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.687420+0000", 2024-09-11T07:33:49.935 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4114e-05 2024-09-11T07:33:49.935 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.935 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.935 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:33:49.935 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.687446+0000", 2024-09-11T07:33:49.935 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5367e-05 2024-09-11T07:33:49.935 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.935 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.935 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:33:49.935 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.691757+0000", 2024-09-11T07:33:49.936 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0043116700000000001 2024-09-11T07:33:49.936 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.936 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.936 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:33:49.936 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.691876+0000", 2024-09-11T07:33:49.936 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000118707 2024-09-11T07:33:49.936 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.936 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.936 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,5", 2024-09-11T07:33:49.936 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.692007+0000", 2024-09-11T07:33:49.936 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013065599999999999 2024-09-11T07:33:49.936 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.937 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.937 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:33:49.937 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695100+0000", 2024-09-11T07:33:49.937 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0030930789999999999 2024-09-11T07:33:49.937 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.937 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.937 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:33:49.937 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.700863+0000", 2024-09-11T07:33:49.937 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0057636079999999999 2024-09-11T07:33:49.937 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.937 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.937 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:33:49.938 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838144+0000", 2024-09-11T07:33:49.938 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13728073099999999 2024-09-11T07:33:49.938 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.938 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.938 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:33:49.938 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838179+0000", 2024-09-11T07:33:49.938 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5129000000000002e-05 2024-09-11T07:33:49.938 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.938 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.938 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:33:49.938 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838210+0000", 2024-09-11T07:33:49.938 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0633999999999998e-05 2024-09-11T07:33:49.939 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:49.939 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:33:49.939 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:49.939 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.939 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.939 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:120408 7.2 7:4165bb05:::benchmark_data_smithi022_77720_object120407:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:33:49.939 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:28.688203+0000", 2024-09-11T07:33:49.939 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 201.23896458900001, 2024-09-11T07:33:49.939 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.149908602, 2024-09-11T07:33:49.939 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:33:49.939 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:33:49.939 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:33:49.939 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:33:49.939 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:33:49.940 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 120408 2024-09-11T07:33:49.940 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.940 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:33:49.940 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.940 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:33:49.940 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.688203+0000", 2024-09-11T07:33:49.940 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:49.940 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.940 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.940 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:33:49.940 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.688203+0000", 2024-09-11T07:33:49.940 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:49.940 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.940 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.941 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:33:49.941 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.688202+0000", 2024-09-11T07:33:49.941 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:33:49.941 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.941 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.941 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:33:49.941 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.688225+0000", 2024-09-11T07:33:49.941 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3796999999999999e-05 2024-09-11T07:33:49.941 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.941 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.941 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:33:49.941 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.688235+0000", 2024-09-11T07:33:49.941 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.7599999999999997e-06 2024-09-11T07:33:49.942 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.942 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.942 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:33:49.942 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.688241+0000", 2024-09-11T07:33:49.942 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6890000000000001e-06 2024-09-11T07:33:49.942 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.942 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.942 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:33:49.942 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.691642+0000", 2024-09-11T07:33:49.942 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.003401669 2024-09-11T07:33:49.942 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.942 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.942 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:33:49.943 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.691766+0000", 2024-09-11T07:33:49.943 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012310700000000001 2024-09-11T07:33:49.943 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.943 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.943 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:33:49.943 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.691893+0000", 2024-09-11T07:33:49.943 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012741099999999999 2024-09-11T07:33:49.943 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.943 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.944 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:33:49.944 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.700857+0000", 2024-09-11T07:33:49.944 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0089635749999999997 2024-09-11T07:33:49.944 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.944 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.944 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:33:49.944 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.701785+0000", 2024-09-11T07:33:49.944 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00092816999999999997 2024-09-11T07:33:49.944 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.944 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.944 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:33:49.944 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838024+0000", 2024-09-11T07:33:49.944 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13623940600000001 2024-09-11T07:33:49.945 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.945 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.945 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:33:49.945 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838072+0000", 2024-09-11T07:33:49.945 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8340999999999997e-05 2024-09-11T07:33:49.945 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.945 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.945 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:33:49.945 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838111+0000", 2024-09-11T07:33:49.945 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8998000000000002e-05 2024-09-11T07:33:49.945 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:49.945 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:33:49.945 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:49.946 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.946 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.946 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:120413 7.a 7:570dbbb8:::benchmark_data_smithi022_77720_object120412:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:33:49.946 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:28.694903+0000", 2024-09-11T07:33:49.946 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 201.23226425300001, 2024-09-11T07:33:49.946 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.14339426699999999, 2024-09-11T07:33:49.946 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:33:49.946 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:33:49.946 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:33:49.946 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:33:49.946 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:49.948 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:33:49.948 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 120413 2024-09-11T07:33:49.948 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.948 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:33:49.948 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.948 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:33:49.948 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.694903+0000", 2024-09-11T07:33:49.948 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:49.948 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.948 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.948 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:33:49.949 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.694903+0000", 2024-09-11T07:33:49.949 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:49.949 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.949 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.949 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:33:49.949 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.694901+0000", 2024-09-11T07:33:49.949 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:33:49.949 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.949 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.949 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:33:49.949 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.694926+0000", 2024-09-11T07:33:49.949 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4505999999999999e-05 2024-09-11T07:33:49.949 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.949 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.950 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:33:49.950 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.694938+0000", 2024-09-11T07:33:49.950 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2326e-05 2024-09-11T07:33:49.950 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.950 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.950 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:33:49.950 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.694944+0000", 2024-09-11T07:33:49.950 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2299999999999996e-06 2024-09-11T07:33:49.950 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.950 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.950 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:33:49.950 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695119+0000", 2024-09-11T07:33:49.950 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00017516499999999999 2024-09-11T07:33:49.950 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.951 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.951 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:33:49.951 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695231+0000", 2024-09-11T07:33:49.951 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011134800000000001 2024-09-11T07:33:49.951 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.951 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.951 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,5", 2024-09-11T07:33:49.951 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695360+0000", 2024-09-11T07:33:49.951 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012901899999999999 2024-09-11T07:33:49.951 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.951 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.951 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:33:49.951 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.698418+0000", 2024-09-11T07:33:49.951 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0030584520000000001 2024-09-11T07:33:49.951 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.952 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.952 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:33:49.952 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.700986+0000", 2024-09-11T07:33:49.952 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0025678440000000001 2024-09-11T07:33:49.952 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.952 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.952 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:33:49.952 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838237+0000", 2024-09-11T07:33:49.952 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13725120900000001 2024-09-11T07:33:49.952 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.952 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.952 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:33:49.952 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838270+0000", 2024-09-11T07:33:49.952 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2852000000000003e-05 2024-09-11T07:33:49.953 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.953 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.953 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:33:49.953 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838297+0000", 2024-09-11T07:33:49.953 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7477000000000001e-05 2024-09-11T07:33:49.953 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:49.953 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:33:49.953 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:49.953 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.953 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.953 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:120415 7.2 7:4b247b8a:::benchmark_data_smithi022_77720_object120414:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:33:49.953 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:28.695503+0000", 2024-09-11T07:33:49.953 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 201.23166432299999, 2024-09-11T07:33:49.953 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.14288304399999999, 2024-09-11T07:33:49.954 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:33:49.954 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:33:49.954 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:33:49.954 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:33:49.954 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:33:49.954 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 120415 2024-09-11T07:33:49.954 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.954 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:33:49.954 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.954 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:33:49.954 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695503+0000", 2024-09-11T07:33:49.954 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:49.954 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.954 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.955 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:33:49.955 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695503+0000", 2024-09-11T07:33:49.955 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:49.955 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.955 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.955 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:33:49.955 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695502+0000", 2024-09-11T07:33:49.955 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:33:49.955 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.955 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.955 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:33:49.955 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695525+0000", 2024-09-11T07:33:49.955 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3680999999999999e-05 2024-09-11T07:33:49.955 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.956 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.956 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:33:49.956 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695537+0000", 2024-09-11T07:33:49.956 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.129e-05 2024-09-11T07:33:49.956 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.956 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.956 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:33:49.956 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695542+0000", 2024-09-11T07:33:49.956 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7899999999999996e-06 2024-09-11T07:33:49.956 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.956 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.956 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:33:49.956 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.697877+0000", 2024-09-11T07:33:49.957 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0023348319999999998 2024-09-11T07:33:49.957 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.957 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.957 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:33:49.957 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.698008+0000", 2024-09-11T07:33:49.957 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000131144 2024-09-11T07:33:49.957 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.957 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.957 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:33:49.957 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.698162+0000", 2024-09-11T07:33:49.957 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000153788 2024-09-11T07:33:49.957 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.957 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.958 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:33:49.958 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.701572+0000", 2024-09-11T07:33:49.958 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.003409568 2024-09-11T07:33:49.958 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.958 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.958 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:33:49.958 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.707469+0000", 2024-09-11T07:33:49.958 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0058970669999999998 2024-09-11T07:33:49.958 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.958 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.958 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:33:49.958 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838325+0000", 2024-09-11T07:33:49.958 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13085648699999999 2024-09-11T07:33:49.959 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.959 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.959 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:33:49.959 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838358+0000", 2024-09-11T07:33:49.959 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2539999999999997e-05 2024-09-11T07:33:49.959 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.959 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.959 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:33:49.959 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838386+0000", 2024-09-11T07:33:49.959 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.828e-05 2024-09-11T07:33:49.959 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:49.959 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:33:49.959 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:49.959 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.960 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.960 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:120425 7.2 7:47ef4c79:::benchmark_data_smithi022_77720_object120424:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:33:49.960 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:28.703929+0000", 2024-09-11T07:33:49.960 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 201.223238044, 2024-09-11T07:33:49.960 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13808211300000001, 2024-09-11T07:33:49.960 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:33:49.960 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:33:49.960 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:33:49.960 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:33:49.960 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:33:49.960 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 120425 2024-09-11T07:33:49.960 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.960 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:33:49.961 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.961 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:33:49.961 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.703929+0000", 2024-09-11T07:33:49.961 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:49.961 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.961 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.961 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:33:49.961 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.703929+0000", 2024-09-11T07:33:49.961 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:49.961 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.961 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.961 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:33:49.961 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.703926+0000", 2024-09-11T07:33:49.962 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999967 2024-09-11T07:33:49.962 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.962 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.962 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:33:49.962 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.703961+0000", 2024-09-11T07:33:49.962 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4691999999999999e-05 2024-09-11T07:33:49.962 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.962 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.962 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:33:49.962 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.703974+0000", 2024-09-11T07:33:49.962 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3237999999999999e-05 2024-09-11T07:33:49.962 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.962 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.962 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:33:49.963 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.703981+0000", 2024-09-11T07:33:49.963 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.776e-06 2024-09-11T07:33:49.963 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.963 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.963 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:33:49.963 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.704039+0000", 2024-09-11T07:33:49.963 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8721000000000002e-05 2024-09-11T07:33:49.963 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.963 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.963 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:33:49.963 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.704175+0000", 2024-09-11T07:33:49.963 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013562000000000001 2024-09-11T07:33:49.963 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.963 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.964 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:33:49.964 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.704408+0000", 2024-09-11T07:33:49.964 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00023285999999999999 2024-09-11T07:33:49.964 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.964 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.964 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:33:49.964 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.707315+0000", 2024-09-11T07:33:49.964 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0029075049999999999 2024-09-11T07:33:49.964 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.964 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.964 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:33:49.964 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.707573+0000", 2024-09-11T07:33:49.964 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00025795400000000002 2024-09-11T07:33:49.965 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.965 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.965 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:33:49.965 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.841933+0000", 2024-09-11T07:33:49.965 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13435992799999999 2024-09-11T07:33:49.965 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.965 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.965 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:33:49.965 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.841979+0000", 2024-09-11T07:33:49.965 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5269000000000003e-05 2024-09-11T07:33:49.965 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.965 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.965 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:33:49.965 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.842011+0000", 2024-09-11T07:33:49.966 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2977e-05 2024-09-11T07:33:49.966 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:49.966 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:33:49.966 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:49.966 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.966 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.966 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:120637 7.a 7:5b8c9ff4:::benchmark_data_smithi022_77720_object120636:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:33:49.966 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.029397+0000", 2024-09-11T07:33:49.966 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 200.89777089500001, 2024-09-11T07:33:49.966 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.22035210899999999, 2024-09-11T07:33:49.966 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:33:49.966 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:33:49.966 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:33:49.967 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:33:49.967 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:33:49.967 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 120637 2024-09-11T07:33:49.967 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.967 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:33:49.967 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.967 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:33:49.967 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.029397+0000", 2024-09-11T07:33:49.967 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:49.967 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.967 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.967 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:33:49.967 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.029397+0000", 2024-09-11T07:33:49.967 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:49.968 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.968 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.968 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:33:49.968 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.029394+0000", 2024-09-11T07:33:49.968 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:33:49.968 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.968 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.968 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:33:49.968 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.029420+0000", 2024-09-11T07:33:49.968 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5621000000000001e-05 2024-09-11T07:33:49.968 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.968 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.968 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:33:49.969 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.029433+0000", 2024-09-11T07:33:49.969 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3437e-05 2024-09-11T07:33:49.969 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.969 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.969 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:33:49.969 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.029440+0000", 2024-09-11T07:33:49.969 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7290000000000003e-06 2024-09-11T07:33:49.969 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.969 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.969 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:33:49.969 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.031329+0000", 2024-09-11T07:33:49.969 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0018888970000000001 2024-09-11T07:33:49.969 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.969 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.970 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:33:49.970 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.031434+0000", 2024-09-11T07:33:49.970 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000104985 2024-09-11T07:33:49.970 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.970 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.970 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,5", 2024-09-11T07:33:49.970 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.031563+0000", 2024-09-11T07:33:49.970 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000128794 2024-09-11T07:33:49.970 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.970 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.970 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:33:49.970 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.040366+0000", 2024-09-11T07:33:49.970 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0088039620000000002 2024-09-11T07:33:49.970 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.971 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.971 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:33:49.971 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.041770+0000", 2024-09-11T07:33:49.971 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001403257 2024-09-11T07:33:49.971 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.971 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.971 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:33:49.971 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.249644+0000", 2024-09-11T07:33:49.971 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20787381199999999 2024-09-11T07:33:49.971 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.971 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.971 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:33:49.971 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.249698+0000", 2024-09-11T07:33:49.972 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.4273e-05 2024-09-11T07:33:49.972 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.972 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.972 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:33:49.972 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.249749+0000", 2024-09-11T07:33:49.972 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.0803000000000001e-05 2024-09-11T07:33:49.972 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:49.972 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:33:49.972 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:49.972 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.972 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.972 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:120649 7.2 7:4a3799e9:::benchmark_data_smithi022_77720_object120648:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:33:49.972 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.034409+0000", 2024-09-11T07:33:49.973 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 200.89275811600001, 2024-09-11T07:33:49.973 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.215573452, 2024-09-11T07:33:49.973 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:33:49.973 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:33:49.973 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:33:49.973 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:33:49.973 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:33:49.973 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 120649 2024-09-11T07:33:49.973 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.973 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:33:49.973 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.973 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:33:49.973 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.034409+0000", 2024-09-11T07:33:49.974 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:49.974 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.974 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.974 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:33:49.974 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.034409+0000", 2024-09-11T07:33:49.974 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:49.974 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.974 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.974 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:33:49.974 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.034407+0000", 2024-09-11T07:33:49.974 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:33:49.974 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.974 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.975 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:33:49.975 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.034433+0000", 2024-09-11T07:33:49.975 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.6526999999999999e-05 2024-09-11T07:33:49.975 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.975 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.975 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:33:49.975 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.034448+0000", 2024-09-11T07:33:49.975 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4080000000000001e-05 2024-09-11T07:33:49.975 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.975 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.975 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:33:49.975 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.034454+0000", 2024-09-11T07:33:49.976 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5429999999999999e-06 2024-09-11T07:33:49.977 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.977 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.977 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:33:49.977 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.036213+0000", 2024-09-11T07:33:49.977 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001758857 2024-09-11T07:33:49.978 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.978 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.978 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:33:49.978 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.036318+0000", 2024-09-11T07:33:49.978 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000105245 2024-09-11T07:33:49.978 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.978 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.978 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:33:49.978 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.036447+0000", 2024-09-11T07:33:49.978 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012836999999999999 2024-09-11T07:33:49.978 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.978 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.979 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:33:49.979 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.042776+0000", 2024-09-11T07:33:49.979 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0063298790000000001 2024-09-11T07:33:49.979 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.979 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.979 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:33:49.979 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.043761+0000", 2024-09-11T07:33:49.979 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00098433900000000009 2024-09-11T07:33:49.979 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.979 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.979 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:33:49.979 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.249911+0000", 2024-09-11T07:33:49.979 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20615025200000001 2024-09-11T07:33:49.979 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.980 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.980 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:33:49.980 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.249951+0000", 2024-09-11T07:33:49.980 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9817000000000002e-05 2024-09-11T07:33:49.980 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.980 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.980 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:33:49.980 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.249983+0000", 2024-09-11T07:33:49.980 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.188e-05 2024-09-11T07:33:49.980 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:49.980 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:33:49.980 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:49.980 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.981 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.981 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120650 7.c e127/92 7:3551ad5b:::benchmark_data_smithi022_77720_object120649:head v 127'7564, mlcod=127'7562)", 2024-09-11T07:33:49.981 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.038754+0000", 2024-09-11T07:33:49.981 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 200.88841295700001, 2024-09-11T07:33:49.981 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.21640670200000001, 2024-09-11T07:33:49.981 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:33:49.981 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:33:49.981 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:33:49.981 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.981 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:33:49.981 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.038754+0000", 2024-09-11T07:33:49.981 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:49.982 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.982 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.982 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:33:49.982 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.038754+0000", 2024-09-11T07:33:49.982 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:49.982 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.982 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.982 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:33:49.982 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.038752+0000", 2024-09-11T07:33:49.982 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:33:49.982 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.982 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.982 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:33:49.983 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.038797+0000", 2024-09-11T07:33:49.983 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4357000000000002e-05 2024-09-11T07:33:49.983 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.983 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.983 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:33:49.983 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.038809+0000", 2024-09-11T07:33:49.983 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1829e-05 2024-09-11T07:33:49.983 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.983 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.983 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:33:49.983 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.038815+0000", 2024-09-11T07:33:49.983 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3579999999999999e-06 2024-09-11T07:33:49.983 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.983 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.983 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:33:49.984 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.038871+0000", 2024-09-11T07:33:49.984 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6193999999999998e-05 2024-09-11T07:33:49.984 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.984 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.984 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:33:49.984 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.038889+0000", 2024-09-11T07:33:49.984 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.7915999999999999e-05 2024-09-11T07:33:49.984 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.984 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.984 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:33:49.984 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.255019+0000", 2024-09-11T07:33:49.984 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.21613015499999999 2024-09-11T07:33:49.984 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.985 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.985 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:33:49.985 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.255161+0000", 2024-09-11T07:33:49.985 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014201900000000001 2024-09-11T07:33:49.985 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:49.985 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:33:49.985 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:49.985 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.985 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.985 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120651 7.c e127/92 7:32aa7de5:::benchmark_data_smithi022_77720_object120650:head v 127'7565, mlcod=127'7562)", 2024-09-11T07:33:49.985 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.041033+0000", 2024-09-11T07:33:49.985 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 200.88613423000001, 2024-09-11T07:33:49.985 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.214179168, 2024-09-11T07:33:49.985 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:33:49.986 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:33:49.986 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:33:49.986 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.986 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:33:49.986 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.041033+0000", 2024-09-11T07:33:49.986 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:49.986 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.986 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.986 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:33:49.986 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.041033+0000", 2024-09-11T07:33:49.986 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:49.986 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.986 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.986 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:33:49.987 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.041032+0000", 2024-09-11T07:33:49.987 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:33:49.987 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.987 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.987 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:33:49.987 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.041062+0000", 2024-09-11T07:33:49.987 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0488999999999999e-05 2024-09-11T07:33:49.987 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.987 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.987 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:33:49.987 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.041072+0000", 2024-09-11T07:33:49.987 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.5629999999999998e-06 2024-09-11T07:33:49.987 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.988 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.988 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:33:49.988 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.041077+0000", 2024-09-11T07:33:49.988 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.0579999999999997e-06 2024-09-11T07:33:49.988 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.988 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.988 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:33:49.988 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.045254+0000", 2024-09-11T07:33:49.988 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0041766110000000002 2024-09-11T07:33:49.988 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.988 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.988 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:33:49.988 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.045273+0000", 2024-09-11T07:33:49.989 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9636e-05 2024-09-11T07:33:49.989 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.989 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.989 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:33:49.989 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.255164+0000", 2024-09-11T07:33:49.989 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20989069099999999 2024-09-11T07:33:49.989 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.989 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.989 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:33:49.989 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.255212+0000", 2024-09-11T07:33:49.989 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8322000000000003e-05 2024-09-11T07:33:49.989 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:49.989 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:33:49.990 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:49.990 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.990 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.990 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120654 7.c e127/92 7:3089e4e7:::benchmark_data_smithi022_77720_object120653:head v 127'7566, mlcod=127'7562)", 2024-09-11T07:33:49.990 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.044199+0000", 2024-09-11T07:33:49.990 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 200.88296885700001, 2024-09-11T07:33:49.990 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.211072754, 2024-09-11T07:33:49.990 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:33:49.990 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:33:49.990 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:33:49.990 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.990 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:33:49.990 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.044199+0000", 2024-09-11T07:33:49.990 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:49.991 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.991 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.991 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:33:49.991 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.044199+0000", 2024-09-11T07:33:49.991 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:49.991 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.991 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.991 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:33:49.991 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.044197+0000", 2024-09-11T07:33:49.991 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:33:49.991 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.991 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.991 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:33:49.992 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.044244+0000", 2024-09-11T07:33:49.992 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6838000000000001e-05 2024-09-11T07:33:49.992 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.992 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.992 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:33:49.992 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.044256+0000", 2024-09-11T07:33:49.992 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2041e-05 2024-09-11T07:33:49.992 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.992 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.992 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:33:49.992 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.044262+0000", 2024-09-11T07:33:49.992 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2110000000000003e-06 2024-09-11T07:33:49.992 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.993 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.993 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:33:49.993 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.048642+0000", 2024-09-11T07:33:49.993 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0043793469999999996 2024-09-11T07:33:49.993 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.993 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.993 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:33:49.993 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.048660+0000", 2024-09-11T07:33:49.993 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8363999999999999e-05 2024-09-11T07:33:49.993 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.993 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.993 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:33:49.993 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.255215+0000", 2024-09-11T07:33:49.994 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20655538200000001 2024-09-11T07:33:49.994 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.994 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.994 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:33:49.994 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.255271+0000", 2024-09-11T07:33:49.994 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6033999999999999e-05 2024-09-11T07:33:49.994 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:49.994 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:33:49.994 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:49.994 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.994 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.994 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:120659 7.3 7:c2975cc8:::benchmark_data_smithi022_77720_object120658:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:33:49.994 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.046561+0000", 2024-09-11T07:33:49.995 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 200.88060621, 2024-09-11T07:33:49.995 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20440429299999999, 2024-09-11T07:33:49.995 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:33:49.995 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:33:49.995 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:33:49.995 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:33:49.995 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:33:49.995 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 120659 2024-09-11T07:33:49.995 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.995 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:33:49.995 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.995 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:33:49.995 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046561+0000", 2024-09-11T07:33:49.996 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:49.996 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.996 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.996 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:33:49.996 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046561+0000", 2024-09-11T07:33:49.996 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:49.996 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.996 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.996 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:33:49.996 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046559+0000", 2024-09-11T07:33:49.996 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:33:49.996 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.996 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.997 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:33:49.997 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046587+0000", 2024-09-11T07:33:49.997 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8705e-05 2024-09-11T07:33:49.997 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.997 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.997 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:33:49.997 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046600+0000", 2024-09-11T07:33:49.997 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2499e-05 2024-09-11T07:33:49.997 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.997 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.997 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:33:49.997 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046606+0000", 2024-09-11T07:33:49.997 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0159999999999997e-06 2024-09-11T07:33:49.998 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.998 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.998 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:33:49.998 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046693+0000", 2024-09-11T07:33:49.998 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.6792999999999994e-05 2024-09-11T07:33:49.998 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.998 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.998 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:33:49.998 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046953+0000", 2024-09-11T07:33:49.998 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00026056300000000001 2024-09-11T07:33:49.998 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.998 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.998 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,4", 2024-09-11T07:33:49.998 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.047062+0000", 2024-09-11T07:33:49.998 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010925999999999999 2024-09-11T07:33:49.999 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.999 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.999 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:33:49.999 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.049336+0000", 2024-09-11T07:33:49.999 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002273691 2024-09-11T07:33:49.999 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.999 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.999 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:33:49.999 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050203+0000", 2024-09-11T07:33:49.999 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000866591 2024-09-11T07:33:49.999 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:49.999 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:49.999 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:33:49.999 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.250805+0000", 2024-09-11T07:33:50.000 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20060213800000001 2024-09-11T07:33:50.000 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.000 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.000 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:33:50.000 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.250854+0000", 2024-09-11T07:33:50.000 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9335000000000001e-05 2024-09-11T07:33:50.000 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.000 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.000 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:33:50.000 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.250965+0000", 2024-09-11T07:33:50.000 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000111242 2024-09-11T07:33:50.000 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:50.000 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:33:50.000 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:50.001 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.001 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.001 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120656 7.c e127/92 7:35364cdf:::benchmark_data_smithi022_77720_object120655:head v 127'7567, mlcod=127'7562)", 2024-09-11T07:33:50.001 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.046959+0000", 2024-09-11T07:33:50.001 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 200.88020819600001, 2024-09-11T07:33:50.001 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.21188748199999999, 2024-09-11T07:33:50.001 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:33:50.001 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:33:50.001 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:33:50.001 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.001 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:33:50.001 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046959+0000", 2024-09-11T07:33:50.001 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:50.002 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.002 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.002 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:33:50.002 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046959+0000", 2024-09-11T07:33:50.002 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:50.002 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.002 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.002 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:33:50.002 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046958+0000", 2024-09-11T07:33:50.002 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:33:50.002 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.002 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.002 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:33:50.002 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.047009+0000", 2024-09-11T07:33:50.003 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1489000000000001e-05 2024-09-11T07:33:50.003 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.003 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.003 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:33:50.003 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.047020+0000", 2024-09-11T07:33:50.003 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.064e-05 2024-09-11T07:33:50.003 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.003 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.003 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:33:50.003 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.047025+0000", 2024-09-11T07:33:50.003 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1279999999999999e-06 2024-09-11T07:33:50.003 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.003 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.004 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:33:50.004 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246150+0000", 2024-09-11T07:33:50.004 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.199124994 2024-09-11T07:33:50.004 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.004 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.004 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:33:50.004 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246173+0000", 2024-09-11T07:33:50.004 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3091999999999998e-05 2024-09-11T07:33:50.004 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.004 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.004 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:33:50.004 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.258588+0000", 2024-09-11T07:33:50.005 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.012415433 2024-09-11T07:33:50.005 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.005 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.005 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:33:50.005 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.258847+0000", 2024-09-11T07:33:50.005 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00025835000000000002 2024-09-11T07:33:50.005 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:50.005 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:33:50.005 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:50.005 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.005 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.005 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120657 7.c e127/92 7:333acda7:::benchmark_data_smithi022_77720_object120656:head v 127'7568, mlcod=127'7562)", 2024-09-11T07:33:50.005 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.050053+0000", 2024-09-11T07:33:50.006 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 200.877114009, 2024-09-11T07:33:50.006 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.22377028800000001, 2024-09-11T07:33:50.006 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:33:50.006 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:33:50.006 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:33:50.006 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.006 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:33:50.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:50.007 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:50.007 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:50.008 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050053+0000", 2024-09-11T07:33:50.008 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:50.008 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.008 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.008 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:33:50.008 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050053+0000", 2024-09-11T07:33:50.008 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:50.008 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.008 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.008 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:33:50.008 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050052+0000", 2024-09-11T07:33:50.009 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:33:50.009 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.009 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.009 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:33:50.009 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050092+0000", 2024-09-11T07:33:50.009 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0565000000000002e-05 2024-09-11T07:33:50.009 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.009 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.009 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:33:50.009 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050102+0000", 2024-09-11T07:33:50.009 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.6150000000000003e-06 2024-09-11T07:33:50.009 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.010 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.010 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:33:50.010 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050106+0000", 2024-09-11T07:33:50.010 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9400000000000004e-06 2024-09-11T07:33:50.010 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.010 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.010 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:33:50.010 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.255309+0000", 2024-09-11T07:33:50.010 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.205202789 2024-09-11T07:33:50.010 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.010 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.010 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:33:50.010 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.255342+0000", 2024-09-11T07:33:50.011 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2854000000000001e-05 2024-09-11T07:33:50.011 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.011 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.011 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:33:50.011 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.273773+0000", 2024-09-11T07:33:50.011 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.018431050000000001 2024-09-11T07:33:50.011 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.011 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.011 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:33:50.011 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.273824+0000", 2024-09-11T07:33:50.011 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1072000000000002e-05 2024-09-11T07:33:50.011 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:50.011 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:33:50.012 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:50.012 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.012 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.012 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120662 7.7 e127/92 7:edc0624b:::benchmark_data_smithi022_77720_object120661:head v 127'7599, mlcod=127'7596)", 2024-09-11T07:33:50.012 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.050439+0000", 2024-09-11T07:33:50.012 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 200.87672817000001, 2024-09-11T07:33:50.012 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20149834799999999, 2024-09-11T07:33:50.012 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:33:50.012 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:33:50.012 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:33:50.012 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.013 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:33:50.013 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050439+0000", 2024-09-11T07:33:50.013 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:50.013 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.013 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.013 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:33:50.013 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050439+0000", 2024-09-11T07:33:50.013 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:50.013 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.013 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.013 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:33:50.013 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050438+0000", 2024-09-11T07:33:50.013 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:33:50.014 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.014 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.014 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:33:50.014 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050526+0000", 2024-09-11T07:33:50.014 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.8176000000000004e-05 2024-09-11T07:33:50.014 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.014 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.014 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:33:50.014 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050535+0000", 2024-09-11T07:33:50.014 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.1050000000000001e-06 2024-09-11T07:33:50.014 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.014 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.015 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:33:50.015 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050540+0000", 2024-09-11T07:33:50.015 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8740000000000003e-06 2024-09-11T07:33:50.015 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.015 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.015 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:33:50.015 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.248240+0000", 2024-09-11T07:33:50.015 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.19770011900000001 2024-09-11T07:33:50.015 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.015 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.015 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:33:50.015 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.248260+0000", 2024-09-11T07:33:50.015 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9468000000000001e-05 2024-09-11T07:33:50.016 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.016 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.016 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:33:50.016 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.251902+0000", 2024-09-11T07:33:50.016 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0036426689999999999 2024-09-11T07:33:50.016 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.016 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.016 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:33:50.016 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.251938+0000", 2024-09-11T07:33:50.016 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5219999999999998e-05 2024-09-11T07:33:50.016 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:50.016 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:33:50.016 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:50.016 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.017 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.017 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120661 7.1 e127/92 7:8ef41966:::benchmark_data_smithi022_77720_object120660:head v 127'7487, mlcod=127'7484)", 2024-09-11T07:33:50.017 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.050540+0000", 2024-09-11T07:33:50.017 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 200.87662788399999, 2024-09-11T07:33:50.017 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20305398899999999, 2024-09-11T07:33:50.017 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:33:50.017 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:33:50.017 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:33:50.017 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.017 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:33:50.017 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050540+0000", 2024-09-11T07:33:50.017 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:50.018 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.018 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.018 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:33:50.018 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050540+0000", 2024-09-11T07:33:50.018 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:50.018 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.018 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.018 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:33:50.018 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050539+0000", 2024-09-11T07:33:50.018 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:33:50.018 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.018 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.018 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:33:50.018 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050602+0000", 2024-09-11T07:33:50.019 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3145000000000006e-05 2024-09-11T07:33:50.019 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.019 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.019 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:33:50.019 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246370+0000", 2024-09-11T07:33:50.019 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.19576812599999999 2024-09-11T07:33:50.019 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.019 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.019 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:33:50.019 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246378+0000", 2024-09-11T07:33:50.019 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.8860000000000004e-06 2024-09-11T07:33:50.019 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.019 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.020 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:33:50.020 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246487+0000", 2024-09-11T07:33:50.020 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000109171 2024-09-11T07:33:50.020 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.020 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.020 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:33:50.020 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246507+0000", 2024-09-11T07:33:50.020 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0064999999999998e-05 2024-09-11T07:33:50.020 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.020 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.020 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:33:50.020 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.253510+0000", 2024-09-11T07:33:50.020 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0070029630000000001 2024-09-11T07:33:50.021 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.021 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.021 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:33:50.021 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.253594+0000", 2024-09-11T07:33:50.021 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3560000000000006e-05 2024-09-11T07:33:50.021 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:50.021 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:33:50.021 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:50.021 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.021 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.021 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24353.0:885 5.14 5:2b04a3e9:::notify.6:head [watch reconnect cookie 94395805760480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-11T07:33:50.021 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:31:02.582152+0000", 2024-09-11T07:33:50.021 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 167.34501590900001, 2024-09-11T07:33:50.022 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.62439777600000002, 2024-09-11T07:33:50.022 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:33:50.022 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:33:50.022 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:33:50.022 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24353", 2024-09-11T07:33:50.022 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1860969011", 2024-09-11T07:33:50.022 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 885 2024-09-11T07:33:50.022 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.022 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:33:50.022 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.022 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:33:50.022 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.582152+0000", 2024-09-11T07:33:50.022 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:50.023 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.023 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.023 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:33:50.023 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.582152+0000", 2024-09-11T07:33:50.023 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:50.023 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.023 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.024 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:33:50.024 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.582149+0000", 2024-09-11T07:33:50.024 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:33:50.024 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.024 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.024 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:33:50.024 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.582157+0000", 2024-09-11T07:33:50.024 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.2050000000000002e-06 2024-09-11T07:33:50.024 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.024 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.024 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:33:50.024 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.582167+0000", 2024-09-11T07:33:50.024 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.6299999999999993e-06 2024-09-11T07:33:50.025 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.025 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.025 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:33:50.025 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.582172+0000", 2024-09-11T07:33:50.025 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1309999999999997e-06 2024-09-11T07:33:50.025 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.025 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.025 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:33:50.025 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.582219+0000", 2024-09-11T07:33:50.026 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7246000000000001e-05 2024-09-11T07:33:50.026 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.026 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.026 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:33:50.026 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.582225+0000", 2024-09-11T07:33:50.026 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8529999999999997e-06 2024-09-11T07:33:50.026 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.026 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.026 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:33:50.026 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.201574+0000", 2024-09-11T07:33:50.026 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.61934906000000001 2024-09-11T07:33:50.027 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.027 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.027 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:33:50.027 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.201736+0000", 2024-09-11T07:33:50.027 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000162118 2024-09-11T07:33:50.027 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.027 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.027 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:33:50.027 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.206549+0000", 2024-09-11T07:33:50.027 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0048128700000000003 2024-09-11T07:33:50.027 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:50.027 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:33:50.027 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:50.028 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.028 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.028 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:14417 8.f 8:fa15f9ae:::benchmark_data_smithi022_108406_object14416:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-11T07:33:50.028 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:45.110515+0000", 2024-09-11T07:33:50.028 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 4.8166519579999996, 2024-09-11T07:33:50.028 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13859276400000001, 2024-09-11T07:33:50.028 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:33:50.028 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:33:50.028 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:33:50.028 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:33:50.028 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:33:50.028 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 14417 2024-09-11T07:33:50.028 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.029 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:33:50.029 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.029 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:33:50.029 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110515+0000", 2024-09-11T07:33:50.029 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:50.029 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.029 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.029 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:33:50.029 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110515+0000", 2024-09-11T07:33:50.029 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:50.029 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.029 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.029 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:33:50.029 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110513+0000", 2024-09-11T07:33:50.030 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:33:50.030 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.030 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.030 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:33:50.030 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110540+0000", 2024-09-11T07:33:50.030 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7021000000000001e-05 2024-09-11T07:33:50.030 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.030 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.030 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:33:50.030 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110553+0000", 2024-09-11T07:33:50.030 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2357999999999999e-05 2024-09-11T07:33:50.030 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.030 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.031 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:33:50.031 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110559+0000", 2024-09-11T07:33:50.031 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4509999999999997e-06 2024-09-11T07:33:50.031 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.031 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.031 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:33:50.031 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110706+0000", 2024-09-11T07:33:50.031 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014704799999999999 2024-09-11T07:33:50.031 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.031 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.031 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:33:50.031 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110841+0000", 2024-09-11T07:33:50.031 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000135053 2024-09-11T07:33:50.031 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.032 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.032 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:33:50.032 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.111155+0000", 2024-09-11T07:33:50.032 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00031389699999999999 2024-09-11T07:33:50.032 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.032 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.032 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:33:50.032 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112755+0000", 2024-09-11T07:33:50.032 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0016004050000000001 2024-09-11T07:33:50.032 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.032 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.032 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:33:50.032 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.113262+0000", 2024-09-11T07:33:50.033 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00050669799999999998 2024-09-11T07:33:50.033 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.033 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.033 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:33:50.033 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.249007+0000", 2024-09-11T07:33:50.033 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13574537 2024-09-11T07:33:50.033 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.033 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.033 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:33:50.033 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.249064+0000", 2024-09-11T07:33:50.033 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6530000000000003e-05 2024-09-11T07:33:50.033 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.033 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.034 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:33:50.034 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.249108+0000", 2024-09-11T07:33:50.034 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4249000000000003e-05 2024-09-11T07:33:50.034 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:50.034 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:33:50.034 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:50.034 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.034 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.034 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:14418 8.0 8:0e01a02b:::benchmark_data_smithi022_108406_object14417:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-11T07:33:50.034 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:45.110638+0000", 2024-09-11T07:33:50.034 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 4.8165298060000001, 2024-09-11T07:33:50.034 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13896979200000001, 2024-09-11T07:33:50.034 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:33:50.034 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:33:50.035 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:33:50.035 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:33:50.035 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:33:50.035 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 14418 2024-09-11T07:33:50.035 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.035 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:33:50.035 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.035 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:33:50.035 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110638+0000", 2024-09-11T07:33:50.035 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:50.035 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.035 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.035 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:33:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110638+0000", 2024-09-11T07:33:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:33:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110636+0000", 2024-09-11T07:33:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:33:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:33:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110658+0000", 2024-09-11T07:33:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2243e-05 2024-09-11T07:33:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.036 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:33:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110668+0000", 2024-09-11T07:33:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.1160000000000005e-06 2024-09-11T07:33:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:33:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110673+0000", 2024-09-11T07:33:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.519e-06 2024-09-11T07:33:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:33:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110717+0000", 2024-09-11T07:33:50.037 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3476999999999997e-05 2024-09-11T07:33:50.038 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.038 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.038 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:33:50.038 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110839+0000", 2024-09-11T07:33:50.038 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012271100000000001 2024-09-11T07:33:50.038 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.039 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.039 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,5", 2024-09-11T07:33:50.039 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110981+0000", 2024-09-11T07:33:50.039 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000141218 2024-09-11T07:33:50.039 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.039 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.039 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:33:50.039 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112249+0000", 2024-09-11T07:33:50.039 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0012689330000000001 2024-09-11T07:33:50.039 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.039 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.039 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:33:50.039 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.114781+0000", 2024-09-11T07:33:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0025319969999999998 2024-09-11T07:33:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:33:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.249534+0000", 2024-09-11T07:33:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.134752701 2024-09-11T07:33:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:33:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.249574+0000", 2024-09-11T07:33:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9972000000000003e-05 2024-09-11T07:33:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.040 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.041 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:33:50.041 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.249607+0000", 2024-09-11T07:33:50.041 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3272999999999998e-05 2024-09-11T07:33:50.041 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:50.041 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:33:50.041 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:50.041 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.041 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.041 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:14419 8.f 8:fe75a06c:::benchmark_data_smithi022_108406_object14418:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-11T07:33:50.041 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:45.110878+0000", 2024-09-11T07:33:50.041 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 4.816289866, 2024-09-11T07:33:50.041 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13866826700000001, 2024-09-11T07:33:50.041 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:33:50.042 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:33:50.042 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:33:50.042 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:33:50.042 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:33:50.042 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 14419 2024-09-11T07:33:50.042 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.042 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:33:50.042 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.042 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:33:50.042 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110878+0000", 2024-09-11T07:33:50.042 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:50.042 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.042 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.043 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:33:50.043 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110878+0000", 2024-09-11T07:33:50.043 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:50.043 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.043 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.043 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:33:50.043 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110876+0000", 2024-09-11T07:33:50.043 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:33:50.043 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.043 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.043 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:33:50.043 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110916+0000", 2024-09-11T07:33:50.043 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0201999999999999e-05 2024-09-11T07:33:50.044 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.044 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.044 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:33:50.044 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110928+0000", 2024-09-11T07:33:50.044 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1099e-05 2024-09-11T07:33:50.044 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.044 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.044 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:33:50.044 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110934+0000", 2024-09-11T07:33:50.044 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2249999999999997e-06 2024-09-11T07:33:50.044 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.044 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.044 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:33:50.045 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.111838+0000", 2024-09-11T07:33:50.045 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00090451400000000003 2024-09-11T07:33:50.045 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.045 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.045 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:33:50.045 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.111957+0000", 2024-09-11T07:33:50.045 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000118888 2024-09-11T07:33:50.045 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.045 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.045 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:33:50.045 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112088+0000", 2024-09-11T07:33:50.045 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013097400000000001 2024-09-11T07:33:50.045 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.046 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.046 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:33:50.046 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112903+0000", 2024-09-11T07:33:50.046 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00081438400000000003 2024-09-11T07:33:50.046 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.046 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.046 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:33:50.046 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.114264+0000", 2024-09-11T07:33:50.046 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001361026 2024-09-11T07:33:50.046 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.046 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.046 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:33:50.046 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.249462+0000", 2024-09-11T07:33:50.047 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13519831800000001 2024-09-11T07:33:50.047 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.047 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.047 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:33:50.047 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.249512+0000", 2024-09-11T07:33:50.047 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9867999999999998e-05 2024-09-11T07:33:50.047 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.047 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.047 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:33:50.047 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.249546+0000", 2024-09-11T07:33:50.047 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4041999999999997e-05 2024-09-11T07:33:50.047 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:50.047 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:33:50.047 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:50.047 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.048 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.048 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:14429 8.0 8:0b62d12a:::benchmark_data_smithi022_108406_object14428:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-11T07:33:50.048 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:45.113804+0000", 2024-09-11T07:33:50.048 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 4.813363442, 2024-09-11T07:33:50.048 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.136458995, 2024-09-11T07:33:50.048 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:33:50.048 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:33:50.048 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:33:50.048 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:33:50.048 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:33:50.048 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 14429 2024-09-11T07:33:50.048 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.049 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:33:50.049 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.049 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:33:50.049 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.113804+0000", 2024-09-11T07:33:50.049 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:50.049 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.049 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.049 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:33:50.049 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.113804+0000", 2024-09-11T07:33:50.049 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:33:50.049 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.049 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.049 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:33:50.050 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.113801+0000", 2024-09-11T07:33:50.050 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:33:50.050 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.050 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.050 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:33:50.050 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.113832+0000", 2024-09-11T07:33:50.050 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0545e-05 2024-09-11T07:33:50.050 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.050 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.050 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:33:50.050 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.113844+0000", 2024-09-11T07:33:50.050 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2575000000000001e-05 2024-09-11T07:33:50.050 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.050 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.051 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:33:50.051 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.113851+0000", 2024-09-11T07:33:50.051 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3799999999999999e-06 2024-09-11T07:33:50.051 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.051 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.051 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:33:50.051 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.113896+0000", 2024-09-11T07:33:50.051 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5454000000000002e-05 2024-09-11T07:33:50.051 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.051 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.051 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:33:50.051 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.114037+0000", 2024-09-11T07:33:50.051 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014135600000000001 2024-09-11T07:33:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,5", 2024-09-11T07:33:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.114183+0000", 2024-09-11T07:33:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000145076 2024-09-11T07:33:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:33:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.114912+0000", 2024-09-11T07:33:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00072963699999999999 2024-09-11T07:33:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.052 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:33:50.053 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.115483+0000", 2024-09-11T07:33:50.053 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00057045999999999996 2024-09-11T07:33:50.053 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.053 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.053 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:33:50.053 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.250193+0000", 2024-09-11T07:33:50.053 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13471022699999999 2024-09-11T07:33:50.053 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.053 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.053 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:33:50.053 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.250227+0000", 2024-09-11T07:33:50.053 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4434000000000003e-05 2024-09-11T07:33:50.053 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:33:50.053 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:33:50.054 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:33:50.054 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.250263+0000", 2024-09-11T07:33:50.054 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5642999999999997e-05 2024-09-11T07:33:50.054 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:50.054 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:33:50.054 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:50.054 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:33:50.054 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:33:50.054 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:33:50.108 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:50.110 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T07:33:50.136 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:50.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:50.137 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:50.200 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 10 16 27055 27039 168.966 138.438 0.0124694 0.00591073 2024-09-11T07:33:50.238 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:50.267 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:50.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:50.269 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:50.369 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:50.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[29462]: 5.9 scrub starts 2024-09-11T07:33:50.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[29462]: 5.9 scrub ok 2024-09-11T07:33:50.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[29462]: 5.0 scrub starts 2024-09-11T07:33:50.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[29462]: 5.0 scrub ok 2024-09-11T07:33:50.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[29462]: 3.1 scrub starts 2024-09-11T07:33:50.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[29462]: 3.1 scrub ok 2024-09-11T07:33:50.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[29462]: 5.1b scrub starts 2024-09-11T07:33:50.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[29462]: 5.1b scrub ok 2024-09-11T07:33:50.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[29462]: osdmap e189: 8 total, 8 up, 8 in 2024-09-11T07:33:50.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[29462]: 5.19 scrub starts 2024-09-11T07:33:50.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[29462]: 5.19 scrub ok 2024-09-11T07:33:50.376 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:50.376 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[29462]: 4.a scrub starts 2024-09-11T07:33:50.376 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[29462]: 4.a scrub ok 2024-09-11T07:33:50.376 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[37330]: 5.9 scrub starts 2024-09-11T07:33:50.376 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[37330]: 5.9 scrub ok 2024-09-11T07:33:50.376 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[37330]: 5.0 scrub starts 2024-09-11T07:33:50.376 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[37330]: 5.0 scrub ok 2024-09-11T07:33:50.376 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[37330]: 3.1 scrub starts 2024-09-11T07:33:50.376 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[37330]: 3.1 scrub ok 2024-09-11T07:33:50.377 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[37330]: 5.1b scrub starts 2024-09-11T07:33:50.377 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[37330]: 5.1b scrub ok 2024-09-11T07:33:50.377 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[37330]: osdmap e189: 8 total, 8 up, 8 in 2024-09-11T07:33:50.377 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[37330]: 5.19 scrub starts 2024-09-11T07:33:50.377 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[37330]: 5.19 scrub ok 2024-09-11T07:33:50.377 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:50.377 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[37330]: 4.a scrub starts 2024-09-11T07:33:50.377 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-mon[37330]: 4.a scrub ok 2024-09-11T07:33:50.421 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:50.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:50.422 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:50.523 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:50.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:50 smithi104 ceph-mon[33507]: 5.9 scrub starts 2024-09-11T07:33:50.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:50 smithi104 ceph-mon[33507]: 5.9 scrub ok 2024-09-11T07:33:50.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:50 smithi104 ceph-mon[33507]: 5.0 scrub starts 2024-09-11T07:33:50.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:50 smithi104 ceph-mon[33507]: 5.0 scrub ok 2024-09-11T07:33:50.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:50 smithi104 ceph-mon[33507]: 3.1 scrub starts 2024-09-11T07:33:50.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:50 smithi104 ceph-mon[33507]: 3.1 scrub ok 2024-09-11T07:33:50.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:50 smithi104 ceph-mon[33507]: 5.1b scrub starts 2024-09-11T07:33:50.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:50 smithi104 ceph-mon[33507]: 5.1b scrub ok 2024-09-11T07:33:50.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:50 smithi104 ceph-mon[33507]: osdmap e189: 8 total, 8 up, 8 in 2024-09-11T07:33:50.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:50 smithi104 ceph-mon[33507]: 5.19 scrub starts 2024-09-11T07:33:50.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:50 smithi104 ceph-mon[33507]: 5.19 scrub ok 2024-09-11T07:33:50.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:50 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:33:50.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:50 smithi104 ceph-mon[33507]: 4.a scrub starts 2024-09-11T07:33:50.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:50 smithi104 ceph-mon[33507]: 4.a scrub ok 2024-09-11T07:33:50.545 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:50.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:50.546 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:50.647 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:50.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:33:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:33:50] "GET /metrics HTTP/1.1" 200 223727 "" "Prometheus/2.43.0" 2024-09-11T07:33:50.676 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:50.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:50.677 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:50.778 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:50.799 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:50.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:50.800 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:50.901 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:50.935 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:50.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:50.936 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:51.037 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:51.058 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:51.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:51.059 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:51.160 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:51.200 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 11 16 28813 28797 163.594 109.875 0.00437654 0.00610522 2024-09-11T07:33:51.201 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:51.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:51.202 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:51.303 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:51.332 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:51.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:51.333 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:51.434 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:51.464 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:51.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:51.465 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:51.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:51 smithi104 ceph-mon[33507]: pgmap v321: 148 pgs: 148 active+clean; 1.5 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.3 KiB/s rd, 209 MiB/s wr, 3.35k op/s 2024-09-11T07:33:51.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:51 smithi104 ceph-mon[33507]: 2.a scrub starts 2024-09-11T07:33:51.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:51 smithi104 ceph-mon[33507]: 2.a scrub ok 2024-09-11T07:33:51.567 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:51.595 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:51.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:51.596 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:51.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:51 smithi022 ceph-mon[29462]: pgmap v321: 148 pgs: 148 active+clean; 1.5 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.3 KiB/s rd, 209 MiB/s wr, 3.35k op/s 2024-09-11T07:33:51.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:51 smithi022 ceph-mon[29462]: 2.a scrub starts 2024-09-11T07:33:51.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:51 smithi022 ceph-mon[29462]: 2.a scrub ok 2024-09-11T07:33:51.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:51 smithi022 ceph-mon[37330]: pgmap v321: 148 pgs: 148 active+clean; 1.5 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.3 KiB/s rd, 209 MiB/s wr, 3.35k op/s 2024-09-11T07:33:51.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:51 smithi022 ceph-mon[37330]: 2.a scrub starts 2024-09-11T07:33:51.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:51 smithi022 ceph-mon[37330]: 2.a scrub ok 2024-09-11T07:33:51.696 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:51.720 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:51.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:51.721 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:51.822 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:51.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:51.851 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:51.851 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:51.952 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:51.980 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:51.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:51.981 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:52.081 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:52.112 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:52.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:52.113 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:52.200 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 12 16 30840 30824 160.518 126.688 0.00493854 0.00622394 2024-09-11T07:33:52.214 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:52.234 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:52.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:52.235 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:52.336 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:52.365 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:52.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:52.366 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:52.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:52 smithi022 ceph-mon[29462]: 4.1d scrub starts 2024-09-11T07:33:52.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:52 smithi022 ceph-mon[29462]: 4.1d scrub ok 2024-09-11T07:33:52.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:52 smithi022 ceph-mon[29462]: 5.3 scrub starts 2024-09-11T07:33:52.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:52 smithi022 ceph-mon[29462]: 5.3 scrub ok 2024-09-11T07:33:52.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:52 smithi022 ceph-mon[29462]: 2.3 scrub starts 2024-09-11T07:33:52.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:52 smithi022 ceph-mon[29462]: 2.3 scrub ok 2024-09-11T07:33:52.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:52 smithi022 ceph-mon[29462]: 5.8 scrub starts 2024-09-11T07:33:52.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:52 smithi022 ceph-mon[29462]: 5.8 scrub ok 2024-09-11T07:33:52.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:52 smithi022 ceph-mon[29462]: 5.12 scrub starts 2024-09-11T07:33:52.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:52 smithi022 ceph-mon[29462]: 5.12 scrub ok 2024-09-11T07:33:52.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:52 smithi022 ceph-mon[37330]: 4.1d scrub starts 2024-09-11T07:33:52.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:52 smithi022 ceph-mon[37330]: 4.1d scrub ok 2024-09-11T07:33:52.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:52 smithi022 ceph-mon[37330]: 5.3 scrub starts 2024-09-11T07:33:52.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:52 smithi022 ceph-mon[37330]: 5.3 scrub ok 2024-09-11T07:33:52.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:52 smithi022 ceph-mon[37330]: 2.3 scrub starts 2024-09-11T07:33:52.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:52 smithi022 ceph-mon[37330]: 2.3 scrub ok 2024-09-11T07:33:52.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:52 smithi022 ceph-mon[37330]: 5.8 scrub starts 2024-09-11T07:33:52.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:52 smithi022 ceph-mon[37330]: 5.8 scrub ok 2024-09-11T07:33:52.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:52 smithi022 ceph-mon[37330]: 5.12 scrub starts 2024-09-11T07:33:52.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:52 smithi022 ceph-mon[37330]: 5.12 scrub ok 2024-09-11T07:33:52.467 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:52.489 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:52.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:52.490 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:52.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:52 smithi104 ceph-mon[33507]: 4.1d scrub starts 2024-09-11T07:33:52.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:52 smithi104 ceph-mon[33507]: 4.1d scrub ok 2024-09-11T07:33:52.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:52 smithi104 ceph-mon[33507]: 5.3 scrub starts 2024-09-11T07:33:52.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:52 smithi104 ceph-mon[33507]: 5.3 scrub ok 2024-09-11T07:33:52.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:52 smithi104 ceph-mon[33507]: 2.3 scrub starts 2024-09-11T07:33:52.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:52 smithi104 ceph-mon[33507]: 2.3 scrub ok 2024-09-11T07:33:52.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:52 smithi104 ceph-mon[33507]: 5.8 scrub starts 2024-09-11T07:33:52.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:52 smithi104 ceph-mon[33507]: 5.8 scrub ok 2024-09-11T07:33:52.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:52 smithi104 ceph-mon[33507]: 5.12 scrub starts 2024-09-11T07:33:52.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:52 smithi104 ceph-mon[33507]: 5.12 scrub ok 2024-09-11T07:33:52.592 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:52.618 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:52.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:52.620 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:52.720 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:52.763 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:52.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:52.765 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:52.865 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:52.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:52.890 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:52.990 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:53.028 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:53.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:53.030 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:53.130 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:53.160 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:53.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:53.161 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:53.200 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 13 15 32606 32591 156.665 110.438 0.00744009 0.00637812 2024-09-11T07:33:53.261 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:53.299 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:53.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:53.301 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:53.402 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:53.423 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:53.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:53.424 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:53.525 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:53.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:53 smithi104 ceph-mon[33507]: 5.4 scrub starts 2024-09-11T07:33:53.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:53 smithi104 ceph-mon[33507]: 5.4 scrub ok 2024-09-11T07:33:53.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:53 smithi104 ceph-mon[33507]: 2.13 scrub starts 2024-09-11T07:33:53.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:53 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:33:53.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:53 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:33:53.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:53 smithi104 ceph-mon[33507]: 2.13 scrub ok 2024-09-11T07:33:53.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:53 smithi104 ceph-mon[33507]: pgmap v322: 148 pgs: 148 active+clean; 1.5 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 161 MiB/s wr, 2.58k op/s 2024-09-11T07:33:53.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:53 smithi104 ceph-mon[33507]: 4.15 scrub starts 2024-09-11T07:33:53.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:53 smithi104 ceph-mon[33507]: 4.15 scrub ok 2024-09-11T07:33:53.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:53 smithi104 ceph-mon[33507]: osdmap e190: 8 total, 8 up, 8 in 2024-09-11T07:33:53.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:33:53.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:33:53.565 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:53.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:53.566 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:53.667 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:53.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:53 smithi022 ceph-mon[37330]: 5.4 scrub starts 2024-09-11T07:33:53.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:53 smithi022 ceph-mon[37330]: 5.4 scrub ok 2024-09-11T07:33:53.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:53 smithi022 ceph-mon[37330]: 2.13 scrub starts 2024-09-11T07:33:53.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:53 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:33:53.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:53 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:33:53.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:53 smithi022 ceph-mon[37330]: 2.13 scrub ok 2024-09-11T07:33:53.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:53 smithi022 ceph-mon[37330]: pgmap v322: 148 pgs: 148 active+clean; 1.5 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 161 MiB/s wr, 2.58k op/s 2024-09-11T07:33:53.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:53 smithi022 ceph-mon[37330]: 4.15 scrub starts 2024-09-11T07:33:53.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:53 smithi022 ceph-mon[37330]: 4.15 scrub ok 2024-09-11T07:33:53.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:53 smithi022 ceph-mon[37330]: osdmap e190: 8 total, 8 up, 8 in 2024-09-11T07:33:53.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:33:53.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:33:53.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:53 smithi022 ceph-mon[29462]: 5.4 scrub starts 2024-09-11T07:33:53.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:53 smithi022 ceph-mon[29462]: 5.4 scrub ok 2024-09-11T07:33:53.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:53 smithi022 ceph-mon[29462]: 2.13 scrub starts 2024-09-11T07:33:53.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:53 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:33:53.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:53 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:33:53.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:53 smithi022 ceph-mon[29462]: 2.13 scrub ok 2024-09-11T07:33:53.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:53 smithi022 ceph-mon[29462]: pgmap v322: 148 pgs: 148 active+clean; 1.5 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 161 MiB/s wr, 2.58k op/s 2024-09-11T07:33:53.670 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:53 smithi022 ceph-mon[29462]: 4.15 scrub starts 2024-09-11T07:33:53.670 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:53 smithi022 ceph-mon[29462]: 4.15 scrub ok 2024-09-11T07:33:53.670 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:53 smithi022 ceph-mon[29462]: osdmap e190: 8 total, 8 up, 8 in 2024-09-11T07:33:53.670 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:33:53.670 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:33:53.707 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:53.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:53.708 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:53.809 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:53.833 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:53.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:53.835 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:53.936 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:53.974 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:53.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:53.975 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:54.076 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:54.098 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:54.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:54.099 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:54.200 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:54.201 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 14 16 34268 34252 152.889 103.812 0.00438181 0.00653583 2024-09-11T07:33:54.241 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:54.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:54.242 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:54.343 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:54.378 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:54.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:54.379 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:54.431 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:33:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:33:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:33:54.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:54 smithi104 ceph-mon[33507]: 2.b scrub starts 2024-09-11T07:33:54.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:54 smithi104 ceph-mon[33507]: 2.b scrub ok 2024-09-11T07:33:54.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:54 smithi104 ceph-mon[33507]: 4.0 scrub starts 2024-09-11T07:33:54.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:54 smithi104 ceph-mon[33507]: 4.0 scrub ok 2024-09-11T07:33:54.432 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:54 smithi104 ceph-mon[33507]: 5.5 scrub starts 2024-09-11T07:33:54.432 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:54 smithi104 ceph-mon[33507]: 5.5 scrub ok 2024-09-11T07:33:54.432 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:54 smithi104 ceph-mon[33507]: 2.18 scrub starts 2024-09-11T07:33:54.432 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:54 smithi104 ceph-mon[33507]: 2.18 scrub ok 2024-09-11T07:33:54.432 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:54 smithi104 ceph-mon[33507]: 4.2 scrub starts 2024-09-11T07:33:54.432 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:54 smithi104 ceph-mon[33507]: 4.2 scrub ok 2024-09-11T07:33:54.432 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:54 smithi104 ceph-mon[33507]: 5.1 scrub starts 2024-09-11T07:33:54.432 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:54 smithi104 ceph-mon[33507]: 5.1 scrub ok 2024-09-11T07:33:54.480 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:54.503 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:54.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:54.504 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:54.604 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:54.645 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:54.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:54.646 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:54.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:54 smithi022 ceph-mon[29462]: 2.b scrub starts 2024-09-11T07:33:54.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:54 smithi022 ceph-mon[29462]: 2.b scrub ok 2024-09-11T07:33:54.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:54 smithi022 ceph-mon[29462]: 4.0 scrub starts 2024-09-11T07:33:54.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:54 smithi022 ceph-mon[29462]: 4.0 scrub ok 2024-09-11T07:33:54.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:54 smithi022 ceph-mon[29462]: 5.5 scrub starts 2024-09-11T07:33:54.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:54 smithi022 ceph-mon[29462]: 5.5 scrub ok 2024-09-11T07:33:54.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:54 smithi022 ceph-mon[29462]: 2.18 scrub starts 2024-09-11T07:33:54.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:54 smithi022 ceph-mon[29462]: 2.18 scrub ok 2024-09-11T07:33:54.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:54 smithi022 ceph-mon[29462]: 4.2 scrub starts 2024-09-11T07:33:54.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:54 smithi022 ceph-mon[29462]: 4.2 scrub ok 2024-09-11T07:33:54.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:54 smithi022 ceph-mon[29462]: 5.1 scrub starts 2024-09-11T07:33:54.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:54 smithi022 ceph-mon[29462]: 5.1 scrub ok 2024-09-11T07:33:54.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:54 smithi022 ceph-mon[37330]: 2.b scrub starts 2024-09-11T07:33:54.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:54 smithi022 ceph-mon[37330]: 2.b scrub ok 2024-09-11T07:33:54.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:54 smithi022 ceph-mon[37330]: 4.0 scrub starts 2024-09-11T07:33:54.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:54 smithi022 ceph-mon[37330]: 4.0 scrub ok 2024-09-11T07:33:54.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:54 smithi022 ceph-mon[37330]: 5.5 scrub starts 2024-09-11T07:33:54.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:54 smithi022 ceph-mon[37330]: 5.5 scrub ok 2024-09-11T07:33:54.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:54 smithi022 ceph-mon[37330]: 2.18 scrub starts 2024-09-11T07:33:54.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:54 smithi022 ceph-mon[37330]: 2.18 scrub ok 2024-09-11T07:33:54.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:54 smithi022 ceph-mon[37330]: 4.2 scrub starts 2024-09-11T07:33:54.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:54 smithi022 ceph-mon[37330]: 4.2 scrub ok 2024-09-11T07:33:54.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:54 smithi022 ceph-mon[37330]: 5.1 scrub starts 2024-09-11T07:33:54.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:54 smithi022 ceph-mon[37330]: 5.1 scrub ok 2024-09-11T07:33:54.747 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:54.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:54.777 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:54.777 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:54.878 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:54.916 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:54.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:54.917 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:55.018 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:55.051 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:55.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:55.053 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:55.153 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:55.190 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:55.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:55.191 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:55.201 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 15 16 35811 35795 149.125 96.4375 0.00711981 0.00669963 2024-09-11T07:33:55.291 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:55.320 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:55.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:55.321 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:55.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:55 smithi022 ceph-mon[29462]: 6.12 scrub starts 2024-09-11T07:33:55.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:55 smithi022 ceph-mon[29462]: 6.12 scrub ok 2024-09-11T07:33:55.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:55 smithi022 ceph-mon[29462]: 4.19 scrub starts 2024-09-11T07:33:55.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:55 smithi022 ceph-mon[29462]: 4.19 scrub ok 2024-09-11T07:33:55.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:55 smithi022 ceph-mon[29462]: 6.10 scrub starts 2024-09-11T07:33:55.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:55 smithi022 ceph-mon[29462]: 6.10 scrub ok 2024-09-11T07:33:55.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:55 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:33:55.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:55 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:33:55.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:55 smithi022 ceph-mon[29462]: pgmap v324: 148 pgs: 148 active+clean; 1.7 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 738 B/s rd, 147 MiB/s wr, 2.35k op/s 2024-09-11T07:33:55.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:55 smithi022 ceph-mon[29462]: osdmap e191: 8 total, 8 up, 8 in 2024-09-11T07:33:55.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:55 smithi022 ceph-mon[29462]: 6.1e scrub starts 2024-09-11T07:33:55.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:55 smithi022 ceph-mon[29462]: 6.1e scrub ok 2024-09-11T07:33:55.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:55 smithi022 ceph-mon[37330]: 6.12 scrub starts 2024-09-11T07:33:55.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:55 smithi022 ceph-mon[37330]: 6.12 scrub ok 2024-09-11T07:33:55.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:55 smithi022 ceph-mon[37330]: 4.19 scrub starts 2024-09-11T07:33:55.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:55 smithi022 ceph-mon[37330]: 4.19 scrub ok 2024-09-11T07:33:55.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:55 smithi022 ceph-mon[37330]: 6.10 scrub starts 2024-09-11T07:33:55.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:55 smithi022 ceph-mon[37330]: 6.10 scrub ok 2024-09-11T07:33:55.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:55 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:33:55.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:55 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:33:55.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:55 smithi022 ceph-mon[37330]: pgmap v324: 148 pgs: 148 active+clean; 1.7 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 738 B/s rd, 147 MiB/s wr, 2.35k op/s 2024-09-11T07:33:55.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:55 smithi022 ceph-mon[37330]: osdmap e191: 8 total, 8 up, 8 in 2024-09-11T07:33:55.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:55 smithi022 ceph-mon[37330]: 6.1e scrub starts 2024-09-11T07:33:55.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:55 smithi022 ceph-mon[37330]: 6.1e scrub ok 2024-09-11T07:33:55.422 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:55.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:55.462 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:55.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:55 smithi104 ceph-mon[33507]: 6.12 scrub starts 2024-09-11T07:33:55.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:55 smithi104 ceph-mon[33507]: 6.12 scrub ok 2024-09-11T07:33:55.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:55 smithi104 ceph-mon[33507]: 4.19 scrub starts 2024-09-11T07:33:55.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:55 smithi104 ceph-mon[33507]: 4.19 scrub ok 2024-09-11T07:33:55.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:55 smithi104 ceph-mon[33507]: 6.10 scrub starts 2024-09-11T07:33:55.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:55 smithi104 ceph-mon[33507]: 6.10 scrub ok 2024-09-11T07:33:55.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:55 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:33:55.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:55 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:33:55.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:55 smithi104 ceph-mon[33507]: pgmap v324: 148 pgs: 148 active+clean; 1.7 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 738 B/s rd, 147 MiB/s wr, 2.35k op/s 2024-09-11T07:33:55.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:55 smithi104 ceph-mon[33507]: osdmap e191: 8 total, 8 up, 8 in 2024-09-11T07:33:55.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:55 smithi104 ceph-mon[33507]: 6.1e scrub starts 2024-09-11T07:33:55.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:55 smithi104 ceph-mon[33507]: 6.1e scrub ok 2024-09-11T07:33:55.563 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:55.586 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:55.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:55.587 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:55.688 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:55.724 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:55.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:55.725 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:55.826 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:55.866 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:55.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:55.867 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:55.968 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:55.999 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:55.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:56.000 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:56.100 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:56.129 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:56.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:56.130 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:56.201 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 16 16 37551 37535 146.601 108.75 0.00325681 0.0068129 2024-09-11T07:33:56.231 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:56.268 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:56.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:56.269 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:56.369 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:56.400 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:56.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:56.401 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:56.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:56 smithi022 ceph-mon[29462]: 6.15 scrub starts 2024-09-11T07:33:56.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:56 smithi022 ceph-mon[29462]: 6.15 scrub ok 2024-09-11T07:33:56.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:56 smithi022 ceph-mon[29462]: 5.17 scrub starts 2024-09-11T07:33:56.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:56 smithi022 ceph-mon[29462]: 5.17 scrub ok 2024-09-11T07:33:56.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:56 smithi022 ceph-mon[29462]: 4.1c scrub starts 2024-09-11T07:33:56.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:56 smithi022 ceph-mon[29462]: 4.1c scrub ok 2024-09-11T07:33:56.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:56 smithi022 ceph-mon[29462]: 2.4 scrub starts 2024-09-11T07:33:56.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:56 smithi022 ceph-mon[29462]: 2.4 scrub ok 2024-09-11T07:33:56.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:56 smithi022 ceph-mon[37330]: 6.15 scrub starts 2024-09-11T07:33:56.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:56 smithi022 ceph-mon[37330]: 6.15 scrub ok 2024-09-11T07:33:56.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:56 smithi022 ceph-mon[37330]: 5.17 scrub starts 2024-09-11T07:33:56.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:56 smithi022 ceph-mon[37330]: 5.17 scrub ok 2024-09-11T07:33:56.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:56 smithi022 ceph-mon[37330]: 4.1c scrub starts 2024-09-11T07:33:56.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:56 smithi022 ceph-mon[37330]: 4.1c scrub ok 2024-09-11T07:33:56.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:56 smithi022 ceph-mon[37330]: 2.4 scrub starts 2024-09-11T07:33:56.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:56 smithi022 ceph-mon[37330]: 2.4 scrub ok 2024-09-11T07:33:56.502 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:56.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:56 smithi104 ceph-mon[33507]: 6.15 scrub starts 2024-09-11T07:33:56.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:56 smithi104 ceph-mon[33507]: 6.15 scrub ok 2024-09-11T07:33:56.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:56 smithi104 ceph-mon[33507]: 5.17 scrub starts 2024-09-11T07:33:56.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:56 smithi104 ceph-mon[33507]: 5.17 scrub ok 2024-09-11T07:33:56.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:56 smithi104 ceph-mon[33507]: 4.1c scrub starts 2024-09-11T07:33:56.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:56 smithi104 ceph-mon[33507]: 4.1c scrub ok 2024-09-11T07:33:56.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:56 smithi104 ceph-mon[33507]: 2.4 scrub starts 2024-09-11T07:33:56.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:56 smithi104 ceph-mon[33507]: 2.4 scrub ok 2024-09-11T07:33:56.531 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:56.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:56.532 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:56.633 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:56.654 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:56.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:56.655 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:56.756 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:56.786 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:56.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:56.787 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:56.888 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:56.931 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:56.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:56.932 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:57.033 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:57.071 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:57.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:57.073 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:57.173 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:57.201 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 17 16 39452 39436 144.966 118.812 0.00384752 0.00689278 2024-09-11T07:33:57.204 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:57.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:57.205 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:57.306 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:57.345 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:57.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:57.346 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:57.447 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:57.486 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:57.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:57.487 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:57.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:57 smithi104 ceph-mon[33507]: 6.5 scrub starts 2024-09-11T07:33:57.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:57 smithi104 ceph-mon[33507]: 6.5 scrub ok 2024-09-11T07:33:57.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:57 smithi104 ceph-mon[33507]: pgmap v326: 148 pgs: 148 active+clean; 2.1 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1.4 KiB/s rd, 156 MiB/s wr, 2.50k op/s 2024-09-11T07:33:57.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:57 smithi104 ceph-mon[33507]: 4.8 scrub starts 2024-09-11T07:33:57.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:57 smithi104 ceph-mon[33507]: 4.8 scrub ok 2024-09-11T07:33:57.588 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:57.622 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:57.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:57.623 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:57.637 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:57 smithi022 ceph-mon[29462]: 6.5 scrub starts 2024-09-11T07:33:57.637 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:57 smithi022 ceph-mon[29462]: 6.5 scrub ok 2024-09-11T07:33:57.638 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:57 smithi022 ceph-mon[29462]: pgmap v326: 148 pgs: 148 active+clean; 2.1 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1.4 KiB/s rd, 156 MiB/s wr, 2.50k op/s 2024-09-11T07:33:57.638 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:57 smithi022 ceph-mon[29462]: 4.8 scrub starts 2024-09-11T07:33:57.638 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:57 smithi022 ceph-mon[29462]: 4.8 scrub ok 2024-09-11T07:33:57.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:57 smithi022 ceph-mon[37330]: 6.5 scrub starts 2024-09-11T07:33:57.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:57 smithi022 ceph-mon[37330]: 6.5 scrub ok 2024-09-11T07:33:57.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:57 smithi022 ceph-mon[37330]: pgmap v326: 148 pgs: 148 active+clean; 2.1 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1.4 KiB/s rd, 156 MiB/s wr, 2.50k op/s 2024-09-11T07:33:57.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:57 smithi022 ceph-mon[37330]: 4.8 scrub starts 2024-09-11T07:33:57.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:57 smithi022 ceph-mon[37330]: 4.8 scrub ok 2024-09-11T07:33:57.684 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-11T07:33:57.724 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:57.764 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:57.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:57.765 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:57.866 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:57.891 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:57.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:57.892 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:57.993 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:58.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:58.034 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:58.034 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:58.055 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:33:58.135 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:33:58.165 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:58.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:58.166 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:33:58.203 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 18 16 41291 41275 143.278 114.938 0.0156581 0.00697365 2024-09-11T07:33:58.267 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:33:58.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:58.297 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:58.297 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:33:58.398 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:58.431 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:58.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:58.432 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:58.442 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:58 smithi104 ceph-mon[33507]: 6.f scrub starts 2024-09-11T07:33:58.442 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:58 smithi104 ceph-mon[33507]: 6.f scrub ok 2024-09-11T07:33:58.442 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:58 smithi104 ceph-mon[33507]: 2.15 scrub starts 2024-09-11T07:33:58.443 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:58 smithi104 ceph-mon[33507]: 2.15 scrub ok 2024-09-11T07:33:58.533 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:58.563 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:58.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:58.564 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:58.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:58 smithi022 ceph-mon[29462]: 6.f scrub starts 2024-09-11T07:33:58.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:58 smithi022 ceph-mon[29462]: 6.f scrub ok 2024-09-11T07:33:58.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:58 smithi022 ceph-mon[29462]: 2.15 scrub starts 2024-09-11T07:33:58.607 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:58 smithi022 ceph-mon[29462]: 2.15 scrub ok 2024-09-11T07:33:58.607 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:58 smithi022 ceph-mon[37330]: 6.f scrub starts 2024-09-11T07:33:58.607 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:58 smithi022 ceph-mon[37330]: 6.f scrub ok 2024-09-11T07:33:58.608 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:58 smithi022 ceph-mon[37330]: 2.15 scrub starts 2024-09-11T07:33:58.608 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:58 smithi022 ceph-mon[37330]: 2.15 scrub ok 2024-09-11T07:33:58.665 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:33:58.729 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:58.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:58.731 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:33:58.734 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:33:58.832 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:33:58.876 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:58.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:58.878 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:33:58.979 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:59.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:59.012 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:59.013 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:33:59.114 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:59.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:59.141 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:59.141 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:59.203 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 19 15 43132 43117 141.796 115.125 0.00621713 0.0070477 2024-09-11T07:33:59.241 INFO:teuthology.orchestra.run.smithi022.stderr:reweighted osd.6 to 1 (10000) 2024-09-11T07:33:59.243 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:59.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:59.289 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:59.290 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:59.390 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:59.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:59 smithi022 ceph-mon[29462]: 2.1d scrub starts 2024-09-11T07:33:59.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:59 smithi022 ceph-mon[29462]: 2.1d scrub ok 2024-09-11T07:33:59.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:59 smithi022 ceph-mon[29462]: pgmap v327: 148 pgs: 148 active+clean; 2.1 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 73 MiB/s wr, 1.17k op/s 2024-09-11T07:33:59.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:59 smithi022 ceph-mon[29462]: 2.0 scrub starts 2024-09-11T07:33:59.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:59 smithi022 ceph-mon[29462]: 2.0 scrub ok 2024-09-11T07:33:59.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:59 smithi022 ceph-mon[29462]: 6.4 scrub starts 2024-09-11T07:33:59.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:59 smithi022 ceph-mon[29462]: 6.4 scrub ok 2024-09-11T07:33:59.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:59 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2507070676' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T07:33:59.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:33:59 smithi022 ceph-mon[29462]: from='client.25093 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T07:33:59.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:59 smithi022 ceph-mon[37330]: 2.1d scrub starts 2024-09-11T07:33:59.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:59 smithi022 ceph-mon[37330]: 2.1d scrub ok 2024-09-11T07:33:59.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:59 smithi022 ceph-mon[37330]: pgmap v327: 148 pgs: 148 active+clean; 2.1 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 73 MiB/s wr, 1.17k op/s 2024-09-11T07:33:59.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:59 smithi022 ceph-mon[37330]: 2.0 scrub starts 2024-09-11T07:33:59.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:59 smithi022 ceph-mon[37330]: 2.0 scrub ok 2024-09-11T07:33:59.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:59 smithi022 ceph-mon[37330]: 6.4 scrub starts 2024-09-11T07:33:59.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:59 smithi022 ceph-mon[37330]: 6.4 scrub ok 2024-09-11T07:33:59.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:59 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2507070676' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T07:33:59.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:33:59 smithi022 ceph-mon[37330]: from='client.25093 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T07:33:59.427 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:59.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:59.428 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:59.529 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:33:59.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:59 smithi104 ceph-mon[33507]: 2.1d scrub starts 2024-09-11T07:33:59.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:59 smithi104 ceph-mon[33507]: 2.1d scrub ok 2024-09-11T07:33:59.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:59 smithi104 ceph-mon[33507]: pgmap v327: 148 pgs: 148 active+clean; 2.1 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 73 MiB/s wr, 1.17k op/s 2024-09-11T07:33:59.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:59 smithi104 ceph-mon[33507]: 2.0 scrub starts 2024-09-11T07:33:59.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:59 smithi104 ceph-mon[33507]: 2.0 scrub ok 2024-09-11T07:33:59.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:59 smithi104 ceph-mon[33507]: 6.4 scrub starts 2024-09-11T07:33:59.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:59 smithi104 ceph-mon[33507]: 6.4 scrub ok 2024-09-11T07:33:59.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:59 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2507070676' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T07:33:59.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:33:59 smithi104 ceph-mon[33507]: from='client.25093 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T07:33:59.531 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:33:59 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:33:59.541 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:33:59.541 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:33:59.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:59.575 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:33:59.578 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:33:59.578 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:33:59.617 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:33:59.617 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:33:59.651 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:33:59.651 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:33:59.676 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:33:59.684 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:33:59.685 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:33:59.717 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:33:59.718 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:33:59.718 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:33:59.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:59.719 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:33:59.745 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:33:59.745 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:33:59.790 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:33:59.790 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:33:59.819 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:33:59.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:59.856 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:59.856 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:33:59.957 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:33:59.990 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:33:59.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:33:59.991 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:00.092 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:00.129 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:00.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:00.131 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:00.204 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:2024-09-11T07:34:00.203060+0000 min lat: 0.00108265 max lat: 0.159168 avg lat: 0.00712792 2024-09-11T07:34:00.204 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T07:34:00.204 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 20 16 44890 44874 140.197 109.812 0.0166453 0.00712792 2024-09-11T07:34:00.219 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:34:00.219 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:34:00.231 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:00.265 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:00.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:00.266 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:00.367 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:00.405 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:00.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:00.406 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:00.507 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:00.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:00 smithi104 ceph-mon[33507]: from='client.25093 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-11T07:34:00.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:00 smithi104 ceph-mon[33507]: osdmap e192: 8 total, 8 up, 8 in 2024-09-11T07:34:00.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:00 smithi104 ceph-mon[33507]: 2.9 scrub starts 2024-09-11T07:34:00.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:00 smithi104 ceph-mon[33507]: 2.9 scrub ok 2024-09-11T07:34:00.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:00 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:00.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:00 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1341955277' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:34:00.542 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:00.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:00.543 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:00.644 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:00.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:34:00 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:34:00] "GET /metrics HTTP/1.1" 200 223951 "" "Prometheus/2.43.0" 2024-09-11T07:34:00.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:00 smithi022 ceph-mon[29462]: from='client.25093 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-11T07:34:00.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:00 smithi022 ceph-mon[29462]: osdmap e192: 8 total, 8 up, 8 in 2024-09-11T07:34:00.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:00 smithi022 ceph-mon[29462]: 2.9 scrub starts 2024-09-11T07:34:00.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:00 smithi022 ceph-mon[29462]: 2.9 scrub ok 2024-09-11T07:34:00.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:00 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:00.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:00 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1341955277' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:34:00.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:00 smithi022 ceph-mon[37330]: from='client.25093 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-11T07:34:00.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:00 smithi022 ceph-mon[37330]: osdmap e192: 8 total, 8 up, 8 in 2024-09-11T07:34:00.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:00 smithi022 ceph-mon[37330]: 2.9 scrub starts 2024-09-11T07:34:00.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:00 smithi022 ceph-mon[37330]: 2.9 scrub ok 2024-09-11T07:34:00.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:00 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:00.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:00 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1341955277' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:34:00.677 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:00.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:00.678 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:00.779 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:00.820 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:00.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:00.821 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:00.922 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:00.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:00.960 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:00.960 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:01.061 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:01.101 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:01.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:01.102 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:01.110 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-11T07:34:01.203 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:01.204 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 21 16 46716 46700 138.955 114.125 0.00853564 0.00719128 2024-09-11T07:34:01.243 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:01.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:01.244 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:01.345 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:01.381 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:01.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:01.382 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:01.483 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:01.519 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:01.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:01.520 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:01.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:01 smithi104 ceph-mon[33507]: pgmap v329: 148 pgs: 148 active+clean; 2.6 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.3 KiB/s rd, 145 MiB/s wr, 2.31k op/s 2024-09-11T07:34:01.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:01 smithi104 ceph-mon[33507]: 5.c scrub starts 2024-09-11T07:34:01.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:01 smithi104 ceph-mon[33507]: 5.c scrub ok 2024-09-11T07:34:01.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:01 smithi104 ceph-mon[33507]: 4.13 scrub starts 2024-09-11T07:34:01.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:01 smithi104 ceph-mon[33507]: 4.13 scrub ok 2024-09-11T07:34:01.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:01 smithi104 ceph-mon[33507]: osdmap e193: 8 total, 8 up, 8 in 2024-09-11T07:34:01.570 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:34:01.621 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:01.648 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:01 smithi022 ceph-mon[29462]: pgmap v329: 148 pgs: 148 active+clean; 2.6 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.3 KiB/s rd, 145 MiB/s wr, 2.31k op/s 2024-09-11T07:34:01.648 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:01 smithi022 ceph-mon[29462]: 5.c scrub starts 2024-09-11T07:34:01.648 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:01 smithi022 ceph-mon[29462]: 5.c scrub ok 2024-09-11T07:34:01.649 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:01 smithi022 ceph-mon[29462]: 4.13 scrub starts 2024-09-11T07:34:01.649 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:01 smithi022 ceph-mon[29462]: 4.13 scrub ok 2024-09-11T07:34:01.649 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:01 smithi022 ceph-mon[29462]: osdmap e193: 8 total, 8 up, 8 in 2024-09-11T07:34:01.649 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:01 smithi022 ceph-mon[37330]: pgmap v329: 148 pgs: 148 active+clean; 2.6 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.3 KiB/s rd, 145 MiB/s wr, 2.31k op/s 2024-09-11T07:34:01.649 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:01 smithi022 ceph-mon[37330]: 5.c scrub starts 2024-09-11T07:34:01.649 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:01 smithi022 ceph-mon[37330]: 5.c scrub ok 2024-09-11T07:34:01.649 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:01 smithi022 ceph-mon[37330]: 4.13 scrub starts 2024-09-11T07:34:01.649 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:01 smithi022 ceph-mon[37330]: 4.13 scrub ok 2024-09-11T07:34:01.649 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:01 smithi022 ceph-mon[37330]: osdmap e193: 8 total, 8 up, 8 in 2024-09-11T07:34:01.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:01.652 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:01.653 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:01.753 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:01.786 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:01.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:01.787 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:01.888 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:01.916 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:01.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:01.917 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:02.018 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:02.049 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:02.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:02.050 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:02.152 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:02.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:02.177 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:02.177 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:02.204 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 22 16 48256 48240 137.014 96.25 0.00240476 0.00728215 2024-09-11T07:34:02.278 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:02.314 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:02.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:02.315 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:02.416 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:02.457 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:02.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:02.458 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:02.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:02 smithi104 ceph-mon[33507]: 2.1c scrub starts 2024-09-11T07:34:02.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:02 smithi104 ceph-mon[33507]: 2.1c scrub ok 2024-09-11T07:34:02.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:02 smithi104 ceph-mon[33507]: 5.1d scrub starts 2024-09-11T07:34:02.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:02 smithi104 ceph-mon[33507]: 5.1d scrub ok 2024-09-11T07:34:02.543 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:02 smithi022 ceph-mon[29462]: 2.1c scrub starts 2024-09-11T07:34:02.543 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:02 smithi022 ceph-mon[29462]: 2.1c scrub ok 2024-09-11T07:34:02.543 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:02 smithi022 ceph-mon[29462]: 5.1d scrub starts 2024-09-11T07:34:02.543 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:02 smithi022 ceph-mon[29462]: 5.1d scrub ok 2024-09-11T07:34:02.544 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:02 smithi022 ceph-mon[37330]: 2.1c scrub starts 2024-09-11T07:34:02.544 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:02 smithi022 ceph-mon[37330]: 2.1c scrub ok 2024-09-11T07:34:02.544 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:02 smithi022 ceph-mon[37330]: 5.1d scrub starts 2024-09-11T07:34:02.544 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:02 smithi022 ceph-mon[37330]: 5.1d scrub ok 2024-09-11T07:34:02.545 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight 7 1 2024-09-11T07:34:02.560 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:02.636 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:02.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:02.637 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:02.738 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:02.777 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:02.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:02.778 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:02.879 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:02.907 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:02.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:02.908 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:03.009 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:34:03.009 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:03.035 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:03.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:03.036 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:03.137 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:03.165 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:03.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:03.166 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:03.204 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 23 16 49769 49753 135.168 94.5625 0.00719002 0.00739286 2024-09-11T07:34:03.266 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:03.283 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is unset 2024-09-11T07:34:03.290 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:34:03.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:03.318 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:03.318 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:03.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:03 smithi022 ceph-mon[29462]: pgmap v331: 148 pgs: 148 active+clean; 2.6 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 649 B/s rd, 97 MiB/s wr, 1.56k op/s 2024-09-11T07:34:03.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:03 smithi022 ceph-mon[29462]: 2.c scrub starts 2024-09-11T07:34:03.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:03 smithi022 ceph-mon[29462]: 2.c scrub ok 2024-09-11T07:34:03.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:03 smithi022 ceph-mon[29462]: 2.12 scrub starts 2024-09-11T07:34:03.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:03 smithi022 ceph-mon[29462]: 2.12 scrub ok 2024-09-11T07:34:03.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:03 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/563790889' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:34:03.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:03 smithi022 ceph-mon[29462]: from='client.25105 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:34:03.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:03 smithi022 ceph-mon[37330]: pgmap v331: 148 pgs: 148 active+clean; 2.6 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 649 B/s rd, 97 MiB/s wr, 1.56k op/s 2024-09-11T07:34:03.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:03 smithi022 ceph-mon[37330]: 2.c scrub starts 2024-09-11T07:34:03.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:03 smithi022 ceph-mon[37330]: 2.c scrub ok 2024-09-11T07:34:03.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:03 smithi022 ceph-mon[37330]: 2.12 scrub starts 2024-09-11T07:34:03.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:03 smithi022 ceph-mon[37330]: 2.12 scrub ok 2024-09-11T07:34:03.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:03 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/563790889' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:34:03.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:03 smithi022 ceph-mon[37330]: from='client.25105 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:34:03.419 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:03.467 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:03.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:03.469 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:03.569 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:03.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:03.627 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:03.627 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:03.683 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:03 smithi104 ceph-mon[33507]: pgmap v331: 148 pgs: 148 active+clean; 2.6 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 649 B/s rd, 97 MiB/s wr, 1.56k op/s 2024-09-11T07:34:03.684 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:03 smithi104 ceph-mon[33507]: 2.c scrub starts 2024-09-11T07:34:03.684 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:03 smithi104 ceph-mon[33507]: 2.c scrub ok 2024-09-11T07:34:03.684 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:03 smithi104 ceph-mon[33507]: 2.12 scrub starts 2024-09-11T07:34:03.684 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:03 smithi104 ceph-mon[33507]: 2.12 scrub ok 2024-09-11T07:34:03.684 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:03 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/563790889' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:34:03.684 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:03 smithi104 ceph-mon[33507]: from='client.25105 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:34:03.728 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:03.767 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:03.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:03.768 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:03.869 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:03.909 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:03.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:03.911 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:04.011 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:04.051 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:04.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:04.052 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:04.153 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:04.173 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:34:04.174 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:34:04.194 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:04.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:04.195 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:04.204 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 24 16 51354 51338 133.663 99.0625 0.00451653 0.00747619 2024-09-11T07:34:04.279 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:34:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:34:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:34:04.296 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:04.328 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:04.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:04.330 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:04.430 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:04.470 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:04.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:04.471 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:04.527 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:04 smithi022 ceph-mon[29462]: 5.1f scrub starts 2024-09-11T07:34:04.528 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:04 smithi022 ceph-mon[29462]: 5.1f scrub ok 2024-09-11T07:34:04.528 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:04 smithi022 ceph-mon[29462]: 2.10 scrub starts 2024-09-11T07:34:04.528 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:04 smithi022 ceph-mon[29462]: 2.10 scrub ok 2024-09-11T07:34:04.528 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:04 smithi022 ceph-mon[29462]: 6.19 scrub starts 2024-09-11T07:34:04.528 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:04 smithi022 ceph-mon[29462]: 6.19 scrub ok 2024-09-11T07:34:04.529 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:04 smithi022 ceph-mon[29462]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:34:04.529 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:04 smithi022 ceph-mon[29462]: Cluster is now healthy 2024-09-11T07:34:04.529 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:04 smithi022 ceph-mon[29462]: from='client.25105 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:34:04.529 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:04 smithi022 ceph-mon[29462]: osdmap e194: 8 total, 8 up, 8 in 2024-09-11T07:34:04.529 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:04 smithi022 ceph-mon[29462]: 4.7 scrub starts 2024-09-11T07:34:04.530 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:04 smithi022 ceph-mon[29462]: 4.7 scrub ok 2024-09-11T07:34:04.530 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:04 smithi022 ceph-mon[37330]: 5.1f scrub starts 2024-09-11T07:34:04.530 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:04 smithi022 ceph-mon[37330]: 5.1f scrub ok 2024-09-11T07:34:04.531 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:04 smithi022 ceph-mon[37330]: 2.10 scrub starts 2024-09-11T07:34:04.531 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:04 smithi022 ceph-mon[37330]: 2.10 scrub ok 2024-09-11T07:34:04.531 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:04 smithi022 ceph-mon[37330]: 6.19 scrub starts 2024-09-11T07:34:04.531 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:04 smithi022 ceph-mon[37330]: 6.19 scrub ok 2024-09-11T07:34:04.531 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:04 smithi022 ceph-mon[37330]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:34:04.531 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:04 smithi022 ceph-mon[37330]: Cluster is now healthy 2024-09-11T07:34:04.532 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:04 smithi022 ceph-mon[37330]: from='client.25105 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:34:04.532 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:04 smithi022 ceph-mon[37330]: osdmap e194: 8 total, 8 up, 8 in 2024-09-11T07:34:04.532 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:04 smithi022 ceph-mon[37330]: 4.7 scrub starts 2024-09-11T07:34:04.532 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:04 smithi022 ceph-mon[37330]: 4.7 scrub ok 2024-09-11T07:34:04.554 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:04 smithi104 ceph-mon[33507]: 5.1f scrub starts 2024-09-11T07:34:04.554 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:04 smithi104 ceph-mon[33507]: 5.1f scrub ok 2024-09-11T07:34:04.554 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:04 smithi104 ceph-mon[33507]: 2.10 scrub starts 2024-09-11T07:34:04.554 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:04 smithi104 ceph-mon[33507]: 2.10 scrub ok 2024-09-11T07:34:04.554 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:04 smithi104 ceph-mon[33507]: 6.19 scrub starts 2024-09-11T07:34:04.554 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:04 smithi104 ceph-mon[33507]: 6.19 scrub ok 2024-09-11T07:34:04.554 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:04 smithi104 ceph-mon[33507]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:34:04.555 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:04 smithi104 ceph-mon[33507]: Cluster is now healthy 2024-09-11T07:34:04.555 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:04 smithi104 ceph-mon[33507]: from='client.25105 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:34:04.555 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:04 smithi104 ceph-mon[33507]: osdmap e194: 8 total, 8 up, 8 in 2024-09-11T07:34:04.555 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:04 smithi104 ceph-mon[33507]: 4.7 scrub starts 2024-09-11T07:34:04.555 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:04 smithi104 ceph-mon[33507]: 4.7 scrub ok 2024-09-11T07:34:04.572 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:04.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:04.617 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:04.617 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:04.718 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:04.748 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:04.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:04.749 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:04.850 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:04.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:04.890 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:04.891 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:04.991 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:05.031 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:05.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:05.032 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:05.088 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-11T07:34:05.133 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:05.144 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:34:05.161 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:05.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:05.162 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:05.208 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 25 16 53072 53056 132.605 107.375 0.00875205 0.00753632 2024-09-11T07:34:05.263 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:05.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:05.306 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:05.306 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:05.407 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:05.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:05.453 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:05.454 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:05.464 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:34:05.552 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:05 smithi022 ceph-mon[37330]: pgmap v333: 148 pgs: 148 active+clean; 2.8 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 125 MiB/s wr, 2.00k op/s 2024-09-11T07:34:05.552 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:05 smithi022 ceph-mon[37330]: 4.f deep-scrub starts 2024-09-11T07:34:05.552 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:05 smithi022 ceph-mon[37330]: 4.f deep-scrub ok 2024-09-11T07:34:05.552 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:05 smithi022 ceph-mon[29462]: pgmap v333: 148 pgs: 148 active+clean; 2.8 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 125 MiB/s wr, 2.00k op/s 2024-09-11T07:34:05.552 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:05 smithi022 ceph-mon[29462]: 4.f deep-scrub starts 2024-09-11T07:34:05.552 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:05 smithi022 ceph-mon[29462]: 4.f deep-scrub ok 2024-09-11T07:34:05.555 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:05.599 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:05 smithi104 ceph-mon[33507]: pgmap v333: 148 pgs: 148 active+clean; 2.8 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 125 MiB/s wr, 2.00k op/s 2024-09-11T07:34:05.599 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:05 smithi104 ceph-mon[33507]: 4.f deep-scrub starts 2024-09-11T07:34:05.599 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:05 smithi104 ceph-mon[33507]: 4.f deep-scrub ok 2024-09-11T07:34:05.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:05.600 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:05.600 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:05.701 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:05.732 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:05.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:05.734 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:05.834 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:05.878 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:05.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:05.879 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:05.980 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:06.012 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:06.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:06.013 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:06.114 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:06.143 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:06.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:06.145 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:06.205 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 26 16 54744 54728 131.524 104.5 0.00967918 0.00759838 2024-09-11T07:34:06.246 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:06.296 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:06.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:06.297 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:06.398 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:06.439 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:06.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:06.440 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:06.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:06 smithi104 ceph-mon[33507]: 5.13 scrub starts 2024-09-11T07:34:06.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:06 smithi104 ceph-mon[33507]: 5.13 scrub ok 2024-09-11T07:34:06.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:06 smithi104 ceph-mon[33507]: osdmap e195: 8 total, 8 up, 8 in 2024-09-11T07:34:06.541 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:06.572 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:06.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:06.573 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:06.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:06 smithi022 ceph-mon[37330]: 5.13 scrub starts 2024-09-11T07:34:06.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:06 smithi022 ceph-mon[37330]: 5.13 scrub ok 2024-09-11T07:34:06.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:06 smithi022 ceph-mon[37330]: osdmap e195: 8 total, 8 up, 8 in 2024-09-11T07:34:06.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:06 smithi022 ceph-mon[29462]: 5.13 scrub starts 2024-09-11T07:34:06.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:06 smithi022 ceph-mon[29462]: 5.13 scrub ok 2024-09-11T07:34:06.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:06 smithi022 ceph-mon[29462]: osdmap e195: 8 total, 8 up, 8 in 2024-09-11T07:34:06.674 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:06.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:06.715 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:06.715 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:06.816 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:06.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:06.857 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:06.857 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:06.958 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:07.001 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:07.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:07.002 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:07.013 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T07:34:07.103 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:07.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:07.148 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:07.149 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:07.206 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 27 16 56255 56239 130.15 94.4375 0.00950991 0.00767844 2024-09-11T07:34:07.249 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:07.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:07.289 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:07.289 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:07.390 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:07.424 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:07.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:07.425 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:07.436 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:07 smithi104 ceph-mon[33507]: pgmap v335: 148 pgs: 148 active+clean; 3.1 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-11T07:34:07.436 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:07 smithi104 ceph-mon[33507]: 5.b scrub starts 2024-09-11T07:34:07.436 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:07 smithi104 ceph-mon[33507]: 5.b scrub ok 2024-09-11T07:34:07.436 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:07 smithi104 ceph-mon[33507]: 6.1d scrub starts 2024-09-11T07:34:07.436 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:07 smithi104 ceph-mon[33507]: 6.1d scrub ok 2024-09-11T07:34:07.436 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:07 smithi104 ceph-mon[33507]: 4.1a scrub starts 2024-09-11T07:34:07.436 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:07 smithi104 ceph-mon[33507]: 4.1a scrub ok 2024-09-11T07:34:07.436 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:07 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/308839046' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T07:34:07.437 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:07 smithi104 ceph-mon[33507]: from='client.25085 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T07:34:07.452 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:07 smithi022 ceph-mon[29462]: pgmap v335: 148 pgs: 148 active+clean; 3.1 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-11T07:34:07.452 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:07 smithi022 ceph-mon[29462]: 5.b scrub starts 2024-09-11T07:34:07.452 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:07 smithi022 ceph-mon[29462]: 5.b scrub ok 2024-09-11T07:34:07.452 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:07 smithi022 ceph-mon[29462]: 6.1d scrub starts 2024-09-11T07:34:07.453 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:07 smithi022 ceph-mon[29462]: 6.1d scrub ok 2024-09-11T07:34:07.453 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:07 smithi022 ceph-mon[29462]: 4.1a scrub starts 2024-09-11T07:34:07.453 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:07 smithi022 ceph-mon[29462]: 4.1a scrub ok 2024-09-11T07:34:07.453 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:07 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/308839046' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T07:34:07.453 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:07 smithi022 ceph-mon[29462]: from='client.25085 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T07:34:07.453 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:07 smithi022 ceph-mon[37330]: pgmap v335: 148 pgs: 148 active+clean; 3.1 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-11T07:34:07.453 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:07 smithi022 ceph-mon[37330]: 5.b scrub starts 2024-09-11T07:34:07.453 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:07 smithi022 ceph-mon[37330]: 5.b scrub ok 2024-09-11T07:34:07.453 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:07 smithi022 ceph-mon[37330]: 6.1d scrub starts 2024-09-11T07:34:07.453 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:07 smithi022 ceph-mon[37330]: 6.1d scrub ok 2024-09-11T07:34:07.453 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:07 smithi022 ceph-mon[37330]: 4.1a scrub starts 2024-09-11T07:34:07.454 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:07 smithi022 ceph-mon[37330]: 4.1a scrub ok 2024-09-11T07:34:07.454 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:07 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/308839046' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T07:34:07.454 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:07 smithi022 ceph-mon[37330]: from='client.25085 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T07:34:07.526 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:07.545 INFO:teuthology.orchestra.run.smithi022.stderr:reweighted osd.7 to 1 (10000) 2024-09-11T07:34:07.586 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:07.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:07.588 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:07.639 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:34:07.639 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:34:07.672 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:34:07.672 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:34:07.689 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:07.702 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:34:07.703 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:34:07.731 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:07.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:07.732 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:07.739 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:34:07.739 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:34:07.769 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:34:07.769 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:34:07.802 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:34:07.803 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:34:07.833 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:07.835 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:34:07.835 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:34:07.864 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:07.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:07.865 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:07.866 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:34:07.866 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:34:07.966 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:08.000 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:08.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:08.001 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:08.102 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:08.122 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:34:08.122 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2024-09-11T07:34:08.123 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2024-09-11T07:34:08.123 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2024-09-11T07:34:08.123 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.123 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:109196 7.1 7:8b29754b:::benchmark_data_smithi022_77720_object109195:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-11T07:34:08.123 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:30:17.188347+0000", 2024-09-11T07:34:08.123 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 230.69004633700001, 2024-09-11T07:34:08.123 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.070326448999999999, 2024-09-11T07:34:08.123 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:08.123 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:08.123 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:08.123 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:34:08.123 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:34:08.123 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 109196 2024-09-11T07:34:08.124 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.124 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:08.124 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.124 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:08.124 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.188347+0000", 2024-09-11T07:34:08.124 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.124 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.124 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.124 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:08.124 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.188347+0000", 2024-09-11T07:34:08.124 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.124 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.124 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.124 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:08.125 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.188344+0000", 2024-09-11T07:34:08.125 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:34:08.125 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.125 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.125 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:08.125 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.188388+0000", 2024-09-11T07:34:08.125 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3974000000000003e-05 2024-09-11T07:34:08.125 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.125 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.125 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:08.125 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.188402+0000", 2024-09-11T07:34:08.125 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.399e-05 2024-09-11T07:34:08.125 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.125 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.126 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:08.126 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.188409+0000", 2024-09-11T07:34:08.126 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1990000000000003e-06 2024-09-11T07:34:08.126 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.126 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.126 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:08.126 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.188452+0000", 2024-09-11T07:34:08.126 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3136999999999997e-05 2024-09-11T07:34:08.126 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.126 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.126 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:08.126 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.188536+0000", 2024-09-11T07:34:08.126 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4202000000000005e-05 2024-09-11T07:34:08.126 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.126 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.127 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,6", 2024-09-11T07:34:08.127 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.188626+0000", 2024-09-11T07:34:08.127 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9784000000000006e-05 2024-09-11T07:34:08.127 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.127 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.127 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:08.127 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.191284+0000", 2024-09-11T07:34:08.128 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0026584149999999999 2024-09-11T07:34:08.128 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.128 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.128 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.128 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.191639+0000", 2024-09-11T07:34:08.128 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00035494699999999999 2024-09-11T07:34:08.128 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.128 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.128 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.128 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.258597+0000", 2024-09-11T07:34:08.128 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.066957711000000003 2024-09-11T07:34:08.128 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.128 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.128 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:08.129 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.258641+0000", 2024-09-11T07:34:08.129 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3970000000000001e-05 2024-09-11T07:34:08.129 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.129 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.129 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:08.129 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.258674+0000", 2024-09-11T07:34:08.129 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2740999999999997e-05 2024-09-11T07:34:08.129 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.129 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:08.129 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.129 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.129 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.129 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:109201 7.1 7:8918cdc0:::benchmark_data_smithi022_77720_object109200:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-11T07:34:08.129 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:30:17.195316+0000", 2024-09-11T07:34:08.130 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 230.68307744500001, 2024-09-11T07:34:08.130 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.063454341999999997, 2024-09-11T07:34:08.130 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:08.130 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:08.130 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:08.130 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:34:08.130 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:34:08.130 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 109201 2024-09-11T07:34:08.130 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.130 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:08.130 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.130 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:08.130 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.195316+0000", 2024-09-11T07:34:08.130 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.130 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.131 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.131 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:08.131 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.195316+0000", 2024-09-11T07:34:08.131 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.131 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.131 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.131 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:08.131 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.195314+0000", 2024-09-11T07:34:08.132 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:34:08.132 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.132 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.132 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:08.132 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.195359+0000", 2024-09-11T07:34:08.132 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5176999999999998e-05 2024-09-11T07:34:08.132 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.132 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.132 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:08.132 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.195369+0000", 2024-09-11T07:34:08.132 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0532e-05 2024-09-11T07:34:08.133 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.133 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.133 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:08.133 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.195374+0000", 2024-09-11T07:34:08.133 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3630000000000004e-06 2024-09-11T07:34:08.133 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.133 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.133 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:08.133 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.195411+0000", 2024-09-11T07:34:08.133 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6835999999999998e-05 2024-09-11T07:34:08.133 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.133 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.134 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:08.134 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.195500+0000", 2024-09-11T07:34:08.134 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9727000000000003e-05 2024-09-11T07:34:08.134 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.134 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.134 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,6", 2024-09-11T07:34:08.134 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.195616+0000", 2024-09-11T07:34:08.134 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000115692 2024-09-11T07:34:08.134 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.134 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.134 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:08.135 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.198515+0000", 2024-09-11T07:34:08.135 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0028987679999999999 2024-09-11T07:34:08.135 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.135 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.135 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.135 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.198738+0000", 2024-09-11T07:34:08.135 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00022306800000000001 2024-09-11T07:34:08.135 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.136 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.136 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.136 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.258705+0000", 2024-09-11T07:34:08.136 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.059967330999999999 2024-09-11T07:34:08.136 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.136 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.136 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:08.136 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.258746+0000", 2024-09-11T07:34:08.136 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0339999999999997e-05 2024-09-11T07:34:08.136 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.136 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.136 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:08.136 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:17.258770+0000", 2024-09-11T07:34:08.136 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.4757999999999999e-05 2024-09-11T07:34:08.137 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.137 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:08.137 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.137 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.137 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.137 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:119875 7.1 7:83da405e:::benchmark_data_smithi022_77720_object119874:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-11T07:34:08.137 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:30:28.063064+0000", 2024-09-11T07:34:08.137 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 219.815329609, 2024-09-11T07:34:08.137 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.11794980100000001, 2024-09-11T07:34:08.137 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:08.137 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:08.137 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:08.137 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:34:08.138 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:34:08.138 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 119875 2024-09-11T07:34:08.138 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.138 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:08.138 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.138 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:08.138 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063064+0000", 2024-09-11T07:34:08.138 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.138 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.138 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.138 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:08.138 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063064+0000", 2024-09-11T07:34:08.138 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.138 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.139 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.139 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:08.139 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063062+0000", 2024-09-11T07:34:08.139 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:34:08.139 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.139 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.139 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:08.139 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063106+0000", 2024-09-11T07:34:08.139 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3788000000000002e-05 2024-09-11T07:34:08.139 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.139 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.139 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:08.139 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063122+0000", 2024-09-11T07:34:08.139 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6390999999999999e-05 2024-09-11T07:34:08.139 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.140 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.140 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:08.140 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063127+0000", 2024-09-11T07:34:08.140 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.374e-06 2024-09-11T07:34:08.140 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.140 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.140 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:08.140 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063153+0000", 2024-09-11T07:34:08.140 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.6754000000000002e-05 2024-09-11T07:34:08.140 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.141 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.141 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:08.141 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063263+0000", 2024-09-11T07:34:08.141 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000109144 2024-09-11T07:34:08.141 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.141 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.141 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,6", 2024-09-11T07:34:08.141 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063372+0000", 2024-09-11T07:34:08.141 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000109143 2024-09-11T07:34:08.141 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.142 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.142 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:08.142 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.066332+0000", 2024-09-11T07:34:08.142 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0029602769999999999 2024-09-11T07:34:08.142 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.142 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.142 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.142 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.066502+0000", 2024-09-11T07:34:08.142 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016962600000000001 2024-09-11T07:34:08.142 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.142 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.142 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.142 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.180920+0000", 2024-09-11T07:34:08.143 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.114418546 2024-09-11T07:34:08.143 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.143 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.143 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:08.143 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.180973+0000", 2024-09-11T07:34:08.143 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2454000000000003e-05 2024-09-11T07:34:08.143 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.143 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.143 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:08.143 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.181014+0000", 2024-09-11T07:34:08.143 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1041999999999998e-05 2024-09-11T07:34:08.143 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.143 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:08.143 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.144 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.144 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.144 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:120661 7.1 7:8ef41966:::benchmark_data_smithi022_77720_object120660:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:34:08.144 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:30:29.046728+0000", 2024-09-11T07:34:08.144 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 218.83166541700001, 2024-09-11T07:34:08.144 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.20704042, 2024-09-11T07:34:08.144 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:08.144 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:08.144 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:08.144 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:34:08.144 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:34:08.144 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 120661 2024-09-11T07:34:08.144 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.144 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:08.145 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.145 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:08.145 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.046728+0000", 2024-09-11T07:34:08.145 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.145 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.145 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.145 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:08.145 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.046728+0000", 2024-09-11T07:34:08.145 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.145 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.145 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.145 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:08.145 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.046727+0000", 2024-09-11T07:34:08.145 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:34:08.146 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.146 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.146 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:08.146 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.046819+0000", 2024-09-11T07:34:08.146 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2175000000000004e-05 2024-09-11T07:34:08.146 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.146 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.146 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:08.146 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.046839+0000", 2024-09-11T07:34:08.146 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9698999999999999e-05 2024-09-11T07:34:08.146 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.146 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.146 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:08.147 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.046846+0000", 2024-09-11T07:34:08.147 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9489999999999999e-06 2024-09-11T07:34:08.147 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.147 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.147 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:08.147 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.050121+0000", 2024-09-11T07:34:08.147 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0032747359999999999 2024-09-11T07:34:08.147 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.147 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.147 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:08.147 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.050236+0000", 2024-09-11T07:34:08.147 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011511899999999999 2024-09-11T07:34:08.147 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.147 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.148 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,6", 2024-09-11T07:34:08.148 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.050381+0000", 2024-09-11T07:34:08.148 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014511999999999999 2024-09-11T07:34:08.148 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.148 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.148 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:08.148 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.052599+0000", 2024-09-11T07:34:08.148 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0022178979999999998 2024-09-11T07:34:08.148 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.148 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.148 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.148 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.055720+0000", 2024-09-11T07:34:08.148 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0031211479999999998 2024-09-11T07:34:08.149 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.149 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.149 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.149 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.253668+0000", 2024-09-11T07:34:08.149 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.19794822000000001 2024-09-11T07:34:08.149 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.149 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.149 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:08.149 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.253726+0000", 2024-09-11T07:34:08.149 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7473999999999999e-05 2024-09-11T07:34:08.149 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.149 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.149 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:08.149 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.253768+0000", 2024-09-11T07:34:08.150 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2888e-05 2024-09-11T07:34:08.150 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.150 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:08.150 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.150 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.150 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.150 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:120666 7.1 7:89a791ca:::benchmark_data_smithi022_77720_object120665:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:34:08.150 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:30:29.052912+0000", 2024-09-11T07:34:08.150 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 218.82548119800001, 2024-09-11T07:34:08.150 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.204864612, 2024-09-11T07:34:08.150 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:08.150 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:08.150 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:08.150 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:34:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:34:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 120666 2024-09-11T07:34:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.052912+0000", 2024-09-11T07:34:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.052912+0000", 2024-09-11T07:34:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.152 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.152 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:08.152 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.052911+0000", 2024-09-11T07:34:08.152 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:34:08.152 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.152 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.152 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:08.152 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.052968+0000", 2024-09-11T07:34:08.152 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7132000000000001e-05 2024-09-11T07:34:08.152 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.052982+0000", 2024-09-11T07:34:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3913999999999999e-05 2024-09-11T07:34:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.052987+0000", 2024-09-11T07:34:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6849999999999998e-06 2024-09-11T07:34:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.053032+0000", 2024-09-11T07:34:08.154 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4495e-05 2024-09-11T07:34:08.154 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.154 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.154 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:08.154 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.053152+0000", 2024-09-11T07:34:08.154 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000120281 2024-09-11T07:34:08.154 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.154 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.154 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,6", 2024-09-11T07:34:08.154 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.053286+0000", 2024-09-11T07:34:08.154 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000133325 2024-09-11T07:34:08.154 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.055758+0000", 2024-09-11T07:34:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0024721669999999999 2024-09-11T07:34:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.055855+0000", 2024-09-11T07:34:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7077999999999994e-05 2024-09-11T07:34:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.257692+0000", 2024-09-11T07:34:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.201837129 2024-09-11T07:34:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.257739+0000", 2024-09-11T07:34:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7064999999999997e-05 2024-09-11T07:34:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.257777+0000", 2024-09-11T07:34:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7870000000000002e-05 2024-09-11T07:34:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:173285 7.1 7:8b9d3b80:::benchmark_data_smithi022_77720_object49530:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-11T07:34:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:31:20.290182+0000", 2024-09-11T07:34:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 167.58821126999999, 2024-09-11T07:34:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.109179046, 2024-09-11T07:34:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:34:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:34:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 173285 2024-09-11T07:34:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.290182+0000", 2024-09-11T07:34:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.290182+0000", 2024-09-11T07:34:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.290180+0000", 2024-09-11T07:34:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:34:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.290188+0000", 2024-09-11T07:34:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9359999999999999e-06 2024-09-11T07:34:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.290201+0000", 2024-09-11T07:34:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2931999999999999e-05 2024-09-11T07:34:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.290207+0000", 2024-09-11T07:34:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5599999999999999e-06 2024-09-11T07:34:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.290794+0000", 2024-09-11T07:34:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00058673600000000003 2024-09-11T07:34:08.160 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.160 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.160 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:08.160 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.290920+0000", 2024-09-11T07:34:08.160 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000126258 2024-09-11T07:34:08.160 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.160 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.160 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,6", 2024-09-11T07:34:08.160 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.291034+0000", 2024-09-11T07:34:08.160 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000114094 2024-09-11T07:34:08.160 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.160 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.294302+0000", 2024-09-11T07:34:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003267407 2024-09-11T07:34:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.295344+0000", 2024-09-11T07:34:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0010417029999999999 2024-09-11T07:34:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.399274+0000", 2024-09-11T07:34:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.103930915 2024-09-11T07:34:08.162 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.162 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.162 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:08.162 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.399324+0000", 2024-09-11T07:34:08.162 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9499999999999997e-05 2024-09-11T07:34:08.162 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.162 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.162 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:08.162 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.399361+0000", 2024-09-11T07:34:08.162 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7253999999999999e-05 2024-09-11T07:34:08.162 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.162 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:08.162 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.162 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.163 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.163 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:173289 7.1 7:8aae1198:::benchmark_data_smithi022_77720_object49534:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-11T07:34:08.163 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:31:20.296860+0000", 2024-09-11T07:34:08.163 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 167.581533331, 2024-09-11T07:34:08.163 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.102627207, 2024-09-11T07:34:08.163 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:08.163 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:08.163 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:08.163 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:34:08.163 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:34:08.163 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 173289 2024-09-11T07:34:08.163 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.163 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:08.163 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.163 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:08.164 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.296860+0000", 2024-09-11T07:34:08.164 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.164 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.164 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.164 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:08.164 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.296860+0000", 2024-09-11T07:34:08.164 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.164 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.164 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.164 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:08.164 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.296858+0000", 2024-09-11T07:34:08.164 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:34:08.164 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.165 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.165 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:08.165 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.296866+0000", 2024-09-11T07:34:08.165 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6910000000000003e-06 2024-09-11T07:34:08.165 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.165 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.165 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:08.165 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.296884+0000", 2024-09-11T07:34:08.165 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7876e-05 2024-09-11T07:34:08.165 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.165 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.165 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:08.165 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.296891+0000", 2024-09-11T07:34:08.166 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.8809999999999998e-06 2024-09-11T07:34:08.166 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.166 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.166 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:08.166 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.296953+0000", 2024-09-11T07:34:08.166 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2155000000000004e-05 2024-09-11T07:34:08.166 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.166 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.166 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:08.166 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.297087+0000", 2024-09-11T07:34:08.166 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000134024 2024-09-11T07:34:08.166 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.167 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.167 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,6", 2024-09-11T07:34:08.167 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.297224+0000", 2024-09-11T07:34:08.167 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013682899999999999 2024-09-11T07:34:08.167 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.167 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.167 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:08.167 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.300299+0000", 2024-09-11T07:34:08.167 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003074509 2024-09-11T07:34:08.167 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.167 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.168 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.168 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.302108+0000", 2024-09-11T07:34:08.168 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001809525 2024-09-11T07:34:08.168 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.168 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.168 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.168 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.399409+0000", 2024-09-11T07:34:08.168 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.097300541000000004 2024-09-11T07:34:08.168 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.168 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.168 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:08.169 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.399463+0000", 2024-09-11T07:34:08.169 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4545e-05 2024-09-11T07:34:08.169 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.169 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.169 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:08.169 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.399487+0000", 2024-09-11T07:34:08.169 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.4159e-05 2024-09-11T07:34:08.169 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.169 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:08.169 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.169 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.169 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.170 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:173300 7.1 7:83f5f869:::benchmark_data_smithi022_77720_object49545:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139)", 2024-09-11T07:34:08.170 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:31:20.310142+0000", 2024-09-11T07:34:08.170 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 167.56825172999999, 2024-09-11T07:34:08.170 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.095482628999999999, 2024-09-11T07:34:08.170 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:08.170 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:08.170 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:08.170 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:34:08.170 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:34:08.170 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 173300 2024-09-11T07:34:08.170 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.170 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:08.170 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.171 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:08.171 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.310142+0000", 2024-09-11T07:34:08.171 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.171 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.171 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.171 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:08.171 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.310142+0000", 2024-09-11T07:34:08.171 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.171 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.171 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.171 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:08.171 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.310140+0000", 2024-09-11T07:34:08.171 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:34:08.171 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.172 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.172 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:08.172 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.310147+0000", 2024-09-11T07:34:08.172 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1940000000000001e-06 2024-09-11T07:34:08.172 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.172 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.172 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:08.172 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.310159+0000", 2024-09-11T07:34:08.172 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1705000000000001e-05 2024-09-11T07:34:08.172 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.172 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.172 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:08.172 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.310164+0000", 2024-09-11T07:34:08.173 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9420000000000003e-06 2024-09-11T07:34:08.173 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.173 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.173 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:08.173 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.310207+0000", 2024-09-11T07:34:08.173 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3059000000000002e-05 2024-09-11T07:34:08.173 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.173 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.173 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:08.173 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.310341+0000", 2024-09-11T07:34:08.173 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013442200000000001 2024-09-11T07:34:08.173 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.173 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.173 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,6", 2024-09-11T07:34:08.174 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.310462+0000", 2024-09-11T07:34:08.174 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012061900000000001 2024-09-11T07:34:08.174 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.174 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.174 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:08.174 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.312563+0000", 2024-09-11T07:34:08.174 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002101665 2024-09-11T07:34:08.174 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.174 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.174 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.174 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.312801+0000", 2024-09-11T07:34:08.174 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00023783800000000001 2024-09-11T07:34:08.174 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.174 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.175 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.175 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.405565+0000", 2024-09-11T07:34:08.175 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.092763797999999995 2024-09-11T07:34:08.175 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.175 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.175 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:08.175 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.405598+0000", 2024-09-11T07:34:08.175 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3435000000000001e-05 2024-09-11T07:34:08.175 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.175 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.175 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:08.175 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.405624+0000", 2024-09-11T07:34:08.175 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.5913000000000001e-05 2024-09-11T07:34:08.175 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.175 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:08.176 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.176 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.176 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.176 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:189701 7.1 7:82b999c7:::benchmark_data_smithi022_77720_object65946:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-11T07:34:08.176 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:31:36.247020+0000", 2024-09-11T07:34:08.176 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 151.631373188, 2024-09-11T07:34:08.176 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.076306626000000002, 2024-09-11T07:34:08.176 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:08.176 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:08.176 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:08.176 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:34:08.176 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:34:08.176 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 189701 2024-09-11T07:34:08.176 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.177 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:08.177 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.177 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:08.177 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247020+0000", 2024-09-11T07:34:08.177 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.177 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.177 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.177 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:08.177 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247020+0000", 2024-09-11T07:34:08.177 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.177 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.177 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.178 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:08.178 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247019+0000", 2024-09-11T07:34:08.178 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:34:08.178 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.178 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.178 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:08.179 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247025+0000", 2024-09-11T07:34:08.179 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2179999999999996e-06 2024-09-11T07:34:08.179 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.179 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.179 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:08.179 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247036+0000", 2024-09-11T07:34:08.179 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0738e-05 2024-09-11T07:34:08.180 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.180 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.180 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:08.180 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247041+0000", 2024-09-11T07:34:08.180 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7579999999999998e-06 2024-09-11T07:34:08.180 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.180 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.181 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:08.181 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247088+0000", 2024-09-11T07:34:08.181 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6962999999999997e-05 2024-09-11T07:34:08.181 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.181 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.181 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:08.181 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247196+0000", 2024-09-11T07:34:08.182 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000108062 2024-09-11T07:34:08.182 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.182 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.182 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,6", 2024-09-11T07:34:08.182 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247313+0000", 2024-09-11T07:34:08.182 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000116697 2024-09-11T07:34:08.182 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.182 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.183 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:08.183 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.249564+0000", 2024-09-11T07:34:08.183 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0022514000000000002 2024-09-11T07:34:08.183 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.183 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.183 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.183 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.249890+0000", 2024-09-11T07:34:08.184 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00032583999999999998 2024-09-11T07:34:08.184 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.184 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.184 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.184 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.323240+0000", 2024-09-11T07:34:08.184 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.073349621000000004 2024-09-11T07:34:08.185 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.185 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.185 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:08.185 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.323288+0000", 2024-09-11T07:34:08.185 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7691000000000001e-05 2024-09-11T07:34:08.185 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.185 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.185 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:08.185 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.323327+0000", 2024-09-11T07:34:08.186 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9317999999999999e-05 2024-09-11T07:34:08.186 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.186 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:08.186 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.186 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.186 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.186 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:196178 7.1 7:8569dcba:::benchmark_data_smithi022_77720_object72423:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e143)", 2024-09-11T07:34:08.186 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:31:42.917360+0000", 2024-09-11T07:34:08.186 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 144.96103339699999, 2024-09-11T07:34:08.186 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.10325635499999999, 2024-09-11T07:34:08.186 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:08.186 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:08.186 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:08.186 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:34:08.187 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:34:08.187 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196178 2024-09-11T07:34:08.187 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.187 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:08.187 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.187 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:08.187 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:42.917360+0000", 2024-09-11T07:34:08.187 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.187 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.187 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.187 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:08.187 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:42.917360+0000", 2024-09-11T07:34:08.187 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.187 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.187 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.187 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:08.188 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:42.917358+0000", 2024-09-11T07:34:08.188 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:34:08.188 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.188 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.188 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:08.188 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:42.917366+0000", 2024-09-11T07:34:08.188 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0299999999999994e-06 2024-09-11T07:34:08.188 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.188 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.188 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:08.188 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:42.917379+0000", 2024-09-11T07:34:08.188 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2947e-05 2024-09-11T07:34:08.188 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.188 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.188 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:08.189 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:42.917385+0000", 2024-09-11T07:34:08.189 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1090000000000003e-06 2024-09-11T07:34:08.189 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.189 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.189 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:08.189 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:42.917432+0000", 2024-09-11T07:34:08.189 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7354000000000001e-05 2024-09-11T07:34:08.189 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.189 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.189 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:08.189 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:42.917517+0000", 2024-09-11T07:34:08.190 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5148999999999994e-05 2024-09-11T07:34:08.190 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.190 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.190 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,6", 2024-09-11T07:34:08.190 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:42.917585+0000", 2024-09-11T07:34:08.190 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7560999999999999e-05 2024-09-11T07:34:08.190 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.190 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.190 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.190 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:42.920183+0000", 2024-09-11T07:34:08.190 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0025980589999999998 2024-09-11T07:34:08.190 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.191 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.191 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:08.191 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:42.920235+0000", 2024-09-11T07:34:08.191 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2064000000000002e-05 2024-09-11T07:34:08.191 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.191 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.192 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.192 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:43.020557+0000", 2024-09-11T07:34:08.192 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.100322171 2024-09-11T07:34:08.192 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.192 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.192 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:08.192 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:43.020593+0000", 2024-09-11T07:34:08.192 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5382000000000001e-05 2024-09-11T07:34:08.193 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.193 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.193 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:08.193 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:43.020616+0000", 2024-09-11T07:34:08.193 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3777999999999998e-05 2024-09-11T07:34:08.193 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.193 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:08.193 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.193 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.194 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.194 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:217956 7.1 7:82fcf72b:::benchmark_data_smithi022_77720_object94201:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-11T07:34:08.194 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:32:07.170042+0000", 2024-09-11T07:34:08.194 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 120.70835128100001, 2024-09-11T07:34:08.194 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.116697231, 2024-09-11T07:34:08.194 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:08.194 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:08.195 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:08.195 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:34:08.195 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:34:08.195 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 217956 2024-09-11T07:34:08.195 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.195 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:08.195 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.196 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:08.196 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:07.170042+0000", 2024-09-11T07:34:08.196 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.196 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.196 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.196 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:08.196 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:07.170042+0000", 2024-09-11T07:34:08.196 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.196 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.196 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.196 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:08.196 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:07.170039+0000", 2024-09-11T07:34:08.196 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999967 2024-09-11T07:34:08.197 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.197 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.197 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:08.197 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:07.170050+0000", 2024-09-11T07:34:08.197 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1063000000000001e-05 2024-09-11T07:34:08.197 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.197 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.197 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:08.197 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:07.170070+0000", 2024-09-11T07:34:08.197 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9533000000000001e-05 2024-09-11T07:34:08.197 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.197 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.197 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:08.198 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:07.170077+0000", 2024-09-11T07:34:08.198 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5680000000000002e-06 2024-09-11T07:34:08.198 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.198 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.198 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:08.198 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:07.170123+0000", 2024-09-11T07:34:08.198 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5630999999999997e-05 2024-09-11T07:34:08.198 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.198 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.198 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:08.198 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:07.170245+0000", 2024-09-11T07:34:08.199 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000122237 2024-09-11T07:34:08.199 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.199 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.199 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,6", 2024-09-11T07:34:08.199 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:07.170362+0000", 2024-09-11T07:34:08.199 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011729400000000001 2024-09-11T07:34:08.199 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.199 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.199 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:08.199 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:07.172903+0000", 2024-09-11T07:34:08.199 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0025406119999999998 2024-09-11T07:34:08.199 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.199 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.199 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.200 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:07.173333+0000", 2024-09-11T07:34:08.200 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00043041899999999997 2024-09-11T07:34:08.200 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.200 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.200 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.200 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:07.286643+0000", 2024-09-11T07:34:08.200 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.113309284 2024-09-11T07:34:08.200 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.200 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.200 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:08.200 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:07.286693+0000", 2024-09-11T07:34:08.200 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0367000000000001e-05 2024-09-11T07:34:08.201 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.201 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.201 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:08.201 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:07.286739+0000", 2024-09-11T07:34:08.201 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6347000000000002e-05 2024-09-11T07:34:08.201 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.201 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:08.201 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.201 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.201 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.201 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:13660 8.2 8:483255c5:::benchmark_data_smithi022_108406_object13659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-11T07:34:08.202 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:33:44.760973+0000", 2024-09-11T07:34:08.202 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 23.117420283000001, 2024-09-11T07:34:08.202 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.105657748, 2024-09-11T07:34:08.202 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:08.202 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:08.202 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:08.202 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:34:08.202 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:34:08.202 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 13660 2024-09-11T07:34:08.202 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.202 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:08.202 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.203 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:08.203 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.760973+0000", 2024-09-11T07:34:08.203 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.203 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.203 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.203 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:08.203 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.760973+0000", 2024-09-11T07:34:08.203 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.203 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.203 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.203 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:08.203 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.760971+0000", 2024-09-11T07:34:08.203 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:34:08.204 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.204 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.204 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:08.204 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.761008+0000", 2024-09-11T07:34:08.204 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7211e-05 2024-09-11T07:34:08.204 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.204 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.204 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:08.204 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.761023+0000", 2024-09-11T07:34:08.204 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4397e-05 2024-09-11T07:34:08.204 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.204 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.205 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:08.205 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.761029+0000", 2024-09-11T07:34:08.205 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2929999999999997e-06 2024-09-11T07:34:08.205 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.205 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.205 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:08.205 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.761071+0000", 2024-09-11T07:34:08.205 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1517000000000002e-05 2024-09-11T07:34:08.205 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.205 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.205 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:08.205 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.761154+0000", 2024-09-11T07:34:08.206 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3677999999999994e-05 2024-09-11T07:34:08.206 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.206 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.206 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:34:08.206 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.761260+0000", 2024-09-11T07:34:08.206 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000105447 2024-09-11T07:34:08.206 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.206 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.206 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:08.206 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.761810+0000", 2024-09-11T07:34:08.206 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00055048800000000002 2024-09-11T07:34:08.206 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.207 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.207 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.207 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.762038+0000", 2024-09-11T07:34:08.207 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00022777100000000001 2024-09-11T07:34:08.207 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.207 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.207 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.207 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.866569+0000", 2024-09-11T07:34:08.207 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.104531086 2024-09-11T07:34:08.207 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.207 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.207 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:08.207 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.866605+0000", 2024-09-11T07:34:08.207 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6482000000000001e-05 2024-09-11T07:34:08.207 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.208 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.208 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:08.208 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.866631+0000", 2024-09-11T07:34:08.208 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.5418e-05 2024-09-11T07:34:08.208 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.208 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:08.208 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.208 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.208 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.208 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:28697 8.2 8:4b5bfabd:::benchmark_data_smithi022_108406_object28696:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e189)", 2024-09-11T07:34:08.208 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:33:51.017713+0000", 2024-09-11T07:34:08.208 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 16.860680820999999, 2024-09-11T07:34:08.209 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13638721500000001, 2024-09-11T07:34:08.209 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:08.209 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:08.209 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:08.209 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:34:08.209 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:34:08.209 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 28697 2024-09-11T07:34:08.209 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.209 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:08.209 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.209 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:08.209 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.017713+0000", 2024-09-11T07:34:08.209 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.210 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.210 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.210 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:08.210 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.017713+0000", 2024-09-11T07:34:08.210 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.210 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.210 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.210 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:08.210 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.017710+0000", 2024-09-11T07:34:08.210 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:34:08.210 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.210 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.210 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:08.211 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.017742+0000", 2024-09-11T07:34:08.211 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2410999999999999e-05 2024-09-11T07:34:08.211 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.211 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.211 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:08.211 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.017752+0000", 2024-09-11T07:34:08.211 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2960000000000004e-06 2024-09-11T07:34:08.211 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.211 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.211 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:08.211 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.017757+0000", 2024-09-11T07:34:08.211 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.023e-06 2024-09-11T07:34:08.212 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.212 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.212 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:08.212 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.017821+0000", 2024-09-11T07:34:08.212 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4518999999999997e-05 2024-09-11T07:34:08.212 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.212 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.212 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:08.212 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.017946+0000", 2024-09-11T07:34:08.212 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000125127 2024-09-11T07:34:08.212 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.212 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.212 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:34:08.213 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.018081+0000", 2024-09-11T07:34:08.213 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013491999999999999 2024-09-11T07:34:08.213 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.213 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.213 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:08.213 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.020234+0000", 2024-09-11T07:34:08.213 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0021525670000000002 2024-09-11T07:34:08.213 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.213 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.213 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.213 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.020972+0000", 2024-09-11T07:34:08.213 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00073804999999999997 2024-09-11T07:34:08.213 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.214 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.214 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.214 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.154017+0000", 2024-09-11T07:34:08.214 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13304534800000001 2024-09-11T07:34:08.214 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.214 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.214 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:08.214 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.154065+0000", 2024-09-11T07:34:08.214 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7428000000000001e-05 2024-09-11T07:34:08.214 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.214 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.214 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:08.215 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.154100+0000", 2024-09-11T07:34:08.215 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5089999999999998e-05 2024-09-11T07:34:08.215 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.215 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:08.215 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.215 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.215 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.215 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:28700 8.2 8:41d2ebaa:::benchmark_data_smithi022_108406_object28699:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e189)", 2024-09-11T07:34:08.215 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:33:51.018182+0000", 2024-09-11T07:34:08.215 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 16.860211093, 2024-09-11T07:34:08.215 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13626432999999999, 2024-09-11T07:34:08.215 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:08.216 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:08.216 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:08.216 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:34:08.216 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:34:08.216 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 28700 2024-09-11T07:34:08.216 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.216 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:08.216 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.216 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:08.216 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.018182+0000", 2024-09-11T07:34:08.216 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.216 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.217 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.217 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:08.217 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.018182+0000", 2024-09-11T07:34:08.217 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.217 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.217 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.217 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:08.217 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.018181+0000", 2024-09-11T07:34:08.217 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:34:08.217 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.217 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.217 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:08.218 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.018255+0000", 2024-09-11T07:34:08.218 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3522999999999994e-05 2024-09-11T07:34:08.218 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.218 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.218 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:08.218 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.018268+0000", 2024-09-11T07:34:08.218 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3397000000000001e-05 2024-09-11T07:34:08.218 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.218 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.218 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:08.218 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.018273+0000", 2024-09-11T07:34:08.218 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.49e-06 2024-09-11T07:34:08.218 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.218 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.218 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:08.219 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.019109+0000", 2024-09-11T07:34:08.219 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00083582700000000003 2024-09-11T07:34:08.219 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.219 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.219 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:08.219 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.019262+0000", 2024-09-11T07:34:08.219 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015234800000000001 2024-09-11T07:34:08.219 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.219 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.219 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:34:08.219 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.019390+0000", 2024-09-11T07:34:08.219 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000128076 2024-09-11T07:34:08.219 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.219 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.220 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:08.220 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.020376+0000", 2024-09-11T07:34:08.220 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00098610199999999999 2024-09-11T07:34:08.220 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.220 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.220 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.220 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.021100+0000", 2024-09-11T07:34:08.220 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00072417200000000003 2024-09-11T07:34:08.220 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.220 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.220 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.220 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.154294+0000", 2024-09-11T07:34:08.220 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13319441900000001 2024-09-11T07:34:08.221 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.221 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.221 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:08.221 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.154409+0000", 2024-09-11T07:34:08.221 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011513199999999999 2024-09-11T07:34:08.221 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.221 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.221 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:08.221 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.154447+0000", 2024-09-11T07:34:08.221 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7205e-05 2024-09-11T07:34:08.221 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.221 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:08.221 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.222 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.222 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.222 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:28704 8.2 8:4568d509:::benchmark_data_smithi022_108406_object28703:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e189)", 2024-09-11T07:34:08.222 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:33:51.021041+0000", 2024-09-11T07:34:08.222 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 16.857352468999999, 2024-09-11T07:34:08.222 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.133545738, 2024-09-11T07:34:08.222 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:08.222 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:08.222 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:08.222 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:34:08.222 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:34:08.222 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 28704 2024-09-11T07:34:08.222 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.222 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:08.223 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.223 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:08.223 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.021041+0000", 2024-09-11T07:34:08.223 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.223 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.223 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.223 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:08.223 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.021041+0000", 2024-09-11T07:34:08.223 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.223 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.223 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.224 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:08.224 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.021039+0000", 2024-09-11T07:34:08.224 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:34:08.224 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.224 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.224 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:08.224 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.021141+0000", 2024-09-11T07:34:08.224 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000102172 2024-09-11T07:34:08.224 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.224 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.225 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:08.225 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.021152+0000", 2024-09-11T07:34:08.225 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1375e-05 2024-09-11T07:34:08.225 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.225 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.225 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:08.225 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.021158+0000", 2024-09-11T07:34:08.226 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.587e-06 2024-09-11T07:34:08.226 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.226 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.226 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:08.226 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.021192+0000", 2024-09-11T07:34:08.226 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4468999999999998e-05 2024-09-11T07:34:08.226 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.226 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.227 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:08.227 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.021266+0000", 2024-09-11T07:34:08.227 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3460999999999993e-05 2024-09-11T07:34:08.227 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.227 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.227 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:34:08.227 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.021369+0000", 2024-09-11T07:34:08.227 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010312 2024-09-11T07:34:08.227 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.228 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.228 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:08.228 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.022203+0000", 2024-09-11T07:34:08.228 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00083434999999999998 2024-09-11T07:34:08.228 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.228 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.228 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.228 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.022628+0000", 2024-09-11T07:34:08.228 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000424864 2024-09-11T07:34:08.229 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.229 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.229 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.229 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.154477+0000", 2024-09-11T07:34:08.229 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13184918800000001 2024-09-11T07:34:08.229 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.229 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.229 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:08.230 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.154554+0000", 2024-09-11T07:34:08.230 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6530000000000001e-05 2024-09-11T07:34:08.230 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.230 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.230 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:08.230 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.154587+0000", 2024-09-11T07:34:08.230 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2919000000000001e-05 2024-09-11T07:34:08.230 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.230 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:08.231 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.231 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.231 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.231 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:31924 8.2 8:4967232f:::benchmark_data_smithi022_108406_object31923:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e190)", 2024-09-11T07:34:08.231 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:33:52.796366+0000", 2024-09-11T07:34:08.231 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 15.082027686, 2024-09-11T07:34:08.231 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.11878032199999999, 2024-09-11T07:34:08.231 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:08.231 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:08.232 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:08.232 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:34:08.232 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:34:08.232 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31924 2024-09-11T07:34:08.232 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.232 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:08.232 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.232 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:08.233 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.796366+0000", 2024-09-11T07:34:08.233 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.233 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.233 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.233 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:08.233 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.796366+0000", 2024-09-11T07:34:08.233 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.233 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.233 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.233 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:08.234 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.796364+0000", 2024-09-11T07:34:08.234 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:34:08.234 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.234 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.234 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:08.234 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.796395+0000", 2024-09-11T07:34:08.234 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.0997000000000002e-05 2024-09-11T07:34:08.234 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.234 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.234 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:08.235 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.796408+0000", 2024-09-11T07:34:08.235 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3334e-05 2024-09-11T07:34:08.235 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.235 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.235 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:08.235 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.796413+0000", 2024-09-11T07:34:08.236 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3770000000000004e-06 2024-09-11T07:34:08.236 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.236 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.236 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:08.236 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.796461+0000", 2024-09-11T07:34:08.236 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7775000000000003e-05 2024-09-11T07:34:08.236 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.236 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.236 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:08.236 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.796616+0000", 2024-09-11T07:34:08.236 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015441399999999999 2024-09-11T07:34:08.237 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.237 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.237 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:34:08.237 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.796760+0000", 2024-09-11T07:34:08.237 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014437 2024-09-11T07:34:08.237 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.237 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.237 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:08.237 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.798096+0000", 2024-09-11T07:34:08.237 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001336056 2024-09-11T07:34:08.237 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.237 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.237 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.238 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.799840+0000", 2024-09-11T07:34:08.238 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0017436870000000001 2024-09-11T07:34:08.238 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.238 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.238 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.238 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.914307+0000", 2024-09-11T07:34:08.238 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.11446765 2024-09-11T07:34:08.238 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.238 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.238 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:08.238 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.915106+0000", 2024-09-11T07:34:08.238 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00079854899999999996 2024-09-11T07:34:08.239 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.239 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.239 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:08.239 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.915146+0000", 2024-09-11T07:34:08.239 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0188999999999998e-05 2024-09-11T07:34:08.239 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.239 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:08.239 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.239 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.239 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.239 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:33552 8.2 8:4687e165:::benchmark_data_smithi022_108406_object33551:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e190)", 2024-09-11T07:34:08.239 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:33:53.715584+0000", 2024-09-11T07:34:08.239 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 14.162808962, 2024-09-11T07:34:08.239 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.115345479, 2024-09-11T07:34:08.240 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:08.240 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:08.240 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:08.240 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:34:08.240 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:34:08.240 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 33552 2024-09-11T07:34:08.240 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.240 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:08.240 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.240 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:08.240 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:53.715584+0000", 2024-09-11T07:34:08.241 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.241 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.241 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.241 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:08.241 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:53.715584+0000", 2024-09-11T07:34:08.241 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.241 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.241 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.241 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:08.241 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:53.715582+0000", 2024-09-11T07:34:08.241 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:34:08.242 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.242 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.242 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:08.242 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:53.715624+0000", 2024-09-11T07:34:08.242 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2088000000000001e-05 2024-09-11T07:34:08.242 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.242 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.242 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:08.242 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:53.715636+0000", 2024-09-11T07:34:08.242 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2033e-05 2024-09-11T07:34:08.242 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.243 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.243 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:08.243 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:53.715642+0000", 2024-09-11T07:34:08.243 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0589999999999999e-06 2024-09-11T07:34:08.243 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.243 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.243 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:08.243 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:53.715684+0000", 2024-09-11T07:34:08.243 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1424000000000002e-05 2024-09-11T07:34:08.243 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.243 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.243 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:08.244 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:53.715772+0000", 2024-09-11T07:34:08.244 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8673999999999998e-05 2024-09-11T07:34:08.244 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.244 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.244 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:34:08.244 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:53.715878+0000", 2024-09-11T07:34:08.244 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000106037 2024-09-11T07:34:08.244 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.244 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.244 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:08.244 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:53.717196+0000", 2024-09-11T07:34:08.244 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001317188 2024-09-11T07:34:08.244 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.244 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.245 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.245 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:53.717794+0000", 2024-09-11T07:34:08.245 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000598041 2024-09-11T07:34:08.245 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.245 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.245 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.245 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:53.830867+0000", 2024-09-11T07:34:08.245 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.113073228 2024-09-11T07:34:08.245 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.245 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.245 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:08.245 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:53.830906+0000", 2024-09-11T07:34:08.245 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8872000000000003e-05 2024-09-11T07:34:08.246 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.246 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.246 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:08.246 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:53.830930+0000", 2024-09-11T07:34:08.246 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.4281e-05 2024-09-11T07:34:08.246 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.246 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:08.246 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.246 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.246 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.246 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:34743 8.2 8:46952494:::benchmark_data_smithi022_108406_object34742:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e191)", 2024-09-11T07:34:08.246 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:33:54.427472+0000", 2024-09-11T07:34:08.246 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 13.450920972, 2024-09-11T07:34:08.246 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.10897390999999999, 2024-09-11T07:34:08.246 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:08.247 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:08.247 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:08.247 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:34:08.247 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:34:08.247 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 34743 2024-09-11T07:34:08.247 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.247 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:08.247 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.247 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:08.247 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.427472+0000", 2024-09-11T07:34:08.247 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.247 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.247 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.247 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:08.248 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.427472+0000", 2024-09-11T07:34:08.248 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.248 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.248 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.248 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:08.248 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.427470+0000", 2024-09-11T07:34:08.248 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:34:08.248 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.248 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.248 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:08.248 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.427510+0000", 2024-09-11T07:34:08.248 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0160000000000002e-05 2024-09-11T07:34:08.248 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.427525+0000", 2024-09-11T07:34:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4408e-05 2024-09-11T07:34:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.427531+0000", 2024-09-11T07:34:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6229999999999999e-06 2024-09-11T07:34:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.427591+0000", 2024-09-11T07:34:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9209999999999997e-05 2024-09-11T07:34:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.427700+0000", 2024-09-11T07:34:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000108834 2024-09-11T07:34:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:34:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.427788+0000", 2024-09-11T07:34:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8500999999999999e-05 2024-09-11T07:34:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.428903+0000", 2024-09-11T07:34:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0011154190000000001 2024-09-11T07:34:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.429130+0000", 2024-09-11T07:34:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000226911 2024-09-11T07:34:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.536374+0000", 2024-09-11T07:34:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.10724367999999999 2024-09-11T07:34:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.536409+0000", 2024-09-11T07:34:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4691999999999999e-05 2024-09-11T07:34:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.536446+0000", 2024-09-11T07:34:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7651999999999998e-05 2024-09-11T07:34:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:34745 8.2 8:4e4bb1b0:::benchmark_data_smithi022_108406_object34744:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e191)", 2024-09-11T07:34:08.253 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:33:54.429487+0000", 2024-09-11T07:34:08.253 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 13.448906329, 2024-09-11T07:34:08.253 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.10703784299999999, 2024-09-11T07:34:08.253 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:08.253 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:08.253 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:08.253 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:34:08.253 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:34:08.253 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 34745 2024-09-11T07:34:08.253 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.253 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.429487+0000", 2024-09-11T07:34:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.429487+0000", 2024-09-11T07:34:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.429486+0000", 2024-09-11T07:34:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:34:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.429548+0000", 2024-09-11T07:34:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2844999999999999e-05 2024-09-11T07:34:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.429558+0000", 2024-09-11T07:34:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4720000000000001e-06 2024-09-11T07:34:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.429563+0000", 2024-09-11T07:34:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2979999999999999e-06 2024-09-11T07:34:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.429596+0000", 2024-09-11T07:34:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2829e-05 2024-09-11T07:34:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.429686+0000", 2024-09-11T07:34:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0305999999999999e-05 2024-09-11T07:34:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:34:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.429769+0000", 2024-09-11T07:34:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2806000000000006e-05 2024-09-11T07:34:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.430682+0000", 2024-09-11T07:34:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000912513 2024-09-11T07:34:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.430991+0000", 2024-09-11T07:34:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000309766 2024-09-11T07:34:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.536471+0000", 2024-09-11T07:34:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.10547975900000001 2024-09-11T07:34:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.536496+0000", 2024-09-11T07:34:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.4604e-05 2024-09-11T07:34:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.536525+0000", 2024-09-11T07:34:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9147000000000001e-05 2024-09-11T07:34:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:08.259 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.259 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.259 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.259 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:35243 8.2 8:40abd99c:::benchmark_data_smithi022_108406_object35242:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e191)", 2024-09-11T07:34:08.259 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:33:54.761431+0000", 2024-09-11T07:34:08.259 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 13.116961999000001, 2024-09-11T07:34:08.259 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.060623949000000003, 2024-09-11T07:34:08.259 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:08.259 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:08.259 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:08.259 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:34:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:34:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 35243 2024-09-11T07:34:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.761431+0000", 2024-09-11T07:34:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.761431+0000", 2024-09-11T07:34:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.261 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.261 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:08.261 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.761429+0000", 2024-09-11T07:34:08.261 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:34:08.261 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.261 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.261 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:08.261 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.761477+0000", 2024-09-11T07:34:08.261 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7252000000000001e-05 2024-09-11T07:34:08.261 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.261 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.261 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:08.261 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.761487+0000", 2024-09-11T07:34:08.262 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0414999999999999e-05 2024-09-11T07:34:08.262 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.262 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.262 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:08.262 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.761492+0000", 2024-09-11T07:34:08.262 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5789999999999997e-06 2024-09-11T07:34:08.262 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.262 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.262 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:08.262 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.761529+0000", 2024-09-11T07:34:08.262 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7329000000000001e-05 2024-09-11T07:34:08.263 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.263 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.263 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:08.263 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.761622+0000", 2024-09-11T07:34:08.263 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.31e-05 2024-09-11T07:34:08.263 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.263 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.263 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:34:08.263 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.761717+0000", 2024-09-11T07:34:08.263 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4396999999999998e-05 2024-09-11T07:34:08.264 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.264 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.264 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:08.264 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.762751+0000", 2024-09-11T07:34:08.264 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001034012 2024-09-11T07:34:08.264 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.264 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.264 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.264 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.763021+0000", 2024-09-11T07:34:08.264 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00027032500000000002 2024-09-11T07:34:08.264 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.264 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.265 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:08.265 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.821955+0000", 2024-09-11T07:34:08.265 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.058933876000000003 2024-09-11T07:34:08.265 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.265 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.265 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:08.265 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.822011+0000", 2024-09-11T07:34:08.265 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6107999999999999e-05 2024-09-11T07:34:08.265 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:08.265 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:08.265 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:08.265 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:54.822055+0000", 2024-09-11T07:34:08.266 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4573000000000002e-05 2024-09-11T07:34:08.266 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.266 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:08.266 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.266 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:08.266 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:08.266 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:34:08.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:08.267 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:08.268 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:08.268 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 28 16 57658 57642 128.634 87.6875 0.00928011 0.00776865 2024-09-11T07:34:08.369 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:08.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:08.398 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:08.398 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:08.499 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:08.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:08.543 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:08.543 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:08.644 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:08.680 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:08.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:08.681 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:08.771 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:34:08.782 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:08.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:08.827 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:08.828 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:08.874 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:08 smithi022 ceph-mon[29462]: 2.1a scrub starts 2024-09-11T07:34:08.874 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:08 smithi022 ceph-mon[29462]: 2.1a scrub ok 2024-09-11T07:34:08.874 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:08 smithi022 ceph-mon[29462]: 2.1 deep-scrub starts 2024-09-11T07:34:08.874 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:08 smithi022 ceph-mon[29462]: 2.1 deep-scrub ok 2024-09-11T07:34:08.875 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:08 smithi022 ceph-mon[29462]: 5.1c scrub starts 2024-09-11T07:34:08.875 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:08 smithi022 ceph-mon[29462]: 5.1c scrub ok 2024-09-11T07:34:08.875 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:08 smithi022 ceph-mon[29462]: from='client.25085 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-11T07:34:08.875 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:08 smithi022 ceph-mon[29462]: osdmap e196: 8 total, 8 up, 8 in 2024-09-11T07:34:08.875 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:08 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2168257148' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:34:08.875 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:34:08.875 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:34:08.875 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:08 smithi022 ceph-mon[29462]: pgmap v337: 148 pgs: 148 active+clean; 3.1 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-11T07:34:08.875 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:08 smithi022 ceph-mon[37330]: 2.1a scrub starts 2024-09-11T07:34:08.875 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:08 smithi022 ceph-mon[37330]: 2.1a scrub ok 2024-09-11T07:34:08.875 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:08 smithi022 ceph-mon[37330]: 2.1 deep-scrub starts 2024-09-11T07:34:08.875 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:08 smithi022 ceph-mon[37330]: 2.1 deep-scrub ok 2024-09-11T07:34:08.876 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:08 smithi022 ceph-mon[37330]: 5.1c scrub starts 2024-09-11T07:34:08.876 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:08 smithi022 ceph-mon[37330]: 5.1c scrub ok 2024-09-11T07:34:08.876 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:08 smithi022 ceph-mon[37330]: from='client.25085 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-11T07:34:08.876 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:08 smithi022 ceph-mon[37330]: osdmap e196: 8 total, 8 up, 8 in 2024-09-11T07:34:08.876 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:08 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2168257148' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:34:08.876 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:34:08.876 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:34:08.876 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:08 smithi022 ceph-mon[37330]: pgmap v337: 148 pgs: 148 active+clean; 3.1 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-11T07:34:08.928 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:08.971 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:08.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:08.972 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:08 smithi104 ceph-mon[33507]: 2.1a scrub starts 2024-09-11T07:34:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:08 smithi104 ceph-mon[33507]: 2.1a scrub ok 2024-09-11T07:34:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:08 smithi104 ceph-mon[33507]: 2.1 deep-scrub starts 2024-09-11T07:34:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:08 smithi104 ceph-mon[33507]: 2.1 deep-scrub ok 2024-09-11T07:34:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:08 smithi104 ceph-mon[33507]: 5.1c scrub starts 2024-09-11T07:34:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:08 smithi104 ceph-mon[33507]: 5.1c scrub ok 2024-09-11T07:34:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:08 smithi104 ceph-mon[33507]: from='client.25085 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-11T07:34:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:08 smithi104 ceph-mon[33507]: osdmap e196: 8 total, 8 up, 8 in 2024-09-11T07:34:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:08 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2168257148' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:34:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:34:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:34:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:08 smithi104 ceph-mon[33507]: pgmap v337: 148 pgs: 148 active+clean; 3.1 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-11T07:34:09.073 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:09.117 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:09.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:09.119 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-11T07:34:09.120 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:09.206 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 29 16 59191 59175 127.502 95.8125 0.00962961 0.0078372 2024-09-11T07:34:09.221 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:09.238 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-11T07:34:09.238 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:34:09.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:09.247 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:09.248 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:09.348 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:09.433 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:09.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:09.434 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:09.471 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:34:09.535 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:09.586 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:09.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:09.587 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:09.631 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:34:09 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:34:09.631 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:09 smithi104 ceph-mon[33507]: 4.5 scrub starts 2024-09-11T07:34:09.632 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:09 smithi104 ceph-mon[33507]: 4.5 scrub ok 2024-09-11T07:34:09.632 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:09 smithi104 ceph-mon[33507]: 4.c scrub starts 2024-09-11T07:34:09.632 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:09 smithi104 ceph-mon[33507]: 4.c scrub ok 2024-09-11T07:34:09.632 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:09 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:09.688 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:09.736 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:09.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:09.737 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:09.838 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:09.869 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:09.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:09.870 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:09.886 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:09 smithi022 ceph-mon[29462]: 4.5 scrub starts 2024-09-11T07:34:09.886 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:09 smithi022 ceph-mon[29462]: 4.5 scrub ok 2024-09-11T07:34:09.886 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:09 smithi022 ceph-mon[29462]: 4.c scrub starts 2024-09-11T07:34:09.887 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:09 smithi022 ceph-mon[29462]: 4.c scrub ok 2024-09-11T07:34:09.887 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:09 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:09.887 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:09 smithi022 ceph-mon[37330]: 4.5 scrub starts 2024-09-11T07:34:09.887 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:09 smithi022 ceph-mon[37330]: 4.5 scrub ok 2024-09-11T07:34:09.887 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:09 smithi022 ceph-mon[37330]: 4.c scrub starts 2024-09-11T07:34:09.887 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:09 smithi022 ceph-mon[37330]: 4.c scrub ok 2024-09-11T07:34:09.887 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:09 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:09.971 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:10.018 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:10.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:10.019 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:10.120 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:10.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:10.160 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:10.160 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:10.206 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 30 16 60439 60423 125.852 78 0.0105819 0.00794054 2024-09-11T07:34:10.261 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:10.295 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:10.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:10.295 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:10.396 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:10.424 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:10.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:10.425 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:10.525 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:10.553 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:10.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:10.554 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:10.655 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:10.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:10 smithi022 ceph-mon[37330]: 2.1f scrub starts 2024-09-11T07:34:10.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:10 smithi022 ceph-mon[37330]: 2.1f scrub ok 2024-09-11T07:34:10.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:10 smithi022 ceph-mon[37330]: pgmap v338: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.5 KiB/s rd, 147 MiB/s wr, 2.35k op/s 2024-09-11T07:34:10.668 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:34:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:34:10] "GET /metrics HTTP/1.1" 200 224029 "" "Prometheus/2.43.0" 2024-09-11T07:34:10.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:10 smithi022 ceph-mon[29462]: 2.1f scrub starts 2024-09-11T07:34:10.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:10 smithi022 ceph-mon[29462]: 2.1f scrub ok 2024-09-11T07:34:10.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:10 smithi022 ceph-mon[29462]: pgmap v338: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.5 KiB/s rd, 147 MiB/s wr, 2.35k op/s 2024-09-11T07:34:10.671 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:34:10.687 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:10.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:10.688 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:10.735 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:34:10.788 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:10.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:10.840 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:10.840 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:10.941 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:10.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:10.965 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:10.966 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:11.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:10 smithi104 ceph-mon[33507]: 2.1f scrub starts 2024-09-11T07:34:11.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:10 smithi104 ceph-mon[33507]: 2.1f scrub ok 2024-09-11T07:34:11.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:10 smithi104 ceph-mon[33507]: pgmap v338: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.5 KiB/s rd, 147 MiB/s wr, 2.35k op/s 2024-09-11T07:34:11.066 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:11.115 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:11.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:11.117 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:11.206 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 31 15 61612 61597 124.158 73.375 0.00406804 0.00804875 2024-09-11T07:34:11.218 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:11.257 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:11.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:11.259 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:11.360 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:11.392 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:11.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:11.393 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:11.494 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:11.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:11.523 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:11.523 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:11.624 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:11.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:11.656 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:11.657 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:11.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:11 smithi022 ceph-mon[29462]: 2.14 deep-scrub starts 2024-09-11T07:34:11.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:11 smithi022 ceph-mon[29462]: 2.14 deep-scrub ok 2024-09-11T07:34:11.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:11 smithi022 ceph-mon[29462]: 6.1 scrub starts 2024-09-11T07:34:11.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:11 smithi022 ceph-mon[29462]: 6.1 scrub ok 2024-09-11T07:34:11.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:11 smithi022 ceph-mon[29462]: osdmap e197: 8 total, 8 up, 8 in 2024-09-11T07:34:11.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:11 smithi022 ceph-mon[37330]: 2.14 deep-scrub starts 2024-09-11T07:34:11.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:11 smithi022 ceph-mon[37330]: 2.14 deep-scrub ok 2024-09-11T07:34:11.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:11 smithi022 ceph-mon[37330]: 6.1 scrub starts 2024-09-11T07:34:11.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:11 smithi022 ceph-mon[37330]: 6.1 scrub ok 2024-09-11T07:34:11.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:11 smithi022 ceph-mon[37330]: osdmap e197: 8 total, 8 up, 8 in 2024-09-11T07:34:11.757 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:11.786 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:11.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:11.787 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:11.888 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:11.918 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:11.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:11.919 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:12.021 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:12.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:11 smithi104 ceph-mon[33507]: 2.14 deep-scrub starts 2024-09-11T07:34:12.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:11 smithi104 ceph-mon[33507]: 2.14 deep-scrub ok 2024-09-11T07:34:12.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:11 smithi104 ceph-mon[33507]: 6.1 scrub starts 2024-09-11T07:34:12.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:11 smithi104 ceph-mon[33507]: 6.1 scrub ok 2024-09-11T07:34:12.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:11 smithi104 ceph-mon[33507]: osdmap e197: 8 total, 8 up, 8 in 2024-09-11T07:34:12.054 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:12.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:12.055 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:12.156 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:12.193 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:12.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:12.194 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:12.206 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 32 16 62877 62861 122.747 79 0.00433893 0.0081422 2024-09-11T07:34:12.295 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:12.332 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:12.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:12.333 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:12.434 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:12.474 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:12.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:12.475 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:12.576 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:12.593 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is set 2024-09-11T07:34:12.621 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:12.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:12.622 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:12.722 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:12.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:12.744 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:12.744 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:12.760 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:12 smithi104 ceph-mon[33507]: 4.d scrub starts 2024-09-11T07:34:12.760 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:12 smithi104 ceph-mon[33507]: 4.d scrub ok 2024-09-11T07:34:12.760 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:12 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3774679601' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:34:12.760 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:12 smithi104 ceph-mon[33507]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:34:12.760 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:12 smithi104 ceph-mon[33507]: pgmap v340: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 779 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-11T07:34:12.845 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:12.869 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:34:12.869 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:34:12.869 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 0 2024-09-11T07:34:12.869 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:34:12.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:12.871 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:12.871 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:12.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:12 smithi022 ceph-mon[29462]: 4.d scrub starts 2024-09-11T07:34:12.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:12 smithi022 ceph-mon[29462]: 4.d scrub ok 2024-09-11T07:34:12.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:12 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3774679601' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:34:12.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:12 smithi022 ceph-mon[29462]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:34:12.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:12 smithi022 ceph-mon[29462]: pgmap v340: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 779 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-11T07:34:12.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:12 smithi022 ceph-mon[37330]: 4.d scrub starts 2024-09-11T07:34:12.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:12 smithi022 ceph-mon[37330]: 4.d scrub ok 2024-09-11T07:34:12.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:12 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3774679601' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:34:12.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:12 smithi022 ceph-mon[37330]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:34:12.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:12 smithi022 ceph-mon[37330]: pgmap v340: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 779 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-11T07:34:12.972 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:13.006 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:13.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:13.007 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:13.108 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:13.129 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:13.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:13.130 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:13.206 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 33 16 64110 64094 121.362 77.0625 0.00620071 0.0082344 2024-09-11T07:34:13.230 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:13.259 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:13.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:13.260 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:13.360 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:13.402 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:13.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:13.403 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:13.504 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:13.536 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:13.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:13.537 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:13.638 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:13.660 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:13.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:13.661 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:13.762 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:13.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:13 smithi104 ceph-mon[33507]: 4.1f scrub starts 2024-09-11T07:34:13.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:13 smithi104 ceph-mon[33507]: 4.1f scrub ok 2024-09-11T07:34:13.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:13 smithi104 ceph-mon[33507]: 5.7 scrub starts 2024-09-11T07:34:13.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:13 smithi104 ceph-mon[33507]: 5.7 scrub ok 2024-09-11T07:34:13.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:13 smithi104 ceph-mon[33507]: 4.12 scrub starts 2024-09-11T07:34:13.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:13 smithi104 ceph-mon[33507]: 4.12 scrub ok 2024-09-11T07:34:13.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:13 smithi104 ceph-mon[33507]: 6.16 scrub starts 2024-09-11T07:34:13.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:13 smithi104 ceph-mon[33507]: 6.16 scrub ok 2024-09-11T07:34:13.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:13 smithi104 ceph-mon[33507]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:34:13.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:13 smithi104 ceph-mon[33507]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:34:13.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:13 smithi104 ceph-mon[33507]: osdmap e198: 8 total, 8 up, 8 in 2024-09-11T07:34:13.792 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:13.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:13.793 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:13.894 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:13.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:13 smithi022 ceph-mon[29462]: 4.1f scrub starts 2024-09-11T07:34:13.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:13 smithi022 ceph-mon[29462]: 4.1f scrub ok 2024-09-11T07:34:13.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:13 smithi022 ceph-mon[29462]: 5.7 scrub starts 2024-09-11T07:34:13.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:13 smithi022 ceph-mon[29462]: 5.7 scrub ok 2024-09-11T07:34:13.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:13 smithi022 ceph-mon[29462]: 4.12 scrub starts 2024-09-11T07:34:13.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:13 smithi022 ceph-mon[29462]: 4.12 scrub ok 2024-09-11T07:34:13.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:13 smithi022 ceph-mon[29462]: 6.16 scrub starts 2024-09-11T07:34:13.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:13 smithi022 ceph-mon[29462]: 6.16 scrub ok 2024-09-11T07:34:13.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:13 smithi022 ceph-mon[29462]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:34:13.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:13 smithi022 ceph-mon[29462]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:34:13.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:13 smithi022 ceph-mon[29462]: osdmap e198: 8 total, 8 up, 8 in 2024-09-11T07:34:13.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:13 smithi022 ceph-mon[37330]: 4.1f scrub starts 2024-09-11T07:34:13.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:13 smithi022 ceph-mon[37330]: 4.1f scrub ok 2024-09-11T07:34:13.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:13 smithi022 ceph-mon[37330]: 5.7 scrub starts 2024-09-11T07:34:13.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:13 smithi022 ceph-mon[37330]: 5.7 scrub ok 2024-09-11T07:34:13.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:13 smithi022 ceph-mon[37330]: 4.12 scrub starts 2024-09-11T07:34:13.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:13 smithi022 ceph-mon[37330]: 4.12 scrub ok 2024-09-11T07:34:13.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:13 smithi022 ceph-mon[37330]: 6.16 scrub starts 2024-09-11T07:34:13.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:13 smithi022 ceph-mon[37330]: 6.16 scrub ok 2024-09-11T07:34:13.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:13 smithi022 ceph-mon[37330]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:34:13.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:13 smithi022 ceph-mon[37330]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:34:13.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:13 smithi022 ceph-mon[37330]: osdmap e198: 8 total, 8 up, 8 in 2024-09-11T07:34:13.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:13.920 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:13.920 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:14.021 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:14.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:14.059 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:14.059 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:14.160 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:14.199 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:14.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:14.200 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:14.206 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 34 15 65349 65334 120.072 77.5 0.0123819 0.00832321 2024-09-11T07:34:14.300 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:34:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:34:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:34:14.301 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:14.330 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:14.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:14.332 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:14.337 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-11T07:34:14.433 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:14.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:14.472 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:14.472 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:14.573 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:14.603 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:14.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:14.604 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:14.705 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:14.708 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:34:14.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:14.738 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:14.738 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:14.839 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:14.876 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:14 smithi104 ceph-mon[33507]: pgmap v342: 148 pgs: 148 active+clean; 3.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 780 B/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-11T07:34:14.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:14.879 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:14.879 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:14.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:14 smithi022 ceph-mon[29462]: pgmap v342: 148 pgs: 148 active+clean; 3.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 780 B/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-11T07:34:14.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:14 smithi022 ceph-mon[37330]: pgmap v342: 148 pgs: 148 active+clean; 3.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 780 B/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-11T07:34:14.980 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:15.007 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:15.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:15.009 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:15.110 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:15.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:15.154 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:15.154 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:15.206 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 35 16 66557 66541 118.797 75.4375 0.00671965 0.00841293 2024-09-11T07:34:15.255 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:15.298 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:15.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:15.299 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:15.400 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:15.430 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:15.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:15.431 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:15.532 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:15.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:15.567 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:15.567 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:15.668 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:15.698 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:15.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:15.699 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:15.800 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:15.839 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:15.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:15.840 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:15.941 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:15.981 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:15.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:15.982 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:16.083 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:16.120 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:16.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:16.121 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:16.206 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 36 16 67827 67811 117.702 79.375 0.00749537 0.00849103 2024-09-11T07:34:16.223 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:16.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:16.245 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:16.245 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:16.346 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:16.376 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:16.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:16.377 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:16.478 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:16.510 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:16.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:16.511 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:16.611 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:16.648 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:16.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:16.649 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:16.713 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:34:16.714 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:34:16.714 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2024-09-11T07:34:16.714 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2024-09-11T07:34:16.714 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:34:16.749 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:16.793 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:16.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:16.794 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:16.822 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:16 smithi022 ceph-mon[29462]: osdmap e199: 8 total, 8 up, 8 in 2024-09-11T07:34:16.822 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:16 smithi022 ceph-mon[29462]: pgmap v344: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 65 MiB/s wr, 1.03k op/s 2024-09-11T07:34:16.823 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:16 smithi022 ceph-mon[37330]: osdmap e199: 8 total, 8 up, 8 in 2024-09-11T07:34:16.823 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:16 smithi022 ceph-mon[37330]: pgmap v344: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 65 MiB/s wr, 1.03k op/s 2024-09-11T07:34:16.895 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:16.927 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:16.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:16.928 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:17.029 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:17.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:16 smithi104 ceph-mon[33507]: osdmap e199: 8 total, 8 up, 8 in 2024-09-11T07:34:17.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:16 smithi104 ceph-mon[33507]: pgmap v344: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 65 MiB/s wr, 1.03k op/s 2024-09-11T07:34:17.073 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:17.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:17.074 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:17.175 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:17.204 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:17.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:17.205 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:17.207 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 37 16 69119 69103 116.703 80.75 0.0116086 0.00856424 2024-09-11T07:34:17.306 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:17.339 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:17.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:17.340 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:17.432 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-11T07:34:17.441 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:17.479 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:17.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:17.480 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:17.581 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:17.620 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:17.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:17.621 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:17.722 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:17.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:17.772 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:17.772 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:17.814 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:34:17.873 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:17.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:17.905 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:17.905 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:18.006 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:18.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:18.041 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:18.041 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:18.142 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:18.169 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:18.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:18.171 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:18.206 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 38 16 70541 70525 115.971 88.875 0.00897168 0.00861807 2024-09-11T07:34:18.272 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:18.312 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:18.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:18.313 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:18.414 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:18.444 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:18.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:18.492 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:18.593 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:18.635 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:18.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:18.636 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:18.737 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:18.768 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:18.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:18.769 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:18.792 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:34:18.869 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:18.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:18.917 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:18.917 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:19.018 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:19.059 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:19.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:19.060 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:34:19.146 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:34:19.161 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:19.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:19.206 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:19.206 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:19.207 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 39 16 71821 71805 115.048 80 0.00911076 0.00868747 2024-09-11T07:34:19.208 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:19 smithi022 ceph-mon[29462]: pgmap v345: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 677 B/s rd, 51 MiB/s wr, 820 op/s 2024-09-11T07:34:19.210 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:19 smithi022 ceph-mon[37330]: pgmap v345: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 677 B/s rd, 51 MiB/s wr, 820 op/s 2024-09-11T07:34:19.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:19 smithi104 ceph-mon[33507]: pgmap v345: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 677 B/s rd, 51 MiB/s wr, 820 op/s 2024-09-11T07:34:19.307 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:19.342 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:19.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:19.343 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:19.444 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:19.469 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:19.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:19.470 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:19.571 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:19.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:19.607 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:19.607 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:19.641 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:34:19 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:34:19.708 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:19.743 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:19.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:19.744 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:19.845 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:19.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:19.872 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:19.873 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:19.973 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:20.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:20.007 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:20.007 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:20.108 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:20.149 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:20.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:20.150 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:20.196 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:34:20.196 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2024-09-11T07:34:20.196 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2024-09-11T07:34:20.197 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2024-09-11T07:34:20.197 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.197 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:119878 7.0 7:0d5a1409:::benchmark_data_smithi022_77720_object119877:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-11T07:34:20.197 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:30:28.063301+0000", 2024-09-11T07:34:20.197 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 232.127060126, 2024-09-11T07:34:20.197 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.12935533599999999, 2024-09-11T07:34:20.197 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:20.197 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:20.197 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:20.197 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:34:20.197 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:34:20.197 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 119878 2024-09-11T07:34:20.197 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.198 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:20.198 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.198 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:20.198 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063301+0000", 2024-09-11T07:34:20.198 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.198 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.198 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.198 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:20.198 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063301+0000", 2024-09-11T07:34:20.198 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.198 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.198 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.198 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:20.198 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063298+0000", 2024-09-11T07:34:20.199 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:34:20.199 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.199 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.199 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:20.199 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063402+0000", 2024-09-11T07:34:20.199 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010310999999999999 2024-09-11T07:34:20.199 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.199 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.199 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:20.199 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063414+0000", 2024-09-11T07:34:20.199 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2237e-05 2024-09-11T07:34:20.199 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.200 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.200 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:20.200 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063419+0000", 2024-09-11T07:34:20.200 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5489999999999997e-06 2024-09-11T07:34:20.200 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.200 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.200 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:20.200 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063454+0000", 2024-09-11T07:34:20.200 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5132000000000002e-05 2024-09-11T07:34:20.200 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.200 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.200 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:20.200 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063565+0000", 2024-09-11T07:34:20.200 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000110246 2024-09-11T07:34:20.201 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.201 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.201 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:34:20.201 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063651+0000", 2024-09-11T07:34:20.201 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6661000000000003e-05 2024-09-11T07:34:20.201 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.201 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.201 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:20.201 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.066483+0000", 2024-09-11T07:34:20.201 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0028320490000000001 2024-09-11T07:34:20.201 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.201 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.201 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.202 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.066815+0000", 2024-09-11T07:34:20.202 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00033145700000000001 2024-09-11T07:34:20.202 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.202 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.202 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.202 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.192594+0000", 2024-09-11T07:34:20.202 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.12577914400000001 2024-09-11T07:34:20.202 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.202 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.202 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:20.202 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.192633+0000", 2024-09-11T07:34:20.202 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9042999999999999e-05 2024-09-11T07:34:20.202 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.203 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.203 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:20.203 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.192656+0000", 2024-09-11T07:34:20.203 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3125999999999998e-05 2024-09-11T07:34:20.203 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.203 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:20.203 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.203 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.203 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.203 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:121102 7.0 7:09266ef0:::benchmark_data_smithi022_77720_object121101:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:34:20.203 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:30:29.710752+0000", 2024-09-11T07:34:20.203 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 230.47960898100001, 2024-09-11T07:34:20.203 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.121706142, 2024-09-11T07:34:20.204 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:20.204 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:20.204 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:20.204 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:34:20.204 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:34:20.204 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 121102 2024-09-11T07:34:20.204 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.204 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:20.204 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.204 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:20.204 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.710752+0000", 2024-09-11T07:34:20.204 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.204 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.204 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.205 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:20.205 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.710752+0000", 2024-09-11T07:34:20.205 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.205 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.205 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.205 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:20.205 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.710749+0000", 2024-09-11T07:34:20.205 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:34:20.205 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.205 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.205 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:20.205 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.710809+0000", 2024-09-11T07:34:20.205 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9255000000000001e-05 2024-09-11T07:34:20.205 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.206 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.206 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:20.206 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.710832+0000", 2024-09-11T07:34:20.206 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3031e-05 2024-09-11T07:34:20.206 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.206 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.206 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:20.206 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.710839+0000", 2024-09-11T07:34:20.206 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3889999999999999e-06 2024-09-11T07:34:20.206 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.206 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.206 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:20.206 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.710883+0000", 2024-09-11T07:34:20.207 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3770999999999998e-05 2024-09-11T07:34:20.207 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.207 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.207 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:20.207 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.711002+0000", 2024-09-11T07:34:20.207 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000119518 2024-09-11T07:34:20.207 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.207 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.207 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:34:20.207 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.711141+0000", 2024-09-11T07:34:20.207 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013841 2024-09-11T07:34:20.207 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.207 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.207 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:20.208 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.713588+0000", 2024-09-11T07:34:20.208 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0024468179999999999 2024-09-11T07:34:20.208 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.208 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.208 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.208 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.713875+0000", 2024-09-11T07:34:20.208 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00028783500000000001 2024-09-11T07:34:20.208 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.208 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.208 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.208 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.832376+0000", 2024-09-11T07:34:20.208 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.11850076800000001 2024-09-11T07:34:20.209 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.209 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.209 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:20.209 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.832424+0000", 2024-09-11T07:34:20.209 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8047999999999998e-05 2024-09-11T07:34:20.209 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.209 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.209 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:20.209 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.832458+0000", 2024-09-11T07:34:20.209 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3849000000000001e-05 2024-09-11T07:34:20.209 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.209 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:20.209 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.209 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.210 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.210 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:173291 7.0 7:0bf19ddd:::benchmark_data_smithi022_77720_object49536:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-11T07:34:20.210 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:31:20.297211+0000", 2024-09-11T07:34:20.210 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 179.89314989600001, 2024-09-11T07:34:20.210 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.110572034, 2024-09-11T07:34:20.210 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:20.210 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:20.210 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:20.210 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:34:20.210 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:34:20.210 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 173291 2024-09-11T07:34:20.210 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.210 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:20.211 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.211 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:20.211 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.297211+0000", 2024-09-11T07:34:20.211 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.211 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.211 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.211 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:20.211 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.297211+0000", 2024-09-11T07:34:20.211 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.211 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.211 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.211 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:20.211 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.297209+0000", 2024-09-11T07:34:20.212 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:34:20.212 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.212 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.212 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:20.212 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.297216+0000", 2024-09-11T07:34:20.212 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5420000000000002e-06 2024-09-11T07:34:20.212 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.212 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.212 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:20.212 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.297226+0000", 2024-09-11T07:34:20.212 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0111e-05 2024-09-11T07:34:20.212 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.212 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.212 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:20.213 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.297231+0000", 2024-09-11T07:34:20.213 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5570000000000003e-06 2024-09-11T07:34:20.213 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.213 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.213 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:20.213 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.297264+0000", 2024-09-11T07:34:20.213 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2778999999999999e-05 2024-09-11T07:34:20.213 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.213 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.213 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:20.213 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.297359+0000", 2024-09-11T07:34:20.213 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5050999999999995e-05 2024-09-11T07:34:20.213 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.214 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.214 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:34:20.214 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.297458+0000", 2024-09-11T07:34:20.214 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8609000000000004e-05 2024-09-11T07:34:20.214 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.214 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.214 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:20.214 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.301513+0000", 2024-09-11T07:34:20.214 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0040550480000000003 2024-09-11T07:34:20.214 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.214 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.214 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.214 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.302635+0000", 2024-09-11T07:34:20.215 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0011224850000000001 2024-09-11T07:34:20.215 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.215 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.215 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.215 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.407729+0000", 2024-09-11T07:34:20.215 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.10509397500000001 2024-09-11T07:34:20.215 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.215 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.215 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:20.215 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.407759+0000", 2024-09-11T07:34:20.215 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9536999999999999e-05 2024-09-11T07:34:20.215 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.215 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.215 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:20.216 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.407783+0000", 2024-09-11T07:34:20.216 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.4362000000000002e-05 2024-09-11T07:34:20.216 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.216 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:20.216 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.216 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.216 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.216 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:173295 7.0 7:0b0dd3c6:::benchmark_data_smithi022_77720_object49540:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139)", 2024-09-11T07:34:20.216 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:31:20.303629+0000", 2024-09-11T07:34:20.216 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 179.88673193299999, 2024-09-11T07:34:20.216 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.104283681, 2024-09-11T07:34:20.216 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:20.216 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:20.217 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:20.217 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:34:20.217 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:34:20.217 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 173295 2024-09-11T07:34:20.217 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.217 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:20.217 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.217 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:20.217 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.303629+0000", 2024-09-11T07:34:20.217 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.217 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.217 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.217 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:20.217 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.303629+0000", 2024-09-11T07:34:20.218 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.218 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.218 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.218 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:20.218 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.303627+0000", 2024-09-11T07:34:20.218 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:34:20.218 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.218 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.218 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:20.218 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.303635+0000", 2024-09-11T07:34:20.218 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3100000000000003e-06 2024-09-11T07:34:20.218 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.218 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.219 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:20.219 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.303644+0000", 2024-09-11T07:34:20.219 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5799999999999998e-06 2024-09-11T07:34:20.219 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.219 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.219 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:20.219 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.303648+0000", 2024-09-11T07:34:20.219 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.002e-06 2024-09-11T07:34:20.219 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.219 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.219 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:20.219 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.303675+0000", 2024-09-11T07:34:20.219 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.6441999999999999e-05 2024-09-11T07:34:20.219 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.220 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.220 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:20.220 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.303764+0000", 2024-09-11T07:34:20.220 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9085999999999994e-05 2024-09-11T07:34:20.220 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.220 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.220 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:34:20.220 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.303841+0000", 2024-09-11T07:34:20.220 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7179000000000001e-05 2024-09-11T07:34:20.220 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.220 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.220 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:20.220 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.306671+0000", 2024-09-11T07:34:20.220 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0028303130000000001 2024-09-11T07:34:20.221 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.221 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.221 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.221 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.307717+0000", 2024-09-11T07:34:20.221 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001046118 2024-09-11T07:34:20.221 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.221 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.221 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.221 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.407819+0000", 2024-09-11T07:34:20.221 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.10010218799999999 2024-09-11T07:34:20.221 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.221 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.221 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:20.222 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.407876+0000", 2024-09-11T07:34:20.222 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6560000000000001e-05 2024-09-11T07:34:20.222 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.222 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.222 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:20.222 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.407913+0000", 2024-09-11T07:34:20.222 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6704e-05 2024-09-11T07:34:20.222 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.222 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:20.222 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.222 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.222 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.222 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:189698 7.0 7:0c18b714:::benchmark_data_smithi022_77720_object65943:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-11T07:34:20.223 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:31:36.241692+0000", 2024-09-11T07:34:20.223 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 163.94866849100001, 2024-09-11T07:34:20.223 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.074615498000000002, 2024-09-11T07:34:20.223 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:20.223 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:20.223 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:20.223 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:34:20.223 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:34:20.223 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 189698 2024-09-11T07:34:20.224 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.224 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:20.224 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.224 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:20.224 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.241692+0000", 2024-09-11T07:34:20.224 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.224 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.224 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.224 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:20.224 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.241692+0000", 2024-09-11T07:34:20.224 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.224 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.225 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.225 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:20.225 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.241690+0000", 2024-09-11T07:34:20.225 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999971 2024-09-11T07:34:20.225 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.225 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.225 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:20.225 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T07:34:20.226 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:2024-09-11T07:34:20.206364+0000 min lat: 0.00108265 max lat: 0.159168 avg lat: 0.0087494 2024-09-11T07:34:20.226 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T07:34:20.226 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 40 16 73134 73118 114.223 82.0625 0.01209 0.0087494 2024-09-11T07:34:20.227 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.241701+0000", 2024-09-11T07:34:20.227 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1127e-05 2024-09-11T07:34:20.227 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.227 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.227 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:20.228 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.241722+0000", 2024-09-11T07:34:20.228 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0616e-05 2024-09-11T07:34:20.228 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.228 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.228 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:20.228 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.241731+0000", 2024-09-11T07:34:20.228 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8030000000000008e-06 2024-09-11T07:34:20.228 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.228 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.228 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:20.228 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.241811+0000", 2024-09-11T07:34:20.228 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9170000000000003e-05 2024-09-11T07:34:20.228 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.229 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.229 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:20.229 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.242037+0000", 2024-09-11T07:34:20.229 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00022672100000000001 2024-09-11T07:34:20.229 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.229 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.229 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:34:20.229 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.242170+0000", 2024-09-11T07:34:20.229 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000132702 2024-09-11T07:34:20.229 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.229 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.229 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:20.229 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.244416+0000", 2024-09-11T07:34:20.229 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0022463769999999999 2024-09-11T07:34:20.230 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.230 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.230 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.230 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.251396+0000", 2024-09-11T07:34:20.230 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0069796290000000002 2024-09-11T07:34:20.230 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.230 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.230 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.230 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.316254+0000", 2024-09-11T07:34:20.230 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.064858117000000007 2024-09-11T07:34:20.230 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.230 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.230 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:20.231 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.316284+0000", 2024-09-11T07:34:20.231 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.025e-05 2024-09-11T07:34:20.231 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.231 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.231 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:20.231 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.316308+0000", 2024-09-11T07:34:20.231 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3632999999999999e-05 2024-09-11T07:34:20.231 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.231 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:20.231 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.231 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.231 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.231 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:189702 7.0 7:03f43c7a:::benchmark_data_smithi022_77720_object65947:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-11T07:34:20.232 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:31:36.247101+0000", 2024-09-11T07:34:20.232 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 163.94325953699999, 2024-09-11T07:34:20.232 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.073677724999999999, 2024-09-11T07:34:20.232 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:20.232 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:20.232 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:20.232 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:34:20.232 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:34:20.232 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 189702 2024-09-11T07:34:20.232 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.232 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:20.232 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.232 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:20.233 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247101+0000", 2024-09-11T07:34:20.233 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.233 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.233 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.233 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:20.233 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247101+0000", 2024-09-11T07:34:20.233 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.233 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.233 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.233 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:20.233 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247100+0000", 2024-09-11T07:34:20.233 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:34:20.233 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.234 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.234 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:20.234 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247108+0000", 2024-09-11T07:34:20.234 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3780000000000001e-06 2024-09-11T07:34:20.234 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.234 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.234 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:20.234 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247120+0000", 2024-09-11T07:34:20.234 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1558e-05 2024-09-11T07:34:20.234 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.234 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.234 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:20.234 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247126+0000", 2024-09-11T07:34:20.235 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2260000000000002e-06 2024-09-11T07:34:20.235 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.235 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.235 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:20.235 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247166+0000", 2024-09-11T07:34:20.235 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0009000000000003e-05 2024-09-11T07:34:20.235 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.235 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.235 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:20.235 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247238+0000", 2024-09-11T07:34:20.235 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.2008999999999994e-05 2024-09-11T07:34:20.235 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.236 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.236 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:34:20.236 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247302+0000", 2024-09-11T07:34:20.236 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4257000000000005e-05 2024-09-11T07:34:20.236 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.236 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.236 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:20.236 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.249620+0000", 2024-09-11T07:34:20.236 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0023178830000000002 2024-09-11T07:34:20.236 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.236 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.236 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.236 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.251426+0000", 2024-09-11T07:34:20.237 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0018057170000000001 2024-09-11T07:34:20.237 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.237 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.237 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.237 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.320727+0000", 2024-09-11T07:34:20.237 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.069301077000000003 2024-09-11T07:34:20.237 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.237 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.237 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:20.237 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.320756+0000", 2024-09-11T07:34:20.237 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9115000000000002e-05 2024-09-11T07:34:20.237 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.237 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.238 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:20.238 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.320779+0000", 2024-09-11T07:34:20.238 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3125999999999998e-05 2024-09-11T07:34:20.238 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.238 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:20.238 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.238 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.238 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.238 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:13641 8.8 8:1997f03a:::benchmark_data_smithi022_108406_object13640:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-11T07:34:20.238 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:33:44.755776+0000", 2024-09-11T07:34:20.238 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 35.434584549999997, 2024-09-11T07:34:20.238 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.102084483, 2024-09-11T07:34:20.238 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:20.239 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:20.239 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:20.239 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:34:20.239 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:34:20.239 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 13641 2024-09-11T07:34:20.239 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.239 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:20.239 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.239 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:20.239 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.755776+0000", 2024-09-11T07:34:20.239 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.239 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.239 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.239 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:20.240 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.755776+0000", 2024-09-11T07:34:20.240 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.240 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.240 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.240 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:20.240 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.755775+0000", 2024-09-11T07:34:20.240 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:34:20.240 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.240 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.240 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:20.241 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.755850+0000", 2024-09-11T07:34:20.241 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5088999999999999e-05 2024-09-11T07:34:20.241 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.241 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.241 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:20.241 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.755867+0000", 2024-09-11T07:34:20.241 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7320999999999999e-05 2024-09-11T07:34:20.241 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.241 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.241 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:20.241 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.755874+0000", 2024-09-11T07:34:20.241 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3629999999999999e-06 2024-09-11T07:34:20.241 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.242 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.242 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:20.242 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.755928+0000", 2024-09-11T07:34:20.242 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4639000000000003e-05 2024-09-11T07:34:20.242 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.242 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.242 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:20.242 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.756083+0000", 2024-09-11T07:34:20.242 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015470800000000001 2024-09-11T07:34:20.242 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.242 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.242 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:34:20.243 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.756225+0000", 2024-09-11T07:34:20.243 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014204500000000001 2024-09-11T07:34:20.243 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.243 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.243 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.243 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.757465+0000", 2024-09-11T07:34:20.243 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0012398120000000001 2024-09-11T07:34:20.243 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.243 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.243 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:20.243 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.757596+0000", 2024-09-11T07:34:20.243 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000131006 2024-09-11T07:34:20.243 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.243 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.244 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.244 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.857783+0000", 2024-09-11T07:34:20.244 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.100186975 2024-09-11T07:34:20.244 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.244 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.244 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:20.244 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.857812+0000", 2024-09-11T07:34:20.244 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9371000000000001e-05 2024-09-11T07:34:20.244 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.244 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.244 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:20.244 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.857861+0000", 2024-09-11T07:34:20.244 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8606999999999997e-05 2024-09-11T07:34:20.244 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.245 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:20.245 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.245 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.245 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.245 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:13647 8.8 8:137d89d6:::benchmark_data_smithi022_108406_object13646:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-11T07:34:20.245 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:33:44.757758+0000", 2024-09-11T07:34:20.245 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 35.432602746999997, 2024-09-11T07:34:20.245 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.10313436400000001, 2024-09-11T07:34:20.245 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:20.245 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:20.245 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:20.245 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:34:20.245 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:34:20.245 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 13647 2024-09-11T07:34:20.246 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.246 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:20.246 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.246 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:20.246 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.757758+0000", 2024-09-11T07:34:20.246 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.246 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.246 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.246 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:20.246 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.757758+0000", 2024-09-11T07:34:20.246 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.246 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.246 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.246 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:20.246 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.757757+0000", 2024-09-11T07:34:20.247 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:34:20.247 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.247 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.247 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:20.247 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.757796+0000", 2024-09-11T07:34:20.247 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9663999999999998e-05 2024-09-11T07:34:20.247 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.247 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.247 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:20.247 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.757812+0000", 2024-09-11T07:34:20.247 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5622000000000001e-05 2024-09-11T07:34:20.247 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.247 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.248 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:20.248 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.757819+0000", 2024-09-11T07:34:20.248 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6340000000000003e-06 2024-09-11T07:34:20.248 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.248 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.248 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:20.248 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.758238+0000", 2024-09-11T07:34:20.248 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00041897199999999999 2024-09-11T07:34:20.248 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.248 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.248 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:20.248 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.758356+0000", 2024-09-11T07:34:20.248 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000118644 2024-09-11T07:34:20.249 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.249 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.249 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:34:20.249 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.758523+0000", 2024-09-11T07:34:20.249 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016667200000000001 2024-09-11T07:34:20.249 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.249 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.249 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:20.249 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.759951+0000", 2024-09-11T07:34:20.249 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0014284639999999999 2024-09-11T07:34:20.249 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.249 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.249 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.250 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.760052+0000", 2024-09-11T07:34:20.250 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000100388 2024-09-11T07:34:20.250 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.250 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.250 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.250 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.860800+0000", 2024-09-11T07:34:20.250 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.100747955 2024-09-11T07:34:20.250 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.250 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.250 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:20.250 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.860855+0000", 2024-09-11T07:34:20.250 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5123000000000001e-05 2024-09-11T07:34:20.250 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.251 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.251 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:20.251 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.860893+0000", 2024-09-11T07:34:20.251 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7762000000000002e-05 2024-09-11T07:34:20.251 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.251 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:20.251 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.251 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.251 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.251 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:28689 8.8 8:1221b283:::benchmark_data_smithi022_108406_object28688:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e189)", 2024-09-11T07:34:20.251 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:33:51.015983+0000", 2024-09-11T07:34:20.251 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 29.174378437000001, 2024-09-11T07:34:20.251 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13574931500000001, 2024-09-11T07:34:20.252 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:20.252 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:20.252 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:20.252 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:34:20.252 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:34:20.252 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 28689 2024-09-11T07:34:20.252 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.252 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:20.252 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.252 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:20.252 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.015983+0000", 2024-09-11T07:34:20.252 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.252 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.253 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.253 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:20.253 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.015983+0000", 2024-09-11T07:34:20.253 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.253 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.253 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.253 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:20.253 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.015981+0000", 2024-09-11T07:34:20.253 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:34:20.253 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.253 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.253 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:20.253 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.016014+0000", 2024-09-11T07:34:20.253 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3164999999999998e-05 2024-09-11T07:34:20.254 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.254 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.254 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:20.254 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.016026+0000", 2024-09-11T07:34:20.254 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1994000000000001e-05 2024-09-11T07:34:20.254 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.254 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.254 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:20.254 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.016032+0000", 2024-09-11T07:34:20.254 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8220000000000003e-06 2024-09-11T07:34:20.254 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.254 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.254 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:20.254 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.016061+0000", 2024-09-11T07:34:20.254 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.8654e-05 2024-09-11T07:34:20.255 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.255 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.255 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:20.255 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.016179+0000", 2024-09-11T07:34:20.255 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000117959 2024-09-11T07:34:20.255 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.255 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.255 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:34:20.255 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.016311+0000", 2024-09-11T07:34:20.255 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013222200000000001 2024-09-11T07:34:20.255 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.255 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.255 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.256 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.018355+0000", 2024-09-11T07:34:20.256 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002044466 2024-09-11T07:34:20.256 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.256 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.256 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:20.256 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.019134+0000", 2024-09-11T07:34:20.256 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00077916300000000003 2024-09-11T07:34:20.256 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.256 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.256 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.256 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.151654+0000", 2024-09-11T07:34:20.256 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.132519682 2024-09-11T07:34:20.256 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.257 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.257 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:20.257 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.151703+0000", 2024-09-11T07:34:20.257 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8458999999999998e-05 2024-09-11T07:34:20.257 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.257 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.257 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:20.257 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.151732+0000", 2024-09-11T07:34:20.257 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9286999999999999e-05 2024-09-11T07:34:20.257 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.257 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:20.257 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.257 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.258 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.258 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:28706 8.8 8:14c85bdd:::benchmark_data_smithi022_108406_object28705:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e189)", 2024-09-11T07:34:20.258 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:33:51.022734+0000", 2024-09-11T07:34:20.258 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 29.167626506000001, 2024-09-11T07:34:20.258 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.135307186, 2024-09-11T07:34:20.258 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:20.258 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:20.258 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:20.258 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:34:20.258 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:34:20.258 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 28706 2024-09-11T07:34:20.258 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.258 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.022734+0000", 2024-09-11T07:34:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.022734+0000", 2024-09-11T07:34:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.022733+0000", 2024-09-11T07:34:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:34:20.260 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.260 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.260 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:20.260 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.022788+0000", 2024-09-11T07:34:20.260 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4837999999999999e-05 2024-09-11T07:34:20.260 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.260 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.260 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:20.260 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.022796+0000", 2024-09-11T07:34:20.260 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6589999999999996e-06 2024-09-11T07:34:20.260 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.260 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.260 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.022801+0000", 2024-09-11T07:34:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3089999999999997e-06 2024-09-11T07:34:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.022849+0000", 2024-09-11T07:34:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8432e-05 2024-09-11T07:34:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.022966+0000", 2024-09-11T07:34:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011678100000000001 2024-09-11T07:34:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:34:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.023103+0000", 2024-09-11T07:34:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013668900000000001 2024-09-11T07:34:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.024203+0000", 2024-09-11T07:34:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001100539 2024-09-11T07:34:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.024892+0000", 2024-09-11T07:34:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00068921200000000003 2024-09-11T07:34:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.157959+0000", 2024-09-11T07:34:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13306637800000001 2024-09-11T07:34:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.158003+0000", 2024-09-11T07:34:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4206999999999999e-05 2024-09-11T07:34:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.158042+0000", 2024-09-11T07:34:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8628999999999999e-05 2024-09-11T07:34:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:29825 8.8 8:1c95b3ee:::benchmark_data_smithi022_108406_object29824:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e189)", 2024-09-11T07:34:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:33:51.652793+0000", 2024-09-11T07:34:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 28.537568015000002, 2024-09-11T07:34:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.075804250000000004, 2024-09-11T07:34:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:34:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:34:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 29825 2024-09-11T07:34:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.652793+0000", 2024-09-11T07:34:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.652793+0000", 2024-09-11T07:34:20.266 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.266 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.266 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.266 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:20.266 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.652792+0000", 2024-09-11T07:34:20.266 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:34:20.266 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.266 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.266 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:20.266 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.652818+0000", 2024-09-11T07:34:20.266 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.6431000000000002e-05 2024-09-11T07:34:20.266 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.266 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.653034+0000", 2024-09-11T07:34:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000215847 2024-09-11T07:34:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.653039+0000", 2024-09-11T07:34:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6369999999999998e-06 2024-09-11T07:34:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.656316+0000", 2024-09-11T07:34:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003276827 2024-09-11T07:34:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.656465+0000", 2024-09-11T07:34:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014931300000000001 2024-09-11T07:34:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:34:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.657134+0000", 2024-09-11T07:34:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00066848799999999996 2024-09-11T07:34:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:20.269 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.658284+0000", 2024-09-11T07:34:20.269 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0011505319999999999 2024-09-11T07:34:20.269 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.269 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.269 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.269 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.658468+0000", 2024-09-11T07:34:20.269 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000183439 2024-09-11T07:34:20.269 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.269 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.269 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.269 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.728499+0000", 2024-09-11T07:34:20.269 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.070031117000000004 2024-09-11T07:34:20.270 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.270 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.270 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:20.270 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.728555+0000", 2024-09-11T07:34:20.270 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:20.272 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:20 smithi022 ceph-mon[29462]: from='client.15237 v1:172.21.15.22:0/397161331' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:34:20.272 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:20 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:20.272 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:20 smithi022 ceph-mon[37330]: from='client.15237 v1:172.21.15.22:0/397161331' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:34:20.272 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:20 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:20.273 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6110999999999999e-05 2024-09-11T07:34:20.273 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.273 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.273 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:20.273 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.728597+0000", 2024-09-11T07:34:20.273 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2400000000000001e-05 2024-09-11T07:34:20.273 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.273 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:20.273 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.273 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.273 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.274 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:31918 8.8 8:16058c3f:::benchmark_data_smithi022_108406_object31917:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e190)", 2024-09-11T07:34:20.274 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:33:52.793111+0000", 2024-09-11T07:34:20.274 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 27.397249723000002, 2024-09-11T07:34:20.274 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.11028637400000001, 2024-09-11T07:34:20.274 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:20.274 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:20.274 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:20.274 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:34:20.274 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:34:20.274 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31918 2024-09-11T07:34:20.274 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.274 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:20.274 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.275 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:20.275 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.793111+0000", 2024-09-11T07:34:20.275 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.275 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.275 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.275 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:20.275 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.793111+0000", 2024-09-11T07:34:20.275 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.275 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.275 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.275 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:20.275 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.793104+0000", 2024-09-11T07:34:20.275 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999933 2024-09-11T07:34:20.275 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.276 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.276 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:20.276 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.793166+0000", 2024-09-11T07:34:20.276 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1296000000000004e-05 2024-09-11T07:34:20.276 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.276 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.276 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:20.276 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.793177+0000", 2024-09-11T07:34:20.276 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1123e-05 2024-09-11T07:34:20.276 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.276 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.276 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:20.276 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.793183+0000", 2024-09-11T07:34:20.277 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7409999999999998e-06 2024-09-11T07:34:20.277 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.277 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.277 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:20.277 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.793218+0000", 2024-09-11T07:34:20.277 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5797999999999998e-05 2024-09-11T07:34:20.277 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.277 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.277 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:20.277 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.793300+0000", 2024-09-11T07:34:20.277 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2150999999999993e-05 2024-09-11T07:34:20.277 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.277 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.278 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:34:20.278 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.793381+0000", 2024-09-11T07:34:20.278 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0320000000000003e-05 2024-09-11T07:34:20.278 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.278 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.278 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.278 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.796146+0000", 2024-09-11T07:34:20.278 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0027649979999999998 2024-09-11T07:34:20.278 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.278 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.278 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:20.278 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.797178+0000", 2024-09-11T07:34:20.278 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001032406 2024-09-11T07:34:20.278 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.279 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.279 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.279 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.903313+0000", 2024-09-11T07:34:20.279 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.106135013 2024-09-11T07:34:20.279 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.279 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.279 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:20.279 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.903362+0000", 2024-09-11T07:34:20.279 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9149e-05 2024-09-11T07:34:20.279 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.279 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.279 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:20.279 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.903398+0000", 2024-09-11T07:34:20.280 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5225000000000002e-05 2024-09-11T07:34:20.280 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.280 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:20.280 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.280 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.280 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.280 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:36507 8.8 8:11b408ae:::benchmark_data_smithi022_108406_object36506:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e191)", 2024-09-11T07:34:20.280 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:33:55.569627+0000", 2024-09-11T07:34:20.280 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 24.620734120000002, 2024-09-11T07:34:20.280 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13017671, 2024-09-11T07:34:20.280 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:20.280 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:20.280 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:20.281 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:34:20.281 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:34:20.281 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 36507 2024-09-11T07:34:20.281 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.281 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:20.281 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.281 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:20.281 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.569627+0000", 2024-09-11T07:34:20.281 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.281 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.281 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.281 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:20.282 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.569627+0000", 2024-09-11T07:34:20.282 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.282 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.282 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.282 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:20.282 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.569624+0000", 2024-09-11T07:34:20.282 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:34:20.282 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.282 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.282 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:20.282 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.569659+0000", 2024-09-11T07:34:20.282 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4276000000000002e-05 2024-09-11T07:34:20.282 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.283 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.283 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:20.283 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.569669+0000", 2024-09-11T07:34:20.283 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0431000000000001e-05 2024-09-11T07:34:20.283 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.283 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.283 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:20.283 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.569675+0000", 2024-09-11T07:34:20.283 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4419999999999997e-06 2024-09-11T07:34:20.283 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.283 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.283 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:20.283 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.569726+0000", 2024-09-11T07:34:20.284 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1289999999999999e-05 2024-09-11T07:34:20.284 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.284 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.284 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:20.284 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.569870+0000", 2024-09-11T07:34:20.284 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000143738 2024-09-11T07:34:20.284 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.284 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.284 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:34:20.284 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.570008+0000", 2024-09-11T07:34:20.284 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000138701 2024-09-11T07:34:20.284 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.284 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.285 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:20.285 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.571300+0000", 2024-09-11T07:34:20.285 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0012921969999999999 2024-09-11T07:34:20.285 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.285 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.285 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.285 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.571517+0000", 2024-09-11T07:34:20.285 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00021640200000000001 2024-09-11T07:34:20.285 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.285 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.285 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.285 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.699718+0000", 2024-09-11T07:34:20.285 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.128201499 2024-09-11T07:34:20.285 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.286 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.286 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:20.286 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.699764+0000", 2024-09-11T07:34:20.286 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5680000000000003e-05 2024-09-11T07:34:20.286 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.286 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.286 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:20.286 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.699804+0000", 2024-09-11T07:34:20.286 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9481999999999999e-05 2024-09-11T07:34:20.286 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.286 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:20.286 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.287 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.287 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.287 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25072.0:62542 8.0 e197/177 8:06b6e595:::benchmark_data_smithi022_108406_object62541:head v 197'3892, mlcod=197'3890)", 2024-09-11T07:34:20.287 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:11.936310+0000", 2024-09-11T07:34:20.287 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 8.2540513440000005, 2024-09-11T07:34:20.287 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.071505043000000004, 2024-09-11T07:34:20.287 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:20.287 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:20.287 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:20.287 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.287 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:20.287 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.936310+0000", 2024-09-11T07:34:20.287 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.288 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.288 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.288 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:20.288 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.936310+0000", 2024-09-11T07:34:20.288 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.288 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.288 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.288 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:20.288 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.936308+0000", 2024-09-11T07:34:20.288 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:34:20.288 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.288 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.288 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:20.289 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.936422+0000", 2024-09-11T07:34:20.289 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000113793 2024-09-11T07:34:20.289 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.289 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.289 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:20.289 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.936432+0000", 2024-09-11T07:34:20.289 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7569999999999999e-06 2024-09-11T07:34:20.289 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.289 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.289 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:20.289 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.936437+0000", 2024-09-11T07:34:20.289 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9470000000000002e-06 2024-09-11T07:34:20.289 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.289 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.290 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:20.290 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.937638+0000", 2024-09-11T07:34:20.290 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0012013429999999999 2024-09-11T07:34:20.290 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.290 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.290 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:20.290 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.937656+0000", 2024-09-11T07:34:20.290 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7533999999999998e-05 2024-09-11T07:34:20.290 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.290 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.290 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:20.290 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.007780+0000", 2024-09-11T07:34:20.290 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.070124461999999999 2024-09-11T07:34:20.291 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.291 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.291 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:20.291 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.007815+0000", 2024-09-11T07:34:20.291 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4609e-05 2024-09-11T07:34:20.291 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.291 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:20.291 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.291 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.291 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.291 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25072.0:62545 8.0 e197/177 8:0bfa4ab9:::benchmark_data_smithi022_108406_object62544:head v 197'3893, mlcod=197'3890)", 2024-09-11T07:34:20.291 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:11.940273+0000", 2024-09-11T07:34:20.291 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 8.2500878839999991, 2024-09-11T07:34:20.291 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.071980819000000001, 2024-09-11T07:34:20.292 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:20.292 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:20.292 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:20.292 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.292 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:20.292 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.940273+0000", 2024-09-11T07:34:20.292 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.292 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.292 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.292 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:20.292 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.940273+0000", 2024-09-11T07:34:20.292 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.292 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.293 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.293 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:20.293 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.940272+0000", 2024-09-11T07:34:20.293 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:34:20.293 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.293 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.293 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:20.293 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.940308+0000", 2024-09-11T07:34:20.293 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6282999999999998e-05 2024-09-11T07:34:20.293 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.293 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.293 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:20.293 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.940318+0000", 2024-09-11T07:34:20.293 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0199e-05 2024-09-11T07:34:20.294 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.294 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.294 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:20.294 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.940324+0000", 2024-09-11T07:34:20.294 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7490000000000004e-06 2024-09-11T07:34:20.294 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.294 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.294 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:20.294 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.940385+0000", 2024-09-11T07:34:20.294 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1266999999999995e-05 2024-09-11T07:34:20.294 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.294 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.294 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:20.294 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.940403+0000", 2024-09-11T07:34:20.295 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7569999999999999e-05 2024-09-11T07:34:20.295 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.295 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.295 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:20.295 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.012219+0000", 2024-09-11T07:34:20.295 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.071815961999999997 2024-09-11T07:34:20.295 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.295 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.295 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:20.295 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.012254+0000", 2024-09-11T07:34:20.295 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5290999999999998e-05 2024-09-11T07:34:20.295 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.295 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:20.296 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.296 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.296 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.296 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:63543 8.8 8:17f4e5d0:::benchmark_data_smithi022_108406_object63542:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-11T07:34:20.296 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:12.693526+0000", 2024-09-11T07:34:20.296 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 7.4968350600000004, 2024-09-11T07:34:20.296 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.085182680999999996, 2024-09-11T07:34:20.296 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:20.296 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:20.296 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:20.296 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:34:20.296 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:34:20.296 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 63543 2024-09-11T07:34:20.297 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.297 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:20.297 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.297 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:20.297 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693526+0000", 2024-09-11T07:34:20.297 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.297 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.297 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.297 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:20.297 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693526+0000", 2024-09-11T07:34:20.297 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.297 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.297 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.297 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:20.298 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693525+0000", 2024-09-11T07:34:20.298 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:34:20.298 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.298 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.298 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:20.298 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693604+0000", 2024-09-11T07:34:20.298 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9684999999999993e-05 2024-09-11T07:34:20.298 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.298 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.298 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:20.298 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693617+0000", 2024-09-11T07:34:20.298 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3125000000000001e-05 2024-09-11T07:34:20.298 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.298 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.299 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:20.299 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693623+0000", 2024-09-11T07:34:20.299 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8760000000000002e-06 2024-09-11T07:34:20.299 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.299 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.299 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:20.299 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693660+0000", 2024-09-11T07:34:20.299 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6451e-05 2024-09-11T07:34:20.299 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.299 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.299 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:20.299 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693789+0000", 2024-09-11T07:34:20.299 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012951 2024-09-11T07:34:20.299 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.300 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.300 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:34:20.300 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693921+0000", 2024-09-11T07:34:20.300 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000131817 2024-09-11T07:34:20.300 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.300 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.300 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.300 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.702633+0000", 2024-09-11T07:34:20.300 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0087122129999999999 2024-09-11T07:34:20.300 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.300 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.300 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:20.300 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.702640+0000", 2024-09-11T07:34:20.300 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.109e-06 2024-09-11T07:34:20.301 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.301 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.301 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.301 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.778634+0000", 2024-09-11T07:34:20.301 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.075993639000000002 2024-09-11T07:34:20.301 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.301 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.301 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:20.301 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.778678+0000", 2024-09-11T07:34:20.301 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4289999999999998e-05 2024-09-11T07:34:20.301 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.301 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.301 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:20.302 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.778709+0000", 2024-09-11T07:34:20.302 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.0273999999999998e-05 2024-09-11T07:34:20.302 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.302 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:20.302 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.302 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.302 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.302 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:63545 8.8 8:1cf778d2:::benchmark_data_smithi022_108406_object63544:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-11T07:34:20.302 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:12.693945+0000", 2024-09-11T07:34:20.302 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 7.4964159170000002, 2024-09-11T07:34:20.302 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.084917197999999999, 2024-09-11T07:34:20.302 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:20.302 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:20.303 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:20.303 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:34:20.303 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:34:20.303 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 63545 2024-09-11T07:34:20.303 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.303 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:20.303 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.303 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:20.303 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693945+0000", 2024-09-11T07:34:20.303 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.303 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.303 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.303 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:20.303 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693945+0000", 2024-09-11T07:34:20.304 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.304 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.304 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.304 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:20.304 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693936+0000", 2024-09-11T07:34:20.304 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999909 2024-09-11T07:34:20.304 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.304 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.304 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:20.304 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693984+0000", 2024-09-11T07:34:20.304 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7815999999999998e-05 2024-09-11T07:34:20.304 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.304 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.305 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:20.305 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693995+0000", 2024-09-11T07:34:20.305 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.06e-05 2024-09-11T07:34:20.305 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.305 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.305 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:20.305 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.694000+0000", 2024-09-11T07:34:20.305 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5910000000000003e-06 2024-09-11T07:34:20.305 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.305 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.305 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:20.305 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.695920+0000", 2024-09-11T07:34:20.305 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001919841 2024-09-11T07:34:20.306 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.306 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.306 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:20.306 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.695991+0000", 2024-09-11T07:34:20.306 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1290000000000004e-05 2024-09-11T07:34:20.306 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.306 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.306 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:34:20.306 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.696090+0000", 2024-09-11T07:34:20.306 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8818000000000002e-05 2024-09-11T07:34:20.306 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.306 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.306 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.306 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.702671+0000", 2024-09-11T07:34:20.307 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0065810449999999998 2024-09-11T07:34:20.307 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.307 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.307 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:20.307 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.705045+0000", 2024-09-11T07:34:20.307 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0023738409999999998 2024-09-11T07:34:20.307 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.307 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.307 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.307 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.778758+0000", 2024-09-11T07:34:20.307 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.073712593000000007 2024-09-11T07:34:20.307 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.307 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.308 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:20.308 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.778812+0000", 2024-09-11T07:34:20.308 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.452e-05 2024-09-11T07:34:20.308 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.308 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.308 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:20.308 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.778862+0000", 2024-09-11T07:34:20.308 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0160000000000001e-05 2024-09-11T07:34:20.308 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.308 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:20.308 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.308 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.308 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.309 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:63549 8.8 8:153fa01c:::benchmark_data_smithi022_108406_object63548:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-11T07:34:20.309 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:12.696485+0000", 2024-09-11T07:34:20.309 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 7.4938756460000002, 2024-09-11T07:34:20.309 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.091098567000000005, 2024-09-11T07:34:20.309 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:20.309 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:20.309 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:20.309 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:34:20.309 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:34:20.309 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 63549 2024-09-11T07:34:20.309 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.309 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:20.309 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.310 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:20.310 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.696485+0000", 2024-09-11T07:34:20.310 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.310 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.310 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.310 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:20.310 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.696485+0000", 2024-09-11T07:34:20.310 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.310 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.310 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.310 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:20.310 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.696484+0000", 2024-09-11T07:34:20.310 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:34:20.311 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.311 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.311 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:20.311 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.696513+0000", 2024-09-11T07:34:20.311 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9009e-05 2024-09-11T07:34:20.311 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.311 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.311 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:20.311 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.696528+0000", 2024-09-11T07:34:20.311 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.483e-05 2024-09-11T07:34:20.311 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.311 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.311 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:20.311 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.696533+0000", 2024-09-11T07:34:20.312 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6439999999999997e-06 2024-09-11T07:34:20.312 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.312 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.312 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:20.312 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.702695+0000", 2024-09-11T07:34:20.312 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0061611790000000001 2024-09-11T07:34:20.312 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.312 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.312 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:20.312 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.702804+0000", 2024-09-11T07:34:20.312 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010943 2024-09-11T07:34:20.312 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.312 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.313 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:34:20.313 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.702940+0000", 2024-09-11T07:34:20.313 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000135938 2024-09-11T07:34:20.313 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.313 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.313 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:20.313 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.705639+0000", 2024-09-11T07:34:20.313 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002699019 2024-09-11T07:34:20.313 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.313 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.313 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.313 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.706887+0000", 2024-09-11T07:34:20.313 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0012476239999999999 2024-09-11T07:34:20.314 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.314 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.314 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.314 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.787529+0000", 2024-09-11T07:34:20.314 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.080642251999999998 2024-09-11T07:34:20.314 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.314 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.314 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:20.314 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.787561+0000", 2024-09-11T07:34:20.314 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1764000000000003e-05 2024-09-11T07:34:20.314 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.314 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.314 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:20.314 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.787584+0000", 2024-09-11T07:34:20.315 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3312e-05 2024-09-11T07:34:20.315 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.315 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:20.315 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.315 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.315 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.315 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:66849 8.8 8:17418fad:::benchmark_data_smithi022_108406_object66848:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-11T07:34:20.315 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:15.448364+0000", 2024-09-11T07:34:20.315 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 4.7419973649999996, 2024-09-11T07:34:20.315 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.102043175, 2024-09-11T07:34:20.315 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:20.315 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:20.315 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:20.316 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:34:20.316 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:34:20.316 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 66849 2024-09-11T07:34:20.316 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.316 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:20.316 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.316 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:20.316 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.448364+0000", 2024-09-11T07:34:20.316 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.316 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.316 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.317 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:20.317 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.448364+0000", 2024-09-11T07:34:20.317 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.317 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.317 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.317 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:20.317 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.448362+0000", 2024-09-11T07:34:20.317 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:34:20.317 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.317 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.317 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:20.317 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.448416+0000", 2024-09-11T07:34:20.317 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4364999999999999e-05 2024-09-11T07:34:20.317 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.318 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.318 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:20.318 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.448426+0000", 2024-09-11T07:34:20.318 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0129000000000001e-05 2024-09-11T07:34:20.318 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.318 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.318 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:20.318 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.448433+0000", 2024-09-11T07:34:20.318 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4590000000000003e-06 2024-09-11T07:34:20.318 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.318 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.318 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:20.319 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.448460+0000", 2024-09-11T07:34:20.319 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.7237999999999999e-05 2024-09-11T07:34:20.319 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.319 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.319 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:20.319 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.448549+0000", 2024-09-11T07:34:20.319 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8811000000000001e-05 2024-09-11T07:34:20.319 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.319 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.319 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:34:20.319 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.448668+0000", 2024-09-11T07:34:20.319 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000119191 2024-09-11T07:34:20.319 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.320 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.320 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.320 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.450969+0000", 2024-09-11T07:34:20.320 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002301432 2024-09-11T07:34:20.320 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.320 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.320 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:20.320 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.450976+0000", 2024-09-11T07:34:20.320 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1590000000000004e-06 2024-09-11T07:34:20.320 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.320 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.320 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.320 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.550242+0000", 2024-09-11T07:34:20.321 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.099265205999999995 2024-09-11T07:34:20.321 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.321 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.321 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:20.321 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.550326+0000", 2024-09-11T07:34:20.321 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4350000000000004e-05 2024-09-11T07:34:20.321 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.321 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.321 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:20.321 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.550407+0000", 2024-09-11T07:34:20.321 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0741999999999994e-05 2024-09-11T07:34:20.321 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.321 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:20.322 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.322 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.322 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.322 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:66854 8.8 8:16b7a44e:::benchmark_data_smithi022_108406_object66853:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-11T07:34:20.322 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:15.453699+0000", 2024-09-11T07:34:20.322 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 4.7366618339999995, 2024-09-11T07:34:20.322 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.096855826000000006, 2024-09-11T07:34:20.322 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:20.322 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:34:20.322 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:34:20.322 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:34:20.322 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:34:20.322 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 66854 2024-09-11T07:34:20.322 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.323 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:20.323 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.323 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:20.323 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.453699+0000", 2024-09-11T07:34:20.323 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.323 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.323 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.323 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:20.323 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.453699+0000", 2024-09-11T07:34:20.323 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:20.323 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.323 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.323 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:20.323 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.453698+0000", 2024-09-11T07:34:20.324 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:34:20.324 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.324 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.324 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:20.324 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.453788+0000", 2024-09-11T07:34:20.324 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0383000000000005e-05 2024-09-11T07:34:20.324 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.324 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.324 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:20.324 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.453799+0000", 2024-09-11T07:34:20.324 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1313e-05 2024-09-11T07:34:20.324 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.324 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.324 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:20.325 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.453804+0000", 2024-09-11T07:34:20.325 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4000000000000002e-06 2024-09-11T07:34:20.325 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.325 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.325 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:20.325 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.453848+0000", 2024-09-11T07:34:20.325 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4224999999999998e-05 2024-09-11T07:34:20.325 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.325 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.325 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:20.325 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.453971+0000", 2024-09-11T07:34:20.325 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012345800000000001 2024-09-11T07:34:20.325 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.326 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.326 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:34:20.326 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.454116+0000", 2024-09-11T07:34:20.326 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014493399999999999 2024-09-11T07:34:20.326 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.326 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.326 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:34:20.326 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.455962+0000", 2024-09-11T07:34:20.326 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001846077 2024-09-11T07:34:20.326 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.326 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.326 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.326 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.456168+0000", 2024-09-11T07:34:20.326 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00020587200000000001 2024-09-11T07:34:20.326 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.327 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.327 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:34:20.327 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.550504+0000", 2024-09-11T07:34:20.327 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.094335666999999998 2024-09-11T07:34:20.327 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.327 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.327 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:34:20.327 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.550534+0000", 2024-09-11T07:34:20.327 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9952000000000001e-05 2024-09-11T07:34:20.327 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:20.327 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:20.327 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:34:20.327 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.550555+0000", 2024-09-11T07:34:20.327 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1015000000000001e-05 2024-09-11T07:34:20.328 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.328 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:20.328 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.328 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:20.328 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:20.328 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:34:20.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:20.356 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:20.357 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:20.457 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:20.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:20.521 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:20.521 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:20.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:20 smithi104 ceph-mon[33507]: from='client.15237 v1:172.21.15.22:0/397161331' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:34:20.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:20 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:20.622 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:20.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:34:20 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:34:20] "GET /metrics HTTP/1.1" 200 224029 "" "Prometheus/2.43.0" 2024-09-11T07:34:20.672 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:20.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:20.674 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:20.775 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:20.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:20.837 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:20.837 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:20.841 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-09-11T07:34:18.101536+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4290582748,"num_objects":65656,"num_object_clones":0,"num_object_copies":196968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65656,"num_whiteouts":0,"num_read":1308,"num_read_kb":1165,"num_write":66003,"num_write_kb":4191111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":591166,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66103,"ondisk_log_size":66103,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":14894616,"kb_used_data":12426240,"kb_used_omap":16,"kb_used_meta":2468271,"kb_avail":735066600,"statfs":{"total":767960285184,"available":752708198400,"internally_reserved":0,"allocated":12724469760,"data_stored":12720240569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16530,"internal_metadata":2527510382},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[9,4,5,10,5,3,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":406257664,"num_objects":6199,"num_object_clones":0,"num_object_copies":18597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6199,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":6199,"num_write_kb":396736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.559510"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":181,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.556979+0000","last_change":"2024-09-11T07:33:08.464899+0000","last_active":"2024-09-11T07:34:10.556979+0000","last_peered":"2024-09-11T07:34:10.556979+0000","last_clean":"2024-09-11T07:34:10.556979+0000","last_became_active":"2024-09-11T07:32:00.501534+0000","last_became_peered":"2024-09-11T07:32:00.501534+0000","last_unstale":"2024-09-11T07:34:10.556979+0000","last_undegraded":"2024-09-11T07:34:10.556979+0000","last_fullsized":"2024-09-11T07:34:10.556979+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:08.464835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:08.464835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:24.698410+0000","scrub_duration":0.002103452,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":206,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.552147+0000","last_change":"2024-09-11T07:34:09.538289+0000","last_active":"2024-09-11T07:34:10.552147+0000","last_peered":"2024-09-11T07:34:10.552147+0000","last_clean":"2024-09-11T07:34:10.552147+0000","last_became_active":"2024-09-11T07:32:00.508572+0000","last_became_peered":"2024-09-11T07:32:00.508572+0000","last_unstale":"2024-09-11T07:34:10.552147+0000","last_undegraded":"2024-09-11T07:34:10.552147+0000","last_fullsized":"2024-09-11T07:34:10.552147+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:09.538205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:09.538205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:23.803280+0000","scrub_duration":0.0045320439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"82'22","reported_seq":210,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.556444+0000","last_change":"2024-09-11T07:33:53.615330+0000","last_active":"2024-09-11T07:34:10.556444+0000","last_peered":"2024-09-11T07:34:10.556444+0000","last_clean":"2024-09-11T07:34:10.556444+0000","last_became_active":"2024-09-11T07:26:33.544627+0000","last_became_peered":"2024-09-11T07:26:33.544627+0000","last_unstale":"2024-09-11T07:34:10.556444+0000","last_undegraded":"2024-09-11T07:34:10.556444+0000","last_fullsized":"2024-09-11T07:34:10.556444+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-09-11T07:33:53.615231+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:33:53.615231+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:22.109465+0000","scrub_duration":0.011220367,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":185,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.538495+0000","last_change":"2024-09-11T07:33:18.247208+0000","last_active":"2024-09-11T07:34:10.538495+0000","last_peered":"2024-09-11T07:34:10.538495+0000","last_clean":"2024-09-11T07:34:10.538495+0000","last_became_active":"2024-09-11T07:26:35.556296+0000","last_became_peered":"2024-09-11T07:26:35.556296+0000","last_unstale":"2024-09-11T07:34:10.538495+0000","last_undegraded":"2024-09-11T07:34:10.538495+0000","last_fullsized":"2024-09-11T07:34:10.538495+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:18.247138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:18.247138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:43.507240+0000","scrub_duration":0.001401198,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":180,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.557767+0000","last_change":"2024-09-11T07:31:39.275089+0000","last_active":"2024-09-11T07:34:10.557767+0000","last_peered":"2024-09-11T07:34:10.557767+0000","last_clean":"2024-09-11T07:34:10.557767+0000","last_became_active":"2024-09-11T07:26:31.420178+0000","last_became_peered":"2024-09-11T07:26:31.420178+0000","last_unstale":"2024-09-11T07:34:10.557767+0000","last_undegraded":"2024-09-11T07:34:10.557767+0000","last_fullsized":"2024-09-11T07:34:10.557767+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:31:39.275011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:31:39.275011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0095928969999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"86'13","reported_seq":197,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.538572+0000","last_change":"2024-09-11T07:31:33.998004+0000","last_active":"2024-09-11T07:34:10.538572+0000","last_peered":"2024-09-11T07:34:10.538572+0000","last_clean":"2024-09-11T07:34:10.538572+0000","last_became_active":"2024-09-11T07:26:33.540732+0000","last_became_peered":"2024-09-11T07:26:33.540732+0000","last_unstale":"2024-09-11T07:34:10.538572+0000","last_undegraded":"2024-09-11T07:34:10.538572+0000","last_fullsized":"2024-09-11T07:34:10.538572+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'13","last_scrub_stamp":"2024-09-11T07:31:33.997949+0000","last_deep_scrub":"86'13","last_deep_scrub_stamp":"2024-09-11T07:28:15.025757+0000","last_clean_scrub_stamp":"2024-09-11T07:31:33.997949+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016545030000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":194,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.544003+0000","last_change":"2024-09-11T07:33:49.237639+0000","last_active":"2024-09-11T07:34:10.544003+0000","last_peered":"2024-09-11T07:34:10.544003+0000","last_clean":"2024-09-11T07:34:10.544003+0000","last_became_active":"2024-09-11T07:32:00.513650+0000","last_became_peered":"2024-09-11T07:32:00.513650+0000","last_unstale":"2024-09-11T07:34:10.544003+0000","last_undegraded":"2024-09-11T07:34:10.544003+0000","last_fullsized":"2024-09-11T07:34:10.544003+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:49.237335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:33:49.237335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:54.939660+0000","scrub_duration":0.0052103979999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":173,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.539703+0000","last_change":"2024-09-11T07:31:49.018958+0000","last_active":"2024-09-11T07:34:10.539703+0000","last_peered":"2024-09-11T07:34:10.539703+0000","last_clean":"2024-09-11T07:34:10.539703+0000","last_became_active":"2024-09-11T07:26:37.566910+0000","last_became_peered":"2024-09-11T07:26:37.566910+0000","last_unstale":"2024-09-11T07:34:10.539703+0000","last_undegraded":"2024-09-11T07:34:10.539703+0000","last_fullsized":"2024-09-11T07:34:10.539703+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:31:49.018877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:31:49.018877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072333409999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":185,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.546793+0000","last_change":"2024-09-11T07:33:57.319606+0000","last_active":"2024-09-11T07:34:10.546793+0000","last_peered":"2024-09-11T07:34:10.546793+0000","last_clean":"2024-09-11T07:34:10.546793+0000","last_became_active":"2024-09-11T07:32:00.516662+0000","last_became_peered":"2024-09-11T07:32:00.516662+0000","last_unstale":"2024-09-11T07:34:10.546793+0000","last_undegraded":"2024-09-11T07:34:10.546793+0000","last_fullsized":"2024-09-11T07:34:10.546793+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:57.319449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:57.319449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:11.083627+0000","scrub_duration":0.007631344,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"82'7","reported_seq":199,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.540192+0000","last_change":"2024-09-11T07:31:43.035948+0000","last_active":"2024-09-11T07:34:10.540192+0000","last_peered":"2024-09-11T07:34:10.540192+0000","last_clean":"2024-09-11T07:34:10.540192+0000","last_became_active":"2024-09-11T07:26:33.540351+0000","last_became_peered":"2024-09-11T07:26:33.540351+0000","last_unstale":"2024-09-11T07:34:10.540192+0000","last_undegraded":"2024-09-11T07:34:10.540192+0000","last_fullsized":"2024-09-11T07:34:10.540192+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'7","last_scrub_stamp":"2024-09-11T07:31:43.035896+0000","last_deep_scrub":"82'7","last_deep_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_clean_scrub_stamp":"2024-09-11T07:31:43.035896+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040919600000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":151,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.544148+0000","last_change":"2024-09-11T07:33:07.124141+0000","last_active":"2024-09-11T07:34:10.544148+0000","last_peered":"2024-09-11T07:34:10.544148+0000","last_clean":"2024-09-11T07:34:10.544148+0000","last_became_active":"2024-09-11T07:32:00.513206+0000","last_became_peered":"2024-09-11T07:32:00.513206+0000","last_unstale":"2024-09-11T07:34:10.544148+0000","last_undegraded":"2024-09-11T07:34:10.544148+0000","last_fullsized":"2024-09-11T07:34:10.544148+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:07.124068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.973323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:07.124068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:29.008509+0000","scrub_duration":0.0019801810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":174,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.541903+0000","last_change":"2024-09-11T07:34:03.257034+0000","last_active":"2024-09-11T07:34:10.541903+0000","last_peered":"2024-09-11T07:34:10.541903+0000","last_clean":"2024-09-11T07:34:10.541903+0000","last_became_active":"2024-09-11T07:26:37.561758+0000","last_became_peered":"2024-09-11T07:26:37.561758+0000","last_unstale":"2024-09-11T07:34:10.541903+0000","last_undegraded":"2024-09-11T07:34:10.541903+0000","last_fullsized":"2024-09-11T07:34:10.541903+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:03.256953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:03.256953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:07.281155+0000","scrub_duration":0.0085144460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":150,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.542529+0000","last_change":"2024-09-11T07:34:01.257321+0000","last_active":"2024-09-11T07:34:10.542529+0000","last_peered":"2024-09-11T07:34:10.542529+0000","last_clean":"2024-09-11T07:34:10.542529+0000","last_became_active":"2024-09-11T07:32:00.505709+0000","last_became_peered":"2024-09-11T07:32:00.505709+0000","last_unstale":"2024-09-11T07:34:10.542529+0000","last_undegraded":"2024-09-11T07:34:10.542529+0000","last_fullsized":"2024-09-11T07:34:10.542529+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:01.257176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_clean_scrub_stamp":"2024-09-11T07:34:01.257176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:03.761234+0000","scrub_duration":0.0018917890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"136'13","reported_seq":207,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.539971+0000","last_change":"2024-09-11T07:34:06.406309+0000","last_active":"2024-09-11T07:34:10.539971+0000","last_peered":"2024-09-11T07:34:10.539971+0000","last_clean":"2024-09-11T07:34:10.539971+0000","last_became_active":"2024-09-11T07:26:33.544386+0000","last_became_peered":"2024-09-11T07:26:33.544386+0000","last_unstale":"2024-09-11T07:34:10.539971+0000","last_undegraded":"2024-09-11T07:34:10.539971+0000","last_fullsized":"2024-09-11T07:34:10.539971+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'13","last_scrub_stamp":"2024-09-11T07:34:06.406196+0000","last_deep_scrub":"136'13","last_deep_scrub_stamp":"2024-09-11T07:31:08.042198+0000","last_clean_scrub_stamp":"2024-09-11T07:34:06.406196+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:08.075604+0000","scrub_duration":0.0079557350000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":196,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.540991+0000","last_change":"2024-09-11T07:33:49.134068+0000","last_active":"2024-09-11T07:34:10.540991+0000","last_peered":"2024-09-11T07:34:10.540991+0000","last_clean":"2024-09-11T07:34:10.540991+0000","last_became_active":"2024-09-11T07:32:00.502337+0000","last_became_peered":"2024-09-11T07:32:00.502337+0000","last_unstale":"2024-09-11T07:34:10.540991+0000","last_undegraded":"2024-09-11T07:34:10.540991+0000","last_fullsized":"2024-09-11T07:34:10.540991+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:49.133995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:49.133995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:52.916415+0000","scrub_duration":0.0028580369999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":190,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.550619+0000","last_change":"2024-09-11T07:33:06.832611+0000","last_active":"2024-09-11T07:34:10.550619+0000","last_peered":"2024-09-11T07:34:10.550619+0000","last_clean":"2024-09-11T07:34:10.550619+0000","last_became_active":"2024-09-11T07:32:00.530335+0000","last_became_peered":"2024-09-11T07:32:00.530335+0000","last_unstale":"2024-09-11T07:34:10.550619+0000","last_undegraded":"2024-09-11T07:34:10.550619+0000","last_fullsized":"2024-09-11T07:34:10.550619+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:06.832540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:06.832540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020659680000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"198'4181","reported_seq":4209,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:14.195306+0000","last_change":"2024-09-11T07:33:29.155481+0000","last_active":"2024-09-11T07:34:14.195306+0000","last_peered":"2024-09-11T07:34:14.195306+0000","last_clean":"2024-09-11T07:34:14.195306+0000","last_became_active":"2024-09-11T07:33:29.155187+0000","last_became_peered":"2024-09-11T07:33:29.155187+0000","last_unstale":"2024-09-11T07:34:14.195306+0000","last_undegraded":"2024-09-11T07:34:14.195306+0000","last_fullsized":"2024-09-11T07:34:14.195306+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4181,"log_dups_size":0,"ondisk_log_size":4181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:56.196806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":274006016,"num_objects":4181,"num_object_clones":0,"num_object_copies":12543,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4181,"num_write_kb":267584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":271,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:11.315591+0000","last_change":"2024-09-11T07:34:11.315441+0000","last_active":"2024-09-11T07:34:11.315591+0000","last_peered":"2024-09-11T07:34:11.315591+0000","last_clean":"2024-09-11T07:34:11.315591+0000","last_became_active":"2024-09-11T07:29:51.482755+0000","last_became_peered":"2024-09-11T07:29:51.482755+0000","last_unstale":"2024-09-11T07:34:11.315591+0000","last_undegraded":"2024-09-11T07:34:11.315591+0000","last_fullsized":"2024-09-11T07:34:11.315591+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-09-11T07:34:11.315334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:34:11.315334+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:17.690755+0000","scrub_duration":0.0071604249999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":180,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.545340+0000","last_change":"2024-09-11T07:33:52.272651+0000","last_active":"2024-09-11T07:34:10.545340+0000","last_peered":"2024-09-11T07:34:10.545340+0000","last_clean":"2024-09-11T07:34:10.545340+0000","last_became_active":"2024-09-11T07:32:00.507191+0000","last_became_peered":"2024-09-11T07:32:00.507191+0000","last_unstale":"2024-09-11T07:34:10.545340+0000","last_undegraded":"2024-09-11T07:34:10.545340+0000","last_fullsized":"2024-09-11T07:34:10.545340+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:52.272573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:52.272573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:17.508902+0000","scrub_duration":0.0023059880000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":184,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.543790+0000","last_change":"2024-09-11T07:34:00.348790+0000","last_active":"2024-09-11T07:34:10.543790+0000","last_peered":"2024-09-11T07:34:10.543790+0000","last_clean":"2024-09-11T07:34:10.543790+0000","last_became_active":"2024-09-11T07:26:35.557989+0000","last_became_peered":"2024-09-11T07:26:35.557989+0000","last_unstale":"2024-09-11T07:34:10.543790+0000","last_undegraded":"2024-09-11T07:34:10.543790+0000","last_fullsized":"2024-09-11T07:34:10.543790+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:00.348684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:00.348684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:30.171094+0000","scrub_duration":0.0036577580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":184,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.538640+0000","last_change":"2024-09-11T07:33:57.219099+0000","last_active":"2024-09-11T07:34:10.538640+0000","last_peered":"2024-09-11T07:34:10.538640+0000","last_clean":"2024-09-11T07:34:10.538640+0000","last_became_active":"2024-09-11T07:26:37.564554+0000","last_became_peered":"2024-09-11T07:26:37.564554+0000","last_unstale":"2024-09-11T07:34:10.538640+0000","last_undegraded":"2024-09-11T07:34:10.538640+0000","last_fullsized":"2024-09-11T07:34:10.538640+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:57.219012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:57.219012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:26.433401+0000","scrub_duration":0.0062201569999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"198'4094","reported_seq":4123,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:14.559665+0000","last_change":"2024-09-11T07:33:29.161604+0000","last_active":"2024-09-11T07:34:14.559665+0000","last_peered":"2024-09-11T07:34:14.559665+0000","last_clean":"2024-09-11T07:34:14.559665+0000","last_became_active":"2024-09-11T07:33:29.161349+0000","last_became_peered":"2024-09-11T07:33:29.161349+0000","last_unstale":"2024-09-11T07:34:14.559665+0000","last_undegraded":"2024-09-11T07:34:14.559665+0000","last_fullsized":"2024-09-11T07:34:14.559665+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4094,"log_dups_size":0,"ondisk_log_size":4094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:37.804059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268304384,"num_objects":4094,"num_object_clones":0,"num_object_copies":12282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4094,"num_write_kb":262016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"112'15","reported_seq":207,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.539484+0000","last_change":"2024-09-11T07:34:08.373729+0000","last_active":"2024-09-11T07:34:10.539484+0000","last_peered":"2024-09-11T07:34:10.539484+0000","last_clean":"2024-09-11T07:34:10.539484+0000","last_became_active":"2024-09-11T07:26:33.535411+0000","last_became_peered":"2024-09-11T07:26:33.535411+0000","last_unstale":"2024-09-11T07:34:10.539484+0000","last_undegraded":"2024-09-11T07:34:10.539484+0000","last_fullsized":"2024-09-11T07:34:10.539484+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"112'15","last_scrub_stamp":"2024-09-11T07:34:08.373652+0000","last_deep_scrub":"112'15","last_deep_scrub_stamp":"2024-09-11T07:31:10.005896+0000","last_clean_scrub_stamp":"2024-09-11T07:34:08.373652+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:35.031237+0000","scrub_duration":0.0048197400000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":198,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.548555+0000","last_change":"2024-09-11T07:33:50.250361+0000","last_active":"2024-09-11T07:34:10.548555+0000","last_peered":"2024-09-11T07:34:10.548555+0000","last_clean":"2024-09-11T07:34:10.548555+0000","last_became_active":"2024-09-11T07:32:00.506615+0000","last_became_peered":"2024-09-11T07:32:00.506615+0000","last_unstale":"2024-09-11T07:34:10.548555+0000","last_undegraded":"2024-09-11T07:34:10.548555+0000","last_fullsized":"2024-09-11T07:34:10.548555+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:50.250289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:50.250289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:01.601495+0000","scrub_duration":0.0080459799999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"151'10","reported_seq":295,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.539780+0000","last_change":"2024-09-11T07:33:01.124548+0000","last_active":"2024-09-11T07:34:10.539780+0000","last_peered":"2024-09-11T07:34:10.539780+0000","last_clean":"2024-09-11T07:34:10.539780+0000","last_became_active":"2024-09-11T07:32:00.507447+0000","last_became_peered":"2024-09-11T07:32:00.507447+0000","last_unstale":"2024-09-11T07:34:10.539780+0000","last_undegraded":"2024-09-11T07:34:10.539780+0000","last_fullsized":"2024-09-11T07:34:10.539780+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:33:01.124483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:01.124483+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020248890000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":173,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.538876+0000","last_change":"2024-09-11T07:31:44.075858+0000","last_active":"2024-09-11T07:34:10.538876+0000","last_peered":"2024-09-11T07:34:10.538876+0000","last_clean":"2024-09-11T07:34:10.538876+0000","last_became_active":"2024-09-11T07:26:37.566919+0000","last_became_peered":"2024-09-11T07:26:37.566919+0000","last_unstale":"2024-09-11T07:34:10.538876+0000","last_undegraded":"2024-09-11T07:34:10.538876+0000","last_fullsized":"2024-09-11T07:34:10.538876+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:31:44.075772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:31:44.075772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053001849999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"198'4055","reported_seq":4083,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:14.173156+0000","last_change":"2024-09-11T07:33:29.154369+0000","last_active":"2024-09-11T07:34:14.173156+0000","last_peered":"2024-09-11T07:34:14.173156+0000","last_clean":"2024-09-11T07:34:14.173156+0000","last_became_active":"2024-09-11T07:33:29.154103+0000","last_became_peered":"2024-09-11T07:33:29.154103+0000","last_unstale":"2024-09-11T07:34:14.173156+0000","last_undegraded":"2024-09-11T07:34:14.173156+0000","last_fullsized":"2024-09-11T07:34:14.173156+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4055,"log_dups_size":0,"ondisk_log_size":4055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:29.121899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":265748480,"num_objects":4055,"num_object_clones":0,"num_object_copies":12165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4055,"num_write_kb":259520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"83'22","reported_seq":218,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.546057+0000","last_change":"2024-09-11T07:34:04.375654+0000","last_active":"2024-09-11T07:34:10.546057+0000","last_peered":"2024-09-11T07:34:10.546057+0000","last_clean":"2024-09-11T07:34:10.546057+0000","last_became_active":"2024-09-11T07:26:33.549733+0000","last_became_peered":"2024-09-11T07:26:33.549733+0000","last_unstale":"2024-09-11T07:34:10.546057+0000","last_undegraded":"2024-09-11T07:34:10.546057+0000","last_fullsized":"2024-09-11T07:34:10.546057+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'22","last_scrub_stamp":"2024-09-11T07:34:04.375457+0000","last_deep_scrub":"83'22","last_deep_scrub_stamp":"2024-09-11T07:34:04.375457+0000","last_clean_scrub_stamp":"2024-09-11T07:34:04.375457+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:29.572580+0000","scrub_duration":0.0086024310000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":208,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.544167+0000","last_change":"2024-09-11T07:33:59.343890+0000","last_active":"2024-09-11T07:34:10.544167+0000","last_peered":"2024-09-11T07:34:10.544167+0000","last_clean":"2024-09-11T07:34:10.544167+0000","last_became_active":"2024-09-11T07:32:00.513472+0000","last_became_peered":"2024-09-11T07:32:00.513472+0000","last_unstale":"2024-09-11T07:34:10.544167+0000","last_undegraded":"2024-09-11T07:34:10.544167+0000","last_fullsized":"2024-09-11T07:34:10.544167+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:59.343676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:59.343676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:04.415703+0000","scrub_duration":0.0067031579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":278,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.539033+0000","last_change":"2024-09-11T07:31:33.055929+0000","last_active":"2024-09-11T07:34:10.539033+0000","last_peered":"2024-09-11T07:34:10.539033+0000","last_clean":"2024-09-11T07:34:10.539033+0000","last_became_active":"2024-09-11T07:26:35.560406+0000","last_became_peered":"2024-09-11T07:26:35.560406+0000","last_unstale":"2024-09-11T07:34:10.539033+0000","last_undegraded":"2024-09-11T07:34:10.539033+0000","last_fullsized":"2024-09-11T07:34:10.539033+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:31:33.055836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:31:33.055836+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010749418,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":174,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.547083+0000","last_change":"2024-09-11T07:33:07.153745+0000","last_active":"2024-09-11T07:34:10.547083+0000","last_peered":"2024-09-11T07:34:10.547083+0000","last_clean":"2024-09-11T07:34:10.547083+0000","last_became_active":"2024-09-11T07:26:37.568005+0000","last_became_peered":"2024-09-11T07:26:37.568005+0000","last_unstale":"2024-09-11T07:34:10.547083+0000","last_undegraded":"2024-09-11T07:34:10.547083+0000","last_fullsized":"2024-09-11T07:34:10.547083+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:07.153674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:31.200685+0000","last_clean_scrub_stamp":"2024-09-11T07:33:07.153674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:20.217755+0000","scrub_duration":0.001744982,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"198'4060","reported_seq":4088,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:13.884164+0000","last_change":"2024-09-11T07:33:29.154414+0000","last_active":"2024-09-11T07:34:13.884164+0000","last_peered":"2024-09-11T07:34:13.884164+0000","last_clean":"2024-09-11T07:34:13.884164+0000","last_became_active":"2024-09-11T07:33:29.154117+0000","last_became_peered":"2024-09-11T07:33:29.154117+0000","last_unstale":"2024-09-11T07:34:13.884164+0000","last_undegraded":"2024-09-11T07:34:13.884164+0000","last_fullsized":"2024-09-11T07:34:13.884164+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4060,"log_dups_size":0,"ondisk_log_size":4060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:37.755024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":266076160,"num_objects":4060,"num_object_clones":0,"num_object_copies":12180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4060,"num_write_kb":259840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"107'16","reported_seq":211,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.539116+0000","last_change":"2024-09-11T07:33:15.278908+0000","last_active":"2024-09-11T07:34:10.539116+0000","last_peered":"2024-09-11T07:34:10.539116+0000","last_clean":"2024-09-11T07:34:10.539116+0000","last_became_active":"2024-09-11T07:26:33.540777+0000","last_became_peered":"2024-09-11T07:26:33.540777+0000","last_unstale":"2024-09-11T07:34:10.539116+0000","last_undegraded":"2024-09-11T07:34:10.539116+0000","last_fullsized":"2024-09-11T07:34:10.539116+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'16","last_scrub_stamp":"2024-09-11T07:33:15.278856+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-09-11T07:29:38.112111+0000","last_clean_scrub_stamp":"2024-09-11T07:33:15.278856+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:17.908263+0000","scrub_duration":0.0016764620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":174,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.543659+0000","last_change":"2024-09-11T07:33:17.262636+0000","last_active":"2024-09-11T07:34:10.543659+0000","last_peered":"2024-09-11T07:34:10.543659+0000","last_clean":"2024-09-11T07:34:10.543659+0000","last_became_active":"2024-09-11T07:32:00.518538+0000","last_became_peered":"2024-09-11T07:32:00.518538+0000","last_unstale":"2024-09-11T07:34:10.543659+0000","last_undegraded":"2024-09-11T07:34:10.543659+0000","last_fullsized":"2024-09-11T07:34:10.543659+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:17.262578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:17.262578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:32.326642+0000","scrub_duration":0.003026361,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":186,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.542661+0000","last_change":"2024-09-11T07:33:10.096952+0000","last_active":"2024-09-11T07:34:10.542661+0000","last_peered":"2024-09-11T07:34:10.542661+0000","last_clean":"2024-09-11T07:34:10.542661+0000","last_became_active":"2024-09-11T07:26:35.569081+0000","last_became_peered":"2024-09-11T07:26:35.569081+0000","last_unstale":"2024-09-11T07:34:10.542661+0000","last_undegraded":"2024-09-11T07:34:10.542661+0000","last_fullsized":"2024-09-11T07:34:10.542661+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:10.096886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:10.096886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:32.795036+0000","scrub_duration":0.0032884910000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":174,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.559089+0000","last_change":"2024-09-11T07:31:45.261381+0000","last_active":"2024-09-11T07:34:10.559089+0000","last_peered":"2024-09-11T07:34:10.559089+0000","last_clean":"2024-09-11T07:34:10.559089+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:34:10.559089+0000","last_undegraded":"2024-09-11T07:34:10.559089+0000","last_fullsized":"2024-09-11T07:34:10.559089+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:31:45.261282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:31:45.261282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086095340000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"198'4084","reported_seq":4113,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:13.826712+0000","last_change":"2024-09-11T07:33:29.205385+0000","last_active":"2024-09-11T07:34:13.826712+0000","last_peered":"2024-09-11T07:34:13.826712+0000","last_clean":"2024-09-11T07:34:13.826712+0000","last_became_active":"2024-09-11T07:33:29.205097+0000","last_became_peered":"2024-09-11T07:33:29.205097+0000","last_unstale":"2024-09-11T07:34:13.826712+0000","last_undegraded":"2024-09-11T07:34:13.826712+0000","last_fullsized":"2024-09-11T07:34:13.826712+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4084,"log_dups_size":0,"ondisk_log_size":4084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:54.401575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":267649024,"num_objects":4084,"num_object_clones":0,"num_object_copies":12252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4084,"num_write_kb":261376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"64'14","reported_seq":198,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.538610+0000","last_change":"2024-09-11T07:31:46.078940+0000","last_active":"2024-09-11T07:34:10.538610+0000","last_peered":"2024-09-11T07:34:10.538610+0000","last_clean":"2024-09-11T07:34:10.538610+0000","last_became_active":"2024-09-11T07:26:33.537850+0000","last_became_peered":"2024-09-11T07:26:33.537850+0000","last_unstale":"2024-09-11T07:34:10.538610+0000","last_undegraded":"2024-09-11T07:34:10.538610+0000","last_fullsized":"2024-09-11T07:34:10.538610+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-09-11T07:31:46.078888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:31:46.078888+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017755819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":200,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.545324+0000","last_change":"2024-09-11T07:33:04.366549+0000","last_active":"2024-09-11T07:34:10.545324+0000","last_peered":"2024-09-11T07:34:10.545324+0000","last_clean":"2024-09-11T07:34:10.545324+0000","last_became_active":"2024-09-11T07:28:09.014995+0000","last_became_peered":"2024-09-11T07:28:09.014995+0000","last_unstale":"2024-09-11T07:34:10.545324+0000","last_undegraded":"2024-09-11T07:34:10.545324+0000","last_fullsized":"2024-09-11T07:34:10.545324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:04.366473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:04.366473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:28.386907+0000","scrub_duration":0.001832272,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"84'9","reported_seq":277,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.562955+0000","last_change":"2024-09-11T07:33:48.510872+0000","last_active":"2024-09-11T07:34:10.562955+0000","last_peered":"2024-09-11T07:34:10.562955+0000","last_clean":"2024-09-11T07:34:10.562955+0000","last_became_active":"2024-09-11T07:26:35.562997+0000","last_became_peered":"2024-09-11T07:26:35.562997+0000","last_unstale":"2024-09-11T07:34:10.562955+0000","last_undegraded":"2024-09-11T07:34:10.562955+0000","last_fullsized":"2024-09-11T07:34:10.562955+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-11T07:33:48.510778+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-11T07:28:05.653237+0000","last_clean_scrub_stamp":"2024-09-11T07:33:48.510778+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:52.240234+0000","scrub_duration":0.0038375390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":148,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.544591+0000","last_change":"2024-09-11T07:32:58.087650+0000","last_active":"2024-09-11T07:34:10.544591+0000","last_peered":"2024-09-11T07:34:10.544591+0000","last_clean":"2024-09-11T07:34:10.544591+0000","last_became_active":"2024-09-11T07:32:00.511005+0000","last_became_peered":"2024-09-11T07:32:00.511005+0000","last_unstale":"2024-09-11T07:34:10.544591+0000","last_undegraded":"2024-09-11T07:34:10.544591+0000","last_fullsized":"2024-09-11T07:34:10.544591+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:32:58.087595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:32:58.087595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001226354,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"198'4134","reported_seq":4160,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:14.565804+0000","last_change":"2024-09-11T07:33:29.205044+0000","last_active":"2024-09-11T07:34:14.565804+0000","last_peered":"2024-09-11T07:34:14.565804+0000","last_clean":"2024-09-11T07:34:14.565804+0000","last_became_active":"2024-09-11T07:33:29.204734+0000","last_became_peered":"2024-09-11T07:33:29.204734+0000","last_unstale":"2024-09-11T07:34:14.565804+0000","last_undegraded":"2024-09-11T07:34:14.565804+0000","last_fullsized":"2024-09-11T07:34:14.565804+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4134,"log_dups_size":0,"ondisk_log_size":4134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:49.210405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270925824,"num_objects":4134,"num_object_clones":0,"num_object_copies":12402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4134,"num_write_kb":264576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"87'26","reported_seq":235,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.563132+0000","last_change":"2024-09-11T07:33:52.609653+0000","last_active":"2024-09-11T07:34:10.563132+0000","last_peered":"2024-09-11T07:34:10.563132+0000","last_clean":"2024-09-11T07:34:10.563132+0000","last_became_active":"2024-09-11T07:32:00.520670+0000","last_became_peered":"2024-09-11T07:32:00.520670+0000","last_unstale":"2024-09-11T07:34:10.563132+0000","last_undegraded":"2024-09-11T07:34:10.563132+0000","last_fullsized":"2024-09-11T07:34:10.563132+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'26","last_scrub_stamp":"2024-09-11T07:33:52.609336+0000","last_deep_scrub":"87'26","last_deep_scrub_stamp":"2024-09-11T07:28:30.788647+0000","last_clean_scrub_stamp":"2024-09-11T07:33:52.609336+0000","objects_scrubbed":8,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:15.124860+0000","scrub_duration":0.010453553000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":188,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.544459+0000","last_change":"2024-09-11T07:33:07.547329+0000","last_active":"2024-09-11T07:34:10.544459+0000","last_peered":"2024-09-11T07:34:10.544459+0000","last_clean":"2024-09-11T07:34:10.544459+0000","last_became_active":"2024-09-11T07:26:31.421500+0000","last_became_peered":"2024-09-11T07:26:31.421500+0000","last_unstale":"2024-09-11T07:34:10.544459+0000","last_undegraded":"2024-09-11T07:34:10.544459+0000","last_fullsized":"2024-09-11T07:34:10.544459+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:07.547259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:07.547259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:34.442417+0000","scrub_duration":0.0019369229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":195,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.538490+0000","last_change":"2024-09-11T07:33:53.449765+0000","last_active":"2024-09-11T07:34:10.538490+0000","last_peered":"2024-09-11T07:34:10.538490+0000","last_clean":"2024-09-11T07:34:10.538490+0000","last_became_active":"2024-09-11T07:32:00.499747+0000","last_became_peered":"2024-09-11T07:32:00.499747+0000","last_unstale":"2024-09-11T07:34:10.538490+0000","last_undegraded":"2024-09-11T07:34:10.538490+0000","last_fullsized":"2024-09-11T07:34:10.538490+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:53.449689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:27.968579+0000","last_clean_scrub_stamp":"2024-09-11T07:33:53.449689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:12.260963+0000","scrub_duration":0.0033513269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":173,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.539904+0000","last_change":"2024-09-11T07:31:41.033292+0000","last_active":"2024-09-11T07:34:10.539904+0000","last_peered":"2024-09-11T07:34:10.539904+0000","last_clean":"2024-09-11T07:34:10.539904+0000","last_became_active":"2024-09-11T07:26:37.564616+0000","last_became_peered":"2024-09-11T07:26:37.564616+0000","last_unstale":"2024-09-11T07:34:10.539904+0000","last_undegraded":"2024-09-11T07:34:10.539904+0000","last_fullsized":"2024-09-11T07:34:10.539904+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:31:41.033237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:29.978798+0000","last_clean_scrub_stamp":"2024-09-11T07:31:41.033237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014647854,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"198'4077","reported_seq":4105,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:14.569710+0000","last_change":"2024-09-11T07:33:29.161660+0000","last_active":"2024-09-11T07:34:14.569710+0000","last_peered":"2024-09-11T07:34:14.569710+0000","last_clean":"2024-09-11T07:34:14.569710+0000","last_became_active":"2024-09-11T07:33:29.161212+0000","last_became_peered":"2024-09-11T07:33:29.161212+0000","last_unstale":"2024-09-11T07:34:14.569710+0000","last_undegraded":"2024-09-11T07:34:14.569710+0000","last_fullsized":"2024-09-11T07:34:14.569710+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4077,"log_dups_size":0,"ondisk_log_size":4077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:50.971554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":267190272,"num_objects":4077,"num_object_clones":0,"num_object_copies":12231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4077,"num_write_kb":260928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":230,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.551346+0000","last_change":"2024-09-11T07:33:10.802985+0000","last_active":"2024-09-11T07:34:10.551346+0000","last_peered":"2024-09-11T07:34:10.551346+0000","last_clean":"2024-09-11T07:34:10.551346+0000","last_became_active":"2024-09-11T07:32:00.506270+0000","last_became_peered":"2024-09-11T07:32:00.506270+0000","last_unstale":"2024-09-11T07:34:10.551346+0000","last_undegraded":"2024-09-11T07:34:10.551346+0000","last_fullsized":"2024-09-11T07:34:10.551346+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-11T07:33:10.802922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:10.802922+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:12.792466+0000","scrub_duration":0.002454899,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":148,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.544385+0000","last_change":"2024-09-11T07:33:13.291210+0000","last_active":"2024-09-11T07:34:10.544385+0000","last_peered":"2024-09-11T07:34:10.544385+0000","last_clean":"2024-09-11T07:34:10.544385+0000","last_became_active":"2024-09-11T07:32:00.510631+0000","last_became_peered":"2024-09-11T07:32:00.510631+0000","last_unstale":"2024-09-11T07:34:10.544385+0000","last_undegraded":"2024-09-11T07:34:10.544385+0000","last_fullsized":"2024-09-11T07:34:10.544385+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:13.291147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:13.291147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:29.681159+0000","scrub_duration":0.0036274660000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":186,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.544135+0000","last_change":"2024-09-11T07:33:01.281595+0000","last_active":"2024-09-11T07:34:10.544135+0000","last_peered":"2024-09-11T07:34:10.544135+0000","last_clean":"2024-09-11T07:34:10.544135+0000","last_became_active":"2024-09-11T07:26:35.565359+0000","last_became_peered":"2024-09-11T07:26:35.565359+0000","last_unstale":"2024-09-11T07:34:10.544135+0000","last_undegraded":"2024-09-11T07:34:10.544135+0000","last_fullsized":"2024-09-11T07:34:10.544135+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:01.281530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:01.281530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:22.674857+0000","scrub_duration":0.00138591,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":182,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.543931+0000","last_change":"2024-09-11T07:34:10.294762+0000","last_active":"2024-09-11T07:34:10.543931+0000","last_peered":"2024-09-11T07:34:10.543931+0000","last_clean":"2024-09-11T07:34:10.543931+0000","last_became_active":"2024-09-11T07:26:37.861279+0000","last_became_peered":"2024-09-11T07:26:37.861279+0000","last_unstale":"2024-09-11T07:34:10.543931+0000","last_undegraded":"2024-09-11T07:34:10.543931+0000","last_fullsized":"2024-09-11T07:34:10.543931+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:10.294658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:39.723598+0000","last_clean_scrub_stamp":"2024-09-11T07:34:10.294658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:25.940771+0000","scrub_duration":0.0026992600000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.e","version":"198'4102","reported_seq":4130,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:14.573826+0000","last_change":"2024-09-11T07:33:29.159841+0000","last_active":"2024-09-11T07:34:14.573826+0000","last_peered":"2024-09-11T07:34:14.573826+0000","last_clean":"2024-09-11T07:34:14.573826+0000","last_became_active":"2024-09-11T07:33:29.159535+0000","last_became_peered":"2024-09-11T07:33:29.159535+0000","last_unstale":"2024-09-11T07:34:14.573826+0000","last_undegraded":"2024-09-11T07:34:14.573826+0000","last_fullsized":"2024-09-11T07:34:14.573826+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4102,"log_dups_size":0,"ondisk_log_size":4102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:54.041505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268828672,"num_objects":4102,"num_object_clones":0,"num_object_copies":12306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4102,"num_write_kb":262528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":204,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.548420+0000","last_change":"2024-09-11T07:33:53.294408+0000","last_active":"2024-09-11T07:34:10.548420+0000","last_peered":"2024-09-11T07:34:10.548420+0000","last_clean":"2024-09-11T07:34:10.548420+0000","last_became_active":"2024-09-11T07:26:33.543063+0000","last_became_peered":"2024-09-11T07:26:33.543063+0000","last_unstale":"2024-09-11T07:34:10.548420+0000","last_undegraded":"2024-09-11T07:34:10.548420+0000","last_fullsized":"2024-09-11T07:34:10.548420+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:33:53.294174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:53.294174+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:17.014241+0000","scrub_duration":0.0074369980000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":215,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.543925+0000","last_change":"2024-09-11T07:33:55.344919+0000","last_active":"2024-09-11T07:34:10.543925+0000","last_peered":"2024-09-11T07:34:10.543925+0000","last_clean":"2024-09-11T07:34:10.543925+0000","last_became_active":"2024-09-11T07:32:00.522456+0000","last_became_peered":"2024-09-11T07:32:00.522456+0000","last_unstale":"2024-09-11T07:34:10.543925+0000","last_undegraded":"2024-09-11T07:34:10.543925+0000","last_fullsized":"2024-09-11T07:34:10.543925+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:33:55.344823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:55.344823+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:19.214723+0000","scrub_duration":0.0025904349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":286,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.594309+0000","last_change":"2024-09-11T07:33:50.686006+0000","last_active":"2024-09-11T07:34:12.594309+0000","last_peered":"2024-09-11T07:34:12.594309+0000","last_clean":"2024-09-11T07:34:12.594309+0000","last_became_active":"2024-09-11T07:26:35.560422+0000","last_became_peered":"2024-09-11T07:26:35.560422+0000","last_unstale":"2024-09-11T07:34:12.594309+0000","last_undegraded":"2024-09-11T07:34:12.594309+0000","last_fullsized":"2024-09-11T07:34:12.594309+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:33:50.685923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:50.685923+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:54.623978+0000","scrub_duration":0.0029026490000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":183,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.550471+0000","last_change":"2024-09-11T07:33:16.738699+0000","last_active":"2024-09-11T07:34:10.550471+0000","last_peered":"2024-09-11T07:34:10.550471+0000","last_clean":"2024-09-11T07:34:10.550471+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:34:10.550471+0000","last_undegraded":"2024-09-11T07:34:10.550471+0000","last_fullsized":"2024-09-11T07:34:10.550471+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:16.738631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_clean_scrub_stamp":"2024-09-11T07:33:16.738631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:20.995420+0000","scrub_duration":0.0015950739999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"198'4041","reported_seq":4070,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:13.900846+0000","last_change":"2024-09-11T07:33:29.205462+0000","last_active":"2024-09-11T07:34:13.900846+0000","last_peered":"2024-09-11T07:34:13.900846+0000","last_clean":"2024-09-11T07:34:13.900846+0000","last_became_active":"2024-09-11T07:33:29.205148+0000","last_became_peered":"2024-09-11T07:33:29.205148+0000","last_unstale":"2024-09-11T07:34:13.900846+0000","last_undegraded":"2024-09-11T07:34:13.900846+0000","last_fullsized":"2024-09-11T07:34:13.900846+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4041,"log_dups_size":0,"ondisk_log_size":4041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:31.106281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264830976,"num_objects":4041,"num_object_clones":0,"num_object_copies":12123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4041,"num_write_kb":258624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"170'29","reported_seq":74,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.542398+0000","last_change":"2024-09-11T07:33:10.026568+0000","last_active":"2024-09-11T07:34:10.542398+0000","last_peered":"2024-09-11T07:34:10.542398+0000","last_clean":"2024-09-11T07:34:10.542398+0000","last_became_active":"2024-09-11T07:33:10.017961+0000","last_became_peered":"2024-09-11T07:33:10.017961+0000","last_unstale":"2024-09-11T07:34:10.542398+0000","last_undegraded":"2024-09-11T07:34:10.542398+0000","last_fullsized":"2024-09-11T07:34:10.542398+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'28","last_scrub_stamp":"2024-09-11T07:31:47.592158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:31:47.592158+0000","objects_scrubbed":10,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052388820000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":190,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.541803+0000","last_change":"2024-09-11T07:33:12.133334+0000","last_active":"2024-09-11T07:34:10.541803+0000","last_peered":"2024-09-11T07:34:10.541803+0000","last_clean":"2024-09-11T07:34:10.541803+0000","last_became_active":"2024-09-11T07:26:31.424618+0000","last_became_peered":"2024-09-11T07:26:31.424618+0000","last_unstale":"2024-09-11T07:34:10.541803+0000","last_undegraded":"2024-09-11T07:34:10.541803+0000","last_fullsized":"2024-09-11T07:34:10.541803+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:12.133258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:12.775067+0000","last_clean_scrub_stamp":"2024-09-11T07:33:12.133258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:13.157614+0000","scrub_duration":0.0013676669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":185,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.551739+0000","last_change":"2024-09-11T07:33:52.629961+0000","last_active":"2024-09-11T07:34:10.551739+0000","last_peered":"2024-09-11T07:34:10.551739+0000","last_clean":"2024-09-11T07:34:10.551739+0000","last_became_active":"2024-09-11T07:26:35.560363+0000","last_became_peered":"2024-09-11T07:26:35.560363+0000","last_unstale":"2024-09-11T07:34:10.551739+0000","last_undegraded":"2024-09-11T07:34:10.551739+0000","last_fullsized":"2024-09-11T07:34:10.551739+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:52.629872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:52.629872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:03.346424+0000","scrub_duration":0.003860858,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":191,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.557589+0000","last_change":"2024-09-11T07:33:13.566605+0000","last_active":"2024-09-11T07:34:10.557589+0000","last_peered":"2024-09-11T07:34:10.557589+0000","last_clean":"2024-09-11T07:34:10.557589+0000","last_became_active":"2024-09-11T07:32:00.505919+0000","last_became_peered":"2024-09-11T07:32:00.505919+0000","last_unstale":"2024-09-11T07:34:10.557589+0000","last_undegraded":"2024-09-11T07:34:10.557589+0000","last_fullsized":"2024-09-11T07:34:10.557589+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:13.566551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:13.566551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:23.787271+0000","scrub_duration":0.0015767380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"198'4172","reported_seq":4201,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:14.551861+0000","last_change":"2024-09-11T07:33:29.161446+0000","last_active":"2024-09-11T07:34:14.551861+0000","last_peered":"2024-09-11T07:34:14.551861+0000","last_clean":"2024-09-11T07:34:14.551861+0000","last_became_active":"2024-09-11T07:33:29.161141+0000","last_became_peered":"2024-09-11T07:33:29.161141+0000","last_unstale":"2024-09-11T07:34:14.551861+0000","last_undegraded":"2024-09-11T07:34:14.551861+0000","last_fullsized":"2024-09-11T07:34:14.551861+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4172,"log_dups_size":0,"ondisk_log_size":4172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:31.128219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":273416192,"num_objects":4172,"num_object_clones":0,"num_object_copies":12516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4172,"num_write_kb":267008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"144'18","reported_seq":218,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.546142+0000","last_change":"2024-09-11T07:34:03.365979+0000","last_active":"2024-09-11T07:34:10.546142+0000","last_peered":"2024-09-11T07:34:10.546142+0000","last_clean":"2024-09-11T07:34:10.546142+0000","last_became_active":"2024-09-11T07:26:33.546126+0000","last_became_peered":"2024-09-11T07:26:33.546126+0000","last_unstale":"2024-09-11T07:34:10.546142+0000","last_undegraded":"2024-09-11T07:34:10.546142+0000","last_fullsized":"2024-09-11T07:34:10.546142+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"144'18","last_scrub_stamp":"2024-09-11T07:34:03.365891+0000","last_deep_scrub":"144'18","last_deep_scrub_stamp":"2024-09-11T07:31:51.603378+0000","last_clean_scrub_stamp":"2024-09-11T07:34:03.365891+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:30.450782+0000","scrub_duration":0.006530999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":651,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.595583+0000","last_change":"2024-09-11T07:33:14.474365+0000","last_active":"2024-09-11T07:34:12.595583+0000","last_peered":"2024-09-11T07:34:12.595583+0000","last_clean":"2024-09-11T07:34:12.595583+0000","last_became_active":"2024-09-11T07:26:31.421276+0000","last_became_peered":"2024-09-11T07:26:31.421276+0000","last_unstale":"2024-09-11T07:34:12.595583+0000","last_undegraded":"2024-09-11T07:34:12.595583+0000","last_fullsized":"2024-09-11T07:34:12.595583+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'5","last_scrub_stamp":"2024-09-11T07:33:14.474301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:14.474301+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:15.895004+0000","scrub_duration":0.0041541269999999996,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":455,"num_read_kb":450,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":200,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.539094+0000","last_change":"2024-09-11T07:34:07.149600+0000","last_active":"2024-09-11T07:34:10.539094+0000","last_peered":"2024-09-11T07:34:10.539094+0000","last_clean":"2024-09-11T07:34:10.539094+0000","last_became_active":"2024-09-11T07:26:31.413354+0000","last_became_peered":"2024-09-11T07:26:31.413354+0000","last_unstale":"2024-09-11T07:34:10.539094+0000","last_undegraded":"2024-09-11T07:34:10.539094+0000","last_fullsized":"2024-09-11T07:34:10.539094+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:07.149367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:07.149367+0000","last_clean_scrub_stamp":"2024-09-11T07:34:07.149367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:12.409913+0000","scrub_duration":0.0082613879999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":193,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.540243+0000","last_change":"2024-09-11T07:33:00.143395+0000","last_active":"2024-09-11T07:34:10.540243+0000","last_peered":"2024-09-11T07:34:10.540243+0000","last_clean":"2024-09-11T07:34:10.540243+0000","last_became_active":"2024-09-11T07:32:00.508398+0000","last_became_peered":"2024-09-11T07:32:00.508398+0000","last_unstale":"2024-09-11T07:34:10.540243+0000","last_undegraded":"2024-09-11T07:34:10.540243+0000","last_fullsized":"2024-09-11T07:34:10.540243+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:00.143339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:00.143339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:28.911771+0000","scrub_duration":0.003474343,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":174,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.544226+0000","last_change":"2024-09-11T07:33:55.344477+0000","last_active":"2024-09-11T07:34:10.544226+0000","last_peered":"2024-09-11T07:34:10.544226+0000","last_clean":"2024-09-11T07:34:10.544226+0000","last_became_active":"2024-09-11T07:32:00.515745+0000","last_became_peered":"2024-09-11T07:32:00.515745+0000","last_unstale":"2024-09-11T07:34:10.544226+0000","last_undegraded":"2024-09-11T07:34:10.544226+0000","last_fullsized":"2024-09-11T07:34:10.544226+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:55.344426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:55.344426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:21.367701+0000","scrub_duration":0.032747216000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"198'4100","reported_seq":4128,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:14.196456+0000","last_change":"2024-09-11T07:33:29.156986+0000","last_active":"2024-09-11T07:34:14.196456+0000","last_peered":"2024-09-11T07:34:14.196456+0000","last_clean":"2024-09-11T07:34:14.196456+0000","last_became_active":"2024-09-11T07:33:29.156567+0000","last_became_peered":"2024-09-11T07:33:29.156567+0000","last_unstale":"2024-09-11T07:34:14.196456+0000","last_undegraded":"2024-09-11T07:34:14.196456+0000","last_fullsized":"2024-09-11T07:34:14.196456+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4100,"log_dups_size":0,"ondisk_log_size":4100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:45.241348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268697600,"num_objects":4100,"num_object_clones":0,"num_object_copies":12300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4100,"num_write_kb":262400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"94'18","reported_seq":235,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.550976+0000","last_change":"2024-09-11T07:33:18.649291+0000","last_active":"2024-09-11T07:34:10.550976+0000","last_peered":"2024-09-11T07:34:10.550976+0000","last_clean":"2024-09-11T07:34:10.550976+0000","last_became_active":"2024-09-11T07:28:09.018134+0000","last_became_peered":"2024-09-11T07:28:09.018134+0000","last_unstale":"2024-09-11T07:34:10.550976+0000","last_undegraded":"2024-09-11T07:34:10.550976+0000","last_fullsized":"2024-09-11T07:34:10.550976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'18","last_scrub_stamp":"2024-09-11T07:33:18.648484+0000","last_deep_scrub":"94'18","last_deep_scrub_stamp":"2024-09-11T07:28:51.437397+0000","last_clean_scrub_stamp":"2024-09-11T07:33:18.648484+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:22.084726+0000","scrub_duration":0.0023327030000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":195,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.554831+0000","last_change":"2024-09-11T07:33:48.749463+0000","last_active":"2024-09-11T07:34:10.554831+0000","last_peered":"2024-09-11T07:34:10.554831+0000","last_clean":"2024-09-11T07:34:10.554831+0000","last_became_active":"2024-09-11T07:26:31.420320+0000","last_became_peered":"2024-09-11T07:26:31.420320+0000","last_unstale":"2024-09-11T07:34:10.554831+0000","last_undegraded":"2024-09-11T07:34:10.554831+0000","last_fullsized":"2024-09-11T07:34:10.554831+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-11T07:33:48.749183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:48.749183+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:14.622124+0000","scrub_duration":0.0059599450000000003,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":185,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.543965+0000","last_change":"2024-09-11T07:33:58.311557+0000","last_active":"2024-09-11T07:34:10.543965+0000","last_peered":"2024-09-11T07:34:10.543965+0000","last_clean":"2024-09-11T07:34:10.543965+0000","last_became_active":"2024-09-11T07:32:00.520035+0000","last_became_peered":"2024-09-11T07:32:00.520035+0000","last_unstale":"2024-09-11T07:34:10.543965+0000","last_undegraded":"2024-09-11T07:34:10.543965+0000","last_fullsized":"2024-09-11T07:34:10.543965+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:58.311344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:16.967492+0000","last_clean_scrub_stamp":"2024-09-11T07:33:58.311344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:22.200383+0000","scrub_duration":0.0030726579999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":176,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.211250+0000","last_change":"2024-09-11T07:34:12.211194+0000","last_active":"2024-09-11T07:34:12.211250+0000","last_peered":"2024-09-11T07:34:12.211250+0000","last_clean":"2024-09-11T07:34:12.211250+0000","last_became_active":"2024-09-11T07:26:35.564178+0000","last_became_peered":"2024-09-11T07:26:35.564178+0000","last_unstale":"2024-09-11T07:34:12.211250+0000","last_undegraded":"2024-09-11T07:34:12.211250+0000","last_fullsized":"2024-09-11T07:34:12.211250+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:12.211119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:12.211119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:16.160923+0000","scrub_duration":0.0062325000000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":185,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.543712+0000","last_change":"2024-09-11T07:33:58.323420+0000","last_active":"2024-09-11T07:34:10.543712+0000","last_peered":"2024-09-11T07:34:10.543712+0000","last_clean":"2024-09-11T07:34:10.543712+0000","last_became_active":"2024-09-11T07:26:37.561378+0000","last_became_peered":"2024-09-11T07:26:37.561378+0000","last_unstale":"2024-09-11T07:34:10.543712+0000","last_undegraded":"2024-09-11T07:34:10.543712+0000","last_fullsized":"2024-09-11T07:34:10.543712+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:58.323318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:58.323318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:00.851560+0000","scrub_duration":0.0083597830000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"198'4058","reported_seq":4087,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:14.046800+0000","last_change":"2024-09-11T07:33:29.154996+0000","last_active":"2024-09-11T07:34:14.046800+0000","last_peered":"2024-09-11T07:34:14.046800+0000","last_clean":"2024-09-11T07:34:14.046800+0000","last_became_active":"2024-09-11T07:33:29.154702+0000","last_became_peered":"2024-09-11T07:33:29.154702+0000","last_unstale":"2024-09-11T07:34:14.046800+0000","last_undegraded":"2024-09-11T07:34:14.046800+0000","last_fullsized":"2024-09-11T07:34:14.046800+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4058,"log_dups_size":0,"ondisk_log_size":4058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:51.999557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":265945088,"num_objects":4058,"num_object_clones":0,"num_object_copies":12174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4058,"num_write_kb":259712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":230,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.544804+0000","last_change":"2024-09-11T07:34:07.759083+0000","last_active":"2024-09-11T07:34:10.544804+0000","last_peered":"2024-09-11T07:34:10.544804+0000","last_clean":"2024-09-11T07:34:10.544804+0000","last_became_active":"2024-09-11T07:32:00.517409+0000","last_became_peered":"2024-09-11T07:32:00.517409+0000","last_unstale":"2024-09-11T07:34:10.544804+0000","last_undegraded":"2024-09-11T07:34:10.544804+0000","last_fullsized":"2024-09-11T07:34:10.544804+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-09-11T07:34:07.758985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:34:07.758985+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:22.987261+0000","scrub_duration":0.0027645090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'3","reported_seq":188,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.556944+0000","last_change":"2024-09-11T07:31:49.316183+0000","last_active":"2024-09-11T07:34:10.556944+0000","last_peered":"2024-09-11T07:34:10.556944+0000","last_clean":"2024-09-11T07:34:10.556944+0000","last_became_active":"2024-09-11T07:26:31.419501+0000","last_became_peered":"2024-09-11T07:26:31.419501+0000","last_unstale":"2024-09-11T07:34:10.556944+0000","last_undegraded":"2024-09-11T07:34:10.556944+0000","last_fullsized":"2024-09-11T07:34:10.556944+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-09-11T07:31:49.316103+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_clean_scrub_stamp":"2024-09-11T07:31:49.316103+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011392093000000001,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":200,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.541261+0000","last_change":"2024-09-11T07:33:51.108800+0000","last_active":"2024-09-11T07:34:10.541261+0000","last_peered":"2024-09-11T07:34:10.541261+0000","last_clean":"2024-09-11T07:34:10.541261+0000","last_became_active":"2024-09-11T07:32:00.498002+0000","last_became_peered":"2024-09-11T07:32:00.498002+0000","last_unstale":"2024-09-11T07:34:10.541261+0000","last_undegraded":"2024-09-11T07:34:10.541261+0000","last_fullsized":"2024-09-11T07:34:10.541261+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:51.108748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:51.108748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:55.524512+0000","scrub_duration":0.0013286089999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"87'81","reported_seq":201,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.543893+0000","last_change":"2024-09-11T07:33:10.189446+0000","last_active":"2024-09-11T07:34:10.543893+0000","last_peered":"2024-09-11T07:34:10.543893+0000","last_clean":"2024-09-11T07:34:10.543893+0000","last_became_active":"2024-09-11T07:32:00.519656+0000","last_became_peered":"2024-09-11T07:32:00.519656+0000","last_unstale":"2024-09-11T07:34:10.543893+0000","last_undegraded":"2024-09-11T07:34:10.543893+0000","last_fullsized":"2024-09-11T07:34:10.543893+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:33:10.189376+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:23.253384+0000","scrub_duration":0.0032570229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":171,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.544524+0000","last_change":"2024-09-11T07:33:51.272916+0000","last_active":"2024-09-11T07:34:10.544524+0000","last_peered":"2024-09-11T07:34:10.544524+0000","last_clean":"2024-09-11T07:34:10.544524+0000","last_became_active":"2024-09-11T07:32:00.510888+0000","last_became_peered":"2024-09-11T07:32:00.510888+0000","last_unstale":"2024-09-11T07:34:10.544524+0000","last_undegraded":"2024-09-11T07:34:10.544524+0000","last_fullsized":"2024-09-11T07:34:10.544524+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:51.272831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:51.272831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:06.075633+0000","scrub_duration":0.0013461180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":174,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.542781+0000","last_change":"2024-09-11T07:32:57.235357+0000","last_active":"2024-09-11T07:34:10.542781+0000","last_peered":"2024-09-11T07:34:10.542781+0000","last_clean":"2024-09-11T07:34:10.542781+0000","last_became_active":"2024-09-11T07:26:37.559920+0000","last_became_peered":"2024-09-11T07:26:37.559920+0000","last_unstale":"2024-09-11T07:34:10.542781+0000","last_undegraded":"2024-09-11T07:34:10.542781+0000","last_fullsized":"2024-09-11T07:34:10.542781+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:32:57.235305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:32:57.235305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:24.383939+0000","scrub_duration":0.0013458579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"198'4064","reported_seq":4090,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:14.573626+0000","last_change":"2024-09-11T07:33:29.204945+0000","last_active":"2024-09-11T07:34:14.573626+0000","last_peered":"2024-09-11T07:34:14.573626+0000","last_clean":"2024-09-11T07:34:14.573626+0000","last_became_active":"2024-09-11T07:33:29.204611+0000","last_became_peered":"2024-09-11T07:33:29.204611+0000","last_unstale":"2024-09-11T07:34:14.573626+0000","last_undegraded":"2024-09-11T07:34:14.573626+0000","last_fullsized":"2024-09-11T07:34:14.573626+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4064,"log_dups_size":0,"ondisk_log_size":4064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:31.427771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":266338304,"num_objects":4064,"num_object_clones":0,"num_object_copies":12192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4064,"num_write_kb":260096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":229,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.544316+0000","last_change":"2024-09-11T07:33:49.612375+0000","last_active":"2024-09-11T07:34:10.544316+0000","last_peered":"2024-09-11T07:34:10.544316+0000","last_clean":"2024-09-11T07:34:10.544316+0000","last_became_active":"2024-09-11T07:32:00.503799+0000","last_became_peered":"2024-09-11T07:32:00.503799+0000","last_unstale":"2024-09-11T07:34:10.544316+0000","last_undegraded":"2024-09-11T07:34:10.544316+0000","last_fullsized":"2024-09-11T07:34:10.544316+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-11T07:33:49.612326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:49.612326+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:15.075880+0000","scrub_duration":0.003204648,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":190,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.542904+0000","last_change":"2024-09-11T07:34:02.192275+0000","last_active":"2024-09-11T07:34:10.542904+0000","last_peered":"2024-09-11T07:34:10.542904+0000","last_clean":"2024-09-11T07:34:10.542904+0000","last_became_active":"2024-09-11T07:26:31.419560+0000","last_became_peered":"2024-09-11T07:26:31.419560+0000","last_unstale":"2024-09-11T07:34:10.542904+0000","last_undegraded":"2024-09-11T07:34:10.542904+0000","last_fullsized":"2024-09-11T07:34:10.542904+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:02.192193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:02.192193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:15.980889+0000","scrub_duration":0.0060418110000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":186,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.538817+0000","last_change":"2024-09-11T07:34:06.134576+0000","last_active":"2024-09-11T07:34:10.538817+0000","last_peered":"2024-09-11T07:34:10.538817+0000","last_clean":"2024-09-11T07:34:10.538817+0000","last_became_active":"2024-09-11T07:26:35.564044+0000","last_became_peered":"2024-09-11T07:26:35.564044+0000","last_unstale":"2024-09-11T07:34:10.538817+0000","last_undegraded":"2024-09-11T07:34:10.538817+0000","last_fullsized":"2024-09-11T07:34:10.538817+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:06.134464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:06.134464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:15.601865+0000","scrub_duration":0.0062969130000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":169,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.544099+0000","last_change":"2024-09-11T07:32:00.520910+0000","last_active":"2024-09-11T07:34:10.544099+0000","last_peered":"2024-09-11T07:34:10.544099+0000","last_clean":"2024-09-11T07:34:10.544099+0000","last_became_active":"2024-09-11T07:32:00.516916+0000","last_became_peered":"2024-09-11T07:32:00.516916+0000","last_unstale":"2024-09-11T07:34:10.544099+0000","last_undegraded":"2024-09-11T07:34:10.544099+0000","last_fullsized":"2024-09-11T07:34:10.544099+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:30:59.302530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:30:59.302530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.7","version":"198'4098","reported_seq":4126,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:14.188394+0000","last_change":"2024-09-11T07:33:29.160658+0000","last_active":"2024-09-11T07:34:14.188394+0000","last_peered":"2024-09-11T07:34:14.188394+0000","last_clean":"2024-09-11T07:34:14.188394+0000","last_became_active":"2024-09-11T07:33:29.160376+0000","last_became_peered":"2024-09-11T07:33:29.160376+0000","last_unstale":"2024-09-11T07:34:14.188394+0000","last_undegraded":"2024-09-11T07:34:14.188394+0000","last_fullsized":"2024-09-11T07:34:14.188394+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4098,"log_dups_size":0,"ondisk_log_size":4098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:55.763414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268566528,"num_objects":4098,"num_object_clones":0,"num_object_copies":12294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4098,"num_write_kb":262272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":193,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.551262+0000","last_change":"2024-09-11T07:33:14.754606+0000","last_active":"2024-09-11T07:34:10.551262+0000","last_peered":"2024-09-11T07:34:10.551262+0000","last_clean":"2024-09-11T07:34:10.551262+0000","last_became_active":"2024-09-11T07:26:33.543148+0000","last_became_peered":"2024-09-11T07:26:33.543148+0000","last_unstale":"2024-09-11T07:34:10.551262+0000","last_undegraded":"2024-09-11T07:34:10.551262+0000","last_fullsized":"2024-09-11T07:34:10.551262+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:33:14.754532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:14.754532+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:25.384577+0000","scrub_duration":0.0034842570000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":188,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.544330+0000","last_change":"2024-09-11T07:33:17.426275+0000","last_active":"2024-09-11T07:34:10.544330+0000","last_peered":"2024-09-11T07:34:10.544330+0000","last_clean":"2024-09-11T07:34:10.544330+0000","last_became_active":"2024-09-11T07:26:31.423056+0000","last_became_peered":"2024-09-11T07:26:31.423056+0000","last_unstale":"2024-09-11T07:34:10.544330+0000","last_undegraded":"2024-09-11T07:34:10.544330+0000","last_fullsized":"2024-09-11T07:34:10.544330+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:17.426220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:17.426220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:19.376100+0000","scrub_duration":0.0035646520000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":186,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.550794+0000","last_change":"2024-09-11T07:33:10.080039+0000","last_active":"2024-09-11T07:34:10.550794+0000","last_peered":"2024-09-11T07:34:10.550794+0000","last_clean":"2024-09-11T07:34:10.550794+0000","last_became_active":"2024-09-11T07:26:35.563182+0000","last_became_peered":"2024-09-11T07:26:35.563182+0000","last_unstale":"2024-09-11T07:34:10.550794+0000","last_undegraded":"2024-09-11T07:34:10.550794+0000","last_fullsized":"2024-09-11T07:34:10.550794+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:10.079984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:10.079984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:14.130284+0000","scrub_duration":0.001676498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":190,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.554664+0000","last_change":"2024-09-11T07:33:11.790118+0000","last_active":"2024-09-11T07:34:10.554664+0000","last_peered":"2024-09-11T07:34:10.554664+0000","last_clean":"2024-09-11T07:34:10.554664+0000","last_became_active":"2024-09-11T07:32:00.505185+0000","last_became_peered":"2024-09-11T07:32:00.505185+0000","last_unstale":"2024-09-11T07:34:10.554664+0000","last_undegraded":"2024-09-11T07:34:10.554664+0000","last_fullsized":"2024-09-11T07:34:10.554664+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:11.790063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:11.790063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:23.070960+0000","scrub_duration":0.001675405,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"198'4088","reported_seq":4117,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:13.886018+0000","last_change":"2024-09-11T07:33:29.153952+0000","last_active":"2024-09-11T07:34:13.886018+0000","last_peered":"2024-09-11T07:34:13.886018+0000","last_clean":"2024-09-11T07:34:13.886018+0000","last_became_active":"2024-09-11T07:33:29.153604+0000","last_became_peered":"2024-09-11T07:33:29.153604+0000","last_unstale":"2024-09-11T07:34:13.886018+0000","last_undegraded":"2024-09-11T07:34:13.886018+0000","last_fullsized":"2024-09-11T07:34:13.886018+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4088,"log_dups_size":0,"ondisk_log_size":4088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:52.795972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":267911168,"num_objects":4088,"num_object_clones":0,"num_object_copies":12264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4088,"num_write_kb":261632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":223,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.542058+0000","last_change":"2024-09-11T07:33:56.187920+0000","last_active":"2024-09-11T07:34:10.542058+0000","last_peered":"2024-09-11T07:34:10.542058+0000","last_clean":"2024-09-11T07:34:10.542058+0000","last_became_active":"2024-09-11T07:32:00.521464+0000","last_became_peered":"2024-09-11T07:32:00.521464+0000","last_unstale":"2024-09-11T07:34:10.542058+0000","last_undegraded":"2024-09-11T07:34:10.542058+0000","last_fullsized":"2024-09-11T07:34:10.542058+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-09-11T07:33:56.187871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:56.187871+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:23.701092+0000","scrub_duration":0.001643175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":207,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.540389+0000","last_change":"2024-09-11T07:33:12.052744+0000","last_active":"2024-09-11T07:34:10.540389+0000","last_peered":"2024-09-11T07:34:10.540389+0000","last_clean":"2024-09-11T07:34:10.540389+0000","last_became_active":"2024-09-11T07:32:00.513949+0000","last_became_peered":"2024-09-11T07:32:00.513949+0000","last_unstale":"2024-09-11T07:34:10.540389+0000","last_undegraded":"2024-09-11T07:34:10.540389+0000","last_fullsized":"2024-09-11T07:34:10.540389+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:12.052670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_clean_scrub_stamp":"2024-09-11T07:33:12.052670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:15.594508+0000","scrub_duration":0.0014794280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"151'10","reported_seq":237,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.543817+0000","last_change":"2024-09-11T07:33:48.261946+0000","last_active":"2024-09-11T07:34:10.543817+0000","last_peered":"2024-09-11T07:34:10.543817+0000","last_clean":"2024-09-11T07:34:10.543817+0000","last_became_active":"2024-09-11T07:32:00.506806+0000","last_became_peered":"2024-09-11T07:32:00.506806+0000","last_unstale":"2024-09-11T07:34:10.543817+0000","last_undegraded":"2024-09-11T07:34:10.543817+0000","last_fullsized":"2024-09-11T07:34:10.543817+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:33:48.261857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:48.261857+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:06.611603+0000","scrub_duration":0.001795774,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":174,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.541494+0000","last_change":"2024-09-11T07:31:37.988572+0000","last_active":"2024-09-11T07:34:10.541494+0000","last_peered":"2024-09-11T07:34:10.541494+0000","last_clean":"2024-09-11T07:34:10.541494+0000","last_became_active":"2024-09-11T07:26:37.861134+0000","last_became_peered":"2024-09-11T07:26:37.861134+0000","last_unstale":"2024-09-11T07:34:10.541494+0000","last_undegraded":"2024-09-11T07:34:10.541494+0000","last_fullsized":"2024-09-11T07:34:10.541494+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:31:37.988491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:31:37.988491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002742899,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.5","version":"198'4052","reported_seq":4081,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:14.019331+0000","last_change":"2024-09-11T07:33:29.205466+0000","last_active":"2024-09-11T07:34:14.019331+0000","last_peered":"2024-09-11T07:34:14.019331+0000","last_clean":"2024-09-11T07:34:14.019331+0000","last_became_active":"2024-09-11T07:33:29.205189+0000","last_became_peered":"2024-09-11T07:33:29.205189+0000","last_unstale":"2024-09-11T07:34:14.019331+0000","last_undegraded":"2024-09-11T07:34:14.019331+0000","last_fullsized":"2024-09-11T07:34:14.019331+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4052,"log_dups_size":0,"ondisk_log_size":4052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:56.812992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":265551872,"num_objects":4052,"num_object_clones":0,"num_object_copies":12156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4052,"num_write_kb":259328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"83'17","reported_seq":205,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.538571+0000","last_change":"2024-09-11T07:31:32.003568+0000","last_active":"2024-09-11T07:34:10.538571+0000","last_peered":"2024-09-11T07:34:10.538571+0000","last_clean":"2024-09-11T07:34:10.538571+0000","last_became_active":"2024-09-11T07:26:33.540363+0000","last_became_peered":"2024-09-11T07:26:33.540363+0000","last_unstale":"2024-09-11T07:34:10.538571+0000","last_undegraded":"2024-09-11T07:34:10.538571+0000","last_fullsized":"2024-09-11T07:34:10.538571+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'17","last_scrub_stamp":"2024-09-11T07:31:32.003516+0000","last_deep_scrub":"83'17","last_deep_scrub_stamp":"2024-09-11T07:29:50.970355+0000","last_clean_scrub_stamp":"2024-09-11T07:31:32.003516+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004484811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":216,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.539097+0000","last_change":"2024-09-11T07:33:48.436215+0000","last_active":"2024-09-11T07:34:10.539097+0000","last_peered":"2024-09-11T07:34:10.539097+0000","last_clean":"2024-09-11T07:34:10.539097+0000","last_became_active":"2024-09-11T07:32:00.504344+0000","last_became_peered":"2024-09-11T07:32:00.504344+0000","last_unstale":"2024-09-11T07:34:10.539097+0000","last_undegraded":"2024-09-11T07:34:10.539097+0000","last_fullsized":"2024-09-11T07:34:10.539097+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:33:48.436127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:48.436127+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:59.296316+0000","scrub_duration":0.002390933,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":186,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.538553+0000","last_change":"2024-09-11T07:33:51.206706+0000","last_active":"2024-09-11T07:34:10.538553+0000","last_peered":"2024-09-11T07:34:10.538553+0000","last_clean":"2024-09-11T07:34:10.538553+0000","last_became_active":"2024-09-11T07:26:35.561967+0000","last_became_peered":"2024-09-11T07:26:35.561967+0000","last_unstale":"2024-09-11T07:34:10.538553+0000","last_undegraded":"2024-09-11T07:34:10.538553+0000","last_fullsized":"2024-09-11T07:34:10.538553+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:51.206632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:44.820354+0000","last_clean_scrub_stamp":"2024-09-11T07:33:51.206632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:01.971369+0000","scrub_duration":0.0065238259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":191,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.559079+0000","last_change":"2024-09-11T07:33:10.531766+0000","last_active":"2024-09-11T07:34:10.559079+0000","last_peered":"2024-09-11T07:34:10.559079+0000","last_clean":"2024-09-11T07:34:10.559079+0000","last_became_active":"2024-09-11T07:32:00.502019+0000","last_became_peered":"2024-09-11T07:32:00.502019+0000","last_unstale":"2024-09-11T07:34:10.559079+0000","last_undegraded":"2024-09-11T07:34:10.559079+0000","last_fullsized":"2024-09-11T07:34:10.559079+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:10.531696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:10.531696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:13.634576+0000","scrub_duration":0.0018694370000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":210,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.550935+0000","last_change":"2024-09-11T07:33:05.880679+0000","last_active":"2024-09-11T07:34:10.550935+0000","last_peered":"2024-09-11T07:34:10.550935+0000","last_clean":"2024-09-11T07:34:10.550935+0000","last_became_active":"2024-09-11T07:32:00.509412+0000","last_became_peered":"2024-09-11T07:32:00.509412+0000","last_unstale":"2024-09-11T07:34:10.550935+0000","last_undegraded":"2024-09-11T07:34:10.550935+0000","last_fullsized":"2024-09-11T07:34:10.550935+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:33:05.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:05.880612+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:35.340902+0000","scrub_duration":0.0021137109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":190,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.538670+0000","last_change":"2024-09-11T07:34:03.167417+0000","last_active":"2024-09-11T07:34:10.538670+0000","last_peered":"2024-09-11T07:34:10.538670+0000","last_clean":"2024-09-11T07:34:10.538670+0000","last_became_active":"2024-09-11T07:26:31.415701+0000","last_became_peered":"2024-09-11T07:26:31.415701+0000","last_unstale":"2024-09-11T07:34:10.538670+0000","last_undegraded":"2024-09-11T07:34:10.538670+0000","last_fullsized":"2024-09-11T07:34:10.538670+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:03.167327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.925200+0000","last_clean_scrub_stamp":"2024-09-11T07:34:03.167327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:28.877249+0000","scrub_duration":0.0047032489999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":193,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.560151+0000","last_change":"2024-09-11T07:33:54.643176+0000","last_active":"2024-09-11T07:34:10.560151+0000","last_peered":"2024-09-11T07:34:10.560151+0000","last_clean":"2024-09-11T07:34:10.560151+0000","last_became_active":"2024-09-11T07:32:00.499598+0000","last_became_peered":"2024-09-11T07:32:00.499598+0000","last_unstale":"2024-09-11T07:34:10.560151+0000","last_undegraded":"2024-09-11T07:34:10.560151+0000","last_fullsized":"2024-09-11T07:34:10.560151+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:54.642486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:35.392360+0000","last_clean_scrub_stamp":"2024-09-11T07:33:54.642486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:23.835294+0000","scrub_duration":0.010529307999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":192,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.539461+0000","last_change":"2024-09-11T07:33:03.061811+0000","last_active":"2024-09-11T07:34:10.539461+0000","last_peered":"2024-09-11T07:34:10.539461+0000","last_clean":"2024-09-11T07:34:10.539461+0000","last_became_active":"2024-09-11T07:32:00.506697+0000","last_became_peered":"2024-09-11T07:32:00.506697+0000","last_unstale":"2024-09-11T07:34:10.539461+0000","last_undegraded":"2024-09-11T07:34:10.539461+0000","last_fullsized":"2024-09-11T07:34:10.539461+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:33:03.061713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:03.061713+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:19.751616+0000","scrub_duration":0.002417888,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":190,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.544933+0000","last_change":"2024-09-11T07:33:03.339633+0000","last_active":"2024-09-11T07:34:10.544933+0000","last_peered":"2024-09-11T07:34:10.544933+0000","last_clean":"2024-09-11T07:34:10.544933+0000","last_became_active":"2024-09-11T07:26:31.423229+0000","last_became_peered":"2024-09-11T07:26:31.423229+0000","last_unstale":"2024-09-11T07:34:10.544933+0000","last_undegraded":"2024-09-11T07:34:10.544933+0000","last_fullsized":"2024-09-11T07:34:10.544933+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:03.339569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:03.339569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:30.079046+0000","scrub_duration":0.001437523,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"82'9","reported_seq":207,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.563468+0000","last_change":"2024-09-11T07:33:11.576715+0000","last_active":"2024-09-11T07:34:10.563468+0000","last_peered":"2024-09-11T07:34:10.563468+0000","last_clean":"2024-09-11T07:34:10.563468+0000","last_became_active":"2024-09-11T07:32:00.501365+0000","last_became_peered":"2024-09-11T07:32:00.501365+0000","last_unstale":"2024-09-11T07:34:10.563468+0000","last_undegraded":"2024-09-11T07:34:10.563468+0000","last_fullsized":"2024-09-11T07:34:10.563468+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'9","last_scrub_stamp":"2024-09-11T07:33:11.576656+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:33:11.576656+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:35.698527+0000","scrub_duration":0.0043542040000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":186,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.541320+0000","last_change":"2024-09-11T07:33:15.113446+0000","last_active":"2024-09-11T07:34:10.541320+0000","last_peered":"2024-09-11T07:34:10.541320+0000","last_clean":"2024-09-11T07:34:10.541320+0000","last_became_active":"2024-09-11T07:26:35.566894+0000","last_became_peered":"2024-09-11T07:26:35.566894+0000","last_unstale":"2024-09-11T07:34:10.541320+0000","last_undegraded":"2024-09-11T07:34:10.541320+0000","last_fullsized":"2024-09-11T07:34:10.541320+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:15.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:33:15.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:25.120342+0000","scrub_duration":0.0011939730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":172,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.546894+0000","last_change":"2024-09-11T07:33:54.283942+0000","last_active":"2024-09-11T07:34:10.546894+0000","last_peered":"2024-09-11T07:34:10.546894+0000","last_clean":"2024-09-11T07:34:10.546894+0000","last_became_active":"2024-09-11T07:32:00.514100+0000","last_became_peered":"2024-09-11T07:32:00.514100+0000","last_unstale":"2024-09-11T07:34:10.546894+0000","last_undegraded":"2024-09-11T07:34:10.546894+0000","last_fullsized":"2024-09-11T07:34:10.546894+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:54.283860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:54.283860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:00.050044+0000","scrub_duration":0.0032251950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"86'15","reported_seq":227,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.559922+0000","last_change":"2024-09-11T07:32:59.286373+0000","last_active":"2024-09-11T07:34:10.559922+0000","last_peered":"2024-09-11T07:34:10.559922+0000","last_clean":"2024-09-11T07:34:10.559922+0000","last_became_active":"2024-09-11T07:32:00.504463+0000","last_became_peered":"2024-09-11T07:32:00.504463+0000","last_unstale":"2024-09-11T07:34:10.559922+0000","last_undegraded":"2024-09-11T07:34:10.559922+0000","last_fullsized":"2024-09-11T07:34:10.559922+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'15","last_scrub_stamp":"2024-09-11T07:32:59.286314+0000","last_deep_scrub":"86'15","last_deep_scrub_stamp":"2024-09-11T07:28:22.858137+0000","last_clean_scrub_stamp":"2024-09-11T07:32:59.286314+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018903660000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":200,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.541761+0000","last_change":"2024-09-11T07:34:02.231632+0000","last_active":"2024-09-11T07:34:10.541761+0000","last_peered":"2024-09-11T07:34:10.541761+0000","last_clean":"2024-09-11T07:34:10.541761+0000","last_became_active":"2024-09-11T07:32:00.498473+0000","last_became_peered":"2024-09-11T07:32:00.498473+0000","last_unstale":"2024-09-11T07:34:10.541761+0000","last_undegraded":"2024-09-11T07:34:10.541761+0000","last_fullsized":"2024-09-11T07:34:10.541761+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:02.231583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:02.231583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:24.160910+0000","scrub_duration":0.0013039099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":276,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.547143+0000","last_change":"2024-09-11T07:31:42.984614+0000","last_active":"2024-09-11T07:34:10.547143+0000","last_peered":"2024-09-11T07:34:10.547143+0000","last_clean":"2024-09-11T07:34:10.547143+0000","last_became_active":"2024-09-11T07:26:35.562245+0000","last_became_peered":"2024-09-11T07:26:35.562245+0000","last_unstale":"2024-09-11T07:34:10.547143+0000","last_undegraded":"2024-09-11T07:34:10.547143+0000","last_fullsized":"2024-09-11T07:34:10.547143+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:31:42.984562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:31:42.984562+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015727580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":200,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.556714+0000","last_change":"2024-09-11T07:34:12.556603+0000","last_active":"2024-09-11T07:34:12.556714+0000","last_peered":"2024-09-11T07:34:12.556714+0000","last_clean":"2024-09-11T07:34:12.556714+0000","last_became_active":"2024-09-11T07:32:00.509295+0000","last_became_peered":"2024-09-11T07:32:00.509295+0000","last_unstale":"2024-09-11T07:34:12.556714+0000","last_undegraded":"2024-09-11T07:34:12.556714+0000","last_fullsized":"2024-09-11T07:34:12.556714+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:12.556370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:12.556370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:14.451889+0000","scrub_duration":0.0041791759999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"83'13","reported_seq":218,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.547166+0000","last_change":"2024-09-11T07:33:52.110471+0000","last_active":"2024-09-11T07:34:10.547166+0000","last_peered":"2024-09-11T07:34:10.547166+0000","last_clean":"2024-09-11T07:34:10.547166+0000","last_became_active":"2024-09-11T07:32:00.505134+0000","last_became_peered":"2024-09-11T07:32:00.505134+0000","last_unstale":"2024-09-11T07:34:10.547166+0000","last_undegraded":"2024-09-11T07:34:10.547166+0000","last_fullsized":"2024-09-11T07:34:10.547166+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-11T07:33:52.110361+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-11T07:28:00.959583+0000","last_clean_scrub_stamp":"2024-09-11T07:33:52.110361+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:04.735778+0000","scrub_duration":0.0035066310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":189,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.551468+0000","last_change":"2024-09-11T07:33:51.733858+0000","last_active":"2024-09-11T07:34:10.551468+0000","last_peered":"2024-09-11T07:34:10.551468+0000","last_clean":"2024-09-11T07:34:10.551468+0000","last_became_active":"2024-09-11T07:26:31.419409+0000","last_became_peered":"2024-09-11T07:26:31.419409+0000","last_unstale":"2024-09-11T07:34:10.551468+0000","last_undegraded":"2024-09-11T07:34:10.551468+0000","last_fullsized":"2024-09-11T07:34:10.551468+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:51.733780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:51.733780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:20.982450+0000","scrub_duration":0.079912094000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"151'10","reported_seq":288,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.560025+0000","last_change":"2024-09-11T07:33:47.546713+0000","last_active":"2024-09-11T07:34:10.560025+0000","last_peered":"2024-09-11T07:34:10.560025+0000","last_clean":"2024-09-11T07:34:10.560025+0000","last_became_active":"2024-09-11T07:32:00.504062+0000","last_became_peered":"2024-09-11T07:32:00.504062+0000","last_unstale":"2024-09-11T07:34:10.560025+0000","last_undegraded":"2024-09-11T07:34:10.560025+0000","last_fullsized":"2024-09-11T07:34:10.560025+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:33:47.546622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:47.546622+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:07.659165+0000","scrub_duration":0.0064760360000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":173,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.538773+0000","last_change":"2024-09-11T07:31:36.995728+0000","last_active":"2024-09-11T07:34:10.538773+0000","last_peered":"2024-09-11T07:34:10.538773+0000","last_clean":"2024-09-11T07:34:10.538773+0000","last_became_active":"2024-09-11T07:26:37.565723+0000","last_became_peered":"2024-09-11T07:26:37.565723+0000","last_unstale":"2024-09-11T07:34:10.538773+0000","last_undegraded":"2024-09-11T07:34:10.538773+0000","last_fullsized":"2024-09-11T07:34:10.538773+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:31:36.995548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:31:36.995548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0093244260000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":212,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.364372+0000","last_change":"2024-09-11T07:34:12.364224+0000","last_active":"2024-09-11T07:34:12.364372+0000","last_peered":"2024-09-11T07:34:12.364372+0000","last_clean":"2024-09-11T07:34:12.364372+0000","last_became_active":"2024-09-11T07:26:33.540688+0000","last_became_peered":"2024-09-11T07:26:33.540688+0000","last_unstale":"2024-09-11T07:34:12.364372+0000","last_undegraded":"2024-09-11T07:34:12.364372+0000","last_fullsized":"2024-09-11T07:34:12.364372+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:34:12.364126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:34:12.364126+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:21.215985+0000","scrub_duration":0.011537936,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":193,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.544565+0000","last_change":"2024-09-11T07:34:09.772110+0000","last_active":"2024-09-11T07:34:10.544565+0000","last_peered":"2024-09-11T07:34:10.544565+0000","last_clean":"2024-09-11T07:34:10.544565+0000","last_became_active":"2024-09-11T07:26:31.425587+0000","last_became_peered":"2024-09-11T07:26:31.425587+0000","last_unstale":"2024-09-11T07:34:10.544565+0000","last_undegraded":"2024-09-11T07:34:10.544565+0000","last_fullsized":"2024-09-11T07:34:10.544565+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:09.772000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:09.772000+0000","last_clean_scrub_stamp":"2024-09-11T07:34:09.772000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:14.926483+0000","scrub_duration":0.0087789909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":176,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.559159+0000","last_change":"2024-09-11T07:34:04.660377+0000","last_active":"2024-09-11T07:34:10.559159+0000","last_peered":"2024-09-11T07:34:10.559159+0000","last_clean":"2024-09-11T07:34:10.559159+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:34:10.559159+0000","last_undegraded":"2024-09-11T07:34:10.559159+0000","last_fullsized":"2024-09-11T07:34:10.559159+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:04.660121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:04.660121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:12.877858+0000","scrub_duration":0.0047831310000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":183,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.550376+0000","last_change":"2024-09-11T07:33:53.667783+0000","last_active":"2024-09-11T07:34:10.550376+0000","last_peered":"2024-09-11T07:34:10.550376+0000","last_clean":"2024-09-11T07:34:10.550376+0000","last_became_active":"2024-09-11T07:26:37.568886+0000","last_became_peered":"2024-09-11T07:26:37.568886+0000","last_unstale":"2024-09-11T07:34:10.550376+0000","last_undegraded":"2024-09-11T07:34:10.550376+0000","last_fullsized":"2024-09-11T07:34:10.550376+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:53.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:53.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:59.734127+0000","scrub_duration":0.0039882809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"136'16","reported_seq":211,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.540186+0000","last_change":"2024-09-11T07:34:00.417426+0000","last_active":"2024-09-11T07:34:10.540186+0000","last_peered":"2024-09-11T07:34:10.540186+0000","last_clean":"2024-09-11T07:34:10.540186+0000","last_became_active":"2024-09-11T07:26:33.540208+0000","last_became_peered":"2024-09-11T07:26:33.540208+0000","last_unstale":"2024-09-11T07:34:10.540186+0000","last_undegraded":"2024-09-11T07:34:10.540186+0000","last_fullsized":"2024-09-11T07:34:10.540186+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'16","last_scrub_stamp":"2024-09-11T07:34:00.417343+0000","last_deep_scrub":"136'16","last_deep_scrub_stamp":"2024-09-11T07:31:06.039023+0000","last_clean_scrub_stamp":"2024-09-11T07:34:00.417343+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:12.571774+0000","scrub_duration":0.0026159130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":191,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.544384+0000","last_change":"2024-09-11T07:33:57.333274+0000","last_active":"2024-09-11T07:34:10.544384+0000","last_peered":"2024-09-11T07:34:10.544384+0000","last_clean":"2024-09-11T07:34:10.544384+0000","last_became_active":"2024-09-11T07:26:31.422671+0000","last_became_peered":"2024-09-11T07:26:31.422671+0000","last_unstale":"2024-09-11T07:34:10.544384+0000","last_undegraded":"2024-09-11T07:34:10.544384+0000","last_fullsized":"2024-09-11T07:34:10.544384+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:57.333166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:33:57.333166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:16.535083+0000","scrub_duration":0.010637648,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":177,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.548230+0000","last_change":"2024-09-11T07:33:51.251429+0000","last_active":"2024-09-11T07:34:10.548230+0000","last_peered":"2024-09-11T07:34:10.548230+0000","last_clean":"2024-09-11T07:34:10.548230+0000","last_became_active":"2024-09-11T07:26:35.566931+0000","last_became_peered":"2024-09-11T07:26:35.566931+0000","last_unstale":"2024-09-11T07:34:10.548230+0000","last_undegraded":"2024-09-11T07:34:10.548230+0000","last_fullsized":"2024-09-11T07:34:10.548230+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:51.251225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.847474+0000","last_clean_scrub_stamp":"2024-09-11T07:33:51.251225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:08.093660+0000","scrub_duration":0.0050645059999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":164,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.556104+0000","last_change":"2024-09-11T07:31:00.433331+0000","last_active":"2024-09-11T07:34:10.556104+0000","last_peered":"2024-09-11T07:34:10.556104+0000","last_clean":"2024-09-11T07:34:10.556104+0000","last_became_active":"2024-09-11T07:26:37.561573+0000","last_became_peered":"2024-09-11T07:26:37.561573+0000","last_unstale":"2024-09-11T07:34:10.556104+0000","last_undegraded":"2024-09-11T07:34:10.556104+0000","last_fullsized":"2024-09-11T07:34:10.556104+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:31:00.432868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:31:00.432868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012162007000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"82'6","reported_seq":186,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.563042+0000","last_change":"2024-09-11T07:31:38.286280+0000","last_active":"2024-09-11T07:34:10.563042+0000","last_peered":"2024-09-11T07:34:10.563042+0000","last_clean":"2024-09-11T07:34:10.563042+0000","last_became_active":"2024-09-11T07:26:33.545830+0000","last_became_peered":"2024-09-11T07:26:33.545830+0000","last_unstale":"2024-09-11T07:34:10.563042+0000","last_undegraded":"2024-09-11T07:34:10.563042+0000","last_fullsized":"2024-09-11T07:34:10.563042+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'6","last_scrub_stamp":"2024-09-11T07:31:38.285969+0000","last_deep_scrub":"82'6","last_deep_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_clean_scrub_stamp":"2024-09-11T07:31:38.285969+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0068909499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":180,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.541605+0000","last_change":"2024-09-11T07:31:35.961700+0000","last_active":"2024-09-11T07:34:10.541605+0000","last_peered":"2024-09-11T07:34:10.541605+0000","last_clean":"2024-09-11T07:34:10.541605+0000","last_became_active":"2024-09-11T07:26:31.420518+0000","last_became_peered":"2024-09-11T07:26:31.420518+0000","last_unstale":"2024-09-11T07:34:10.541605+0000","last_undegraded":"2024-09-11T07:34:10.541605+0000","last_fullsized":"2024-09-11T07:34:10.541605+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:31:35.961609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_clean_scrub_stamp":"2024-09-11T07:31:35.961609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001830932,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":196,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.544257+0000","last_change":"2024-09-11T07:33:16.555518+0000","last_active":"2024-09-11T07:34:10.544257+0000","last_peered":"2024-09-11T07:34:10.544257+0000","last_clean":"2024-09-11T07:34:10.544257+0000","last_became_active":"2024-09-11T07:32:00.501969+0000","last_became_peered":"2024-09-11T07:32:00.501969+0000","last_unstale":"2024-09-11T07:34:10.544257+0000","last_undegraded":"2024-09-11T07:34:10.544257+0000","last_fullsized":"2024-09-11T07:34:10.544257+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:16.555461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:16.555461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:18.844094+0000","scrub_duration":0.001257347,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":174,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.544217+0000","last_change":"2024-09-11T07:33:53.290535+0000","last_active":"2024-09-11T07:34:10.544217+0000","last_peered":"2024-09-11T07:34:10.544217+0000","last_clean":"2024-09-11T07:34:10.544217+0000","last_became_active":"2024-09-11T07:32:00.511407+0000","last_became_peered":"2024-09-11T07:32:00.511407+0000","last_unstale":"2024-09-11T07:34:10.544217+0000","last_undegraded":"2024-09-11T07:34:10.544217+0000","last_fullsized":"2024-09-11T07:34:10.544217+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:53.290379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:53.290379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:58.283671+0000","scrub_duration":0.002605291,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":204,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.556280+0000","last_change":"2024-09-11T07:33:03.304492+0000","last_active":"2024-09-11T07:34:10.556280+0000","last_peered":"2024-09-11T07:34:10.556280+0000","last_clean":"2024-09-11T07:34:10.556280+0000","last_became_active":"2024-09-11T07:32:00.502081+0000","last_became_peered":"2024-09-11T07:32:00.502081+0000","last_unstale":"2024-09-11T07:34:10.556280+0000","last_undegraded":"2024-09-11T07:34:10.556280+0000","last_fullsized":"2024-09-11T07:34:10.556280+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:33:03.304404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:03.304404+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:29.580187+0000","scrub_duration":0.0023973169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":190,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.545254+0000","last_change":"2024-09-11T07:33:15.559485+0000","last_active":"2024-09-11T07:34:10.545254+0000","last_peered":"2024-09-11T07:34:10.545254+0000","last_clean":"2024-09-11T07:34:10.545254+0000","last_became_active":"2024-09-11T07:26:31.425876+0000","last_became_peered":"2024-09-11T07:26:31.425876+0000","last_unstale":"2024-09-11T07:34:10.545254+0000","last_undegraded":"2024-09-11T07:34:10.545254+0000","last_fullsized":"2024-09-11T07:34:10.545254+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:15.559426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:15.559426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:40.363895+0000","scrub_duration":0.0013828619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":147,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.543693+0000","last_change":"2024-09-11T07:33:11.235639+0000","last_active":"2024-09-11T07:34:10.543693+0000","last_peered":"2024-09-11T07:34:10.543693+0000","last_clean":"2024-09-11T07:34:10.543693+0000","last_became_active":"2024-09-11T07:32:00.520534+0000","last_became_peered":"2024-09-11T07:32:00.520534+0000","last_unstale":"2024-09-11T07:34:10.543693+0000","last_undegraded":"2024-09-11T07:34:10.543693+0000","last_fullsized":"2024-09-11T07:34:10.543693+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:11.235570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:39.021876+0000","last_clean_scrub_stamp":"2024-09-11T07:33:11.235570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:18.432537+0000","scrub_duration":0.001531335,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":164,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.557062+0000","last_change":"2024-09-11T07:31:05.392160+0000","last_active":"2024-09-11T07:34:10.557062+0000","last_peered":"2024-09-11T07:34:10.557062+0000","last_clean":"2024-09-11T07:34:10.557062+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:34:10.557062+0000","last_undegraded":"2024-09-11T07:34:10.557062+0000","last_fullsized":"2024-09-11T07:34:10.557062+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:31:05.391713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:31:05.391713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050876960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":151,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.544224+0000","last_change":"2024-09-11T07:33:15.248196+0000","last_active":"2024-09-11T07:34:10.544224+0000","last_peered":"2024-09-11T07:34:10.544224+0000","last_clean":"2024-09-11T07:34:10.544224+0000","last_became_active":"2024-09-11T07:32:00.511258+0000","last_became_peered":"2024-09-11T07:32:00.511258+0000","last_unstale":"2024-09-11T07:34:10.544224+0000","last_undegraded":"2024-09-11T07:34:10.544224+0000","last_fullsized":"2024-09-11T07:34:10.544224+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:33:15.248113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:15.248113+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:18.239937+0000","scrub_duration":0.0017074939999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":209,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.542246+0000","last_change":"2024-09-11T07:33:53.155118+0000","last_active":"2024-09-11T07:34:10.542246+0000","last_peered":"2024-09-11T07:34:10.542246+0000","last_clean":"2024-09-11T07:34:10.542246+0000","last_became_active":"2024-09-11T07:32:00.518307+0000","last_became_peered":"2024-09-11T07:32:00.518307+0000","last_unstale":"2024-09-11T07:34:10.542246+0000","last_undegraded":"2024-09-11T07:34:10.542246+0000","last_fullsized":"2024-09-11T07:34:10.542246+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:33:53.155024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:53.155024+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:20.182620+0000","scrub_duration":0.0038397209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":209,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.550887+0000","last_change":"2024-09-11T07:33:07.805709+0000","last_active":"2024-09-11T07:34:10.550887+0000","last_peered":"2024-09-11T07:34:10.550887+0000","last_clean":"2024-09-11T07:34:10.550887+0000","last_became_active":"2024-09-11T07:32:00.508233+0000","last_became_peered":"2024-09-11T07:32:00.508233+0000","last_unstale":"2024-09-11T07:34:10.550887+0000","last_undegraded":"2024-09-11T07:34:10.550887+0000","last_fullsized":"2024-09-11T07:34:10.550887+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:33:07.805635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:07.805635+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:28.351452+0000","scrub_duration":0.002375144,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"151'10","reported_seq":301,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.545101+0000","last_change":"2024-09-11T07:34:02.797981+0000","last_active":"2024-09-11T07:34:10.545101+0000","last_peered":"2024-09-11T07:34:10.545101+0000","last_clean":"2024-09-11T07:34:10.545101+0000","last_became_active":"2024-09-11T07:32:00.502049+0000","last_became_peered":"2024-09-11T07:32:00.502049+0000","last_unstale":"2024-09-11T07:34:10.545101+0000","last_undegraded":"2024-09-11T07:34:10.545101+0000","last_fullsized":"2024-09-11T07:34:10.545101+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:34:02.797889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:02.797889+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:19.482223+0000","scrub_duration":0.001718904,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":182,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.544076+0000","last_change":"2024-09-11T07:34:06.381128+0000","last_active":"2024-09-11T07:34:10.544076+0000","last_peered":"2024-09-11T07:34:10.544076+0000","last_clean":"2024-09-11T07:34:10.544076+0000","last_became_active":"2024-09-11T07:26:37.562501+0000","last_became_peered":"2024-09-11T07:26:37.562501+0000","last_unstale":"2024-09-11T07:34:10.544076+0000","last_undegraded":"2024-09-11T07:34:10.544076+0000","last_fullsized":"2024-09-11T07:34:10.544076+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:06.381043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:06.381043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:06.803518+0000","scrub_duration":0.0046834980000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":180,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.556552+0000","last_change":"2024-09-11T07:31:52.334717+0000","last_active":"2024-09-11T07:34:10.556552+0000","last_peered":"2024-09-11T07:34:10.556552+0000","last_clean":"2024-09-11T07:34:10.556552+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:34:10.556552+0000","last_undegraded":"2024-09-11T07:34:10.556552+0000","last_fullsized":"2024-09-11T07:34:10.556552+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:31:52.334517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:31:52.334517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029011779999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":261,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:11.760487+0000","last_change":"2024-09-11T07:34:11.760411+0000","last_active":"2024-09-11T07:34:11.760487+0000","last_peered":"2024-09-11T07:34:11.760487+0000","last_clean":"2024-09-11T07:34:11.760487+0000","last_became_active":"2024-09-11T07:30:28.599840+0000","last_became_peered":"2024-09-11T07:30:28.599840+0000","last_unstale":"2024-09-11T07:34:11.760487+0000","last_undegraded":"2024-09-11T07:34:11.760487+0000","last_fullsized":"2024-09-11T07:34:11.760487+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:34:11.760315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:34:11.760315+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:31.329350+0000","scrub_duration":0.007126177,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":192,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.550808+0000","last_change":"2024-09-11T07:33:13.776627+0000","last_active":"2024-09-11T07:34:10.550808+0000","last_peered":"2024-09-11T07:34:10.550808+0000","last_clean":"2024-09-11T07:34:10.550808+0000","last_became_active":"2024-09-11T07:32:00.507050+0000","last_became_peered":"2024-09-11T07:32:00.507050+0000","last_unstale":"2024-09-11T07:34:10.550808+0000","last_undegraded":"2024-09-11T07:34:10.550808+0000","last_fullsized":"2024-09-11T07:34:10.550808+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:13.776566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:24.575210+0000","last_clean_scrub_stamp":"2024-09-11T07:33:13.776566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:43.729373+0000","scrub_duration":0.001406963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":183,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.538954+0000","last_change":"2024-09-11T07:33:54.459273+0000","last_active":"2024-09-11T07:34:10.538954+0000","last_peered":"2024-09-11T07:34:10.538954+0000","last_clean":"2024-09-11T07:34:10.538954+0000","last_became_active":"2024-09-11T07:26:37.862373+0000","last_became_peered":"2024-09-11T07:26:37.862373+0000","last_unstale":"2024-09-11T07:34:10.538954+0000","last_undegraded":"2024-09-11T07:34:10.538954+0000","last_fullsized":"2024-09-11T07:34:10.538954+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:54.459223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:54.459223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:00.855517+0000","scrub_duration":0.001318018,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":210,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.544444+0000","last_change":"2024-09-11T07:34:06.757693+0000","last_active":"2024-09-11T07:34:10.544444+0000","last_peered":"2024-09-11T07:34:10.544444+0000","last_clean":"2024-09-11T07:34:10.544444+0000","last_became_active":"2024-09-11T07:32:00.503749+0000","last_became_peered":"2024-09-11T07:32:00.503749+0000","last_unstale":"2024-09-11T07:34:10.544444+0000","last_undegraded":"2024-09-11T07:34:10.544444+0000","last_fullsized":"2024-09-11T07:34:10.544444+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:06.757621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:06.757621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:18.674821+0000","scrub_duration":0.0015661609999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":213,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.542678+0000","last_change":"2024-09-11T07:33:55.212599+0000","last_active":"2024-09-11T07:34:10.542678+0000","last_peered":"2024-09-11T07:34:10.542678+0000","last_clean":"2024-09-11T07:34:10.542678+0000","last_became_active":"2024-09-11T07:26:33.542544+0000","last_became_peered":"2024-09-11T07:26:33.542544+0000","last_unstale":"2024-09-11T07:34:10.542678+0000","last_undegraded":"2024-09-11T07:34:10.542678+0000","last_fullsized":"2024-09-11T07:34:10.542678+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-09-11T07:33:55.212499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:55.212499+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:23.471681+0000","scrub_duration":0.004594473,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":184,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.544235+0000","last_change":"2024-09-11T07:34:01.366655+0000","last_active":"2024-09-11T07:34:10.544235+0000","last_peered":"2024-09-11T07:34:10.544235+0000","last_clean":"2024-09-11T07:34:10.544235+0000","last_became_active":"2024-09-11T07:26:35.555862+0000","last_became_peered":"2024-09-11T07:26:35.555862+0000","last_unstale":"2024-09-11T07:34:10.544235+0000","last_undegraded":"2024-09-11T07:34:10.544235+0000","last_fullsized":"2024-09-11T07:34:10.544235+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:01.366571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:01.366571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:19.675146+0000","scrub_duration":0.0059672780000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":174,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.560088+0000","last_change":"2024-09-11T07:33:17.489228+0000","last_active":"2024-09-11T07:34:10.560088+0000","last_peered":"2024-09-11T07:34:10.560088+0000","last_clean":"2024-09-11T07:34:10.560088+0000","last_became_active":"2024-09-11T07:26:37.861647+0000","last_became_peered":"2024-09-11T07:26:37.861647+0000","last_unstale":"2024-09-11T07:34:10.560088+0000","last_undegraded":"2024-09-11T07:34:10.560088+0000","last_fullsized":"2024-09-11T07:34:10.560088+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:17.489174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:17.489174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:20.520796+0000","scrub_duration":0.0029380980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":197,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.559345+0000","last_change":"2024-09-11T07:33:09.499435+0000","last_active":"2024-09-11T07:34:10.559345+0000","last_peered":"2024-09-11T07:34:10.559345+0000","last_clean":"2024-09-11T07:34:10.559345+0000","last_became_active":"2024-09-11T07:32:00.502947+0000","last_became_peered":"2024-09-11T07:32:00.502947+0000","last_unstale":"2024-09-11T07:34:10.559345+0000","last_undegraded":"2024-09-11T07:34:10.559345+0000","last_fullsized":"2024-09-11T07:34:10.559345+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:09.499379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:09.499379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:25.553360+0000","scrub_duration":0.001485933,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"127'17","reported_seq":206,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.558105+0000","last_change":"2024-09-11T07:33:50.604574+0000","last_active":"2024-09-11T07:34:10.558105+0000","last_peered":"2024-09-11T07:34:10.558105+0000","last_clean":"2024-09-11T07:34:10.558105+0000","last_became_active":"2024-09-11T07:26:33.542711+0000","last_became_peered":"2024-09-11T07:26:33.542711+0000","last_unstale":"2024-09-11T07:34:10.558105+0000","last_undegraded":"2024-09-11T07:34:10.558105+0000","last_fullsized":"2024-09-11T07:34:10.558105+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'17","last_scrub_stamp":"2024-09-11T07:33:50.604493+0000","last_deep_scrub":"127'17","last_deep_scrub_stamp":"2024-09-11T07:30:30.260552+0000","last_clean_scrub_stamp":"2024-09-11T07:33:50.604493+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:03.711168+0000","scrub_duration":0.0045862999999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":185,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-11T07:34:10.539428+0000","last_change":"2024-09-11T07:34:07.375537+0000","last_active":"2024-09-11T07:34:10.539428+0000","last_peered":"2024-09-11T07:34:10.539428+0000","last_clean":"2024-09-11T07:34:10.539428+0000","last_became_active":"2024-09-11T07:26:35.562037+0000","last_became_peered":"2024-09-11T07:26:35.562037+0000","last_unstale":"2024-09-11T07:34:10.539428+0000","last_undegraded":"2024-09-11T07:34:10.539428+0000","last_fullsized":"2024-09-11T07:34:10.539428+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:07.375490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:07.375490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:16.626440+0000","scrub_duration":0.0042989400000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":4289986560,"num_objects":65460,"num_object_clones":0,"num_object_copies":196380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65460,"num_write_kb":4189440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12857769984,"data_stored":12857769984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65460,"ondisk_log_size":65460,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":798,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":474,"ondisk_log_size":474,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":455,"num_read_kb":450,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":147,"seq":631360192540,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1603476,"kb_used_data":1283148,"kb_used_omap":4,"kb_used_meta":320315,"kb_avail":92141676,"statfs":{"total":95995035648,"available":94353076224,"internally_reserved":0,"allocated":1313943552,"data_stored":1313410393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":328003078},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":3.2189999999999999,"15min":3.617},"min":{"1min":0.246,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":2.7250000000000001,"5min":49.573,"15min":49.573},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":2.9049999999999998,"15min":3.2869999999999999},"min":{"1min":0.311,"5min":0.22,"15min":0.22},"max":{"1min":2.1299999999999999,"5min":49.881,"15min":49.881},"last":5.0069999999999997}]},{"osd":1,"last update":"Wed Sep 11 07:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.65100000000000002,"15min":0.61499999999999999},"min":{"1min":0.29399999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.677,"5min":3.677,"15min":3.677},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.78200000000000003,"15min":0.79600000000000004},"min":{"1min":0.26000000000000001,"5min":0.246,"15min":0.246},"max":{"1min":1.919,"5min":4.6740000000000004,"15min":4.6740000000000004},"last":0.35499999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.92900000000000005,"15min":0.96799999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.141,"5min":3.6579999999999999,"15min":3.6579999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.90300000000000002,"15min":0.91300000000000003},"min":{"1min":0.318,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.141,"5min":5.0149999999999997,"15min":5.0149999999999997},"last":1.28}]},{"osd":3,"last update":"Wed Sep 11 07:34:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":3.0920000000000001,"15min":3.4300000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":4.4580000000000002,"5min":49.798999999999999,"15min":49.798999999999999},"last":0.317},{"interface":"front","average":{"1min":1.4570000000000001,"5min":3.0619999999999998,"15min":3.3300000000000001},"min":{"1min":0.32800000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":15.316000000000001,"5min":49.524000000000001,"15min":49.524000000000001},"last":0.44600000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.42299999999999999,"15min":0.41799999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.64200000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.62,"15min":0.60799999999999998},"min":{"1min":0.33800000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.9340000000000002,"5min":3.9340000000000002,"15min":3.9340000000000002},"last":0.52200000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.46200000000000002,"15min":0.44700000000000001},"min":{"1min":0.32300000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.374},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.64500000000000002,"15min":0.65300000000000002},"min":{"1min":0.33400000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.111,"5min":3.5640000000000001,"15min":3.5640000000000001},"last":0.40400000000000003}]},{"osd":6,"last update":"Wed Sep 11 07:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.54000000000000004,"15min":0.52000000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.55800000000000005,"15min":0.54400000000000004},"min":{"1min":0.35399999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.503}]}]},{"osd":6,"up_from":44,"seq":188978561125,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1600132,"kb_used_data":1294456,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":92145020,"statfs":{"total":95995035648,"available":94356500480,"internally_reserved":0,"allocated":1325522944,"data_stored":1325010498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:34:07 2024","interfaces":[{"interface":"back","average":{"1min":7.5389999999999997,"5min":3.7440000000000002,"15min":1.5680000000000001},"min":{"1min":0.26500000000000001,"5min":0.192,"15min":0.192},"max":{"1min":142.29599999999999,"5min":142.29599999999999,"15min":142.29599999999999},"last":3.2469999999999999},{"interface":"front","average":{"1min":7.5039999999999996,"5min":3.819,"15min":1.6799999999999999},"min":{"1min":0.39800000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":142.68799999999999,"5min":142.68799999999999,"15min":142.68799999999999},"last":0.41099999999999998}]},{"osd":1,"last update":"Wed Sep 11 07:34:07 2024","interfaces":[{"interface":"back","average":{"1min":6.8109999999999999,"5min":2.0009999999999999,"15min":1.0629999999999999},"min":{"1min":0.317,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":127.92100000000001,"5min":127.92100000000001,"15min":127.92100000000001},"last":0.253},{"interface":"front","average":{"1min":6.8289999999999997,"5min":2.226,"15min":1.0740000000000001},"min":{"1min":0.24099999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":127.746,"5min":127.746,"15min":127.746},"last":0.70099999999999996}]},{"osd":2,"last update":"Wed Sep 11 07:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":1.1160000000000001,"15min":0.78300000000000003},"min":{"1min":0.29399999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.748,"5min":26.491,"15min":26.491},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":1.355,"15min":0.84999999999999998},"min":{"1min":0.36399999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.7969999999999999,"5min":26.561,"15min":26.561},"last":1.512}]},{"osd":3,"last update":"Wed Sep 11 07:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":1.399,"15min":0.81100000000000005},"min":{"1min":0.27600000000000002,"5min":0.254,"15min":0.254},"max":{"1min":2.1970000000000001,"5min":31.268999999999998,"15min":31.268999999999998},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":1.718,"15min":0.998},"min":{"1min":0.34100000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.6479999999999999,"5min":31.872,"15min":31.872},"last":0.375}]},{"osd":4,"last update":"Wed Sep 11 07:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.439,"15min":0.53700000000000003},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":4.4720000000000004},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.435,"15min":0.42699999999999999},"min":{"1min":0.27700000000000002,"5min":0.25,"15min":0.25},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.55400000000000005}]},{"osd":5,"last update":"Wed Sep 11 07:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.53600000000000003,"15min":0.61899999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":2.9809999999999999,"5min":2.9809999999999999,"15min":5.7329999999999997},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.33500000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":3.4129999999999998,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":0.39500000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.309,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.514,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.70499999999999996,"15min":0.71699999999999997},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":2.9449999999999998,"5min":3.6030000000000002,"15min":3.6030000000000002},"last":0.43099999999999999}]}]},{"osd":5,"up_from":37,"seq":158913790059,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2348304,"kb_used_data":2046792,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":91396848,"statfs":{"total":95995035648,"available":93590372352,"internally_reserved":0,"allocated":2095915008,"data_stored":2095386802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:33:39 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":4.3449999999999998,"15min":1.788},"min":{"1min":0.253,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.65000000000000002,"5min":185.68199999999999,"15min":185.68199999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":4.556,"15min":1.8520000000000001},"min":{"1min":0.28199999999999997,"5min":0.218,"15min":0.218},"max":{"1min":0.59699999999999998,"5min":186.12200000000001,"15min":186.12200000000001},"last":0.35199999999999998}]},{"osd":1,"last update":"Wed Sep 11 07:33:39 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.93999999999999995,"15min":0.66500000000000004},"min":{"1min":0.33400000000000002,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":9.0969999999999995,"5min":9.0969999999999995,"15min":9.0969999999999995},"last":1.339},{"interface":"front","average":{"1min":0.496,"5min":1.069,"15min":0.68500000000000005},"min":{"1min":0.29099999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.81899999999999995,"5min":19.119,"15min":19.119},"last":1.0920000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:33:39 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":3.2389999999999999,"15min":1.4259999999999999},"min":{"1min":0.30199999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.64700000000000002,"5min":187.99199999999999,"15min":187.99199999999999},"last":1.319},{"interface":"front","average":{"1min":0.50800000000000001,"5min":3.1800000000000002,"15min":1.417},"min":{"1min":0.318,"5min":0.254,"15min":0.254},"max":{"1min":0.624,"5min":186.375,"15min":186.375},"last":0.503}]},{"osd":3,"last update":"Wed Sep 11 07:33:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":4.5960000000000001,"15min":1.889},"min":{"1min":0.39800000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.74099999999999999,"5min":188.11500000000001,"15min":188.11500000000001},"last":1.248},{"interface":"front","average":{"1min":0.53500000000000003,"5min":4.5519999999999996,"15min":1.8740000000000001},"min":{"1min":0.39200000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.69899999999999995,"5min":186.44200000000001,"15min":186.44200000000001},"last":0.52400000000000002}]},{"osd":4,"last update":"Wed Sep 11 07:33:39 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.44500000000000001,"15min":0.39700000000000002},"min":{"1min":0.30199999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.86099999999999999,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.435,"15min":0.42399999999999999},"min":{"1min":0.26600000000000001,"5min":0.253,"15min":0.23200000000000001},"max":{"1min":0.79800000000000004,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.2969999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:34:08 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.503,"15min":0.44400000000000001},"min":{"1min":0.32300000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.784,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":1.2769999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.51700000000000002,"15min":0.45800000000000002},"min":{"1min":0.28699999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":2.0369999999999999,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":0.47899999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.772,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61299999999999999,"15min":0.60599999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":2.226,"5min":2.226,"15min":2.226},"last":1.357}]}]},{"osd":4,"up_from":31,"seq":133143986287,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1593432,"kb_used_data":1287756,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":92151720,"statfs":{"total":95995035648,"available":94363361280,"internally_reserved":0,"allocated":1318662144,"data_stored":1318129685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":1.6080000000000001,"15min":0.88200000000000001},"min":{"1min":0.253,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":5.1020000000000003,"5min":35.308,"15min":35.308},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.52900000000000003,"5min":1.4279999999999999,"15min":0.80900000000000005},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.95799999999999996,"5min":35.597999999999999,"15min":35.597999999999999},"last":0.34300000000000003}]},{"osd":1,"last update":"Wed Sep 11 07:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":1.1120000000000001,"15min":0.70299999999999996},"min":{"1min":0.28799999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":4.5330000000000004,"5min":9.4550000000000001,"15min":9.4550000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.30199999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.998,"5min":4.5780000000000003,"15min":4.5780000000000003},"last":0.70899999999999996}]},{"osd":2,"last update":"Wed Sep 11 07:33:29 2024","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.3120000000000001,"15min":0.79200000000000004},"min":{"1min":0.29499999999999998,"5min":0.25800000000000001,"15min":0.22600000000000001},"max":{"1min":11.425000000000001,"5min":19.927,"15min":19.927},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":1.169,"15min":0.73499999999999999},"min":{"1min":0.308,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":3.0510000000000002,"5min":20.02,"15min":20.02},"last":0.27200000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":1.46,"15min":0.84399999999999997},"min":{"1min":0.378,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":9.0099999999999998,"5min":19.895,"15min":19.895},"last":3.7250000000000001},{"interface":"front","average":{"1min":0.75,"5min":1.2749999999999999,"15min":0.76300000000000001},"min":{"1min":0.34100000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":3.4169999999999998,"5min":20.260000000000002,"15min":20.260000000000002},"last":0.93500000000000005}]},{"osd":5,"last update":"Wed Sep 11 07:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.40200000000000002,"15min":0.36699999999999999},"min":{"1min":0.219,"5min":0.216,"15min":0.216},"max":{"1min":0.747,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.247},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.41699999999999998,"15min":0.39300000000000002},"min":{"1min":0.25,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.28799999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.44600000000000001,"15min":0.42799999999999999},"min":{"1min":0.32100000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.80900000000000005,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.45500000000000002,"15min":0.44},"min":{"1min":0.29599999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.81699999999999995,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.25800000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55900000000000005,"15min":0.55000000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.57099999999999995,"15min":0.55600000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":2.4020000000000001,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":0.32300000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182516,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1865760,"kb_used_data":1555924,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":91879392,"statfs":{"total":95995035648,"available":94084497408,"internally_reserved":0,"allocated":1593266176,"data_stored":1592724989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,2,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":2.202,"15min":0.99099999999999999},"min":{"1min":0.21199999999999999,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":2.4740000000000002,"5min":38.432000000000002,"15min":38.432000000000002},"last":3.5990000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":3.044,"15min":1.2989999999999999},"min":{"1min":0.21099999999999999,"5min":0.20300000000000001,"15min":0.19500000000000001},"max":{"1min":2.7650000000000001,"5min":91.876999999999995,"15min":91.876999999999995},"last":3.9380000000000002}]},{"osd":1,"last update":"Wed Sep 11 07:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":1.831,"15min":0.89600000000000002},"min":{"1min":0.23499999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":2.8170000000000002,"5min":10.475,"15min":10.475},"last":2.6970000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":1.4710000000000001,"15min":0.76900000000000002},"min":{"1min":0.25,"5min":0.215,"15min":0.215},"max":{"1min":1.998,"5min":8.1899999999999995,"15min":8.1899999999999995},"last":0.192}]},{"osd":2,"last update":"Wed Sep 11 07:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":1.843,"15min":0.91600000000000004},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.23200000000000001},"max":{"1min":1.9650000000000001,"5min":8.282,"15min":8.282},"last":1.2949999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":1.659,"15min":0.85799999999999998},"min":{"1min":0.26200000000000001,"5min":0.21199999999999999,"15min":0.20999999999999999},"max":{"1min":2.569,"5min":8.7110000000000003,"15min":8.7110000000000003},"last":3.274}]},{"osd":4,"last update":"Wed Sep 11 07:33:27 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.202,"15min":0.83499999999999996},"min":{"1min":0.307,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":5.5099999999999998,"5min":5.7670000000000003,"15min":21.856000000000002},"last":3.2389999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":1.429,"15min":0.96399999999999997},"min":{"1min":0.40600000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":2.0129999999999999,"5min":8.0709999999999997,"15min":19.146000000000001},"last":2.7589999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:33:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":1.544,"15min":0.88700000000000001},"min":{"1min":0.38700000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.1949999999999998,"5min":32.325000000000003,"15min":32.325000000000003},"last":3.3039999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":1.4730000000000001,"15min":0.92100000000000004},"min":{"1min":0.379,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.27,"5min":7.9870000000000001,"15min":18.416},"last":2.7330000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":1.617,"15min":1.4119999999999999},"min":{"1min":0.34899999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":2.5590000000000002,"5min":32.350999999999999,"15min":32.350999999999999},"last":0.315},{"interface":"front","average":{"1min":0.82599999999999996,"5min":1.631,"15min":1.4239999999999999},"min":{"1min":0.42399999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":2.609,"5min":32.402999999999999,"15min":32.402999999999999},"last":3.331}]},{"osd":7,"last update":"Wed Sep 11 07:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":1.2689999999999999,"15min":1.349},"min":{"1min":0.44600000000000001,"5min":0.308,"15min":0.308},"max":{"1min":2.8759999999999999,"5min":8.0429999999999993,"15min":8.0429999999999993},"last":2.7799999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":2.1659999999999999,"15min":2.3929999999999998},"min":{"1min":0.39400000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.5209999999999999,"5min":32.289999999999999,"15min":32.289999999999999},"last":0.38100000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411449,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1871100,"kb_used_data":1569588,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":91874052,"statfs":{"total":95995035648,"available":94079029248,"internally_reserved":0,"allocated":1607258112,"data_stored":1606749345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:33:39 2024","interfaces":[{"interface":"back","average":{"1min":3.4910000000000001,"5min":2.3260000000000001,"15min":1.0189999999999999},"min":{"1min":0.20499999999999999,"5min":0.183,"15min":0.17000000000000001},"max":{"1min":56.704000000000001,"5min":56.704000000000001,"15min":56.704000000000001},"last":2.75},{"interface":"front","average":{"1min":3.5139999999999998,"5min":2.3490000000000002,"15min":1.0329999999999999},"min":{"1min":0.27000000000000002,"5min":0.216,"15min":0.216},"max":{"1min":56.816000000000003,"5min":56.816000000000003,"15min":56.816000000000003},"last":0.52000000000000002}]},{"osd":1,"last update":"Wed Sep 11 07:33:39 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":1.6040000000000001,"15min":0.79300000000000004},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.23200000000000001},"max":{"1min":0.53700000000000003,"5min":22.175000000000001,"15min":22.175000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":1.728,"15min":0.83699999999999997},"min":{"1min":0.25700000000000001,"5min":0.216,"15min":0.20699999999999999},"max":{"1min":0.746,"5min":22.609999999999999,"15min":22.609999999999999},"last":0.63300000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:34:11 2024","interfaces":[{"interface":"back","average":{"1min":2.8420000000000001,"5min":2.4980000000000002,"15min":1.1160000000000001},"min":{"1min":0.33200000000000002,"5min":0.27400000000000002,"15min":0.26900000000000002},"max":{"1min":44.405999999999999,"5min":44.405999999999999,"15min":44.405999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":3.1960000000000002,"5min":2.649,"15min":1.1839999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":43.295999999999999,"5min":43.295999999999999,"15min":43.295999999999999},"last":0.54000000000000004}]},{"osd":4,"last update":"Wed Sep 11 07:33:26 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.7270000000000001,"15min":0.94599999999999995},"min":{"1min":0.34599999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":6.7599999999999998,"5min":30.946999999999999,"15min":30.946999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":1.4119999999999999,"15min":0.85399999999999998},"min":{"1min":0.44800000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.0960000000000001,"5min":22.434000000000001,"15min":22.434000000000001},"last":1.605}]},{"osd":5,"last update":"Wed Sep 11 07:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":1.7030000000000001,"15min":0.95399999999999996},"min":{"1min":0.38300000000000001,"5min":0.34000000000000002,"15min":0.31900000000000001},"max":{"1min":0.70899999999999996,"5min":31.789000000000001,"15min":31.789000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":1.6990000000000001,"15min":1.022},"min":{"1min":0.371,"5min":0.31,"15min":0.31},"max":{"1min":0.73199999999999998,"5min":30.898,"15min":30.898},"last":1.825}]},{"osd":6,"last update":"Wed Sep 11 07:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":1.6459999999999999,"15min":0.96799999999999997},"min":{"1min":0.432,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.7030000000000001,"5min":22.399000000000001,"15min":22.399000000000001},"last":1.726},{"interface":"front","average":{"1min":0.72299999999999998,"5min":1.5940000000000001,"15min":0.94999999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.714,"5min":25.739000000000001,"15min":25.739000000000001},"last":1.659}]},{"osd":7,"last update":"Wed Sep 11 07:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":2.0139999999999998,"15min":2.226},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.6719999999999999,"5min":22.484000000000002,"15min":22.484000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":2.331,"15min":2.593},"min":{"1min":0.438,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.7850000000000001,"5min":25.670000000000002,"15min":25.670000000000002},"last":0.60799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574972,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2121320,"kb_used_data":1807320,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":91623832,"statfs":{"total":95995035648,"available":93822803968,"internally_reserved":0,"allocated":1850695680,"data_stored":1850151211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":3.282,"15min":1.3160000000000001},"min":{"1min":0.192,"5min":0.192,"15min":0.186},"max":{"1min":3.7719999999999998,"5min":82.218999999999994,"15min":82.218999999999994},"last":0.34799999999999998},{"interface":"front","average":{"1min":1.4990000000000001,"5min":3.2349999999999999,"15min":1.3169999999999999},"min":{"1min":0.23599999999999999,"5min":0.191,"15min":0.191},"max":{"1min":4.6050000000000004,"5min":82.278999999999996,"15min":82.278999999999996},"last":0.57399999999999995}]},{"osd":2,"last update":"Wed Sep 11 07:33:40 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":1.6060000000000001,"15min":0.81000000000000005},"min":{"1min":0.27100000000000002,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":4.907,"5min":6.7610000000000001,"15min":6.7610000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":1.8580000000000001,"15min":0.89000000000000001},"min":{"1min":0.24199999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.633,"5min":8.9559999999999995,"15min":8.9559999999999995},"last":0.70099999999999996}]},{"osd":3,"last update":"Wed Sep 11 07:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":3.4409999999999998,"15min":1.4399999999999999},"min":{"1min":0.28000000000000003,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":5.4690000000000003,"5min":81.882999999999996,"15min":81.882999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.0820000000000001,"5min":3.488,"15min":1.4370000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.20999999999999999},"max":{"1min":7.2809999999999997,"5min":82.400000000000006,"15min":82.400000000000006},"last":0.318}]},{"osd":4,"last update":"Wed Sep 11 07:33:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0529999999999999,"15min":0.71499999999999997},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":4.0250000000000004,"5min":6.7309999999999999,"15min":6.7309999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.81000000000000005,"5min":1.3879999999999999,"15min":0.83299999999999996},"min":{"1min":0.34300000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.1680000000000001,"5min":8.9039999999999999,"15min":8.9039999999999999},"last":1.6220000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:33:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":1.159,"15min":0.753},"min":{"1min":0.38300000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.109,"5min":6.6909999999999998,"15min":6.6909999999999998},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":1.4099999999999999,"15min":0.84499999999999997},"min":{"1min":0.32400000000000001,"5min":0.308,"15min":0.308},"max":{"1min":1.758,"5min":8.8759999999999994,"15min":8.8759999999999994},"last":1.754}]},{"osd":6,"last update":"Wed Sep 11 07:34:05 2024","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.6839999999999999,"15min":0.94299999999999995},"min":{"1min":0.35799999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":5.3609999999999998,"5min":24.018000000000001,"15min":24.018000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":1.444,"15min":0.85099999999999998},"min":{"1min":0.42799999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.5430000000000001,"5min":23.681000000000001,"15min":23.681000000000001},"last":0.46700000000000003}]},{"osd":7,"last update":"Wed Sep 11 07:34:05 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.3080000000000001,"15min":1.3380000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":7.0940000000000003,"5min":7.0940000000000003,"15min":7.0940000000000003},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.462,"15min":1.5089999999999999},"min":{"1min":0.45700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":7.4050000000000002,"5min":7.4050000000000002,"15min":7.4050000000000002},"last":1.655}]}]},{"osd":0,"up_from":8,"seq":34359738498,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1891092,"kb_used_data":1581256,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":91854060,"statfs":{"total":95995035648,"available":94058557440,"internally_reserved":0,"allocated":1619206144,"data_stored":1618677646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":1.5149999999999999,"15min":0.74399999999999999},"min":{"1min":0.21199999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":2.2229999999999999,"5min":9.7710000000000008,"15min":9.7710000000000008},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":1.585,"15min":0.76300000000000001},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":3.8769999999999998,"5min":10.275,"15min":10.275},"last":0.34399999999999997}]},{"osd":2,"last update":"Wed Sep 11 07:33:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":1.9630000000000001,"15min":0.92200000000000004},"min":{"1min":0.23300000000000001,"5min":0.20399999999999999,"15min":0.20200000000000001},"max":{"1min":4.0570000000000004,"5min":19.527999999999999,"15min":19.527999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":1.964,"15min":0.92300000000000004},"min":{"1min":0.32300000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":3.391,"5min":19.556999999999999,"15min":19.556999999999999},"last":0.65200000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:34:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":3.6000000000000001,"15min":1.506},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":4.359,"5min":45.185000000000002,"15min":45.185000000000002},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":2.927,"15min":1.2849999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":2.7400000000000002,"5min":43.228999999999999,"15min":43.228999999999999},"last":0.56899999999999995}]},{"osd":4,"last update":"Wed Sep 11 07:33:19 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.355,"15min":0.82899999999999996},"min":{"1min":0.35899999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":6.8319999999999999,"5min":19.411999999999999,"15min":19.411999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":1.129,"5min":1.7090000000000001,"15min":0.96899999999999997},"min":{"1min":0.34000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":4.0960000000000001,"5min":19.396000000000001,"15min":19.396000000000001},"last":0.58999999999999997}]},{"osd":5,"last update":"Wed Sep 11 07:33:43 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.601,"15min":0.92000000000000004},"min":{"1min":0.371,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":6.9400000000000004,"5min":19.381,"15min":19.381},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.97599999999999998,"5min":1.8360000000000001,"15min":1.018},"min":{"1min":0.40899999999999997,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":4.1689999999999996,"5min":19.437000000000001,"15min":19.437000000000001},"last":0.84299999999999997}]},{"osd":6,"last update":"Wed Sep 11 07:34:13 2024","interfaces":[{"interface":"back","average":{"1min":2.048,"5min":1.9750000000000001,"15min":1.0509999999999999},"min":{"1min":0.442,"5min":0.316,"15min":0.316},"max":{"1min":23.190999999999999,"5min":23.190999999999999,"15min":23.190999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":2.0819999999999999,"5min":1.984,"15min":1.0600000000000001},"min":{"1min":0.47299999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":23.632000000000001,"5min":23.632000000000001,"15min":23.632000000000001},"last":0.67900000000000005}]},{"osd":7,"last update":"Wed Sep 11 07:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":1.641,"15min":1.7769999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":4.2350000000000003,"5min":5.3890000000000002,"15min":5.3890000000000002},"last":0.87},{"interface":"front","average":{"1min":0.754,"5min":1.6919999999999999,"15min":1.8480000000000001},"min":{"1min":0.52500000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.2010000000000001,"5min":5.2759999999999998,"15min":5.2759999999999998},"last":0.61799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1620377600,"data_stored":1620377600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1879375872,"data_stored":1879375872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1613430784,"data_stored":1613430784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1619001344,"data_stored":1619001344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1335558144,"data_stored":1335558144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2130313216,"data_stored":2130313216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1328349184,"data_stored":1328349184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1331363840,"data_stored":1331363840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:34:20.849 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:34:20.937 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:20.981 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:20.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:20.982 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:21.083 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:21.117 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:21.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:21.118 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:21.121 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:34:21.208 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 41 16 74359 74343 113.305 76.5625 0.0115486 0.00881954 2024-09-11T07:34:21.218 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:21.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:21.246 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:21.246 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:21.272 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:21 smithi022 ceph-mon[29462]: pgmap v346: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-11T07:34:21.272 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:21 smithi022 ceph-mon[37330]: pgmap v346: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-11T07:34:21.299 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-11T07:34:21.348 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:21.402 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:21.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:21.403 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:21.468 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:34:21.468 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:34:21.505 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:21 smithi104 ceph-mon[33507]: pgmap v346: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-11T07:34:21.532 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:21.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:21.533 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:21.549 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:34:21.549 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:34:21.582 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:34:21.582 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:34:21.613 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:34:21.613 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:34:21.634 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:21.645 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:34:21.645 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:34:21.660 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:34:21.675 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:34:21.675 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:34:21.677 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:21.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:21.678 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:21.706 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:34:21.706 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:34:21.738 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:34:21.739 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:34:21.779 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:21.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:21.810 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:21.810 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:21.911 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:21.942 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:21.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:21.943 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:22.044 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:22.078 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:22.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:22.079 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:22.180 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:22.207 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 42 16 75546 75530 112.373 74.1875 0.0191472 0.00889404 2024-09-11T07:34:22.209 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:22.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:22.210 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:22.311 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:22.344 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:22.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:22.345 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:22.363 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:22 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3772067555' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:34:22.364 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:22 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3772067555' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:34:22.445 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:22.478 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:22.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:22.479 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:22.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:22 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3772067555' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:34:22.580 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:22.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:22.611 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:22.612 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:22.713 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:22.742 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:22.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:22.743 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:22.844 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:22.876 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:22.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:22.877 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:22.978 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:23.019 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:23.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:23.020 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:23.121 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:23.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:23.158 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:23.207 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 43 16 76885 76869 111.706 83.6875 0.0198732 0.00894678 2024-09-11T07:34:23.259 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:23.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:23.291 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:23.291 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:23.320 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:23 smithi022 ceph-mon[29462]: pgmap v347: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.0 KiB/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-11T07:34:23.320 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:34:23.320 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:34:23.321 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:23 smithi022 ceph-mon[37330]: pgmap v347: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.0 KiB/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-11T07:34:23.321 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:34:23.321 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:34:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:34:23.392 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:23.414 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:23 smithi104 ceph-mon[33507]: pgmap v347: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.0 KiB/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-11T07:34:23.414 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:34:23.415 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:34:23.426 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:23.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:23.427 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:23.528 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:23.553 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:34:23.553 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:34:23.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:23.566 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:23.567 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:23.667 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:23.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:23.700 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:23.700 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:23.801 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:23.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:23.868 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:23.868 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:23.969 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:24.010 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:24.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:24.011 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:24.112 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:24.151 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:24.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:24.152 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:24.207 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 44 16 78172 78156 110.995 80.4375 0.0157349 0.00900408 2024-09-11T07:34:24.253 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:24.279 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:34:24 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:34:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:34:24.281 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:24.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:24.282 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:24.377 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-11T07:34:24.382 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:24.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:24.414 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:24.414 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:24.515 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:24.559 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:24.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:24.560 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:24.661 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:24.702 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:24.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:24.703 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:24.742 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:34:24.804 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:24.844 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:24.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:24.845 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:24.946 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:24.987 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:24.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:24.988 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:25.089 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:25.128 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:25.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:25.130 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:25.207 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 45 16 79387 79371 110.216 75.9375 0.00964521 0.00906797 2024-09-11T07:34:25.217 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is set 2024-09-11T07:34:25.231 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:25.276 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:25.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:25.277 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:25.322 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:34:25.363 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:25 smithi022 ceph-mon[29462]: pgmap v348: 148 pgs: 148 active+clean; 4.5 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-11T07:34:25.363 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:25 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2483971475' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:34:25.363 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:25 smithi022 ceph-mon[37330]: pgmap v348: 148 pgs: 148 active+clean; 4.5 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-11T07:34:25.363 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:25 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2483971475' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:34:25.378 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:25.429 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:25.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:25.430 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:25.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:25 smithi104 ceph-mon[33507]: pgmap v348: 148 pgs: 148 active+clean; 4.5 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-11T07:34:25.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:25 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2483971475' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:34:25.531 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:25.592 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:25.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:25.593 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:25.693 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:25.740 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:25.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:25.742 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:25.842 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:25.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:25.882 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:25.882 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:25.983 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:26.021 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:26.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:26.022 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:26.123 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:26.165 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:26.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:26.166 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:26.208 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 46 16 80565 80549 109.418 73.625 0.0152024 0.00913428 2024-09-11T07:34:26.267 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:26.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:26 smithi104 ceph-mon[33507]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:34:26.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:26 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2483971475' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:34:26.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:26 smithi104 ceph-mon[33507]: osdmap e200: 8 total, 8 up, 8 in 2024-09-11T07:34:26.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:26 smithi104 ceph-mon[33507]: osdmap e201: 8 total, 8 up, 8 in 2024-09-11T07:34:26.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:26.311 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:26.311 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:26.364 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:26 smithi022 ceph-mon[29462]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:34:26.364 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:26 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2483971475' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:34:26.364 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:26 smithi022 ceph-mon[29462]: osdmap e200: 8 total, 8 up, 8 in 2024-09-11T07:34:26.364 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:26 smithi022 ceph-mon[29462]: osdmap e201: 8 total, 8 up, 8 in 2024-09-11T07:34:26.364 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:26 smithi022 ceph-mon[37330]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:34:26.365 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:26 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2483971475' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:34:26.365 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:26 smithi022 ceph-mon[37330]: osdmap e200: 8 total, 8 up, 8 in 2024-09-11T07:34:26.365 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:26 smithi022 ceph-mon[37330]: osdmap e201: 8 total, 8 up, 8 in 2024-09-11T07:34:26.412 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:26.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:26.442 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:26.442 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:26.543 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:26.579 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:26.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:26.580 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:26.619 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:34:26.619 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:34:26.677 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:34:26.680 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:26.728 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:26.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:26.729 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:26.830 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:26.865 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:26.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:26.866 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:26.967 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:26.995 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:26.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:26.996 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:27.001 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:34:27.009 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:34:27.097 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:27.144 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:27.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:27.146 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:27.209 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 47 16 81683 81667 108.577 69.875 0.0212803 0.00920488 2024-09-11T07:34:27.246 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:27.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:27 smithi104 ceph-mon[33507]: pgmap v351: 148 pgs: 148 active+clean; 4.8 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-11T07:34:27.289 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:27.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:27.290 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:27.352 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-11T07:34:27.390 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:27.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:27 smithi022 ceph-mon[29462]: pgmap v351: 148 pgs: 148 active+clean; 4.8 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-11T07:34:27.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:27 smithi022 ceph-mon[37330]: pgmap v351: 148 pgs: 148 active+clean; 4.8 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-11T07:34:27.437 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:27.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:27.438 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:27.539 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:27.578 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:27.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:27.579 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:27.680 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:27.716 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:27.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:27.718 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:27.722 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:34:27.818 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:27.857 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:27.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:27.858 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:27.959 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:27.990 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:27.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:27.991 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:28.091 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:28.129 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T07:34:28.131 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:28.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:28.133 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:28.191 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2024-09-11T07:34:26.104061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5129050332,"num_objects":78450,"num_object_clones":0,"num_object_copies":235350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78450,"num_whiteouts":0,"num_read":1318,"num_read_kb":1175,"num_write":78797,"num_write_kb":5009927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":591166,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78897,"ondisk_log_size":78897,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":17291808,"kb_used_data":14840032,"kb_used_omap":16,"kb_used_meta":2451631,"kb_avail":732669408,"statfs":{"total":767960285184,"available":750253473792,"internally_reserved":0,"allocated":15196192768,"data_stored":15191920265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16530,"internal_metadata":2510471022},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":838467584,"num_objects":12794,"num_object_clones":0,"num_object_copies":38382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12794,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":12794,"num_write_kb":818816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002525"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":182,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.602291+0000","last_change":"2024-09-11T07:33:08.464899+0000","last_active":"2024-09-11T07:34:12.602291+0000","last_peered":"2024-09-11T07:34:12.602291+0000","last_clean":"2024-09-11T07:34:12.602291+0000","last_became_active":"2024-09-11T07:32:00.501534+0000","last_became_peered":"2024-09-11T07:32:00.501534+0000","last_unstale":"2024-09-11T07:34:12.602291+0000","last_undegraded":"2024-09-11T07:34:12.602291+0000","last_fullsized":"2024-09-11T07:34:12.602291+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:08.464835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:08.464835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:24.698410+0000","scrub_duration":0.002103452,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":207,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.595962+0000","last_change":"2024-09-11T07:34:09.538289+0000","last_active":"2024-09-11T07:34:12.595962+0000","last_peered":"2024-09-11T07:34:12.595962+0000","last_clean":"2024-09-11T07:34:12.595962+0000","last_became_active":"2024-09-11T07:32:00.508572+0000","last_became_peered":"2024-09-11T07:32:00.508572+0000","last_unstale":"2024-09-11T07:34:12.595962+0000","last_undegraded":"2024-09-11T07:34:12.595962+0000","last_fullsized":"2024-09-11T07:34:12.595962+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:09.538205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:09.538205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:23.803280+0000","scrub_duration":0.0045320439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"82'22","reported_seq":211,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.599004+0000","last_change":"2024-09-11T07:33:53.615330+0000","last_active":"2024-09-11T07:34:12.599004+0000","last_peered":"2024-09-11T07:34:12.599004+0000","last_clean":"2024-09-11T07:34:12.599004+0000","last_became_active":"2024-09-11T07:26:33.544627+0000","last_became_peered":"2024-09-11T07:26:33.544627+0000","last_unstale":"2024-09-11T07:34:12.599004+0000","last_undegraded":"2024-09-11T07:34:12.599004+0000","last_fullsized":"2024-09-11T07:34:12.599004+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-09-11T07:33:53.615231+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:33:53.615231+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:22.109465+0000","scrub_duration":0.011220367,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":186,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.588709+0000","last_change":"2024-09-11T07:33:18.247208+0000","last_active":"2024-09-11T07:34:12.588709+0000","last_peered":"2024-09-11T07:34:12.588709+0000","last_clean":"2024-09-11T07:34:12.588709+0000","last_became_active":"2024-09-11T07:26:35.556296+0000","last_became_peered":"2024-09-11T07:26:35.556296+0000","last_unstale":"2024-09-11T07:34:12.588709+0000","last_undegraded":"2024-09-11T07:34:12.588709+0000","last_fullsized":"2024-09-11T07:34:12.588709+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:18.247138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:18.247138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:43.507240+0000","scrub_duration":0.001401198,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":181,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.598550+0000","last_change":"2024-09-11T07:31:39.275089+0000","last_active":"2024-09-11T07:34:12.598550+0000","last_peered":"2024-09-11T07:34:12.598550+0000","last_clean":"2024-09-11T07:34:12.598550+0000","last_became_active":"2024-09-11T07:26:31.420178+0000","last_became_peered":"2024-09-11T07:26:31.420178+0000","last_unstale":"2024-09-11T07:34:12.598550+0000","last_undegraded":"2024-09-11T07:34:12.598550+0000","last_fullsized":"2024-09-11T07:34:12.598550+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:31:39.275011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:31:39.275011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0095928969999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"86'13","reported_seq":198,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.588464+0000","last_change":"2024-09-11T07:31:33.998004+0000","last_active":"2024-09-11T07:34:12.588464+0000","last_peered":"2024-09-11T07:34:12.588464+0000","last_clean":"2024-09-11T07:34:12.588464+0000","last_became_active":"2024-09-11T07:26:33.540732+0000","last_became_peered":"2024-09-11T07:26:33.540732+0000","last_unstale":"2024-09-11T07:34:12.588464+0000","last_undegraded":"2024-09-11T07:34:12.588464+0000","last_fullsized":"2024-09-11T07:34:12.588464+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'13","last_scrub_stamp":"2024-09-11T07:31:33.997949+0000","last_deep_scrub":"86'13","last_deep_scrub_stamp":"2024-09-11T07:28:15.025757+0000","last_clean_scrub_stamp":"2024-09-11T07:31:33.997949+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016545030000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":195,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.590061+0000","last_change":"2024-09-11T07:33:49.237639+0000","last_active":"2024-09-11T07:34:12.590061+0000","last_peered":"2024-09-11T07:34:12.590061+0000","last_clean":"2024-09-11T07:34:12.590061+0000","last_became_active":"2024-09-11T07:32:00.513650+0000","last_became_peered":"2024-09-11T07:32:00.513650+0000","last_unstale":"2024-09-11T07:34:12.590061+0000","last_undegraded":"2024-09-11T07:34:12.590061+0000","last_fullsized":"2024-09-11T07:34:12.590061+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:49.237335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:33:49.237335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:54.939660+0000","scrub_duration":0.0052103979999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":174,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.588659+0000","last_change":"2024-09-11T07:31:49.018958+0000","last_active":"2024-09-11T07:34:12.588659+0000","last_peered":"2024-09-11T07:34:12.588659+0000","last_clean":"2024-09-11T07:34:12.588659+0000","last_became_active":"2024-09-11T07:26:37.566910+0000","last_became_peered":"2024-09-11T07:26:37.566910+0000","last_unstale":"2024-09-11T07:34:12.588659+0000","last_undegraded":"2024-09-11T07:34:12.588659+0000","last_fullsized":"2024-09-11T07:34:12.588659+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:31:49.018877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:31:49.018877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072333409999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":186,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.602960+0000","last_change":"2024-09-11T07:33:57.319606+0000","last_active":"2024-09-11T07:34:12.602960+0000","last_peered":"2024-09-11T07:34:12.602960+0000","last_clean":"2024-09-11T07:34:12.602960+0000","last_became_active":"2024-09-11T07:32:00.516662+0000","last_became_peered":"2024-09-11T07:32:00.516662+0000","last_unstale":"2024-09-11T07:34:12.602960+0000","last_undegraded":"2024-09-11T07:34:12.602960+0000","last_fullsized":"2024-09-11T07:34:12.602960+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:57.319449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:57.319449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:11.083627+0000","scrub_duration":0.007631344,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"82'7","reported_seq":200,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.588964+0000","last_change":"2024-09-11T07:31:43.035948+0000","last_active":"2024-09-11T07:34:12.588964+0000","last_peered":"2024-09-11T07:34:12.588964+0000","last_clean":"2024-09-11T07:34:12.588964+0000","last_became_active":"2024-09-11T07:26:33.540351+0000","last_became_peered":"2024-09-11T07:26:33.540351+0000","last_unstale":"2024-09-11T07:34:12.588964+0000","last_undegraded":"2024-09-11T07:34:12.588964+0000","last_fullsized":"2024-09-11T07:34:12.588964+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'7","last_scrub_stamp":"2024-09-11T07:31:43.035896+0000","last_deep_scrub":"82'7","last_deep_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_clean_scrub_stamp":"2024-09-11T07:31:43.035896+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040919600000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":152,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.601083+0000","last_change":"2024-09-11T07:33:07.124141+0000","last_active":"2024-09-11T07:34:12.601083+0000","last_peered":"2024-09-11T07:34:12.601083+0000","last_clean":"2024-09-11T07:34:12.601083+0000","last_became_active":"2024-09-11T07:32:00.513206+0000","last_became_peered":"2024-09-11T07:32:00.513206+0000","last_unstale":"2024-09-11T07:34:12.601083+0000","last_undegraded":"2024-09-11T07:34:12.601083+0000","last_fullsized":"2024-09-11T07:34:12.601083+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:07.124068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.973323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:07.124068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:29.008509+0000","scrub_duration":0.0019801810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":175,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.595009+0000","last_change":"2024-09-11T07:34:03.257034+0000","last_active":"2024-09-11T07:34:12.595009+0000","last_peered":"2024-09-11T07:34:12.595009+0000","last_clean":"2024-09-11T07:34:12.595009+0000","last_became_active":"2024-09-11T07:26:37.561758+0000","last_became_peered":"2024-09-11T07:26:37.561758+0000","last_unstale":"2024-09-11T07:34:12.595009+0000","last_undegraded":"2024-09-11T07:34:12.595009+0000","last_fullsized":"2024-09-11T07:34:12.595009+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:03.256953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:03.256953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:07.281155+0000","scrub_duration":0.0085144460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":151,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.595156+0000","last_change":"2024-09-11T07:34:01.257321+0000","last_active":"2024-09-11T07:34:12.595156+0000","last_peered":"2024-09-11T07:34:12.595156+0000","last_clean":"2024-09-11T07:34:12.595156+0000","last_became_active":"2024-09-11T07:32:00.505709+0000","last_became_peered":"2024-09-11T07:32:00.505709+0000","last_unstale":"2024-09-11T07:34:12.595156+0000","last_undegraded":"2024-09-11T07:34:12.595156+0000","last_fullsized":"2024-09-11T07:34:12.595156+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:01.257176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_clean_scrub_stamp":"2024-09-11T07:34:01.257176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:03.761234+0000","scrub_duration":0.0018917890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"136'13","reported_seq":208,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.588917+0000","last_change":"2024-09-11T07:34:06.406309+0000","last_active":"2024-09-11T07:34:12.588917+0000","last_peered":"2024-09-11T07:34:12.588917+0000","last_clean":"2024-09-11T07:34:12.588917+0000","last_became_active":"2024-09-11T07:26:33.544386+0000","last_became_peered":"2024-09-11T07:26:33.544386+0000","last_unstale":"2024-09-11T07:34:12.588917+0000","last_undegraded":"2024-09-11T07:34:12.588917+0000","last_fullsized":"2024-09-11T07:34:12.588917+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'13","last_scrub_stamp":"2024-09-11T07:34:06.406196+0000","last_deep_scrub":"136'13","last_deep_scrub_stamp":"2024-09-11T07:31:08.042198+0000","last_clean_scrub_stamp":"2024-09-11T07:34:06.406196+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:08.075604+0000","scrub_duration":0.0079557350000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":197,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.595026+0000","last_change":"2024-09-11T07:33:49.134068+0000","last_active":"2024-09-11T07:34:12.595026+0000","last_peered":"2024-09-11T07:34:12.595026+0000","last_clean":"2024-09-11T07:34:12.595026+0000","last_became_active":"2024-09-11T07:32:00.502337+0000","last_became_peered":"2024-09-11T07:32:00.502337+0000","last_unstale":"2024-09-11T07:34:12.595026+0000","last_undegraded":"2024-09-11T07:34:12.595026+0000","last_fullsized":"2024-09-11T07:34:12.595026+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:49.133995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:49.133995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:52.916415+0000","scrub_duration":0.0028580369999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":191,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.593484+0000","last_change":"2024-09-11T07:33:06.832611+0000","last_active":"2024-09-11T07:34:12.593484+0000","last_peered":"2024-09-11T07:34:12.593484+0000","last_clean":"2024-09-11T07:34:12.593484+0000","last_became_active":"2024-09-11T07:32:00.530335+0000","last_became_peered":"2024-09-11T07:32:00.530335+0000","last_unstale":"2024-09-11T07:34:12.593484+0000","last_undegraded":"2024-09-11T07:34:12.593484+0000","last_fullsized":"2024-09-11T07:34:12.593484+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:06.832540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:06.832540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020659680000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"199'5007","reported_seq":5035,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T07:34:24.189611+0000","last_change":"2024-09-11T07:33:29.155481+0000","last_active":"2024-09-11T07:34:24.189611+0000","last_peered":"2024-09-11T07:34:24.189611+0000","last_clean":"2024-09-11T07:34:24.189611+0000","last_became_active":"2024-09-11T07:33:29.155187+0000","last_became_peered":"2024-09-11T07:33:29.155187+0000","last_unstale":"2024-09-11T07:34:24.189611+0000","last_undegraded":"2024-09-11T07:34:24.189611+0000","last_fullsized":"2024-09-11T07:34:24.189611+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":5007,"log_dups_size":0,"ondisk_log_size":5007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:56.196806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328138752,"num_objects":5007,"num_object_clones":0,"num_object_copies":15021,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5007,"num_write_kb":320448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":272,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.589277+0000","last_change":"2024-09-11T07:34:11.315441+0000","last_active":"2024-09-11T07:34:12.589277+0000","last_peered":"2024-09-11T07:34:12.589277+0000","last_clean":"2024-09-11T07:34:12.589277+0000","last_became_active":"2024-09-11T07:29:51.482755+0000","last_became_peered":"2024-09-11T07:29:51.482755+0000","last_unstale":"2024-09-11T07:34:12.589277+0000","last_undegraded":"2024-09-11T07:34:12.589277+0000","last_fullsized":"2024-09-11T07:34:12.589277+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-09-11T07:34:11.315334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:34:11.315334+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:17.690755+0000","scrub_duration":0.0071604249999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":181,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.602026+0000","last_change":"2024-09-11T07:33:52.272651+0000","last_active":"2024-09-11T07:34:12.602026+0000","last_peered":"2024-09-11T07:34:12.602026+0000","last_clean":"2024-09-11T07:34:12.602026+0000","last_became_active":"2024-09-11T07:32:00.507191+0000","last_became_peered":"2024-09-11T07:32:00.507191+0000","last_unstale":"2024-09-11T07:34:12.602026+0000","last_undegraded":"2024-09-11T07:34:12.602026+0000","last_fullsized":"2024-09-11T07:34:12.602026+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:52.272573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:52.272573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:17.508902+0000","scrub_duration":0.0023059880000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":185,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.590116+0000","last_change":"2024-09-11T07:34:00.348790+0000","last_active":"2024-09-11T07:34:12.590116+0000","last_peered":"2024-09-11T07:34:12.590116+0000","last_clean":"2024-09-11T07:34:12.590116+0000","last_became_active":"2024-09-11T07:26:35.557989+0000","last_became_peered":"2024-09-11T07:26:35.557989+0000","last_unstale":"2024-09-11T07:34:12.590116+0000","last_undegraded":"2024-09-11T07:34:12.590116+0000","last_fullsized":"2024-09-11T07:34:12.590116+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:00.348684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:00.348684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:30.171094+0000","scrub_duration":0.0036577580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":185,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.602578+0000","last_change":"2024-09-11T07:33:57.219099+0000","last_active":"2024-09-11T07:34:12.602578+0000","last_peered":"2024-09-11T07:34:12.602578+0000","last_clean":"2024-09-11T07:34:12.602578+0000","last_became_active":"2024-09-11T07:26:37.564554+0000","last_became_peered":"2024-09-11T07:26:37.564554+0000","last_unstale":"2024-09-11T07:34:12.602578+0000","last_undegraded":"2024-09-11T07:34:12.602578+0000","last_fullsized":"2024-09-11T07:34:12.602578+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:57.219012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:57.219012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:26.433401+0000","scrub_duration":0.0062201569999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"199'4877","reported_seq":4907,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T07:34:24.573728+0000","last_change":"2024-09-11T07:33:29.161604+0000","last_active":"2024-09-11T07:34:24.573728+0000","last_peered":"2024-09-11T07:34:24.573728+0000","last_clean":"2024-09-11T07:34:24.573728+0000","last_became_active":"2024-09-11T07:33:29.161349+0000","last_became_peered":"2024-09-11T07:33:29.161349+0000","last_unstale":"2024-09-11T07:34:24.573728+0000","last_undegraded":"2024-09-11T07:34:24.573728+0000","last_fullsized":"2024-09-11T07:34:24.573728+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4877,"log_dups_size":0,"ondisk_log_size":4877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:37.804059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319619072,"num_objects":4877,"num_object_clones":0,"num_object_copies":14631,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4877,"num_write_kb":312128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"112'15","reported_seq":208,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.589100+0000","last_change":"2024-09-11T07:34:08.373729+0000","last_active":"2024-09-11T07:34:12.589100+0000","last_peered":"2024-09-11T07:34:12.589100+0000","last_clean":"2024-09-11T07:34:12.589100+0000","last_became_active":"2024-09-11T07:26:33.535411+0000","last_became_peered":"2024-09-11T07:26:33.535411+0000","last_unstale":"2024-09-11T07:34:12.589100+0000","last_undegraded":"2024-09-11T07:34:12.589100+0000","last_fullsized":"2024-09-11T07:34:12.589100+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"112'15","last_scrub_stamp":"2024-09-11T07:34:08.373652+0000","last_deep_scrub":"112'15","last_deep_scrub_stamp":"2024-09-11T07:31:10.005896+0000","last_clean_scrub_stamp":"2024-09-11T07:34:08.373652+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:35.031237+0000","scrub_duration":0.0048197400000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":199,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.591796+0000","last_change":"2024-09-11T07:33:50.250361+0000","last_active":"2024-09-11T07:34:12.591796+0000","last_peered":"2024-09-11T07:34:12.591796+0000","last_clean":"2024-09-11T07:34:12.591796+0000","last_became_active":"2024-09-11T07:32:00.506615+0000","last_became_peered":"2024-09-11T07:32:00.506615+0000","last_unstale":"2024-09-11T07:34:12.591796+0000","last_undegraded":"2024-09-11T07:34:12.591796+0000","last_fullsized":"2024-09-11T07:34:12.591796+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:50.250289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:50.250289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:01.601495+0000","scrub_duration":0.0080459799999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"151'10","reported_seq":298,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T07:34:15.544854+0000","last_change":"2024-09-11T07:33:01.124548+0000","last_active":"2024-09-11T07:34:15.544854+0000","last_peered":"2024-09-11T07:34:15.544854+0000","last_clean":"2024-09-11T07:34:15.544854+0000","last_became_active":"2024-09-11T07:32:00.507447+0000","last_became_peered":"2024-09-11T07:32:00.507447+0000","last_unstale":"2024-09-11T07:34:15.544854+0000","last_undegraded":"2024-09-11T07:34:15.544854+0000","last_fullsized":"2024-09-11T07:34:15.544854+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:33:01.124483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:01.124483+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020248890000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":174,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.589347+0000","last_change":"2024-09-11T07:31:44.075858+0000","last_active":"2024-09-11T07:34:12.589347+0000","last_peered":"2024-09-11T07:34:12.589347+0000","last_clean":"2024-09-11T07:34:12.589347+0000","last_became_active":"2024-09-11T07:26:37.566919+0000","last_became_peered":"2024-09-11T07:26:37.566919+0000","last_unstale":"2024-09-11T07:34:12.589347+0000","last_undegraded":"2024-09-11T07:34:12.589347+0000","last_fullsized":"2024-09-11T07:34:12.589347+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:31:44.075772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:31:44.075772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053001849999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"199'4882","reported_seq":4911,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T07:34:24.215559+0000","last_change":"2024-09-11T07:33:29.154369+0000","last_active":"2024-09-11T07:34:24.215559+0000","last_peered":"2024-09-11T07:34:24.215559+0000","last_clean":"2024-09-11T07:34:24.215559+0000","last_became_active":"2024-09-11T07:33:29.154103+0000","last_became_peered":"2024-09-11T07:33:29.154103+0000","last_unstale":"2024-09-11T07:34:24.215559+0000","last_undegraded":"2024-09-11T07:34:24.215559+0000","last_fullsized":"2024-09-11T07:34:24.215559+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4882,"log_dups_size":0,"ondisk_log_size":4882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:29.121899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319946752,"num_objects":4882,"num_object_clones":0,"num_object_copies":14646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4882,"num_write_kb":312448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"83'22","reported_seq":219,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.595216+0000","last_change":"2024-09-11T07:34:04.375654+0000","last_active":"2024-09-11T07:34:12.595216+0000","last_peered":"2024-09-11T07:34:12.595216+0000","last_clean":"2024-09-11T07:34:12.595216+0000","last_became_active":"2024-09-11T07:26:33.549733+0000","last_became_peered":"2024-09-11T07:26:33.549733+0000","last_unstale":"2024-09-11T07:34:12.595216+0000","last_undegraded":"2024-09-11T07:34:12.595216+0000","last_fullsized":"2024-09-11T07:34:12.595216+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'22","last_scrub_stamp":"2024-09-11T07:34:04.375457+0000","last_deep_scrub":"83'22","last_deep_scrub_stamp":"2024-09-11T07:34:04.375457+0000","last_clean_scrub_stamp":"2024-09-11T07:34:04.375457+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:29.572580+0000","scrub_duration":0.0086024310000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":209,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.590253+0000","last_change":"2024-09-11T07:33:59.343890+0000","last_active":"2024-09-11T07:34:12.590253+0000","last_peered":"2024-09-11T07:34:12.590253+0000","last_clean":"2024-09-11T07:34:12.590253+0000","last_became_active":"2024-09-11T07:32:00.513472+0000","last_became_peered":"2024-09-11T07:32:00.513472+0000","last_unstale":"2024-09-11T07:34:12.590253+0000","last_undegraded":"2024-09-11T07:34:12.590253+0000","last_fullsized":"2024-09-11T07:34:12.590253+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:59.343676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:59.343676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:04.415703+0000","scrub_duration":0.0067031579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":281,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T07:34:15.536884+0000","last_change":"2024-09-11T07:31:33.055929+0000","last_active":"2024-09-11T07:34:15.536884+0000","last_peered":"2024-09-11T07:34:15.536884+0000","last_clean":"2024-09-11T07:34:15.536884+0000","last_became_active":"2024-09-11T07:26:35.560406+0000","last_became_peered":"2024-09-11T07:26:35.560406+0000","last_unstale":"2024-09-11T07:34:15.536884+0000","last_undegraded":"2024-09-11T07:34:15.536884+0000","last_fullsized":"2024-09-11T07:34:15.536884+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:31:33.055836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:31:33.055836+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010749418,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":175,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.595539+0000","last_change":"2024-09-11T07:33:07.153745+0000","last_active":"2024-09-11T07:34:12.595539+0000","last_peered":"2024-09-11T07:34:12.595539+0000","last_clean":"2024-09-11T07:34:12.595539+0000","last_became_active":"2024-09-11T07:26:37.568005+0000","last_became_peered":"2024-09-11T07:26:37.568005+0000","last_unstale":"2024-09-11T07:34:12.595539+0000","last_undegraded":"2024-09-11T07:34:12.595539+0000","last_fullsized":"2024-09-11T07:34:12.595539+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:07.153674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:31.200685+0000","last_clean_scrub_stamp":"2024-09-11T07:33:07.153674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:20.217755+0000","scrub_duration":0.001744982,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"199'4837","reported_seq":4866,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T07:34:24.004554+0000","last_change":"2024-09-11T07:33:29.154414+0000","last_active":"2024-09-11T07:34:24.004554+0000","last_peered":"2024-09-11T07:34:24.004554+0000","last_clean":"2024-09-11T07:34:24.004554+0000","last_became_active":"2024-09-11T07:33:29.154117+0000","last_became_peered":"2024-09-11T07:33:29.154117+0000","last_unstale":"2024-09-11T07:34:24.004554+0000","last_undegraded":"2024-09-11T07:34:24.004554+0000","last_fullsized":"2024-09-11T07:34:24.004554+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4837,"log_dups_size":0,"ondisk_log_size":4837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:37.755024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":316997632,"num_objects":4837,"num_object_clones":0,"num_object_copies":14511,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4837,"num_write_kb":309568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"107'16","reported_seq":212,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.589556+0000","last_change":"2024-09-11T07:33:15.278908+0000","last_active":"2024-09-11T07:34:12.589556+0000","last_peered":"2024-09-11T07:34:12.589556+0000","last_clean":"2024-09-11T07:34:12.589556+0000","last_became_active":"2024-09-11T07:26:33.540777+0000","last_became_peered":"2024-09-11T07:26:33.540777+0000","last_unstale":"2024-09-11T07:34:12.589556+0000","last_undegraded":"2024-09-11T07:34:12.589556+0000","last_fullsized":"2024-09-11T07:34:12.589556+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'16","last_scrub_stamp":"2024-09-11T07:33:15.278856+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-09-11T07:29:38.112111+0000","last_clean_scrub_stamp":"2024-09-11T07:33:15.278856+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:17.908263+0000","scrub_duration":0.0016764620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":175,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.602815+0000","last_change":"2024-09-11T07:33:17.262636+0000","last_active":"2024-09-11T07:34:12.602815+0000","last_peered":"2024-09-11T07:34:12.602815+0000","last_clean":"2024-09-11T07:34:12.602815+0000","last_became_active":"2024-09-11T07:32:00.518538+0000","last_became_peered":"2024-09-11T07:32:00.518538+0000","last_unstale":"2024-09-11T07:34:12.602815+0000","last_undegraded":"2024-09-11T07:34:12.602815+0000","last_fullsized":"2024-09-11T07:34:12.602815+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:17.262578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:17.262578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:32.326642+0000","scrub_duration":0.003026361,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":187,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.596543+0000","last_change":"2024-09-11T07:33:10.096952+0000","last_active":"2024-09-11T07:34:12.596543+0000","last_peered":"2024-09-11T07:34:12.596543+0000","last_clean":"2024-09-11T07:34:12.596543+0000","last_became_active":"2024-09-11T07:26:35.569081+0000","last_became_peered":"2024-09-11T07:26:35.569081+0000","last_unstale":"2024-09-11T07:34:12.596543+0000","last_undegraded":"2024-09-11T07:34:12.596543+0000","last_fullsized":"2024-09-11T07:34:12.596543+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:10.096886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:10.096886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:32.795036+0000","scrub_duration":0.0032884910000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":175,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.599919+0000","last_change":"2024-09-11T07:31:45.261381+0000","last_active":"2024-09-11T07:34:12.599919+0000","last_peered":"2024-09-11T07:34:12.599919+0000","last_clean":"2024-09-11T07:34:12.599919+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:34:12.599919+0000","last_undegraded":"2024-09-11T07:34:12.599919+0000","last_fullsized":"2024-09-11T07:34:12.599919+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:31:45.261282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:31:45.261282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086095340000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"199'4912","reported_seq":4942,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T07:34:23.949831+0000","last_change":"2024-09-11T07:33:29.205385+0000","last_active":"2024-09-11T07:34:23.949831+0000","last_peered":"2024-09-11T07:34:23.949831+0000","last_clean":"2024-09-11T07:34:23.949831+0000","last_became_active":"2024-09-11T07:33:29.205097+0000","last_became_peered":"2024-09-11T07:33:29.205097+0000","last_unstale":"2024-09-11T07:34:23.949831+0000","last_undegraded":"2024-09-11T07:34:23.949831+0000","last_fullsized":"2024-09-11T07:34:23.949831+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4912,"log_dups_size":0,"ondisk_log_size":4912,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:54.401575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":321912832,"num_objects":4912,"num_object_clones":0,"num_object_copies":14736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4912,"num_write_kb":314368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"64'14","reported_seq":199,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.588603+0000","last_change":"2024-09-11T07:31:46.078940+0000","last_active":"2024-09-11T07:34:12.588603+0000","last_peered":"2024-09-11T07:34:12.588603+0000","last_clean":"2024-09-11T07:34:12.588603+0000","last_became_active":"2024-09-11T07:26:33.537850+0000","last_became_peered":"2024-09-11T07:26:33.537850+0000","last_unstale":"2024-09-11T07:34:12.588603+0000","last_undegraded":"2024-09-11T07:34:12.588603+0000","last_fullsized":"2024-09-11T07:34:12.588603+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-09-11T07:31:46.078888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:31:46.078888+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017755819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":201,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.594697+0000","last_change":"2024-09-11T07:33:04.366549+0000","last_active":"2024-09-11T07:34:12.594697+0000","last_peered":"2024-09-11T07:34:12.594697+0000","last_clean":"2024-09-11T07:34:12.594697+0000","last_became_active":"2024-09-11T07:28:09.014995+0000","last_became_peered":"2024-09-11T07:28:09.014995+0000","last_unstale":"2024-09-11T07:34:12.594697+0000","last_undegraded":"2024-09-11T07:34:12.594697+0000","last_fullsized":"2024-09-11T07:34:12.594697+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:04.366473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:04.366473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:28.386907+0000","scrub_duration":0.001832272,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"84'9","reported_seq":281,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T07:34:19.591516+0000","last_change":"2024-09-11T07:33:48.510872+0000","last_active":"2024-09-11T07:34:19.591516+0000","last_peered":"2024-09-11T07:34:19.591516+0000","last_clean":"2024-09-11T07:34:19.591516+0000","last_became_active":"2024-09-11T07:26:35.562997+0000","last_became_peered":"2024-09-11T07:26:35.562997+0000","last_unstale":"2024-09-11T07:34:19.591516+0000","last_undegraded":"2024-09-11T07:34:19.591516+0000","last_fullsized":"2024-09-11T07:34:19.591516+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-11T07:33:48.510778+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-11T07:28:05.653237+0000","last_clean_scrub_stamp":"2024-09-11T07:33:48.510778+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:52.240234+0000","scrub_duration":0.0038375390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":149,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.601792+0000","last_change":"2024-09-11T07:32:58.087650+0000","last_active":"2024-09-11T07:34:12.601792+0000","last_peered":"2024-09-11T07:34:12.601792+0000","last_clean":"2024-09-11T07:34:12.601792+0000","last_became_active":"2024-09-11T07:32:00.511005+0000","last_became_peered":"2024-09-11T07:32:00.511005+0000","last_unstale":"2024-09-11T07:34:12.601792+0000","last_undegraded":"2024-09-11T07:34:12.601792+0000","last_fullsized":"2024-09-11T07:34:12.601792+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:32:58.087595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:32:58.087595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001226354,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"199'4941","reported_seq":4970,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T07:34:24.574216+0000","last_change":"2024-09-11T07:33:29.205044+0000","last_active":"2024-09-11T07:34:24.574216+0000","last_peered":"2024-09-11T07:34:24.574216+0000","last_clean":"2024-09-11T07:34:24.574216+0000","last_became_active":"2024-09-11T07:33:29.204734+0000","last_became_peered":"2024-09-11T07:33:29.204734+0000","last_unstale":"2024-09-11T07:34:24.574216+0000","last_undegraded":"2024-09-11T07:34:24.574216+0000","last_fullsized":"2024-09-11T07:34:24.574216+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4941,"log_dups_size":0,"ondisk_log_size":4941,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:49.210405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":323813376,"num_objects":4941,"num_object_clones":0,"num_object_copies":14823,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4941,"num_write_kb":316224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"87'26","reported_seq":236,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.599717+0000","last_change":"2024-09-11T07:33:52.609653+0000","last_active":"2024-09-11T07:34:12.599717+0000","last_peered":"2024-09-11T07:34:12.599717+0000","last_clean":"2024-09-11T07:34:12.599717+0000","last_became_active":"2024-09-11T07:32:00.520670+0000","last_became_peered":"2024-09-11T07:32:00.520670+0000","last_unstale":"2024-09-11T07:34:12.599717+0000","last_undegraded":"2024-09-11T07:34:12.599717+0000","last_fullsized":"2024-09-11T07:34:12.599717+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'26","last_scrub_stamp":"2024-09-11T07:33:52.609336+0000","last_deep_scrub":"87'26","last_deep_scrub_stamp":"2024-09-11T07:28:30.788647+0000","last_clean_scrub_stamp":"2024-09-11T07:33:52.609336+0000","objects_scrubbed":8,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:15.124860+0000","scrub_duration":0.010453553000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":189,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.591096+0000","last_change":"2024-09-11T07:33:07.547329+0000","last_active":"2024-09-11T07:34:12.591096+0000","last_peered":"2024-09-11T07:34:12.591096+0000","last_clean":"2024-09-11T07:34:12.591096+0000","last_became_active":"2024-09-11T07:26:31.421500+0000","last_became_peered":"2024-09-11T07:26:31.421500+0000","last_unstale":"2024-09-11T07:34:12.591096+0000","last_undegraded":"2024-09-11T07:34:12.591096+0000","last_fullsized":"2024-09-11T07:34:12.591096+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:07.547259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:07.547259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:34.442417+0000","scrub_duration":0.0019369229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":196,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.588511+0000","last_change":"2024-09-11T07:33:53.449765+0000","last_active":"2024-09-11T07:34:12.588511+0000","last_peered":"2024-09-11T07:34:12.588511+0000","last_clean":"2024-09-11T07:34:12.588511+0000","last_became_active":"2024-09-11T07:32:00.499747+0000","last_became_peered":"2024-09-11T07:32:00.499747+0000","last_unstale":"2024-09-11T07:34:12.588511+0000","last_undegraded":"2024-09-11T07:34:12.588511+0000","last_fullsized":"2024-09-11T07:34:12.588511+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:53.449689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:27.968579+0000","last_clean_scrub_stamp":"2024-09-11T07:33:53.449689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:12.260963+0000","scrub_duration":0.0033513269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":174,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.588820+0000","last_change":"2024-09-11T07:31:41.033292+0000","last_active":"2024-09-11T07:34:12.588820+0000","last_peered":"2024-09-11T07:34:12.588820+0000","last_clean":"2024-09-11T07:34:12.588820+0000","last_became_active":"2024-09-11T07:26:37.564616+0000","last_became_peered":"2024-09-11T07:26:37.564616+0000","last_unstale":"2024-09-11T07:34:12.588820+0000","last_undegraded":"2024-09-11T07:34:12.588820+0000","last_fullsized":"2024-09-11T07:34:12.588820+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:31:41.033237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:29.978798+0000","last_clean_scrub_stamp":"2024-09-11T07:31:41.033237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014647854,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"199'4838","reported_seq":4868,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T07:34:24.562333+0000","last_change":"2024-09-11T07:33:29.161660+0000","last_active":"2024-09-11T07:34:24.562333+0000","last_peered":"2024-09-11T07:34:24.562333+0000","last_clean":"2024-09-11T07:34:24.562333+0000","last_became_active":"2024-09-11T07:33:29.161212+0000","last_became_peered":"2024-09-11T07:33:29.161212+0000","last_unstale":"2024-09-11T07:34:24.562333+0000","last_undegraded":"2024-09-11T07:34:24.562333+0000","last_fullsized":"2024-09-11T07:34:24.562333+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4838,"log_dups_size":0,"ondisk_log_size":4838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:50.971554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":317063168,"num_objects":4838,"num_object_clones":0,"num_object_copies":14514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4838,"num_write_kb":309632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":231,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.594460+0000","last_change":"2024-09-11T07:33:10.802985+0000","last_active":"2024-09-11T07:34:12.594460+0000","last_peered":"2024-09-11T07:34:12.594460+0000","last_clean":"2024-09-11T07:34:12.594460+0000","last_became_active":"2024-09-11T07:32:00.506270+0000","last_became_peered":"2024-09-11T07:32:00.506270+0000","last_unstale":"2024-09-11T07:34:12.594460+0000","last_undegraded":"2024-09-11T07:34:12.594460+0000","last_fullsized":"2024-09-11T07:34:12.594460+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-11T07:33:10.802922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:10.802922+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002454899,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":149,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.602852+0000","last_change":"2024-09-11T07:33:13.291210+0000","last_active":"2024-09-11T07:34:12.602852+0000","last_peered":"2024-09-11T07:34:12.602852+0000","last_clean":"2024-09-11T07:34:12.602852+0000","last_became_active":"2024-09-11T07:32:00.510631+0000","last_became_peered":"2024-09-11T07:32:00.510631+0000","last_unstale":"2024-09-11T07:34:12.602852+0000","last_undegraded":"2024-09-11T07:34:12.602852+0000","last_fullsized":"2024-09-11T07:34:12.602852+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:13.291147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:13.291147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:29.681159+0000","scrub_duration":0.0036274660000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":187,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.593588+0000","last_change":"2024-09-11T07:33:01.281595+0000","last_active":"2024-09-11T07:34:12.593588+0000","last_peered":"2024-09-11T07:34:12.593588+0000","last_clean":"2024-09-11T07:34:12.593588+0000","last_became_active":"2024-09-11T07:26:35.565359+0000","last_became_peered":"2024-09-11T07:26:35.565359+0000","last_unstale":"2024-09-11T07:34:12.593588+0000","last_undegraded":"2024-09-11T07:34:12.593588+0000","last_fullsized":"2024-09-11T07:34:12.593588+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:01.281530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:01.281530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:22.674857+0000","scrub_duration":0.00138591,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":183,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.589992+0000","last_change":"2024-09-11T07:34:10.294762+0000","last_active":"2024-09-11T07:34:12.589992+0000","last_peered":"2024-09-11T07:34:12.589992+0000","last_clean":"2024-09-11T07:34:12.589992+0000","last_became_active":"2024-09-11T07:26:37.861279+0000","last_became_peered":"2024-09-11T07:26:37.861279+0000","last_unstale":"2024-09-11T07:34:12.589992+0000","last_undegraded":"2024-09-11T07:34:12.589992+0000","last_fullsized":"2024-09-11T07:34:12.589992+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:10.294658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:39.723598+0000","last_clean_scrub_stamp":"2024-09-11T07:34:10.294658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:25.940771+0000","scrub_duration":0.0026992600000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.e","version":"199'4882","reported_seq":4909,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T07:34:24.585527+0000","last_change":"2024-09-11T07:33:29.159841+0000","last_active":"2024-09-11T07:34:24.585527+0000","last_peered":"2024-09-11T07:34:24.585527+0000","last_clean":"2024-09-11T07:34:24.585527+0000","last_became_active":"2024-09-11T07:33:29.159535+0000","last_became_peered":"2024-09-11T07:33:29.159535+0000","last_unstale":"2024-09-11T07:34:24.585527+0000","last_undegraded":"2024-09-11T07:34:24.585527+0000","last_fullsized":"2024-09-11T07:34:24.585527+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4882,"log_dups_size":0,"ondisk_log_size":4882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:54.041505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319946752,"num_objects":4882,"num_object_clones":0,"num_object_copies":14646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4882,"num_write_kb":312448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":205,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.591639+0000","last_change":"2024-09-11T07:33:53.294408+0000","last_active":"2024-09-11T07:34:12.591639+0000","last_peered":"2024-09-11T07:34:12.591639+0000","last_clean":"2024-09-11T07:34:12.591639+0000","last_became_active":"2024-09-11T07:26:33.543063+0000","last_became_peered":"2024-09-11T07:26:33.543063+0000","last_unstale":"2024-09-11T07:34:12.591639+0000","last_undegraded":"2024-09-11T07:34:12.591639+0000","last_fullsized":"2024-09-11T07:34:12.591639+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:33:53.294174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:53.294174+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:17.014241+0000","scrub_duration":0.0074369980000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":216,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.590255+0000","last_change":"2024-09-11T07:33:55.344919+0000","last_active":"2024-09-11T07:34:12.590255+0000","last_peered":"2024-09-11T07:34:12.590255+0000","last_clean":"2024-09-11T07:34:12.590255+0000","last_became_active":"2024-09-11T07:32:00.522456+0000","last_became_peered":"2024-09-11T07:32:00.522456+0000","last_unstale":"2024-09-11T07:34:12.590255+0000","last_undegraded":"2024-09-11T07:34:12.590255+0000","last_fullsized":"2024-09-11T07:34:12.590255+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:33:55.344823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:55.344823+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:19.214723+0000","scrub_duration":0.0025904349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":289,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T07:34:19.591218+0000","last_change":"2024-09-11T07:33:50.686006+0000","last_active":"2024-09-11T07:34:19.591218+0000","last_peered":"2024-09-11T07:34:19.591218+0000","last_clean":"2024-09-11T07:34:19.591218+0000","last_became_active":"2024-09-11T07:26:35.560422+0000","last_became_peered":"2024-09-11T07:26:35.560422+0000","last_unstale":"2024-09-11T07:34:19.591218+0000","last_undegraded":"2024-09-11T07:34:19.591218+0000","last_fullsized":"2024-09-11T07:34:19.591218+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:33:50.685923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:50.685923+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:54.623978+0000","scrub_duration":0.0029026490000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":184,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.593364+0000","last_change":"2024-09-11T07:33:16.738699+0000","last_active":"2024-09-11T07:34:12.593364+0000","last_peered":"2024-09-11T07:34:12.593364+0000","last_clean":"2024-09-11T07:34:12.593364+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:34:12.593364+0000","last_undegraded":"2024-09-11T07:34:12.593364+0000","last_fullsized":"2024-09-11T07:34:12.593364+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:16.738631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_clean_scrub_stamp":"2024-09-11T07:33:16.738631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:20.995420+0000","scrub_duration":0.0015950739999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"199'4845","reported_seq":4875,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T07:34:24.018556+0000","last_change":"2024-09-11T07:33:29.205462+0000","last_active":"2024-09-11T07:34:24.018556+0000","last_peered":"2024-09-11T07:34:24.018556+0000","last_clean":"2024-09-11T07:34:24.018556+0000","last_became_active":"2024-09-11T07:33:29.205148+0000","last_became_peered":"2024-09-11T07:33:29.205148+0000","last_unstale":"2024-09-11T07:34:24.018556+0000","last_undegraded":"2024-09-11T07:34:24.018556+0000","last_fullsized":"2024-09-11T07:34:24.018556+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4845,"log_dups_size":0,"ondisk_log_size":4845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:31.106281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":317521920,"num_objects":4845,"num_object_clones":0,"num_object_copies":14535,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4845,"num_write_kb":310080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"170'29","reported_seq":77,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T07:34:15.556119+0000","last_change":"2024-09-11T07:33:10.026568+0000","last_active":"2024-09-11T07:34:15.556119+0000","last_peered":"2024-09-11T07:34:15.556119+0000","last_clean":"2024-09-11T07:34:15.556119+0000","last_became_active":"2024-09-11T07:33:10.017961+0000","last_became_peered":"2024-09-11T07:33:10.017961+0000","last_unstale":"2024-09-11T07:34:15.556119+0000","last_undegraded":"2024-09-11T07:34:15.556119+0000","last_fullsized":"2024-09-11T07:34:15.556119+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'28","last_scrub_stamp":"2024-09-11T07:31:47.592158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:31:47.592158+0000","objects_scrubbed":10,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052388820000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":191,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.595550+0000","last_change":"2024-09-11T07:33:12.133334+0000","last_active":"2024-09-11T07:34:12.595550+0000","last_peered":"2024-09-11T07:34:12.595550+0000","last_clean":"2024-09-11T07:34:12.595550+0000","last_became_active":"2024-09-11T07:26:31.424618+0000","last_became_peered":"2024-09-11T07:26:31.424618+0000","last_unstale":"2024-09-11T07:34:12.595550+0000","last_undegraded":"2024-09-11T07:34:12.595550+0000","last_fullsized":"2024-09-11T07:34:12.595550+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:12.133258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:12.775067+0000","last_clean_scrub_stamp":"2024-09-11T07:33:12.133258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013676669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":186,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.595206+0000","last_change":"2024-09-11T07:33:52.629961+0000","last_active":"2024-09-11T07:34:12.595206+0000","last_peered":"2024-09-11T07:34:12.595206+0000","last_clean":"2024-09-11T07:34:12.595206+0000","last_became_active":"2024-09-11T07:26:35.560363+0000","last_became_peered":"2024-09-11T07:26:35.560363+0000","last_unstale":"2024-09-11T07:34:12.595206+0000","last_undegraded":"2024-09-11T07:34:12.595206+0000","last_fullsized":"2024-09-11T07:34:12.595206+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:52.629872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:52.629872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:03.346424+0000","scrub_duration":0.003860858,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":192,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.598378+0000","last_change":"2024-09-11T07:33:13.566605+0000","last_active":"2024-09-11T07:34:12.598378+0000","last_peered":"2024-09-11T07:34:12.598378+0000","last_clean":"2024-09-11T07:34:12.598378+0000","last_became_active":"2024-09-11T07:32:00.505919+0000","last_became_peered":"2024-09-11T07:32:00.505919+0000","last_unstale":"2024-09-11T07:34:12.598378+0000","last_undegraded":"2024-09-11T07:34:12.598378+0000","last_fullsized":"2024-09-11T07:34:12.598378+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:13.566551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:13.566551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:23.787271+0000","scrub_duration":0.0015767380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"199'5003","reported_seq":5032,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T07:34:24.587470+0000","last_change":"2024-09-11T07:33:29.161446+0000","last_active":"2024-09-11T07:34:24.587470+0000","last_peered":"2024-09-11T07:34:24.587470+0000","last_clean":"2024-09-11T07:34:24.587470+0000","last_became_active":"2024-09-11T07:33:29.161141+0000","last_became_peered":"2024-09-11T07:33:29.161141+0000","last_unstale":"2024-09-11T07:34:24.587470+0000","last_undegraded":"2024-09-11T07:34:24.587470+0000","last_fullsized":"2024-09-11T07:34:24.587470+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":5003,"log_dups_size":0,"ondisk_log_size":5003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:31.128219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327876608,"num_objects":5003,"num_object_clones":0,"num_object_copies":15009,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5003,"num_write_kb":320192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"144'18","reported_seq":219,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.595294+0000","last_change":"2024-09-11T07:34:03.365979+0000","last_active":"2024-09-11T07:34:12.595294+0000","last_peered":"2024-09-11T07:34:12.595294+0000","last_clean":"2024-09-11T07:34:12.595294+0000","last_became_active":"2024-09-11T07:26:33.546126+0000","last_became_peered":"2024-09-11T07:26:33.546126+0000","last_unstale":"2024-09-11T07:34:12.595294+0000","last_undegraded":"2024-09-11T07:34:12.595294+0000","last_fullsized":"2024-09-11T07:34:12.595294+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"144'18","last_scrub_stamp":"2024-09-11T07:34:03.365891+0000","last_deep_scrub":"144'18","last_deep_scrub_stamp":"2024-09-11T07:31:51.603378+0000","last_clean_scrub_stamp":"2024-09-11T07:34:03.365891+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:30.450782+0000","scrub_duration":0.006530999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":662,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T07:34:22.612146+0000","last_change":"2024-09-11T07:33:14.474365+0000","last_active":"2024-09-11T07:34:22.612146+0000","last_peered":"2024-09-11T07:34:22.612146+0000","last_clean":"2024-09-11T07:34:22.612146+0000","last_became_active":"2024-09-11T07:26:31.421276+0000","last_became_peered":"2024-09-11T07:26:31.421276+0000","last_unstale":"2024-09-11T07:34:22.612146+0000","last_undegraded":"2024-09-11T07:34:22.612146+0000","last_fullsized":"2024-09-11T07:34:22.612146+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'5","last_scrub_stamp":"2024-09-11T07:33:14.474301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:14.474301+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041541269999999996,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":465,"num_read_kb":460,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":201,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.603461+0000","last_change":"2024-09-11T07:34:07.149600+0000","last_active":"2024-09-11T07:34:12.603461+0000","last_peered":"2024-09-11T07:34:12.603461+0000","last_clean":"2024-09-11T07:34:12.603461+0000","last_became_active":"2024-09-11T07:26:31.413354+0000","last_became_peered":"2024-09-11T07:26:31.413354+0000","last_unstale":"2024-09-11T07:34:12.603461+0000","last_undegraded":"2024-09-11T07:34:12.603461+0000","last_fullsized":"2024-09-11T07:34:12.603461+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:07.149367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:07.149367+0000","last_clean_scrub_stamp":"2024-09-11T07:34:07.149367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:12.409913+0000","scrub_duration":0.0082613879999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":194,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.601599+0000","last_change":"2024-09-11T07:33:00.143395+0000","last_active":"2024-09-11T07:34:12.601599+0000","last_peered":"2024-09-11T07:34:12.601599+0000","last_clean":"2024-09-11T07:34:12.601599+0000","last_became_active":"2024-09-11T07:32:00.508398+0000","last_became_peered":"2024-09-11T07:32:00.508398+0000","last_unstale":"2024-09-11T07:34:12.601599+0000","last_undegraded":"2024-09-11T07:34:12.601599+0000","last_fullsized":"2024-09-11T07:34:12.601599+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:00.143339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:00.143339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:28.911771+0000","scrub_duration":0.003474343,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":175,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.602680+0000","last_change":"2024-09-11T07:33:55.344477+0000","last_active":"2024-09-11T07:34:12.602680+0000","last_peered":"2024-09-11T07:34:12.602680+0000","last_clean":"2024-09-11T07:34:12.602680+0000","last_became_active":"2024-09-11T07:32:00.515745+0000","last_became_peered":"2024-09-11T07:32:00.515745+0000","last_unstale":"2024-09-11T07:34:12.602680+0000","last_undegraded":"2024-09-11T07:34:12.602680+0000","last_fullsized":"2024-09-11T07:34:12.602680+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:55.344426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:55.344426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:21.367701+0000","scrub_duration":0.032747216000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"199'4924","reported_seq":4953,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T07:34:24.135951+0000","last_change":"2024-09-11T07:33:29.156986+0000","last_active":"2024-09-11T07:34:24.135951+0000","last_peered":"2024-09-11T07:34:24.135951+0000","last_clean":"2024-09-11T07:34:24.135951+0000","last_became_active":"2024-09-11T07:33:29.156567+0000","last_became_peered":"2024-09-11T07:33:29.156567+0000","last_unstale":"2024-09-11T07:34:24.135951+0000","last_undegraded":"2024-09-11T07:34:24.135951+0000","last_fullsized":"2024-09-11T07:34:24.135951+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4924,"log_dups_size":0,"ondisk_log_size":4924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:45.241348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":322699264,"num_objects":4924,"num_object_clones":0,"num_object_copies":14772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4924,"num_write_kb":315136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"94'18","reported_seq":236,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.595484+0000","last_change":"2024-09-11T07:33:18.649291+0000","last_active":"2024-09-11T07:34:12.595484+0000","last_peered":"2024-09-11T07:34:12.595484+0000","last_clean":"2024-09-11T07:34:12.595484+0000","last_became_active":"2024-09-11T07:28:09.018134+0000","last_became_peered":"2024-09-11T07:28:09.018134+0000","last_unstale":"2024-09-11T07:34:12.595484+0000","last_undegraded":"2024-09-11T07:34:12.595484+0000","last_fullsized":"2024-09-11T07:34:12.595484+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'18","last_scrub_stamp":"2024-09-11T07:33:18.648484+0000","last_deep_scrub":"94'18","last_deep_scrub_stamp":"2024-09-11T07:28:51.437397+0000","last_clean_scrub_stamp":"2024-09-11T07:33:18.648484+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:22.084726+0000","scrub_duration":0.0023327030000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":196,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.594873+0000","last_change":"2024-09-11T07:33:48.749463+0000","last_active":"2024-09-11T07:34:12.594873+0000","last_peered":"2024-09-11T07:34:12.594873+0000","last_clean":"2024-09-11T07:34:12.594873+0000","last_became_active":"2024-09-11T07:26:31.420320+0000","last_became_peered":"2024-09-11T07:26:31.420320+0000","last_unstale":"2024-09-11T07:34:12.594873+0000","last_undegraded":"2024-09-11T07:34:12.594873+0000","last_fullsized":"2024-09-11T07:34:12.594873+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-11T07:33:48.749183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:48.749183+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:14.622124+0000","scrub_duration":0.0059599450000000003,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":186,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.602246+0000","last_change":"2024-09-11T07:33:58.311557+0000","last_active":"2024-09-11T07:34:12.602246+0000","last_peered":"2024-09-11T07:34:12.602246+0000","last_clean":"2024-09-11T07:34:12.602246+0000","last_became_active":"2024-09-11T07:32:00.520035+0000","last_became_peered":"2024-09-11T07:32:00.520035+0000","last_unstale":"2024-09-11T07:34:12.602246+0000","last_undegraded":"2024-09-11T07:34:12.602246+0000","last_fullsized":"2024-09-11T07:34:12.602246+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:58.311344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:16.967492+0000","last_clean_scrub_stamp":"2024-09-11T07:33:58.311344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:22.200383+0000","scrub_duration":0.0030726579999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":177,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.596608+0000","last_change":"2024-09-11T07:34:12.211194+0000","last_active":"2024-09-11T07:34:12.596608+0000","last_peered":"2024-09-11T07:34:12.596608+0000","last_clean":"2024-09-11T07:34:12.596608+0000","last_became_active":"2024-09-11T07:26:35.564178+0000","last_became_peered":"2024-09-11T07:26:35.564178+0000","last_unstale":"2024-09-11T07:34:12.596608+0000","last_undegraded":"2024-09-11T07:34:12.596608+0000","last_fullsized":"2024-09-11T07:34:12.596608+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:12.211119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:12.211119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:16.160923+0000","scrub_duration":0.0062325000000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":186,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.590043+0000","last_change":"2024-09-11T07:33:58.323420+0000","last_active":"2024-09-11T07:34:12.590043+0000","last_peered":"2024-09-11T07:34:12.590043+0000","last_clean":"2024-09-11T07:34:12.590043+0000","last_became_active":"2024-09-11T07:26:37.561378+0000","last_became_peered":"2024-09-11T07:26:37.561378+0000","last_unstale":"2024-09-11T07:34:12.590043+0000","last_undegraded":"2024-09-11T07:34:12.590043+0000","last_fullsized":"2024-09-11T07:34:12.590043+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:58.323318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:58.323318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:00.851560+0000","scrub_duration":0.0083597830000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"199'4843","reported_seq":4873,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T07:34:24.084158+0000","last_change":"2024-09-11T07:33:29.154996+0000","last_active":"2024-09-11T07:34:24.084158+0000","last_peered":"2024-09-11T07:34:24.084158+0000","last_clean":"2024-09-11T07:34:24.084158+0000","last_became_active":"2024-09-11T07:33:29.154702+0000","last_became_peered":"2024-09-11T07:33:29.154702+0000","last_unstale":"2024-09-11T07:34:24.084158+0000","last_undegraded":"2024-09-11T07:34:24.084158+0000","last_fullsized":"2024-09-11T07:34:24.084158+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4843,"log_dups_size":0,"ondisk_log_size":4843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:51.999557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":317390848,"num_objects":4843,"num_object_clones":0,"num_object_copies":14529,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4843,"num_write_kb":309952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":231,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.594339+0000","last_change":"2024-09-11T07:34:07.759083+0000","last_active":"2024-09-11T07:34:12.594339+0000","last_peered":"2024-09-11T07:34:12.594339+0000","last_clean":"2024-09-11T07:34:12.594339+0000","last_became_active":"2024-09-11T07:32:00.517409+0000","last_became_peered":"2024-09-11T07:32:00.517409+0000","last_unstale":"2024-09-11T07:34:12.594339+0000","last_undegraded":"2024-09-11T07:34:12.594339+0000","last_fullsized":"2024-09-11T07:34:12.594339+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-09-11T07:34:07.758985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:34:07.758985+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:22.987261+0000","scrub_duration":0.0027645090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'3","reported_seq":189,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.598816+0000","last_change":"2024-09-11T07:31:49.316183+0000","last_active":"2024-09-11T07:34:12.598816+0000","last_peered":"2024-09-11T07:34:12.598816+0000","last_clean":"2024-09-11T07:34:12.598816+0000","last_became_active":"2024-09-11T07:26:31.419501+0000","last_became_peered":"2024-09-11T07:26:31.419501+0000","last_unstale":"2024-09-11T07:34:12.598816+0000","last_undegraded":"2024-09-11T07:34:12.598816+0000","last_fullsized":"2024-09-11T07:34:12.598816+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-09-11T07:31:49.316103+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_clean_scrub_stamp":"2024-09-11T07:31:49.316103+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011392093000000001,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":201,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.595246+0000","last_change":"2024-09-11T07:33:51.108800+0000","last_active":"2024-09-11T07:34:12.595246+0000","last_peered":"2024-09-11T07:34:12.595246+0000","last_clean":"2024-09-11T07:34:12.595246+0000","last_became_active":"2024-09-11T07:32:00.498002+0000","last_became_peered":"2024-09-11T07:32:00.498002+0000","last_unstale":"2024-09-11T07:34:12.595246+0000","last_undegraded":"2024-09-11T07:34:12.595246+0000","last_fullsized":"2024-09-11T07:34:12.595246+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:51.108748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:51.108748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:55.524512+0000","scrub_duration":0.0013286089999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"87'81","reported_seq":202,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.602180+0000","last_change":"2024-09-11T07:33:10.189446+0000","last_active":"2024-09-11T07:34:12.602180+0000","last_peered":"2024-09-11T07:34:12.602180+0000","last_clean":"2024-09-11T07:34:12.602180+0000","last_became_active":"2024-09-11T07:32:00.519656+0000","last_became_peered":"2024-09-11T07:32:00.519656+0000","last_unstale":"2024-09-11T07:34:12.602180+0000","last_undegraded":"2024-09-11T07:34:12.602180+0000","last_fullsized":"2024-09-11T07:34:12.602180+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:33:10.189376+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:23.253384+0000","scrub_duration":0.0032570229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":172,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.602430+0000","last_change":"2024-09-11T07:33:51.272916+0000","last_active":"2024-09-11T07:34:12.602430+0000","last_peered":"2024-09-11T07:34:12.602430+0000","last_clean":"2024-09-11T07:34:12.602430+0000","last_became_active":"2024-09-11T07:32:00.510888+0000","last_became_peered":"2024-09-11T07:32:00.510888+0000","last_unstale":"2024-09-11T07:34:12.602430+0000","last_undegraded":"2024-09-11T07:34:12.602430+0000","last_fullsized":"2024-09-11T07:34:12.602430+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:51.272831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:51.272831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:06.075633+0000","scrub_duration":0.0013461180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":175,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.596676+0000","last_change":"2024-09-11T07:32:57.235357+0000","last_active":"2024-09-11T07:34:12.596676+0000","last_peered":"2024-09-11T07:34:12.596676+0000","last_clean":"2024-09-11T07:34:12.596676+0000","last_became_active":"2024-09-11T07:26:37.559920+0000","last_became_peered":"2024-09-11T07:26:37.559920+0000","last_unstale":"2024-09-11T07:34:12.596676+0000","last_undegraded":"2024-09-11T07:34:12.596676+0000","last_fullsized":"2024-09-11T07:34:12.596676+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:32:57.235305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:32:57.235305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:24.383939+0000","scrub_duration":0.0013458579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"199'4886","reported_seq":4915,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T07:34:24.557926+0000","last_change":"2024-09-11T07:33:29.204945+0000","last_active":"2024-09-11T07:34:24.557926+0000","last_peered":"2024-09-11T07:34:24.557926+0000","last_clean":"2024-09-11T07:34:24.557926+0000","last_became_active":"2024-09-11T07:33:29.204611+0000","last_became_peered":"2024-09-11T07:33:29.204611+0000","last_unstale":"2024-09-11T07:34:24.557926+0000","last_undegraded":"2024-09-11T07:34:24.557926+0000","last_fullsized":"2024-09-11T07:34:24.557926+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4886,"log_dups_size":0,"ondisk_log_size":4886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:31.427771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":320208896,"num_objects":4886,"num_object_clones":0,"num_object_copies":14658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4886,"num_write_kb":312704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":230,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.593780+0000","last_change":"2024-09-11T07:33:49.612375+0000","last_active":"2024-09-11T07:34:12.593780+0000","last_peered":"2024-09-11T07:34:12.593780+0000","last_clean":"2024-09-11T07:34:12.593780+0000","last_became_active":"2024-09-11T07:32:00.503799+0000","last_became_peered":"2024-09-11T07:32:00.503799+0000","last_unstale":"2024-09-11T07:34:12.593780+0000","last_undegraded":"2024-09-11T07:34:12.593780+0000","last_fullsized":"2024-09-11T07:34:12.593780+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-11T07:33:49.612326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:49.612326+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:15.075880+0000","scrub_duration":0.003204648,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":191,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.602404+0000","last_change":"2024-09-11T07:34:02.192275+0000","last_active":"2024-09-11T07:34:12.602404+0000","last_peered":"2024-09-11T07:34:12.602404+0000","last_clean":"2024-09-11T07:34:12.602404+0000","last_became_active":"2024-09-11T07:26:31.419560+0000","last_became_peered":"2024-09-11T07:26:31.419560+0000","last_unstale":"2024-09-11T07:34:12.602404+0000","last_undegraded":"2024-09-11T07:34:12.602404+0000","last_fullsized":"2024-09-11T07:34:12.602404+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:02.192193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:02.192193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:15.980889+0000","scrub_duration":0.0060418110000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":187,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.601329+0000","last_change":"2024-09-11T07:34:06.134576+0000","last_active":"2024-09-11T07:34:12.601329+0000","last_peered":"2024-09-11T07:34:12.601329+0000","last_clean":"2024-09-11T07:34:12.601329+0000","last_became_active":"2024-09-11T07:26:35.564044+0000","last_became_peered":"2024-09-11T07:26:35.564044+0000","last_unstale":"2024-09-11T07:34:12.601329+0000","last_undegraded":"2024-09-11T07:34:12.601329+0000","last_fullsized":"2024-09-11T07:34:12.601329+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:06.134464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:06.134464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:15.601865+0000","scrub_duration":0.0062969130000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":170,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.602107+0000","last_change":"2024-09-11T07:32:00.520910+0000","last_active":"2024-09-11T07:34:12.602107+0000","last_peered":"2024-09-11T07:34:12.602107+0000","last_clean":"2024-09-11T07:34:12.602107+0000","last_became_active":"2024-09-11T07:32:00.516916+0000","last_became_peered":"2024-09-11T07:32:00.516916+0000","last_unstale":"2024-09-11T07:34:12.602107+0000","last_undegraded":"2024-09-11T07:34:12.602107+0000","last_fullsized":"2024-09-11T07:34:12.602107+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:30:59.302530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:30:59.302530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.7","version":"199'4866","reported_seq":4895,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T07:34:24.180406+0000","last_change":"2024-09-11T07:33:29.160658+0000","last_active":"2024-09-11T07:34:24.180406+0000","last_peered":"2024-09-11T07:34:24.180406+0000","last_clean":"2024-09-11T07:34:24.180406+0000","last_became_active":"2024-09-11T07:33:29.160376+0000","last_became_peered":"2024-09-11T07:33:29.160376+0000","last_unstale":"2024-09-11T07:34:24.180406+0000","last_undegraded":"2024-09-11T07:34:24.180406+0000","last_fullsized":"2024-09-11T07:34:24.180406+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4866,"log_dups_size":0,"ondisk_log_size":4866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:55.763414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318898176,"num_objects":4866,"num_object_clones":0,"num_object_copies":14598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4866,"num_write_kb":311424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":194,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.594385+0000","last_change":"2024-09-11T07:33:14.754606+0000","last_active":"2024-09-11T07:34:12.594385+0000","last_peered":"2024-09-11T07:34:12.594385+0000","last_clean":"2024-09-11T07:34:12.594385+0000","last_became_active":"2024-09-11T07:26:33.543148+0000","last_became_peered":"2024-09-11T07:26:33.543148+0000","last_unstale":"2024-09-11T07:34:12.594385+0000","last_undegraded":"2024-09-11T07:34:12.594385+0000","last_fullsized":"2024-09-11T07:34:12.594385+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:33:14.754532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:14.754532+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:25.384577+0000","scrub_duration":0.0034842570000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":189,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.595132+0000","last_change":"2024-09-11T07:33:17.426275+0000","last_active":"2024-09-11T07:34:12.595132+0000","last_peered":"2024-09-11T07:34:12.595132+0000","last_clean":"2024-09-11T07:34:12.595132+0000","last_became_active":"2024-09-11T07:26:31.423056+0000","last_became_peered":"2024-09-11T07:26:31.423056+0000","last_unstale":"2024-09-11T07:34:12.595132+0000","last_undegraded":"2024-09-11T07:34:12.595132+0000","last_fullsized":"2024-09-11T07:34:12.595132+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:17.426220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:17.426220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:19.376100+0000","scrub_duration":0.0035646520000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":187,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.603915+0000","last_change":"2024-09-11T07:33:10.080039+0000","last_active":"2024-09-11T07:34:12.603915+0000","last_peered":"2024-09-11T07:34:12.603915+0000","last_clean":"2024-09-11T07:34:12.603915+0000","last_became_active":"2024-09-11T07:26:35.563182+0000","last_became_peered":"2024-09-11T07:26:35.563182+0000","last_unstale":"2024-09-11T07:34:12.603915+0000","last_undegraded":"2024-09-11T07:34:12.603915+0000","last_fullsized":"2024-09-11T07:34:12.603915+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:10.079984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:10.079984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001676498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":191,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.595958+0000","last_change":"2024-09-11T07:33:11.790118+0000","last_active":"2024-09-11T07:34:12.595958+0000","last_peered":"2024-09-11T07:34:12.595958+0000","last_clean":"2024-09-11T07:34:12.595958+0000","last_became_active":"2024-09-11T07:32:00.505185+0000","last_became_peered":"2024-09-11T07:32:00.505185+0000","last_unstale":"2024-09-11T07:34:12.595958+0000","last_undegraded":"2024-09-11T07:34:12.595958+0000","last_fullsized":"2024-09-11T07:34:12.595958+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:11.790063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:11.790063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:23.070960+0000","scrub_duration":0.001675405,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"199'4904","reported_seq":4934,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T07:34:24.013661+0000","last_change":"2024-09-11T07:33:29.153952+0000","last_active":"2024-09-11T07:34:24.013661+0000","last_peered":"2024-09-11T07:34:24.013661+0000","last_clean":"2024-09-11T07:34:24.013661+0000","last_became_active":"2024-09-11T07:33:29.153604+0000","last_became_peered":"2024-09-11T07:33:29.153604+0000","last_unstale":"2024-09-11T07:34:24.013661+0000","last_undegraded":"2024-09-11T07:34:24.013661+0000","last_fullsized":"2024-09-11T07:34:24.013661+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4904,"log_dups_size":0,"ondisk_log_size":4904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:52.795972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":321388544,"num_objects":4904,"num_object_clones":0,"num_object_copies":14712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4904,"num_write_kb":313856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":224,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.599370+0000","last_change":"2024-09-11T07:33:56.187920+0000","last_active":"2024-09-11T07:34:12.599370+0000","last_peered":"2024-09-11T07:34:12.599370+0000","last_clean":"2024-09-11T07:34:12.599370+0000","last_became_active":"2024-09-11T07:32:00.521464+0000","last_became_peered":"2024-09-11T07:32:00.521464+0000","last_unstale":"2024-09-11T07:34:12.599370+0000","last_undegraded":"2024-09-11T07:34:12.599370+0000","last_fullsized":"2024-09-11T07:34:12.599370+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-09-11T07:33:56.187871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:56.187871+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:23.701092+0000","scrub_duration":0.001643175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":208,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.601770+0000","last_change":"2024-09-11T07:33:12.052744+0000","last_active":"2024-09-11T07:34:12.601770+0000","last_peered":"2024-09-11T07:34:12.601770+0000","last_clean":"2024-09-11T07:34:12.601770+0000","last_became_active":"2024-09-11T07:32:00.513949+0000","last_became_peered":"2024-09-11T07:32:00.513949+0000","last_unstale":"2024-09-11T07:34:12.601770+0000","last_undegraded":"2024-09-11T07:34:12.601770+0000","last_fullsized":"2024-09-11T07:34:12.601770+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:12.052670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_clean_scrub_stamp":"2024-09-11T07:33:12.052670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:15.594508+0000","scrub_duration":0.0014794280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"151'10","reported_seq":240,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T07:34:15.557749+0000","last_change":"2024-09-11T07:33:48.261946+0000","last_active":"2024-09-11T07:34:15.557749+0000","last_peered":"2024-09-11T07:34:15.557749+0000","last_clean":"2024-09-11T07:34:15.557749+0000","last_became_active":"2024-09-11T07:32:00.506806+0000","last_became_peered":"2024-09-11T07:32:00.506806+0000","last_unstale":"2024-09-11T07:34:15.557749+0000","last_undegraded":"2024-09-11T07:34:15.557749+0000","last_fullsized":"2024-09-11T07:34:15.557749+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:33:48.261857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:48.261857+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:06.611603+0000","scrub_duration":0.001795774,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":175,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.594710+0000","last_change":"2024-09-11T07:31:37.988572+0000","last_active":"2024-09-11T07:34:12.594710+0000","last_peered":"2024-09-11T07:34:12.594710+0000","last_clean":"2024-09-11T07:34:12.594710+0000","last_became_active":"2024-09-11T07:26:37.861134+0000","last_became_peered":"2024-09-11T07:26:37.861134+0000","last_unstale":"2024-09-11T07:34:12.594710+0000","last_undegraded":"2024-09-11T07:34:12.594710+0000","last_fullsized":"2024-09-11T07:34:12.594710+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:31:37.988491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:31:37.988491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002742899,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.5","version":"199'4807","reported_seq":4836,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T07:34:24.109198+0000","last_change":"2024-09-11T07:33:29.205466+0000","last_active":"2024-09-11T07:34:24.109198+0000","last_peered":"2024-09-11T07:34:24.109198+0000","last_clean":"2024-09-11T07:34:24.109198+0000","last_became_active":"2024-09-11T07:33:29.205189+0000","last_became_peered":"2024-09-11T07:33:29.205189+0000","last_unstale":"2024-09-11T07:34:24.109198+0000","last_undegraded":"2024-09-11T07:34:24.109198+0000","last_fullsized":"2024-09-11T07:34:24.109198+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4807,"log_dups_size":0,"ondisk_log_size":4807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:56.812992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":315031552,"num_objects":4807,"num_object_clones":0,"num_object_copies":14421,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4807,"num_write_kb":307648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"83'17","reported_seq":206,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.588582+0000","last_change":"2024-09-11T07:31:32.003568+0000","last_active":"2024-09-11T07:34:12.588582+0000","last_peered":"2024-09-11T07:34:12.588582+0000","last_clean":"2024-09-11T07:34:12.588582+0000","last_became_active":"2024-09-11T07:26:33.540363+0000","last_became_peered":"2024-09-11T07:26:33.540363+0000","last_unstale":"2024-09-11T07:34:12.588582+0000","last_undegraded":"2024-09-11T07:34:12.588582+0000","last_fullsized":"2024-09-11T07:34:12.588582+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'17","last_scrub_stamp":"2024-09-11T07:31:32.003516+0000","last_deep_scrub":"83'17","last_deep_scrub_stamp":"2024-09-11T07:29:50.970355+0000","last_clean_scrub_stamp":"2024-09-11T07:31:32.003516+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004484811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":217,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.589535+0000","last_change":"2024-09-11T07:33:48.436215+0000","last_active":"2024-09-11T07:34:12.589535+0000","last_peered":"2024-09-11T07:34:12.589535+0000","last_clean":"2024-09-11T07:34:12.589535+0000","last_became_active":"2024-09-11T07:32:00.504344+0000","last_became_peered":"2024-09-11T07:32:00.504344+0000","last_unstale":"2024-09-11T07:34:12.589535+0000","last_undegraded":"2024-09-11T07:34:12.589535+0000","last_fullsized":"2024-09-11T07:34:12.589535+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:33:48.436127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:48.436127+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:59.296316+0000","scrub_duration":0.002390933,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":187,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.601482+0000","last_change":"2024-09-11T07:33:51.206706+0000","last_active":"2024-09-11T07:34:12.601482+0000","last_peered":"2024-09-11T07:34:12.601482+0000","last_clean":"2024-09-11T07:34:12.601482+0000","last_became_active":"2024-09-11T07:26:35.561967+0000","last_became_peered":"2024-09-11T07:26:35.561967+0000","last_unstale":"2024-09-11T07:34:12.601482+0000","last_undegraded":"2024-09-11T07:34:12.601482+0000","last_fullsized":"2024-09-11T07:34:12.601482+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:51.206632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:44.820354+0000","last_clean_scrub_stamp":"2024-09-11T07:33:51.206632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:01.971369+0000","scrub_duration":0.0065238259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":192,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.602435+0000","last_change":"2024-09-11T07:33:10.531766+0000","last_active":"2024-09-11T07:34:12.602435+0000","last_peered":"2024-09-11T07:34:12.602435+0000","last_clean":"2024-09-11T07:34:12.602435+0000","last_became_active":"2024-09-11T07:32:00.502019+0000","last_became_peered":"2024-09-11T07:32:00.502019+0000","last_unstale":"2024-09-11T07:34:12.602435+0000","last_undegraded":"2024-09-11T07:34:12.602435+0000","last_fullsized":"2024-09-11T07:34:12.602435+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:10.531696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:10.531696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018694370000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":211,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.595393+0000","last_change":"2024-09-11T07:33:05.880679+0000","last_active":"2024-09-11T07:34:12.595393+0000","last_peered":"2024-09-11T07:34:12.595393+0000","last_clean":"2024-09-11T07:34:12.595393+0000","last_became_active":"2024-09-11T07:32:00.509412+0000","last_became_peered":"2024-09-11T07:32:00.509412+0000","last_unstale":"2024-09-11T07:34:12.595393+0000","last_undegraded":"2024-09-11T07:34:12.595393+0000","last_fullsized":"2024-09-11T07:34:12.595393+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:33:05.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:05.880612+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:35.340902+0000","scrub_duration":0.0021137109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":191,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.601623+0000","last_change":"2024-09-11T07:34:03.167417+0000","last_active":"2024-09-11T07:34:12.601623+0000","last_peered":"2024-09-11T07:34:12.601623+0000","last_clean":"2024-09-11T07:34:12.601623+0000","last_became_active":"2024-09-11T07:26:31.415701+0000","last_became_peered":"2024-09-11T07:26:31.415701+0000","last_unstale":"2024-09-11T07:34:12.601623+0000","last_undegraded":"2024-09-11T07:34:12.601623+0000","last_fullsized":"2024-09-11T07:34:12.601623+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:03.167327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.925200+0000","last_clean_scrub_stamp":"2024-09-11T07:34:03.167327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:28.877249+0000","scrub_duration":0.0047032489999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":194,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.598783+0000","last_change":"2024-09-11T07:33:54.643176+0000","last_active":"2024-09-11T07:34:12.598783+0000","last_peered":"2024-09-11T07:34:12.598783+0000","last_clean":"2024-09-11T07:34:12.598783+0000","last_became_active":"2024-09-11T07:32:00.499598+0000","last_became_peered":"2024-09-11T07:32:00.499598+0000","last_unstale":"2024-09-11T07:34:12.598783+0000","last_undegraded":"2024-09-11T07:34:12.598783+0000","last_fullsized":"2024-09-11T07:34:12.598783+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:54.642486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:35.392360+0000","last_clean_scrub_stamp":"2024-09-11T07:33:54.642486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:23.835294+0000","scrub_duration":0.010529307999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":193,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.601927+0000","last_change":"2024-09-11T07:33:03.061811+0000","last_active":"2024-09-11T07:34:12.601927+0000","last_peered":"2024-09-11T07:34:12.601927+0000","last_clean":"2024-09-11T07:34:12.601927+0000","last_became_active":"2024-09-11T07:32:00.506697+0000","last_became_peered":"2024-09-11T07:32:00.506697+0000","last_unstale":"2024-09-11T07:34:12.601927+0000","last_undegraded":"2024-09-11T07:34:12.601927+0000","last_fullsized":"2024-09-11T07:34:12.601927+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:33:03.061713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:03.061713+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:19.751616+0000","scrub_duration":0.002417888,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":191,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.593626+0000","last_change":"2024-09-11T07:33:03.339633+0000","last_active":"2024-09-11T07:34:12.593626+0000","last_peered":"2024-09-11T07:34:12.593626+0000","last_clean":"2024-09-11T07:34:12.593626+0000","last_became_active":"2024-09-11T07:26:31.423229+0000","last_became_peered":"2024-09-11T07:26:31.423229+0000","last_unstale":"2024-09-11T07:34:12.593626+0000","last_undegraded":"2024-09-11T07:34:12.593626+0000","last_fullsized":"2024-09-11T07:34:12.593626+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:03.339569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:03.339569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:30.079046+0000","scrub_duration":0.001437523,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"82'9","reported_seq":208,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.598913+0000","last_change":"2024-09-11T07:33:11.576715+0000","last_active":"2024-09-11T07:34:12.598913+0000","last_peered":"2024-09-11T07:34:12.598913+0000","last_clean":"2024-09-11T07:34:12.598913+0000","last_became_active":"2024-09-11T07:32:00.501365+0000","last_became_peered":"2024-09-11T07:32:00.501365+0000","last_unstale":"2024-09-11T07:34:12.598913+0000","last_undegraded":"2024-09-11T07:34:12.598913+0000","last_fullsized":"2024-09-11T07:34:12.598913+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'9","last_scrub_stamp":"2024-09-11T07:33:11.576656+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:33:11.576656+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:35.698527+0000","scrub_duration":0.0043542040000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":187,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.596970+0000","last_change":"2024-09-11T07:33:15.113446+0000","last_active":"2024-09-11T07:34:12.596970+0000","last_peered":"2024-09-11T07:34:12.596970+0000","last_clean":"2024-09-11T07:34:12.596970+0000","last_became_active":"2024-09-11T07:26:35.566894+0000","last_became_peered":"2024-09-11T07:26:35.566894+0000","last_unstale":"2024-09-11T07:34:12.596970+0000","last_undegraded":"2024-09-11T07:34:12.596970+0000","last_fullsized":"2024-09-11T07:34:12.596970+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:15.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:33:15.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:25.120342+0000","scrub_duration":0.0011939730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":173,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.603022+0000","last_change":"2024-09-11T07:33:54.283942+0000","last_active":"2024-09-11T07:34:12.603022+0000","last_peered":"2024-09-11T07:34:12.603022+0000","last_clean":"2024-09-11T07:34:12.603022+0000","last_became_active":"2024-09-11T07:32:00.514100+0000","last_became_peered":"2024-09-11T07:32:00.514100+0000","last_unstale":"2024-09-11T07:34:12.603022+0000","last_undegraded":"2024-09-11T07:34:12.603022+0000","last_fullsized":"2024-09-11T07:34:12.603022+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:54.283860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:54.283860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:00.050044+0000","scrub_duration":0.0032251950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"86'15","reported_seq":228,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.600340+0000","last_change":"2024-09-11T07:32:59.286373+0000","last_active":"2024-09-11T07:34:12.600340+0000","last_peered":"2024-09-11T07:34:12.600340+0000","last_clean":"2024-09-11T07:34:12.600340+0000","last_became_active":"2024-09-11T07:32:00.504463+0000","last_became_peered":"2024-09-11T07:32:00.504463+0000","last_unstale":"2024-09-11T07:34:12.600340+0000","last_undegraded":"2024-09-11T07:34:12.600340+0000","last_fullsized":"2024-09-11T07:34:12.600340+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'15","last_scrub_stamp":"2024-09-11T07:32:59.286314+0000","last_deep_scrub":"86'15","last_deep_scrub_stamp":"2024-09-11T07:28:22.858137+0000","last_clean_scrub_stamp":"2024-09-11T07:32:59.286314+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018903660000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":201,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.595482+0000","last_change":"2024-09-11T07:34:02.231632+0000","last_active":"2024-09-11T07:34:12.595482+0000","last_peered":"2024-09-11T07:34:12.595482+0000","last_clean":"2024-09-11T07:34:12.595482+0000","last_became_active":"2024-09-11T07:32:00.498473+0000","last_became_peered":"2024-09-11T07:32:00.498473+0000","last_unstale":"2024-09-11T07:34:12.595482+0000","last_undegraded":"2024-09-11T07:34:12.595482+0000","last_fullsized":"2024-09-11T07:34:12.595482+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:02.231583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:02.231583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:24.160910+0000","scrub_duration":0.0013039099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":279,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T07:34:15.556602+0000","last_change":"2024-09-11T07:31:42.984614+0000","last_active":"2024-09-11T07:34:15.556602+0000","last_peered":"2024-09-11T07:34:15.556602+0000","last_clean":"2024-09-11T07:34:15.556602+0000","last_became_active":"2024-09-11T07:26:35.562245+0000","last_became_peered":"2024-09-11T07:26:35.562245+0000","last_unstale":"2024-09-11T07:34:15.556602+0000","last_undegraded":"2024-09-11T07:34:15.556602+0000","last_fullsized":"2024-09-11T07:34:15.556602+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:31:42.984562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:31:42.984562+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015727580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":201,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.594526+0000","last_change":"2024-09-11T07:34:12.556603+0000","last_active":"2024-09-11T07:34:12.594526+0000","last_peered":"2024-09-11T07:34:12.594526+0000","last_clean":"2024-09-11T07:34:12.594526+0000","last_became_active":"2024-09-11T07:32:00.509295+0000","last_became_peered":"2024-09-11T07:32:00.509295+0000","last_unstale":"2024-09-11T07:34:12.594526+0000","last_undegraded":"2024-09-11T07:34:12.594526+0000","last_fullsized":"2024-09-11T07:34:12.594526+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:12.556370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:12.556370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:14.451889+0000","scrub_duration":0.0041791759999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"83'13","reported_seq":219,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.595746+0000","last_change":"2024-09-11T07:33:52.110471+0000","last_active":"2024-09-11T07:34:12.595746+0000","last_peered":"2024-09-11T07:34:12.595746+0000","last_clean":"2024-09-11T07:34:12.595746+0000","last_became_active":"2024-09-11T07:32:00.505134+0000","last_became_peered":"2024-09-11T07:32:00.505134+0000","last_unstale":"2024-09-11T07:34:12.595746+0000","last_undegraded":"2024-09-11T07:34:12.595746+0000","last_fullsized":"2024-09-11T07:34:12.595746+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-11T07:33:52.110361+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-11T07:28:00.959583+0000","last_clean_scrub_stamp":"2024-09-11T07:33:52.110361+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:04.735778+0000","scrub_duration":0.0035066310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":190,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.594590+0000","last_change":"2024-09-11T07:33:51.733858+0000","last_active":"2024-09-11T07:34:12.594590+0000","last_peered":"2024-09-11T07:34:12.594590+0000","last_clean":"2024-09-11T07:34:12.594590+0000","last_became_active":"2024-09-11T07:26:31.419409+0000","last_became_peered":"2024-09-11T07:26:31.419409+0000","last_unstale":"2024-09-11T07:34:12.594590+0000","last_undegraded":"2024-09-11T07:34:12.594590+0000","last_fullsized":"2024-09-11T07:34:12.594590+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:51.733780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:51.733780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:20.982450+0000","scrub_duration":0.079912094000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"151'10","reported_seq":292,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T07:34:19.591295+0000","last_change":"2024-09-11T07:33:47.546713+0000","last_active":"2024-09-11T07:34:19.591295+0000","last_peered":"2024-09-11T07:34:19.591295+0000","last_clean":"2024-09-11T07:34:19.591295+0000","last_became_active":"2024-09-11T07:32:00.504062+0000","last_became_peered":"2024-09-11T07:32:00.504062+0000","last_unstale":"2024-09-11T07:34:19.591295+0000","last_undegraded":"2024-09-11T07:34:19.591295+0000","last_fullsized":"2024-09-11T07:34:19.591295+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:33:47.546622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:47.546622+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:07.659165+0000","scrub_duration":0.0064760360000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":174,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.589205+0000","last_change":"2024-09-11T07:31:36.995728+0000","last_active":"2024-09-11T07:34:12.589205+0000","last_peered":"2024-09-11T07:34:12.589205+0000","last_clean":"2024-09-11T07:34:12.589205+0000","last_became_active":"2024-09-11T07:26:37.565723+0000","last_became_peered":"2024-09-11T07:26:37.565723+0000","last_unstale":"2024-09-11T07:34:12.589205+0000","last_undegraded":"2024-09-11T07:34:12.589205+0000","last_fullsized":"2024-09-11T07:34:12.589205+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:31:36.995548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:31:36.995548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0093244260000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":213,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.592861+0000","last_change":"2024-09-11T07:34:12.364224+0000","last_active":"2024-09-11T07:34:12.592861+0000","last_peered":"2024-09-11T07:34:12.592861+0000","last_clean":"2024-09-11T07:34:12.592861+0000","last_became_active":"2024-09-11T07:26:33.540688+0000","last_became_peered":"2024-09-11T07:26:33.540688+0000","last_unstale":"2024-09-11T07:34:12.592861+0000","last_undegraded":"2024-09-11T07:34:12.592861+0000","last_fullsized":"2024-09-11T07:34:12.592861+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:34:12.364126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:34:12.364126+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:21.215985+0000","scrub_duration":0.011537936,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":194,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.594118+0000","last_change":"2024-09-11T07:34:09.772110+0000","last_active":"2024-09-11T07:34:12.594118+0000","last_peered":"2024-09-11T07:34:12.594118+0000","last_clean":"2024-09-11T07:34:12.594118+0000","last_became_active":"2024-09-11T07:26:31.425587+0000","last_became_peered":"2024-09-11T07:26:31.425587+0000","last_unstale":"2024-09-11T07:34:12.594118+0000","last_undegraded":"2024-09-11T07:34:12.594118+0000","last_fullsized":"2024-09-11T07:34:12.594118+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:09.772000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:09.772000+0000","last_clean_scrub_stamp":"2024-09-11T07:34:09.772000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:14.926483+0000","scrub_duration":0.0087789909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":177,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.602501+0000","last_change":"2024-09-11T07:34:04.660377+0000","last_active":"2024-09-11T07:34:12.602501+0000","last_peered":"2024-09-11T07:34:12.602501+0000","last_clean":"2024-09-11T07:34:12.602501+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:34:12.602501+0000","last_undegraded":"2024-09-11T07:34:12.602501+0000","last_fullsized":"2024-09-11T07:34:12.602501+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:04.660121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:04.660121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:12.877858+0000","scrub_duration":0.0047831310000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":184,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.593283+0000","last_change":"2024-09-11T07:33:53.667783+0000","last_active":"2024-09-11T07:34:12.593283+0000","last_peered":"2024-09-11T07:34:12.593283+0000","last_clean":"2024-09-11T07:34:12.593283+0000","last_became_active":"2024-09-11T07:26:37.568886+0000","last_became_peered":"2024-09-11T07:26:37.568886+0000","last_unstale":"2024-09-11T07:34:12.593283+0000","last_undegraded":"2024-09-11T07:34:12.593283+0000","last_fullsized":"2024-09-11T07:34:12.593283+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:53.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:53.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:59.734127+0000","scrub_duration":0.0039882809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"136'16","reported_seq":212,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.589095+0000","last_change":"2024-09-11T07:34:00.417426+0000","last_active":"2024-09-11T07:34:12.589095+0000","last_peered":"2024-09-11T07:34:12.589095+0000","last_clean":"2024-09-11T07:34:12.589095+0000","last_became_active":"2024-09-11T07:26:33.540208+0000","last_became_peered":"2024-09-11T07:26:33.540208+0000","last_unstale":"2024-09-11T07:34:12.589095+0000","last_undegraded":"2024-09-11T07:34:12.589095+0000","last_fullsized":"2024-09-11T07:34:12.589095+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'16","last_scrub_stamp":"2024-09-11T07:34:00.417343+0000","last_deep_scrub":"136'16","last_deep_scrub_stamp":"2024-09-11T07:31:06.039023+0000","last_clean_scrub_stamp":"2024-09-11T07:34:00.417343+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:12.571774+0000","scrub_duration":0.0026159130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":192,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.591174+0000","last_change":"2024-09-11T07:33:57.333274+0000","last_active":"2024-09-11T07:34:12.591174+0000","last_peered":"2024-09-11T07:34:12.591174+0000","last_clean":"2024-09-11T07:34:12.591174+0000","last_became_active":"2024-09-11T07:26:31.422671+0000","last_became_peered":"2024-09-11T07:26:31.422671+0000","last_unstale":"2024-09-11T07:34:12.591174+0000","last_undegraded":"2024-09-11T07:34:12.591174+0000","last_fullsized":"2024-09-11T07:34:12.591174+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:57.333166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:33:57.333166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:16.535083+0000","scrub_duration":0.010637648,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":178,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.590587+0000","last_change":"2024-09-11T07:33:51.251429+0000","last_active":"2024-09-11T07:34:12.590587+0000","last_peered":"2024-09-11T07:34:12.590587+0000","last_clean":"2024-09-11T07:34:12.590587+0000","last_became_active":"2024-09-11T07:26:35.566931+0000","last_became_peered":"2024-09-11T07:26:35.566931+0000","last_unstale":"2024-09-11T07:34:12.590587+0000","last_undegraded":"2024-09-11T07:34:12.590587+0000","last_fullsized":"2024-09-11T07:34:12.590587+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:51.251225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.847474+0000","last_clean_scrub_stamp":"2024-09-11T07:33:51.251225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:08.093660+0000","scrub_duration":0.0050645059999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":165,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.597946+0000","last_change":"2024-09-11T07:31:00.433331+0000","last_active":"2024-09-11T07:34:12.597946+0000","last_peered":"2024-09-11T07:34:12.597946+0000","last_clean":"2024-09-11T07:34:12.597946+0000","last_became_active":"2024-09-11T07:26:37.561573+0000","last_became_peered":"2024-09-11T07:26:37.561573+0000","last_unstale":"2024-09-11T07:34:12.597946+0000","last_undegraded":"2024-09-11T07:34:12.597946+0000","last_fullsized":"2024-09-11T07:34:12.597946+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:31:00.432868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:31:00.432868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012162007000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"82'6","reported_seq":187,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.599295+0000","last_change":"2024-09-11T07:31:38.286280+0000","last_active":"2024-09-11T07:34:12.599295+0000","last_peered":"2024-09-11T07:34:12.599295+0000","last_clean":"2024-09-11T07:34:12.599295+0000","last_became_active":"2024-09-11T07:26:33.545830+0000","last_became_peered":"2024-09-11T07:26:33.545830+0000","last_unstale":"2024-09-11T07:34:12.599295+0000","last_undegraded":"2024-09-11T07:34:12.599295+0000","last_fullsized":"2024-09-11T07:34:12.599295+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'6","last_scrub_stamp":"2024-09-11T07:31:38.285969+0000","last_deep_scrub":"82'6","last_deep_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_clean_scrub_stamp":"2024-09-11T07:31:38.285969+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0068909499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":181,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.597129+0000","last_change":"2024-09-11T07:31:35.961700+0000","last_active":"2024-09-11T07:34:12.597129+0000","last_peered":"2024-09-11T07:34:12.597129+0000","last_clean":"2024-09-11T07:34:12.597129+0000","last_became_active":"2024-09-11T07:26:31.420518+0000","last_became_peered":"2024-09-11T07:26:31.420518+0000","last_unstale":"2024-09-11T07:34:12.597129+0000","last_undegraded":"2024-09-11T07:34:12.597129+0000","last_fullsized":"2024-09-11T07:34:12.597129+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:31:35.961609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_clean_scrub_stamp":"2024-09-11T07:31:35.961609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001830932,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":197,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.592455+0000","last_change":"2024-09-11T07:33:16.555518+0000","last_active":"2024-09-11T07:34:12.592455+0000","last_peered":"2024-09-11T07:34:12.592455+0000","last_clean":"2024-09-11T07:34:12.592455+0000","last_became_active":"2024-09-11T07:32:00.501969+0000","last_became_peered":"2024-09-11T07:32:00.501969+0000","last_unstale":"2024-09-11T07:34:12.592455+0000","last_undegraded":"2024-09-11T07:34:12.592455+0000","last_fullsized":"2024-09-11T07:34:12.592455+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:16.555461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:16.555461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:18.844094+0000","scrub_duration":0.001257347,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":175,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.601152+0000","last_change":"2024-09-11T07:33:53.290535+0000","last_active":"2024-09-11T07:34:12.601152+0000","last_peered":"2024-09-11T07:34:12.601152+0000","last_clean":"2024-09-11T07:34:12.601152+0000","last_became_active":"2024-09-11T07:32:00.511407+0000","last_became_peered":"2024-09-11T07:32:00.511407+0000","last_unstale":"2024-09-11T07:34:12.601152+0000","last_undegraded":"2024-09-11T07:34:12.601152+0000","last_fullsized":"2024-09-11T07:34:12.601152+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:53.290379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:53.290379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:58.283671+0000","scrub_duration":0.002605291,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":205,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.598117+0000","last_change":"2024-09-11T07:33:03.304492+0000","last_active":"2024-09-11T07:34:12.598117+0000","last_peered":"2024-09-11T07:34:12.598117+0000","last_clean":"2024-09-11T07:34:12.598117+0000","last_became_active":"2024-09-11T07:32:00.502081+0000","last_became_peered":"2024-09-11T07:32:00.502081+0000","last_unstale":"2024-09-11T07:34:12.598117+0000","last_undegraded":"2024-09-11T07:34:12.598117+0000","last_fullsized":"2024-09-11T07:34:12.598117+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:33:03.304404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:03.304404+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:29.580187+0000","scrub_duration":0.0023973169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":191,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.594633+0000","last_change":"2024-09-11T07:33:15.559485+0000","last_active":"2024-09-11T07:34:12.594633+0000","last_peered":"2024-09-11T07:34:12.594633+0000","last_clean":"2024-09-11T07:34:12.594633+0000","last_became_active":"2024-09-11T07:26:31.425876+0000","last_became_peered":"2024-09-11T07:26:31.425876+0000","last_unstale":"2024-09-11T07:34:12.594633+0000","last_undegraded":"2024-09-11T07:34:12.594633+0000","last_fullsized":"2024-09-11T07:34:12.594633+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:15.559426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:15.559426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:40.363895+0000","scrub_duration":0.0013828619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":148,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.601881+0000","last_change":"2024-09-11T07:33:11.235639+0000","last_active":"2024-09-11T07:34:12.601881+0000","last_peered":"2024-09-11T07:34:12.601881+0000","last_clean":"2024-09-11T07:34:12.601881+0000","last_became_active":"2024-09-11T07:32:00.520534+0000","last_became_peered":"2024-09-11T07:32:00.520534+0000","last_unstale":"2024-09-11T07:34:12.601881+0000","last_undegraded":"2024-09-11T07:34:12.601881+0000","last_fullsized":"2024-09-11T07:34:12.601881+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:11.235570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:39.021876+0000","last_clean_scrub_stamp":"2024-09-11T07:33:11.235570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:18.432537+0000","scrub_duration":0.001531335,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":165,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.602366+0000","last_change":"2024-09-11T07:31:05.392160+0000","last_active":"2024-09-11T07:34:12.602366+0000","last_peered":"2024-09-11T07:34:12.602366+0000","last_clean":"2024-09-11T07:34:12.602366+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:34:12.602366+0000","last_undegraded":"2024-09-11T07:34:12.602366+0000","last_fullsized":"2024-09-11T07:34:12.602366+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:31:05.391713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:31:05.391713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050876960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":152,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.600933+0000","last_change":"2024-09-11T07:33:15.248196+0000","last_active":"2024-09-11T07:34:12.600933+0000","last_peered":"2024-09-11T07:34:12.600933+0000","last_clean":"2024-09-11T07:34:12.600933+0000","last_became_active":"2024-09-11T07:32:00.511258+0000","last_became_peered":"2024-09-11T07:32:00.511258+0000","last_unstale":"2024-09-11T07:34:12.600933+0000","last_undegraded":"2024-09-11T07:34:12.600933+0000","last_fullsized":"2024-09-11T07:34:12.600933+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:33:15.248113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:15.248113+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:18.239937+0000","scrub_duration":0.0017074939999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":210,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.599677+0000","last_change":"2024-09-11T07:33:53.155118+0000","last_active":"2024-09-11T07:34:12.599677+0000","last_peered":"2024-09-11T07:34:12.599677+0000","last_clean":"2024-09-11T07:34:12.599677+0000","last_became_active":"2024-09-11T07:32:00.518307+0000","last_became_peered":"2024-09-11T07:32:00.518307+0000","last_unstale":"2024-09-11T07:34:12.599677+0000","last_undegraded":"2024-09-11T07:34:12.599677+0000","last_fullsized":"2024-09-11T07:34:12.599677+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:33:53.155024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:53.155024+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:20.182620+0000","scrub_duration":0.0038397209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":210,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.594834+0000","last_change":"2024-09-11T07:33:07.805709+0000","last_active":"2024-09-11T07:34:12.594834+0000","last_peered":"2024-09-11T07:34:12.594834+0000","last_clean":"2024-09-11T07:34:12.594834+0000","last_became_active":"2024-09-11T07:32:00.508233+0000","last_became_peered":"2024-09-11T07:32:00.508233+0000","last_unstale":"2024-09-11T07:34:12.594834+0000","last_undegraded":"2024-09-11T07:34:12.594834+0000","last_fullsized":"2024-09-11T07:34:12.594834+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:33:07.805635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:07.805635+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:28.351452+0000","scrub_duration":0.002375144,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"151'10","reported_seq":304,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-11T07:34:15.563442+0000","last_change":"2024-09-11T07:34:02.797981+0000","last_active":"2024-09-11T07:34:15.563442+0000","last_peered":"2024-09-11T07:34:15.563442+0000","last_clean":"2024-09-11T07:34:15.563442+0000","last_became_active":"2024-09-11T07:32:00.502049+0000","last_became_peered":"2024-09-11T07:32:00.502049+0000","last_unstale":"2024-09-11T07:34:15.563442+0000","last_undegraded":"2024-09-11T07:34:15.563442+0000","last_fullsized":"2024-09-11T07:34:15.563442+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:34:02.797889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:02.797889+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:19.482223+0000","scrub_duration":0.001718904,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":183,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.590415+0000","last_change":"2024-09-11T07:34:06.381128+0000","last_active":"2024-09-11T07:34:12.590415+0000","last_peered":"2024-09-11T07:34:12.590415+0000","last_clean":"2024-09-11T07:34:12.590415+0000","last_became_active":"2024-09-11T07:26:37.562501+0000","last_became_peered":"2024-09-11T07:26:37.562501+0000","last_unstale":"2024-09-11T07:34:12.590415+0000","last_undegraded":"2024-09-11T07:34:12.590415+0000","last_fullsized":"2024-09-11T07:34:12.590415+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:06.381043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:06.381043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:06.803518+0000","scrub_duration":0.0046834980000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":181,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.599119+0000","last_change":"2024-09-11T07:31:52.334717+0000","last_active":"2024-09-11T07:34:12.599119+0000","last_peered":"2024-09-11T07:34:12.599119+0000","last_clean":"2024-09-11T07:34:12.599119+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:34:12.599119+0000","last_undegraded":"2024-09-11T07:34:12.599119+0000","last_fullsized":"2024-09-11T07:34:12.599119+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:31:52.334517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:31:52.334517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029011779999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":262,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.594567+0000","last_change":"2024-09-11T07:34:11.760411+0000","last_active":"2024-09-11T07:34:12.594567+0000","last_peered":"2024-09-11T07:34:12.594567+0000","last_clean":"2024-09-11T07:34:12.594567+0000","last_became_active":"2024-09-11T07:30:28.599840+0000","last_became_peered":"2024-09-11T07:30:28.599840+0000","last_unstale":"2024-09-11T07:34:12.594567+0000","last_undegraded":"2024-09-11T07:34:12.594567+0000","last_fullsized":"2024-09-11T07:34:12.594567+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:34:11.760315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:34:11.760315+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:31.329350+0000","scrub_duration":0.007126177,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":193,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.594749+0000","last_change":"2024-09-11T07:33:13.776627+0000","last_active":"2024-09-11T07:34:12.594749+0000","last_peered":"2024-09-11T07:34:12.594749+0000","last_clean":"2024-09-11T07:34:12.594749+0000","last_became_active":"2024-09-11T07:32:00.507050+0000","last_became_peered":"2024-09-11T07:32:00.507050+0000","last_unstale":"2024-09-11T07:34:12.594749+0000","last_undegraded":"2024-09-11T07:34:12.594749+0000","last_fullsized":"2024-09-11T07:34:12.594749+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:13.776566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:24.575210+0000","last_clean_scrub_stamp":"2024-09-11T07:33:13.776566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:43.729373+0000","scrub_duration":0.001406963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":184,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.589418+0000","last_change":"2024-09-11T07:33:54.459273+0000","last_active":"2024-09-11T07:34:12.589418+0000","last_peered":"2024-09-11T07:34:12.589418+0000","last_clean":"2024-09-11T07:34:12.589418+0000","last_became_active":"2024-09-11T07:26:37.862373+0000","last_became_peered":"2024-09-11T07:26:37.862373+0000","last_unstale":"2024-09-11T07:34:12.589418+0000","last_undegraded":"2024-09-11T07:34:12.589418+0000","last_fullsized":"2024-09-11T07:34:12.589418+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:54.459223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:54.459223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:00.855517+0000","scrub_duration":0.001318018,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":211,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.593924+0000","last_change":"2024-09-11T07:34:06.757693+0000","last_active":"2024-09-11T07:34:12.593924+0000","last_peered":"2024-09-11T07:34:12.593924+0000","last_clean":"2024-09-11T07:34:12.593924+0000","last_became_active":"2024-09-11T07:32:00.503749+0000","last_became_peered":"2024-09-11T07:32:00.503749+0000","last_unstale":"2024-09-11T07:34:12.593924+0000","last_undegraded":"2024-09-11T07:34:12.593924+0000","last_fullsized":"2024-09-11T07:34:12.593924+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:06.757621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:06.757621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:18.674821+0000","scrub_duration":0.0015661609999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":214,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.602193+0000","last_change":"2024-09-11T07:33:55.212599+0000","last_active":"2024-09-11T07:34:12.602193+0000","last_peered":"2024-09-11T07:34:12.602193+0000","last_clean":"2024-09-11T07:34:12.602193+0000","last_became_active":"2024-09-11T07:26:33.542544+0000","last_became_peered":"2024-09-11T07:26:33.542544+0000","last_unstale":"2024-09-11T07:34:12.602193+0000","last_undegraded":"2024-09-11T07:34:12.602193+0000","last_fullsized":"2024-09-11T07:34:12.602193+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-09-11T07:33:55.212499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:55.212499+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:23.471681+0000","scrub_duration":0.004594473,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":185,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.590624+0000","last_change":"2024-09-11T07:34:01.366655+0000","last_active":"2024-09-11T07:34:12.590624+0000","last_peered":"2024-09-11T07:34:12.590624+0000","last_clean":"2024-09-11T07:34:12.590624+0000","last_became_active":"2024-09-11T07:26:35.555862+0000","last_became_peered":"2024-09-11T07:26:35.555862+0000","last_unstale":"2024-09-11T07:34:12.590624+0000","last_undegraded":"2024-09-11T07:34:12.590624+0000","last_fullsized":"2024-09-11T07:34:12.590624+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:01.366571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:01.366571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:19.675146+0000","scrub_duration":0.0059672780000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":175,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.598699+0000","last_change":"2024-09-11T07:33:17.489228+0000","last_active":"2024-09-11T07:34:12.598699+0000","last_peered":"2024-09-11T07:34:12.598699+0000","last_clean":"2024-09-11T07:34:12.598699+0000","last_became_active":"2024-09-11T07:26:37.861647+0000","last_became_peered":"2024-09-11T07:26:37.861647+0000","last_unstale":"2024-09-11T07:34:12.598699+0000","last_undegraded":"2024-09-11T07:34:12.598699+0000","last_fullsized":"2024-09-11T07:34:12.598699+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:17.489174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:17.489174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:20.520796+0000","scrub_duration":0.0029380980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":198,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.602674+0000","last_change":"2024-09-11T07:33:09.499435+0000","last_active":"2024-09-11T07:34:12.602674+0000","last_peered":"2024-09-11T07:34:12.602674+0000","last_clean":"2024-09-11T07:34:12.602674+0000","last_became_active":"2024-09-11T07:32:00.502947+0000","last_became_peered":"2024-09-11T07:32:00.502947+0000","last_unstale":"2024-09-11T07:34:12.602674+0000","last_undegraded":"2024-09-11T07:34:12.602674+0000","last_fullsized":"2024-09-11T07:34:12.602674+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:09.499379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:09.499379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:25.553360+0000","scrub_duration":0.001485933,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"127'17","reported_seq":207,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.598666+0000","last_change":"2024-09-11T07:33:50.604574+0000","last_active":"2024-09-11T07:34:12.598666+0000","last_peered":"2024-09-11T07:34:12.598666+0000","last_clean":"2024-09-11T07:34:12.598666+0000","last_became_active":"2024-09-11T07:26:33.542711+0000","last_became_peered":"2024-09-11T07:26:33.542711+0000","last_unstale":"2024-09-11T07:34:12.598666+0000","last_undegraded":"2024-09-11T07:34:12.598666+0000","last_fullsized":"2024-09-11T07:34:12.598666+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'17","last_scrub_stamp":"2024-09-11T07:33:50.604493+0000","last_deep_scrub":"127'17","last_deep_scrub_stamp":"2024-09-11T07:30:30.260552+0000","last_clean_scrub_stamp":"2024-09-11T07:33:50.604493+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:03.711168+0000","scrub_duration":0.0045862999999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":186,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-11T07:34:12.589024+0000","last_change":"2024-09-11T07:34:07.375537+0000","last_active":"2024-09-11T07:34:12.589024+0000","last_peered":"2024-09-11T07:34:12.589024+0000","last_clean":"2024-09-11T07:34:12.589024+0000","last_became_active":"2024-09-11T07:26:35.562037+0000","last_became_peered":"2024-09-11T07:26:35.562037+0000","last_unstale":"2024-09-11T07:34:12.589024+0000","last_undegraded":"2024-09-11T07:34:12.589024+0000","last_fullsized":"2024-09-11T07:34:12.589024+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:07.375490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:07.375490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:16.626440+0000","scrub_duration":0.0042989400000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":5128454144,"num_objects":78254,"num_object_clones":0,"num_object_copies":234762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78254,"num_write_kb":5008256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15375663104,"data_stored":15375663104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78254,"ondisk_log_size":78254,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":798,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":474,"ondisk_log_size":474,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":465,"num_read_kb":460,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":147,"seq":631360192542,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1857120,"kb_used_data":1536792,"kb_used_omap":4,"kb_used_meta":320315,"kb_avail":91888032,"statfs":{"total":95995035648,"available":94093344768,"internally_reserved":0,"allocated":1573675008,"data_stored":1573136435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":328003078},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":3.2189999999999999,"15min":3.617},"min":{"1min":0.246,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":2.7250000000000001,"5min":49.573,"15min":49.573},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":2.9049999999999998,"15min":3.2869999999999999},"min":{"1min":0.311,"5min":0.22,"15min":0.22},"max":{"1min":2.1299999999999999,"5min":49.881,"15min":49.881},"last":0.44700000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.65100000000000002,"15min":0.61499999999999999},"min":{"1min":0.29399999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.677,"5min":3.677,"15min":3.677},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.78200000000000003,"15min":0.79600000000000004},"min":{"1min":0.26000000000000001,"5min":0.246,"15min":0.246},"max":{"1min":1.919,"5min":4.6740000000000004,"15min":4.6740000000000004},"last":2.097}]},{"osd":2,"last update":"Wed Sep 11 07:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.92900000000000005,"15min":0.96799999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.141,"5min":3.6579999999999999,"15min":3.6579999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.90300000000000002,"15min":0.91300000000000003},"min":{"1min":0.318,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.141,"5min":5.0149999999999997,"15min":5.0149999999999997},"last":0.90100000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:34:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":3.0920000000000001,"15min":3.4300000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":4.4580000000000002,"5min":49.798999999999999,"15min":49.798999999999999},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":3.0619999999999998,"15min":3.3300000000000001},"min":{"1min":0.32800000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":15.316000000000001,"5min":49.524000000000001,"15min":49.524000000000001},"last":0.29299999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.42299999999999999,"15min":0.41799999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.64200000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.503},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.62,"15min":0.60799999999999998},"min":{"1min":0.33800000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.9340000000000002,"5min":3.9340000000000002,"15min":3.9340000000000002},"last":0.65700000000000003}]},{"osd":5,"last update":"Wed Sep 11 07:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.46200000000000002,"15min":0.44700000000000001},"min":{"1min":0.32300000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.64500000000000002,"15min":0.65300000000000002},"min":{"1min":0.33400000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.111,"5min":3.5640000000000001,"15min":3.5640000000000001},"last":0.63100000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.54000000000000004,"15min":0.52000000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.55800000000000005,"15min":0.54400000000000004},"min":{"1min":0.35399999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.47999999999999998}]}]},{"osd":6,"up_from":44,"seq":188978561127,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1845516,"kb_used_data":1543940,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":91899636,"statfs":{"total":95995035648,"available":94105227264,"internally_reserved":0,"allocated":1580994560,"data_stored":1580476700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:34:07 2024","interfaces":[{"interface":"back","average":{"1min":7.5389999999999997,"5min":3.7440000000000002,"15min":1.5680000000000001},"min":{"1min":0.26500000000000001,"5min":0.192,"15min":0.192},"max":{"1min":142.29599999999999,"5min":142.29599999999999,"15min":142.29599999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":7.5039999999999996,"5min":3.819,"15min":1.6799999999999999},"min":{"1min":0.39800000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":142.68799999999999,"5min":142.68799999999999,"15min":142.68799999999999},"last":0.30299999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:34:07 2024","interfaces":[{"interface":"back","average":{"1min":6.8109999999999999,"5min":2.0009999999999999,"15min":1.0629999999999999},"min":{"1min":0.317,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":127.92100000000001,"5min":127.92100000000001,"15min":127.92100000000001},"last":1.4330000000000001},{"interface":"front","average":{"1min":6.8289999999999997,"5min":2.226,"15min":1.0740000000000001},"min":{"1min":0.24099999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":127.746,"5min":127.746,"15min":127.746},"last":0.375}]},{"osd":2,"last update":"Wed Sep 11 07:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":1.1160000000000001,"15min":0.78300000000000003},"min":{"1min":0.29399999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.748,"5min":26.491,"15min":26.491},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":1.355,"15min":0.84999999999999998},"min":{"1min":0.36399999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.7969999999999999,"5min":26.561,"15min":26.561},"last":1.9590000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":1.399,"15min":0.81100000000000005},"min":{"1min":0.27600000000000002,"5min":0.254,"15min":0.254},"max":{"1min":2.1970000000000001,"5min":31.268999999999998,"15min":31.268999999999998},"last":2.1579999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":1.718,"15min":0.998},"min":{"1min":0.34100000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.6479999999999999,"5min":31.872,"15min":31.872},"last":0.64700000000000002}]},{"osd":4,"last update":"Wed Sep 11 07:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.439,"15min":0.53700000000000003},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":4.4720000000000004},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.435,"15min":0.42699999999999999},"min":{"1min":0.27700000000000002,"5min":0.25,"15min":0.25},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.55700000000000005}]},{"osd":5,"last update":"Wed Sep 11 07:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.53600000000000003,"15min":0.61899999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":2.9809999999999999,"5min":2.9809999999999999,"15min":5.7329999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.33500000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":3.4129999999999998,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":0.35599999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.309,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.514,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.70499999999999996,"15min":0.71699999999999997},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":2.9449999999999998,"5min":3.6030000000000002,"15min":3.6030000000000002},"last":0.34000000000000002}]}]},{"osd":5,"up_from":37,"seq":158913790061,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2761244,"kb_used_data":2455572,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":90983908,"statfs":{"total":95995035648,"available":93167521792,"internally_reserved":0,"allocated":2514505728,"data_stored":2513972108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:33:39 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":4.3449999999999998,"15min":1.788},"min":{"1min":0.253,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.65000000000000002,"5min":185.68199999999999,"15min":185.68199999999999},"last":1.292},{"interface":"front","average":{"1min":0.46000000000000002,"5min":4.556,"15min":1.8520000000000001},"min":{"1min":0.28199999999999997,"5min":0.218,"15min":0.218},"max":{"1min":0.59699999999999998,"5min":186.12200000000001,"15min":186.12200000000001},"last":1.859}]},{"osd":1,"last update":"Wed Sep 11 07:33:39 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.93999999999999995,"15min":0.66500000000000004},"min":{"1min":0.33400000000000002,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":9.0969999999999995,"5min":9.0969999999999995,"15min":9.0969999999999995},"last":1.3640000000000001},{"interface":"front","average":{"1min":0.496,"5min":1.069,"15min":0.68500000000000005},"min":{"1min":0.29099999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.81899999999999995,"5min":19.119,"15min":19.119},"last":1.0640000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:33:39 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":3.2389999999999999,"15min":1.4259999999999999},"min":{"1min":0.30199999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.64700000000000002,"5min":187.99199999999999,"15min":187.99199999999999},"last":3.8050000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":3.1800000000000002,"15min":1.417},"min":{"1min":0.318,"5min":0.254,"15min":0.254},"max":{"1min":0.624,"5min":186.375,"15min":186.375},"last":1.7290000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:33:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":4.5960000000000001,"15min":1.889},"min":{"1min":0.39800000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.74099999999999999,"5min":188.11500000000001,"15min":188.11500000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53500000000000003,"5min":4.5519999999999996,"15min":1.8740000000000001},"min":{"1min":0.39200000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.69899999999999995,"5min":186.44200000000001,"15min":186.44200000000001},"last":0.44400000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:33:39 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.44500000000000001,"15min":0.39700000000000002},"min":{"1min":0.30199999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.86099999999999999,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.435,"15min":0.42399999999999999},"min":{"1min":0.26600000000000001,"5min":0.253,"15min":0.23200000000000001},"max":{"1min":0.79800000000000004,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.66700000000000004}]},{"osd":6,"last update":"Wed Sep 11 07:34:08 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.503,"15min":0.44400000000000001},"min":{"1min":0.32300000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.784,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.51700000000000002,"15min":0.45800000000000002},"min":{"1min":0.28699999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":2.0369999999999999,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":0.34599999999999997}]},{"osd":7,"last update":"Wed Sep 11 07:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.772,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.375},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61299999999999999,"15min":0.60599999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":2.226,"5min":2.226,"15min":2.226},"last":0.63600000000000001}]}]},{"osd":4,"up_from":31,"seq":133143986289,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1844896,"kb_used_data":1539224,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":91900256,"statfs":{"total":95995035648,"available":94105862144,"internally_reserved":0,"allocated":1576165376,"data_stored":1575627503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":1.6080000000000001,"15min":0.88200000000000001},"min":{"1min":0.253,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":5.1020000000000003,"5min":35.308,"15min":35.308},"last":1.601},{"interface":"front","average":{"1min":0.52900000000000003,"5min":1.4279999999999999,"15min":0.80900000000000005},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.95799999999999996,"5min":35.597999999999999,"15min":35.597999999999999},"last":1.446}]},{"osd":1,"last update":"Wed Sep 11 07:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":1.1120000000000001,"15min":0.70299999999999996},"min":{"1min":0.28799999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":4.5330000000000004,"5min":9.4550000000000001,"15min":9.4550000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.30199999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.998,"5min":4.5780000000000003,"15min":4.5780000000000003},"last":1.351}]},{"osd":2,"last update":"Wed Sep 11 07:33:29 2024","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.3120000000000001,"15min":0.79200000000000004},"min":{"1min":0.29499999999999998,"5min":0.25800000000000001,"15min":0.22600000000000001},"max":{"1min":11.425000000000001,"5min":19.927,"15min":19.927},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":1.169,"15min":0.73499999999999999},"min":{"1min":0.308,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":3.0510000000000002,"5min":20.02,"15min":20.02},"last":0.315}]},{"osd":3,"last update":"Wed Sep 11 07:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":1.46,"15min":0.84399999999999997},"min":{"1min":0.378,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":9.0099999999999998,"5min":19.895,"15min":19.895},"last":1.1930000000000001},{"interface":"front","average":{"1min":0.75,"5min":1.2749999999999999,"15min":0.76300000000000001},"min":{"1min":0.34100000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":3.4169999999999998,"5min":20.260000000000002,"15min":20.260000000000002},"last":1.5069999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.40200000000000002,"15min":0.36699999999999999},"min":{"1min":0.219,"5min":0.216,"15min":0.216},"max":{"1min":0.747,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.376},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.41699999999999998,"15min":0.39300000000000002},"min":{"1min":0.25,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.73499999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:34:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":0.57799999999999996,"15min":0.47599999999999998},"min":{"1min":0.23999999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":7.8650000000000002,"5min":7.8650000000000002,"15min":7.8650000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":1.1279999999999999,"5min":0.57899999999999996,"15min":0.48699999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":9.3770000000000007,"5min":9.3770000000000007,"15min":9.3770000000000007},"last":0.58199999999999996}]},{"osd":7,"last update":"Wed Sep 11 07:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55900000000000005,"15min":0.55000000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.57099999999999995,"15min":0.55600000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":2.4020000000000001,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":0.40200000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182518,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2168936,"kb_used_data":1859104,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":91576216,"statfs":{"total":95995035648,"available":93774045184,"internally_reserved":0,"allocated":1903722496,"data_stored":1903175895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":2.202,"15min":0.99099999999999999},"min":{"1min":0.21199999999999999,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":2.4740000000000002,"5min":38.432000000000002,"15min":38.432000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":3.044,"15min":1.2989999999999999},"min":{"1min":0.21099999999999999,"5min":0.20300000000000001,"15min":0.19500000000000001},"max":{"1min":2.7650000000000001,"5min":91.876999999999995,"15min":91.876999999999995},"last":0.53600000000000003}]},{"osd":1,"last update":"Wed Sep 11 07:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":1.831,"15min":0.89600000000000002},"min":{"1min":0.23499999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":2.8170000000000002,"5min":10.475,"15min":10.475},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":1.4710000000000001,"15min":0.76900000000000002},"min":{"1min":0.25,"5min":0.215,"15min":0.215},"max":{"1min":1.998,"5min":8.1899999999999995,"15min":8.1899999999999995},"last":0.70799999999999996}]},{"osd":2,"last update":"Wed Sep 11 07:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":1.843,"15min":0.91600000000000004},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.23200000000000001},"max":{"1min":1.9650000000000001,"5min":8.282,"15min":8.282},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":1.659,"15min":0.85799999999999998},"min":{"1min":0.26200000000000001,"5min":0.21199999999999999,"15min":0.20999999999999999},"max":{"1min":2.569,"5min":8.7110000000000003,"15min":8.7110000000000003},"last":3.0099999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:33:27 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.202,"15min":0.83499999999999996},"min":{"1min":0.307,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":5.5099999999999998,"5min":5.7670000000000003,"15min":21.856000000000002},"last":2.9239999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":1.429,"15min":0.96399999999999997},"min":{"1min":0.40600000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":2.0129999999999999,"5min":8.0709999999999997,"15min":19.146000000000001},"last":0.56899999999999995}]},{"osd":5,"last update":"Wed Sep 11 07:33:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":1.544,"15min":0.88700000000000001},"min":{"1min":0.38700000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.1949999999999998,"5min":32.325000000000003,"15min":32.325000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":1.4730000000000001,"15min":0.92100000000000004},"min":{"1min":0.379,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.27,"5min":7.9870000000000001,"15min":18.416},"last":0.59799999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":1.617,"15min":1.4119999999999999},"min":{"1min":0.34899999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":2.5590000000000002,"5min":32.350999999999999,"15min":32.350999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.82599999999999996,"5min":1.631,"15min":1.4239999999999999},"min":{"1min":0.42399999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":2.609,"5min":32.402999999999999,"15min":32.402999999999999},"last":0.497}]},{"osd":7,"last update":"Wed Sep 11 07:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":1.2689999999999999,"15min":1.349},"min":{"1min":0.44600000000000001,"5min":0.308,"15min":0.308},"max":{"1min":2.8759999999999999,"5min":8.0429999999999993,"15min":8.0429999999999993},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":2.1659999999999999,"15min":2.3929999999999998},"min":{"1min":0.39400000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.5209999999999999,"5min":32.289999999999999,"15min":32.289999999999999},"last":0.91200000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411451,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2181128,"kb_used_data":1879616,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":91564024,"statfs":{"total":95995035648,"available":93761560576,"internally_reserved":0,"allocated":1924726784,"data_stored":1924212603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:33:39 2024","interfaces":[{"interface":"back","average":{"1min":3.4910000000000001,"5min":2.3260000000000001,"15min":1.0189999999999999},"min":{"1min":0.20499999999999999,"5min":0.183,"15min":0.17000000000000001},"max":{"1min":56.704000000000001,"5min":56.704000000000001,"15min":56.704000000000001},"last":3.4489999999999998},{"interface":"front","average":{"1min":3.5139999999999998,"5min":2.3490000000000002,"15min":1.0329999999999999},"min":{"1min":0.27000000000000002,"5min":0.216,"15min":0.216},"max":{"1min":56.816000000000003,"5min":56.816000000000003,"15min":56.816000000000003},"last":2.96}]},{"osd":1,"last update":"Wed Sep 11 07:33:39 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":1.6040000000000001,"15min":0.79300000000000004},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.23200000000000001},"max":{"1min":0.53700000000000003,"5min":22.175000000000001,"15min":22.175000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.39700000000000002,"5min":1.728,"15min":0.83699999999999997},"min":{"1min":0.25700000000000001,"5min":0.216,"15min":0.20699999999999999},"max":{"1min":0.746,"5min":22.609999999999999,"15min":22.609999999999999},"last":3.4820000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:34:11 2024","interfaces":[{"interface":"back","average":{"1min":2.8420000000000001,"5min":2.4980000000000002,"15min":1.1160000000000001},"min":{"1min":0.33200000000000002,"5min":0.27400000000000002,"15min":0.26900000000000002},"max":{"1min":44.405999999999999,"5min":44.405999999999999,"15min":44.405999999999999},"last":1.0780000000000001},{"interface":"front","average":{"1min":3.1960000000000002,"5min":2.649,"15min":1.1839999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":43.295999999999999,"5min":43.295999999999999,"15min":43.295999999999999},"last":2.6469999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:33:26 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.7270000000000001,"15min":0.94599999999999995},"min":{"1min":0.34599999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":6.7599999999999998,"5min":30.946999999999999,"15min":30.946999999999999},"last":2.71},{"interface":"front","average":{"1min":0.77500000000000002,"5min":1.4119999999999999,"15min":0.85399999999999998},"min":{"1min":0.44800000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.0960000000000001,"5min":22.434000000000001,"15min":22.434000000000001},"last":0.48699999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":1.7030000000000001,"15min":0.95399999999999996},"min":{"1min":0.38300000000000001,"5min":0.34000000000000002,"15min":0.31900000000000001},"max":{"1min":0.70899999999999996,"5min":31.789000000000001,"15min":31.789000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":1.6990000000000001,"15min":1.022},"min":{"1min":0.371,"5min":0.31,"15min":0.31},"max":{"1min":0.73199999999999998,"5min":30.898,"15min":30.898},"last":2.7429999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":1.6459999999999999,"15min":0.96799999999999997},"min":{"1min":0.432,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.7030000000000001,"5min":22.399000000000001,"15min":22.399000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":1.5940000000000001,"15min":0.94999999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.714,"5min":25.739000000000001,"15min":25.739000000000001},"last":2.7669999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":2.0139999999999998,"15min":2.226},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.6719999999999999,"5min":22.484000000000002,"15min":22.484000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":2.331,"15min":2.593},"min":{"1min":0.438,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.7850000000000001,"5min":25.670000000000002,"15min":25.670000000000002},"last":2.7919999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574974,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2460652,"kb_used_data":2163300,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":91284500,"statfs":{"total":95995035648,"available":93475328000,"internally_reserved":0,"allocated":2215219200,"data_stored":2214669317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:34:15 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":3.4580000000000002,"15min":1.379},"min":{"1min":0.248,"5min":0.192,"15min":0.186},"max":{"1min":8.0549999999999997,"5min":82.218999999999994,"15min":82.218999999999994},"last":0.23899999999999999},{"interface":"front","average":{"1min":1.5,"5min":3.4569999999999999,"15min":1.3939999999999999},"min":{"1min":0.22700000000000001,"5min":0.191,"15min":0.191},"max":{"1min":7.5679999999999996,"5min":82.278999999999996,"15min":82.278999999999996},"last":2.9980000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:33:40 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":1.6060000000000001,"15min":0.81000000000000005},"min":{"1min":0.27100000000000002,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":4.907,"5min":6.7610000000000001,"15min":6.7610000000000001},"last":2.7410000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":1.8580000000000001,"15min":0.89000000000000001},"min":{"1min":0.24199999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.633,"5min":8.9559999999999995,"15min":8.9559999999999995},"last":0.65600000000000003}]},{"osd":3,"last update":"Wed Sep 11 07:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":3.4409999999999998,"15min":1.4399999999999999},"min":{"1min":0.28000000000000003,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":5.4690000000000003,"5min":81.882999999999996,"15min":81.882999999999996},"last":5.9880000000000004},{"interface":"front","average":{"1min":1.0820000000000001,"5min":3.488,"15min":1.4370000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.20999999999999999},"max":{"1min":7.2809999999999997,"5min":82.400000000000006,"15min":82.400000000000006},"last":0.69999999999999996}]},{"osd":4,"last update":"Wed Sep 11 07:34:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.1879999999999999,"15min":0.75800000000000001},"min":{"1min":0.36599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":7.3630000000000004,"5min":7.3630000000000004,"15min":7.3630000000000004},"last":2.6819999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.5189999999999999,"15min":0.878},"min":{"1min":0.34399999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.2960000000000003,"5min":8.9039999999999999,"15min":8.9039999999999999},"last":0.61399999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:33:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":1.159,"15min":0.753},"min":{"1min":0.38300000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.109,"5min":6.6909999999999998,"15min":6.6909999999999998},"last":2.6419999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":1.4099999999999999,"15min":0.84499999999999997},"min":{"1min":0.32400000000000001,"5min":0.308,"15min":0.308},"max":{"1min":1.758,"5min":8.8759999999999994,"15min":8.8759999999999994},"last":0.38300000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:34:05 2024","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.6839999999999999,"15min":0.94299999999999995},"min":{"1min":0.35799999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":5.3609999999999998,"5min":24.018000000000001,"15min":24.018000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":1.444,"15min":0.85099999999999998},"min":{"1min":0.42799999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.5430000000000001,"5min":23.681000000000001,"15min":23.681000000000001},"last":0.72599999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:34:05 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.3080000000000001,"15min":1.3380000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":7.0940000000000003,"5min":7.0940000000000003,"15min":7.0940000000000003},"last":0.751},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.462,"15min":1.5089999999999999},"min":{"1min":0.45700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":7.4050000000000002,"5min":7.4050000000000002,"15min":7.4050000000000002},"last":2.7109999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738500,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2172316,"kb_used_data":1862484,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":91572836,"statfs":{"total":95995035648,"available":93770584064,"internally_reserved":0,"allocated":1907183616,"data_stored":1906649704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":1.5149999999999999,"15min":0.74399999999999999},"min":{"1min":0.21199999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":2.2229999999999999,"5min":9.7710000000000008,"15min":9.7710000000000008},"last":2.6909999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":1.585,"15min":0.76300000000000001},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":3.8769999999999998,"5min":10.275,"15min":10.275},"last":3.5329999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:33:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":1.9630000000000001,"15min":0.92200000000000004},"min":{"1min":0.23300000000000001,"5min":0.20399999999999999,"15min":0.20200000000000001},"max":{"1min":4.0570000000000004,"5min":19.527999999999999,"15min":19.527999999999999},"last":2.1869999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":1.964,"15min":0.92300000000000004},"min":{"1min":0.32300000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":3.391,"5min":19.556999999999999,"15min":19.556999999999999},"last":2.875}]},{"osd":3,"last update":"Wed Sep 11 07:34:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":3.6000000000000001,"15min":1.506},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":4.359,"5min":45.185000000000002,"15min":45.185000000000002},"last":2.4369999999999998},{"interface":"front","average":{"1min":0.80400000000000005,"5min":2.927,"15min":1.2849999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":2.7400000000000002,"5min":43.228999999999999,"15min":43.228999999999999},"last":2.085}]},{"osd":4,"last update":"Wed Sep 11 07:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":1.401,"15min":0.84299999999999997},"min":{"1min":0.27200000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":2.613,"5min":19.411999999999999,"15min":19.411999999999999},"last":2.0049999999999999},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.768,"15min":0.99399999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":4.1669999999999998,"5min":19.396000000000001,"15min":19.396000000000001},"last":2.2090000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:33:43 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.601,"15min":0.92000000000000004},"min":{"1min":0.371,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":6.9400000000000004,"5min":19.381,"15min":19.381},"last":1.968},{"interface":"front","average":{"1min":0.97599999999999998,"5min":1.8360000000000001,"15min":1.018},"min":{"1min":0.40899999999999997,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":4.1689999999999996,"5min":19.437000000000001,"15min":19.437000000000001},"last":2.2909999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:34:13 2024","interfaces":[{"interface":"back","average":{"1min":2.048,"5min":1.9750000000000001,"15min":1.0509999999999999},"min":{"1min":0.442,"5min":0.316,"15min":0.316},"max":{"1min":23.190999999999999,"5min":23.190999999999999,"15min":23.190999999999999},"last":2.27},{"interface":"front","average":{"1min":2.0819999999999999,"5min":1.984,"15min":1.0600000000000001},"min":{"1min":0.47299999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":23.632000000000001,"5min":23.632000000000001,"15min":23.632000000000001},"last":2.0310000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":1.641,"15min":1.7769999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":4.2350000000000003,"5min":5.3890000000000002,"15min":5.3890000000000002},"last":2.2480000000000002},{"interface":"front","average":{"1min":0.754,"5min":1.6919999999999999,"15min":1.8480000000000001},"min":{"1min":0.52500000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.2010000000000001,"5min":5.2759999999999998,"15min":5.2759999999999998},"last":2.1099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1934753792,"data_stored":1934753792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2249261056,"data_stored":2249261056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1931149312,"data_stored":1931149312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1929773056,"data_stored":1929773056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1594556416,"data_stored":1594556416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2552233984,"data_stored":2552233984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1588133888,"data_stored":1588133888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1595801600,"data_stored":1595801600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:34:28.198 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-11T07:34:28.198 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:34:28.198 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-11T07:34:28.198 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-11T07:34:28.210 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 48 16 82961 82945 107.976 79.875 0.0208041 0.00925536 2024-09-11T07:34:28.233 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:28.324 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:28.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:28.325 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:28.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:28 smithi022 ceph-mon[29462]: from='client.25141 v1:172.21.15.22:0/2286445656' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:34:28.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:28 smithi022 ceph-mon[37330]: from='client.25141 v1:172.21.15.22:0/2286445656' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:34:28.426 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:28.453 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:28.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:28.454 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:28.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:28 smithi104 ceph-mon[33507]: from='client.25141 v1:172.21.15.22:0/2286445656' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:34:28.555 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:28.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:28.605 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:28.605 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:28.706 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:28.807 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:28.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:28.808 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:28.843 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:34:28.909 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:28.956 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:28.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:28.957 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:29.058 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:29.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:29.096 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:29.096 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:29.197 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:29.210 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 49 16 84029 84013 107.135 66.75 0.0154691 0.00932896 2024-09-11T07:34:29.226 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:29.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:29.227 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:29.328 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:29.356 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:29.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:29.357 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:29.457 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:29.458 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:34:29 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:34:29.459 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:29 smithi104 ceph-mon[33507]: pgmap v352: 148 pgs: 148 active+clean; 4.8 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 49 MiB/s wr, 790 op/s 2024-09-11T07:34:29.459 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:29 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:34:29.459 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:29 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:34:29.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:29 smithi022 ceph-mon[29462]: pgmap v352: 148 pgs: 148 active+clean; 4.8 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 49 MiB/s wr, 790 op/s 2024-09-11T07:34:29.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:29 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:34:29.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:29 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:34:29.487 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:29 smithi022 ceph-mon[37330]: pgmap v352: 148 pgs: 148 active+clean; 4.8 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 49 MiB/s wr, 790 op/s 2024-09-11T07:34:29.487 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:29 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:34:29.487 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:29 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:34:29.499 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:29.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:29.500 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:29.601 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:29.636 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:29.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:29.637 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:29.738 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:29.770 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:29.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:29.772 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:29.872 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:29.900 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:29.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:29.901 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:30.002 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:30.016 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:34:30.016 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:34:30.042 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:30.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:30.043 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:30.144 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:30.182 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:30.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:30.183 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:30.210 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 50 16 85231 85215 106.494 75.125 0.00710241 0.00938515 2024-09-11T07:34:30.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:30 smithi104 ceph-mon[33507]: osdmap e202: 8 total, 8 up, 8 in 2024-09-11T07:34:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:30 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:30 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:34:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:30 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:34:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:30 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1051623266' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:34:30.283 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:30.325 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:30.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:30.326 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:30.427 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:30.459 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:30.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:30.460 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:30.561 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:30.589 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:30.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:30.590 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:30.628 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:34:30.628 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:34:30.628 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:34:30.628 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:34:30.629 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:34:30.629 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:34:30.629 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:34:30.629 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:34:30.629 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:34:30.629 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:34:30.629 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:34:30.629 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:34:30.629 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:34:30.629 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:34:30.629 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:34:30.630 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:34:30.630 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:30 smithi022 ceph-mon[29462]: osdmap e202: 8 total, 8 up, 8 in 2024-09-11T07:34:30.630 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:30 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:30.630 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:30 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:34:30.630 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:30 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:34:30.630 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:30 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1051623266' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:34:30.630 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:34:30 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:34:30] "GET /metrics HTTP/1.1" 200 224088 "" "Prometheus/2.43.0" 2024-09-11T07:34:30.630 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:30 smithi022 ceph-mon[37330]: osdmap e202: 8 total, 8 up, 8 in 2024-09-11T07:34:30.631 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:30 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:30.631 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:30 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:34:30.631 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:30 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:34:30.631 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:30 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1051623266' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:34:30.691 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:30.723 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:30.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:30.724 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:30.809 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-11T07:34:30.824 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:30.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:30.865 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:30.865 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:30.966 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:31.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:31.013 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:31.013 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:31.114 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:31.157 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:31.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:31.158 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:31.188 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:34:31.210 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 51 16 86480 86464 105.937 78.0625 0.00774822 0.00943494 2024-09-11T07:34:31.259 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:31.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:31.300 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:31.300 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:31.401 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:31.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:31.430 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:31.430 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:31.440 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:34:31.496 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:31 smithi104 ceph-mon[33507]: pgmap v354: 148 pgs: 148 active+clean; 5.1 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.7 KiB/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-11T07:34:31.497 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:31 smithi104 ceph-mon[33507]: osdmap e203: 8 total, 8 up, 8 in 2024-09-11T07:34:31.531 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:31.564 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:31.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:31.565 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:31.581 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:31 smithi022 ceph-mon[29462]: pgmap v354: 148 pgs: 148 active+clean; 5.1 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.7 KiB/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-11T07:34:31.581 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:31 smithi022 ceph-mon[29462]: osdmap e203: 8 total, 8 up, 8 in 2024-09-11T07:34:31.582 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:31 smithi022 ceph-mon[37330]: pgmap v354: 148 pgs: 148 active+clean; 5.1 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.7 KiB/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-11T07:34:31.582 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:31 smithi022 ceph-mon[37330]: osdmap e203: 8 total, 8 up, 8 in 2024-09-11T07:34:31.666 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:31.698 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:31.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:31.699 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:31.800 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:31.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:31.853 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:31.854 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:31.954 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:31.999 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:32.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:32.001 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:32.102 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:32.132 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:32.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:32.133 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:32.210 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 52 16 87670 87654 105.33 74.375 0.014117 0.00948937 2024-09-11T07:34:32.234 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:32.258 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:32.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:32.259 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:32.360 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:32.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:32.399 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:32.400 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:32.500 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:32.540 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:32.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:32.541 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:32.642 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:32.671 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:32.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:32.672 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:32.773 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:32.812 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:32.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:32.813 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:32.914 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:32.943 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:32.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:32.944 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:33.045 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:33.077 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:33.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:33.078 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:33.179 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:33.210 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 53 16 88844 88828 104.727 73.375 0.0135766 0.00954413 2024-09-11T07:34:33.222 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:33.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:33.223 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:33.256 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is unset 2024-09-11T07:34:33.259 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:33 smithi022 ceph-mon[29462]: pgmap v356: 148 pgs: 148 active+clean; 5.1 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 779 B/s rd, 56 MiB/s wr, 897 op/s 2024-09-11T07:34:33.259 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:33 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2687370906' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:34:33.260 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:33 smithi022 ceph-mon[37330]: pgmap v356: 148 pgs: 148 active+clean; 5.1 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 779 B/s rd, 56 MiB/s wr, 897 op/s 2024-09-11T07:34:33.260 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:33 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2687370906' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:34:33.260 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:33 smithi022 ceph-mon[37330]: from='client.25118 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:34:33.324 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:33.371 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:33.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:33.372 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:33.398 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:33 smithi104 ceph-mon[33507]: pgmap v356: 148 pgs: 148 active+clean; 5.1 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 779 B/s rd, 56 MiB/s wr, 897 op/s 2024-09-11T07:34:33.398 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:33 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2687370906' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:34:33.398 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:33 smithi104 ceph-mon[33507]: from='client.25118 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:34:33.471 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:34:33.473 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:33.509 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:33.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:33.510 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:33.539 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:34:33.539 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2024-09-11T07:34:33.539 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.539 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25072.0:89190 8.6 e204/177 8:6ed78a6c:::benchmark_data_smithi022_108406_object89189:head v 204'5525, mlcod=204'5522)", 2024-09-11T07:34:33.539 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:33.531489+0000", 2024-09-11T07:34:33.539 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.0046714460000000001, 2024-09-11T07:34:33.539 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0047073560000000002, 2024-09-11T07:34:33.539 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:33.539 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2024-09-11T07:34:33.540 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:33.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.540 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:33.540 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.531489+0000", 2024-09-11T07:34:33.540 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:33.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.540 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:33.540 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.531489+0000", 2024-09-11T07:34:33.540 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:33.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.540 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:33.540 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.531487+0000", 2024-09-11T07:34:33.540 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:34:33.541 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.541 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.541 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:33.541 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.531520+0000", 2024-09-11T07:34:33.541 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2098000000000003e-05 2024-09-11T07:34:33.541 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.541 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.541 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:33.541 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.531536+0000", 2024-09-11T07:34:33.541 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5999e-05 2024-09-11T07:34:33.541 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.541 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.541 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:33.541 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.531543+0000", 2024-09-11T07:34:33.542 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1210000000000001e-06 2024-09-11T07:34:33.542 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.542 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.542 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:33.542 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.532314+0000", 2024-09-11T07:34:33.542 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00077120699999999999 2024-09-11T07:34:33.542 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.542 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.542 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:33.542 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.532330+0000", 2024-09-11T07:34:33.542 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6401000000000001e-05 2024-09-11T07:34:33.542 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:33.542 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:33.543 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:33.543 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.543 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.543 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25072.0:89194 8.6 e204/177 8:69f18b85:::benchmark_data_smithi022_108406_object89193:head v 204'5526, mlcod=204'5522)", 2024-09-11T07:34:33.543 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:33.535350+0000", 2024-09-11T07:34:33.543 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.00081083999999999995, 2024-09-11T07:34:33.543 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000908258, 2024-09-11T07:34:33.543 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:33.543 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2024-09-11T07:34:33.543 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:33.543 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.543 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:33.543 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.535350+0000", 2024-09-11T07:34:33.543 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:33.544 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.544 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.544 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:33.544 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.535350+0000", 2024-09-11T07:34:33.544 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:33.544 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.544 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.544 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:33.544 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.535348+0000", 2024-09-11T07:34:33.544 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:34:33.544 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.545 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:33.545 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.535384+0000", 2024-09-11T07:34:33.545 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5917000000000002e-05 2024-09-11T07:34:33.545 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.545 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:33.545 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.535395+0000", 2024-09-11T07:34:33.545 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1669e-05 2024-09-11T07:34:33.545 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.545 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:33.546 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.535402+0000", 2024-09-11T07:34:33.546 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.472e-06 2024-09-11T07:34:33.546 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.546 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.546 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:33.546 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.535539+0000", 2024-09-11T07:34:33.546 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013674100000000001 2024-09-11T07:34:33.546 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.546 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.546 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:33.546 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.535559+0000", 2024-09-11T07:34:33.546 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0344999999999999e-05 2024-09-11T07:34:33.546 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:33.547 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:33.547 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:33.547 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.547 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.547 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25072.0:89197 8.d e204/177 8:b3bb63fb:::benchmark_data_smithi022_108406_object89196:head v 204'5605, mlcod=204'5603)", 2024-09-11T07:34:33.547 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:33.535421+0000", 2024-09-11T07:34:33.547 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.000739801, 2024-09-11T07:34:33.547 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00089088999999999998, 2024-09-11T07:34:33.547 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:33.547 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2024-09-11T07:34:33.547 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:33.547 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.547 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:33.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.535421+0000", 2024-09-11T07:34:33.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:33.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.548 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.548 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:33.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.535421+0000", 2024-09-11T07:34:33.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:33.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.548 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.548 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:33.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.535420+0000", 2024-09-11T07:34:33.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:34:33.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.549 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:33.549 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.535444+0000", 2024-09-11T07:34:33.549 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3757e-05 2024-09-11T07:34:33.549 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.549 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:33.549 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.535451+0000", 2024-09-11T07:34:33.549 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5190000000000003e-06 2024-09-11T07:34:33.549 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.549 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:33.549 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.535456+0000", 2024-09-11T07:34:33.550 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5630000000000002e-06 2024-09-11T07:34:33.550 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.550 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.550 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:33.550 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.535496+0000", 2024-09-11T07:34:33.550 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9977e-05 2024-09-11T07:34:33.550 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.550 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.550 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:33.550 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.535514+0000", 2024-09-11T07:34:33.550 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8859e-05 2024-09-11T07:34:33.550 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:33.550 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:33.550 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:33.551 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.551 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.551 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25072.0:89182 8.c e204/177 8:3ddbe849:::benchmark_data_smithi022_108406_object89181:head v 204'5616, mlcod=204'5611)", 2024-09-11T07:34:33.551 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:33.529191+0000", 2024-09-11T07:34:33.551 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.0069688679999999996, 2024-09-11T07:34:33.551 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0071748070000000001, 2024-09-11T07:34:33.551 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:34:33.551 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2024-09-11T07:34:33.551 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:34:33.551 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.551 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:34:33.551 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.529191+0000", 2024-09-11T07:34:33.551 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:33.552 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.552 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.552 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:34:33.552 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.529191+0000", 2024-09-11T07:34:33.552 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:34:33.552 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.552 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.552 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:34:33.552 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.529190+0000", 2024-09-11T07:34:33.552 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:34:33.552 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.552 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.553 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:34:33.553 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.529324+0000", 2024-09-11T07:34:33.553 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000133967 2024-09-11T07:34:33.553 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.553 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.553 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:34:33.553 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.529337+0000", 2024-09-11T07:34:33.553 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3087e-05 2024-09-11T07:34:33.553 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.553 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.553 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:34:33.553 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.529343+0000", 2024-09-11T07:34:33.553 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8050000000000003e-06 2024-09-11T07:34:33.554 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.554 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.554 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:34:33.554 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.529413+0000", 2024-09-11T07:34:33.554 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9967000000000003e-05 2024-09-11T07:34:33.554 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:34:33.554 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:34:33.554 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:34:33.554 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:33.529431+0000", 2024-09-11T07:34:33.554 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8235000000000001e-05 2024-09-11T07:34:33.554 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:33.554 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:34:33.554 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:33.554 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:34:33.554 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2024-09-11T07:34:33.555 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 4 2024-09-11T07:34:33.555 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:34:33.611 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:33.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:33.639 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:33.640 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:33.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:33 smithi022 ceph-mon[29462]: from='client.25118 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:34:33.740 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:33.766 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:33.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:33.768 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:33.868 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:33.901 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:33.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:33.902 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:34.003 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:34.040 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:34.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:34.041 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:34.142 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:34.171 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:34.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:34.172 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:34.212 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 54 16 90002 89986 104.128 72.375 0.00902955 0.0095985 2024-09-11T07:34:34.262 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:34:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:34:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:34:34.273 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:34.298 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-11T07:34:34.328 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:34.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:34.329 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:34.429 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:34.467 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:34.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:34.468 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:34.470 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:34 smithi022 ceph-mon[29462]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:34:34.470 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:34 smithi022 ceph-mon[29462]: from='client.25118 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:34:34.470 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:34 smithi022 ceph-mon[29462]: osdmap e204: 8 total, 8 up, 8 in 2024-09-11T07:34:34.470 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:34 smithi022 ceph-mon[37330]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:34:34.470 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:34 smithi022 ceph-mon[37330]: from='client.25118 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:34:34.470 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:34 smithi022 ceph-mon[37330]: osdmap e204: 8 total, 8 up, 8 in 2024-09-11T07:34:34.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:34 smithi104 ceph-mon[33507]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:34:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:34 smithi104 ceph-mon[33507]: from='client.25118 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:34:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:34 smithi104 ceph-mon[33507]: osdmap e204: 8 total, 8 up, 8 in 2024-09-11T07:34:34.569 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:34.581 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:34:34.628 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:34.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:34.629 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:34.661 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:34:34.730 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:34.761 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:34.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:34.762 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:34.858 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:34:34.859 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:34:34.859 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 204, 2024-09-11T07:34:34.859 INFO:teuthology.orchestra.run.smithi022.stdout: "fsid": "12a285d8-700e-11ef-bcea-c7b262605968", 2024-09-11T07:34:34.859 INFO:teuthology.orchestra.run.smithi022.stdout: "created": "2024-09-11T07:21:18.695332+0000", 2024-09-11T07:34:34.859 INFO:teuthology.orchestra.run.smithi022.stdout: "modified": "2024-09-11T07:34:33.245588+0000", 2024-09-11T07:34:34.859 INFO:teuthology.orchestra.run.smithi022.stdout: "last_up_change": "2024-09-11T07:32:00.070825+0000", 2024-09-11T07:34:34.859 INFO:teuthology.orchestra.run.smithi022.stdout: "last_in_change": "2024-09-11T07:25:58.551813+0000", 2024-09-11T07:34:34.859 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-11T07:34:34.859 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_num": 5804032, 2024-09-11T07:34:34.859 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_set": [ 2024-09-11T07:34:34.860 INFO:teuthology.orchestra.run.smithi022.stdout: "nodeep-scrub", 2024-09-11T07:34:34.860 INFO:teuthology.orchestra.run.smithi022.stdout: "pglog_hardlimit", 2024-09-11T07:34:34.860 INFO:teuthology.orchestra.run.smithi022.stdout: "purged_snapdirs", 2024-09-11T07:34:34.860 INFO:teuthology.orchestra.run.smithi022.stdout: "recovery_deletes", 2024-09-11T07:34:34.860 INFO:teuthology.orchestra.run.smithi022.stdout: "sortbitwise" 2024-09-11T07:34:34.860 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:34:34.860 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_version": 19, 2024-09-11T07:34:34.860 INFO:teuthology.orchestra.run.smithi022.stdout: "full_ratio": 0.94999998807907104, 2024-09-11T07:34:34.860 INFO:teuthology.orchestra.run.smithi022.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-11T07:34:34.860 INFO:teuthology.orchestra.run.smithi022.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-11T07:34:34.860 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_snapshot": "", 2024-09-11T07:34:34.860 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_max": 8, 2024-09-11T07:34:34.860 INFO:teuthology.orchestra.run.smithi022.stdout: "max_osd": 8, 2024-09-11T07:34:34.861 INFO:teuthology.orchestra.run.smithi022.stdout: "require_min_compat_client": "luminous", 2024-09-11T07:34:34.861 INFO:teuthology.orchestra.run.smithi022.stdout: "min_compat_client": "luminous", 2024-09-11T07:34:34.861 INFO:teuthology.orchestra.run.smithi022.stdout: "require_osd_release": "quincy", 2024-09-11T07:34:34.861 INFO:teuthology.orchestra.run.smithi022.stdout: "pools": [ 2024-09-11T07:34:34.861 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.861 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 1, 2024-09-11T07:34:34.861 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": ".mgr", 2024-09-11T07:34:34.861 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:24:21.772426+0000", 2024-09-11T07:34:34.861 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:34:34.861 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:34:34.861 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:34:34.861 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:34:34.861 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:34:34.862 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:34:34.862 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:34:34.862 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:34:34.862 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:34:34.862 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:34:34.862 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:34:34.862 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:34:34.862 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 1, 2024-09-11T07:34:34.862 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 1, 2024-09-11T07:34:34.862 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 1, 2024-09-11T07:34:34.862 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 1, 2024-09-11T07:34:34.862 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 1, 2024-09-11T07:34:34.862 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:34:34.863 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:34:34.863 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:34:34.863 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:34:34.863 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:34:34.863 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:34:34.863 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:34:34.863 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.863 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "21", 2024-09-11T07:34:34.863 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:34:34.863 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:34:34.863 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:34:34.863 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:34:34.863 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:34:34.863 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:34:34.864 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:34:34.864 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:34:34.864 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:34:34.864 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:34:34.864 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:34:34.864 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:34:34.864 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:34:34.864 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:34:34.864 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:34:34.864 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:34:34.864 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:34:34.864 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:34:34.864 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:34:34.865 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:34:34.865 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:34:34.865 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:34:34.865 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:34:34.865 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:34:34.865 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:34:34.865 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:34:34.865 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.865 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:34:34.865 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:34:34.865 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:34:34.865 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:34:34.866 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:34:34.866 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:34:34.866 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:34:34.866 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:34:34.866 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:34:34.866 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:34:34.866 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:34:34.866 INFO:teuthology.orchestra.run.smithi022.stdout: "options": { 2024-09-11T07:34:34.866 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_max": 32, 2024-09-11T07:34:34.866 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_min": 1 2024-09-11T07:34:34.866 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.866 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:34:34.866 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": {} 2024-09-11T07:34:34.866 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.867 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.867 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.867 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 2, 2024-09-11T07:34:34.867 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": ".rgw.root", 2024-09-11T07:34:34.867 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:29.941354+0000", 2024-09-11T07:34:34.867 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:34:34.867 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:34:34.867 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:34:34.867 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:34:34.867 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:34:34.867 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:34:34.867 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:34:34.867 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:34:34.868 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:34:34.868 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:34:34.868 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:34:34.868 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:34:34.868 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:34:34.868 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:34:34.868 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:34:34.868 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:34:34.868 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:34:34.868 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:34:34.868 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:34:34.868 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:34:34.868 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:34:34.868 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:34:34.869 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:34:34.869 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:34:34.869 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.869 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "58", 2024-09-11T07:34:34.869 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:34:34.869 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:34:34.869 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:34:34.869 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:34:34.869 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:34:34.869 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:34:34.869 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:34:34.869 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:34:34.869 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:34:34.870 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:34:34.870 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:34:34.870 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:34:34.870 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:34:34.870 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:34:34.870 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:34:34.870 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:34:34.870 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:34:34.870 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:34:34.870 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:34:34.870 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:34:34.870 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:34:34.870 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:34:34.870 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:34:34.871 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:34:34.871 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:34:34.871 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:34:34.871 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.871 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:34:34.871 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:34:34.871 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:34:34.871 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:34:34.871 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:34:34.871 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:34:34.871 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:34:34.871 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:34:34.871 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:34:34.871 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:34:34.872 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:34:34.872 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:34:34.872 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:34:34.872 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:34:34.872 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.872 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.872 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.872 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 3, 2024-09-11T07:34:34.872 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "datapool", 2024-09-11T07:34:34.872 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:30.396859+0000", 2024-09-11T07:34:34.872 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 8193, 2024-09-11T07:34:34.872 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T07:34:34.872 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:34:34.872 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:34:34.873 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:34:34.873 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:34:34.873 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:34:34.873 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:34:34.873 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:34:34.873 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:34:34.873 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:34:34.873 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:34:34.873 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 3, 2024-09-11T07:34:34.873 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 3, 2024-09-11T07:34:34.873 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 3, 2024-09-11T07:34:34.873 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 3, 2024-09-11T07:34:34.874 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 3, 2024-09-11T07:34:34.874 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:34:34.874 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:34:34.874 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:34:34.874 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:34:34.874 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:34:34.874 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:34:34.874 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:34:34.874 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.874 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "61", 2024-09-11T07:34:34.874 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:34:34.874 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:34:34.874 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:34:34.874 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:34:34.875 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:34:34.875 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 3, 2024-09-11T07:34:34.875 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 61, 2024-09-11T07:34:34.875 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:34:34.875 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:34:34.875 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:34:34.875 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:34:34.875 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:34:34.875 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:34:34.875 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:34:34.875 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:34:34.875 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:34:34.875 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:34:34.875 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:34:34.876 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:34:34.876 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:34:34.876 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:34:34.876 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:34:34.876 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:34:34.876 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:34:34.876 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:34:34.876 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:34:34.876 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.876 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:34.877 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:34:34.877 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:34:34.877 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:34:34.878 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:34:34.878 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:34:34.878 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:34:34.878 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:34:34.878 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:34:34.878 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:34:34.878 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:34:34.878 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:34:34.878 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:34:34.878 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:34:34.878 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd": {} 2024-09-11T07:34:34.878 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.878 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.879 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.879 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 4, 2024-09-11T07:34:34.879 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.log", 2024-09-11T07:34:34.879 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:31.922926+0000", 2024-09-11T07:34:34.879 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:34:34.879 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:34:34.879 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:34:34.879 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:34:34.879 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:34:34.879 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:34:34.879 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:34:34.879 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:34:34.879 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:34:34.880 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:34:34.880 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:34:34.880 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:34:34.880 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:34:34.880 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:34:34.880 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:34:34.880 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:34:34.880 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:34:34.880 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:34:34.880 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:34:34.880 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:34:34.880 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:34:34.880 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:34:34.881 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:34:34.881 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:34:34.881 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.881 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "60", 2024-09-11T07:34:34.881 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:34:34.881 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:34:34.881 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:34:34.881 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:34:34.881 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:34:34.881 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:34:34.881 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:34:34.881 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:34:34.881 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:34:34.881 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:34:34.882 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:34:34.882 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:34:34.882 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:34:34.882 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:34:34.882 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:34:34.882 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:34:34.882 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:34:34.882 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:34:34.882 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:34:34.882 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:34:34.882 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:34:34.882 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:34:34.882 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:34:34.882 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:34:34.883 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:34:34.883 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:34:34.883 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.883 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:34:34.883 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:34:34.883 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:34:34.883 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:34:34.883 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:34:34.883 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:34:34.883 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:34:34.883 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:34:34.883 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:34:34.883 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:34:34.884 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:34:34.884 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:34:34.884 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:34:34.884 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:34:34.884 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.884 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.884 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.884 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 5, 2024-09-11T07:34:34.884 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.control", 2024-09-11T07:34:34.884 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:34.007746+0000", 2024-09-11T07:34:34.884 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:34:34.884 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:34:34.884 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:34:34.884 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:34:34.885 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:34:34.885 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:34:34.885 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:34:34.885 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:34:34.885 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:34:34.885 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:34:34.885 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:34:34.885 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:34:34.885 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:34:34.885 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:34:34.885 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:34:34.885 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:34:34.885 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:34:34.886 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:34:34.886 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:34:34.886 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:34:34.886 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:34:34.886 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:34:34.886 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:34:34.886 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:34:34.886 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.886 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "62", 2024-09-11T07:34:34.886 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:34:34.886 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:34:34.886 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:34:34.886 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:34:34.886 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:34:34.887 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:34:34.887 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:34:34.887 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:34:34.887 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:34:34.887 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:34:34.887 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:34:34.887 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:34:34.887 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:34:34.887 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:34:34.887 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:34:34.887 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:34:34.887 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:34:34.887 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:34:34.887 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:34:34.888 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:34:34.888 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:34:34.888 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:34:34.888 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:34:34.888 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:34:34.888 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:34:34.888 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:34:34.888 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.888 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:34:34.888 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:34:34.888 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:34:34.888 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:34:34.888 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:34:34.888 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:34:34.888 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:34:34.889 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:34:34.889 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:34:34.889 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:34:34.889 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:34:34.889 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:34:34.889 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:34:34.889 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:34:34.889 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.889 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.889 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.889 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 6, 2024-09-11T07:34:34.889 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.meta", 2024-09-11T07:34:34.889 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:36.077215+0000", 2024-09-11T07:34:34.889 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:34:34.890 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:34:34.890 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:34:34.890 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:34:34.890 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:34:34.890 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:34:34.890 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:34:34.890 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:34:34.890 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:34:34.890 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:34:34.890 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:34:34.890 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:34:34.890 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:34:34.890 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:34:34.890 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:34:34.891 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:34:34.891 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:34:34.891 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:34:34.891 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:34:34.891 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:34:34.891 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:34:34.891 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:34:34.891 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:34:34.891 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:34:34.891 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.891 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "64", 2024-09-11T07:34:34.891 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:34:34.891 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:34:34.892 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:34:34.892 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:34:34.892 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:34:34.892 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:34:34.892 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:34:34.892 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:34:34.892 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:34:34.892 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:34:34.892 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:34:34.892 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:34:34.892 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:34:34.892 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:34:34.892 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:34:34.893 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:34:34.893 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:34:34.893 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:34:34.893 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:34:34.893 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:34:34.893 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:34:34.893 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:34:34.893 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:34:34.893 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:34:34.893 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:34:34.893 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:34:34.893 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.893 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:34:34.894 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:34:34.894 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:34:34.894 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:34:34.894 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:34:34.894 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:34:34.894 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:34:34.894 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:34:34.894 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:34:34.894 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:34:34.894 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:34:34.895 INFO:teuthology.orchestra.run.smithi022.stdout: "options": { 2024-09-11T07:34:34.895 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_bias": 4 2024-09-11T07:34:34.895 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.895 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:34:34.895 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:34:34.896 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.896 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.896 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.896 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 8, 2024-09-11T07:34:34.896 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "unique_pool_1", 2024-09-11T07:34:34.896 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:33:28.041392+0000", 2024-09-11T07:34:34.896 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:34:34.896 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:34:34.896 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:34:34.896 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:34:34.896 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:34:34.896 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:34:34.897 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:34:34.897 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:34:34.897 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:34:34.897 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:34:34.897 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:34:34.897 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:34:34.897 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 16, 2024-09-11T07:34:34.897 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 16, 2024-09-11T07:34:34.897 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 16, 2024-09-11T07:34:34.897 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 16, 2024-09-11T07:34:34.897 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 16, 2024-09-11T07:34:34.897 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:34:34.898 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:34:34.898 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:34:34.898 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:34:34.898 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:34:34.898 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:34:34.898 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:34:34.898 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.898 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "184", 2024-09-11T07:34:34.898 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:34:34.898 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:34:34.898 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:34:34.898 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:34:34.899 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:34:34.899 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:34:34.899 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:34:34.899 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:34:34.899 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:34:34.899 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:34:34.899 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:34:34.899 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:34:34.899 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:34:34.899 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:34:34.899 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:34:34.899 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:34:34.900 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:34:34.900 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:34:34.900 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:34:34.900 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:34:34.900 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:34:34.900 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:34:34.900 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:34:34.900 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:34:34.900 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:34:34.900 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:34:34.900 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.900 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:34:34.900 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:34:34.901 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:34:34.901 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:34:34.901 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:34:34.901 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:34:34.901 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:34:34.901 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:34:34.901 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:34:34.901 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:34:34.901 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:34:34.901 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:34:34.901 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:34:34.901 INFO:teuthology.orchestra.run.smithi022.stdout: "rados": {} 2024-09-11T07:34:34.901 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.902 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.902 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:34:34.902 INFO:teuthology.orchestra.run.smithi022.stdout: "osds": [ 2024-09-11T07:34:34.902 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.902 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 0, 2024-09-11T07:34:34.902 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "09075f16-32f4-4f88-a116-c3d7f977a0e0", 2024-09-11T07:34:34.902 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:34:34.902 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:34:34.902 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:34:34.902 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:34:34.902 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:34:34.902 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:34:34.902 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 8, 2024-09-11T07:34:34.903 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 177, 2024-09-11T07:34:34.903 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:34:34.903 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:34:34.903 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:34:34.903 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.903 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.903 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.903 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6801", 2024-09-11T07:34:34.903 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:34:34.903 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.903 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.903 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.903 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:34:34.903 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.904 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.904 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.904 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6802", 2024-09-11T07:34:34.904 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:34:34.904 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.904 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.904 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.904 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:34:34.904 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.904 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.904 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.904 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6804", 2024-09-11T07:34:34.905 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:34:34.905 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.905 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.905 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.905 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:34:34.905 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.905 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.905 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.905 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6803", 2024-09-11T07:34:34.905 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:34:34.905 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.905 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.905 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.906 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6801/2821493591", 2024-09-11T07:34:34.906 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6802/2821493591", 2024-09-11T07:34:34.906 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6804/2821493591", 2024-09-11T07:34:34.906 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6803/2821493591", 2024-09-11T07:34:34.908 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:34:34.908 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:34:34.908 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:34:34.908 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.908 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.908 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.908 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 1, 2024-09-11T07:34:34.908 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "52cc1e06-9506-48c0-9f7b-d4270dff4b9f", 2024-09-11T07:34:34.909 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:34:34.909 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:34:34.909 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:34:34.909 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:34:34.909 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:34:34.909 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:34:34.909 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 13, 2024-09-11T07:34:34.909 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 177, 2024-09-11T07:34:34.909 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:34:34.909 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:34:34.909 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:34:34.909 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.910 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.910 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.910 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6805", 2024-09-11T07:34:34.910 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:34:34.910 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.910 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.910 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.910 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:34:34.910 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.910 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.910 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.910 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6806", 2024-09-11T07:34:34.911 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:34:34.911 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.911 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.911 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.911 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:34:34.911 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.911 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.911 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.911 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6808", 2024-09-11T07:34:34.911 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:34:34.911 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.911 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.912 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.912 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:34:34.912 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.912 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.912 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.912 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6807", 2024-09-11T07:34:34.912 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:34:34.912 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.912 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.912 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.912 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6805/4037911317", 2024-09-11T07:34:34.913 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6806/4037911317", 2024-09-11T07:34:34.913 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6808/4037911317", 2024-09-11T07:34:34.913 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6807/4037911317", 2024-09-11T07:34:34.913 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:34:34.913 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:34:34.913 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:34:34.913 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.913 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.913 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.913 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 2, 2024-09-11T07:34:34.913 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "fa2b2730-2c1d-45d0-be8a-494a60dc6b9b", 2024-09-11T07:34:34.913 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:34:34.914 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:34:34.914 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:34:34.914 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:34:34.914 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:34:34.914 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:34:34.914 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 18, 2024-09-11T07:34:34.914 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 161, 2024-09-11T07:34:34.914 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:34:34.914 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:34:34.914 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:34:34.914 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.914 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.915 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.915 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6809", 2024-09-11T07:34:34.915 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:34:34.915 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.915 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.915 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.915 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:34:34.915 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.915 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.915 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.915 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6810", 2024-09-11T07:34:34.915 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:34:34.915 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.916 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.916 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.916 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:34:34.916 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.916 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.916 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.916 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6812", 2024-09-11T07:34:34.916 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:34:34.916 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.916 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.916 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.916 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:34:34.917 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.917 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.917 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.917 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6811", 2024-09-11T07:34:34.917 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:34:34.917 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.917 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.917 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.917 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6809/1675799666", 2024-09-11T07:34:34.917 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6810/1675799666", 2024-09-11T07:34:34.917 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6812/1675799666", 2024-09-11T07:34:34.917 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6811/1675799666", 2024-09-11T07:34:34.918 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:34:34.918 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:34:34.918 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:34:34.918 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.918 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.918 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.918 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 3, 2024-09-11T07:34:34.919 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "8d53d3fc-e401-47df-b70b-4b4b1f4539df", 2024-09-11T07:34:34.919 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:34:34.919 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:34:34.919 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:34:34.920 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:34:34.920 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:34:34.920 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:34:34.920 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 25, 2024-09-11T07:34:34.920 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 177, 2024-09-11T07:34:34.920 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:34:34.920 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:34:34.920 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:34:34.920 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.920 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.920 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.920 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6813", 2024-09-11T07:34:34.921 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:34:34.921 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.921 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.921 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.921 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:34:34.921 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.921 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.921 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.921 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6814", 2024-09-11T07:34:34.921 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:34:34.921 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.922 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.922 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.922 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:34:34.922 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.922 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.922 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.922 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6816", 2024-09-11T07:34:34.922 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:34:34.922 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.922 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.922 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.922 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:34:34.922 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.922 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.922 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.923 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6815", 2024-09-11T07:34:34.923 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:34:34.923 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.923 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.923 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.923 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6813/1501962752", 2024-09-11T07:34:34.923 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6814/1501962752", 2024-09-11T07:34:34.923 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6816/1501962752", 2024-09-11T07:34:34.923 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6815/1501962752", 2024-09-11T07:34:34.923 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:34:34.923 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:34:34.923 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:34:34.924 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.924 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.924 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.924 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 4, 2024-09-11T07:34:34.924 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "d3207fe0-8b97-4067-a755-ba6d449c53c7", 2024-09-11T07:34:34.924 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:34:34.924 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:34:34.924 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:34:34.924 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:34:34.924 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:34:34.924 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:34:34.924 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 31, 2024-09-11T07:34:34.924 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 177, 2024-09-11T07:34:34.924 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:34:34.925 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:34:34.925 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:34:34.925 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.925 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.925 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.925 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6800", 2024-09-11T07:34:34.925 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:34:34.925 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.925 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.925 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.925 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:34:34.925 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.926 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.926 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.926 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6801", 2024-09-11T07:34:34.926 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:34:34.926 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.926 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.926 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.926 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:34:34.926 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.926 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.926 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.926 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6803", 2024-09-11T07:34:34.926 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:34:34.926 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.927 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.927 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.927 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:34:34.927 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.927 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.927 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.927 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6802", 2024-09-11T07:34:34.927 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:34:34.927 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.927 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.927 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.927 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6800/1210229443", 2024-09-11T07:34:34.927 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6801/1210229443", 2024-09-11T07:34:34.928 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6803/1210229443", 2024-09-11T07:34:34.928 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6802/1210229443", 2024-09-11T07:34:34.928 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:34:34.928 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:34:34.928 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:34:34.928 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.928 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.928 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.928 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 5, 2024-09-11T07:34:34.928 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "89c9352b-30b2-48c6-869e-047d07a23a43", 2024-09-11T07:34:34.928 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:34:34.928 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:34:34.928 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:34:34.929 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:34:34.929 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:34:34.929 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:34:34.929 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 37, 2024-09-11T07:34:34.929 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 177, 2024-09-11T07:34:34.929 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:34:34.929 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:34:34.929 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:34:34.929 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.929 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.929 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.929 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6804", 2024-09-11T07:34:34.930 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:34:34.930 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.930 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.930 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.930 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:34:34.930 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.930 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.930 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.930 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6805", 2024-09-11T07:34:34.930 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:34:34.930 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.930 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.930 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.931 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:34:34.931 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.931 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.931 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.931 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6807", 2024-09-11T07:34:34.931 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:34:34.931 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.931 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.931 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.931 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:34:34.931 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.931 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.932 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.932 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6806", 2024-09-11T07:34:34.932 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:34:34.932 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.932 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.932 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.932 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6804/2735328732", 2024-09-11T07:34:34.932 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6805/2735328732", 2024-09-11T07:34:34.932 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6807/2735328732", 2024-09-11T07:34:34.932 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6806/2735328732", 2024-09-11T07:34:34.932 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:34:34.933 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:34:34.933 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:34:34.933 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.933 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.933 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.933 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 6, 2024-09-11T07:34:34.933 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "704563de-d88f-480a-b469-720135704d42", 2024-09-11T07:34:34.933 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:34:34.933 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:34:34.933 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:34:34.933 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:34:34.933 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:34:34.934 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:34:34.934 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 44, 2024-09-11T07:34:34.934 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 177, 2024-09-11T07:34:34.934 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:34:34.934 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:34:34.934 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:34:34.934 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.934 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.934 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.934 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6808", 2024-09-11T07:34:34.934 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:34:34.934 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.935 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.935 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.935 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:34:34.935 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.935 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.935 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.935 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6809", 2024-09-11T07:34:34.935 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:34:34.935 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.935 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.935 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.935 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:34:34.935 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.936 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.936 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.936 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6811", 2024-09-11T07:34:34.936 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:34:34.936 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.936 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.936 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.936 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:34:34.936 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.936 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.936 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.936 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6810", 2024-09-11T07:34:34.936 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:34:34.937 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.937 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.937 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.937 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6808/721493010", 2024-09-11T07:34:34.937 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6809/721493010", 2024-09-11T07:34:34.937 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6811/721493010", 2024-09-11T07:34:34.937 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6810/721493010", 2024-09-11T07:34:34.937 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:34:34.937 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:34:34.937 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:34:34.937 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.937 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.937 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.938 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 7, 2024-09-11T07:34:34.938 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "18199e47-0052-49f0-b45f-0fb201e0d0f4", 2024-09-11T07:34:34.938 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:34:34.938 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:34:34.938 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:34:34.938 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:34:34.938 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 52, 2024-09-11T07:34:34.938 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 133, 2024-09-11T07:34:34.938 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 147, 2024-09-11T07:34:34.938 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 167, 2024-09-11T07:34:34.938 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 134, 2024-09-11T07:34:34.938 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:34:34.938 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:34:34.939 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.939 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.939 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.939 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6812", 2024-09-11T07:34:34.939 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2218863853 2024-09-11T07:34:34.939 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.939 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.939 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.939 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:34:34.939 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.939 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.939 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.939 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6813", 2024-09-11T07:34:34.940 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2218863853 2024-09-11T07:34:34.940 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.940 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.940 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.940 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:34:34.940 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.940 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.940 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.940 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6815", 2024-09-11T07:34:34.940 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2218863853 2024-09-11T07:34:34.940 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.940 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.941 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.941 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:34:34.941 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:34:34.941 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.941 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:34:34.941 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6814", 2024-09-11T07:34:34.941 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2218863853 2024-09-11T07:34:34.941 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.941 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.941 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.941 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6812/2218863853", 2024-09-11T07:34:34.941 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6813/2218863853", 2024-09-11T07:34:34.941 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6815/2218863853", 2024-09-11T07:34:34.941 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6814/2218863853", 2024-09-11T07:34:34.942 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:34:34.942 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:34:34.942 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:34:34.942 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.942 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.942 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:34:34.942 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_xinfo": [ 2024-09-11T07:34:34.942 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.942 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 0, 2024-09-11T07:34:34.942 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:34:34.942 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:34:34.942 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:34:34.943 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:34:34.943 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:34:34.943 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:33:53.829080+0000", 2024-09-11T07:34:34.943 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:34:34.943 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.943 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.943 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 1, 2024-09-11T07:34:34.943 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:34:34.943 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:34:34.943 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:34:34.943 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:34:34.943 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:34:34.943 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:33:41.323675+0000", 2024-09-11T07:34:34.943 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:34:34.944 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.944 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.944 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 2, 2024-09-11T07:34:34.944 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:34:34.944 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:34:34.944 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:34:34.944 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:34:34.944 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:34:34.944 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:33:51.706870+0000", 2024-09-11T07:34:34.944 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:34:34.944 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.944 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.944 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 3, 2024-09-11T07:34:34.944 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:34:34.945 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:34:34.945 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:34:34.945 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:34:34.945 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:34:34.945 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:33:48.417037+0000", 2024-09-11T07:34:34.945 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:34:34.945 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.945 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.945 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 4, 2024-09-11T07:34:34.945 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:34:34.945 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:34:34.945 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:34:34.945 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:34:34.946 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:34:34.946 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:33:32.098156+0000", 2024-09-11T07:34:34.946 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:34:34.946 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.946 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.946 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 5, 2024-09-11T07:34:34.946 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:34:34.946 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:34:34.946 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:34:34.946 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:34:34.946 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:34:34.946 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:34:29.436631+0000", 2024-09-11T07:34:34.947 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:34:34.947 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.947 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.947 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 6, 2024-09-11T07:34:34.947 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:34:34.947 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:34:34.947 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:34:34.947 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:34:34.947 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:34:34.947 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:34:28.486370+0000", 2024-09-11T07:34:34.947 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:34:34.947 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.947 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.948 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 7, 2024-09-11T07:34:34.948 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:34:34.948 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:34:34.948 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:34:34.948 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:34:34.948 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:34:34.948 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:33:29.278265+0000", 2024-09-11T07:34:34.948 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 133 2024-09-11T07:34:34.948 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.948 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:34:34.948 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_upmap": [ 2024-09-11T07:34:34.948 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.948 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "2.1c", 2024-09-11T07:34:34.949 INFO:teuthology.orchestra.run.smithi022.stdout: "osds": [ 2024-09-11T07:34:34.949 INFO:teuthology.orchestra.run.smithi022.stdout: 5, 2024-09-11T07:34:34.949 INFO:teuthology.orchestra.run.smithi022.stdout: 2, 2024-09-11T07:34:34.949 INFO:teuthology.orchestra.run.smithi022.stdout: 7 2024-09-11T07:34:34.949 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.949 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.949 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:34:34.949 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_upmap_items": [ 2024-09-11T07:34:34.949 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.949 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "2.7", 2024-09-11T07:34:34.949 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:34:34.949 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.949 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 7, 2024-09-11T07:34:34.950 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 4 2024-09-11T07:34:34.950 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.950 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.950 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.950 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.950 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.4", 2024-09-11T07:34:34.950 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:34:34.950 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.950 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 1, 2024-09-11T07:34:34.950 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 5 2024-09-11T07:34:34.950 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.950 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.951 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.951 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.951 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.6", 2024-09-11T07:34:34.951 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:34:34.951 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.951 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 1, 2024-09-11T07:34:34.951 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 5 2024-09-11T07:34:34.951 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.951 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.951 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.951 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.951 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.14", 2024-09-11T07:34:34.951 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:34:34.951 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:34:34.952 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 1, 2024-09-11T07:34:34.952 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 2 2024-09-11T07:34:34.952 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.952 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:34:34.952 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.952 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:34:34.952 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_temp": [], 2024-09-11T07:34:34.952 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_temp": [], 2024-09-11T07:34:34.952 INFO:teuthology.orchestra.run.smithi022.stdout: "blocklist": { 2024-09-11T07:34:34.952 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/240999923": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:34:34.952 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/4016369460": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:34:34.952 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/585929238": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:34:34.952 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/544031024": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:34:34.953 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/2504196132": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:34:34.953 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/1451388349": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:34:34.953 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/639336081": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:34:34.953 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2104069730": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:34:34.953 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/4236450731": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:34:34.953 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/3573193776": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:34:34.953 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2661764422": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:34:34.953 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2197488530": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:34:34.953 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2968034970": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:34:34.953 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/599318175": "2024-09-12T07:27:07.919998+0000" 2024-09-11T07:34:34.953 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.953 INFO:teuthology.orchestra.run.smithi022.stdout: "range_blocklist": {}, 2024-09-11T07:34:34.953 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profiles": { 2024-09-11T07:34:34.954 INFO:teuthology.orchestra.run.smithi022.stdout: "default": { 2024-09-11T07:34:34.954 INFO:teuthology.orchestra.run.smithi022.stdout: "crush-failure-domain": "osd", 2024-09-11T07:34:34.954 INFO:teuthology.orchestra.run.smithi022.stdout: "k": "2", 2024-09-11T07:34:34.954 INFO:teuthology.orchestra.run.smithi022.stdout: "m": "1", 2024-09-11T07:34:34.954 INFO:teuthology.orchestra.run.smithi022.stdout: "plugin": "jerasure", 2024-09-11T07:34:34.954 INFO:teuthology.orchestra.run.smithi022.stdout: "technique": "reed_sol_van" 2024-09-11T07:34:34.954 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.954 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:34:34.954 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps_queue": [], 2024-09-11T07:34:34.954 INFO:teuthology.orchestra.run.smithi022.stdout: "new_removed_snaps": [], 2024-09-11T07:34:34.954 INFO:teuthology.orchestra.run.smithi022.stdout: "new_purged_snaps": [], 2024-09-11T07:34:34.954 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_node_flags": {}, 2024-09-11T07:34:34.954 INFO:teuthology.orchestra.run.smithi022.stdout: "device_class_flags": {}, 2024-09-11T07:34:34.954 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode": { 2024-09-11T07:34:34.955 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode_enabled": false, 2024-09-11T07:34:34.955 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_bucket_count": 0, 2024-09-11T07:34:34.955 INFO:teuthology.orchestra.run.smithi022.stdout: "degraded_stretch_mode": 0, 2024-09-11T07:34:34.955 INFO:teuthology.orchestra.run.smithi022.stdout: "recovering_stretch_mode": 0, 2024-09-11T07:34:34.955 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode_bucket": 0 2024-09-11T07:34:34.955 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:34:34.955 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:34:34.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:34.956 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:34.956 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:35.057 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:35.083 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:35.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:35.084 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:35.185 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:35.210 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 55 16 91114 91098 103.498 69.5 0.0075448 0.00965729 2024-09-11T07:34:35.225 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:35.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:35.226 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:35.326 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:35.368 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:35.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:35.369 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:35.470 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:35.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:35.510 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:35.511 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:35.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:35 smithi104 ceph-mon[33507]: 6.11 scrub starts 2024-09-11T07:34:35.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:35 smithi104 ceph-mon[33507]: 6.11 scrub ok 2024-09-11T07:34:35.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:35 smithi104 ceph-mon[33507]: 6.18 scrub starts 2024-09-11T07:34:35.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:35 smithi104 ceph-mon[33507]: 6.18 scrub ok 2024-09-11T07:34:35.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:35 smithi104 ceph-mon[33507]: 5.d scrub starts 2024-09-11T07:34:35.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:35 smithi104 ceph-mon[33507]: 5.d scrub ok 2024-09-11T07:34:35.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:35 smithi104 ceph-mon[33507]: pgmap v358: 148 pgs: 148 active+clean; 5.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-11T07:34:35.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:35 smithi104 ceph-mon[33507]: 4.4 scrub starts 2024-09-11T07:34:35.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:35 smithi104 ceph-mon[33507]: 4.4 scrub ok 2024-09-11T07:34:35.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:35 smithi104 ceph-mon[33507]: 4.18 scrub starts 2024-09-11T07:34:35.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:35 smithi104 ceph-mon[33507]: 4.18 scrub ok 2024-09-11T07:34:35.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:35 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2009024933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:34:35.611 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:35.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:35.644 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:35.644 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:35.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:35 smithi022 ceph-mon[29462]: 6.11 scrub starts 2024-09-11T07:34:35.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:35 smithi022 ceph-mon[29462]: 6.11 scrub ok 2024-09-11T07:34:35.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:35 smithi022 ceph-mon[29462]: 6.18 scrub starts 2024-09-11T07:34:35.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:35 smithi022 ceph-mon[29462]: 6.18 scrub ok 2024-09-11T07:34:35.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:35 smithi022 ceph-mon[29462]: 5.d scrub starts 2024-09-11T07:34:35.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:35 smithi022 ceph-mon[29462]: 5.d scrub ok 2024-09-11T07:34:35.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:35 smithi022 ceph-mon[29462]: pgmap v358: 148 pgs: 148 active+clean; 5.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-11T07:34:35.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:35 smithi022 ceph-mon[29462]: 4.4 scrub starts 2024-09-11T07:34:35.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:35 smithi022 ceph-mon[29462]: 4.4 scrub ok 2024-09-11T07:34:35.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:35 smithi022 ceph-mon[29462]: 4.18 scrub starts 2024-09-11T07:34:35.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:35 smithi022 ceph-mon[29462]: 4.18 scrub ok 2024-09-11T07:34:35.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:35 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2009024933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:34:35.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:35 smithi022 ceph-mon[37330]: 6.11 scrub starts 2024-09-11T07:34:35.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:35 smithi022 ceph-mon[37330]: 6.11 scrub ok 2024-09-11T07:34:35.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:35 smithi022 ceph-mon[37330]: 6.18 scrub starts 2024-09-11T07:34:35.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:35 smithi022 ceph-mon[37330]: 6.18 scrub ok 2024-09-11T07:34:35.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:35 smithi022 ceph-mon[37330]: 5.d scrub starts 2024-09-11T07:34:35.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:35 smithi022 ceph-mon[37330]: 5.d scrub ok 2024-09-11T07:34:35.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:35 smithi022 ceph-mon[37330]: pgmap v358: 148 pgs: 148 active+clean; 5.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-11T07:34:35.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:35 smithi022 ceph-mon[37330]: 4.4 scrub starts 2024-09-11T07:34:35.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:35 smithi022 ceph-mon[37330]: 4.4 scrub ok 2024-09-11T07:34:35.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:35 smithi022 ceph-mon[37330]: 4.18 scrub starts 2024-09-11T07:34:35.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:35 smithi022 ceph-mon[37330]: 4.18 scrub ok 2024-09-11T07:34:35.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:35 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2009024933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:34:35.745 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:35.766 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:35.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:35.767 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:35.868 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:35.919 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:35.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:35.920 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:35.944 INFO:tasks.thrashosds.thrasher:j is {'epoch': 204, 'fsid': '12a285d8-700e-11ef-bcea-c7b262605968', 'created': '2024-09-11T07:21:18.695332+0000', 'modified': '2024-09-11T07:34:33.245588+0000', 'last_up_change': '2024-09-11T07:32:00.070825+0000', 'last_in_change': '2024-09-11T07:25:58.551813+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 19, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 8, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T07:24:21.772426+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': '.rgw.root', 'create_time': '2024-09-11T07:26:29.941354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 3, 'pool_name': 'datapool', 'create_time': '2024-09-11T07:26:30.396859+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-11T07:26:31.922926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-11T07:26:34.007746+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-11T07:26:36.077215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}, {'pool': 8, 'pool_name': 'unique_pool_1', 'create_time': '2024-09-11T07:33:28.041392+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '184', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '09075f16-32f4-4f88-a116-c3d7f977a0e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 177, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6801', 'nonce': 2821493591}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6802', 'nonce': 2821493591}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6804', 'nonce': 2821493591}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6803', 'nonce': 2821493591}]}, 'public_addr': '172.21.15.22:6801/2821493591', 'cluster_addr': '172.21.15.22:6802/2821493591', 'heartbeat_back_addr': '172.21.15.22:6804/2821493591', 'heartbeat_front_addr': '172.21.15.22:6803/2821493591', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '52cc1e06-9506-48c0-9f7b-d4270dff4b9f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 177, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6805', 'nonce': 4037911317}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6806', 'nonce': 4037911317}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6808', 'nonce': 4037911317}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6807', 'nonce': 4037911317}]}, 'public_addr': '172.21.15.22:6805/4037911317', 'cluster_addr': '172.21.15.22:6806/4037911317', 'heartbeat_back_addr': '172.21.15.22:6808/4037911317', 'heartbeat_front_addr': '172.21.15.22:6807/4037911317', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'fa2b2730-2c1d-45d0-be8a-494a60dc6b9b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 161, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6809', 'nonce': 1675799666}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6810', 'nonce': 1675799666}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6812', 'nonce': 1675799666}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6811', 'nonce': 1675799666}]}, 'public_addr': '172.21.15.22:6809/1675799666', 'cluster_addr': '172.21.15.22:6810/1675799666', 'heartbeat_back_addr': '172.21.15.22:6812/1675799666', 'heartbeat_front_addr': '172.21.15.22:6811/1675799666', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '8d53d3fc-e401-47df-b70b-4b4b1f4539df', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 25, 'up_thru': 177, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6813', 'nonce': 1501962752}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6814', 'nonce': 1501962752}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6816', 'nonce': 1501962752}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6815', 'nonce': 1501962752}]}, 'public_addr': '172.21.15.22:6813/1501962752', 'cluster_addr': '172.21.15.22:6814/1501962752', 'heartbeat_back_addr': '172.21.15.22:6816/1501962752', 'heartbeat_front_addr': '172.21.15.22:6815/1501962752', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'd3207fe0-8b97-4067-a755-ba6d449c53c7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 177, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6800', 'nonce': 1210229443}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6801', 'nonce': 1210229443}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6803', 'nonce': 1210229443}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6802', 'nonce': 1210229443}]}, 'public_addr': '172.21.15.104:6800/1210229443', 'cluster_addr': '172.21.15.104:6801/1210229443', 'heartbeat_back_addr': '172.21.15.104:6803/1210229443', 'heartbeat_front_addr': '172.21.15.104:6802/1210229443', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '89c9352b-30b2-48c6-869e-047d07a23a43', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 37, 'up_thru': 177, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6804', 'nonce': 2735328732}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6805', 'nonce': 2735328732}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6807', 'nonce': 2735328732}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6806', 'nonce': 2735328732}]}, 'public_addr': '172.21.15.104:6804/2735328732', 'cluster_addr': '172.21.15.104:6805/2735328732', 'heartbeat_back_addr': '172.21.15.104:6807/2735328732', 'heartbeat_front_addr': '172.21.15.104:6806/2735328732', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '704563de-d88f-480a-b469-720135704d42', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 44, 'up_thru': 177, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6808', 'nonce': 721493010}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6809', 'nonce': 721493010}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6811', 'nonce': 721493010}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6810', 'nonce': 721493010}]}, 'public_addr': '172.21.15.104:6808/721493010', 'cluster_addr': '172.21.15.104:6809/721493010', 'heartbeat_back_addr': '172.21.15.104:6811/721493010', 'heartbeat_front_addr': '172.21.15.104:6810/721493010', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '18199e47-0052-49f0-b45f-0fb201e0d0f4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 52, 'last_clean_end': 133, 'up_from': 147, 'up_thru': 167, 'down_at': 134, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6812', 'nonce': 2218863853}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6813', 'nonce': 2218863853}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6815', 'nonce': 2218863853}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6814', 'nonce': 2218863853}]}, 'public_addr': '172.21.15.104:6812/2218863853', 'cluster_addr': '172.21.15.104:6813/2218863853', 'heartbeat_back_addr': '172.21.15.104:6815/2218863853', 'heartbeat_front_addr': '172.21.15.104:6814/2218863853', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:33:53.829080+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:33:41.323675+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:33:51.706870+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:33:48.417037+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:33:32.098156+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:34:29.436631+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:34:28.486370+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:33:29.278265+0000', 'dead_epoch': 133}], 'pg_upmap': [{'pgid': '2.1c', 'osds': [5, 2, 7]}], 'pg_upmap_items': [{'pgid': '2.7', 'mappings': [{'from': 7, 'to': 4}]}, {'pgid': '4.4', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.6', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.14', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.22:0/240999923': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:6800/4016369460': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/585929238': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/544031024': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:6800/2504196132': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/1451388349': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/639336081': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/2104069730': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/4236450731': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:6800/3573193776': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/2661764422': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/2197488530': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/2968034970': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/599318175': '2024-09-12T07:27:07.919998+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-11T07:34:35.945 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:34:36.021 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:36.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:36.075 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:36.075 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:36.176 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:36.200 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:36.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:36.202 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:36.210 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 56 15 92245 92230 102.913 70.75 0.00670056 0.00971152 2024-09-11T07:34:36.251 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T07:34:36.303 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:36.396 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:36.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:36.397 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:36.498 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:36.524 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:36.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:36.525 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:36.626 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:36.653 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:36.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:36.655 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:36.755 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:36.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:36.792 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:36.793 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:36.893 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:36.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:36 smithi022 ceph-mon[29462]: 6.13 scrub starts 2024-09-11T07:34:36.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:36 smithi022 ceph-mon[29462]: 6.13 scrub ok 2024-09-11T07:34:36.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:36 smithi022 ceph-mon[29462]: 6.a scrub starts 2024-09-11T07:34:36.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:36 smithi022 ceph-mon[29462]: 6.a scrub ok 2024-09-11T07:34:36.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:36 smithi022 ceph-mon[29462]: osdmap e205: 8 total, 8 up, 8 in 2024-09-11T07:34:36.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:36 smithi022 ceph-mon[29462]: 4.3 scrub starts 2024-09-11T07:34:36.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:36 smithi022 ceph-mon[29462]: 4.3 scrub ok 2024-09-11T07:34:36.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:36 smithi022 ceph-mon[29462]: pgmap v360: 148 pgs: 148 active+clean; 5.5 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 62 MiB/s wr, 994 op/s 2024-09-11T07:34:36.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:36 smithi022 ceph-mon[37330]: 6.13 scrub starts 2024-09-11T07:34:36.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:36 smithi022 ceph-mon[37330]: 6.13 scrub ok 2024-09-11T07:34:36.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:36 smithi022 ceph-mon[37330]: 6.a scrub starts 2024-09-11T07:34:36.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:36 smithi022 ceph-mon[37330]: 6.a scrub ok 2024-09-11T07:34:36.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:36 smithi022 ceph-mon[37330]: osdmap e205: 8 total, 8 up, 8 in 2024-09-11T07:34:36.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:36 smithi022 ceph-mon[37330]: 4.3 scrub starts 2024-09-11T07:34:36.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:36 smithi022 ceph-mon[37330]: 4.3 scrub ok 2024-09-11T07:34:36.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:36 smithi022 ceph-mon[37330]: pgmap v360: 148 pgs: 148 active+clean; 5.5 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 62 MiB/s wr, 994 op/s 2024-09-11T07:34:36.936 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:36.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:36.937 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:36 smithi104 ceph-mon[33507]: 6.13 scrub starts 2024-09-11T07:34:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:36 smithi104 ceph-mon[33507]: 6.13 scrub ok 2024-09-11T07:34:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:36 smithi104 ceph-mon[33507]: 6.a scrub starts 2024-09-11T07:34:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:36 smithi104 ceph-mon[33507]: 6.a scrub ok 2024-09-11T07:34:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:36 smithi104 ceph-mon[33507]: osdmap e205: 8 total, 8 up, 8 in 2024-09-11T07:34:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:36 smithi104 ceph-mon[33507]: 4.3 scrub starts 2024-09-11T07:34:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:36 smithi104 ceph-mon[33507]: 4.3 scrub ok 2024-09-11T07:34:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:36 smithi104 ceph-mon[33507]: pgmap v360: 148 pgs: 148 active+clean; 5.5 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 62 MiB/s wr, 994 op/s 2024-09-11T07:34:37.038 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:37.069 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:37.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:37.070 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:37.171 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:37.211 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 57 16 93405 93389 102.378 72.4375 0.0107893 0.00976203 2024-09-11T07:34:37.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:37.226 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:37.227 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:37.327 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:37.379 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:37.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:37.380 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:37.481 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:37.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:37.517 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:37.517 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:37.618 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:37.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:37 smithi022 ceph-mon[29462]: 5.11 scrub starts 2024-09-11T07:34:37.661 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:37 smithi022 ceph-mon[29462]: 5.11 scrub ok 2024-09-11T07:34:37.661 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:37 smithi022 ceph-mon[29462]: 5.a scrub starts 2024-09-11T07:34:37.661 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:37 smithi022 ceph-mon[29462]: 5.a scrub ok 2024-09-11T07:34:37.661 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:37 smithi022 ceph-mon[37330]: 5.11 scrub starts 2024-09-11T07:34:37.661 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:37 smithi022 ceph-mon[37330]: 5.11 scrub ok 2024-09-11T07:34:37.661 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:37 smithi022 ceph-mon[37330]: 5.a scrub starts 2024-09-11T07:34:37.661 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:37 smithi022 ceph-mon[37330]: 5.a scrub ok 2024-09-11T07:34:37.668 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:37.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:37.669 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:37.701 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:37 smithi104 ceph-mon[33507]: 5.11 scrub starts 2024-09-11T07:34:37.701 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:37 smithi104 ceph-mon[33507]: 5.11 scrub ok 2024-09-11T07:34:37.702 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:37 smithi104 ceph-mon[33507]: 5.a scrub starts 2024-09-11T07:34:37.702 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:37 smithi104 ceph-mon[33507]: 5.a scrub ok 2024-09-11T07:34:37.726 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:34:37.726 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:34:37.726 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2024-09-11T07:34:37.726 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2024-09-11T07:34:37.727 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:34:37.770 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:37.799 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:37.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:37.801 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:37.901 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:37.938 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:37.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:37.939 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:38.040 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:38.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:38.081 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:38.082 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:38.169 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:34:38.169 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:34:38.170 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:34:38.170 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:34:38.170 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:34:38.170 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:34:38.170 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:34:38.170 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:34:38.170 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:34:38.170 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:34:38.170 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:34:38.170 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:34:38.183 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:38.191 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:34:38.202 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:34:38.202 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:34:38.207 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:38.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:38.208 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:38.211 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 58 16 94480 94464 101.771 67.1875 0.00993556 0.00981989 2024-09-11T07:34:38.232 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:34:38.233 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:34:38.260 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:34:38.309 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:38.339 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:38.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:38.341 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:38.441 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:38.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:38.475 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:38.475 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:38.576 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:38.605 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:38.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:38.606 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:38.707 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:38.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:38.736 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:38.736 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:38.832 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:38 smithi022 ceph-mon[29462]: 2.e scrub starts 2024-09-11T07:34:38.832 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:38 smithi022 ceph-mon[29462]: 2.e scrub ok 2024-09-11T07:34:38.832 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:38 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1022403543' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:34:38.832 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:34:38.832 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:38 smithi022 ceph-mon[29462]: pgmap v361: 148 pgs: 148 active+clean; 5.5 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 650 B/s rd, 47 MiB/s wr, 758 op/s 2024-09-11T07:34:38.833 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:38 smithi022 ceph-mon[29462]: 6.8 scrub starts 2024-09-11T07:34:38.833 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:38 smithi022 ceph-mon[29462]: 6.8 scrub ok 2024-09-11T07:34:38.833 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:38 smithi022 ceph-mon[37330]: 2.e scrub starts 2024-09-11T07:34:38.833 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:38 smithi022 ceph-mon[37330]: 2.e scrub ok 2024-09-11T07:34:38.833 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:38 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1022403543' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:34:38.833 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:34:38.833 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:38 smithi022 ceph-mon[37330]: pgmap v361: 148 pgs: 148 active+clean; 5.5 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 650 B/s rd, 47 MiB/s wr, 758 op/s 2024-09-11T07:34:38.833 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:38 smithi022 ceph-mon[37330]: 6.8 scrub starts 2024-09-11T07:34:38.833 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:38 smithi022 ceph-mon[37330]: 6.8 scrub ok 2024-09-11T07:34:38.837 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:38.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:38.887 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:38.887 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:38.988 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:39.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:38 smithi104 ceph-mon[33507]: 2.e scrub starts 2024-09-11T07:34:39.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:38 smithi104 ceph-mon[33507]: 2.e scrub ok 2024-09-11T07:34:39.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:38 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1022403543' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:34:39.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:34:39.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:38 smithi104 ceph-mon[33507]: pgmap v361: 148 pgs: 148 active+clean; 5.5 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 650 B/s rd, 47 MiB/s wr, 758 op/s 2024-09-11T07:34:39.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:38 smithi104 ceph-mon[33507]: 6.8 scrub starts 2024-09-11T07:34:39.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:38 smithi104 ceph-mon[33507]: 6.8 scrub ok 2024-09-11T07:34:39.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:39.041 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:39.042 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:39.142 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:39.172 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:39.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:39.173 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:39.212 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 59 16 95530 95514 101.158 65.625 0.0176461 0.00987998 2024-09-11T07:34:39.274 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:39.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:39.318 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:39.318 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:39.419 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:39.450 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:39.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:39.451 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:39.552 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:39.583 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:39.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:39.584 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:39.614 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-11T07:34:39.682 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:39 smithi104 ceph-mon[33507]: 6.3 scrub starts 2024-09-11T07:34:39.682 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:39 smithi104 ceph-mon[33507]: 6.3 scrub ok 2024-09-11T07:34:39.682 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:39 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:39.682 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:34:39 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:34:39.685 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:39.728 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:39.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:39.729 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:39.830 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:39.864 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:39.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:39.865 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:39.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:39 smithi022 ceph-mon[29462]: 6.3 scrub starts 2024-09-11T07:34:39.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:39 smithi022 ceph-mon[29462]: 6.3 scrub ok 2024-09-11T07:34:39.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:39 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:39.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:39 smithi022 ceph-mon[37330]: 6.3 scrub starts 2024-09-11T07:34:39.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:39 smithi022 ceph-mon[37330]: 6.3 scrub ok 2024-09-11T07:34:39.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:39 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:39.966 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:39.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:39.999 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:39.999 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:40.026 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:34:40.100 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:40.139 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:40.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:40.140 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:40.211 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:2024-09-11T07:34:40.210936+0000 min lat: 0.00108265 max lat: 0.159168 avg lat: 0.00994114 2024-09-11T07:34:40.212 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T07:34:40.212 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 60 16 96558 96542 100.543 64.25 0.0111992 0.00994114 2024-09-11T07:34:40.241 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:40.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:40.273 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:40.273 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:40.373 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:40.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:40.417 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:40.417 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:40.417 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:34:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:34:40] "GET /metrics HTTP/1.1" 200 224105 "" "Prometheus/2.43.0" 2024-09-11T07:34:40.518 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:40.548 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:40.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:40.549 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:40.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:40 smithi104 ceph-mon[33507]: 2.1e scrub starts 2024-09-11T07:34:40.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:40 smithi104 ceph-mon[33507]: 5.2 scrub starts 2024-09-11T07:34:40.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:40 smithi104 ceph-mon[33507]: 5.2 scrub ok 2024-09-11T07:34:40.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:40 smithi104 ceph-mon[33507]: 2.1e scrub ok 2024-09-11T07:34:40.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:40 smithi104 ceph-mon[33507]: 5.e scrub starts 2024-09-11T07:34:40.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:40 smithi104 ceph-mon[33507]: 5.e scrub ok 2024-09-11T07:34:40.606 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:40 smithi104 ceph-mon[33507]: pgmap v362: 148 pgs: 148 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-11T07:34:40.650 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:40.691 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:40.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:40.692 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:40.793 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:40.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:40.823 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:40.823 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:40.913 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:40 smithi022 ceph-mon[29462]: 2.1e scrub starts 2024-09-11T07:34:40.913 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:40 smithi022 ceph-mon[29462]: 5.2 scrub starts 2024-09-11T07:34:40.913 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:40 smithi022 ceph-mon[29462]: 5.2 scrub ok 2024-09-11T07:34:40.914 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:40 smithi022 ceph-mon[29462]: 2.1e scrub ok 2024-09-11T07:34:40.914 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:40 smithi022 ceph-mon[29462]: 5.e scrub starts 2024-09-11T07:34:40.914 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:40 smithi022 ceph-mon[29462]: 5.e scrub ok 2024-09-11T07:34:40.914 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:40 smithi022 ceph-mon[29462]: pgmap v362: 148 pgs: 148 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-11T07:34:40.914 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:40 smithi022 ceph-mon[37330]: 2.1e scrub starts 2024-09-11T07:34:40.914 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:40 smithi022 ceph-mon[37330]: 5.2 scrub starts 2024-09-11T07:34:40.914 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:40 smithi022 ceph-mon[37330]: 5.2 scrub ok 2024-09-11T07:34:40.914 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:40 smithi022 ceph-mon[37330]: 2.1e scrub ok 2024-09-11T07:34:40.914 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:40 smithi022 ceph-mon[37330]: 5.e scrub starts 2024-09-11T07:34:40.914 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:40 smithi022 ceph-mon[37330]: 5.e scrub ok 2024-09-11T07:34:40.915 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:40 smithi022 ceph-mon[37330]: pgmap v362: 148 pgs: 148 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-11T07:34:40.924 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:40.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:40.971 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:40.972 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:41.072 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:41.104 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:41.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:41.105 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:41.206 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:41.211 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 61 15 97665 97650 100.03 69.25 0.0094361 0.00999224 2024-09-11T07:34:41.243 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:41.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:41.244 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:41.345 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:41.383 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:41.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:41.384 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:41.485 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:41.511 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:41.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:41.512 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:41.612 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:41.638 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:41.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:41.639 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:41 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:34:41.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:41 smithi022 ceph-mon[29462]: 4.10 scrub starts 2024-09-11T07:34:41.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:41 smithi022 ceph-mon[29462]: 4.10 scrub ok 2024-09-11T07:34:41.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:41 smithi022 ceph-mon[29462]: 6.1c scrub starts 2024-09-11T07:34:41.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:41 smithi022 ceph-mon[29462]: 6.1c scrub ok 2024-09-11T07:34:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:41 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:34:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:41 smithi022 ceph-mon[37330]: 4.10 scrub starts 2024-09-11T07:34:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:41 smithi022 ceph-mon[37330]: 4.10 scrub ok 2024-09-11T07:34:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:41 smithi022 ceph-mon[37330]: 6.1c scrub starts 2024-09-11T07:34:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:41 smithi022 ceph-mon[37330]: 6.1c scrub ok 2024-09-11T07:34:41.740 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:41.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:41.766 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:41.766 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:41.866 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:41.907 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:41.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:41.908 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:42.008 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:42.011 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:41 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:34:42.011 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:41 smithi104 ceph-mon[33507]: 4.10 scrub starts 2024-09-11T07:34:42.011 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:41 smithi104 ceph-mon[33507]: 4.10 scrub ok 2024-09-11T07:34:42.011 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:41 smithi104 ceph-mon[33507]: 6.1c scrub starts 2024-09-11T07:34:42.011 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:41 smithi104 ceph-mon[33507]: 6.1c scrub ok 2024-09-11T07:34:42.042 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:42.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:42.043 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:42.144 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:42.183 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:42.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:42.185 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:42.212 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 62 16 98740 98724 99.4994 67.125 0.0150237 0.0100454 2024-09-11T07:34:42.285 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:42.317 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:42.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:42.318 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:42.419 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:42.456 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:42.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:42.457 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:42.462 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:34:42.462 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:34:42.558 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:42.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:42.605 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:42.605 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:42.706 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:42.744 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:42.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:42.745 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:42.846 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:42.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:42.887 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:42.887 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:42.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:42 smithi022 ceph-mon[29462]: 6.14 scrub starts 2024-09-11T07:34:42.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:42 smithi022 ceph-mon[29462]: 6.14 scrub ok 2024-09-11T07:34:42.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:42 smithi022 ceph-mon[29462]: 6.1a scrub starts 2024-09-11T07:34:42.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:42 smithi022 ceph-mon[29462]: 6.1a scrub ok 2024-09-11T07:34:42.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:42 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:34:42.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:42 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:34:42.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:42 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:34:42.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:42 smithi022 ceph-mon[29462]: pgmap v363: 148 pgs: 148 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.1 KiB/s rd, 81 MiB/s wr, 1.29k op/s 2024-09-11T07:34:42.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:42 smithi022 ceph-mon[37330]: 6.14 scrub starts 2024-09-11T07:34:42.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:42 smithi022 ceph-mon[37330]: 6.14 scrub ok 2024-09-11T07:34:42.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:42 smithi022 ceph-mon[37330]: 6.1a scrub starts 2024-09-11T07:34:42.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:42 smithi022 ceph-mon[37330]: 6.1a scrub ok 2024-09-11T07:34:42.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:42 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:34:42.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:42 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:34:42.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:42 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:34:42.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:42 smithi022 ceph-mon[37330]: pgmap v363: 148 pgs: 148 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.1 KiB/s rd, 81 MiB/s wr, 1.29k op/s 2024-09-11T07:34:42.966 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:34:42.988 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:43.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:42 smithi104 ceph-mon[33507]: 6.14 scrub starts 2024-09-11T07:34:43.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:42 smithi104 ceph-mon[33507]: 6.14 scrub ok 2024-09-11T07:34:43.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:42 smithi104 ceph-mon[33507]: 6.1a scrub starts 2024-09-11T07:34:43.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:42 smithi104 ceph-mon[33507]: 6.1a scrub ok 2024-09-11T07:34:43.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:42 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:34:43.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:42 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:34:43.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:42 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:34:43.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:42 smithi104 ceph-mon[33507]: pgmap v363: 148 pgs: 148 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.1 KiB/s rd, 81 MiB/s wr, 1.29k op/s 2024-09-11T07:34:43.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:43.074 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:43.075 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:43.175 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:43.212 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 63 16 99932 99916 99.1021 74.5 0.0114872 0.0100855 2024-09-11T07:34:43.215 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-11T07:34:43.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:43.217 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:43.217 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:43.318 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:43.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:43.341 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:43.341 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:43.442 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:43.489 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:43.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:43.492 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:43.593 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:43.639 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:43.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:43.640 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:43.741 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:43.765 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:43.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:43.766 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:43.867 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:43.909 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:43.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:43.910 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:44.010 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:44.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:43 smithi104 ceph-mon[33507]: 2.16 scrub starts 2024-09-11T07:34:44.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:43 smithi104 ceph-mon[33507]: 2.16 scrub ok 2024-09-11T07:34:44.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:43 smithi104 ceph-mon[33507]: 6.17 scrub starts 2024-09-11T07:34:44.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:43 smithi104 ceph-mon[33507]: 6.17 scrub ok 2024-09-11T07:34:44.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:43 smithi104 ceph-mon[33507]: 3.0 scrub starts 2024-09-11T07:34:44.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:43 smithi104 ceph-mon[33507]: 3.0 scrub ok 2024-09-11T07:34:44.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:43 smithi104 ceph-mon[33507]: 5.6 scrub starts 2024-09-11T07:34:44.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:43 smithi104 ceph-mon[33507]: 5.6 scrub ok 2024-09-11T07:34:44.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:43 smithi104 ceph-mon[33507]: 5.10 scrub starts 2024-09-11T07:34:44.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:43 smithi104 ceph-mon[33507]: 5.10 scrub ok 2024-09-11T07:34:44.047 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:44.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:44.048 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:44.149 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:44.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:43 smithi022 ceph-mon[29462]: 2.16 scrub starts 2024-09-11T07:34:44.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:43 smithi022 ceph-mon[29462]: 2.16 scrub ok 2024-09-11T07:34:44.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:43 smithi022 ceph-mon[29462]: 6.17 scrub starts 2024-09-11T07:34:44.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:43 smithi022 ceph-mon[29462]: 6.17 scrub ok 2024-09-11T07:34:44.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:43 smithi022 ceph-mon[29462]: 3.0 scrub starts 2024-09-11T07:34:44.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:43 smithi022 ceph-mon[29462]: 3.0 scrub ok 2024-09-11T07:34:44.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:43 smithi022 ceph-mon[29462]: 5.6 scrub starts 2024-09-11T07:34:44.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:43 smithi022 ceph-mon[29462]: 5.6 scrub ok 2024-09-11T07:34:44.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:43 smithi022 ceph-mon[29462]: 5.10 scrub starts 2024-09-11T07:34:44.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:43 smithi022 ceph-mon[29462]: 5.10 scrub ok 2024-09-11T07:34:44.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:43 smithi022 ceph-mon[37330]: 2.16 scrub starts 2024-09-11T07:34:44.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:43 smithi022 ceph-mon[37330]: 2.16 scrub ok 2024-09-11T07:34:44.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:43 smithi022 ceph-mon[37330]: 6.17 scrub starts 2024-09-11T07:34:44.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:43 smithi022 ceph-mon[37330]: 6.17 scrub ok 2024-09-11T07:34:44.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:43 smithi022 ceph-mon[37330]: 3.0 scrub starts 2024-09-11T07:34:44.171 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:43 smithi022 ceph-mon[37330]: 3.0 scrub ok 2024-09-11T07:34:44.171 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:43 smithi022 ceph-mon[37330]: 5.6 scrub starts 2024-09-11T07:34:44.171 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:43 smithi022 ceph-mon[37330]: 5.6 scrub ok 2024-09-11T07:34:44.171 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:43 smithi022 ceph-mon[37330]: 5.10 scrub starts 2024-09-11T07:34:44.171 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:43 smithi022 ceph-mon[37330]: 5.10 scrub ok 2024-09-11T07:34:44.185 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:44.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:44.187 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:44.212 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 64 16 101024 101008 98.62 68.25 0.00639488 0.0101351 2024-09-11T07:34:44.288 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:44.317 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:44.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:44.318 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:44.419 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:44.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:44.443 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:44.443 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:44.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:34:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:34:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:34:44.543 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:44.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:44.575 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:44.576 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:44.676 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:44.709 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:44.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:44.710 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:44.811 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:44.843 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:44.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:44.844 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:44.945 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:44.972 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:44.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:44.973 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:45.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:44 smithi104 ceph-mon[33507]: 2.d scrub starts 2024-09-11T07:34:45.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:44 smithi104 ceph-mon[33507]: 2.d scrub ok 2024-09-11T07:34:45.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:44 smithi104 ceph-mon[33507]: 3.2 scrub starts 2024-09-11T07:34:45.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:44 smithi104 ceph-mon[33507]: 3.2 scrub ok 2024-09-11T07:34:45.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:44 smithi104 ceph-mon[33507]: 6.0 scrub starts 2024-09-11T07:34:45.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:44 smithi104 ceph-mon[33507]: 6.0 scrub ok 2024-09-11T07:34:45.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:44 smithi104 ceph-mon[33507]: pgmap v364: 148 pgs: 148 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 71 MiB/s wr, 1.13k op/s 2024-09-11T07:34:45.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:44 smithi104 ceph-mon[33507]: 1.0 scrub starts 2024-09-11T07:34:45.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:44 smithi104 ceph-mon[33507]: 1.0 scrub ok 2024-09-11T07:34:45.074 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:45.101 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:44 smithi022 ceph-mon[29462]: 2.d scrub starts 2024-09-11T07:34:45.101 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:44 smithi022 ceph-mon[29462]: 2.d scrub ok 2024-09-11T07:34:45.101 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:44 smithi022 ceph-mon[29462]: 3.2 scrub starts 2024-09-11T07:34:45.101 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:44 smithi022 ceph-mon[29462]: 3.2 scrub ok 2024-09-11T07:34:45.101 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:44 smithi022 ceph-mon[29462]: 6.0 scrub starts 2024-09-11T07:34:45.101 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:44 smithi022 ceph-mon[29462]: 6.0 scrub ok 2024-09-11T07:34:45.101 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:44 smithi022 ceph-mon[29462]: pgmap v364: 148 pgs: 148 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 71 MiB/s wr, 1.13k op/s 2024-09-11T07:34:45.101 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:44 smithi022 ceph-mon[29462]: 1.0 scrub starts 2024-09-11T07:34:45.101 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:44 smithi022 ceph-mon[29462]: 1.0 scrub ok 2024-09-11T07:34:45.102 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:44 smithi022 ceph-mon[37330]: 2.d scrub starts 2024-09-11T07:34:45.102 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:44 smithi022 ceph-mon[37330]: 2.d scrub ok 2024-09-11T07:34:45.102 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:44 smithi022 ceph-mon[37330]: 3.2 scrub starts 2024-09-11T07:34:45.102 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:44 smithi022 ceph-mon[37330]: 3.2 scrub ok 2024-09-11T07:34:45.102 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:44 smithi022 ceph-mon[37330]: 6.0 scrub starts 2024-09-11T07:34:45.102 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:44 smithi022 ceph-mon[37330]: 6.0 scrub ok 2024-09-11T07:34:45.102 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:44 smithi022 ceph-mon[37330]: pgmap v364: 148 pgs: 148 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 71 MiB/s wr, 1.13k op/s 2024-09-11T07:34:45.102 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:44 smithi022 ceph-mon[37330]: 1.0 scrub starts 2024-09-11T07:34:45.102 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:44 smithi022 ceph-mon[37330]: 1.0 scrub ok 2024-09-11T07:34:45.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:45.118 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:45.118 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:45.212 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 65 16 102223 102207 98.2554 74.9375 0.00626887 0.0101728 2024-09-11T07:34:45.219 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:45.248 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:45.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:45.249 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:45.350 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:45.380 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:45.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:45.381 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:45.482 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:45.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:45.525 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:45.525 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:45.626 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:45.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:45.677 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:45.677 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:45.778 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:45.812 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:45 smithi022 ceph-mon[29462]: 2.7 scrub starts 2024-09-11T07:34:45.812 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:45 smithi022 ceph-mon[29462]: 2.7 scrub ok 2024-09-11T07:34:45.812 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:45 smithi022 ceph-mon[29462]: 6.e scrub starts 2024-09-11T07:34:45.812 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:45 smithi022 ceph-mon[29462]: 6.e scrub ok 2024-09-11T07:34:45.812 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:45 smithi022 ceph-mon[29462]: 4.6 scrub starts 2024-09-11T07:34:45.812 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:45 smithi022 ceph-mon[29462]: 4.6 scrub ok 2024-09-11T07:34:45.813 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:45 smithi022 ceph-mon[37330]: 2.7 scrub starts 2024-09-11T07:34:45.813 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:45 smithi022 ceph-mon[37330]: 2.7 scrub ok 2024-09-11T07:34:45.813 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:45 smithi022 ceph-mon[37330]: 6.e scrub starts 2024-09-11T07:34:45.813 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:45 smithi022 ceph-mon[37330]: 6.e scrub ok 2024-09-11T07:34:45.813 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:45 smithi022 ceph-mon[37330]: 4.6 scrub starts 2024-09-11T07:34:45.813 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:45 smithi022 ceph-mon[37330]: 4.6 scrub ok 2024-09-11T07:34:45.818 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:45.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:45.820 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:45.920 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:45.951 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:45.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:45.952 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:46.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:45 smithi104 ceph-mon[33507]: 2.7 scrub starts 2024-09-11T07:34:46.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:45 smithi104 ceph-mon[33507]: 2.7 scrub ok 2024-09-11T07:34:46.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:45 smithi104 ceph-mon[33507]: 6.e scrub starts 2024-09-11T07:34:46.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:45 smithi104 ceph-mon[33507]: 6.e scrub ok 2024-09-11T07:34:46.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:45 smithi104 ceph-mon[33507]: 4.6 scrub starts 2024-09-11T07:34:46.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:45 smithi104 ceph-mon[33507]: 4.6 scrub ok 2024-09-11T07:34:46.053 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:46.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:46.101 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:46.101 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:46.202 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:46.212 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 66 16 103336 103320 97.8207 69.5625 0.0218469 0.0102178 2024-09-11T07:34:46.228 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:46.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:46.229 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:46.329 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:46.357 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:46.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:46.359 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:46.459 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:46.517 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:34:46.526 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:34:46.541 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:46.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:46.543 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:46.620 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:34:46.623 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:34:46.644 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:46.692 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:46.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:46.693 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:46.723 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is unset 2024-09-11T07:34:46.794 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:46.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:46.844 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:46.844 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:46.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:46 smithi022 ceph-mon[29462]: 2.11 scrub starts 2024-09-11T07:34:46.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:46 smithi022 ceph-mon[29462]: 2.11 scrub ok 2024-09-11T07:34:46.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:46 smithi022 ceph-mon[29462]: 5.15 scrub starts 2024-09-11T07:34:46.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:46 smithi022 ceph-mon[29462]: 5.15 scrub ok 2024-09-11T07:34:46.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:46 smithi022 ceph-mon[29462]: pgmap v365: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 968 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-11T07:34:46.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:46 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1700454585' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:34:46.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:46 smithi022 ceph-mon[37330]: 2.11 scrub starts 2024-09-11T07:34:46.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:46 smithi022 ceph-mon[37330]: 2.11 scrub ok 2024-09-11T07:34:46.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:46 smithi022 ceph-mon[37330]: 5.15 scrub starts 2024-09-11T07:34:46.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:46 smithi022 ceph-mon[37330]: 5.15 scrub ok 2024-09-11T07:34:46.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:46 smithi022 ceph-mon[37330]: pgmap v365: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 968 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-11T07:34:46.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:46 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1700454585' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:34:46.946 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:46.980 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:46.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:46.981 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:47.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:46 smithi104 ceph-mon[33507]: 2.11 scrub starts 2024-09-11T07:34:47.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:46 smithi104 ceph-mon[33507]: 2.11 scrub ok 2024-09-11T07:34:47.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:46 smithi104 ceph-mon[33507]: 5.15 scrub starts 2024-09-11T07:34:47.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:46 smithi104 ceph-mon[33507]: 5.15 scrub ok 2024-09-11T07:34:47.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:46 smithi104 ceph-mon[33507]: pgmap v365: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 968 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-11T07:34:47.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:46 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1700454585' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:34:47.082 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:47.107 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:47.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:47.108 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:47.209 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:47.212 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 67 16 104368 104352 97.3233 64.5 0.00775536 0.0102704 2024-09-11T07:34:47.249 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:47.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:47.250 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:47.351 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:47.378 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:47.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:47.379 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:47.480 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:47.516 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:47.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:47.517 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:47.618 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:47.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:47.661 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:47.661 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:47.762 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:47.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:47.802 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:47.802 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:47.903 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:47.930 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:47.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:47.931 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:48.011 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:47 smithi022 ceph-mon[29462]: 2.17 scrub starts 2024-09-11T07:34:48.012 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:47 smithi022 ceph-mon[29462]: 2.17 scrub ok 2024-09-11T07:34:48.012 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:47 smithi022 ceph-mon[29462]: from='client.25174 v1:172.21.15.22:0/3865119750' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:34:48.012 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:47 smithi022 ceph-mon[29462]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:34:48.012 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:47 smithi022 ceph-mon[29462]: Cluster is now healthy 2024-09-11T07:34:48.012 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:47 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1700454585' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:34:48.012 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:47 smithi022 ceph-mon[29462]: osdmap e206: 8 total, 8 up, 8 in 2024-09-11T07:34:48.012 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:47 smithi022 ceph-mon[29462]: 6.c deep-scrub starts 2024-09-11T07:34:48.012 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:47 smithi022 ceph-mon[29462]: 6.c deep-scrub ok 2024-09-11T07:34:48.012 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:47 smithi022 ceph-mon[29462]: 6.9 deep-scrub starts 2024-09-11T07:34:48.012 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:47 smithi022 ceph-mon[29462]: 6.9 deep-scrub ok 2024-09-11T07:34:48.012 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:47 smithi022 ceph-mon[29462]: 5.1a scrub starts 2024-09-11T07:34:48.013 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:47 smithi022 ceph-mon[29462]: 5.1a scrub ok 2024-09-11T07:34:48.013 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:47 smithi022 ceph-mon[37330]: 2.17 scrub starts 2024-09-11T07:34:48.013 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:47 smithi022 ceph-mon[37330]: 2.17 scrub ok 2024-09-11T07:34:48.013 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:47 smithi022 ceph-mon[37330]: from='client.25174 v1:172.21.15.22:0/3865119750' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:34:48.013 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:47 smithi022 ceph-mon[37330]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:34:48.013 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:47 smithi022 ceph-mon[37330]: Cluster is now healthy 2024-09-11T07:34:48.013 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:47 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1700454585' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:34:48.013 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:47 smithi022 ceph-mon[37330]: osdmap e206: 8 total, 8 up, 8 in 2024-09-11T07:34:48.013 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:47 smithi022 ceph-mon[37330]: 6.c deep-scrub starts 2024-09-11T07:34:48.013 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:47 smithi022 ceph-mon[37330]: 6.c deep-scrub ok 2024-09-11T07:34:48.013 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:47 smithi022 ceph-mon[37330]: 6.9 deep-scrub starts 2024-09-11T07:34:48.014 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:47 smithi022 ceph-mon[37330]: 6.9 deep-scrub ok 2024-09-11T07:34:48.014 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:47 smithi022 ceph-mon[37330]: 5.1a scrub starts 2024-09-11T07:34:48.014 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:47 smithi022 ceph-mon[37330]: 5.1a scrub ok 2024-09-11T07:34:48.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:47 smithi104 ceph-mon[33507]: 2.17 scrub starts 2024-09-11T07:34:48.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:47 smithi104 ceph-mon[33507]: 2.17 scrub ok 2024-09-11T07:34:48.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:47 smithi104 ceph-mon[33507]: from='client.25174 v1:172.21.15.22:0/3865119750' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:34:48.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:47 smithi104 ceph-mon[33507]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:34:48.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:47 smithi104 ceph-mon[33507]: Cluster is now healthy 2024-09-11T07:34:48.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:47 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1700454585' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:34:48.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:47 smithi104 ceph-mon[33507]: osdmap e206: 8 total, 8 up, 8 in 2024-09-11T07:34:48.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:47 smithi104 ceph-mon[33507]: 6.c deep-scrub starts 2024-09-11T07:34:48.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:47 smithi104 ceph-mon[33507]: 6.c deep-scrub ok 2024-09-11T07:34:48.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:47 smithi104 ceph-mon[33507]: 6.9 deep-scrub starts 2024-09-11T07:34:48.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:47 smithi104 ceph-mon[33507]: 6.9 deep-scrub ok 2024-09-11T07:34:48.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:47 smithi104 ceph-mon[33507]: 5.1a scrub starts 2024-09-11T07:34:48.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:47 smithi104 ceph-mon[33507]: 5.1a scrub ok 2024-09-11T07:34:48.032 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:48.067 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:48.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:48.068 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:48.169 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:48.210 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:48.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:48.211 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:48.212 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 68 16 105634 105618 97.0556 79.125 0.0171008 0.0102984 2024-09-11T07:34:48.312 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:48.351 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:48.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:48.352 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:48.453 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:48.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:48.491 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:48.491 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:48.592 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:48.619 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:48.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:48.620 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:48.721 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:48.759 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:48.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:48.760 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:48.860 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:48.885 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:48.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:48.886 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:48.897 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-09-11T07:34:46.110094+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6628776156,"num_objects":101334,"num_object_clones":0,"num_object_copies":304002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101334,"num_whiteouts":0,"num_read":1338,"num_read_kb":1195,"num_write":101681,"num_write_kb":6474503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":591166,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86081,"ondisk_log_size":86081,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":21705760,"kb_used_data":19262368,"kb_used_omap":16,"kb_used_meta":2443311,"kb_avail":728255456,"statfs":{"total":767960285184,"available":745733586944,"internally_reserved":0,"allocated":19724664832,"data_stored":19720133257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16530,"internal_metadata":2501951342},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":723255296,"num_objects":11036,"num_object_clones":0,"num_object_copies":33108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11036,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":11036,"num_write_kb":706304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.567899"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":188,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.258983+0000","last_change":"2024-09-11T07:33:08.464899+0000","last_active":"2024-09-11T07:34:33.258983+0000","last_peered":"2024-09-11T07:34:33.258983+0000","last_clean":"2024-09-11T07:34:33.258983+0000","last_became_active":"2024-09-11T07:32:00.501534+0000","last_became_peered":"2024-09-11T07:32:00.501534+0000","last_unstale":"2024-09-11T07:34:33.258983+0000","last_undegraded":"2024-09-11T07:34:33.258983+0000","last_fullsized":"2024-09-11T07:34:33.258983+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:08.464835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:08.464835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002103452,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":213,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.254231+0000","last_change":"2024-09-11T07:34:09.538289+0000","last_active":"2024-09-11T07:34:33.254231+0000","last_peered":"2024-09-11T07:34:33.254231+0000","last_clean":"2024-09-11T07:34:33.254231+0000","last_became_active":"2024-09-11T07:32:00.508572+0000","last_became_peered":"2024-09-11T07:32:00.508572+0000","last_unstale":"2024-09-11T07:34:33.254231+0000","last_undegraded":"2024-09-11T07:34:33.254231+0000","last_fullsized":"2024-09-11T07:34:33.254231+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:09.538205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:09.538205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:23.803280+0000","scrub_duration":0.0045320439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"82'22","reported_seq":217,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.259624+0000","last_change":"2024-09-11T07:33:53.615330+0000","last_active":"2024-09-11T07:34:33.259624+0000","last_peered":"2024-09-11T07:34:33.259624+0000","last_clean":"2024-09-11T07:34:33.259624+0000","last_became_active":"2024-09-11T07:26:33.544627+0000","last_became_peered":"2024-09-11T07:26:33.544627+0000","last_unstale":"2024-09-11T07:34:33.259624+0000","last_undegraded":"2024-09-11T07:34:33.259624+0000","last_fullsized":"2024-09-11T07:34:33.259624+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-09-11T07:33:53.615231+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:33:53.615231+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:22.109465+0000","scrub_duration":0.011220367,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":192,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.254495+0000","last_change":"2024-09-11T07:33:18.247208+0000","last_active":"2024-09-11T07:34:33.254495+0000","last_peered":"2024-09-11T07:34:33.254495+0000","last_clean":"2024-09-11T07:34:33.254495+0000","last_became_active":"2024-09-11T07:26:35.556296+0000","last_became_peered":"2024-09-11T07:26:35.556296+0000","last_unstale":"2024-09-11T07:34:33.254495+0000","last_undegraded":"2024-09-11T07:34:33.254495+0000","last_fullsized":"2024-09-11T07:34:33.254495+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:18.247138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:18.247138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:43.507240+0000","scrub_duration":0.001401198,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":197,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:38.766322+0000","last_change":"2024-09-11T07:34:38.766278+0000","last_active":"2024-09-11T07:34:38.766322+0000","last_peered":"2024-09-11T07:34:38.766322+0000","last_clean":"2024-09-11T07:34:38.766322+0000","last_became_active":"2024-09-11T07:26:31.420178+0000","last_became_peered":"2024-09-11T07:26:31.420178+0000","last_unstale":"2024-09-11T07:34:38.766322+0000","last_undegraded":"2024-09-11T07:34:38.766322+0000","last_fullsized":"2024-09-11T07:34:38.766322+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:38.766188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:38.766188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:46.651091+0000","scrub_duration":0.022341764,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"86'13","reported_seq":214,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:34.251770+0000","last_change":"2024-09-11T07:34:34.251712+0000","last_active":"2024-09-11T07:34:34.251770+0000","last_peered":"2024-09-11T07:34:34.251770+0000","last_clean":"2024-09-11T07:34:34.251770+0000","last_became_active":"2024-09-11T07:26:33.540732+0000","last_became_peered":"2024-09-11T07:26:33.540732+0000","last_unstale":"2024-09-11T07:34:34.251770+0000","last_undegraded":"2024-09-11T07:34:34.251770+0000","last_fullsized":"2024-09-11T07:34:34.251770+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'13","last_scrub_stamp":"2024-09-11T07:34:34.251622+0000","last_deep_scrub":"86'13","last_deep_scrub_stamp":"2024-09-11T07:28:15.025757+0000","last_clean_scrub_stamp":"2024-09-11T07:34:34.251622+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:34.755765+0000","scrub_duration":0.0040986499999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":201,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.252836+0000","last_change":"2024-09-11T07:33:49.237639+0000","last_active":"2024-09-11T07:34:33.252836+0000","last_peered":"2024-09-11T07:34:33.252836+0000","last_clean":"2024-09-11T07:34:33.252836+0000","last_became_active":"2024-09-11T07:32:00.513650+0000","last_became_peered":"2024-09-11T07:32:00.513650+0000","last_unstale":"2024-09-11T07:34:33.252836+0000","last_undegraded":"2024-09-11T07:34:33.252836+0000","last_fullsized":"2024-09-11T07:34:33.252836+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:49.237335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:33:49.237335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:54.939660+0000","scrub_duration":0.0052103979999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":190,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:41.315756+0000","last_change":"2024-09-11T07:34:41.315649+0000","last_active":"2024-09-11T07:34:41.315756+0000","last_peered":"2024-09-11T07:34:41.315756+0000","last_clean":"2024-09-11T07:34:41.315756+0000","last_became_active":"2024-09-11T07:26:37.566910+0000","last_became_peered":"2024-09-11T07:26:37.566910+0000","last_unstale":"2024-09-11T07:34:41.315756+0000","last_undegraded":"2024-09-11T07:34:41.315756+0000","last_fullsized":"2024-09-11T07:34:41.315756+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:41.315562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:41.315562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:44.355231+0000","scrub_duration":0.002327371,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":192,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.264285+0000","last_change":"2024-09-11T07:33:57.319606+0000","last_active":"2024-09-11T07:34:33.264285+0000","last_peered":"2024-09-11T07:34:33.264285+0000","last_clean":"2024-09-11T07:34:33.264285+0000","last_became_active":"2024-09-11T07:32:00.516662+0000","last_became_peered":"2024-09-11T07:32:00.516662+0000","last_unstale":"2024-09-11T07:34:33.264285+0000","last_undegraded":"2024-09-11T07:34:33.264285+0000","last_fullsized":"2024-09-11T07:34:33.264285+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:57.319449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:57.319449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:11.083627+0000","scrub_duration":0.007631344,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"82'7","reported_seq":206,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.255804+0000","last_change":"2024-09-11T07:31:43.035948+0000","last_active":"2024-09-11T07:34:33.255804+0000","last_peered":"2024-09-11T07:34:33.255804+0000","last_clean":"2024-09-11T07:34:33.255804+0000","last_became_active":"2024-09-11T07:26:33.540351+0000","last_became_peered":"2024-09-11T07:26:33.540351+0000","last_unstale":"2024-09-11T07:34:33.255804+0000","last_undegraded":"2024-09-11T07:34:33.255804+0000","last_fullsized":"2024-09-11T07:34:33.255804+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'7","last_scrub_stamp":"2024-09-11T07:31:43.035896+0000","last_deep_scrub":"82'7","last_deep_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_clean_scrub_stamp":"2024-09-11T07:31:43.035896+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040919600000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":158,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.259950+0000","last_change":"2024-09-11T07:33:07.124141+0000","last_active":"2024-09-11T07:34:33.259950+0000","last_peered":"2024-09-11T07:34:33.259950+0000","last_clean":"2024-09-11T07:34:33.259950+0000","last_became_active":"2024-09-11T07:32:00.513206+0000","last_became_peered":"2024-09-11T07:32:00.513206+0000","last_unstale":"2024-09-11T07:34:33.259950+0000","last_undegraded":"2024-09-11T07:34:33.259950+0000","last_fullsized":"2024-09-11T07:34:33.259950+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:07.124068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.973323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:07.124068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019801810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":181,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.258097+0000","last_change":"2024-09-11T07:34:03.257034+0000","last_active":"2024-09-11T07:34:33.258097+0000","last_peered":"2024-09-11T07:34:33.258097+0000","last_clean":"2024-09-11T07:34:33.258097+0000","last_became_active":"2024-09-11T07:26:37.561758+0000","last_became_peered":"2024-09-11T07:26:37.561758+0000","last_unstale":"2024-09-11T07:34:33.258097+0000","last_undegraded":"2024-09-11T07:34:33.258097+0000","last_fullsized":"2024-09-11T07:34:33.258097+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:03.256953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:03.256953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:07.281155+0000","scrub_duration":0.0085144460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":157,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.257748+0000","last_change":"2024-09-11T07:34:01.257321+0000","last_active":"2024-09-11T07:34:33.257748+0000","last_peered":"2024-09-11T07:34:33.257748+0000","last_clean":"2024-09-11T07:34:33.257748+0000","last_became_active":"2024-09-11T07:32:00.505709+0000","last_became_peered":"2024-09-11T07:32:00.505709+0000","last_unstale":"2024-09-11T07:34:33.257748+0000","last_undegraded":"2024-09-11T07:34:33.257748+0000","last_fullsized":"2024-09-11T07:34:33.257748+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:01.257176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_clean_scrub_stamp":"2024-09-11T07:34:01.257176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:03.761234+0000","scrub_duration":0.0018917890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"136'13","reported_seq":214,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.255222+0000","last_change":"2024-09-11T07:34:06.406309+0000","last_active":"2024-09-11T07:34:33.255222+0000","last_peered":"2024-09-11T07:34:33.255222+0000","last_clean":"2024-09-11T07:34:33.255222+0000","last_became_active":"2024-09-11T07:26:33.544386+0000","last_became_peered":"2024-09-11T07:26:33.544386+0000","last_unstale":"2024-09-11T07:34:33.255222+0000","last_undegraded":"2024-09-11T07:34:33.255222+0000","last_fullsized":"2024-09-11T07:34:33.255222+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'13","last_scrub_stamp":"2024-09-11T07:34:06.406196+0000","last_deep_scrub":"136'13","last_deep_scrub_stamp":"2024-09-11T07:31:08.042198+0000","last_clean_scrub_stamp":"2024-09-11T07:34:06.406196+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:08.075604+0000","scrub_duration":0.0079557350000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":203,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.257266+0000","last_change":"2024-09-11T07:33:49.134068+0000","last_active":"2024-09-11T07:34:33.257266+0000","last_peered":"2024-09-11T07:34:33.257266+0000","last_clean":"2024-09-11T07:34:33.257266+0000","last_became_active":"2024-09-11T07:32:00.502337+0000","last_became_peered":"2024-09-11T07:32:00.502337+0000","last_unstale":"2024-09-11T07:34:33.257266+0000","last_undegraded":"2024-09-11T07:34:33.257266+0000","last_fullsized":"2024-09-11T07:34:33.257266+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:49.133995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:49.133995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:52.916415+0000","scrub_duration":0.0028580369999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":207,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.740785+0000","last_change":"2024-09-11T07:34:33.740699+0000","last_active":"2024-09-11T07:34:33.740785+0000","last_peered":"2024-09-11T07:34:33.740785+0000","last_clean":"2024-09-11T07:34:33.740785+0000","last_became_active":"2024-09-11T07:32:00.530335+0000","last_became_peered":"2024-09-11T07:32:00.530335+0000","last_unstale":"2024-09-11T07:34:33.740785+0000","last_undegraded":"2024-09-11T07:34:33.740785+0000","last_fullsized":"2024-09-11T07:34:33.740785+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:33.740593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:33.740593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:36.438261+0000","scrub_duration":0.014153393,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"205'6497","reported_seq":6532,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:44.214837+0000","last_change":"2024-09-11T07:33:29.155481+0000","last_active":"2024-09-11T07:34:44.214837+0000","last_peered":"2024-09-11T07:34:44.214837+0000","last_clean":"2024-09-11T07:34:44.214837+0000","last_became_active":"2024-09-11T07:33:29.155187+0000","last_became_peered":"2024-09-11T07:33:29.155187+0000","last_unstale":"2024-09-11T07:34:44.214837+0000","last_undegraded":"2024-09-11T07:34:44.214837+0000","last_fullsized":"2024-09-11T07:34:44.214837+0000","mapping_epoch":177,"log_start":"188'1400","ondisk_log_start":"188'1400","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":5097,"log_dups_size":0,"ondisk_log_size":5097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:56.196806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":425787392,"num_objects":6497,"num_object_clones":0,"num_object_copies":19491,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6497,"num_write_kb":415808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":278,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.259510+0000","last_change":"2024-09-11T07:34:11.315441+0000","last_active":"2024-09-11T07:34:33.259510+0000","last_peered":"2024-09-11T07:34:33.259510+0000","last_clean":"2024-09-11T07:34:33.259510+0000","last_became_active":"2024-09-11T07:29:51.482755+0000","last_became_peered":"2024-09-11T07:29:51.482755+0000","last_unstale":"2024-09-11T07:34:33.259510+0000","last_undegraded":"2024-09-11T07:34:33.259510+0000","last_fullsized":"2024-09-11T07:34:33.259510+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-09-11T07:34:11.315334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:34:11.315334+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:17.690755+0000","scrub_duration":0.0071604249999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":187,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.260926+0000","last_change":"2024-09-11T07:33:52.272651+0000","last_active":"2024-09-11T07:34:33.260926+0000","last_peered":"2024-09-11T07:34:33.260926+0000","last_clean":"2024-09-11T07:34:33.260926+0000","last_became_active":"2024-09-11T07:32:00.507191+0000","last_became_peered":"2024-09-11T07:32:00.507191+0000","last_unstale":"2024-09-11T07:34:33.260926+0000","last_undegraded":"2024-09-11T07:34:33.260926+0000","last_fullsized":"2024-09-11T07:34:33.260926+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:52.272573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:52.272573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:17.508902+0000","scrub_duration":0.0023059880000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":191,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.252808+0000","last_change":"2024-09-11T07:34:00.348790+0000","last_active":"2024-09-11T07:34:33.252808+0000","last_peered":"2024-09-11T07:34:33.252808+0000","last_clean":"2024-09-11T07:34:33.252808+0000","last_became_active":"2024-09-11T07:26:35.557989+0000","last_became_peered":"2024-09-11T07:26:35.557989+0000","last_unstale":"2024-09-11T07:34:33.252808+0000","last_undegraded":"2024-09-11T07:34:33.252808+0000","last_fullsized":"2024-09-11T07:34:33.252808+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:00.348684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:00.348684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:30.171094+0000","scrub_duration":0.0036577580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":191,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.261390+0000","last_change":"2024-09-11T07:33:57.219099+0000","last_active":"2024-09-11T07:34:33.261390+0000","last_peered":"2024-09-11T07:34:33.261390+0000","last_clean":"2024-09-11T07:34:33.261390+0000","last_became_active":"2024-09-11T07:26:37.564554+0000","last_became_peered":"2024-09-11T07:26:37.564554+0000","last_unstale":"2024-09-11T07:34:33.261390+0000","last_undegraded":"2024-09-11T07:34:33.261390+0000","last_fullsized":"2024-09-11T07:34:33.261390+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:57.219012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:57.219012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:26.433401+0000","scrub_duration":0.0062201569999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"205'6307","reported_seq":6342,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:44.605853+0000","last_change":"2024-09-11T07:33:29.161604+0000","last_active":"2024-09-11T07:34:44.605853+0000","last_peered":"2024-09-11T07:34:44.605853+0000","last_clean":"2024-09-11T07:34:44.605853+0000","last_became_active":"2024-09-11T07:33:29.161349+0000","last_became_peered":"2024-09-11T07:33:29.161349+0000","last_unstale":"2024-09-11T07:34:44.605853+0000","last_undegraded":"2024-09-11T07:34:44.605853+0000","last_fullsized":"2024-09-11T07:34:44.605853+0000","mapping_epoch":177,"log_start":"188'1300","ondisk_log_start":"188'1300","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":5007,"log_dups_size":0,"ondisk_log_size":5007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413335552,"num_objects":6307,"num_object_clones":0,"num_object_copies":18921,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6307,"num_write_kb":403648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"112'15","reported_seq":214,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.256076+0000","last_change":"2024-09-11T07:34:08.373729+0000","last_active":"2024-09-11T07:34:33.256076+0000","last_peered":"2024-09-11T07:34:33.256076+0000","last_clean":"2024-09-11T07:34:33.256076+0000","last_became_active":"2024-09-11T07:26:33.535411+0000","last_became_peered":"2024-09-11T07:26:33.535411+0000","last_unstale":"2024-09-11T07:34:33.256076+0000","last_undegraded":"2024-09-11T07:34:33.256076+0000","last_fullsized":"2024-09-11T07:34:33.256076+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"112'15","last_scrub_stamp":"2024-09-11T07:34:08.373652+0000","last_deep_scrub":"112'15","last_deep_scrub_stamp":"2024-09-11T07:31:10.005896+0000","last_clean_scrub_stamp":"2024-09-11T07:34:08.373652+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:35.031237+0000","scrub_duration":0.0048197400000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":205,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.253792+0000","last_change":"2024-09-11T07:33:50.250361+0000","last_active":"2024-09-11T07:34:33.253792+0000","last_peered":"2024-09-11T07:34:33.253792+0000","last_clean":"2024-09-11T07:34:33.253792+0000","last_became_active":"2024-09-11T07:32:00.506615+0000","last_became_peered":"2024-09-11T07:32:00.506615+0000","last_unstale":"2024-09-11T07:34:33.253792+0000","last_undegraded":"2024-09-11T07:34:33.253792+0000","last_fullsized":"2024-09-11T07:34:33.253792+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:50.250289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:50.250289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:01.601495+0000","scrub_duration":0.0080459799999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"151'10","reported_seq":318,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:35.557122+0000","last_change":"2024-09-11T07:34:33.960973+0000","last_active":"2024-09-11T07:34:35.557122+0000","last_peered":"2024-09-11T07:34:35.557122+0000","last_clean":"2024-09-11T07:34:35.557122+0000","last_became_active":"2024-09-11T07:32:00.507447+0000","last_became_peered":"2024-09-11T07:32:00.507447+0000","last_unstale":"2024-09-11T07:34:35.557122+0000","last_undegraded":"2024-09-11T07:34:35.557122+0000","last_fullsized":"2024-09-11T07:34:35.557122+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:34:33.960877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:33.960877+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:44.216730+0000","scrub_duration":0.0063436389999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":180,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.254791+0000","last_change":"2024-09-11T07:31:44.075858+0000","last_active":"2024-09-11T07:34:33.254791+0000","last_peered":"2024-09-11T07:34:33.254791+0000","last_clean":"2024-09-11T07:34:33.254791+0000","last_became_active":"2024-09-11T07:26:37.566919+0000","last_became_peered":"2024-09-11T07:26:37.566919+0000","last_unstale":"2024-09-11T07:34:33.254791+0000","last_undegraded":"2024-09-11T07:34:33.254791+0000","last_fullsized":"2024-09-11T07:34:33.254791+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:31:44.075772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:31:44.075772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053001849999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"205'6349","reported_seq":6383,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:44.239792+0000","last_change":"2024-09-11T07:33:29.154369+0000","last_active":"2024-09-11T07:34:44.239792+0000","last_peered":"2024-09-11T07:34:44.239792+0000","last_clean":"2024-09-11T07:34:44.239792+0000","last_became_active":"2024-09-11T07:33:29.154103+0000","last_became_peered":"2024-09-11T07:33:29.154103+0000","last_unstale":"2024-09-11T07:34:44.239792+0000","last_undegraded":"2024-09-11T07:34:44.239792+0000","last_fullsized":"2024-09-11T07:34:44.239792+0000","mapping_epoch":177,"log_start":"188'1300","ondisk_log_start":"188'1300","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":5049,"log_dups_size":0,"ondisk_log_size":5049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416088064,"num_objects":6349,"num_object_clones":0,"num_object_copies":19047,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6349,"num_write_kb":406336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"83'22","reported_seq":225,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.255457+0000","last_change":"2024-09-11T07:34:04.375654+0000","last_active":"2024-09-11T07:34:33.255457+0000","last_peered":"2024-09-11T07:34:33.255457+0000","last_clean":"2024-09-11T07:34:33.255457+0000","last_became_active":"2024-09-11T07:26:33.549733+0000","last_became_peered":"2024-09-11T07:26:33.549733+0000","last_unstale":"2024-09-11T07:34:33.255457+0000","last_undegraded":"2024-09-11T07:34:33.255457+0000","last_fullsized":"2024-09-11T07:34:33.255457+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'22","last_scrub_stamp":"2024-09-11T07:34:04.375457+0000","last_deep_scrub":"83'22","last_deep_scrub_stamp":"2024-09-11T07:34:04.375457+0000","last_clean_scrub_stamp":"2024-09-11T07:34:04.375457+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:29.572580+0000","scrub_duration":0.0086024310000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":215,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.253060+0000","last_change":"2024-09-11T07:33:59.343890+0000","last_active":"2024-09-11T07:34:33.253060+0000","last_peered":"2024-09-11T07:34:33.253060+0000","last_clean":"2024-09-11T07:34:33.253060+0000","last_became_active":"2024-09-11T07:32:00.513472+0000","last_became_peered":"2024-09-11T07:32:00.513472+0000","last_unstale":"2024-09-11T07:34:33.253060+0000","last_undegraded":"2024-09-11T07:34:33.253060+0000","last_fullsized":"2024-09-11T07:34:33.253060+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:59.343676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:59.343676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:04.415703+0000","scrub_duration":0.0067031579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":301,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:39.263233+0000","last_change":"2024-09-11T07:34:39.263174+0000","last_active":"2024-09-11T07:34:39.263233+0000","last_peered":"2024-09-11T07:34:39.263233+0000","last_clean":"2024-09-11T07:34:39.263233+0000","last_became_active":"2024-09-11T07:26:35.560406+0000","last_became_peered":"2024-09-11T07:26:35.560406+0000","last_unstale":"2024-09-11T07:34:39.263233+0000","last_undegraded":"2024-09-11T07:34:39.263233+0000","last_fullsized":"2024-09-11T07:34:39.263233+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:34:39.263118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:39.263118+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:08.406869+0000","scrub_duration":0.002665307,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":181,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.266265+0000","last_change":"2024-09-11T07:33:07.153745+0000","last_active":"2024-09-11T07:34:33.266265+0000","last_peered":"2024-09-11T07:34:33.266265+0000","last_clean":"2024-09-11T07:34:33.266265+0000","last_became_active":"2024-09-11T07:26:37.568005+0000","last_became_peered":"2024-09-11T07:26:37.568005+0000","last_unstale":"2024-09-11T07:34:33.266265+0000","last_undegraded":"2024-09-11T07:34:33.266265+0000","last_fullsized":"2024-09-11T07:34:33.266265+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:07.153674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:31.200685+0000","last_clean_scrub_stamp":"2024-09-11T07:33:07.153674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001744982,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"205'6279","reported_seq":6314,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:44.001790+0000","last_change":"2024-09-11T07:33:29.154414+0000","last_active":"2024-09-11T07:34:44.001790+0000","last_peered":"2024-09-11T07:34:44.001790+0000","last_clean":"2024-09-11T07:34:44.001790+0000","last_became_active":"2024-09-11T07:33:29.154117+0000","last_became_peered":"2024-09-11T07:33:29.154117+0000","last_unstale":"2024-09-11T07:34:44.001790+0000","last_undegraded":"2024-09-11T07:34:44.001790+0000","last_fullsized":"2024-09-11T07:34:44.001790+0000","mapping_epoch":177,"log_start":"187'1100","ondisk_log_start":"187'1100","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":5179,"log_dups_size":0,"ondisk_log_size":5179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":411500544,"num_objects":6279,"num_object_clones":0,"num_object_copies":18837,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6279,"num_write_kb":401856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"107'16","reported_seq":218,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.255473+0000","last_change":"2024-09-11T07:33:15.278908+0000","last_active":"2024-09-11T07:34:33.255473+0000","last_peered":"2024-09-11T07:34:33.255473+0000","last_clean":"2024-09-11T07:34:33.255473+0000","last_became_active":"2024-09-11T07:26:33.540777+0000","last_became_peered":"2024-09-11T07:26:33.540777+0000","last_unstale":"2024-09-11T07:34:33.255473+0000","last_undegraded":"2024-09-11T07:34:33.255473+0000","last_fullsized":"2024-09-11T07:34:33.255473+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'16","last_scrub_stamp":"2024-09-11T07:33:15.278856+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-09-11T07:29:38.112111+0000","last_clean_scrub_stamp":"2024-09-11T07:33:15.278856+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016764620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":181,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.260179+0000","last_change":"2024-09-11T07:33:17.262636+0000","last_active":"2024-09-11T07:34:33.260179+0000","last_peered":"2024-09-11T07:34:33.260179+0000","last_clean":"2024-09-11T07:34:33.260179+0000","last_became_active":"2024-09-11T07:32:00.518538+0000","last_became_peered":"2024-09-11T07:32:00.518538+0000","last_unstale":"2024-09-11T07:34:33.260179+0000","last_undegraded":"2024-09-11T07:34:33.260179+0000","last_fullsized":"2024-09-11T07:34:33.260179+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:17.262578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:17.262578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003026361,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":193,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.258260+0000","last_change":"2024-09-11T07:33:10.096952+0000","last_active":"2024-09-11T07:34:33.258260+0000","last_peered":"2024-09-11T07:34:33.258260+0000","last_clean":"2024-09-11T07:34:33.258260+0000","last_became_active":"2024-09-11T07:26:35.569081+0000","last_became_peered":"2024-09-11T07:26:35.569081+0000","last_unstale":"2024-09-11T07:34:33.258260+0000","last_undegraded":"2024-09-11T07:34:33.258260+0000","last_fullsized":"2024-09-11T07:34:33.258260+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:10.096886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:10.096886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032884910000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":181,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.259955+0000","last_change":"2024-09-11T07:31:45.261381+0000","last_active":"2024-09-11T07:34:33.259955+0000","last_peered":"2024-09-11T07:34:33.259955+0000","last_clean":"2024-09-11T07:34:33.259955+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:34:33.259955+0000","last_undegraded":"2024-09-11T07:34:33.259955+0000","last_fullsized":"2024-09-11T07:34:33.259955+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:31:45.261282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:31:45.261282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086095340000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"205'6315","reported_seq":6351,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:44.022972+0000","last_change":"2024-09-11T07:33:29.205385+0000","last_active":"2024-09-11T07:34:44.022972+0000","last_peered":"2024-09-11T07:34:44.022972+0000","last_clean":"2024-09-11T07:34:44.022972+0000","last_became_active":"2024-09-11T07:33:29.205097+0000","last_became_peered":"2024-09-11T07:33:29.205097+0000","last_unstale":"2024-09-11T07:34:44.022972+0000","last_undegraded":"2024-09-11T07:34:44.022972+0000","last_fullsized":"2024-09-11T07:34:44.022972+0000","mapping_epoch":177,"log_start":"187'1100","ondisk_log_start":"187'1100","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":5215,"log_dups_size":0,"ondisk_log_size":5215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:54.401575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413859840,"num_objects":6315,"num_object_clones":0,"num_object_copies":18945,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6315,"num_write_kb":404160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"64'14","reported_seq":205,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.254671+0000","last_change":"2024-09-11T07:31:46.078940+0000","last_active":"2024-09-11T07:34:33.254671+0000","last_peered":"2024-09-11T07:34:33.254671+0000","last_clean":"2024-09-11T07:34:33.254671+0000","last_became_active":"2024-09-11T07:26:33.537850+0000","last_became_peered":"2024-09-11T07:26:33.537850+0000","last_unstale":"2024-09-11T07:34:33.254671+0000","last_undegraded":"2024-09-11T07:34:33.254671+0000","last_fullsized":"2024-09-11T07:34:33.254671+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-09-11T07:31:46.078888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:31:46.078888+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017755819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":217,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:43.819938+0000","last_change":"2024-09-11T07:34:43.819898+0000","last_active":"2024-09-11T07:34:43.819938+0000","last_peered":"2024-09-11T07:34:43.819938+0000","last_clean":"2024-09-11T07:34:43.819938+0000","last_became_active":"2024-09-11T07:28:09.014995+0000","last_became_peered":"2024-09-11T07:28:09.014995+0000","last_unstale":"2024-09-11T07:34:43.819938+0000","last_undegraded":"2024-09-11T07:34:43.819938+0000","last_fullsized":"2024-09-11T07:34:43.819938+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:43.819809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:43.819809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:04.954329+0000","scrub_duration":0.0047161240000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"84'9","reported_seq":291,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:39.601690+0000","last_change":"2024-09-11T07:33:48.510872+0000","last_active":"2024-09-11T07:34:39.601690+0000","last_peered":"2024-09-11T07:34:39.601690+0000","last_clean":"2024-09-11T07:34:39.601690+0000","last_became_active":"2024-09-11T07:26:35.562997+0000","last_became_peered":"2024-09-11T07:26:35.562997+0000","last_unstale":"2024-09-11T07:34:39.601690+0000","last_undegraded":"2024-09-11T07:34:39.601690+0000","last_fullsized":"2024-09-11T07:34:39.601690+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-11T07:33:48.510778+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-11T07:28:05.653237+0000","last_clean_scrub_stamp":"2024-09-11T07:33:48.510778+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:52.240234+0000","scrub_duration":0.0038375390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":165,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:39.170019+0000","last_change":"2024-09-11T07:34:39.169972+0000","last_active":"2024-09-11T07:34:39.170019+0000","last_peered":"2024-09-11T07:34:39.170019+0000","last_clean":"2024-09-11T07:34:39.170019+0000","last_became_active":"2024-09-11T07:32:00.511005+0000","last_became_peered":"2024-09-11T07:32:00.511005+0000","last_unstale":"2024-09-11T07:34:39.170019+0000","last_undegraded":"2024-09-11T07:34:39.170019+0000","last_fullsized":"2024-09-11T07:34:39.170019+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:39.169891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:39.169891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:53.143370+0000","scrub_duration":0.0052103009999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"205'6421","reported_seq":6456,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:44.573185+0000","last_change":"2024-09-11T07:33:29.205044+0000","last_active":"2024-09-11T07:34:44.573185+0000","last_peered":"2024-09-11T07:34:44.573185+0000","last_clean":"2024-09-11T07:34:44.573185+0000","last_became_active":"2024-09-11T07:33:29.204734+0000","last_became_peered":"2024-09-11T07:33:29.204734+0000","last_unstale":"2024-09-11T07:34:44.573185+0000","last_undegraded":"2024-09-11T07:34:44.573185+0000","last_fullsized":"2024-09-11T07:34:44.573185+0000","mapping_epoch":177,"log_start":"187'700","ondisk_log_start":"187'700","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":5721,"log_dups_size":0,"ondisk_log_size":5721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:49.210405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420806656,"num_objects":6421,"num_object_clones":0,"num_object_copies":19263,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6421,"num_write_kb":410944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"87'26","reported_seq":242,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.267485+0000","last_change":"2024-09-11T07:33:52.609653+0000","last_active":"2024-09-11T07:34:33.267485+0000","last_peered":"2024-09-11T07:34:33.267485+0000","last_clean":"2024-09-11T07:34:33.267485+0000","last_became_active":"2024-09-11T07:32:00.520670+0000","last_became_peered":"2024-09-11T07:32:00.520670+0000","last_unstale":"2024-09-11T07:34:33.267485+0000","last_undegraded":"2024-09-11T07:34:33.267485+0000","last_fullsized":"2024-09-11T07:34:33.267485+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'26","last_scrub_stamp":"2024-09-11T07:33:52.609336+0000","last_deep_scrub":"87'26","last_deep_scrub_stamp":"2024-09-11T07:28:30.788647+0000","last_clean_scrub_stamp":"2024-09-11T07:33:52.609336+0000","objects_scrubbed":8,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:15.124860+0000","scrub_duration":0.010453553000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":195,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.253450+0000","last_change":"2024-09-11T07:33:07.547329+0000","last_active":"2024-09-11T07:34:33.253450+0000","last_peered":"2024-09-11T07:34:33.253450+0000","last_clean":"2024-09-11T07:34:33.253450+0000","last_became_active":"2024-09-11T07:26:31.421500+0000","last_became_peered":"2024-09-11T07:26:31.421500+0000","last_unstale":"2024-09-11T07:34:33.253450+0000","last_undegraded":"2024-09-11T07:34:33.253450+0000","last_fullsized":"2024-09-11T07:34:33.253450+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:07.547259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:07.547259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019369229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":202,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.254478+0000","last_change":"2024-09-11T07:33:53.449765+0000","last_active":"2024-09-11T07:34:33.254478+0000","last_peered":"2024-09-11T07:34:33.254478+0000","last_clean":"2024-09-11T07:34:33.254478+0000","last_became_active":"2024-09-11T07:32:00.499747+0000","last_became_peered":"2024-09-11T07:32:00.499747+0000","last_unstale":"2024-09-11T07:34:33.254478+0000","last_undegraded":"2024-09-11T07:34:33.254478+0000","last_fullsized":"2024-09-11T07:34:33.254478+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:53.449689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:27.968579+0000","last_clean_scrub_stamp":"2024-09-11T07:33:53.449689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:12.260963+0000","scrub_duration":0.0033513269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":180,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.255114+0000","last_change":"2024-09-11T07:31:41.033292+0000","last_active":"2024-09-11T07:34:33.255114+0000","last_peered":"2024-09-11T07:34:33.255114+0000","last_clean":"2024-09-11T07:34:33.255114+0000","last_became_active":"2024-09-11T07:26:37.564616+0000","last_became_peered":"2024-09-11T07:26:37.564616+0000","last_unstale":"2024-09-11T07:34:33.255114+0000","last_undegraded":"2024-09-11T07:34:33.255114+0000","last_fullsized":"2024-09-11T07:34:33.255114+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:31:41.033237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:29.978798+0000","last_clean_scrub_stamp":"2024-09-11T07:31:41.033237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014647854,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"205'6291","reported_seq":6327,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:44.611866+0000","last_change":"2024-09-11T07:33:29.161660+0000","last_active":"2024-09-11T07:34:44.611866+0000","last_peered":"2024-09-11T07:34:44.611866+0000","last_clean":"2024-09-11T07:34:44.611866+0000","last_became_active":"2024-09-11T07:33:29.161212+0000","last_became_peered":"2024-09-11T07:33:29.161212+0000","last_unstale":"2024-09-11T07:34:44.611866+0000","last_undegraded":"2024-09-11T07:34:44.611866+0000","last_fullsized":"2024-09-11T07:34:44.611866+0000","mapping_epoch":177,"log_start":"188'1300","ondisk_log_start":"188'1300","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4991,"log_dups_size":0,"ondisk_log_size":4991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:50.971554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":412286976,"num_objects":6291,"num_object_clones":0,"num_object_copies":18873,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6291,"num_write_kb":402624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":247,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:35.790054+0000","last_change":"2024-09-11T07:34:35.790009+0000","last_active":"2024-09-11T07:34:35.790054+0000","last_peered":"2024-09-11T07:34:35.790054+0000","last_clean":"2024-09-11T07:34:35.790054+0000","last_became_active":"2024-09-11T07:32:00.506270+0000","last_became_peered":"2024-09-11T07:32:00.506270+0000","last_unstale":"2024-09-11T07:34:35.790054+0000","last_undegraded":"2024-09-11T07:34:35.790054+0000","last_fullsized":"2024-09-11T07:34:35.790054+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-11T07:34:35.789921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:34:35.789921+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:58.796684+0000","scrub_duration":0.015782836000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":155,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.262579+0000","last_change":"2024-09-11T07:33:13.291210+0000","last_active":"2024-09-11T07:34:33.262579+0000","last_peered":"2024-09-11T07:34:33.262579+0000","last_clean":"2024-09-11T07:34:33.262579+0000","last_became_active":"2024-09-11T07:32:00.510631+0000","last_became_peered":"2024-09-11T07:32:00.510631+0000","last_unstale":"2024-09-11T07:34:33.262579+0000","last_undegraded":"2024-09-11T07:34:33.262579+0000","last_fullsized":"2024-09-11T07:34:33.262579+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:13.291147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:13.291147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036274660000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":203,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:38.765222+0000","last_change":"2024-09-11T07:34:38.765181+0000","last_active":"2024-09-11T07:34:38.765222+0000","last_peered":"2024-09-11T07:34:38.765222+0000","last_clean":"2024-09-11T07:34:38.765222+0000","last_became_active":"2024-09-11T07:26:35.565359+0000","last_became_peered":"2024-09-11T07:26:35.565359+0000","last_unstale":"2024-09-11T07:34:38.765222+0000","last_undegraded":"2024-09-11T07:34:38.765222+0000","last_fullsized":"2024-09-11T07:34:38.765222+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:38.765093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:38.765093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:03.577721+0000","scrub_duration":0.0083567410000000009,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":189,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.252757+0000","last_change":"2024-09-11T07:34:10.294762+0000","last_active":"2024-09-11T07:34:33.252757+0000","last_peered":"2024-09-11T07:34:33.252757+0000","last_clean":"2024-09-11T07:34:33.252757+0000","last_became_active":"2024-09-11T07:26:37.861279+0000","last_became_peered":"2024-09-11T07:26:37.861279+0000","last_unstale":"2024-09-11T07:34:33.252757+0000","last_undegraded":"2024-09-11T07:34:33.252757+0000","last_fullsized":"2024-09-11T07:34:33.252757+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:10.294658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:39.723598+0000","last_clean_scrub_stamp":"2024-09-11T07:34:10.294658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:25.940771+0000","scrub_duration":0.0026992600000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.e","version":"205'6305","reported_seq":6339,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:44.624507+0000","last_change":"2024-09-11T07:33:29.159841+0000","last_active":"2024-09-11T07:34:44.624507+0000","last_peered":"2024-09-11T07:34:44.624507+0000","last_clean":"2024-09-11T07:34:44.624507+0000","last_became_active":"2024-09-11T07:33:29.159535+0000","last_became_peered":"2024-09-11T07:33:29.159535+0000","last_unstale":"2024-09-11T07:34:44.624507+0000","last_undegraded":"2024-09-11T07:34:44.624507+0000","last_fullsized":"2024-09-11T07:34:44.624507+0000","mapping_epoch":177,"log_start":"187'600","ondisk_log_start":"187'600","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":5705,"log_dups_size":0,"ondisk_log_size":5705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:54.041505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413204480,"num_objects":6305,"num_object_clones":0,"num_object_copies":18915,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6305,"num_write_kb":403520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":211,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.253649+0000","last_change":"2024-09-11T07:33:53.294408+0000","last_active":"2024-09-11T07:34:33.253649+0000","last_peered":"2024-09-11T07:34:33.253649+0000","last_clean":"2024-09-11T07:34:33.253649+0000","last_became_active":"2024-09-11T07:26:33.543063+0000","last_became_peered":"2024-09-11T07:26:33.543063+0000","last_unstale":"2024-09-11T07:34:33.253649+0000","last_undegraded":"2024-09-11T07:34:33.253649+0000","last_fullsized":"2024-09-11T07:34:33.253649+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:33:53.294174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:53.294174+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:17.014241+0000","scrub_duration":0.0074369980000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":222,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.253041+0000","last_change":"2024-09-11T07:33:55.344919+0000","last_active":"2024-09-11T07:34:33.253041+0000","last_peered":"2024-09-11T07:34:33.253041+0000","last_clean":"2024-09-11T07:34:33.253041+0000","last_became_active":"2024-09-11T07:32:00.522456+0000","last_became_peered":"2024-09-11T07:32:00.522456+0000","last_unstale":"2024-09-11T07:34:33.253041+0000","last_undegraded":"2024-09-11T07:34:33.253041+0000","last_fullsized":"2024-09-11T07:34:33.253041+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:33:55.344823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:55.344823+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:19.214723+0000","scrub_duration":0.0025904349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":299,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:39.600652+0000","last_change":"2024-09-11T07:33:50.686006+0000","last_active":"2024-09-11T07:34:39.600652+0000","last_peered":"2024-09-11T07:34:39.600652+0000","last_clean":"2024-09-11T07:34:39.600652+0000","last_became_active":"2024-09-11T07:26:35.560422+0000","last_became_peered":"2024-09-11T07:26:35.560422+0000","last_unstale":"2024-09-11T07:34:39.600652+0000","last_undegraded":"2024-09-11T07:34:39.600652+0000","last_fullsized":"2024-09-11T07:34:39.600652+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:33:50.685923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:50.685923+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:54.623978+0000","scrub_duration":0.0029026490000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":200,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:43.901466+0000","last_change":"2024-09-11T07:34:43.901422+0000","last_active":"2024-09-11T07:34:43.901466+0000","last_peered":"2024-09-11T07:34:43.901466+0000","last_clean":"2024-09-11T07:34:43.901466+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:34:43.901466+0000","last_undegraded":"2024-09-11T07:34:43.901466+0000","last_fullsized":"2024-09-11T07:34:43.901466+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:43.901333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_clean_scrub_stamp":"2024-09-11T07:34:43.901333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:50.099382+0000","scrub_duration":0.0036903819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"205'6225","reported_seq":6261,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:44.012555+0000","last_change":"2024-09-11T07:33:29.205462+0000","last_active":"2024-09-11T07:34:44.012555+0000","last_peered":"2024-09-11T07:34:44.012555+0000","last_clean":"2024-09-11T07:34:44.012555+0000","last_became_active":"2024-09-11T07:33:29.205148+0000","last_became_peered":"2024-09-11T07:33:29.205148+0000","last_unstale":"2024-09-11T07:34:44.012555+0000","last_undegraded":"2024-09-11T07:34:44.012555+0000","last_fullsized":"2024-09-11T07:34:44.012555+0000","mapping_epoch":177,"log_start":"187'1000","ondisk_log_start":"187'1000","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":5225,"log_dups_size":0,"ondisk_log_size":5225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":407961600,"num_objects":6225,"num_object_clones":0,"num_object_copies":18675,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6225,"num_write_kb":398400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"170'29","reported_seq":97,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:35.548720+0000","last_change":"2024-09-11T07:34:34.192084+0000","last_active":"2024-09-11T07:34:35.548720+0000","last_peered":"2024-09-11T07:34:35.548720+0000","last_clean":"2024-09-11T07:34:35.548720+0000","last_became_active":"2024-09-11T07:33:10.017961+0000","last_became_peered":"2024-09-11T07:33:10.017961+0000","last_unstale":"2024-09-11T07:34:35.548720+0000","last_undegraded":"2024-09-11T07:34:35.548720+0000","last_fullsized":"2024-09-11T07:34:35.548720+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"170'29","last_scrub_stamp":"2024-09-11T07:34:34.192009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:34:34.192009+0000","objects_scrubbed":10,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:45.027744+0000","scrub_duration":0.0099986009999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":197,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.258374+0000","last_change":"2024-09-11T07:33:12.133334+0000","last_active":"2024-09-11T07:34:33.258374+0000","last_peered":"2024-09-11T07:34:33.258374+0000","last_clean":"2024-09-11T07:34:33.258374+0000","last_became_active":"2024-09-11T07:26:31.424618+0000","last_became_peered":"2024-09-11T07:26:31.424618+0000","last_unstale":"2024-09-11T07:34:33.258374+0000","last_undegraded":"2024-09-11T07:34:33.258374+0000","last_fullsized":"2024-09-11T07:34:33.258374+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:12.133258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:12.775067+0000","last_clean_scrub_stamp":"2024-09-11T07:33:12.133258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013676669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":192,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.254556+0000","last_change":"2024-09-11T07:33:52.629961+0000","last_active":"2024-09-11T07:34:33.254556+0000","last_peered":"2024-09-11T07:34:33.254556+0000","last_clean":"2024-09-11T07:34:33.254556+0000","last_became_active":"2024-09-11T07:26:35.560363+0000","last_became_peered":"2024-09-11T07:26:35.560363+0000","last_unstale":"2024-09-11T07:34:33.254556+0000","last_undegraded":"2024-09-11T07:34:33.254556+0000","last_fullsized":"2024-09-11T07:34:33.254556+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:52.629872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:52.629872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:03.346424+0000","scrub_duration":0.003860858,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":198,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.259116+0000","last_change":"2024-09-11T07:33:13.566605+0000","last_active":"2024-09-11T07:34:33.259116+0000","last_peered":"2024-09-11T07:34:33.259116+0000","last_clean":"2024-09-11T07:34:33.259116+0000","last_became_active":"2024-09-11T07:32:00.505919+0000","last_became_peered":"2024-09-11T07:32:00.505919+0000","last_unstale":"2024-09-11T07:34:33.259116+0000","last_undegraded":"2024-09-11T07:34:33.259116+0000","last_fullsized":"2024-09-11T07:34:33.259116+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:13.566551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:13.566551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015767380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"205'6429","reported_seq":6465,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:44.617808+0000","last_change":"2024-09-11T07:33:29.161446+0000","last_active":"2024-09-11T07:34:44.617808+0000","last_peered":"2024-09-11T07:34:44.617808+0000","last_clean":"2024-09-11T07:34:44.617808+0000","last_became_active":"2024-09-11T07:33:29.161141+0000","last_became_peered":"2024-09-11T07:33:29.161141+0000","last_unstale":"2024-09-11T07:34:44.617808+0000","last_undegraded":"2024-09-11T07:34:44.617808+0000","last_fullsized":"2024-09-11T07:34:44.617808+0000","mapping_epoch":177,"log_start":"188'1500","ondisk_log_start":"188'1500","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4929,"log_dups_size":0,"ondisk_log_size":4929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421330944,"num_objects":6429,"num_object_clones":0,"num_object_copies":19287,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6429,"num_write_kb":411456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"144'18","reported_seq":225,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.255547+0000","last_change":"2024-09-11T07:34:03.365979+0000","last_active":"2024-09-11T07:34:33.255547+0000","last_peered":"2024-09-11T07:34:33.255547+0000","last_clean":"2024-09-11T07:34:33.255547+0000","last_became_active":"2024-09-11T07:26:33.546126+0000","last_became_peered":"2024-09-11T07:26:33.546126+0000","last_unstale":"2024-09-11T07:34:33.255547+0000","last_undegraded":"2024-09-11T07:34:33.255547+0000","last_fullsized":"2024-09-11T07:34:33.255547+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"144'18","last_scrub_stamp":"2024-09-11T07:34:03.365891+0000","last_deep_scrub":"144'18","last_deep_scrub_stamp":"2024-09-11T07:31:51.603378+0000","last_clean_scrub_stamp":"2024-09-11T07:34:03.365891+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:30.450782+0000","scrub_duration":0.006530999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":698,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:42.654039+0000","last_change":"2024-09-11T07:34:42.459557+0000","last_active":"2024-09-11T07:34:42.654039+0000","last_peered":"2024-09-11T07:34:42.654039+0000","last_clean":"2024-09-11T07:34:42.654039+0000","last_became_active":"2024-09-11T07:26:31.421276+0000","last_became_peered":"2024-09-11T07:26:31.421276+0000","last_unstale":"2024-09-11T07:34:42.654039+0000","last_undegraded":"2024-09-11T07:34:42.654039+0000","last_fullsized":"2024-09-11T07:34:42.654039+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'5","last_scrub_stamp":"2024-09-11T07:34:42.459494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:42.459494+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:07.298507+0000","scrub_duration":0.0049235640000000001,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":485,"num_read_kb":480,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":207,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.261365+0000","last_change":"2024-09-11T07:34:07.149600+0000","last_active":"2024-09-11T07:34:33.261365+0000","last_peered":"2024-09-11T07:34:33.261365+0000","last_clean":"2024-09-11T07:34:33.261365+0000","last_became_active":"2024-09-11T07:26:31.413354+0000","last_became_peered":"2024-09-11T07:26:31.413354+0000","last_unstale":"2024-09-11T07:34:33.261365+0000","last_undegraded":"2024-09-11T07:34:33.261365+0000","last_fullsized":"2024-09-11T07:34:33.261365+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:07.149367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:07.149367+0000","last_clean_scrub_stamp":"2024-09-11T07:34:07.149367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:12.409913+0000","scrub_duration":0.0082613879999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":210,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:43.005551+0000","last_change":"2024-09-11T07:34:43.004662+0000","last_active":"2024-09-11T07:34:43.005551+0000","last_peered":"2024-09-11T07:34:43.005551+0000","last_clean":"2024-09-11T07:34:43.005551+0000","last_became_active":"2024-09-11T07:32:00.508398+0000","last_became_peered":"2024-09-11T07:32:00.508398+0000","last_unstale":"2024-09-11T07:34:43.005551+0000","last_undegraded":"2024-09-11T07:34:43.005551+0000","last_fullsized":"2024-09-11T07:34:43.005551+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:43.003030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:43.003030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:03.470264+0000","scrub_duration":0.010745073000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":181,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.262405+0000","last_change":"2024-09-11T07:33:55.344477+0000","last_active":"2024-09-11T07:34:33.262405+0000","last_peered":"2024-09-11T07:34:33.262405+0000","last_clean":"2024-09-11T07:34:33.262405+0000","last_became_active":"2024-09-11T07:32:00.515745+0000","last_became_peered":"2024-09-11T07:32:00.515745+0000","last_unstale":"2024-09-11T07:34:33.262405+0000","last_undegraded":"2024-09-11T07:34:33.262405+0000","last_fullsized":"2024-09-11T07:34:33.262405+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:55.344426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:55.344426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:21.367701+0000","scrub_duration":0.032747216000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"205'6347","reported_seq":6382,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:44.226487+0000","last_change":"2024-09-11T07:33:29.156986+0000","last_active":"2024-09-11T07:34:44.226487+0000","last_peered":"2024-09-11T07:34:44.226487+0000","last_clean":"2024-09-11T07:34:44.226487+0000","last_became_active":"2024-09-11T07:33:29.156567+0000","last_became_peered":"2024-09-11T07:33:29.156567+0000","last_unstale":"2024-09-11T07:34:44.226487+0000","last_undegraded":"2024-09-11T07:34:44.226487+0000","last_fullsized":"2024-09-11T07:34:44.226487+0000","mapping_epoch":177,"log_start":"188'1300","ondisk_log_start":"188'1300","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":5047,"log_dups_size":0,"ondisk_log_size":5047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:45.241348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":415956992,"num_objects":6347,"num_object_clones":0,"num_object_copies":19041,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6347,"num_write_kb":406208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"94'18","reported_seq":242,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.254566+0000","last_change":"2024-09-11T07:33:18.649291+0000","last_active":"2024-09-11T07:34:33.254566+0000","last_peered":"2024-09-11T07:34:33.254566+0000","last_clean":"2024-09-11T07:34:33.254566+0000","last_became_active":"2024-09-11T07:28:09.018134+0000","last_became_peered":"2024-09-11T07:28:09.018134+0000","last_unstale":"2024-09-11T07:34:33.254566+0000","last_undegraded":"2024-09-11T07:34:33.254566+0000","last_fullsized":"2024-09-11T07:34:33.254566+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'18","last_scrub_stamp":"2024-09-11T07:33:18.648484+0000","last_deep_scrub":"94'18","last_deep_scrub_stamp":"2024-09-11T07:28:51.437397+0000","last_clean_scrub_stamp":"2024-09-11T07:33:18.648484+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023327030000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":202,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.256300+0000","last_change":"2024-09-11T07:33:48.749463+0000","last_active":"2024-09-11T07:34:33.256300+0000","last_peered":"2024-09-11T07:34:33.256300+0000","last_clean":"2024-09-11T07:34:33.256300+0000","last_became_active":"2024-09-11T07:26:31.420320+0000","last_became_peered":"2024-09-11T07:26:31.420320+0000","last_unstale":"2024-09-11T07:34:33.256300+0000","last_undegraded":"2024-09-11T07:34:33.256300+0000","last_fullsized":"2024-09-11T07:34:33.256300+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-11T07:33:48.749183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:48.749183+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:14.622124+0000","scrub_duration":0.0059599450000000003,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":192,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.260607+0000","last_change":"2024-09-11T07:33:58.311557+0000","last_active":"2024-09-11T07:34:33.260607+0000","last_peered":"2024-09-11T07:34:33.260607+0000","last_clean":"2024-09-11T07:34:33.260607+0000","last_became_active":"2024-09-11T07:32:00.520035+0000","last_became_peered":"2024-09-11T07:32:00.520035+0000","last_unstale":"2024-09-11T07:34:33.260607+0000","last_undegraded":"2024-09-11T07:34:33.260607+0000","last_fullsized":"2024-09-11T07:34:33.260607+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:58.311344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:16.967492+0000","last_clean_scrub_stamp":"2024-09-11T07:33:58.311344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:22.200383+0000","scrub_duration":0.0030726579999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":183,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.258325+0000","last_change":"2024-09-11T07:34:12.211194+0000","last_active":"2024-09-11T07:34:33.258325+0000","last_peered":"2024-09-11T07:34:33.258325+0000","last_clean":"2024-09-11T07:34:33.258325+0000","last_became_active":"2024-09-11T07:26:35.564178+0000","last_became_peered":"2024-09-11T07:26:35.564178+0000","last_unstale":"2024-09-11T07:34:33.258325+0000","last_undegraded":"2024-09-11T07:34:33.258325+0000","last_fullsized":"2024-09-11T07:34:33.258325+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:12.211119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:12.211119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:16.160923+0000","scrub_duration":0.0062325000000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":192,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.252878+0000","last_change":"2024-09-11T07:33:58.323420+0000","last_active":"2024-09-11T07:34:33.252878+0000","last_peered":"2024-09-11T07:34:33.252878+0000","last_clean":"2024-09-11T07:34:33.252878+0000","last_became_active":"2024-09-11T07:26:37.561378+0000","last_became_peered":"2024-09-11T07:26:37.561378+0000","last_unstale":"2024-09-11T07:34:33.252878+0000","last_undegraded":"2024-09-11T07:34:33.252878+0000","last_fullsized":"2024-09-11T07:34:33.252878+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:58.323318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:58.323318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:00.851560+0000","scrub_duration":0.0083597830000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"205'6303","reported_seq":6339,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:44.134849+0000","last_change":"2024-09-11T07:33:29.154996+0000","last_active":"2024-09-11T07:34:44.134849+0000","last_peered":"2024-09-11T07:34:44.134849+0000","last_clean":"2024-09-11T07:34:44.134849+0000","last_became_active":"2024-09-11T07:33:29.154702+0000","last_became_peered":"2024-09-11T07:33:29.154702+0000","last_unstale":"2024-09-11T07:34:44.134849+0000","last_undegraded":"2024-09-11T07:34:44.134849+0000","last_fullsized":"2024-09-11T07:34:44.134849+0000","mapping_epoch":177,"log_start":"185'100","ondisk_log_start":"185'100","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":6203,"log_dups_size":0,"ondisk_log_size":6203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:51.999557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413073408,"num_objects":6303,"num_object_clones":0,"num_object_copies":18909,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6303,"num_write_kb":403392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":237,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.263189+0000","last_change":"2024-09-11T07:34:07.759083+0000","last_active":"2024-09-11T07:34:33.263189+0000","last_peered":"2024-09-11T07:34:33.263189+0000","last_clean":"2024-09-11T07:34:33.263189+0000","last_became_active":"2024-09-11T07:32:00.517409+0000","last_became_peered":"2024-09-11T07:32:00.517409+0000","last_unstale":"2024-09-11T07:34:33.263189+0000","last_undegraded":"2024-09-11T07:34:33.263189+0000","last_fullsized":"2024-09-11T07:34:33.263189+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-09-11T07:34:07.758985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:34:07.758985+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:22.987261+0000","scrub_duration":0.0027645090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'3","reported_seq":205,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:43.819999+0000","last_change":"2024-09-11T07:34:43.819953+0000","last_active":"2024-09-11T07:34:43.819999+0000","last_peered":"2024-09-11T07:34:43.819999+0000","last_clean":"2024-09-11T07:34:43.819999+0000","last_became_active":"2024-09-11T07:26:31.419501+0000","last_became_peered":"2024-09-11T07:26:31.419501+0000","last_unstale":"2024-09-11T07:34:43.819999+0000","last_undegraded":"2024-09-11T07:34:43.819999+0000","last_fullsized":"2024-09-11T07:34:43.819999+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-09-11T07:34:43.819873+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_clean_scrub_stamp":"2024-09-11T07:34:43.819873+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:07.822773+0000","scrub_duration":0.010008270999999999,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":207,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.257482+0000","last_change":"2024-09-11T07:33:51.108800+0000","last_active":"2024-09-11T07:34:33.257482+0000","last_peered":"2024-09-11T07:34:33.257482+0000","last_clean":"2024-09-11T07:34:33.257482+0000","last_became_active":"2024-09-11T07:32:00.498002+0000","last_became_peered":"2024-09-11T07:32:00.498002+0000","last_unstale":"2024-09-11T07:34:33.257482+0000","last_undegraded":"2024-09-11T07:34:33.257482+0000","last_fullsized":"2024-09-11T07:34:33.257482+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:51.108748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:51.108748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:55.524512+0000","scrub_duration":0.0013286089999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"87'81","reported_seq":208,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.260536+0000","last_change":"2024-09-11T07:33:10.189446+0000","last_active":"2024-09-11T07:34:33.260536+0000","last_peered":"2024-09-11T07:34:33.260536+0000","last_clean":"2024-09-11T07:34:33.260536+0000","last_became_active":"2024-09-11T07:32:00.519656+0000","last_became_peered":"2024-09-11T07:32:00.519656+0000","last_unstale":"2024-09-11T07:34:33.260536+0000","last_undegraded":"2024-09-11T07:34:33.260536+0000","last_fullsized":"2024-09-11T07:34:33.260536+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:33:10.189376+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032570229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":178,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.261433+0000","last_change":"2024-09-11T07:33:51.272916+0000","last_active":"2024-09-11T07:34:33.261433+0000","last_peered":"2024-09-11T07:34:33.261433+0000","last_clean":"2024-09-11T07:34:33.261433+0000","last_became_active":"2024-09-11T07:32:00.510888+0000","last_became_peered":"2024-09-11T07:32:00.510888+0000","last_unstale":"2024-09-11T07:34:33.261433+0000","last_undegraded":"2024-09-11T07:34:33.261433+0000","last_fullsized":"2024-09-11T07:34:33.261433+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:51.272831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:51.272831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:06.075633+0000","scrub_duration":0.0013461180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":181,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.258393+0000","last_change":"2024-09-11T07:32:57.235357+0000","last_active":"2024-09-11T07:34:33.258393+0000","last_peered":"2024-09-11T07:34:33.258393+0000","last_clean":"2024-09-11T07:34:33.258393+0000","last_became_active":"2024-09-11T07:26:37.559920+0000","last_became_peered":"2024-09-11T07:26:37.559920+0000","last_unstale":"2024-09-11T07:34:33.258393+0000","last_undegraded":"2024-09-11T07:34:33.258393+0000","last_fullsized":"2024-09-11T07:34:33.258393+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:32:57.235305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:32:57.235305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013458579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"205'6285","reported_seq":6320,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:44.593816+0000","last_change":"2024-09-11T07:33:29.204945+0000","last_active":"2024-09-11T07:34:44.593816+0000","last_peered":"2024-09-11T07:34:44.593816+0000","last_clean":"2024-09-11T07:34:44.593816+0000","last_became_active":"2024-09-11T07:33:29.204611+0000","last_became_peered":"2024-09-11T07:33:29.204611+0000","last_unstale":"2024-09-11T07:34:44.593816+0000","last_undegraded":"2024-09-11T07:34:44.593816+0000","last_fullsized":"2024-09-11T07:34:44.593816+0000","mapping_epoch":177,"log_start":"187'600","ondisk_log_start":"187'600","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":5685,"log_dups_size":0,"ondisk_log_size":5685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":411893760,"num_objects":6285,"num_object_clones":0,"num_object_copies":18855,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6285,"num_write_kb":402240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":236,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.262809+0000","last_change":"2024-09-11T07:33:49.612375+0000","last_active":"2024-09-11T07:34:33.262809+0000","last_peered":"2024-09-11T07:34:33.262809+0000","last_clean":"2024-09-11T07:34:33.262809+0000","last_became_active":"2024-09-11T07:32:00.503799+0000","last_became_peered":"2024-09-11T07:32:00.503799+0000","last_unstale":"2024-09-11T07:34:33.262809+0000","last_undegraded":"2024-09-11T07:34:33.262809+0000","last_fullsized":"2024-09-11T07:34:33.262809+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-11T07:33:49.612326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:49.612326+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:15.075880+0000","scrub_duration":0.003204648,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":197,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.261291+0000","last_change":"2024-09-11T07:34:02.192275+0000","last_active":"2024-09-11T07:34:33.261291+0000","last_peered":"2024-09-11T07:34:33.261291+0000","last_clean":"2024-09-11T07:34:33.261291+0000","last_became_active":"2024-09-11T07:26:31.419560+0000","last_became_peered":"2024-09-11T07:26:31.419560+0000","last_unstale":"2024-09-11T07:34:33.261291+0000","last_undegraded":"2024-09-11T07:34:33.261291+0000","last_fullsized":"2024-09-11T07:34:33.261291+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:02.192193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:02.192193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:15.980889+0000","scrub_duration":0.0060418110000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":193,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.260617+0000","last_change":"2024-09-11T07:34:06.134576+0000","last_active":"2024-09-11T07:34:33.260617+0000","last_peered":"2024-09-11T07:34:33.260617+0000","last_clean":"2024-09-11T07:34:33.260617+0000","last_became_active":"2024-09-11T07:26:35.564044+0000","last_became_peered":"2024-09-11T07:26:35.564044+0000","last_unstale":"2024-09-11T07:34:33.260617+0000","last_undegraded":"2024-09-11T07:34:33.260617+0000","last_fullsized":"2024-09-11T07:34:33.260617+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:06.134464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:06.134464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:15.601865+0000","scrub_duration":0.0062969130000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":186,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:38.220057+0000","last_change":"2024-09-11T07:34:38.219945+0000","last_active":"2024-09-11T07:34:38.220057+0000","last_peered":"2024-09-11T07:34:38.220057+0000","last_clean":"2024-09-11T07:34:38.220057+0000","last_became_active":"2024-09-11T07:32:00.516916+0000","last_became_peered":"2024-09-11T07:32:00.516916+0000","last_unstale":"2024-09-11T07:34:38.220057+0000","last_undegraded":"2024-09-11T07:34:38.220057+0000","last_fullsized":"2024-09-11T07:34:38.220057+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:38.219876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:38.219876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:49.345830+0000","scrub_duration":0.020549118000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.7","version":"205'6273","reported_seq":6308,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:44.218650+0000","last_change":"2024-09-11T07:33:29.160658+0000","last_active":"2024-09-11T07:34:44.218650+0000","last_peered":"2024-09-11T07:34:44.218650+0000","last_clean":"2024-09-11T07:34:44.218650+0000","last_became_active":"2024-09-11T07:33:29.160376+0000","last_became_peered":"2024-09-11T07:33:29.160376+0000","last_unstale":"2024-09-11T07:34:44.218650+0000","last_undegraded":"2024-09-11T07:34:44.218650+0000","last_fullsized":"2024-09-11T07:34:44.218650+0000","mapping_epoch":177,"log_start":"188'1200","ondisk_log_start":"188'1200","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":5073,"log_dups_size":0,"ondisk_log_size":5073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:55.763414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":411107328,"num_objects":6273,"num_object_clones":0,"num_object_copies":18819,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6273,"num_write_kb":401472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":200,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.254087+0000","last_change":"2024-09-11T07:33:14.754606+0000","last_active":"2024-09-11T07:34:33.254087+0000","last_peered":"2024-09-11T07:34:33.254087+0000","last_clean":"2024-09-11T07:34:33.254087+0000","last_became_active":"2024-09-11T07:26:33.543148+0000","last_became_peered":"2024-09-11T07:26:33.543148+0000","last_unstale":"2024-09-11T07:34:33.254087+0000","last_undegraded":"2024-09-11T07:34:33.254087+0000","last_fullsized":"2024-09-11T07:34:33.254087+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:33:14.754532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:14.754532+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034842570000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":205,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:43.479878+0000","last_change":"2024-09-11T07:34:43.479834+0000","last_active":"2024-09-11T07:34:43.479878+0000","last_peered":"2024-09-11T07:34:43.479878+0000","last_clean":"2024-09-11T07:34:43.479878+0000","last_became_active":"2024-09-11T07:26:31.423056+0000","last_became_peered":"2024-09-11T07:26:31.423056+0000","last_unstale":"2024-09-11T07:34:43.479878+0000","last_undegraded":"2024-09-11T07:34:43.479878+0000","last_fullsized":"2024-09-11T07:34:43.479878+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:43.479741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:43.479741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:10.044799+0000","scrub_duration":0.0073695460000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":203,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:35.918914+0000","last_change":"2024-09-11T07:34:35.918872+0000","last_active":"2024-09-11T07:34:35.918914+0000","last_peered":"2024-09-11T07:34:35.918914+0000","last_clean":"2024-09-11T07:34:35.918914+0000","last_became_active":"2024-09-11T07:26:35.563182+0000","last_became_peered":"2024-09-11T07:26:35.563182+0000","last_unstale":"2024-09-11T07:34:35.918914+0000","last_undegraded":"2024-09-11T07:34:35.918914+0000","last_fullsized":"2024-09-11T07:34:35.918914+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:35.918794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:35.918794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:00.878344+0000","scrub_duration":0.0031958289999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":197,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.256218+0000","last_change":"2024-09-11T07:33:11.790118+0000","last_active":"2024-09-11T07:34:33.256218+0000","last_peered":"2024-09-11T07:34:33.256218+0000","last_clean":"2024-09-11T07:34:33.256218+0000","last_became_active":"2024-09-11T07:32:00.505185+0000","last_became_peered":"2024-09-11T07:32:00.505185+0000","last_unstale":"2024-09-11T07:34:33.256218+0000","last_undegraded":"2024-09-11T07:34:33.256218+0000","last_fullsized":"2024-09-11T07:34:33.256218+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:11.790063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:11.790063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001675405,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"205'6283","reported_seq":6319,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:44.008205+0000","last_change":"2024-09-11T07:33:29.153952+0000","last_active":"2024-09-11T07:34:44.008205+0000","last_peered":"2024-09-11T07:34:44.008205+0000","last_clean":"2024-09-11T07:34:44.008205+0000","last_became_active":"2024-09-11T07:33:29.153604+0000","last_became_peered":"2024-09-11T07:33:29.153604+0000","last_unstale":"2024-09-11T07:34:44.008205+0000","last_undegraded":"2024-09-11T07:34:44.008205+0000","last_fullsized":"2024-09-11T07:34:44.008205+0000","mapping_epoch":177,"log_start":"187'1100","ondisk_log_start":"187'1100","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":5183,"log_dups_size":0,"ondisk_log_size":5183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:52.795972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":411762688,"num_objects":6283,"num_object_clones":0,"num_object_copies":18849,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6283,"num_write_kb":402112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":230,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.262059+0000","last_change":"2024-09-11T07:33:56.187920+0000","last_active":"2024-09-11T07:34:33.262059+0000","last_peered":"2024-09-11T07:34:33.262059+0000","last_clean":"2024-09-11T07:34:33.262059+0000","last_became_active":"2024-09-11T07:32:00.521464+0000","last_became_peered":"2024-09-11T07:32:00.521464+0000","last_unstale":"2024-09-11T07:34:33.262059+0000","last_undegraded":"2024-09-11T07:34:33.262059+0000","last_fullsized":"2024-09-11T07:34:33.262059+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-09-11T07:33:56.187871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:56.187871+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:23.701092+0000","scrub_duration":0.001643175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":224,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:36.912114+0000","last_change":"2024-09-11T07:34:36.912017+0000","last_active":"2024-09-11T07:34:36.912114+0000","last_peered":"2024-09-11T07:34:36.912114+0000","last_clean":"2024-09-11T07:34:36.912114+0000","last_became_active":"2024-09-11T07:32:00.513949+0000","last_became_peered":"2024-09-11T07:32:00.513949+0000","last_unstale":"2024-09-11T07:34:36.912114+0000","last_undegraded":"2024-09-11T07:34:36.912114+0000","last_fullsized":"2024-09-11T07:34:36.912114+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:36.911933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_clean_scrub_stamp":"2024-09-11T07:34:36.911933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:57.170517+0000","scrub_duration":0.0036783879999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"151'10","reported_seq":250,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:35.550501+0000","last_change":"2024-09-11T07:33:48.261946+0000","last_active":"2024-09-11T07:34:35.550501+0000","last_peered":"2024-09-11T07:34:35.550501+0000","last_clean":"2024-09-11T07:34:35.550501+0000","last_became_active":"2024-09-11T07:32:00.506806+0000","last_became_peered":"2024-09-11T07:32:00.506806+0000","last_unstale":"2024-09-11T07:34:35.550501+0000","last_undegraded":"2024-09-11T07:34:35.550501+0000","last_fullsized":"2024-09-11T07:34:35.550501+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:33:48.261857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:48.261857+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:06.611603+0000","scrub_duration":0.001795774,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":191,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:35.190926+0000","last_change":"2024-09-11T07:34:35.190891+0000","last_active":"2024-09-11T07:34:35.190926+0000","last_peered":"2024-09-11T07:34:35.190926+0000","last_clean":"2024-09-11T07:34:35.190926+0000","last_became_active":"2024-09-11T07:26:37.861134+0000","last_became_peered":"2024-09-11T07:26:37.861134+0000","last_unstale":"2024-09-11T07:34:35.190926+0000","last_undegraded":"2024-09-11T07:34:35.190926+0000","last_fullsized":"2024-09-11T07:34:35.190926+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:35.190837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:35.190837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:48.885204+0000","scrub_duration":0.003575679,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.5","version":"205'6229","reported_seq":6265,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:44.124721+0000","last_change":"2024-09-11T07:33:29.205466+0000","last_active":"2024-09-11T07:34:44.124721+0000","last_peered":"2024-09-11T07:34:44.124721+0000","last_clean":"2024-09-11T07:34:44.124721+0000","last_became_active":"2024-09-11T07:33:29.205189+0000","last_became_peered":"2024-09-11T07:33:29.205189+0000","last_unstale":"2024-09-11T07:34:44.124721+0000","last_undegraded":"2024-09-11T07:34:44.124721+0000","last_fullsized":"2024-09-11T07:34:44.124721+0000","mapping_epoch":177,"log_start":"185'100","ondisk_log_start":"185'100","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":6129,"log_dups_size":0,"ondisk_log_size":6129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:56.812992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":408223744,"num_objects":6229,"num_object_clones":0,"num_object_copies":18687,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6229,"num_write_kb":398656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"83'17","reported_seq":212,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.254498+0000","last_change":"2024-09-11T07:31:32.003568+0000","last_active":"2024-09-11T07:34:33.254498+0000","last_peered":"2024-09-11T07:34:33.254498+0000","last_clean":"2024-09-11T07:34:33.254498+0000","last_became_active":"2024-09-11T07:26:33.540363+0000","last_became_peered":"2024-09-11T07:26:33.540363+0000","last_unstale":"2024-09-11T07:34:33.254498+0000","last_undegraded":"2024-09-11T07:34:33.254498+0000","last_fullsized":"2024-09-11T07:34:33.254498+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'17","last_scrub_stamp":"2024-09-11T07:31:32.003516+0000","last_deep_scrub":"83'17","last_deep_scrub_stamp":"2024-09-11T07:29:50.970355+0000","last_clean_scrub_stamp":"2024-09-11T07:31:32.003516+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004484811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":223,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.255246+0000","last_change":"2024-09-11T07:33:48.436215+0000","last_active":"2024-09-11T07:34:33.255246+0000","last_peered":"2024-09-11T07:34:33.255246+0000","last_clean":"2024-09-11T07:34:33.255246+0000","last_became_active":"2024-09-11T07:32:00.504344+0000","last_became_peered":"2024-09-11T07:32:00.504344+0000","last_unstale":"2024-09-11T07:34:33.255246+0000","last_undegraded":"2024-09-11T07:34:33.255246+0000","last_fullsized":"2024-09-11T07:34:33.255246+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:33:48.436127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:48.436127+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:59.296316+0000","scrub_duration":0.002390933,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":193,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.260351+0000","last_change":"2024-09-11T07:33:51.206706+0000","last_active":"2024-09-11T07:34:33.260351+0000","last_peered":"2024-09-11T07:34:33.260351+0000","last_clean":"2024-09-11T07:34:33.260351+0000","last_became_active":"2024-09-11T07:26:35.561967+0000","last_became_peered":"2024-09-11T07:26:35.561967+0000","last_unstale":"2024-09-11T07:34:33.260351+0000","last_undegraded":"2024-09-11T07:34:33.260351+0000","last_fullsized":"2024-09-11T07:34:33.260351+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:51.206632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:44.820354+0000","last_clean_scrub_stamp":"2024-09-11T07:33:51.206632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:01.971369+0000","scrub_duration":0.0065238259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":198,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.259126+0000","last_change":"2024-09-11T07:33:10.531766+0000","last_active":"2024-09-11T07:34:33.259126+0000","last_peered":"2024-09-11T07:34:33.259126+0000","last_clean":"2024-09-11T07:34:33.259126+0000","last_became_active":"2024-09-11T07:32:00.502019+0000","last_became_peered":"2024-09-11T07:32:00.502019+0000","last_unstale":"2024-09-11T07:34:33.259126+0000","last_undegraded":"2024-09-11T07:34:33.259126+0000","last_fullsized":"2024-09-11T07:34:33.259126+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:10.531696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:10.531696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018694370000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":217,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.254492+0000","last_change":"2024-09-11T07:33:05.880679+0000","last_active":"2024-09-11T07:34:33.254492+0000","last_peered":"2024-09-11T07:34:33.254492+0000","last_clean":"2024-09-11T07:34:33.254492+0000","last_became_active":"2024-09-11T07:32:00.509412+0000","last_became_peered":"2024-09-11T07:32:00.509412+0000","last_unstale":"2024-09-11T07:34:33.254492+0000","last_undegraded":"2024-09-11T07:34:33.254492+0000","last_fullsized":"2024-09-11T07:34:33.254492+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:33:05.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:05.880612+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021137109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":197,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.260481+0000","last_change":"2024-09-11T07:34:03.167417+0000","last_active":"2024-09-11T07:34:33.260481+0000","last_peered":"2024-09-11T07:34:33.260481+0000","last_clean":"2024-09-11T07:34:33.260481+0000","last_became_active":"2024-09-11T07:26:31.415701+0000","last_became_peered":"2024-09-11T07:26:31.415701+0000","last_unstale":"2024-09-11T07:34:33.260481+0000","last_undegraded":"2024-09-11T07:34:33.260481+0000","last_fullsized":"2024-09-11T07:34:33.260481+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:03.167327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.925200+0000","last_clean_scrub_stamp":"2024-09-11T07:34:03.167327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:28.877249+0000","scrub_duration":0.0047032489999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":200,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.259697+0000","last_change":"2024-09-11T07:33:54.643176+0000","last_active":"2024-09-11T07:34:33.259697+0000","last_peered":"2024-09-11T07:34:33.259697+0000","last_clean":"2024-09-11T07:34:33.259697+0000","last_became_active":"2024-09-11T07:32:00.499598+0000","last_became_peered":"2024-09-11T07:32:00.499598+0000","last_unstale":"2024-09-11T07:34:33.259697+0000","last_undegraded":"2024-09-11T07:34:33.259697+0000","last_fullsized":"2024-09-11T07:34:33.259697+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:54.642486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:35.392360+0000","last_clean_scrub_stamp":"2024-09-11T07:33:54.642486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:23.835294+0000","scrub_duration":0.010529307999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":209,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:40.972730+0000","last_change":"2024-09-11T07:34:40.972629+0000","last_active":"2024-09-11T07:34:40.972730+0000","last_peered":"2024-09-11T07:34:40.972730+0000","last_clean":"2024-09-11T07:34:40.972730+0000","last_became_active":"2024-09-11T07:32:00.506697+0000","last_became_peered":"2024-09-11T07:32:00.506697+0000","last_unstale":"2024-09-11T07:34:40.972730+0000","last_undegraded":"2024-09-11T07:34:40.972730+0000","last_fullsized":"2024-09-11T07:34:40.972730+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:34:40.971610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:40.971610+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:41.756990+0000","scrub_duration":0.0083037709999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":197,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.262572+0000","last_change":"2024-09-11T07:33:03.339633+0000","last_active":"2024-09-11T07:34:33.262572+0000","last_peered":"2024-09-11T07:34:33.262572+0000","last_clean":"2024-09-11T07:34:33.262572+0000","last_became_active":"2024-09-11T07:26:31.423229+0000","last_became_peered":"2024-09-11T07:26:31.423229+0000","last_unstale":"2024-09-11T07:34:33.262572+0000","last_undegraded":"2024-09-11T07:34:33.262572+0000","last_fullsized":"2024-09-11T07:34:33.262572+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:03.339569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:03.339569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001437523,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"82'9","reported_seq":214,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.259872+0000","last_change":"2024-09-11T07:33:11.576715+0000","last_active":"2024-09-11T07:34:33.259872+0000","last_peered":"2024-09-11T07:34:33.259872+0000","last_clean":"2024-09-11T07:34:33.259872+0000","last_became_active":"2024-09-11T07:32:00.501365+0000","last_became_peered":"2024-09-11T07:32:00.501365+0000","last_unstale":"2024-09-11T07:34:33.259872+0000","last_undegraded":"2024-09-11T07:34:33.259872+0000","last_fullsized":"2024-09-11T07:34:33.259872+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'9","last_scrub_stamp":"2024-09-11T07:33:11.576656+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:33:11.576656+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:35.698527+0000","scrub_duration":0.0043542040000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":193,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.258747+0000","last_change":"2024-09-11T07:33:15.113446+0000","last_active":"2024-09-11T07:34:33.258747+0000","last_peered":"2024-09-11T07:34:33.258747+0000","last_clean":"2024-09-11T07:34:33.258747+0000","last_became_active":"2024-09-11T07:26:35.566894+0000","last_became_peered":"2024-09-11T07:26:35.566894+0000","last_unstale":"2024-09-11T07:34:33.258747+0000","last_undegraded":"2024-09-11T07:34:33.258747+0000","last_fullsized":"2024-09-11T07:34:33.258747+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:15.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:33:15.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011939730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":179,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.263541+0000","last_change":"2024-09-11T07:33:54.283942+0000","last_active":"2024-09-11T07:34:33.263541+0000","last_peered":"2024-09-11T07:34:33.263541+0000","last_clean":"2024-09-11T07:34:33.263541+0000","last_became_active":"2024-09-11T07:32:00.514100+0000","last_became_peered":"2024-09-11T07:32:00.514100+0000","last_unstale":"2024-09-11T07:34:33.263541+0000","last_undegraded":"2024-09-11T07:34:33.263541+0000","last_fullsized":"2024-09-11T07:34:33.263541+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:54.283860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:54.283860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:00.050044+0000","scrub_duration":0.0032251950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"86'15","reported_seq":234,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.260363+0000","last_change":"2024-09-11T07:32:59.286373+0000","last_active":"2024-09-11T07:34:33.260363+0000","last_peered":"2024-09-11T07:34:33.260363+0000","last_clean":"2024-09-11T07:34:33.260363+0000","last_became_active":"2024-09-11T07:32:00.504463+0000","last_became_peered":"2024-09-11T07:32:00.504463+0000","last_unstale":"2024-09-11T07:34:33.260363+0000","last_undegraded":"2024-09-11T07:34:33.260363+0000","last_fullsized":"2024-09-11T07:34:33.260363+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'15","last_scrub_stamp":"2024-09-11T07:32:59.286314+0000","last_deep_scrub":"86'15","last_deep_scrub_stamp":"2024-09-11T07:28:22.858137+0000","last_clean_scrub_stamp":"2024-09-11T07:32:59.286314+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018903660000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":207,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.258305+0000","last_change":"2024-09-11T07:34:02.231632+0000","last_active":"2024-09-11T07:34:33.258305+0000","last_peered":"2024-09-11T07:34:33.258305+0000","last_clean":"2024-09-11T07:34:33.258305+0000","last_became_active":"2024-09-11T07:32:00.498473+0000","last_became_peered":"2024-09-11T07:32:00.498473+0000","last_unstale":"2024-09-11T07:34:33.258305+0000","last_undegraded":"2024-09-11T07:34:33.258305+0000","last_fullsized":"2024-09-11T07:34:33.258305+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:02.231583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:02.231583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:24.160910+0000","scrub_duration":0.0013039099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":289,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:35.548206+0000","last_change":"2024-09-11T07:31:42.984614+0000","last_active":"2024-09-11T07:34:35.548206+0000","last_peered":"2024-09-11T07:34:35.548206+0000","last_clean":"2024-09-11T07:34:35.548206+0000","last_became_active":"2024-09-11T07:26:35.562245+0000","last_became_peered":"2024-09-11T07:26:35.562245+0000","last_unstale":"2024-09-11T07:34:35.548206+0000","last_undegraded":"2024-09-11T07:34:35.548206+0000","last_fullsized":"2024-09-11T07:34:35.548206+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:31:42.984562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:31:42.984562+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015727580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":207,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.254063+0000","last_change":"2024-09-11T07:34:12.556603+0000","last_active":"2024-09-11T07:34:33.254063+0000","last_peered":"2024-09-11T07:34:33.254063+0000","last_clean":"2024-09-11T07:34:33.254063+0000","last_became_active":"2024-09-11T07:32:00.509295+0000","last_became_peered":"2024-09-11T07:32:00.509295+0000","last_unstale":"2024-09-11T07:34:33.254063+0000","last_undegraded":"2024-09-11T07:34:33.254063+0000","last_fullsized":"2024-09-11T07:34:33.254063+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:12.556370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:12.556370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:14.451889+0000","scrub_duration":0.0041791759999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"83'13","reported_seq":225,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.266474+0000","last_change":"2024-09-11T07:33:52.110471+0000","last_active":"2024-09-11T07:34:33.266474+0000","last_peered":"2024-09-11T07:34:33.266474+0000","last_clean":"2024-09-11T07:34:33.266474+0000","last_became_active":"2024-09-11T07:32:00.505134+0000","last_became_peered":"2024-09-11T07:32:00.505134+0000","last_unstale":"2024-09-11T07:34:33.266474+0000","last_undegraded":"2024-09-11T07:34:33.266474+0000","last_fullsized":"2024-09-11T07:34:33.266474+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-11T07:33:52.110361+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-11T07:28:00.959583+0000","last_clean_scrub_stamp":"2024-09-11T07:33:52.110361+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:04.735778+0000","scrub_duration":0.0035066310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":196,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.254293+0000","last_change":"2024-09-11T07:33:51.733858+0000","last_active":"2024-09-11T07:34:33.254293+0000","last_peered":"2024-09-11T07:34:33.254293+0000","last_clean":"2024-09-11T07:34:33.254293+0000","last_became_active":"2024-09-11T07:26:31.419409+0000","last_became_peered":"2024-09-11T07:26:31.419409+0000","last_unstale":"2024-09-11T07:34:33.254293+0000","last_undegraded":"2024-09-11T07:34:33.254293+0000","last_fullsized":"2024-09-11T07:34:33.254293+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:51.733780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:51.733780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:20.982450+0000","scrub_duration":0.079912094000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"151'10","reported_seq":302,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:39.602212+0000","last_change":"2024-09-11T07:33:47.546713+0000","last_active":"2024-09-11T07:34:39.602212+0000","last_peered":"2024-09-11T07:34:39.602212+0000","last_clean":"2024-09-11T07:34:39.602212+0000","last_became_active":"2024-09-11T07:32:00.504062+0000","last_became_peered":"2024-09-11T07:32:00.504062+0000","last_unstale":"2024-09-11T07:34:39.602212+0000","last_undegraded":"2024-09-11T07:34:39.602212+0000","last_fullsized":"2024-09-11T07:34:39.602212+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:33:47.546622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:47.546622+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:07.659165+0000","scrub_duration":0.0064760360000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":190,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:42.324759+0000","last_change":"2024-09-11T07:34:42.324717+0000","last_active":"2024-09-11T07:34:42.324759+0000","last_peered":"2024-09-11T07:34:42.324759+0000","last_clean":"2024-09-11T07:34:42.324759+0000","last_became_active":"2024-09-11T07:26:37.565723+0000","last_became_peered":"2024-09-11T07:26:37.565723+0000","last_unstale":"2024-09-11T07:34:42.324759+0000","last_undegraded":"2024-09-11T07:34:42.324759+0000","last_fullsized":"2024-09-11T07:34:42.324759+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:42.324639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:42.324639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:59.418722+0000","scrub_duration":0.011135859,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":219,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.254124+0000","last_change":"2024-09-11T07:34:12.364224+0000","last_active":"2024-09-11T07:34:33.254124+0000","last_peered":"2024-09-11T07:34:33.254124+0000","last_clean":"2024-09-11T07:34:33.254124+0000","last_became_active":"2024-09-11T07:26:33.540688+0000","last_became_peered":"2024-09-11T07:26:33.540688+0000","last_unstale":"2024-09-11T07:34:33.254124+0000","last_undegraded":"2024-09-11T07:34:33.254124+0000","last_fullsized":"2024-09-11T07:34:33.254124+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:34:12.364126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:34:12.364126+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:21.215985+0000","scrub_duration":0.011537936,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":200,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.263257+0000","last_change":"2024-09-11T07:34:09.772110+0000","last_active":"2024-09-11T07:34:33.263257+0000","last_peered":"2024-09-11T07:34:33.263257+0000","last_clean":"2024-09-11T07:34:33.263257+0000","last_became_active":"2024-09-11T07:26:31.425587+0000","last_became_peered":"2024-09-11T07:26:31.425587+0000","last_unstale":"2024-09-11T07:34:33.263257+0000","last_undegraded":"2024-09-11T07:34:33.263257+0000","last_fullsized":"2024-09-11T07:34:33.263257+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:09.772000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:09.772000+0000","last_clean_scrub_stamp":"2024-09-11T07:34:09.772000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:14.926483+0000","scrub_duration":0.0087789909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":183,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.259194+0000","last_change":"2024-09-11T07:34:04.660377+0000","last_active":"2024-09-11T07:34:33.259194+0000","last_peered":"2024-09-11T07:34:33.259194+0000","last_clean":"2024-09-11T07:34:33.259194+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:34:33.259194+0000","last_undegraded":"2024-09-11T07:34:33.259194+0000","last_fullsized":"2024-09-11T07:34:33.259194+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:04.660121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:04.660121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:12.877858+0000","scrub_duration":0.0047831310000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":190,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.254816+0000","last_change":"2024-09-11T07:33:53.667783+0000","last_active":"2024-09-11T07:34:33.254816+0000","last_peered":"2024-09-11T07:34:33.254816+0000","last_clean":"2024-09-11T07:34:33.254816+0000","last_became_active":"2024-09-11T07:26:37.568886+0000","last_became_peered":"2024-09-11T07:26:37.568886+0000","last_unstale":"2024-09-11T07:34:33.254816+0000","last_undegraded":"2024-09-11T07:34:33.254816+0000","last_fullsized":"2024-09-11T07:34:33.254816+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:53.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:53.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:59.734127+0000","scrub_duration":0.0039882809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"136'16","reported_seq":218,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.255893+0000","last_change":"2024-09-11T07:34:00.417426+0000","last_active":"2024-09-11T07:34:33.255893+0000","last_peered":"2024-09-11T07:34:33.255893+0000","last_clean":"2024-09-11T07:34:33.255893+0000","last_became_active":"2024-09-11T07:26:33.540208+0000","last_became_peered":"2024-09-11T07:26:33.540208+0000","last_unstale":"2024-09-11T07:34:33.255893+0000","last_undegraded":"2024-09-11T07:34:33.255893+0000","last_fullsized":"2024-09-11T07:34:33.255893+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'16","last_scrub_stamp":"2024-09-11T07:34:00.417343+0000","last_deep_scrub":"136'16","last_deep_scrub_stamp":"2024-09-11T07:31:06.039023+0000","last_clean_scrub_stamp":"2024-09-11T07:34:00.417343+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:12.571774+0000","scrub_duration":0.0026159130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":198,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.253226+0000","last_change":"2024-09-11T07:33:57.333274+0000","last_active":"2024-09-11T07:34:33.253226+0000","last_peered":"2024-09-11T07:34:33.253226+0000","last_clean":"2024-09-11T07:34:33.253226+0000","last_became_active":"2024-09-11T07:26:31.422671+0000","last_became_peered":"2024-09-11T07:26:31.422671+0000","last_unstale":"2024-09-11T07:34:33.253226+0000","last_undegraded":"2024-09-11T07:34:33.253226+0000","last_fullsized":"2024-09-11T07:34:33.253226+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:57.333166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:33:57.333166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:16.535083+0000","scrub_duration":0.010637648,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":184,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.253330+0000","last_change":"2024-09-11T07:33:51.251429+0000","last_active":"2024-09-11T07:34:33.253330+0000","last_peered":"2024-09-11T07:34:33.253330+0000","last_clean":"2024-09-11T07:34:33.253330+0000","last_became_active":"2024-09-11T07:26:35.566931+0000","last_became_peered":"2024-09-11T07:26:35.566931+0000","last_unstale":"2024-09-11T07:34:33.253330+0000","last_undegraded":"2024-09-11T07:34:33.253330+0000","last_fullsized":"2024-09-11T07:34:33.253330+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:51.251225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.847474+0000","last_clean_scrub_stamp":"2024-09-11T07:33:51.251225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:08.093660+0000","scrub_duration":0.0050645059999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":181,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.688678+0000","last_change":"2024-09-11T07:34:33.688599+0000","last_active":"2024-09-11T07:34:33.688678+0000","last_peered":"2024-09-11T07:34:33.688678+0000","last_clean":"2024-09-11T07:34:33.688678+0000","last_became_active":"2024-09-11T07:26:37.561573+0000","last_became_peered":"2024-09-11T07:26:37.561573+0000","last_unstale":"2024-09-11T07:34:33.688678+0000","last_undegraded":"2024-09-11T07:34:33.688678+0000","last_fullsized":"2024-09-11T07:34:33.688678+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:33.688496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:33.688496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:52.913118+0000","scrub_duration":0.0094425380000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"82'6","reported_seq":203,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:40.773894+0000","last_change":"2024-09-11T07:34:40.773406+0000","last_active":"2024-09-11T07:34:40.773894+0000","last_peered":"2024-09-11T07:34:40.773894+0000","last_clean":"2024-09-11T07:34:40.773894+0000","last_became_active":"2024-09-11T07:26:33.545830+0000","last_became_peered":"2024-09-11T07:26:33.545830+0000","last_unstale":"2024-09-11T07:34:40.773894+0000","last_undegraded":"2024-09-11T07:34:40.773894+0000","last_fullsized":"2024-09-11T07:34:40.773894+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'6","last_scrub_stamp":"2024-09-11T07:34:40.772239+0000","last_deep_scrub":"82'6","last_deep_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_clean_scrub_stamp":"2024-09-11T07:34:40.772239+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:00.092871+0000","scrub_duration":0.005304563,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":197,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:42.048613+0000","last_change":"2024-09-11T07:34:42.048593+0000","last_active":"2024-09-11T07:34:42.048613+0000","last_peered":"2024-09-11T07:34:42.048613+0000","last_clean":"2024-09-11T07:34:42.048613+0000","last_became_active":"2024-09-11T07:26:31.420518+0000","last_became_peered":"2024-09-11T07:26:31.420518+0000","last_unstale":"2024-09-11T07:34:42.048613+0000","last_undegraded":"2024-09-11T07:34:42.048613+0000","last_fullsized":"2024-09-11T07:34:42.048613+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:42.048544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_clean_scrub_stamp":"2024-09-11T07:34:42.048544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:53.207069+0000","scrub_duration":0.0012192539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":213,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:35.809840+0000","last_change":"2024-09-11T07:34:35.809784+0000","last_active":"2024-09-11T07:34:35.809840+0000","last_peered":"2024-09-11T07:34:35.809840+0000","last_clean":"2024-09-11T07:34:35.809840+0000","last_became_active":"2024-09-11T07:32:00.501969+0000","last_became_peered":"2024-09-11T07:32:00.501969+0000","last_unstale":"2024-09-11T07:34:35.809840+0000","last_undegraded":"2024-09-11T07:34:35.809840+0000","last_fullsized":"2024-09-11T07:34:35.809840+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:35.809630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:35.809630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:55.273618+0000","scrub_duration":0.0051714539999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":181,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.260018+0000","last_change":"2024-09-11T07:33:53.290535+0000","last_active":"2024-09-11T07:34:33.260018+0000","last_peered":"2024-09-11T07:34:33.260018+0000","last_clean":"2024-09-11T07:34:33.260018+0000","last_became_active":"2024-09-11T07:32:00.511407+0000","last_became_peered":"2024-09-11T07:32:00.511407+0000","last_unstale":"2024-09-11T07:34:33.260018+0000","last_undegraded":"2024-09-11T07:34:33.260018+0000","last_fullsized":"2024-09-11T07:34:33.260018+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:53.290379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:53.290379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:58.283671+0000","scrub_duration":0.002605291,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":211,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.259439+0000","last_change":"2024-09-11T07:33:03.304492+0000","last_active":"2024-09-11T07:34:33.259439+0000","last_peered":"2024-09-11T07:34:33.259439+0000","last_clean":"2024-09-11T07:34:33.259439+0000","last_became_active":"2024-09-11T07:32:00.502081+0000","last_became_peered":"2024-09-11T07:32:00.502081+0000","last_unstale":"2024-09-11T07:34:33.259439+0000","last_undegraded":"2024-09-11T07:34:33.259439+0000","last_fullsized":"2024-09-11T07:34:33.259439+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:33:03.304404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:03.304404+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023973169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":197,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.263611+0000","last_change":"2024-09-11T07:33:15.559485+0000","last_active":"2024-09-11T07:34:33.263611+0000","last_peered":"2024-09-11T07:34:33.263611+0000","last_clean":"2024-09-11T07:34:33.263611+0000","last_became_active":"2024-09-11T07:26:31.425876+0000","last_became_peered":"2024-09-11T07:26:31.425876+0000","last_unstale":"2024-09-11T07:34:33.263611+0000","last_undegraded":"2024-09-11T07:34:33.263611+0000","last_fullsized":"2024-09-11T07:34:33.263611+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:15.559426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:15.559426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:40.363895+0000","scrub_duration":0.0013828619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":164,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:43.214763+0000","last_change":"2024-09-11T07:34:43.214735+0000","last_active":"2024-09-11T07:34:43.214763+0000","last_peered":"2024-09-11T07:34:43.214763+0000","last_clean":"2024-09-11T07:34:43.214763+0000","last_became_active":"2024-09-11T07:32:00.520534+0000","last_became_peered":"2024-09-11T07:32:00.520534+0000","last_unstale":"2024-09-11T07:34:43.214763+0000","last_undegraded":"2024-09-11T07:34:43.214763+0000","last_fullsized":"2024-09-11T07:34:43.214763+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:43.214661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:39.021876+0000","last_clean_scrub_stamp":"2024-09-11T07:34:43.214661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:54.001779+0000","scrub_duration":0.0017089449999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":181,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:34.678738+0000","last_change":"2024-09-11T07:34:34.678649+0000","last_active":"2024-09-11T07:34:34.678738+0000","last_peered":"2024-09-11T07:34:34.678738+0000","last_clean":"2024-09-11T07:34:34.678738+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:34:34.678738+0000","last_undegraded":"2024-09-11T07:34:34.678738+0000","last_fullsized":"2024-09-11T07:34:34.678738+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:34.678233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:34.678233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:00.120684+0000","scrub_duration":0.0075729409999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":168,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:41.207870+0000","last_change":"2024-09-11T07:34:41.207758+0000","last_active":"2024-09-11T07:34:41.207870+0000","last_peered":"2024-09-11T07:34:41.207870+0000","last_clean":"2024-09-11T07:34:41.207870+0000","last_became_active":"2024-09-11T07:32:00.511258+0000","last_became_peered":"2024-09-11T07:32:00.511258+0000","last_unstale":"2024-09-11T07:34:41.207870+0000","last_undegraded":"2024-09-11T07:34:41.207870+0000","last_fullsized":"2024-09-11T07:34:41.207870+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:34:41.207591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:41.207591+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:09.601160+0000","scrub_duration":0.0027997759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":216,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.262205+0000","last_change":"2024-09-11T07:33:53.155118+0000","last_active":"2024-09-11T07:34:33.262205+0000","last_peered":"2024-09-11T07:34:33.262205+0000","last_clean":"2024-09-11T07:34:33.262205+0000","last_became_active":"2024-09-11T07:32:00.518307+0000","last_became_peered":"2024-09-11T07:32:00.518307+0000","last_unstale":"2024-09-11T07:34:33.262205+0000","last_undegraded":"2024-09-11T07:34:33.262205+0000","last_fullsized":"2024-09-11T07:34:33.262205+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:33:53.155024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:53.155024+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:20.182620+0000","scrub_duration":0.0038397209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":216,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.254418+0000","last_change":"2024-09-11T07:33:07.805709+0000","last_active":"2024-09-11T07:34:33.254418+0000","last_peered":"2024-09-11T07:34:33.254418+0000","last_clean":"2024-09-11T07:34:33.254418+0000","last_became_active":"2024-09-11T07:32:00.508233+0000","last_became_peered":"2024-09-11T07:32:00.508233+0000","last_unstale":"2024-09-11T07:34:33.254418+0000","last_undegraded":"2024-09-11T07:34:33.254418+0000","last_fullsized":"2024-09-11T07:34:33.254418+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:33:07.805635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:07.805635+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002375144,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"151'10","reported_seq":314,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-11T07:34:35.554224+0000","last_change":"2024-09-11T07:34:02.797981+0000","last_active":"2024-09-11T07:34:35.554224+0000","last_peered":"2024-09-11T07:34:35.554224+0000","last_clean":"2024-09-11T07:34:35.554224+0000","last_became_active":"2024-09-11T07:32:00.502049+0000","last_became_peered":"2024-09-11T07:32:00.502049+0000","last_unstale":"2024-09-11T07:34:35.554224+0000","last_undegraded":"2024-09-11T07:34:35.554224+0000","last_fullsized":"2024-09-11T07:34:35.554224+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:34:02.797889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:02.797889+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:19.482223+0000","scrub_duration":0.001718904,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":189,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.252874+0000","last_change":"2024-09-11T07:34:06.381128+0000","last_active":"2024-09-11T07:34:33.252874+0000","last_peered":"2024-09-11T07:34:33.252874+0000","last_clean":"2024-09-11T07:34:33.252874+0000","last_became_active":"2024-09-11T07:26:37.562501+0000","last_became_peered":"2024-09-11T07:26:37.562501+0000","last_unstale":"2024-09-11T07:34:33.252874+0000","last_undegraded":"2024-09-11T07:34:33.252874+0000","last_fullsized":"2024-09-11T07:34:33.252874+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:06.381043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:06.381043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:06.803518+0000","scrub_duration":0.0046834980000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":187,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.259757+0000","last_change":"2024-09-11T07:31:52.334717+0000","last_active":"2024-09-11T07:34:33.259757+0000","last_peered":"2024-09-11T07:34:33.259757+0000","last_clean":"2024-09-11T07:34:33.259757+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:34:33.259757+0000","last_undegraded":"2024-09-11T07:34:33.259757+0000","last_fullsized":"2024-09-11T07:34:33.259757+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:31:52.334517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:31:52.334517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029011779999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":268,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.263550+0000","last_change":"2024-09-11T07:34:11.760411+0000","last_active":"2024-09-11T07:34:33.263550+0000","last_peered":"2024-09-11T07:34:33.263550+0000","last_clean":"2024-09-11T07:34:33.263550+0000","last_became_active":"2024-09-11T07:30:28.599840+0000","last_became_peered":"2024-09-11T07:30:28.599840+0000","last_unstale":"2024-09-11T07:34:33.263550+0000","last_undegraded":"2024-09-11T07:34:33.263550+0000","last_fullsized":"2024-09-11T07:34:33.263550+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:34:11.760315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:34:11.760315+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:31.329350+0000","scrub_duration":0.007126177,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":199,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.254350+0000","last_change":"2024-09-11T07:33:13.776627+0000","last_active":"2024-09-11T07:34:33.254350+0000","last_peered":"2024-09-11T07:34:33.254350+0000","last_clean":"2024-09-11T07:34:33.254350+0000","last_became_active":"2024-09-11T07:32:00.507050+0000","last_became_peered":"2024-09-11T07:32:00.507050+0000","last_unstale":"2024-09-11T07:34:33.254350+0000","last_undegraded":"2024-09-11T07:34:33.254350+0000","last_fullsized":"2024-09-11T07:34:33.254350+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:13.776566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:24.575210+0000","last_clean_scrub_stamp":"2024-09-11T07:33:13.776566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:34:43.729373+0000","scrub_duration":0.001406963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":190,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.254868+0000","last_change":"2024-09-11T07:33:54.459273+0000","last_active":"2024-09-11T07:34:33.254868+0000","last_peered":"2024-09-11T07:34:33.254868+0000","last_clean":"2024-09-11T07:34:33.254868+0000","last_became_active":"2024-09-11T07:26:37.862373+0000","last_became_peered":"2024-09-11T07:26:37.862373+0000","last_unstale":"2024-09-11T07:34:33.254868+0000","last_undegraded":"2024-09-11T07:34:33.254868+0000","last_fullsized":"2024-09-11T07:34:33.254868+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:54.459223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:54.459223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:00.855517+0000","scrub_duration":0.001318018,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":217,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.263087+0000","last_change":"2024-09-11T07:34:06.757693+0000","last_active":"2024-09-11T07:34:33.263087+0000","last_peered":"2024-09-11T07:34:33.263087+0000","last_clean":"2024-09-11T07:34:33.263087+0000","last_became_active":"2024-09-11T07:32:00.503749+0000","last_became_peered":"2024-09-11T07:32:00.503749+0000","last_unstale":"2024-09-11T07:34:33.263087+0000","last_undegraded":"2024-09-11T07:34:33.263087+0000","last_fullsized":"2024-09-11T07:34:33.263087+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:06.757621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:06.757621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:18.674821+0000","scrub_duration":0.0015661609999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":220,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.261098+0000","last_change":"2024-09-11T07:33:55.212599+0000","last_active":"2024-09-11T07:34:33.261098+0000","last_peered":"2024-09-11T07:34:33.261098+0000","last_clean":"2024-09-11T07:34:33.261098+0000","last_became_active":"2024-09-11T07:26:33.542544+0000","last_became_peered":"2024-09-11T07:26:33.542544+0000","last_unstale":"2024-09-11T07:34:33.261098+0000","last_undegraded":"2024-09-11T07:34:33.261098+0000","last_fullsized":"2024-09-11T07:34:33.261098+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-09-11T07:33:55.212499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:55.212499+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:23.471681+0000","scrub_duration":0.004594473,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":191,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.253100+0000","last_change":"2024-09-11T07:34:01.366655+0000","last_active":"2024-09-11T07:34:33.253100+0000","last_peered":"2024-09-11T07:34:33.253100+0000","last_clean":"2024-09-11T07:34:33.253100+0000","last_became_active":"2024-09-11T07:26:35.555862+0000","last_became_peered":"2024-09-11T07:26:35.555862+0000","last_unstale":"2024-09-11T07:34:33.253100+0000","last_undegraded":"2024-09-11T07:34:33.253100+0000","last_fullsized":"2024-09-11T07:34:33.253100+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:01.366571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:01.366571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:19.675146+0000","scrub_duration":0.0059672780000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":181,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.259529+0000","last_change":"2024-09-11T07:33:17.489228+0000","last_active":"2024-09-11T07:34:33.259529+0000","last_peered":"2024-09-11T07:34:33.259529+0000","last_clean":"2024-09-11T07:34:33.259529+0000","last_became_active":"2024-09-11T07:26:37.861647+0000","last_became_peered":"2024-09-11T07:26:37.861647+0000","last_unstale":"2024-09-11T07:34:33.259529+0000","last_undegraded":"2024-09-11T07:34:33.259529+0000","last_fullsized":"2024-09-11T07:34:33.259529+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:17.489174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:17.489174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029380980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":204,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.259370+0000","last_change":"2024-09-11T07:33:09.499435+0000","last_active":"2024-09-11T07:34:33.259370+0000","last_peered":"2024-09-11T07:34:33.259370+0000","last_clean":"2024-09-11T07:34:33.259370+0000","last_became_active":"2024-09-11T07:32:00.502947+0000","last_became_peered":"2024-09-11T07:32:00.502947+0000","last_unstale":"2024-09-11T07:34:33.259370+0000","last_undegraded":"2024-09-11T07:34:33.259370+0000","last_fullsized":"2024-09-11T07:34:33.259370+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:09.499379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:09.499379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001485933,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"127'17","reported_seq":213,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.258736+0000","last_change":"2024-09-11T07:33:50.604574+0000","last_active":"2024-09-11T07:34:33.258736+0000","last_peered":"2024-09-11T07:34:33.258736+0000","last_clean":"2024-09-11T07:34:33.258736+0000","last_became_active":"2024-09-11T07:26:33.542711+0000","last_became_peered":"2024-09-11T07:26:33.542711+0000","last_unstale":"2024-09-11T07:34:33.258736+0000","last_undegraded":"2024-09-11T07:34:33.258736+0000","last_fullsized":"2024-09-11T07:34:33.258736+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'17","last_scrub_stamp":"2024-09-11T07:33:50.604493+0000","last_deep_scrub":"127'17","last_deep_scrub_stamp":"2024-09-11T07:30:30.260552+0000","last_clean_scrub_stamp":"2024-09-11T07:33:50.604493+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:03.711168+0000","scrub_duration":0.0045862999999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":192,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-11T07:34:33.254969+0000","last_change":"2024-09-11T07:34:07.375537+0000","last_active":"2024-09-11T07:34:33.254969+0000","last_peered":"2024-09-11T07:34:33.254969+0000","last_clean":"2024-09-11T07:34:33.254969+0000","last_became_active":"2024-09-11T07:26:35.562037+0000","last_became_peered":"2024-09-11T07:26:35.562037+0000","last_unstale":"2024-09-11T07:34:33.254969+0000","last_undegraded":"2024-09-11T07:34:33.254969+0000","last_fullsized":"2024-09-11T07:34:33.254969+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:07.375490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:07.375490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:16.626440+0000","scrub_duration":0.0042989400000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":6628179968,"num_objects":101138,"num_object_clones":0,"num_object_copies":303414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101138,"num_write_kb":6472832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19872219136,"data_stored":19872219136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85438,"ondisk_log_size":85438,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":798,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":474,"ondisk_log_size":474,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":485,"num_read_kb":480,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":147,"seq":631360192546,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2321704,"kb_used_data":1993056,"kb_used_omap":4,"kb_used_meta":328635,"kb_avail":91423448,"statfs":{"total":95995035648,"available":93617610752,"internally_reserved":0,"allocated":2040889344,"data_stored":2040318387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":336522758},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":3.2189999999999999,"15min":3.617},"min":{"1min":0.246,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":2.7250000000000001,"5min":49.573,"15min":49.573},"last":3.4140000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":2.9049999999999998,"15min":3.2869999999999999},"min":{"1min":0.311,"5min":0.22,"15min":0.22},"max":{"1min":2.1299999999999999,"5min":49.881,"15min":49.881},"last":0.48599999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.65100000000000002,"15min":0.61499999999999999},"min":{"1min":0.29399999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.677,"5min":3.677,"15min":3.677},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.78200000000000003,"15min":0.79600000000000004},"min":{"1min":0.26000000000000001,"5min":0.246,"15min":0.246},"max":{"1min":1.919,"5min":4.6740000000000004,"15min":4.6740000000000004},"last":0.437}]},{"osd":2,"last update":"Wed Sep 11 07:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.92900000000000005,"15min":0.96799999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.141,"5min":3.6579999999999999,"15min":3.6579999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.90300000000000002,"15min":0.91300000000000003},"min":{"1min":0.318,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.141,"5min":5.0149999999999997,"15min":5.0149999999999997},"last":0.374}]},{"osd":3,"last update":"Wed Sep 11 07:34:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":3.0920000000000001,"15min":3.4300000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":4.4580000000000002,"5min":49.798999999999999,"15min":49.798999999999999},"last":3.2749999999999999},{"interface":"front","average":{"1min":1.4570000000000001,"5min":3.0619999999999998,"15min":3.3300000000000001},"min":{"1min":0.32800000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":15.316000000000001,"5min":49.524000000000001,"15min":49.524000000000001},"last":0.45300000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.42299999999999999,"15min":0.41799999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.64200000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.62,"15min":0.60799999999999998},"min":{"1min":0.33800000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.9340000000000002,"5min":3.9340000000000002,"15min":3.9340000000000002},"last":0.46899999999999997}]},{"osd":5,"last update":"Wed Sep 11 07:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.46200000000000002,"15min":0.44700000000000001},"min":{"1min":0.32300000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.64500000000000002,"15min":0.65300000000000002},"min":{"1min":0.33400000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.111,"5min":3.5640000000000001,"15min":3.5640000000000001},"last":0.503}]},{"osd":6,"last update":"Wed Sep 11 07:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.54000000000000004,"15min":0.52000000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.55800000000000005,"15min":0.54400000000000004},"min":{"1min":0.35399999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.29099999999999998}]}]},{"osd":6,"up_from":44,"seq":188978561131,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2298260,"kb_used_data":2000908,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":91446892,"statfs":{"total":95995035648,"available":93641617408,"internally_reserved":0,"allocated":2048929792,"data_stored":2048379548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:34:07 2024","interfaces":[{"interface":"back","average":{"1min":7.5389999999999997,"5min":3.7440000000000002,"15min":1.5680000000000001},"min":{"1min":0.26500000000000001,"5min":0.192,"15min":0.192},"max":{"1min":142.29599999999999,"5min":142.29599999999999,"15min":142.29599999999999},"last":2.3340000000000001},{"interface":"front","average":{"1min":7.5039999999999996,"5min":3.819,"15min":1.6799999999999999},"min":{"1min":0.39800000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":142.68799999999999,"5min":142.68799999999999,"15min":142.68799999999999},"last":0.84199999999999997}]},{"osd":1,"last update":"Wed Sep 11 07:34:07 2024","interfaces":[{"interface":"back","average":{"1min":6.8109999999999999,"5min":2.0009999999999999,"15min":1.0629999999999999},"min":{"1min":0.317,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":127.92100000000001,"5min":127.92100000000001,"15min":127.92100000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":6.8289999999999997,"5min":2.226,"15min":1.0740000000000001},"min":{"1min":0.24099999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":127.746,"5min":127.746,"15min":127.746},"last":0.71799999999999997}]},{"osd":2,"last update":"Wed Sep 11 07:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":1.1160000000000001,"15min":0.78300000000000003},"min":{"1min":0.29399999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.748,"5min":26.491,"15min":26.491},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":1.355,"15min":0.84999999999999998},"min":{"1min":0.36399999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.7969999999999999,"5min":26.561,"15min":26.561},"last":2.6669999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":1.399,"15min":0.81100000000000005},"min":{"1min":0.27600000000000002,"5min":0.254,"15min":0.254},"max":{"1min":2.1970000000000001,"5min":31.268999999999998,"15min":31.268999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":1.718,"15min":0.998},"min":{"1min":0.34100000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.6479999999999999,"5min":31.872,"15min":31.872},"last":0.42499999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.439,"15min":0.53700000000000003},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":4.4720000000000004},"last":0.998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.435,"15min":0.42699999999999999},"min":{"1min":0.27700000000000002,"5min":0.25,"15min":0.25},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.623}]},{"osd":5,"last update":"Wed Sep 11 07:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.53600000000000003,"15min":0.61899999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":2.9809999999999999,"5min":2.9809999999999999,"15min":5.7329999999999997},"last":8.891},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.33500000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":3.4129999999999998,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":14.597}]},{"osd":7,"last update":"Wed Sep 11 07:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.309,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.514,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.70499999999999996,"15min":0.71699999999999997},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":2.9449999999999998,"5min":3.6030000000000002,"15min":3.6030000000000002},"last":0.51500000000000001}]}]},{"osd":5,"up_from":37,"seq":158913790065,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3488164,"kb_used_data":3186652,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":90256988,"statfs":{"total":95995035648,"available":92423155712,"internally_reserved":0,"allocated":3263131648,"data_stored":3262565644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:34:41 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":4.4749999999999996,"15min":1.837},"min":{"1min":0.215,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":10.342000000000001,"5min":185.68199999999999,"15min":185.68199999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":1.8140000000000001,"5min":4.7720000000000002,"15min":1.9390000000000001},"min":{"1min":0.254,"5min":0.218,"15min":0.218},"max":{"1min":10.355,"5min":186.12200000000001,"15min":186.12200000000001},"last":0.92400000000000004}]},{"osd":1,"last update":"Wed Sep 11 07:34:41 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.0449999999999999,"15min":0.70299999999999996},"min":{"1min":0.23799999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":5.8550000000000004,"5min":9.0969999999999995,"15min":9.0969999999999995},"last":0.35599999999999998},{"interface":"front","average":{"1min":1.458,"5min":1.2190000000000001,"15min":0.749},"min":{"1min":0.28899999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":5.8689999999999998,"5min":19.119,"15min":19.119},"last":0.505}]},{"osd":2,"last update":"Wed Sep 11 07:34:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":3.3109999999999999,"15min":1.4650000000000001},"min":{"1min":0.312,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":3.8050000000000002,"5min":187.99199999999999,"15min":187.99199999999999},"last":2.3119999999999998},{"interface":"front","average":{"1min":0.90700000000000003,"5min":3.2210000000000001,"15min":1.4419999999999999},"min":{"1min":0.27600000000000002,"5min":0.254,"15min":0.254},"max":{"1min":2.778,"5min":186.375,"15min":186.375},"last":1.8759999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:34:41 2024","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":4.6150000000000002,"15min":1.9430000000000001},"min":{"1min":0.26400000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":4.4790000000000001,"5min":188.11500000000001,"15min":188.11500000000001},"last":0.373},{"interface":"front","average":{"1min":1.236,"5min":4.6020000000000003,"15min":1.9199999999999999},"min":{"1min":0.39400000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":5.3949999999999996,"5min":186.44200000000001,"15min":186.44200000000001},"last":0.59399999999999997}]},{"osd":4,"last update":"Wed Sep 11 07:34:41 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.53500000000000003,"15min":0.42799999999999999},"min":{"1min":0.253,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":4.7060000000000004,"5min":4.7060000000000004,"15min":4.7060000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.50900000000000001,"15min":0.44900000000000001},"min":{"1min":0.29099999999999998,"5min":0.253,"15min":0.23200000000000001},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":0.40999999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:34:08 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.503,"15min":0.44400000000000001},"min":{"1min":0.32300000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.784,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.51700000000000002,"15min":0.45800000000000002},"min":{"1min":0.28699999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":2.0369999999999999,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":0.42399999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.772,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61299999999999999,"15min":0.60599999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":2.226,"5min":2.226,"15min":2.226},"last":0.435}]}]},{"osd":4,"up_from":31,"seq":133143986293,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2300520,"kb_used_data":1999008,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":91444632,"statfs":{"total":95995035648,"available":93639303168,"internally_reserved":0,"allocated":2046984192,"data_stored":2046413935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:34:32 2024","interfaces":[{"interface":"back","average":{"1min":2.391,"5min":1.968,"15min":1.0069999999999999},"min":{"1min":0.27700000000000002,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":32.892000000000003,"5min":35.308,"15min":35.308},"last":1.0940000000000001},{"interface":"front","average":{"1min":2.2749999999999999,"5min":1.782,"15min":0.92800000000000005},"min":{"1min":0.30599999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":32.860999999999997,"5min":35.597999999999999,"15min":35.597999999999999},"last":0.434}]},{"osd":1,"last update":"Wed Sep 11 07:34:32 2024","interfaces":[{"interface":"back","average":{"1min":3.0739999999999998,"5min":1.617,"15min":0.875},"min":{"1min":0.23599999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":24.581,"5min":24.581,"15min":24.581},"last":0.29199999999999998},{"interface":"front","average":{"1min":2.1749999999999998,"5min":1.032,"15min":0.80700000000000005},"min":{"1min":0.26700000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":24.652000000000001,"5min":24.652000000000001,"15min":24.652000000000001},"last":2.327}]},{"osd":2,"last update":"Wed Sep 11 07:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":1.369,"15min":0.81000000000000005},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.22600000000000001},"max":{"1min":2.9380000000000002,"5min":19.927,"15min":19.927},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.286,"15min":0.77700000000000002},"min":{"1min":0.27200000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":4.4080000000000004,"5min":20.02,"15min":20.02},"last":0.92100000000000004}]},{"osd":3,"last update":"Wed Sep 11 07:34:32 2024","interfaces":[{"interface":"back","average":{"1min":2.3690000000000002,"5min":1.774,"15min":0.96599999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":24.029,"5min":24.029,"15min":24.029},"last":0.40899999999999997},{"interface":"front","average":{"1min":1.97,"5min":1.5660000000000001,"15min":0.86099999999999999},"min":{"1min":0.32100000000000001,"5min":0.309,"15min":0.28699999999999998},"max":{"1min":24.265000000000001,"5min":24.265000000000001,"15min":24.265000000000001},"last":0.94799999999999995}]},{"osd":5,"last update":"Wed Sep 11 07:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.40200000000000002,"15min":0.36699999999999999},"min":{"1min":0.219,"5min":0.216,"15min":0.216},"max":{"1min":0.747,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.41699999999999998,"15min":0.39300000000000002},"min":{"1min":0.25,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.39300000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:34:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":0.57799999999999996,"15min":0.47599999999999998},"min":{"1min":0.23999999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":7.8650000000000002,"5min":7.8650000000000002,"15min":7.8650000000000002},"last":0.42199999999999999},{"interface":"front","average":{"1min":1.1279999999999999,"5min":0.57899999999999996,"15min":0.48699999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":9.3770000000000007,"5min":9.3770000000000007,"15min":9.3770000000000007},"last":0.45400000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55900000000000005,"15min":0.55000000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":0.63},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.57099999999999995,"15min":0.55600000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":2.4020000000000001,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":0.33200000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182522,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2720624,"kb_used_data":2410792,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":91024528,"statfs":{"total":95995035648,"available":93209116672,"internally_reserved":0,"allocated":2468651008,"data_stored":2468072023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":2.202,"15min":0.99099999999999999},"min":{"1min":0.21199999999999999,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":2.4740000000000002,"5min":38.432000000000002,"15min":38.432000000000002},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":3.044,"15min":1.2989999999999999},"min":{"1min":0.21099999999999999,"5min":0.20300000000000001,"15min":0.19500000000000001},"max":{"1min":2.7650000000000001,"5min":91.876999999999995,"15min":91.876999999999995},"last":0.65600000000000003}]},{"osd":1,"last update":"Wed Sep 11 07:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":1.831,"15min":0.89600000000000002},"min":{"1min":0.23499999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":2.8170000000000002,"5min":10.475,"15min":10.475},"last":1.3839999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":1.4710000000000001,"15min":0.76900000000000002},"min":{"1min":0.25,"5min":0.215,"15min":0.215},"max":{"1min":1.998,"5min":8.1899999999999995,"15min":8.1899999999999995},"last":0.502}]},{"osd":2,"last update":"Wed Sep 11 07:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":1.843,"15min":0.91600000000000004},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.23200000000000001},"max":{"1min":1.9650000000000001,"5min":8.282,"15min":8.282},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":1.659,"15min":0.85799999999999998},"min":{"1min":0.26200000000000001,"5min":0.21199999999999999,"15min":0.20999999999999999},"max":{"1min":2.569,"5min":8.7110000000000003,"15min":8.7110000000000003},"last":0.32500000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:34:27 2024","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.371,"15min":0.89300000000000002},"min":{"1min":0.29599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":3.2389999999999999,"5min":5.7670000000000003,"15min":21.856000000000002},"last":0.36399999999999999},{"interface":"front","average":{"1min":1.018,"5min":1.468,"15min":0.99399999999999999},"min":{"1min":0.36599999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":2.8479999999999999,"5min":8.0709999999999997,"15min":19.146000000000001},"last":0.83999999999999997}]},{"osd":5,"last update":"Wed Sep 11 07:34:43 2024","interfaces":[{"interface":"back","average":{"1min":1.6259999999999999,"5min":1.7,"15min":0.95799999999999996},"min":{"1min":0.377,"5min":0.30599999999999999,"15min":0.29599999999999999},"max":{"1min":5.0209999999999999,"5min":32.325000000000003,"15min":32.325000000000003},"last":0.40999999999999998},{"interface":"front","average":{"1min":1.841,"5min":1.617,"15min":1.006},"min":{"1min":0.54600000000000004,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":4.8490000000000002,"5min":7.9870000000000001,"15min":18.416},"last":0.89800000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":1.617,"15min":1.4119999999999999},"min":{"1min":0.34899999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":2.5590000000000002,"5min":32.350999999999999,"15min":32.350999999999999},"last":0.443},{"interface":"front","average":{"1min":0.82599999999999996,"5min":1.631,"15min":1.4239999999999999},"min":{"1min":0.42399999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":2.609,"5min":32.402999999999999,"15min":32.402999999999999},"last":0.38700000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":1.2689999999999999,"15min":1.349},"min":{"1min":0.44600000000000001,"5min":0.308,"15min":0.308},"max":{"1min":2.8759999999999999,"5min":8.0429999999999993,"15min":8.0429999999999993},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":2.1659999999999999,"15min":2.3929999999999998},"min":{"1min":0.39400000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.5209999999999999,"5min":32.289999999999999,"15min":32.289999999999999},"last":0.47399999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411455,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2725328,"kb_used_data":2427976,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":91019824,"statfs":{"total":95995035648,"available":93204299776,"internally_reserved":0,"allocated":2486247424,"data_stored":2485700859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:34:41 2024","interfaces":[{"interface":"back","average":{"1min":3.5289999999999999,"5min":2.8010000000000002,"15min":1.23},"min":{"1min":0.23000000000000001,"5min":0.183,"15min":0.17000000000000001},"max":{"1min":43.151000000000003,"5min":56.704000000000001,"15min":56.704000000000001},"last":2.0329999999999999},{"interface":"front","average":{"1min":3.5939999999999999,"5min":2.8690000000000002,"15min":1.248},"min":{"1min":0.32600000000000001,"5min":0.216,"15min":0.216},"max":{"1min":43.213999999999999,"5min":56.816000000000003,"15min":56.816000000000003},"last":0.78000000000000003}]},{"osd":1,"last update":"Wed Sep 11 07:34:41 2024","interfaces":[{"interface":"back","average":{"1min":4.468,"5min":2.3679999999999999,"15min":1.0660000000000001},"min":{"1min":0.374,"5min":0.24299999999999999,"15min":0.23200000000000001},"max":{"1min":65.153999999999996,"5min":65.153999999999996,"15min":65.153999999999996},"last":2.75},{"interface":"front","average":{"1min":4.96,"5min":2.5489999999999999,"15min":1.143},"min":{"1min":0.35299999999999998,"5min":0.216,"15min":0.20699999999999999},"max":{"1min":63.311999999999998,"5min":63.311999999999998,"15min":63.311999999999998},"last":4.6440000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:34:11 2024","interfaces":[{"interface":"back","average":{"1min":2.8420000000000001,"5min":2.4980000000000002,"15min":1.1160000000000001},"min":{"1min":0.33200000000000002,"5min":0.27400000000000002,"15min":0.26900000000000002},"max":{"1min":44.405999999999999,"5min":44.405999999999999,"15min":44.405999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":3.1960000000000002,"5min":2.649,"15min":1.1839999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":43.295999999999999,"5min":43.295999999999999,"15min":43.295999999999999},"last":2.0960000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:34:27 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":1.8009999999999999,"15min":0.97099999999999997},"min":{"1min":0.32300000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":2.71,"5min":30.946999999999999,"15min":30.946999999999999},"last":2.1190000000000002},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.488,"15min":0.88},"min":{"1min":0.31900000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.5339999999999998,"5min":22.434000000000001,"15min":22.434000000000001},"last":0.52400000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":1.7030000000000001,"15min":0.95399999999999996},"min":{"1min":0.38300000000000001,"5min":0.34000000000000002,"15min":0.31900000000000001},"max":{"1min":0.70899999999999996,"5min":31.789000000000001,"15min":31.789000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":1.6990000000000001,"15min":1.022},"min":{"1min":0.371,"5min":0.31,"15min":0.31},"max":{"1min":0.73199999999999998,"5min":30.898,"15min":30.898},"last":2.069}]},{"osd":6,"last update":"Wed Sep 11 07:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":1.6459999999999999,"15min":0.96799999999999997},"min":{"1min":0.432,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.7030000000000001,"5min":22.399000000000001,"15min":22.399000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":1.5940000000000001,"15min":0.94999999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.714,"5min":25.739000000000001,"15min":25.739000000000001},"last":2.1680000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":2.0139999999999998,"15min":2.226},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.6719999999999999,"5min":22.484000000000002,"15min":22.484000000000002},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":2.331,"15min":2.593},"min":{"1min":0.438,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.7850000000000001,"5min":25.670000000000002,"15min":25.670000000000002},"last":2.1419999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574978,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3119476,"kb_used_data":2817964,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":90625676,"statfs":{"total":95995035648,"available":92800692224,"internally_reserved":0,"allocated":2885595136,"data_stored":2885012869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:34:15 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":3.4580000000000002,"15min":1.379},"min":{"1min":0.248,"5min":0.192,"15min":0.186},"max":{"1min":8.0549999999999997,"5min":82.218999999999994,"15min":82.218999999999994},"last":0.29499999999999998},{"interface":"front","average":{"1min":1.5,"5min":3.4569999999999999,"15min":1.3939999999999999},"min":{"1min":0.22700000000000001,"5min":0.191,"15min":0.191},"max":{"1min":7.5679999999999996,"5min":82.278999999999996,"15min":82.278999999999996},"last":0.193}]},{"osd":2,"last update":"Wed Sep 11 07:33:40 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":1.6060000000000001,"15min":0.81000000000000005},"min":{"1min":0.27100000000000002,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":4.907,"5min":6.7610000000000001,"15min":6.7610000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":1.8580000000000001,"15min":0.89000000000000001},"min":{"1min":0.24199999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.633,"5min":8.9559999999999995,"15min":8.9559999999999995},"last":2.1589999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":3.4409999999999998,"15min":1.4399999999999999},"min":{"1min":0.28000000000000003,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":5.4690000000000003,"5min":81.882999999999996,"15min":81.882999999999996},"last":2.194},{"interface":"front","average":{"1min":1.0820000000000001,"5min":3.488,"15min":1.4370000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.20999999999999999},"max":{"1min":7.2809999999999997,"5min":82.400000000000006,"15min":82.400000000000006},"last":0.75}]},{"osd":4,"last update":"Wed Sep 11 07:34:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.1879999999999999,"15min":0.75800000000000001},"min":{"1min":0.36599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":7.3630000000000004,"5min":7.3630000000000004,"15min":7.3630000000000004},"last":0.65500000000000003},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.5189999999999999,"15min":0.878},"min":{"1min":0.34399999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.2960000000000003,"5min":8.9039999999999999,"15min":8.9039999999999999},"last":2.2170000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:33:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":1.159,"15min":0.753},"min":{"1min":0.38300000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.109,"5min":6.6909999999999998,"15min":6.6909999999999998},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":1.4099999999999999,"15min":0.84499999999999997},"min":{"1min":0.32400000000000001,"5min":0.308,"15min":0.308},"max":{"1min":1.758,"5min":8.8759999999999994,"15min":8.8759999999999994},"last":2.2400000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:34:05 2024","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.6839999999999999,"15min":0.94299999999999995},"min":{"1min":0.35799999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":5.3609999999999998,"5min":24.018000000000001,"15min":24.018000000000001},"last":3.0739999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":1.444,"15min":0.85099999999999998},"min":{"1min":0.42799999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.5430000000000001,"5min":23.681000000000001,"15min":23.681000000000001},"last":0.371}]},{"osd":7,"last update":"Wed Sep 11 07:34:05 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.3080000000000001,"15min":1.3380000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":7.0940000000000003,"5min":7.0940000000000003,"15min":7.0940000000000003},"last":2.5739999999999998},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.462,"15min":1.5089999999999999},"min":{"1min":0.45700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":7.4050000000000002,"5min":7.4050000000000002,"15min":7.4050000000000002},"last":0.313}]}]},{"osd":0,"up_from":8,"seq":34359738504,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2731684,"kb_used_data":2426012,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":91013468,"statfs":{"total":95995035648,"available":93197791232,"internally_reserved":0,"allocated":2484236288,"data_stored":2483669992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":1.5149999999999999,"15min":0.74399999999999999},"min":{"1min":0.21199999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":2.2229999999999999,"5min":9.7710000000000008,"15min":9.7710000000000008},"last":4.5389999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":1.585,"15min":0.76300000000000001},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":3.8769999999999998,"5min":10.275,"15min":10.275},"last":1.5329999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:34:44 2024","interfaces":[{"interface":"back","average":{"1min":1.657,"5min":2.1200000000000001,"15min":1.0049999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.20200000000000001},"max":{"1min":5.923,"5min":19.527999999999999,"15min":19.527999999999999},"last":5.2199999999999998},{"interface":"front","average":{"1min":1.401,"5min":2.008,"15min":0.98899999999999999},"min":{"1min":0.29799999999999999,"5min":0.22700000000000001,"15min":0.20999999999999999},"max":{"1min":5.0800000000000001,"5min":19.556999999999999,"15min":19.556999999999999},"last":0.29799999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:34:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":3.6000000000000001,"15min":1.506},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":4.359,"5min":45.185000000000002,"15min":45.185000000000002},"last":5.2489999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":2.927,"15min":1.2849999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":2.7400000000000002,"5min":43.228999999999999,"15min":43.228999999999999},"last":0.23999999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:34:22 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":1.401,"15min":0.84299999999999997},"min":{"1min":0.27200000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":2.613,"5min":19.411999999999999,"15min":19.411999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.768,"15min":0.99399999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":4.1669999999999998,"5min":19.396000000000001,"15min":19.396000000000001},"last":5.1310000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":1.54,"15min":0.94099999999999995},"min":{"1min":0.40899999999999997,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.843,"5min":19.381,"15min":19.381},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.9330000000000001,"15min":1.0660000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":5.0919999999999996,"5min":19.437000000000001,"15min":19.437000000000001},"last":5.0919999999999996}]},{"osd":6,"last update":"Wed Sep 11 07:34:13 2024","interfaces":[{"interface":"back","average":{"1min":2.048,"5min":1.9750000000000001,"15min":1.0509999999999999},"min":{"1min":0.442,"5min":0.316,"15min":0.316},"max":{"1min":23.190999999999999,"5min":23.190999999999999,"15min":23.190999999999999},"last":5.1630000000000003},{"interface":"front","average":{"1min":2.0819999999999999,"5min":1.984,"15min":1.0600000000000001},"min":{"1min":0.47299999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":23.632000000000001,"5min":23.632000000000001,"15min":23.632000000000001},"last":0.42799999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":1.641,"15min":1.7769999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":4.2350000000000003,"5min":5.3890000000000002,"15min":5.3890000000000002},"last":5.1920000000000002},{"interface":"front","average":{"1min":0.754,"5min":1.6919999999999999,"15min":1.8480000000000001},"min":{"1min":0.52500000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.2010000000000001,"5min":5.2759999999999998,"15min":5.2759999999999998},"last":0.46000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2503737344,"data_stored":2503737344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2910781440,"data_stored":2910781440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2492465152,"data_stored":2492465152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2491744256,"data_stored":2491744256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2060713984,"data_stored":2060713984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3297116160,"data_stored":3297116160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2056388608,"data_stored":2056388608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2059272192,"data_stored":2059272192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:34:48.904 INFO:tasks.thrashosds.thrasher:Setting 4.1e to [4, 5, 3] 2024-09-11T07:34:48.904 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '4.1e', '4', '5', '3'] 2024-09-11T07:34:48.904 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pg-upmap 4.1e 4 5 3 2024-09-11T07:34:48.987 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:49.012 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:49.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:49.013 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:49.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:48 smithi104 ceph-mon[33507]: 4.9 scrub starts 2024-09-11T07:34:49.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:48 smithi104 ceph-mon[33507]: 4.9 scrub ok 2024-09-11T07:34:49.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:48 smithi104 ceph-mon[33507]: 2.19 scrub starts 2024-09-11T07:34:49.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:48 smithi104 ceph-mon[33507]: 2.19 scrub ok 2024-09-11T07:34:49.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:48 smithi104 ceph-mon[33507]: 4.b scrub starts 2024-09-11T07:34:49.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:48 smithi104 ceph-mon[33507]: 4.b scrub ok 2024-09-11T07:34:49.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:48 smithi104 ceph-mon[33507]: pgmap v367: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 69 MiB/s wr, 1.10k op/s 2024-09-11T07:34:49.067 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:48 smithi022 ceph-mon[29462]: 4.9 scrub starts 2024-09-11T07:34:49.067 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:48 smithi022 ceph-mon[29462]: 4.9 scrub ok 2024-09-11T07:34:49.067 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:48 smithi022 ceph-mon[29462]: 2.19 scrub starts 2024-09-11T07:34:49.067 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:48 smithi022 ceph-mon[29462]: 2.19 scrub ok 2024-09-11T07:34:49.067 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:48 smithi022 ceph-mon[29462]: 4.b scrub starts 2024-09-11T07:34:49.067 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:48 smithi022 ceph-mon[29462]: 4.b scrub ok 2024-09-11T07:34:49.067 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:48 smithi022 ceph-mon[29462]: pgmap v367: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 69 MiB/s wr, 1.10k op/s 2024-09-11T07:34:49.068 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:48 smithi022 ceph-mon[37330]: 4.9 scrub starts 2024-09-11T07:34:49.068 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:48 smithi022 ceph-mon[37330]: 4.9 scrub ok 2024-09-11T07:34:49.068 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:48 smithi022 ceph-mon[37330]: 2.19 scrub starts 2024-09-11T07:34:49.068 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:48 smithi022 ceph-mon[37330]: 2.19 scrub ok 2024-09-11T07:34:49.068 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:48 smithi022 ceph-mon[37330]: 4.b scrub starts 2024-09-11T07:34:49.068 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:48 smithi022 ceph-mon[37330]: 4.b scrub ok 2024-09-11T07:34:49.068 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:48 smithi022 ceph-mon[37330]: pgmap v367: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 69 MiB/s wr, 1.10k op/s 2024-09-11T07:34:49.114 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:49.138 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:49.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:49.139 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:49.213 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 69 16 106800 106784 96.7051 72.875 0.00399224 0.0103345 2024-09-11T07:34:49.239 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:49.261 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:49.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:49.262 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:49.363 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:49.400 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:49.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:49.401 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:49.502 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:49.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:49.542 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:49.542 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:49.643 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:49.678 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:49.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:49.679 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:49.723 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:34:49 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:34:49.779 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:49.822 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:49.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:49.823 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:49.923 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:49.949 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:49.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:49.951 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:50.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:49 smithi104 ceph-mon[33507]: 2.2 scrub starts 2024-09-11T07:34:50.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:49 smithi104 ceph-mon[33507]: 2.2 scrub ok 2024-09-11T07:34:50.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:49 smithi104 ceph-mon[33507]: 2.5 deep-scrub starts 2024-09-11T07:34:50.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:49 smithi104 ceph-mon[33507]: 2.5 deep-scrub ok 2024-09-11T07:34:50.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:49 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:50.049 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:49 smithi022 ceph-mon[29462]: 2.2 scrub starts 2024-09-11T07:34:50.049 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:49 smithi022 ceph-mon[29462]: 2.2 scrub ok 2024-09-11T07:34:50.049 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:49 smithi022 ceph-mon[29462]: 2.5 deep-scrub starts 2024-09-11T07:34:50.049 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:49 smithi022 ceph-mon[29462]: 2.5 deep-scrub ok 2024-09-11T07:34:50.049 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:49 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:50.050 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:49 smithi022 ceph-mon[37330]: 2.2 scrub starts 2024-09-11T07:34:50.050 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:49 smithi022 ceph-mon[37330]: 2.2 scrub ok 2024-09-11T07:34:50.050 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:49 smithi022 ceph-mon[37330]: 2.5 deep-scrub starts 2024-09-11T07:34:50.050 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:49 smithi022 ceph-mon[37330]: 2.5 deep-scrub ok 2024-09-11T07:34:50.050 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:49 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:34:50.052 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:50.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:50.094 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:50.094 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:50.195 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:50.213 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 70 16 107960 107944 96.3587 72.5 0.019193 0.0103732 2024-09-11T07:34:50.239 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:50.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:50.240 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:50.341 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:50.371 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:50.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:50.372 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:50.424 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T07:34:50.473 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:50.498 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:50.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:50.499 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:50.599 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:50.627 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:50.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:50.629 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:50.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:34:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:34:50] "GET /metrics HTTP/1.1" 200 224105 "" "Prometheus/2.43.0" 2024-09-11T07:34:50.730 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:50.762 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:50.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:50.763 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:50.863 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:50.893 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:50.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:50.895 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:50.995 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:51.022 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:51.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:51.023 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:51.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:50 smithi104 ceph-mon[33507]: pgmap v368: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 71 MiB/s wr, 1.13k op/s 2024-09-11T07:34:51.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:50 smithi104 ceph-mon[33507]: osdmap e207: 8 total, 8 up, 8 in 2024-09-11T07:34:51.035 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:50 smithi022 ceph-mon[29462]: pgmap v368: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 71 MiB/s wr, 1.13k op/s 2024-09-11T07:34:51.035 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:50 smithi022 ceph-mon[29462]: osdmap e207: 8 total, 8 up, 8 in 2024-09-11T07:34:51.036 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:50 smithi022 ceph-mon[37330]: pgmap v368: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 71 MiB/s wr, 1.13k op/s 2024-09-11T07:34:51.036 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:50 smithi022 ceph-mon[37330]: osdmap e207: 8 total, 8 up, 8 in 2024-09-11T07:34:51.124 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:51.148 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:51.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:51.149 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:51.213 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 71 16 109109 109093 96.013 71.8125 0.00749108 0.0104101 2024-09-11T07:34:51.250 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:51.274 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:51.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:51.275 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:51.376 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:51.391 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:34:51.453 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:51.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:51.454 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:51.457 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:34:51.458 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:34:51.554 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:51.580 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:51.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:51.581 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:51.682 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:51.684 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:34:51.684 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:34:51.712 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:51.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:51.713 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:51.717 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:34:51.717 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:34:51.750 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:34:51.750 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:34:51.783 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:34:51.783 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:34:51.814 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:51.816 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:34:51.816 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:34:51.831 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:34:51.843 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:51.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:51.844 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:51.851 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:51 smithi022 ceph-mon[29462]: 6.d scrub starts 2024-09-11T07:34:51.851 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:51 smithi022 ceph-mon[29462]: 6.d scrub ok 2024-09-11T07:34:51.851 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:51 smithi022 ceph-mon[29462]: 6.2 scrub starts 2024-09-11T07:34:51.851 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:51 smithi022 ceph-mon[29462]: 6.2 scrub ok 2024-09-11T07:34:51.851 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:51 smithi022 ceph-mon[29462]: 4.14 scrub starts 2024-09-11T07:34:51.851 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:51 smithi022 ceph-mon[29462]: 4.14 scrub ok 2024-09-11T07:34:51.852 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:51 smithi022 ceph-mon[29462]: 2.8 scrub starts 2024-09-11T07:34:51.852 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:51 smithi022 ceph-mon[29462]: 2.8 scrub ok 2024-09-11T07:34:51.852 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:51 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1016133593' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:34:51.852 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:51 smithi022 ceph-mon[37330]: 6.d scrub starts 2024-09-11T07:34:51.852 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:51 smithi022 ceph-mon[37330]: 6.d scrub ok 2024-09-11T07:34:51.852 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:51 smithi022 ceph-mon[37330]: 6.2 scrub starts 2024-09-11T07:34:51.852 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:51 smithi022 ceph-mon[37330]: 6.2 scrub ok 2024-09-11T07:34:51.852 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:51 smithi022 ceph-mon[37330]: 4.14 scrub starts 2024-09-11T07:34:51.852 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:51 smithi022 ceph-mon[37330]: 4.14 scrub ok 2024-09-11T07:34:51.852 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:51 smithi022 ceph-mon[37330]: 2.8 scrub starts 2024-09-11T07:34:51.853 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:51 smithi022 ceph-mon[37330]: 2.8 scrub ok 2024-09-11T07:34:51.853 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:51 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1016133593' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:34:51.853 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:34:51.853 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:34:51.888 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:34:51.888 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:34:51.921 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:34:51.921 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:34:51.945 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:51.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:51.998 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:51.998 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:52.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:51 smithi104 ceph-mon[33507]: 6.d scrub starts 2024-09-11T07:34:52.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:51 smithi104 ceph-mon[33507]: 6.d scrub ok 2024-09-11T07:34:52.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:51 smithi104 ceph-mon[33507]: 6.2 scrub starts 2024-09-11T07:34:52.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:51 smithi104 ceph-mon[33507]: 6.2 scrub ok 2024-09-11T07:34:52.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:51 smithi104 ceph-mon[33507]: 4.14 scrub starts 2024-09-11T07:34:52.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:51 smithi104 ceph-mon[33507]: 4.14 scrub ok 2024-09-11T07:34:52.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:51 smithi104 ceph-mon[33507]: 2.8 scrub starts 2024-09-11T07:34:52.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:51 smithi104 ceph-mon[33507]: 2.8 scrub ok 2024-09-11T07:34:52.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:51 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1016133593' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:34:52.098 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:52.137 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:52.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:52.138 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:52.213 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 72 15 110225 110210 95.6491 69.8125 0.0142902 0.0104502 2024-09-11T07:34:52.239 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:52.275 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:52.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:52.277 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:52.377 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:52.407 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:52.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:52.408 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:52.508 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:52.549 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:52.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:52.550 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:52.651 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:52.678 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:52.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:52.679 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:52.780 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:52.815 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:52.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:52.816 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:52.917 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:52.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:52 smithi022 ceph-mon[29462]: pgmap v370: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-11T07:34:52.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:52 smithi022 ceph-mon[37330]: pgmap v370: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-11T07:34:52.950 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:52.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:52.951 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:53.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:52 smithi104 ceph-mon[33507]: pgmap v370: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-11T07:34:53.052 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:53.092 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:53.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:53.093 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:53.193 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:53.213 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 73 16 111384 111368 95.3299 72.375 0.0283322 0.0104844 2024-09-11T07:34:53.234 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:53.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:53.235 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:53.336 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:53.372 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:53.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:53.373 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:53.474 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:53.515 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:53.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:53.516 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:53.616 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:53.659 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:53.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:53.660 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:53.761 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:53.806 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:53.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:53.807 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:53.867 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-11T07:34:53.908 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:53.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:53.957 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:53.957 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:54.058 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:54.098 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:54.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:54.099 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:54.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:54 smithi022 ceph-mon[29462]: 8.3 scrub starts 2024-09-11T07:34:54.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:54 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:34:54.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:54 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:34:54.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:54 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:34:54.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:54 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:34:54.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:54 smithi022 ceph-mon[29462]: osdmap e208: 8 total, 8 up, 8 in 2024-09-11T07:34:54.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:54 smithi022 ceph-mon[37330]: 8.3 scrub starts 2024-09-11T07:34:54.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:54 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:34:54.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:54 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:34:54.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:54 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:34:54.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:54 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:34:54.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:54 smithi022 ceph-mon[37330]: osdmap e208: 8 total, 8 up, 8 in 2024-09-11T07:34:54.200 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:54.215 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 74 16 112483 112467 94.968 68.6875 0.0117545 0.0105253 2024-09-11T07:34:54.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:54.240 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:54.240 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:54.279 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:34:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:34:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:34:54.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:54 smithi104 ceph-mon[33507]: 8.3 scrub starts 2024-09-11T07:34:54.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:54 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:34:54.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:54 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:34:54.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:54 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:34:54.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:54 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:34:54.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:54 smithi104 ceph-mon[33507]: osdmap e208: 8 total, 8 up, 8 in 2024-09-11T07:34:54.341 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:54.391 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:54.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:54.392 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:54.493 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:54.532 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:54.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:54.533 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:54.634 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:54.669 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:54.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:54.670 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:54.708 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:34:54.771 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:54.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:54.830 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:54.830 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:54.931 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:54.956 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:54.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:54.957 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:55.034 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:55 smithi022 ceph-mon[37330]: 8.9 scrub starts 2024-09-11T07:34:55.034 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:55 smithi022 ceph-mon[37330]: 4.1e scrub starts 2024-09-11T07:34:55.034 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:55 smithi022 ceph-mon[37330]: 4.1e scrub ok 2024-09-11T07:34:55.034 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:55 smithi022 ceph-mon[37330]: pgmap v372: 148 pgs: 148 active+clean; 6.6 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 691 B/s rd, 60 MiB/s wr, 962 op/s 2024-09-11T07:34:55.057 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:55.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:55.096 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:55.096 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:55.197 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:55.215 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 75 16 113452 113436 94.5093 60.5625 0.0180213 0.0105751 2024-09-11T07:34:55.235 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:55.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:55.236 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:55.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:55 smithi104 ceph-mon[33507]: 8.9 scrub starts 2024-09-11T07:34:55.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:55 smithi104 ceph-mon[33507]: 4.1e scrub starts 2024-09-11T07:34:55.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:55 smithi104 ceph-mon[33507]: 4.1e scrub ok 2024-09-11T07:34:55.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:55 smithi104 ceph-mon[33507]: pgmap v372: 148 pgs: 148 active+clean; 6.6 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 691 B/s rd, 60 MiB/s wr, 962 op/s 2024-09-11T07:34:55.338 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:55.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:55.377 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:55.377 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:55.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:55 smithi022 ceph-mon[29462]: 8.9 scrub starts 2024-09-11T07:34:55.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:55 smithi022 ceph-mon[29462]: 4.1e scrub starts 2024-09-11T07:34:55.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:55 smithi022 ceph-mon[29462]: 4.1e scrub ok 2024-09-11T07:34:55.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:55 smithi022 ceph-mon[29462]: pgmap v372: 148 pgs: 148 active+clean; 6.6 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 691 B/s rd, 60 MiB/s wr, 962 op/s 2024-09-11T07:34:55.478 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:55.508 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:55.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:55.509 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:55.610 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:55.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:55.644 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:55.644 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:55.745 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:55.782 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:55.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:55.783 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:55.884 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:55.925 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:34:55.931 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:55.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:55.933 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:56.033 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:56.069 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:56.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:56.070 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:56.109 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:56 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:34:56.110 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:56 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:34:56.110 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:56 smithi022 ceph-mon[29462]: 6.b scrub starts 2024-09-11T07:34:56.110 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:56 smithi022 ceph-mon[29462]: 6.b scrub ok 2024-09-11T07:34:56.110 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:56 smithi022 ceph-mon[29462]: 8.6 deep-scrub starts 2024-09-11T07:34:56.110 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:56 smithi022 ceph-mon[29462]: osdmap e209: 8 total, 8 up, 8 in 2024-09-11T07:34:56.110 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:56 smithi022 ceph-mon[29462]: 6.7 scrub starts 2024-09-11T07:34:56.110 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:56 smithi022 ceph-mon[29462]: 6.7 scrub ok 2024-09-11T07:34:56.110 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:56 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:34:56.110 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:56 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:34:56.110 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:56 smithi022 ceph-mon[37330]: 6.b scrub starts 2024-09-11T07:34:56.111 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:56 smithi022 ceph-mon[37330]: 6.b scrub ok 2024-09-11T07:34:56.111 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:56 smithi022 ceph-mon[37330]: 8.6 deep-scrub starts 2024-09-11T07:34:56.111 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:56 smithi022 ceph-mon[37330]: osdmap e209: 8 total, 8 up, 8 in 2024-09-11T07:34:56.111 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:56 smithi022 ceph-mon[37330]: 6.7 scrub starts 2024-09-11T07:34:56.111 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:56 smithi022 ceph-mon[37330]: 6.7 scrub ok 2024-09-11T07:34:56.170 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:56.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:56.205 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:56.205 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:56.215 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 76 16 114359 114343 94.0116 56.6875 0.00559342 0.0106314 2024-09-11T07:34:56.306 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:56.351 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:56.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:56.352 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:56.453 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:56.505 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:56.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:56.506 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:56.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:56 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:34:56.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:56 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:34:56.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:56 smithi104 ceph-mon[33507]: 6.b scrub starts 2024-09-11T07:34:56.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:56 smithi104 ceph-mon[33507]: 6.b scrub ok 2024-09-11T07:34:56.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:56 smithi104 ceph-mon[33507]: 8.6 deep-scrub starts 2024-09-11T07:34:56.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:56 smithi104 ceph-mon[33507]: osdmap e209: 8 total, 8 up, 8 in 2024-09-11T07:34:56.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:56 smithi104 ceph-mon[33507]: 6.7 scrub starts 2024-09-11T07:34:56.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:56 smithi104 ceph-mon[33507]: 6.7 scrub ok 2024-09-11T07:34:56.607 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:56.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:56.653 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:56.654 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:56.754 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:56.785 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:56.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:56.786 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:56.887 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:56.918 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:56.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:56.919 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:57.020 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:57.060 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is set 2024-09-11T07:34:57.064 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:57.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:57.065 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:57.166 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:57.197 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:57.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:57.198 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:57.215 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 77 16 115377 115361 93.617 63.625 0.0157295 0.0106771 2024-09-11T07:34:57.299 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:57.329 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:57.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:57.330 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:57.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:57 smithi022 ceph-mon[29462]: 8.3 scrub ok 2024-09-11T07:34:57.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:57 smithi022 ceph-mon[29462]: pgmap v374: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 59 MiB/s wr, 947 op/s 2024-09-11T07:34:57.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:57 smithi022 ceph-mon[29462]: 5.16 scrub starts 2024-09-11T07:34:57.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:57 smithi022 ceph-mon[29462]: 5.16 scrub ok 2024-09-11T07:34:57.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:57 smithi022 ceph-mon[29462]: 8.9 scrub ok 2024-09-11T07:34:57.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:57 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3890885155' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:34:57.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:57 smithi022 ceph-mon[29462]: from='client.25148 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:34:57.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:57 smithi022 ceph-mon[37330]: 8.3 scrub ok 2024-09-11T07:34:57.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:57 smithi022 ceph-mon[37330]: pgmap v374: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 59 MiB/s wr, 947 op/s 2024-09-11T07:34:57.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:57 smithi022 ceph-mon[37330]: 5.16 scrub starts 2024-09-11T07:34:57.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:57 smithi022 ceph-mon[37330]: 5.16 scrub ok 2024-09-11T07:34:57.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:57 smithi022 ceph-mon[37330]: 8.9 scrub ok 2024-09-11T07:34:57.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:57 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3890885155' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:34:57.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:57 smithi022 ceph-mon[37330]: from='client.25148 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:34:57.432 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:57.477 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:57.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:57.478 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:57.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:57 smithi104 ceph-mon[33507]: 8.3 scrub ok 2024-09-11T07:34:57.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:57 smithi104 ceph-mon[33507]: pgmap v374: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 59 MiB/s wr, 947 op/s 2024-09-11T07:34:57.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:57 smithi104 ceph-mon[33507]: 5.16 scrub starts 2024-09-11T07:34:57.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:57 smithi104 ceph-mon[33507]: 5.16 scrub ok 2024-09-11T07:34:57.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:57 smithi104 ceph-mon[33507]: 8.9 scrub ok 2024-09-11T07:34:57.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:57 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3890885155' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:34:57.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:57 smithi104 ceph-mon[33507]: from='client.25148 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:34:57.579 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:57.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:57.618 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:57.618 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:57.719 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:57.755 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:57.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:57.756 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:57.857 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:57.887 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:57.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:57.888 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:57.989 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:34:58.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:58.019 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:58.019 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:34:58.120 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:58.151 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:58.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:58.152 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:58.215 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 78 15 116492 116477 93.311 69.75 0.00835761 0.010712 2024-09-11T07:34:58.253 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:58.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:58.297 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:58.398 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:58.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:58 smithi022 ceph-mon[29462]: 8.5 scrub starts 2024-09-11T07:34:58.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:58 smithi022 ceph-mon[29462]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:34:58.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:58 smithi022 ceph-mon[29462]: from='client.25148 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:34:58.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:58 smithi022 ceph-mon[29462]: osdmap e210: 8 total, 8 up, 8 in 2024-09-11T07:34:58.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:58 smithi022 ceph-mon[37330]: 8.5 scrub starts 2024-09-11T07:34:58.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:58 smithi022 ceph-mon[37330]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:34:58.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:58 smithi022 ceph-mon[37330]: from='client.25148 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:34:58.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:58 smithi022 ceph-mon[37330]: osdmap e210: 8 total, 8 up, 8 in 2024-09-11T07:34:58.432 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:58.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:58.433 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:58.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:58 smithi104 ceph-mon[33507]: 8.5 scrub starts 2024-09-11T07:34:58.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:58 smithi104 ceph-mon[33507]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:34:58.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:58 smithi104 ceph-mon[33507]: from='client.25148 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:34:58.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:58 smithi104 ceph-mon[33507]: osdmap e210: 8 total, 8 up, 8 in 2024-09-11T07:34:58.533 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:34:58.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:58.587 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:58.587 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:34:58.682 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:34:58.688 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:58.716 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:58.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:58.717 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:58.817 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:58.843 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:58.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:58.844 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:34:58.945 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:58.972 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:58.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:58.973 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:59.073 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:59.101 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:59.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:59.102 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:59.203 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:34:59.216 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 79 16 117479 117463 92.9095 61.625 0.00477362 0.0107572 2024-09-11T07:34:59.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:59.241 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:59.241 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:34:59.342 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:34:59.349 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:59 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:34:59.350 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:59 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:34:59.350 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:59 smithi022 ceph-mon[29462]: osdmap e211: 8 total, 8 up, 8 in 2024-09-11T07:34:59.350 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:34:59 smithi022 ceph-mon[29462]: pgmap v377: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.1 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-11T07:34:59.350 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:59 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:34:59.350 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:59 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:34:59.350 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:59 smithi022 ceph-mon[37330]: osdmap e211: 8 total, 8 up, 8 in 2024-09-11T07:34:59.350 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:34:59 smithi022 ceph-mon[37330]: pgmap v377: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.1 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-11T07:34:59.369 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:59.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:59.371 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:34:59.420 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:59 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:34:59.420 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:59 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:34:59.420 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:59 smithi104 ceph-mon[33507]: osdmap e211: 8 total, 8 up, 8 in 2024-09-11T07:34:59.420 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:34:59 smithi104 ceph-mon[33507]: pgmap v377: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.1 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-11T07:34:59.472 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:34:59.506 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:59.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:59.507 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:34:59.608 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:34:59.638 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:59.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:59.639 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:34:59.740 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:34:59.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:34:59 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:34:59.788 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:34:59.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:59.789 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:34:59.890 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:34:59.922 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:34:59.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:34:59.923 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:00.004 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T07:35:00.024 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:00.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:00.092 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:00.092 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:00.103 INFO:teuthology.orchestra.run.smithi022.stderr:set 4.1e pg_upmap mapping to [4,5,3] 2024-09-11T07:35:00.182 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:35:00.182 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:35:00.183 INFO:teuthology.orchestra.run.smithi022.stdout: "complaint_time": 30, 2024-09-11T07:35:00.183 INFO:teuthology.orchestra.run.smithi022.stdout: "num_blocked_ops": 0 2024-09-11T07:35:00.183 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:35:00.193 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:00.216 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:2024-09-11T07:35:00.215323+0000 min lat: 0.00108265 max lat: 0.159168 avg lat: 0.0107999 2024-09-11T07:35:00.216 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T07:35:00.216 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 80 16 118473 118457 92.5247 62.125 0.0122713 0.0107999 2024-09-11T07:35:00.237 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:00.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:00.238 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:00.339 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:00.372 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:00.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:00.373 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:00.374 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:00 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4274728318' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "4.1e", "id": [4, 5, 3]}]: dispatch 2024-09-11T07:35:00.374 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:00 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:00.375 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:00 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4274728318' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "4.1e", "id": [4, 5, 3]}]: dispatch 2024-09-11T07:35:00.375 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:00 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:00.473 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:00.504 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:00.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:00.508 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:00.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:00 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4274728318' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "4.1e", "id": [4, 5, 3]}]: dispatch 2024-09-11T07:35:00.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:00 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:00.609 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:00.646 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:00.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:00.648 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:00.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:35:00 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:35:00] "GET /metrics HTTP/1.1" 200 224095 "" "Prometheus/2.43.0" 2024-09-11T07:35:00.748 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:00.784 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:00.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:00.785 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:00.886 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:00.916 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:00.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:00.917 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:01.018 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:01.055 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:01.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:01.056 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:01.157 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:01.197 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:01.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:01.198 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:01.216 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 81 16 119571 119555 92.2296 68.625 0.00516646 0.0108371 2024-09-11T07:35:01.274 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-11T07:35:01.299 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:01.333 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:01.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:01.335 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:01.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:01 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4274728318' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "4.1e", "id": [4, 5, 3]}]': finished 2024-09-11T07:35:01.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:01 smithi022 ceph-mon[29462]: osdmap e212: 8 total, 8 up, 8 in 2024-09-11T07:35:01.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:01 smithi022 ceph-mon[29462]: pgmap v379: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1004 B/s rd, 61 MiB/s wr, 983 op/s 2024-09-11T07:35:01.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:01 smithi022 ceph-mon[29462]: osdmap e213: 8 total, 8 up, 8 in 2024-09-11T07:35:01.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:01 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4274728318' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "4.1e", "id": [4, 5, 3]}]': finished 2024-09-11T07:35:01.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:01 smithi022 ceph-mon[37330]: osdmap e212: 8 total, 8 up, 8 in 2024-09-11T07:35:01.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:01 smithi022 ceph-mon[37330]: pgmap v379: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1004 B/s rd, 61 MiB/s wr, 983 op/s 2024-09-11T07:35:01.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:01 smithi022 ceph-mon[37330]: osdmap e213: 8 total, 8 up, 8 in 2024-09-11T07:35:01.435 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:01.461 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:01.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:01.462 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:01.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:01 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4274728318' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "4.1e", "id": [4, 5, 3]}]': finished 2024-09-11T07:35:01.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:01 smithi104 ceph-mon[33507]: osdmap e212: 8 total, 8 up, 8 in 2024-09-11T07:35:01.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:01 smithi104 ceph-mon[33507]: pgmap v379: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1004 B/s rd, 61 MiB/s wr, 983 op/s 2024-09-11T07:35:01.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:01 smithi104 ceph-mon[33507]: osdmap e213: 8 total, 8 up, 8 in 2024-09-11T07:35:01.564 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:01.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:01.616 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:01.616 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:01.717 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:01.760 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:01.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:01.761 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:01.861 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:01.895 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:01.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:01.896 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:01.997 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:02.021 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:02.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:02.022 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:02.123 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:02.152 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:02.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:02.153 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:02.216 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 82 16 120481 120465 91.7984 56.875 0.0182677 0.0108886 2024-09-11T07:35:02.253 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:02.296 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:02.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:02.298 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:02.398 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:02.425 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:02.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:02.426 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:02.527 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:02.556 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:02.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:02.557 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:02.658 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:02.686 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:02.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:02.687 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:02.788 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:02.825 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:02.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:02.827 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:02.927 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:02.957 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:02.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:02.959 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:03.060 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:03.095 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:03.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:03.096 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:03.197 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:03.216 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 83 16 121500 121484 91.4597 63.6875 0.00507647 0.0109275 2024-09-11T07:35:03.226 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:03.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:03.227 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:03.328 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:03.359 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:03.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:03.361 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:03.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:03 smithi022 ceph-mon[29462]: pgmap v381: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1009 B/s rd, 62 MiB/s wr, 988 op/s 2024-09-11T07:35:03.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:03 smithi022 ceph-mon[37330]: pgmap v381: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1009 B/s rd, 62 MiB/s wr, 988 op/s 2024-09-11T07:35:03.461 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:03.492 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:03.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:03.493 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:03.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:03 smithi104 ceph-mon[33507]: pgmap v381: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1009 B/s rd, 62 MiB/s wr, 988 op/s 2024-09-11T07:35:03.567 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:35:03.568 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:35:03.581 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:35:03.593 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:03.597 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:35:03.597 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:35:03.598 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:35:03.618 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:03.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:03.619 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:03.628 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:35:03.628 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:35:03.664 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:35:03.665 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:35:03.694 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:35:03.694 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:35:03.720 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:03.721 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:35:03.722 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:35:03.753 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:35:03.754 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:35:03.756 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:03.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:03.757 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:03.783 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:35:03.783 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:35:03.858 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:03.904 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:03.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:03.906 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:04.006 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:04.039 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:04.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:04.040 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:04.140 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:04.183 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:04.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:04.184 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:04.216 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 84 16 122516 122500 91.1268 63.5 0.00643531 0.0109682 2024-09-11T07:35:04.285 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:04.310 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:04.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:04.312 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:04.412 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:04.446 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:04.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:04.448 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:04.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:35:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:35:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:35:04.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:04 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4136317243' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:35:04.548 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:04.581 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:04.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:04.582 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:04.614 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:04 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4136317243' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:35:04.616 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:04 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4136317243' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:35:04.683 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:04.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:04.710 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:04.710 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:04.813 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:04.842 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:04.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:04.843 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:04.944 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:04.988 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:04.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:04.989 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:05.090 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:05.116 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:05.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:05.117 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:05.216 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 85 16 123619 123603 90.8658 68.9375 0.0142309 0.0109998 2024-09-11T07:35:05.218 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:05.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:05.252 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:05.252 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:05.353 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:05.387 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:05.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:05.388 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:05.489 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:05.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:05 smithi022 ceph-mon[29462]: pgmap v382: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.3 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 843 B/s rd, 64 MiB/s wr, 1.02k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:35:05.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:05 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:35:05.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:05 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:35:05.504 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:05 smithi022 ceph-mon[37330]: pgmap v382: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.3 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 843 B/s rd, 64 MiB/s wr, 1.02k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:35:05.504 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:05 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:35:05.505 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:05 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:35:05.520 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:05.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:05.521 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:05.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:05 smithi104 ceph-mon[33507]: pgmap v382: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.3 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 843 B/s rd, 64 MiB/s wr, 1.02k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:35:05.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:05 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:35:05.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:05 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:35:05.623 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:05.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:05.659 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:05.659 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:05.760 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:05.791 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:05.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:05.792 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:05.893 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:05.923 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:05.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:05.924 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:06.024 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:06.052 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:06.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:06.053 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:06.154 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:06.199 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:06.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:06.200 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:06.216 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 86 16 124708 124692 90.6005 68.0625 0.016945 0.0110312 2024-09-11T07:35:06.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:06 smithi022 ceph-mon[29462]: osdmap e214: 8 total, 8 up, 8 in 2024-09-11T07:35:06.281 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:06 smithi022 ceph-mon[29462]: osdmap e215: 8 total, 8 up, 8 in 2024-09-11T07:35:06.282 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:06 smithi022 ceph-mon[37330]: osdmap e214: 8 total, 8 up, 8 in 2024-09-11T07:35:06.282 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:06 smithi022 ceph-mon[37330]: osdmap e215: 8 total, 8 up, 8 in 2024-09-11T07:35:06.301 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:06.331 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:06.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:06.333 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:06.434 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:06.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:06.467 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:06.467 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:06.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:06 smithi104 ceph-mon[33507]: osdmap e214: 8 total, 8 up, 8 in 2024-09-11T07:35:06.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:06 smithi104 ceph-mon[33507]: osdmap e215: 8 total, 8 up, 8 in 2024-09-11T07:35:06.568 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:06.611 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:06.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:06.612 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:06.712 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:06.745 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:06.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:06.746 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:06.847 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:06.880 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:06.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:06.881 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:06.982 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:07.008 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:07.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:07.009 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:07.110 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:07.142 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:07.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:07.143 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:07.217 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 87 16 125787 125771 90.3342 67.4375 0.00482264 0.011063 2024-09-11T07:35:07.244 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:07.280 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:07.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:07.281 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:07.352 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:35:07.382 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:07.402 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:07.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:07.403 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:07.504 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:07.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:07 smithi104 ceph-mon[33507]: pgmap v385: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.5 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 852 B/s rd, 52 MiB/s wr, 840 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:35:07.533 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:07.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:07.534 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:07.563 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:07 smithi022 ceph-mon[37330]: pgmap v385: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.5 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 852 B/s rd, 52 MiB/s wr, 840 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:35:07.563 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:07 smithi022 ceph-mon[29462]: pgmap v385: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.5 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 852 B/s rd, 52 MiB/s wr, 840 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:35:07.634 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:07.681 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:07.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:07.682 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:07.783 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:07.805 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:07.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:07.806 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:07.907 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:07.942 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:07.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:07.943 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:08.044 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:08.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:08.084 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:08.084 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:08.185 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:08.186 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:35:08.186 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:35:08.186 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-11T07:35:08.186 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-11T07:35:08.217 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 88 16 126780 126764 90.0128 62.0625 0.00888609 0.0111035 2024-09-11T07:35:08.227 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:08.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:08.228 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:08.329 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:08.352 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:08.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:08.354 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:08.454 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:08.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:08.486 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:08.486 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:08.587 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:08.619 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:08.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:08.620 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:08.721 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:08.748 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:08.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:08.749 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:08.850 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:08.897 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:08.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:08.899 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:08.999 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:09.026 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:09.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:09.027 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:09.100 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:35:09.101 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:35:09.128 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:09.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:09.159 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:09.160 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:09.187 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:09 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:35:09.188 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:09 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:35:09.188 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:09 smithi022 ceph-mon[37330]: pgmap v386: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.5 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 676 B/s rd, 42 MiB/s wr, 667 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:35:09.188 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:09 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:35:09.188 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:09 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:35:09.189 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:09 smithi022 ceph-mon[29462]: pgmap v386: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.5 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 676 B/s rd, 42 MiB/s wr, 667 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:35:09.217 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 89 16 127668 127652 89.625 55.5 0.00826917 0.0111509 2024-09-11T07:35:09.261 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:09.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:09 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:35:09.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:09 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:35:09.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:09 smithi104 ceph-mon[33507]: pgmap v386: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.5 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 676 B/s rd, 42 MiB/s wr, 667 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:35:09.290 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:09.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:09.291 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:09.392 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:09.421 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:09.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:09.422 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:09.523 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:09.550 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:09.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:09.551 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:09.652 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:09.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:09.721 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:09.721 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:09.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:35:09 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:35:09.822 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:09.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:09.860 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:09.860 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:09.961 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:10.005 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:10.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:10.006 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:10.054 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-11T07:35:10.107 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:10.132 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:10.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:10.133 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:10.217 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 90 3 128528 128525 89.2354 54.5625 0.0177833 0.0112004 2024-09-11T07:35:10.233 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:10.255 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:10.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:10.256 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:10.304 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:35:10.305 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:35:10.357 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:10.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:10.385 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:10.385 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:10.481 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:35:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:35:10] "GET /metrics HTTP/1.1" 200 224153 "" "Prometheus/2.43.0" 2024-09-11T07:35:10.486 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:10.507 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:10.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:10.508 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:10.609 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:10.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:10.632 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:10.632 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:10.733 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:10.755 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:10.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:10.756 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:10.857 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:10.880 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:10.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:10.881 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:10.982 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:11.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:11.005 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:11.006 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:11.106 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:11.131 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:11.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:11.132 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:11.180 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is set 2024-09-11T07:35:11.217 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Total time run: 90.0512 2024-09-11T07:35:11.217 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Total writes made: 128528 2024-09-11T07:35:11.217 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Write size: 65536 2024-09-11T07:35:11.217 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Object size: 65536 2024-09-11T07:35:11.217 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Bandwidth (MB/sec): 89.2048 2024-09-11T07:35:11.217 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Stddev Bandwidth: 33.2495 2024-09-11T07:35:11.217 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Max bandwidth (MB/sec): 210.5 2024-09-11T07:35:11.217 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Min bandwidth (MB/sec): 54.5625 2024-09-11T07:35:11.218 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Average IOPS: 1427 2024-09-11T07:35:11.218 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Stddev IOPS: 531.992 2024-09-11T07:35:11.218 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Max IOPS: 3368 2024-09-11T07:35:11.218 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Min IOPS: 873 2024-09-11T07:35:11.218 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Average Latency(s): 0.0112019 2024-09-11T07:35:11.218 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Stddev Latency(s): 0.0100195 2024-09-11T07:35:11.218 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Max latency(s): 0.159168 2024-09-11T07:35:11.218 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Min latency(s): 0.00108265 2024-09-11T07:35:11.223 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Cleaning up (deleting benchmark objects) 2024-09-11T07:35:11.234 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:11.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:11.262 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:11.262 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:11.363 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:11.403 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:11.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:11.404 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:11.505 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:11.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:11 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:11.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:11 smithi104 ceph-mon[33507]: pgmap v387: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.8 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 80 MiB/s wr, 1.27k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:35:11.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:11 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1686778688' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:35:11.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:11 smithi104 ceph-mon[33507]: from='client.25163 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:35:11.532 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:11.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:11.533 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:11.559 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:11 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:11.559 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:11 smithi022 ceph-mon[29462]: pgmap v387: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.8 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 80 MiB/s wr, 1.27k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:35:11.559 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:11 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1686778688' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:35:11.559 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:11 smithi022 ceph-mon[29462]: from='client.25163 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:35:11.559 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:11 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:11.559 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:11 smithi022 ceph-mon[37330]: pgmap v387: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.8 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 80 MiB/s wr, 1.27k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:35:11.559 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:11 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1686778688' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:35:11.560 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:11 smithi022 ceph-mon[37330]: from='client.25163 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:35:11.634 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:11.655 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:11.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:11.656 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:11.757 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:11.792 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:11.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:11.793 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:11.894 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:11.915 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:11.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:11.916 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:12.017 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:12.052 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:12.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:12.053 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:12.154 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:12.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:12.191 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:12.191 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:12.292 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:12.324 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:12.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:12.326 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:12.427 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:12.451 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:12.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:12.452 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:12.522 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:12 smithi022 ceph-mon[29462]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:35:12.522 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:12 smithi022 ceph-mon[29462]: from='client.25163 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:35:12.522 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:12 smithi022 ceph-mon[29462]: osdmap e216: 8 total, 8 up, 8 in 2024-09-11T07:35:12.522 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:12 smithi022 ceph-mon[37330]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:35:12.522 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:12 smithi022 ceph-mon[37330]: from='client.25163 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:35:12.523 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:12 smithi022 ceph-mon[37330]: osdmap e216: 8 total, 8 up, 8 in 2024-09-11T07:35:12.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:12 smithi104 ceph-mon[33507]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:35:12.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:12 smithi104 ceph-mon[33507]: from='client.25163 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:35:12.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:12 smithi104 ceph-mon[33507]: osdmap e216: 8 total, 8 up, 8 in 2024-09-11T07:35:12.553 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:12.590 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:12.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:12.591 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:12.692 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:12.721 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:12.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:12.722 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:12.823 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:12.856 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:12.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:12.857 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:12.958 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:12.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:12.992 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:12.992 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:13.093 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:13.129 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:13.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:13.130 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:13.231 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:13.268 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:13.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:13.269 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:13.370 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:13.401 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:13.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:13.402 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:13.503 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:13.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:13 smithi104 ceph-mon[33507]: pgmap v389: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.8 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 739 B/s rd, 46 MiB/s wr, 743 op/s 2024-09-11T07:35:13.540 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:13.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:13.541 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:13.642 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:13.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:13 smithi022 ceph-mon[37330]: pgmap v389: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.8 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 739 B/s rd, 46 MiB/s wr, 743 op/s 2024-09-11T07:35:13.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:13 smithi022 ceph-mon[29462]: pgmap v389: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.8 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 739 B/s rd, 46 MiB/s wr, 743 op/s 2024-09-11T07:35:13.680 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:13.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:13.681 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:13.782 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:13.822 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:13.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:13.823 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:13.924 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:13.962 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:13.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:13.963 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:14.064 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:14.106 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:14.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:14.107 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:14.208 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:14.243 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:14.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:14.244 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:14.345 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:14.375 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:14.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:14.376 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:14.476 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:14.507 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:14.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:14.508 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:14.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:35:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:35:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:35:14.609 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:14.635 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:14.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:14.636 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:14.737 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:14.766 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:14.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:14.767 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:14.868 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:14.895 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:14.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:14.896 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:14.997 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:15.041 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:15.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:15.042 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:15.142 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:15.173 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:15.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:15.174 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:15.275 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:15.286 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:15 smithi022 ceph-mon[29462]: pgmap v390: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.8 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 717 B/s rd, 39 MiB/s wr, 732 op/s 2024-09-11T07:35:15.287 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:15 smithi022 ceph-mon[37330]: pgmap v390: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.8 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 717 B/s rd, 39 MiB/s wr, 732 op/s 2024-09-11T07:35:15.317 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:15.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:15.332 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:15.432 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:15.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:15.476 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:15.476 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:15.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:15 smithi104 ceph-mon[33507]: pgmap v390: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.8 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 717 B/s rd, 39 MiB/s wr, 732 op/s 2024-09-11T07:35:15.577 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:15.619 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:15.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:15.620 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:15.721 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:15.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:15.763 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:15.763 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:15.864 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:15.904 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:15.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:15.905 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:16.005 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:16.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:16.034 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:16.034 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:16.096 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:35:16.097 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:35:16.097 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:35:16.097 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:35:16.097 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:35:16.097 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:35:16.097 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:35:16.097 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:35:16.097 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:35:16.098 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:35:16.098 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:35:16.098 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:35:16.098 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:35:16.098 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:35:16.098 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:35:16.098 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:35:16.134 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:16.174 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:16.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:16.176 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:16.277 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:16.312 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:16.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:16.313 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:16.414 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:16.440 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:16.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:16.441 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:16.542 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:16.569 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:16.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:16.570 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:16.627 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:16 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/660684196' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:35:16.627 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:16 smithi022 ceph-mon[29462]: pgmap v391: 148 pgs: 148 active+clean; 7.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.1 KiB/s rd, 37 MiB/s wr, 985 op/s 2024-09-11T07:35:16.630 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:16 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/660684196' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:35:16.630 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:16 smithi022 ceph-mon[37330]: pgmap v391: 148 pgs: 148 active+clean; 7.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.1 KiB/s rd, 37 MiB/s wr, 985 op/s 2024-09-11T07:35:16.672 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:16.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:16.701 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:16.701 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:16.802 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:16.841 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:16.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:16.843 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:16.943 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:16.986 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:16.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:16.987 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:17.013 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T07:35:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:16 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/660684196' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:35:17.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:16 smithi104 ceph-mon[33507]: pgmap v391: 148 pgs: 148 active+clean; 7.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.1 KiB/s rd, 37 MiB/s wr, 985 op/s 2024-09-11T07:35:17.052 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:35:17.089 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:17.118 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:17.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:17.119 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:17.220 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:17.251 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:17.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:17.253 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:17.354 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:17.404 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:17.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:17.405 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:17.506 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:17.535 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:17.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:17.536 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:17.636 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:17.656 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:35:17.657 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:35:17.657 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 216, 2024-09-11T07:35:17.657 INFO:teuthology.orchestra.run.smithi022.stdout: "fsid": "12a285d8-700e-11ef-bcea-c7b262605968", 2024-09-11T07:35:17.658 INFO:teuthology.orchestra.run.smithi022.stdout: "created": "2024-09-11T07:21:18.695332+0000", 2024-09-11T07:35:17.658 INFO:teuthology.orchestra.run.smithi022.stdout: "modified": "2024-09-11T07:35:11.174281+0000", 2024-09-11T07:35:17.658 INFO:teuthology.orchestra.run.smithi022.stdout: "last_up_change": "2024-09-11T07:32:00.070825+0000", 2024-09-11T07:35:17.658 INFO:teuthology.orchestra.run.smithi022.stdout: "last_in_change": "2024-09-11T07:25:58.551813+0000", 2024-09-11T07:35:17.658 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-11T07:35:17.658 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_num": 5806080, 2024-09-11T07:35:17.658 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_set": [ 2024-09-11T07:35:17.658 INFO:teuthology.orchestra.run.smithi022.stdout: "nodeep-scrub", 2024-09-11T07:35:17.658 INFO:teuthology.orchestra.run.smithi022.stdout: "noscrub", 2024-09-11T07:35:17.658 INFO:teuthology.orchestra.run.smithi022.stdout: "pglog_hardlimit", 2024-09-11T07:35:17.658 INFO:teuthology.orchestra.run.smithi022.stdout: "purged_snapdirs", 2024-09-11T07:35:17.658 INFO:teuthology.orchestra.run.smithi022.stdout: "recovery_deletes", 2024-09-11T07:35:17.659 INFO:teuthology.orchestra.run.smithi022.stdout: "sortbitwise" 2024-09-11T07:35:17.659 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:35:17.659 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_version": 19, 2024-09-11T07:35:17.659 INFO:teuthology.orchestra.run.smithi022.stdout: "full_ratio": 0.94999998807907104, 2024-09-11T07:35:17.659 INFO:teuthology.orchestra.run.smithi022.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-11T07:35:17.659 INFO:teuthology.orchestra.run.smithi022.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-11T07:35:17.659 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_snapshot": "", 2024-09-11T07:35:17.659 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_max": 8, 2024-09-11T07:35:17.659 INFO:teuthology.orchestra.run.smithi022.stdout: "max_osd": 8, 2024-09-11T07:35:17.659 INFO:teuthology.orchestra.run.smithi022.stdout: "require_min_compat_client": "luminous", 2024-09-11T07:35:17.659 INFO:teuthology.orchestra.run.smithi022.stdout: "min_compat_client": "luminous", 2024-09-11T07:35:17.659 INFO:teuthology.orchestra.run.smithi022.stdout: "require_osd_release": "quincy", 2024-09-11T07:35:17.659 INFO:teuthology.orchestra.run.smithi022.stdout: "pools": [ 2024-09-11T07:35:17.660 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.660 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 1, 2024-09-11T07:35:17.660 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": ".mgr", 2024-09-11T07:35:17.660 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:24:21.772426+0000", 2024-09-11T07:35:17.660 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:35:17.660 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:35:17.660 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:35:17.660 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:35:17.660 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:35:17.660 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:35:17.660 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:35:17.660 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:35:17.661 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:35:17.661 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:35:17.661 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:35:17.661 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:35:17.661 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 1, 2024-09-11T07:35:17.661 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 1, 2024-09-11T07:35:17.661 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 1, 2024-09-11T07:35:17.661 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 1, 2024-09-11T07:35:17.661 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 1, 2024-09-11T07:35:17.661 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:35:17.661 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:35:17.661 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:35:17.662 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:35:17.662 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:35:17.662 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:35:17.662 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:35:17.662 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.662 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "21", 2024-09-11T07:35:17.662 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:35:17.662 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:35:17.662 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:35:17.662 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:35:17.662 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:35:17.662 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:35:17.662 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:35:17.662 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:35:17.662 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:35:17.663 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:35:17.663 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:35:17.663 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:35:17.663 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:35:17.663 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:35:17.663 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:35:17.663 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:35:17.663 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:35:17.663 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:35:17.663 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:35:17.663 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:35:17.663 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:35:17.663 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:35:17.663 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:35:17.664 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:35:17.664 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:35:17.664 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:35:17.664 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.664 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:35:17.664 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:35:17.664 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:35:17.664 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:35:17.664 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:35:17.664 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:35:17.664 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:35:17.664 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:35:17.664 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:35:17.665 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:35:17.665 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:35:17.665 INFO:teuthology.orchestra.run.smithi022.stdout: "options": { 2024-09-11T07:35:17.665 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_max": 32, 2024-09-11T07:35:17.665 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_min": 1 2024-09-11T07:35:17.665 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.665 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:35:17.665 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": {} 2024-09-11T07:35:17.665 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.665 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.665 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.665 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 2, 2024-09-11T07:35:17.666 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": ".rgw.root", 2024-09-11T07:35:17.666 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:29.941354+0000", 2024-09-11T07:35:17.666 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:35:17.666 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:35:17.666 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:35:17.666 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:35:17.666 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:35:17.666 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:35:17.666 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:35:17.666 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:35:17.666 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:35:17.666 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:35:17.666 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:35:17.666 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:35:17.667 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:35:17.667 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:35:17.667 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:35:17.667 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:35:17.667 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:35:17.667 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:35:17.667 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:35:17.667 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:35:17.667 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:35:17.667 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:35:17.667 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:35:17.667 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:35:17.667 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.667 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "58", 2024-09-11T07:35:17.668 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:35:17.668 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:35:17.668 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:35:17.668 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:35:17.668 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:35:17.668 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:35:17.668 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:35:17.668 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:35:17.668 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:35:17.668 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:35:17.668 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:35:17.668 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:35:17.668 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:35:17.669 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:35:17.669 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:35:17.669 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:35:17.669 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:35:17.669 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:35:17.669 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:35:17.669 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:35:17.669 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:35:17.669 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:35:17.669 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:35:17.669 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:35:17.669 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:35:17.669 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:35:17.670 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.670 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:35:17.670 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:35:17.670 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:35:17.670 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:35:17.670 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:35:17.670 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:35:17.670 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:35:17.670 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:35:17.670 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:35:17.671 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:35:17.671 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:35:17.671 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:35:17.671 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:35:17.671 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:35:17.671 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.671 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.671 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.671 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 3, 2024-09-11T07:35:17.671 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "datapool", 2024-09-11T07:35:17.671 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:30.396859+0000", 2024-09-11T07:35:17.671 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 8193, 2024-09-11T07:35:17.671 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T07:35:17.672 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:35:17.672 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:35:17.672 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:35:17.672 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:35:17.672 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:35:17.672 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:35:17.672 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:35:17.672 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:35:17.672 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:35:17.672 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:35:17.672 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 3, 2024-09-11T07:35:17.672 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 3, 2024-09-11T07:35:17.672 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 3, 2024-09-11T07:35:17.673 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 3, 2024-09-11T07:35:17.673 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 3, 2024-09-11T07:35:17.673 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:35:17.673 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:35:17.673 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:35:17.673 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:35:17.673 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:35:17.673 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:35:17.673 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:35:17.673 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.673 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "61", 2024-09-11T07:35:17.673 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:35:17.673 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:35:17.673 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:35:17.674 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:35:17.674 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:35:17.674 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 3, 2024-09-11T07:35:17.674 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 61, 2024-09-11T07:35:17.674 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:35:17.674 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:35:17.674 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:35:17.674 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:35:17.674 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:35:17.674 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:35:17.674 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:35:17.674 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:35:17.674 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:35:17.674 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:35:17.675 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:35:17.675 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:35:17.675 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:35:17.675 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:35:17.675 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:35:17.675 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:35:17.675 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:35:17.675 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:35:17.675 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:35:17.676 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.676 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:35:17.676 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:35:17.677 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:35:17.677 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:35:17.677 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:35:17.677 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:35:17.677 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:35:17.677 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:35:17.677 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:35:17.677 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:35:17.677 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:35:17.677 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:35:17.677 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:35:17.677 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd": {} 2024-09-11T07:35:17.677 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.678 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.678 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.678 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 4, 2024-09-11T07:35:17.678 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.log", 2024-09-11T07:35:17.678 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:31.922926+0000", 2024-09-11T07:35:17.678 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:35:17.678 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:35:17.678 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:35:17.678 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:35:17.678 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:35:17.678 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:35:17.678 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:35:17.678 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:35:17.678 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:35:17.679 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:35:17.679 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:35:17.679 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:35:17.679 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:35:17.679 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:35:17.679 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:35:17.679 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:35:17.679 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:35:17.679 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:35:17.679 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:35:17.679 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:35:17.679 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:35:17.679 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:35:17.680 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:35:17.680 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:35:17.680 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.680 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "60", 2024-09-11T07:35:17.680 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:35:17.680 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:35:17.680 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:35:17.680 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:35:17.680 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:35:17.680 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:35:17.680 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:35:17.680 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:35:17.680 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:35:17.681 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:35:17.681 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:35:17.681 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:35:17.681 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:35:17.681 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:35:17.681 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:35:17.681 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:35:17.681 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:35:17.681 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:35:17.681 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:35:17.681 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:35:17.681 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:35:17.681 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:35:17.682 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:35:17.682 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:35:17.682 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:35:17.682 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:35:17.682 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.682 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:35:17.682 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:35:17.682 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:35:17.682 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:35:17.682 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:35:17.682 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:35:17.682 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:35:17.682 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:35:17.683 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:35:17.683 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:35:17.683 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:35:17.683 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:35:17.683 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:35:17.683 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:35:17.683 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.683 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.683 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.683 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 5, 2024-09-11T07:35:17.683 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.control", 2024-09-11T07:35:17.683 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:34.007746+0000", 2024-09-11T07:35:17.683 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:35:17.684 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:35:17.684 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:35:17.684 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:35:17.684 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:35:17.684 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:35:17.684 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:35:17.684 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:35:17.684 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:35:17.684 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:35:17.684 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:35:17.684 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:35:17.684 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:35:17.684 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:35:17.685 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:35:17.685 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:35:17.685 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:35:17.685 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:35:17.685 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:35:17.685 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:35:17.685 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:35:17.685 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:35:17.685 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:35:17.685 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:35:17.685 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.685 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "62", 2024-09-11T07:35:17.685 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:35:17.686 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:35:17.686 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:35:17.686 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:35:17.686 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:35:17.686 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:35:17.686 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:35:17.686 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:35:17.686 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:35:17.686 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:35:17.686 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:35:17.686 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:35:17.686 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:35:17.686 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:35:17.686 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:35:17.687 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:35:17.687 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:35:17.687 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:35:17.687 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:35:17.687 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:35:17.687 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:35:17.687 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:35:17.687 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:35:17.687 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:35:17.687 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:35:17.687 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:35:17.687 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.687 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:35:17.687 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:35:17.688 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:35:17.688 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:35:17.688 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:35:17.688 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:35:17.688 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:35:17.688 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:35:17.688 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:35:17.688 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:35:17.688 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:35:17.688 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:35:17.688 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:35:17.688 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:35:17.688 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.688 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.689 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.689 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 6, 2024-09-11T07:35:17.689 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.meta", 2024-09-11T07:35:17.689 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:36.077215+0000", 2024-09-11T07:35:17.689 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:35:17.689 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:35:17.689 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:35:17.689 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:35:17.689 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:35:17.689 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:35:17.689 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:35:17.689 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:35:17.689 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:35:17.689 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:35:17.690 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:35:17.690 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:35:17.690 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:35:17.690 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:35:17.690 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:35:17.690 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:35:17.690 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:35:17.690 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:35:17.690 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:35:17.690 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:35:17.690 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:35:17.690 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:35:17.690 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:35:17.690 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:35:17.691 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.691 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "64", 2024-09-11T07:35:17.691 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:35:17.691 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:35:17.691 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:35:17.691 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:35:17.691 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:35:17.691 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:35:17.691 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:35:17.691 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:35:17.691 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:35:17.691 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:35:17.691 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:35:17.691 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:35:17.692 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:35:17.692 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:35:17.692 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:35:17.692 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:35:17.692 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:35:17.692 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:35:17.692 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:35:17.692 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:35:17.692 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:35:17.692 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:35:17.692 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:35:17.692 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:35:17.692 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:35:17.693 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:35:17.693 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.693 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:35:17.693 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:35:17.693 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:35:17.693 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:35:17.693 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:35:17.693 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:35:17.693 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:35:17.693 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:35:17.693 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:35:17.693 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:35:17.693 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:35:17.694 INFO:teuthology.orchestra.run.smithi022.stdout: "options": { 2024-09-11T07:35:17.694 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_bias": 4 2024-09-11T07:35:17.694 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.694 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:35:17.694 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:35:17.694 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.694 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.694 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.694 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 8, 2024-09-11T07:35:17.694 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "unique_pool_1", 2024-09-11T07:35:17.694 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:33:28.041392+0000", 2024-09-11T07:35:17.694 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:35:17.694 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:35:17.694 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:35:17.695 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:35:17.695 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:35:17.695 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:35:17.695 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:35:17.695 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:35:17.695 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:35:17.695 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:35:17.695 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:35:17.695 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:35:17.695 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 16, 2024-09-11T07:35:17.695 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 16, 2024-09-11T07:35:17.695 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 16, 2024-09-11T07:35:17.695 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 16, 2024-09-11T07:35:17.695 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 16, 2024-09-11T07:35:17.696 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:35:17.696 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:35:17.696 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:35:17.696 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:35:17.696 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:35:17.696 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:35:17.696 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:35:17.696 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.696 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "184", 2024-09-11T07:35:17.696 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:35:17.697 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:35:17.697 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:35:17.697 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:35:17.697 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:35:17.697 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:35:17.697 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:35:17.697 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:35:17.697 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:35:17.697 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:35:17.697 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:35:17.697 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:35:17.697 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:35:17.698 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:35:17.698 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:35:17.698 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:35:17.698 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:35:17.698 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:35:17.698 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:35:17.698 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:35:17.698 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:35:17.698 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:35:17.698 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:35:17.698 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:35:17.698 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:35:17.698 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:35:17.698 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.699 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:35:17.699 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:35:17.699 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:35:17.699 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:35:17.699 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:35:17.699 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:35:17.699 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:35:17.699 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:35:17.699 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:35:17.699 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:35:17.699 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:35:17.699 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:35:17.699 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:35:17.700 INFO:teuthology.orchestra.run.smithi022.stdout: "rados": {} 2024-09-11T07:35:17.700 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.700 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.700 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:35:17.700 INFO:teuthology.orchestra.run.smithi022.stdout: "osds": [ 2024-09-11T07:35:17.700 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.700 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 0, 2024-09-11T07:35:17.700 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "09075f16-32f4-4f88-a116-c3d7f977a0e0", 2024-09-11T07:35:17.700 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:35:17.700 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:35:17.700 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:35:17.700 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:35:17.700 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:35:17.700 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:35:17.701 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 8, 2024-09-11T07:35:17.701 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 177, 2024-09-11T07:35:17.701 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:35:17.701 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:35:17.701 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:35:17.701 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.701 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.701 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.701 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6801", 2024-09-11T07:35:17.701 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:35:17.701 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.701 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.702 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.702 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:35:17.702 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.702 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.702 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.702 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6802", 2024-09-11T07:35:17.702 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:35:17.702 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.702 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.702 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.702 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:35:17.702 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.702 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.703 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.703 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6804", 2024-09-11T07:35:17.703 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:35:17.703 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.703 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.703 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.703 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:35:17.703 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.703 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.703 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.703 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6803", 2024-09-11T07:35:17.703 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:35:17.703 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.703 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.704 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.704 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6801/2821493591", 2024-09-11T07:35:17.704 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6802/2821493591", 2024-09-11T07:35:17.704 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6804/2821493591", 2024-09-11T07:35:17.704 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6803/2821493591", 2024-09-11T07:35:17.704 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:35:17.704 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:35:17.704 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:35:17.704 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.704 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.704 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.704 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 1, 2024-09-11T07:35:17.705 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "52cc1e06-9506-48c0-9f7b-d4270dff4b9f", 2024-09-11T07:35:17.705 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:35:17.705 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:35:17.705 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:35:17.705 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:35:17.705 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:35:17.705 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:35:17.705 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 13, 2024-09-11T07:35:17.705 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 177, 2024-09-11T07:35:17.705 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:35:17.705 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:35:17.705 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:35:17.706 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.706 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.706 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.706 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6805", 2024-09-11T07:35:17.706 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:35:17.706 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.706 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.706 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.706 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:35:17.706 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.706 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.706 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.706 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6806", 2024-09-11T07:35:17.706 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:35:17.707 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.707 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.707 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.707 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:35:17.707 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.707 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.707 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.707 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6808", 2024-09-11T07:35:17.707 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:35:17.707 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.707 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.707 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.707 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:35:17.708 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.708 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.708 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.708 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6807", 2024-09-11T07:35:17.708 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:35:17.708 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.708 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.708 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.708 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6805/4037911317", 2024-09-11T07:35:17.708 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6806/4037911317", 2024-09-11T07:35:17.708 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6808/4037911317", 2024-09-11T07:35:17.708 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6807/4037911317", 2024-09-11T07:35:17.708 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:35:17.709 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:35:17.709 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:35:17.709 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.709 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.709 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.709 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 2, 2024-09-11T07:35:17.709 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "fa2b2730-2c1d-45d0-be8a-494a60dc6b9b", 2024-09-11T07:35:17.709 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:35:17.709 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:35:17.709 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:35:17.709 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:35:17.709 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:35:17.710 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:35:17.710 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 18, 2024-09-11T07:35:17.710 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 161, 2024-09-11T07:35:17.710 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:35:17.710 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:35:17.710 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:35:17.710 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.710 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.710 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.710 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6809", 2024-09-11T07:35:17.710 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:35:17.710 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.710 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.711 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.711 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:35:17.711 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.711 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.711 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.711 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6810", 2024-09-11T07:35:17.711 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:35:17.711 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.711 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.711 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.711 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:35:17.711 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.711 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.712 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.712 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6812", 2024-09-11T07:35:17.712 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:35:17.712 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.712 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.712 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.712 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:35:17.712 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.712 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.712 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.712 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6811", 2024-09-11T07:35:17.712 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:35:17.712 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.713 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.713 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.713 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6809/1675799666", 2024-09-11T07:35:17.713 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6810/1675799666", 2024-09-11T07:35:17.713 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6812/1675799666", 2024-09-11T07:35:17.713 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6811/1675799666", 2024-09-11T07:35:17.713 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:35:17.713 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:35:17.713 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:35:17.713 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.713 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.713 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:17.714 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:17.714 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:17.715 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 3, 2024-09-11T07:35:17.715 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "8d53d3fc-e401-47df-b70b-4b4b1f4539df", 2024-09-11T07:35:17.715 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:35:17.715 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:35:17.715 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:35:17.715 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:35:17.715 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:35:17.715 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:35:17.715 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 25, 2024-09-11T07:35:17.715 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 177, 2024-09-11T07:35:17.715 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:35:17.716 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:35:17.716 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:35:17.716 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.716 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.716 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.716 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6813", 2024-09-11T07:35:17.716 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:35:17.716 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.716 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.716 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.716 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:35:17.716 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.716 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.717 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.717 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6814", 2024-09-11T07:35:17.717 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:35:17.717 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.717 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.717 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.717 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:35:17.717 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.717 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.717 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.717 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6816", 2024-09-11T07:35:17.717 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:35:17.718 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.718 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.718 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.718 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:35:17.718 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.718 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.718 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.718 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6815", 2024-09-11T07:35:17.718 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:35:17.718 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.718 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.718 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.718 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6813/1501962752", 2024-09-11T07:35:17.719 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6814/1501962752", 2024-09-11T07:35:17.719 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6816/1501962752", 2024-09-11T07:35:17.719 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6815/1501962752", 2024-09-11T07:35:17.719 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:35:17.719 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:35:17.719 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:35:17.719 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.719 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.719 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.719 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 4, 2024-09-11T07:35:17.719 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "d3207fe0-8b97-4067-a755-ba6d449c53c7", 2024-09-11T07:35:17.719 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:35:17.719 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:35:17.720 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:35:17.720 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:35:17.720 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:35:17.720 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:35:17.720 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 31, 2024-09-11T07:35:17.720 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 212, 2024-09-11T07:35:17.720 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:35:17.720 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:35:17.720 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:35:17.720 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.720 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.720 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.720 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6800", 2024-09-11T07:35:17.721 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:35:17.721 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.721 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.721 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.721 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:35:17.721 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.721 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.721 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.724 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6801", 2024-09-11T07:35:17.724 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:35:17.724 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.724 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.724 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.724 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:35:17.724 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.725 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.725 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.725 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6803", 2024-09-11T07:35:17.725 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:35:17.725 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.725 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.725 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.725 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:35:17.725 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.725 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.725 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.725 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6802", 2024-09-11T07:35:17.725 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:35:17.725 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.726 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.726 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.726 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6800/1210229443", 2024-09-11T07:35:17.726 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6801/1210229443", 2024-09-11T07:35:17.726 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6803/1210229443", 2024-09-11T07:35:17.726 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6802/1210229443", 2024-09-11T07:35:17.726 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:35:17.726 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:35:17.726 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:35:17.726 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.726 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.726 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.726 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 5, 2024-09-11T07:35:17.726 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "89c9352b-30b2-48c6-869e-047d07a23a43", 2024-09-11T07:35:17.726 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:35:17.727 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:35:17.727 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:35:17.727 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:35:17.727 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:35:17.727 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:35:17.727 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 37, 2024-09-11T07:35:17.727 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 177, 2024-09-11T07:35:17.727 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:35:17.727 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:35:17.727 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:35:17.727 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.727 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.727 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.727 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6804", 2024-09-11T07:35:17.728 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:35:17.728 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.728 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.728 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.728 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:35:17.728 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.728 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.728 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.728 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6805", 2024-09-11T07:35:17.728 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:35:17.728 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.728 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.728 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.728 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:35:17.729 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.729 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.729 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.729 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6807", 2024-09-11T07:35:17.729 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:35:17.729 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.729 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.729 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.729 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:35:17.729 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.729 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.729 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.729 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6806", 2024-09-11T07:35:17.729 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:35:17.730 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.730 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.730 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.730 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6804/2735328732", 2024-09-11T07:35:17.730 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6805/2735328732", 2024-09-11T07:35:17.730 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6807/2735328732", 2024-09-11T07:35:17.730 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6806/2735328732", 2024-09-11T07:35:17.730 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:35:17.730 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:35:17.730 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:35:17.730 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.730 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.730 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.730 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 6, 2024-09-11T07:35:17.731 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "704563de-d88f-480a-b469-720135704d42", 2024-09-11T07:35:17.731 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:35:17.731 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:35:17.731 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:35:17.731 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:35:17.731 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:35:17.731 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:35:17.731 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 44, 2024-09-11T07:35:17.731 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 177, 2024-09-11T07:35:17.731 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:35:17.731 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:35:17.731 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:35:17.731 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.731 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.732 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.732 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6808", 2024-09-11T07:35:17.732 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:35:17.732 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.732 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.732 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.732 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:35:17.732 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.732 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.732 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.732 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6809", 2024-09-11T07:35:17.732 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:35:17.732 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.732 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.733 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.733 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:35:17.733 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.733 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.733 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.733 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6811", 2024-09-11T07:35:17.733 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:35:17.733 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.733 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.733 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.733 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:35:17.733 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.733 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.734 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.734 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6810", 2024-09-11T07:35:17.734 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:35:17.734 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.734 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.734 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.734 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6808/721493010", 2024-09-11T07:35:17.734 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6809/721493010", 2024-09-11T07:35:17.734 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6811/721493010", 2024-09-11T07:35:17.734 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6810/721493010", 2024-09-11T07:35:17.734 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:35:17.734 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:35:17.734 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:35:17.735 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.735 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.735 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.735 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 7, 2024-09-11T07:35:17.735 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "18199e47-0052-49f0-b45f-0fb201e0d0f4", 2024-09-11T07:35:17.735 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:35:17.735 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:35:17.735 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:35:17.735 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:35:17.735 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 52, 2024-09-11T07:35:17.735 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 133, 2024-09-11T07:35:17.735 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 147, 2024-09-11T07:35:17.736 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 167, 2024-09-11T07:35:17.736 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 134, 2024-09-11T07:35:17.736 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:35:17.736 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:35:17.736 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.736 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.736 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.736 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6812", 2024-09-11T07:35:17.736 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2218863853 2024-09-11T07:35:17.736 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.736 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.736 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.737 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:35:17.737 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.737 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.737 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.737 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6813", 2024-09-11T07:35:17.737 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2218863853 2024-09-11T07:35:17.737 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.737 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.737 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.737 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:35:17.737 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.737 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.737 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.738 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6815", 2024-09-11T07:35:17.738 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2218863853 2024-09-11T07:35:17.738 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.738 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.738 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.738 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:35:17.738 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:35:17.738 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.738 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:35:17.738 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6814", 2024-09-11T07:35:17.738 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2218863853 2024-09-11T07:35:17.738 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.738 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.738 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.739 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6812/2218863853", 2024-09-11T07:35:17.739 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6813/2218863853", 2024-09-11T07:35:17.739 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6815/2218863853", 2024-09-11T07:35:17.739 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6814/2218863853", 2024-09-11T07:35:17.739 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:35:17.739 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:35:17.739 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:35:17.739 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.739 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.739 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:35:17.739 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_xinfo": [ 2024-09-11T07:35:17.739 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.739 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 0, 2024-09-11T07:35:17.740 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:35:17.740 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:35:17.740 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:35:17.740 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:35:17.740 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:35:17.740 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:33:53.829080+0000", 2024-09-11T07:35:17.740 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:35:17.740 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.740 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.740 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 1, 2024-09-11T07:35:17.740 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:35:17.740 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:35:17.741 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:35:17.741 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:35:17.741 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:35:17.741 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:35:04.552209+0000", 2024-09-11T07:35:17.741 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:35:17.741 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.741 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.741 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 2, 2024-09-11T07:35:17.741 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:35:17.742 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:35:17.742 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:35:17.742 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:35:17.742 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:35:17.742 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:33:51.706870+0000", 2024-09-11T07:35:17.742 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:35:17.742 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.742 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.742 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 3, 2024-09-11T07:35:17.742 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:35:17.743 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:35:17.743 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:35:17.743 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:35:17.743 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:35:17.743 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:34:52.765101+0000", 2024-09-11T07:35:17.743 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:35:17.743 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.743 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.743 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 4, 2024-09-11T07:35:17.743 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:35:17.743 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:35:17.743 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:35:17.743 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:35:17.743 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:35:17.744 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:34:57.918667+0000", 2024-09-11T07:35:17.744 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:35:17.744 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.744 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.744 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 5, 2024-09-11T07:35:17.744 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:35:17.744 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:35:17.744 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:35:17.744 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:35:17.744 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:35:17.744 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:34:29.436631+0000", 2024-09-11T07:35:17.744 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:35:17.744 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.745 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.745 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 6, 2024-09-11T07:35:17.745 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:35:17.745 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:35:17.745 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:35:17.745 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:35:17.745 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:35:17.745 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:34:28.486370+0000", 2024-09-11T07:35:17.745 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:35:17.745 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.745 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.745 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 7, 2024-09-11T07:35:17.745 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:35:17.745 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:35:17.746 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:35:17.746 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:35:17.746 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:35:17.746 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:34:54.457031+0000", 2024-09-11T07:35:17.746 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 133 2024-09-11T07:35:17.746 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.746 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:35:17.746 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_upmap": [ 2024-09-11T07:35:17.746 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.746 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "2.1c", 2024-09-11T07:35:17.746 INFO:teuthology.orchestra.run.smithi022.stdout: "osds": [ 2024-09-11T07:35:17.746 INFO:teuthology.orchestra.run.smithi022.stdout: 5, 2024-09-11T07:35:17.746 INFO:teuthology.orchestra.run.smithi022.stdout: 2, 2024-09-11T07:35:17.746 INFO:teuthology.orchestra.run.smithi022.stdout: 7 2024-09-11T07:35:17.747 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.747 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.747 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.747 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.1e", 2024-09-11T07:35:17.747 INFO:teuthology.orchestra.run.smithi022.stdout: "osds": [ 2024-09-11T07:35:17.747 INFO:teuthology.orchestra.run.smithi022.stdout: 4, 2024-09-11T07:35:17.747 INFO:teuthology.orchestra.run.smithi022.stdout: 5, 2024-09-11T07:35:17.747 INFO:teuthology.orchestra.run.smithi022.stdout: 3 2024-09-11T07:35:17.747 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.747 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.747 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:35:17.747 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_upmap_items": [ 2024-09-11T07:35:17.747 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.747 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "2.7", 2024-09-11T07:35:17.748 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:35:17.748 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.748 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 7, 2024-09-11T07:35:17.748 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 4 2024-09-11T07:35:17.748 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.748 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.748 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.748 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.748 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.4", 2024-09-11T07:35:17.748 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:35:17.748 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.748 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 1, 2024-09-11T07:35:17.748 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 5 2024-09-11T07:35:17.748 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.749 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.749 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.749 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.749 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.6", 2024-09-11T07:35:17.749 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:35:17.749 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.749 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 1, 2024-09-11T07:35:17.749 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 5 2024-09-11T07:35:17.749 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.749 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.749 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.749 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.749 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.14", 2024-09-11T07:35:17.749 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:35:17.750 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:17.750 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 1, 2024-09-11T07:35:17.750 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 2 2024-09-11T07:35:17.750 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.750 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:17.750 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.750 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:35:17.750 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_temp": [], 2024-09-11T07:35:17.750 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_temp": [], 2024-09-11T07:35:17.750 INFO:teuthology.orchestra.run.smithi022.stdout: "blocklist": { 2024-09-11T07:35:17.750 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/240999923": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:35:17.750 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/4016369460": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:35:17.750 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/585929238": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:35:17.751 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/544031024": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:35:17.751 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/2504196132": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:35:17.751 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/1451388349": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:35:17.751 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/639336081": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:35:17.751 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2104069730": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:35:17.751 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/4236450731": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:35:17.751 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/3573193776": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:35:17.751 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2661764422": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:35:17.751 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2197488530": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:35:17.751 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2968034970": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:35:17.751 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/599318175": "2024-09-12T07:27:07.919998+0000" 2024-09-11T07:35:17.751 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.751 INFO:teuthology.orchestra.run.smithi022.stdout: "range_blocklist": {}, 2024-09-11T07:35:17.751 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profiles": { 2024-09-11T07:35:17.752 INFO:teuthology.orchestra.run.smithi022.stdout: "default": { 2024-09-11T07:35:17.752 INFO:teuthology.orchestra.run.smithi022.stdout: "crush-failure-domain": "osd", 2024-09-11T07:35:17.752 INFO:teuthology.orchestra.run.smithi022.stdout: "k": "2", 2024-09-11T07:35:17.752 INFO:teuthology.orchestra.run.smithi022.stdout: "m": "1", 2024-09-11T07:35:17.752 INFO:teuthology.orchestra.run.smithi022.stdout: "plugin": "jerasure", 2024-09-11T07:35:17.752 INFO:teuthology.orchestra.run.smithi022.stdout: "technique": "reed_sol_van" 2024-09-11T07:35:17.752 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.752 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:17.752 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps_queue": [], 2024-09-11T07:35:17.752 INFO:teuthology.orchestra.run.smithi022.stdout: "new_removed_snaps": [], 2024-09-11T07:35:17.752 INFO:teuthology.orchestra.run.smithi022.stdout: "new_purged_snaps": [], 2024-09-11T07:35:17.752 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_node_flags": {}, 2024-09-11T07:35:17.752 INFO:teuthology.orchestra.run.smithi022.stdout: "device_class_flags": {}, 2024-09-11T07:35:17.753 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode": { 2024-09-11T07:35:17.753 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode_enabled": false, 2024-09-11T07:35:17.753 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_bucket_count": 0, 2024-09-11T07:35:17.753 INFO:teuthology.orchestra.run.smithi022.stdout: "degraded_stretch_mode": 0, 2024-09-11T07:35:17.753 INFO:teuthology.orchestra.run.smithi022.stdout: "recovering_stretch_mode": 0, 2024-09-11T07:35:17.753 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode_bucket": 0 2024-09-11T07:35:17.753 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:17.753 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:35:17.815 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:17.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:17.858 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:17 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1045138967' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:35:17.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:17 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1045138967' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:35:17.959 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:17.999 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:18.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:18.000 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:18.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:17 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1045138967' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:35:18.101 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:18.128 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:18.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:18.129 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:18.230 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:18.261 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:18.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:18.262 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:18.363 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:18.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:18.408 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:18.408 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:18.474 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:35:18.509 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:18.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:18.549 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:18.549 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:18.650 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:18.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:18.684 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:18.684 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:18.785 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:18.813 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:18.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:18.814 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:18.904 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:18 smithi022 ceph-mon[37330]: pgmap v392: 148 pgs: 148 active+clean; 7.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.1 KiB/s rd, 37 MiB/s wr, 985 op/s 2024-09-11T07:35:18.905 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:18 smithi022 ceph-mon[29462]: pgmap v392: 148 pgs: 148 active+clean; 7.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.1 KiB/s rd, 37 MiB/s wr, 985 op/s 2024-09-11T07:35:18.915 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:18.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:18.943 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:18.943 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:19.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:18 smithi104 ceph-mon[33507]: pgmap v392: 148 pgs: 148 active+clean; 7.6 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.1 KiB/s rd, 37 MiB/s wr, 985 op/s 2024-09-11T07:35:19.044 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:19.090 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:35:19.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:19.102 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:19.102 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:19.203 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:19.227 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:19.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:19.228 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:19.329 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:19.362 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:19.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:19.365 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:19.465 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:19.511 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:19.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:19.513 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:19.613 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:19.653 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:19.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:19.655 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:19.755 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:19.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:35:19 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:35:19.790 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:19.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:19.791 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:19.892 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:19.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:19.925 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:19.925 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:20.026 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:20.060 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:20.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:20.061 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:20.162 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:20.198 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:20.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:20.199 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:20.300 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:20.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:20.331 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:20.331 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:20.432 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:20.464 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:20.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:20.465 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:20.565 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:20.606 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:35:20 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:35:20] "GET /metrics HTTP/1.1" 200 224153 "" "Prometheus/2.43.0" 2024-09-11T07:35:20.611 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:20.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:20.613 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:20.713 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:20.761 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:20.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:20.764 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:20.865 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:20.866 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:35:20.866 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:35:20.866 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.866 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.25072.0:140555 8.0 e217/177 8:08e010d1:::benchmark_data_smithi022_108406_object12024:head v 217'8732, mlcod=217'8730)", 2024-09-11T07:35:20.866 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:35:20.862922+0000", 2024-09-11T07:35:20.866 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0011883180000000001, 2024-09-11T07:35:20.867 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001215406, 2024-09-11T07:35:20.867 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:35:20.867 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:35:20.867 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:35:20.867 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.867 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:35:20.867 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.862922+0000", 2024-09-11T07:35:20.867 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:20.867 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.867 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.867 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:35:20.867 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.862922+0000", 2024-09-11T07:35:20.868 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:20.868 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.868 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.868 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:35:20.868 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.862920+0000", 2024-09-11T07:35:20.868 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:35:20.868 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.868 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.868 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:35:20.868 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.862936+0000", 2024-09-11T07:35:20.868 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5515e-05 2024-09-11T07:35:20.868 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.868 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.869 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:35:20.869 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.862946+0000", 2024-09-11T07:35:20.869 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0156000000000001e-05 2024-09-11T07:35:20.869 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.869 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.869 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:35:20.869 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.862951+0000", 2024-09-11T07:35:20.869 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6509999999999998e-06 2024-09-11T07:35:20.870 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.870 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.870 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:35:20.870 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.863010+0000", 2024-09-11T07:35:20.870 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8857999999999998e-05 2024-09-11T07:35:20.870 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.870 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.870 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:35:20.870 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.863029+0000", 2024-09-11T07:35:20.870 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8287999999999998e-05 2024-09-11T07:35:20.871 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:20.871 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:20.871 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:20.871 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.871 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.871 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:140540 8.7 8:e4542597:::benchmark_data_smithi022_108406_object12009:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-11T07:35:20.871 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:35:20.849515+0000", 2024-09-11T07:35:20.871 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.014595227000000001, 2024-09-11T07:35:20.871 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.014688886, 2024-09-11T07:35:20.871 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:35:20.871 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "waiting for sub ops", 2024-09-11T07:35:20.871 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:35:20.871 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:35:20.871 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:35:20.872 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 140540 2024-09-11T07:35:20.872 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.872 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:35:20.872 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.872 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:35:20.872 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.849515+0000", 2024-09-11T07:35:20.872 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:20.872 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.872 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.872 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:35:20.872 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.849515+0000", 2024-09-11T07:35:20.872 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:20.873 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.873 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.873 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:35:20.873 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.849513+0000", 2024-09-11T07:35:20.873 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:35:20.873 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.873 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.873 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:35:20.873 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.849520+0000", 2024-09-11T07:35:20.873 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5740000000000001e-06 2024-09-11T07:35:20.873 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.873 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.873 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:35:20.874 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.849537+0000", 2024-09-11T07:35:20.874 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.6688999999999999e-05 2024-09-11T07:35:20.874 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.874 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.874 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:35:20.874 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.849543+0000", 2024-09-11T07:35:20.874 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5390000000000004e-06 2024-09-11T07:35:20.874 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.874 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.874 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:35:20.874 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.852935+0000", 2024-09-11T07:35:20.874 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0033915830000000001 2024-09-11T07:35:20.874 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.874 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.875 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:35:20.875 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.853063+0000", 2024-09-11T07:35:20.875 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012850000000000001 2024-09-11T07:35:20.875 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.875 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.875 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,6", 2024-09-11T07:35:20.875 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.853218+0000", 2024-09-11T07:35:20.875 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015502499999999999 2024-09-11T07:35:20.875 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.875 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.875 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:20.875 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.859400+0000", 2024-09-11T07:35:20.875 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0061820929999999996 2024-09-11T07:35:20.876 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.876 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.876 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:35:20.876 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.859857+0000", 2024-09-11T07:35:20.876 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00045628900000000002 2024-09-11T07:35:20.876 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:20.876 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:20.876 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:20.876 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.876 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.876 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:140550 8.3 8:c70af2c7:::benchmark_data_smithi022_108406_object12019:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-11T07:35:20.876 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:35:20.858908+0000", 2024-09-11T07:35:20.876 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0052025379999999996, 2024-09-11T07:35:20.876 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0053773919999999999, 2024-09-11T07:35:20.877 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:35:20.877 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "waiting for sub ops", 2024-09-11T07:35:20.877 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:35:20.877 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:35:20.877 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:35:20.877 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 140550 2024-09-11T07:35:20.877 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.877 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:35:20.877 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.878 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:35:20.878 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.858908+0000", 2024-09-11T07:35:20.878 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:20.878 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.878 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.878 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:35:20.878 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.858908+0000", 2024-09-11T07:35:20.878 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:20.878 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.878 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.879 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:35:20.879 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.858906+0000", 2024-09-11T07:35:20.879 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:35:20.879 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.879 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.879 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:35:20.879 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.858912+0000", 2024-09-11T07:35:20.879 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4849999999999997e-06 2024-09-11T07:35:20.879 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.880 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.880 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:35:20.880 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.858922+0000", 2024-09-11T07:35:20.880 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.7000000000000003e-06 2024-09-11T07:35:20.880 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.880 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.880 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:35:20.880 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.858927+0000", 2024-09-11T07:35:20.881 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3029999999999999e-06 2024-09-11T07:35:20.881 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.881 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.881 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:35:20.881 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.859352+0000", 2024-09-11T07:35:20.881 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00042482799999999998 2024-09-11T07:35:20.881 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.881 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.881 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:35:20.881 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.859465+0000", 2024-09-11T07:35:20.881 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000112763 2024-09-11T07:35:20.881 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.881 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.881 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:35:20.882 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.859578+0000", 2024-09-11T07:35:20.882 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011346200000000001 2024-09-11T07:35:20.882 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:20.882 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:20.882 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:35:20.882 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:20.863464+0000", 2024-09-11T07:35:20.882 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0038857879999999998 2024-09-11T07:35:20.882 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:20.882 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:20.882 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:20.882 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:20.882 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:35:20.882 INFO:teuthology.orchestra.run.smithi022.stdout: "num_ops": 3 2024-09-11T07:35:20.882 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:35:20.918 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:20.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:20.919 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:21.020 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:21.058 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:21.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:21.059 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:21.160 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:21.189 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:21.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:21.190 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:21.211 INFO:tasks.thrashosds.thrasher:j is {'epoch': 216, 'fsid': '12a285d8-700e-11ef-bcea-c7b262605968', 'created': '2024-09-11T07:21:18.695332+0000', 'modified': '2024-09-11T07:35:11.174281+0000', 'last_up_change': '2024-09-11T07:32:00.070825+0000', 'last_in_change': '2024-09-11T07:25:58.551813+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 19, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 8, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T07:24:21.772426+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': '.rgw.root', 'create_time': '2024-09-11T07:26:29.941354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 3, 'pool_name': 'datapool', 'create_time': '2024-09-11T07:26:30.396859+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-11T07:26:31.922926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-11T07:26:34.007746+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-11T07:26:36.077215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}, {'pool': 8, 'pool_name': 'unique_pool_1', 'create_time': '2024-09-11T07:33:28.041392+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '184', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '09075f16-32f4-4f88-a116-c3d7f977a0e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 177, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6801', 'nonce': 2821493591}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6802', 'nonce': 2821493591}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6804', 'nonce': 2821493591}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6803', 'nonce': 2821493591}]}, 'public_addr': '172.21.15.22:6801/2821493591', 'cluster_addr': '172.21.15.22:6802/2821493591', 'heartbeat_back_addr': '172.21.15.22:6804/2821493591', 'heartbeat_front_addr': '172.21.15.22:6803/2821493591', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '52cc1e06-9506-48c0-9f7b-d4270dff4b9f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 177, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6805', 'nonce': 4037911317}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6806', 'nonce': 4037911317}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6808', 'nonce': 4037911317}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6807', 'nonce': 4037911317}]}, 'public_addr': '172.21.15.22:6805/4037911317', 'cluster_addr': '172.21.15.22:6806/4037911317', 'heartbeat_back_addr': '172.21.15.22:6808/4037911317', 'heartbeat_front_addr': '172.21.15.22:6807/4037911317', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'fa2b2730-2c1d-45d0-be8a-494a60dc6b9b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 161, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6809', 'nonce': 1675799666}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6810', 'nonce': 1675799666}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6812', 'nonce': 1675799666}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6811', 'nonce': 1675799666}]}, 'public_addr': '172.21.15.22:6809/1675799666', 'cluster_addr': '172.21.15.22:6810/1675799666', 'heartbeat_back_addr': '172.21.15.22:6812/1675799666', 'heartbeat_front_addr': '172.21.15.22:6811/1675799666', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '8d53d3fc-e401-47df-b70b-4b4b1f4539df', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 25, 'up_thru': 177, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6813', 'nonce': 1501962752}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6814', 'nonce': 1501962752}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6816', 'nonce': 1501962752}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6815', 'nonce': 1501962752}]}, 'public_addr': '172.21.15.22:6813/1501962752', 'cluster_addr': '172.21.15.22:6814/1501962752', 'heartbeat_back_addr': '172.21.15.22:6816/1501962752', 'heartbeat_front_addr': '172.21.15.22:6815/1501962752', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'd3207fe0-8b97-4067-a755-ba6d449c53c7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 212, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6800', 'nonce': 1210229443}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6801', 'nonce': 1210229443}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6803', 'nonce': 1210229443}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6802', 'nonce': 1210229443}]}, 'public_addr': '172.21.15.104:6800/1210229443', 'cluster_addr': '172.21.15.104:6801/1210229443', 'heartbeat_back_addr': '172.21.15.104:6803/1210229443', 'heartbeat_front_addr': '172.21.15.104:6802/1210229443', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '89c9352b-30b2-48c6-869e-047d07a23a43', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 37, 'up_thru': 177, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6804', 'nonce': 2735328732}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6805', 'nonce': 2735328732}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6807', 'nonce': 2735328732}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6806', 'nonce': 2735328732}]}, 'public_addr': '172.21.15.104:6804/2735328732', 'cluster_addr': '172.21.15.104:6805/2735328732', 'heartbeat_back_addr': '172.21.15.104:6807/2735328732', 'heartbeat_front_addr': '172.21.15.104:6806/2735328732', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '704563de-d88f-480a-b469-720135704d42', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 44, 'up_thru': 177, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6808', 'nonce': 721493010}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6809', 'nonce': 721493010}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6811', 'nonce': 721493010}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6810', 'nonce': 721493010}]}, 'public_addr': '172.21.15.104:6808/721493010', 'cluster_addr': '172.21.15.104:6809/721493010', 'heartbeat_back_addr': '172.21.15.104:6811/721493010', 'heartbeat_front_addr': '172.21.15.104:6810/721493010', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '18199e47-0052-49f0-b45f-0fb201e0d0f4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 52, 'last_clean_end': 133, 'up_from': 147, 'up_thru': 167, 'down_at': 134, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6812', 'nonce': 2218863853}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6813', 'nonce': 2218863853}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6815', 'nonce': 2218863853}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6814', 'nonce': 2218863853}]}, 'public_addr': '172.21.15.104:6812/2218863853', 'cluster_addr': '172.21.15.104:6813/2218863853', 'heartbeat_back_addr': '172.21.15.104:6815/2218863853', 'heartbeat_front_addr': '172.21.15.104:6814/2218863853', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:33:53.829080+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:35:04.552209+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:33:51.706870+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:34:52.765101+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:34:57.918667+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:34:29.436631+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:34:28.486370+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:34:54.457031+0000', 'dead_epoch': 133}], 'pg_upmap': [{'pgid': '2.1c', 'osds': [5, 2, 7]}, {'pgid': '4.1e', 'osds': [4, 5, 3]}], 'pg_upmap_items': [{'pgid': '2.7', 'mappings': [{'from': 7, 'to': 4}]}, {'pgid': '4.4', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.6', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.14', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.22:0/240999923': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:6800/4016369460': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/585929238': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/544031024': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:6800/2504196132': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/1451388349': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/639336081': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/2104069730': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/4236450731': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:6800/3573193776': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/2661764422': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/2197488530': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/2968034970': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/599318175': '2024-09-12T07:27:07.919998+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-11T07:35:21.211 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:35:21.291 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:21.320 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:21.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:21.321 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:21.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:21 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:21.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:21 smithi022 ceph-mon[29462]: pgmap v393: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 4.6 MiB/s wr, 1.09k op/s 2024-09-11T07:35:21.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:21 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:35:21.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:21 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:35:21.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:21 smithi022 ceph-mon[29462]: osdmap e217: 8 total, 8 up, 8 in 2024-09-11T07:35:21.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:21 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:21.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:21 smithi022 ceph-mon[37330]: pgmap v393: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 4.6 MiB/s wr, 1.09k op/s 2024-09-11T07:35:21.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:21 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:35:21.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:21 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:35:21.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:21 smithi022 ceph-mon[37330]: osdmap e217: 8 total, 8 up, 8 in 2024-09-11T07:35:21.421 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:21.479 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:21.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:21.480 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:21.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:21 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:21 smithi104 ceph-mon[33507]: pgmap v393: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 4.6 MiB/s wr, 1.09k op/s 2024-09-11T07:35:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:21 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:35:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:21 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:35:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:21 smithi104 ceph-mon[33507]: osdmap e217: 8 total, 8 up, 8 in 2024-09-11T07:35:21.581 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:21.632 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:21.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:21.634 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:21.734 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:21.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:21.759 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:21.759 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:21.860 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:21.898 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:21.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:21.899 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:22.000 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:22.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:22.050 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:22.051 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:22.152 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:22.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:22.184 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:22.184 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:22.285 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:22.313 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:22.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:22.314 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:22.415 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:22.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:22.449 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:22.449 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:22.451 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:35:22.459 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-11T07:35:22.550 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:22.573 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:22.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:22.574 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:22.675 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:22.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:22.702 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:22.702 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:22.803 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:22.848 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:22.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:22.849 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:22.949 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:22.978 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:22.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:22.979 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:23.080 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:23.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:23.121 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:23.121 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:23.221 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is unset 2024-09-11T07:35:23.223 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:23.237 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:35:23.278 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:23.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:23.279 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:23.380 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:23.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:23.417 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:23.417 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:23.519 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:23.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:23 smithi104 ceph-mon[33507]: pgmap v395: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 4.6 MiB/s wr, 1.09k op/s 2024-09-11T07:35:23.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:23 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2725394625' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:35:23.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:23 smithi104 ceph-mon[33507]: from='client.25181 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:35:23.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:35:23.556 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:23.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:23.557 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:23.560 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:23 smithi022 ceph-mon[29462]: pgmap v395: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 4.6 MiB/s wr, 1.09k op/s 2024-09-11T07:35:23.560 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:23 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2725394625' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:35:23.560 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:23 smithi022 ceph-mon[29462]: from='client.25181 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:35:23.560 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:35:23.560 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:23 smithi022 ceph-mon[37330]: pgmap v395: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 4.6 MiB/s wr, 1.09k op/s 2024-09-11T07:35:23.561 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:23 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2725394625' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:35:23.561 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:23 smithi022 ceph-mon[37330]: from='client.25181 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:35:23.561 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:35:23.658 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:23.697 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:23.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:23.698 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:23.799 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:23.834 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:23.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:23.836 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:23.936 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:23.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:23.977 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:23.977 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:24.078 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:24.117 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:24.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:24.118 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:24.219 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:24.264 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:24.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:24.266 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:24.367 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:24.411 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:24.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:24.412 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:24.431 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:35:24 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:35:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:35:24.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:24 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:35:24.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:24 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:35:24.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:24 smithi104 ceph-mon[33507]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:35:24.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:24 smithi104 ceph-mon[33507]: from='client.25181 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:35:24.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:24 smithi104 ceph-mon[33507]: osdmap e218: 8 total, 8 up, 8 in 2024-09-11T07:35:24.465 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:24 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:35:24.465 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:24 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:35:24.465 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:24 smithi022 ceph-mon[29462]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:35:24.465 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:24 smithi022 ceph-mon[29462]: from='client.25181 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:35:24.465 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:24 smithi022 ceph-mon[29462]: osdmap e218: 8 total, 8 up, 8 in 2024-09-11T07:35:24.466 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:24 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:35:24.466 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:24 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:35:24.466 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:24 smithi022 ceph-mon[37330]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:35:24.466 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:24 smithi022 ceph-mon[37330]: from='client.25181 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:35:24.466 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:24 smithi022 ceph-mon[37330]: osdmap e218: 8 total, 8 up, 8 in 2024-09-11T07:35:24.513 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:24.545 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:24.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:24.546 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:24.647 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:24.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:24.684 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:24.684 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:24.785 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:24.818 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:24.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:24.820 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:24.920 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:24.970 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:24.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:24.971 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:25.072 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:25.114 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:25.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:25.115 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:25.216 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:25.248 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:25.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:25.249 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:25.260 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:35:25.350 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:25.360 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:25 smithi104 ceph-mon[33507]: pgmap v397: 148 pgs: 148 active+clean; 7.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 957 op/s 2024-09-11T07:35:25.360 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:25 smithi104 ceph-mon[33507]: 4.1 scrub starts 2024-09-11T07:35:25.360 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:25 smithi104 ceph-mon[33507]: 4.1 scrub ok 2024-09-11T07:35:25.394 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:25.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:25.395 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:25.496 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:25.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:25.534 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:25.534 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:25.635 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:25.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:25 smithi022 ceph-mon[29462]: pgmap v397: 148 pgs: 148 active+clean; 7.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 957 op/s 2024-09-11T07:35:25.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:25 smithi022 ceph-mon[29462]: 4.1 scrub starts 2024-09-11T07:35:25.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:25 smithi022 ceph-mon[29462]: 4.1 scrub ok 2024-09-11T07:35:25.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:25 smithi022 ceph-mon[37330]: pgmap v397: 148 pgs: 148 active+clean; 7.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 957 op/s 2024-09-11T07:35:25.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:25 smithi022 ceph-mon[37330]: 4.1 scrub starts 2024-09-11T07:35:25.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:25 smithi022 ceph-mon[37330]: 4.1 scrub ok 2024-09-11T07:35:25.674 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:25.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:25.675 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:25.776 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:25.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:25.813 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:25.814 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:25.914 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:25.964 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:25.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:25.966 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:26.066 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:26.102 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:26.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:26.103 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:26.204 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:26.224 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T07:35:26.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:26.241 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:26.242 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:26.342 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:26.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:26.402 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:26.402 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:26.503 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:26.547 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:26.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:26.550 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:26.642 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:26 smithi022 ceph-mon[29462]: 8.5 scrub starts 2024-09-11T07:35:26.642 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:26 smithi022 ceph-mon[29462]: 4.1b scrub starts 2024-09-11T07:35:26.643 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:26 smithi022 ceph-mon[29462]: 4.1b scrub ok 2024-09-11T07:35:26.643 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:26 smithi022 ceph-mon[29462]: pgmap v398: 148 pgs: 148 active+clean; 6.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.53k op/s 2024-09-11T07:35:26.643 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:26 smithi022 ceph-mon[29462]: 4.16 scrub starts 2024-09-11T07:35:26.643 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:26 smithi022 ceph-mon[29462]: 4.16 scrub ok 2024-09-11T07:35:26.643 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:26 smithi022 ceph-mon[37330]: 8.5 scrub starts 2024-09-11T07:35:26.643 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:26 smithi022 ceph-mon[37330]: 4.1b scrub starts 2024-09-11T07:35:26.643 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:26 smithi022 ceph-mon[37330]: 4.1b scrub ok 2024-09-11T07:35:26.643 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:26 smithi022 ceph-mon[37330]: pgmap v398: 148 pgs: 148 active+clean; 6.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.53k op/s 2024-09-11T07:35:26.643 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:26 smithi022 ceph-mon[37330]: 4.16 scrub starts 2024-09-11T07:35:26.643 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:26 smithi022 ceph-mon[37330]: 4.16 scrub ok 2024-09-11T07:35:26.651 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:26.697 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:26.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:26.699 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:26.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:26 smithi104 ceph-mon[33507]: 8.5 scrub starts 2024-09-11T07:35:26.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:26 smithi104 ceph-mon[33507]: 4.1b scrub starts 2024-09-11T07:35:26.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:26 smithi104 ceph-mon[33507]: 4.1b scrub ok 2024-09-11T07:35:26.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:26 smithi104 ceph-mon[33507]: pgmap v398: 148 pgs: 148 active+clean; 6.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.53k op/s 2024-09-11T07:35:26.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:26 smithi104 ceph-mon[33507]: 4.16 scrub starts 2024-09-11T07:35:26.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:26 smithi104 ceph-mon[33507]: 4.16 scrub ok 2024-09-11T07:35:26.800 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:26.814 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:35:26.815 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:35:26.815 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:35:26.815 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:35:26.815 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:35:26.815 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:35:26.815 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:35:26.815 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:35:26.815 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:35:26.815 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:35:26.817 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:35:26.817 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:35:26.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:26.840 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:26.840 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:26.849 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:35:26.849 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:35:26.879 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:35:26.880 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:35:26.941 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:26.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:26.973 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:26.973 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:27.074 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:27.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:27.102 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:27.103 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:27.203 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:27.229 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:27.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:27.230 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:27.331 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:27.371 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:27.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:27.372 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:27.473 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:27.511 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:27.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:27.513 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:27.571 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:27 smithi022 ceph-mon[29462]: 6.1f scrub starts 2024-09-11T07:35:27.572 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:27 smithi022 ceph-mon[29462]: 6.1f scrub ok 2024-09-11T07:35:27.613 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:27.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:27.647 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:27.647 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:27.747 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:27.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:27.789 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:27.789 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:27.893 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:27.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:27 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/108741193' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:35:27.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:27 smithi022 ceph-mon[37330]: 6.1f scrub starts 2024-09-11T07:35:27.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:27 smithi022 ceph-mon[37330]: 6.1f scrub ok 2024-09-11T07:35:27.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:27 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/108741193' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:35:27.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:27.933 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:27.934 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:28.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:27 smithi104 ceph-mon[33507]: 6.1f scrub starts 2024-09-11T07:35:28.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:27 smithi104 ceph-mon[33507]: 6.1f scrub ok 2024-09-11T07:35:28.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:27 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/108741193' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:35:28.034 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:28.075 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:28.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:28.076 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:28.177 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:28.219 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:28.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:28.220 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:28.321 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:28.352 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:28.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:28.353 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:28.454 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:28.497 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:28.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:28.498 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:28.599 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:28.618 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:35:28.652 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:28.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:28.653 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:28.714 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:28 smithi022 ceph-mon[29462]: 8.5 scrub ok 2024-09-11T07:35:28.714 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:28 smithi022 ceph-mon[29462]: 8.8 scrub starts 2024-09-11T07:35:28.714 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:28 smithi022 ceph-mon[29462]: 2.6 scrub starts 2024-09-11T07:35:28.714 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:28 smithi022 ceph-mon[29462]: 2.6 scrub ok 2024-09-11T07:35:28.715 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:28 smithi022 ceph-mon[29462]: pgmap v399: 148 pgs: 148 active+clean; 6.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 762 op/s 2024-09-11T07:35:28.715 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:28 smithi022 ceph-mon[37330]: 8.5 scrub ok 2024-09-11T07:35:28.715 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:28 smithi022 ceph-mon[37330]: 8.8 scrub starts 2024-09-11T07:35:28.715 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:28 smithi022 ceph-mon[37330]: 2.6 scrub starts 2024-09-11T07:35:28.715 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:28 smithi022 ceph-mon[37330]: 2.6 scrub ok 2024-09-11T07:35:28.715 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:28 smithi022 ceph-mon[37330]: pgmap v399: 148 pgs: 148 active+clean; 6.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 762 op/s 2024-09-11T07:35:28.754 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:28.803 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:28.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:28.804 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:28.905 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:28.936 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:28.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:28.938 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:29.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:28 smithi104 ceph-mon[33507]: 8.5 scrub ok 2024-09-11T07:35:29.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:28 smithi104 ceph-mon[33507]: 8.8 scrub starts 2024-09-11T07:35:29.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:28 smithi104 ceph-mon[33507]: 2.6 scrub starts 2024-09-11T07:35:29.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:28 smithi104 ceph-mon[33507]: 2.6 scrub ok 2024-09-11T07:35:29.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:28 smithi104 ceph-mon[33507]: pgmap v399: 148 pgs: 148 active+clean; 6.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 762 op/s 2024-09-11T07:35:29.038 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:29.072 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:29.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:29.074 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:29.175 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:29.207 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:29.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:29.208 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:29.308 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:29.343 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:29.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:29.344 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:29.445 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:29.470 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:29.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:29.471 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:29.572 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:29.615 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:29.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:29.616 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:29.717 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:29.746 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:29.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:29.747 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:29.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:35:29 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:35:29.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:29 smithi104 ceph-mon[33507]: 6.6 scrub starts 2024-09-11T07:35:29.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:29 smithi104 ceph-mon[33507]: 6.6 scrub ok 2024-09-11T07:35:29.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:29 smithi104 ceph-mon[33507]: 2.14 scrub starts 2024-09-11T07:35:29.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:29 smithi104 ceph-mon[33507]: 2.14 scrub ok 2024-09-11T07:35:29.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:29 smithi104 ceph-mon[33507]: 5.8 scrub starts 2024-09-11T07:35:29.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:29 smithi104 ceph-mon[33507]: 5.8 scrub ok 2024-09-11T07:35:29.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:29 smithi104 ceph-mon[33507]: 4.e scrub starts 2024-09-11T07:35:29.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:29 smithi104 ceph-mon[33507]: 4.e scrub ok 2024-09-11T07:35:29.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:29 smithi104 ceph-mon[33507]: 8.a scrub starts 2024-09-11T07:35:29.803 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:29 smithi022 ceph-mon[29462]: 6.6 scrub starts 2024-09-11T07:35:29.803 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:29 smithi022 ceph-mon[29462]: 6.6 scrub ok 2024-09-11T07:35:29.803 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:29 smithi022 ceph-mon[29462]: 2.14 scrub starts 2024-09-11T07:35:29.803 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:29 smithi022 ceph-mon[29462]: 2.14 scrub ok 2024-09-11T07:35:29.803 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:29 smithi022 ceph-mon[29462]: 5.8 scrub starts 2024-09-11T07:35:29.804 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:29 smithi022 ceph-mon[29462]: 5.8 scrub ok 2024-09-11T07:35:29.804 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:29 smithi022 ceph-mon[29462]: 4.e scrub starts 2024-09-11T07:35:29.804 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:29 smithi022 ceph-mon[29462]: 4.e scrub ok 2024-09-11T07:35:29.804 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:29 smithi022 ceph-mon[29462]: 8.a scrub starts 2024-09-11T07:35:29.804 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:29 smithi022 ceph-mon[37330]: 6.6 scrub starts 2024-09-11T07:35:29.804 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:29 smithi022 ceph-mon[37330]: 6.6 scrub ok 2024-09-11T07:35:29.804 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:29 smithi022 ceph-mon[37330]: 2.14 scrub starts 2024-09-11T07:35:29.804 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:29 smithi022 ceph-mon[37330]: 2.14 scrub ok 2024-09-11T07:35:29.804 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:29 smithi022 ceph-mon[37330]: 5.8 scrub starts 2024-09-11T07:35:29.804 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:29 smithi022 ceph-mon[37330]: 5.8 scrub ok 2024-09-11T07:35:29.804 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:29 smithi022 ceph-mon[37330]: 4.e scrub starts 2024-09-11T07:35:29.805 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:29 smithi022 ceph-mon[37330]: 4.e scrub ok 2024-09-11T07:35:29.805 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:29 smithi022 ceph-mon[37330]: 8.a scrub starts 2024-09-11T07:35:29.848 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:29.875 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:29.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:29.876 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:29.977 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:29.985 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:35:29.988 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:35:30.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:30.026 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:30.026 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:30.053 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:35:30.127 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:30.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:30.180 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:30.181 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:30.282 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:30.310 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:30.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:30.312 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:30.413 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:30.439 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:30.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:30.440 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:30.541 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:30.573 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:30.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:30.574 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:30.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:35:30] "GET /metrics HTTP/1.1" 200 224126 "" "Prometheus/2.43.0" 2024-09-11T07:35:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[29462]: 4.a scrub starts 2024-09-11T07:35:30.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[29462]: 4.a scrub ok 2024-09-11T07:35:30.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[29462]: 5.f scrub starts 2024-09-11T07:35:30.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[29462]: 5.f scrub ok 2024-09-11T07:35:30.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:30.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[29462]: 8.8 scrub ok 2024-09-11T07:35:30.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[29462]: 5.19 scrub starts 2024-09-11T07:35:30.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[29462]: 5.19 scrub ok 2024-09-11T07:35:30.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[29462]: from='client.15330 v1:172.21.15.22:0/1822719373' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:35:30.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[29462]: pgmap v400: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T07:35:30.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[29462]: 5.1e scrub starts 2024-09-11T07:35:30.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[29462]: 5.1e scrub ok 2024-09-11T07:35:30.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[29462]: 6.12 scrub starts 2024-09-11T07:35:30.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[29462]: 6.12 scrub ok 2024-09-11T07:35:30.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[37330]: 4.a scrub starts 2024-09-11T07:35:30.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[37330]: 4.a scrub ok 2024-09-11T07:35:30.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[37330]: 5.f scrub starts 2024-09-11T07:35:30.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[37330]: 5.f scrub ok 2024-09-11T07:35:30.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:30.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[37330]: 8.8 scrub ok 2024-09-11T07:35:30.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[37330]: 5.19 scrub starts 2024-09-11T07:35:30.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[37330]: 5.19 scrub ok 2024-09-11T07:35:30.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[37330]: from='client.15330 v1:172.21.15.22:0/1822719373' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:35:30.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[37330]: pgmap v400: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T07:35:30.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[37330]: 5.1e scrub starts 2024-09-11T07:35:30.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[37330]: 5.1e scrub ok 2024-09-11T07:35:30.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[37330]: 6.12 scrub starts 2024-09-11T07:35:30.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:30 smithi022 ceph-mon[37330]: 6.12 scrub ok 2024-09-11T07:35:30.675 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:30.711 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:30.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:30.712 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:30.813 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:30.846 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:30.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:30.847 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:30.948 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:31.002 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:31.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:31.003 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:31.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:30 smithi104 ceph-mon[33507]: 4.a scrub starts 2024-09-11T07:35:31.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:30 smithi104 ceph-mon[33507]: 4.a scrub ok 2024-09-11T07:35:31.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:30 smithi104 ceph-mon[33507]: 5.f scrub starts 2024-09-11T07:35:31.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:30 smithi104 ceph-mon[33507]: 5.f scrub ok 2024-09-11T07:35:31.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:30 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:31.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:30 smithi104 ceph-mon[33507]: 8.8 scrub ok 2024-09-11T07:35:31.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:30 smithi104 ceph-mon[33507]: 5.19 scrub starts 2024-09-11T07:35:31.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:30 smithi104 ceph-mon[33507]: 5.19 scrub ok 2024-09-11T07:35:31.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:30 smithi104 ceph-mon[33507]: from='client.15330 v1:172.21.15.22:0/1822719373' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:35:31.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:30 smithi104 ceph-mon[33507]: pgmap v400: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T07:35:31.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:30 smithi104 ceph-mon[33507]: 5.1e scrub starts 2024-09-11T07:35:31.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:30 smithi104 ceph-mon[33507]: 5.1e scrub ok 2024-09-11T07:35:31.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:30 smithi104 ceph-mon[33507]: 6.12 scrub starts 2024-09-11T07:35:31.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:30 smithi104 ceph-mon[33507]: 6.12 scrub ok 2024-09-11T07:35:31.041 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:35:31.041 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:35:31.104 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:31.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:31.156 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:31.156 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:31.257 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:31.293 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:31.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:31.295 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:31.395 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:31.430 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:31.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:31.432 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:31.533 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:31.550 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-09-11T07:35:28.125787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7360813300,"num_objects":112505,"num_object_clones":0,"num_object_copies":337515,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112505,"num_whiteouts":0,"num_read":1379,"num_read_kb":1236,"num_write":145292,"num_write_kb":8227464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":591166,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86046,"ondisk_log_size":86046,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":24188616,"kb_used_data":21728048,"kb_used_omap":16,"kb_used_meta":2459951,"kb_avail":725772600,"statfs":{"total":767960285184,"available":743191142400,"internally_reserved":0,"allocated":22249521152,"data_stored":22246577164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16530,"internal_metadata":2518990702},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,5,10,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-399507456,"num_objects":-6096,"num_object_clones":0,"num_object_copies":-18288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6096,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":6096,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004432"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":201,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.211986+0000","last_change":"2024-09-11T07:33:08.464899+0000","last_active":"2024-09-11T07:35:20.211986+0000","last_peered":"2024-09-11T07:35:20.211986+0000","last_clean":"2024-09-11T07:35:20.211986+0000","last_became_active":"2024-09-11T07:32:00.501534+0000","last_became_peered":"2024-09-11T07:32:00.501534+0000","last_unstale":"2024-09-11T07:35:20.211986+0000","last_undegraded":"2024-09-11T07:35:20.211986+0000","last_fullsized":"2024-09-11T07:35:20.211986+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:08.464835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:08.464835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002103452,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":226,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.214535+0000","last_change":"2024-09-11T07:34:09.538289+0000","last_active":"2024-09-11T07:35:20.214535+0000","last_peered":"2024-09-11T07:35:20.214535+0000","last_clean":"2024-09-11T07:35:20.214535+0000","last_became_active":"2024-09-11T07:32:00.508572+0000","last_became_peered":"2024-09-11T07:32:00.508572+0000","last_unstale":"2024-09-11T07:35:20.214535+0000","last_undegraded":"2024-09-11T07:35:20.214535+0000","last_fullsized":"2024-09-11T07:35:20.214535+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:09.538205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:09.538205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:23.803280+0000","scrub_duration":0.0045320439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"82'22","reported_seq":230,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.212085+0000","last_change":"2024-09-11T07:33:53.615330+0000","last_active":"2024-09-11T07:35:20.212085+0000","last_peered":"2024-09-11T07:35:20.212085+0000","last_clean":"2024-09-11T07:35:20.212085+0000","last_became_active":"2024-09-11T07:26:33.544627+0000","last_became_peered":"2024-09-11T07:26:33.544627+0000","last_unstale":"2024-09-11T07:35:20.212085+0000","last_undegraded":"2024-09-11T07:35:20.212085+0000","last_fullsized":"2024-09-11T07:35:20.212085+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-09-11T07:33:53.615231+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:33:53.615231+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011220367,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":205,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.210473+0000","last_change":"2024-09-11T07:33:18.247208+0000","last_active":"2024-09-11T07:35:20.210473+0000","last_peered":"2024-09-11T07:35:20.210473+0000","last_clean":"2024-09-11T07:35:20.210473+0000","last_became_active":"2024-09-11T07:26:35.556296+0000","last_became_peered":"2024-09-11T07:26:35.556296+0000","last_unstale":"2024-09-11T07:35:20.210473+0000","last_undegraded":"2024-09-11T07:35:20.210473+0000","last_fullsized":"2024-09-11T07:35:20.210473+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:18.247138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:18.247138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001401198,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":210,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.213150+0000","last_change":"2024-09-11T07:34:38.766278+0000","last_active":"2024-09-11T07:35:20.213150+0000","last_peered":"2024-09-11T07:35:20.213150+0000","last_clean":"2024-09-11T07:35:20.213150+0000","last_became_active":"2024-09-11T07:26:31.420178+0000","last_became_peered":"2024-09-11T07:26:31.420178+0000","last_unstale":"2024-09-11T07:35:20.213150+0000","last_undegraded":"2024-09-11T07:35:20.213150+0000","last_fullsized":"2024-09-11T07:35:20.213150+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:38.766188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:38.766188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:46.651091+0000","scrub_duration":0.022341764,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"86'13","reported_seq":227,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.210541+0000","last_change":"2024-09-11T07:34:34.251712+0000","last_active":"2024-09-11T07:35:20.210541+0000","last_peered":"2024-09-11T07:35:20.210541+0000","last_clean":"2024-09-11T07:35:20.210541+0000","last_became_active":"2024-09-11T07:26:33.540732+0000","last_became_peered":"2024-09-11T07:26:33.540732+0000","last_unstale":"2024-09-11T07:35:20.210541+0000","last_undegraded":"2024-09-11T07:35:20.210541+0000","last_fullsized":"2024-09-11T07:35:20.210541+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'13","last_scrub_stamp":"2024-09-11T07:34:34.251622+0000","last_deep_scrub":"86'13","last_deep_scrub_stamp":"2024-09-11T07:28:15.025757+0000","last_clean_scrub_stamp":"2024-09-11T07:34:34.251622+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:34.755765+0000","scrub_duration":0.0040986499999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":214,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.224353+0000","last_change":"2024-09-11T07:33:49.237639+0000","last_active":"2024-09-11T07:35:20.224353+0000","last_peered":"2024-09-11T07:35:20.224353+0000","last_clean":"2024-09-11T07:35:20.224353+0000","last_became_active":"2024-09-11T07:32:00.513650+0000","last_became_peered":"2024-09-11T07:32:00.513650+0000","last_unstale":"2024-09-11T07:35:20.224353+0000","last_undegraded":"2024-09-11T07:35:20.224353+0000","last_fullsized":"2024-09-11T07:35:20.224353+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:49.237335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:33:49.237335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052103979999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":203,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.212437+0000","last_change":"2024-09-11T07:34:41.315649+0000","last_active":"2024-09-11T07:35:20.212437+0000","last_peered":"2024-09-11T07:35:20.212437+0000","last_clean":"2024-09-11T07:35:20.212437+0000","last_became_active":"2024-09-11T07:26:37.566910+0000","last_became_peered":"2024-09-11T07:26:37.566910+0000","last_unstale":"2024-09-11T07:35:20.212437+0000","last_undegraded":"2024-09-11T07:35:20.212437+0000","last_fullsized":"2024-09-11T07:35:20.212437+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:41.315562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:41.315562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:44.355231+0000","scrub_duration":0.002327371,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":205,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.226746+0000","last_change":"2024-09-11T07:33:57.319606+0000","last_active":"2024-09-11T07:35:20.226746+0000","last_peered":"2024-09-11T07:35:20.226746+0000","last_clean":"2024-09-11T07:35:20.226746+0000","last_became_active":"2024-09-11T07:32:00.516662+0000","last_became_peered":"2024-09-11T07:32:00.516662+0000","last_unstale":"2024-09-11T07:35:20.226746+0000","last_undegraded":"2024-09-11T07:35:20.226746+0000","last_fullsized":"2024-09-11T07:35:20.226746+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:57.319449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:57.319449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007631344,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"82'7","reported_seq":219,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.211987+0000","last_change":"2024-09-11T07:31:43.035948+0000","last_active":"2024-09-11T07:35:20.211987+0000","last_peered":"2024-09-11T07:35:20.211987+0000","last_clean":"2024-09-11T07:35:20.211987+0000","last_became_active":"2024-09-11T07:26:33.540351+0000","last_became_peered":"2024-09-11T07:26:33.540351+0000","last_unstale":"2024-09-11T07:35:20.211987+0000","last_undegraded":"2024-09-11T07:35:20.211987+0000","last_fullsized":"2024-09-11T07:35:20.211987+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'7","last_scrub_stamp":"2024-09-11T07:31:43.035896+0000","last_deep_scrub":"82'7","last_deep_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_clean_scrub_stamp":"2024-09-11T07:31:43.035896+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040919600000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":181,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.225166+0000","last_change":"2024-09-11T07:34:47.230562+0000","last_active":"2024-09-11T07:35:20.225166+0000","last_peered":"2024-09-11T07:35:20.225166+0000","last_clean":"2024-09-11T07:35:20.225166+0000","last_became_active":"2024-09-11T07:32:00.513206+0000","last_became_peered":"2024-09-11T07:32:00.513206+0000","last_unstale":"2024-09-11T07:35:20.225166+0000","last_undegraded":"2024-09-11T07:35:20.225166+0000","last_fullsized":"2024-09-11T07:35:20.225166+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:47.230449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.973323+0000","last_clean_scrub_stamp":"2024-09-11T07:34:47.230449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:47.951327+0000","scrub_duration":0.0044176479999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":194,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.218749+0000","last_change":"2024-09-11T07:34:03.257034+0000","last_active":"2024-09-11T07:35:20.218749+0000","last_peered":"2024-09-11T07:35:20.218749+0000","last_clean":"2024-09-11T07:35:20.218749+0000","last_became_active":"2024-09-11T07:26:37.561758+0000","last_became_peered":"2024-09-11T07:26:37.561758+0000","last_unstale":"2024-09-11T07:35:20.218749+0000","last_undegraded":"2024-09-11T07:35:20.218749+0000","last_fullsized":"2024-09-11T07:35:20.218749+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:03.256953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:03.256953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0085144460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":170,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.220186+0000","last_change":"2024-09-11T07:34:01.257321+0000","last_active":"2024-09-11T07:35:20.220186+0000","last_peered":"2024-09-11T07:35:20.220186+0000","last_clean":"2024-09-11T07:35:20.220186+0000","last_became_active":"2024-09-11T07:32:00.505709+0000","last_became_peered":"2024-09-11T07:32:00.505709+0000","last_unstale":"2024-09-11T07:35:20.220186+0000","last_undegraded":"2024-09-11T07:35:20.220186+0000","last_fullsized":"2024-09-11T07:35:20.220186+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:01.257176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_clean_scrub_stamp":"2024-09-11T07:34:01.257176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018917890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"136'13","reported_seq":227,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.212774+0000","last_change":"2024-09-11T07:34:06.406309+0000","last_active":"2024-09-11T07:35:20.212774+0000","last_peered":"2024-09-11T07:35:20.212774+0000","last_clean":"2024-09-11T07:35:20.212774+0000","last_became_active":"2024-09-11T07:26:33.544386+0000","last_became_peered":"2024-09-11T07:26:33.544386+0000","last_unstale":"2024-09-11T07:35:20.212774+0000","last_undegraded":"2024-09-11T07:35:20.212774+0000","last_fullsized":"2024-09-11T07:35:20.212774+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'13","last_scrub_stamp":"2024-09-11T07:34:06.406196+0000","last_deep_scrub":"136'13","last_deep_scrub_stamp":"2024-09-11T07:31:08.042198+0000","last_clean_scrub_stamp":"2024-09-11T07:34:06.406196+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079557350000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":216,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.219057+0000","last_change":"2024-09-11T07:33:49.134068+0000","last_active":"2024-09-11T07:35:20.219057+0000","last_peered":"2024-09-11T07:35:20.219057+0000","last_clean":"2024-09-11T07:35:20.219057+0000","last_became_active":"2024-09-11T07:32:00.502337+0000","last_became_peered":"2024-09-11T07:32:00.502337+0000","last_unstale":"2024-09-11T07:35:20.219057+0000","last_undegraded":"2024-09-11T07:35:20.219057+0000","last_fullsized":"2024-09-11T07:35:20.219057+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:49.133995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:49.133995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028580369999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":220,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.214649+0000","last_change":"2024-09-11T07:34:33.740699+0000","last_active":"2024-09-11T07:35:20.214649+0000","last_peered":"2024-09-11T07:35:20.214649+0000","last_clean":"2024-09-11T07:35:20.214649+0000","last_became_active":"2024-09-11T07:32:00.530335+0000","last_became_peered":"2024-09-11T07:32:00.530335+0000","last_unstale":"2024-09-11T07:35:20.214649+0000","last_undegraded":"2024-09-11T07:35:20.214649+0000","last_fullsized":"2024-09-11T07:35:20.214649+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:33.740593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:33.740593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:36.438261+0000","scrub_duration":0.014153393,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"218'9306","reported_seq":9354,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T07:35:24.227976+0000","last_change":"2024-09-11T07:33:29.155481+0000","last_active":"2024-09-11T07:35:24.227976+0000","last_peered":"2024-09-11T07:35:24.227976+0000","last_clean":"2024-09-11T07:35:24.227976+0000","last_became_active":"2024-09-11T07:33:29.155187+0000","last_became_peered":"2024-09-11T07:33:29.155187+0000","last_unstale":"2024-09-11T07:35:24.227976+0000","last_undegraded":"2024-09-11T07:35:24.227976+0000","last_fullsized":"2024-09-11T07:35:24.227976+0000","mapping_epoch":177,"log_start":"199'4300","ondisk_log_start":"199'4300","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":5006,"log_dups_size":0,"ondisk_log_size":5006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":475004928,"num_objects":7248,"num_object_clones":0,"num_object_copies":21744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9306,"num_write_kb":529728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":291,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.213095+0000","last_change":"2024-09-11T07:34:11.315441+0000","last_active":"2024-09-11T07:35:20.213095+0000","last_peered":"2024-09-11T07:35:20.213095+0000","last_clean":"2024-09-11T07:35:20.213095+0000","last_became_active":"2024-09-11T07:29:51.482755+0000","last_became_peered":"2024-09-11T07:29:51.482755+0000","last_unstale":"2024-09-11T07:35:20.213095+0000","last_undegraded":"2024-09-11T07:35:20.213095+0000","last_fullsized":"2024-09-11T07:35:20.213095+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-09-11T07:34:11.315334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:34:11.315334+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071604249999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":200,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.225713+0000","last_change":"2024-09-11T07:33:52.272651+0000","last_active":"2024-09-11T07:35:20.225713+0000","last_peered":"2024-09-11T07:35:20.225713+0000","last_clean":"2024-09-11T07:35:20.225713+0000","last_became_active":"2024-09-11T07:32:00.507191+0000","last_became_peered":"2024-09-11T07:32:00.507191+0000","last_unstale":"2024-09-11T07:35:20.225713+0000","last_undegraded":"2024-09-11T07:35:20.225713+0000","last_fullsized":"2024-09-11T07:35:20.225713+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:52.272573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:52.272573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023059880000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":204,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.222563+0000","last_change":"2024-09-11T07:34:00.348790+0000","last_active":"2024-09-11T07:35:20.222563+0000","last_peered":"2024-09-11T07:35:20.222563+0000","last_clean":"2024-09-11T07:35:20.222563+0000","last_became_active":"2024-09-11T07:26:35.557989+0000","last_became_peered":"2024-09-11T07:26:35.557989+0000","last_unstale":"2024-09-11T07:35:20.222563+0000","last_undegraded":"2024-09-11T07:35:20.222563+0000","last_fullsized":"2024-09-11T07:35:20.222563+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:00.348684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:00.348684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:30.171094+0000","scrub_duration":0.0036577580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":204,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.224329+0000","last_change":"2024-09-11T07:33:57.219099+0000","last_active":"2024-09-11T07:35:20.224329+0000","last_peered":"2024-09-11T07:35:20.224329+0000","last_clean":"2024-09-11T07:35:20.224329+0000","last_became_active":"2024-09-11T07:26:37.564554+0000","last_became_peered":"2024-09-11T07:26:37.564554+0000","last_unstale":"2024-09-11T07:35:20.224329+0000","last_undegraded":"2024-09-11T07:35:20.224329+0000","last_fullsized":"2024-09-11T07:35:20.224329+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:57.219012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:57.219012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:26.433401+0000","scrub_duration":0.0062201569999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"218'9006","reported_seq":9055,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T07:35:24.650074+0000","last_change":"2024-09-11T07:33:29.161604+0000","last_active":"2024-09-11T07:35:24.650074+0000","last_peered":"2024-09-11T07:35:24.650074+0000","last_clean":"2024-09-11T07:35:24.650074+0000","last_became_active":"2024-09-11T07:33:29.161349+0000","last_became_peered":"2024-09-11T07:33:29.161349+0000","last_unstale":"2024-09-11T07:35:24.650074+0000","last_undegraded":"2024-09-11T07:35:24.650074+0000","last_fullsized":"2024-09-11T07:35:24.650074+0000","mapping_epoch":177,"log_start":"198'4000","ondisk_log_start":"198'4000","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":5006,"log_dups_size":0,"ondisk_log_size":5006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":456654848,"num_objects":6968,"num_object_clones":0,"num_object_copies":20904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9006,"num_write_kb":511168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"112'15","reported_seq":227,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.212404+0000","last_change":"2024-09-11T07:34:08.373729+0000","last_active":"2024-09-11T07:35:20.212404+0000","last_peered":"2024-09-11T07:35:20.212404+0000","last_clean":"2024-09-11T07:35:20.212404+0000","last_became_active":"2024-09-11T07:26:33.535411+0000","last_became_peered":"2024-09-11T07:26:33.535411+0000","last_unstale":"2024-09-11T07:35:20.212404+0000","last_undegraded":"2024-09-11T07:35:20.212404+0000","last_fullsized":"2024-09-11T07:35:20.212404+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"112'15","last_scrub_stamp":"2024-09-11T07:34:08.373652+0000","last_deep_scrub":"112'15","last_deep_scrub_stamp":"2024-09-11T07:31:10.005896+0000","last_clean_scrub_stamp":"2024-09-11T07:34:08.373652+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:35.031237+0000","scrub_duration":0.0048197400000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":218,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.226454+0000","last_change":"2024-09-11T07:33:50.250361+0000","last_active":"2024-09-11T07:35:20.226454+0000","last_peered":"2024-09-11T07:35:20.226454+0000","last_clean":"2024-09-11T07:35:20.226454+0000","last_became_active":"2024-09-11T07:32:00.506615+0000","last_became_peered":"2024-09-11T07:32:00.506615+0000","last_unstale":"2024-09-11T07:35:20.226454+0000","last_undegraded":"2024-09-11T07:35:20.226454+0000","last_fullsized":"2024-09-11T07:35:20.226454+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:50.250289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:50.250289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0080459799999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"151'10","reported_seq":339,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.234322+0000","last_change":"2024-09-11T07:34:33.960973+0000","last_active":"2024-09-11T07:35:20.234322+0000","last_peered":"2024-09-11T07:35:20.234322+0000","last_clean":"2024-09-11T07:35:20.234322+0000","last_became_active":"2024-09-11T07:32:00.507447+0000","last_became_peered":"2024-09-11T07:32:00.507447+0000","last_unstale":"2024-09-11T07:35:20.234322+0000","last_undegraded":"2024-09-11T07:35:20.234322+0000","last_fullsized":"2024-09-11T07:35:20.234322+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:34:33.960877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:33.960877+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:44.216730+0000","scrub_duration":0.0063436389999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":203,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.212685+0000","last_change":"2024-09-11T07:34:44.344252+0000","last_active":"2024-09-11T07:35:20.212685+0000","last_peered":"2024-09-11T07:35:20.212685+0000","last_clean":"2024-09-11T07:35:20.212685+0000","last_became_active":"2024-09-11T07:26:37.566919+0000","last_became_peered":"2024-09-11T07:26:37.566919+0000","last_unstale":"2024-09-11T07:35:20.212685+0000","last_undegraded":"2024-09-11T07:35:20.212685+0000","last_fullsized":"2024-09-11T07:35:20.212685+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:44.344176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:44.344176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:57.046656+0000","scrub_duration":0.0023200949999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"218'9089","reported_seq":9430,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T07:35:24.282597+0000","last_change":"2024-09-11T07:34:55.517249+0000","last_active":"2024-09-11T07:35:24.282597+0000","last_peered":"2024-09-11T07:35:24.282597+0000","last_clean":"2024-09-11T07:35:24.282597+0000","last_became_active":"2024-09-11T07:33:29.154103+0000","last_became_peered":"2024-09-11T07:33:29.154103+0000","last_unstale":"2024-09-11T07:35:24.282597+0000","last_undegraded":"2024-09-11T07:35:24.282597+0000","last_fullsized":"2024-09-11T07:35:24.282597+0000","mapping_epoch":177,"log_start":"198'4000","ondisk_log_start":"198'4000","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"209'7164","last_scrub_stamp":"2024-09-11T07:34:55.517157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:34:55.517157+0000","objects_scrubbed":7074,"log_size":5089,"log_dups_size":0,"ondisk_log_size":5089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:07.513741+0000","scrub_duration":2.936802589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":461832192,"num_objects":7047,"num_object_clones":0,"num_object_copies":21141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9089,"num_write_kb":516352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"83'22","reported_seq":238,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.225904+0000","last_change":"2024-09-11T07:34:04.375654+0000","last_active":"2024-09-11T07:35:20.225904+0000","last_peered":"2024-09-11T07:35:20.225904+0000","last_clean":"2024-09-11T07:35:20.225904+0000","last_became_active":"2024-09-11T07:26:33.549733+0000","last_became_peered":"2024-09-11T07:26:33.549733+0000","last_unstale":"2024-09-11T07:35:20.225904+0000","last_undegraded":"2024-09-11T07:35:20.225904+0000","last_fullsized":"2024-09-11T07:35:20.225904+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'22","last_scrub_stamp":"2024-09-11T07:34:04.375457+0000","last_deep_scrub":"83'22","last_deep_scrub_stamp":"2024-09-11T07:34:04.375457+0000","last_clean_scrub_stamp":"2024-09-11T07:34:04.375457+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:29.572580+0000","scrub_duration":0.0086024310000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":228,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.224514+0000","last_change":"2024-09-11T07:33:59.343890+0000","last_active":"2024-09-11T07:35:20.224514+0000","last_peered":"2024-09-11T07:35:20.224514+0000","last_clean":"2024-09-11T07:35:20.224514+0000","last_became_active":"2024-09-11T07:32:00.513472+0000","last_became_peered":"2024-09-11T07:32:00.513472+0000","last_unstale":"2024-09-11T07:35:20.224514+0000","last_undegraded":"2024-09-11T07:35:20.224514+0000","last_fullsized":"2024-09-11T07:35:20.224514+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:59.343676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:59.343676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0067031579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":322,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.212923+0000","last_change":"2024-09-11T07:34:39.263174+0000","last_active":"2024-09-11T07:35:20.212923+0000","last_peered":"2024-09-11T07:35:20.212923+0000","last_clean":"2024-09-11T07:35:20.212923+0000","last_became_active":"2024-09-11T07:26:35.560406+0000","last_became_peered":"2024-09-11T07:26:35.560406+0000","last_unstale":"2024-09-11T07:35:20.212923+0000","last_undegraded":"2024-09-11T07:35:20.212923+0000","last_fullsized":"2024-09-11T07:35:20.212923+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:34:39.263118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:39.263118+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:08.406869+0000","scrub_duration":0.002665307,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":204,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.221763+0000","last_change":"2024-09-11T07:34:50.112677+0000","last_active":"2024-09-11T07:35:20.221763+0000","last_peered":"2024-09-11T07:35:20.221763+0000","last_clean":"2024-09-11T07:35:20.221763+0000","last_became_active":"2024-09-11T07:26:37.568005+0000","last_became_peered":"2024-09-11T07:26:37.568005+0000","last_unstale":"2024-09-11T07:35:20.221763+0000","last_undegraded":"2024-09-11T07:35:20.221763+0000","last_fullsized":"2024-09-11T07:35:20.221763+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:50.112594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:31.200685+0000","last_clean_scrub_stamp":"2024-09-11T07:34:50.112594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:16.653839+0000","scrub_duration":0.0084012830000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"218'8887","reported_seq":8935,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T07:35:24.032562+0000","last_change":"2024-09-11T07:33:29.154414+0000","last_active":"2024-09-11T07:35:24.032562+0000","last_peered":"2024-09-11T07:35:24.032562+0000","last_clean":"2024-09-11T07:35:24.032562+0000","last_became_active":"2024-09-11T07:33:29.154117+0000","last_became_peered":"2024-09-11T07:33:29.154117+0000","last_unstale":"2024-09-11T07:35:24.032562+0000","last_undegraded":"2024-09-11T07:35:24.032562+0000","last_fullsized":"2024-09-11T07:35:24.032562+0000","mapping_epoch":177,"log_start":"196'3784","ondisk_log_start":"196'3784","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":5103,"log_dups_size":0,"ondisk_log_size":5103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":460914688,"num_objects":7033,"num_object_clones":0,"num_object_copies":21099,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8887,"num_write_kb":509440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"107'16","reported_seq":231,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.212997+0000","last_change":"2024-09-11T07:33:15.278908+0000","last_active":"2024-09-11T07:35:20.212997+0000","last_peered":"2024-09-11T07:35:20.212997+0000","last_clean":"2024-09-11T07:35:20.212997+0000","last_became_active":"2024-09-11T07:26:33.540777+0000","last_became_peered":"2024-09-11T07:26:33.540777+0000","last_unstale":"2024-09-11T07:35:20.212997+0000","last_undegraded":"2024-09-11T07:35:20.212997+0000","last_fullsized":"2024-09-11T07:35:20.212997+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'16","last_scrub_stamp":"2024-09-11T07:33:15.278856+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-09-11T07:29:38.112111+0000","last_clean_scrub_stamp":"2024-09-11T07:33:15.278856+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016764620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":204,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.224054+0000","last_change":"2024-09-11T07:34:51.212752+0000","last_active":"2024-09-11T07:35:20.224054+0000","last_peered":"2024-09-11T07:35:20.224054+0000","last_clean":"2024-09-11T07:35:20.224054+0000","last_became_active":"2024-09-11T07:32:00.518538+0000","last_became_peered":"2024-09-11T07:32:00.518538+0000","last_unstale":"2024-09-11T07:35:20.224054+0000","last_undegraded":"2024-09-11T07:35:20.224054+0000","last_fullsized":"2024-09-11T07:35:20.224054+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:51.212673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:51.212673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:54.661479+0000","scrub_duration":0.002821179,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":206,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.219901+0000","last_change":"2024-09-11T07:33:10.096952+0000","last_active":"2024-09-11T07:35:20.219901+0000","last_peered":"2024-09-11T07:35:20.219901+0000","last_clean":"2024-09-11T07:35:20.219901+0000","last_became_active":"2024-09-11T07:26:35.569081+0000","last_became_peered":"2024-09-11T07:26:35.569081+0000","last_unstale":"2024-09-11T07:35:20.219901+0000","last_undegraded":"2024-09-11T07:35:20.219901+0000","last_fullsized":"2024-09-11T07:35:20.219901+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:10.096886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:10.096886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032884910000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":204,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.211851+0000","last_change":"2024-09-11T07:34:46.853471+0000","last_active":"2024-09-11T07:35:20.211851+0000","last_peered":"2024-09-11T07:35:20.211851+0000","last_clean":"2024-09-11T07:35:20.211851+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:35:20.211851+0000","last_undegraded":"2024-09-11T07:35:20.211851+0000","last_fullsized":"2024-09-11T07:35:20.211851+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:46.853387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:46.853387+0000","last_clean_scrub_stamp":"2024-09-11T07:34:46.853387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:05.043352+0000","scrub_duration":0.0046894370000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"218'8983","reported_seq":9033,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T07:35:24.048947+0000","last_change":"2024-09-11T07:33:29.205385+0000","last_active":"2024-09-11T07:35:24.048947+0000","last_peered":"2024-09-11T07:35:24.048947+0000","last_clean":"2024-09-11T07:35:24.048947+0000","last_became_active":"2024-09-11T07:33:29.205097+0000","last_became_peered":"2024-09-11T07:33:29.205097+0000","last_unstale":"2024-09-11T07:35:24.048947+0000","last_undegraded":"2024-09-11T07:35:24.048947+0000","last_fullsized":"2024-09-11T07:35:24.048947+0000","mapping_epoch":177,"log_start":"197'3830","ondisk_log_start":"197'3830","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":5153,"log_dups_size":0,"ondisk_log_size":5153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":456654872,"num_objects":6969,"num_object_clones":0,"num_object_copies":20907,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6969,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8983,"num_write_kb":510401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"64'14","reported_seq":218,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.210886+0000","last_change":"2024-09-11T07:31:46.078940+0000","last_active":"2024-09-11T07:35:20.210886+0000","last_peered":"2024-09-11T07:35:20.210886+0000","last_clean":"2024-09-11T07:35:20.210886+0000","last_became_active":"2024-09-11T07:26:33.537850+0000","last_became_peered":"2024-09-11T07:26:33.537850+0000","last_unstale":"2024-09-11T07:35:20.210886+0000","last_undegraded":"2024-09-11T07:35:20.210886+0000","last_fullsized":"2024-09-11T07:35:20.210886+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-09-11T07:31:46.078888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:31:46.078888+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017755819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":230,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.218533+0000","last_change":"2024-09-11T07:34:43.819898+0000","last_active":"2024-09-11T07:35:20.218533+0000","last_peered":"2024-09-11T07:35:20.218533+0000","last_clean":"2024-09-11T07:35:20.218533+0000","last_became_active":"2024-09-11T07:28:09.014995+0000","last_became_peered":"2024-09-11T07:28:09.014995+0000","last_unstale":"2024-09-11T07:35:20.218533+0000","last_undegraded":"2024-09-11T07:35:20.218533+0000","last_fullsized":"2024-09-11T07:35:20.218533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:43.819809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:43.819809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:04.954329+0000","scrub_duration":0.0047161240000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"84'9","reported_seq":312,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T07:35:23.239191+0000","last_change":"2024-09-11T07:33:48.510872+0000","last_active":"2024-09-11T07:35:23.239191+0000","last_peered":"2024-09-11T07:35:23.239191+0000","last_clean":"2024-09-11T07:35:23.239191+0000","last_became_active":"2024-09-11T07:26:35.562997+0000","last_became_peered":"2024-09-11T07:26:35.562997+0000","last_unstale":"2024-09-11T07:35:23.239191+0000","last_undegraded":"2024-09-11T07:35:23.239191+0000","last_fullsized":"2024-09-11T07:35:23.239191+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-11T07:33:48.510778+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-11T07:28:05.653237+0000","last_clean_scrub_stamp":"2024-09-11T07:33:48.510778+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038375390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":178,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.225504+0000","last_change":"2024-09-11T07:34:39.169972+0000","last_active":"2024-09-11T07:35:20.225504+0000","last_peered":"2024-09-11T07:35:20.225504+0000","last_clean":"2024-09-11T07:35:20.225504+0000","last_became_active":"2024-09-11T07:32:00.511005+0000","last_became_peered":"2024-09-11T07:32:00.511005+0000","last_unstale":"2024-09-11T07:35:20.225504+0000","last_undegraded":"2024-09-11T07:35:20.225504+0000","last_fullsized":"2024-09-11T07:35:20.225504+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:39.169891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:39.169891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:53.143370+0000","scrub_duration":0.0052103009999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"218'9281","reported_seq":9329,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T07:35:24.708507+0000","last_change":"2024-09-11T07:33:29.205044+0000","last_active":"2024-09-11T07:35:24.708507+0000","last_peered":"2024-09-11T07:35:24.708507+0000","last_clean":"2024-09-11T07:35:24.708507+0000","last_became_active":"2024-09-11T07:33:29.204734+0000","last_became_peered":"2024-09-11T07:33:29.204734+0000","last_unstale":"2024-09-11T07:35:24.708507+0000","last_undegraded":"2024-09-11T07:35:24.708507+0000","last_fullsized":"2024-09-11T07:35:24.708507+0000","mapping_epoch":177,"log_start":"195'3500","ondisk_log_start":"195'3500","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":5781,"log_dups_size":0,"ondisk_log_size":5781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":466157568,"num_objects":7113,"num_object_clones":0,"num_object_copies":21339,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9281,"num_write_kb":524608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"87'26","reported_seq":255,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.215036+0000","last_change":"2024-09-11T07:33:52.609653+0000","last_active":"2024-09-11T07:35:20.215036+0000","last_peered":"2024-09-11T07:35:20.215036+0000","last_clean":"2024-09-11T07:35:20.215036+0000","last_became_active":"2024-09-11T07:32:00.520670+0000","last_became_peered":"2024-09-11T07:32:00.520670+0000","last_unstale":"2024-09-11T07:35:20.215036+0000","last_undegraded":"2024-09-11T07:35:20.215036+0000","last_fullsized":"2024-09-11T07:35:20.215036+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'26","last_scrub_stamp":"2024-09-11T07:33:52.609336+0000","last_deep_scrub":"87'26","last_deep_scrub_stamp":"2024-09-11T07:28:30.788647+0000","last_clean_scrub_stamp":"2024-09-11T07:33:52.609336+0000","objects_scrubbed":8,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010453553000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":208,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.226003+0000","last_change":"2024-09-11T07:33:07.547329+0000","last_active":"2024-09-11T07:35:20.226003+0000","last_peered":"2024-09-11T07:35:20.226003+0000","last_clean":"2024-09-11T07:35:20.226003+0000","last_became_active":"2024-09-11T07:26:31.421500+0000","last_became_peered":"2024-09-11T07:26:31.421500+0000","last_unstale":"2024-09-11T07:35:20.226003+0000","last_undegraded":"2024-09-11T07:35:20.226003+0000","last_fullsized":"2024-09-11T07:35:20.226003+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:07.547259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:07.547259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019369229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":215,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.210744+0000","last_change":"2024-09-11T07:33:53.449765+0000","last_active":"2024-09-11T07:35:20.210744+0000","last_peered":"2024-09-11T07:35:20.210744+0000","last_clean":"2024-09-11T07:35:20.210744+0000","last_became_active":"2024-09-11T07:32:00.499747+0000","last_became_peered":"2024-09-11T07:32:00.499747+0000","last_unstale":"2024-09-11T07:35:20.210744+0000","last_undegraded":"2024-09-11T07:35:20.210744+0000","last_fullsized":"2024-09-11T07:35:20.210744+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:53.449689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:27.968579+0000","last_clean_scrub_stamp":"2024-09-11T07:33:53.449689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033513269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":203,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.212655+0000","last_change":"2024-09-11T07:34:50.305507+0000","last_active":"2024-09-11T07:35:20.212655+0000","last_peered":"2024-09-11T07:35:20.212655+0000","last_clean":"2024-09-11T07:35:20.212655+0000","last_became_active":"2024-09-11T07:26:37.564616+0000","last_became_peered":"2024-09-11T07:26:37.564616+0000","last_unstale":"2024-09-11T07:35:20.212655+0000","last_undegraded":"2024-09-11T07:35:20.212655+0000","last_fullsized":"2024-09-11T07:35:20.212655+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:50.305429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:29.978798+0000","last_clean_scrub_stamp":"2024-09-11T07:34:50.305429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:04.185413+0000","scrub_duration":0.0037429310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"218'9037","reported_seq":9086,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T07:35:24.652199+0000","last_change":"2024-09-11T07:33:29.161660+0000","last_active":"2024-09-11T07:35:24.652199+0000","last_peered":"2024-09-11T07:35:24.652199+0000","last_clean":"2024-09-11T07:35:24.652199+0000","last_became_active":"2024-09-11T07:33:29.161212+0000","last_became_peered":"2024-09-11T07:33:29.161212+0000","last_unstale":"2024-09-11T07:35:24.652199+0000","last_undegraded":"2024-09-11T07:35:24.652199+0000","last_fullsized":"2024-09-11T07:35:24.652199+0000","mapping_epoch":177,"log_start":"198'4100","ondisk_log_start":"198'4100","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4937,"log_dups_size":0,"ondisk_log_size":4937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":460521472,"num_objects":7027,"num_object_clones":0,"num_object_copies":21081,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9037,"num_write_kb":514048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"64'19","reported_seq":260,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.216419+0000","last_change":"2024-09-11T07:34:35.790009+0000","last_active":"2024-09-11T07:35:20.216419+0000","last_peered":"2024-09-11T07:35:20.216419+0000","last_clean":"2024-09-11T07:35:20.216419+0000","last_became_active":"2024-09-11T07:32:00.506270+0000","last_became_peered":"2024-09-11T07:32:00.506270+0000","last_unstale":"2024-09-11T07:35:20.216419+0000","last_undegraded":"2024-09-11T07:35:20.216419+0000","last_fullsized":"2024-09-11T07:35:20.216419+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-11T07:34:35.789921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:34:35.789921+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:58.796684+0000","scrub_duration":0.015782836000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":178,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.226571+0000","last_change":"2024-09-11T07:34:49.144125+0000","last_active":"2024-09-11T07:35:20.226571+0000","last_peered":"2024-09-11T07:35:20.226571+0000","last_clean":"2024-09-11T07:35:20.226571+0000","last_became_active":"2024-09-11T07:32:00.510631+0000","last_became_peered":"2024-09-11T07:32:00.510631+0000","last_unstale":"2024-09-11T07:35:20.226571+0000","last_undegraded":"2024-09-11T07:35:20.226571+0000","last_fullsized":"2024-09-11T07:35:20.226571+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:49.144041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:49.144041+0000","last_clean_scrub_stamp":"2024-09-11T07:34:49.144041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:16.180178+0000","scrub_duration":0.0079798890000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":216,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.217471+0000","last_change":"2024-09-11T07:34:38.765181+0000","last_active":"2024-09-11T07:35:20.217471+0000","last_peered":"2024-09-11T07:35:20.217471+0000","last_clean":"2024-09-11T07:35:20.217471+0000","last_became_active":"2024-09-11T07:26:35.565359+0000","last_became_peered":"2024-09-11T07:26:35.565359+0000","last_unstale":"2024-09-11T07:35:20.217471+0000","last_undegraded":"2024-09-11T07:35:20.217471+0000","last_fullsized":"2024-09-11T07:35:20.217471+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:38.765093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:38.765093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:03.577721+0000","scrub_duration":0.0083567410000000009,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":202,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.224284+0000","last_change":"2024-09-11T07:34:10.294762+0000","last_active":"2024-09-11T07:35:20.224284+0000","last_peered":"2024-09-11T07:35:20.224284+0000","last_clean":"2024-09-11T07:35:20.224284+0000","last_became_active":"2024-09-11T07:26:37.861279+0000","last_became_peered":"2024-09-11T07:26:37.861279+0000","last_unstale":"2024-09-11T07:35:20.224284+0000","last_undegraded":"2024-09-11T07:35:20.224284+0000","last_fullsized":"2024-09-11T07:35:20.224284+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:10.294658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:39.723598+0000","last_clean_scrub_stamp":"2024-09-11T07:34:10.294658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:25.940771+0000","scrub_duration":0.0026992600000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.e","version":"218'9047","reported_seq":9095,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T07:35:24.727297+0000","last_change":"2024-09-11T07:33:29.159841+0000","last_active":"2024-09-11T07:35:24.727297+0000","last_peered":"2024-09-11T07:35:24.727297+0000","last_clean":"2024-09-11T07:35:24.727297+0000","last_became_active":"2024-09-11T07:33:29.159535+0000","last_became_peered":"2024-09-11T07:33:29.159535+0000","last_unstale":"2024-09-11T07:35:24.727297+0000","last_undegraded":"2024-09-11T07:35:24.727297+0000","last_fullsized":"2024-09-11T07:35:24.727297+0000","mapping_epoch":177,"log_start":"194'3200","ondisk_log_start":"194'3200","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":5847,"log_dups_size":0,"ondisk_log_size":5847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458031104,"num_objects":6989,"num_object_clones":0,"num_object_copies":20967,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9047,"num_write_kb":513152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"64'10","reported_seq":224,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.226323+0000","last_change":"2024-09-11T07:33:53.294408+0000","last_active":"2024-09-11T07:35:20.226323+0000","last_peered":"2024-09-11T07:35:20.226323+0000","last_clean":"2024-09-11T07:35:20.226323+0000","last_became_active":"2024-09-11T07:26:33.543063+0000","last_became_peered":"2024-09-11T07:26:33.543063+0000","last_unstale":"2024-09-11T07:35:20.226323+0000","last_undegraded":"2024-09-11T07:35:20.226323+0000","last_fullsized":"2024-09-11T07:35:20.226323+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:33:53.294174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:53.294174+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0074369980000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":235,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.222699+0000","last_change":"2024-09-11T07:33:55.344919+0000","last_active":"2024-09-11T07:35:20.222699+0000","last_peered":"2024-09-11T07:35:20.222699+0000","last_clean":"2024-09-11T07:35:20.222699+0000","last_became_active":"2024-09-11T07:32:00.522456+0000","last_became_peered":"2024-09-11T07:32:00.522456+0000","last_unstale":"2024-09-11T07:35:20.222699+0000","last_undegraded":"2024-09-11T07:35:20.222699+0000","last_fullsized":"2024-09-11T07:35:20.222699+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:33:55.344823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:55.344823+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025904349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":320,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T07:35:23.216480+0000","last_change":"2024-09-11T07:33:50.686006+0000","last_active":"2024-09-11T07:35:23.216480+0000","last_peered":"2024-09-11T07:35:23.216480+0000","last_clean":"2024-09-11T07:35:23.216480+0000","last_became_active":"2024-09-11T07:26:35.560422+0000","last_became_peered":"2024-09-11T07:26:35.560422+0000","last_unstale":"2024-09-11T07:35:23.216480+0000","last_undegraded":"2024-09-11T07:35:23.216480+0000","last_fullsized":"2024-09-11T07:35:23.216480+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:33:50.685923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:50.685923+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029026490000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":213,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.213867+0000","last_change":"2024-09-11T07:34:43.901422+0000","last_active":"2024-09-11T07:35:20.213867+0000","last_peered":"2024-09-11T07:35:20.213867+0000","last_clean":"2024-09-11T07:35:20.213867+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:35:20.213867+0000","last_undegraded":"2024-09-11T07:35:20.213867+0000","last_fullsized":"2024-09-11T07:35:20.213867+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:43.901333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_clean_scrub_stamp":"2024-09-11T07:34:43.901333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:50.099382+0000","scrub_duration":0.0036903819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"218'8843","reported_seq":8892,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T07:35:24.040219+0000","last_change":"2024-09-11T07:33:29.205462+0000","last_active":"2024-09-11T07:35:24.040219+0000","last_peered":"2024-09-11T07:35:24.040219+0000","last_clean":"2024-09-11T07:35:24.040219+0000","last_became_active":"2024-09-11T07:33:29.205148+0000","last_became_peered":"2024-09-11T07:33:29.205148+0000","last_unstale":"2024-09-11T07:35:24.040219+0000","last_undegraded":"2024-09-11T07:35:24.040219+0000","last_fullsized":"2024-09-11T07:35:24.040219+0000","mapping_epoch":177,"log_start":"196'3700","ondisk_log_start":"196'3700","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":5143,"log_dups_size":0,"ondisk_log_size":5143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":451346432,"num_objects":6887,"num_object_clones":0,"num_object_copies":20661,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8843,"num_write_kb":503360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"170'29","reported_seq":118,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.219223+0000","last_change":"2024-09-11T07:34:34.192084+0000","last_active":"2024-09-11T07:35:20.219223+0000","last_peered":"2024-09-11T07:35:20.219223+0000","last_clean":"2024-09-11T07:35:20.219223+0000","last_became_active":"2024-09-11T07:33:10.017961+0000","last_became_peered":"2024-09-11T07:33:10.017961+0000","last_unstale":"2024-09-11T07:35:20.219223+0000","last_undegraded":"2024-09-11T07:35:20.219223+0000","last_fullsized":"2024-09-11T07:35:20.219223+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"170'29","last_scrub_stamp":"2024-09-11T07:34:34.192009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:34:34.192009+0000","objects_scrubbed":10,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:45.027744+0000","scrub_duration":0.0099986009999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":220,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.225542+0000","last_change":"2024-09-11T07:34:48.082772+0000","last_active":"2024-09-11T07:35:20.225542+0000","last_peered":"2024-09-11T07:35:20.225542+0000","last_clean":"2024-09-11T07:35:20.225542+0000","last_became_active":"2024-09-11T07:26:31.424618+0000","last_became_peered":"2024-09-11T07:26:31.424618+0000","last_unstale":"2024-09-11T07:35:20.225542+0000","last_undegraded":"2024-09-11T07:35:20.225542+0000","last_fullsized":"2024-09-11T07:35:20.225542+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:48.082697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:12.775067+0000","last_clean_scrub_stamp":"2024-09-11T07:34:48.082697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:02.915785+0000","scrub_duration":0.017453455,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":205,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.214154+0000","last_change":"2024-09-11T07:33:52.629961+0000","last_active":"2024-09-11T07:35:20.214154+0000","last_peered":"2024-09-11T07:35:20.214154+0000","last_clean":"2024-09-11T07:35:20.214154+0000","last_became_active":"2024-09-11T07:26:35.560363+0000","last_became_peered":"2024-09-11T07:26:35.560363+0000","last_unstale":"2024-09-11T07:35:20.214154+0000","last_undegraded":"2024-09-11T07:35:20.214154+0000","last_fullsized":"2024-09-11T07:35:20.214154+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:52.629872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:52.629872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003860858,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":211,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.212982+0000","last_change":"2024-09-11T07:33:13.566605+0000","last_active":"2024-09-11T07:35:20.212982+0000","last_peered":"2024-09-11T07:35:20.212982+0000","last_clean":"2024-09-11T07:35:20.212982+0000","last_became_active":"2024-09-11T07:32:00.505919+0000","last_became_peered":"2024-09-11T07:32:00.505919+0000","last_unstale":"2024-09-11T07:35:20.212982+0000","last_undegraded":"2024-09-11T07:35:20.212982+0000","last_fullsized":"2024-09-11T07:35:20.212982+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:13.566551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:13.566551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015767380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"218'9187","reported_seq":9236,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T07:35:24.656991+0000","last_change":"2024-09-11T07:33:29.161446+0000","last_active":"2024-09-11T07:35:24.656991+0000","last_peered":"2024-09-11T07:35:24.656991+0000","last_clean":"2024-09-11T07:35:24.656991+0000","last_became_active":"2024-09-11T07:33:29.161141+0000","last_became_peered":"2024-09-11T07:33:29.161141+0000","last_unstale":"2024-09-11T07:35:24.656991+0000","last_undegraded":"2024-09-11T07:35:24.656991+0000","last_fullsized":"2024-09-11T07:35:24.656991+0000","mapping_epoch":177,"log_start":"198'4200","ondisk_log_start":"198'4200","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":4987,"log_dups_size":0,"ondisk_log_size":4987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":467468288,"num_objects":7133,"num_object_clones":0,"num_object_copies":21399,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9187,"num_write_kb":522240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"144'18","reported_seq":238,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.226178+0000","last_change":"2024-09-11T07:34:03.365979+0000","last_active":"2024-09-11T07:35:20.226178+0000","last_peered":"2024-09-11T07:35:20.226178+0000","last_clean":"2024-09-11T07:35:20.226178+0000","last_became_active":"2024-09-11T07:26:33.546126+0000","last_became_peered":"2024-09-11T07:26:33.546126+0000","last_unstale":"2024-09-11T07:35:20.226178+0000","last_undegraded":"2024-09-11T07:35:20.226178+0000","last_fullsized":"2024-09-11T07:35:20.226178+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"144'18","last_scrub_stamp":"2024-09-11T07:34:03.365891+0000","last_deep_scrub":"144'18","last_deep_scrub_stamp":"2024-09-11T07:31:51.603378+0000","last_clean_scrub_stamp":"2024-09-11T07:34:03.365891+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:30.450782+0000","scrub_duration":0.006530999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":751,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T07:35:23.230032+0000","last_change":"2024-09-11T07:34:42.459557+0000","last_active":"2024-09-11T07:35:23.230032+0000","last_peered":"2024-09-11T07:35:23.230032+0000","last_clean":"2024-09-11T07:35:23.230032+0000","last_became_active":"2024-09-11T07:26:31.421276+0000","last_became_peered":"2024-09-11T07:26:31.421276+0000","last_unstale":"2024-09-11T07:35:23.230032+0000","last_undegraded":"2024-09-11T07:35:23.230032+0000","last_fullsized":"2024-09-11T07:35:23.230032+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'5","last_scrub_stamp":"2024-09-11T07:34:42.459494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:42.459494+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:07.298507+0000","scrub_duration":0.0049235640000000001,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":525,"num_read_kb":520,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":220,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.228464+0000","last_change":"2024-09-11T07:34:07.149600+0000","last_active":"2024-09-11T07:35:20.228464+0000","last_peered":"2024-09-11T07:35:20.228464+0000","last_clean":"2024-09-11T07:35:20.228464+0000","last_became_active":"2024-09-11T07:26:31.413354+0000","last_became_peered":"2024-09-11T07:26:31.413354+0000","last_unstale":"2024-09-11T07:35:20.228464+0000","last_undegraded":"2024-09-11T07:35:20.228464+0000","last_fullsized":"2024-09-11T07:35:20.228464+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:07.149367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:07.149367+0000","last_clean_scrub_stamp":"2024-09-11T07:34:07.149367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0082613879999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":223,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.226352+0000","last_change":"2024-09-11T07:34:43.004662+0000","last_active":"2024-09-11T07:35:20.226352+0000","last_peered":"2024-09-11T07:35:20.226352+0000","last_clean":"2024-09-11T07:35:20.226352+0000","last_became_active":"2024-09-11T07:32:00.508398+0000","last_became_peered":"2024-09-11T07:32:00.508398+0000","last_unstale":"2024-09-11T07:35:20.226352+0000","last_undegraded":"2024-09-11T07:35:20.226352+0000","last_fullsized":"2024-09-11T07:35:20.226352+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:43.003030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:43.003030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:03.470264+0000","scrub_duration":0.010745073000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":194,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.226419+0000","last_change":"2024-09-11T07:33:55.344477+0000","last_active":"2024-09-11T07:35:20.226419+0000","last_peered":"2024-09-11T07:35:20.226419+0000","last_clean":"2024-09-11T07:35:20.226419+0000","last_became_active":"2024-09-11T07:32:00.515745+0000","last_became_peered":"2024-09-11T07:32:00.515745+0000","last_unstale":"2024-09-11T07:35:20.226419+0000","last_undegraded":"2024-09-11T07:35:20.226419+0000","last_fullsized":"2024-09-11T07:35:20.226419+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:55.344426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:55.344426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.032747216000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"218'9139","reported_seq":9185,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T07:35:24.286798+0000","last_change":"2024-09-11T07:33:29.156986+0000","last_active":"2024-09-11T07:35:24.286798+0000","last_peered":"2024-09-11T07:35:24.286798+0000","last_clean":"2024-09-11T07:35:24.286798+0000","last_became_active":"2024-09-11T07:33:29.156567+0000","last_became_peered":"2024-09-11T07:33:29.156567+0000","last_unstale":"2024-09-11T07:35:24.286798+0000","last_undegraded":"2024-09-11T07:35:24.286798+0000","last_fullsized":"2024-09-11T07:35:24.286798+0000","mapping_epoch":177,"log_start":"198'4100","ondisk_log_start":"198'4100","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":5039,"log_dups_size":0,"ondisk_log_size":5039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":460652544,"num_objects":7029,"num_object_clones":0,"num_object_copies":21087,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9139,"num_write_kb":517376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"94'18","reported_seq":265,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.216853+0000","last_change":"2024-09-11T07:34:44.944741+0000","last_active":"2024-09-11T07:35:20.216853+0000","last_peered":"2024-09-11T07:35:20.216853+0000","last_clean":"2024-09-11T07:35:20.216853+0000","last_became_active":"2024-09-11T07:28:09.018134+0000","last_became_peered":"2024-09-11T07:28:09.018134+0000","last_unstale":"2024-09-11T07:35:20.216853+0000","last_undegraded":"2024-09-11T07:35:20.216853+0000","last_fullsized":"2024-09-11T07:35:20.216853+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'18","last_scrub_stamp":"2024-09-11T07:34:44.944644+0000","last_deep_scrub":"94'18","last_deep_scrub_stamp":"2024-09-11T07:28:51.437397+0000","last_clean_scrub_stamp":"2024-09-11T07:34:44.944644+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:52.368609+0000","scrub_duration":0.0043360300000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":215,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.214953+0000","last_change":"2024-09-11T07:33:48.749463+0000","last_active":"2024-09-11T07:35:20.214953+0000","last_peered":"2024-09-11T07:35:20.214953+0000","last_clean":"2024-09-11T07:35:20.214953+0000","last_became_active":"2024-09-11T07:26:31.420320+0000","last_became_peered":"2024-09-11T07:26:31.420320+0000","last_unstale":"2024-09-11T07:35:20.214953+0000","last_undegraded":"2024-09-11T07:35:20.214953+0000","last_fullsized":"2024-09-11T07:35:20.214953+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-11T07:33:48.749183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:48.749183+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059599450000000003,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":205,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.226129+0000","last_change":"2024-09-11T07:33:58.311557+0000","last_active":"2024-09-11T07:35:20.226129+0000","last_peered":"2024-09-11T07:35:20.226129+0000","last_clean":"2024-09-11T07:35:20.226129+0000","last_became_active":"2024-09-11T07:32:00.520035+0000","last_became_peered":"2024-09-11T07:32:00.520035+0000","last_unstale":"2024-09-11T07:35:20.226129+0000","last_undegraded":"2024-09-11T07:35:20.226129+0000","last_fullsized":"2024-09-11T07:35:20.226129+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:58.311344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:16.967492+0000","last_clean_scrub_stamp":"2024-09-11T07:33:58.311344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030726579999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":196,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.219324+0000","last_change":"2024-09-11T07:34:12.211194+0000","last_active":"2024-09-11T07:35:20.219324+0000","last_peered":"2024-09-11T07:35:20.219324+0000","last_clean":"2024-09-11T07:35:20.219324+0000","last_became_active":"2024-09-11T07:26:35.564178+0000","last_became_peered":"2024-09-11T07:26:35.564178+0000","last_unstale":"2024-09-11T07:35:20.219324+0000","last_undegraded":"2024-09-11T07:35:20.219324+0000","last_fullsized":"2024-09-11T07:35:20.219324+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:12.211119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:12.211119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0062325000000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":205,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.222494+0000","last_change":"2024-09-11T07:33:58.323420+0000","last_active":"2024-09-11T07:35:20.222494+0000","last_peered":"2024-09-11T07:35:20.222494+0000","last_clean":"2024-09-11T07:35:20.222494+0000","last_became_active":"2024-09-11T07:26:37.561378+0000","last_became_peered":"2024-09-11T07:26:37.561378+0000","last_unstale":"2024-09-11T07:35:20.222494+0000","last_undegraded":"2024-09-11T07:35:20.222494+0000","last_fullsized":"2024-09-11T07:35:20.222494+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:58.323318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:58.323318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0083597830000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"218'8995","reported_seq":9336,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T07:35:24.164934+0000","last_change":"2024-09-11T07:34:56.391596+0000","last_active":"2024-09-11T07:35:24.164934+0000","last_peered":"2024-09-11T07:35:24.164934+0000","last_clean":"2024-09-11T07:35:24.164934+0000","last_became_active":"2024-09-11T07:33:29.154702+0000","last_became_peered":"2024-09-11T07:33:29.154702+0000","last_unstale":"2024-09-11T07:35:24.164934+0000","last_undegraded":"2024-09-11T07:35:24.164934+0000","last_fullsized":"2024-09-11T07:35:24.164934+0000","mapping_epoch":177,"log_start":"192'2800","ondisk_log_start":"192'2800","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"209'7126","last_scrub_stamp":"2024-09-11T07:34:56.391542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:34:56.391542+0000","objects_scrubbed":7063,"log_size":6195,"log_dups_size":0,"ondisk_log_size":6195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:23.443491+0000","scrub_duration":2.590155212,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458555392,"num_objects":6997,"num_object_clones":0,"num_object_copies":20991,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8995,"num_write_kb":511744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":250,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.218374+0000","last_change":"2024-09-11T07:34:07.759083+0000","last_active":"2024-09-11T07:35:20.218374+0000","last_peered":"2024-09-11T07:35:20.218374+0000","last_clean":"2024-09-11T07:35:20.218374+0000","last_became_active":"2024-09-11T07:32:00.517409+0000","last_became_peered":"2024-09-11T07:32:00.517409+0000","last_unstale":"2024-09-11T07:35:20.218374+0000","last_undegraded":"2024-09-11T07:35:20.218374+0000","last_fullsized":"2024-09-11T07:35:20.218374+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-09-11T07:34:07.758985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:34:07.758985+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027645090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'3","reported_seq":218,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.215375+0000","last_change":"2024-09-11T07:34:43.819953+0000","last_active":"2024-09-11T07:35:20.215375+0000","last_peered":"2024-09-11T07:35:20.215375+0000","last_clean":"2024-09-11T07:35:20.215375+0000","last_became_active":"2024-09-11T07:26:31.419501+0000","last_became_peered":"2024-09-11T07:26:31.419501+0000","last_unstale":"2024-09-11T07:35:20.215375+0000","last_undegraded":"2024-09-11T07:35:20.215375+0000","last_fullsized":"2024-09-11T07:35:20.215375+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-09-11T07:34:43.819873+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_clean_scrub_stamp":"2024-09-11T07:34:43.819873+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:07.822773+0000","scrub_duration":0.010008270999999999,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":220,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.219319+0000","last_change":"2024-09-11T07:33:51.108800+0000","last_active":"2024-09-11T07:35:20.219319+0000","last_peered":"2024-09-11T07:35:20.219319+0000","last_clean":"2024-09-11T07:35:20.219319+0000","last_became_active":"2024-09-11T07:32:00.498002+0000","last_became_peered":"2024-09-11T07:32:00.498002+0000","last_unstale":"2024-09-11T07:35:20.219319+0000","last_undegraded":"2024-09-11T07:35:20.219319+0000","last_fullsized":"2024-09-11T07:35:20.219319+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:51.108748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:51.108748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013286089999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"87'81","reported_seq":231,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.226063+0000","last_change":"2024-09-11T07:34:44.231140+0000","last_active":"2024-09-11T07:35:20.226063+0000","last_peered":"2024-09-11T07:35:20.226063+0000","last_clean":"2024-09-11T07:35:20.226063+0000","last_became_active":"2024-09-11T07:32:00.519656+0000","last_became_peered":"2024-09-11T07:32:00.519656+0000","last_unstale":"2024-09-11T07:35:20.226063+0000","last_undegraded":"2024-09-11T07:35:20.226063+0000","last_fullsized":"2024-09-11T07:35:20.226063+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:34:44.230767+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:34:44.230767+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:12.546729+0000","scrub_duration":0.0048225719999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":191,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.226383+0000","last_change":"2024-09-11T07:33:51.272916+0000","last_active":"2024-09-11T07:35:20.226383+0000","last_peered":"2024-09-11T07:35:20.226383+0000","last_clean":"2024-09-11T07:35:20.226383+0000","last_became_active":"2024-09-11T07:32:00.510888+0000","last_became_peered":"2024-09-11T07:32:00.510888+0000","last_unstale":"2024-09-11T07:35:20.226383+0000","last_undegraded":"2024-09-11T07:35:20.226383+0000","last_fullsized":"2024-09-11T07:35:20.226383+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:51.272831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:51.272831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013461180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":204,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.219400+0000","last_change":"2024-09-11T07:34:55.205744+0000","last_active":"2024-09-11T07:35:20.219400+0000","last_peered":"2024-09-11T07:35:20.219400+0000","last_clean":"2024-09-11T07:35:20.219400+0000","last_became_active":"2024-09-11T07:26:37.559920+0000","last_became_peered":"2024-09-11T07:26:37.559920+0000","last_unstale":"2024-09-11T07:35:20.219400+0000","last_undegraded":"2024-09-11T07:35:20.219400+0000","last_fullsized":"2024-09-11T07:35:20.219400+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:55.205661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:55.205661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:18.914870+0000","scrub_duration":0.0089229300000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"218'11309","reported_seq":11487,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T07:35:24.727021+0000","last_change":"2024-09-11T07:35:11.216353+0000","last_active":"2024-09-11T07:35:24.727021+0000","last_peered":"2024-09-11T07:35:24.727021+0000","last_clean":"2024-09-11T07:35:24.727021+0000","last_became_active":"2024-09-11T07:33:29.204611+0000","last_became_peered":"2024-09-11T07:33:29.204611+0000","last_unstale":"2024-09-11T07:35:24.727021+0000","last_undegraded":"2024-09-11T07:35:24.727021+0000","last_fullsized":"2024-09-11T07:35:24.727021+0000","mapping_epoch":177,"log_start":"204'5501","ondisk_log_start":"204'5501","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":3073,"log_size":5808,"log_dups_size":0,"ondisk_log_size":5808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454033408,"num_objects":6928,"num_object_clones":0,"num_object_copies":20784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9068,"num_write_kb":511872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"64'19","reported_seq":249,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.217654+0000","last_change":"2024-09-11T07:33:49.612375+0000","last_active":"2024-09-11T07:35:20.217654+0000","last_peered":"2024-09-11T07:35:20.217654+0000","last_clean":"2024-09-11T07:35:20.217654+0000","last_became_active":"2024-09-11T07:32:00.503799+0000","last_became_peered":"2024-09-11T07:32:00.503799+0000","last_unstale":"2024-09-11T07:35:20.217654+0000","last_undegraded":"2024-09-11T07:35:20.217654+0000","last_fullsized":"2024-09-11T07:35:20.217654+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-11T07:33:49.612326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:49.612326+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003204648,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":210,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.230299+0000","last_change":"2024-09-11T07:34:02.192275+0000","last_active":"2024-09-11T07:35:20.230299+0000","last_peered":"2024-09-11T07:35:20.230299+0000","last_clean":"2024-09-11T07:35:20.230299+0000","last_became_active":"2024-09-11T07:26:31.419560+0000","last_became_peered":"2024-09-11T07:26:31.419560+0000","last_unstale":"2024-09-11T07:35:20.230299+0000","last_undegraded":"2024-09-11T07:35:20.230299+0000","last_fullsized":"2024-09-11T07:35:20.230299+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:02.192193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:02.192193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060418110000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":206,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.224075+0000","last_change":"2024-09-11T07:34:06.134576+0000","last_active":"2024-09-11T07:35:20.224075+0000","last_peered":"2024-09-11T07:35:20.224075+0000","last_clean":"2024-09-11T07:35:20.224075+0000","last_became_active":"2024-09-11T07:26:35.564044+0000","last_became_peered":"2024-09-11T07:26:35.564044+0000","last_unstale":"2024-09-11T07:35:20.224075+0000","last_undegraded":"2024-09-11T07:35:20.224075+0000","last_fullsized":"2024-09-11T07:35:20.224075+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:06.134464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:06.134464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0062969130000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":199,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.226077+0000","last_change":"2024-09-11T07:34:38.219945+0000","last_active":"2024-09-11T07:35:20.226077+0000","last_peered":"2024-09-11T07:35:20.226077+0000","last_clean":"2024-09-11T07:35:20.226077+0000","last_became_active":"2024-09-11T07:32:00.516916+0000","last_became_peered":"2024-09-11T07:32:00.516916+0000","last_unstale":"2024-09-11T07:35:20.226077+0000","last_undegraded":"2024-09-11T07:35:20.226077+0000","last_fullsized":"2024-09-11T07:35:20.226077+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:38.219876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:38.219876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:49.345830+0000","scrub_duration":0.020549118000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.7","version":"218'8964","reported_seq":9012,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T07:35:24.262205+0000","last_change":"2024-09-11T07:33:29.160658+0000","last_active":"2024-09-11T07:35:24.262205+0000","last_peered":"2024-09-11T07:35:24.262205+0000","last_clean":"2024-09-11T07:35:24.262205+0000","last_became_active":"2024-09-11T07:33:29.160376+0000","last_became_peered":"2024-09-11T07:33:29.160376+0000","last_unstale":"2024-09-11T07:35:24.262205+0000","last_undegraded":"2024-09-11T07:35:24.262205+0000","last_fullsized":"2024-09-11T07:35:24.262205+0000","mapping_epoch":177,"log_start":"197'3900","ondisk_log_start":"197'3900","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":5064,"log_dups_size":0,"ondisk_log_size":5064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":457048064,"num_objects":6974,"num_object_clones":0,"num_object_copies":20922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8964,"num_write_kb":510016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'9","reported_seq":223,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.216100+0000","last_change":"2024-09-11T07:34:47.984112+0000","last_active":"2024-09-11T07:35:20.216100+0000","last_peered":"2024-09-11T07:35:20.216100+0000","last_clean":"2024-09-11T07:35:20.216100+0000","last_became_active":"2024-09-11T07:26:33.543148+0000","last_became_peered":"2024-09-11T07:26:33.543148+0000","last_unstale":"2024-09-11T07:35:20.216100+0000","last_undegraded":"2024-09-11T07:35:20.216100+0000","last_fullsized":"2024-09-11T07:35:20.216100+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:34:47.984050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:34:47.984050+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:57.229110+0000","scrub_duration":0.0070263900000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":218,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.225269+0000","last_change":"2024-09-11T07:34:43.479834+0000","last_active":"2024-09-11T07:35:20.225269+0000","last_peered":"2024-09-11T07:35:20.225269+0000","last_clean":"2024-09-11T07:35:20.225269+0000","last_became_active":"2024-09-11T07:26:31.423056+0000","last_became_peered":"2024-09-11T07:26:31.423056+0000","last_unstale":"2024-09-11T07:35:20.225269+0000","last_undegraded":"2024-09-11T07:35:20.225269+0000","last_fullsized":"2024-09-11T07:35:20.225269+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:43.479741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:43.479741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:10.044799+0000","scrub_duration":0.0073695460000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":216,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.225716+0000","last_change":"2024-09-11T07:34:35.918872+0000","last_active":"2024-09-11T07:35:20.225716+0000","last_peered":"2024-09-11T07:35:20.225716+0000","last_clean":"2024-09-11T07:35:20.225716+0000","last_became_active":"2024-09-11T07:26:35.563182+0000","last_became_peered":"2024-09-11T07:26:35.563182+0000","last_unstale":"2024-09-11T07:35:20.225716+0000","last_undegraded":"2024-09-11T07:35:20.225716+0000","last_fullsized":"2024-09-11T07:35:20.225716+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:35.918794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:35.918794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:00.878344+0000","scrub_duration":0.0031958289999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":220,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.214730+0000","last_change":"2024-09-11T07:34:46.944862+0000","last_active":"2024-09-11T07:35:20.214730+0000","last_peered":"2024-09-11T07:35:20.214730+0000","last_clean":"2024-09-11T07:35:20.214730+0000","last_became_active":"2024-09-11T07:32:00.505185+0000","last_became_peered":"2024-09-11T07:32:00.505185+0000","last_unstale":"2024-09-11T07:35:20.214730+0000","last_undegraded":"2024-09-11T07:35:20.214730+0000","last_fullsized":"2024-09-11T07:35:20.214730+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:46.944763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:46.944763+0000","last_clean_scrub_stamp":"2024-09-11T07:34:46.944763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:48.104421+0000","scrub_duration":0.0075379239999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"218'8966","reported_seq":9015,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T07:35:24.014500+0000","last_change":"2024-09-11T07:33:29.153952+0000","last_active":"2024-09-11T07:35:24.014500+0000","last_peered":"2024-09-11T07:35:24.014500+0000","last_clean":"2024-09-11T07:35:24.014500+0000","last_became_active":"2024-09-11T07:33:29.153604+0000","last_became_peered":"2024-09-11T07:33:29.153604+0000","last_unstale":"2024-09-11T07:35:24.014500+0000","last_undegraded":"2024-09-11T07:35:24.014500+0000","last_fullsized":"2024-09-11T07:35:24.014500+0000","mapping_epoch":177,"log_start":"197'3872","ondisk_log_start":"197'3872","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":0,"log_size":5094,"log_dups_size":0,"ondisk_log_size":5094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":453246976,"num_objects":6916,"num_object_clones":0,"num_object_copies":20748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8966,"num_write_kb":508224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"64'15","reported_seq":243,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.220210+0000","last_change":"2024-09-11T07:33:56.187920+0000","last_active":"2024-09-11T07:35:20.220210+0000","last_peered":"2024-09-11T07:35:20.220210+0000","last_clean":"2024-09-11T07:35:20.220210+0000","last_became_active":"2024-09-11T07:32:00.521464+0000","last_became_peered":"2024-09-11T07:32:00.521464+0000","last_unstale":"2024-09-11T07:35:20.220210+0000","last_undegraded":"2024-09-11T07:35:20.220210+0000","last_fullsized":"2024-09-11T07:35:20.220210+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-09-11T07:33:56.187871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:56.187871+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:23.701092+0000","scrub_duration":0.001643175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":237,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.226975+0000","last_change":"2024-09-11T07:34:36.912017+0000","last_active":"2024-09-11T07:35:20.226975+0000","last_peered":"2024-09-11T07:35:20.226975+0000","last_clean":"2024-09-11T07:35:20.226975+0000","last_became_active":"2024-09-11T07:32:00.513949+0000","last_became_peered":"2024-09-11T07:32:00.513949+0000","last_unstale":"2024-09-11T07:35:20.226975+0000","last_undegraded":"2024-09-11T07:35:20.226975+0000","last_fullsized":"2024-09-11T07:35:20.226975+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:36.911933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_clean_scrub_stamp":"2024-09-11T07:34:36.911933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:57.170517+0000","scrub_duration":0.0036783879999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"151'10","reported_seq":271,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.225449+0000","last_change":"2024-09-11T07:33:48.261946+0000","last_active":"2024-09-11T07:35:20.225449+0000","last_peered":"2024-09-11T07:35:20.225449+0000","last_clean":"2024-09-11T07:35:20.225449+0000","last_became_active":"2024-09-11T07:32:00.506806+0000","last_became_peered":"2024-09-11T07:32:00.506806+0000","last_unstale":"2024-09-11T07:35:20.225449+0000","last_undegraded":"2024-09-11T07:35:20.225449+0000","last_fullsized":"2024-09-11T07:35:20.225449+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:33:48.261857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:48.261857+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001795774,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":204,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.219572+0000","last_change":"2024-09-11T07:34:35.190891+0000","last_active":"2024-09-11T07:35:20.219572+0000","last_peered":"2024-09-11T07:35:20.219572+0000","last_clean":"2024-09-11T07:35:20.219572+0000","last_became_active":"2024-09-11T07:26:37.861134+0000","last_became_peered":"2024-09-11T07:26:37.861134+0000","last_unstale":"2024-09-11T07:35:20.219572+0000","last_undegraded":"2024-09-11T07:35:20.219572+0000","last_fullsized":"2024-09-11T07:35:20.219572+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:35.190837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:35.190837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:48.885204+0000","scrub_duration":0.003575679,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.5","version":"218'8951","reported_seq":9030,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T07:35:24.139947+0000","last_change":"2024-09-11T07:34:57.053543+0000","last_active":"2024-09-11T07:35:24.139947+0000","last_peered":"2024-09-11T07:35:24.139947+0000","last_clean":"2024-09-11T07:35:24.139947+0000","last_became_active":"2024-09-11T07:33:29.205189+0000","last_became_peered":"2024-09-11T07:33:29.205189+0000","last_unstale":"2024-09-11T07:35:24.139947+0000","last_undegraded":"2024-09-11T07:35:24.139947+0000","last_fullsized":"2024-09-11T07:35:24.139947+0000","mapping_epoch":177,"log_start":"192'2800","ondisk_log_start":"192'2800","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:33:28.139035+0000","last_clean_scrub_stamp":"2024-09-11T07:33:28.139035+0000","objects_scrubbed":599,"log_size":6151,"log_dups_size":0,"ondisk_log_size":6151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":462094336,"num_objects":7051,"num_object_clones":0,"num_object_copies":21153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8951,"num_write_kb":512064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"83'17","reported_seq":235,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.210809+0000","last_change":"2024-09-11T07:34:47.310667+0000","last_active":"2024-09-11T07:35:20.210809+0000","last_peered":"2024-09-11T07:35:20.210809+0000","last_clean":"2024-09-11T07:35:20.210809+0000","last_became_active":"2024-09-11T07:26:33.540363+0000","last_became_peered":"2024-09-11T07:26:33.540363+0000","last_unstale":"2024-09-11T07:35:20.210809+0000","last_undegraded":"2024-09-11T07:35:20.210809+0000","last_fullsized":"2024-09-11T07:35:20.210809+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'17","last_scrub_stamp":"2024-09-11T07:34:47.310608+0000","last_deep_scrub":"83'17","last_deep_scrub_stamp":"2024-09-11T07:29:50.970355+0000","last_clean_scrub_stamp":"2024-09-11T07:34:47.310608+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:15.970815+0000","scrub_duration":0.0054521589999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":236,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.212441+0000","last_change":"2024-09-11T07:33:48.436215+0000","last_active":"2024-09-11T07:35:20.212441+0000","last_peered":"2024-09-11T07:35:20.212441+0000","last_clean":"2024-09-11T07:35:20.212441+0000","last_became_active":"2024-09-11T07:32:00.504344+0000","last_became_peered":"2024-09-11T07:32:00.504344+0000","last_unstale":"2024-09-11T07:35:20.212441+0000","last_undegraded":"2024-09-11T07:35:20.212441+0000","last_fullsized":"2024-09-11T07:35:20.212441+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:33:48.436127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:48.436127+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002390933,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":206,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.223793+0000","last_change":"2024-09-11T07:33:51.206706+0000","last_active":"2024-09-11T07:35:20.223793+0000","last_peered":"2024-09-11T07:35:20.223793+0000","last_clean":"2024-09-11T07:35:20.223793+0000","last_became_active":"2024-09-11T07:26:35.561967+0000","last_became_peered":"2024-09-11T07:26:35.561967+0000","last_unstale":"2024-09-11T07:35:20.223793+0000","last_undegraded":"2024-09-11T07:35:20.223793+0000","last_fullsized":"2024-09-11T07:35:20.223793+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:51.206632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:44.820354+0000","last_clean_scrub_stamp":"2024-09-11T07:33:51.206632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0065238259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":221,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.212126+0000","last_change":"2024-09-11T07:34:54.748541+0000","last_active":"2024-09-11T07:35:20.212126+0000","last_peered":"2024-09-11T07:35:20.212126+0000","last_clean":"2024-09-11T07:35:20.212126+0000","last_became_active":"2024-09-11T07:32:00.502019+0000","last_became_peered":"2024-09-11T07:32:00.502019+0000","last_unstale":"2024-09-11T07:35:20.212126+0000","last_undegraded":"2024-09-11T07:35:20.212126+0000","last_fullsized":"2024-09-11T07:35:20.212126+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:54.748316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:54.748316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:19.772161+0000","scrub_duration":0.0033172589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"64'9","reported_seq":230,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.216814+0000","last_change":"2024-09-11T07:33:05.880679+0000","last_active":"2024-09-11T07:35:20.216814+0000","last_peered":"2024-09-11T07:35:20.216814+0000","last_clean":"2024-09-11T07:35:20.216814+0000","last_became_active":"2024-09-11T07:32:00.509412+0000","last_became_peered":"2024-09-11T07:32:00.509412+0000","last_unstale":"2024-09-11T07:35:20.216814+0000","last_undegraded":"2024-09-11T07:35:20.216814+0000","last_fullsized":"2024-09-11T07:35:20.216814+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:33:05.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:05.880612+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021137109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":210,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.223932+0000","last_change":"2024-09-11T07:34:03.167417+0000","last_active":"2024-09-11T07:35:20.223932+0000","last_peered":"2024-09-11T07:35:20.223932+0000","last_clean":"2024-09-11T07:35:20.223932+0000","last_became_active":"2024-09-11T07:26:31.415701+0000","last_became_peered":"2024-09-11T07:26:31.415701+0000","last_unstale":"2024-09-11T07:35:20.223932+0000","last_undegraded":"2024-09-11T07:35:20.223932+0000","last_fullsized":"2024-09-11T07:35:20.223932+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:03.167327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.925200+0000","last_clean_scrub_stamp":"2024-09-11T07:34:03.167327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:28.877249+0000","scrub_duration":0.0047032489999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":213,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.214837+0000","last_change":"2024-09-11T07:33:54.643176+0000","last_active":"2024-09-11T07:35:20.214837+0000","last_peered":"2024-09-11T07:35:20.214837+0000","last_clean":"2024-09-11T07:35:20.214837+0000","last_became_active":"2024-09-11T07:32:00.499598+0000","last_became_peered":"2024-09-11T07:32:00.499598+0000","last_unstale":"2024-09-11T07:35:20.214837+0000","last_undegraded":"2024-09-11T07:35:20.214837+0000","last_fullsized":"2024-09-11T07:35:20.214837+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:54.642486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:35.392360+0000","last_clean_scrub_stamp":"2024-09-11T07:33:54.642486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:23.835294+0000","scrub_duration":0.010529307999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":222,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.227168+0000","last_change":"2024-09-11T07:34:40.972629+0000","last_active":"2024-09-11T07:35:20.227168+0000","last_peered":"2024-09-11T07:35:20.227168+0000","last_clean":"2024-09-11T07:35:20.227168+0000","last_became_active":"2024-09-11T07:32:00.506697+0000","last_became_peered":"2024-09-11T07:32:00.506697+0000","last_unstale":"2024-09-11T07:35:20.227168+0000","last_undegraded":"2024-09-11T07:35:20.227168+0000","last_fullsized":"2024-09-11T07:35:20.227168+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:34:40.971610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:40.971610+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:41.756990+0000","scrub_duration":0.0083037709999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":220,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.217527+0000","last_change":"2024-09-11T07:34:44.772163+0000","last_active":"2024-09-11T07:35:20.217527+0000","last_peered":"2024-09-11T07:35:20.217527+0000","last_clean":"2024-09-11T07:35:20.217527+0000","last_became_active":"2024-09-11T07:26:31.423229+0000","last_became_peered":"2024-09-11T07:26:31.423229+0000","last_unstale":"2024-09-11T07:35:20.217527+0000","last_undegraded":"2024-09-11T07:35:20.217527+0000","last_fullsized":"2024-09-11T07:35:20.217527+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:44.772088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:44.772088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:56.364459+0000","scrub_duration":0.0054747839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"82'9","reported_seq":227,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.215031+0000","last_change":"2024-09-11T07:33:11.576715+0000","last_active":"2024-09-11T07:35:20.215031+0000","last_peered":"2024-09-11T07:35:20.215031+0000","last_clean":"2024-09-11T07:35:20.215031+0000","last_became_active":"2024-09-11T07:32:00.501365+0000","last_became_peered":"2024-09-11T07:32:00.501365+0000","last_unstale":"2024-09-11T07:35:20.215031+0000","last_undegraded":"2024-09-11T07:35:20.215031+0000","last_fullsized":"2024-09-11T07:35:20.215031+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'9","last_scrub_stamp":"2024-09-11T07:33:11.576656+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:33:11.576656+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043542040000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":216,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.219718+0000","last_change":"2024-09-11T07:34:56.186434+0000","last_active":"2024-09-11T07:35:20.219718+0000","last_peered":"2024-09-11T07:35:20.219718+0000","last_clean":"2024-09-11T07:35:20.219718+0000","last_became_active":"2024-09-11T07:26:35.566894+0000","last_became_peered":"2024-09-11T07:26:35.566894+0000","last_unstale":"2024-09-11T07:35:20.219718+0000","last_undegraded":"2024-09-11T07:35:20.219718+0000","last_fullsized":"2024-09-11T07:35:20.219718+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:56.186343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:34:56.186343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:06.950239+0000","scrub_duration":0.0077966119999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":192,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.226816+0000","last_change":"2024-09-11T07:33:54.283942+0000","last_active":"2024-09-11T07:35:20.226816+0000","last_peered":"2024-09-11T07:35:20.226816+0000","last_clean":"2024-09-11T07:35:20.226816+0000","last_became_active":"2024-09-11T07:32:00.514100+0000","last_became_peered":"2024-09-11T07:32:00.514100+0000","last_unstale":"2024-09-11T07:35:20.226816+0000","last_undegraded":"2024-09-11T07:35:20.226816+0000","last_fullsized":"2024-09-11T07:35:20.226816+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:54.283860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:54.283860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032251950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"86'15","reported_seq":257,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.212279+0000","last_change":"2024-09-11T07:34:50.757345+0000","last_active":"2024-09-11T07:35:20.212279+0000","last_peered":"2024-09-11T07:35:20.212279+0000","last_clean":"2024-09-11T07:35:20.212279+0000","last_became_active":"2024-09-11T07:32:00.504463+0000","last_became_peered":"2024-09-11T07:32:00.504463+0000","last_unstale":"2024-09-11T07:35:20.212279+0000","last_undegraded":"2024-09-11T07:35:20.212279+0000","last_fullsized":"2024-09-11T07:35:20.212279+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'15","last_scrub_stamp":"2024-09-11T07:34:50.757258+0000","last_deep_scrub":"86'15","last_deep_scrub_stamp":"2024-09-11T07:28:22.858137+0000","last_clean_scrub_stamp":"2024-09-11T07:34:50.757258+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:19.272535+0000","scrub_duration":0.0058766319999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":220,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.225481+0000","last_change":"2024-09-11T07:34:02.231632+0000","last_active":"2024-09-11T07:35:20.225481+0000","last_peered":"2024-09-11T07:35:20.225481+0000","last_clean":"2024-09-11T07:35:20.225481+0000","last_became_active":"2024-09-11T07:32:00.498473+0000","last_became_peered":"2024-09-11T07:32:00.498473+0000","last_unstale":"2024-09-11T07:35:20.225481+0000","last_undegraded":"2024-09-11T07:35:20.225481+0000","last_fullsized":"2024-09-11T07:35:20.225481+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:02.231583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:02.231583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:24.160910+0000","scrub_duration":0.0013039099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":320,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.221900+0000","last_change":"2024-09-11T07:34:45.055042+0000","last_active":"2024-09-11T07:35:20.221900+0000","last_peered":"2024-09-11T07:35:20.221900+0000","last_clean":"2024-09-11T07:35:20.221900+0000","last_became_active":"2024-09-11T07:26:35.562245+0000","last_became_peered":"2024-09-11T07:26:35.562245+0000","last_unstale":"2024-09-11T07:35:20.221900+0000","last_undegraded":"2024-09-11T07:35:20.221900+0000","last_fullsized":"2024-09-11T07:35:20.221900+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:34:45.054966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:45.054966+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:06.712119+0000","scrub_duration":0.0055567680000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":220,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.216459+0000","last_change":"2024-09-11T07:34:12.556603+0000","last_active":"2024-09-11T07:35:20.216459+0000","last_peered":"2024-09-11T07:35:20.216459+0000","last_clean":"2024-09-11T07:35:20.216459+0000","last_became_active":"2024-09-11T07:32:00.509295+0000","last_became_peered":"2024-09-11T07:32:00.509295+0000","last_unstale":"2024-09-11T07:35:20.216459+0000","last_undegraded":"2024-09-11T07:35:20.216459+0000","last_fullsized":"2024-09-11T07:35:20.216459+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:12.556370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:12.556370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041791759999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"83'13","reported_seq":238,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.221970+0000","last_change":"2024-09-11T07:33:52.110471+0000","last_active":"2024-09-11T07:35:20.221970+0000","last_peered":"2024-09-11T07:35:20.221970+0000","last_clean":"2024-09-11T07:35:20.221970+0000","last_became_active":"2024-09-11T07:32:00.505134+0000","last_became_peered":"2024-09-11T07:32:00.505134+0000","last_unstale":"2024-09-11T07:35:20.221970+0000","last_undegraded":"2024-09-11T07:35:20.221970+0000","last_fullsized":"2024-09-11T07:35:20.221970+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-11T07:33:52.110361+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-11T07:28:00.959583+0000","last_clean_scrub_stamp":"2024-09-11T07:33:52.110361+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035066310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":209,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.216573+0000","last_change":"2024-09-11T07:33:51.733858+0000","last_active":"2024-09-11T07:35:20.216573+0000","last_peered":"2024-09-11T07:35:20.216573+0000","last_clean":"2024-09-11T07:35:20.216573+0000","last_became_active":"2024-09-11T07:26:31.419409+0000","last_became_peered":"2024-09-11T07:26:31.419409+0000","last_unstale":"2024-09-11T07:35:20.216573+0000","last_undegraded":"2024-09-11T07:35:20.216573+0000","last_fullsized":"2024-09-11T07:35:20.216573+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:51.733780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:51.733780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.079912094000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"151'10","reported_seq":323,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-11T07:35:23.238094+0000","last_change":"2024-09-11T07:33:47.546713+0000","last_active":"2024-09-11T07:35:23.238094+0000","last_peered":"2024-09-11T07:35:23.238094+0000","last_clean":"2024-09-11T07:35:23.238094+0000","last_became_active":"2024-09-11T07:32:00.504062+0000","last_became_peered":"2024-09-11T07:32:00.504062+0000","last_unstale":"2024-09-11T07:35:23.238094+0000","last_undegraded":"2024-09-11T07:35:23.238094+0000","last_fullsized":"2024-09-11T07:35:23.238094+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:33:47.546622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:33:47.546622+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0064760360000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":203,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.212109+0000","last_change":"2024-09-11T07:34:42.324717+0000","last_active":"2024-09-11T07:35:20.212109+0000","last_peered":"2024-09-11T07:35:20.212109+0000","last_clean":"2024-09-11T07:35:20.212109+0000","last_became_active":"2024-09-11T07:26:37.565723+0000","last_became_peered":"2024-09-11T07:26:37.565723+0000","last_unstale":"2024-09-11T07:35:20.212109+0000","last_undegraded":"2024-09-11T07:35:20.212109+0000","last_fullsized":"2024-09-11T07:35:20.212109+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:42.324639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:42.324639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:59.418722+0000","scrub_duration":0.011135859,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":232,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.226242+0000","last_change":"2024-09-11T07:34:12.364224+0000","last_active":"2024-09-11T07:35:20.226242+0000","last_peered":"2024-09-11T07:35:20.226242+0000","last_clean":"2024-09-11T07:35:20.226242+0000","last_became_active":"2024-09-11T07:26:33.540688+0000","last_became_peered":"2024-09-11T07:26:33.540688+0000","last_unstale":"2024-09-11T07:35:20.226242+0000","last_undegraded":"2024-09-11T07:35:20.226242+0000","last_fullsized":"2024-09-11T07:35:20.226242+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-11T07:34:12.364126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:34:12.364126+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011537936,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":213,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.217668+0000","last_change":"2024-09-11T07:34:09.772110+0000","last_active":"2024-09-11T07:35:20.217668+0000","last_peered":"2024-09-11T07:35:20.217668+0000","last_clean":"2024-09-11T07:35:20.217668+0000","last_became_active":"2024-09-11T07:26:31.425587+0000","last_became_peered":"2024-09-11T07:26:31.425587+0000","last_unstale":"2024-09-11T07:35:20.217668+0000","last_undegraded":"2024-09-11T07:35:20.217668+0000","last_fullsized":"2024-09-11T07:35:20.217668+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:09.772000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:09.772000+0000","last_clean_scrub_stamp":"2024-09-11T07:34:09.772000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0087789909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":196,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.212195+0000","last_change":"2024-09-11T07:34:04.660377+0000","last_active":"2024-09-11T07:35:20.212195+0000","last_peered":"2024-09-11T07:35:20.212195+0000","last_clean":"2024-09-11T07:35:20.212195+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:35:20.212195+0000","last_undegraded":"2024-09-11T07:35:20.212195+0000","last_fullsized":"2024-09-11T07:35:20.212195+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:04.660121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:04.660121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047831310000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":203,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.213787+0000","last_change":"2024-09-11T07:33:53.667783+0000","last_active":"2024-09-11T07:35:20.213787+0000","last_peered":"2024-09-11T07:35:20.213787+0000","last_clean":"2024-09-11T07:35:20.213787+0000","last_became_active":"2024-09-11T07:26:37.568886+0000","last_became_peered":"2024-09-11T07:26:37.568886+0000","last_unstale":"2024-09-11T07:35:20.213787+0000","last_undegraded":"2024-09-11T07:35:20.213787+0000","last_fullsized":"2024-09-11T07:35:20.213787+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:53.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:53.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039882809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"136'16","reported_seq":231,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.212067+0000","last_change":"2024-09-11T07:34:00.417426+0000","last_active":"2024-09-11T07:35:20.212067+0000","last_peered":"2024-09-11T07:35:20.212067+0000","last_clean":"2024-09-11T07:35:20.212067+0000","last_became_active":"2024-09-11T07:26:33.540208+0000","last_became_peered":"2024-09-11T07:26:33.540208+0000","last_unstale":"2024-09-11T07:35:20.212067+0000","last_undegraded":"2024-09-11T07:35:20.212067+0000","last_fullsized":"2024-09-11T07:35:20.212067+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'16","last_scrub_stamp":"2024-09-11T07:34:00.417343+0000","last_deep_scrub":"136'16","last_deep_scrub_stamp":"2024-09-11T07:31:06.039023+0000","last_clean_scrub_stamp":"2024-09-11T07:34:00.417343+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026159130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":211,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.225201+0000","last_change":"2024-09-11T07:33:57.333274+0000","last_active":"2024-09-11T07:35:20.225201+0000","last_peered":"2024-09-11T07:35:20.225201+0000","last_clean":"2024-09-11T07:35:20.225201+0000","last_became_active":"2024-09-11T07:26:31.422671+0000","last_became_peered":"2024-09-11T07:26:31.422671+0000","last_unstale":"2024-09-11T07:35:20.225201+0000","last_undegraded":"2024-09-11T07:35:20.225201+0000","last_fullsized":"2024-09-11T07:35:20.225201+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:57.333166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:33:57.333166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010637648,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":197,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.226107+0000","last_change":"2024-09-11T07:33:51.251429+0000","last_active":"2024-09-11T07:35:20.226107+0000","last_peered":"2024-09-11T07:35:20.226107+0000","last_clean":"2024-09-11T07:35:20.226107+0000","last_became_active":"2024-09-11T07:26:35.566931+0000","last_became_peered":"2024-09-11T07:26:35.566931+0000","last_unstale":"2024-09-11T07:35:20.226107+0000","last_undegraded":"2024-09-11T07:35:20.226107+0000","last_fullsized":"2024-09-11T07:35:20.226107+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:51.251225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.847474+0000","last_clean_scrub_stamp":"2024-09-11T07:33:51.251225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050645059999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":194,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.211682+0000","last_change":"2024-09-11T07:34:33.688599+0000","last_active":"2024-09-11T07:35:20.211682+0000","last_peered":"2024-09-11T07:35:20.211682+0000","last_clean":"2024-09-11T07:35:20.211682+0000","last_became_active":"2024-09-11T07:26:37.561573+0000","last_became_peered":"2024-09-11T07:26:37.561573+0000","last_unstale":"2024-09-11T07:35:20.211682+0000","last_undegraded":"2024-09-11T07:35:20.211682+0000","last_fullsized":"2024-09-11T07:35:20.211682+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:33.688496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:33.688496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:52.913118+0000","scrub_duration":0.0094425380000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"82'6","reported_seq":216,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.214953+0000","last_change":"2024-09-11T07:34:40.773406+0000","last_active":"2024-09-11T07:35:20.214953+0000","last_peered":"2024-09-11T07:35:20.214953+0000","last_clean":"2024-09-11T07:35:20.214953+0000","last_became_active":"2024-09-11T07:26:33.545830+0000","last_became_peered":"2024-09-11T07:26:33.545830+0000","last_unstale":"2024-09-11T07:35:20.214953+0000","last_undegraded":"2024-09-11T07:35:20.214953+0000","last_fullsized":"2024-09-11T07:35:20.214953+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'6","last_scrub_stamp":"2024-09-11T07:34:40.772239+0000","last_deep_scrub":"82'6","last_deep_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_clean_scrub_stamp":"2024-09-11T07:34:40.772239+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:00.092871+0000","scrub_duration":0.005304563,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":210,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.219870+0000","last_change":"2024-09-11T07:34:42.048593+0000","last_active":"2024-09-11T07:35:20.219870+0000","last_peered":"2024-09-11T07:35:20.219870+0000","last_clean":"2024-09-11T07:35:20.219870+0000","last_became_active":"2024-09-11T07:26:31.420518+0000","last_became_peered":"2024-09-11T07:26:31.420518+0000","last_unstale":"2024-09-11T07:35:20.219870+0000","last_undegraded":"2024-09-11T07:35:20.219870+0000","last_fullsized":"2024-09-11T07:35:20.219870+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:42.048544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_clean_scrub_stamp":"2024-09-11T07:34:42.048544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:53.207069+0000","scrub_duration":0.0012192539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":226,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.217240+0000","last_change":"2024-09-11T07:34:35.809784+0000","last_active":"2024-09-11T07:35:20.217240+0000","last_peered":"2024-09-11T07:35:20.217240+0000","last_clean":"2024-09-11T07:35:20.217240+0000","last_became_active":"2024-09-11T07:32:00.501969+0000","last_became_peered":"2024-09-11T07:32:00.501969+0000","last_unstale":"2024-09-11T07:35:20.217240+0000","last_undegraded":"2024-09-11T07:35:20.217240+0000","last_fullsized":"2024-09-11T07:35:20.217240+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:35.809630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:35.809630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:55.273618+0000","scrub_duration":0.0051714539999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":194,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.225246+0000","last_change":"2024-09-11T07:33:53.290535+0000","last_active":"2024-09-11T07:35:20.225246+0000","last_peered":"2024-09-11T07:35:20.225246+0000","last_clean":"2024-09-11T07:35:20.225246+0000","last_became_active":"2024-09-11T07:32:00.511407+0000","last_became_peered":"2024-09-11T07:32:00.511407+0000","last_unstale":"2024-09-11T07:35:20.225246+0000","last_undegraded":"2024-09-11T07:35:20.225246+0000","last_fullsized":"2024-09-11T07:35:20.225246+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:53.290379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:53.290379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002605291,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"64'11","reported_seq":224,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.211883+0000","last_change":"2024-09-11T07:33:03.304492+0000","last_active":"2024-09-11T07:35:20.211883+0000","last_peered":"2024-09-11T07:35:20.211883+0000","last_clean":"2024-09-11T07:35:20.211883+0000","last_became_active":"2024-09-11T07:32:00.502081+0000","last_became_peered":"2024-09-11T07:32:00.502081+0000","last_unstale":"2024-09-11T07:35:20.211883+0000","last_undegraded":"2024-09-11T07:35:20.211883+0000","last_fullsized":"2024-09-11T07:35:20.211883+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:33:03.304404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:03.304404+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023973169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":220,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.218477+0000","last_change":"2024-09-11T07:34:45.805554+0000","last_active":"2024-09-11T07:35:20.218477+0000","last_peered":"2024-09-11T07:35:20.218477+0000","last_clean":"2024-09-11T07:35:20.218477+0000","last_became_active":"2024-09-11T07:26:31.425876+0000","last_became_peered":"2024-09-11T07:26:31.425876+0000","last_unstale":"2024-09-11T07:35:20.218477+0000","last_undegraded":"2024-09-11T07:35:20.218477+0000","last_fullsized":"2024-09-11T07:35:20.218477+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:45.805472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:45.805472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:48.261758+0000","scrub_duration":0.0063397319999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":177,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.224121+0000","last_change":"2024-09-11T07:34:43.214735+0000","last_active":"2024-09-11T07:35:20.224121+0000","last_peered":"2024-09-11T07:35:20.224121+0000","last_clean":"2024-09-11T07:35:20.224121+0000","last_became_active":"2024-09-11T07:32:00.520534+0000","last_became_peered":"2024-09-11T07:32:00.520534+0000","last_unstale":"2024-09-11T07:35:20.224121+0000","last_undegraded":"2024-09-11T07:35:20.224121+0000","last_fullsized":"2024-09-11T07:35:20.224121+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:43.214661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:39.021876+0000","last_clean_scrub_stamp":"2024-09-11T07:34:43.214661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:54.001779+0000","scrub_duration":0.0017089449999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":194,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.212057+0000","last_change":"2024-09-11T07:34:34.678649+0000","last_active":"2024-09-11T07:35:20.212057+0000","last_peered":"2024-09-11T07:35:20.212057+0000","last_clean":"2024-09-11T07:35:20.212057+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:35:20.212057+0000","last_undegraded":"2024-09-11T07:35:20.212057+0000","last_fullsized":"2024-09-11T07:35:20.212057+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:34.678233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:34.678233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:00.120684+0000","scrub_duration":0.0075729409999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":181,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.225419+0000","last_change":"2024-09-11T07:34:41.207758+0000","last_active":"2024-09-11T07:35:20.225419+0000","last_peered":"2024-09-11T07:35:20.225419+0000","last_clean":"2024-09-11T07:35:20.225419+0000","last_became_active":"2024-09-11T07:32:00.511258+0000","last_became_peered":"2024-09-11T07:32:00.511258+0000","last_unstale":"2024-09-11T07:35:20.225419+0000","last_undegraded":"2024-09-11T07:35:20.225419+0000","last_fullsized":"2024-09-11T07:35:20.225419+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:34:41.207591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:41.207591+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:09.601160+0000","scrub_duration":0.0027997759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":229,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.220311+0000","last_change":"2024-09-11T07:33:53.155118+0000","last_active":"2024-09-11T07:35:20.220311+0000","last_peered":"2024-09-11T07:35:20.220311+0000","last_clean":"2024-09-11T07:35:20.220311+0000","last_became_active":"2024-09-11T07:32:00.518307+0000","last_became_peered":"2024-09-11T07:32:00.518307+0000","last_unstale":"2024-09-11T07:35:20.220311+0000","last_undegraded":"2024-09-11T07:35:20.220311+0000","last_fullsized":"2024-09-11T07:35:20.220311+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:33:53.155024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:53.155024+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038397209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"64'10","reported_seq":32,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.212565+0000","last_change":"2024-09-11T07:35:00.589816+0000","last_active":"2024-09-11T07:35:20.212565+0000","last_peered":"2024-09-11T07:35:20.212565+0000","last_clean":"2024-09-11T07:35:20.212565+0000","last_became_active":"2024-09-11T07:35:00.561615+0000","last_became_peered":"2024-09-11T07:35:00.561615+0000","last_unstale":"2024-09-11T07:35:20.212565+0000","last_undegraded":"2024-09-11T07:35:20.212565+0000","last_fullsized":"2024-09-11T07:35:20.212565+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:34:53.937466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:34:53.937466+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:36:05.896110+0000","scrub_duration":0.010397707000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1f","version":"151'10","reported_seq":335,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.218342+0000","last_change":"2024-09-11T07:34:02.797981+0000","last_active":"2024-09-11T07:35:20.218342+0000","last_peered":"2024-09-11T07:35:20.218342+0000","last_clean":"2024-09-11T07:35:20.218342+0000","last_became_active":"2024-09-11T07:32:00.502049+0000","last_became_peered":"2024-09-11T07:32:00.502049+0000","last_unstale":"2024-09-11T07:35:20.218342+0000","last_undegraded":"2024-09-11T07:35:20.218342+0000","last_fullsized":"2024-09-11T07:35:20.218342+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:34:02.797889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:02.797889+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001718904,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":202,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.224861+0000","last_change":"2024-09-11T07:34:06.381128+0000","last_active":"2024-09-11T07:35:20.224861+0000","last_peered":"2024-09-11T07:35:20.224861+0000","last_clean":"2024-09-11T07:35:20.224861+0000","last_became_active":"2024-09-11T07:26:37.562501+0000","last_became_peered":"2024-09-11T07:26:37.562501+0000","last_unstale":"2024-09-11T07:35:20.224861+0000","last_undegraded":"2024-09-11T07:35:20.224861+0000","last_fullsized":"2024-09-11T07:35:20.224861+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:06.381043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:34:06.381043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046834980000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":210,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.212199+0000","last_change":"2024-09-11T07:34:47.832839+0000","last_active":"2024-09-11T07:35:20.212199+0000","last_peered":"2024-09-11T07:35:20.212199+0000","last_clean":"2024-09-11T07:35:20.212199+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:35:20.212199+0000","last_undegraded":"2024-09-11T07:35:20.212199+0000","last_fullsized":"2024-09-11T07:35:20.212199+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:47.832712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:47.832712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:52.133010+0000","scrub_duration":0.013167129,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'11","reported_seq":281,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.218417+0000","last_change":"2024-09-11T07:34:11.760411+0000","last_active":"2024-09-11T07:35:20.218417+0000","last_peered":"2024-09-11T07:35:20.218417+0000","last_clean":"2024-09-11T07:35:20.218417+0000","last_became_active":"2024-09-11T07:30:28.599840+0000","last_became_peered":"2024-09-11T07:30:28.599840+0000","last_unstale":"2024-09-11T07:35:20.218417+0000","last_undegraded":"2024-09-11T07:35:20.218417+0000","last_fullsized":"2024-09-11T07:35:20.218417+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:34:11.760315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:34:11.760315+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:31.329350+0000","scrub_duration":0.007126177,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":212,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.216714+0000","last_change":"2024-09-11T07:33:13.776627+0000","last_active":"2024-09-11T07:35:20.216714+0000","last_peered":"2024-09-11T07:35:20.216714+0000","last_clean":"2024-09-11T07:35:20.216714+0000","last_became_active":"2024-09-11T07:32:00.507050+0000","last_became_peered":"2024-09-11T07:32:00.507050+0000","last_unstale":"2024-09-11T07:35:20.216714+0000","last_undegraded":"2024-09-11T07:35:20.216714+0000","last_fullsized":"2024-09-11T07:35:20.216714+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:13.776566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:24.575210+0000","last_clean_scrub_stamp":"2024-09-11T07:33:13.776566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001406963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":203,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.212798+0000","last_change":"2024-09-11T07:33:54.459273+0000","last_active":"2024-09-11T07:35:20.212798+0000","last_peered":"2024-09-11T07:35:20.212798+0000","last_clean":"2024-09-11T07:35:20.212798+0000","last_became_active":"2024-09-11T07:26:37.862373+0000","last_became_peered":"2024-09-11T07:26:37.862373+0000","last_unstale":"2024-09-11T07:35:20.212798+0000","last_undegraded":"2024-09-11T07:35:20.212798+0000","last_fullsized":"2024-09-11T07:35:20.212798+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:54.459223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:54.459223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001318018,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":230,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.217746+0000","last_change":"2024-09-11T07:34:06.757693+0000","last_active":"2024-09-11T07:35:20.217746+0000","last_peered":"2024-09-11T07:35:20.217746+0000","last_clean":"2024-09-11T07:35:20.217746+0000","last_became_active":"2024-09-11T07:32:00.503749+0000","last_became_peered":"2024-09-11T07:32:00.503749+0000","last_unstale":"2024-09-11T07:35:20.217746+0000","last_undegraded":"2024-09-11T07:35:20.217746+0000","last_fullsized":"2024-09-11T07:35:20.217746+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:06.757621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:34:06.757621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015661609999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":233,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.230100+0000","last_change":"2024-09-11T07:33:55.212599+0000","last_active":"2024-09-11T07:35:20.230100+0000","last_peered":"2024-09-11T07:35:20.230100+0000","last_clean":"2024-09-11T07:35:20.230100+0000","last_became_active":"2024-09-11T07:26:33.542544+0000","last_became_peered":"2024-09-11T07:26:33.542544+0000","last_unstale":"2024-09-11T07:35:20.230100+0000","last_undegraded":"2024-09-11T07:35:20.230100+0000","last_fullsized":"2024-09-11T07:35:20.230100+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-09-11T07:33:55.212499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:33:55.212499+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:35:23.471681+0000","scrub_duration":0.004594473,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":204,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.225080+0000","last_change":"2024-09-11T07:34:01.366655+0000","last_active":"2024-09-11T07:35:20.225080+0000","last_peered":"2024-09-11T07:35:20.225080+0000","last_clean":"2024-09-11T07:35:20.225080+0000","last_became_active":"2024-09-11T07:26:35.555862+0000","last_became_peered":"2024-09-11T07:26:35.555862+0000","last_unstale":"2024-09-11T07:35:20.225080+0000","last_undegraded":"2024-09-11T07:35:20.225080+0000","last_fullsized":"2024-09-11T07:35:20.225080+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:01.366571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:01.366571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059672780000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":194,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.214766+0000","last_change":"2024-09-11T07:33:17.489228+0000","last_active":"2024-09-11T07:35:20.214766+0000","last_peered":"2024-09-11T07:35:20.214766+0000","last_clean":"2024-09-11T07:35:20.214766+0000","last_became_active":"2024-09-11T07:26:37.861647+0000","last_became_peered":"2024-09-11T07:26:37.861647+0000","last_unstale":"2024-09-11T07:35:20.214766+0000","last_undegraded":"2024-09-11T07:35:20.214766+0000","last_fullsized":"2024-09-11T07:35:20.214766+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:17.489174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:33:17.489174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029380980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":217,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.212471+0000","last_change":"2024-09-11T07:33:09.499435+0000","last_active":"2024-09-11T07:35:20.212471+0000","last_peered":"2024-09-11T07:35:20.212471+0000","last_clean":"2024-09-11T07:35:20.212471+0000","last_became_active":"2024-09-11T07:32:00.502947+0000","last_became_peered":"2024-09-11T07:32:00.502947+0000","last_unstale":"2024-09-11T07:35:20.212471+0000","last_undegraded":"2024-09-11T07:35:20.212471+0000","last_fullsized":"2024-09-11T07:35:20.212471+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:33:09.499379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:33:09.499379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001485933,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"127'17","reported_seq":226,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.212688+0000","last_change":"2024-09-11T07:33:50.604574+0000","last_active":"2024-09-11T07:35:20.212688+0000","last_peered":"2024-09-11T07:35:20.212688+0000","last_clean":"2024-09-11T07:35:20.212688+0000","last_became_active":"2024-09-11T07:26:33.542711+0000","last_became_peered":"2024-09-11T07:26:33.542711+0000","last_unstale":"2024-09-11T07:35:20.212688+0000","last_undegraded":"2024-09-11T07:35:20.212688+0000","last_fullsized":"2024-09-11T07:35:20.212688+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'17","last_scrub_stamp":"2024-09-11T07:33:50.604493+0000","last_deep_scrub":"127'17","last_deep_scrub_stamp":"2024-09-11T07:30:30.260552+0000","last_clean_scrub_stamp":"2024-09-11T07:33:50.604493+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045862999999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":205,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-11T07:35:20.212375+0000","last_change":"2024-09-11T07:34:07.375537+0000","last_active":"2024-09-11T07:35:20.212375+0000","last_peered":"2024-09-11T07:35:20.212375+0000","last_clean":"2024-09-11T07:35:20.212375+0000","last_became_active":"2024-09-11T07:26:35.562037+0000","last_became_peered":"2024-09-11T07:26:35.562037+0000","last_unstale":"2024-09-11T07:35:20.212375+0000","last_undegraded":"2024-09-11T07:35:20.212375+0000","last_fullsized":"2024-09-11T07:35:20.212375+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:07.375490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:07.375490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042989400000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":7360217112,"num_objects":112309,"num_object_clones":0,"num_object_copies":336927,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112309,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":144749,"num_write_kb":8225793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":22095933440,"data_stored":22095921224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85403,"ondisk_log_size":85403,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":798,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":474,"ondisk_log_size":474,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":525,"num_read_kb":520,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":147,"seq":631360192554,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2593836,"kb_used_data":2264992,"kb_used_omap":4,"kb_used_meta":328635,"kb_avail":91151316,"statfs":{"total":95995035648,"available":93338947584,"internally_reserved":0,"allocated":2319351808,"data_stored":2318974706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":336522758},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":2.6179999999999999,"15min":3.4159999999999999},"min":{"1min":0.23999999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":3.4140000000000001,"5min":49.573,"15min":49.573},"last":0.19500000000000001},{"interface":"front","average":{"1min":1.75,"5min":2.5600000000000001,"15min":3.1720000000000002},"min":{"1min":0.223,"5min":0.22,"15min":0.22},"max":{"1min":5.0069999999999997,"5min":49.881,"15min":49.881},"last":0.66300000000000003}]},{"osd":1,"last update":"Wed Sep 11 07:35:04 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.73799999999999999,"15min":0.64400000000000002},"min":{"1min":0.316,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":2.367,"5min":3.677,"15min":3.677},"last":0.40200000000000002},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.82699999999999996,"15min":0.81100000000000005},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":3.0670000000000002,"5min":4.6740000000000004,"15min":4.6740000000000004},"last":0.27200000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.92900000000000005,"15min":0.96799999999999997},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":3.5,"5min":3.6579999999999999,"15min":3.6579999999999999},"last":0.432},{"interface":"front","average":{"1min":1.1359999999999999,"5min":0.94699999999999995,"15min":0.92700000000000005},"min":{"1min":0.31,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.9929999999999999,"5min":5.0149999999999997,"15min":5.0149999999999997},"last":0.308}]},{"osd":3,"last update":"Wed Sep 11 07:35:04 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":2.6360000000000001,"15min":3.278},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":4.6589999999999998,"5min":49.798999999999999,"15min":49.798999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":1.4299999999999999,"5min":2.6549999999999998,"15min":3.194},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":4.3280000000000003,"5min":49.524000000000001,"15min":49.524000000000001},"last":0.75900000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.47999999999999998,"15min":0.437},"min":{"1min":0.29299999999999998,"5min":0.253,"15min":0.253},"max":{"1min":3.5409999999999999,"5min":3.5409999999999999,"15min":3.5409999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.624,"15min":0.60899999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.3849999999999998,"5min":3.9340000000000002,"15min":3.9340000000000002},"last":1.238}]},{"osd":5,"last update":"Wed Sep 11 07:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.54900000000000004,"15min":0.47599999999999998},"min":{"1min":0.35099999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.6280000000000001,"5min":3.6280000000000001,"15min":3.6280000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.70699999999999996,"15min":0.67400000000000004},"min":{"1min":0.33500000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.79,"5min":4.79,"15min":4.79},"last":0.36599999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54800000000000004,"15min":0.52300000000000002},"min":{"1min":0.29499999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.58499999999999996,"15min":0.55300000000000005},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":3.5800000000000001,"5min":3.5800000000000001,"15min":3.5800000000000001},"last":0.73699999999999999}]}]},{"osd":6,"up_from":44,"seq":188978561139,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2559760,"kb_used_data":2249796,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":91185392,"statfs":{"total":95995035648,"available":93373841408,"internally_reserved":0,"allocated":2303791104,"data_stored":2303441128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":3.1840000000000002,"15min":1.5980000000000001},"min":{"1min":0.20699999999999999,"5min":0.192,"15min":0.192},"max":{"1min":3.2469999999999999,"5min":142.29599999999999,"15min":142.29599999999999},"last":0.751},{"interface":"front","average":{"1min":1.708,"5min":3.395,"15min":1.7509999999999999},"min":{"1min":0.25900000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":10.404,"5min":142.68799999999999,"15min":142.68799999999999},"last":0.36899999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:35:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":2.0470000000000002,"15min":1.097},"min":{"1min":0.249,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":2.4500000000000002,"5min":127.92100000000001,"15min":127.92100000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":2.218,"15min":1.087},"min":{"1min":0.245,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.831,"5min":127.746,"15min":127.746},"last":0.82099999999999995}]},{"osd":2,"last update":"Wed Sep 11 07:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":1.123,"15min":0.79700000000000004},"min":{"1min":0.26900000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":3.1030000000000002,"5min":26.491,"15min":26.491},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.502,"15min":0.91400000000000003},"min":{"1min":0.36799999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":3.7639999999999998,"5min":26.561,"15min":26.561},"last":0.96699999999999997}]},{"osd":3,"last update":"Wed Sep 11 07:35:09 2024","interfaces":[{"interface":"back","average":{"1min":1.7949999999999999,"5min":1.4350000000000001,"15min":0.89600000000000002},"min":{"1min":0.33500000000000002,"5min":0.254,"15min":0.254},"max":{"1min":6.7519999999999998,"5min":31.268999999999998,"15min":31.268999999999998},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.538,"15min":1.0189999999999999},"min":{"1min":0.317,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":2.8100000000000001,"5min":31.872,"15min":31.872},"last":1.1180000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.48499999999999999,"15min":0.54200000000000004},"min":{"1min":0.25900000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":2.1629999999999998,"5min":2.1629999999999998,"15min":4.4720000000000004},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.45600000000000002,"15min":0.435},"min":{"1min":0.27500000000000002,"5min":0.26600000000000001,"15min":0.25},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.52900000000000003}]},{"osd":5,"last update":"Wed Sep 11 07:35:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":0.67200000000000004,"15min":0.64800000000000002},"min":{"1min":0.317,"5min":0.255,"15min":0.255},"max":{"1min":8.891,"5min":8.891,"15min":8.891},"last":0.39700000000000002},{"interface":"front","average":{"1min":1.4750000000000001,"5min":0.71999999999999997,"15min":0.59199999999999997},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":14.597,"5min":14.597,"15min":14.597},"last":0.70999999999999996}]},{"osd":7,"last update":"Wed Sep 11 07:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.194,"5min":2.194,"15min":2.194},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.67700000000000005,"15min":0.70799999999999996},"min":{"1min":0.34000000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.3069999999999999,"5min":3.6030000000000002,"15min":3.6030000000000002},"last":0.41899999999999998}]}]},{"osd":5,"up_from":37,"seq":158913790073,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3909668,"kb_used_data":3612312,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":89835484,"statfs":{"total":95995035648,"available":91991535616,"internally_reserved":0,"allocated":3699007488,"data_stored":3698637680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:34:41 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":4.4749999999999996,"15min":1.837},"min":{"1min":0.215,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":10.342000000000001,"5min":185.68199999999999,"15min":185.68199999999999},"last":0.23799999999999999},{"interface":"front","average":{"1min":1.8140000000000001,"5min":4.7720000000000002,"15min":1.9390000000000001},"min":{"1min":0.254,"5min":0.218,"15min":0.218},"max":{"1min":10.355,"5min":186.12200000000001,"15min":186.12200000000001},"last":2.0739999999999998}]},{"osd":1,"last update":"Wed Sep 11 07:34:41 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.0449999999999999,"15min":0.70299999999999996},"min":{"1min":0.23799999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":5.8550000000000004,"5min":9.0969999999999995,"15min":9.0969999999999995},"last":3.1299999999999999},{"interface":"front","average":{"1min":1.458,"5min":1.2190000000000001,"15min":0.749},"min":{"1min":0.28899999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":5.8689999999999998,"5min":19.119,"15min":19.119},"last":3.0369999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:34:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":3.3109999999999999,"15min":1.4650000000000001},"min":{"1min":0.312,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":3.8050000000000002,"5min":187.99199999999999,"15min":187.99199999999999},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.90700000000000003,"5min":3.2210000000000001,"15min":1.4419999999999999},"min":{"1min":0.27600000000000002,"5min":0.254,"15min":0.254},"max":{"1min":2.778,"5min":186.375,"15min":186.375},"last":0.379}]},{"osd":3,"last update":"Wed Sep 11 07:34:41 2024","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":4.6150000000000002,"15min":1.9430000000000001},"min":{"1min":0.26400000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":4.4790000000000001,"5min":188.11500000000001,"15min":188.11500000000001},"last":3.427},{"interface":"front","average":{"1min":1.236,"5min":4.6020000000000003,"15min":1.9199999999999999},"min":{"1min":0.39400000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":5.3949999999999996,"5min":186.44200000000001,"15min":186.44200000000001},"last":3.339}]},{"osd":4,"last update":"Wed Sep 11 07:34:41 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.53500000000000003,"15min":0.42799999999999999},"min":{"1min":0.253,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":4.7060000000000004,"5min":4.7060000000000004,"15min":4.7060000000000004},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.50900000000000001,"15min":0.44900000000000001},"min":{"1min":0.29099999999999998,"5min":0.253,"15min":0.23200000000000001},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":0.39000000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.55500000000000005,"15min":0.46700000000000003},"min":{"1min":0.376,"5min":0.27200000000000002,"15min":0.26000000000000001},"max":{"1min":1.9830000000000001,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.318},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.57099999999999995,"15min":0.47699999999999998},"min":{"1min":0.28599999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.9570000000000001,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":0.34999999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:35:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.67300000000000004,"15min":0.61699999999999999},"min":{"1min":0.30599999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.7919999999999998,"5min":4.7919999999999998,"15min":4.7919999999999998},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.65200000000000002,"15min":0.61899999999999999},"min":{"1min":0.35699999999999998,"5min":0.316,"15min":0.316},"max":{"1min":2.02,"5min":2.226,"15min":2.226},"last":0.41899999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986301,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2574112,"kb_used_data":2268440,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":91171040,"statfs":{"total":95995035648,"available":93359144960,"internally_reserved":0,"allocated":2322882560,"data_stored":2322512571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:34:32 2024","interfaces":[{"interface":"back","average":{"1min":2.391,"5min":1.968,"15min":1.0069999999999999},"min":{"1min":0.27700000000000002,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":32.892000000000003,"5min":35.308,"15min":35.308},"last":0.53100000000000003},{"interface":"front","average":{"1min":2.2749999999999999,"5min":1.782,"15min":0.92800000000000005},"min":{"1min":0.30599999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":32.860999999999997,"5min":35.597999999999999,"15min":35.597999999999999},"last":1.4399999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:34:32 2024","interfaces":[{"interface":"back","average":{"1min":3.0739999999999998,"5min":1.617,"15min":0.875},"min":{"1min":0.23599999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":24.581,"5min":24.581,"15min":24.581},"last":3.9430000000000001},{"interface":"front","average":{"1min":2.1749999999999998,"5min":1.032,"15min":0.80700000000000005},"min":{"1min":0.26700000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":24.652000000000001,"5min":24.652000000000001,"15min":24.652000000000001},"last":0.26300000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":1.369,"15min":0.81000000000000005},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.22600000000000001},"max":{"1min":2.9380000000000002,"5min":19.927,"15min":19.927},"last":2.9630000000000001},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.286,"15min":0.77700000000000002},"min":{"1min":0.27200000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":4.4080000000000004,"5min":20.02,"15min":20.02},"last":0.39100000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:34:32 2024","interfaces":[{"interface":"back","average":{"1min":2.3690000000000002,"5min":1.774,"15min":0.96599999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":24.029,"5min":24.029,"15min":24.029},"last":0.41899999999999998},{"interface":"front","average":{"1min":1.97,"5min":1.5660000000000001,"15min":0.86099999999999999},"min":{"1min":0.32100000000000001,"5min":0.309,"15min":0.28699999999999998},"max":{"1min":24.265000000000001,"5min":24.265000000000001,"15min":24.265000000000001},"last":0.45100000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.48299999999999998,"15min":0.39900000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":1.7709999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.50900000000000001,"15min":0.42799999999999999},"min":{"1min":0.25,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":5.7320000000000002,"5min":5.7320000000000002,"15min":5.7320000000000002},"last":0.32600000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.621,"15min":0.49099999999999999},"min":{"1min":0.33500000000000002,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.3440000000000001,"5min":7.8650000000000002,"15min":7.8650000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60499999999999998,"15min":0.498},"min":{"1min":0.27100000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":2.5670000000000002,"5min":9.3770000000000007,"15min":9.3770000000000007},"last":0.35299999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:35:06 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.57799999999999996,"15min":0.55600000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.5920000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.504},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.59899999999999998,"15min":0.56499999999999995},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.4740000000000002,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":0.48199999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182530,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3018476,"kb_used_data":2716896,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":90726676,"statfs":{"total":95995035648,"available":92904116224,"internally_reserved":0,"allocated":2782101504,"data_stored":2781722787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:35:13 2024","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":2.1819999999999999,"15min":1.056},"min":{"1min":0.20899999999999999,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":3.5990000000000002,"5min":38.432000000000002,"15min":38.432000000000002},"last":0.40400000000000003},{"interface":"front","average":{"1min":1.8180000000000001,"5min":3.056,"15min":1.391},"min":{"1min":0.22900000000000001,"5min":0.20300000000000001,"15min":0.19500000000000001},"max":{"1min":4.7119999999999997,"5min":91.876999999999995,"15min":91.876999999999995},"last":0.51500000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:35:13 2024","interfaces":[{"interface":"back","average":{"1min":1.885,"5min":1.929,"15min":0.99299999999999999},"min":{"1min":0.31,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":4.734,"5min":10.475,"15min":10.475},"last":0.621},{"interface":"front","average":{"1min":1.6739999999999999,"5min":1.5269999999999999,"15min":0.85099999999999998},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":5.6680000000000001,"5min":8.1899999999999995,"15min":8.1899999999999995},"last":2.6509999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:35:13 2024","interfaces":[{"interface":"back","average":{"1min":1.6080000000000001,"5min":1.7769999999999999,"15min":0.99199999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.23200000000000001},"max":{"1min":5.5789999999999997,"5min":8.282,"15min":8.282},"last":2.677},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.6819999999999999,"15min":0.91900000000000004},"min":{"1min":0.26800000000000002,"5min":0.21199999999999999,"15min":0.20999999999999999},"max":{"1min":5.008,"5min":8.7110000000000003,"15min":8.7110000000000003},"last":0.371}]},{"osd":4,"last update":"Wed Sep 11 07:34:27 2024","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.371,"15min":0.89300000000000002},"min":{"1min":0.29599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":3.2389999999999999,"5min":5.7670000000000003,"15min":21.856000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":1.018,"5min":1.468,"15min":0.99399999999999999},"min":{"1min":0.36599999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":2.8479999999999999,"5min":8.0709999999999997,"15min":19.146000000000001},"last":0.54700000000000004}]},{"osd":5,"last update":"Wed Sep 11 07:34:43 2024","interfaces":[{"interface":"back","average":{"1min":1.6259999999999999,"5min":1.7,"15min":0.95799999999999996},"min":{"1min":0.377,"5min":0.30599999999999999,"15min":0.29599999999999999},"max":{"1min":5.0209999999999999,"5min":32.325000000000003,"15min":32.325000000000003},"last":0.48399999999999999},{"interface":"front","average":{"1min":1.841,"5min":1.617,"15min":1.006},"min":{"1min":0.54600000000000004,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":4.8490000000000002,"5min":7.9870000000000001,"15min":18.416},"last":0.57199999999999995}]},{"osd":6,"last update":"Wed Sep 11 07:35:13 2024","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.5900000000000001,"15min":1.407},"min":{"1min":0.315,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":5.4539999999999997,"5min":32.350999999999999,"15min":32.350999999999999},"last":2.5840000000000001},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.611,"15min":1.3879999999999999},"min":{"1min":0.38500000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.331,"5min":32.402999999999999,"15min":32.402999999999999},"last":0.433}]},{"osd":7,"last update":"Wed Sep 11 07:35:07 2024","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.288,"15min":1.3560000000000001},"min":{"1min":0.53500000000000003,"5min":0.308,"15min":0.308},"max":{"1min":4.6859999999999999,"5min":8.0429999999999993,"15min":8.0429999999999993},"last":0.59599999999999997},{"interface":"front","average":{"1min":1.653,"5min":1.9950000000000001,"15min":2.3359999999999999},"min":{"1min":0.38100000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":5.5309999999999997,"5min":32.289999999999999,"15min":32.289999999999999},"last":2.6240000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411463,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3015244,"kb_used_data":2705284,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":90729908,"statfs":{"total":95995035648,"available":92907425792,"internally_reserved":0,"allocated":2770210816,"data_stored":2769860447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:34:41 2024","interfaces":[{"interface":"back","average":{"1min":3.5289999999999999,"5min":2.8010000000000002,"15min":1.23},"min":{"1min":0.23000000000000001,"5min":0.183,"15min":0.17000000000000001},"max":{"1min":43.151000000000003,"5min":56.704000000000001,"15min":56.704000000000001},"last":0.997},{"interface":"front","average":{"1min":3.5939999999999999,"5min":2.8690000000000002,"15min":1.248},"min":{"1min":0.32600000000000001,"5min":0.216,"15min":0.216},"max":{"1min":43.213999999999999,"5min":56.816000000000003,"15min":56.816000000000003},"last":0.76800000000000002}]},{"osd":1,"last update":"Wed Sep 11 07:34:41 2024","interfaces":[{"interface":"back","average":{"1min":4.468,"5min":2.3679999999999999,"15min":1.0660000000000001},"min":{"1min":0.374,"5min":0.24299999999999999,"15min":0.23200000000000001},"max":{"1min":65.153999999999996,"5min":65.153999999999996,"15min":65.153999999999996},"last":0.42899999999999999},{"interface":"front","average":{"1min":4.96,"5min":2.5489999999999999,"15min":1.143},"min":{"1min":0.35299999999999998,"5min":0.216,"15min":0.20699999999999999},"max":{"1min":63.311999999999998,"5min":63.311999999999998,"15min":63.311999999999998},"last":0.88700000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:35:12 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":2.4089999999999998,"15min":1.171},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.26900000000000002},"max":{"1min":4.6180000000000003,"5min":44.405999999999999,"15min":44.405999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":1.2629999999999999,"5min":2.5379999999999998,"15min":1.238},"min":{"1min":0.311,"5min":0.253,"15min":0.253},"max":{"1min":4.8029999999999999,"5min":43.295999999999999,"15min":43.295999999999999},"last":0.93100000000000005}]},{"osd":4,"last update":"Wed Sep 11 07:34:27 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":1.8009999999999999,"15min":0.97099999999999997},"min":{"1min":0.32300000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":2.71,"5min":30.946999999999999,"15min":30.946999999999999},"last":1.024},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.488,"15min":0.88},"min":{"1min":0.31900000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.5339999999999998,"5min":22.434000000000001,"15min":22.434000000000001},"last":1.153}]},{"osd":5,"last update":"Wed Sep 11 07:34:46 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":1.7430000000000001,"15min":0.97699999999999998},"min":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.31900000000000001},"max":{"1min":2.4249999999999998,"5min":31.789000000000001,"15min":31.789000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":1.254,"5min":1.7929999999999999,"15min":1.0589999999999999},"min":{"1min":0.34100000000000003,"5min":0.31,"15min":0.31},"max":{"1min":2.7679999999999998,"5min":30.898,"15min":30.898},"last":1.0489999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:35:12 2024","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.5369999999999999,"15min":1},"min":{"1min":0.42799999999999999,"5min":0.376,"15min":0.36099999999999999},"max":{"1min":2.4700000000000002,"5min":22.399000000000001,"15min":22.399000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.4730000000000001,"15min":0.98999999999999999},"min":{"1min":0.39000000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.7669999999999999,"5min":25.739000000000001,"15min":25.739000000000001},"last":0.97199999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:35:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.7529999999999999,"15min":2.1389999999999998},"min":{"1min":0.45200000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.3820000000000001,"5min":22.484000000000002,"15min":22.484000000000002},"last":0.45800000000000002},{"interface":"front","average":{"1min":1.3520000000000001,"5min":2.0569999999999999,"15min":2.5009999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":2.7919999999999998,"5min":25.670000000000002,"15min":25.670000000000002},"last":1.0760000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574986,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3492592,"kb_used_data":3191076,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":90252560,"statfs":{"total":95995035648,"available":92418621440,"internally_reserved":0,"allocated":3267661824,"data_stored":3267279825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:35:17 2024","interfaces":[{"interface":"back","average":{"1min":1.6970000000000001,"5min":3.4020000000000001,"15min":1.47},"min":{"1min":0.20599999999999999,"5min":0.192,"15min":0.186},"max":{"1min":7.069,"5min":82.218999999999994,"15min":82.218999999999994},"last":0.72299999999999998},{"interface":"front","average":{"1min":1.601,"5min":3.379,"15min":1.4770000000000001},"min":{"1min":0.193,"5min":0.191,"15min":0.191},"max":{"1min":4.3579999999999997,"5min":82.278999999999996,"15min":82.278999999999996},"last":0.83599999999999997}]},{"osd":2,"last update":"Wed Sep 11 07:34:45 2024","interfaces":[{"interface":"back","average":{"1min":1.6799999999999999,"5min":1.798,"15min":0.89500000000000002},"min":{"1min":0.22700000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":7.2229999999999999,"5min":7.2229999999999999,"15min":7.2229999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.9530000000000001,"15min":0.96199999999999997},"min":{"1min":0.32300000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":5.4059999999999997,"5min":8.7949999999999999,"15min":8.9559999999999995},"last":0.78300000000000003}]},{"osd":3,"last update":"Wed Sep 11 07:35:04 2024","interfaces":[{"interface":"back","average":{"1min":2.0499999999999998,"5min":3.5009999999999999,"15min":1.5469999999999999},"min":{"1min":0.60499999999999998,"5min":0.28000000000000003,"15min":0.25800000000000001},"max":{"1min":6.8109999999999999,"5min":81.882999999999996,"15min":81.882999999999996},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":3.4079999999999999,"15min":1.5089999999999999},"min":{"1min":0.22800000000000001,"5min":0.22600000000000001,"15min":0.20999999999999999},"max":{"1min":6.3460000000000001,"5min":82.400000000000006,"15min":82.400000000000006},"last":0.61599999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:35:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.194,"15min":0.79200000000000004},"min":{"1min":0.30199999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":3.98,"5min":7.3630000000000004,"15min":7.3630000000000004},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.351,"5min":1.456,"15min":0.93100000000000005},"min":{"1min":0.34999999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":4.883,"5min":6.2930000000000001,"15min":8.9039999999999999},"last":0.76100000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:34:47 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.2929999999999999,"15min":0.81000000000000005},"min":{"1min":0.33500000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":7.4379999999999997,"5min":7.4379999999999997,"15min":7.4379999999999997},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.659,"5min":1.554,"15min":0.91700000000000004},"min":{"1min":0.38300000000000001,"5min":0.316,"15min":0.308},"max":{"1min":5.4400000000000004,"5min":6.9850000000000003,"15min":8.8759999999999994},"last":1.4330000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:35:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.4399999999999999,"15min":0.98999999999999999},"min":{"1min":0.32400000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":4.601,"5min":6.3769999999999998,"15min":24.018000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.873,"5min":1.143,"15min":0.872},"min":{"1min":0.36699999999999999,"5min":0.30499999999999999,"15min":0.29999999999999999},"max":{"1min":2.8119999999999998,"5min":5.6449999999999996,"15min":23.681000000000001},"last":0.80400000000000005}]},{"osd":7,"last update":"Wed Sep 11 07:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.234,"15min":1.3129999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":2.8479999999999999,"5min":7.0940000000000003,"15min":7.0940000000000003},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.389,"15min":1.4850000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":4.0140000000000002,"5min":7.4050000000000002,"15min":7.4050000000000002},"last":0.91200000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738512,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3024928,"kb_used_data":2719252,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":90720224,"statfs":{"total":95995035648,"available":92897509376,"internally_reserved":0,"allocated":2784514048,"data_stored":2784148020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:35:14 2024","interfaces":[{"interface":"back","average":{"1min":1.71,"5min":1.5449999999999999,"15min":0.83599999999999997},"min":{"1min":0.28299999999999997,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":4.5389999999999997,"5min":9.7710000000000008,"15min":9.7710000000000008},"last":2.0299999999999998},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4470000000000001,"15min":0.83899999999999997},"min":{"1min":0.19800000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":4.7560000000000002,"5min":9.5310000000000006,"15min":10.275},"last":0.77600000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:34:44 2024","interfaces":[{"interface":"back","average":{"1min":1.657,"5min":2.1200000000000001,"15min":1.0049999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.20200000000000001},"max":{"1min":5.923,"5min":19.527999999999999,"15min":19.527999999999999},"last":2.3610000000000002},{"interface":"front","average":{"1min":1.401,"5min":2.008,"15min":0.98899999999999999},"min":{"1min":0.29799999999999999,"5min":0.22700000000000001,"15min":0.20999999999999999},"max":{"1min":5.0800000000000001,"5min":19.556999999999999,"15min":19.556999999999999},"last":0.21099999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:35:14 2024","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":2.7490000000000001,"15min":1.5609999999999999},"min":{"1min":0.38700000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":5.2489999999999997,"5min":45.185000000000002,"15min":45.185000000000002},"last":1.196},{"interface":"front","average":{"1min":1.6890000000000001,"5min":2.3679999999999999,"15min":1.365},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":8.0999999999999996,"5min":43.228999999999999,"15min":43.228999999999999},"last":2.161}]},{"osd":4,"last update":"Wed Sep 11 07:35:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.3180000000000001,"15min":0.88200000000000001},"min":{"1min":0.28399999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":4.8230000000000004,"5min":19.411999999999999,"15min":19.411999999999999},"last":0.26800000000000002},{"interface":"front","average":{"1min":1.089,"5min":1.6220000000000001,"15min":1.0269999999999999},"min":{"1min":0.30599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":5.1310000000000002,"5min":19.396000000000001,"15min":19.396000000000001},"last":0.47599999999999998}]},{"osd":5,"last update":"Wed Sep 11 07:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":1.54,"15min":0.94099999999999995},"min":{"1min":0.40899999999999997,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.843,"5min":19.381,"15min":19.381},"last":0.29599999999999999},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.9330000000000001,"15min":1.0660000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":5.0919999999999996,"5min":19.437000000000001,"15min":19.437000000000001},"last":0.44600000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:35:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.8169999999999999,"15min":1.079},"min":{"1min":0.33300000000000002,"5min":0.316,"15min":0.316},"max":{"1min":5.1630000000000003,"5min":23.190999999999999,"15min":23.190999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.96,"15min":1.1080000000000001},"min":{"1min":0.38900000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":3.778,"5min":23.632000000000001,"15min":23.632000000000001},"last":0.35999999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:35:06 2024","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.502,"15min":1.7310000000000001},"min":{"1min":0.41999999999999998,"5min":0.374,"15min":0.374},"max":{"1min":5.1920000000000002,"5min":5.3890000000000002,"15min":5.3890000000000002},"last":0.501},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.544,"15min":1.7989999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":3.677,"5min":5.2759999999999998,"15min":5.2759999999999998},"last":0.315}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2763325440,"data_stored":2763325440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3245867008,"data_stored":3245867008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2765557760,"data_stored":2765553688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2753495040,"data_stored":2753495040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2308505600,"data_stored":2308505600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3666874368,"data_stored":3666870296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2298019840,"data_stored":2298019840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2294288384,"data_stored":2294284312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:35:31.617 INFO:tasks.thrashosds.thrasher:Setting 4.17 to [2, 6, 4, 1, 3, 5] 2024-09-11T07:35:31.617 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '4.17', '2', '6', '4', '1', '3', '5'] 2024-09-11T07:35:31.617 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pg-upmap-items 4.17 2 6 4 1 3 5 2024-09-11T07:35:31.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:31.619 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:31.619 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:31.720 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:31.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:31.750 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:31.750 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:31.851 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:31.894 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:31.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:31.896 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:31.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:31 smithi022 ceph-mon[29462]: 8.4 scrub starts 2024-09-11T07:35:31.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:31 smithi022 ceph-mon[29462]: 2.1a scrub starts 2024-09-11T07:35:31.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:31 smithi022 ceph-mon[29462]: 2.1a scrub ok 2024-09-11T07:35:31.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:31 smithi022 ceph-mon[29462]: 8.2 scrub starts 2024-09-11T07:35:31.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:31 smithi022 ceph-mon[29462]: 8.7 scrub starts 2024-09-11T07:35:31.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:31 smithi022 ceph-mon[29462]: 8.c scrub starts 2024-09-11T07:35:31.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:31 smithi022 ceph-mon[29462]: 6.15 scrub starts 2024-09-11T07:35:31.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:31 smithi022 ceph-mon[37330]: 8.4 scrub starts 2024-09-11T07:35:31.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:31 smithi022 ceph-mon[37330]: 2.1a scrub starts 2024-09-11T07:35:31.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:31 smithi022 ceph-mon[37330]: 2.1a scrub ok 2024-09-11T07:35:31.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:31 smithi022 ceph-mon[37330]: 8.2 scrub starts 2024-09-11T07:35:31.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:31 smithi022 ceph-mon[37330]: 8.7 scrub starts 2024-09-11T07:35:31.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:31 smithi022 ceph-mon[37330]: 8.c scrub starts 2024-09-11T07:35:31.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:31 smithi022 ceph-mon[37330]: 6.15 scrub starts 2024-09-11T07:35:31.996 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:32.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:31 smithi104 ceph-mon[33507]: 8.4 scrub starts 2024-09-11T07:35:32.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:31 smithi104 ceph-mon[33507]: 2.1a scrub starts 2024-09-11T07:35:32.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:31 smithi104 ceph-mon[33507]: 2.1a scrub ok 2024-09-11T07:35:32.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:31 smithi104 ceph-mon[33507]: 8.2 scrub starts 2024-09-11T07:35:32.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:31 smithi104 ceph-mon[33507]: 8.7 scrub starts 2024-09-11T07:35:32.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:31 smithi104 ceph-mon[33507]: 8.c scrub starts 2024-09-11T07:35:32.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:31 smithi104 ceph-mon[33507]: 6.15 scrub starts 2024-09-11T07:35:32.056 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:32.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:32.057 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:32.158 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:32.203 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:32.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:32.204 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:32.305 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:32.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:32.337 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:32.337 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:32.438 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:32.489 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:32.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:32.490 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:32.591 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:32.619 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:32.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:32.620 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:32.721 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:32.753 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:32.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:32.754 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:32.855 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:32.890 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:32 smithi022 ceph-mon[29462]: 6.1b scrub starts 2024-09-11T07:35:32.890 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:32 smithi022 ceph-mon[29462]: 6.1b scrub ok 2024-09-11T07:35:32.890 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:32 smithi022 ceph-mon[29462]: 6.15 scrub ok 2024-09-11T07:35:32.891 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:32 smithi022 ceph-mon[29462]: 8.1 scrub starts 2024-09-11T07:35:32.891 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:32 smithi022 ceph-mon[29462]: 8.a scrub ok 2024-09-11T07:35:32.891 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:32 smithi022 ceph-mon[29462]: pgmap v401: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.22k op/s 2024-09-11T07:35:32.891 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:32 smithi022 ceph-mon[37330]: 6.1b scrub starts 2024-09-11T07:35:32.891 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:32 smithi022 ceph-mon[37330]: 6.1b scrub ok 2024-09-11T07:35:32.891 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:32 smithi022 ceph-mon[37330]: 6.15 scrub ok 2024-09-11T07:35:32.892 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:32 smithi022 ceph-mon[37330]: 8.1 scrub starts 2024-09-11T07:35:32.892 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:32 smithi022 ceph-mon[37330]: 8.a scrub ok 2024-09-11T07:35:32.892 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:32 smithi022 ceph-mon[37330]: pgmap v401: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.22k op/s 2024-09-11T07:35:32.895 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:32.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:32.896 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:32.997 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:33.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:32 smithi104 ceph-mon[33507]: 6.1b scrub starts 2024-09-11T07:35:33.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:32 smithi104 ceph-mon[33507]: 6.1b scrub ok 2024-09-11T07:35:33.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:32 smithi104 ceph-mon[33507]: 6.15 scrub ok 2024-09-11T07:35:33.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:32 smithi104 ceph-mon[33507]: 8.1 scrub starts 2024-09-11T07:35:33.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:32 smithi104 ceph-mon[33507]: 8.a scrub ok 2024-09-11T07:35:33.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:32 smithi104 ceph-mon[33507]: pgmap v401: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.22k op/s 2024-09-11T07:35:33.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:33.042 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:33.042 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:33.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:33.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:33.185 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:33.185 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:33.286 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:33.309 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:33.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:33.310 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:33.411 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:33.438 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:33.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:33.439 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:33.540 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:33.569 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:35:33.603 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-11T07:35:33.604 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:33.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:33.607 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:33.708 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:33.732 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:33.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:33.733 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:33.798 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:33 smithi022 ceph-mon[37330]: 2.1b scrub starts 2024-09-11T07:35:33.799 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:33 smithi022 ceph-mon[37330]: 2.1b scrub ok 2024-09-11T07:35:33.799 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:33 smithi022 ceph-mon[37330]: 8.4 scrub ok 2024-09-11T07:35:33.799 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:33 smithi022 ceph-mon[37330]: 5.1f scrub starts 2024-09-11T07:35:33.799 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:33 smithi022 ceph-mon[37330]: 5.1f scrub ok 2024-09-11T07:35:33.799 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:33 smithi022 ceph-mon[37330]: 8.2 scrub ok 2024-09-11T07:35:33.799 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:33 smithi022 ceph-mon[29462]: 2.1b scrub starts 2024-09-11T07:35:33.799 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:33 smithi022 ceph-mon[29462]: 2.1b scrub ok 2024-09-11T07:35:33.799 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:33 smithi022 ceph-mon[29462]: 8.4 scrub ok 2024-09-11T07:35:33.799 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:33 smithi022 ceph-mon[29462]: 5.1f scrub starts 2024-09-11T07:35:33.799 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:33 smithi022 ceph-mon[29462]: 5.1f scrub ok 2024-09-11T07:35:33.799 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:33 smithi022 ceph-mon[29462]: 8.2 scrub ok 2024-09-11T07:35:33.834 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:33.856 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:33.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:33.858 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:33.958 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:33.986 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:33.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:33.986 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:34.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:33 smithi104 ceph-mon[33507]: 2.1b scrub starts 2024-09-11T07:35:34.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:33 smithi104 ceph-mon[33507]: 2.1b scrub ok 2024-09-11T07:35:34.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:33 smithi104 ceph-mon[33507]: 8.4 scrub ok 2024-09-11T07:35:34.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:33 smithi104 ceph-mon[33507]: 5.1f scrub starts 2024-09-11T07:35:34.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:33 smithi104 ceph-mon[33507]: 5.1f scrub ok 2024-09-11T07:35:34.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:33 smithi104 ceph-mon[33507]: 8.2 scrub ok 2024-09-11T07:35:34.087 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:34.113 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:34.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:34.114 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:34.215 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:34.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:34.261 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:34.261 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:34.362 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:34.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:34.391 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:34.391 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:34.492 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:34.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:34.518 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:34.519 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:34.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:35:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:35:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:35:34.620 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:34.646 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:34.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:34.647 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:34.748 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:34.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:34.793 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:34.793 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:34.894 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:34.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:34 smithi022 ceph-mon[29462]: 4.11 scrub starts 2024-09-11T07:35:34.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:34 smithi022 ceph-mon[29462]: 4.11 scrub ok 2024-09-11T07:35:34.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:34 smithi022 ceph-mon[29462]: pgmap v402: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.4 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 938 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T07:35:34.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:34 smithi022 ceph-mon[29462]: 8.c scrub ok 2024-09-11T07:35:34.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:34 smithi022 ceph-mon[37330]: 4.11 scrub starts 2024-09-11T07:35:34.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:34 smithi022 ceph-mon[37330]: 4.11 scrub ok 2024-09-11T07:35:34.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:34 smithi022 ceph-mon[37330]: pgmap v402: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.4 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 938 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T07:35:34.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:34 smithi022 ceph-mon[37330]: 8.c scrub ok 2024-09-11T07:35:34.922 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:34.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:34.923 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:35.024 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:35.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:34 smithi104 ceph-mon[33507]: 4.11 scrub starts 2024-09-11T07:35:35.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:34 smithi104 ceph-mon[33507]: 4.11 scrub ok 2024-09-11T07:35:35.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:34 smithi104 ceph-mon[33507]: pgmap v402: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.4 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 938 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T07:35:35.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:34 smithi104 ceph-mon[33507]: 8.c scrub ok 2024-09-11T07:35:35.066 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:35.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:35.067 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:35.168 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:35.208 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:35.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:35.209 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:35.310 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:35.347 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:35.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:35.348 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:35.448 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:35.480 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:35.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:35.481 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:35.581 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:35.603 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:35.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:35.604 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:35.705 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:35.743 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:35.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:35.744 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:35.845 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:35.875 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:35.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:35.876 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:35.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:35 smithi022 ceph-mon[29462]: 5.1b scrub starts 2024-09-11T07:35:35.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:35 smithi022 ceph-mon[29462]: 5.1b scrub ok 2024-09-11T07:35:35.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:35 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4123264703' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:35:35.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:35 smithi022 ceph-mon[29462]: 8.b scrub starts 2024-09-11T07:35:35.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:35 smithi022 ceph-mon[29462]: 8.6 scrub starts 2024-09-11T07:35:35.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:35 smithi022 ceph-mon[37330]: 5.1b scrub starts 2024-09-11T07:35:35.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:35 smithi022 ceph-mon[37330]: 5.1b scrub ok 2024-09-11T07:35:35.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:35 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4123264703' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:35:35.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:35 smithi022 ceph-mon[37330]: 8.b scrub starts 2024-09-11T07:35:35.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:35 smithi022 ceph-mon[37330]: 8.6 scrub starts 2024-09-11T07:35:35.976 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:36.004 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:36.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:36.005 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:36.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:35 smithi104 ceph-mon[33507]: 5.1b scrub starts 2024-09-11T07:35:36.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:35 smithi104 ceph-mon[33507]: 5.1b scrub ok 2024-09-11T07:35:36.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:35 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4123264703' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:35:36.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:35 smithi104 ceph-mon[33507]: 8.b scrub starts 2024-09-11T07:35:36.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:35 smithi104 ceph-mon[33507]: 8.6 scrub starts 2024-09-11T07:35:36.106 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:36.139 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:36.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:36.140 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:36.241 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:36.278 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:36.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:36.279 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:36.380 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:36.410 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:36.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:36.412 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:36.512 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:36.554 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:36.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:36.556 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:36.656 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:36.687 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:36.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:36.688 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:36.789 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:36.811 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:36.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:36.812 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:36.912 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:36.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[29462]: 8.d scrub starts 2024-09-11T07:35:36.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[29462]: 8.7 scrub ok 2024-09-11T07:35:36.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[29462]: 4.5 scrub starts 2024-09-11T07:35:36.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[29462]: 4.5 scrub ok 2024-09-11T07:35:36.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[29462]: 5.4 scrub starts 2024-09-11T07:35:36.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[29462]: 5.4 scrub ok 2024-09-11T07:35:36.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[29462]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:35:36.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[29462]: Cluster is now healthy 2024-09-11T07:35:36.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4123264703' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:35:36.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[29462]: osdmap e219: 8 total, 8 up, 8 in 2024-09-11T07:35:36.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[29462]: 8.1 scrub ok 2024-09-11T07:35:36.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[29462]: 6.4 scrub starts 2024-09-11T07:35:36.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[29462]: 6.4 scrub ok 2024-09-11T07:35:36.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[29462]: 8.e scrub starts 2024-09-11T07:35:36.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[29462]: pgmap v404: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.11k op/s 2024-09-11T07:35:36.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[37330]: 8.d scrub starts 2024-09-11T07:35:36.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[37330]: 8.7 scrub ok 2024-09-11T07:35:36.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[37330]: 4.5 scrub starts 2024-09-11T07:35:36.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[37330]: 4.5 scrub ok 2024-09-11T07:35:36.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[37330]: 5.4 scrub starts 2024-09-11T07:35:36.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[37330]: 5.4 scrub ok 2024-09-11T07:35:36.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[37330]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:35:36.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[37330]: Cluster is now healthy 2024-09-11T07:35:36.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4123264703' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:35:36.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[37330]: osdmap e219: 8 total, 8 up, 8 in 2024-09-11T07:35:36.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[37330]: 8.1 scrub ok 2024-09-11T07:35:36.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[37330]: 6.4 scrub starts 2024-09-11T07:35:36.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[37330]: 6.4 scrub ok 2024-09-11T07:35:36.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[37330]: 8.e scrub starts 2024-09-11T07:35:36.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:36 smithi022 ceph-mon[37330]: pgmap v404: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.11k op/s 2024-09-11T07:35:36.953 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:36.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:36.954 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:37.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:36 smithi104 ceph-mon[33507]: 8.d scrub starts 2024-09-11T07:35:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:36 smithi104 ceph-mon[33507]: 8.7 scrub ok 2024-09-11T07:35:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:36 smithi104 ceph-mon[33507]: 4.5 scrub starts 2024-09-11T07:35:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:36 smithi104 ceph-mon[33507]: 4.5 scrub ok 2024-09-11T07:35:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:36 smithi104 ceph-mon[33507]: 5.4 scrub starts 2024-09-11T07:35:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:36 smithi104 ceph-mon[33507]: 5.4 scrub ok 2024-09-11T07:35:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:36 smithi104 ceph-mon[33507]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:35:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:36 smithi104 ceph-mon[33507]: Cluster is now healthy 2024-09-11T07:35:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:36 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4123264703' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:35:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:36 smithi104 ceph-mon[33507]: osdmap e219: 8 total, 8 up, 8 in 2024-09-11T07:35:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:36 smithi104 ceph-mon[33507]: 8.1 scrub ok 2024-09-11T07:35:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:36 smithi104 ceph-mon[33507]: 6.4 scrub starts 2024-09-11T07:35:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:36 smithi104 ceph-mon[33507]: 6.4 scrub ok 2024-09-11T07:35:37.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:36 smithi104 ceph-mon[33507]: 8.e scrub starts 2024-09-11T07:35:37.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:36 smithi104 ceph-mon[33507]: pgmap v404: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.11k op/s 2024-09-11T07:35:37.055 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:37.078 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:37.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:37.079 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:37.179 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:37.218 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:37.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:37.219 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:37.320 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:37.361 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:37.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:37.362 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:37.463 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:37.485 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:37.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:37.486 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:37.587 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:37.613 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:37.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:37.614 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:37.715 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:37.742 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:37.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:37.743 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:37.844 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:37.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:37.889 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:37.890 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:37.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:37 smithi022 ceph-mon[29462]: 4.1f scrub starts 2024-09-11T07:35:37.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:37 smithi022 ceph-mon[29462]: 4.1f scrub ok 2024-09-11T07:35:37.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:37 smithi022 ceph-mon[29462]: 2.1 scrub starts 2024-09-11T07:35:37.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:37 smithi022 ceph-mon[29462]: 2.1 scrub ok 2024-09-11T07:35:37.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:37 smithi022 ceph-mon[29462]: 4.17 scrub starts 2024-09-11T07:35:37.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:37 smithi022 ceph-mon[29462]: 4.17 scrub ok 2024-09-11T07:35:37.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:37 smithi022 ceph-mon[37330]: 4.1f scrub starts 2024-09-11T07:35:37.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:37 smithi022 ceph-mon[37330]: 4.1f scrub ok 2024-09-11T07:35:37.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:37 smithi022 ceph-mon[37330]: 2.1 scrub starts 2024-09-11T07:35:37.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:37 smithi022 ceph-mon[37330]: 2.1 scrub ok 2024-09-11T07:35:37.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:37 smithi022 ceph-mon[37330]: 4.17 scrub starts 2024-09-11T07:35:37.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:37 smithi022 ceph-mon[37330]: 4.17 scrub ok 2024-09-11T07:35:37.990 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:38.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:38.025 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:38.025 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:38.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:37 smithi104 ceph-mon[33507]: 4.1f scrub starts 2024-09-11T07:35:38.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:37 smithi104 ceph-mon[33507]: 4.1f scrub ok 2024-09-11T07:35:38.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:37 smithi104 ceph-mon[33507]: 2.1 scrub starts 2024-09-11T07:35:38.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:37 smithi104 ceph-mon[33507]: 2.1 scrub ok 2024-09-11T07:35:38.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:37 smithi104 ceph-mon[33507]: 4.17 scrub starts 2024-09-11T07:35:38.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:37 smithi104 ceph-mon[33507]: 4.17 scrub ok 2024-09-11T07:35:38.126 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:38.159 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:38.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:38.161 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:38.261 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:38.294 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:38.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:38.295 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:38.396 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:38.423 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:38.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:38.424 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:38.525 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:38.549 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:38.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:38.550 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:38.651 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:38.687 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:38.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:38.689 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:38.696 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is unset 2024-09-11T07:35:38.696 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:38 smithi022 ceph-mon[29462]: 2.3 scrub starts 2024-09-11T07:35:38.697 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:38 smithi022 ceph-mon[29462]: 2.3 scrub ok 2024-09-11T07:35:38.697 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:38 smithi022 ceph-mon[29462]: 8.d scrub ok 2024-09-11T07:35:38.697 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:38 smithi022 ceph-mon[29462]: 5.b scrub starts 2024-09-11T07:35:38.697 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:38 smithi022 ceph-mon[29462]: 5.b scrub ok 2024-09-11T07:35:38.697 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:38 smithi022 ceph-mon[29462]: 5.9 scrub starts 2024-09-11T07:35:38.697 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:38 smithi022 ceph-mon[29462]: 5.9 scrub ok 2024-09-11T07:35:38.697 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:38 smithi022 ceph-mon[29462]: 8.0 scrub starts 2024-09-11T07:35:38.697 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:35:38.697 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:35:38.697 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:38 smithi022 ceph-mon[29462]: pgmap v405: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.11k op/s 2024-09-11T07:35:38.698 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:38 smithi022 ceph-mon[29462]: 8.6 scrub ok 2024-09-11T07:35:38.698 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:38 smithi022 ceph-mon[37330]: 2.3 scrub starts 2024-09-11T07:35:38.698 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:38 smithi022 ceph-mon[37330]: 2.3 scrub ok 2024-09-11T07:35:38.698 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:38 smithi022 ceph-mon[37330]: 8.d scrub ok 2024-09-11T07:35:38.698 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:38 smithi022 ceph-mon[37330]: 5.b scrub starts 2024-09-11T07:35:38.698 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:38 smithi022 ceph-mon[37330]: 5.b scrub ok 2024-09-11T07:35:38.698 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:38 smithi022 ceph-mon[37330]: 5.9 scrub starts 2024-09-11T07:35:38.698 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:38 smithi022 ceph-mon[37330]: 5.9 scrub ok 2024-09-11T07:35:38.698 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:38 smithi022 ceph-mon[37330]: 8.0 scrub starts 2024-09-11T07:35:38.699 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:35:38.699 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:35:38.699 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:38 smithi022 ceph-mon[37330]: pgmap v405: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.11k op/s 2024-09-11T07:35:38.699 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:38 smithi022 ceph-mon[37330]: 8.6 scrub ok 2024-09-11T07:35:38.789 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:38.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:38.832 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:38.832 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:38.933 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:38.954 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:38.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:38.955 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:39.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:38 smithi104 ceph-mon[33507]: 2.3 scrub starts 2024-09-11T07:35:39.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:38 smithi104 ceph-mon[33507]: 2.3 scrub ok 2024-09-11T07:35:39.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:38 smithi104 ceph-mon[33507]: 8.d scrub ok 2024-09-11T07:35:39.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:38 smithi104 ceph-mon[33507]: 5.b scrub starts 2024-09-11T07:35:39.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:38 smithi104 ceph-mon[33507]: 5.b scrub ok 2024-09-11T07:35:39.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:38 smithi104 ceph-mon[33507]: 5.9 scrub starts 2024-09-11T07:35:39.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:38 smithi104 ceph-mon[33507]: 5.9 scrub ok 2024-09-11T07:35:39.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:38 smithi104 ceph-mon[33507]: 8.0 scrub starts 2024-09-11T07:35:39.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:35:39.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:35:39.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:38 smithi104 ceph-mon[33507]: pgmap v405: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.11k op/s 2024-09-11T07:35:39.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:38 smithi104 ceph-mon[33507]: 8.6 scrub ok 2024-09-11T07:35:39.056 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:39.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:39.084 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:39.084 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:39.185 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:39.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:39.224 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:39.325 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:39.350 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:39.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:39.351 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:39.452 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:39.477 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:39.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:39.478 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:39.579 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:39.607 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:39.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:39.608 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:39.709 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:39.740 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:39.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:39.740 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:39.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:35:39 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:35:39.841 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:39.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:39.869 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:39.869 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:39.970 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:39.995 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:39.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:39.997 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:40.097 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:40.128 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:40.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:40.129 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:40.230 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:40.251 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:40.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:40.252 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:40.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:40 smithi104 ceph-mon[33507]: 8.f scrub starts 2024-09-11T07:35:40.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:40 smithi104 ceph-mon[33507]: 8.b scrub ok 2024-09-11T07:35:40.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:40 smithi104 ceph-mon[33507]: 2.1c scrub starts 2024-09-11T07:35:40.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:40 smithi104 ceph-mon[33507]: 2.1c scrub ok 2024-09-11T07:35:40.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:40 smithi104 ceph-mon[33507]: 5.3 scrub starts 2024-09-11T07:35:40.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:40 smithi104 ceph-mon[33507]: 5.3 scrub ok 2024-09-11T07:35:40.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:40 smithi104 ceph-mon[33507]: 2.c scrub starts 2024-09-11T07:35:40.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:40 smithi104 ceph-mon[33507]: 2.c scrub ok 2024-09-11T07:35:40.353 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:40.376 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:40 smithi022 ceph-mon[29462]: 8.f scrub starts 2024-09-11T07:35:40.376 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:40 smithi022 ceph-mon[29462]: 8.b scrub ok 2024-09-11T07:35:40.377 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:40 smithi022 ceph-mon[29462]: 2.1c scrub starts 2024-09-11T07:35:40.377 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:40 smithi022 ceph-mon[29462]: 2.1c scrub ok 2024-09-11T07:35:40.377 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:40 smithi022 ceph-mon[29462]: 5.3 scrub starts 2024-09-11T07:35:40.377 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:40 smithi022 ceph-mon[29462]: 5.3 scrub ok 2024-09-11T07:35:40.377 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:40 smithi022 ceph-mon[29462]: 2.c scrub starts 2024-09-11T07:35:40.377 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:40 smithi022 ceph-mon[29462]: 2.c scrub ok 2024-09-11T07:35:40.377 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:40 smithi022 ceph-mon[37330]: 8.f scrub starts 2024-09-11T07:35:40.377 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:40 smithi022 ceph-mon[37330]: 8.b scrub ok 2024-09-11T07:35:40.377 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:40 smithi022 ceph-mon[37330]: 2.1c scrub starts 2024-09-11T07:35:40.377 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:40 smithi022 ceph-mon[37330]: 2.1c scrub ok 2024-09-11T07:35:40.377 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:40 smithi022 ceph-mon[37330]: 5.3 scrub starts 2024-09-11T07:35:40.377 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:40 smithi022 ceph-mon[37330]: 5.3 scrub ok 2024-09-11T07:35:40.378 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:40 smithi022 ceph-mon[37330]: 2.c scrub starts 2024-09-11T07:35:40.378 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:40 smithi022 ceph-mon[37330]: 2.c scrub ok 2024-09-11T07:35:40.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:40.384 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:40.384 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:40.485 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:40.526 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:40.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:40.527 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:40.627 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:40.656 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:40.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:40.658 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:40.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:35:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:35:40] "GET /metrics HTTP/1.1" 200 224111 "" "Prometheus/2.43.0" 2024-09-11T07:35:40.758 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:40.802 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:40.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:40.803 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:40.904 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:40.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:40.930 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:40.930 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:41.031 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:41.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:41.060 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:41.060 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:41.161 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:41.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:41 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:41.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:41 smithi022 ceph-mon[29462]: 2.a scrub starts 2024-09-11T07:35:41.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:41 smithi022 ceph-mon[29462]: 8.e scrub ok 2024-09-11T07:35:41.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:41 smithi022 ceph-mon[29462]: 2.a scrub ok 2024-09-11T07:35:41.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:41 smithi022 ceph-mon[29462]: 5.0 deep-scrub starts 2024-09-11T07:35:41.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:41 smithi022 ceph-mon[29462]: 5.0 deep-scrub ok 2024-09-11T07:35:41.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:41 smithi022 ceph-mon[29462]: 4.15 scrub starts 2024-09-11T07:35:41.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:41 smithi022 ceph-mon[29462]: 4.15 scrub ok 2024-09-11T07:35:41.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:41 smithi022 ceph-mon[29462]: pgmap v406: 148 pgs: 3 active+clean+scrubbing, 145 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 956 op/s 2024-09-11T07:35:41.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:41 smithi022 ceph-mon[29462]: 4.1c scrub starts 2024-09-11T07:35:41.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:41 smithi022 ceph-mon[29462]: 4.1c scrub ok 2024-09-11T07:35:41.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:41 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:41.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:41 smithi022 ceph-mon[37330]: 2.a scrub starts 2024-09-11T07:35:41.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:41 smithi022 ceph-mon[37330]: 8.e scrub ok 2024-09-11T07:35:41.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:41 smithi022 ceph-mon[37330]: 2.a scrub ok 2024-09-11T07:35:41.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:41 smithi022 ceph-mon[37330]: 5.0 deep-scrub starts 2024-09-11T07:35:41.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:41 smithi022 ceph-mon[37330]: 5.0 deep-scrub ok 2024-09-11T07:35:41.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:41 smithi022 ceph-mon[37330]: 4.15 scrub starts 2024-09-11T07:35:41.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:41 smithi022 ceph-mon[37330]: 4.15 scrub ok 2024-09-11T07:35:41.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:41 smithi022 ceph-mon[37330]: pgmap v406: 148 pgs: 3 active+clean+scrubbing, 145 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 956 op/s 2024-09-11T07:35:41.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:41 smithi022 ceph-mon[37330]: 4.1c scrub starts 2024-09-11T07:35:41.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:41 smithi022 ceph-mon[37330]: 4.1c scrub ok 2024-09-11T07:35:41.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:41.211 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:41.212 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:41.313 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:41.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:41.351 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:41.351 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:41.395 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:35:41.452 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:41.479 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:41.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:41.480 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:41 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:41 smithi104 ceph-mon[33507]: 2.a scrub starts 2024-09-11T07:35:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:41 smithi104 ceph-mon[33507]: 8.e scrub ok 2024-09-11T07:35:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:41 smithi104 ceph-mon[33507]: 2.a scrub ok 2024-09-11T07:35:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:41 smithi104 ceph-mon[33507]: 5.0 deep-scrub starts 2024-09-11T07:35:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:41 smithi104 ceph-mon[33507]: 5.0 deep-scrub ok 2024-09-11T07:35:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:41 smithi104 ceph-mon[33507]: 4.15 scrub starts 2024-09-11T07:35:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:41 smithi104 ceph-mon[33507]: 4.15 scrub ok 2024-09-11T07:35:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:41 smithi104 ceph-mon[33507]: pgmap v406: 148 pgs: 3 active+clean+scrubbing, 145 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 956 op/s 2024-09-11T07:35:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:41 smithi104 ceph-mon[33507]: 4.1c scrub starts 2024-09-11T07:35:41.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:41 smithi104 ceph-mon[33507]: 4.1c scrub ok 2024-09-11T07:35:41.581 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:41.608 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:41.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:41.609 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:41.709 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:41.749 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:41.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:41.750 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:41.851 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:41.876 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:41.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:41.877 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:41.891 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:35:41.979 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:42.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:42.012 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:42.012 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:42.113 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:42.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:42 smithi022 ceph-mon[29462]: 2.1d scrub starts 2024-09-11T07:35:42.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:42 smithi022 ceph-mon[29462]: 2.1d scrub ok 2024-09-11T07:35:42.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:42 smithi022 ceph-mon[29462]: 2.9 scrub starts 2024-09-11T07:35:42.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:42 smithi022 ceph-mon[29462]: 2.9 scrub ok 2024-09-11T07:35:42.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:42 smithi022 ceph-mon[29462]: 8.0 scrub ok 2024-09-11T07:35:42.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:42 smithi022 ceph-mon[29462]: 5.18 scrub starts 2024-09-11T07:35:42.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:42 smithi022 ceph-mon[29462]: 5.18 scrub ok 2024-09-11T07:35:42.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:42 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:35:42.120 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:42 smithi022 ceph-mon[37330]: 2.1d scrub starts 2024-09-11T07:35:42.120 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:42 smithi022 ceph-mon[37330]: 2.1d scrub ok 2024-09-11T07:35:42.120 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:42 smithi022 ceph-mon[37330]: 2.9 scrub starts 2024-09-11T07:35:42.120 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:42 smithi022 ceph-mon[37330]: 2.9 scrub ok 2024-09-11T07:35:42.120 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:42 smithi022 ceph-mon[37330]: 8.0 scrub ok 2024-09-11T07:35:42.120 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:42 smithi022 ceph-mon[37330]: 5.18 scrub starts 2024-09-11T07:35:42.120 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:42 smithi022 ceph-mon[37330]: 5.18 scrub ok 2024-09-11T07:35:42.120 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:42 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:35:42.159 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:42.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:42.160 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:42.261 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:42.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:42.303 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:42.303 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:42.404 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:42.435 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:42.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:42.437 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:42.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:42 smithi104 ceph-mon[33507]: 2.1d scrub starts 2024-09-11T07:35:42.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:42 smithi104 ceph-mon[33507]: 2.1d scrub ok 2024-09-11T07:35:42.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:42 smithi104 ceph-mon[33507]: 2.9 scrub starts 2024-09-11T07:35:42.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:42 smithi104 ceph-mon[33507]: 2.9 scrub ok 2024-09-11T07:35:42.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:42 smithi104 ceph-mon[33507]: 8.0 scrub ok 2024-09-11T07:35:42.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:42 smithi104 ceph-mon[33507]: 5.18 scrub starts 2024-09-11T07:35:42.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:42 smithi104 ceph-mon[33507]: 5.18 scrub ok 2024-09-11T07:35:42.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:42 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:35:42.537 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:42.566 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:42.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:42.567 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:42.667 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:42.697 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:42.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:42.698 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:42.799 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:42.834 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:42.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:42.836 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:42.936 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:42.971 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:42.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:42.972 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:43.072 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:43.107 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:43.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:43.108 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:43.209 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:43.235 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:43.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:43.237 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:43.337 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:43.375 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:43.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:43.376 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:43.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:43 smithi022 ceph-mon[29462]: 2.b scrub starts 2024-09-11T07:35:43.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:43 smithi022 ceph-mon[29462]: 2.b scrub ok 2024-09-11T07:35:43.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:43 smithi022 ceph-mon[29462]: pgmap v407: 148 pgs: 3 active+clean+scrubbing, 145 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 956 op/s 2024-09-11T07:35:43.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:43 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:35:43.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:43 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:35:43.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:43 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:35:43.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:43 smithi022 ceph-mon[37330]: 2.b scrub starts 2024-09-11T07:35:43.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:43 smithi022 ceph-mon[37330]: 2.b scrub ok 2024-09-11T07:35:43.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:43 smithi022 ceph-mon[37330]: pgmap v407: 148 pgs: 3 active+clean+scrubbing, 145 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 956 op/s 2024-09-11T07:35:43.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:43 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:35:43.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:43 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:35:43.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:43 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:35:43.477 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:43.504 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:43.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:43.506 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:43.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:43 smithi104 ceph-mon[33507]: 2.b scrub starts 2024-09-11T07:35:43.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:43 smithi104 ceph-mon[33507]: 2.b scrub ok 2024-09-11T07:35:43.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:43 smithi104 ceph-mon[33507]: pgmap v407: 148 pgs: 3 active+clean+scrubbing, 145 active+clean; 5.9 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 956 op/s 2024-09-11T07:35:43.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:43 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:35:43.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:43 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:35:43.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:43 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:35:43.607 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:43.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:43.636 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:43.636 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:43.737 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:43.761 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:43.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:43.762 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:43.862 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T07:35:43.862 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:43.888 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:43.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:43.889 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:43.990 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:44.017 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:44.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:44.018 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:44.072 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:35:44.073 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:35:44.112 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:35:44.112 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:35:44.118 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:44.146 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:35:44.146 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:35:44.147 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:44 smithi022 ceph-mon[29462]: 6.5 scrub starts 2024-09-11T07:35:44.147 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:44 smithi022 ceph-mon[29462]: 6.5 scrub ok 2024-09-11T07:35:44.147 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:44 smithi022 ceph-mon[29462]: 8.f scrub ok 2024-09-11T07:35:44.147 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:44 smithi022 ceph-mon[29462]: 6.f scrub starts 2024-09-11T07:35:44.147 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:44 smithi022 ceph-mon[29462]: 6.f scrub ok 2024-09-11T07:35:44.148 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:44 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4196139735' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:35:44.148 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:44 smithi022 ceph-mon[37330]: 6.5 scrub starts 2024-09-11T07:35:44.148 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:44 smithi022 ceph-mon[37330]: 6.5 scrub ok 2024-09-11T07:35:44.148 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:44 smithi022 ceph-mon[37330]: 8.f scrub ok 2024-09-11T07:35:44.148 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:44 smithi022 ceph-mon[37330]: 6.f scrub starts 2024-09-11T07:35:44.148 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:44 smithi022 ceph-mon[37330]: 6.f scrub ok 2024-09-11T07:35:44.148 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:44 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4196139735' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:35:44.175 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:44.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:44.176 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:44.182 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:35:44.182 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:35:44.213 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:35:44.213 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:35:44.246 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:35:44.247 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:35:44.277 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:35:44.277 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:35:44.278 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:44.310 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:35:44.310 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:35:44.323 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:44.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:44.324 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:44.425 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:44.463 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:44.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:44.464 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:44.530 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:35:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:35:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:35:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:44 smithi104 ceph-mon[33507]: 6.5 scrub starts 2024-09-11T07:35:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:44 smithi104 ceph-mon[33507]: 6.5 scrub ok 2024-09-11T07:35:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:44 smithi104 ceph-mon[33507]: 8.f scrub ok 2024-09-11T07:35:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:44 smithi104 ceph-mon[33507]: 6.f scrub starts 2024-09-11T07:35:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:44 smithi104 ceph-mon[33507]: 6.f scrub ok 2024-09-11T07:35:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:44 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4196139735' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:35:44.565 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:44.600 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:44.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:44.601 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:44.702 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:44.738 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:44.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:44.739 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:44.840 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:44.882 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:44.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:44.883 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:44.984 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:45.018 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:45.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:45.019 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:45.120 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:45.147 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:45.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:45.148 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:45.249 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:45.313 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:45.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:45.314 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:45.316 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:45 smithi022 ceph-mon[29462]: 2.0 scrub starts 2024-09-11T07:35:45.316 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:45 smithi022 ceph-mon[29462]: 2.0 scrub ok 2024-09-11T07:35:45.316 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:45 smithi022 ceph-mon[29462]: 4.1d deep-scrub starts 2024-09-11T07:35:45.316 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:45 smithi022 ceph-mon[29462]: 4.1d deep-scrub ok 2024-09-11T07:35:45.316 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:45 smithi022 ceph-mon[29462]: pgmap v408: 148 pgs: 3 active+clean+scrubbing, 145 active+clean; 5.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 965 op/s 2024-09-11T07:35:45.317 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:45 smithi022 ceph-mon[37330]: 2.0 scrub starts 2024-09-11T07:35:45.317 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:45 smithi022 ceph-mon[37330]: 2.0 scrub ok 2024-09-11T07:35:45.317 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:45 smithi022 ceph-mon[37330]: 4.1d deep-scrub starts 2024-09-11T07:35:45.317 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:45 smithi022 ceph-mon[37330]: 4.1d deep-scrub ok 2024-09-11T07:35:45.317 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:45 smithi022 ceph-mon[37330]: pgmap v408: 148 pgs: 3 active+clean+scrubbing, 145 active+clean; 5.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 965 op/s 2024-09-11T07:35:45.415 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:45.459 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:45.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:45.460 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:45 smithi104 ceph-mon[33507]: 2.0 scrub starts 2024-09-11T07:35:45.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:45 smithi104 ceph-mon[33507]: 2.0 scrub ok 2024-09-11T07:35:45.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:45 smithi104 ceph-mon[33507]: 4.1d deep-scrub starts 2024-09-11T07:35:45.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:45 smithi104 ceph-mon[33507]: 4.1d deep-scrub ok 2024-09-11T07:35:45.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:45 smithi104 ceph-mon[33507]: pgmap v408: 148 pgs: 3 active+clean+scrubbing, 145 active+clean; 5.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 965 op/s 2024-09-11T07:35:45.560 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:45.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:45.589 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:45.590 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:45.690 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:45.717 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:45.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:45.718 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:45.819 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:45.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:45.869 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:45.869 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:45.970 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:46.012 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:46.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:46.014 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:46.114 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:46.153 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:46.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:46.155 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:46.255 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:46.281 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:46.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:46.282 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:46.365 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:46 smithi022 ceph-mon[29462]: 6.19 deep-scrub starts 2024-09-11T07:35:46.365 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:46 smithi022 ceph-mon[29462]: 6.19 deep-scrub ok 2024-09-11T07:35:46.365 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:46 smithi022 ceph-mon[37330]: 6.19 deep-scrub starts 2024-09-11T07:35:46.365 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:46 smithi022 ceph-mon[37330]: 6.19 deep-scrub ok 2024-09-11T07:35:46.382 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:46.409 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:46.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:46.410 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:46.510 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:46.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:46 smithi104 ceph-mon[33507]: 6.19 deep-scrub starts 2024-09-11T07:35:46.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:46 smithi104 ceph-mon[33507]: 6.19 deep-scrub ok 2024-09-11T07:35:46.538 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:46.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:46.539 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:46.640 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:46.666 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:46.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:46.667 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:46.766 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:35:46.766 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 20, 2024-09-11T07:35:46.767 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 600, 2024-09-11T07:35:46.767 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:35:46.767 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.767 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:14416 8.1 8:8210eb53:::benchmark_data_smithi022_108406_object14415:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-11T07:35:46.767 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:45.110334+0000", 2024-09-11T07:35:46.767 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 121.418307676, 2024-09-11T07:35:46.767 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.15313612800000001, 2024-09-11T07:35:46.767 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:35:46.767 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:35:46.767 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:35:46.767 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:35:46.767 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:35:46.767 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 14416 2024-09-11T07:35:46.768 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.768 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:35:46.768 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.768 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:35:46.768 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110334+0000", 2024-09-11T07:35:46.768 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.768 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.768 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.768 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:35:46.768 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110334+0000", 2024-09-11T07:35:46.768 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.768 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.768 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.769 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:35:46.769 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110332+0000", 2024-09-11T07:35:46.769 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:35:46.769 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.769 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.769 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:35:46.769 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110363+0000", 2024-09-11T07:35:46.769 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.1136999999999997e-05 2024-09-11T07:35:46.769 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.769 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.769 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:35:46.769 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110374+0000", 2024-09-11T07:35:46.769 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.059e-05 2024-09-11T07:35:46.769 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.770 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.770 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:35:46.770 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110380+0000", 2024-09-11T07:35:46.770 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2539999999999998e-06 2024-09-11T07:35:46.770 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.770 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.770 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:35:46.770 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110564+0000", 2024-09-11T07:35:46.770 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000183399 2024-09-11T07:35:46.770 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.770 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.770 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:35:46.770 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110679+0000", 2024-09-11T07:35:46.770 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000115593 2024-09-11T07:35:46.771 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.771 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.771 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2", 2024-09-11T07:35:46.771 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110834+0000", 2024-09-11T07:35:46.771 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000154544 2024-09-11T07:35:46.771 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.771 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.771 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:46.771 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112819+0000", 2024-09-11T07:35:46.771 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0019854180000000001 2024-09-11T07:35:46.771 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.771 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.771 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:35:46.772 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.248545+0000", 2024-09-11T07:35:46.772 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13572600800000001 2024-09-11T07:35:46.772 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.772 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.772 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:46.772 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.263341+0000", 2024-09-11T07:35:46.772 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.014795523999999999 2024-09-11T07:35:46.772 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.772 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.772 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:35:46.772 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.263431+0000", 2024-09-11T07:35:46.772 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.0829999999999996e-05 2024-09-11T07:35:46.772 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.773 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.773 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:35:46.773 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.263470+0000", 2024-09-11T07:35:46.773 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8503e-05 2024-09-11T07:35:46.773 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.773 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:46.773 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.773 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.773 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.773 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.25072.0:14418 8.0 e187/177 8:0e01a02b:::benchmark_data_smithi022_108406_object14417:head v 187'897, mlcod=187'895)", 2024-09-11T07:35:46.773 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:45.111913+0000", 2024-09-11T07:35:46.773 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 121.41672817600001, 2024-09-11T07:35:46.774 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13747295400000001, 2024-09-11T07:35:46.774 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:35:46.774 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:35:46.774 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:35:46.774 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.774 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:35:46.774 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.111913+0000", 2024-09-11T07:35:46.774 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.774 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.774 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.774 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:35:46.774 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.111913+0000", 2024-09-11T07:35:46.774 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.775 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.775 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.775 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:35:46.775 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.111912+0000", 2024-09-11T07:35:46.775 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:35:46.775 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.775 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.775 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:35:46.775 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112806+0000", 2024-09-11T07:35:46.775 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00089435000000000003 2024-09-11T07:35:46.775 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.775 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.775 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:35:46.775 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112835+0000", 2024-09-11T07:35:46.775 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8394999999999998e-05 2024-09-11T07:35:46.776 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.776 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.776 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:35:46.776 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112841+0000", 2024-09-11T07:35:46.776 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0179999999999998e-06 2024-09-11T07:35:46.776 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.776 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.776 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:35:46.776 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112901+0000", 2024-09-11T07:35:46.776 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0677000000000002e-05 2024-09-11T07:35:46.776 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.776 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.776 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:35:46.776 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112920+0000", 2024-09-11T07:35:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8190999999999999e-05 2024-09-11T07:35:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:35:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.249346+0000", 2024-09-11T07:35:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13642599399999999 2024-09-11T07:35:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:35:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.249386+0000", 2024-09-11T07:35:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0661999999999998e-05 2024-09-11T07:35:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:14426 8.3 8:ce95c1b3:::benchmark_data_smithi022_108406_object14425:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-11T07:35:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:45.112842+0000", 2024-09-11T07:35:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 121.41579998, 2024-09-11T07:35:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13800079700000001, 2024-09-11T07:35:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:35:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:35:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:35:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:35:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:35:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 14426 2024-09-11T07:35:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:35:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:35:46.779 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112842+0000", 2024-09-11T07:35:46.779 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.779 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.779 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.779 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:35:46.779 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112842+0000", 2024-09-11T07:35:46.779 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.779 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.779 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.779 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:35:46.779 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112839+0000", 2024-09-11T07:35:46.780 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:35:46.780 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.780 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.780 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:35:46.780 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112865+0000", 2024-09-11T07:35:46.780 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5456000000000002e-05 2024-09-11T07:35:46.780 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.780 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.780 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:35:46.780 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112876+0000", 2024-09-11T07:35:46.780 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1080000000000001e-05 2024-09-11T07:35:46.780 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.781 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.781 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:35:46.781 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112882+0000", 2024-09-11T07:35:46.781 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6529999999999999e-06 2024-09-11T07:35:46.781 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.781 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.781 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:35:46.781 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112935+0000", 2024-09-11T07:35:46.781 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3224999999999999e-05 2024-09-11T07:35:46.781 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.781 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.781 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:35:46.782 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.113954+0000", 2024-09-11T07:35:46.782 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0010188510000000001 2024-09-11T07:35:46.782 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.782 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.782 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:35:46.782 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.248598+0000", 2024-09-11T07:35:46.782 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.134644186 2024-09-11T07:35:46.782 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.782 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.782 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:46.782 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.250736+0000", 2024-09-11T07:35:46.782 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0021376670000000002 2024-09-11T07:35:46.782 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.782 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.783 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:46.783 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.250771+0000", 2024-09-11T07:35:46.783 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5676999999999997e-05 2024-09-11T07:35:46.783 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.783 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.783 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:35:46.783 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.250778+0000", 2024-09-11T07:35:46.783 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1899999999999998e-06 2024-09-11T07:35:46.783 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.783 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.783 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:35:46.783 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.250811+0000", 2024-09-11T07:35:46.783 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2561000000000002e-05 2024-09-11T07:35:46.784 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.784 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.784 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:35:46.784 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.250842+0000", 2024-09-11T07:35:46.784 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.1313999999999999e-05 2024-09-11T07:35:46.784 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.784 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:46.784 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.784 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.784 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.784 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.25072.0:14417 8.f e187/177 8:fa15f9ae:::benchmark_data_smithi022_108406_object14416:head v 187'872, mlcod=187'870)", 2024-09-11T07:35:46.784 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:45.112869+0000", 2024-09-11T07:35:46.784 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 121.415772644, 2024-09-11T07:35:46.785 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.136026383, 2024-09-11T07:35:46.785 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:35:46.785 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:35:46.785 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:35:46.785 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.785 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:35:46.785 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112869+0000", 2024-09-11T07:35:46.785 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.785 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.785 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.785 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:35:46.785 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112869+0000", 2024-09-11T07:35:46.785 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.786 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.786 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.786 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:35:46.786 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112868+0000", 2024-09-11T07:35:46.786 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:35:46.786 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.786 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.786 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:35:46.786 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112894+0000", 2024-09-11T07:35:46.786 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5984000000000001e-05 2024-09-11T07:35:46.786 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.786 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.786 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:35:46.787 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112902+0000", 2024-09-11T07:35:46.787 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.661e-06 2024-09-11T07:35:46.787 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.787 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.787 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:35:46.787 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112906+0000", 2024-09-11T07:35:46.787 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7129999999999996e-06 2024-09-11T07:35:46.787 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.787 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.787 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:35:46.787 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112954+0000", 2024-09-11T07:35:46.787 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7562999999999998e-05 2024-09-11T07:35:46.787 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.788 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.788 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:35:46.788 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112973+0000", 2024-09-11T07:35:46.788 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8728000000000001e-05 2024-09-11T07:35:46.788 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.788 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.788 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:35:46.788 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.248865+0000", 2024-09-11T07:35:46.788 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.135891822 2024-09-11T07:35:46.788 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.788 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.788 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:35:46.788 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.248895+0000", 2024-09-11T07:35:46.788 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0667000000000003e-05 2024-09-11T07:35:46.789 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.789 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:46.789 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.789 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.789 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.789 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.25072.0:14419 8.f e187/177 8:fe75a06c:::benchmark_data_smithi022_108406_object14418:head v 187'873, mlcod=187'870)", 2024-09-11T07:35:46.789 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:45.112918+0000", 2024-09-11T07:35:46.789 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 121.415723551, 2024-09-11T07:35:46.789 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13626408100000001, 2024-09-11T07:35:46.789 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:35:46.789 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:35:46.789 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:35:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:35:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112918+0000", 2024-09-11T07:35:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:35:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112918+0000", 2024-09-11T07:35:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:35:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112917+0000", 2024-09-11T07:35:46.791 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:35:46.791 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.791 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.791 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:35:46.791 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112941+0000", 2024-09-11T07:35:46.791 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4009e-05 2024-09-11T07:35:46.791 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.791 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.791 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:35:46.791 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112947+0000", 2024-09-11T07:35:46.791 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2310000000000001e-06 2024-09-11T07:35:46.791 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.791 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.791 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:35:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112951+0000", 2024-09-11T07:35:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.534e-06 2024-09-11T07:35:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:35:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.113565+0000", 2024-09-11T07:35:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00061437799999999995 2024-09-11T07:35:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:35:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.113582+0000", 2024-09-11T07:35:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.7005999999999999e-05 2024-09-11T07:35:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:35:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.249154+0000", 2024-09-11T07:35:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.135571951 2024-09-11T07:35:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:35:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.249182+0000", 2024-09-11T07:35:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.779e-05 2024-09-11T07:35:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:14427 8.3 8:c0deb9a5:::benchmark_data_smithi022_108406_object14426:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-11T07:35:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:45.113496+0000", 2024-09-11T07:35:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 121.415145197, 2024-09-11T07:35:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.138501123, 2024-09-11T07:35:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:35:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:35:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:35:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:35:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:35:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 14427 2024-09-11T07:35:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:35:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:35:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.113496+0000", 2024-09-11T07:35:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:35:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.113496+0000", 2024-09-11T07:35:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:35:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.113495+0000", 2024-09-11T07:35:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:35:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:35:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.113519+0000", 2024-09-11T07:35:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4513000000000001e-05 2024-09-11T07:35:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:35:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.113530+0000", 2024-09-11T07:35:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1184e-05 2024-09-11T07:35:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:35:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.113540+0000", 2024-09-11T07:35:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.3200000000000006e-06 2024-09-11T07:35:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:35:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.249715+0000", 2024-09-11T07:35:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13617541799999999 2024-09-11T07:35:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:35:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.249864+0000", 2024-09-11T07:35:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0001485 2024-09-11T07:35:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:35:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.250007+0000", 2024-09-11T07:35:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000143506 2024-09-11T07:35:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:35:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.250857+0000", 2024-09-11T07:35:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00084956699999999997 2024-09-11T07:35:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.251805+0000", 2024-09-11T07:35:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000948699 2024-09-11T07:35:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.251933+0000", 2024-09-11T07:35:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012777399999999999 2024-09-11T07:35:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:35:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.251976+0000", 2024-09-11T07:35:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2935999999999997e-05 2024-09-11T07:35:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:35:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.251997+0000", 2024-09-11T07:35:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1409000000000001e-05 2024-09-11T07:35:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.800 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.800 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:14428 8.3 8:cb8a67ae:::benchmark_data_smithi022_108406_object14427:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-11T07:35:46.800 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:45.114053+0000", 2024-09-11T07:35:46.800 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 121.414588587, 2024-09-11T07:35:46.800 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13808647900000001, 2024-09-11T07:35:46.800 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:35:46.800 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:35:46.800 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:35:46.800 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:35:46.800 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:35:46.800 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 14428 2024-09-11T07:35:46.800 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.800 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:35:46.800 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.801 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:35:46.801 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.114053+0000", 2024-09-11T07:35:46.801 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.801 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.801 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.801 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:35:46.801 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.114053+0000", 2024-09-11T07:35:46.801 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.801 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.801 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.801 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:35:46.801 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.114052+0000", 2024-09-11T07:35:46.801 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:35:46.801 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.802 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.802 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:35:46.802 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.114076+0000", 2024-09-11T07:35:46.802 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4151e-05 2024-09-11T07:35:46.802 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.802 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.802 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:35:46.802 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.248565+0000", 2024-09-11T07:35:46.802 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13448981099999999 2024-09-11T07:35:46.802 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.802 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.802 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:35:46.802 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.248575+0000", 2024-09-11T07:35:46.802 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.1060000000000006e-06 2024-09-11T07:35:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:35:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.250884+0000", 2024-09-11T07:35:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0023091040000000002 2024-09-11T07:35:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:35:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.251008+0000", 2024-09-11T07:35:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012423800000000001 2024-09-11T07:35:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:35:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.251143+0000", 2024-09-11T07:35:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000135093 2024-09-11T07:35:46.804 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.804 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.804 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:46.804 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.252039+0000", 2024-09-11T07:35:46.804 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00089547900000000002 2024-09-11T07:35:46.804 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.804 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.804 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:35:46.804 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.252045+0000", 2024-09-11T07:35:46.804 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7710000000000001e-06 2024-09-11T07:35:46.804 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.804 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.804 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:46.805 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.252069+0000", 2024-09-11T07:35:46.805 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3805e-05 2024-09-11T07:35:46.805 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.805 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.805 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:35:46.805 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.252109+0000", 2024-09-11T07:35:46.805 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9674e-05 2024-09-11T07:35:46.805 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.805 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.805 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:35:46.805 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.252139+0000", 2024-09-11T07:35:46.805 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0633000000000003e-05 2024-09-11T07:35:46.805 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.805 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.25072.0:14429 8.0 e187/177 8:0b62d12a:::benchmark_data_smithi022_108406_object14428:head v 187'898, mlcod=187'895)", 2024-09-11T07:35:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:45.114263+0000", 2024-09-11T07:35:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 121.41437891, 2024-09-11T07:35:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13582166000000001, 2024-09-11T07:35:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:35:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:35:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:35:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:35:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.114263+0000", 2024-09-11T07:35:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.807 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.807 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.807 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:35:46.807 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.114263+0000", 2024-09-11T07:35:46.807 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.807 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.807 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.807 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:35:46.807 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.114261+0000", 2024-09-11T07:35:46.807 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:35:46.807 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.807 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.807 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:35:46.807 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.114287+0000", 2024-09-11T07:35:46.808 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.6108000000000001e-05 2024-09-11T07:35:46.808 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.808 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.808 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:35:46.808 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.248531+0000", 2024-09-11T07:35:46.808 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.134243208 2024-09-11T07:35:46.808 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.808 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.808 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:35:46.808 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.248542+0000", 2024-09-11T07:35:46.808 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1124000000000001e-05 2024-09-11T07:35:46.808 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.808 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:35:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.248648+0000", 2024-09-11T07:35:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010611 2024-09-11T07:35:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:35:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.248672+0000", 2024-09-11T07:35:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4176999999999999e-05 2024-09-11T07:35:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:35:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.250055+0000", 2024-09-11T07:35:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0013828029999999999 2024-09-11T07:35:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:35:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.250084+0000", 2024-09-11T07:35:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.9532000000000001e-05 2024-09-11T07:35:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:28693 8.1 8:8e0bd374:::benchmark_data_smithi022_108406_object28692:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e189)", 2024-09-11T07:35:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:51.016429+0000", 2024-09-11T07:35:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 115.512212576, 2024-09-11T07:35:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13608294000000001, 2024-09-11T07:35:46.811 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:35:46.811 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:35:46.811 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:35:46.811 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:35:46.811 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:35:46.811 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 28693 2024-09-11T07:35:46.811 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.811 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:35:46.811 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.811 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:35:46.811 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.016429+0000", 2024-09-11T07:35:46.811 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.811 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.811 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.812 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:35:46.812 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.016429+0000", 2024-09-11T07:35:46.812 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.812 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.812 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.812 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:35:46.812 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.016427+0000", 2024-09-11T07:35:46.812 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:35:46.812 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.812 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.812 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:35:46.812 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.016450+0000", 2024-09-11T07:35:46.812 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3689999999999998e-05 2024-09-11T07:35:46.813 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.813 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.813 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:35:46.813 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.016459+0000", 2024-09-11T07:35:46.813 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.9139999999999997e-06 2024-09-11T07:35:46.813 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.813 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.813 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:35:46.813 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.016463+0000", 2024-09-11T07:35:46.813 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0389999999999998e-06 2024-09-11T07:35:46.813 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.813 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.813 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:35:46.814 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.016504+0000", 2024-09-11T07:35:46.814 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0431e-05 2024-09-11T07:35:46.814 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.814 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.814 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:35:46.814 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.016643+0000", 2024-09-11T07:35:46.814 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013941600000000001 2024-09-11T07:35:46.814 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.814 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.814 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2", 2024-09-11T07:35:46.814 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.016792+0000", 2024-09-11T07:35:46.814 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014857299999999999 2024-09-11T07:35:46.814 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.814 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:35:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.018986+0000", 2024-09-11T07:35:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0021943700000000002 2024-09-11T07:35:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.019367+0000", 2024-09-11T07:35:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00038110500000000002 2024-09-11T07:35:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.152423+0000", 2024-09-11T07:35:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13305526600000001 2024-09-11T07:35:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.816 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.816 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:35:46.816 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.152475+0000", 2024-09-11T07:35:46.816 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.24e-05 2024-09-11T07:35:46.816 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.816 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.816 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:35:46.816 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.152512+0000", 2024-09-11T07:35:46.816 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6959000000000003e-05 2024-09-11T07:35:46.816 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.816 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:46.816 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.816 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.817 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.817 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:28701 8.7 8:ecc1746b:::benchmark_data_smithi022_108406_object28700:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e189)", 2024-09-11T07:35:46.817 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:51.018786+0000", 2024-09-11T07:35:46.817 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 115.50985523999999, 2024-09-11T07:35:46.817 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13377114300000001, 2024-09-11T07:35:46.817 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:35:46.817 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:35:46.817 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:35:46.817 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:35:46.817 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:35:46.817 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 28701 2024-09-11T07:35:46.817 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.817 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:35:46.818 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.818 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:35:46.818 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.018786+0000", 2024-09-11T07:35:46.818 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.818 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.818 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.818 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:35:46.818 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.018786+0000", 2024-09-11T07:35:46.818 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.818 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.818 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.818 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:35:46.818 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.018783+0000", 2024-09-11T07:35:46.818 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999967 2024-09-11T07:35:46.819 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.819 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.819 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:35:46.819 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.018809+0000", 2024-09-11T07:35:46.819 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.6658000000000001e-05 2024-09-11T07:35:46.819 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.819 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.819 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:35:46.819 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.018821+0000", 2024-09-11T07:35:46.819 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1901e-05 2024-09-11T07:35:46.819 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.819 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.819 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:35:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.018827+0000", 2024-09-11T07:35:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8409999999999996e-06 2024-09-11T07:35:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:35:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.018889+0000", 2024-09-11T07:35:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2242999999999993e-05 2024-09-11T07:35:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:35:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.019011+0000", 2024-09-11T07:35:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000121698 2024-09-11T07:35:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.821 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,6", 2024-09-11T07:35:46.821 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.019194+0000", 2024-09-11T07:35:46.821 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00018337799999999999 2024-09-11T07:35:46.821 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.821 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.821 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:35:46.821 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.020267+0000", 2024-09-11T07:35:46.821 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0010720580000000001 2024-09-11T07:35:46.821 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.821 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.821 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:46.821 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.020979+0000", 2024-09-11T07:35:46.822 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00071237700000000004 2024-09-11T07:35:46.822 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.822 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.822 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:46.822 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.152477+0000", 2024-09-11T07:35:46.822 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13149818399999999 2024-09-11T07:35:46.822 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.822 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.822 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:35:46.822 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.152522+0000", 2024-09-11T07:35:46.822 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4687000000000001e-05 2024-09-11T07:35:46.822 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.822 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.823 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:35:46.823 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.152557+0000", 2024-09-11T07:35:46.823 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5649999999999999e-05 2024-09-11T07:35:46.823 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.823 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:46.823 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.823 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.823 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.823 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:28707 8.1 8:89790372:::benchmark_data_smithi022_108406_object28706:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e189)", 2024-09-11T07:35:46.823 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:51.024536+0000", 2024-09-11T07:35:46.823 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 115.504105526, 2024-09-11T07:35:46.823 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13417187999999999, 2024-09-11T07:35:46.823 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:35:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:35:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:35:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:35:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:35:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 28707 2024-09-11T07:35:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:35:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:35:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.024536+0000", 2024-09-11T07:35:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:35:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.024536+0000", 2024-09-11T07:35:46.825 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.825 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.825 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.825 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:35:46.825 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.024535+0000", 2024-09-11T07:35:46.825 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:35:46.825 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.825 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.825 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:35:46.825 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.024555+0000", 2024-09-11T07:35:46.825 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0119000000000002e-05 2024-09-11T07:35:46.826 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.826 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.826 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:35:46.826 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.024564+0000", 2024-09-11T07:35:46.826 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.0250000000000008e-06 2024-09-11T07:35:46.826 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.826 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.826 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:35:46.826 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.024568+0000", 2024-09-11T07:35:46.826 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9380000000000002e-06 2024-09-11T07:35:46.826 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.826 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.826 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:35:46.827 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.024602+0000", 2024-09-11T07:35:46.827 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3686999999999998e-05 2024-09-11T07:35:46.827 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.827 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.827 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:35:46.827 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.024678+0000", 2024-09-11T07:35:46.827 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.6631999999999994e-05 2024-09-11T07:35:46.827 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.827 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.827 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2", 2024-09-11T07:35:46.827 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.024764+0000", 2024-09-11T07:35:46.827 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.5568000000000005e-05 2024-09-11T07:35:46.827 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.827 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.827 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:35:46.828 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.025621+0000", 2024-09-11T07:35:46.828 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00085749000000000003 2024-09-11T07:35:46.828 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.828 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.828 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:46.828 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.025859+0000", 2024-09-11T07:35:46.828 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000237696 2024-09-11T07:35:46.828 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.828 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.828 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:46.828 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.158613+0000", 2024-09-11T07:35:46.828 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.132754016 2024-09-11T07:35:46.829 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.829 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.829 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:35:46.829 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.158667+0000", 2024-09-11T07:35:46.829 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.4109e-05 2024-09-11T07:35:46.829 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.829 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.829 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:35:46.829 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.158708+0000", 2024-09-11T07:35:46.829 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0812000000000002e-05 2024-09-11T07:35:46.829 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.829 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:46.829 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.829 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.830 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.830 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:33531 8.3 8:c1221af9:::benchmark_data_smithi022_108406_object33530:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e190)", 2024-09-11T07:35:46.830 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:53.707041+0000", 2024-09-11T07:35:46.830 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 112.821600921, 2024-09-11T07:35:46.830 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.128257924, 2024-09-11T07:35:46.830 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:35:46.830 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:35:46.830 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:35:46.830 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:35:46.830 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:35:46.830 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 33531 2024-09-11T07:35:46.830 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.830 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:35:46.830 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.831 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:35:46.831 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:53.707041+0000", 2024-09-11T07:35:46.831 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.831 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.831 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.831 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:35:46.831 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:53.707041+0000", 2024-09-11T07:35:46.831 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.831 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.831 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.831 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:35:46.831 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:53.707040+0000", 2024-09-11T07:35:46.831 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:35:46.832 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.832 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.832 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:35:46.832 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:53.707068+0000", 2024-09-11T07:35:46.832 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8279000000000001e-05 2024-09-11T07:35:46.832 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.832 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.832 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:35:46.832 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:53.707075+0000", 2024-09-11T07:35:46.832 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.3679999999999996e-06 2024-09-11T07:35:46.832 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.832 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.832 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:35:46.832 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:53.707080+0000", 2024-09-11T07:35:46.832 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1239999999999998e-06 2024-09-11T07:35:46.833 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.833 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.833 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:35:46.833 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:53.709101+0000", 2024-09-11T07:35:46.833 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002021213 2024-09-11T07:35:46.833 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.833 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.833 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:35:46.833 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:53.709233+0000", 2024-09-11T07:35:46.833 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013198300000000001 2024-09-11T07:35:46.833 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.833 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.833 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:35:46.833 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:53.709384+0000", 2024-09-11T07:35:46.834 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015112799999999999 2024-09-11T07:35:46.834 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.834 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.834 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:35:46.834 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:53.710865+0000", 2024-09-11T07:35:46.834 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0014811329999999999 2024-09-11T07:35:46.834 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.834 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.834 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:46.834 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:53.835125+0000", 2024-09-11T07:35:46.834 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.124260359 2024-09-11T07:35:46.834 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.834 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:53.835158+0000", 2024-09-11T07:35:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2580999999999999e-05 2024-09-11T07:35:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:35:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:53.835259+0000", 2024-09-11T07:35:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000101096 2024-09-11T07:35:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:35:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:53.835299+0000", 2024-09-11T07:35:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9433e-05 2024-09-11T07:35:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.836 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.836 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.836 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:36490 8.1 8:86daf699:::benchmark_data_smithi022_108406_object36489:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e191)", 2024-09-11T07:35:46.836 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:55.563456+0000", 2024-09-11T07:35:46.836 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 110.965185664, 2024-09-11T07:35:46.836 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13399622999999999, 2024-09-11T07:35:46.836 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:35:46.836 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:35:46.836 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:35:46.836 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:35:46.836 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:35:46.836 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 36490 2024-09-11T07:35:46.837 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.837 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:35:46.837 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.837 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:35:46.837 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.563456+0000", 2024-09-11T07:35:46.837 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.837 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.837 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.837 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:35:46.837 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.563456+0000", 2024-09-11T07:35:46.837 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.837 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.838 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.838 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:35:46.838 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.563453+0000", 2024-09-11T07:35:46.838 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:35:46.838 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.838 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.838 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:35:46.838 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.563483+0000", 2024-09-11T07:35:46.838 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.9445e-05 2024-09-11T07:35:46.838 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.838 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.838 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:35:46.838 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.563495+0000", 2024-09-11T07:35:46.839 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2473000000000001e-05 2024-09-11T07:35:46.839 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.839 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.839 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:35:46.839 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.563502+0000", 2024-09-11T07:35:46.839 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.652e-06 2024-09-11T07:35:46.839 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.839 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.839 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:35:46.839 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.563541+0000", 2024-09-11T07:35:46.839 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8560999999999999e-05 2024-09-11T07:35:46.839 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.839 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.840 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:35:46.840 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.563650+0000", 2024-09-11T07:35:46.840 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010945199999999999 2024-09-11T07:35:46.840 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.840 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.840 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2", 2024-09-11T07:35:46.840 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.563806+0000", 2024-09-11T07:35:46.840 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000155752 2024-09-11T07:35:46.840 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.840 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.840 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:35:46.840 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.565913+0000", 2024-09-11T07:35:46.841 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0021068969999999999 2024-09-11T07:35:46.841 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.841 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.841 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:46.841 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.691714+0000", 2024-09-11T07:35:46.841 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.12580175599999999 2024-09-11T07:35:46.841 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.841 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.841 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:46.841 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.697319+0000", 2024-09-11T07:35:46.841 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0056043509999999996 2024-09-11T07:35:46.841 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:35:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.697411+0000", 2024-09-11T07:35:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.2373000000000005e-05 2024-09-11T07:35:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:35:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.697452+0000", 2024-09-11T07:35:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0908000000000002e-05 2024-09-11T07:35:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:36497 8.a 8:59ddddbf:::benchmark_data_smithi022_108406_object36496:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e191)", 2024-09-11T07:35:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:55.564602+0000", 2024-09-11T07:35:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 110.96403979599999, 2024-09-11T07:35:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13276951000000001, 2024-09-11T07:35:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:35:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:35:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:35:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:35:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:35:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 36497 2024-09-11T07:35:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:35:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.844 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:35:46.844 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.564602+0000", 2024-09-11T07:35:46.844 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.844 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.844 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.844 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:35:46.844 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.564602+0000", 2024-09-11T07:35:46.844 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.844 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.844 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.844 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:35:46.844 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.564600+0000", 2024-09-11T07:35:46.845 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:35:46.845 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.845 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.845 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:35:46.845 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.564624+0000", 2024-09-11T07:35:46.845 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4294000000000002e-05 2024-09-11T07:35:46.845 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.845 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.845 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:35:46.845 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.564635+0000", 2024-09-11T07:35:46.845 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1174e-05 2024-09-11T07:35:46.845 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.845 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.846 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:35:46.846 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.564642+0000", 2024-09-11T07:35:46.846 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1700000000000002e-06 2024-09-11T07:35:46.846 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.846 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.846 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:35:46.846 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.564686+0000", 2024-09-11T07:35:46.846 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4421e-05 2024-09-11T07:35:46.846 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.846 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.846 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:35:46.846 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.564810+0000", 2024-09-11T07:35:46.846 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012408699999999999 2024-09-11T07:35:46.847 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.847 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.847 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,5", 2024-09-11T07:35:46.847 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.565099+0000", 2024-09-11T07:35:46.847 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00028865599999999998 2024-09-11T07:35:46.847 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.847 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.847 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:35:46.847 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.567071+0000", 2024-09-11T07:35:46.847 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0019723100000000001 2024-09-11T07:35:46.847 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.847 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.847 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:46.848 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.692272+0000", 2024-09-11T07:35:46.848 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.12520094700000001 2024-09-11T07:35:46.848 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.848 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.848 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:46.848 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.697244+0000", 2024-09-11T07:35:46.848 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0049720490000000001 2024-09-11T07:35:46.848 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.848 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.848 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:35:46.848 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.697333+0000", 2024-09-11T07:35:46.848 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.9124999999999998e-05 2024-09-11T07:35:46.848 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.849 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.849 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:35:46.849 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.697371+0000", 2024-09-11T07:35:46.849 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7997000000000002e-05 2024-09-11T07:35:46.849 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.849 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:46.849 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.849 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.849 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.849 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:36498 8.7 8:e8a774b4:::benchmark_data_smithi022_108406_object36497:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e191)", 2024-09-11T07:35:46.849 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:55.564944+0000", 2024-09-11T07:35:46.849 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 110.963697114, 2024-09-11T07:35:46.849 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.127705296, 2024-09-11T07:35:46.850 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:35:46.850 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:35:46.850 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:35:46.850 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:35:46.850 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:35:46.850 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 36498 2024-09-11T07:35:46.850 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.850 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:35:46.850 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.850 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:35:46.850 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.564944+0000", 2024-09-11T07:35:46.850 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.850 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:35:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.564944+0000", 2024-09-11T07:35:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:35:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.564943+0000", 2024-09-11T07:35:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:35:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:35:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.564975+0000", 2024-09-11T07:35:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2348999999999998e-05 2024-09-11T07:35:46.852 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.852 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.852 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:35:46.852 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.564994+0000", 2024-09-11T07:35:46.852 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8700000000000001e-05 2024-09-11T07:35:46.852 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.852 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.852 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:35:46.852 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.565000+0000", 2024-09-11T07:35:46.852 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1510000000000001e-06 2024-09-11T07:35:46.852 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.852 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.852 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:35:46.853 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.565642+0000", 2024-09-11T07:35:46.853 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00064224799999999999 2024-09-11T07:35:46.853 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.853 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.853 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:35:46.853 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.565773+0000", 2024-09-11T07:35:46.853 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000130452 2024-09-11T07:35:46.853 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.853 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.853 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,6", 2024-09-11T07:35:46.853 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.565908+0000", 2024-09-11T07:35:46.853 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013499399999999999 2024-09-11T07:35:46.853 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.853 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.691840+0000", 2024-09-11T07:35:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.125931974 2024-09-11T07:35:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:35:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.691846+0000", 2024-09-11T07:35:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.669e-06 2024-09-11T07:35:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.692510+0000", 2024-09-11T07:35:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00066316699999999999 2024-09-11T07:35:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:35:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.692605+0000", 2024-09-11T07:35:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.5859999999999994e-05 2024-09-11T07:35:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:35:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.692650+0000", 2024-09-11T07:35:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4187999999999998e-05 2024-09-11T07:35:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.25072.0:36496 8.b e191/177 8:d3b7d23a:::benchmark_data_smithi022_108406_object36495:head v 191'2305, mlcod=191'2303)", 2024-09-11T07:35:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:55.566309+0000", 2024-09-11T07:35:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 110.962332137, 2024-09-11T07:35:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.12722481799999999, 2024-09-11T07:35:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:35:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:35:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:35:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:35:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.566309+0000", 2024-09-11T07:35:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:35:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.566309+0000", 2024-09-11T07:35:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:35:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.566308+0000", 2024-09-11T07:35:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:35:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:35:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.566336+0000", 2024-09-11T07:35:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8328999999999999e-05 2024-09-11T07:35:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:35:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.566348+0000", 2024-09-11T07:35:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1333e-05 2024-09-11T07:35:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:35:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.566354+0000", 2024-09-11T07:35:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9279999999999998e-06 2024-09-11T07:35:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:35:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.566521+0000", 2024-09-11T07:35:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000167727 2024-09-11T07:35:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:35:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.566539+0000", 2024-09-11T07:35:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8097999999999999e-05 2024-09-11T07:35:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:35:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.693498+0000", 2024-09-11T07:35:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.12695861 2024-09-11T07:35:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:35:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.693534+0000", 2024-09-11T07:35:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6232000000000001e-05 2024-09-11T07:35:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop_reply(client.25072.0:36497 8.a e191/177 ondisk, result = 0)", 2024-09-11T07:35:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:55.568263+0000", 2024-09-11T07:35:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 110.960378668, 2024-09-11T07:35:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.12911460699999999, 2024-09-11T07:35:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:35:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:35:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:35:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:35:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.568263+0000", 2024-09-11T07:35:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.861 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.861 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:35:46.861 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.568263+0000", 2024-09-11T07:35:46.861 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.861 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.861 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.861 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:35:46.861 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.568262+0000", 2024-09-11T07:35:46.861 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:35:46.861 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.861 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.861 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:35:46.861 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.568267+0000", 2024-09-11T07:35:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.5639999999999996e-06 2024-09-11T07:35:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:35:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.692225+0000", 2024-09-11T07:35:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.123958031 2024-09-11T07:35:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:35:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.692235+0000", 2024-09-11T07:35:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.7739999999999999e-06 2024-09-11T07:35:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:35:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.695411+0000", 2024-09-11T07:35:46.863 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.003176468 2024-09-11T07:35:46.863 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.863 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.863 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:35:46.863 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.697235+0000", 2024-09-11T07:35:46.863 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001823638 2024-09-11T07:35:46.863 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.863 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.863 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:35:46.863 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.697377+0000", 2024-09-11T07:35:46.863 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014236000000000001 2024-09-11T07:35:46.863 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.863 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:46.864 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.864 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.864 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.864 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.25072.0:36501 8.b e191/177 8:d3d50775:::benchmark_data_smithi022_108406_object36500:head v 191'2306, mlcod=191'2303)", 2024-09-11T07:35:46.864 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:55.568324+0000", 2024-09-11T07:35:46.864 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 110.96031752099999, 2024-09-11T07:35:46.864 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.12858783700000001, 2024-09-11T07:35:46.864 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:35:46.864 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:35:46.864 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:35:46.864 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.864 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:35:46.864 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.568324+0000", 2024-09-11T07:35:46.864 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.865 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.865 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.865 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:35:46.865 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.568324+0000", 2024-09-11T07:35:46.865 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.865 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.865 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.865 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:35:46.865 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.568323+0000", 2024-09-11T07:35:46.865 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:35:46.865 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.865 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.866 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:35:46.866 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.568347+0000", 2024-09-11T07:35:46.866 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4671000000000001e-05 2024-09-11T07:35:46.866 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.866 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.866 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:35:46.866 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.692288+0000", 2024-09-11T07:35:46.866 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.123940074 2024-09-11T07:35:46.866 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.866 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.866 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:35:46.866 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.692297+0000", 2024-09-11T07:35:46.866 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.4890000000000001e-06 2024-09-11T07:35:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:35:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.693558+0000", 2024-09-11T07:35:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001260705 2024-09-11T07:35:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:35:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.693576+0000", 2024-09-11T07:35:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.7954000000000001e-05 2024-09-11T07:35:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:35:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.696889+0000", 2024-09-11T07:35:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.003312939 2024-09-11T07:35:46.868 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.868 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.868 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:35:46.868 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:55.696912+0000", 2024-09-11T07:35:46.868 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3204999999999999e-05 2024-09-11T07:35:46.868 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.868 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:46.868 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.868 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.868 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.868 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:65081 8.1 8:8ee42688:::benchmark_data_smithi022_108406_object65080:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-11T07:35:46.868 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:34:13.886298+0000", 2024-09-11T07:35:46.868 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 92.642343726999997, 2024-09-11T07:35:46.869 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13628242400000001, 2024-09-11T07:35:46.869 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:35:46.869 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:35:46.869 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:35:46.869 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:35:46.869 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:35:46.869 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 65081 2024-09-11T07:35:46.869 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.869 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:35:46.869 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.869 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:35:46.869 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:34:13.886298+0000", 2024-09-11T07:35:46.869 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:35:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:34:13.886298+0000", 2024-09-11T07:35:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:35:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:34:13.886295+0000", 2024-09-11T07:35:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:35:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:35:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:34:13.886324+0000", 2024-09-11T07:35:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8421000000000001e-05 2024-09-11T07:35:46.871 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.871 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.871 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:35:46.871 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:34:13.886337+0000", 2024-09-11T07:35:46.871 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2995e-05 2024-09-11T07:35:46.871 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.871 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.871 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:35:46.871 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:34:13.886343+0000", 2024-09-11T07:35:46.871 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.6259999999999997e-06 2024-09-11T07:35:46.871 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.871 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.871 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:35:46.871 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:34:13.886485+0000", 2024-09-11T07:35:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014125800000000001 2024-09-11T07:35:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:35:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:34:13.886606+0000", 2024-09-11T07:35:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000121159 2024-09-11T07:35:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2", 2024-09-11T07:35:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:34:13.886745+0000", 2024-09-11T07:35:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013929599999999999 2024-09-11T07:35:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:35:46.873 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:34:13.889246+0000", 2024-09-11T07:35:46.873 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0025009390000000002 2024-09-11T07:35:46.873 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.873 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.873 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:46.873 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:34:13.892696+0000", 2024-09-11T07:35:46.873 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0034497680000000002 2024-09-11T07:35:46.873 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.873 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.873 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:46.873 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:34:14.022418+0000", 2024-09-11T07:35:46.873 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.129722107 2024-09-11T07:35:46.873 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:35:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:34:14.022513+0000", 2024-09-11T07:35:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.5413000000000003e-05 2024-09-11T07:35:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:35:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:34:14.022580+0000", 2024-09-11T07:35:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.6843999999999999e-05 2024-09-11T07:35:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.875 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:120161 8.1 8:86cdb084:::benchmark_data_smithi022_108406_object120160:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e213)", 2024-09-11T07:35:46.875 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:35:01.857835+0000", 2024-09-11T07:35:46.875 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 44.670806335000002, 2024-09-11T07:35:46.875 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.079876928999999999, 2024-09-11T07:35:46.875 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:35:46.875 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:35:46.875 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:35:46.875 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:35:46.875 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:35:46.875 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 120161 2024-09-11T07:35:46.875 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.876 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:35:46.876 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.876 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:35:46.876 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:01.857835+0000", 2024-09-11T07:35:46.876 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.876 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.876 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.876 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:35:46.876 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:01.857835+0000", 2024-09-11T07:35:46.876 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:46.876 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.876 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.876 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:35:46.876 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:01.857833+0000", 2024-09-11T07:35:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:35:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:35:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:01.857857+0000", 2024-09-11T07:35:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3213999999999998e-05 2024-09-11T07:35:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:35:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:01.857872+0000", 2024-09-11T07:35:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4849e-05 2024-09-11T07:35:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.878 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:35:46.878 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:01.857877+0000", 2024-09-11T07:35:46.878 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.4829999999999999e-06 2024-09-11T07:35:46.878 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.878 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.878 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:35:46.878 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:01.859572+0000", 2024-09-11T07:35:46.878 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0016953059999999999 2024-09-11T07:35:46.878 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.878 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.878 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:35:46.878 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:01.859706+0000", 2024-09-11T07:35:46.878 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013352899999999999 2024-09-11T07:35:46.879 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.879 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.879 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2", 2024-09-11T07:35:46.879 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:01.860169+0000", 2024-09-11T07:35:46.879 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00046300699999999998 2024-09-11T07:35:46.879 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.879 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.879 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:46.879 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:01.864947+0000", 2024-09-11T07:35:46.879 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.004778203 2024-09-11T07:35:46.879 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.879 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.879 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:35:46.880 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:01.865017+0000", 2024-09-11T07:35:46.880 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0375000000000003e-05 2024-09-11T07:35:46.880 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.880 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.880 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:35:46.880 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:01.937588+0000", 2024-09-11T07:35:46.880 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.072570203 2024-09-11T07:35:46.880 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.880 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.881 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:35:46.881 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:01.937671+0000", 2024-09-11T07:35:46.881 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.2917999999999995e-05 2024-09-11T07:35:46.881 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:46.881 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:46.881 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:35:46.881 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:01.937712+0000", 2024-09-11T07:35:46.881 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1563999999999997e-05 2024-09-11T07:35:46.881 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.881 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:46.881 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.881 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:46.881 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:46.882 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:35:46.882 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:46.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:46.910 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:46.911 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:47.011 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:47.053 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:47.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:47.054 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:47.155 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:47.183 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:47.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:47.184 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:47.285 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:47.314 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:47.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:47.315 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:47.416 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:47.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:47 smithi022 ceph-mon[29462]: pgmap v409: 148 pgs: 148 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 976 B/s rd, 0 B/s wr, 962 op/s 2024-09-11T07:35:47.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:47 smithi022 ceph-mon[29462]: 2.10 scrub starts 2024-09-11T07:35:47.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:47 smithi022 ceph-mon[29462]: 2.10 scrub ok 2024-09-11T07:35:47.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:47 smithi022 ceph-mon[29462]: 2.f scrub starts 2024-09-11T07:35:47.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:47 smithi022 ceph-mon[29462]: 2.f scrub ok 2024-09-11T07:35:47.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:47 smithi022 ceph-mon[37330]: pgmap v409: 148 pgs: 148 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 976 B/s rd, 0 B/s wr, 962 op/s 2024-09-11T07:35:47.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:47 smithi022 ceph-mon[37330]: 2.10 scrub starts 2024-09-11T07:35:47.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:47 smithi022 ceph-mon[37330]: 2.10 scrub ok 2024-09-11T07:35:47.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:47 smithi022 ceph-mon[37330]: 2.f scrub starts 2024-09-11T07:35:47.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:47 smithi022 ceph-mon[37330]: 2.f scrub ok 2024-09-11T07:35:47.468 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:47.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:47.469 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:47.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:47 smithi104 ceph-mon[33507]: pgmap v409: 148 pgs: 148 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 976 B/s rd, 0 B/s wr, 962 op/s 2024-09-11T07:35:47.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:47 smithi104 ceph-mon[33507]: 2.10 scrub starts 2024-09-11T07:35:47.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:47 smithi104 ceph-mon[33507]: 2.10 scrub ok 2024-09-11T07:35:47.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:47 smithi104 ceph-mon[33507]: 2.f scrub starts 2024-09-11T07:35:47.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:47 smithi104 ceph-mon[33507]: 2.f scrub ok 2024-09-11T07:35:47.570 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:47.608 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:47.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:47.609 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:47.710 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:47.752 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:47.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:47.753 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:47.797 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:35:47.854 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:47.864 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-11T07:35:47.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:47.912 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:47.912 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:48.013 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:48.036 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:48.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:48.037 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:48.138 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:48.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:48.165 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:48.166 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:48.266 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:48.307 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:48.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:48.308 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:48.409 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:48.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:48 smithi022 ceph-mon[29462]: 5.14 scrub starts 2024-09-11T07:35:48.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:48 smithi022 ceph-mon[29462]: 5.14 scrub ok 2024-09-11T07:35:48.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:48 smithi022 ceph-mon[37330]: 5.14 scrub starts 2024-09-11T07:35:48.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:48 smithi022 ceph-mon[37330]: 5.14 scrub ok 2024-09-11T07:35:48.436 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:48.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:48.437 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:48.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:48 smithi104 ceph-mon[33507]: 5.14 scrub starts 2024-09-11T07:35:48.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:48 smithi104 ceph-mon[33507]: 5.14 scrub ok 2024-09-11T07:35:48.538 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:48.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:48.578 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:48.578 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:48.679 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:48.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:48.718 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:48.718 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:48.819 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:48.851 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:48.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:48.852 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:48.953 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:48.975 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:48.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:48.976 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:49.077 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:49.102 INFO:teuthology.orchestra.run.smithi022.stderr:set 4.17 pg_upmap_items mapping to [2->6,4->1,3->5] 2024-09-11T07:35:49.102 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:35:49.107 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:49.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:49.108 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:49.208 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:49.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:49.236 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:49.236 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:49.337 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:49.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:49.367 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:49.367 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:49.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:49 smithi022 ceph-mon[29462]: pgmap v410: 148 pgs: 148 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 840 op/s 2024-09-11T07:35:49.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:49 smithi022 ceph-mon[29462]: 2.17 deep-scrub starts 2024-09-11T07:35:49.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:49 smithi022 ceph-mon[29462]: 2.17 deep-scrub ok 2024-09-11T07:35:49.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:49 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/854667535' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "4.17", "id": [2, 6, 4, 1, 3, 5]}]: dispatch 2024-09-11T07:35:49.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:49 smithi022 ceph-mon[29462]: from='client.25208 ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "4.17", "id": [2, 6, 4, 1, 3, 5]}]: dispatch 2024-09-11T07:35:49.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:49 smithi022 ceph-mon[37330]: pgmap v410: 148 pgs: 148 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 840 op/s 2024-09-11T07:35:49.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:49 smithi022 ceph-mon[37330]: 2.17 deep-scrub starts 2024-09-11T07:35:49.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:49 smithi022 ceph-mon[37330]: 2.17 deep-scrub ok 2024-09-11T07:35:49.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:49 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/854667535' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "4.17", "id": [2, 6, 4, 1, 3, 5]}]: dispatch 2024-09-11T07:35:49.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:49 smithi022 ceph-mon[37330]: from='client.25208 ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "4.17", "id": [2, 6, 4, 1, 3, 5]}]: dispatch 2024-09-11T07:35:49.468 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:49.472 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:49 smithi104 ceph-mon[33507]: pgmap v410: 148 pgs: 148 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 840 op/s 2024-09-11T07:35:49.472 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:49 smithi104 ceph-mon[33507]: 2.17 deep-scrub starts 2024-09-11T07:35:49.472 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:49 smithi104 ceph-mon[33507]: 2.17 deep-scrub ok 2024-09-11T07:35:49.472 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:49 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/854667535' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "4.17", "id": [2, 6, 4, 1, 3, 5]}]: dispatch 2024-09-11T07:35:49.472 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:49 smithi104 ceph-mon[33507]: from='client.25208 ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "4.17", "id": [2, 6, 4, 1, 3, 5]}]: dispatch 2024-09-11T07:35:49.497 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:49.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:49.498 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:49.599 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:49.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:49.639 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:49.639 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:49.740 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:49.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:49.770 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:49.770 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:49.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:35:49 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:35:49.871 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:49.910 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:49.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:49.911 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:50.012 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:50.042 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:50.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:50.043 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:50.144 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:50.184 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:50.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:50.185 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:50.286 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:50.313 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:50.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:50.314 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:50.372 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:50 smithi022 ceph-mon[29462]: from='client.25208 ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "4.17", "id": [2, 6, 4, 1, 3, 5]}]': finished 2024-09-11T07:35:50.372 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:50 smithi022 ceph-mon[29462]: osdmap e220: 8 total, 8 up, 8 in 2024-09-11T07:35:50.372 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:50 smithi022 ceph-mon[29462]: 6.10 scrub starts 2024-09-11T07:35:50.372 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:50 smithi022 ceph-mon[29462]: 6.10 scrub ok 2024-09-11T07:35:50.372 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:50 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:35:50.373 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:50 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:35:50.373 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:50 smithi022 ceph-mon[29462]: 6.1e scrub starts 2024-09-11T07:35:50.373 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:50 smithi022 ceph-mon[29462]: 6.1e scrub ok 2024-09-11T07:35:50.373 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:50 smithi022 ceph-mon[37330]: from='client.25208 ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "4.17", "id": [2, 6, 4, 1, 3, 5]}]': finished 2024-09-11T07:35:50.373 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:50 smithi022 ceph-mon[37330]: osdmap e220: 8 total, 8 up, 8 in 2024-09-11T07:35:50.373 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:50 smithi022 ceph-mon[37330]: 6.10 scrub starts 2024-09-11T07:35:50.373 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:50 smithi022 ceph-mon[37330]: 6.10 scrub ok 2024-09-11T07:35:50.373 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:50 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:35:50.373 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:50 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:35:50.373 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:50 smithi022 ceph-mon[37330]: 6.1e scrub starts 2024-09-11T07:35:50.373 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:50 smithi022 ceph-mon[37330]: 6.1e scrub ok 2024-09-11T07:35:50.414 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:50.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:50.459 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:50.459 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:50.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:50 smithi104 ceph-mon[33507]: from='client.25208 ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "4.17", "id": [2, 6, 4, 1, 3, 5]}]': finished 2024-09-11T07:35:50.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:50 smithi104 ceph-mon[33507]: osdmap e220: 8 total, 8 up, 8 in 2024-09-11T07:35:50.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:50 smithi104 ceph-mon[33507]: 6.10 scrub starts 2024-09-11T07:35:50.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:50 smithi104 ceph-mon[33507]: 6.10 scrub ok 2024-09-11T07:35:50.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:50 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:35:50.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:50 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:35:50.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:50 smithi104 ceph-mon[33507]: 6.1e scrub starts 2024-09-11T07:35:50.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:50 smithi104 ceph-mon[33507]: 6.1e scrub ok 2024-09-11T07:35:50.560 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:50.602 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:50.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:50.603 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:50.664 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:35:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:35:50] "GET /metrics HTTP/1.1" 200 224111 "" "Prometheus/2.43.0" 2024-09-11T07:35:50.704 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:50.731 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:50.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:50.732 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:50.833 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:50.867 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:50.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:50.868 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:50.968 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:51.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:51.010 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:51.011 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:51.081 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:35:51.112 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:51.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:51.160 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:51.160 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:51.261 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:51.287 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:51.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:51.288 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:51.389 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:51.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:51 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:51.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:51 smithi022 ceph-mon[37330]: osdmap e221: 8 total, 8 up, 8 in 2024-09-11T07:35:51.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:51 smithi022 ceph-mon[37330]: pgmap v413: 148 pgs: 148 active+clean; 5.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-11T07:35:51.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:51 smithi022 ceph-mon[37330]: 5.5 scrub starts 2024-09-11T07:35:51.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:51 smithi022 ceph-mon[37330]: 5.5 scrub ok 2024-09-11T07:35:51.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:51 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:35:51.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:51 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:35:51.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:51 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:51.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:51 smithi022 ceph-mon[29462]: osdmap e221: 8 total, 8 up, 8 in 2024-09-11T07:35:51.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:51 smithi022 ceph-mon[29462]: pgmap v413: 148 pgs: 148 active+clean; 5.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-11T07:35:51.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:51 smithi022 ceph-mon[29462]: 5.5 scrub starts 2024-09-11T07:35:51.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:51 smithi022 ceph-mon[29462]: 5.5 scrub ok 2024-09-11T07:35:51.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:51 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:35:51.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:51 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:35:51.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:51.429 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:51.429 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:51.529 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:51.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:51 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:35:51.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:51 smithi104 ceph-mon[33507]: osdmap e221: 8 total, 8 up, 8 in 2024-09-11T07:35:51.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:51 smithi104 ceph-mon[33507]: pgmap v413: 148 pgs: 148 active+clean; 5.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-11T07:35:51.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:51 smithi104 ceph-mon[33507]: 5.5 scrub starts 2024-09-11T07:35:51.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:51 smithi104 ceph-mon[33507]: 5.5 scrub ok 2024-09-11T07:35:51.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:51 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:35:51.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:51 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:35:51.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:51.585 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:51.586 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:51.686 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:51.719 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:51.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:51.720 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:51.821 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:51.847 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:51.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:51.848 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:51.949 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:51.992 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:51.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:51.994 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:52.094 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:52.131 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:52.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:52.132 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:52.232 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:52.302 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:52.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:52.303 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:52.404 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:52.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:52 smithi022 ceph-mon[29462]: 5.1a scrub starts 2024-09-11T07:35:52.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:52 smithi022 ceph-mon[29462]: 5.1a scrub ok 2024-09-11T07:35:52.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:52 smithi022 ceph-mon[29462]: osdmap e222: 8 total, 8 up, 8 in 2024-09-11T07:35:52.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:52 smithi022 ceph-mon[29462]: 5.7 deep-scrub starts 2024-09-11T07:35:52.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:52 smithi022 ceph-mon[29462]: 5.7 deep-scrub ok 2024-09-11T07:35:52.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:52 smithi022 ceph-mon[37330]: 5.1a scrub starts 2024-09-11T07:35:52.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:52 smithi022 ceph-mon[37330]: 5.1a scrub ok 2024-09-11T07:35:52.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:52 smithi022 ceph-mon[37330]: osdmap e222: 8 total, 8 up, 8 in 2024-09-11T07:35:52.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:52 smithi022 ceph-mon[37330]: 5.7 deep-scrub starts 2024-09-11T07:35:52.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:52 smithi022 ceph-mon[37330]: 5.7 deep-scrub ok 2024-09-11T07:35:52.437 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:52.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:52.438 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:52.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:52 smithi104 ceph-mon[33507]: 5.1a scrub starts 2024-09-11T07:35:52.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:52 smithi104 ceph-mon[33507]: 5.1a scrub ok 2024-09-11T07:35:52.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:52 smithi104 ceph-mon[33507]: osdmap e222: 8 total, 8 up, 8 in 2024-09-11T07:35:52.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:52 smithi104 ceph-mon[33507]: 5.7 deep-scrub starts 2024-09-11T07:35:52.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:52 smithi104 ceph-mon[33507]: 5.7 deep-scrub ok 2024-09-11T07:35:52.538 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:52.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:52.570 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:52.570 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:52.671 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:52.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:52.708 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:52.708 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:52.810 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:52.850 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:52.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:52.851 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:52.952 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:52.984 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:52.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:52.985 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:53.086 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:53.110 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:53.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:53.111 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:53.212 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:53.260 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:53.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:53.261 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:53.324 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:35:53.362 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:53.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:53.398 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:53.398 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:53.405 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:53 smithi022 ceph-mon[37330]: 5.13 scrub starts 2024-09-11T07:35:53.405 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:53 smithi022 ceph-mon[37330]: 5.13 scrub ok 2024-09-11T07:35:53.405 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:53 smithi022 ceph-mon[37330]: pgmap v415: 148 pgs: 148 active+clean; 5.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 974 op/s 2024-09-11T07:35:53.405 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:53 smithi022 ceph-mon[37330]: 6.16 scrub starts 2024-09-11T07:35:53.405 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:53 smithi022 ceph-mon[37330]: 6.16 scrub ok 2024-09-11T07:35:53.405 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:35:53.414 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:53 smithi022 ceph-mon[29462]: 5.13 scrub starts 2024-09-11T07:35:53.414 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:53 smithi022 ceph-mon[29462]: 5.13 scrub ok 2024-09-11T07:35:53.414 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:53 smithi022 ceph-mon[29462]: pgmap v415: 148 pgs: 148 active+clean; 5.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 974 op/s 2024-09-11T07:35:53.414 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:53 smithi022 ceph-mon[29462]: 6.16 scrub starts 2024-09-11T07:35:53.415 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:53 smithi022 ceph-mon[29462]: 6.16 scrub ok 2024-09-11T07:35:53.415 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:35:53.499 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:53.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:53 smithi104 ceph-mon[33507]: 5.13 scrub starts 2024-09-11T07:35:53.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:53 smithi104 ceph-mon[33507]: 5.13 scrub ok 2024-09-11T07:35:53.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:53 smithi104 ceph-mon[33507]: pgmap v415: 148 pgs: 148 active+clean; 5.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 974 op/s 2024-09-11T07:35:53.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:53 smithi104 ceph-mon[33507]: 6.16 scrub starts 2024-09-11T07:35:53.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:53 smithi104 ceph-mon[33507]: 6.16 scrub ok 2024-09-11T07:35:53.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:35:53.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:53.540 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:53.541 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:53.641 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:53.682 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:53.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:53.683 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:53.783 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:53.830 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:53.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:53.831 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:53.932 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:53.973 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:53.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:53.975 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:54.076 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:54.102 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:54.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:54.103 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:54.141 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is set 2024-09-11T07:35:54.205 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:54.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:54.239 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:54.240 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:54.340 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:54.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:54.369 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:54.370 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:54.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:54 smithi022 ceph-mon[29462]: 4.0 deep-scrub starts 2024-09-11T07:35:54.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:54 smithi022 ceph-mon[29462]: 4.0 deep-scrub ok 2024-09-11T07:35:54.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:54 smithi022 ceph-mon[29462]: 3.1 scrub starts 2024-09-11T07:35:54.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:54 smithi022 ceph-mon[29462]: 3.1 scrub ok 2024-09-11T07:35:54.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:54 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3324058028' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:35:54.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:54 smithi022 ceph-mon[29462]: from='client.25214 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:35:54.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:54 smithi022 ceph-mon[37330]: 4.0 deep-scrub starts 2024-09-11T07:35:54.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:54 smithi022 ceph-mon[37330]: 4.0 deep-scrub ok 2024-09-11T07:35:54.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:54 smithi022 ceph-mon[37330]: 3.1 scrub starts 2024-09-11T07:35:54.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:54 smithi022 ceph-mon[37330]: 3.1 scrub ok 2024-09-11T07:35:54.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:54 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3324058028' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:35:54.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:54 smithi022 ceph-mon[37330]: from='client.25214 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:35:54.430 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:35:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:35:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:35:54.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:54 smithi104 ceph-mon[33507]: 4.0 deep-scrub starts 2024-09-11T07:35:54.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:54 smithi104 ceph-mon[33507]: 4.0 deep-scrub ok 2024-09-11T07:35:54.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:54 smithi104 ceph-mon[33507]: 3.1 scrub starts 2024-09-11T07:35:54.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:54 smithi104 ceph-mon[33507]: 3.1 scrub ok 2024-09-11T07:35:54.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:54 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3324058028' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:35:54.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:54 smithi104 ceph-mon[33507]: from='client.25214 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:35:54.471 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:54.497 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:54.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:54.499 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:54.599 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:54.635 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:54.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:54.636 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:54.737 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:54.777 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:54.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:54.778 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:54.879 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:54.916 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:54.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:54.917 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:55.018 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:55.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:55.050 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:55.050 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:55.151 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:55.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:55.203 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:55.204 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:55.233 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:35:55.233 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:35:55.234 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-11T07:35:55.234 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 0.000000 2024-09-11T07:35:55.234 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 6 0 2024-09-11T07:35:55.305 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:55.335 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:55.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:55.336 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:55.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:55 smithi022 ceph-mon[29462]: 6.1d deep-scrub starts 2024-09-11T07:35:55.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:55 smithi022 ceph-mon[29462]: 6.1d deep-scrub ok 2024-09-11T07:35:55.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:55 smithi022 ceph-mon[29462]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:35:55.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:55 smithi022 ceph-mon[29462]: pgmap v417: 148 pgs: 148 active+clean; 5.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1015 B/s rd, 0 B/s wr, 1.16k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:35:55.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:55 smithi022 ceph-mon[29462]: from='client.25214 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:35:55.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:55 smithi022 ceph-mon[29462]: osdmap e223: 8 total, 8 up, 8 in 2024-09-11T07:35:55.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:55 smithi022 ceph-mon[37330]: 6.1d deep-scrub starts 2024-09-11T07:35:55.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:55 smithi022 ceph-mon[37330]: 6.1d deep-scrub ok 2024-09-11T07:35:55.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:55 smithi022 ceph-mon[37330]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:35:55.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:55 smithi022 ceph-mon[37330]: pgmap v417: 148 pgs: 148 active+clean; 5.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1015 B/s rd, 0 B/s wr, 1.16k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:35:55.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:55 smithi022 ceph-mon[37330]: from='client.25214 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:35:55.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:55 smithi022 ceph-mon[37330]: osdmap e223: 8 total, 8 up, 8 in 2024-09-11T07:35:55.437 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:55.464 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:35:55.464 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:35:55.464 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:55.464 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.25072.0:177473 8.a e222/177 8:58558c42:::benchmark_data_smithi022_108406_object48942:head v 222'11201, mlcod=222'11197)", 2024-09-11T07:35:55.464 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:35:54.066158+0000", 2024-09-11T07:35:55.464 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.005965143, 2024-09-11T07:35:55.464 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0059925569999999999, 2024-09-11T07:35:55.464 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:35:55.464 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:35:55.464 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:35:55.465 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:55.465 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:35:55.465 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:54.066158+0000", 2024-09-11T07:35:55.465 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:55.465 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:55.465 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:55.465 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:35:55.465 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:54.066158+0000", 2024-09-11T07:35:55.465 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:55.465 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:55.465 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:55.465 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:35:55.465 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:54.066157+0000", 2024-09-11T07:35:55.466 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:35:55.466 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:55.466 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:55.466 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:35:55.466 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:54.066165+0000", 2024-09-11T07:35:55.466 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.2709999999999992e-06 2024-09-11T07:35:55.466 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:55.466 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:55.466 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:35:55.466 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:54.066179+0000", 2024-09-11T07:35:55.466 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3885e-05 2024-09-11T07:35:55.466 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:55.466 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:55.466 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:35:55.466 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:54.066186+0000", 2024-09-11T07:35:55.467 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3210000000000001e-06 2024-09-11T07:35:55.467 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:55.467 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:55.467 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:35:55.467 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:54.066251+0000", 2024-09-11T07:35:55.467 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5432999999999995e-05 2024-09-11T07:35:55.467 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:55.467 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:55.467 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:35:55.467 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:54.066276+0000", 2024-09-11T07:35:55.467 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5055e-05 2024-09-11T07:35:55.467 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:55.467 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:55.468 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:55.468 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:55.468 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:55.468 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.25072.0:177488 8.e e222/177 8:75f14b80:::benchmark_data_smithi022_108406_object48957:head v 222'11067, mlcod=222'11065)", 2024-09-11T07:35:55.468 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:35:54.071909+0000", 2024-09-11T07:35:55.468 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.000214952, 2024-09-11T07:35:55.468 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00031566199999999997, 2024-09-11T07:35:55.468 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:35:55.468 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:35:55.468 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:35:55.468 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:55.468 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:35:55.468 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:54.071909+0000", 2024-09-11T07:35:55.468 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:55.469 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:55.469 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:55.469 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:35:55.469 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:54.071909+0000", 2024-09-11T07:35:55.469 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:55.469 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:55.469 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:55.469 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:35:55.469 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:54.071907+0000", 2024-09-11T07:35:55.469 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:35:55.469 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:55.469 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:55.469 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:35:55.470 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:54.071916+0000", 2024-09-11T07:35:55.470 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.1220000000000001e-06 2024-09-11T07:35:55.470 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:55.470 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:55.470 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:35:55.470 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:54.071931+0000", 2024-09-11T07:35:55.470 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5271e-05 2024-09-11T07:35:55.470 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:55.470 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:55.470 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:35:55.470 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:54.071938+0000", 2024-09-11T07:35:55.470 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.6930000000000001e-06 2024-09-11T07:35:55.470 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:55.470 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:55.470 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:35:55.471 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:54.071988+0000", 2024-09-11T07:35:55.471 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.0374000000000003e-05 2024-09-11T07:35:55.471 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:55.471 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:55.471 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:35:55.471 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:54.072007+0000", 2024-09-11T07:35:55.471 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8958e-05 2024-09-11T07:35:55.471 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:55.471 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:55.471 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:55.471 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:55.471 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:55.471 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.25072.0:177484 8.a e222/177)", 2024-09-11T07:35:55.471 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:35:54.071954+0000", 2024-09-11T07:35:55.472 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.00016946299999999999, 2024-09-11T07:35:55.472 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00033074800000000001, 2024-09-11T07:35:55.472 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:35:55.472 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "queued for pg", 2024-09-11T07:35:55.472 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:35:55.472 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:55.472 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:35:55.472 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:54.071954+0000", 2024-09-11T07:35:55.472 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:55.472 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:55.472 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:55.472 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:35:55.472 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:54.071954+0000", 2024-09-11T07:35:55.472 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:35:55.473 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:55.473 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:55.473 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:35:55.473 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:54.071953+0000", 2024-09-11T07:35:55.473 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:35:55.473 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:55.473 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:55.473 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:35:55.473 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:54.071959+0000", 2024-09-11T07:35:55.473 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1259999999999997e-06 2024-09-11T07:35:55.473 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:55.473 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:55.473 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:35:55.473 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:54.071967+0000", 2024-09-11T07:35:55.474 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.9640000000000003e-06 2024-09-11T07:35:55.474 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:35:55.474 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:35:55.474 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:35:55.474 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:35:54.071970+0000", 2024-09-11T07:35:55.474 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7079999999999999e-06 2024-09-11T07:35:55.474 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:55.474 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:35:55.474 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:55.474 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:35:55.474 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:35:55.474 INFO:teuthology.orchestra.run.smithi022.stdout: "num_ops": 3 2024-09-11T07:35:55.474 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:35:55.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:55.488 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:55.488 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:55.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:55 smithi104 ceph-mon[33507]: 6.1d deep-scrub starts 2024-09-11T07:35:55.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:55 smithi104 ceph-mon[33507]: 6.1d deep-scrub ok 2024-09-11T07:35:55.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:55 smithi104 ceph-mon[33507]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:35:55.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:55 smithi104 ceph-mon[33507]: pgmap v417: 148 pgs: 148 active+clean; 5.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1015 B/s rd, 0 B/s wr, 1.16k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:35:55.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:55 smithi104 ceph-mon[33507]: from='client.25214 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:35:55.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:55 smithi104 ceph-mon[33507]: osdmap e223: 8 total, 8 up, 8 in 2024-09-11T07:35:55.588 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:55.632 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:55.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:55.634 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:55.735 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:55.777 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:55.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:55.778 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:55.879 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:55.908 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:55.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:55.909 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:56.009 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:56.041 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:56.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:56.042 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:56.142 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:56.188 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:56.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:56.190 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:56.291 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:56.336 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:56.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:56.337 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:56.438 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:56.464 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:56.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:56.465 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:56.566 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:56.594 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:56.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:56.595 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:56.665 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:56 smithi022 ceph-mon[37330]: pgmap v418: 148 pgs: 148 active+clean; 4.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 847 B/s rd, 0 B/s wr, 1.01k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:35:56.665 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:56 smithi022 ceph-mon[29462]: pgmap v418: 148 pgs: 148 active+clean; 4.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 847 B/s rd, 0 B/s wr, 1.01k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:35:56.696 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:56.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:56.742 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:56.742 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:56.843 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:56.874 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:56.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:56.876 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:56.977 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:57.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:57.020 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:57.020 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:57.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:56 smithi104 ceph-mon[33507]: pgmap v418: 148 pgs: 148 active+clean; 4.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 847 B/s rd, 0 B/s wr, 1.01k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:35:57.097 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-11T07:35:57.121 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:57.183 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:57.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:57.185 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:57.236 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:35:57.237 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:35:57.237 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:35:57.237 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:35:57.237 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:35:57.237 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:35:57.237 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:35:57.237 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:35:57.237 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:35:57.237 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:35:57.238 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:35:57.238 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:35:57.238 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:35:57.238 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:35:57.238 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:35:57.238 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:35:57.285 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:57.332 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:57.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:57.334 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:57.434 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:57.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:57.462 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:57.462 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:57.563 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:57.609 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:57.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:57.610 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:57.711 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:57.753 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:57.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:57.754 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:57.855 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:57.888 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:57.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:57.892 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:57.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:57 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2752051998' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:35:57.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:57 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2752051998' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:35:57.993 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:35:58.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:57 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2752051998' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:35:58.033 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:35:58.071 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:58.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:58.072 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:35:58.103 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T07:35:58.172 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:58.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:58.217 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:58.217 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:58.318 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:58.356 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:35:58.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:58.382 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:58.382 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:58.483 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:58.510 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:58.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:58.511 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:58.612 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:35:58.654 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:58.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:58.655 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:35:58.756 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:35:58.793 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:58.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:58.795 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:35:58.895 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:35:58.920 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:35:58 smithi022 ceph-mon[29462]: pgmap v419: 148 pgs: 148 active+clean; 4.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 765 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:35:58.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:35:58 smithi022 ceph-mon[37330]: pgmap v419: 148 pgs: 148 active+clean; 4.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 765 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:35:58.928 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:58.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:58.929 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:35:59.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:35:58 smithi104 ceph-mon[33507]: pgmap v419: 148 pgs: 148 active+clean; 4.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 765 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:35:59.030 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:35:59.060 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:59.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:59.061 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:35:59.162 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:59.188 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:59.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:59.189 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:59.290 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:35:59.320 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:59.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:59.321 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:35:59.422 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:59.454 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:35:59.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:59.455 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:35:59.555 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:59.595 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:59.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:59.596 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:59.697 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:59.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:59.750 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:59.750 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:59.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:35:59 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:35:59.851 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:35:59.877 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:35:59.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:35:59.878 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:35:59.979 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:35:59.983 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:36:00.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:00.010 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:00.010 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:00.111 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:00.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:00.149 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:00.152 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:00.253 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:00.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:00.278 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:00.278 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:00.379 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:00.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:00.423 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:00.424 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:00.525 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:00.563 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:00.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:00.565 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:00.666 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:00.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:36:00 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:36:00] "GET /metrics HTTP/1.1" 200 224099 "" "Prometheus/2.43.0" 2024-09-11T07:36:00.698 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:00.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:00.699 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:00.799 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:00.838 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:00.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:00.840 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:00.941 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:00.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:00.974 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:00.974 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:01.075 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:01.104 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:01.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:01.107 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:01.208 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:01.244 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:01.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:01.245 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:01.346 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:01.376 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:01.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:01.377 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:01.478 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:01.505 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:01.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:01.506 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:01.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:01 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:01.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:01 smithi104 ceph-mon[33507]: pgmap v420: 148 pgs: 148 active+clean; 4.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.35k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:36:01.607 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:01.638 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:01.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:01.639 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:01.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:01 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:01.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:01 smithi022 ceph-mon[29462]: pgmap v420: 148 pgs: 148 active+clean; 4.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.35k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:36:01.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:01 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:01.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:01 smithi022 ceph-mon[37330]: pgmap v420: 148 pgs: 148 active+clean; 4.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.35k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:36:01.740 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:01.775 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:01.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:01.776 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:01.877 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:01.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:01.914 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:01.915 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:02.015 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:02.059 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:02.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:02.061 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:02.161 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:02.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:02.193 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:02.194 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:02.294 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:02.319 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:02.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:02.320 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:02.421 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:02.433 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:36:02.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:02.487 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:02.487 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:02.588 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:02.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:02.622 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:02.622 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:02.723 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:02.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:02.769 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:02.769 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:02.870 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:02.896 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:02.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:02.897 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:02.998 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:03.029 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:03.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:03.030 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:03.131 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:03.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:03.174 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:03.175 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:03.275 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:03.316 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:03.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:03.317 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:03.383 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:03 smithi022 ceph-mon[37330]: pgmap v421: 148 pgs: 148 active+clean; 4.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.22k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:36:03.383 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:03 smithi022 ceph-mon[29462]: pgmap v421: 148 pgs: 148 active+clean; 4.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.22k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:36:03.418 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:03.458 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:03.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:03.459 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:03.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:03 smithi104 ceph-mon[33507]: pgmap v421: 148 pgs: 148 active+clean; 4.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.22k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:36:03.560 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:03.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:03.599 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:03.599 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:03.700 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:03.731 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:03.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:03.733 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:03.833 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:03.865 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:03.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:03.866 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:03.968 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:04.002 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:04.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:04.003 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:04.104 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:04.135 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:04.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:04.136 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:04.203 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:36:04.221 INFO:teuthology.orchestra.run.smithi022.stderr:set osd.6 primary-affinity to 0 (802) 2024-09-11T07:36:04.237 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:04.242 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:04 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/156553277' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]: dispatch 2024-09-11T07:36:04.242 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:04 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/156553277' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]: dispatch 2024-09-11T07:36:04.288 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:04.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:04.290 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:04.390 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:04.412 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:04.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:04.413 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:04.514 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:04.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:04 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/156553277' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]: dispatch 2024-09-11T07:36:04.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:36:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:36:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:36:04.535 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:04.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:04.536 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:04.636 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:04.660 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:04.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:04.661 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:04.762 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:04.784 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:04.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:04.786 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:04.886 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:04.909 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:04.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:04.910 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:05.011 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:05.035 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:05.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:05.036 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:05.137 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:05.159 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:05.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:05.160 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:05.261 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:05.288 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:05.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:05.289 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:05.389 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:05.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:05.415 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:05.415 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:05.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:05 smithi022 ceph-mon[29462]: pgmap v422: 148 pgs: 148 active+clean; 4.4 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.22k op/s 2024-09-11T07:36:05.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:05 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/156553277' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]': finished 2024-09-11T07:36:05.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:05 smithi022 ceph-mon[29462]: osdmap e224: 8 total, 8 up, 8 in 2024-09-11T07:36:05.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:05 smithi022 ceph-mon[37330]: pgmap v422: 148 pgs: 148 active+clean; 4.4 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.22k op/s 2024-09-11T07:36:05.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:05 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/156553277' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]': finished 2024-09-11T07:36:05.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:05 smithi022 ceph-mon[37330]: osdmap e224: 8 total, 8 up, 8 in 2024-09-11T07:36:05.515 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:05.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:05 smithi104 ceph-mon[33507]: pgmap v422: 148 pgs: 148 active+clean; 4.4 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.22k op/s 2024-09-11T07:36:05.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:05 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/156553277' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.0}]': finished 2024-09-11T07:36:05.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:05 smithi104 ceph-mon[33507]: osdmap e224: 8 total, 8 up, 8 in 2024-09-11T07:36:05.550 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:05.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:05.551 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:05.652 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:05.692 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:36:05.692 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:36:05.693 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:05.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:05.695 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:05.795 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:05.827 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:05.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:05.828 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:05.928 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:05.954 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:05.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:05.956 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:06.056 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:06.100 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:06.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:06.104 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:06.205 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:06.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:06.244 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:06.244 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:06.345 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:06.382 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:06.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:06.383 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:06.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:06 smithi022 ceph-mon[29462]: osdmap e225: 8 total, 8 up, 8 in 2024-09-11T07:36:06.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:06 smithi022 ceph-mon[37330]: osdmap e225: 8 total, 8 up, 8 in 2024-09-11T07:36:06.484 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:06.490 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-11T07:36:06.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:06.527 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:06.527 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:06.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:06 smithi104 ceph-mon[33507]: osdmap e225: 8 total, 8 up, 8 in 2024-09-11T07:36:06.628 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:06.681 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:06.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:06.682 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:06.782 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:06.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:06.835 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:06.835 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:06.936 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:06.958 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:06.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:06.959 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:07.060 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:07.089 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:07.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:07.090 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:07.191 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:07.215 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:07.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:07.216 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:07.317 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:07.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:07.358 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:07.359 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:07.459 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:07.490 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:07.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:07.491 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:07.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:07 smithi104 ceph-mon[33507]: pgmap v425: 148 pgs: 3 peering, 145 active+clean; 4.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.50k op/s 2024-09-11T07:36:07.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:07 smithi104 ceph-mon[33507]: Health check failed: Reduced data availability: 2 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-09-11T07:36:07.577 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:07 smithi022 ceph-mon[29462]: pgmap v425: 148 pgs: 3 peering, 145 active+clean; 4.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.50k op/s 2024-09-11T07:36:07.577 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:07 smithi022 ceph-mon[29462]: Health check failed: Reduced data availability: 2 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-09-11T07:36:07.577 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:07 smithi022 ceph-mon[37330]: pgmap v425: 148 pgs: 3 peering, 145 active+clean; 4.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.50k op/s 2024-09-11T07:36:07.577 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:07 smithi022 ceph-mon[37330]: Health check failed: Reduced data availability: 2 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-09-11T07:36:07.592 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:07.634 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:07.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:07.634 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:07.735 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:07.778 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:07.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:07.779 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:07.880 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:07.908 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:07.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:07.909 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:08.010 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:08.033 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:08.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:08.035 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:08.136 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:08.162 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:08.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:08.163 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:08.264 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:08.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:08.315 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:08.315 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:08.416 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:08.461 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:08.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:08.462 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:08.563 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:08.595 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:08.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:08.596 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:08.697 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:08.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:08.727 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:08.727 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:08.828 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:08.857 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:08.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:08.858 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:08.959 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:08.978 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:36:08.998 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:08.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:08.999 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:09.052 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is set 2024-09-11T07:36:09.100 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:09.149 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:09.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:09.151 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:09.251 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:09.293 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:09 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:36:09.293 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:09 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:36:09.293 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:09 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/158577785' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:36:09.294 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:09 smithi022 ceph-mon[29462]: from='client.25255 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:36:09.294 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:09 smithi022 ceph-mon[29462]: pgmap v426: 148 pgs: 3 peering, 145 active+clean; 4.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 740 op/s 2024-09-11T07:36:09.294 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:09.294 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:09 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:36:09.294 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:09 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:36:09.294 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:09 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/158577785' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:36:09.295 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:09 smithi022 ceph-mon[37330]: from='client.25255 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:36:09.295 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:09 smithi022 ceph-mon[37330]: pgmap v426: 148 pgs: 3 peering, 145 active+clean; 4.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 740 op/s 2024-09-11T07:36:09.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:09.296 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:09.396 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:09.440 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:09.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:09.441 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:09.494 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:09 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:36:09.494 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:09 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:36:09.494 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:09 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/158577785' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:36:09.494 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:09 smithi104 ceph-mon[33507]: from='client.25255 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:36:09.494 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:09 smithi104 ceph-mon[33507]: pgmap v426: 148 pgs: 3 peering, 145 active+clean; 4.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 740 op/s 2024-09-11T07:36:09.542 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:09.567 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:09.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:09.569 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:09.669 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:09.695 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:09.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:09.696 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:09.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:36:09 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:36:09.797 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:09.821 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:09.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:09.822 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:09.923 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:09.963 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:09.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:09.964 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:10.065 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:10.096 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:10.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:10.097 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:10.198 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:10.226 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:10.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:10.227 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:10.328 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:10.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:10.364 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:10.364 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:10.373 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:10 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:36:10.373 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:10 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:36:10.373 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:10 smithi022 ceph-mon[29462]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:36:10.373 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:10 smithi022 ceph-mon[29462]: from='client.25255 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:36:10.373 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:10 smithi022 ceph-mon[29462]: osdmap e226: 8 total, 8 up, 8 in 2024-09-11T07:36:10.373 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:10 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:36:10.374 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:10 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:36:10.374 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:10 smithi022 ceph-mon[37330]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:36:10.374 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:10 smithi022 ceph-mon[37330]: from='client.25255 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:36:10.374 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:10 smithi022 ceph-mon[37330]: osdmap e226: 8 total, 8 up, 8 in 2024-09-11T07:36:10.465 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:10.490 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:10.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:10.491 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:10.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:10 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:36:10.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:10 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:36:10.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:10 smithi104 ceph-mon[33507]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:36:10.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:10 smithi104 ceph-mon[33507]: from='client.25255 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:36:10.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:10 smithi104 ceph-mon[33507]: osdmap e226: 8 total, 8 up, 8 in 2024-09-11T07:36:10.592 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:10.624 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:10.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:10.625 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:10.658 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:36:10.658 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight 0 1 2024-09-11T07:36:10.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:36:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:36:10] "GET /metrics HTTP/1.1" 200 224188 "" "Prometheus/2.43.0" 2024-09-11T07:36:10.726 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:10.752 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:10.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:10.753 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:10.854 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:10.882 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:10.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:10.883 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:10.984 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:11.019 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:11.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:11.020 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:11.121 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:11.161 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:11.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:11.162 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:11.263 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:11.288 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:11.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:11.289 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:11.390 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:11.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:11 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:11.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:11 smithi022 ceph-mon[29462]: pgmap v428: 148 pgs: 148 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.53k op/s 2024-09-11T07:36:11.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:11 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:11.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:11 smithi022 ceph-mon[37330]: pgmap v428: 148 pgs: 148 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.53k op/s 2024-09-11T07:36:11.419 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:11.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:11.421 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:11.521 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:11.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:11 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:11.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:11 smithi104 ceph-mon[33507]: pgmap v428: 148 pgs: 148 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.53k op/s 2024-09-11T07:36:11.561 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:11.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:11.562 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:11.663 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:11.701 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:11.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:11.702 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:11.803 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:11.844 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:11.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:11.845 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:11.945 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:11.985 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:11.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:11.986 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:12.087 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:12.114 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:12.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:12.116 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:12.216 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:12.244 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:12.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:12.245 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:12.346 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:12.369 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:12 smithi022 ceph-mon[29462]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 3 pgs peering) 2024-09-11T07:36:12.369 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:12 smithi022 ceph-mon[37330]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 3 pgs peering) 2024-09-11T07:36:12.382 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:12.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:12.383 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:12.484 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:12.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:12.514 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:12.514 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:12.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:12 smithi104 ceph-mon[33507]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 3 pgs peering) 2024-09-11T07:36:12.615 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:12.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:12.658 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:12.658 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:12.759 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:12.799 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:12.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:12.800 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:12.900 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:12.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:12.939 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:12.939 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:13.040 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:13.074 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:13.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:13.075 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:13.176 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:13.201 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:36:13.201 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:36:13.201 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:36:13.201 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:36:13.201 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:36:13.201 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:36:13.202 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:36:13.202 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:36:13.202 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:36:13.202 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:36:13.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:13.204 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:13.204 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:13.233 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:36:13.233 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:36:13.263 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:36:13.264 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:36:13.292 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:36:13.293 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:36:13.304 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:13.341 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:13.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:13.342 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:13.360 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:13 smithi022 ceph-mon[37330]: pgmap v429: 148 pgs: 148 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 903 B/s rd, 0 B/s wr, 1.07k op/s 2024-09-11T07:36:13.361 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:13 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2130624141' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:36:13.361 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:13 smithi022 ceph-mon[29462]: pgmap v429: 148 pgs: 148 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 903 B/s rd, 0 B/s wr, 1.07k op/s 2024-09-11T07:36:13.361 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:13 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2130624141' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:36:13.443 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:13.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:13.501 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:13.501 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:13.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:13 smithi104 ceph-mon[33507]: pgmap v429: 148 pgs: 148 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 903 B/s rd, 0 B/s wr, 1.07k op/s 2024-09-11T07:36:13.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:13 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2130624141' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:36:13.602 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:13.627 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:13.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:13.628 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:13.729 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:13.764 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:13.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:13.765 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:13.865 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:13.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:13.899 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:13.899 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:13.999 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:14.031 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:14.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:14.032 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:14.133 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:14.163 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:14.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:14.164 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:14.265 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:14.289 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:14.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:14.290 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:14.391 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:14.412 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T07:36:14.416 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:14.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:14.417 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:14.519 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:14.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:36:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:36:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:36:14.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:14.572 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:14.573 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:14.673 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:14.674 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:36:14.696 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:14.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:14.697 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:14.728 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:36:14.728 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 20, 2024-09-11T07:36:14.728 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 600, 2024-09-11T07:36:14.728 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:36:14.728 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.728 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120408 7.2 e127/92 7:4165bb05:::benchmark_data_smithi022_77720_object120407:head v 127'7646, mlcod=127'7644)", 2024-09-11T07:36:14.728 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:28.698889+0000", 2024-09-11T07:36:14.729 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 346.02271166499997, 2024-09-11T07:36:14.729 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13833764600000001, 2024-09-11T07:36:14.729 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:14.729 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:14.729 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:14.729 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.729 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:14.730 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.698889+0000", 2024-09-11T07:36:14.730 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.730 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.730 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.730 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:14.730 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.698889+0000", 2024-09-11T07:36:14.730 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.731 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.731 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.731 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:14.731 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.698888+0000", 2024-09-11T07:36:14.731 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:36:14.731 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.731 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.731 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:14.731 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.698916+0000", 2024-09-11T07:36:14.731 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8008999999999999e-05 2024-09-11T07:36:14.731 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.731 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.732 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:14.732 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.698924+0000", 2024-09-11T07:36:14.732 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.6410000000000002e-06 2024-09-11T07:36:14.732 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.732 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.732 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:14.732 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.698927+0000", 2024-09-11T07:36:14.732 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2990000000000001e-06 2024-09-11T07:36:14.732 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.732 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.732 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.732 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.699047+0000", 2024-09-11T07:36:14.732 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000119882 2024-09-11T07:36:14.733 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.733 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.733 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:14.733 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.699066+0000", 2024-09-11T07:36:14.733 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9477999999999999e-05 2024-09-11T07:36:14.733 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.733 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.733 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:14.733 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.837173+0000", 2024-09-11T07:36:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.138106542 2024-09-11T07:36:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.837227+0000", 2024-09-11T07:36:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3729e-05 2024-09-11T07:36:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120403 7.a e127/92 7:53556683:::benchmark_data_smithi022_77720_object120402:head v 127'7440, mlcod=127'7437)", 2024-09-11T07:36:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:28.698935+0000", 2024-09-11T07:36:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 346.02266564899998, 2024-09-11T07:36:14.735 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13831479799999999, 2024-09-11T07:36:14.735 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:14.735 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:14.735 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:14.735 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.735 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:14.735 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.698935+0000", 2024-09-11T07:36:14.735 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.735 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.735 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.736 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:14.736 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.698935+0000", 2024-09-11T07:36:14.736 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.736 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.736 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.736 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:14.736 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.698935+0000", 2024-09-11T07:36:14.736 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:36:14.737 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.737 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.737 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:14.737 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.698955+0000", 2024-09-11T07:36:14.737 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0023000000000001e-05 2024-09-11T07:36:14.737 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.737 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.737 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:14.737 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.698958+0000", 2024-09-11T07:36:14.737 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4400000000000001e-06 2024-09-11T07:36:14.737 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.738 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.738 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:14.738 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.698961+0000", 2024-09-11T07:36:14.738 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.6029999999999999e-06 2024-09-11T07:36:14.738 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.738 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.738 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.738 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.699746+0000", 2024-09-11T07:36:14.738 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00078507000000000002 2024-09-11T07:36:14.738 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.738 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.738 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:14.738 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.699765+0000", 2024-09-11T07:36:14.738 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8814999999999999e-05 2024-09-11T07:36:14.739 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.739 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.739 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:14.739 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.837230+0000", 2024-09-11T07:36:14.739 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13746528099999999 2024-09-11T07:36:14.739 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.739 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.739 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:14.739 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.837250+0000", 2024-09-11T07:36:14.739 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9967999999999999e-05 2024-09-11T07:36:14.739 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.739 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:14.739 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.740 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.740 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.740 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120637 7.a e127/92 7:5b8c9ff4:::benchmark_data_smithi022_77720_object120636:head v 127'7458, mlcod=127'7454)", 2024-09-11T07:36:14.740 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.033623+0000", 2024-09-11T07:36:14.740 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 345.68797775799999, 2024-09-11T07:36:14.740 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.21590059, 2024-09-11T07:36:14.740 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:14.740 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:14.740 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:14.740 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.740 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:14.740 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.033623+0000", 2024-09-11T07:36:14.740 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.033623+0000", 2024-09-11T07:36:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.033622+0000", 2024-09-11T07:36:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:36:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.033782+0000", 2024-09-11T07:36:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00016013799999999999 2024-09-11T07:36:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.033791+0000", 2024-09-11T07:36:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.1319999999999994e-06 2024-09-11T07:36:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.033796+0000", 2024-09-11T07:36:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.0189999999999997e-06 2024-09-11T07:36:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.743 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.743 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.034483+0000", 2024-09-11T07:36:14.743 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00068758600000000001 2024-09-11T07:36:14.743 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.743 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.743 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:14.743 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.034508+0000", 2024-09-11T07:36:14.743 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4709e-05 2024-09-11T07:36:14.743 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.743 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.743 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:14.743 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.249434+0000", 2024-09-11T07:36:14.743 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.21492656199999999 2024-09-11T07:36:14.744 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.744 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.744 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:14.744 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.249524+0000", 2024-09-11T07:36:14.744 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.9195000000000002e-05 2024-09-11T07:36:14.744 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.744 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:14.744 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.744 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.745 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.745 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:120646 7.8 7:1a637839:::benchmark_data_smithi022_77720_object120645:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:36:14.745 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.033864+0000", 2024-09-11T07:36:14.745 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 345.68773663500002, 2024-09-11T07:36:14.745 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.21216812400000001, 2024-09-11T07:36:14.745 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:14.745 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:14.745 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:14.745 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:36:14.745 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:36:14.745 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 120646 2024-09-11T07:36:14.745 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.033864+0000", 2024-09-11T07:36:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.033864+0000", 2024-09-11T07:36:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.033862+0000", 2024-09-11T07:36:14.747 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:36:14.747 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.747 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.747 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:14.747 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.033886+0000", 2024-09-11T07:36:14.747 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3734e-05 2024-09-11T07:36:14.747 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.747 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.747 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:14.747 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.033896+0000", 2024-09-11T07:36:14.747 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0633e-05 2024-09-11T07:36:14.748 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.748 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.748 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:14.748 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.033902+0000", 2024-09-11T07:36:14.748 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6409999999999999e-06 2024-09-11T07:36:14.748 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.748 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.749 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.749 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.035936+0000", 2024-09-11T07:36:14.749 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002033993 2024-09-11T07:36:14.749 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.749 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.749 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:14.750 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.036068+0000", 2024-09-11T07:36:14.750 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000131617 2024-09-11T07:36:14.750 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.750 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.750 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 4,7", 2024-09-11T07:36:14.750 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.036237+0000", 2024-09-11T07:36:14.751 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00016937000000000001 2024-09-11T07:36:14.751 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.751 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.751 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:36:14.751 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.039994+0000", 2024-09-11T07:36:14.751 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0037572880000000001 2024-09-11T07:36:14.752 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.752 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.752 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:14.752 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.040299+0000", 2024-09-11T07:36:14.752 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00030425199999999997 2024-09-11T07:36:14.752 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.753 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.753 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:14.753 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.245941+0000", 2024-09-11T07:36:14.753 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.205642886 2024-09-11T07:36:14.753 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.753 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.754 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:14.754 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.245989+0000", 2024-09-11T07:36:14.754 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7794999999999999e-05 2024-09-11T07:36:14.754 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.754 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.755 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:14.755 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246032+0000", 2024-09-11T07:36:14.755 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3050999999999998e-05 2024-09-11T07:36:14.755 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.755 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:14.755 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.756 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.756 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.756 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "rep_scrubmap(7.e e92 from shard 0)", 2024-09-11T07:36:14.756 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.040295+0000", 2024-09-11T07:36:14.756 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 345.68130615899997, 2024-09-11T07:36:14.756 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.205869997, 2024-09-11T07:36:14.757 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:14.757 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:36:14.757 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:14.757 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.757 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:14.758 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.040295+0000", 2024-09-11T07:36:14.758 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.758 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.758 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.758 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:14.759 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.040295+0000", 2024-09-11T07:36:14.759 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.759 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.759 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.759 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:14.759 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.040293+0000", 2024-09-11T07:36:14.760 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:36:14.760 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.760 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.760 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:14.760 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.040316+0000", 2024-09-11T07:36:14.760 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.283e-05 2024-09-11T07:36:14.761 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.761 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.761 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:14.761 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.040326+0000", 2024-09-11T07:36:14.761 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0156999999999999e-05 2024-09-11T07:36:14.762 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.762 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.762 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:14.762 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.040332+0000", 2024-09-11T07:36:14.762 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2700000000000004e-06 2024-09-11T07:36:14.762 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.763 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.763 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.763 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.040384+0000", 2024-09-11T07:36:14.763 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2212000000000001e-05 2024-09-11T07:36:14.763 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.763 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.763 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:14.763 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246049+0000", 2024-09-11T07:36:14.764 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.205664815 2024-09-11T07:36:14.764 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.764 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.764 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:14.764 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246165+0000", 2024-09-11T07:36:14.764 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011607000000000001 2024-09-11T07:36:14.765 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.765 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:14.765 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.765 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.765 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.765 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop_reply(client.24778.0:120646 7.8 e127/92 ondisk, result = 0)", 2024-09-11T07:36:14.765 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.040494+0000", 2024-09-11T07:36:14.765 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 345.68110664199997, 2024-09-11T07:36:14.765 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20554557700000001, 2024-09-11T07:36:14.765 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:14.766 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:36:14.766 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:14.766 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.766 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:14.766 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.040494+0000", 2024-09-11T07:36:14.766 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.766 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.766 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.766 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:14.766 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.040494+0000", 2024-09-11T07:36:14.766 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.040493+0000", 2024-09-11T07:36:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:36:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.040502+0000", 2024-09-11T07:36:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.7870000000000007e-06 2024-09-11T07:36:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.245702+0000", 2024-09-11T07:36:14.768 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20519998 2024-09-11T07:36:14.768 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.768 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.768 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:14.768 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.245717+0000", 2024-09-11T07:36:14.768 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5067e-05 2024-09-11T07:36:14.768 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.768 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.768 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.768 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.245926+0000", 2024-09-11T07:36:14.768 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00020971999999999999 2024-09-11T07:36:14.768 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.768 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.769 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:14.769 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.245935+0000", 2024-09-11T07:36:14.769 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.2709999999999992e-06 2024-09-11T07:36:14.769 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.769 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.769 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:14.769 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246040+0000", 2024-09-11T07:36:14.769 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000105062 2024-09-11T07:36:14.769 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.769 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:14.769 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.769 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.769 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.770 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120649 7.2 e127/92 7:4a3799e9:::benchmark_data_smithi022_77720_object120648:head v 127'7662, mlcod=127'7660)", 2024-09-11T07:36:14.770 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.040796+0000", 2024-09-11T07:36:14.770 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 345.680804443, 2024-09-11T07:36:14.770 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20897986599999999, 2024-09-11T07:36:14.771 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:14.771 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:14.771 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:14.771 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.771 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:14.772 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.040796+0000", 2024-09-11T07:36:14.772 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.772 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.772 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.772 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:14.772 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.040796+0000", 2024-09-11T07:36:14.772 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.773 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.773 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.773 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:14.773 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.040795+0000", 2024-09-11T07:36:14.773 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:36:14.773 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.773 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.773 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:14.774 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.040823+0000", 2024-09-11T07:36:14.774 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8140000000000002e-05 2024-09-11T07:36:14.774 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.774 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.774 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:14.774 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.245868+0000", 2024-09-11T07:36:14.774 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20504457200000001 2024-09-11T07:36:14.774 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.775 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.775 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:14.775 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.245875+0000", 2024-09-11T07:36:14.775 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9609999999999999e-06 2024-09-11T07:36:14.775 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.775 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.775 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.775 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246266+0000", 2024-09-11T07:36:14.776 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00039140300000000002 2024-09-11T07:36:14.776 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.776 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.776 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:14.776 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246284+0000", 2024-09-11T07:36:14.776 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8260000000000001e-05 2024-09-11T07:36:14.776 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.777 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.777 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:14.777 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.249688+0000", 2024-09-11T07:36:14.777 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0034039090000000001 2024-09-11T07:36:14.777 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.777 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.778 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:14.778 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.249776+0000", 2024-09-11T07:36:14.778 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.8109999999999995e-05 2024-09-11T07:36:14.778 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.778 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:14.778 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.778 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.779 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.779 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:155163 7.8 7:18a9ebac:::benchmark_data_smithi022_77720_object31408:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-11T07:36:14.779 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:31:02.178759+0000", 2024-09-11T07:36:14.779 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 312.54284132200002, 2024-09-11T07:36:14.779 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.053786128, 2024-09-11T07:36:14.779 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:14.779 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:36:14.779 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:14.779 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:36:14.779 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:36:14.779 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 155163 2024-09-11T07:36:14.780 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.780 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:14.780 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.780 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:14.780 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.178759+0000", 2024-09-11T07:36:14.780 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.780 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.780 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.780 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:14.780 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.178759+0000", 2024-09-11T07:36:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.178757+0000", 2024-09-11T07:36:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:36:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.178766+0000", 2024-09-11T07:36:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.499e-06 2024-09-11T07:36:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.782 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:14.782 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.178781+0000", 2024-09-11T07:36:14.782 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4473e-05 2024-09-11T07:36:14.782 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.782 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.782 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:14.782 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.178788+0000", 2024-09-11T07:36:14.782 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.815e-06 2024-09-11T07:36:14.782 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.782 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.179125+0000", 2024-09-11T07:36:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00033711499999999997 2024-09-11T07:36:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.179226+0000", 2024-09-11T07:36:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000101075 2024-09-11T07:36:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 4,7", 2024-09-11T07:36:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.179325+0000", 2024-09-11T07:36:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.8962000000000006e-05 2024-09-11T07:36:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.784 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.784 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.784 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.214158+0000", 2024-09-11T07:36:14.784 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.034832954999999999 2024-09-11T07:36:14.784 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.784 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.784 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:36:14.784 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.214165+0000", 2024-09-11T07:36:14.784 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.3209999999999999e-06 2024-09-11T07:36:14.784 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.784 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.784 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.785 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.232477+0000", 2024-09-11T07:36:14.785 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.018311864 2024-09-11T07:36:14.785 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.785 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.785 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:14.785 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.232546+0000", 2024-09-11T07:36:14.785 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.8683999999999995e-05 2024-09-11T07:36:14.785 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.785 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:14.785 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.785 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.785 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.785 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:155175 7.8 7:1ce8d305:::benchmark_data_smithi022_77720_object31420:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-11T07:36:14.786 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:31:02.184282+0000", 2024-09-11T07:36:14.786 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 312.537318486, 2024-09-11T07:36:14.786 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0560903399999999, 2024-09-11T07:36:14.786 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:14.786 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:36:14.786 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:14.786 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:36:14.786 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:36:14.786 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 155175 2024-09-11T07:36:14.786 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.786 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:14.786 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.786 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.184282+0000", 2024-09-11T07:36:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.184282+0000", 2024-09-11T07:36:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.184280+0000", 2024-09-11T07:36:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:36:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.788 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.788 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:14.788 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.184289+0000", 2024-09-11T07:36:14.788 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.9590000000000001e-06 2024-09-11T07:36:14.788 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.788 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.788 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:14.788 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.184314+0000", 2024-09-11T07:36:14.788 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.472e-05 2024-09-11T07:36:14.789 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.789 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.789 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:14.789 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.184320+0000", 2024-09-11T07:36:14.789 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2450000000000003e-06 2024-09-11T07:36:14.789 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.789 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.789 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.789 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.187770+0000", 2024-09-11T07:36:14.789 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0034499370000000001 2024-09-11T07:36:14.789 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.789 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.789 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:14.790 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.187887+0000", 2024-09-11T07:36:14.790 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000117359 2024-09-11T07:36:14.790 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.790 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.790 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 4,7", 2024-09-11T07:36:14.790 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.188004+0000", 2024-09-11T07:36:14.790 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011642 2024-09-11T07:36:14.790 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.790 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.790 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.790 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.214681+0000", 2024-09-11T07:36:14.790 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.026676934999999999 2024-09-11T07:36:14.791 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.791 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.791 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:36:14.791 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.214687+0000", 2024-09-11T07:36:14.791 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2929999999999997e-06 2024-09-11T07:36:14.791 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.791 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.791 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.791 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.232757+0000", 2024-09-11T07:36:14.791 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0180701839999999 2024-09-11T07:36:14.792 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.792 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.792 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for missing object", 2024-09-11T07:36:14.792 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.232866+0000", 2024-09-11T07:36:14.792 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010851799999999999 2024-09-11T07:36:14.792 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.793 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.793 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.793 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.240308+0000", 2024-09-11T07:36:14.793 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0074420119999999996 2024-09-11T07:36:14.793 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.793 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.794 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:14.794 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.240373+0000", 2024-09-11T07:36:14.794 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5028000000000001e-05 2024-09-11T07:36:14.794 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.794 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:14.794 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.795 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.795 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.795 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:155180 7.8 7:17985a55:::benchmark_data_smithi022_77720_object31425:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-11T07:36:14.795 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:31:02.209024+0000", 2024-09-11T07:36:14.795 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 312.51257643999998, 2024-09-11T07:36:14.795 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.047352785, 2024-09-11T07:36:14.796 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:14.796 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:14.796 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:14.796 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:36:14.796 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:36:14.796 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 155180 2024-09-11T07:36:14.797 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.797 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:14.797 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.797 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:14.797 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.209024+0000", 2024-09-11T07:36:14.797 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.797 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.797 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.798 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:14.798 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.209024+0000", 2024-09-11T07:36:14.798 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.798 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.798 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.798 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:14.798 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.209023+0000", 2024-09-11T07:36:14.798 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:36:14.799 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.799 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.799 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:14.799 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.209029+0000", 2024-09-11T07:36:14.799 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.139e-06 2024-09-11T07:36:14.799 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.799 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.799 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:14.800 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.209036+0000", 2024-09-11T07:36:14.800 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5320000000000001e-06 2024-09-11T07:36:14.800 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.800 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.800 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:14.800 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.209041+0000", 2024-09-11T07:36:14.800 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9989999999999999e-06 2024-09-11T07:36:14.800 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.801 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.801 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.801 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.214714+0000", 2024-09-11T07:36:14.801 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0056723880000000004 2024-09-11T07:36:14.801 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.801 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.801 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:36:14.801 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.214725+0000", 2024-09-11T07:36:14.802 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1027e-05 2024-09-11T07:36:14.802 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.802 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.802 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.802 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.232892+0000", 2024-09-11T07:36:14.802 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0181671370000001 2024-09-11T07:36:14.802 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.802 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.803 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:14.803 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.232994+0000", 2024-09-11T07:36:14.803 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000101758 2024-09-11T07:36:14.803 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.803 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.803 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 4", 2024-09-11T07:36:14.803 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.233100+0000", 2024-09-11T07:36:14.803 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000105917 2024-09-11T07:36:14.803 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.803 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.803 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:14.803 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.248918+0000", 2024-09-11T07:36:14.803 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.015818125999999998 2024-09-11T07:36:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:36:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.256255+0000", 2024-09-11T07:36:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0073373910000000004 2024-09-11T07:36:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.256298+0000", 2024-09-11T07:36:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3430000000000003e-05 2024-09-11T07:36:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:14.805 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.256377+0000", 2024-09-11T07:36:14.805 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.8665999999999995e-05 2024-09-11T07:36:14.805 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.805 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:14.805 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.805 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.805 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.805 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:155176 7.5 7:ac874f27:::benchmark_data_smithi022_77720_object31421:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e134)", 2024-09-11T07:36:14.805 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:31:02.209045+0000", 2024-09-11T07:36:14.805 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 312.51255559999998, 2024-09-11T07:36:14.805 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.013530177, 2024-09-11T07:36:14.805 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:14.805 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:36:14.806 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:14.806 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:36:14.806 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:36:14.806 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 155176 2024-09-11T07:36:14.806 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.806 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:14.806 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.806 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:14.806 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.209045+0000", 2024-09-11T07:36:14.806 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.806 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.209045+0000", 2024-09-11T07:36:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.209045+0000", 2024-09-11T07:36:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:36:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.209048+0000", 2024-09-11T07:36:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0479999999999999e-06 2024-09-11T07:36:14.808 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.808 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.808 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:14.808 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.209052+0000", 2024-09-11T07:36:14.808 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6759999999999998e-06 2024-09-11T07:36:14.808 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.808 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.808 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:14.808 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.209056+0000", 2024-09-11T07:36:14.808 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.602e-06 2024-09-11T07:36:14.808 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.808 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.809 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.809 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.209744+0000", 2024-09-11T07:36:14.809 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00068834400000000002 2024-09-11T07:36:14.809 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.809 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.809 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:36:14.809 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.209753+0000", 2024-09-11T07:36:14.809 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.5960000000000004e-06 2024-09-11T07:36:14.809 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.810 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.810 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.810 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.220849+0000", 2024-09-11T07:36:14.810 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0110962889999999 2024-09-11T07:36:14.810 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.810 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.810 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:14.810 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.222575+0000", 2024-09-11T07:36:14.810 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001726053 2024-09-11T07:36:14.810 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.810 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:14.810 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.811 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.811 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.811 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:155185 7.8 7:1096647c:::benchmark_data_smithi022_77720_object31430:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-11T07:36:14.811 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:31:02.212649+0000", 2024-09-11T07:36:14.811 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 312.508951389, 2024-09-11T07:36:14.811 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.043859104, 2024-09-11T07:36:14.811 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:14.811 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:14.811 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:14.811 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:36:14.811 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:36:14.811 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 155185 2024-09-11T07:36:14.811 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.811 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:14.812 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.812 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:14.812 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.212649+0000", 2024-09-11T07:36:14.812 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.812 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.812 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.812 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:14.812 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.212649+0000", 2024-09-11T07:36:14.812 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.812 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.812 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.812 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:14.812 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.212647+0000", 2024-09-11T07:36:14.813 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:36:14.813 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.813 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.813 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:14.813 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.212655+0000", 2024-09-11T07:36:14.813 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3580000000000003e-06 2024-09-11T07:36:14.813 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.813 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.813 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:14.813 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.212668+0000", 2024-09-11T07:36:14.813 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2611e-05 2024-09-11T07:36:14.813 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.814 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.814 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:14.814 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.212675+0000", 2024-09-11T07:36:14.814 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4849999999999997e-06 2024-09-11T07:36:14.814 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.814 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.814 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.814 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.214745+0000", 2024-09-11T07:36:14.814 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0020704909999999998 2024-09-11T07:36:14.814 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.814 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.814 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:36:14.814 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.214750+0000", 2024-09-11T07:36:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8500000000000002e-06 2024-09-11T07:36:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.236541+0000", 2024-09-11T07:36:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.021790894 2024-09-11T07:36:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.236663+0000", 2024-09-11T07:36:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012232000000000001 2024-09-11T07:36:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 4", 2024-09-11T07:36:14.816 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.236783+0000", 2024-09-11T07:36:14.816 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000119498 2024-09-11T07:36:14.816 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.816 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.816 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:14.816 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.248946+0000", 2024-09-11T07:36:14.816 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.012163386999999999 2024-09-11T07:36:14.816 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.816 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.816 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:36:14.817 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.256388+0000", 2024-09-11T07:36:14.817 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0074417199999999998 2024-09-11T07:36:14.817 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.817 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.817 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:14.817 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.256430+0000", 2024-09-11T07:36:14.817 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2661999999999999e-05 2024-09-11T07:36:14.817 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.817 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.817 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:14.817 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.256508+0000", 2024-09-11T07:36:14.817 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.8145999999999993e-05 2024-09-11T07:36:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:155186 7.8 7:1b18abab:::benchmark_data_smithi022_77720_object31431:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-11T07:36:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:31:02.214826+0000", 2024-09-11T07:36:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 312.50677443000001, 2024-09-11T07:36:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0418078559999999, 2024-09-11T07:36:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:36:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:36:14.819 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 155186 2024-09-11T07:36:14.819 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.819 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:14.819 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.819 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:14.819 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.214826+0000", 2024-09-11T07:36:14.819 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.819 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.819 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.819 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:14.819 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.214826+0000", 2024-09-11T07:36:14.820 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.820 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.820 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.820 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:14.820 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.214824+0000", 2024-09-11T07:36:14.820 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:36:14.820 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.820 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.820 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:14.820 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.214831+0000", 2024-09-11T07:36:14.820 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0759999999999997e-06 2024-09-11T07:36:14.820 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.820 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.821 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:14.821 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.214840+0000", 2024-09-11T07:36:14.821 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.9390000000000003e-06 2024-09-11T07:36:14.821 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.821 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.821 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:14.821 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.214846+0000", 2024-09-11T07:36:14.821 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.5160000000000002e-06 2024-09-11T07:36:14.821 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.822 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.822 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.822 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.214888+0000", 2024-09-11T07:36:14.822 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2395000000000003e-05 2024-09-11T07:36:14.822 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.822 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.822 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:36:14.822 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.214894+0000", 2024-09-11T07:36:14.823 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9830000000000001e-06 2024-09-11T07:36:14.823 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.823 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.823 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.823 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.240396+0000", 2024-09-11T07:36:14.823 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0255020459999999 2024-09-11T07:36:14.823 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.823 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.823 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:14.824 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.240500+0000", 2024-09-11T07:36:14.824 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000103738 2024-09-11T07:36:14.824 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.824 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.824 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 4", 2024-09-11T07:36:14.824 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.240618+0000", 2024-09-11T07:36:14.824 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000118532 2024-09-11T07:36:14.824 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.825 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.825 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:14.825 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.248973+0000", 2024-09-11T07:36:14.825 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0083549569999999997 2024-09-11T07:36:14.825 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.825 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.825 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:36:14.825 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.256515+0000", 2024-09-11T07:36:14.826 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0075419249999999997 2024-09-11T07:36:14.826 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.826 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.826 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:14.826 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.256558+0000", 2024-09-11T07:36:14.826 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2889999999999998e-05 2024-09-11T07:36:14.826 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.827 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.827 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:14.827 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.256634+0000", 2024-09-11T07:36:14.827 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5957999999999993e-05 2024-09-11T07:36:14.827 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.827 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:14.827 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.827 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.828 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.828 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:155193 7.5 7:af5a3de3:::benchmark_data_smithi022_77720_object31438:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-11T07:36:14.828 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:31:02.215609+0000", 2024-09-11T07:36:14.828 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 312.50599197000003, 2024-09-11T07:36:14.828 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.024030797, 2024-09-11T07:36:14.828 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:14.829 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:14.831 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:14.831 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:14.831 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:36:14.831 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:36:14.831 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 155193 2024-09-11T07:36:14.831 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.831 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:14.831 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.832 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:14.832 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.215609+0000", 2024-09-11T07:36:14.832 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.832 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.832 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.832 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:14.832 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.215609+0000", 2024-09-11T07:36:14.833 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.833 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.833 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.833 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:14.833 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.215608+0000", 2024-09-11T07:36:14.833 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:36:14.833 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.833 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.834 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:14.834 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.215613+0000", 2024-09-11T07:36:14.834 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1290000000000004e-06 2024-09-11T07:36:14.834 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.834 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.834 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:14.834 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.215624+0000", 2024-09-11T07:36:14.834 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1420000000000001e-05 2024-09-11T07:36:14.834 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.834 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.215630+0000", 2024-09-11T07:36:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3210000000000001e-06 2024-09-11T07:36:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.215670+0000", 2024-09-11T07:36:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9478999999999999e-05 2024-09-11T07:36:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:36:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.215676+0000", 2024-09-11T07:36:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4250000000000003e-06 2024-09-11T07:36:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.222601+0000", 2024-09-11T07:36:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.006924717 2024-09-11T07:36:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.222991+0000", 2024-09-11T07:36:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00038962699999999997 2024-09-11T07:36:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0", 2024-09-11T07:36:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.223086+0000", 2024-09-11T07:36:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.5458999999999995e-05 2024-09-11T07:36:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:36:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.228698+0000", 2024-09-11T07:36:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0056120720000000001 2024-09-11T07:36:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.239458+0000", 2024-09-11T07:36:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.010759805000000001 2024-09-11T07:36:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.239554+0000", 2024-09-11T07:36:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.5567000000000002e-05 2024-09-11T07:36:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.239640+0000", 2024-09-11T07:36:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.6024000000000002e-05 2024-09-11T07:36:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:155198 7.5 7:a0f48ad5:::benchmark_data_smithi022_77720_object31443:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-11T07:36:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:31:02.220995+0000", 2024-09-11T07:36:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 312.500606275, 2024-09-11T07:36:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0188490109999999, 2024-09-11T07:36:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:36:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:36:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 155198 2024-09-11T07:36:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.220995+0000", 2024-09-11T07:36:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.220995+0000", 2024-09-11T07:36:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.220993+0000", 2024-09-11T07:36:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:36:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.841 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:14.841 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.221000+0000", 2024-09-11T07:36:14.841 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.2609999999999996e-06 2024-09-11T07:36:14.841 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.841 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.841 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:14.841 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.221013+0000", 2024-09-11T07:36:14.841 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2907e-05 2024-09-11T07:36:14.841 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.841 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.841 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:14.841 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.221019+0000", 2024-09-11T07:36:14.842 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0750000000000003e-06 2024-09-11T07:36:14.842 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.842 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.842 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.842 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.221069+0000", 2024-09-11T07:36:14.842 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9598999999999997e-05 2024-09-11T07:36:14.842 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.842 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.842 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:36:14.842 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.221075+0000", 2024-09-11T07:36:14.842 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.6950000000000003e-06 2024-09-11T07:36:14.842 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.842 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.843 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.843 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.225633+0000", 2024-09-11T07:36:14.843 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.004557396 2024-09-11T07:36:14.843 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.843 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.843 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:14.843 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.225738+0000", 2024-09-11T07:36:14.843 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000105157 2024-09-11T07:36:14.843 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.843 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.843 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0", 2024-09-11T07:36:14.843 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.225840+0000", 2024-09-11T07:36:14.843 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000102688 2024-09-11T07:36:14.844 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.844 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.844 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:36:14.844 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.228705+0000", 2024-09-11T07:36:14.844 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0028649719999999999 2024-09-11T07:36:14.844 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.844 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.844 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:14.844 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.239671+0000", 2024-09-11T07:36:14.844 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.010965803999999999 2024-09-11T07:36:14.844 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.844 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.844 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:14.845 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.239756+0000", 2024-09-11T07:36:14.845 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.4844000000000003e-05 2024-09-11T07:36:14.845 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.845 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.845 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:14.845 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.239844+0000", 2024-09-11T07:36:14.845 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.7430999999999997e-05 2024-09-11T07:36:14.845 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.845 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:14.845 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.845 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.845 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.845 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:155209 7.8 7:1292d403:::benchmark_data_smithi022_77720_object31454:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-11T07:36:14.846 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:31:02.245397+0000", 2024-09-11T07:36:14.846 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 312.47620378200003, 2024-09-11T07:36:14.846 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0113786339999999, 2024-09-11T07:36:14.846 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:14.846 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:14.846 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:14.846 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:36:14.846 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:36:14.846 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 155209 2024-09-11T07:36:14.846 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.846 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:14.846 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.846 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:14.847 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.245397+0000", 2024-09-11T07:36:14.847 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.847 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.847 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.847 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:14.847 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.245397+0000", 2024-09-11T07:36:14.847 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.847 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.847 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.847 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:14.847 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.245395+0000", 2024-09-11T07:36:14.847 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:36:14.847 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.848 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.848 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:14.848 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.245402+0000", 2024-09-11T07:36:14.848 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.3939999999999998e-06 2024-09-11T07:36:14.848 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.848 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.848 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:14.848 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.245413+0000", 2024-09-11T07:36:14.848 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0821e-05 2024-09-11T07:36:14.848 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.848 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.848 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:14.848 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.245418+0000", 2024-09-11T07:36:14.849 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1800000000000004e-06 2024-09-11T07:36:14.849 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.849 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.849 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.849 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.245450+0000", 2024-09-11T07:36:14.849 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.1825000000000001e-05 2024-09-11T07:36:14.849 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.849 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.849 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:36:14.849 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.245455+0000", 2024-09-11T07:36:14.850 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2560000000000002e-06 2024-09-11T07:36:14.850 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.850 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.850 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.850 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.244298+0000", 2024-09-11T07:36:14.850 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.998842915 2024-09-11T07:36:14.850 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.850 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.850 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:14.850 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.244405+0000", 2024-09-11T07:36:14.850 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010728100000000001 2024-09-11T07:36:14.850 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.850 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.850 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 4", 2024-09-11T07:36:14.851 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.244517+0000", 2024-09-11T07:36:14.851 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000111592 2024-09-11T07:36:14.851 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.851 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.851 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:14.851 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.249003+0000", 2024-09-11T07:36:14.851 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0044859019999999999 2024-09-11T07:36:14.851 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.851 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.851 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:36:14.851 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.256641+0000", 2024-09-11T07:36:14.851 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0076381640000000002 2024-09-11T07:36:14.852 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.852 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.852 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:14.852 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.256689+0000", 2024-09-11T07:36:14.852 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7976000000000003e-05 2024-09-11T07:36:14.852 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.852 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.852 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:14.852 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.256776+0000", 2024-09-11T07:36:14.852 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.6549000000000001e-05 2024-09-11T07:36:14.852 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.852 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:14.852 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.853 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.853 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.853 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24353.0:884 5.d 5:b4812045:::notify.4:head [watch reconnect cookie 94395805755184 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-11T07:36:14.853 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:31:02.582133+0000", 2024-09-11T07:36:14.853 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 312.13946772499997, 2024-09-11T07:36:14.853 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.646530991, 2024-09-11T07:36:14.853 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:14.853 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:36:14.853 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:14.853 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24353", 2024-09-11T07:36:14.853 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1860969011", 2024-09-11T07:36:14.853 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 884 2024-09-11T07:36:14.854 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.854 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:14.854 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.854 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:14.854 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.582133+0000", 2024-09-11T07:36:14.854 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.854 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.856 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.856 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:14.856 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.582133+0000", 2024-09-11T07:36:14.856 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.857 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.857 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.857 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:14.857 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.582131+0000", 2024-09-11T07:36:14.857 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:36:14.857 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.857 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.857 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:14.857 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.582139+0000", 2024-09-11T07:36:14.857 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.8460000000000008e-06 2024-09-11T07:36:14.857 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.857 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.858 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:14.858 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.582149+0000", 2024-09-11T07:36:14.858 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0518e-05 2024-09-11T07:36:14.858 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.858 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.858 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:14.858 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.582155+0000", 2024-09-11T07:36:14.858 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1550000000000004e-06 2024-09-11T07:36:14.858 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.858 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.858 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.858 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.582218+0000", 2024-09-11T07:36:14.858 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2255999999999995e-05 2024-09-11T07:36:14.858 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.859 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.859 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:36:14.859 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.582227+0000", 2024-09-11T07:36:14.859 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.4350000000000003e-06 2024-09-11T07:36:14.859 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.859 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.859 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.859 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.200239+0000", 2024-09-11T07:36:14.859 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.61801146200000001 2024-09-11T07:36:14.859 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.859 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.859 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:14.859 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.202383+0000", 2024-09-11T07:36:14.860 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0021441960000000001 2024-09-11T07:36:14.860 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.860 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.860 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:14.860 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.228664+0000", 2024-09-11T07:36:14.860 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.026281262999999999 2024-09-11T07:36:14.860 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.860 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:14.860 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.860 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.860 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.860 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:213451 7.8 7:13d4c186:::benchmark_data_smithi022_77720_object89696:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-11T07:36:14.860 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:00.074514+0000", 2024-09-11T07:36:14.860 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 254.64708720300001, 2024-09-11T07:36:14.861 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9915458479999999, 2024-09-11T07:36:14.861 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:14.861 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:36:14.861 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:14.861 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:36:14.861 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:36:14.861 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213451 2024-09-11T07:36:14.861 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.861 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:14.861 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.861 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:14.861 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.074514+0000", 2024-09-11T07:36:14.861 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.861 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.862 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.862 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:14.862 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.074514+0000", 2024-09-11T07:36:14.862 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.862 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.862 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.862 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:14.862 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.074511+0000", 2024-09-11T07:36:14.862 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:36:14.862 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.862 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.862 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:14.862 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.074519+0000", 2024-09-11T07:36:14.862 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.0989999999999999e-06 2024-09-11T07:36:14.863 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.863 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.863 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:14.863 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.074533+0000", 2024-09-11T07:36:14.863 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3464e-05 2024-09-11T07:36:14.863 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.863 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.863 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:14.863 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.074539+0000", 2024-09-11T07:36:14.863 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9730000000000002e-06 2024-09-11T07:36:14.863 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.863 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.863 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.863 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.075492+0000", 2024-09-11T07:36:14.863 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000953078 2024-09-11T07:36:14.864 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.864 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.864 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:14.864 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.075600+0000", 2024-09-11T07:36:14.864 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000108399 2024-09-11T07:36:14.864 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.864 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.864 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 4", 2024-09-11T07:36:14.864 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.075709+0000", 2024-09-11T07:36:14.864 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000108416 2024-09-11T07:36:14.864 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.864 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.864 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.864 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.103173+0000", 2024-09-11T07:36:14.865 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.027464691999999999 2024-09-11T07:36:14.865 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.865 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.865 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:36:14.865 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.103183+0000", 2024-09-11T07:36:14.865 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.6209999999999999e-06 2024-09-11T07:36:14.865 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.865 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.865 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.865 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.491700+0000", 2024-09-11T07:36:14.865 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.388517271 2024-09-11T07:36:14.865 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.865 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.866 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:36:14.866 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.491713+0000", 2024-09-11T07:36:14.866 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2479e-05 2024-09-11T07:36:14.866 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.866 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.866 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.866 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.033294+0000", 2024-09-11T07:36:14.866 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5415816490000001 2024-09-11T07:36:14.866 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.866 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.866 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for missing object", 2024-09-11T07:36:14.866 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.033458+0000", 2024-09-11T07:36:14.866 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000163392 2024-09-11T07:36:14.866 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.866 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.867 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.867 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.065939+0000", 2024-09-11T07:36:14.867 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.032481253000000002 2024-09-11T07:36:14.867 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.867 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.867 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:14.867 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.066059+0000", 2024-09-11T07:36:14.867 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012038499999999999 2024-09-11T07:36:14.867 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.867 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:14.867 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.867 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.867 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.867 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:213469 7.8 7:1e57e751:::benchmark_data_smithi022_77720_object89714:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-11T07:36:14.868 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:00.108060+0000", 2024-09-11T07:36:14.868 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 254.61354084000001, 2024-09-11T07:36:14.868 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.958251086, 2024-09-11T07:36:14.868 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:14.868 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:14.868 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:14.868 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:36:14.868 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:36:14.868 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213469 2024-09-11T07:36:14.868 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.868 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:14.868 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.868 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:14.868 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.108060+0000", 2024-09-11T07:36:14.869 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.869 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.869 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.869 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:14.869 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.108060+0000", 2024-09-11T07:36:14.869 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.869 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.869 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.869 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:14.869 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.108059+0000", 2024-09-11T07:36:14.869 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:36:14.869 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.869 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.870 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:14.870 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.108063+0000", 2024-09-11T07:36:14.870 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.393e-06 2024-09-11T07:36:14.870 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.870 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.870 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:14.870 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.108068+0000", 2024-09-11T07:36:14.870 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.198e-06 2024-09-11T07:36:14.870 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.870 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.870 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:14.870 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.108072+0000", 2024-09-11T07:36:14.870 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9299999999999996e-06 2024-09-11T07:36:14.870 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.870 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.871 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.871 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.108526+0000", 2024-09-11T07:36:14.871 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00045391499999999999 2024-09-11T07:36:14.871 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.871 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.871 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:36:14.871 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.108534+0000", 2024-09-11T07:36:14.871 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.0090000000000007e-06 2024-09-11T07:36:14.871 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.871 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.871 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.871 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.491756+0000", 2024-09-11T07:36:14.871 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.38322225799999998 2024-09-11T07:36:14.871 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.871 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.872 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:36:14.872 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.491764+0000", 2024-09-11T07:36:14.872 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.6520000000000006e-06 2024-09-11T07:36:14.872 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.872 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.872 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.872 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.033884+0000", 2024-09-11T07:36:14.872 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5421203960000001 2024-09-11T07:36:14.872 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.872 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.872 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:14.872 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.034019+0000", 2024-09-11T07:36:14.872 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000134432 2024-09-11T07:36:14.873 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.873 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.873 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 4,7", 2024-09-11T07:36:14.873 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.034179+0000", 2024-09-11T07:36:14.873 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00016033700000000001 2024-09-11T07:36:14.873 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.873 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.873 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:36:14.873 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.045351+0000", 2024-09-11T07:36:14.873 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.011172375999999999 2024-09-11T07:36:14.873 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.873 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.873 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:14.873 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.047887+0000", 2024-09-11T07:36:14.874 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002535271 2024-09-11T07:36:14.874 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.874 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.874 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:14.874 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.066092+0000", 2024-09-11T07:36:14.874 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.018205559999999999 2024-09-11T07:36:14.874 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.874 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.874 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:14.874 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.066170+0000", 2024-09-11T07:36:14.874 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.7770999999999998e-05 2024-09-11T07:36:14.874 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.874 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.874 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:14.874 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.066311+0000", 2024-09-11T07:36:14.875 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014108799999999999 2024-09-11T07:36:14.875 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.875 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:14.875 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.875 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.875 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.875 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:213474 7.8 7:1fd17165:::benchmark_data_smithi022_77720_object89719:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-11T07:36:14.875 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:00.117937+0000", 2024-09-11T07:36:14.875 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 254.60366405600001, 2024-09-11T07:36:14.875 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.948734757, 2024-09-11T07:36:14.875 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:14.875 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:14.875 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:14.875 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:36:14.876 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:36:14.876 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213474 2024-09-11T07:36:14.876 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.876 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:14.876 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.876 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:14.876 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.117937+0000", 2024-09-11T07:36:14.876 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.876 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.876 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.876 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:14.876 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.117937+0000", 2024-09-11T07:36:14.876 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:14.876 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.877 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.877 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:14.877 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.117935+0000", 2024-09-11T07:36:14.877 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:36:14.877 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.877 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.877 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:14.877 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.117942+0000", 2024-09-11T07:36:14.877 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5889999999999997e-06 2024-09-11T07:36:14.877 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.877 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.877 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:14.878 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.117954+0000", 2024-09-11T07:36:14.878 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1943000000000001e-05 2024-09-11T07:36:14.878 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.878 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.878 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:14.878 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.117960+0000", 2024-09-11T07:36:14.878 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6740000000000002e-06 2024-09-11T07:36:14.878 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.878 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.878 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.878 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.118025+0000", 2024-09-11T07:36:14.878 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4932999999999996e-05 2024-09-11T07:36:14.879 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.879 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.879 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:36:14.879 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.118042+0000", 2024-09-11T07:36:14.879 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.6725999999999999e-05 2024-09-11T07:36:14.879 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.879 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.879 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.879 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.491808+0000", 2024-09-11T07:36:14.879 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.373765864 2024-09-11T07:36:14.879 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.879 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.880 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:36:14.880 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.491816+0000", 2024-09-11T07:36:14.880 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3140000000000004e-06 2024-09-11T07:36:14.880 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.880 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.880 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:14.880 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.038131+0000", 2024-09-11T07:36:14.880 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5463155749999999 2024-09-11T07:36:14.880 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.880 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.880 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:14.880 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.038254+0000", 2024-09-11T07:36:14.880 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012281799999999999 2024-09-11T07:36:14.881 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.881 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.881 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 4,7", 2024-09-11T07:36:14.881 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.038383+0000", 2024-09-11T07:36:14.881 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012845099999999999 2024-09-11T07:36:14.881 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.881 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.881 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:36:14.882 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.045356+0000", 2024-09-11T07:36:14.882 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.006973595 2024-09-11T07:36:14.882 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.882 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.882 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:14.882 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.047917+0000", 2024-09-11T07:36:14.882 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0025606610000000001 2024-09-11T07:36:14.882 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.882 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.882 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:14.882 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.066346+0000", 2024-09-11T07:36:14.883 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.018428863 2024-09-11T07:36:14.883 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.883 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.883 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:14.883 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.066496+0000", 2024-09-11T07:36:14.883 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014982 2024-09-11T07:36:14.883 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:14.883 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:14.883 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:14.883 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.066671+0000", 2024-09-11T07:36:14.884 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00017581000000000001 2024-09-11T07:36:14.884 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.884 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:14.884 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.884 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:14.884 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:14.884 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:36:14.923 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:14.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:14.925 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:15.025 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:15.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:15.063 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:15.064 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:15.164 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:15.190 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:15.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:15.191 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:15.292 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:15.320 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:15.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:15.321 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:15.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:15 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:36:15.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:15 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:36:15.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:15 smithi022 ceph-mon[37330]: pgmap v430: 148 pgs: 148 active+clean; 3.7 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 344 B/s rd, 0 B/s wr, 782 op/s 2024-09-11T07:36:15.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:15 smithi022 ceph-mon[37330]: osdmap e227: 8 total, 8 up, 8 in 2024-09-11T07:36:15.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:15 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:36:15.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:15 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:36:15.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:15 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:36:15.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:15 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:36:15.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:15 smithi022 ceph-mon[29462]: pgmap v430: 148 pgs: 148 active+clean; 3.7 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 344 B/s rd, 0 B/s wr, 782 op/s 2024-09-11T07:36:15.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:15 smithi022 ceph-mon[29462]: osdmap e227: 8 total, 8 up, 8 in 2024-09-11T07:36:15.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:15 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:36:15.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:15 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:36:15.422 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:15.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:15.457 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:15.457 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:15.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:15 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:36:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:15 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:36:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:15 smithi104 ceph-mon[33507]: pgmap v430: 148 pgs: 148 active+clean; 3.7 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 344 B/s rd, 0 B/s wr, 782 op/s 2024-09-11T07:36:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:15 smithi104 ceph-mon[33507]: osdmap e227: 8 total, 8 up, 8 in 2024-09-11T07:36:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:15 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:36:15.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:15 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:36:15.558 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:15.603 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:15.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:15.604 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:15.705 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:15.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:15.740 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:15.741 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:15.842 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:15.873 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:15.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:15.874 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:15.975 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:16.009 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:16.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:16.011 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:16.111 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:16.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:36:16.149 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-11T07:36:16.158 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:16.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:16.160 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:16.261 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:16.289 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:16.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:16.290 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:16.390 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:16 smithi022 ceph-mon[29462]: osdmap e228: 8 total, 8 up, 8 in 2024-09-11T07:36:16.391 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:16.391 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:16 smithi022 ceph-mon[37330]: osdmap e228: 8 total, 8 up, 8 in 2024-09-11T07:36:16.437 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:16.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:16.438 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:16.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:16 smithi104 ceph-mon[33507]: osdmap e228: 8 total, 8 up, 8 in 2024-09-11T07:36:16.539 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:16.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:16.586 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:16.586 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:16.687 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:16.731 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:16.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:16.733 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:16.833 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:16.872 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:16.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:16.873 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:16.973 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:17.007 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:17.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:17.008 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:17.109 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:17.138 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:17.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:17.139 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:17.240 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:17.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:17.269 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:17.269 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:17.372 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:17.399 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:17.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:17.400 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:17.501 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:17.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:17 smithi104 ceph-mon[33507]: pgmap v433: 148 pgs: 148 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.52k op/s 2024-09-11T07:36:17.532 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:17.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:17.533 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:17.614 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:17 smithi022 ceph-mon[29462]: pgmap v433: 148 pgs: 148 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.52k op/s 2024-09-11T07:36:17.614 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:17 smithi022 ceph-mon[37330]: pgmap v433: 148 pgs: 148 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.52k op/s 2024-09-11T07:36:17.633 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:17.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:17.682 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:17.682 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:17.721 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:36:17.758 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:36:17.783 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:17.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:17.810 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:17.810 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:17.911 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:17.957 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:17.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:17.958 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:18.059 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:18.089 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:18.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:18.090 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:18.190 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:18.220 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:18.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:18.222 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:18.322 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:18.364 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:18.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:18.365 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:18.466 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:18.491 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:18.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:18.492 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:18.593 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:18.618 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:18.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:18.619 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:18.720 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:18.723 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:36:18.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:18.751 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:18.751 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:18.852 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:18.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:18.904 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:18.905 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:19.009 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:19.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:19.056 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:19.056 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:19.157 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:19.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:19.198 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:19.198 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:19.237 INFO:teuthology.orchestra.run.smithi022.stderr:reweighted osd.0 to 1 (10000) 2024-09-11T07:36:19.299 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:19.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:19.330 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:19.330 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:19.431 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:19.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:19.458 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:19.458 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:19.505 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:19 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:36:19.505 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:19 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:36:19.505 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:19 smithi104 ceph-mon[33507]: pgmap v434: 148 pgs: 148 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 745 op/s 2024-09-11T07:36:19.505 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:19 smithi104 ceph-mon[33507]: osdmap e229: 8 total, 8 up, 8 in 2024-09-11T07:36:19.505 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:19 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2425786234' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T07:36:19.558 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:19.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:19.605 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:19.606 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:19.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:19 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:36:19.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:19 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:36:19.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:19 smithi022 ceph-mon[29462]: pgmap v434: 148 pgs: 148 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 745 op/s 2024-09-11T07:36:19.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:19 smithi022 ceph-mon[29462]: osdmap e229: 8 total, 8 up, 8 in 2024-09-11T07:36:19.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:19 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2425786234' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T07:36:19.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:19 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:36:19.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:19 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:36:19.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:19 smithi022 ceph-mon[37330]: pgmap v434: 148 pgs: 148 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 745 op/s 2024-09-11T07:36:19.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:19 smithi022 ceph-mon[37330]: osdmap e229: 8 total, 8 up, 8 in 2024-09-11T07:36:19.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:19 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2425786234' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T07:36:19.706 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:19.741 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:19.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:19.742 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:19.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:36:19 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:36:19.843 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:19.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:19.889 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:19.890 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:19.990 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:20.032 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:20.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:20.033 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:20.137 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:20.185 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:20.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:20.186 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:20.287 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:20.317 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:20.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:20.318 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:20.373 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:20 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2425786234' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-11T07:36:20.374 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:20 smithi022 ceph-mon[29462]: osdmap e230: 8 total, 8 up, 8 in 2024-09-11T07:36:20.374 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:20 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2425786234' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-11T07:36:20.374 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:20 smithi022 ceph-mon[37330]: osdmap e230: 8 total, 8 up, 8 in 2024-09-11T07:36:20.419 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:20.461 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:20.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:20.462 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:20.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:20 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2425786234' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-11T07:36:20.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:20 smithi104 ceph-mon[33507]: osdmap e230: 8 total, 8 up, 8 in 2024-09-11T07:36:20.563 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:20.609 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:20.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:20.612 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:20.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:36:20 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:36:20] "GET /metrics HTTP/1.1" 200 224188 "" "Prometheus/2.43.0" 2024-09-11T07:36:20.713 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:20.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:20.740 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:20.740 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:20.841 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:20.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:20.875 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:20.875 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:20.975 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:20.998 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight 1 1 2024-09-11T07:36:21.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:21.017 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:21.017 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:21.118 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:21.147 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:21.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:21.148 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:21.249 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:21.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:21.298 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:21.298 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:21.399 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:21.454 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:21.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:21.455 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:21.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:21 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:21.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:21 smithi022 ceph-mon[29462]: pgmap v437: 148 pgs: 148 active+clean; 3.1 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.65k op/s 2024-09-11T07:36:21.504 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:21 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:21.504 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:21 smithi022 ceph-mon[37330]: pgmap v437: 148 pgs: 148 active+clean; 3.1 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.65k op/s 2024-09-11T07:36:21.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:21 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:21.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:21 smithi104 ceph-mon[33507]: pgmap v437: 148 pgs: 148 active+clean; 3.1 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.65k op/s 2024-09-11T07:36:21.556 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:21.587 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:21.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:21.588 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:21.659 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:36:21.659 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:36:21.659 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.659 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.25072.0:209499 8.8 e230/177 8:1a589377:::benchmark_data_smithi022_108406_object80968:head v 230'12903, mlcod=230'12901)", 2024-09-11T07:36:21.659 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:36:21.650592+0000", 2024-09-11T07:36:21.659 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.005315111, 2024-09-11T07:36:21.659 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0053419390000000004, 2024-09-11T07:36:21.659 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:21.659 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:36:21.659 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:21.659 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.659 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:21.660 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.650592+0000", 2024-09-11T07:36:21.660 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:21.660 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.660 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.660 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:21.660 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.650592+0000", 2024-09-11T07:36:21.660 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:21.660 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.660 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.660 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:21.661 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.650590+0000", 2024-09-11T07:36:21.661 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:36:21.661 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.661 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.661 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:21.661 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.650599+0000", 2024-09-11T07:36:21.661 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.3579999999999992e-06 2024-09-11T07:36:21.661 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.661 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.661 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:21.661 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.650615+0000", 2024-09-11T07:36:21.661 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5481999999999998e-05 2024-09-11T07:36:21.662 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.662 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.662 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:21.662 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.650621+0000", 2024-09-11T07:36:21.662 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.6490000000000002e-06 2024-09-11T07:36:21.662 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.662 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.662 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:21.662 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.650868+0000", 2024-09-11T07:36:21.662 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00024636800000000001 2024-09-11T07:36:21.662 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.662 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.662 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:21.663 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.650884+0000", 2024-09-11T07:36:21.663 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.6671999999999999e-05 2024-09-11T07:36:21.663 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:21.663 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:21.663 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:21.663 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.663 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.663 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.25072.0:209506 8.8 e230/177 8:17a30268:::benchmark_data_smithi022_108406_object80975:head v 230'12904, mlcod=230'12901)", 2024-09-11T07:36:21.663 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:36:21.655776+0000", 2024-09-11T07:36:21.663 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.00013098600000000001, 2024-09-11T07:36:21.663 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000232074, 2024-09-11T07:36:21.663 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:21.663 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:36:21.664 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:21.664 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.664 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:21.664 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.655776+0000", 2024-09-11T07:36:21.664 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:21.664 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.664 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.664 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:21.664 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.655776+0000", 2024-09-11T07:36:21.664 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:21.664 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.664 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.664 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:21.664 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.655775+0000", 2024-09-11T07:36:21.665 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:36:21.665 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.665 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.665 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:21.665 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.655782+0000", 2024-09-11T07:36:21.665 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.3649999999999998e-06 2024-09-11T07:36:21.665 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.665 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.665 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:21.665 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.655796+0000", 2024-09-11T07:36:21.665 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4073999999999999e-05 2024-09-11T07:36:21.665 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.665 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.666 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:21.666 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.655803+0000", 2024-09-11T07:36:21.666 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5860000000000001e-06 2024-09-11T07:36:21.666 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.666 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.666 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:21.666 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.655863+0000", 2024-09-11T07:36:21.666 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0102000000000002e-05 2024-09-11T07:36:21.666 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.666 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.666 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:21.666 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.655885+0000", 2024-09-11T07:36:21.666 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2152999999999999e-05 2024-09-11T07:36:21.666 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:21.666 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:21.667 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:21.667 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.667 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.667 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.25072.0:209505 8.7 e230/177 8:e62a7d56:::benchmark_data_smithi022_108406_object80974:head v 230'13005, mlcod=230'13003)", 2024-09-11T07:36:21.667 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:36:21.655853+0000", 2024-09-11T07:36:21.667 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 5.3522e-05, 2024-09-11T07:36:21.667 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00022122, 2024-09-11T07:36:21.667 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:21.667 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:36:21.667 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:21.667 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.667 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:21.667 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.655853+0000", 2024-09-11T07:36:21.667 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:21.667 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.668 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.668 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:21.668 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.655853+0000", 2024-09-11T07:36:21.668 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:21.668 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.668 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.668 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:21.668 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.655852+0000", 2024-09-11T07:36:21.668 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:36:21.668 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.668 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.668 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:21.668 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.655859+0000", 2024-09-11T07:36:21.668 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.348e-06 2024-09-11T07:36:21.669 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.669 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.672 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:21.672 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.655865+0000", 2024-09-11T07:36:21.672 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.878e-06 2024-09-11T07:36:21.672 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.672 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.672 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:21.672 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.655871+0000", 2024-09-11T07:36:21.673 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9689999999999999e-06 2024-09-11T07:36:21.673 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.673 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.673 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:21.673 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.655932+0000", 2024-09-11T07:36:21.673 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0943999999999998e-05 2024-09-11T07:36:21.673 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.673 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.673 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:21.673 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.655949+0000", 2024-09-11T07:36:21.673 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.7016000000000001e-05 2024-09-11T07:36:21.673 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:21.673 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:21.673 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:21.674 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.674 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.674 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:209469 8.b 8:dc4bd1cf:::benchmark_data_smithi022_108406_object80938:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e230)", 2024-09-11T07:36:21.674 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:36:21.618510+0000", 2024-09-11T07:36:21.674 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.037396236999999999, 2024-09-11T07:36:21.674 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.037632789, 2024-09-11T07:36:21.674 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:21.674 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "waiting for sub ops", 2024-09-11T07:36:21.674 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:21.674 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:36:21.674 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:36:21.674 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 209469 2024-09-11T07:36:21.674 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.675 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:21.675 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.675 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:21.675 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.618510+0000", 2024-09-11T07:36:21.675 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:21.675 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.675 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.675 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:21.675 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.618510+0000", 2024-09-11T07:36:21.675 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:21.675 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.675 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.675 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:21.675 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.618508+0000", 2024-09-11T07:36:21.676 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:36:21.676 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.676 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.676 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:21.676 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.618515+0000", 2024-09-11T07:36:21.676 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.985e-06 2024-09-11T07:36:21.676 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.676 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.676 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:21.676 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.618533+0000", 2024-09-11T07:36:21.676 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.7560000000000001e-05 2024-09-11T07:36:21.676 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.676 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.677 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:21.677 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.618540+0000", 2024-09-11T07:36:21.677 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.8680000000000001e-06 2024-09-11T07:36:21.677 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.677 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.677 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:21.677 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.635089+0000", 2024-09-11T07:36:21.677 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.016548625000000001 2024-09-11T07:36:21.677 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.677 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.677 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:21.677 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.635379+0000", 2024-09-11T07:36:21.677 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00029074199999999998 2024-09-11T07:36:21.678 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.678 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.678 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,4", 2024-09-11T07:36:21.678 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.635927+0000", 2024-09-11T07:36:21.678 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00054763399999999997 2024-09-11T07:36:21.678 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.678 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.678 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:36:21.678 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.644790+0000", 2024-09-11T07:36:21.678 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0088635239999999994 2024-09-11T07:36:21.678 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.678 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.678 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:21.678 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.644829+0000", 2024-09-11T07:36:21.679 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8698000000000001e-05 2024-09-11T07:36:21.679 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:21.679 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:21.679 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:21.679 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.679 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.679 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:209495 8.b 8:de733ca5:::benchmark_data_smithi022_108406_object80964:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e230)", 2024-09-11T07:36:21.679 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:36:21.645336+0000", 2024-09-11T07:36:21.679 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.010571041, 2024-09-11T07:36:21.679 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.010903161999999999, 2024-09-11T07:36:21.679 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:21.679 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "waiting for sub ops", 2024-09-11T07:36:21.679 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:21.679 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:36:21.680 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:36:21.680 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 209495 2024-09-11T07:36:21.680 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.680 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:21.680 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.680 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:21.680 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.645336+0000", 2024-09-11T07:36:21.680 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:21.680 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.680 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.680 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:21.680 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.645336+0000", 2024-09-11T07:36:21.680 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:21.681 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.681 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.681 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:21.681 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.645335+0000", 2024-09-11T07:36:21.681 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:36:21.681 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.681 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.681 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:21.681 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.645338+0000", 2024-09-11T07:36:21.681 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.027e-06 2024-09-11T07:36:21.681 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.681 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.681 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:21.681 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.645343+0000", 2024-09-11T07:36:21.682 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9840000000000001e-06 2024-09-11T07:36:21.682 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.682 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.682 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:21.682 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.645347+0000", 2024-09-11T07:36:21.682 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.512e-06 2024-09-11T07:36:21.682 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.682 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.682 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:21.682 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.645385+0000", 2024-09-11T07:36:21.682 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8659999999999999e-05 2024-09-11T07:36:21.682 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.682 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.683 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:21.683 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.645490+0000", 2024-09-11T07:36:21.683 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000104518 2024-09-11T07:36:21.683 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.683 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.683 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,4", 2024-09-11T07:36:21.683 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.645588+0000", 2024-09-11T07:36:21.683 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.8684000000000006e-05 2024-09-11T07:36:21.683 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.683 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.683 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:36:21.683 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.648982+0000", 2024-09-11T07:36:21.683 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0033936639999999998 2024-09-11T07:36:21.683 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:21.684 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:21.684 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:21.684 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:36:21.651276+0000", 2024-09-11T07:36:21.684 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0022941709999999998 2024-09-11T07:36:21.684 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:21.684 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:21.684 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:21.684 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:21.684 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:36:21.684 INFO:teuthology.orchestra.run.smithi022.stdout: "num_ops": 5 2024-09-11T07:36:21.684 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:36:21.689 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:21.714 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:21.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:21.715 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:21.816 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:21.854 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:21.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:21.856 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:21.957 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:21.999 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:22.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:22.000 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:22.101 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:22.144 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:22.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:22.145 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:22.246 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:22.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:22.289 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:22.290 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:22.390 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:22.460 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:22.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:22.461 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:22.561 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:22.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:22.603 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:22.603 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:22.704 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:22.742 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:22.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:22.743 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:22.844 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:22.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:22.878 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:22.878 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:22.979 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:23.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:23.014 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:23.014 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:23.115 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:23.143 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:23.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:23.144 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:23.245 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:23.288 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:23.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:23.289 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:23.390 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:23.414 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:23.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:23.415 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:23.516 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:23.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:23 smithi104 ceph-mon[33507]: pgmap v438: 148 pgs: 148 active+clean; 3.1 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 737 B/s rd, 0 B/s wr, 866 op/s 2024-09-11T07:36:23.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:36:23.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:36:23.541 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:23.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:23.542 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:23.643 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:23 smithi022 ceph-mon[29462]: pgmap v438: 148 pgs: 148 active+clean; 3.1 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 737 B/s rd, 0 B/s wr, 866 op/s 2024-09-11T07:36:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:36:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:36:23.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:23 smithi022 ceph-mon[37330]: pgmap v438: 148 pgs: 148 active+clean; 3.1 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 737 B/s rd, 0 B/s wr, 866 op/s 2024-09-11T07:36:23.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:36:23.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:36:23.675 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:23.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:23.676 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:23.777 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:23.813 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:23.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:23.814 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:23.915 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:23.957 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:23.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:23.958 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:24.059 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:24.085 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:24.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:24.086 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:24.187 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:24.227 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:24.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:24.228 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:24.329 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:24.367 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:24.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:24.368 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:24.431 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:36:24 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:36:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:36:24.469 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:24.512 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:24.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:24.513 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:24.614 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:24.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:24.643 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:24.643 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:24.654 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-11T07:36:24.743 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:24.791 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:24.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:24.792 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:24.893 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:24.922 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:24.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:24.923 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:25.024 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:25.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:25.071 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:25.071 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:25.172 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:25.197 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:25.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:25.198 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:25.299 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:25.361 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:25 smithi022 ceph-mon[29462]: pgmap v439: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-11T07:36:25.362 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:25 smithi022 ceph-mon[37330]: pgmap v439: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-11T07:36:25.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:25.363 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:25.363 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:25.464 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:25.491 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:25.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:25.492 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:25.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:25 smithi104 ceph-mon[33507]: pgmap v439: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-11T07:36:25.593 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:25.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:25.648 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:25.648 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:25.749 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:25.785 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:25.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:25.786 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:25.887 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:25.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:25.919 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:25.919 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:26.020 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:26.058 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:26.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:26.059 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:26.074 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:36:26.075 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:36:26.075 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:36:26.075 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:36:26.075 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:36:26.075 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:36:26.075 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:36:26.075 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:36:26.075 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:36:26.075 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:36:26.083 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:36:26.105 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:36:26.105 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:36:26.146 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:36:26.146 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:36:26.160 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:26.179 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:36:26.179 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:36:26.220 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:26.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:26.222 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:26.322 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:26.352 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:26.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:26.353 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:26.454 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:26.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:26.498 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:26.498 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:26.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:26 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:36:26.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:26 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:36:26.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:26 smithi104 ceph-mon[33507]: osdmap e231: 8 total, 8 up, 8 in 2024-09-11T07:36:26.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:26 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/658285424' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:36:26.599 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:26.626 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:26.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:26.627 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:26.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:26 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:36:26.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:26 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:36:26.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:26 smithi022 ceph-mon[29462]: osdmap e231: 8 total, 8 up, 8 in 2024-09-11T07:36:26.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:26 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/658285424' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:36:26.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:26 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:36:26.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:26 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:36:26.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:26 smithi022 ceph-mon[37330]: osdmap e231: 8 total, 8 up, 8 in 2024-09-11T07:36:26.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:26 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/658285424' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:36:26.728 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:26.766 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:26.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:26.768 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:26.869 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:26.902 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:26.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:26.903 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:27.004 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:27.036 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:27.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:27.037 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:27.138 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:27.177 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:27.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:27.178 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:27.279 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:27.318 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:27.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:27.319 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:27.397 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:36:27.420 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:27.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:27.443 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:27.444 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:27.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:27 smithi104 ceph-mon[33507]: pgmap v441: 148 pgs: 148 active+clean; 2.7 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.54k op/s 2024-09-11T07:36:27.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:27 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1983002589' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:36:27.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:27 smithi104 ceph-mon[33507]: from='client.25276 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:36:27.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:27 smithi022 ceph-mon[29462]: pgmap v441: 148 pgs: 148 active+clean; 2.7 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.54k op/s 2024-09-11T07:36:27.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:27 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1983002589' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:36:27.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:27 smithi022 ceph-mon[29462]: from='client.25276 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:36:27.533 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:27 smithi022 ceph-mon[37330]: pgmap v441: 148 pgs: 148 active+clean; 2.7 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.54k op/s 2024-09-11T07:36:27.534 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:27 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1983002589' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:36:27.534 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:27 smithi022 ceph-mon[37330]: from='client.25276 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:36:27.544 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:27.575 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:27.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:27.576 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:27.593 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is unset 2024-09-11T07:36:27.677 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:27.711 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:27.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:27.713 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:27.813 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:27.854 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:27.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:27.855 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:27.956 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:27.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:27.983 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:27.984 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:28.084 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:28.121 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:28.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:28.122 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:28.223 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:28.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:28.251 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:28.251 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:28.352 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:28.394 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:28.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:28.395 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:28.496 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:28.526 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:28.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:28.528 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:28.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:28 smithi104 ceph-mon[33507]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:36:28.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:28 smithi104 ceph-mon[33507]: from='client.25276 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:36:28.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:28 smithi104 ceph-mon[33507]: osdmap e232: 8 total, 8 up, 8 in 2024-09-11T07:36:28.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:28 smithi104 ceph-mon[33507]: 4.1a scrub starts 2024-09-11T07:36:28.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:28 smithi104 ceph-mon[33507]: 4.1a scrub ok 2024-09-11T07:36:28.628 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:28.661 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:28.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:28.662 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:28.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:28 smithi022 ceph-mon[29462]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:36:28.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:28 smithi022 ceph-mon[29462]: from='client.25276 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:36:28.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:28 smithi022 ceph-mon[29462]: osdmap e232: 8 total, 8 up, 8 in 2024-09-11T07:36:28.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:28 smithi022 ceph-mon[29462]: 4.1a scrub starts 2024-09-11T07:36:28.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:28 smithi022 ceph-mon[29462]: 4.1a scrub ok 2024-09-11T07:36:28.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:28 smithi022 ceph-mon[37330]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:36:28.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:28 smithi022 ceph-mon[37330]: from='client.25276 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:36:28.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:28 smithi022 ceph-mon[37330]: osdmap e232: 8 total, 8 up, 8 in 2024-09-11T07:36:28.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:28 smithi022 ceph-mon[37330]: 4.1a scrub starts 2024-09-11T07:36:28.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:28 smithi022 ceph-mon[37330]: 4.1a scrub ok 2024-09-11T07:36:28.763 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:28.800 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:28.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:28.802 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:28.903 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:28.941 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:28.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:28.942 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:29.044 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:29.075 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:29.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:29.076 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:29.177 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:29.214 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:29.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:29.215 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:29.316 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:29.359 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:29.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:29.360 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:29.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:29 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:36:29.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:29 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:36:29.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:29 smithi022 ceph-mon[29462]: 2.13 scrub starts 2024-09-11T07:36:29.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:29 smithi022 ceph-mon[29462]: 2.13 scrub ok 2024-09-11T07:36:29.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:29 smithi022 ceph-mon[29462]: pgmap v443: 148 pgs: 148 active+clean; 2.7 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 773 op/s 2024-09-11T07:36:29.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:29 smithi022 ceph-mon[29462]: osdmap e233: 8 total, 8 up, 8 in 2024-09-11T07:36:29.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:29 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:36:29.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:29 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:36:29.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:29 smithi022 ceph-mon[37330]: 2.13 scrub starts 2024-09-11T07:36:29.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:29 smithi022 ceph-mon[37330]: 2.13 scrub ok 2024-09-11T07:36:29.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:29 smithi022 ceph-mon[37330]: pgmap v443: 148 pgs: 148 active+clean; 2.7 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 773 op/s 2024-09-11T07:36:29.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:29 smithi022 ceph-mon[37330]: osdmap e233: 8 total, 8 up, 8 in 2024-09-11T07:36:29.435 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:36:29.461 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:29.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:36:29 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:36:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:29 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:36:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:29 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:36:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:29 smithi104 ceph-mon[33507]: 2.13 scrub starts 2024-09-11T07:36:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:29 smithi104 ceph-mon[33507]: 2.13 scrub ok 2024-09-11T07:36:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:29 smithi104 ceph-mon[33507]: pgmap v443: 148 pgs: 148 active+clean; 2.7 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 773 op/s 2024-09-11T07:36:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:29 smithi104 ceph-mon[33507]: osdmap e233: 8 total, 8 up, 8 in 2024-09-11T07:36:29.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:29.531 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:29.531 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:29.632 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:29.658 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:29.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:29.660 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:29.760 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:29.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:29.811 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:29.811 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:29.912 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:29.953 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:29.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:29.954 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:30.055 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:30.081 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:30.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:30.082 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:30.183 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:30.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:30.221 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:30.222 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:30.322 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:30.360 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:30.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:30.362 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:30.462 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:30.499 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:30.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:30.500 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:30.601 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:30.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:30.633 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:30.634 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:30.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:36:30 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:36:30] "GET /metrics HTTP/1.1" 200 224231 "" "Prometheus/2.43.0" 2024-09-11T07:36:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:30 smithi022 ceph-mon[29462]: 4.19 scrub starts 2024-09-11T07:36:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:30 smithi022 ceph-mon[29462]: 4.19 scrub ok 2024-09-11T07:36:30.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:30 smithi022 ceph-mon[37330]: 4.19 scrub starts 2024-09-11T07:36:30.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:30 smithi022 ceph-mon[37330]: 4.19 scrub ok 2024-09-11T07:36:30.735 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:30.766 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:30.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:30.767 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:30.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:30 smithi104 ceph-mon[33507]: 4.19 scrub starts 2024-09-11T07:36:30.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:30 smithi104 ceph-mon[33507]: 4.19 scrub ok 2024-09-11T07:36:30.868 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:30.914 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:30.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:30.915 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:31.016 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:31.043 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:31.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:31.044 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:31.145 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:31.170 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:31.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:31.171 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:31.272 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:31.284 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:31 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:31.284 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:31 smithi022 ceph-mon[37330]: pgmap v445: 148 pgs: 148 active+clean; 2.3 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.66k op/s 2024-09-11T07:36:31.284 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:31 smithi022 ceph-mon[37330]: 5.1 scrub starts 2024-09-11T07:36:31.284 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:31 smithi022 ceph-mon[37330]: 5.1 scrub ok 2024-09-11T07:36:31.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:31.320 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:31.320 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:31.422 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:31.459 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:31.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:31.460 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:31.561 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:31.603 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:31.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:31.604 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:31.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:31 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:31.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:31 smithi022 ceph-mon[29462]: pgmap v445: 148 pgs: 148 active+clean; 2.3 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.66k op/s 2024-09-11T07:36:31.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:31 smithi022 ceph-mon[29462]: 5.1 scrub starts 2024-09-11T07:36:31.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:31 smithi022 ceph-mon[29462]: 5.1 scrub ok 2024-09-11T07:36:31.705 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:31.754 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:31.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:31.755 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:31.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:31 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:31.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:31 smithi104 ceph-mon[33507]: pgmap v445: 148 pgs: 148 active+clean; 2.3 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.66k op/s 2024-09-11T07:36:31.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:31 smithi104 ceph-mon[33507]: 5.1 scrub starts 2024-09-11T07:36:31.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:31 smithi104 ceph-mon[33507]: 5.1 scrub ok 2024-09-11T07:36:31.782 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:36:31.857 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:31.901 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:31.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:31.902 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:32.003 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:32.053 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:32.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:32.054 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:32.154 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:32.186 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:32.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:32.187 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:32.288 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:32.317 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:32 smithi022 ceph-mon[29462]: 2.1f scrub starts 2024-09-11T07:36:32.317 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:32 smithi022 ceph-mon[29462]: 2.1f scrub ok 2024-09-11T07:36:32.318 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:32 smithi022 ceph-mon[29462]: 2.18 scrub starts 2024-09-11T07:36:32.318 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:32 smithi022 ceph-mon[29462]: 2.18 scrub ok 2024-09-11T07:36:32.318 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:32 smithi022 ceph-mon[29462]: 6.8 scrub starts 2024-09-11T07:36:32.318 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:32 smithi022 ceph-mon[29462]: 6.8 scrub ok 2024-09-11T07:36:32.318 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:32 smithi022 ceph-mon[29462]: 4.13 scrub starts 2024-09-11T07:36:32.318 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:32 smithi022 ceph-mon[29462]: 4.13 scrub ok 2024-09-11T07:36:32.318 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:32 smithi022 ceph-mon[37330]: 2.1f scrub starts 2024-09-11T07:36:32.318 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:32 smithi022 ceph-mon[37330]: 2.1f scrub ok 2024-09-11T07:36:32.318 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:32 smithi022 ceph-mon[37330]: 2.18 scrub starts 2024-09-11T07:36:32.319 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:32 smithi022 ceph-mon[37330]: 2.18 scrub ok 2024-09-11T07:36:32.319 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:32 smithi022 ceph-mon[37330]: 6.8 scrub starts 2024-09-11T07:36:32.319 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:32 smithi022 ceph-mon[37330]: 6.8 scrub ok 2024-09-11T07:36:32.319 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:32 smithi022 ceph-mon[37330]: 4.13 scrub starts 2024-09-11T07:36:32.319 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:32 smithi022 ceph-mon[37330]: 4.13 scrub ok 2024-09-11T07:36:32.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:32.331 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:32.332 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:32.432 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:32.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:32.461 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:32.461 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:32.562 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:32.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:32.604 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:32.605 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:32.705 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:32.733 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:32.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:32.734 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:32.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:32 smithi104 ceph-mon[33507]: 2.1f scrub starts 2024-09-11T07:36:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:32 smithi104 ceph-mon[33507]: 2.1f scrub ok 2024-09-11T07:36:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:32 smithi104 ceph-mon[33507]: 2.18 scrub starts 2024-09-11T07:36:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:32 smithi104 ceph-mon[33507]: 2.18 scrub ok 2024-09-11T07:36:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:32 smithi104 ceph-mon[33507]: 6.8 scrub starts 2024-09-11T07:36:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:32 smithi104 ceph-mon[33507]: 6.8 scrub ok 2024-09-11T07:36:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:32 smithi104 ceph-mon[33507]: 4.13 scrub starts 2024-09-11T07:36:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:32 smithi104 ceph-mon[33507]: 4.13 scrub ok 2024-09-11T07:36:32.835 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:32.871 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:32.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:32.872 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:32.973 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:33.015 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:33.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:33.016 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:33.041 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T07:36:33.059 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:36:33.060 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:36:33.117 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:33.155 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:33.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:33.157 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:33.257 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:33.284 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:33.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:33.285 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:33.307 INFO:teuthology.orchestra.run.smithi022.stderr:reweighted osd.1 to 1 (10000) 2024-09-11T07:36:33.386 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:33.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:33.425 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:33.426 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:33.526 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:33.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:33.558 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:33.558 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:33.659 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:33.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:33 smithi022 ceph-mon[29462]: 6.18 scrub starts 2024-09-11T07:36:33.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:33 smithi022 ceph-mon[29462]: 6.18 scrub ok 2024-09-11T07:36:33.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:33 smithi022 ceph-mon[29462]: pgmap v446: 148 pgs: 148 active+clean; 2.3 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 743 B/s rd, 0 B/s wr, 877 op/s 2024-09-11T07:36:33.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:33 smithi022 ceph-mon[29462]: 4.8 scrub starts 2024-09-11T07:36:33.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:33 smithi022 ceph-mon[29462]: 4.8 scrub ok 2024-09-11T07:36:33.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:33 smithi022 ceph-mon[29462]: 6.3 scrub starts 2024-09-11T07:36:33.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:33 smithi022 ceph-mon[29462]: 6.3 scrub ok 2024-09-11T07:36:33.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:33 smithi022 ceph-mon[29462]: 5.17 scrub starts 2024-09-11T07:36:33.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:33 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1981838617' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T07:36:33.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:33 smithi022 ceph-mon[37330]: 6.18 scrub starts 2024-09-11T07:36:33.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:33 smithi022 ceph-mon[37330]: 6.18 scrub ok 2024-09-11T07:36:33.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:33 smithi022 ceph-mon[37330]: pgmap v446: 148 pgs: 148 active+clean; 2.3 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 743 B/s rd, 0 B/s wr, 877 op/s 2024-09-11T07:36:33.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:33 smithi022 ceph-mon[37330]: 4.8 scrub starts 2024-09-11T07:36:33.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:33 smithi022 ceph-mon[37330]: 4.8 scrub ok 2024-09-11T07:36:33.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:33 smithi022 ceph-mon[37330]: 6.3 scrub starts 2024-09-11T07:36:33.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:33 smithi022 ceph-mon[37330]: 6.3 scrub ok 2024-09-11T07:36:33.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:33 smithi022 ceph-mon[37330]: 5.17 scrub starts 2024-09-11T07:36:33.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:33 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1981838617' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T07:36:33.690 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:33.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:33.691 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:33.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:33 smithi104 ceph-mon[33507]: 6.18 scrub starts 2024-09-11T07:36:33.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:33 smithi104 ceph-mon[33507]: 6.18 scrub ok 2024-09-11T07:36:33.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:33 smithi104 ceph-mon[33507]: pgmap v446: 148 pgs: 148 active+clean; 2.3 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 743 B/s rd, 0 B/s wr, 877 op/s 2024-09-11T07:36:33.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:33 smithi104 ceph-mon[33507]: 4.8 scrub starts 2024-09-11T07:36:33.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:33 smithi104 ceph-mon[33507]: 4.8 scrub ok 2024-09-11T07:36:33.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:33 smithi104 ceph-mon[33507]: 6.3 scrub starts 2024-09-11T07:36:33.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:33 smithi104 ceph-mon[33507]: 6.3 scrub ok 2024-09-11T07:36:33.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:33 smithi104 ceph-mon[33507]: 5.17 scrub starts 2024-09-11T07:36:33.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:33 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1981838617' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T07:36:33.792 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:33.831 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:33.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:33.832 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:33.933 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:33.979 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:33.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:33.980 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:34.081 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:34.123 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:34.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:34.124 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:34.224 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:34.259 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:34.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:34.260 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:34.303 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:36:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:36:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:36:34.361 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:34.398 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:34.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:34.399 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:34.500 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:34.539 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:34.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:34.540 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:34.641 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:34.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:34 smithi022 ceph-mon[29462]: 5.17 scrub ok 2024-09-11T07:36:34.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:34 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1981838617' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-11T07:36:34.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:34 smithi022 ceph-mon[29462]: osdmap e234: 8 total, 8 up, 8 in 2024-09-11T07:36:34.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:34 smithi022 ceph-mon[29462]: 5.12 scrub starts 2024-09-11T07:36:34.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:34 smithi022 ceph-mon[29462]: 5.12 scrub ok 2024-09-11T07:36:34.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:34 smithi022 ceph-mon[37330]: 5.17 scrub ok 2024-09-11T07:36:34.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:34 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1981838617' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-11T07:36:34.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:34 smithi022 ceph-mon[37330]: osdmap e234: 8 total, 8 up, 8 in 2024-09-11T07:36:34.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:34 smithi022 ceph-mon[37330]: 5.12 scrub starts 2024-09-11T07:36:34.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:34 smithi022 ceph-mon[37330]: 5.12 scrub ok 2024-09-11T07:36:34.669 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:34.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:34.670 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:34.770 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:34.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:34 smithi104 ceph-mon[33507]: 5.17 scrub ok 2024-09-11T07:36:34.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:34 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1981838617' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-11T07:36:34.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:34 smithi104 ceph-mon[33507]: osdmap e234: 8 total, 8 up, 8 in 2024-09-11T07:36:34.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:34 smithi104 ceph-mon[33507]: 5.12 scrub starts 2024-09-11T07:36:34.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:34 smithi104 ceph-mon[33507]: 5.12 scrub ok 2024-09-11T07:36:34.799 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:34.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:34.800 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:34.901 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:34.929 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:34.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:34.930 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:35.031 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:35.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:35.066 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:35.066 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:35.167 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:35.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:35.195 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:35.196 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:35.297 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:35.340 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:35.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:35.341 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:35.442 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:35.472 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:35.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:35.473 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:35.574 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:35.606 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:35.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:35.607 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:35.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:35 smithi022 ceph-mon[29462]: 2.1e scrub starts 2024-09-11T07:36:35.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:35 smithi022 ceph-mon[29462]: 2.1e scrub ok 2024-09-11T07:36:35.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:35 smithi022 ceph-mon[29462]: pgmap v448: 148 pgs: 148 active+clean; 2.2 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 779 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T07:36:35.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:35 smithi022 ceph-mon[29462]: 2.12 scrub starts 2024-09-11T07:36:35.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:35 smithi022 ceph-mon[29462]: 2.12 scrub ok 2024-09-11T07:36:35.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:35 smithi022 ceph-mon[37330]: 2.1e scrub starts 2024-09-11T07:36:35.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:35 smithi022 ceph-mon[37330]: 2.1e scrub ok 2024-09-11T07:36:35.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:35 smithi022 ceph-mon[37330]: pgmap v448: 148 pgs: 148 active+clean; 2.2 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 779 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T07:36:35.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:35 smithi022 ceph-mon[37330]: 2.12 scrub starts 2024-09-11T07:36:35.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:35 smithi022 ceph-mon[37330]: 2.12 scrub ok 2024-09-11T07:36:35.707 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:35.739 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:35.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:35.740 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:35.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:35 smithi104 ceph-mon[33507]: 2.1e scrub starts 2024-09-11T07:36:35.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:35 smithi104 ceph-mon[33507]: 2.1e scrub ok 2024-09-11T07:36:35.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:35 smithi104 ceph-mon[33507]: pgmap v448: 148 pgs: 148 active+clean; 2.2 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 779 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T07:36:35.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:35 smithi104 ceph-mon[33507]: 2.12 scrub starts 2024-09-11T07:36:35.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:35 smithi104 ceph-mon[33507]: 2.12 scrub ok 2024-09-11T07:36:35.841 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:35.868 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:35.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:35.869 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:35.970 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:36.007 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:36.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:36.008 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:36.109 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:36.140 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:36.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:36.142 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:36.243 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:36.280 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:36.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:36.281 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:36.382 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:36.414 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:36.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:36.416 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:36.516 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:36.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:36.551 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:36.551 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:36.652 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:36.662 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight 2 1 2024-09-11T07:36:36.689 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:36.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:36.692 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:36.793 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:36.821 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:36.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:36.822 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:36.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:36 smithi022 ceph-mon[29462]: 6.14 scrub starts 2024-09-11T07:36:36.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:36 smithi022 ceph-mon[29462]: 6.14 scrub ok 2024-09-11T07:36:36.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:36 smithi022 ceph-mon[29462]: 2.15 scrub starts 2024-09-11T07:36:36.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:36 smithi022 ceph-mon[29462]: 2.15 scrub ok 2024-09-11T07:36:36.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:36 smithi022 ceph-mon[29462]: 6.9 scrub starts 2024-09-11T07:36:36.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:36 smithi022 ceph-mon[29462]: 6.9 scrub ok 2024-09-11T07:36:36.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:36 smithi022 ceph-mon[29462]: pgmap v449: 148 pgs: 148 active+clean; 2.0 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.50k op/s 2024-09-11T07:36:36.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:36 smithi022 ceph-mon[37330]: 6.14 scrub starts 2024-09-11T07:36:36.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:36 smithi022 ceph-mon[37330]: 6.14 scrub ok 2024-09-11T07:36:36.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:36 smithi022 ceph-mon[37330]: 2.15 scrub starts 2024-09-11T07:36:36.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:36 smithi022 ceph-mon[37330]: 2.15 scrub ok 2024-09-11T07:36:36.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:36 smithi022 ceph-mon[37330]: 6.9 scrub starts 2024-09-11T07:36:36.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:36 smithi022 ceph-mon[37330]: 6.9 scrub ok 2024-09-11T07:36:36.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:36 smithi022 ceph-mon[37330]: pgmap v449: 148 pgs: 148 active+clean; 2.0 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.50k op/s 2024-09-11T07:36:36.922 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:36.949 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:36.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:36.950 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:37.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:36 smithi104 ceph-mon[33507]: 6.14 scrub starts 2024-09-11T07:36:37.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:36 smithi104 ceph-mon[33507]: 6.14 scrub ok 2024-09-11T07:36:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:36 smithi104 ceph-mon[33507]: 2.15 scrub starts 2024-09-11T07:36:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:36 smithi104 ceph-mon[33507]: 2.15 scrub ok 2024-09-11T07:36:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:36 smithi104 ceph-mon[33507]: 6.9 scrub starts 2024-09-11T07:36:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:36 smithi104 ceph-mon[33507]: 6.9 scrub ok 2024-09-11T07:36:37.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:36 smithi104 ceph-mon[33507]: pgmap v449: 148 pgs: 148 active+clean; 2.0 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.50k op/s 2024-09-11T07:36:37.051 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:37.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:37.092 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:37.092 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:37.193 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:37.233 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:37.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:37.235 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:37.336 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:37.369 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:37.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:37.370 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:37.471 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:37.494 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:37.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:37.495 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:37.596 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:37.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:37.624 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:37.624 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:37.725 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:37.767 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:37.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:37.768 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:37.869 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:37.899 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:37.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:37.901 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:37.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:37 smithi022 ceph-mon[29462]: 5.d scrub starts 2024-09-11T07:36:37.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:37 smithi022 ceph-mon[29462]: 5.d scrub ok 2024-09-11T07:36:37.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:37 smithi022 ceph-mon[29462]: 5.10 scrub starts 2024-09-11T07:36:37.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:37 smithi022 ceph-mon[29462]: 5.10 scrub ok 2024-09-11T07:36:37.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:37 smithi022 ceph-mon[37330]: 5.d scrub starts 2024-09-11T07:36:37.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:37 smithi022 ceph-mon[37330]: 5.d scrub ok 2024-09-11T07:36:37.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:37 smithi022 ceph-mon[37330]: 5.10 scrub starts 2024-09-11T07:36:37.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:37 smithi022 ceph-mon[37330]: 5.10 scrub ok 2024-09-11T07:36:38.002 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:38.026 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:38.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:38.029 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:38.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:37 smithi104 ceph-mon[33507]: 5.d scrub starts 2024-09-11T07:36:38.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:37 smithi104 ceph-mon[33507]: 5.d scrub ok 2024-09-11T07:36:38.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:37 smithi104 ceph-mon[33507]: 5.10 scrub starts 2024-09-11T07:36:38.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:37 smithi104 ceph-mon[33507]: 5.10 scrub ok 2024-09-11T07:36:38.031 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-11T07:36:38.130 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:38.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:38.185 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:38.185 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:38.286 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:38.298 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:36:38.308 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:38.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:38.309 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:38.410 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:38.463 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:38.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:38.465 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:38.566 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:38.590 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:38.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:38.591 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:38.659 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:36:38.659 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:36:38.692 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:36:38.692 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:36:38.692 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:38.717 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:38.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:38.718 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:38.728 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:36:38.728 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:36:38.765 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:36:38.765 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:36:38.798 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:36:38.798 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:36:38.819 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:38.834 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:38 smithi022 ceph-mon[29462]: 2.e scrub starts 2024-09-11T07:36:38.834 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:38 smithi022 ceph-mon[29462]: 2.e scrub ok 2024-09-11T07:36:38.834 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:38 smithi022 ceph-mon[29462]: 2.8 scrub starts 2024-09-11T07:36:38.834 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:38 smithi022 ceph-mon[29462]: 2.8 scrub ok 2024-09-11T07:36:38.834 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:38 smithi022 ceph-mon[29462]: 5.1c scrub starts 2024-09-11T07:36:38.834 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:38 smithi022 ceph-mon[29462]: 5.1c scrub ok 2024-09-11T07:36:38.834 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:38 smithi022 ceph-mon[29462]: 4.2 scrub starts 2024-09-11T07:36:38.835 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:38 smithi022 ceph-mon[29462]: 4.2 scrub ok 2024-09-11T07:36:38.835 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:36:38.835 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:38 smithi022 ceph-mon[29462]: pgmap v450: 148 pgs: 148 active+clean; 2.0 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T07:36:38.835 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:38 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3556520546' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:36:38.835 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:36:38.835 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:36:38.836 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:38 smithi022 ceph-mon[37330]: 2.e scrub starts 2024-09-11T07:36:38.836 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:38 smithi022 ceph-mon[37330]: 2.e scrub ok 2024-09-11T07:36:38.836 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:38 smithi022 ceph-mon[37330]: 2.8 scrub starts 2024-09-11T07:36:38.836 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:38 smithi022 ceph-mon[37330]: 2.8 scrub ok 2024-09-11T07:36:38.836 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:38 smithi022 ceph-mon[37330]: 5.1c scrub starts 2024-09-11T07:36:38.836 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:38 smithi022 ceph-mon[37330]: 5.1c scrub ok 2024-09-11T07:36:38.836 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:38 smithi022 ceph-mon[37330]: 4.2 scrub starts 2024-09-11T07:36:38.836 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:38 smithi022 ceph-mon[37330]: 4.2 scrub ok 2024-09-11T07:36:38.836 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:36:38.836 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:38 smithi022 ceph-mon[37330]: pgmap v450: 148 pgs: 148 active+clean; 2.0 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T07:36:38.836 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:38 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3556520546' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:36:38.862 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:36:38.862 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:36:38.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:38.867 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:38.868 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:38.894 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:36:38.894 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:36:38.968 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:38.993 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:38.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:38.994 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:39.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:38 smithi104 ceph-mon[33507]: 2.e scrub starts 2024-09-11T07:36:39.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:38 smithi104 ceph-mon[33507]: 2.e scrub ok 2024-09-11T07:36:39.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:38 smithi104 ceph-mon[33507]: 2.8 scrub starts 2024-09-11T07:36:39.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:38 smithi104 ceph-mon[33507]: 2.8 scrub ok 2024-09-11T07:36:39.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:38 smithi104 ceph-mon[33507]: 5.1c scrub starts 2024-09-11T07:36:39.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:38 smithi104 ceph-mon[33507]: 5.1c scrub ok 2024-09-11T07:36:39.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:38 smithi104 ceph-mon[33507]: 4.2 scrub starts 2024-09-11T07:36:39.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:38 smithi104 ceph-mon[33507]: 4.2 scrub ok 2024-09-11T07:36:39.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:36:39.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:38 smithi104 ceph-mon[33507]: pgmap v450: 148 pgs: 148 active+clean; 2.0 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T07:36:39.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:38 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3556520546' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:36:39.095 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:39.135 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:39.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:39.136 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:39.237 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:39.261 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:39.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:39.262 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:39.363 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:39.393 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:39.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:39.394 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:39.495 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:39.524 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:39.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:39.525 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:39.626 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:39.652 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:39.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:39.653 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:39.754 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:39.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:36:39 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:36:39.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:39 smithi104 ceph-mon[33507]: 2.4 scrub starts 2024-09-11T07:36:39.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:39 smithi104 ceph-mon[33507]: 2.4 scrub ok 2024-09-11T07:36:39.781 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:39.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:39.782 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:39.883 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:39.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:39 smithi022 ceph-mon[37330]: 2.4 scrub starts 2024-09-11T07:36:39.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:39 smithi022 ceph-mon[37330]: 2.4 scrub ok 2024-09-11T07:36:39.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:39 smithi022 ceph-mon[29462]: 2.4 scrub starts 2024-09-11T07:36:39.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:39 smithi022 ceph-mon[29462]: 2.4 scrub ok 2024-09-11T07:36:39.923 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:39.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:39.924 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:40.025 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:40.060 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:36:40.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:40.079 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:40.079 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:40.180 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:40.205 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:40.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:40.206 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:40.307 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:40.346 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:40.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:40.347 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:40.448 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:40.485 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:40.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:40.487 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:40.587 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:40.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:40.619 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:40.619 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:40.641 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:36:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:36:40] "GET /metrics HTTP/1.1" 200 224206 "" "Prometheus/2.43.0" 2024-09-11T07:36:40.719 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:40.768 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:36:40.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:40.772 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:40.773 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:40.873 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:40.893 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:40 smithi022 ceph-mon[29462]: 2.19 scrub starts 2024-09-11T07:36:40.893 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:40 smithi022 ceph-mon[29462]: 2.19 scrub ok 2024-09-11T07:36:40.893 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:40 smithi022 ceph-mon[29462]: 5.a scrub starts 2024-09-11T07:36:40.893 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:40 smithi022 ceph-mon[29462]: 5.a scrub ok 2024-09-11T07:36:40.893 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:40 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:40.893 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:40 smithi022 ceph-mon[29462]: 4.d scrub starts 2024-09-11T07:36:40.893 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:40 smithi022 ceph-mon[29462]: 4.d scrub ok 2024-09-11T07:36:40.894 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:40 smithi022 ceph-mon[29462]: 6.0 scrub starts 2024-09-11T07:36:40.894 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:40 smithi022 ceph-mon[29462]: 6.0 scrub ok 2024-09-11T07:36:40.894 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:40 smithi022 ceph-mon[29462]: pgmap v451: 148 pgs: 148 active+clean; 1.6 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 1.22k op/s 2024-09-11T07:36:40.894 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:40 smithi022 ceph-mon[37330]: 2.19 scrub starts 2024-09-11T07:36:40.894 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:40 smithi022 ceph-mon[37330]: 2.19 scrub ok 2024-09-11T07:36:40.894 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:40 smithi022 ceph-mon[37330]: 5.a scrub starts 2024-09-11T07:36:40.894 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:40 smithi022 ceph-mon[37330]: 5.a scrub ok 2024-09-11T07:36:40.894 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:40 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:40.894 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:40 smithi022 ceph-mon[37330]: 4.d scrub starts 2024-09-11T07:36:40.894 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:40 smithi022 ceph-mon[37330]: 4.d scrub ok 2024-09-11T07:36:40.894 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:40 smithi022 ceph-mon[37330]: 6.0 scrub starts 2024-09-11T07:36:40.895 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:40 smithi022 ceph-mon[37330]: 6.0 scrub ok 2024-09-11T07:36:40.895 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:40 smithi022 ceph-mon[37330]: pgmap v451: 148 pgs: 148 active+clean; 1.6 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 1.22k op/s 2024-09-11T07:36:40.913 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:40.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:40.914 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:41.015 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:41.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:40 smithi104 ceph-mon[33507]: 2.19 scrub starts 2024-09-11T07:36:41.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:40 smithi104 ceph-mon[33507]: 2.19 scrub ok 2024-09-11T07:36:41.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:40 smithi104 ceph-mon[33507]: 5.a scrub starts 2024-09-11T07:36:41.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:40 smithi104 ceph-mon[33507]: 5.a scrub ok 2024-09-11T07:36:41.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:40 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:41.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:40 smithi104 ceph-mon[33507]: 4.d scrub starts 2024-09-11T07:36:41.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:40 smithi104 ceph-mon[33507]: 4.d scrub ok 2024-09-11T07:36:41.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:40 smithi104 ceph-mon[33507]: 6.0 scrub starts 2024-09-11T07:36:41.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:40 smithi104 ceph-mon[33507]: 6.0 scrub ok 2024-09-11T07:36:41.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:40 smithi104 ceph-mon[33507]: pgmap v451: 148 pgs: 148 active+clean; 1.6 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 1.22k op/s 2024-09-11T07:36:41.070 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:41.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:41.071 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:41.172 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:41.203 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:41.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:41.204 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:41.305 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:41.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:41.344 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:41.345 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:41.445 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:41.488 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:41.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:41.489 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:41.590 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:41.620 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:41.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:41.621 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:41.721 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:41.746 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:41.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:41.747 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:41.769 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:36:41.811 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:41 smithi022 ceph-mon[29462]: 5.6 scrub starts 2024-09-11T07:36:41.811 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:41 smithi022 ceph-mon[29462]: 5.6 scrub ok 2024-09-11T07:36:41.811 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:41 smithi022 ceph-mon[37330]: 5.6 scrub starts 2024-09-11T07:36:41.811 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:41 smithi022 ceph-mon[37330]: 5.6 scrub ok 2024-09-11T07:36:41.848 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:41.872 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:41.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:41.873 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:41.974 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:41.997 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:41.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:41.999 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:42.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:41 smithi104 ceph-mon[33507]: 5.6 scrub starts 2024-09-11T07:36:42.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:41 smithi104 ceph-mon[33507]: 5.6 scrub ok 2024-09-11T07:36:42.100 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:42.130 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:42.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:42.131 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:42.232 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:42.261 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:42.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:42.262 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:42.363 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:42.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:42.403 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:42.403 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:42.504 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:42.544 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:42.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:42.545 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:42.646 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:42.670 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is unset 2024-09-11T07:36:42.672 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:42.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:42.673 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:42.774 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:42.807 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:42.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:42.808 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:42.909 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:42.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:42 smithi022 ceph-mon[37330]: 4.4 scrub starts 2024-09-11T07:36:42.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:42 smithi022 ceph-mon[37330]: 4.4 scrub ok 2024-09-11T07:36:42.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:42 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3341095941' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:36:42.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:42 smithi022 ceph-mon[37330]: pgmap v452: 148 pgs: 148 active+clean; 1.6 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 1.22k op/s 2024-09-11T07:36:42.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:42 smithi022 ceph-mon[29462]: 4.4 scrub starts 2024-09-11T07:36:42.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:42 smithi022 ceph-mon[29462]: 4.4 scrub ok 2024-09-11T07:36:42.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:42 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3341095941' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:36:42.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:42 smithi022 ceph-mon[29462]: pgmap v452: 148 pgs: 148 active+clean; 1.6 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 1.22k op/s 2024-09-11T07:36:42.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:42.943 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:42.943 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:43.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:42 smithi104 ceph-mon[33507]: 4.4 scrub starts 2024-09-11T07:36:43.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:42 smithi104 ceph-mon[33507]: 4.4 scrub ok 2024-09-11T07:36:43.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:42 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3341095941' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:36:43.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:42 smithi104 ceph-mon[33507]: pgmap v452: 148 pgs: 148 active+clean; 1.6 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 1.22k op/s 2024-09-11T07:36:43.044 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:43.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:43.076 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:43.077 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:43.177 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:43.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:43.265 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:43.265 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:43.366 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:43.400 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:43.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:43.401 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:43.491 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:36:43.502 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:43.526 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:43.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:43.527 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:43.628 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:43.672 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:43.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:43.674 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:43.774 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:43.800 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:43.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:43.801 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:43.820 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:43 smithi022 ceph-mon[29462]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:36:43.820 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:43 smithi022 ceph-mon[29462]: Cluster is now healthy 2024-09-11T07:36:43.820 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:43 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3341095941' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:36:43.820 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:43 smithi022 ceph-mon[29462]: osdmap e235: 8 total, 8 up, 8 in 2024-09-11T07:36:43.820 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:43 smithi022 ceph-mon[29462]: 4.18 deep-scrub starts 2024-09-11T07:36:43.820 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:43 smithi022 ceph-mon[29462]: 4.18 deep-scrub ok 2024-09-11T07:36:43.820 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:43 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:36:43.820 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:43 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:36:43.820 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:43 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:36:43.820 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:43 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:36:43.821 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:43 smithi022 ceph-mon[37330]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:36:43.821 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:43 smithi022 ceph-mon[37330]: Cluster is now healthy 2024-09-11T07:36:43.821 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:43 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3341095941' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:36:43.821 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:43 smithi022 ceph-mon[37330]: osdmap e235: 8 total, 8 up, 8 in 2024-09-11T07:36:43.821 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:43 smithi022 ceph-mon[37330]: 4.18 deep-scrub starts 2024-09-11T07:36:43.821 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:43 smithi022 ceph-mon[37330]: 4.18 deep-scrub ok 2024-09-11T07:36:43.821 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:43 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:36:43.821 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:43 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:36:43.822 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:43 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:36:43.822 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:43 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:36:43.902 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:43.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:43.941 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:43.942 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:44.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:43 smithi104 ceph-mon[33507]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:36:44.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:43 smithi104 ceph-mon[33507]: Cluster is now healthy 2024-09-11T07:36:44.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:43 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3341095941' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:36:44.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:43 smithi104 ceph-mon[33507]: osdmap e235: 8 total, 8 up, 8 in 2024-09-11T07:36:44.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:43 smithi104 ceph-mon[33507]: 4.18 deep-scrub starts 2024-09-11T07:36:44.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:43 smithi104 ceph-mon[33507]: 4.18 deep-scrub ok 2024-09-11T07:36:44.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:43 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:36:44.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:43 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:36:44.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:43 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:36:44.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:43 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:36:44.043 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:44.074 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:44.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:44.075 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:44.176 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:44.205 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:44.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:44.206 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:44.307 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:44.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:44.338 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:44.338 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:44.439 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:44.469 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:44.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:44.470 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:44.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:36:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:36:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:36:44.571 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:44.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:44.620 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:44.621 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:44.722 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:44.762 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:44.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:44.763 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:44.864 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:44.911 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:44.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:44.912 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:44.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:44 smithi022 ceph-mon[29462]: 5.8 scrub starts 2024-09-11T07:36:44.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:44 smithi022 ceph-mon[29462]: 5.8 scrub ok 2024-09-11T07:36:44.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:44 smithi022 ceph-mon[29462]: 4.c scrub starts 2024-09-11T07:36:44.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:44 smithi022 ceph-mon[29462]: 4.c scrub ok 2024-09-11T07:36:44.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:44 smithi022 ceph-mon[29462]: pgmap v454: 148 pgs: 148 active+clean; 1.5 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 31 KiB/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T07:36:44.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:44 smithi022 ceph-mon[37330]: 5.8 scrub starts 2024-09-11T07:36:44.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:44 smithi022 ceph-mon[37330]: 5.8 scrub ok 2024-09-11T07:36:44.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:44 smithi022 ceph-mon[37330]: 4.c scrub starts 2024-09-11T07:36:44.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:44 smithi022 ceph-mon[37330]: 4.c scrub ok 2024-09-11T07:36:44.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:44 smithi022 ceph-mon[37330]: pgmap v454: 148 pgs: 148 active+clean; 1.5 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 31 KiB/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T07:36:45.013 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:45.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:44 smithi104 ceph-mon[33507]: 5.8 scrub starts 2024-09-11T07:36:45.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:44 smithi104 ceph-mon[33507]: 5.8 scrub ok 2024-09-11T07:36:45.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:44 smithi104 ceph-mon[33507]: 4.c scrub starts 2024-09-11T07:36:45.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:44 smithi104 ceph-mon[33507]: 4.c scrub ok 2024-09-11T07:36:45.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:44 smithi104 ceph-mon[33507]: pgmap v454: 148 pgs: 148 active+clean; 1.5 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 31 KiB/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T07:36:45.039 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:45.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:45.040 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:45.141 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:45.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:45.206 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:45.207 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:45.307 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:45.338 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:45.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:45.339 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:45.440 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:45.484 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:45.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:45.485 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:45.585 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:45.628 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:45.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:45.629 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:45.688 INFO:teuthology.orchestra.run.smithi022.stderr:reweighted osd.2 to 1 (10000) 2024-09-11T07:36:45.730 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:45.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:45.766 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:45.766 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:45.867 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:45.908 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:45.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:45.909 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:45.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[29462]: 6.11 scrub starts 2024-09-11T07:36:45.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[29462]: 6.11 scrub ok 2024-09-11T07:36:45.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[29462]: 6.a scrub starts 2024-09-11T07:36:45.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[29462]: 6.a scrub ok 2024-09-11T07:36:45.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[29462]: 6.1c scrub starts 2024-09-11T07:36:45.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[29462]: 6.1c scrub ok 2024-09-11T07:36:45.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[29462]: 5.1d scrub starts 2024-09-11T07:36:45.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[29462]: 5.1d scrub ok 2024-09-11T07:36:45.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2922927103' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T07:36:45.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[29462]: from='client.25271 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T07:36:45.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[29462]: 2.1 scrub starts 2024-09-11T07:36:45.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[29462]: 2.1 scrub ok 2024-09-11T07:36:45.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[29462]: 4.6 scrub starts 2024-09-11T07:36:45.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[29462]: 4.6 scrub ok 2024-09-11T07:36:45.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[37330]: 6.11 scrub starts 2024-09-11T07:36:45.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[37330]: 6.11 scrub ok 2024-09-11T07:36:45.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[37330]: 6.a scrub starts 2024-09-11T07:36:45.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[37330]: 6.a scrub ok 2024-09-11T07:36:45.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[37330]: 6.1c scrub starts 2024-09-11T07:36:45.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[37330]: 6.1c scrub ok 2024-09-11T07:36:45.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[37330]: 5.1d scrub starts 2024-09-11T07:36:45.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[37330]: 5.1d scrub ok 2024-09-11T07:36:45.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2922927103' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T07:36:45.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[37330]: from='client.25271 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T07:36:45.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[37330]: 2.1 scrub starts 2024-09-11T07:36:45.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[37330]: 2.1 scrub ok 2024-09-11T07:36:45.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[37330]: 4.6 scrub starts 2024-09-11T07:36:45.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:45 smithi022 ceph-mon[37330]: 4.6 scrub ok 2024-09-11T07:36:46.010 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:46.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:45 smithi104 ceph-mon[33507]: 6.11 scrub starts 2024-09-11T07:36:46.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:45 smithi104 ceph-mon[33507]: 6.11 scrub ok 2024-09-11T07:36:46.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:45 smithi104 ceph-mon[33507]: 6.a scrub starts 2024-09-11T07:36:46.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:45 smithi104 ceph-mon[33507]: 6.a scrub ok 2024-09-11T07:36:46.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:45 smithi104 ceph-mon[33507]: 6.1c scrub starts 2024-09-11T07:36:46.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:45 smithi104 ceph-mon[33507]: 6.1c scrub ok 2024-09-11T07:36:46.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:45 smithi104 ceph-mon[33507]: 5.1d scrub starts 2024-09-11T07:36:46.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:45 smithi104 ceph-mon[33507]: 5.1d scrub ok 2024-09-11T07:36:46.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:45 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2922927103' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T07:36:46.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:45 smithi104 ceph-mon[33507]: from='client.25271 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T07:36:46.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:45 smithi104 ceph-mon[33507]: 2.1 scrub starts 2024-09-11T07:36:46.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:45 smithi104 ceph-mon[33507]: 2.1 scrub ok 2024-09-11T07:36:46.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:45 smithi104 ceph-mon[33507]: 4.6 scrub starts 2024-09-11T07:36:46.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:45 smithi104 ceph-mon[33507]: 4.6 scrub ok 2024-09-11T07:36:46.046 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:46.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:46.048 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:46.148 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:46.178 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T07:36:46.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:46.179 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:46.179 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:46.280 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:46.310 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:46.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:46.311 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:46.412 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:46.439 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:46.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:46.440 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:46.541 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:46.588 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:46.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:46.589 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:46.690 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:46.716 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:46.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:46.717 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:46.818 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:46.856 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:46.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:46.857 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:46.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:46 smithi022 ceph-mon[29462]: 1.0 scrub starts 2024-09-11T07:36:46.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:46 smithi022 ceph-mon[29462]: 1.0 scrub ok 2024-09-11T07:36:46.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:46 smithi022 ceph-mon[29462]: from='client.25271 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-11T07:36:46.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:46 smithi022 ceph-mon[29462]: osdmap e236: 8 total, 8 up, 8 in 2024-09-11T07:36:46.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:46 smithi022 ceph-mon[29462]: 4.b scrub starts 2024-09-11T07:36:46.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:46 smithi022 ceph-mon[29462]: 4.b scrub ok 2024-09-11T07:36:46.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:46 smithi022 ceph-mon[29462]: pgmap v456: 148 pgs: 148 active+clean; 1.2 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 1.55k op/s 2024-09-11T07:36:46.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:46 smithi022 ceph-mon[37330]: 1.0 scrub starts 2024-09-11T07:36:46.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:46 smithi022 ceph-mon[37330]: 1.0 scrub ok 2024-09-11T07:36:46.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:46 smithi022 ceph-mon[37330]: from='client.25271 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-11T07:36:46.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:46 smithi022 ceph-mon[37330]: osdmap e236: 8 total, 8 up, 8 in 2024-09-11T07:36:46.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:46 smithi022 ceph-mon[37330]: 4.b scrub starts 2024-09-11T07:36:46.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:46 smithi022 ceph-mon[37330]: 4.b scrub ok 2024-09-11T07:36:46.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:46 smithi022 ceph-mon[37330]: pgmap v456: 148 pgs: 148 active+clean; 1.2 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 1.55k op/s 2024-09-11T07:36:46.958 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:46.995 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:46.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:46.996 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:47.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:46 smithi104 ceph-mon[33507]: 1.0 scrub starts 2024-09-11T07:36:47.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:46 smithi104 ceph-mon[33507]: 1.0 scrub ok 2024-09-11T07:36:47.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:46 smithi104 ceph-mon[33507]: from='client.25271 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-11T07:36:47.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:46 smithi104 ceph-mon[33507]: osdmap e236: 8 total, 8 up, 8 in 2024-09-11T07:36:47.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:46 smithi104 ceph-mon[33507]: 4.b scrub starts 2024-09-11T07:36:47.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:46 smithi104 ceph-mon[33507]: 4.b scrub ok 2024-09-11T07:36:47.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:46 smithi104 ceph-mon[33507]: pgmap v456: 148 pgs: 148 active+clean; 1.2 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 1.55k op/s 2024-09-11T07:36:47.097 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:47.126 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:47.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:47.127 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:47.227 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:47.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:47.274 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:47.274 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:47.339 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:36:47.340 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 20, 2024-09-11T07:36:47.340 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 600, 2024-09-11T07:36:47.340 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:36:47.340 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.340 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:120403 7.a 7:53556683:::benchmark_data_smithi022_77720_object120402:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:36:47.341 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:28.687289+0000", 2024-09-11T07:36:47.341 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 378.64675616599999, 2024-09-11T07:36:47.341 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.15092119800000001, 2024-09-11T07:36:47.341 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:47.341 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:47.341 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:47.342 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:36:47.342 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:36:47.342 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 120403 2024-09-11T07:36:47.342 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.342 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:47.342 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.343 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:47.343 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.687289+0000", 2024-09-11T07:36:47.343 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.343 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.343 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.343 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:47.344 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.687289+0000", 2024-09-11T07:36:47.344 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.344 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.344 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.344 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:47.345 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.687288+0000", 2024-09-11T07:36:47.345 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:36:47.345 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.345 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.345 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:47.345 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.687396+0000", 2024-09-11T07:36:47.346 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010856 2024-09-11T07:36:47.346 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.346 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.346 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:47.346 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.687420+0000", 2024-09-11T07:36:47.347 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4114e-05 2024-09-11T07:36:47.347 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.347 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.347 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:47.347 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.687446+0000", 2024-09-11T07:36:47.347 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5367e-05 2024-09-11T07:36:47.348 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.348 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.348 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:47.348 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.691757+0000", 2024-09-11T07:36:47.348 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0043116700000000001 2024-09-11T07:36:47.348 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.349 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.349 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:47.349 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.691876+0000", 2024-09-11T07:36:47.349 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000118707 2024-09-11T07:36:47.349 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.349 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.349 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,5", 2024-09-11T07:36:47.350 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.692007+0000", 2024-09-11T07:36:47.350 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013065599999999999 2024-09-11T07:36:47.350 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.350 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.350 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:47.350 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695100+0000", 2024-09-11T07:36:47.351 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0030930789999999999 2024-09-11T07:36:47.351 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.351 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.351 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:36:47.351 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.700863+0000", 2024-09-11T07:36:47.351 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0057636079999999999 2024-09-11T07:36:47.351 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.351 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.351 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:47.351 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838144+0000", 2024-09-11T07:36:47.352 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13728073099999999 2024-09-11T07:36:47.352 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.352 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.352 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:47.352 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838179+0000", 2024-09-11T07:36:47.352 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5129000000000002e-05 2024-09-11T07:36:47.352 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.352 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.352 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:47.352 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838210+0000", 2024-09-11T07:36:47.352 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0633999999999998e-05 2024-09-11T07:36:47.352 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.352 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:47.353 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.353 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.353 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.353 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:120408 7.2 7:4165bb05:::benchmark_data_smithi022_77720_object120407:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:36:47.353 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:28.688203+0000", 2024-09-11T07:36:47.353 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 378.64584190300002, 2024-09-11T07:36:47.353 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.149908602, 2024-09-11T07:36:47.353 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:47.353 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:47.353 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:47.353 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:36:47.353 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:36:47.353 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 120408 2024-09-11T07:36:47.354 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.354 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:47.354 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.354 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:47.354 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.688203+0000", 2024-09-11T07:36:47.354 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.354 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.354 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.354 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:47.354 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.688203+0000", 2024-09-11T07:36:47.354 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.354 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.354 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.354 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:47.355 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.688202+0000", 2024-09-11T07:36:47.355 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:36:47.355 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.355 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.355 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:47.355 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.688225+0000", 2024-09-11T07:36:47.355 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3796999999999999e-05 2024-09-11T07:36:47.355 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.355 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.355 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:47.355 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.688235+0000", 2024-09-11T07:36:47.355 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.7599999999999997e-06 2024-09-11T07:36:47.356 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.356 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.356 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:47.356 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.688241+0000", 2024-09-11T07:36:47.356 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6890000000000001e-06 2024-09-11T07:36:47.356 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.356 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.356 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:47.356 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.691642+0000", 2024-09-11T07:36:47.356 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.003401669 2024-09-11T07:36:47.356 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.356 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.356 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:47.357 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.691766+0000", 2024-09-11T07:36:47.357 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012310700000000001 2024-09-11T07:36:47.357 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.357 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.357 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:36:47.357 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.691893+0000", 2024-09-11T07:36:47.357 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012741099999999999 2024-09-11T07:36:47.357 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.357 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.357 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:36:47.357 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.700857+0000", 2024-09-11T07:36:47.357 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0089635749999999997 2024-09-11T07:36:47.357 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.357 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.358 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:47.358 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.701785+0000", 2024-09-11T07:36:47.358 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00092816999999999997 2024-09-11T07:36:47.358 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.358 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.358 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:47.358 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838024+0000", 2024-09-11T07:36:47.358 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13623940600000001 2024-09-11T07:36:47.358 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.358 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.358 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:47.358 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838072+0000", 2024-09-11T07:36:47.358 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8340999999999997e-05 2024-09-11T07:36:47.359 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.359 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.359 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:47.359 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838111+0000", 2024-09-11T07:36:47.359 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8998000000000002e-05 2024-09-11T07:36:47.359 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.359 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:47.359 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.359 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.359 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.359 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:120413 7.a 7:570dbbb8:::benchmark_data_smithi022_77720_object120412:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:36:47.359 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:28.694903+0000", 2024-09-11T07:36:47.359 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 378.63914156700002, 2024-09-11T07:36:47.359 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.14339426699999999, 2024-09-11T07:36:47.359 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:47.359 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:47.360 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:47.360 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:36:47.360 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:36:47.360 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 120413 2024-09-11T07:36:47.360 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.360 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:47.360 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.360 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:47.360 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.694903+0000", 2024-09-11T07:36:47.360 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.360 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.360 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.360 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:47.361 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.694903+0000", 2024-09-11T07:36:47.361 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.361 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.361 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.361 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:47.361 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.694901+0000", 2024-09-11T07:36:47.361 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:36:47.361 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.361 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.361 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:47.361 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.694926+0000", 2024-09-11T07:36:47.361 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4505999999999999e-05 2024-09-11T07:36:47.361 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.362 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.362 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:47.362 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.694938+0000", 2024-09-11T07:36:47.362 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2326e-05 2024-09-11T07:36:47.362 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.362 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.362 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:47.362 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.694944+0000", 2024-09-11T07:36:47.362 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2299999999999996e-06 2024-09-11T07:36:47.362 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.363 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.363 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:47.363 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695119+0000", 2024-09-11T07:36:47.363 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00017516499999999999 2024-09-11T07:36:47.363 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.363 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.363 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:47.363 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695231+0000", 2024-09-11T07:36:47.363 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011134800000000001 2024-09-11T07:36:47.363 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.363 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.363 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,5", 2024-09-11T07:36:47.363 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695360+0000", 2024-09-11T07:36:47.363 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012901899999999999 2024-09-11T07:36:47.364 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.364 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.364 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:47.364 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.698418+0000", 2024-09-11T07:36:47.364 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0030584520000000001 2024-09-11T07:36:47.364 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.364 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.364 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:36:47.364 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.700986+0000", 2024-09-11T07:36:47.364 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0025678440000000001 2024-09-11T07:36:47.364 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.364 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.364 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:47.365 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838237+0000", 2024-09-11T07:36:47.365 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13725120900000001 2024-09-11T07:36:47.365 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.365 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.365 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:47.365 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838270+0000", 2024-09-11T07:36:47.365 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2852000000000003e-05 2024-09-11T07:36:47.365 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.365 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.365 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:47.366 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838297+0000", 2024-09-11T07:36:47.366 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7477000000000001e-05 2024-09-11T07:36:47.366 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.366 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:47.366 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.366 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.366 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.366 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:120415 7.2 7:4b247b8a:::benchmark_data_smithi022_77720_object120414:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:36:47.366 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:28.695503+0000", 2024-09-11T07:36:47.366 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 378.638541637, 2024-09-11T07:36:47.366 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.14288304399999999, 2024-09-11T07:36:47.366 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:47.366 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:47.367 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:47.367 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:36:47.367 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:36:47.367 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 120415 2024-09-11T07:36:47.367 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.367 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:47.367 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.367 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:47.367 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695503+0000", 2024-09-11T07:36:47.367 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.367 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.367 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.367 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:47.368 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695503+0000", 2024-09-11T07:36:47.368 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.368 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.368 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.368 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:47.368 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695502+0000", 2024-09-11T07:36:47.368 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:36:47.368 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.368 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.368 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:47.368 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695525+0000", 2024-09-11T07:36:47.368 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3680999999999999e-05 2024-09-11T07:36:47.368 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.369 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.369 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:47.369 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695537+0000", 2024-09-11T07:36:47.369 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.129e-05 2024-09-11T07:36:47.369 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.369 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.369 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:47.369 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.695542+0000", 2024-09-11T07:36:47.369 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7899999999999996e-06 2024-09-11T07:36:47.369 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.369 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.369 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:47.369 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.697877+0000", 2024-09-11T07:36:47.370 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0023348319999999998 2024-09-11T07:36:47.370 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.370 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.370 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:47.370 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.698008+0000", 2024-09-11T07:36:47.370 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000131144 2024-09-11T07:36:47.370 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.370 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.370 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:36:47.370 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.698162+0000", 2024-09-11T07:36:47.370 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000153788 2024-09-11T07:36:47.371 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.371 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.371 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:36:47.371 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.701572+0000", 2024-09-11T07:36:47.371 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.003409568 2024-09-11T07:36:47.371 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.371 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.371 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:47.371 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.707469+0000", 2024-09-11T07:36:47.371 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0058970669999999998 2024-09-11T07:36:47.371 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.371 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.372 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:47.372 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838325+0000", 2024-09-11T07:36:47.372 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13085648699999999 2024-09-11T07:36:47.372 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.372 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.372 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:47.372 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838358+0000", 2024-09-11T07:36:47.372 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2539999999999997e-05 2024-09-11T07:36:47.372 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.372 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.372 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:47.372 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:28.838386+0000", 2024-09-11T07:36:47.373 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.828e-05 2024-09-11T07:36:47.373 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.373 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:47.373 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.373 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.373 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.373 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:120637 7.a 7:5b8c9ff4:::benchmark_data_smithi022_77720_object120636:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:36:47.373 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.029397+0000", 2024-09-11T07:36:47.373 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 378.30464820899999, 2024-09-11T07:36:47.373 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.22035210899999999, 2024-09-11T07:36:47.373 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:47.373 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:47.373 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:47.374 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:36:47.374 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:36:47.374 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 120637 2024-09-11T07:36:47.374 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.374 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:47.374 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.374 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:47.374 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.029397+0000", 2024-09-11T07:36:47.374 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.374 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.374 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.375 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:47.375 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.029397+0000", 2024-09-11T07:36:47.375 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.375 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.375 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.375 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:47.375 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.029394+0000", 2024-09-11T07:36:47.375 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:47.376 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:36:47.376 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.377 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.377 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:47.377 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.029420+0000", 2024-09-11T07:36:47.377 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5621000000000001e-05 2024-09-11T07:36:47.377 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.377 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.377 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:47.377 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.029433+0000", 2024-09-11T07:36:47.377 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3437e-05 2024-09-11T07:36:47.377 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.377 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.377 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:47.377 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.029440+0000", 2024-09-11T07:36:47.378 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7290000000000003e-06 2024-09-11T07:36:47.378 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.378 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.378 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:47.378 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.031329+0000", 2024-09-11T07:36:47.378 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0018888970000000001 2024-09-11T07:36:47.378 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.378 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.378 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:47.378 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.031434+0000", 2024-09-11T07:36:47.378 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000104985 2024-09-11T07:36:47.378 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.379 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.379 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,5", 2024-09-11T07:36:47.379 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.031563+0000", 2024-09-11T07:36:47.379 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000128794 2024-09-11T07:36:47.379 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.379 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.379 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:47.379 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.040366+0000", 2024-09-11T07:36:47.379 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0088039620000000002 2024-09-11T07:36:47.379 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.379 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.379 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:36:47.379 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.041770+0000", 2024-09-11T07:36:47.380 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001403257 2024-09-11T07:36:47.380 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.380 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.380 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:47.380 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.249644+0000", 2024-09-11T07:36:47.380 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20787381199999999 2024-09-11T07:36:47.380 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.380 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.380 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:47.381 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.249698+0000", 2024-09-11T07:36:47.381 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.4273e-05 2024-09-11T07:36:47.381 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.381 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.381 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:47.381 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.249749+0000", 2024-09-11T07:36:47.381 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.0803000000000001e-05 2024-09-11T07:36:47.381 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.381 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:47.381 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.381 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.381 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.382 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:120649 7.2 7:4a3799e9:::benchmark_data_smithi022_77720_object120648:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:36:47.382 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.034409+0000", 2024-09-11T07:36:47.382 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 378.29963543000002, 2024-09-11T07:36:47.382 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.215573452, 2024-09-11T07:36:47.382 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:47.382 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:47.382 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:47.382 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:36:47.382 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:36:47.382 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 120649 2024-09-11T07:36:47.382 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.382 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:47.382 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.382 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:47.383 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.034409+0000", 2024-09-11T07:36:47.383 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.383 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.383 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.383 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:47.383 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.034409+0000", 2024-09-11T07:36:47.383 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.383 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.383 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.383 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:47.383 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.034407+0000", 2024-09-11T07:36:47.383 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:36:47.383 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.383 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.384 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:47.384 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.034433+0000", 2024-09-11T07:36:47.384 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.6526999999999999e-05 2024-09-11T07:36:47.384 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.384 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.384 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:47.384 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.034448+0000", 2024-09-11T07:36:47.384 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4080000000000001e-05 2024-09-11T07:36:47.384 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.384 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.384 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:47.384 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.034454+0000", 2024-09-11T07:36:47.384 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5429999999999999e-06 2024-09-11T07:36:47.384 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.384 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.385 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:47.385 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.036213+0000", 2024-09-11T07:36:47.385 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001758857 2024-09-11T07:36:47.385 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.385 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.385 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:47.385 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.036318+0000", 2024-09-11T07:36:47.385 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000105245 2024-09-11T07:36:47.385 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.385 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.385 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:36:47.385 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.036447+0000", 2024-09-11T07:36:47.385 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012836999999999999 2024-09-11T07:36:47.385 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.386 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.386 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:47.386 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.042776+0000", 2024-09-11T07:36:47.386 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0063298790000000001 2024-09-11T07:36:47.386 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.386 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.386 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:36:47.386 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.043761+0000", 2024-09-11T07:36:47.386 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00098433900000000009 2024-09-11T07:36:47.387 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.387 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.387 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:47.387 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.249911+0000", 2024-09-11T07:36:47.387 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20615025200000001 2024-09-11T07:36:47.387 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.387 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.387 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:47.387 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.249951+0000", 2024-09-11T07:36:47.387 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9817000000000002e-05 2024-09-11T07:36:47.387 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.387 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.387 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:47.387 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.249983+0000", 2024-09-11T07:36:47.388 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.188e-05 2024-09-11T07:36:47.388 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.388 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:47.388 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.388 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.388 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.388 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120650 7.c e127/92 7:3551ad5b:::benchmark_data_smithi022_77720_object120649:head v 127'7564, mlcod=127'7562)", 2024-09-11T07:36:47.388 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.038754+0000", 2024-09-11T07:36:47.388 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 378.295290271, 2024-09-11T07:36:47.388 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.21640670200000001, 2024-09-11T07:36:47.388 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:47.388 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:47.388 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:47.389 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.389 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:47.389 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.038754+0000", 2024-09-11T07:36:47.389 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.389 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.389 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.389 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:47.389 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.038754+0000", 2024-09-11T07:36:47.389 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.389 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.389 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.389 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:47.389 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.038752+0000", 2024-09-11T07:36:47.390 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:36:47.390 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.390 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.390 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:47.390 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.038797+0000", 2024-09-11T07:36:47.390 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4357000000000002e-05 2024-09-11T07:36:47.390 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.390 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.390 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:47.390 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.038809+0000", 2024-09-11T07:36:47.390 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1829e-05 2024-09-11T07:36:47.390 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.390 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.390 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:47.390 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.038815+0000", 2024-09-11T07:36:47.391 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3579999999999999e-06 2024-09-11T07:36:47.391 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.391 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.391 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:47.391 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.038871+0000", 2024-09-11T07:36:47.391 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6193999999999998e-05 2024-09-11T07:36:47.391 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.391 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.391 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:47.391 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.038889+0000", 2024-09-11T07:36:47.391 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.7915999999999999e-05 2024-09-11T07:36:47.391 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.391 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.391 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:47.392 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.255019+0000", 2024-09-11T07:36:47.392 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.21613015499999999 2024-09-11T07:36:47.392 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.392 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.392 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:47.392 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.255161+0000", 2024-09-11T07:36:47.392 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014201900000000001 2024-09-11T07:36:47.392 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.392 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:47.392 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.392 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.392 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.392 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120651 7.c e127/92 7:32aa7de5:::benchmark_data_smithi022_77720_object120650:head v 127'7565, mlcod=127'7562)", 2024-09-11T07:36:47.393 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.041033+0000", 2024-09-11T07:36:47.393 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 378.29301154400002, 2024-09-11T07:36:47.393 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.214179168, 2024-09-11T07:36:47.393 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:47.393 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:47.393 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:47.393 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.393 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:47.393 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.041033+0000", 2024-09-11T07:36:47.393 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.393 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.393 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.393 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:47.394 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.041033+0000", 2024-09-11T07:36:47.394 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.394 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.394 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.394 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:47.394 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.041032+0000", 2024-09-11T07:36:47.394 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:36:47.394 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.394 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.395 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:47.395 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.041062+0000", 2024-09-11T07:36:47.395 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0488999999999999e-05 2024-09-11T07:36:47.395 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.395 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.395 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:47.395 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.041072+0000", 2024-09-11T07:36:47.395 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.5629999999999998e-06 2024-09-11T07:36:47.395 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.395 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.395 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:47.395 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.041077+0000", 2024-09-11T07:36:47.396 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.0579999999999997e-06 2024-09-11T07:36:47.396 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.396 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.396 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:47.396 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.045254+0000", 2024-09-11T07:36:47.396 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0041766110000000002 2024-09-11T07:36:47.396 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.396 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.396 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:47.396 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.045273+0000", 2024-09-11T07:36:47.396 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9636e-05 2024-09-11T07:36:47.396 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.396 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.397 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:47.397 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.255164+0000", 2024-09-11T07:36:47.397 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20989069099999999 2024-09-11T07:36:47.397 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.397 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.397 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:47.397 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.255212+0000", 2024-09-11T07:36:47.397 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8322000000000003e-05 2024-09-11T07:36:47.397 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.397 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:47.397 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.398 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.398 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.398 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120654 7.c e127/92 7:3089e4e7:::benchmark_data_smithi022_77720_object120653:head v 127'7566, mlcod=127'7562)", 2024-09-11T07:36:47.398 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.044199+0000", 2024-09-11T07:36:47.398 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 378.28984617100002, 2024-09-11T07:36:47.398 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.211072754, 2024-09-11T07:36:47.398 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:47.398 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:47.398 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:47.399 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.399 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:47.399 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.044199+0000", 2024-09-11T07:36:47.399 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.399 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.399 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.399 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:47.399 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.044199+0000", 2024-09-11T07:36:47.399 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.399 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.399 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.399 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:47.400 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.044197+0000", 2024-09-11T07:36:47.400 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:36:47.400 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.400 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.400 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:47.400 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.044244+0000", 2024-09-11T07:36:47.400 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6838000000000001e-05 2024-09-11T07:36:47.400 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.400 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.400 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:47.400 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.044256+0000", 2024-09-11T07:36:47.400 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2041e-05 2024-09-11T07:36:47.401 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.401 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.401 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:47.401 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.044262+0000", 2024-09-11T07:36:47.401 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2110000000000003e-06 2024-09-11T07:36:47.401 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.401 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.401 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:47.401 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.048642+0000", 2024-09-11T07:36:47.401 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0043793469999999996 2024-09-11T07:36:47.401 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.401 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.402 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:47.402 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.048660+0000", 2024-09-11T07:36:47.402 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8363999999999999e-05 2024-09-11T07:36:47.402 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.402 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.402 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:47.402 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.255215+0000", 2024-09-11T07:36:47.402 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20655538200000001 2024-09-11T07:36:47.402 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.402 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.402 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:47.403 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.255271+0000", 2024-09-11T07:36:47.403 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6033999999999999e-05 2024-09-11T07:36:47.403 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.403 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:47.403 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.403 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.403 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.403 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:120659 7.3 7:c2975cc8:::benchmark_data_smithi022_77720_object120658:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:36:47.403 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.046561+0000", 2024-09-11T07:36:47.403 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 378.28748352399998, 2024-09-11T07:36:47.403 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20440429299999999, 2024-09-11T07:36:47.404 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:47.404 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:47.404 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:47.404 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:36:47.404 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:36:47.404 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 120659 2024-09-11T07:36:47.404 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.404 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:47.404 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.404 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:47.404 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046561+0000", 2024-09-11T07:36:47.404 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.405 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.405 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.405 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:47.405 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046561+0000", 2024-09-11T07:36:47.405 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.405 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.405 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.405 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:47.405 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046559+0000", 2024-09-11T07:36:47.405 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:36:47.405 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.405 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.406 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:47.406 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046587+0000", 2024-09-11T07:36:47.406 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8705e-05 2024-09-11T07:36:47.406 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.406 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.406 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:47.406 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046600+0000", 2024-09-11T07:36:47.406 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2499e-05 2024-09-11T07:36:47.406 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.406 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.406 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:47.406 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046606+0000", 2024-09-11T07:36:47.406 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0159999999999997e-06 2024-09-11T07:36:47.407 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.407 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.407 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:47.407 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046693+0000", 2024-09-11T07:36:47.407 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.6792999999999994e-05 2024-09-11T07:36:47.407 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.407 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.407 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:47.407 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046953+0000", 2024-09-11T07:36:47.407 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00026056300000000001 2024-09-11T07:36:47.407 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.407 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.407 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,4", 2024-09-11T07:36:47.407 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.047062+0000", 2024-09-11T07:36:47.408 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010925999999999999 2024-09-11T07:36:47.408 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.408 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.408 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:36:47.408 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.049336+0000", 2024-09-11T07:36:47.408 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002273691 2024-09-11T07:36:47.408 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.408 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.408 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:47.408 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050203+0000", 2024-09-11T07:36:47.408 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000866591 2024-09-11T07:36:47.408 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.409 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.409 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:47.409 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.250805+0000", 2024-09-11T07:36:47.409 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20060213800000001 2024-09-11T07:36:47.409 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.409 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.409 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:47.409 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.250854+0000", 2024-09-11T07:36:47.409 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9335000000000001e-05 2024-09-11T07:36:47.409 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.409 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.409 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:47.409 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.250965+0000", 2024-09-11T07:36:47.410 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000111242 2024-09-11T07:36:47.410 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.410 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:47.410 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.410 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.410 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.410 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120656 7.c e127/92 7:35364cdf:::benchmark_data_smithi022_77720_object120655:head v 127'7567, mlcod=127'7562)", 2024-09-11T07:36:47.410 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.046959+0000", 2024-09-11T07:36:47.410 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 378.28708551, 2024-09-11T07:36:47.410 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.21188748199999999, 2024-09-11T07:36:47.410 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:47.410 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:47.410 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:47.410 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.411 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:47.411 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046959+0000", 2024-09-11T07:36:47.411 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.411 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.411 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.411 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:47.411 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046959+0000", 2024-09-11T07:36:47.411 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.411 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.411 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.411 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:47.411 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.046958+0000", 2024-09-11T07:36:47.411 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:36:47.412 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.412 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.412 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:47.412 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.047009+0000", 2024-09-11T07:36:47.412 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1489000000000001e-05 2024-09-11T07:36:47.412 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.412 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.412 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:47.412 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.047020+0000", 2024-09-11T07:36:47.412 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.064e-05 2024-09-11T07:36:47.412 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.412 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.412 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:47.412 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.047025+0000", 2024-09-11T07:36:47.413 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1279999999999999e-06 2024-09-11T07:36:47.413 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.413 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.413 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:47.413 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246150+0000", 2024-09-11T07:36:47.413 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.199124994 2024-09-11T07:36:47.413 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.413 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.413 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:47.413 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246173+0000", 2024-09-11T07:36:47.413 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3091999999999998e-05 2024-09-11T07:36:47.413 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.413 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.414 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:47.414 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.258588+0000", 2024-09-11T07:36:47.414 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.012415433 2024-09-11T07:36:47.414 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.414 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.414 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:47.414 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.258847+0000", 2024-09-11T07:36:47.414 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00025835000000000002 2024-09-11T07:36:47.414 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.414 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:47.414 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.414 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.414 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.414 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120657 7.c e127/92 7:333acda7:::benchmark_data_smithi022_77720_object120656:head v 127'7568, mlcod=127'7562)", 2024-09-11T07:36:47.415 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.050053+0000", 2024-09-11T07:36:47.415 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 378.28399132300001, 2024-09-11T07:36:47.415 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.22377028800000001, 2024-09-11T07:36:47.415 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:47.415 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:47.415 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:47.415 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.415 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:47.415 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050053+0000", 2024-09-11T07:36:47.415 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.415 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.415 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.415 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:47.416 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050053+0000", 2024-09-11T07:36:47.416 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.416 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.416 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.416 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:47.416 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050052+0000", 2024-09-11T07:36:47.416 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:36:47.416 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.416 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.416 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:47.416 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050092+0000", 2024-09-11T07:36:47.417 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0565000000000002e-05 2024-09-11T07:36:47.417 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.417 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.417 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:47.417 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050102+0000", 2024-09-11T07:36:47.417 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.6150000000000003e-06 2024-09-11T07:36:47.417 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.417 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.417 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:47.417 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050106+0000", 2024-09-11T07:36:47.417 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9400000000000004e-06 2024-09-11T07:36:47.417 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.418 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.418 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:47.418 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.255309+0000", 2024-09-11T07:36:47.418 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.205202789 2024-09-11T07:36:47.418 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.418 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.418 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:47.418 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.255342+0000", 2024-09-11T07:36:47.418 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2854000000000001e-05 2024-09-11T07:36:47.418 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.418 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.419 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:47.419 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.273773+0000", 2024-09-11T07:36:47.419 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.018431050000000001 2024-09-11T07:36:47.419 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.419 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.419 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:47.419 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.273824+0000", 2024-09-11T07:36:47.419 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1072000000000002e-05 2024-09-11T07:36:47.419 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.419 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:47.419 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.419 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.419 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.419 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120662 7.7 e127/92 7:edc0624b:::benchmark_data_smithi022_77720_object120661:head v 127'7599, mlcod=127'7596)", 2024-09-11T07:36:47.420 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.050439+0000", 2024-09-11T07:36:47.420 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 378.28360548400002, 2024-09-11T07:36:47.420 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20149834799999999, 2024-09-11T07:36:47.420 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:47.420 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:47.420 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:47.420 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.420 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:47.420 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050439+0000", 2024-09-11T07:36:47.420 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.420 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.420 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.420 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:47.420 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050439+0000", 2024-09-11T07:36:47.421 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.421 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.421 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.421 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:47.421 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050438+0000", 2024-09-11T07:36:47.421 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:36:47.421 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.421 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.421 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:47.421 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050526+0000", 2024-09-11T07:36:47.421 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.8176000000000004e-05 2024-09-11T07:36:47.421 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.421 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.421 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:47.422 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050535+0000", 2024-09-11T07:36:47.422 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.1050000000000001e-06 2024-09-11T07:36:47.422 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.422 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.422 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:47.422 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050540+0000", 2024-09-11T07:36:47.422 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8740000000000003e-06 2024-09-11T07:36:47.422 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.423 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.423 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:47.423 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.248240+0000", 2024-09-11T07:36:47.423 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.19770011900000001 2024-09-11T07:36:47.423 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.424 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.424 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:47.424 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.248260+0000", 2024-09-11T07:36:47.424 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9468000000000001e-05 2024-09-11T07:36:47.424 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.424 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.424 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:47.424 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.251902+0000", 2024-09-11T07:36:47.424 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0036426689999999999 2024-09-11T07:36:47.424 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.424 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.424 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:47.424 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.251938+0000", 2024-09-11T07:36:47.425 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5219999999999998e-05 2024-09-11T07:36:47.425 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.425 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:47.425 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.425 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.425 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.425 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120661 7.1 e127/92 7:8ef41966:::benchmark_data_smithi022_77720_object120660:head v 127'7487, mlcod=127'7484)", 2024-09-11T07:36:47.425 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.050540+0000", 2024-09-11T07:36:47.425 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 378.283505198, 2024-09-11T07:36:47.425 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20305398899999999, 2024-09-11T07:36:47.425 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:47.425 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:47.425 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:47.425 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.426 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:47.426 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050540+0000", 2024-09-11T07:36:47.426 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.426 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.426 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.426 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:47.426 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050540+0000", 2024-09-11T07:36:47.426 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.426 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.426 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.426 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:47.426 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050539+0000", 2024-09-11T07:36:47.426 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:36:47.426 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.427 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.427 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:47.427 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050602+0000", 2024-09-11T07:36:47.427 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3145000000000006e-05 2024-09-11T07:36:47.427 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.427 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.427 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:47.427 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246370+0000", 2024-09-11T07:36:47.427 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.19576812599999999 2024-09-11T07:36:47.427 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.427 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.427 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:47.428 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246378+0000", 2024-09-11T07:36:47.428 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.8860000000000004e-06 2024-09-11T07:36:47.428 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.428 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.428 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:47.428 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246487+0000", 2024-09-11T07:36:47.428 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000109171 2024-09-11T07:36:47.428 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.428 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.428 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:47.428 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246507+0000", 2024-09-11T07:36:47.428 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0064999999999998e-05 2024-09-11T07:36:47.428 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.428 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.429 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:47.429 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.253510+0000", 2024-09-11T07:36:47.429 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0070029630000000001 2024-09-11T07:36:47.429 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.429 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.429 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:47.429 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.253594+0000", 2024-09-11T07:36:47.429 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3560000000000006e-05 2024-09-11T07:36:47.429 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.429 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:47.429 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.429 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.429 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.429 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24353.0:885 5.14 5:2b04a3e9:::notify.6:head [watch reconnect cookie 94395805760480 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-11T07:36:47.430 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:31:02.582152+0000", 2024-09-11T07:36:47.430 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 344.75189322300002, 2024-09-11T07:36:47.430 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.62439777600000002, 2024-09-11T07:36:47.430 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:47.430 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:36:47.430 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:47.430 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24353", 2024-09-11T07:36:47.430 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1860969011", 2024-09-11T07:36:47.430 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 885 2024-09-11T07:36:47.430 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.430 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:47.430 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.430 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:47.430 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.582152+0000", 2024-09-11T07:36:47.431 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.431 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.431 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.431 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:47.431 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.582152+0000", 2024-09-11T07:36:47.431 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.431 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.431 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.431 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:47.431 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.582149+0000", 2024-09-11T07:36:47.431 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:36:47.431 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.431 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.432 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:47.432 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.582157+0000", 2024-09-11T07:36:47.432 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.2050000000000002e-06 2024-09-11T07:36:47.432 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.432 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.432 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:47.432 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.582167+0000", 2024-09-11T07:36:47.432 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.6299999999999993e-06 2024-09-11T07:36:47.432 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.432 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.432 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:47.432 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.582172+0000", 2024-09-11T07:36:47.432 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1309999999999997e-06 2024-09-11T07:36:47.432 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.433 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.433 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:47.433 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.582219+0000", 2024-09-11T07:36:47.433 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7246000000000001e-05 2024-09-11T07:36:47.433 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.433 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.433 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:36:47.433 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.582225+0000", 2024-09-11T07:36:47.433 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8529999999999997e-06 2024-09-11T07:36:47.433 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.433 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.433 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:47.433 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.201574+0000", 2024-09-11T07:36:47.434 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.61934906000000001 2024-09-11T07:36:47.434 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.434 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.434 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:47.434 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.201736+0000", 2024-09-11T07:36:47.434 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000162118 2024-09-11T07:36:47.434 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.434 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.434 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:47.434 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.206549+0000", 2024-09-11T07:36:47.434 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0048128700000000003 2024-09-11T07:36:47.434 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.434 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:47.434 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.435 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.435 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.435 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:14418 8.0 8:0e01a02b:::benchmark_data_smithi022_108406_object14417:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-11T07:36:47.435 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:45.110638+0000", 2024-09-11T07:36:47.435 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 182.22340711999999, 2024-09-11T07:36:47.435 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13896979200000001, 2024-09-11T07:36:47.435 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:47.435 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:47.435 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:47.435 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:36:47.435 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:36:47.435 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 14418 2024-09-11T07:36:47.435 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.436 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:47.436 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.436 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:47.436 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110638+0000", 2024-09-11T07:36:47.436 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.436 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.436 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.436 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:47.436 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110638+0000", 2024-09-11T07:36:47.436 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.436 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.436 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.436 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:47.437 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110636+0000", 2024-09-11T07:36:47.437 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:36:47.437 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.437 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.437 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:47.437 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110658+0000", 2024-09-11T07:36:47.437 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2243e-05 2024-09-11T07:36:47.437 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.437 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.437 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:47.437 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110668+0000", 2024-09-11T07:36:47.437 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.1160000000000005e-06 2024-09-11T07:36:47.437 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.437 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.438 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:47.438 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110673+0000", 2024-09-11T07:36:47.438 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.519e-06 2024-09-11T07:36:47.438 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.438 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.438 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:47.438 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110717+0000", 2024-09-11T07:36:47.438 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3476999999999997e-05 2024-09-11T07:36:47.438 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.438 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.438 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:47.438 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110839+0000", 2024-09-11T07:36:47.438 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012271100000000001 2024-09-11T07:36:47.438 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.439 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.439 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,5", 2024-09-11T07:36:47.439 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110981+0000", 2024-09-11T07:36:47.439 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000141218 2024-09-11T07:36:47.439 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.439 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.439 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:36:47.439 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112249+0000", 2024-09-11T07:36:47.439 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0012689330000000001 2024-09-11T07:36:47.439 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.439 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.439 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:47.439 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.114781+0000", 2024-09-11T07:36:47.439 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0025319969999999998 2024-09-11T07:36:47.440 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.440 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.440 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:47.440 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.249534+0000", 2024-09-11T07:36:47.440 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.134752701 2024-09-11T07:36:47.440 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.440 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.440 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:47.440 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.249574+0000", 2024-09-11T07:36:47.440 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9972000000000003e-05 2024-09-11T07:36:47.440 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.440 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.440 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:47.441 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.249607+0000", 2024-09-11T07:36:47.441 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3272999999999998e-05 2024-09-11T07:36:47.441 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.441 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:47.441 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.441 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.441 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.441 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:14419 8.f 8:fe75a06c:::benchmark_data_smithi022_108406_object14418:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-11T07:36:47.441 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:45.110878+0000", 2024-09-11T07:36:47.441 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 182.22316717999999, 2024-09-11T07:36:47.441 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13866826700000001, 2024-09-11T07:36:47.441 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:47.441 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:47.442 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:47.442 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:36:47.442 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:36:47.442 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 14419 2024-09-11T07:36:47.442 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.442 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:47.442 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.442 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:47.442 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110878+0000", 2024-09-11T07:36:47.442 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.442 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.442 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.442 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:47.442 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110878+0000", 2024-09-11T07:36:47.442 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.443 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.443 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.443 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:47.443 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110876+0000", 2024-09-11T07:36:47.443 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:36:47.443 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.443 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.443 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:47.443 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110916+0000", 2024-09-11T07:36:47.443 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0201999999999999e-05 2024-09-11T07:36:47.443 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.443 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.443 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:47.443 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110928+0000", 2024-09-11T07:36:47.443 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1099e-05 2024-09-11T07:36:47.444 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.444 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.444 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:47.444 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.110934+0000", 2024-09-11T07:36:47.444 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2249999999999997e-06 2024-09-11T07:36:47.444 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.444 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.444 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:47.444 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.111838+0000", 2024-09-11T07:36:47.444 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00090451400000000003 2024-09-11T07:36:47.444 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.444 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.444 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:47.445 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.111957+0000", 2024-09-11T07:36:47.445 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000118888 2024-09-11T07:36:47.445 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.445 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.445 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:36:47.445 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112088+0000", 2024-09-11T07:36:47.445 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013097400000000001 2024-09-11T07:36:47.445 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.445 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.445 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:36:47.445 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112903+0000", 2024-09-11T07:36:47.445 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00081438400000000003 2024-09-11T07:36:47.445 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.446 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.446 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:47.446 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.114264+0000", 2024-09-11T07:36:47.446 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001361026 2024-09-11T07:36:47.446 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.446 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.446 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:47.446 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.249462+0000", 2024-09-11T07:36:47.446 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13519831800000001 2024-09-11T07:36:47.446 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.446 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.446 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:47.446 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.249512+0000", 2024-09-11T07:36:47.447 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9867999999999998e-05 2024-09-11T07:36:47.447 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.447 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.447 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:47.447 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.249546+0000", 2024-09-11T07:36:47.447 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4041999999999997e-05 2024-09-11T07:36:47.447 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.447 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:47.447 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.447 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.447 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.447 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:28679 8.b 8:dc4b7282:::benchmark_data_smithi022_108406_object28678:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e189)", 2024-09-11T07:36:47.447 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:51.011649+0000", 2024-09-11T07:36:47.448 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 176.32239569199999, 2024-09-11T07:36:47.448 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.139338726, 2024-09-11T07:36:47.448 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:47.448 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:47.448 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:47.448 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:36:47.448 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:36:47.448 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 28679 2024-09-11T07:36:47.448 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.448 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:47.448 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.448 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:47.448 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.011649+0000", 2024-09-11T07:36:47.448 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.449 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.449 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.449 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:47.449 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.011649+0000", 2024-09-11T07:36:47.449 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.449 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.449 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.449 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:47.449 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.011648+0000", 2024-09-11T07:36:47.449 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:36:47.449 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.449 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.449 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:47.450 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.011681+0000", 2024-09-11T07:36:47.450 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3272999999999998e-05 2024-09-11T07:36:47.450 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.450 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.450 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:47.450 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.011690+0000", 2024-09-11T07:36:47.450 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.3349999999999996e-06 2024-09-11T07:36:47.450 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.450 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.450 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:47.450 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.011696+0000", 2024-09-11T07:36:47.451 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3929999999999999e-06 2024-09-11T07:36:47.451 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.451 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.451 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:47.451 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.014538+0000", 2024-09-11T07:36:47.451 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0028426189999999998 2024-09-11T07:36:47.451 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.451 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.451 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:47.451 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.014667+0000", 2024-09-11T07:36:47.451 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012869100000000001 2024-09-11T07:36:47.451 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.451 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.452 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,4", 2024-09-11T07:36:47.452 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.014820+0000", 2024-09-11T07:36:47.452 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015319500000000001 2024-09-11T07:36:47.452 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.452 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.452 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:36:47.452 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.016371+0000", 2024-09-11T07:36:47.452 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001551083 2024-09-11T07:36:47.452 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.452 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.452 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:47.452 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.150864+0000", 2024-09-11T07:36:47.452 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.134493115 2024-09-11T07:36:47.452 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.453 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.453 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:47.453 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.150903+0000", 2024-09-11T07:36:47.453 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8238999999999998e-05 2024-09-11T07:36:47.453 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.453 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.453 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:47.453 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.150953+0000", 2024-09-11T07:36:47.453 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.0130999999999998e-05 2024-09-11T07:36:47.453 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.453 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.453 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:47.454 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.150988+0000", 2024-09-11T07:36:47.454 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5023e-05 2024-09-11T07:36:47.454 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.454 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:47.454 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.454 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.454 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.454 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:28694 8.0 8:0c7e1fa9:::benchmark_data_smithi022_108406_object28693:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e189)", 2024-09-11T07:36:47.454 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:51.016651+0000", 2024-09-11T07:36:47.454 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 176.317394087, 2024-09-11T07:36:47.454 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.139306556, 2024-09-11T07:36:47.454 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:47.455 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:47.455 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:47.455 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:36:47.455 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:36:47.455 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 28694 2024-09-11T07:36:47.455 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.455 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:47.455 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.455 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:47.455 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.016651+0000", 2024-09-11T07:36:47.455 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.455 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.455 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.456 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:47.456 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.016651+0000", 2024-09-11T07:36:47.456 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.456 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.456 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.456 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:47.456 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.016649+0000", 2024-09-11T07:36:47.456 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:36:47.456 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.456 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.456 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:47.456 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.016673+0000", 2024-09-11T07:36:47.456 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3693000000000002e-05 2024-09-11T07:36:47.457 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.457 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.457 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:47.457 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.016682+0000", 2024-09-11T07:36:47.457 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.5310000000000007e-06 2024-09-11T07:36:47.457 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.457 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.457 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:47.457 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.016693+0000", 2024-09-11T07:36:47.457 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0978999999999999e-05 2024-09-11T07:36:47.457 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.457 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.457 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:47.458 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.018193+0000", 2024-09-11T07:36:47.458 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0014991539999999999 2024-09-11T07:36:47.458 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.458 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.458 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:47.458 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.018291+0000", 2024-09-11T07:36:47.458 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.8911000000000002e-05 2024-09-11T07:36:47.458 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.458 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.458 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,5", 2024-09-11T07:36:47.458 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.018408+0000", 2024-09-11T07:36:47.458 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000116772 2024-09-11T07:36:47.458 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.459 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.459 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:47.459 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.154883+0000", 2024-09-11T07:36:47.459 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13647477599999999 2024-09-11T07:36:47.459 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.459 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.459 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:36:47.459 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.154896+0000", 2024-09-11T07:36:47.459 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2592000000000001e-05 2024-09-11T07:36:47.459 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.459 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.459 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:47.459 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.155810+0000", 2024-09-11T07:36:47.459 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00091425899999999995 2024-09-11T07:36:47.460 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.460 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.460 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:47.460 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.155914+0000", 2024-09-11T07:36:47.460 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000104151 2024-09-11T07:36:47.460 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.460 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.460 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:47.460 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.155957+0000", 2024-09-11T07:36:47.460 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3161000000000002e-05 2024-09-11T07:36:47.460 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.460 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:47.460 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.460 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.461 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.461 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:28698 8.0 8:0331bb21:::benchmark_data_smithi022_108406_object28697:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e189)", 2024-09-11T07:36:47.461 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:51.017934+0000", 2024-09-11T07:36:47.461 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 176.31611048900001, 2024-09-11T07:36:47.461 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.14166325299999999, 2024-09-11T07:36:47.461 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:36:47.461 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:36:47.461 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:36:47.461 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:36:47.461 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:36:47.461 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 28698 2024-09-11T07:36:47.461 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.461 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:36:47.462 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.462 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:36:47.462 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.017934+0000", 2024-09-11T07:36:47.462 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.462 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.462 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.462 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:36:47.462 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.017934+0000", 2024-09-11T07:36:47.462 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:36:47.462 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.462 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.462 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:36:47.462 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.017933+0000", 2024-09-11T07:36:47.462 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:36:47.463 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.463 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.463 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:36:47.463 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.017957+0000", 2024-09-11T07:36:47.463 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4321999999999999e-05 2024-09-11T07:36:47.463 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.463 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.463 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:36:47.463 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.017967+0000", 2024-09-11T07:36:47.463 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0197000000000001e-05 2024-09-11T07:36:47.464 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.464 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.464 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:36:47.464 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.017973+0000", 2024-09-11T07:36:47.464 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9900000000000002e-06 2024-09-11T07:36:47.464 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.464 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.464 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:36:47.464 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.151440+0000", 2024-09-11T07:36:47.464 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13346665699999999 2024-09-11T07:36:47.464 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.464 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.464 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:36:47.465 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.152279+0000", 2024-09-11T07:36:47.465 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00083917499999999995 2024-09-11T07:36:47.465 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.465 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.465 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,5", 2024-09-11T07:36:47.465 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.152733+0000", 2024-09-11T07:36:47.465 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00045426400000000001 2024-09-11T07:36:47.465 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.465 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.465 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:47.465 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.156016+0000", 2024-09-11T07:36:47.466 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0032824999999999998 2024-09-11T07:36:47.466 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.466 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.466 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:36:47.466 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.156022+0000", 2024-09-11T07:36:47.466 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0929999999999999e-06 2024-09-11T07:36:47.466 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.466 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.466 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:36:47.466 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.159494+0000", 2024-09-11T07:36:47.466 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0034718180000000002 2024-09-11T07:36:47.466 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.466 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.467 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:36:47.467 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.159560+0000", 2024-09-11T07:36:47.467 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.6452999999999995e-05 2024-09-11T07:36:47.467 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:36:47.467 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:36:47.467 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:36:47.467 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:51.159597+0000", 2024-09-11T07:36:47.467 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7444999999999998e-05 2024-09-11T07:36:47.467 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.467 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:47.467 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.467 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:36:47.467 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:36:47.468 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:36:47.469 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:47.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:47.470 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:47.571 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:47.598 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:47.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:47.599 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:47.699 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:47.735 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:47.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:47.736 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:47.837 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:47.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:47.881 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:47.881 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:47.982 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:48.010 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:48.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:48.011 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:48.112 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:48.145 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:48.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:48.146 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:48.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:47 smithi022 ceph-mon[29462]: 4.10 scrub starts 2024-09-11T07:36:48.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:47 smithi022 ceph-mon[29462]: 4.10 scrub ok 2024-09-11T07:36:48.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:47 smithi022 ceph-mon[37330]: 4.10 scrub starts 2024-09-11T07:36:48.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:47 smithi022 ceph-mon[37330]: 4.10 scrub ok 2024-09-11T07:36:48.247 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:48.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:47 smithi104 ceph-mon[33507]: 4.10 scrub starts 2024-09-11T07:36:48.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:47 smithi104 ceph-mon[33507]: 4.10 scrub ok 2024-09-11T07:36:48.285 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:48.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:48.286 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:48.387 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:48.418 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:48.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:48.419 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:48.516 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-11T07:36:48.521 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:48.573 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:48.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:48.574 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:48.675 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:48.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:48.714 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:48.814 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:48.865 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:48.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:48.866 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:48.884 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:36:48.967 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:49.002 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:49.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:49.003 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:49.016 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:48 smithi104 ceph-mon[33507]: 5.11 deep-scrub starts 2024-09-11T07:36:49.016 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:48 smithi104 ceph-mon[33507]: 5.11 deep-scrub ok 2024-09-11T07:36:49.016 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:48 smithi104 ceph-mon[33507]: 2.c scrub starts 2024-09-11T07:36:49.017 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:48 smithi104 ceph-mon[33507]: 2.c scrub ok 2024-09-11T07:36:49.017 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:48 smithi104 ceph-mon[33507]: pgmap v457: 148 pgs: 148 active+clean; 1.2 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 38 KiB/s rd, 0 B/s wr, 787 op/s 2024-09-11T07:36:49.104 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:49.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:49.135 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:49.135 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:49.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:48 smithi022 ceph-mon[29462]: 5.11 deep-scrub starts 2024-09-11T07:36:49.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:48 smithi022 ceph-mon[29462]: 5.11 deep-scrub ok 2024-09-11T07:36:49.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:48 smithi022 ceph-mon[29462]: 2.c scrub starts 2024-09-11T07:36:49.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:48 smithi022 ceph-mon[29462]: 2.c scrub ok 2024-09-11T07:36:49.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:48 smithi022 ceph-mon[29462]: pgmap v457: 148 pgs: 148 active+clean; 1.2 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 38 KiB/s rd, 0 B/s wr, 787 op/s 2024-09-11T07:36:49.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:48 smithi022 ceph-mon[37330]: 5.11 deep-scrub starts 2024-09-11T07:36:49.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:48 smithi022 ceph-mon[37330]: 5.11 deep-scrub ok 2024-09-11T07:36:49.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:48 smithi022 ceph-mon[37330]: 2.c scrub starts 2024-09-11T07:36:49.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:48 smithi022 ceph-mon[37330]: 2.c scrub ok 2024-09-11T07:36:49.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:48 smithi022 ceph-mon[37330]: pgmap v457: 148 pgs: 148 active+clean; 1.2 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 38 KiB/s rd, 0 B/s wr, 787 op/s 2024-09-11T07:36:49.236 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:49.273 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:49.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:49.274 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:49.375 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:49.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:49.407 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:49.408 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:49.508 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:49.538 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:49.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:49.539 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:49.640 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:49.681 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:49.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:49.682 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:49.737 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight 3 1 2024-09-11T07:36:49.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:36:49 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:36:49.784 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:49.836 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:49.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:49.837 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:49.938 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:49.965 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:49.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:49.966 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:50.067 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:50.071 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:49 smithi104 ceph-mon[33507]: 2.5 scrub starts 2024-09-11T07:36:50.072 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:49 smithi104 ceph-mon[33507]: 2.5 scrub ok 2024-09-11T07:36:50.072 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:49 smithi104 ceph-mon[33507]: 6.1a scrub starts 2024-09-11T07:36:50.072 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:49 smithi104 ceph-mon[33507]: 6.1a scrub ok 2024-09-11T07:36:50.091 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:49 smithi022 ceph-mon[37330]: 2.5 scrub starts 2024-09-11T07:36:50.091 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:49 smithi022 ceph-mon[37330]: 2.5 scrub ok 2024-09-11T07:36:50.091 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:49 smithi022 ceph-mon[37330]: 6.1a scrub starts 2024-09-11T07:36:50.091 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:49 smithi022 ceph-mon[37330]: 6.1a scrub ok 2024-09-11T07:36:50.091 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:49 smithi022 ceph-mon[29462]: 2.5 scrub starts 2024-09-11T07:36:50.091 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:49 smithi022 ceph-mon[29462]: 2.5 scrub ok 2024-09-11T07:36:50.092 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:49 smithi022 ceph-mon[29462]: 6.1a scrub starts 2024-09-11T07:36:50.092 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:49 smithi022 ceph-mon[29462]: 6.1a scrub ok 2024-09-11T07:36:50.098 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:50.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:50.099 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:50.199 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:50.217 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:36:50.224 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:50.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:50.226 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:50.296 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:36:50.296 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:36:50.326 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:50.328 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:36:50.328 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:36:50.355 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:36:50.355 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:36:50.356 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:50.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:50.357 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:50.390 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:36:50.390 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:36:50.413 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:36:50.431 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:36:50.431 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:36:50.458 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:50.461 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:36:50.461 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:36:50.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:50.485 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:50.486 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:50.496 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:36:50.496 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:36:50.533 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:36:50.534 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:36:50.586 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:50.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:50.637 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:50.637 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:50.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:36:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:36:50] "GET /metrics HTTP/1.1" 200 224206 "" "Prometheus/2.43.0" 2024-09-11T07:36:50.738 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:50.775 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:50.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:50.777 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:50.878 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:50.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:50.914 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:50.914 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:50.938 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:36:50.938 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:36:51.015 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:51.051 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:51.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:51.052 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:51.063 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:50 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:51.063 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:50 smithi022 ceph-mon[29462]: 4.12 deep-scrub starts 2024-09-11T07:36:51.063 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:50 smithi022 ceph-mon[29462]: 4.12 deep-scrub ok 2024-09-11T07:36:51.063 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:50 smithi022 ceph-mon[29462]: 6.13 scrub starts 2024-09-11T07:36:51.063 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:50 smithi022 ceph-mon[29462]: 6.13 scrub ok 2024-09-11T07:36:51.063 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:50 smithi022 ceph-mon[29462]: 4.3 deep-scrub starts 2024-09-11T07:36:51.064 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:50 smithi022 ceph-mon[29462]: 4.3 deep-scrub ok 2024-09-11T07:36:51.064 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:50 smithi022 ceph-mon[29462]: pgmap v458: 148 pgs: 148 active+clean; 890 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 39 KiB/s rd, 0 B/s wr, 1.55k op/s 2024-09-11T07:36:51.064 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:50 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/151564629' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:36:51.064 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:50 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:51.064 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:50 smithi022 ceph-mon[37330]: 4.12 deep-scrub starts 2024-09-11T07:36:51.064 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:50 smithi022 ceph-mon[37330]: 4.12 deep-scrub ok 2024-09-11T07:36:51.064 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:50 smithi022 ceph-mon[37330]: 6.13 scrub starts 2024-09-11T07:36:51.065 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:50 smithi022 ceph-mon[37330]: 6.13 scrub ok 2024-09-11T07:36:51.065 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:50 smithi022 ceph-mon[37330]: 4.3 deep-scrub starts 2024-09-11T07:36:51.065 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:50 smithi022 ceph-mon[37330]: 4.3 deep-scrub ok 2024-09-11T07:36:51.065 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:50 smithi022 ceph-mon[37330]: pgmap v458: 148 pgs: 148 active+clean; 890 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 39 KiB/s rd, 0 B/s wr, 1.55k op/s 2024-09-11T07:36:51.065 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:50 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/151564629' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:36:51.153 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:51.190 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:51.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:51.191 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:51.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:50 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:36:51.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:50 smithi104 ceph-mon[33507]: 4.12 deep-scrub starts 2024-09-11T07:36:51.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:50 smithi104 ceph-mon[33507]: 4.12 deep-scrub ok 2024-09-11T07:36:51.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:50 smithi104 ceph-mon[33507]: 6.13 scrub starts 2024-09-11T07:36:51.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:50 smithi104 ceph-mon[33507]: 6.13 scrub ok 2024-09-11T07:36:51.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:50 smithi104 ceph-mon[33507]: 4.3 deep-scrub starts 2024-09-11T07:36:51.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:50 smithi104 ceph-mon[33507]: 4.3 deep-scrub ok 2024-09-11T07:36:51.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:50 smithi104 ceph-mon[33507]: pgmap v458: 148 pgs: 148 active+clean; 890 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 39 KiB/s rd, 0 B/s wr, 1.55k op/s 2024-09-11T07:36:51.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:50 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/151564629' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:36:51.291 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:51.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:51.330 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:51.431 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:51.469 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:51.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:51.470 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:51.571 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:51.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:51.609 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:51.609 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:51.710 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:51.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:51.751 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:51.751 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:51.823 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-11T07:36:51.844 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:51 smithi104 ceph-mon[33507]: 2.16 scrub starts 2024-09-11T07:36:51.845 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:51 smithi104 ceph-mon[33507]: 2.16 scrub ok 2024-09-11T07:36:51.845 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:51 smithi104 ceph-mon[33507]: 6.12 scrub starts 2024-09-11T07:36:51.845 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:51 smithi104 ceph-mon[33507]: 6.12 scrub ok 2024-09-11T07:36:51.845 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:51 smithi104 ceph-mon[33507]: 6.c scrub starts 2024-09-11T07:36:51.845 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:51 smithi104 ceph-mon[33507]: 6.c scrub ok 2024-09-11T07:36:51.851 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:51.889 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:51.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:51.891 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:51.991 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:52.030 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:52.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:52.031 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:52.132 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:52.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:51 smithi022 ceph-mon[29462]: 2.16 scrub starts 2024-09-11T07:36:52.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:51 smithi022 ceph-mon[29462]: 2.16 scrub ok 2024-09-11T07:36:52.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:51 smithi022 ceph-mon[29462]: 6.12 scrub starts 2024-09-11T07:36:52.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:51 smithi022 ceph-mon[29462]: 6.12 scrub ok 2024-09-11T07:36:52.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:51 smithi022 ceph-mon[29462]: 6.c scrub starts 2024-09-11T07:36:52.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:51 smithi022 ceph-mon[29462]: 6.c scrub ok 2024-09-11T07:36:52.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:51 smithi022 ceph-mon[37330]: 2.16 scrub starts 2024-09-11T07:36:52.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:51 smithi022 ceph-mon[37330]: 2.16 scrub ok 2024-09-11T07:36:52.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:51 smithi022 ceph-mon[37330]: 6.12 scrub starts 2024-09-11T07:36:52.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:51 smithi022 ceph-mon[37330]: 6.12 scrub ok 2024-09-11T07:36:52.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:51 smithi022 ceph-mon[37330]: 6.c scrub starts 2024-09-11T07:36:52.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:51 smithi022 ceph-mon[37330]: 6.c scrub ok 2024-09-11T07:36:52.170 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:52.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:52.170 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:52.199 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:36:52.272 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:52.306 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:52.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:52.308 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:52.408 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:52.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:52.449 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:52.449 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:52.550 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:52.593 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:52.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:52.594 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:52.695 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:52.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:52.729 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:52.729 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:52.830 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:52.861 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:52.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:52.862 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:52.963 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:52.994 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:52.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:52.995 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:53.016 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:52 smithi104 ceph-mon[33507]: 2.2 deep-scrub starts 2024-09-11T07:36:53.016 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:52 smithi104 ceph-mon[33507]: 2.2 deep-scrub ok 2024-09-11T07:36:53.016 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:52 smithi104 ceph-mon[33507]: 2.b scrub starts 2024-09-11T07:36:53.016 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:52 smithi104 ceph-mon[33507]: 2.b scrub ok 2024-09-11T07:36:53.016 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:52 smithi104 ceph-mon[33507]: 2.11 scrub starts 2024-09-11T07:36:53.016 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:52 smithi104 ceph-mon[33507]: 2.11 scrub ok 2024-09-11T07:36:53.016 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:52 smithi104 ceph-mon[33507]: pgmap v459: 148 pgs: 148 active+clean; 890 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 1.31k op/s 2024-09-11T07:36:53.086 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:52 smithi022 ceph-mon[29462]: 2.2 deep-scrub starts 2024-09-11T07:36:53.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:52 smithi022 ceph-mon[29462]: 2.2 deep-scrub ok 2024-09-11T07:36:53.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:52 smithi022 ceph-mon[29462]: 2.b scrub starts 2024-09-11T07:36:53.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:52 smithi022 ceph-mon[29462]: 2.b scrub ok 2024-09-11T07:36:53.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:52 smithi022 ceph-mon[29462]: 2.11 scrub starts 2024-09-11T07:36:53.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:52 smithi022 ceph-mon[29462]: 2.11 scrub ok 2024-09-11T07:36:53.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:52 smithi022 ceph-mon[29462]: pgmap v459: 148 pgs: 148 active+clean; 890 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 1.31k op/s 2024-09-11T07:36:53.120 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:52 smithi022 ceph-mon[37330]: 2.2 deep-scrub starts 2024-09-11T07:36:53.120 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:52 smithi022 ceph-mon[37330]: 2.2 deep-scrub ok 2024-09-11T07:36:53.120 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:52 smithi022 ceph-mon[37330]: 2.b scrub starts 2024-09-11T07:36:53.120 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:52 smithi022 ceph-mon[37330]: 2.b scrub ok 2024-09-11T07:36:53.120 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:52 smithi022 ceph-mon[37330]: 2.11 scrub starts 2024-09-11T07:36:53.120 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:52 smithi022 ceph-mon[37330]: 2.11 scrub ok 2024-09-11T07:36:53.120 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:52 smithi022 ceph-mon[37330]: pgmap v459: 148 pgs: 148 active+clean; 890 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 1.31k op/s 2024-09-11T07:36:53.121 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:53.159 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:53.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:53.161 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:53.261 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:53.303 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:53.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:53.305 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:53.405 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:53.436 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:53.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:53.437 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:53.538 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:53.566 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:53.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:53.567 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:53.668 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:53.714 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:53.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:53.716 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:53.817 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:53.859 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:53.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:53.860 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:53.961 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:53.996 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:53.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:53.997 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:54.098 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:54.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:54.138 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:54.138 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:54.239 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:54.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:54.277 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:54.278 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:54.280 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:36:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:36:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:36:54.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:54 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:36:54.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:54 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:36:54.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:54 smithi104 ceph-mon[33507]: 2.10 scrub starts 2024-09-11T07:36:54.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:54 smithi104 ceph-mon[33507]: 2.10 scrub ok 2024-09-11T07:36:54.378 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:54.410 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:54.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:54.411 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:54.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:54 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:36:54.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:54 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:36:54.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:54 smithi022 ceph-mon[29462]: 2.10 scrub starts 2024-09-11T07:36:54.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:54 smithi022 ceph-mon[29462]: 2.10 scrub ok 2024-09-11T07:36:54.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:54 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:36:54.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:54 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:36:54.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:54 smithi022 ceph-mon[37330]: 2.10 scrub starts 2024-09-11T07:36:54.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:54 smithi022 ceph-mon[37330]: 2.10 scrub ok 2024-09-11T07:36:54.512 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:54.560 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:54.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:54.561 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:54.585 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:36:54.585 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:36:54.662 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:54.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:54.705 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:54.706 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:54.806 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:54.843 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:54.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:54.844 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:54.871 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:36:54.945 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:54.972 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:54.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:54.973 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:55.074 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:55.095 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:55.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:55.096 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:55.197 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:55.233 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:55.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:55.234 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:55.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:55 smithi104 ceph-mon[33507]: pgmap v460: 148 pgs: 148 active+clean; 763 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 1.20k op/s 2024-09-11T07:36:55.335 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:36:55.369 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:55.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:55.371 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:36:55.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:55 smithi022 ceph-mon[29462]: pgmap v460: 148 pgs: 148 active+clean; 763 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 1.20k op/s 2024-09-11T07:36:55.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:55 smithi022 ceph-mon[37330]: pgmap v460: 148 pgs: 148 active+clean; 763 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 1.20k op/s 2024-09-11T07:36:55.424 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-11T07:36:55.471 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:55.513 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:55.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:55.514 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:55.615 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:55.653 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:55.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:55.655 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:55.756 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:55.791 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:55.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:55.792 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:55.792 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:36:55.893 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:55.931 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:55.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:55.933 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:56.034 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:56.066 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:56.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:56.067 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:56.167 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:56.203 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:56.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:56.204 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:56.220 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:56 smithi104 ceph-mon[33507]: 6.1 scrub starts 2024-09-11T07:36:56.220 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:56 smithi104 ceph-mon[33507]: 6.1 scrub ok 2024-09-11T07:36:56.220 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:56 smithi104 ceph-mon[33507]: 3.2 scrub starts 2024-09-11T07:36:56.220 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:56 smithi104 ceph-mon[33507]: 3.2 scrub ok 2024-09-11T07:36:56.220 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:56 smithi104 ceph-mon[33507]: 5.2 scrub starts 2024-09-11T07:36:56.220 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:56 smithi104 ceph-mon[33507]: 5.2 scrub ok 2024-09-11T07:36:56.305 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:56.341 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:56.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:56.342 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:56.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:56 smithi022 ceph-mon[29462]: 6.1 scrub starts 2024-09-11T07:36:56.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:56 smithi022 ceph-mon[29462]: 6.1 scrub ok 2024-09-11T07:36:56.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:56 smithi022 ceph-mon[29462]: 3.2 scrub starts 2024-09-11T07:36:56.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:56 smithi022 ceph-mon[29462]: 3.2 scrub ok 2024-09-11T07:36:56.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:56 smithi022 ceph-mon[29462]: 5.2 scrub starts 2024-09-11T07:36:56.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:56 smithi022 ceph-mon[29462]: 5.2 scrub ok 2024-09-11T07:36:56.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:56 smithi022 ceph-mon[37330]: 6.1 scrub starts 2024-09-11T07:36:56.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:56 smithi022 ceph-mon[37330]: 6.1 scrub ok 2024-09-11T07:36:56.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:56 smithi022 ceph-mon[37330]: 3.2 scrub starts 2024-09-11T07:36:56.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:56 smithi022 ceph-mon[37330]: 3.2 scrub ok 2024-09-11T07:36:56.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:56 smithi022 ceph-mon[37330]: 5.2 scrub starts 2024-09-11T07:36:56.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:56 smithi022 ceph-mon[37330]: 5.2 scrub ok 2024-09-11T07:36:56.443 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:56.464 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:56.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:56.465 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:56.566 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:56.606 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:56.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:56.608 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:56.708 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:56.734 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:56.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:56.735 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:56.835 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:56.875 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:56.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:56.876 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:56.977 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:57.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:57.008 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:57.008 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:57.109 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:57.139 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:57.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:57.140 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:57.241 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:57.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:57.274 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:57.274 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:57.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:57 smithi104 ceph-mon[33507]: 4.14 scrub starts 2024-09-11T07:36:57.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:57 smithi104 ceph-mon[33507]: 4.14 scrub ok 2024-09-11T07:36:57.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:57 smithi104 ceph-mon[33507]: pgmap v461: 148 pgs: 148 active+clean; 536 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 976 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-11T07:36:57.375 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:57.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:57.410 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:57.410 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:57.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:57 smithi022 ceph-mon[29462]: 4.14 scrub starts 2024-09-11T07:36:57.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:57 smithi022 ceph-mon[29462]: 4.14 scrub ok 2024-09-11T07:36:57.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:57 smithi022 ceph-mon[29462]: pgmap v461: 148 pgs: 148 active+clean; 536 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 976 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-11T07:36:57.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:57 smithi022 ceph-mon[37330]: 4.14 scrub starts 2024-09-11T07:36:57.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:57 smithi022 ceph-mon[37330]: 4.14 scrub ok 2024-09-11T07:36:57.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:57 smithi022 ceph-mon[37330]: pgmap v461: 148 pgs: 148 active+clean; 536 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 976 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-11T07:36:57.511 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:57.540 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:57.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:57.541 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:57.642 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:57.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:57.664 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:57.664 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:57.765 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:57.799 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:57.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:57.800 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:57.901 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:57.903 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:36:57.929 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:57.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:57.930 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:58.012 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:36:58.013 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:36:58.031 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:58.059 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is set 2024-09-11T07:36:58.065 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:58.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:58.066 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:58.093 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:58 smithi022 ceph-mon[29462]: 6.5 deep-scrub starts 2024-09-11T07:36:58.093 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:58 smithi022 ceph-mon[29462]: 6.5 deep-scrub ok 2024-09-11T07:36:58.093 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:58 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/724433185' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:36:58.093 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:58 smithi022 ceph-mon[29462]: from='client.25283 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:36:58.093 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:58 smithi022 ceph-mon[37330]: 6.5 deep-scrub starts 2024-09-11T07:36:58.093 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:58 smithi022 ceph-mon[37330]: 6.5 deep-scrub ok 2024-09-11T07:36:58.093 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:58 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/724433185' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:36:58.093 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:58 smithi022 ceph-mon[37330]: from='client.25283 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:36:58.167 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:58.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:58.197 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:58.197 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:58.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:58 smithi104 ceph-mon[33507]: 6.5 deep-scrub starts 2024-09-11T07:36:58.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:58 smithi104 ceph-mon[33507]: 6.5 deep-scrub ok 2024-09-11T07:36:58.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:58 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/724433185' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:36:58.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:58 smithi104 ceph-mon[33507]: from='client.25283 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:36:58.298 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:58.336 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:58.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:58.337 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:58.437 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:36:58.465 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:58.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:58.466 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:36:58.567 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:58.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:58.605 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:58.605 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:58.706 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:58.736 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:58.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:58.737 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:58.838 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:58.871 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:58.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:58.872 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:36:58.907 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-11T07:36:58.972 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:36:59.008 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:59.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:59.009 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:36:59.110 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:59.138 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:59.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:59.140 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:59.241 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:59.277 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:59.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:59.278 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:59.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:59 smithi104 ceph-mon[33507]: 6.15 scrub starts 2024-09-11T07:36:59.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:59 smithi104 ceph-mon[33507]: 6.15 scrub ok 2024-09-11T07:36:59.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:59 smithi104 ceph-mon[33507]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:36:59.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:59 smithi104 ceph-mon[33507]: from='client.25283 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:36:59.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:59 smithi104 ceph-mon[33507]: osdmap e237: 8 total, 8 up, 8 in 2024-09-11T07:36:59.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:36:59 smithi104 ceph-mon[33507]: pgmap v463: 148 pgs: 148 active+clean; 536 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-11T07:36:59.341 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:36:59.376 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:59 smithi022 ceph-mon[29462]: 6.15 scrub starts 2024-09-11T07:36:59.376 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:59 smithi022 ceph-mon[29462]: 6.15 scrub ok 2024-09-11T07:36:59.376 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:59 smithi022 ceph-mon[29462]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:36:59.376 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:59 smithi022 ceph-mon[29462]: from='client.25283 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:36:59.376 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:59 smithi022 ceph-mon[29462]: osdmap e237: 8 total, 8 up, 8 in 2024-09-11T07:36:59.377 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:36:59 smithi022 ceph-mon[29462]: pgmap v463: 148 pgs: 148 active+clean; 536 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-11T07:36:59.377 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:59 smithi022 ceph-mon[37330]: 6.15 scrub starts 2024-09-11T07:36:59.377 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:59 smithi022 ceph-mon[37330]: 6.15 scrub ok 2024-09-11T07:36:59.377 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:59 smithi022 ceph-mon[37330]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:36:59.377 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:59 smithi022 ceph-mon[37330]: from='client.25283 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:36:59.377 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:59 smithi022 ceph-mon[37330]: osdmap e237: 8 total, 8 up, 8 in 2024-09-11T07:36:59.377 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:36:59 smithi022 ceph-mon[37330]: pgmap v463: 148 pgs: 148 active+clean; 536 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-11T07:36:59.378 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:36:59.414 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:59.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:59.416 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:36:59.516 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:36:59.542 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:59.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:59.543 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:36:59.644 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:36:59.677 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:59.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:59.678 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:36:59.779 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:36:59.780 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:36:59 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:36:59.817 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:36:59.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:59.818 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:36:59.919 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:36:59.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:36:59.949 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:36:59.949 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:00.050 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:00.080 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:00.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:00.081 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:00.181 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:00.223 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:00.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:00.224 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:00.325 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:00.350 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:00.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:00.351 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:00.386 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:37:00 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:37:00] "GET /metrics HTTP/1.1" 200 224227 "" "Prometheus/2.43.0" 2024-09-11T07:37:00.452 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:00.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:00.495 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:00.495 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:00.596 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:00.626 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:00.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:00.627 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:00.727 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:00.767 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:00.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:00.768 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:00.869 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:00.894 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:00.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:00.895 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:00.942 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:37:00.942 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:37:00.979 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:37:00.979 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:37:00.996 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:01.018 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:37:01.018 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:37:01.033 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:01.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:01.034 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:01.056 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:37:01.056 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:37:01.090 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:37:01.090 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:37:01.121 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:37:01.121 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:37:01.134 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:01.155 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:37:01.155 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:37:01.173 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:01.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:01.174 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:01.185 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:37:01.188 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:37:01.202 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T07:37:01.233 INFO:teuthology.orchestra.run.smithi022.stderr:reweighted osd.3 to 1 (10000) 2024-09-11T07:37:01.235 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:37:01.235 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:37:01.236 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 0 2024-09-11T07:37:01.236 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:37:01.275 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:01.305 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:01.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:01.306 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:01.407 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:01.433 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:01.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:01.434 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:01.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:01 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:01.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:01 smithi104 ceph-mon[33507]: pgmap v464: 148 pgs: 148 active+clean; 171 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.15k op/s 2024-09-11T07:37:01.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:01 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1126739270' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T07:37:01.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:01 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2040121980' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:37:01.535 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:01.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:01.577 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:01.577 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:01.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:01 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:01.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:01 smithi022 ceph-mon[29462]: pgmap v464: 148 pgs: 148 active+clean; 171 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.15k op/s 2024-09-11T07:37:01.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:01 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1126739270' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T07:37:01.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:01 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2040121980' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:37:01.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:01 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:01.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:01 smithi022 ceph-mon[37330]: pgmap v464: 148 pgs: 148 active+clean; 171 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.15k op/s 2024-09-11T07:37:01.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:01 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1126739270' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T07:37:01.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:01 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2040121980' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:37:01.677 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:01.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:01.728 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:01.729 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:01.822 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Removed 128528 objects 2024-09-11T07:37:01.823 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Clean up completed and total clean up time :110.599 2024-09-11T07:37:01.829 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:01.850 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:01.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:01.851 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:01.952 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:01.955 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-11T07:37:01.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:01.973 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:01.974 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:02.075 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:02.097 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:02.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:02.098 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:02.198 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:02.216 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:37:02.224 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:02.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:02.226 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:02.241 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight 4 1 2024-09-11T07:37:02.326 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:02.349 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:02.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:02.350 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:02.450 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:02.473 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:02.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:02.474 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:02.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:02 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1126739270' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-11T07:37:02.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:02 smithi104 ceph-mon[33507]: osdmap e238: 8 total, 8 up, 8 in 2024-09-11T07:37:02.575 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:02 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1126739270' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-11T07:37:02.575 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:02 smithi022 ceph-mon[37330]: osdmap e238: 8 total, 8 up, 8 in 2024-09-11T07:37:02.575 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:02 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1126739270' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-11T07:37:02.575 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:02 smithi022 ceph-mon[29462]: osdmap e238: 8 total, 8 up, 8 in 2024-09-11T07:37:02.575 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:02.597 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:02.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:02.598 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:02.699 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:02.721 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:02.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:02.722 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:02.824 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:02.847 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:02.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:02.848 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:02.949 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:02.962 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:37:02.970 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:02.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:02.971 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:03.072 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:03.093 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:03.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:03.094 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:03.195 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:03.216 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:03.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:03.217 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:03.318 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:03.341 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:03.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:03.342 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:03.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:03 smithi104 ceph-mon[33507]: pgmap v466: 148 pgs: 148 active+clean; 171 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.18k op/s 2024-09-11T07:37:03.443 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:03.466 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:03.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:03.467 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:03.567 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:03.588 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:03.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:03.589 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:03.600 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:03 smithi022 ceph-mon[29462]: pgmap v466: 148 pgs: 148 active+clean; 171 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.18k op/s 2024-09-11T07:37:03.601 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:03 smithi022 ceph-mon[37330]: pgmap v466: 148 pgs: 148 active+clean; 171 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.18k op/s 2024-09-11T07:37:03.690 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:03.715 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:03.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:03.716 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:03.817 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:03.840 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:03.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:03.841 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:03.941 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:03.942 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:37:03.942 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:37:03.943 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2024-09-11T07:37:03.943 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2024-09-11T07:37:03.943 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:37:03.964 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:03.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:03.965 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:04.066 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:04.087 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:04.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:04.088 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:04.189 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:04.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:04.214 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:04.214 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:04.280 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:37:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:37:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:37:04.314 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:04.336 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:04.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:04.337 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:04.438 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:04.461 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:04.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:04.462 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:04.516 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:37:04.563 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:04.584 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:04.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:04.586 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:04.686 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:04.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:04.710 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:04.710 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:04.811 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:04.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:04.838 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:04.838 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:04.939 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:04.954 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:37:04.965 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:04.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:04.966 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:05.067 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:05.093 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:05.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:05.094 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:05.195 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:05.217 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:05.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:05.218 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:05.319 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:05.341 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:05.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:05.342 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:05.443 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:05.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:05.465 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:05.466 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:05.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:05 smithi022 ceph-mon[29462]: pgmap v467: 148 pgs: 148 active+clean; 100 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 873 op/s 2024-09-11T07:37:05.502 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:05 smithi022 ceph-mon[37330]: pgmap v467: 148 pgs: 148 active+clean; 100 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 873 op/s 2024-09-11T07:37:05.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:05 smithi104 ceph-mon[33507]: pgmap v467: 148 pgs: 148 active+clean; 100 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 873 op/s 2024-09-11T07:37:05.567 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:05.589 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:05.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:05.590 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:05.691 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:05.713 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:05.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:05.714 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:05.764 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-11T07:37:05.815 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:05.839 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:05.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:05.840 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:05.941 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:05.963 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:05.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:05.964 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:06.006 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:37:06.065 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:06.087 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:06.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:06.088 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:06.189 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:06.211 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:06.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:06.212 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:06.245 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:37:06.312 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:06.335 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:06.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:06.336 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:06.437 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:06.460 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:06.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:06.465 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:06.566 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:06.587 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:06.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:06.588 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:06.596 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is set 2024-09-11T07:37:06.689 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:06.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:06.713 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:06.713 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:06.740 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:06 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2725381082' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:37:06.740 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:06 smithi022 ceph-mon[29462]: pgmap v468: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.06k op/s 2024-09-11T07:37:06.740 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:06 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2725381082' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:37:06.740 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:06 smithi022 ceph-mon[37330]: pgmap v468: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.06k op/s 2024-09-11T07:37:06.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:06 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2725381082' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:37:06.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:06 smithi104 ceph-mon[33507]: pgmap v468: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.06k op/s 2024-09-11T07:37:06.817 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:06.839 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:06.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:06.840 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:06.941 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:06.963 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:06.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:06.964 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:07.065 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:07.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:07.089 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:07.089 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:07.190 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:07.212 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:07.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:07.213 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:07.314 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:07.338 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:07.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:07.339 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:07.440 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:07.463 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:07.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:07.464 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:07.564 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:07.586 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_1 2024-09-11T07:37:07.587 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pool rm unique_pool_1 unique_pool_1 --yes-i-really-really-mean-it 2024-09-11T07:37:07.588 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:07.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:07.589 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:07.597 INFO:teuthology.orchestra.run.smithi022.stderr:reweighted osd.4 to 1 (10000) 2024-09-11T07:37:07.690 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:07.712 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:07.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:07.713 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:07.732 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:07 smithi104 ceph-mon[33507]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:37:07.732 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:07 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2725381082' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:37:07.732 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:07 smithi104 ceph-mon[33507]: osdmap e239: 8 total, 8 up, 8 in 2024-09-11T07:37:07.732 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:07 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/353833050' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T07:37:07.773 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:07 smithi022 ceph-mon[29462]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:37:07.773 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:07 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2725381082' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:37:07.773 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:07 smithi022 ceph-mon[29462]: osdmap e239: 8 total, 8 up, 8 in 2024-09-11T07:37:07.773 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:07 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/353833050' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T07:37:07.773 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:07 smithi022 ceph-mon[37330]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:37:07.773 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:07 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2725381082' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:37:07.773 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:07 smithi022 ceph-mon[37330]: osdmap e239: 8 total, 8 up, 8 in 2024-09-11T07:37:07.773 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:07 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/353833050' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T07:37:07.814 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:07.829 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:37:07.829 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2024-09-11T07:37:07.829 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2024-09-11T07:37:07.830 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2024-09-11T07:37:07.830 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.830 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:119878 7.0 7:0d5a1409:::benchmark_data_smithi022_77720_object119877:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-11T07:37:07.830 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:30:28.063301+0000", 2024-09-11T07:37:07.830 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 399.76345804699997, 2024-09-11T07:37:07.830 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.12935533599999999, 2024-09-11T07:37:07.830 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:07.830 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:07.830 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:37:07.831 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:37:07.831 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:37:07.831 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 119878 2024-09-11T07:37:07.831 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.831 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:07.831 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.831 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:07.831 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063301+0000", 2024-09-11T07:37:07.831 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.831 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.831 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.831 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:07.832 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063301+0000", 2024-09-11T07:37:07.832 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.832 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.832 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.832 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:07.832 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063298+0000", 2024-09-11T07:37:07.832 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:37:07.832 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.832 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.832 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:07.832 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063402+0000", 2024-09-11T07:37:07.832 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010310999999999999 2024-09-11T07:37:07.833 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.833 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.833 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:07.833 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063414+0000", 2024-09-11T07:37:07.833 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2237e-05 2024-09-11T07:37:07.833 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.833 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.833 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:07.833 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063419+0000", 2024-09-11T07:37:07.833 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5489999999999997e-06 2024-09-11T07:37:07.833 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.834 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.834 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:07.834 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063454+0000", 2024-09-11T07:37:07.834 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5132000000000002e-05 2024-09-11T07:37:07.834 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.835 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.835 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:07.835 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063565+0000", 2024-09-11T07:37:07.835 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000110246 2024-09-11T07:37:07.835 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.835 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.836 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:37:07.836 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.063651+0000", 2024-09-11T07:37:07.836 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6661000000000003e-05 2024-09-11T07:37:07.836 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.836 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.836 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:37:07.836 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.066483+0000", 2024-09-11T07:37:07.837 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0028320490000000001 2024-09-11T07:37:07.837 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.837 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.837 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.837 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.066815+0000", 2024-09-11T07:37:07.837 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00033145700000000001 2024-09-11T07:37:07.838 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.838 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.838 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.838 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.192594+0000", 2024-09-11T07:37:07.838 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.12577914400000001 2024-09-11T07:37:07.838 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.838 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.838 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:07.839 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.192633+0000", 2024-09-11T07:37:07.839 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9042999999999999e-05 2024-09-11T07:37:07.839 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.839 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.839 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:07.839 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:28.192656+0000", 2024-09-11T07:37:07.840 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3125999999999998e-05 2024-09-11T07:37:07.840 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.840 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:07.840 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.840 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.840 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.840 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:121102 7.0 7:09266ef0:::benchmark_data_smithi022_77720_object121101:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:37:07.841 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:30:29.710752+0000", 2024-09-11T07:37:07.841 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 398.11600690199998, 2024-09-11T07:37:07.841 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.121706142, 2024-09-11T07:37:07.841 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:07.841 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:07.841 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:37:07.842 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:37:07.842 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:37:07.842 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 121102 2024-09-11T07:37:07.842 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.842 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:07.842 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.842 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:07.842 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.710752+0000", 2024-09-11T07:37:07.842 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.842 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.842 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.842 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:07.842 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.710752+0000", 2024-09-11T07:37:07.843 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.843 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.843 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.843 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:07.843 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.710749+0000", 2024-09-11T07:37:07.843 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:37:07.843 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.843 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.843 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:07.843 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.710809+0000", 2024-09-11T07:37:07.843 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9255000000000001e-05 2024-09-11T07:37:07.843 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.844 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.844 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:07.844 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.710832+0000", 2024-09-11T07:37:07.844 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3031e-05 2024-09-11T07:37:07.844 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.844 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.844 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:07.844 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.710839+0000", 2024-09-11T07:37:07.844 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3889999999999999e-06 2024-09-11T07:37:07.844 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.844 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.844 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:07.844 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.710883+0000", 2024-09-11T07:37:07.844 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3770999999999998e-05 2024-09-11T07:37:07.845 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.845 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.845 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:07.845 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.711002+0000", 2024-09-11T07:37:07.845 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000119518 2024-09-11T07:37:07.845 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.845 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.845 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:37:07.845 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.711141+0000", 2024-09-11T07:37:07.845 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013841 2024-09-11T07:37:07.845 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.845 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.845 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:37:07.845 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.713588+0000", 2024-09-11T07:37:07.846 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0024468179999999999 2024-09-11T07:37:07.846 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.846 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.846 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.846 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.713875+0000", 2024-09-11T07:37:07.846 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00028783500000000001 2024-09-11T07:37:07.846 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.846 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.846 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.846 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.832376+0000", 2024-09-11T07:37:07.846 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.11850076800000001 2024-09-11T07:37:07.846 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.846 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.846 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:07.846 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.832424+0000", 2024-09-11T07:37:07.846 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8047999999999998e-05 2024-09-11T07:37:07.847 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.847 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.847 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:07.847 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.832458+0000", 2024-09-11T07:37:07.847 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3849000000000001e-05 2024-09-11T07:37:07.847 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.847 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:07.847 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.847 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.847 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.847 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:173291 7.0 7:0bf19ddd:::benchmark_data_smithi022_77720_object49536:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-11T07:37:07.847 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:31:20.297211+0000", 2024-09-11T07:37:07.847 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 347.52954781699998, 2024-09-11T07:37:07.847 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.110572034, 2024-09-11T07:37:07.848 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:07.848 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:07.848 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:37:07.848 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:37:07.848 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:37:07.848 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 173291 2024-09-11T07:37:07.849 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.849 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:07.849 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.849 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:07.849 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.297211+0000", 2024-09-11T07:37:07.849 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.849 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.849 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.849 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:07.849 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.297211+0000", 2024-09-11T07:37:07.849 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.849 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.849 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.850 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:07.850 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.297209+0000", 2024-09-11T07:37:07.850 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:37:07.850 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.850 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.850 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:07.850 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.297216+0000", 2024-09-11T07:37:07.850 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5420000000000002e-06 2024-09-11T07:37:07.850 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.850 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.851 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:07.851 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.297226+0000", 2024-09-11T07:37:07.851 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0111e-05 2024-09-11T07:37:07.851 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.851 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.851 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:07.851 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.297231+0000", 2024-09-11T07:37:07.851 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5570000000000003e-06 2024-09-11T07:37:07.851 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.851 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.851 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:07.851 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.297264+0000", 2024-09-11T07:37:07.851 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2778999999999999e-05 2024-09-11T07:37:07.851 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.852 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.852 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:07.852 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.297359+0000", 2024-09-11T07:37:07.852 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5050999999999995e-05 2024-09-11T07:37:07.852 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.852 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.852 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:37:07.852 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.297458+0000", 2024-09-11T07:37:07.852 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8609000000000004e-05 2024-09-11T07:37:07.852 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.852 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.853 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:37:07.853 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.301513+0000", 2024-09-11T07:37:07.853 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0040550480000000003 2024-09-11T07:37:07.853 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.853 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.853 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.853 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.302635+0000", 2024-09-11T07:37:07.853 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0011224850000000001 2024-09-11T07:37:07.853 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.854 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.854 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.854 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.407729+0000", 2024-09-11T07:37:07.854 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.10509397500000001 2024-09-11T07:37:07.854 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.854 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.854 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:07.854 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.407759+0000", 2024-09-11T07:37:07.854 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9536999999999999e-05 2024-09-11T07:37:07.854 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.854 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.854 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:07.854 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.407783+0000", 2024-09-11T07:37:07.855 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.4362000000000002e-05 2024-09-11T07:37:07.855 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.855 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:07.855 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.855 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.855 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.855 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:173295 7.0 7:0b0dd3c6:::benchmark_data_smithi022_77720_object49540:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139)", 2024-09-11T07:37:07.855 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:31:20.303629+0000", 2024-09-11T07:37:07.855 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 347.52312985399999, 2024-09-11T07:37:07.855 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.104283681, 2024-09-11T07:37:07.855 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:07.855 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:07.856 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:37:07.856 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:37:07.856 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:37:07.856 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 173295 2024-09-11T07:37:07.856 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.856 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:07.856 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.856 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:07.856 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.303629+0000", 2024-09-11T07:37:07.856 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.856 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.856 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.856 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:07.857 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.303629+0000", 2024-09-11T07:37:07.857 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.857 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.857 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.857 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:07.857 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.303627+0000", 2024-09-11T07:37:07.857 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:37:07.857 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.857 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.857 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:07.857 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.303635+0000", 2024-09-11T07:37:07.857 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3100000000000003e-06 2024-09-11T07:37:07.857 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.858 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.858 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:07.858 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.303644+0000", 2024-09-11T07:37:07.858 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5799999999999998e-06 2024-09-11T07:37:07.858 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.858 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.858 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:07.858 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.303648+0000", 2024-09-11T07:37:07.858 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.002e-06 2024-09-11T07:37:07.858 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.858 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.858 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:07.858 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.303675+0000", 2024-09-11T07:37:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.6441999999999999e-05 2024-09-11T07:37:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.303764+0000", 2024-09-11T07:37:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9085999999999994e-05 2024-09-11T07:37:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:37:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.303841+0000", 2024-09-11T07:37:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7179000000000001e-05 2024-09-11T07:37:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:37:07.859 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.306671+0000", 2024-09-11T07:37:07.860 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0028303130000000001 2024-09-11T07:37:07.860 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.860 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.860 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.860 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.307717+0000", 2024-09-11T07:37:07.860 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001046118 2024-09-11T07:37:07.860 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.860 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.860 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.860 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.407819+0000", 2024-09-11T07:37:07.860 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.10010218799999999 2024-09-11T07:37:07.860 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.860 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.861 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:07.861 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.407876+0000", 2024-09-11T07:37:07.861 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6560000000000001e-05 2024-09-11T07:37:07.861 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.861 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.861 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:07.861 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:20.407913+0000", 2024-09-11T07:37:07.861 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6704e-05 2024-09-11T07:37:07.861 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.861 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:07.861 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.861 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.861 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.862 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:189698 7.0 7:0c18b714:::benchmark_data_smithi022_77720_object65943:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-11T07:37:07.862 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:31:36.241692+0000", 2024-09-11T07:37:07.862 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 331.585066412, 2024-09-11T07:37:07.862 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.074615498000000002, 2024-09-11T07:37:07.862 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:07.862 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:07.862 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:37:07.862 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:37:07.862 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:37:07.862 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 189698 2024-09-11T07:37:07.862 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.862 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:07.862 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.862 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:07.863 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.241692+0000", 2024-09-11T07:37:07.863 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.863 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.863 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.863 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:07.863 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.241692+0000", 2024-09-11T07:37:07.863 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.863 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.863 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.863 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:07.863 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.241690+0000", 2024-09-11T07:37:07.863 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999971 2024-09-11T07:37:07.863 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.863 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.864 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:07.864 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.241701+0000", 2024-09-11T07:37:07.864 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1127e-05 2024-09-11T07:37:07.864 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.864 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.864 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:07.864 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.241722+0000", 2024-09-11T07:37:07.864 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0616e-05 2024-09-11T07:37:07.864 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.864 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.865 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:07.865 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.241731+0000", 2024-09-11T07:37:07.865 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8030000000000008e-06 2024-09-11T07:37:07.865 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.865 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.865 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:07.865 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.241811+0000", 2024-09-11T07:37:07.865 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9170000000000003e-05 2024-09-11T07:37:07.865 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.865 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.865 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:07.865 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.242037+0000", 2024-09-11T07:37:07.865 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00022672100000000001 2024-09-11T07:37:07.866 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.866 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.866 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:37:07.866 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.242170+0000", 2024-09-11T07:37:07.866 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000132702 2024-09-11T07:37:07.866 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.866 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.866 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:37:07.866 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.244416+0000", 2024-09-11T07:37:07.866 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0022463769999999999 2024-09-11T07:37:07.866 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.866 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.866 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.866 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.251396+0000", 2024-09-11T07:37:07.867 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0069796290000000002 2024-09-11T07:37:07.867 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.867 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.867 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.867 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.316254+0000", 2024-09-11T07:37:07.867 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.064858117000000007 2024-09-11T07:37:07.867 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.867 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.867 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:07.867 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.316284+0000", 2024-09-11T07:37:07.867 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.025e-05 2024-09-11T07:37:07.867 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.867 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.867 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:07.867 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.316308+0000", 2024-09-11T07:37:07.868 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3632999999999999e-05 2024-09-11T07:37:07.868 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.868 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:07.868 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.868 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.868 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.868 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:189702 7.0 7:03f43c7a:::benchmark_data_smithi022_77720_object65947:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-11T07:37:07.868 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:31:36.247101+0000", 2024-09-11T07:37:07.868 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 331.57965745799999, 2024-09-11T07:37:07.868 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.073677724999999999, 2024-09-11T07:37:07.868 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:07.868 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:07.868 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:37:07.868 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:37:07.869 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:37:07.869 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 189702 2024-09-11T07:37:07.869 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.869 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:07.869 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.869 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:07.869 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247101+0000", 2024-09-11T07:37:07.869 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.869 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.869 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.869 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:07.869 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247101+0000", 2024-09-11T07:37:07.869 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.870 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.870 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.870 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:07.870 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247100+0000", 2024-09-11T07:37:07.870 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:07.870 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.870 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.870 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:07.870 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247108+0000", 2024-09-11T07:37:07.870 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3780000000000001e-06 2024-09-11T07:37:07.871 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.871 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.871 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:07.871 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247120+0000", 2024-09-11T07:37:07.871 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1558e-05 2024-09-11T07:37:07.871 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.871 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.871 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:07.871 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247126+0000", 2024-09-11T07:37:07.871 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2260000000000002e-06 2024-09-11T07:37:07.871 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.871 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.872 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:07.872 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247166+0000", 2024-09-11T07:37:07.872 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0009000000000003e-05 2024-09-11T07:37:07.872 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.872 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.872 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:07.872 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247238+0000", 2024-09-11T07:37:07.872 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.2008999999999994e-05 2024-09-11T07:37:07.873 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.873 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.873 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:37:07.873 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.247302+0000", 2024-09-11T07:37:07.873 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4257000000000005e-05 2024-09-11T07:37:07.873 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.873 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.873 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:37:07.873 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.249620+0000", 2024-09-11T07:37:07.873 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0023178830000000002 2024-09-11T07:37:07.873 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.873 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.873 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.873 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.251426+0000", 2024-09-11T07:37:07.874 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0018057170000000001 2024-09-11T07:37:07.874 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.874 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.874 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.874 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.320727+0000", 2024-09-11T07:37:07.874 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.069301077000000003 2024-09-11T07:37:07.874 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.874 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.874 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:07.874 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.320756+0000", 2024-09-11T07:37:07.874 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9115000000000002e-05 2024-09-11T07:37:07.874 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.874 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.874 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:07.874 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:31:36.320779+0000", 2024-09-11T07:37:07.875 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3125999999999998e-05 2024-09-11T07:37:07.875 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.875 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:07.875 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.875 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.875 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.875 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:13641 8.8 8:1997f03a:::benchmark_data_smithi022_108406_object13640:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-11T07:37:07.875 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:33:44.755776+0000", 2024-09-11T07:37:07.875 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 203.07098247100001, 2024-09-11T07:37:07.875 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.102084483, 2024-09-11T07:37:07.875 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:07.875 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:07.875 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:37:07.876 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:37:07.876 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:37:07.876 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 13641 2024-09-11T07:37:07.876 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.876 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:07.876 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.876 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:07.876 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.755776+0000", 2024-09-11T07:37:07.876 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.877 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.877 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.877 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:07.877 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.755776+0000", 2024-09-11T07:37:07.877 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.877 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.877 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.877 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:07.877 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.755775+0000", 2024-09-11T07:37:07.877 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:07.877 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.877 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.877 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:07.878 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.755850+0000", 2024-09-11T07:37:07.878 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5088999999999999e-05 2024-09-11T07:37:07.878 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.878 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.878 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:07.878 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.755867+0000", 2024-09-11T07:37:07.878 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7320999999999999e-05 2024-09-11T07:37:07.878 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.878 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.878 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:07.878 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.755874+0000", 2024-09-11T07:37:07.878 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3629999999999999e-06 2024-09-11T07:37:07.878 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.879 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.879 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:07.879 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.755928+0000", 2024-09-11T07:37:07.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:07.880 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:07.880 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:07.881 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4639000000000003e-05 2024-09-11T07:37:07.881 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.881 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.881 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:07.882 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.756083+0000", 2024-09-11T07:37:07.882 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015470800000000001 2024-09-11T07:37:07.882 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.882 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.882 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:37:07.882 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.756225+0000", 2024-09-11T07:37:07.882 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014204500000000001 2024-09-11T07:37:07.882 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.882 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.882 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.882 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.757465+0000", 2024-09-11T07:37:07.882 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0012398120000000001 2024-09-11T07:37:07.883 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.883 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.883 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:37:07.883 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.757596+0000", 2024-09-11T07:37:07.883 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000131006 2024-09-11T07:37:07.883 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.883 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.883 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.883 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.857783+0000", 2024-09-11T07:37:07.883 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.100186975 2024-09-11T07:37:07.883 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.883 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.884 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:07.884 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.857812+0000", 2024-09-11T07:37:07.884 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9371000000000001e-05 2024-09-11T07:37:07.884 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.884 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.884 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:07.884 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.857861+0000", 2024-09-11T07:37:07.884 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8606999999999997e-05 2024-09-11T07:37:07.884 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.885 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:07.885 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.885 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.885 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.885 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:13647 8.8 8:137d89d6:::benchmark_data_smithi022_108406_object13646:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-11T07:37:07.885 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:33:44.757758+0000", 2024-09-11T07:37:07.885 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 203.069000668, 2024-09-11T07:37:07.885 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.10313436400000001, 2024-09-11T07:37:07.885 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:07.885 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:07.885 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:37:07.885 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:37:07.886 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:37:07.886 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 13647 2024-09-11T07:37:07.886 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.886 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:07.886 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.886 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:07.886 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.757758+0000", 2024-09-11T07:37:07.886 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.886 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.886 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.886 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:07.886 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.757758+0000", 2024-09-11T07:37:07.886 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.887 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.887 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.887 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:07.887 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.757757+0000", 2024-09-11T07:37:07.887 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:07.887 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.887 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.887 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:07.887 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.757796+0000", 2024-09-11T07:37:07.887 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9663999999999998e-05 2024-09-11T07:37:07.887 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.888 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.888 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:07.888 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.757812+0000", 2024-09-11T07:37:07.888 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5622000000000001e-05 2024-09-11T07:37:07.888 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.888 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.888 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:07.888 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.757819+0000", 2024-09-11T07:37:07.888 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6340000000000003e-06 2024-09-11T07:37:07.888 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.888 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.888 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:07.889 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.758238+0000", 2024-09-11T07:37:07.889 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00041897199999999999 2024-09-11T07:37:07.889 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.889 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.889 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:07.889 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.758356+0000", 2024-09-11T07:37:07.889 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000118644 2024-09-11T07:37:07.889 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.889 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.889 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:37:07.890 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.758523+0000", 2024-09-11T07:37:07.890 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016667200000000001 2024-09-11T07:37:07.890 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.890 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.890 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:37:07.890 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.759951+0000", 2024-09-11T07:37:07.890 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0014284639999999999 2024-09-11T07:37:07.890 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.890 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.890 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.890 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.760052+0000", 2024-09-11T07:37:07.890 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000100388 2024-09-11T07:37:07.890 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.891 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.891 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.891 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.860800+0000", 2024-09-11T07:37:07.891 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.100747955 2024-09-11T07:37:07.891 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.891 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.891 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:07.891 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.860855+0000", 2024-09-11T07:37:07.891 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5123000000000001e-05 2024-09-11T07:37:07.891 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.891 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.892 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:07.892 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:44.860893+0000", 2024-09-11T07:37:07.892 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7762000000000002e-05 2024-09-11T07:37:07.892 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.892 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:07.892 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.892 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.892 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.892 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:28689 8.8 8:1221b283:::benchmark_data_smithi022_108406_object28688:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e189)", 2024-09-11T07:37:07.892 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:33:51.015983+0000", 2024-09-11T07:37:07.892 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 196.810776358, 2024-09-11T07:37:07.892 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13574931500000001, 2024-09-11T07:37:07.893 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:07.893 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:07.893 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:37:07.893 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:37:07.893 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:37:07.893 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 28689 2024-09-11T07:37:07.893 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.893 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:07.893 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.893 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:07.893 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.015983+0000", 2024-09-11T07:37:07.894 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.894 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.894 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.894 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:07.894 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.015983+0000", 2024-09-11T07:37:07.894 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.894 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.894 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.894 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:07.894 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.015981+0000", 2024-09-11T07:37:07.894 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:07.894 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.894 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.894 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:07.895 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.016014+0000", 2024-09-11T07:37:07.895 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3164999999999998e-05 2024-09-11T07:37:07.895 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.895 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.895 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:07.895 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.016026+0000", 2024-09-11T07:37:07.895 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1994000000000001e-05 2024-09-11T07:37:07.895 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.895 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.895 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:07.895 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.016032+0000", 2024-09-11T07:37:07.895 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8220000000000003e-06 2024-09-11T07:37:07.895 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.896 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.896 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:07.896 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.016061+0000", 2024-09-11T07:37:07.896 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.8654e-05 2024-09-11T07:37:07.896 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.896 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.896 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:07.896 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.016179+0000", 2024-09-11T07:37:07.896 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000117959 2024-09-11T07:37:07.896 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.897 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.897 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:37:07.897 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.016311+0000", 2024-09-11T07:37:07.897 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013222200000000001 2024-09-11T07:37:07.897 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.897 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.897 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.897 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.018355+0000", 2024-09-11T07:37:07.897 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002044466 2024-09-11T07:37:07.897 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.897 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.897 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:37:07.898 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.019134+0000", 2024-09-11T07:37:07.898 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00077916300000000003 2024-09-11T07:37:07.898 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.898 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.898 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.898 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.151654+0000", 2024-09-11T07:37:07.898 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.132519682 2024-09-11T07:37:07.898 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.898 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.898 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:07.898 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.151703+0000", 2024-09-11T07:37:07.898 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8458999999999998e-05 2024-09-11T07:37:07.898 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.898 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.899 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:07.899 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.151732+0000", 2024-09-11T07:37:07.899 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9286999999999999e-05 2024-09-11T07:37:07.899 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.899 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:07.899 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.899 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.899 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.899 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:28706 8.8 8:14c85bdd:::benchmark_data_smithi022_108406_object28705:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e189)", 2024-09-11T07:37:07.899 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:33:51.022734+0000", 2024-09-11T07:37:07.899 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 196.804024427, 2024-09-11T07:37:07.899 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.135307186, 2024-09-11T07:37:07.899 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:07.899 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:07.899 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:37:07.900 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:37:07.900 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:37:07.900 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 28706 2024-09-11T07:37:07.900 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.900 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:07.900 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.900 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:07.900 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.022734+0000", 2024-09-11T07:37:07.900 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.900 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.900 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.900 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:07.900 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.022734+0000", 2024-09-11T07:37:07.901 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.901 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.901 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.901 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:07.901 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.022733+0000", 2024-09-11T07:37:07.901 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:07.901 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.901 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.901 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:07.901 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.022788+0000", 2024-09-11T07:37:07.901 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4837999999999999e-05 2024-09-11T07:37:07.901 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.901 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.902 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:07.902 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.022796+0000", 2024-09-11T07:37:07.902 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6589999999999996e-06 2024-09-11T07:37:07.902 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.902 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.902 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:07.902 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.022801+0000", 2024-09-11T07:37:07.902 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3089999999999997e-06 2024-09-11T07:37:07.902 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.902 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.902 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:07.902 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.022849+0000", 2024-09-11T07:37:07.902 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8432e-05 2024-09-11T07:37:07.902 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.902 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.903 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:07.903 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.022966+0000", 2024-09-11T07:37:07.903 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011678100000000001 2024-09-11T07:37:07.903 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.903 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.903 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:37:07.903 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.023103+0000", 2024-09-11T07:37:07.903 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013668900000000001 2024-09-11T07:37:07.903 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.903 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.903 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:37:07.903 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.024203+0000", 2024-09-11T07:37:07.903 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001100539 2024-09-11T07:37:07.903 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.904 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.904 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.904 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.024892+0000", 2024-09-11T07:37:07.904 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00068921200000000003 2024-09-11T07:37:07.904 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.904 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.904 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.904 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.157959+0000", 2024-09-11T07:37:07.904 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13306637800000001 2024-09-11T07:37:07.904 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.904 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.904 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:07.904 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.158003+0000", 2024-09-11T07:37:07.905 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4206999999999999e-05 2024-09-11T07:37:07.905 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.905 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.905 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:07.905 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.158042+0000", 2024-09-11T07:37:07.905 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8628999999999999e-05 2024-09-11T07:37:07.905 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.905 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:07.905 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.905 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.905 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.905 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:29825 8.8 8:1c95b3ee:::benchmark_data_smithi022_108406_object29824:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e189)", 2024-09-11T07:37:07.905 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:33:51.652793+0000", 2024-09-11T07:37:07.906 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 196.173965936, 2024-09-11T07:37:07.906 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.075804250000000004, 2024-09-11T07:37:07.906 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:07.906 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:07.906 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:37:07.906 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:37:07.906 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:37:07.906 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 29825 2024-09-11T07:37:07.906 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.906 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:07.906 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.906 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:07.906 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.652793+0000", 2024-09-11T07:37:07.906 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.906 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.907 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.907 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:07.907 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.652793+0000", 2024-09-11T07:37:07.907 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.907 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.907 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.907 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:07.907 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.652792+0000", 2024-09-11T07:37:07.907 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:37:07.907 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.907 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.907 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:07.907 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.652818+0000", 2024-09-11T07:37:07.907 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.6431000000000002e-05 2024-09-11T07:37:07.908 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.908 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.908 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:07.908 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.653034+0000", 2024-09-11T07:37:07.908 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000215847 2024-09-11T07:37:07.908 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.908 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.908 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:07.908 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.653039+0000", 2024-09-11T07:37:07.908 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6369999999999998e-06 2024-09-11T07:37:07.908 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.908 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.908 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:07.908 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.656316+0000", 2024-09-11T07:37:07.909 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003276827 2024-09-11T07:37:07.909 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.909 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.909 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:07.909 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.656465+0000", 2024-09-11T07:37:07.909 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014931300000000001 2024-09-11T07:37:07.909 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.909 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.909 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:37:07.909 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.657134+0000", 2024-09-11T07:37:07.909 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00066848799999999996 2024-09-11T07:37:07.909 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.909 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.910 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:37:07.910 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.658284+0000", 2024-09-11T07:37:07.910 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0011505319999999999 2024-09-11T07:37:07.910 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.910 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.910 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.910 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.658468+0000", 2024-09-11T07:37:07.910 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000183439 2024-09-11T07:37:07.910 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.910 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.910 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.910 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.728499+0000", 2024-09-11T07:37:07.910 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.070031117000000004 2024-09-11T07:37:07.910 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.911 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.911 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:07.911 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.728555+0000", 2024-09-11T07:37:07.911 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6110999999999999e-05 2024-09-11T07:37:07.911 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.911 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.911 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:07.911 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.728597+0000", 2024-09-11T07:37:07.911 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2400000000000001e-05 2024-09-11T07:37:07.911 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.911 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:07.911 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.911 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.912 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.912 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:31918 8.8 8:16058c3f:::benchmark_data_smithi022_108406_object31917:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e190)", 2024-09-11T07:37:07.912 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:33:52.793111+0000", 2024-09-11T07:37:07.912 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 195.03364764400001, 2024-09-11T07:37:07.912 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.11028637400000001, 2024-09-11T07:37:07.912 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:07.912 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:07.912 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:37:07.912 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:37:07.912 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:37:07.913 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 31918 2024-09-11T07:37:07.913 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.913 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:07.913 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.913 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:07.913 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.793111+0000", 2024-09-11T07:37:07.913 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.913 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.913 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.913 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:07.913 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.793111+0000", 2024-09-11T07:37:07.913 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.913 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.914 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.914 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:07.914 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.793104+0000", 2024-09-11T07:37:07.914 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999933 2024-09-11T07:37:07.914 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.914 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.914 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:07.914 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.793166+0000", 2024-09-11T07:37:07.915 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1296000000000004e-05 2024-09-11T07:37:07.915 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.915 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.915 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:07.915 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.793177+0000", 2024-09-11T07:37:07.915 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1123e-05 2024-09-11T07:37:07.915 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.915 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.915 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:07.915 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.793183+0000", 2024-09-11T07:37:07.916 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7409999999999998e-06 2024-09-11T07:37:07.916 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.916 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.916 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:07.916 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.793218+0000", 2024-09-11T07:37:07.916 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5797999999999998e-05 2024-09-11T07:37:07.916 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.916 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.916 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:07.916 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.793300+0000", 2024-09-11T07:37:07.916 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2150999999999993e-05 2024-09-11T07:37:07.916 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.917 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.917 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:37:07.917 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.793381+0000", 2024-09-11T07:37:07.917 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0320000000000003e-05 2024-09-11T07:37:07.917 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.917 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.917 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.917 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.796146+0000", 2024-09-11T07:37:07.917 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0027649979999999998 2024-09-11T07:37:07.917 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.917 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.917 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:37:07.917 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.797178+0000", 2024-09-11T07:37:07.918 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001032406 2024-09-11T07:37:07.918 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.918 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.918 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.918 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.903313+0000", 2024-09-11T07:37:07.918 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.106135013 2024-09-11T07:37:07.918 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.918 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.918 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:07.918 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.903362+0000", 2024-09-11T07:37:07.918 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9149e-05 2024-09-11T07:37:07.918 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.919 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.919 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:07.919 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:52.903398+0000", 2024-09-11T07:37:07.919 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5225000000000002e-05 2024-09-11T07:37:07.919 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.919 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:07.919 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.919 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.919 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.919 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:36507 8.8 8:11b408ae:::benchmark_data_smithi022_108406_object36506:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e191)", 2024-09-11T07:37:07.919 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:33:55.569627+0000", 2024-09-11T07:37:07.919 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 192.25713204100001, 2024-09-11T07:37:07.919 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13017671, 2024-09-11T07:37:07.919 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:07.920 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:07.920 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:37:07.920 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:37:07.920 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:37:07.920 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 36507 2024-09-11T07:37:07.920 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.920 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:07.920 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.920 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:07.920 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.569627+0000", 2024-09-11T07:37:07.920 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.920 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.920 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.921 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:07.921 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.569627+0000", 2024-09-11T07:37:07.921 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.921 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.921 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.921 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:07.921 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.569624+0000", 2024-09-11T07:37:07.921 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:37:07.921 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.921 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.921 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:07.921 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.569659+0000", 2024-09-11T07:37:07.921 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4276000000000002e-05 2024-09-11T07:37:07.922 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.922 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.922 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:07.922 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.569669+0000", 2024-09-11T07:37:07.922 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0431000000000001e-05 2024-09-11T07:37:07.922 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.922 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.922 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:07.922 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.569675+0000", 2024-09-11T07:37:07.922 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4419999999999997e-06 2024-09-11T07:37:07.922 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.922 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.922 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:07.923 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.569726+0000", 2024-09-11T07:37:07.923 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1289999999999999e-05 2024-09-11T07:37:07.923 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.923 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.923 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:07.923 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.569870+0000", 2024-09-11T07:37:07.923 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000143738 2024-09-11T07:37:07.923 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.923 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.923 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:37:07.923 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.570008+0000", 2024-09-11T07:37:07.923 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000138701 2024-09-11T07:37:07.923 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.923 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.923 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:37:07.924 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.571300+0000", 2024-09-11T07:37:07.924 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0012921969999999999 2024-09-11T07:37:07.924 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.924 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.924 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.924 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.571517+0000", 2024-09-11T07:37:07.924 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00021640200000000001 2024-09-11T07:37:07.924 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.924 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.924 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.924 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.699718+0000", 2024-09-11T07:37:07.924 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.128201499 2024-09-11T07:37:07.924 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.925 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.925 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:07.925 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.699764+0000", 2024-09-11T07:37:07.925 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5680000000000003e-05 2024-09-11T07:37:07.925 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.925 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.925 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:07.925 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:55.699804+0000", 2024-09-11T07:37:07.925 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9481999999999999e-05 2024-09-11T07:37:07.925 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.925 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:07.925 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.926 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.926 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.926 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25072.0:62542 8.0 e197/177 8:06b6e595:::benchmark_data_smithi022_108406_object62541:head v 197'3892, mlcod=197'3890)", 2024-09-11T07:37:07.926 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:11.936310+0000", 2024-09-11T07:37:07.926 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 175.890449265, 2024-09-11T07:37:07.926 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.071505043000000004, 2024-09-11T07:37:07.926 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:07.926 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:07.926 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:07.926 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.926 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:07.926 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.936310+0000", 2024-09-11T07:37:07.926 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.927 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.927 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.927 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:07.927 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.936310+0000", 2024-09-11T07:37:07.927 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.927 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.927 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.927 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:07.927 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.936308+0000", 2024-09-11T07:37:07.927 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:07.927 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.927 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.927 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:07.927 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.936422+0000", 2024-09-11T07:37:07.928 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000113793 2024-09-11T07:37:07.928 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.928 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.928 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:07.928 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.936432+0000", 2024-09-11T07:37:07.928 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7569999999999999e-06 2024-09-11T07:37:07.928 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.928 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.928 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:07.928 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.936437+0000", 2024-09-11T07:37:07.928 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9470000000000002e-06 2024-09-11T07:37:07.928 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.928 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.928 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:07.928 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.937638+0000", 2024-09-11T07:37:07.929 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0012013429999999999 2024-09-11T07:37:07.929 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.929 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.929 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:07.929 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.937656+0000", 2024-09-11T07:37:07.929 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7533999999999998e-05 2024-09-11T07:37:07.929 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.929 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.929 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:07.929 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.007780+0000", 2024-09-11T07:37:07.929 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.070124461999999999 2024-09-11T07:37:07.929 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.929 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.930 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:07.930 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.007815+0000", 2024-09-11T07:37:07.930 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4609e-05 2024-09-11T07:37:07.930 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.930 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:07.930 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.930 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.930 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.930 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25072.0:62545 8.0 e197/177 8:0bfa4ab9:::benchmark_data_smithi022_108406_object62544:head v 197'3893, mlcod=197'3890)", 2024-09-11T07:37:07.930 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:11.940273+0000", 2024-09-11T07:37:07.930 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 175.88648580500001, 2024-09-11T07:37:07.930 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.071980819000000001, 2024-09-11T07:37:07.930 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:07.930 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:07.931 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:07.931 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.931 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:07.931 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.940273+0000", 2024-09-11T07:37:07.931 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.931 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.931 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.931 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:07.931 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.940273+0000", 2024-09-11T07:37:07.931 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.931 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.931 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.931 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:07.932 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.940272+0000", 2024-09-11T07:37:07.932 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:07.932 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.932 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.932 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:07.932 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.940308+0000", 2024-09-11T07:37:07.932 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6282999999999998e-05 2024-09-11T07:37:07.932 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.932 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.932 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:07.932 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.940318+0000", 2024-09-11T07:37:07.932 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0199e-05 2024-09-11T07:37:07.932 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.933 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.933 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:07.933 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.940324+0000", 2024-09-11T07:37:07.933 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7490000000000004e-06 2024-09-11T07:37:07.933 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.933 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.933 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:07.933 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.940385+0000", 2024-09-11T07:37:07.933 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1266999999999995e-05 2024-09-11T07:37:07.933 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.933 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.933 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:07.933 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:11.940403+0000", 2024-09-11T07:37:07.934 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7569999999999999e-05 2024-09-11T07:37:07.934 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.934 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.934 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:07.934 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.012219+0000", 2024-09-11T07:37:07.934 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.071815961999999997 2024-09-11T07:37:07.934 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.934 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.934 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:07.934 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.012254+0000", 2024-09-11T07:37:07.934 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5290999999999998e-05 2024-09-11T07:37:07.934 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.935 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:07.935 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.935 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.935 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.935 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:63543 8.8 8:17f4e5d0:::benchmark_data_smithi022_108406_object63542:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-11T07:37:07.935 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:12.693526+0000", 2024-09-11T07:37:07.935 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 175.13323298099999, 2024-09-11T07:37:07.935 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.085182680999999996, 2024-09-11T07:37:07.935 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:07.935 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:07.935 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:37:07.935 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:37:07.935 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:37:07.935 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 63543 2024-09-11T07:37:07.936 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.936 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:07.936 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.936 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:07.936 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693526+0000", 2024-09-11T07:37:07.936 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.936 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.936 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.936 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:07.936 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693526+0000", 2024-09-11T07:37:07.936 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.936 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.936 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.936 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:07.936 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693525+0000", 2024-09-11T07:37:07.937 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:07.937 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.937 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.937 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:07.937 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693604+0000", 2024-09-11T07:37:07.937 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9684999999999993e-05 2024-09-11T07:37:07.937 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.937 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.937 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:07.937 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693617+0000", 2024-09-11T07:37:07.937 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3125000000000001e-05 2024-09-11T07:37:07.937 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.938 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.938 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:07.938 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693623+0000", 2024-09-11T07:37:07.938 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8760000000000002e-06 2024-09-11T07:37:07.938 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.938 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.938 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:07.938 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693660+0000", 2024-09-11T07:37:07.938 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6451e-05 2024-09-11T07:37:07.938 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.938 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.938 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:07.938 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693789+0000", 2024-09-11T07:37:07.938 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012951 2024-09-11T07:37:07.938 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.939 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.939 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:37:07.939 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693921+0000", 2024-09-11T07:37:07.939 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000131817 2024-09-11T07:37:07.939 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.939 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.939 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.939 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.702633+0000", 2024-09-11T07:37:07.939 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0087122129999999999 2024-09-11T07:37:07.939 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.939 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.939 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:37:07.939 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.702640+0000", 2024-09-11T07:37:07.939 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.109e-06 2024-09-11T07:37:07.939 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.940 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.940 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.940 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.778634+0000", 2024-09-11T07:37:07.940 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.075993639000000002 2024-09-11T07:37:07.940 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.940 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.940 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:07.940 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.778678+0000", 2024-09-11T07:37:07.940 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4289999999999998e-05 2024-09-11T07:37:07.940 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.940 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.940 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:07.940 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.778709+0000", 2024-09-11T07:37:07.940 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.0273999999999998e-05 2024-09-11T07:37:07.940 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.941 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:07.941 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.941 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.941 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.941 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:63545 8.8 8:1cf778d2:::benchmark_data_smithi022_108406_object63544:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-11T07:37:07.941 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:12.693945+0000", 2024-09-11T07:37:07.941 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 175.132813838, 2024-09-11T07:37:07.941 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.084917197999999999, 2024-09-11T07:37:07.941 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:07.941 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:07.941 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:37:07.941 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:37:07.941 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:37:07.942 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 63545 2024-09-11T07:37:07.942 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.942 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:07.942 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.942 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:07.942 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693945+0000", 2024-09-11T07:37:07.942 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.942 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.942 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.942 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:07.942 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693945+0000", 2024-09-11T07:37:07.942 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.942 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.942 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.942 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:07.943 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693936+0000", 2024-09-11T07:37:07.943 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999909 2024-09-11T07:37:07.943 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.943 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.943 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:07.943 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693984+0000", 2024-09-11T07:37:07.943 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7815999999999998e-05 2024-09-11T07:37:07.943 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.943 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.943 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:07.943 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.693995+0000", 2024-09-11T07:37:07.943 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.06e-05 2024-09-11T07:37:07.943 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.944 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.944 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:07.944 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.694000+0000", 2024-09-11T07:37:07.944 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5910000000000003e-06 2024-09-11T07:37:07.944 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.944 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.944 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:07.944 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.695920+0000", 2024-09-11T07:37:07.944 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001919841 2024-09-11T07:37:07.944 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.944 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.944 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:07.944 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.695991+0000", 2024-09-11T07:37:07.944 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1290000000000004e-05 2024-09-11T07:37:07.945 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.945 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.945 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:37:07.945 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.696090+0000", 2024-09-11T07:37:07.945 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8818000000000002e-05 2024-09-11T07:37:07.945 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.945 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.945 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.945 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.702671+0000", 2024-09-11T07:37:07.945 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0065810449999999998 2024-09-11T07:37:07.945 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.945 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.945 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:37:07.945 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.705045+0000", 2024-09-11T07:37:07.946 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0023738409999999998 2024-09-11T07:37:07.946 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.946 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.946 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.946 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.778758+0000", 2024-09-11T07:37:07.946 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.073712593000000007 2024-09-11T07:37:07.946 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.946 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.946 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:07.946 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.778812+0000", 2024-09-11T07:37:07.946 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.452e-05 2024-09-11T07:37:07.946 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.947 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.947 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:07.947 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.778862+0000", 2024-09-11T07:37:07.947 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0160000000000001e-05 2024-09-11T07:37:07.947 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.947 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:07.947 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.947 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.947 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.947 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:63549 8.8 8:153fa01c:::benchmark_data_smithi022_108406_object63548:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-11T07:37:07.947 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:12.696485+0000", 2024-09-11T07:37:07.947 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 175.13027356699999, 2024-09-11T07:37:07.947 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.091098567000000005, 2024-09-11T07:37:07.947 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:07.947 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:07.948 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:37:07.948 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:37:07.948 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:37:07.948 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 63549 2024-09-11T07:37:07.948 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.948 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:07.948 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.948 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:07.948 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.696485+0000", 2024-09-11T07:37:07.948 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.948 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.948 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.948 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:07.948 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.696485+0000", 2024-09-11T07:37:07.949 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.949 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.949 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.949 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:07.949 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.696484+0000", 2024-09-11T07:37:07.949 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:07.949 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.949 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.949 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:07.949 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.696513+0000", 2024-09-11T07:37:07.949 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9009e-05 2024-09-11T07:37:07.949 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.949 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.949 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:07.949 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.696528+0000", 2024-09-11T07:37:07.950 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.483e-05 2024-09-11T07:37:07.950 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.950 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.950 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:07.950 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.696533+0000", 2024-09-11T07:37:07.950 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6439999999999997e-06 2024-09-11T07:37:07.950 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.950 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.950 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:07.950 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.702695+0000", 2024-09-11T07:37:07.950 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0061611790000000001 2024-09-11T07:37:07.950 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.951 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.951 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:07.951 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.702804+0000", 2024-09-11T07:37:07.951 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010943 2024-09-11T07:37:07.951 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.951 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.951 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:37:07.951 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.702940+0000", 2024-09-11T07:37:07.951 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000135938 2024-09-11T07:37:07.951 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.951 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.951 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:37:07.951 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.705639+0000", 2024-09-11T07:37:07.952 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002699019 2024-09-11T07:37:07.952 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.952 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.952 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.952 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.706887+0000", 2024-09-11T07:37:07.952 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0012476239999999999 2024-09-11T07:37:07.952 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.952 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.952 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.952 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.787529+0000", 2024-09-11T07:37:07.952 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.080642251999999998 2024-09-11T07:37:07.952 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.952 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.952 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:07.952 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.787561+0000", 2024-09-11T07:37:07.953 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1764000000000003e-05 2024-09-11T07:37:07.953 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.953 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.953 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:07.953 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:12.787584+0000", 2024-09-11T07:37:07.953 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3312e-05 2024-09-11T07:37:07.953 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.953 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:07.953 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.953 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.953 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.953 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:66849 8.8 8:17418fad:::benchmark_data_smithi022_108406_object66848:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-11T07:37:07.953 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:15.448364+0000", 2024-09-11T07:37:07.953 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 172.378395286, 2024-09-11T07:37:07.953 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.102043175, 2024-09-11T07:37:07.954 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:07.954 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:07.954 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:37:07.954 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:37:07.954 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:37:07.954 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 66849 2024-09-11T07:37:07.954 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.954 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:07.954 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.954 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:07.954 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.448364+0000", 2024-09-11T07:37:07.954 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.954 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.954 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.954 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:07.955 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.448364+0000", 2024-09-11T07:37:07.955 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.955 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.955 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.955 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:07.955 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.448362+0000", 2024-09-11T07:37:07.955 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:37:07.955 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.955 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.955 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:07.955 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.448416+0000", 2024-09-11T07:37:07.955 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4364999999999999e-05 2024-09-11T07:37:07.956 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.956 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.956 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:07.956 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.448426+0000", 2024-09-11T07:37:07.956 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0129000000000001e-05 2024-09-11T07:37:07.956 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.956 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.956 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:07.956 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.448433+0000", 2024-09-11T07:37:07.956 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4590000000000003e-06 2024-09-11T07:37:07.956 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.956 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.956 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:07.957 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.448460+0000", 2024-09-11T07:37:07.957 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.7237999999999999e-05 2024-09-11T07:37:07.957 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.957 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.957 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:07.957 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.448549+0000", 2024-09-11T07:37:07.957 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8811000000000001e-05 2024-09-11T07:37:07.957 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.957 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.957 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:37:07.957 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.448668+0000", 2024-09-11T07:37:07.957 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000119191 2024-09-11T07:37:07.957 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.957 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.957 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.958 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.450969+0000", 2024-09-11T07:37:07.958 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002301432 2024-09-11T07:37:07.958 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.958 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.958 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:37:07.958 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.450976+0000", 2024-09-11T07:37:07.958 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1590000000000004e-06 2024-09-11T07:37:07.958 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.958 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.958 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.958 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.550242+0000", 2024-09-11T07:37:07.958 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.099265205999999995 2024-09-11T07:37:07.958 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.958 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.958 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:07.959 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.550326+0000", 2024-09-11T07:37:07.959 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4350000000000004e-05 2024-09-11T07:37:07.959 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.959 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.959 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:07.959 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.550407+0000", 2024-09-11T07:37:07.959 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0741999999999994e-05 2024-09-11T07:37:07.959 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.959 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:07.959 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.959 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.959 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.959 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:66854 8.8 8:16b7a44e:::benchmark_data_smithi022_108406_object66853:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-11T07:37:07.959 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:15.453699+0000", 2024-09-11T07:37:07.959 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 172.37305975500001, 2024-09-11T07:37:07.960 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.096855826000000006, 2024-09-11T07:37:07.960 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:07.960 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:07.960 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:37:07.960 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:37:07.960 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:37:07.960 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 66854 2024-09-11T07:37:07.960 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.960 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:07.960 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.960 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:07.960 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.453699+0000", 2024-09-11T07:37:07.960 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.960 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.960 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.961 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:07.961 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.453699+0000", 2024-09-11T07:37:07.961 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:07.961 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.961 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.961 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:07.961 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.453698+0000", 2024-09-11T07:37:07.961 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:07.961 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.961 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.961 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:07.961 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.453788+0000", 2024-09-11T07:37:07.961 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0383000000000005e-05 2024-09-11T07:37:07.961 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.962 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.962 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:07.962 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.453799+0000", 2024-09-11T07:37:07.962 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1313e-05 2024-09-11T07:37:07.962 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.962 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.962 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:07.962 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.453804+0000", 2024-09-11T07:37:07.962 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4000000000000002e-06 2024-09-11T07:37:07.962 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.962 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.962 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:07.962 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.453848+0000", 2024-09-11T07:37:07.962 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4224999999999998e-05 2024-09-11T07:37:07.962 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.963 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.963 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:07.963 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.453971+0000", 2024-09-11T07:37:07.963 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012345800000000001 2024-09-11T07:37:07.963 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.963 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.963 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:37:07.963 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.454116+0000", 2024-09-11T07:37:07.963 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014493399999999999 2024-09-11T07:37:07.963 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.963 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.963 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:37:07.963 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.455962+0000", 2024-09-11T07:37:07.963 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001846077 2024-09-11T07:37:07.964 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.964 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.964 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.964 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.456168+0000", 2024-09-11T07:37:07.964 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00020587200000000001 2024-09-11T07:37:07.964 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.964 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.964 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:07.964 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.550504+0000", 2024-09-11T07:37:07.964 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.094335666999999998 2024-09-11T07:37:07.965 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.965 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.965 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:07.965 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.550534+0000", 2024-09-11T07:37:07.965 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9952000000000001e-05 2024-09-11T07:37:07.965 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:07.965 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:07.965 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:07.965 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:15.550555+0000", 2024-09-11T07:37:07.965 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1015000000000001e-05 2024-09-11T07:37:07.965 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.965 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:07.965 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.965 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:07.966 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:07.966 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:37:07.981 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:08.003 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:08.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:08.005 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:08.105 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:08.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:08.129 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:08.129 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:08.230 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:08.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:08.251 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:08.252 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:08.352 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:08.373 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:08.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:08.374 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:08.475 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:08.495 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:08.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:08.496 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:08.597 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:08.618 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:08.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:08.619 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:08.720 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:08.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:08 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/353833050' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-11T07:37:08.735 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:08 smithi022 ceph-mon[29462]: osdmap e240: 8 total, 8 up, 8 in 2024-09-11T07:37:08.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:37:08.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:08.736 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:08 smithi022 ceph-mon[29462]: pgmap v471: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 737 B/s rd, 0 B/s wr, 394 op/s 2024-09-11T07:37:08.736 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:08 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/353833050' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-11T07:37:08.736 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:08 smithi022 ceph-mon[37330]: osdmap e240: 8 total, 8 up, 8 in 2024-09-11T07:37:08.736 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:37:08.736 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:08.737 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:08 smithi022 ceph-mon[37330]: pgmap v471: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 737 B/s rd, 0 B/s wr, 394 op/s 2024-09-11T07:37:08.742 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:08.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:08.743 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:08.748 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-11T07:37:08.845 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:08.866 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:08.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:08.867 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:08.968 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:08.989 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:08.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:08.990 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:09.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:08 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/353833050' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-11T07:37:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:08 smithi104 ceph-mon[33507]: osdmap e240: 8 total, 8 up, 8 in 2024-09-11T07:37:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:37:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:08 smithi104 ceph-mon[33507]: pgmap v471: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 737 B/s rd, 0 B/s wr, 394 op/s 2024-09-11T07:37:09.034 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:37:09.092 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:09.113 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:09.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:09.114 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:09.215 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:09.236 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:09.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:09.236 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:09.337 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:09.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:09.359 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:09.359 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:09.460 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:09.491 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:09.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:09.492 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:09.505 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight 5 1 2024-09-11T07:37:09.593 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:09.614 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:09.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:09.615 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:09.715 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:09.736 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:09.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:09.737 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:09.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:09 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:37:09.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:09 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:37:09.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:09 smithi022 ceph-mon[29462]: osdmap e241: 8 total, 8 up, 8 in 2024-09-11T07:37:09.758 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:09 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:37:09.758 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:09 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:37:09.758 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:09 smithi022 ceph-mon[37330]: osdmap e241: 8 total, 8 up, 8 in 2024-09-11T07:37:09.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:37:09 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:37:09.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:09 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:37:09.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:09 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:37:09.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:09 smithi104 ceph-mon[33507]: osdmap e241: 8 total, 8 up, 8 in 2024-09-11T07:37:09.804 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T07:37:09.839 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:09.860 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:09.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:09.861 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:09.962 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:09.989 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:09.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:09.990 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:10.072 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:37:10.091 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:10.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:10.114 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:10.114 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:10.215 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:10.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:10.239 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:10.239 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:10.340 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:10.362 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:10.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:10.363 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:10.417 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:37:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:37:10] "GET /metrics HTTP/1.1" 200 224112 "" "Prometheus/2.43.0" 2024-09-11T07:37:10.463 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:10.486 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:10.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:10.487 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:10.588 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:10.614 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:10.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:10.615 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:10.716 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:10.731 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:37:10.731 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:37:10.731 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 0 2024-09-11T07:37:10.731 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:37:10.738 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:10.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:10.739 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:10.739 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:10 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:37:10.739 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:10 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:37:10.739 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:10 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:10.739 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:10 smithi104 ceph-mon[33507]: osdmap e242: 8 total, 8 up, 8 in 2024-09-11T07:37:10.740 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:10 smithi104 ceph-mon[33507]: pgmap v474: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:10.799 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:10 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:37:10.799 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:10 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:37:10.800 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:10 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:10.800 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:10 smithi022 ceph-mon[29462]: osdmap e242: 8 total, 8 up, 8 in 2024-09-11T07:37:10.800 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:10 smithi022 ceph-mon[29462]: pgmap v474: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:10.800 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:10 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:37:10.800 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:10 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:37:10.800 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:10 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:10.800 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:10 smithi022 ceph-mon[37330]: osdmap e242: 8 total, 8 up, 8 in 2024-09-11T07:37:10.800 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:10 smithi022 ceph-mon[37330]: pgmap v474: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:10.840 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:10.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:10.863 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:10.863 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:10.964 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:10.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:10.986 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:10.987 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:11.088 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:11.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:11.110 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:11.110 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:11.211 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:11.217 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:37:11.217 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:37:11.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:11.244 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:11.244 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:11.261 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:37:11.262 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:37:11.277 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:37:11.277 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:37:11.291 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:37:11.292 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:37:11.307 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:37:11.307 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:37:11.321 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:37:11.322 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:37:11.337 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:37:11.337 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:37:11.345 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:11.353 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:37:11.353 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:37:11.368 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:11.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:11.369 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:11.369 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:37:11.370 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:37:11.379 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-11T07:37:11.470 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:11.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:11.494 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:11.494 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:11.595 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:11.617 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:11.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:11.618 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:11.660 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:37:11.719 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:11.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:11.741 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:11.741 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:11.842 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:11.865 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:11.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:11.866 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:11.967 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:11.989 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:11.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:11.990 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:12.082 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:12 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1290966650' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:37:12.091 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:12.115 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:12.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:12.116 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:12.217 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:12.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:12.239 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:12.239 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:12.311 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:12 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1290966650' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:37:12.311 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:12 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1290966650' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:37:12.339 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:12.361 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:12.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:12.363 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:12.463 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:12.485 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:12.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:12.486 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:12.587 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:12.608 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:12.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:12.609 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:12.710 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:12.732 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:12.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:12.733 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:12.834 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:12.856 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:12.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:12.860 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:12.961 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:12.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:12.984 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:12.984 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:13.085 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:13.108 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:13.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:13.109 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:13.210 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:13.233 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:13.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:13.234 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:13.240 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:37:13.240 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:37:13.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:13 smithi104 ceph-mon[33507]: pgmap v475: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 917 B/s rd, 0 op/s 2024-09-11T07:37:13.334 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:13.356 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:13.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:13.357 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:13.388 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:13 smithi022 ceph-mon[29462]: pgmap v475: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 917 B/s rd, 0 op/s 2024-09-11T07:37:13.389 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:13 smithi022 ceph-mon[37330]: pgmap v475: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 917 B/s rd, 0 op/s 2024-09-11T07:37:13.457 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:13.480 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:13.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:13.481 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:13.582 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:13.604 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:13.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:13.605 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:13.705 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:13.729 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:13.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:13.730 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:13.831 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:13.853 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:13.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:13.854 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:13.874 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-11T07:37:13.955 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:13.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:13.977 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:13.977 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:14.036 INFO:teuthology.orchestra.run.smithi022.stderr:reweighted osd.5 to 1 (10000) 2024-09-11T07:37:14.078 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:14.101 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:14.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:14.102 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:14.121 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:37:14.143 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:14 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1966454325' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T07:37:14.143 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:14 smithi022 ceph-mon[29462]: from='client.25316 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T07:37:14.143 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:14 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1966454325' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T07:37:14.144 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:14 smithi022 ceph-mon[37330]: from='client.25316 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T07:37:14.203 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:14.225 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:14.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:14.226 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:14.242 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:37:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:37:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:37:14.242 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:14 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1966454325' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T07:37:14.242 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:14 smithi104 ceph-mon[33507]: from='client.25316 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T07:37:14.327 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:14.350 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:14.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:14.351 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:14.451 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:14.474 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:14.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:14.475 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:14.575 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:14.598 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:14.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:14.599 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:14.699 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:14.721 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:14.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:14.722 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:14.763 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight 6 1 2024-09-11T07:37:14.822 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:14.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:14.844 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:14.845 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:14.946 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:14.967 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:14.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:14.968 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:15.068 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:15.091 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:15.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:15.092 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:15.193 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:15.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:15.215 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:15.216 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:15.225 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:15 smithi104 ceph-mon[33507]: from='client.25316 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-11T07:37:15.225 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:15 smithi104 ceph-mon[33507]: osdmap e243: 8 total, 8 up, 8 in 2024-09-11T07:37:15.225 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:15 smithi104 ceph-mon[33507]: pgmap v477: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:15.317 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:15.339 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:15.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:15.340 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:15.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:15 smithi022 ceph-mon[29462]: from='client.25316 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-11T07:37:15.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:15 smithi022 ceph-mon[29462]: osdmap e243: 8 total, 8 up, 8 in 2024-09-11T07:37:15.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:15 smithi022 ceph-mon[29462]: pgmap v477: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:15.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:15 smithi022 ceph-mon[37330]: from='client.25316 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-11T07:37:15.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:15 smithi022 ceph-mon[37330]: osdmap e243: 8 total, 8 up, 8 in 2024-09-11T07:37:15.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:15 smithi022 ceph-mon[37330]: pgmap v477: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:15.441 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:15.463 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:15.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:15.464 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:15.565 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:15.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:15.587 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:15.587 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:15.688 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:15.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:15.711 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:15.711 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:15.806 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:37:15.807 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:37:15.812 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:15.834 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:15.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:15.835 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:15.936 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:15.958 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:15.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:15.959 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:16.060 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:16.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:16.084 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:16.084 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:16.169 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:37:16.185 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:16.234 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:16.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:16.235 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:16.336 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:16.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:16.359 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:16.359 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:16.423 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-11T07:37:16.460 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:16.483 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:16.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:16.484 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:16.552 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:37:16.585 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:16.600 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is unset 2024-09-11T07:37:16.607 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:16.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:16.608 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:16.673 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:37:16.709 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:16.734 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:16.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:16.735 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:16.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:16 smithi104 ceph-mon[33507]: pgmap v478: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T07:37:16.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:16 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2083960900' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:37:16.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:16 smithi104 ceph-mon[33507]: from='client.25322 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:37:16.813 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:16 smithi022 ceph-mon[29462]: pgmap v478: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T07:37:16.813 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:16 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2083960900' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:37:16.813 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:16 smithi022 ceph-mon[29462]: from='client.25322 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:37:16.813 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:16 smithi022 ceph-mon[37330]: pgmap v478: 148 pgs: 148 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T07:37:16.814 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:16 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2083960900' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:37:16.814 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:16 smithi022 ceph-mon[37330]: from='client.25322 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:37:16.836 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:16.858 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:16.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:16.859 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:16.960 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:16.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:16.982 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:16.982 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:17.083 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:17.106 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:17.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:17.107 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:17.207 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:17.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:17.230 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:17.230 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:17.331 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:17.354 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:17.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:17.355 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:17.456 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:17.477 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:17.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:17.478 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:17.579 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:17.603 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'unique_pool_1' removed 2024-09-11T07:37:17.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:17.608 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:17.608 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:17.709 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:17.731 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:17.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:17.732 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:17.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:17 smithi104 ceph-mon[33507]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:37:17.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:17 smithi104 ceph-mon[33507]: from='client.25322 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:37:17.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:17 smithi104 ceph-mon[33507]: osdmap e244: 8 total, 8 up, 8 in 2024-09-11T07:37:17.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:17 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3742497172' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:37:17.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:17 smithi104 ceph-mon[33507]: from='client.25328 ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:37:17.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:17 smithi104 ceph-mon[33507]: 4.f scrub starts 2024-09-11T07:37:17.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:17 smithi104 ceph-mon[33507]: 4.f scrub ok 2024-09-11T07:37:17.833 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:17.856 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:17.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:17.857 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:17 smithi022 ceph-mon[29462]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:37:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:17 smithi022 ceph-mon[29462]: from='client.25322 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:37:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:17 smithi022 ceph-mon[29462]: osdmap e244: 8 total, 8 up, 8 in 2024-09-11T07:37:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:17 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3742497172' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:37:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:17 smithi022 ceph-mon[29462]: from='client.25328 ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:37:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:17 smithi022 ceph-mon[29462]: 4.f scrub starts 2024-09-11T07:37:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:17 smithi022 ceph-mon[29462]: 4.f scrub ok 2024-09-11T07:37:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:17 smithi022 ceph-mon[37330]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:37:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:17 smithi022 ceph-mon[37330]: from='client.25322 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:37:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:17 smithi022 ceph-mon[37330]: osdmap e244: 8 total, 8 up, 8 in 2024-09-11T07:37:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:17 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3742497172' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:37:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:17 smithi022 ceph-mon[37330]: from='client.25328 ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:37:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:17 smithi022 ceph-mon[37330]: 4.f scrub starts 2024-09-11T07:37:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:17 smithi022 ceph-mon[37330]: 4.f scrub ok 2024-09-11T07:37:17.958 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:17.981 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:17.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:17.982 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:18.082 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:18.109 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:18.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:18.110 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:18.211 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:18.235 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:18.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:18.236 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:18.337 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:18.363 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:18.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:18.364 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:18.465 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:18.488 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:18.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:18.489 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:18.590 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:18.613 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:18.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:18.614 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:18.621 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:37:18.621 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:37:18.715 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:18.737 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:18.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:18.738 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:18 smithi104 ceph-mon[33507]: 6.b scrub starts 2024-09-11T07:37:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:18 smithi104 ceph-mon[33507]: 6.b scrub ok 2024-09-11T07:37:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:18 smithi104 ceph-mon[33507]: 5.15 scrub starts 2024-09-11T07:37:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:18 smithi104 ceph-mon[33507]: 5.15 scrub ok 2024-09-11T07:37:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:18 smithi104 ceph-mon[33507]: 5.b scrub starts 2024-09-11T07:37:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:18 smithi104 ceph-mon[33507]: 5.b scrub ok 2024-09-11T07:37:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:18 smithi104 ceph-mon[33507]: from='client.25328 ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T07:37:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:18 smithi104 ceph-mon[33507]: osdmap e245: 8 total, 8 up, 8 in 2024-09-11T07:37:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:18 smithi104 ceph-mon[33507]: 5.c scrub starts 2024-09-11T07:37:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:18 smithi104 ceph-mon[33507]: pgmap v481: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-11T07:37:18.839 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:18.854 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:18 smithi022 ceph-mon[29462]: 6.b scrub starts 2024-09-11T07:37:18.854 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:18 smithi022 ceph-mon[29462]: 6.b scrub ok 2024-09-11T07:37:18.854 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:18 smithi022 ceph-mon[29462]: 5.15 scrub starts 2024-09-11T07:37:18.854 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:18 smithi022 ceph-mon[29462]: 5.15 scrub ok 2024-09-11T07:37:18.854 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:18 smithi022 ceph-mon[29462]: 5.b scrub starts 2024-09-11T07:37:18.854 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:18 smithi022 ceph-mon[29462]: 5.b scrub ok 2024-09-11T07:37:18.854 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:18 smithi022 ceph-mon[29462]: from='client.25328 ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T07:37:18.854 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:18 smithi022 ceph-mon[29462]: osdmap e245: 8 total, 8 up, 8 in 2024-09-11T07:37:18.854 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:18 smithi022 ceph-mon[29462]: 5.c scrub starts 2024-09-11T07:37:18.854 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:18 smithi022 ceph-mon[29462]: pgmap v481: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-11T07:37:18.854 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:18 smithi022 ceph-mon[37330]: 6.b scrub starts 2024-09-11T07:37:18.855 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:18 smithi022 ceph-mon[37330]: 6.b scrub ok 2024-09-11T07:37:18.855 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:18 smithi022 ceph-mon[37330]: 5.15 scrub starts 2024-09-11T07:37:18.855 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:18 smithi022 ceph-mon[37330]: 5.15 scrub ok 2024-09-11T07:37:18.855 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:18 smithi022 ceph-mon[37330]: 5.b scrub starts 2024-09-11T07:37:18.855 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:18 smithi022 ceph-mon[37330]: 5.b scrub ok 2024-09-11T07:37:18.855 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:18 smithi022 ceph-mon[37330]: from='client.25328 ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T07:37:18.855 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:18 smithi022 ceph-mon[37330]: osdmap e245: 8 total, 8 up, 8 in 2024-09-11T07:37:18.855 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:18 smithi022 ceph-mon[37330]: 5.c scrub starts 2024-09-11T07:37:18.855 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:18 smithi022 ceph-mon[37330]: pgmap v481: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-11T07:37:18.862 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:18.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:18.863 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:18.877 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-11T07:37:18.878 INFO:tasks.radosbench:Beginning radosbench... 2024-09-11T07:37:18.879 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_2 2024-09-11T07:37:18.879 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pool create unique_pool_2 16 2024-09-11T07:37:18.921 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:37:18.963 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:18.986 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:18.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:18.987 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:19.087 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:19.110 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:19.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:19.111 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:19.212 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-11T07:37:19.212 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:19.235 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:19.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:19.236 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:19.336 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:19.359 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:19.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:19.360 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:19.455 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:37:19.460 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:19.482 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:19.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:19.483 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:19.584 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:19.606 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:19.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:19.607 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:19.708 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:19.731 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:19.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:19.732 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:19.812 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:19 smithi022 ceph-mon[29462]: 5.16 scrub starts 2024-09-11T07:37:19.812 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:19 smithi022 ceph-mon[29462]: 5.16 scrub ok 2024-09-11T07:37:19.812 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:19 smithi022 ceph-mon[29462]: 4.1c scrub starts 2024-09-11T07:37:19.812 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:19 smithi022 ceph-mon[29462]: 4.1c scrub ok 2024-09-11T07:37:19.812 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:19 smithi022 ceph-mon[29462]: 4.16 scrub starts 2024-09-11T07:37:19.812 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:19 smithi022 ceph-mon[29462]: 4.16 scrub ok 2024-09-11T07:37:19.812 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:19 smithi022 ceph-mon[37330]: 5.16 scrub starts 2024-09-11T07:37:19.813 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:19 smithi022 ceph-mon[37330]: 5.16 scrub ok 2024-09-11T07:37:19.813 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:19 smithi022 ceph-mon[37330]: 4.1c scrub starts 2024-09-11T07:37:19.813 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:19 smithi022 ceph-mon[37330]: 4.1c scrub ok 2024-09-11T07:37:19.813 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:19 smithi022 ceph-mon[37330]: 4.16 scrub starts 2024-09-11T07:37:19.813 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:19 smithi022 ceph-mon[37330]: 4.16 scrub ok 2024-09-11T07:37:19.833 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:19.855 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:19.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:19.856 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:19.884 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:37:19 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:37:19.884 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:19 smithi104 ceph-mon[33507]: 5.16 scrub starts 2024-09-11T07:37:19.884 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:19 smithi104 ceph-mon[33507]: 5.16 scrub ok 2024-09-11T07:37:19.884 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:19 smithi104 ceph-mon[33507]: 4.1c scrub starts 2024-09-11T07:37:19.884 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:19 smithi104 ceph-mon[33507]: 4.1c scrub ok 2024-09-11T07:37:19.884 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:19 smithi104 ceph-mon[33507]: 4.16 scrub starts 2024-09-11T07:37:19.884 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:19 smithi104 ceph-mon[33507]: 4.16 scrub ok 2024-09-11T07:37:19.957 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:19.978 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:19.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:19.979 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:20.080 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:20.101 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:20.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:20.102 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:20.203 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:20.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:20.225 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:20.225 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:20.295 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T07:37:20.326 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:20.347 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:20.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:20.348 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:20.449 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:20.470 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:20.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:20.471 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:20.572 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:20.597 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:20.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:20.598 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:20.612 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:37:20 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:37:20] "GET /metrics HTTP/1.1" 200 224112 "" "Prometheus/2.43.0" 2024-09-11T07:37:20.612 INFO:teuthology.orchestra.run.smithi022.stderr:reweighted osd.6 to 1 (10000) 2024-09-11T07:37:20.699 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:20.720 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:20.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:20.721 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:20.824 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:20.848 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:20.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:20.849 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:20.884 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:20 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:20.884 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:20 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1949106965' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T07:37:20.884 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:20 smithi022 ceph-mon[29462]: from='client.25334 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T07:37:20.885 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:20 smithi022 ceph-mon[29462]: pgmap v482: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-11T07:37:20.885 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:20 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:20.885 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:20 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1949106965' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T07:37:20.885 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:20 smithi022 ceph-mon[37330]: from='client.25334 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T07:37:20.885 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:20 smithi022 ceph-mon[37330]: pgmap v482: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-11T07:37:20.946 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:20 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:20.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:20 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1949106965' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T07:37:20.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:20 smithi104 ceph-mon[33507]: from='client.25334 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T07:37:20.947 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:20 smithi104 ceph-mon[33507]: pgmap v482: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-11T07:37:20.950 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:20.972 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:20.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:20.973 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:21.073 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:21.096 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:21.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:21.097 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:21.198 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:21.220 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:21.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:21.222 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:21.322 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:21.324 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:37:21.324 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:37:21.334 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:37:21.345 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:21.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:21.346 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:21.348 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight 7 1 2024-09-11T07:37:21.447 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:21.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:21.470 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:21.470 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:21.571 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:21.592 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:21.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:21.593 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:21.694 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:21.719 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:21.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:21.721 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:21.822 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:21.843 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:21.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:21.844 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:21.880 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:21 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:37:21.880 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:21 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:37:21.881 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:21 smithi104 ceph-mon[33507]: 6.d scrub starts 2024-09-11T07:37:21.881 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:21 smithi104 ceph-mon[33507]: 6.d scrub ok 2024-09-11T07:37:21.881 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:21 smithi104 ceph-mon[33507]: 2.7 scrub starts 2024-09-11T07:37:21.881 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:21 smithi104 ceph-mon[33507]: 2.7 scrub ok 2024-09-11T07:37:21.881 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:21 smithi104 ceph-mon[33507]: from='client.25334 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-11T07:37:21.881 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:21 smithi104 ceph-mon[33507]: osdmap e246: 8 total, 8 up, 8 in 2024-09-11T07:37:21.881 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:21 smithi104 ceph-mon[33507]: 4.7 scrub starts 2024-09-11T07:37:21.881 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:21 smithi104 ceph-mon[33507]: 4.7 scrub ok 2024-09-11T07:37:21.881 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:21 smithi104 ceph-mon[33507]: 5.1e scrub starts 2024-09-11T07:37:21.881 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:21 smithi104 ceph-mon[33507]: 5.1e scrub ok 2024-09-11T07:37:21.904 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-11T07:37:21.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:21 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:37:21.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:21 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:37:21.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:21 smithi022 ceph-mon[29462]: 6.d scrub starts 2024-09-11T07:37:21.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:21 smithi022 ceph-mon[29462]: 6.d scrub ok 2024-09-11T07:37:21.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:21 smithi022 ceph-mon[29462]: 2.7 scrub starts 2024-09-11T07:37:21.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:21 smithi022 ceph-mon[29462]: 2.7 scrub ok 2024-09-11T07:37:21.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:21 smithi022 ceph-mon[29462]: from='client.25334 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-11T07:37:21.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:21 smithi022 ceph-mon[29462]: osdmap e246: 8 total, 8 up, 8 in 2024-09-11T07:37:21.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:21 smithi022 ceph-mon[29462]: 4.7 scrub starts 2024-09-11T07:37:21.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:21 smithi022 ceph-mon[29462]: 4.7 scrub ok 2024-09-11T07:37:21.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:21 smithi022 ceph-mon[29462]: 5.1e scrub starts 2024-09-11T07:37:21.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:21 smithi022 ceph-mon[29462]: 5.1e scrub ok 2024-09-11T07:37:21.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:21 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:37:21.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:21 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:37:21.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:21 smithi022 ceph-mon[37330]: 6.d scrub starts 2024-09-11T07:37:21.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:21 smithi022 ceph-mon[37330]: 6.d scrub ok 2024-09-11T07:37:21.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:21 smithi022 ceph-mon[37330]: 2.7 scrub starts 2024-09-11T07:37:21.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:21 smithi022 ceph-mon[37330]: 2.7 scrub ok 2024-09-11T07:37:21.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:21 smithi022 ceph-mon[37330]: from='client.25334 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-11T07:37:21.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:21 smithi022 ceph-mon[37330]: osdmap e246: 8 total, 8 up, 8 in 2024-09-11T07:37:21.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:21 smithi022 ceph-mon[37330]: 4.7 scrub starts 2024-09-11T07:37:21.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:21 smithi022 ceph-mon[37330]: 4.7 scrub ok 2024-09-11T07:37:21.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:21 smithi022 ceph-mon[37330]: 5.1e scrub starts 2024-09-11T07:37:21.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:21 smithi022 ceph-mon[37330]: 5.1e scrub ok 2024-09-11T07:37:21.945 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:21.966 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:21.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:21.967 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:22.068 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:22.092 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:22.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:22.094 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:22.186 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:37:22.195 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:22.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:22.218 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:22.219 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:22.320 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:22.342 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:22.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:22.343 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:22.444 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:22.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:22.469 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:22.469 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:22.500 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:37:22.571 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:22.593 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:22.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:22.594 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:22.695 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:22.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:22.719 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:22.720 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:22.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:22 smithi022 ceph-mon[29462]: 6.f scrub starts 2024-09-11T07:37:22.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:22 smithi022 ceph-mon[29462]: 6.f scrub ok 2024-09-11T07:37:22.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:22 smithi022 ceph-mon[29462]: pgmap v484: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-11T07:37:22.751 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:22 smithi022 ceph-mon[37330]: 6.f scrub starts 2024-09-11T07:37:22.751 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:22 smithi022 ceph-mon[37330]: 6.f scrub ok 2024-09-11T07:37:22.752 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:22 smithi022 ceph-mon[37330]: pgmap v484: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-11T07:37:22.820 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:22.843 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:22.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:22.844 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:22.938 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:22 smithi104 ceph-mon[33507]: 6.f scrub starts 2024-09-11T07:37:22.938 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:22 smithi104 ceph-mon[33507]: 6.f scrub ok 2024-09-11T07:37:22.938 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:22 smithi104 ceph-mon[33507]: pgmap v484: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-11T07:37:22.944 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:22.968 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:22.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:22.969 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:23.069 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:23.092 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:23.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:23.093 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:23.194 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:23.215 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:23.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:23.216 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:23.317 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:23.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:23.340 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:23.340 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:23.441 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:23.462 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:23.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:23.463 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:23.504 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:37:23.505 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:37:23.505 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:37:23.505 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:37:23.505 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:37:23.505 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:37:23.505 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:37:23.505 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:37:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:37:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:37:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:37:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:37:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:37:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:37:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:37:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:37:23.564 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:23.587 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:23.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:23.588 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:23.688 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:23.710 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:23.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:23.711 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:23.771 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:37:23.812 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:23.846 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:23.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:23.848 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:23.849 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:23 smithi104 ceph-mon[33507]: 6.e scrub starts 2024-09-11T07:37:23.849 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:23 smithi104 ceph-mon[33507]: 6.e scrub ok 2024-09-11T07:37:23.849 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:23 smithi104 ceph-mon[33507]: 3.0 scrub starts 2024-09-11T07:37:23.849 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:23 smithi104 ceph-mon[33507]: 3.0 scrub ok 2024-09-11T07:37:23.850 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:23.850 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:23 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1961946107' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:37:23.883 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:23 smithi022 ceph-mon[29462]: 6.e scrub starts 2024-09-11T07:37:23.883 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:23 smithi022 ceph-mon[29462]: 6.e scrub ok 2024-09-11T07:37:23.883 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:23 smithi022 ceph-mon[29462]: 3.0 scrub starts 2024-09-11T07:37:23.883 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:23 smithi022 ceph-mon[29462]: 3.0 scrub ok 2024-09-11T07:37:23.883 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:23.883 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:23 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1961946107' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:37:23.884 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:23 smithi022 ceph-mon[37330]: 6.e scrub starts 2024-09-11T07:37:23.884 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:23 smithi022 ceph-mon[37330]: 6.e scrub ok 2024-09-11T07:37:23.884 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:23 smithi022 ceph-mon[37330]: 3.0 scrub starts 2024-09-11T07:37:23.884 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:23 smithi022 ceph-mon[37330]: 3.0 scrub ok 2024-09-11T07:37:23.884 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:23.884 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:23 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1961946107' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:37:23.949 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:23.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:23.972 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:23.972 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:24.034 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:37:24.034 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2024-09-11T07:37:24.034 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2024-09-11T07:37:24.034 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2024-09-11T07:37:24.034 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.034 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.24778.0:213545 7.6 e149/148 7:6fa89040:::benchmark_data_smithi022_77720_object89790:head v 149'13293, mlcod=133'9652)", 2024-09-11T07:37:24.034 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:32:02.095868+0000", 2024-09-11T07:37:24.034 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 321.847009711, 2024-09-11T07:37:24.034 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.051695970000000001, 2024-09-11T07:37:24.034 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:24.035 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:24.035 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:24.035 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.035 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:24.035 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.095868+0000", 2024-09-11T07:37:24.035 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.035 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.035 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.035 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:24.035 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.095868+0000", 2024-09-11T07:37:24.035 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.035 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.036 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.036 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:24.036 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.095867+0000", 2024-09-11T07:37:24.036 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:24.036 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.036 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.036 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:24.036 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.095876+0000", 2024-09-11T07:37:24.036 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2709999999999998e-06 2024-09-11T07:37:24.036 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.036 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.036 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:24.036 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.095889+0000", 2024-09-11T07:37:24.037 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2816000000000001e-05 2024-09-11T07:37:24.037 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.037 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.037 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:24.037 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.095895+0000", 2024-09-11T07:37:24.037 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3829999999999997e-06 2024-09-11T07:37:24.037 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.037 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.037 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:24.037 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.108719+0000", 2024-09-11T07:37:24.037 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.012824242 2024-09-11T07:37:24.037 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.038 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.038 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:24.038 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.108828+0000", 2024-09-11T07:37:24.038 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010869 2024-09-11T07:37:24.038 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.038 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.038 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:24.038 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.147436+0000", 2024-09-11T07:37:24.038 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.038607439 2024-09-11T07:37:24.038 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.038 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.038 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:24.038 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.147564+0000", 2024-09-11T07:37:24.039 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012844899999999999 2024-09-11T07:37:24.039 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.039 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:24.039 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.039 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.039 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.039 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.24778.0:213775 7.8 e149/148 7:1968d51c:::benchmark_data_smithi022_77720_object90020:head v 149'13168, mlcod=135'9590)", 2024-09-11T07:37:24.039 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:32:02.361931+0000", 2024-09-11T07:37:24.039 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 321.58094712600001, 2024-09-11T07:37:24.039 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.044073216999999998, 2024-09-11T07:37:24.039 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:24.039 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:24.039 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:24.040 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.040 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:24.040 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.361931+0000", 2024-09-11T07:37:24.040 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.040 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.040 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.040 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:24.040 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.361931+0000", 2024-09-11T07:37:24.040 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.040 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.040 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.040 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:24.041 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.361929+0000", 2024-09-11T07:37:24.041 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:24.041 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.041 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.041 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:24.041 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.361937+0000", 2024-09-11T07:37:24.041 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5730000000000002e-06 2024-09-11T07:37:24.041 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.041 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.041 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:24.041 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.361948+0000", 2024-09-11T07:37:24.041 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1589e-05 2024-09-11T07:37:24.041 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.042 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.042 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:24.042 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.361956+0000", 2024-09-11T07:37:24.042 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5639999999999999e-06 2024-09-11T07:37:24.042 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.042 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.042 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:24.042 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.372866+0000", 2024-09-11T07:37:24.042 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.010909746999999999 2024-09-11T07:37:24.042 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.042 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.042 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:24.042 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.372915+0000", 2024-09-11T07:37:24.043 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8848999999999999e-05 2024-09-11T07:37:24.043 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.043 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.043 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:24.043 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.405949+0000", 2024-09-11T07:37:24.043 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.033034926999999999 2024-09-11T07:37:24.043 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.043 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.043 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:24.043 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.406004+0000", 2024-09-11T07:37:24.043 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4373999999999998e-05 2024-09-11T07:37:24.043 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.043 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:24.043 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.044 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.044 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.044 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.24778.0:214149 7.5 e149/148 7:a7d92c18:::benchmark_data_smithi022_77720_object90394:head v 149'13617, mlcod=135'9932)", 2024-09-11T07:37:24.044 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:32:02.780188+0000", 2024-09-11T07:37:24.044 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 321.16268964699998, 2024-09-11T07:37:24.044 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.043553145000000001, 2024-09-11T07:37:24.044 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:24.044 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:24.044 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:24.044 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.044 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:24.044 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.780188+0000", 2024-09-11T07:37:24.044 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.045 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.045 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.045 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:24.045 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.780188+0000", 2024-09-11T07:37:24.045 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.045 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.045 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.045 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:24.045 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.780187+0000", 2024-09-11T07:37:24.045 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:24.045 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.045 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.045 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:24.046 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.780196+0000", 2024-09-11T07:37:24.046 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4860000000000003e-06 2024-09-11T07:37:24.046 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.046 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.046 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:24.046 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.781639+0000", 2024-09-11T07:37:24.046 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0014425 2024-09-11T07:37:24.046 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.046 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.046 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:24.046 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.781647+0000", 2024-09-11T07:37:24.046 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5720000000000003e-06 2024-09-11T07:37:24.046 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.046 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.047 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:24.047 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.781730+0000", 2024-09-11T07:37:24.047 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2927000000000001e-05 2024-09-11T07:37:24.047 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.047 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.047 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:24.047 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.781753+0000", 2024-09-11T07:37:24.047 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.2849000000000001e-05 2024-09-11T07:37:24.047 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.047 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.047 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:24.047 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.823716+0000", 2024-09-11T07:37:24.047 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.041962770000000003 2024-09-11T07:37:24.048 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.048 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.048 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:24.048 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.823741+0000", 2024-09-11T07:37:24.048 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.5573000000000001e-05 2024-09-11T07:37:24.048 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.048 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:24.048 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.048 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.048 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.048 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.24778.0:214150 7.5 e149/148 7:a01e7734:::benchmark_data_smithi022_77720_object90395:head v 149'13618, mlcod=135'9932)", 2024-09-11T07:37:24.048 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:32:02.788409+0000", 2024-09-11T07:37:24.048 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 321.15446917899999, 2024-09-11T07:37:24.049 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.040148282, 2024-09-11T07:37:24.049 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:24.049 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:24.049 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:24.049 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.049 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:24.049 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.788409+0000", 2024-09-11T07:37:24.049 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.049 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.049 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.049 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:24.049 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.788409+0000", 2024-09-11T07:37:24.049 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.050 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.050 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.050 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:24.050 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.788407+0000", 2024-09-11T07:37:24.050 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:24.050 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.050 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.050 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:24.050 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.788415+0000", 2024-09-11T07:37:24.050 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7049999999999999e-06 2024-09-11T07:37:24.050 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.050 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.050 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:24.051 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.788427+0000", 2024-09-11T07:37:24.051 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1800000000000001e-05 2024-09-11T07:37:24.051 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.051 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.051 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:24.051 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.788432+0000", 2024-09-11T07:37:24.051 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2249999999999999e-06 2024-09-11T07:37:24.051 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.051 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.051 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:24.051 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.788542+0000", 2024-09-11T07:37:24.051 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000109848 2024-09-11T07:37:24.051 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.051 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.052 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:24.052 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.788563+0000", 2024-09-11T07:37:24.052 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1243e-05 2024-09-11T07:37:24.052 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.052 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.052 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:24.052 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.828465+0000", 2024-09-11T07:37:24.052 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.039902285000000003 2024-09-11T07:37:24.052 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.052 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.052 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:24.052 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:32:02.828557+0000", 2024-09-11T07:37:24.052 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.1380999999999998e-05 2024-09-11T07:37:24.052 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.053 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:24.053 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.053 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.053 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.053 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(osd.0.0:85734 8.6 e208/177 8:60093b3e:::benchmark_data_smithi022_108406_object27139:head v 208'7039, mlcod=208'7023)", 2024-09-11T07:37:24.053 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:55.079028+0000", 2024-09-11T07:37:24.053 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 148.86384926100001, 2024-09-11T07:37:24.053 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.041301155999999999, 2024-09-11T07:37:24.053 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:24.053 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:24.053 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:24.053 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.053 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:24.054 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.079028+0000", 2024-09-11T07:37:24.054 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.054 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.054 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.054 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:24.054 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.079028+0000", 2024-09-11T07:37:24.054 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.054 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.054 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.054 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:24.054 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.079027+0000", 2024-09-11T07:37:24.054 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:24.054 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.055 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.055 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:24.055 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.079034+0000", 2024-09-11T07:37:24.055 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3030000000000002e-06 2024-09-11T07:37:24.055 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.055 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.055 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:24.055 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.079044+0000", 2024-09-11T07:37:24.055 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2159999999999995e-06 2024-09-11T07:37:24.055 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.055 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.055 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:24.055 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.079048+0000", 2024-09-11T07:37:24.056 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7979999999999996e-06 2024-09-11T07:37:24.056 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.056 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.056 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:24.056 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.083662+0000", 2024-09-11T07:37:24.056 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0046132789999999996 2024-09-11T07:37:24.056 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.056 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.056 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:24.056 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.083680+0000", 2024-09-11T07:37:24.056 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8658e-05 2024-09-11T07:37:24.056 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.056 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.057 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:24.057 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.120261+0000", 2024-09-11T07:37:24.057 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.036580895000000002 2024-09-11T07:37:24.057 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.057 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.057 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:24.057 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.120330+0000", 2024-09-11T07:37:24.057 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.8473000000000007e-05 2024-09-11T07:37:24.057 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.057 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:24.057 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.057 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.057 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.058 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(osd.0.0:85735 8.6 e208/177 8:600a9589:::benchmark_data_smithi022_108406_object22364:head v 208'7040, mlcod=208'7023)", 2024-09-11T07:37:24.058 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:55.081697+0000", 2024-09-11T07:37:24.058 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 148.86118096600001, 2024-09-11T07:37:24.058 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.041451518, 2024-09-11T07:37:24.058 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:24.058 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:24.058 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:24.058 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.058 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:24.058 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.081697+0000", 2024-09-11T07:37:24.058 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.058 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.058 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.058 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:24.059 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.081697+0000", 2024-09-11T07:37:24.059 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.059 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.059 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.059 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:24.059 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.081696+0000", 2024-09-11T07:37:24.059 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:37:24.059 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.059 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.059 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:24.059 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.081702+0000", 2024-09-11T07:37:24.059 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4230000000000002e-06 2024-09-11T07:37:24.059 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.059 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.060 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:24.060 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.081712+0000", 2024-09-11T07:37:24.060 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.9369999999999999e-06 2024-09-11T07:37:24.060 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.060 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.060 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:24.060 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.081718+0000", 2024-09-11T07:37:24.060 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8139999999999997e-06 2024-09-11T07:37:24.060 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.060 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.060 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:24.060 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.086430+0000", 2024-09-11T07:37:24.060 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0047120629999999998 2024-09-11T07:37:24.061 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.061 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.061 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:24.061 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.086448+0000", 2024-09-11T07:37:24.061 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7527999999999999e-05 2024-09-11T07:37:24.061 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.061 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.061 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:24.061 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.123113+0000", 2024-09-11T07:37:24.061 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.036665455 2024-09-11T07:37:24.061 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.061 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.061 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:24.061 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.123148+0000", 2024-09-11T07:37:24.062 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5290000000000003e-05 2024-09-11T07:37:24.062 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.062 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:24.062 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.062 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.062 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.062 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25072.0:113487 8.6 e209/177 8:668139ac:::benchmark_data_smithi022_108406_object113486:head v 209'7075, mlcod=209'7072)", 2024-09-11T07:37:24.062 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:55.258253+0000", 2024-09-11T07:37:24.062 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 148.68462486499999, 2024-09-11T07:37:24.062 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.040077862999999998, 2024-09-11T07:37:24.062 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:24.062 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:24.062 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:24.062 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.063 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:24.063 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.258253+0000", 2024-09-11T07:37:24.063 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.063 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.063 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.063 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:24.063 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.258253+0000", 2024-09-11T07:37:24.063 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.063 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.063 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.063 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:24.063 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.258252+0000", 2024-09-11T07:37:24.063 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:24.064 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.064 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.064 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:24.064 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.260870+0000", 2024-09-11T07:37:24.064 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002618127 2024-09-11T07:37:24.064 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.064 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.064 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:24.064 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.260887+0000", 2024-09-11T07:37:24.064 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6969999999999998e-05 2024-09-11T07:37:24.064 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.064 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.065 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:24.065 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.260893+0000", 2024-09-11T07:37:24.065 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1369999999999999e-06 2024-09-11T07:37:24.065 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.065 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.065 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:24.065 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.261049+0000", 2024-09-11T07:37:24.065 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015628800000000001 2024-09-11T07:37:24.065 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.065 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.065 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:24.065 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.261100+0000", 2024-09-11T07:37:24.065 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.045e-05 2024-09-11T07:37:24.065 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.066 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.066 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:24.066 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.298218+0000", 2024-09-11T07:37:24.066 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.037118901000000003 2024-09-11T07:37:24.066 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.066 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.066 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:24.066 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.298331+0000", 2024-09-11T07:37:24.066 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000112339 2024-09-11T07:37:24.066 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.066 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:24.066 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.066 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.067 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.067 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25072.0:113644 8.6 e209/177 8:6c7db1dd:::benchmark_data_smithi022_108406_object113643:head v 209'7103, mlcod=209'7101)", 2024-09-11T07:37:24.067 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:55.411452+0000", 2024-09-11T07:37:24.067 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 148.53142586499999, 2024-09-11T07:37:24.067 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.044860757000000001, 2024-09-11T07:37:24.067 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:24.067 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:24.067 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:24.067 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.067 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:24.067 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.411452+0000", 2024-09-11T07:37:24.067 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.067 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.068 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.068 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:24.068 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.411452+0000", 2024-09-11T07:37:24.068 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.068 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.068 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.068 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:24.068 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.411450+0000", 2024-09-11T07:37:24.068 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:24.068 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.068 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.068 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:24.068 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.411510+0000", 2024-09-11T07:37:24.069 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9617999999999997e-05 2024-09-11T07:37:24.069 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.069 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.069 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:24.069 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.411523+0000", 2024-09-11T07:37:24.069 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3395e-05 2024-09-11T07:37:24.069 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.069 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.069 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:24.069 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.411530+0000", 2024-09-11T07:37:24.069 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2380000000000002e-06 2024-09-11T07:37:24.069 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.069 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.069 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:24.070 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.435528+0000", 2024-09-11T07:37:24.070 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.023998150999999999 2024-09-11T07:37:24.070 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.070 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.070 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:24.070 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.435559+0000", 2024-09-11T07:37:24.070 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1495999999999998e-05 2024-09-11T07:37:24.070 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.070 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.070 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:24.070 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.456271+0000", 2024-09-11T07:37:24.070 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.020711956 2024-09-11T07:37:24.070 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.070 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.071 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:24.071 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.456313+0000", 2024-09-11T07:37:24.071 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1418000000000002e-05 2024-09-11T07:37:24.071 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.071 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:24.071 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.071 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.071 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.071 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25072.0:113655 8.6 e209/177 8:68767f94:::benchmark_data_smithi022_108406_object113654:head v 209'7104, mlcod=209'7101)", 2024-09-11T07:37:24.071 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:55.414988+0000", 2024-09-11T07:37:24.071 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 148.527889284, 2024-09-11T07:37:24.071 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.041345882, 2024-09-11T07:37:24.071 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:24.072 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:24.072 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:24.072 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.072 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:24.072 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.414988+0000", 2024-09-11T07:37:24.072 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.072 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.072 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.072 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:24.072 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.414988+0000", 2024-09-11T07:37:24.072 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.072 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.072 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.072 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:24.073 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.414987+0000", 2024-09-11T07:37:24.073 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:24.073 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.073 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.073 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:24.073 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.415023+0000", 2024-09-11T07:37:24.073 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6291999999999997e-05 2024-09-11T07:37:24.073 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.073 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.073 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:24.073 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.415035+0000", 2024-09-11T07:37:24.073 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2063e-05 2024-09-11T07:37:24.073 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.074 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.074 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:24.074 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.415041+0000", 2024-09-11T07:37:24.074 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5110000000000003e-06 2024-09-11T07:37:24.074 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.074 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.074 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:24.074 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.438336+0000", 2024-09-11T07:37:24.074 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.023295118 2024-09-11T07:37:24.074 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.074 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.074 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:24.074 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.438356+0000", 2024-09-11T07:37:24.075 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0271e-05 2024-09-11T07:37:24.075 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.075 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.075 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:24.075 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.456316+0000", 2024-09-11T07:37:24.075 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.017959573 2024-09-11T07:37:24.075 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.075 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.075 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:24.075 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.456334+0000", 2024-09-11T07:37:24.075 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8491e-05 2024-09-11T07:37:24.075 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.075 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:24.076 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.076 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.076 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.076 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25072.0:113833 8.6 e209/177 8:6bda8f3b:::benchmark_data_smithi022_108406_object113832:head v 209'7149, mlcod=209'7142)", 2024-09-11T07:37:24.076 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:55.630322+0000", 2024-09-11T07:37:24.076 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 148.31255558199999, 2024-09-11T07:37:24.076 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.040139299000000003, 2024-09-11T07:37:24.076 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:24.076 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:24.076 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:24.076 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.076 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:24.076 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.630322+0000", 2024-09-11T07:37:24.076 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.077 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.077 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.077 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:24.077 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.630322+0000", 2024-09-11T07:37:24.077 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.077 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.077 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.077 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:24.077 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.630321+0000", 2024-09-11T07:37:24.077 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:24.077 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.077 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.077 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:24.078 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.630364+0000", 2024-09-11T07:37:24.078 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2724e-05 2024-09-11T07:37:24.078 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.078 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.078 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:24.078 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.630373+0000", 2024-09-11T07:37:24.078 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2e-06 2024-09-11T07:37:24.078 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.078 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.078 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:24.078 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.630378+0000", 2024-09-11T07:37:24.078 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9899999999999997e-06 2024-09-11T07:37:24.078 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.079 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.079 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:24.079 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.637263+0000", 2024-09-11T07:37:24.079 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.006884968 2024-09-11T07:37:24.079 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.079 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.079 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:24.079 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.637323+0000", 2024-09-11T07:37:24.079 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0560999999999998e-05 2024-09-11T07:37:24.079 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.079 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.079 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:24.079 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.670216+0000", 2024-09-11T07:37:24.079 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.032893146999999998 2024-09-11T07:37:24.080 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.080 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.080 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:24.080 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.670461+0000", 2024-09-11T07:37:24.080 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00024504799999999999 2024-09-11T07:37:24.080 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.080 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:24.080 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.080 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.080 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.080 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(osd.0.0:85925 8.6 e209/177 8:6052a415:::benchmark_data_smithi022_108406_object15637:head v 209'7162, mlcod=209'7148)", 2024-09-11T07:37:24.080 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:55.714613+0000", 2024-09-11T07:37:24.080 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 148.22826431499999, 2024-09-11T07:37:24.081 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.046135466999999999, 2024-09-11T07:37:24.081 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:24.081 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:24.081 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:24.081 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.081 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:24.081 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.714613+0000", 2024-09-11T07:37:24.081 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.081 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.081 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.081 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:24.081 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.714613+0000", 2024-09-11T07:37:24.081 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.082 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.082 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.082 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:24.082 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.714612+0000", 2024-09-11T07:37:24.082 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:24.082 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.082 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.082 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:24.082 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.714621+0000", 2024-09-11T07:37:24.082 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0240000000000003e-06 2024-09-11T07:37:24.082 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.082 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.082 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:24.083 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.714633+0000", 2024-09-11T07:37:24.083 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1958e-05 2024-09-11T07:37:24.083 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.083 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.083 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:24.083 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.714639+0000", 2024-09-11T07:37:24.083 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6640000000000003e-06 2024-09-11T07:37:24.083 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.083 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.083 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:24.083 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.718953+0000", 2024-09-11T07:37:24.083 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0043147239999999998 2024-09-11T07:37:24.083 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.084 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.084 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:24.084 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.718971+0000", 2024-09-11T07:37:24.084 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7549000000000001e-05 2024-09-11T07:37:24.084 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.084 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.084 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:24.084 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.760634+0000", 2024-09-11T07:37:24.084 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.041663444000000001 2024-09-11T07:37:24.084 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.084 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.084 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:24.084 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.760749+0000", 2024-09-11T07:37:24.085 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000114627 2024-09-11T07:37:24.085 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.085 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:24.085 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.085 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.085 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.085 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(osd.0.0:85926 8.6 e209/177 8:6052eb17:::benchmark_data_smithi022_108406_object56620:head v 209'7163, mlcod=209'7148)", 2024-09-11T07:37:24.085 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:55.714644+0000", 2024-09-11T07:37:24.085 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 148.228234222, 2024-09-11T07:37:24.085 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.049064611000000001, 2024-09-11T07:37:24.085 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:24.085 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:24.085 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:24.085 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.086 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:24.086 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.714644+0000", 2024-09-11T07:37:24.086 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.086 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.086 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.086 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:24.086 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.714644+0000", 2024-09-11T07:37:24.086 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.086 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.086 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.086 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:24.086 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.714643+0000", 2024-09-11T07:37:24.086 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:37:24.086 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.087 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.087 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:24.087 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.714649+0000", 2024-09-11T07:37:24.087 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5010000000000004e-06 2024-09-11T07:37:24.087 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.087 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.087 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:24.087 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.714653+0000", 2024-09-11T07:37:24.087 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1189999999999999e-06 2024-09-11T07:37:24.087 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.087 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.087 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:24.087 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.714656+0000", 2024-09-11T07:37:24.087 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2590000000000001e-06 2024-09-11T07:37:24.088 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.088 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.088 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:24.088 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.721551+0000", 2024-09-11T07:37:24.088 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0068946809999999997 2024-09-11T07:37:24.088 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.088 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.088 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:24.088 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.721569+0000", 2024-09-11T07:37:24.088 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7856999999999999e-05 2024-09-11T07:37:24.088 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.088 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.088 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:24.088 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.760752+0000", 2024-09-11T07:37:24.088 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.039183533 2024-09-11T07:37:24.089 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.089 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.089 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:24.089 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.763708+0000", 2024-09-11T07:37:24.089 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0029560649999999999 2024-09-11T07:37:24.089 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.089 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:24.089 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.089 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.089 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.089 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(osd.0.0:85927 8.6 e209/177 8:6053f009:::benchmark_data_smithi022_108406_object5904:head v 209'7164, mlcod=209'7148)", 2024-09-11T07:37:24.089 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:55.717452+0000", 2024-09-11T07:37:24.089 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 148.22542579099999, 2024-09-11T07:37:24.090 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.046347212999999998, 2024-09-11T07:37:24.090 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:24.090 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:24.090 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:24.090 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.090 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:24.090 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.717452+0000", 2024-09-11T07:37:24.090 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.090 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.090 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.090 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:24.090 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.717452+0000", 2024-09-11T07:37:24.090 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.090 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.091 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.091 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:24.091 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.717451+0000", 2024-09-11T07:37:24.091 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:37:24.091 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.091 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.091 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:24.091 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.717457+0000", 2024-09-11T07:37:24.091 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6699999999999999e-06 2024-09-11T07:37:24.091 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.091 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.091 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:24.091 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.717463+0000", 2024-09-11T07:37:24.091 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.699e-06 2024-09-11T07:37:24.092 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.092 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.092 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:24.092 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.717466+0000", 2024-09-11T07:37:24.092 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9509999999999999e-06 2024-09-11T07:37:24.092 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.092 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.092 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:24.092 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.724128+0000", 2024-09-11T07:37:24.092 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0066619840000000001 2024-09-11T07:37:24.092 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.092 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.093 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:24.093 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.724181+0000", 2024-09-11T07:37:24.093 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2352000000000003e-05 2024-09-11T07:37:24.093 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.093 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.093 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:24.093 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.763714+0000", 2024-09-11T07:37:24.093 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.039532678000000002 2024-09-11T07:37:24.093 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.093 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.093 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:24.093 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.763799+0000", 2024-09-11T07:37:24.093 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5661999999999994e-05 2024-09-11T07:37:24.094 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.094 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:24.094 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.094 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.094 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.094 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(osd.0.0:85928 8.6 e209/177 8:6054aaeb:::benchmark_data_smithi022_108406_object763:head v 209'7165, mlcod=209'7148)", 2024-09-11T07:37:24.094 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:55.719984+0000", 2024-09-11T07:37:24.094 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 148.22289336599999, 2024-09-11T07:37:24.094 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.043901813999999997, 2024-09-11T07:37:24.094 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:24.094 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:24.094 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:24.094 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.095 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:24.095 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.719984+0000", 2024-09-11T07:37:24.095 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.095 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.095 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.095 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:24.095 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.719984+0000", 2024-09-11T07:37:24.095 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.095 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.095 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.095 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:24.095 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.719983+0000", 2024-09-11T07:37:24.095 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:37:24.096 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.096 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.096 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:24.096 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.719990+0000", 2024-09-11T07:37:24.096 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1620000000000002e-06 2024-09-11T07:37:24.096 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.096 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.096 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:24.096 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.719999+0000", 2024-09-11T07:37:24.096 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3350000000000007e-06 2024-09-11T07:37:24.096 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.096 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.096 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:24.097 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.720004+0000", 2024-09-11T07:37:24.097 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1919999999999996e-06 2024-09-11T07:37:24.097 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.097 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.097 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:24.097 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.727009+0000", 2024-09-11T07:37:24.097 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.007004784 2024-09-11T07:37:24.097 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.097 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.097 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:24.097 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.727055+0000", 2024-09-11T07:37:24.097 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6539000000000003e-05 2024-09-11T07:37:24.097 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.098 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.098 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:24.098 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.763802+0000", 2024-09-11T07:37:24.098 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.036746276000000001 2024-09-11T07:37:24.098 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.098 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.098 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:24.098 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.763886+0000", 2024-09-11T07:37:24.098 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4656999999999999e-05 2024-09-11T07:37:24.098 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.098 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:24.098 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.098 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.098 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.098 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(osd.0.0:85929 8.6 e209/177 8:605648de:::benchmark_data_smithi022_108406_object62683:head v 209'7166, mlcod=209'7148)", 2024-09-11T07:37:24.099 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:55.721473+0000", 2024-09-11T07:37:24.099 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 148.221404713, 2024-09-11T07:37:24.099 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.04250785, 2024-09-11T07:37:24.099 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:24.099 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:24.099 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:24.099 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.099 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:24.099 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.721473+0000", 2024-09-11T07:37:24.099 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.099 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.099 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.099 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:24.099 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.721473+0000", 2024-09-11T07:37:24.100 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.100 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.100 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.100 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:24.100 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.721472+0000", 2024-09-11T07:37:24.100 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:37:24.100 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.100 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.100 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:24.100 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.721478+0000", 2024-09-11T07:37:24.100 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.48e-06 2024-09-11T07:37:24.100 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.100 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.100 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:24.101 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.721484+0000", 2024-09-11T07:37:24.101 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.737e-06 2024-09-11T07:37:24.101 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.101 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.101 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:24.101 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.721488+0000", 2024-09-11T07:37:24.101 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0890000000000002e-06 2024-09-11T07:37:24.101 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.101 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.101 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:24.104 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.729606+0000", 2024-09-11T07:37:24.104 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0081172220000000003 2024-09-11T07:37:24.104 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.104 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.104 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:24.104 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.729622+0000", 2024-09-11T07:37:24.104 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6396e-05 2024-09-11T07:37:24.104 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.104 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.105 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:24.105 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.763889+0000", 2024-09-11T07:37:24.105 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.03426655 2024-09-11T07:37:24.105 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.105 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.105 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:24.105 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.763981+0000", 2024-09-11T07:37:24.105 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2232999999999996e-05 2024-09-11T07:37:24.105 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.105 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:24.105 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.105 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.105 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.106 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25072.0:113895 8.6 e209/177 8:6c4908cb:::benchmark_data_smithi022_108406_object113894:head v 209'7171, mlcod=209'7159)", 2024-09-11T07:37:24.106 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:55.734399+0000", 2024-09-11T07:37:24.106 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 148.20847903200001, 2024-09-11T07:37:24.106 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.069776878000000001, 2024-09-11T07:37:24.106 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:24.106 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:24.106 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:24.106 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.106 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:24.106 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.734399+0000", 2024-09-11T07:37:24.106 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.106 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.106 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.107 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:24.107 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.734399+0000", 2024-09-11T07:37:24.107 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.107 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.107 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.107 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:24.107 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.734397+0000", 2024-09-11T07:37:24.107 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:24.107 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.107 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.107 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:24.107 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.734432+0000", 2024-09-11T07:37:24.107 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4827999999999999e-05 2024-09-11T07:37:24.107 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.108 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.108 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:24.108 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.734448+0000", 2024-09-11T07:37:24.108 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6464e-05 2024-09-11T07:37:24.108 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.108 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.108 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:24.108 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.734454+0000", 2024-09-11T07:37:24.108 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0220000000000001e-06 2024-09-11T07:37:24.108 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.108 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.108 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:24.108 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.754655+0000", 2024-09-11T07:37:24.109 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.020200215000000001 2024-09-11T07:37:24.109 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.109 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.109 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:24.109 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.754671+0000", 2024-09-11T07:37:24.109 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6557000000000001e-05 2024-09-11T07:37:24.109 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.109 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.109 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:24.109 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.804007+0000", 2024-09-11T07:37:24.109 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.049336049999999999 2024-09-11T07:37:24.109 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.109 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.110 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:24.110 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.804176+0000", 2024-09-11T07:37:24.110 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000168388 2024-09-11T07:37:24.110 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.110 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:24.110 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.110 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.110 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.110 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25072.0:113949 8.6 e209/177 8:6402dfe1:::benchmark_data_smithi022_108406_object113948:head v 209'7172, mlcod=209'7159)", 2024-09-11T07:37:24.110 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:55.740162+0000", 2024-09-11T07:37:24.110 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 148.20271586000001, 2024-09-11T07:37:24.110 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.064141514999999996, 2024-09-11T07:37:24.110 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:24.111 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:24.111 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:24.111 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.111 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:24.111 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.740162+0000", 2024-09-11T07:37:24.111 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.111 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.111 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.111 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:24.111 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.740162+0000", 2024-09-11T07:37:24.111 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.111 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.111 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.111 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:24.112 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.740161+0000", 2024-09-11T07:37:24.112 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:24.112 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.112 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.112 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:24.112 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.740216+0000", 2024-09-11T07:37:24.112 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4951999999999998e-05 2024-09-11T07:37:24.112 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.112 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.112 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:24.112 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.740224+0000", 2024-09-11T07:37:24.112 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3270000000000001e-06 2024-09-11T07:37:24.112 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.113 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.113 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:24.113 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.740229+0000", 2024-09-11T07:37:24.113 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8230000000000002e-06 2024-09-11T07:37:24.113 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.113 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.113 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:24.113 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.757491+0000", 2024-09-11T07:37:24.113 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.017262394 2024-09-11T07:37:24.113 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.113 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.113 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:24.113 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.757611+0000", 2024-09-11T07:37:24.114 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000120099 2024-09-11T07:37:24.114 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.114 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.114 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:24.114 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.804181+0000", 2024-09-11T07:37:24.114 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.046569630000000001 2024-09-11T07:37:24.114 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.114 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.114 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:24.114 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:55.804303+0000", 2024-09-11T07:37:24.114 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012262199999999999 2024-09-11T07:37:24.114 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.114 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:24.114 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.115 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.115 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.115 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25072.0:121555 8.6 e213/177 8:6d5eba34:::benchmark_data_smithi022_108406_object121554:head v 213'8535, mlcod=213'8533)", 2024-09-11T07:37:24.115 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:35:03.267811+0000", 2024-09-11T07:37:24.115 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 140.675066673, 2024-09-11T07:37:24.115 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.049213188999999997, 2024-09-11T07:37:24.115 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:24.115 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:24.115 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:24.115 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.115 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:24.115 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.267811+0000", 2024-09-11T07:37:24.115 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.115 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.116 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.116 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:24.116 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.267811+0000", 2024-09-11T07:37:24.116 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.116 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.116 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.116 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:24.116 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.267810+0000", 2024-09-11T07:37:24.116 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:37:24.116 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.116 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.116 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:24.116 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.267869+0000", 2024-09-11T07:37:24.116 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8520000000000002e-05 2024-09-11T07:37:24.117 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.117 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.117 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:24.117 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.267876+0000", 2024-09-11T07:37:24.117 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.8140000000000001e-06 2024-09-11T07:37:24.117 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.117 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.117 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:24.117 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.267881+0000", 2024-09-11T07:37:24.117 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4870000000000004e-06 2024-09-11T07:37:24.117 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.117 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.117 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:24.117 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.275616+0000", 2024-09-11T07:37:24.118 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0077351349999999998 2024-09-11T07:37:24.118 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.118 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.118 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:24.118 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.275634+0000", 2024-09-11T07:37:24.118 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8159999999999999e-05 2024-09-11T07:37:24.118 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.118 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.118 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:24.118 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.316967+0000", 2024-09-11T07:37:24.118 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.041332801000000002 2024-09-11T07:37:24.118 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.118 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.118 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:24.119 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.317024+0000", 2024-09-11T07:37:24.119 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7346000000000003e-05 2024-09-11T07:37:24.119 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.119 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:24.119 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.119 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.119 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.119 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25072.0:121557 8.6 e213/177 8:6f954906:::benchmark_data_smithi022_108406_object121556:head v 213'8536, mlcod=213'8533)", 2024-09-11T07:37:24.119 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:35:03.272456+0000", 2024-09-11T07:37:24.119 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 140.670421981, 2024-09-11T07:37:24.119 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.044598120999999998, 2024-09-11T07:37:24.119 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:24.119 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:24.120 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:24.120 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.120 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:24.120 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.272456+0000", 2024-09-11T07:37:24.120 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.120 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.120 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.120 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:24.120 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.272456+0000", 2024-09-11T07:37:24.120 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.120 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.120 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.120 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:24.120 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.272455+0000", 2024-09-11T07:37:24.121 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:24.121 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.121 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.121 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:24.121 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.272535+0000", 2024-09-11T07:37:24.121 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0671000000000001e-05 2024-09-11T07:37:24.121 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.121 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.121 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:24.121 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.272546+0000", 2024-09-11T07:37:24.121 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0811e-05 2024-09-11T07:37:24.121 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.121 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.121 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:24.122 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.272551+0000", 2024-09-11T07:37:24.122 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3870000000000003e-06 2024-09-11T07:37:24.122 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.122 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.122 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:24.122 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.280885+0000", 2024-09-11T07:37:24.122 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0083337040000000008 2024-09-11T07:37:24.122 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.122 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.122 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:24.122 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.280905+0000", 2024-09-11T07:37:24.122 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9415e-05 2024-09-11T07:37:24.122 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.122 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.123 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:24.123 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.317029+0000", 2024-09-11T07:37:24.123 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.036124725000000003 2024-09-11T07:37:24.123 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.123 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.123 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:24.123 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.317054+0000", 2024-09-11T07:37:24.123 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.4612000000000001e-05 2024-09-11T07:37:24.123 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.123 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:24.123 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.123 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.123 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.124 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25072.0:121559 8.6 e213/177 8:6b3365aa:::benchmark_data_smithi022_108406_object121558:head v 213'8537, mlcod=213'8533)", 2024-09-11T07:37:24.124 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:35:03.275507+0000", 2024-09-11T07:37:24.124 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 140.66737033699999, 2024-09-11T07:37:24.124 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.041568991, 2024-09-11T07:37:24.124 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:37:24.124 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:24.124 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:37:24.124 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.124 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:37:24.124 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.275507+0000", 2024-09-11T07:37:24.124 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.124 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.124 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.125 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:37:24.125 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.275507+0000", 2024-09-11T07:37:24.125 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:37:24.125 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.125 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.125 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:37:24.125 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.275506+0000", 2024-09-11T07:37:24.125 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:24.125 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.125 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.125 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:37:24.125 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.275570+0000", 2024-09-11T07:37:24.125 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4511999999999995e-05 2024-09-11T07:37:24.125 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.126 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.126 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:37:24.126 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.275579+0000", 2024-09-11T07:37:24.126 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3930000000000008e-06 2024-09-11T07:37:24.126 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.126 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.126 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:37:24.126 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.275584+0000", 2024-09-11T07:37:24.126 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9660000000000004e-06 2024-09-11T07:37:24.126 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.126 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.126 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:37:24.126 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.286517+0000", 2024-09-11T07:37:24.126 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.010933346 2024-09-11T07:37:24.127 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.127 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.127 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:37:24.127 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.286534+0000", 2024-09-11T07:37:24.127 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6591000000000001e-05 2024-09-11T07:37:24.127 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.127 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.127 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:37:24.127 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.317058+0000", 2024-09-11T07:37:24.127 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.030524279000000001 2024-09-11T07:37:24.127 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:37:24.127 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:37:24.127 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:37:24.127 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:35:03.317076+0000", 2024-09-11T07:37:24.128 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8434e-05 2024-09-11T07:37:24.128 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.128 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:24.128 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.128 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:37:24.128 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:37:24.128 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:37:24.128 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:24.152 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:24.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:24.153 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:24.254 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:24.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:24.276 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:24.276 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:24.279 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:37:24 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:37:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:37:24.377 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:24.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:24.399 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:24.400 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:24.500 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:24.521 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:24.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:24.522 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:24.623 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:24.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:24.644 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:24.645 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:24.745 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:24.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:24.769 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:24.769 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:24.843 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'unique_pool_2' created 2024-09-11T07:37:24.871 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:24.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:24.894 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:24.894 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:24.995 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:25.017 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:25.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:25.018 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:25.023 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-11T07:37:25.119 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:25.140 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:25.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:25.142 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:25.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:24 smithi022 ceph-mon[37330]: 2.14 scrub starts 2024-09-11T07:37:25.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:24 smithi022 ceph-mon[37330]: 2.14 scrub ok 2024-09-11T07:37:25.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:24 smithi022 ceph-mon[37330]: 6.7 scrub starts 2024-09-11T07:37:25.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:24 smithi022 ceph-mon[37330]: 6.7 scrub ok 2024-09-11T07:37:25.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:24 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:37:25.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:24 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:37:25.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:24 smithi022 ceph-mon[29462]: 2.14 scrub starts 2024-09-11T07:37:25.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:24 smithi022 ceph-mon[29462]: 2.14 scrub ok 2024-09-11T07:37:25.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:24 smithi022 ceph-mon[29462]: 6.7 scrub starts 2024-09-11T07:37:25.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:24 smithi022 ceph-mon[29462]: 6.7 scrub ok 2024-09-11T07:37:25.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:24 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:37:25.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:24 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:37:25.243 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:25.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:25.265 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:25.265 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:25.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:24 smithi104 ceph-mon[33507]: 2.14 scrub starts 2024-09-11T07:37:25.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:24 smithi104 ceph-mon[33507]: 2.14 scrub ok 2024-09-11T07:37:25.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:24 smithi104 ceph-mon[33507]: 6.7 scrub starts 2024-09-11T07:37:25.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:24 smithi104 ceph-mon[33507]: 6.7 scrub ok 2024-09-11T07:37:25.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:24 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:37:25.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:24 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:37:25.365 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:25.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:25.390 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:25.390 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:25.491 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:25.512 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:25.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:25.513 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:25.614 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:25.635 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:25.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:25.636 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:25.736 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:25.758 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:25.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:25.759 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:25.860 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:25.882 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:25.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:25.883 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:25.984 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:26.006 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:26.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:26.007 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:26.108 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:26.130 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:26.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:26.131 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:26.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:25 smithi022 ceph-mon[29462]: pgmap v485: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 540 B/s rd, 0 op/s 2024-09-11T07:37:26.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:25 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2245019477' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-09-11T07:37:26.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:25 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2245019477' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-09-11T07:37:26.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:25 smithi022 ceph-mon[29462]: osdmap e247: 8 total, 8 up, 8 in 2024-09-11T07:37:26.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:25 smithi022 ceph-mon[37330]: pgmap v485: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 540 B/s rd, 0 op/s 2024-09-11T07:37:26.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:25 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2245019477' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-09-11T07:37:26.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:25 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2245019477' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-09-11T07:37:26.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:25 smithi022 ceph-mon[37330]: osdmap e247: 8 total, 8 up, 8 in 2024-09-11T07:37:26.231 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:26.253 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:26.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:26.254 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:26.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:25 smithi104 ceph-mon[33507]: pgmap v485: 132 pgs: 132 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 540 B/s rd, 0 op/s 2024-09-11T07:37:26.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:25 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2245019477' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-09-11T07:37:26.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:25 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2245019477' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-09-11T07:37:26.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:25 smithi104 ceph-mon[33507]: osdmap e247: 8 total, 8 up, 8 in 2024-09-11T07:37:26.354 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:26.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:26.378 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:26.379 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:26.479 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:26.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:26.502 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:26.503 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:26.604 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:26.626 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:26.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:26.628 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:26.700 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pool application enable unique_pool_2 rados --yes-i-really-mean-it || true 2024-09-11T07:37:26.729 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:26.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:26.767 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:26.767 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:26.868 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:26.892 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:26.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:26.893 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:26.937 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:26 smithi022 ceph-mon[29462]: osdmap e248: 8 total, 8 up, 8 in 2024-09-11T07:37:26.937 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:26 smithi022 ceph-mon[29462]: 2.d scrub starts 2024-09-11T07:37:26.938 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:26 smithi022 ceph-mon[29462]: 2.d scrub ok 2024-09-11T07:37:26.938 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:26 smithi022 ceph-mon[29462]: 5.3 scrub starts 2024-09-11T07:37:26.938 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:26 smithi022 ceph-mon[29462]: 5.3 scrub ok 2024-09-11T07:37:26.938 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:26 smithi022 ceph-mon[29462]: pgmap v488: 148 pgs: 3 creating+peering, 13 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:26.938 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:26 smithi022 ceph-mon[29462]: 4.17 scrub starts 2024-09-11T07:37:26.938 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:26 smithi022 ceph-mon[29462]: 4.17 scrub ok 2024-09-11T07:37:26.938 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:26 smithi022 ceph-mon[29462]: 6.17 scrub starts 2024-09-11T07:37:26.939 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:26 smithi022 ceph-mon[29462]: 6.17 scrub ok 2024-09-11T07:37:26.939 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:26 smithi022 ceph-mon[37330]: osdmap e248: 8 total, 8 up, 8 in 2024-09-11T07:37:26.939 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:26 smithi022 ceph-mon[37330]: 2.d scrub starts 2024-09-11T07:37:26.939 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:26 smithi022 ceph-mon[37330]: 2.d scrub ok 2024-09-11T07:37:26.939 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:26 smithi022 ceph-mon[37330]: 5.3 scrub starts 2024-09-11T07:37:26.939 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:26 smithi022 ceph-mon[37330]: 5.3 scrub ok 2024-09-11T07:37:26.940 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:26 smithi022 ceph-mon[37330]: pgmap v488: 148 pgs: 3 creating+peering, 13 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:26.940 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:26 smithi022 ceph-mon[37330]: 4.17 scrub starts 2024-09-11T07:37:26.940 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:26 smithi022 ceph-mon[37330]: 4.17 scrub ok 2024-09-11T07:37:26.940 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:26 smithi022 ceph-mon[37330]: 6.17 scrub starts 2024-09-11T07:37:26.940 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:26 smithi022 ceph-mon[37330]: 6.17 scrub ok 2024-09-11T07:37:26.994 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:27.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:27.017 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:27.018 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:27.118 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:27.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:27.144 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:27.145 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:27.246 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:27.269 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:27.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:27.271 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:27.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:26 smithi104 ceph-mon[33507]: osdmap e248: 8 total, 8 up, 8 in 2024-09-11T07:37:27.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:26 smithi104 ceph-mon[33507]: 2.d scrub starts 2024-09-11T07:37:27.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:26 smithi104 ceph-mon[33507]: 2.d scrub ok 2024-09-11T07:37:27.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:26 smithi104 ceph-mon[33507]: 5.3 scrub starts 2024-09-11T07:37:27.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:26 smithi104 ceph-mon[33507]: 5.3 scrub ok 2024-09-11T07:37:27.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:26 smithi104 ceph-mon[33507]: pgmap v488: 148 pgs: 3 creating+peering, 13 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:27.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:26 smithi104 ceph-mon[33507]: 4.17 scrub starts 2024-09-11T07:37:27.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:26 smithi104 ceph-mon[33507]: 4.17 scrub ok 2024-09-11T07:37:27.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:26 smithi104 ceph-mon[33507]: 6.17 scrub starts 2024-09-11T07:37:27.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:26 smithi104 ceph-mon[33507]: 6.17 scrub ok 2024-09-11T07:37:27.372 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:27.394 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:27.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:27.396 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:27.496 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:27.520 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:27.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:27.521 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:27.577 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:37:27.622 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:27.645 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:27.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:27.647 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:27.700 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:37:27.748 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:27.774 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:27.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:27.775 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:27.876 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:27.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:27.902 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:27.902 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:28.003 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:28.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:28.027 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:28.028 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:28.128 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:28.149 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:27 smithi022 ceph-mon[29462]: 2.0 scrub starts 2024-09-11T07:37:28.149 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:27 smithi022 ceph-mon[29462]: 2.0 scrub ok 2024-09-11T07:37:28.149 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:27 smithi022 ceph-mon[29462]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:37:28.149 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:27 smithi022 ceph-mon[29462]: osdmap e249: 8 total, 8 up, 8 in 2024-09-11T07:37:28.149 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:27 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:37:28.149 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:27 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:37:28.150 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:27 smithi022 ceph-mon[37330]: 2.0 scrub starts 2024-09-11T07:37:28.150 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:27 smithi022 ceph-mon[37330]: 2.0 scrub ok 2024-09-11T07:37:28.150 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:27 smithi022 ceph-mon[37330]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:37:28.150 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:27 smithi022 ceph-mon[37330]: osdmap e249: 8 total, 8 up, 8 in 2024-09-11T07:37:28.150 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:27 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:37:28.150 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:27 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:37:28.152 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:28.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:28.153 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:28.253 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:28.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:27 smithi104 ceph-mon[33507]: 2.0 scrub starts 2024-09-11T07:37:28.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:27 smithi104 ceph-mon[33507]: 2.0 scrub ok 2024-09-11T07:37:28.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:27 smithi104 ceph-mon[33507]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:37:28.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:27 smithi104 ceph-mon[33507]: osdmap e249: 8 total, 8 up, 8 in 2024-09-11T07:37:28.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:27 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:37:28.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:27 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:37:28.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:28.281 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:28.281 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:28.362 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:37:28.382 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:28.407 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:28.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:28.408 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:28.509 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:28.531 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:28.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:28.532 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:28.633 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:28.658 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:28.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:28.659 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:28.760 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:28.784 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:28.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:28.785 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:28.859 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is unset 2024-09-11T07:37:28.860 INFO:teuthology.orchestra.run.smithi022.stderr:reweighted osd.7 to 1 (10000) 2024-09-11T07:37:28.885 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:28.912 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:28.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:28.913 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:29.014 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:29.036 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:29.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:29.038 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:29.102 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:28 smithi022 ceph-mon[29462]: osdmap e250: 8 total, 8 up, 8 in 2024-09-11T07:37:29.102 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:28 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3664823118' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T07:37:29.102 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:28 smithi022 ceph-mon[29462]: from='client.25352 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T07:37:29.102 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:28 smithi022 ceph-mon[29462]: pgmap v491: 148 pgs: 3 creating+peering, 13 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:29.102 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:28 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2184936154' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:37:29.102 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:28 smithi022 ceph-mon[29462]: 6.2 scrub starts 2024-09-11T07:37:29.102 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:28 smithi022 ceph-mon[29462]: 6.2 scrub ok 2024-09-11T07:37:29.103 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:28 smithi022 ceph-mon[37330]: osdmap e250: 8 total, 8 up, 8 in 2024-09-11T07:37:29.103 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:28 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3664823118' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T07:37:29.103 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:28 smithi022 ceph-mon[37330]: from='client.25352 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T07:37:29.103 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:28 smithi022 ceph-mon[37330]: pgmap v491: 148 pgs: 3 creating+peering, 13 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:29.103 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:28 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2184936154' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:37:29.103 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:28 smithi022 ceph-mon[37330]: 6.2 scrub starts 2024-09-11T07:37:29.103 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:28 smithi022 ceph-mon[37330]: 6.2 scrub ok 2024-09-11T07:37:29.139 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:29.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:29.163 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:29.163 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:29.264 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:29.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:28 smithi104 ceph-mon[33507]: osdmap e250: 8 total, 8 up, 8 in 2024-09-11T07:37:29.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:28 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3664823118' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T07:37:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:28 smithi104 ceph-mon[33507]: from='client.25352 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T07:37:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:28 smithi104 ceph-mon[33507]: pgmap v491: 148 pgs: 3 creating+peering, 13 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:28 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2184936154' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:37:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:28 smithi104 ceph-mon[33507]: 6.2 scrub starts 2024-09-11T07:37:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:28 smithi104 ceph-mon[33507]: 6.2 scrub ok 2024-09-11T07:37:29.286 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:29.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:29.287 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:29.388 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:29.414 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:29.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:29.415 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:29.516 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:29.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:29.539 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:29.540 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:29.640 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:29.668 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:29.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:29.669 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:29.770 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:29.791 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:29.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:29.793 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:29.862 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:37:29 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:37:29.893 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:29.916 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:29.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:29.918 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:30.019 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:30.041 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:30.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:30.042 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:30.143 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:30.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:29 smithi022 ceph-mon[37330]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:37:30.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:29 smithi022 ceph-mon[37330]: from='client.25352 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-11T07:37:30.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:29 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2184936154' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:37:30.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:29 smithi022 ceph-mon[37330]: osdmap e251: 8 total, 8 up, 8 in 2024-09-11T07:37:30.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:29 smithi022 ceph-mon[37330]: 6.1b deep-scrub starts 2024-09-11T07:37:30.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:29 smithi022 ceph-mon[37330]: 6.1b deep-scrub ok 2024-09-11T07:37:30.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:29 smithi022 ceph-mon[37330]: 4.1e scrub starts 2024-09-11T07:37:30.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:29 smithi022 ceph-mon[37330]: 4.1e scrub ok 2024-09-11T07:37:30.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:29 smithi022 ceph-mon[29462]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:37:30.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:29 smithi022 ceph-mon[29462]: from='client.25352 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-11T07:37:30.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:29 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2184936154' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:37:30.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:29 smithi022 ceph-mon[29462]: osdmap e251: 8 total, 8 up, 8 in 2024-09-11T07:37:30.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:29 smithi022 ceph-mon[29462]: 6.1b deep-scrub starts 2024-09-11T07:37:30.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:29 smithi022 ceph-mon[29462]: 6.1b deep-scrub ok 2024-09-11T07:37:30.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:29 smithi022 ceph-mon[29462]: 4.1e scrub starts 2024-09-11T07:37:30.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:29 smithi022 ceph-mon[29462]: 4.1e scrub ok 2024-09-11T07:37:30.169 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:30.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:30.170 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:30.271 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:30.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:29 smithi104 ceph-mon[33507]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:37:30.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:29 smithi104 ceph-mon[33507]: from='client.25352 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-11T07:37:30.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:29 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2184936154' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:37:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:29 smithi104 ceph-mon[33507]: osdmap e251: 8 total, 8 up, 8 in 2024-09-11T07:37:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:29 smithi104 ceph-mon[33507]: 6.1b deep-scrub starts 2024-09-11T07:37:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:29 smithi104 ceph-mon[33507]: 6.1b deep-scrub ok 2024-09-11T07:37:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:29 smithi104 ceph-mon[33507]: 4.1e scrub starts 2024-09-11T07:37:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:29 smithi104 ceph-mon[33507]: 4.1e scrub ok 2024-09-11T07:37:30.296 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:30.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:30.297 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:30.398 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:30.421 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:30.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:30.422 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:30.516 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:37:30.523 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:30.546 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:30.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:30.547 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:30.648 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:30.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:37:30 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:37:30] "GET /metrics HTTP/1.1" 200 221752 "" "Prometheus/2.43.0" 2024-09-11T07:37:30.674 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:30.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:30.675 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:30.776 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:30.801 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:30.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:30.802 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:30.896 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-11T07:37:30.896 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:37:30.902 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:30.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:30.962 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:30.962 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:31.063 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:31.088 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:31.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:31.089 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:31.129 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:30 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:31.129 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:30 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:37:31.129 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:30 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:37:31.129 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:30 smithi022 ceph-mon[37330]: osdmap e252: 8 total, 8 up, 8 in 2024-09-11T07:37:31.129 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:30 smithi022 ceph-mon[37330]: pgmap v494: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:31.129 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:30 smithi022 ceph-mon[37330]: 5.e scrub starts 2024-09-11T07:37:31.130 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:30 smithi022 ceph-mon[37330]: 5.e scrub ok 2024-09-11T07:37:31.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:30 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:31.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:30 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:37:31.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:30 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:37:31.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:30 smithi022 ceph-mon[29462]: osdmap e252: 8 total, 8 up, 8 in 2024-09-11T07:37:31.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:30 smithi022 ceph-mon[29462]: pgmap v494: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:31.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:30 smithi022 ceph-mon[29462]: 5.e scrub starts 2024-09-11T07:37:31.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:30 smithi022 ceph-mon[29462]: 5.e scrub ok 2024-09-11T07:37:31.190 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:31.214 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:31.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:31.215 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:31.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:30 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:31.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:30 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:37:31.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:30 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:37:31.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:30 smithi104 ceph-mon[33507]: osdmap e252: 8 total, 8 up, 8 in 2024-09-11T07:37:31.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:30 smithi104 ceph-mon[33507]: pgmap v494: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:37:31.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:30 smithi104 ceph-mon[33507]: 5.e scrub starts 2024-09-11T07:37:31.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:30 smithi104 ceph-mon[33507]: 5.e scrub ok 2024-09-11T07:37:31.315 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:31.338 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:31.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:31.339 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:31.440 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:31.464 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:31.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:31.465 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:31.566 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:31.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:31.591 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:31.591 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:31.692 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:31.717 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:31.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:31.718 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:31.819 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:31.841 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:31.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:31.842 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:31.917 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T07:37:31.942 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:31.984 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:31.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:31.985 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:32.086 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:32.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:32.114 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:32.114 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:32.215 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:32.241 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:32.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:32.242 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:32.343 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:32.368 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:32.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:32.369 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:32.470 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:32.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:32.493 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:32.493 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:32.594 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:32.616 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:32.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:32.617 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:32.718 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:32.744 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:32.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:32.745 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:32.846 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:32.868 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:32.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:32.869 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:32.933 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:37:32.970 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:32.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:32.994 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:32.994 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:33.095 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:33.117 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:33.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:33.118 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:33.219 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:33.240 INFO:teuthology.orchestra.run.smithi022.stderr:enabled application 'rados' on pool 'unique_pool_2' 2024-09-11T07:37:33.245 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:33.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:33.246 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:33.347 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:33.373 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:33.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:33.374 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:33.475 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:33.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:33 smithi022 ceph-mon[29462]: 6.1f scrub starts 2024-09-11T07:37:33.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:33 smithi022 ceph-mon[29462]: 6.1f scrub ok 2024-09-11T07:37:33.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:33 smithi022 ceph-mon[29462]: pgmap v495: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 961 B/s rd, 0 op/s 2024-09-11T07:37:33.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:33 smithi022 ceph-mon[29462]: 6.10 scrub starts 2024-09-11T07:37:33.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:33 smithi022 ceph-mon[29462]: 6.10 scrub ok 2024-09-11T07:37:33.484 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:33 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/723883873' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:37:33.485 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:33 smithi022 ceph-mon[37330]: 6.1f scrub starts 2024-09-11T07:37:33.485 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:33 smithi022 ceph-mon[37330]: 6.1f scrub ok 2024-09-11T07:37:33.485 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:33 smithi022 ceph-mon[37330]: pgmap v495: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 961 B/s rd, 0 op/s 2024-09-11T07:37:33.485 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:33 smithi022 ceph-mon[37330]: 6.10 scrub starts 2024-09-11T07:37:33.485 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:33 smithi022 ceph-mon[37330]: 6.10 scrub ok 2024-09-11T07:37:33.485 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:33 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/723883873' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:37:33.500 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:33.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:33.501 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:33.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:33 smithi104 ceph-mon[33507]: 6.1f scrub starts 2024-09-11T07:37:33.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:33 smithi104 ceph-mon[33507]: 6.1f scrub ok 2024-09-11T07:37:33.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:33 smithi104 ceph-mon[33507]: pgmap v495: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 961 B/s rd, 0 op/s 2024-09-11T07:37:33.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:33 smithi104 ceph-mon[33507]: 6.10 scrub starts 2024-09-11T07:37:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:33 smithi104 ceph-mon[33507]: 6.10 scrub ok 2024-09-11T07:37:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:33 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/723883873' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:37:33.602 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:33.627 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:33.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:33.629 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:33.729 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:33.755 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:33.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:33.756 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:33.856 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:33.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:33.880 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:33.880 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:33.981 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:34.007 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:34.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:34.008 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:34.108 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:34.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:34.133 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:34.133 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:34.234 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:34.259 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:34.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:34.260 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:34.361 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:34.385 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:34.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:34.386 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:34.487 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:34.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:34.513 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:34.514 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:34.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:37:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:37:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:37:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:34 smithi104 ceph-mon[33507]: 2.6 scrub starts 2024-09-11T07:37:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:34 smithi104 ceph-mon[33507]: 2.6 scrub ok 2024-09-11T07:37:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:34 smithi104 ceph-mon[33507]: 5.5 scrub starts 2024-09-11T07:37:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:34 smithi104 ceph-mon[33507]: 5.5 scrub ok 2024-09-11T07:37:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:34 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/723883873' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:37:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:34 smithi104 ceph-mon[33507]: osdmap e253: 8 total, 8 up, 8 in 2024-09-11T07:37:34.615 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:34.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:34 smithi022 ceph-mon[29462]: 2.6 scrub starts 2024-09-11T07:37:34.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:34 smithi022 ceph-mon[29462]: 2.6 scrub ok 2024-09-11T07:37:34.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:34 smithi022 ceph-mon[29462]: 5.5 scrub starts 2024-09-11T07:37:34.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:34 smithi022 ceph-mon[29462]: 5.5 scrub ok 2024-09-11T07:37:34.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:34 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/723883873' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:37:34.632 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:34 smithi022 ceph-mon[29462]: osdmap e253: 8 total, 8 up, 8 in 2024-09-11T07:37:34.632 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:34 smithi022 ceph-mon[37330]: 2.6 scrub starts 2024-09-11T07:37:34.632 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:34 smithi022 ceph-mon[37330]: 2.6 scrub ok 2024-09-11T07:37:34.632 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:34 smithi022 ceph-mon[37330]: 5.5 scrub starts 2024-09-11T07:37:34.632 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:34 smithi022 ceph-mon[37330]: 5.5 scrub ok 2024-09-11T07:37:34.632 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:34 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/723883873' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:37:34.632 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:34 smithi022 ceph-mon[37330]: osdmap e253: 8 total, 8 up, 8 in 2024-09-11T07:37:34.640 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:34.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:34.641 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:34.742 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:34.766 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:34.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:34.767 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:34.848 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:37:34.848 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:37:34.848 INFO:teuthology.orchestra.run.smithi022.stdout: "num_ops": 0 2024-09-11T07:37:34.848 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:37:34.868 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:34.894 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:34.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:34.895 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:34.996 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:35.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:35.020 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:35.021 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:35.121 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:35.144 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:35.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:35.145 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:35.245 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:35.268 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:35.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:35.269 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:35.370 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:35.394 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:35.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:35.395 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:35.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:35 smithi022 ceph-mon[37330]: pgmap v497: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:35.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:35 smithi022 ceph-mon[37330]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:37:35.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:35 smithi022 ceph-mon[37330]: Cluster is now healthy 2024-09-11T07:37:35.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:35 smithi022 ceph-mon[37330]: 4.9 deep-scrub starts 2024-09-11T07:37:35.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:35 smithi022 ceph-mon[37330]: 4.9 deep-scrub ok 2024-09-11T07:37:35.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:35 smithi022 ceph-mon[29462]: pgmap v497: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:35.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:35 smithi022 ceph-mon[29462]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:37:35.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:35 smithi022 ceph-mon[29462]: Cluster is now healthy 2024-09-11T07:37:35.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:35 smithi022 ceph-mon[29462]: 4.9 deep-scrub starts 2024-09-11T07:37:35.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:35 smithi022 ceph-mon[29462]: 4.9 deep-scrub ok 2024-09-11T07:37:35.495 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:35.518 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:35.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:35.519 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:35.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:35 smithi104 ceph-mon[33507]: pgmap v497: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:37:35.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:35 smithi104 ceph-mon[33507]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:37:35.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:35 smithi104 ceph-mon[33507]: Cluster is now healthy 2024-09-11T07:37:35.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:35 smithi104 ceph-mon[33507]: 4.9 deep-scrub starts 2024-09-11T07:37:35.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:35 smithi104 ceph-mon[33507]: 4.9 deep-scrub ok 2024-09-11T07:37:35.539 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-11T07:37:35.620 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:35.644 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:35.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:35.645 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:35.746 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:35.769 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:35.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:35.770 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:35.871 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:35.893 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:35.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:35.894 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:35.995 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:36.021 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:36.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:36.022 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:36.123 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:36.145 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:36.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:36.146 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:36.247 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:36.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:36.270 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:36.270 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:36.371 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:36.397 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:36.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:36.398 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:36.499 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:36.512 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:36 smithi022 ceph-mon[29462]: 2.1d scrub starts 2024-09-11T07:37:36.512 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:36 smithi022 ceph-mon[29462]: 2.1d scrub ok 2024-09-11T07:37:36.512 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:36 smithi022 ceph-mon[29462]: 2.3 scrub starts 2024-09-11T07:37:36.512 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:36 smithi022 ceph-mon[29462]: 2.3 scrub ok 2024-09-11T07:37:36.512 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:36 smithi022 ceph-mon[29462]: 4.1 scrub starts 2024-09-11T07:37:36.512 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:36 smithi022 ceph-mon[29462]: 4.1 scrub ok 2024-09-11T07:37:36.512 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:36 smithi022 ceph-mon[37330]: 2.1d scrub starts 2024-09-11T07:37:36.512 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:36 smithi022 ceph-mon[37330]: 2.1d scrub ok 2024-09-11T07:37:36.513 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:36 smithi022 ceph-mon[37330]: 2.3 scrub starts 2024-09-11T07:37:36.513 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:36 smithi022 ceph-mon[37330]: 2.3 scrub ok 2024-09-11T07:37:36.513 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:36 smithi022 ceph-mon[37330]: 4.1 scrub starts 2024-09-11T07:37:36.513 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:36 smithi022 ceph-mon[37330]: 4.1 scrub ok 2024-09-11T07:37:36.524 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:36.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:36.525 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:36.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:36 smithi104 ceph-mon[33507]: 2.1d scrub starts 2024-09-11T07:37:36.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:36 smithi104 ceph-mon[33507]: 2.1d scrub ok 2024-09-11T07:37:36.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:36 smithi104 ceph-mon[33507]: 2.3 scrub starts 2024-09-11T07:37:36.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:36 smithi104 ceph-mon[33507]: 2.3 scrub ok 2024-09-11T07:37:36.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:36 smithi104 ceph-mon[33507]: 4.1 scrub starts 2024-09-11T07:37:36.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:36 smithi104 ceph-mon[33507]: 4.1 scrub ok 2024-09-11T07:37:36.620 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:37:36.620 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:37:36.626 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:36.629 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:37:36.629 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:37:36.648 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:37:36.648 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:37:36.650 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:36.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:36.651 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:36.663 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:37:36.663 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:37:36.678 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:37:36.679 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:37:36.694 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:37:36.694 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:37:36.710 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:37:36.710 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:37:36.726 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:37:36.726 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:37:36.751 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:36.774 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:36.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:36.775 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:36.876 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:36.900 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:36.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:36.901 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:37.001 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:37.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:37.026 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:37.026 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:37.127 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:37.150 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:37.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:37.151 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:37.164 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:37:37.252 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:37.276 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:37.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:37.278 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:37.379 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:37.404 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:37.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:37.405 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:37.423 DEBUG:teuthology.orchestra.run.smithi022:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_2 bench 90 write' 2024-09-11T07:37:37.426 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-11T07:37:37.426 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-11T07:37:37.472 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:hints = 1 2024-09-11T07:37:37.472 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-11T07:37:37.472 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Object prefix: benchmark_data_smithi022_133943 2024-09-11T07:37:37.473 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T07:37:37.473 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 0 0 0 0 0 0 - 0 2024-09-11T07:37:37.506 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:37.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:37 smithi104 ceph-mon[33507]: 2.a scrub starts 2024-09-11T07:37:37.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:37 smithi104 ceph-mon[33507]: 2.a scrub ok 2024-09-11T07:37:37.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:37 smithi104 ceph-mon[33507]: pgmap v498: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T07:37:37.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:37 smithi104 ceph-mon[33507]: 6.19 scrub starts 2024-09-11T07:37:37.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:37 smithi104 ceph-mon[33507]: 6.19 scrub ok 2024-09-11T07:37:37.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:37 smithi104 ceph-mon[33507]: 4.e scrub starts 2024-09-11T07:37:37.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:37 smithi104 ceph-mon[33507]: 4.e scrub ok 2024-09-11T07:37:37.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:37 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2572230187' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:37:37.545 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:37.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:37.547 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:37.590 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:37 smithi022 ceph-mon[29462]: 2.a scrub starts 2024-09-11T07:37:37.590 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:37 smithi022 ceph-mon[29462]: 2.a scrub ok 2024-09-11T07:37:37.590 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:37 smithi022 ceph-mon[29462]: pgmap v498: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T07:37:37.590 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:37 smithi022 ceph-mon[29462]: 6.19 scrub starts 2024-09-11T07:37:37.590 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:37 smithi022 ceph-mon[29462]: 6.19 scrub ok 2024-09-11T07:37:37.590 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:37 smithi022 ceph-mon[29462]: 4.e scrub starts 2024-09-11T07:37:37.590 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:37 smithi022 ceph-mon[29462]: 4.e scrub ok 2024-09-11T07:37:37.590 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:37 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2572230187' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:37:37.590 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:37 smithi022 ceph-mon[37330]: 2.a scrub starts 2024-09-11T07:37:37.591 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:37 smithi022 ceph-mon[37330]: 2.a scrub ok 2024-09-11T07:37:37.591 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:37 smithi022 ceph-mon[37330]: pgmap v498: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T07:37:37.591 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:37 smithi022 ceph-mon[37330]: 6.19 scrub starts 2024-09-11T07:37:37.591 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:37 smithi022 ceph-mon[37330]: 6.19 scrub ok 2024-09-11T07:37:37.591 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:37 smithi022 ceph-mon[37330]: 4.e scrub starts 2024-09-11T07:37:37.591 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:37 smithi022 ceph-mon[37330]: 4.e scrub ok 2024-09-11T07:37:37.591 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:37 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2572230187' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:37:37.647 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:37.648 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:37:37.696 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:37.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:37.697 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:37.798 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:37.831 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:37.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:37.832 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:37.933 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:37.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:37.987 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:37.987 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:38.088 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:38.127 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:38.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:38.128 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:38.229 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:38.272 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:38.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:38.274 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:38.375 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:38.404 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:38.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:38.406 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:38.473 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 1 16 3648 3632 226.987 227 0.00752537 0.00439041 2024-09-11T07:37:38.504 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:38 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:37:38.504 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:38 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:37:38.505 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:38 smithi022 ceph-mon[37330]: 5.0 scrub starts 2024-09-11T07:37:38.505 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:38 smithi022 ceph-mon[37330]: 5.0 scrub ok 2024-09-11T07:37:38.505 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:38 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:37:38.505 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:38 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:37:38.505 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:38 smithi022 ceph-mon[37330]: 4.1b scrub starts 2024-09-11T07:37:38.505 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:38 smithi022 ceph-mon[37330]: 4.1b scrub ok 2024-09-11T07:37:38.505 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:38 smithi022 ceph-mon[37330]: osdmap e254: 8 total, 8 up, 8 in 2024-09-11T07:37:38.505 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:37:38.505 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:38.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:38 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:37:38.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:38 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:37:38.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:38 smithi022 ceph-mon[29462]: 5.0 scrub starts 2024-09-11T07:37:38.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:38 smithi022 ceph-mon[29462]: 5.0 scrub ok 2024-09-11T07:37:38.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:38 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:37:38.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:38 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:37:38.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:38 smithi022 ceph-mon[29462]: 4.1b scrub starts 2024-09-11T07:37:38.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:38 smithi022 ceph-mon[29462]: 4.1b scrub ok 2024-09-11T07:37:38.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:38 smithi022 ceph-mon[29462]: osdmap e254: 8 total, 8 up, 8 in 2024-09-11T07:37:38.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:37:38.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:38.506 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:38.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:38 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:37:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:38 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:37:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:38 smithi104 ceph-mon[33507]: 5.0 scrub starts 2024-09-11T07:37:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:38 smithi104 ceph-mon[33507]: 5.0 scrub ok 2024-09-11T07:37:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:38 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:37:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:38 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:37:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:38 smithi104 ceph-mon[33507]: 4.1b scrub starts 2024-09-11T07:37:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:38 smithi104 ceph-mon[33507]: 4.1b scrub ok 2024-09-11T07:37:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:38 smithi104 ceph-mon[33507]: osdmap e254: 8 total, 8 up, 8 in 2024-09-11T07:37:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:37:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:38.537 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:38.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:38.538 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:38.638 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:38.681 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:38.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:38.682 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:38.783 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:38.813 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:38.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:38.814 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:38.915 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:38.954 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:38.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:38.957 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:39.058 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:37:39.060 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:39.064 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:37:39.094 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:39.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:39.095 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:39.196 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:39.239 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:39.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:39.240 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:39.340 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:39.378 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:39.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:39.379 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:39.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:39 smithi022 ceph-mon[37330]: pgmap v500: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:37:39.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:39 smithi022 ceph-mon[29462]: pgmap v500: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:37:39.473 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 2 15 6640 6625 207.016 187.062 0.00224209 0.00481958 2024-09-11T07:37:39.480 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:39.511 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:39.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:39.512 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:39.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:39 smithi104 ceph-mon[33507]: pgmap v500: 148 pgs: 148 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:37:39.613 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:39.641 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:37:39.645 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:39.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:39.646 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:39.747 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:39.812 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:39.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:39.813 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:39.914 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:39.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:39.956 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:39.957 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:40.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:37:39 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:37:40.057 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:40.086 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:40.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:40.088 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:40.188 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:40.221 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:40.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:40.222 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:40.322 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:40.360 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:40.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:40.361 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:40.462 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:40.473 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 3 16 9582 9566 199.276 183.812 0.00670012 0.00500906 2024-09-11T07:37:40.492 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:40.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:40.494 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:40.516 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:37:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:37:40] "GET /metrics HTTP/1.1" 200 223998 "" "Prometheus/2.43.0" 2024-09-11T07:37:40.516 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:40 smithi022 ceph-mon[29462]: 5.9 scrub starts 2024-09-11T07:37:40.516 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:40 smithi022 ceph-mon[29462]: 5.9 scrub ok 2024-09-11T07:37:40.516 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:40 smithi022 ceph-mon[29462]: from='client.25408 v1:172.21.15.22:0/1085927834' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:37:40.516 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:40 smithi022 ceph-mon[29462]: 6.6 scrub starts 2024-09-11T07:37:40.516 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:40 smithi022 ceph-mon[29462]: 6.6 scrub ok 2024-09-11T07:37:40.517 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:40 smithi022 ceph-mon[29462]: 5.18 scrub starts 2024-09-11T07:37:40.517 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:40 smithi022 ceph-mon[29462]: 5.18 scrub ok 2024-09-11T07:37:40.517 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:40 smithi022 ceph-mon[37330]: 5.9 scrub starts 2024-09-11T07:37:40.517 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:40 smithi022 ceph-mon[37330]: 5.9 scrub ok 2024-09-11T07:37:40.517 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:40 smithi022 ceph-mon[37330]: from='client.25408 v1:172.21.15.22:0/1085927834' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:37:40.517 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:40 smithi022 ceph-mon[37330]: 6.6 scrub starts 2024-09-11T07:37:40.517 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:40 smithi022 ceph-mon[37330]: 6.6 scrub ok 2024-09-11T07:37:40.518 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:40 smithi022 ceph-mon[37330]: 5.18 scrub starts 2024-09-11T07:37:40.518 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:40 smithi022 ceph-mon[37330]: 5.18 scrub ok 2024-09-11T07:37:40.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:40 smithi104 ceph-mon[33507]: 5.9 scrub starts 2024-09-11T07:37:40.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:40 smithi104 ceph-mon[33507]: 5.9 scrub ok 2024-09-11T07:37:40.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:40 smithi104 ceph-mon[33507]: from='client.25408 v1:172.21.15.22:0/1085927834' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:37:40.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:40 smithi104 ceph-mon[33507]: 6.6 scrub starts 2024-09-11T07:37:40.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:40 smithi104 ceph-mon[33507]: 6.6 scrub ok 2024-09-11T07:37:40.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:40 smithi104 ceph-mon[33507]: 5.18 scrub starts 2024-09-11T07:37:40.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:40 smithi104 ceph-mon[33507]: 5.18 scrub ok 2024-09-11T07:37:40.594 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:40.602 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":500,"stamp":"2024-09-11T07:37:38.177515+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1934,"num_read_kb":1791,"num_write":829,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":590946,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":934,"ondisk_log_size":934,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2463888,"kb_used_data":7988,"kb_used_omap":16,"kb_used_meta":2455791,"kb_avail":747497328,"statfs":{"total":767960285184,"available":765437263872,"internally_reserved":0,"allocated":8179712,"data_stored":5054892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16530,"internal_metadata":2514730862},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002025"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":255,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121702+0000","last_change":"2024-09-11T07:37:29.007467+0000","last_active":"2024-09-11T07:37:30.121702+0000","last_peered":"2024-09-11T07:37:30.121702+0000","last_clean":"2024-09-11T07:37:30.121702+0000","last_became_active":"2024-09-11T07:32:00.501534+0000","last_became_peered":"2024-09-11T07:32:00.501534+0000","last_unstale":"2024-09-11T07:37:30.121702+0000","last_undegraded":"2024-09-11T07:37:30.121702+0000","last_fullsized":"2024-09-11T07:37:30.121702+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:29.007411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:29.007411+0000","last_clean_scrub_stamp":"2024-09-11T07:37:29.007411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:43.805924+0000","scrub_duration":0.003998723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":271,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121588+0000","last_change":"2024-09-11T07:36:30.991280+0000","last_active":"2024-09-11T07:37:30.121588+0000","last_peered":"2024-09-11T07:37:30.121588+0000","last_clean":"2024-09-11T07:37:30.121588+0000","last_became_active":"2024-09-11T07:32:00.508572+0000","last_became_peered":"2024-09-11T07:32:00.508572+0000","last_unstale":"2024-09-11T07:37:30.121588+0000","last_undegraded":"2024-09-11T07:37:30.121588+0000","last_fullsized":"2024-09-11T07:37:30.121588+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:30.991202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:30.991202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:34.752711+0000","scrub_duration":0.0090469859999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"234'34","reported_seq":292,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121207+0000","last_change":"2024-09-11T07:36:28.772225+0000","last_active":"2024-09-11T07:37:30.121207+0000","last_peered":"2024-09-11T07:37:30.121207+0000","last_clean":"2024-09-11T07:37:30.121207+0000","last_became_active":"2024-09-11T07:26:33.544627+0000","last_became_peered":"2024-09-11T07:26:33.544627+0000","last_unstale":"2024-09-11T07:37:30.121207+0000","last_undegraded":"2024-09-11T07:37:30.121207+0000","last_fullsized":"2024-09-11T07:37:30.121207+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-09-11T07:36:28.772126+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:36:28.772126+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:51.960040+0000","scrub_duration":0.016007773999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":250,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868127+0000","last_change":"2024-09-11T07:35:41.639146+0000","last_active":"2024-09-11T07:37:29.868127+0000","last_peered":"2024-09-11T07:37:29.868127+0000","last_clean":"2024-09-11T07:37:29.868127+0000","last_became_active":"2024-09-11T07:26:35.556296+0000","last_became_peered":"2024-09-11T07:26:35.556296+0000","last_unstale":"2024-09-11T07:37:29.868127+0000","last_undegraded":"2024-09-11T07:37:29.868127+0000","last_fullsized":"2024-09-11T07:37:29.868127+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:41.639074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:35:41.639074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058749240000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":254,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121608+0000","last_change":"2024-09-11T07:36:33.755073+0000","last_active":"2024-09-11T07:37:30.121608+0000","last_peered":"2024-09-11T07:37:30.121608+0000","last_clean":"2024-09-11T07:37:30.121608+0000","last_became_active":"2024-09-11T07:26:31.420178+0000","last_became_peered":"2024-09-11T07:26:31.420178+0000","last_unstale":"2024-09-11T07:37:30.121608+0000","last_undegraded":"2024-09-11T07:37:30.121608+0000","last_fullsized":"2024-09-11T07:37:30.121608+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:33.754983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:33.754983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:43.787572+0000","scrub_duration":0.0034784899999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"234'19","reported_seq":281,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868149+0000","last_change":"2024-09-11T07:36:42.709007+0000","last_active":"2024-09-11T07:37:29.868149+0000","last_peered":"2024-09-11T07:37:29.868149+0000","last_clean":"2024-09-11T07:37:29.868149+0000","last_became_active":"2024-09-11T07:26:33.540732+0000","last_became_peered":"2024-09-11T07:26:33.540732+0000","last_unstale":"2024-09-11T07:37:29.868149+0000","last_undegraded":"2024-09-11T07:37:29.868149+0000","last_fullsized":"2024-09-11T07:37:29.868149+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:36:42.708941+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:36:42.708941+0000","last_clean_scrub_stamp":"2024-09-11T07:36:42.708941+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:47.828260+0000","scrub_duration":0.0088444879999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":259,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.867785+0000","last_change":"2024-09-11T07:35:29.741680+0000","last_active":"2024-09-11T07:37:29.867785+0000","last_peered":"2024-09-11T07:37:29.867785+0000","last_clean":"2024-09-11T07:37:29.867785+0000","last_became_active":"2024-09-11T07:32:00.513650+0000","last_became_peered":"2024-09-11T07:32:00.513650+0000","last_unstale":"2024-09-11T07:37:29.867785+0000","last_undegraded":"2024-09-11T07:37:29.867785+0000","last_fullsized":"2024-09-11T07:37:29.867785+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:29.741578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:35:29.741578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030495850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":248,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868313+0000","last_change":"2024-09-11T07:36:48.684925+0000","last_active":"2024-09-11T07:37:29.868313+0000","last_peered":"2024-09-11T07:37:29.868313+0000","last_clean":"2024-09-11T07:37:29.868313+0000","last_became_active":"2024-09-11T07:26:37.566910+0000","last_became_peered":"2024-09-11T07:26:37.566910+0000","last_unstale":"2024-09-11T07:37:29.868313+0000","last_undegraded":"2024-09-11T07:37:29.868313+0000","last_fullsized":"2024-09-11T07:37:29.868313+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:48.684838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:48.684838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:55.308938+0000","scrub_duration":0.0058146650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":249,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122730+0000","last_change":"2024-09-11T07:35:40.292130+0000","last_active":"2024-09-11T07:37:30.122730+0000","last_peered":"2024-09-11T07:37:30.122730+0000","last_clean":"2024-09-11T07:37:30.122730+0000","last_became_active":"2024-09-11T07:32:00.516662+0000","last_became_peered":"2024-09-11T07:32:00.516662+0000","last_unstale":"2024-09-11T07:37:30.122730+0000","last_undegraded":"2024-09-11T07:37:30.122730+0000","last_fullsized":"2024-09-11T07:37:30.122730+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:40.292055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:35:40.292055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00373822,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"234'9","reported_seq":267,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868467+0000","last_change":"2024-09-11T07:35:25.599700+0000","last_active":"2024-09-11T07:37:29.868467+0000","last_peered":"2024-09-11T07:37:29.868467+0000","last_clean":"2024-09-11T07:37:29.868467+0000","last_became_active":"2024-09-11T07:26:33.540351+0000","last_became_peered":"2024-09-11T07:26:33.540351+0000","last_unstale":"2024-09-11T07:37:29.868467+0000","last_undegraded":"2024-09-11T07:37:29.868467+0000","last_fullsized":"2024-09-11T07:37:29.868467+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'7","last_scrub_stamp":"2024-09-11T07:35:25.599616+0000","last_deep_scrub":"82'7","last_deep_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_clean_scrub_stamp":"2024-09-11T07:35:25.599616+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0081033730000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":225,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121169+0000","last_change":"2024-09-11T07:35:50.332326+0000","last_active":"2024-09-11T07:37:30.121169+0000","last_peered":"2024-09-11T07:37:30.121169+0000","last_clean":"2024-09-11T07:37:30.121169+0000","last_became_active":"2024-09-11T07:32:00.513206+0000","last_became_peered":"2024-09-11T07:32:00.513206+0000","last_unstale":"2024-09-11T07:37:30.121169+0000","last_undegraded":"2024-09-11T07:37:30.121169+0000","last_fullsized":"2024-09-11T07:37:30.121169+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:50.332244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.973323+0000","last_clean_scrub_stamp":"2024-09-11T07:35:50.332244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034655860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":238,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122301+0000","last_change":"2024-09-11T07:35:45.150059+0000","last_active":"2024-09-11T07:37:30.122301+0000","last_peered":"2024-09-11T07:37:30.122301+0000","last_clean":"2024-09-11T07:37:30.122301+0000","last_became_active":"2024-09-11T07:26:37.561758+0000","last_became_peered":"2024-09-11T07:26:37.561758+0000","last_unstale":"2024-09-11T07:37:30.122301+0000","last_undegraded":"2024-09-11T07:37:30.122301+0000","last_fullsized":"2024-09-11T07:37:30.122301+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:45.149916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:45.149916+0000","last_clean_scrub_stamp":"2024-09-11T07:35:45.149916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035215020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":214,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121648+0000","last_change":"2024-09-11T07:35:39.094710+0000","last_active":"2024-09-11T07:37:30.121648+0000","last_peered":"2024-09-11T07:37:30.121648+0000","last_clean":"2024-09-11T07:37:30.121648+0000","last_became_active":"2024-09-11T07:32:00.505709+0000","last_became_peered":"2024-09-11T07:32:00.505709+0000","last_unstale":"2024-09-11T07:37:30.121648+0000","last_undegraded":"2024-09-11T07:37:30.121648+0000","last_fullsized":"2024-09-11T07:37:30.121648+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:39.094661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_clean_scrub_stamp":"2024-09-11T07:35:39.094661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0074675389999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"234'19","reported_seq":281,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868846+0000","last_change":"2024-09-11T07:36:27.746378+0000","last_active":"2024-09-11T07:37:29.868846+0000","last_peered":"2024-09-11T07:37:29.868846+0000","last_clean":"2024-09-11T07:37:29.868846+0000","last_became_active":"2024-09-11T07:26:33.544386+0000","last_became_peered":"2024-09-11T07:26:33.544386+0000","last_unstale":"2024-09-11T07:37:29.868846+0000","last_undegraded":"2024-09-11T07:37:29.868846+0000","last_fullsized":"2024-09-11T07:37:29.868846+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'13","last_scrub_stamp":"2024-09-11T07:36:27.746295+0000","last_deep_scrub":"136'13","last_deep_scrub_stamp":"2024-09-11T07:31:08.042198+0000","last_clean_scrub_stamp":"2024-09-11T07:36:27.746295+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:54.572101+0000","scrub_duration":0.0089695690000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":260,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121281+0000","last_change":"2024-09-11T07:35:34.150965+0000","last_active":"2024-09-11T07:37:30.121281+0000","last_peered":"2024-09-11T07:37:30.121281+0000","last_clean":"2024-09-11T07:37:30.121281+0000","last_became_active":"2024-09-11T07:32:00.502337+0000","last_became_peered":"2024-09-11T07:32:00.502337+0000","last_unstale":"2024-09-11T07:37:30.121281+0000","last_undegraded":"2024-09-11T07:37:30.121281+0000","last_fullsized":"2024-09-11T07:37:30.121281+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:34.150289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:35:34.150289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023057149999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":265,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.120759+0000","last_change":"2024-09-11T07:36:31.977319+0000","last_active":"2024-09-11T07:37:30.120759+0000","last_peered":"2024-09-11T07:37:30.120759+0000","last_clean":"2024-09-11T07:37:30.120759+0000","last_became_active":"2024-09-11T07:32:00.530335+0000","last_became_peered":"2024-09-11T07:32:00.530335+0000","last_unstale":"2024-09-11T07:37:30.120759+0000","last_undegraded":"2024-09-11T07:37:30.120759+0000","last_fullsized":"2024-09-11T07:37:30.120759+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:31.977240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:31.977240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:42.807853+0000","scrub_duration":0.0074114259999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.867034+0000","last_change":"2024-09-11T07:37:26.029984+0000","last_active":"2024-09-11T07:37:29.867034+0000","last_peered":"2024-09-11T07:37:29.867034+0000","last_clean":"2024-09-11T07:37:29.867034+0000","last_became_active":"2024-09-11T07:37:26.029625+0000","last_became_peered":"2024-09-11T07:37:26.029625+0000","last_unstale":"2024-09-11T07:37:29.867034+0000","last_undegraded":"2024-09-11T07:37:29.867034+0000","last_fullsized":"2024-09-11T07:37:29.867034+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:33.495400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"234'31","reported_seq":357,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.869199+0000","last_change":"2024-09-11T07:36:39.737025+0000","last_active":"2024-09-11T07:37:29.869199+0000","last_peered":"2024-09-11T07:37:29.869199+0000","last_clean":"2024-09-11T07:37:29.869199+0000","last_became_active":"2024-09-11T07:29:51.482755+0000","last_became_peered":"2024-09-11T07:29:51.482755+0000","last_unstale":"2024-09-11T07:37:29.869199+0000","last_undegraded":"2024-09-11T07:37:29.869199+0000","last_fullsized":"2024-09-11T07:37:29.869199+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'21","last_scrub_stamp":"2024-09-11T07:36:39.736944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:36:39.736944+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:56.164639+0000","scrub_duration":0.0034170279999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":254,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122183+0000","last_change":"2024-09-11T07:36:51.418345+0000","last_active":"2024-09-11T07:37:30.122183+0000","last_peered":"2024-09-11T07:37:30.122183+0000","last_clean":"2024-09-11T07:37:30.122183+0000","last_became_active":"2024-09-11T07:32:00.507191+0000","last_became_peered":"2024-09-11T07:32:00.507191+0000","last_unstale":"2024-09-11T07:37:30.122183+0000","last_undegraded":"2024-09-11T07:37:30.122183+0000","last_fullsized":"2024-09-11T07:37:30.122183+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:51.418258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:51.418258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:58.646693+0000","scrub_duration":0.001500181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":242,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868026+0000","last_change":"2024-09-11T07:37:17.980251+0000","last_active":"2024-09-11T07:37:29.868026+0000","last_peered":"2024-09-11T07:37:29.868026+0000","last_clean":"2024-09-11T07:37:29.868026+0000","last_became_active":"2024-09-11T07:26:35.557989+0000","last_became_peered":"2024-09-11T07:26:35.557989+0000","last_unstale":"2024-09-11T07:37:29.868026+0000","last_undegraded":"2024-09-11T07:37:29.868026+0000","last_fullsized":"2024-09-11T07:37:29.868026+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:00.348684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:00.348684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036577580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":258,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.867978+0000","last_change":"2024-09-11T07:37:21.385329+0000","last_active":"2024-09-11T07:37:29.867978+0000","last_peered":"2024-09-11T07:37:29.867978+0000","last_clean":"2024-09-11T07:37:29.867978+0000","last_became_active":"2024-09-11T07:26:37.564554+0000","last_became_peered":"2024-09-11T07:26:37.564554+0000","last_unstale":"2024-09-11T07:37:29.867978+0000","last_undegraded":"2024-09-11T07:37:29.867978+0000","last_fullsized":"2024-09-11T07:37:29.867978+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:21.385271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:21.385271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:27.423807+0000","scrub_duration":0.001463322,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.120942+0000","last_change":"2024-09-11T07:37:26.029943+0000","last_active":"2024-09-11T07:37:30.120942+0000","last_peered":"2024-09-11T07:37:30.120942+0000","last_clean":"2024-09-11T07:37:30.120942+0000","last_became_active":"2024-09-11T07:37:26.029574+0000","last_became_peered":"2024-09-11T07:37:26.029574+0000","last_unstale":"2024-09-11T07:37:30.120942+0000","last_undegraded":"2024-09-11T07:37:30.120942+0000","last_fullsized":"2024-09-11T07:37:30.120942+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:25.412197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"234'25","reported_seq":287,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.869163+0000","last_change":"2024-09-11T07:36:43.704258+0000","last_active":"2024-09-11T07:37:29.869163+0000","last_peered":"2024-09-11T07:37:29.869163+0000","last_clean":"2024-09-11T07:37:29.869163+0000","last_became_active":"2024-09-11T07:26:33.535411+0000","last_became_peered":"2024-09-11T07:26:33.535411+0000","last_unstale":"2024-09-11T07:37:29.869163+0000","last_undegraded":"2024-09-11T07:37:29.869163+0000","last_fullsized":"2024-09-11T07:37:29.869163+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'25","last_scrub_stamp":"2024-09-11T07:36:43.704170+0000","last_deep_scrub":"112'15","last_deep_scrub_stamp":"2024-09-11T07:31:10.005896+0000","last_clean_scrub_stamp":"2024-09-11T07:36:43.704170+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:48.004221+0000","scrub_duration":0.0048642039999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":263,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.869038+0000","last_change":"2024-09-11T07:35:39.792438+0000","last_active":"2024-09-11T07:37:29.869038+0000","last_peered":"2024-09-11T07:37:29.869038+0000","last_clean":"2024-09-11T07:37:29.869038+0000","last_became_active":"2024-09-11T07:32:00.506615+0000","last_became_peered":"2024-09-11T07:32:00.506615+0000","last_unstale":"2024-09-11T07:37:29.869038+0000","last_undegraded":"2024-09-11T07:37:29.869038+0000","last_fullsized":"2024-09-11T07:37:29.869038+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:39.792365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:35:39.792365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010496709,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"151'10","reported_seq":409,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.867666+0000","last_change":"2024-09-11T07:36:36.141087+0000","last_active":"2024-09-11T07:37:29.867666+0000","last_peered":"2024-09-11T07:37:29.867666+0000","last_clean":"2024-09-11T07:37:29.867666+0000","last_became_active":"2024-09-11T07:32:00.507447+0000","last_became_peered":"2024-09-11T07:32:00.507447+0000","last_unstale":"2024-09-11T07:37:29.867666+0000","last_undegraded":"2024-09-11T07:37:29.867666+0000","last_fullsized":"2024-09-11T07:37:29.867666+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:36:36.140998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:36:36.140998+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:42.959982+0000","scrub_duration":0.0071827749999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":248,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868763+0000","last_change":"2024-09-11T07:37:22.546982+0000","last_active":"2024-09-11T07:37:29.868763+0000","last_peered":"2024-09-11T07:37:29.868763+0000","last_clean":"2024-09-11T07:37:29.868763+0000","last_became_active":"2024-09-11T07:26:37.566919+0000","last_became_peered":"2024-09-11T07:26:37.566919+0000","last_unstale":"2024-09-11T07:37:29.868763+0000","last_undegraded":"2024-09-11T07:37:29.868763+0000","last_fullsized":"2024-09-11T07:37:29.868763+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:22.546931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:22.546931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:42.991401+0000","scrub_duration":0.001345527,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121302+0000","last_change":"2024-09-11T07:37:25.851826+0000","last_active":"2024-09-11T07:37:30.121302+0000","last_peered":"2024-09-11T07:37:30.121302+0000","last_clean":"2024-09-11T07:37:30.121302+0000","last_became_active":"2024-09-11T07:37:25.851536+0000","last_became_peered":"2024-09-11T07:37:25.851536+0000","last_unstale":"2024-09-11T07:37:30.121302+0000","last_undegraded":"2024-09-11T07:37:30.121302+0000","last_fullsized":"2024-09-11T07:37:30.121302+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:42.549849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"234'34","reported_seq":301,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.869150+0000","last_change":"2024-09-11T07:37:16.938565+0000","last_active":"2024-09-11T07:37:29.869150+0000","last_peered":"2024-09-11T07:37:29.869150+0000","last_clean":"2024-09-11T07:37:29.869150+0000","last_became_active":"2024-09-11T07:26:33.549733+0000","last_became_peered":"2024-09-11T07:26:33.549733+0000","last_unstale":"2024-09-11T07:37:29.869150+0000","last_undegraded":"2024-09-11T07:37:29.869150+0000","last_fullsized":"2024-09-11T07:37:29.869150+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'34","last_scrub_stamp":"2024-09-11T07:37:16.938496+0000","last_deep_scrub":"83'22","last_deep_scrub_stamp":"2024-09-11T07:34:04.375457+0000","last_clean_scrub_stamp":"2024-09-11T07:37:16.938496+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:44.726347+0000","scrub_duration":0.0028468529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":273,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.867982+0000","last_change":"2024-09-11T07:35:40.786431+0000","last_active":"2024-09-11T07:37:29.867982+0000","last_peered":"2024-09-11T07:37:29.867982+0000","last_clean":"2024-09-11T07:37:29.867982+0000","last_became_active":"2024-09-11T07:32:00.513472+0000","last_became_peered":"2024-09-11T07:32:00.513472+0000","last_unstale":"2024-09-11T07:37:29.867982+0000","last_undegraded":"2024-09-11T07:37:29.867982+0000","last_fullsized":"2024-09-11T07:37:29.867982+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:40.786358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:35:40.786358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014952305000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":393,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.657658+0000","last_change":"2024-09-11T07:37:30.657587+0000","last_active":"2024-09-11T07:37:30.657658+0000","last_peered":"2024-09-11T07:37:30.657658+0000","last_clean":"2024-09-11T07:37:30.657658+0000","last_became_active":"2024-09-11T07:26:35.560406+0000","last_became_peered":"2024-09-11T07:26:35.560406+0000","last_unstale":"2024-09-11T07:37:30.657658+0000","last_undegraded":"2024-09-11T07:37:30.657658+0000","last_fullsized":"2024-09-11T07:37:30.657658+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:37:30.657477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:30.657477+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:57.486387+0000","scrub_duration":0.001943012,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":248,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121868+0000","last_change":"2024-09-11T07:37:20.369255+0000","last_active":"2024-09-11T07:37:30.121868+0000","last_peered":"2024-09-11T07:37:30.121868+0000","last_clean":"2024-09-11T07:37:30.121868+0000","last_became_active":"2024-09-11T07:26:37.568005+0000","last_became_peered":"2024-09-11T07:26:37.568005+0000","last_unstale":"2024-09-11T07:37:30.121868+0000","last_undegraded":"2024-09-11T07:37:30.121868+0000","last_fullsized":"2024-09-11T07:37:30.121868+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:20.369199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:31.200685+0000","last_clean_scrub_stamp":"2024-09-11T07:37:20.369199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:32.604085+0000","scrub_duration":0.001371688,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121692+0000","last_change":"2024-09-11T07:37:25.855865+0000","last_active":"2024-09-11T07:37:30.121692+0000","last_peered":"2024-09-11T07:37:30.121692+0000","last_clean":"2024-09-11T07:37:30.121692+0000","last_became_active":"2024-09-11T07:37:25.855577+0000","last_became_peered":"2024-09-11T07:37:25.855577+0000","last_unstale":"2024-09-11T07:37:30.121692+0000","last_undegraded":"2024-09-11T07:37:30.121692+0000","last_fullsized":"2024-09-11T07:37:30.121692+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:48.134227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"234'24","reported_seq":288,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.869176+0000","last_change":"2024-09-11T07:35:28.635340+0000","last_active":"2024-09-11T07:37:29.869176+0000","last_peered":"2024-09-11T07:37:29.869176+0000","last_clean":"2024-09-11T07:37:29.869176+0000","last_became_active":"2024-09-11T07:26:33.540777+0000","last_became_peered":"2024-09-11T07:26:33.540777+0000","last_unstale":"2024-09-11T07:37:29.869176+0000","last_undegraded":"2024-09-11T07:37:29.869176+0000","last_fullsized":"2024-09-11T07:37:29.869176+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'16","last_scrub_stamp":"2024-09-11T07:35:28.635269+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-09-11T07:29:38.112111+0000","last_clean_scrub_stamp":"2024-09-11T07:35:28.635269+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013024327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":248,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121162+0000","last_change":"2024-09-11T07:36:37.445237+0000","last_active":"2024-09-11T07:37:30.121162+0000","last_peered":"2024-09-11T07:37:30.121162+0000","last_clean":"2024-09-11T07:37:30.121162+0000","last_became_active":"2024-09-11T07:32:00.518538+0000","last_became_peered":"2024-09-11T07:32:00.518538+0000","last_unstale":"2024-09-11T07:37:30.121162+0000","last_undegraded":"2024-09-11T07:37:30.121162+0000","last_fullsized":"2024-09-11T07:37:30.121162+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:37.444760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:37.444760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:47.891356+0000","scrub_duration":0.012636722,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":250,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122473+0000","last_change":"2024-09-11T07:35:29.166617+0000","last_active":"2024-09-11T07:37:30.122473+0000","last_peered":"2024-09-11T07:37:30.122473+0000","last_clean":"2024-09-11T07:37:30.122473+0000","last_became_active":"2024-09-11T07:26:35.569081+0000","last_became_peered":"2024-09-11T07:26:35.569081+0000","last_unstale":"2024-09-11T07:37:30.122473+0000","last_undegraded":"2024-09-11T07:37:30.122473+0000","last_fullsized":"2024-09-11T07:37:30.122473+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:29.166488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:35:29.166488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059891079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":248,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121791+0000","last_change":"2024-09-11T07:36:50.920711+0000","last_active":"2024-09-11T07:37:30.121791+0000","last_peered":"2024-09-11T07:37:30.121791+0000","last_clean":"2024-09-11T07:37:30.121791+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:37:30.121791+0000","last_undegraded":"2024-09-11T07:37:30.121791+0000","last_fullsized":"2024-09-11T07:37:30.121791+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:50.920625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:46.853387+0000","last_clean_scrub_stamp":"2024-09-11T07:36:50.920625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:19.177270+0000","scrub_duration":0.012404324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121714+0000","last_change":"2024-09-11T07:37:25.855661+0000","last_active":"2024-09-11T07:37:30.121714+0000","last_peered":"2024-09-11T07:37:30.121714+0000","last_clean":"2024-09-11T07:37:30.121714+0000","last_became_active":"2024-09-11T07:37:25.854796+0000","last_became_peered":"2024-09-11T07:37:25.854796+0000","last_unstale":"2024-09-11T07:37:30.121714+0000","last_undegraded":"2024-09-11T07:37:30.121714+0000","last_fullsized":"2024-09-11T07:37:30.121714+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:31.723924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"234'28","reported_seq":284,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868284+0000","last_change":"2024-09-11T07:35:24.587712+0000","last_active":"2024-09-11T07:37:29.868284+0000","last_peered":"2024-09-11T07:37:29.868284+0000","last_clean":"2024-09-11T07:37:29.868284+0000","last_became_active":"2024-09-11T07:26:33.537850+0000","last_became_peered":"2024-09-11T07:26:33.537850+0000","last_unstale":"2024-09-11T07:37:29.868284+0000","last_undegraded":"2024-09-11T07:37:29.868284+0000","last_fullsized":"2024-09-11T07:37:29.868284+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-09-11T07:35:24.587622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:35:24.587622+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012975469999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":100,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868763+0000","last_change":"2024-09-11T07:37:20.527569+0000","last_active":"2024-09-11T07:37:29.868763+0000","last_peered":"2024-09-11T07:37:29.868763+0000","last_clean":"2024-09-11T07:37:29.868763+0000","last_became_active":"2024-09-11T07:36:05.226733+0000","last_became_peered":"2024-09-11T07:36:05.226733+0000","last_unstale":"2024-09-11T07:37:29.868763+0000","last_undegraded":"2024-09-11T07:37:29.868763+0000","last_fullsized":"2024-09-11T07:37:29.868763+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:20.527511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:20.527511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:31.232016+0000","scrub_duration":0.0011632039999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"84'9","reported_seq":382,"reported_epoch":253,"state":"active+clean","last_fresh":"2024-09-11T07:37:33.239446+0000","last_change":"2024-09-11T07:35:39.833322+0000","last_active":"2024-09-11T07:37:33.239446+0000","last_peered":"2024-09-11T07:37:33.239446+0000","last_clean":"2024-09-11T07:37:33.239446+0000","last_became_active":"2024-09-11T07:26:35.562997+0000","last_became_peered":"2024-09-11T07:26:35.562997+0000","last_unstale":"2024-09-11T07:37:33.239446+0000","last_undegraded":"2024-09-11T07:37:33.239446+0000","last_fullsized":"2024-09-11T07:37:33.239446+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-11T07:35:39.832547+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-11T07:35:39.832547+0000","last_clean_scrub_stamp":"2024-09-11T07:35:39.832547+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015158308000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":222,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121929+0000","last_change":"2024-09-11T07:36:32.419622+0000","last_active":"2024-09-11T07:37:30.121929+0000","last_peered":"2024-09-11T07:37:30.121929+0000","last_clean":"2024-09-11T07:37:30.121929+0000","last_became_active":"2024-09-11T07:32:00.511005+0000","last_became_peered":"2024-09-11T07:32:00.511005+0000","last_unstale":"2024-09-11T07:37:30.121929+0000","last_undegraded":"2024-09-11T07:37:30.121929+0000","last_fullsized":"2024-09-11T07:37:30.121929+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:32.419555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:32.419555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:35.199598+0000","scrub_duration":0.0046427200000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":13,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122628+0000","last_change":"2024-09-11T07:37:25.854170+0000","last_active":"2024-09-11T07:37:30.122628+0000","last_peered":"2024-09-11T07:37:30.122628+0000","last_clean":"2024-09-11T07:37:30.122628+0000","last_became_active":"2024-09-11T07:37:25.853857+0000","last_became_peered":"2024-09-11T07:37:25.853857+0000","last_unstale":"2024-09-11T07:37:30.122628+0000","last_undegraded":"2024-09-11T07:37:30.122628+0000","last_fullsized":"2024-09-11T07:37:30.122628+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:29.440611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"234'38","reported_seq":317,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.120827+0000","last_change":"2024-09-11T07:35:52.942431+0000","last_active":"2024-09-11T07:37:30.120827+0000","last_peered":"2024-09-11T07:37:30.120827+0000","last_clean":"2024-09-11T07:37:30.120827+0000","last_became_active":"2024-09-11T07:32:00.520670+0000","last_became_peered":"2024-09-11T07:32:00.520670+0000","last_unstale":"2024-09-11T07:37:30.120827+0000","last_undegraded":"2024-09-11T07:37:30.120827+0000","last_fullsized":"2024-09-11T07:37:30.120827+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'26","last_scrub_stamp":"2024-09-11T07:35:52.942347+0000","last_deep_scrub":"87'26","last_deep_scrub_stamp":"2024-09-11T07:35:52.942347+0000","last_clean_scrub_stamp":"2024-09-11T07:35:52.942347+0000","objects_scrubbed":8,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010845960999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":263,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:33.081918+0000","last_change":"2024-09-11T07:37:33.081869+0000","last_active":"2024-09-11T07:37:33.081918+0000","last_peered":"2024-09-11T07:37:33.081918+0000","last_clean":"2024-09-11T07:37:33.081918+0000","last_became_active":"2024-09-11T07:26:31.421500+0000","last_became_peered":"2024-09-11T07:26:31.421500+0000","last_unstale":"2024-09-11T07:37:33.081918+0000","last_undegraded":"2024-09-11T07:37:33.081918+0000","last_fullsized":"2024-09-11T07:37:33.081918+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:33.081815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:33.081815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:54.240779+0000","scrub_duration":0.001426379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":260,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868207+0000","last_change":"2024-09-11T07:36:30.786071+0000","last_active":"2024-09-11T07:37:29.868207+0000","last_peered":"2024-09-11T07:37:29.868207+0000","last_clean":"2024-09-11T07:37:29.868207+0000","last_became_active":"2024-09-11T07:32:00.499747+0000","last_became_peered":"2024-09-11T07:32:00.499747+0000","last_unstale":"2024-09-11T07:37:29.868207+0000","last_undegraded":"2024-09-11T07:37:29.868207+0000","last_fullsized":"2024-09-11T07:37:29.868207+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:30.785991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:27.968579+0000","last_clean_scrub_stamp":"2024-09-11T07:36:30.785991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:53.321816+0000","scrub_duration":0.011893737999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":248,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868615+0000","last_change":"2024-09-11T07:37:28.623070+0000","last_active":"2024-09-11T07:37:29.868615+0000","last_peered":"2024-09-11T07:37:29.868615+0000","last_clean":"2024-09-11T07:37:29.868615+0000","last_became_active":"2024-09-11T07:26:37.564616+0000","last_became_peered":"2024-09-11T07:26:37.564616+0000","last_unstale":"2024-09-11T07:37:29.868615+0000","last_undegraded":"2024-09-11T07:37:29.868615+0000","last_fullsized":"2024-09-11T07:37:29.868615+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:28.622998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:29.978798+0000","last_clean_scrub_stamp":"2024-09-11T07:37:28.622998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:42.675263+0000","scrub_duration":0.001843798,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122625+0000","last_change":"2024-09-11T07:37:25.854301+0000","last_active":"2024-09-11T07:37:30.122625+0000","last_peered":"2024-09-11T07:37:30.122625+0000","last_clean":"2024-09-11T07:37:30.122625+0000","last_became_active":"2024-09-11T07:37:25.853844+0000","last_became_peered":"2024-09-11T07:37:25.853844+0000","last_unstale":"2024-09-11T07:37:30.122625+0000","last_undegraded":"2024-09-11T07:37:30.122625+0000","last_fullsized":"2024-09-11T07:37:30.122625+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:45.631635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"236'32","reported_seq":323,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121385+0000","last_change":"2024-09-11T07:36:50.024003+0000","last_active":"2024-09-11T07:37:30.121385+0000","last_peered":"2024-09-11T07:37:30.121385+0000","last_clean":"2024-09-11T07:37:30.121385+0000","last_became_active":"2024-09-11T07:32:00.506270+0000","last_became_peered":"2024-09-11T07:32:00.506270+0000","last_unstale":"2024-09-11T07:37:30.121385+0000","last_undegraded":"2024-09-11T07:37:30.121385+0000","last_fullsized":"2024-09-11T07:37:30.121385+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'32","last_scrub_stamp":"2024-09-11T07:36:50.023909+0000","last_deep_scrub":"236'32","last_deep_scrub_stamp":"2024-09-11T07:36:50.023909+0000","last_clean_scrub_stamp":"2024-09-11T07:36:50.023909+0000","objects_scrubbed":8,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:00.989529+0000","scrub_duration":0.030217668999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":222,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122939+0000","last_change":"2024-09-11T07:36:48.434408+0000","last_active":"2024-09-11T07:37:30.122939+0000","last_peered":"2024-09-11T07:37:30.122939+0000","last_clean":"2024-09-11T07:37:30.122939+0000","last_became_active":"2024-09-11T07:32:00.510631+0000","last_became_peered":"2024-09-11T07:32:00.510631+0000","last_unstale":"2024-09-11T07:37:30.122939+0000","last_undegraded":"2024-09-11T07:37:30.122939+0000","last_fullsized":"2024-09-11T07:37:30.122939+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:48.434320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:49.144041+0000","last_clean_scrub_stamp":"2024-09-11T07:36:48.434320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:02.587912+0000","scrub_duration":0.0066312669999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":51,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121004+0000","last_change":"2024-09-11T07:36:55.163926+0000","last_active":"2024-09-11T07:37:30.121004+0000","last_peered":"2024-09-11T07:37:30.121004+0000","last_clean":"2024-09-11T07:37:30.121004+0000","last_became_active":"2024-09-11T07:36:05.225840+0000","last_became_peered":"2024-09-11T07:36:05.225840+0000","last_unstale":"2024-09-11T07:37:30.121004+0000","last_undegraded":"2024-09-11T07:37:30.121004+0000","last_fullsized":"2024-09-11T07:37:30.121004+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:55.163476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:36:55.163476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:57.584983+0000","scrub_duration":0.0082978570000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":247,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.867701+0000","last_change":"2024-09-11T07:36:54.828435+0000","last_active":"2024-09-11T07:37:29.867701+0000","last_peered":"2024-09-11T07:37:29.867701+0000","last_clean":"2024-09-11T07:37:29.867701+0000","last_became_active":"2024-09-11T07:26:37.861279+0000","last_became_peered":"2024-09-11T07:26:37.861279+0000","last_unstale":"2024-09-11T07:37:29.867701+0000","last_undegraded":"2024-09-11T07:37:29.867701+0000","last_fullsized":"2024-09-11T07:37:29.867701+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:54.827454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:39.723598+0000","last_clean_scrub_stamp":"2024-09-11T07:36:54.827454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:06.664883+0000","scrub_duration":0.016642015,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121957+0000","last_change":"2024-09-11T07:37:25.856504+0000","last_active":"2024-09-11T07:37:30.121957+0000","last_peered":"2024-09-11T07:37:30.121957+0000","last_clean":"2024-09-11T07:37:30.121957+0000","last_became_active":"2024-09-11T07:37:25.856214+0000","last_became_peered":"2024-09-11T07:37:25.856214+0000","last_unstale":"2024-09-11T07:37:30.121957+0000","last_undegraded":"2024-09-11T07:37:30.121957+0000","last_fullsized":"2024-09-11T07:37:30.121957+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:48.819382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"234'20","reported_seq":284,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868907+0000","last_change":"2024-09-11T07:36:37.911537+0000","last_active":"2024-09-11T07:37:29.868907+0000","last_peered":"2024-09-11T07:37:29.868907+0000","last_clean":"2024-09-11T07:37:29.868907+0000","last_became_active":"2024-09-11T07:26:33.543063+0000","last_became_peered":"2024-09-11T07:26:33.543063+0000","last_unstale":"2024-09-11T07:37:29.868907+0000","last_undegraded":"2024-09-11T07:37:29.868907+0000","last_fullsized":"2024-09-11T07:37:29.868907+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:36:37.911329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:36:37.911329+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:44.764318+0000","scrub_duration":0.01198222,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":280,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868156+0000","last_change":"2024-09-11T07:36:38.909472+0000","last_active":"2024-09-11T07:37:29.868156+0000","last_peered":"2024-09-11T07:37:29.868156+0000","last_clean":"2024-09-11T07:37:29.868156+0000","last_became_active":"2024-09-11T07:32:00.522456+0000","last_became_peered":"2024-09-11T07:32:00.522456+0000","last_unstale":"2024-09-11T07:37:29.868156+0000","last_undegraded":"2024-09-11T07:37:29.868156+0000","last_fullsized":"2024-09-11T07:37:29.868156+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:36:38.909348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:38.909348+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:03.548160+0000","scrub_duration":0.0031018040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":401,"reported_epoch":253,"state":"active+clean","last_fresh":"2024-09-11T07:37:33.897258+0000","last_change":"2024-09-11T07:37:26.141508+0000","last_active":"2024-09-11T07:37:33.897258+0000","last_peered":"2024-09-11T07:37:33.897258+0000","last_clean":"2024-09-11T07:37:33.897258+0000","last_became_active":"2024-09-11T07:26:35.560422+0000","last_became_peered":"2024-09-11T07:26:35.560422+0000","last_unstale":"2024-09-11T07:37:33.897258+0000","last_undegraded":"2024-09-11T07:37:33.897258+0000","last_fullsized":"2024-09-11T07:37:33.897258+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:37:26.141447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.141447+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:28.626906+0000","scrub_duration":0.0015408170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":258,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.120626+0000","last_change":"2024-09-11T07:36:40.027067+0000","last_active":"2024-09-11T07:37:30.120626+0000","last_peered":"2024-09-11T07:37:30.120626+0000","last_clean":"2024-09-11T07:37:30.120626+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:37:30.120626+0000","last_undegraded":"2024-09-11T07:37:30.120626+0000","last_fullsized":"2024-09-11T07:37:30.120626+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:40.026982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_clean_scrub_stamp":"2024-09-11T07:36:40.026982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:00.546972+0000","scrub_duration":0.0090395279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.120706+0000","last_change":"2024-09-11T07:37:25.854831+0000","last_active":"2024-09-11T07:37:30.120706+0000","last_peered":"2024-09-11T07:37:30.120706+0000","last_clean":"2024-09-11T07:37:30.120706+0000","last_became_active":"2024-09-11T07:37:25.854472+0000","last_became_peered":"2024-09-11T07:37:25.854472+0000","last_unstale":"2024-09-11T07:37:30.120706+0000","last_undegraded":"2024-09-11T07:37:30.120706+0000","last_fullsized":"2024-09-11T07:37:30.120706+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:54.556048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"234'41","reported_seq":206,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121379+0000","last_change":"2024-09-11T07:36:41.343699+0000","last_active":"2024-09-11T07:37:30.121379+0000","last_peered":"2024-09-11T07:37:30.121379+0000","last_clean":"2024-09-11T07:37:30.121379+0000","last_became_active":"2024-09-11T07:33:10.017961+0000","last_became_peered":"2024-09-11T07:33:10.017961+0000","last_unstale":"2024-09-11T07:37:30.121379+0000","last_undegraded":"2024-09-11T07:37:30.121379+0000","last_fullsized":"2024-09-11T07:37:30.121379+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'41","last_scrub_stamp":"2024-09-11T07:36:41.343611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:36:41.343611+0000","objects_scrubbed":10,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:57.163851+0000","scrub_duration":0.0043492390000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":264,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122721+0000","last_change":"2024-09-11T07:36:51.308567+0000","last_active":"2024-09-11T07:37:30.122721+0000","last_peered":"2024-09-11T07:37:30.122721+0000","last_clean":"2024-09-11T07:37:30.122721+0000","last_became_active":"2024-09-11T07:26:31.424618+0000","last_became_peered":"2024-09-11T07:26:31.424618+0000","last_unstale":"2024-09-11T07:37:30.122721+0000","last_undegraded":"2024-09-11T07:37:30.122721+0000","last_fullsized":"2024-09-11T07:37:30.122721+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:51.308492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:51.308492+0000","last_clean_scrub_stamp":"2024-09-11T07:36:51.308492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:51.904303+0000","scrub_duration":0.0057563680000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":260,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:33.144982+0000","last_change":"2024-09-11T07:37:33.144930+0000","last_active":"2024-09-11T07:37:33.144982+0000","last_peered":"2024-09-11T07:37:33.144982+0000","last_clean":"2024-09-11T07:37:33.144982+0000","last_became_active":"2024-09-11T07:26:35.560363+0000","last_became_peered":"2024-09-11T07:26:35.560363+0000","last_unstale":"2024-09-11T07:37:33.144982+0000","last_undegraded":"2024-09-11T07:37:33.144982+0000","last_fullsized":"2024-09-11T07:37:33.144982+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:33.144873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:33.144873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:42.928193+0000","scrub_duration":0.001754675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":255,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121507+0000","last_change":"2024-09-11T07:35:27.765100+0000","last_active":"2024-09-11T07:37:30.121507+0000","last_peered":"2024-09-11T07:37:30.121507+0000","last_clean":"2024-09-11T07:37:30.121507+0000","last_became_active":"2024-09-11T07:32:00.505919+0000","last_became_peered":"2024-09-11T07:32:00.505919+0000","last_unstale":"2024-09-11T07:37:30.121507+0000","last_undegraded":"2024-09-11T07:37:30.121507+0000","last_fullsized":"2024-09-11T07:37:30.121507+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:27.765020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:35:27.765020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001488807,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122448+0000","last_change":"2024-09-11T07:37:25.855294+0000","last_active":"2024-09-11T07:37:30.122448+0000","last_peered":"2024-09-11T07:37:30.122448+0000","last_clean":"2024-09-11T07:37:30.122448+0000","last_became_active":"2024-09-11T07:37:25.855005+0000","last_became_peered":"2024-09-11T07:37:25.855005+0000","last_unstale":"2024-09-11T07:37:30.122448+0000","last_undegraded":"2024-09-11T07:37:30.122448+0000","last_fullsized":"2024-09-11T07:37:30.122448+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:26.698988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"234'22","reported_seq":289,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.869152+0000","last_change":"2024-09-11T07:37:20.965215+0000","last_active":"2024-09-11T07:37:29.869152+0000","last_peered":"2024-09-11T07:37:29.869152+0000","last_clean":"2024-09-11T07:37:29.869152+0000","last_became_active":"2024-09-11T07:26:33.546126+0000","last_became_peered":"2024-09-11T07:26:33.546126+0000","last_unstale":"2024-09-11T07:37:29.869152+0000","last_undegraded":"2024-09-11T07:37:29.869152+0000","last_fullsized":"2024-09-11T07:37:29.869152+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'22","last_scrub_stamp":"2024-09-11T07:37:20.965147+0000","last_deep_scrub":"144'18","last_deep_scrub_stamp":"2024-09-11T07:31:51.603378+0000","last_clean_scrub_stamp":"2024-09-11T07:37:20.965147+0000","objects_scrubbed":5,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:26.343736+0000","scrub_duration":0.0019365999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":926,"reported_epoch":253,"state":"active+clean","last_fresh":"2024-09-11T07:37:34.032025+0000","last_change":"2024-09-11T07:37:22.990093+0000","last_active":"2024-09-11T07:37:34.032025+0000","last_peered":"2024-09-11T07:37:34.032025+0000","last_clean":"2024-09-11T07:37:34.032025+0000","last_became_active":"2024-09-11T07:26:31.421276+0000","last_became_peered":"2024-09-11T07:26:31.421276+0000","last_unstale":"2024-09-11T07:37:34.032025+0000","last_undegraded":"2024-09-11T07:37:34.032025+0000","last_fullsized":"2024-09-11T07:37:34.032025+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'5","last_scrub_stamp":"2024-09-11T07:37:22.990033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:22.990033+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:51.516885+0000","scrub_duration":0.001646954,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":655,"num_read_kb":650,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":274,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.867749+0000","last_change":"2024-09-11T07:36:45.059852+0000","last_active":"2024-09-11T07:37:29.867749+0000","last_peered":"2024-09-11T07:37:29.867749+0000","last_clean":"2024-09-11T07:37:29.867749+0000","last_became_active":"2024-09-11T07:26:31.413354+0000","last_became_peered":"2024-09-11T07:26:31.413354+0000","last_unstale":"2024-09-11T07:37:29.867749+0000","last_undegraded":"2024-09-11T07:37:29.867749+0000","last_fullsized":"2024-09-11T07:37:29.867749+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:45.059537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:07.149367+0000","last_clean_scrub_stamp":"2024-09-11T07:36:45.059537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:06.540727+0000","scrub_duration":0.013492418000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":267,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868155+0000","last_change":"2024-09-11T07:36:40.189921+0000","last_active":"2024-09-11T07:37:29.868155+0000","last_peered":"2024-09-11T07:37:29.868155+0000","last_clean":"2024-09-11T07:37:29.868155+0000","last_became_active":"2024-09-11T07:32:00.508398+0000","last_became_peered":"2024-09-11T07:32:00.508398+0000","last_unstale":"2024-09-11T07:37:29.868155+0000","last_undegraded":"2024-09-11T07:37:29.868155+0000","last_fullsized":"2024-09-11T07:37:29.868155+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:40.189372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:36:40.189372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:47.482245+0000","scrub_duration":0.0077070330000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":248,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122466+0000","last_change":"2024-09-11T07:36:56.490992+0000","last_active":"2024-09-11T07:37:30.122466+0000","last_peered":"2024-09-11T07:37:30.122466+0000","last_clean":"2024-09-11T07:37:30.122466+0000","last_became_active":"2024-09-11T07:32:00.515745+0000","last_became_peered":"2024-09-11T07:32:00.515745+0000","last_unstale":"2024-09-11T07:37:30.122466+0000","last_undegraded":"2024-09-11T07:37:30.122466+0000","last_fullsized":"2024-09-11T07:37:30.122466+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:56.490940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:56.490940+0000","last_clean_scrub_stamp":"2024-09-11T07:36:56.490940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:58.832227+0000","scrub_duration":0.037860923999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":14,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121219+0000","last_change":"2024-09-11T07:37:26.029935+0000","last_active":"2024-09-11T07:37:30.121219+0000","last_peered":"2024-09-11T07:37:30.121219+0000","last_clean":"2024-09-11T07:37:30.121219+0000","last_became_active":"2024-09-11T07:37:26.029608+0000","last_became_peered":"2024-09-11T07:37:26.029608+0000","last_unstale":"2024-09-11T07:37:30.121219+0000","last_undegraded":"2024-09-11T07:37:30.121219+0000","last_fullsized":"2024-09-11T07:37:30.121219+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:38.232917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"234'30","reported_seq":328,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122026+0000","last_change":"2024-09-11T07:36:45.088277+0000","last_active":"2024-09-11T07:37:30.122026+0000","last_peered":"2024-09-11T07:37:30.122026+0000","last_clean":"2024-09-11T07:37:30.122026+0000","last_became_active":"2024-09-11T07:28:09.018134+0000","last_became_peered":"2024-09-11T07:28:09.018134+0000","last_unstale":"2024-09-11T07:37:30.122026+0000","last_undegraded":"2024-09-11T07:37:30.122026+0000","last_fullsized":"2024-09-11T07:37:30.122026+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'30","last_scrub_stamp":"2024-09-11T07:36:45.087995+0000","last_deep_scrub":"94'18","last_deep_scrub_stamp":"2024-09-11T07:28:51.437397+0000","last_clean_scrub_stamp":"2024-09-11T07:36:45.087995+0000","objects_scrubbed":6,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:47.102679+0000","scrub_duration":0.027742936999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":260,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121174+0000","last_change":"2024-09-11T07:35:53.257684+0000","last_active":"2024-09-11T07:37:30.121174+0000","last_peered":"2024-09-11T07:37:30.121174+0000","last_clean":"2024-09-11T07:37:30.121174+0000","last_became_active":"2024-09-11T07:26:31.420320+0000","last_became_peered":"2024-09-11T07:26:31.420320+0000","last_unstale":"2024-09-11T07:37:30.121174+0000","last_undegraded":"2024-09-11T07:37:30.121174+0000","last_fullsized":"2024-09-11T07:37:30.121174+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-11T07:35:53.257474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:35:53.257474+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053710479999999998,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":259,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121657+0000","last_change":"2024-09-11T07:37:26.557037+0000","last_active":"2024-09-11T07:37:30.121657+0000","last_peered":"2024-09-11T07:37:30.121657+0000","last_clean":"2024-09-11T07:37:30.121657+0000","last_became_active":"2024-09-11T07:32:00.520035+0000","last_became_peered":"2024-09-11T07:32:00.520035+0000","last_unstale":"2024-09-11T07:37:30.121657+0000","last_undegraded":"2024-09-11T07:37:30.121657+0000","last_fullsized":"2024-09-11T07:37:30.121657+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:26.556972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:16.967492+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.556972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:43.963183+0000","scrub_duration":0.001503206,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":240,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122565+0000","last_change":"2024-09-11T07:35:51.187203+0000","last_active":"2024-09-11T07:37:30.122565+0000","last_peered":"2024-09-11T07:37:30.122565+0000","last_clean":"2024-09-11T07:37:30.122565+0000","last_became_active":"2024-09-11T07:26:35.564178+0000","last_became_peered":"2024-09-11T07:26:35.564178+0000","last_unstale":"2024-09-11T07:37:30.122565+0000","last_undegraded":"2024-09-11T07:37:30.122565+0000","last_fullsized":"2024-09-11T07:37:30.122565+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:51.187127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:51.187127+0000","last_clean_scrub_stamp":"2024-09-11T07:35:51.187127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0055970839999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":250,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.867961+0000","last_change":"2024-09-11T07:35:35.789115+0000","last_active":"2024-09-11T07:37:29.867961+0000","last_peered":"2024-09-11T07:37:29.867961+0000","last_clean":"2024-09-11T07:37:29.867961+0000","last_became_active":"2024-09-11T07:26:37.561378+0000","last_became_peered":"2024-09-11T07:26:37.561378+0000","last_unstale":"2024-09-11T07:37:29.867961+0000","last_undegraded":"2024-09-11T07:37:29.867961+0000","last_fullsized":"2024-09-11T07:37:29.867961+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:35.789024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:35:35.789024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079175110000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.120623+0000","last_change":"2024-09-11T07:37:25.854273+0000","last_active":"2024-09-11T07:37:30.120623+0000","last_peered":"2024-09-11T07:37:30.120623+0000","last_clean":"2024-09-11T07:37:30.120623+0000","last_became_active":"2024-09-11T07:37:25.853947+0000","last_became_peered":"2024-09-11T07:37:25.853947+0000","last_unstale":"2024-09-11T07:37:30.120623+0000","last_undegraded":"2024-09-11T07:37:30.120623+0000","last_fullsized":"2024-09-11T07:37:30.120623+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:38.389157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"234'26","reported_seq":93,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.869066+0000","last_change":"2024-09-11T07:36:05.225033+0000","last_active":"2024-09-11T07:37:29.869066+0000","last_peered":"2024-09-11T07:37:29.869066+0000","last_clean":"2024-09-11T07:37:29.869066+0000","last_became_active":"2024-09-11T07:36:05.224704+0000","last_became_peered":"2024-09-11T07:36:05.224704+0000","last_unstale":"2024-09-11T07:37:29.869066+0000","last_undegraded":"2024-09-11T07:37:29.869066+0000","last_fullsized":"2024-09-11T07:37:29.869066+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-09-11T07:35:35.243253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:35:35.243253+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'3","reported_seq":262,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121661+0000","last_change":"2024-09-11T07:36:54.937298+0000","last_active":"2024-09-11T07:37:30.121661+0000","last_peered":"2024-09-11T07:37:30.121661+0000","last_clean":"2024-09-11T07:37:30.121661+0000","last_became_active":"2024-09-11T07:26:31.419501+0000","last_became_peered":"2024-09-11T07:26:31.419501+0000","last_unstale":"2024-09-11T07:37:30.121661+0000","last_undegraded":"2024-09-11T07:37:30.121661+0000","last_fullsized":"2024-09-11T07:37:30.121661+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-09-11T07:36:54.937201+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_clean_scrub_stamp":"2024-09-11T07:36:54.937201+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:15.893124+0000","scrub_duration":0.007045239,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":264,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121554+0000","last_change":"2024-09-11T07:35:37.139617+0000","last_active":"2024-09-11T07:37:30.121554+0000","last_peered":"2024-09-11T07:37:30.121554+0000","last_clean":"2024-09-11T07:37:30.121554+0000","last_became_active":"2024-09-11T07:32:00.498002+0000","last_became_peered":"2024-09-11T07:32:00.498002+0000","last_unstale":"2024-09-11T07:37:30.121554+0000","last_undegraded":"2024-09-11T07:37:30.121554+0000","last_fullsized":"2024-09-11T07:37:30.121554+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:37.139518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:35:37.139518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.019842492,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"87'81","reported_seq":275,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121574+0000","last_change":"2024-09-11T07:36:45.398752+0000","last_active":"2024-09-11T07:37:30.121574+0000","last_peered":"2024-09-11T07:37:30.121574+0000","last_clean":"2024-09-11T07:37:30.121574+0000","last_became_active":"2024-09-11T07:32:00.519656+0000","last_became_peered":"2024-09-11T07:32:00.519656+0000","last_unstale":"2024-09-11T07:37:30.121574+0000","last_undegraded":"2024-09-11T07:37:30.121574+0000","last_fullsized":"2024-09-11T07:37:30.121574+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:36:45.398664+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:36:45.398664+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:47.953625+0000","scrub_duration":0.0041388420000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":235,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122655+0000","last_change":"2024-09-11T07:35:35.328370+0000","last_active":"2024-09-11T07:37:30.122655+0000","last_peered":"2024-09-11T07:37:30.122655+0000","last_clean":"2024-09-11T07:37:30.122655+0000","last_became_active":"2024-09-11T07:32:00.510888+0000","last_became_peered":"2024-09-11T07:32:00.510888+0000","last_unstale":"2024-09-11T07:37:30.122655+0000","last_undegraded":"2024-09-11T07:37:30.122655+0000","last_fullsized":"2024-09-11T07:37:30.122655+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:35.328281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:35:35.328281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011807075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":248,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122656+0000","last_change":"2024-09-11T07:37:23.436064+0000","last_active":"2024-09-11T07:37:30.122656+0000","last_peered":"2024-09-11T07:37:30.122656+0000","last_clean":"2024-09-11T07:37:30.122656+0000","last_became_active":"2024-09-11T07:26:37.559920+0000","last_became_peered":"2024-09-11T07:26:37.559920+0000","last_unstale":"2024-09-11T07:37:30.122656+0000","last_undegraded":"2024-09-11T07:37:30.122656+0000","last_fullsized":"2024-09-11T07:37:30.122656+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:23.436012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:23.436012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:25.352176+0000","scrub_duration":0.001213481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122567+0000","last_change":"2024-09-11T07:37:25.855953+0000","last_active":"2024-09-11T07:37:30.122567+0000","last_peered":"2024-09-11T07:37:30.122567+0000","last_clean":"2024-09-11T07:37:30.122567+0000","last_became_active":"2024-09-11T07:37:25.855615+0000","last_became_peered":"2024-09-11T07:37:25.855615+0000","last_unstale":"2024-09-11T07:37:30.122567+0000","last_undegraded":"2024-09-11T07:37:30.122567+0000","last_fullsized":"2024-09-11T07:37:30.122567+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:25.900228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"234'35","reported_seq":108,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868819+0000","last_change":"2024-09-11T07:36:05.226411+0000","last_active":"2024-09-11T07:37:29.868819+0000","last_peered":"2024-09-11T07:37:29.868819+0000","last_clean":"2024-09-11T07:37:29.868819+0000","last_became_active":"2024-09-11T07:36:05.226021+0000","last_became_peered":"2024-09-11T07:36:05.226021+0000","last_unstale":"2024-09-11T07:37:29.868819+0000","last_undegraded":"2024-09-11T07:37:29.868819+0000","last_fullsized":"2024-09-11T07:37:29.868819+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-11T07:35:29.151274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:35:29.151274+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":264,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868968+0000","last_change":"2024-09-11T07:36:48.089096+0000","last_active":"2024-09-11T07:37:29.868968+0000","last_peered":"2024-09-11T07:37:29.868968+0000","last_clean":"2024-09-11T07:37:29.868968+0000","last_became_active":"2024-09-11T07:26:31.419560+0000","last_became_peered":"2024-09-11T07:26:31.419560+0000","last_unstale":"2024-09-11T07:37:29.868968+0000","last_undegraded":"2024-09-11T07:37:29.868968+0000","last_fullsized":"2024-09-11T07:37:29.868968+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:48.088795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:48.088795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:06.111523+0000","scrub_duration":0.0081450789999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":260,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.867513+0000","last_change":"2024-09-11T07:37:17.420289+0000","last_active":"2024-09-11T07:37:29.867513+0000","last_peered":"2024-09-11T07:37:29.867513+0000","last_clean":"2024-09-11T07:37:29.867513+0000","last_became_active":"2024-09-11T07:26:35.564044+0000","last_became_peered":"2024-09-11T07:26:35.564044+0000","last_unstale":"2024-09-11T07:37:29.867513+0000","last_undegraded":"2024-09-11T07:37:29.867513+0000","last_fullsized":"2024-09-11T07:37:29.867513+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:17.420236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:17.420236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:47.156712+0000","scrub_duration":0.0012713080000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":243,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121481+0000","last_change":"2024-09-11T07:36:31.411708+0000","last_active":"2024-09-11T07:37:30.121481+0000","last_peered":"2024-09-11T07:37:30.121481+0000","last_clean":"2024-09-11T07:37:30.121481+0000","last_became_active":"2024-09-11T07:32:00.516916+0000","last_became_peered":"2024-09-11T07:32:00.516916+0000","last_unstale":"2024-09-11T07:37:30.121481+0000","last_undegraded":"2024-09-11T07:37:30.121481+0000","last_fullsized":"2024-09-11T07:37:30.121481+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:31.411554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:31.411554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:49.766518+0000","scrub_duration":0.002948132,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122262+0000","last_change":"2024-09-11T07:37:25.854298+0000","last_active":"2024-09-11T07:37:30.122262+0000","last_peered":"2024-09-11T07:37:30.122262+0000","last_clean":"2024-09-11T07:37:30.122262+0000","last_became_active":"2024-09-11T07:37:25.854037+0000","last_became_peered":"2024-09-11T07:37:25.854037+0000","last_unstale":"2024-09-11T07:37:30.122262+0000","last_undegraded":"2024-09-11T07:37:30.122262+0000","last_fullsized":"2024-09-11T07:37:30.122262+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:45.294877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"234'15","reported_seq":277,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121302+0000","last_change":"2024-09-11T07:36:46.033480+0000","last_active":"2024-09-11T07:37:30.121302+0000","last_peered":"2024-09-11T07:37:30.121302+0000","last_clean":"2024-09-11T07:37:30.121302+0000","last_became_active":"2024-09-11T07:26:33.543148+0000","last_became_peered":"2024-09-11T07:26:33.543148+0000","last_unstale":"2024-09-11T07:37:30.121302+0000","last_undegraded":"2024-09-11T07:37:30.121302+0000","last_fullsized":"2024-09-11T07:37:30.121302+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:36:46.033388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:36:46.033388+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:00.259650+0000","scrub_duration":0.0084720030000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":263,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.869031+0000","last_change":"2024-09-11T07:37:26.019680+0000","last_active":"2024-09-11T07:37:29.869031+0000","last_peered":"2024-09-11T07:37:29.869031+0000","last_clean":"2024-09-11T07:37:29.869031+0000","last_became_active":"2024-09-11T07:26:31.423056+0000","last_became_peered":"2024-09-11T07:26:31.423056+0000","last_unstale":"2024-09-11T07:37:29.869031+0000","last_undegraded":"2024-09-11T07:37:29.869031+0000","last_fullsized":"2024-09-11T07:37:29.869031+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:26.019623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.019623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:43.097013+0000","scrub_duration":0.0013393599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":260,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.867518+0000","last_change":"2024-09-11T07:36:39.141707+0000","last_active":"2024-09-11T07:37:29.867518+0000","last_peered":"2024-09-11T07:37:29.867518+0000","last_clean":"2024-09-11T07:37:29.867518+0000","last_became_active":"2024-09-11T07:26:35.563182+0000","last_became_peered":"2024-09-11T07:26:35.563182+0000","last_unstale":"2024-09-11T07:37:29.867518+0000","last_undegraded":"2024-09-11T07:37:29.867518+0000","last_fullsized":"2024-09-11T07:37:29.867518+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:39.141620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:36:39.141620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:54.791536+0000","scrub_duration":0.01162813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":265,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121094+0000","last_change":"2024-09-11T07:36:36.023079+0000","last_active":"2024-09-11T07:37:30.121094+0000","last_peered":"2024-09-11T07:37:30.121094+0000","last_clean":"2024-09-11T07:37:30.121094+0000","last_became_active":"2024-09-11T07:32:00.505185+0000","last_became_peered":"2024-09-11T07:32:00.505185+0000","last_unstale":"2024-09-11T07:37:30.121094+0000","last_undegraded":"2024-09-11T07:37:30.121094+0000","last_fullsized":"2024-09-11T07:37:30.121094+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:36.023006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:46.944763+0000","last_clean_scrub_stamp":"2024-09-11T07:36:36.023006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:02.683265+0000","scrub_duration":0.011934019000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121215+0000","last_change":"2024-09-11T07:37:25.854781+0000","last_active":"2024-09-11T07:37:30.121215+0000","last_peered":"2024-09-11T07:37:30.121215+0000","last_clean":"2024-09-11T07:37:30.121215+0000","last_became_active":"2024-09-11T07:37:25.854037+0000","last_became_peered":"2024-09-11T07:37:25.854037+0000","last_unstale":"2024-09-11T07:37:30.121215+0000","last_undegraded":"2024-09-11T07:37:30.121215+0000","last_fullsized":"2024-09-11T07:37:30.121215+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:46.537669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"234'27","reported_seq":305,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.120963+0000","last_change":"2024-09-11T07:36:32.341522+0000","last_active":"2024-09-11T07:37:30.120963+0000","last_peered":"2024-09-11T07:37:30.120963+0000","last_clean":"2024-09-11T07:37:30.120963+0000","last_became_active":"2024-09-11T07:32:00.521464+0000","last_became_peered":"2024-09-11T07:32:00.521464+0000","last_unstale":"2024-09-11T07:37:30.120963+0000","last_undegraded":"2024-09-11T07:37:30.120963+0000","last_fullsized":"2024-09-11T07:37:30.120963+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-09-11T07:36:32.341462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:36:32.341462+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:51.691479+0000","scrub_duration":0.0043984430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":281,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868395+0000","last_change":"2024-09-11T07:36:37.142765+0000","last_active":"2024-09-11T07:37:29.868395+0000","last_peered":"2024-09-11T07:37:29.868395+0000","last_clean":"2024-09-11T07:37:29.868395+0000","last_became_active":"2024-09-11T07:32:00.513949+0000","last_became_peered":"2024-09-11T07:32:00.513949+0000","last_unstale":"2024-09-11T07:37:29.868395+0000","last_undegraded":"2024-09-11T07:37:29.868395+0000","last_fullsized":"2024-09-11T07:37:29.868395+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:37.142658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_clean_scrub_stamp":"2024-09-11T07:36:37.142658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:49.494264+0000","scrub_duration":0.011370480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"151'10","reported_seq":341,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121979+0000","last_change":"2024-09-11T07:35:37.294973+0000","last_active":"2024-09-11T07:37:30.121979+0000","last_peered":"2024-09-11T07:37:30.121979+0000","last_clean":"2024-09-11T07:37:30.121979+0000","last_became_active":"2024-09-11T07:32:00.506806+0000","last_became_peered":"2024-09-11T07:32:00.506806+0000","last_unstale":"2024-09-11T07:37:30.121979+0000","last_undegraded":"2024-09-11T07:37:30.121979+0000","last_fullsized":"2024-09-11T07:37:30.121979+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:35:37.294908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:35:37.294908+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054099159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":248,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122081+0000","last_change":"2024-09-11T07:36:44.295056+0000","last_active":"2024-09-11T07:37:30.122081+0000","last_peered":"2024-09-11T07:37:30.122081+0000","last_clean":"2024-09-11T07:37:30.122081+0000","last_became_active":"2024-09-11T07:26:37.861134+0000","last_became_peered":"2024-09-11T07:26:37.861134+0000","last_unstale":"2024-09-11T07:37:30.122081+0000","last_undegraded":"2024-09-11T07:37:30.122081+0000","last_fullsized":"2024-09-11T07:37:30.122081+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:44.293653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:44.293653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:57.404591+0000","scrub_duration":0.0074032500000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.867891+0000","last_change":"2024-09-11T07:37:25.852069+0000","last_active":"2024-09-11T07:37:29.867891+0000","last_peered":"2024-09-11T07:37:29.867891+0000","last_clean":"2024-09-11T07:37:29.867891+0000","last_became_active":"2024-09-11T07:37:25.851789+0000","last_became_peered":"2024-09-11T07:37:25.851789+0000","last_unstale":"2024-09-11T07:37:29.867891+0000","last_undegraded":"2024-09-11T07:37:29.867891+0000","last_fullsized":"2024-09-11T07:37:29.867891+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:40.599431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"234'23","reported_seq":279,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868262+0000","last_change":"2024-09-11T07:34:47.310667+0000","last_active":"2024-09-11T07:37:29.868262+0000","last_peered":"2024-09-11T07:37:29.868262+0000","last_clean":"2024-09-11T07:37:29.868262+0000","last_became_active":"2024-09-11T07:26:33.540363+0000","last_became_peered":"2024-09-11T07:26:33.540363+0000","last_unstale":"2024-09-11T07:37:29.868262+0000","last_undegraded":"2024-09-11T07:37:29.868262+0000","last_fullsized":"2024-09-11T07:37:29.868262+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'17","last_scrub_stamp":"2024-09-11T07:34:47.310608+0000","last_deep_scrub":"83'17","last_deep_scrub_stamp":"2024-09-11T07:29:50.970355+0000","last_clean_scrub_stamp":"2024-09-11T07:34:47.310608+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054521589999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":281,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.869274+0000","last_change":"2024-09-11T07:35:46.769144+0000","last_active":"2024-09-11T07:37:29.869274+0000","last_peered":"2024-09-11T07:37:29.869274+0000","last_clean":"2024-09-11T07:37:29.869274+0000","last_became_active":"2024-09-11T07:32:00.504344+0000","last_became_peered":"2024-09-11T07:32:00.504344+0000","last_unstale":"2024-09-11T07:37:29.869274+0000","last_undegraded":"2024-09-11T07:37:29.869274+0000","last_fullsized":"2024-09-11T07:37:29.869274+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:35:46.769070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:35:46.769070+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049213720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":260,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.867233+0000","last_change":"2024-09-11T07:36:43.125248+0000","last_active":"2024-09-11T07:37:29.867233+0000","last_peered":"2024-09-11T07:37:29.867233+0000","last_clean":"2024-09-11T07:37:29.867233+0000","last_became_active":"2024-09-11T07:26:35.561967+0000","last_became_peered":"2024-09-11T07:26:35.561967+0000","last_unstale":"2024-09-11T07:37:29.867233+0000","last_undegraded":"2024-09-11T07:37:29.867233+0000","last_fullsized":"2024-09-11T07:37:29.867233+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:43.125187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:44.820354+0000","last_clean_scrub_stamp":"2024-09-11T07:36:43.125187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:59.275217+0000","scrub_duration":0.0061424100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":265,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121837+0000","last_change":"2024-09-11T07:37:16.860986+0000","last_active":"2024-09-11T07:37:30.121837+0000","last_peered":"2024-09-11T07:37:30.121837+0000","last_clean":"2024-09-11T07:37:30.121837+0000","last_became_active":"2024-09-11T07:32:00.502019+0000","last_became_peered":"2024-09-11T07:32:00.502019+0000","last_unstale":"2024-09-11T07:37:30.121837+0000","last_undegraded":"2024-09-11T07:37:30.121837+0000","last_fullsized":"2024-09-11T07:37:30.121837+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:16.860930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:16.860930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:35.856490+0000","scrub_duration":0.0024991000000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"234'15","reported_seq":294,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121946+0000","last_change":"2024-09-11T07:37:19.096890+0000","last_active":"2024-09-11T07:37:30.121946+0000","last_peered":"2024-09-11T07:37:30.121946+0000","last_clean":"2024-09-11T07:37:30.121946+0000","last_became_active":"2024-09-11T07:32:00.509412+0000","last_became_peered":"2024-09-11T07:32:00.509412+0000","last_unstale":"2024-09-11T07:37:30.121946+0000","last_undegraded":"2024-09-11T07:37:30.121946+0000","last_fullsized":"2024-09-11T07:37:30.121946+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:37:19.096817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:19.096817+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:38.083588+0000","scrub_duration":0.00175015,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":264,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.867342+0000","last_change":"2024-09-11T07:36:53.065329+0000","last_active":"2024-09-11T07:37:29.867342+0000","last_peered":"2024-09-11T07:37:29.867342+0000","last_clean":"2024-09-11T07:37:29.867342+0000","last_became_active":"2024-09-11T07:26:31.415701+0000","last_became_peered":"2024-09-11T07:26:31.415701+0000","last_unstale":"2024-09-11T07:37:29.867342+0000","last_undegraded":"2024-09-11T07:37:29.867342+0000","last_fullsized":"2024-09-11T07:37:29.867342+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:53.065247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.925200+0000","last_clean_scrub_stamp":"2024-09-11T07:36:53.065247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:59.828393+0000","scrub_duration":0.0023028290000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":257,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122062+0000","last_change":"2024-09-11T07:36:32.708578+0000","last_active":"2024-09-11T07:37:30.122062+0000","last_peered":"2024-09-11T07:37:30.122062+0000","last_clean":"2024-09-11T07:37:30.122062+0000","last_became_active":"2024-09-11T07:32:00.499598+0000","last_became_peered":"2024-09-11T07:32:00.499598+0000","last_unstale":"2024-09-11T07:37:30.122062+0000","last_undegraded":"2024-09-11T07:37:30.122062+0000","last_fullsized":"2024-09-11T07:37:30.122062+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:32.708500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:35.392360+0000","last_clean_scrub_stamp":"2024-09-11T07:36:32.708500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:55.503565+0000","scrub_duration":0.0032820800000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":266,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868642+0000","last_change":"2024-09-11T07:36:35.186527+0000","last_active":"2024-09-11T07:37:29.868642+0000","last_peered":"2024-09-11T07:37:29.868642+0000","last_clean":"2024-09-11T07:37:29.868642+0000","last_became_active":"2024-09-11T07:32:00.506697+0000","last_became_peered":"2024-09-11T07:32:00.506697+0000","last_unstale":"2024-09-11T07:37:29.868642+0000","last_undegraded":"2024-09-11T07:37:29.868642+0000","last_fullsized":"2024-09-11T07:37:29.868642+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:36:35.185293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:35.185293+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:50.456483+0000","scrub_duration":0.0039808639999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":51,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868379+0000","last_change":"2024-09-11T07:36:51.617809+0000","last_active":"2024-09-11T07:37:29.868379+0000","last_peered":"2024-09-11T07:37:29.868379+0000","last_clean":"2024-09-11T07:37:29.868379+0000","last_became_active":"2024-09-11T07:36:05.225791+0000","last_became_peered":"2024-09-11T07:36:05.225791+0000","last_unstale":"2024-09-11T07:37:29.868379+0000","last_undegraded":"2024-09-11T07:37:29.868379+0000","last_fullsized":"2024-09-11T07:37:29.868379+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:51.617730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:51.617730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:06.231089+0000","scrub_duration":0.001776197,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"234'15","reported_seq":68,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122290+0000","last_change":"2024-09-11T07:37:26.450184+0000","last_active":"2024-09-11T07:37:30.122290+0000","last_peered":"2024-09-11T07:37:30.122290+0000","last_clean":"2024-09-11T07:37:30.122290+0000","last_became_active":"2024-09-11T07:35:50.100154+0000","last_became_peered":"2024-09-11T07:35:50.100154+0000","last_unstale":"2024-09-11T07:37:30.122290+0000","last_undegraded":"2024-09-11T07:37:30.122290+0000","last_fullsized":"2024-09-11T07:37:30.122290+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:37:26.450122+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.450122+0000","objects_scrubbed":3,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:32.005494+0000","scrub_duration":0.001711052,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":260,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122190+0000","last_change":"2024-09-11T07:37:18.340651+0000","last_active":"2024-09-11T07:37:30.122190+0000","last_peered":"2024-09-11T07:37:30.122190+0000","last_clean":"2024-09-11T07:37:30.122190+0000","last_became_active":"2024-09-11T07:26:35.566894+0000","last_became_peered":"2024-09-11T07:26:35.566894+0000","last_unstale":"2024-09-11T07:37:30.122190+0000","last_undegraded":"2024-09-11T07:37:30.122190+0000","last_fullsized":"2024-09-11T07:37:30.122190+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:18.340578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:37:18.340578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:21.567929+0000","scrub_duration":0.0054750240000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":246,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122803+0000","last_change":"2024-09-11T07:36:57.461696+0000","last_active":"2024-09-11T07:37:30.122803+0000","last_peered":"2024-09-11T07:37:30.122803+0000","last_clean":"2024-09-11T07:37:30.122803+0000","last_became_active":"2024-09-11T07:32:00.514100+0000","last_became_peered":"2024-09-11T07:32:00.514100+0000","last_unstale":"2024-09-11T07:37:30.122803+0000","last_undegraded":"2024-09-11T07:37:30.122803+0000","last_fullsized":"2024-09-11T07:37:30.122803+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:57.461617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:57.461617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:06.770396+0000","scrub_duration":0.028762672999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"234'25","reported_seq":316,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122057+0000","last_change":"2024-09-11T07:36:55.966810+0000","last_active":"2024-09-11T07:37:30.122057+0000","last_peered":"2024-09-11T07:37:30.122057+0000","last_clean":"2024-09-11T07:37:30.122057+0000","last_became_active":"2024-09-11T07:32:00.504463+0000","last_became_peered":"2024-09-11T07:32:00.504463+0000","last_unstale":"2024-09-11T07:37:30.122057+0000","last_undegraded":"2024-09-11T07:37:30.122057+0000","last_fullsized":"2024-09-11T07:37:30.122057+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'25","last_scrub_stamp":"2024-09-11T07:36:55.966595+0000","last_deep_scrub":"86'15","last_deep_scrub_stamp":"2024-09-11T07:28:22.858137+0000","last_clean_scrub_stamp":"2024-09-11T07:36:55.966595+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:24.512562+0000","scrub_duration":0.032608074000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":264,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122598+0000","last_change":"2024-09-11T07:36:34.405309+0000","last_active":"2024-09-11T07:37:30.122598+0000","last_peered":"2024-09-11T07:37:30.122598+0000","last_clean":"2024-09-11T07:37:30.122598+0000","last_became_active":"2024-09-11T07:32:00.498473+0000","last_became_peered":"2024-09-11T07:32:00.498473+0000","last_unstale":"2024-09-11T07:37:30.122598+0000","last_undegraded":"2024-09-11T07:37:30.122598+0000","last_fullsized":"2024-09-11T07:37:30.122598+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:34.405241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:34.405241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:39.596038+0000","scrub_duration":0.0039370459999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":390,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122049+0000","last_change":"2024-09-11T07:37:17.360240+0000","last_active":"2024-09-11T07:37:30.122049+0000","last_peered":"2024-09-11T07:37:30.122049+0000","last_clean":"2024-09-11T07:37:30.122049+0000","last_became_active":"2024-09-11T07:26:35.562245+0000","last_became_peered":"2024-09-11T07:26:35.562245+0000","last_unstale":"2024-09-11T07:37:30.122049+0000","last_undegraded":"2024-09-11T07:37:30.122049+0000","last_fullsized":"2024-09-11T07:37:30.122049+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:37:17.360183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:17.360183+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:33.736883+0000","scrub_duration":0.0016828819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":265,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121738+0000","last_change":"2024-09-11T07:35:52.247401+0000","last_active":"2024-09-11T07:37:30.121738+0000","last_peered":"2024-09-11T07:37:30.121738+0000","last_clean":"2024-09-11T07:37:30.121738+0000","last_became_active":"2024-09-11T07:32:00.509295+0000","last_became_peered":"2024-09-11T07:32:00.509295+0000","last_unstale":"2024-09-11T07:37:30.121738+0000","last_undegraded":"2024-09-11T07:37:30.121738+0000","last_fullsized":"2024-09-11T07:37:30.121738+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:52.247036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:35:52.247036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.032039428000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"234'19","reported_seq":291,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122029+0000","last_change":"2024-09-11T07:35:40.096134+0000","last_active":"2024-09-11T07:37:30.122029+0000","last_peered":"2024-09-11T07:37:30.122029+0000","last_clean":"2024-09-11T07:37:30.122029+0000","last_became_active":"2024-09-11T07:32:00.505134+0000","last_became_peered":"2024-09-11T07:32:00.505134+0000","last_unstale":"2024-09-11T07:37:30.122029+0000","last_undegraded":"2024-09-11T07:37:30.122029+0000","last_fullsized":"2024-09-11T07:37:30.122029+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-11T07:35:40.096061+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-11T07:28:00.959583+0000","last_clean_scrub_stamp":"2024-09-11T07:35:40.096061+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045362370000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":254,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121532+0000","last_change":"2024-09-11T07:36:27.958468+0000","last_active":"2024-09-11T07:37:30.121532+0000","last_peered":"2024-09-11T07:37:30.121532+0000","last_clean":"2024-09-11T07:37:30.121532+0000","last_became_active":"2024-09-11T07:26:31.419409+0000","last_became_peered":"2024-09-11T07:26:31.419409+0000","last_unstale":"2024-09-11T07:37:30.121532+0000","last_undegraded":"2024-09-11T07:37:30.121532+0000","last_fullsized":"2024-09-11T07:37:30.121532+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:27.958386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:27.958386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:55.062218+0000","scrub_duration":0.0093493710000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"151'10","reported_seq":393,"reported_epoch":253,"state":"active+clean","last_fresh":"2024-09-11T07:37:33.241212+0000","last_change":"2024-09-11T07:35:46.977819+0000","last_active":"2024-09-11T07:37:33.241212+0000","last_peered":"2024-09-11T07:37:33.241212+0000","last_clean":"2024-09-11T07:37:33.241212+0000","last_became_active":"2024-09-11T07:32:00.504062+0000","last_became_peered":"2024-09-11T07:32:00.504062+0000","last_unstale":"2024-09-11T07:37:33.241212+0000","last_undegraded":"2024-09-11T07:37:33.241212+0000","last_fullsized":"2024-09-11T07:37:33.241212+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:35:46.977611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:35:46.977611+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012282416000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":248,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868897+0000","last_change":"2024-09-11T07:37:26.659394+0000","last_active":"2024-09-11T07:37:29.868897+0000","last_peered":"2024-09-11T07:37:29.868897+0000","last_clean":"2024-09-11T07:37:29.868897+0000","last_became_active":"2024-09-11T07:26:37.565723+0000","last_became_peered":"2024-09-11T07:26:37.565723+0000","last_unstale":"2024-09-11T07:37:29.868897+0000","last_undegraded":"2024-09-11T07:37:29.868897+0000","last_fullsized":"2024-09-11T07:37:29.868897+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:26.659337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.659337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:32.875069+0000","scrub_duration":0.0012563839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"236'16","reported_seq":287,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868865+0000","last_change":"2024-09-11T07:36:49.809594+0000","last_active":"2024-09-11T07:37:29.868865+0000","last_peered":"2024-09-11T07:37:29.868865+0000","last_clean":"2024-09-11T07:37:29.868865+0000","last_became_active":"2024-09-11T07:26:33.540688+0000","last_became_peered":"2024-09-11T07:26:33.540688+0000","last_unstale":"2024-09-11T07:37:29.868865+0000","last_undegraded":"2024-09-11T07:37:29.868865+0000","last_fullsized":"2024-09-11T07:37:29.868865+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'16","last_scrub_stamp":"2024-09-11T07:36:49.809484+0000","last_deep_scrub":"236'16","last_deep_scrub_stamp":"2024-09-11T07:36:49.809484+0000","last_clean_scrub_stamp":"2024-09-11T07:36:49.809484+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:12.361926+0000","scrub_duration":0.017423357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":50,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122082+0000","last_change":"2024-09-11T07:37:22.914840+0000","last_active":"2024-09-11T07:37:30.122082+0000","last_peered":"2024-09-11T07:37:30.122082+0000","last_clean":"2024-09-11T07:37:30.122082+0000","last_became_active":"2024-09-11T07:36:05.225465+0000","last_became_peered":"2024-09-11T07:36:05.225465+0000","last_unstale":"2024-09-11T07:37:30.122082+0000","last_undegraded":"2024-09-11T07:37:30.122082+0000","last_fullsized":"2024-09-11T07:37:30.122082+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:22.914781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:09.772000+0000","last_clean_scrub_stamp":"2024-09-11T07:37:22.914781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:37.135943+0000","scrub_duration":0.001446864,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":240,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121850+0000","last_change":"2024-09-11T07:35:51.886845+0000","last_active":"2024-09-11T07:37:30.121850+0000","last_peered":"2024-09-11T07:37:30.121850+0000","last_clean":"2024-09-11T07:37:30.121850+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:37:30.121850+0000","last_undegraded":"2024-09-11T07:37:30.121850+0000","last_fullsized":"2024-09-11T07:37:30.121850+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:51.886407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:35:51.886407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002900241,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":258,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:32.186686+0000","last_change":"2024-09-11T07:37:32.186622+0000","last_active":"2024-09-11T07:37:32.186686+0000","last_peered":"2024-09-11T07:37:32.186686+0000","last_clean":"2024-09-11T07:37:32.186686+0000","last_became_active":"2024-09-11T07:26:37.568886+0000","last_became_peered":"2024-09-11T07:26:37.568886+0000","last_unstale":"2024-09-11T07:37:32.186686+0000","last_undegraded":"2024-09-11T07:37:32.186686+0000","last_fullsized":"2024-09-11T07:37:32.186686+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:32.186561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:32.186561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:34.230574+0000","scrub_duration":0.0014950059999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"234'24","reported_seq":288,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868470+0000","last_change":"2024-09-11T07:36:31.789902+0000","last_active":"2024-09-11T07:37:29.868470+0000","last_peered":"2024-09-11T07:37:29.868470+0000","last_clean":"2024-09-11T07:37:29.868470+0000","last_became_active":"2024-09-11T07:26:33.540208+0000","last_became_peered":"2024-09-11T07:26:33.540208+0000","last_unstale":"2024-09-11T07:37:29.868470+0000","last_undegraded":"2024-09-11T07:37:29.868470+0000","last_fullsized":"2024-09-11T07:37:29.868470+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'16","last_scrub_stamp":"2024-09-11T07:36:31.789841+0000","last_deep_scrub":"136'16","last_deep_scrub_stamp":"2024-09-11T07:31:06.039023+0000","last_clean_scrub_stamp":"2024-09-11T07:36:31.789841+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:58.106789+0000","scrub_duration":0.0048561150000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":256,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868950+0000","last_change":"2024-09-11T07:36:35.957044+0000","last_active":"2024-09-11T07:37:29.868950+0000","last_peered":"2024-09-11T07:37:29.868950+0000","last_clean":"2024-09-11T07:37:29.868950+0000","last_became_active":"2024-09-11T07:26:31.422671+0000","last_became_peered":"2024-09-11T07:26:31.422671+0000","last_unstale":"2024-09-11T07:37:29.868950+0000","last_undegraded":"2024-09-11T07:37:29.868950+0000","last_fullsized":"2024-09-11T07:37:29.868950+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:35.956943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:36:35.956943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:37.115885+0000","scrub_duration":0.0053643689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":242,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868786+0000","last_change":"2024-09-11T07:36:33.978210+0000","last_active":"2024-09-11T07:37:29.868786+0000","last_peered":"2024-09-11T07:37:29.868786+0000","last_clean":"2024-09-11T07:37:29.868786+0000","last_became_active":"2024-09-11T07:26:35.566931+0000","last_became_peered":"2024-09-11T07:26:35.566931+0000","last_unstale":"2024-09-11T07:37:29.868786+0000","last_undegraded":"2024-09-11T07:37:29.868786+0000","last_fullsized":"2024-09-11T07:37:29.868786+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:33.978003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.847474+0000","last_clean_scrub_stamp":"2024-09-11T07:36:33.978003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:44.376038+0000","scrub_duration":0.012287529,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":238,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.120741+0000","last_change":"2024-09-11T07:36:43.814458+0000","last_active":"2024-09-11T07:37:30.120741+0000","last_peered":"2024-09-11T07:37:30.120741+0000","last_clean":"2024-09-11T07:37:30.120741+0000","last_became_active":"2024-09-11T07:26:37.561573+0000","last_became_peered":"2024-09-11T07:26:37.561573+0000","last_unstale":"2024-09-11T07:37:30.120741+0000","last_undegraded":"2024-09-11T07:37:30.120741+0000","last_fullsized":"2024-09-11T07:37:30.120741+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:43.814370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:43.814370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:10.518232+0000","scrub_duration":0.0093820789999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"234'10","reported_seq":266,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.120774+0000","last_change":"2024-09-11T07:36:46.900027+0000","last_active":"2024-09-11T07:37:30.120774+0000","last_peered":"2024-09-11T07:37:30.120774+0000","last_clean":"2024-09-11T07:37:30.120774+0000","last_became_active":"2024-09-11T07:26:33.545830+0000","last_became_peered":"2024-09-11T07:26:33.545830+0000","last_unstale":"2024-09-11T07:37:30.120774+0000","last_undegraded":"2024-09-11T07:37:30.120774+0000","last_fullsized":"2024-09-11T07:37:30.120774+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'10","last_scrub_stamp":"2024-09-11T07:36:46.899947+0000","last_deep_scrub":"82'6","last_deep_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_clean_scrub_stamp":"2024-09-11T07:36:46.899947+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:02.797069+0000","scrub_duration":0.003412277,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":254,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122447+0000","last_change":"2024-09-11T07:36:50.285861+0000","last_active":"2024-09-11T07:37:30.122447+0000","last_peered":"2024-09-11T07:37:30.122447+0000","last_clean":"2024-09-11T07:37:30.122447+0000","last_became_active":"2024-09-11T07:26:31.420518+0000","last_became_peered":"2024-09-11T07:26:31.420518+0000","last_unstale":"2024-09-11T07:37:30.122447+0000","last_undegraded":"2024-09-11T07:37:30.122447+0000","last_fullsized":"2024-09-11T07:37:30.122447+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:50.285756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_clean_scrub_stamp":"2024-09-11T07:36:50.285756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:51.131474+0000","scrub_duration":0.0020130360000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":175,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868138+0000","last_change":"2024-09-11T07:36:47.675100+0000","last_active":"2024-09-11T07:37:29.868138+0000","last_peered":"2024-09-11T07:37:29.868138+0000","last_clean":"2024-09-11T07:37:29.868138+0000","last_became_active":"2024-09-11T07:36:05.226194+0000","last_became_peered":"2024-09-11T07:36:05.226194+0000","last_unstale":"2024-09-11T07:37:29.868138+0000","last_undegraded":"2024-09-11T07:37:29.868138+0000","last_fullsized":"2024-09-11T07:37:29.868138+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:47.675050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:47.675050+0000","last_clean_scrub_stamp":"2024-09-11T07:36:47.675050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:50.604543+0000","scrub_duration":0.00141387,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":248,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121271+0000","last_change":"2024-09-11T07:36:50.407702+0000","last_active":"2024-09-11T07:37:30.121271+0000","last_peered":"2024-09-11T07:37:30.121271+0000","last_clean":"2024-09-11T07:37:30.121271+0000","last_became_active":"2024-09-11T07:32:00.511407+0000","last_became_peered":"2024-09-11T07:32:00.511407+0000","last_unstale":"2024-09-11T07:37:30.121271+0000","last_undegraded":"2024-09-11T07:37:30.121271+0000","last_fullsized":"2024-09-11T07:37:30.121271+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:50.407649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:50.407649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:52.000990+0000","scrub_duration":0.01288723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"234'19","reported_seq":280,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121023+0000","last_change":"2024-09-11T07:35:32.805542+0000","last_active":"2024-09-11T07:37:30.121023+0000","last_peered":"2024-09-11T07:37:30.121023+0000","last_clean":"2024-09-11T07:37:30.121023+0000","last_became_active":"2024-09-11T07:32:00.502081+0000","last_became_peered":"2024-09-11T07:32:00.502081+0000","last_unstale":"2024-09-11T07:37:30.121023+0000","last_undegraded":"2024-09-11T07:37:30.121023+0000","last_fullsized":"2024-09-11T07:37:30.121023+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:35:32.805318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:35:32.805318+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0068191889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":40,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122748+0000","last_change":"2024-09-11T07:36:05.228927+0000","last_active":"2024-09-11T07:37:30.122748+0000","last_peered":"2024-09-11T07:37:30.122748+0000","last_clean":"2024-09-11T07:37:30.122748+0000","last_became_active":"2024-09-11T07:36:05.228360+0000","last_became_peered":"2024-09-11T07:36:05.228360+0000","last_unstale":"2024-09-11T07:37:30.122748+0000","last_undegraded":"2024-09-11T07:37:30.122748+0000","last_fullsized":"2024-09-11T07:37:30.122748+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:48.366931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:48.366931+0000","last_clean_scrub_stamp":"2024-09-11T07:35:48.366931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":221,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121255+0000","last_change":"2024-09-11T07:36:36.444436+0000","last_active":"2024-09-11T07:37:30.121255+0000","last_peered":"2024-09-11T07:37:30.121255+0000","last_clean":"2024-09-11T07:37:30.121255+0000","last_became_active":"2024-09-11T07:32:00.520534+0000","last_became_peered":"2024-09-11T07:32:00.520534+0000","last_unstale":"2024-09-11T07:37:30.121255+0000","last_undegraded":"2024-09-11T07:37:30.121255+0000","last_fullsized":"2024-09-11T07:37:30.121255+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:36.444301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:39.021876+0000","last_clean_scrub_stamp":"2024-09-11T07:36:36.444301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:01.603071+0000","scrub_duration":0.0015961,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":238,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121711+0000","last_change":"2024-09-11T07:36:49.899993+0000","last_active":"2024-09-11T07:37:30.121711+0000","last_peered":"2024-09-11T07:37:30.121711+0000","last_clean":"2024-09-11T07:37:30.121711+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:37:30.121711+0000","last_undegraded":"2024-09-11T07:37:30.121711+0000","last_fullsized":"2024-09-11T07:37:30.121711+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:49.899620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:49.899620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:12.310909+0000","scrub_duration":0.014021288999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":225,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121386+0000","last_change":"2024-09-11T07:36:44.447910+0000","last_active":"2024-09-11T07:37:30.121386+0000","last_peered":"2024-09-11T07:37:30.121386+0000","last_clean":"2024-09-11T07:37:30.121386+0000","last_became_active":"2024-09-11T07:32:00.511258+0000","last_became_peered":"2024-09-11T07:32:00.511258+0000","last_unstale":"2024-09-11T07:37:30.121386+0000","last_undegraded":"2024-09-11T07:37:30.121386+0000","last_fullsized":"2024-09-11T07:37:30.121386+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:36:44.447809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:44.447809+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:07.090809+0000","scrub_duration":0.005540555,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":273,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121021+0000","last_change":"2024-09-11T07:36:31.353614+0000","last_active":"2024-09-11T07:37:30.121021+0000","last_peered":"2024-09-11T07:37:30.121021+0000","last_clean":"2024-09-11T07:37:30.121021+0000","last_became_active":"2024-09-11T07:32:00.518307+0000","last_became_peered":"2024-09-11T07:32:00.518307+0000","last_unstale":"2024-09-11T07:37:30.121021+0000","last_undegraded":"2024-09-11T07:37:30.121021+0000","last_fullsized":"2024-09-11T07:37:30.121021+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:36:31.353426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:31.353426+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:54.521250+0000","scrub_duration":0.0063773889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"234'20","reported_seq":92,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868782+0000","last_change":"2024-09-11T07:37:29.661799+0000","last_active":"2024-09-11T07:37:29.868782+0000","last_peered":"2024-09-11T07:37:29.868782+0000","last_clean":"2024-09-11T07:37:29.868782+0000","last_became_active":"2024-09-11T07:35:00.561615+0000","last_became_peered":"2024-09-11T07:35:00.561615+0000","last_unstale":"2024-09-11T07:37:29.868782+0000","last_undegraded":"2024-09-11T07:37:29.868782+0000","last_fullsized":"2024-09-11T07:37:29.868782+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'20","last_scrub_stamp":"2024-09-11T07:37:29.661671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:29.661671+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:58.370347+0000","scrub_duration":0.0019609900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1f","version":"225'11","reported_seq":82,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.869267+0000","last_change":"2024-09-11T07:36:05.227568+0000","last_active":"2024-09-11T07:37:29.869267+0000","last_peered":"2024-09-11T07:37:29.869267+0000","last_clean":"2024-09-11T07:37:29.869267+0000","last_became_active":"2024-09-11T07:36:05.227140+0000","last_became_peered":"2024-09-11T07:36:05.227140+0000","last_unstale":"2024-09-11T07:37:29.869267+0000","last_undegraded":"2024-09-11T07:37:29.869267+0000","last_fullsized":"2024-09-11T07:37:29.869267+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:35:32.194121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:35:32.194121+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":247,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868855+0000","last_change":"2024-09-11T07:35:53.875376+0000","last_active":"2024-09-11T07:37:29.868855+0000","last_peered":"2024-09-11T07:37:29.868855+0000","last_clean":"2024-09-11T07:37:29.868855+0000","last_became_active":"2024-09-11T07:26:37.562501+0000","last_became_peered":"2024-09-11T07:26:37.562501+0000","last_unstale":"2024-09-11T07:37:29.868855+0000","last_undegraded":"2024-09-11T07:37:29.868855+0000","last_fullsized":"2024-09-11T07:37:29.868855+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:53.875300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:53.875300+0000","last_clean_scrub_stamp":"2024-09-11T07:35:53.875300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019957579999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":254,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121325+0000","last_change":"2024-09-11T07:36:38.841363+0000","last_active":"2024-09-11T07:37:30.121325+0000","last_peered":"2024-09-11T07:37:30.121325+0000","last_clean":"2024-09-11T07:37:30.121325+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:37:30.121325+0000","last_undegraded":"2024-09-11T07:37:30.121325+0000","last_fullsized":"2024-09-11T07:37:30.121325+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:38.841250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:38.841250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:44.208784+0000","scrub_duration":0.014360088999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"234'19","reported_seq":83,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122658+0000","last_change":"2024-09-11T07:36:05.229126+0000","last_active":"2024-09-11T07:37:30.122658+0000","last_peered":"2024-09-11T07:37:30.122658+0000","last_clean":"2024-09-11T07:37:30.122658+0000","last_became_active":"2024-09-11T07:36:05.228746+0000","last_became_peered":"2024-09-11T07:36:05.228746+0000","last_unstale":"2024-09-11T07:37:30.122658+0000","last_undegraded":"2024-09-11T07:37:30.122658+0000","last_fullsized":"2024-09-11T07:37:30.122658+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:35:36.232923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:35:36.232923+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":267,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.122119+0000","last_change":"2024-09-11T07:37:21.150801+0000","last_active":"2024-09-11T07:37:30.122119+0000","last_peered":"2024-09-11T07:37:30.122119+0000","last_clean":"2024-09-11T07:37:30.122119+0000","last_became_active":"2024-09-11T07:32:00.507050+0000","last_became_peered":"2024-09-11T07:32:00.507050+0000","last_unstale":"2024-09-11T07:37:30.122119+0000","last_undegraded":"2024-09-11T07:37:30.122119+0000","last_fullsized":"2024-09-11T07:37:30.122119+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:21.150732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:24.575210+0000","last_clean_scrub_stamp":"2024-09-11T07:37:21.150732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:42.313821+0000","scrub_duration":0.001360691,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":248,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868925+0000","last_change":"2024-09-11T07:35:49.723748+0000","last_active":"2024-09-11T07:37:29.868925+0000","last_peered":"2024-09-11T07:37:29.868925+0000","last_clean":"2024-09-11T07:37:29.868925+0000","last_became_active":"2024-09-11T07:26:37.862373+0000","last_became_peered":"2024-09-11T07:26:37.862373+0000","last_unstale":"2024-09-11T07:37:29.868925+0000","last_undegraded":"2024-09-11T07:37:29.868925+0000","last_fullsized":"2024-09-11T07:37:29.868925+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:49.723695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:35:49.723695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00410707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":179,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.869007+0000","last_change":"2024-09-11T07:36:05.224568+0000","last_active":"2024-09-11T07:37:29.869007+0000","last_peered":"2024-09-11T07:37:29.869007+0000","last_clean":"2024-09-11T07:37:29.869007+0000","last_became_active":"2024-09-11T07:36:05.224085+0000","last_became_peered":"2024-09-11T07:36:05.224085+0000","last_unstale":"2024-09-11T07:37:29.869007+0000","last_undegraded":"2024-09-11T07:37:29.869007+0000","last_fullsized":"2024-09-11T07:37:29.869007+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:30.179373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:35:30.179373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001295526,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"234'27","reported_seq":305,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868834+0000","last_change":"2024-09-11T07:37:18.388968+0000","last_active":"2024-09-11T07:37:29.868834+0000","last_peered":"2024-09-11T07:37:29.868834+0000","last_clean":"2024-09-11T07:37:29.868834+0000","last_became_active":"2024-09-11T07:26:33.542544+0000","last_became_peered":"2024-09-11T07:26:33.542544+0000","last_unstale":"2024-09-11T07:37:29.868834+0000","last_undegraded":"2024-09-11T07:37:29.868834+0000","last_fullsized":"2024-09-11T07:37:29.868834+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'27","last_scrub_stamp":"2024-09-11T07:37:18.388604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:18.388604+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:32.272894+0000","scrub_duration":0.0019030239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":249,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868823+0000","last_change":"2024-09-11T07:36:44.853517+0000","last_active":"2024-09-11T07:37:29.868823+0000","last_peered":"2024-09-11T07:37:29.868823+0000","last_clean":"2024-09-11T07:37:29.868823+0000","last_became_active":"2024-09-11T07:26:35.555862+0000","last_became_peered":"2024-09-11T07:26:35.555862+0000","last_unstale":"2024-09-11T07:37:29.868823+0000","last_undegraded":"2024-09-11T07:37:29.868823+0000","last_fullsized":"2024-09-11T07:37:29.868823+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:44.850049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:36:44.850049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:46.370556+0000","scrub_duration":0.020441075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":248,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:32.004235+0000","last_change":"2024-09-11T07:37:32.004184+0000","last_active":"2024-09-11T07:37:32.004235+0000","last_peered":"2024-09-11T07:37:32.004235+0000","last_clean":"2024-09-11T07:37:32.004235+0000","last_became_active":"2024-09-11T07:26:37.861647+0000","last_became_peered":"2024-09-11T07:26:37.861647+0000","last_unstale":"2024-09-11T07:37:32.004235+0000","last_undegraded":"2024-09-11T07:37:32.004235+0000","last_fullsized":"2024-09-11T07:37:32.004235+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:32.004125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:32.004125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:34.761692+0000","scrub_duration":0.0019485520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":261,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121998+0000","last_change":"2024-09-11T07:35:31.798326+0000","last_active":"2024-09-11T07:37:30.121998+0000","last_peered":"2024-09-11T07:37:30.121998+0000","last_clean":"2024-09-11T07:37:30.121998+0000","last_became_active":"2024-09-11T07:32:00.502947+0000","last_became_peered":"2024-09-11T07:32:00.502947+0000","last_unstale":"2024-09-11T07:37:30.121998+0000","last_undegraded":"2024-09-11T07:37:30.121998+0000","last_fullsized":"2024-09-11T07:37:30.121998+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:31.798138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:35:31.798138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015353789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"234'23","reported_seq":279,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:30.121360+0000","last_change":"2024-09-11T07:35:43.872310+0000","last_active":"2024-09-11T07:37:30.121360+0000","last_peered":"2024-09-11T07:37:30.121360+0000","last_clean":"2024-09-11T07:37:30.121360+0000","last_became_active":"2024-09-11T07:26:33.542711+0000","last_became_peered":"2024-09-11T07:26:33.542711+0000","last_unstale":"2024-09-11T07:37:30.121360+0000","last_undegraded":"2024-09-11T07:37:30.121360+0000","last_fullsized":"2024-09-11T07:37:30.121360+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'17","last_scrub_stamp":"2024-09-11T07:35:43.871918+0000","last_deep_scrub":"127'17","last_deep_scrub_stamp":"2024-09-11T07:35:43.871918+0000","last_clean_scrub_stamp":"2024-09-11T07:35:43.871918+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010390129999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":250,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-11T07:37:29.868934+0000","last_change":"2024-09-11T07:36:37.811087+0000","last_active":"2024-09-11T07:37:29.868934+0000","last_peered":"2024-09-11T07:37:29.868934+0000","last_clean":"2024-09-11T07:37:29.868934+0000","last_became_active":"2024-09-11T07:26:35.562037+0000","last_became_peered":"2024-09-11T07:26:35.562037+0000","last_unstale":"2024-09-11T07:37:29.868934+0000","last_undegraded":"2024-09-11T07:37:29.868934+0000","last_fullsized":"2024-09-11T07:37:29.868934+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:37.811009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:36:37.811009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:38.320888+0000","scrub_duration":0.0039204729999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1124,"num_read_kb":881,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":764,"ondisk_log_size":764,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":655,"num_read_kb":650,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":147,"seq":631360192580,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":334176,"kb_used_data":1360,"kb_used_omap":4,"kb_used_meta":332795,"kb_avail":93410976,"statfs":{"total":95995035648,"available":95652839424,"internally_reserved":0,"allocated":1392640,"data_stored":997969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":340782598},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.8,"15min":3.1440000000000001},"min":{"1min":0.22,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":3.0910000000000002,"5min":49.573,"15min":49.573},"last":0.46700000000000003},{"interface":"front","average":{"1min":1.177,"5min":1.833,"15min":2.9289999999999998},"min":{"1min":0.24199999999999999,"5min":0.22,"15min":0.22},"max":{"1min":6.1470000000000002,"5min":49.881,"15min":49.881},"last":0.55000000000000004}]},{"osd":1,"last update":"Wed Sep 11 07:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.98599999999999999,"15min":0.72599999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":2.9910000000000001,"5min":10.063000000000001,"15min":10.063000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.206,"15min":0.93799999999999994},"min":{"1min":0.252,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":3.6499999999999999,"5min":9.6059999999999999,"15min":9.6059999999999999},"last":0.626}]},{"osd":2,"last update":"Wed Sep 11 07:37:14 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0649999999999999,"15min":1.0129999999999999},"min":{"1min":0.27100000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":5.6909999999999998,"5min":13.99,"15min":13.99},"last":0.57799999999999996},{"interface":"front","average":{"1min":1.111,"5min":1.196,"15min":1.01},"min":{"1min":0.28499999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.2989999999999999,"5min":14.67,"15min":14.67},"last":0.59799999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":1.661,"15min":2.9529999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":3.3279999999999998,"5min":49.798999999999999,"15min":49.798999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.7210000000000001,"15min":2.883},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.4079999999999999,"5min":49.524000000000001,"15min":49.524000000000001},"last":0.499}]},{"osd":4,"last update":"Wed Sep 11 07:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.64500000000000002,"15min":0.49199999999999999},"min":{"1min":0.28799999999999998,"5min":0.253,"15min":0.253},"max":{"1min":6.0220000000000002,"5min":6.0220000000000002,"15min":6.0220000000000002},"last":0.438},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.66700000000000004,"15min":0.623},"min":{"1min":0.30199999999999999,"5min":0.247,"15min":0.247},"max":{"1min":3.044,"5min":3.9340000000000002,"15min":3.9340000000000002},"last":0.68000000000000005}]},{"osd":5,"last update":"Wed Sep 11 07:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.70899999999999996,"15min":0.52900000000000003},"min":{"1min":0.309,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.9909999999999997,"5min":5.9909999999999997,"15min":5.9909999999999997},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.77500000000000002,"15min":0.69699999999999995},"min":{"1min":0.32100000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.0149999999999997,"5min":4.9969999999999999,"15min":4.9969999999999999},"last":0.70699999999999996}]},{"osd":6,"last update":"Wed Sep 11 07:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66800000000000004,"15min":0.56299999999999994},"min":{"1min":0.34399999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":3.3490000000000002,"5min":4.6440000000000001,"15min":4.6440000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.72299999999999998,"15min":0.59899999999999998},"min":{"1min":0.34399999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":3.5470000000000002,"5min":3.694,"15min":3.694},"last":0.65400000000000003}]}]},{"osd":6,"up_from":44,"seq":188978561165,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298704,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446448,"statfs":{"total":95995035648,"available":95689162752,"internally_reserved":0,"allocated":1380352,"data_stored":1003755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:37:11 2024","interfaces":[{"interface":"back","average":{"1min":1.96,"5min":2.984,"15min":1.7549999999999999},"min":{"1min":0.22,"5min":0.20699999999999999,"15min":0.192},"max":{"1min":20.675999999999998,"5min":142.29599999999999,"15min":142.29599999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.9450000000000001,"5min":3.2999999999999998,"15min":1.8919999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":20.663,"5min":142.68799999999999,"15min":142.68799999999999},"last":0.70899999999999996}]},{"osd":1,"last update":"Wed Sep 11 07:37:11 2024","interfaces":[{"interface":"back","average":{"1min":2.964,"5min":2.645,"15min":1.3260000000000001},"min":{"1min":0.249,"5min":0.24099999999999999,"15min":0.23599999999999999},"max":{"1min":39.390000000000001,"5min":127.92100000000001,"15min":127.92100000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":3.0800000000000001,"5min":2.4790000000000001,"15min":1.3140000000000001},"min":{"1min":0.27200000000000002,"5min":0.24099999999999999,"15min":0.23000000000000001},"max":{"1min":40.186999999999998,"5min":127.746,"15min":127.746},"last":0.68899999999999995}]},{"osd":2,"last update":"Wed Sep 11 07:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.92700000000000005,"15min":0.85899999999999999},"min":{"1min":0.27300000000000002,"5min":0.248,"15min":0.248},"max":{"1min":3.6539999999999999,"5min":4.3220000000000001,"15min":26.491},"last":0.58599999999999997},{"interface":"front","average":{"1min":1.363,"5min":1.198,"15min":1.018},"min":{"1min":0.27100000000000002,"5min":0.26500000000000001,"15min":0.26400000000000001},"max":{"1min":3.9689999999999999,"5min":6.0819999999999999,"15min":26.561},"last":0.73299999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:37:11 2024","interfaces":[{"interface":"back","average":{"1min":2.274,"5min":1.4750000000000001,"15min":1.0660000000000001},"min":{"1min":0.26100000000000001,"5min":0.25600000000000001,"15min":0.254},"max":{"1min":20.734000000000002,"5min":20.734000000000002,"15min":31.268999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":2.0840000000000001,"5min":1.3149999999999999,"15min":1.1499999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":20.701000000000001,"5min":20.701000000000001,"15min":31.872},"last":0.50700000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.626,"15min":0.55700000000000005},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":1.6719999999999999,"5min":3.3599999999999999,"15min":4.4720000000000004},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56100000000000005,"15min":0.46500000000000002},"min":{"1min":0.25900000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.329,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":0.61499999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:37:11 2024","interfaces":[{"interface":"back","average":{"1min":3.6349999999999998,"5min":1.417,"15min":0.85499999999999998},"min":{"1min":0.30199999999999999,"5min":0.255,"15min":0.255},"max":{"1min":57.436,"5min":57.436,"15min":57.436},"last":0.63900000000000001},{"interface":"front","average":{"1min":3.359,"5min":1.429,"15min":0.81399999999999995},"min":{"1min":0.32300000000000001,"5min":0.251,"15min":0.251},"max":{"1min":56.822000000000003,"5min":56.822000000000003,"15min":56.822000000000003},"last":0.90200000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.68700000000000006,"15min":0.66600000000000004},"min":{"1min":0.312,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.3109999999999999,"5min":3.3140000000000001,"15min":3.3140000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73099999999999998,"15min":0.72599999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":3.6749999999999998,"5min":5.0439999999999996,"15min":5.0439999999999996},"last":0.77300000000000002}]}]},{"osd":5,"up_from":37,"seq":158913790099,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":302292,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93442860,"statfs":{"total":95995035648,"available":95685488640,"internally_reserved":0,"allocated":794624,"data_stored":407247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:36:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":2.609,"15min":2.0289999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.20799999999999999},"max":{"1min":2.7440000000000002,"5min":118.086,"15min":185.68199999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":1.6359999999999999,"5min":3.0459999999999998,"15min":2.1760000000000002},"min":{"1min":0.223,"5min":0.223,"15min":0.218},"max":{"1min":6.0279999999999996,"5min":118.004,"15min":186.12200000000001},"last":0.48899999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:36:46 2024","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.26,"15min":0.85699999999999998},"min":{"1min":0.27300000000000002,"5min":0.22700000000000001,"15min":0.20899999999999999},"max":{"1min":2.5699999999999998,"5min":26.286999999999999,"15min":26.286999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.747,"5min":1.421,"15min":0.89300000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":2.044,"5min":24.672999999999998,"15min":24.672999999999998},"last":0.45000000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:36:46 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.1890000000000001,"15min":1.556},"min":{"1min":0.309,"5min":0.29099999999999998,"15min":0.22700000000000001},"max":{"1min":4.585,"5min":12.122,"15min":187.99199999999999},"last":0.432},{"interface":"front","average":{"1min":1.2090000000000001,"5min":0.97399999999999998,"15min":1.5109999999999999},"min":{"1min":0.375,"5min":0.254,"15min":0.254},"max":{"1min":4.6749999999999998,"5min":12.038,"15min":186.375},"last":0.57299999999999995}]},{"osd":3,"last update":"Wed Sep 11 07:36:46 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":2.6539999999999999,"15min":2.1379999999999999},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.23599999999999999},"max":{"1min":3.3130000000000002,"5min":116.33799999999999,"15min":188.11500000000001},"last":0.503},{"interface":"front","average":{"1min":1.3200000000000001,"5min":2.6909999999999998,"15min":2.109},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.298,"5min":116.15600000000001,"15min":186.44200000000001},"last":0.53400000000000003}]},{"osd":4,"last update":"Wed Sep 11 07:36:46 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.46200000000000002},"min":{"1min":0.28000000000000003,"5min":0.247,"15min":0.19900000000000001},"max":{"1min":2.0920000000000001,"5min":4.7060000000000004,"15min":4.7060000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.63900000000000001,"15min":0.48999999999999999},"min":{"1min":0.248,"5min":0.24199999999999999,"15min":0.23200000000000001},"max":{"1min":2.1749999999999998,"5min":2.8439999999999999,"15min":2.8439999999999999},"last":0.56000000000000005}]},{"osd":6,"last update":"Wed Sep 11 07:37:12 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.70599999999999996,"15min":0.51900000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.26000000000000001},"max":{"1min":1.7849999999999999,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.65100000000000002,"15min":0.51200000000000001},"min":{"1min":0.33900000000000002,"5min":0.26800000000000002,"15min":0.25800000000000001},"max":{"1min":4.3120000000000003,"5min":4.3120000000000003,"15min":4.3120000000000003},"last":0.623}]},{"osd":7,"last update":"Wed Sep 11 07:37:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.745,"15min":0.64100000000000001},"min":{"1min":0.314,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":2.464,"5min":4.7919999999999998,"15min":4.7919999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.76700000000000002,"15min":0.65700000000000003},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.8140000000000001,"5min":4.0110000000000001,"15min":4.0110000000000001},"last":0.61299999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986327,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298132,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447020,"statfs":{"total":95995035648,"available":95689748480,"internally_reserved":0,"allocated":798720,"data_stored":407090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:36:34 2024","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.7869999999999999,"15min":1.093},"min":{"1min":0.217,"5min":0.217,"15min":0.20799999999999999},"max":{"1min":4.0960000000000001,"5min":35.308,"15min":35.308},"last":0.628},{"interface":"front","average":{"1min":0.90000000000000002,"5min":1.788,"15min":1.0549999999999999},"min":{"1min":0.23999999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":3.1640000000000001,"5min":35.597999999999999,"15min":35.597999999999999},"last":0.54800000000000004}]},{"osd":1,"last update":"Wed Sep 11 07:36:34 2024","interfaces":[{"interface":"back","average":{"1min":1.5609999999999999,"5min":1.46,"15min":0.96399999999999997},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":3.657,"5min":24.581,"15min":24.581},"last":0.53100000000000003},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.137,"15min":0.86099999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.23699999999999999},"max":{"1min":3.2330000000000001,"5min":24.652000000000001,"15min":24.652000000000001},"last":0.61699999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:36:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.2909999999999999,"15min":0.89000000000000001},"min":{"1min":0.29299999999999998,"5min":0.223,"15min":0.223},"max":{"1min":3.0030000000000001,"5min":19.927,"15min":19.927},"last":0.58799999999999997},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.333,"15min":0.89000000000000001},"min":{"1min":0.30599999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":3.9350000000000001,"5min":20.02,"15min":20.02},"last":0.57299999999999995}]},{"osd":3,"last update":"Wed Sep 11 07:36:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.6359999999999999,"15min":1.0409999999999999},"min":{"1min":0.28499999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":3.153,"5min":24.029,"15min":24.029},"last":0.51400000000000001},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.607,"15min":0.97999999999999998},"min":{"1min":0.34300000000000003,"5min":0.32100000000000001,"15min":0.28699999999999998},"max":{"1min":4.0830000000000002,"5min":24.265000000000001,"15min":24.265000000000001},"last":0.60399999999999998}]},{"osd":5,"last update":"Wed Sep 11 07:36:50 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.65700000000000003,"15min":0.46100000000000002},"min":{"1min":0.223,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":3.0529999999999999,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66900000000000004,"15min":0.48299999999999998},"min":{"1min":0.28299999999999997,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.52,"5min":5.7320000000000002,"15min":5.7320000000000002},"last":0.47899999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:37:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.72499999999999998,"15min":0.52700000000000002},"min":{"1min":0.255,"5min":0.23999999999999999,"15min":0.20899999999999999},"max":{"1min":1.96,"5min":7.8650000000000002,"15min":7.8650000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.73199999999999998,"15min":0.53800000000000003},"min":{"1min":0.27500000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":3.7949999999999999,"5min":9.3770000000000007,"15min":9.3770000000000007},"last":0.65400000000000003}]},{"osd":7,"last update":"Wed Sep 11 07:37:09 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.65000000000000002,"15min":0.57999999999999996},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":2.4470000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70699999999999996,"15min":0.60099999999999998},"min":{"1min":0.29799999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.2949999999999999,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":0.66400000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182556,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":306476,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93438676,"statfs":{"total":95995035648,"available":95681204224,"internally_reserved":0,"allocated":819200,"data_stored":413862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:37:16 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.6299999999999999,"15min":1.1919999999999999},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":4.2709999999999999,"5min":38.432000000000002,"15min":38.432000000000002},"last":0.26100000000000001},{"interface":"front","average":{"1min":1.353,"5min":1.7190000000000001,"15min":1.546},"min":{"1min":0.26300000000000001,"5min":0.20300000000000001,"15min":0.19500000000000001},"max":{"1min":4.657,"5min":33.426000000000002,"15min":91.876999999999995},"last":0.316}]},{"osd":1,"last update":"Wed Sep 11 07:37:16 2024","interfaces":[{"interface":"back","average":{"1min":5.1669999999999998,"5min":2.2480000000000002,"15min":1.3979999999999999},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":67.305000000000007,"5min":67.305000000000007,"15min":67.305000000000007},"last":0.34799999999999998},{"interface":"front","average":{"1min":1.252,"5min":1.484,"15min":1.0209999999999999},"min":{"1min":0.23799999999999999,"5min":0.192,"15min":0.192},"max":{"1min":4.4619999999999997,"5min":9.3450000000000006,"15min":9.3450000000000006},"last":0.36299999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":1.387,"15min":1.0980000000000001},"min":{"1min":0.24199999999999999,"5min":0.23699999999999999,"15min":0.23200000000000001},"max":{"1min":4.4240000000000004,"5min":8.282,"15min":8.282},"last":0.39600000000000002},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.2669999999999999,"15min":1.03},"min":{"1min":0.315,"5min":0.26200000000000001,"15min":0.20999999999999999},"max":{"1min":5.1520000000000001,"5min":6.5640000000000001,"15min":8.7110000000000003},"last":0.29499999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:37:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.99099999999999999,"15min":0.95499999999999996},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":2.1760000000000002,"5min":5.5099999999999998,"15min":21.856000000000002},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":1.006,"15min":1.105},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":2.7200000000000002,"5min":4.6219999999999999,"15min":19.146000000000001},"last":0.46999999999999997}]},{"osd":5,"last update":"Wed Sep 11 07:36:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.516,"15min":1.016},"min":{"1min":0.309,"5min":0.309,"15min":0.29599999999999999},"max":{"1min":4.1589999999999998,"5min":32.325000000000003,"15min":32.325000000000003},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.3440000000000001,"15min":1.091},"min":{"1min":0.28599999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.6800000000000002,"5min":7.9870000000000001,"15min":18.416},"last":0.51200000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:37:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.3740000000000001,"15min":1.3500000000000001},"min":{"1min":0.372,"5min":0.308,"15min":0.27000000000000002},"max":{"1min":4.5179999999999998,"5min":32.350999999999999,"15min":32.350999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":1.3220000000000001,"15min":1.3240000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.33100000000000002},"max":{"1min":2.2290000000000001,"5min":32.402999999999999,"15min":32.402999999999999},"last":0.48399999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:37:09 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":1.194,"15min":1.3240000000000001},"min":{"1min":0.32300000000000001,"5min":0.308,"15min":0.308},"max":{"1min":2.7679999999999998,"5min":8.0429999999999993,"15min":8.0429999999999993},"last":0.53100000000000003},{"interface":"front","average":{"1min":1.077,"5min":1.4570000000000001,"15min":2.157},"min":{"1min":0.39400000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":4.4909999999999997,"5min":32.289999999999999,"15min":32.289999999999999},"last":0.57699999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411489,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":302284,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93442868,"statfs":{"total":95995035648,"available":95685496832,"internally_reserved":0,"allocated":786432,"data_stored":413514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:36:43 2024","interfaces":[{"interface":"back","average":{"1min":2.3500000000000001,"5min":2.8170000000000002,"15min":1.4490000000000001},"min":{"1min":0.27100000000000002,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":11.17,"5min":56.704000000000001,"15min":56.704000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":2.5179999999999998,"5min":2.8730000000000002,"15min":1.5089999999999999},"min":{"1min":0.36699999999999999,"5min":0.22700000000000001,"15min":0.216},"max":{"1min":10.801,"5min":56.816000000000003,"15min":56.816000000000003},"last":0.29399999999999998}]},{"osd":1,"last update":"Wed Sep 11 07:36:43 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.9339999999999999,"15min":1.151},"min":{"1min":0.313,"5min":0.24299999999999999,"15min":0.23200000000000001},"max":{"1min":2.1739999999999999,"5min":65.153999999999996,"15min":65.153999999999996},"last":0.32700000000000001},{"interface":"front","average":{"1min":2.1499999999999999,"5min":2.4980000000000002,"15min":1.3400000000000001},"min":{"1min":0.501,"5min":0.25700000000000001,"15min":0.20699999999999999},"max":{"1min":11.302,"5min":63.311999999999998,"15min":63.311999999999998},"last":0.27400000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:37:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":2.0619999999999998,"15min":1.3220000000000001},"min":{"1min":0.28100000000000003,"5min":0.27400000000000002,"15min":0.26900000000000002},"max":{"1min":6.4960000000000004,"5min":44.405999999999999,"15min":44.405999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":1.0349999999999999,"5min":2.0259999999999998,"15min":1.3520000000000001},"min":{"1min":0.27100000000000002,"5min":0.253,"15min":0.253},"max":{"1min":3.222,"5min":43.295999999999999,"15min":43.295999999999999},"last":0.41399999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:37:34 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.90400000000000003,"15min":1.036},"min":{"1min":0.34000000000000002,"5min":0.30599999999999999,"15min":0.27500000000000002},"max":{"1min":1.004,"5min":6.7599999999999998,"15min":30.946999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.86899999999999999,"15min":0.93899999999999995},"min":{"1min":0.41599999999999998,"5min":0.31900000000000001,"15min":0.29999999999999999},"max":{"1min":1.8500000000000001,"5min":3.0960000000000001,"15min":22.434000000000001},"last":0.58099999999999996}]},{"osd":5,"last update":"Wed Sep 11 07:36:48 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.167,"15min":1.028},"min":{"1min":0.36899999999999999,"5min":0.35099999999999998,"15min":0.31900000000000001},"max":{"1min":2.585,"5min":22.460000000000001,"15min":31.789000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.3089999999999999,"15min":1.1100000000000001},"min":{"1min":0.378,"5min":0.33600000000000002,"15min":0.31},"max":{"1min":3.4319999999999999,"5min":25.696999999999999,"15min":30.898},"last":0.52800000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:37:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.2549999999999999,"15min":1.0489999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.36099999999999999},"max":{"1min":2.9430000000000001,"5min":22.399000000000001,"15min":22.399000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.3819999999999999,"15min":1.048},"min":{"1min":0.311,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.0680000000000001,"5min":25.739000000000001,"15min":25.739000000000001},"last":0.56000000000000005}]},{"osd":7,"last update":"Wed Sep 11 07:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.204,"15min":1.956},"min":{"1min":0.435,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.6600000000000001,"5min":22.484000000000002,"15min":22.484000000000002},"last":0.629},{"interface":"front","average":{"1min":1.081,"5min":1.4410000000000001,"15min":2.2959999999999998},"min":{"1min":0.40699999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":3.1949999999999998,"5min":25.670000000000002,"15min":25.670000000000002},"last":0.59399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575012,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":310624,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93434528,"statfs":{"total":95995035648,"available":95676956672,"internally_reserved":0,"allocated":811008,"data_stored":407368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:37:18 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.6180000000000001,"15min":1.6850000000000001},"min":{"1min":0.24099999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":7.9429999999999996,"5min":8.0549999999999997,"15min":82.218999999999994},"last":0.32400000000000001},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.7470000000000001,"15min":1.706},"min":{"1min":0.217,"5min":0.193,"15min":0.191},"max":{"1min":5.5019999999999998,"5min":8.0739999999999998,"15min":82.278999999999996},"last":0.40200000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:36:48 2024","interfaces":[{"interface":"back","average":{"1min":2.9500000000000002,"5min":1.8819999999999999,"15min":1.161},"min":{"1min":0.437,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":9.0969999999999995,"5min":9.0969999999999995,"15min":9.0969999999999995},"last":0.34300000000000003},{"interface":"front","average":{"1min":1.8029999999999999,"5min":1.835,"15min":1.2130000000000001},"min":{"1min":0.25800000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":9.1310000000000002,"5min":9.1310000000000002,"15min":9.1310000000000002},"last":0.41899999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:37:09 2024","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.736,"15min":1.7709999999999999},"min":{"1min":0.29199999999999998,"5min":0.28000000000000003,"15min":0.25800000000000001},"max":{"1min":6.1360000000000001,"5min":9.2889999999999997,"15min":81.882999999999996},"last":0.45000000000000001},{"interface":"front","average":{"1min":1.423,"5min":1.7170000000000001,"15min":1.7230000000000001},"min":{"1min":0.36299999999999999,"5min":0.22600000000000001,"15min":0.20999999999999999},"max":{"1min":5.2359999999999998,"5min":14.428000000000001,"15min":82.400000000000006},"last":0.39000000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:37:26 2024","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.331,"15min":0.94399999999999995},"min":{"1min":0.33000000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":5.5609999999999999,"5min":7.3630000000000004,"15min":7.3630000000000004},"last":0.52300000000000002},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.3400000000000001,"15min":1.0780000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.26500000000000001},"max":{"1min":4.4459999999999997,"5min":9.2289999999999992,"15min":9.2289999999999992},"last":0.56899999999999995}]},{"osd":5,"last update":"Wed Sep 11 07:36:51 2024","interfaces":[{"interface":"back","average":{"1min":2.2799999999999998,"5min":1.51,"15min":0.98099999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":6.7469999999999999,"5min":7.4379999999999997,"15min":7.4379999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":1.8280000000000001,"5min":1.4399999999999999,"15min":1.069},"min":{"1min":0.46500000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":9.1929999999999996,"5min":9.1929999999999996,"15min":9.1929999999999996},"last":0.60499999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:37:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.341,"15min":1.133},"min":{"1min":0.41399999999999998,"5min":0.316,"15min":0.30099999999999999},"max":{"1min":2.6299999999999999,"5min":9.2569999999999997,"15min":24.018000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.0349999999999999,"15min":0.94899999999999995},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.29999999999999999},"max":{"1min":5.2709999999999999,"5min":5.2709999999999999,"15min":23.681000000000001},"last":0.61399999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:37:09 2024","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.196,"15min":1.3},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":5.3079999999999998,"5min":7.0940000000000003,"15min":7.0940000000000003},"last":0.64000000000000001},{"interface":"front","average":{"1min":1.7689999999999999,"5min":1.4770000000000001,"15min":1.514},"min":{"1min":0.36899999999999999,"5min":0.313,"15min":0.313},"max":{"1min":5.5860000000000003,"5min":7.4050000000000002,"15min":7.4050000000000002},"last":0.56399999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738538,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":311200,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93433952,"statfs":{"total":95995035648,"available":95676366848,"internally_reserved":0,"allocated":1396736,"data_stored":1004087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:37:19 2024","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.2689999999999999,"15min":0.98399999999999999},"min":{"1min":0.32300000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":5.4630000000000001,"5min":5.4630000000000001,"15min":9.7710000000000008},"last":0.253},{"interface":"front","average":{"1min":1.8080000000000001,"5min":1.333,"15min":1.0129999999999999},"min":{"1min":0.26800000000000002,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":5.1479999999999997,"5min":5.1479999999999997,"15min":10.275},"last":0.35299999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:36:46 2024","interfaces":[{"interface":"back","average":{"1min":1.756,"5min":1.706,"15min":1.163},"min":{"1min":0.217,"5min":0.21299999999999999,"15min":0.20200000000000001},"max":{"1min":4.9930000000000003,"5min":11.397,"15min":19.527999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":1.859,"5min":1.4930000000000001,"15min":1.1140000000000001},"min":{"1min":0.23699999999999999,"5min":0.21099999999999999,"15min":0.20999999999999999},"max":{"1min":6.5960000000000001,"5min":9.7309999999999999,"15min":19.556999999999999},"last":0.28899999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:37:19 2024","interfaces":[{"interface":"back","average":{"1min":1.8140000000000001,"5min":1.55,"15min":1.7310000000000001},"min":{"1min":0.35199999999999998,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":9.4199999999999999,"5min":9.4920000000000009,"15min":45.185000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":1.379,"5min":1.383,"15min":1.4950000000000001},"min":{"1min":0.378,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.2460000000000004,"5min":8.0999999999999996,"15min":43.228999999999999},"last":0.32200000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:37:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.218,"15min":0.995},"min":{"1min":0.29799999999999999,"5min":0.26800000000000002,"15min":0.26700000000000002},"max":{"1min":4.8109999999999999,"5min":6.8319999999999999,"15min":19.411999999999999},"last":0.443},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.196,"15min":1.1319999999999999},"min":{"1min":0.35999999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":2.9260000000000002,"5min":9.5619999999999994,"15min":19.396000000000001},"last":0.60299999999999998}]},{"osd":5,"last update":"Wed Sep 11 07:36:46 2024","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.258,"15min":1.0660000000000001},"min":{"1min":0.33800000000000002,"5min":0.29599999999999999,"15min":0.28999999999999998},"max":{"1min":4.8360000000000003,"5min":6.9400000000000004,"15min":19.381},"last":0.49299999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.456,"15min":1.1659999999999999},"min":{"1min":0.36899999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":4.6790000000000003,"5min":9.6289999999999996,"15min":19.437000000000001},"last":0.57799999999999996}]},{"osd":6,"last update":"Wed Sep 11 07:37:19 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.337,"15min":1.153},"min":{"1min":0.45000000000000001,"5min":0.33300000000000002,"15min":0.316},"max":{"1min":2.29,"5min":23.190999999999999,"15min":23.190999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.995,"5min":1.4350000000000001,"15min":1.1599999999999999},"min":{"1min":0.373,"5min":0.35999999999999999,"15min":0.34200000000000003},"max":{"1min":4.1619999999999999,"5min":23.632000000000001,"15min":23.632000000000001},"last":0.53700000000000003}]},{"osd":7,"last update":"Wed Sep 11 07:37:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.288,"15min":1.6599999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":2.9540000000000002,"5min":9.6530000000000005,"15min":9.6530000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":1.282,"5min":1.2769999999999999,"15min":1.71},"min":{"1min":0.42699999999999999,"5min":0.315,"15min":0.315},"max":{"1min":4.194,"5min":5.2759999999999998,"15min":5.2759999999999998},"last":0.47799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:37:40.610 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:37:40.626 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:40.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:40.627 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:40.728 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:40.774 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:40.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:40.776 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:40.876 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:40.907 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:40.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:40.908 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:41.009 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:41.050 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:41.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:41.051 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:41.152 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:41.182 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:41.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:41.183 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:41.284 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:41.329 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:41.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:41.330 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:41.354 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:41 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:41.354 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:41 smithi022 ceph-mon[37330]: pgmap v501: 148 pgs: 148 active+clean; 419 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 52 MiB/s wr, 837 op/s 2024-09-11T07:37:41.354 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:41 smithi022 ceph-mon[37330]: 2.1c scrub starts 2024-09-11T07:37:41.354 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:41 smithi022 ceph-mon[37330]: 2.1c scrub ok 2024-09-11T07:37:41.354 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:41 smithi022 ceph-mon[37330]: 4.5 scrub starts 2024-09-11T07:37:41.354 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:41 smithi022 ceph-mon[37330]: 4.5 scrub ok 2024-09-11T07:37:41.354 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:41 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:41.354 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:41 smithi022 ceph-mon[29462]: pgmap v501: 148 pgs: 148 active+clean; 419 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 52 MiB/s wr, 837 op/s 2024-09-11T07:37:41.355 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:41 smithi022 ceph-mon[29462]: 2.1c scrub starts 2024-09-11T07:37:41.355 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:41 smithi022 ceph-mon[29462]: 2.1c scrub ok 2024-09-11T07:37:41.355 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:41 smithi022 ceph-mon[29462]: 4.5 scrub starts 2024-09-11T07:37:41.355 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:41 smithi022 ceph-mon[29462]: 4.5 scrub ok 2024-09-11T07:37:41.431 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:41.462 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:41.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:41.463 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:41.473 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 4 16 12648 12632 197.358 191.625 0.0041222 0.0050608 2024-09-11T07:37:41.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:41 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:41 smithi104 ceph-mon[33507]: pgmap v501: 148 pgs: 148 active+clean; 419 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 52 MiB/s wr, 837 op/s 2024-09-11T07:37:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:41 smithi104 ceph-mon[33507]: 2.1c scrub starts 2024-09-11T07:37:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:41 smithi104 ceph-mon[33507]: 2.1c scrub ok 2024-09-11T07:37:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:41 smithi104 ceph-mon[33507]: 4.5 scrub starts 2024-09-11T07:37:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:41 smithi104 ceph-mon[33507]: 4.5 scrub ok 2024-09-11T07:37:41.563 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:41.596 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:41.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:41.597 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:41.698 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:41.731 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:41.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:41.732 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:41.833 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:41.862 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:41.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:41.864 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:41.965 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:41.997 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:41.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:41.998 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:42.099 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:42.142 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:42.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:42.143 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:42.243 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:42.274 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:42.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:42.275 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:42.371 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:42 smithi022 ceph-mon[29462]: 4.15 deep-scrub starts 2024-09-11T07:37:42.371 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:42 smithi022 ceph-mon[29462]: 4.15 deep-scrub ok 2024-09-11T07:37:42.371 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:42 smithi022 ceph-mon[29462]: 5.4 scrub starts 2024-09-11T07:37:42.371 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:42 smithi022 ceph-mon[29462]: 5.4 scrub ok 2024-09-11T07:37:42.372 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:42 smithi022 ceph-mon[29462]: 2.1a scrub starts 2024-09-11T07:37:42.372 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:42 smithi022 ceph-mon[29462]: 2.1a scrub ok 2024-09-11T07:37:42.372 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:42 smithi022 ceph-mon[37330]: 4.15 deep-scrub starts 2024-09-11T07:37:42.372 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:42 smithi022 ceph-mon[37330]: 4.15 deep-scrub ok 2024-09-11T07:37:42.372 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:42 smithi022 ceph-mon[37330]: 5.4 scrub starts 2024-09-11T07:37:42.372 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:42 smithi022 ceph-mon[37330]: 5.4 scrub ok 2024-09-11T07:37:42.372 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:42 smithi022 ceph-mon[37330]: 2.1a scrub starts 2024-09-11T07:37:42.372 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:42 smithi022 ceph-mon[37330]: 2.1a scrub ok 2024-09-11T07:37:42.376 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:42.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:42.414 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:42.415 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:42.473 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 5 16 15555 15539 194.221 181.688 0.00275523 0.00514157 2024-09-11T07:37:42.516 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:42.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:42 smithi104 ceph-mon[33507]: 4.15 deep-scrub starts 2024-09-11T07:37:42.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:42 smithi104 ceph-mon[33507]: 4.15 deep-scrub ok 2024-09-11T07:37:42.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:42 smithi104 ceph-mon[33507]: 5.4 scrub starts 2024-09-11T07:37:42.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:42 smithi104 ceph-mon[33507]: 5.4 scrub ok 2024-09-11T07:37:42.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:42 smithi104 ceph-mon[33507]: 2.1a scrub starts 2024-09-11T07:37:42.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:42 smithi104 ceph-mon[33507]: 2.1a scrub ok 2024-09-11T07:37:42.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:42.562 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:42.562 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:42.663 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:42.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:42.698 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:42.699 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:42.799 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:42.827 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:42.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:42.828 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:42.929 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:42.960 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:42.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:42.961 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:43.062 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:43.102 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:43.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:43.103 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:43.204 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:43.235 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:43.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:43.237 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:43.269 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:37:43.269 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:37:43.284 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:37:43.290 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:37:43.311 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is set 2024-09-11T07:37:43.337 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:43.367 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:43.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:43.368 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:43.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:43 smithi022 ceph-mon[29462]: 2.1b scrub starts 2024-09-11T07:37:43.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:43 smithi022 ceph-mon[29462]: 2.1b scrub ok 2024-09-11T07:37:43.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:43 smithi022 ceph-mon[29462]: 6.16 scrub starts 2024-09-11T07:37:43.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:43 smithi022 ceph-mon[29462]: 6.16 scrub ok 2024-09-11T07:37:43.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:43 smithi022 ceph-mon[29462]: pgmap v502: 148 pgs: 148 active+clean; 419 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 47 MiB/s wr, 749 op/s 2024-09-11T07:37:43.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:43 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/308496175' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:37:43.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:43 smithi022 ceph-mon[29462]: from='client.25411 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:37:43.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:43 smithi022 ceph-mon[29462]: 5.19 scrub starts 2024-09-11T07:37:43.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:43 smithi022 ceph-mon[29462]: 5.19 scrub ok 2024-09-11T07:37:43.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:43 smithi022 ceph-mon[37330]: 2.1b scrub starts 2024-09-11T07:37:43.420 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:43 smithi022 ceph-mon[37330]: 2.1b scrub ok 2024-09-11T07:37:43.420 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:43 smithi022 ceph-mon[37330]: 6.16 scrub starts 2024-09-11T07:37:43.420 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:43 smithi022 ceph-mon[37330]: 6.16 scrub ok 2024-09-11T07:37:43.420 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:43 smithi022 ceph-mon[37330]: pgmap v502: 148 pgs: 148 active+clean; 419 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 47 MiB/s wr, 749 op/s 2024-09-11T07:37:43.420 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:43 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/308496175' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:37:43.420 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:43 smithi022 ceph-mon[37330]: from='client.25411 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:37:43.420 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:43 smithi022 ceph-mon[37330]: 5.19 scrub starts 2024-09-11T07:37:43.420 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:43 smithi022 ceph-mon[37330]: 5.19 scrub ok 2024-09-11T07:37:43.469 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:43.473 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 6 16 18094 18078 188.296 158.688 0.00693783 0.00530385 2024-09-11T07:37:43.502 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:43.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:43.503 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:43.604 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:43.645 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:43.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:43.647 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:43.747 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:43.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:43 smithi104 ceph-mon[33507]: 2.1b scrub starts 2024-09-11T07:37:43.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:43 smithi104 ceph-mon[33507]: 2.1b scrub ok 2024-09-11T07:37:43.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:43 smithi104 ceph-mon[33507]: 6.16 scrub starts 2024-09-11T07:37:43.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:43 smithi104 ceph-mon[33507]: 6.16 scrub ok 2024-09-11T07:37:43.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:43 smithi104 ceph-mon[33507]: pgmap v502: 148 pgs: 148 active+clean; 419 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 47 MiB/s wr, 749 op/s 2024-09-11T07:37:43.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:43 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/308496175' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:37:43.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:43 smithi104 ceph-mon[33507]: from='client.25411 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:37:43.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:43 smithi104 ceph-mon[33507]: 5.19 scrub starts 2024-09-11T07:37:43.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:43 smithi104 ceph-mon[33507]: 5.19 scrub ok 2024-09-11T07:37:43.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:43.784 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:43.785 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:43.885 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:43.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:43.923 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:43.923 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:44.024 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:44.073 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:44.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:44.074 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:44.177 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:44.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:44.223 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:44.223 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:44.324 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:44.356 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:44.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:44.357 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:44.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:44 smithi022 ceph-mon[29462]: 4.1d scrub starts 2024-09-11T07:37:44.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:44 smithi022 ceph-mon[29462]: 4.1d scrub ok 2024-09-11T07:37:44.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:44 smithi022 ceph-mon[29462]: 5.d scrub starts 2024-09-11T07:37:44.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:44 smithi022 ceph-mon[29462]: 5.d scrub ok 2024-09-11T07:37:44.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:44 smithi022 ceph-mon[29462]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:37:44.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:44 smithi022 ceph-mon[29462]: from='client.25411 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:37:44.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:44 smithi022 ceph-mon[29462]: osdmap e255: 8 total, 8 up, 8 in 2024-09-11T07:37:44.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:44 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:37:44.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:44 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:37:44.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:44 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:37:44.420 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:44 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:37:44.420 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:44 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:37:44.420 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:44 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:37:44.420 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:44 smithi022 ceph-mon[37330]: 4.1d scrub starts 2024-09-11T07:37:44.421 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:44 smithi022 ceph-mon[37330]: 4.1d scrub ok 2024-09-11T07:37:44.421 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:44 smithi022 ceph-mon[37330]: 5.d scrub starts 2024-09-11T07:37:44.421 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:44 smithi022 ceph-mon[37330]: 5.d scrub ok 2024-09-11T07:37:44.421 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:44 smithi022 ceph-mon[37330]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:37:44.421 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:44 smithi022 ceph-mon[37330]: from='client.25411 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:37:44.421 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:44 smithi022 ceph-mon[37330]: osdmap e255: 8 total, 8 up, 8 in 2024-09-11T07:37:44.422 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:44 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:37:44.422 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:44 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:37:44.422 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:44 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:37:44.422 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:44 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:37:44.422 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:44 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:37:44.422 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:44 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:37:44.458 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:44.473 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 7 16 20665 20649 184.35 160.688 0.00328803 0.00541694 2024-09-11T07:37:44.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:44.497 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:44.497 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:44.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:37:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:37:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:37:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:44 smithi104 ceph-mon[33507]: 4.1d scrub starts 2024-09-11T07:37:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:44 smithi104 ceph-mon[33507]: 4.1d scrub ok 2024-09-11T07:37:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:44 smithi104 ceph-mon[33507]: 5.d scrub starts 2024-09-11T07:37:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:44 smithi104 ceph-mon[33507]: 5.d scrub ok 2024-09-11T07:37:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:44 smithi104 ceph-mon[33507]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:37:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:44 smithi104 ceph-mon[33507]: from='client.25411 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:37:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:44 smithi104 ceph-mon[33507]: osdmap e255: 8 total, 8 up, 8 in 2024-09-11T07:37:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:44 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:37:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:44 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:37:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:44 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:37:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:44 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:37:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:44 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:37:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:44 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:37:44.598 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:44.641 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:44.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:44.642 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:44.743 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:44.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:44.778 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:44.778 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:44.879 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:44.911 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:44.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:44.913 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:45.014 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:45.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:45.046 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:45.046 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:45.141 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-11T07:37:45.146 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:45.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:45.242 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:45.242 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:45.343 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:45.372 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:45.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:45.374 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:45.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:45 smithi022 ceph-mon[29462]: pgmap v504: 148 pgs: 148 active+clean; 643 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-11T07:37:45.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:45 smithi022 ceph-mon[29462]: Adjusting osd_memory_target on smithi104 to 3248M 2024-09-11T07:37:45.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:45 smithi022 ceph-mon[37330]: pgmap v504: 148 pgs: 148 active+clean; 643 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-11T07:37:45.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:45 smithi022 ceph-mon[37330]: Adjusting osd_memory_target on smithi104 to 3248M 2024-09-11T07:37:45.473 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 8 16 23155 23139 180.758 155.625 0.00561626 0.00552571 2024-09-11T07:37:45.475 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:45.508 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:45.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:45.509 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:45.610 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:45.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:45.641 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:45.642 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:45.742 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:45.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:45.777 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:45.777 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:45.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:45 smithi104 ceph-mon[33507]: pgmap v504: 148 pgs: 148 active+clean; 643 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-11T07:37:45.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:45 smithi104 ceph-mon[33507]: Adjusting osd_memory_target on smithi104 to 3248M 2024-09-11T07:37:45.878 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:45.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:45.910 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:45.910 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:46.011 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:46.044 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:46.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:46.045 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:46.146 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:46.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:46.180 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:46.180 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:46.281 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:46.314 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:46.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:46.315 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:46.415 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:46.444 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:46.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:46.446 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:46.473 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 9 16 25677 25661 178.186 157.625 0.00492342 0.00560576 2024-09-11T07:37:46.546 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:46.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:46.592 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:46.592 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:46.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:46 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:37:46.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:46 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:37:46.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:46 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:37:46.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:46 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:37:46.661 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:46 smithi022 ceph-mon[29462]: Adjusting osd_memory_target on smithi022 to 3249M 2024-09-11T07:37:46.661 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:46 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:37:46.661 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:46 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:37:46.661 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:46 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:37:46.661 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:46 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:37:46.661 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:46 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:37:46.661 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:46 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:37:46.661 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:46 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:37:46.661 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:46 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:37:46.662 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:46 smithi022 ceph-mon[37330]: Adjusting osd_memory_target on smithi022 to 3249M 2024-09-11T07:37:46.662 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:46 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:37:46.662 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:46 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:37:46.662 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:46 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:37:46.662 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:46 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:37:46.693 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:46.727 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:46.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:46.728 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:46.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:46 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:37:46.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:46 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:37:46.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:46 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:37:46.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:46 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:37:46.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:46 smithi104 ceph-mon[33507]: Adjusting osd_memory_target on smithi022 to 3249M 2024-09-11T07:37:46.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:46 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:37:46.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:46 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:37:46.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:46 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:37:46.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:46 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:37:46.829 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:46.870 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:46.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:46.871 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:46.972 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:46.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:46.997 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:46.997 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:47.098 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:47.135 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:47.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:47.137 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:47.237 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:47.278 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:47.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:47.279 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:47.380 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:47.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:47.412 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:47.412 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:47.439 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:37:47.473 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 10 16 27939 27923 174.504 141.375 0.00834731 0.00572404 2024-09-11T07:37:47.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:47 smithi022 ceph-mon[29462]: pgmap v505: 148 pgs: 148 active+clean; 1.3 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 152 MiB/s wr, 2.42k op/s 2024-09-11T07:37:47.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:47 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2541588813' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:37:47.484 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:47 smithi022 ceph-mon[37330]: pgmap v505: 148 pgs: 148 active+clean; 1.3 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 152 MiB/s wr, 2.42k op/s 2024-09-11T07:37:47.484 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:47 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2541588813' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:37:47.513 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:47.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:47.555 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:47.555 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:47.573 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:37:47.573 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:37:47.573 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:37:47.574 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:37:47.574 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:37:47.574 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:37:47.574 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:37:47.574 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:37:47.574 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:37:47.574 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:37:47.574 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:37:47.574 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:37:47.574 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:37:47.574 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:37:47.574 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:37:47.575 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:37:47.656 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:47.688 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:47.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:47.689 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:47.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:47 smithi104 ceph-mon[33507]: pgmap v505: 148 pgs: 148 active+clean; 1.3 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 152 MiB/s wr, 2.42k op/s 2024-09-11T07:37:47.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:47 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2541588813' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:37:47.790 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:47.821 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:47.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:47.823 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:47.924 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T07:37:47.924 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:47.967 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:47.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:47.968 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:48.068 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:48.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:48.118 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:48.119 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:48.219 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:48.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:48.251 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:48.251 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:48.352 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:48.385 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:48.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:48.386 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:48.474 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 11 16 30214 30198 171.56 142.188 0.0108085 0.00582341 2024-09-11T07:37:48.487 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:48.525 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:48.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:48.526 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:48.627 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:48.656 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:48.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:48.657 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:48.758 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:48.797 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:48.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:48.799 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:48.899 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:48.932 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:48.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:48.933 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:49.034 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:49.056 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:49.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:49.057 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:49.157 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:49.190 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:49.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:49.191 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:49.292 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:49.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:49.320 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:49.320 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:49.421 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:49.474 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 12 16 31849 31833 165.775 102.188 0.00537589 0.006027 2024-09-11T07:37:49.484 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:49.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:49.486 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:49.586 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:49.588 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:49 smithi022 ceph-mon[29462]: pgmap v506: 148 pgs: 148 active+clean; 1.3 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 130 MiB/s wr, 2.08k op/s 2024-09-11T07:37:49.589 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:49 smithi022 ceph-mon[37330]: pgmap v506: 148 pgs: 148 active+clean; 1.3 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 130 MiB/s wr, 2.08k op/s 2024-09-11T07:37:49.598 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:49 smithi104 ceph-mon[33507]: pgmap v506: 148 pgs: 148 active+clean; 1.3 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 130 MiB/s wr, 2.08k op/s 2024-09-11T07:37:49.622 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:49.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:49.623 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:49.724 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:49.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:49.762 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:49.762 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:49.863 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:49.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:49.897 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:49.897 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:49.997 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:50.025 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:50.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:50.026 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:50.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:37:49 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:37:50.128 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:50.161 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:50.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:50.162 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:50.263 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:50.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:50.288 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:50.288 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:50.390 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:50.424 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:50.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:50.425 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:50.452 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:37:50.474 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 13 16 33550 33534 161.201 106.312 0.00402424 0.00619775 2024-09-11T07:37:50.526 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:50.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:50.570 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:50.570 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:50.632 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:37:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:37:50] "GET /metrics HTTP/1.1" 200 223998 "" "Prometheus/2.43.0" 2024-09-11T07:37:50.671 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:50.693 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:50.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:50.694 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:50.795 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:50.842 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:50.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:50.843 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:50.944 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:50.983 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:50.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:50.984 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:51.085 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:51.118 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:51.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:51.119 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:51.220 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:51.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:51.250 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:51.250 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:51.351 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:51.389 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:51.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:51.390 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:51.474 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 14 16 35361 35345 157.771 113.188 0.00360461 0.00633245 2024-09-11T07:37:51.492 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:51.528 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:51.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:51.529 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:51.630 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:51.662 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:51.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:51.663 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:51.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:51 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:51.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:51 smithi022 ceph-mon[29462]: pgmap v507: 148 pgs: 148 active+clean; 2.0 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 158 MiB/s wr, 2.53k op/s 2024-09-11T07:37:51.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:51 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:51.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:51 smithi022 ceph-mon[37330]: pgmap v507: 148 pgs: 148 active+clean; 2.0 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 158 MiB/s wr, 2.53k op/s 2024-09-11T07:37:51.765 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:51.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:51 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:37:51.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:51 smithi104 ceph-mon[33507]: pgmap v507: 148 pgs: 148 active+clean; 2.0 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 158 MiB/s wr, 2.53k op/s 2024-09-11T07:37:51.792 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:51.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:51.793 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:51.894 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:51.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:51.937 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:51.937 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:52.038 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:52.060 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:52.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:52.061 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:52.162 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:52.201 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:52.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:52.202 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:52.303 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:52.324 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:52.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:52.325 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:52.426 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:52.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:52.458 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:52.459 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:52.474 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 15 16 36962 36946 153.923 100.062 0.00883885 0.00649108 2024-09-11T07:37:52.559 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:52.583 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:52.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:52.584 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:52.685 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:52.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:52.718 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:52.719 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:52.819 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:52.851 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:52.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:52.853 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:52.954 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:52.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:52.987 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:52.987 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:53.088 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:53.122 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:53.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:53.123 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:53.224 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:53.263 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:53.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:53.264 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:53.365 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:53.405 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:53.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:53.407 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:53.474 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 16 16 38592 38576 150.669 101.875 0.0778467 0.00662462 2024-09-11T07:37:53.508 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:53.547 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:53.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:53.548 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:53.649 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:53.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:53 smithi022 ceph-mon[29462]: pgmap v508: 148 pgs: 148 active+clean; 2.0 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 158 MiB/s wr, 2.53k op/s 2024-09-11T07:37:53.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:37:53.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:53.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:53 smithi022 ceph-mon[37330]: pgmap v508: 148 pgs: 148 active+clean; 2.0 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 158 MiB/s wr, 2.53k op/s 2024-09-11T07:37:53.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:37:53.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:53.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:53.677 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:53.677 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:53.778 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:53.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:53 smithi104 ceph-mon[33507]: pgmap v508: 148 pgs: 148 active+clean; 2.0 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 158 MiB/s wr, 2.53k op/s 2024-09-11T07:37:53.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:37:53.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:37:53.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:53.829 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:53.829 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:53.930 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:53.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:53.960 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:53.961 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:54.061 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:54.093 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:54.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:54.094 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:54.195 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:54.226 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:54.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:54.227 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:54.328 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:54.362 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:54.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:54.363 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:54.464 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:54.477 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 17 16 40530 40514 148.911 121.125 0.00669545 0.00670758 2024-09-11T07:37:54.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:54.493 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:54.493 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:54.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:37:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:37:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:37:54.594 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:54.637 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:54.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:54.638 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:54.739 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:54.777 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:54.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:54.778 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:54.878 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:54.917 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:54.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:54.918 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:55.019 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:55.065 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:55.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:55.066 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:55.166 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:55.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:55.203 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:55.203 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:55.263 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:37:55 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=sqlstore.transactions t=2024-09-11T07:37:55.05832754Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T07:37:55.263 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:37:55 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=cleanup t=2024-09-11T07:37:55.060658667Z level=info msg="Completed cleanup jobs" duration=12.661247ms 2024-09-11T07:37:55.304 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:55.332 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:55.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:55.333 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:55.434 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:55.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:55.467 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:55.467 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:55.477 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 18 16 42352 42336 146.965 113.875 0.00653544 0.00679905 2024-09-11T07:37:55.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:55 smithi104 ceph-mon[33507]: pgmap v509: 148 pgs: 148 active+clean; 2.1 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 941 B/s rd, 157 MiB/s wr, 2.52k op/s 2024-09-11T07:37:55.558 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:55 smithi022 ceph-mon[29462]: pgmap v509: 148 pgs: 148 active+clean; 2.1 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 941 B/s rd, 157 MiB/s wr, 2.52k op/s 2024-09-11T07:37:55.559 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:55 smithi022 ceph-mon[37330]: pgmap v509: 148 pgs: 148 active+clean; 2.1 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 941 B/s rd, 157 MiB/s wr, 2.52k op/s 2024-09-11T07:37:55.568 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:55.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:55.626 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:55.626 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:55.727 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:55.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:55.768 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:55.768 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:55.869 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:55.899 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:55.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:55.900 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:56.001 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:56.045 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:56.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:56.046 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:56.147 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:56.147 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:37:56.171 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:56.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:56.172 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:56.205 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:37:56.206 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:37:56.206 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 20, 2024-09-11T07:37:56.206 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 600, 2024-09-11T07:37:56.206 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:37:56.206 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.206 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120660 7.7 e127/92 7:e4ae0ab7:::benchmark_data_smithi022_77720_object120659:head v 127'7598, mlcod=127'7596)", 2024-09-11T07:37:56.206 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.047036+0000", 2024-09-11T07:37:56.206 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 447.15111616500002, 2024-09-11T07:37:56.206 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20266092499999999, 2024-09-11T07:37:56.206 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:37:56.206 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:56.207 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:37:56.207 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.207 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:37:56.207 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.047036+0000", 2024-09-11T07:37:56.207 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.207 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.207 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.207 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:37:56.207 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.047036+0000", 2024-09-11T07:37:56.207 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.207 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.207 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.207 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:37:56.208 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.047035+0000", 2024-09-11T07:37:56.208 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:56.208 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.208 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.208 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:37:56.208 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.047080+0000", 2024-09-11T07:37:56.208 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5188999999999997e-05 2024-09-11T07:37:56.208 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.208 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.208 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:37:56.208 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.047088+0000", 2024-09-11T07:37:56.208 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.9419999999999995e-06 2024-09-11T07:37:56.208 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.209 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.209 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:37:56.209 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.047092+0000", 2024-09-11T07:37:56.209 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8419999999999998e-06 2024-09-11T07:37:56.209 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.209 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.209 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.209 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.047125+0000", 2024-09-11T07:37:56.209 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3547000000000003e-05 2024-09-11T07:37:56.209 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.209 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.209 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:37:56.209 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.047140+0000", 2024-09-11T07:37:56.209 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4297999999999999e-05 2024-09-11T07:37:56.210 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.210 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.210 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:37:56.210 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.249655+0000", 2024-09-11T07:37:56.210 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.20251532999999999 2024-09-11T07:37:56.210 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.210 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.210 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:37:56.210 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.249697+0000", 2024-09-11T07:37:56.210 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2027999999999998e-05 2024-09-11T07:37:56.210 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.210 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:37:56.210 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.211 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.211 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.211 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120664 7.5 e127/92 7:a3fec1e5:::benchmark_data_smithi022_77720_object120663:head v 127'7704, mlcod=127'7702)", 2024-09-11T07:37:56.211 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.047523+0000", 2024-09-11T07:37:56.211 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 447.150629348, 2024-09-11T07:37:56.211 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.200962582, 2024-09-11T07:37:56.211 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:37:56.211 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:56.211 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:37:56.211 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.211 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:37:56.211 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.047523+0000", 2024-09-11T07:37:56.211 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.212 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.212 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.212 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:37:56.212 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.047523+0000", 2024-09-11T07:37:56.212 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.212 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.212 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.212 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:37:56.212 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.047522+0000", 2024-09-11T07:37:56.212 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:56.212 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.212 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.212 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:37:56.212 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.047615+0000", 2024-09-11T07:37:56.213 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.3702999999999994e-05 2024-09-11T07:37:56.213 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.213 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.213 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:37:56.213 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.047625+0000", 2024-09-11T07:37:56.213 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.8120000000000002e-06 2024-09-11T07:37:56.213 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.213 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.213 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:37:56.213 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.047630+0000", 2024-09-11T07:37:56.213 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.0139999999999998e-06 2024-09-11T07:37:56.213 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.213 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.214 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.214 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.245656+0000", 2024-09-11T07:37:56.214 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.19802588099999999 2024-09-11T07:37:56.214 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.214 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.214 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:37:56.214 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.245694+0000", 2024-09-11T07:37:56.214 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8216000000000001e-05 2024-09-11T07:37:56.214 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.214 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.214 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:37:56.215 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.248442+0000", 2024-09-11T07:37:56.215 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0027474750000000001 2024-09-11T07:37:56.215 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.215 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.215 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:37:56.215 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.248485+0000", 2024-09-11T07:37:56.215 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3826999999999999e-05 2024-09-11T07:37:56.215 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.215 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:37:56.215 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.215 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.215 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.215 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.24778.0:120662 7.7 e127/92 7:edc0624b:::benchmark_data_smithi022_77720_object120661:head v 127'7599, mlcod=127'7596)", 2024-09-11T07:37:56.215 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:30:29.050417+0000", 2024-09-11T07:37:56.216 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 447.14773553600003, 2024-09-11T07:37:56.216 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.19994145499999999, 2024-09-11T07:37:56.216 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:37:56.216 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:56.216 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:37:56.216 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.216 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:37:56.216 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050417+0000", 2024-09-11T07:37:56.216 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.216 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.216 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.216 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:37:56.216 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050417+0000", 2024-09-11T07:37:56.216 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.217 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.217 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.217 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:37:56.217 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050415+0000", 2024-09-11T07:37:56.217 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:56.217 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.217 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.217 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:37:56.217 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.050512+0000", 2024-09-11T07:37:56.217 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.7014000000000003e-05 2024-09-11T07:37:56.217 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.217 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.217 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:37:56.218 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246090+0000", 2024-09-11T07:37:56.218 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.19557713600000001 2024-09-11T07:37:56.218 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.218 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.218 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:37:56.218 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246099+0000", 2024-09-11T07:37:56.218 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.5310000000000007e-06 2024-09-11T07:37:56.218 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.218 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.218 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.218 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246543+0000", 2024-09-11T07:37:56.218 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00044386700000000002 2024-09-11T07:37:56.218 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.218 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.219 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:37:56.219 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.246564+0000", 2024-09-11T07:37:56.219 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1358000000000001e-05 2024-09-11T07:37:56.219 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.219 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.219 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:37:56.219 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.250325+0000", 2024-09-11T07:37:56.219 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0037608369999999999 2024-09-11T07:37:56.219 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.219 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.219 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:37:56.219 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:30:29.250358+0000", 2024-09-11T07:37:56.219 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2997999999999998e-05 2024-09-11T07:37:56.219 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.220 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:37:56.220 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.220 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.220 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.220 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:155168 7.6 7:61ee2fa0:::benchmark_data_smithi022_77720_object31413:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-11T07:37:56.220 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:31:02.178228+0000", 2024-09-11T07:37:56.220 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 414.01992463300002, 2024-09-11T07:37:56.220 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.045960706, 2024-09-11T07:37:56.220 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:37:56.220 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:37:56.220 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:37:56.220 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:37:56.220 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:37:56.221 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 155168 2024-09-11T07:37:56.221 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.221 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:37:56.221 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.221 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:37:56.221 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.178228+0000", 2024-09-11T07:37:56.221 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.221 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.221 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.221 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:37:56.221 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.178228+0000", 2024-09-11T07:37:56.221 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.221 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.221 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.222 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:37:56.222 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.178226+0000", 2024-09-11T07:37:56.222 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:56.222 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.222 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.222 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:37:56.222 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.178232+0000", 2024-09-11T07:37:56.222 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1989999999999997e-06 2024-09-11T07:37:56.222 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.222 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.222 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:37:56.222 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.178269+0000", 2024-09-11T07:37:56.222 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7027000000000003e-05 2024-09-11T07:37:56.222 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.223 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.223 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:37:56.223 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.178275+0000", 2024-09-11T07:37:56.223 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.545e-06 2024-09-11T07:37:56.223 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.223 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.223 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.223 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.189545+0000", 2024-09-11T07:37:56.223 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.011269926 2024-09-11T07:37:56.223 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.223 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.223 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:37:56.223 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.189656+0000", 2024-09-11T07:37:56.223 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000110881 2024-09-11T07:37:56.224 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.224 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.224 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:37:56.224 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.189768+0000", 2024-09-11T07:37:56.224 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000111753 2024-09-11T07:37:56.224 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.224 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.224 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.224 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.212902+0000", 2024-09-11T07:37:56.224 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.023134310000000002 2024-09-11T07:37:56.224 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.224 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.224 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:37:56.224 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.212910+0000", 2024-09-11T07:37:56.225 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.1750000000000005e-06 2024-09-11T07:37:56.225 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.225 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.225 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.225 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.223890+0000", 2024-09-11T07:37:56.225 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0109797270000001 2024-09-11T07:37:56.225 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.225 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.225 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:37:56.225 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.224188+0000", 2024-09-11T07:37:56.225 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00029837800000000002 2024-09-11T07:37:56.225 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.225 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:37:56.226 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.226 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.226 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.226 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:155187 7.6 7:66f58159:::benchmark_data_smithi022_77720_object31432:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-11T07:37:56.226 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:31:02.215571+0000", 2024-09-11T07:37:56.226 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 413.98258074500001, 2024-09-11T07:37:56.226 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0231388610000001, 2024-09-11T07:37:56.226 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:37:56.226 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:56.226 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:37:56.226 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:37:56.226 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:37:56.226 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 155187 2024-09-11T07:37:56.227 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.227 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:37:56.227 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.227 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:37:56.227 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.215571+0000", 2024-09-11T07:37:56.227 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.227 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.227 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.227 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:37:56.227 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.215571+0000", 2024-09-11T07:37:56.227 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.227 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.227 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.228 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:37:56.228 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.215570+0000", 2024-09-11T07:37:56.228 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:37:56.228 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.228 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.228 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:37:56.228 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.215576+0000", 2024-09-11T07:37:56.228 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9229999999999999e-06 2024-09-11T07:37:56.228 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.228 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.228 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:37:56.228 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.215584+0000", 2024-09-11T07:37:56.228 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.5290000000000008e-06 2024-09-11T07:37:56.228 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.229 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.229 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:37:56.229 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.215590+0000", 2024-09-11T07:37:56.229 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.5709999999999997e-06 2024-09-11T07:37:56.229 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.229 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.229 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.229 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.215669+0000", 2024-09-11T07:37:56.229 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.9553000000000002e-05 2024-09-11T07:37:56.229 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.229 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.229 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:37:56.229 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.215675+0000", 2024-09-11T07:37:56.229 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1340000000000001e-06 2024-09-11T07:37:56.230 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.230 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.230 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.230 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.224218+0000", 2024-09-11T07:37:56.230 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0085421480000001 2024-09-11T07:37:56.230 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.230 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.230 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:37:56.230 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.224373+0000", 2024-09-11T07:37:56.230 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000155296 2024-09-11T07:37:56.230 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.230 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.230 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3", 2024-09-11T07:37:56.230 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.224524+0000", 2024-09-11T07:37:56.231 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000151426 2024-09-11T07:37:56.231 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.231 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.231 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:37:56.231 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.232366+0000", 2024-09-11T07:37:56.231 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0078421169999999991 2024-09-11T07:37:56.231 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.231 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.231 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:56.231 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.238570+0000", 2024-09-11T07:37:56.231 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0062034220000000001 2024-09-11T07:37:56.231 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.231 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.231 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:37:56.232 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.238619+0000", 2024-09-11T07:37:56.232 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9261000000000002e-05 2024-09-11T07:37:56.232 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.232 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.232 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:37:56.232 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.238710+0000", 2024-09-11T07:37:56.232 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.1223000000000004e-05 2024-09-11T07:37:56.232 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.232 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:37:56.232 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.232 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.232 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.232 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:155191 7.6 7:6429dde4:::benchmark_data_smithi022_77720_object31436:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-11T07:37:56.232 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:31:02.215623+0000", 2024-09-11T07:37:56.233 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 413.98252968700001, 2024-09-11T07:37:56.233 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.025335731, 2024-09-11T07:37:56.233 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:37:56.233 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:56.233 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:37:56.233 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:37:56.233 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:37:56.233 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 155191 2024-09-11T07:37:56.233 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.233 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:37:56.233 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.233 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:37:56.233 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.215623+0000", 2024-09-11T07:37:56.233 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.234 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.234 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.234 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:37:56.234 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.215623+0000", 2024-09-11T07:37:56.234 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.234 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.234 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.234 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:37:56.235 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.215622+0000", 2024-09-11T07:37:56.235 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:37:56.235 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.235 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.235 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:37:56.235 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.215626+0000", 2024-09-11T07:37:56.235 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5930000000000002e-06 2024-09-11T07:37:56.235 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.235 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.235 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:37:56.235 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.215633+0000", 2024-09-11T07:37:56.236 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9990000000000002e-06 2024-09-11T07:37:56.236 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.236 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.236 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:37:56.236 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.215637+0000", 2024-09-11T07:37:56.236 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9770000000000002e-06 2024-09-11T07:37:56.236 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.236 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.236 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.236 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.215747+0000", 2024-09-11T07:37:56.236 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000110132 2024-09-11T07:37:56.236 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.236 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.237 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:37:56.237 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.215753+0000", 2024-09-11T07:37:56.237 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3559999999999998e-06 2024-09-11T07:37:56.237 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.237 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.237 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.237 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.227329+0000", 2024-09-11T07:37:56.237 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.011575541 2024-09-11T07:37:56.237 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.237 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.237 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:37:56.237 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.227425+0000", 2024-09-11T07:37:56.237 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.6819999999999998e-05 2024-09-11T07:37:56.237 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.238 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.238 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3", 2024-09-11T07:37:56.238 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.228257+0000", 2024-09-11T07:37:56.238 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00083145099999999998 2024-09-11T07:37:56.238 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.238 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.238 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:37:56.238 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.240631+0000", 2024-09-11T07:37:56.238 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.012373996 2024-09-11T07:37:56.238 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.238 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.238 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:56.238 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.240877+0000", 2024-09-11T07:37:56.238 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000246164 2024-09-11T07:37:56.238 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.239 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.239 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:37:56.239 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.240915+0000", 2024-09-11T07:37:56.239 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7722000000000002e-05 2024-09-11T07:37:56.239 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.239 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.239 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:37:56.239 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.240958+0000", 2024-09-11T07:37:56.239 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3550000000000001e-05 2024-09-11T07:37:56.239 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.239 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:37:56.239 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.239 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.240 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.240 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:155197 7.f 7:f79f8502:::benchmark_data_smithi022_77720_object31442:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-11T07:37:56.240 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:31:02.220893+0000", 2024-09-11T07:37:56.240 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 413.97725876999999, 2024-09-11T07:37:56.240 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0070183349999999, 2024-09-11T07:37:56.240 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:37:56.240 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:56.240 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:37:56.240 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:37:56.240 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:37:56.240 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 155197 2024-09-11T07:37:56.240 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.241 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:37:56.241 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.241 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:37:56.241 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.220893+0000", 2024-09-11T07:37:56.241 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.241 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.241 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.241 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:37:56.241 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.220893+0000", 2024-09-11T07:37:56.241 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.241 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.241 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.241 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:37:56.241 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.220892+0000", 2024-09-11T07:37:56.242 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:37:56.242 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.242 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.242 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:37:56.242 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.220897+0000", 2024-09-11T07:37:56.242 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.0760000000000002e-06 2024-09-11T07:37:56.242 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.242 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.242 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:37:56.242 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.220906+0000", 2024-09-11T07:37:56.242 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.2239999999999995e-06 2024-09-11T07:37:56.242 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.242 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.243 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:37:56.243 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.220911+0000", 2024-09-11T07:37:56.243 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2279999999999998e-06 2024-09-11T07:37:56.243 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.243 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.243 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.243 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.220967+0000", 2024-09-11T07:37:56.243 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6687999999999997e-05 2024-09-11T07:37:56.243 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.243 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.243 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:37:56.243 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.220974+0000", 2024-09-11T07:37:56.243 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4300000000000003e-06 2024-09-11T07:37:56.243 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.244 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.244 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.244 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.220660+0000", 2024-09-11T07:37:56.244 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.99968644600000001 2024-09-11T07:37:56.244 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.244 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.244 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:37:56.244 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.220818+0000", 2024-09-11T07:37:56.244 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000157577 2024-09-11T07:37:56.244 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.244 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.244 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3", 2024-09-11T07:37:56.244 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.220965+0000", 2024-09-11T07:37:56.244 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014718500000000001 2024-09-11T07:37:56.245 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.245 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.245 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:37:56.245 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.227095+0000", 2024-09-11T07:37:56.245 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0061301819999999996 2024-09-11T07:37:56.245 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.245 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.245 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:56.245 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.227779+0000", 2024-09-11T07:37:56.245 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00068379200000000002 2024-09-11T07:37:56.245 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.245 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.245 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:37:56.245 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.227829+0000", 2024-09-11T07:37:56.246 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.0216999999999997e-05 2024-09-11T07:37:56.246 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.246 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.246 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:37:56.246 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.227912+0000", 2024-09-11T07:37:56.246 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.2458000000000002e-05 2024-09-11T07:37:56.246 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.246 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:37:56.246 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.246 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.246 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.246 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:155200 7.f 7:f6f49d85:::benchmark_data_smithi022_77720_object31445:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-11T07:37:56.246 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:31:02.224246+0000", 2024-09-11T07:37:56.246 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 413.973906709, 2024-09-11T07:37:56.247 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.005115016, 2024-09-11T07:37:56.247 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:37:56.247 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:56.247 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:37:56.247 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:37:56.247 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:37:56.247 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 155200 2024-09-11T07:37:56.247 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.247 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:37:56.247 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.247 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:37:56.247 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.224246+0000", 2024-09-11T07:37:56.247 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.247 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.248 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.248 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:37:56.248 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.224246+0000", 2024-09-11T07:37:56.248 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.248 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.248 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.248 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:37:56.248 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.224243+0000", 2024-09-11T07:37:56.248 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:37:56.248 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.248 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.248 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:37:56.248 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.224252+0000", 2024-09-11T07:37:56.248 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.1650000000000006e-06 2024-09-11T07:37:56.249 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.249 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.249 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:37:56.249 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.224263+0000", 2024-09-11T07:37:56.249 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1208e-05 2024-09-11T07:37:56.249 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.249 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.249 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:37:56.249 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.224269+0000", 2024-09-11T07:37:56.249 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1689999999999997e-06 2024-09-11T07:37:56.249 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.249 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.250 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.250 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.224326+0000", 2024-09-11T07:37:56.250 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7294999999999999e-05 2024-09-11T07:37:56.250 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.250 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.250 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:37:56.250 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:02.224335+0000", 2024-09-11T07:37:56.250 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.9220000000000003e-06 2024-09-11T07:37:56.250 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.250 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.250 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.250 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.224131+0000", 2024-09-11T07:37:56.250 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.99979618400000003 2024-09-11T07:37:56.250 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.251 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.251 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:37:56.251 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.224232+0000", 2024-09-11T07:37:56.251 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000100807 2024-09-11T07:37:56.251 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.251 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.251 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3", 2024-09-11T07:37:56.251 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.224341+0000", 2024-09-11T07:37:56.251 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000108658 2024-09-11T07:37:56.251 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.251 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.251 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:37:56.251 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.227944+0000", 2024-09-11T07:37:56.251 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.003602788 2024-09-11T07:37:56.252 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.252 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.252 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:56.252 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.229232+0000", 2024-09-11T07:37:56.252 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001287961 2024-09-11T07:37:56.252 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.252 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.252 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:37:56.252 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.229278+0000", 2024-09-11T07:37:56.252 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6462999999999999e-05 2024-09-11T07:37:56.252 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.252 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.252 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:37:56.252 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:31:03.229361+0000", 2024-09-11T07:37:56.253 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.2527000000000005e-05 2024-09-11T07:37:56.253 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.253 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:37:56.253 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.253 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.253 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.253 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:213455 7.f 7:f4abd973:::benchmark_data_smithi022_77720_object89700:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-11T07:37:56.253 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:00.080874+0000", 2024-09-11T07:37:56.253 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 356.11727858099999, 2024-09-11T07:37:56.253 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9603731720000002, 2024-09-11T07:37:56.253 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:37:56.253 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:37:56.253 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:37:56.254 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:37:56.254 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:37:56.254 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213455 2024-09-11T07:37:56.254 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.254 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:37:56.254 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.254 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:37:56.254 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.080874+0000", 2024-09-11T07:37:56.254 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.254 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.254 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.254 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:37:56.254 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.080874+0000", 2024-09-11T07:37:56.254 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.255 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.255 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.255 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:37:56.255 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.080871+0000", 2024-09-11T07:37:56.255 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:37:56.255 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.255 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.255 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:37:56.255 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.080880+0000", 2024-09-11T07:37:56.255 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.3540000000000006e-06 2024-09-11T07:37:56.255 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.255 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.255 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:37:56.255 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.080894+0000", 2024-09-11T07:37:56.256 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3073e-05 2024-09-11T07:37:56.256 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.256 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.256 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:37:56.256 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.080901+0000", 2024-09-11T07:37:56.256 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0609999999999998e-06 2024-09-11T07:37:56.256 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.256 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.256 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.256 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.080993+0000", 2024-09-11T07:37:56.256 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.2237000000000005e-05 2024-09-11T07:37:56.256 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.256 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.256 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:37:56.257 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.081121+0000", 2024-09-11T07:37:56.257 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012860000000000001 2024-09-11T07:37:56.257 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.257 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.257 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3", 2024-09-11T07:37:56.257 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.081242+0000", 2024-09-11T07:37:56.257 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000120677 2024-09-11T07:37:56.257 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.257 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.258 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:37:56.258 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.099147+0000", 2024-09-11T07:37:56.258 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.017904863 2024-09-11T07:37:56.258 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.258 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.258 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.258 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.104850+0000", 2024-09-11T07:37:56.258 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0057033930000000002 2024-09-11T07:37:56.258 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.258 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.258 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:37:56.258 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.104859+0000", 2024-09-11T07:37:56.258 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.0180000000000007e-06 2024-09-11T07:37:56.259 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.259 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.259 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.259 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502755+0000", 2024-09-11T07:37:56.259 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.39789565399999999 2024-09-11T07:37:56.259 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.259 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.259 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:37:56.259 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502764+0000", 2024-09-11T07:37:56.259 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.9460000000000004e-06 2024-09-11T07:37:56.259 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.259 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.259 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.259 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:01.995532+0000", 2024-09-11T07:37:56.260 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.492767924 2024-09-11T07:37:56.260 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.260 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.260 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for missing object", 2024-09-11T07:37:56.260 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:01.995669+0000", 2024-09-11T07:37:56.260 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013712899999999999 2024-09-11T07:37:56.260 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.260 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.260 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.260 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.041144+0000", 2024-09-11T07:37:56.260 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.045474964 2024-09-11T07:37:56.260 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.260 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.261 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:37:56.261 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.041247+0000", 2024-09-11T07:37:56.261 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000102835 2024-09-11T07:37:56.261 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.261 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:37:56.261 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.261 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.261 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.261 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:213456 7.f 7:fe3cd8b0:::benchmark_data_smithi022_77720_object89701:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-11T07:37:56.261 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:00.087849+0000", 2024-09-11T07:37:56.261 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 356.11030281799998, 2024-09-11T07:37:56.261 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9475085029999999, 2024-09-11T07:37:56.261 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:37:56.261 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:56.262 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:37:56.262 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:37:56.262 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:37:56.262 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213456 2024-09-11T07:37:56.262 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.262 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:37:56.262 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.262 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:37:56.262 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.087849+0000", 2024-09-11T07:37:56.262 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.262 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.262 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.262 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:37:56.263 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.087849+0000", 2024-09-11T07:37:56.263 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.263 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.263 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.263 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:37:56.263 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.087847+0000", 2024-09-11T07:37:56.263 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:37:56.263 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.263 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.263 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:37:56.263 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.087855+0000", 2024-09-11T07:37:56.263 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.2199999999999992e-06 2024-09-11T07:37:56.263 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.264 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.264 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:37:56.264 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.087868+0000", 2024-09-11T07:37:56.264 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2447e-05 2024-09-11T07:37:56.264 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.264 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.264 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:37:56.264 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.087875+0000", 2024-09-11T07:37:56.264 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.759e-06 2024-09-11T07:37:56.264 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.264 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.264 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.264 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.107015+0000", 2024-09-11T07:37:56.265 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.019140834999999998 2024-09-11T07:37:56.265 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.265 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.268 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:37:56.268 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.107025+0000", 2024-09-11T07:37:56.268 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.064e-06 2024-09-11T07:37:56.268 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.268 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.268 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.268 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502790+0000", 2024-09-11T07:37:56.268 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.39576567200000001 2024-09-11T07:37:56.268 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.268 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.269 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:37:56.269 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502797+0000", 2024-09-11T07:37:56.269 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1439999999999997e-06 2024-09-11T07:37:56.269 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.269 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.269 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.269 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:01.995707+0000", 2024-09-11T07:37:56.269 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4929092989999999 2024-09-11T07:37:56.269 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.269 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.269 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:37:56.269 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:01.995811+0000", 2024-09-11T07:37:56.269 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010403099999999999 2024-09-11T07:37:56.270 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.270 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.270 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:37:56.270 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:01.995891+0000", 2024-09-11T07:37:56.270 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.0668999999999996e-05 2024-09-11T07:37:56.270 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.270 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.270 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:37:56.270 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.008367+0000", 2024-09-11T07:37:56.270 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.012475966999999999 2024-09-11T07:37:56.270 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.270 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.270 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:56.270 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.008945+0000", 2024-09-11T07:37:56.270 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00057729099999999996 2024-09-11T07:37:56.271 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.271 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.271 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:56.271 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.034874+0000", 2024-09-11T07:37:56.271 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.025929285999999999 2024-09-11T07:37:56.271 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.271 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.271 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:37:56.271 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.034966+0000", 2024-09-11T07:37:56.271 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.2115999999999997e-05 2024-09-11T07:37:56.271 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.271 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.271 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:37:56.271 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.035358+0000", 2024-09-11T07:37:56.272 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00039192399999999998 2024-09-11T07:37:56.272 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.272 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:37:56.272 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.272 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.272 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.272 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:213461 7.f 7:f62d3026:::benchmark_data_smithi022_77720_object89706:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-11T07:37:56.272 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:00.102257+0000", 2024-09-11T07:37:56.272 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 356.095895676, 2024-09-11T07:37:56.272 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9333541919999999, 2024-09-11T07:37:56.272 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:37:56.272 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:56.272 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:37:56.272 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:37:56.273 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:37:56.273 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213461 2024-09-11T07:37:56.273 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.273 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:37:56.273 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.273 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:37:56.273 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.102257+0000", 2024-09-11T07:37:56.273 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.273 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.273 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.273 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:37:56.273 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.102257+0000", 2024-09-11T07:37:56.274 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.274 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.274 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.274 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:37:56.274 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.102255+0000", 2024-09-11T07:37:56.274 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:37:56.274 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.274 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.274 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:37:56.274 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.102261+0000", 2024-09-11T07:37:56.274 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5309999999999998e-06 2024-09-11T07:37:56.274 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.274 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.274 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:37:56.275 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.102271+0000", 2024-09-11T07:37:56.275 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0169e-05 2024-09-11T07:37:56.275 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.275 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.275 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:37:56.275 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.102292+0000", 2024-09-11T07:37:56.275 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0650999999999999e-05 2024-09-11T07:37:56.275 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.275 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.275 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.275 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.107047+0000", 2024-09-11T07:37:56.275 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0047554040000000004 2024-09-11T07:37:56.276 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.276 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.276 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:37:56.276 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.107053+0000", 2024-09-11T07:37:56.276 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0639999999999999e-06 2024-09-11T07:37:56.276 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.276 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.276 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.276 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502817+0000", 2024-09-11T07:37:56.276 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.39576341199999998 2024-09-11T07:37:56.276 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.276 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.276 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:37:56.277 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502823+0000", 2024-09-11T07:37:56.277 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.725e-06 2024-09-11T07:37:56.277 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.277 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.277 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.277 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:01.998668+0000", 2024-09-11T07:37:56.277 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4958450189999999 2024-09-11T07:37:56.277 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.277 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.277 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:37:56.277 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:01.998813+0000", 2024-09-11T07:37:56.277 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014467399999999999 2024-09-11T07:37:56.277 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.277 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.278 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:37:56.278 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:01.998964+0000", 2024-09-11T07:37:56.278 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015080700000000001 2024-09-11T07:37:56.278 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.278 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.278 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:37:56.278 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.008955+0000", 2024-09-11T07:37:56.278 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0099908700000000007 2024-09-11T07:37:56.278 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.278 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.278 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:56.278 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.009680+0000", 2024-09-11T07:37:56.278 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00072526700000000003 2024-09-11T07:37:56.279 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.279 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.279 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:56.279 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.035411+0000", 2024-09-11T07:37:56.279 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.025730755000000001 2024-09-11T07:37:56.279 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.279 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.279 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:37:56.279 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.035457+0000", 2024-09-11T07:37:56.279 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6254999999999997e-05 2024-09-11T07:37:56.279 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.279 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.279 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:37:56.279 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.035611+0000", 2024-09-11T07:37:56.279 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000153621 2024-09-11T07:37:56.280 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.280 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:37:56.280 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.280 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.280 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.280 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:213466 7.6 7:66df2139:::benchmark_data_smithi022_77720_object89711:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-11T07:37:56.280 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:00.108031+0000", 2024-09-11T07:37:56.280 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:56.281 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 356.09012075800001, 2024-09-11T07:37:56.282 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9748235599999999, 2024-09-11T07:37:56.282 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:37:56.282 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:56.282 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:37:56.282 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:37:56.282 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:37:56.282 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213466 2024-09-11T07:37:56.282 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.282 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:37:56.282 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.282 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:37:56.283 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.108031+0000", 2024-09-11T07:37:56.283 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.283 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.283 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.283 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:37:56.283 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.108031+0000", 2024-09-11T07:37:56.283 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.283 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.283 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.283 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:37:56.283 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.108029+0000", 2024-09-11T07:37:56.283 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:37:56.283 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.284 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.284 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:37:56.284 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.108037+0000", 2024-09-11T07:37:56.284 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.9510000000000006e-06 2024-09-11T07:37:56.284 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.284 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.284 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:37:56.284 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.108047+0000", 2024-09-11T07:37:56.284 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.7249999999999992e-06 2024-09-11T07:37:56.284 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.284 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.284 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:37:56.284 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.108053+0000", 2024-09-11T07:37:56.284 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7030000000000003e-06 2024-09-11T07:37:56.285 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.285 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.285 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.285 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.111646+0000", 2024-09-11T07:37:56.285 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0035932659999999999 2024-09-11T07:37:56.285 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.285 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.285 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:37:56.285 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.111655+0000", 2024-09-11T07:37:56.285 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.4900000000000006e-06 2024-09-11T07:37:56.285 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.285 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.286 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.286 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502475+0000", 2024-09-11T07:37:56.286 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.39081970199999999 2024-09-11T07:37:56.286 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.286 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.286 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:37:56.286 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502486+0000", 2024-09-11T07:37:56.286 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1391e-05 2024-09-11T07:37:56.286 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.286 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.286 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.286 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.055238+0000", 2024-09-11T07:37:56.286 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5527511650000001 2024-09-11T07:37:56.287 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.287 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.287 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:37:56.287 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.055350+0000", 2024-09-11T07:37:56.287 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000112248 2024-09-11T07:37:56.287 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.287 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.287 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:37:56.287 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.055469+0000", 2024-09-11T07:37:56.287 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000119067 2024-09-11T07:37:56.287 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.287 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.287 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:37:56.287 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.064868+0000", 2024-09-11T07:37:56.288 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0093990099999999993 2024-09-11T07:37:56.288 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.288 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.288 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:56.288 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.079028+0000", 2024-09-11T07:37:56.288 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.014160104 2024-09-11T07:37:56.288 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.288 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.288 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:56.288 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.082289+0000", 2024-09-11T07:37:56.288 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0032610529999999999 2024-09-11T07:37:56.288 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.288 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.289 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:37:56.289 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.082491+0000", 2024-09-11T07:37:56.289 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00020178899999999999 2024-09-11T07:37:56.289 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.289 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.289 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:37:56.289 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.082855+0000", 2024-09-11T07:37:56.289 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000364144 2024-09-11T07:37:56.289 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.289 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:37:56.289 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.289 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.289 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.290 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:213482 7.6 7:68f51978:::benchmark_data_smithi022_77720_object89727:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-11T07:37:56.290 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:00.121732+0000", 2024-09-11T07:37:56.290 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 356.07642053799998, 2024-09-11T07:37:56.290 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9616346930000002, 2024-09-11T07:37:56.290 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:37:56.290 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:56.290 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:37:56.290 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:37:56.290 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:37:56.290 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213482 2024-09-11T07:37:56.290 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.290 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:37:56.290 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.290 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:37:56.290 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.121732+0000", 2024-09-11T07:37:56.291 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.291 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.291 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.291 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:37:56.291 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.121732+0000", 2024-09-11T07:37:56.291 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.291 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.291 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.291 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:37:56.291 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.121731+0000", 2024-09-11T07:37:56.291 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:37:56.291 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.291 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.291 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:37:56.292 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.121735+0000", 2024-09-11T07:37:56.292 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7999999999999998e-06 2024-09-11T07:37:56.292 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.292 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.292 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:37:56.292 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.121743+0000", 2024-09-11T07:37:56.292 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.588e-06 2024-09-11T07:37:56.292 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.292 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.292 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:37:56.292 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.121747+0000", 2024-09-11T07:37:56.292 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2390000000000004e-06 2024-09-11T07:37:56.292 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.292 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.293 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.293 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.122374+0000", 2024-09-11T07:37:56.293 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00062680600000000004 2024-09-11T07:37:56.293 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.293 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.293 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:37:56.293 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.122382+0000", 2024-09-11T07:37:56.293 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.2889999999999998e-06 2024-09-11T07:37:56.293 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.293 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.293 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.293 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502524+0000", 2024-09-11T07:37:56.293 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.38014126300000001 2024-09-11T07:37:56.294 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.294 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.294 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:37:56.294 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502531+0000", 2024-09-11T07:37:56.294 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1740000000000003e-06 2024-09-11T07:37:56.294 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.294 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.294 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.294 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.058510+0000", 2024-09-11T07:37:56.294 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.555978825 2024-09-11T07:37:56.294 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.294 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.294 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:37:56.294 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.058651+0000", 2024-09-11T07:37:56.295 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000141395 2024-09-11T07:37:56.295 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.295 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.295 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:37:56.295 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.058837+0000", 2024-09-11T07:37:56.295 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00018597500000000001 2024-09-11T07:37:56.295 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.295 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.295 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:37:56.295 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.073988+0000", 2024-09-11T07:37:56.295 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.015151009999999999 2024-09-11T07:37:56.295 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.296 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.296 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:56.296 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.082894+0000", 2024-09-11T07:37:56.296 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0089062640000000005 2024-09-11T07:37:56.296 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.296 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.296 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:56.296 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.083176+0000", 2024-09-11T07:37:56.296 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00028186800000000001 2024-09-11T07:37:56.296 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.296 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.296 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:37:56.296 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.083216+0000", 2024-09-11T07:37:56.296 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.019e-05 2024-09-11T07:37:56.297 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.297 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.297 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:37:56.297 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.083366+0000", 2024-09-11T07:37:56.297 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015016300000000001 2024-09-11T07:37:56.297 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.297 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:37:56.297 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.297 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.297 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.297 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:213492 7.6 7:63c90940:::benchmark_data_smithi022_77720_object89737:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-11T07:37:56.297 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:00.132840+0000", 2024-09-11T07:37:56.297 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 356.06531204599997, 2024-09-11T07:37:56.297 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.950725611, 2024-09-11T07:37:56.297 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:37:56.298 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:56.298 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:37:56.298 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:37:56.298 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:37:56.298 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213492 2024-09-11T07:37:56.298 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.298 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:37:56.298 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.298 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:37:56.298 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.132840+0000", 2024-09-11T07:37:56.298 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.298 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.298 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.299 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:37:56.299 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.132840+0000", 2024-09-11T07:37:56.299 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.299 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.299 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.299 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:37:56.299 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.132839+0000", 2024-09-11T07:37:56.299 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:56.299 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.299 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.299 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:37:56.299 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.132845+0000", 2024-09-11T07:37:56.299 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.798e-06 2024-09-11T07:37:56.299 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.300 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.300 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:37:56.300 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.132861+0000", 2024-09-11T07:37:56.300 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.6086000000000002e-05 2024-09-11T07:37:56.300 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.300 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.300 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:37:56.300 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.132868+0000", 2024-09-11T07:37:56.300 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2129999999999996e-06 2024-09-11T07:37:56.300 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.300 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.300 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.300 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.132953+0000", 2024-09-11T07:37:56.301 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.5389999999999994e-05 2024-09-11T07:37:56.301 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.301 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.301 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:37:56.301 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.132963+0000", 2024-09-11T07:37:56.301 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.9389999999999992e-06 2024-09-11T07:37:56.301 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.301 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.301 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.301 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502555+0000", 2024-09-11T07:37:56.301 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.36959232199999997 2024-09-11T07:37:56.301 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.301 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.301 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:37:56.301 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502562+0000", 2024-09-11T07:37:56.302 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1160000000000001e-06 2024-09-11T07:37:56.302 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.302 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.302 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.302 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.061705+0000", 2024-09-11T07:37:56.302 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5591422750000001 2024-09-11T07:37:56.302 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.302 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.302 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:37:56.302 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.061818+0000", 2024-09-11T07:37:56.302 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000113824 2024-09-11T07:37:56.302 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.302 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.302 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:37:56.303 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.061942+0000", 2024-09-11T07:37:56.303 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000123125 2024-09-11T07:37:56.303 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.303 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.303 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:37:56.303 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.074019+0000", 2024-09-11T07:37:56.303 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.012077048999999999 2024-09-11T07:37:56.303 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.303 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.303 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:56.303 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.082921+0000", 2024-09-11T07:37:56.303 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0089027959999999993 2024-09-11T07:37:56.303 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.304 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.304 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:56.304 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.083399+0000", 2024-09-11T07:37:56.304 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00047718000000000001 2024-09-11T07:37:56.304 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.304 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.304 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:37:56.304 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.083434+0000", 2024-09-11T07:37:56.304 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5805e-05 2024-09-11T07:37:56.304 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.304 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.304 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:37:56.304 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.083566+0000", 2024-09-11T07:37:56.304 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013135999999999999 2024-09-11T07:37:56.304 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.305 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:37:56.305 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.305 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.305 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.305 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:213495 7.6 7:6bab33dc:::benchmark_data_smithi022_77720_object89740:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-11T07:37:56.305 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:00.133603+0000", 2024-09-11T07:37:56.305 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 356.06454917799999, 2024-09-11T07:37:56.305 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.950173537, 2024-09-11T07:37:56.305 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:37:56.305 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:56.305 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:37:56.305 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:37:56.305 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:37:56.305 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213495 2024-09-11T07:37:56.306 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.306 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:37:56.306 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.306 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:37:56.306 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.133603+0000", 2024-09-11T07:37:56.306 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.306 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.306 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.306 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:37:56.306 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.133603+0000", 2024-09-11T07:37:56.306 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.306 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.306 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.306 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:37:56.307 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.133602+0000", 2024-09-11T07:37:56.307 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:56.307 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.307 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.307 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:37:56.307 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.133608+0000", 2024-09-11T07:37:56.307 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9540000000000001e-06 2024-09-11T07:37:56.307 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.307 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.307 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:37:56.307 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.133618+0000", 2024-09-11T07:37:56.307 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0414000000000001e-05 2024-09-11T07:37:56.307 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.307 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.307 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:37:56.308 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.133624+0000", 2024-09-11T07:37:56.308 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8030000000000002e-06 2024-09-11T07:37:56.308 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.308 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.308 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.308 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.133667+0000", 2024-09-11T07:37:56.308 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3158000000000003e-05 2024-09-11T07:37:56.308 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.308 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.308 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:37:56.308 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.133693+0000", 2024-09-11T07:37:56.308 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.637e-05 2024-09-11T07:37:56.308 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.308 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.309 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.309 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502586+0000", 2024-09-11T07:37:56.309 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.36889280699999999 2024-09-11T07:37:56.309 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.309 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.309 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:37:56.309 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502593+0000", 2024-09-11T07:37:56.309 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.2150000000000004e-06 2024-09-11T07:37:56.309 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.309 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.309 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.309 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.064896+0000", 2024-09-11T07:37:56.309 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5623021349999999 2024-09-11T07:37:56.309 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.310 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.310 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:37:56.310 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.065019+0000", 2024-09-11T07:37:56.310 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000122958 2024-09-11T07:37:56.310 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.310 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.310 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:37:56.310 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.065150+0000", 2024-09-11T07:37:56.310 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013193999999999999 2024-09-11T07:37:56.310 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.310 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.310 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:56.310 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.082948+0000", 2024-09-11T07:37:56.310 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.017797149000000002 2024-09-11T07:37:56.311 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.311 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.311 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:37:56.311 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.083004+0000", 2024-09-11T07:37:56.311 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6184999999999998e-05 2024-09-11T07:37:56.311 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.311 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.311 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:56.311 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.083598+0000", 2024-09-11T07:37:56.311 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00059438899999999996 2024-09-11T07:37:56.311 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.311 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.311 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:37:56.311 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.083638+0000", 2024-09-11T07:37:56.312 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9570999999999998e-05 2024-09-11T07:37:56.312 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.312 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.312 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:37:56.312 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.083777+0000", 2024-09-11T07:37:56.312 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013881399999999999 2024-09-11T07:37:56.312 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.312 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:37:56.312 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.312 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.312 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.312 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:213498 7.f 7:fce9e756:::benchmark_data_smithi022_77720_object89743:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-11T07:37:56.312 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:00.138124+0000", 2024-09-11T07:37:56.313 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 356.06002797899998, 2024-09-11T07:37:56.313 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8977508329999999, 2024-09-11T07:37:56.313 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:37:56.313 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:56.313 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:37:56.313 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:37:56.313 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:37:56.313 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213498 2024-09-11T07:37:56.313 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.313 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:37:56.313 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.313 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:37:56.313 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.138124+0000", 2024-09-11T07:37:56.313 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.314 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.314 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.314 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:37:56.314 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.138124+0000", 2024-09-11T07:37:56.314 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.314 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.314 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.314 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:37:56.314 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.138122+0000", 2024-09-11T07:37:56.314 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:37:56.314 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.314 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.314 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:37:56.314 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.138130+0000", 2024-09-11T07:37:56.315 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.8380000000000002e-06 2024-09-11T07:37:56.315 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.315 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.315 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:37:56.315 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.138142+0000", 2024-09-11T07:37:56.315 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2143999999999999e-05 2024-09-11T07:37:56.315 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.315 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.315 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:37:56.315 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.138149+0000", 2024-09-11T07:37:56.315 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9709999999999998e-06 2024-09-11T07:37:56.315 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.315 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.315 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.316 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.138199+0000", 2024-09-11T07:37:56.316 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.0308e-05 2024-09-11T07:37:56.316 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.316 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.316 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:37:56.316 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.138208+0000", 2024-09-11T07:37:56.316 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.1790000000000008e-06 2024-09-11T07:37:56.316 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.316 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.316 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.316 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502844+0000", 2024-09-11T07:37:56.316 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.36463590600000001 2024-09-11T07:37:56.316 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.316 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.317 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:37:56.317 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502851+0000", 2024-09-11T07:37:56.317 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5240000000000003e-06 2024-09-11T07:37:56.317 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.317 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.317 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.317 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.001786+0000", 2024-09-11T07:37:56.317 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.498934945 2024-09-11T07:37:56.317 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.317 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.317 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:37:56.317 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.001928+0000", 2024-09-11T07:37:56.317 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000142368 2024-09-11T07:37:56.317 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.318 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.318 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:37:56.318 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.002065+0000", 2024-09-11T07:37:56.318 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000136694 2024-09-11T07:37:56.318 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.318 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.318 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:37:56.318 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.008965+0000", 2024-09-11T07:37:56.318 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0068995339999999997 2024-09-11T07:37:56.318 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.318 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.318 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:56.318 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.009714+0000", 2024-09-11T07:37:56.318 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00074906300000000001 2024-09-11T07:37:56.318 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.319 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.319 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:56.319 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.035662+0000", 2024-09-11T07:37:56.319 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.02594861 2024-09-11T07:37:56.319 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.319 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.319 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:37:56.319 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.035691+0000", 2024-09-11T07:37:56.319 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8841000000000001e-05 2024-09-11T07:37:56.319 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.319 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.319 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:37:56.319 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.035875+0000", 2024-09-11T07:37:56.319 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000183938 2024-09-11T07:37:56.320 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.320 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:37:56.320 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.320 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.320 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.320 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:213504 7.f 7:f173a132:::benchmark_data_smithi022_77720_object89749:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-11T07:37:56.320 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:00.147571+0000", 2024-09-11T07:37:56.320 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 356.05058124800001, 2024-09-11T07:37:56.320 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.888562385, 2024-09-11T07:37:56.320 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:37:56.320 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:56.320 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:37:56.320 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:37:56.320 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:37:56.321 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213504 2024-09-11T07:37:56.321 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.321 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:37:56.321 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.321 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:37:56.321 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.147571+0000", 2024-09-11T07:37:56.321 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.321 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.321 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.321 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:37:56.321 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.147571+0000", 2024-09-11T07:37:56.321 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.321 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.321 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.322 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:37:56.322 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.147569+0000", 2024-09-11T07:37:56.322 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:56.322 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.322 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.322 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:37:56.322 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.147575+0000", 2024-09-11T07:37:56.322 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9020000000000004e-06 2024-09-11T07:37:56.322 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.322 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.322 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:37:56.322 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.147586+0000", 2024-09-11T07:37:56.322 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0256e-05 2024-09-11T07:37:56.322 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.323 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.323 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:37:56.323 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.147590+0000", 2024-09-11T07:37:56.323 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5750000000000002e-06 2024-09-11T07:37:56.323 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.323 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.323 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.329 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:37:56.331 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:37:56.333 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.147633+0000", 2024-09-11T07:37:56.333 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3205999999999999e-05 2024-09-11T07:37:56.334 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.334 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.334 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:37:56.334 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.147641+0000", 2024-09-11T07:37:56.334 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.7020000000000001e-06 2024-09-11T07:37:56.334 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.334 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.334 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.334 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502872+0000", 2024-09-11T07:37:56.334 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.35523062300000002 2024-09-11T07:37:56.334 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.334 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.334 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:37:56.335 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502879+0000", 2024-09-11T07:37:56.335 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.8900000000000001e-06 2024-09-11T07:37:56.335 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.335 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.335 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.335 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.004748+0000", 2024-09-11T07:37:56.335 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.501869117 2024-09-11T07:37:56.335 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.335 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.335 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:37:56.335 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.004874+0000", 2024-09-11T07:37:56.335 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012605799999999999 2024-09-11T07:37:56.335 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.335 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.335 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:37:56.336 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.005321+0000", 2024-09-11T07:37:56.336 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00044770500000000001 2024-09-11T07:37:56.336 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.336 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.336 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:37:56.336 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.009129+0000", 2024-09-11T07:37:56.336 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0038073949999999999 2024-09-11T07:37:56.336 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.336 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.336 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:56.336 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.009797+0000", 2024-09-11T07:37:56.336 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00066787400000000005 2024-09-11T07:37:56.336 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.337 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.337 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:56.337 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.035927+0000", 2024-09-11T07:37:56.337 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.026129896999999999 2024-09-11T07:37:56.337 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.337 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.337 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:37:56.337 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.035974+0000", 2024-09-11T07:37:56.337 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7679999999999998e-05 2024-09-11T07:37:56.337 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.337 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.337 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:37:56.337 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.036133+0000", 2024-09-11T07:37:56.337 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015902599999999999 2024-09-11T07:37:56.338 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.338 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:37:56.338 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.338 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.338 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.338 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:213505 7.6 7:62fc8d5e:::benchmark_data_smithi022_77720_object89750:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-11T07:37:56.338 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:00.148250+0000", 2024-09-11T07:37:56.338 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 356.04990211500001, 2024-09-11T07:37:56.338 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9638549599999999, 2024-09-11T07:37:56.338 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:37:56.338 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:56.338 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:37:56.338 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:37:56.338 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:37:56.339 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213505 2024-09-11T07:37:56.339 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.339 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:37:56.339 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.339 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:37:56.339 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.148250+0000", 2024-09-11T07:37:56.339 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.339 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.339 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.339 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:37:56.339 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.148250+0000", 2024-09-11T07:37:56.339 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.339 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.339 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.340 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:37:56.340 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.148249+0000", 2024-09-11T07:37:56.340 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:37:56.340 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.340 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.340 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:37:56.340 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.148254+0000", 2024-09-11T07:37:56.340 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7910000000000004e-06 2024-09-11T07:37:56.340 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.340 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.340 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:37:56.340 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.148263+0000", 2024-09-11T07:37:56.340 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.7709999999999995e-06 2024-09-11T07:37:56.340 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.341 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.341 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:37:56.341 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.148266+0000", 2024-09-11T07:37:56.341 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5580000000000001e-06 2024-09-11T07:37:56.341 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.341 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.341 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.341 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.148301+0000", 2024-09-11T07:37:56.341 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5200999999999997e-05 2024-09-11T07:37:56.341 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.341 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.341 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:37:56.341 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.148307+0000", 2024-09-11T07:37:56.341 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.4559999999999999e-06 2024-09-11T07:37:56.342 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.342 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.342 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.342 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502614+0000", 2024-09-11T07:37:56.342 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.35430687100000002 2024-09-11T07:37:56.342 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.342 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.342 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:37:56.342 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502621+0000", 2024-09-11T07:37:56.342 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.8789999999999997e-06 2024-09-11T07:37:56.342 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.342 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.342 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.342 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.068237+0000", 2024-09-11T07:37:56.343 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5656168400000001 2024-09-11T07:37:56.343 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.343 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.343 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:37:56.343 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.068353+0000", 2024-09-11T07:37:56.343 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000115969 2024-09-11T07:37:56.343 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.343 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.343 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:37:56.343 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.068475+0000", 2024-09-11T07:37:56.343 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012209699999999999 2024-09-11T07:37:56.343 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.343 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.343 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:56.344 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.082974+0000", 2024-09-11T07:37:56.344 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.014498176999999999 2024-09-11T07:37:56.344 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.344 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.344 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:37:56.344 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.083010+0000", 2024-09-11T07:37:56.344 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6448000000000001e-05 2024-09-11T07:37:56.344 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.344 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.344 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:56.344 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.111738+0000", 2024-09-11T07:37:56.344 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.028728331999999999 2024-09-11T07:37:56.344 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.345 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.345 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:37:56.345 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.111794+0000", 2024-09-11T07:37:56.345 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.5967000000000001e-05 2024-09-11T07:37:56.345 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.345 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.345 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:37:56.345 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.112105+0000", 2024-09-11T07:37:56.345 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000310713 2024-09-11T07:37:56.345 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.345 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:37:56.345 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.345 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.346 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.346 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:14406 8.6 8:61ad7f40:::benchmark_data_smithi022_108406_object14405:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-11T07:37:56.346 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:45.106407+0000", 2024-09-11T07:37:56.346 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 251.09174491300001, 2024-09-11T07:37:56.346 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.15729849200000001, 2024-09-11T07:37:56.346 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:37:56.346 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:56.346 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:37:56.346 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:37:56.346 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:37:56.346 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 14406 2024-09-11T07:37:56.346 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.346 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:37:56.346 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.346 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:37:56.347 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.106407+0000", 2024-09-11T07:37:56.347 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.347 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.347 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.347 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:37:56.347 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.106407+0000", 2024-09-11T07:37:56.347 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.347 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.347 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.347 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:37:56.347 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.106406+0000", 2024-09-11T07:37:56.347 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:37:56.347 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.348 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.348 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:37:56.348 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.106540+0000", 2024-09-11T07:37:56.348 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000134602 2024-09-11T07:37:56.348 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.348 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.348 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:37:56.348 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.106551+0000", 2024-09-11T07:37:56.348 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0993e-05 2024-09-11T07:37:56.348 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.348 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.348 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:37:56.348 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.106557+0000", 2024-09-11T07:37:56.348 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9869999999999996e-06 2024-09-11T07:37:56.349 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.349 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.349 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.349 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.111407+0000", 2024-09-11T07:37:56.349 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.004849346 2024-09-11T07:37:56.349 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.349 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.349 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:37:56.349 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.111549+0000", 2024-09-11T07:37:56.349 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000141898 2024-09-11T07:37:56.349 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.349 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.349 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 6,7", 2024-09-11T07:37:56.349 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.111666+0000", 2024-09-11T07:37:56.350 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011752300000000001 2024-09-11T07:37:56.350 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.350 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.350 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:37:56.350 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.114787+0000", 2024-09-11T07:37:56.350 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0031213270000000001 2024-09-11T07:37:56.350 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.350 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.350 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:56.350 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.114864+0000", 2024-09-11T07:37:56.350 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.6822000000000004e-05 2024-09-11T07:37:56.350 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.350 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.350 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:56.350 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.263099+0000", 2024-09-11T07:37:56.351 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.14823464 2024-09-11T07:37:56.351 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.351 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.351 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:37:56.351 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.263667+0000", 2024-09-11T07:37:56.351 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00056761400000000003 2024-09-11T07:37:56.351 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.351 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.351 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:37:56.351 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.263706+0000", 2024-09-11T07:37:56.351 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9289000000000003e-05 2024-09-11T07:37:56.351 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.352 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:37:56.352 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.352 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.352 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.352 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25072.0:14407 8.6 8:6878de25:::benchmark_data_smithi022_108406_object14406:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-11T07:37:56.352 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:33:45.106563+0000", 2024-09-11T07:37:56.352 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 251.09158900700001, 2024-09-11T07:37:56.352 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.158880085, 2024-09-11T07:37:56.352 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:37:56.352 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:37:56.352 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:37:56.352 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25072", 2024-09-11T07:37:56.352 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:37:56.352 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 14407 2024-09-11T07:37:56.353 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.353 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:37:56.353 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.353 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:37:56.353 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.106563+0000", 2024-09-11T07:37:56.353 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.353 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.353 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.353 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:37:56.353 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.106563+0000", 2024-09-11T07:37:56.353 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:37:56.353 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.353 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.354 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:37:56.354 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.106562+0000", 2024-09-11T07:37:56.354 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:37:56.354 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.354 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.354 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:37:56.354 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.106582+0000", 2024-09-11T07:37:56.354 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9241000000000001e-05 2024-09-11T07:37:56.354 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.354 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.354 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:37:56.354 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.106587+0000", 2024-09-11T07:37:56.354 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8209999999999998e-06 2024-09-11T07:37:56.354 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.355 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.355 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:37:56.355 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.106591+0000", 2024-09-11T07:37:56.355 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7179999999999998e-06 2024-09-11T07:37:56.355 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.355 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.355 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:37:56.355 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112442+0000", 2024-09-11T07:37:56.355 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0058511470000000001 2024-09-11T07:37:56.355 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.355 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.355 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:37:56.355 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112545+0000", 2024-09-11T07:37:56.356 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000102713 2024-09-11T07:37:56.356 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.356 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.356 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 6,7", 2024-09-11T07:37:56.356 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.112674+0000", 2024-09-11T07:37:56.356 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012848199999999999 2024-09-11T07:37:56.356 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.356 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.356 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:37:56.356 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.114791+0000", 2024-09-11T07:37:56.356 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0021177100000000001 2024-09-11T07:37:56.356 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.356 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.356 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:56.357 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.264236+0000", 2024-09-11T07:37:56.357 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.149444942 2024-09-11T07:37:56.357 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.357 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.357 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:37:56.357 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.265173+0000", 2024-09-11T07:37:56.357 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000937213 2024-09-11T07:37:56.357 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.357 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.357 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:37:56.357 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.265352+0000", 2024-09-11T07:37:56.357 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00017889100000000001 2024-09-11T07:37:56.357 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:37:56.358 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:37:56.358 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:37:56.358 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:33:45.265443+0000", 2024-09-11T07:37:56.358 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.1003999999999998e-05 2024-09-11T07:37:56.358 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.358 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:37:56.358 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.358 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:37:56.358 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:37:56.358 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:37:56.373 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:56.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:56.375 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:56.476 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:56.477 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 19 16 43888 43872 144.282 96 0.00866633 0.00692411 2024-09-11T07:37:56.514 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:56.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:56.516 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:56.616 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:56.645 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:56.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:56.646 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:56.747 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:56.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:56.790 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:56.790 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:56.846 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:56 smithi022 ceph-mon[29462]: pgmap v510: 148 pgs: 148 active+clean; 2.5 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 158 MiB/s wr, 2.52k op/s 2024-09-11T07:37:56.846 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:56 smithi022 ceph-mon[29462]: from='client.15528 v1:172.21.15.22:0/3824404318' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:37:56.847 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:56 smithi022 ceph-mon[37330]: pgmap v510: 148 pgs: 148 active+clean; 2.5 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 158 MiB/s wr, 2.52k op/s 2024-09-11T07:37:56.847 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:56 smithi022 ceph-mon[37330]: from='client.15528 v1:172.21.15.22:0/3824404318' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:37:56.891 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:56.935 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:56.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:56.936 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:57.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:56 smithi104 ceph-mon[33507]: pgmap v510: 148 pgs: 148 active+clean; 2.5 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 158 MiB/s wr, 2.52k op/s 2024-09-11T07:37:57.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:56 smithi104 ceph-mon[33507]: from='client.15528 v1:172.21.15.22:0/3824404318' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:37:57.037 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:57.062 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:57.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:57.064 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:57.164 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:37:57.200 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:57.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:57.201 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:37:57.302 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:57.335 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:57.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:57.336 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:57.437 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:57.463 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:57.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:57.464 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:57.477 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:2024-09-11T07:37:57.476594+0000 min lat: 0.00139335 max lat: 0.147805 avg lat: 0.00703099 2024-09-11T07:37:57.477 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T07:37:57.477 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 20 16 45471 45455 142.015 98.9375 0.00597789 0.00703099 2024-09-11T07:37:57.565 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:57.603 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:57.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:57.604 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:57.705 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:57.734 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:57.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:57.735 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:57.835 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:57.857 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:57.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:57.857 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:57.958 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:57.989 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:57.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:57.990 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:58.090 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:58.122 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:58.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:58.123 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:58.224 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:58.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:58.258 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:58.258 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:58.359 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:37:58.391 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:58.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:58.392 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:37:58.477 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 21 16 46939 46923 139.621 91.75 0.00610657 0.00715707 2024-09-11T07:37:58.492 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:37:58.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:58.525 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:58.525 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:37:58.626 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:37:58.649 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:58.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:58.650 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:37:58.751 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:58.786 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:58.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:58.787 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:58.888 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:58.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:58.928 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:58.929 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:59.029 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:59.062 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:59.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:59.063 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:59.164 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:59.195 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:59.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:59.196 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:59.297 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:37:59.329 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:59.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:59.330 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:37:59.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:37:59 smithi022 ceph-mon[29462]: pgmap v511: 148 pgs: 148 active+clean; 2.5 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-11T07:37:59.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:37:59 smithi022 ceph-mon[37330]: pgmap v511: 148 pgs: 148 active+clean; 2.5 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-11T07:37:59.430 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:37:59.470 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:59.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:59.471 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:37:59.477 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 22 16 48499 48483 137.707 97.5 0.0156372 0.00725638 2024-09-11T07:37:59.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:37:59 smithi104 ceph-mon[33507]: pgmap v511: 148 pgs: 148 active+clean; 2.5 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-11T07:37:59.573 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:59.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:59.612 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:59.612 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:59.713 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:37:59.734 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:37:59.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:59.735 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:37:59.836 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:37:59.872 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:37:59.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:37:59.873 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:37:59.974 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:00.003 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:00.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:00.004 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:00.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:37:59 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:38:00.105 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:00.137 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:00.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:00.139 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:00.239 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:00.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:00.272 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:00.272 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:00.373 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:00.404 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:00.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:00.405 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:00.426 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":510,"stamp":"2024-09-11T07:37:56.183571+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2659653852,"num_objects":40770,"num_object_clones":0,"num_object_copies":122310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40770,"num_whiteouts":0,"num_read":1954,"num_read_kb":1811,"num_write":41403,"num_write_kb":2598407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":590946,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41508,"ondisk_log_size":41508,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9996772,"kb_used_data":7532580,"kb_used_omap":16,"kb_used_meta":2464111,"kb_avail":739964444,"statfs":{"total":767960285184,"available":757723590656,"internally_reserved":0,"allocated":7713361920,"data_stored":7710132860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16530,"internal_metadata":2523250542},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[13,6,10,10],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1985806336,"num_objects":30301,"num_object_clones":0,"num_object_copies":90903,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30301,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":30301,"num_write_kb":1939264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003724"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":257,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.605256+0000","last_change":"2024-09-11T07:37:29.007467+0000","last_active":"2024-09-11T07:37:37.605256+0000","last_peered":"2024-09-11T07:37:37.605256+0000","last_clean":"2024-09-11T07:37:37.605256+0000","last_became_active":"2024-09-11T07:32:00.501534+0000","last_became_peered":"2024-09-11T07:32:00.501534+0000","last_unstale":"2024-09-11T07:37:37.605256+0000","last_undegraded":"2024-09-11T07:37:37.605256+0000","last_fullsized":"2024-09-11T07:37:37.605256+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:29.007411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:29.007411+0000","last_clean_scrub_stamp":"2024-09-11T07:37:29.007411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:43.805924+0000","scrub_duration":0.003998723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":273,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.615103+0000","last_change":"2024-09-11T07:36:30.991280+0000","last_active":"2024-09-11T07:37:37.615103+0000","last_peered":"2024-09-11T07:37:37.615103+0000","last_clean":"2024-09-11T07:37:37.615103+0000","last_became_active":"2024-09-11T07:32:00.508572+0000","last_became_peered":"2024-09-11T07:32:00.508572+0000","last_unstale":"2024-09-11T07:37:37.615103+0000","last_undegraded":"2024-09-11T07:37:37.615103+0000","last_fullsized":"2024-09-11T07:37:37.615103+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:30.991202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:30.991202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090469859999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"234'34","reported_seq":294,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.605561+0000","last_change":"2024-09-11T07:36:28.772225+0000","last_active":"2024-09-11T07:37:37.605561+0000","last_peered":"2024-09-11T07:37:37.605561+0000","last_clean":"2024-09-11T07:37:37.605561+0000","last_became_active":"2024-09-11T07:26:33.544627+0000","last_became_peered":"2024-09-11T07:26:33.544627+0000","last_unstale":"2024-09-11T07:37:37.605561+0000","last_undegraded":"2024-09-11T07:37:37.605561+0000","last_fullsized":"2024-09-11T07:37:37.605561+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-09-11T07:36:28.772126+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:36:28.772126+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:51.960040+0000","scrub_duration":0.016007773999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":262,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:39.609505+0000","last_change":"2024-09-11T07:37:39.609450+0000","last_active":"2024-09-11T07:37:39.609505+0000","last_peered":"2024-09-11T07:37:39.609505+0000","last_clean":"2024-09-11T07:37:39.609505+0000","last_became_active":"2024-09-11T07:26:35.556296+0000","last_became_peered":"2024-09-11T07:26:35.556296+0000","last_unstale":"2024-09-11T07:37:39.609505+0000","last_undegraded":"2024-09-11T07:37:39.609505+0000","last_fullsized":"2024-09-11T07:37:39.609505+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:39.609357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:39.609357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:46.568905+0000","scrub_duration":0.0026059439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":256,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.605729+0000","last_change":"2024-09-11T07:36:33.755073+0000","last_active":"2024-09-11T07:37:37.605729+0000","last_peered":"2024-09-11T07:37:37.605729+0000","last_clean":"2024-09-11T07:37:37.605729+0000","last_became_active":"2024-09-11T07:26:31.420178+0000","last_became_peered":"2024-09-11T07:26:31.420178+0000","last_unstale":"2024-09-11T07:37:37.605729+0000","last_undegraded":"2024-09-11T07:37:37.605729+0000","last_fullsized":"2024-09-11T07:37:37.605729+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:33.754983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:33.754983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:43.787572+0000","scrub_duration":0.0034784899999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"234'19","reported_seq":283,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.604114+0000","last_change":"2024-09-11T07:36:42.709007+0000","last_active":"2024-09-11T07:37:37.604114+0000","last_peered":"2024-09-11T07:37:37.604114+0000","last_clean":"2024-09-11T07:37:37.604114+0000","last_became_active":"2024-09-11T07:26:33.540732+0000","last_became_peered":"2024-09-11T07:26:33.540732+0000","last_unstale":"2024-09-11T07:37:37.604114+0000","last_undegraded":"2024-09-11T07:37:37.604114+0000","last_fullsized":"2024-09-11T07:37:37.604114+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:36:42.708941+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:36:42.708941+0000","last_clean_scrub_stamp":"2024-09-11T07:36:42.708941+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:47.828260+0000","scrub_duration":0.0088444879999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":271,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.087488+0000","last_change":"2024-09-11T07:37:43.087423+0000","last_active":"2024-09-11T07:37:43.087488+0000","last_peered":"2024-09-11T07:37:43.087488+0000","last_clean":"2024-09-11T07:37:43.087488+0000","last_became_active":"2024-09-11T07:32:00.513650+0000","last_became_peered":"2024-09-11T07:32:00.513650+0000","last_unstale":"2024-09-11T07:37:43.087488+0000","last_undegraded":"2024-09-11T07:37:43.087488+0000","last_fullsized":"2024-09-11T07:37:43.087488+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:43.087302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:37:43.087302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:52.305128+0000","scrub_duration":0.00405653,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":250,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.605284+0000","last_change":"2024-09-11T07:36:48.684925+0000","last_active":"2024-09-11T07:37:37.605284+0000","last_peered":"2024-09-11T07:37:37.605284+0000","last_clean":"2024-09-11T07:37:37.605284+0000","last_became_active":"2024-09-11T07:26:37.566910+0000","last_became_peered":"2024-09-11T07:26:37.566910+0000","last_unstale":"2024-09-11T07:37:37.605284+0000","last_undegraded":"2024-09-11T07:37:37.605284+0000","last_fullsized":"2024-09-11T07:37:37.605284+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:48.684838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:48.684838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:55.308938+0000","scrub_duration":0.0058146650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":261,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.609844+0000","last_change":"2024-09-11T07:37:34.542508+0000","last_active":"2024-09-11T07:37:37.609844+0000","last_peered":"2024-09-11T07:37:37.609844+0000","last_clean":"2024-09-11T07:37:37.609844+0000","last_became_active":"2024-09-11T07:32:00.516662+0000","last_became_peered":"2024-09-11T07:32:00.516662+0000","last_unstale":"2024-09-11T07:37:37.609844+0000","last_undegraded":"2024-09-11T07:37:37.609844+0000","last_fullsized":"2024-09-11T07:37:37.609844+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:34.542441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:34.542441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:53.707135+0000","scrub_duration":0.0015726010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"234'9","reported_seq":279,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.604334+0000","last_change":"2024-09-11T07:37:37.558147+0000","last_active":"2024-09-11T07:37:37.604334+0000","last_peered":"2024-09-11T07:37:37.604334+0000","last_clean":"2024-09-11T07:37:37.604334+0000","last_became_active":"2024-09-11T07:26:33.540351+0000","last_became_peered":"2024-09-11T07:26:33.540351+0000","last_unstale":"2024-09-11T07:37:37.604334+0000","last_undegraded":"2024-09-11T07:37:37.604334+0000","last_fullsized":"2024-09-11T07:37:37.604334+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'9","last_scrub_stamp":"2024-09-11T07:37:37.558036+0000","last_deep_scrub":"82'7","last_deep_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_clean_scrub_stamp":"2024-09-11T07:37:37.558036+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:49.821269+0000","scrub_duration":0.0034408770000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":227,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.608328+0000","last_change":"2024-09-11T07:35:50.332326+0000","last_active":"2024-09-11T07:37:37.608328+0000","last_peered":"2024-09-11T07:37:37.608328+0000","last_clean":"2024-09-11T07:37:37.608328+0000","last_became_active":"2024-09-11T07:32:00.513206+0000","last_became_peered":"2024-09-11T07:32:00.513206+0000","last_unstale":"2024-09-11T07:37:37.608328+0000","last_undegraded":"2024-09-11T07:37:37.608328+0000","last_fullsized":"2024-09-11T07:37:37.608328+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:50.332244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.973323+0000","last_clean_scrub_stamp":"2024-09-11T07:35:50.332244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034655860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":250,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.610488+0000","last_change":"2024-09-11T07:37:36.462873+0000","last_active":"2024-09-11T07:37:37.610488+0000","last_peered":"2024-09-11T07:37:37.610488+0000","last_clean":"2024-09-11T07:37:37.610488+0000","last_became_active":"2024-09-11T07:26:37.561758+0000","last_became_peered":"2024-09-11T07:26:37.561758+0000","last_unstale":"2024-09-11T07:37:37.610488+0000","last_undegraded":"2024-09-11T07:37:37.610488+0000","last_fullsized":"2024-09-11T07:37:37.610488+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:36.462794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:45.149916+0000","last_clean_scrub_stamp":"2024-09-11T07:37:36.462794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:39.725240+0000","scrub_duration":0.001718252,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":226,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:40.336193+0000","last_change":"2024-09-11T07:37:40.336139+0000","last_active":"2024-09-11T07:37:40.336193+0000","last_peered":"2024-09-11T07:37:40.336193+0000","last_clean":"2024-09-11T07:37:40.336193+0000","last_became_active":"2024-09-11T07:32:00.505709+0000","last_became_peered":"2024-09-11T07:32:00.505709+0000","last_unstale":"2024-09-11T07:37:40.336193+0000","last_undegraded":"2024-09-11T07:37:40.336193+0000","last_fullsized":"2024-09-11T07:37:40.336193+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:40.336050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_clean_scrub_stamp":"2024-09-11T07:37:40.336050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:58.024969+0000","scrub_duration":0.0021490649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"234'19","reported_seq":283,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.605446+0000","last_change":"2024-09-11T07:36:27.746378+0000","last_active":"2024-09-11T07:37:37.605446+0000","last_peered":"2024-09-11T07:37:37.605446+0000","last_clean":"2024-09-11T07:37:37.605446+0000","last_became_active":"2024-09-11T07:26:33.544386+0000","last_became_peered":"2024-09-11T07:26:33.544386+0000","last_unstale":"2024-09-11T07:37:37.605446+0000","last_undegraded":"2024-09-11T07:37:37.605446+0000","last_fullsized":"2024-09-11T07:37:37.605446+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'13","last_scrub_stamp":"2024-09-11T07:36:27.746295+0000","last_deep_scrub":"136'13","last_deep_scrub_stamp":"2024-09-11T07:31:08.042198+0000","last_clean_scrub_stamp":"2024-09-11T07:36:27.746295+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:54.572101+0000","scrub_duration":0.0089695690000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":262,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.610601+0000","last_change":"2024-09-11T07:35:34.150965+0000","last_active":"2024-09-11T07:37:37.610601+0000","last_peered":"2024-09-11T07:37:37.610601+0000","last_clean":"2024-09-11T07:37:37.610601+0000","last_became_active":"2024-09-11T07:32:00.502337+0000","last_became_peered":"2024-09-11T07:32:00.502337+0000","last_unstale":"2024-09-11T07:37:37.610601+0000","last_undegraded":"2024-09-11T07:37:37.610601+0000","last_fullsized":"2024-09-11T07:37:37.610601+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:34.150289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:35:34.150289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023057149999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":267,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.613819+0000","last_change":"2024-09-11T07:36:31.977319+0000","last_active":"2024-09-11T07:37:37.613819+0000","last_peered":"2024-09-11T07:37:37.613819+0000","last_clean":"2024-09-11T07:37:37.613819+0000","last_became_active":"2024-09-11T07:32:00.530335+0000","last_became_peered":"2024-09-11T07:32:00.530335+0000","last_unstale":"2024-09-11T07:37:37.613819+0000","last_undegraded":"2024-09-11T07:37:37.613819+0000","last_fullsized":"2024-09-11T07:37:37.613819+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:31.977240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:31.977240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0074114259999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"255'2530","reported_seq":2546,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:54.434629+0000","last_change":"2024-09-11T07:37:26.029984+0000","last_active":"2024-09-11T07:37:54.434629+0000","last_peered":"2024-09-11T07:37:54.434629+0000","last_clean":"2024-09-11T07:37:54.434629+0000","last_became_active":"2024-09-11T07:37:26.029625+0000","last_became_peered":"2024-09-11T07:37:26.029625+0000","last_unstale":"2024-09-11T07:37:54.434629+0000","last_undegraded":"2024-09-11T07:37:54.434629+0000","last_fullsized":"2024-09-11T07:37:54.434629+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":2530,"log_dups_size":0,"ondisk_log_size":2530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:33.495400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165806080,"num_objects":2530,"num_object_clones":0,"num_object_copies":7590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2530,"num_write_kb":161920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"234'31","reported_seq":359,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.605101+0000","last_change":"2024-09-11T07:36:39.737025+0000","last_active":"2024-09-11T07:37:37.605101+0000","last_peered":"2024-09-11T07:37:37.605101+0000","last_clean":"2024-09-11T07:37:37.605101+0000","last_became_active":"2024-09-11T07:29:51.482755+0000","last_became_peered":"2024-09-11T07:29:51.482755+0000","last_unstale":"2024-09-11T07:37:37.605101+0000","last_undegraded":"2024-09-11T07:37:37.605101+0000","last_fullsized":"2024-09-11T07:37:37.605101+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'21","last_scrub_stamp":"2024-09-11T07:36:39.736944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:36:39.736944+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:56.164639+0000","scrub_duration":0.0034170279999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":256,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.609387+0000","last_change":"2024-09-11T07:36:51.418345+0000","last_active":"2024-09-11T07:37:37.609387+0000","last_peered":"2024-09-11T07:37:37.609387+0000","last_clean":"2024-09-11T07:37:37.609387+0000","last_became_active":"2024-09-11T07:32:00.507191+0000","last_became_peered":"2024-09-11T07:32:00.507191+0000","last_unstale":"2024-09-11T07:37:37.609387+0000","last_undegraded":"2024-09-11T07:37:37.609387+0000","last_fullsized":"2024-09-11T07:37:37.609387+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:51.418258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:51.418258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:58.646693+0000","scrub_duration":0.001500181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":244,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.609322+0000","last_change":"2024-09-11T07:37:17.980251+0000","last_active":"2024-09-11T07:37:37.609322+0000","last_peered":"2024-09-11T07:37:37.609322+0000","last_clean":"2024-09-11T07:37:37.609322+0000","last_became_active":"2024-09-11T07:26:35.557989+0000","last_became_peered":"2024-09-11T07:26:35.557989+0000","last_unstale":"2024-09-11T07:37:37.609322+0000","last_undegraded":"2024-09-11T07:37:37.609322+0000","last_fullsized":"2024-09-11T07:37:37.609322+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:00.348684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:00.348684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036577580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":260,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.606075+0000","last_change":"2024-09-11T07:37:21.385329+0000","last_active":"2024-09-11T07:37:37.606075+0000","last_peered":"2024-09-11T07:37:37.606075+0000","last_clean":"2024-09-11T07:37:37.606075+0000","last_became_active":"2024-09-11T07:26:37.564554+0000","last_became_peered":"2024-09-11T07:26:37.564554+0000","last_unstale":"2024-09-11T07:37:37.606075+0000","last_undegraded":"2024-09-11T07:37:37.606075+0000","last_fullsized":"2024-09-11T07:37:37.606075+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:21.385271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:21.385271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:27.423807+0000","scrub_duration":0.001463322,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"255'2607","reported_seq":2622,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:54.828943+0000","last_change":"2024-09-11T07:37:26.029943+0000","last_active":"2024-09-11T07:37:54.828943+0000","last_peered":"2024-09-11T07:37:54.828943+0000","last_clean":"2024-09-11T07:37:54.828943+0000","last_became_active":"2024-09-11T07:37:26.029574+0000","last_became_peered":"2024-09-11T07:37:26.029574+0000","last_unstale":"2024-09-11T07:37:54.828943+0000","last_undegraded":"2024-09-11T07:37:54.828943+0000","last_fullsized":"2024-09-11T07:37:54.828943+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":2607,"log_dups_size":0,"ondisk_log_size":2607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:25.412197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170852352,"num_objects":2607,"num_object_clones":0,"num_object_copies":7821,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2607,"num_write_kb":166848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"234'25","reported_seq":289,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.604913+0000","last_change":"2024-09-11T07:36:43.704258+0000","last_active":"2024-09-11T07:37:37.604913+0000","last_peered":"2024-09-11T07:37:37.604913+0000","last_clean":"2024-09-11T07:37:37.604913+0000","last_became_active":"2024-09-11T07:26:33.535411+0000","last_became_peered":"2024-09-11T07:26:33.535411+0000","last_unstale":"2024-09-11T07:37:37.604913+0000","last_undegraded":"2024-09-11T07:37:37.604913+0000","last_fullsized":"2024-09-11T07:37:37.604913+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'25","last_scrub_stamp":"2024-09-11T07:36:43.704170+0000","last_deep_scrub":"112'15","last_deep_scrub_stamp":"2024-09-11T07:31:10.005896+0000","last_clean_scrub_stamp":"2024-09-11T07:36:43.704170+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:48.004221+0000","scrub_duration":0.0048642039999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":275,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.609884+0000","last_change":"2024-09-11T07:37:36.098229+0000","last_active":"2024-09-11T07:37:37.609884+0000","last_peered":"2024-09-11T07:37:37.609884+0000","last_clean":"2024-09-11T07:37:37.609884+0000","last_became_active":"2024-09-11T07:32:00.506615+0000","last_became_peered":"2024-09-11T07:32:00.506615+0000","last_unstale":"2024-09-11T07:37:37.609884+0000","last_undegraded":"2024-09-11T07:37:37.609884+0000","last_fullsized":"2024-09-11T07:37:37.609884+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:36.098175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:36.098175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:54.532896+0000","scrub_duration":0.0017555839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"151'10","reported_seq":425,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:44.638429+0000","last_change":"2024-09-11T07:37:43.141088+0000","last_active":"2024-09-11T07:37:44.638429+0000","last_peered":"2024-09-11T07:37:44.638429+0000","last_clean":"2024-09-11T07:37:44.638429+0000","last_became_active":"2024-09-11T07:32:00.507447+0000","last_became_peered":"2024-09-11T07:32:00.507447+0000","last_unstale":"2024-09-11T07:37:44.638429+0000","last_undegraded":"2024-09-11T07:37:44.638429+0000","last_fullsized":"2024-09-11T07:37:44.638429+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:37:43.140999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:43.140999+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:57.947183+0000","scrub_duration":0.0039651130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":250,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.604962+0000","last_change":"2024-09-11T07:37:22.546982+0000","last_active":"2024-09-11T07:37:37.604962+0000","last_peered":"2024-09-11T07:37:37.604962+0000","last_clean":"2024-09-11T07:37:37.604962+0000","last_became_active":"2024-09-11T07:26:37.566919+0000","last_became_peered":"2024-09-11T07:26:37.566919+0000","last_unstale":"2024-09-11T07:37:37.604962+0000","last_undegraded":"2024-09-11T07:37:37.604962+0000","last_fullsized":"2024-09-11T07:37:37.604962+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:22.546931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:22.546931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:42.991401+0000","scrub_duration":0.001345527,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"255'2572","reported_seq":2587,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:54.848348+0000","last_change":"2024-09-11T07:37:25.851826+0000","last_active":"2024-09-11T07:37:54.848348+0000","last_peered":"2024-09-11T07:37:54.848348+0000","last_clean":"2024-09-11T07:37:54.848348+0000","last_became_active":"2024-09-11T07:37:25.851536+0000","last_became_peered":"2024-09-11T07:37:25.851536+0000","last_unstale":"2024-09-11T07:37:54.848348+0000","last_undegraded":"2024-09-11T07:37:54.848348+0000","last_fullsized":"2024-09-11T07:37:54.848348+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":2572,"log_dups_size":0,"ondisk_log_size":2572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:42.549849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168558592,"num_objects":2572,"num_object_clones":0,"num_object_copies":7716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2572,"num_write_kb":164608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"234'34","reported_seq":303,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.609844+0000","last_change":"2024-09-11T07:37:16.938565+0000","last_active":"2024-09-11T07:37:37.609844+0000","last_peered":"2024-09-11T07:37:37.609844+0000","last_clean":"2024-09-11T07:37:37.609844+0000","last_became_active":"2024-09-11T07:26:33.549733+0000","last_became_peered":"2024-09-11T07:26:33.549733+0000","last_unstale":"2024-09-11T07:37:37.609844+0000","last_undegraded":"2024-09-11T07:37:37.609844+0000","last_fullsized":"2024-09-11T07:37:37.609844+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'34","last_scrub_stamp":"2024-09-11T07:37:16.938496+0000","last_deep_scrub":"83'22","last_deep_scrub_stamp":"2024-09-11T07:34:04.375457+0000","last_clean_scrub_stamp":"2024-09-11T07:37:16.938496+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:44.726347+0000","scrub_duration":0.0028468529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":275,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.609528+0000","last_change":"2024-09-11T07:35:40.786431+0000","last_active":"2024-09-11T07:37:37.609528+0000","last_peered":"2024-09-11T07:37:37.609528+0000","last_clean":"2024-09-11T07:37:37.609528+0000","last_became_active":"2024-09-11T07:32:00.513472+0000","last_became_peered":"2024-09-11T07:32:00.513472+0000","last_unstale":"2024-09-11T07:37:37.609528+0000","last_undegraded":"2024-09-11T07:37:37.609528+0000","last_fullsized":"2024-09-11T07:37:37.609528+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:40.786358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:35:40.786358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014952305000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":399,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:44.638843+0000","last_change":"2024-09-11T07:37:30.657587+0000","last_active":"2024-09-11T07:37:44.638843+0000","last_peered":"2024-09-11T07:37:44.638843+0000","last_clean":"2024-09-11T07:37:44.638843+0000","last_became_active":"2024-09-11T07:26:35.560406+0000","last_became_peered":"2024-09-11T07:26:35.560406+0000","last_unstale":"2024-09-11T07:37:44.638843+0000","last_undegraded":"2024-09-11T07:37:44.638843+0000","last_fullsized":"2024-09-11T07:37:44.638843+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:37:30.657477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:30.657477+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:57.486387+0000","scrub_duration":0.001943012,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":250,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.611036+0000","last_change":"2024-09-11T07:37:20.369255+0000","last_active":"2024-09-11T07:37:37.611036+0000","last_peered":"2024-09-11T07:37:37.611036+0000","last_clean":"2024-09-11T07:37:37.611036+0000","last_became_active":"2024-09-11T07:26:37.568005+0000","last_became_peered":"2024-09-11T07:26:37.568005+0000","last_unstale":"2024-09-11T07:37:37.611036+0000","last_undegraded":"2024-09-11T07:37:37.611036+0000","last_fullsized":"2024-09-11T07:37:37.611036+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:20.369199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:31.200685+0000","last_clean_scrub_stamp":"2024-09-11T07:37:20.369199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:32.604085+0000","scrub_duration":0.001371688,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"255'2509","reported_seq":2524,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:54.205626+0000","last_change":"2024-09-11T07:37:25.855865+0000","last_active":"2024-09-11T07:37:54.205626+0000","last_peered":"2024-09-11T07:37:54.205626+0000","last_clean":"2024-09-11T07:37:54.205626+0000","last_became_active":"2024-09-11T07:37:25.855577+0000","last_became_peered":"2024-09-11T07:37:25.855577+0000","last_unstale":"2024-09-11T07:37:54.205626+0000","last_undegraded":"2024-09-11T07:37:54.205626+0000","last_fullsized":"2024-09-11T07:37:54.205626+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":2509,"log_dups_size":0,"ondisk_log_size":2509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:48.134227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":164429824,"num_objects":2509,"num_object_clones":0,"num_object_copies":7527,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2509,"num_write_kb":160576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"234'24","reported_seq":300,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.605170+0000","last_change":"2024-09-11T07:37:36.542263+0000","last_active":"2024-09-11T07:37:37.605170+0000","last_peered":"2024-09-11T07:37:37.605170+0000","last_clean":"2024-09-11T07:37:37.605170+0000","last_became_active":"2024-09-11T07:26:33.540777+0000","last_became_peered":"2024-09-11T07:26:33.540777+0000","last_unstale":"2024-09-11T07:37:37.605170+0000","last_undegraded":"2024-09-11T07:37:37.605170+0000","last_fullsized":"2024-09-11T07:37:37.605170+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'24","last_scrub_stamp":"2024-09-11T07:37:36.542161+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-09-11T07:29:38.112111+0000","last_clean_scrub_stamp":"2024-09-11T07:37:36.542161+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:55.053665+0000","scrub_duration":0.001886759,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":250,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.608237+0000","last_change":"2024-09-11T07:36:37.445237+0000","last_active":"2024-09-11T07:37:37.608237+0000","last_peered":"2024-09-11T07:37:37.608237+0000","last_clean":"2024-09-11T07:37:37.608237+0000","last_became_active":"2024-09-11T07:32:00.518538+0000","last_became_peered":"2024-09-11T07:32:00.518538+0000","last_unstale":"2024-09-11T07:37:37.608237+0000","last_undegraded":"2024-09-11T07:37:37.608237+0000","last_fullsized":"2024-09-11T07:37:37.608237+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:37.444760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:37.444760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:47.891356+0000","scrub_duration":0.012636722,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":252,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.611804+0000","last_change":"2024-09-11T07:35:29.166617+0000","last_active":"2024-09-11T07:37:37.611804+0000","last_peered":"2024-09-11T07:37:37.611804+0000","last_clean":"2024-09-11T07:37:37.611804+0000","last_became_active":"2024-09-11T07:26:35.569081+0000","last_became_peered":"2024-09-11T07:26:35.569081+0000","last_unstale":"2024-09-11T07:37:37.611804+0000","last_undegraded":"2024-09-11T07:37:37.611804+0000","last_fullsized":"2024-09-11T07:37:37.611804+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:29.166488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:35:29.166488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059891079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":250,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.607544+0000","last_change":"2024-09-11T07:36:50.920711+0000","last_active":"2024-09-11T07:37:37.607544+0000","last_peered":"2024-09-11T07:37:37.607544+0000","last_clean":"2024-09-11T07:37:37.607544+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:37:37.607544+0000","last_undegraded":"2024-09-11T07:37:37.607544+0000","last_fullsized":"2024-09-11T07:37:37.607544+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:50.920625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:46.853387+0000","last_clean_scrub_stamp":"2024-09-11T07:36:50.920625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:19.177270+0000","scrub_duration":0.012404324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"255'2553","reported_seq":2567,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:54.851473+0000","last_change":"2024-09-11T07:37:25.855661+0000","last_active":"2024-09-11T07:37:54.851473+0000","last_peered":"2024-09-11T07:37:54.851473+0000","last_clean":"2024-09-11T07:37:54.851473+0000","last_became_active":"2024-09-11T07:37:25.854796+0000","last_became_peered":"2024-09-11T07:37:25.854796+0000","last_unstale":"2024-09-11T07:37:54.851473+0000","last_undegraded":"2024-09-11T07:37:54.851473+0000","last_fullsized":"2024-09-11T07:37:54.851473+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":2553,"log_dups_size":0,"ondisk_log_size":2553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:31.723924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167313408,"num_objects":2553,"num_object_clones":0,"num_object_copies":7659,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2553,"num_write_kb":163392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"234'28","reported_seq":296,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.604065+0000","last_change":"2024-09-11T07:37:35.580526+0000","last_active":"2024-09-11T07:37:37.604065+0000","last_peered":"2024-09-11T07:37:37.604065+0000","last_clean":"2024-09-11T07:37:37.604065+0000","last_became_active":"2024-09-11T07:26:33.537850+0000","last_became_peered":"2024-09-11T07:26:33.537850+0000","last_unstale":"2024-09-11T07:37:37.604065+0000","last_undegraded":"2024-09-11T07:37:37.604065+0000","last_fullsized":"2024-09-11T07:37:37.604065+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'28","last_scrub_stamp":"2024-09-11T07:37:35.580463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:35.580463+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:50.893847+0000","scrub_duration":0.0019235789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":102,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.604860+0000","last_change":"2024-09-11T07:37:20.527569+0000","last_active":"2024-09-11T07:37:37.604860+0000","last_peered":"2024-09-11T07:37:37.604860+0000","last_clean":"2024-09-11T07:37:37.604860+0000","last_became_active":"2024-09-11T07:36:05.226733+0000","last_became_peered":"2024-09-11T07:36:05.226733+0000","last_unstale":"2024-09-11T07:37:37.604860+0000","last_undegraded":"2024-09-11T07:37:37.604860+0000","last_fullsized":"2024-09-11T07:37:37.604860+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:20.527511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:20.527511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:31.232016+0000","scrub_duration":0.0011632039999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"84'9","reported_seq":398,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:49.639204+0000","last_change":"2024-09-11T07:37:37.000676+0000","last_active":"2024-09-11T07:37:49.639204+0000","last_peered":"2024-09-11T07:37:49.639204+0000","last_clean":"2024-09-11T07:37:49.639204+0000","last_became_active":"2024-09-11T07:26:35.562997+0000","last_became_peered":"2024-09-11T07:26:35.562997+0000","last_unstale":"2024-09-11T07:37:49.639204+0000","last_undegraded":"2024-09-11T07:37:49.639204+0000","last_fullsized":"2024-09-11T07:37:49.639204+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-11T07:37:37.000619+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-11T07:35:39.832547+0000","last_clean_scrub_stamp":"2024-09-11T07:37:37.000619+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:57.784083+0000","scrub_duration":0.0021426219999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":224,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.608986+0000","last_change":"2024-09-11T07:36:32.419622+0000","last_active":"2024-09-11T07:37:37.608986+0000","last_peered":"2024-09-11T07:37:37.608986+0000","last_clean":"2024-09-11T07:37:37.608986+0000","last_became_active":"2024-09-11T07:32:00.511005+0000","last_became_peered":"2024-09-11T07:32:00.511005+0000","last_unstale":"2024-09-11T07:37:37.608986+0000","last_undegraded":"2024-09-11T07:37:37.608986+0000","last_fullsized":"2024-09-11T07:37:37.608986+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:32.419555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:32.419555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046427200000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"255'2475","reported_seq":2490,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:54.214604+0000","last_change":"2024-09-11T07:37:25.854170+0000","last_active":"2024-09-11T07:37:54.214604+0000","last_peered":"2024-09-11T07:37:54.214604+0000","last_clean":"2024-09-11T07:37:54.214604+0000","last_became_active":"2024-09-11T07:37:25.853857+0000","last_became_peered":"2024-09-11T07:37:25.853857+0000","last_unstale":"2024-09-11T07:37:54.214604+0000","last_undegraded":"2024-09-11T07:37:54.214604+0000","last_fullsized":"2024-09-11T07:37:54.214604+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":2475,"log_dups_size":0,"ondisk_log_size":2475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:29.440611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":162201600,"num_objects":2475,"num_object_clones":0,"num_object_copies":7425,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2475,"num_write_kb":158400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"234'38","reported_seq":319,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.605138+0000","last_change":"2024-09-11T07:35:52.942431+0000","last_active":"2024-09-11T07:37:37.605138+0000","last_peered":"2024-09-11T07:37:37.605138+0000","last_clean":"2024-09-11T07:37:37.605138+0000","last_became_active":"2024-09-11T07:32:00.520670+0000","last_became_peered":"2024-09-11T07:32:00.520670+0000","last_unstale":"2024-09-11T07:37:37.605138+0000","last_undegraded":"2024-09-11T07:37:37.605138+0000","last_fullsized":"2024-09-11T07:37:37.605138+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'26","last_scrub_stamp":"2024-09-11T07:35:52.942347+0000","last_deep_scrub":"87'26","last_deep_scrub_stamp":"2024-09-11T07:35:52.942347+0000","last_clean_scrub_stamp":"2024-09-11T07:35:52.942347+0000","objects_scrubbed":8,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010845960999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":265,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.610905+0000","last_change":"2024-09-11T07:37:33.081869+0000","last_active":"2024-09-11T07:37:37.610905+0000","last_peered":"2024-09-11T07:37:37.610905+0000","last_clean":"2024-09-11T07:37:37.610905+0000","last_became_active":"2024-09-11T07:26:31.421500+0000","last_became_peered":"2024-09-11T07:26:31.421500+0000","last_unstale":"2024-09-11T07:37:37.610905+0000","last_undegraded":"2024-09-11T07:37:37.610905+0000","last_fullsized":"2024-09-11T07:37:37.610905+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:33.081815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:33.081815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:54.240779+0000","scrub_duration":0.001426379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":262,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.603910+0000","last_change":"2024-09-11T07:36:30.786071+0000","last_active":"2024-09-11T07:37:37.603910+0000","last_peered":"2024-09-11T07:37:37.603910+0000","last_clean":"2024-09-11T07:37:37.603910+0000","last_became_active":"2024-09-11T07:32:00.499747+0000","last_became_peered":"2024-09-11T07:32:00.499747+0000","last_unstale":"2024-09-11T07:37:37.603910+0000","last_undegraded":"2024-09-11T07:37:37.603910+0000","last_fullsized":"2024-09-11T07:37:37.603910+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:30.785991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:27.968579+0000","last_clean_scrub_stamp":"2024-09-11T07:36:30.785991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:53.321816+0000","scrub_duration":0.011893737999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":250,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.605464+0000","last_change":"2024-09-11T07:37:28.623070+0000","last_active":"2024-09-11T07:37:37.605464+0000","last_peered":"2024-09-11T07:37:37.605464+0000","last_clean":"2024-09-11T07:37:37.605464+0000","last_became_active":"2024-09-11T07:26:37.564616+0000","last_became_peered":"2024-09-11T07:26:37.564616+0000","last_unstale":"2024-09-11T07:37:37.605464+0000","last_undegraded":"2024-09-11T07:37:37.605464+0000","last_fullsized":"2024-09-11T07:37:37.605464+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:28.622998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:29.978798+0000","last_clean_scrub_stamp":"2024-09-11T07:37:28.622998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:42.675263+0000","scrub_duration":0.001843798,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"255'2504","reported_seq":2518,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:54.219589+0000","last_change":"2024-09-11T07:37:25.854301+0000","last_active":"2024-09-11T07:37:54.219589+0000","last_peered":"2024-09-11T07:37:54.219589+0000","last_clean":"2024-09-11T07:37:54.219589+0000","last_became_active":"2024-09-11T07:37:25.853844+0000","last_became_peered":"2024-09-11T07:37:25.853844+0000","last_unstale":"2024-09-11T07:37:54.219589+0000","last_undegraded":"2024-09-11T07:37:54.219589+0000","last_fullsized":"2024-09-11T07:37:54.219589+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":2504,"log_dups_size":0,"ondisk_log_size":2504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:45.631635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":164102144,"num_objects":2504,"num_object_clones":0,"num_object_copies":7512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2504,"num_write_kb":160256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"236'32","reported_seq":325,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.615044+0000","last_change":"2024-09-11T07:36:50.024003+0000","last_active":"2024-09-11T07:37:37.615044+0000","last_peered":"2024-09-11T07:37:37.615044+0000","last_clean":"2024-09-11T07:37:37.615044+0000","last_became_active":"2024-09-11T07:32:00.506270+0000","last_became_peered":"2024-09-11T07:32:00.506270+0000","last_unstale":"2024-09-11T07:37:37.615044+0000","last_undegraded":"2024-09-11T07:37:37.615044+0000","last_fullsized":"2024-09-11T07:37:37.615044+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'32","last_scrub_stamp":"2024-09-11T07:36:50.023909+0000","last_deep_scrub":"236'32","last_deep_scrub_stamp":"2024-09-11T07:36:50.023909+0000","last_clean_scrub_stamp":"2024-09-11T07:36:50.023909+0000","objects_scrubbed":8,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:00.989529+0000","scrub_duration":0.030217668999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":224,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.609836+0000","last_change":"2024-09-11T07:36:48.434408+0000","last_active":"2024-09-11T07:37:37.609836+0000","last_peered":"2024-09-11T07:37:37.609836+0000","last_clean":"2024-09-11T07:37:37.609836+0000","last_became_active":"2024-09-11T07:32:00.510631+0000","last_became_peered":"2024-09-11T07:32:00.510631+0000","last_unstale":"2024-09-11T07:37:37.609836+0000","last_undegraded":"2024-09-11T07:37:37.609836+0000","last_fullsized":"2024-09-11T07:37:37.609836+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:48.434320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:49.144041+0000","last_clean_scrub_stamp":"2024-09-11T07:36:48.434320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:02.587912+0000","scrub_duration":0.0066312669999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":53,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.614457+0000","last_change":"2024-09-11T07:36:55.163926+0000","last_active":"2024-09-11T07:37:37.614457+0000","last_peered":"2024-09-11T07:37:37.614457+0000","last_clean":"2024-09-11T07:37:37.614457+0000","last_became_active":"2024-09-11T07:36:05.225840+0000","last_became_peered":"2024-09-11T07:36:05.225840+0000","last_unstale":"2024-09-11T07:37:37.614457+0000","last_undegraded":"2024-09-11T07:37:37.614457+0000","last_fullsized":"2024-09-11T07:37:37.614457+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:55.163476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:36:55.163476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:57.584983+0000","scrub_duration":0.0082978570000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":249,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.608921+0000","last_change":"2024-09-11T07:36:54.828435+0000","last_active":"2024-09-11T07:37:37.608921+0000","last_peered":"2024-09-11T07:37:37.608921+0000","last_clean":"2024-09-11T07:37:37.608921+0000","last_became_active":"2024-09-11T07:26:37.861279+0000","last_became_peered":"2024-09-11T07:26:37.861279+0000","last_unstale":"2024-09-11T07:37:37.608921+0000","last_undegraded":"2024-09-11T07:37:37.608921+0000","last_fullsized":"2024-09-11T07:37:37.608921+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:54.827454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:39.723598+0000","last_clean_scrub_stamp":"2024-09-11T07:36:54.827454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:06.664883+0000","scrub_duration":0.016642015,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.f","version":"255'2380","reported_seq":2395,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:54.123855+0000","last_change":"2024-09-11T07:37:25.856504+0000","last_active":"2024-09-11T07:37:54.123855+0000","last_peered":"2024-09-11T07:37:54.123855+0000","last_clean":"2024-09-11T07:37:54.123855+0000","last_became_active":"2024-09-11T07:37:25.856214+0000","last_became_peered":"2024-09-11T07:37:25.856214+0000","last_unstale":"2024-09-11T07:37:54.123855+0000","last_undegraded":"2024-09-11T07:37:54.123855+0000","last_fullsized":"2024-09-11T07:37:54.123855+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":2380,"log_dups_size":0,"ondisk_log_size":2380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:48.819382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":155975680,"num_objects":2380,"num_object_clones":0,"num_object_copies":7140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2380,"num_write_kb":152320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"234'20","reported_seq":286,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.609754+0000","last_change":"2024-09-11T07:36:37.911537+0000","last_active":"2024-09-11T07:37:37.609754+0000","last_peered":"2024-09-11T07:37:37.609754+0000","last_clean":"2024-09-11T07:37:37.609754+0000","last_became_active":"2024-09-11T07:26:33.543063+0000","last_became_peered":"2024-09-11T07:26:33.543063+0000","last_unstale":"2024-09-11T07:37:37.609754+0000","last_undegraded":"2024-09-11T07:37:37.609754+0000","last_fullsized":"2024-09-11T07:37:37.609754+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:36:37.911329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:36:37.911329+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:44.764318+0000","scrub_duration":0.01198222,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":282,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.609465+0000","last_change":"2024-09-11T07:36:38.909472+0000","last_active":"2024-09-11T07:37:37.609465+0000","last_peered":"2024-09-11T07:37:37.609465+0000","last_clean":"2024-09-11T07:37:37.609465+0000","last_became_active":"2024-09-11T07:32:00.522456+0000","last_became_peered":"2024-09-11T07:32:00.522456+0000","last_unstale":"2024-09-11T07:37:37.609465+0000","last_undegraded":"2024-09-11T07:37:37.609465+0000","last_fullsized":"2024-09-11T07:37:37.609465+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:36:38.909348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:38.909348+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:03.548160+0000","scrub_duration":0.0031018040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":407,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:49.639681+0000","last_change":"2024-09-11T07:37:26.141508+0000","last_active":"2024-09-11T07:37:49.639681+0000","last_peered":"2024-09-11T07:37:49.639681+0000","last_clean":"2024-09-11T07:37:49.639681+0000","last_became_active":"2024-09-11T07:26:35.560422+0000","last_became_peered":"2024-09-11T07:26:35.560422+0000","last_unstale":"2024-09-11T07:37:49.639681+0000","last_undegraded":"2024-09-11T07:37:49.639681+0000","last_fullsized":"2024-09-11T07:37:49.639681+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:37:26.141447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.141447+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:28.626906+0000","scrub_duration":0.0015408170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":260,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.614232+0000","last_change":"2024-09-11T07:36:40.027067+0000","last_active":"2024-09-11T07:37:37.614232+0000","last_peered":"2024-09-11T07:37:37.614232+0000","last_clean":"2024-09-11T07:37:37.614232+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:37:37.614232+0000","last_undegraded":"2024-09-11T07:37:37.614232+0000","last_fullsized":"2024-09-11T07:37:37.614232+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:40.026982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_clean_scrub_stamp":"2024-09-11T07:36:40.026982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:00.546972+0000","scrub_duration":0.0090395279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.9","version":"255'2641","reported_seq":2657,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:54.926264+0000","last_change":"2024-09-11T07:37:25.854831+0000","last_active":"2024-09-11T07:37:54.926264+0000","last_peered":"2024-09-11T07:37:54.926264+0000","last_clean":"2024-09-11T07:37:54.926264+0000","last_became_active":"2024-09-11T07:37:25.854472+0000","last_became_peered":"2024-09-11T07:37:25.854472+0000","last_unstale":"2024-09-11T07:37:54.926264+0000","last_undegraded":"2024-09-11T07:37:54.926264+0000","last_fullsized":"2024-09-11T07:37:54.926264+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":2641,"log_dups_size":0,"ondisk_log_size":2641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:54.556048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":173080576,"num_objects":2641,"num_object_clones":0,"num_object_copies":7923,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2641,"num_write_kb":169024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"234'41","reported_seq":212,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:44.638486+0000","last_change":"2024-09-11T07:36:41.343699+0000","last_active":"2024-09-11T07:37:44.638486+0000","last_peered":"2024-09-11T07:37:44.638486+0000","last_clean":"2024-09-11T07:37:44.638486+0000","last_became_active":"2024-09-11T07:33:10.017961+0000","last_became_peered":"2024-09-11T07:33:10.017961+0000","last_unstale":"2024-09-11T07:37:44.638486+0000","last_undegraded":"2024-09-11T07:37:44.638486+0000","last_fullsized":"2024-09-11T07:37:44.638486+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'41","last_scrub_stamp":"2024-09-11T07:36:41.343611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:36:41.343611+0000","objects_scrubbed":10,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:57.163851+0000","scrub_duration":0.0043492390000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":266,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.612363+0000","last_change":"2024-09-11T07:36:51.308567+0000","last_active":"2024-09-11T07:37:37.612363+0000","last_peered":"2024-09-11T07:37:37.612363+0000","last_clean":"2024-09-11T07:37:37.612363+0000","last_became_active":"2024-09-11T07:26:31.424618+0000","last_became_peered":"2024-09-11T07:26:31.424618+0000","last_unstale":"2024-09-11T07:37:37.612363+0000","last_undegraded":"2024-09-11T07:37:37.612363+0000","last_fullsized":"2024-09-11T07:37:37.612363+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:51.308492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:51.308492+0000","last_clean_scrub_stamp":"2024-09-11T07:36:51.308492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:51.904303+0000","scrub_duration":0.0057563680000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":262,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.615513+0000","last_change":"2024-09-11T07:37:33.144930+0000","last_active":"2024-09-11T07:37:37.615513+0000","last_peered":"2024-09-11T07:37:37.615513+0000","last_clean":"2024-09-11T07:37:37.615513+0000","last_became_active":"2024-09-11T07:26:35.560363+0000","last_became_peered":"2024-09-11T07:26:35.560363+0000","last_unstale":"2024-09-11T07:37:37.615513+0000","last_undegraded":"2024-09-11T07:37:37.615513+0000","last_fullsized":"2024-09-11T07:37:37.615513+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:33.144873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:33.144873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:42.928193+0000","scrub_duration":0.001754675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":267,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:39.054076+0000","last_change":"2024-09-11T07:37:39.053981+0000","last_active":"2024-09-11T07:37:39.054076+0000","last_peered":"2024-09-11T07:37:39.054076+0000","last_clean":"2024-09-11T07:37:39.054076+0000","last_became_active":"2024-09-11T07:32:00.505919+0000","last_became_peered":"2024-09-11T07:32:00.505919+0000","last_unstale":"2024-09-11T07:37:39.054076+0000","last_undegraded":"2024-09-11T07:37:39.054076+0000","last_fullsized":"2024-09-11T07:37:39.054076+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:39.053900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:39.053900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:49.165548+0000","scrub_duration":0.0050142590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"255'2528","reported_seq":2543,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:54.116577+0000","last_change":"2024-09-11T07:37:25.855294+0000","last_active":"2024-09-11T07:37:54.116577+0000","last_peered":"2024-09-11T07:37:54.116577+0000","last_clean":"2024-09-11T07:37:54.116577+0000","last_became_active":"2024-09-11T07:37:25.855005+0000","last_became_peered":"2024-09-11T07:37:25.855005+0000","last_unstale":"2024-09-11T07:37:54.116577+0000","last_undegraded":"2024-09-11T07:37:54.116577+0000","last_fullsized":"2024-09-11T07:37:54.116577+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":2528,"log_dups_size":0,"ondisk_log_size":2528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:26.698988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165675008,"num_objects":2528,"num_object_clones":0,"num_object_copies":7584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2528,"num_write_kb":161792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"234'22","reported_seq":291,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.609922+0000","last_change":"2024-09-11T07:37:20.965215+0000","last_active":"2024-09-11T07:37:37.609922+0000","last_peered":"2024-09-11T07:37:37.609922+0000","last_clean":"2024-09-11T07:37:37.609922+0000","last_became_active":"2024-09-11T07:26:33.546126+0000","last_became_peered":"2024-09-11T07:26:33.546126+0000","last_unstale":"2024-09-11T07:37:37.609922+0000","last_undegraded":"2024-09-11T07:37:37.609922+0000","last_fullsized":"2024-09-11T07:37:37.609922+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'22","last_scrub_stamp":"2024-09-11T07:37:20.965147+0000","last_deep_scrub":"144'18","last_deep_scrub_stamp":"2024-09-11T07:31:51.603378+0000","last_clean_scrub_stamp":"2024-09-11T07:37:20.965147+0000","objects_scrubbed":5,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:26.343736+0000","scrub_duration":0.0019365999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":948,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:53.125572+0000","last_change":"2024-09-11T07:37:22.990093+0000","last_active":"2024-09-11T07:37:53.125572+0000","last_peered":"2024-09-11T07:37:53.125572+0000","last_clean":"2024-09-11T07:37:53.125572+0000","last_became_active":"2024-09-11T07:26:31.421276+0000","last_became_peered":"2024-09-11T07:26:31.421276+0000","last_unstale":"2024-09-11T07:37:53.125572+0000","last_undegraded":"2024-09-11T07:37:53.125572+0000","last_fullsized":"2024-09-11T07:37:53.125572+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'5","last_scrub_stamp":"2024-09-11T07:37:22.990033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:22.990033+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:51.516885+0000","scrub_duration":0.001646954,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":675,"num_read_kb":670,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":276,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.605633+0000","last_change":"2024-09-11T07:36:45.059852+0000","last_active":"2024-09-11T07:37:37.605633+0000","last_peered":"2024-09-11T07:37:37.605633+0000","last_clean":"2024-09-11T07:37:37.605633+0000","last_became_active":"2024-09-11T07:26:31.413354+0000","last_became_peered":"2024-09-11T07:26:31.413354+0000","last_unstale":"2024-09-11T07:37:37.605633+0000","last_undegraded":"2024-09-11T07:37:37.605633+0000","last_fullsized":"2024-09-11T07:37:37.605633+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:45.059537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:07.149367+0000","last_clean_scrub_stamp":"2024-09-11T07:36:45.059537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:06.540727+0000","scrub_duration":0.013492418000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":269,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.606799+0000","last_change":"2024-09-11T07:36:40.189921+0000","last_active":"2024-09-11T07:37:37.606799+0000","last_peered":"2024-09-11T07:37:37.606799+0000","last_clean":"2024-09-11T07:37:37.606799+0000","last_became_active":"2024-09-11T07:32:00.508398+0000","last_became_peered":"2024-09-11T07:32:00.508398+0000","last_unstale":"2024-09-11T07:37:37.606799+0000","last_undegraded":"2024-09-11T07:37:37.606799+0000","last_fullsized":"2024-09-11T07:37:37.606799+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:40.189372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:36:40.189372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:47.482245+0000","scrub_duration":0.0077070330000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":250,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.609666+0000","last_change":"2024-09-11T07:36:56.490992+0000","last_active":"2024-09-11T07:37:37.609666+0000","last_peered":"2024-09-11T07:37:37.609666+0000","last_clean":"2024-09-11T07:37:37.609666+0000","last_became_active":"2024-09-11T07:32:00.515745+0000","last_became_peered":"2024-09-11T07:32:00.515745+0000","last_unstale":"2024-09-11T07:37:37.609666+0000","last_undegraded":"2024-09-11T07:37:37.609666+0000","last_fullsized":"2024-09-11T07:37:37.609666+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:56.490940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:56.490940+0000","last_clean_scrub_stamp":"2024-09-11T07:36:56.490940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:58.832227+0000","scrub_duration":0.037860923999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"255'2631","reported_seq":2647,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:54.911718+0000","last_change":"2024-09-11T07:37:26.029935+0000","last_active":"2024-09-11T07:37:54.911718+0000","last_peered":"2024-09-11T07:37:54.911718+0000","last_clean":"2024-09-11T07:37:54.911718+0000","last_became_active":"2024-09-11T07:37:26.029608+0000","last_became_peered":"2024-09-11T07:37:26.029608+0000","last_unstale":"2024-09-11T07:37:54.911718+0000","last_undegraded":"2024-09-11T07:37:54.911718+0000","last_fullsized":"2024-09-11T07:37:54.911718+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":2631,"log_dups_size":0,"ondisk_log_size":2631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:38.232917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":172425216,"num_objects":2631,"num_object_clones":0,"num_object_copies":7893,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2631,"num_write_kb":168384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"234'30","reported_seq":330,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.616529+0000","last_change":"2024-09-11T07:36:45.088277+0000","last_active":"2024-09-11T07:37:37.616529+0000","last_peered":"2024-09-11T07:37:37.616529+0000","last_clean":"2024-09-11T07:37:37.616529+0000","last_became_active":"2024-09-11T07:28:09.018134+0000","last_became_peered":"2024-09-11T07:28:09.018134+0000","last_unstale":"2024-09-11T07:37:37.616529+0000","last_undegraded":"2024-09-11T07:37:37.616529+0000","last_fullsized":"2024-09-11T07:37:37.616529+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'30","last_scrub_stamp":"2024-09-11T07:36:45.087995+0000","last_deep_scrub":"94'18","last_deep_scrub_stamp":"2024-09-11T07:28:51.437397+0000","last_clean_scrub_stamp":"2024-09-11T07:36:45.087995+0000","objects_scrubbed":6,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:47.102679+0000","scrub_duration":0.027742936999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":262,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.615382+0000","last_change":"2024-09-11T07:35:53.257684+0000","last_active":"2024-09-11T07:37:37.615382+0000","last_peered":"2024-09-11T07:37:37.615382+0000","last_clean":"2024-09-11T07:37:37.615382+0000","last_became_active":"2024-09-11T07:26:31.420320+0000","last_became_peered":"2024-09-11T07:26:31.420320+0000","last_unstale":"2024-09-11T07:37:37.615382+0000","last_undegraded":"2024-09-11T07:37:37.615382+0000","last_fullsized":"2024-09-11T07:37:37.615382+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-11T07:35:53.257474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:35:53.257474+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053710479999999998,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":261,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.608489+0000","last_change":"2024-09-11T07:37:26.557037+0000","last_active":"2024-09-11T07:37:37.608489+0000","last_peered":"2024-09-11T07:37:37.608489+0000","last_clean":"2024-09-11T07:37:37.608489+0000","last_became_active":"2024-09-11T07:32:00.520035+0000","last_became_peered":"2024-09-11T07:32:00.520035+0000","last_unstale":"2024-09-11T07:37:37.608489+0000","last_undegraded":"2024-09-11T07:37:37.608489+0000","last_fullsized":"2024-09-11T07:37:37.608489+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:26.556972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:16.967492+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.556972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:43.963183+0000","scrub_duration":0.001503206,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":242,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.611890+0000","last_change":"2024-09-11T07:35:51.187203+0000","last_active":"2024-09-11T07:37:37.611890+0000","last_peered":"2024-09-11T07:37:37.611890+0000","last_clean":"2024-09-11T07:37:37.611890+0000","last_became_active":"2024-09-11T07:26:35.564178+0000","last_became_peered":"2024-09-11T07:26:35.564178+0000","last_unstale":"2024-09-11T07:37:37.611890+0000","last_undegraded":"2024-09-11T07:37:37.611890+0000","last_fullsized":"2024-09-11T07:37:37.611890+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:51.187127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:51.187127+0000","last_clean_scrub_stamp":"2024-09-11T07:35:51.187127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0055970839999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":252,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.609241+0000","last_change":"2024-09-11T07:35:35.789115+0000","last_active":"2024-09-11T07:37:37.609241+0000","last_peered":"2024-09-11T07:37:37.609241+0000","last_clean":"2024-09-11T07:37:37.609241+0000","last_became_active":"2024-09-11T07:26:37.561378+0000","last_became_peered":"2024-09-11T07:26:37.561378+0000","last_unstale":"2024-09-11T07:37:37.609241+0000","last_undegraded":"2024-09-11T07:37:37.609241+0000","last_fullsized":"2024-09-11T07:37:37.609241+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:35.789024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:35:35.789024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079175110000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"255'2526","reported_seq":2540,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:54.847770+0000","last_change":"2024-09-11T07:37:25.854273+0000","last_active":"2024-09-11T07:37:54.847770+0000","last_peered":"2024-09-11T07:37:54.847770+0000","last_clean":"2024-09-11T07:37:54.847770+0000","last_became_active":"2024-09-11T07:37:25.853947+0000","last_became_peered":"2024-09-11T07:37:25.853947+0000","last_unstale":"2024-09-11T07:37:54.847770+0000","last_undegraded":"2024-09-11T07:37:54.847770+0000","last_fullsized":"2024-09-11T07:37:54.847770+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":2526,"log_dups_size":0,"ondisk_log_size":2526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:38.389157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165543936,"num_objects":2526,"num_object_clones":0,"num_object_copies":7578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2526,"num_write_kb":161664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"234'26","reported_seq":105,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:40.649151+0000","last_change":"2024-09-11T07:37:40.649065+0000","last_active":"2024-09-11T07:37:40.649151+0000","last_peered":"2024-09-11T07:37:40.649151+0000","last_clean":"2024-09-11T07:37:40.649151+0000","last_became_active":"2024-09-11T07:36:05.224704+0000","last_became_peered":"2024-09-11T07:36:05.224704+0000","last_unstale":"2024-09-11T07:37:40.649151+0000","last_undegraded":"2024-09-11T07:37:40.649151+0000","last_fullsized":"2024-09-11T07:37:40.649151+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'26","last_scrub_stamp":"2024-09-11T07:37:40.648986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:40.648986+0000","objects_scrubbed":7,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:09.297479+0000","scrub_duration":0.0030578620000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'3","reported_seq":264,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.606811+0000","last_change":"2024-09-11T07:36:54.937298+0000","last_active":"2024-09-11T07:37:37.606811+0000","last_peered":"2024-09-11T07:37:37.606811+0000","last_clean":"2024-09-11T07:37:37.606811+0000","last_became_active":"2024-09-11T07:26:31.419501+0000","last_became_peered":"2024-09-11T07:26:31.419501+0000","last_unstale":"2024-09-11T07:37:37.606811+0000","last_undegraded":"2024-09-11T07:37:37.606811+0000","last_fullsized":"2024-09-11T07:37:37.606811+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-09-11T07:36:54.937201+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_clean_scrub_stamp":"2024-09-11T07:36:54.937201+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:15.893124+0000","scrub_duration":0.007045239,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":276,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.610747+0000","last_change":"2024-09-11T07:37:35.504299+0000","last_active":"2024-09-11T07:37:37.610747+0000","last_peered":"2024-09-11T07:37:37.610747+0000","last_clean":"2024-09-11T07:37:37.610747+0000","last_became_active":"2024-09-11T07:32:00.498002+0000","last_became_peered":"2024-09-11T07:32:00.498002+0000","last_unstale":"2024-09-11T07:37:37.610747+0000","last_undegraded":"2024-09-11T07:37:37.610747+0000","last_fullsized":"2024-09-11T07:37:37.610747+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:35.504238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:35.504238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:46.132916+0000","scrub_duration":0.0013066169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"87'81","reported_seq":277,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.608414+0000","last_change":"2024-09-11T07:36:45.398752+0000","last_active":"2024-09-11T07:37:37.608414+0000","last_peered":"2024-09-11T07:37:37.608414+0000","last_clean":"2024-09-11T07:37:37.608414+0000","last_became_active":"2024-09-11T07:32:00.519656+0000","last_became_peered":"2024-09-11T07:32:00.519656+0000","last_unstale":"2024-09-11T07:37:37.608414+0000","last_undegraded":"2024-09-11T07:37:37.608414+0000","last_fullsized":"2024-09-11T07:37:37.608414+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:36:45.398664+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:36:45.398664+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:47.953625+0000","scrub_duration":0.0041388420000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":247,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:41.562419+0000","last_change":"2024-09-11T07:37:41.562374+0000","last_active":"2024-09-11T07:37:41.562419+0000","last_peered":"2024-09-11T07:37:41.562419+0000","last_clean":"2024-09-11T07:37:41.562419+0000","last_became_active":"2024-09-11T07:32:00.510888+0000","last_became_peered":"2024-09-11T07:32:00.510888+0000","last_unstale":"2024-09-11T07:37:41.562419+0000","last_undegraded":"2024-09-11T07:37:41.562419+0000","last_fullsized":"2024-09-11T07:37:41.562419+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:41.562322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:41.562322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:07.245074+0000","scrub_duration":0.003022734,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":250,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.612043+0000","last_change":"2024-09-11T07:37:23.436064+0000","last_active":"2024-09-11T07:37:37.612043+0000","last_peered":"2024-09-11T07:37:37.612043+0000","last_clean":"2024-09-11T07:37:37.612043+0000","last_became_active":"2024-09-11T07:26:37.559920+0000","last_became_peered":"2024-09-11T07:26:37.559920+0000","last_unstale":"2024-09-11T07:37:37.612043+0000","last_undegraded":"2024-09-11T07:37:37.612043+0000","last_fullsized":"2024-09-11T07:37:37.612043+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:23.436012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:23.436012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:25.352176+0000","scrub_duration":0.001213481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.7","version":"255'2475","reported_seq":2489,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:54.106430+0000","last_change":"2024-09-11T07:37:25.855953+0000","last_active":"2024-09-11T07:37:54.106430+0000","last_peered":"2024-09-11T07:37:54.106430+0000","last_clean":"2024-09-11T07:37:54.106430+0000","last_became_active":"2024-09-11T07:37:25.855615+0000","last_became_peered":"2024-09-11T07:37:25.855615+0000","last_unstale":"2024-09-11T07:37:54.106430+0000","last_undegraded":"2024-09-11T07:37:54.106430+0000","last_fullsized":"2024-09-11T07:37:54.106430+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":2475,"log_dups_size":0,"ondisk_log_size":2475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:25.900228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":162201600,"num_objects":2475,"num_object_clones":0,"num_object_copies":7425,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2475,"num_write_kb":158400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"234'35","reported_seq":110,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.609584+0000","last_change":"2024-09-11T07:36:05.226411+0000","last_active":"2024-09-11T07:37:37.609584+0000","last_peered":"2024-09-11T07:37:37.609584+0000","last_clean":"2024-09-11T07:37:37.609584+0000","last_became_active":"2024-09-11T07:36:05.226021+0000","last_became_peered":"2024-09-11T07:36:05.226021+0000","last_unstale":"2024-09-11T07:37:37.609584+0000","last_undegraded":"2024-09-11T07:37:37.609584+0000","last_fullsized":"2024-09-11T07:37:37.609584+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-11T07:35:29.151274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:35:29.151274+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":266,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.606503+0000","last_change":"2024-09-11T07:36:48.089096+0000","last_active":"2024-09-11T07:37:37.606503+0000","last_peered":"2024-09-11T07:37:37.606503+0000","last_clean":"2024-09-11T07:37:37.606503+0000","last_became_active":"2024-09-11T07:26:31.419560+0000","last_became_peered":"2024-09-11T07:26:31.419560+0000","last_unstale":"2024-09-11T07:37:37.606503+0000","last_undegraded":"2024-09-11T07:37:37.606503+0000","last_fullsized":"2024-09-11T07:37:37.606503+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:48.088795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:48.088795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:06.111523+0000","scrub_duration":0.0081450789999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":262,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.606326+0000","last_change":"2024-09-11T07:37:17.420289+0000","last_active":"2024-09-11T07:37:37.606326+0000","last_peered":"2024-09-11T07:37:37.606326+0000","last_clean":"2024-09-11T07:37:37.606326+0000","last_became_active":"2024-09-11T07:26:35.564044+0000","last_became_peered":"2024-09-11T07:26:35.564044+0000","last_unstale":"2024-09-11T07:37:37.606326+0000","last_undegraded":"2024-09-11T07:37:37.606326+0000","last_fullsized":"2024-09-11T07:37:37.606326+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:17.420236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:17.420236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:47.156712+0000","scrub_duration":0.0012713080000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":245,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.608380+0000","last_change":"2024-09-11T07:36:31.411708+0000","last_active":"2024-09-11T07:37:37.608380+0000","last_peered":"2024-09-11T07:37:37.608380+0000","last_clean":"2024-09-11T07:37:37.608380+0000","last_became_active":"2024-09-11T07:32:00.516916+0000","last_became_peered":"2024-09-11T07:32:00.516916+0000","last_unstale":"2024-09-11T07:37:37.608380+0000","last_undegraded":"2024-09-11T07:37:37.608380+0000","last_fullsized":"2024-09-11T07:37:37.608380+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:31.411554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:31.411554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:49.766518+0000","scrub_duration":0.002948132,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"255'2594","reported_seq":2609,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:54.129469+0000","last_change":"2024-09-11T07:37:25.854298+0000","last_active":"2024-09-11T07:37:54.129469+0000","last_peered":"2024-09-11T07:37:54.129469+0000","last_clean":"2024-09-11T07:37:54.129469+0000","last_became_active":"2024-09-11T07:37:25.854037+0000","last_became_peered":"2024-09-11T07:37:25.854037+0000","last_unstale":"2024-09-11T07:37:54.129469+0000","last_undegraded":"2024-09-11T07:37:54.129469+0000","last_fullsized":"2024-09-11T07:37:54.129469+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":2594,"log_dups_size":0,"ondisk_log_size":2594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:45.294877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170000384,"num_objects":2594,"num_object_clones":0,"num_object_copies":7782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2594,"num_write_kb":166016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"234'15","reported_seq":279,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.614961+0000","last_change":"2024-09-11T07:36:46.033480+0000","last_active":"2024-09-11T07:37:37.614961+0000","last_peered":"2024-09-11T07:37:37.614961+0000","last_clean":"2024-09-11T07:37:37.614961+0000","last_became_active":"2024-09-11T07:26:33.543148+0000","last_became_peered":"2024-09-11T07:26:33.543148+0000","last_unstale":"2024-09-11T07:37:37.614961+0000","last_undegraded":"2024-09-11T07:37:37.614961+0000","last_fullsized":"2024-09-11T07:37:37.614961+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:36:46.033388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:36:46.033388+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:00.259650+0000","scrub_duration":0.0084720030000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":265,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.610564+0000","last_change":"2024-09-11T07:37:26.019680+0000","last_active":"2024-09-11T07:37:37.610564+0000","last_peered":"2024-09-11T07:37:37.610564+0000","last_clean":"2024-09-11T07:37:37.610564+0000","last_became_active":"2024-09-11T07:26:31.423056+0000","last_became_peered":"2024-09-11T07:26:31.423056+0000","last_unstale":"2024-09-11T07:37:37.610564+0000","last_undegraded":"2024-09-11T07:37:37.610564+0000","last_fullsized":"2024-09-11T07:37:37.610564+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:26.019623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.019623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:43.097013+0000","scrub_duration":0.0013393599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":262,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.605915+0000","last_change":"2024-09-11T07:36:39.141707+0000","last_active":"2024-09-11T07:37:37.605915+0000","last_peered":"2024-09-11T07:37:37.605915+0000","last_clean":"2024-09-11T07:37:37.605915+0000","last_became_active":"2024-09-11T07:26:35.563182+0000","last_became_peered":"2024-09-11T07:26:35.563182+0000","last_unstale":"2024-09-11T07:37:37.605915+0000","last_undegraded":"2024-09-11T07:37:37.605915+0000","last_fullsized":"2024-09-11T07:37:37.605915+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:39.141620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:36:39.141620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:54.791536+0000","scrub_duration":0.01162813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":267,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.615352+0000","last_change":"2024-09-11T07:36:36.023079+0000","last_active":"2024-09-11T07:37:37.615352+0000","last_peered":"2024-09-11T07:37:37.615352+0000","last_clean":"2024-09-11T07:37:37.615352+0000","last_became_active":"2024-09-11T07:32:00.505185+0000","last_became_peered":"2024-09-11T07:32:00.505185+0000","last_unstale":"2024-09-11T07:37:37.615352+0000","last_undegraded":"2024-09-11T07:37:37.615352+0000","last_fullsized":"2024-09-11T07:37:37.615352+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:36.023006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:46.944763+0000","last_clean_scrub_stamp":"2024-09-11T07:36:36.023006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:02.683265+0000","scrub_duration":0.011934019000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.5","version":"255'2548","reported_seq":2564,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:54.900696+0000","last_change":"2024-09-11T07:37:25.854781+0000","last_active":"2024-09-11T07:37:54.900696+0000","last_peered":"2024-09-11T07:37:54.900696+0000","last_clean":"2024-09-11T07:37:54.900696+0000","last_became_active":"2024-09-11T07:37:25.854037+0000","last_became_peered":"2024-09-11T07:37:25.854037+0000","last_unstale":"2024-09-11T07:37:54.900696+0000","last_undegraded":"2024-09-11T07:37:54.900696+0000","last_fullsized":"2024-09-11T07:37:54.900696+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":2548,"log_dups_size":0,"ondisk_log_size":2548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:46.537669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":166985728,"num_objects":2548,"num_object_clones":0,"num_object_copies":7644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2548,"num_write_kb":163072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"234'27","reported_seq":307,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.610151+0000","last_change":"2024-09-11T07:36:32.341522+0000","last_active":"2024-09-11T07:37:37.610151+0000","last_peered":"2024-09-11T07:37:37.610151+0000","last_clean":"2024-09-11T07:37:37.610151+0000","last_became_active":"2024-09-11T07:32:00.521464+0000","last_became_peered":"2024-09-11T07:32:00.521464+0000","last_unstale":"2024-09-11T07:37:37.610151+0000","last_undegraded":"2024-09-11T07:37:37.610151+0000","last_fullsized":"2024-09-11T07:37:37.610151+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-09-11T07:36:32.341462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:36:32.341462+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:51.691479+0000","scrub_duration":0.0043984430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":283,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.606891+0000","last_change":"2024-09-11T07:36:37.142765+0000","last_active":"2024-09-11T07:37:37.606891+0000","last_peered":"2024-09-11T07:37:37.606891+0000","last_clean":"2024-09-11T07:37:37.606891+0000","last_became_active":"2024-09-11T07:32:00.513949+0000","last_became_peered":"2024-09-11T07:32:00.513949+0000","last_unstale":"2024-09-11T07:37:37.606891+0000","last_undegraded":"2024-09-11T07:37:37.606891+0000","last_fullsized":"2024-09-11T07:37:37.606891+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:37.142658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_clean_scrub_stamp":"2024-09-11T07:36:37.142658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:49.494264+0000","scrub_duration":0.011370480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"151'10","reported_seq":357,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:44.639747+0000","last_change":"2024-09-11T07:37:38.544127+0000","last_active":"2024-09-11T07:37:44.639747+0000","last_peered":"2024-09-11T07:37:44.639747+0000","last_clean":"2024-09-11T07:37:44.639747+0000","last_became_active":"2024-09-11T07:32:00.506806+0000","last_became_peered":"2024-09-11T07:32:00.506806+0000","last_unstale":"2024-09-11T07:37:44.639747+0000","last_undegraded":"2024-09-11T07:37:44.639747+0000","last_fullsized":"2024-09-11T07:37:44.639747+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:37:38.544072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:38.544072+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:47.377252+0000","scrub_duration":0.0027401579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":250,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.610741+0000","last_change":"2024-09-11T07:36:44.295056+0000","last_active":"2024-09-11T07:37:37.610741+0000","last_peered":"2024-09-11T07:37:37.610741+0000","last_clean":"2024-09-11T07:37:37.610741+0000","last_became_active":"2024-09-11T07:26:37.861134+0000","last_became_peered":"2024-09-11T07:26:37.861134+0000","last_unstale":"2024-09-11T07:37:37.610741+0000","last_undegraded":"2024-09-11T07:37:37.610741+0000","last_fullsized":"2024-09-11T07:37:37.610741+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:44.293653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:44.293653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:57.404591+0000","scrub_duration":0.0074032500000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.4","version":"255'2501","reported_seq":2516,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:54.449342+0000","last_change":"2024-09-11T07:37:25.852069+0000","last_active":"2024-09-11T07:37:54.449342+0000","last_peered":"2024-09-11T07:37:54.449342+0000","last_clean":"2024-09-11T07:37:54.449342+0000","last_became_active":"2024-09-11T07:37:25.851789+0000","last_became_peered":"2024-09-11T07:37:25.851789+0000","last_unstale":"2024-09-11T07:37:54.449342+0000","last_undegraded":"2024-09-11T07:37:54.449342+0000","last_fullsized":"2024-09-11T07:37:54.449342+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":2501,"log_dups_size":0,"ondisk_log_size":2501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:40.599431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163905536,"num_objects":2501,"num_object_clones":0,"num_object_copies":7503,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2501,"num_write_kb":160064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"234'23","reported_seq":291,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.603989+0000","last_change":"2024-09-11T07:37:34.613363+0000","last_active":"2024-09-11T07:37:37.603989+0000","last_peered":"2024-09-11T07:37:37.603989+0000","last_clean":"2024-09-11T07:37:37.603989+0000","last_became_active":"2024-09-11T07:26:33.540363+0000","last_became_peered":"2024-09-11T07:26:33.540363+0000","last_unstale":"2024-09-11T07:37:37.603989+0000","last_undegraded":"2024-09-11T07:37:37.603989+0000","last_fullsized":"2024-09-11T07:37:37.603989+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'23","last_scrub_stamp":"2024-09-11T07:37:34.613280+0000","last_deep_scrub":"234'23","last_deep_scrub_stamp":"2024-09-11T07:37:34.613280+0000","last_clean_scrub_stamp":"2024-09-11T07:37:34.613280+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:45.821230+0000","scrub_duration":0.002722604,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":283,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.605343+0000","last_change":"2024-09-11T07:35:46.769144+0000","last_active":"2024-09-11T07:37:37.605343+0000","last_peered":"2024-09-11T07:37:37.605343+0000","last_clean":"2024-09-11T07:37:37.605343+0000","last_became_active":"2024-09-11T07:32:00.504344+0000","last_became_peered":"2024-09-11T07:32:00.504344+0000","last_unstale":"2024-09-11T07:37:37.605343+0000","last_undegraded":"2024-09-11T07:37:37.605343+0000","last_fullsized":"2024-09-11T07:37:37.605343+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:35:46.769070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:35:46.769070+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049213720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":262,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.605615+0000","last_change":"2024-09-11T07:36:43.125248+0000","last_active":"2024-09-11T07:37:37.605615+0000","last_peered":"2024-09-11T07:37:37.605615+0000","last_clean":"2024-09-11T07:37:37.605615+0000","last_became_active":"2024-09-11T07:26:35.561967+0000","last_became_peered":"2024-09-11T07:26:35.561967+0000","last_unstale":"2024-09-11T07:37:37.605615+0000","last_undegraded":"2024-09-11T07:37:37.605615+0000","last_fullsized":"2024-09-11T07:37:37.605615+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:43.125187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:44.820354+0000","last_clean_scrub_stamp":"2024-09-11T07:36:43.125187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:59.275217+0000","scrub_duration":0.0061424100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":267,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.605339+0000","last_change":"2024-09-11T07:37:16.860986+0000","last_active":"2024-09-11T07:37:37.605339+0000","last_peered":"2024-09-11T07:37:37.605339+0000","last_clean":"2024-09-11T07:37:37.605339+0000","last_became_active":"2024-09-11T07:32:00.502019+0000","last_became_peered":"2024-09-11T07:32:00.502019+0000","last_unstale":"2024-09-11T07:37:37.605339+0000","last_undegraded":"2024-09-11T07:37:37.605339+0000","last_fullsized":"2024-09-11T07:37:37.605339+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:16.860930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:16.860930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:35.856490+0000","scrub_duration":0.0024991000000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"234'15","reported_seq":296,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.616443+0000","last_change":"2024-09-11T07:37:19.096890+0000","last_active":"2024-09-11T07:37:37.616443+0000","last_peered":"2024-09-11T07:37:37.616443+0000","last_clean":"2024-09-11T07:37:37.616443+0000","last_became_active":"2024-09-11T07:32:00.509412+0000","last_became_peered":"2024-09-11T07:32:00.509412+0000","last_unstale":"2024-09-11T07:37:37.616443+0000","last_undegraded":"2024-09-11T07:37:37.616443+0000","last_fullsized":"2024-09-11T07:37:37.616443+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:37:19.096817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:19.096817+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:38.083588+0000","scrub_duration":0.00175015,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":266,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.605675+0000","last_change":"2024-09-11T07:36:53.065329+0000","last_active":"2024-09-11T07:37:37.605675+0000","last_peered":"2024-09-11T07:37:37.605675+0000","last_clean":"2024-09-11T07:37:37.605675+0000","last_became_active":"2024-09-11T07:26:31.415701+0000","last_became_peered":"2024-09-11T07:26:31.415701+0000","last_unstale":"2024-09-11T07:37:37.605675+0000","last_undegraded":"2024-09-11T07:37:37.605675+0000","last_fullsized":"2024-09-11T07:37:37.605675+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:53.065247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.925200+0000","last_clean_scrub_stamp":"2024-09-11T07:36:53.065247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:59.828393+0000","scrub_duration":0.0023028290000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":259,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.606157+0000","last_change":"2024-09-11T07:36:32.708578+0000","last_active":"2024-09-11T07:37:37.606157+0000","last_peered":"2024-09-11T07:37:37.606157+0000","last_clean":"2024-09-11T07:37:37.606157+0000","last_became_active":"2024-09-11T07:32:00.499598+0000","last_became_peered":"2024-09-11T07:32:00.499598+0000","last_unstale":"2024-09-11T07:37:37.606157+0000","last_undegraded":"2024-09-11T07:37:37.606157+0000","last_fullsized":"2024-09-11T07:37:37.606157+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:32.708500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:35.392360+0000","last_clean_scrub_stamp":"2024-09-11T07:36:32.708500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:55.503565+0000","scrub_duration":0.0032820800000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":268,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.606060+0000","last_change":"2024-09-11T07:36:35.186527+0000","last_active":"2024-09-11T07:37:37.606060+0000","last_peered":"2024-09-11T07:37:37.606060+0000","last_clean":"2024-09-11T07:37:37.606060+0000","last_became_active":"2024-09-11T07:32:00.506697+0000","last_became_peered":"2024-09-11T07:32:00.506697+0000","last_unstale":"2024-09-11T07:37:37.606060+0000","last_undegraded":"2024-09-11T07:37:37.606060+0000","last_fullsized":"2024-09-11T07:37:37.606060+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:36:35.185293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:35.185293+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:50.456483+0000","scrub_duration":0.0039808639999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":53,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.604138+0000","last_change":"2024-09-11T07:36:51.617809+0000","last_active":"2024-09-11T07:37:37.604138+0000","last_peered":"2024-09-11T07:37:37.604138+0000","last_clean":"2024-09-11T07:37:37.604138+0000","last_became_active":"2024-09-11T07:36:05.225791+0000","last_became_peered":"2024-09-11T07:36:05.225791+0000","last_unstale":"2024-09-11T07:37:37.604138+0000","last_undegraded":"2024-09-11T07:37:37.604138+0000","last_fullsized":"2024-09-11T07:37:37.604138+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:51.617730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:51.617730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:06.231089+0000","scrub_duration":0.001776197,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"234'15","reported_seq":70,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.611618+0000","last_change":"2024-09-11T07:37:26.450184+0000","last_active":"2024-09-11T07:37:37.611618+0000","last_peered":"2024-09-11T07:37:37.611618+0000","last_clean":"2024-09-11T07:37:37.611618+0000","last_became_active":"2024-09-11T07:35:50.100154+0000","last_became_peered":"2024-09-11T07:35:50.100154+0000","last_unstale":"2024-09-11T07:37:37.611618+0000","last_undegraded":"2024-09-11T07:37:37.611618+0000","last_fullsized":"2024-09-11T07:37:37.611618+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:37:26.450122+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.450122+0000","objects_scrubbed":3,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:32.005494+0000","scrub_duration":0.001711052,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":262,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.610914+0000","last_change":"2024-09-11T07:37:18.340651+0000","last_active":"2024-09-11T07:37:37.610914+0000","last_peered":"2024-09-11T07:37:37.610914+0000","last_clean":"2024-09-11T07:37:37.610914+0000","last_became_active":"2024-09-11T07:26:35.566894+0000","last_became_peered":"2024-09-11T07:26:35.566894+0000","last_unstale":"2024-09-11T07:37:37.610914+0000","last_undegraded":"2024-09-11T07:37:37.610914+0000","last_fullsized":"2024-09-11T07:37:37.610914+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:18.340578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:37:18.340578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:21.567929+0000","scrub_duration":0.0054750240000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":248,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.609927+0000","last_change":"2024-09-11T07:36:57.461696+0000","last_active":"2024-09-11T07:37:37.609927+0000","last_peered":"2024-09-11T07:37:37.609927+0000","last_clean":"2024-09-11T07:37:37.609927+0000","last_became_active":"2024-09-11T07:32:00.514100+0000","last_became_peered":"2024-09-11T07:32:00.514100+0000","last_unstale":"2024-09-11T07:37:37.609927+0000","last_undegraded":"2024-09-11T07:37:37.609927+0000","last_fullsized":"2024-09-11T07:37:37.609927+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:57.461617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:57.461617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:06.770396+0000","scrub_duration":0.028762672999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"234'25","reported_seq":318,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.607386+0000","last_change":"2024-09-11T07:36:55.966810+0000","last_active":"2024-09-11T07:37:37.607386+0000","last_peered":"2024-09-11T07:37:37.607386+0000","last_clean":"2024-09-11T07:37:37.607386+0000","last_became_active":"2024-09-11T07:32:00.504463+0000","last_became_peered":"2024-09-11T07:32:00.504463+0000","last_unstale":"2024-09-11T07:37:37.607386+0000","last_undegraded":"2024-09-11T07:37:37.607386+0000","last_fullsized":"2024-09-11T07:37:37.607386+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'25","last_scrub_stamp":"2024-09-11T07:36:55.966595+0000","last_deep_scrub":"86'15","last_deep_scrub_stamp":"2024-09-11T07:28:22.858137+0000","last_clean_scrub_stamp":"2024-09-11T07:36:55.966595+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:24.512562+0000","scrub_duration":0.032608074000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":266,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.612377+0000","last_change":"2024-09-11T07:36:34.405309+0000","last_active":"2024-09-11T07:37:37.612377+0000","last_peered":"2024-09-11T07:37:37.612377+0000","last_clean":"2024-09-11T07:37:37.612377+0000","last_became_active":"2024-09-11T07:32:00.498473+0000","last_became_peered":"2024-09-11T07:32:00.498473+0000","last_unstale":"2024-09-11T07:37:37.612377+0000","last_undegraded":"2024-09-11T07:37:37.612377+0000","last_fullsized":"2024-09-11T07:37:37.612377+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:34.405241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:34.405241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039370459999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":396,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:44.638799+0000","last_change":"2024-09-11T07:37:17.360240+0000","last_active":"2024-09-11T07:37:44.638799+0000","last_peered":"2024-09-11T07:37:44.638799+0000","last_clean":"2024-09-11T07:37:44.638799+0000","last_became_active":"2024-09-11T07:26:35.562245+0000","last_became_peered":"2024-09-11T07:26:35.562245+0000","last_unstale":"2024-09-11T07:37:44.638799+0000","last_undegraded":"2024-09-11T07:37:44.638799+0000","last_fullsized":"2024-09-11T07:37:44.638799+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:37:17.360183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:17.360183+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:33.736883+0000","scrub_duration":0.0016828819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":277,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:42.084471+0000","last_change":"2024-09-11T07:37:42.084410+0000","last_active":"2024-09-11T07:37:42.084471+0000","last_peered":"2024-09-11T07:37:42.084471+0000","last_clean":"2024-09-11T07:37:42.084471+0000","last_became_active":"2024-09-11T07:32:00.509295+0000","last_became_peered":"2024-09-11T07:32:00.509295+0000","last_unstale":"2024-09-11T07:37:42.084471+0000","last_undegraded":"2024-09-11T07:37:42.084471+0000","last_fullsized":"2024-09-11T07:37:42.084471+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:42.084305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:42.084305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:53.270508+0000","scrub_duration":0.0047035039999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"234'19","reported_seq":303,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:41.338249+0000","last_change":"2024-09-11T07:37:41.338132+0000","last_active":"2024-09-11T07:37:41.338249+0000","last_peered":"2024-09-11T07:37:41.338249+0000","last_clean":"2024-09-11T07:37:41.338249+0000","last_became_active":"2024-09-11T07:32:00.505134+0000","last_became_peered":"2024-09-11T07:32:00.505134+0000","last_unstale":"2024-09-11T07:37:41.338249+0000","last_undegraded":"2024-09-11T07:37:41.338249+0000","last_fullsized":"2024-09-11T07:37:41.338249+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:37:41.338036+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:37:41.338036+0000","last_clean_scrub_stamp":"2024-09-11T07:37:41.338036+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:08.290067+0000","scrub_duration":0.0053417279999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":256,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.615170+0000","last_change":"2024-09-11T07:36:27.958468+0000","last_active":"2024-09-11T07:37:37.615170+0000","last_peered":"2024-09-11T07:37:37.615170+0000","last_clean":"2024-09-11T07:37:37.615170+0000","last_became_active":"2024-09-11T07:26:31.419409+0000","last_became_peered":"2024-09-11T07:26:31.419409+0000","last_unstale":"2024-09-11T07:37:37.615170+0000","last_undegraded":"2024-09-11T07:37:37.615170+0000","last_fullsized":"2024-09-11T07:37:37.615170+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:27.958386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:27.958386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:55.062218+0000","scrub_duration":0.0093493710000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"151'10","reported_seq":399,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:49.639223+0000","last_change":"2024-09-11T07:35:46.977819+0000","last_active":"2024-09-11T07:37:49.639223+0000","last_peered":"2024-09-11T07:37:49.639223+0000","last_clean":"2024-09-11T07:37:49.639223+0000","last_became_active":"2024-09-11T07:32:00.504062+0000","last_became_peered":"2024-09-11T07:32:00.504062+0000","last_unstale":"2024-09-11T07:37:49.639223+0000","last_undegraded":"2024-09-11T07:37:49.639223+0000","last_fullsized":"2024-09-11T07:37:49.639223+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:35:46.977611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:35:46.977611+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012282416000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":250,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.604989+0000","last_change":"2024-09-11T07:37:26.659394+0000","last_active":"2024-09-11T07:37:37.604989+0000","last_peered":"2024-09-11T07:37:37.604989+0000","last_clean":"2024-09-11T07:37:37.604989+0000","last_became_active":"2024-09-11T07:26:37.565723+0000","last_became_peered":"2024-09-11T07:26:37.565723+0000","last_unstale":"2024-09-11T07:37:37.604989+0000","last_undegraded":"2024-09-11T07:37:37.604989+0000","last_fullsized":"2024-09-11T07:37:37.604989+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:26.659337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.659337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:32.875069+0000","scrub_duration":0.0012563839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"236'16","reported_seq":289,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.609664+0000","last_change":"2024-09-11T07:36:49.809594+0000","last_active":"2024-09-11T07:37:37.609664+0000","last_peered":"2024-09-11T07:37:37.609664+0000","last_clean":"2024-09-11T07:37:37.609664+0000","last_became_active":"2024-09-11T07:26:33.540688+0000","last_became_peered":"2024-09-11T07:26:33.540688+0000","last_unstale":"2024-09-11T07:37:37.609664+0000","last_undegraded":"2024-09-11T07:37:37.609664+0000","last_fullsized":"2024-09-11T07:37:37.609664+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'16","last_scrub_stamp":"2024-09-11T07:36:49.809484+0000","last_deep_scrub":"236'16","last_deep_scrub_stamp":"2024-09-11T07:36:49.809484+0000","last_clean_scrub_stamp":"2024-09-11T07:36:49.809484+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:12.361926+0000","scrub_duration":0.017423357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":52,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.607873+0000","last_change":"2024-09-11T07:37:22.914840+0000","last_active":"2024-09-11T07:37:37.607873+0000","last_peered":"2024-09-11T07:37:37.607873+0000","last_clean":"2024-09-11T07:37:37.607873+0000","last_became_active":"2024-09-11T07:36:05.225465+0000","last_became_peered":"2024-09-11T07:36:05.225465+0000","last_unstale":"2024-09-11T07:37:37.607873+0000","last_undegraded":"2024-09-11T07:37:37.607873+0000","last_fullsized":"2024-09-11T07:37:37.607873+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:22.914781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:09.772000+0000","last_clean_scrub_stamp":"2024-09-11T07:37:22.914781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:37.135943+0000","scrub_duration":0.001446864,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":242,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.605416+0000","last_change":"2024-09-11T07:35:51.886845+0000","last_active":"2024-09-11T07:37:37.605416+0000","last_peered":"2024-09-11T07:37:37.605416+0000","last_clean":"2024-09-11T07:37:37.605416+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:37:37.605416+0000","last_undegraded":"2024-09-11T07:37:37.605416+0000","last_fullsized":"2024-09-11T07:37:37.605416+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:51.886407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:35:51.886407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002900241,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":260,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.613587+0000","last_change":"2024-09-11T07:37:32.186622+0000","last_active":"2024-09-11T07:37:37.613587+0000","last_peered":"2024-09-11T07:37:37.613587+0000","last_clean":"2024-09-11T07:37:37.613587+0000","last_became_active":"2024-09-11T07:26:37.568886+0000","last_became_peered":"2024-09-11T07:26:37.568886+0000","last_unstale":"2024-09-11T07:37:37.613587+0000","last_undegraded":"2024-09-11T07:37:37.613587+0000","last_fullsized":"2024-09-11T07:37:37.613587+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:32.186561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:32.186561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:34.230574+0000","scrub_duration":0.0014950059999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"234'24","reported_seq":290,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.604412+0000","last_change":"2024-09-11T07:36:31.789902+0000","last_active":"2024-09-11T07:37:37.604412+0000","last_peered":"2024-09-11T07:37:37.604412+0000","last_clean":"2024-09-11T07:37:37.604412+0000","last_became_active":"2024-09-11T07:26:33.540208+0000","last_became_peered":"2024-09-11T07:26:33.540208+0000","last_unstale":"2024-09-11T07:37:37.604412+0000","last_undegraded":"2024-09-11T07:37:37.604412+0000","last_fullsized":"2024-09-11T07:37:37.604412+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'16","last_scrub_stamp":"2024-09-11T07:36:31.789841+0000","last_deep_scrub":"136'16","last_deep_scrub_stamp":"2024-09-11T07:31:06.039023+0000","last_clean_scrub_stamp":"2024-09-11T07:36:31.789841+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:58.106789+0000","scrub_duration":0.0048561150000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":258,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.610494+0000","last_change":"2024-09-11T07:36:35.957044+0000","last_active":"2024-09-11T07:37:37.610494+0000","last_peered":"2024-09-11T07:37:37.610494+0000","last_clean":"2024-09-11T07:37:37.610494+0000","last_became_active":"2024-09-11T07:26:31.422671+0000","last_became_peered":"2024-09-11T07:26:31.422671+0000","last_unstale":"2024-09-11T07:37:37.610494+0000","last_undegraded":"2024-09-11T07:37:37.610494+0000","last_fullsized":"2024-09-11T07:37:37.610494+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:35.956943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:36:35.956943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053643689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":244,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.609114+0000","last_change":"2024-09-11T07:36:33.978210+0000","last_active":"2024-09-11T07:37:37.609114+0000","last_peered":"2024-09-11T07:37:37.609114+0000","last_clean":"2024-09-11T07:37:37.609114+0000","last_became_active":"2024-09-11T07:26:35.566931+0000","last_became_peered":"2024-09-11T07:26:35.566931+0000","last_unstale":"2024-09-11T07:37:37.609114+0000","last_undegraded":"2024-09-11T07:37:37.609114+0000","last_fullsized":"2024-09-11T07:37:37.609114+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:33.978003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.847474+0000","last_clean_scrub_stamp":"2024-09-11T07:36:33.978003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:44.376038+0000","scrub_duration":0.012287529,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":240,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.605023+0000","last_change":"2024-09-11T07:36:43.814458+0000","last_active":"2024-09-11T07:37:37.605023+0000","last_peered":"2024-09-11T07:37:37.605023+0000","last_clean":"2024-09-11T07:37:37.605023+0000","last_became_active":"2024-09-11T07:26:37.561573+0000","last_became_peered":"2024-09-11T07:26:37.561573+0000","last_unstale":"2024-09-11T07:37:37.605023+0000","last_undegraded":"2024-09-11T07:37:37.605023+0000","last_fullsized":"2024-09-11T07:37:37.605023+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:43.814370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:43.814370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:10.518232+0000","scrub_duration":0.0093820789999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"234'10","reported_seq":268,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.605093+0000","last_change":"2024-09-11T07:36:46.900027+0000","last_active":"2024-09-11T07:37:37.605093+0000","last_peered":"2024-09-11T07:37:37.605093+0000","last_clean":"2024-09-11T07:37:37.605093+0000","last_became_active":"2024-09-11T07:26:33.545830+0000","last_became_peered":"2024-09-11T07:26:33.545830+0000","last_unstale":"2024-09-11T07:37:37.605093+0000","last_undegraded":"2024-09-11T07:37:37.605093+0000","last_fullsized":"2024-09-11T07:37:37.605093+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'10","last_scrub_stamp":"2024-09-11T07:36:46.899947+0000","last_deep_scrub":"82'6","last_deep_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_clean_scrub_stamp":"2024-09-11T07:36:46.899947+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:02.797069+0000","scrub_duration":0.003412277,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":256,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.611578+0000","last_change":"2024-09-11T07:36:50.285861+0000","last_active":"2024-09-11T07:37:37.611578+0000","last_peered":"2024-09-11T07:37:37.611578+0000","last_clean":"2024-09-11T07:37:37.611578+0000","last_became_active":"2024-09-11T07:26:31.420518+0000","last_became_peered":"2024-09-11T07:26:31.420518+0000","last_unstale":"2024-09-11T07:37:37.611578+0000","last_undegraded":"2024-09-11T07:37:37.611578+0000","last_fullsized":"2024-09-11T07:37:37.611578+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:50.285756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_clean_scrub_stamp":"2024-09-11T07:36:50.285756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:51.131474+0000","scrub_duration":0.0020130360000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":177,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.603836+0000","last_change":"2024-09-11T07:36:47.675100+0000","last_active":"2024-09-11T07:37:37.603836+0000","last_peered":"2024-09-11T07:37:37.603836+0000","last_clean":"2024-09-11T07:37:37.603836+0000","last_became_active":"2024-09-11T07:36:05.226194+0000","last_became_peered":"2024-09-11T07:36:05.226194+0000","last_unstale":"2024-09-11T07:37:37.603836+0000","last_undegraded":"2024-09-11T07:37:37.603836+0000","last_fullsized":"2024-09-11T07:37:37.603836+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:47.675050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:47.675050+0000","last_clean_scrub_stamp":"2024-09-11T07:36:47.675050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:50.604543+0000","scrub_duration":0.00141387,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":250,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.608408+0000","last_change":"2024-09-11T07:36:50.407702+0000","last_active":"2024-09-11T07:37:37.608408+0000","last_peered":"2024-09-11T07:37:37.608408+0000","last_clean":"2024-09-11T07:37:37.608408+0000","last_became_active":"2024-09-11T07:32:00.511407+0000","last_became_peered":"2024-09-11T07:32:00.511407+0000","last_unstale":"2024-09-11T07:37:37.608408+0000","last_undegraded":"2024-09-11T07:37:37.608408+0000","last_fullsized":"2024-09-11T07:37:37.608408+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:50.407649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:50.407649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:52.000990+0000","scrub_duration":0.01288723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"234'19","reported_seq":282,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.605772+0000","last_change":"2024-09-11T07:35:32.805542+0000","last_active":"2024-09-11T07:37:37.605772+0000","last_peered":"2024-09-11T07:37:37.605772+0000","last_clean":"2024-09-11T07:37:37.605772+0000","last_became_active":"2024-09-11T07:32:00.502081+0000","last_became_peered":"2024-09-11T07:32:00.502081+0000","last_unstale":"2024-09-11T07:37:37.605772+0000","last_undegraded":"2024-09-11T07:37:37.605772+0000","last_fullsized":"2024-09-11T07:37:37.605772+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:35:32.805318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:35:32.805318+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0068191889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":42,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.612248+0000","last_change":"2024-09-11T07:36:05.228927+0000","last_active":"2024-09-11T07:37:37.612248+0000","last_peered":"2024-09-11T07:37:37.612248+0000","last_clean":"2024-09-11T07:37:37.612248+0000","last_became_active":"2024-09-11T07:36:05.228360+0000","last_became_peered":"2024-09-11T07:36:05.228360+0000","last_unstale":"2024-09-11T07:37:37.612248+0000","last_undegraded":"2024-09-11T07:37:37.612248+0000","last_fullsized":"2024-09-11T07:37:37.612248+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:48.366931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:48.366931+0000","last_clean_scrub_stamp":"2024-09-11T07:35:48.366931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":223,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.608232+0000","last_change":"2024-09-11T07:36:36.444436+0000","last_active":"2024-09-11T07:37:37.608232+0000","last_peered":"2024-09-11T07:37:37.608232+0000","last_clean":"2024-09-11T07:37:37.608232+0000","last_became_active":"2024-09-11T07:32:00.520534+0000","last_became_peered":"2024-09-11T07:32:00.520534+0000","last_unstale":"2024-09-11T07:37:37.608232+0000","last_undegraded":"2024-09-11T07:37:37.608232+0000","last_fullsized":"2024-09-11T07:37:37.608232+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:36.444301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:39.021876+0000","last_clean_scrub_stamp":"2024-09-11T07:36:36.444301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:01.603071+0000","scrub_duration":0.0015961,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":240,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.605692+0000","last_change":"2024-09-11T07:36:49.899993+0000","last_active":"2024-09-11T07:37:37.605692+0000","last_peered":"2024-09-11T07:37:37.605692+0000","last_clean":"2024-09-11T07:37:37.605692+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:37:37.605692+0000","last_undegraded":"2024-09-11T07:37:37.605692+0000","last_fullsized":"2024-09-11T07:37:37.605692+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:49.899620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:49.899620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:12.310909+0000","scrub_duration":0.014021288999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":227,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.609020+0000","last_change":"2024-09-11T07:36:44.447910+0000","last_active":"2024-09-11T07:37:37.609020+0000","last_peered":"2024-09-11T07:37:37.609020+0000","last_clean":"2024-09-11T07:37:37.609020+0000","last_became_active":"2024-09-11T07:32:00.511258+0000","last_became_peered":"2024-09-11T07:32:00.511258+0000","last_unstale":"2024-09-11T07:37:37.609020+0000","last_undegraded":"2024-09-11T07:37:37.609020+0000","last_fullsized":"2024-09-11T07:37:37.609020+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:36:44.447809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:44.447809+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:07.090809+0000","scrub_duration":0.005540555,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":275,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.610232+0000","last_change":"2024-09-11T07:36:31.353614+0000","last_active":"2024-09-11T07:37:37.610232+0000","last_peered":"2024-09-11T07:37:37.610232+0000","last_clean":"2024-09-11T07:37:37.610232+0000","last_became_active":"2024-09-11T07:32:00.518307+0000","last_became_peered":"2024-09-11T07:32:00.518307+0000","last_unstale":"2024-09-11T07:37:37.610232+0000","last_undegraded":"2024-09-11T07:37:37.610232+0000","last_fullsized":"2024-09-11T07:37:37.610232+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:36:31.353426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:31.353426+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:54.521250+0000","scrub_duration":0.0063773889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"234'20","reported_seq":94,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.604838+0000","last_change":"2024-09-11T07:37:29.661799+0000","last_active":"2024-09-11T07:37:37.604838+0000","last_peered":"2024-09-11T07:37:37.604838+0000","last_clean":"2024-09-11T07:37:37.604838+0000","last_became_active":"2024-09-11T07:35:00.561615+0000","last_became_peered":"2024-09-11T07:35:00.561615+0000","last_unstale":"2024-09-11T07:37:37.604838+0000","last_undegraded":"2024-09-11T07:37:37.604838+0000","last_fullsized":"2024-09-11T07:37:37.604838+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'20","last_scrub_stamp":"2024-09-11T07:37:29.661671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:29.661671+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:58.370347+0000","scrub_duration":0.0019609900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1f","version":"225'11","reported_seq":88,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:44.638638+0000","last_change":"2024-09-11T07:36:05.227568+0000","last_active":"2024-09-11T07:37:44.638638+0000","last_peered":"2024-09-11T07:37:44.638638+0000","last_clean":"2024-09-11T07:37:44.638638+0000","last_became_active":"2024-09-11T07:36:05.227140+0000","last_became_peered":"2024-09-11T07:36:05.227140+0000","last_unstale":"2024-09-11T07:37:44.638638+0000","last_undegraded":"2024-09-11T07:37:44.638638+0000","last_fullsized":"2024-09-11T07:37:44.638638+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:35:32.194121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:35:32.194121+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":249,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.610147+0000","last_change":"2024-09-11T07:35:53.875376+0000","last_active":"2024-09-11T07:37:37.610147+0000","last_peered":"2024-09-11T07:37:37.610147+0000","last_clean":"2024-09-11T07:37:37.610147+0000","last_became_active":"2024-09-11T07:26:37.562501+0000","last_became_peered":"2024-09-11T07:26:37.562501+0000","last_unstale":"2024-09-11T07:37:37.610147+0000","last_undegraded":"2024-09-11T07:37:37.610147+0000","last_fullsized":"2024-09-11T07:37:37.610147+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:53.875300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:53.875300+0000","last_clean_scrub_stamp":"2024-09-11T07:35:53.875300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019957579999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":256,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.606585+0000","last_change":"2024-09-11T07:36:38.841363+0000","last_active":"2024-09-11T07:37:37.606585+0000","last_peered":"2024-09-11T07:37:37.606585+0000","last_clean":"2024-09-11T07:37:37.606585+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:37:37.606585+0000","last_undegraded":"2024-09-11T07:37:37.606585+0000","last_fullsized":"2024-09-11T07:37:37.606585+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:38.841250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:38.841250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:44.208784+0000","scrub_duration":0.014360088999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"234'19","reported_seq":85,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.612122+0000","last_change":"2024-09-11T07:36:05.229126+0000","last_active":"2024-09-11T07:37:37.612122+0000","last_peered":"2024-09-11T07:37:37.612122+0000","last_clean":"2024-09-11T07:37:37.612122+0000","last_became_active":"2024-09-11T07:36:05.228746+0000","last_became_peered":"2024-09-11T07:36:05.228746+0000","last_unstale":"2024-09-11T07:37:37.612122+0000","last_undegraded":"2024-09-11T07:37:37.612122+0000","last_fullsized":"2024-09-11T07:37:37.612122+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:35:36.232923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:35:36.232923+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":269,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.616365+0000","last_change":"2024-09-11T07:37:21.150801+0000","last_active":"2024-09-11T07:37:37.616365+0000","last_peered":"2024-09-11T07:37:37.616365+0000","last_clean":"2024-09-11T07:37:37.616365+0000","last_became_active":"2024-09-11T07:32:00.507050+0000","last_became_peered":"2024-09-11T07:32:00.507050+0000","last_unstale":"2024-09-11T07:37:37.616365+0000","last_undegraded":"2024-09-11T07:37:37.616365+0000","last_fullsized":"2024-09-11T07:37:37.616365+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:21.150732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:24.575210+0000","last_clean_scrub_stamp":"2024-09-11T07:37:21.150732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:42.313821+0000","scrub_duration":0.001360691,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":250,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.605044+0000","last_change":"2024-09-11T07:35:49.723748+0000","last_active":"2024-09-11T07:37:37.605044+0000","last_peered":"2024-09-11T07:37:37.605044+0000","last_clean":"2024-09-11T07:37:37.605044+0000","last_became_active":"2024-09-11T07:26:37.862373+0000","last_became_peered":"2024-09-11T07:26:37.862373+0000","last_unstale":"2024-09-11T07:37:37.605044+0000","last_undegraded":"2024-09-11T07:37:37.605044+0000","last_fullsized":"2024-09-11T07:37:37.605044+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:49.723695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:35:49.723695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00410707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":191,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:41.657881+0000","last_change":"2024-09-11T07:37:41.657806+0000","last_active":"2024-09-11T07:37:41.657881+0000","last_peered":"2024-09-11T07:37:41.657881+0000","last_clean":"2024-09-11T07:37:41.657881+0000","last_became_active":"2024-09-11T07:36:05.224085+0000","last_became_peered":"2024-09-11T07:36:05.224085+0000","last_unstale":"2024-09-11T07:37:41.657881+0000","last_undegraded":"2024-09-11T07:37:41.657881+0000","last_fullsized":"2024-09-11T07:37:41.657881+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:41.657719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:41.657719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:44.656666+0000","scrub_duration":0.002776653,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"234'27","reported_seq":307,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.606308+0000","last_change":"2024-09-11T07:37:18.388968+0000","last_active":"2024-09-11T07:37:37.606308+0000","last_peered":"2024-09-11T07:37:37.606308+0000","last_clean":"2024-09-11T07:37:37.606308+0000","last_became_active":"2024-09-11T07:26:33.542544+0000","last_became_peered":"2024-09-11T07:26:33.542544+0000","last_unstale":"2024-09-11T07:37:37.606308+0000","last_undegraded":"2024-09-11T07:37:37.606308+0000","last_fullsized":"2024-09-11T07:37:37.606308+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'27","last_scrub_stamp":"2024-09-11T07:37:18.388604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:18.388604+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:32.272894+0000","scrub_duration":0.0019030239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":251,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.610364+0000","last_change":"2024-09-11T07:36:44.853517+0000","last_active":"2024-09-11T07:37:37.610364+0000","last_peered":"2024-09-11T07:37:37.610364+0000","last_clean":"2024-09-11T07:37:37.610364+0000","last_became_active":"2024-09-11T07:26:35.555862+0000","last_became_peered":"2024-09-11T07:26:35.555862+0000","last_unstale":"2024-09-11T07:37:37.610364+0000","last_undegraded":"2024-09-11T07:37:37.610364+0000","last_fullsized":"2024-09-11T07:37:37.610364+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:44.850049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:36:44.850049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:37:46.370556+0000","scrub_duration":0.020441075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":250,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.606083+0000","last_change":"2024-09-11T07:37:32.004184+0000","last_active":"2024-09-11T07:37:37.606083+0000","last_peered":"2024-09-11T07:37:37.606083+0000","last_clean":"2024-09-11T07:37:37.606083+0000","last_became_active":"2024-09-11T07:26:37.861647+0000","last_became_peered":"2024-09-11T07:26:37.861647+0000","last_unstale":"2024-09-11T07:37:37.606083+0000","last_undegraded":"2024-09-11T07:37:37.606083+0000","last_fullsized":"2024-09-11T07:37:37.606083+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:32.004125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:32.004125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:34.761692+0000","scrub_duration":0.0019485520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":273,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:42.079009+0000","last_change":"2024-09-11T07:37:42.078889+0000","last_active":"2024-09-11T07:37:42.079009+0000","last_peered":"2024-09-11T07:37:42.079009+0000","last_clean":"2024-09-11T07:37:42.079009+0000","last_became_active":"2024-09-11T07:32:00.502947+0000","last_became_peered":"2024-09-11T07:32:00.502947+0000","last_unstale":"2024-09-11T07:37:42.079009+0000","last_undegraded":"2024-09-11T07:37:42.079009+0000","last_fullsized":"2024-09-11T07:37:42.079009+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:42.078765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:42.078765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:42.683014+0000","scrub_duration":0.0062011560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"234'23","reported_seq":291,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.079051+0000","last_change":"2024-09-11T07:37:43.078969+0000","last_active":"2024-09-11T07:37:43.079051+0000","last_peered":"2024-09-11T07:37:43.079051+0000","last_clean":"2024-09-11T07:37:43.079051+0000","last_became_active":"2024-09-11T07:26:33.542711+0000","last_became_peered":"2024-09-11T07:26:33.542711+0000","last_unstale":"2024-09-11T07:37:43.079051+0000","last_undegraded":"2024-09-11T07:37:43.079051+0000","last_fullsized":"2024-09-11T07:37:43.079051+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'23","last_scrub_stamp":"2024-09-11T07:37:43.078872+0000","last_deep_scrub":"127'17","last_deep_scrub_stamp":"2024-09-11T07:35:43.871918+0000","last_clean_scrub_stamp":"2024-09-11T07:37:43.078872+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:09.927640+0000","scrub_duration":0.0051065670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":252,"reported_epoch":254,"state":"active+clean","last_fresh":"2024-09-11T07:37:37.604837+0000","last_change":"2024-09-11T07:36:37.811087+0000","last_active":"2024-09-11T07:37:37.604837+0000","last_peered":"2024-09-11T07:37:37.604837+0000","last_clean":"2024-09-11T07:37:37.604837+0000","last_became_active":"2024-09-11T07:26:35.562037+0000","last_became_peered":"2024-09-11T07:26:35.562037+0000","last_unstale":"2024-09-11T07:37:37.604837+0000","last_undegraded":"2024-09-11T07:37:37.604837+0000","last_fullsized":"2024-09-11T07:37:37.604837+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:37.811009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:36:37.811009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039204729999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":2659057664,"num_objects":40574,"num_object_clones":0,"num_object_copies":121722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40574,"num_write_kb":2596736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":7941390336,"data_stored":7941390336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40574,"ondisk_log_size":40574,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1124,"num_read_kb":881,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":764,"ondisk_log_size":764,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":675,"num_read_kb":670,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":147,"seq":631360192584,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1719676,"kb_used_data":1391028,"kb_used_omap":4,"kb_used_meta":328635,"kb_avail":92025476,"statfs":{"total":95995035648,"available":94234087424,"internally_reserved":0,"allocated":1424412672,"data_stored":1424002481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":336522758},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.8,"15min":3.1440000000000001},"min":{"1min":0.22,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":3.0910000000000002,"5min":49.573,"15min":49.573},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.177,"5min":1.833,"15min":2.9289999999999998},"min":{"1min":0.24199999999999999,"5min":0.22,"15min":0.22},"max":{"1min":6.1470000000000002,"5min":49.881,"15min":49.881},"last":0.26200000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.98599999999999999,"15min":0.72599999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":2.9910000000000001,"5min":10.063000000000001,"15min":10.063000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.206,"15min":0.93799999999999994},"min":{"1min":0.252,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":3.6499999999999999,"5min":9.6059999999999999,"15min":9.6059999999999999},"last":0.501}]},{"osd":2,"last update":"Wed Sep 11 07:37:14 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0649999999999999,"15min":1.0129999999999999},"min":{"1min":0.27100000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":5.6909999999999998,"5min":13.99,"15min":13.99},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.196,"15min":1.01},"min":{"1min":0.28499999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.2989999999999999,"5min":14.67,"15min":14.67},"last":3.8759999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":1.661,"15min":2.9529999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":3.3279999999999998,"5min":49.798999999999999,"15min":49.798999999999999},"last":2.6850000000000001},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.7210000000000001,"15min":2.883},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.4079999999999999,"5min":49.524000000000001,"15min":49.524000000000001},"last":0.30099999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.64500000000000002,"15min":0.49199999999999999},"min":{"1min":0.28799999999999998,"5min":0.253,"15min":0.253},"max":{"1min":6.0220000000000002,"5min":6.0220000000000002,"15min":6.0220000000000002},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.66700000000000004,"15min":0.623},"min":{"1min":0.30199999999999999,"5min":0.247,"15min":0.247},"max":{"1min":3.044,"5min":3.9340000000000002,"15min":3.9340000000000002},"last":0.76200000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.70899999999999996,"15min":0.52900000000000003},"min":{"1min":0.309,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.9909999999999997,"5min":5.9909999999999997,"15min":5.9909999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.77500000000000002,"15min":0.69699999999999995},"min":{"1min":0.32100000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.0149999999999997,"5min":4.9969999999999999,"15min":4.9969999999999999},"last":0.79500000000000004}]},{"osd":6,"last update":"Wed Sep 11 07:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66800000000000004,"15min":0.56299999999999994},"min":{"1min":0.34399999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":3.3490000000000002,"5min":4.6440000000000001,"15min":4.6440000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.72299999999999998,"15min":0.59899999999999998},"min":{"1min":0.34399999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":3.5470000000000002,"5min":3.694,"15min":3.694},"last":0.32900000000000001}]}]},{"osd":6,"up_from":44,"seq":188978561169,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":795556,"kb_used_data":481564,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":92949596,"statfs":{"total":95995035648,"available":95180386304,"internally_reserved":0,"allocated":493121536,"data_stored":492734399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:37:11 2024","interfaces":[{"interface":"back","average":{"1min":1.96,"5min":2.984,"15min":1.7549999999999999},"min":{"1min":0.22,"5min":0.20699999999999999,"15min":0.192},"max":{"1min":20.675999999999998,"5min":142.29599999999999,"15min":142.29599999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":1.9450000000000001,"5min":3.2999999999999998,"15min":1.8919999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":20.663,"5min":142.68799999999999,"15min":142.68799999999999},"last":0.35999999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:37:11 2024","interfaces":[{"interface":"back","average":{"1min":2.964,"5min":2.645,"15min":1.3260000000000001},"min":{"1min":0.249,"5min":0.24099999999999999,"15min":0.23599999999999999},"max":{"1min":39.390000000000001,"5min":127.92100000000001,"15min":127.92100000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":3.0800000000000001,"5min":2.4790000000000001,"15min":1.3140000000000001},"min":{"1min":0.27200000000000002,"5min":0.24099999999999999,"15min":0.23000000000000001},"max":{"1min":40.186999999999998,"5min":127.746,"15min":127.746},"last":0.749}]},{"osd":2,"last update":"Wed Sep 11 07:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.92700000000000005,"15min":0.85899999999999999},"min":{"1min":0.27300000000000002,"5min":0.248,"15min":0.248},"max":{"1min":3.6539999999999999,"5min":4.3220000000000001,"15min":26.491},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.363,"5min":1.198,"15min":1.018},"min":{"1min":0.27100000000000002,"5min":0.26500000000000001,"15min":0.26400000000000001},"max":{"1min":3.9689999999999999,"5min":6.0819999999999999,"15min":26.561},"last":0.28299999999999997}]},{"osd":3,"last update":"Wed Sep 11 07:37:11 2024","interfaces":[{"interface":"back","average":{"1min":2.274,"5min":1.4750000000000001,"15min":1.0660000000000001},"min":{"1min":0.26100000000000001,"5min":0.25600000000000001,"15min":0.254},"max":{"1min":20.734000000000002,"5min":20.734000000000002,"15min":31.268999999999998},"last":0.71499999999999997},{"interface":"front","average":{"1min":2.0840000000000001,"5min":1.3149999999999999,"15min":1.1499999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":20.701000000000001,"5min":20.701000000000001,"15min":31.872},"last":0.86199999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.626,"15min":0.55700000000000005},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":1.6719999999999999,"5min":3.3599999999999999,"15min":4.4720000000000004},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56100000000000005,"15min":0.46500000000000002},"min":{"1min":0.25900000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.329,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":0.438}]},{"osd":5,"last update":"Wed Sep 11 07:37:11 2024","interfaces":[{"interface":"back","average":{"1min":3.6349999999999998,"5min":1.417,"15min":0.85499999999999998},"min":{"1min":0.30199999999999999,"5min":0.255,"15min":0.255},"max":{"1min":57.436,"5min":57.436,"15min":57.436},"last":1.0229999999999999},{"interface":"front","average":{"1min":3.359,"5min":1.429,"15min":0.81399999999999995},"min":{"1min":0.32300000000000001,"5min":0.251,"15min":0.251},"max":{"1min":56.822000000000003,"5min":56.822000000000003,"15min":56.822000000000003},"last":0.46000000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.68700000000000006,"15min":0.66600000000000004},"min":{"1min":0.312,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.3109999999999999,"5min":3.3140000000000001,"15min":3.3140000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73099999999999998,"15min":0.72599999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":3.6749999999999998,"5min":5.0439999999999996,"15min":5.0439999999999996},"last":0.88500000000000001}]}]},{"osd":5,"up_from":37,"seq":158913790103,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1686708,"kb_used_data":1376876,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":92058444,"statfs":{"total":95995035648,"available":94267846656,"internally_reserved":0,"allocated":1409921024,"data_stored":1409518127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,2,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.891,"5min":1.5089999999999999,"15min":2.121},"min":{"1min":0.28000000000000003,"5min":0.21299999999999999,"15min":0.20799999999999999},"max":{"1min":25.808,"5min":31.369,"15min":185.68199999999999},"last":1.45},{"interface":"front","average":{"1min":2.0059999999999998,"5min":1.766,"15min":2.2759999999999998},"min":{"1min":0.22900000000000001,"5min":0.223,"15min":0.218},"max":{"1min":25.722999999999999,"5min":25.722999999999999,"15min":186.12200000000001},"last":1.343}]},{"osd":1,"last update":"Wed Sep 11 07:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3540000000000001,"15min":0.90600000000000003},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.20899999999999999},"max":{"1min":13.297000000000001,"5min":26.286999999999999,"15min":26.286999999999999},"last":1.538},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2629999999999999,"15min":0.94199999999999995},"min":{"1min":0.255,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":13.281000000000001,"5min":24.672999999999998,"15min":24.672999999999998},"last":2.181}]},{"osd":2,"last update":"Wed Sep 11 07:37:49 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.91500000000000004,"15min":1.5589999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.22700000000000001},"max":{"1min":1.1719999999999999,"5min":4.585,"15min":187.99199999999999},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.0620000000000001,"5min":0.91900000000000004,"15min":1.546},"min":{"1min":0.28399999999999997,"5min":0.27600000000000002,"15min":0.254},"max":{"1min":7.8120000000000003,"5min":7.8120000000000003,"15min":186.375},"last":1.4870000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.865,"5min":1.55,"15min":2.226},"min":{"1min":0.26800000000000002,"5min":0.26400000000000001,"15min":0.23599999999999999},"max":{"1min":28.978000000000002,"5min":28.978000000000002,"15min":188.11500000000001},"last":2.0209999999999999},{"interface":"front","average":{"1min":1.9610000000000001,"5min":1.528,"15min":2.2040000000000002},"min":{"1min":0.32700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":25.837,"5min":26.936,"15min":186.44200000000001},"last":2.0670000000000002}]},{"osd":4,"last update":"Wed Sep 11 07:37:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.60599999999999998,"15min":0.47199999999999998},"min":{"1min":0.25,"5min":0.247,"15min":0.19900000000000001},"max":{"1min":1.4890000000000001,"5min":4.7060000000000004,"15min":4.7060000000000004},"last":1.3899999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.67100000000000004,"15min":0.505},"min":{"1min":0.32600000000000001,"5min":0.24199999999999999,"15min":0.23200000000000001},"max":{"1min":2.0830000000000002,"5min":2.8439999999999999,"15min":2.8439999999999999},"last":1.4039999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:37:12 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.70599999999999996,"15min":0.51900000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.26000000000000001},"max":{"1min":1.7849999999999999,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":1.4350000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.65100000000000002,"15min":0.51200000000000001},"min":{"1min":0.33900000000000002,"5min":0.26800000000000002,"15min":0.25800000000000001},"max":{"1min":4.3120000000000003,"5min":4.3120000000000003,"15min":4.3120000000000003},"last":1.5129999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:37:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.745,"15min":0.64100000000000001},"min":{"1min":0.314,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":2.464,"5min":4.7919999999999998,"15min":4.7919999999999998},"last":1.4239999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.76700000000000002,"15min":0.65700000000000003},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.8140000000000001,"5min":4.0110000000000001,"15min":4.0110000000000001},"last":1.5900000000000001}]}]},{"osd":4,"up_from":31,"seq":133143986331,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":775160,"kb_used_data":465328,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":92969992,"statfs":{"total":95995035648,"available":95201271808,"internally_reserved":0,"allocated":476495872,"data_stored":476088722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:37:36 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":1.242,"15min":1.1080000000000001},"min":{"1min":0.27300000000000002,"5min":0.217,"15min":0.20799999999999999},"max":{"1min":2.7530000000000001,"5min":32.892000000000003,"15min":35.308},"last":1.2350000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.337,"15min":1.087},"min":{"1min":0.249,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":4.5620000000000003,"5min":32.860999999999997,"15min":35.597999999999999},"last":0.36099999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:37:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":1.365,"15min":0.97599999999999998},"min":{"1min":0.23100000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":2.1240000000000001,"5min":24.581,"15min":24.581},"last":0.42599999999999999},{"interface":"front","average":{"1min":1.23,"5min":1.28,"15min":0.89300000000000002},"min":{"1min":0.38,"5min":0.24399999999999999,"15min":0.23699999999999999},"max":{"1min":5.8010000000000002,"5min":24.652000000000001,"15min":24.652000000000001},"last":4.4390000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:37:36 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.036,"15min":0.92200000000000004},"min":{"1min":0.23699999999999999,"5min":0.223,"15min":0.223},"max":{"1min":3.726,"5min":11.425000000000001,"15min":19.927},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.94399999999999995,"5min":1.0960000000000001,"15min":0.91800000000000004},"min":{"1min":0.28999999999999998,"5min":0.27200000000000002,"15min":0.24399999999999999},"max":{"1min":4.101,"5min":13.423999999999999,"15min":20.02},"last":0.52300000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:37:36 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.3109999999999999,"15min":1.0720000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.673,"5min":24.029,"15min":24.029},"last":0.621},{"interface":"front","average":{"1min":0.70499999999999996,"5min":1.2390000000000001,"15min":0.99299999999999999},"min":{"1min":0.33900000000000002,"5min":0.32100000000000001,"15min":0.28699999999999998},"max":{"1min":2.0070000000000001,"5min":24.265000000000001,"15min":24.265000000000001},"last":0.53900000000000003}]},{"osd":5,"last update":"Wed Sep 11 07:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.72999999999999998,"15min":0.49099999999999999},"min":{"1min":0.23699999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":4.6079999999999997,"5min":4.6079999999999997,"15min":4.6079999999999997},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73599999999999999,"15min":0.50800000000000001},"min":{"1min":0.28399999999999997,"5min":0.25,"15min":0.22700000000000001},"max":{"1min":2.9990000000000001,"5min":5.7320000000000002,"15min":5.7320000000000002},"last":0.54700000000000004}]},{"osd":6,"last update":"Wed Sep 11 07:37:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.72499999999999998,"15min":0.52700000000000002},"min":{"1min":0.255,"5min":0.23999999999999999,"15min":0.20899999999999999},"max":{"1min":1.96,"5min":7.8650000000000002,"15min":7.8650000000000002},"last":1.0569999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.73199999999999998,"15min":0.53800000000000003},"min":{"1min":0.27500000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":3.7949999999999999,"5min":9.3770000000000007,"15min":9.3770000000000007},"last":0.46999999999999997}]},{"osd":7,"last update":"Wed Sep 11 07:37:09 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.65000000000000002,"15min":0.57999999999999996},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":2.4470000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70699999999999996,"15min":0.60099999999999998},"min":{"1min":0.29799999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.2949999999999999,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":0.45400000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182560,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1568192,"kb_used_data":1266680,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":92176960,"statfs":{"total":95995035648,"available":94389207040,"internally_reserved":0,"allocated":1297080320,"data_stored":1296664442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:37:16 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.6299999999999999,"15min":1.1919999999999999},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":4.2709999999999999,"5min":38.432000000000002,"15min":38.432000000000002},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.353,"5min":1.7190000000000001,"15min":1.546},"min":{"1min":0.26300000000000001,"5min":0.20300000000000001,"15min":0.19500000000000001},"max":{"1min":4.657,"5min":33.426000000000002,"15min":91.876999999999995},"last":3.1629999999999998}]},{"osd":1,"last update":"Wed Sep 11 07:37:16 2024","interfaces":[{"interface":"back","average":{"1min":5.1669999999999998,"5min":2.2480000000000002,"15min":1.3979999999999999},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":67.305000000000007,"5min":67.305000000000007,"15min":67.305000000000007},"last":3.3130000000000002},{"interface":"front","average":{"1min":1.252,"5min":1.484,"15min":1.0209999999999999},"min":{"1min":0.23799999999999999,"5min":0.192,"15min":0.192},"max":{"1min":4.4619999999999997,"5min":9.3450000000000006,"15min":9.3450000000000006},"last":3.2320000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":1.387,"15min":1.0980000000000001},"min":{"1min":0.24199999999999999,"5min":0.23699999999999999,"15min":0.23200000000000001},"max":{"1min":4.4240000000000004,"5min":8.282,"15min":8.282},"last":2.6040000000000001},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.2669999999999999,"15min":1.03},"min":{"1min":0.315,"5min":0.26200000000000001,"15min":0.20999999999999999},"max":{"1min":5.1520000000000001,"5min":6.5640000000000001,"15min":8.7110000000000003},"last":0.63200000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:37:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.99099999999999999,"15min":0.95499999999999996},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":2.1760000000000002,"5min":5.5099999999999998,"15min":21.856000000000002},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.74099999999999999,"5min":1.006,"15min":1.105},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":2.7200000000000002,"5min":4.6219999999999999,"15min":19.146000000000001},"last":3.2080000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:37:47 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":1.0309999999999999,"15min":1.0369999999999999},"min":{"1min":0.42599999999999999,"5min":0.309,"15min":0.29599999999999999},"max":{"1min":2.25,"5min":5.0209999999999999,"15min":32.325000000000003},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":1.1439999999999999,"15min":1.109},"min":{"1min":0.39000000000000001,"5min":0.28599999999999998,"15min":0.28299999999999997},"max":{"1min":2.3109999999999999,"5min":4.8490000000000002,"15min":18.416},"last":3.2810000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:37:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.3740000000000001,"15min":1.3500000000000001},"min":{"1min":0.372,"5min":0.308,"15min":0.27000000000000002},"max":{"1min":4.5179999999999998,"5min":32.350999999999999,"15min":32.350999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":1.3220000000000001,"15min":1.3240000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.33100000000000002},"max":{"1min":2.2290000000000001,"5min":32.402999999999999,"15min":32.402999999999999},"last":0.71399999999999997}]},{"osd":7,"last update":"Wed Sep 11 07:37:09 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":1.194,"15min":1.3240000000000001},"min":{"1min":0.32300000000000001,"5min":0.308,"15min":0.308},"max":{"1min":2.7679999999999998,"5min":8.0429999999999993,"15min":8.0429999999999993},"last":3.246},{"interface":"front","average":{"1min":1.077,"5min":1.4570000000000001,"15min":2.157},"min":{"1min":0.39400000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":4.4909999999999997,"5min":32.289999999999999,"15min":32.289999999999999},"last":0.91200000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411493,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":779872,"kb_used_data":482520,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":92965280,"statfs":{"total":95995035648,"available":95196446720,"internally_reserved":0,"allocated":494100480,"data_stored":493717022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:37:48 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":2.3290000000000002,"15min":1.4670000000000001},"min":{"1min":0.185,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":1.958,"5min":56.704000000000001,"15min":56.704000000000001},"last":1.704},{"interface":"front","average":{"1min":0.90300000000000002,"5min":2.532,"15min":1.5449999999999999},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":4.5199999999999996,"5min":56.816000000000003,"15min":56.816000000000003},"last":0.25700000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:37:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":1.4970000000000001,"15min":1.1679999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.23200000000000001},"max":{"1min":1.55,"5min":65.153999999999996,"15min":65.153999999999996},"last":2.1269999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":1.946,"15min":1.3600000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.20699999999999999},"max":{"1min":2.8540000000000001,"5min":63.311999999999998,"15min":63.311999999999998},"last":1.923}]},{"osd":3,"last update":"Wed Sep 11 07:37:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":2.0619999999999998,"15min":1.3220000000000001},"min":{"1min":0.28100000000000003,"5min":0.27400000000000002,"15min":0.26900000000000002},"max":{"1min":6.4960000000000004,"5min":44.405999999999999,"15min":44.405999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":1.0349999999999999,"5min":2.0259999999999998,"15min":1.3520000000000001},"min":{"1min":0.27100000000000002,"5min":0.253,"15min":0.253},"max":{"1min":3.222,"5min":43.295999999999999,"15min":43.295999999999999},"last":1.6679999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:37:34 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.90400000000000003,"15min":1.036},"min":{"1min":0.34000000000000002,"5min":0.30599999999999999,"15min":0.27500000000000002},"max":{"1min":1.004,"5min":6.7599999999999998,"15min":30.946999999999999},"last":0.442},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.86899999999999999,"15min":0.93899999999999995},"min":{"1min":0.41599999999999998,"5min":0.31900000000000001,"15min":0.29999999999999999},"max":{"1min":1.8500000000000001,"5min":3.0960000000000001,"15min":22.434000000000001},"last":0.59599999999999997}]},{"osd":5,"last update":"Wed Sep 11 07:37:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.85999999999999999,"15min":1.048},"min":{"1min":0.436,"5min":0.35099999999999998,"15min":0.31900000000000001},"max":{"1min":3.3290000000000002,"5min":3.3290000000000002,"15min":31.789000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0129999999999999,"15min":1.133},"min":{"1min":0.376,"5min":0.34100000000000003,"15min":0.31},"max":{"1min":3.931,"5min":4.1479999999999997,"15min":30.898},"last":0.628}]},{"osd":6,"last update":"Wed Sep 11 07:37:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.2549999999999999,"15min":1.0489999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.36099999999999999},"max":{"1min":2.9430000000000001,"5min":22.399000000000001,"15min":22.399000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.3819999999999999,"15min":1.048},"min":{"1min":0.311,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.0680000000000001,"5min":25.739000000000001,"15min":25.739000000000001},"last":0.47199999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.204,"15min":1.956},"min":{"1min":0.435,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.6600000000000001,"5min":22.484000000000002,"15min":22.484000000000002},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.081,"5min":1.4410000000000001,"15min":2.2959999999999998},"min":{"1min":0.40699999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":3.1949999999999998,"5min":25.670000000000002,"15min":25.670000000000002},"last":0.70599999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575016,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1737220,"kb_used_data":1431548,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":92007932,"statfs":{"total":95995035648,"available":94216122368,"internally_reserved":0,"allocated":1465905152,"data_stored":1465485992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:37:18 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.6180000000000001,"15min":1.6850000000000001},"min":{"1min":0.24099999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":7.9429999999999996,"5min":8.0549999999999997,"15min":82.218999999999994},"last":0.32300000000000001},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.7470000000000001,"15min":1.706},"min":{"1min":0.217,"5min":0.193,"15min":0.191},"max":{"1min":5.5019999999999998,"5min":8.0739999999999998,"15min":82.278999999999996},"last":0.45100000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:37:51 2024","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.6910000000000001,"15min":1.2190000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.19800000000000001},"max":{"1min":6.0279999999999996,"5min":9.0969999999999995,"15min":9.0969999999999995},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.5,"15min":1.25},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":4.0119999999999996,"5min":9.1310000000000002,"15min":9.1310000000000002},"last":2.2440000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:37:09 2024","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.736,"15min":1.7709999999999999},"min":{"1min":0.29199999999999998,"5min":0.28000000000000003,"15min":0.25800000000000001},"max":{"1min":6.1360000000000001,"5min":9.2889999999999997,"15min":81.882999999999996},"last":2.9820000000000002},{"interface":"front","average":{"1min":1.423,"5min":1.7170000000000001,"15min":1.7230000000000001},"min":{"1min":0.36299999999999999,"5min":0.22600000000000001,"15min":0.20999999999999999},"max":{"1min":5.2359999999999998,"5min":14.428000000000001,"15min":82.400000000000006},"last":1.0189999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:37:26 2024","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.331,"15min":0.94399999999999995},"min":{"1min":0.33000000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":5.5609999999999999,"5min":7.3630000000000004,"15min":7.3630000000000004},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.3400000000000001,"15min":1.0780000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.26500000000000001},"max":{"1min":4.4459999999999997,"5min":9.2289999999999992,"15min":9.2289999999999992},"last":2.351}]},{"osd":5,"last update":"Wed Sep 11 07:37:51 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.3779999999999999,"15min":1.0169999999999999},"min":{"1min":0.371,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":4.5099999999999998,"5min":7.4379999999999997,"15min":7.4379999999999997},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.3899999999999999,"15min":1.113},"min":{"1min":0.34499999999999997,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":4.6799999999999997,"5min":9.1929999999999996,"15min":9.1929999999999996},"last":2.3079999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:37:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.341,"15min":1.133},"min":{"1min":0.41399999999999998,"5min":0.316,"15min":0.30099999999999999},"max":{"1min":2.6299999999999999,"5min":9.2569999999999997,"15min":24.018000000000001},"last":2.5760000000000001},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.0349999999999999,"15min":0.94899999999999995},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.29999999999999999},"max":{"1min":5.2709999999999999,"5min":5.2709999999999999,"15min":23.681000000000001},"last":0.38900000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:37:09 2024","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.196,"15min":1.3},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":5.3079999999999998,"5min":7.0940000000000003,"15min":7.0940000000000003},"last":0.41599999999999998},{"interface":"front","average":{"1min":1.7689999999999999,"5min":1.4770000000000001,"15min":1.514},"min":{"1min":0.36899999999999999,"5min":0.313,"15min":0.313},"max":{"1min":5.5860000000000003,"5min":7.4050000000000002,"15min":7.4050000000000002},"last":0.93300000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738542,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":934388,"kb_used_data":637036,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":92810764,"statfs":{"total":95995035648,"available":95038222336,"internally_reserved":0,"allocated":652324864,"data_stored":651921675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:37:19 2024","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.2689999999999999,"15min":0.98399999999999999},"min":{"1min":0.32300000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":5.4630000000000001,"5min":5.4630000000000001,"15min":9.7710000000000008},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.8080000000000001,"5min":1.333,"15min":1.0129999999999999},"min":{"1min":0.26800000000000002,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":5.1479999999999997,"5min":5.1479999999999997,"15min":10.275},"last":0.39300000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:37:50 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.3100000000000001,"15min":1.208},"min":{"1min":0.23000000000000001,"5min":0.21299999999999999,"15min":0.20200000000000001},"max":{"1min":5.3040000000000003,"5min":11.397,"15min":19.527999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.878,"5min":1.1160000000000001,"15min":1.145},"min":{"1min":0.224,"5min":0.21099999999999999,"15min":0.20999999999999999},"max":{"1min":4.2999999999999998,"5min":6.5960000000000001,"15min":19.556999999999999},"last":0.246}]},{"osd":3,"last update":"Wed Sep 11 07:37:19 2024","interfaces":[{"interface":"back","average":{"1min":1.8140000000000001,"5min":1.55,"15min":1.7310000000000001},"min":{"1min":0.35199999999999998,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":9.4199999999999999,"5min":9.4920000000000009,"15min":45.185000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":1.379,"5min":1.383,"15min":1.4950000000000001},"min":{"1min":0.378,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.2460000000000004,"5min":8.0999999999999996,"15min":43.228999999999999},"last":0.27700000000000002}]},{"osd":4,"last update":"Wed Sep 11 07:37:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.218,"15min":0.995},"min":{"1min":0.29799999999999999,"5min":0.26800000000000002,"15min":0.26700000000000002},"max":{"1min":4.8109999999999999,"5min":6.8319999999999999,"15min":19.411999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.196,"15min":1.1319999999999999},"min":{"1min":0.35999999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":2.9260000000000002,"5min":9.5619999999999994,"15min":19.396000000000001},"last":0.42899999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:37:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.246,"15min":1.103},"min":{"1min":0.34100000000000003,"5min":0.29599999999999999,"15min":0.28999999999999998},"max":{"1min":5.0830000000000002,"5min":6.9400000000000004,"15min":19.381},"last":1.5569999999999999},{"interface":"front","average":{"1min":0.86499999999999999,"5min":1.1839999999999999,"15min":1.1830000000000001},"min":{"1min":0.40000000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.0910000000000002,"5min":9.6289999999999996,"15min":19.437000000000001},"last":0.46400000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:37:19 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.337,"15min":1.153},"min":{"1min":0.45000000000000001,"5min":0.33300000000000002,"15min":0.316},"max":{"1min":2.29,"5min":23.190999999999999,"15min":23.190999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.995,"5min":1.4350000000000001,"15min":1.1599999999999999},"min":{"1min":0.373,"5min":0.35999999999999999,"15min":0.34200000000000003},"max":{"1min":4.1619999999999999,"5min":23.632000000000001,"15min":23.632000000000001},"last":0.41399999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:37:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.288,"15min":1.6599999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":2.9540000000000002,"5min":9.6530000000000005,"15min":9.6530000000000005},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.282,"5min":1.2769999999999999,"15min":1.71},"min":{"1min":0.42699999999999999,"5min":0.315,"15min":0.315},"max":{"1min":4.194,"5min":5.2759999999999998,"15min":5.2759999999999998},"last":0.36899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":678952960,"data_stored":678952960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1492123648,"data_stored":1492123648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":504037376,"data_stored":504037376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1338507264,"data_stored":1338507264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":485621760,"data_stored":485621760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1469382656,"data_stored":1469382656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":494862336,"data_stored":494862336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1477902336,"data_stored":1477902336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:38:00.434 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-11T07:38:00.434 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:38:00.434 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_2 2024-09-11T07:38:00.434 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:38:00.477 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 23 16 50103 50087 136.078 100.25 0.0101103 0.00734162 2024-09-11T07:38:00.506 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:00.555 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:00.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:00.556 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:00.657 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:00.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:38:00 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:38:00] "GET /metrics HTTP/1.1" 200 224290 "" "Prometheus/2.43.0" 2024-09-11T07:38:00.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:00.703 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:00.703 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:00.804 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:00.832 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:00.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:00.833 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:00.934 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:00.985 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:00.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:00.986 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:01.087 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:01.117 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:01.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:01.118 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:01.219 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:01.255 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:01.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:01.256 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:01.357 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:01.399 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:01.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:01.400 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:01.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:01 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:38:01.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:01 smithi022 ceph-mon[29462]: pgmap v512: 148 pgs: 148 active+clean; 3.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 144 MiB/s wr, 2.30k op/s 2024-09-11T07:38:01.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:01 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:38:01.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:01 smithi022 ceph-mon[37330]: pgmap v512: 148 pgs: 148 active+clean; 3.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 144 MiB/s wr, 2.30k op/s 2024-09-11T07:38:01.477 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 24 15 51785 51770 134.791 105.188 0.00611071 0.00741344 2024-09-11T07:38:01.501 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:01.519 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-11T07:38:01.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:01 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:38:01.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:01 smithi104 ceph-mon[33507]: pgmap v512: 148 pgs: 148 active+clean; 3.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 144 MiB/s wr, 2.30k op/s 2024-09-11T07:38:01.545 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:01.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:01.546 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:01.647 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:01.698 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:01.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:01.699 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:01.800 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:01.827 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:01.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:01.828 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:01.929 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:01.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:01.963 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:01.963 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:02.064 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:02.096 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:02.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:02.097 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:02.198 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:02.240 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:02.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:02.241 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:02.342 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:02.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:02.391 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:02.391 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:02.477 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 25 15 53446 53431 133.551 103.812 0.0114455 0.00748225 2024-09-11T07:38:02.492 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:02.524 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:02.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:02.525 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:02.626 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:02.667 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:02.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:02.668 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:02.769 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:02.800 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:02.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:02.801 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:02.902 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:02.935 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:02.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:02.937 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:03.037 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:03.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:03.082 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:03.082 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:03.183 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:03.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:03.225 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:03.225 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:03.230 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:38:03.231 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:38:03.271 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:38:03.271 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:38:03.327 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:03.350 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:03.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:03.351 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:03.410 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:38:03.410 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:38:03.440 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:38:03.440 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:38:03.451 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:03.472 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:38:03.473 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:38:03.473 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:03 smithi022 ceph-mon[29462]: pgmap v513: 148 pgs: 148 active+clean; 3.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 86 MiB/s wr, 1.37k op/s 2024-09-11T07:38:03.473 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:03 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1681590580' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:38:03.473 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:03 smithi022 ceph-mon[37330]: pgmap v513: 148 pgs: 148 active+clean; 3.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 86 MiB/s wr, 1.37k op/s 2024-09-11T07:38:03.473 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:03 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1681590580' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:38:03.477 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 26 16 54792 54776 131.648 84.0625 0.00579351 0.00759098 2024-09-11T07:38:03.483 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:03.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:03.484 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:03.507 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:38:03.507 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:38:03.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:03 smithi104 ceph-mon[33507]: pgmap v513: 148 pgs: 148 active+clean; 3.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 86 MiB/s wr, 1.37k op/s 2024-09-11T07:38:03.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:03 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1681590580' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:38:03.537 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:38:03.537 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:38:03.573 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:38:03.574 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:38:03.585 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:03.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:03.629 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:03.629 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:03.730 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:03.765 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:03.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:03.766 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:03.867 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:03.906 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:03.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:03.907 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:04.008 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:04.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:04.043 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:04.043 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:04.144 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:04.178 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:04.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:04.179 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:04.280 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:04.325 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:04.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:04.326 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:04.426 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:04.457 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:04.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:04.458 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:04.477 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 27 16 56219 56203 130.075 89.1875 0.00644004 0.00768286 2024-09-11T07:38:04.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:04 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2659447588' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:38:04.530 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:38:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:38:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:38:04.559 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:04.592 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:04.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:04.593 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:04.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:04 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2659447588' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:38:04.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:04 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2659447588' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:38:04.693 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:04.717 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is set 2024-09-11T07:38:04.730 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:04.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:04.732 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:04.832 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:04.868 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:04.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:04.869 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:04.970 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:04.998 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:04.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:04.999 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:05.100 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:05.140 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:05.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:05.142 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:05.242 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:05.270 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:05.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:05.271 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:05.371 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:05.402 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:05.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:05.403 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:05.478 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 28 16 57804 57788 128.967 99.0625 0.00866562 0.00774891 2024-09-11T07:38:05.504 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:05.524 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:38:05.527 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:05 smithi022 ceph-mon[29462]: pgmap v514: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-11T07:38:05.527 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:05 smithi022 ceph-mon[29462]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:38:05.527 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:05 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2659447588' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:38:05.527 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:05 smithi022 ceph-mon[29462]: osdmap e256: 8 total, 8 up, 8 in 2024-09-11T07:38:05.527 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:05 smithi022 ceph-mon[37330]: pgmap v514: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-11T07:38:05.527 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:05 smithi022 ceph-mon[37330]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:38:05.528 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:05 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2659447588' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:38:05.528 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:05 smithi022 ceph-mon[37330]: osdmap e256: 8 total, 8 up, 8 in 2024-09-11T07:38:05.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:05 smithi104 ceph-mon[33507]: pgmap v514: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-11T07:38:05.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:05 smithi104 ceph-mon[33507]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:38:05.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:05 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2659447588' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:38:05.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:05 smithi104 ceph-mon[33507]: osdmap e256: 8 total, 8 up, 8 in 2024-09-11T07:38:05.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:05.547 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:05.547 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:05.648 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:05.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:05.670 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:05.670 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:05.770 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:05.806 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:05.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:05.807 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:05.907 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:05.943 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:05.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:05.944 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:06.045 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:06.054 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:38:06.074 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:06.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:06.075 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:06.176 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:06.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:06.217 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:06.218 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:06.318 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:06.358 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:06.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:06.360 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:06.460 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:06.478 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 29 16 59312 59296 127.769 94.25 0.00745634 0.00782089 2024-09-11T07:38:06.499 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:06.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:06.501 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:06.602 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:06.628 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:06.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:06.629 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:06.729 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:06.765 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:06.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:06.766 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:06.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:06 smithi104 ceph-mon[33507]: pgmap v516: 148 pgs: 148 active+clean; 3.4 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-11T07:38:06.867 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:06.905 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:06 smithi022 ceph-mon[37330]: pgmap v516: 148 pgs: 148 active+clean; 3.4 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-11T07:38:06.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:06 smithi022 ceph-mon[29462]: pgmap v516: 148 pgs: 148 active+clean; 3.4 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-11T07:38:06.913 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:06.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:06.915 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:07.015 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:07.055 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:07.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:07.057 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:07.157 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:07.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:07.190 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:07.190 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:07.291 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:07.318 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:07.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:07.319 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:07.420 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:07.453 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:07.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:07.454 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:07.478 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 30 16 60927 60911 126.874 100.938 0.00985892 0.00787692 2024-09-11T07:38:07.554 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:07.585 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:07.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:07.586 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:07.687 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:07.730 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:07.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:07.731 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:07.832 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:07.870 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:07.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:07.871 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:07.972 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:08.006 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:08.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:08.007 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:08.108 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:08.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:08.159 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:08.159 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:08.260 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:08.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:08.299 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:08.300 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:08.400 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:08.416 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:38:08.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:38:08.445 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:08.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:08.446 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:08.478 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 31 16 62492 62476 125.937 97.8125 0.0103394 0.00793581 2024-09-11T07:38:08.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:38:08.547 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:08.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:08.589 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:08.589 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:08.649 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:38:08.690 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:08.725 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:08.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:08.726 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:08.826 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:08.854 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:08.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:08.855 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:08.956 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:08.983 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:08.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:08.984 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:09.084 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:09.121 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:09.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:09.122 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:09.223 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:09.254 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:09.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:09.256 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:09.352 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T07:38:09.357 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:09.387 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:09.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:09.388 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:09.391 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:09 smithi022 ceph-mon[29462]: pgmap v517: 148 pgs: 148 active+clean; 3.4 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-11T07:38:09.391 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:09 smithi022 ceph-mon[37330]: pgmap v517: 148 pgs: 148 active+clean; 3.4 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-11T07:38:09.481 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 32 16 63973 63957 124.89 92.5625 0.00713583 0.00800162 2024-09-11T07:38:09.489 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:09.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:09 smithi104 ceph-mon[33507]: pgmap v517: 148 pgs: 148 active+clean; 3.4 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-11T07:38:09.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:09.539 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:09.539 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:09.640 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:09.699 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:09.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:09.700 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:09.801 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:09.836 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:09.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:09.837 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:09.938 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:09.997 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:09.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:09.998 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:10.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:38:09 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:38:10.099 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:10.151 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:10.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:10.152 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:10.252 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:10.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:10.280 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:10.280 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:10.381 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:10.416 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:10.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:10.417 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:10.479 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 33 16 65330 65314 123.675 84.8125 0.0150185 0.00808054 2024-09-11T07:38:10.517 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:10.553 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:10.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:10.554 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:10.631 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:38:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:38:10] "GET /metrics HTTP/1.1" 200 224281 "" "Prometheus/2.43.0" 2024-09-11T07:38:10.655 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:10.682 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:10.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:10.684 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:10.784 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:10.811 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:10.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:10.812 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:10.913 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:10.947 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:10.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:10.948 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:11.048 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:11.075 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:11.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:11.077 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:11.177 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:11.208 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:11.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:11.209 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:11.310 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:11.338 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:11.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:11.339 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:11.440 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:11.473 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:11.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:11.474 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:11.479 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 34 16 66546 66530 122.273 76 0.0081984 0.00817294 2024-09-11T07:38:11.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:11 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:38:11.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:11 smithi104 ceph-mon[33507]: pgmap v518: 148 pgs: 148 active+clean; 3.9 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 97 MiB/s wr, 1.56k op/s 2024-09-11T07:38:11.575 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:11.603 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:11.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:11.604 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:11.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:11 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:38:11.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:11 smithi022 ceph-mon[29462]: pgmap v518: 148 pgs: 148 active+clean; 3.9 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 97 MiB/s wr, 1.56k op/s 2024-09-11T07:38:11.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:11 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:38:11.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:11 smithi022 ceph-mon[37330]: pgmap v518: 148 pgs: 148 active+clean; 3.9 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 97 MiB/s wr, 1.56k op/s 2024-09-11T07:38:11.706 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:11.739 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:11.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:11.740 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:11.841 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:11.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:11.872 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:11.872 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:11.900 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:38:11.911 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:38:11.973 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:12.002 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:12.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:12.004 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:12.104 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:12.128 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:12.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:12.129 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:12.230 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:12.254 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:12.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:12.255 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:12.356 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:12.385 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:12.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:12.386 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:12.479 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 35 16 67527 67511 120.531 61.3125 0.00714547 0.00829151 2024-09-11T07:38:12.487 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:12.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:12 smithi022 ceph-mon[29462]: from='client.25435 v1:172.21.15.22:0/781561039' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:38:12.505 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:12 smithi022 ceph-mon[37330]: from='client.25435 v1:172.21.15.22:0/781561039' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:38:12.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:12.529 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:12.529 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:12.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:12 smithi104 ceph-mon[33507]: from='client.25435 v1:172.21.15.22:0/781561039' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:38:12.630 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:12.663 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:12.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:12.664 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:12.764 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:12.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:12.797 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:12.797 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:12.879 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:38:12.879 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:38:12.898 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:12.929 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:12.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:12.930 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:13.031 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:13.069 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:13.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:13.070 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:13.171 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:13.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:13.208 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:13.208 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:13.309 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:13.340 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:13.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:13.343 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:13.444 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:13.476 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:13.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:13.478 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:13.480 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 36 16 69073 69057 119.867 96.625 0.014701 0.00833793 2024-09-11T07:38:13.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:13 smithi104 ceph-mon[33507]: pgmap v519: 148 pgs: 148 active+clean; 3.9 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 97 MiB/s wr, 1.56k op/s 2024-09-11T07:38:13.579 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:13.602 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:13 smithi022 ceph-mon[29462]: pgmap v519: 148 pgs: 148 active+clean; 3.9 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 97 MiB/s wr, 1.56k op/s 2024-09-11T07:38:13.603 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:13 smithi022 ceph-mon[37330]: pgmap v519: 148 pgs: 148 active+clean; 3.9 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 97 MiB/s wr, 1.56k op/s 2024-09-11T07:38:13.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:13.619 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:13.619 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:13.658 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-11T07:38:13.720 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:13.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:13.789 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:13.789 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:13.890 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:13.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:13.922 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:13.922 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:14.023 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:14.073 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:14.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:14.074 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:14.123 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":518,"stamp":"2024-09-11T07:38:10.188195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4196473052,"num_objects":64220,"num_object_clones":0,"num_object_copies":192660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64220,"num_whiteouts":0,"num_read":1969,"num_read_kb":1826,"num_write":64853,"num_write_kb":4099207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":590946,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64958,"ondisk_log_size":64958,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":14584900,"kb_used_data":12137348,"kb_used_omap":16,"kb_used_meta":2447471,"kb_avail":735376316,"statfs":{"total":767960285184,"available":753025347584,"internally_reserved":0,"allocated":12428644352,"data_stored":12425372732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16530,"internal_metadata":2506211182},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1021706240,"num_objects":15590,"num_object_clones":0,"num_object_copies":46770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15590,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":15590,"num_write_kb":997760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003451"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":258,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.308120+0000","last_change":"2024-09-11T07:37:29.007467+0000","last_active":"2024-09-11T07:37:43.308120+0000","last_peered":"2024-09-11T07:37:43.308120+0000","last_clean":"2024-09-11T07:37:43.308120+0000","last_became_active":"2024-09-11T07:32:00.501534+0000","last_became_peered":"2024-09-11T07:32:00.501534+0000","last_unstale":"2024-09-11T07:37:43.308120+0000","last_undegraded":"2024-09-11T07:37:43.308120+0000","last_fullsized":"2024-09-11T07:37:43.308120+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:29.007411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:29.007411+0000","last_clean_scrub_stamp":"2024-09-11T07:37:29.007411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:43.805924+0000","scrub_duration":0.003998723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":274,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.324708+0000","last_change":"2024-09-11T07:36:30.991280+0000","last_active":"2024-09-11T07:37:43.324708+0000","last_peered":"2024-09-11T07:37:43.324708+0000","last_clean":"2024-09-11T07:37:43.324708+0000","last_became_active":"2024-09-11T07:32:00.508572+0000","last_became_peered":"2024-09-11T07:32:00.508572+0000","last_unstale":"2024-09-11T07:37:43.324708+0000","last_undegraded":"2024-09-11T07:37:43.324708+0000","last_fullsized":"2024-09-11T07:37:43.324708+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:30.991202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:30.991202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090469859999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"234'34","reported_seq":295,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.309050+0000","last_change":"2024-09-11T07:36:28.772225+0000","last_active":"2024-09-11T07:37:43.309050+0000","last_peered":"2024-09-11T07:37:43.309050+0000","last_clean":"2024-09-11T07:37:43.309050+0000","last_became_active":"2024-09-11T07:26:33.544627+0000","last_became_peered":"2024-09-11T07:26:33.544627+0000","last_unstale":"2024-09-11T07:37:43.309050+0000","last_undegraded":"2024-09-11T07:37:43.309050+0000","last_fullsized":"2024-09-11T07:37:43.309050+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-09-11T07:36:28.772126+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:36:28.772126+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016007773999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":263,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.307034+0000","last_change":"2024-09-11T07:37:39.609450+0000","last_active":"2024-09-11T07:37:43.307034+0000","last_peered":"2024-09-11T07:37:43.307034+0000","last_clean":"2024-09-11T07:37:43.307034+0000","last_became_active":"2024-09-11T07:26:35.556296+0000","last_became_peered":"2024-09-11T07:26:35.556296+0000","last_unstale":"2024-09-11T07:37:43.307034+0000","last_undegraded":"2024-09-11T07:37:43.307034+0000","last_fullsized":"2024-09-11T07:37:43.307034+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:39.609357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:39.609357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:46.568905+0000","scrub_duration":0.0026059439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":257,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.309171+0000","last_change":"2024-09-11T07:36:33.755073+0000","last_active":"2024-09-11T07:37:43.309171+0000","last_peered":"2024-09-11T07:37:43.309171+0000","last_clean":"2024-09-11T07:37:43.309171+0000","last_became_active":"2024-09-11T07:26:31.420178+0000","last_became_peered":"2024-09-11T07:26:31.420178+0000","last_unstale":"2024-09-11T07:37:43.309171+0000","last_undegraded":"2024-09-11T07:37:43.309171+0000","last_fullsized":"2024-09-11T07:37:43.309171+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:33.754983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:33.754983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034784899999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"234'19","reported_seq":284,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.307158+0000","last_change":"2024-09-11T07:36:42.709007+0000","last_active":"2024-09-11T07:37:43.307158+0000","last_peered":"2024-09-11T07:37:43.307158+0000","last_clean":"2024-09-11T07:37:43.307158+0000","last_became_active":"2024-09-11T07:26:33.540732+0000","last_became_peered":"2024-09-11T07:26:33.540732+0000","last_unstale":"2024-09-11T07:37:43.307158+0000","last_undegraded":"2024-09-11T07:37:43.307158+0000","last_fullsized":"2024-09-11T07:37:43.307158+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:36:42.708941+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:36:42.708941+0000","last_clean_scrub_stamp":"2024-09-11T07:36:42.708941+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0088444879999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":272,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.321195+0000","last_change":"2024-09-11T07:37:43.087423+0000","last_active":"2024-09-11T07:37:43.321195+0000","last_peered":"2024-09-11T07:37:43.321195+0000","last_clean":"2024-09-11T07:37:43.321195+0000","last_became_active":"2024-09-11T07:32:00.513650+0000","last_became_peered":"2024-09-11T07:32:00.513650+0000","last_unstale":"2024-09-11T07:37:43.321195+0000","last_undegraded":"2024-09-11T07:37:43.321195+0000","last_fullsized":"2024-09-11T07:37:43.321195+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:43.087302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:37:43.087302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:52.305128+0000","scrub_duration":0.00405653,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":251,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.307156+0000","last_change":"2024-09-11T07:36:48.684925+0000","last_active":"2024-09-11T07:37:43.307156+0000","last_peered":"2024-09-11T07:37:43.307156+0000","last_clean":"2024-09-11T07:37:43.307156+0000","last_became_active":"2024-09-11T07:26:37.566910+0000","last_became_peered":"2024-09-11T07:26:37.566910+0000","last_unstale":"2024-09-11T07:37:43.307156+0000","last_undegraded":"2024-09-11T07:37:43.307156+0000","last_fullsized":"2024-09-11T07:37:43.307156+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:48.684838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:48.684838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058146650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":262,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.315571+0000","last_change":"2024-09-11T07:37:34.542508+0000","last_active":"2024-09-11T07:37:43.315571+0000","last_peered":"2024-09-11T07:37:43.315571+0000","last_clean":"2024-09-11T07:37:43.315571+0000","last_became_active":"2024-09-11T07:32:00.516662+0000","last_became_peered":"2024-09-11T07:32:00.516662+0000","last_unstale":"2024-09-11T07:37:43.315571+0000","last_undegraded":"2024-09-11T07:37:43.315571+0000","last_fullsized":"2024-09-11T07:37:43.315571+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:34.542441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:34.542441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:53.707135+0000","scrub_duration":0.0015726010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"234'9","reported_seq":280,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.307357+0000","last_change":"2024-09-11T07:37:37.558147+0000","last_active":"2024-09-11T07:37:43.307357+0000","last_peered":"2024-09-11T07:37:43.307357+0000","last_clean":"2024-09-11T07:37:43.307357+0000","last_became_active":"2024-09-11T07:26:33.540351+0000","last_became_peered":"2024-09-11T07:26:33.540351+0000","last_unstale":"2024-09-11T07:37:43.307357+0000","last_undegraded":"2024-09-11T07:37:43.307357+0000","last_fullsized":"2024-09-11T07:37:43.307357+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'9","last_scrub_stamp":"2024-09-11T07:37:37.558036+0000","last_deep_scrub":"82'7","last_deep_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_clean_scrub_stamp":"2024-09-11T07:37:37.558036+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:49.821269+0000","scrub_duration":0.0034408770000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":228,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.313860+0000","last_change":"2024-09-11T07:35:50.332326+0000","last_active":"2024-09-11T07:37:43.313860+0000","last_peered":"2024-09-11T07:37:43.313860+0000","last_clean":"2024-09-11T07:37:43.313860+0000","last_became_active":"2024-09-11T07:32:00.513206+0000","last_became_peered":"2024-09-11T07:32:00.513206+0000","last_unstale":"2024-09-11T07:37:43.313860+0000","last_undegraded":"2024-09-11T07:37:43.313860+0000","last_fullsized":"2024-09-11T07:37:43.313860+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:50.332244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.973323+0000","last_clean_scrub_stamp":"2024-09-11T07:35:50.332244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034655860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":251,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311302+0000","last_change":"2024-09-11T07:37:36.462873+0000","last_active":"2024-09-11T07:37:43.311302+0000","last_peered":"2024-09-11T07:37:43.311302+0000","last_clean":"2024-09-11T07:37:43.311302+0000","last_became_active":"2024-09-11T07:26:37.561758+0000","last_became_peered":"2024-09-11T07:26:37.561758+0000","last_unstale":"2024-09-11T07:37:43.311302+0000","last_undegraded":"2024-09-11T07:37:43.311302+0000","last_fullsized":"2024-09-11T07:37:43.311302+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:36.462794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:45.149916+0000","last_clean_scrub_stamp":"2024-09-11T07:37:36.462794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:39.725240+0000","scrub_duration":0.001718252,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":227,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311612+0000","last_change":"2024-09-11T07:37:40.336139+0000","last_active":"2024-09-11T07:37:43.311612+0000","last_peered":"2024-09-11T07:37:43.311612+0000","last_clean":"2024-09-11T07:37:43.311612+0000","last_became_active":"2024-09-11T07:32:00.505709+0000","last_became_peered":"2024-09-11T07:32:00.505709+0000","last_unstale":"2024-09-11T07:37:43.311612+0000","last_undegraded":"2024-09-11T07:37:43.311612+0000","last_fullsized":"2024-09-11T07:37:43.311612+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:40.336050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_clean_scrub_stamp":"2024-09-11T07:37:40.336050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:58.024969+0000","scrub_duration":0.0021490649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"234'19","reported_seq":284,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.307698+0000","last_change":"2024-09-11T07:36:27.746378+0000","last_active":"2024-09-11T07:37:43.307698+0000","last_peered":"2024-09-11T07:37:43.307698+0000","last_clean":"2024-09-11T07:37:43.307698+0000","last_became_active":"2024-09-11T07:26:33.544386+0000","last_became_peered":"2024-09-11T07:26:33.544386+0000","last_unstale":"2024-09-11T07:37:43.307698+0000","last_undegraded":"2024-09-11T07:37:43.307698+0000","last_fullsized":"2024-09-11T07:37:43.307698+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'13","last_scrub_stamp":"2024-09-11T07:36:27.746295+0000","last_deep_scrub":"136'13","last_deep_scrub_stamp":"2024-09-11T07:31:08.042198+0000","last_clean_scrub_stamp":"2024-09-11T07:36:27.746295+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0089695690000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":263,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311444+0000","last_change":"2024-09-11T07:35:34.150965+0000","last_active":"2024-09-11T07:37:43.311444+0000","last_peered":"2024-09-11T07:37:43.311444+0000","last_clean":"2024-09-11T07:37:43.311444+0000","last_became_active":"2024-09-11T07:32:00.502337+0000","last_became_peered":"2024-09-11T07:32:00.502337+0000","last_unstale":"2024-09-11T07:37:43.311444+0000","last_undegraded":"2024-09-11T07:37:43.311444+0000","last_fullsized":"2024-09-11T07:37:43.311444+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:34.150289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:35:34.150289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023057149999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":268,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.321084+0000","last_change":"2024-09-11T07:36:31.977319+0000","last_active":"2024-09-11T07:37:43.321084+0000","last_peered":"2024-09-11T07:37:43.321084+0000","last_clean":"2024-09-11T07:37:43.321084+0000","last_became_active":"2024-09-11T07:32:00.530335+0000","last_became_peered":"2024-09-11T07:32:00.530335+0000","last_unstale":"2024-09-11T07:37:43.321084+0000","last_undegraded":"2024-09-11T07:37:43.321084+0000","last_fullsized":"2024-09-11T07:37:43.321084+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:31.977240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:31.977240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0074114259999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"256'3979","reported_seq":3996,"reported_epoch":256,"state":"active+clean","last_fresh":"2024-09-11T07:38:09.456533+0000","last_change":"2024-09-11T07:37:26.029984+0000","last_active":"2024-09-11T07:38:09.456533+0000","last_peered":"2024-09-11T07:38:09.456533+0000","last_clean":"2024-09-11T07:38:09.456533+0000","last_became_active":"2024-09-11T07:37:26.029625+0000","last_became_peered":"2024-09-11T07:37:26.029625+0000","last_unstale":"2024-09-11T07:38:09.456533+0000","last_undegraded":"2024-09-11T07:38:09.456533+0000","last_fullsized":"2024-09-11T07:38:09.456533+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":3979,"log_dups_size":0,"ondisk_log_size":3979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:33.495400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":260767744,"num_objects":3979,"num_object_clones":0,"num_object_copies":11937,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3979,"num_write_kb":254656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"234'31","reported_seq":360,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.308037+0000","last_change":"2024-09-11T07:36:39.737025+0000","last_active":"2024-09-11T07:37:43.308037+0000","last_peered":"2024-09-11T07:37:43.308037+0000","last_clean":"2024-09-11T07:37:43.308037+0000","last_became_active":"2024-09-11T07:29:51.482755+0000","last_became_peered":"2024-09-11T07:29:51.482755+0000","last_unstale":"2024-09-11T07:37:43.308037+0000","last_undegraded":"2024-09-11T07:37:43.308037+0000","last_fullsized":"2024-09-11T07:37:43.308037+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'21","last_scrub_stamp":"2024-09-11T07:36:39.736944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:36:39.736944+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034170279999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":257,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.315259+0000","last_change":"2024-09-11T07:36:51.418345+0000","last_active":"2024-09-11T07:37:43.315259+0000","last_peered":"2024-09-11T07:37:43.315259+0000","last_clean":"2024-09-11T07:37:43.315259+0000","last_became_active":"2024-09-11T07:32:00.507191+0000","last_became_peered":"2024-09-11T07:32:00.507191+0000","last_unstale":"2024-09-11T07:37:43.315259+0000","last_undegraded":"2024-09-11T07:37:43.315259+0000","last_fullsized":"2024-09-11T07:37:43.315259+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:51.418258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:51.418258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001500181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":245,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.322059+0000","last_change":"2024-09-11T07:37:17.980251+0000","last_active":"2024-09-11T07:37:43.322059+0000","last_peered":"2024-09-11T07:37:43.322059+0000","last_clean":"2024-09-11T07:37:43.322059+0000","last_became_active":"2024-09-11T07:26:35.557989+0000","last_became_peered":"2024-09-11T07:26:35.557989+0000","last_unstale":"2024-09-11T07:37:43.322059+0000","last_undegraded":"2024-09-11T07:37:43.322059+0000","last_fullsized":"2024-09-11T07:37:43.322059+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:34:00.348684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:34:00.348684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036577580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":261,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311383+0000","last_change":"2024-09-11T07:37:21.385329+0000","last_active":"2024-09-11T07:37:43.311383+0000","last_peered":"2024-09-11T07:37:43.311383+0000","last_clean":"2024-09-11T07:37:43.311383+0000","last_became_active":"2024-09-11T07:26:37.564554+0000","last_became_peered":"2024-09-11T07:26:37.564554+0000","last_unstale":"2024-09-11T07:37:43.311383+0000","last_undegraded":"2024-09-11T07:37:43.311383+0000","last_fullsized":"2024-09-11T07:37:43.311383+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:21.385271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:21.385271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:27.423807+0000","scrub_duration":0.001463322,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"256'3995","reported_seq":4011,"reported_epoch":256,"state":"active+clean","last_fresh":"2024-09-11T07:38:09.859328+0000","last_change":"2024-09-11T07:37:26.029943+0000","last_active":"2024-09-11T07:38:09.859328+0000","last_peered":"2024-09-11T07:38:09.859328+0000","last_clean":"2024-09-11T07:38:09.859328+0000","last_became_active":"2024-09-11T07:37:26.029574+0000","last_became_peered":"2024-09-11T07:37:26.029574+0000","last_unstale":"2024-09-11T07:38:09.859328+0000","last_undegraded":"2024-09-11T07:38:09.859328+0000","last_fullsized":"2024-09-11T07:38:09.859328+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":3995,"log_dups_size":0,"ondisk_log_size":3995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:25.412197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":261816320,"num_objects":3995,"num_object_clones":0,"num_object_copies":11985,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3995,"num_write_kb":255680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"234'25","reported_seq":290,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.307993+0000","last_change":"2024-09-11T07:36:43.704258+0000","last_active":"2024-09-11T07:37:43.307993+0000","last_peered":"2024-09-11T07:37:43.307993+0000","last_clean":"2024-09-11T07:37:43.307993+0000","last_became_active":"2024-09-11T07:26:33.535411+0000","last_became_peered":"2024-09-11T07:26:33.535411+0000","last_unstale":"2024-09-11T07:37:43.307993+0000","last_undegraded":"2024-09-11T07:37:43.307993+0000","last_fullsized":"2024-09-11T07:37:43.307993+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'25","last_scrub_stamp":"2024-09-11T07:36:43.704170+0000","last_deep_scrub":"112'15","last_deep_scrub_stamp":"2024-09-11T07:31:10.005896+0000","last_clean_scrub_stamp":"2024-09-11T07:36:43.704170+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048642039999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":276,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.321763+0000","last_change":"2024-09-11T07:37:36.098229+0000","last_active":"2024-09-11T07:37:43.321763+0000","last_peered":"2024-09-11T07:37:43.321763+0000","last_clean":"2024-09-11T07:37:43.321763+0000","last_became_active":"2024-09-11T07:32:00.506615+0000","last_became_peered":"2024-09-11T07:32:00.506615+0000","last_unstale":"2024-09-11T07:37:43.321763+0000","last_undegraded":"2024-09-11T07:37:43.321763+0000","last_fullsized":"2024-09-11T07:37:43.321763+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:36.098175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:36.098175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:54.532896+0000","scrub_duration":0.0017555839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"151'10","reported_seq":429,"reported_epoch":256,"state":"active+clean","last_fresh":"2024-09-11T07:38:04.291182+0000","last_change":"2024-09-11T07:37:43.141088+0000","last_active":"2024-09-11T07:38:04.291182+0000","last_peered":"2024-09-11T07:38:04.291182+0000","last_clean":"2024-09-11T07:38:04.291182+0000","last_became_active":"2024-09-11T07:32:00.507447+0000","last_became_peered":"2024-09-11T07:32:00.507447+0000","last_unstale":"2024-09-11T07:38:04.291182+0000","last_undegraded":"2024-09-11T07:38:04.291182+0000","last_fullsized":"2024-09-11T07:38:04.291182+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:37:43.140999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:43.140999+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:57.947183+0000","scrub_duration":0.0039651130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":251,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.308734+0000","last_change":"2024-09-11T07:37:22.546982+0000","last_active":"2024-09-11T07:37:43.308734+0000","last_peered":"2024-09-11T07:37:43.308734+0000","last_clean":"2024-09-11T07:37:43.308734+0000","last_became_active":"2024-09-11T07:26:37.566919+0000","last_became_peered":"2024-09-11T07:26:37.566919+0000","last_unstale":"2024-09-11T07:37:43.308734+0000","last_undegraded":"2024-09-11T07:37:43.308734+0000","last_fullsized":"2024-09-11T07:37:43.308734+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:22.546931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:22.546931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:42.991401+0000","scrub_duration":0.001345527,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"256'4043","reported_seq":4059,"reported_epoch":256,"state":"active+clean","last_fresh":"2024-09-11T07:38:09.853591+0000","last_change":"2024-09-11T07:37:25.851826+0000","last_active":"2024-09-11T07:38:09.853591+0000","last_peered":"2024-09-11T07:38:09.853591+0000","last_clean":"2024-09-11T07:38:09.853591+0000","last_became_active":"2024-09-11T07:37:25.851536+0000","last_became_peered":"2024-09-11T07:37:25.851536+0000","last_unstale":"2024-09-11T07:38:09.853591+0000","last_undegraded":"2024-09-11T07:38:09.853591+0000","last_fullsized":"2024-09-11T07:38:09.853591+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4043,"log_dups_size":0,"ondisk_log_size":4043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:42.549849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264962048,"num_objects":4043,"num_object_clones":0,"num_object_copies":12129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4043,"num_write_kb":258752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"234'34","reported_seq":304,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.321541+0000","last_change":"2024-09-11T07:37:16.938565+0000","last_active":"2024-09-11T07:37:43.321541+0000","last_peered":"2024-09-11T07:37:43.321541+0000","last_clean":"2024-09-11T07:37:43.321541+0000","last_became_active":"2024-09-11T07:26:33.549733+0000","last_became_peered":"2024-09-11T07:26:33.549733+0000","last_unstale":"2024-09-11T07:37:43.321541+0000","last_undegraded":"2024-09-11T07:37:43.321541+0000","last_fullsized":"2024-09-11T07:37:43.321541+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'34","last_scrub_stamp":"2024-09-11T07:37:16.938496+0000","last_deep_scrub":"83'22","last_deep_scrub_stamp":"2024-09-11T07:34:04.375457+0000","last_clean_scrub_stamp":"2024-09-11T07:37:16.938496+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:44.726347+0000","scrub_duration":0.0028468529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":276,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.321380+0000","last_change":"2024-09-11T07:35:40.786431+0000","last_active":"2024-09-11T07:37:43.321380+0000","last_peered":"2024-09-11T07:37:43.321380+0000","last_clean":"2024-09-11T07:37:43.321380+0000","last_became_active":"2024-09-11T07:32:00.513472+0000","last_became_peered":"2024-09-11T07:32:00.513472+0000","last_unstale":"2024-09-11T07:37:43.321380+0000","last_undegraded":"2024-09-11T07:37:43.321380+0000","last_fullsized":"2024-09-11T07:37:43.321380+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:40.786358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:35:40.786358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014952305000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":403,"reported_epoch":256,"state":"active+clean","last_fresh":"2024-09-11T07:38:04.262312+0000","last_change":"2024-09-11T07:37:30.657587+0000","last_active":"2024-09-11T07:38:04.262312+0000","last_peered":"2024-09-11T07:38:04.262312+0000","last_clean":"2024-09-11T07:38:04.262312+0000","last_became_active":"2024-09-11T07:26:35.560406+0000","last_became_peered":"2024-09-11T07:26:35.560406+0000","last_unstale":"2024-09-11T07:38:04.262312+0000","last_undegraded":"2024-09-11T07:38:04.262312+0000","last_fullsized":"2024-09-11T07:38:04.262312+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:37:30.657477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:30.657477+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:57.486387+0000","scrub_duration":0.001943012,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":251,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311686+0000","last_change":"2024-09-11T07:37:20.369255+0000","last_active":"2024-09-11T07:37:43.311686+0000","last_peered":"2024-09-11T07:37:43.311686+0000","last_clean":"2024-09-11T07:37:43.311686+0000","last_became_active":"2024-09-11T07:26:37.568005+0000","last_became_peered":"2024-09-11T07:26:37.568005+0000","last_unstale":"2024-09-11T07:37:43.311686+0000","last_undegraded":"2024-09-11T07:37:43.311686+0000","last_fullsized":"2024-09-11T07:37:43.311686+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:20.369199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:31.200685+0000","last_clean_scrub_stamp":"2024-09-11T07:37:20.369199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:32.604085+0000","scrub_duration":0.001371688,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"256'3917","reported_seq":3932,"reported_epoch":256,"state":"active+clean","last_fresh":"2024-09-11T07:38:09.202247+0000","last_change":"2024-09-11T07:37:25.855865+0000","last_active":"2024-09-11T07:38:09.202247+0000","last_peered":"2024-09-11T07:38:09.202247+0000","last_clean":"2024-09-11T07:38:09.202247+0000","last_became_active":"2024-09-11T07:37:25.855577+0000","last_became_peered":"2024-09-11T07:37:25.855577+0000","last_unstale":"2024-09-11T07:38:09.202247+0000","last_undegraded":"2024-09-11T07:38:09.202247+0000","last_fullsized":"2024-09-11T07:38:09.202247+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":3917,"log_dups_size":0,"ondisk_log_size":3917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:48.134227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":256704512,"num_objects":3917,"num_object_clones":0,"num_object_copies":11751,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3917,"num_write_kb":250688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"234'24","reported_seq":301,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.308877+0000","last_change":"2024-09-11T07:37:36.542263+0000","last_active":"2024-09-11T07:37:43.308877+0000","last_peered":"2024-09-11T07:37:43.308877+0000","last_clean":"2024-09-11T07:37:43.308877+0000","last_became_active":"2024-09-11T07:26:33.540777+0000","last_became_peered":"2024-09-11T07:26:33.540777+0000","last_unstale":"2024-09-11T07:37:43.308877+0000","last_undegraded":"2024-09-11T07:37:43.308877+0000","last_fullsized":"2024-09-11T07:37:43.308877+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'24","last_scrub_stamp":"2024-09-11T07:37:36.542161+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-09-11T07:29:38.112111+0000","last_clean_scrub_stamp":"2024-09-11T07:37:36.542161+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:55.053665+0000","scrub_duration":0.001886759,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":251,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.313663+0000","last_change":"2024-09-11T07:36:37.445237+0000","last_active":"2024-09-11T07:37:43.313663+0000","last_peered":"2024-09-11T07:37:43.313663+0000","last_clean":"2024-09-11T07:37:43.313663+0000","last_became_active":"2024-09-11T07:32:00.518538+0000","last_became_peered":"2024-09-11T07:32:00.518538+0000","last_unstale":"2024-09-11T07:37:43.313663+0000","last_undegraded":"2024-09-11T07:37:43.313663+0000","last_fullsized":"2024-09-11T07:37:43.313663+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:37.444760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:37.444760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012636722,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":253,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311923+0000","last_change":"2024-09-11T07:35:29.166617+0000","last_active":"2024-09-11T07:37:43.311923+0000","last_peered":"2024-09-11T07:37:43.311923+0000","last_clean":"2024-09-11T07:37:43.311923+0000","last_became_active":"2024-09-11T07:26:35.569081+0000","last_became_peered":"2024-09-11T07:26:35.569081+0000","last_unstale":"2024-09-11T07:37:43.311923+0000","last_undegraded":"2024-09-11T07:37:43.311923+0000","last_fullsized":"2024-09-11T07:37:43.311923+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:29.166488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:35:29.166488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059891079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":251,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.310761+0000","last_change":"2024-09-11T07:36:50.920711+0000","last_active":"2024-09-11T07:37:43.310761+0000","last_peered":"2024-09-11T07:37:43.310761+0000","last_clean":"2024-09-11T07:37:43.310761+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:37:43.310761+0000","last_undegraded":"2024-09-11T07:37:43.310761+0000","last_fullsized":"2024-09-11T07:37:43.310761+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:50.920625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:46.853387+0000","last_clean_scrub_stamp":"2024-09-11T07:36:50.920625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:19.177270+0000","scrub_duration":0.012404324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"256'3989","reported_seq":4005,"reported_epoch":256,"state":"active+clean","last_fresh":"2024-09-11T07:38:09.859756+0000","last_change":"2024-09-11T07:37:25.855661+0000","last_active":"2024-09-11T07:38:09.859756+0000","last_peered":"2024-09-11T07:38:09.859756+0000","last_clean":"2024-09-11T07:38:09.859756+0000","last_became_active":"2024-09-11T07:37:25.854796+0000","last_became_peered":"2024-09-11T07:37:25.854796+0000","last_unstale":"2024-09-11T07:38:09.859756+0000","last_undegraded":"2024-09-11T07:38:09.859756+0000","last_fullsized":"2024-09-11T07:38:09.859756+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":3989,"log_dups_size":0,"ondisk_log_size":3989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:31.723924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":261423104,"num_objects":3989,"num_object_clones":0,"num_object_copies":11967,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3989,"num_write_kb":255296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"234'28","reported_seq":297,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.307233+0000","last_change":"2024-09-11T07:37:35.580526+0000","last_active":"2024-09-11T07:37:43.307233+0000","last_peered":"2024-09-11T07:37:43.307233+0000","last_clean":"2024-09-11T07:37:43.307233+0000","last_became_active":"2024-09-11T07:26:33.537850+0000","last_became_peered":"2024-09-11T07:26:33.537850+0000","last_unstale":"2024-09-11T07:37:43.307233+0000","last_undegraded":"2024-09-11T07:37:43.307233+0000","last_fullsized":"2024-09-11T07:37:43.307233+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'28","last_scrub_stamp":"2024-09-11T07:37:35.580463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:35.580463+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:50.893847+0000","scrub_duration":0.0019235789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":103,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.308816+0000","last_change":"2024-09-11T07:37:20.527569+0000","last_active":"2024-09-11T07:37:43.308816+0000","last_peered":"2024-09-11T07:37:43.308816+0000","last_clean":"2024-09-11T07:37:43.308816+0000","last_became_active":"2024-09-11T07:36:05.226733+0000","last_became_peered":"2024-09-11T07:36:05.226733+0000","last_unstale":"2024-09-11T07:37:43.308816+0000","last_undegraded":"2024-09-11T07:37:43.308816+0000","last_fullsized":"2024-09-11T07:37:43.308816+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:20.527511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:20.527511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:31.232016+0000","scrub_duration":0.0011632039999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"84'9","reported_seq":402,"reported_epoch":256,"state":"active+clean","last_fresh":"2024-09-11T07:38:04.644809+0000","last_change":"2024-09-11T07:37:37.000676+0000","last_active":"2024-09-11T07:38:04.644809+0000","last_peered":"2024-09-11T07:38:04.644809+0000","last_clean":"2024-09-11T07:38:04.644809+0000","last_became_active":"2024-09-11T07:26:35.562997+0000","last_became_peered":"2024-09-11T07:26:35.562997+0000","last_unstale":"2024-09-11T07:38:04.644809+0000","last_undegraded":"2024-09-11T07:38:04.644809+0000","last_fullsized":"2024-09-11T07:38:04.644809+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-11T07:37:37.000619+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-11T07:35:39.832547+0000","last_clean_scrub_stamp":"2024-09-11T07:37:37.000619+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:57.784083+0000","scrub_duration":0.0021426219999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":225,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.315060+0000","last_change":"2024-09-11T07:36:32.419622+0000","last_active":"2024-09-11T07:37:43.315060+0000","last_peered":"2024-09-11T07:37:43.315060+0000","last_clean":"2024-09-11T07:37:43.315060+0000","last_became_active":"2024-09-11T07:32:00.511005+0000","last_became_peered":"2024-09-11T07:32:00.511005+0000","last_unstale":"2024-09-11T07:37:43.315060+0000","last_undegraded":"2024-09-11T07:37:43.315060+0000","last_fullsized":"2024-09-11T07:37:43.315060+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:32.419555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:32.419555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046427200000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"256'3911","reported_seq":3927,"reported_epoch":256,"state":"active+clean","last_fresh":"2024-09-11T07:38:09.195073+0000","last_change":"2024-09-11T07:37:25.854170+0000","last_active":"2024-09-11T07:38:09.195073+0000","last_peered":"2024-09-11T07:38:09.195073+0000","last_clean":"2024-09-11T07:38:09.195073+0000","last_became_active":"2024-09-11T07:37:25.853857+0000","last_became_peered":"2024-09-11T07:37:25.853857+0000","last_unstale":"2024-09-11T07:38:09.195073+0000","last_undegraded":"2024-09-11T07:38:09.195073+0000","last_fullsized":"2024-09-11T07:38:09.195073+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":3911,"log_dups_size":0,"ondisk_log_size":3911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:29.440611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":256311296,"num_objects":3911,"num_object_clones":0,"num_object_copies":11733,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3911,"num_write_kb":250304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"234'38","reported_seq":320,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.309977+0000","last_change":"2024-09-11T07:35:52.942431+0000","last_active":"2024-09-11T07:37:43.309977+0000","last_peered":"2024-09-11T07:37:43.309977+0000","last_clean":"2024-09-11T07:37:43.309977+0000","last_became_active":"2024-09-11T07:32:00.520670+0000","last_became_peered":"2024-09-11T07:32:00.520670+0000","last_unstale":"2024-09-11T07:37:43.309977+0000","last_undegraded":"2024-09-11T07:37:43.309977+0000","last_fullsized":"2024-09-11T07:37:43.309977+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'26","last_scrub_stamp":"2024-09-11T07:35:52.942347+0000","last_deep_scrub":"87'26","last_deep_scrub_stamp":"2024-09-11T07:35:52.942347+0000","last_clean_scrub_stamp":"2024-09-11T07:35:52.942347+0000","objects_scrubbed":8,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010845960999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":266,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.326779+0000","last_change":"2024-09-11T07:37:33.081869+0000","last_active":"2024-09-11T07:37:43.326779+0000","last_peered":"2024-09-11T07:37:43.326779+0000","last_clean":"2024-09-11T07:37:43.326779+0000","last_became_active":"2024-09-11T07:26:31.421500+0000","last_became_peered":"2024-09-11T07:26:31.421500+0000","last_unstale":"2024-09-11T07:37:43.326779+0000","last_undegraded":"2024-09-11T07:37:43.326779+0000","last_fullsized":"2024-09-11T07:37:43.326779+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:33.081815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:33.081815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:54.240779+0000","scrub_duration":0.001426379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":263,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.307077+0000","last_change":"2024-09-11T07:36:30.786071+0000","last_active":"2024-09-11T07:37:43.307077+0000","last_peered":"2024-09-11T07:37:43.307077+0000","last_clean":"2024-09-11T07:37:43.307077+0000","last_became_active":"2024-09-11T07:32:00.499747+0000","last_became_peered":"2024-09-11T07:32:00.499747+0000","last_unstale":"2024-09-11T07:37:43.307077+0000","last_undegraded":"2024-09-11T07:37:43.307077+0000","last_fullsized":"2024-09-11T07:37:43.307077+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:30.785991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:27.968579+0000","last_clean_scrub_stamp":"2024-09-11T07:36:30.785991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011893737999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":251,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.307625+0000","last_change":"2024-09-11T07:37:28.623070+0000","last_active":"2024-09-11T07:37:43.307625+0000","last_peered":"2024-09-11T07:37:43.307625+0000","last_clean":"2024-09-11T07:37:43.307625+0000","last_became_active":"2024-09-11T07:26:37.564616+0000","last_became_peered":"2024-09-11T07:26:37.564616+0000","last_unstale":"2024-09-11T07:37:43.307625+0000","last_undegraded":"2024-09-11T07:37:43.307625+0000","last_fullsized":"2024-09-11T07:37:43.307625+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:28.622998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:29.978798+0000","last_clean_scrub_stamp":"2024-09-11T07:37:28.622998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:42.675263+0000","scrub_duration":0.001843798,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"256'3937","reported_seq":3953,"reported_epoch":256,"state":"active+clean","last_fresh":"2024-09-11T07:38:09.220476+0000","last_change":"2024-09-11T07:37:25.854301+0000","last_active":"2024-09-11T07:38:09.220476+0000","last_peered":"2024-09-11T07:38:09.220476+0000","last_clean":"2024-09-11T07:38:09.220476+0000","last_became_active":"2024-09-11T07:37:25.853844+0000","last_became_peered":"2024-09-11T07:37:25.853844+0000","last_unstale":"2024-09-11T07:38:09.220476+0000","last_undegraded":"2024-09-11T07:38:09.220476+0000","last_fullsized":"2024-09-11T07:38:09.220476+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":3937,"log_dups_size":0,"ondisk_log_size":3937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:45.631635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258015232,"num_objects":3937,"num_object_clones":0,"num_object_copies":11811,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3937,"num_write_kb":251968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"236'32","reported_seq":326,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.324593+0000","last_change":"2024-09-11T07:36:50.024003+0000","last_active":"2024-09-11T07:37:43.324593+0000","last_peered":"2024-09-11T07:37:43.324593+0000","last_clean":"2024-09-11T07:37:43.324593+0000","last_became_active":"2024-09-11T07:32:00.506270+0000","last_became_peered":"2024-09-11T07:32:00.506270+0000","last_unstale":"2024-09-11T07:37:43.324593+0000","last_undegraded":"2024-09-11T07:37:43.324593+0000","last_fullsized":"2024-09-11T07:37:43.324593+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'32","last_scrub_stamp":"2024-09-11T07:36:50.023909+0000","last_deep_scrub":"236'32","last_deep_scrub_stamp":"2024-09-11T07:36:50.023909+0000","last_clean_scrub_stamp":"2024-09-11T07:36:50.023909+0000","objects_scrubbed":8,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.030217668999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":225,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.315502+0000","last_change":"2024-09-11T07:36:48.434408+0000","last_active":"2024-09-11T07:37:43.315502+0000","last_peered":"2024-09-11T07:37:43.315502+0000","last_clean":"2024-09-11T07:37:43.315502+0000","last_became_active":"2024-09-11T07:32:00.510631+0000","last_became_peered":"2024-09-11T07:32:00.510631+0000","last_unstale":"2024-09-11T07:37:43.315502+0000","last_undegraded":"2024-09-11T07:37:43.315502+0000","last_fullsized":"2024-09-11T07:37:43.315502+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:48.434320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:49.144041+0000","last_clean_scrub_stamp":"2024-09-11T07:36:48.434320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066312669999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":54,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.320761+0000","last_change":"2024-09-11T07:36:55.163926+0000","last_active":"2024-09-11T07:37:43.320761+0000","last_peered":"2024-09-11T07:37:43.320761+0000","last_clean":"2024-09-11T07:37:43.320761+0000","last_became_active":"2024-09-11T07:36:05.225840+0000","last_became_peered":"2024-09-11T07:36:05.225840+0000","last_unstale":"2024-09-11T07:37:43.320761+0000","last_undegraded":"2024-09-11T07:37:43.320761+0000","last_fullsized":"2024-09-11T07:37:43.320761+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:55.163476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:36:55.163476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0082978570000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":250,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.321122+0000","last_change":"2024-09-11T07:36:54.828435+0000","last_active":"2024-09-11T07:37:43.321122+0000","last_peered":"2024-09-11T07:37:43.321122+0000","last_clean":"2024-09-11T07:37:43.321122+0000","last_became_active":"2024-09-11T07:26:37.861279+0000","last_became_peered":"2024-09-11T07:26:37.861279+0000","last_unstale":"2024-09-11T07:37:43.321122+0000","last_undegraded":"2024-09-11T07:37:43.321122+0000","last_fullsized":"2024-09-11T07:37:43.321122+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:54.827454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:39.723598+0000","last_clean_scrub_stamp":"2024-09-11T07:36:54.827454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:06.664883+0000","scrub_duration":0.016642015,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.f","version":"256'3839","reported_seq":3855,"reported_epoch":256,"state":"active+clean","last_fresh":"2024-09-11T07:38:09.134194+0000","last_change":"2024-09-11T07:37:25.856504+0000","last_active":"2024-09-11T07:38:09.134194+0000","last_peered":"2024-09-11T07:38:09.134194+0000","last_clean":"2024-09-11T07:38:09.134194+0000","last_became_active":"2024-09-11T07:37:25.856214+0000","last_became_peered":"2024-09-11T07:37:25.856214+0000","last_unstale":"2024-09-11T07:38:09.134194+0000","last_undegraded":"2024-09-11T07:38:09.134194+0000","last_fullsized":"2024-09-11T07:38:09.134194+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":3839,"log_dups_size":0,"ondisk_log_size":3839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:48.819382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":251592704,"num_objects":3839,"num_object_clones":0,"num_object_copies":11517,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3839,"num_write_kb":245696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"234'20","reported_seq":287,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.321618+0000","last_change":"2024-09-11T07:36:37.911537+0000","last_active":"2024-09-11T07:37:43.321618+0000","last_peered":"2024-09-11T07:37:43.321618+0000","last_clean":"2024-09-11T07:37:43.321618+0000","last_became_active":"2024-09-11T07:26:33.543063+0000","last_became_peered":"2024-09-11T07:26:33.543063+0000","last_unstale":"2024-09-11T07:37:43.321618+0000","last_undegraded":"2024-09-11T07:37:43.321618+0000","last_fullsized":"2024-09-11T07:37:43.321618+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:36:37.911329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:36:37.911329+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01198222,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":283,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.322196+0000","last_change":"2024-09-11T07:36:38.909472+0000","last_active":"2024-09-11T07:37:43.322196+0000","last_peered":"2024-09-11T07:37:43.322196+0000","last_clean":"2024-09-11T07:37:43.322196+0000","last_became_active":"2024-09-11T07:32:00.522456+0000","last_became_peered":"2024-09-11T07:32:00.522456+0000","last_unstale":"2024-09-11T07:37:43.322196+0000","last_undegraded":"2024-09-11T07:37:43.322196+0000","last_fullsized":"2024-09-11T07:37:43.322196+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:36:38.909348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:38.909348+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031018040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":411,"reported_epoch":256,"state":"active+clean","last_fresh":"2024-09-11T07:38:04.644236+0000","last_change":"2024-09-11T07:37:26.141508+0000","last_active":"2024-09-11T07:38:04.644236+0000","last_peered":"2024-09-11T07:38:04.644236+0000","last_clean":"2024-09-11T07:38:04.644236+0000","last_became_active":"2024-09-11T07:26:35.560422+0000","last_became_peered":"2024-09-11T07:26:35.560422+0000","last_unstale":"2024-09-11T07:38:04.644236+0000","last_undegraded":"2024-09-11T07:38:04.644236+0000","last_fullsized":"2024-09-11T07:38:04.644236+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:37:26.141447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.141447+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:28.626906+0000","scrub_duration":0.0015408170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":261,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.320958+0000","last_change":"2024-09-11T07:36:40.027067+0000","last_active":"2024-09-11T07:37:43.320958+0000","last_peered":"2024-09-11T07:37:43.320958+0000","last_clean":"2024-09-11T07:37:43.320958+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:37:43.320958+0000","last_undegraded":"2024-09-11T07:37:43.320958+0000","last_fullsized":"2024-09-11T07:37:43.320958+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:40.026982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_clean_scrub_stamp":"2024-09-11T07:36:40.026982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090395279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.9","version":"256'4102","reported_seq":4119,"reported_epoch":256,"state":"active+clean","last_fresh":"2024-09-11T07:38:09.901404+0000","last_change":"2024-09-11T07:37:25.854831+0000","last_active":"2024-09-11T07:38:09.901404+0000","last_peered":"2024-09-11T07:38:09.901404+0000","last_clean":"2024-09-11T07:38:09.901404+0000","last_became_active":"2024-09-11T07:37:25.854472+0000","last_became_peered":"2024-09-11T07:37:25.854472+0000","last_unstale":"2024-09-11T07:38:09.901404+0000","last_undegraded":"2024-09-11T07:38:09.901404+0000","last_fullsized":"2024-09-11T07:38:09.901404+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4102,"log_dups_size":0,"ondisk_log_size":4102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:54.556048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268828672,"num_objects":4102,"num_object_clones":0,"num_object_copies":12306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4102,"num_write_kb":262528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"234'41","reported_seq":216,"reported_epoch":256,"state":"active+clean","last_fresh":"2024-09-11T07:38:04.285712+0000","last_change":"2024-09-11T07:36:41.343699+0000","last_active":"2024-09-11T07:38:04.285712+0000","last_peered":"2024-09-11T07:38:04.285712+0000","last_clean":"2024-09-11T07:38:04.285712+0000","last_became_active":"2024-09-11T07:33:10.017961+0000","last_became_peered":"2024-09-11T07:33:10.017961+0000","last_unstale":"2024-09-11T07:38:04.285712+0000","last_undegraded":"2024-09-11T07:38:04.285712+0000","last_fullsized":"2024-09-11T07:38:04.285712+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'41","last_scrub_stamp":"2024-09-11T07:36:41.343611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:36:41.343611+0000","objects_scrubbed":10,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043492390000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":267,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311467+0000","last_change":"2024-09-11T07:36:51.308567+0000","last_active":"2024-09-11T07:37:43.311467+0000","last_peered":"2024-09-11T07:37:43.311467+0000","last_clean":"2024-09-11T07:37:43.311467+0000","last_became_active":"2024-09-11T07:26:31.424618+0000","last_became_peered":"2024-09-11T07:26:31.424618+0000","last_unstale":"2024-09-11T07:37:43.311467+0000","last_undegraded":"2024-09-11T07:37:43.311467+0000","last_fullsized":"2024-09-11T07:37:43.311467+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:51.308492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:51.308492+0000","last_clean_scrub_stamp":"2024-09-11T07:36:51.308492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057563680000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":263,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.326098+0000","last_change":"2024-09-11T07:37:33.144930+0000","last_active":"2024-09-11T07:37:43.326098+0000","last_peered":"2024-09-11T07:37:43.326098+0000","last_clean":"2024-09-11T07:37:43.326098+0000","last_became_active":"2024-09-11T07:26:35.560363+0000","last_became_peered":"2024-09-11T07:26:35.560363+0000","last_unstale":"2024-09-11T07:37:43.326098+0000","last_undegraded":"2024-09-11T07:37:43.326098+0000","last_fullsized":"2024-09-11T07:37:43.326098+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:33.144873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:33.144873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:42.928193+0000","scrub_duration":0.001754675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":268,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.308990+0000","last_change":"2024-09-11T07:37:39.053981+0000","last_active":"2024-09-11T07:37:43.308990+0000","last_peered":"2024-09-11T07:37:43.308990+0000","last_clean":"2024-09-11T07:37:43.308990+0000","last_became_active":"2024-09-11T07:32:00.505919+0000","last_became_peered":"2024-09-11T07:32:00.505919+0000","last_unstale":"2024-09-11T07:37:43.308990+0000","last_undegraded":"2024-09-11T07:37:43.308990+0000","last_fullsized":"2024-09-11T07:37:43.308990+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:39.053900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:39.053900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:49.165548+0000","scrub_duration":0.0050142590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"256'4051","reported_seq":4067,"reported_epoch":256,"state":"active+clean","last_fresh":"2024-09-11T07:38:09.118355+0000","last_change":"2024-09-11T07:37:25.855294+0000","last_active":"2024-09-11T07:38:09.118355+0000","last_peered":"2024-09-11T07:38:09.118355+0000","last_clean":"2024-09-11T07:38:09.118355+0000","last_became_active":"2024-09-11T07:37:25.855005+0000","last_became_peered":"2024-09-11T07:37:25.855005+0000","last_unstale":"2024-09-11T07:38:09.118355+0000","last_undegraded":"2024-09-11T07:38:09.118355+0000","last_fullsized":"2024-09-11T07:38:09.118355+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4051,"log_dups_size":0,"ondisk_log_size":4051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:26.698988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":265486336,"num_objects":4051,"num_object_clones":0,"num_object_copies":12153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4051,"num_write_kb":259264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"234'22","reported_seq":292,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.321623+0000","last_change":"2024-09-11T07:37:20.965215+0000","last_active":"2024-09-11T07:37:43.321623+0000","last_peered":"2024-09-11T07:37:43.321623+0000","last_clean":"2024-09-11T07:37:43.321623+0000","last_became_active":"2024-09-11T07:26:33.546126+0000","last_became_peered":"2024-09-11T07:26:33.546126+0000","last_unstale":"2024-09-11T07:37:43.321623+0000","last_undegraded":"2024-09-11T07:37:43.321623+0000","last_fullsized":"2024-09-11T07:37:43.321623+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'22","last_scrub_stamp":"2024-09-11T07:37:20.965147+0000","last_deep_scrub":"144'18","last_deep_scrub_stamp":"2024-09-11T07:31:51.603378+0000","last_clean_scrub_stamp":"2024-09-11T07:37:20.965147+0000","objects_scrubbed":5,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:26.343736+0000","scrub_duration":0.0019365999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":964,"reported_epoch":256,"state":"active+clean","last_fresh":"2024-09-11T07:38:08.160369+0000","last_change":"2024-09-11T07:37:22.990093+0000","last_active":"2024-09-11T07:38:08.160369+0000","last_peered":"2024-09-11T07:38:08.160369+0000","last_clean":"2024-09-11T07:38:08.160369+0000","last_became_active":"2024-09-11T07:26:31.421276+0000","last_became_peered":"2024-09-11T07:26:31.421276+0000","last_unstale":"2024-09-11T07:38:08.160369+0000","last_undegraded":"2024-09-11T07:38:08.160369+0000","last_fullsized":"2024-09-11T07:38:08.160369+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'5","last_scrub_stamp":"2024-09-11T07:37:22.990033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:22.990033+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:51.516885+0000","scrub_duration":0.001646954,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":690,"num_read_kb":685,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":277,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.310989+0000","last_change":"2024-09-11T07:36:45.059852+0000","last_active":"2024-09-11T07:37:43.310989+0000","last_peered":"2024-09-11T07:37:43.310989+0000","last_clean":"2024-09-11T07:37:43.310989+0000","last_became_active":"2024-09-11T07:26:31.413354+0000","last_became_peered":"2024-09-11T07:26:31.413354+0000","last_unstale":"2024-09-11T07:37:43.310989+0000","last_undegraded":"2024-09-11T07:37:43.310989+0000","last_fullsized":"2024-09-11T07:37:43.310989+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:45.059537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:07.149367+0000","last_clean_scrub_stamp":"2024-09-11T07:36:45.059537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:06.540727+0000","scrub_duration":0.013492418000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":270,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311701+0000","last_change":"2024-09-11T07:36:40.189921+0000","last_active":"2024-09-11T07:37:43.311701+0000","last_peered":"2024-09-11T07:37:43.311701+0000","last_clean":"2024-09-11T07:37:43.311701+0000","last_became_active":"2024-09-11T07:32:00.508398+0000","last_became_peered":"2024-09-11T07:32:00.508398+0000","last_unstale":"2024-09-11T07:37:43.311701+0000","last_undegraded":"2024-09-11T07:37:43.311701+0000","last_fullsized":"2024-09-11T07:37:43.311701+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:40.189372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:36:40.189372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077070330000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":251,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.315172+0000","last_change":"2024-09-11T07:36:56.490992+0000","last_active":"2024-09-11T07:37:43.315172+0000","last_peered":"2024-09-11T07:37:43.315172+0000","last_clean":"2024-09-11T07:37:43.315172+0000","last_became_active":"2024-09-11T07:32:00.515745+0000","last_became_peered":"2024-09-11T07:32:00.515745+0000","last_unstale":"2024-09-11T07:37:43.315172+0000","last_undegraded":"2024-09-11T07:37:43.315172+0000","last_fullsized":"2024-09-11T07:37:43.315172+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:56.490940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:56.490940+0000","last_clean_scrub_stamp":"2024-09-11T07:36:56.490940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.037860923999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"256'4134","reported_seq":4151,"reported_epoch":256,"state":"active+clean","last_fresh":"2024-09-11T07:38:09.930151+0000","last_change":"2024-09-11T07:37:26.029935+0000","last_active":"2024-09-11T07:38:09.930151+0000","last_peered":"2024-09-11T07:38:09.930151+0000","last_clean":"2024-09-11T07:38:09.930151+0000","last_became_active":"2024-09-11T07:37:26.029608+0000","last_became_peered":"2024-09-11T07:37:26.029608+0000","last_unstale":"2024-09-11T07:38:09.930151+0000","last_undegraded":"2024-09-11T07:38:09.930151+0000","last_fullsized":"2024-09-11T07:38:09.930151+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4134,"log_dups_size":0,"ondisk_log_size":4134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:38.232917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270925824,"num_objects":4134,"num_object_clones":0,"num_object_copies":12402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4134,"num_write_kb":264576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"234'30","reported_seq":331,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.324755+0000","last_change":"2024-09-11T07:36:45.088277+0000","last_active":"2024-09-11T07:37:43.324755+0000","last_peered":"2024-09-11T07:37:43.324755+0000","last_clean":"2024-09-11T07:37:43.324755+0000","last_became_active":"2024-09-11T07:28:09.018134+0000","last_became_peered":"2024-09-11T07:28:09.018134+0000","last_unstale":"2024-09-11T07:37:43.324755+0000","last_undegraded":"2024-09-11T07:37:43.324755+0000","last_fullsized":"2024-09-11T07:37:43.324755+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'30","last_scrub_stamp":"2024-09-11T07:36:45.087995+0000","last_deep_scrub":"94'18","last_deep_scrub_stamp":"2024-09-11T07:28:51.437397+0000","last_clean_scrub_stamp":"2024-09-11T07:36:45.087995+0000","objects_scrubbed":6,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.027742936999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":263,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.321547+0000","last_change":"2024-09-11T07:35:53.257684+0000","last_active":"2024-09-11T07:37:43.321547+0000","last_peered":"2024-09-11T07:37:43.321547+0000","last_clean":"2024-09-11T07:37:43.321547+0000","last_became_active":"2024-09-11T07:26:31.420320+0000","last_became_peered":"2024-09-11T07:26:31.420320+0000","last_unstale":"2024-09-11T07:37:43.321547+0000","last_undegraded":"2024-09-11T07:37:43.321547+0000","last_fullsized":"2024-09-11T07:37:43.321547+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-11T07:35:53.257474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:35:53.257474+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053710479999999998,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":262,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.313961+0000","last_change":"2024-09-11T07:37:26.557037+0000","last_active":"2024-09-11T07:37:43.313961+0000","last_peered":"2024-09-11T07:37:43.313961+0000","last_clean":"2024-09-11T07:37:43.313961+0000","last_became_active":"2024-09-11T07:32:00.520035+0000","last_became_peered":"2024-09-11T07:32:00.520035+0000","last_unstale":"2024-09-11T07:37:43.313961+0000","last_undegraded":"2024-09-11T07:37:43.313961+0000","last_fullsized":"2024-09-11T07:37:43.313961+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:26.556972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:16.967492+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.556972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:43.963183+0000","scrub_duration":0.001503206,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":243,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311948+0000","last_change":"2024-09-11T07:35:51.187203+0000","last_active":"2024-09-11T07:37:43.311948+0000","last_peered":"2024-09-11T07:37:43.311948+0000","last_clean":"2024-09-11T07:37:43.311948+0000","last_became_active":"2024-09-11T07:26:35.564178+0000","last_became_peered":"2024-09-11T07:26:35.564178+0000","last_unstale":"2024-09-11T07:37:43.311948+0000","last_undegraded":"2024-09-11T07:37:43.311948+0000","last_fullsized":"2024-09-11T07:37:43.311948+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:51.187127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:51.187127+0000","last_clean_scrub_stamp":"2024-09-11T07:35:51.187127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0055970839999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":253,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.321989+0000","last_change":"2024-09-11T07:35:35.789115+0000","last_active":"2024-09-11T07:37:43.321989+0000","last_peered":"2024-09-11T07:37:43.321989+0000","last_clean":"2024-09-11T07:37:43.321989+0000","last_became_active":"2024-09-11T07:26:37.561378+0000","last_became_peered":"2024-09-11T07:26:37.561378+0000","last_unstale":"2024-09-11T07:37:43.321989+0000","last_undegraded":"2024-09-11T07:37:43.321989+0000","last_fullsized":"2024-09-11T07:37:43.321989+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:35.789024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:35:35.789024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079175110000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"256'3988","reported_seq":4003,"reported_epoch":256,"state":"active+clean","last_fresh":"2024-09-11T07:38:09.838600+0000","last_change":"2024-09-11T07:37:25.854273+0000","last_active":"2024-09-11T07:38:09.838600+0000","last_peered":"2024-09-11T07:38:09.838600+0000","last_clean":"2024-09-11T07:38:09.838600+0000","last_became_active":"2024-09-11T07:37:25.853947+0000","last_became_peered":"2024-09-11T07:37:25.853947+0000","last_unstale":"2024-09-11T07:38:09.838600+0000","last_undegraded":"2024-09-11T07:38:09.838600+0000","last_fullsized":"2024-09-11T07:38:09.838600+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":3988,"log_dups_size":0,"ondisk_log_size":3988,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:38.389157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":261357568,"num_objects":3988,"num_object_clones":0,"num_object_copies":11964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3988,"num_write_kb":255232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"234'26","reported_seq":106,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.307942+0000","last_change":"2024-09-11T07:37:40.649065+0000","last_active":"2024-09-11T07:37:43.307942+0000","last_peered":"2024-09-11T07:37:43.307942+0000","last_clean":"2024-09-11T07:37:43.307942+0000","last_became_active":"2024-09-11T07:36:05.224704+0000","last_became_peered":"2024-09-11T07:36:05.224704+0000","last_unstale":"2024-09-11T07:37:43.307942+0000","last_undegraded":"2024-09-11T07:37:43.307942+0000","last_fullsized":"2024-09-11T07:37:43.307942+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'26","last_scrub_stamp":"2024-09-11T07:37:40.648986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:40.648986+0000","objects_scrubbed":7,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:09.297479+0000","scrub_duration":0.0030578620000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'3","reported_seq":265,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.310157+0000","last_change":"2024-09-11T07:36:54.937298+0000","last_active":"2024-09-11T07:37:43.310157+0000","last_peered":"2024-09-11T07:37:43.310157+0000","last_clean":"2024-09-11T07:37:43.310157+0000","last_became_active":"2024-09-11T07:26:31.419501+0000","last_became_peered":"2024-09-11T07:26:31.419501+0000","last_unstale":"2024-09-11T07:37:43.310157+0000","last_undegraded":"2024-09-11T07:37:43.310157+0000","last_fullsized":"2024-09-11T07:37:43.310157+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-09-11T07:36:54.937201+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_clean_scrub_stamp":"2024-09-11T07:36:54.937201+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:15.893124+0000","scrub_duration":0.007045239,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":277,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311556+0000","last_change":"2024-09-11T07:37:35.504299+0000","last_active":"2024-09-11T07:37:43.311556+0000","last_peered":"2024-09-11T07:37:43.311556+0000","last_clean":"2024-09-11T07:37:43.311556+0000","last_became_active":"2024-09-11T07:32:00.498002+0000","last_became_peered":"2024-09-11T07:32:00.498002+0000","last_unstale":"2024-09-11T07:37:43.311556+0000","last_undegraded":"2024-09-11T07:37:43.311556+0000","last_fullsized":"2024-09-11T07:37:43.311556+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:35.504238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:35.504238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:46.132916+0000","scrub_duration":0.0013066169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"87'81","reported_seq":278,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.313886+0000","last_change":"2024-09-11T07:36:45.398752+0000","last_active":"2024-09-11T07:37:43.313886+0000","last_peered":"2024-09-11T07:37:43.313886+0000","last_clean":"2024-09-11T07:37:43.313886+0000","last_became_active":"2024-09-11T07:32:00.519656+0000","last_became_peered":"2024-09-11T07:32:00.519656+0000","last_unstale":"2024-09-11T07:37:43.313886+0000","last_undegraded":"2024-09-11T07:37:43.313886+0000","last_fullsized":"2024-09-11T07:37:43.313886+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:36:45.398664+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:36:45.398664+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041388420000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":248,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.314753+0000","last_change":"2024-09-11T07:37:41.562374+0000","last_active":"2024-09-11T07:37:43.314753+0000","last_peered":"2024-09-11T07:37:43.314753+0000","last_clean":"2024-09-11T07:37:43.314753+0000","last_became_active":"2024-09-11T07:32:00.510888+0000","last_became_peered":"2024-09-11T07:32:00.510888+0000","last_unstale":"2024-09-11T07:37:43.314753+0000","last_undegraded":"2024-09-11T07:37:43.314753+0000","last_fullsized":"2024-09-11T07:37:43.314753+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:41.562322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:41.562322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:07.245074+0000","scrub_duration":0.003022734,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":251,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311990+0000","last_change":"2024-09-11T07:37:23.436064+0000","last_active":"2024-09-11T07:37:43.311990+0000","last_peered":"2024-09-11T07:37:43.311990+0000","last_clean":"2024-09-11T07:37:43.311990+0000","last_became_active":"2024-09-11T07:26:37.559920+0000","last_became_peered":"2024-09-11T07:26:37.559920+0000","last_unstale":"2024-09-11T07:37:43.311990+0000","last_undegraded":"2024-09-11T07:37:43.311990+0000","last_fullsized":"2024-09-11T07:37:43.311990+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:23.436012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:23.436012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:25.352176+0000","scrub_duration":0.001213481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.7","version":"256'3984","reported_seq":4000,"reported_epoch":256,"state":"active+clean","last_fresh":"2024-09-11T07:38:09.127591+0000","last_change":"2024-09-11T07:37:25.855953+0000","last_active":"2024-09-11T07:38:09.127591+0000","last_peered":"2024-09-11T07:38:09.127591+0000","last_clean":"2024-09-11T07:38:09.127591+0000","last_became_active":"2024-09-11T07:37:25.855615+0000","last_became_peered":"2024-09-11T07:37:25.855615+0000","last_unstale":"2024-09-11T07:38:09.127591+0000","last_undegraded":"2024-09-11T07:38:09.127591+0000","last_fullsized":"2024-09-11T07:38:09.127591+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":3984,"log_dups_size":0,"ondisk_log_size":3984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:25.900228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":261095424,"num_objects":3984,"num_object_clones":0,"num_object_copies":11952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3984,"num_write_kb":254976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"234'35","reported_seq":111,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.321466+0000","last_change":"2024-09-11T07:36:05.226411+0000","last_active":"2024-09-11T07:37:43.321466+0000","last_peered":"2024-09-11T07:37:43.321466+0000","last_clean":"2024-09-11T07:37:43.321466+0000","last_became_active":"2024-09-11T07:36:05.226021+0000","last_became_peered":"2024-09-11T07:36:05.226021+0000","last_unstale":"2024-09-11T07:37:43.321466+0000","last_undegraded":"2024-09-11T07:37:43.321466+0000","last_fullsized":"2024-09-11T07:37:43.321466+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-11T07:35:29.151274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:35:29.151274+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":267,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311493+0000","last_change":"2024-09-11T07:36:48.089096+0000","last_active":"2024-09-11T07:37:43.311493+0000","last_peered":"2024-09-11T07:37:43.311493+0000","last_clean":"2024-09-11T07:37:43.311493+0000","last_became_active":"2024-09-11T07:26:31.419560+0000","last_became_peered":"2024-09-11T07:26:31.419560+0000","last_unstale":"2024-09-11T07:37:43.311493+0000","last_undegraded":"2024-09-11T07:37:43.311493+0000","last_fullsized":"2024-09-11T07:37:43.311493+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:48.088795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:48.088795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:06.111523+0000","scrub_duration":0.0081450789999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":263,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311753+0000","last_change":"2024-09-11T07:37:17.420289+0000","last_active":"2024-09-11T07:37:43.311753+0000","last_peered":"2024-09-11T07:37:43.311753+0000","last_clean":"2024-09-11T07:37:43.311753+0000","last_became_active":"2024-09-11T07:26:35.564044+0000","last_became_peered":"2024-09-11T07:26:35.564044+0000","last_unstale":"2024-09-11T07:37:43.311753+0000","last_undegraded":"2024-09-11T07:37:43.311753+0000","last_fullsized":"2024-09-11T07:37:43.311753+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:17.420236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:17.420236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:47.156712+0000","scrub_duration":0.0012713080000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":246,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.313800+0000","last_change":"2024-09-11T07:36:31.411708+0000","last_active":"2024-09-11T07:37:43.313800+0000","last_peered":"2024-09-11T07:37:43.313800+0000","last_clean":"2024-09-11T07:37:43.313800+0000","last_became_active":"2024-09-11T07:32:00.516916+0000","last_became_peered":"2024-09-11T07:32:00.516916+0000","last_unstale":"2024-09-11T07:37:43.313800+0000","last_undegraded":"2024-09-11T07:37:43.313800+0000","last_fullsized":"2024-09-11T07:37:43.313800+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:31.411554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:31.411554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002948132,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"256'4093","reported_seq":4109,"reported_epoch":256,"state":"active+clean","last_fresh":"2024-09-11T07:38:09.103397+0000","last_change":"2024-09-11T07:37:25.854298+0000","last_active":"2024-09-11T07:38:09.103397+0000","last_peered":"2024-09-11T07:38:09.103397+0000","last_clean":"2024-09-11T07:38:09.103397+0000","last_became_active":"2024-09-11T07:37:25.854037+0000","last_became_peered":"2024-09-11T07:37:25.854037+0000","last_unstale":"2024-09-11T07:38:09.103397+0000","last_undegraded":"2024-09-11T07:38:09.103397+0000","last_fullsized":"2024-09-11T07:38:09.103397+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4093,"log_dups_size":0,"ondisk_log_size":4093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:45.294877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268238848,"num_objects":4093,"num_object_clones":0,"num_object_copies":12279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4093,"num_write_kb":261952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"234'15","reported_seq":280,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.324515+0000","last_change":"2024-09-11T07:36:46.033480+0000","last_active":"2024-09-11T07:37:43.324515+0000","last_peered":"2024-09-11T07:37:43.324515+0000","last_clean":"2024-09-11T07:37:43.324515+0000","last_became_active":"2024-09-11T07:26:33.543148+0000","last_became_peered":"2024-09-11T07:26:33.543148+0000","last_unstale":"2024-09-11T07:37:43.324515+0000","last_undegraded":"2024-09-11T07:37:43.324515+0000","last_fullsized":"2024-09-11T07:37:43.324515+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:36:46.033388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:36:46.033388+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0084720030000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":266,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.322264+0000","last_change":"2024-09-11T07:37:26.019680+0000","last_active":"2024-09-11T07:37:43.322264+0000","last_peered":"2024-09-11T07:37:43.322264+0000","last_clean":"2024-09-11T07:37:43.322264+0000","last_became_active":"2024-09-11T07:26:31.423056+0000","last_became_peered":"2024-09-11T07:26:31.423056+0000","last_unstale":"2024-09-11T07:37:43.322264+0000","last_undegraded":"2024-09-11T07:37:43.322264+0000","last_fullsized":"2024-09-11T07:37:43.322264+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:26.019623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.019623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:43.097013+0000","scrub_duration":0.0013393599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":263,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311606+0000","last_change":"2024-09-11T07:36:39.141707+0000","last_active":"2024-09-11T07:37:43.311606+0000","last_peered":"2024-09-11T07:37:43.311606+0000","last_clean":"2024-09-11T07:37:43.311606+0000","last_became_active":"2024-09-11T07:26:35.563182+0000","last_became_peered":"2024-09-11T07:26:35.563182+0000","last_unstale":"2024-09-11T07:37:43.311606+0000","last_undegraded":"2024-09-11T07:37:43.311606+0000","last_fullsized":"2024-09-11T07:37:43.311606+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:39.141620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:36:39.141620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01162813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":268,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.321473+0000","last_change":"2024-09-11T07:36:36.023079+0000","last_active":"2024-09-11T07:37:43.321473+0000","last_peered":"2024-09-11T07:37:43.321473+0000","last_clean":"2024-09-11T07:37:43.321473+0000","last_became_active":"2024-09-11T07:32:00.505185+0000","last_became_peered":"2024-09-11T07:32:00.505185+0000","last_unstale":"2024-09-11T07:37:43.321473+0000","last_undegraded":"2024-09-11T07:37:43.321473+0000","last_fullsized":"2024-09-11T07:37:43.321473+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:36.023006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:46.944763+0000","last_clean_scrub_stamp":"2024-09-11T07:36:36.023006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011934019000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.5","version":"256'4051","reported_seq":4067,"reported_epoch":256,"state":"active+clean","last_fresh":"2024-09-11T07:38:09.930703+0000","last_change":"2024-09-11T07:37:25.854781+0000","last_active":"2024-09-11T07:38:09.930703+0000","last_peered":"2024-09-11T07:38:09.930703+0000","last_clean":"2024-09-11T07:38:09.930703+0000","last_became_active":"2024-09-11T07:37:25.854037+0000","last_became_peered":"2024-09-11T07:37:25.854037+0000","last_unstale":"2024-09-11T07:38:09.930703+0000","last_undegraded":"2024-09-11T07:38:09.930703+0000","last_fullsized":"2024-09-11T07:38:09.930703+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4051,"log_dups_size":0,"ondisk_log_size":4051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:46.537669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":265486336,"num_objects":4051,"num_object_clones":0,"num_object_copies":12153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4051,"num_write_kb":259264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"234'27","reported_seq":308,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311275+0000","last_change":"2024-09-11T07:36:32.341522+0000","last_active":"2024-09-11T07:37:43.311275+0000","last_peered":"2024-09-11T07:37:43.311275+0000","last_clean":"2024-09-11T07:37:43.311275+0000","last_became_active":"2024-09-11T07:32:00.521464+0000","last_became_peered":"2024-09-11T07:32:00.521464+0000","last_unstale":"2024-09-11T07:37:43.311275+0000","last_undegraded":"2024-09-11T07:37:43.311275+0000","last_fullsized":"2024-09-11T07:37:43.311275+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-09-11T07:36:32.341462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:36:32.341462+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043984430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":284,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311941+0000","last_change":"2024-09-11T07:36:37.142765+0000","last_active":"2024-09-11T07:37:43.311941+0000","last_peered":"2024-09-11T07:37:43.311941+0000","last_clean":"2024-09-11T07:37:43.311941+0000","last_became_active":"2024-09-11T07:32:00.513949+0000","last_became_peered":"2024-09-11T07:32:00.513949+0000","last_unstale":"2024-09-11T07:37:43.311941+0000","last_undegraded":"2024-09-11T07:37:43.311941+0000","last_fullsized":"2024-09-11T07:37:43.311941+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:37.142658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_clean_scrub_stamp":"2024-09-11T07:36:37.142658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011370480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"151'10","reported_seq":361,"reported_epoch":256,"state":"active+clean","last_fresh":"2024-09-11T07:38:04.287690+0000","last_change":"2024-09-11T07:37:38.544127+0000","last_active":"2024-09-11T07:38:04.287690+0000","last_peered":"2024-09-11T07:38:04.287690+0000","last_clean":"2024-09-11T07:38:04.287690+0000","last_became_active":"2024-09-11T07:32:00.506806+0000","last_became_peered":"2024-09-11T07:32:00.506806+0000","last_unstale":"2024-09-11T07:38:04.287690+0000","last_undegraded":"2024-09-11T07:38:04.287690+0000","last_fullsized":"2024-09-11T07:38:04.287690+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:37:38.544072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:38.544072+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:47.377252+0000","scrub_duration":0.0027401579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":251,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311281+0000","last_change":"2024-09-11T07:36:44.295056+0000","last_active":"2024-09-11T07:37:43.311281+0000","last_peered":"2024-09-11T07:37:43.311281+0000","last_clean":"2024-09-11T07:37:43.311281+0000","last_became_active":"2024-09-11T07:26:37.861134+0000","last_became_peered":"2024-09-11T07:26:37.861134+0000","last_unstale":"2024-09-11T07:37:43.311281+0000","last_undegraded":"2024-09-11T07:37:43.311281+0000","last_fullsized":"2024-09-11T07:37:43.311281+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:44.293653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:44.293653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0074032500000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.4","version":"256'4011","reported_seq":4027,"reported_epoch":256,"state":"active+clean","last_fresh":"2024-09-11T07:38:09.427868+0000","last_change":"2024-09-11T07:37:25.852069+0000","last_active":"2024-09-11T07:38:09.427868+0000","last_peered":"2024-09-11T07:38:09.427868+0000","last_clean":"2024-09-11T07:38:09.427868+0000","last_became_active":"2024-09-11T07:37:25.851789+0000","last_became_peered":"2024-09-11T07:37:25.851789+0000","last_unstale":"2024-09-11T07:38:09.427868+0000","last_undegraded":"2024-09-11T07:38:09.427868+0000","last_fullsized":"2024-09-11T07:38:09.427868+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4011,"log_dups_size":0,"ondisk_log_size":4011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:40.599431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262864896,"num_objects":4011,"num_object_clones":0,"num_object_copies":12033,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4011,"num_write_kb":256704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"234'23","reported_seq":292,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.307220+0000","last_change":"2024-09-11T07:37:34.613363+0000","last_active":"2024-09-11T07:37:43.307220+0000","last_peered":"2024-09-11T07:37:43.307220+0000","last_clean":"2024-09-11T07:37:43.307220+0000","last_became_active":"2024-09-11T07:26:33.540363+0000","last_became_peered":"2024-09-11T07:26:33.540363+0000","last_unstale":"2024-09-11T07:37:43.307220+0000","last_undegraded":"2024-09-11T07:37:43.307220+0000","last_fullsized":"2024-09-11T07:37:43.307220+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'23","last_scrub_stamp":"2024-09-11T07:37:34.613280+0000","last_deep_scrub":"234'23","last_deep_scrub_stamp":"2024-09-11T07:37:34.613280+0000","last_clean_scrub_stamp":"2024-09-11T07:37:34.613280+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:45.821230+0000","scrub_duration":0.002722604,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":284,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.309079+0000","last_change":"2024-09-11T07:35:46.769144+0000","last_active":"2024-09-11T07:37:43.309079+0000","last_peered":"2024-09-11T07:37:43.309079+0000","last_clean":"2024-09-11T07:37:43.309079+0000","last_became_active":"2024-09-11T07:32:00.504344+0000","last_became_peered":"2024-09-11T07:32:00.504344+0000","last_unstale":"2024-09-11T07:37:43.309079+0000","last_undegraded":"2024-09-11T07:37:43.309079+0000","last_fullsized":"2024-09-11T07:37:43.309079+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:35:46.769070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:35:46.769070+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049213720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":263,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311268+0000","last_change":"2024-09-11T07:36:43.125248+0000","last_active":"2024-09-11T07:37:43.311268+0000","last_peered":"2024-09-11T07:37:43.311268+0000","last_clean":"2024-09-11T07:37:43.311268+0000","last_became_active":"2024-09-11T07:26:35.561967+0000","last_became_peered":"2024-09-11T07:26:35.561967+0000","last_unstale":"2024-09-11T07:37:43.311268+0000","last_undegraded":"2024-09-11T07:37:43.311268+0000","last_fullsized":"2024-09-11T07:37:43.311268+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:43.125187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:44.820354+0000","last_clean_scrub_stamp":"2024-09-11T07:36:43.125187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061424100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":268,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.308197+0000","last_change":"2024-09-11T07:37:16.860986+0000","last_active":"2024-09-11T07:37:43.308197+0000","last_peered":"2024-09-11T07:37:43.308197+0000","last_clean":"2024-09-11T07:37:43.308197+0000","last_became_active":"2024-09-11T07:32:00.502019+0000","last_became_peered":"2024-09-11T07:32:00.502019+0000","last_unstale":"2024-09-11T07:37:43.308197+0000","last_undegraded":"2024-09-11T07:37:43.308197+0000","last_fullsized":"2024-09-11T07:37:43.308197+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:16.860930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:16.860930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:35.856490+0000","scrub_duration":0.0024991000000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"234'15","reported_seq":297,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.324659+0000","last_change":"2024-09-11T07:37:19.096890+0000","last_active":"2024-09-11T07:37:43.324659+0000","last_peered":"2024-09-11T07:37:43.324659+0000","last_clean":"2024-09-11T07:37:43.324659+0000","last_became_active":"2024-09-11T07:32:00.509412+0000","last_became_peered":"2024-09-11T07:32:00.509412+0000","last_unstale":"2024-09-11T07:37:43.324659+0000","last_undegraded":"2024-09-11T07:37:43.324659+0000","last_fullsized":"2024-09-11T07:37:43.324659+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:37:19.096817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:19.096817+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:38.083588+0000","scrub_duration":0.00175015,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":267,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311408+0000","last_change":"2024-09-11T07:36:53.065329+0000","last_active":"2024-09-11T07:37:43.311408+0000","last_peered":"2024-09-11T07:37:43.311408+0000","last_clean":"2024-09-11T07:37:43.311408+0000","last_became_active":"2024-09-11T07:26:31.415701+0000","last_became_peered":"2024-09-11T07:26:31.415701+0000","last_unstale":"2024-09-11T07:37:43.311408+0000","last_undegraded":"2024-09-11T07:37:43.311408+0000","last_fullsized":"2024-09-11T07:37:43.311408+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:53.065247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.925200+0000","last_clean_scrub_stamp":"2024-09-11T07:36:53.065247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023028290000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":260,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.312974+0000","last_change":"2024-09-11T07:36:32.708578+0000","last_active":"2024-09-11T07:37:43.312974+0000","last_peered":"2024-09-11T07:37:43.312974+0000","last_clean":"2024-09-11T07:37:43.312974+0000","last_became_active":"2024-09-11T07:32:00.499598+0000","last_became_peered":"2024-09-11T07:32:00.499598+0000","last_unstale":"2024-09-11T07:37:43.312974+0000","last_undegraded":"2024-09-11T07:37:43.312974+0000","last_fullsized":"2024-09-11T07:37:43.312974+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:32.708500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:35.392360+0000","last_clean_scrub_stamp":"2024-09-11T07:36:32.708500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032820800000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":269,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311008+0000","last_change":"2024-09-11T07:36:35.186527+0000","last_active":"2024-09-11T07:37:43.311008+0000","last_peered":"2024-09-11T07:37:43.311008+0000","last_clean":"2024-09-11T07:37:43.311008+0000","last_became_active":"2024-09-11T07:32:00.506697+0000","last_became_peered":"2024-09-11T07:32:00.506697+0000","last_unstale":"2024-09-11T07:37:43.311008+0000","last_undegraded":"2024-09-11T07:37:43.311008+0000","last_fullsized":"2024-09-11T07:37:43.311008+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:36:35.185293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:35.185293+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039808639999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":54,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.307303+0000","last_change":"2024-09-11T07:36:51.617809+0000","last_active":"2024-09-11T07:37:43.307303+0000","last_peered":"2024-09-11T07:37:43.307303+0000","last_clean":"2024-09-11T07:37:43.307303+0000","last_became_active":"2024-09-11T07:36:05.225791+0000","last_became_peered":"2024-09-11T07:36:05.225791+0000","last_unstale":"2024-09-11T07:37:43.307303+0000","last_undegraded":"2024-09-11T07:37:43.307303+0000","last_fullsized":"2024-09-11T07:37:43.307303+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:51.617730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:51.617730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:06.231089+0000","scrub_duration":0.001776197,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"234'15","reported_seq":71,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311777+0000","last_change":"2024-09-11T07:37:26.450184+0000","last_active":"2024-09-11T07:37:43.311777+0000","last_peered":"2024-09-11T07:37:43.311777+0000","last_clean":"2024-09-11T07:37:43.311777+0000","last_became_active":"2024-09-11T07:35:50.100154+0000","last_became_peered":"2024-09-11T07:35:50.100154+0000","last_unstale":"2024-09-11T07:37:43.311777+0000","last_undegraded":"2024-09-11T07:37:43.311777+0000","last_fullsized":"2024-09-11T07:37:43.311777+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:37:26.450122+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.450122+0000","objects_scrubbed":3,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:32.005494+0000","scrub_duration":0.001711052,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":263,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311847+0000","last_change":"2024-09-11T07:37:18.340651+0000","last_active":"2024-09-11T07:37:43.311847+0000","last_peered":"2024-09-11T07:37:43.311847+0000","last_clean":"2024-09-11T07:37:43.311847+0000","last_became_active":"2024-09-11T07:26:35.566894+0000","last_became_peered":"2024-09-11T07:26:35.566894+0000","last_unstale":"2024-09-11T07:37:43.311847+0000","last_undegraded":"2024-09-11T07:37:43.311847+0000","last_fullsized":"2024-09-11T07:37:43.311847+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:18.340578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:37:18.340578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:21.567929+0000","scrub_duration":0.0054750240000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":249,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.315644+0000","last_change":"2024-09-11T07:36:57.461696+0000","last_active":"2024-09-11T07:37:43.315644+0000","last_peered":"2024-09-11T07:37:43.315644+0000","last_clean":"2024-09-11T07:37:43.315644+0000","last_became_active":"2024-09-11T07:32:00.514100+0000","last_became_peered":"2024-09-11T07:32:00.514100+0000","last_unstale":"2024-09-11T07:37:43.315644+0000","last_undegraded":"2024-09-11T07:37:43.315644+0000","last_fullsized":"2024-09-11T07:37:43.315644+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:57.461617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:57.461617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:06.770396+0000","scrub_duration":0.028762672999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"234'25","reported_seq":319,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311131+0000","last_change":"2024-09-11T07:36:55.966810+0000","last_active":"2024-09-11T07:37:43.311131+0000","last_peered":"2024-09-11T07:37:43.311131+0000","last_clean":"2024-09-11T07:37:43.311131+0000","last_became_active":"2024-09-11T07:32:00.504463+0000","last_became_peered":"2024-09-11T07:32:00.504463+0000","last_unstale":"2024-09-11T07:37:43.311131+0000","last_undegraded":"2024-09-11T07:37:43.311131+0000","last_fullsized":"2024-09-11T07:37:43.311131+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'25","last_scrub_stamp":"2024-09-11T07:36:55.966595+0000","last_deep_scrub":"86'15","last_deep_scrub_stamp":"2024-09-11T07:28:22.858137+0000","last_clean_scrub_stamp":"2024-09-11T07:36:55.966595+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:24.512562+0000","scrub_duration":0.032608074000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":267,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311424+0000","last_change":"2024-09-11T07:36:34.405309+0000","last_active":"2024-09-11T07:37:43.311424+0000","last_peered":"2024-09-11T07:37:43.311424+0000","last_clean":"2024-09-11T07:37:43.311424+0000","last_became_active":"2024-09-11T07:32:00.498473+0000","last_became_peered":"2024-09-11T07:32:00.498473+0000","last_unstale":"2024-09-11T07:37:43.311424+0000","last_undegraded":"2024-09-11T07:37:43.311424+0000","last_fullsized":"2024-09-11T07:37:43.311424+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:34.405241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:34.405241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039370459999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":400,"reported_epoch":256,"state":"active+clean","last_fresh":"2024-09-11T07:38:04.286289+0000","last_change":"2024-09-11T07:37:17.360240+0000","last_active":"2024-09-11T07:38:04.286289+0000","last_peered":"2024-09-11T07:38:04.286289+0000","last_clean":"2024-09-11T07:38:04.286289+0000","last_became_active":"2024-09-11T07:26:35.562245+0000","last_became_peered":"2024-09-11T07:26:35.562245+0000","last_unstale":"2024-09-11T07:38:04.286289+0000","last_undegraded":"2024-09-11T07:38:04.286289+0000","last_fullsized":"2024-09-11T07:38:04.286289+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:37:17.360183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:17.360183+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:33.736883+0000","scrub_duration":0.0016828819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":278,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.324407+0000","last_change":"2024-09-11T07:37:42.084410+0000","last_active":"2024-09-11T07:37:43.324407+0000","last_peered":"2024-09-11T07:37:43.324407+0000","last_clean":"2024-09-11T07:37:43.324407+0000","last_became_active":"2024-09-11T07:32:00.509295+0000","last_became_peered":"2024-09-11T07:32:00.509295+0000","last_unstale":"2024-09-11T07:37:43.324407+0000","last_undegraded":"2024-09-11T07:37:43.324407+0000","last_fullsized":"2024-09-11T07:37:43.324407+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:42.084305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:42.084305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:53.270508+0000","scrub_duration":0.0047035039999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"234'19","reported_seq":304,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311869+0000","last_change":"2024-09-11T07:37:41.338132+0000","last_active":"2024-09-11T07:37:43.311869+0000","last_peered":"2024-09-11T07:37:43.311869+0000","last_clean":"2024-09-11T07:37:43.311869+0000","last_became_active":"2024-09-11T07:32:00.505134+0000","last_became_peered":"2024-09-11T07:32:00.505134+0000","last_unstale":"2024-09-11T07:37:43.311869+0000","last_undegraded":"2024-09-11T07:37:43.311869+0000","last_fullsized":"2024-09-11T07:37:43.311869+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:37:41.338036+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:37:41.338036+0000","last_clean_scrub_stamp":"2024-09-11T07:37:41.338036+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:08.290067+0000","scrub_duration":0.0053417279999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":257,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.324740+0000","last_change":"2024-09-11T07:36:27.958468+0000","last_active":"2024-09-11T07:37:43.324740+0000","last_peered":"2024-09-11T07:37:43.324740+0000","last_clean":"2024-09-11T07:37:43.324740+0000","last_became_active":"2024-09-11T07:26:31.419409+0000","last_became_peered":"2024-09-11T07:26:31.419409+0000","last_unstale":"2024-09-11T07:37:43.324740+0000","last_undegraded":"2024-09-11T07:37:43.324740+0000","last_fullsized":"2024-09-11T07:37:43.324740+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:27.958386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:27.958386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0093493710000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"151'10","reported_seq":403,"reported_epoch":256,"state":"active+clean","last_fresh":"2024-09-11T07:38:04.648876+0000","last_change":"2024-09-11T07:35:46.977819+0000","last_active":"2024-09-11T07:38:04.648876+0000","last_peered":"2024-09-11T07:38:04.648876+0000","last_clean":"2024-09-11T07:38:04.648876+0000","last_became_active":"2024-09-11T07:32:00.504062+0000","last_became_peered":"2024-09-11T07:32:00.504062+0000","last_unstale":"2024-09-11T07:38:04.648876+0000","last_undegraded":"2024-09-11T07:38:04.648876+0000","last_fullsized":"2024-09-11T07:38:04.648876+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:35:46.977611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:35:46.977611+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012282416000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":251,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.308914+0000","last_change":"2024-09-11T07:37:26.659394+0000","last_active":"2024-09-11T07:37:43.308914+0000","last_peered":"2024-09-11T07:37:43.308914+0000","last_clean":"2024-09-11T07:37:43.308914+0000","last_became_active":"2024-09-11T07:26:37.565723+0000","last_became_peered":"2024-09-11T07:26:37.565723+0000","last_unstale":"2024-09-11T07:37:43.308914+0000","last_undegraded":"2024-09-11T07:37:43.308914+0000","last_fullsized":"2024-09-11T07:37:43.308914+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:26.659337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.659337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:32.875069+0000","scrub_duration":0.0012563839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"236'16","reported_seq":290,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.321539+0000","last_change":"2024-09-11T07:36:49.809594+0000","last_active":"2024-09-11T07:37:43.321539+0000","last_peered":"2024-09-11T07:37:43.321539+0000","last_clean":"2024-09-11T07:37:43.321539+0000","last_became_active":"2024-09-11T07:26:33.540688+0000","last_became_peered":"2024-09-11T07:26:33.540688+0000","last_unstale":"2024-09-11T07:37:43.321539+0000","last_undegraded":"2024-09-11T07:37:43.321539+0000","last_fullsized":"2024-09-11T07:37:43.321539+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'16","last_scrub_stamp":"2024-09-11T07:36:49.809484+0000","last_deep_scrub":"236'16","last_deep_scrub_stamp":"2024-09-11T07:36:49.809484+0000","last_clean_scrub_stamp":"2024-09-11T07:36:49.809484+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:12.361926+0000","scrub_duration":0.017423357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":53,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311056+0000","last_change":"2024-09-11T07:37:22.914840+0000","last_active":"2024-09-11T07:37:43.311056+0000","last_peered":"2024-09-11T07:37:43.311056+0000","last_clean":"2024-09-11T07:37:43.311056+0000","last_became_active":"2024-09-11T07:36:05.225465+0000","last_became_peered":"2024-09-11T07:36:05.225465+0000","last_unstale":"2024-09-11T07:37:43.311056+0000","last_undegraded":"2024-09-11T07:37:43.311056+0000","last_fullsized":"2024-09-11T07:37:43.311056+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:22.914781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:09.772000+0000","last_clean_scrub_stamp":"2024-09-11T07:37:22.914781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:37.135943+0000","scrub_duration":0.001446864,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":243,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.308401+0000","last_change":"2024-09-11T07:35:51.886845+0000","last_active":"2024-09-11T07:37:43.308401+0000","last_peered":"2024-09-11T07:37:43.308401+0000","last_clean":"2024-09-11T07:37:43.308401+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:37:43.308401+0000","last_undegraded":"2024-09-11T07:37:43.308401+0000","last_fullsized":"2024-09-11T07:37:43.308401+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:51.886407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:35:51.886407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002900241,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":261,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.320881+0000","last_change":"2024-09-11T07:37:32.186622+0000","last_active":"2024-09-11T07:37:43.320881+0000","last_peered":"2024-09-11T07:37:43.320881+0000","last_clean":"2024-09-11T07:37:43.320881+0000","last_became_active":"2024-09-11T07:26:37.568886+0000","last_became_peered":"2024-09-11T07:26:37.568886+0000","last_unstale":"2024-09-11T07:37:43.320881+0000","last_undegraded":"2024-09-11T07:37:43.320881+0000","last_fullsized":"2024-09-11T07:37:43.320881+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:32.186561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:32.186561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:34.230574+0000","scrub_duration":0.0014950059999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"234'24","reported_seq":291,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.307395+0000","last_change":"2024-09-11T07:36:31.789902+0000","last_active":"2024-09-11T07:37:43.307395+0000","last_peered":"2024-09-11T07:37:43.307395+0000","last_clean":"2024-09-11T07:37:43.307395+0000","last_became_active":"2024-09-11T07:26:33.540208+0000","last_became_peered":"2024-09-11T07:26:33.540208+0000","last_unstale":"2024-09-11T07:37:43.307395+0000","last_undegraded":"2024-09-11T07:37:43.307395+0000","last_fullsized":"2024-09-11T07:37:43.307395+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'16","last_scrub_stamp":"2024-09-11T07:36:31.789841+0000","last_deep_scrub":"136'16","last_deep_scrub_stamp":"2024-09-11T07:31:06.039023+0000","last_clean_scrub_stamp":"2024-09-11T07:36:31.789841+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048561150000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":259,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.322186+0000","last_change":"2024-09-11T07:36:35.957044+0000","last_active":"2024-09-11T07:37:43.322186+0000","last_peered":"2024-09-11T07:37:43.322186+0000","last_clean":"2024-09-11T07:37:43.322186+0000","last_became_active":"2024-09-11T07:26:31.422671+0000","last_became_peered":"2024-09-11T07:26:31.422671+0000","last_unstale":"2024-09-11T07:37:43.322186+0000","last_undegraded":"2024-09-11T07:37:43.322186+0000","last_fullsized":"2024-09-11T07:37:43.322186+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:35.956943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:36:35.956943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053643689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":245,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.321388+0000","last_change":"2024-09-11T07:36:33.978210+0000","last_active":"2024-09-11T07:37:43.321388+0000","last_peered":"2024-09-11T07:37:43.321388+0000","last_clean":"2024-09-11T07:37:43.321388+0000","last_became_active":"2024-09-11T07:26:35.566931+0000","last_became_peered":"2024-09-11T07:26:35.566931+0000","last_unstale":"2024-09-11T07:37:43.321388+0000","last_undegraded":"2024-09-11T07:37:43.321388+0000","last_fullsized":"2024-09-11T07:37:43.321388+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:33.978003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.847474+0000","last_clean_scrub_stamp":"2024-09-11T07:36:33.978003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012287529,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":241,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.308579+0000","last_change":"2024-09-11T07:36:43.814458+0000","last_active":"2024-09-11T07:37:43.308579+0000","last_peered":"2024-09-11T07:37:43.308579+0000","last_clean":"2024-09-11T07:37:43.308579+0000","last_became_active":"2024-09-11T07:26:37.561573+0000","last_became_peered":"2024-09-11T07:26:37.561573+0000","last_unstale":"2024-09-11T07:37:43.308579+0000","last_undegraded":"2024-09-11T07:37:43.308579+0000","last_fullsized":"2024-09-11T07:37:43.308579+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:43.814370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:43.814370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:10.518232+0000","scrub_duration":0.0093820789999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"234'10","reported_seq":269,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.309898+0000","last_change":"2024-09-11T07:36:46.900027+0000","last_active":"2024-09-11T07:37:43.309898+0000","last_peered":"2024-09-11T07:37:43.309898+0000","last_clean":"2024-09-11T07:37:43.309898+0000","last_became_active":"2024-09-11T07:26:33.545830+0000","last_became_peered":"2024-09-11T07:26:33.545830+0000","last_unstale":"2024-09-11T07:37:43.309898+0000","last_undegraded":"2024-09-11T07:37:43.309898+0000","last_fullsized":"2024-09-11T07:37:43.309898+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'10","last_scrub_stamp":"2024-09-11T07:36:46.899947+0000","last_deep_scrub":"82'6","last_deep_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_clean_scrub_stamp":"2024-09-11T07:36:46.899947+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003412277,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":257,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311965+0000","last_change":"2024-09-11T07:36:50.285861+0000","last_active":"2024-09-11T07:37:43.311965+0000","last_peered":"2024-09-11T07:37:43.311965+0000","last_clean":"2024-09-11T07:37:43.311965+0000","last_became_active":"2024-09-11T07:26:31.420518+0000","last_became_peered":"2024-09-11T07:26:31.420518+0000","last_unstale":"2024-09-11T07:37:43.311965+0000","last_undegraded":"2024-09-11T07:37:43.311965+0000","last_fullsized":"2024-09-11T07:37:43.311965+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:50.285756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_clean_scrub_stamp":"2024-09-11T07:36:50.285756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020130360000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":178,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.307064+0000","last_change":"2024-09-11T07:36:47.675100+0000","last_active":"2024-09-11T07:37:43.307064+0000","last_peered":"2024-09-11T07:37:43.307064+0000","last_clean":"2024-09-11T07:37:43.307064+0000","last_became_active":"2024-09-11T07:36:05.226194+0000","last_became_peered":"2024-09-11T07:36:05.226194+0000","last_unstale":"2024-09-11T07:37:43.307064+0000","last_undegraded":"2024-09-11T07:37:43.307064+0000","last_fullsized":"2024-09-11T07:37:43.307064+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:47.675050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:47.675050+0000","last_clean_scrub_stamp":"2024-09-11T07:36:47.675050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00141387,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":251,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.313939+0000","last_change":"2024-09-11T07:36:50.407702+0000","last_active":"2024-09-11T07:37:43.313939+0000","last_peered":"2024-09-11T07:37:43.313939+0000","last_clean":"2024-09-11T07:37:43.313939+0000","last_became_active":"2024-09-11T07:32:00.511407+0000","last_became_peered":"2024-09-11T07:32:00.511407+0000","last_unstale":"2024-09-11T07:37:43.313939+0000","last_undegraded":"2024-09-11T07:37:43.313939+0000","last_fullsized":"2024-09-11T07:37:43.313939+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:50.407649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:50.407649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01288723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"234'19","reported_seq":283,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.308862+0000","last_change":"2024-09-11T07:35:32.805542+0000","last_active":"2024-09-11T07:37:43.308862+0000","last_peered":"2024-09-11T07:37:43.308862+0000","last_clean":"2024-09-11T07:37:43.308862+0000","last_became_active":"2024-09-11T07:32:00.502081+0000","last_became_peered":"2024-09-11T07:32:00.502081+0000","last_unstale":"2024-09-11T07:37:43.308862+0000","last_undegraded":"2024-09-11T07:37:43.308862+0000","last_fullsized":"2024-09-11T07:37:43.308862+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:35:32.805318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:35:32.805318+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0068191889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":43,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.312051+0000","last_change":"2024-09-11T07:36:05.228927+0000","last_active":"2024-09-11T07:37:43.312051+0000","last_peered":"2024-09-11T07:37:43.312051+0000","last_clean":"2024-09-11T07:37:43.312051+0000","last_became_active":"2024-09-11T07:36:05.228360+0000","last_became_peered":"2024-09-11T07:36:05.228360+0000","last_unstale":"2024-09-11T07:37:43.312051+0000","last_undegraded":"2024-09-11T07:37:43.312051+0000","last_fullsized":"2024-09-11T07:37:43.312051+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:48.366931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:48.366931+0000","last_clean_scrub_stamp":"2024-09-11T07:35:48.366931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":224,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.313676+0000","last_change":"2024-09-11T07:36:36.444436+0000","last_active":"2024-09-11T07:37:43.313676+0000","last_peered":"2024-09-11T07:37:43.313676+0000","last_clean":"2024-09-11T07:37:43.313676+0000","last_became_active":"2024-09-11T07:32:00.520534+0000","last_became_peered":"2024-09-11T07:32:00.520534+0000","last_unstale":"2024-09-11T07:37:43.313676+0000","last_undegraded":"2024-09-11T07:37:43.313676+0000","last_fullsized":"2024-09-11T07:37:43.313676+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:36.444301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:39.021876+0000","last_clean_scrub_stamp":"2024-09-11T07:36:36.444301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015961,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":241,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.308185+0000","last_change":"2024-09-11T07:36:49.899993+0000","last_active":"2024-09-11T07:37:43.308185+0000","last_peered":"2024-09-11T07:37:43.308185+0000","last_clean":"2024-09-11T07:37:43.308185+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:37:43.308185+0000","last_undegraded":"2024-09-11T07:37:43.308185+0000","last_fullsized":"2024-09-11T07:37:43.308185+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:49.899620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:49.899620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:12.310909+0000","scrub_duration":0.014021288999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":228,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.314579+0000","last_change":"2024-09-11T07:36:44.447910+0000","last_active":"2024-09-11T07:37:43.314579+0000","last_peered":"2024-09-11T07:37:43.314579+0000","last_clean":"2024-09-11T07:37:43.314579+0000","last_became_active":"2024-09-11T07:32:00.511258+0000","last_became_peered":"2024-09-11T07:32:00.511258+0000","last_unstale":"2024-09-11T07:37:43.314579+0000","last_undegraded":"2024-09-11T07:37:43.314579+0000","last_fullsized":"2024-09-11T07:37:43.314579+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:36:44.447809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:44.447809+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:07.090809+0000","scrub_duration":0.005540555,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":276,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311277+0000","last_change":"2024-09-11T07:36:31.353614+0000","last_active":"2024-09-11T07:37:43.311277+0000","last_peered":"2024-09-11T07:37:43.311277+0000","last_clean":"2024-09-11T07:37:43.311277+0000","last_became_active":"2024-09-11T07:32:00.518307+0000","last_became_peered":"2024-09-11T07:32:00.518307+0000","last_unstale":"2024-09-11T07:37:43.311277+0000","last_undegraded":"2024-09-11T07:37:43.311277+0000","last_fullsized":"2024-09-11T07:37:43.311277+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:36:31.353426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:31.353426+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0063773889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"234'20","reported_seq":95,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.308699+0000","last_change":"2024-09-11T07:37:29.661799+0000","last_active":"2024-09-11T07:37:43.308699+0000","last_peered":"2024-09-11T07:37:43.308699+0000","last_clean":"2024-09-11T07:37:43.308699+0000","last_became_active":"2024-09-11T07:35:00.561615+0000","last_became_peered":"2024-09-11T07:35:00.561615+0000","last_unstale":"2024-09-11T07:37:43.308699+0000","last_undegraded":"2024-09-11T07:37:43.308699+0000","last_fullsized":"2024-09-11T07:37:43.308699+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'20","last_scrub_stamp":"2024-09-11T07:37:29.661671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:29.661671+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:58.370347+0000","scrub_duration":0.0019609900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1f","version":"225'11","reported_seq":92,"reported_epoch":256,"state":"active+clean","last_fresh":"2024-09-11T07:38:04.262311+0000","last_change":"2024-09-11T07:36:05.227568+0000","last_active":"2024-09-11T07:38:04.262311+0000","last_peered":"2024-09-11T07:38:04.262311+0000","last_clean":"2024-09-11T07:38:04.262311+0000","last_became_active":"2024-09-11T07:36:05.227140+0000","last_became_peered":"2024-09-11T07:36:05.227140+0000","last_unstale":"2024-09-11T07:38:04.262311+0000","last_undegraded":"2024-09-11T07:38:04.262311+0000","last_fullsized":"2024-09-11T07:38:04.262311+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:35:32.194121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:35:32.194121+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":250,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.321806+0000","last_change":"2024-09-11T07:35:53.875376+0000","last_active":"2024-09-11T07:37:43.321806+0000","last_peered":"2024-09-11T07:37:43.321806+0000","last_clean":"2024-09-11T07:37:43.321806+0000","last_became_active":"2024-09-11T07:26:37.562501+0000","last_became_peered":"2024-09-11T07:26:37.562501+0000","last_unstale":"2024-09-11T07:37:43.321806+0000","last_undegraded":"2024-09-11T07:37:43.321806+0000","last_fullsized":"2024-09-11T07:37:43.321806+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:53.875300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:53.875300+0000","last_clean_scrub_stamp":"2024-09-11T07:35:53.875300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019957579999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":257,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.309133+0000","last_change":"2024-09-11T07:36:38.841363+0000","last_active":"2024-09-11T07:37:43.309133+0000","last_peered":"2024-09-11T07:37:43.309133+0000","last_clean":"2024-09-11T07:37:43.309133+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:37:43.309133+0000","last_undegraded":"2024-09-11T07:37:43.309133+0000","last_fullsized":"2024-09-11T07:37:43.309133+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:38.841250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:38.841250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014360088999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"234'19","reported_seq":86,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.312014+0000","last_change":"2024-09-11T07:36:05.229126+0000","last_active":"2024-09-11T07:37:43.312014+0000","last_peered":"2024-09-11T07:37:43.312014+0000","last_clean":"2024-09-11T07:37:43.312014+0000","last_became_active":"2024-09-11T07:36:05.228746+0000","last_became_peered":"2024-09-11T07:36:05.228746+0000","last_unstale":"2024-09-11T07:37:43.312014+0000","last_undegraded":"2024-09-11T07:37:43.312014+0000","last_fullsized":"2024-09-11T07:37:43.312014+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-11T07:35:36.232923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:35:36.232923+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":270,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.324566+0000","last_change":"2024-09-11T07:37:21.150801+0000","last_active":"2024-09-11T07:37:43.324566+0000","last_peered":"2024-09-11T07:37:43.324566+0000","last_clean":"2024-09-11T07:37:43.324566+0000","last_became_active":"2024-09-11T07:32:00.507050+0000","last_became_peered":"2024-09-11T07:32:00.507050+0000","last_unstale":"2024-09-11T07:37:43.324566+0000","last_undegraded":"2024-09-11T07:37:43.324566+0000","last_fullsized":"2024-09-11T07:37:43.324566+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:21.150732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:24.575210+0000","last_clean_scrub_stamp":"2024-09-11T07:37:21.150732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:42.313821+0000","scrub_duration":0.001360691,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":251,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.308785+0000","last_change":"2024-09-11T07:35:49.723748+0000","last_active":"2024-09-11T07:37:43.308785+0000","last_peered":"2024-09-11T07:37:43.308785+0000","last_clean":"2024-09-11T07:37:43.308785+0000","last_became_active":"2024-09-11T07:26:37.862373+0000","last_became_peered":"2024-09-11T07:26:37.862373+0000","last_unstale":"2024-09-11T07:37:43.308785+0000","last_undegraded":"2024-09-11T07:37:43.308785+0000","last_fullsized":"2024-09-11T07:37:43.308785+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:49.723695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:35:49.723695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00410707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":192,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.307749+0000","last_change":"2024-09-11T07:37:41.657806+0000","last_active":"2024-09-11T07:37:43.307749+0000","last_peered":"2024-09-11T07:37:43.307749+0000","last_clean":"2024-09-11T07:37:43.307749+0000","last_became_active":"2024-09-11T07:36:05.224085+0000","last_became_peered":"2024-09-11T07:36:05.224085+0000","last_unstale":"2024-09-11T07:37:43.307749+0000","last_undegraded":"2024-09-11T07:37:43.307749+0000","last_fullsized":"2024-09-11T07:37:43.307749+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:41.657719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:41.657719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:44.656666+0000","scrub_duration":0.002776653,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"234'27","reported_seq":308,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.311270+0000","last_change":"2024-09-11T07:37:18.388968+0000","last_active":"2024-09-11T07:37:43.311270+0000","last_peered":"2024-09-11T07:37:43.311270+0000","last_clean":"2024-09-11T07:37:43.311270+0000","last_became_active":"2024-09-11T07:26:33.542544+0000","last_became_peered":"2024-09-11T07:26:33.542544+0000","last_unstale":"2024-09-11T07:37:43.311270+0000","last_undegraded":"2024-09-11T07:37:43.311270+0000","last_fullsized":"2024-09-11T07:37:43.311270+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'27","last_scrub_stamp":"2024-09-11T07:37:18.388604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:18.388604+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:32.272894+0000","scrub_duration":0.0019030239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":252,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.322053+0000","last_change":"2024-09-11T07:36:44.853517+0000","last_active":"2024-09-11T07:37:43.322053+0000","last_peered":"2024-09-11T07:37:43.322053+0000","last_clean":"2024-09-11T07:37:43.322053+0000","last_became_active":"2024-09-11T07:26:35.555862+0000","last_became_peered":"2024-09-11T07:26:35.555862+0000","last_unstale":"2024-09-11T07:37:43.322053+0000","last_undegraded":"2024-09-11T07:37:43.322053+0000","last_fullsized":"2024-09-11T07:37:43.322053+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:44.850049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:36:44.850049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020441075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":251,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.310683+0000","last_change":"2024-09-11T07:37:32.004184+0000","last_active":"2024-09-11T07:37:43.310683+0000","last_peered":"2024-09-11T07:37:43.310683+0000","last_clean":"2024-09-11T07:37:43.310683+0000","last_became_active":"2024-09-11T07:26:37.861647+0000","last_became_peered":"2024-09-11T07:26:37.861647+0000","last_unstale":"2024-09-11T07:37:43.310683+0000","last_undegraded":"2024-09-11T07:37:43.310683+0000","last_fullsized":"2024-09-11T07:37:43.310683+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:32.004125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:32.004125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:34.761692+0000","scrub_duration":0.0019485520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":274,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.308528+0000","last_change":"2024-09-11T07:37:42.078889+0000","last_active":"2024-09-11T07:37:43.308528+0000","last_peered":"2024-09-11T07:37:43.308528+0000","last_clean":"2024-09-11T07:37:43.308528+0000","last_became_active":"2024-09-11T07:32:00.502947+0000","last_became_peered":"2024-09-11T07:32:00.502947+0000","last_unstale":"2024-09-11T07:37:43.308528+0000","last_undegraded":"2024-09-11T07:37:43.308528+0000","last_fullsized":"2024-09-11T07:37:43.308528+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:42.078765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:42.078765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:42.683014+0000","scrub_duration":0.0062011560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"234'23","reported_seq":292,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.308684+0000","last_change":"2024-09-11T07:37:43.078969+0000","last_active":"2024-09-11T07:37:43.308684+0000","last_peered":"2024-09-11T07:37:43.308684+0000","last_clean":"2024-09-11T07:37:43.308684+0000","last_became_active":"2024-09-11T07:26:33.542711+0000","last_became_peered":"2024-09-11T07:26:33.542711+0000","last_unstale":"2024-09-11T07:37:43.308684+0000","last_undegraded":"2024-09-11T07:37:43.308684+0000","last_fullsized":"2024-09-11T07:37:43.308684+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'23","last_scrub_stamp":"2024-09-11T07:37:43.078872+0000","last_deep_scrub":"127'17","last_deep_scrub_stamp":"2024-09-11T07:35:43.871918+0000","last_clean_scrub_stamp":"2024-09-11T07:37:43.078872+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:09.927640+0000","scrub_duration":0.0051065670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":253,"reported_epoch":255,"state":"active+clean","last_fresh":"2024-09-11T07:37:43.307984+0000","last_change":"2024-09-11T07:36:37.811087+0000","last_active":"2024-09-11T07:37:43.307984+0000","last_peered":"2024-09-11T07:37:43.307984+0000","last_clean":"2024-09-11T07:37:43.307984+0000","last_became_active":"2024-09-11T07:26:35.562037+0000","last_became_peered":"2024-09-11T07:26:35.562037+0000","last_unstale":"2024-09-11T07:37:43.307984+0000","last_undegraded":"2024-09-11T07:37:43.307984+0000","last_fullsized":"2024-09-11T07:37:43.307984+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:37.811009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:36:37.811009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039204729999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":4195876864,"num_objects":64024,"num_object_clones":0,"num_object_copies":192072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64024,"num_write_kb":4097536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12560564224,"data_stored":12560564224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64024,"ondisk_log_size":64024,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1124,"num_read_kb":881,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":764,"ondisk_log_size":764,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":690,"num_read_kb":685,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":147,"seq":631360192587,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2582088,"kb_used_data":2253440,"kb_used_omap":4,"kb_used_meta":328635,"kb_avail":91163064,"statfs":{"total":95995035648,"available":93350977536,"internally_reserved":0,"allocated":2307522560,"data_stored":2307107049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":336522758},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.8,"15min":3.1440000000000001},"min":{"1min":0.22,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":3.0910000000000002,"5min":49.573,"15min":49.573},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.177,"5min":1.833,"15min":2.9289999999999998},"min":{"1min":0.24199999999999999,"5min":0.22,"15min":0.22},"max":{"1min":6.1470000000000002,"5min":49.881,"15min":49.881},"last":0.80400000000000005}]},{"osd":1,"last update":"Wed Sep 11 07:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.98599999999999999,"15min":0.72599999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":2.9910000000000001,"5min":10.063000000000001,"15min":10.063000000000001},"last":0.30499999999999999},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.206,"15min":0.93799999999999994},"min":{"1min":0.252,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":3.6499999999999999,"5min":9.6059999999999999,"15min":9.6059999999999999},"last":0.38800000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:37:14 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0649999999999999,"15min":1.0129999999999999},"min":{"1min":0.27100000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":5.6909999999999998,"5min":13.99,"15min":13.99},"last":0.33100000000000002},{"interface":"front","average":{"1min":1.111,"5min":1.196,"15min":1.01},"min":{"1min":0.28499999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.2989999999999999,"5min":14.67,"15min":14.67},"last":1.028}]},{"osd":3,"last update":"Wed Sep 11 07:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":1.661,"15min":2.9529999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":3.3279999999999998,"5min":49.798999999999999,"15min":49.798999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.7210000000000001,"15min":2.883},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.4079999999999999,"5min":49.524000000000001,"15min":49.524000000000001},"last":0.93100000000000005}]},{"osd":4,"last update":"Wed Sep 11 07:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.64500000000000002,"15min":0.49199999999999999},"min":{"1min":0.28799999999999998,"5min":0.253,"15min":0.253},"max":{"1min":6.0220000000000002,"5min":6.0220000000000002,"15min":6.0220000000000002},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.66700000000000004,"15min":0.623},"min":{"1min":0.30199999999999999,"5min":0.247,"15min":0.247},"max":{"1min":3.044,"5min":3.9340000000000002,"15min":3.9340000000000002},"last":0.35099999999999998}]},{"osd":5,"last update":"Wed Sep 11 07:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.70899999999999996,"15min":0.52900000000000003},"min":{"1min":0.309,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.9909999999999997,"5min":5.9909999999999997,"15min":5.9909999999999997},"last":1.353},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.77500000000000002,"15min":0.69699999999999995},"min":{"1min":0.32100000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.0149999999999997,"5min":4.9969999999999999,"15min":4.9969999999999999},"last":0.83099999999999996}]},{"osd":6,"last update":"Wed Sep 11 07:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66800000000000004,"15min":0.56299999999999994},"min":{"1min":0.34399999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":3.3490000000000002,"5min":4.6440000000000001,"15min":4.6440000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.72299999999999998,"15min":0.59899999999999998},"min":{"1min":0.34399999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":3.5470000000000002,"5min":3.694,"15min":3.694},"last":0.73099999999999998}]}]},{"osd":6,"up_from":44,"seq":188978561172,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1060272,"kb_used_data":762920,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":92684880,"statfs":{"total":95995035648,"available":94909317120,"internally_reserved":0,"allocated":781230080,"data_stored":780837623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:37:11 2024","interfaces":[{"interface":"back","average":{"1min":1.96,"5min":2.984,"15min":1.7549999999999999},"min":{"1min":0.22,"5min":0.20699999999999999,"15min":0.192},"max":{"1min":20.675999999999998,"5min":142.29599999999999,"15min":142.29599999999999},"last":0.33800000000000002},{"interface":"front","average":{"1min":1.9450000000000001,"5min":3.2999999999999998,"15min":1.8919999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":20.663,"5min":142.68799999999999,"15min":142.68799999999999},"last":0.26800000000000002}]},{"osd":1,"last update":"Wed Sep 11 07:37:11 2024","interfaces":[{"interface":"back","average":{"1min":2.964,"5min":2.645,"15min":1.3260000000000001},"min":{"1min":0.249,"5min":0.24099999999999999,"15min":0.23599999999999999},"max":{"1min":39.390000000000001,"5min":127.92100000000001,"15min":127.92100000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":3.0800000000000001,"5min":2.4790000000000001,"15min":1.3140000000000001},"min":{"1min":0.27200000000000002,"5min":0.24099999999999999,"15min":0.23000000000000001},"max":{"1min":40.186999999999998,"5min":127.746,"15min":127.746},"last":0.499}]},{"osd":2,"last update":"Wed Sep 11 07:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.92700000000000005,"15min":0.85899999999999999},"min":{"1min":0.27300000000000002,"5min":0.248,"15min":0.248},"max":{"1min":3.6539999999999999,"5min":4.3220000000000001,"15min":26.491},"last":0.39200000000000002},{"interface":"front","average":{"1min":1.363,"5min":1.198,"15min":1.018},"min":{"1min":0.27100000000000002,"5min":0.26500000000000001,"15min":0.26400000000000001},"max":{"1min":3.9689999999999999,"5min":6.0819999999999999,"15min":26.561},"last":0.54000000000000004}]},{"osd":3,"last update":"Wed Sep 11 07:37:11 2024","interfaces":[{"interface":"back","average":{"1min":2.274,"5min":1.4750000000000001,"15min":1.0660000000000001},"min":{"1min":0.26100000000000001,"5min":0.25600000000000001,"15min":0.254},"max":{"1min":20.734000000000002,"5min":20.734000000000002,"15min":31.268999999999998},"last":0.378},{"interface":"front","average":{"1min":2.0840000000000001,"5min":1.3149999999999999,"15min":1.1499999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":20.701000000000001,"5min":20.701000000000001,"15min":31.872},"last":0.72199999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.626,"15min":0.55700000000000005},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":1.6719999999999999,"5min":3.3599999999999999,"15min":4.4720000000000004},"last":2.1000000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56100000000000005,"15min":0.46500000000000002},"min":{"1min":0.25900000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.329,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":0.70799999999999996}]},{"osd":5,"last update":"Wed Sep 11 07:37:11 2024","interfaces":[{"interface":"back","average":{"1min":3.6349999999999998,"5min":1.417,"15min":0.85499999999999998},"min":{"1min":0.30199999999999999,"5min":0.255,"15min":0.255},"max":{"1min":57.436,"5min":57.436,"15min":57.436},"last":0.41899999999999998},{"interface":"front","average":{"1min":3.359,"5min":1.429,"15min":0.81399999999999995},"min":{"1min":0.32300000000000001,"5min":0.251,"15min":0.251},"max":{"1min":56.822000000000003,"5min":56.822000000000003,"15min":56.822000000000003},"last":0.28599999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.68700000000000006,"15min":0.66600000000000004},"min":{"1min":0.312,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.3109999999999999,"5min":3.3140000000000001,"15min":3.3140000000000001},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73099999999999998,"15min":0.72599999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":3.6749999999999998,"5min":5.0439999999999996,"15min":5.0439999999999996},"last":0.44700000000000001}]}]},{"osd":5,"up_from":37,"seq":158913790106,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2589248,"kb_used_data":2291896,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":91155904,"statfs":{"total":95995035648,"available":93343645696,"internally_reserved":0,"allocated":2346901504,"data_stored":2346493287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.891,"5min":1.5089999999999999,"15min":2.121},"min":{"1min":0.28000000000000003,"5min":0.21299999999999999,"15min":0.20799999999999999},"max":{"1min":25.808,"5min":31.369,"15min":185.68199999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":2.0059999999999998,"5min":1.766,"15min":2.2759999999999998},"min":{"1min":0.22900000000000001,"5min":0.223,"15min":0.218},"max":{"1min":25.722999999999999,"5min":25.722999999999999,"15min":186.12200000000001},"last":0.66000000000000003}]},{"osd":1,"last update":"Wed Sep 11 07:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3540000000000001,"15min":0.90600000000000003},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.20899999999999999},"max":{"1min":13.297000000000001,"5min":26.286999999999999,"15min":26.286999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2629999999999999,"15min":0.94199999999999995},"min":{"1min":0.255,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":13.281000000000001,"5min":24.672999999999998,"15min":24.672999999999998},"last":2.9660000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:37:49 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.91500000000000004,"15min":1.5589999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.22700000000000001},"max":{"1min":1.1719999999999999,"5min":4.585,"15min":187.99199999999999},"last":1.9970000000000001},{"interface":"front","average":{"1min":1.0620000000000001,"5min":0.91900000000000004,"15min":1.546},"min":{"1min":0.28399999999999997,"5min":0.27600000000000002,"15min":0.254},"max":{"1min":7.8120000000000003,"5min":7.8120000000000003,"15min":186.375},"last":0.33500000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:37:49 2024","interfaces":[{"interface":"back","average":{"1min":1.865,"5min":1.55,"15min":2.226},"min":{"1min":0.26800000000000002,"5min":0.26400000000000001,"15min":0.23599999999999999},"max":{"1min":28.978000000000002,"5min":28.978000000000002,"15min":188.11500000000001},"last":2.2509999999999999},{"interface":"front","average":{"1min":1.9610000000000001,"5min":1.528,"15min":2.2040000000000002},"min":{"1min":0.32700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":25.837,"5min":26.936,"15min":186.44200000000001},"last":0.52500000000000002}]},{"osd":4,"last update":"Wed Sep 11 07:37:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.60599999999999998,"15min":0.47199999999999998},"min":{"1min":0.25,"5min":0.247,"15min":0.19900000000000001},"max":{"1min":1.4890000000000001,"5min":4.7060000000000004,"15min":4.7060000000000004},"last":2.867},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.67100000000000004,"15min":0.505},"min":{"1min":0.32600000000000001,"5min":0.24199999999999999,"15min":0.23200000000000001},"max":{"1min":2.0830000000000002,"5min":2.8439999999999999,"15min":2.8439999999999999},"last":1.9350000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:37:12 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.70599999999999996,"15min":0.51900000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.26000000000000001},"max":{"1min":1.7849999999999999,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":2.02},{"interface":"front","average":{"1min":0.747,"5min":0.65100000000000002,"15min":0.51200000000000001},"min":{"1min":0.33900000000000002,"5min":0.26800000000000002,"15min":0.25800000000000001},"max":{"1min":4.3120000000000003,"5min":4.3120000000000003,"15min":4.3120000000000003},"last":0.69799999999999995}]},{"osd":7,"last update":"Wed Sep 11 07:37:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.745,"15min":0.64100000000000001},"min":{"1min":0.314,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":2.464,"5min":4.7919999999999998,"15min":4.7919999999999998},"last":2.8959999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.76700000000000002,"15min":0.65700000000000003},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.8140000000000001,"5min":4.0110000000000001,"15min":4.0110000000000001},"last":1.962}]}]},{"osd":4,"up_from":31,"seq":133143986334,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1056644,"kb_used_data":746812,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":92688508,"statfs":{"total":95995035648,"available":94913032192,"internally_reserved":0,"allocated":764735488,"data_stored":764323018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:37:36 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":1.242,"15min":1.1080000000000001},"min":{"1min":0.27300000000000002,"5min":0.217,"15min":0.20799999999999999},"max":{"1min":2.7530000000000001,"5min":32.892000000000003,"15min":35.308},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.337,"15min":1.087},"min":{"1min":0.249,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":4.5620000000000003,"5min":32.860999999999997,"15min":35.597999999999999},"last":0.22700000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:37:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":1.365,"15min":0.97599999999999998},"min":{"1min":0.23100000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":2.1240000000000001,"5min":24.581,"15min":24.581},"last":0.54500000000000004},{"interface":"front","average":{"1min":1.23,"5min":1.28,"15min":0.89300000000000002},"min":{"1min":0.38,"5min":0.24399999999999999,"15min":0.23699999999999999},"max":{"1min":5.8010000000000002,"5min":24.652000000000001,"15min":24.652000000000001},"last":1.2110000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:37:36 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.036,"15min":0.92200000000000004},"min":{"1min":0.23699999999999999,"5min":0.223,"15min":0.223},"max":{"1min":3.726,"5min":11.425000000000001,"15min":19.927},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.94399999999999995,"5min":1.0960000000000001,"15min":0.91800000000000004},"min":{"1min":0.28999999999999998,"5min":0.27200000000000002,"15min":0.24399999999999999},"max":{"1min":4.101,"5min":13.423999999999999,"15min":20.02},"last":0.91500000000000004}]},{"osd":3,"last update":"Wed Sep 11 07:37:36 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.3109999999999999,"15min":1.0720000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.673,"5min":24.029,"15min":24.029},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":1.2390000000000001,"15min":0.99299999999999999},"min":{"1min":0.33900000000000002,"5min":0.32100000000000001,"15min":0.28699999999999998},"max":{"1min":2.0070000000000001,"5min":24.265000000000001,"15min":24.265000000000001},"last":0.51900000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.72999999999999998,"15min":0.49099999999999999},"min":{"1min":0.23699999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":4.6079999999999997,"5min":4.6079999999999997,"15min":4.6079999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73599999999999999,"15min":0.50800000000000001},"min":{"1min":0.28399999999999997,"5min":0.25,"15min":0.22700000000000001},"max":{"1min":2.9990000000000001,"5min":5.7320000000000002,"15min":5.7320000000000002},"last":0.39200000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:37:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.72499999999999998,"15min":0.52700000000000002},"min":{"1min":0.255,"5min":0.23999999999999999,"15min":0.20899999999999999},"max":{"1min":1.96,"5min":7.8650000000000002,"15min":7.8650000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.73199999999999998,"15min":0.53800000000000003},"min":{"1min":0.27500000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":3.7949999999999999,"5min":9.3770000000000007,"15min":9.3770000000000007},"last":0.28599999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:37:09 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.65000000000000002,"15min":0.57999999999999996},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":2.4470000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.311},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70699999999999996,"15min":0.60099999999999998},"min":{"1min":0.29799999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.2949999999999999,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":0.57099999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182563,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2320908,"kb_used_data":2015236,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":91424244,"statfs":{"total":95995035648,"available":93618425856,"internally_reserved":0,"allocated":2063601664,"data_stored":2063180466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:37:16 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.6299999999999999,"15min":1.1919999999999999},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":4.2709999999999999,"5min":38.432000000000002,"15min":38.432000000000002},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.353,"5min":1.7190000000000001,"15min":1.546},"min":{"1min":0.26300000000000001,"5min":0.20300000000000001,"15min":0.19500000000000001},"max":{"1min":4.657,"5min":33.426000000000002,"15min":91.876999999999995},"last":2.9119999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:37:16 2024","interfaces":[{"interface":"back","average":{"1min":5.1669999999999998,"5min":2.2480000000000002,"15min":1.3979999999999999},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":67.305000000000007,"5min":67.305000000000007,"15min":67.305000000000007},"last":2.9430000000000001},{"interface":"front","average":{"1min":1.252,"5min":1.484,"15min":1.0209999999999999},"min":{"1min":0.23799999999999999,"5min":0.192,"15min":0.192},"max":{"1min":4.4619999999999997,"5min":9.3450000000000006,"15min":9.3450000000000006},"last":2.7959999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":1.387,"15min":1.0980000000000001},"min":{"1min":0.24199999999999999,"5min":0.23699999999999999,"15min":0.23200000000000001},"max":{"1min":4.4240000000000004,"5min":8.282,"15min":8.282},"last":2.7469999999999999},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.2669999999999999,"15min":1.03},"min":{"1min":0.315,"5min":0.26200000000000001,"15min":0.20999999999999999},"max":{"1min":5.1520000000000001,"5min":6.5640000000000001,"15min":8.7110000000000003},"last":1.587}]},{"osd":4,"last update":"Wed Sep 11 07:37:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.99099999999999999,"15min":0.95499999999999996},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":2.1760000000000002,"5min":5.5099999999999998,"15min":21.856000000000002},"last":1.5389999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":1.006,"15min":1.105},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":2.7200000000000002,"5min":4.6219999999999999,"15min":19.146000000000001},"last":0.47899999999999998}]},{"osd":5,"last update":"Wed Sep 11 07:37:47 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":1.0309999999999999,"15min":1.0369999999999999},"min":{"1min":0.42599999999999999,"5min":0.309,"15min":0.29599999999999999},"max":{"1min":2.25,"5min":5.0209999999999999,"15min":32.325000000000003},"last":1.5069999999999999},{"interface":"front","average":{"1min":0.83699999999999997,"5min":1.1439999999999999,"15min":1.109},"min":{"1min":0.39000000000000001,"5min":0.28599999999999998,"15min":0.28299999999999997},"max":{"1min":2.3109999999999999,"5min":4.8490000000000002,"15min":18.416},"last":0.50900000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:37:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.3740000000000001,"15min":1.3500000000000001},"min":{"1min":0.372,"5min":0.308,"15min":0.27000000000000002},"max":{"1min":4.5179999999999998,"5min":32.350999999999999,"15min":32.350999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.76900000000000002,"5min":1.3220000000000001,"15min":1.3240000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.33100000000000002},"max":{"1min":2.2290000000000001,"5min":32.402999999999999,"15min":32.402999999999999},"last":1.5620000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:37:09 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":1.194,"15min":1.3240000000000001},"min":{"1min":0.32300000000000001,"5min":0.308,"15min":0.308},"max":{"1min":2.7679999999999998,"5min":8.0429999999999993,"15min":8.0429999999999993},"last":0.53200000000000003},{"interface":"front","average":{"1min":1.077,"5min":1.4570000000000001,"15min":2.157},"min":{"1min":0.39400000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":4.4909999999999997,"5min":32.289999999999999,"15min":32.289999999999999},"last":0.58899999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411496,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1069292,"kb_used_data":767780,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":92675860,"statfs":{"total":95995035648,"available":94900080640,"internally_reserved":0,"allocated":786206720,"data_stored":785817942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:37:48 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":2.3290000000000002,"15min":1.4670000000000001},"min":{"1min":0.185,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":1.958,"5min":56.704000000000001,"15min":56.704000000000001},"last":0.20699999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":2.532,"15min":1.5449999999999999},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":4.5199999999999996,"5min":56.816000000000003,"15min":56.816000000000003},"last":4.6059999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:37:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":1.4970000000000001,"15min":1.1679999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.23200000000000001},"max":{"1min":1.55,"5min":65.153999999999996,"15min":65.153999999999996},"last":0.87},{"interface":"front","average":{"1min":0.67000000000000004,"5min":1.946,"15min":1.3600000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.20699999999999999},"max":{"1min":2.8540000000000001,"5min":63.311999999999998,"15min":63.311999999999998},"last":0.69999999999999996}]},{"osd":3,"last update":"Wed Sep 11 07:37:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":2.0619999999999998,"15min":1.3220000000000001},"min":{"1min":0.28100000000000003,"5min":0.27400000000000002,"15min":0.26900000000000002},"max":{"1min":6.4960000000000004,"5min":44.405999999999999,"15min":44.405999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":1.0349999999999999,"5min":2.0259999999999998,"15min":1.3520000000000001},"min":{"1min":0.27100000000000002,"5min":0.253,"15min":0.253},"max":{"1min":3.222,"5min":43.295999999999999,"15min":43.295999999999999},"last":0.40699999999999997}]},{"osd":4,"last update":"Wed Sep 11 07:37:34 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.90400000000000003,"15min":1.036},"min":{"1min":0.34000000000000002,"5min":0.30599999999999999,"15min":0.27500000000000002},"max":{"1min":1.004,"5min":6.7599999999999998,"15min":30.946999999999999},"last":0.437},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.86899999999999999,"15min":0.93899999999999995},"min":{"1min":0.41599999999999998,"5min":0.31900000000000001,"15min":0.29999999999999999},"max":{"1min":1.8500000000000001,"5min":3.0960000000000001,"15min":22.434000000000001},"last":0.45000000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:37:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.85999999999999999,"15min":1.048},"min":{"1min":0.436,"5min":0.35099999999999998,"15min":0.31900000000000001},"max":{"1min":3.3290000000000002,"5min":3.3290000000000002,"15min":31.789000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0129999999999999,"15min":1.133},"min":{"1min":0.376,"5min":0.34100000000000003,"15min":0.31},"max":{"1min":3.931,"5min":4.1479999999999997,"15min":30.898},"last":0.72399999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:37:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.2549999999999999,"15min":1.0489999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.36099999999999999},"max":{"1min":2.9430000000000001,"5min":22.399000000000001,"15min":22.399000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.3819999999999999,"15min":1.048},"min":{"1min":0.311,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.0680000000000001,"5min":25.739000000000001,"15min":25.739000000000001},"last":0.51100000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.85999999999999999,"15min":1.8420000000000001},"min":{"1min":0.47499999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.97199999999999998,"5min":2.6600000000000001,"15min":22.484000000000002},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.749,"5min":1.046,"15min":2.1640000000000001},"min":{"1min":0.46100000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.7869999999999999,"5min":4.1719999999999997,"15min":25.670000000000002},"last":0.55300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575019,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2576464,"kb_used_data":2274952,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":91168688,"statfs":{"total":95995035648,"available":93356736512,"internally_reserved":0,"allocated":2329550848,"data_stored":2329126368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:37:18 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.6180000000000001,"15min":1.6850000000000001},"min":{"1min":0.24099999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":7.9429999999999996,"5min":8.0549999999999997,"15min":82.218999999999994},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.7470000000000001,"15min":1.706},"min":{"1min":0.217,"5min":0.193,"15min":0.191},"max":{"1min":5.5019999999999998,"5min":8.0739999999999998,"15min":82.278999999999996},"last":0.45400000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:37:51 2024","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.6910000000000001,"15min":1.2190000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.19800000000000001},"max":{"1min":6.0279999999999996,"5min":9.0969999999999995,"15min":9.0969999999999995},"last":0.502},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.5,"15min":1.25},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":4.0119999999999996,"5min":9.1310000000000002,"15min":9.1310000000000002},"last":0.65100000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:37:09 2024","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.736,"15min":1.7709999999999999},"min":{"1min":0.29199999999999998,"5min":0.28000000000000003,"15min":0.25800000000000001},"max":{"1min":6.1360000000000001,"5min":9.2889999999999997,"15min":81.882999999999996},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.423,"5min":1.7170000000000001,"15min":1.7230000000000001},"min":{"1min":0.36299999999999999,"5min":0.22600000000000001,"15min":0.20999999999999999},"max":{"1min":5.2359999999999998,"5min":14.428000000000001,"15min":82.400000000000006},"last":1.3049999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:37:26 2024","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.331,"15min":0.94399999999999995},"min":{"1min":0.33000000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":5.5609999999999999,"5min":7.3630000000000004,"15min":7.3630000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.3400000000000001,"15min":1.0780000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.26500000000000001},"max":{"1min":4.4459999999999997,"5min":9.2289999999999992,"15min":9.2289999999999992},"last":0.78500000000000003}]},{"osd":5,"last update":"Wed Sep 11 07:37:51 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.3779999999999999,"15min":1.0169999999999999},"min":{"1min":0.371,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":4.5099999999999998,"5min":7.4379999999999997,"15min":7.4379999999999997},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.3899999999999999,"15min":1.113},"min":{"1min":0.34499999999999997,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":4.6799999999999997,"5min":9.1929999999999996,"15min":9.1929999999999996},"last":0.90000000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:37:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.341,"15min":1.133},"min":{"1min":0.41399999999999998,"5min":0.316,"15min":0.30099999999999999},"max":{"1min":2.6299999999999999,"5min":9.2569999999999997,"15min":24.018000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.0349999999999999,"15min":0.94899999999999995},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.29999999999999999},"max":{"1min":5.2709999999999999,"5min":5.2709999999999999,"15min":23.681000000000001},"last":1.2010000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:37:09 2024","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.196,"15min":1.3},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":5.3079999999999998,"5min":7.0940000000000003,"15min":7.0940000000000003},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.7689999999999999,"5min":1.4770000000000001,"15min":1.514},"min":{"1min":0.36899999999999999,"5min":0.313,"15min":0.313},"max":{"1min":5.5860000000000003,"5min":7.4050000000000002,"15min":7.4050000000000002},"last":1.3340000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738545,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1329984,"kb_used_data":1024312,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":92415168,"statfs":{"total":95995035648,"available":94633132032,"internally_reserved":0,"allocated":1048895488,"data_stored":1048486979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:37:19 2024","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.2689999999999999,"15min":0.98399999999999999},"min":{"1min":0.32300000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":5.4630000000000001,"5min":5.4630000000000001,"15min":9.7710000000000008},"last":1.05},{"interface":"front","average":{"1min":1.8080000000000001,"5min":1.333,"15min":1.0129999999999999},"min":{"1min":0.26800000000000002,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":5.1479999999999997,"5min":5.1479999999999997,"15min":10.275},"last":0.89300000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:37:50 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.3100000000000001,"15min":1.208},"min":{"1min":0.23000000000000001,"5min":0.21299999999999999,"15min":0.20200000000000001},"max":{"1min":5.3040000000000003,"5min":11.397,"15min":19.527999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.878,"5min":1.1160000000000001,"15min":1.145},"min":{"1min":0.224,"5min":0.21099999999999999,"15min":0.20999999999999999},"max":{"1min":4.2999999999999998,"5min":6.5960000000000001,"15min":19.556999999999999},"last":1.75}]},{"osd":3,"last update":"Wed Sep 11 07:37:19 2024","interfaces":[{"interface":"back","average":{"1min":1.8140000000000001,"5min":1.55,"15min":1.7310000000000001},"min":{"1min":0.35199999999999998,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":9.4199999999999999,"5min":9.4920000000000009,"15min":45.185000000000002},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.379,"5min":1.383,"15min":1.4950000000000001},"min":{"1min":0.378,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.2460000000000004,"5min":8.0999999999999996,"15min":43.228999999999999},"last":0.74299999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:37:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.218,"15min":0.995},"min":{"1min":0.29799999999999999,"5min":0.26800000000000002,"15min":0.26700000000000002},"max":{"1min":4.8109999999999999,"5min":6.8319999999999999,"15min":19.411999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.196,"15min":1.1319999999999999},"min":{"1min":0.35999999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":2.9260000000000002,"5min":9.5619999999999994,"15min":19.396000000000001},"last":0.77200000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:37:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.246,"15min":1.103},"min":{"1min":0.34100000000000003,"5min":0.29599999999999999,"15min":0.28999999999999998},"max":{"1min":5.0830000000000002,"5min":6.9400000000000004,"15min":19.381},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.86499999999999999,"5min":1.1839999999999999,"15min":1.1830000000000001},"min":{"1min":0.40000000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.0910000000000002,"5min":9.6289999999999996,"15min":19.437000000000001},"last":0.878}]},{"osd":6,"last update":"Wed Sep 11 07:37:19 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.337,"15min":1.153},"min":{"1min":0.45000000000000001,"5min":0.33300000000000002,"15min":0.316},"max":{"1min":2.29,"5min":23.190999999999999,"15min":23.190999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.995,"5min":1.4350000000000001,"15min":1.1599999999999999},"min":{"1min":0.373,"5min":0.35999999999999999,"15min":0.34200000000000003},"max":{"1min":4.1619999999999999,"5min":23.632000000000001,"15min":23.632000000000001},"last":1.02}]},{"osd":7,"last update":"Wed Sep 11 07:37:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.288,"15min":1.6599999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":2.9540000000000002,"5min":9.6530000000000005,"15min":9.6530000000000005},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.2769999999999999,"15min":1.71},"min":{"1min":0.42699999999999999,"5min":0.315,"15min":0.315},"max":{"1min":4.194,"5min":5.2759999999999998,"15min":5.2759999999999998},"last":0.95199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1067515904,"data_stored":1067515904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2352545792,"data_stored":2352545792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":794230784,"data_stored":794230784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2095185920,"data_stored":2095185920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":775421952,"data_stored":775421952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2350448640,"data_stored":2350448640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":783613952,"data_stored":783613952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2341601280,"data_stored":2341601280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:38:14.131 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-11T07:38:14.131 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pool set unique_pool_2 pg_num 26 2024-09-11T07:38:14.139 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:38:14.175 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:14.196 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:38:14.261 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:14.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:14.262 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:14.362 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:14.414 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:14.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:14.415 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:14.482 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 37 16 70353 70337 118.788 80 0.0119528 0.00841337 2024-09-11T07:38:14.516 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:14.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:38:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:38:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:38:14.542 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:14.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:14.543 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:14.644 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:14.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:14.678 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:14.678 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:14.779 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:14.808 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:14.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:14.809 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:14.910 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:14.950 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:14.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:14.952 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:15.052 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:15.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:15.084 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:15.084 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:15.185 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:15.225 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:15.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:15.226 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:15.327 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:15.355 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:15.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:15.356 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:15.457 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:15.480 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 38 16 71727 71711 117.922 85.875 0.00714895 0.00847477 2024-09-11T07:38:15.498 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:15.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:15.499 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:15.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:15 smithi104 ceph-mon[33507]: pgmap v520: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-11T07:38:15.600 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:15.646 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:15.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:15.647 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:15.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:15 smithi022 ceph-mon[29462]: pgmap v520: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-11T07:38:15.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:15 smithi022 ceph-mon[37330]: pgmap v520: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-11T07:38:15.747 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:15.778 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:15.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:15.779 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:15.880 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:15.917 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:15.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:15.918 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:16.019 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:16.057 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:16.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:16.058 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:16.159 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:16.196 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:16.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:16.197 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:16.298 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:16.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:16.335 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:16.336 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:16.436 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:16.480 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 39 16 73173 73157 117.215 90.375 0.00652532 0.00852576 2024-09-11T07:38:16.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:16.481 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:16.482 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:16.583 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:16.622 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:16.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:16.623 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:16.724 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:16.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:16.763 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:16.763 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:16.864 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:16.891 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:16 smithi022 ceph-mon[29462]: pgmap v521: 148 pgs: 148 active+clean; 4.3 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.3 KiB/s rd, 97 MiB/s wr, 1.56k op/s 2024-09-11T07:38:16.892 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:16 smithi022 ceph-mon[37330]: pgmap v521: 148 pgs: 148 active+clean; 4.3 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.3 KiB/s rd, 97 MiB/s wr, 1.56k op/s 2024-09-11T07:38:16.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:16.902 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:16.902 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:16.984 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:38:17.003 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:17.030 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:17.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:17.032 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:17.032 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:16 smithi104 ceph-mon[33507]: pgmap v521: 148 pgs: 148 active+clean; 4.3 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.3 KiB/s rd, 97 MiB/s wr, 1.56k op/s 2024-09-11T07:38:17.034 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:38:17.133 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:17.170 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:17.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:17.171 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:17.272 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:17.301 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:17.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:17.302 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:17.403 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:17.443 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:17.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:17.444 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:17.480 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:2024-09-11T07:38:17.479922+0000 min lat: 0.00139335 max lat: 0.183392 avg lat: 0.00858052 2024-09-11T07:38:17.480 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T07:38:17.481 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 40 16 74571 74555 116.469 87.375 0.00558985 0.00858052 2024-09-11T07:38:17.545 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:17.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:17.585 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:17.585 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:17.686 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:17.718 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:17.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:17.719 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:17.820 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:17.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:17.853 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:17.853 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:17.954 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:17.997 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:17.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:17.998 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:18.098 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:18.135 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:18.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:18.136 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:18.237 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:18.268 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:18.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:18.269 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:18.370 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:18.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:18.400 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:18.400 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:18.480 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 41 16 75859 75843 115.592 80.5 0.0139637 0.00864603 2024-09-11T07:38:18.501 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:18.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:18.535 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:18.536 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:18.636 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:18.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:18.677 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:18.677 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:18.779 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:18.815 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:18.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:18.817 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:18.917 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:18.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:18.951 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:18.951 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:19.052 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:19.093 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:19.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:19.094 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:19.195 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:19.249 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:19.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:19.250 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:19.351 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:19.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:19.387 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:19.387 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:19.462 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:19 smithi022 ceph-mon[29462]: pgmap v522: 148 pgs: 148 active+clean; 4.3 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 73 MiB/s wr, 1.18k op/s 2024-09-11T07:38:19.463 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:19 smithi022 ceph-mon[37330]: pgmap v522: 148 pgs: 148 active+clean; 4.3 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 73 MiB/s wr, 1.18k op/s 2024-09-11T07:38:19.480 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 42 16 77184 77168 114.811 82.8125 0.00530658 0.00870498 2024-09-11T07:38:19.488 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:19.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:19 smithi104 ceph-mon[33507]: pgmap v522: 148 pgs: 148 active+clean; 4.3 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 73 MiB/s wr, 1.18k op/s 2024-09-11T07:38:19.534 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:19.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:19.534 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:19.635 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:19.675 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:19.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:19.677 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:19.777 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:19.815 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:19.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:19.817 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:19.917 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:19.954 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:19.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:19.955 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:20.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:38:19 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:38:20.056 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:20.102 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:20.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:20.146 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:20.231 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:38:20.231 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:38:20.247 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:20.268 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:38:20.268 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:38:20.277 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:20.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:20.277 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:20.300 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:38:20.300 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:38:20.342 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:38:20.343 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:38:20.374 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:38:20.374 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:38:20.378 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:20.407 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:20 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2762890459' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:38:20.407 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:38:20 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:38:20] "GET /metrics HTTP/1.1" 200 224281 "" "Prometheus/2.43.0" 2024-09-11T07:38:20.408 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:38:20.408 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:38:20.409 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:20 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2762890459' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:38:20.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:20.413 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:20.413 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:20.442 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:38:20.442 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:38:20.480 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:38:20.480 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:38:20.481 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 43 15 78388 78373 113.893 75.3125 0.0122923 0.00877511 2024-09-11T07:38:20.514 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:20.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:20 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2762890459' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:38:20.555 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:20.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:20.556 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:20.657 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:20.688 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:20.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:20.689 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:20.790 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:20.818 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:20.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:20.819 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:20.920 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:20.965 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:20.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:20.966 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:21.067 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:21.104 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:21.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:21.105 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:21.205 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:21.238 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:21.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:21.239 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:21.281 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is unset 2024-09-11T07:38:21.340 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:21.375 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:21.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:21.376 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:21.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:21 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:38:21.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:21 smithi022 ceph-mon[29462]: pgmap v523: 148 pgs: 148 active+clean; 4.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 109 MiB/s wr, 1.74k op/s 2024-09-11T07:38:21.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:21 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3015932608' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:38:21.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:21 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:38:21.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:21 smithi022 ceph-mon[37330]: pgmap v523: 148 pgs: 148 active+clean; 4.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 109 MiB/s wr, 1.74k op/s 2024-09-11T07:38:21.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:21 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3015932608' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:38:21.477 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:21.480 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 44 16 79734 79718 113.215 84.0625 0.0222754 0.00882758 2024-09-11T07:38:21.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:21.514 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:21.514 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:21.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:21 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:38:21.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:21 smithi104 ceph-mon[33507]: pgmap v523: 148 pgs: 148 active+clean; 4.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 109 MiB/s wr, 1.74k op/s 2024-09-11T07:38:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:21 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3015932608' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:38:21.616 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:21.659 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:21.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:21.660 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:21.761 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:21.795 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:21.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:21.796 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:21.896 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:21.935 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:21.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:21.936 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:22.037 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:22.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:22.071 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:22.071 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:22.172 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:22.212 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:22.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:22.213 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:22.314 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:22.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:22.345 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:22.345 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:22.446 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:22.472 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:22.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:22.473 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:22.481 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 45 16 81060 81044 112.54 82.875 0.0209992 0.00887919 2024-09-11T07:38:22.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:22 smithi104 ceph-mon[33507]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:38:22.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:22 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3015932608' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:38:22.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:22 smithi104 ceph-mon[33507]: osdmap e257: 8 total, 8 up, 8 in 2024-09-11T07:38:22.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:22 smithi104 ceph-mon[33507]: 3.1 scrub starts 2024-09-11T07:38:22.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:22 smithi104 ceph-mon[33507]: 3.1 scrub ok 2024-09-11T07:38:22.553 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:22 smithi022 ceph-mon[29462]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:38:22.553 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:22 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3015932608' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:38:22.553 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:22 smithi022 ceph-mon[29462]: osdmap e257: 8 total, 8 up, 8 in 2024-09-11T07:38:22.553 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:22 smithi022 ceph-mon[29462]: 3.1 scrub starts 2024-09-11T07:38:22.553 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:22 smithi022 ceph-mon[29462]: 3.1 scrub ok 2024-09-11T07:38:22.553 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:22 smithi022 ceph-mon[37330]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:38:22.554 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:22 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3015932608' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:38:22.554 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:22 smithi022 ceph-mon[37330]: osdmap e257: 8 total, 8 up, 8 in 2024-09-11T07:38:22.554 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:22 smithi022 ceph-mon[37330]: 3.1 scrub starts 2024-09-11T07:38:22.554 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:22 smithi022 ceph-mon[37330]: 3.1 scrub ok 2024-09-11T07:38:22.574 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:22.607 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:22.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:22.608 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:22.708 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:22.745 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:22.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:22.746 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:22.847 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:22.875 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:22.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:22.876 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:22.977 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:23.011 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:23.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:23.012 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:23.113 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:23.143 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:23.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:23.144 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:23.245 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:23.288 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:23.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:23.289 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:23.390 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:23.422 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:23.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:23.423 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:23.482 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 46 16 82386 82370 111.891 82.875 0.011829 0.00893193 2024-09-11T07:38:23.524 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:23.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:23 smithi104 ceph-mon[33507]: 5.c scrub starts 2024-09-11T07:38:23.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:23 smithi104 ceph-mon[33507]: 5.c scrub ok 2024-09-11T07:38:23.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:23 smithi104 ceph-mon[33507]: pgmap v525: 148 pgs: 148 active+clean; 4.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 83 MiB/s wr, 1.32k op/s 2024-09-11T07:38:23.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:23 smithi104 ceph-mon[33507]: 5.1f scrub starts 2024-09-11T07:38:23.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:23 smithi104 ceph-mon[33507]: 5.1f scrub ok 2024-09-11T07:38:23.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:38:23.557 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:23.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:23.558 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:23.659 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:23 smithi022 ceph-mon[29462]: 5.c scrub starts 2024-09-11T07:38:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:23 smithi022 ceph-mon[29462]: 5.c scrub ok 2024-09-11T07:38:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:23 smithi022 ceph-mon[29462]: pgmap v525: 148 pgs: 148 active+clean; 4.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 83 MiB/s wr, 1.32k op/s 2024-09-11T07:38:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:23 smithi022 ceph-mon[29462]: 5.1f scrub starts 2024-09-11T07:38:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:23 smithi022 ceph-mon[29462]: 5.1f scrub ok 2024-09-11T07:38:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:38:23.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:23 smithi022 ceph-mon[37330]: 5.c scrub starts 2024-09-11T07:38:23.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:23 smithi022 ceph-mon[37330]: 5.c scrub ok 2024-09-11T07:38:23.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:23 smithi022 ceph-mon[37330]: pgmap v525: 148 pgs: 148 active+clean; 4.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 83 MiB/s wr, 1.32k op/s 2024-09-11T07:38:23.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:23 smithi022 ceph-mon[37330]: 5.1f scrub starts 2024-09-11T07:38:23.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:23 smithi022 ceph-mon[37330]: 5.1f scrub ok 2024-09-11T07:38:23.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:38:23.698 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:23.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:23.700 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:23.800 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:23.840 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:23.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:23.841 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:23.942 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:24.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:24.029 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:24.030 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:24.130 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:24.161 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:24.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:24.162 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:24.262 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:24.287 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:24.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:24.288 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:24.388 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:24.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:24.419 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:24.419 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:24.430 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:38:24 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:38:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:38:24.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:24 smithi104 ceph-mon[33507]: 5.7 scrub starts 2024-09-11T07:38:24.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:24 smithi104 ceph-mon[33507]: 5.7 scrub ok 2024-09-11T07:38:24.482 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 47 15 83631 83616 111.168 77.875 0.0144414 0.00898966 2024-09-11T07:38:24.520 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:24.551 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:24.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:24.552 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:24.564 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:24 smithi022 ceph-mon[37330]: 5.7 scrub starts 2024-09-11T07:38:24.564 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:24 smithi022 ceph-mon[37330]: 5.7 scrub ok 2024-09-11T07:38:24.565 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:24 smithi022 ceph-mon[29462]: 5.7 scrub starts 2024-09-11T07:38:24.565 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:24 smithi022 ceph-mon[29462]: 5.7 scrub ok 2024-09-11T07:38:24.652 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:24.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:24.694 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:24.694 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:24.795 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:24.827 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:24.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:24.828 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:24.929 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:24.976 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:24.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:24.977 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:25.079 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:25.113 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:25.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:25.114 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:25.215 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:25.248 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:25.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:25.249 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:25.265 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:38:25.265 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:38:25.265 INFO:teuthology.orchestra.run.smithi022.stdout: "complaint_time": 30, 2024-09-11T07:38:25.265 INFO:teuthology.orchestra.run.smithi022.stdout: "num_blocked_ops": 0 2024-09-11T07:38:25.265 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:38:25.304 INFO:teuthology.orchestra.run.smithi022.stderr:set pool 9 pg_num to 26 2024-09-11T07:38:25.349 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:25.391 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:25.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:25.392 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:25.482 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 48 16 84910 84894 110.516 79.875 0.00961061 0.00904387 2024-09-11T07:38:25.493 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:25.521 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:25.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:25.522 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:25.581 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:25 smithi022 ceph-mon[29462]: 4.a scrub starts 2024-09-11T07:38:25.581 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:25 smithi022 ceph-mon[29462]: 4.a scrub ok 2024-09-11T07:38:25.581 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:25 smithi022 ceph-mon[29462]: 4.0 scrub starts 2024-09-11T07:38:25.582 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:25 smithi022 ceph-mon[29462]: 4.0 scrub ok 2024-09-11T07:38:25.582 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:25 smithi022 ceph-mon[29462]: 5.f scrub starts 2024-09-11T07:38:25.582 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:25 smithi022 ceph-mon[29462]: 5.f scrub ok 2024-09-11T07:38:25.582 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:25 smithi022 ceph-mon[29462]: pgmap v526: 148 pgs: 148 active+clean; 4.8 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-11T07:38:25.582 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:25 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2317435641' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"}]: dispatch 2024-09-11T07:38:25.582 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:25 smithi022 ceph-mon[37330]: 4.a scrub starts 2024-09-11T07:38:25.582 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:25 smithi022 ceph-mon[37330]: 4.a scrub ok 2024-09-11T07:38:25.582 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:25 smithi022 ceph-mon[37330]: 4.0 scrub starts 2024-09-11T07:38:25.582 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:25 smithi022 ceph-mon[37330]: 4.0 scrub ok 2024-09-11T07:38:25.582 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:25 smithi022 ceph-mon[37330]: 5.f scrub starts 2024-09-11T07:38:25.582 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:25 smithi022 ceph-mon[37330]: 5.f scrub ok 2024-09-11T07:38:25.582 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:25 smithi022 ceph-mon[37330]: pgmap v526: 148 pgs: 148 active+clean; 4.8 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-11T07:38:25.583 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:25 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2317435641' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"}]: dispatch 2024-09-11T07:38:25.623 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:25.661 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:25.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:25.662 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:25.763 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:25.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:25 smithi104 ceph-mon[33507]: 4.a scrub starts 2024-09-11T07:38:25.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:25 smithi104 ceph-mon[33507]: 4.a scrub ok 2024-09-11T07:38:25.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:25 smithi104 ceph-mon[33507]: 4.0 scrub starts 2024-09-11T07:38:25.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:25 smithi104 ceph-mon[33507]: 4.0 scrub ok 2024-09-11T07:38:25.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:25 smithi104 ceph-mon[33507]: 5.f scrub starts 2024-09-11T07:38:25.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:25 smithi104 ceph-mon[33507]: 5.f scrub ok 2024-09-11T07:38:25.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:25 smithi104 ceph-mon[33507]: pgmap v526: 148 pgs: 148 active+clean; 4.8 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-11T07:38:25.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:25 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2317435641' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"}]: dispatch 2024-09-11T07:38:25.796 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:25.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:25.797 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:25.898 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:25.931 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:25.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:25.932 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:25.944 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:38:26.032 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:26.060 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:26.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:26.061 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:26.162 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:26.198 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:26.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:26.199 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:26.300 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:26.328 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:26 smithi022 ceph-mon[29462]: 5.6 scrub starts 2024-09-11T07:38:26.329 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:26 smithi022 ceph-mon[29462]: 5.6 scrub ok 2024-09-11T07:38:26.329 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:26 smithi022 ceph-mon[29462]: 5.14 scrub starts 2024-09-11T07:38:26.329 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:26 smithi022 ceph-mon[29462]: 5.14 scrub ok 2024-09-11T07:38:26.329 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:26 smithi022 ceph-mon[29462]: 4.1f scrub starts 2024-09-11T07:38:26.329 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:26 smithi022 ceph-mon[29462]: 4.1f scrub ok 2024-09-11T07:38:26.329 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:26 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2317435641' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"}]': finished 2024-09-11T07:38:26.329 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:26 smithi022 ceph-mon[29462]: osdmap e258: 8 total, 8 up, 8 in 2024-09-11T07:38:26.329 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:26 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-11T07:38:26.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:26.344 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:26.344 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:26.445 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:26.472 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:26.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:26.473 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:26.482 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 49 16 85954 85938 109.592 65.25 0.0568388 0.00909164 2024-09-11T07:38:26.574 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:26.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:26.612 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:26.612 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:26.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:26 smithi022 ceph-mon[37330]: 5.6 scrub starts 2024-09-11T07:38:26.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:26 smithi022 ceph-mon[37330]: 5.6 scrub ok 2024-09-11T07:38:26.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:26 smithi022 ceph-mon[37330]: 5.14 scrub starts 2024-09-11T07:38:26.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:26 smithi022 ceph-mon[37330]: 5.14 scrub ok 2024-09-11T07:38:26.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:26 smithi022 ceph-mon[37330]: 4.1f scrub starts 2024-09-11T07:38:26.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:26 smithi022 ceph-mon[37330]: 4.1f scrub ok 2024-09-11T07:38:26.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:26 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2317435641' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"}]': finished 2024-09-11T07:38:26.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:26 smithi022 ceph-mon[37330]: osdmap e258: 8 total, 8 up, 8 in 2024-09-11T07:38:26.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:26 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-11T07:38:26.712 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:26.734 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:26.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:26.735 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:26.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:26 smithi104 ceph-mon[33507]: 5.6 scrub starts 2024-09-11T07:38:26.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:26 smithi104 ceph-mon[33507]: 5.6 scrub ok 2024-09-11T07:38:26.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:26 smithi104 ceph-mon[33507]: 5.14 scrub starts 2024-09-11T07:38:26.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:26 smithi104 ceph-mon[33507]: 5.14 scrub ok 2024-09-11T07:38:26.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:26 smithi104 ceph-mon[33507]: 4.1f scrub starts 2024-09-11T07:38:26.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:26 smithi104 ceph-mon[33507]: 4.1f scrub ok 2024-09-11T07:38:26.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:26 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2317435641' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"}]': finished 2024-09-11T07:38:26.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:26 smithi104 ceph-mon[33507]: osdmap e258: 8 total, 8 up, 8 in 2024-09-11T07:38:26.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:26 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-11T07:38:26.835 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:26.859 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:26.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:26.860 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:26.961 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:26.984 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:26.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:26.985 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:27.018 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T07:38:27.086 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:27.108 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:27.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:27.109 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:27.210 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:27.231 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:27.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:27.232 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:27.332 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:27.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:27.368 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:27.368 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:27.469 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:27.483 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 50 16 86159 86143 107.656 12.8125 0.00747089 0.00928434 2024-09-11T07:38:27.509 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:27.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:27.510 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:27.610 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:27.631 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:27.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:27.632 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:27.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:27 smithi022 ceph-mon[29462]: 6.4 scrub starts 2024-09-11T07:38:27.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:27 smithi022 ceph-mon[29462]: 6.4 scrub ok 2024-09-11T07:38:27.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:27 smithi022 ceph-mon[29462]: pgmap v528: 148 pgs: 148 active+clean; 5.1 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 104 MiB/s wr, 1.66k op/s 2024-09-11T07:38:27.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:27 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-11T07:38:27.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:27 smithi022 ceph-mon[29462]: osdmap e259: 8 total, 8 up, 8 in 2024-09-11T07:38:27.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:27 smithi022 ceph-mon[37330]: 6.4 scrub starts 2024-09-11T07:38:27.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:27 smithi022 ceph-mon[37330]: 6.4 scrub ok 2024-09-11T07:38:27.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:27 smithi022 ceph-mon[37330]: pgmap v528: 148 pgs: 148 active+clean; 5.1 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 104 MiB/s wr, 1.66k op/s 2024-09-11T07:38:27.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:27 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-11T07:38:27.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:27 smithi022 ceph-mon[37330]: osdmap e259: 8 total, 8 up, 8 in 2024-09-11T07:38:27.733 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:27.756 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:27.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:27.756 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:27.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:27 smithi104 ceph-mon[33507]: 6.4 scrub starts 2024-09-11T07:38:27.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:27 smithi104 ceph-mon[33507]: 6.4 scrub ok 2024-09-11T07:38:27.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:27 smithi104 ceph-mon[33507]: pgmap v528: 148 pgs: 148 active+clean; 5.1 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 104 MiB/s wr, 1.66k op/s 2024-09-11T07:38:27.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:27 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-11T07:38:27.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:27 smithi104 ceph-mon[33507]: osdmap e259: 8 total, 8 up, 8 in 2024-09-11T07:38:27.857 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:27.890 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:27.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:27.891 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:27.992 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:27.998 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-11T07:38:28.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:28.020 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:28.020 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:28.122 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:28.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:28.174 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:28.174 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:28.275 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:28.304 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:28.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:28.305 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:28.406 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:28.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:28.428 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:28.428 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:28.483 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 51 16 87257 87241 106.891 68.625 0.00199103 0.00933041 2024-09-11T07:38:28.528 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:28.549 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:28.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:28.550 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:28.648 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:28 smithi022 ceph-mon[29462]: 6.1d scrub starts 2024-09-11T07:38:28.648 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:28 smithi022 ceph-mon[29462]: 6.1d scrub ok 2024-09-11T07:38:28.648 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:28 smithi022 ceph-mon[29462]: 5.1b scrub starts 2024-09-11T07:38:28.648 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:28 smithi022 ceph-mon[29462]: 5.1b scrub ok 2024-09-11T07:38:28.648 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:28 smithi022 ceph-mon[29462]: osdmap e260: 8 total, 8 up, 8 in 2024-09-11T07:38:28.648 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:28 smithi022 ceph-mon[29462]: 2.1f scrub starts 2024-09-11T07:38:28.648 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:28 smithi022 ceph-mon[29462]: 2.1f scrub ok 2024-09-11T07:38:28.649 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:28 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-11T07:38:28.649 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:28 smithi022 ceph-mon[37330]: 6.1d scrub starts 2024-09-11T07:38:28.649 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:28 smithi022 ceph-mon[37330]: 6.1d scrub ok 2024-09-11T07:38:28.649 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:28 smithi022 ceph-mon[37330]: 5.1b scrub starts 2024-09-11T07:38:28.649 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:28 smithi022 ceph-mon[37330]: 5.1b scrub ok 2024-09-11T07:38:28.649 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:28 smithi022 ceph-mon[37330]: osdmap e260: 8 total, 8 up, 8 in 2024-09-11T07:38:28.649 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:28 smithi022 ceph-mon[37330]: 2.1f scrub starts 2024-09-11T07:38:28.649 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:28 smithi022 ceph-mon[37330]: 2.1f scrub ok 2024-09-11T07:38:28.649 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:28 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-11T07:38:28.651 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:28.671 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:28.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:28.672 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:28.720 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:38:28.748 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:38:28.773 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:28.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:28 smithi104 ceph-mon[33507]: 6.1d scrub starts 2024-09-11T07:38:28.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:28 smithi104 ceph-mon[33507]: 6.1d scrub ok 2024-09-11T07:38:28.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:28 smithi104 ceph-mon[33507]: 5.1b scrub starts 2024-09-11T07:38:28.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:28 smithi104 ceph-mon[33507]: 5.1b scrub ok 2024-09-11T07:38:28.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:28 smithi104 ceph-mon[33507]: osdmap e260: 8 total, 8 up, 8 in 2024-09-11T07:38:28.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:28 smithi104 ceph-mon[33507]: 2.1f scrub starts 2024-09-11T07:38:28.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:28 smithi104 ceph-mon[33507]: 2.1f scrub ok 2024-09-11T07:38:28.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:28 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-11T07:38:28.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:28.797 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:28.797 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:28.898 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:28.920 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:28.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:28.921 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:28.977 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:38:29.022 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:29.044 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:29.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:29.045 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:29.146 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:29.168 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:29.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:29.169 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:29.270 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:29.293 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:29.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:29.293 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:29.394 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:29.417 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:29.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:29.418 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:29.483 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 52 16 87257 87241 104.835 0 - 0.00933041 2024-09-11T07:38:29.519 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:29.541 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:29.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:29.542 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:29.634 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:29 smithi104 ceph-mon[33507]: 2.e scrub starts 2024-09-11T07:38:29.634 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:29 smithi104 ceph-mon[33507]: 2.e scrub ok 2024-09-11T07:38:29.635 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:29 smithi104 ceph-mon[33507]: 4.11 scrub starts 2024-09-11T07:38:29.635 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:29 smithi104 ceph-mon[33507]: 4.11 scrub ok 2024-09-11T07:38:29.635 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:29 smithi104 ceph-mon[33507]: pgmap v531: 158 pgs: 10 unknown, 148 active+clean; 5.1 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 852 B/s rd, 67 MiB/s wr, 1.08k op/s 2024-09-11T07:38:29.635 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:29 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-11T07:38:29.635 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:29 smithi104 ceph-mon[33507]: osdmap e261: 8 total, 8 up, 8 in 2024-09-11T07:38:29.635 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:29 smithi104 ceph-mon[33507]: osdmap e262: 8 total, 8 up, 8 in 2024-09-11T07:38:29.638 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:29 smithi022 ceph-mon[29462]: 2.e scrub starts 2024-09-11T07:38:29.638 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:29 smithi022 ceph-mon[29462]: 2.e scrub ok 2024-09-11T07:38:29.638 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:29 smithi022 ceph-mon[29462]: 4.11 scrub starts 2024-09-11T07:38:29.638 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:29 smithi022 ceph-mon[29462]: 4.11 scrub ok 2024-09-11T07:38:29.638 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:29 smithi022 ceph-mon[29462]: pgmap v531: 158 pgs: 10 unknown, 148 active+clean; 5.1 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 852 B/s rd, 67 MiB/s wr, 1.08k op/s 2024-09-11T07:38:29.638 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:29 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-11T07:38:29.638 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:29 smithi022 ceph-mon[29462]: osdmap e261: 8 total, 8 up, 8 in 2024-09-11T07:38:29.638 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:29 smithi022 ceph-mon[29462]: osdmap e262: 8 total, 8 up, 8 in 2024-09-11T07:38:29.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:29 smithi022 ceph-mon[37330]: 2.e scrub starts 2024-09-11T07:38:29.639 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:29 smithi022 ceph-mon[37330]: 2.e scrub ok 2024-09-11T07:38:29.639 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:29 smithi022 ceph-mon[37330]: 4.11 scrub starts 2024-09-11T07:38:29.639 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:29 smithi022 ceph-mon[37330]: 4.11 scrub ok 2024-09-11T07:38:29.639 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:29 smithi022 ceph-mon[37330]: pgmap v531: 158 pgs: 10 unknown, 148 active+clean; 5.1 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 852 B/s rd, 67 MiB/s wr, 1.08k op/s 2024-09-11T07:38:29.639 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:29 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-11T07:38:29.639 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:29 smithi022 ceph-mon[37330]: osdmap e261: 8 total, 8 up, 8 in 2024-09-11T07:38:29.639 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:29 smithi022 ceph-mon[37330]: osdmap e262: 8 total, 8 up, 8 in 2024-09-11T07:38:29.642 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:29.666 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:29.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:29.666 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:29.767 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:29.789 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:29.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:29.790 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:29.891 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:29.913 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:29.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:29.915 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:30.015 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:30.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:38:29 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:38:30.039 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:30.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:30.040 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:30.141 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:30.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:30.164 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:30.165 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:30.265 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:30.290 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:30.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:30.291 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:30.392 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:30.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:30.435 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:30.436 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:30.483 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 53 16 87290 87274 102.897 1.03125 0.00478252 0.00961522 2024-09-11T07:38:30.537 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:30.576 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:30.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:30.577 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:30.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:38:30 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:38:30] "GET /metrics HTTP/1.1" 200 224299 "" "Prometheus/2.43.0" 2024-09-11T07:38:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:30 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:38:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:30 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:38:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:30 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:38:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:30 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:38:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:30 smithi022 ceph-mon[29462]: osdmap e263: 8 total, 8 up, 8 in 2024-09-11T07:38:30.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:30 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:38:30.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:30 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:38:30.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:30 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:38:30.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:30 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:38:30.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:30 smithi022 ceph-mon[37330]: osdmap e263: 8 total, 8 up, 8 in 2024-09-11T07:38:30.678 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:30.712 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:30.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:30.713 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:30.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:30 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:38:30.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:30 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:38:30.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:30 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:38:30.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:30 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:38:30.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:30 smithi104 ceph-mon[33507]: osdmap e263: 8 total, 8 up, 8 in 2024-09-11T07:38:30.814 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:30.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:30.853 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:30.853 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:30.954 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:30.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:30.994 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:30.994 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:31.095 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:31.129 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:31.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:31.130 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:31.231 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:31.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:31.262 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:31.262 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:31.363 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:31.399 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:31.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:31.400 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:31.483 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 54 16 88564 88548 102.465 79.625 0.0147191 0.00975435 2024-09-11T07:38:31.501 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:31.537 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:31.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:31.538 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:31.639 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:31.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:38:31.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[29462]: 2.9 scrub starts 2024-09-11T07:38:31.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[29462]: 6.14 scrub starts 2024-09-11T07:38:31.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[29462]: 2.9 scrub ok 2024-09-11T07:38:31.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[29462]: 6.14 scrub ok 2024-09-11T07:38:31.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:38:31.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:38:31.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[29462]: pgmap v534: 158 pgs: 4 peering, 5 remapped+peering, 149 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 55 MiB/s wr, 877 op/s; 32 KiB/s, 0 objects/s recovering 2024-09-11T07:38:31.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[29462]: 9.10 starting backfill to osd.0 from (0'0,0'0] MAX to 260'5479 2024-09-11T07:38:31.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[29462]: 9.16 starting backfill to osd.4 from (0'0,0'0] MAX to 260'5566 2024-09-11T07:38:31.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[29462]: 9.12 starting backfill to osd.0 from (0'0,0'0] MAX to 260'5360 2024-09-11T07:38:31.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[29462]: 9.17 starting backfill to osd.0 from (0'0,0'0] MAX to 260'5442 2024-09-11T07:38:31.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[29462]: 9.18 starting backfill to osd.6 from (0'0,0'0] MAX to 260'5382 2024-09-11T07:38:31.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[29462]: 9.13 starting backfill to osd.5 from (0'0,0'0] MAX to 260'5345 2024-09-11T07:38:31.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[29462]: 9.18 starting backfill to osd.7 from (0'0,0'0] MAX to 260'5382 2024-09-11T07:38:31.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[29462]: 9.10 starting backfill to osd.4 from (0'0,0'0] MAX to 260'5479 2024-09-11T07:38:31.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[29462]: 9.17 starting backfill to osd.1 from (0'0,0'0] MAX to 260'5442 2024-09-11T07:38:31.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[29462]: 9.13 starting backfill to osd.6 from (0'0,0'0] MAX to 260'5345 2024-09-11T07:38:31.670 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[29462]: 9.12 starting backfill to osd.5 from (0'0,0'0] MAX to 260'5360 2024-09-11T07:38:31.670 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[29462]: osdmap e264: 8 total, 8 up, 8 in 2024-09-11T07:38:31.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:38:31.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[37330]: 2.9 scrub starts 2024-09-11T07:38:31.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[37330]: 6.14 scrub starts 2024-09-11T07:38:31.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[37330]: 2.9 scrub ok 2024-09-11T07:38:31.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[37330]: 6.14 scrub ok 2024-09-11T07:38:31.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:38:31.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:38:31.671 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[37330]: pgmap v534: 158 pgs: 4 peering, 5 remapped+peering, 149 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 55 MiB/s wr, 877 op/s; 32 KiB/s, 0 objects/s recovering 2024-09-11T07:38:31.671 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[37330]: 9.10 starting backfill to osd.0 from (0'0,0'0] MAX to 260'5479 2024-09-11T07:38:31.671 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[37330]: 9.16 starting backfill to osd.4 from (0'0,0'0] MAX to 260'5566 2024-09-11T07:38:31.671 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[37330]: 9.12 starting backfill to osd.0 from (0'0,0'0] MAX to 260'5360 2024-09-11T07:38:31.671 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[37330]: 9.17 starting backfill to osd.0 from (0'0,0'0] MAX to 260'5442 2024-09-11T07:38:31.671 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[37330]: 9.18 starting backfill to osd.6 from (0'0,0'0] MAX to 260'5382 2024-09-11T07:38:31.671 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[37330]: 9.13 starting backfill to osd.5 from (0'0,0'0] MAX to 260'5345 2024-09-11T07:38:31.671 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[37330]: 9.18 starting backfill to osd.7 from (0'0,0'0] MAX to 260'5382 2024-09-11T07:38:31.671 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[37330]: 9.10 starting backfill to osd.4 from (0'0,0'0] MAX to 260'5479 2024-09-11T07:38:31.671 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[37330]: 9.17 starting backfill to osd.1 from (0'0,0'0] MAX to 260'5442 2024-09-11T07:38:31.671 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[37330]: 9.13 starting backfill to osd.6 from (0'0,0'0] MAX to 260'5345 2024-09-11T07:38:31.671 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[37330]: 9.12 starting backfill to osd.5 from (0'0,0'0] MAX to 260'5360 2024-09-11T07:38:31.671 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:31 smithi022 ceph-mon[37330]: osdmap e264: 8 total, 8 up, 8 in 2024-09-11T07:38:31.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:31.685 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:31.685 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:31.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:31 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:38:31.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:31 smithi104 ceph-mon[33507]: 2.9 scrub starts 2024-09-11T07:38:31.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:31 smithi104 ceph-mon[33507]: 6.14 scrub starts 2024-09-11T07:38:31.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:31 smithi104 ceph-mon[33507]: 2.9 scrub ok 2024-09-11T07:38:31.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:31 smithi104 ceph-mon[33507]: 6.14 scrub ok 2024-09-11T07:38:31.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:31 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:38:31.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:31 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:38:31.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:31 smithi104 ceph-mon[33507]: pgmap v534: 158 pgs: 4 peering, 5 remapped+peering, 149 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 55 MiB/s wr, 877 op/s; 32 KiB/s, 0 objects/s recovering 2024-09-11T07:38:31.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:31 smithi104 ceph-mon[33507]: 9.10 starting backfill to osd.0 from (0'0,0'0] MAX to 260'5479 2024-09-11T07:38:31.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:31 smithi104 ceph-mon[33507]: 9.16 starting backfill to osd.4 from (0'0,0'0] MAX to 260'5566 2024-09-11T07:38:31.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:31 smithi104 ceph-mon[33507]: 9.12 starting backfill to osd.0 from (0'0,0'0] MAX to 260'5360 2024-09-11T07:38:31.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:31 smithi104 ceph-mon[33507]: 9.17 starting backfill to osd.0 from (0'0,0'0] MAX to 260'5442 2024-09-11T07:38:31.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:31 smithi104 ceph-mon[33507]: 9.18 starting backfill to osd.6 from (0'0,0'0] MAX to 260'5382 2024-09-11T07:38:31.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:31 smithi104 ceph-mon[33507]: 9.13 starting backfill to osd.5 from (0'0,0'0] MAX to 260'5345 2024-09-11T07:38:31.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:31 smithi104 ceph-mon[33507]: 9.18 starting backfill to osd.7 from (0'0,0'0] MAX to 260'5382 2024-09-11T07:38:31.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:31 smithi104 ceph-mon[33507]: 9.10 starting backfill to osd.4 from (0'0,0'0] MAX to 260'5479 2024-09-11T07:38:31.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:31 smithi104 ceph-mon[33507]: 9.17 starting backfill to osd.1 from (0'0,0'0] MAX to 260'5442 2024-09-11T07:38:31.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:31 smithi104 ceph-mon[33507]: 9.13 starting backfill to osd.6 from (0'0,0'0] MAX to 260'5345 2024-09-11T07:38:31.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:31 smithi104 ceph-mon[33507]: 9.12 starting backfill to osd.5 from (0'0,0'0] MAX to 260'5360 2024-09-11T07:38:31.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:31 smithi104 ceph-mon[33507]: osdmap e264: 8 total, 8 up, 8 in 2024-09-11T07:38:31.786 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:31.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:31.818 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:31.818 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:31.919 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:31.952 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:31.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:31.953 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:32.054 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:32.096 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:32.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:32.097 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:32.198 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:32.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:32.238 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:32.238 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:32.339 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:32.377 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:32.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:32.378 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:32.480 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:32.483 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 55 16 89867 89851 102.083 81.4375 0.0109988 0.00979046 2024-09-11T07:38:32.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:32.515 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:32.515 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:32.616 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:32.654 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:32.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:32.655 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:32.659 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:32 smithi022 ceph-mon[29462]: 9.10 starting backfill to osd.7 from (0'0,0'0] MAX to 260'5479 2024-09-11T07:38:32.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:32 smithi022 ceph-mon[29462]: 5.a scrub starts 2024-09-11T07:38:32.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:32 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:38:32.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:32 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:38:32.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:32 smithi022 ceph-mon[29462]: osdmap e265: 8 total, 8 up, 8 in 2024-09-11T07:38:32.660 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:32 smithi022 ceph-mon[37330]: 9.10 starting backfill to osd.7 from (0'0,0'0] MAX to 260'5479 2024-09-11T07:38:32.660 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:32 smithi022 ceph-mon[37330]: 5.a scrub starts 2024-09-11T07:38:32.660 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:32 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:38:32.660 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:32 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:38:32.660 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:32 smithi022 ceph-mon[37330]: osdmap e265: 8 total, 8 up, 8 in 2024-09-11T07:38:32.757 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:32.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:32 smithi104 ceph-mon[33507]: 9.10 starting backfill to osd.7 from (0'0,0'0] MAX to 260'5479 2024-09-11T07:38:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:32 smithi104 ceph-mon[33507]: 5.a scrub starts 2024-09-11T07:38:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:32 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:38:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:32 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:38:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:32 smithi104 ceph-mon[33507]: osdmap e265: 8 total, 8 up, 8 in 2024-09-11T07:38:32.794 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:32.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:32.795 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:32.896 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:32.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:32.936 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:32.936 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:33.037 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:33.065 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:33.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:33.066 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:33.167 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:33.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:33.209 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:33.209 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:33.310 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:33.339 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:33.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:33.341 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:33.441 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:33.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:33.476 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:33.476 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:33.483 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 56 16 90960 90944 101.48 68.3125 0.00733789 0.00984916 2024-09-11T07:38:33.511 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:33 smithi022 ceph-mon[29462]: pgmap v538: 158 pgs: 4 peering, 5 remapped+peering, 149 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.3 KiB/s rd, 57 MiB/s wr, 908 op/s; 33 KiB/s, 0 objects/s recovering 2024-09-11T07:38:33.511 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:33 smithi022 ceph-mon[37330]: pgmap v538: 158 pgs: 4 peering, 5 remapped+peering, 149 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.3 KiB/s rd, 57 MiB/s wr, 908 op/s; 33 KiB/s, 0 objects/s recovering 2024-09-11T07:38:33.577 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:33.623 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:33.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:33.624 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:33.677 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:38:33.678 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:38:33.678 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-11T07:38:33.678 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T07:38:33.678 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T07:38:33.678 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 config set filestore_inject_stall 3 2024-09-11T07:38:33.707 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:33 smithi104 ceph-mon[33507]: pgmap v538: 158 pgs: 4 peering, 5 remapped+peering, 149 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.3 KiB/s rd, 57 MiB/s wr, 908 op/s; 33 KiB/s, 0 objects/s recovering 2024-09-11T07:38:33.725 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:33.760 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:33.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:33.762 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:33.862 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:33.903 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:33.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:33.904 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:33.930 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:38:33.930 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 20, 2024-09-11T07:38:33.930 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 600, 2024-09-11T07:38:33.930 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:38:33.930 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.930 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:85939 9.4 9:2a2e89c7:::benchmark_data_smithi022_133943_object85938:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e258)", 2024-09-11T07:38:33.930 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:26.315621+0000", 2024-09-11T07:38:33.931 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 6.9597123190000003, 2024-09-11T07:38:33.931 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.246332512, 2024-09-11T07:38:33.931 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:38:33.931 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:38:33.931 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:38:33.931 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:38:33.931 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:38:33.931 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 85939 2024-09-11T07:38:33.931 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.931 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:38:33.931 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.931 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:38:33.932 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.315621+0000", 2024-09-11T07:38:33.932 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:33.932 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.932 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.932 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:38:33.932 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.315621+0000", 2024-09-11T07:38:33.932 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:33.932 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.932 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.932 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:38:33.932 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.315619+0000", 2024-09-11T07:38:33.933 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:38:33.933 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.933 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.933 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:38:33.933 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.315644+0000", 2024-09-11T07:38:33.933 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4423e-05 2024-09-11T07:38:33.933 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.933 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.933 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:38:33.933 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.315651+0000", 2024-09-11T07:38:33.933 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.8760000000000005e-06 2024-09-11T07:38:33.933 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.933 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.933 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:38:33.934 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.315656+0000", 2024-09-11T07:38:33.934 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.994e-06 2024-09-11T07:38:33.934 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.934 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.934 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:33.934 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.316852+0000", 2024-09-11T07:38:33.934 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0011958069999999999 2024-09-11T07:38:33.934 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.934 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.934 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:38:33.934 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.316959+0000", 2024-09-11T07:38:33.934 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000106331 2024-09-11T07:38:33.935 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.935 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.935 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,4", 2024-09-11T07:38:33.935 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.317086+0000", 2024-09-11T07:38:33.935 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012746799999999999 2024-09-11T07:38:33.935 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.935 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.935 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:33.935 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.324395+0000", 2024-09-11T07:38:33.935 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0073091609999999998 2024-09-11T07:38:33.935 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.935 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.936 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:38:33.936 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.324403+0000", 2024-09-11T07:38:33.936 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.8909999999999995e-06 2024-09-11T07:38:33.936 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.936 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.936 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:33.936 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.561941+0000", 2024-09-11T07:38:33.936 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.23753767200000001 2024-09-11T07:38:33.936 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.936 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.936 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:38:33.936 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.561954+0000", 2024-09-11T07:38:33.937 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2724e-05 2024-09-11T07:38:33.937 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:33.937 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:38:33.937 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:33.937 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.937 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.937 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:85943 9.4 9:27627e86:::benchmark_data_smithi022_133943_object85942:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e258)", 2024-09-11T07:38:33.937 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:26.316745+0000", 2024-09-11T07:38:33.937 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 6.9585885840000001, 2024-09-11T07:38:33.937 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.24524643600000001, 2024-09-11T07:38:33.937 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:38:33.937 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:38:33.938 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:38:33.938 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:38:33.938 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:38:33.938 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 85943 2024-09-11T07:38:33.938 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.938 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:38:33.938 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.938 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:38:33.938 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.316745+0000", 2024-09-11T07:38:33.938 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:33.938 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.938 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.938 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:38:33.939 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.316745+0000", 2024-09-11T07:38:33.939 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:33.939 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.939 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.939 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:38:33.939 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.316744+0000", 2024-09-11T07:38:33.939 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:38:33.939 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.939 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.939 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:38:33.939 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.316767+0000", 2024-09-11T07:38:33.939 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3521000000000001e-05 2024-09-11T07:38:33.939 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.940 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.940 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:38:33.940 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.316782+0000", 2024-09-11T07:38:33.940 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5197999999999999e-05 2024-09-11T07:38:33.940 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.940 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.940 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:38:33.940 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.316788+0000", 2024-09-11T07:38:33.940 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.5849999999999999e-06 2024-09-11T07:38:33.940 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.940 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.940 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:33.941 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.319845+0000", 2024-09-11T07:38:33.941 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0030569780000000001 2024-09-11T07:38:33.941 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.941 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.941 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:38:33.941 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.320154+0000", 2024-09-11T07:38:33.941 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000309652 2024-09-11T07:38:33.941 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.941 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.941 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,4", 2024-09-11T07:38:33.941 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.320308+0000", 2024-09-11T07:38:33.941 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000153385 2024-09-11T07:38:33.942 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.942 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.942 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:38:33.942 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.327061+0000", 2024-09-11T07:38:33.942 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.006753434 2024-09-11T07:38:33.942 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.942 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.942 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:33.942 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.561985+0000", 2024-09-11T07:38:33.942 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.234923785 2024-09-11T07:38:33.942 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.942 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.942 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:38:33.943 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.561991+0000", 2024-09-11T07:38:33.943 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0990000000000004e-06 2024-09-11T07:38:33.943 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:33.943 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:38:33.943 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:33.943 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.943 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.943 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:85950 9.4 9:22f2c04f:::benchmark_data_smithi022_133943_object85949:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e258)", 2024-09-11T07:38:33.943 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:26.329634+0000", 2024-09-11T07:38:33.943 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 6.9456995890000002, 2024-09-11T07:38:33.943 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.23238773700000001, 2024-09-11T07:38:33.944 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:38:33.944 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:38:33.944 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:38:33.944 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:38:33.944 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:38:33.944 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 85950 2024-09-11T07:38:33.944 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.944 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:38:33.944 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.944 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:38:33.944 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.329634+0000", 2024-09-11T07:38:33.944 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:33.945 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.945 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.945 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:38:33.945 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.329634+0000", 2024-09-11T07:38:33.945 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:33.945 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.945 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.945 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:38:33.945 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.329631+0000", 2024-09-11T07:38:33.945 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:38:33.945 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.945 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.946 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:38:33.946 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.329662+0000", 2024-09-11T07:38:33.946 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.1368000000000002e-05 2024-09-11T07:38:33.946 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.946 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.946 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:38:33.946 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.329676+0000", 2024-09-11T07:38:33.946 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3857e-05 2024-09-11T07:38:33.946 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.946 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.946 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:38:33.946 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.329684+0000", 2024-09-11T07:38:33.947 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5220000000000001e-06 2024-09-11T07:38:33.947 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.947 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.947 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:33.947 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.329751+0000", 2024-09-11T07:38:33.947 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7501000000000003e-05 2024-09-11T07:38:33.947 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.947 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.947 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:38:33.947 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.329902+0000", 2024-09-11T07:38:33.947 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015110500000000001 2024-09-11T07:38:33.947 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.947 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.947 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,4", 2024-09-11T07:38:33.948 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.330067+0000", 2024-09-11T07:38:33.948 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00016449499999999999 2024-09-11T07:38:33.948 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.948 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.948 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:38:33.948 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.334684+0000", 2024-09-11T07:38:33.948 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0046172690000000002 2024-09-11T07:38:33.948 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.948 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.948 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:33.948 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.562015+0000", 2024-09-11T07:38:33.948 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.227331492 2024-09-11T07:38:33.949 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.949 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.949 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:38:33.949 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.562021+0000", 2024-09-11T07:38:33.949 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0630000000000002e-06 2024-09-11T07:38:33.949 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:33.949 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:38:33.949 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:33.949 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.949 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.949 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:85939 9.14 9:2a2e89c7:::benchmark_data_smithi022_133943_object85938:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e259)", 2024-09-11T07:38:33.949 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:26.333855+0000", 2024-09-11T07:38:33.949 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 6.9414782060000002, 2024-09-11T07:38:33.949 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.058456533, 2024-09-11T07:38:33.950 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:38:33.950 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:38:33.950 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:38:33.950 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:38:33.950 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:38:33.950 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 85939 2024-09-11T07:38:33.950 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.950 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:38:33.950 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.950 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:38:33.950 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333855+0000", 2024-09-11T07:38:33.950 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:33.951 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.951 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.951 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:38:33.951 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333855+0000", 2024-09-11T07:38:33.951 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:33.951 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.951 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.951 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:38:33.951 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333852+0000", 2024-09-11T07:38:33.951 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:38:33.951 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.951 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.951 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:38:33.952 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333883+0000", 2024-09-11T07:38:33.952 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0558000000000001e-05 2024-09-11T07:38:33.952 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.952 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.952 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:38:33.952 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333898+0000", 2024-09-11T07:38:33.952 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5258e-05 2024-09-11T07:38:33.952 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.952 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.952 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:38:33.952 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333906+0000", 2024-09-11T07:38:33.952 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.892e-06 2024-09-11T07:38:33.953 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.953 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.953 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:33.953 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.562799+0000", 2024-09-11T07:38:33.953 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.228892701 2024-09-11T07:38:33.953 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.953 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.953 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:38:33.953 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.562805+0000", 2024-09-11T07:38:33.953 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0310000000000004e-06 2024-09-11T07:38:33.953 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.953 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.953 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:33.953 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.392245+0000", 2024-09-11T07:38:33.954 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.829440124 2024-09-11T07:38:33.954 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.954 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.954 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:38:33.954 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.392312+0000", 2024-09-11T07:38:33.954 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.6910999999999997e-05 2024-09-11T07:38:33.954 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:33.954 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:38:33.954 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:33.954 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.954 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.954 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:85943 9.4 9:27627e86:::benchmark_data_smithi022_133943_object85942:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e259)", 2024-09-11T07:38:33.954 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:26.333925+0000", 2024-09-11T07:38:33.955 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 6.9414081330000004, 2024-09-11T07:38:33.955 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.04661077, 2024-09-11T07:38:33.955 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:38:33.955 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:38:33.955 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:38:33.955 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:38:33.955 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:38:33.955 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 85943 2024-09-11T07:38:33.955 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.955 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:38:33.955 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.956 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:38:33.956 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333925+0000", 2024-09-11T07:38:33.956 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:33.956 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.956 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.956 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:38:33.956 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333925+0000", 2024-09-11T07:38:33.956 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:33.956 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.956 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.956 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:38:33.956 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333924+0000", 2024-09-11T07:38:33.956 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:38:33.957 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.957 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.957 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:38:33.957 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333948+0000", 2024-09-11T07:38:33.957 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3336000000000002e-05 2024-09-11T07:38:33.957 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.957 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.957 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:38:33.957 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333955+0000", 2024-09-11T07:38:33.957 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.6850000000000001e-06 2024-09-11T07:38:33.957 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.957 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.958 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:38:33.958 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333960+0000", 2024-09-11T07:38:33.958 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7119999999999999e-06 2024-09-11T07:38:33.958 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.958 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.958 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:33.958 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.334072+0000", 2024-09-11T07:38:33.958 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011152799999999999 2024-09-11T07:38:33.958 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.958 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.958 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op must wait for map", 2024-09-11T07:38:33.958 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.334077+0000", 2024-09-11T07:38:33.958 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8919999999999997e-06 2024-09-11T07:38:33.959 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.959 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.959 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:33.959 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.562040+0000", 2024-09-11T07:38:33.959 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.227962259 2024-09-11T07:38:33.959 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.959 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.959 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:38:33.959 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.562047+0000", 2024-09-11T07:38:33.959 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.537e-06 2024-09-11T07:38:33.959 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.959 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.960 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:33.960 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.375671+0000", 2024-09-11T07:38:33.960 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.81362374599999998 2024-09-11T07:38:33.960 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.960 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.960 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for degraded object", 2024-09-11T07:38:33.960 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.376570+0000", 2024-09-11T07:38:33.960 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00089861200000000002 2024-09-11T07:38:33.960 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.960 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.960 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:33.960 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.380412+0000", 2024-09-11T07:38:33.961 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.003842322 2024-09-11T07:38:33.961 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.961 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.961 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:38:33.961 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.380536+0000", 2024-09-11T07:38:33.961 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000123991 2024-09-11T07:38:33.961 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:33.961 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:38:33.961 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:33.961 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.961 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.961 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:85950 9.4 9:22f2c04f:::benchmark_data_smithi022_133943_object85949:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e259)", 2024-09-11T07:38:33.962 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:26.333965+0000", 2024-09-11T07:38:33.962 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 6.9413680859999998, 2024-09-11T07:38:33.962 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0552972730000001, 2024-09-11T07:38:33.962 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:38:33.962 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:38:33.962 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:38:33.962 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:38:33.962 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:38:33.962 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 85950 2024-09-11T07:38:33.962 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.962 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:38:33.962 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.962 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:38:33.963 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333965+0000", 2024-09-11T07:38:33.963 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:33.963 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.963 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.963 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:38:33.963 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333965+0000", 2024-09-11T07:38:33.963 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:33.963 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.963 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.963 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:38:33.963 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333965+0000", 2024-09-11T07:38:33.963 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:38:33.964 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.964 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.964 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:38:33.964 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333987+0000", 2024-09-11T07:38:33.964 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1985e-05 2024-09-11T07:38:33.964 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.964 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.964 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:38:33.964 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333993+0000", 2024-09-11T07:38:33.964 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7150000000000001e-06 2024-09-11T07:38:33.964 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.964 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.964 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:38:33.965 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333998+0000", 2024-09-11T07:38:33.965 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3649999999999997e-06 2024-09-11T07:38:33.965 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.965 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.965 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:33.965 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.334119+0000", 2024-09-11T07:38:33.965 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012089 2024-09-11T07:38:33.965 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.965 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.965 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting_for_map not empty", 2024-09-11T07:38:33.965 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.334123+0000", 2024-09-11T07:38:33.965 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1949999999999996e-06 2024-09-11T07:38:33.965 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.966 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.966 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:33.966 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.562063+0000", 2024-09-11T07:38:33.966 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.22794028499999999 2024-09-11T07:38:33.966 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.966 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.966 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:38:33.966 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.562068+0000", 2024-09-11T07:38:33.966 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1669999999999998e-06 2024-09-11T07:38:33.966 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.966 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.966 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:33.966 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.376594+0000", 2024-09-11T07:38:33.966 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.81452606699999996 2024-09-11T07:38:33.967 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.967 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.967 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for degraded object", 2024-09-11T07:38:33.967 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.379280+0000", 2024-09-11T07:38:33.967 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0026856340000000001 2024-09-11T07:38:33.967 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.967 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.967 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:33.967 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.389179+0000", 2024-09-11T07:38:33.967 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0098992579999999993 2024-09-11T07:38:33.967 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.967 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.967 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:38:33.967 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.389263+0000", 2024-09-11T07:38:33.968 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3319000000000006e-05 2024-09-11T07:38:33.968 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:33.968 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:38:33.968 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:33.968 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.968 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.968 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:85956 9.10 9:0bcec34e:::benchmark_data_smithi022_133943_object85955:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e259)", 2024-09-11T07:38:33.968 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:26.613254+0000", 2024-09-11T07:38:33.968 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 6.6620789689999995, 2024-09-11T07:38:33.968 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.76771919700000002, 2024-09-11T07:38:33.968 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:38:33.968 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:38:33.969 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:38:33.969 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:38:33.969 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:38:33.969 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 85956 2024-09-11T07:38:33.969 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.969 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:38:33.969 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.969 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:38:33.969 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.613254+0000", 2024-09-11T07:38:33.969 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:33.969 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.969 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.969 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:38:33.969 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.613254+0000", 2024-09-11T07:38:33.970 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:33.970 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.970 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.970 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:38:33.970 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.613251+0000", 2024-09-11T07:38:33.970 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999967 2024-09-11T07:38:33.970 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.970 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.970 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:38:33.970 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.613286+0000", 2024-09-11T07:38:33.970 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4677e-05 2024-09-11T07:38:33.970 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.970 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.970 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:38:33.970 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.613303+0000", 2024-09-11T07:38:33.971 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.7679000000000001e-05 2024-09-11T07:38:33.971 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.971 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.971 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:38:33.971 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.613312+0000", 2024-09-11T07:38:33.971 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.9409999999999996e-06 2024-09-11T07:38:33.971 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.971 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.971 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:33.971 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.613380+0000", 2024-09-11T07:38:33.971 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7385000000000006e-05 2024-09-11T07:38:33.971 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.971 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.971 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:38:33.972 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.613388+0000", 2024-09-11T07:38:33.972 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.4700000000000002e-06 2024-09-11T07:38:33.972 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.972 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.972 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:33.972 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.373573+0000", 2024-09-11T07:38:33.972 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.76018452000000003 2024-09-11T07:38:33.972 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.972 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.972 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:38:33.972 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.373717+0000", 2024-09-11T07:38:33.972 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014447700000000001 2024-09-11T07:38:33.972 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.972 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.973 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:38:33.973 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.374003+0000", 2024-09-11T07:38:33.973 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00028560800000000002 2024-09-11T07:38:33.973 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.973 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.973 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:38:33.973 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.376306+0000", 2024-09-11T07:38:33.973 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0023028010000000002 2024-09-11T07:38:33.973 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.973 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.973 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:33.973 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.380861+0000", 2024-09-11T07:38:33.973 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0045556930000000004 2024-09-11T07:38:33.974 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.974 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.974 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:33.974 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.380890+0000", 2024-09-11T07:38:33.974 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8954000000000001e-05 2024-09-11T07:38:33.974 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.974 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.974 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:38:33.974 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.380944+0000", 2024-09-11T07:38:33.974 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3301999999999999e-05 2024-09-11T07:38:33.974 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.974 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.974 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:38:33.974 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.380974+0000", 2024-09-11T07:38:33.975 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.9918000000000001e-05 2024-09-11T07:38:33.975 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:33.975 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:38:33.975 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:33.975 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.975 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.975 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:86219 9.4 9:20d92912:::benchmark_data_smithi022_133943_object86218:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:38:33.975 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:27.527573+0000", 2024-09-11T07:38:33.975 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 5.7477603410000002, 2024-09-11T07:38:33.975 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.24278649199999999, 2024-09-11T07:38:33.975 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:38:33.975 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:38:33.975 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:38:33.975 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:38:33.976 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:38:33.976 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 86219 2024-09-11T07:38:33.976 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.976 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:38:33.976 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.976 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:38:33.976 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.527573+0000", 2024-09-11T07:38:33.976 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:33.976 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.976 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.976 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:38:33.976 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.527573+0000", 2024-09-11T07:38:33.976 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:33.976 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.977 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.977 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:38:33.977 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.527571+0000", 2024-09-11T07:38:33.977 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:38:33.977 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.977 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.977 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:38:33.977 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.527664+0000", 2024-09-11T07:38:33.977 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.3212000000000001e-05 2024-09-11T07:38:33.977 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.977 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.977 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:38:33.977 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.527676+0000", 2024-09-11T07:38:33.977 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.183e-05 2024-09-11T07:38:33.978 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.978 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.978 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:38:33.978 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.527683+0000", 2024-09-11T07:38:33.978 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.8600000000000004e-06 2024-09-11T07:38:33.978 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.978 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.978 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:33.978 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.527751+0000", 2024-09-11T07:38:33.978 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.8007000000000001e-05 2024-09-11T07:38:33.978 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.978 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.978 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:38:33.978 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.527875+0000", 2024-09-11T07:38:33.979 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012437 2024-09-11T07:38:33.979 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.979 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.979 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,4", 2024-09-11T07:38:33.979 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.528233+0000", 2024-09-11T07:38:33.979 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00035774599999999999 2024-09-11T07:38:33.979 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.979 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.979 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:33.979 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.532258+0000", 2024-09-11T07:38:33.979 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0040247360000000001 2024-09-11T07:38:33.979 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.979 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.979 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:38:33.980 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.532475+0000", 2024-09-11T07:38:33.980 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000217087 2024-09-11T07:38:33.980 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.980 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.980 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:33.980 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.770264+0000", 2024-09-11T07:38:33.980 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.23778978000000001 2024-09-11T07:38:33.980 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.980 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.980 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:38:33.980 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.770319+0000", 2024-09-11T07:38:33.980 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.4707999999999999e-05 2024-09-11T07:38:33.981 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.981 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.981 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:38:33.981 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.770359+0000", 2024-09-11T07:38:33.981 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0290000000000002e-05 2024-09-11T07:38:33.981 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:33.981 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:38:33.981 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:33.981 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.981 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.981 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.15504.0:86227 9.e e260/247 9:74b4edf6:::benchmark_data_smithi022_133943_object86226:head v 260'5379, mlcod=260'5377)", 2024-09-11T07:38:33.981 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:27.530626+0000", 2024-09-11T07:38:33.981 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 5.7447073529999999, 2024-09-11T07:38:33.982 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.18189477100000001, 2024-09-11T07:38:33.982 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:38:33.982 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:38:33.982 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:38:33.982 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.982 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:38:33.982 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.530626+0000", 2024-09-11T07:38:33.982 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:33.982 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.982 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.982 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:38:33.982 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.530626+0000", 2024-09-11T07:38:33.982 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:33.982 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.983 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.983 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:38:33.983 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.530625+0000", 2024-09-11T07:38:33.983 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:38:33.983 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.983 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.983 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:38:33.983 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.530697+0000", 2024-09-11T07:38:33.983 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1501000000000005e-05 2024-09-11T07:38:33.983 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.983 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.983 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:38:33.983 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.530705+0000", 2024-09-11T07:38:33.983 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3389999999999993e-06 2024-09-11T07:38:33.984 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.984 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.984 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:38:33.984 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.530710+0000", 2024-09-11T07:38:33.984 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9219999999999996e-06 2024-09-11T07:38:33.984 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.984 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.984 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:33.984 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.531793+0000", 2024-09-11T07:38:33.984 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0010826639999999999 2024-09-11T07:38:33.984 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.984 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.984 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:38:33.984 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.531811+0000", 2024-09-11T07:38:33.985 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.7946000000000001e-05 2024-09-11T07:38:33.985 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.985 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.985 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:38:33.985 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.712483+0000", 2024-09-11T07:38:33.985 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.180672676 2024-09-11T07:38:33.985 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.985 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.985 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:38:33.985 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.712521+0000", 2024-09-11T07:38:33.985 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7557e-05 2024-09-11T07:38:33.985 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:33.985 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:38:33.985 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:33.986 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.986 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.986 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.15504.0:86233 9.e e260/247 9:746045db:::benchmark_data_smithi022_133943_object86232:head v 260'5380, mlcod=260'5377)", 2024-09-11T07:38:33.986 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:27.533548+0000", 2024-09-11T07:38:33.986 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 5.7417852040000001, 2024-09-11T07:38:33.986 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.17906865299999999, 2024-09-11T07:38:33.986 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:38:33.986 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:38:33.986 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:38:33.986 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.986 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:38:33.986 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.533548+0000", 2024-09-11T07:38:33.986 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:33.986 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.986 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.987 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:38:33.987 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.533548+0000", 2024-09-11T07:38:33.987 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:33.987 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.987 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.987 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:38:33.987 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.533547+0000", 2024-09-11T07:38:33.987 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:38:33.987 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.987 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.987 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:38:33.987 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.533577+0000", 2024-09-11T07:38:33.987 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.9759000000000001e-05 2024-09-11T07:38:33.987 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.987 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.988 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:38:33.988 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.533587+0000", 2024-09-11T07:38:33.988 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0626e-05 2024-09-11T07:38:33.988 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.988 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.988 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:38:33.988 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.533593+0000", 2024-09-11T07:38:33.988 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.48e-06 2024-09-11T07:38:33.988 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.988 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.988 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:33.988 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.534933+0000", 2024-09-11T07:38:33.988 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0013407650000000001 2024-09-11T07:38:33.989 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.989 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.989 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:38:33.989 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.534946+0000", 2024-09-11T07:38:33.989 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2473999999999999e-05 2024-09-11T07:38:33.989 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.989 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.989 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:38:33.989 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.712560+0000", 2024-09-11T07:38:33.989 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.177614567 2024-09-11T07:38:33.989 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.989 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.989 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:38:33.990 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.712617+0000", 2024-09-11T07:38:33.990 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6271999999999999e-05 2024-09-11T07:38:33.990 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:33.990 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:38:33.990 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:33.990 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.990 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.990 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:86239 9.4 9:20fc3301:::benchmark_data_smithi022_133943_object86238:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:38:33.990 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:27.536368+0000", 2024-09-11T07:38:33.990 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 5.7389652230000001, 2024-09-11T07:38:33.990 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.237280292, 2024-09-11T07:38:33.990 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:38:33.990 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:38:33.990 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:38:33.991 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:38:33.991 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:38:33.991 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 86239 2024-09-11T07:38:33.991 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.991 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:38:33.991 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.991 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:38:33.991 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536368+0000", 2024-09-11T07:38:33.991 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:33.991 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.991 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.991 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:38:33.991 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536368+0000", 2024-09-11T07:38:33.991 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:33.992 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.992 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.992 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:38:33.992 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536357+0000", 2024-09-11T07:38:33.992 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999989 2024-09-11T07:38:33.992 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.992 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.992 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:38:33.992 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536419+0000", 2024-09-11T07:38:33.992 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1382000000000003e-05 2024-09-11T07:38:33.992 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.992 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.992 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:38:33.992 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536431+0000", 2024-09-11T07:38:33.992 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2183e-05 2024-09-11T07:38:33.993 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.993 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.993 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:38:33.993 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536437+0000", 2024-09-11T07:38:33.993 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7180000000000001e-06 2024-09-11T07:38:33.993 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.993 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.993 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:33.993 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536640+0000", 2024-09-11T07:38:33.993 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00020332299999999999 2024-09-11T07:38:33.993 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.993 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.994 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:38:33.994 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.712490+0000", 2024-09-11T07:38:33.994 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.175849852 2024-09-11T07:38:33.994 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.994 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.994 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,4", 2024-09-11T07:38:33.994 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.712882+0000", 2024-09-11T07:38:33.994 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00039207499999999998 2024-09-11T07:38:33.994 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.994 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.994 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:38:33.994 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.715809+0000", 2024-09-11T07:38:33.994 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0029270530000000002 2024-09-11T07:38:33.994 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.994 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.995 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:33.995 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.715927+0000", 2024-09-11T07:38:33.995 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011765999999999999 2024-09-11T07:38:33.995 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.995 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.995 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:33.995 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.773553+0000", 2024-09-11T07:38:33.995 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.057626068000000003 2024-09-11T07:38:33.995 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.995 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.995 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:38:33.995 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.773603+0000", 2024-09-11T07:38:33.995 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.0757000000000002e-05 2024-09-11T07:38:33.996 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.996 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.996 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:38:33.996 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.773648+0000", 2024-09-11T07:38:33.996 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5009000000000002e-05 2024-09-11T07:38:33.996 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:33.996 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:38:33.996 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:33.996 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.996 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.996 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:86240 9.10 9:08d9731a:::benchmark_data_smithi022_133943_object86239:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:38:33.996 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:27.536452+0000", 2024-09-11T07:38:33.996 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 5.7388812439999999, 2024-09-11T07:38:33.996 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.17924583699999999, 2024-09-11T07:38:33.997 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:38:33.997 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:38:33.997 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:38:33.997 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:38:33.997 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:38:33.997 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 86240 2024-09-11T07:38:33.997 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.997 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:38:33.997 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.997 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:38:33.997 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536452+0000", 2024-09-11T07:38:33.997 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:33.997 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.997 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.998 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:38:33.998 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536452+0000", 2024-09-11T07:38:33.998 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:33.998 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.998 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.998 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:38:33.998 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536451+0000", 2024-09-11T07:38:33.998 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:38:33.998 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.998 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.998 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:38:33.998 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536473+0000", 2024-09-11T07:38:33.998 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1780000000000002e-05 2024-09-11T07:38:33.998 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.999 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.999 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:38:33.999 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536480+0000", 2024-09-11T07:38:33.999 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7270000000000001e-06 2024-09-11T07:38:33.999 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.999 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.999 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:38:33.999 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536484+0000", 2024-09-11T07:38:33.999 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5750000000000002e-06 2024-09-11T07:38:33.999 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:33.999 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:33.999 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:33.999 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536533+0000", 2024-09-11T07:38:33.999 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8683000000000001e-05 2024-09-11T07:38:34.000 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.000 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.000 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:38:34.000 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536654+0000", 2024-09-11T07:38:34.000 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012094100000000001 2024-09-11T07:38:34.000 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.000 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.000 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:38:34.000 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.712535+0000", 2024-09-11T07:38:34.000 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.175881438 2024-09-11T07:38:34.000 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.000 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.000 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:38:34.001 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.715519+0000", 2024-09-11T07:38:34.001 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0029831580000000001 2024-09-11T07:38:34.001 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.001 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.001 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.001 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.715591+0000", 2024-09-11T07:38:34.001 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.2584999999999997e-05 2024-09-11T07:38:34.001 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.001 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.001 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.001 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.715625+0000", 2024-09-11T07:38:34.001 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3492999999999999e-05 2024-09-11T07:38:34.001 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.002 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.002 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:38:34.002 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.715672+0000", 2024-09-11T07:38:34.002 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6848999999999998e-05 2024-09-11T07:38:34.002 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.002 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.002 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:38:34.002 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.715698+0000", 2024-09-11T07:38:34.002 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.6339e-05 2024-09-11T07:38:34.002 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:34.002 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:38:34.002 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:34.002 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.003 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.003 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:86243 9.14 9:2ca0fb1c:::benchmark_data_smithi022_133943_object86242:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:38:34.003 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:27.536553+0000", 2024-09-11T07:38:34.003 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 5.7387803990000004, 2024-09-11T07:38:34.003 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.23395485899999999, 2024-09-11T07:38:34.003 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:38:34.003 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:38:34.003 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:38:34.003 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:38:34.003 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:38:34.003 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 86243 2024-09-11T07:38:34.003 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.003 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:38:34.003 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.003 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:38:34.004 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536553+0000", 2024-09-11T07:38:34.004 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:34.004 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.004 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.004 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:38:34.004 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536553+0000", 2024-09-11T07:38:34.004 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:34.004 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.004 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.004 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:38:34.004 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536551+0000", 2024-09-11T07:38:34.004 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:38:34.004 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.005 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.005 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:38:34.005 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536572+0000", 2024-09-11T07:38:34.005 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0335000000000001e-05 2024-09-11T07:38:34.005 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.005 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.005 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:38:34.005 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536581+0000", 2024-09-11T07:38:34.005 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.0149999999999992e-06 2024-09-11T07:38:34.005 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.005 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.005 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:38:34.005 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536585+0000", 2024-09-11T07:38:34.006 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3610000000000003e-06 2024-09-11T07:38:34.006 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.006 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.006 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:34.006 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536657+0000", 2024-09-11T07:38:34.006 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1688999999999998e-05 2024-09-11T07:38:34.006 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.006 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.006 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:38:34.006 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536785+0000", 2024-09-11T07:38:34.006 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012829700000000001 2024-09-11T07:38:34.006 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.006 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.007 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,4", 2024-09-11T07:38:34.007 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.712682+0000", 2024-09-11T07:38:34.007 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.17589640000000001 2024-09-11T07:38:34.007 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.007 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.007 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.007 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.714522+0000", 2024-09-11T07:38:34.007 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001840455 2024-09-11T07:38:34.008 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.008 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.008 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:38:34.008 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.714804+0000", 2024-09-11T07:38:34.008 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00028165599999999997 2024-09-11T07:38:34.008 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.008 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.008 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.008 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.770440+0000", 2024-09-11T07:38:34.008 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.055636471 2024-09-11T07:38:34.008 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.008 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.008 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:38:34.009 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.770477+0000", 2024-09-11T07:38:34.009 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6569000000000002e-05 2024-09-11T07:38:34.009 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.009 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.009 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:38:34.009 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.770508+0000", 2024-09-11T07:38:34.009 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.1087000000000002e-05 2024-09-11T07:38:34.009 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:34.009 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:38:34.009 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:34.009 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.009 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.009 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.15504.0:86238 9.e e260/247 9:76c6605d:::benchmark_data_smithi022_133943_object86237:head v 260'5381, mlcod=260'5377)", 2024-09-11T07:38:34.010 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:27.536652+0000", 2024-09-11T07:38:34.010 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 5.7386809589999999, 2024-09-11T07:38:34.010 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.17920635300000001, 2024-09-11T07:38:34.010 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:38:34.010 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:38:34.010 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:38:34.010 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.010 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:38:34.010 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536652+0000", 2024-09-11T07:38:34.010 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:34.010 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.010 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.010 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:38:34.011 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536652+0000", 2024-09-11T07:38:34.011 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:34.011 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.011 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.011 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:38:34.011 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536651+0000", 2024-09-11T07:38:34.011 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:38:34.011 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.011 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.011 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:38:34.011 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536709+0000", 2024-09-11T07:38:34.011 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8118999999999997e-05 2024-09-11T07:38:34.011 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.012 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.012 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:38:34.012 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536745+0000", 2024-09-11T07:38:34.012 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6145e-05 2024-09-11T07:38:34.012 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.012 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.012 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:38:34.012 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.536751+0000", 2024-09-11T07:38:34.012 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3709999999999999e-06 2024-09-11T07:38:34.012 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.012 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.012 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:34.012 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.713570+0000", 2024-09-11T07:38:34.013 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.17681907799999999 2024-09-11T07:38:34.013 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.013 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.013 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:38:34.013 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.713591+0000", 2024-09-11T07:38:34.013 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1049000000000001e-05 2024-09-11T07:38:34.013 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.013 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.013 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:38:34.013 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.715815+0000", 2024-09-11T07:38:34.013 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0022245429999999998 2024-09-11T07:38:34.013 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.013 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.014 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:38:34.014 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.715859+0000", 2024-09-11T07:38:34.014 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3383999999999997e-05 2024-09-11T07:38:34.014 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:34.014 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:38:34.014 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:34.014 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.014 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.014 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87214 9.10 9:08bc93d3:::benchmark_data_smithi022_133943_object87213:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:38:34.014 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.342684+0000", 2024-09-11T07:38:34.014 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 3.9326488689999999, 2024-09-11T07:38:34.014 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1203623110000001, 2024-09-11T07:38:34.014 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:38:34.014 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:38:34.015 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:38:34.015 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:38:34.015 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:38:34.015 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87214 2024-09-11T07:38:34.015 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.015 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:38:34.015 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.015 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:38:34.015 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342684+0000", 2024-09-11T07:38:34.015 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:34.015 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.015 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.015 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:38:34.015 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342684+0000", 2024-09-11T07:38:34.016 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:34.016 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.016 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.016 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:38:34.016 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342682+0000", 2024-09-11T07:38:34.016 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:38:34.016 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.016 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.016 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:38:34.016 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342765+0000", 2024-09-11T07:38:34.016 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3146999999999995e-05 2024-09-11T07:38:34.016 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.016 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.017 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:38:34.017 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342777+0000", 2024-09-11T07:38:34.017 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1494999999999999e-05 2024-09-11T07:38:34.017 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.017 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.017 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:38:34.017 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342784+0000", 2024-09-11T07:38:34.017 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9750000000000001e-06 2024-09-11T07:38:34.017 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.017 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.017 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:34.017 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343439+0000", 2024-09-11T07:38:34.017 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00065492600000000001 2024-09-11T07:38:34.018 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.018 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.018 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:38:34.018 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343443+0000", 2024-09-11T07:38:34.018 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.583e-06 2024-09-11T07:38:34.018 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.018 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.018 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:34.018 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.449163+0000", 2024-09-11T07:38:34.018 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.105719841 2024-09-11T07:38:34.018 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.018 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.018 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:38:34.018 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.449317+0000", 2024-09-11T07:38:34.019 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000154358 2024-09-11T07:38:34.019 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.019 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.019 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2,4,6,7", 2024-09-11T07:38:34.019 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.450361+0000", 2024-09-11T07:38:34.019 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0010434000000000001 2024-09-11T07:38:34.019 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.019 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.019 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:38:34.019 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.456929+0000", 2024-09-11T07:38:34.019 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0065678389999999998 2024-09-11T07:38:34.020 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.020 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.020 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.020 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.457159+0000", 2024-09-11T07:38:34.020 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000230518 2024-09-11T07:38:34.020 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.020 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.020 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.020 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461254+0000", 2024-09-11T07:38:34.020 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0040944909999999996 2024-09-11T07:38:34.020 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.020 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.021 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.021 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461887+0000", 2024-09-11T07:38:34.021 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000633455 2024-09-11T07:38:34.021 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.021 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.021 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.021 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461946+0000", 2024-09-11T07:38:34.021 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9064999999999998e-05 2024-09-11T07:38:34.021 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.021 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.021 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.021 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.462748+0000", 2024-09-11T07:38:34.021 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00080188799999999995 2024-09-11T07:38:34.022 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.022 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.022 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:38:34.022 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.462836+0000", 2024-09-11T07:38:34.022 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.7596999999999995e-05 2024-09-11T07:38:34.022 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.022 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.022 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:38:34.022 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.463047+0000", 2024-09-11T07:38:34.022 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000211196 2024-09-11T07:38:34.022 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:34.022 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:38:34.022 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:34.022 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.023 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.023 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87241 9.16 9:6d9f9b4a:::benchmark_data_smithi022_133943_object87240:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:38:34.023 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.342793+0000", 2024-09-11T07:38:34.023 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 3.932540795, 2024-09-11T07:38:34.023 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.087127435, 2024-09-11T07:38:34.023 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:38:34.023 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:38:34.023 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:38:34.023 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:38:34.023 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:38:34.023 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87241 2024-09-11T07:38:34.023 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.023 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:38:34.024 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.024 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:38:34.024 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342793+0000", 2024-09-11T07:38:34.024 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:34.024 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.024 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.024 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:38:34.024 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342793+0000", 2024-09-11T07:38:34.024 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:34.024 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.024 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.024 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:38:34.024 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342792+0000", 2024-09-11T07:38:34.025 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:38:34.025 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.025 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.025 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:38:34.025 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342816+0000", 2024-09-11T07:38:34.025 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4511e-05 2024-09-11T07:38:34.025 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.025 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.025 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:38:34.025 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342823+0000", 2024-09-11T07:38:34.025 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.2189999999999998e-06 2024-09-11T07:38:34.025 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.025 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.025 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:38:34.025 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342828+0000", 2024-09-11T07:38:34.026 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5950000000000001e-06 2024-09-11T07:38:34.026 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.026 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.026 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:34.026 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.346264+0000", 2024-09-11T07:38:34.026 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0034359199999999999 2024-09-11T07:38:34.026 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.026 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.026 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:38:34.026 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.346271+0000", 2024-09-11T07:38:34.026 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9070000000000001e-06 2024-09-11T07:38:34.026 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.026 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.026 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:34.027 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.416568+0000", 2024-09-11T07:38:34.027 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.07029756 2024-09-11T07:38:34.027 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.027 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.027 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:38:34.027 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.416798+0000", 2024-09-11T07:38:34.027 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00022939600000000001 2024-09-11T07:38:34.027 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.027 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.027 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,4,5", 2024-09-11T07:38:34.027 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.417111+0000", 2024-09-11T07:38:34.027 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000313374 2024-09-11T07:38:34.027 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.027 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.028 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:38:34.028 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.420512+0000", 2024-09-11T07:38:34.028 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0034004019999999999 2024-09-11T07:38:34.028 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.028 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.028 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.028 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.424231+0000", 2024-09-11T07:38:34.028 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0037196479999999999 2024-09-11T07:38:34.028 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.028 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.028 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.028 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.424639+0000", 2024-09-11T07:38:34.028 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00040762699999999998 2024-09-11T07:38:34.028 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.029 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.029 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.029 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429504+0000", 2024-09-11T07:38:34.029 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0048647830000000001 2024-09-11T07:38:34.029 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.029 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.029 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:38:34.029 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429776+0000", 2024-09-11T07:38:34.029 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00027279600000000001 2024-09-11T07:38:34.029 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.029 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.029 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:38:34.029 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429920+0000", 2024-09-11T07:38:34.029 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014360299999999999 2024-09-11T07:38:34.030 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:34.030 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:38:34.030 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:34.030 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.030 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.030 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87242 9.16 9:6d646f5a:::benchmark_data_smithi022_133943_object87241:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:38:34.030 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.342889+0000", 2024-09-11T07:38:34.030 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 3.932444421, 2024-09-11T07:38:34.030 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0872590680000001, 2024-09-11T07:38:34.030 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:38:34.030 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:38:34.030 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:38:34.030 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:38:34.031 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:38:34.031 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87242 2024-09-11T07:38:34.031 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.031 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:38:34.031 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.031 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:38:34.031 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342889+0000", 2024-09-11T07:38:34.031 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:34.031 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.031 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.031 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:38:34.031 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342889+0000", 2024-09-11T07:38:34.031 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:34.032 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.032 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.032 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:38:34.032 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342888+0000", 2024-09-11T07:38:34.032 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:38:34.032 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.032 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.032 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:38:34.032 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342914+0000", 2024-09-11T07:38:34.032 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.6143999999999999e-05 2024-09-11T07:38:34.032 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.032 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.032 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:38:34.032 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342923+0000", 2024-09-11T07:38:34.033 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.7050000000000005e-06 2024-09-11T07:38:34.033 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.033 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.033 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:38:34.033 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342928+0000", 2024-09-11T07:38:34.033 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.13e-06 2024-09-11T07:38:34.033 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.033 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.033 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:34.033 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.346523+0000", 2024-09-11T07:38:34.033 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0035955869999999999 2024-09-11T07:38:34.033 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.033 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.033 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:38:34.034 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.346533+0000", 2024-09-11T07:38:34.034 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.4290000000000007e-06 2024-09-11T07:38:34.034 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.034 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.034 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:34.034 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.418238+0000", 2024-09-11T07:38:34.034 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0717048899999999 2024-09-11T07:38:34.034 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.034 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.034 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:38:34.034 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.418354+0000", 2024-09-11T07:38:34.034 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011647899999999999 2024-09-11T07:38:34.034 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.035 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.035 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,4,5", 2024-09-11T07:38:34.035 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.418586+0000", 2024-09-11T07:38:34.035 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000231618 2024-09-11T07:38:34.035 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.035 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.035 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.035 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.424264+0000", 2024-09-11T07:38:34.035 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0056779370000000001 2024-09-11T07:38:34.035 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.035 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.035 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.035 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.424711+0000", 2024-09-11T07:38:34.035 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00044714499999999998 2024-09-11T07:38:34.036 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.036 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.036 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:38:34.036 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429400+0000", 2024-09-11T07:38:34.036 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0046887550000000002 2024-09-11T07:38:34.036 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.036 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.036 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.036 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429971+0000", 2024-09-11T07:38:34.036 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00057166200000000002 2024-09-11T07:38:34.036 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.036 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.036 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:38:34.037 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.430024+0000", 2024-09-11T07:38:34.037 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3009e-05 2024-09-11T07:38:34.037 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.037 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.037 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:38:34.037 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.430148+0000", 2024-09-11T07:38:34.037 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012362300000000001 2024-09-11T07:38:34.037 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:34.037 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:38:34.037 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:34.037 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.037 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.037 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87252 9.12 9:4a1febda:::benchmark_data_smithi022_133943_object87251:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:38:34.038 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.342933+0000", 2024-09-11T07:38:34.038 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 3.9324007500000002, 2024-09-11T07:38:34.038 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0912417050000001, 2024-09-11T07:38:34.038 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:38:34.038 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:38:34.038 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:38:34.038 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:38:34.038 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:38:34.038 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87252 2024-09-11T07:38:34.038 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.038 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:38:34.038 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.038 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:38:34.039 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342933+0000", 2024-09-11T07:38:34.039 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:34.039 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.039 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.039 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:38:34.039 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342933+0000", 2024-09-11T07:38:34.039 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:34.039 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.039 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.039 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:38:34.039 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342932+0000", 2024-09-11T07:38:34.039 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:38:34.039 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.039 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.039 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:38:34.040 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342953+0000", 2024-09-11T07:38:34.040 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0792999999999999e-05 2024-09-11T07:38:34.040 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.040 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.040 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:38:34.040 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342959+0000", 2024-09-11T07:38:34.040 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4869999999999999e-06 2024-09-11T07:38:34.040 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.040 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.040 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:38:34.040 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342964+0000", 2024-09-11T07:38:34.040 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4100000000000001e-06 2024-09-11T07:38:34.040 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.041 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.041 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:34.041 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.344041+0000", 2024-09-11T07:38:34.041 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0010773219999999999 2024-09-11T07:38:34.041 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.041 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.041 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:38:34.041 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.344048+0000", 2024-09-11T07:38:34.041 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7220000000000002e-06 2024-09-11T07:38:34.041 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.041 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.041 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:34.041 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.423752+0000", 2024-09-11T07:38:34.041 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.079704123 2024-09-11T07:38:34.042 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.042 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.042 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:38:34.042 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.423914+0000", 2024-09-11T07:38:34.042 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000161972 2024-09-11T07:38:34.042 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.042 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.042 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2,3,5", 2024-09-11T07:38:34.042 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.424186+0000", 2024-09-11T07:38:34.042 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000272238 2024-09-11T07:38:34.042 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.042 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.042 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.042 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429530+0000", 2024-09-11T07:38:34.042 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0053444929999999996 2024-09-11T07:38:34.043 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.043 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.043 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.043 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429824+0000", 2024-09-11T07:38:34.043 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00029352499999999999 2024-09-11T07:38:34.043 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.043 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.043 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.043 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.430566+0000", 2024-09-11T07:38:34.043 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00074221400000000005 2024-09-11T07:38:34.043 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.043 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.043 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:38:34.043 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.431126+0000", 2024-09-11T07:38:34.044 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00055992399999999995 2024-09-11T07:38:34.044 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.044 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.044 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.044 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.433998+0000", 2024-09-11T07:38:34.044 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0028716219999999999 2024-09-11T07:38:34.044 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.044 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.044 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:38:34.044 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.434055+0000", 2024-09-11T07:38:34.044 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7658999999999998e-05 2024-09-11T07:38:34.044 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.044 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.045 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:38:34.045 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.434174+0000", 2024-09-11T07:38:34.045 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000118891 2024-09-11T07:38:34.045 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:34.045 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:38:34.045 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:34.045 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.045 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.045 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87253 9.10 9:0ceb58b8:::benchmark_data_smithi022_133943_object87252:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:38:34.045 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.342970+0000", 2024-09-11T07:38:34.045 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 3.932363295, 2024-09-11T07:38:34.045 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.123611441, 2024-09-11T07:38:34.045 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:38:34.045 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:38:34.046 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:38:34.046 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:38:34.046 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:38:34.046 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87253 2024-09-11T07:38:34.046 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.046 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:38:34.046 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.046 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:38:34.046 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342970+0000", 2024-09-11T07:38:34.046 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:34.046 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.046 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.046 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:38:34.046 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342970+0000", 2024-09-11T07:38:34.047 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:34.047 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.047 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.047 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:38:34.047 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342969+0000", 2024-09-11T07:38:34.047 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:38:34.047 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.047 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.047 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:38:34.047 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342991+0000", 2024-09-11T07:38:34.047 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1855999999999999e-05 2024-09-11T07:38:34.047 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.047 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.047 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:38:34.048 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342998+0000", 2024-09-11T07:38:34.048 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7769999999999997e-06 2024-09-11T07:38:34.048 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.048 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.048 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:38:34.048 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343002+0000", 2024-09-11T07:38:34.048 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0860000000000004e-06 2024-09-11T07:38:34.048 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.048 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.048 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:34.048 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343467+0000", 2024-09-11T07:38:34.048 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00046544499999999999 2024-09-11T07:38:34.048 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.049 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.049 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:38:34.049 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343475+0000", 2024-09-11T07:38:34.049 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1400000000000002e-06 2024-09-11T07:38:34.049 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.049 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.049 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:34.049 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.455239+0000", 2024-09-11T07:38:34.049 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1117646999999999 2024-09-11T07:38:34.049 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.049 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.049 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:38:34.049 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.455378+0000", 2024-09-11T07:38:34.050 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013862299999999999 2024-09-11T07:38:34.050 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.050 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.050 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2,4,6,7", 2024-09-11T07:38:34.050 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.455674+0000", 2024-09-11T07:38:34.050 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00029640200000000003 2024-09-11T07:38:34.050 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.050 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.050 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.050 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.459952+0000", 2024-09-11T07:38:34.050 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0042780409999999998 2024-09-11T07:38:34.050 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.050 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.051 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.051 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461424+0000", 2024-09-11T07:38:34.051 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001471483 2024-09-11T07:38:34.051 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.051 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.051 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.051 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461918+0000", 2024-09-11T07:38:34.051 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00049364099999999996 2024-09-11T07:38:34.051 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.051 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.051 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.051 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.463082+0000", 2024-09-11T07:38:34.051 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00116457 2024-09-11T07:38:34.052 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.052 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.052 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.052 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.464588+0000", 2024-09-11T07:38:34.052 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0015063889999999999 2024-09-11T07:38:34.052 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.052 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.052 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:38:34.052 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.466379+0000", 2024-09-11T07:38:34.052 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00179065 2024-09-11T07:38:34.052 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.052 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.052 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:38:34.053 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.466455+0000", 2024-09-11T07:38:34.053 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5956000000000003e-05 2024-09-11T07:38:34.053 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.053 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.053 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:38:34.053 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.466581+0000", 2024-09-11T07:38:34.053 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012639699999999999 2024-09-11T07:38:34.053 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:34.053 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:38:34.053 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:34.053 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.053 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.053 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87254 9.16 9:6e7c07b3:::benchmark_data_smithi022_133943_object87253:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:38:34.053 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.343006+0000", 2024-09-11T07:38:34.053 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 3.9323269119999997, 2024-09-11T07:38:34.054 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.094570789, 2024-09-11T07:38:34.054 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:38:34.054 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:38:34.054 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:38:34.054 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:38:34.054 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:38:34.054 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87254 2024-09-11T07:38:34.054 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.054 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:38:34.054 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.054 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:38:34.054 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343006+0000", 2024-09-11T07:38:34.054 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:34.054 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.054 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.055 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:38:34.055 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343006+0000", 2024-09-11T07:38:34.055 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:38:34.055 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.055 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.055 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:38:34.055 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343006+0000", 2024-09-11T07:38:34.055 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:38:34.055 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.055 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.055 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:38:34.055 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343027+0000", 2024-09-11T07:38:34.055 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1491000000000001e-05 2024-09-11T07:38:34.055 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.056 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.056 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:38:34.056 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343034+0000", 2024-09-11T07:38:34.056 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7009999999999999e-06 2024-09-11T07:38:34.056 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.056 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.056 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:38:34.056 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343038+0000", 2024-09-11T07:38:34.056 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.98e-06 2024-09-11T07:38:34.056 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.056 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.056 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:34.056 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.346555+0000", 2024-09-11T07:38:34.056 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00351688 2024-09-11T07:38:34.057 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.057 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.057 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:38:34.057 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.346561+0000", 2024-09-11T07:38:34.057 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7359999999999998e-06 2024-09-11T07:38:34.057 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.057 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.057 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:38:34.057 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.420577+0000", 2024-09-11T07:38:34.057 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0740159579999999 2024-09-11T07:38:34.057 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.057 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.057 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:38:34.057 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.420669+0000", 2024-09-11T07:38:34.058 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.2723e-05 2024-09-11T07:38:34.058 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.058 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.058 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,4,5", 2024-09-11T07:38:34.058 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.420997+0000", 2024-09-11T07:38:34.058 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00032735499999999999 2024-09-11T07:38:34.058 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.058 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.058 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:38:34.058 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429409+0000", 2024-09-11T07:38:34.058 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0084126630000000008 2024-09-11T07:38:34.058 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.058 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.058 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.058 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429463+0000", 2024-09-11T07:38:34.059 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3375999999999999e-05 2024-09-11T07:38:34.059 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.059 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.059 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.059 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.430192+0000", 2024-09-11T07:38:34.059 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00072935800000000005 2024-09-11T07:38:34.059 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.059 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.059 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:38:34.059 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.437414+0000", 2024-09-11T07:38:34.059 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0072219939999999998 2024-09-11T07:38:34.059 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.059 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.059 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:38:34.059 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.437450+0000", 2024-09-11T07:38:34.060 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5957000000000001e-05 2024-09-11T07:38:34.060 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:38:34.060 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:38:34.060 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:38:34.060 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.437577+0000", 2024-09-11T07:38:34.060 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012722100000000001 2024-09-11T07:38:34.060 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:34.060 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:38:34.060 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:34.060 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:38:34.060 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:38:34.060 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:38:34.061 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:34.098 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:34.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:34.099 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:34.109 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:38:34.200 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:34.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:34.243 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:34.243 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:34.344 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:34.379 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:34.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:34.381 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:34.389 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is unset 2024-09-11T07:38:34.395 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:38:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:38:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:38:34.482 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:34.484 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 57 16 92131 92115 100.983 73.1875 0.0115712 0.00989752 2024-09-11T07:38:34.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:34.525 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:34.525 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:34.626 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:34.659 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:34 smithi104 ceph-mon[33507]: 5.8 scrub starts 2024-09-11T07:38:34.659 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:34 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/141316484' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:38:34.659 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:34 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:38:34.659 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:34 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:38:34.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:34 smithi022 ceph-mon[29462]: 5.8 scrub starts 2024-09-11T07:38:34.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:34 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/141316484' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:38:34.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:34 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:38:34.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:34 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:38:34.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:34 smithi022 ceph-mon[37330]: 5.8 scrub starts 2024-09-11T07:38:34.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:34 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/141316484' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:38:34.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:34 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:38:34.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:34 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:38:34.668 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:34.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:34.669 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:34.770 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:34.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:34.809 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:34.810 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:34.879 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-11T07:38:34.911 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:34.963 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:34.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:34.964 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:35.065 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:35.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:35.112 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:35.112 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:35.213 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:35.247 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:35.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:35.248 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:35.349 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:35.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:35.391 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:35.392 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:35.484 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 58 16 93313 93297 100.516 73.875 0.0155323 0.00994408 2024-09-11T07:38:35.560 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:35.561 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:35 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:38:35.561 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:35 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:38:35.561 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:35 smithi104 ceph-mon[33507]: 2.10 scrub starts 2024-09-11T07:38:35.561 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:35 smithi104 ceph-mon[33507]: pgmap v539: 158 pgs: 4 peering, 5 remapped+peering, 149 active+clean; 5.4 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 26 KiB/s, 0 objects/s recovering 2024-09-11T07:38:35.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:35 smithi104 ceph-mon[33507]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:38:35.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:35 smithi104 ceph-mon[33507]: Cluster is now healthy 2024-09-11T07:38:35.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:35 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/141316484' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:38:35.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:35 smithi104 ceph-mon[33507]: osdmap e266: 8 total, 8 up, 8 in 2024-09-11T07:38:35.601 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:35.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:35.602 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:35.663 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:35 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:38:35.664 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:35 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:38:35.664 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:35 smithi022 ceph-mon[29462]: 2.10 scrub starts 2024-09-11T07:38:35.664 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:35 smithi022 ceph-mon[29462]: pgmap v539: 158 pgs: 4 peering, 5 remapped+peering, 149 active+clean; 5.4 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 26 KiB/s, 0 objects/s recovering 2024-09-11T07:38:35.664 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:35 smithi022 ceph-mon[29462]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:38:35.664 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:35 smithi022 ceph-mon[29462]: Cluster is now healthy 2024-09-11T07:38:35.664 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:35 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/141316484' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:38:35.664 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:35 smithi022 ceph-mon[29462]: osdmap e266: 8 total, 8 up, 8 in 2024-09-11T07:38:35.664 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:35 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:38:35.664 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:35 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:38:35.664 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:35 smithi022 ceph-mon[37330]: 2.10 scrub starts 2024-09-11T07:38:35.665 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:35 smithi022 ceph-mon[37330]: pgmap v539: 158 pgs: 4 peering, 5 remapped+peering, 149 active+clean; 5.4 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 26 KiB/s, 0 objects/s recovering 2024-09-11T07:38:35.665 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:35 smithi022 ceph-mon[37330]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:38:35.665 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:35 smithi022 ceph-mon[37330]: Cluster is now healthy 2024-09-11T07:38:35.665 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:35 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/141316484' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:38:35.665 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:35 smithi022 ceph-mon[37330]: osdmap e266: 8 total, 8 up, 8 in 2024-09-11T07:38:35.703 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:35.744 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:35.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:35.746 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:35.846 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:35.887 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:35.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:35.888 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:35.945 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:38:35.946 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:38:35.946 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:38:35.946 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:38:35.946 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:38:35.946 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:38:35.946 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:38:35.946 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:38:35.946 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:38:35.947 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:38:35.947 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:38:35.947 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:38:35.947 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:38:35.947 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:38:35.947 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:38:35.947 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:38:35.989 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:36.034 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:36.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:36.035 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:36.067 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:38:36.068 INFO:teuthology.orchestra.run.smithi104.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-11T07:38:36.068 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:38:36.136 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:36.175 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:36.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:36.176 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:36.277 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:36.308 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:36.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:36.309 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:36.410 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:36.447 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:36.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:36.449 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:36.484 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 59 15 94387 94372 99.951 67.1875 0.00874163 0.00999933 2024-09-11T07:38:36.549 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:36.591 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:36.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:36.592 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:36.594 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:36 smithi022 ceph-mon[29462]: 2.c scrub starts 2024-09-11T07:38:36.594 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:36 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2540859721' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:38:36.594 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:36 smithi022 ceph-mon[37330]: 2.c scrub starts 2024-09-11T07:38:36.594 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:36 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2540859721' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:38:36.693 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:36.725 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:36 smithi104 ceph-mon[33507]: 2.c scrub starts 2024-09-11T07:38:36.725 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:36 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2540859721' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:38:36.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:36.733 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:36.734 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:36.835 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:36.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:36.872 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:36.872 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:36.973 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:37.012 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:37.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:37.013 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:37.114 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:37.150 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:37.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:37.151 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:37.252 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:37.286 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:37.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:37.287 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:37.388 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:37.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:37.422 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:37.423 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:37.484 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:2024-09-11T07:38:37.483563+0000 min lat: 0.00139335 max lat: 2.21963 avg lat: 0.0100554 2024-09-11T07:38:37.484 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T07:38:37.484 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 60 16 95454 95438 99.3955 66.625 0.00591582 0.0100554 2024-09-11T07:38:37.524 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:37.561 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:37.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:37.562 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:37.663 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:37.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:37 smithi022 ceph-mon[29462]: pgmap v541: 158 pgs: 2 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 150 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 873 B/s rd, 52 MiB/s wr, 840 op/s; 42805/277089 objects misplaced (15.448%); 19 MiB/s, 303 objects/s recovering 2024-09-11T07:38:37.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:37 smithi022 ceph-mon[29462]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:38:37.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:37 smithi022 ceph-mon[37330]: pgmap v541: 158 pgs: 2 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 150 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 873 B/s rd, 52 MiB/s wr, 840 op/s; 42805/277089 objects misplaced (15.448%); 19 MiB/s, 303 objects/s recovering 2024-09-11T07:38:37.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:37 smithi022 ceph-mon[37330]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:38:37.705 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:37.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:37.706 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:37.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:37 smithi104 ceph-mon[33507]: pgmap v541: 158 pgs: 2 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 150 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 873 B/s rd, 52 MiB/s wr, 840 op/s; 42805/277089 objects misplaced (15.448%); 19 MiB/s, 303 objects/s recovering 2024-09-11T07:38:37.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:37 smithi104 ceph-mon[33507]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:38:37.807 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:37.852 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:37.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:37.853 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:37.955 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:37.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:37.997 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:37.997 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:38.098 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:38.100 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:38:38.147 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:38.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:38.148 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:38.249 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:38.294 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:38.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:38.295 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:38.396 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:38.436 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:38.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:38.437 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:38.484 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 61 16 96668 96652 99.0099 75.875 0.00685194 0.0100951 2024-09-11T07:38:38.538 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:38.575 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:38.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:38.577 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:38.677 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:38.719 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:38.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:38.721 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:38.821 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:38.861 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:38.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:38.862 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:38.963 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:39.005 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:39.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:39.006 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:39.107 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:39.151 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:39.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:39.152 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:39.253 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:39.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:39 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:38:39.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:39 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:38:39.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:39 smithi104 ceph-mon[33507]: pgmap v542: 158 pgs: 2 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 150 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 674 B/s rd, 41 MiB/s wr, 649 op/s; 42805/277089 objects misplaced (15.448%); 15 MiB/s, 234 objects/s recovering 2024-09-11T07:38:39.291 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:39.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:39.292 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:39.306 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T07:38:39.371 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:39 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:38:39.371 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:39 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:38:39.371 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:39 smithi022 ceph-mon[29462]: pgmap v542: 158 pgs: 2 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 150 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 674 B/s rd, 41 MiB/s wr, 649 op/s; 42805/277089 objects misplaced (15.448%); 15 MiB/s, 234 objects/s recovering 2024-09-11T07:38:39.371 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:39 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:38:39.372 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:39 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:38:39.372 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:39 smithi022 ceph-mon[37330]: pgmap v542: 158 pgs: 2 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 150 active+clean; 5.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 674 B/s rd, 41 MiB/s wr, 649 op/s; 42805/277089 objects misplaced (15.448%); 15 MiB/s, 234 objects/s recovering 2024-09-11T07:38:39.393 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:39.426 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:39.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:39.427 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:39.484 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 62 16 97839 97823 98.5933 73.1875 0.00986775 0.010138 2024-09-11T07:38:39.528 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:39.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:39.573 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:39.573 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:39.674 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:39.726 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:39.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:39.727 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:39.828 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:39.866 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:39.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:39.867 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:39.901 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:38:39.967 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:39.991 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:39.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:39.992 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:40.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:38:39 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:38:40.092 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:40.127 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:40.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:40.128 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:40.229 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:40.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:40.277 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:40.277 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:40.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:40 smithi104 ceph-mon[33507]: 2.1 scrub starts 2024-09-11T07:38:40.374 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:40 smithi022 ceph-mon[29462]: 2.1 scrub starts 2024-09-11T07:38:40.374 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:40 smithi022 ceph-mon[37330]: 2.1 scrub starts 2024-09-11T07:38:40.378 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:40.424 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:40.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:40.426 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:40.484 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 63 16 98927 98911 98.1077 68 0.0194096 0.0101885 2024-09-11T07:38:40.527 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:40.562 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:40.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:40.563 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:40.664 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:40.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:38:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:38:40] "GET /metrics HTTP/1.1" 200 224514 "" "Prometheus/2.43.0" 2024-09-11T07:38:40.695 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:40.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:40.696 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:40.797 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:40.831 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:40.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:40.832 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:40.932 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:40.963 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:40.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:40.964 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:41.065 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:41.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:41.092 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:41.093 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:41.193 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:41.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:41.216 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:41.217 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:41.317 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:41.358 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:41.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:41.359 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:41.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:41 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:38:41.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:41 smithi022 ceph-mon[29462]: pgmap v543: 158 pgs: 2 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 150 active+clean; 6.0 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 77 MiB/s wr, 1.24k op/s; 41488/294207 objects misplaced (14.102%); 28 MiB/s, 444 objects/s recovering 2024-09-11T07:38:41.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:41 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:38:41.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:41 smithi022 ceph-mon[37330]: pgmap v543: 158 pgs: 2 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 150 active+clean; 6.0 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 77 MiB/s wr, 1.24k op/s; 41488/294207 objects misplaced (14.102%); 28 MiB/s, 444 objects/s recovering 2024-09-11T07:38:41.460 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:41.484 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 64 16 99728 99712 97.357 50.0625 0.00817928 0.0102665 2024-09-11T07:38:41.497 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:41.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:41.499 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:41.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:41 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:38:41.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:41 smithi104 ceph-mon[33507]: pgmap v543: 158 pgs: 2 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 150 active+clean; 6.0 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 77 MiB/s wr, 1.24k op/s; 41488/294207 objects misplaced (14.102%); 28 MiB/s, 444 objects/s recovering 2024-09-11T07:38:41.600 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:41.623 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:41.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:41.624 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:41.725 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:41.760 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:41.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:41.762 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:41.767 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:38:41.767 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:38:41.767 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-11T07:38:41.767 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T07:38:41.767 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T07:38:41.768 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 config set filestore_inject_stall 3 2024-09-11T07:38:41.862 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:41.893 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:41.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:41.894 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:41.995 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:42.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:42.042 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:42.042 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:42.143 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:42.174 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:42.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:42.175 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:42.276 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:42.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:42.321 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:42.321 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:42.422 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:42.461 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:42.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:42.462 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:42.484 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 65 16 100749 100733 96.8409 63.8125 0.0139454 0.0103207 2024-09-11T07:38:42.563 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:42.602 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:42.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:42.603 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:42.704 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:42.739 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:42.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:42.741 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:42.842 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:42.883 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:42.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:42.884 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:42.985 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:43.027 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:43.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:43.028 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:43.129 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:43.163 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:43.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:43.164 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:43.265 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:43.302 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:43.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:43.303 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:43.404 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:43.427 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:43.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:43.428 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:43.485 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 66 16 101819 101803 96.3868 66.875 0.00855003 0.0103701 2024-09-11T07:38:43.528 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:43.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:43 smithi104 ceph-mon[33507]: pgmap v544: 158 pgs: 2 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 150 active+clean; 6.0 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 66 MiB/s wr, 1.06k op/s; 41488/294207 objects misplaced (14.102%); 24 MiB/s, 381 objects/s recovering 2024-09-11T07:38:43.575 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:43.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:43.576 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:43.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:43 smithi022 ceph-mon[29462]: pgmap v544: 158 pgs: 2 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 150 active+clean; 6.0 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 66 MiB/s wr, 1.06k op/s; 41488/294207 objects misplaced (14.102%); 24 MiB/s, 381 objects/s recovering 2024-09-11T07:38:43.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:43 smithi022 ceph-mon[37330]: pgmap v544: 158 pgs: 2 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 150 active+clean; 6.0 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 66 MiB/s wr, 1.06k op/s; 41488/294207 objects misplaced (14.102%); 24 MiB/s, 381 objects/s recovering 2024-09-11T07:38:43.676 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:43.705 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:43.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:43.706 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:43.806 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:43.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:43.836 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:43.837 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:43.937 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:43.975 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:43.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:43.976 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:44.077 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:44.099 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:44.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:44.100 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:44.201 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:44.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:44.237 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:44.237 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:44.338 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:44.361 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:44.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:44.362 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:44.463 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:44.484 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 67 16 102635 102619 95.7094 51 0.0102894 0.0104335 2024-09-11T07:38:44.490 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:44.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:44.491 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:44.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:44 smithi104 ceph-mon[33507]: 6.f scrub starts 2024-09-11T07:38:44.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:38:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:38:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:38:44.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:44 smithi022 ceph-mon[29462]: 6.f scrub starts 2024-09-11T07:38:44.532 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:44 smithi022 ceph-mon[37330]: 6.f scrub starts 2024-09-11T07:38:44.591 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:44.617 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:44.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:44.618 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:44.719 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:44.747 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:38:44.747 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:38:44.750 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:44.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:44.751 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:44.816 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:38:44.853 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:44.878 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:44.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:44.880 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:44.980 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:45.001 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:45.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:45.002 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:45.103 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:45.126 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:45.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:45.127 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:45.227 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:45.252 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:45.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:45.253 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:45.354 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:45.392 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:45.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:45.393 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:45.485 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 68 16 102943 102927 94.585 19.25 0.00549819 0.0105562 2024-09-11T07:38:45.494 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:45.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:45.521 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:45.522 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:45.622 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:45.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:45 smithi022 ceph-mon[29462]: pgmap v545: 158 pgs: 2 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 150 active+clean; 6.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 67 MiB/s wr, 1.08k op/s; 41488/297051 objects misplaced (13.967%); 24 MiB/s, 381 objects/s recovering 2024-09-11T07:38:45.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:45 smithi022 ceph-mon[29462]: osdmap e267: 8 total, 8 up, 8 in 2024-09-11T07:38:45.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:45 smithi022 ceph-mon[37330]: pgmap v545: 158 pgs: 2 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 150 active+clean; 6.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 67 MiB/s wr, 1.08k op/s; 41488/297051 objects misplaced (13.967%); 24 MiB/s, 381 objects/s recovering 2024-09-11T07:38:45.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:45 smithi022 ceph-mon[37330]: osdmap e267: 8 total, 8 up, 8 in 2024-09-11T07:38:45.668 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:45.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:45.670 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:45.770 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:45.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:45 smithi104 ceph-mon[33507]: pgmap v545: 158 pgs: 2 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 150 active+clean; 6.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 67 MiB/s wr, 1.08k op/s; 41488/297051 objects misplaced (13.967%); 24 MiB/s, 381 objects/s recovering 2024-09-11T07:38:45.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:45 smithi104 ceph-mon[33507]: osdmap e267: 8 total, 8 up, 8 in 2024-09-11T07:38:45.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:45.813 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:45.813 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:45.914 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:45.936 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:45.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:45.937 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:46.038 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:46.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:46.092 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:46.092 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:46.193 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:46.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:46.233 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:46.233 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:46.334 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:46.355 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:46.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:46.356 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:46.457 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:46.484 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 69 16 103969 103953 94.1435 64.125 0.010285 0.0106175 2024-09-11T07:38:46.490 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:46.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:46.491 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:46.592 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:46.624 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:46.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:46.625 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:46.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:46 smithi022 ceph-mon[29462]: 4.1c scrub starts 2024-09-11T07:38:46.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:46 smithi022 ceph-mon[29462]: osdmap e268: 8 total, 8 up, 8 in 2024-09-11T07:38:46.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:46 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:38:46.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:46 smithi022 ceph-mon[37330]: 4.1c scrub starts 2024-09-11T07:38:46.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:46 smithi022 ceph-mon[37330]: osdmap e268: 8 total, 8 up, 8 in 2024-09-11T07:38:46.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:46 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:38:46.726 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:46.750 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:46.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:46.752 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:46.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:46 smithi104 ceph-mon[33507]: 4.1c scrub starts 2024-09-11T07:38:46.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:46 smithi104 ceph-mon[33507]: osdmap e268: 8 total, 8 up, 8 in 2024-09-11T07:38:46.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:46 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:38:46.853 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:46.892 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:46.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:46.893 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:46.994 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:47.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:47.024 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:47.024 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:47.125 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:47.153 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:47.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:47.154 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:47.255 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:47.279 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:47.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:47.281 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:47.381 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:47.404 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:47.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:47.405 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:47.485 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 70 16 104587 104571 93.3504 38.625 0.00297953 0.0106213 2024-09-11T07:38:47.505 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:47.529 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:47.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:47 smithi104 ceph-mon[33507]: 5.a scrub starts 2024-09-11T07:38:47.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:47 smithi104 ceph-mon[33507]: pgmap v548: 158 pgs: 1 active+remapped+backfilling, 1 peering, 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 150 active+clean; 6.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s; 38802/308355 objects misplaced (12.584%); 35 MiB/s, 564 objects/s recovering 2024-09-11T07:38:47.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:47 smithi104 ceph-mon[33507]: 2.f scrub starts 2024-09-11T07:38:47.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:47 smithi104 ceph-mon[33507]: osdmap e269: 8 total, 8 up, 8 in 2024-09-11T07:38:47.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:47 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:38:47.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:47 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:38:47.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:47.531 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:47.589 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:47 smithi022 ceph-mon[29462]: 5.a scrub starts 2024-09-11T07:38:47.589 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:47 smithi022 ceph-mon[29462]: pgmap v548: 158 pgs: 1 active+remapped+backfilling, 1 peering, 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 150 active+clean; 6.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s; 38802/308355 objects misplaced (12.584%); 35 MiB/s, 564 objects/s recovering 2024-09-11T07:38:47.589 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:47 smithi022 ceph-mon[29462]: 2.f scrub starts 2024-09-11T07:38:47.589 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:47 smithi022 ceph-mon[29462]: osdmap e269: 8 total, 8 up, 8 in 2024-09-11T07:38:47.589 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:47 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:38:47.590 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:47 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:38:47.590 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:47 smithi022 ceph-mon[37330]: 5.a scrub starts 2024-09-11T07:38:47.590 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:47 smithi022 ceph-mon[37330]: pgmap v548: 158 pgs: 1 active+remapped+backfilling, 1 peering, 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 150 active+clean; 6.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s; 38802/308355 objects misplaced (12.584%); 35 MiB/s, 564 objects/s recovering 2024-09-11T07:38:47.590 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:47 smithi022 ceph-mon[37330]: 2.f scrub starts 2024-09-11T07:38:47.590 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:47 smithi022 ceph-mon[37330]: osdmap e269: 8 total, 8 up, 8 in 2024-09-11T07:38:47.590 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:47 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:38:47.590 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:47 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:38:47.632 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:47.656 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:47.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:47.657 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:47.758 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:47.789 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:47.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:47.790 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:47.891 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:47.934 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:47.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:47.935 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:48.036 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:48.064 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:48.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:48.065 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:48.166 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:48.221 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:38:48.221 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:38:48.226 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:48.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:48.227 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:48.328 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:48.368 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:48.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:48.369 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:48.470 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:48.485 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 71 16 105572 105556 92.9027 61.5625 0.0159835 0.0107594 2024-09-11T07:38:48.500 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:48.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:48.501 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:48.602 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:48.642 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:48.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:48.643 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:48.744 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:48.771 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:48.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:48.772 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:48.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:48 smithi104 ceph-mon[33507]: 6.1e scrub starts 2024-09-11T07:38:48.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:48 smithi104 ceph-mon[33507]: osdmap e270: 8 total, 8 up, 8 in 2024-09-11T07:38:48.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:48 smithi104 ceph-mon[33507]: pgmap v551: 158 pgs: 1 active+remapped+backfilling, 1 peering, 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 150 active+clean; 6.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 59 MiB/s wr, 942 op/s; 38802/308355 objects misplaced (12.584%); 39 MiB/s, 619 objects/s recovering 2024-09-11T07:38:48.873 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:48.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:48.908 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:48.908 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:48.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:48 smithi022 ceph-mon[29462]: 6.1e scrub starts 2024-09-11T07:38:48.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:48 smithi022 ceph-mon[29462]: osdmap e270: 8 total, 8 up, 8 in 2024-09-11T07:38:48.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:48 smithi022 ceph-mon[29462]: pgmap v551: 158 pgs: 1 active+remapped+backfilling, 1 peering, 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 150 active+clean; 6.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 59 MiB/s wr, 942 op/s; 38802/308355 objects misplaced (12.584%); 39 MiB/s, 619 objects/s recovering 2024-09-11T07:38:48.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:48 smithi022 ceph-mon[37330]: 6.1e scrub starts 2024-09-11T07:38:48.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:48 smithi022 ceph-mon[37330]: osdmap e270: 8 total, 8 up, 8 in 2024-09-11T07:38:48.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:48 smithi022 ceph-mon[37330]: pgmap v551: 158 pgs: 1 active+remapped+backfilling, 1 peering, 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 150 active+clean; 6.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 59 MiB/s wr, 942 op/s; 38802/308355 objects misplaced (12.584%); 39 MiB/s, 619 objects/s recovering 2024-09-11T07:38:49.009 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:49.050 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:49.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:49.051 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:49.152 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:49.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:49.193 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:49.193 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:49.294 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:49.326 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:49.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:49.327 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:49.428 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:49.466 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:49.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:49.467 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:49.485 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 72 16 106812 106796 92.6887 77.5 0.00813355 0.010784 2024-09-11T07:38:49.568 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:49.592 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:49.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:49.593 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:49.694 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:49.731 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:49.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:49.733 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:49.834 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:49.858 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:49.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:49.859 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:49.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:49 smithi022 ceph-mon[37330]: 5.8 scrub starts 2024-09-11T07:38:49.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:49 smithi022 ceph-mon[37330]: 5.1c scrub starts 2024-09-11T07:38:49.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:49 smithi022 ceph-mon[29462]: 5.8 scrub starts 2024-09-11T07:38:49.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:49 smithi022 ceph-mon[29462]: 5.1c scrub starts 2024-09-11T07:38:49.959 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:49.998 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:49.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:49.999 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:50.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:38:49 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:38:50.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:49 smithi104 ceph-mon[33507]: 5.8 scrub starts 2024-09-11T07:38:50.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:49 smithi104 ceph-mon[33507]: 5.1c scrub starts 2024-09-11T07:38:50.099 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:50.134 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:50.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:50.135 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:50.236 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:50.259 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:50.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:50.260 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:50.361 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:50.401 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:50.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:50.402 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:50.485 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 73 16 107724 107708 92.1998 57 0.0182113 0.0108416 2024-09-11T07:38:50.502 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:50.543 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:50.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:50.544 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:50.644 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:50.654 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:38:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:38:50] "GET /metrics HTTP/1.1" 200 224514 "" "Prometheus/2.43.0" 2024-09-11T07:38:50.676 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:50.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:50.677 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:50.778 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:50.807 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:50.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:50.809 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:50.910 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:50.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:50 smithi022 ceph-mon[29462]: 2.10 scrub starts 2024-09-11T07:38:50.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:50 smithi022 ceph-mon[29462]: 4.18 scrub starts 2024-09-11T07:38:50.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:50 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:38:50.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:50 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:38:50.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:50 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:38:50.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:50 smithi022 ceph-mon[29462]: pgmap v552: 158 pgs: 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 152 active+clean; 6.5 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.5 KiB/s rd, 84 MiB/s wr, 1.34k op/s; 38478/320943 objects misplaced (11.989%); 37 MiB/s, 592 objects/s recovering 2024-09-11T07:38:50.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:50 smithi022 ceph-mon[37330]: 2.10 scrub starts 2024-09-11T07:38:50.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:50 smithi022 ceph-mon[37330]: 4.18 scrub starts 2024-09-11T07:38:50.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:50 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:38:50.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:50 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:38:50.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:50 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:38:50.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:50 smithi022 ceph-mon[37330]: pgmap v552: 158 pgs: 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 152 active+clean; 6.5 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.5 KiB/s rd, 84 MiB/s wr, 1.34k op/s; 38478/320943 objects misplaced (11.989%); 37 MiB/s, 592 objects/s recovering 2024-09-11T07:38:50.941 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:50.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:50.942 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:51.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:50 smithi104 ceph-mon[33507]: 2.10 scrub starts 2024-09-11T07:38:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:50 smithi104 ceph-mon[33507]: 4.18 scrub starts 2024-09-11T07:38:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:50 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:38:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:50 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:38:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:50 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:38:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:50 smithi104 ceph-mon[33507]: pgmap v552: 158 pgs: 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 152 active+clean; 6.5 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.5 KiB/s rd, 84 MiB/s wr, 1.34k op/s; 38478/320943 objects misplaced (11.989%); 37 MiB/s, 592 objects/s recovering 2024-09-11T07:38:51.043 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:51.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:51.079 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:51.079 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:51.180 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:51.216 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:51.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:51.217 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:51.318 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:51.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:51.362 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:51.362 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:51.463 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:51.485 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 74 16 108737 108721 91.8094 63.3125 0.0181659 0.0108877 2024-09-11T07:38:51.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:51.497 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:51.498 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:51.598 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:51.634 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:51.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:51.635 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:51.736 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:51.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:51.779 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:51.779 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:51.853 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:51 smithi022 ceph-mon[29462]: 2.c scrub starts 2024-09-11T07:38:51.853 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:51 smithi022 ceph-mon[29462]: osdmap e271: 8 total, 8 up, 8 in 2024-09-11T07:38:51.853 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:51 smithi022 ceph-mon[37330]: 2.c scrub starts 2024-09-11T07:38:51.853 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:51 smithi022 ceph-mon[37330]: osdmap e271: 8 total, 8 up, 8 in 2024-09-11T07:38:51.881 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:51.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:51.910 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:51.910 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:52.011 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:52.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:51 smithi104 ceph-mon[33507]: 2.c scrub starts 2024-09-11T07:38:52.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:51 smithi104 ceph-mon[33507]: osdmap e271: 8 total, 8 up, 8 in 2024-09-11T07:38:52.053 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:52.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:52.054 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:52.155 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:52.184 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:52.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:52.186 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:52.287 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:52.323 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:52.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:52.324 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:52.425 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:52.449 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:52.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:52.450 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:52.485 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 75 16 109793 109777 91.4652 66 0.014174 0.010927 2024-09-11T07:38:52.552 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:52.582 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:52.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:52.583 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:52.675 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is set 2024-09-11T07:38:52.683 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:52.708 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:52.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:52.709 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:52.810 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:52.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:52.851 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:52.851 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:52.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:52 smithi022 ceph-mon[29462]: pgmap v554: 158 pgs: 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 152 active+clean; 6.5 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 682 B/s rd, 44 MiB/s wr, 699 op/s; 38478/320943 objects misplaced (11.989%); 11 MiB/s, 173 objects/s recovering 2024-09-11T07:38:52.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:52 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3104495706' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:38:52.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:52 smithi022 ceph-mon[29462]: from='client.25465 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:38:52.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:52 smithi022 ceph-mon[37330]: pgmap v554: 158 pgs: 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 152 active+clean; 6.5 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 682 B/s rd, 44 MiB/s wr, 699 op/s; 38478/320943 objects misplaced (11.989%); 11 MiB/s, 173 objects/s recovering 2024-09-11T07:38:52.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:52 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3104495706' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:38:52.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:52 smithi022 ceph-mon[37330]: from='client.25465 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:38:52.952 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:52.995 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:52.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:52.997 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:53.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:52 smithi104 ceph-mon[33507]: pgmap v554: 158 pgs: 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 152 active+clean; 6.5 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 682 B/s rd, 44 MiB/s wr, 699 op/s; 38478/320943 objects misplaced (11.989%); 11 MiB/s, 173 objects/s recovering 2024-09-11T07:38:53.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:52 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3104495706' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:38:53.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:52 smithi104 ceph-mon[33507]: from='client.25465 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:38:53.098 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:53.137 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:53.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:53.138 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:53.238 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:53.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:53.280 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:53.280 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:53.381 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:53.425 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:53.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:53.427 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:53.485 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 76 16 110999 110983 91.2534 75.375 0.00276137 0.010949 2024-09-11T07:38:53.527 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:53.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:53.564 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:53.564 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:53.665 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:53.686 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:53.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:53.687 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:53.787 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:53.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:53.828 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:53.828 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:53.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:53 smithi022 ceph-mon[29462]: 4.c scrub starts 2024-09-11T07:38:53.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:53 smithi022 ceph-mon[29462]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:38:53.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:53 smithi022 ceph-mon[29462]: from='client.25465 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:38:53.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:53 smithi022 ceph-mon[29462]: osdmap e272: 8 total, 8 up, 8 in 2024-09-11T07:38:53.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:38:53.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:38:53.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:53 smithi022 ceph-mon[37330]: 4.c scrub starts 2024-09-11T07:38:53.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:53 smithi022 ceph-mon[37330]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:38:53.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:53 smithi022 ceph-mon[37330]: from='client.25465 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:38:53.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:53 smithi022 ceph-mon[37330]: osdmap e272: 8 total, 8 up, 8 in 2024-09-11T07:38:53.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:38:53.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:38:53.929 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:53.953 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:53.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:53.954 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:54.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:53 smithi104 ceph-mon[33507]: 4.c scrub starts 2024-09-11T07:38:54.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:53 smithi104 ceph-mon[33507]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:38:54.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:53 smithi104 ceph-mon[33507]: from='client.25465 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:38:54.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:53 smithi104 ceph-mon[33507]: osdmap e272: 8 total, 8 up, 8 in 2024-09-11T07:38:54.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:38:54.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:38:54.054 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:54.076 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:54.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:54.077 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:54.178 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:54.211 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:54.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:54.213 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:54.313 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:38:54.335 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:54.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:54.336 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:38:54.437 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:54.476 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:54.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:54.477 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:54.485 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 77 16 111920 111904 90.8158 57.5625 0.0169591 0.0110067 2024-09-11T07:38:54.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:38:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:38:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:38:54.578 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:54.620 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:54.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:54.621 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:54.722 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:54.745 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:54.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:54.746 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:54.846 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:54.895 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:54.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:54.896 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:54.904 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:54 smithi104 ceph-mon[33507]: pgmap v556: 158 pgs: 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 152 active+clean; 6.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 622 B/s rd, 51 MiB/s wr, 813 op/s; 38478/324420 objects misplaced (11.861%); 9.9 MiB/s, 158 objects/s recovering 2024-09-11T07:38:54.998 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:55.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:55.027 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:55.027 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:55.128 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:55.165 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:55.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:55.166 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:55.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:54 smithi022 ceph-mon[29462]: pgmap v556: 158 pgs: 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 152 active+clean; 6.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 622 B/s rd, 51 MiB/s wr, 813 op/s; 38478/324420 objects misplaced (11.861%); 9.9 MiB/s, 158 objects/s recovering 2024-09-11T07:38:55.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:54 smithi022 ceph-mon[37330]: pgmap v556: 158 pgs: 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 152 active+clean; 6.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 622 B/s rd, 51 MiB/s wr, 813 op/s; 38478/324420 objects misplaced (11.861%); 9.9 MiB/s, 158 objects/s recovering 2024-09-11T07:38:55.267 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:55.304 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:55.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:55.305 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:55.405 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:55.446 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:55.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:55.447 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:55.486 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 78 16 112947 112931 90.4737 64.1875 0.0188532 0.0110483 2024-09-11T07:38:55.548 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:55.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:55.589 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:55.589 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:55.690 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:55.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:55.738 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:55.738 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:55.744 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-11T07:38:55.839 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:55.878 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:55.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:55.879 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:55.979 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:56.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:56.018 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:56.019 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:56.119 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:56.163 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:56.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:56.164 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:56.265 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:56.301 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:56.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:56.302 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:56.403 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:56.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:56.449 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:56.450 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:56.486 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 79 16 114059 114043 90.2081 69.5 0.0145388 0.0110806 2024-09-11T07:38:56.550 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:56.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:56.587 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:56.587 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:56.688 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:56.713 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:56.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:56.714 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:56.748 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:56 smithi104 ceph-mon[33507]: pgmap v557: 158 pgs: 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 152 active+clean; 6.8 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.1 KiB/s rd, 73 MiB/s wr, 1.17k op/s; 39001/336495 objects misplaced (11.590%); 11 MiB/s, 181 objects/s recovering 2024-09-11T07:38:56.816 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:56.854 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:56.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:56.855 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:56.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:56 smithi022 ceph-mon[29462]: pgmap v557: 158 pgs: 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 152 active+clean; 6.8 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.1 KiB/s rd, 73 MiB/s wr, 1.17k op/s; 39001/336495 objects misplaced (11.590%); 11 MiB/s, 181 objects/s recovering 2024-09-11T07:38:56.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:56 smithi022 ceph-mon[37330]: pgmap v557: 158 pgs: 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 152 active+clean; 6.8 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.1 KiB/s rd, 73 MiB/s wr, 1.17k op/s; 39001/336495 objects misplaced (11.590%); 11 MiB/s, 181 objects/s recovering 2024-09-11T07:38:56.955 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:57.002 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:57.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:57.003 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:57.104 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:57.143 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:57.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:57.144 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:57.245 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:57.284 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:57.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:57.285 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:57.385 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:57.429 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:57.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:57.430 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:57.486 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:2024-09-11T07:38:57.485936+0000 min lat: 0.00139335 max lat: 2.21963 avg lat: 0.0111157 2024-09-11T07:38:57.486 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T07:38:57.486 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 80 16 115133 115117 89.9196 67.125 0.014557 0.0111157 2024-09-11T07:38:57.531 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:57.571 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:57.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:57.572 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:57.673 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:57.708 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:57.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:57.710 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:57.810 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:57.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:57.847 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:57.847 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:57.947 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:58.003 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:58.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:58.004 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:58.105 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:38:58.141 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:38:58.141 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "" 2024-09-11T07:38:58.142 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:38:58.144 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:58.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:58.146 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:38:58.246 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:58.298 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:58.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:58.301 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:58.345 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:38:58.402 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:58.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:58.446 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:58.446 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:58.488 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 81 16 116335 116319 89.7368 75.125 0.00924347 0.0111392 2024-09-11T07:38:58.547 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:38:58.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:58.589 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:58.589 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:38:58.692 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:38:58.735 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:58.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:58.737 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:38:58.837 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:58.876 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:58.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:58.877 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:58.978 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:59.010 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:59.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:59.011 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:59.112 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:59.143 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:59.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:59.144 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:59.245 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:38:59.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:58 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:38:59.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:58 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:38:59.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:58 smithi022 ceph-mon[29462]: pgmap v558: 158 pgs: 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 152 active+clean; 6.8 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 40 MiB/s wr, 648 op/s; 39001/336495 objects misplaced (11.590%); 3.2 MiB/s, 50 objects/s recovering 2024-09-11T07:38:59.252 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:58 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:38:59.252 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:58 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:38:59.253 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:58 smithi022 ceph-mon[37330]: pgmap v558: 158 pgs: 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 152 active+clean; 6.8 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 40 MiB/s wr, 648 op/s; 39001/336495 objects misplaced (11.590%); 3.2 MiB/s, 50 objects/s recovering 2024-09-11T07:38:59.275 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:59.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:59.276 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:38:59.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:58 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:38:59.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:58 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:38:59.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:58 smithi104 ceph-mon[33507]: pgmap v558: 158 pgs: 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 152 active+clean; 6.8 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 40 MiB/s wr, 648 op/s; 39001/336495 objects misplaced (11.590%); 3.2 MiB/s, 50 objects/s recovering 2024-09-11T07:38:59.341 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:38:59.341 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:38:59.377 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:38:59.381 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:38:59.381 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:38:59.409 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:59.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:59.410 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:38:59.415 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:38:59.415 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:38:59.449 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:38:59.449 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:38:59.472 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T07:38:59.482 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:38:59.482 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:38:59.486 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 82 16 117582 117566 89.5928 77.9375 0.0122768 0.011157 2024-09-11T07:38:59.511 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:59.518 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:38:59.518 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:38:59.540 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:59.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:59.541 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:59.549 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:38:59.549 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:38:59.582 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:38:59.582 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:38:59.641 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:59.670 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:59.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:59.671 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:59.772 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:38:59.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:59.821 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:38:59.821 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:38:59.921 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:38:59.957 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:38:59.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:38:59.958 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:38:59.958 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:38:59 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:39:00.059 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:00.102 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:00.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:00.103 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:00.204 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:00.239 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:00.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:00.240 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:00.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:38:59 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1693900239' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:39:00.341 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:00.363 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:00.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:00.364 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:00.373 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:38:59 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1693900239' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:39:00.373 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:38:59 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1693900239' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:39:00.465 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:00.487 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 83 16 118653 118637 89.3198 66.9375 0.00521443 0.011182 2024-09-11T07:39:00.487 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:00.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:00.488 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:00.589 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:00.631 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:00.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:00.633 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:00.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:39:00 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:39:00] "GET /metrics HTTP/1.1" 200 224538 "" "Prometheus/2.43.0" 2024-09-11T07:39:00.736 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:00.765 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:00.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:00.766 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:00.867 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:00.907 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:00.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:00.908 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:01.009 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:01.042 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:01.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:01.043 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:01.144 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:01.183 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:01.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:01.185 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:01.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:00 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:39:01.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:00 smithi104 ceph-mon[33507]: pgmap v559: 158 pgs: 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 152 active+clean; 7.2 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 1.0 KiB/s rd, 71 MiB/s wr, 1.14k op/s; 39816/353556 objects misplaced (11.262%); 5.2 MiB/s, 83 objects/s recovering 2024-09-11T07:39:01.285 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:01.322 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:01.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:01.323 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:01.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:00 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:39:01.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:00 smithi022 ceph-mon[29462]: pgmap v559: 158 pgs: 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 152 active+clean; 7.2 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 1.0 KiB/s rd, 71 MiB/s wr, 1.14k op/s; 39816/353556 objects misplaced (11.262%); 5.2 MiB/s, 83 objects/s recovering 2024-09-11T07:39:01.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:00 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:39:01.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:00 smithi022 ceph-mon[37330]: pgmap v559: 158 pgs: 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 152 active+clean; 7.2 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 1.0 KiB/s rd, 71 MiB/s wr, 1.14k op/s; 39816/353556 objects misplaced (11.262%); 5.2 MiB/s, 83 objects/s recovering 2024-09-11T07:39:01.424 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:01.454 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:01.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:01.456 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:01.487 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 84 16 119794 119778 89.1053 71.3125 0.0159911 0.0112181 2024-09-11T07:39:01.556 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:01.581 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:01.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:01.582 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:01.683 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:01.720 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:01.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:01.721 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:01.821 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:01.865 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:01.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:01.866 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:01.967 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:01.995 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:01.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:01.996 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:02.097 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:02.131 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:02.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:02.133 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:02.233 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:02.269 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:02.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:02.270 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:02.371 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:02.409 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:02.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:02.410 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:02.487 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 85 16 120953 120937 88.9092 72.4375 0.00834795 0.0112427 2024-09-11T07:39:02.511 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:02.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:02.545 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:02.545 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:02.645 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:02.694 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:02.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:02.696 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:02.796 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:02.843 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:02.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:02.844 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:02.946 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:02.980 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:02.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:02.982 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:03.082 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:03.126 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:03.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:03.127 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:03.228 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:03.288 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:03.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:03.289 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:03.390 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:03.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:03.437 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:03.437 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:03.487 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 86 16 122186 122170 88.7713 77.0625 0.0102329 0.01126 2024-09-11T07:39:03.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:03 smithi104 ceph-mon[33507]: pgmap v560: 158 pgs: 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 152 active+clean; 7.2 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 68 MiB/s wr, 1.09k op/s; 39816/353556 objects misplaced (11.262%); 5.0 MiB/s, 79 objects/s recovering 2024-09-11T07:39:03.538 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:03.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:03.579 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:03.579 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:03.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:03 smithi022 ceph-mon[29462]: pgmap v560: 158 pgs: 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 152 active+clean; 7.2 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 68 MiB/s wr, 1.09k op/s; 39816/353556 objects misplaced (11.262%); 5.0 MiB/s, 79 objects/s recovering 2024-09-11T07:39:03.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:03 smithi022 ceph-mon[37330]: pgmap v560: 158 pgs: 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 152 active+clean; 7.2 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 68 MiB/s wr, 1.09k op/s; 39816/353556 objects misplaced (11.262%); 5.0 MiB/s, 79 objects/s recovering 2024-09-11T07:39:03.679 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:03.711 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:03.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:03.712 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:03.797 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:39:03.813 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:03.828 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:39:03.828 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:39:03.829 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_2 2024-09-11T07:39:03.829 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:39:03.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:03.843 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:03.843 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:03.944 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:03.978 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:03.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:03.979 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:04.080 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:04.109 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:04.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:04.110 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:04.211 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:04.244 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:04.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:04.245 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:04.346 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:04.368 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:39:04.379 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:04.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:04.380 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:04.481 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:04.487 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 87 16 123299 123283 88.5505 69.5625 0.0074078 0.0112879 2024-09-11T07:39:04.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:39:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:39:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:39:04.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:04.539 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:04.539 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:04.640 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:04.710 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:04.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:04.711 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:04.812 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:04.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:04.848 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:04.849 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:04.950 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:04.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:04.988 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:04.989 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:05.090 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:05.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:05.122 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:05.122 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:05.223 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:05.262 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:05.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:05.263 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:05.364 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:05.390 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:39:05.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:05.440 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:05.440 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:05.487 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 88 16 124380 124364 88.312 67.5625 0.0305462 0.0113187 2024-09-11T07:39:05.542 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:05.555 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:05 smithi022 ceph-mon[29462]: pgmap v561: 158 pgs: 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 152 active+clean; 7.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 886 B/s rd, 66 MiB/s wr, 1.05k op/s; 39816/357261 objects misplaced (11.145%); 4.3 MiB/s, 69 objects/s recovering 2024-09-11T07:39:05.555 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:05 smithi022 ceph-mon[37330]: pgmap v561: 158 pgs: 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 152 active+clean; 7.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 886 B/s rd, 66 MiB/s wr, 1.05k op/s; 39816/357261 objects misplaced (11.145%); 4.3 MiB/s, 69 objects/s recovering 2024-09-11T07:39:05.580 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:05.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:05.581 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:05.682 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:05.734 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:05.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:05.735 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:05.746 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:39:05.746 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:39:05.746 INFO:teuthology.orchestra.run.smithi022.stdout: "complaint_time": 30, 2024-09-11T07:39:05.747 INFO:teuthology.orchestra.run.smithi022.stdout: "num_blocked_ops": 0 2024-09-11T07:39:05.747 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:39:05.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:05 smithi104 ceph-mon[33507]: pgmap v561: 158 pgs: 2 active+recovering+undersized+remapped, 4 active+remapped+backfill_toofull, 152 active+clean; 7.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 886 B/s rd, 66 MiB/s wr, 1.05k op/s; 39816/357261 objects misplaced (11.145%); 4.3 MiB/s, 69 objects/s recovering 2024-09-11T07:39:05.836 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:05.890 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:05.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:05.891 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:05.992 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:06.029 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:06.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:06.030 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:06.130 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:06.183 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:06.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:06.279 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:06.379 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:06.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:06 smithi022 ceph-mon[29462]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:39:06.399 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:06 smithi022 ceph-mon[37330]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:39:06.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:06.409 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:06.409 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:06.487 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 89 16 125517 125501 88.1181 71.0625 0.00952202 0.0113403 2024-09-11T07:39:06.510 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:06.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:06.563 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:06.563 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:06.664 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:06.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:06.718 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:06.718 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:06.769 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:06 smithi104 ceph-mon[33507]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:39:06.819 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:06.854 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:06.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:06.855 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:06.956 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:06.988 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:06.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:06.989 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:07.090 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:07.121 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:07.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:07.122 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:07.223 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:07.243 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:07.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:07.244 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:07.345 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:07.385 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:07.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:07.386 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:07.487 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:07.488 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 90 11 126588 126577 87.8862 67.25 0.00447286 0.0113723 2024-09-11T07:39:07.516 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:07.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:07.517 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:07.617 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:07 smithi022 ceph-mon[29462]: pgmap v562: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 7.5 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 80 MiB/s wr, 1.28k op/s; 39484/370626 objects misplaced (10.653%); 9.3 MiB/s, 148 objects/s recovering 2024-09-11T07:39:07.617 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:07 smithi022 ceph-mon[37330]: pgmap v562: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 7.5 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 80 MiB/s wr, 1.28k op/s; 39484/370626 objects misplaced (10.653%); 9.3 MiB/s, 148 objects/s recovering 2024-09-11T07:39:07.618 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:07.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:07.649 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:07.650 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:07.750 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:07.775 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:07.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:07.776 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:07.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:07 smithi104 ceph-mon[33507]: pgmap v562: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 7.5 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 80 MiB/s wr, 1.28k op/s; 39484/370626 objects misplaced (10.653%); 9.3 MiB/s, 148 objects/s recovering 2024-09-11T07:39:07.877 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:07.900 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:07.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:07.901 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:08.002 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:08.025 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:08.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:08.026 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:08.127 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:08.151 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:08.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:08.152 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:08.252 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:08.276 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:08.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:08.277 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:08.378 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:08.406 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:08.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:08.407 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:08.487 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Total time run: 90.0308 2024-09-11T07:39:08.487 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Total writes made: 126588 2024-09-11T07:39:08.487 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Write size: 65536 2024-09-11T07:39:08.488 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Object size: 65536 2024-09-11T07:39:08.488 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Bandwidth (MB/sec): 87.8782 2024-09-11T07:39:08.488 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Stddev Bandwidth: 38.7527 2024-09-11T07:39:08.488 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Max bandwidth (MB/sec): 227 2024-09-11T07:39:08.488 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Min bandwidth (MB/sec): 0 2024-09-11T07:39:08.488 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Average IOPS: 1406 2024-09-11T07:39:08.488 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Stddev IOPS: 620.056 2024-09-11T07:39:08.488 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Max IOPS: 3632 2024-09-11T07:39:08.488 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Min IOPS: 0 2024-09-11T07:39:08.488 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Average Latency(s): 0.011374 2024-09-11T07:39:08.488 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Stddev Latency(s): 0.031069 2024-09-11T07:39:08.488 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Max latency(s): 2.21963 2024-09-11T07:39:08.488 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Min latency(s): 0.00139335 2024-09-11T07:39:08.492 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Cleaning up (deleting benchmark objects) 2024-09-11T07:39:08.507 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:08.545 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:08.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:08.546 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:08.647 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:08.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:39:08.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:39:08.689 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:08.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:08.691 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:08.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:39:08.792 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:08.830 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:08.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:08.831 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:08.932 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:08.953 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:08.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:08.954 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:09.056 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:09.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:09.091 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:09.091 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:09.192 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:09.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:09.226 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:09.226 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:09.327 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:09.356 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:09.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:09.357 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:09.458 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:09.498 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:09.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:09.499 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:09.600 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:09.622 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:09.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:09.624 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:09.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:09 smithi022 ceph-mon[29462]: pgmap v563: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 7.5 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 853 B/s rd, 59 MiB/s wr, 948 op/s; 39484/370626 objects misplaced (10.653%); 7.1 MiB/s, 114 objects/s recovering 2024-09-11T07:39:09.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:09 smithi022 ceph-mon[37330]: pgmap v563: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 7.5 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 853 B/s rd, 59 MiB/s wr, 948 op/s; 39484/370626 objects misplaced (10.653%); 7.1 MiB/s, 114 objects/s recovering 2024-09-11T07:39:09.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:09 smithi104 ceph-mon[33507]: pgmap v563: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 7.5 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 853 B/s rd, 59 MiB/s wr, 948 op/s; 39484/370626 objects misplaced (10.653%); 7.1 MiB/s, 114 objects/s recovering 2024-09-11T07:39:09.724 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:09.756 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:09.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:09.757 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:09.858 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:09.892 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:09.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:09.893 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:09.994 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:10.025 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:10.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:10.026 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:10.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:39:09 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:39:10.127 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:10.167 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:10.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:10.168 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:10.269 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:10.299 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:10.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:10.300 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:10.401 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:10.434 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:10.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:10.435 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:10.536 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:10.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:10.577 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:10.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:39:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:39:10] "GET /metrics HTTP/1.1" 200 224535 "" "Prometheus/2.43.0" 2024-09-11T07:39:10.678 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:10.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:10.711 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:10.711 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:10.791 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:39:10.812 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:10.867 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:10.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:10.868 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:10.969 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:11.001 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:11.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:11.002 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:11.103 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:11.142 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:11.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:11.144 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:11.244 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:11.293 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:11.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:11.294 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:11.395 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:11.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:11.428 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:11.428 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:11.529 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:11.560 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:11.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:11.561 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:11.575 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:11 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:39:11.575 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:11 smithi022 ceph-mon[29462]: pgmap v564: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 7.7 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.3 KiB/s rd, 76 MiB/s wr, 1.32k op/s; 36978/376599 objects misplaced (9.819%); 15 MiB/s, 241 objects/s recovering 2024-09-11T07:39:11.575 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:11 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:39:11.575 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:11 smithi022 ceph-mon[37330]: pgmap v564: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 7.7 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.3 KiB/s rd, 76 MiB/s wr, 1.32k op/s; 36978/376599 objects misplaced (9.819%); 15 MiB/s, 241 objects/s recovering 2024-09-11T07:39:11.662 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:11.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:11.711 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:11.711 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:11.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:11 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:39:11.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:11 smithi104 ceph-mon[33507]: pgmap v564: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 7.7 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.3 KiB/s rd, 76 MiB/s wr, 1.32k op/s; 36978/376599 objects misplaced (9.819%); 15 MiB/s, 241 objects/s recovering 2024-09-11T07:39:11.812 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:11.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:11.855 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:11.855 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:11.956 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:11.986 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:11.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:11.987 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:12.088 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:12.119 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:12.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:12.120 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:12.221 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:12.250 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:12.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:12.251 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:12.352 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:12.374 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is set 2024-09-11T07:39:12.411 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:12.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:12.414 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:12.515 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:12.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:12.549 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:12.549 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:12.641 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:12 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2457315668' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:39:12.642 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:12 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2457315668' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:39:12.653 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:12.655 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:39:12.658 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:39:12.735 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:12.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:12.736 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:12.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:12 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2457315668' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:39:12.837 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:12.867 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:12.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:12.868 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:12.969 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:13.008 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:13.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:13.009 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:13.110 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:13.139 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:13.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:13.140 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:13.241 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:13.283 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:13.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:13.284 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:13.385 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:13.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:13.445 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:13.445 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:13.546 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:13.594 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:13.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:13.595 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:13.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:13 smithi022 ceph-mon[29462]: pgmap v565: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 7.7 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 938 B/s rd, 47 MiB/s wr, 849 op/s; 36978/376599 objects misplaced (9.819%); 13 MiB/s, 208 objects/s recovering 2024-09-11T07:39:13.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:13 smithi022 ceph-mon[29462]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:39:13.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:13 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2457315668' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:39:13.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:13 smithi022 ceph-mon[29462]: osdmap e273: 8 total, 8 up, 8 in 2024-09-11T07:39:13.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:13 smithi022 ceph-mon[37330]: pgmap v565: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 7.7 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 938 B/s rd, 47 MiB/s wr, 849 op/s; 36978/376599 objects misplaced (9.819%); 13 MiB/s, 208 objects/s recovering 2024-09-11T07:39:13.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:13 smithi022 ceph-mon[37330]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:39:13.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:13 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2457315668' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:39:13.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:13 smithi022 ceph-mon[37330]: osdmap e273: 8 total, 8 up, 8 in 2024-09-11T07:39:13.696 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:13.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:13.735 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:13.735 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:13.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:13 smithi104 ceph-mon[33507]: pgmap v565: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 7.7 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 938 B/s rd, 47 MiB/s wr, 849 op/s; 36978/376599 objects misplaced (9.819%); 13 MiB/s, 208 objects/s recovering 2024-09-11T07:39:13.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:13 smithi104 ceph-mon[33507]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:39:13.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:13 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2457315668' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:39:13.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:13 smithi104 ceph-mon[33507]: osdmap e273: 8 total, 8 up, 8 in 2024-09-11T07:39:13.836 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:13.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:13.868 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:13.868 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:13.969 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:13.993 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-11T07:39:13.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:13.998 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:13.999 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:14.100 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:14.153 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:14.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:14.155 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:14.256 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:14.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:14.287 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:14.287 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:14.373 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:39:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:39:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:39:14.388 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:14.436 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:14.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:14.437 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:14.538 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:14.581 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:14.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:14.582 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:14.649 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:14 smithi022 ceph-mon[37330]: from='client.25480 v1:172.21.15.22:0/63087126' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:39:14.649 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:14 smithi022 ceph-mon[29462]: from='client.25480 v1:172.21.15.22:0/63087126' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:39:14.683 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:14.714 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:14.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:14.715 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:14.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:14 smithi104 ceph-mon[33507]: from='client.25480 v1:172.21.15.22:0/63087126' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:39:14.816 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:14.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:14.866 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:14.866 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:14.967 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:15.008 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:15.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:15.009 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:15.110 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:15.153 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:15.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:15.154 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:15.255 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:15.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:15.287 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:15.287 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:15.297 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":566,"stamp":"2024-09-11T07:39:12.359120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8214616308,"num_objects":125533,"num_object_clones":0,"num_object_copies":376599,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":36978,"num_objects_unfound":0,"num_objects_dirty":125533,"num_whiteouts":0,"num_read":2029,"num_read_kb":1886,"num_write":128670,"num_write_kb":8103304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10690,"num_bytes_recovered":698090594,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154135,"ondisk_log_size":154135,"up":474,"acting":472,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":490,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":28045744,"kb_used_data":25334576,"kb_used_omap":16,"kb_used_meta":2710895,"kb_avail":721915472,"statfs":{"total":767960285184,"available":739241443328,"internally_reserved":0,"allocated":25942605824,"data_stored":25939752844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16530,"internal_metadata":2775957358},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,6,3,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":503316504,"num_objects":7681,"num_object_clones":0,"num_object_copies":23043,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-2838,"num_objects_unfound":0,"num_objects_dirty":7681,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":10185,"num_write_kb":571649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2505,"num_bytes_recovered":164167680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.144332"},"pg_stats":[{"pgid":"9.14","version":"272'6703","reported_seq":6743,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:39:09.505831+0000","last_change":"2024-09-11T07:38:27.393848+0000","last_active":"2024-09-11T07:39:09.505831+0000","last_peered":"2024-09-11T07:39:09.505831+0000","last_clean":"2024-09-11T07:39:09.505831+0000","last_became_active":"2024-09-11T07:38:27.390349+0000","last_became_peered":"2024-09-11T07:38:27.390349+0000","last_unstale":"2024-09-11T07:39:09.505831+0000","last_undegraded":"2024-09-11T07:39:09.505831+0000","last_fullsized":"2024-09-11T07:39:09.505831+0000","mapping_epoch":259,"log_start":"254'500","ondisk_log_start":"254'500","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6203,"log_dups_size":0,"ondisk_log_size":6203,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258211840,"num_objects":3940,"num_object_clones":0,"num_object_copies":11820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4016,"num_write_kb":254592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":274,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.665873+0000","last_change":"2024-09-11T07:37:29.007467+0000","last_active":"2024-09-11T07:38:50.665873+0000","last_peered":"2024-09-11T07:38:50.665873+0000","last_clean":"2024-09-11T07:38:50.665873+0000","last_became_active":"2024-09-11T07:32:00.501534+0000","last_became_peered":"2024-09-11T07:32:00.501534+0000","last_unstale":"2024-09-11T07:38:50.665873+0000","last_undegraded":"2024-09-11T07:38:50.665873+0000","last_fullsized":"2024-09-11T07:38:50.665873+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:29.007411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:29.007411+0000","last_clean_scrub_stamp":"2024-09-11T07:37:29.007411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003998723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":300,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.669671+0000","last_change":"2024-09-11T07:38:27.418908+0000","last_active":"2024-09-11T07:38:50.669671+0000","last_peered":"2024-09-11T07:38:50.669671+0000","last_clean":"2024-09-11T07:38:50.669671+0000","last_became_active":"2024-09-11T07:32:00.508572+0000","last_became_peered":"2024-09-11T07:32:00.508572+0000","last_unstale":"2024-09-11T07:38:50.669671+0000","last_undegraded":"2024-09-11T07:38:50.669671+0000","last_fullsized":"2024-09-11T07:38:50.669671+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:27.418829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:38:27.418829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:43.594097+0000","scrub_duration":0.0036972480000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"234'34","reported_seq":311,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.669262+0000","last_change":"2024-09-11T07:36:28.772225+0000","last_active":"2024-09-11T07:38:50.669262+0000","last_peered":"2024-09-11T07:38:50.669262+0000","last_clean":"2024-09-11T07:38:50.669262+0000","last_became_active":"2024-09-11T07:26:33.544627+0000","last_became_peered":"2024-09-11T07:26:33.544627+0000","last_unstale":"2024-09-11T07:38:50.669262+0000","last_undegraded":"2024-09-11T07:38:50.669262+0000","last_fullsized":"2024-09-11T07:38:50.669262+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-09-11T07:36:28.772126+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:36:28.772126+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016007773999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":279,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.665055+0000","last_change":"2024-09-11T07:37:39.609450+0000","last_active":"2024-09-11T07:38:50.665055+0000","last_peered":"2024-09-11T07:38:50.665055+0000","last_clean":"2024-09-11T07:38:50.665055+0000","last_became_active":"2024-09-11T07:26:35.556296+0000","last_became_peered":"2024-09-11T07:26:35.556296+0000","last_unstale":"2024-09-11T07:38:50.665055+0000","last_undegraded":"2024-09-11T07:38:50.665055+0000","last_fullsized":"2024-09-11T07:38:50.665055+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:39.609357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:39.609357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026059439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.15","version":"272'6719","reported_seq":10934,"reported_epoch":272,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T07:39:09.286148+0000","last_change":"2024-09-11T07:38:30.550092+0000","last_active":"2024-09-11T07:39:09.286148+0000","last_peered":"2024-09-11T07:39:09.286148+0000","last_clean":"2024-09-11T07:38:28.297716+0000","last_became_active":"2024-09-11T07:38:30.505780+0000","last_became_peered":"2024-09-11T07:38:30.505780+0000","last_unstale":"2024-09-11T07:39:09.286148+0000","last_undegraded":"2024-09-11T07:39:09.286148+0000","last_fullsized":"2024-09-11T07:38:30.343082+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6719,"log_dups_size":0,"ondisk_log_size":6719,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":259588096,"num_objects":3961,"num_object_clones":0,"num_object_copies":11883,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2728,"num_objects_unfound":0,"num_objects_dirty":3961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4005,"num_write_kb":254912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1388,"num_bytes_recovered":90963968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1],"avail_no_missing":["7","0","5"],"object_location_counts":[{"shards":"0,5,7","objects":1364},{"shards":"1,7","objects":2597}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":273,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.667793+0000","last_change":"2024-09-11T07:36:33.755073+0000","last_active":"2024-09-11T07:38:50.667793+0000","last_peered":"2024-09-11T07:38:50.667793+0000","last_clean":"2024-09-11T07:38:50.667793+0000","last_became_active":"2024-09-11T07:26:31.420178+0000","last_became_peered":"2024-09-11T07:26:31.420178+0000","last_unstale":"2024-09-11T07:38:50.667793+0000","last_undegraded":"2024-09-11T07:38:50.667793+0000","last_fullsized":"2024-09-11T07:38:50.667793+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:33.754983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:33.754983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034784899999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"234'19","reported_seq":303,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.665131+0000","last_change":"2024-09-11T07:38:49.475617+0000","last_active":"2024-09-11T07:38:50.665131+0000","last_peered":"2024-09-11T07:38:50.665131+0000","last_clean":"2024-09-11T07:38:50.665131+0000","last_became_active":"2024-09-11T07:26:33.540732+0000","last_became_peered":"2024-09-11T07:26:33.540732+0000","last_unstale":"2024-09-11T07:38:50.665131+0000","last_undegraded":"2024-09-11T07:38:50.665131+0000","last_fullsized":"2024-09-11T07:38:50.665131+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:36:42.708941+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:36:42.708941+0000","last_clean_scrub_stamp":"2024-09-11T07:36:42.708941+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0088444879999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":288,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.667865+0000","last_change":"2024-09-11T07:37:43.087423+0000","last_active":"2024-09-11T07:38:50.667865+0000","last_peered":"2024-09-11T07:38:50.667865+0000","last_clean":"2024-09-11T07:38:50.667865+0000","last_became_active":"2024-09-11T07:32:00.513650+0000","last_became_peered":"2024-09-11T07:32:00.513650+0000","last_unstale":"2024-09-11T07:38:50.667865+0000","last_undegraded":"2024-09-11T07:38:50.667865+0000","last_fullsized":"2024-09-11T07:38:50.667865+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:43.087302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:37:43.087302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00405653,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":267,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.665576+0000","last_change":"2024-09-11T07:36:48.684925+0000","last_active":"2024-09-11T07:38:50.665576+0000","last_peered":"2024-09-11T07:38:50.665576+0000","last_clean":"2024-09-11T07:38:50.665576+0000","last_became_active":"2024-09-11T07:26:37.566910+0000","last_became_peered":"2024-09-11T07:26:37.566910+0000","last_unstale":"2024-09-11T07:38:50.665576+0000","last_undegraded":"2024-09-11T07:38:50.665576+0000","last_fullsized":"2024-09-11T07:38:50.665576+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:48.684838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:48.684838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058146650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"272'6808","reported_seq":13014,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:39:09.082348+0000","last_change":"2024-09-11T07:38:47.691052+0000","last_active":"2024-09-11T07:39:09.082348+0000","last_peered":"2024-09-11T07:39:09.082348+0000","last_clean":"2024-09-11T07:39:09.082348+0000","last_became_active":"2024-09-11T07:38:47.687489+0000","last_became_peered":"2024-09-11T07:38:47.687489+0000","last_unstale":"2024-09-11T07:39:09.082348+0000","last_undegraded":"2024-09-11T07:39:09.082348+0000","last_fullsized":"2024-09-11T07:39:09.082348+0000","mapping_epoch":269,"log_start":"254'600","ondisk_log_start":"254'600","created":259,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262799360,"num_objects":4010,"num_object_clones":0,"num_object_copies":12030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4044,"num_write_kb":257728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3072,"num_bytes_recovered":201326592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":278,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.660624+0000","last_change":"2024-09-11T07:37:34.542508+0000","last_active":"2024-09-11T07:38:50.660624+0000","last_peered":"2024-09-11T07:38:50.660624+0000","last_clean":"2024-09-11T07:38:50.660624+0000","last_became_active":"2024-09-11T07:32:00.516662+0000","last_became_peered":"2024-09-11T07:32:00.516662+0000","last_unstale":"2024-09-11T07:38:50.660624+0000","last_undegraded":"2024-09-11T07:38:50.660624+0000","last_fullsized":"2024-09-11T07:38:50.660624+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:34.542441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:34.542441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:53.707135+0000","scrub_duration":0.0015726010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"234'9","reported_seq":296,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.666071+0000","last_change":"2024-09-11T07:37:37.558147+0000","last_active":"2024-09-11T07:38:50.666071+0000","last_peered":"2024-09-11T07:38:50.666071+0000","last_clean":"2024-09-11T07:38:50.666071+0000","last_became_active":"2024-09-11T07:26:33.540351+0000","last_became_peered":"2024-09-11T07:26:33.540351+0000","last_unstale":"2024-09-11T07:38:50.666071+0000","last_undegraded":"2024-09-11T07:38:50.666071+0000","last_fullsized":"2024-09-11T07:38:50.666071+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'9","last_scrub_stamp":"2024-09-11T07:37:37.558036+0000","last_deep_scrub":"82'7","last_deep_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_clean_scrub_stamp":"2024-09-11T07:37:37.558036+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034408770000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":244,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.658987+0000","last_change":"2024-09-11T07:35:50.332326+0000","last_active":"2024-09-11T07:38:50.658987+0000","last_peered":"2024-09-11T07:38:50.658987+0000","last_clean":"2024-09-11T07:38:50.658987+0000","last_became_active":"2024-09-11T07:32:00.513206+0000","last_became_peered":"2024-09-11T07:32:00.513206+0000","last_unstale":"2024-09-11T07:38:50.658987+0000","last_undegraded":"2024-09-11T07:38:50.658987+0000","last_fullsized":"2024-09-11T07:38:50.658987+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:50.332244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.973323+0000","last_clean_scrub_stamp":"2024-09-11T07:35:50.332244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034655860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":267,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.669028+0000","last_change":"2024-09-11T07:37:36.462873+0000","last_active":"2024-09-11T07:38:50.669028+0000","last_peered":"2024-09-11T07:38:50.669028+0000","last_clean":"2024-09-11T07:38:50.669028+0000","last_became_active":"2024-09-11T07:26:37.561758+0000","last_became_peered":"2024-09-11T07:26:37.561758+0000","last_unstale":"2024-09-11T07:38:50.669028+0000","last_undegraded":"2024-09-11T07:38:50.669028+0000","last_fullsized":"2024-09-11T07:38:50.669028+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:36.462794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:45.149916+0000","last_clean_scrub_stamp":"2024-09-11T07:37:36.462794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001718252,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.17","version":"272'6715","reported_seq":6774,"reported_epoch":272,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-11T07:39:09.257244+0000","last_change":"2024-09-11T07:39:00.429569+0000","last_active":"2024-09-11T07:39:09.257244+0000","last_peered":"2024-09-11T07:39:09.257244+0000","last_clean":"2024-09-11T07:38:28.336632+0000","last_became_active":"2024-09-11T07:38:30.420725+0000","last_became_peered":"2024-09-11T07:38:30.420725+0000","last_unstale":"2024-09-11T07:39:09.257244+0000","last_undegraded":"2024-09-11T07:39:09.257244+0000","last_fullsized":"2024-09-11T07:39:09.257244+0000","mapping_epoch":262,"log_start":"254'400","ondisk_log_start":"254'400","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6315,"log_dups_size":0,"ondisk_log_size":6315,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":259063808,"num_objects":3953,"num_object_clones":0,"num_object_copies":11859,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7906,"num_objects_unfound":0,"num_objects_dirty":3953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4011,"num_write_kb":254848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,3,5],"avail_no_missing":["7","3","5"],"object_location_counts":[{"shards":"3,5,7","objects":3953}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":243,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.669464+0000","last_change":"2024-09-11T07:37:40.336139+0000","last_active":"2024-09-11T07:38:50.669464+0000","last_peered":"2024-09-11T07:38:50.669464+0000","last_clean":"2024-09-11T07:38:50.669464+0000","last_became_active":"2024-09-11T07:32:00.505709+0000","last_became_peered":"2024-09-11T07:32:00.505709+0000","last_unstale":"2024-09-11T07:38:50.669464+0000","last_undegraded":"2024-09-11T07:38:50.669464+0000","last_fullsized":"2024-09-11T07:38:50.669464+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:40.336050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_clean_scrub_stamp":"2024-09-11T07:37:40.336050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:58.024969+0000","scrub_duration":0.0021490649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"234'19","reported_seq":300,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.665850+0000","last_change":"2024-09-11T07:36:27.746378+0000","last_active":"2024-09-11T07:38:50.665850+0000","last_peered":"2024-09-11T07:38:50.665850+0000","last_clean":"2024-09-11T07:38:50.665850+0000","last_became_active":"2024-09-11T07:26:33.544386+0000","last_became_peered":"2024-09-11T07:26:33.544386+0000","last_unstale":"2024-09-11T07:38:50.665850+0000","last_undegraded":"2024-09-11T07:38:50.665850+0000","last_fullsized":"2024-09-11T07:38:50.665850+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'13","last_scrub_stamp":"2024-09-11T07:36:27.746295+0000","last_deep_scrub":"136'13","last_deep_scrub_stamp":"2024-09-11T07:31:08.042198+0000","last_clean_scrub_stamp":"2024-09-11T07:36:27.746295+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0089695690000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":289,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.669333+0000","last_change":"2024-09-11T07:38:27.145934+0000","last_active":"2024-09-11T07:38:50.669333+0000","last_peered":"2024-09-11T07:38:50.669333+0000","last_clean":"2024-09-11T07:38:50.669333+0000","last_became_active":"2024-09-11T07:32:00.502337+0000","last_became_peered":"2024-09-11T07:32:00.502337+0000","last_unstale":"2024-09-11T07:38:50.669333+0000","last_undegraded":"2024-09-11T07:38:50.669333+0000","last_fullsized":"2024-09-11T07:38:50.669333+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:27.145883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:38:27.145883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:54.477246+0000","scrub_duration":0.0011677339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":284,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.668970+0000","last_change":"2024-09-11T07:36:31.977319+0000","last_active":"2024-09-11T07:38:50.668970+0000","last_peered":"2024-09-11T07:38:50.668970+0000","last_clean":"2024-09-11T07:38:50.668970+0000","last_became_active":"2024-09-11T07:32:00.530335+0000","last_became_peered":"2024-09-11T07:32:00.530335+0000","last_unstale":"2024-09-11T07:38:50.668970+0000","last_undegraded":"2024-09-11T07:38:50.668970+0000","last_fullsized":"2024-09-11T07:38:50.668970+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:31.977240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:31.977240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0074114259999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"272'6739","reported_seq":6783,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:39:09.492231+0000","last_change":"2024-09-11T07:38:27.363670+0000","last_active":"2024-09-11T07:39:09.492231+0000","last_peered":"2024-09-11T07:39:09.492231+0000","last_clean":"2024-09-11T07:39:09.492231+0000","last_became_active":"2024-09-11T07:38:27.360167+0000","last_became_peered":"2024-09-11T07:38:27.360167+0000","last_unstale":"2024-09-11T07:39:09.492231+0000","last_undegraded":"2024-09-11T07:39:09.492231+0000","last_fullsized":"2024-09-11T07:39:09.492231+0000","mapping_epoch":247,"log_start":"254'500","ondisk_log_start":"254'500","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6239,"log_dups_size":0,"ondisk_log_size":6239,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":257294336,"num_objects":3926,"num_object_clones":0,"num_object_copies":11778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4020,"num_write_kb":254272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"234'31","reported_seq":376,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.666671+0000","last_change":"2024-09-11T07:36:39.737025+0000","last_active":"2024-09-11T07:38:50.666671+0000","last_peered":"2024-09-11T07:38:50.666671+0000","last_clean":"2024-09-11T07:38:50.666671+0000","last_became_active":"2024-09-11T07:29:51.482755+0000","last_became_peered":"2024-09-11T07:29:51.482755+0000","last_unstale":"2024-09-11T07:38:50.666671+0000","last_undegraded":"2024-09-11T07:38:50.666671+0000","last_fullsized":"2024-09-11T07:38:50.666671+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'21","last_scrub_stamp":"2024-09-11T07:36:39.736944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:36:39.736944+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034170279999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":273,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.660456+0000","last_change":"2024-09-11T07:36:51.418345+0000","last_active":"2024-09-11T07:38:50.660456+0000","last_peered":"2024-09-11T07:38:50.660456+0000","last_clean":"2024-09-11T07:38:50.660456+0000","last_became_active":"2024-09-11T07:32:00.507191+0000","last_became_peered":"2024-09-11T07:32:00.507191+0000","last_unstale":"2024-09-11T07:38:50.660456+0000","last_undegraded":"2024-09-11T07:38:50.660456+0000","last_fullsized":"2024-09-11T07:38:50.660456+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:51.418258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:51.418258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001500181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":271,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.670710+0000","last_change":"2024-09-11T07:38:22.027909+0000","last_active":"2024-09-11T07:38:50.670710+0000","last_peered":"2024-09-11T07:38:50.670710+0000","last_clean":"2024-09-11T07:38:50.670710+0000","last_became_active":"2024-09-11T07:26:35.557989+0000","last_became_peered":"2024-09-11T07:26:35.557989+0000","last_unstale":"2024-09-11T07:38:50.670710+0000","last_undegraded":"2024-09-11T07:38:50.670710+0000","last_fullsized":"2024-09-11T07:38:50.670710+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:22.027769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:38:22.027769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:26.275069+0000","scrub_duration":0.0060071020000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":280,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.663367+0000","last_change":"2024-09-11T07:38:42.988126+0000","last_active":"2024-09-11T07:38:50.663367+0000","last_peered":"2024-09-11T07:38:50.663367+0000","last_clean":"2024-09-11T07:38:50.663367+0000","last_became_active":"2024-09-11T07:26:37.564554+0000","last_became_peered":"2024-09-11T07:26:37.564554+0000","last_unstale":"2024-09-11T07:38:50.663367+0000","last_undegraded":"2024-09-11T07:38:50.663367+0000","last_fullsized":"2024-09-11T07:38:50.663367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:21.385271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:21.385271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001463322,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"272'6609","reported_seq":6652,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:39:09.855389+0000","last_change":"2024-09-11T07:38:27.369381+0000","last_active":"2024-09-11T07:39:09.855389+0000","last_peered":"2024-09-11T07:39:09.855389+0000","last_clean":"2024-09-11T07:39:09.855389+0000","last_became_active":"2024-09-11T07:38:27.365510+0000","last_became_peered":"2024-09-11T07:38:27.365510+0000","last_unstale":"2024-09-11T07:39:09.855389+0000","last_undegraded":"2024-09-11T07:39:09.855389+0000","last_fullsized":"2024-09-11T07:39:09.855389+0000","mapping_epoch":247,"log_start":"254'500","ondisk_log_start":"254'500","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6109,"log_dups_size":0,"ondisk_log_size":6109,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":251265024,"num_objects":3834,"num_object_clones":0,"num_object_copies":11502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3932,"num_write_kb":248512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"234'25","reported_seq":309,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:52.528116+0000","last_change":"2024-09-11T07:38:52.528090+0000","last_active":"2024-09-11T07:38:52.528116+0000","last_peered":"2024-09-11T07:38:52.528116+0000","last_clean":"2024-09-11T07:38:52.528116+0000","last_became_active":"2024-09-11T07:26:33.535411+0000","last_became_peered":"2024-09-11T07:26:33.535411+0000","last_unstale":"2024-09-11T07:38:52.528116+0000","last_undegraded":"2024-09-11T07:38:52.528116+0000","last_fullsized":"2024-09-11T07:38:52.528116+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'25","last_scrub_stamp":"2024-09-11T07:36:43.704170+0000","last_deep_scrub":"112'15","last_deep_scrub_stamp":"2024-09-11T07:31:10.005896+0000","last_clean_scrub_stamp":"2024-09-11T07:36:43.704170+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048642039999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":292,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.669764+0000","last_change":"2024-09-11T07:37:36.098229+0000","last_active":"2024-09-11T07:38:50.669764+0000","last_peered":"2024-09-11T07:38:50.669764+0000","last_clean":"2024-09-11T07:38:50.669764+0000","last_became_active":"2024-09-11T07:32:00.506615+0000","last_became_peered":"2024-09-11T07:32:00.506615+0000","last_unstale":"2024-09-11T07:38:50.669764+0000","last_undegraded":"2024-09-11T07:38:50.669764+0000","last_fullsized":"2024-09-11T07:38:50.669764+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:36.098175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:36.098175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:54.532896+0000","scrub_duration":0.0017555839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"151'10","reported_seq":457,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:38:59.653394+0000","last_change":"2024-09-11T07:37:43.141088+0000","last_active":"2024-09-11T07:38:59.653394+0000","last_peered":"2024-09-11T07:38:59.653394+0000","last_clean":"2024-09-11T07:38:59.653394+0000","last_became_active":"2024-09-11T07:32:00.507447+0000","last_became_peered":"2024-09-11T07:32:00.507447+0000","last_unstale":"2024-09-11T07:38:59.653394+0000","last_undegraded":"2024-09-11T07:38:59.653394+0000","last_fullsized":"2024-09-11T07:38:59.653394+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:37:43.140999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:43.140999+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039651130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":267,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.667438+0000","last_change":"2024-09-11T07:37:22.546982+0000","last_active":"2024-09-11T07:38:50.667438+0000","last_peered":"2024-09-11T07:38:50.667438+0000","last_clean":"2024-09-11T07:38:50.667438+0000","last_became_active":"2024-09-11T07:26:37.566919+0000","last_became_peered":"2024-09-11T07:26:37.566919+0000","last_unstale":"2024-09-11T07:38:50.667438+0000","last_undegraded":"2024-09-11T07:38:50.667438+0000","last_fullsized":"2024-09-11T07:38:50.667438+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:22.546931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:22.546931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001345527,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"272'6626","reported_seq":6668,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:39:09.891511+0000","last_change":"2024-09-11T07:38:27.363945+0000","last_active":"2024-09-11T07:39:09.891511+0000","last_peered":"2024-09-11T07:39:09.891511+0000","last_clean":"2024-09-11T07:39:09.891511+0000","last_became_active":"2024-09-11T07:38:27.353859+0000","last_became_peered":"2024-09-11T07:38:27.353859+0000","last_unstale":"2024-09-11T07:39:09.891511+0000","last_undegraded":"2024-09-11T07:39:09.891511+0000","last_fullsized":"2024-09-11T07:39:09.891511+0000","mapping_epoch":247,"log_start":"254'400","ondisk_log_start":"254'400","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6226,"log_dups_size":0,"ondisk_log_size":6226,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":252444672,"num_objects":3852,"num_object_clones":0,"num_object_copies":11556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3964,"num_write_kb":250112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"234'34","reported_seq":320,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.668703+0000","last_change":"2024-09-11T07:37:16.938565+0000","last_active":"2024-09-11T07:38:50.668703+0000","last_peered":"2024-09-11T07:38:50.668703+0000","last_clean":"2024-09-11T07:38:50.668703+0000","last_became_active":"2024-09-11T07:26:33.549733+0000","last_became_peered":"2024-09-11T07:26:33.549733+0000","last_unstale":"2024-09-11T07:38:50.668703+0000","last_undegraded":"2024-09-11T07:38:50.668703+0000","last_fullsized":"2024-09-11T07:38:50.668703+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'34","last_scrub_stamp":"2024-09-11T07:37:16.938496+0000","last_deep_scrub":"83'22","last_deep_scrub_stamp":"2024-09-11T07:34:04.375457+0000","last_clean_scrub_stamp":"2024-09-11T07:37:16.938496+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028468529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":302,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.668039+0000","last_change":"2024-09-11T07:38:29.939925+0000","last_active":"2024-09-11T07:38:50.668039+0000","last_peered":"2024-09-11T07:38:50.668039+0000","last_clean":"2024-09-11T07:38:50.668039+0000","last_became_active":"2024-09-11T07:32:00.513472+0000","last_became_peered":"2024-09-11T07:32:00.513472+0000","last_unstale":"2024-09-11T07:38:50.668039+0000","last_undegraded":"2024-09-11T07:38:50.668039+0000","last_fullsized":"2024-09-11T07:38:50.668039+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:29.939826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:38:29.939826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:57.256068+0000","scrub_duration":0.0017750210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":431,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:38:59.653491+0000","last_change":"2024-09-11T07:37:30.657587+0000","last_active":"2024-09-11T07:38:59.653491+0000","last_peered":"2024-09-11T07:38:59.653491+0000","last_clean":"2024-09-11T07:38:59.653491+0000","last_became_active":"2024-09-11T07:26:35.560406+0000","last_became_peered":"2024-09-11T07:26:35.560406+0000","last_unstale":"2024-09-11T07:38:59.653491+0000","last_undegraded":"2024-09-11T07:38:59.653491+0000","last_fullsized":"2024-09-11T07:38:59.653491+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:37:30.657477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:30.657477+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001943012,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":267,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.668701+0000","last_change":"2024-09-11T07:37:20.369255+0000","last_active":"2024-09-11T07:38:50.668701+0000","last_peered":"2024-09-11T07:38:50.668701+0000","last_clean":"2024-09-11T07:38:50.668701+0000","last_became_active":"2024-09-11T07:26:37.568005+0000","last_became_peered":"2024-09-11T07:26:37.568005+0000","last_unstale":"2024-09-11T07:38:50.668701+0000","last_undegraded":"2024-09-11T07:38:50.668701+0000","last_fullsized":"2024-09-11T07:38:50.668701+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:20.369199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:31.200685+0000","last_clean_scrub_stamp":"2024-09-11T07:37:20.369199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001371688,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"272'6562","reported_seq":6605,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:39:09.279236+0000","last_change":"2024-09-11T07:38:27.363719+0000","last_active":"2024-09-11T07:39:09.279236+0000","last_peered":"2024-09-11T07:39:09.279236+0000","last_clean":"2024-09-11T07:39:09.279236+0000","last_became_active":"2024-09-11T07:38:27.361246+0000","last_became_peered":"2024-09-11T07:38:27.361246+0000","last_unstale":"2024-09-11T07:39:09.279236+0000","last_undegraded":"2024-09-11T07:39:09.279236+0000","last_fullsized":"2024-09-11T07:39:09.279236+0000","mapping_epoch":247,"log_start":"254'600","ondisk_log_start":"254'600","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":5962,"log_dups_size":0,"ondisk_log_size":5962,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":250576896,"num_objects":3823,"num_object_clones":0,"num_object_copies":11469,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3909,"num_write_kb":247456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"234'24","reported_seq":317,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.668246+0000","last_change":"2024-09-11T07:37:36.542263+0000","last_active":"2024-09-11T07:38:50.668246+0000","last_peered":"2024-09-11T07:38:50.668246+0000","last_clean":"2024-09-11T07:38:50.668246+0000","last_became_active":"2024-09-11T07:26:33.540777+0000","last_became_peered":"2024-09-11T07:26:33.540777+0000","last_unstale":"2024-09-11T07:38:50.668246+0000","last_undegraded":"2024-09-11T07:38:50.668246+0000","last_fullsized":"2024-09-11T07:38:50.668246+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'24","last_scrub_stamp":"2024-09-11T07:37:36.542161+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-09-11T07:29:38.112111+0000","last_clean_scrub_stamp":"2024-09-11T07:37:36.542161+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:55.053665+0000","scrub_duration":0.001886759,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":267,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.660031+0000","last_change":"2024-09-11T07:36:37.445237+0000","last_active":"2024-09-11T07:38:50.660031+0000","last_peered":"2024-09-11T07:38:50.660031+0000","last_clean":"2024-09-11T07:38:50.660031+0000","last_became_active":"2024-09-11T07:32:00.518538+0000","last_became_peered":"2024-09-11T07:32:00.518538+0000","last_unstale":"2024-09-11T07:38:50.660031+0000","last_undegraded":"2024-09-11T07:38:50.660031+0000","last_fullsized":"2024-09-11T07:38:50.660031+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:37.444760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:37.444760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012636722,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":279,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.674558+0000","last_change":"2024-09-11T07:38:24.129487+0000","last_active":"2024-09-11T07:38:50.674558+0000","last_peered":"2024-09-11T07:38:50.674558+0000","last_clean":"2024-09-11T07:38:50.674558+0000","last_became_active":"2024-09-11T07:26:35.569081+0000","last_became_peered":"2024-09-11T07:26:35.569081+0000","last_unstale":"2024-09-11T07:38:50.674558+0000","last_undegraded":"2024-09-11T07:38:50.674558+0000","last_fullsized":"2024-09-11T07:38:50.674558+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:24.129047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:38:24.129047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:26.792533+0000","scrub_duration":0.003571196,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":267,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.669205+0000","last_change":"2024-09-11T07:36:50.920711+0000","last_active":"2024-09-11T07:38:50.669205+0000","last_peered":"2024-09-11T07:38:50.669205+0000","last_clean":"2024-09-11T07:38:50.669205+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:38:50.669205+0000","last_undegraded":"2024-09-11T07:38:50.669205+0000","last_fullsized":"2024-09-11T07:38:50.669205+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:50.920625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:46.853387+0000","last_clean_scrub_stamp":"2024-09-11T07:36:50.920625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012404324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"272'7928","reported_seq":7960,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:39:09.900199+0000","last_change":"2024-09-11T07:37:25.855661+0000","last_active":"2024-09-11T07:39:09.900199+0000","last_peered":"2024-09-11T07:39:09.900199+0000","last_clean":"2024-09-11T07:39:09.900199+0000","last_became_active":"2024-09-11T07:37:25.854796+0000","last_became_peered":"2024-09-11T07:37:25.854796+0000","last_unstale":"2024-09-11T07:39:09.900199+0000","last_undegraded":"2024-09-11T07:39:09.900199+0000","last_fullsized":"2024-09-11T07:39:09.900199+0000","mapping_epoch":247,"log_start":"256'3606","ondisk_log_start":"256'3606","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4322,"log_dups_size":0,"ondisk_log_size":4322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":507117568,"num_objects":7738,"num_object_clones":0,"num_object_copies":23214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7928,"num_write_kb":501312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"234'28","reported_seq":313,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.667587+0000","last_change":"2024-09-11T07:37:35.580526+0000","last_active":"2024-09-11T07:38:50.667587+0000","last_peered":"2024-09-11T07:38:50.667587+0000","last_clean":"2024-09-11T07:38:50.667587+0000","last_became_active":"2024-09-11T07:26:33.537850+0000","last_became_peered":"2024-09-11T07:26:33.537850+0000","last_unstale":"2024-09-11T07:38:50.667587+0000","last_undegraded":"2024-09-11T07:38:50.667587+0000","last_fullsized":"2024-09-11T07:38:50.667587+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'28","last_scrub_stamp":"2024-09-11T07:37:35.580463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:35.580463+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019235789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":119,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.666858+0000","last_change":"2024-09-11T07:37:20.527569+0000","last_active":"2024-09-11T07:38:50.666858+0000","last_peered":"2024-09-11T07:38:50.666858+0000","last_clean":"2024-09-11T07:38:50.666858+0000","last_became_active":"2024-09-11T07:36:05.226733+0000","last_became_peered":"2024-09-11T07:36:05.226733+0000","last_unstale":"2024-09-11T07:38:50.666858+0000","last_undegraded":"2024-09-11T07:38:50.666858+0000","last_fullsized":"2024-09-11T07:38:50.666858+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:20.527511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:20.527511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011632039999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"84'9","reported_seq":430,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:39:04.658968+0000","last_change":"2024-09-11T07:37:37.000676+0000","last_active":"2024-09-11T07:39:04.658968+0000","last_peered":"2024-09-11T07:39:04.658968+0000","last_clean":"2024-09-11T07:39:04.658968+0000","last_became_active":"2024-09-11T07:26:35.562997+0000","last_became_peered":"2024-09-11T07:26:35.562997+0000","last_unstale":"2024-09-11T07:39:04.658968+0000","last_undegraded":"2024-09-11T07:39:04.658968+0000","last_fullsized":"2024-09-11T07:39:04.658968+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-11T07:37:37.000619+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-11T07:35:39.832547+0000","last_clean_scrub_stamp":"2024-09-11T07:37:37.000619+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021426219999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":241,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.660284+0000","last_change":"2024-09-11T07:36:32.419622+0000","last_active":"2024-09-11T07:38:50.660284+0000","last_peered":"2024-09-11T07:38:50.660284+0000","last_clean":"2024-09-11T07:38:50.660284+0000","last_became_active":"2024-09-11T07:32:00.511005+0000","last_became_peered":"2024-09-11T07:32:00.511005+0000","last_unstale":"2024-09-11T07:38:50.660284+0000","last_undegraded":"2024-09-11T07:38:50.660284+0000","last_fullsized":"2024-09-11T07:38:50.660284+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:32.419555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:32.419555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046427200000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"272'7830","reported_seq":7862,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:39:09.283761+0000","last_change":"2024-09-11T07:37:25.854170+0000","last_active":"2024-09-11T07:39:09.283761+0000","last_peered":"2024-09-11T07:39:09.283761+0000","last_clean":"2024-09-11T07:39:09.283761+0000","last_became_active":"2024-09-11T07:37:25.853857+0000","last_became_peered":"2024-09-11T07:37:25.853857+0000","last_unstale":"2024-09-11T07:39:09.283761+0000","last_undegraded":"2024-09-11T07:39:09.283761+0000","last_fullsized":"2024-09-11T07:39:09.283761+0000","mapping_epoch":247,"log_start":"256'3544","ondisk_log_start":"256'3544","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4286,"log_dups_size":0,"ondisk_log_size":4286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":505872408,"num_objects":7720,"num_object_clones":0,"num_object_copies":23160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7720,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7830,"num_write_kb":497537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"234'38","reported_seq":346,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.667497+0000","last_change":"2024-09-11T07:38:24.105120+0000","last_active":"2024-09-11T07:38:50.667497+0000","last_peered":"2024-09-11T07:38:50.667497+0000","last_clean":"2024-09-11T07:38:50.667497+0000","last_became_active":"2024-09-11T07:32:00.520670+0000","last_became_peered":"2024-09-11T07:32:00.520670+0000","last_unstale":"2024-09-11T07:38:50.667497+0000","last_undegraded":"2024-09-11T07:38:50.667497+0000","last_fullsized":"2024-09-11T07:38:50.667497+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'38","last_scrub_stamp":"2024-09-11T07:38:24.103612+0000","last_deep_scrub":"87'26","last_deep_scrub_stamp":"2024-09-11T07:35:52.942347+0000","last_clean_scrub_stamp":"2024-09-11T07:38:24.103612+0000","objects_scrubbed":8,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:41.936975+0000","scrub_duration":0.022875121000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":282,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.670274+0000","last_change":"2024-09-11T07:37:33.081869+0000","last_active":"2024-09-11T07:38:50.670274+0000","last_peered":"2024-09-11T07:38:50.670274+0000","last_clean":"2024-09-11T07:38:50.670274+0000","last_became_active":"2024-09-11T07:26:31.421500+0000","last_became_peered":"2024-09-11T07:26:31.421500+0000","last_unstale":"2024-09-11T07:38:50.670274+0000","last_undegraded":"2024-09-11T07:38:50.670274+0000","last_fullsized":"2024-09-11T07:38:50.670274+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:33.081815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:33.081815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:54.240779+0000","scrub_duration":0.001426379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":279,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.667439+0000","last_change":"2024-09-11T07:36:30.786071+0000","last_active":"2024-09-11T07:38:50.667439+0000","last_peered":"2024-09-11T07:38:50.667439+0000","last_clean":"2024-09-11T07:38:50.667439+0000","last_became_active":"2024-09-11T07:32:00.499747+0000","last_became_peered":"2024-09-11T07:32:00.499747+0000","last_unstale":"2024-09-11T07:38:50.667439+0000","last_undegraded":"2024-09-11T07:38:50.667439+0000","last_fullsized":"2024-09-11T07:38:50.667439+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:30.785991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:27.968579+0000","last_clean_scrub_stamp":"2024-09-11T07:36:30.785991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011893737999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":267,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.665766+0000","last_change":"2024-09-11T07:37:28.623070+0000","last_active":"2024-09-11T07:38:50.665766+0000","last_peered":"2024-09-11T07:38:50.665766+0000","last_clean":"2024-09-11T07:38:50.665766+0000","last_became_active":"2024-09-11T07:26:37.564616+0000","last_became_peered":"2024-09-11T07:26:37.564616+0000","last_unstale":"2024-09-11T07:38:50.665766+0000","last_undegraded":"2024-09-11T07:38:50.665766+0000","last_fullsized":"2024-09-11T07:38:50.665766+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:28.622998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:29.978798+0000","last_clean_scrub_stamp":"2024-09-11T07:37:28.622998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001843798,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"272'7889","reported_seq":7920,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:39:09.256426+0000","last_change":"2024-09-11T07:37:25.854301+0000","last_active":"2024-09-11T07:39:09.256426+0000","last_peered":"2024-09-11T07:39:09.256426+0000","last_clean":"2024-09-11T07:39:09.256426+0000","last_became_active":"2024-09-11T07:37:25.853844+0000","last_became_peered":"2024-09-11T07:37:25.853844+0000","last_unstale":"2024-09-11T07:39:09.256426+0000","last_undegraded":"2024-09-11T07:39:09.256426+0000","last_fullsized":"2024-09-11T07:39:09.256426+0000","mapping_epoch":247,"log_start":"256'3657","ondisk_log_start":"256'3657","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4232,"log_dups_size":0,"ondisk_log_size":4232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":510459904,"num_objects":7789,"num_object_clones":0,"num_object_copies":23367,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7889,"num_write_kb":501696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"236'32","reported_seq":342,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.667979+0000","last_change":"2024-09-11T07:36:50.024003+0000","last_active":"2024-09-11T07:38:50.667979+0000","last_peered":"2024-09-11T07:38:50.667979+0000","last_clean":"2024-09-11T07:38:50.667979+0000","last_became_active":"2024-09-11T07:32:00.506270+0000","last_became_peered":"2024-09-11T07:32:00.506270+0000","last_unstale":"2024-09-11T07:38:50.667979+0000","last_undegraded":"2024-09-11T07:38:50.667979+0000","last_fullsized":"2024-09-11T07:38:50.667979+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'32","last_scrub_stamp":"2024-09-11T07:36:50.023909+0000","last_deep_scrub":"236'32","last_deep_scrub_stamp":"2024-09-11T07:36:50.023909+0000","last_clean_scrub_stamp":"2024-09-11T07:36:50.023909+0000","objects_scrubbed":8,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.030217668999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":241,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.660567+0000","last_change":"2024-09-11T07:36:48.434408+0000","last_active":"2024-09-11T07:38:50.660567+0000","last_peered":"2024-09-11T07:38:50.660567+0000","last_clean":"2024-09-11T07:38:50.660567+0000","last_became_active":"2024-09-11T07:32:00.510631+0000","last_became_peered":"2024-09-11T07:32:00.510631+0000","last_unstale":"2024-09-11T07:38:50.660567+0000","last_undegraded":"2024-09-11T07:38:50.660567+0000","last_fullsized":"2024-09-11T07:38:50.660567+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:48.434320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:49.144041+0000","last_clean_scrub_stamp":"2024-09-11T07:36:48.434320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066312669999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":70,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.667493+0000","last_change":"2024-09-11T07:36:55.163926+0000","last_active":"2024-09-11T07:38:50.667493+0000","last_peered":"2024-09-11T07:38:50.667493+0000","last_clean":"2024-09-11T07:38:50.667493+0000","last_became_active":"2024-09-11T07:36:05.225840+0000","last_became_peered":"2024-09-11T07:36:05.225840+0000","last_unstale":"2024-09-11T07:38:50.667493+0000","last_undegraded":"2024-09-11T07:38:50.667493+0000","last_fullsized":"2024-09-11T07:38:50.667493+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:55.163476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:36:55.163476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0082978570000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":266,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.667790+0000","last_change":"2024-09-11T07:36:54.828435+0000","last_active":"2024-09-11T07:38:50.667790+0000","last_peered":"2024-09-11T07:38:50.667790+0000","last_clean":"2024-09-11T07:38:50.667790+0000","last_became_active":"2024-09-11T07:26:37.861279+0000","last_became_peered":"2024-09-11T07:26:37.861279+0000","last_unstale":"2024-09-11T07:38:50.667790+0000","last_undegraded":"2024-09-11T07:38:50.667790+0000","last_fullsized":"2024-09-11T07:38:50.667790+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:54.827454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:39.723598+0000","last_clean_scrub_stamp":"2024-09-11T07:36:54.827454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016642015,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.f","version":"272'7863","reported_seq":7895,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:39:09.140760+0000","last_change":"2024-09-11T07:37:25.856504+0000","last_active":"2024-09-11T07:39:09.140760+0000","last_peered":"2024-09-11T07:39:09.140760+0000","last_clean":"2024-09-11T07:39:09.140760+0000","last_became_active":"2024-09-11T07:37:25.856214+0000","last_became_peered":"2024-09-11T07:37:25.856214+0000","last_unstale":"2024-09-11T07:39:09.140760+0000","last_undegraded":"2024-09-11T07:39:09.140760+0000","last_fullsized":"2024-09-11T07:39:09.140760+0000","mapping_epoch":247,"log_start":"256'3441","ondisk_log_start":"256'3441","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4422,"log_dups_size":0,"ondisk_log_size":4422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":508887040,"num_objects":7765,"num_object_clones":0,"num_object_copies":23295,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7863,"num_write_kb":500096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"234'20","reported_seq":303,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.669488+0000","last_change":"2024-09-11T07:36:37.911537+0000","last_active":"2024-09-11T07:38:50.669488+0000","last_peered":"2024-09-11T07:38:50.669488+0000","last_clean":"2024-09-11T07:38:50.669488+0000","last_became_active":"2024-09-11T07:26:33.543063+0000","last_became_peered":"2024-09-11T07:26:33.543063+0000","last_unstale":"2024-09-11T07:38:50.669488+0000","last_undegraded":"2024-09-11T07:38:50.669488+0000","last_fullsized":"2024-09-11T07:38:50.669488+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-11T07:36:37.911329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:36:37.911329+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01198222,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":299,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.670826+0000","last_change":"2024-09-11T07:36:38.909472+0000","last_active":"2024-09-11T07:38:50.670826+0000","last_peered":"2024-09-11T07:38:50.670826+0000","last_clean":"2024-09-11T07:38:50.670826+0000","last_became_active":"2024-09-11T07:32:00.522456+0000","last_became_peered":"2024-09-11T07:32:00.522456+0000","last_unstale":"2024-09-11T07:38:50.670826+0000","last_undegraded":"2024-09-11T07:38:50.670826+0000","last_fullsized":"2024-09-11T07:38:50.670826+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:36:38.909348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:38.909348+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031018040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":439,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:39:04.656859+0000","last_change":"2024-09-11T07:37:26.141508+0000","last_active":"2024-09-11T07:39:04.656859+0000","last_peered":"2024-09-11T07:39:04.656859+0000","last_clean":"2024-09-11T07:39:04.656859+0000","last_became_active":"2024-09-11T07:26:35.560422+0000","last_became_peered":"2024-09-11T07:26:35.560422+0000","last_unstale":"2024-09-11T07:39:04.656859+0000","last_undegraded":"2024-09-11T07:39:04.656859+0000","last_fullsized":"2024-09-11T07:39:04.656859+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:37:26.141447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.141447+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015408170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":277,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.668591+0000","last_change":"2024-09-11T07:36:40.027067+0000","last_active":"2024-09-11T07:38:50.668591+0000","last_peered":"2024-09-11T07:38:50.668591+0000","last_clean":"2024-09-11T07:38:50.668591+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:38:50.668591+0000","last_undegraded":"2024-09-11T07:38:50.668591+0000","last_fullsized":"2024-09-11T07:38:50.668591+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:40.026982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_clean_scrub_stamp":"2024-09-11T07:36:40.026982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090395279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.9","version":"272'6776","reported_seq":6820,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:39:10.076074+0000","last_change":"2024-09-11T07:38:27.369273+0000","last_active":"2024-09-11T07:39:10.076074+0000","last_peered":"2024-09-11T07:39:10.076074+0000","last_clean":"2024-09-11T07:39:10.076074+0000","last_became_active":"2024-09-11T07:38:27.364955+0000","last_became_peered":"2024-09-11T07:38:27.364955+0000","last_unstale":"2024-09-11T07:39:10.076074+0000","last_undegraded":"2024-09-11T07:39:10.076074+0000","last_fullsized":"2024-09-11T07:39:10.076074+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6776,"log_dups_size":0,"ondisk_log_size":6776,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258342912,"num_objects":3942,"num_object_clones":0,"num_object_copies":11826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4068,"num_write_kb":256320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"234'41","reported_seq":244,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:38:59.653276+0000","last_change":"2024-09-11T07:36:41.343699+0000","last_active":"2024-09-11T07:38:59.653276+0000","last_peered":"2024-09-11T07:38:59.653276+0000","last_clean":"2024-09-11T07:38:59.653276+0000","last_became_active":"2024-09-11T07:33:10.017961+0000","last_became_peered":"2024-09-11T07:33:10.017961+0000","last_unstale":"2024-09-11T07:38:59.653276+0000","last_undegraded":"2024-09-11T07:38:59.653276+0000","last_fullsized":"2024-09-11T07:38:59.653276+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'41","last_scrub_stamp":"2024-09-11T07:36:41.343611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:36:41.343611+0000","objects_scrubbed":10,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043492390000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":283,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.674248+0000","last_change":"2024-09-11T07:36:51.308567+0000","last_active":"2024-09-11T07:38:50.674248+0000","last_peered":"2024-09-11T07:38:50.674248+0000","last_clean":"2024-09-11T07:38:50.674248+0000","last_became_active":"2024-09-11T07:26:31.424618+0000","last_became_peered":"2024-09-11T07:26:31.424618+0000","last_unstale":"2024-09-11T07:38:50.674248+0000","last_undegraded":"2024-09-11T07:38:50.674248+0000","last_fullsized":"2024-09-11T07:38:50.674248+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:51.308492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:51.308492+0000","last_clean_scrub_stamp":"2024-09-11T07:36:51.308492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057563680000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":279,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.668930+0000","last_change":"2024-09-11T07:37:33.144930+0000","last_active":"2024-09-11T07:38:50.668930+0000","last_peered":"2024-09-11T07:38:50.668930+0000","last_clean":"2024-09-11T07:38:50.668930+0000","last_became_active":"2024-09-11T07:26:35.560363+0000","last_became_peered":"2024-09-11T07:26:35.560363+0000","last_unstale":"2024-09-11T07:38:50.668930+0000","last_undegraded":"2024-09-11T07:38:50.668930+0000","last_fullsized":"2024-09-11T07:38:50.668930+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:33.144873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:33.144873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001754675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":284,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.667602+0000","last_change":"2024-09-11T07:37:39.053981+0000","last_active":"2024-09-11T07:38:50.667602+0000","last_peered":"2024-09-11T07:38:50.667602+0000","last_clean":"2024-09-11T07:38:50.667602+0000","last_became_active":"2024-09-11T07:32:00.505919+0000","last_became_peered":"2024-09-11T07:32:00.505919+0000","last_unstale":"2024-09-11T07:38:50.667602+0000","last_undegraded":"2024-09-11T07:38:50.667602+0000","last_fullsized":"2024-09-11T07:38:50.667602+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:39.053900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:39.053900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050142590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"272'8065","reported_seq":8096,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:39:09.180932+0000","last_change":"2024-09-11T07:37:25.855294+0000","last_active":"2024-09-11T07:39:09.180932+0000","last_peered":"2024-09-11T07:39:09.180932+0000","last_clean":"2024-09-11T07:39:09.180932+0000","last_became_active":"2024-09-11T07:37:25.855005+0000","last_became_peered":"2024-09-11T07:37:25.855005+0000","last_unstale":"2024-09-11T07:39:09.180932+0000","last_undegraded":"2024-09-11T07:39:09.180932+0000","last_fullsized":"2024-09-11T07:39:09.180932+0000","mapping_epoch":247,"log_start":"256'3618","ondisk_log_start":"256'3618","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4447,"log_dups_size":0,"ondisk_log_size":4447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":520552448,"num_objects":7943,"num_object_clones":0,"num_object_copies":23829,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8065,"num_write_kb":512256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"234'22","reported_seq":308,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.668787+0000","last_change":"2024-09-11T07:37:20.965215+0000","last_active":"2024-09-11T07:38:50.668787+0000","last_peered":"2024-09-11T07:38:50.668787+0000","last_clean":"2024-09-11T07:38:50.668787+0000","last_became_active":"2024-09-11T07:26:33.546126+0000","last_became_peered":"2024-09-11T07:26:33.546126+0000","last_unstale":"2024-09-11T07:38:50.668787+0000","last_undegraded":"2024-09-11T07:38:50.668787+0000","last_fullsized":"2024-09-11T07:38:50.668787+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'22","last_scrub_stamp":"2024-09-11T07:37:20.965147+0000","last_deep_scrub":"144'18","last_deep_scrub_stamp":"2024-09-11T07:31:51.603378+0000","last_clean_scrub_stamp":"2024-09-11T07:37:20.965147+0000","objects_scrubbed":5,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019365999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":1039,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:39:07.557247+0000","last_change":"2024-09-11T07:37:22.990093+0000","last_active":"2024-09-11T07:39:07.557247+0000","last_peered":"2024-09-11T07:39:07.557247+0000","last_clean":"2024-09-11T07:39:07.557247+0000","last_became_active":"2024-09-11T07:26:31.421276+0000","last_became_peered":"2024-09-11T07:26:31.421276+0000","last_unstale":"2024-09-11T07:39:07.557247+0000","last_undegraded":"2024-09-11T07:39:07.557247+0000","last_fullsized":"2024-09-11T07:39:07.557247+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'5","last_scrub_stamp":"2024-09-11T07:37:22.990033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:22.990033+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001646954,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":749,"num_read_kb":744,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":296,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.661696+0000","last_change":"2024-09-11T07:38:38.915997+0000","last_active":"2024-09-11T07:38:50.661696+0000","last_peered":"2024-09-11T07:38:50.661696+0000","last_clean":"2024-09-11T07:38:50.661696+0000","last_became_active":"2024-09-11T07:26:31.413354+0000","last_became_peered":"2024-09-11T07:26:31.413354+0000","last_unstale":"2024-09-11T07:38:50.661696+0000","last_undegraded":"2024-09-11T07:38:50.661696+0000","last_fullsized":"2024-09-11T07:38:50.661696+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:45.059537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:07.149367+0000","last_clean_scrub_stamp":"2024-09-11T07:36:45.059537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013492418000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":296,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.662609+0000","last_change":"2024-09-11T07:38:24.877020+0000","last_active":"2024-09-11T07:38:50.662609+0000","last_peered":"2024-09-11T07:38:50.662609+0000","last_clean":"2024-09-11T07:38:50.662609+0000","last_became_active":"2024-09-11T07:32:00.508398+0000","last_became_peered":"2024-09-11T07:32:00.508398+0000","last_unstale":"2024-09-11T07:38:50.662609+0000","last_undegraded":"2024-09-11T07:38:50.662609+0000","last_fullsized":"2024-09-11T07:38:50.662609+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:24.876930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:38:24.876930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:30.500296+0000","scrub_duration":0.0095042219999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":267,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.660337+0000","last_change":"2024-09-11T07:36:56.490992+0000","last_active":"2024-09-11T07:38:50.660337+0000","last_peered":"2024-09-11T07:38:50.660337+0000","last_clean":"2024-09-11T07:38:50.660337+0000","last_became_active":"2024-09-11T07:32:00.515745+0000","last_became_peered":"2024-09-11T07:32:00.515745+0000","last_unstale":"2024-09-11T07:38:50.660337+0000","last_undegraded":"2024-09-11T07:38:50.660337+0000","last_fullsized":"2024-09-11T07:38:50.660337+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:56.490940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:56.490940+0000","last_clean_scrub_stamp":"2024-09-11T07:36:56.490940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.037860923999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"272'8126","reported_seq":8158,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:39:10.072650+0000","last_change":"2024-09-11T07:37:26.029935+0000","last_active":"2024-09-11T07:39:10.072650+0000","last_peered":"2024-09-11T07:39:10.072650+0000","last_clean":"2024-09-11T07:39:10.072650+0000","last_became_active":"2024-09-11T07:37:26.029608+0000","last_became_peered":"2024-09-11T07:37:26.029608+0000","last_unstale":"2024-09-11T07:39:10.072650+0000","last_undegraded":"2024-09-11T07:39:10.072650+0000","last_fullsized":"2024-09-11T07:39:10.072650+0000","mapping_epoch":247,"log_start":"255'2697","ondisk_log_start":"255'2697","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":5429,"log_dups_size":0,"ondisk_log_size":5429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":517865472,"num_objects":7902,"num_object_clones":0,"num_object_copies":23706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8126,"num_write_kb":512896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"234'30","reported_seq":347,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.669341+0000","last_change":"2024-09-11T07:36:45.088277+0000","last_active":"2024-09-11T07:38:50.669341+0000","last_peered":"2024-09-11T07:38:50.669341+0000","last_clean":"2024-09-11T07:38:50.669341+0000","last_became_active":"2024-09-11T07:28:09.018134+0000","last_became_peered":"2024-09-11T07:28:09.018134+0000","last_unstale":"2024-09-11T07:38:50.669341+0000","last_undegraded":"2024-09-11T07:38:50.669341+0000","last_fullsized":"2024-09-11T07:38:50.669341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'30","last_scrub_stamp":"2024-09-11T07:36:45.087995+0000","last_deep_scrub":"94'18","last_deep_scrub_stamp":"2024-09-11T07:28:51.437397+0000","last_clean_scrub_stamp":"2024-09-11T07:36:45.087995+0000","objects_scrubbed":6,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.027742936999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":289,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.674650+0000","last_change":"2024-09-11T07:38:21.385905+0000","last_active":"2024-09-11T07:38:50.674650+0000","last_peered":"2024-09-11T07:38:50.674650+0000","last_clean":"2024-09-11T07:38:50.674650+0000","last_became_active":"2024-09-11T07:26:31.420320+0000","last_became_peered":"2024-09-11T07:26:31.420320+0000","last_unstale":"2024-09-11T07:38:50.674650+0000","last_undegraded":"2024-09-11T07:38:50.674650+0000","last_fullsized":"2024-09-11T07:38:50.674650+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-11T07:38:21.385813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:38:21.385813+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:33.800360+0000","scrub_duration":0.0060130330000000001,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":278,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.659805+0000","last_change":"2024-09-11T07:37:26.557037+0000","last_active":"2024-09-11T07:38:50.659805+0000","last_peered":"2024-09-11T07:38:50.659805+0000","last_clean":"2024-09-11T07:38:50.659805+0000","last_became_active":"2024-09-11T07:32:00.520035+0000","last_became_peered":"2024-09-11T07:32:00.520035+0000","last_unstale":"2024-09-11T07:38:50.659805+0000","last_undegraded":"2024-09-11T07:38:50.659805+0000","last_fullsized":"2024-09-11T07:38:50.659805+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:26.556972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:16.967492+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.556972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001503206,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":269,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.674622+0000","last_change":"2024-09-11T07:38:23.100816+0000","last_active":"2024-09-11T07:38:50.674622+0000","last_peered":"2024-09-11T07:38:50.674622+0000","last_clean":"2024-09-11T07:38:50.674622+0000","last_became_active":"2024-09-11T07:26:35.564178+0000","last_became_peered":"2024-09-11T07:26:35.564178+0000","last_unstale":"2024-09-11T07:38:50.674622+0000","last_undegraded":"2024-09-11T07:38:50.674622+0000","last_fullsized":"2024-09-11T07:38:50.674622+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:23.100768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:51.187127+0000","last_clean_scrub_stamp":"2024-09-11T07:38:23.100768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:49.705352+0000","scrub_duration":0.012057399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":279,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.670642+0000","last_change":"2024-09-11T07:38:25.908008+0000","last_active":"2024-09-11T07:38:50.670642+0000","last_peered":"2024-09-11T07:38:50.670642+0000","last_clean":"2024-09-11T07:38:50.670642+0000","last_became_active":"2024-09-11T07:26:37.561378+0000","last_became_peered":"2024-09-11T07:26:37.561378+0000","last_unstale":"2024-09-11T07:38:50.670642+0000","last_undegraded":"2024-09-11T07:38:50.670642+0000","last_fullsized":"2024-09-11T07:38:50.670642+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:25.907711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:38:25.907711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:29.941232+0000","scrub_duration":0.0069833569999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"272'6683","reported_seq":6725,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:39:09.909858+0000","last_change":"2024-09-11T07:38:27.358414+0000","last_active":"2024-09-11T07:39:09.909858+0000","last_peered":"2024-09-11T07:39:09.909858+0000","last_clean":"2024-09-11T07:39:09.909858+0000","last_became_active":"2024-09-11T07:38:27.355364+0000","last_became_peered":"2024-09-11T07:38:27.355364+0000","last_unstale":"2024-09-11T07:39:09.909858+0000","last_undegraded":"2024-09-11T07:39:09.909858+0000","last_fullsized":"2024-09-11T07:39:09.909858+0000","mapping_epoch":247,"log_start":"254'500","ondisk_log_start":"254'500","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6183,"log_dups_size":0,"ondisk_log_size":6183,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":254869504,"num_objects":3889,"num_object_clones":0,"num_object_copies":11667,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4009,"num_write_kb":252736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"234'26","reported_seq":122,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.666540+0000","last_change":"2024-09-11T07:37:40.649065+0000","last_active":"2024-09-11T07:38:50.666540+0000","last_peered":"2024-09-11T07:38:50.666540+0000","last_clean":"2024-09-11T07:38:50.666540+0000","last_became_active":"2024-09-11T07:36:05.224704+0000","last_became_peered":"2024-09-11T07:36:05.224704+0000","last_unstale":"2024-09-11T07:38:50.666540+0000","last_undegraded":"2024-09-11T07:38:50.666540+0000","last_fullsized":"2024-09-11T07:38:50.666540+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'26","last_scrub_stamp":"2024-09-11T07:37:40.648986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:40.648986+0000","objects_scrubbed":7,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:09.297479+0000","scrub_duration":0.0030578620000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'3","reported_seq":281,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.666214+0000","last_change":"2024-09-11T07:36:54.937298+0000","last_active":"2024-09-11T07:38:50.666214+0000","last_peered":"2024-09-11T07:38:50.666214+0000","last_clean":"2024-09-11T07:38:50.666214+0000","last_became_active":"2024-09-11T07:26:31.419501+0000","last_became_peered":"2024-09-11T07:26:31.419501+0000","last_unstale":"2024-09-11T07:38:50.666214+0000","last_undegraded":"2024-09-11T07:38:50.666214+0000","last_fullsized":"2024-09-11T07:38:50.666214+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-09-11T07:36:54.937201+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_clean_scrub_stamp":"2024-09-11T07:36:54.937201+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007045239,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":293,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.669557+0000","last_change":"2024-09-11T07:37:35.504299+0000","last_active":"2024-09-11T07:38:50.669557+0000","last_peered":"2024-09-11T07:38:50.669557+0000","last_clean":"2024-09-11T07:38:50.669557+0000","last_became_active":"2024-09-11T07:32:00.498002+0000","last_became_peered":"2024-09-11T07:32:00.498002+0000","last_unstale":"2024-09-11T07:38:50.669557+0000","last_undegraded":"2024-09-11T07:38:50.669557+0000","last_fullsized":"2024-09-11T07:38:50.669557+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:35.504238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:35.504238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013066169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"87'81","reported_seq":294,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.659741+0000","last_change":"2024-09-11T07:36:45.398752+0000","last_active":"2024-09-11T07:38:50.659741+0000","last_peered":"2024-09-11T07:38:50.659741+0000","last_clean":"2024-09-11T07:38:50.659741+0000","last_became_active":"2024-09-11T07:32:00.519656+0000","last_became_peered":"2024-09-11T07:32:00.519656+0000","last_unstale":"2024-09-11T07:38:50.659741+0000","last_undegraded":"2024-09-11T07:38:50.659741+0000","last_fullsized":"2024-09-11T07:38:50.659741+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:36:45.398664+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:36:45.398664+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041388420000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":264,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.659501+0000","last_change":"2024-09-11T07:37:41.562374+0000","last_active":"2024-09-11T07:38:50.659501+0000","last_peered":"2024-09-11T07:38:50.659501+0000","last_clean":"2024-09-11T07:38:50.659501+0000","last_became_active":"2024-09-11T07:32:00.510888+0000","last_became_peered":"2024-09-11T07:32:00.510888+0000","last_unstale":"2024-09-11T07:38:50.659501+0000","last_undegraded":"2024-09-11T07:38:50.659501+0000","last_fullsized":"2024-09-11T07:38:50.659501+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:41.562322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:41.562322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:07.245074+0000","scrub_duration":0.003022734,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":267,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.675713+0000","last_change":"2024-09-11T07:37:23.436064+0000","last_active":"2024-09-11T07:38:50.675713+0000","last_peered":"2024-09-11T07:38:50.675713+0000","last_clean":"2024-09-11T07:38:50.675713+0000","last_became_active":"2024-09-11T07:26:37.559920+0000","last_became_peered":"2024-09-11T07:26:37.559920+0000","last_unstale":"2024-09-11T07:38:50.675713+0000","last_undegraded":"2024-09-11T07:38:50.675713+0000","last_fullsized":"2024-09-11T07:38:50.675713+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:23.436012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:23.436012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001213481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.7","version":"272'6648","reported_seq":6690,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:39:09.101604+0000","last_change":"2024-09-11T07:38:27.358935+0000","last_active":"2024-09-11T07:39:09.101604+0000","last_peered":"2024-09-11T07:39:09.101604+0000","last_clean":"2024-09-11T07:39:09.101604+0000","last_became_active":"2024-09-11T07:38:27.356333+0000","last_became_peered":"2024-09-11T07:38:27.356333+0000","last_unstale":"2024-09-11T07:39:09.101604+0000","last_undegraded":"2024-09-11T07:39:09.101604+0000","last_fullsized":"2024-09-11T07:39:09.101604+0000","mapping_epoch":247,"log_start":"254'400","ondisk_log_start":"254'400","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6248,"log_dups_size":0,"ondisk_log_size":6248,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255721472,"num_objects":3902,"num_object_clones":0,"num_object_copies":11706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3944,"num_write_kb":251072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"234'35","reported_seq":137,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.669336+0000","last_change":"2024-09-11T07:38:23.944823+0000","last_active":"2024-09-11T07:38:50.669336+0000","last_peered":"2024-09-11T07:38:50.669336+0000","last_clean":"2024-09-11T07:38:50.669336+0000","last_became_active":"2024-09-11T07:36:05.226021+0000","last_became_peered":"2024-09-11T07:36:05.226021+0000","last_unstale":"2024-09-11T07:38:50.669336+0000","last_undegraded":"2024-09-11T07:38:50.669336+0000","last_fullsized":"2024-09-11T07:38:50.669336+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'35","last_scrub_stamp":"2024-09-11T07:38:23.944683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:38:23.944683+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:46.043876+0000","scrub_duration":0.003078154,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":289,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.663735+0000","last_change":"2024-09-11T07:38:50.102639+0000","last_active":"2024-09-11T07:38:50.663735+0000","last_peered":"2024-09-11T07:38:50.663735+0000","last_clean":"2024-09-11T07:38:50.663735+0000","last_became_active":"2024-09-11T07:26:31.419560+0000","last_became_peered":"2024-09-11T07:26:31.419560+0000","last_unstale":"2024-09-11T07:38:50.663735+0000","last_undegraded":"2024-09-11T07:38:50.663735+0000","last_fullsized":"2024-09-11T07:38:50.663735+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:48.088795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:48.088795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0081450789999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":279,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.662271+0000","last_change":"2024-09-11T07:37:17.420289+0000","last_active":"2024-09-11T07:38:50.662271+0000","last_peered":"2024-09-11T07:38:50.662271+0000","last_clean":"2024-09-11T07:38:50.662271+0000","last_became_active":"2024-09-11T07:26:35.564044+0000","last_became_peered":"2024-09-11T07:26:35.564044+0000","last_unstale":"2024-09-11T07:38:50.662271+0000","last_undegraded":"2024-09-11T07:38:50.662271+0000","last_fullsized":"2024-09-11T07:38:50.662271+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:17.420236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:17.420236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012713080000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":262,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.659677+0000","last_change":"2024-09-11T07:36:31.411708+0000","last_active":"2024-09-11T07:38:50.659677+0000","last_peered":"2024-09-11T07:38:50.659677+0000","last_clean":"2024-09-11T07:38:50.659677+0000","last_became_active":"2024-09-11T07:32:00.516916+0000","last_became_peered":"2024-09-11T07:32:00.516916+0000","last_unstale":"2024-09-11T07:38:50.659677+0000","last_undegraded":"2024-09-11T07:38:50.659677+0000","last_fullsized":"2024-09-11T07:38:50.659677+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:31.411554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:31.411554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002948132,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"272'6848","reported_seq":6890,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:39:09.143731+0000","last_change":"2024-09-11T07:38:27.354103+0000","last_active":"2024-09-11T07:39:09.143731+0000","last_peered":"2024-09-11T07:39:09.143731+0000","last_clean":"2024-09-11T07:39:09.143731+0000","last_became_active":"2024-09-11T07:38:27.351200+0000","last_became_peered":"2024-09-11T07:38:27.351200+0000","last_unstale":"2024-09-11T07:39:09.143731+0000","last_undegraded":"2024-09-11T07:39:09.143731+0000","last_fullsized":"2024-09-11T07:39:09.143731+0000","mapping_epoch":247,"log_start":"254'600","ondisk_log_start":"254'600","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6248,"log_dups_size":0,"ondisk_log_size":6248,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264372224,"num_objects":4034,"num_object_clones":0,"num_object_copies":12102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4084,"num_write_kb":259776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"234'15","reported_seq":296,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.667900+0000","last_change":"2024-09-11T07:36:46.033480+0000","last_active":"2024-09-11T07:38:50.667900+0000","last_peered":"2024-09-11T07:38:50.667900+0000","last_clean":"2024-09-11T07:38:50.667900+0000","last_became_active":"2024-09-11T07:26:33.543148+0000","last_became_peered":"2024-09-11T07:26:33.543148+0000","last_unstale":"2024-09-11T07:38:50.667900+0000","last_undegraded":"2024-09-11T07:38:50.667900+0000","last_fullsized":"2024-09-11T07:38:50.667900+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:36:46.033388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:36:46.033388+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0084720030000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":282,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.670706+0000","last_change":"2024-09-11T07:37:26.019680+0000","last_active":"2024-09-11T07:38:50.670706+0000","last_peered":"2024-09-11T07:38:50.670706+0000","last_clean":"2024-09-11T07:38:50.670706+0000","last_became_active":"2024-09-11T07:26:31.423056+0000","last_became_peered":"2024-09-11T07:26:31.423056+0000","last_unstale":"2024-09-11T07:38:50.670706+0000","last_undegraded":"2024-09-11T07:38:50.670706+0000","last_fullsized":"2024-09-11T07:38:50.670706+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:26.019623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.019623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013393599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":285,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.662261+0000","last_change":"2024-09-11T07:38:46.052238+0000","last_active":"2024-09-11T07:38:50.662261+0000","last_peered":"2024-09-11T07:38:50.662261+0000","last_clean":"2024-09-11T07:38:50.662261+0000","last_became_active":"2024-09-11T07:26:35.563182+0000","last_became_peered":"2024-09-11T07:26:35.563182+0000","last_unstale":"2024-09-11T07:38:50.662261+0000","last_undegraded":"2024-09-11T07:38:50.662261+0000","last_fullsized":"2024-09-11T07:38:50.662261+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:39.141620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:36:39.141620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01162813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":284,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.674575+0000","last_change":"2024-09-11T07:36:36.023079+0000","last_active":"2024-09-11T07:38:50.674575+0000","last_peered":"2024-09-11T07:38:50.674575+0000","last_clean":"2024-09-11T07:38:50.674575+0000","last_became_active":"2024-09-11T07:32:00.505185+0000","last_became_peered":"2024-09-11T07:32:00.505185+0000","last_unstale":"2024-09-11T07:38:50.674575+0000","last_undegraded":"2024-09-11T07:38:50.674575+0000","last_fullsized":"2024-09-11T07:38:50.674575+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:36.023006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:46.944763+0000","last_clean_scrub_stamp":"2024-09-11T07:36:36.023006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011934019000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.5","version":"272'6787","reported_seq":6829,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:39:10.109563+0000","last_change":"2024-09-11T07:38:27.362381+0000","last_active":"2024-09-11T07:39:10.109563+0000","last_peered":"2024-09-11T07:39:10.109563+0000","last_clean":"2024-09-11T07:39:10.109563+0000","last_became_active":"2024-09-11T07:38:27.359105+0000","last_became_peered":"2024-09-11T07:38:27.359105+0000","last_unstale":"2024-09-11T07:39:10.109563+0000","last_undegraded":"2024-09-11T07:39:10.109563+0000","last_fullsized":"2024-09-11T07:39:10.109563+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6787,"log_dups_size":0,"ondisk_log_size":6787,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":260505600,"num_objects":3975,"num_object_clones":0,"num_object_copies":11925,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4073,"num_write_kb":257536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"234'27","reported_seq":324,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.668599+0000","last_change":"2024-09-11T07:36:32.341522+0000","last_active":"2024-09-11T07:38:50.668599+0000","last_peered":"2024-09-11T07:38:50.668599+0000","last_clean":"2024-09-11T07:38:50.668599+0000","last_became_active":"2024-09-11T07:32:00.521464+0000","last_became_peered":"2024-09-11T07:32:00.521464+0000","last_unstale":"2024-09-11T07:38:50.668599+0000","last_undegraded":"2024-09-11T07:38:50.668599+0000","last_fullsized":"2024-09-11T07:38:50.668599+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-09-11T07:36:32.341462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:36:32.341462+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043984430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":310,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.662854+0000","last_change":"2024-09-11T07:38:27.889904+0000","last_active":"2024-09-11T07:38:50.662854+0000","last_peered":"2024-09-11T07:38:50.662854+0000","last_clean":"2024-09-11T07:38:50.662854+0000","last_became_active":"2024-09-11T07:32:00.513949+0000","last_became_peered":"2024-09-11T07:32:00.513949+0000","last_unstale":"2024-09-11T07:38:50.662854+0000","last_undegraded":"2024-09-11T07:38:50.662854+0000","last_fullsized":"2024-09-11T07:38:50.662854+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:27.889798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_clean_scrub_stamp":"2024-09-11T07:38:27.889798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:32.678437+0000","scrub_duration":0.0040959359999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"151'10","reported_seq":389,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:38:59.653194+0000","last_change":"2024-09-11T07:37:38.544127+0000","last_active":"2024-09-11T07:38:59.653194+0000","last_peered":"2024-09-11T07:38:59.653194+0000","last_clean":"2024-09-11T07:38:59.653194+0000","last_became_active":"2024-09-11T07:32:00.506806+0000","last_became_peered":"2024-09-11T07:32:00.506806+0000","last_unstale":"2024-09-11T07:38:59.653194+0000","last_undegraded":"2024-09-11T07:38:59.653194+0000","last_fullsized":"2024-09-11T07:38:59.653194+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:37:38.544072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:38.544072+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027401579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":267,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.673791+0000","last_change":"2024-09-11T07:36:44.295056+0000","last_active":"2024-09-11T07:38:50.673791+0000","last_peered":"2024-09-11T07:38:50.673791+0000","last_clean":"2024-09-11T07:38:50.673791+0000","last_became_active":"2024-09-11T07:26:37.861134+0000","last_became_peered":"2024-09-11T07:26:37.861134+0000","last_unstale":"2024-09-11T07:38:50.673791+0000","last_undegraded":"2024-09-11T07:38:50.673791+0000","last_fullsized":"2024-09-11T07:38:50.673791+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:44.293653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:44.293653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0074032500000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.4","version":"272'6707","reported_seq":6755,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:39:09.512464+0000","last_change":"2024-09-11T07:38:27.395872+0000","last_active":"2024-09-11T07:39:09.512464+0000","last_peered":"2024-09-11T07:39:09.512464+0000","last_clean":"2024-09-11T07:39:09.512464+0000","last_became_active":"2024-09-11T07:38:27.372060+0000","last_became_peered":"2024-09-11T07:38:27.372060+0000","last_unstale":"2024-09-11T07:39:09.512464+0000","last_undegraded":"2024-09-11T07:39:09.512464+0000","last_fullsized":"2024-09-11T07:39:09.512464+0000","mapping_epoch":247,"log_start":"254'500","ondisk_log_start":"254'500","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6207,"log_dups_size":0,"ondisk_log_size":6207,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258473984,"num_objects":3944,"num_object_clones":0,"num_object_copies":11832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4020,"num_write_kb":254848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"234'23","reported_seq":308,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.667512+0000","last_change":"2024-09-11T07:37:34.613363+0000","last_active":"2024-09-11T07:38:50.667512+0000","last_peered":"2024-09-11T07:38:50.667512+0000","last_clean":"2024-09-11T07:38:50.667512+0000","last_became_active":"2024-09-11T07:26:33.540363+0000","last_became_peered":"2024-09-11T07:26:33.540363+0000","last_unstale":"2024-09-11T07:38:50.667512+0000","last_undegraded":"2024-09-11T07:38:50.667512+0000","last_fullsized":"2024-09-11T07:38:50.667512+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'23","last_scrub_stamp":"2024-09-11T07:37:34.613280+0000","last_deep_scrub":"234'23","last_deep_scrub_stamp":"2024-09-11T07:37:34.613280+0000","last_clean_scrub_stamp":"2024-09-11T07:37:34.613280+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002722604,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":303,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.667345+0000","last_change":"2024-09-11T07:38:46.492654+0000","last_active":"2024-09-11T07:38:50.667345+0000","last_peered":"2024-09-11T07:38:50.667345+0000","last_clean":"2024-09-11T07:38:50.667345+0000","last_became_active":"2024-09-11T07:32:00.504344+0000","last_became_peered":"2024-09-11T07:32:00.504344+0000","last_unstale":"2024-09-11T07:38:50.667345+0000","last_undegraded":"2024-09-11T07:38:50.667345+0000","last_fullsized":"2024-09-11T07:38:50.667345+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:35:46.769070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:35:46.769070+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049213720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":285,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.663357+0000","last_change":"2024-09-11T07:38:48.023909+0000","last_active":"2024-09-11T07:38:50.663357+0000","last_peered":"2024-09-11T07:38:50.663357+0000","last_clean":"2024-09-11T07:38:50.663357+0000","last_became_active":"2024-09-11T07:26:35.561967+0000","last_became_peered":"2024-09-11T07:26:35.561967+0000","last_unstale":"2024-09-11T07:38:50.663357+0000","last_undegraded":"2024-09-11T07:38:50.663357+0000","last_fullsized":"2024-09-11T07:38:50.663357+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:43.125187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:44.820354+0000","last_clean_scrub_stamp":"2024-09-11T07:36:43.125187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061424100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":284,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.666617+0000","last_change":"2024-09-11T07:37:16.860986+0000","last_active":"2024-09-11T07:38:50.666617+0000","last_peered":"2024-09-11T07:38:50.666617+0000","last_clean":"2024-09-11T07:38:50.666617+0000","last_became_active":"2024-09-11T07:32:00.502019+0000","last_became_peered":"2024-09-11T07:32:00.502019+0000","last_unstale":"2024-09-11T07:38:50.666617+0000","last_undegraded":"2024-09-11T07:38:50.666617+0000","last_fullsized":"2024-09-11T07:38:50.666617+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:16.860930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:16.860930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024991000000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"234'15","reported_seq":313,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.669266+0000","last_change":"2024-09-11T07:37:19.096890+0000","last_active":"2024-09-11T07:38:50.669266+0000","last_peered":"2024-09-11T07:38:50.669266+0000","last_clean":"2024-09-11T07:38:50.669266+0000","last_became_active":"2024-09-11T07:32:00.509412+0000","last_became_peered":"2024-09-11T07:32:00.509412+0000","last_unstale":"2024-09-11T07:38:50.669266+0000","last_undegraded":"2024-09-11T07:38:50.669266+0000","last_fullsized":"2024-09-11T07:38:50.669266+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:37:19.096817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:19.096817+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00175015,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":289,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.663447+0000","last_change":"2024-09-11T07:38:49.055084+0000","last_active":"2024-09-11T07:38:50.663447+0000","last_peered":"2024-09-11T07:38:50.663447+0000","last_clean":"2024-09-11T07:38:50.663447+0000","last_became_active":"2024-09-11T07:26:31.415701+0000","last_became_peered":"2024-09-11T07:26:31.415701+0000","last_unstale":"2024-09-11T07:38:50.663447+0000","last_undegraded":"2024-09-11T07:38:50.663447+0000","last_fullsized":"2024-09-11T07:38:50.663447+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:53.065247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.925200+0000","last_clean_scrub_stamp":"2024-09-11T07:36:53.065247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023028290000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":276,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.668520+0000","last_change":"2024-09-11T07:36:32.708578+0000","last_active":"2024-09-11T07:38:50.668520+0000","last_peered":"2024-09-11T07:38:50.668520+0000","last_clean":"2024-09-11T07:38:50.668520+0000","last_became_active":"2024-09-11T07:32:00.499598+0000","last_became_peered":"2024-09-11T07:32:00.499598+0000","last_unstale":"2024-09-11T07:38:50.668520+0000","last_undegraded":"2024-09-11T07:38:50.668520+0000","last_fullsized":"2024-09-11T07:38:50.668520+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:32.708500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:35.392360+0000","last_clean_scrub_stamp":"2024-09-11T07:36:32.708500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032820800000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":295,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.662523+0000","last_change":"2024-09-11T07:38:29.941071+0000","last_active":"2024-09-11T07:38:50.662523+0000","last_peered":"2024-09-11T07:38:50.662523+0000","last_clean":"2024-09-11T07:38:50.662523+0000","last_became_active":"2024-09-11T07:32:00.506697+0000","last_became_peered":"2024-09-11T07:32:00.506697+0000","last_unstale":"2024-09-11T07:38:50.662523+0000","last_undegraded":"2024-09-11T07:38:50.662523+0000","last_fullsized":"2024-09-11T07:38:50.662523+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:38:29.941008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:38:29.941008+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:55.438942+0000","scrub_duration":0.0016886850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":70,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.667655+0000","last_change":"2024-09-11T07:36:51.617809+0000","last_active":"2024-09-11T07:38:50.667655+0000","last_peered":"2024-09-11T07:38:50.667655+0000","last_clean":"2024-09-11T07:38:50.667655+0000","last_became_active":"2024-09-11T07:36:05.225791+0000","last_became_peered":"2024-09-11T07:36:05.225791+0000","last_unstale":"2024-09-11T07:38:50.667655+0000","last_undegraded":"2024-09-11T07:38:50.667655+0000","last_fullsized":"2024-09-11T07:38:50.667655+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:51.617730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:51.617730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001776197,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"234'15","reported_seq":87,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.674310+0000","last_change":"2024-09-11T07:37:26.450184+0000","last_active":"2024-09-11T07:38:50.674310+0000","last_peered":"2024-09-11T07:38:50.674310+0000","last_clean":"2024-09-11T07:38:50.674310+0000","last_became_active":"2024-09-11T07:35:50.100154+0000","last_became_peered":"2024-09-11T07:35:50.100154+0000","last_unstale":"2024-09-11T07:38:50.674310+0000","last_undegraded":"2024-09-11T07:38:50.674310+0000","last_fullsized":"2024-09-11T07:38:50.674310+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:37:26.450122+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.450122+0000","objects_scrubbed":3,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001711052,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":279,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.669288+0000","last_change":"2024-09-11T07:37:18.340651+0000","last_active":"2024-09-11T07:38:50.669288+0000","last_peered":"2024-09-11T07:38:50.669288+0000","last_clean":"2024-09-11T07:38:50.669288+0000","last_became_active":"2024-09-11T07:26:35.566894+0000","last_became_peered":"2024-09-11T07:26:35.566894+0000","last_unstale":"2024-09-11T07:38:50.669288+0000","last_undegraded":"2024-09-11T07:38:50.669288+0000","last_fullsized":"2024-09-11T07:38:50.669288+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:18.340578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:37:18.340578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054750240000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":265,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.660683+0000","last_change":"2024-09-11T07:36:57.461696+0000","last_active":"2024-09-11T07:38:50.660683+0000","last_peered":"2024-09-11T07:38:50.660683+0000","last_clean":"2024-09-11T07:38:50.660683+0000","last_became_active":"2024-09-11T07:32:00.514100+0000","last_became_peered":"2024-09-11T07:32:00.514100+0000","last_unstale":"2024-09-11T07:38:50.660683+0000","last_undegraded":"2024-09-11T07:38:50.660683+0000","last_fullsized":"2024-09-11T07:38:50.660683+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:57.461617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:57.461617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.028762672999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.19","version":"272'6816","reported_seq":11746,"reported_epoch":272,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T07:39:10.097873+0000","last_change":"2024-09-11T07:38:30.582397+0000","last_active":"2024-09-11T07:39:10.097873+0000","last_peered":"2024-09-11T07:39:10.097873+0000","last_clean":"2024-09-11T07:38:28.328791+0000","last_became_active":"2024-09-11T07:38:30.512286+0000","last_became_peered":"2024-09-11T07:38:30.512286+0000","last_unstale":"2024-09-11T07:39:10.097873+0000","last_undegraded":"2024-09-11T07:39:10.097873+0000","last_fullsized":"2024-09-11T07:38:30.343646+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6816,"log_dups_size":0,"ondisk_log_size":6816,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":259522560,"num_objects":3960,"num_object_clones":0,"num_object_copies":11880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2282,"num_objects_unfound":0,"num_objects_dirty":3960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4108,"num_write_kb":258176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":106561536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[0,5],"avail_no_missing":["0","5","7"],"object_location_counts":[{"shards":"0,5","objects":2819},{"shards":"0,5,7","objects":1141}],"blocked_by":[],"up_primary":3,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"234'25","reported_seq":335,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.669690+0000","last_change":"2024-09-11T07:36:55.966810+0000","last_active":"2024-09-11T07:38:50.669690+0000","last_peered":"2024-09-11T07:38:50.669690+0000","last_clean":"2024-09-11T07:38:50.669690+0000","last_became_active":"2024-09-11T07:32:00.504463+0000","last_became_peered":"2024-09-11T07:32:00.504463+0000","last_unstale":"2024-09-11T07:38:50.669690+0000","last_undegraded":"2024-09-11T07:38:50.669690+0000","last_fullsized":"2024-09-11T07:38:50.669690+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'25","last_scrub_stamp":"2024-09-11T07:36:55.966595+0000","last_deep_scrub":"86'15","last_deep_scrub_stamp":"2024-09-11T07:28:22.858137+0000","last_clean_scrub_stamp":"2024-09-11T07:36:55.966595+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.032608074000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":283,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.674222+0000","last_change":"2024-09-11T07:36:34.405309+0000","last_active":"2024-09-11T07:38:50.674222+0000","last_peered":"2024-09-11T07:38:50.674222+0000","last_clean":"2024-09-11T07:38:50.674222+0000","last_became_active":"2024-09-11T07:32:00.498473+0000","last_became_peered":"2024-09-11T07:32:00.498473+0000","last_unstale":"2024-09-11T07:38:50.674222+0000","last_undegraded":"2024-09-11T07:38:50.674222+0000","last_fullsized":"2024-09-11T07:38:50.674222+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:34.405241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:34.405241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039370459999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":428,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:38:59.653262+0000","last_change":"2024-09-11T07:37:17.360240+0000","last_active":"2024-09-11T07:38:59.653262+0000","last_peered":"2024-09-11T07:38:59.653262+0000","last_clean":"2024-09-11T07:38:59.653262+0000","last_became_active":"2024-09-11T07:26:35.562245+0000","last_became_peered":"2024-09-11T07:26:35.562245+0000","last_unstale":"2024-09-11T07:38:59.653262+0000","last_undegraded":"2024-09-11T07:38:59.653262+0000","last_fullsized":"2024-09-11T07:38:59.653262+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:37:17.360183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:17.360183+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016828819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":294,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.668649+0000","last_change":"2024-09-11T07:37:42.084410+0000","last_active":"2024-09-11T07:38:50.668649+0000","last_peered":"2024-09-11T07:38:50.668649+0000","last_clean":"2024-09-11T07:38:50.668649+0000","last_became_active":"2024-09-11T07:32:00.509295+0000","last_became_peered":"2024-09-11T07:32:00.509295+0000","last_unstale":"2024-09-11T07:38:50.668649+0000","last_undegraded":"2024-09-11T07:38:50.668649+0000","last_fullsized":"2024-09-11T07:38:50.668649+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:42.084305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:42.084305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:53.270508+0000","scrub_duration":0.0047035039999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"272'6676","reported_seq":11519,"reported_epoch":272,"state":"active+remapped+backfilling","last_fresh":"2024-09-11T07:39:10.106501+0000","last_change":"2024-09-11T07:39:00.461377+0000","last_active":"2024-09-11T07:39:10.106501+0000","last_peered":"2024-09-11T07:39:10.106501+0000","last_clean":"2024-09-11T07:38:28.299325+0000","last_became_active":"2024-09-11T07:38:30.447489+0000","last_became_peered":"2024-09-11T07:38:30.447489+0000","last_unstale":"2024-09-11T07:39:10.106501+0000","last_undegraded":"2024-09-11T07:39:10.106501+0000","last_fullsized":"2024-09-11T07:39:10.106501+0000","mapping_epoch":262,"log_start":"254'500","ondisk_log_start":"254'500","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6176,"log_dups_size":0,"ondisk_log_size":6176,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255197184,"num_objects":3894,"num_object_clones":0,"num_object_copies":11682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4558,"num_objects_unfound":0,"num_objects_dirty":3894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4002,"num_write_kb":252672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1596,"num_bytes_recovered":104595456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[0,5,3],"avail_no_missing":["0","3","5"],"object_location_counts":[{"shards":"0,3,5","objects":3894}],"blocked_by":[],"up_primary":7,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"234'19","reported_seq":320,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.669025+0000","last_change":"2024-09-11T07:37:41.338132+0000","last_active":"2024-09-11T07:38:50.669025+0000","last_peered":"2024-09-11T07:38:50.669025+0000","last_clean":"2024-09-11T07:38:50.669025+0000","last_became_active":"2024-09-11T07:32:00.505134+0000","last_became_peered":"2024-09-11T07:32:00.505134+0000","last_unstale":"2024-09-11T07:38:50.669025+0000","last_undegraded":"2024-09-11T07:38:50.669025+0000","last_fullsized":"2024-09-11T07:38:50.669025+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:37:41.338036+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:37:41.338036+0000","last_clean_scrub_stamp":"2024-09-11T07:37:41.338036+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:08.290067+0000","scrub_duration":0.0053417279999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":273,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.668096+0000","last_change":"2024-09-11T07:36:27.958468+0000","last_active":"2024-09-11T07:38:50.668096+0000","last_peered":"2024-09-11T07:38:50.668096+0000","last_clean":"2024-09-11T07:38:50.668096+0000","last_became_active":"2024-09-11T07:26:31.419409+0000","last_became_peered":"2024-09-11T07:26:31.419409+0000","last_unstale":"2024-09-11T07:38:50.668096+0000","last_undegraded":"2024-09-11T07:38:50.668096+0000","last_fullsized":"2024-09-11T07:38:50.668096+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:27.958386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:27.958386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0093493710000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"151'10","reported_seq":441,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:39:04.664819+0000","last_change":"2024-09-11T07:38:25.065156+0000","last_active":"2024-09-11T07:39:04.664819+0000","last_peered":"2024-09-11T07:39:04.664819+0000","last_clean":"2024-09-11T07:39:04.664819+0000","last_became_active":"2024-09-11T07:32:00.504062+0000","last_became_peered":"2024-09-11T07:32:00.504062+0000","last_unstale":"2024-09-11T07:39:04.664819+0000","last_undegraded":"2024-09-11T07:39:04.664819+0000","last_fullsized":"2024-09-11T07:39:04.664819+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:38:25.065071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:38:25.065071+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:47.572471+0000","scrub_duration":0.019573179,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":267,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.666987+0000","last_change":"2024-09-11T07:37:26.659394+0000","last_active":"2024-09-11T07:38:50.666987+0000","last_peered":"2024-09-11T07:38:50.666987+0000","last_clean":"2024-09-11T07:38:50.666987+0000","last_became_active":"2024-09-11T07:26:37.565723+0000","last_became_peered":"2024-09-11T07:26:37.565723+0000","last_unstale":"2024-09-11T07:38:50.666987+0000","last_undegraded":"2024-09-11T07:38:50.666987+0000","last_fullsized":"2024-09-11T07:38:50.666987+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:26.659337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.659337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012563839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"236'16","reported_seq":306,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.669413+0000","last_change":"2024-09-11T07:36:49.809594+0000","last_active":"2024-09-11T07:38:50.669413+0000","last_peered":"2024-09-11T07:38:50.669413+0000","last_clean":"2024-09-11T07:38:50.669413+0000","last_became_active":"2024-09-11T07:26:33.540688+0000","last_became_peered":"2024-09-11T07:26:33.540688+0000","last_unstale":"2024-09-11T07:38:50.669413+0000","last_undegraded":"2024-09-11T07:38:50.669413+0000","last_fullsized":"2024-09-11T07:38:50.669413+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'16","last_scrub_stamp":"2024-09-11T07:36:49.809484+0000","last_deep_scrub":"236'16","last_deep_scrub_stamp":"2024-09-11T07:36:49.809484+0000","last_clean_scrub_stamp":"2024-09-11T07:36:49.809484+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017423357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":69,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.669775+0000","last_change":"2024-09-11T07:37:22.914840+0000","last_active":"2024-09-11T07:38:50.669775+0000","last_peered":"2024-09-11T07:38:50.669775+0000","last_clean":"2024-09-11T07:38:50.669775+0000","last_became_active":"2024-09-11T07:36:05.225465+0000","last_became_peered":"2024-09-11T07:36:05.225465+0000","last_unstale":"2024-09-11T07:38:50.669775+0000","last_undegraded":"2024-09-11T07:38:50.669775+0000","last_fullsized":"2024-09-11T07:38:50.669775+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:22.914781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:09.772000+0000","last_clean_scrub_stamp":"2024-09-11T07:37:22.914781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001446864,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":259,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.666990+0000","last_change":"2024-09-11T07:35:51.886845+0000","last_active":"2024-09-11T07:38:50.666990+0000","last_peered":"2024-09-11T07:38:50.666990+0000","last_clean":"2024-09-11T07:38:50.666990+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:38:50.666990+0000","last_undegraded":"2024-09-11T07:38:50.666990+0000","last_fullsized":"2024-09-11T07:38:50.666990+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:51.886407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:35:51.886407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002900241,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":277,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.668512+0000","last_change":"2024-09-11T07:37:32.186622+0000","last_active":"2024-09-11T07:38:50.668512+0000","last_peered":"2024-09-11T07:38:50.668512+0000","last_clean":"2024-09-11T07:38:50.668512+0000","last_became_active":"2024-09-11T07:26:37.568886+0000","last_became_peered":"2024-09-11T07:26:37.568886+0000","last_unstale":"2024-09-11T07:38:50.668512+0000","last_undegraded":"2024-09-11T07:38:50.668512+0000","last_fullsized":"2024-09-11T07:38:50.668512+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:32.186561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:32.186561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014950059999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"234'24","reported_seq":307,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.666150+0000","last_change":"2024-09-11T07:36:31.789902+0000","last_active":"2024-09-11T07:38:50.666150+0000","last_peered":"2024-09-11T07:38:50.666150+0000","last_clean":"2024-09-11T07:38:50.666150+0000","last_became_active":"2024-09-11T07:26:33.540208+0000","last_became_peered":"2024-09-11T07:26:33.540208+0000","last_unstale":"2024-09-11T07:38:50.666150+0000","last_undegraded":"2024-09-11T07:38:50.666150+0000","last_fullsized":"2024-09-11T07:38:50.666150+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'16","last_scrub_stamp":"2024-09-11T07:36:31.789841+0000","last_deep_scrub":"136'16","last_deep_scrub_stamp":"2024-09-11T07:31:06.039023+0000","last_clean_scrub_stamp":"2024-09-11T07:36:31.789841+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048561150000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":275,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.670567+0000","last_change":"2024-09-11T07:36:35.957044+0000","last_active":"2024-09-11T07:38:50.670567+0000","last_peered":"2024-09-11T07:38:50.670567+0000","last_clean":"2024-09-11T07:38:50.670567+0000","last_became_active":"2024-09-11T07:26:31.422671+0000","last_became_peered":"2024-09-11T07:26:31.422671+0000","last_unstale":"2024-09-11T07:38:50.670567+0000","last_undegraded":"2024-09-11T07:38:50.670567+0000","last_fullsized":"2024-09-11T07:38:50.670567+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:35.956943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:36:35.956943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053643689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":261,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.669072+0000","last_change":"2024-09-11T07:36:33.978210+0000","last_active":"2024-09-11T07:38:50.669072+0000","last_peered":"2024-09-11T07:38:50.669072+0000","last_clean":"2024-09-11T07:38:50.669072+0000","last_became_active":"2024-09-11T07:26:35.566931+0000","last_became_peered":"2024-09-11T07:26:35.566931+0000","last_unstale":"2024-09-11T07:38:50.669072+0000","last_undegraded":"2024-09-11T07:38:50.669072+0000","last_fullsized":"2024-09-11T07:38:50.669072+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:33.978003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.847474+0000","last_clean_scrub_stamp":"2024-09-11T07:36:33.978003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012287529,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":257,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.668746+0000","last_change":"2024-09-11T07:36:43.814458+0000","last_active":"2024-09-11T07:38:50.668746+0000","last_peered":"2024-09-11T07:38:50.668746+0000","last_clean":"2024-09-11T07:38:50.668746+0000","last_became_active":"2024-09-11T07:26:37.561573+0000","last_became_peered":"2024-09-11T07:26:37.561573+0000","last_unstale":"2024-09-11T07:38:50.668746+0000","last_undegraded":"2024-09-11T07:38:50.668746+0000","last_fullsized":"2024-09-11T07:38:50.668746+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:43.814370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:43.814370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0093820789999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"234'10","reported_seq":285,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.666195+0000","last_change":"2024-09-11T07:36:46.900027+0000","last_active":"2024-09-11T07:38:50.666195+0000","last_peered":"2024-09-11T07:38:50.666195+0000","last_clean":"2024-09-11T07:38:50.666195+0000","last_became_active":"2024-09-11T07:26:33.545830+0000","last_became_peered":"2024-09-11T07:26:33.545830+0000","last_unstale":"2024-09-11T07:38:50.666195+0000","last_undegraded":"2024-09-11T07:38:50.666195+0000","last_fullsized":"2024-09-11T07:38:50.666195+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'10","last_scrub_stamp":"2024-09-11T07:36:46.899947+0000","last_deep_scrub":"82'6","last_deep_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_clean_scrub_stamp":"2024-09-11T07:36:46.899947+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003412277,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":273,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.669522+0000","last_change":"2024-09-11T07:36:50.285861+0000","last_active":"2024-09-11T07:38:50.669522+0000","last_peered":"2024-09-11T07:38:50.669522+0000","last_clean":"2024-09-11T07:38:50.669522+0000","last_became_active":"2024-09-11T07:26:31.420518+0000","last_became_peered":"2024-09-11T07:26:31.420518+0000","last_unstale":"2024-09-11T07:38:50.669522+0000","last_undegraded":"2024-09-11T07:38:50.669522+0000","last_fullsized":"2024-09-11T07:38:50.669522+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:50.285756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_clean_scrub_stamp":"2024-09-11T07:36:50.285756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020130360000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":194,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.664856+0000","last_change":"2024-09-11T07:36:47.675100+0000","last_active":"2024-09-11T07:38:50.664856+0000","last_peered":"2024-09-11T07:38:50.664856+0000","last_clean":"2024-09-11T07:38:50.664856+0000","last_became_active":"2024-09-11T07:36:05.226194+0000","last_became_peered":"2024-09-11T07:36:05.226194+0000","last_unstale":"2024-09-11T07:38:50.664856+0000","last_undegraded":"2024-09-11T07:38:50.664856+0000","last_fullsized":"2024-09-11T07:38:50.664856+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:47.675050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:47.675050+0000","last_clean_scrub_stamp":"2024-09-11T07:36:47.675050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00141387,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":267,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.659064+0000","last_change":"2024-09-11T07:36:50.407702+0000","last_active":"2024-09-11T07:38:50.659064+0000","last_peered":"2024-09-11T07:38:50.659064+0000","last_clean":"2024-09-11T07:38:50.659064+0000","last_became_active":"2024-09-11T07:32:00.511407+0000","last_became_peered":"2024-09-11T07:32:00.511407+0000","last_unstale":"2024-09-11T07:38:50.659064+0000","last_undegraded":"2024-09-11T07:38:50.659064+0000","last_fullsized":"2024-09-11T07:38:50.659064+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:50.407649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:50.407649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01288723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"234'19","reported_seq":309,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.668985+0000","last_change":"2024-09-11T07:38:28.017180+0000","last_active":"2024-09-11T07:38:50.668985+0000","last_peered":"2024-09-11T07:38:50.668985+0000","last_clean":"2024-09-11T07:38:50.668985+0000","last_became_active":"2024-09-11T07:32:00.502081+0000","last_became_peered":"2024-09-11T07:32:00.502081+0000","last_unstale":"2024-09-11T07:38:50.668985+0000","last_undegraded":"2024-09-11T07:38:50.668985+0000","last_fullsized":"2024-09-11T07:38:50.668985+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:38:28.017124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:38:28.017124+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:35.454534+0000","scrub_duration":0.0018425620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":59,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.675925+0000","last_change":"2024-09-11T07:36:05.228927+0000","last_active":"2024-09-11T07:38:50.675925+0000","last_peered":"2024-09-11T07:38:50.675925+0000","last_clean":"2024-09-11T07:38:50.675925+0000","last_became_active":"2024-09-11T07:36:05.228360+0000","last_became_peered":"2024-09-11T07:36:05.228360+0000","last_unstale":"2024-09-11T07:38:50.675925+0000","last_undegraded":"2024-09-11T07:38:50.675925+0000","last_fullsized":"2024-09-11T07:38:50.675925+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:48.366931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:48.366931+0000","last_clean_scrub_stamp":"2024-09-11T07:35:48.366931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":240,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.659447+0000","last_change":"2024-09-11T07:36:36.444436+0000","last_active":"2024-09-11T07:38:50.659447+0000","last_peered":"2024-09-11T07:38:50.659447+0000","last_clean":"2024-09-11T07:38:50.659447+0000","last_became_active":"2024-09-11T07:32:00.520534+0000","last_became_peered":"2024-09-11T07:32:00.520534+0000","last_unstale":"2024-09-11T07:38:50.659447+0000","last_undegraded":"2024-09-11T07:38:50.659447+0000","last_fullsized":"2024-09-11T07:38:50.659447+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:36.444301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:39.021876+0000","last_clean_scrub_stamp":"2024-09-11T07:36:36.444301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015961,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":257,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.666473+0000","last_change":"2024-09-11T07:36:49.899993+0000","last_active":"2024-09-11T07:38:50.666473+0000","last_peered":"2024-09-11T07:38:50.666473+0000","last_clean":"2024-09-11T07:38:50.666473+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:38:50.666473+0000","last_undegraded":"2024-09-11T07:38:50.666473+0000","last_fullsized":"2024-09-11T07:38:50.666473+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:49.899620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:49.899620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014021288999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"272'6588","reported_seq":15536,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:39:09.268395+0000","last_change":"2024-09-11T07:38:45.294553+0000","last_active":"2024-09-11T07:39:09.268395+0000","last_peered":"2024-09-11T07:39:09.268395+0000","last_clean":"2024-09-11T07:39:09.268395+0000","last_became_active":"2024-09-11T07:38:45.291334+0000","last_became_peered":"2024-09-11T07:38:45.291334+0000","last_unstale":"2024-09-11T07:39:09.268395+0000","last_undegraded":"2024-09-11T07:39:09.268395+0000","last_fullsized":"2024-09-11T07:39:09.268395+0000","mapping_epoch":267,"log_start":"254'600","ondisk_log_start":"254'600","created":259,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":5988,"log_dups_size":0,"ondisk_log_size":5988,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":253853696,"num_objects":3874,"num_object_clones":0,"num_object_copies":11622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3936,"num_write_kb":249888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2959,"num_bytes_recovered":193921024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":244,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.659006+0000","last_change":"2024-09-11T07:36:44.447910+0000","last_active":"2024-09-11T07:38:50.659006+0000","last_peered":"2024-09-11T07:38:50.659006+0000","last_clean":"2024-09-11T07:38:50.659006+0000","last_became_active":"2024-09-11T07:32:00.511258+0000","last_became_peered":"2024-09-11T07:32:00.511258+0000","last_unstale":"2024-09-11T07:38:50.659006+0000","last_undegraded":"2024-09-11T07:38:50.659006+0000","last_fullsized":"2024-09-11T07:38:50.659006+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:36:44.447809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:44.447809+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005540555,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":292,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.668723+0000","last_change":"2024-09-11T07:36:31.353614+0000","last_active":"2024-09-11T07:38:50.668723+0000","last_peered":"2024-09-11T07:38:50.668723+0000","last_clean":"2024-09-11T07:38:50.668723+0000","last_became_active":"2024-09-11T07:32:00.518307+0000","last_became_peered":"2024-09-11T07:32:00.518307+0000","last_unstale":"2024-09-11T07:38:50.668723+0000","last_undegraded":"2024-09-11T07:38:50.668723+0000","last_fullsized":"2024-09-11T07:38:50.668723+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:36:31.353426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:31.353426+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0063773889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"234'20","reported_seq":111,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.667308+0000","last_change":"2024-09-11T07:37:29.661799+0000","last_active":"2024-09-11T07:38:50.667308+0000","last_peered":"2024-09-11T07:38:50.667308+0000","last_clean":"2024-09-11T07:38:50.667308+0000","last_became_active":"2024-09-11T07:35:00.561615+0000","last_became_peered":"2024-09-11T07:35:00.561615+0000","last_unstale":"2024-09-11T07:38:50.667308+0000","last_undegraded":"2024-09-11T07:38:50.667308+0000","last_fullsized":"2024-09-11T07:38:50.667308+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'20","last_scrub_stamp":"2024-09-11T07:37:29.661671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:29.661671+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:38:58.370347+0000","scrub_duration":0.0019609900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1f","version":"225'11","reported_seq":130,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:38:59.653291+0000","last_change":"2024-09-11T07:38:22.504282+0000","last_active":"2024-09-11T07:38:59.653291+0000","last_peered":"2024-09-11T07:38:59.653291+0000","last_clean":"2024-09-11T07:38:59.653291+0000","last_became_active":"2024-09-11T07:36:05.227140+0000","last_became_peered":"2024-09-11T07:36:05.227140+0000","last_unstale":"2024-09-11T07:38:59.653291+0000","last_undegraded":"2024-09-11T07:38:59.653291+0000","last_fullsized":"2024-09-11T07:38:59.653291+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"225'11","last_scrub_stamp":"2024-09-11T07:38:22.504199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:38:22.504199+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:36.665110+0000","scrub_duration":0.0020188300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"272'6609","reported_seq":6665,"reported_epoch":272,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-11T07:39:09.490056+0000","last_change":"2024-09-11T07:39:00.434624+0000","last_active":"2024-09-11T07:39:09.490056+0000","last_peered":"2024-09-11T07:39:09.490056+0000","last_clean":"2024-09-11T07:38:28.269880+0000","last_became_active":"2024-09-11T07:38:30.421449+0000","last_became_peered":"2024-09-11T07:38:30.421449+0000","last_unstale":"2024-09-11T07:39:09.490056+0000","last_undegraded":"2024-09-11T07:39:09.490056+0000","last_fullsized":"2024-09-11T07:39:09.490056+0000","mapping_epoch":262,"log_start":"254'400","ondisk_log_start":"254'400","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6209,"log_dups_size":0,"ondisk_log_size":6209,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":253165568,"num_objects":3863,"num_object_clones":0,"num_object_copies":11589,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7726,"num_objects_unfound":0,"num_objects_dirty":3863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3947,"num_write_kb":249920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[1,2,3],"avail_no_missing":["1","2","3"],"object_location_counts":[{"shards":"1,2,3","objects":3863}],"blocked_by":[],"up_primary":5,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":276,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.670102+0000","last_change":"2024-09-11T07:38:26.939048+0000","last_active":"2024-09-11T07:38:50.670102+0000","last_peered":"2024-09-11T07:38:50.670102+0000","last_clean":"2024-09-11T07:38:50.670102+0000","last_became_active":"2024-09-11T07:26:37.562501+0000","last_became_peered":"2024-09-11T07:26:37.562501+0000","last_unstale":"2024-09-11T07:38:50.670102+0000","last_undegraded":"2024-09-11T07:38:50.670102+0000","last_fullsized":"2024-09-11T07:38:50.670102+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:26.938987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:53.875300+0000","last_clean_scrub_stamp":"2024-09-11T07:38:26.938987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:43.951909+0000","scrub_duration":0.0016308410000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":273,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.669371+0000","last_change":"2024-09-11T07:36:38.841363+0000","last_active":"2024-09-11T07:38:50.669371+0000","last_peered":"2024-09-11T07:38:50.669371+0000","last_clean":"2024-09-11T07:38:50.669371+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:38:50.669371+0000","last_undegraded":"2024-09-11T07:38:50.669371+0000","last_fullsized":"2024-09-11T07:38:50.669371+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:38.841250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:38.841250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014360088999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"234'19","reported_seq":112,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.675800+0000","last_change":"2024-09-11T07:38:25.113379+0000","last_active":"2024-09-11T07:38:50.675800+0000","last_peered":"2024-09-11T07:38:50.675800+0000","last_clean":"2024-09-11T07:38:50.675800+0000","last_became_active":"2024-09-11T07:36:05.228746+0000","last_became_peered":"2024-09-11T07:36:05.228746+0000","last_unstale":"2024-09-11T07:38:50.675800+0000","last_undegraded":"2024-09-11T07:38:50.675800+0000","last_fullsized":"2024-09-11T07:38:50.675800+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:38:25.113300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:38:25.113300+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:40.256440+0000","scrub_duration":0.0063902960000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":286,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.669193+0000","last_change":"2024-09-11T07:37:21.150801+0000","last_active":"2024-09-11T07:38:50.669193+0000","last_peered":"2024-09-11T07:38:50.669193+0000","last_clean":"2024-09-11T07:38:50.669193+0000","last_became_active":"2024-09-11T07:32:00.507050+0000","last_became_peered":"2024-09-11T07:32:00.507050+0000","last_unstale":"2024-09-11T07:38:50.669193+0000","last_undegraded":"2024-09-11T07:38:50.669193+0000","last_fullsized":"2024-09-11T07:38:50.669193+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:21.150732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:24.575210+0000","last_clean_scrub_stamp":"2024-09-11T07:37:21.150732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001360691,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.11","version":"272'6731","reported_seq":6782,"reported_epoch":272,"state":"active+clean","last_fresh":"2024-09-11T07:39:09.282772+0000","last_change":"2024-09-11T07:38:29.349114+0000","last_active":"2024-09-11T07:39:09.282772+0000","last_peered":"2024-09-11T07:39:09.282772+0000","last_clean":"2024-09-11T07:39:09.282772+0000","last_became_active":"2024-09-11T07:38:29.346889+0000","last_became_peered":"2024-09-11T07:38:29.346889+0000","last_unstale":"2024-09-11T07:39:09.282772+0000","last_undegraded":"2024-09-11T07:39:09.282772+0000","last_fullsized":"2024-09-11T07:39:09.282772+0000","mapping_epoch":261,"log_start":"254'500","ondisk_log_start":"254'500","created":259,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6231,"log_dups_size":0,"ondisk_log_size":6231,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":260702208,"num_objects":3978,"num_object_clones":0,"num_object_copies":11934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4054,"num_write_kb":257024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":270,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.667509+0000","last_change":"2024-09-11T07:38:47.462111+0000","last_active":"2024-09-11T07:38:50.667509+0000","last_peered":"2024-09-11T07:38:50.667509+0000","last_clean":"2024-09-11T07:38:50.667509+0000","last_became_active":"2024-09-11T07:26:37.862373+0000","last_became_peered":"2024-09-11T07:26:37.862373+0000","last_unstale":"2024-09-11T07:38:50.667509+0000","last_undegraded":"2024-09-11T07:38:50.667509+0000","last_fullsized":"2024-09-11T07:38:50.667509+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:49.723695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:35:49.723695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00410707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":208,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.665922+0000","last_change":"2024-09-11T07:37:41.657806+0000","last_active":"2024-09-11T07:38:50.665922+0000","last_peered":"2024-09-11T07:38:50.665922+0000","last_clean":"2024-09-11T07:38:50.665922+0000","last_became_active":"2024-09-11T07:36:05.224085+0000","last_became_peered":"2024-09-11T07:36:05.224085+0000","last_unstale":"2024-09-11T07:38:50.665922+0000","last_undegraded":"2024-09-11T07:38:50.665922+0000","last_fullsized":"2024-09-11T07:38:50.665922+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:41.657719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:41.657719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002776653,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"234'27","reported_seq":327,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.662780+0000","last_change":"2024-09-11T07:38:45.004976+0000","last_active":"2024-09-11T07:38:50.662780+0000","last_peered":"2024-09-11T07:38:50.662780+0000","last_clean":"2024-09-11T07:38:50.662780+0000","last_became_active":"2024-09-11T07:26:33.542544+0000","last_became_peered":"2024-09-11T07:26:33.542544+0000","last_unstale":"2024-09-11T07:38:50.662780+0000","last_undegraded":"2024-09-11T07:38:50.662780+0000","last_fullsized":"2024-09-11T07:38:50.662780+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'27","last_scrub_stamp":"2024-09-11T07:37:18.388604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:18.388604+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019030239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":268,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.670430+0000","last_change":"2024-09-11T07:36:44.853517+0000","last_active":"2024-09-11T07:38:50.670430+0000","last_peered":"2024-09-11T07:38:50.670430+0000","last_clean":"2024-09-11T07:38:50.670430+0000","last_became_active":"2024-09-11T07:26:35.555862+0000","last_became_peered":"2024-09-11T07:26:35.555862+0000","last_unstale":"2024-09-11T07:38:50.670430+0000","last_undegraded":"2024-09-11T07:38:50.670430+0000","last_fullsized":"2024-09-11T07:38:50.670430+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:44.850049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:36:44.850049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020441075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"272'6713","reported_seq":6773,"reported_epoch":272,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-11T07:39:09.486890+0000","last_change":"2024-09-11T07:39:00.462758+0000","last_active":"2024-09-11T07:39:09.486890+0000","last_peered":"2024-09-11T07:39:09.486890+0000","last_clean":"2024-09-11T07:38:28.331892+0000","last_became_active":"2024-09-11T07:38:30.445788+0000","last_became_peered":"2024-09-11T07:38:30.445788+0000","last_unstale":"2024-09-11T07:39:09.486890+0000","last_undegraded":"2024-09-11T07:39:09.486890+0000","last_fullsized":"2024-09-11T07:39:09.486890+0000","mapping_epoch":262,"log_start":"254'500","ondisk_log_start":"254'500","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6213,"log_dups_size":0,"ondisk_log_size":6213,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":257294336,"num_objects":3926,"num_object_clones":0,"num_object_copies":11778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":11778,"num_objects_unfound":0,"num_objects_dirty":3926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3994,"num_write_kb":253440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[1,2,6],"avail_no_missing":["1","2","6"],"object_location_counts":[{"shards":"1,2,6","objects":3926}],"blocked_by":[],"up_primary":4,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":267,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.668448+0000","last_change":"2024-09-11T07:37:32.004184+0000","last_active":"2024-09-11T07:38:50.668448+0000","last_peered":"2024-09-11T07:38:50.668448+0000","last_clean":"2024-09-11T07:38:50.668448+0000","last_became_active":"2024-09-11T07:26:37.861647+0000","last_became_peered":"2024-09-11T07:26:37.861647+0000","last_unstale":"2024-09-11T07:38:50.668448+0000","last_undegraded":"2024-09-11T07:38:50.668448+0000","last_fullsized":"2024-09-11T07:38:50.668448+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:32.004125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:32.004125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019485520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":290,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.667156+0000","last_change":"2024-09-11T07:37:42.078889+0000","last_active":"2024-09-11T07:38:50.667156+0000","last_peered":"2024-09-11T07:38:50.667156+0000","last_clean":"2024-09-11T07:38:50.667156+0000","last_became_active":"2024-09-11T07:32:00.502947+0000","last_became_peered":"2024-09-11T07:32:00.502947+0000","last_unstale":"2024-09-11T07:38:50.667156+0000","last_undegraded":"2024-09-11T07:38:50.667156+0000","last_fullsized":"2024-09-11T07:38:50.667156+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:42.078765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:42.078765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0062011560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"234'23","reported_seq":308,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.667979+0000","last_change":"2024-09-11T07:37:43.078969+0000","last_active":"2024-09-11T07:38:50.667979+0000","last_peered":"2024-09-11T07:38:50.667979+0000","last_clean":"2024-09-11T07:38:50.667979+0000","last_became_active":"2024-09-11T07:26:33.542711+0000","last_became_peered":"2024-09-11T07:26:33.542711+0000","last_unstale":"2024-09-11T07:38:50.667979+0000","last_undegraded":"2024-09-11T07:38:50.667979+0000","last_fullsized":"2024-09-11T07:38:50.667979+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'23","last_scrub_stamp":"2024-09-11T07:37:43.078872+0000","last_deep_scrub":"127'17","last_deep_scrub_stamp":"2024-09-11T07:35:43.871918+0000","last_clean_scrub_stamp":"2024-09-11T07:37:43.078872+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:39:09.927640+0000","scrub_duration":0.0051065670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":272,"reported_epoch":271,"state":"active+clean","last_fresh":"2024-09-11T07:38:50.666863+0000","last_change":"2024-09-11T07:38:48.487630+0000","last_active":"2024-09-11T07:38:50.666863+0000","last_peered":"2024-09-11T07:38:50.666863+0000","last_clean":"2024-09-11T07:38:50.666863+0000","last_became_active":"2024-09-11T07:26:35.562037+0000","last_became_peered":"2024-09-11T07:26:35.562037+0000","last_unstale":"2024-09-11T07:38:50.666863+0000","last_undegraded":"2024-09-11T07:38:50.666863+0000","last_fullsized":"2024-09-11T07:38:50.666863+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:37.811009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:36:37.811009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039204729999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":26,"stat_sum":{"num_bytes":8214020120,"num_objects":125337,"num_object_clones":0,"num_object_copies":376011,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":36978,"num_objects_unfound":0,"num_objects_dirty":125337,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":127841,"num_write_kb":8101633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10643,"num_bytes_recovered":697499648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":25826308096,"data_stored":25826295880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153201,"ondisk_log_size":153201,"up":78,"acting":76,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1124,"num_read_kb":881,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":764,"ondisk_log_size":764,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":749,"num_read_kb":744,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":147,"seq":631360192599,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4877856,"kb_used_data":4557400,"kb_used_omap":4,"kb_used_meta":320315,"kb_avail":88867296,"statfs":{"total":95995035648,"available":91000111104,"internally_reserved":0,"allocated":4666777600,"data_stored":4666411874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":328003078},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:38:14 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":1.1850000000000001,"15min":2.9390000000000001},"min":{"1min":0.223,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":2.1150000000000002,"5min":14.086,"15min":49.573},"last":1.831},{"interface":"front","average":{"1min":0.70099999999999996,"5min":1.278,"15min":2.7440000000000002},"min":{"1min":0.26200000000000001,"5min":0.223,"15min":0.22},"max":{"1min":1.9550000000000001,"5min":9.5030000000000001,"15min":49.881},"last":1.756}]},{"osd":1,"last update":"Wed Sep 11 07:38:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":1.004,"15min":0.73199999999999998},"min":{"1min":0.30499999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":2.2320000000000002,"5min":10.063000000000001,"15min":10.063000000000001},"last":1.5820000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":1.1890000000000001,"15min":0.93200000000000005},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":2.2160000000000002,"5min":9.6059999999999999,"15min":9.6059999999999999},"last":0.25700000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:38:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.98499999999999999,"15min":0.98699999999999999},"min":{"1min":0.33100000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.6220000000000001,"5min":13.99,"15min":13.99},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.83099999999999996,"5min":1.179,"15min":1.0049999999999999},"min":{"1min":0.29099999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.8759999999999999,"5min":14.67,"15min":14.67},"last":0.35099999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:38:14 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.157,"15min":2.7850000000000001},"min":{"1min":0.307,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":2.7629999999999999,"5min":4.6589999999999998,"15min":49.798999999999999},"last":1.8859999999999999},{"interface":"front","average":{"1min":1.04,"5min":1.236,"15min":2.7210000000000001},"min":{"1min":0.30099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":5.891,"5min":15.316000000000001,"15min":49.524000000000001},"last":2.0459999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:38:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.66300000000000003,"15min":0.498},"min":{"1min":0.25700000000000001,"5min":0.253,"15min":0.253},"max":{"1min":1.506,"5min":6.0220000000000002,"15min":6.0220000000000002},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.65900000000000003,"15min":0.621},"min":{"1min":0.35099999999999998,"5min":0.247,"15min":0.247},"max":{"1min":1.536,"5min":3.9340000000000002,"15min":3.9340000000000002},"last":0.36499999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:38:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.747,"15min":0.54200000000000004},"min":{"1min":0.35299999999999998,"5min":0.309,"15min":0.26500000000000001},"max":{"1min":1.6120000000000001,"5min":5.9909999999999997,"15min":5.9909999999999997},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.78400000000000003,"15min":0.69999999999999996},"min":{"1min":0.32300000000000001,"5min":0.27900000000000003,"15min":0.27600000000000002},"max":{"1min":2.1230000000000002,"5min":4.9969999999999999,"15min":4.9969999999999999},"last":0.39600000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:38:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.67700000000000005,"15min":0.56599999999999995},"min":{"1min":0.36399999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.3440000000000001,"5min":4.6440000000000001,"15min":4.6440000000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.73599999999999999,"15min":0.60399999999999998},"min":{"1min":0.32900000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.591,"5min":3.694,"15min":3.694},"last":0.376}]}]},{"osd":6,"up_from":44,"seq":188978561184,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2345540,"kb_used_data":2048188,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":91399612,"statfs":{"total":95995035648,"available":93593202688,"internally_reserved":0,"allocated":2097344512,"data_stored":2097005912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:38:16 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":2.5190000000000001,"15min":1.778},"min":{"1min":0.311,"5min":0.20699999999999999,"15min":0.192},"max":{"1min":3.379,"5min":142.29599999999999,"15min":142.29599999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.98899999999999999,"5min":2.7170000000000001,"15min":1.9159999999999999},"min":{"1min":0.26800000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":4.3810000000000002,"5min":142.68799999999999,"15min":142.68799999999999},"last":3.1400000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:38:16 2024","interfaces":[{"interface":"back","average":{"1min":2.8069999999999999,"5min":3.081,"15min":1.472},"min":{"1min":0.30399999999999999,"5min":0.249,"15min":0.23599999999999999},"max":{"1min":37.350999999999999,"5min":127.92100000000001,"15min":127.92100000000001},"last":0.27500000000000002},{"interface":"front","average":{"1min":2.8610000000000002,"5min":2.9550000000000001,"15min":1.4710000000000001},"min":{"1min":0.33700000000000002,"5min":0.24099999999999999,"15min":0.23000000000000001},"max":{"1min":39.262999999999998,"5min":127.746,"15min":127.746},"last":0.36899999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:38:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.83799999999999997,"15min":0.85299999999999998},"min":{"1min":0.254,"5min":0.248,"15min":0.248},"max":{"1min":0.88600000000000001,"5min":3.6539999999999999,"15min":26.491},"last":2.1989999999999998},{"interface":"front","average":{"1min":0.84799999999999998,"5min":1.1930000000000001,"15min":1.0329999999999999},"min":{"1min":0.28299999999999997,"5min":0.27100000000000002,"15min":0.26400000000000001},"max":{"1min":3.2200000000000002,"5min":6.0819999999999999,"15min":26.561},"last":0.39300000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:38:16 2024","interfaces":[{"interface":"back","average":{"1min":2.7050000000000001,"5min":1.7549999999999999,"15min":1.212},"min":{"1min":0.29699999999999999,"5min":0.26100000000000001,"15min":0.254},"max":{"1min":40.212000000000003,"5min":40.212000000000003,"15min":40.212000000000003},"last":2.278},{"interface":"front","average":{"1min":2.6760000000000002,"5min":1.534,"15min":1.284},"min":{"1min":0.34599999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":37.567999999999998,"5min":37.567999999999998,"15min":37.567999999999998},"last":2.1549999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:38:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.65900000000000003,"15min":0.55500000000000005},"min":{"1min":0.28100000000000003,"5min":0.223,"15min":0.223},"max":{"1min":2.1000000000000001,"5min":3.3599999999999999,"15min":4.4720000000000004},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.56799999999999995,"15min":0.46999999999999997},"min":{"1min":0.34000000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.369,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":0.443}]},{"osd":5,"last update":"Wed Sep 11 07:38:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":1.4199999999999999,"15min":0.84699999999999998},"min":{"1min":0.32700000000000001,"5min":0.255,"15min":0.255},"max":{"1min":1.145,"5min":57.436,"15min":57.436},"last":2.5859999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":1.462,"15min":0.82199999999999995},"min":{"1min":0.28599999999999998,"5min":0.251,"15min":0.251},"max":{"1min":2.637,"5min":56.822000000000003,"15min":56.822000000000003},"last":0.52300000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:38:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.67800000000000005,"15min":0.66300000000000003},"min":{"1min":0.309,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.4319999999999999,"5min":3.3140000000000001,"15min":3.3140000000000001},"last":2.133},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.70899999999999996,"15min":0.71899999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.1319999999999999,"5min":5.0439999999999996,"15min":5.0439999999999996},"last":2.2389999999999999}]}]},{"osd":5,"up_from":37,"seq":158913790118,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4945880,"kb_used_data":4644368,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":88799272,"statfs":{"total":95995035648,"available":90930454528,"internally_reserved":0,"allocated":4755832832,"data_stored":4755474400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:38:52 2024","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.7,"15min":2.1829999999999998},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":10.65,"5min":31.369,"15min":185.68199999999999},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.874,"5min":1.8480000000000001,"15min":2.3010000000000002},"min":{"1min":0.23400000000000001,"5min":0.223,"15min":0.218},"max":{"1min":5.5739999999999998,"5min":25.722999999999999,"15min":186.12200000000001},"last":0.437}]},{"osd":1,"last update":"Wed Sep 11 07:38:52 2024","interfaces":[{"interface":"back","average":{"1min":2.536,"5min":1.657,"15min":1.0389999999999999},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":13.058,"5min":26.286999999999999,"15min":26.286999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.4399999999999999,"15min":1.0009999999999999},"min":{"1min":0.28499999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":5.6470000000000002,"5min":24.672999999999998,"15min":24.672999999999998},"last":0.28299999999999997}]},{"osd":2,"last update":"Wed Sep 11 07:38:52 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.0629999999999999,"15min":1.607},"min":{"1min":0.315,"5min":0.28399999999999997,"15min":0.22700000000000001},"max":{"1min":5.726,"5min":5.726,"15min":187.99199999999999},"last":0.34799999999999998},{"interface":"front","average":{"1min":1.399,"5min":1.097,"15min":1.603},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":13.016,"5min":13.016,"15min":186.375},"last":0.38500000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:38:52 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.675,"15min":2.2669999999999999},"min":{"1min":0.26900000000000002,"5min":0.26400000000000001,"15min":0.23599999999999999},"max":{"1min":5.593,"5min":28.978000000000002,"15min":188.11500000000001},"last":12.983000000000001},{"interface":"front","average":{"1min":2.2970000000000002,"5min":1.881,"15min":2.3210000000000002},"min":{"1min":0.30099999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":13.081,"5min":26.936,"15min":186.44200000000001},"last":13.765000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:38:52 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.67900000000000005,"15min":0.5},"min":{"1min":0.255,"5min":0.247,"15min":0.19900000000000001},"max":{"1min":5.4359999999999999,"5min":5.4359999999999999,"15min":5.4359999999999999},"last":0.246},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71699999999999997,"15min":0.52400000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.9350000000000001,"5min":2.8439999999999999,"15min":2.8439999999999999},"last":0.29099999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:38:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.73399999999999999,"15min":0.54000000000000004},"min":{"1min":0.314,"5min":0.29699999999999999,"15min":0.26000000000000001},"max":{"1min":2.02,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66200000000000003,"15min":0.52800000000000002},"min":{"1min":0.32700000000000001,"5min":0.26800000000000002,"15min":0.25800000000000001},"max":{"1min":2.2229999999999999,"5min":4.3120000000000003,"15min":4.3120000000000003},"last":0.36399999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:38:12 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.77600000000000002,"15min":0.65200000000000002},"min":{"1min":0.30099999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":2.8959999999999999,"5min":4.7919999999999998,"15min":4.7919999999999998},"last":0.308},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.81899999999999995,"15min":0.67500000000000004},"min":{"1min":0.39000000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":2.641,"5min":4.0110000000000001,"15min":4.0110000000000001},"last":0.31900000000000001}]}]},{"osd":4,"up_from":31,"seq":133143986346,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2075032,"kb_used_data":1769360,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":91670120,"statfs":{"total":95995035648,"available":93870202880,"internally_reserved":0,"allocated":1811824640,"data_stored":1811466027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:38:38 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.331,"15min":1.155},"min":{"1min":0.26700000000000002,"5min":0.217,"15min":0.20799999999999999},"max":{"1min":9.2330000000000005,"5min":32.892000000000003,"15min":35.308},"last":0.499},{"interface":"front","average":{"1min":0.80100000000000005,"5min":1.391,"15min":1.107},"min":{"1min":0.223,"5min":0.223,"15min":0.20499999999999999},"max":{"1min":2.3860000000000001,"5min":32.860999999999997,"15min":35.597999999999999},"last":0.222}]},{"osd":1,"last update":"Wed Sep 11 07:38:38 2024","interfaces":[{"interface":"back","average":{"1min":2.004,"5min":1.613,"15min":1.077},"min":{"1min":0.23100000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":21.651,"5min":24.581,"15min":24.581},"last":0.52600000000000002},{"interface":"front","average":{"1min":2.2330000000000001,"5min":1.587,"15min":0.99199999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":21.824000000000002,"5min":24.652000000000001,"15min":24.652000000000001},"last":3.2290000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:38:38 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.93300000000000005,"15min":0.92700000000000005},"min":{"1min":0.25700000000000001,"5min":0.223,"15min":0.223},"max":{"1min":1.5029999999999999,"5min":6.0490000000000004,"15min":19.927},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":1.0840000000000001,"15min":0.92300000000000004},"min":{"1min":0.31,"5min":0.27200000000000002,"15min":0.24399999999999999},"max":{"1min":1.532,"5min":13.423999999999999,"15min":20.02},"last":1.0640000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:38:38 2024","interfaces":[{"interface":"back","average":{"1min":1.7250000000000001,"5min":1.462,"15min":1.151},"min":{"1min":0.34300000000000003,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":13.853,"5min":24.029,"15min":24.029},"last":1.028},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.4119999999999999,"15min":1.0680000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.28699999999999998},"max":{"1min":13.651,"5min":24.265000000000001,"15min":24.265000000000001},"last":0.47299999999999998}]},{"osd":5,"last update":"Wed Sep 11 07:38:56 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.73799999999999999,"15min":0.501},"min":{"1min":0.23799999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.84199999999999997,"5min":4.6079999999999997,"15min":4.6079999999999997},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.78700000000000003,"15min":0.53300000000000003},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.22700000000000001},"max":{"1min":2.3010000000000002,"5min":5.7320000000000002,"15min":5.7320000000000002},"last":0.39300000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.72499999999999998,"15min":0.53100000000000003},"min":{"1min":0.27000000000000002,"5min":0.23999999999999999,"15min":0.20899999999999999},"max":{"1min":1.0569999999999999,"5min":7.8650000000000002,"15min":7.8650000000000002},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.73799999999999999,"15min":0.54400000000000004},"min":{"1min":0.28599999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.5449999999999999,"5min":9.3770000000000007,"15min":9.3770000000000007},"last":0.748}]},{"osd":7,"last update":"Wed Sep 11 07:38:11 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.63800000000000001,"15min":0.57599999999999996},"min":{"1min":0.311,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63700000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.1859999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.70599999999999996,"15min":0.60099999999999998},"min":{"1min":0.36399999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.93000000000000005,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":0.44600000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182575,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4850592,"kb_used_data":4264856,"kb_used_omap":1,"kb_used_meta":585726,"kb_avail":88894560,"statfs":{"total":95995035648,"available":91028029440,"internally_reserved":0,"allocated":4367212544,"data_stored":4366845203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":599783768},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":1.153,"15min":1.2290000000000001},"min":{"1min":0.26100000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":2.883,"5min":7.3289999999999997,"15min":38.432000000000002},"last":18.704000000000001},{"interface":"front","average":{"1min":1.26,"5min":1.3779999999999999,"15min":1.6000000000000001},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.19500000000000001},"max":{"1min":3.456,"5min":6.9119999999999999,"15min":91.876999999999995},"last":14.749000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:38:18 2024","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":2.1850000000000001,"15min":1.4630000000000001},"min":{"1min":0.26100000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":3.5720000000000001,"5min":67.305000000000007,"15min":67.305000000000007},"last":13.664},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4159999999999999,"15min":1.085},"min":{"1min":0.24099999999999999,"5min":0.192,"15min":0.192},"max":{"1min":4.859,"5min":9.3450000000000006,"15min":9.3450000000000006},"last":14.334}]},{"osd":2,"last update":"Wed Sep 11 07:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.1559999999999999,"15min":1.1299999999999999},"min":{"1min":0.33000000000000002,"5min":0.23699999999999999,"15min":0.23200000000000001},"max":{"1min":2.7469999999999999,"5min":6.6619999999999999,"15min":8.282},"last":0.248},{"interface":"front","average":{"1min":0.91600000000000004,"5min":1.127,"15min":1.0589999999999999},"min":{"1min":0.29299999999999998,"5min":0.26200000000000001,"15min":0.20999999999999999},"max":{"1min":3.6419999999999999,"5min":6.5640000000000001,"15min":8.7110000000000003},"last":0.25600000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:38:36 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.042,"15min":0.997},"min":{"1min":0.42999999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":2.956,"5min":4.125,"15min":21.856000000000002},"last":0.34200000000000003},{"interface":"front","average":{"1min":1.6739999999999999,"5min":1.2090000000000001,"15min":1.1779999999999999},"min":{"1min":0.45700000000000002,"5min":0.255,"15min":0.255},"max":{"1min":3.4889999999999999,"5min":4.6219999999999999,"15min":19.146000000000001},"last":0.36699999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:38:50 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.133,"15min":1.079},"min":{"1min":0.36299999999999999,"5min":0.309,"15min":0.29599999999999999},"max":{"1min":2.9239999999999999,"5min":5.0209999999999999,"15min":32.325000000000003},"last":0.41999999999999998},{"interface":"front","average":{"1min":1.54,"5min":1.329,"15min":1.173},"min":{"1min":0.29199999999999998,"5min":0.28599999999999998,"15min":0.28299999999999997},"max":{"1min":3.5499999999999998,"5min":4.8490000000000002,"15min":18.416},"last":0.33000000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":1.0369999999999999,"15min":1.3049999999999999},"min":{"1min":0.38300000000000001,"5min":0.315,"15min":0.27000000000000002},"max":{"1min":2.6949999999999998,"5min":5.4539999999999997,"15min":32.350999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96199999999999997,"15min":1.2869999999999999},"min":{"1min":0.39700000000000002,"5min":0.34699999999999998,"15min":0.33100000000000002},"max":{"1min":2.9870000000000001,"5min":4.5199999999999996,"15min":32.402999999999999},"last":0.38}]},{"osd":7,"last update":"Wed Sep 11 07:38:13 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.1479999999999999,"15min":1.3089999999999999},"min":{"1min":0.42399999999999999,"5min":0.32300000000000001,"15min":0.308},"max":{"1min":3.5259999999999998,"5min":4.6859999999999999,"15min":8.0429999999999993},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":1.129,"15min":2.0470000000000002},"min":{"1min":0.41899999999999998,"5min":0.36599999999999999,"15min":0.36099999999999999},"max":{"1min":2.7250000000000001,"5min":5.5309999999999997,"15min":32.289999999999999},"last":0.38900000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411508,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1813824,"kb_used_data":1516472,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":91931328,"statfs":{"total":95995035648,"available":94137679872,"internally_reserved":0,"allocated":1552867328,"data_stored":1552532407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:38:50 2024","interfaces":[{"interface":"back","average":{"1min":4.056,"5min":2.4420000000000002,"15min":1.714},"min":{"1min":0.17799999999999999,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":57.802999999999997,"5min":57.802999999999997,"15min":57.802999999999997},"last":0.32400000000000001},{"interface":"front","average":{"1min":4.157,"5min":2.661,"15min":1.7969999999999999},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":57.759,"5min":57.759,"15min":57.759},"last":0.40600000000000003}]},{"osd":1,"last update":"Wed Sep 11 07:38:50 2024","interfaces":[{"interface":"back","average":{"1min":4.4169999999999998,"5min":2.3100000000000001,"15min":1.4370000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":48.042000000000002,"5min":65.153999999999996,"15min":65.153999999999996},"last":3.6549999999999998},{"interface":"front","average":{"1min":4.5170000000000003,"5min":2.77,"15min":1.6359999999999999},"min":{"1min":0.39900000000000002,"5min":0.255,"15min":0.20699999999999999},"max":{"1min":50.201000000000001,"5min":63.311999999999998,"15min":63.311999999999998},"last":0.86899999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:38:18 2024","interfaces":[{"interface":"back","average":{"1min":2.444,"5min":1.9239999999999999,"15min":1.4570000000000001},"min":{"1min":0.30399999999999999,"5min":0.27400000000000002,"15min":0.26900000000000002},"max":{"1min":18.329000000000001,"5min":44.405999999999999,"15min":44.405999999999999},"last":0.31},{"interface":"front","average":{"1min":1.7849999999999999,"5min":1.77,"15min":1.4410000000000001},"min":{"1min":0.27600000000000002,"5min":0.253,"15min":0.253},"max":{"1min":17.896999999999998,"5min":43.295999999999999,"15min":43.295999999999999},"last":0.35899999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:38:37 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.88300000000000001,"15min":1.0569999999999999},"min":{"1min":0.437,"5min":0.30599999999999999,"15min":0.27500000000000002},"max":{"1min":2.6400000000000001,"5min":4.0940000000000003,"15min":30.946999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.86199999999999999,"15min":0.94999999999999996},"min":{"1min":0.39900000000000002,"5min":0.31900000000000001,"15min":0.29999999999999999},"max":{"1min":3.1669999999999998,"5min":3.1669999999999998,"15min":22.434000000000001},"last":0.48499999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:38:50 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.91200000000000003,"15min":1.0640000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.31900000000000001},"max":{"1min":3.214,"5min":3.3290000000000002,"15min":31.789000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.81599999999999995,"5min":1.0580000000000001,"15min":1.141},"min":{"1min":0.41099999999999998,"5min":0.34100000000000003,"15min":0.31},"max":{"1min":2.1760000000000002,"5min":4.1479999999999997,"15min":30.898},"last":0.90000000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.90700000000000003,"15min":1.0509999999999999},"min":{"1min":0.42499999999999999,"5min":0.373,"15min":0.36099999999999999},"max":{"1min":2.3599999999999999,"5min":3.0129999999999999,"15min":22.399000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.96399999999999997,"15min":1.054},"min":{"1min":0.44900000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.7559999999999998,"5min":4.1230000000000002,"15min":25.739000000000001},"last":0.42399999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.85999999999999999,"15min":1.8420000000000001},"min":{"1min":0.47499999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.97199999999999998,"5min":2.6600000000000001,"15min":22.484000000000002},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.749,"5min":1.046,"15min":2.1640000000000001},"min":{"1min":0.46100000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.7869999999999999,"5min":4.1719999999999997,"15min":25.670000000000002},"last":0.82299999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575031,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4882536,"kb_used_data":4585120,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":88862616,"statfs":{"total":95995035648,"available":90995318784,"internally_reserved":0,"allocated":4695162880,"data_stored":4694788185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":1.5569999999999999,"15min":1.726},"min":{"1min":0.17999999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":3.9199999999999999,"5min":8.0549999999999997,"15min":82.218999999999994},"last":4.899},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.71,"15min":1.77},"min":{"1min":0.193,"5min":0.193,"15min":0.191},"max":{"1min":6.9880000000000004,"5min":8.0739999999999998,"15min":82.278999999999996},"last":0.35999999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:38:56 2024","interfaces":[{"interface":"back","average":{"1min":1.827,"5min":1.923,"15min":1.3129999999999999},"min":{"1min":0.26700000000000002,"5min":0.22,"15min":0.19800000000000001},"max":{"1min":6.7480000000000002,"5min":9.0969999999999995,"15min":9.0969999999999995},"last":0.308},{"interface":"front","average":{"1min":2.222,"5min":1.849,"15min":1.371},"min":{"1min":0.254,"5min":0.191,"15min":0.191},"max":{"1min":20.518000000000001,"5min":20.518000000000001,"15min":20.518000000000001},"last":2.5150000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:38:11 2024","interfaces":[{"interface":"back","average":{"1min":5.0529999999999999,"5min":2.4710000000000001,"15min":2.0779999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.25800000000000001},"max":{"1min":67.120999999999995,"5min":67.120999999999995,"15min":81.882999999999996},"last":2.6139999999999999},{"interface":"front","average":{"1min":5.0010000000000003,"5min":2.3239999999999998,"15min":2.0289999999999999},"min":{"1min":0.27000000000000002,"5min":0.22600000000000001,"15min":0.20999999999999999},"max":{"1min":66.888999999999996,"5min":66.888999999999996,"15min":82.400000000000006},"last":3.1219999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:38:29 2024","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.4019999999999999,"15min":0.999},"min":{"1min":0.30499999999999999,"5min":0.27500000000000002,"15min":0.27400000000000002},"max":{"1min":5.3550000000000004,"5min":7.3630000000000004,"15min":7.3630000000000004},"last":2.0350000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.4370000000000001,"15min":1.127},"min":{"1min":0.37,"5min":0.313,"15min":0.26500000000000001},"max":{"1min":4.0759999999999996,"5min":9.2289999999999992,"15min":9.2289999999999992},"last":3.9540000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:38:56 2024","interfaces":[{"interface":"back","average":{"1min":2.1059999999999999,"5min":1.655,"15min":1.121},"min":{"1min":0.442,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":6.718,"5min":7.4379999999999997,"15min":7.4379999999999997},"last":2.0649999999999999},{"interface":"front","average":{"1min":1.583,"5min":1.5740000000000001,"15min":1.181},"min":{"1min":0.47999999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":4.0519999999999996,"5min":9.1929999999999996,"15min":9.1929999999999996},"last":2.661}]},{"osd":6,"last update":"Wed Sep 11 07:38:11 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.395,"15min":1.181},"min":{"1min":0.38500000000000001,"5min":0.32400000000000001,"15min":0.30099999999999999},"max":{"1min":4.0999999999999996,"5min":9.2569999999999997,"15min":24.018000000000001},"last":2.6400000000000001},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.006,"15min":0.98599999999999999},"min":{"1min":0.38900000000000001,"5min":0.313,"15min":0.29999999999999999},"max":{"1min":3.9609999999999999,"5min":5.2709999999999999,"15min":23.681000000000001},"last":2.98}]},{"osd":7,"last update":"Wed Sep 11 07:38:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.161,"15min":1.2889999999999999},"min":{"1min":0.41599999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":4.048,"5min":7.0940000000000003,"15min":7.0940000000000003},"last":3.0409999999999999},{"interface":"front","average":{"1min":1.548,"5min":1.48,"15min":1.5149999999999999},"min":{"1min":0.44400000000000001,"5min":0.313,"15min":0.313},"max":{"1min":5.3220000000000001,"5min":7.4050000000000002,"15min":7.4050000000000002},"last":2.089}]}]},{"osd":0,"up_from":8,"seq":34359738557,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2254484,"kb_used_data":1948812,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":91490668,"statfs":{"total":95995035648,"available":93686444032,"internally_reserved":0,"allocated":1995583488,"data_stored":1995228836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:38:21 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.248,"15min":1.024},"min":{"1min":0.20000000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":3.8580000000000001,"5min":5.4630000000000001,"15min":9.7710000000000008},"last":0.497},{"interface":"front","average":{"1min":1.252,"5min":1.3540000000000001,"15min":1.0740000000000001},"min":{"1min":0.22900000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":5.4279999999999999,"5min":5.4279999999999999,"15min":10.275},"last":0.77800000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:38:51 2024","interfaces":[{"interface":"back","average":{"1min":11.262,"5min":3.4390000000000001,"15min":1.9319999999999999},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":200.339,"5min":200.339,"15min":200.339},"last":0.47699999999999998},{"interface":"front","average":{"1min":11.214,"5min":3.2349999999999999,"15min":1.8660000000000001},"min":{"1min":0.246,"5min":0.21099999999999999,"15min":0.20999999999999999},"max":{"1min":200.40100000000001,"5min":200.40100000000001,"15min":200.40100000000001},"last":0.57999999999999996}]},{"osd":3,"last update":"Wed Sep 11 07:38:21 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.419,"15min":1.7829999999999999},"min":{"1min":0.253,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":5.5250000000000004,"5min":9.4920000000000009,"15min":45.185000000000002},"last":0.61799999999999999},{"interface":"front","average":{"1min":1.077,"5min":1.2989999999999999,"15min":1.534},"min":{"1min":0.24299999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":3.8300000000000001,"5min":8.0999999999999996,"15min":43.228999999999999},"last":0.55600000000000005}]},{"osd":4,"last update":"Wed Sep 11 07:38:33 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.1379999999999999,"15min":1.018},"min":{"1min":0.318,"5min":0.26800000000000002,"15min":0.26700000000000002},"max":{"1min":5.1139999999999999,"5min":5.1139999999999999,"15min":19.411999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.1619999999999999,"15min":1.1559999999999999},"min":{"1min":0.32800000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":5.3529999999999998,"5min":9.5619999999999994,"15min":19.396000000000001},"last":0.71899999999999997}]},{"osd":5,"last update":"Wed Sep 11 07:38:51 2024","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.278,"15min":1.151},"min":{"1min":0.35399999999999998,"5min":0.29599999999999999,"15min":0.28999999999999998},"max":{"1min":4.9749999999999996,"5min":5.0830000000000002,"15min":19.381},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.607,"5min":1.3100000000000001,"15min":1.248},"min":{"1min":0.38800000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":5.4100000000000001,"5min":9.6289999999999996,"15min":19.437000000000001},"last":0.68700000000000006}]},{"osd":6,"last update":"Wed Sep 11 07:38:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.284,"15min":1.1830000000000001},"min":{"1min":0.42499999999999999,"5min":0.33300000000000002,"15min":0.316},"max":{"1min":5.4640000000000004,"5min":23.190999999999999,"15min":23.190999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":1.2430000000000001,"15min":1.1739999999999999},"min":{"1min":0.36599999999999999,"5min":0.35999999999999999,"15min":0.34200000000000003},"max":{"1min":3.7170000000000001,"5min":23.632000000000001,"15min":23.632000000000001},"last":0.66500000000000004}]},{"osd":7,"last update":"Wed Sep 11 07:38:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1559999999999999,"15min":1.6160000000000001},"min":{"1min":0.46400000000000002,"5min":0.373,"15min":0.373},"max":{"1min":5.4400000000000004,"5min":9.6530000000000005,"15min":9.6530000000000005},"last":0.746},{"interface":"front","average":{"1min":0.88100000000000001,"5min":1.0680000000000001,"15min":1.6399999999999999},"min":{"1min":0.36899999999999999,"5min":0.315,"15min":0.315},"max":{"1min":3.79,"5min":4.194,"15min":5.2759999999999998},"last":0.73099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1985806336,"data_stored":1985806336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4682616832,"data_stored":4682612760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1547042816,"data_stored":1547042816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4327735296,"data_stored":4327735296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1802895360,"data_stored":1802895360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4747169792,"data_stored":4747165720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2091909120,"data_stored":2091909120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4641132544,"data_stored":4641128472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:39:15.311 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pool set unique_pool_2 pgp_num 26 2024-09-11T07:39:15.387 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:15.418 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:15.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:15.419 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:15.425 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T07:39:15.520 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:15.547 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:15.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:15.548 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:15.579 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:15 smithi022 ceph-mon[37330]: pgmap v567: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 7.6 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 48 MiB/s wr, 1.03k op/s; 36978/372624 objects misplaced (9.924%); 16 MiB/s, 250 objects/s recovering 2024-09-11T07:39:15.579 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:15 smithi022 ceph-mon[29462]: pgmap v567: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 7.6 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 48 MiB/s wr, 1.03k op/s; 36978/372624 objects misplaced (9.924%); 16 MiB/s, 250 objects/s recovering 2024-09-11T07:39:15.649 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:15.672 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:15.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:15.673 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:15.774 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:15.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:15 smithi104 ceph-mon[33507]: pgmap v567: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 7.6 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 48 MiB/s wr, 1.03k op/s; 36978/372624 objects misplaced (9.924%); 16 MiB/s, 250 objects/s recovering 2024-09-11T07:39:15.802 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:15.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:15.803 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:15.904 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:15.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:15.937 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:15.937 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:16.039 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:16.093 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:16.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:16.095 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:16.196 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:16.239 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:16.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:16.240 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:16.341 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:16.369 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:16.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:16.370 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:16.471 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:16.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:16.516 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:16.516 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:16.617 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:16.660 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:39:16.660 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:39:16.674 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:16.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:16.675 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:16.701 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:39:16.701 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:39:16.737 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:16 smithi022 ceph-mon[37330]: pgmap v568: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 7.3 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 20 MiB/s wr, 1.06k op/s; 33176/358335 objects misplaced (9.258%); 17 MiB/s, 266 objects/s recovering 2024-09-11T07:39:16.737 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:16 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3492851929' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:39:16.737 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:39:16.737 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:39:16.738 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:16 smithi022 ceph-mon[29462]: pgmap v568: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 7.3 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 20 MiB/s wr, 1.06k op/s; 33176/358335 objects misplaced (9.258%); 17 MiB/s, 266 objects/s recovering 2024-09-11T07:39:16.738 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:16 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3492851929' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:39:16.768 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:39:16.768 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:39:16.776 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:16.798 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:39:16.798 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:39:16.824 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:16.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:16.825 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:16.835 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:39:16.835 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:39:16.864 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:39:16.864 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:39:16.896 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:39:16.897 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:39:16.925 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:16.956 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:16.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:16.957 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:16 smithi104 ceph-mon[33507]: pgmap v568: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 7.3 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 20 MiB/s wr, 1.06k op/s; 33176/358335 objects misplaced (9.258%); 17 MiB/s, 266 objects/s recovering 2024-09-11T07:39:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:16 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3492851929' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:39:17.059 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:17.092 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:17.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:17.093 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:17.194 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:17.234 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:17.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:17.235 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:17.336 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:17.376 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:17.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:17.377 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:17.478 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:17.507 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:39:17.508 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:39:17.508 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:39:17.538 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:17.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:17.539 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:17.640 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:17.674 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:17.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:17.675 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:17.776 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:17.808 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:17.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:17.809 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:17.910 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:17.957 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:17.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:17.958 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:18.059 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:18.136 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:18.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:18.139 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:18.240 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:18.286 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:18.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:18.287 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:18.388 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:18.418 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:18.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:18.419 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:18.520 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:18.559 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:18.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:18.560 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:18.661 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:18.699 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:18.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:18.700 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:18 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:39:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:18 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:39:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:18 smithi104 ceph-mon[33507]: pgmap v569: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 7.3 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 20 MiB/s wr, 1.06k op/s; 33176/358335 objects misplaced (9.258%); 17 MiB/s, 266 objects/s recovering 2024-09-11T07:39:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:18 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:39:18.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:18 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:39:18.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:18 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:39:18.801 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:18.841 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:18.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:18.842 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:18.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:18 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:39:18.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:18 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:39:18.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:18 smithi022 ceph-mon[29462]: pgmap v569: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 7.3 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 20 MiB/s wr, 1.06k op/s; 33176/358335 objects misplaced (9.258%); 17 MiB/s, 266 objects/s recovering 2024-09-11T07:39:18.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:18 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:39:18.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:18 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:39:18.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:18 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:39:18.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:18 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:39:18.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:18 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:39:18.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:18 smithi022 ceph-mon[37330]: pgmap v569: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 7.3 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 20 MiB/s wr, 1.06k op/s; 33176/358335 objects misplaced (9.258%); 17 MiB/s, 266 objects/s recovering 2024-09-11T07:39:18.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:18 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:39:18.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:18 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:39:18.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:18 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:39:18.943 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:18.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:18.985 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:19.086 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:19.133 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:19.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:19.134 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:19.234 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:19.265 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:19.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:19.266 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:19.368 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:19.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:19.408 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:19.408 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:19.509 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:19.551 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:19.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:19.552 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:19.652 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:19.689 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:19.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:19.690 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:19.791 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:19.826 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:19.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:19.827 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:19.928 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:19.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:19.958 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:19.958 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:20.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:39:19 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:39:20.059 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:20.104 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:20.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:20.105 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:20.126 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:39:20.206 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:20.258 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:20.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:20.259 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:20.360 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:20.386 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:20.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:20.387 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:20.488 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:20.491 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:39:20 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:39:20] "GET /metrics HTTP/1.1" 200 224535 "" "Prometheus/2.43.0" 2024-09-11T07:39:20.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:20.519 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:20.519 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:20.620 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:20.647 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:20.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:20.648 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:20.749 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:20.771 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:39:20.771 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:39:20.778 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:20.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:20.779 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:20.880 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:20.915 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:20.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:20.916 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:21.017 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:21.064 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:21.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:21.065 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:21.166 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:21.203 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:21.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:21.204 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:21.305 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:21.326 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:21.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:21.327 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:21.428 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:21.465 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:21.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:21.467 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:21.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:21 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:39:21.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:21 smithi104 ceph-mon[33507]: pgmap v570: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 6.9 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.20k op/s; 29293/340719 objects misplaced (8.597%); 14 MiB/s, 231 objects/s recovering 2024-09-11T07:39:21.567 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:21.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:21.602 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:21.602 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:21.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:21 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:39:21.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:21 smithi022 ceph-mon[29462]: pgmap v570: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 6.9 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.20k op/s; 29293/340719 objects misplaced (8.597%); 14 MiB/s, 231 objects/s recovering 2024-09-11T07:39:21.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:21 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:39:21.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:21 smithi022 ceph-mon[37330]: pgmap v570: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 6.9 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.20k op/s; 29293/340719 objects misplaced (8.597%); 14 MiB/s, 231 objects/s recovering 2024-09-11T07:39:21.703 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:21.728 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:21.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:21.729 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:21.830 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:21.865 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:21.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:21.866 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:21.967 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:22.002 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:22.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:22.003 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:22.104 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:22.144 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:22.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:22.145 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:22.246 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:22.281 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:22.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:22.282 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:22.383 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:22.421 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:22.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:22.422 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:22.523 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:22.553 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:22.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:22.554 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:22.655 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:22.694 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:22.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:22.695 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:22.796 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:22.836 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:22.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:22.837 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:22.875 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-11T07:39:22.937 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:23.018 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:23.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:23.019 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:23.120 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:23.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:23.183 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:23.183 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:23.284 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:23.316 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:23.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:23.317 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:23.418 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:23.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:23.469 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:23.470 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:23.571 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:23.610 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:23.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:23.611 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:23 smithi022 ceph-mon[29462]: pgmap v571: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 6.9 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.20k op/s; 29293/340719 objects misplaced (8.597%); 14 MiB/s, 231 objects/s recovering 2024-09-11T07:39:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:39:23.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:23 smithi022 ceph-mon[37330]: pgmap v571: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 6.9 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.20k op/s; 29293/340719 objects misplaced (8.597%); 14 MiB/s, 231 objects/s recovering 2024-09-11T07:39:23.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:39:23.712 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:23.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:23.740 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:23.740 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:23.744 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:39:23.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:23 smithi104 ceph-mon[33507]: pgmap v571: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 6.9 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.20k op/s; 29293/340719 objects misplaced (8.597%); 14 MiB/s, 231 objects/s recovering 2024-09-11T07:39:23.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:39:23.842 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:23.873 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:23.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:23.874 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:23.975 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:24.008 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:24.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:24.009 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:24.110 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:24.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:24.155 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:24.155 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:24.256 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:24.298 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:24.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:24.299 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:24.400 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:24.445 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:24.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:24.446 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:24.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:39:24 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:39:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:39:24.547 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:24.592 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:24.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:24.593 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:24.693 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:24.742 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:24.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:24.743 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:24.844 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:24.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:24.894 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:24.894 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:24.995 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:25.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:25.037 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:25.037 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:25.138 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:25.161 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:25.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:25.163 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:25.264 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:25.303 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:25.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:25.305 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:25.314 INFO:teuthology.orchestra.run.smithi022.stderr:set pool 9 pgp_num to 26 2024-09-11T07:39:25.314 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is unset 2024-09-11T07:39:25.405 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:25.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:25.445 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:25.445 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:25.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:25 smithi104 ceph-mon[33507]: pgmap v572: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 6.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 863 B/s rd, 0 B/s wr, 1.12k op/s; 29293/336726 objects misplaced (8.699%); 12 MiB/s, 195 objects/s recovering 2024-09-11T07:39:25.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:25 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3833142030' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "26"}]: dispatch 2024-09-11T07:39:25.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:25 smithi104 ceph-mon[33507]: from='client.25489 ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "26"}]: dispatch 2024-09-11T07:39:25.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:25 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3528713561' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:39:25.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:25 smithi104 ceph-mon[33507]: from='client.25495 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:39:25.546 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:25.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:25.588 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:25.588 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:25.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:25 smithi022 ceph-mon[29462]: pgmap v572: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 6.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 863 B/s rd, 0 B/s wr, 1.12k op/s; 29293/336726 objects misplaced (8.699%); 12 MiB/s, 195 objects/s recovering 2024-09-11T07:39:25.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:25 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3833142030' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "26"}]: dispatch 2024-09-11T07:39:25.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:25 smithi022 ceph-mon[29462]: from='client.25489 ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "26"}]: dispatch 2024-09-11T07:39:25.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:25 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3528713561' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:39:25.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:25 smithi022 ceph-mon[29462]: from='client.25495 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:39:25.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:25 smithi022 ceph-mon[37330]: pgmap v572: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 6.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 863 B/s rd, 0 B/s wr, 1.12k op/s; 29293/336726 objects misplaced (8.699%); 12 MiB/s, 195 objects/s recovering 2024-09-11T07:39:25.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:25 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3833142030' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "26"}]: dispatch 2024-09-11T07:39:25.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:25 smithi022 ceph-mon[37330]: from='client.25489 ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "26"}]: dispatch 2024-09-11T07:39:25.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:25 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3528713561' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:39:25.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:25 smithi022 ceph-mon[37330]: from='client.25495 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:39:25.689 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:25.726 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:25.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:25.728 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:25.828 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:25.861 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:25.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:25.862 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:25.963 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:25.993 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:25.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:25.994 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:26.095 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:26.122 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:26.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:26.123 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:26.224 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:26.250 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:26.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:26.251 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:26.351 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:26.375 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:26.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:26.376 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:26.477 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:26.499 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:26.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:26.500 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:26.601 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:26.624 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:26.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:26.626 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:26.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:26 smithi022 ceph-mon[29462]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:39:26.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:26 smithi022 ceph-mon[29462]: from='client.25489 ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "26"}]': finished 2024-09-11T07:39:26.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:26 smithi022 ceph-mon[29462]: from='client.25495 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:39:26.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:26 smithi022 ceph-mon[29462]: osdmap e274: 8 total, 8 up, 8 in 2024-09-11T07:39:26.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:26 smithi022 ceph-mon[29462]: osdmap e275: 8 total, 8 up, 8 in 2024-09-11T07:39:26.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:26 smithi022 ceph-mon[37330]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:39:26.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:26 smithi022 ceph-mon[37330]: from='client.25489 ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "26"}]': finished 2024-09-11T07:39:26.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:26 smithi022 ceph-mon[37330]: from='client.25495 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:39:26.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:26 smithi022 ceph-mon[37330]: osdmap e274: 8 total, 8 up, 8 in 2024-09-11T07:39:26.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:26 smithi022 ceph-mon[37330]: osdmap e275: 8 total, 8 up, 8 in 2024-09-11T07:39:26.726 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:26.749 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:26.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:26.750 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:26.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:26 smithi104 ceph-mon[33507]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:39:26.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:26 smithi104 ceph-mon[33507]: from='client.25489 ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "26"}]': finished 2024-09-11T07:39:26.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:26 smithi104 ceph-mon[33507]: from='client.25495 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:39:26.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:26 smithi104 ceph-mon[33507]: osdmap e274: 8 total, 8 up, 8 in 2024-09-11T07:39:26.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:26 smithi104 ceph-mon[33507]: osdmap e275: 8 total, 8 up, 8 in 2024-09-11T07:39:26.851 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:26.890 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:26.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:26.891 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:26.992 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:27.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:27.034 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:27.034 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:27.135 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:27.174 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:27.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:27.175 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:27.276 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:27.319 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:27.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:27.320 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:27.421 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:27.457 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:27.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:27.458 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:27.559 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:27.597 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:27.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:27.599 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:27.613 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:27 smithi022 ceph-mon[29462]: 5.a scrub starts 2024-09-11T07:39:27.613 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:27 smithi022 ceph-mon[29462]: pgmap v575: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 6.6 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.47k op/s; 25529/323175 objects misplaced (7.899%); 19 MiB/s, 299 objects/s recovering 2024-09-11T07:39:27.614 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:27 smithi022 ceph-mon[29462]: osdmap e276: 8 total, 8 up, 8 in 2024-09-11T07:39:27.614 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:27 smithi022 ceph-mon[37330]: 5.a scrub starts 2024-09-11T07:39:27.614 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:27 smithi022 ceph-mon[37330]: pgmap v575: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 6.6 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.47k op/s; 25529/323175 objects misplaced (7.899%); 19 MiB/s, 299 objects/s recovering 2024-09-11T07:39:27.614 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:27 smithi022 ceph-mon[37330]: osdmap e276: 8 total, 8 up, 8 in 2024-09-11T07:39:27.700 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:27.721 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:27.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:27.723 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:27.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:27 smithi104 ceph-mon[33507]: 5.a scrub starts 2024-09-11T07:39:27.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:27 smithi104 ceph-mon[33507]: pgmap v575: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 6.6 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.47k op/s; 25529/323175 objects misplaced (7.899%); 19 MiB/s, 299 objects/s recovering 2024-09-11T07:39:27.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:27 smithi104 ceph-mon[33507]: osdmap e276: 8 total, 8 up, 8 in 2024-09-11T07:39:27.823 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:27.857 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:27.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:27.858 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:27.959 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:27.984 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:27.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:27.985 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:28.086 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:28.118 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:28.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:28.119 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:28.220 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:28.241 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:28.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:28.242 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:28.343 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:28.387 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:28.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:28.388 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:28.489 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:28.515 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:28.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:28.516 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:28.617 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:28.651 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:28.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:28.652 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:28.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:28 smithi022 ceph-mon[29462]: 2.f scrub starts 2024-09-11T07:39:28.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:28 smithi022 ceph-mon[37330]: 2.f scrub starts 2024-09-11T07:39:28.752 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:28.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:28 smithi104 ceph-mon[33507]: 2.f scrub starts 2024-09-11T07:39:28.786 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:28.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:28.787 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:28.888 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:28.919 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:28.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:28.920 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:29.021 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:29.043 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:29.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:29.044 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:29.145 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:29.166 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:29.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:29.167 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:29.267 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:29.291 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:29.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:29.292 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:29.393 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:29.426 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:29.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:29.427 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:29.528 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:29.558 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:29.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:29.559 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:29.660 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:29.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:29 smithi022 ceph-mon[29462]: pgmap v577: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 6.6 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 975 op/s; 25529/323175 objects misplaced (7.899%); 13 MiB/s, 202 objects/s recovering 2024-09-11T07:39:29.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:29 smithi022 ceph-mon[29462]: osdmap e277: 8 total, 8 up, 8 in 2024-09-11T07:39:29.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:29 smithi022 ceph-mon[29462]: 6.1e scrub starts 2024-09-11T07:39:29.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:29 smithi022 ceph-mon[29462]: 6.1e scrub ok 2024-09-11T07:39:29.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:29 smithi022 ceph-mon[37330]: pgmap v577: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 6.6 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 975 op/s; 25529/323175 objects misplaced (7.899%); 13 MiB/s, 202 objects/s recovering 2024-09-11T07:39:29.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:29 smithi022 ceph-mon[37330]: osdmap e277: 8 total, 8 up, 8 in 2024-09-11T07:39:29.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:29 smithi022 ceph-mon[37330]: 6.1e scrub starts 2024-09-11T07:39:29.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:29 smithi022 ceph-mon[37330]: 6.1e scrub ok 2024-09-11T07:39:29.694 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:29.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:29.695 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:29.700 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:29 smithi104 ceph-mon[33507]: pgmap v577: 158 pgs: 1 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 6.6 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 975 op/s; 25529/323175 objects misplaced (7.899%); 13 MiB/s, 202 objects/s recovering 2024-09-11T07:39:29.700 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:29 smithi104 ceph-mon[33507]: osdmap e277: 8 total, 8 up, 8 in 2024-09-11T07:39:29.700 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:29 smithi104 ceph-mon[33507]: 6.1e scrub starts 2024-09-11T07:39:29.700 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:29 smithi104 ceph-mon[33507]: 6.1e scrub ok 2024-09-11T07:39:29.796 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:29.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:29.832 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:29.832 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:29.933 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:29.970 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:29.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:29.971 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:30.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:39:29 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:39:30.072 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:30.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:30.093 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:30.094 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:30.194 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:30.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:30.232 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:30.232 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:30.333 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:30.377 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:30.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:30.378 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:30.479 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:30.515 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:30.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:30.516 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:30.617 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:30.658 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:30.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:30.659 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:30.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:39:30 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:39:30] "GET /metrics HTTP/1.1" 200 224534 "" "Prometheus/2.43.0" 2024-09-11T07:39:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:30 smithi022 ceph-mon[29462]: 2.17 scrub starts 2024-09-11T07:39:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:30 smithi022 ceph-mon[29462]: 2.17 scrub ok 2024-09-11T07:39:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:30 smithi022 ceph-mon[29462]: osdmap e278: 8 total, 8 up, 8 in 2024-09-11T07:39:30.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:30 smithi022 ceph-mon[37330]: 2.17 scrub starts 2024-09-11T07:39:30.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:30 smithi022 ceph-mon[37330]: 2.17 scrub ok 2024-09-11T07:39:30.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:30 smithi022 ceph-mon[37330]: osdmap e278: 8 total, 8 up, 8 in 2024-09-11T07:39:30.760 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:30.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:30 smithi104 ceph-mon[33507]: 2.17 scrub starts 2024-09-11T07:39:30.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:30 smithi104 ceph-mon[33507]: 2.17 scrub ok 2024-09-11T07:39:30.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:30 smithi104 ceph-mon[33507]: osdmap e278: 8 total, 8 up, 8 in 2024-09-11T07:39:30.798 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:30.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:30.799 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:30.900 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:30.931 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:30.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:30.932 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:31.033 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:31.069 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:31.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:31.071 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:31.171 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:31.213 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:31.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:31.214 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:31.314 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:31.342 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:31.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:31.343 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:31.444 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:31.465 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:31.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:31.466 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:31.567 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:31.591 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:31.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:31.592 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:31.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:31 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:39:31.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:31 smithi022 ceph-mon[29462]: pgmap v580: 158 pgs: 1 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 6.3 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 820 op/s; 22704/311874 objects misplaced (7.280%); 12 MiB/s, 191 objects/s recovering 2024-09-11T07:39:31.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:31 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:39:31.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:31 smithi022 ceph-mon[37330]: pgmap v580: 158 pgs: 1 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 6.3 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 820 op/s; 22704/311874 objects misplaced (7.280%); 12 MiB/s, 191 objects/s recovering 2024-09-11T07:39:31.693 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:31.731 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:31.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:31.732 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:31.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:31 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:39:32.115 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:31 smithi104 ceph-mon[33507]: pgmap v580: 158 pgs: 1 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 6.3 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 820 op/s; 22704/311874 objects misplaced (7.280%); 12 MiB/s, 191 objects/s recovering 2024-09-11T07:39:32.115 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:32.139 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:32.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:32.140 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:32.241 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:32.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:32.282 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:32.282 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:32.383 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:32.405 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:32.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:32.406 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:32.507 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:32.543 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:32.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:32.545 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:32.645 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:32.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:32 smithi022 ceph-mon[29462]: 5.1c scrub starts 2024-09-11T07:39:32.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:32 smithi022 ceph-mon[37330]: 5.1c scrub starts 2024-09-11T07:39:32.671 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:32.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:32.672 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:32.773 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:32 smithi104 ceph-mon[33507]: 5.1c scrub starts 2024-09-11T07:39:32.798 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:32.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:32.799 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:32.900 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:32.923 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:32.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:32.924 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:33.025 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:33.048 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:33.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:33.049 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:33.149 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:33.172 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:33.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:33.173 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:33.273 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:33.298 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:33.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:33.299 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:33.400 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:33.435 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:33.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:33.436 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:33.537 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:33.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:33.570 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:33.570 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:33.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:33 smithi022 ceph-mon[29462]: pgmap v581: 158 pgs: 1 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 6.3 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 726 op/s; 22704/310116 objects misplaced (7.321%); 9.2 MiB/s, 146 objects/s recovering 2024-09-11T07:39:33.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:33 smithi022 ceph-mon[29462]: Health check failed: Degraded data redundancy: 1 pg undersized (PG_DEGRADED) 2024-09-11T07:39:33.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:33 smithi022 ceph-mon[29462]: osdmap e279: 8 total, 8 up, 8 in 2024-09-11T07:39:33.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:33 smithi022 ceph-mon[29462]: 5.1a scrub starts 2024-09-11T07:39:33.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:33 smithi022 ceph-mon[29462]: 4.18 scrub starts 2024-09-11T07:39:33.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:33 smithi022 ceph-mon[37330]: pgmap v581: 158 pgs: 1 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 6.3 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 726 op/s; 22704/310116 objects misplaced (7.321%); 9.2 MiB/s, 146 objects/s recovering 2024-09-11T07:39:33.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:33 smithi022 ceph-mon[37330]: Health check failed: Degraded data redundancy: 1 pg undersized (PG_DEGRADED) 2024-09-11T07:39:33.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:33 smithi022 ceph-mon[37330]: osdmap e279: 8 total, 8 up, 8 in 2024-09-11T07:39:33.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:33 smithi022 ceph-mon[37330]: 5.1a scrub starts 2024-09-11T07:39:33.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:33 smithi022 ceph-mon[37330]: 4.18 scrub starts 2024-09-11T07:39:33.671 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:33.711 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:33.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:33.712 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:33.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:33 smithi104 ceph-mon[33507]: pgmap v581: 158 pgs: 1 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 6.3 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 726 op/s; 22704/310116 objects misplaced (7.321%); 9.2 MiB/s, 146 objects/s recovering 2024-09-11T07:39:33.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:33 smithi104 ceph-mon[33507]: Health check failed: Degraded data redundancy: 1 pg undersized (PG_DEGRADED) 2024-09-11T07:39:33.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:33 smithi104 ceph-mon[33507]: osdmap e279: 8 total, 8 up, 8 in 2024-09-11T07:39:33.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:33 smithi104 ceph-mon[33507]: 5.1a scrub starts 2024-09-11T07:39:33.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:33 smithi104 ceph-mon[33507]: 4.18 scrub starts 2024-09-11T07:39:33.812 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:33.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:33.847 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:33.847 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:33.948 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:33.990 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:33.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:33.991 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:34.092 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:34.105 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:39:34.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:34.137 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:34.138 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:34.238 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:34.283 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:34.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:34.284 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:34.349 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:39:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:39:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:39:34.385 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:34.415 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:34.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:34.416 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:34.516 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:34.551 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:34.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:34.552 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:34.620 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:34 smithi022 ceph-mon[29462]: 5.8 scrub starts 2024-09-11T07:39:34.620 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:34 smithi022 ceph-mon[29462]: osdmap e280: 8 total, 8 up, 8 in 2024-09-11T07:39:34.620 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:34 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:39:34.620 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:34 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:39:34.620 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:34 smithi022 ceph-mon[29462]: 4.c scrub starts 2024-09-11T07:39:34.620 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:34 smithi022 ceph-mon[29462]: 4.c scrub ok 2024-09-11T07:39:34.620 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:34 smithi022 ceph-mon[37330]: 5.8 scrub starts 2024-09-11T07:39:34.620 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:34 smithi022 ceph-mon[37330]: osdmap e280: 8 total, 8 up, 8 in 2024-09-11T07:39:34.620 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:34 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:39:34.621 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:34 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:39:34.621 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:34 smithi022 ceph-mon[37330]: 4.c scrub starts 2024-09-11T07:39:34.621 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:34 smithi022 ceph-mon[37330]: 4.c scrub ok 2024-09-11T07:39:34.653 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:34.690 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:34.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:34.691 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:34.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:34 smithi104 ceph-mon[33507]: 5.8 scrub starts 2024-09-11T07:39:34.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:34 smithi104 ceph-mon[33507]: osdmap e280: 8 total, 8 up, 8 in 2024-09-11T07:39:34.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:34 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:39:34.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:34 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:39:34.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:34 smithi104 ceph-mon[33507]: 4.c scrub starts 2024-09-11T07:39:34.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:34 smithi104 ceph-mon[33507]: 4.c scrub ok 2024-09-11T07:39:34.792 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:34.833 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:34.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:34.835 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:34.936 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:34.984 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:35.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:35.003 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:35.104 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:35.144 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:35.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:35.145 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:35.246 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:35.276 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:35.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:35.277 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:35.330 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T07:39:35.377 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:35.378 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:39:35.379 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:39:35.419 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:39:35.420 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:39:35.427 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:35.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:35.428 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:35.455 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:39:35.456 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:39:35.456 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:35 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:39:35.456 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:35 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:39:35.457 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:35 smithi022 ceph-mon[37330]: pgmap v584: 158 pgs: 1 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 6.2 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 865 B/s rd, 0 B/s wr, 943 op/s; 22704/306450 objects misplaced (7.409%); 9.3 MiB/s, 149 objects/s recovering 2024-09-11T07:39:35.457 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:35 smithi022 ceph-mon[37330]: 2.10 scrub starts 2024-09-11T07:39:35.457 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:35 smithi022 ceph-mon[37330]: osdmap e281: 8 total, 8 up, 8 in 2024-09-11T07:39:35.457 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:35 smithi022 ceph-mon[37330]: 5.11 scrub starts 2024-09-11T07:39:35.457 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:35 smithi022 ceph-mon[37330]: 5.11 scrub ok 2024-09-11T07:39:35.457 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:35 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2358605498' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:39:35.458 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:35 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:39:35.458 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:35 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:39:35.458 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:35 smithi022 ceph-mon[29462]: pgmap v584: 158 pgs: 1 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 6.2 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 865 B/s rd, 0 B/s wr, 943 op/s; 22704/306450 objects misplaced (7.409%); 9.3 MiB/s, 149 objects/s recovering 2024-09-11T07:39:35.458 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:35 smithi022 ceph-mon[29462]: 2.10 scrub starts 2024-09-11T07:39:35.458 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:35 smithi022 ceph-mon[29462]: osdmap e281: 8 total, 8 up, 8 in 2024-09-11T07:39:35.458 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:35 smithi022 ceph-mon[29462]: 5.11 scrub starts 2024-09-11T07:39:35.458 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:35 smithi022 ceph-mon[29462]: 5.11 scrub ok 2024-09-11T07:39:35.458 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:35 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2358605498' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:39:35.491 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:39:35.491 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:39:35.523 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:39:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:39:35.529 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:35.562 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:39:35.563 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:39:35.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:35.586 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:35.587 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:35.597 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:39:35.597 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:39:35.636 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:39:35.637 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:39:35.687 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:35.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:35.719 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:35.719 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:35.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:35 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:39:35.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:35 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:39:35.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:35 smithi104 ceph-mon[33507]: pgmap v584: 158 pgs: 1 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 6.2 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 865 B/s rd, 0 B/s wr, 943 op/s; 22704/306450 objects misplaced (7.409%); 9.3 MiB/s, 149 objects/s recovering 2024-09-11T07:39:35.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:35 smithi104 ceph-mon[33507]: 2.10 scrub starts 2024-09-11T07:39:35.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:35 smithi104 ceph-mon[33507]: osdmap e281: 8 total, 8 up, 8 in 2024-09-11T07:39:35.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:35 smithi104 ceph-mon[33507]: 5.11 scrub starts 2024-09-11T07:39:35.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:35 smithi104 ceph-mon[33507]: 5.11 scrub ok 2024-09-11T07:39:35.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:35 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2358605498' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:39:35.820 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:35.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:35.854 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:35.854 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:35.955 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:35.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:35.999 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:35.999 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:36.100 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:36.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:36.144 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:36.146 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:36.247 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:36.282 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:36.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:36.283 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:36.384 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:36.421 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:36.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:36.422 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:36.523 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:36.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:36.562 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:36.562 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:36.663 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:36.701 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:36.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:36.702 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:36.803 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:36.840 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:36.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:36.841 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:36.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:36 smithi022 ceph-mon[29462]: 2.c scrub starts 2024-09-11T07:39:36.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:36 smithi022 ceph-mon[29462]: 6.3 scrub starts 2024-09-11T07:39:36.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:36 smithi022 ceph-mon[29462]: 6.3 scrub ok 2024-09-11T07:39:36.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:36 smithi022 ceph-mon[29462]: pgmap v586: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 6.0 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.02k op/s; 19384/293544 objects misplaced (6.603%); 11 MiB/s, 182 objects/s recovering 2024-09-11T07:39:36.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:36 smithi022 ceph-mon[37330]: 2.c scrub starts 2024-09-11T07:39:36.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:36 smithi022 ceph-mon[37330]: 6.3 scrub starts 2024-09-11T07:39:36.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:36 smithi022 ceph-mon[37330]: 6.3 scrub ok 2024-09-11T07:39:36.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:36 smithi022 ceph-mon[37330]: pgmap v586: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 6.0 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.02k op/s; 19384/293544 objects misplaced (6.603%); 11 MiB/s, 182 objects/s recovering 2024-09-11T07:39:36.942 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:36.984 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:36.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:36.986 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:37.015 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:36 smithi104 ceph-mon[33507]: 2.c scrub starts 2024-09-11T07:39:37.016 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:36 smithi104 ceph-mon[33507]: 6.3 scrub starts 2024-09-11T07:39:37.016 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:36 smithi104 ceph-mon[33507]: 6.3 scrub ok 2024-09-11T07:39:37.016 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:36 smithi104 ceph-mon[33507]: pgmap v586: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 6.0 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.02k op/s; 19384/293544 objects misplaced (6.603%); 11 MiB/s, 182 objects/s recovering 2024-09-11T07:39:37.087 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:37.116 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:37.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:37.117 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:37.218 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:37.253 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:37.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:37.255 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:37.356 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:37.398 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:37.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:37.399 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:37.500 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:37.536 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:37.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:37.537 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:37.638 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:37.649 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:37 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:39:37.660 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:37.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:37.661 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:37.762 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:37.802 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:37.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:37.803 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:37.904 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:37.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:37 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:39:37.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:37 smithi022 ceph-mon[29462]: 2.8 scrub starts 2024-09-11T07:39:37.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:37 smithi022 ceph-mon[29462]: 5.1 scrub starts 2024-09-11T07:39:37.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:37 smithi022 ceph-mon[29462]: 5.1 scrub ok 2024-09-11T07:39:37.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:37 smithi022 ceph-mon[29462]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:39:37.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:37 smithi022 ceph-mon[29462]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1 pg undersized) 2024-09-11T07:39:37.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:37 smithi022 ceph-mon[29462]: osdmap e282: 8 total, 8 up, 8 in 2024-09-11T07:39:37.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:37 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:39:37.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:37 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:39:37.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:37 smithi022 ceph-mon[37330]: 2.8 scrub starts 2024-09-11T07:39:37.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:37 smithi022 ceph-mon[37330]: 5.1 scrub starts 2024-09-11T07:39:37.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:37 smithi022 ceph-mon[37330]: 5.1 scrub ok 2024-09-11T07:39:37.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:37 smithi022 ceph-mon[37330]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:39:37.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:37 smithi022 ceph-mon[37330]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1 pg undersized) 2024-09-11T07:39:37.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:37 smithi022 ceph-mon[37330]: osdmap e282: 8 total, 8 up, 8 in 2024-09-11T07:39:37.939 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:37.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:37.940 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:38.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:37 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:39:38.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:37 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:39:38.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:37 smithi104 ceph-mon[33507]: 2.8 scrub starts 2024-09-11T07:39:38.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:37 smithi104 ceph-mon[33507]: 5.1 scrub starts 2024-09-11T07:39:38.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:37 smithi104 ceph-mon[33507]: 5.1 scrub ok 2024-09-11T07:39:38.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:37 smithi104 ceph-mon[33507]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:39:38.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:37 smithi104 ceph-mon[33507]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1 pg undersized) 2024-09-11T07:39:38.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:37 smithi104 ceph-mon[33507]: osdmap e282: 8 total, 8 up, 8 in 2024-09-11T07:39:38.041 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:38.077 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:38.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:38.078 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:38.178 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:38.200 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:38.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:38.202 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:38.302 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:38.341 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:38.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:38.343 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:38.443 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:38.474 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:38.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:38.475 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:38.576 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:38.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:38.606 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:38.606 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:38.707 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:38.741 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:38.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:38.742 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:38.843 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:38.864 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:38.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:38.865 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:38.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:38 smithi022 ceph-mon[29462]: 1.0 scrub starts 2024-09-11T07:39:38.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:39:38.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:38 smithi022 ceph-mon[29462]: pgmap v588: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 6.0 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 869 B/s rd, 0 B/s wr, 939 op/s; 19384/293544 objects misplaced (6.603%); 12 MiB/s, 186 objects/s recovering 2024-09-11T07:39:38.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:38 smithi022 ceph-mon[37330]: 1.0 scrub starts 2024-09-11T07:39:38.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:39:38.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:38 smithi022 ceph-mon[37330]: pgmap v588: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 6.0 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 869 B/s rd, 0 B/s wr, 939 op/s; 19384/293544 objects misplaced (6.603%); 12 MiB/s, 186 objects/s recovering 2024-09-11T07:39:38.966 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:39.007 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:39.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:39.008 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:39.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:38 smithi104 ceph-mon[33507]: 1.0 scrub starts 2024-09-11T07:39:39.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:39:39.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:38 smithi104 ceph-mon[33507]: pgmap v588: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 6.0 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 869 B/s rd, 0 B/s wr, 939 op/s; 19384/293544 objects misplaced (6.603%); 12 MiB/s, 186 objects/s recovering 2024-09-11T07:39:39.109 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:39.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:39.146 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:39.146 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:39.247 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:39.285 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:39.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:39.286 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:39.387 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:39.424 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:39.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:39.425 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:39.526 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:39.560 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:39.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:39.561 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:39.662 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:39.663 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:39:39.663 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:39:39.663 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-11T07:39:39.663 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd test-reweight-by-utilization 1000 3.0 2024-09-11T07:39:39.692 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:39.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:39.693 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:39.794 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:39.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:39.845 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:39.845 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:39.946 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:39.995 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:39.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:39.996 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:40.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:39 smithi104 ceph-mon[33507]: 2.1 scrub starts 2024-09-11T07:39:40.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:39 smithi104 ceph-mon[33507]: 4.1a scrub starts 2024-09-11T07:39:40.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:39:39 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:39:40.097 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:40.103 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:39 smithi022 ceph-mon[29462]: 2.1 scrub starts 2024-09-11T07:39:40.103 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:39 smithi022 ceph-mon[29462]: 4.1a scrub starts 2024-09-11T07:39:40.104 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:39 smithi022 ceph-mon[37330]: 2.1 scrub starts 2024-09-11T07:39:40.104 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:39 smithi022 ceph-mon[37330]: 4.1a scrub starts 2024-09-11T07:39:40.125 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:40.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:40.126 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:40.166 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:39:40.166 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:39:40.166 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.166 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:162691 9.2 9:43695054:::benchmark_data_smithi022_133943_object36100:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e282)", 2024-09-11T07:39:40.166 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:39:40.132403+0000", 2024-09-11T07:39:40.166 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.028941178000000001, 2024-09-11T07:39:40.167 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.028973972000000001, 2024-09-11T07:39:40.167 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:39:40.167 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "waiting for sub ops", 2024-09-11T07:39:40.167 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:39:40.167 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:39:40.167 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:39:40.167 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 162691 2024-09-11T07:39:40.167 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.167 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:39:40.167 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.167 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:39:40.167 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.132403+0000", 2024-09-11T07:39:40.167 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:39:40.168 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.168 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.168 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:39:40.168 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.132403+0000", 2024-09-11T07:39:40.168 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:39:40.168 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.168 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.168 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:39:40.168 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.132401+0000", 2024-09-11T07:39:40.168 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:39:40.168 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.168 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.169 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:39:40.169 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.132408+0000", 2024-09-11T07:39:40.169 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9360000000000002e-06 2024-09-11T07:39:40.169 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.169 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.169 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:39:40.169 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.132419+0000", 2024-09-11T07:39:40.169 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0351e-05 2024-09-11T07:39:40.169 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.169 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.169 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:39:40.169 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.132424+0000", 2024-09-11T07:39:40.169 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.587e-06 2024-09-11T07:39:40.170 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.170 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.170 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:39:40.170 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.140601+0000", 2024-09-11T07:39:40.170 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.008176638 2024-09-11T07:39:40.170 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.170 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.170 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:39:40.170 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.140733+0000", 2024-09-11T07:39:40.170 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000131922 2024-09-11T07:39:40.170 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.170 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.170 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:39:40.170 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.141138+0000", 2024-09-11T07:39:40.171 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00040547599999999999 2024-09-11T07:39:40.171 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.171 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.171 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:39:40.171 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.148949+0000", 2024-09-11T07:39:40.171 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0078109579999999998 2024-09-11T07:39:40.171 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.171 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.171 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:39:40.171 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.150036+0000", 2024-09-11T07:39:40.171 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001087253 2024-09-11T07:39:40.171 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:39:40.171 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:39:40.171 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:39:40.172 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.172 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.172 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.15504.0:162713 9.3 e282/259 9:c6789e98:::benchmark_data_smithi022_133943_object36122:head v 282'7646, mlcod=282'7644)", 2024-09-11T07:39:40.172 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:39:40.155286+0000", 2024-09-11T07:39:40.172 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0060580979999999996, 2024-09-11T07:39:40.172 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0061574680000000001, 2024-09-11T07:39:40.172 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:39:40.172 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:39:40.172 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:39:40.172 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.172 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:39:40.172 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.155286+0000", 2024-09-11T07:39:40.172 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:39:40.172 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.173 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.173 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:39:40.173 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.155286+0000", 2024-09-11T07:39:40.173 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:39:40.173 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.173 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.173 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:39:40.173 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.155285+0000", 2024-09-11T07:39:40.173 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:39:40.173 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.173 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.173 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:39:40.173 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.155293+0000", 2024-09-11T07:39:40.173 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5290000000000003e-06 2024-09-11T07:39:40.174 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.174 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.174 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:39:40.174 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.155302+0000", 2024-09-11T07:39:40.174 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.1300000000000007e-06 2024-09-11T07:39:40.174 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.174 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.174 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:39:40.174 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.155307+0000", 2024-09-11T07:39:40.174 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3340000000000001e-06 2024-09-11T07:39:40.174 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.174 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.174 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:39:40.174 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.156227+0000", 2024-09-11T07:39:40.175 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00092005099999999996 2024-09-11T07:39:40.175 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.175 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.175 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:39:40.175 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.156245+0000", 2024-09-11T07:39:40.175 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.7906000000000001e-05 2024-09-11T07:39:40.175 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:39:40.175 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:39:40.175 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:39:40.175 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.175 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.175 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:162680 9.2 9:429f0e15:::benchmark_data_smithi022_133943_object36089:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e282)", 2024-09-11T07:39:40.175 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:39:40.123732+0000", 2024-09-11T07:39:40.175 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.037612079, 2024-09-11T07:39:40.176 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.037746296999999998, 2024-09-11T07:39:40.176 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:39:40.176 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "waiting for sub ops", 2024-09-11T07:39:40.176 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:39:40.176 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:39:40.176 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:39:40.176 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 162680 2024-09-11T07:39:40.176 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.176 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:39:40.176 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.176 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:39:40.176 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.123732+0000", 2024-09-11T07:39:40.176 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:39:40.176 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.177 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.177 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:39:40.177 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.123732+0000", 2024-09-11T07:39:40.177 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:39:40.177 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.177 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.177 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:39:40.177 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.123730+0000", 2024-09-11T07:39:40.177 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:39:40.177 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.177 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.177 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:39:40.177 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.123738+0000", 2024-09-11T07:39:40.177 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.4360000000000002e-06 2024-09-11T07:39:40.177 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.178 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.178 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:39:40.178 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.123751+0000", 2024-09-11T07:39:40.178 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2347999999999999e-05 2024-09-11T07:39:40.178 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.178 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.178 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:39:40.178 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.123757+0000", 2024-09-11T07:39:40.178 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3169999999999998e-06 2024-09-11T07:39:40.178 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.178 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.178 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:39:40.179 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.135448+0000", 2024-09-11T07:39:40.179 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.011690531000000001 2024-09-11T07:39:40.179 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.179 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.179 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:39:40.179 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.135567+0000", 2024-09-11T07:39:40.179 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000119263 2024-09-11T07:39:40.179 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.179 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.179 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:39:40.179 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.135981+0000", 2024-09-11T07:39:40.179 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00041448899999999998 2024-09-11T07:39:40.179 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.179 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.180 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:39:40.180 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.148885+0000", 2024-09-11T07:39:40.180 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.012903282 2024-09-11T07:39:40.180 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.180 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.180 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:39:40.180 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.148918+0000", 2024-09-11T07:39:40.180 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3475000000000001e-05 2024-09-11T07:39:40.180 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:39:40.180 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:39:40.180 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:39:40.180 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.181 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.181 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:162719 9.c 9:3f995b26:::benchmark_data_smithi022_133943_object36128:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e282)", 2024-09-11T07:39:40.181 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:39:40.159368+0000", 2024-09-11T07:39:40.181 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0019759909999999999, 2024-09-11T07:39:40.181 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0021591679999999999, 2024-09-11T07:39:40.181 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:39:40.181 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "waiting for sub ops", 2024-09-11T07:39:40.181 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:39:40.181 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:39:40.181 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:39:40.181 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 162719 2024-09-11T07:39:40.181 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.181 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:39:40.181 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.182 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:39:40.182 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.159368+0000", 2024-09-11T07:39:40.182 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:39:40.182 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.182 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.182 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:39:40.182 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.159368+0000", 2024-09-11T07:39:40.182 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:39:40.182 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.182 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.182 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:39:40.182 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.159366+0000", 2024-09-11T07:39:40.182 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:39:40.182 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.183 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.183 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:39:40.183 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.159374+0000", 2024-09-11T07:39:40.183 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5490000000000001e-06 2024-09-11T07:39:40.183 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.183 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.183 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:39:40.183 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.159389+0000", 2024-09-11T07:39:40.183 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4944e-05 2024-09-11T07:39:40.183 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.183 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.183 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:39:40.183 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.159395+0000", 2024-09-11T07:39:40.184 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0430000000000004e-06 2024-09-11T07:39:40.184 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.184 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.184 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:39:40.184 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.159462+0000", 2024-09-11T07:39:40.184 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7113000000000006e-05 2024-09-11T07:39:40.184 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.184 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.184 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:39:40.184 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.159578+0000", 2024-09-11T07:39:40.184 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000116319 2024-09-11T07:39:40.184 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:39:40.184 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:39:40.184 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 4,7", 2024-09-11T07:39:40.185 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:40.159805+0000", 2024-09-11T07:39:40.185 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00022719099999999999 2024-09-11T07:39:40.185 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:39:40.185 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:39:40.185 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:39:40.185 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:39:40.185 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:39:40.185 INFO:teuthology.orchestra.run.smithi022.stdout: "num_ops": 4 2024-09-11T07:39:40.185 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:39:40.227 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:40.263 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:40.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:40.265 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:40.365 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:40.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:40.397 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:40.398 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:40.498 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:40.538 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:40.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:40.539 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:40.640 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:40.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:39:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:39:40] "GET /metrics HTTP/1.1" 200 224566 "" "Prometheus/2.43.0" 2024-09-11T07:39:40.676 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:40.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:40.677 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:40.778 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:40.825 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:40.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:40.826 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:40.839 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:39:40.840 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:39:40.872 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-11T07:39:40.926 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:41.007 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:41.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:41.008 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:41.109 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:41.135 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:40 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:39:41.135 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:40 smithi022 ceph-mon[29462]: 5.13 scrub starts 2024-09-11T07:39:41.136 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:40 smithi022 ceph-mon[29462]: pgmap v589: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.62k op/s; 17161/276654 objects misplaced (6.203%); 14 MiB/s, 229 objects/s recovering 2024-09-11T07:39:41.136 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:40 smithi022 ceph-mon[29462]: 6.1a scrub starts 2024-09-11T07:39:41.143 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:41.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:41.144 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:41.144 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:40 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:39:41.144 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:40 smithi022 ceph-mon[37330]: 5.13 scrub starts 2024-09-11T07:39:41.144 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:40 smithi022 ceph-mon[37330]: pgmap v589: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.62k op/s; 17161/276654 objects misplaced (6.203%); 14 MiB/s, 229 objects/s recovering 2024-09-11T07:39:41.144 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:40 smithi022 ceph-mon[37330]: 6.1a scrub starts 2024-09-11T07:39:41.245 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:41.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:40 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:39:41.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:40 smithi104 ceph-mon[33507]: 5.13 scrub starts 2024-09-11T07:39:41.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:40 smithi104 ceph-mon[33507]: pgmap v589: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.62k op/s; 17161/276654 objects misplaced (6.203%); 14 MiB/s, 229 objects/s recovering 2024-09-11T07:39:41.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:40 smithi104 ceph-mon[33507]: 6.1a scrub starts 2024-09-11T07:39:41.306 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:41.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:41.307 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:41.408 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:41.443 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:41.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:41.444 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:41.545 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:41.571 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:39:41.578 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:41.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:41.580 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:41.681 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:41.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:41.730 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:41.730 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:41.831 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:41.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:41.892 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:41.892 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:41.993 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:42.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:42.026 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:42.026 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:42.127 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:42.163 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:42.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:42.165 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:42.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:41 smithi022 ceph-mon[29462]: 6.f scrub starts 2024-09-11T07:39:42.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:41 smithi022 ceph-mon[29462]: 6.f scrub ok 2024-09-11T07:39:42.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:41 smithi022 ceph-mon[29462]: 4.d scrub starts 2024-09-11T07:39:42.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:41 smithi022 ceph-mon[37330]: 6.f scrub starts 2024-09-11T07:39:42.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:41 smithi022 ceph-mon[37330]: 6.f scrub ok 2024-09-11T07:39:42.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:41 smithi022 ceph-mon[37330]: 4.d scrub starts 2024-09-11T07:39:42.265 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:42.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:41 smithi104 ceph-mon[33507]: 6.f scrub starts 2024-09-11T07:39:42.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:41 smithi104 ceph-mon[33507]: 6.f scrub ok 2024-09-11T07:39:42.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:41 smithi104 ceph-mon[33507]: 4.d scrub starts 2024-09-11T07:39:42.299 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:42.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:42.300 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:42.400 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:42.453 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:42.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:42.454 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:42.555 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:42.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:42.591 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:42.591 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:42.692 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:42.727 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:42.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:42.728 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:42.828 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:42.864 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:42.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:42.865 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:42.966 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:43.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:43.001 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:43.001 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:43.102 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:43.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:43.137 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:43.137 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:43.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:42 smithi022 ceph-mon[37330]: 6.8 scrub starts 2024-09-11T07:39:43.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:42 smithi022 ceph-mon[37330]: 6.8 scrub ok 2024-09-11T07:39:43.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:42 smithi022 ceph-mon[37330]: pgmap v590: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s; 17161/274251 objects misplaced (6.257%); 12 MiB/s, 197 objects/s recovering 2024-09-11T07:39:43.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:42 smithi022 ceph-mon[37330]: 4.1c scrub starts 2024-09-11T07:39:43.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:42 smithi022 ceph-mon[37330]: 4.13 scrub starts 2024-09-11T07:39:43.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:42 smithi022 ceph-mon[29462]: 6.8 scrub starts 2024-09-11T07:39:43.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:42 smithi022 ceph-mon[29462]: 6.8 scrub ok 2024-09-11T07:39:43.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:42 smithi022 ceph-mon[29462]: pgmap v590: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s; 17161/274251 objects misplaced (6.257%); 12 MiB/s, 197 objects/s recovering 2024-09-11T07:39:43.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:42 smithi022 ceph-mon[29462]: 4.1c scrub starts 2024-09-11T07:39:43.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:42 smithi022 ceph-mon[29462]: 4.13 scrub starts 2024-09-11T07:39:43.238 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:43.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:42 smithi104 ceph-mon[33507]: 6.8 scrub starts 2024-09-11T07:39:43.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:42 smithi104 ceph-mon[33507]: 6.8 scrub ok 2024-09-11T07:39:43.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:42 smithi104 ceph-mon[33507]: pgmap v590: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s; 17161/274251 objects misplaced (6.257%); 12 MiB/s, 197 objects/s recovering 2024-09-11T07:39:43.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:42 smithi104 ceph-mon[33507]: 4.1c scrub starts 2024-09-11T07:39:43.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:42 smithi104 ceph-mon[33507]: 4.13 scrub starts 2024-09-11T07:39:43.280 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:43.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:43.281 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:43.382 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:43.418 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:43.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:43.419 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:43.520 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:43.546 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:43.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:43.547 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:43.648 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:43.687 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:43.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:43.688 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:43.789 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:43.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:43.823 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:43.824 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:43.924 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:43.965 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:43.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:43.966 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:44.066 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:44.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:44.095 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:44.095 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:44.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:43 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:39:44.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:43 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:39:44.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:43 smithi022 ceph-mon[29462]: osdmap e283: 8 total, 8 up, 8 in 2024-09-11T07:39:44.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:43 smithi022 ceph-mon[29462]: 5.b scrub starts 2024-09-11T07:39:44.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:43 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:39:44.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:43 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:39:44.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:43 smithi022 ceph-mon[37330]: osdmap e283: 8 total, 8 up, 8 in 2024-09-11T07:39:44.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:43 smithi022 ceph-mon[37330]: 5.b scrub starts 2024-09-11T07:39:44.196 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:44.235 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:44.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:44.236 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:44.279 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:39:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:39:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:39:44.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:43 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:39:44.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:43 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:39:44.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:43 smithi104 ceph-mon[33507]: osdmap e283: 8 total, 8 up, 8 in 2024-09-11T07:39:44.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:43 smithi104 ceph-mon[33507]: 5.b scrub starts 2024-09-11T07:39:44.337 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:44.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:44.374 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:44.374 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:44.475 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:44.512 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:44.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:44.513 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:44.613 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:44.650 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:44.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:44.651 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:44.752 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:44.821 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:44.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:44.822 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:44.923 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:44.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:44.963 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:44.963 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:45.064 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:45.092 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:45.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:45.093 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:45.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:44 smithi022 ceph-mon[29462]: 6.12 scrub starts 2024-09-11T07:39:45.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:44 smithi022 ceph-mon[29462]: 6.12 scrub ok 2024-09-11T07:39:45.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:44 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:39:45.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:44 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:39:45.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:44 smithi022 ceph-mon[29462]: osdmap e284: 8 total, 8 up, 8 in 2024-09-11T07:39:45.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:44 smithi022 ceph-mon[29462]: pgmap v593: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 5.5 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 675 B/s rd, 0 B/s wr, 1.04k op/s; 17161/269826 objects misplaced (6.360%); 4.0 MiB/s, 63 objects/s recovering 2024-09-11T07:39:45.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:44 smithi022 ceph-mon[29462]: 5.d scrub starts 2024-09-11T07:39:45.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:44 smithi022 ceph-mon[37330]: 6.12 scrub starts 2024-09-11T07:39:45.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:44 smithi022 ceph-mon[37330]: 6.12 scrub ok 2024-09-11T07:39:45.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:44 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:39:45.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:44 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:39:45.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:44 smithi022 ceph-mon[37330]: osdmap e284: 8 total, 8 up, 8 in 2024-09-11T07:39:45.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:44 smithi022 ceph-mon[37330]: pgmap v593: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 5.5 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 675 B/s rd, 0 B/s wr, 1.04k op/s; 17161/269826 objects misplaced (6.360%); 4.0 MiB/s, 63 objects/s recovering 2024-09-11T07:39:45.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:44 smithi022 ceph-mon[37330]: 5.d scrub starts 2024-09-11T07:39:45.193 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:45.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:45.229 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:45.229 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:45.249 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:39:45.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:44 smithi104 ceph-mon[33507]: 6.12 scrub starts 2024-09-11T07:39:45.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:44 smithi104 ceph-mon[33507]: 6.12 scrub ok 2024-09-11T07:39:45.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:44 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:39:45.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:44 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:39:45.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:44 smithi104 ceph-mon[33507]: osdmap e284: 8 total, 8 up, 8 in 2024-09-11T07:39:45.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:44 smithi104 ceph-mon[33507]: pgmap v593: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 5.5 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 675 B/s rd, 0 B/s wr, 1.04k op/s; 17161/269826 objects misplaced (6.360%); 4.0 MiB/s, 63 objects/s recovering 2024-09-11T07:39:45.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:44 smithi104 ceph-mon[33507]: 5.d scrub starts 2024-09-11T07:39:45.330 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:45.363 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:45.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:45.364 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:45.465 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:45.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:45.511 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:45.511 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:45.612 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:45.649 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:45.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:45.650 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:45.752 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:45.786 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:45.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:45.787 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:45.888 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:45.919 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:45.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:45.921 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:46.021 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:46.051 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:46.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:46.053 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:46.153 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:46.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:45 smithi022 ceph-mon[29462]: 2.b scrub starts 2024-09-11T07:39:46.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:45 smithi022 ceph-mon[29462]: 5.6 scrub starts 2024-09-11T07:39:46.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:45 smithi022 ceph-mon[37330]: 2.b scrub starts 2024-09-11T07:39:46.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:45 smithi022 ceph-mon[37330]: 5.6 scrub starts 2024-09-11T07:39:46.182 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:46.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:46.183 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:46.191 INFO:teuthology.orchestra.run.smithi022.stdout:moved 0 / 474 (0%) 2024-09-11T07:39:46.191 INFO:teuthology.orchestra.run.smithi022.stdout:avg 59.25 2024-09-11T07:39:46.192 INFO:teuthology.orchestra.run.smithi022.stdout:stddev 9.07951 -> 9.07951 (expected baseline 7.20026) 2024-09-11T07:39:46.192 INFO:teuthology.orchestra.run.smithi022.stdout:min osd.2 with 42 -> 42 pgs (0.708861 -> 0.708861 * mean) 2024-09-11T07:39:46.192 INFO:teuthology.orchestra.run.smithi022.stdout:max osd.7 with 70 -> 70 pgs (1.18143 -> 1.18143 * mean) 2024-09-11T07:39:46.192 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:39:46.192 INFO:teuthology.orchestra.run.smithi022.stdout:oload 1000 2024-09-11T07:39:46.192 INFO:teuthology.orchestra.run.smithi022.stdout:max_change 3 2024-09-11T07:39:46.192 INFO:teuthology.orchestra.run.smithi022.stdout:max_change_osds 4 2024-09-11T07:39:46.192 INFO:teuthology.orchestra.run.smithi022.stdout:average_utilization 0.0289 2024-09-11T07:39:46.192 INFO:teuthology.orchestra.run.smithi022.stdout:overload_utilization 0.2891 2024-09-11T07:39:46.192 INFO:teuthology.orchestra.run.smithi022.stderr:no change 2024-09-11T07:39:46.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:45 smithi104 ceph-mon[33507]: 2.b scrub starts 2024-09-11T07:39:46.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:45 smithi104 ceph-mon[33507]: 5.6 scrub starts 2024-09-11T07:39:46.285 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:46.326 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:46.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:46.327 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:46.428 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:46.470 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:46.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:46.472 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:46.573 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:46.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:46.611 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:46.611 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:46.636 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:39:46.712 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:46.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:46.755 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:46.756 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:46.856 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:46.899 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:46.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:46.900 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:47.001 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:47.041 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:47.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:47.042 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:47.143 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:47.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:46 smithi022 ceph-mon[29462]: from='client.25507 v1:172.21.15.22:0/986929163' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:39:47.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:46 smithi022 ceph-mon[29462]: pgmap v594: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 5.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.64k op/s; 14934/254277 objects misplaced (5.873%); 7.7 MiB/s, 123 objects/s recovering 2024-09-11T07:39:47.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:46 smithi022 ceph-mon[29462]: 2.e scrub starts 2024-09-11T07:39:47.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:46 smithi022 ceph-mon[29462]: 2.e scrub ok 2024-09-11T07:39:47.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:46 smithi022 ceph-mon[29462]: 2.11 scrub starts 2024-09-11T07:39:47.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:46 smithi022 ceph-mon[37330]: from='client.25507 v1:172.21.15.22:0/986929163' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:39:47.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:46 smithi022 ceph-mon[37330]: pgmap v594: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 5.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.64k op/s; 14934/254277 objects misplaced (5.873%); 7.7 MiB/s, 123 objects/s recovering 2024-09-11T07:39:47.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:46 smithi022 ceph-mon[37330]: 2.e scrub starts 2024-09-11T07:39:47.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:46 smithi022 ceph-mon[37330]: 2.e scrub ok 2024-09-11T07:39:47.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:46 smithi022 ceph-mon[37330]: 2.11 scrub starts 2024-09-11T07:39:47.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:47.182 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:47.182 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:47.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:46 smithi104 ceph-mon[33507]: from='client.25507 v1:172.21.15.22:0/986929163' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:39:47.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:46 smithi104 ceph-mon[33507]: pgmap v594: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 5.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.64k op/s; 14934/254277 objects misplaced (5.873%); 7.7 MiB/s, 123 objects/s recovering 2024-09-11T07:39:47.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:46 smithi104 ceph-mon[33507]: 2.e scrub starts 2024-09-11T07:39:47.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:46 smithi104 ceph-mon[33507]: 2.e scrub ok 2024-09-11T07:39:47.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:46 smithi104 ceph-mon[33507]: 2.11 scrub starts 2024-09-11T07:39:47.284 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:47.336 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:47.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:47.338 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:47.439 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:47.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:47.479 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:47.479 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:47.580 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:47.618 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:47.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:47.619 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:47.719 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:47.765 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:47.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:47.766 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-11T07:39:47.766 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight-by-utilization 110 3.0 2024-09-11T07:39:47.767 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:47.867 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:47.926 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:47.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:47.927 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:48.004 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:47 smithi022 ceph-mon[37330]: 2.1e scrub starts 2024-09-11T07:39:48.004 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:47 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:39:48.005 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:47 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:39:48.005 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:47 smithi022 ceph-mon[29462]: 2.1e scrub starts 2024-09-11T07:39:48.005 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:47 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:39:48.005 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:47 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:39:48.028 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:48.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:48.089 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:48.090 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:48.190 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:48.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:48.232 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:48.233 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:48.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:47 smithi104 ceph-mon[33507]: 2.1e scrub starts 2024-09-11T07:39:48.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:47 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:39:48.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:47 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:39:48.333 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:48.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:48.381 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:48.381 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:48.482 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:48.513 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:48.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:48.514 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:48.615 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:48.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:48.653 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:48.653 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:48.754 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:48.783 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:48.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:48.784 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:48.885 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:48.918 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:48.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:48.920 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:49.020 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:49.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:49.059 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:49.060 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:49.159 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:39:49.161 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:49.166 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:48 smithi022 ceph-mon[37330]: osdmap e285: 8 total, 8 up, 8 in 2024-09-11T07:39:49.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:48 smithi022 ceph-mon[37330]: pgmap v596: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 5.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.11k op/s; 14934/254277 objects misplaced (5.873%); 5.2 MiB/s, 83 objects/s recovering 2024-09-11T07:39:49.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:48 smithi022 ceph-mon[37330]: 2.19 scrub starts 2024-09-11T07:39:49.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:48 smithi022 ceph-mon[29462]: osdmap e285: 8 total, 8 up, 8 in 2024-09-11T07:39:49.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:48 smithi022 ceph-mon[29462]: pgmap v596: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 5.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.11k op/s; 14934/254277 objects misplaced (5.873%); 5.2 MiB/s, 83 objects/s recovering 2024-09-11T07:39:49.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:48 smithi022 ceph-mon[29462]: 2.19 scrub starts 2024-09-11T07:39:49.191 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:49.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:49.192 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:49.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:48 smithi104 ceph-mon[33507]: osdmap e285: 8 total, 8 up, 8 in 2024-09-11T07:39:49.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:48 smithi104 ceph-mon[33507]: pgmap v596: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 5.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.11k op/s; 14934/254277 objects misplaced (5.873%); 5.2 MiB/s, 83 objects/s recovering 2024-09-11T07:39:49.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:48 smithi104 ceph-mon[33507]: 2.19 scrub starts 2024-09-11T07:39:49.293 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:49.333 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:49.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:49.334 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:49.436 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:49.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:49.476 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:49.477 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:49.577 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:49.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:49.620 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:49.620 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:49.721 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:49.762 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:49.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:49.763 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:49.819 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:39:49.819 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:39:49.820 INFO:teuthology.orchestra.run.smithi022.stdout: "complaint_time": 30, 2024-09-11T07:39:49.820 INFO:teuthology.orchestra.run.smithi022.stdout: "num_blocked_ops": 0 2024-09-11T07:39:49.820 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:39:49.864 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:49.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:49.910 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:49.910 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:49.972 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:39:49 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:39:50.011 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:50.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:50.033 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:50.033 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:50.134 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:50.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:49 smithi022 ceph-mon[29462]: 2.7 scrub starts 2024-09-11T07:39:50.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:49 smithi022 ceph-mon[29462]: 2.7 scrub ok 2024-09-11T07:39:50.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:49 smithi022 ceph-mon[37330]: 2.7 scrub starts 2024-09-11T07:39:50.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:49 smithi022 ceph-mon[37330]: 2.7 scrub ok 2024-09-11T07:39:50.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:50.168 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:50.168 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:50.269 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:50.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:49 smithi104 ceph-mon[33507]: 2.7 scrub starts 2024-09-11T07:39:50.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:49 smithi104 ceph-mon[33507]: 2.7 scrub ok 2024-09-11T07:39:50.305 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:50.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:50.307 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:50.407 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:50.443 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:50.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:50.444 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:50.545 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:50.578 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:50.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:50.579 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:50.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:39:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:39:50] "GET /metrics HTTP/1.1" 200 224566 "" "Prometheus/2.43.0" 2024-09-11T07:39:50.679 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:50.711 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:50.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:50.712 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:50.813 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:50.849 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:50.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:50.850 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:50.951 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:50.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:50.988 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:50.988 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:51.089 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:51.133 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:51.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:51.134 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:51.235 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:51.250 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:50 smithi022 ceph-mon[37330]: 6.5 scrub starts 2024-09-11T07:39:51.250 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:50 smithi022 ceph-mon[37330]: 6.5 scrub ok 2024-09-11T07:39:51.250 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:50 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:39:51.250 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:50 smithi022 ceph-mon[37330]: pgmap v597: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 4.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.71k op/s; 12751/236742 objects misplaced (5.386%); 8.6 MiB/s, 138 objects/s recovering 2024-09-11T07:39:51.250 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:50 smithi022 ceph-mon[37330]: 6.17 scrub starts 2024-09-11T07:39:51.250 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:50 smithi022 ceph-mon[37330]: 4.19 scrub starts 2024-09-11T07:39:51.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:50 smithi022 ceph-mon[29462]: 6.5 scrub starts 2024-09-11T07:39:51.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:50 smithi022 ceph-mon[29462]: 6.5 scrub ok 2024-09-11T07:39:51.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:50 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:39:51.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:50 smithi022 ceph-mon[29462]: pgmap v597: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 4.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.71k op/s; 12751/236742 objects misplaced (5.386%); 8.6 MiB/s, 138 objects/s recovering 2024-09-11T07:39:51.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:50 smithi022 ceph-mon[29462]: 6.17 scrub starts 2024-09-11T07:39:51.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:50 smithi022 ceph-mon[29462]: 4.19 scrub starts 2024-09-11T07:39:51.268 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:51.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:51.269 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:51.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:50 smithi104 ceph-mon[33507]: 6.5 scrub starts 2024-09-11T07:39:51.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:50 smithi104 ceph-mon[33507]: 6.5 scrub ok 2024-09-11T07:39:51.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:50 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:39:51.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:50 smithi104 ceph-mon[33507]: pgmap v597: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 4.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.71k op/s; 12751/236742 objects misplaced (5.386%); 8.6 MiB/s, 138 objects/s recovering 2024-09-11T07:39:51.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:50 smithi104 ceph-mon[33507]: 6.17 scrub starts 2024-09-11T07:39:51.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:50 smithi104 ceph-mon[33507]: 4.19 scrub starts 2024-09-11T07:39:51.369 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:51.410 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:51.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:51.411 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:51.511 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:51.548 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:51.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:51.549 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:51.650 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:51.695 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:51.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:51.696 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:51.797 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:51.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:51.852 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:51.852 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:51.952 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:51.973 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:51.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:51.975 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:52.075 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:52.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:52.108 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:52.108 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:52.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:51 smithi022 ceph-mon[29462]: 6.2 scrub starts 2024-09-11T07:39:52.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:51 smithi022 ceph-mon[29462]: 6.2 scrub ok 2024-09-11T07:39:52.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:51 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1184565116' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:39:52.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:51 smithi022 ceph-mon[37330]: 6.2 scrub starts 2024-09-11T07:39:52.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:51 smithi022 ceph-mon[37330]: 6.2 scrub ok 2024-09-11T07:39:52.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:51 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1184565116' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:39:52.210 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:52.239 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:52.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:52.240 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:52.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:51 smithi104 ceph-mon[33507]: 6.2 scrub starts 2024-09-11T07:39:52.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:51 smithi104 ceph-mon[33507]: 6.2 scrub ok 2024-09-11T07:39:52.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:51 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1184565116' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:39:52.341 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:52.363 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:52.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:52.364 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:52.465 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:52.489 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:52.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:52.490 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:52.591 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:52.613 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:52.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:52.614 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:52.715 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:39:52.736 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:52.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:52.737 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:39:52.838 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:52.861 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:52.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:52.862 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:52.893 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-11T07:39:52.961 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:39:52.962 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:39:52.962 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:39:52.962 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:39:52.962 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:39:52.962 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:39:52.962 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:39:52.962 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:39:52.963 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:39:52.963 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:39:52.963 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:39:52.963 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:39:52.963 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:39:52.963 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:39:52.963 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:39:52.963 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:39:52.963 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:52.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:52.986 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:52.986 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:53.000 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is unset 2024-09-11T07:39:53.087 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:53.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:53.129 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:53.129 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:53.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:52 smithi022 ceph-mon[29462]: 2.15 scrub starts 2024-09-11T07:39:53.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:52 smithi022 ceph-mon[29462]: 2.15 scrub ok 2024-09-11T07:39:53.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:52 smithi022 ceph-mon[29462]: 6.18 scrub starts 2024-09-11T07:39:53.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:52 smithi022 ceph-mon[29462]: 6.18 scrub ok 2024-09-11T07:39:53.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:52 smithi022 ceph-mon[29462]: osdmap e286: 8 total, 8 up, 8 in 2024-09-11T07:39:53.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:52 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3056969119' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:39:53.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:52 smithi022 ceph-mon[29462]: from='client.25496 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:39:53.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:52 smithi022 ceph-mon[29462]: pgmap v599: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 4.8 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.48k op/s; 12751/234270 objects misplaced (5.443%); 7.9 MiB/s, 126 objects/s recovering 2024-09-11T07:39:53.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:52 smithi022 ceph-mon[29462]: 5.a scrub starts 2024-09-11T07:39:53.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:52 smithi022 ceph-mon[29462]: 5.a scrub ok 2024-09-11T07:39:53.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:52 smithi022 ceph-mon[29462]: 6.e scrub starts 2024-09-11T07:39:53.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:52 smithi022 ceph-mon[29462]: 6.e scrub ok 2024-09-11T07:39:53.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:52 smithi022 ceph-mon[37330]: 2.15 scrub starts 2024-09-11T07:39:53.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:52 smithi022 ceph-mon[37330]: 2.15 scrub ok 2024-09-11T07:39:53.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:52 smithi022 ceph-mon[37330]: 6.18 scrub starts 2024-09-11T07:39:53.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:52 smithi022 ceph-mon[37330]: 6.18 scrub ok 2024-09-11T07:39:53.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:52 smithi022 ceph-mon[37330]: osdmap e286: 8 total, 8 up, 8 in 2024-09-11T07:39:53.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:52 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3056969119' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:39:53.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:52 smithi022 ceph-mon[37330]: from='client.25496 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:39:53.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:52 smithi022 ceph-mon[37330]: pgmap v599: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 4.8 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.48k op/s; 12751/234270 objects misplaced (5.443%); 7.9 MiB/s, 126 objects/s recovering 2024-09-11T07:39:53.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:52 smithi022 ceph-mon[37330]: 5.a scrub starts 2024-09-11T07:39:53.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:52 smithi022 ceph-mon[37330]: 5.a scrub ok 2024-09-11T07:39:53.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:52 smithi022 ceph-mon[37330]: 6.e scrub starts 2024-09-11T07:39:53.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:52 smithi022 ceph-mon[37330]: 6.e scrub ok 2024-09-11T07:39:53.230 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:53.274 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:53.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:53.276 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:53.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:52 smithi104 ceph-mon[33507]: 2.15 scrub starts 2024-09-11T07:39:53.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:52 smithi104 ceph-mon[33507]: 2.15 scrub ok 2024-09-11T07:39:53.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:52 smithi104 ceph-mon[33507]: 6.18 scrub starts 2024-09-11T07:39:53.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:52 smithi104 ceph-mon[33507]: 6.18 scrub ok 2024-09-11T07:39:53.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:52 smithi104 ceph-mon[33507]: osdmap e286: 8 total, 8 up, 8 in 2024-09-11T07:39:53.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:52 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3056969119' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:39:53.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:52 smithi104 ceph-mon[33507]: from='client.25496 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:39:53.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:52 smithi104 ceph-mon[33507]: pgmap v599: 158 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 155 active+clean; 4.8 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.48k op/s; 12751/234270 objects misplaced (5.443%); 7.9 MiB/s, 126 objects/s recovering 2024-09-11T07:39:53.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:52 smithi104 ceph-mon[33507]: 5.a scrub starts 2024-09-11T07:39:53.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:52 smithi104 ceph-mon[33507]: 5.a scrub ok 2024-09-11T07:39:53.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:52 smithi104 ceph-mon[33507]: 6.e scrub starts 2024-09-11T07:39:53.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:52 smithi104 ceph-mon[33507]: 6.e scrub ok 2024-09-11T07:39:53.377 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:53.413 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:53.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:53.414 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:53.515 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:53.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:53.548 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:53.548 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:53.649 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:53.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:53.685 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:53.685 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:53.786 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:53.821 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:53.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:53.822 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:53.923 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:53.951 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:53.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:53.952 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:54.053 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:54.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:54.095 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:54.195 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:54.236 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:54.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:54.237 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:54.279 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:39:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:39:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:39:54.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:53 smithi104 ceph-mon[33507]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:39:54.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:53 smithi104 ceph-mon[33507]: from='client.25496 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:39:54.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:53 smithi104 ceph-mon[33507]: osdmap e287: 8 total, 8 up, 8 in 2024-09-11T07:39:54.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:39:54.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:39:54.337 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:54.375 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:54.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:54.376 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:54.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:53 smithi022 ceph-mon[29462]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:39:54.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:53 smithi022 ceph-mon[29462]: from='client.25496 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:39:54.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:53 smithi022 ceph-mon[29462]: osdmap e287: 8 total, 8 up, 8 in 2024-09-11T07:39:54.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:39:54.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:39:54.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:53 smithi022 ceph-mon[37330]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:39:54.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:53 smithi022 ceph-mon[37330]: from='client.25496 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:39:54.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:53 smithi022 ceph-mon[37330]: osdmap e287: 8 total, 8 up, 8 in 2024-09-11T07:39:54.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:39:54.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:39:54.477 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:54.545 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:54.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:54.546 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:54.647 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:54.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:54.682 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:54.683 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:54.784 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:54.819 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:54.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:54.820 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:54.921 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:54.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:54.961 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:54.962 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:55.035 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:39:55.063 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:55.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:55.099 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:55.100 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:55.193 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:55 smithi104 ceph-mon[33507]: pgmap v601: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 4.7 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 815 B/s rd, 0 B/s wr, 1.28k op/s; 12135/230142 objects misplaced (5.273%); 8.1 MiB/s, 129 objects/s recovering 2024-09-11T07:39:55.193 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:55 smithi104 ceph-mon[33507]: 2.12 deep-scrub starts 2024-09-11T07:39:55.194 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:55 smithi104 ceph-mon[33507]: 2.12 deep-scrub ok 2024-09-11T07:39:55.194 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:55 smithi104 ceph-mon[33507]: 2.1a scrub starts 2024-09-11T07:39:55.194 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:55 smithi104 ceph-mon[33507]: 2.1a scrub ok 2024-09-11T07:39:55.200 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:55.234 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:55.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:55.235 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:55.267 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:55 smithi022 ceph-mon[37330]: pgmap v601: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 4.7 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 815 B/s rd, 0 B/s wr, 1.28k op/s; 12135/230142 objects misplaced (5.273%); 8.1 MiB/s, 129 objects/s recovering 2024-09-11T07:39:55.267 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:55 smithi022 ceph-mon[37330]: 2.12 deep-scrub starts 2024-09-11T07:39:55.267 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:55 smithi022 ceph-mon[37330]: 2.12 deep-scrub ok 2024-09-11T07:39:55.267 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:55 smithi022 ceph-mon[37330]: 2.1a scrub starts 2024-09-11T07:39:55.267 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:55 smithi022 ceph-mon[37330]: 2.1a scrub ok 2024-09-11T07:39:55.268 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:55 smithi022 ceph-mon[29462]: pgmap v601: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 4.7 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 815 B/s rd, 0 B/s wr, 1.28k op/s; 12135/230142 objects misplaced (5.273%); 8.1 MiB/s, 129 objects/s recovering 2024-09-11T07:39:55.268 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:55 smithi022 ceph-mon[29462]: 2.12 deep-scrub starts 2024-09-11T07:39:55.268 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:55 smithi022 ceph-mon[29462]: 2.12 deep-scrub ok 2024-09-11T07:39:55.268 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:55 smithi022 ceph-mon[29462]: 2.1a scrub starts 2024-09-11T07:39:55.268 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:55 smithi022 ceph-mon[29462]: 2.1a scrub ok 2024-09-11T07:39:55.336 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:55.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:55.373 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:55.374 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:55.475 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:55.505 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:55.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:55.506 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:55.607 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:55.644 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:55.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:55.645 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:55.746 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:55.782 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:55.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:55.783 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:55.884 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:55.920 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:55.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:55.921 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:55.971 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T07:39:56.022 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:56.052 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:56.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:56.053 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:56.154 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:56.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:56.189 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:56.190 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:56.290 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:56.322 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:56.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:56.323 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:56.424 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:56.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:56.463 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:56.463 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:56.564 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:56.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:56.597 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:56.598 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:56.666 INFO:teuthology.orchestra.run.smithi022.stdout:moved 69 / 474 (14.557%) 2024-09-11T07:39:56.667 INFO:teuthology.orchestra.run.smithi022.stdout:avg 59.25 2024-09-11T07:39:56.667 INFO:teuthology.orchestra.run.smithi022.stdout:stddev 9.07951 -> 4.94343 (expected baseline 7.20026) 2024-09-11T07:39:56.667 INFO:teuthology.orchestra.run.smithi022.stdout:min osd.2 with 42 -> 54 pgs (0.708861 -> 0.911392 * mean) 2024-09-11T07:39:56.667 INFO:teuthology.orchestra.run.smithi022.stdout:max osd.7 with 70 -> 55 pgs (1.18143 -> 0.92827 * mean) 2024-09-11T07:39:56.667 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:39:56.667 INFO:teuthology.orchestra.run.smithi022.stdout:oload 110 2024-09-11T07:39:56.667 INFO:teuthology.orchestra.run.smithi022.stdout:max_change 3 2024-09-11T07:39:56.667 INFO:teuthology.orchestra.run.smithi022.stdout:max_change_osds 4 2024-09-11T07:39:56.667 INFO:teuthology.orchestra.run.smithi022.stdout:average_utilization 0.0261 2024-09-11T07:39:56.667 INFO:teuthology.orchestra.run.smithi022.stdout:overload_utilization 0.0287 2024-09-11T07:39:56.667 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 weight 1.0000 -> 0.6975 2024-09-11T07:39:56.668 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 weight 1.0000 -> 0.7009 2024-09-11T07:39:56.668 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 weight 1.0000 -> 0.7364 2024-09-11T07:39:56.668 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7 weight 1.0000 -> 0.7364 2024-09-11T07:39:56.698 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:56.730 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:56.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:56.731 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:56.832 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:56.855 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:56.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:56.856 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:56.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:56 smithi022 ceph-mon[37330]: 5.8 scrub starts 2024-09-11T07:39:56.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:56 smithi022 ceph-mon[37330]: 5.8 scrub ok 2024-09-11T07:39:56.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:56 smithi022 ceph-mon[37330]: 4.9 scrub starts 2024-09-11T07:39:56.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:56 smithi022 ceph-mon[37330]: 4.9 scrub ok 2024-09-11T07:39:56.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:56 smithi022 ceph-mon[37330]: 5.12 deep-scrub starts 2024-09-11T07:39:56.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:56 smithi022 ceph-mon[37330]: 5.12 deep-scrub ok 2024-09-11T07:39:56.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:56 smithi022 ceph-mon[37330]: from='client.15630 v1:172.21.15.22:0/2340678829' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:39:56.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:56 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'1':'45712','3':'48258','5':'45937','7':'48262'}"}]: dispatch 2024-09-11T07:39:56.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:56 smithi022 ceph-mon[37330]: pgmap v602: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 4.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.55k op/s; 11274/217167 objects misplaced (5.191%); 6.4 MiB/s, 101 objects/s recovering 2024-09-11T07:39:56.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:56 smithi022 ceph-mon[37330]: 5.10 scrub starts 2024-09-11T07:39:56.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:56 smithi022 ceph-mon[37330]: 5.10 scrub ok 2024-09-11T07:39:56.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:56 smithi022 ceph-mon[29462]: 5.8 scrub starts 2024-09-11T07:39:56.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:56 smithi022 ceph-mon[29462]: 5.8 scrub ok 2024-09-11T07:39:56.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:56 smithi022 ceph-mon[29462]: 4.9 scrub starts 2024-09-11T07:39:56.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:56 smithi022 ceph-mon[29462]: 4.9 scrub ok 2024-09-11T07:39:56.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:56 smithi022 ceph-mon[29462]: 5.12 deep-scrub starts 2024-09-11T07:39:56.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:56 smithi022 ceph-mon[29462]: 5.12 deep-scrub ok 2024-09-11T07:39:56.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:56 smithi022 ceph-mon[29462]: from='client.15630 v1:172.21.15.22:0/2340678829' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:39:56.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:56 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'1':'45712','3':'48258','5':'45937','7':'48262'}"}]: dispatch 2024-09-11T07:39:56.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:56 smithi022 ceph-mon[29462]: pgmap v602: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 4.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.55k op/s; 11274/217167 objects misplaced (5.191%); 6.4 MiB/s, 101 objects/s recovering 2024-09-11T07:39:56.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:56 smithi022 ceph-mon[29462]: 5.10 scrub starts 2024-09-11T07:39:56.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:56 smithi022 ceph-mon[29462]: 5.10 scrub ok 2024-09-11T07:39:56.956 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:56.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:56.980 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:56.980 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:56.997 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:39:57.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:56 smithi104 ceph-mon[33507]: 5.8 scrub starts 2024-09-11T07:39:57.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:56 smithi104 ceph-mon[33507]: 5.8 scrub ok 2024-09-11T07:39:57.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:56 smithi104 ceph-mon[33507]: 4.9 scrub starts 2024-09-11T07:39:57.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:56 smithi104 ceph-mon[33507]: 4.9 scrub ok 2024-09-11T07:39:57.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:56 smithi104 ceph-mon[33507]: 5.12 deep-scrub starts 2024-09-11T07:39:57.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:56 smithi104 ceph-mon[33507]: 5.12 deep-scrub ok 2024-09-11T07:39:57.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:56 smithi104 ceph-mon[33507]: from='client.15630 v1:172.21.15.22:0/2340678829' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:39:57.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:56 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'1':'45712','3':'48258','5':'45937','7':'48262'}"}]: dispatch 2024-09-11T07:39:57.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:56 smithi104 ceph-mon[33507]: pgmap v602: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 4.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.55k op/s; 11274/217167 objects misplaced (5.191%); 6.4 MiB/s, 101 objects/s recovering 2024-09-11T07:39:57.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:56 smithi104 ceph-mon[33507]: 5.10 scrub starts 2024-09-11T07:39:57.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:56 smithi104 ceph-mon[33507]: 5.10 scrub ok 2024-09-11T07:39:57.082 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:57.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:57.105 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:57.105 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:57.206 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:57.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:57.231 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:57.231 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:57.332 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:57.356 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:57.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:57.357 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:57.458 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:57.480 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:57.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:57.481 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:57.581 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:57.603 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:57.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:57.604 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:57.704 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:57.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:57.726 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:57.727 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:57.828 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:39:57.851 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:57.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:57.852 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:39:57.953 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:39:57.976 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:57.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:57.976 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:39:58.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:57 smithi104 ceph-mon[33507]: 2.16 scrub starts 2024-09-11T07:39:58.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:57 smithi104 ceph-mon[33507]: 2.16 scrub ok 2024-09-11T07:39:58.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:57 smithi104 ceph-mon[33507]: 5.18 scrub starts 2024-09-11T07:39:58.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:57 smithi104 ceph-mon[33507]: 5.18 scrub ok 2024-09-11T07:39:58.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:57 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'1':'45712','3':'48258','5':'45937','7':'48262'}"}]': finished 2024-09-11T07:39:58.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:57 smithi104 ceph-mon[33507]: osdmap e288: 8 total, 8 up, 8 in 2024-09-11T07:39:58.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:57 smithi104 ceph-mon[33507]: 2.5 scrub starts 2024-09-11T07:39:58.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:57 smithi104 ceph-mon[33507]: 2.5 scrub ok 2024-09-11T07:39:58.077 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:58.100 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:58.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:58.101 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:58.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:57 smithi022 ceph-mon[29462]: 2.16 scrub starts 2024-09-11T07:39:58.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:57 smithi022 ceph-mon[29462]: 2.16 scrub ok 2024-09-11T07:39:58.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:57 smithi022 ceph-mon[29462]: 5.18 scrub starts 2024-09-11T07:39:58.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:57 smithi022 ceph-mon[29462]: 5.18 scrub ok 2024-09-11T07:39:58.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:57 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'1':'45712','3':'48258','5':'45937','7':'48262'}"}]': finished 2024-09-11T07:39:58.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:57 smithi022 ceph-mon[29462]: osdmap e288: 8 total, 8 up, 8 in 2024-09-11T07:39:58.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:57 smithi022 ceph-mon[29462]: 2.5 scrub starts 2024-09-11T07:39:58.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:57 smithi022 ceph-mon[29462]: 2.5 scrub ok 2024-09-11T07:39:58.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:57 smithi022 ceph-mon[37330]: 2.16 scrub starts 2024-09-11T07:39:58.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:57 smithi022 ceph-mon[37330]: 2.16 scrub ok 2024-09-11T07:39:58.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:57 smithi022 ceph-mon[37330]: 5.18 scrub starts 2024-09-11T07:39:58.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:57 smithi022 ceph-mon[37330]: 5.18 scrub ok 2024-09-11T07:39:58.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:57 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'1':'45712','3':'48258','5':'45937','7':'48262'}"}]': finished 2024-09-11T07:39:58.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:57 smithi022 ceph-mon[37330]: osdmap e288: 8 total, 8 up, 8 in 2024-09-11T07:39:58.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:57 smithi022 ceph-mon[37330]: 2.5 scrub starts 2024-09-11T07:39:58.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:57 smithi022 ceph-mon[37330]: 2.5 scrub ok 2024-09-11T07:39:58.201 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:58.225 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:58.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:58.226 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:58.327 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:58.350 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:58.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:58.351 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:58.452 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:58.475 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:58.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:58.476 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:58.577 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:39:58.600 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:58.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:58.601 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:39:58.702 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:58.754 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:58.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:58.755 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:58.856 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:58.878 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:58.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:58.879 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:58.980 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:59.017 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:59.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:59.018 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:39:59.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:58 smithi104 ceph-mon[33507]: 2.10 scrub starts 2024-09-11T07:39:59.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:58 smithi104 ceph-mon[33507]: 2.10 scrub ok 2024-09-11T07:39:59.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:58 smithi104 ceph-mon[33507]: 2.2 scrub starts 2024-09-11T07:39:59.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:58 smithi104 ceph-mon[33507]: 2.2 scrub ok 2024-09-11T07:39:59.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:58 smithi104 ceph-mon[33507]: osdmap e289: 8 total, 8 up, 8 in 2024-09-11T07:39:59.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:58 smithi104 ceph-mon[33507]: pgmap v605: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 4.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 950 op/s; 11274/217167 objects misplaced (5.191%); 3.2 MiB/s, 51 objects/s recovering 2024-09-11T07:39:59.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:58 smithi104 ceph-mon[33507]: osdmap e290: 8 total, 8 up, 8 in 2024-09-11T07:39:59.118 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:59.153 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:59.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:59.154 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:59.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:58 smithi022 ceph-mon[29462]: 2.10 scrub starts 2024-09-11T07:39:59.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:58 smithi022 ceph-mon[29462]: 2.10 scrub ok 2024-09-11T07:39:59.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:58 smithi022 ceph-mon[29462]: 2.2 scrub starts 2024-09-11T07:39:59.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:58 smithi022 ceph-mon[29462]: 2.2 scrub ok 2024-09-11T07:39:59.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:58 smithi022 ceph-mon[29462]: osdmap e289: 8 total, 8 up, 8 in 2024-09-11T07:39:59.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:58 smithi022 ceph-mon[29462]: pgmap v605: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 4.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 950 op/s; 11274/217167 objects misplaced (5.191%); 3.2 MiB/s, 51 objects/s recovering 2024-09-11T07:39:59.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:58 smithi022 ceph-mon[29462]: osdmap e290: 8 total, 8 up, 8 in 2024-09-11T07:39:59.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:58 smithi022 ceph-mon[37330]: 2.10 scrub starts 2024-09-11T07:39:59.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:58 smithi022 ceph-mon[37330]: 2.10 scrub ok 2024-09-11T07:39:59.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:58 smithi022 ceph-mon[37330]: 2.2 scrub starts 2024-09-11T07:39:59.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:58 smithi022 ceph-mon[37330]: 2.2 scrub ok 2024-09-11T07:39:59.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:58 smithi022 ceph-mon[37330]: osdmap e289: 8 total, 8 up, 8 in 2024-09-11T07:39:59.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:58 smithi022 ceph-mon[37330]: pgmap v605: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 4.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 950 op/s; 11274/217167 objects misplaced (5.191%); 3.2 MiB/s, 51 objects/s recovering 2024-09-11T07:39:59.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:58 smithi022 ceph-mon[37330]: osdmap e290: 8 total, 8 up, 8 in 2024-09-11T07:39:59.255 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:59.291 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:59.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:59.292 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:59.393 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:39:59.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:59.427 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:59.427 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:39:59.528 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:59.566 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:59.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:59.567 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:59.667 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:39:59.701 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:39:59.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:59.702 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:39:59.803 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:39:59.843 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:59.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:59.844 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:39:59.945 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:39:59.983 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:39:59.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:39:59.984 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:00.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:39:59 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:40:00.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:59 smithi104 ceph-mon[33507]: 4.2 scrub starts 2024-09-11T07:40:00.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:59 smithi104 ceph-mon[33507]: 4.2 scrub ok 2024-09-11T07:40:00.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:59 smithi104 ceph-mon[33507]: 9.1 starting backfill to osd.4 from (0'0,0'0] MAX to 287'8294 2024-09-11T07:40:00.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:59 smithi104 ceph-mon[33507]: 9.0 starting backfill to osd.0 from (0'0,0'0] MAX to 287'8439 2024-09-11T07:40:00.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:59 smithi104 ceph-mon[33507]: 9.19 starting backfill to osd.4 from (0'0,0'0] MAX to 287'8567 2024-09-11T07:40:00.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:59 smithi104 ceph-mon[33507]: 9.12 starting backfill to osd.2 from (255'2376,285'8233] MIN to 287'8397 2024-09-11T07:40:00.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:59 smithi104 ceph-mon[33507]: 9.16 starting backfill to osd.6 from (0'0,0'0] MAX to 287'8611 2024-09-11T07:40:00.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:59 smithi104 ceph-mon[33507]: 9.14 starting backfill to osd.5 from (0'0,0'0] MAX to 287'8413 2024-09-11T07:40:00.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:59 smithi104 ceph-mon[33507]: 9.18 starting backfill to osd.3 from (254'500,274'7232] MIN to 287'8346 2024-09-11T07:40:00.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:59 smithi104 ceph-mon[33507]: 9.d starting backfill to osd.4 from (0'0,0'0] MAX to 287'11271 2024-09-11T07:40:00.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:59 smithi104 ceph-mon[33507]: 9.8 starting backfill to osd.1 from (0'0,0'0] MAX to 287'8424 2024-09-11T07:40:00.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:59 smithi104 ceph-mon[33507]: 9.2 starting backfill to osd.7 from (0'0,0'0] MAX to 287'8312 2024-09-11T07:40:00.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:59 smithi104 ceph-mon[33507]: 9.7 starting backfill to osd.1 from (0'0,0'0] MAX to 287'8378 2024-09-11T07:40:00.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:59 smithi104 ceph-mon[33507]: 9.5 starting backfill to osd.1 from (0'0,0'0] MAX to 287'8516 2024-09-11T07:40:00.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:59 smithi104 ceph-mon[33507]: 9.15 starting backfill to osd.4 from (0'0,0'0] MAX to 287'8488 2024-09-11T07:40:00.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:59 smithi104 ceph-mon[33507]: 9.d starting backfill to osd.6 from (0'0,0'0] MAX to 287'11271 2024-09-11T07:40:00.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:59 smithi104 ceph-mon[33507]: 9.a starting backfill to osd.3 from (0'0,0'0] MAX to 287'11645 2024-09-11T07:40:00.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:59 smithi104 ceph-mon[33507]: 9.7 starting backfill to osd.2 from (0'0,0'0] MAX to 287'8378 2024-09-11T07:40:00.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:59 smithi104 ceph-mon[33507]: 9.5 starting backfill to osd.3 from (0'0,0'0] MAX to 287'8516 2024-09-11T07:40:00.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:59 smithi104 ceph-mon[33507]: 9.8 starting backfill to osd.7 from (0'0,0'0] MAX to 287'8424 2024-09-11T07:40:00.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:39:59 smithi104 ceph-mon[33507]: 5.1d scrub starts 2024-09-11T07:40:00.085 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:00.132 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:00.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:00.133 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:00.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[29462]: 4.2 scrub starts 2024-09-11T07:40:00.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[29462]: 4.2 scrub ok 2024-09-11T07:40:00.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[29462]: 9.1 starting backfill to osd.4 from (0'0,0'0] MAX to 287'8294 2024-09-11T07:40:00.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[29462]: 9.0 starting backfill to osd.0 from (0'0,0'0] MAX to 287'8439 2024-09-11T07:40:00.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[29462]: 9.19 starting backfill to osd.4 from (0'0,0'0] MAX to 287'8567 2024-09-11T07:40:00.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[29462]: 9.12 starting backfill to osd.2 from (255'2376,285'8233] MIN to 287'8397 2024-09-11T07:40:00.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[29462]: 9.16 starting backfill to osd.6 from (0'0,0'0] MAX to 287'8611 2024-09-11T07:40:00.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[29462]: 9.14 starting backfill to osd.5 from (0'0,0'0] MAX to 287'8413 2024-09-11T07:40:00.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[29462]: 9.18 starting backfill to osd.3 from (254'500,274'7232] MIN to 287'8346 2024-09-11T07:40:00.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[29462]: 9.d starting backfill to osd.4 from (0'0,0'0] MAX to 287'11271 2024-09-11T07:40:00.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[29462]: 9.8 starting backfill to osd.1 from (0'0,0'0] MAX to 287'8424 2024-09-11T07:40:00.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[29462]: 9.2 starting backfill to osd.7 from (0'0,0'0] MAX to 287'8312 2024-09-11T07:40:00.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[29462]: 9.7 starting backfill to osd.1 from (0'0,0'0] MAX to 287'8378 2024-09-11T07:40:00.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[29462]: 9.5 starting backfill to osd.1 from (0'0,0'0] MAX to 287'8516 2024-09-11T07:40:00.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[29462]: 9.15 starting backfill to osd.4 from (0'0,0'0] MAX to 287'8488 2024-09-11T07:40:00.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[29462]: 9.d starting backfill to osd.6 from (0'0,0'0] MAX to 287'11271 2024-09-11T07:40:00.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[29462]: 9.a starting backfill to osd.3 from (0'0,0'0] MAX to 287'11645 2024-09-11T07:40:00.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[29462]: 9.7 starting backfill to osd.2 from (0'0,0'0] MAX to 287'8378 2024-09-11T07:40:00.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[29462]: 9.5 starting backfill to osd.3 from (0'0,0'0] MAX to 287'8516 2024-09-11T07:40:00.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[29462]: 9.8 starting backfill to osd.7 from (0'0,0'0] MAX to 287'8424 2024-09-11T07:40:00.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[29462]: 5.1d scrub starts 2024-09-11T07:40:00.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[37330]: 4.2 scrub starts 2024-09-11T07:40:00.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[37330]: 4.2 scrub ok 2024-09-11T07:40:00.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[37330]: 9.1 starting backfill to osd.4 from (0'0,0'0] MAX to 287'8294 2024-09-11T07:40:00.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[37330]: 9.0 starting backfill to osd.0 from (0'0,0'0] MAX to 287'8439 2024-09-11T07:40:00.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[37330]: 9.19 starting backfill to osd.4 from (0'0,0'0] MAX to 287'8567 2024-09-11T07:40:00.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[37330]: 9.12 starting backfill to osd.2 from (255'2376,285'8233] MIN to 287'8397 2024-09-11T07:40:00.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[37330]: 9.16 starting backfill to osd.6 from (0'0,0'0] MAX to 287'8611 2024-09-11T07:40:00.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[37330]: 9.14 starting backfill to osd.5 from (0'0,0'0] MAX to 287'8413 2024-09-11T07:40:00.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[37330]: 9.18 starting backfill to osd.3 from (254'500,274'7232] MIN to 287'8346 2024-09-11T07:40:00.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[37330]: 9.d starting backfill to osd.4 from (0'0,0'0] MAX to 287'11271 2024-09-11T07:40:00.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[37330]: 9.8 starting backfill to osd.1 from (0'0,0'0] MAX to 287'8424 2024-09-11T07:40:00.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[37330]: 9.2 starting backfill to osd.7 from (0'0,0'0] MAX to 287'8312 2024-09-11T07:40:00.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[37330]: 9.7 starting backfill to osd.1 from (0'0,0'0] MAX to 287'8378 2024-09-11T07:40:00.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[37330]: 9.5 starting backfill to osd.1 from (0'0,0'0] MAX to 287'8516 2024-09-11T07:40:00.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[37330]: 9.15 starting backfill to osd.4 from (0'0,0'0] MAX to 287'8488 2024-09-11T07:40:00.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[37330]: 9.d starting backfill to osd.6 from (0'0,0'0] MAX to 287'11271 2024-09-11T07:40:00.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[37330]: 9.a starting backfill to osd.3 from (0'0,0'0] MAX to 287'11645 2024-09-11T07:40:00.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[37330]: 9.7 starting backfill to osd.2 from (0'0,0'0] MAX to 287'8378 2024-09-11T07:40:00.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[37330]: 9.5 starting backfill to osd.3 from (0'0,0'0] MAX to 287'8516 2024-09-11T07:40:00.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[37330]: 9.8 starting backfill to osd.7 from (0'0,0'0] MAX to 287'8424 2024-09-11T07:40:00.170 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:39:59 smithi022 ceph-mon[37330]: 5.1d scrub starts 2024-09-11T07:40:00.234 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:00.260 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:00.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:00.261 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:00.362 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:00.384 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:00.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:00.385 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:00.486 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:00.506 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:40:00 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:40:00] "GET /metrics HTTP/1.1" 200 224560 "" "Prometheus/2.43.0" 2024-09-11T07:40:00.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:00.528 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:00.528 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:00.629 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:00.670 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:00.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:00.672 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:00.772 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:00.806 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:00.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:00.807 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:00.908 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:00.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:00 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:40:00.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:00 smithi022 ceph-mon[37330]: Health detail: HEALTH_WARN Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull 2024-09-11T07:40:00.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:00 smithi022 ceph-mon[37330]: [WRN] PG_BACKFILL_FULL: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull 2024-09-11T07:40:00.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:00 smithi022 ceph-mon[37330]: pg 9.10 is active+remapped+backfill_toofull, acting [1,2,6] 2024-09-11T07:40:00.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:00 smithi022 ceph-mon[37330]: pg 9.17 is active+remapped+backfill_toofull, acting [7,3,5] 2024-09-11T07:40:00.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:00 smithi022 ceph-mon[37330]: pgmap v607: 158 pgs: 1 active+remapped+backfill_wait, 5 active+remapped+backfilling, 8 active+remapped+backfill_toofull, 144 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.19k op/s; 46948/208710 objects misplaced (22.494%); 4.8 MiB/s, 87 objects/s recovering 2024-09-11T07:40:00.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:00 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:40:00.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:00 smithi022 ceph-mon[29462]: Health detail: HEALTH_WARN Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull 2024-09-11T07:40:00.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:00 smithi022 ceph-mon[29462]: [WRN] PG_BACKFILL_FULL: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull 2024-09-11T07:40:00.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:00 smithi022 ceph-mon[29462]: pg 9.10 is active+remapped+backfill_toofull, acting [1,2,6] 2024-09-11T07:40:00.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:00 smithi022 ceph-mon[29462]: pg 9.17 is active+remapped+backfill_toofull, acting [7,3,5] 2024-09-11T07:40:00.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:00 smithi022 ceph-mon[29462]: pgmap v607: 158 pgs: 1 active+remapped+backfill_wait, 5 active+remapped+backfilling, 8 active+remapped+backfill_toofull, 144 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.19k op/s; 46948/208710 objects misplaced (22.494%); 4.8 MiB/s, 87 objects/s recovering 2024-09-11T07:40:00.946 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:00.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:00.947 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:01.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:00 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:40:01.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:00 smithi104 ceph-mon[33507]: Health detail: HEALTH_WARN Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull 2024-09-11T07:40:01.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:00 smithi104 ceph-mon[33507]: [WRN] PG_BACKFILL_FULL: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull 2024-09-11T07:40:01.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:00 smithi104 ceph-mon[33507]: pg 9.10 is active+remapped+backfill_toofull, acting [1,2,6] 2024-09-11T07:40:01.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:00 smithi104 ceph-mon[33507]: pg 9.17 is active+remapped+backfill_toofull, acting [7,3,5] 2024-09-11T07:40:01.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:00 smithi104 ceph-mon[33507]: pgmap v607: 158 pgs: 1 active+remapped+backfill_wait, 5 active+remapped+backfilling, 8 active+remapped+backfill_toofull, 144 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.19k op/s; 46948/208710 objects misplaced (22.494%); 4.8 MiB/s, 87 objects/s recovering 2024-09-11T07:40:01.049 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:01.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:01.091 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:01.091 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:01.192 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:01.233 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:01.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:01.235 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:01.336 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:01.369 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:01.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:01.370 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:01.390 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:40:01.390 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:40:01.471 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:01.505 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-11T07:40:01.505 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight-by-utilization 110 0.05 2024-09-11T07:40:01.510 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:01.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:01.513 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:01.614 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:01.651 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:01.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:01.652 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:01.753 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:01.766 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:01 smithi022 ceph-mon[29462]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 8 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:40:01.766 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:01 smithi022 ceph-mon[37330]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 8 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:40:01.795 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:01.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:01.796 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:01.897 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:01.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:01.938 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:01.938 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:02.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:01 smithi104 ceph-mon[33507]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 8 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:40:02.039 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:02.082 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:02.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:02.083 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:02.184 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:02.243 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-11T07:40:02.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:02.245 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:02.245 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:02.283 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:40:02.326 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:40:02.341 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:40:02.345 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:02.385 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:02.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:02.386 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:02.487 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:02.541 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:02.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:02.542 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:02.643 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:02.686 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:02.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:02.687 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:02.693 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:40:02.789 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:02.838 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:02.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:02.841 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:02.941 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:02.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:02.983 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:02.983 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:03.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:02 smithi104 ceph-mon[33507]: pgmap v608: 158 pgs: 1 active+remapped+backfill_wait, 5 active+remapped+backfilling, 10 active+remapped+backfill_toofull, 142 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 0 B/s wr, 554 op/s; 55570/207177 objects misplaced (26.822%); 4.9 MiB/s, 90 objects/s recovering 2024-09-11T07:40:03.084 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:03.129 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:03.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:03.130 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:03.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:02 smithi022 ceph-mon[29462]: pgmap v608: 158 pgs: 1 active+remapped+backfill_wait, 5 active+remapped+backfilling, 10 active+remapped+backfill_toofull, 142 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 0 B/s wr, 554 op/s; 55570/207177 objects misplaced (26.822%); 4.9 MiB/s, 90 objects/s recovering 2024-09-11T07:40:03.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:02 smithi022 ceph-mon[37330]: pgmap v608: 158 pgs: 1 active+remapped+backfill_wait, 5 active+remapped+backfilling, 10 active+remapped+backfill_toofull, 142 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 0 B/s wr, 554 op/s; 55570/207177 objects misplaced (26.822%); 4.9 MiB/s, 90 objects/s recovering 2024-09-11T07:40:03.231 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:03.270 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:03.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:03.271 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:03.372 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:03.412 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:03.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:03.413 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:03.514 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:03.553 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:03.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:03.554 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:03.655 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:03.687 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:03.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:03.689 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:03.726 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:03 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:40:03.789 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:03.826 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:03.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:03.827 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:03.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:03 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:40:03.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:03 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:40:03.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:03 smithi022 ceph-mon[29462]: osdmap e291: 8 total, 8 up, 8 in 2024-09-11T07:40:03.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:03 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:40:03.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:03 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:40:03.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:03 smithi022 ceph-mon[37330]: osdmap e291: 8 total, 8 up, 8 in 2024-09-11T07:40:03.929 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:03.964 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:03.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:03.965 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:03.979 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:03 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:40:03.979 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:03 smithi104 ceph-mon[33507]: osdmap e291: 8 total, 8 up, 8 in 2024-09-11T07:40:04.067 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:04.105 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:04.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:04.106 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:04.207 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:04.251 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:04.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:04.252 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:04.279 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:40:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:40:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:40:04.353 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:04.393 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:04.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:04.394 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:04.495 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:04.535 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:04.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:04.536 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:04.637 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:04.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:04.678 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:04.679 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:04.779 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:04.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:04 smithi104 ceph-mon[33507]: pgmap v610: 158 pgs: 1 active+remapped+backfill_wait, 5 active+remapped+backfilling, 10 active+remapped+backfill_toofull, 142 active+clean; 4.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 0 B/s wr, 698 op/s; 53435/203400 objects misplaced (26.271%); 13 MiB/s, 222 objects/s recovering 2024-09-11T07:40:04.812 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:04.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:04.813 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:04.914 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:04.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:04 smithi022 ceph-mon[29462]: pgmap v610: 158 pgs: 1 active+remapped+backfill_wait, 5 active+remapped+backfilling, 10 active+remapped+backfill_toofull, 142 active+clean; 4.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 0 B/s wr, 698 op/s; 53435/203400 objects misplaced (26.271%); 13 MiB/s, 222 objects/s recovering 2024-09-11T07:40:04.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:04 smithi022 ceph-mon[37330]: pgmap v610: 158 pgs: 1 active+remapped+backfill_wait, 5 active+remapped+backfilling, 10 active+remapped+backfill_toofull, 142 active+clean; 4.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 0 B/s wr, 698 op/s; 53435/203400 objects misplaced (26.271%); 13 MiB/s, 222 objects/s recovering 2024-09-11T07:40:04.940 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:04.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:04.941 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:05.042 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:05.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:05.076 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:05.076 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:05.177 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:05.205 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:05.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:05.206 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:05.208 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:40:05.208 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:40:05.307 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:05.341 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:05.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:05.342 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:05.443 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:05.481 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:05.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:05.482 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:05.583 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:05.610 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:05.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:05.611 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:05.712 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:05.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:05.745 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:05.745 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:05.846 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:05.871 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:05.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:05.872 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:05.926 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-11T07:40:05.973 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:06.004 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:06.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:06.006 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:06.106 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:06.143 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:06.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:06.144 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:06.245 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:06.278 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:06.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:06.279 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:06.341 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:40:06.380 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:06.402 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:06.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:06.403 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:06.504 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:06.543 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:06.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:06.544 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:06.645 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:06.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:06.680 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:06.680 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:06.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:06 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:40:06.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:06 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:40:06.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:06 smithi104 ceph-mon[33507]: pgmap v611: 158 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 8 active+remapped+backfill_toofull, 142 active+clean; 3.9 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 1.02k op/s; 47330/192642 objects misplaced (24.569%); 36 MiB/s, 585 objects/s recovering 2024-09-11T07:40:06.781 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:06.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:06.818 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:06.819 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:06 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:40:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:06 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:40:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:06 smithi022 ceph-mon[29462]: pgmap v611: 158 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 8 active+remapped+backfill_toofull, 142 active+clean; 3.9 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 1.02k op/s; 47330/192642 objects misplaced (24.569%); 36 MiB/s, 585 objects/s recovering 2024-09-11T07:40:06.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:06 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:40:06.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:06 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:40:06.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:06 smithi022 ceph-mon[37330]: pgmap v611: 158 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 8 active+remapped+backfill_toofull, 142 active+clean; 3.9 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 1.02k op/s; 47330/192642 objects misplaced (24.569%); 36 MiB/s, 585 objects/s recovering 2024-09-11T07:40:06.920 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:06.953 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:06.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:06.954 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:07.054 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:07.085 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:07.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:07.086 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:07.187 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:07.212 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:07.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:07.213 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:07.314 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:07.340 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:07.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:07.341 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:07.441 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:07.467 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:07.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:07.468 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:07.568 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:07.593 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:07.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:07.594 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:07.694 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:07.725 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:07.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:07.726 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:07.762 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:07 smithi104 ceph-mon[33507]: 4.12 scrub starts 2024-09-11T07:40:07.762 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:07 smithi104 ceph-mon[33507]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 9 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:40:07.762 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:07 smithi104 ceph-mon[33507]: osdmap e292: 8 total, 8 up, 8 in 2024-09-11T07:40:07.827 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:07.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:07.875 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:07.875 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:07.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:07 smithi022 ceph-mon[29462]: 4.12 scrub starts 2024-09-11T07:40:07.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:07 smithi022 ceph-mon[29462]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 9 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:40:07.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:07 smithi022 ceph-mon[29462]: osdmap e292: 8 total, 8 up, 8 in 2024-09-11T07:40:07.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:07 smithi022 ceph-mon[37330]: 4.12 scrub starts 2024-09-11T07:40:07.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:07 smithi022 ceph-mon[37330]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 9 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:40:07.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:07 smithi022 ceph-mon[37330]: osdmap e292: 8 total, 8 up, 8 in 2024-09-11T07:40:07.976 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:08.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:08.016 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:08.016 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:08.117 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:08.169 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:08.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:08.170 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:08.271 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:08.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:08.320 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:08.320 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:08.421 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:08.453 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:40:08.453 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:40:08.456 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:08.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:08.457 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:08.558 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:08.597 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:08.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:08.598 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:08.699 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:08.733 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:08.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:08.734 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:08.835 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:08.868 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:08.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:08.869 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:08.911 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:08 smithi022 ceph-mon[29462]: 4.4 scrub starts 2024-09-11T07:40:08.911 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:08 smithi022 ceph-mon[29462]: osdmap e293: 8 total, 8 up, 8 in 2024-09-11T07:40:08.911 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:40:08.911 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:08 smithi022 ceph-mon[29462]: pgmap v614: 158 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 8 active+remapped+backfill_toofull, 142 active+clean; 3.9 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 807 op/s; 47330/192642 objects misplaced (24.569%); 43 MiB/s, 689 objects/s recovering 2024-09-11T07:40:08.911 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:08 smithi022 ceph-mon[37330]: 4.4 scrub starts 2024-09-11T07:40:08.911 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:08 smithi022 ceph-mon[37330]: osdmap e293: 8 total, 8 up, 8 in 2024-09-11T07:40:08.911 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:40:08.911 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:08 smithi022 ceph-mon[37330]: pgmap v614: 158 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 8 active+remapped+backfill_toofull, 142 active+clean; 3.9 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 807 op/s; 47330/192642 objects misplaced (24.569%); 43 MiB/s, 689 objects/s recovering 2024-09-11T07:40:08.970 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:09.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:09.015 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:09.016 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:09.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:08 smithi104 ceph-mon[33507]: 4.4 scrub starts 2024-09-11T07:40:09.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:08 smithi104 ceph-mon[33507]: osdmap e293: 8 total, 8 up, 8 in 2024-09-11T07:40:09.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:40:09.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:08 smithi104 ceph-mon[33507]: pgmap v614: 158 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfill_wait, 6 active+remapped+backfilling, 8 active+remapped+backfill_toofull, 142 active+clean; 3.9 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 807 op/s; 47330/192642 objects misplaced (24.569%); 43 MiB/s, 689 objects/s recovering 2024-09-11T07:40:09.117 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:09.153 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:09.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:09.154 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:09.255 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:09.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:09.292 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:09.293 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:09.393 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:09.425 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:09.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:09.426 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:09.527 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:09.552 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:09.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:09.553 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:09.654 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:09.686 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:09.686 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is set 2024-09-11T07:40:09.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:09.687 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:09.788 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:09.818 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:09.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:09.819 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:09.921 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:09.933 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:09 smithi022 ceph-mon[29462]: 2.c scrub starts 2024-09-11T07:40:09.933 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:09 smithi022 ceph-mon[29462]: osdmap e294: 8 total, 8 up, 8 in 2024-09-11T07:40:09.934 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:09 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/505455417' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:40:09.934 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:09 smithi022 ceph-mon[37330]: 2.c scrub starts 2024-09-11T07:40:09.934 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:09 smithi022 ceph-mon[37330]: osdmap e294: 8 total, 8 up, 8 in 2024-09-11T07:40:09.934 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:09 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/505455417' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:40:09.954 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:09.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:09.955 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:10.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:40:09 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:40:10.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:09 smithi104 ceph-mon[33507]: 2.c scrub starts 2024-09-11T07:40:10.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:09 smithi104 ceph-mon[33507]: osdmap e294: 8 total, 8 up, 8 in 2024-09-11T07:40:10.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:09 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/505455417' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:40:10.055 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:10.083 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:10.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:10.084 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:10.185 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:10.211 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:10.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:10.213 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:10.313 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:10.340 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:10.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:10.341 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:10.442 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:10.463 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:40:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:40:10] "GET /metrics HTTP/1.1" 200 224713 "" "Prometheus/2.43.0" 2024-09-11T07:40:10.466 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:10.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:10.468 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:10.568 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:10.598 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:10.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:10.599 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:10.691 INFO:teuthology.orchestra.run.smithi022.stdout:moved 16 / 474 (3.37553%) 2024-09-11T07:40:10.691 INFO:teuthology.orchestra.run.smithi022.stdout:avg 59.25 2024-09-11T07:40:10.691 INFO:teuthology.orchestra.run.smithi022.stdout:stddev 4.94343 -> 7.47914 (expected baseline 7.20026) 2024-09-11T07:40:10.691 INFO:teuthology.orchestra.run.smithi022.stdout:min osd.5 with 53 -> 49 pgs (0.894515 -> 0.827004 * mean) 2024-09-11T07:40:10.691 INFO:teuthology.orchestra.run.smithi022.stdout:max osd.4 with 68 -> 69 pgs (1.14768 -> 1.16456 * mean) 2024-09-11T07:40:10.691 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:40:10.692 INFO:teuthology.orchestra.run.smithi022.stdout:oload 110 2024-09-11T07:40:10.692 INFO:teuthology.orchestra.run.smithi022.stdout:max_change 0.05 2024-09-11T07:40:10.692 INFO:teuthology.orchestra.run.smithi022.stdout:max_change_osds 4 2024-09-11T07:40:10.692 INFO:teuthology.orchestra.run.smithi022.stdout:average_utilization 0.0230 2024-09-11T07:40:10.692 INFO:teuthology.orchestra.run.smithi022.stdout:overload_utilization 0.0253 2024-09-11T07:40:10.692 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 weight 0.7364 -> 0.6864 2024-09-11T07:40:10.692 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 weight 0.7009 -> 0.6510 2024-09-11T07:40:10.692 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 weight 0.6975 -> 0.6475 2024-09-11T07:40:10.692 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7 weight 0.7364 -> 0.6864 2024-09-11T07:40:10.700 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:10.720 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:10.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:10.721 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:10.822 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:10.848 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:10 smithi022 ceph-mon[37330]: 2.1 scrub starts 2024-09-11T07:40:10.849 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:10 smithi022 ceph-mon[37330]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:40:10.849 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:10 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/505455417' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:40:10.849 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:10 smithi022 ceph-mon[37330]: osdmap e295: 8 total, 8 up, 8 in 2024-09-11T07:40:10.849 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:10 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:40:10.849 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:10 smithi022 ceph-mon[37330]: pgmap v617: 158 pgs: 2 peering, 1 active+clean+remapped, 3 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 143 active+clean; 3.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 689 op/s; 37559/184386 objects misplaced (20.370%); 86 MiB/s, 1.37k objects/s recovering 2024-09-11T07:40:10.849 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:10 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'1':'42436','3':'44982','5':'42661','7':'44986'}"}]: dispatch 2024-09-11T07:40:10.849 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:10 smithi022 ceph-mon[29462]: 2.1 scrub starts 2024-09-11T07:40:10.849 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:10 smithi022 ceph-mon[29462]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:40:10.850 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:10 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/505455417' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:40:10.850 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:10 smithi022 ceph-mon[29462]: osdmap e295: 8 total, 8 up, 8 in 2024-09-11T07:40:10.850 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:10 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:40:10.850 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:10 smithi022 ceph-mon[29462]: pgmap v617: 158 pgs: 2 peering, 1 active+clean+remapped, 3 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 143 active+clean; 3.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 689 op/s; 37559/184386 objects misplaced (20.370%); 86 MiB/s, 1.37k objects/s recovering 2024-09-11T07:40:10.850 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:10 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'1':'42436','3':'44982','5':'42661','7':'44986'}"}]: dispatch 2024-09-11T07:40:10.856 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:10.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:10.858 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:10.958 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:10.984 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:10.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:10.985 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:11.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:10 smithi104 ceph-mon[33507]: 2.1 scrub starts 2024-09-11T07:40:11.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:10 smithi104 ceph-mon[33507]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:40:11.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:10 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/505455417' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:40:11.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:10 smithi104 ceph-mon[33507]: osdmap e295: 8 total, 8 up, 8 in 2024-09-11T07:40:11.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:10 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:40:11.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:10 smithi104 ceph-mon[33507]: pgmap v617: 158 pgs: 2 peering, 1 active+clean+remapped, 3 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 143 active+clean; 3.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 689 op/s; 37559/184386 objects misplaced (20.370%); 86 MiB/s, 1.37k objects/s recovering 2024-09-11T07:40:11.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:10 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'1':'42436','3':'44982','5':'42661','7':'44986'}"}]: dispatch 2024-09-11T07:40:11.086 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:11.110 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:11.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:11.111 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:11.212 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:11.237 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:11.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:11.238 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:11.338 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:11.363 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:11.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:11.364 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:11.376 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-11T07:40:11.465 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:11.490 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:11.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:11.491 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:11.516 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-11T07:40:11.516 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd test-reweight-by-utilization 1000 3.0 2024-09-11T07:40:11.592 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:11.626 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:11.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:11.627 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:11.670 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:40:11.701 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:11 smithi104 ceph-mon[33507]: from='client.25525 v1:172.21.15.22:0/2239294522' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:40:11.701 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:11 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'1':'42436','3':'44982','5':'42661','7':'44986'}"}]': finished 2024-09-11T07:40:11.702 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:11 smithi104 ceph-mon[33507]: osdmap e296: 8 total, 8 up, 8 in 2024-09-11T07:40:11.728 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:11.756 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:11.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:11.757 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:11.858 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:11.884 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:11.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:11.885 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:11.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:11 smithi022 ceph-mon[29462]: from='client.25525 v1:172.21.15.22:0/2239294522' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:40:11.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:11 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'1':'42436','3':'44982','5':'42661','7':'44986'}"}]': finished 2024-09-11T07:40:11.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:11 smithi022 ceph-mon[29462]: osdmap e296: 8 total, 8 up, 8 in 2024-09-11T07:40:11.908 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:11 smithi022 ceph-mon[37330]: from='client.25525 v1:172.21.15.22:0/2239294522' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:40:11.908 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:11 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'1':'42436','3':'44982','5':'42661','7':'44986'}"}]': finished 2024-09-11T07:40:11.908 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:11 smithi022 ceph-mon[37330]: osdmap e296: 8 total, 8 up, 8 in 2024-09-11T07:40:11.976 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:40:11.986 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:12.009 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:12.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:12.010 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:12.111 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:12.133 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:12.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:12.134 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:12.235 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:12.257 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:12.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:12.258 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:12.359 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:12.381 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:12.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:12.382 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:12.483 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:12.509 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:12.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:12.510 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:12.611 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:12.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:12.634 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:12.635 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:12.735 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:12.759 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:12.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:12.760 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:12.861 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:12.885 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:12.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:12.887 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:12.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:12 smithi022 ceph-mon[29462]: osdmap e297: 8 total, 8 up, 8 in 2024-09-11T07:40:12.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:12 smithi022 ceph-mon[29462]: 9.17 continuing backfill to osd.0 from (255'3107,295'8785] 9:edc6e533:::benchmark_data_smithi022_133943_object125380:head to 295'8785 2024-09-11T07:40:12.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:12 smithi022 ceph-mon[29462]: 9.17 continuing backfill to osd.1 from (255'3107,295'8785] 9:edc83675:::benchmark_data_smithi022_133943_object75782:head to 295'8785 2024-09-11T07:40:12.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:12 smithi022 ceph-mon[29462]: 9.17 starting backfill to osd.2 from (0'0,0'0] MAX to 295'8785 2024-09-11T07:40:12.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:12 smithi022 ceph-mon[29462]: pgmap v620: 158 pgs: 2 peering, 1 active+clean+remapped, 3 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 143 active+clean; 3.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 809 op/s; 37167/182943 objects misplaced (20.316%); 86 MiB/s, 1.37k objects/s recovering 2024-09-11T07:40:12.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:12 smithi022 ceph-mon[29462]: osdmap e298: 8 total, 8 up, 8 in 2024-09-11T07:40:12.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:12 smithi022 ceph-mon[37330]: osdmap e297: 8 total, 8 up, 8 in 2024-09-11T07:40:12.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:12 smithi022 ceph-mon[37330]: 9.17 continuing backfill to osd.0 from (255'3107,295'8785] 9:edc6e533:::benchmark_data_smithi022_133943_object125380:head to 295'8785 2024-09-11T07:40:12.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:12 smithi022 ceph-mon[37330]: 9.17 continuing backfill to osd.1 from (255'3107,295'8785] 9:edc83675:::benchmark_data_smithi022_133943_object75782:head to 295'8785 2024-09-11T07:40:12.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:12 smithi022 ceph-mon[37330]: 9.17 starting backfill to osd.2 from (0'0,0'0] MAX to 295'8785 2024-09-11T07:40:12.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:12 smithi022 ceph-mon[37330]: pgmap v620: 158 pgs: 2 peering, 1 active+clean+remapped, 3 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 143 active+clean; 3.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 809 op/s; 37167/182943 objects misplaced (20.316%); 86 MiB/s, 1.37k objects/s recovering 2024-09-11T07:40:12.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:12 smithi022 ceph-mon[37330]: osdmap e298: 8 total, 8 up, 8 in 2024-09-11T07:40:12.987 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:13.024 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:13.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:13.025 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:13.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:12 smithi104 ceph-mon[33507]: osdmap e297: 8 total, 8 up, 8 in 2024-09-11T07:40:13.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:12 smithi104 ceph-mon[33507]: 9.17 continuing backfill to osd.0 from (255'3107,295'8785] 9:edc6e533:::benchmark_data_smithi022_133943_object125380:head to 295'8785 2024-09-11T07:40:13.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:12 smithi104 ceph-mon[33507]: 9.17 continuing backfill to osd.1 from (255'3107,295'8785] 9:edc83675:::benchmark_data_smithi022_133943_object75782:head to 295'8785 2024-09-11T07:40:13.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:12 smithi104 ceph-mon[33507]: 9.17 starting backfill to osd.2 from (0'0,0'0] MAX to 295'8785 2024-09-11T07:40:13.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:12 smithi104 ceph-mon[33507]: pgmap v620: 158 pgs: 2 peering, 1 active+clean+remapped, 3 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 143 active+clean; 3.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 809 op/s; 37167/182943 objects misplaced (20.316%); 86 MiB/s, 1.37k objects/s recovering 2024-09-11T07:40:13.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:12 smithi104 ceph-mon[33507]: osdmap e298: 8 total, 8 up, 8 in 2024-09-11T07:40:13.126 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:13.141 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:40:13.141 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:40:13.158 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:13.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:13.159 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:13.167 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:40:13.167 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:40:13.190 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:40:13.190 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:40:13.213 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:40:13.213 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:40:13.240 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:40:13.240 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:40:13.260 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:13.264 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:40:13.264 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:40:13.284 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:40:13.284 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:40:13.286 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:13.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:13.287 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:13.303 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:40:13.303 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:40:13.388 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:13.413 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:13.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:13.414 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:13.515 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:13.539 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:13.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:13.540 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:13.641 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:13.669 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:13.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:13.670 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:13.771 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:13.807 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:13.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:13.808 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:13.908 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:13.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:13 smithi022 ceph-mon[29462]: 9.14 starting backfill to osd.2 from (0'0,0'0] MAX to 295'8687 2024-09-11T07:40:13.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:13 smithi022 ceph-mon[29462]: 9.e starting backfill to osd.0 from (0'0,0'0] MAX to 295'11946 2024-09-11T07:40:13.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:13 smithi022 ceph-mon[29462]: 9.13 starting backfill to osd.0 from (0'0,0'0] MAX to 295'8590 2024-09-11T07:40:13.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:13 smithi022 ceph-mon[29462]: 9.14 continuing backfill to osd.5 from (255'2886,295'8687] MIN to 295'8687 2024-09-11T07:40:13.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:13 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1976695111' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:40:13.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:13 smithi022 ceph-mon[29462]: osdmap e299: 8 total, 8 up, 8 in 2024-09-11T07:40:13.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:13 smithi022 ceph-mon[37330]: 9.14 starting backfill to osd.2 from (0'0,0'0] MAX to 295'8687 2024-09-11T07:40:13.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:13 smithi022 ceph-mon[37330]: 9.e starting backfill to osd.0 from (0'0,0'0] MAX to 295'11946 2024-09-11T07:40:13.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:13 smithi022 ceph-mon[37330]: 9.13 starting backfill to osd.0 from (0'0,0'0] MAX to 295'8590 2024-09-11T07:40:13.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:13 smithi022 ceph-mon[37330]: 9.14 continuing backfill to osd.5 from (255'2886,295'8687] MIN to 295'8687 2024-09-11T07:40:13.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:13 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1976695111' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:40:13.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:13 smithi022 ceph-mon[37330]: osdmap e299: 8 total, 8 up, 8 in 2024-09-11T07:40:13.940 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T07:40:13.955 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:13.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:13.956 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:14.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:13 smithi104 ceph-mon[33507]: 9.14 starting backfill to osd.2 from (0'0,0'0] MAX to 295'8687 2024-09-11T07:40:14.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:13 smithi104 ceph-mon[33507]: 9.e starting backfill to osd.0 from (0'0,0'0] MAX to 295'11946 2024-09-11T07:40:14.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:13 smithi104 ceph-mon[33507]: 9.13 starting backfill to osd.0 from (0'0,0'0] MAX to 295'8590 2024-09-11T07:40:14.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:13 smithi104 ceph-mon[33507]: 9.14 continuing backfill to osd.5 from (255'2886,295'8687] MIN to 295'8687 2024-09-11T07:40:14.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:13 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1976695111' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:40:14.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:13 smithi104 ceph-mon[33507]: osdmap e299: 8 total, 8 up, 8 in 2024-09-11T07:40:14.057 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:14.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:14.096 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:14.096 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:14.197 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:14.234 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:14.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:14.235 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:14.336 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:14.373 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:14.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:14.374 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:14.475 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:14.511 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:14.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:14.512 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:14.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:40:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:40:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:40:14.613 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:14.656 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:14.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:14.657 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:14.758 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:14.819 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:14.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:14.820 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:14.921 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:14.957 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:14.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:14.958 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:15.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:14 smithi104 ceph-mon[33507]: pgmap v623: 158 pgs: 1 peering, 1 active+clean+remapped, 5 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 142 active+clean; 3.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 0 B/s wr, 223 op/s; 40234/181707 objects misplaced (22.142%); 38 MiB/s, 609 objects/s recovering 2024-09-11T07:40:15.059 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:15.092 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:15.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:15.093 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:15.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:14 smithi022 ceph-mon[29462]: pgmap v623: 158 pgs: 1 peering, 1 active+clean+remapped, 5 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 142 active+clean; 3.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 0 B/s wr, 223 op/s; 40234/181707 objects misplaced (22.142%); 38 MiB/s, 609 objects/s recovering 2024-09-11T07:40:15.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:14 smithi022 ceph-mon[37330]: pgmap v623: 158 pgs: 1 peering, 1 active+clean+remapped, 5 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 142 active+clean; 3.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 0 B/s wr, 223 op/s; 40234/181707 objects misplaced (22.142%); 38 MiB/s, 609 objects/s recovering 2024-09-11T07:40:15.194 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:15.237 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:15.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:15.238 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:15.338 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:15.377 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:15.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:15.378 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:15.479 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:15.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:15.522 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:15.523 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:15.623 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:15.650 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:15.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:15.651 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:15.752 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:15.790 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:15.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:15.792 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:15.893 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:15.933 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:15.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:15.934 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:16.035 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:16.070 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:16.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:16.071 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:16.172 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:16.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:16.215 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:16.215 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:16.316 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:16.351 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:16.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:16.352 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:16.456 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:16.486 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:16.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:16.487 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:16.588 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:16.625 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:16.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:16.626 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:16.727 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:16.751 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:40:16.755 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:16.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:16.756 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:16.785 INFO:teuthology.orchestra.run.smithi022.stdout:moved 0 / 474 (0%) 2024-09-11T07:40:16.785 INFO:teuthology.orchestra.run.smithi022.stdout:avg 59.25 2024-09-11T07:40:16.785 INFO:teuthology.orchestra.run.smithi022.stdout:stddev 7.47914 -> 7.47914 (expected baseline 7.20026) 2024-09-11T07:40:16.785 INFO:teuthology.orchestra.run.smithi022.stdout:min osd.7 with 48 -> 48 pgs (0.810127 -> 0.810127 * mean) 2024-09-11T07:40:16.785 INFO:teuthology.orchestra.run.smithi022.stdout:max osd.4 with 69 -> 69 pgs (1.16456 -> 1.16456 * mean) 2024-09-11T07:40:16.785 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:40:16.785 INFO:teuthology.orchestra.run.smithi022.stdout:oload 1000 2024-09-11T07:40:16.785 INFO:teuthology.orchestra.run.smithi022.stdout:max_change 3 2024-09-11T07:40:16.786 INFO:teuthology.orchestra.run.smithi022.stdout:max_change_osds 4 2024-09-11T07:40:16.786 INFO:teuthology.orchestra.run.smithi022.stdout:average_utilization 0.0230 2024-09-11T07:40:16.786 INFO:teuthology.orchestra.run.smithi022.stdout:overload_utilization 0.2304 2024-09-11T07:40:16.786 INFO:teuthology.orchestra.run.smithi022.stderr:no change 2024-09-11T07:40:16.856 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:16.894 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:16.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:16.895 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:16.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:16 smithi022 ceph-mon[29462]: from='client.25537 v1:172.21.15.22:0/2714266899' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:40:16.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:16 smithi022 ceph-mon[29462]: pgmap v624: 158 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 145 active+clean; 3.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 251 op/s; 38134/178758 objects misplaced (21.333%); 61 MiB/s, 980 objects/s recovering 2024-09-11T07:40:16.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:16 smithi022 ceph-mon[37330]: from='client.25537 v1:172.21.15.22:0/2714266899' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:40:16.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:16 smithi022 ceph-mon[37330]: pgmap v624: 158 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 145 active+clean; 3.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 251 op/s; 38134/178758 objects misplaced (21.333%); 61 MiB/s, 980 objects/s recovering 2024-09-11T07:40:16.996 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:16 smithi104 ceph-mon[33507]: from='client.25537 v1:172.21.15.22:0/2714266899' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:40:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:16 smithi104 ceph-mon[33507]: pgmap v624: 158 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 145 active+clean; 3.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 921 B/s rd, 0 B/s wr, 251 op/s; 38134/178758 objects misplaced (21.333%); 61 MiB/s, 980 objects/s recovering 2024-09-11T07:40:17.048 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:17.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:17.049 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:17.150 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:17.188 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:17.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:17.189 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:17.220 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:40:17.289 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:17.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:17.325 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:17.326 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:17.426 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:17.469 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:17.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:17.470 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:17.570 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:17.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:17.593 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:17.594 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:17.694 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:17.730 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:17.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:17.731 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:17.735 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-11T07:40:17.735 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd test-reweight-by-utilization 1000 3.0 2024-09-11T07:40:17.832 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:17.855 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:17.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:17.856 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:17.957 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:18.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:18.014 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:18.014 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:18.115 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:18.176 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:18.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:18.177 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:18.278 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:18.315 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:18.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:18.316 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:18.380 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:40:18.417 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:18.451 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:18.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:18.452 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:18.552 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:18.593 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:18.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:18.594 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:18.695 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:18.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:18.754 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:18.755 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:18.855 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:18.895 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:18.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:18.896 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:40:18.998 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:19.035 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:19.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:19.036 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:19.137 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:19.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:19.176 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:19.177 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:19.277 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:19.334 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:19.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:19.335 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:19.436 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:19.473 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:19 smithi022 ceph-mon[29462]: pgmap v625: 158 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 145 active+clean; 3.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 780 B/s rd, 0 B/s wr, 213 op/s; 38134/178758 objects misplaced (21.333%); 52 MiB/s, 830 objects/s recovering 2024-09-11T07:40:19.473 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:19 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:40:19.474 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:19 smithi022 ceph-mon[37330]: pgmap v625: 158 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 145 active+clean; 3.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 780 B/s rd, 0 B/s wr, 213 op/s; 38134/178758 objects misplaced (21.333%); 52 MiB/s, 830 objects/s recovering 2024-09-11T07:40:19.474 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:19 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:40:19.476 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:19 smithi104 ceph-mon[33507]: pgmap v625: 158 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 145 active+clean; 3.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 780 B/s rd, 0 B/s wr, 213 op/s; 38134/178758 objects misplaced (21.333%); 52 MiB/s, 830 objects/s recovering 2024-09-11T07:40:19.476 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:19 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:40:19.484 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:19.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:19.485 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:19.586 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:19.617 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:19.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:19.618 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:19.719 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:19.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:19.759 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:19.759 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:19.860 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:19.903 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:19.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:19.904 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:19.931 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:40:19.931 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2024-09-11T07:40:19.931 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2024-09-11T07:40:19.931 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2024-09-11T07:40:19.931 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.931 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:120661 7.1 7:8ef41966:::benchmark_data_smithi022_77720_object120660:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:40:19.931 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:30:29.046728+0000", 2024-09-11T07:40:19.931 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 590.88023256500003, 2024-09-11T07:40:19.931 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.20704042, 2024-09-11T07:40:19.932 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:19.932 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:19.932 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:19.932 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:40:19.932 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:40:19.932 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 120661 2024-09-11T07:40:19.932 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.932 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:19.932 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.932 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:19.932 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.046728+0000", 2024-09-11T07:40:19.932 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:19.933 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.933 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.933 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:19.933 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.046728+0000", 2024-09-11T07:40:19.933 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:19.933 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.934 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.934 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:19.934 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.046727+0000", 2024-09-11T07:40:19.934 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:40:19.934 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.934 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.934 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:19.934 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.046819+0000", 2024-09-11T07:40:19.934 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2175000000000004e-05 2024-09-11T07:40:19.934 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.935 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.935 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:19.935 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.046839+0000", 2024-09-11T07:40:19.935 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9698999999999999e-05 2024-09-11T07:40:19.935 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.935 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.935 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:19.935 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.046846+0000", 2024-09-11T07:40:19.935 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9489999999999999e-06 2024-09-11T07:40:19.936 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.936 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.936 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:19.936 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.050121+0000", 2024-09-11T07:40:19.936 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0032747359999999999 2024-09-11T07:40:19.936 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.936 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.936 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:19.936 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.050236+0000", 2024-09-11T07:40:19.936 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011511899999999999 2024-09-11T07:40:19.936 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.936 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.937 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,6", 2024-09-11T07:40:19.937 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.050381+0000", 2024-09-11T07:40:19.937 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014511999999999999 2024-09-11T07:40:19.937 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.937 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.937 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:19.937 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.052599+0000", 2024-09-11T07:40:19.937 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0022178979999999998 2024-09-11T07:40:19.937 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.937 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.937 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:19.937 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.055720+0000", 2024-09-11T07:40:19.937 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0031211479999999998 2024-09-11T07:40:19.937 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.938 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.938 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:19.938 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.253668+0000", 2024-09-11T07:40:19.938 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.19794822000000001 2024-09-11T07:40:19.938 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.938 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.938 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:19.938 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.253726+0000", 2024-09-11T07:40:19.938 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7473999999999999e-05 2024-09-11T07:40:19.938 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.938 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.938 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:19.938 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.253768+0000", 2024-09-11T07:40:19.938 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2888e-05 2024-09-11T07:40:19.939 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:19.939 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:19.939 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:19.939 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.939 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.939 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24778.0:120666 7.1 7:89a791ca:::benchmark_data_smithi022_77720_object120665:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-11T07:40:19.939 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:30:29.052912+0000", 2024-09-11T07:40:19.939 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 590.874048346, 2024-09-11T07:40:19.939 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.204864612, 2024-09-11T07:40:19.939 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:19.940 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:19.940 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:19.940 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24778", 2024-09-11T07:40:19.940 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:40:19.940 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 120666 2024-09-11T07:40:19.940 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.940 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:19.940 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.940 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:19.940 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.052912+0000", 2024-09-11T07:40:19.941 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:19.941 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.941 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.941 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:19.941 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.052912+0000", 2024-09-11T07:40:19.941 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:19.941 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.941 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.941 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:19.941 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.052911+0000", 2024-09-11T07:40:19.941 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:40:19.941 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.941 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.942 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:19.942 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.052968+0000", 2024-09-11T07:40:19.942 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7132000000000001e-05 2024-09-11T07:40:19.942 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.942 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.942 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:19.942 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.052982+0000", 2024-09-11T07:40:19.942 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3913999999999999e-05 2024-09-11T07:40:19.942 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.942 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.942 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:19.943 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.052987+0000", 2024-09-11T07:40:19.943 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6849999999999998e-06 2024-09-11T07:40:19.943 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.943 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.943 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:19.943 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.053032+0000", 2024-09-11T07:40:19.943 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4495e-05 2024-09-11T07:40:19.943 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.943 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.943 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:19.943 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.053152+0000", 2024-09-11T07:40:19.943 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000120281 2024-09-11T07:40:19.943 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.943 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.944 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 3,6", 2024-09-11T07:40:19.944 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.053286+0000", 2024-09-11T07:40:19.944 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000133325 2024-09-11T07:40:19.944 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.944 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.944 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:19.944 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.055758+0000", 2024-09-11T07:40:19.944 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0024721669999999999 2024-09-11T07:40:19.944 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.944 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.944 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:19.944 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.055855+0000", 2024-09-11T07:40:19.944 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7077999999999994e-05 2024-09-11T07:40:19.945 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.945 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.945 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:19.945 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.257692+0000", 2024-09-11T07:40:19.945 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.201837129 2024-09-11T07:40:19.945 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.945 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.945 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:19.945 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.257739+0000", 2024-09-11T07:40:19.945 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7064999999999997e-05 2024-09-11T07:40:19.945 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.945 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.946 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:19.946 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:30:29.257777+0000", 2024-09-11T07:40:19.946 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7870000000000002e-05 2024-09-11T07:40:19.946 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:19.946 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:19.946 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:19.946 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.946 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.946 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:28697 8.2 8:4b5bfabd:::benchmark_data_smithi022_108406_object28696:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e189)", 2024-09-11T07:40:19.946 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:33:51.017713+0000", 2024-09-11T07:40:19.946 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 388.90924796899998, 2024-09-11T07:40:19.946 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13638721500000001, 2024-09-11T07:40:19.947 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:19.947 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:19.947 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:19.947 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:40:19.948 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:40:19.948 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 28697 2024-09-11T07:40:19.948 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.948 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:19.948 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.948 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:19.948 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.017713+0000", 2024-09-11T07:40:19.948 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:19.948 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.949 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.949 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:19.949 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.017713+0000", 2024-09-11T07:40:19.949 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:19.949 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.949 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.950 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:19.950 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.017710+0000", 2024-09-11T07:40:19.950 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:40:19.950 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.950 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.950 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:19.950 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.017742+0000", 2024-09-11T07:40:19.950 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2410999999999999e-05 2024-09-11T07:40:19.951 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.951 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.951 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:19.951 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.017752+0000", 2024-09-11T07:40:19.951 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2960000000000004e-06 2024-09-11T07:40:19.951 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.951 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.951 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:19.951 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.017757+0000", 2024-09-11T07:40:19.951 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.023e-06 2024-09-11T07:40:19.951 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.951 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.952 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:19.952 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.017821+0000", 2024-09-11T07:40:19.952 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4518999999999997e-05 2024-09-11T07:40:19.952 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.952 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.952 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:19.952 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.017946+0000", 2024-09-11T07:40:19.952 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000125127 2024-09-11T07:40:19.952 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.952 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.952 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:40:19.952 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.018081+0000", 2024-09-11T07:40:19.952 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013491999999999999 2024-09-11T07:40:19.953 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.953 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.953 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:19.953 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.020234+0000", 2024-09-11T07:40:19.953 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0021525670000000002 2024-09-11T07:40:19.953 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.953 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.953 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:19.953 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.020972+0000", 2024-09-11T07:40:19.953 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00073804999999999997 2024-09-11T07:40:19.954 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.954 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.954 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:19.954 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.154017+0000", 2024-09-11T07:40:19.954 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13304534800000001 2024-09-11T07:40:19.954 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.954 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.954 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:19.954 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.154065+0000", 2024-09-11T07:40:19.954 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7428000000000001e-05 2024-09-11T07:40:19.954 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.954 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.954 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:19.954 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:33:51.154100+0000", 2024-09-11T07:40:19.955 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5089999999999998e-05 2024-09-11T07:40:19.955 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:19.955 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:19.955 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:19.955 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.955 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.955 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:65082 8.2 8:4db2ff4f:::benchmark_data_smithi022_108406_object65081:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-11T07:40:19.955 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:13.886436+0000", 2024-09-11T07:40:19.955 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 366.04052488799999, 2024-09-11T07:40:19.955 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.139453254, 2024-09-11T07:40:19.955 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:19.955 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:19.955 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:19.955 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:40:19.956 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:40:19.956 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 65082 2024-09-11T07:40:19.956 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.956 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:19.956 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.956 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:19.956 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.886436+0000", 2024-09-11T07:40:19.956 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:19.956 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.956 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.957 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:19.957 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.886436+0000", 2024-09-11T07:40:19.957 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:19.957 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.957 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.957 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:19.957 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.886433+0000", 2024-09-11T07:40:19.957 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:40:19.957 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.957 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.957 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:19.957 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.886487+0000", 2024-09-11T07:40:19.957 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3309000000000001e-05 2024-09-11T07:40:19.957 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.958 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.958 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:19.958 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.886497+0000", 2024-09-11T07:40:19.958 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0139999999999999e-05 2024-09-11T07:40:19.958 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.958 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.958 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:19.958 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.886502+0000", 2024-09-11T07:40:19.958 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3550000000000004e-06 2024-09-11T07:40:19.958 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.958 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.958 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:19.958 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.886576+0000", 2024-09-11T07:40:19.959 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4184000000000007e-05 2024-09-11T07:40:19.959 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.959 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.959 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:19.959 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.886696+0000", 2024-09-11T07:40:19.959 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000119938 2024-09-11T07:40:19.959 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.959 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.959 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:40:19.959 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.886784+0000", 2024-09-11T07:40:19.959 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8166999999999998e-05 2024-09-11T07:40:19.959 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.959 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.959 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:19.959 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.890708+0000", 2024-09-11T07:40:19.960 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0039232260000000001 2024-09-11T07:40:19.960 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.960 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.960 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:19.960 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.892953+0000", 2024-09-11T07:40:19.960 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002245613 2024-09-11T07:40:19.960 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.960 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.960 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:19.961 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:14.025783+0000", 2024-09-11T07:40:19.961 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.132830107 2024-09-11T07:40:19.961 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.961 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.961 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:19.961 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:14.025846+0000", 2024-09-11T07:40:19.961 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2768999999999995e-05 2024-09-11T07:40:19.961 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.961 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.961 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:19.961 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:14.025889+0000", 2024-09-11T07:40:19.961 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2821000000000002e-05 2024-09-11T07:40:19.962 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:19.962 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:19.962 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:19.962 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.962 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.962 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:65085 8.2 8:4f7da7cb:::benchmark_data_smithi022_108406_object65084:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-11T07:40:19.962 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:13.886854+0000", 2024-09-11T07:40:19.962 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 366.04010664899999, 2024-09-11T07:40:19.962 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13916767199999999, 2024-09-11T07:40:19.962 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:19.962 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:19.962 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:19.962 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:40:19.962 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:40:19.963 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 65085 2024-09-11T07:40:19.963 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.963 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:19.963 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.963 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:19.963 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.886854+0000", 2024-09-11T07:40:19.963 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:19.963 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.963 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.963 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:19.963 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.886854+0000", 2024-09-11T07:40:19.964 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:19.964 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.964 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.964 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:19.964 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.886852+0000", 2024-09-11T07:40:19.964 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:40:19.964 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.964 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.964 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:19.964 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.886913+0000", 2024-09-11T07:40:19.964 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0532000000000003e-05 2024-09-11T07:40:19.964 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.964 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.965 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:19.965 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.886929+0000", 2024-09-11T07:40:19.965 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5744999999999999e-05 2024-09-11T07:40:19.965 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.965 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.965 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:19.965 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.886934+0000", 2024-09-11T07:40:19.965 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6540000000000004e-06 2024-09-11T07:40:19.965 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.965 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.965 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:19.965 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.888454+0000", 2024-09-11T07:40:19.965 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0015198449999999999 2024-09-11T07:40:19.965 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.966 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.966 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:19.966 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.888586+0000", 2024-09-11T07:40:19.966 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013134900000000001 2024-09-11T07:40:19.966 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.966 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.966 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:40:19.966 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.888737+0000", 2024-09-11T07:40:19.966 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015161499999999999 2024-09-11T07:40:19.967 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.967 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.967 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:19.967 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.892918+0000", 2024-09-11T07:40:19.967 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0041806990000000004 2024-09-11T07:40:19.967 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.967 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.967 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:19.967 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.893005+0000", 2024-09-11T07:40:19.967 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7029999999999999e-05 2024-09-11T07:40:19.967 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.967 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.967 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:19.967 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:14.025938+0000", 2024-09-11T07:40:19.968 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13293284499999999 2024-09-11T07:40:19.968 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.968 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.968 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:19.968 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:14.025985+0000", 2024-09-11T07:40:19.968 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7468e-05 2024-09-11T07:40:19.968 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.968 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.968 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:19.968 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:14.026022+0000", 2024-09-11T07:40:19.968 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6350000000000003e-05 2024-09-11T07:40:19.968 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:19.968 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:19.968 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:19.969 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.969 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.969 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25072.0:65088 8.2 8:4f03b95a:::benchmark_data_smithi022_108406_object65087:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-11T07:40:19.969 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:34:13.888672+0000", 2024-09-11T07:40:19.969 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 366.03828852499998, 2024-09-11T07:40:19.969 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.141509143, 2024-09-11T07:40:19.969 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:19.969 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:19.969 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:19.969 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25072", 2024-09-11T07:40:19.969 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2669833463", 2024-09-11T07:40:19.969 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 65088 2024-09-11T07:40:19.969 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.969 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:19.970 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.970 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:19.970 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.888672+0000", 2024-09-11T07:40:19.970 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:19.970 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.970 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.970 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:19.970 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.888672+0000", 2024-09-11T07:40:19.970 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:19.970 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.971 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.971 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:19.971 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.888671+0000", 2024-09-11T07:40:19.971 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:40:19.971 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.971 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.971 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:19.971 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.888720+0000", 2024-09-11T07:40:19.971 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9740999999999997e-05 2024-09-11T07:40:19.971 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.971 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.971 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:19.972 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.888733+0000", 2024-09-11T07:40:19.972 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2593e-05 2024-09-11T07:40:19.972 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.972 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.972 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:19.972 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.888739+0000", 2024-09-11T07:40:19.972 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5860000000000004e-06 2024-09-11T07:40:19.972 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.972 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.972 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:19.972 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.890740+0000", 2024-09-11T07:40:19.972 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002001152 2024-09-11T07:40:19.972 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.972 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.973 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:19.973 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.890894+0000", 2024-09-11T07:40:19.973 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015384900000000001 2024-09-11T07:40:19.973 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.973 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.973 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,5", 2024-09-11T07:40:19.973 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.891048+0000", 2024-09-11T07:40:19.973 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015442099999999999 2024-09-11T07:40:19.973 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.973 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.973 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:19.973 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.893103+0000", 2024-09-11T07:40:19.974 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0020548710000000002 2024-09-11T07:40:19.974 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.974 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.974 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:19.974 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:13.893212+0000", 2024-09-11T07:40:19.974 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010898799999999999 2024-09-11T07:40:19.974 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.974 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.974 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:19.975 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:14.030114+0000", 2024-09-11T07:40:19.975 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.136901784 2024-09-11T07:40:19.975 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.975 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.975 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:19.975 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:14.030145+0000", 2024-09-11T07:40:19.975 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1510999999999997e-05 2024-09-11T07:40:19.975 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.975 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.975 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:19.975 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:34:14.030181+0000", 2024-09-11T07:40:19.975 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6133000000000001e-05 2024-09-11T07:40:19.975 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:19.976 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:19.976 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:19.976 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.976 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.976 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87214 9.10 9.cbc93d10 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:40:19.976 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.358568+0000", 2024-09-11T07:40:19.976 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 111.56839237600001, 2024-09-11T07:40:19.976 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.98231133800000003, 2024-09-11T07:40:19.976 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:19.976 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:40:19.976 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:19.976 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:19.976 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:19.977 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87214 2024-09-11T07:40:19.977 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.977 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:19.977 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.977 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:19.977 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.358568+0000", 2024-09-11T07:40:19.977 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:19.977 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.977 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.977 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:19.978 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.358568+0000", 2024-09-11T07:40:19.978 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:19.978 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.978 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.978 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:19.978 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.358565+0000", 2024-09-11T07:40:19.978 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999971 2024-09-11T07:40:19.978 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.978 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.978 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:19.979 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.358689+0000", 2024-09-11T07:40:19.979 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012315399999999999 2024-09-11T07:40:19.979 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.979 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.979 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:19.979 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.358704+0000", 2024-09-11T07:40:19.979 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5931e-05 2024-09-11T07:40:19.979 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.979 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.979 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:19.979 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.358711+0000", 2024-09-11T07:40:19.979 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7410000000000003e-06 2024-09-11T07:40:19.979 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.979 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.979 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:19.980 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.358765+0000", 2024-09-11T07:40:19.980 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3458000000000002e-05 2024-09-11T07:40:19.980 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.980 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.980 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:40:19.980 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.358773+0000", 2024-09-11T07:40:19.980 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.892e-06 2024-09-11T07:40:19.980 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.980 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.980 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:19.980 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.340867+0000", 2024-09-11T07:40:19.980 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.98209449400000004 2024-09-11T07:40:19.981 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.981 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.981 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:19.981 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.340880+0000", 2024-09-11T07:40:19.981 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2479e-05 2024-09-11T07:40:19.981 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:19.981 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:19.981 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:19.981 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.981 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.982 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87241 9.16 9.52d9f9b6 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:40:19.982 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.372509+0000", 2024-09-11T07:40:19.982 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 111.55445194399999, 2024-09-11T07:40:19.982 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.97069455699999996, 2024-09-11T07:40:19.982 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:19.982 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:40:19.982 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:19.982 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:19.982 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:19.982 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87241 2024-09-11T07:40:19.982 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.982 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:19.982 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.982 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:19.983 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372509+0000", 2024-09-11T07:40:19.983 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:19.983 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.983 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.983 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:19.983 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372509+0000", 2024-09-11T07:40:19.983 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:19.983 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.983 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.983 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:19.983 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372506+0000", 2024-09-11T07:40:19.983 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999971 2024-09-11T07:40:19.983 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.983 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.983 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:19.984 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372572+0000", 2024-09-11T07:40:19.984 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6177e-05 2024-09-11T07:40:19.984 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.984 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.984 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:19.984 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372586+0000", 2024-09-11T07:40:19.984 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3866000000000001e-05 2024-09-11T07:40:19.984 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.984 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.984 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:19.984 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372592+0000", 2024-09-11T07:40:19.984 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8490000000000002e-06 2024-09-11T07:40:19.984 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.984 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.984 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:19.985 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372635+0000", 2024-09-11T07:40:19.985 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3566999999999998e-05 2024-09-11T07:40:19.985 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.985 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.985 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:40:19.985 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372644+0000", 2024-09-11T07:40:19.985 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4020000000000002e-06 2024-09-11T07:40:19.985 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.985 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.985 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:19.985 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.343195+0000", 2024-09-11T07:40:19.985 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.97055083499999995 2024-09-11T07:40:19.985 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.985 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.986 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:19.986 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.343203+0000", 2024-09-11T07:40:19.986 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6880000000000005e-06 2024-09-11T07:40:19.986 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:19.986 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:19.986 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:19.986 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.986 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.986 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87242 9.16 9.5af626b6 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:40:19.986 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.372676+0000", 2024-09-11T07:40:19.986 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 111.554284451, 2024-09-11T07:40:19.986 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.970557585, 2024-09-11T07:40:19.986 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:19.986 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:40:19.986 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:19.986 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:19.987 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:19.987 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87242 2024-09-11T07:40:19.987 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.987 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:19.987 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.987 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:19.987 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372676+0000", 2024-09-11T07:40:19.987 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:19.987 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.987 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.987 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:19.988 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372676+0000", 2024-09-11T07:40:19.988 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:19.988 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.988 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.988 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:19.988 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372675+0000", 2024-09-11T07:40:19.988 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:40:19.988 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.988 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.988 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:19.988 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372801+0000", 2024-09-11T07:40:19.988 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000126692 2024-09-11T07:40:19.988 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.988 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.988 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:19.989 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372815+0000", 2024-09-11T07:40:19.989 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4235e-05 2024-09-11T07:40:19.989 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.989 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.989 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:19.989 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372829+0000", 2024-09-11T07:40:19.989 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3794999999999999e-05 2024-09-11T07:40:19.989 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.989 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.989 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:19.989 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372894+0000", 2024-09-11T07:40:19.989 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5038999999999999e-05 2024-09-11T07:40:19.989 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.990 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.990 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:40:19.990 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372902+0000", 2024-09-11T07:40:19.990 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5909999999999998e-06 2024-09-11T07:40:19.990 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.990 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.990 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:19.990 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.343226+0000", 2024-09-11T07:40:19.990 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.97032444799999995 2024-09-11T07:40:19.990 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.990 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.990 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:19.990 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.343234+0000", 2024-09-11T07:40:19.991 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4150000000000002e-06 2024-09-11T07:40:19.991 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:19.991 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:19.991 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:19.991 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.991 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.991 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87253 9.10 9.1d1ad730 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:40:19.991 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.378537+0000", 2024-09-11T07:40:19.992 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 111.548423511, 2024-09-11T07:40:19.992 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.96243146199999996, 2024-09-11T07:40:19.992 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:19.992 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:40:19.992 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:19.992 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:19.992 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:19.992 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87253 2024-09-11T07:40:19.992 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.992 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:19.992 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.992 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:19.992 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378537+0000", 2024-09-11T07:40:19.992 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:19.993 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.993 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.993 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:19.993 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378537+0000", 2024-09-11T07:40:19.993 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:19.993 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.993 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.993 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:19.993 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378534+0000", 2024-09-11T07:40:19.993 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999971 2024-09-11T07:40:19.993 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.993 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.993 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:19.993 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378616+0000", 2024-09-11T07:40:19.994 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1242999999999994e-05 2024-09-11T07:40:19.994 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.994 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.994 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:19.994 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378630+0000", 2024-09-11T07:40:19.994 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4463e-05 2024-09-11T07:40:19.994 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.994 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.994 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:19.994 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378637+0000", 2024-09-11T07:40:19.994 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5300000000000002e-06 2024-09-11T07:40:19.994 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.994 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.995 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:19.995 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378682+0000", 2024-09-11T07:40:19.995 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5768e-05 2024-09-11T07:40:19.995 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.995 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.995 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:40:19.995 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378688+0000", 2024-09-11T07:40:19.996 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.259e-06 2024-09-11T07:40:19.996 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.996 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.996 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:19.996 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.340958+0000", 2024-09-11T07:40:19.996 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.96227069300000001 2024-09-11T07:40:19.996 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.996 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.996 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:19.996 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.340969+0000", 2024-09-11T07:40:19.996 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0175e-05 2024-09-11T07:40:19.996 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:19.996 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:19.996 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:19.997 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.997 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.997 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87254 9.16 9.cde03e76 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:40:19.997 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.379589+0000", 2024-09-11T07:40:19.997 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 111.547371974, 2024-09-11T07:40:19.997 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.96368716799999998, 2024-09-11T07:40:19.997 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:19.997 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:40:19.997 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:19.997 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:19.997 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:19.997 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87254 2024-09-11T07:40:19.997 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:19.997 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:19.998 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:19.998 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:19.998 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.379589+0000", 2024-09-11T07:40:19.998 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:20.001 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.001 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.001 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:20.001 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.379589+0000", 2024-09-11T07:40:20.001 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:20.001 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.001 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.002 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:20.002 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.379587+0000", 2024-09-11T07:40:20.002 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:40:20.002 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.002 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.002 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:20.002 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.379677+0000", 2024-09-11T07:40:20.002 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9827000000000006e-05 2024-09-11T07:40:20.002 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.002 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.002 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:20.003 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.379690+0000", 2024-09-11T07:40:20.003 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3224000000000001e-05 2024-09-11T07:40:20.003 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.003 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.003 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:20.003 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.379696+0000", 2024-09-11T07:40:20.003 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.288e-06 2024-09-11T07:40:20.003 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.003 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.003 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:20.003 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.379728+0000", 2024-09-11T07:40:20.004 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2771000000000002e-05 2024-09-11T07:40:20.004 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.004 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.004 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:40:20.004 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.379734+0000", 2024-09-11T07:40:20.004 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2000000000000002e-06 2024-09-11T07:40:20.004 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.004 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.004 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:20.004 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.343266+0000", 2024-09-11T07:40:20.004 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.96353272199999995 2024-09-11T07:40:20.004 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.004 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.004 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:20.004 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.343276+0000", 2024-09-11T07:40:20.005 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4029999999999996e-06 2024-09-11T07:40:20.005 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:20.005 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:20.005 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:20.005 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.005 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.005 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196443 9.e 9:7b5d3f45:::benchmark_data_smithi022_133943_object69852:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:40:20.005 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.517056+0000", 2024-09-11T07:40:20.005 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 2.409904891, 2024-09-11T07:40:20.005 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14592654299999999, 2024-09-11T07:40:20.005 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:20.005 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:20.005 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:20.005 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:20.006 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:20.006 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196443 2024-09-11T07:40:20.006 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.006 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:20.006 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.006 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:20.006 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.517056+0000", 2024-09-11T07:40:20.006 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:20.006 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.006 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.006 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:20.006 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.517056+0000", 2024-09-11T07:40:20.006 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:20.006 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.007 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.007 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:20.007 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.517053+0000", 2024-09-11T07:40:20.007 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:40:20.007 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.007 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.007 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:20.007 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.517062+0000", 2024-09-11T07:40:20.007 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.371e-06 2024-09-11T07:40:20.007 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.007 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.007 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.517074+0000", 2024-09-11T07:40:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2235999999999999e-05 2024-09-11T07:40:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.517080+0000", 2024-09-11T07:40:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0580000000000003e-06 2024-09-11T07:40:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.517198+0000", 2024-09-11T07:40:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000118362 2024-09-11T07:40:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.009 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:20.009 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.517321+0000", 2024-09-11T07:40:20.009 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012251800000000001 2024-09-11T07:40:20.009 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.009 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.009 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,1,7", 2024-09-11T07:40:20.009 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.517457+0000", 2024-09-11T07:40:20.009 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000135761 2024-09-11T07:40:20.009 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.010 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.010 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:20.010 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.519778+0000", 2024-09-11T07:40:20.010 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0023211490000000002 2024-09-11T07:40:20.010 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.010 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.010 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:20.010 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.519955+0000", 2024-09-11T07:40:20.011 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017744899999999999 2024-09-11T07:40:20.011 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.011 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.011 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:20.011 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.662743+0000", 2024-09-11T07:40:20.011 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14278789 2024-09-11T07:40:20.011 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.011 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.011 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:20.011 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.662813+0000", 2024-09-11T07:40:20.011 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9832000000000005e-05 2024-09-11T07:40:20.012 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.012 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.012 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:20.012 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.662868+0000", 2024-09-11T07:40:20.012 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5009999999999997e-05 2024-09-11T07:40:20.012 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.012 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.012 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:20.012 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.662982+0000", 2024-09-11T07:40:20.013 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000114218 2024-09-11T07:40:20.013 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:20.013 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:20.013 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:20.013 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.013 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.013 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196447 9.e 9:7b61690c:::benchmark_data_smithi022_133943_object69856:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:40:20.013 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.520458+0000", 2024-09-11T07:40:20.013 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 2.406502846, 2024-09-11T07:40:20.013 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14738014299999999, 2024-09-11T07:40:20.014 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:20.014 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:20.014 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:20.014 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:20.014 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:20.014 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196447 2024-09-11T07:40:20.014 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.014 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:20.015 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.015 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:20.015 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.520458+0000", 2024-09-11T07:40:20.015 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:20.015 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.015 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.015 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:20.015 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.520458+0000", 2024-09-11T07:40:20.016 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:20.016 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.016 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.016 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:20.016 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.520456+0000", 2024-09-11T07:40:20.016 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:40:20.016 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.016 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.016 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:20.017 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.520462+0000", 2024-09-11T07:40:20.017 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6149999999999996e-06 2024-09-11T07:40:20.017 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.017 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.017 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:20.017 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.520471+0000", 2024-09-11T07:40:20.017 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6810000000000004e-06 2024-09-11T07:40:20.017 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.017 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.017 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:20.017 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.520475+0000", 2024-09-11T07:40:20.018 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7389999999999999e-06 2024-09-11T07:40:20.018 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.018 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.018 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:20.018 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.659279+0000", 2024-09-11T07:40:20.018 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13880337800000001 2024-09-11T07:40:20.018 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.018 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.018 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:20.018 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.659737+0000", 2024-09-11T07:40:20.018 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000458777 2024-09-11T07:40:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,1,7", 2024-09-11T07:40:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.659882+0000", 2024-09-11T07:40:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014431200000000001 2024-09-11T07:40:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.663028+0000", 2024-09-11T07:40:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0031457619999999999 2024-09-11T07:40:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.663321+0000", 2024-09-11T07:40:20.020 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000293171 2024-09-11T07:40:20.020 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.020 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.020 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:20.020 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.663327+0000", 2024-09-11T07:40:20.020 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7730000000000002e-06 2024-09-11T07:40:20.020 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.020 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.020 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:20.020 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.667683+0000", 2024-09-11T07:40:20.020 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0043554149999999996 2024-09-11T07:40:20.020 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.020 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.021 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:20.021 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.667732+0000", 2024-09-11T07:40:20.021 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8890000000000001e-05 2024-09-11T07:40:20.021 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.021 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.021 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:20.021 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.667838+0000", 2024-09-11T07:40:20.021 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010613 2024-09-11T07:40:20.021 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:20.021 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:20.021 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:20.021 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.022 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.022 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196451 9.14 9:2f54c23c:::benchmark_data_smithi022_133943_object69860:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:40:20.022 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.521504+0000", 2024-09-11T07:40:20.022 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 2.4054569699999999, 2024-09-11T07:40:20.022 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14478313400000001, 2024-09-11T07:40:20.022 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:20.022 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:20.022 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:20.022 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:20.023 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:20.023 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196451 2024-09-11T07:40:20.023 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.023 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:20.023 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.023 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:20.023 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.521504+0000", 2024-09-11T07:40:20.023 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:20.024 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.024 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.024 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:20.024 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.521504+0000", 2024-09-11T07:40:20.024 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:20.024 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.024 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.024 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:20.024 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.521503+0000", 2024-09-11T07:40:20.024 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:40:20.024 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.024 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.024 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:20.024 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.521507+0000", 2024-09-11T07:40:20.025 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9489999999999998e-06 2024-09-11T07:40:20.025 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.025 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.025 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:20.025 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.658345+0000", 2024-09-11T07:40:20.025 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.136838707 2024-09-11T07:40:20.025 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.025 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.025 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:20.025 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.658358+0000", 2024-09-11T07:40:20.025 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3057999999999999e-05 2024-09-11T07:40:20.025 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.025 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.025 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:20.026 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.658624+0000", 2024-09-11T07:40:20.026 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000265741 2024-09-11T07:40:20.026 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.026 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.026 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:20.026 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.658774+0000", 2024-09-11T07:40:20.026 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015002599999999999 2024-09-11T07:40:20.026 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.026 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.026 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2,3,5", 2024-09-11T07:40:20.026 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.659279+0000", 2024-09-11T07:40:20.026 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00050533700000000002 2024-09-11T07:40:20.026 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.026 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.026 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:20.027 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.662537+0000", 2024-09-11T07:40:20.027 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003257369 2024-09-11T07:40:20.027 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.027 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.027 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:20.027 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.662570+0000", 2024-09-11T07:40:20.027 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3417999999999998e-05 2024-09-11T07:40:20.027 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.027 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.027 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:20.027 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.666021+0000", 2024-09-11T07:40:20.027 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0034508479999999998 2024-09-11T07:40:20.027 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.027 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.027 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:20.028 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.666050+0000", 2024-09-11T07:40:20.028 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9124000000000001e-05 2024-09-11T07:40:20.028 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.028 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.028 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:20.028 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.666078+0000", 2024-09-11T07:40:20.028 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.8055000000000002e-05 2024-09-11T07:40:20.028 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.028 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.028 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:20.028 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.666170+0000", 2024-09-11T07:40:20.028 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.1722e-05 2024-09-11T07:40:20.028 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.029 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.029 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:20.029 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.666287+0000", 2024-09-11T07:40:20.029 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000116731 2024-09-11T07:40:20.029 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:20.029 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:20.029 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:20.029 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.029 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.029 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.15504.0:196448 9.c e299/247 9:349e6908:::benchmark_data_smithi022_133943_object69857:head v 299'12117, mlcod=299'12114)", 2024-09-11T07:40:20.029 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.521649+0000", 2024-09-11T07:40:20.029 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 2.4053111060000001, 2024-09-11T07:40:20.029 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14425960900000001, 2024-09-11T07:40:20.029 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:20.030 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:20.030 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:20.030 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.030 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:20.030 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.521649+0000", 2024-09-11T07:40:20.030 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:20.030 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.030 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.030 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:20.030 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.521649+0000", 2024-09-11T07:40:20.031 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:20.031 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.031 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.031 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:20.031 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.521649+0000", 2024-09-11T07:40:20.031 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:40:20.031 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.031 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.031 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:20.031 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.521654+0000", 2024-09-11T07:40:20.031 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5679999999999999e-06 2024-09-11T07:40:20.031 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.031 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.032 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:20.032 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.658717+0000", 2024-09-11T07:40:20.032 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13706264400000001 2024-09-11T07:40:20.032 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.032 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.032 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:20.032 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.658725+0000", 2024-09-11T07:40:20.032 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0609999999999996e-06 2024-09-11T07:40:20.032 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.032 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.032 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:20.032 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.659855+0000", 2024-09-11T07:40:20.033 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001130217 2024-09-11T07:40:20.033 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.033 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.033 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:20.033 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.659877+0000", 2024-09-11T07:40:20.033 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1682999999999999e-05 2024-09-11T07:40:20.033 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.033 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.033 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:20.033 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.665873+0000", 2024-09-11T07:40:20.033 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.005995935 2024-09-11T07:40:20.033 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.033 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.034 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:20.034 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.665909+0000", 2024-09-11T07:40:20.034 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6295999999999999e-05 2024-09-11T07:40:20.034 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:20.034 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:20.034 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:20.034 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.034 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.034 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196571 9.e 9:76985fef:::benchmark_data_smithi022_133943_object69980:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:40:20.034 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.778036+0000", 2024-09-11T07:40:20.034 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 2.148924659, 2024-09-11T07:40:20.034 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14644922099999999, 2024-09-11T07:40:20.034 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:20.035 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:20.035 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:20.035 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:20.035 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:20.035 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196571 2024-09-11T07:40:20.035 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.035 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:20.035 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.035 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:20.035 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.778036+0000", 2024-09-11T07:40:20.035 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:20.035 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.035 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.035 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:20.036 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.778036+0000", 2024-09-11T07:40:20.036 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:20.036 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.036 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.036 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:20.036 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.778033+0000", 2024-09-11T07:40:20.036 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999971 2024-09-11T07:40:20.036 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.036 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.036 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:20.036 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.778041+0000", 2024-09-11T07:40:20.036 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9540000000000004e-06 2024-09-11T07:40:20.036 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.036 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.037 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:20.037 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.778054+0000", 2024-09-11T07:40:20.037 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2507000000000001e-05 2024-09-11T07:40:20.037 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.037 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.037 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:20.037 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.778060+0000", 2024-09-11T07:40:20.037 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3049999999999998e-06 2024-09-11T07:40:20.037 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.037 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.037 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:20.037 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.778106+0000", 2024-09-11T07:40:20.037 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5747000000000001e-05 2024-09-11T07:40:20.037 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.038 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.038 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:20.038 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.778222+0000", 2024-09-11T07:40:20.038 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000116451 2024-09-11T07:40:20.038 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.038 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.038 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,1,7", 2024-09-11T07:40:20.038 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.778364+0000", 2024-09-11T07:40:20.038 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000142235 2024-09-11T07:40:20.038 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.038 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.038 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:20.038 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.780969+0000", 2024-09-11T07:40:20.038 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0026042359999999998 2024-09-11T07:40:20.039 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.039 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.039 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:20.039 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.781415+0000", 2024-09-11T07:40:20.039 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000446155 2024-09-11T07:40:20.039 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.039 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.039 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:20.039 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.783888+0000", 2024-09-11T07:40:20.039 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0024732209999999998 2024-09-11T07:40:20.039 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.039 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.039 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:20.039 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924311+0000", 2024-09-11T07:40:20.040 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14042258499999999 2024-09-11T07:40:20.040 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.040 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.040 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:20.040 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924371+0000", 2024-09-11T07:40:20.040 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0375999999999999e-05 2024-09-11T07:40:20.040 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.040 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.040 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:20.040 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924485+0000", 2024-09-11T07:40:20.040 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000114178 2024-09-11T07:40:20.040 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:20.040 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:20.041 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:20.041 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.041 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.041 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196578 9.16 9:6c2b6157:::benchmark_data_smithi022_133943_object69987:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:40:20.041 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.785362+0000", 2024-09-11T07:40:20.041 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 2.1415984200000002, 2024-09-11T07:40:20.041 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13935903199999999, 2024-09-11T07:40:20.041 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:20.041 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:20.041 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:20.041 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:20.041 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:20.041 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196578 2024-09-11T07:40:20.042 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.042 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:20.042 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.042 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:20.042 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785362+0000", 2024-09-11T07:40:20.042 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:20.042 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.042 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.042 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:20.042 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785362+0000", 2024-09-11T07:40:20.042 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:20.042 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.042 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.042 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:20.043 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785360+0000", 2024-09-11T07:40:20.043 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:40:20.043 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.043 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.043 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:20.043 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785367+0000", 2024-09-11T07:40:20.043 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1570000000000003e-06 2024-09-11T07:40:20.043 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.043 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.043 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:20.043 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785377+0000", 2024-09-11T07:40:20.043 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0148e-05 2024-09-11T07:40:20.043 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.043 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.044 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:20.044 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785382+0000", 2024-09-11T07:40:20.044 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1529999999999996e-06 2024-09-11T07:40:20.044 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.044 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.044 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:20.044 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785534+0000", 2024-09-11T07:40:20.044 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000151581 2024-09-11T07:40:20.044 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.044 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.044 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:20.044 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785684+0000", 2024-09-11T07:40:20.044 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015024400000000001 2024-09-11T07:40:20.044 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.045 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.045 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2,6", 2024-09-11T07:40:20.045 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.786069+0000", 2024-09-11T07:40:20.045 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:40:19 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:40:20.046 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00038500699999999998 2024-09-11T07:40:20.046 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.046 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.046 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:20.046 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.788778+0000", 2024-09-11T07:40:20.046 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0027084710000000001 2024-09-11T07:40:20.046 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.046 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.046 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:20.046 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.790798+0000", 2024-09-11T07:40:20.046 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002019842 2024-09-11T07:40:20.046 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.046 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.047 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:20.047 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.790850+0000", 2024-09-11T07:40:20.047 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2769000000000002e-05 2024-09-11T07:40:20.047 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.047 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.047 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:20.047 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924560+0000", 2024-09-11T07:40:20.047 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13370979399999999 2024-09-11T07:40:20.047 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.047 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.047 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:20.047 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924606+0000", 2024-09-11T07:40:20.048 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6025000000000001e-05 2024-09-11T07:40:20.048 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.048 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.048 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:20.048 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924721+0000", 2024-09-11T07:40:20.048 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011505099999999999 2024-09-11T07:40:20.048 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:20.048 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:20.048 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:20.048 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.048 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.048 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196579 9.e 9:7d796217:::benchmark_data_smithi022_133943_object69988:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:40:20.048 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.788300+0000", 2024-09-11T07:40:20.049 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 2.1386609650000001, 2024-09-11T07:40:20.049 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13661493, 2024-09-11T07:40:20.049 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:20.049 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:20.049 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:20.049 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:20.049 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:20.049 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196579 2024-09-11T07:40:20.049 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.049 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:20.049 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.049 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:20.050 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.788300+0000", 2024-09-11T07:40:20.050 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:20.050 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.050 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.050 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:20.050 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.788300+0000", 2024-09-11T07:40:20.050 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:20.050 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.050 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.050 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:20.051 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.788298+0000", 2024-09-11T07:40:20.051 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:40:20.051 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.051 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.051 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:20.051 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.788304+0000", 2024-09-11T07:40:20.051 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2899999999999999e-06 2024-09-11T07:40:20.051 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.051 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.052 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:20.052 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.788313+0000", 2024-09-11T07:40:20.052 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3950000000000007e-06 2024-09-11T07:40:20.052 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.052 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.052 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:20.052 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.788318+0000", 2024-09-11T07:40:20.052 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1300000000000003e-06 2024-09-11T07:40:20.052 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.052 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.052 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:20.052 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.788747+0000", 2024-09-11T07:40:20.052 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00042964199999999999 2024-09-11T07:40:20.053 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.053 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.053 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:20.053 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.788862+0000", 2024-09-11T07:40:20.053 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000114957 2024-09-11T07:40:20.053 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.053 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.053 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,1,7", 2024-09-11T07:40:20.053 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.788979+0000", 2024-09-11T07:40:20.053 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000116563 2024-09-11T07:40:20.053 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.053 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.053 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:20.054 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.793510+0000", 2024-09-11T07:40:20.054 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0045317789999999997 2024-09-11T07:40:20.054 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.054 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.054 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:20.054 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924144+0000", 2024-09-11T07:40:20.054 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.130633691 2024-09-11T07:40:20.054 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.054 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.054 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:20.054 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924238+0000", 2024-09-11T07:40:20.054 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4191000000000007e-05 2024-09-11T07:40:20.054 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.055 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.055 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:20.055 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924786+0000", 2024-09-11T07:40:20.055 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00054762300000000001 2024-09-11T07:40:20.055 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.055 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.055 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:20.055 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924834+0000", 2024-09-11T07:40:20.055 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7620999999999997e-05 2024-09-11T07:40:20.055 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.055 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.055 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:20.055 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924915+0000", 2024-09-11T07:40:20.055 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0952999999999996e-05 2024-09-11T07:40:20.056 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:20.056 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:20.056 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:20.056 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.056 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.056 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196583 9.e 9:7380ed4c:::benchmark_data_smithi022_133943_object69992:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:40:20.056 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.788321+0000", 2024-09-11T07:40:20.056 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 2.1386392359999999, 2024-09-11T07:40:20.056 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.137590622, 2024-09-11T07:40:20.056 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:20.056 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:20.056 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:20.056 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:20.057 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:20.057 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196583 2024-09-11T07:40:20.057 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.057 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:20.057 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.057 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:20.057 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.788321+0000", 2024-09-11T07:40:20.057 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:20.057 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.057 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.057 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:20.057 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.788321+0000", 2024-09-11T07:40:20.057 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:20.057 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.058 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.058 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:20.058 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.788321+0000", 2024-09-11T07:40:20.058 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:40:20.058 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.058 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.058 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:20.058 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.788323+0000", 2024-09-11T07:40:20.058 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.812e-06 2024-09-11T07:40:20.058 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.058 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.058 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:20.058 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.788326+0000", 2024-09-11T07:40:20.058 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.8940000000000002e-06 2024-09-11T07:40:20.058 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.059 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.059 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:20.059 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.788328+0000", 2024-09-11T07:40:20.059 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0109999999999999e-06 2024-09-11T07:40:20.059 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.059 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.059 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:20.059 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.790818+0000", 2024-09-11T07:40:20.059 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0024900600000000001 2024-09-11T07:40:20.059 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.059 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.059 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:20.059 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.790942+0000", 2024-09-11T07:40:20.059 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012420199999999999 2024-09-11T07:40:20.060 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.060 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.060 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,1,7", 2024-09-11T07:40:20.060 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.791072+0000", 2024-09-11T07:40:20.060 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013024400000000001 2024-09-11T07:40:20.060 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.060 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.060 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:20.060 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.793518+0000", 2024-09-11T07:40:20.060 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0024458470000000001 2024-09-11T07:40:20.060 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.060 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.060 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:20.061 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924525+0000", 2024-09-11T07:40:20.061 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13100734 2024-09-11T07:40:20.061 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.061 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.061 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:20.061 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924759+0000", 2024-09-11T07:40:20.061 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000233219 2024-09-11T07:40:20.061 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.061 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.061 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:20.061 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.925765+0000", 2024-09-11T07:40:20.061 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0010064060000000001 2024-09-11T07:40:20.061 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.062 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.062 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:20.062 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.925807+0000", 2024-09-11T07:40:20.062 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1474000000000003e-05 2024-09-11T07:40:20.062 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.062 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.062 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:20.062 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.925912+0000", 2024-09-11T07:40:20.062 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000105455 2024-09-11T07:40:20.062 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:20.062 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:20.062 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:20.062 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.062 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.063 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196589 9.16 9:6adb73f3:::benchmark_data_smithi022_133943_object69998:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:40:20.063 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.793862+0000", 2024-09-11T07:40:20.063 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 2.1330986649999999, 2024-09-11T07:40:20.063 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.138043053, 2024-09-11T07:40:20.063 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:20.063 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:20.063 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:20.063 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:20.063 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:20.063 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196589 2024-09-11T07:40:20.063 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.063 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:20.063 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.063 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:20.063 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.793862+0000", 2024-09-11T07:40:20.064 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:20.064 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.064 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.064 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:20.064 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.793862+0000", 2024-09-11T07:40:20.064 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:20.064 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.064 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.064 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:20.064 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.793860+0000", 2024-09-11T07:40:20.064 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:40:20.064 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.064 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.065 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:20.065 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.793867+0000", 2024-09-11T07:40:20.065 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.8850000000000001e-06 2024-09-11T07:40:20.065 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.065 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.065 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:20.065 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.793876+0000", 2024-09-11T07:40:20.065 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.6269999999999995e-06 2024-09-11T07:40:20.065 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.065 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.065 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:20.065 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.793882+0000", 2024-09-11T07:40:20.065 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5289999999999999e-06 2024-09-11T07:40:20.066 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.066 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.066 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:20.066 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.925112+0000", 2024-09-11T07:40:20.066 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.131230177 2024-09-11T07:40:20.066 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.066 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.066 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:20.066 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.925223+0000", 2024-09-11T07:40:20.066 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011082900000000001 2024-09-11T07:40:20.066 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.066 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.067 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2,6", 2024-09-11T07:40:20.067 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.925668+0000", 2024-09-11T07:40:20.067 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00044549100000000001 2024-09-11T07:40:20.067 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.067 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.067 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:20.067 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.931400+0000", 2024-09-11T07:40:20.067 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0057312220000000002 2024-09-11T07:40:20.067 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.067 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.067 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:20.067 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.931449+0000", 2024-09-11T07:40:20.067 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9753000000000003e-05 2024-09-11T07:40:20.067 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.068 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.068 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:20.068 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.931484+0000", 2024-09-11T07:40:20.068 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4764000000000001e-05 2024-09-11T07:40:20.068 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.068 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.068 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:20.068 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.931684+0000", 2024-09-11T07:40:20.068 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00020028299999999999 2024-09-11T07:40:20.068 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.068 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.068 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:20.068 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.931787+0000", 2024-09-11T07:40:20.069 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010264699999999999 2024-09-11T07:40:20.069 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:20.069 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:20.069 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:20.069 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.931905+0000", 2024-09-11T07:40:20.069 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000117892 2024-09-11T07:40:20.069 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:20.069 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:20.069 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:20.069 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:20.069 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:20.069 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:40:20.070 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:20.108 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:20.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:20.109 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:20.210 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:20.242 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:20.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:20.243 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:20.344 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:20.383 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:20.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:20.384 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:20.416 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:40:20 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:40:20] "GET /metrics HTTP/1.1" 200 224713 "" "Prometheus/2.43.0" 2024-09-11T07:40:20.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:20 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:40:20.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:20 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:40:20.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:20 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:40:20.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:20 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:40:20.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:20 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:40:20.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:20 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:40:20.485 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:20.523 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:20.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:20.524 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:20.624 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:20.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:20.667 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:20.667 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:20.745 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:20 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:40:20.745 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:20 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:40:20.745 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:20 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:40:20.768 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:20.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:20.808 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:20.808 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:20.909 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:20.940 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:20.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:20.941 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:20.983 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-11T07:40:21.042 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:21.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:21.082 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:21.082 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:21.183 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:21.218 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:21.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:21.219 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:21.320 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:21.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:21.366 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:21.366 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:21.366 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is set 2024-09-11T07:40:21.438 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:40:21.467 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:21.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:21.501 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:21.501 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:21.602 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:21.633 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:21 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:40:21.633 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:21 smithi104 ceph-mon[33507]: pgmap v626: 158 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 145 active+clean; 3.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 641 op/s; 32999/167583 objects misplaced (19.691%); 59 MiB/s, 949 objects/s recovering 2024-09-11T07:40:21.633 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:21 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1135331086' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:40:21.633 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:21 smithi104 ceph-mon[33507]: from='client.25540 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:40:21.644 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:21.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:21.645 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:21.665 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:21 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:40:21.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:21 smithi022 ceph-mon[29462]: pgmap v626: 158 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 145 active+clean; 3.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 641 op/s; 32999/167583 objects misplaced (19.691%); 59 MiB/s, 949 objects/s recovering 2024-09-11T07:40:21.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:21 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1135331086' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:40:21.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:21 smithi022 ceph-mon[29462]: from='client.25540 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:40:21.666 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:21 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:40:21.666 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:21 smithi022 ceph-mon[37330]: pgmap v626: 158 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 145 active+clean; 3.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 641 op/s; 32999/167583 objects misplaced (19.691%); 59 MiB/s, 949 objects/s recovering 2024-09-11T07:40:21.666 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:21 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1135331086' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:40:21.666 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:21 smithi022 ceph-mon[37330]: from='client.25540 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:40:21.746 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:21.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:21.778 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:21.778 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:21.879 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:21.914 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:21.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:21.915 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:22.016 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:22.045 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:22.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:22.047 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:22.147 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:22.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:22.187 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:22.187 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:22.288 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:22.320 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:22.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:22.321 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:22.422 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:22.461 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:22.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:22.462 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:22.562 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:22.598 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:22.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:22.599 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:22.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:22 smithi022 ceph-mon[37330]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:40:22.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:22 smithi022 ceph-mon[37330]: from='client.25540 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:40:22.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:22 smithi022 ceph-mon[37330]: osdmap e300: 8 total, 8 up, 8 in 2024-09-11T07:40:22.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:22 smithi022 ceph-mon[29462]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:40:22.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:22 smithi022 ceph-mon[29462]: from='client.25540 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:40:22.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:22 smithi022 ceph-mon[29462]: osdmap e300: 8 total, 8 up, 8 in 2024-09-11T07:40:22.692 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:22 smithi104 ceph-mon[33507]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:40:22.692 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:22 smithi104 ceph-mon[33507]: from='client.25540 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:40:22.692 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:22 smithi104 ceph-mon[33507]: osdmap e300: 8 total, 8 up, 8 in 2024-09-11T07:40:22.700 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:22.745 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:22.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:22.746 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:22.847 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:22.889 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:22.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:22.890 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:22.991 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:23.034 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:23.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:23.036 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:23.137 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:23.161 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:23.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:23.162 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:23.263 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:23.301 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:23.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:23.302 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:23.402 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:23.413 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:23 smithi022 ceph-mon[29462]: pgmap v628: 158 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 145 active+clean; 3.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 678 op/s; 32431/165555 objects misplaced (19.589%); 55 MiB/s, 887 objects/s recovering 2024-09-11T07:40:23.413 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:40:23.414 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:40:23.414 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:23 smithi022 ceph-mon[37330]: pgmap v628: 158 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 145 active+clean; 3.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 678 op/s; 32431/165555 objects misplaced (19.589%); 55 MiB/s, 887 objects/s recovering 2024-09-11T07:40:23.414 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:40:23.414 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:40:23.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:23.471 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:23.471 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:23.572 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:23.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:23.623 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:23.623 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:23.668 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:23 smithi104 ceph-mon[33507]: pgmap v628: 158 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 145 active+clean; 3.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 678 op/s; 32431/165555 objects misplaced (19.589%); 55 MiB/s, 887 objects/s recovering 2024-09-11T07:40:23.668 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:40:23.669 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:40:23.700 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:40:23.700 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2024-09-11T07:40:23.700 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.700 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:202397 9.d 9:b4b3aeaa:::benchmark_data_smithi022_133943_object75806:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e300)", 2024-09-11T07:40:23.700 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:23.694901+0000", 2024-09-11T07:40:23.701 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.002575538, 2024-09-11T07:40:23.701 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0026028090000000002, 2024-09-11T07:40:23.701 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:23.701 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:40:23.701 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:23.701 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:23.701 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:23.701 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 202397 2024-09-11T07:40:23.702 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.702 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:23.702 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.702 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:23.702 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694901+0000", 2024-09-11T07:40:23.702 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:23.702 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.702 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.702 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:23.702 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694901+0000", 2024-09-11T07:40:23.702 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:23.702 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.702 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.702 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:23.703 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694899+0000", 2024-09-11T07:40:23.703 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:40:23.703 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.703 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.703 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:23.703 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694904+0000", 2024-09-11T07:40:23.703 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9830000000000004e-06 2024-09-11T07:40:23.703 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.703 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.703 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:23.703 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694933+0000", 2024-09-11T07:40:23.703 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.8407000000000001e-05 2024-09-11T07:40:23.704 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.704 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.704 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:23.704 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694937+0000", 2024-09-11T07:40:23.704 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6720000000000003e-06 2024-09-11T07:40:23.704 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.704 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.704 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:23.704 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.697341+0000", 2024-09-11T07:40:23.704 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0024039880000000001 2024-09-11T07:40:23.704 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:23.704 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:23.704 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:23.705 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.705 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.705 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:202398 9.6 9:62e179e6:::benchmark_data_smithi022_133943_object75807:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e300)", 2024-09-11T07:40:23.705 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:23.694948+0000", 2024-09-11T07:40:23.705 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.002528705, 2024-09-11T07:40:23.705 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002627795, 2024-09-11T07:40:23.705 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:23.705 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2024-09-11T07:40:23.705 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:23.705 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:23.705 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:23.705 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 202398 2024-09-11T07:40:23.706 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.706 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:23.706 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.706 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:23.706 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694948+0000", 2024-09-11T07:40:23.706 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:23.706 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.706 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.706 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:23.706 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694948+0000", 2024-09-11T07:40:23.706 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:23.706 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.706 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.706 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:23.707 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694947+0000", 2024-09-11T07:40:23.707 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:40:23.707 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.707 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.707 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:23.707 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694950+0000", 2024-09-11T07:40:23.707 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.0699999999999998e-06 2024-09-11T07:40:23.707 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.707 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.707 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:23.707 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694955+0000", 2024-09-11T07:40:23.707 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7570000000000004e-06 2024-09-11T07:40:23.708 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.708 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.708 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:23.708 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694958+0000", 2024-09-11T07:40:23.708 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2550000000000002e-06 2024-09-11T07:40:23.708 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.708 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.708 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:23.708 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.695020+0000", 2024-09-11T07:40:23.708 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2003999999999998e-05 2024-09-11T07:40:23.708 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.709 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.709 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:23.709 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.695118+0000", 2024-09-11T07:40:23.709 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8222999999999998e-05 2024-09-11T07:40:23.709 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.709 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.709 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:40:23.709 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.695567+0000", 2024-09-11T07:40:23.709 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00044811200000000002 2024-09-11T07:40:23.709 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:23.709 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:23.709 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:23.709 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.710 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.710 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:202399 9.d 9.36a65ffd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e300)", 2024-09-11T07:40:23.710 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:23.694966+0000", 2024-09-11T07:40:23.710 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.0025102140000000002, 2024-09-11T07:40:23.710 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002673464, 2024-09-11T07:40:23.710 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:23.710 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "queued for pg", 2024-09-11T07:40:23.710 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:23.710 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:23.710 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:23.710 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 202399 2024-09-11T07:40:23.710 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.710 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:23.710 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.711 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:23.711 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694966+0000", 2024-09-11T07:40:23.711 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:23.711 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.711 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.711 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:23.711 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694966+0000", 2024-09-11T07:40:23.711 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:23.711 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.711 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.711 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:23.711 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694966+0000", 2024-09-11T07:40:23.711 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:40:23.712 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.712 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.712 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:23.712 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694968+0000", 2024-09-11T07:40:23.712 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.729e-06 2024-09-11T07:40:23.712 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.712 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.712 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:23.712 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694973+0000", 2024-09-11T07:40:23.712 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4449999999999998e-06 2024-09-11T07:40:23.712 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.712 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.712 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:23.712 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694976+0000", 2024-09-11T07:40:23.713 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.0350000000000002e-06 2024-09-11T07:40:23.713 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:23.713 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:23.713 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:23.713 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.713 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.713 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:202401 9.f 9:fde55345:::benchmark_data_smithi022_133943_object75810:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e300)", 2024-09-11T07:40:23.713 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:23.694979+0000", 2024-09-11T07:40:23.713 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.002497113, 2024-09-11T07:40:23.713 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00272712, 2024-09-11T07:40:23.713 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:23.713 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2024-09-11T07:40:23.713 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:23.713 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:23.714 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:23.714 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 202401 2024-09-11T07:40:23.714 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.714 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:23.714 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.714 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:23.714 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694979+0000", 2024-09-11T07:40:23.714 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:23.714 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.714 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.714 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:23.714 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694979+0000", 2024-09-11T07:40:23.714 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:23.715 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.715 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.715 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:23.715 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694979+0000", 2024-09-11T07:40:23.715 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:40:23.715 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.715 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.715 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:23.715 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694981+0000", 2024-09-11T07:40:23.715 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.4540000000000001e-06 2024-09-11T07:40:23.715 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.716 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.716 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:23.716 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694985+0000", 2024-09-11T07:40:23.716 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1679999999999997e-06 2024-09-11T07:40:23.716 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.716 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.717 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:23.717 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694989+0000", 2024-09-11T07:40:23.717 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.174e-06 2024-09-11T07:40:23.717 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.717 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.717 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:23.717 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.696090+0000", 2024-09-11T07:40:23.717 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0011016979999999999 2024-09-11T07:40:23.717 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.717 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.717 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:23.717 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.696204+0000", 2024-09-11T07:40:23.718 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011400099999999999 2024-09-11T07:40:23.718 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.718 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.718 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:40:23.718 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.696314+0000", 2024-09-11T07:40:23.718 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010939599999999999 2024-09-11T07:40:23.718 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:23.718 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:23.718 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:23.718 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.718 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.718 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:202402 9.d 9.1b3b1f5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e300)", 2024-09-11T07:40:23.718 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:23.695618+0000", 2024-09-11T07:40:23.718 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.001858221, 2024-09-11T07:40:23.719 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0021522799999999999, 2024-09-11T07:40:23.719 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:23.719 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "queued for pg", 2024-09-11T07:40:23.719 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:23.719 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:23.719 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:23.719 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 202402 2024-09-11T07:40:23.719 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.719 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:23.719 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.719 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:23.719 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.695618+0000", 2024-09-11T07:40:23.719 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:23.720 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.720 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.720 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:23.720 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.695618+0000", 2024-09-11T07:40:23.720 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:23.720 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.720 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.720 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:23.720 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.695617+0000", 2024-09-11T07:40:23.720 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:40:23.720 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.720 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.720 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:23.721 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.695622+0000", 2024-09-11T07:40:23.721 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8149999999999996e-06 2024-09-11T07:40:23.721 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.721 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.721 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:23.721 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.695629+0000", 2024-09-11T07:40:23.721 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.8750000000000002e-06 2024-09-11T07:40:23.721 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.721 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.721 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:23.721 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.695634+0000", 2024-09-11T07:40:23.721 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1429999999999997e-06 2024-09-11T07:40:23.721 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:23.722 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:23.722 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:23.722 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.722 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.722 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:202403 9.f 9.35bc5faf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e300)", 2024-09-11T07:40:23.722 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:23.695639+0000", 2024-09-11T07:40:23.722 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.001837628, 2024-09-11T07:40:23.722 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002178269, 2024-09-11T07:40:23.722 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:23.723 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "queued for pg", 2024-09-11T07:40:23.723 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:23.723 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:23.723 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:23.723 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 202403 2024-09-11T07:40:23.723 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.723 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:23.723 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.723 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:23.723 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.695639+0000", 2024-09-11T07:40:23.723 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:23.723 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.723 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.724 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:23.724 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.695639+0000", 2024-09-11T07:40:23.724 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:23.724 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.724 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.724 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:23.724 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.695638+0000", 2024-09-11T07:40:23.724 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:40:23.724 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.724 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.725 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:23.725 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.695641+0000", 2024-09-11T07:40:23.725 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.559e-06 2024-09-11T07:40:23.725 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.725 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.725 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:23.725 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.695645+0000", 2024-09-11T07:40:23.725 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3649999999999997e-06 2024-09-11T07:40:23.725 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.725 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.725 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:23.725 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.695648+0000", 2024-09-11T07:40:23.725 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2990000000000001e-06 2024-09-11T07:40:23.725 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:23.726 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:23.726 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:23.726 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.726 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.726 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:202387 9.d 9:b47b00e4:::benchmark_data_smithi022_133943_object75796:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e300)", 2024-09-11T07:40:23.726 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:23.685601+0000", 2024-09-11T07:40:23.726 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.01187529, 2024-09-11T07:40:23.726 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.012272280999999999, 2024-09-11T07:40:23.726 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:23.726 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2024-09-11T07:40:23.726 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:23.726 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:23.727 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:23.727 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 202387 2024-09-11T07:40:23.727 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.727 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:23.727 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.727 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:23.727 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.685601+0000", 2024-09-11T07:40:23.727 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:23.727 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.728 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.728 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:23.728 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.685601+0000", 2024-09-11T07:40:23.728 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:23.728 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.728 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.728 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:23.728 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.685599+0000", 2024-09-11T07:40:23.728 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:40:23.728 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.728 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.728 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:23.729 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.685606+0000", 2024-09-11T07:40:23.729 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.516e-06 2024-09-11T07:40:23.729 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.729 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.729 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:23.729 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.685616+0000", 2024-09-11T07:40:23.729 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0317e-05 2024-09-11T07:40:23.729 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.729 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.729 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:23.729 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.685621+0000", 2024-09-11T07:40:23.729 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4020000000000001e-06 2024-09-11T07:40:23.730 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.730 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.730 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:23.730 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.687095+0000", 2024-09-11T07:40:23.730 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0014732339999999999 2024-09-11T07:40:23.730 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.730 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.730 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:23.730 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.687270+0000", 2024-09-11T07:40:23.730 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017485800000000001 2024-09-11T07:40:23.730 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.730 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.730 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,4,6,7", 2024-09-11T07:40:23.730 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.687405+0000", 2024-09-11T07:40:23.731 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013555899999999999 2024-09-11T07:40:23.731 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.731 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.731 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:23.731 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694442+0000", 2024-09-11T07:40:23.731 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.007036825 2024-09-11T07:40:23.731 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.731 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.731 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:23.731 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694459+0000", 2024-09-11T07:40:23.731 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6988999999999999e-05 2024-09-11T07:40:23.731 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.731 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.732 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:23.732 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694533+0000", 2024-09-11T07:40:23.732 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3602000000000004e-05 2024-09-11T07:40:23.732 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.732 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.732 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:23.732 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.694873+0000", 2024-09-11T07:40:23.732 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000340615 2024-09-11T07:40:23.732 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:23.732 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:23.732 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:23.732 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.732 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.733 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.15504.0:202385 9.17 e300/296 9:ee15d27d:::benchmark_data_smithi022_133943_object75794:head v 300'9100, mlcod=300'9098)", 2024-09-11T07:40:23.733 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:23.689948+0000", 2024-09-11T07:40:23.733 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.0075283590000000001, 2024-09-11T07:40:23.733 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0080112499999999993, 2024-09-11T07:40:23.733 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:23.733 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2024-09-11T07:40:23.733 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:23.733 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.733 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:23.733 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.689948+0000", 2024-09-11T07:40:23.733 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:23.733 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.733 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.734 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:23.734 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.689948+0000", 2024-09-11T07:40:23.734 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:23.734 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.734 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.734 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:23.734 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.689947+0000", 2024-09-11T07:40:23.734 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:40:23.734 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.734 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.734 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:23.734 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.689955+0000", 2024-09-11T07:40:23.734 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0029999999999994e-06 2024-09-11T07:40:23.734 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.735 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.735 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:23.735 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.689967+0000", 2024-09-11T07:40:23.735 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1899000000000001e-05 2024-09-11T07:40:23.735 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.735 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.735 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:23.735 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.689973+0000", 2024-09-11T07:40:23.735 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4980000000000003e-06 2024-09-11T07:40:23.735 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.735 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.735 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:23.735 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.692380+0000", 2024-09-11T07:40:23.735 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0024064690000000001 2024-09-11T07:40:23.736 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:23.736 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:23.736 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:23.736 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:23.692415+0000", 2024-09-11T07:40:23.736 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5298e-05 2024-09-11T07:40:23.736 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:23.736 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:23.736 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:23.736 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:23.736 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2024-09-11T07:40:23.736 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 8 2024-09-11T07:40:23.736 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:40:23.737 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:23.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:23.775 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:23.776 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:23.876 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:23.910 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:23.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:23.911 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:24.012 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:24.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:24.068 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:24.069 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:24.076 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:40:24.076 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:40:24.126 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:40:24.127 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:40:24.165 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:40:24.165 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:40:24.170 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:24.205 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:40:24.205 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:40:24.218 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:24.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:24.219 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:24.245 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:40:24.246 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:40:24.288 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:40:24.288 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:40:24.310 INFO:teuthology.orchestra.run.smithi022.stdout:moved 0 / 474 (0%) 2024-09-11T07:40:24.311 INFO:teuthology.orchestra.run.smithi022.stdout:avg 59.25 2024-09-11T07:40:24.311 INFO:teuthology.orchestra.run.smithi022.stdout:stddev 7.47914 -> 7.47914 (expected baseline 7.20026) 2024-09-11T07:40:24.311 INFO:teuthology.orchestra.run.smithi022.stdout:min osd.7 with 48 -> 48 pgs (0.810127 -> 0.810127 * mean) 2024-09-11T07:40:24.311 INFO:teuthology.orchestra.run.smithi022.stdout:max osd.4 with 69 -> 69 pgs (1.16456 -> 1.16456 * mean) 2024-09-11T07:40:24.311 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:40:24.311 INFO:teuthology.orchestra.run.smithi022.stdout:oload 1000 2024-09-11T07:40:24.311 INFO:teuthology.orchestra.run.smithi022.stdout:max_change 3 2024-09-11T07:40:24.311 INFO:teuthology.orchestra.run.smithi022.stdout:max_change_osds 4 2024-09-11T07:40:24.311 INFO:teuthology.orchestra.run.smithi022.stdout:average_utilization 0.0220 2024-09-11T07:40:24.311 INFO:teuthology.orchestra.run.smithi022.stdout:overload_utilization 0.2199 2024-09-11T07:40:24.311 INFO:teuthology.orchestra.run.smithi022.stderr:no change 2024-09-11T07:40:24.320 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:24.326 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:40:24.326 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:40:24.354 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:24.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:24.355 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:24.359 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:40:24.359 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:40:24.386 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:40:24 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:40:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:40:24.457 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:24.494 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:24.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:24.495 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:24.547 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-11T07:40:24.596 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:24.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:24 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1242928662' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:40:24.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:24 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1242928662' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:40:24.679 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:24.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:24.680 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:24.776 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T07:40:24.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:24 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1242928662' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:40:24.780 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:24.863 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:24.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:24.864 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:24.965 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:25.009 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:25.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:25.010 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:25.023 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:40:25.111 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:25.134 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:25.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:25.135 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:25.236 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:25.277 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:25.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:25.278 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:25.379 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:25.421 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:25.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:25.422 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:25.477 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:25 smithi104 ceph-mon[33507]: pgmap v629: 158 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 145 active+clean; 3.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 600 op/s; 31169/163725 objects misplaced (19.037%); 36 MiB/s, 573 objects/s recovering 2024-09-11T07:40:25.477 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:25 smithi104 ceph-mon[33507]: from='client.25535 v1:172.21.15.22:0/2612777007' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:40:25.523 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:25.568 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:25.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:25.570 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:25.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:25 smithi022 ceph-mon[29462]: pgmap v629: 158 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 145 active+clean; 3.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 600 op/s; 31169/163725 objects misplaced (19.037%); 36 MiB/s, 573 objects/s recovering 2024-09-11T07:40:25.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:25 smithi022 ceph-mon[29462]: from='client.25535 v1:172.21.15.22:0/2612777007' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:40:25.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:25 smithi022 ceph-mon[37330]: pgmap v629: 158 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 145 active+clean; 3.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 600 op/s; 31169/163725 objects misplaced (19.037%); 36 MiB/s, 573 objects/s recovering 2024-09-11T07:40:25.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:25 smithi022 ceph-mon[37330]: from='client.25535 v1:172.21.15.22:0/2612777007' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:40:25.671 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:25.707 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:25.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:25.708 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:25.809 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:25.840 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:25.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:25.841 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:25.942 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:25.974 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:25.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:25.975 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:26.076 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:26.112 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:26.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:26.113 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:26.214 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:26.240 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:26.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:26.241 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:26.342 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:26.366 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:26.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:26.367 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:26.467 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:26.501 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:26.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:26.502 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:26.597 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:40:26.603 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:26.628 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:26.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:26.629 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:26.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:26 smithi022 ceph-mon[29462]: osdmap e301: 8 total, 8 up, 8 in 2024-09-11T07:40:26.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:26 smithi022 ceph-mon[37330]: osdmap e301: 8 total, 8 up, 8 in 2024-09-11T07:40:26.709 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:26 smithi104 ceph-mon[33507]: osdmap e301: 8 total, 8 up, 8 in 2024-09-11T07:40:26.730 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:26.753 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:26.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:26.754 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:26.855 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:26.878 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:26.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:26.879 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:26.977 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:40:26.977 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:40:26.980 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:27.005 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:27.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:27.006 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:27.109 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:27.131 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:27.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:27.132 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:27.233 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:27.256 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:27.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:27.257 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:27.358 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:27.380 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:27.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:27.381 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:27.482 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:27.525 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:27.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:27.526 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:27.627 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:27.641 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:27 smithi104 ceph-mon[33507]: pgmap v631: 158 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 145 active+clean; 3.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.18k op/s; 26179/150408 objects misplaced (17.405%); 33 MiB/s, 524 objects/s recovering 2024-09-11T07:40:27.641 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:27 smithi104 ceph-mon[33507]: osdmap e302: 8 total, 8 up, 8 in 2024-09-11T07:40:27.665 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:27.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:27.667 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:27.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:27 smithi022 ceph-mon[29462]: pgmap v631: 158 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 145 active+clean; 3.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.18k op/s; 26179/150408 objects misplaced (17.405%); 33 MiB/s, 524 objects/s recovering 2024-09-11T07:40:27.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:27 smithi022 ceph-mon[29462]: osdmap e302: 8 total, 8 up, 8 in 2024-09-11T07:40:27.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:27 smithi022 ceph-mon[37330]: pgmap v631: 158 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 145 active+clean; 3.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.18k op/s; 26179/150408 objects misplaced (17.405%); 33 MiB/s, 524 objects/s recovering 2024-09-11T07:40:27.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:27 smithi022 ceph-mon[37330]: osdmap e302: 8 total, 8 up, 8 in 2024-09-11T07:40:27.681 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-11T07:40:27.769 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:27.813 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:27.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:27.814 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:27.915 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:27.956 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:27.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:27.957 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:28.058 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:28.104 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:28.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:28.105 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:28.108 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:40:28.206 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:28.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:28.236 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:28.236 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:28.337 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:28.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:28.389 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:28.390 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:28.491 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:28.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:28 smithi104 ceph-mon[33507]: osdmap e303: 8 total, 8 up, 8 in 2024-09-11T07:40:28.531 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:28.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:28.533 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:28.540 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:40:28.619 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:28 smithi022 ceph-mon[37330]: osdmap e303: 8 total, 8 up, 8 in 2024-09-11T07:40:28.620 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:28 smithi022 ceph-mon[29462]: osdmap e303: 8 total, 8 up, 8 in 2024-09-11T07:40:28.633 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:28.664 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:28.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:28.665 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:28.766 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:28.787 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:28.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:28.789 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:28.890 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:28.922 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:28.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:28.924 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:28.946 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:40:29.024 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:29.066 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:29.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:29.067 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:29.168 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:29.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:29.197 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:29.197 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:29.298 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:29.321 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:29.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:29.322 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:29.423 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:29.437 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is unset 2024-09-11T07:40:29.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:29.468 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:29.468 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:29.569 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:29.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:29.607 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:29.608 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:29.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:29 smithi022 ceph-mon[29462]: pgmap v634: 158 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 145 active+clean; 3.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 842 op/s; 26179/150408 objects misplaced (17.405%); 21 MiB/s, 340 objects/s recovering 2024-09-11T07:40:29.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:29 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1135878900' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:40:29.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:29 smithi022 ceph-mon[37330]: pgmap v634: 158 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 145 active+clean; 3.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 842 op/s; 26179/150408 objects misplaced (17.405%); 21 MiB/s, 340 objects/s recovering 2024-09-11T07:40:29.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:29 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1135878900' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:40:29.709 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:29.720 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:29 smithi104 ceph-mon[33507]: pgmap v634: 158 pgs: 4 active+remapped+backfilling, 9 active+remapped+backfill_toofull, 145 active+clean; 3.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 842 op/s; 26179/150408 objects misplaced (17.405%); 21 MiB/s, 340 objects/s recovering 2024-09-11T07:40:29.720 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:29 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1135878900' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:40:29.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:29.747 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:29.748 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:29.848 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:29.891 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:29.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:29.892 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:29.993 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:30.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:30.014 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:30.015 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:30.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:40:29 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:40:30.115 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:30.168 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:30.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:30.169 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:30.270 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:30.292 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:30.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:30.293 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:30.394 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:30.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:30.434 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:30.434 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:30.535 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:30.557 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:30.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:30.558 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:30.659 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:30.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:40:30 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:40:30] "GET /metrics HTTP/1.1" 200 224772 "" "Prometheus/2.43.0" 2024-09-11T07:40:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:30 smithi022 ceph-mon[29462]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:40:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:30 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1135878900' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:40:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:30 smithi022 ceph-mon[29462]: osdmap e304: 8 total, 8 up, 8 in 2024-09-11T07:40:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:30 smithi022 ceph-mon[29462]: 4.12 scrub starts 2024-09-11T07:40:30.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:30 smithi022 ceph-mon[37330]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:40:30.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:30 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1135878900' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:40:30.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:30 smithi022 ceph-mon[37330]: osdmap e304: 8 total, 8 up, 8 in 2024-09-11T07:40:30.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:30 smithi022 ceph-mon[37330]: 4.12 scrub starts 2024-09-11T07:40:30.689 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:30.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:30.690 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:30.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:30 smithi104 ceph-mon[33507]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:40:30.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:30 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1135878900' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:40:30.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:30 smithi104 ceph-mon[33507]: osdmap e304: 8 total, 8 up, 8 in 2024-09-11T07:40:30.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:30 smithi104 ceph-mon[33507]: 4.12 scrub starts 2024-09-11T07:40:30.791 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:30.831 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:30.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:30.832 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:30.933 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:30.972 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:30.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:30.973 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:31.056 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:40:31.057 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:40:31.057 INFO:tasks.thrashosds.thrasher:Reweighting osd 7 to 0.6657611370104746 2024-09-11T07:40:31.057 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight 7 0.6657611370104746 2024-09-11T07:40:31.073 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:31.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:31.126 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:31.126 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:31.227 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:31.250 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:31.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:31.251 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:31.352 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:31.383 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:31.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:31.384 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:31.485 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:31.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:31.517 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:31.517 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:31.618 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:31.650 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:31.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:31.651 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:31.752 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:31.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:31 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:40:31.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:31 smithi104 ceph-mon[33507]: pgmap v636: 158 pgs: 3 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 148 active+clean; 2.9 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 656 op/s; 22358/140862 objects misplaced (15.872%); 27 MiB/s, 425 objects/s recovering 2024-09-11T07:40:31.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:31 smithi104 ceph-mon[33507]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 7 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:40:31.787 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:31.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:31.788 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:31.889 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:31.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:31 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:40:31.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:31 smithi022 ceph-mon[29462]: pgmap v636: 158 pgs: 3 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 148 active+clean; 2.9 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 656 op/s; 22358/140862 objects misplaced (15.872%); 27 MiB/s, 425 objects/s recovering 2024-09-11T07:40:31.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:31 smithi022 ceph-mon[29462]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 7 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:40:31.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:31 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:40:31.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:31 smithi022 ceph-mon[37330]: pgmap v636: 158 pgs: 3 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 148 active+clean; 2.9 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 656 op/s; 22358/140862 objects misplaced (15.872%); 27 MiB/s, 425 objects/s recovering 2024-09-11T07:40:31.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:31 smithi022 ceph-mon[37330]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 7 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:40:31.930 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:31.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:31.931 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:32.032 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:32.060 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:32.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:32.061 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:32.162 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:32.200 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:32.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:32.201 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:32.302 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:32.338 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:32.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:32.339 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:32.440 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:32.480 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:32.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:32.481 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:32.582 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:32.619 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:32.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:32.620 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:32.721 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:32.747 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:32.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:32.748 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:32.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:32 smithi104 ceph-mon[33507]: 5.13 scrub starts 2024-09-11T07:40:32.849 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:32.871 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:32.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:32.872 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:32.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:32 smithi022 ceph-mon[37330]: 5.13 scrub starts 2024-09-11T07:40:32.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:32 smithi022 ceph-mon[29462]: 5.13 scrub starts 2024-09-11T07:40:32.973 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:33.003 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:33.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:33.005 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:33.105 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:33.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:33.141 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:33.141 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:33.242 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:33.264 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:33.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:33.265 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:33.366 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:33.394 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:33.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:33.395 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:33.496 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:33.519 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:33.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:33.520 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:33.621 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:33.652 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:33.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:33.653 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:33.754 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:33.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:33 smithi022 ceph-mon[29462]: pgmap v637: 158 pgs: 3 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 148 active+clean; 2.8 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 624 op/s; 21902/139176 objects misplaced (15.737%); 22 MiB/s, 344 objects/s recovering 2024-09-11T07:40:33.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:33 smithi022 ceph-mon[29462]: 2.18 scrub starts 2024-09-11T07:40:33.767 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:33 smithi022 ceph-mon[37330]: pgmap v637: 158 pgs: 3 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 148 active+clean; 2.8 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 624 op/s; 21902/139176 objects misplaced (15.737%); 22 MiB/s, 344 objects/s recovering 2024-09-11T07:40:33.767 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:33 smithi022 ceph-mon[37330]: 2.18 scrub starts 2024-09-11T07:40:33.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:33 smithi104 ceph-mon[33507]: pgmap v637: 158 pgs: 3 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 148 active+clean; 2.8 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 624 op/s; 21902/139176 objects misplaced (15.737%); 22 MiB/s, 344 objects/s recovering 2024-09-11T07:40:33.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:33 smithi104 ceph-mon[33507]: 2.18 scrub starts 2024-09-11T07:40:33.795 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:33.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:33.796 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:33.897 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:33.919 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:33.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:33.920 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:34.021 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:34.043 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:34.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:34.044 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:34.145 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:34.169 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:34.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:34.170 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:34.271 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:34.293 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:34.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:34.295 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:34.396 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:34.418 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:34.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:34.420 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:34.473 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:40:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:40:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:40:34.520 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:34.548 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:34.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:34.549 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:34.649 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:34.697 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:34.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:34.698 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:34.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:34 smithi022 ceph-mon[29462]: 6.a scrub starts 2024-09-11T07:40:34.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:34 smithi022 ceph-mon[29462]: 4.4 scrub starts 2024-09-11T07:40:34.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:34 smithi022 ceph-mon[29462]: osdmap e305: 8 total, 8 up, 8 in 2024-09-11T07:40:34.737 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:34 smithi022 ceph-mon[37330]: 6.a scrub starts 2024-09-11T07:40:34.737 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:34 smithi022 ceph-mon[37330]: 4.4 scrub starts 2024-09-11T07:40:34.737 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:34 smithi022 ceph-mon[37330]: osdmap e305: 8 total, 8 up, 8 in 2024-09-11T07:40:34.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:34 smithi104 ceph-mon[33507]: 6.a scrub starts 2024-09-11T07:40:34.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:34 smithi104 ceph-mon[33507]: 4.4 scrub starts 2024-09-11T07:40:34.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:34 smithi104 ceph-mon[33507]: osdmap e305: 8 total, 8 up, 8 in 2024-09-11T07:40:34.799 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:34.841 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:34.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:34.842 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:34.943 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:34.974 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:34.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:34.975 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:35.076 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:35.123 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:40:35.127 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:35.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:35.128 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:35.156 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:40:35.156 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:40:35.188 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:40:35.188 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:40:35.218 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:40:35.218 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:40:35.229 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:35.249 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:40:35.249 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:40:35.265 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:35.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:35.266 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:35.285 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:40:35.286 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:40:35.317 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:40:35.317 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:40:35.354 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:40:35.354 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:40:35.366 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:35.387 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:40:35.388 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:40:35.393 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:35.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:35.394 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:35.495 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:35.533 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:35.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:35.534 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:35.635 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:35.655 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:35 smithi104 ceph-mon[33507]: pgmap v639: 158 pgs: 1 peering, 2 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 148 active+clean; 2.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 747 B/s rd, 0 B/s wr, 616 op/s; 20015/137745 objects misplaced (14.530%); 33 MiB/s, 526 objects/s recovering 2024-09-11T07:40:35.656 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:35 smithi104 ceph-mon[33507]: osdmap e306: 8 total, 8 up, 8 in 2024-09-11T07:40:35.656 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:35 smithi104 ceph-mon[33507]: 2.19 scrub starts 2024-09-11T07:40:35.656 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:35 smithi104 ceph-mon[33507]: 2.19 scrub ok 2024-09-11T07:40:35.656 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:35 smithi104 ceph-mon[33507]: 4.6 scrub starts 2024-09-11T07:40:35.656 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:35 smithi104 ceph-mon[33507]: 4.6 scrub ok 2024-09-11T07:40:35.656 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:35 smithi104 ceph-mon[33507]: 4.8 scrub starts 2024-09-11T07:40:35.656 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:35 smithi104 ceph-mon[33507]: 4.8 scrub ok 2024-09-11T07:40:35.656 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:35 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2858232790' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:40:35.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:35 smithi022 ceph-mon[29462]: pgmap v639: 158 pgs: 1 peering, 2 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 148 active+clean; 2.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 747 B/s rd, 0 B/s wr, 616 op/s; 20015/137745 objects misplaced (14.530%); 33 MiB/s, 526 objects/s recovering 2024-09-11T07:40:35.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:35 smithi022 ceph-mon[29462]: osdmap e306: 8 total, 8 up, 8 in 2024-09-11T07:40:35.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:35 smithi022 ceph-mon[29462]: 2.19 scrub starts 2024-09-11T07:40:35.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:35 smithi022 ceph-mon[29462]: 2.19 scrub ok 2024-09-11T07:40:35.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:35 smithi022 ceph-mon[29462]: 4.6 scrub starts 2024-09-11T07:40:35.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:35 smithi022 ceph-mon[29462]: 4.6 scrub ok 2024-09-11T07:40:35.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:35 smithi022 ceph-mon[29462]: 4.8 scrub starts 2024-09-11T07:40:35.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:35 smithi022 ceph-mon[29462]: 4.8 scrub ok 2024-09-11T07:40:35.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:35 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2858232790' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:40:35.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:35 smithi022 ceph-mon[37330]: pgmap v639: 158 pgs: 1 peering, 2 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 148 active+clean; 2.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 747 B/s rd, 0 B/s wr, 616 op/s; 20015/137745 objects misplaced (14.530%); 33 MiB/s, 526 objects/s recovering 2024-09-11T07:40:35.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:35 smithi022 ceph-mon[37330]: osdmap e306: 8 total, 8 up, 8 in 2024-09-11T07:40:35.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:35 smithi022 ceph-mon[37330]: 2.19 scrub starts 2024-09-11T07:40:35.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:35 smithi022 ceph-mon[37330]: 2.19 scrub ok 2024-09-11T07:40:35.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:35 smithi022 ceph-mon[37330]: 4.6 scrub starts 2024-09-11T07:40:35.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:35 smithi022 ceph-mon[37330]: 4.6 scrub ok 2024-09-11T07:40:35.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:35 smithi022 ceph-mon[37330]: 4.8 scrub starts 2024-09-11T07:40:35.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:35 smithi022 ceph-mon[37330]: 4.8 scrub ok 2024-09-11T07:40:35.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:35 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2858232790' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:40:35.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:35.669 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:35.670 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:35.770 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:35.794 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T07:40:35.802 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:35.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:35.803 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:35.905 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:35.932 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:35.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:35.933 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:36.034 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:36.035 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:40:36.035 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2024-09-11T07:40:36.036 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2024-09-11T07:40:36.036 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2024-09-11T07:40:36.036 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.036 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:85934 9.f 9:fdbe5b3c:::benchmark_data_smithi022_133943_object85933:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e258)", 2024-09-11T07:40:36.036 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:26.311544+0000", 2024-09-11T07:40:36.036 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 129.72051293300001, 2024-09-11T07:40:36.036 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.30120598399999998, 2024-09-11T07:40:36.036 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:36.036 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:36.036 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:36.036 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:36.036 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:36.037 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 85934 2024-09-11T07:40:36.037 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.037 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:36.037 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.037 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:36.037 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.311544+0000", 2024-09-11T07:40:36.037 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.037 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.037 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.037 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:36.037 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.311544+0000", 2024-09-11T07:40:36.037 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.037 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.037 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.038 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:36.038 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.311542+0000", 2024-09-11T07:40:36.038 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:40:36.038 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.038 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.038 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:36.038 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.311634+0000", 2024-09-11T07:40:36.038 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.1664999999999997e-05 2024-09-11T07:40:36.038 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.038 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.038 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:36.038 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.311653+0000", 2024-09-11T07:40:36.038 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8987e-05 2024-09-11T07:40:36.039 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.039 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.039 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:36.039 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.311665+0000", 2024-09-11T07:40:36.039 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1657e-05 2024-09-11T07:40:36.039 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.039 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.039 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:36.039 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.317571+0000", 2024-09-11T07:40:36.039 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0059061249999999999 2024-09-11T07:40:36.039 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.039 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.039 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:36.039 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.317705+0000", 2024-09-11T07:40:36.040 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013474 2024-09-11T07:40:36.040 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.040 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.040 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:40:36.040 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.317854+0000", 2024-09-11T07:40:36.040 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000148627 2024-09-11T07:40:36.040 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.040 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.040 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.040 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.325240+0000", 2024-09-11T07:40:36.040 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0073861480000000004 2024-09-11T07:40:36.040 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.040 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.041 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:36.041 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.325252+0000", 2024-09-11T07:40:36.041 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1678000000000001e-05 2024-09-11T07:40:36.041 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.041 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.041 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.041 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.612633+0000", 2024-09-11T07:40:36.041 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.28738107099999999 2024-09-11T07:40:36.041 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.041 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.041 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:36.041 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.612702+0000", 2024-09-11T07:40:36.041 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9142999999999999e-05 2024-09-11T07:40:36.041 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.042 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.042 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:36.042 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.612750+0000", 2024-09-11T07:40:36.042 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7559000000000003e-05 2024-09-11T07:40:36.042 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.042 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:36.042 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.042 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.042 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.042 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:85942 9.f 9:fc70c519:::benchmark_data_smithi022_133943_object85941:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e258)", 2024-09-11T07:40:36.043 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:26.316250+0000", 2024-09-11T07:40:36.043 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 129.71580630299999, 2024-09-11T07:40:36.043 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.29663829400000002, 2024-09-11T07:40:36.043 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:36.043 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:36.043 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:36.043 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:36.043 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:36.043 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 85942 2024-09-11T07:40:36.043 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.043 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:36.043 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.043 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:36.043 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.316250+0000", 2024-09-11T07:40:36.044 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.044 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.044 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.044 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:36.044 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.316250+0000", 2024-09-11T07:40:36.044 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.044 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.044 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.044 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:36.044 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.316249+0000", 2024-09-11T07:40:36.044 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:40:36.044 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.044 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.045 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:36.045 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.316331+0000", 2024-09-11T07:40:36.045 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2365000000000002e-05 2024-09-11T07:40:36.045 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.045 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.045 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:36.045 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.316343+0000", 2024-09-11T07:40:36.045 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1596e-05 2024-09-11T07:40:36.045 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.045 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.045 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:36.045 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.316348+0000", 2024-09-11T07:40:36.045 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6470000000000003e-06 2024-09-11T07:40:36.045 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.046 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.046 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:36.046 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.322428+0000", 2024-09-11T07:40:36.046 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0060797029999999997 2024-09-11T07:40:36.046 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.046 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.046 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:36.046 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.322544+0000", 2024-09-11T07:40:36.046 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000116296 2024-09-11T07:40:36.046 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.046 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.046 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:40:36.046 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.322682+0000", 2024-09-11T07:40:36.046 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013749500000000001 2024-09-11T07:40:36.047 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.047 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.047 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:36.047 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.325339+0000", 2024-09-11T07:40:36.047 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0026568189999999999 2024-09-11T07:40:36.047 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.047 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.047 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.047 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.328284+0000", 2024-09-11T07:40:36.047 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0029453029999999998 2024-09-11T07:40:36.047 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.047 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.047 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.047 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.612802+0000", 2024-09-11T07:40:36.048 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.28451834599999998 2024-09-11T07:40:36.048 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.048 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.048 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:36.048 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.612852+0000", 2024-09-11T07:40:36.048 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0139999999999998e-05 2024-09-11T07:40:36.048 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.048 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.048 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:36.048 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.612889+0000", 2024-09-11T07:40:36.048 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6168999999999999e-05 2024-09-11T07:40:36.048 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.048 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:36.049 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.049 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.049 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.049 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:85953 9.f 9:fc848a32:::benchmark_data_smithi022_133943_object85952:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e258)", 2024-09-11T07:40:36.049 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:26.331015+0000", 2024-09-11T07:40:36.049 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 129.70104154800001, 2024-09-11T07:40:36.049 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.28199429199999998, 2024-09-11T07:40:36.049 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:36.049 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:36.049 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:36.049 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:36.049 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:36.049 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 85953 2024-09-11T07:40:36.050 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.050 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:36.050 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.050 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:36.050 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.331015+0000", 2024-09-11T07:40:36.050 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.050 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.050 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.050 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:36.050 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.331015+0000", 2024-09-11T07:40:36.050 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.050 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.050 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.051 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:36.051 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.331013+0000", 2024-09-11T07:40:36.051 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:40:36.051 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.051 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.051 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:36.051 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.331046+0000", 2024-09-11T07:40:36.051 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3189999999999999e-05 2024-09-11T07:40:36.051 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.051 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.051 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:36.051 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.331057+0000", 2024-09-11T07:40:36.051 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0703999999999999e-05 2024-09-11T07:40:36.051 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.052 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.052 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:36.052 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.331062+0000", 2024-09-11T07:40:36.052 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5439999999999998e-06 2024-09-11T07:40:36.052 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.052 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.052 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:36.052 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.332439+0000", 2024-09-11T07:40:36.052 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0013765870000000001 2024-09-11T07:40:36.052 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.052 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.052 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:36.052 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.332578+0000", 2024-09-11T07:40:36.053 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013949000000000001 2024-09-11T07:40:36.053 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.053 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.053 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:40:36.053 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.332730+0000", 2024-09-11T07:40:36.053 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015161100000000001 2024-09-11T07:40:36.053 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.053 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.053 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.053 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.454340+0000", 2024-09-11T07:40:36.053 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.12160982100000001 2024-09-11T07:40:36.053 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.053 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.054 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:36.054 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.454786+0000", 2024-09-11T07:40:36.054 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00044578 2024-09-11T07:40:36.054 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.054 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.054 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.054 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.612928+0000", 2024-09-11T07:40:36.054 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.15814265899999999 2024-09-11T07:40:36.054 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.054 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.054 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:36.054 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.612973+0000", 2024-09-11T07:40:36.054 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4904999999999998e-05 2024-09-11T07:40:36.055 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.055 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.055 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:36.055 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.613009+0000", 2024-09-11T07:40:36.055 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6177000000000003e-05 2024-09-11T07:40:36.055 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.055 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:36.055 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.055 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.055 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.055 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:85932 9.7 9:e6d503e6:::benchmark_data_smithi022_133943_object85931:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e259)", 2024-09-11T07:40:36.055 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:26.333730+0000", 2024-09-11T07:40:36.055 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 129.69832674400001, 2024-09-11T07:40:36.055 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.025048288, 2024-09-11T07:40:36.056 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:36.056 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:40:36.056 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:36.056 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:36.056 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:36.056 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 85932 2024-09-11T07:40:36.056 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.056 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:36.056 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.056 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:36.056 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333730+0000", 2024-09-11T07:40:36.056 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.057 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.057 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.057 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:36.057 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333730+0000", 2024-09-11T07:40:36.057 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.057 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.057 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.057 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:36.057 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333727+0000", 2024-09-11T07:40:36.057 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999967 2024-09-11T07:40:36.057 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.057 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.057 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:36.057 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333773+0000", 2024-09-11T07:40:36.058 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6940999999999997e-05 2024-09-11T07:40:36.058 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.058 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.058 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:36.058 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333787+0000", 2024-09-11T07:40:36.058 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3107e-05 2024-09-11T07:40:36.058 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.058 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.058 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:36.058 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333794+0000", 2024-09-11T07:40:36.058 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4059999999999999e-06 2024-09-11T07:40:36.058 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.058 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.058 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:36.059 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.454708+0000", 2024-09-11T07:40:36.059 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.120914359 2024-09-11T07:40:36.059 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.059 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.059 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:40:36.059 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.454717+0000", 2024-09-11T07:40:36.059 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2830000000000001e-06 2024-09-11T07:40:36.059 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.059 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.059 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:36.059 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.358693+0000", 2024-09-11T07:40:36.059 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.90397602300000002 2024-09-11T07:40:36.059 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.059 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.060 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:36.060 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.358778+0000", 2024-09-11T07:40:36.060 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5457999999999994e-05 2024-09-11T07:40:36.060 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.060 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:36.060 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.060 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.060 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.060 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:85948 9.6 9:6409403d:::benchmark_data_smithi022_133943_object85947:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e259)", 2024-09-11T07:40:36.060 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:26.333803+0000", 2024-09-11T07:40:36.060 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 129.698253924, 2024-09-11T07:40:36.060 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0201702269999999, 2024-09-11T07:40:36.060 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:36.061 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:40:36.061 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:36.061 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:36.061 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:36.061 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 85948 2024-09-11T07:40:36.061 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.061 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:36.061 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.061 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:36.061 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333803+0000", 2024-09-11T07:40:36.061 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.061 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.061 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.061 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:36.062 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333803+0000", 2024-09-11T07:40:36.062 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.062 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.062 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.062 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:36.062 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333802+0000", 2024-09-11T07:40:36.062 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:40:36.062 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.062 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.062 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:36.062 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333842+0000", 2024-09-11T07:40:36.062 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0456e-05 2024-09-11T07:40:36.062 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.062 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.063 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:36.063 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333849+0000", 2024-09-11T07:40:36.063 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.511e-06 2024-09-11T07:40:36.063 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.063 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.063 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:36.063 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333853+0000", 2024-09-11T07:40:36.063 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.963e-06 2024-09-11T07:40:36.063 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.063 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.063 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:36.063 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.469490+0000", 2024-09-11T07:40:36.063 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13563673200000001 2024-09-11T07:40:36.063 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.065 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.065 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:40:36.065 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.469495+0000", 2024-09-11T07:40:36.065 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3410000000000002e-06 2024-09-11T07:40:36.065 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.065 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.065 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:36.065 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.353902+0000", 2024-09-11T07:40:36.066 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.88440743099999997 2024-09-11T07:40:36.066 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.066 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.066 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:36.066 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.353973+0000", 2024-09-11T07:40:36.066 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0479000000000001e-05 2024-09-11T07:40:36.066 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.066 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:36.066 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.066 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.066 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.066 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:86232 9.6 9:6062b21b:::benchmark_data_smithi022_133943_object86231:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:40:36.067 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:27.532402+0000", 2024-09-11T07:40:36.067 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 128.49965417199999, 2024-09-11T07:40:36.067 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.180620107, 2024-09-11T07:40:36.067 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:36.067 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:36.067 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:36.067 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:36.067 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:36.067 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 86232 2024-09-11T07:40:36.067 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.068 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:36.068 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.068 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:36.068 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532402+0000", 2024-09-11T07:40:36.068 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.068 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.068 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.068 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:36.068 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532402+0000", 2024-09-11T07:40:36.068 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.068 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.068 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.068 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:36.068 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532400+0000", 2024-09-11T07:40:36.069 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:40:36.069 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.069 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.069 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:36.069 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532456+0000", 2024-09-11T07:40:36.069 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5791000000000002e-05 2024-09-11T07:40:36.069 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.069 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.069 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:36.069 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532470+0000", 2024-09-11T07:40:36.069 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4280999999999999e-05 2024-09-11T07:40:36.069 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.069 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.070 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:36.070 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532477+0000", 2024-09-11T07:40:36.070 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3350000000000003e-06 2024-09-11T07:40:36.070 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.070 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.070 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:36.070 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532524+0000", 2024-09-11T07:40:36.070 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7128e-05 2024-09-11T07:40:36.070 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.070 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.070 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:36.070 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532627+0000", 2024-09-11T07:40:36.070 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010340099999999999 2024-09-11T07:40:36.070 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.071 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.071 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:40:36.071 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532919+0000", 2024-09-11T07:40:36.071 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00029145000000000001 2024-09-11T07:40:36.071 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.071 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.071 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:36.071 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.534532+0000", 2024-09-11T07:40:36.071 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0016128519999999999 2024-09-11T07:40:36.071 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.071 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.071 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.071 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.537151+0000", 2024-09-11T07:40:36.072 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0026190900000000001 2024-09-11T07:40:36.072 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.072 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.072 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.072 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.712780+0000", 2024-09-11T07:40:36.072 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.175629435 2024-09-11T07:40:36.072 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.072 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.072 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:36.072 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.712894+0000", 2024-09-11T07:40:36.072 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000113725 2024-09-11T07:40:36.073 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.073 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.073 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:36.073 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.713022+0000", 2024-09-11T07:40:36.073 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012873499999999999 2024-09-11T07:40:36.073 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.073 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:36.073 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.073 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.074 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.074 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:86234 9.6 9:6677583e:::benchmark_data_smithi022_133943_object86233:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:40:36.074 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:27.532659+0000", 2024-09-11T07:40:36.074 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 128.49939789199999, 2024-09-11T07:40:36.074 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18307527700000001, 2024-09-11T07:40:36.074 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:36.074 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:36.074 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:36.074 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:36.074 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:36.074 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 86234 2024-09-11T07:40:36.074 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.074 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:36.075 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.075 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:36.075 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532659+0000", 2024-09-11T07:40:36.075 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.075 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.075 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.075 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:36.075 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532659+0000", 2024-09-11T07:40:36.075 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.075 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.075 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.075 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:36.075 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532657+0000", 2024-09-11T07:40:36.075 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:40:36.076 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.076 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.076 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:36.076 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532714+0000", 2024-09-11T07:40:36.076 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6298999999999997e-05 2024-09-11T07:40:36.076 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.076 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.076 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:36.076 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532725+0000", 2024-09-11T07:40:36.076 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1216000000000001e-05 2024-09-11T07:40:36.076 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.076 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.076 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:36.076 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532730+0000", 2024-09-11T07:40:36.077 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5849999999999999e-06 2024-09-11T07:40:36.077 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.077 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.077 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:36.077 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.535694+0000", 2024-09-11T07:40:36.077 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0029634069999999999 2024-09-11T07:40:36.077 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.077 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.077 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:36.077 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.535830+0000", 2024-09-11T07:40:36.077 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013649300000000001 2024-09-11T07:40:36.077 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.077 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.078 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:40:36.078 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536159+0000", 2024-09-11T07:40:36.078 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00032905400000000001 2024-09-11T07:40:36.078 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.078 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.078 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:36.078 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.537278+0000", 2024-09-11T07:40:36.078 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0011182830000000001 2024-09-11T07:40:36.078 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.078 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.078 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.078 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.537813+0000", 2024-09-11T07:40:36.078 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00053518700000000001 2024-09-11T07:40:36.078 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.079 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.079 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.079 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.715656+0000", 2024-09-11T07:40:36.079 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17784266000000001 2024-09-11T07:40:36.079 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.079 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.079 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:36.079 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.715700+0000", 2024-09-11T07:40:36.079 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4508000000000001e-05 2024-09-11T07:40:36.079 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.079 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.079 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:36.079 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.715734+0000", 2024-09-11T07:40:36.079 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3877000000000001e-05 2024-09-11T07:40:36.080 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.080 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:36.080 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.080 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.080 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.080 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:86244 9.a 9:5743f4ed:::benchmark_data_smithi022_133943_object86243:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:40:36.080 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:27.540150+0000", 2024-09-11T07:40:36.080 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 128.491906984, 2024-09-11T07:40:36.080 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.177488494, 2024-09-11T07:40:36.080 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:36.080 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:36.080 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:36.080 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:36.081 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:36.081 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 86244 2024-09-11T07:40:36.081 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.081 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:36.081 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.081 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:36.081 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540150+0000", 2024-09-11T07:40:36.081 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.081 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.081 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.081 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:36.081 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540150+0000", 2024-09-11T07:40:36.081 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.081 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.082 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.082 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:36.082 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540148+0000", 2024-09-11T07:40:36.082 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:40:36.082 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.082 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.082 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:36.082 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540199+0000", 2024-09-11T07:40:36.082 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1332999999999998e-05 2024-09-11T07:40:36.082 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.082 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.082 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:36.082 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540210+0000", 2024-09-11T07:40:36.082 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1323e-05 2024-09-11T07:40:36.083 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.083 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.083 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:36.083 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540215+0000", 2024-09-11T07:40:36.083 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6280000000000004e-06 2024-09-11T07:40:36.083 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.083 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.083 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:36.083 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540253+0000", 2024-09-11T07:40:36.083 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8096000000000003e-05 2024-09-11T07:40:36.083 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.083 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.083 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:36.084 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540335+0000", 2024-09-11T07:40:36.084 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1703e-05 2024-09-11T07:40:36.084 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.084 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.084 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,7", 2024-09-11T07:40:36.084 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540427+0000", 2024-09-11T07:40:36.084 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2069000000000002e-05 2024-09-11T07:40:36.084 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.084 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.084 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:36.084 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.542279+0000", 2024-09-11T07:40:36.084 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001852245 2024-09-11T07:40:36.084 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.085 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.085 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.085 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.542406+0000", 2024-09-11T07:40:36.085 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012750700000000001 2024-09-11T07:40:36.085 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.085 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.086 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.086 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.717564+0000", 2024-09-11T07:40:36.086 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17515773600000001 2024-09-11T07:40:36.086 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.086 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.086 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:36.086 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.717600+0000", 2024-09-11T07:40:36.086 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5410000000000001e-05 2024-09-11T07:40:36.087 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.087 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.087 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:36.087 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.717638+0000", 2024-09-11T07:40:36.087 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8473999999999998e-05 2024-09-11T07:40:36.087 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.087 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:36.087 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.087 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.087 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.087 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87252 9.12 9.5bd7f852 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:40:36.087 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.378195+0000", 2024-09-11T07:40:36.087 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 127.653861436, 2024-09-11T07:40:36.088 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.96275559600000005, 2024-09-11T07:40:36.088 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:36.088 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:40:36.088 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:36.088 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:36.088 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:36.088 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87252 2024-09-11T07:40:36.088 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.088 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:36.088 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.088 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:36.088 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378195+0000", 2024-09-11T07:40:36.088 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.089 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.089 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.089 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:36.089 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378195+0000", 2024-09-11T07:40:36.089 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.089 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.089 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.089 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:36.089 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378193+0000", 2024-09-11T07:40:36.089 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:40:36.089 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.089 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.089 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:36.090 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378254+0000", 2024-09-11T07:40:36.090 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0699999999999998e-05 2024-09-11T07:40:36.090 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.090 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.090 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:36.090 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378263+0000", 2024-09-11T07:40:36.091 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7579999999999998e-06 2024-09-11T07:40:36.091 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.091 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.092 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:36.092 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378268+0000", 2024-09-11T07:40:36.092 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9830000000000004e-06 2024-09-11T07:40:36.092 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.092 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.092 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:36.092 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378303+0000", 2024-09-11T07:40:36.092 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5503999999999998e-05 2024-09-11T07:40:36.093 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.093 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.093 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:40:36.093 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378308+0000", 2024-09-11T07:40:36.093 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.33e-06 2024-09-11T07:40:36.093 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.093 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.093 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:36.093 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.340939+0000", 2024-09-11T07:40:36.093 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.96263122800000001 2024-09-11T07:40:36.093 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.093 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.093 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:36.094 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.340951+0000", 2024-09-11T07:40:36.094 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1800999999999999e-05 2024-09-11T07:40:36.094 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.094 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:36.094 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.094 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.094 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.094 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196432 9.d 9:b43c5e98:::benchmark_data_smithi022_133943_object69841:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:40:36.094 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.509114+0000", 2024-09-11T07:40:36.094 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 18.522942208, 2024-09-11T07:40:36.094 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.152507894, 2024-09-11T07:40:36.094 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:36.094 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:36.095 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:36.095 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:36.095 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:36.095 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196432 2024-09-11T07:40:36.095 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.095 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:36.095 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.095 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:36.095 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509114+0000", 2024-09-11T07:40:36.095 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.095 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.095 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.095 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:36.095 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509114+0000", 2024-09-11T07:40:36.095 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.096 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.096 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.096 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:36.096 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509112+0000", 2024-09-11T07:40:36.096 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:40:36.096 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.096 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.096 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:36.096 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509119+0000", 2024-09-11T07:40:36.096 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3439999999999997e-06 2024-09-11T07:40:36.096 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.096 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.096 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:36.096 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509130+0000", 2024-09-11T07:40:36.097 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1066999999999999e-05 2024-09-11T07:40:36.097 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.097 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.097 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:36.097 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509136+0000", 2024-09-11T07:40:36.097 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0460000000000002e-06 2024-09-11T07:40:36.097 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.097 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.097 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:36.097 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509237+0000", 2024-09-11T07:40:36.097 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000101338 2024-09-11T07:40:36.098 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.098 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.098 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:36.098 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509361+0000", 2024-09-11T07:40:36.098 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000123287 2024-09-11T07:40:36.098 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.098 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.098 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,4,6,7", 2024-09-11T07:40:36.098 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509470+0000", 2024-09-11T07:40:36.098 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000109986 2024-09-11T07:40:36.098 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.098 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.098 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.099 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.515270+0000", 2024-09-11T07:40:36.099 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0057995310000000001 2024-09-11T07:40:36.099 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.099 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.099 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.099 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.515306+0000", 2024-09-11T07:40:36.099 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5920999999999997e-05 2024-09-11T07:40:36.099 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.099 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.099 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:36.099 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.515314+0000", 2024-09-11T07:40:36.099 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7680000000000008e-06 2024-09-11T07:40:36.099 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.099 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.099 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.100 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.659682+0000", 2024-09-11T07:40:36.100 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14436806299999999 2024-09-11T07:40:36.100 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.100 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.100 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.100 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.661408+0000", 2024-09-11T07:40:36.100 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0017266499999999999 2024-09-11T07:40:36.100 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.100 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.100 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:36.100 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.661490+0000", 2024-09-11T07:40:36.100 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1951000000000002e-05 2024-09-11T07:40:36.100 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.100 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.100 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:36.101 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.661622+0000", 2024-09-11T07:40:36.101 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013185900000000001 2024-09-11T07:40:36.101 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.101 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:36.101 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.101 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.101 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.101 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196436 9.f 9:f28fa10b:::benchmark_data_smithi022_133943_object69845:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:40:36.101 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.509149+0000", 2024-09-11T07:40:36.101 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 18.522908025, 2024-09-11T07:40:36.101 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.15385757, 2024-09-11T07:40:36.101 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:36.101 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:36.102 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:36.102 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:36.102 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:36.102 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196436 2024-09-11T07:40:36.102 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.102 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:36.102 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.102 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:36.102 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509149+0000", 2024-09-11T07:40:36.102 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.102 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.102 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.102 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:36.102 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509149+0000", 2024-09-11T07:40:36.103 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.103 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.103 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.103 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:36.103 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509148+0000", 2024-09-11T07:40:36.103 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:40:36.103 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.103 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.103 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:36.103 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509152+0000", 2024-09-11T07:40:36.103 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3309999999999997e-06 2024-09-11T07:40:36.103 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.103 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.104 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:36.104 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509158+0000", 2024-09-11T07:40:36.104 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0090000000000004e-06 2024-09-11T07:40:36.104 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.104 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.104 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:36.104 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509162+0000", 2024-09-11T07:40:36.104 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9149999999999998e-06 2024-09-11T07:40:36.104 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.104 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.104 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:36.104 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509226+0000", 2024-09-11T07:40:36.104 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3969999999999999e-05 2024-09-11T07:40:36.105 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.105 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.105 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:36.105 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509327+0000", 2024-09-11T07:40:36.105 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000101217 2024-09-11T07:40:36.105 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.105 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.105 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:40:36.105 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509426+0000", 2024-09-11T07:40:36.105 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8525999999999998e-05 2024-09-11T07:40:36.105 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.105 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.105 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:36.106 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.512876+0000", 2024-09-11T07:40:36.106 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003449939 2024-09-11T07:40:36.106 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.106 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.106 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.106 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.658382+0000", 2024-09-11T07:40:36.106 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.145506042 2024-09-11T07:40:36.106 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.106 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.106 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.106 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.662896+0000", 2024-09-11T07:40:36.106 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0045144510000000001 2024-09-11T07:40:36.106 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.106 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.107 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:36.107 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.662971+0000", 2024-09-11T07:40:36.107 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4895999999999996e-05 2024-09-11T07:40:36.107 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.107 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.107 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:36.107 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.663006+0000", 2024-09-11T07:40:36.107 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5099999999999999e-05 2024-09-11T07:40:36.107 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.107 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:36.107 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.107 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.107 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.107 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196438 9.7 9:e639c272:::benchmark_data_smithi022_133943_object69847:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:40:36.108 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.509172+0000", 2024-09-11T07:40:36.108 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 18.522884875999999, 2024-09-11T07:40:36.108 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.154085528, 2024-09-11T07:40:36.108 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:36.108 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:36.108 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:36.108 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:36.108 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:36.108 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196438 2024-09-11T07:40:36.108 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.108 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:36.108 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.108 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:36.109 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509172+0000", 2024-09-11T07:40:36.109 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.109 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.109 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.109 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:36.109 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509172+0000", 2024-09-11T07:40:36.109 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.109 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.109 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.109 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:36.109 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509171+0000", 2024-09-11T07:40:36.109 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:40:36.110 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.110 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.110 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:36.110 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509174+0000", 2024-09-11T07:40:36.110 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2679999999999999e-06 2024-09-11T07:40:36.110 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.110 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.110 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:36.110 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509179+0000", 2024-09-11T07:40:36.110 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9119999999999997e-06 2024-09-11T07:40:36.110 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.110 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.110 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:36.111 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509183+0000", 2024-09-11T07:40:36.111 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4410000000000002e-06 2024-09-11T07:40:36.111 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.111 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.111 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:36.111 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.510817+0000", 2024-09-11T07:40:36.111 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0016344020000000001 2024-09-11T07:40:36.111 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.111 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.111 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:36.111 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.510945+0000", 2024-09-11T07:40:36.111 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012800999999999999 2024-09-11T07:40:36.111 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.112 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.112 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2,3,7", 2024-09-11T07:40:36.112 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.511375+0000", 2024-09-11T07:40:36.112 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00042995300000000001 2024-09-11T07:40:36.112 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.112 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.112 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:36.112 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.659619+0000", 2024-09-11T07:40:36.112 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.148244129 2024-09-11T07:40:36.112 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.112 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.112 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.112 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.659674+0000", 2024-09-11T07:40:36.112 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4605000000000003e-05 2024-09-11T07:40:36.113 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.113 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.113 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.113 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.659715+0000", 2024-09-11T07:40:36.113 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0935e-05 2024-09-11T07:40:36.113 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.113 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.113 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.113 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.662817+0000", 2024-09-11T07:40:36.113 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0031017319999999998 2024-09-11T07:40:36.113 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.113 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.114 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.114 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.663041+0000", 2024-09-11T07:40:36.114 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000224303 2024-09-11T07:40:36.114 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.114 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.114 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:36.114 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.663089+0000", 2024-09-11T07:40:36.114 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8464000000000002e-05 2024-09-11T07:40:36.114 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.114 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.114 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:36.114 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.663257+0000", 2024-09-11T07:40:36.114 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016791399999999999 2024-09-11T07:40:36.115 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.115 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:36.115 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.115 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.115 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.115 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196440 9.6 9:64e1f8bb:::benchmark_data_smithi022_133943_object69849:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:40:36.115 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.512090+0000", 2024-09-11T07:40:36.115 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 18.519966611000001, 2024-09-11T07:40:36.115 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14799346099999999, 2024-09-11T07:40:36.115 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:36.115 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:36.115 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:36.115 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:36.115 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:36.116 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196440 2024-09-11T07:40:36.116 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.116 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:36.116 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.116 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:36.116 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.512090+0000", 2024-09-11T07:40:36.116 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.116 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.116 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.116 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:36.116 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.512090+0000", 2024-09-11T07:40:36.116 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.116 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.117 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.117 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:36.117 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.512088+0000", 2024-09-11T07:40:36.117 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:40:36.117 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.117 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.117 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:36.117 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.512094+0000", 2024-09-11T07:40:36.117 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2809999999999999e-06 2024-09-11T07:40:36.117 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.117 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.117 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:36.117 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.512100+0000", 2024-09-11T07:40:36.118 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2960000000000004e-06 2024-09-11T07:40:36.118 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.118 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.118 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:36.118 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.512104+0000", 2024-09-11T07:40:36.118 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9570000000000004e-06 2024-09-11T07:40:36.118 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.118 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.118 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:36.118 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.512323+0000", 2024-09-11T07:40:36.118 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00021935600000000001 2024-09-11T07:40:36.118 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.118 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.118 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:36.119 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.512429+0000", 2024-09-11T07:40:36.119 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010613800000000001 2024-09-11T07:40:36.119 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.119 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.119 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:40:36.119 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.512856+0000", 2024-09-11T07:40:36.119 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00042626900000000001 2024-09-11T07:40:36.119 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.119 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.119 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:36.119 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.657518+0000", 2024-09-11T07:40:36.119 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14466178900000001 2024-09-11T07:40:36.119 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.120 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.120 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.120 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.658304+0000", 2024-09-11T07:40:36.120 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00078643699999999997 2024-09-11T07:40:36.120 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.120 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.120 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.120 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.660002+0000", 2024-09-11T07:40:36.120 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0016980249999999999 2024-09-11T07:40:36.120 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.120 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.120 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:36.120 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.660046+0000", 2024-09-11T07:40:36.121 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4304999999999997e-05 2024-09-11T07:40:36.121 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.121 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.121 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:36.121 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.660083+0000", 2024-09-11T07:40:36.121 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7125000000000001e-05 2024-09-11T07:40:36.121 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.121 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:36.121 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.121 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.121 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.121 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.15504.0:196437 9.8 e299/289 9:1765339f:::benchmark_data_smithi022_133943_object69846:head v 299'8840, mlcod=299'8838)", 2024-09-11T07:40:36.121 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.513692+0000", 2024-09-11T07:40:36.121 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 18.518364730999998, 2024-09-11T07:40:36.122 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.146462602, 2024-09-11T07:40:36.122 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:36.122 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:36.122 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:36.122 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.122 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:36.122 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.513692+0000", 2024-09-11T07:40:36.122 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.122 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.122 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.122 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:36.122 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.513692+0000", 2024-09-11T07:40:36.122 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.123 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.123 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.123 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:36.123 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.513691+0000", 2024-09-11T07:40:36.123 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:40:36.123 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.123 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.123 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:36.123 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.513698+0000", 2024-09-11T07:40:36.123 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.238e-06 2024-09-11T07:40:36.123 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.123 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.123 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:36.123 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.513707+0000", 2024-09-11T07:40:36.124 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7849999999999997e-06 2024-09-11T07:40:36.124 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.124 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.124 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:36.124 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.513712+0000", 2024-09-11T07:40:36.124 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0250000000000002e-06 2024-09-11T07:40:36.124 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.124 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.124 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:36.124 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.514773+0000", 2024-09-11T07:40:36.124 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001061602 2024-09-11T07:40:36.124 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.124 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.125 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:36.125 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.514816+0000", 2024-09-11T07:40:36.125 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3081999999999998e-05 2024-09-11T07:40:36.125 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.125 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.125 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:36.125 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.660113+0000", 2024-09-11T07:40:36.125 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.145296709 2024-09-11T07:40:36.125 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.125 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.125 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:36.125 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.660154+0000", 2024-09-11T07:40:36.125 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1479e-05 2024-09-11T07:40:36.126 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.126 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:36.126 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.126 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.126 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.126 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.15504.0:196431 9.b e299/247 9:d2784b45:::benchmark_data_smithi022_133943_object69840:head v 299'12483, mlcod=299'12479)", 2024-09-11T07:40:36.126 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.513721+0000", 2024-09-11T07:40:36.126 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 18.51833543, 2024-09-11T07:40:36.126 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14646970300000001, 2024-09-11T07:40:36.126 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:36.126 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:36.126 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:36.127 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.127 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:36.127 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.513721+0000", 2024-09-11T07:40:36.127 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.127 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.127 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.127 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:36.127 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.513721+0000", 2024-09-11T07:40:36.127 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.127 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.127 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.127 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:36.127 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.513721+0000", 2024-09-11T07:40:36.128 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:40:36.128 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.128 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.128 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:36.128 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.513724+0000", 2024-09-11T07:40:36.128 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.489e-06 2024-09-11T07:40:36.128 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.128 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.128 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:36.128 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.513729+0000", 2024-09-11T07:40:36.128 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4039999999999997e-06 2024-09-11T07:40:36.128 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.128 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.129 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:36.129 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.513732+0000", 2024-09-11T07:40:36.129 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.0319999999999999e-06 2024-09-11T07:40:36.129 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.129 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.129 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:36.129 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.513780+0000", 2024-09-11T07:40:36.129 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8207000000000001e-05 2024-09-11T07:40:36.129 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.129 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.129 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:36.129 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.513796+0000", 2024-09-11T07:40:36.129 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6396999999999999e-05 2024-09-11T07:40:36.130 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.130 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.130 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:36.130 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.660170+0000", 2024-09-11T07:40:36.130 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.146373223 2024-09-11T07:40:36.130 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.130 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.130 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:36.130 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.660191+0000", 2024-09-11T07:40:36.130 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1251999999999999e-05 2024-09-11T07:40:36.130 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.130 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:36.130 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.130 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.131 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.131 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196563 9.6 9:63ebaa83:::benchmark_data_smithi022_133943_object69972:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:40:36.131 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.771425+0000", 2024-09-11T07:40:36.131 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 18.260631060000001, 2024-09-11T07:40:36.131 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.153416993, 2024-09-11T07:40:36.131 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:36.131 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:36.131 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:36.131 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:36.131 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:36.131 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196563 2024-09-11T07:40:36.131 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.131 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:36.131 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.132 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:36.132 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771425+0000", 2024-09-11T07:40:36.132 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.132 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.132 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.132 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:36.132 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771425+0000", 2024-09-11T07:40:36.132 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.132 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.132 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.132 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:36.132 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771424+0000", 2024-09-11T07:40:36.132 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:40:36.133 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.133 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.133 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:36.133 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771430+0000", 2024-09-11T07:40:36.133 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4540000000000004e-06 2024-09-11T07:40:36.133 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.133 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.133 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:36.133 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771438+0000", 2024-09-11T07:40:36.133 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.418e-06 2024-09-11T07:40:36.133 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.133 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.133 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:36.133 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771442+0000", 2024-09-11T07:40:36.134 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.622e-06 2024-09-11T07:40:36.134 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.134 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.134 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:36.134 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771560+0000", 2024-09-11T07:40:36.134 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000117298 2024-09-11T07:40:36.134 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.134 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.134 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:36.134 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771669+0000", 2024-09-11T07:40:36.134 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000109386 2024-09-11T07:40:36.134 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.134 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.135 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:40:36.135 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.772085+0000", 2024-09-11T07:40:36.135 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00041578699999999998 2024-09-11T07:40:36.135 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.135 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.135 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:36.135 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.776041+0000", 2024-09-11T07:40:36.135 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0039559069999999998 2024-09-11T07:40:36.135 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.135 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.135 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.135 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.782755+0000", 2024-09-11T07:40:36.135 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0067147760000000004 2024-09-11T07:40:36.136 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.136 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.136 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.136 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924754+0000", 2024-09-11T07:40:36.136 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14199824599999999 2024-09-11T07:40:36.136 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.136 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.136 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:36.136 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924797+0000", 2024-09-11T07:40:36.136 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2988999999999998e-05 2024-09-11T07:40:36.136 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.136 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.136 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:36.137 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924842+0000", 2024-09-11T07:40:36.137 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5884000000000003e-05 2024-09-11T07:40:36.137 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.137 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:36.137 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.137 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.137 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.137 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196573 9.6 9:60e6e15d:::benchmark_data_smithi022_133943_object69982:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:40:36.137 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.777975+0000", 2024-09-11T07:40:36.137 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 18.254081246999998, 2024-09-11T07:40:36.137 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.146964759, 2024-09-11T07:40:36.137 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:36.137 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:36.137 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:36.137 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:36.138 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:36.138 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196573 2024-09-11T07:40:36.138 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.138 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:36.138 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.138 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:36.138 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.777975+0000", 2024-09-11T07:40:36.138 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.138 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.138 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.138 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:36.138 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.777975+0000", 2024-09-11T07:40:36.138 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.139 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.139 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.139 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:36.139 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.777975+0000", 2024-09-11T07:40:36.139 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:40:36.139 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.139 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.139 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:36.139 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.777981+0000", 2024-09-11T07:40:36.139 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7769999999999997e-06 2024-09-11T07:40:36.139 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.139 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.139 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:36.139 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.777988+0000", 2024-09-11T07:40:36.141 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4060000000000002e-06 2024-09-11T07:40:36.141 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.141 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.141 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:36.141 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.777997+0000", 2024-09-11T07:40:36.141 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3270000000000007e-06 2024-09-11T07:40:36.141 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.142 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.142 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:36.142 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.778084+0000", 2024-09-11T07:40:36.142 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6769999999999998e-05 2024-09-11T07:40:36.142 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.142 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.142 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:36.142 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.778192+0000", 2024-09-11T07:40:36.142 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010845600000000001 2024-09-11T07:40:36.142 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.142 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.142 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:40:36.142 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.778643+0000", 2024-09-11T07:40:36.142 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000450469 2024-09-11T07:40:36.142 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.143 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.143 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:36.143 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.780930+0000", 2024-09-11T07:40:36.143 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0022869409999999998 2024-09-11T07:40:36.143 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.143 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.143 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.143 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785416+0000", 2024-09-11T07:40:36.143 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0044864800000000002 2024-09-11T07:40:36.143 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.143 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.143 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.143 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924878+0000", 2024-09-11T07:40:36.143 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13946214200000001 2024-09-11T07:40:36.144 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.144 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.144 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:36.144 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924907+0000", 2024-09-11T07:40:36.144 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.8802e-05 2024-09-11T07:40:36.144 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.144 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.144 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:36.144 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924940+0000", 2024-09-11T07:40:36.144 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2943e-05 2024-09-11T07:40:36.144 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.144 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:36.144 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.144 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.145 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.145 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196574 9.13 9:cb380b75:::benchmark_data_smithi022_133943_object69983:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:40:36.145 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.782920+0000", 2024-09-11T07:40:36.145 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 18.249136287999999, 2024-09-11T07:40:36.145 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.145638404, 2024-09-11T07:40:36.145 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:36.145 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:36.145 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:36.145 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:36.145 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:36.145 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196574 2024-09-11T07:40:36.145 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.145 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:36.145 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.146 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:36.146 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.782920+0000", 2024-09-11T07:40:36.146 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.146 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.146 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.146 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:36.146 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.782920+0000", 2024-09-11T07:40:36.146 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.146 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.146 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.146 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:36.146 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.782919+0000", 2024-09-11T07:40:36.146 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:40:36.147 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.147 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.147 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:36.147 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.782929+0000", 2024-09-11T07:40:36.147 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0107e-05 2024-09-11T07:40:36.147 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.147 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.147 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:36.147 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.782936+0000", 2024-09-11T07:40:36.147 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0439999999999998e-06 2024-09-11T07:40:36.147 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.147 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.147 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:36.147 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.782939+0000", 2024-09-11T07:40:36.148 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2629999999999999e-06 2024-09-11T07:40:36.148 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.148 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.148 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:36.148 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.782971+0000", 2024-09-11T07:40:36.148 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1900999999999998e-05 2024-09-11T07:40:36.148 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.148 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.148 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:36.148 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.783052+0000", 2024-09-11T07:40:36.148 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0481000000000004e-05 2024-09-11T07:40:36.148 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.148 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.148 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,6,7", 2024-09-11T07:40:36.149 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.783444+0000", 2024-09-11T07:40:36.149 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00039258199999999999 2024-09-11T07:40:36.149 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.149 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.149 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:36.149 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.787141+0000", 2024-09-11T07:40:36.149 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0036968280000000001 2024-09-11T07:40:36.149 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.149 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.149 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.149 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924179+0000", 2024-09-11T07:40:36.149 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.137037457 2024-09-11T07:40:36.149 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.150 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.150 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.150 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924214+0000", 2024-09-11T07:40:36.150 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5756000000000001e-05 2024-09-11T07:40:36.150 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.150 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.150 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.150 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.928381+0000", 2024-09-11T07:40:36.150 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0041661249999999997 2024-09-11T07:40:36.150 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.150 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.150 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:36.150 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.928430+0000", 2024-09-11T07:40:36.151 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9651000000000003e-05 2024-09-11T07:40:36.151 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.151 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.151 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:36.151 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.928559+0000", 2024-09-11T07:40:36.151 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012844000000000001 2024-09-11T07:40:36.151 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.151 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:36.151 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.151 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.151 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.151 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196577 9.13 9:c894ba9e:::benchmark_data_smithi022_133943_object69986:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:40:36.151 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.785754+0000", 2024-09-11T07:40:36.151 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 18.246302941, 2024-09-11T07:40:36.152 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.167572637, 2024-09-11T07:40:36.152 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:36.152 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:36.152 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:36.152 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:36.152 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:36.152 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196577 2024-09-11T07:40:36.152 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.152 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:36.152 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.152 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:36.152 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785754+0000", 2024-09-11T07:40:36.152 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.153 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.153 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.153 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:36.153 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785754+0000", 2024-09-11T07:40:36.153 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.153 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.153 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.153 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:36.153 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785752+0000", 2024-09-11T07:40:36.153 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:40:36.153 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.153 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.153 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:36.153 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785757+0000", 2024-09-11T07:40:36.153 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0010000000000001e-06 2024-09-11T07:40:36.154 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.154 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.154 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:36.154 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785765+0000", 2024-09-11T07:40:36.154 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2670000000000006e-06 2024-09-11T07:40:36.154 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.154 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.154 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:36.154 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785771+0000", 2024-09-11T07:40:36.154 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.242e-06 2024-09-11T07:40:36.154 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.154 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.154 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:36.154 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924515+0000", 2024-09-11T07:40:36.154 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13874467300000001 2024-09-11T07:40:36.155 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.155 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.155 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:36.155 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924668+0000", 2024-09-11T07:40:36.155 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015245999999999999 2024-09-11T07:40:36.155 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.155 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.155 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,6,7", 2024-09-11T07:40:36.155 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.925194+0000", 2024-09-11T07:40:36.155 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00052637400000000003 2024-09-11T07:40:36.155 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.155 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.155 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.156 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.930777+0000", 2024-09-11T07:40:36.156 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00558319 2024-09-11T07:40:36.156 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.156 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.156 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.156 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.930807+0000", 2024-09-11T07:40:36.156 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9363999999999999e-05 2024-09-11T07:40:36.156 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.156 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.156 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:36.156 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.931376+0000", 2024-09-11T07:40:36.156 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00056952599999999999 2024-09-11T07:40:36.156 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.156 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.157 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.157 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.953159+0000", 2024-09-11T07:40:36.157 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.021782632999999999 2024-09-11T07:40:36.157 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.157 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.157 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:36.157 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.953206+0000", 2024-09-11T07:40:36.157 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7624999999999999e-05 2024-09-11T07:40:36.157 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.157 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.157 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:36.157 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.953326+0000", 2024-09-11T07:40:36.157 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000119883 2024-09-11T07:40:36.158 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.158 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:36.158 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.158 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.158 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.158 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196580 9.19 9:9d127f5e:::benchmark_data_smithi022_133943_object69989:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:40:36.158 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.788082+0000", 2024-09-11T07:40:36.158 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 18.243974215000001, 2024-09-11T07:40:36.158 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14594933199999999, 2024-09-11T07:40:36.158 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:36.158 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:36.158 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:36.158 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:36.159 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:36.159 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196580 2024-09-11T07:40:36.159 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.159 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:36.159 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.159 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:36.159 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.788082+0000", 2024-09-11T07:40:36.159 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.159 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.159 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.159 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:36.159 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.788082+0000", 2024-09-11T07:40:36.159 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:36.159 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.159 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.160 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:36.160 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.788080+0000", 2024-09-11T07:40:36.160 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:40:36.160 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.160 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.160 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:36.160 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.788090+0000", 2024-09-11T07:40:36.160 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4809999999999995e-06 2024-09-11T07:40:36.160 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.160 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.160 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:36.160 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924528+0000", 2024-09-11T07:40:36.160 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13643872700000001 2024-09-11T07:40:36.161 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.161 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.161 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:36.161 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924539+0000", 2024-09-11T07:40:36.161 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0772e-05 2024-09-11T07:40:36.161 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.161 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.161 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:36.161 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924601+0000", 2024-09-11T07:40:36.161 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1959999999999996e-05 2024-09-11T07:40:36.161 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.161 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.161 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:36.162 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924701+0000", 2024-09-11T07:40:36.162 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000100197 2024-09-11T07:40:36.162 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.162 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.162 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 4,6", 2024-09-11T07:40:36.162 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.925161+0000", 2024-09-11T07:40:36.162 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00045958799999999998 2024-09-11T07:40:36.162 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.162 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.162 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:36.162 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.930378+0000", 2024-09-11T07:40:36.162 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0052174639999999998 2024-09-11T07:40:36.162 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.162 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.163 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.163 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.930728+0000", 2024-09-11T07:40:36.163 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00034939200000000002 2024-09-11T07:40:36.163 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.163 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.163 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:36.163 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.933941+0000", 2024-09-11T07:40:36.163 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0032131009999999999 2024-09-11T07:40:36.163 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.163 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.163 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:36.163 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.933999+0000", 2024-09-11T07:40:36.163 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7933999999999998e-05 2024-09-11T07:40:36.163 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:36.164 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:36.164 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:36.164 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.934032+0000", 2024-09-11T07:40:36.164 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2851000000000001e-05 2024-09-11T07:40:36.164 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.164 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:36.164 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.164 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:36.164 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:36.164 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:40:36.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:36.165 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:36.165 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:36.266 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:36.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:36.289 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:36.289 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:36.390 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:36.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:36.413 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:36.413 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:36.514 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:36.544 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:36.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:36.545 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:36.646 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:36.677 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:36.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:36.678 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:36.778 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:36.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:36 smithi104 ceph-mon[33507]: 2.c scrub starts 2024-09-11T07:40:36.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:36 smithi104 ceph-mon[33507]: 2.c scrub ok 2024-09-11T07:40:36.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:36 smithi104 ceph-mon[33507]: osdmap e307: 8 total, 8 up, 8 in 2024-09-11T07:40:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:36 smithi104 ceph-mon[33507]: 2.1e scrub starts 2024-09-11T07:40:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:36 smithi104 ceph-mon[33507]: 2.1e scrub ok 2024-09-11T07:40:36.816 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:36.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:36.818 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:36.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:36 smithi022 ceph-mon[29462]: 2.c scrub starts 2024-09-11T07:40:36.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:36 smithi022 ceph-mon[29462]: 2.c scrub ok 2024-09-11T07:40:36.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:36 smithi022 ceph-mon[29462]: osdmap e307: 8 total, 8 up, 8 in 2024-09-11T07:40:36.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:36 smithi022 ceph-mon[29462]: 2.1e scrub starts 2024-09-11T07:40:36.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:36 smithi022 ceph-mon[29462]: 2.1e scrub ok 2024-09-11T07:40:36.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:36 smithi022 ceph-mon[37330]: 2.c scrub starts 2024-09-11T07:40:36.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:36 smithi022 ceph-mon[37330]: 2.c scrub ok 2024-09-11T07:40:36.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:36 smithi022 ceph-mon[37330]: osdmap e307: 8 total, 8 up, 8 in 2024-09-11T07:40:36.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:36 smithi022 ceph-mon[37330]: 2.1e scrub starts 2024-09-11T07:40:36.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:36 smithi022 ceph-mon[37330]: 2.1e scrub ok 2024-09-11T07:40:36.919 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:36.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:36.954 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:36.954 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:37.055 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:37.074 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-11T07:40:37.080 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:37.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:37.081 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:37.182 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:37.204 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:37.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:37.205 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:37.306 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:37.320 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:40:37.329 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:37.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:37.330 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:37.430 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:37.452 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:37.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:37.454 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:37.554 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:37.589 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:37.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:37.590 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:37.636 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:40:37.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:37 smithi022 ceph-mon[29462]: pgmap v642: 158 pgs: 1 peering, 2 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 148 active+clean; 2.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 714 op/s; 17775/128016 objects misplaced (13.885%); 24 MiB/s, 391 objects/s recovering 2024-09-11T07:40:37.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:37 smithi022 ceph-mon[29462]: osdmap e308: 8 total, 8 up, 8 in 2024-09-11T07:40:37.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:37 smithi022 ceph-mon[29462]: 6.1a scrub starts 2024-09-11T07:40:37.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:37 smithi022 ceph-mon[29462]: 6.1a scrub ok 2024-09-11T07:40:37.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:37 smithi022 ceph-mon[37330]: pgmap v642: 158 pgs: 1 peering, 2 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 148 active+clean; 2.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 714 op/s; 17775/128016 objects misplaced (13.885%); 24 MiB/s, 391 objects/s recovering 2024-09-11T07:40:37.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:37 smithi022 ceph-mon[37330]: osdmap e308: 8 total, 8 up, 8 in 2024-09-11T07:40:37.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:37 smithi022 ceph-mon[37330]: 6.1a scrub starts 2024-09-11T07:40:37.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:37 smithi022 ceph-mon[37330]: 6.1a scrub ok 2024-09-11T07:40:37.691 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:37.713 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:37 smithi104 ceph-mon[33507]: pgmap v642: 158 pgs: 1 peering, 2 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 148 active+clean; 2.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 714 op/s; 17775/128016 objects misplaced (13.885%); 24 MiB/s, 391 objects/s recovering 2024-09-11T07:40:37.713 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:37 smithi104 ceph-mon[33507]: osdmap e308: 8 total, 8 up, 8 in 2024-09-11T07:40:37.713 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:37 smithi104 ceph-mon[33507]: 6.1a scrub starts 2024-09-11T07:40:37.713 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:37 smithi104 ceph-mon[33507]: 6.1a scrub ok 2024-09-11T07:40:37.742 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:37.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:37.744 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:37.844 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:37.890 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:37.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:37.891 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:37.992 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:38.020 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:38.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:38.021 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:38.122 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:38.154 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:38.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:38.155 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:38.256 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:38.293 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:38.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:38.294 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:38.395 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:38.441 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:38.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:38.442 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:38.515 INFO:teuthology.orchestra.run.smithi022.stderr:reweighted osd.7 to 0.665761 (aa6f) 2024-09-11T07:40:38.542 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:38.578 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:38.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:38.579 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:38.588 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:40:38.680 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:38.727 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:38.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:38.728 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:38.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:38 smithi104 ceph-mon[33507]: 6.15 scrub starts 2024-09-11T07:40:38.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:38 smithi104 ceph-mon[33507]: 6.15 scrub ok 2024-09-11T07:40:38.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:38 smithi104 ceph-mon[33507]: osdmap e309: 8 total, 8 up, 8 in 2024-09-11T07:40:38.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:40:38.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:38 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3364564420' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 0.6657611370104746}]: dispatch 2024-09-11T07:40:38.829 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:38.865 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:38.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:38.866 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:38.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:38 smithi022 ceph-mon[29462]: 6.15 scrub starts 2024-09-11T07:40:38.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:38 smithi022 ceph-mon[29462]: 6.15 scrub ok 2024-09-11T07:40:38.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:38 smithi022 ceph-mon[29462]: osdmap e309: 8 total, 8 up, 8 in 2024-09-11T07:40:38.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:40:38.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:38 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3364564420' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 0.6657611370104746}]: dispatch 2024-09-11T07:40:38.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:38 smithi022 ceph-mon[37330]: 6.15 scrub starts 2024-09-11T07:40:38.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:38 smithi022 ceph-mon[37330]: 6.15 scrub ok 2024-09-11T07:40:38.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:38 smithi022 ceph-mon[37330]: osdmap e309: 8 total, 8 up, 8 in 2024-09-11T07:40:38.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:40:38.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:38 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3364564420' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 0.6657611370104746}]: dispatch 2024-09-11T07:40:38.967 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:38.995 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:38.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:38.996 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:39.097 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:39.121 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:39.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:39.122 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:39.149 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:40:39.149 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:40:39.150 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 0 2024-09-11T07:40:39.150 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:40:39.223 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:39.245 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:39.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:39.246 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:39.347 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:39.372 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:39.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:39.373 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:39.474 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:39.497 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:39.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:39.498 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:39.598 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:39.622 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:39.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:39.623 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:39.724 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:39.749 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:39.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:39.750 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:39.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:39 smithi104 ceph-mon[33507]: pgmap v645: 158 pgs: 1 peering, 2 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 148 active+clean; 2.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 811 op/s; 17775/128016 objects misplaced (13.885%); 13 MiB/s, 202 objects/s recovering 2024-09-11T07:40:39.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:39 smithi104 ceph-mon[33507]: 2.1 scrub starts 2024-09-11T07:40:39.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:39 smithi104 ceph-mon[33507]: 2.1 scrub ok 2024-09-11T07:40:39.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:39 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3364564420' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 0.6657611370104746}]': finished 2024-09-11T07:40:39.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:39 smithi104 ceph-mon[33507]: osdmap e310: 8 total, 8 up, 8 in 2024-09-11T07:40:39.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:39 smithi104 ceph-mon[33507]: 4.e scrub starts 2024-09-11T07:40:39.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:39 smithi104 ceph-mon[33507]: 4.e scrub ok 2024-09-11T07:40:39.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:39 smithi104 ceph-mon[33507]: 1.0 scrub starts 2024-09-11T07:40:39.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:39 smithi104 ceph-mon[33507]: 1.0 scrub ok 2024-09-11T07:40:39.850 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:39.872 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:39.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:39.873 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:39.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:39 smithi022 ceph-mon[29462]: pgmap v645: 158 pgs: 1 peering, 2 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 148 active+clean; 2.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 811 op/s; 17775/128016 objects misplaced (13.885%); 13 MiB/s, 202 objects/s recovering 2024-09-11T07:40:39.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:39 smithi022 ceph-mon[29462]: 2.1 scrub starts 2024-09-11T07:40:39.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:39 smithi022 ceph-mon[29462]: 2.1 scrub ok 2024-09-11T07:40:39.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:39 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3364564420' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 0.6657611370104746}]': finished 2024-09-11T07:40:39.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:39 smithi022 ceph-mon[29462]: osdmap e310: 8 total, 8 up, 8 in 2024-09-11T07:40:39.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:39 smithi022 ceph-mon[29462]: 4.e scrub starts 2024-09-11T07:40:39.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:39 smithi022 ceph-mon[29462]: 4.e scrub ok 2024-09-11T07:40:39.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:39 smithi022 ceph-mon[29462]: 1.0 scrub starts 2024-09-11T07:40:39.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:39 smithi022 ceph-mon[29462]: 1.0 scrub ok 2024-09-11T07:40:39.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:39 smithi022 ceph-mon[37330]: pgmap v645: 158 pgs: 1 peering, 2 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 148 active+clean; 2.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 811 op/s; 17775/128016 objects misplaced (13.885%); 13 MiB/s, 202 objects/s recovering 2024-09-11T07:40:39.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:39 smithi022 ceph-mon[37330]: 2.1 scrub starts 2024-09-11T07:40:39.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:39 smithi022 ceph-mon[37330]: 2.1 scrub ok 2024-09-11T07:40:39.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:39 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3364564420' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 0.6657611370104746}]': finished 2024-09-11T07:40:39.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:39 smithi022 ceph-mon[37330]: osdmap e310: 8 total, 8 up, 8 in 2024-09-11T07:40:39.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:39 smithi022 ceph-mon[37330]: 4.e scrub starts 2024-09-11T07:40:39.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:39 smithi022 ceph-mon[37330]: 4.e scrub ok 2024-09-11T07:40:39.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:39 smithi022 ceph-mon[37330]: 1.0 scrub starts 2024-09-11T07:40:39.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:39 smithi022 ceph-mon[37330]: 1.0 scrub ok 2024-09-11T07:40:39.974 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:39.994 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:39.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:39.995 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:40.054 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:40:39 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:40:40.076 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-11T07:40:40.095 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:40:40.096 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:40.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:40.121 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:40.121 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:40.222 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:40.243 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:40.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:40.244 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:40.320 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:40:40.345 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:40.368 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:40.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:40.369 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:40.470 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:40.492 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:40.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:40.493 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:40.517 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is unset 2024-09-11T07:40:40.518 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:40:40] "GET /metrics HTTP/1.1" 200 224777 "" "Prometheus/2.43.0" 2024-09-11T07:40:40.593 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:40.617 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:40.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:40.618 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:40.719 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:40.745 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:40.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:40.746 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:40.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:40 smithi104 ceph-mon[33507]: 5.1d scrub starts 2024-09-11T07:40:40.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:40 smithi104 ceph-mon[33507]: 5.1d scrub ok 2024-09-11T07:40:40.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:40 smithi104 ceph-mon[33507]: 5.16 scrub starts 2024-09-11T07:40:40.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:40 smithi104 ceph-mon[33507]: 5.16 scrub ok 2024-09-11T07:40:40.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:40 smithi104 ceph-mon[33507]: 4.1c scrub starts 2024-09-11T07:40:40.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:40 smithi104 ceph-mon[33507]: 4.1c scrub ok 2024-09-11T07:40:40.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:40 smithi104 ceph-mon[33507]: osdmap e311: 8 total, 8 up, 8 in 2024-09-11T07:40:40.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:40 smithi104 ceph-mon[33507]: 4.19 scrub starts 2024-09-11T07:40:40.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:40 smithi104 ceph-mon[33507]: 4.19 scrub ok 2024-09-11T07:40:40.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:40 smithi104 ceph-mon[33507]: 9.e scrub starts 2024-09-11T07:40:40.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:40 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1166374929' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:40:40.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:40 smithi104 ceph-mon[33507]: 9.e scrub ok 2024-09-11T07:40:40.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:40 smithi104 ceph-mon[33507]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:40:40.846 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:40.887 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:40.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:40.888 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:40.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-mon[29462]: 5.1d scrub starts 2024-09-11T07:40:40.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-mon[29462]: 5.1d scrub ok 2024-09-11T07:40:40.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-mon[29462]: 5.16 scrub starts 2024-09-11T07:40:40.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-mon[29462]: 5.16 scrub ok 2024-09-11T07:40:40.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-mon[29462]: 4.1c scrub starts 2024-09-11T07:40:40.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-mon[29462]: 4.1c scrub ok 2024-09-11T07:40:40.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-mon[29462]: osdmap e311: 8 total, 8 up, 8 in 2024-09-11T07:40:40.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-mon[29462]: 4.19 scrub starts 2024-09-11T07:40:40.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-mon[29462]: 4.19 scrub ok 2024-09-11T07:40:40.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-mon[29462]: 9.e scrub starts 2024-09-11T07:40:40.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1166374929' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:40:40.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-mon[29462]: 9.e scrub ok 2024-09-11T07:40:40.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-mon[29462]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:40:40.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-mon[37330]: 5.1d scrub starts 2024-09-11T07:40:40.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-mon[37330]: 5.1d scrub ok 2024-09-11T07:40:40.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-mon[37330]: 5.16 scrub starts 2024-09-11T07:40:40.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-mon[37330]: 5.16 scrub ok 2024-09-11T07:40:40.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-mon[37330]: 4.1c scrub starts 2024-09-11T07:40:40.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-mon[37330]: 4.1c scrub ok 2024-09-11T07:40:40.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-mon[37330]: osdmap e311: 8 total, 8 up, 8 in 2024-09-11T07:40:40.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-mon[37330]: 4.19 scrub starts 2024-09-11T07:40:40.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-mon[37330]: 4.19 scrub ok 2024-09-11T07:40:40.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-mon[37330]: 9.e scrub starts 2024-09-11T07:40:40.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1166374929' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:40:40.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-mon[37330]: 9.e scrub ok 2024-09-11T07:40:40.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:40 smithi022 ceph-mon[37330]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:40:40.990 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:41.030 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:41.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:41.031 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:41.131 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:41.170 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:41.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:41.171 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:41.272 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:41.314 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:41.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:41.315 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:41.416 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:41.451 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:41.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:41.452 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:41.553 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:41.591 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:41.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:41.593 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:41 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:40:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:41 smithi022 ceph-mon[29462]: 5.17 scrub starts 2024-09-11T07:40:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:41 smithi022 ceph-mon[29462]: 5.17 scrub ok 2024-09-11T07:40:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:41 smithi022 ceph-mon[29462]: pgmap v648: 158 pgs: 1 remapped+peering, 7 active+remapped+backfill_toofull, 150 active+clean; 2.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 760 op/s; 16074/118908 objects misplaced (13.518%); 10 MiB/s, 163 objects/s recovering 2024-09-11T07:40:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:41 smithi022 ceph-mon[29462]: 5.b scrub starts 2024-09-11T07:40:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:41 smithi022 ceph-mon[29462]: 5.b scrub ok 2024-09-11T07:40:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:41 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1166374929' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:40:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:41 smithi022 ceph-mon[29462]: osdmap e312: 8 total, 8 up, 8 in 2024-09-11T07:40:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:41 smithi022 ceph-mon[29462]: 9.18 starting backfill to osd.2 from (0'0,0'0] MAX to 309'9298 2024-09-11T07:40:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:41 smithi022 ceph-mon[29462]: 4.1 scrub starts 2024-09-11T07:40:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:41 smithi022 ceph-mon[29462]: 4.1 scrub ok 2024-09-11T07:40:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:41 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:40:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:41 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:40:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:41 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:40:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:41 smithi022 ceph-mon[37330]: 5.17 scrub starts 2024-09-11T07:40:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:41 smithi022 ceph-mon[37330]: 5.17 scrub ok 2024-09-11T07:40:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:41 smithi022 ceph-mon[37330]: pgmap v648: 158 pgs: 1 remapped+peering, 7 active+remapped+backfill_toofull, 150 active+clean; 2.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 760 op/s; 16074/118908 objects misplaced (13.518%); 10 MiB/s, 163 objects/s recovering 2024-09-11T07:40:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:41 smithi022 ceph-mon[37330]: 5.b scrub starts 2024-09-11T07:40:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:41 smithi022 ceph-mon[37330]: 5.b scrub ok 2024-09-11T07:40:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:41 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1166374929' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:40:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:41 smithi022 ceph-mon[37330]: osdmap e312: 8 total, 8 up, 8 in 2024-09-11T07:40:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:41 smithi022 ceph-mon[37330]: 9.18 starting backfill to osd.2 from (0'0,0'0] MAX to 309'9298 2024-09-11T07:40:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:41 smithi022 ceph-mon[37330]: 4.1 scrub starts 2024-09-11T07:40:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:41 smithi022 ceph-mon[37330]: 4.1 scrub ok 2024-09-11T07:40:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:41 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:40:41.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:41 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:40:41.694 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:41.721 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:41.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:41.722 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:41.759 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:41 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:40:41.759 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:41 smithi104 ceph-mon[33507]: 5.17 scrub starts 2024-09-11T07:40:41.760 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:41 smithi104 ceph-mon[33507]: 5.17 scrub ok 2024-09-11T07:40:41.760 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:41 smithi104 ceph-mon[33507]: pgmap v648: 158 pgs: 1 remapped+peering, 7 active+remapped+backfill_toofull, 150 active+clean; 2.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 760 op/s; 16074/118908 objects misplaced (13.518%); 10 MiB/s, 163 objects/s recovering 2024-09-11T07:40:41.760 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:41 smithi104 ceph-mon[33507]: 5.b scrub starts 2024-09-11T07:40:41.760 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:41 smithi104 ceph-mon[33507]: 5.b scrub ok 2024-09-11T07:40:41.760 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:41 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1166374929' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:40:41.760 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:41 smithi104 ceph-mon[33507]: osdmap e312: 8 total, 8 up, 8 in 2024-09-11T07:40:41.760 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:41 smithi104 ceph-mon[33507]: 9.18 starting backfill to osd.2 from (0'0,0'0] MAX to 309'9298 2024-09-11T07:40:41.760 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:41 smithi104 ceph-mon[33507]: 4.1 scrub starts 2024-09-11T07:40:41.760 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:41 smithi104 ceph-mon[33507]: 4.1 scrub ok 2024-09-11T07:40:41.760 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:41 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:40:41.760 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:41 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:40:41.823 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:41.863 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:41.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:41.865 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:41.965 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:42.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:42.000 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:42.001 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:42.101 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:42.141 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:42.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:42.143 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:42.243 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:42.272 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:42.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:42.273 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:42.374 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:42.406 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:42.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:42.407 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:42.508 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:42.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:42.540 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:42.540 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:42.618 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:40:42.619 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:40:42.641 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:42.650 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:42 smithi104 ceph-mon[33507]: 6.7 scrub starts 2024-09-11T07:40:42.650 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:42 smithi104 ceph-mon[33507]: osdmap e313: 8 total, 8 up, 8 in 2024-09-11T07:40:42.650 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:42 smithi104 ceph-mon[33507]: pgmap v651: 158 pgs: 1 remapped+peering, 7 active+remapped+backfill_toofull, 150 active+clean; 2.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 883 op/s; 15694/117429 objects misplaced (13.365%); 10 MiB/s, 163 objects/s recovering 2024-09-11T07:40:42.659 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:40:42.660 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:40:42.685 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:42.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:42.686 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:42.698 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:40:42.698 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:40:42.729 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:40:42.729 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:40:42.771 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:40:42.771 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:40:42.786 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:42.807 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:40:42.807 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:40:42.807 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:42 smithi022 ceph-mon[29462]: 6.7 scrub starts 2024-09-11T07:40:42.808 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:42 smithi022 ceph-mon[29462]: osdmap e313: 8 total, 8 up, 8 in 2024-09-11T07:40:42.808 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:42 smithi022 ceph-mon[29462]: pgmap v651: 158 pgs: 1 remapped+peering, 7 active+remapped+backfill_toofull, 150 active+clean; 2.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 883 op/s; 15694/117429 objects misplaced (13.365%); 10 MiB/s, 163 objects/s recovering 2024-09-11T07:40:42.808 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:42 smithi022 ceph-mon[37330]: 6.7 scrub starts 2024-09-11T07:40:42.808 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:42 smithi022 ceph-mon[37330]: osdmap e313: 8 total, 8 up, 8 in 2024-09-11T07:40:42.808 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:42 smithi022 ceph-mon[37330]: pgmap v651: 158 pgs: 1 remapped+peering, 7 active+remapped+backfill_toofull, 150 active+clean; 2.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 883 op/s; 15694/117429 objects misplaced (13.365%); 10 MiB/s, 163 objects/s recovering 2024-09-11T07:40:42.815 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:42.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:42.816 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:42.838 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:40:42.838 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:40:42.873 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:40:42.873 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:40:42.917 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:42.953 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:42.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:42.955 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:43.056 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:43.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:43.089 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:43.089 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:43.190 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:43.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:43.226 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:43.226 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:43.327 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:43.334 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:40:43.334 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:40:43.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:43.369 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:43.370 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:43.470 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:43.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:43.507 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:43.507 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:43.551 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T07:40:43.608 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:43.636 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:43.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:43.637 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:43.738 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:43.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:43.766 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:43.767 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:43.867 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:43.898 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:43.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:43.899 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:43.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:43 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/369884132' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:40:43.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:43 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/369884132' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:40:43.999 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:44.024 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:43 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/369884132' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:40:44.033 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:44.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:44.034 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:44.038 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:40:44.061 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-11T07:40:44.135 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:44.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:44.175 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:44.175 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:44.275 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:44.279 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:40:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:40:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:40:44.310 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:44.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:44.311 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:44.411 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:44.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:44.453 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:44.453 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:44.504 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:40:44.554 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:44.594 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:44.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:44.595 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:44.696 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:44.741 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:44.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:44.742 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:44.842 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:44.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:44.886 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:44.886 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:44.914 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:44 smithi104 ceph-mon[33507]: 4.7 scrub starts 2024-09-11T07:40:44.914 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:44 smithi104 ceph-mon[33507]: pgmap v652: 158 pgs: 1 remapped+peering, 7 active+remapped+backfill_toofull, 150 active+clean; 2.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 891 B/s rd, 0 B/s wr, 701 op/s; 15428/115938 objects misplaced (13.307%); 7.1 MiB/s, 113 objects/s recovering 2024-09-11T07:40:44.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:44 smithi022 ceph-mon[29462]: 4.7 scrub starts 2024-09-11T07:40:44.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:44 smithi022 ceph-mon[29462]: pgmap v652: 158 pgs: 1 remapped+peering, 7 active+remapped+backfill_toofull, 150 active+clean; 2.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 891 B/s rd, 0 B/s wr, 701 op/s; 15428/115938 objects misplaced (13.307%); 7.1 MiB/s, 113 objects/s recovering 2024-09-11T07:40:44.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:44 smithi022 ceph-mon[37330]: 4.7 scrub starts 2024-09-11T07:40:44.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:44 smithi022 ceph-mon[37330]: pgmap v652: 158 pgs: 1 remapped+peering, 7 active+remapped+backfill_toofull, 150 active+clean; 2.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 891 B/s rd, 0 B/s wr, 701 op/s; 15428/115938 objects misplaced (13.307%); 7.1 MiB/s, 113 objects/s recovering 2024-09-11T07:40:44.987 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:45.010 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:40:45.011 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:40:45.011 INFO:tasks.thrashosds.thrasher:Shrinking pool .mgr 2024-09-11T07:40:45.011 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool .mgr by 10 min 8 2024-09-11T07:40:45.011 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:40:45.021 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:45.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:45.022 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:45.123 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:45.164 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:45.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:45.165 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:45.266 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:45.296 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:45.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:45.297 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:45.398 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:45.425 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:45.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:45.426 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:45.527 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:45.535 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:40:45.556 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:45.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:45.557 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:45.643 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:45 smithi022 ceph-mon[37330]: 6.1c scrub starts 2024-09-11T07:40:45.643 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:45 smithi022 ceph-mon[37330]: 2.13 scrub starts 2024-09-11T07:40:45.644 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:45 smithi022 ceph-mon[29462]: 6.1c scrub starts 2024-09-11T07:40:45.644 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:45 smithi022 ceph-mon[29462]: 2.13 scrub starts 2024-09-11T07:40:45.657 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:45.659 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:40:45.726 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:45.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:45.727 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:45.828 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:45.885 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:45.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:45.886 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:45.974 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:45 smithi104 ceph-mon[33507]: 6.1c scrub starts 2024-09-11T07:40:45.974 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:45 smithi104 ceph-mon[33507]: 2.13 scrub starts 2024-09-11T07:40:45.987 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:46.026 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:46.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:46.027 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:46.128 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:46.158 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:46.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:46.159 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:46.260 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:46.308 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:46.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:46.309 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:46.410 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:46.458 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:46.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:46.459 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:46.560 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:46.595 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:46.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:46.596 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:46.697 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:46.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:46.729 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:46.730 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:46.822 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:40:46.830 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:46.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:46.859 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:46.859 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:46.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:46 smithi022 ceph-mon[29462]: 2.4 scrub starts 2024-09-11T07:40:46.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:46 smithi022 ceph-mon[29462]: 2.4 scrub ok 2024-09-11T07:40:46.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:46 smithi022 ceph-mon[29462]: 9.6 scrub starts 2024-09-11T07:40:46.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:46 smithi022 ceph-mon[29462]: 9.3 scrub starts 2024-09-11T07:40:46.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:46 smithi022 ceph-mon[29462]: 5.2 scrub starts 2024-09-11T07:40:46.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:46 smithi022 ceph-mon[29462]: pgmap v653: 158 pgs: 1 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 150 active+clean; 2.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 968 op/s; 14242/103299 objects misplaced (13.787%); 9.8 MiB/s, 157 objects/s recovering 2024-09-11T07:40:46.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:46 smithi022 ceph-mon[37330]: 2.4 scrub starts 2024-09-11T07:40:46.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:46 smithi022 ceph-mon[37330]: 2.4 scrub ok 2024-09-11T07:40:46.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:46 smithi022 ceph-mon[37330]: 9.6 scrub starts 2024-09-11T07:40:46.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:46 smithi022 ceph-mon[37330]: 9.3 scrub starts 2024-09-11T07:40:46.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:46 smithi022 ceph-mon[37330]: 5.2 scrub starts 2024-09-11T07:40:46.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:46 smithi022 ceph-mon[37330]: pgmap v653: 158 pgs: 1 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 150 active+clean; 2.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 968 op/s; 14242/103299 objects misplaced (13.787%); 9.8 MiB/s, 157 objects/s recovering 2024-09-11T07:40:46.959 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:46.980 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:46 smithi104 ceph-mon[33507]: 2.4 scrub starts 2024-09-11T07:40:46.980 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:46 smithi104 ceph-mon[33507]: 2.4 scrub ok 2024-09-11T07:40:46.980 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:46 smithi104 ceph-mon[33507]: 9.6 scrub starts 2024-09-11T07:40:46.980 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:46 smithi104 ceph-mon[33507]: 9.3 scrub starts 2024-09-11T07:40:46.980 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:46 smithi104 ceph-mon[33507]: 5.2 scrub starts 2024-09-11T07:40:46.980 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:46 smithi104 ceph-mon[33507]: pgmap v653: 158 pgs: 1 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 150 active+clean; 2.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 968 op/s; 14242/103299 objects misplaced (13.787%); 9.8 MiB/s, 157 objects/s recovering 2024-09-11T07:40:47.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:47.026 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:47.026 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:47.127 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:47.168 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:47.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:47.169 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:47.270 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:47.298 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:40:47.298 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:40:47.303 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:47.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:47.305 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:47.405 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:47.436 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:47.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:47.437 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:47.538 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:47.576 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:47.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:47.577 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:47.678 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:47.679 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is set 2024-09-11T07:40:47.706 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:47.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:47.707 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:47.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:47 smithi022 ceph-mon[29462]: 5.2 scrub ok 2024-09-11T07:40:47.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:47 smithi022 ceph-mon[29462]: 6.1 scrub starts 2024-09-11T07:40:47.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:47 smithi022 ceph-mon[29462]: 5.15 scrub starts 2024-09-11T07:40:47.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:47 smithi022 ceph-mon[29462]: 5.15 scrub ok 2024-09-11T07:40:47.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:47 smithi022 ceph-mon[29462]: 5.e scrub starts 2024-09-11T07:40:47.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:47 smithi022 ceph-mon[29462]: 5.e scrub ok 2024-09-11T07:40:47.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:47 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2948158219' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:40:47.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:47 smithi022 ceph-mon[29462]: 4.b scrub starts 2024-09-11T07:40:47.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:47 smithi022 ceph-mon[29462]: 4.b scrub ok 2024-09-11T07:40:47.748 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:47 smithi022 ceph-mon[37330]: 5.2 scrub ok 2024-09-11T07:40:47.748 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:47 smithi022 ceph-mon[37330]: 6.1 scrub starts 2024-09-11T07:40:47.748 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:47 smithi022 ceph-mon[37330]: 5.15 scrub starts 2024-09-11T07:40:47.748 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:47 smithi022 ceph-mon[37330]: 5.15 scrub ok 2024-09-11T07:40:47.748 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:47 smithi022 ceph-mon[37330]: 5.e scrub starts 2024-09-11T07:40:47.748 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:47 smithi022 ceph-mon[37330]: 5.e scrub ok 2024-09-11T07:40:47.748 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:47 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2948158219' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:40:47.748 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:47 smithi022 ceph-mon[37330]: 4.b scrub starts 2024-09-11T07:40:47.748 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:47 smithi022 ceph-mon[37330]: 4.b scrub ok 2024-09-11T07:40:47.808 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:47.841 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:47.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:47.842 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:47.942 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:47.985 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:47.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:47.986 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:48.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:47 smithi104 ceph-mon[33507]: 5.2 scrub ok 2024-09-11T07:40:48.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:47 smithi104 ceph-mon[33507]: 6.1 scrub starts 2024-09-11T07:40:48.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:47 smithi104 ceph-mon[33507]: 5.15 scrub starts 2024-09-11T07:40:48.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:47 smithi104 ceph-mon[33507]: 5.15 scrub ok 2024-09-11T07:40:48.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:47 smithi104 ceph-mon[33507]: 5.e scrub starts 2024-09-11T07:40:48.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:47 smithi104 ceph-mon[33507]: 5.e scrub ok 2024-09-11T07:40:48.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:47 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2948158219' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:40:48.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:47 smithi104 ceph-mon[33507]: 4.b scrub starts 2024-09-11T07:40:48.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:47 smithi104 ceph-mon[33507]: 4.b scrub ok 2024-09-11T07:40:48.087 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:48.127 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-11T07:40:48.128 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:48.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:48.130 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:48.230 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:48.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:48.268 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:48.268 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:48.369 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:48.394 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:48.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:48.395 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:48.464 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:40:48.497 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:48.520 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:48.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:48.521 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:48.622 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:48.658 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:48.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:48.659 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:48.760 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:48.795 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:48.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:48.796 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:48.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:48 smithi104 ceph-mon[33507]: 9.4 scrub starts 2024-09-11T07:40:48.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:48 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:40:48.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:48 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:40:48.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:48 smithi104 ceph-mon[33507]: 4.1e scrub starts 2024-09-11T07:40:48.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:48 smithi104 ceph-mon[33507]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:40:48.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:48 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2948158219' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:40:48.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:48 smithi104 ceph-mon[33507]: osdmap e314: 8 total, 8 up, 8 in 2024-09-11T07:40:48.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:48 smithi104 ceph-mon[33507]: pgmap v655: 158 pgs: 1 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 150 active+clean; 2.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 661 B/s rd, 0 B/s wr, 609 op/s; 14242/103299 objects misplaced (13.787%); 4.9 MiB/s, 78 objects/s recovering 2024-09-11T07:40:48.897 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:48.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:48 smithi022 ceph-mon[29462]: 9.4 scrub starts 2024-09-11T07:40:48.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:48 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:40:48.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:48 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:40:48.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:48 smithi022 ceph-mon[29462]: 4.1e scrub starts 2024-09-11T07:40:48.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:48 smithi022 ceph-mon[29462]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:40:48.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:48 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2948158219' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:40:48.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:48 smithi022 ceph-mon[29462]: osdmap e314: 8 total, 8 up, 8 in 2024-09-11T07:40:48.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:48 smithi022 ceph-mon[29462]: pgmap v655: 158 pgs: 1 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 150 active+clean; 2.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 661 B/s rd, 0 B/s wr, 609 op/s; 14242/103299 objects misplaced (13.787%); 4.9 MiB/s, 78 objects/s recovering 2024-09-11T07:40:48.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:48 smithi022 ceph-mon[37330]: 9.4 scrub starts 2024-09-11T07:40:48.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:48 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:40:48.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:48 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:40:48.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:48 smithi022 ceph-mon[37330]: 4.1e scrub starts 2024-09-11T07:40:48.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:48 smithi022 ceph-mon[37330]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:40:48.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:48 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2948158219' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:40:48.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:48 smithi022 ceph-mon[37330]: osdmap e314: 8 total, 8 up, 8 in 2024-09-11T07:40:48.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:48 smithi022 ceph-mon[37330]: pgmap v655: 158 pgs: 1 active+remapped+backfilling, 7 active+remapped+backfill_toofull, 150 active+clean; 2.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 661 B/s rd, 0 B/s wr, 609 op/s; 14242/103299 objects misplaced (13.787%); 4.9 MiB/s, 78 objects/s recovering 2024-09-11T07:40:48.925 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:48.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:48.926 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:49.027 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:49.060 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:49.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:49.061 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:49.162 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:49.186 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:49.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:49.187 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:49.288 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:49.309 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:49.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:49.310 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:49.411 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:49.433 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:49.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:49.434 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:49.535 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:49.558 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:49.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:49.559 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:49.660 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:49.681 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:49.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:49.682 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:49.783 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:49.828 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:49.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:49.829 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:49.929 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:49.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:49.963 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:49.963 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:49.993 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:49 smithi104 ceph-mon[33507]: osdmap e315: 8 total, 8 up, 8 in 2024-09-11T07:40:49.993 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:40:49 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:40:50.065 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:50.100 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:50.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:50.101 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:50.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:49 smithi022 ceph-mon[29462]: osdmap e315: 8 total, 8 up, 8 in 2024-09-11T07:40:50.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:49 smithi022 ceph-mon[37330]: osdmap e315: 8 total, 8 up, 8 in 2024-09-11T07:40:50.202 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:50.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:50.241 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:50.241 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:50.314 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:40:50.314 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:40:50.342 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:50.376 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:50.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:50.377 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:50.478 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:50.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:50.512 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:50.513 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:50.614 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:50.643 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:50.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:50.644 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:50.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:40:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:40:50] "GET /metrics HTTP/1.1" 200 224777 "" "Prometheus/2.43.0" 2024-09-11T07:40:50.744 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:50.793 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:50.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:50.796 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:50.898 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:50.936 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:50.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:50.937 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:50.967 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:50 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:40:50.967 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:50 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:40:50.967 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:50 smithi104 ceph-mon[33507]: osdmap e316: 8 total, 8 up, 8 in 2024-09-11T07:40:50.967 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:50 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:40:50.967 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:50 smithi104 ceph-mon[33507]: pgmap v658: 158 pgs: 7 active+remapped+backfill_toofull, 151 active+clean; 1.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.41k op/s; 12319/90582 objects misplaced (13.600%); 11 MiB/s, 179 objects/s recovering 2024-09-11T07:40:50.967 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:50 smithi104 ceph-mon[33507]: osdmap e317: 8 total, 8 up, 8 in 2024-09-11T07:40:50.979 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:50 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:40:50.979 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:50 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:40:50.979 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:50 smithi022 ceph-mon[29462]: osdmap e316: 8 total, 8 up, 8 in 2024-09-11T07:40:50.979 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:50 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:40:50.979 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:50 smithi022 ceph-mon[29462]: pgmap v658: 158 pgs: 7 active+remapped+backfill_toofull, 151 active+clean; 1.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.41k op/s; 12319/90582 objects misplaced (13.600%); 11 MiB/s, 179 objects/s recovering 2024-09-11T07:40:50.979 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:50 smithi022 ceph-mon[29462]: osdmap e317: 8 total, 8 up, 8 in 2024-09-11T07:40:50.979 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:50 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:40:50.979 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:50 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:40:50.980 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:50 smithi022 ceph-mon[37330]: osdmap e316: 8 total, 8 up, 8 in 2024-09-11T07:40:50.980 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:50 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:40:50.980 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:50 smithi022 ceph-mon[37330]: pgmap v658: 158 pgs: 7 active+remapped+backfill_toofull, 151 active+clean; 1.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.41k op/s; 12319/90582 objects misplaced (13.600%); 11 MiB/s, 179 objects/s recovering 2024-09-11T07:40:50.980 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:50 smithi022 ceph-mon[37330]: osdmap e317: 8 total, 8 up, 8 in 2024-09-11T07:40:50.999 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-11T07:40:51.039 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:51.079 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:51.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:51.081 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:51.181 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:51.217 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:51.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:51.218 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:51.245 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:40:51.248 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:40:51.320 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:51.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:51.365 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:51.365 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:51.366 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:40:51.466 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:51.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:51.499 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:51.499 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:51.600 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:51.646 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:51.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:51.647 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:51.744 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:40:51.745 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:40:51.748 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:51.764 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:40:51.764 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:40:51.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:51 smithi104 ceph-mon[33507]: from='client.25582 v1:172.21.15.22:0/1048350781' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:40:51.783 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:51.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:51.784 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:51.798 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:40:51.798 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:40:51.829 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:40:51.830 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:40:51.860 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:40:51.861 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:40:51.861 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:51 smithi022 ceph-mon[29462]: from='client.25582 v1:172.21.15.22:0/1048350781' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:40:51.861 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:51 smithi022 ceph-mon[37330]: from='client.25582 v1:172.21.15.22:0/1048350781' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:40:51.885 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:51.893 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:40:51.893 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:40:51.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:51.922 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:51.922 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:51.929 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:40:51.929 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:40:51.958 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:40:51.958 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:40:52.023 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:52.052 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:52.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:52.053 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:52.154 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:52.182 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:52.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:52.183 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:52.210 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":659,"stamp":"2024-09-11T07:40:50.658428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1914640628,"num_objects":29403,"num_object_clones":0,"num_object_copies":88209,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":11693,"num_objects_unfound":0,"num_objects_dirty":29403,"num_whiteouts":0,"num_read":2123,"num_read_kb":1980,"num_write":224800,"num_write_kb":8103304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43953,"num_bytes_recovered":2872578873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136120,"ondisk_log_size":136120,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":510,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":11468888,"kb_used_data":7554348,"kb_used_omap":16,"kb_used_meta":3914159,"kb_avail":738492328,"statfs":{"total":767960285184,"available":756216143872,"internally_reserved":0,"allocated":7735652352,"data_stored":7732379481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16530,"internal_metadata":4008099694},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-329646080,"num_objects":-5030,"num_object_clones":0,"num_object_copies":-15090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-2549,"num_objects_unfound":0,"num_objects_dirty":-5030,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5030,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":469,"num_bytes_recovered":30736384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.404466"},"pg_stats":[{"pgid":"9.14","version":"315'9673","reported_seq":9785,"reported_epoch":315,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-11T07:40:49.072354+0000","last_change":"2024-09-11T07:40:42.790526+0000","last_active":"2024-09-11T07:40:49.072354+0000","last_peered":"2024-09-11T07:40:49.072354+0000","last_clean":"2024-09-11T07:39:56.654544+0000","last_became_active":"2024-09-11T07:40:12.755113+0000","last_became_peered":"2024-09-11T07:40:12.755113+0000","last_unstale":"2024-09-11T07:40:49.072354+0000","last_undegraded":"2024-09-11T07:40:49.072354+0000","last_fullsized":"2024-09-11T07:40:49.072354+0000","mapping_epoch":297,"log_start":"257'5254","ondisk_log_start":"257'5254","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4419,"log_dups_size":0,"ondisk_log_size":4419,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63569920,"num_objects":970,"num_object_clones":0,"num_object_copies":2910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1940,"num_objects_unfound":0,"num_objects_dirty":970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6986,"num_write_kb":254592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,2],"acting":[4,1,3],"avail_no_missing":["4","1","3"],"object_location_counts":[{"shards":"1,3,4","objects":970}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":133,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.687286+0000","last_change":"2024-09-11T07:40:39.605681+0000","last_active":"2024-09-11T07:40:47.687286+0000","last_peered":"2024-09-11T07:40:47.687286+0000","last_clean":"2024-09-11T07:40:47.687286+0000","last_became_active":"2024-09-11T07:40:39.605361+0000","last_became_peered":"2024-09-11T07:40:39.605361+0000","last_unstale":"2024-09-11T07:40:47.687286+0000","last_undegraded":"2024-09-11T07:40:47.687286+0000","last_fullsized":"2024-09-11T07:40:47.687286+0000","mapping_epoch":310,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":311,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:29.007411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:29.007411+0000","last_clean_scrub_stamp":"2024-09-11T07:37:29.007411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001472253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":344,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.692730+0000","last_change":"2024-09-11T07:38:27.418908+0000","last_active":"2024-09-11T07:40:48.692730+0000","last_peered":"2024-09-11T07:40:48.692730+0000","last_clean":"2024-09-11T07:40:48.692730+0000","last_became_active":"2024-09-11T07:32:00.508572+0000","last_became_peered":"2024-09-11T07:32:00.508572+0000","last_unstale":"2024-09-11T07:40:48.692730+0000","last_undegraded":"2024-09-11T07:40:48.692730+0000","last_fullsized":"2024-09-11T07:40:48.692730+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:27.418829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:38:27.418829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036972480000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"234'34","reported_seq":368,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.702180+0000","last_change":"2024-09-11T07:40:39.607640+0000","last_active":"2024-09-11T07:40:48.702180+0000","last_peered":"2024-09-11T07:40:48.702180+0000","last_clean":"2024-09-11T07:40:48.702180+0000","last_became_active":"2024-09-11T07:26:33.544627+0000","last_became_peered":"2024-09-11T07:26:33.544627+0000","last_unstale":"2024-09-11T07:40:48.702180+0000","last_undegraded":"2024-09-11T07:40:48.702180+0000","last_fullsized":"2024-09-11T07:40:48.702180+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'34","last_scrub_stamp":"2024-09-11T07:40:39.607579+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:40:39.607579+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:51.954396+0000","scrub_duration":0.0054431660000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":332,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.686323+0000","last_change":"2024-09-11T07:39:56.545948+0000","last_active":"2024-09-11T07:40:47.686323+0000","last_peered":"2024-09-11T07:40:47.686323+0000","last_clean":"2024-09-11T07:40:47.686323+0000","last_became_active":"2024-09-11T07:26:35.556296+0000","last_became_peered":"2024-09-11T07:26:35.556296+0000","last_unstale":"2024-09-11T07:40:47.686323+0000","last_undegraded":"2024-09-11T07:40:47.686323+0000","last_fullsized":"2024-09-11T07:40:47.686323+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:56.545872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:39:56.545872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:24.540468+0000","scrub_duration":0.002240283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.15","version":"315'9777","reported_seq":22253,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:49.216950+0000","last_change":"2024-09-11T07:40:10.796286+0000","last_active":"2024-09-11T07:40:49.216950+0000","last_peered":"2024-09-11T07:40:49.216950+0000","last_clean":"2024-09-11T07:40:49.216950+0000","last_became_active":"2024-09-11T07:40:10.762538+0000","last_became_peered":"2024-09-11T07:40:10.762538+0000","last_unstale":"2024-09-11T07:40:49.216950+0000","last_undegraded":"2024-09-11T07:40:49.216950+0000","last_fullsized":"2024-09-11T07:40:49.216950+0000","mapping_epoch":295,"log_start":"255'3199","ondisk_log_start":"255'3199","created":259,"last_epoch_clean":296,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6578,"log_dups_size":0,"ondisk_log_size":6578,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59179008,"num_objects":903,"num_object_clones":0,"num_object_copies":2709,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7063,"num_write_kb":254912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4795,"num_bytes_recovered":314179584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":340,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.691422+0000","last_change":"2024-09-11T07:40:35.645201+0000","last_active":"2024-09-11T07:40:48.691422+0000","last_peered":"2024-09-11T07:40:48.691422+0000","last_clean":"2024-09-11T07:40:48.691422+0000","last_became_active":"2024-09-11T07:39:57.767806+0000","last_became_peered":"2024-09-11T07:39:57.767806+0000","last_unstale":"2024-09-11T07:40:48.691422+0000","last_undegraded":"2024-09-11T07:40:48.691422+0000","last_fullsized":"2024-09-11T07:40:48.691422+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:35.645119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:40:35.645119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:00.870748+0000","scrub_duration":0.0071309060000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"234'19","reported_seq":349,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.686398+0000","last_change":"2024-09-11T07:39:32.589939+0000","last_active":"2024-09-11T07:40:47.686398+0000","last_peered":"2024-09-11T07:40:47.686398+0000","last_clean":"2024-09-11T07:40:47.686398+0000","last_became_active":"2024-09-11T07:26:33.540732+0000","last_became_peered":"2024-09-11T07:26:33.540732+0000","last_unstale":"2024-09-11T07:40:47.686398+0000","last_undegraded":"2024-09-11T07:40:47.686398+0000","last_fullsized":"2024-09-11T07:40:47.686398+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:36:42.708941+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:36:42.708941+0000","last_clean_scrub_stamp":"2024-09-11T07:36:42.708941+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0088444879999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":331,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.682865+0000","last_change":"2024-09-11T07:37:43.087423+0000","last_active":"2024-09-11T07:40:47.682865+0000","last_peered":"2024-09-11T07:40:47.682865+0000","last_clean":"2024-09-11T07:40:47.682865+0000","last_became_active":"2024-09-11T07:32:00.513650+0000","last_became_peered":"2024-09-11T07:32:00.513650+0000","last_unstale":"2024-09-11T07:40:47.682865+0000","last_undegraded":"2024-09-11T07:40:47.682865+0000","last_fullsized":"2024-09-11T07:40:47.682865+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:43.087302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:37:43.087302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00405653,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":343,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.686448+0000","last_change":"2024-09-11T07:40:36.708489+0000","last_active":"2024-09-11T07:40:47.686448+0000","last_peered":"2024-09-11T07:40:47.686448+0000","last_clean":"2024-09-11T07:40:47.686448+0000","last_became_active":"2024-09-11T07:40:11.698235+0000","last_became_peered":"2024-09-11T07:40:11.698235+0000","last_unstale":"2024-09-11T07:40:47.686448+0000","last_undegraded":"2024-09-11T07:40:47.686448+0000","last_fullsized":"2024-09-11T07:40:47.686448+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:36.708406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:40:36.708406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:50.355594+0000","scrub_duration":0.0020771840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"315'9899","reported_seq":18949,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:49.091771+0000","last_change":"2024-09-11T07:40:36.515700+0000","last_active":"2024-09-11T07:40:49.091771+0000","last_peered":"2024-09-11T07:40:49.091771+0000","last_clean":"2024-09-11T07:40:49.091771+0000","last_became_active":"2024-09-11T07:40:36.499497+0000","last_became_peered":"2024-09-11T07:40:36.499497+0000","last_unstale":"2024-09-11T07:40:49.091771+0000","last_undegraded":"2024-09-11T07:40:49.091771+0000","last_fullsized":"2024-09-11T07:40:49.091771+0000","mapping_epoch":307,"log_start":"258'5465","ondisk_log_start":"258'5465","created":259,"last_epoch_clean":308,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4434,"log_dups_size":0,"ondisk_log_size":4434,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60227584,"num_objects":919,"num_object_clones":0,"num_object_copies":2757,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7135,"num_write_kb":257728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4458,"num_bytes_recovered":292159488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":321,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.679066+0000","last_change":"2024-09-11T07:37:34.542508+0000","last_active":"2024-09-11T07:40:47.679066+0000","last_peered":"2024-09-11T07:40:47.679066+0000","last_clean":"2024-09-11T07:40:47.679066+0000","last_became_active":"2024-09-11T07:32:00.516662+0000","last_became_peered":"2024-09-11T07:32:00.516662+0000","last_unstale":"2024-09-11T07:40:47.679066+0000","last_undegraded":"2024-09-11T07:40:47.679066+0000","last_fullsized":"2024-09-11T07:40:47.679066+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:34.542441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:34.542441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015726010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"234'9","reported_seq":355,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.686713+0000","last_change":"2024-09-11T07:39:57.773949+0000","last_active":"2024-09-11T07:40:47.686713+0000","last_peered":"2024-09-11T07:40:47.686713+0000","last_clean":"2024-09-11T07:40:47.686713+0000","last_became_active":"2024-09-11T07:39:57.765606+0000","last_became_peered":"2024-09-11T07:39:57.765606+0000","last_unstale":"2024-09-11T07:40:47.686713+0000","last_undegraded":"2024-09-11T07:40:47.686713+0000","last_fullsized":"2024-09-11T07:40:47.686713+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'9","last_scrub_stamp":"2024-09-11T07:37:37.558036+0000","last_deep_scrub":"82'7","last_deep_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_clean_scrub_stamp":"2024-09-11T07:37:37.558036+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034408770000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":290,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.678641+0000","last_change":"2024-09-11T07:39:32.482789+0000","last_active":"2024-09-11T07:40:47.678641+0000","last_peered":"2024-09-11T07:40:47.678641+0000","last_clean":"2024-09-11T07:40:47.678641+0000","last_became_active":"2024-09-11T07:32:00.513206+0000","last_became_peered":"2024-09-11T07:32:00.513206+0000","last_unstale":"2024-09-11T07:40:47.678641+0000","last_undegraded":"2024-09-11T07:40:47.678641+0000","last_fullsized":"2024-09-11T07:40:47.678641+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:50.332244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.973323+0000","last_clean_scrub_stamp":"2024-09-11T07:35:50.332244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034655860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":49,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.682704+0000","last_change":"2024-09-11T07:40:11.703754+0000","last_active":"2024-09-11T07:40:47.682704+0000","last_peered":"2024-09-11T07:40:47.682704+0000","last_clean":"2024-09-11T07:40:47.682704+0000","last_became_active":"2024-09-11T07:40:11.701560+0000","last_became_peered":"2024-09-11T07:40:11.701560+0000","last_unstale":"2024-09-11T07:40:47.682704+0000","last_undegraded":"2024-09-11T07:40:47.682704+0000","last_fullsized":"2024-09-11T07:40:47.682704+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:36.462794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:45.149916+0000","last_clean_scrub_stamp":"2024-09-11T07:37:36.462794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"315'9768","reported_seq":18750,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:49.126145+0000","last_change":"2024-09-11T07:40:26.422385+0000","last_active":"2024-09-11T07:40:49.126145+0000","last_peered":"2024-09-11T07:40:49.126145+0000","last_clean":"2024-09-11T07:40:49.126145+0000","last_became_active":"2024-09-11T07:40:26.404009+0000","last_became_peered":"2024-09-11T07:40:26.404009+0000","last_unstale":"2024-09-11T07:40:49.126145+0000","last_undegraded":"2024-09-11T07:40:49.126145+0000","last_fullsized":"2024-09-11T07:40:49.126145+0000","mapping_epoch":301,"log_start":"256'4915","ondisk_log_start":"256'4915","created":259,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4853,"log_dups_size":0,"ondisk_log_size":4853,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58982400,"num_objects":900,"num_object_clones":0,"num_object_copies":2700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7064,"num_write_kb":254848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3243,"num_bytes_recovered":212533248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":286,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.681978+0000","last_change":"2024-09-11T07:37:40.336139+0000","last_active":"2024-09-11T07:40:47.681978+0000","last_peered":"2024-09-11T07:40:47.681978+0000","last_clean":"2024-09-11T07:40:47.681978+0000","last_became_active":"2024-09-11T07:32:00.505709+0000","last_became_peered":"2024-09-11T07:32:00.505709+0000","last_unstale":"2024-09-11T07:40:47.681978+0000","last_undegraded":"2024-09-11T07:40:47.681978+0000","last_fullsized":"2024-09-11T07:40:47.681978+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:40.336050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_clean_scrub_stamp":"2024-09-11T07:37:40.336050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021490649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"234'19","reported_seq":346,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.686525+0000","last_change":"2024-09-11T07:39:39.571267+0000","last_active":"2024-09-11T07:40:47.686525+0000","last_peered":"2024-09-11T07:40:47.686525+0000","last_clean":"2024-09-11T07:40:47.686525+0000","last_became_active":"2024-09-11T07:26:33.544386+0000","last_became_peered":"2024-09-11T07:26:33.544386+0000","last_unstale":"2024-09-11T07:40:47.686525+0000","last_undegraded":"2024-09-11T07:40:47.686525+0000","last_fullsized":"2024-09-11T07:40:47.686525+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'13","last_scrub_stamp":"2024-09-11T07:36:27.746295+0000","last_deep_scrub":"136'13","last_deep_scrub_stamp":"2024-09-11T07:31:08.042198+0000","last_clean_scrub_stamp":"2024-09-11T07:36:27.746295+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0089695690000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":339,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.692136+0000","last_change":"2024-09-11T07:40:11.700127+0000","last_active":"2024-09-11T07:40:48.692136+0000","last_peered":"2024-09-11T07:40:48.692136+0000","last_clean":"2024-09-11T07:40:48.692136+0000","last_became_active":"2024-09-11T07:40:11.699772+0000","last_became_peered":"2024-09-11T07:40:11.699772+0000","last_unstale":"2024-09-11T07:40:48.692136+0000","last_undegraded":"2024-09-11T07:40:48.692136+0000","last_fullsized":"2024-09-11T07:40:48.692136+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:27.145883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:38:27.145883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011677339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":338,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.692857+0000","last_change":"2024-09-11T07:39:51.811562+0000","last_active":"2024-09-11T07:40:48.692857+0000","last_peered":"2024-09-11T07:40:48.692857+0000","last_clean":"2024-09-11T07:40:48.692857+0000","last_became_active":"2024-09-11T07:32:00.530335+0000","last_became_peered":"2024-09-11T07:32:00.530335+0000","last_unstale":"2024-09-11T07:40:48.692857+0000","last_undegraded":"2024-09-11T07:40:48.692857+0000","last_fullsized":"2024-09-11T07:40:48.692857+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:51.811287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:39:51.811287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:21.373975+0000","scrub_duration":0.0069635549999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"315'9697","reported_seq":9801,"reported_epoch":315,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-11T07:40:49.203163+0000","last_change":"2024-09-11T07:40:28.935451+0000","last_active":"2024-09-11T07:40:49.203163+0000","last_peered":"2024-09-11T07:40:49.203163+0000","last_clean":"2024-09-11T07:39:56.598112+0000","last_became_active":"2024-09-11T07:39:58.829214+0000","last_became_peered":"2024-09-11T07:39:58.829214+0000","last_unstale":"2024-09-11T07:40:49.203163+0000","last_undegraded":"2024-09-11T07:40:49.203163+0000","last_fullsized":"2024-09-11T07:40:49.203163+0000","mapping_epoch":289,"log_start":"256'3568","ondisk_log_start":"256'3568","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6129,"log_dups_size":0,"ondisk_log_size":6129,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63438848,"num_objects":968,"num_object_clones":0,"num_object_copies":2904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":968,"num_objects_unfound":0,"num_objects_dirty":968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6978,"num_write_kb":254272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,1],"avail_no_missing":["2","1","6"],"object_location_counts":[{"shards":"1,2,6","objects":968}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"234'31","reported_seq":422,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.687245+0000","last_change":"2024-09-11T07:39:41.579223+0000","last_active":"2024-09-11T07:40:47.687245+0000","last_peered":"2024-09-11T07:40:47.687245+0000","last_clean":"2024-09-11T07:40:47.687245+0000","last_became_active":"2024-09-11T07:29:51.482755+0000","last_became_peered":"2024-09-11T07:29:51.482755+0000","last_unstale":"2024-09-11T07:40:47.687245+0000","last_undegraded":"2024-09-11T07:40:47.687245+0000","last_fullsized":"2024-09-11T07:40:47.687245+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'21","last_scrub_stamp":"2024-09-11T07:36:39.736944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:36:39.736944+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034170279999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":319,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.677716+0000","last_change":"2024-09-11T07:39:44.385573+0000","last_active":"2024-09-11T07:40:47.677716+0000","last_peered":"2024-09-11T07:40:47.677716+0000","last_clean":"2024-09-11T07:40:47.677716+0000","last_became_active":"2024-09-11T07:32:00.507191+0000","last_became_peered":"2024-09-11T07:32:00.507191+0000","last_unstale":"2024-09-11T07:40:47.677716+0000","last_undegraded":"2024-09-11T07:40:47.677716+0000","last_fullsized":"2024-09-11T07:40:47.677716+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:51.418258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:51.418258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001500181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":314,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.696017+0000","last_change":"2024-09-11T07:38:22.027909+0000","last_active":"2024-09-11T07:40:47.696017+0000","last_peered":"2024-09-11T07:40:47.696017+0000","last_clean":"2024-09-11T07:40:47.696017+0000","last_became_active":"2024-09-11T07:26:35.557989+0000","last_became_peered":"2024-09-11T07:26:35.557989+0000","last_unstale":"2024-09-11T07:40:47.696017+0000","last_undegraded":"2024-09-11T07:40:47.696017+0000","last_fullsized":"2024-09-11T07:40:47.696017+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:22.027769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:38:22.027769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060071020000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":333,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.685049+0000","last_change":"2024-09-11T07:39:41.220688+0000","last_active":"2024-09-11T07:40:47.685049+0000","last_peered":"2024-09-11T07:40:47.685049+0000","last_clean":"2024-09-11T07:40:47.685049+0000","last_became_active":"2024-09-11T07:26:37.564554+0000","last_became_peered":"2024-09-11T07:26:37.564554+0000","last_unstale":"2024-09-11T07:40:47.685049+0000","last_undegraded":"2024-09-11T07:40:47.685049+0000","last_fullsized":"2024-09-11T07:40:47.685049+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:41.220589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:39:41.220589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066621019999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"316'9452","reported_seq":13400,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-09-11T07:40:49.891540+0000","last_change":"2024-09-11T07:40:10.728235+0000","last_active":"2024-09-11T07:40:49.891540+0000","last_peered":"2024-09-11T07:40:49.891540+0000","last_clean":"2024-09-11T07:40:49.891540+0000","last_became_active":"2024-09-11T07:40:10.693596+0000","last_became_peered":"2024-09-11T07:40:10.693596+0000","last_unstale":"2024-09-11T07:40:49.891540+0000","last_undegraded":"2024-09-11T07:40:49.891540+0000","last_fullsized":"2024-09-11T07:40:49.891540+0000","mapping_epoch":295,"log_start":"256'4585","ondisk_log_start":"256'4585","created":247,"last_epoch_clean":296,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4867,"log_dups_size":0,"ondisk_log_size":4867,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64946176,"num_objects":991,"num_object_clones":0,"num_object_copies":2973,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6775,"num_write_kb":248512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1918,"num_bytes_recovered":125698048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"234'25","reported_seq":362,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.687432+0000","last_change":"2024-09-11T07:39:33.667725+0000","last_active":"2024-09-11T07:40:47.687432+0000","last_peered":"2024-09-11T07:40:47.687432+0000","last_clean":"2024-09-11T07:40:47.687432+0000","last_became_active":"2024-09-11T07:26:33.535411+0000","last_became_peered":"2024-09-11T07:26:33.535411+0000","last_unstale":"2024-09-11T07:40:47.687432+0000","last_undegraded":"2024-09-11T07:40:47.687432+0000","last_fullsized":"2024-09-11T07:40:47.687432+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'25","last_scrub_stamp":"2024-09-11T07:39:33.667530+0000","last_deep_scrub":"112'15","last_deep_scrub_stamp":"2024-09-11T07:31:10.005896+0000","last_clean_scrub_stamp":"2024-09-11T07:39:33.667530+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.041705696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":335,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.695439+0000","last_change":"2024-09-11T07:37:36.098229+0000","last_active":"2024-09-11T07:40:47.695439+0000","last_peered":"2024-09-11T07:40:47.695439+0000","last_clean":"2024-09-11T07:40:47.695439+0000","last_became_active":"2024-09-11T07:32:00.506615+0000","last_became_peered":"2024-09-11T07:32:00.506615+0000","last_unstale":"2024-09-11T07:40:47.695439+0000","last_undegraded":"2024-09-11T07:40:47.695439+0000","last_fullsized":"2024-09-11T07:40:47.695439+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:36.098175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:36.098175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017555839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"299'11","reported_seq":537,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.683157+0000","last_change":"2024-09-11T07:40:11.703968+0000","last_active":"2024-09-11T07:40:47.683157+0000","last_peered":"2024-09-11T07:40:47.683157+0000","last_clean":"2024-09-11T07:40:47.683157+0000","last_became_active":"2024-09-11T07:40:11.697568+0000","last_became_peered":"2024-09-11T07:40:11.697568+0000","last_unstale":"2024-09-11T07:40:47.683157+0000","last_undegraded":"2024-09-11T07:40:47.683157+0000","last_fullsized":"2024-09-11T07:40:47.683157+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:37:43.140999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:43.140999+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039651130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":330,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.687520+0000","last_change":"2024-09-11T07:39:57.698127+0000","last_active":"2024-09-11T07:40:47.687520+0000","last_peered":"2024-09-11T07:40:47.687520+0000","last_clean":"2024-09-11T07:40:47.687520+0000","last_became_active":"2024-09-11T07:39:57.697588+0000","last_became_peered":"2024-09-11T07:39:57.697588+0000","last_unstale":"2024-09-11T07:40:47.687520+0000","last_undegraded":"2024-09-11T07:40:47.687520+0000","last_fullsized":"2024-09-11T07:40:47.687520+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:52.605883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:39:52.605883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:21.117617+0000","scrub_duration":0.0013626420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"315'9577","reported_seq":12295,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:49.037743+0000","last_change":"2024-09-11T07:40:37.540503+0000","last_active":"2024-09-11T07:40:49.037743+0000","last_peered":"2024-09-11T07:40:49.037743+0000","last_clean":"2024-09-11T07:40:49.037743+0000","last_became_active":"2024-09-11T07:40:37.499654+0000","last_became_peered":"2024-09-11T07:40:37.499654+0000","last_unstale":"2024-09-11T07:40:49.037743+0000","last_undegraded":"2024-09-11T07:40:49.037743+0000","last_fullsized":"2024-09-11T07:40:49.037743+0000","mapping_epoch":308,"log_start":"256'4631","ondisk_log_start":"256'4631","created":247,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4946,"log_dups_size":0,"ondisk_log_size":4946,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59047936,"num_objects":901,"num_object_clones":0,"num_object_copies":2703,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6915,"num_write_kb":250112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1302,"num_bytes_recovered":85327872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"234'34","reported_seq":363,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.683161+0000","last_change":"2024-09-11T07:37:16.938565+0000","last_active":"2024-09-11T07:40:47.683161+0000","last_peered":"2024-09-11T07:40:47.683161+0000","last_clean":"2024-09-11T07:40:47.683161+0000","last_became_active":"2024-09-11T07:26:33.549733+0000","last_became_peered":"2024-09-11T07:26:33.549733+0000","last_unstale":"2024-09-11T07:40:47.683161+0000","last_undegraded":"2024-09-11T07:40:47.683161+0000","last_fullsized":"2024-09-11T07:40:47.683161+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'34","last_scrub_stamp":"2024-09-11T07:37:16.938496+0000","last_deep_scrub":"83'22","last_deep_scrub_stamp":"2024-09-11T07:34:04.375457+0000","last_clean_scrub_stamp":"2024-09-11T07:37:16.938496+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028468529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":345,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.683041+0000","last_change":"2024-09-11T07:38:29.939925+0000","last_active":"2024-09-11T07:40:47.683041+0000","last_peered":"2024-09-11T07:40:47.683041+0000","last_clean":"2024-09-11T07:40:47.683041+0000","last_became_active":"2024-09-11T07:32:00.513472+0000","last_became_peered":"2024-09-11T07:32:00.513472+0000","last_unstale":"2024-09-11T07:40:47.683041+0000","last_undegraded":"2024-09-11T07:40:47.683041+0000","last_fullsized":"2024-09-11T07:40:47.683041+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:29.939826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:38:29.939826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017750210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":504,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.687598+0000","last_change":"2024-09-11T07:40:46.534047+0000","last_active":"2024-09-11T07:40:47.687598+0000","last_peered":"2024-09-11T07:40:47.687598+0000","last_clean":"2024-09-11T07:40:47.687598+0000","last_became_active":"2024-09-11T07:26:35.560406+0000","last_became_peered":"2024-09-11T07:26:35.560406+0000","last_unstale":"2024-09-11T07:40:47.687598+0000","last_undegraded":"2024-09-11T07:40:47.687598+0000","last_fullsized":"2024-09-11T07:40:47.687598+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:40:46.533966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:40:46.533966+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:51.243657+0000","scrub_duration":0.0050834449999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":320,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.682371+0000","last_change":"2024-09-11T07:39:57.762607+0000","last_active":"2024-09-11T07:40:47.682371+0000","last_peered":"2024-09-11T07:40:47.682371+0000","last_clean":"2024-09-11T07:40:47.682371+0000","last_became_active":"2024-09-11T07:39:57.762309+0000","last_became_peered":"2024-09-11T07:39:57.762309+0000","last_unstale":"2024-09-11T07:40:47.682371+0000","last_undegraded":"2024-09-11T07:40:47.682371+0000","last_fullsized":"2024-09-11T07:40:47.682371+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:20.369199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:31.200685+0000","last_clean_scrub_stamp":"2024-09-11T07:37:20.369199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001371688,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"315'9495","reported_seq":9584,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:49.137913+0000","last_change":"2024-09-11T07:40:45.528413+0000","last_active":"2024-09-11T07:40:49.137913+0000","last_peered":"2024-09-11T07:40:49.137913+0000","last_clean":"2024-09-11T07:40:49.137913+0000","last_became_active":"2024-09-11T07:38:27.361246+0000","last_became_peered":"2024-09-11T07:38:27.361246+0000","last_unstale":"2024-09-11T07:40:49.137913+0000","last_undegraded":"2024-09-11T07:40:49.137913+0000","last_fullsized":"2024-09-11T07:40:49.137913+0000","mapping_epoch":247,"log_start":"255'2822","ondisk_log_start":"255'2822","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6673,"log_dups_size":0,"ondisk_log_size":6673,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58359808,"num_objects":890,"num_object_clones":0,"num_object_copies":2670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6842,"num_write_kb":247456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"234'24","reported_seq":392,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.687638+0000","last_change":"2024-09-11T07:40:38.740191+0000","last_active":"2024-09-11T07:40:47.687638+0000","last_peered":"2024-09-11T07:40:47.687638+0000","last_clean":"2024-09-11T07:40:47.687638+0000","last_became_active":"2024-09-11T07:39:57.766134+0000","last_became_peered":"2024-09-11T07:39:57.766134+0000","last_unstale":"2024-09-11T07:40:47.687638+0000","last_undegraded":"2024-09-11T07:40:47.687638+0000","last_fullsized":"2024-09-11T07:40:47.687638+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'24","last_scrub_stamp":"2024-09-11T07:40:38.740127+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-09-11T07:29:38.112111+0000","last_clean_scrub_stamp":"2024-09-11T07:40:38.740127+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:52.114070+0000","scrub_duration":0.0044589670000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":313,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.677124+0000","last_change":"2024-09-11T07:39:36.451885+0000","last_active":"2024-09-11T07:40:47.677124+0000","last_peered":"2024-09-11T07:40:47.677124+0000","last_clean":"2024-09-11T07:40:47.677124+0000","last_became_active":"2024-09-11T07:32:00.518538+0000","last_became_peered":"2024-09-11T07:32:00.518538+0000","last_unstale":"2024-09-11T07:40:47.677124+0000","last_undegraded":"2024-09-11T07:40:47.677124+0000","last_fullsized":"2024-09-11T07:40:47.677124+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:37.444760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:37.444760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012636722,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":322,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.682619+0000","last_change":"2024-09-11T07:38:24.129487+0000","last_active":"2024-09-11T07:40:47.682619+0000","last_peered":"2024-09-11T07:40:47.682619+0000","last_clean":"2024-09-11T07:40:47.682619+0000","last_became_active":"2024-09-11T07:26:35.569081+0000","last_became_peered":"2024-09-11T07:26:35.569081+0000","last_unstale":"2024-09-11T07:40:47.682619+0000","last_undegraded":"2024-09-11T07:40:47.682619+0000","last_fullsized":"2024-09-11T07:40:47.682619+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:24.129047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:38:24.129047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003571196,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":311,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.701063+0000","last_change":"2024-09-11T07:36:50.920711+0000","last_active":"2024-09-11T07:40:48.701063+0000","last_peered":"2024-09-11T07:40:48.701063+0000","last_clean":"2024-09-11T07:40:48.701063+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:40:48.701063+0000","last_undegraded":"2024-09-11T07:40:48.701063+0000","last_fullsized":"2024-09-11T07:40:48.701063+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:50.920625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:46.853387+0000","last_clean_scrub_stamp":"2024-09-11T07:36:50.920625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012404324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"316'13846","reported_seq":13922,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-09-11T07:40:50.000397+0000","last_change":"2024-09-11T07:37:25.855661+0000","last_active":"2024-09-11T07:40:50.000397+0000","last_peered":"2024-09-11T07:40:50.000397+0000","last_clean":"2024-09-11T07:40:50.000397+0000","last_became_active":"2024-09-11T07:37:25.854796+0000","last_became_peered":"2024-09-11T07:37:25.854796+0000","last_unstale":"2024-09-11T07:40:50.000397+0000","last_undegraded":"2024-09-11T07:40:50.000397+0000","last_fullsized":"2024-09-11T07:40:50.000397+0000","mapping_epoch":247,"log_start":"278'9306","ondisk_log_start":"278'9306","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4540,"log_dups_size":0,"ondisk_log_size":4540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":119275520,"num_objects":1820,"num_object_clones":0,"num_object_copies":5460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13846,"num_write_kb":501312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"234'28","reported_seq":366,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.686585+0000","last_change":"2024-09-11T07:40:40.689386+0000","last_active":"2024-09-11T07:40:47.686585+0000","last_peered":"2024-09-11T07:40:47.686585+0000","last_clean":"2024-09-11T07:40:47.686585+0000","last_became_active":"2024-09-11T07:26:33.537850+0000","last_became_peered":"2024-09-11T07:26:33.537850+0000","last_unstale":"2024-09-11T07:40:47.686585+0000","last_undegraded":"2024-09-11T07:40:47.686585+0000","last_fullsized":"2024-09-11T07:40:47.686585+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'28","last_scrub_stamp":"2024-09-11T07:40:40.689301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:40:40.689301+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:48.883615+0000","scrub_duration":0.0071558330000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":92,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.685160+0000","last_change":"2024-09-11T07:39:57.768072+0000","last_active":"2024-09-11T07:40:47.685160+0000","last_peered":"2024-09-11T07:40:47.685160+0000","last_clean":"2024-09-11T07:40:47.685160+0000","last_became_active":"2024-09-11T07:39:57.767741+0000","last_became_peered":"2024-09-11T07:39:57.767741+0000","last_unstale":"2024-09-11T07:40:47.685160+0000","last_undegraded":"2024-09-11T07:40:47.685160+0000","last_fullsized":"2024-09-11T07:40:47.685160+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:49.609120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:39:49.609120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:40:52.200046+0000","scrub_duration":0.0032789239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3],"acting":[2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"84'9","reported_seq":494,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:49.675162+0000","last_change":"2024-09-11T07:37:37.000676+0000","last_active":"2024-09-11T07:40:49.675162+0000","last_peered":"2024-09-11T07:40:49.675162+0000","last_clean":"2024-09-11T07:40:49.675162+0000","last_became_active":"2024-09-11T07:26:35.562997+0000","last_became_peered":"2024-09-11T07:26:35.562997+0000","last_unstale":"2024-09-11T07:40:49.675162+0000","last_undegraded":"2024-09-11T07:40:49.675162+0000","last_fullsized":"2024-09-11T07:40:49.675162+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-11T07:37:37.000619+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-11T07:35:39.832547+0000","last_clean_scrub_stamp":"2024-09-11T07:37:37.000619+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021426219999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":294,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.677515+0000","last_change":"2024-09-11T07:39:35.486304+0000","last_active":"2024-09-11T07:40:47.677515+0000","last_peered":"2024-09-11T07:40:47.677515+0000","last_clean":"2024-09-11T07:40:47.677515+0000","last_became_active":"2024-09-11T07:32:00.511005+0000","last_became_peered":"2024-09-11T07:32:00.511005+0000","last_unstale":"2024-09-11T07:40:47.677515+0000","last_undegraded":"2024-09-11T07:40:47.677515+0000","last_fullsized":"2024-09-11T07:40:47.677515+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:35.486250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:39:35.486250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:40:48.848225+0000","scrub_duration":0.0042980279999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"315'13720","reported_seq":24313,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:49.144196+0000","last_change":"2024-09-11T07:40:27.473852+0000","last_active":"2024-09-11T07:40:49.144196+0000","last_peered":"2024-09-11T07:40:49.144196+0000","last_clean":"2024-09-11T07:40:49.144196+0000","last_became_active":"2024-09-11T07:40:27.407183+0000","last_became_peered":"2024-09-11T07:40:27.407183+0000","last_unstale":"2024-09-11T07:40:49.144196+0000","last_undegraded":"2024-09-11T07:40:49.144196+0000","last_fullsized":"2024-09-11T07:40:49.144196+0000","mapping_epoch":302,"log_start":"273'8444","ondisk_log_start":"273'8444","created":247,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":5276,"log_dups_size":0,"ondisk_log_size":5276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":119865368,"num_objects":1830,"num_object_clones":0,"num_object_copies":5490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1830,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13720,"num_write_kb":497537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3497,"num_bytes_recovered":229113880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"234'38","reported_seq":363,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.692943+0000","last_change":"2024-09-11T07:39:57.792218+0000","last_active":"2024-09-11T07:40:48.692943+0000","last_peered":"2024-09-11T07:40:48.692943+0000","last_clean":"2024-09-11T07:40:48.692943+0000","last_became_active":"2024-09-11T07:39:57.775541+0000","last_became_peered":"2024-09-11T07:39:57.775541+0000","last_unstale":"2024-09-11T07:40:48.692943+0000","last_undegraded":"2024-09-11T07:40:48.692943+0000","last_fullsized":"2024-09-11T07:40:48.692943+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'38","last_scrub_stamp":"2024-09-11T07:38:24.103612+0000","last_deep_scrub":"87'26","last_deep_scrub_stamp":"2024-09-11T07:35:52.942347+0000","last_clean_scrub_stamp":"2024-09-11T07:38:24.103612+0000","objects_scrubbed":8,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010845960999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":51,"num_read_kb":39,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":31,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.687687+0000","last_change":"2024-09-11T07:40:11.698799+0000","last_active":"2024-09-11T07:40:47.687687+0000","last_peered":"2024-09-11T07:40:47.687687+0000","last_clean":"2024-09-11T07:40:47.687687+0000","last_became_active":"2024-09-11T07:40:11.698558+0000","last_became_peered":"2024-09-11T07:40:11.698558+0000","last_unstale":"2024-09-11T07:40:47.687687+0000","last_undegraded":"2024-09-11T07:40:47.687687+0000","last_fullsized":"2024-09-11T07:40:47.687687+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:33.081815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:33.081815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":342,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.686413+0000","last_change":"2024-09-11T07:39:57.767026+0000","last_active":"2024-09-11T07:40:47.686413+0000","last_peered":"2024-09-11T07:40:47.686413+0000","last_clean":"2024-09-11T07:40:47.686413+0000","last_became_active":"2024-09-11T07:39:57.766717+0000","last_became_peered":"2024-09-11T07:39:57.766717+0000","last_unstale":"2024-09-11T07:40:47.686413+0000","last_undegraded":"2024-09-11T07:40:47.686413+0000","last_fullsized":"2024-09-11T07:40:47.686413+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:36.585153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:27.968579+0000","last_clean_scrub_stamp":"2024-09-11T07:39:36.585153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0067939640000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":330,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.687594+0000","last_change":"2024-09-11T07:39:57.692130+0000","last_active":"2024-09-11T07:40:47.687594+0000","last_peered":"2024-09-11T07:40:47.687594+0000","last_clean":"2024-09-11T07:40:47.687594+0000","last_became_active":"2024-09-11T07:39:57.691831+0000","last_became_peered":"2024-09-11T07:39:57.691831+0000","last_unstale":"2024-09-11T07:40:47.687594+0000","last_undegraded":"2024-09-11T07:40:47.687594+0000","last_fullsized":"2024-09-11T07:40:47.687594+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:51.576754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:29.978798+0000","last_clean_scrub_stamp":"2024-09-11T07:39:51.576754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:03.752909+0000","scrub_duration":0.0036155279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"315'13917","reported_seq":20348,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:49.107331+0000","last_change":"2024-09-11T07:40:39.907214+0000","last_active":"2024-09-11T07:40:49.107331+0000","last_peered":"2024-09-11T07:40:49.107331+0000","last_clean":"2024-09-11T07:40:49.107331+0000","last_became_active":"2024-09-11T07:40:34.480392+0000","last_became_peered":"2024-09-11T07:40:34.480392+0000","last_unstale":"2024-09-11T07:40:49.107331+0000","last_undegraded":"2024-09-11T07:40:49.107331+0000","last_fullsized":"2024-09-11T07:40:49.107331+0000","mapping_epoch":305,"log_start":"278'9499","ondisk_log_start":"278'9499","created":247,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"310'13316","last_scrub_stamp":"2024-09-11T07:40:39.907152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:40:39.907152+0000","objects_scrubbed":2362,"log_size":4418,"log_dups_size":0,"ondisk_log_size":4418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:05.320495+0000","scrub_duration":0.211466654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115408896,"num_objects":1761,"num_object_clones":0,"num_object_copies":5283,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13917,"num_write_kb":501696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3114,"num_bytes_recovered":204079104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"236'32","reported_seq":386,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.692062+0000","last_change":"2024-09-11T07:36:50.024003+0000","last_active":"2024-09-11T07:40:48.692062+0000","last_peered":"2024-09-11T07:40:48.692062+0000","last_clean":"2024-09-11T07:40:48.692062+0000","last_became_active":"2024-09-11T07:32:00.506270+0000","last_became_peered":"2024-09-11T07:32:00.506270+0000","last_unstale":"2024-09-11T07:40:48.692062+0000","last_undegraded":"2024-09-11T07:40:48.692062+0000","last_fullsized":"2024-09-11T07:40:48.692062+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'32","last_scrub_stamp":"2024-09-11T07:36:50.023909+0000","last_deep_scrub":"236'32","last_deep_scrub_stamp":"2024-09-11T07:36:50.023909+0000","last_clean_scrub_stamp":"2024-09-11T07:36:50.023909+0000","objects_scrubbed":8,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.030217668999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":294,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.679020+0000","last_change":"2024-09-11T07:39:57.281547+0000","last_active":"2024-09-11T07:40:47.679020+0000","last_peered":"2024-09-11T07:40:47.679020+0000","last_clean":"2024-09-11T07:40:47.679020+0000","last_became_active":"2024-09-11T07:32:00.510631+0000","last_became_peered":"2024-09-11T07:32:00.510631+0000","last_unstale":"2024-09-11T07:40:47.679020+0000","last_undegraded":"2024-09-11T07:40:47.679020+0000","last_fullsized":"2024-09-11T07:40:47.679020+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:57.281493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:49.144041+0000","last_clean_scrub_stamp":"2024-09-11T07:39:57.281493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:40:59.294334+0000","scrub_duration":0.001207484,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":124,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.692152+0000","last_change":"2024-09-11T07:40:45.799295+0000","last_active":"2024-09-11T07:40:48.692152+0000","last_peered":"2024-09-11T07:40:48.692152+0000","last_clean":"2024-09-11T07:40:48.692152+0000","last_became_active":"2024-09-11T07:36:05.225840+0000","last_became_peered":"2024-09-11T07:36:05.225840+0000","last_unstale":"2024-09-11T07:40:48.692152+0000","last_undegraded":"2024-09-11T07:40:48.692152+0000","last_fullsized":"2024-09-11T07:40:48.692152+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:45.799189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:40:45.799189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:54.908201+0000","scrub_duration":0.047262133999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":312,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.682790+0000","last_change":"2024-09-11T07:40:45.844765+0000","last_active":"2024-09-11T07:40:47.682790+0000","last_peered":"2024-09-11T07:40:47.682790+0000","last_clean":"2024-09-11T07:40:47.682790+0000","last_became_active":"2024-09-11T07:26:37.861279+0000","last_became_peered":"2024-09-11T07:26:37.861279+0000","last_unstale":"2024-09-11T07:40:47.682790+0000","last_undegraded":"2024-09-11T07:40:47.682790+0000","last_fullsized":"2024-09-11T07:40:47.682790+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:54.827454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:39.723598+0000","last_clean_scrub_stamp":"2024-09-11T07:36:54.827454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016642015,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.f","version":"315'13737","reported_seq":13812,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:49.104385+0000","last_change":"2024-09-11T07:37:25.856504+0000","last_active":"2024-09-11T07:40:49.104385+0000","last_peered":"2024-09-11T07:40:49.104385+0000","last_clean":"2024-09-11T07:40:49.104385+0000","last_became_active":"2024-09-11T07:37:25.856214+0000","last_became_peered":"2024-09-11T07:37:25.856214+0000","last_unstale":"2024-09-11T07:40:49.104385+0000","last_undegraded":"2024-09-11T07:40:49.104385+0000","last_fullsized":"2024-09-11T07:40:49.104385+0000","mapping_epoch":247,"log_start":"273'8441","ondisk_log_start":"273'8441","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":5296,"log_dups_size":0,"ondisk_log_size":5296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123928576,"num_objects":1891,"num_object_clones":0,"num_object_copies":5673,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13737,"num_write_kb":500096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"234'20","reported_seq":329,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.686276+0000","last_change":"2024-09-11T07:39:58.520372+0000","last_active":"2024-09-11T07:40:47.686276+0000","last_peered":"2024-09-11T07:40:47.686276+0000","last_clean":"2024-09-11T07:40:47.686276+0000","last_became_active":"2024-09-11T07:39:57.692071+0000","last_became_peered":"2024-09-11T07:39:57.692071+0000","last_unstale":"2024-09-11T07:40:47.686276+0000","last_undegraded":"2024-09-11T07:40:47.686276+0000","last_fullsized":"2024-09-11T07:40:47.686276+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'20","last_scrub_stamp":"2024-09-11T07:39:58.520311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:39:58.520311+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:12.044084+0000","scrub_duration":0.0024543820000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2],"acting":[4,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":352,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.696142+0000","last_change":"2024-09-11T07:40:44.863008+0000","last_active":"2024-09-11T07:40:47.696142+0000","last_peered":"2024-09-11T07:40:47.696142+0000","last_clean":"2024-09-11T07:40:47.696142+0000","last_became_active":"2024-09-11T07:32:00.522456+0000","last_became_peered":"2024-09-11T07:32:00.522456+0000","last_unstale":"2024-09-11T07:40:47.696142+0000","last_undegraded":"2024-09-11T07:40:47.696142+0000","last_fullsized":"2024-09-11T07:40:47.696142+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:40:44.862646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:40:44.862646+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:02.689591+0000","scrub_duration":0.005065918,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"290'9","reported_seq":517,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:49.675174+0000","last_change":"2024-09-11T07:39:57.780023+0000","last_active":"2024-09-11T07:40:49.675174+0000","last_peered":"2024-09-11T07:40:49.675174+0000","last_clean":"2024-09-11T07:40:49.675174+0000","last_became_active":"2024-09-11T07:39:57.768155+0000","last_became_peered":"2024-09-11T07:39:57.768155+0000","last_unstale":"2024-09-11T07:40:49.675174+0000","last_undegraded":"2024-09-11T07:40:49.675174+0000","last_fullsized":"2024-09-11T07:40:49.675174+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:37:26.141447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.141447+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015408170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":321,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.692386+0000","last_change":"2024-09-11T07:36:40.027067+0000","last_active":"2024-09-11T07:40:48.692386+0000","last_peered":"2024-09-11T07:40:48.692386+0000","last_clean":"2024-09-11T07:40:48.692386+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:40:48.692386+0000","last_undegraded":"2024-09-11T07:40:48.692386+0000","last_fullsized":"2024-09-11T07:40:48.692386+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:40.026982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_clean_scrub_stamp":"2024-09-11T07:36:40.026982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090395279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.9","version":"316'9820","reported_seq":9908,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-09-11T07:40:50.209999+0000","last_change":"2024-09-11T07:38:27.369273+0000","last_active":"2024-09-11T07:40:50.209999+0000","last_peered":"2024-09-11T07:40:50.209999+0000","last_clean":"2024-09-11T07:40:50.209999+0000","last_became_active":"2024-09-11T07:38:27.364955+0000","last_became_peered":"2024-09-11T07:38:27.364955+0000","last_unstale":"2024-09-11T07:40:50.209999+0000","last_undegraded":"2024-09-11T07:40:50.209999+0000","last_fullsized":"2024-09-11T07:40:50.209999+0000","mapping_epoch":247,"log_start":"260'5426","ondisk_log_start":"260'5426","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4394,"log_dups_size":0,"ondisk_log_size":4394,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58851328,"num_objects":898,"num_object_clones":0,"num_object_copies":2694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7112,"num_write_kb":256320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"290'42","reported_seq":243,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.683393+0000","last_change":"2024-09-11T07:40:33.449484+0000","last_active":"2024-09-11T07:40:47.683393+0000","last_peered":"2024-09-11T07:40:47.683393+0000","last_clean":"2024-09-11T07:40:47.683393+0000","last_became_active":"2024-09-11T07:39:57.768267+0000","last_became_peered":"2024-09-11T07:39:57.768267+0000","last_unstale":"2024-09-11T07:40:47.683393+0000","last_undegraded":"2024-09-11T07:40:47.683393+0000","last_fullsized":"2024-09-11T07:40:47.683393+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'41","last_scrub_stamp":"2024-09-11T07:36:41.343611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:36:41.343611+0000","objects_scrubbed":10,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0099986009999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":336,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.681833+0000","last_change":"2024-09-11T07:39:57.468860+0000","last_active":"2024-09-11T07:40:47.681833+0000","last_peered":"2024-09-11T07:40:47.681833+0000","last_clean":"2024-09-11T07:40:47.681833+0000","last_became_active":"2024-09-11T07:26:31.424618+0000","last_became_peered":"2024-09-11T07:26:31.424618+0000","last_unstale":"2024-09-11T07:40:47.681833+0000","last_undegraded":"2024-09-11T07:40:47.681833+0000","last_fullsized":"2024-09-11T07:40:47.681833+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:57.468791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:51.308492+0000","last_clean_scrub_stamp":"2024-09-11T07:39:57.468791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:21.538207+0000","scrub_duration":0.0013109230000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":323,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.693694+0000","last_change":"2024-09-11T07:37:33.144930+0000","last_active":"2024-09-11T07:40:48.693694+0000","last_peered":"2024-09-11T07:40:48.693694+0000","last_clean":"2024-09-11T07:40:48.693694+0000","last_became_active":"2024-09-11T07:26:35.560363+0000","last_became_peered":"2024-09-11T07:26:35.560363+0000","last_unstale":"2024-09-11T07:40:48.693694+0000","last_undegraded":"2024-09-11T07:40:48.693694+0000","last_fullsized":"2024-09-11T07:40:48.693694+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:33.144873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:33.144873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001754675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":328,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.691346+0000","last_change":"2024-09-11T07:37:39.053981+0000","last_active":"2024-09-11T07:40:48.691346+0000","last_peered":"2024-09-11T07:40:48.691346+0000","last_clean":"2024-09-11T07:40:48.691346+0000","last_became_active":"2024-09-11T07:32:00.505919+0000","last_became_peered":"2024-09-11T07:32:00.505919+0000","last_unstale":"2024-09-11T07:40:48.691346+0000","last_undegraded":"2024-09-11T07:40:48.691346+0000","last_fullsized":"2024-09-11T07:40:48.691346+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:39.053900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:39.053900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050142590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"315'14133","reported_seq":22021,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:49.199451+0000","last_change":"2024-09-11T07:40:13.749548+0000","last_active":"2024-09-11T07:40:49.199451+0000","last_peered":"2024-09-11T07:40:49.199451+0000","last_clean":"2024-09-11T07:40:49.199451+0000","last_became_active":"2024-09-11T07:40:13.698098+0000","last_became_peered":"2024-09-11T07:40:13.698098+0000","last_unstale":"2024-09-11T07:40:49.199451+0000","last_undegraded":"2024-09-11T07:40:49.199451+0000","last_fullsized":"2024-09-11T07:40:49.199451+0000","mapping_epoch":298,"log_start":"276'9418","ondisk_log_start":"276'9418","created":247,"last_epoch_clean":299,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4715,"log_dups_size":0,"ondisk_log_size":4715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":122880000,"num_objects":1875,"num_object_clones":0,"num_object_copies":5625,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14133,"num_write_kb":512256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3894,"num_bytes_recovered":255197184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"234'22","reported_seq":334,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.682853+0000","last_change":"2024-09-11T07:40:43.269210+0000","last_active":"2024-09-11T07:40:47.682853+0000","last_peered":"2024-09-11T07:40:47.682853+0000","last_clean":"2024-09-11T07:40:47.682853+0000","last_became_active":"2024-09-11T07:40:11.704155+0000","last_became_peered":"2024-09-11T07:40:11.704155+0000","last_unstale":"2024-09-11T07:40:47.682853+0000","last_undegraded":"2024-09-11T07:40:47.682853+0000","last_fullsized":"2024-09-11T07:40:47.682853+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'22","last_scrub_stamp":"2024-09-11T07:37:20.965147+0000","last_deep_scrub":"144'18","last_deep_scrub_stamp":"2024-09-11T07:31:51.603378+0000","last_clean_scrub_stamp":"2024-09-11T07:37:20.965147+0000","objects_scrubbed":5,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006530999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":1182,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.694581+0000","last_change":"2024-09-11T07:37:22.990093+0000","last_active":"2024-09-11T07:40:48.694581+0000","last_peered":"2024-09-11T07:40:48.694581+0000","last_clean":"2024-09-11T07:40:48.694581+0000","last_became_active":"2024-09-11T07:26:31.421276+0000","last_became_peered":"2024-09-11T07:26:31.421276+0000","last_unstale":"2024-09-11T07:40:48.694581+0000","last_undegraded":"2024-09-11T07:40:48.694581+0000","last_fullsized":"2024-09-11T07:40:48.694581+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'5","last_scrub_stamp":"2024-09-11T07:37:22.990033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:22.990033+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001646954,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":849,"num_read_kb":844,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":355,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.683875+0000","last_change":"2024-09-11T07:40:38.385663+0000","last_active":"2024-09-11T07:40:47.683875+0000","last_peered":"2024-09-11T07:40:47.683875+0000","last_clean":"2024-09-11T07:40:47.683875+0000","last_became_active":"2024-09-11T07:26:31.413354+0000","last_became_peered":"2024-09-11T07:26:31.413354+0000","last_unstale":"2024-09-11T07:40:47.683875+0000","last_undegraded":"2024-09-11T07:40:47.683875+0000","last_fullsized":"2024-09-11T07:40:47.683875+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:38.385583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:07.149367+0000","last_clean_scrub_stamp":"2024-09-11T07:40:38.385583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:53.440444+0000","scrub_duration":0.014349707999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":342,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.683334+0000","last_change":"2024-09-11T07:39:45.351040+0000","last_active":"2024-09-11T07:40:47.683334+0000","last_peered":"2024-09-11T07:40:47.683334+0000","last_clean":"2024-09-11T07:40:47.683334+0000","last_became_active":"2024-09-11T07:32:00.508398+0000","last_became_peered":"2024-09-11T07:32:00.508398+0000","last_unstale":"2024-09-11T07:40:47.683334+0000","last_undegraded":"2024-09-11T07:40:47.683334+0000","last_fullsized":"2024-09-11T07:40:47.683334+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:24.876930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:38:24.876930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0095042219999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":320,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.678734+0000","last_change":"2024-09-11T07:39:49.399814+0000","last_active":"2024-09-11T07:40:47.678734+0000","last_peered":"2024-09-11T07:40:47.678734+0000","last_clean":"2024-09-11T07:40:47.678734+0000","last_became_active":"2024-09-11T07:32:00.515745+0000","last_became_peered":"2024-09-11T07:32:00.515745+0000","last_unstale":"2024-09-11T07:40:47.678734+0000","last_undegraded":"2024-09-11T07:40:47.678734+0000","last_fullsized":"2024-09-11T07:40:47.678734+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:49.399708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:56.490940+0000","last_clean_scrub_stamp":"2024-09-11T07:39:49.399708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:16.075069+0000","scrub_duration":0.0053800089999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"316'14231","reported_seq":14308,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-09-11T07:40:50.282935+0000","last_change":"2024-09-11T07:37:26.029935+0000","last_active":"2024-09-11T07:40:50.282935+0000","last_peered":"2024-09-11T07:40:50.282935+0000","last_clean":"2024-09-11T07:40:50.282935+0000","last_became_active":"2024-09-11T07:37:26.029608+0000","last_became_peered":"2024-09-11T07:37:26.029608+0000","last_unstale":"2024-09-11T07:40:50.282935+0000","last_undegraded":"2024-09-11T07:40:50.282935+0000","last_fullsized":"2024-09-11T07:40:50.282935+0000","mapping_epoch":247,"log_start":"281'9844","ondisk_log_start":"281'9844","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4387,"log_dups_size":0,"ondisk_log_size":4387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117768192,"num_objects":1797,"num_object_clones":0,"num_object_copies":5391,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14231,"num_write_kb":512896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"234'30","reported_seq":401,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.694433+0000","last_change":"2024-09-11T07:40:34.716884+0000","last_active":"2024-09-11T07:40:48.694433+0000","last_peered":"2024-09-11T07:40:48.694433+0000","last_clean":"2024-09-11T07:40:48.694433+0000","last_became_active":"2024-09-11T07:28:09.018134+0000","last_became_peered":"2024-09-11T07:28:09.018134+0000","last_unstale":"2024-09-11T07:40:48.694433+0000","last_undegraded":"2024-09-11T07:40:48.694433+0000","last_fullsized":"2024-09-11T07:40:48.694433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'30","last_scrub_stamp":"2024-09-11T07:40:34.716804+0000","last_deep_scrub":"94'18","last_deep_scrub_stamp":"2024-09-11T07:28:51.437397+0000","last_clean_scrub_stamp":"2024-09-11T07:40:34.716804+0000","objects_scrubbed":6,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:00.562358+0000","scrub_duration":0.0055680800000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":347,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.692014+0000","last_change":"2024-09-11T07:39:57.769695+0000","last_active":"2024-09-11T07:40:48.692014+0000","last_peered":"2024-09-11T07:40:48.692014+0000","last_clean":"2024-09-11T07:40:48.692014+0000","last_became_active":"2024-09-11T07:39:57.758029+0000","last_became_peered":"2024-09-11T07:39:57.758029+0000","last_unstale":"2024-09-11T07:40:48.692014+0000","last_undegraded":"2024-09-11T07:40:48.692014+0000","last_fullsized":"2024-09-11T07:40:48.692014+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-11T07:38:21.385813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:38:21.385813+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060130330000000001,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":331,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.677262+0000","last_change":"2024-09-11T07:39:57.772569+0000","last_active":"2024-09-11T07:40:47.677262+0000","last_peered":"2024-09-11T07:40:47.677262+0000","last_clean":"2024-09-11T07:40:47.677262+0000","last_became_active":"2024-09-11T07:39:57.772244+0000","last_became_peered":"2024-09-11T07:39:57.772244+0000","last_unstale":"2024-09-11T07:40:47.677262+0000","last_undegraded":"2024-09-11T07:40:47.677262+0000","last_fullsized":"2024-09-11T07:40:47.677262+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:26.556972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:16.967492+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.556972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001503206,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":39,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.682938+0000","last_change":"2024-09-11T07:39:57.688960+0000","last_active":"2024-09-11T07:40:47.682938+0000","last_peered":"2024-09-11T07:40:47.682938+0000","last_clean":"2024-09-11T07:40:47.682938+0000","last_became_active":"2024-09-11T07:39:57.688619+0000","last_became_peered":"2024-09-11T07:39:57.688619+0000","last_unstale":"2024-09-11T07:40:47.682938+0000","last_undegraded":"2024-09-11T07:40:47.682938+0000","last_fullsized":"2024-09-11T07:40:47.682938+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:23.100768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:51.187127+0000","last_clean_scrub_stamp":"2024-09-11T07:38:23.100768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":39,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.681526+0000","last_change":"2024-09-11T07:39:57.772562+0000","last_active":"2024-09-11T07:40:47.681526+0000","last_peered":"2024-09-11T07:40:47.681526+0000","last_clean":"2024-09-11T07:40:47.681526+0000","last_became_active":"2024-09-11T07:39:57.772223+0000","last_became_peered":"2024-09-11T07:39:57.772223+0000","last_unstale":"2024-09-11T07:40:47.681526+0000","last_undegraded":"2024-09-11T07:40:47.681526+0000","last_fullsized":"2024-09-11T07:40:47.681526+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:25.907711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:38:25.907711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"316'9709","reported_seq":9813,"reported_epoch":316,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-11T07:40:50.246270+0000","last_change":"2024-09-11T07:40:28.945683+0000","last_active":"2024-09-11T07:40:50.246270+0000","last_peered":"2024-09-11T07:40:50.246270+0000","last_clean":"2024-09-11T07:39:56.626867+0000","last_became_active":"2024-09-11T07:39:58.903159+0000","last_became_peered":"2024-09-11T07:39:58.903159+0000","last_unstale":"2024-09-11T07:40:50.246270+0000","last_undegraded":"2024-09-11T07:40:50.246270+0000","last_fullsized":"2024-09-11T07:40:50.246270+0000","mapping_epoch":289,"log_start":"258'5324","ondisk_log_start":"258'5324","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4385,"log_dups_size":0,"ondisk_log_size":4385,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56557568,"num_objects":863,"num_object_clones":0,"num_object_copies":2589,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1726,"num_objects_unfound":0,"num_objects_dirty":863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7035,"num_write_kb":252736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,3,5],"avail_no_missing":["0","3","5"],"object_location_counts":[{"shards":"0,3,5","objects":863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"234'26","reported_seq":165,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.687347+0000","last_change":"2024-09-11T07:37:40.649065+0000","last_active":"2024-09-11T07:40:47.687347+0000","last_peered":"2024-09-11T07:40:47.687347+0000","last_clean":"2024-09-11T07:40:47.687347+0000","last_became_active":"2024-09-11T07:36:05.224704+0000","last_became_peered":"2024-09-11T07:36:05.224704+0000","last_unstale":"2024-09-11T07:40:47.687347+0000","last_undegraded":"2024-09-11T07:40:47.687347+0000","last_fullsized":"2024-09-11T07:40:47.687347+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'26","last_scrub_stamp":"2024-09-11T07:37:40.648986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:40.648986+0000","objects_scrubbed":7,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030578620000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'3","reported_seq":339,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.694295+0000","last_change":"2024-09-11T07:40:11.706429+0000","last_active":"2024-09-11T07:40:48.694295+0000","last_peered":"2024-09-11T07:40:48.694295+0000","last_clean":"2024-09-11T07:40:48.694295+0000","last_became_active":"2024-09-11T07:40:11.701416+0000","last_became_peered":"2024-09-11T07:40:11.701416+0000","last_unstale":"2024-09-11T07:40:48.694295+0000","last_undegraded":"2024-09-11T07:40:48.694295+0000","last_fullsized":"2024-09-11T07:40:48.694295+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-09-11T07:36:54.937201+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_clean_scrub_stamp":"2024-09-11T07:36:54.937201+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007045239,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":346,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.681978+0000","last_change":"2024-09-11T07:40:11.706063+0000","last_active":"2024-09-11T07:40:47.681978+0000","last_peered":"2024-09-11T07:40:47.681978+0000","last_clean":"2024-09-11T07:40:47.681978+0000","last_became_active":"2024-09-11T07:40:11.705286+0000","last_became_peered":"2024-09-11T07:40:11.705286+0000","last_unstale":"2024-09-11T07:40:47.681978+0000","last_undegraded":"2024-09-11T07:40:47.681978+0000","last_fullsized":"2024-09-11T07:40:47.681978+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:35.504238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:35.504238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013066169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"87'81","reported_seq":164,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.693357+0000","last_change":"2024-09-11T07:40:38.751275+0000","last_active":"2024-09-11T07:40:48.693357+0000","last_peered":"2024-09-11T07:40:48.693357+0000","last_clean":"2024-09-11T07:40:48.693357+0000","last_became_active":"2024-09-11T07:39:57.761901+0000","last_became_peered":"2024-09-11T07:39:57.761901+0000","last_unstale":"2024-09-11T07:40:48.693357+0000","last_undegraded":"2024-09-11T07:40:48.693357+0000","last_fullsized":"2024-09-11T07:40:48.693357+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:40:38.751193+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:40:38.751193+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:44.042328+0000","scrub_duration":0.010400779000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":142,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.683648+0000","last_change":"2024-09-11T07:39:57.749041+0000","last_active":"2024-09-11T07:40:47.683648+0000","last_peered":"2024-09-11T07:40:47.683648+0000","last_clean":"2024-09-11T07:40:47.683648+0000","last_became_active":"2024-09-11T07:39:57.748658+0000","last_became_peered":"2024-09-11T07:39:57.748658+0000","last_unstale":"2024-09-11T07:40:47.683648+0000","last_undegraded":"2024-09-11T07:40:47.683648+0000","last_fullsized":"2024-09-11T07:40:47.683648+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:41.562322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:41.562322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":313,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.682695+0000","last_change":"2024-09-11T07:40:41.208608+0000","last_active":"2024-09-11T07:40:47.682695+0000","last_peered":"2024-09-11T07:40:47.682695+0000","last_clean":"2024-09-11T07:40:47.682695+0000","last_became_active":"2024-09-11T07:26:37.559920+0000","last_became_peered":"2024-09-11T07:26:37.559920+0000","last_unstale":"2024-09-11T07:40:47.682695+0000","last_undegraded":"2024-09-11T07:40:47.682695+0000","last_fullsized":"2024-09-11T07:40:47.682695+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:23.436012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:23.436012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001213481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.7","version":"315'9622","reported_seq":9726,"reported_epoch":315,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-11T07:40:49.148051+0000","last_change":"2024-09-11T07:40:29.005492+0000","last_active":"2024-09-11T07:40:49.148051+0000","last_peered":"2024-09-11T07:40:49.148051+0000","last_clean":"2024-09-11T07:39:56.662527+0000","last_became_active":"2024-09-11T07:39:58.961246+0000","last_became_peered":"2024-09-11T07:39:58.961246+0000","last_unstale":"2024-09-11T07:40:49.148051+0000","last_undegraded":"2024-09-11T07:40:49.148051+0000","last_fullsized":"2024-09-11T07:40:49.148051+0000","mapping_epoch":289,"log_start":"255'3279","ondisk_log_start":"255'3279","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6343,"log_dups_size":0,"ondisk_log_size":6343,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60817408,"num_objects":928,"num_object_clones":0,"num_object_copies":2784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1856,"num_objects_unfound":0,"num_objects_dirty":928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6918,"num_write_kb":251072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,3,7],"avail_no_missing":["5","3","7"],"object_location_counts":[{"shards":"3,5,7","objects":928}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"234'35","reported_seq":210,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.695313+0000","last_change":"2024-09-11T07:39:57.784855+0000","last_active":"2024-09-11T07:40:47.695313+0000","last_peered":"2024-09-11T07:40:47.695313+0000","last_clean":"2024-09-11T07:40:47.695313+0000","last_became_active":"2024-09-11T07:39:57.759856+0000","last_became_peered":"2024-09-11T07:39:57.759856+0000","last_unstale":"2024-09-11T07:40:47.695313+0000","last_undegraded":"2024-09-11T07:40:47.695313+0000","last_fullsized":"2024-09-11T07:40:47.695313+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'35","last_scrub_stamp":"2024-09-11T07:38:23.944683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:38:23.944683+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003078154,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":348,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.683716+0000","last_change":"2024-09-11T07:40:35.376311+0000","last_active":"2024-09-11T07:40:47.683716+0000","last_peered":"2024-09-11T07:40:47.683716+0000","last_clean":"2024-09-11T07:40:47.683716+0000","last_became_active":"2024-09-11T07:26:31.419560+0000","last_became_peered":"2024-09-11T07:26:31.419560+0000","last_unstale":"2024-09-11T07:40:47.683716+0000","last_undegraded":"2024-09-11T07:40:47.683716+0000","last_fullsized":"2024-09-11T07:40:47.683716+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:35.376219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:40:35.376219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:44.257357+0000","scrub_duration":0.003361682,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":345,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.682969+0000","last_change":"2024-09-11T07:40:40.411425+0000","last_active":"2024-09-11T07:40:47.682969+0000","last_peered":"2024-09-11T07:40:47.682969+0000","last_clean":"2024-09-11T07:40:47.682969+0000","last_became_active":"2024-09-11T07:39:57.748953+0000","last_became_peered":"2024-09-11T07:39:57.748953+0000","last_unstale":"2024-09-11T07:40:47.682969+0000","last_undegraded":"2024-09-11T07:40:47.682969+0000","last_fullsized":"2024-09-11T07:40:47.682969+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:40.411364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:40:40.411364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:40.631026+0000","scrub_duration":0.0012767259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":150,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.684203+0000","last_change":"2024-09-11T07:39:57.767088+0000","last_active":"2024-09-11T07:40:47.684203+0000","last_peered":"2024-09-11T07:40:47.684203+0000","last_clean":"2024-09-11T07:40:47.684203+0000","last_became_active":"2024-09-11T07:39:57.766605+0000","last_became_peered":"2024-09-11T07:39:57.766605+0000","last_unstale":"2024-09-11T07:40:47.684203+0000","last_undegraded":"2024-09-11T07:40:47.684203+0000","last_fullsized":"2024-09-11T07:40:47.684203+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:41.428866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:39:41.428866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:40:52.853076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"315'9966","reported_seq":10054,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:49.057103+0000","last_change":"2024-09-11T07:40:45.283673+0000","last_active":"2024-09-11T07:40:49.057103+0000","last_peered":"2024-09-11T07:40:49.057103+0000","last_clean":"2024-09-11T07:40:49.057103+0000","last_became_active":"2024-09-11T07:38:27.351200+0000","last_became_peered":"2024-09-11T07:38:27.351200+0000","last_unstale":"2024-09-11T07:40:49.057103+0000","last_undegraded":"2024-09-11T07:40:49.057103+0000","last_fullsized":"2024-09-11T07:40:49.057103+0000","mapping_epoch":247,"log_start":"256'3872","ondisk_log_start":"256'3872","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6094,"log_dups_size":0,"ondisk_log_size":6094,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60030976,"num_objects":916,"num_object_clones":0,"num_object_copies":2748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7202,"num_write_kb":259776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"234'15","reported_seq":350,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.691986+0000","last_change":"2024-09-11T07:40:46.759444+0000","last_active":"2024-09-11T07:40:48.691986+0000","last_peered":"2024-09-11T07:40:48.691986+0000","last_clean":"2024-09-11T07:40:48.691986+0000","last_became_active":"2024-09-11T07:26:33.543148+0000","last_became_peered":"2024-09-11T07:26:33.543148+0000","last_unstale":"2024-09-11T07:40:48.691986+0000","last_undegraded":"2024-09-11T07:40:48.691986+0000","last_fullsized":"2024-09-11T07:40:48.691986+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:40:46.759189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:40:46.759189+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:55.039295+0000","scrub_duration":0.007573715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":325,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.687188+0000","last_change":"2024-09-11T07:37:26.019680+0000","last_active":"2024-09-11T07:40:47.687188+0000","last_peered":"2024-09-11T07:40:47.687188+0000","last_clean":"2024-09-11T07:40:47.687188+0000","last_became_active":"2024-09-11T07:26:31.423056+0000","last_became_peered":"2024-09-11T07:26:31.423056+0000","last_unstale":"2024-09-11T07:40:47.687188+0000","last_undegraded":"2024-09-11T07:40:47.687188+0000","last_fullsized":"2024-09-11T07:40:47.687188+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:26.019623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.019623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013393599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":341,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.684289+0000","last_change":"2024-09-11T07:39:52.485307+0000","last_active":"2024-09-11T07:40:47.684289+0000","last_peered":"2024-09-11T07:40:47.684289+0000","last_clean":"2024-09-11T07:40:47.684289+0000","last_became_active":"2024-09-11T07:26:35.563182+0000","last_became_peered":"2024-09-11T07:26:35.563182+0000","last_unstale":"2024-09-11T07:40:47.684289+0000","last_undegraded":"2024-09-11T07:40:47.684289+0000","last_fullsized":"2024-09-11T07:40:47.684289+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:52.485217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:39:52.485217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:10.747603+0000","scrub_duration":0.0013231569999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":328,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.691938+0000","last_change":"2024-09-11T07:36:36.023079+0000","last_active":"2024-09-11T07:40:48.691938+0000","last_peered":"2024-09-11T07:40:48.691938+0000","last_clean":"2024-09-11T07:40:48.691938+0000","last_became_active":"2024-09-11T07:32:00.505185+0000","last_became_peered":"2024-09-11T07:32:00.505185+0000","last_unstale":"2024-09-11T07:40:48.691938+0000","last_undegraded":"2024-09-11T07:40:48.691938+0000","last_fullsized":"2024-09-11T07:40:48.691938+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:36.023006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:46.944763+0000","last_clean_scrub_stamp":"2024-09-11T07:36:36.023006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011934019000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.5","version":"316'9866","reported_seq":9970,"reported_epoch":316,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-11T07:40:50.267013+0000","last_change":"2024-09-11T07:40:28.981797+0000","last_active":"2024-09-11T07:40:50.267013+0000","last_peered":"2024-09-11T07:40:50.267013+0000","last_clean":"2024-09-11T07:39:56.563478+0000","last_became_active":"2024-09-11T07:39:58.941275+0000","last_became_peered":"2024-09-11T07:39:58.941275+0000","last_unstale":"2024-09-11T07:40:50.267013+0000","last_undegraded":"2024-09-11T07:40:50.267013+0000","last_fullsized":"2024-09-11T07:40:50.267013+0000","mapping_epoch":289,"log_start":"264'5492","ondisk_log_start":"264'5492","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4374,"log_dups_size":0,"ondisk_log_size":4374,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720256,"num_objects":896,"num_object_clones":0,"num_object_copies":2688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1792,"num_objects_unfound":0,"num_objects_dirty":896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7152,"num_write_kb":257536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,5,7],"avail_no_missing":["0","5","7"],"object_location_counts":[{"shards":"0,5,7","objects":896}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"234'27","reported_seq":69,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.685987+0000","last_change":"2024-09-11T07:40:34.735380+0000","last_active":"2024-09-11T07:40:47.685987+0000","last_peered":"2024-09-11T07:40:47.685987+0000","last_clean":"2024-09-11T07:40:47.685987+0000","last_became_active":"2024-09-11T07:39:57.766550+0000","last_became_peered":"2024-09-11T07:39:57.766550+0000","last_unstale":"2024-09-11T07:40:47.685987+0000","last_undegraded":"2024-09-11T07:40:47.685987+0000","last_fullsized":"2024-09-11T07:40:47.685987+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'27","last_scrub_stamp":"2024-09-11T07:40:34.735292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:40:34.735292+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:40.999720+0000","scrub_duration":0.0055354389999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":383,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.683499+0000","last_change":"2024-09-11T07:40:11.702110+0000","last_active":"2024-09-11T07:40:47.683499+0000","last_peered":"2024-09-11T07:40:47.683499+0000","last_clean":"2024-09-11T07:40:47.683499+0000","last_became_active":"2024-09-11T07:40:11.701330+0000","last_became_peered":"2024-09-11T07:40:11.701330+0000","last_unstale":"2024-09-11T07:40:47.683499+0000","last_undegraded":"2024-09-11T07:40:47.683499+0000","last_fullsized":"2024-09-11T07:40:47.683499+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:46.402060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_clean_scrub_stamp":"2024-09-11T07:39:46.402060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:14.503163+0000","scrub_duration":0.0055267119999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"151'10","reported_seq":452,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.677543+0000","last_change":"2024-09-11T07:37:38.544127+0000","last_active":"2024-09-11T07:40:47.677543+0000","last_peered":"2024-09-11T07:40:47.677543+0000","last_clean":"2024-09-11T07:40:47.677543+0000","last_became_active":"2024-09-11T07:32:00.506806+0000","last_became_peered":"2024-09-11T07:32:00.506806+0000","last_unstale":"2024-09-11T07:40:47.677543+0000","last_undegraded":"2024-09-11T07:40:47.677543+0000","last_fullsized":"2024-09-11T07:40:47.677543+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:37:38.544072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:38.544072+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027401579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":313,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.681280+0000","last_change":"2024-09-11T07:40:33.314374+0000","last_active":"2024-09-11T07:40:47.681280+0000","last_peered":"2024-09-11T07:40:47.681280+0000","last_clean":"2024-09-11T07:40:47.681280+0000","last_became_active":"2024-09-11T07:26:37.861134+0000","last_became_peered":"2024-09-11T07:26:37.861134+0000","last_unstale":"2024-09-11T07:40:47.681280+0000","last_undegraded":"2024-09-11T07:40:47.681280+0000","last_fullsized":"2024-09-11T07:40:47.681280+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:44.293653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:44.293653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0074032500000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.4","version":"315'9755","reported_seq":9849,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:49.093327+0000","last_change":"2024-09-11T07:40:46.843129+0000","last_active":"2024-09-11T07:40:49.093327+0000","last_peered":"2024-09-11T07:40:49.093327+0000","last_clean":"2024-09-11T07:40:49.093327+0000","last_became_active":"2024-09-11T07:38:27.372060+0000","last_became_peered":"2024-09-11T07:38:27.372060+0000","last_unstale":"2024-09-11T07:40:49.093327+0000","last_undegraded":"2024-09-11T07:40:49.093327+0000","last_fullsized":"2024-09-11T07:40:49.093327+0000","mapping_epoch":247,"log_start":"256'4904","ondisk_log_start":"256'4904","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4851,"log_dups_size":0,"ondisk_log_size":4851,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720256,"num_objects":896,"num_object_clones":0,"num_object_copies":2688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7068,"num_write_kb":254848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"234'23","reported_seq":383,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.686490+0000","last_change":"2024-09-11T07:39:57.768799+0000","last_active":"2024-09-11T07:40:47.686490+0000","last_peered":"2024-09-11T07:40:47.686490+0000","last_clean":"2024-09-11T07:40:47.686490+0000","last_became_active":"2024-09-11T07:39:57.691713+0000","last_became_peered":"2024-09-11T07:39:57.691713+0000","last_unstale":"2024-09-11T07:40:47.686490+0000","last_undegraded":"2024-09-11T07:40:47.686490+0000","last_fullsized":"2024-09-11T07:40:47.686490+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'23","last_scrub_stamp":"2024-09-11T07:39:55.578519+0000","last_deep_scrub":"234'23","last_deep_scrub_stamp":"2024-09-11T07:37:34.613280+0000","last_clean_scrub_stamp":"2024-09-11T07:39:55.578519+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:15.441265+0000","scrub_duration":0.0030590740000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":349,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.687312+0000","last_change":"2024-09-11T07:39:27.582662+0000","last_active":"2024-09-11T07:40:47.687312+0000","last_peered":"2024-09-11T07:40:47.687312+0000","last_clean":"2024-09-11T07:40:47.687312+0000","last_became_active":"2024-09-11T07:32:00.504344+0000","last_became_peered":"2024-09-11T07:32:00.504344+0000","last_unstale":"2024-09-11T07:40:47.687312+0000","last_undegraded":"2024-09-11T07:40:47.687312+0000","last_fullsized":"2024-09-11T07:40:47.687312+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:35:46.769070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:35:46.769070+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049213720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":341,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.684266+0000","last_change":"2024-09-11T07:39:55.453328+0000","last_active":"2024-09-11T07:40:47.684266+0000","last_peered":"2024-09-11T07:40:47.684266+0000","last_clean":"2024-09-11T07:40:47.684266+0000","last_became_active":"2024-09-11T07:26:35.561967+0000","last_became_peered":"2024-09-11T07:26:35.561967+0000","last_unstale":"2024-09-11T07:40:47.684266+0000","last_undegraded":"2024-09-11T07:40:47.684266+0000","last_fullsized":"2024-09-11T07:40:47.684266+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:55.453221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:44.820354+0000","last_clean_scrub_stamp":"2024-09-11T07:39:55.453221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:11.983136+0000","scrub_duration":0.0014898649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":338,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.697464+0000","last_change":"2024-09-11T07:39:57.776965+0000","last_active":"2024-09-11T07:40:48.697464+0000","last_peered":"2024-09-11T07:40:48.697464+0000","last_clean":"2024-09-11T07:40:48.697464+0000","last_became_active":"2024-09-11T07:39:57.776576+0000","last_became_peered":"2024-09-11T07:39:57.776576+0000","last_unstale":"2024-09-11T07:40:48.697464+0000","last_undegraded":"2024-09-11T07:40:48.697464+0000","last_fullsized":"2024-09-11T07:40:48.697464+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:16.860930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:16.860930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024991000000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"234'15","reported_seq":377,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.694264+0000","last_change":"2024-09-11T07:39:57.786497+0000","last_active":"2024-09-11T07:40:48.694264+0000","last_peered":"2024-09-11T07:40:48.694264+0000","last_clean":"2024-09-11T07:40:48.694264+0000","last_became_active":"2024-09-11T07:39:57.767661+0000","last_became_peered":"2024-09-11T07:39:57.767661+0000","last_unstale":"2024-09-11T07:40:48.694264+0000","last_undegraded":"2024-09-11T07:40:48.694264+0000","last_fullsized":"2024-09-11T07:40:48.694264+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:37:19.096817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:19.096817+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00175015,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":345,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.684381+0000","last_change":"2024-09-11T07:39:57.443485+0000","last_active":"2024-09-11T07:40:47.684381+0000","last_peered":"2024-09-11T07:40:47.684381+0000","last_clean":"2024-09-11T07:40:47.684381+0000","last_became_active":"2024-09-11T07:26:31.415701+0000","last_became_peered":"2024-09-11T07:26:31.415701+0000","last_unstale":"2024-09-11T07:40:47.684381+0000","last_undegraded":"2024-09-11T07:40:47.684381+0000","last_fullsized":"2024-09-11T07:40:47.684381+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:57.443424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.925200+0000","last_clean_scrub_stamp":"2024-09-11T07:39:57.443424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:03.724321+0000","scrub_duration":0.0013832989999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":165,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.682864+0000","last_change":"2024-09-11T07:40:39.893675+0000","last_active":"2024-09-11T07:40:47.682864+0000","last_peered":"2024-09-11T07:40:47.682864+0000","last_clean":"2024-09-11T07:40:47.682864+0000","last_became_active":"2024-09-11T07:40:11.711154+0000","last_became_peered":"2024-09-11T07:40:11.711154+0000","last_unstale":"2024-09-11T07:40:47.682864+0000","last_undegraded":"2024-09-11T07:40:47.682864+0000","last_fullsized":"2024-09-11T07:40:47.682864+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:39.893619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:35.392360+0000","last_clean_scrub_stamp":"2024-09-11T07:40:39.893619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:46.233742+0000","scrub_duration":0.001330475,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":338,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.683200+0000","last_change":"2024-09-11T07:38:29.941071+0000","last_active":"2024-09-11T07:40:47.683200+0000","last_peered":"2024-09-11T07:40:47.683200+0000","last_clean":"2024-09-11T07:40:47.683200+0000","last_became_active":"2024-09-11T07:32:00.506697+0000","last_became_peered":"2024-09-11T07:32:00.506697+0000","last_unstale":"2024-09-11T07:40:47.683200+0000","last_undegraded":"2024-09-11T07:40:47.683200+0000","last_fullsized":"2024-09-11T07:40:47.683200+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:38:29.941008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:38:29.941008+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016886850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":116,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.686584+0000","last_change":"2024-09-11T07:39:46.613784+0000","last_active":"2024-09-11T07:40:47.686584+0000","last_peered":"2024-09-11T07:40:47.686584+0000","last_clean":"2024-09-11T07:40:47.686584+0000","last_became_active":"2024-09-11T07:36:05.225791+0000","last_became_peered":"2024-09-11T07:36:05.225791+0000","last_unstale":"2024-09-11T07:40:47.686584+0000","last_undegraded":"2024-09-11T07:40:47.686584+0000","last_fullsized":"2024-09-11T07:40:47.686584+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:51.617730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:51.617730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001776197,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"234'15","reported_seq":88,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.683074+0000","last_change":"2024-09-11T07:39:57.773581+0000","last_active":"2024-09-11T07:40:47.683074+0000","last_peered":"2024-09-11T07:40:47.683074+0000","last_clean":"2024-09-11T07:40:47.683074+0000","last_became_active":"2024-09-11T07:39:57.691830+0000","last_became_peered":"2024-09-11T07:39:57.691830+0000","last_unstale":"2024-09-11T07:40:47.683074+0000","last_undegraded":"2024-09-11T07:40:47.683074+0000","last_fullsized":"2024-09-11T07:40:47.683074+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:37:26.450122+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.450122+0000","objects_scrubbed":3,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":332,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.682585+0000","last_change":"2024-09-11T07:40:39.267299+0000","last_active":"2024-09-11T07:40:47.682585+0000","last_peered":"2024-09-11T07:40:47.682585+0000","last_clean":"2024-09-11T07:40:47.682585+0000","last_became_active":"2024-09-11T07:26:35.566894+0000","last_became_peered":"2024-09-11T07:26:35.566894+0000","last_unstale":"2024-09-11T07:40:47.682585+0000","last_undegraded":"2024-09-11T07:40:47.682585+0000","last_fullsized":"2024-09-11T07:40:47.682585+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:39.267248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:40:39.267248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:55.182199+0000","scrub_duration":0.0012024570000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":318,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.679088+0000","last_change":"2024-09-11T07:40:37.436683+0000","last_active":"2024-09-11T07:40:47.679088+0000","last_peered":"2024-09-11T07:40:47.679088+0000","last_clean":"2024-09-11T07:40:47.679088+0000","last_became_active":"2024-09-11T07:32:00.514100+0000","last_became_peered":"2024-09-11T07:32:00.514100+0000","last_unstale":"2024-09-11T07:40:47.679088+0000","last_undegraded":"2024-09-11T07:40:47.679088+0000","last_fullsized":"2024-09-11T07:40:47.679088+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:37.436628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:40:37.436628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:37.557030+0000","scrub_duration":0.001208518,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.19","version":"315'9863","reported_seq":22474,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:49.101018+0000","last_change":"2024-09-11T07:40:09.708552+0000","last_active":"2024-09-11T07:40:49.101018+0000","last_peered":"2024-09-11T07:40:49.101018+0000","last_clean":"2024-09-11T07:40:49.101018+0000","last_became_active":"2024-09-11T07:40:09.688156+0000","last_became_peered":"2024-09-11T07:40:09.688156+0000","last_unstale":"2024-09-11T07:40:49.101018+0000","last_undegraded":"2024-09-11T07:40:49.101018+0000","last_fullsized":"2024-09-11T07:40:49.101018+0000","mapping_epoch":294,"log_start":"256'3793","ondisk_log_start":"256'3793","created":259,"last_epoch_clean":295,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6070,"log_dups_size":0,"ondisk_log_size":6070,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59834368,"num_objects":913,"num_object_clones":0,"num_object_copies":2739,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7155,"num_write_kb":258176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4857,"num_bytes_recovered":318308352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"234'25","reported_seq":389,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.701407+0000","last_change":"2024-09-11T07:39:57.777199+0000","last_active":"2024-09-11T07:40:48.701407+0000","last_peered":"2024-09-11T07:40:48.701407+0000","last_clean":"2024-09-11T07:40:48.701407+0000","last_became_active":"2024-09-11T07:39:57.771009+0000","last_became_peered":"2024-09-11T07:39:57.771009+0000","last_unstale":"2024-09-11T07:40:48.701407+0000","last_undegraded":"2024-09-11T07:40:48.701407+0000","last_fullsized":"2024-09-11T07:40:48.701407+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'25","last_scrub_stamp":"2024-09-11T07:36:55.966595+0000","last_deep_scrub":"86'15","last_deep_scrub_stamp":"2024-09-11T07:28:22.858137+0000","last_clean_scrub_stamp":"2024-09-11T07:36:55.966595+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.032608074000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":346,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.681701+0000","last_change":"2024-09-11T07:39:57.764438+0000","last_active":"2024-09-11T07:40:47.681701+0000","last_peered":"2024-09-11T07:40:47.681701+0000","last_clean":"2024-09-11T07:40:47.681701+0000","last_became_active":"2024-09-11T07:39:57.764063+0000","last_became_peered":"2024-09-11T07:39:57.764063+0000","last_unstale":"2024-09-11T07:40:47.681701+0000","last_undegraded":"2024-09-11T07:40:47.681701+0000","last_fullsized":"2024-09-11T07:40:47.681701+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:54.493614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:39:54.493614+0000","last_clean_scrub_stamp":"2024-09-11T07:39:54.493614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:22.925815+0000","scrub_duration":0.0049403759999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":501,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.682442+0000","last_change":"2024-09-11T07:40:46.321277+0000","last_active":"2024-09-11T07:40:47.682442+0000","last_peered":"2024-09-11T07:40:47.682442+0000","last_clean":"2024-09-11T07:40:47.682442+0000","last_became_active":"2024-09-11T07:26:35.562245+0000","last_became_peered":"2024-09-11T07:26:35.562245+0000","last_unstale":"2024-09-11T07:40:47.682442+0000","last_undegraded":"2024-09-11T07:40:47.682442+0000","last_fullsized":"2024-09-11T07:40:47.682442+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:40:46.321189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:40:46.321189+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:03.756105+0000","scrub_duration":0.024243521000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":348,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.694083+0000","last_change":"2024-09-11T07:39:57.759567+0000","last_active":"2024-09-11T07:40:48.694083+0000","last_peered":"2024-09-11T07:40:48.694083+0000","last_clean":"2024-09-11T07:40:48.694083+0000","last_became_active":"2024-09-11T07:39:57.758650+0000","last_became_peered":"2024-09-11T07:39:57.758650+0000","last_unstale":"2024-09-11T07:40:48.694083+0000","last_undegraded":"2024-09-11T07:40:48.694083+0000","last_fullsized":"2024-09-11T07:40:48.694083+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:42.084305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:42.084305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047035039999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"316'9629","reported_seq":24198,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-09-11T07:40:50.024891+0000","last_change":"2024-09-11T07:40:49.712435+0000","last_active":"2024-09-11T07:40:50.024891+0000","last_peered":"2024-09-11T07:40:50.024891+0000","last_clean":"2024-09-11T07:40:50.024891+0000","last_became_active":"2024-09-11T07:40:49.697184+0000","last_became_peered":"2024-09-11T07:40:49.697184+0000","last_unstale":"2024-09-11T07:40:50.024891+0000","last_undegraded":"2024-09-11T07:40:50.024891+0000","last_fullsized":"2024-09-11T07:40:50.024891+0000","mapping_epoch":315,"log_start":"257'4987","ondisk_log_start":"257'4987","created":259,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4642,"log_dups_size":0,"ondisk_log_size":4642,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61669376,"num_objects":941,"num_object_clones":0,"num_object_copies":2823,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6955,"num_write_kb":252672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5363,"num_bytes_recovered":351469568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"234'19","reported_seq":320,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.678948+0000","last_change":"2024-09-11T07:39:57.780187+0000","last_active":"2024-09-11T07:40:47.678948+0000","last_peered":"2024-09-11T07:40:47.678948+0000","last_clean":"2024-09-11T07:40:47.678948+0000","last_became_active":"2024-09-11T07:39:57.765149+0000","last_became_peered":"2024-09-11T07:39:57.765149+0000","last_unstale":"2024-09-11T07:40:47.678948+0000","last_undegraded":"2024-09-11T07:40:47.678948+0000","last_fullsized":"2024-09-11T07:40:47.678948+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:37:41.338036+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:37:41.338036+0000","last_clean_scrub_stamp":"2024-09-11T07:37:41.338036+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045362370000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":320,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.692209+0000","last_change":"2024-09-11T07:40:44.709140+0000","last_active":"2024-09-11T07:40:48.692209+0000","last_peered":"2024-09-11T07:40:48.692209+0000","last_clean":"2024-09-11T07:40:48.692209+0000","last_became_active":"2024-09-11T07:26:31.419409+0000","last_became_peered":"2024-09-11T07:26:31.419409+0000","last_unstale":"2024-09-11T07:40:48.692209+0000","last_undegraded":"2024-09-11T07:40:48.692209+0000","last_fullsized":"2024-09-11T07:40:48.692209+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:27.958386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:27.958386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0093493710000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"290'11","reported_seq":243,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.678024+0000","last_change":"2024-09-11T07:39:57.778860+0000","last_active":"2024-09-11T07:40:47.678024+0000","last_peered":"2024-09-11T07:40:47.678024+0000","last_clean":"2024-09-11T07:40:47.678024+0000","last_became_active":"2024-09-11T07:39:57.772790+0000","last_became_peered":"2024-09-11T07:39:57.772790+0000","last_unstale":"2024-09-11T07:40:47.678024+0000","last_undegraded":"2024-09-11T07:40:47.678024+0000","last_fullsized":"2024-09-11T07:40:47.678024+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:38:25.065071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:38:25.065071+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020633370000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":313,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.686997+0000","last_change":"2024-09-11T07:39:50.572103+0000","last_active":"2024-09-11T07:40:47.686997+0000","last_peered":"2024-09-11T07:40:47.686997+0000","last_clean":"2024-09-11T07:40:47.686997+0000","last_became_active":"2024-09-11T07:26:37.565723+0000","last_became_peered":"2024-09-11T07:26:37.565723+0000","last_unstale":"2024-09-11T07:40:47.686997+0000","last_undegraded":"2024-09-11T07:40:47.686997+0000","last_fullsized":"2024-09-11T07:40:47.686997+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:26.659337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.659337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012563839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"236'16","reported_seq":318,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.684371+0000","last_change":"2024-09-11T07:40:29.564753+0000","last_active":"2024-09-11T07:40:47.684371+0000","last_peered":"2024-09-11T07:40:47.684371+0000","last_clean":"2024-09-11T07:40:47.684371+0000","last_became_active":"2024-09-11T07:39:57.689424+0000","last_became_peered":"2024-09-11T07:39:57.689424+0000","last_unstale":"2024-09-11T07:40:47.684371+0000","last_undegraded":"2024-09-11T07:40:47.684371+0000","last_fullsized":"2024-09-11T07:40:47.684371+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'16","last_scrub_stamp":"2024-09-11T07:36:49.809484+0000","last_deep_scrub":"236'16","last_deep_scrub_stamp":"2024-09-11T07:36:49.809484+0000","last_clean_scrub_stamp":"2024-09-11T07:36:49.809484+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011537936,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":123,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.701475+0000","last_change":"2024-09-11T07:39:57.779694+0000","last_active":"2024-09-11T07:40:48.701475+0000","last_peered":"2024-09-11T07:40:48.701475+0000","last_clean":"2024-09-11T07:40:48.701475+0000","last_became_active":"2024-09-11T07:39:57.776841+0000","last_became_peered":"2024-09-11T07:39:57.776841+0000","last_unstale":"2024-09-11T07:40:48.701475+0000","last_undegraded":"2024-09-11T07:40:48.701475+0000","last_fullsized":"2024-09-11T07:40:48.701475+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:22.914781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:09.772000+0000","last_clean_scrub_stamp":"2024-09-11T07:37:22.914781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001446864,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":309,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.697531+0000","last_change":"2024-09-11T07:40:31.747754+0000","last_active":"2024-09-11T07:40:48.697531+0000","last_peered":"2024-09-11T07:40:48.697531+0000","last_clean":"2024-09-11T07:40:48.697531+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:40:48.697531+0000","last_undegraded":"2024-09-11T07:40:48.697531+0000","last_fullsized":"2024-09-11T07:40:48.697531+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:51.886407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:35:51.886407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002900241,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":331,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.692312+0000","last_change":"2024-09-11T07:39:57.765790+0000","last_active":"2024-09-11T07:40:48.692312+0000","last_peered":"2024-09-11T07:40:48.692312+0000","last_clean":"2024-09-11T07:40:48.692312+0000","last_became_active":"2024-09-11T07:39:57.761754+0000","last_became_peered":"2024-09-11T07:39:57.761754+0000","last_unstale":"2024-09-11T07:40:48.692312+0000","last_undegraded":"2024-09-11T07:40:48.692312+0000","last_fullsized":"2024-09-11T07:40:48.692312+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:32.186561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:32.186561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014950059999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"234'24","reported_seq":353,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.686737+0000","last_change":"2024-09-11T07:39:42.612220+0000","last_active":"2024-09-11T07:40:47.686737+0000","last_peered":"2024-09-11T07:40:47.686737+0000","last_clean":"2024-09-11T07:40:47.686737+0000","last_became_active":"2024-09-11T07:26:33.540208+0000","last_became_peered":"2024-09-11T07:26:33.540208+0000","last_unstale":"2024-09-11T07:40:47.686737+0000","last_undegraded":"2024-09-11T07:40:47.686737+0000","last_fullsized":"2024-09-11T07:40:47.686737+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'16","last_scrub_stamp":"2024-09-11T07:36:31.789841+0000","last_deep_scrub":"136'16","last_deep_scrub_stamp":"2024-09-11T07:31:06.039023+0000","last_clean_scrub_stamp":"2024-09-11T07:36:31.789841+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048561150000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":328,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.687019+0000","last_change":"2024-09-11T07:39:51.779556+0000","last_active":"2024-09-11T07:40:47.687019+0000","last_peered":"2024-09-11T07:40:47.687019+0000","last_clean":"2024-09-11T07:40:47.687019+0000","last_became_active":"2024-09-11T07:26:31.422671+0000","last_became_peered":"2024-09-11T07:26:31.422671+0000","last_unstale":"2024-09-11T07:40:47.687019+0000","last_undegraded":"2024-09-11T07:40:47.687019+0000","last_fullsized":"2024-09-11T07:40:47.687019+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:51.779468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:39:51.779468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:40:52.296173+0000","scrub_duration":0.0089535459999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":324,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.695040+0000","last_change":"2024-09-11T07:39:57.761360+0000","last_active":"2024-09-11T07:40:47.695040+0000","last_peered":"2024-09-11T07:40:47.695040+0000","last_clean":"2024-09-11T07:40:47.695040+0000","last_became_active":"2024-09-11T07:39:57.759390+0000","last_became_peered":"2024-09-11T07:39:57.759390+0000","last_unstale":"2024-09-11T07:40:47.695040+0000","last_undegraded":"2024-09-11T07:40:47.695040+0000","last_fullsized":"2024-09-11T07:40:47.695040+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:55.895853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:39:55.895853+0000","last_clean_scrub_stamp":"2024-09-11T07:39:55.895853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:17.232667+0000","scrub_duration":0.011708260999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":50,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.691613+0000","last_change":"2024-09-11T07:40:11.701200+0000","last_active":"2024-09-11T07:40:48.691613+0000","last_peered":"2024-09-11T07:40:48.691613+0000","last_clean":"2024-09-11T07:40:48.691613+0000","last_became_active":"2024-09-11T07:40:11.700676+0000","last_became_peered":"2024-09-11T07:40:11.700676+0000","last_unstale":"2024-09-11T07:40:48.691613+0000","last_undegraded":"2024-09-11T07:40:48.691613+0000","last_fullsized":"2024-09-11T07:40:48.691613+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:43.814370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:43.814370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"234'10","reported_seq":329,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.693430+0000","last_change":"2024-09-11T07:36:46.900027+0000","last_active":"2024-09-11T07:40:48.693430+0000","last_peered":"2024-09-11T07:40:48.693430+0000","last_clean":"2024-09-11T07:40:48.693430+0000","last_became_active":"2024-09-11T07:26:33.545830+0000","last_became_peered":"2024-09-11T07:26:33.545830+0000","last_unstale":"2024-09-11T07:40:48.693430+0000","last_undegraded":"2024-09-11T07:40:48.693430+0000","last_fullsized":"2024-09-11T07:40:48.693430+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'10","last_scrub_stamp":"2024-09-11T07:36:46.899947+0000","last_deep_scrub":"82'6","last_deep_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_clean_scrub_stamp":"2024-09-11T07:36:46.899947+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003412277,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":326,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.682657+0000","last_change":"2024-09-11T07:39:56.466407+0000","last_active":"2024-09-11T07:40:47.682657+0000","last_peered":"2024-09-11T07:40:47.682657+0000","last_clean":"2024-09-11T07:40:47.682657+0000","last_became_active":"2024-09-11T07:26:31.420518+0000","last_became_peered":"2024-09-11T07:26:31.420518+0000","last_unstale":"2024-09-11T07:40:47.682657+0000","last_undegraded":"2024-09-11T07:40:47.682657+0000","last_fullsized":"2024-09-11T07:40:47.682657+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:56.466067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_clean_scrub_stamp":"2024-09-11T07:39:56.466067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:40:59.945617+0000","scrub_duration":0.0030505950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":257,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.686342+0000","last_change":"2024-09-11T07:40:11.700029+0000","last_active":"2024-09-11T07:40:47.686342+0000","last_peered":"2024-09-11T07:40:47.686342+0000","last_clean":"2024-09-11T07:40:47.686342+0000","last_became_active":"2024-09-11T07:40:11.699702+0000","last_became_peered":"2024-09-11T07:40:11.699702+0000","last_unstale":"2024-09-11T07:40:47.686342+0000","last_undegraded":"2024-09-11T07:40:47.686342+0000","last_fullsized":"2024-09-11T07:40:47.686342+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:34.658586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:47.675050+0000","last_clean_scrub_stamp":"2024-09-11T07:39:34.658586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047285640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":320,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.678326+0000","last_change":"2024-09-11T07:39:43.398492+0000","last_active":"2024-09-11T07:40:47.678326+0000","last_peered":"2024-09-11T07:40:47.678326+0000","last_clean":"2024-09-11T07:40:47.678326+0000","last_became_active":"2024-09-11T07:32:00.511407+0000","last_became_peered":"2024-09-11T07:32:00.511407+0000","last_unstale":"2024-09-11T07:40:47.678326+0000","last_undegraded":"2024-09-11T07:40:47.678326+0000","last_fullsized":"2024-09-11T07:40:47.678326+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:43.398419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:39:43.398419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:06.930545+0000","scrub_duration":0.0064759270000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"234'19","reported_seq":375,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.701657+0000","last_change":"2024-09-11T07:39:57.791332+0000","last_active":"2024-09-11T07:40:48.701657+0000","last_peered":"2024-09-11T07:40:48.701657+0000","last_clean":"2024-09-11T07:40:48.701657+0000","last_became_active":"2024-09-11T07:39:57.767928+0000","last_became_peered":"2024-09-11T07:39:57.767928+0000","last_unstale":"2024-09-11T07:40:48.701657+0000","last_undegraded":"2024-09-11T07:40:48.701657+0000","last_fullsized":"2024-09-11T07:40:48.701657+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:38:28.017124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:38:28.017124+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018425620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":112,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.682873+0000","last_change":"2024-09-11T07:39:29.286259+0000","last_active":"2024-09-11T07:40:47.682873+0000","last_peered":"2024-09-11T07:40:47.682873+0000","last_clean":"2024-09-11T07:40:47.682873+0000","last_became_active":"2024-09-11T07:36:05.228360+0000","last_became_peered":"2024-09-11T07:36:05.228360+0000","last_unstale":"2024-09-11T07:40:47.682873+0000","last_undegraded":"2024-09-11T07:40:47.682873+0000","last_fullsized":"2024-09-11T07:40:47.682873+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:29.286202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:48.366931+0000","last_clean_scrub_stamp":"2024-09-11T07:39:29.286202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001125411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":293,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.677163+0000","last_change":"2024-09-11T07:39:56.318341+0000","last_active":"2024-09-11T07:40:47.677163+0000","last_peered":"2024-09-11T07:40:47.677163+0000","last_clean":"2024-09-11T07:40:47.677163+0000","last_became_active":"2024-09-11T07:32:00.520534+0000","last_became_peered":"2024-09-11T07:32:00.520534+0000","last_unstale":"2024-09-11T07:40:47.677163+0000","last_undegraded":"2024-09-11T07:40:47.677163+0000","last_fullsized":"2024-09-11T07:40:47.677163+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:56.318259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:39.021876+0000","last_clean_scrub_stamp":"2024-09-11T07:39:56.318259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:25.586713+0000","scrub_duration":0.0056755970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":301,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.697388+0000","last_change":"2024-09-11T07:36:49.899993+0000","last_active":"2024-09-11T07:40:48.697388+0000","last_peered":"2024-09-11T07:40:48.697388+0000","last_clean":"2024-09-11T07:40:48.697388+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:40:48.697388+0000","last_undegraded":"2024-09-11T07:40:48.697388+0000","last_fullsized":"2024-09-11T07:40:48.697388+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:49.899620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:49.899620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014021288999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"315'9574","reported_seq":21970,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:49.081853+0000","last_change":"2024-09-11T07:40:27.421619+0000","last_active":"2024-09-11T07:40:49.081853+0000","last_peered":"2024-09-11T07:40:49.081853+0000","last_clean":"2024-09-11T07:40:49.081853+0000","last_became_active":"2024-09-11T07:40:27.404959+0000","last_became_peered":"2024-09-11T07:40:27.404959+0000","last_unstale":"2024-09-11T07:40:49.081853+0000","last_undegraded":"2024-09-11T07:40:49.081853+0000","last_fullsized":"2024-09-11T07:40:49.081853+0000","mapping_epoch":302,"log_start":"255'3335","ondisk_log_start":"255'3335","created":259,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6239,"log_dups_size":0,"ondisk_log_size":6239,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58163200,"num_objects":888,"num_object_clones":0,"num_object_copies":2664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6922,"num_write_kb":249888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4649,"num_bytes_recovered":304676864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":290,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.677849+0000","last_change":"2024-09-11T07:40:44.505650+0000","last_active":"2024-09-11T07:40:47.677849+0000","last_peered":"2024-09-11T07:40:47.677849+0000","last_clean":"2024-09-11T07:40:47.677849+0000","last_became_active":"2024-09-11T07:32:00.511258+0000","last_became_peered":"2024-09-11T07:32:00.511258+0000","last_unstale":"2024-09-11T07:40:47.677849+0000","last_undegraded":"2024-09-11T07:40:47.677849+0000","last_fullsized":"2024-09-11T07:40:47.677849+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:36:44.447809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:44.447809+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005540555,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":352,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.682181+0000","last_change":"2024-09-11T07:40:32.337500+0000","last_active":"2024-09-11T07:40:47.682181+0000","last_peered":"2024-09-11T07:40:47.682181+0000","last_clean":"2024-09-11T07:40:47.682181+0000","last_became_active":"2024-09-11T07:39:57.696805+0000","last_became_peered":"2024-09-11T07:39:57.696805+0000","last_unstale":"2024-09-11T07:40:47.682181+0000","last_undegraded":"2024-09-11T07:40:47.682181+0000","last_fullsized":"2024-09-11T07:40:47.682181+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:36:31.353426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:31.353426+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0063773889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"234'20","reported_seq":157,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.687435+0000","last_change":"2024-09-11T07:40:47.544520+0000","last_active":"2024-09-11T07:40:47.687435+0000","last_peered":"2024-09-11T07:40:47.687435+0000","last_clean":"2024-09-11T07:40:47.687435+0000","last_became_active":"2024-09-11T07:35:00.561615+0000","last_became_peered":"2024-09-11T07:35:00.561615+0000","last_unstale":"2024-09-11T07:40:47.687435+0000","last_undegraded":"2024-09-11T07:40:47.687435+0000","last_fullsized":"2024-09-11T07:40:47.687435+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'20","last_scrub_stamp":"2024-09-11T07:37:29.661671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:29.661671+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019609900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1f","version":"290'12","reported_seq":207,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.687231+0000","last_change":"2024-09-11T07:39:57.749653+0000","last_active":"2024-09-11T07:40:47.687231+0000","last_peered":"2024-09-11T07:40:47.687231+0000","last_clean":"2024-09-11T07:40:47.687231+0000","last_became_active":"2024-09-11T07:39:57.687809+0000","last_became_peered":"2024-09-11T07:39:57.687809+0000","last_unstale":"2024-09-11T07:40:47.687231+0000","last_undegraded":"2024-09-11T07:40:47.687231+0000","last_fullsized":"2024-09-11T07:40:47.687231+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"225'11","last_scrub_stamp":"2024-09-11T07:38:22.504199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:38:22.504199+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020188300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"315'9572","reported_seq":17875,"reported_epoch":315,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-11T07:40:49.089700+0000","last_change":"2024-09-11T07:40:28.985055+0000","last_active":"2024-09-11T07:40:49.089700+0000","last_peered":"2024-09-11T07:40:49.089700+0000","last_clean":"2024-09-11T07:39:56.654039+0000","last_became_active":"2024-09-11T07:39:58.924616+0000","last_became_peered":"2024-09-11T07:39:58.924616+0000","last_unstale":"2024-09-11T07:40:49.089700+0000","last_undegraded":"2024-09-11T07:40:49.089700+0000","last_fullsized":"2024-09-11T07:40:49.089700+0000","mapping_epoch":289,"log_start":"256'4552","ondisk_log_start":"256'4552","created":259,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":5020,"log_dups_size":0,"ondisk_log_size":5020,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58982400,"num_objects":900,"num_object_clones":0,"num_object_copies":2700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":900,"num_objects_unfound":0,"num_objects_dirty":900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6910,"num_write_kb":249920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2724,"num_bytes_recovered":178520064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,5],"avail_no_missing":["1","0","3","5"],"object_location_counts":[{"shards":"0,1,5","objects":900}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":319,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.683274+0000","last_change":"2024-09-11T07:38:26.939048+0000","last_active":"2024-09-11T07:40:47.683274+0000","last_peered":"2024-09-11T07:40:47.683274+0000","last_clean":"2024-09-11T07:40:47.683274+0000","last_became_active":"2024-09-11T07:26:37.562501+0000","last_became_peered":"2024-09-11T07:26:37.562501+0000","last_unstale":"2024-09-11T07:40:47.683274+0000","last_undegraded":"2024-09-11T07:40:47.683274+0000","last_fullsized":"2024-09-11T07:40:47.683274+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:26.938987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:53.875300+0000","last_clean_scrub_stamp":"2024-09-11T07:38:26.938987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016308410000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":330,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.702301+0000","last_change":"2024-09-11T07:40:34.645930+0000","last_active":"2024-09-11T07:40:48.702301+0000","last_peered":"2024-09-11T07:40:48.702301+0000","last_clean":"2024-09-11T07:40:48.702301+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:40:48.702301+0000","last_undegraded":"2024-09-11T07:40:48.702301+0000","last_fullsized":"2024-09-11T07:40:48.702301+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:34.645645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:40:34.645645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:53.530666+0000","scrub_duration":0.0077905500000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"234'19","reported_seq":155,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.682752+0000","last_change":"2024-09-11T07:38:25.113379+0000","last_active":"2024-09-11T07:40:47.682752+0000","last_peered":"2024-09-11T07:40:47.682752+0000","last_clean":"2024-09-11T07:40:47.682752+0000","last_became_active":"2024-09-11T07:36:05.228746+0000","last_became_peered":"2024-09-11T07:36:05.228746+0000","last_unstale":"2024-09-11T07:40:47.682752+0000","last_undegraded":"2024-09-11T07:40:47.682752+0000","last_fullsized":"2024-09-11T07:40:47.682752+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:38:25.113300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:38:25.113300+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0063902960000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":330,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.694195+0000","last_change":"2024-09-11T07:37:21.150801+0000","last_active":"2024-09-11T07:40:48.694195+0000","last_peered":"2024-09-11T07:40:48.694195+0000","last_clean":"2024-09-11T07:40:48.694195+0000","last_became_active":"2024-09-11T07:32:00.507050+0000","last_became_peered":"2024-09-11T07:32:00.507050+0000","last_unstale":"2024-09-11T07:40:48.694195+0000","last_undegraded":"2024-09-11T07:40:48.694195+0000","last_fullsized":"2024-09-11T07:40:48.694195+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:21.150732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:24.575210+0000","last_clean_scrub_stamp":"2024-09-11T07:37:21.150732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001360691,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.11","version":"315'9794","reported_seq":9888,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:49.075015+0000","last_change":"2024-09-11T07:38:29.349114+0000","last_active":"2024-09-11T07:40:49.075015+0000","last_peered":"2024-09-11T07:40:49.075015+0000","last_clean":"2024-09-11T07:40:49.075015+0000","last_became_active":"2024-09-11T07:38:29.346889+0000","last_became_peered":"2024-09-11T07:38:29.346889+0000","last_unstale":"2024-09-11T07:40:49.075015+0000","last_undegraded":"2024-09-11T07:40:49.075015+0000","last_fullsized":"2024-09-11T07:40:49.075015+0000","mapping_epoch":261,"log_start":"255'3397","ondisk_log_start":"255'3397","created":259,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6397,"log_dups_size":0,"ondisk_log_size":6397,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59965440,"num_objects":915,"num_object_clones":0,"num_object_copies":2745,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7117,"num_write_kb":257024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":323,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.687558+0000","last_change":"2024-09-11T07:39:28.550044+0000","last_active":"2024-09-11T07:40:47.687558+0000","last_peered":"2024-09-11T07:40:47.687558+0000","last_clean":"2024-09-11T07:40:47.687558+0000","last_became_active":"2024-09-11T07:26:37.862373+0000","last_became_peered":"2024-09-11T07:26:37.862373+0000","last_unstale":"2024-09-11T07:40:47.687558+0000","last_undegraded":"2024-09-11T07:40:47.687558+0000","last_fullsized":"2024-09-11T07:40:47.687558+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:28.549952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:39:28.549952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:40:52.918785+0000","scrub_duration":0.001250695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":261,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.686598+0000","last_change":"2024-09-11T07:39:54.632584+0000","last_active":"2024-09-11T07:40:47.686598+0000","last_peered":"2024-09-11T07:40:47.686598+0000","last_clean":"2024-09-11T07:40:47.686598+0000","last_became_active":"2024-09-11T07:36:05.224085+0000","last_became_peered":"2024-09-11T07:36:05.224085+0000","last_unstale":"2024-09-11T07:40:47.686598+0000","last_undegraded":"2024-09-11T07:40:47.686598+0000","last_fullsized":"2024-09-11T07:40:47.686598+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:54.632499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:39:54.632499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:06.525855+0000","scrub_duration":0.013616645,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"234'27","reported_seq":383,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.683467+0000","last_change":"2024-09-11T07:40:39.395515+0000","last_active":"2024-09-11T07:40:47.683467+0000","last_peered":"2024-09-11T07:40:47.683467+0000","last_clean":"2024-09-11T07:40:47.683467+0000","last_became_active":"2024-09-11T07:26:33.542544+0000","last_became_peered":"2024-09-11T07:26:33.542544+0000","last_unstale":"2024-09-11T07:40:47.683467+0000","last_undegraded":"2024-09-11T07:40:47.683467+0000","last_fullsized":"2024-09-11T07:40:47.683467+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'27","last_scrub_stamp":"2024-09-11T07:40:39.395458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:40:39.395458+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:04.483004+0000","scrub_duration":0.0018350700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":324,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.683444+0000","last_change":"2024-09-11T07:40:38.884375+0000","last_active":"2024-09-11T07:40:47.683444+0000","last_peered":"2024-09-11T07:40:47.683444+0000","last_clean":"2024-09-11T07:40:47.683444+0000","last_became_active":"2024-09-11T07:26:35.555862+0000","last_became_peered":"2024-09-11T07:26:35.555862+0000","last_unstale":"2024-09-11T07:40:47.683444+0000","last_undegraded":"2024-09-11T07:40:47.683444+0000","last_fullsized":"2024-09-11T07:40:47.683444+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:38.884278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:40:38.884278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:53.219499+0000","scrub_duration":0.0017591729999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"315'9802","reported_seq":9917,"reported_epoch":315,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-11T07:40:49.158910+0000","last_change":"2024-09-11T07:40:38.661172+0000","last_active":"2024-09-11T07:40:49.158910+0000","last_peered":"2024-09-11T07:40:49.158910+0000","last_clean":"2024-09-11T07:38:28.331892+0000","last_became_active":"2024-09-11T07:38:30.445788+0000","last_became_peered":"2024-09-11T07:38:30.445788+0000","last_unstale":"2024-09-11T07:40:49.158910+0000","last_undegraded":"2024-09-11T07:40:49.158910+0000","last_fullsized":"2024-09-11T07:40:49.158910+0000","mapping_epoch":262,"log_start":"256'4961","ondisk_log_start":"256'4961","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4841,"log_dups_size":0,"ondisk_log_size":4841,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54853632,"num_objects":837,"num_object_clones":0,"num_object_copies":2511,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2511,"num_objects_unfound":0,"num_objects_dirty":837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7083,"num_write_kb":253440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[1,2,6],"avail_no_missing":["1","2","6"],"object_location_counts":[{"shards":"1,2,6","objects":837}],"blocked_by":[],"up_primary":4,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":321,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.694467+0000","last_change":"2024-09-11T07:39:57.769864+0000","last_active":"2024-09-11T07:40:48.694467+0000","last_peered":"2024-09-11T07:40:48.694467+0000","last_clean":"2024-09-11T07:40:48.694467+0000","last_became_active":"2024-09-11T07:39:57.768994+0000","last_became_peered":"2024-09-11T07:39:57.768994+0000","last_unstale":"2024-09-11T07:40:48.694467+0000","last_undegraded":"2024-09-11T07:40:48.694467+0000","last_fullsized":"2024-09-11T07:40:48.694467+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:32.004125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:32.004125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019485520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":344,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.697700+0000","last_change":"2024-09-11T07:39:57.768623+0000","last_active":"2024-09-11T07:40:48.697700+0000","last_peered":"2024-09-11T07:40:48.697700+0000","last_clean":"2024-09-11T07:40:48.697700+0000","last_became_active":"2024-09-11T07:39:57.766915+0000","last_became_peered":"2024-09-11T07:39:57.766915+0000","last_unstale":"2024-09-11T07:40:48.697700+0000","last_undegraded":"2024-09-11T07:40:48.697700+0000","last_fullsized":"2024-09-11T07:40:48.697700+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:42.078765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:42.078765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0062011560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"234'23","reported_seq":352,"reported_epoch":315,"state":"active+clean","last_fresh":"2024-09-11T07:40:48.691819+0000","last_change":"2024-09-11T07:37:43.078969+0000","last_active":"2024-09-11T07:40:48.691819+0000","last_peered":"2024-09-11T07:40:48.691819+0000","last_clean":"2024-09-11T07:40:48.691819+0000","last_became_active":"2024-09-11T07:26:33.542711+0000","last_became_peered":"2024-09-11T07:26:33.542711+0000","last_unstale":"2024-09-11T07:40:48.691819+0000","last_undegraded":"2024-09-11T07:40:48.691819+0000","last_fullsized":"2024-09-11T07:40:48.691819+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'23","last_scrub_stamp":"2024-09-11T07:37:43.078872+0000","last_deep_scrub":"127'17","last_deep_scrub_stamp":"2024-09-11T07:35:43.871918+0000","last_clean_scrub_stamp":"2024-09-11T07:37:43.078872+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051065670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":318,"reported_epoch":314,"state":"active+clean","last_fresh":"2024-09-11T07:40:47.687345+0000","last_change":"2024-09-11T07:39:31.586959+0000","last_active":"2024-09-11T07:40:47.687345+0000","last_peered":"2024-09-11T07:40:47.687345+0000","last_clean":"2024-09-11T07:40:47.687345+0000","last_became_active":"2024-09-11T07:26:35.562037+0000","last_became_peered":"2024-09-11T07:26:35.562037+0000","last_unstale":"2024-09-11T07:40:47.687345+0000","last_undegraded":"2024-09-11T07:40:47.687345+0000","last_fullsized":"2024-09-11T07:40:47.687345+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:37.811009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:36:37.811009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039204729999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":26,"stat_sum":{"num_bytes":1914044440,"num_objects":29207,"num_object_clones":0,"num_object_copies":87621,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":11693,"num_objects_unfound":0,"num_objects_dirty":29207,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":223971,"num_write_kb":8101633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43816,"num_bytes_recovered":2871394328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":7626117120,"data_stored":7626096760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135181,"ondisk_log_size":135181,"up":78,"acting":78,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1118,"num_read_kb":875,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":2916,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":765,"ondisk_log_size":765,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":849,"num_read_kb":844,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":147,"seq":631360192619,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1545860,"kb_used_data":1217212,"kb_used_omap":4,"kb_used_meta":328635,"kb_avail":92199292,"statfs":{"total":95995035648,"available":94412075008,"internally_reserved":0,"allocated":1246425088,"data_stored":1246043109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":336522758},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:40:17 2024","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.375,"15min":2.6030000000000002},"min":{"1min":0.23100000000000001,"5min":0.191,"15min":0.191},"max":{"1min":4.609,"5min":14.086,"15min":49.573},"last":0.49099999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.488,"15min":2.508},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":4.069,"5min":16.841999999999999,"15min":49.881},"last":0.47599999999999998}]},{"osd":1,"last update":"Wed Sep 11 07:40:17 2024","interfaces":[{"interface":"back","average":{"1min":1.677,"5min":1.2050000000000001,"15min":0.84699999999999998},"min":{"1min":0.28299999999999997,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":5.0039999999999996,"5min":10.063000000000001,"15min":10.063000000000001},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.361,"5min":1.2789999999999999,"15min":0.96999999999999997},"min":{"1min":0.25600000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.5149999999999997,"5min":9.6059999999999999,"15min":9.6059999999999999},"last":0.45700000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:40:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":3.6030000000000002,"15min":1.8400000000000001},"min":{"1min":0.25700000000000001,"5min":0.249,"15min":0.22800000000000001},"max":{"1min":4.282,"5min":227.79300000000001,"15min":227.79300000000001},"last":3.46},{"interface":"front","average":{"1min":1.1779999999999999,"5min":3.7250000000000001,"15min":1.863},"min":{"1min":0.313,"5min":0.246,"15min":0.246},"max":{"1min":4.5529999999999999,"5min":227.22,"15min":227.22},"last":0.79800000000000004}]},{"osd":3,"last update":"Wed Sep 11 07:40:17 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.206,"15min":2.48},"min":{"1min":0.32800000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":4.6349999999999998,"5min":5.181,"15min":49.798999999999999},"last":2.383},{"interface":"front","average":{"1min":2.0680000000000001,"5min":1.339,"15min":2.4860000000000002},"min":{"1min":0.32100000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":7.3369999999999997,"5min":7.3369999999999997,"15min":49.524000000000001},"last":2.0289999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:40:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":0.85799999999999998,"15min":0.58399999999999996},"min":{"1min":0.27300000000000002,"5min":0.25700000000000001,"15min":0.253},"max":{"1min":3.9399999999999999,"5min":6.0220000000000002,"15min":6.0220000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":1.5629999999999999,"5min":0.86699999999999999,"15min":0.69699999999999995},"min":{"1min":0.33700000000000002,"5min":0.247,"15min":0.247},"max":{"1min":6.9000000000000004,"5min":6.9000000000000004,"15min":6.9000000000000004},"last":0.34399999999999997}]},{"osd":5,"last update":"Wed Sep 11 07:40:17 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.014,"15min":0.66700000000000004},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":5.2320000000000002,"5min":7.2160000000000002,"15min":7.2160000000000002},"last":0.503},{"interface":"front","average":{"1min":1.149,"5min":0.91100000000000003,"15min":0.75800000000000001},"min":{"1min":0.36299999999999999,"5min":0.27900000000000003,"15min":0.27600000000000002},"max":{"1min":4.4610000000000003,"5min":4.9969999999999999,"15min":4.9969999999999999},"last":0.41399999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:40:17 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":0.81999999999999995,"15min":0.626},"min":{"1min":0.312,"5min":0.31,"15min":0.29099999999999998},"max":{"1min":5.1909999999999998,"5min":5.1909999999999998,"15min":5.1909999999999998},"last":0.432},{"interface":"front","average":{"1min":1.091,"5min":0.92000000000000004,"15min":0.68100000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29099999999999998},"max":{"1min":4.2030000000000003,"5min":4.2030000000000003,"15min":4.2030000000000003},"last":0.64200000000000002}]}]},{"osd":6,"up_from":44,"seq":188978561204,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1026112,"kb_used_data":716272,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":92719040,"statfs":{"total":95995035648,"available":94944296960,"internally_reserved":0,"allocated":733462528,"data_stored":733036531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:40:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":1.081,"15min":1.7989999999999999},"min":{"1min":0.223,"5min":0.218,"15min":0.192},"max":{"1min":1.405,"5min":20.675999999999998,"15min":142.29599999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.2709999999999999,"15min":1.964},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":3.754,"5min":20.663,"15min":142.68799999999999},"last":0.91100000000000003}]},{"osd":1,"last update":"Wed Sep 11 07:40:18 2024","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.9990000000000001,"15min":1.5589999999999999},"min":{"1min":0.29399999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":4.7350000000000003,"5min":39.390000000000001,"15min":127.92100000000001},"last":1.0700000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":1.853,"15min":1.5389999999999999},"min":{"1min":0.29099999999999998,"5min":0.26900000000000002,"15min":0.23000000000000001},"max":{"1min":4.6859999999999999,"5min":40.186999999999998,"15min":127.746},"last":0.78700000000000003}]},{"osd":2,"last update":"Wed Sep 11 07:40:18 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.83599999999999997,"15min":0.86099999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":2.1699999999999999,"5min":3.6539999999999999,"15min":26.491},"last":1.3129999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":1.0169999999999999,"15min":1.048},"min":{"1min":0.313,"5min":0.26800000000000002,"15min":0.26400000000000001},"max":{"1min":2.6960000000000002,"5min":6.0819999999999999,"15min":26.561},"last":1.0289999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:40:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":2.1960000000000002,"15min":1.4550000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":4.0419999999999998,"5min":49.109000000000002,"15min":49.109000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":1.8480000000000001,"5min":2.3210000000000002,"15min":1.571},"min":{"1min":0.29299999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":7.0659999999999998,"5min":48.963999999999999,"15min":48.963999999999999},"last":0.38400000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:40:18 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.72699999999999998,"15min":0.57699999999999996},"min":{"1min":0.23899999999999999,"5min":0.223,"15min":0.223},"max":{"1min":2.444,"5min":3.3599999999999999,"15min":4.4720000000000004},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.69999999999999996,"15min":0.52800000000000002},"min":{"1min":0.32100000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":2.597,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":0.44700000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:40:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.395,"15min":0.874},"min":{"1min":0.33500000000000002,"5min":0.30199999999999999,"15min":0.255},"max":{"1min":3.7189999999999999,"5min":57.436,"15min":57.436},"last":0.52500000000000002},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.4159999999999999,"15min":0.88700000000000001},"min":{"1min":0.28899999999999998,"5min":0.28599999999999998,"15min":0.251},"max":{"1min":8.5489999999999995,"5min":56.822000000000003,"15min":56.822000000000003},"last":1.581}]},{"osd":7,"last update":"Wed Sep 11 07:40:18 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":0.84199999999999997,"15min":0.71799999999999997},"min":{"1min":0.373,"5min":0.309,"15min":0.28999999999999998},"max":{"1min":5.4290000000000003,"5min":5.4290000000000003,"15min":5.4290000000000003},"last":1.3320000000000001},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.86199999999999999,"15min":0.754},"min":{"1min":0.308,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.2869999999999999,"5min":6.2869999999999999,"15min":6.2869999999999999},"last":0.65600000000000003}]}]},{"osd":5,"up_from":37,"seq":158913790138,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1819548,"kb_used_data":1221332,"kb_used_omap":1,"kb_used_meta":598206,"kb_avail":91925604,"statfs":{"total":95995035648,"available":94131818496,"internally_reserved":0,"allocated":1250643968,"data_stored":1250245548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":612563288},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:39:54 2024","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.7490000000000001,"15min":2.2480000000000002},"min":{"1min":0.216,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":8.7300000000000004,"5min":31.369,"15min":185.68199999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.6890000000000001,"15min":2.3359999999999999},"min":{"1min":0.27200000000000002,"5min":0.223,"15min":0.218},"max":{"1min":3.0329999999999999,"5min":25.722999999999999,"15min":186.12200000000001},"last":0.60699999999999998}]},{"osd":1,"last update":"Wed Sep 11 07:39:54 2024","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.752,"15min":1.1080000000000001},"min":{"1min":0.25800000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":11.234999999999999,"5min":26.286999999999999,"15min":26.286999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":2.0790000000000002,"5min":1.5640000000000001,"15min":1.107},"min":{"1min":0.247,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":10.657999999999999,"5min":24.672999999999998,"15min":24.672999999999998},"last":0.94699999999999995}]},{"osd":2,"last update":"Wed Sep 11 07:39:54 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":1.0189999999999999,"15min":1.631},"min":{"1min":0.312,"5min":0.28399999999999997,"15min":0.22700000000000001},"max":{"1min":3.452,"5min":5.726,"15min":187.99199999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":1.048,"5min":1.125,"15min":1.637},"min":{"1min":0.30399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":4.04,"5min":13.016,"15min":186.375},"last":2.645}]},{"osd":3,"last update":"Wed Sep 11 07:39:54 2024","interfaces":[{"interface":"back","average":{"1min":1.877,"5min":1.7789999999999999,"15min":2.3559999999999999},"min":{"1min":0.28699999999999998,"5min":0.26800000000000002,"15min":0.23599999999999999},"max":{"1min":12.983000000000001,"5min":28.978000000000002,"15min":188.11500000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":1.804,"5min":1.994,"15min":2.4060000000000001},"min":{"1min":0.28100000000000003,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":13.765000000000001,"5min":26.936,"15min":186.44200000000001},"last":0.499}]},{"osd":4,"last update":"Wed Sep 11 07:39:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":0.80300000000000005,"15min":0.57299999999999995},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.19900000000000001},"max":{"1min":11.103,"5min":11.103,"15min":11.103},"last":0.441},{"interface":"front","average":{"1min":1.601,"5min":0.879,"15min":0.60299999999999998},"min":{"1min":0.29099999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":10.305,"5min":10.305,"15min":10.305},"last":0.41899999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:40:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":0.873,"15min":0.624},"min":{"1min":0.32900000000000001,"5min":0.29199999999999998,"15min":0.26000000000000001},"max":{"1min":3.5019999999999998,"5min":5.6859999999999999,"15min":5.6859999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":1.1439999999999999,"5min":0.84899999999999998,"15min":0.62},"min":{"1min":0.28799999999999998,"5min":0.26800000000000002,"15min":0.25800000000000001},"max":{"1min":5.415,"5min":10.613,"15min":10.613},"last":0.52500000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:40:15 2024","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.375,"15min":0.879},"min":{"1min":0.42299999999999999,"5min":0.30099999999999999,"15min":0.28000000000000003},"max":{"1min":5.9290000000000003,"5min":50.338999999999999,"15min":50.338999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.369,"15min":0.875},"min":{"1min":0.36199999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":3.5249999999999999,"5min":50.298999999999999,"15min":50.298999999999999},"last":0.58199999999999996}]}]},{"osd":4,"up_from":31,"seq":133143986366,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1019308,"kb_used_data":717664,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":92725844,"statfs":{"total":95995035648,"available":94951264256,"internally_reserved":0,"allocated":734887936,"data_stored":734470245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:40:40 2024","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.042,"15min":1.222},"min":{"1min":0.22500000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":4.423,"5min":9.2330000000000005,"15min":35.308},"last":0.441},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.89100000000000001,"15min":1.159},"min":{"1min":0.26900000000000002,"5min":0.222,"15min":0.20499999999999999},"max":{"1min":2.6379999999999999,"5min":4.5620000000000003,"15min":35.597999999999999},"last":0.61599999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:40:40 2024","interfaces":[{"interface":"back","average":{"1min":5.6479999999999997,"5min":2.2629999999999999,"15min":1.484},"min":{"1min":0.35699999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":73.447999999999993,"5min":73.447999999999993,"15min":73.447999999999993},"last":1.242},{"interface":"front","average":{"1min":5.492,"5min":2.3130000000000002,"15min":1.3460000000000001},"min":{"1min":0.26600000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":73.557000000000002,"5min":73.557000000000002,"15min":73.557000000000002},"last":0.83999999999999997}]},{"osd":2,"last update":"Wed Sep 11 07:40:40 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":0.90000000000000002,"15min":0.97099999999999997},"min":{"1min":0.24099999999999999,"5min":0.23699999999999999,"15min":0.223},"max":{"1min":4.1550000000000002,"5min":4.1550000000000002,"15min":19.927},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":0.93899999999999995,"15min":0.98099999999999998},"min":{"1min":0.34100000000000003,"5min":0.28899999999999998,"15min":0.24399999999999999},"max":{"1min":4.0730000000000004,"5min":4.101,"15min":20.02},"last":0.28799999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:40:40 2024","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3140000000000001,"15min":1.2390000000000001},"min":{"1min":0.30499999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.0339999999999998,"5min":13.853,"15min":24.029},"last":0.53500000000000003},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.236,"15min":1.157},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":2.9020000000000001,"5min":13.651,"15min":24.265000000000001},"last":1.278}]},{"osd":5,"last update":"Wed Sep 11 07:40:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":0.80900000000000005,"15min":0.55800000000000005},"min":{"1min":0.224,"5min":0.223,"15min":0.21099999999999999},"max":{"1min":4.1299999999999999,"5min":4.6079999999999997,"15min":4.6079999999999997},"last":0.318},{"interface":"front","average":{"1min":1.0660000000000001,"5min":0.82299999999999995,"15min":0.57899999999999996},"min":{"1min":0.28499999999999998,"5min":0.23300000000000001,"15min":0.22700000000000001},"max":{"1min":4.1840000000000002,"5min":5.4560000000000004,"15min":5.7320000000000002},"last":0.39800000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:40:30 2024","interfaces":[{"interface":"back","average":{"1min":1.9059999999999999,"5min":0.85499999999999998,"15min":0.63700000000000001},"min":{"1min":0.28399999999999997,"5min":0.255,"15min":0.20899999999999999},"max":{"1min":9.0579999999999998,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":0.34599999999999997},{"interface":"front","average":{"1min":1.133,"5min":0.77200000000000002,"15min":0.61399999999999999},"min":{"1min":0.33900000000000002,"5min":0.27500000000000002,"15min":0.25800000000000001},"max":{"1min":4.2140000000000004,"5min":4.2140000000000004,"15min":9.3770000000000007},"last":0.35499999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:40:16 2024","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":0.89400000000000002,"15min":0.67300000000000004},"min":{"1min":0.39200000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":8.0129999999999999,"5min":8.0129999999999999,"15min":8.0129999999999999},"last":0.30099999999999999},{"interface":"front","average":{"1min":1.1919999999999999,"5min":0.80500000000000005,"15min":0.65100000000000002},"min":{"1min":0.312,"5min":0.29799999999999999,"15min":0.29599999999999999},"max":{"1min":3.274,"5min":3.274,"15min":3.274},"last":0.47399999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182595,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2087096,"kb_used_data":1188204,"kb_used_omap":1,"kb_used_meta":898878,"kb_avail":91658056,"statfs":{"total":95995035648,"available":93857849344,"internally_reserved":0,"allocated":1216720896,"data_stored":1216297450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":920451416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:40:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.468,"15min":1.413},"min":{"1min":0.18099999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":9.0790000000000006,"5min":18.704000000000001,"15min":38.432000000000002},"last":0.24399999999999999},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.7230000000000001,"15min":1.8160000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.19500000000000001},"max":{"1min":11.895,"5min":14.749000000000001,"15min":91.876999999999995},"last":3.6949999999999998}]},{"osd":1,"last update":"Wed Sep 11 07:40:23 2024","interfaces":[{"interface":"back","average":{"1min":2.2309999999999999,"5min":3.1509999999999998,"15min":1.8959999999999999},"min":{"1min":0.28499999999999998,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":11.917999999999999,"5min":67.305000000000007,"15min":67.305000000000007},"last":0.41999999999999998},{"interface":"front","average":{"1min":1.1850000000000001,"5min":2.169,"15min":1.4279999999999999},"min":{"1min":0.27500000000000002,"5min":0.23799999999999999,"15min":0.192},"max":{"1min":6.9569999999999999,"5min":44.325000000000003,"15min":44.325000000000003},"last":3.6379999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:40:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.139,"15min":1.214},"min":{"1min":0.246,"5min":0.23699999999999999,"15min":0.23200000000000001},"max":{"1min":4.6799999999999997,"5min":6.6619999999999999,"15min":8.282},"last":3.6709999999999998},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.1619999999999999,"15min":1.147},"min":{"1min":0.29299999999999998,"5min":0.252,"15min":0.20999999999999999},"max":{"1min":5.992,"5min":6.5640000000000001,"15min":8.7110000000000003},"last":0.54200000000000004}]},{"osd":4,"last update":"Wed Sep 11 07:40:43 2024","interfaces":[{"interface":"back","average":{"1min":11.355,"5min":3.04,"15min":1.746},"min":{"1min":0.33300000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":132.76400000000001,"5min":132.76400000000001,"15min":132.76400000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":11.542,"5min":3.1960000000000002,"15min":1.9330000000000001},"min":{"1min":0.311,"5min":0.255,"15min":0.255},"max":{"1min":132.154,"5min":132.154,"15min":132.154},"last":3.7320000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:39:51 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.038,"15min":1.119},"min":{"1min":0.34799999999999998,"5min":0.309,"15min":0.29599999999999999},"max":{"1min":4.5439999999999996,"5min":4.5439999999999996,"15min":32.325000000000003},"last":3.488},{"interface":"front","average":{"1min":1.194,"5min":1.1990000000000001,"15min":1.214},"min":{"1min":0.309,"5min":0.28599999999999998,"15min":0.28299999999999997},"max":{"1min":3.7050000000000001,"5min":3.9830000000000001,"15min":18.416},"last":3.9529999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:40:23 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.131,"15min":1.2849999999999999},"min":{"1min":0.36199999999999999,"5min":0.313,"15min":0.27000000000000002},"max":{"1min":6.8680000000000003,"5min":6.8680000000000003,"15min":32.350999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.069,"15min":1.2410000000000001},"min":{"1min":0.34200000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":4.5199999999999996,"5min":4.5199999999999996,"15min":32.402999999999999},"last":3.5230000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:40:15 2024","interfaces":[{"interface":"back","average":{"1min":1.673,"5min":1.3580000000000001,"15min":1.345},"min":{"1min":0.35699999999999998,"5min":0.32300000000000001,"15min":0.308},"max":{"1min":11.864000000000001,"5min":11.864000000000001,"15min":11.864000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.2070000000000001,"15min":1.903},"min":{"1min":0.38700000000000001,"5min":0.36599999999999999,"15min":0.36099999999999999},"max":{"1min":7.0300000000000002,"5min":7.0300000000000002,"15min":32.289999999999999},"last":0.76800000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411528,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":848800,"kb_used_data":547280,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":92896352,"statfs":{"total":95995035648,"available":95125864448,"internally_reserved":0,"allocated":560414720,"data_stored":560019708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:39:53 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":1.897,"15min":1.744},"min":{"1min":0.221,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":4.1020000000000003,"5min":57.802999999999997,"15min":57.802999999999997},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.84499999999999997,"5min":2.1110000000000002,"15min":1.829},"min":{"1min":0.217,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":4.2859999999999996,"5min":57.759,"15min":57.759},"last":0.79700000000000004}]},{"osd":1,"last update":"Wed Sep 11 07:39:53 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":1.5900000000000001,"15min":1.47},"min":{"1min":0.23000000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":3.6549999999999998,"5min":48.042000000000002,"15min":65.153999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":2.0150000000000001,"15min":1.6899999999999999},"min":{"1min":0.28299999999999997,"5min":0.255,"15min":0.20699999999999999},"max":{"1min":5.1180000000000003,"5min":50.201000000000001,"15min":63.311999999999998},"last":0.23699999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:40:23 2024","interfaces":[{"interface":"back","average":{"1min":4.2480000000000002,"5min":2.1909999999999998,"15min":1.7629999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":67.552999999999997,"5min":67.552999999999997,"15min":67.552999999999997},"last":0.71099999999999997},{"interface":"front","average":{"1min":4.3449999999999998,"5min":2.0139999999999998,"15min":1.7589999999999999},"min":{"1min":0.23999999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":67.352999999999994,"5min":67.352999999999994,"15min":67.352999999999994},"last":0.85599999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:40:41 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.79500000000000004,"15min":1.083},"min":{"1min":0.315,"5min":0.30599999999999999,"15min":0.27500000000000002},"max":{"1min":2.4790000000000001,"5min":3.254,"15min":30.946999999999999},"last":0.316},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.81200000000000006,"15min":0.98099999999999998},"min":{"1min":0.33900000000000002,"5min":0.313,"15min":0.29999999999999999},"max":{"1min":2.7429999999999999,"5min":3.1669999999999998,"15min":22.434000000000001},"last":0.51900000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:39:53 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.92600000000000005,"15min":1.0920000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.31900000000000001},"max":{"1min":3.1960000000000002,"5min":3.3290000000000002,"15min":31.789000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":1.02,"5min":1.0109999999999999,"15min":1.1619999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.31},"max":{"1min":2.4540000000000002,"5min":4.1479999999999997,"15min":30.898},"last":0.34499999999999997}]},{"osd":6,"last update":"Wed Sep 11 07:40:23 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.89000000000000001,"15min":1.0820000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":6.891,"5min":6.891,"15min":22.399000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.875,"5min":0.92000000000000004,"15min":1.0860000000000001},"min":{"1min":0.377,"5min":0.311,"15min":0.29999999999999999},"max":{"1min":2.2400000000000002,"5min":4.1230000000000002,"15min":25.739000000000001},"last":0.44600000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:40:11 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.82299999999999995,"15min":1.6359999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.1349999999999998,"5min":2.6600000000000001,"15min":22.484000000000002},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.325,"5min":1.107,"15min":1.962},"min":{"1min":0.46000000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.9809999999999999,"5min":4.1719999999999997,"15min":25.670000000000002},"last":0.48599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575051,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1734644,"kb_used_data":1144740,"kb_used_omap":1,"kb_used_meta":589886,"kb_avail":92010508,"statfs":{"total":95995035648,"available":94218760192,"internally_reserved":0,"allocated":1172213760,"data_stored":1171799338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":604043608},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:40:22 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.6779999999999999,"15min":1.921},"min":{"1min":0.185,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":3.944,"5min":9.2669999999999995,"15min":82.218999999999994},"last":1.99},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.7869999999999999,"15min":1.956},"min":{"1min":0.255,"5min":0.186,"15min":0.186},"max":{"1min":5.0789999999999997,"5min":8.0739999999999998,"15min":82.278999999999996},"last":0.67500000000000004}]},{"osd":2,"last update":"Wed Sep 11 07:39:58 2024","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.8480000000000001,"15min":1.3720000000000001},"min":{"1min":0.308,"5min":0.22,"15min":0.19800000000000001},"max":{"1min":5.0190000000000001,"5min":9.0969999999999995,"15min":9.0969999999999995},"last":0.41699999999999998},{"interface":"front","average":{"1min":1.321,"5min":1.8160000000000001,"15min":1.4319999999999999},"min":{"1min":0.23899999999999999,"5min":0.191,"15min":0.191},"max":{"1min":4.4299999999999997,"5min":20.518000000000001,"15min":20.518000000000001},"last":1.835}]},{"osd":3,"last update":"Wed Sep 11 07:40:19 2024","interfaces":[{"interface":"back","average":{"1min":1.796,"5min":2.7080000000000002,"15min":2.2999999999999998},"min":{"1min":0.33000000000000002,"5min":0.27600000000000002,"15min":0.25800000000000001},"max":{"1min":4.774,"5min":67.120999999999995,"15min":81.882999999999996},"last":1.526},{"interface":"front","average":{"1min":1.1200000000000001,"5min":2.6080000000000001,"15min":2.2400000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.20999999999999999},"max":{"1min":4.3639999999999999,"5min":66.888999999999996,"15min":82.400000000000006},"last":2.0209999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:40:33 2024","interfaces":[{"interface":"back","average":{"1min":2.4940000000000002,"5min":1.8620000000000001,"15min":1.2290000000000001},"min":{"1min":0.30299999999999999,"5min":0.27500000000000002,"15min":0.27400000000000002},"max":{"1min":7.8390000000000004,"5min":7.8390000000000004,"15min":7.8390000000000004},"last":0.625},{"interface":"front","average":{"1min":2.2450000000000001,"5min":1.7729999999999999,"15min":1.3360000000000001},"min":{"1min":0.38700000000000001,"5min":0.29999999999999999,"15min":0.26500000000000001},"max":{"1min":7.8239999999999998,"5min":9.2289999999999992,"15min":9.2289999999999992},"last":7.1280000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:39:58 2024","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.704,"15min":1.194},"min":{"1min":0.28599999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":5.101,"5min":6.7469999999999999,"15min":7.4379999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":2.0259999999999998,"5min":1.6479999999999999,"15min":1.278},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":5.4790000000000001,"5min":9.1929999999999996,"15min":9.1929999999999996},"last":6.0919999999999996}]},{"osd":6,"last update":"Wed Sep 11 07:40:19 2024","interfaces":[{"interface":"back","average":{"1min":2.0219999999999998,"5min":1.6639999999999999,"15min":1.3520000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.30099999999999999},"max":{"1min":9.1850000000000005,"5min":9.2569999999999997,"15min":24.018000000000001},"last":1.252},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.3540000000000001,"15min":1.141},"min":{"1min":0.318,"5min":0.313,"15min":0.29999999999999999},"max":{"1min":4.968,"5min":9.3339999999999996,"15min":23.681000000000001},"last":0.79500000000000004}]},{"osd":7,"last update":"Wed Sep 11 07:40:19 2024","interfaces":[{"interface":"back","average":{"1min":2.6000000000000001,"5min":1.6659999999999999,"15min":1.417},"min":{"1min":0.36099999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":9.0489999999999995,"5min":9.3629999999999995,"15min":9.3629999999999995},"last":1.9359999999999999},{"interface":"front","average":{"1min":2.9340000000000002,"5min":2.0670000000000002,"15min":1.663},"min":{"1min":0.46300000000000002,"5min":0.36899999999999999,"15min":0.313},"max":{"1min":8.2219999999999995,"5min":8.2219999999999995,"15min":8.2219999999999995},"last":0.71599999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738577,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1387520,"kb_used_data":801644,"kb_used_omap":1,"kb_used_meta":585726,"kb_avail":92357632,"statfs":{"total":95995035648,"available":94574215168,"internally_reserved":0,"allocated":820883456,"data_stored":820467552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":599783768},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:40:27 2024","interfaces":[{"interface":"back","average":{"1min":2.9449999999999998,"5min":1.6479999999999999,"15min":1.274},"min":{"1min":0.24399999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":15.522,"5min":15.522,"15min":15.522},"last":1.611},{"interface":"front","average":{"1min":2.222,"5min":1.72,"15min":1.3009999999999999},"min":{"1min":0.254,"5min":0.20499999999999999,"15min":0.16600000000000001},"max":{"1min":13.988,"5min":13.988,"15min":13.988},"last":1.032}]},{"osd":2,"last update":"Wed Sep 11 07:39:53 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":3.3170000000000002,"15min":1.974},"min":{"1min":0.21199999999999999,"5min":0.188,"15min":0.188},"max":{"1min":2.6560000000000001,"5min":200.339,"15min":200.339},"last":1.2310000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":3.117,"15min":1.893},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":4.5529999999999999,"5min":200.40100000000001,"15min":200.40100000000001},"last":0.54300000000000004}]},{"osd":3,"last update":"Wed Sep 11 07:40:27 2024","interfaces":[{"interface":"back","average":{"1min":1.99,"5min":1.6619999999999999,"15min":1.9550000000000001},"min":{"1min":0.23300000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":8.8070000000000004,"5min":9.4920000000000009,"15min":45.185000000000002},"last":1.26},{"interface":"front","average":{"1min":2.1789999999999998,"5min":1.6319999999999999,"15min":1.746},"min":{"1min":0.251,"5min":0.24299999999999999,"15min":0.22800000000000001},"max":{"1min":15.252000000000001,"5min":15.252000000000001,"15min":43.228999999999999},"last":3.964}]},{"osd":4,"last update":"Wed Sep 11 07:40:36 2024","interfaces":[{"interface":"back","average":{"1min":3.0369999999999999,"5min":1.673,"15min":1.2490000000000001},"min":{"1min":0.311,"5min":0.26800000000000002,"15min":0.26700000000000002},"max":{"1min":30.315000000000001,"5min":30.315000000000001,"15min":30.315000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":3.3159999999999998,"5min":1.6919999999999999,"15min":1.3899999999999999},"min":{"1min":0.36899999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":30.363,"5min":30.363,"15min":30.363},"last":0.28599999999999998}]},{"osd":5,"last update":"Wed Sep 11 07:39:53 2024","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.3879999999999999,"15min":1.2090000000000001},"min":{"1min":0.32100000000000001,"5min":0.29599999999999999,"15min":0.28999999999999998},"max":{"1min":3.7629999999999999,"5min":5.0830000000000002,"15min":19.381},"last":0.47799999999999998},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3169999999999999,"15min":1.2989999999999999},"min":{"1min":0.442,"5min":0.36899999999999999,"15min":0.27200000000000002},"max":{"1min":3.528,"5min":9.6289999999999996,"15min":19.437000000000001},"last":0.38700000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:40:27 2024","interfaces":[{"interface":"back","average":{"1min":1.76,"5min":1.353,"15min":1.331},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.316},"max":{"1min":5.5010000000000003,"5min":9.5990000000000002,"15min":23.190999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":1.865,"5min":1.198,"15min":1.306},"min":{"1min":0.375,"5min":0.35299999999999998,"15min":0.34200000000000003},"max":{"1min":15.404,"5min":15.404,"15min":23.632000000000001},"last":0.33300000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:40:16 2024","interfaces":[{"interface":"back","average":{"1min":2.9039999999999999,"5min":1.7609999999999999,"15min":1.7030000000000001},"min":{"1min":0.47999999999999998,"5min":0.373,"15min":0.373},"max":{"1min":30.43,"5min":30.43,"15min":30.43},"last":0.432},{"interface":"front","average":{"1min":3.3410000000000002,"5min":1.7649999999999999,"15min":1.7450000000000001},"min":{"1min":0.36199999999999999,"5min":0.315,"15min":0.315},"max":{"1min":31.065000000000001,"5min":31.065000000000001,"15min":31.065000000000001},"last":0.35999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":801701888,"data_stored":801701888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1154289664,"data_stored":1154285592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":559546368,"data_stored":559546368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1198456832,"data_stored":1198456832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":718213120,"data_stored":718209048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1249775616,"data_stored":1249771544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":725356544,"data_stored":725352472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1218777088,"data_stored":1218773016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:40:52.217 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj -9, currently 1 2024-09-11T07:40:52.284 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:52.321 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:52.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:52.322 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:52.422 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:52.454 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:52.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:52.455 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:52.556 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:52.583 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:52.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:52.584 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:52.685 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:52.726 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:52.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:52.727 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:52.828 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:52.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:52.868 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:52.868 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:52.872 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:52 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:40:52.872 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:52 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:40:52.873 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:52 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:40:52.873 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:52 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:40:52.873 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:52 smithi022 ceph-mon[29462]: osdmap e318: 8 total, 8 up, 8 in 2024-09-11T07:40:52.873 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:52 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4075694366' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:40:52.873 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:52 smithi022 ceph-mon[29462]: pgmap v661: 158 pgs: 7 active+remapped+backfill_toofull, 151 active+clean; 1.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.26k op/s; 11693/88209 objects misplaced (13.256%); 7.3 MiB/s, 117 objects/s recovering 2024-09-11T07:40:52.873 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T07:40:52.874 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:52 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:40:52.874 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:52 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:40:52.874 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:52 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:40:52.874 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:52 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:40:52.874 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:52 smithi022 ceph-mon[37330]: osdmap e318: 8 total, 8 up, 8 in 2024-09-11T07:40:52.874 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:52 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4075694366' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:40:52.874 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:52 smithi022 ceph-mon[37330]: pgmap v661: 158 pgs: 7 active+remapped+backfill_toofull, 151 active+clean; 1.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.26k op/s; 11693/88209 objects misplaced (13.256%); 7.3 MiB/s, 117 objects/s recovering 2024-09-11T07:40:52.969 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:52.997 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:52.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:52.998 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:53.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:52 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:40:53.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:52 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:40:53.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:52 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:40:53.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:52 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:40:53.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:52 smithi104 ceph-mon[33507]: osdmap e318: 8 total, 8 up, 8 in 2024-09-11T07:40:53.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:52 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4075694366' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:40:53.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:52 smithi104 ceph-mon[33507]: pgmap v661: 158 pgs: 7 active+remapped+backfill_toofull, 151 active+clean; 1.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.26k op/s; 11693/88209 objects misplaced (13.256%); 7.3 MiB/s, 117 objects/s recovering 2024-09-11T07:40:53.098 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:53.147 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:53.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:53.149 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:53.250 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:53.282 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:53.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:53.283 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:53.332 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:40:53.384 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:53.443 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:53.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:53.445 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:53.545 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:53.586 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:53.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:53.587 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:53.688 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:53.724 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:53.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:53.725 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:53.726 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:40:53.826 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:53.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:53.870 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:53.870 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:53.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:40:53.971 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:54.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:54.016 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:54.016 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:54.117 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:54.149 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:54.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:54.150 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:54.166 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:40:54.240 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:40:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:40:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:40:54.251 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:54.273 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:54.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:54.274 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:54.348 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:40:54.349 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:40:54.349 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2024-09-11T07:40:54.349 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2024-09-11T07:40:54.349 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:40:54.375 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:54.416 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:54.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:54.417 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:54.518 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:54.554 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:54.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:54.555 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:54.656 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:54.701 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:54.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:54.702 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:54.803 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:54.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:54.841 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:54.842 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:54.909 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:54 smithi104 ceph-mon[33507]: pgmap v662: 158 pgs: 7 active+remapped+backfill_toofull, 151 active+clean; 1.7 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 918 B/s rd, 0 B/s wr, 1.01k op/s; 11369/86367 objects misplaced (13.164%); 5.3 MiB/s, 84 objects/s recovering 2024-09-11T07:40:54.916 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:40:54.943 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:54.975 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:54.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:54.976 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:55.077 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:55.086 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-11T07:40:55.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:55.104 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:55.104 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:55.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:54 smithi022 ceph-mon[29462]: pgmap v662: 158 pgs: 7 active+remapped+backfill_toofull, 151 active+clean; 1.7 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 918 B/s rd, 0 B/s wr, 1.01k op/s; 11369/86367 objects misplaced (13.164%); 5.3 MiB/s, 84 objects/s recovering 2024-09-11T07:40:55.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:54 smithi022 ceph-mon[37330]: pgmap v662: 158 pgs: 7 active+remapped+backfill_toofull, 151 active+clean; 1.7 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 918 B/s rd, 0 B/s wr, 1.01k op/s; 11369/86367 objects misplaced (13.164%); 5.3 MiB/s, 84 objects/s recovering 2024-09-11T07:40:55.205 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:55.252 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:55.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:55.253 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:55.354 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:55.391 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:55.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:55.392 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:55.493 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:55.524 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:55.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:55.525 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:55.536 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:40:55.625 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:55.668 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:55.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:55.669 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:55.770 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:55.806 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:55.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:55.807 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:55.907 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:55.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:55.946 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:55.946 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:56.047 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:56.081 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:56.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:56.082 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:56.184 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:56.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:56.225 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:56.226 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:56.327 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:56.351 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:56.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:56.352 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:56.453 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:56.490 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:56.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:56.491 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:56.592 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:56.625 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:56.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:56.626 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:56.727 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:56.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:56.752 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:56.752 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:56.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:56 smithi104 ceph-mon[33507]: osdmap e319: 8 total, 8 up, 8 in 2024-09-11T07:40:56.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:56 smithi104 ceph-mon[33507]: pgmap v664: 158 pgs: 7 active+remapped+backfill_toofull, 151 active+clean; 1.5 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.03k op/s; 9521/72027 objects misplaced (13.219%) 2024-09-11T07:40:56.853 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:56.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:56.905 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:56.905 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:56.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:56 smithi022 ceph-mon[29462]: osdmap e319: 8 total, 8 up, 8 in 2024-09-11T07:40:56.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:56 smithi022 ceph-mon[29462]: pgmap v664: 158 pgs: 7 active+remapped+backfill_toofull, 151 active+clean; 1.5 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.03k op/s; 9521/72027 objects misplaced (13.219%) 2024-09-11T07:40:56.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:56 smithi022 ceph-mon[37330]: osdmap e319: 8 total, 8 up, 8 in 2024-09-11T07:40:56.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:56 smithi022 ceph-mon[37330]: pgmap v664: 158 pgs: 7 active+remapped+backfill_toofull, 151 active+clean; 1.5 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.03k op/s; 9521/72027 objects misplaced (13.219%) 2024-09-11T07:40:57.007 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:57.029 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:57.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:57.030 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:57.131 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:57.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:57.171 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:57.171 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:57.218 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:40:57.218 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:40:57.219 INFO:tasks.thrashosds.thrasher:Shrinking pool .mgr 2024-09-11T07:40:57.219 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool .mgr by 10 min 8 2024-09-11T07:40:57.219 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:40:57.272 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:57.327 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:57.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:57.331 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:57.431 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:57.441 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:40:57.441 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2024-09-11T07:40:57.441 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2024-09-11T07:40:57.441 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2024-09-11T07:40:57.441 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.441 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:45929 9.e 9:7bc47a8b:::benchmark_data_smithi022_133943_object45928:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e255)", 2024-09-11T07:40:57.441 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:37:57.776927+0000", 2024-09-11T07:40:57.441 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 179.66045786800001, 2024-09-11T07:40:57.442 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13309727099999999, 2024-09-11T07:40:57.442 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:57.442 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:57.442 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:57.442 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:57.442 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:57.442 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 45929 2024-09-11T07:40:57.442 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.442 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:57.442 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.442 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:57.442 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:37:57.776927+0000", 2024-09-11T07:40:57.442 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.442 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.443 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.443 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:57.443 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:37:57.776927+0000", 2024-09-11T07:40:57.443 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.443 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.443 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.443 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:57.443 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:37:57.776926+0000", 2024-09-11T07:40:57.443 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:40:57.443 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.443 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.443 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:57.443 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:37:57.776959+0000", 2024-09-11T07:40:57.444 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2648999999999999e-05 2024-09-11T07:40:57.444 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.444 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.444 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:57.444 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:37:57.776965+0000", 2024-09-11T07:40:57.444 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2190000000000001e-06 2024-09-11T07:40:57.444 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.444 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.444 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:57.444 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:37:57.776969+0000", 2024-09-11T07:40:57.444 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.048e-06 2024-09-11T07:40:57.444 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.444 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.445 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.445 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:37:57.777081+0000", 2024-09-11T07:40:57.445 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011231299999999999 2024-09-11T07:40:57.445 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.445 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.445 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:57.445 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:37:57.777199+0000", 2024-09-11T07:40:57.445 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000117409 2024-09-11T07:40:57.445 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.445 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.445 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,4", 2024-09-11T07:40:57.445 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:37:57.777325+0000", 2024-09-11T07:40:57.445 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012632799999999999 2024-09-11T07:40:57.446 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.446 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.446 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:57.446 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:37:57.778905+0000", 2024-09-11T07:40:57.446 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0015801369999999999 2024-09-11T07:40:57.446 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.446 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.446 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.446 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:37:57.779390+0000", 2024-09-11T07:40:57.446 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000484909 2024-09-11T07:40:57.446 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.446 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.446 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.446 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:37:57.909938+0000", 2024-09-11T07:40:57.447 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13054824800000001 2024-09-11T07:40:57.447 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.447 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.447 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:57.447 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:37:57.909987+0000", 2024-09-11T07:40:57.447 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8730000000000003e-05 2024-09-11T07:40:57.447 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.447 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.447 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:57.447 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:37:57.910024+0000", 2024-09-11T07:40:57.447 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6998000000000001e-05 2024-09-11T07:40:57.447 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.447 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:57.447 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.448 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.448 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.448 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:86227 9.e 9:74b4edf6:::benchmark_data_smithi022_133943_object86226:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:40:57.448 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:27.528103+0000", 2024-09-11T07:40:57.448 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 149.90928194, 2024-09-11T07:40:57.448 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18483545900000001, 2024-09-11T07:40:57.448 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:57.448 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:57.448 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:57.448 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:57.448 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:57.449 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 86227 2024-09-11T07:40:57.449 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.449 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:57.449 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.449 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:57.449 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.528103+0000", 2024-09-11T07:40:57.449 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.449 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.449 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.450 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:57.450 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.528103+0000", 2024-09-11T07:40:57.450 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.450 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.450 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.450 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:57.450 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.528101+0000", 2024-09-11T07:40:57.450 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:40:57.450 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.450 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.451 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:57.451 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.528241+0000", 2024-09-11T07:40:57.451 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014016300000000001 2024-09-11T07:40:57.451 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.451 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.451 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:57.451 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.528250+0000", 2024-09-11T07:40:57.451 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8279999999999992e-06 2024-09-11T07:40:57.451 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.451 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.451 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:57.452 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.528255+0000", 2024-09-11T07:40:57.452 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0200000000000002e-06 2024-09-11T07:40:57.452 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.452 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.452 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.452 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.528294+0000", 2024-09-11T07:40:57.452 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8683000000000002e-05 2024-09-11T07:40:57.452 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.452 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.452 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:57.452 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.528397+0000", 2024-09-11T07:40:57.452 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010292400000000001 2024-09-11T07:40:57.452 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.452 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.453 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,4", 2024-09-11T07:40:57.453 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.528530+0000", 2024-09-11T07:40:57.453 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000132729 2024-09-11T07:40:57.453 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.453 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.453 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:57.453 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.531084+0000", 2024-09-11T07:40:57.453 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0025538969999999999 2024-09-11T07:40:57.453 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.453 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.453 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.453 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.531301+0000", 2024-09-11T07:40:57.453 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00021777799999999999 2024-09-11T07:40:57.453 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.454 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.454 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.454 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.712803+0000", 2024-09-11T07:40:57.454 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18150128800000001 2024-09-11T07:40:57.454 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.454 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.454 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:57.454 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.712872+0000", 2024-09-11T07:40:57.454 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9064000000000001e-05 2024-09-11T07:40:57.454 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.454 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.454 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:57.454 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.712938+0000", 2024-09-11T07:40:57.454 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6431000000000001e-05 2024-09-11T07:40:57.454 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.455 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:57.455 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.455 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.455 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.455 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:86231 9.3 9:c7d2fbd6:::benchmark_data_smithi022_133943_object86230:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:40:57.455 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:27.531792+0000", 2024-09-11T07:40:57.455 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 149.90559248400001, 2024-09-11T07:40:57.455 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.24124353800000001, 2024-09-11T07:40:57.455 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:57.455 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:57.455 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:57.455 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:57.456 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:57.456 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 86231 2024-09-11T07:40:57.456 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.456 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:57.456 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.456 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:57.457 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.531792+0000", 2024-09-11T07:40:57.457 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.457 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.457 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.457 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:57.457 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.531792+0000", 2024-09-11T07:40:57.457 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.457 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.457 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.457 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:57.457 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.531790+0000", 2024-09-11T07:40:57.457 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:40:57.458 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.458 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.458 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:57.458 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.531895+0000", 2024-09-11T07:40:57.458 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000105342 2024-09-11T07:40:57.458 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.458 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.458 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:57.458 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.531909+0000", 2024-09-11T07:40:57.458 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3998e-05 2024-09-11T07:40:57.458 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.458 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.458 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:57.458 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.531915+0000", 2024-09-11T07:40:57.459 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9800000000000003e-06 2024-09-11T07:40:57.459 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.459 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.459 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.459 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.531955+0000", 2024-09-11T07:40:57.459 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9527999999999998e-05 2024-09-11T07:40:57.459 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.459 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.459 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:57.459 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532091+0000", 2024-09-11T07:40:57.459 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013630399999999999 2024-09-11T07:40:57.459 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.459 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.460 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:40:57.460 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532398+0000", 2024-09-11T07:40:57.460 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00030691399999999998 2024-09-11T07:40:57.460 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.460 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.460 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.460 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536109+0000", 2024-09-11T07:40:57.460 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0037104379999999999 2024-09-11T07:40:57.460 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.460 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.460 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:57.460 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.537537+0000", 2024-09-11T07:40:57.460 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001428594 2024-09-11T07:40:57.460 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.461 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.461 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.461 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.772927+0000", 2024-09-11T07:40:57.461 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.235389549 2024-09-11T07:40:57.461 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.461 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.461 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:57.461 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.772991+0000", 2024-09-11T07:40:57.461 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4746999999999995e-05 2024-09-11T07:40:57.461 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.461 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.462 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:57.462 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.773036+0000", 2024-09-11T07:40:57.462 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4199000000000002e-05 2024-09-11T07:40:57.462 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.462 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:57.462 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.462 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.462 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.462 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:86233 9.e 9:746045db:::benchmark_data_smithi022_133943_object86232:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:40:57.462 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:27.532483+0000", 2024-09-11T07:40:57.462 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 149.904901131, 2024-09-11T07:40:57.462 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18061364699999999, 2024-09-11T07:40:57.462 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:57.463 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:57.463 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:57.463 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:57.463 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:57.463 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 86233 2024-09-11T07:40:57.463 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.463 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:57.463 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.463 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:57.463 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532483+0000", 2024-09-11T07:40:57.463 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.463 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.463 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.463 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:57.463 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532483+0000", 2024-09-11T07:40:57.464 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.464 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.464 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.464 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:57.464 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532482+0000", 2024-09-11T07:40:57.464 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:40:57.464 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.464 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.464 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:57.464 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532896+0000", 2024-09-11T07:40:57.464 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000413557 2024-09-11T07:40:57.464 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.464 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.464 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:57.465 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532910+0000", 2024-09-11T07:40:57.465 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4083999999999999e-05 2024-09-11T07:40:57.465 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.465 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.465 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:57.465 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532916+0000", 2024-09-11T07:40:57.465 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9939999999999997e-06 2024-09-11T07:40:57.465 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.465 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.465 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.465 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532971+0000", 2024-09-11T07:40:57.465 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5179999999999997e-05 2024-09-11T07:40:57.465 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.466 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.466 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:57.466 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.533088+0000", 2024-09-11T07:40:57.466 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000116618 2024-09-11T07:40:57.466 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.466 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.466 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,4", 2024-09-11T07:40:57.466 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.533232+0000", 2024-09-11T07:40:57.466 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000144191 2024-09-11T07:40:57.466 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.467 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.467 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:57.467 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.535865+0000", 2024-09-11T07:40:57.467 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0026336250000000001 2024-09-11T07:40:57.467 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.467 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.467 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.467 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.538553+0000", 2024-09-11T07:40:57.467 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0026877300000000002 2024-09-11T07:40:57.467 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.467 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.467 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.467 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.713002+0000", 2024-09-11T07:40:57.467 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17444870100000001 2024-09-11T07:40:57.468 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.468 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.468 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:57.468 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.713065+0000", 2024-09-11T07:40:57.468 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3140999999999997e-05 2024-09-11T07:40:57.468 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.468 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.468 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:57.468 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.713097+0000", 2024-09-11T07:40:57.468 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2076000000000003e-05 2024-09-11T07:40:57.468 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.468 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:57.468 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.468 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.469 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.469 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:86237 9.3 9:c1b2182b:::benchmark_data_smithi022_133943_object86236:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:40:57.469 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:27.535933+0000", 2024-09-11T07:40:57.469 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 149.90145150399999, 2024-09-11T07:40:57.469 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.23897332099999999, 2024-09-11T07:40:57.469 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:57.469 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:57.469 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:57.469 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:57.469 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:57.469 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 86237 2024-09-11T07:40:57.469 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.470 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:57.470 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.470 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:57.470 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.535933+0000", 2024-09-11T07:40:57.470 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.470 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.470 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.470 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:57.470 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.535933+0000", 2024-09-11T07:40:57.470 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.470 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.470 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.470 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:57.470 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.535931+0000", 2024-09-11T07:40:57.471 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:40:57.471 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.471 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.471 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:57.471 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.535972+0000", 2024-09-11T07:40:57.471 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0861000000000001e-05 2024-09-11T07:40:57.471 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.471 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.471 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:57.471 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.535985+0000", 2024-09-11T07:40:57.471 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2336e-05 2024-09-11T07:40:57.471 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.471 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.471 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:57.472 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.535990+0000", 2024-09-11T07:40:57.472 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.772e-06 2024-09-11T07:40:57.472 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.472 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.472 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.472 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536127+0000", 2024-09-11T07:40:57.472 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013698900000000001 2024-09-11T07:40:57.472 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.472 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.472 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:57.472 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536211+0000", 2024-09-11T07:40:57.472 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3901999999999997e-05 2024-09-11T07:40:57.472 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.472 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.473 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:40:57.473 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536455+0000", 2024-09-11T07:40:57.473 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00024371199999999999 2024-09-11T07:40:57.473 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.473 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.473 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:57.473 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.537667+0000", 2024-09-11T07:40:57.473 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00121188 2024-09-11T07:40:57.473 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.473 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.473 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.474 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.715691+0000", 2024-09-11T07:40:57.474 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.178024185 2024-09-11T07:40:57.474 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.474 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.474 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.474 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.774831+0000", 2024-09-11T07:40:57.474 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.059139518000000002 2024-09-11T07:40:57.474 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.474 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.474 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:57.474 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.774874+0000", 2024-09-11T07:40:57.474 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3080000000000001e-05 2024-09-11T07:40:57.475 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.475 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.475 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:57.475 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.774906+0000", 2024-09-11T07:40:57.475 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2827000000000003e-05 2024-09-11T07:40:57.475 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.475 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:57.475 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.475 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.475 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.475 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:86238 9.e 9:76c6605d:::benchmark_data_smithi022_133943_object86237:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:40:57.475 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:27.536029+0000", 2024-09-11T07:40:57.475 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 149.90135538600001, 2024-09-11T07:40:57.475 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18003709600000001, 2024-09-11T07:40:57.476 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:57.476 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:57.476 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:57.476 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:57.476 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:57.476 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 86238 2024-09-11T07:40:57.476 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.476 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:57.476 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.476 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:57.476 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536029+0000", 2024-09-11T07:40:57.476 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.477 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.477 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.477 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:57.477 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536029+0000", 2024-09-11T07:40:57.477 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.477 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.477 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.477 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:57.477 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536028+0000", 2024-09-11T07:40:57.477 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:40:57.478 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.478 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.478 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:57.478 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536060+0000", 2024-09-11T07:40:57.478 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1294000000000002e-05 2024-09-11T07:40:57.478 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.478 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.478 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:57.478 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536072+0000", 2024-09-11T07:40:57.478 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2031000000000001e-05 2024-09-11T07:40:57.478 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.478 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.479 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:57.479 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536077+0000", 2024-09-11T07:40:57.479 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0939999999999998e-06 2024-09-11T07:40:57.479 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.479 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.479 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.479 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536120+0000", 2024-09-11T07:40:57.479 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3315000000000001e-05 2024-09-11T07:40:57.479 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.479 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.479 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:57.480 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536227+0000", 2024-09-11T07:40:57.480 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010718 2024-09-11T07:40:57.480 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.480 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.480 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,4", 2024-09-11T07:40:57.480 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536331+0000", 2024-09-11T07:40:57.480 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000104227 2024-09-11T07:40:57.480 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.480 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.480 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:57.480 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.538631+0000", 2024-09-11T07:40:57.480 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002299784 2024-09-11T07:40:57.480 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.481 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.481 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.481 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.538870+0000", 2024-09-11T07:40:57.481 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00023881600000000001 2024-09-11T07:40:57.481 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.481 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.481 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.481 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.715974+0000", 2024-09-11T07:40:57.481 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17710394400000001 2024-09-11T07:40:57.481 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.482 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.482 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:57.482 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.716028+0000", 2024-09-11T07:40:57.482 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3665999999999997e-05 2024-09-11T07:40:57.482 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.482 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.482 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:57.482 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.716066+0000", 2024-09-11T07:40:57.482 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8702000000000003e-05 2024-09-11T07:40:57.482 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.482 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:57.482 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.482 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.482 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.482 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:86241 9.13 9:cc909b6c:::benchmark_data_smithi022_133943_object86240:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:40:57.483 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:27.536369+0000", 2024-09-11T07:40:57.483 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 149.9010159, 2024-09-11T07:40:57.483 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.23673264699999999, 2024-09-11T07:40:57.483 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:57.483 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:57.483 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:57.483 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:57.483 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:57.483 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 86241 2024-09-11T07:40:57.483 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.483 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:57.483 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.483 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:57.484 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536369+0000", 2024-09-11T07:40:57.484 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.484 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.484 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.484 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:57.484 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536369+0000", 2024-09-11T07:40:57.484 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.484 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.484 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.484 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:57.484 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536367+0000", 2024-09-11T07:40:57.484 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:40:57.484 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.484 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.485 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:57.485 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536426+0000", 2024-09-11T07:40:57.485 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8866999999999997e-05 2024-09-11T07:40:57.485 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.485 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.485 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:57.485 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536434+0000", 2024-09-11T07:40:57.485 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9389999999999997e-06 2024-09-11T07:40:57.485 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.485 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.485 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:57.485 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536440+0000", 2024-09-11T07:40:57.485 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2769999999999996e-06 2024-09-11T07:40:57.486 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.486 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.486 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.486 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.537555+0000", 2024-09-11T07:40:57.486 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0011153420000000001 2024-09-11T07:40:57.486 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.486 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.486 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:57.486 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.537650+0000", 2024-09-11T07:40:57.486 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4883999999999995e-05 2024-09-11T07:40:57.486 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.486 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.486 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:40:57.486 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.537882+0000", 2024-09-11T07:40:57.487 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000231952 2024-09-11T07:40:57.487 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.487 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.487 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:57.487 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.538889+0000", 2024-09-11T07:40:57.487 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0010068939999999999 2024-09-11T07:40:57.487 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.487 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.487 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.487 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.715709+0000", 2024-09-11T07:40:57.487 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17682025700000001 2024-09-11T07:40:57.487 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.487 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.487 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.488 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.772956+0000", 2024-09-11T07:40:57.488 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.057247167000000002 2024-09-11T07:40:57.488 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.488 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.488 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:57.488 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.773061+0000", 2024-09-11T07:40:57.488 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010540299999999999 2024-09-11T07:40:57.488 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.488 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.488 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:57.488 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.773101+0000", 2024-09-11T07:40:57.488 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9910999999999998e-05 2024-09-11T07:40:57.488 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.488 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:57.489 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.489 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.489 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.489 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87203 9.13 9:cc4de378:::benchmark_data_smithi022_133943_object87202:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:40:57.489 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.339440+0000", 2024-09-11T07:40:57.489 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 149.09794428699999, 2024-09-11T07:40:57.489 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1017081860000002, 2024-09-11T07:40:57.489 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:57.489 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:40:57.489 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:57.489 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:57.490 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:57.490 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87203 2024-09-11T07:40:57.490 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.490 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:57.490 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.490 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:57.490 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339440+0000", 2024-09-11T07:40:57.490 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.490 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.490 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.490 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:57.490 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339440+0000", 2024-09-11T07:40:57.490 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.491 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.491 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.491 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:57.491 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339440+0000", 2024-09-11T07:40:57.491 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:40:57.491 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.491 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.491 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:57.491 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339474+0000", 2024-09-11T07:40:57.491 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4149000000000001e-05 2024-09-11T07:40:57.491 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.491 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.491 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:57.491 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339480+0000", 2024-09-11T07:40:57.492 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3550000000000001e-06 2024-09-11T07:40:57.492 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.492 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.492 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:57.492 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339484+0000", 2024-09-11T07:40:57.492 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1239999999999998e-06 2024-09-11T07:40:57.492 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.492 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.492 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.492 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339567+0000", 2024-09-11T07:40:57.492 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3145000000000004e-05 2024-09-11T07:40:57.492 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.492 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.492 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:57.493 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339673+0000", 2024-09-11T07:40:57.493 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010586699999999999 2024-09-11T07:40:57.493 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.493 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.493 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:40:57.493 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339947+0000", 2024-09-11T07:40:57.493 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00027409800000000002 2024-09-11T07:40:57.493 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.493 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.493 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:57.493 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.341767+0000", 2024-09-11T07:40:57.493 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001819883 2024-09-11T07:40:57.493 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.494 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.494 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.494 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.342624+0000", 2024-09-11T07:40:57.494 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00085697099999999999 2024-09-11T07:40:57.494 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.494 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.494 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.494 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.347306+0000", 2024-09-11T07:40:57.494 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0046816230000000002 2024-09-11T07:40:57.494 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.494 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.494 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:40:57.494 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.347315+0000", 2024-09-11T07:40:57.494 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.1600000000000004e-06 2024-09-11T07:40:57.495 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.495 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.495 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.495 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.346303+0000", 2024-09-11T07:40:57.495 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.99898775399999995 2024-09-11T07:40:57.495 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.495 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.495 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:40:57.495 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.346312+0000", 2024-09-11T07:40:57.495 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.6889999999999999e-06 2024-09-11T07:40:57.495 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.495 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.495 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.495 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.436501+0000", 2024-09-11T07:40:57.496 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.090188951 2024-09-11T07:40:57.496 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.496 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.496 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for degraded object", 2024-09-11T07:40:57.496 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.437198+0000", 2024-09-11T07:40:57.496 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00069633100000000001 2024-09-11T07:40:57.496 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.496 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.496 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.496 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.441056+0000", 2024-09-11T07:40:57.496 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0038578359999999999 2024-09-11T07:40:57.496 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.496 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.496 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:57.497 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.441149+0000", 2024-09-11T07:40:57.497 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2905000000000006e-05 2024-09-11T07:40:57.497 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.497 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:57.497 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.497 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.497 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.497 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87197 9.17 9:ede1dd36:::benchmark_data_smithi022_133943_object87196:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:40:57.497 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.345260+0000", 2024-09-11T07:40:57.497 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 149.09212433799999, 2024-09-11T07:40:57.497 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0776407649999999, 2024-09-11T07:40:57.497 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:57.498 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:40:57.498 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:57.498 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:57.498 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:57.498 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87197 2024-09-11T07:40:57.498 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.498 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:57.498 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.498 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:57.498 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.345260+0000", 2024-09-11T07:40:57.498 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.498 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.498 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.498 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:57.499 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.345260+0000", 2024-09-11T07:40:57.499 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.499 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.499 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.499 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:57.499 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.345258+0000", 2024-09-11T07:40:57.499 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:40:57.499 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.499 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.499 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:57.499 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.345346+0000", 2024-09-11T07:40:57.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:57.501 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7859e-05 2024-09-11T07:40:57.501 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.501 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.501 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:57.501 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.345356+0000", 2024-09-11T07:40:57.502 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0205e-05 2024-09-11T07:40:57.502 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.502 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.502 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:57.502 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.345361+0000", 2024-09-11T07:40:57.502 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4199999999999998e-06 2024-09-11T07:40:57.502 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.502 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.502 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.502 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.347877+0000", 2024-09-11T07:40:57.502 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0025154270000000002 2024-09-11T07:40:57.502 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.502 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.502 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:40:57.503 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.347884+0000", 2024-09-11T07:40:57.503 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4699999999999996e-06 2024-09-11T07:40:57.503 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.503 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.503 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.503 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.344661+0000", 2024-09-11T07:40:57.503 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.99677682199999995 2024-09-11T07:40:57.503 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.503 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.503 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:40:57.503 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.344670+0000", 2024-09-11T07:40:57.503 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3449999999999995e-06 2024-09-11T07:40:57.503 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.504 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.504 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.504 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.422818+0000", 2024-09-11T07:40:57.504 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0781474900000001 2024-09-11T07:40:57.504 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.504 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.504 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:57.504 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.422901+0000", 2024-09-11T07:40:57.504 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3071000000000005e-05 2024-09-11T07:40:57.504 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.504 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:57.504 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.504 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.505 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.505 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87222 9.18 9.9a7e3098 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:40:57.505 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.362550+0000", 2024-09-11T07:40:57.505 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 149.07483412299999, 2024-09-11T07:40:57.505 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.98180590199999995, 2024-09-11T07:40:57.505 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:57.505 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:40:57.505 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:57.505 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:57.505 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:57.505 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87222 2024-09-11T07:40:57.505 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.505 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:57.505 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.506 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:57.506 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.362550+0000", 2024-09-11T07:40:57.506 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.506 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.506 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.506 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:57.506 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.362550+0000", 2024-09-11T07:40:57.506 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.506 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.506 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.506 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:57.506 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.362549+0000", 2024-09-11T07:40:57.506 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:40:57.506 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.506 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.507 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:57.507 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.362658+0000", 2024-09-11T07:40:57.507 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000108529 2024-09-11T07:40:57.507 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.507 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.507 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:57.507 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.362667+0000", 2024-09-11T07:40:57.507 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9020000000000005e-06 2024-09-11T07:40:57.507 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.507 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.507 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:57.507 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.362670+0000", 2024-09-11T07:40:57.507 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4340000000000001e-06 2024-09-11T07:40:57.507 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.507 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.507 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.508 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.362714+0000", 2024-09-11T07:40:57.508 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3615000000000002e-05 2024-09-11T07:40:57.508 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.508 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.508 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:40:57.508 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.362719+0000", 2024-09-11T07:40:57.508 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0350000000000001e-06 2024-09-11T07:40:57.508 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.508 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.508 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.508 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.344348+0000", 2024-09-11T07:40:57.508 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.98162881999999996 2024-09-11T07:40:57.508 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.508 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.508 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:57.509 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.344356+0000", 2024-09-11T07:40:57.509 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6580000000000008e-06 2024-09-11T07:40:57.509 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.509 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:57.509 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.509 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.509 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.509 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87236 9.15 9:aa3be195:::benchmark_data_smithi022_133943_object87235:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:40:57.509 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.369488+0000", 2024-09-11T07:40:57.509 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 149.067897045, 2024-09-11T07:40:57.509 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.156802254, 2024-09-11T07:40:57.509 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:57.509 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:57.509 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:57.509 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:57.510 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:57.510 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87236 2024-09-11T07:40:57.510 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.510 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:57.510 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.510 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:57.510 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369488+0000", 2024-09-11T07:40:57.510 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.510 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.510 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.510 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:57.510 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369488+0000", 2024-09-11T07:40:57.510 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.510 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.510 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.511 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:57.511 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369486+0000", 2024-09-11T07:40:57.511 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:40:57.511 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.511 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.511 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:57.511 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369534+0000", 2024-09-11T07:40:57.511 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8127000000000002e-05 2024-09-11T07:40:57.511 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.511 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.511 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:57.511 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369545+0000", 2024-09-11T07:40:57.511 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0920999999999999e-05 2024-09-11T07:40:57.511 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.512 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.512 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:57.512 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369549+0000", 2024-09-11T07:40:57.512 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4580000000000003e-06 2024-09-11T07:40:57.512 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.512 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.512 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.512 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369610+0000", 2024-09-11T07:40:57.512 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0631000000000003e-05 2024-09-11T07:40:57.512 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.512 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.512 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:40:57.512 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369618+0000", 2024-09-11T07:40:57.512 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7420000000000003e-06 2024-09-11T07:40:57.513 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.513 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.513 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.513 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.345692+0000", 2024-09-11T07:40:57.513 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.97607389 2024-09-11T07:40:57.513 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.513 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.513 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:40:57.513 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.345702+0000", 2024-09-11T07:40:57.513 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0086e-05 2024-09-11T07:40:57.513 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.513 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.513 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.514 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.510602+0000", 2024-09-11T07:40:57.514 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1648999250000001 2024-09-11T07:40:57.514 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.514 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.514 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:57.514 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.510767+0000", 2024-09-11T07:40:57.514 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000164656 2024-09-11T07:40:57.514 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.514 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.514 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:40:57.514 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.511002+0000", 2024-09-11T07:40:57.514 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000234699 2024-09-11T07:40:57.514 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.514 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.515 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:57.515 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.514166+0000", 2024-09-11T07:40:57.515 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0031640069999999999 2024-09-11T07:40:57.515 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.515 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.515 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.515 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.523959+0000", 2024-09-11T07:40:57.515 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0097934619999999993 2024-09-11T07:40:57.515 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.515 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.515 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.515 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.523981+0000", 2024-09-11T07:40:57.516 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.181e-05 2024-09-11T07:40:57.516 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.516 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.516 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:57.516 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.524012+0000", 2024-09-11T07:40:57.516 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1241000000000001e-05 2024-09-11T07:40:57.516 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.516 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.516 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:57.516 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.526290+0000", 2024-09-11T07:40:57.516 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0022775819999999998 2024-09-11T07:40:57.516 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.516 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:57.516 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.517 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.517 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.517 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87239 9.17 9:eb730478:::benchmark_data_smithi022_133943_object87238:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:40:57.517 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.369713+0000", 2024-09-11T07:40:57.517 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 149.06767192999999, 2024-09-11T07:40:57.517 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0664777230000002, 2024-09-11T07:40:57.517 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:57.517 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:57.517 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:57.517 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:57.517 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:57.517 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87239 2024-09-11T07:40:57.517 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.517 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:57.518 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.518 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:57.518 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369713+0000", 2024-09-11T07:40:57.518 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.518 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.518 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.518 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:57.518 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369713+0000", 2024-09-11T07:40:57.518 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.518 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.518 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.518 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:57.518 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369712+0000", 2024-09-11T07:40:57.518 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:40:57.518 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.519 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.519 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:57.519 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369781+0000", 2024-09-11T07:40:57.519 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9408999999999999e-05 2024-09-11T07:40:57.519 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.519 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.519 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:57.519 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369796+0000", 2024-09-11T07:40:57.519 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4355999999999999e-05 2024-09-11T07:40:57.519 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.519 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.519 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:57.519 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369801+0000", 2024-09-11T07:40:57.519 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1410000000000004e-06 2024-09-11T07:40:57.519 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.520 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.520 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.520 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369852+0000", 2024-09-11T07:40:57.520 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0983000000000003e-05 2024-09-11T07:40:57.520 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.520 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.520 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:40:57.520 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369860+0000", 2024-09-11T07:40:57.520 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6200000000000005e-06 2024-09-11T07:40:57.520 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.520 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.520 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.520 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.344696+0000", 2024-09-11T07:40:57.520 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.97483578000000004 2024-09-11T07:40:57.520 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.521 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.521 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:40:57.521 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.344703+0000", 2024-09-11T07:40:57.521 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4240000000000004e-06 2024-09-11T07:40:57.521 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.521 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.521 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.521 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.422922+0000", 2024-09-11T07:40:57.521 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0782185689999999 2024-09-11T07:40:57.521 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.521 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.521 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:57.521 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.423049+0000", 2024-09-11T07:40:57.521 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012664800000000001 2024-09-11T07:40:57.521 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.522 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.522 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,1,3,5", 2024-09-11T07:40:57.522 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.423321+0000", 2024-09-11T07:40:57.522 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000272355 2024-09-11T07:40:57.522 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.522 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.522 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.522 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.428425+0000", 2024-09-11T07:40:57.522 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0051037970000000002 2024-09-11T07:40:57.522 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.522 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.522 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.522 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.429833+0000", 2024-09-11T07:40:57.522 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0014079769999999999 2024-09-11T07:40:57.523 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.523 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.523 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.523 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.435362+0000", 2024-09-11T07:40:57.523 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0055289069999999996 2024-09-11T07:40:57.523 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.523 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.523 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.523 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.435951+0000", 2024-09-11T07:40:57.523 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000589158 2024-09-11T07:40:57.523 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.523 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.523 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:57.523 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.435960+0000", 2024-09-11T07:40:57.523 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9600000000000006e-06 2024-09-11T07:40:57.523 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.524 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.524 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:57.524 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.436059+0000", 2024-09-11T07:40:57.524 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8784999999999997e-05 2024-09-11T07:40:57.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.524 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.524 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:57.524 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.436190+0000", 2024-09-11T07:40:57.524 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000131772 2024-09-11T07:40:57.524 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.524 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:57.524 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.524 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.524 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87240 9.18 9.2c746ab8 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:40:57.525 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.372326+0000", 2024-09-11T07:40:57.525 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 149.06505853499999, 2024-09-11T07:40:57.525 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.97205600599999997, 2024-09-11T07:40:57.525 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:57.525 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:40:57.525 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:57.525 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:57.525 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:57.525 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87240 2024-09-11T07:40:57.525 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.525 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:57.525 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.525 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:57.525 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372326+0000", 2024-09-11T07:40:57.525 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.526 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.526 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.526 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:57.526 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372326+0000", 2024-09-11T07:40:57.526 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.526 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.526 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.526 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:57.526 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372324+0000", 2024-09-11T07:40:57.526 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:40:57.526 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.526 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.526 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:57.527 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372381+0000", 2024-09-11T07:40:57.527 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7188000000000002e-05 2024-09-11T07:40:57.527 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.527 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.527 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:57.527 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372394+0000", 2024-09-11T07:40:57.527 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2901999999999999e-05 2024-09-11T07:40:57.527 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.527 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.527 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:57.527 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372400+0000", 2024-09-11T07:40:57.527 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.699e-06 2024-09-11T07:40:57.527 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.527 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.527 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.528 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372439+0000", 2024-09-11T07:40:57.528 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9839000000000003e-05 2024-09-11T07:40:57.528 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.528 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.528 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:40:57.528 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372445+0000", 2024-09-11T07:40:57.528 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9490000000000001e-06 2024-09-11T07:40:57.528 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.528 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.528 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.529 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.344376+0000", 2024-09-11T07:40:57.529 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.97193081100000001 2024-09-11T07:40:57.529 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.529 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.529 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:57.529 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.344382+0000", 2024-09-11T07:40:57.529 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9070000000000004e-06 2024-09-11T07:40:57.529 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.529 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:57.529 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.530 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.530 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.530 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87250 9.15 9:ac3c2ae6:::benchmark_data_smithi022_133943_object87249:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:40:57.530 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.377199+0000", 2024-09-11T07:40:57.530 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 149.06018610699999, 2024-09-11T07:40:57.530 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1512879869999999, 2024-09-11T07:40:57.530 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:57.530 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:57.530 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:57.530 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:57.530 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:57.530 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87250 2024-09-11T07:40:57.530 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.530 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:57.531 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.531 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:57.531 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.377199+0000", 2024-09-11T07:40:57.531 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.531 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.531 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.531 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:57.531 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.377199+0000", 2024-09-11T07:40:57.531 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.531 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.531 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.531 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:57.531 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.377197+0000", 2024-09-11T07:40:57.532 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:40:57.532 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.532 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.532 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:57.532 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.377241+0000", 2024-09-11T07:40:57.532 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3498999999999998e-05 2024-09-11T07:40:57.532 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.532 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.532 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:57.532 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.377249+0000", 2024-09-11T07:40:57.532 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0989999999999999e-06 2024-09-11T07:40:57.532 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.532 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.533 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:57.533 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.377254+0000", 2024-09-11T07:40:57.533 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1220000000000003e-06 2024-09-11T07:40:57.533 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.533 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.533 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.533 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.377302+0000", 2024-09-11T07:40:57.533 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7479999999999999e-05 2024-09-11T07:40:57.533 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.533 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.533 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:40:57.533 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.377311+0000", 2024-09-11T07:40:57.533 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2359999999999993e-06 2024-09-11T07:40:57.534 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.534 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.534 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.534 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.345770+0000", 2024-09-11T07:40:57.534 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.968458763 2024-09-11T07:40:57.534 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.534 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.534 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:40:57.534 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.345779+0000", 2024-09-11T07:40:57.534 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0869999999999996e-06 2024-09-11T07:40:57.534 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.534 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.534 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.535 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.512718+0000", 2024-09-11T07:40:57.535 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.166939323 2024-09-11T07:40:57.535 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.535 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.535 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:57.535 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.512902+0000", 2024-09-11T07:40:57.535 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018403899999999999 2024-09-11T07:40:57.535 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.535 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.535 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:40:57.535 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.513149+0000", 2024-09-11T07:40:57.535 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00024737899999999999 2024-09-11T07:40:57.536 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.536 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.536 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:57.536 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.519581+0000", 2024-09-11T07:40:57.536 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0064314819999999997 2024-09-11T07:40:57.536 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.536 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.536 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.536 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.526384+0000", 2024-09-11T07:40:57.536 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0068032070000000004 2024-09-11T07:40:57.536 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.536 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.537 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.537 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.526419+0000", 2024-09-11T07:40:57.537 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5036000000000001e-05 2024-09-11T07:40:57.537 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.537 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.537 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:57.537 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.526478+0000", 2024-09-11T07:40:57.537 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8462999999999999e-05 2024-09-11T07:40:57.537 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.537 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.537 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:57.537 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.528486+0000", 2024-09-11T07:40:57.537 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0020089330000000001 2024-09-11T07:40:57.538 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.538 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:57.538 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.538 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.538 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.538 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87257 9.13 9:ca17476d:::benchmark_data_smithi022_133943_object87256:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:40:57.538 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.384938+0000", 2024-09-11T07:40:57.538 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 149.052447005, 2024-09-11T07:40:57.538 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0584020930000002, 2024-09-11T07:40:57.538 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:57.538 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:57.538 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:57.538 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:57.538 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:57.539 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87257 2024-09-11T07:40:57.539 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.539 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:57.539 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.539 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:57.539 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.384938+0000", 2024-09-11T07:40:57.539 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.539 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.539 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.539 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:57.539 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.384938+0000", 2024-09-11T07:40:57.539 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.539 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.539 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.540 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:57.540 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.384936+0000", 2024-09-11T07:40:57.540 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:40:57.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.540 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:57.540 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.384998+0000", 2024-09-11T07:40:57.540 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1383000000000005e-05 2024-09-11T07:40:57.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.540 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:57.540 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.385007+0000", 2024-09-11T07:40:57.540 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3370000000000006e-06 2024-09-11T07:40:57.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.541 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.541 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:57.541 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.385011+0000", 2024-09-11T07:40:57.541 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9840000000000003e-06 2024-09-11T07:40:57.541 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.541 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.541 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.541 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.385067+0000", 2024-09-11T07:40:57.541 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6425999999999998e-05 2024-09-11T07:40:57.541 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.541 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.541 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:40:57.542 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.385079+0000", 2024-09-11T07:40:57.542 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.155e-05 2024-09-11T07:40:57.542 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.542 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.542 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.542 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.346702+0000", 2024-09-11T07:40:57.542 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.96162294299999995 2024-09-11T07:40:57.542 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.542 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.542 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:40:57.542 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.346713+0000", 2024-09-11T07:40:57.542 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0688e-05 2024-09-11T07:40:57.543 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.543 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.543 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.543 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.437750+0000", 2024-09-11T07:40:57.543 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0910372879999999 2024-09-11T07:40:57.543 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.543 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.543 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:57.543 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.437940+0000", 2024-09-11T07:40:57.543 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019046499999999999 2024-09-11T07:40:57.543 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.544 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.544 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3,5,6", 2024-09-11T07:40:57.544 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.438830+0000", 2024-09-11T07:40:57.544 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00088921499999999995 2024-09-11T07:40:57.544 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.544 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.544 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:57.544 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.440627+0000", 2024-09-11T07:40:57.544 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001797873 2024-09-11T07:40:57.544 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.544 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.544 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.544 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.441186+0000", 2024-09-11T07:40:57.544 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00055897300000000004 2024-09-11T07:40:57.545 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.545 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.545 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.441752+0000", 2024-09-11T07:40:57.545 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00056608299999999999 2024-09-11T07:40:57.545 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.545 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.545 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.442310+0000", 2024-09-11T07:40:57.545 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00055787000000000002 2024-09-11T07:40:57.545 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.545 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.545 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.443120+0000", 2024-09-11T07:40:57.546 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00080973899999999997 2024-09-11T07:40:57.546 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.546 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.546 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:57.546 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.443167+0000", 2024-09-11T07:40:57.546 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7302999999999998e-05 2024-09-11T07:40:57.546 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.546 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.546 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:57.546 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.443340+0000", 2024-09-11T07:40:57.546 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017236000000000001 2024-09-11T07:40:57.546 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.546 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:57.546 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.547 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.547 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.547 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196445 9.17 9:edf5eae4:::benchmark_data_smithi022_133943_object69854:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:40:57.547 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.519449+0000", 2024-09-11T07:40:57.547 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 39.917935753000002, 2024-09-11T07:40:57.547 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.161483352, 2024-09-11T07:40:57.547 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:57.547 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:57.547 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:57.547 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:57.547 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:57.547 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196445 2024-09-11T07:40:57.547 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.547 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:57.547 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.548 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:57.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.519449+0000", 2024-09-11T07:40:57.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.548 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.548 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:57.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.519449+0000", 2024-09-11T07:40:57.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.548 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.548 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:57.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.519447+0000", 2024-09-11T07:40:57.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:40:57.549 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.549 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:57.549 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.519453+0000", 2024-09-11T07:40:57.549 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0619999999999997e-06 2024-09-11T07:40:57.549 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.549 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:57.549 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.519462+0000", 2024-09-11T07:40:57.549 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.1009999999999998e-06 2024-09-11T07:40:57.550 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.550 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.550 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:57.550 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.519468+0000", 2024-09-11T07:40:57.550 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7950000000000004e-06 2024-09-11T07:40:57.550 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.550 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.550 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.550 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.522372+0000", 2024-09-11T07:40:57.550 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0029042389999999999 2024-09-11T07:40:57.550 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.550 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.550 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:57.551 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.522451+0000", 2024-09-11T07:40:57.551 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9005000000000007e-05 2024-09-11T07:40:57.551 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.551 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.551 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,1,2,3,5", 2024-09-11T07:40:57.551 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.658722+0000", 2024-09-11T07:40:57.551 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.136270488 2024-09-11T07:40:57.552 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.552 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.552 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.552 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.665102+0000", 2024-09-11T07:40:57.552 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0063799490000000002 2024-09-11T07:40:57.552 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.552 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.552 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.552 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.665138+0000", 2024-09-11T07:40:57.552 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6230999999999999e-05 2024-09-11T07:40:57.552 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.552 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.552 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.552 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.665177+0000", 2024-09-11T07:40:57.553 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9316000000000001e-05 2024-09-11T07:40:57.553 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.553 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.553 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.553 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.670335+0000", 2024-09-11T07:40:57.553 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0051581659999999996 2024-09-11T07:40:57.553 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.553 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.553 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:57.553 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.670342+0000", 2024-09-11T07:40:57.553 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0600000000000002e-06 2024-09-11T07:40:57.553 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.553 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.554 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.554 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.680708+0000", 2024-09-11T07:40:57.554 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.010365592 2024-09-11T07:40:57.554 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.554 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.554 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:57.554 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.680765+0000", 2024-09-11T07:40:57.554 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6641999999999997e-05 2024-09-11T07:40:57.554 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.554 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.554 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:57.554 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.680932+0000", 2024-09-11T07:40:57.554 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016753800000000001 2024-09-11T07:40:57.554 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.554 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:57.555 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.555 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.555 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.555 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.15504.0:196448 9.c e299/247 9:349e6908:::benchmark_data_smithi022_133943_object69857:head v 299'12117, mlcod=299'12114)", 2024-09-11T07:40:57.555 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.521681+0000", 2024-09-11T07:40:57.555 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 39.915703284000003, 2024-09-11T07:40:57.555 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13854418399999999, 2024-09-11T07:40:57.555 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:57.555 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:57.555 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:57.555 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.556 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:57.556 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.521681+0000", 2024-09-11T07:40:57.556 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.556 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.556 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.556 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:57.556 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.521681+0000", 2024-09-11T07:40:57.556 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.556 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.556 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.556 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:57.556 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.521680+0000", 2024-09-11T07:40:57.556 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:40:57.556 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.557 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.557 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:57.557 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.521686+0000", 2024-09-11T07:40:57.557 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5740000000000003e-06 2024-09-11T07:40:57.557 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.557 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.557 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:57.557 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.521693+0000", 2024-09-11T07:40:57.557 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0099999999999998e-06 2024-09-11T07:40:57.557 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.557 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.557 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:57.557 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.521697+0000", 2024-09-11T07:40:57.557 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6600000000000001e-06 2024-09-11T07:40:57.558 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.558 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.558 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.558 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.521723+0000", 2024-09-11T07:40:57.558 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.6754000000000002e-05 2024-09-11T07:40:57.558 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.558 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.558 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:57.558 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.521735+0000", 2024-09-11T07:40:57.558 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1131000000000001e-05 2024-09-11T07:40:57.558 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.558 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.558 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:57.558 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.660106+0000", 2024-09-11T07:40:57.559 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13837102200000001 2024-09-11T07:40:57.559 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.559 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.559 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:57.559 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.660226+0000", 2024-09-11T07:40:57.559 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000119925 2024-09-11T07:40:57.559 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.559 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:57.559 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.559 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.559 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.559 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.15504.0:196449 9.c e299/247 9:3c3917ff:::benchmark_data_smithi022_133943_object69858:head v 299'12118, mlcod=299'12114)", 2024-09-11T07:40:57.559 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.523914+0000", 2024-09-11T07:40:57.559 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 39.913470754000002, 2024-09-11T07:40:57.560 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.143445717, 2024-09-11T07:40:57.560 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:57.560 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:57.560 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:57.560 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.560 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:57.560 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.523914+0000", 2024-09-11T07:40:57.560 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.560 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.560 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.561 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:57.561 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.523914+0000", 2024-09-11T07:40:57.561 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.561 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.561 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.561 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:57.561 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.523912+0000", 2024-09-11T07:40:57.561 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:40:57.561 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.561 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.561 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:57.561 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.523923+0000", 2024-09-11T07:40:57.561 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1362999999999999e-05 2024-09-11T07:40:57.562 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.562 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.562 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:57.562 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.658139+0000", 2024-09-11T07:40:57.562 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13421586299999999 2024-09-11T07:40:57.562 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.562 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.562 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:57.562 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.658149+0000", 2024-09-11T07:40:57.562 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4360000000000008e-06 2024-09-11T07:40:57.562 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.563 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.563 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.563 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.660255+0000", 2024-09-11T07:40:57.563 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0021059709999999999 2024-09-11T07:40:57.563 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.563 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.563 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:57.563 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.660276+0000", 2024-09-11T07:40:57.563 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0928e-05 2024-09-11T07:40:57.563 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.563 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.563 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:57.563 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.667249+0000", 2024-09-11T07:40:57.563 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0069737710000000001 2024-09-11T07:40:57.564 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.564 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.564 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:57.564 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.667360+0000", 2024-09-11T07:40:57.564 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000110208 2024-09-11T07:40:57.564 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.564 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:57.564 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.564 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.564 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.564 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:228990 9.3 9:c0b226b6:::benchmark_data_smithi022_133943_object102399:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e318)", 2024-09-11T07:40:57.564 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:54.232834+0000", 2024-09-11T07:40:57.564 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 3.2045507789999998, 2024-09-11T07:40:57.564 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13338493700000001, 2024-09-11T07:40:57.565 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:57.565 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:57.565 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:57.565 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:57.565 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:57.565 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 228990 2024-09-11T07:40:57.565 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.565 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:57.565 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.565 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:57.565 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:54.232834+0000", 2024-09-11T07:40:57.565 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.565 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.566 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.566 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:57.566 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:54.232834+0000", 2024-09-11T07:40:57.566 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.566 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.566 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.566 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:57.567 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:54.232833+0000", 2024-09-11T07:40:57.567 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:40:57.567 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.567 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.567 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:57.568 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:54.232837+0000", 2024-09-11T07:40:57.568 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1520000000000002e-06 2024-09-11T07:40:57.568 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.568 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.568 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:57.568 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:54.232843+0000", 2024-09-11T07:40:57.568 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8429999999999998e-06 2024-09-11T07:40:57.569 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.569 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.569 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:57.569 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:54.232848+0000", 2024-09-11T07:40:57.569 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7110000000000003e-06 2024-09-11T07:40:57.569 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.570 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.570 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.570 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:54.235895+0000", 2024-09-11T07:40:57.570 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0030476370000000002 2024-09-11T07:40:57.570 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.570 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.571 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:57.571 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:54.235992+0000", 2024-09-11T07:40:57.571 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.6280999999999995e-05 2024-09-11T07:40:57.571 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.571 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.571 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:40:57.572 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:54.236430+0000", 2024-09-11T07:40:57.572 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000438009 2024-09-11T07:40:57.572 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.572 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.572 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:57.572 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:54.245051+0000", 2024-09-11T07:40:57.572 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0086219049999999992 2024-09-11T07:40:57.572 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.573 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.573 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.573 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:54.245089+0000", 2024-09-11T07:40:57.573 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7081e-05 2024-09-11T07:40:57.573 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.573 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.573 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.574 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:54.366073+0000", 2024-09-11T07:40:57.574 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.120984379 2024-09-11T07:40:57.574 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.574 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.574 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:57.574 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:54.366163+0000", 2024-09-11T07:40:57.574 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0025999999999995e-05 2024-09-11T07:40:57.574 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.575 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.575 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:57.575 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:54.366219+0000", 2024-09-11T07:40:57.575 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5853999999999998e-05 2024-09-11T07:40:57.575 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.575 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:57.575 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.575 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.576 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.576 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:231574 9.3 9:c191c235:::benchmark_data_smithi022_133943_object104983:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:40:57.576 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:56.615171+0000", 2024-09-11T07:40:57.576 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.82221378499999997, 2024-09-11T07:40:57.576 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.136530492, 2024-09-11T07:40:57.576 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:40:57.576 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:40:57.576 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:40:57.577 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:40:57.577 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:40:57.577 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 231574 2024-09-11T07:40:57.577 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.577 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:40:57.577 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.577 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:40:57.577 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.615171+0000", 2024-09-11T07:40:57.577 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.578 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.578 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:57.578 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:57.579 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.579 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:40:57.579 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.615171+0000", 2024-09-11T07:40:57.579 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:40:57.579 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.579 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.580 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:40:57.580 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.615168+0000", 2024-09-11T07:40:57.580 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:40:57.580 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.580 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.581 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:40:57.581 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.615176+0000", 2024-09-11T07:40:57.581 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1210000000000007e-06 2024-09-11T07:40:57.581 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.581 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.581 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:40:57.581 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.615193+0000", 2024-09-11T07:40:57.581 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6796e-05 2024-09-11T07:40:57.581 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.581 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.582 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:40:57.582 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.615200+0000", 2024-09-11T07:40:57.582 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3840000000000002e-06 2024-09-11T07:40:57.582 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.582 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.582 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:40:57.582 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.615242+0000", 2024-09-11T07:40:57.582 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2320000000000001e-05 2024-09-11T07:40:57.582 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.582 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.582 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:40:57.582 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.615351+0000", 2024-09-11T07:40:57.583 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000108884 2024-09-11T07:40:57.583 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.583 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.583 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:40:57.583 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.615873+0000", 2024-09-11T07:40:57.583 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00052220700000000001 2024-09-11T07:40:57.583 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.583 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.583 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:40:57.583 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.618792+0000", 2024-09-11T07:40:57.583 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0029193180000000002 2024-09-11T07:40:57.583 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.583 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.583 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.584 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.620178+0000", 2024-09-11T07:40:57.584 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001386204 2024-09-11T07:40:57.584 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.584 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.584 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:40:57.584 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.751623+0000", 2024-09-11T07:40:57.584 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13144478400000001 2024-09-11T07:40:57.584 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.584 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.584 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:40:57.584 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.751676+0000", 2024-09-11T07:40:57.584 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3176e-05 2024-09-11T07:40:57.584 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:40:57.584 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:40:57.585 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:40:57.585 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.751701+0000", 2024-09-11T07:40:57.585 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.4905999999999998e-05 2024-09-11T07:40:57.585 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.585 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:57.585 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.585 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:40:57.585 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:40:57.585 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:40:57.679 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:57.704 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:57.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:57.705 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:57.806 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:57.828 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:57.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:57.829 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:57.930 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:57.961 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:57.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:57.962 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:58.063 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:58.099 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:58.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:58.101 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:58.201 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:58.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:58.229 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:58.229 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:58.331 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:40:58.361 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:58.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:58.363 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:40:58.463 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:40:58.466 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-11T07:40:58.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:58.512 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:58.512 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:40:58.613 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:58.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:58.635 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:58.636 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:58.737 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:58.766 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:58.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:58.767 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:58.868 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:58.890 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:58.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:58.891 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:58.992 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:59.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:59.032 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:59.033 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:40:59.133 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:40:59.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:59.156 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:59.156 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:40:59.257 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:59.289 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:59.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:59.290 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:59.391 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:40:59.420 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:59.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:59.420 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:40:59.521 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:40:59.551 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:59.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:59.553 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:40:59.653 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:40:59.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:40:59 smithi022 ceph-mon[29462]: pgmap v665: 158 pgs: 7 active+remapped+backfill_toofull, 151 active+clean; 1.5 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 673 B/s rd, 0 B/s wr, 710 op/s; 9521/72027 objects misplaced (13.219%) 2024-09-11T07:40:59.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:40:59 smithi022 ceph-mon[37330]: pgmap v665: 158 pgs: 7 active+remapped+backfill_toofull, 151 active+clean; 1.5 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 673 B/s rd, 0 B/s wr, 710 op/s; 9521/72027 objects misplaced (13.219%) 2024-09-11T07:40:59.689 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:59.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:59.690 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:40:59.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:40:59 smithi104 ceph-mon[33507]: pgmap v665: 158 pgs: 7 active+remapped+backfill_toofull, 151 active+clean; 1.5 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 673 B/s rd, 0 B/s wr, 710 op/s; 9521/72027 objects misplaced (13.219%) 2024-09-11T07:40:59.791 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:40:59.812 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:40:59.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:59.813 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:40:59.914 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:40:59.936 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:40:59.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:40:59.937 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:00.038 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:00.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:00.061 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:00.061 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:00.162 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:00.183 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:00.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:00.185 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:00.279 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:40:59 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:41:00.288 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:00.309 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:00.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:00.310 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:00.411 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:00.433 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:00.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:00.434 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:00.535 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:00.572 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:00.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:00.574 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:00.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:00 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:41:00.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:00 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:41:00.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:41:00 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:41:00] "GET /metrics HTTP/1.1" 200 224774 "" "Prometheus/2.43.0" 2024-09-11T07:41:00.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:00 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:41:00.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:00 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:41:00.674 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:00.708 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:00.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:00.709 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:00.765 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:41:00.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:00 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:41:00.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:00 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:41:00.806 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:41:00.810 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:00.846 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:00.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:00.847 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:00.948 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:00.974 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:00.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:00.975 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:01.076 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:01.107 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:01.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:01.108 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:01.209 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:01.244 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:01.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:01.245 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:01.346 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:01.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:01.374 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:01.374 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:01.475 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:01.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:01.507 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:01.507 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:01.580 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:01 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:41:01.580 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:01 smithi022 ceph-mon[29462]: pgmap v666: 158 pgs: 2 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 151 active+clean; 1.2 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.13k op/s; 7896/59283 objects misplaced (13.319%); 486 KiB/s, 7 objects/s recovering 2024-09-11T07:41:01.580 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:01 smithi022 ceph-mon[29462]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 5 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:41:01.580 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:01 smithi022 ceph-mon[29462]: osdmap e320: 8 total, 8 up, 8 in 2024-09-11T07:41:01.580 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:01 smithi022 ceph-mon[29462]: osdmap e321: 8 total, 8 up, 8 in 2024-09-11T07:41:01.580 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:01 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:41:01.580 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:01 smithi022 ceph-mon[37330]: pgmap v666: 158 pgs: 2 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 151 active+clean; 1.2 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.13k op/s; 7896/59283 objects misplaced (13.319%); 486 KiB/s, 7 objects/s recovering 2024-09-11T07:41:01.581 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:01 smithi022 ceph-mon[37330]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 5 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:41:01.581 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:01 smithi022 ceph-mon[37330]: osdmap e320: 8 total, 8 up, 8 in 2024-09-11T07:41:01.581 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:01 smithi022 ceph-mon[37330]: osdmap e321: 8 total, 8 up, 8 in 2024-09-11T07:41:01.608 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:01.643 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:01.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:01.644 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:01.745 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:01.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:01 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:41:01.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:01 smithi104 ceph-mon[33507]: pgmap v666: 158 pgs: 2 active+remapped+backfilling, 5 active+remapped+backfill_toofull, 151 active+clean; 1.2 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.13k op/s; 7896/59283 objects misplaced (13.319%); 486 KiB/s, 7 objects/s recovering 2024-09-11T07:41:01.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:01 smithi104 ceph-mon[33507]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 5 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:41:01.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:01 smithi104 ceph-mon[33507]: osdmap e320: 8 total, 8 up, 8 in 2024-09-11T07:41:01.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:01 smithi104 ceph-mon[33507]: osdmap e321: 8 total, 8 up, 8 in 2024-09-11T07:41:01.781 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:01.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:01.783 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:01.884 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:01.909 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:01.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:01.910 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:02.011 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:02.042 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:02.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:02.043 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:02.144 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:02.165 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:02.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:02.166 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:02.267 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:02.289 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:02.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:02.290 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:02.391 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:02.412 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:02.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:02.414 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:02.515 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:02.537 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:02.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:02.538 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:02.639 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:02.661 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:02.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:02.662 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:02.685 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is set 2024-09-11T07:41:02.763 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:02.800 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:02.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:02.801 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:02.902 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:02.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:02 smithi022 ceph-mon[29462]: osdmap e322: 8 total, 8 up, 8 in 2024-09-11T07:41:02.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:02 smithi022 ceph-mon[29462]: pgmap v670: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 1.2 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 825 op/s; 6946/57174 objects misplaced (12.149%); 2.8 MiB/s, 45 objects/s recovering 2024-09-11T07:41:02.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:02 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4255827210' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:41:02.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:02 smithi022 ceph-mon[29462]: from='client.25594 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:41:02.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:02 smithi022 ceph-mon[37330]: osdmap e322: 8 total, 8 up, 8 in 2024-09-11T07:41:02.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:02 smithi022 ceph-mon[37330]: pgmap v670: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 1.2 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 825 op/s; 6946/57174 objects misplaced (12.149%); 2.8 MiB/s, 45 objects/s recovering 2024-09-11T07:41:02.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:02 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4255827210' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:41:02.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:02 smithi022 ceph-mon[37330]: from='client.25594 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:41:02.925 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:02.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:02.926 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:02.945 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:41:03.026 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:03.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:02 smithi104 ceph-mon[33507]: osdmap e322: 8 total, 8 up, 8 in 2024-09-11T07:41:03.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:02 smithi104 ceph-mon[33507]: pgmap v670: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 1.2 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 825 op/s; 6946/57174 objects misplaced (12.149%); 2.8 MiB/s, 45 objects/s recovering 2024-09-11T07:41:03.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:02 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4255827210' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:41:03.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:02 smithi104 ceph-mon[33507]: from='client.25594 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:41:03.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:03.048 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:03.049 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:03.152 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:03.173 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:03.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:03.174 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:03.275 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:03.300 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:03.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:03.301 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:03.401 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:03.424 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:03.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:03.425 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:03.526 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:03.549 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:03.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:03.550 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:03.651 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:03.673 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:03.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:03.675 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:03.776 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:03.811 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:03.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:03.813 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:03.913 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:03.949 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:03.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:03.951 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:03.955 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:03 smithi022 ceph-mon[29462]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:41:03.955 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:03 smithi022 ceph-mon[29462]: from='client.25594 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:41:03.955 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:03 smithi022 ceph-mon[29462]: osdmap e323: 8 total, 8 up, 8 in 2024-09-11T07:41:03.956 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:03 smithi022 ceph-mon[37330]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:41:03.956 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:03 smithi022 ceph-mon[37330]: from='client.25594 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:41:03.956 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:03 smithi022 ceph-mon[37330]: osdmap e323: 8 total, 8 up, 8 in 2024-09-11T07:41:04.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:03 smithi104 ceph-mon[33507]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:41:04.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:03 smithi104 ceph-mon[33507]: from='client.25594 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:41:04.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:03 smithi104 ceph-mon[33507]: osdmap e323: 8 total, 8 up, 8 in 2024-09-11T07:41:04.052 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:04.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:04.084 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:04.084 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:04.185 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:04.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:04.223 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:04.224 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:04.324 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:04.365 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:04.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:04.366 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:04.468 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:04.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:04.520 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:04.520 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:04.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:41:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:41:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:41:04.621 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:04.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:04.650 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:04.651 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:04.751 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:04.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:04.796 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:04.796 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:04.898 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:04.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:04 smithi022 ceph-mon[29462]: osdmap e324: 8 total, 8 up, 8 in 2024-09-11T07:41:04.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:04 smithi022 ceph-mon[29462]: pgmap v673: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 1.1 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 0 B/s wr, 243 op/s; 6816/56406 objects misplaced (12.084%); 3.3 MiB/s, 52 objects/s recovering 2024-09-11T07:41:04.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:04 smithi022 ceph-mon[37330]: osdmap e324: 8 total, 8 up, 8 in 2024-09-11T07:41:04.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:04 smithi022 ceph-mon[37330]: pgmap v673: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 1.1 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 0 B/s wr, 243 op/s; 6816/56406 objects misplaced (12.084%); 3.3 MiB/s, 52 objects/s recovering 2024-09-11T07:41:04.927 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:04.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:04.928 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:05.029 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:05.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:04 smithi104 ceph-mon[33507]: osdmap e324: 8 total, 8 up, 8 in 2024-09-11T07:41:05.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:04 smithi104 ceph-mon[33507]: pgmap v673: 158 pgs: 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 151 active+clean; 1.1 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 0 B/s wr, 243 op/s; 6816/56406 objects misplaced (12.084%); 3.3 MiB/s, 52 objects/s recovering 2024-09-11T07:41:05.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:05.071 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:05.072 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:05.172 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:05.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:05.230 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:05.230 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:05.331 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:05.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:05.381 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:05.381 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:05.482 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:05.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:05.525 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:05.525 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:05.626 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:05.668 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:05.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:05.669 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:05.769 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:05.802 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:05.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:05.803 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:05.871 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:05 smithi022 ceph-mon[29462]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:41:05.871 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:05 smithi022 ceph-mon[37330]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:41:05.903 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:05.937 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:05.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:05.938 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:05.950 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:41:05.951 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:41:05.984 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:41:05.984 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:41:06.017 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:41:06.017 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:41:06.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:05 smithi104 ceph-mon[33507]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:41:06.039 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:06.055 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:41:06.055 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:41:06.079 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:06.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:06.080 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:06.084 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:41:06.084 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:41:06.117 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:41:06.117 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:41:06.147 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:41:06.147 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:41:06.177 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:41:06.177 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:41:06.182 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:06.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:06.215 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:06.215 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:06.316 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:06.353 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:06.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:06.354 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:06.427 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T07:41:06.455 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:06.482 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:06.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:06.483 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:06.583 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:06.616 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:06.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:06.617 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:06.718 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:06.747 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:06.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:06.748 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:06.849 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:06.902 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:06.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:06.903 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:06.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:06 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1807381033' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:41:06.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:06 smithi022 ceph-mon[29462]: pgmap v674: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 1013 MiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 914 B/s rd, 0 B/s wr, 476 op/s; 3551/49188 objects misplaced (7.219%); 17 MiB/s, 264 objects/s recovering 2024-09-11T07:41:06.988 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:06 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1807381033' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:41:06.988 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:06 smithi022 ceph-mon[37330]: pgmap v674: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 1013 MiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 914 B/s rd, 0 B/s wr, 476 op/s; 3551/49188 objects misplaced (7.219%); 17 MiB/s, 264 objects/s recovering 2024-09-11T07:41:07.004 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:07.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:06 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1807381033' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:41:07.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:06 smithi104 ceph-mon[33507]: pgmap v674: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 1013 MiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 914 B/s rd, 0 B/s wr, 476 op/s; 3551/49188 objects misplaced (7.219%); 17 MiB/s, 264 objects/s recovering 2024-09-11T07:41:07.032 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:07.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:07.033 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:07.134 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:07.176 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:07.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:07.177 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:07.257 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:41:07.257 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:41:07.277 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:07.311 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:07.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:07.312 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:07.413 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:07.453 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:07.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:07.454 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:07.555 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:07.594 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:07.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:07.595 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:07.696 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:07.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:07.728 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:07.728 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:07.829 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:07.860 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:07.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:07.861 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:07.890 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-11T07:41:07.961 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:07.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:07.997 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:07.997 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:08.099 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:08.127 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:08.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:08.128 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:08.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:41:08.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:41:08.228 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:08.254 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:08.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:08.255 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:08.356 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:08.386 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:08.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:08.387 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:08.488 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:08.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:41:08.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:08.542 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:08.542 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:08.643 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:08.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:08.674 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:08.674 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:08.775 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:08.806 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:08.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:08.807 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:08.832 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:41:08.836 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:41:08.908 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:08.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:08.957 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:08.957 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:09.058 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:09.097 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:09.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:09.098 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:09.199 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:09.240 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:09.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:09.241 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:09.342 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:09.374 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:09.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:09.375 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:09.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:09 smithi022 ceph-mon[29462]: pgmap v675: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 1013 MiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 777 B/s rd, 0 B/s wr, 404 op/s; 3551/49188 objects misplaced (7.219%); 14 MiB/s, 224 objects/s recovering 2024-09-11T07:41:09.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:09 smithi022 ceph-mon[37330]: pgmap v675: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 1013 MiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 777 B/s rd, 0 B/s wr, 404 op/s; 3551/49188 objects misplaced (7.219%); 14 MiB/s, 224 objects/s recovering 2024-09-11T07:41:09.475 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:09.519 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:09.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:09.521 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:09.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:09 smithi104 ceph-mon[33507]: pgmap v675: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 1013 MiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 777 B/s rd, 0 B/s wr, 404 op/s; 3551/49188 objects misplaced (7.219%); 14 MiB/s, 224 objects/s recovering 2024-09-11T07:41:09.621 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:09.658 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:09.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:09.659 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:09.728 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":675,"stamp":"2024-09-11T07:41:08.261295+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1062213876,"num_objects":16396,"num_object_clones":0,"num_object_copies":49188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3551,"num_objects_unfound":0,"num_objects_dirty":16396,"num_whiteouts":0,"num_read":2139,"num_read_kb":1996,"num_write":237807,"num_write_kb":8103304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45705,"num_bytes_recovered":2987332409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131322,"ondisk_log_size":131322,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":509,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":8766132,"kb_used_data":4537344,"kb_used_omap":16,"kb_used_meta":4228591,"kb_avail":741195084,"statfs":{"total":767960285184,"available":758983766016,"internally_reserved":0,"allocated":4646240256,"data_stored":4642598158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16530,"internal_metadata":4330078062},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,8,3,1,0,0,0,0,16],"upper_bound":2048},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-174456832,"num_objects":-2662,"num_object_clones":0,"num_object_copies":-7986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-3395,"num_objects_unfound":0,"num_objects_dirty":-2662,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2662,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1481,"num_bytes_recovered":96993280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.588336"},"pg_stats":[{"pgid":"9.14","version":"324'10071","reported_seq":10192,"reported_epoch":324,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-11T07:41:04.114730+0000","last_change":"2024-09-11T07:40:42.790526+0000","last_active":"2024-09-11T07:41:04.114730+0000","last_peered":"2024-09-11T07:41:04.114730+0000","last_clean":"2024-09-11T07:39:56.654544+0000","last_became_active":"2024-09-11T07:40:12.755113+0000","last_became_peered":"2024-09-11T07:40:12.755113+0000","last_unstale":"2024-09-11T07:41:04.114730+0000","last_undegraded":"2024-09-11T07:41:04.114730+0000","last_fullsized":"2024-09-11T07:41:04.114730+0000","mapping_epoch":297,"log_start":"266'5632","ondisk_log_start":"266'5632","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4439,"log_dups_size":0,"ondisk_log_size":4439,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37486592,"num_objects":572,"num_object_clones":0,"num_object_copies":1716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1144,"num_objects_unfound":0,"num_objects_dirty":572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7384,"num_write_kb":254592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,2],"acting":[4,1,3],"avail_no_missing":["4","1","3"],"object_location_counts":[{"shards":"1,3,4","objects":572}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":142,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.680188+0000","last_change":"2024-09-11T07:40:39.605681+0000","last_active":"2024-09-11T07:41:02.680188+0000","last_peered":"2024-09-11T07:41:02.680188+0000","last_clean":"2024-09-11T07:41:02.680188+0000","last_became_active":"2024-09-11T07:40:39.605361+0000","last_became_peered":"2024-09-11T07:40:39.605361+0000","last_unstale":"2024-09-11T07:41:02.680188+0000","last_undegraded":"2024-09-11T07:41:02.680188+0000","last_fullsized":"2024-09-11T07:41:02.680188+0000","mapping_epoch":310,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":311,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:29.007411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:29.007411+0000","last_clean_scrub_stamp":"2024-09-11T07:37:29.007411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001472253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":352,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.683973+0000","last_change":"2024-09-11T07:38:27.418908+0000","last_active":"2024-09-11T07:41:02.683973+0000","last_peered":"2024-09-11T07:41:02.683973+0000","last_clean":"2024-09-11T07:41:02.683973+0000","last_became_active":"2024-09-11T07:32:00.508572+0000","last_became_peered":"2024-09-11T07:32:00.508572+0000","last_unstale":"2024-09-11T07:41:02.683973+0000","last_undegraded":"2024-09-11T07:41:02.683973+0000","last_fullsized":"2024-09-11T07:41:02.683973+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:27.418829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:38:27.418829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036972480000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"234'34","reported_seq":376,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.687205+0000","last_change":"2024-09-11T07:40:39.607640+0000","last_active":"2024-09-11T07:41:02.687205+0000","last_peered":"2024-09-11T07:41:02.687205+0000","last_clean":"2024-09-11T07:41:02.687205+0000","last_became_active":"2024-09-11T07:26:33.544627+0000","last_became_peered":"2024-09-11T07:26:33.544627+0000","last_unstale":"2024-09-11T07:41:02.687205+0000","last_undegraded":"2024-09-11T07:41:02.687205+0000","last_fullsized":"2024-09-11T07:41:02.687205+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'34","last_scrub_stamp":"2024-09-11T07:40:39.607579+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:40:39.607579+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:51.954396+0000","scrub_duration":0.0054431660000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":341,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.718927+0000","last_change":"2024-09-11T07:39:56.545948+0000","last_active":"2024-09-11T07:41:02.718927+0000","last_peered":"2024-09-11T07:41:02.718927+0000","last_clean":"2024-09-11T07:41:02.718927+0000","last_became_active":"2024-09-11T07:26:35.556296+0000","last_became_peered":"2024-09-11T07:26:35.556296+0000","last_unstale":"2024-09-11T07:41:02.718927+0000","last_undegraded":"2024-09-11T07:41:02.718927+0000","last_fullsized":"2024-09-11T07:41:02.718927+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:56.545872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:39:56.545872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:24.540468+0000","scrub_duration":0.002240283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.15","version":"324'10161","reported_seq":22646,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T07:41:04.415368+0000","last_change":"2024-09-11T07:40:10.796286+0000","last_active":"2024-09-11T07:41:04.415368+0000","last_peered":"2024-09-11T07:41:04.415368+0000","last_clean":"2024-09-11T07:41:04.415368+0000","last_became_active":"2024-09-11T07:40:10.762538+0000","last_became_peered":"2024-09-11T07:40:10.762538+0000","last_unstale":"2024-09-11T07:41:04.415368+0000","last_undegraded":"2024-09-11T07:41:04.415368+0000","last_fullsized":"2024-09-11T07:41:04.415368+0000","mapping_epoch":295,"log_start":"256'3961","ondisk_log_start":"256'3961","created":259,"last_epoch_clean":296,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6200,"log_dups_size":0,"ondisk_log_size":6200,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34013184,"num_objects":519,"num_object_clones":0,"num_object_copies":1557,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7447,"num_write_kb":254912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4795,"num_bytes_recovered":314179584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":348,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.688782+0000","last_change":"2024-09-11T07:40:35.645201+0000","last_active":"2024-09-11T07:41:02.688782+0000","last_peered":"2024-09-11T07:41:02.688782+0000","last_clean":"2024-09-11T07:41:02.688782+0000","last_became_active":"2024-09-11T07:39:57.767806+0000","last_became_peered":"2024-09-11T07:39:57.767806+0000","last_unstale":"2024-09-11T07:41:02.688782+0000","last_undegraded":"2024-09-11T07:41:02.688782+0000","last_fullsized":"2024-09-11T07:41:02.688782+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:35.645119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:40:35.645119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:00.870748+0000","scrub_duration":0.0071309060000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"234'19","reported_seq":358,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.719001+0000","last_change":"2024-09-11T07:39:32.589939+0000","last_active":"2024-09-11T07:41:02.719001+0000","last_peered":"2024-09-11T07:41:02.719001+0000","last_clean":"2024-09-11T07:41:02.719001+0000","last_became_active":"2024-09-11T07:26:33.540732+0000","last_became_peered":"2024-09-11T07:26:33.540732+0000","last_unstale":"2024-09-11T07:41:02.719001+0000","last_undegraded":"2024-09-11T07:41:02.719001+0000","last_fullsized":"2024-09-11T07:41:02.719001+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:36:42.708941+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:36:42.708941+0000","last_clean_scrub_stamp":"2024-09-11T07:36:42.708941+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0088444879999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":340,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.680368+0000","last_change":"2024-09-11T07:37:43.087423+0000","last_active":"2024-09-11T07:41:02.680368+0000","last_peered":"2024-09-11T07:41:02.680368+0000","last_clean":"2024-09-11T07:41:02.680368+0000","last_became_active":"2024-09-11T07:32:00.513650+0000","last_became_peered":"2024-09-11T07:32:00.513650+0000","last_unstale":"2024-09-11T07:41:02.680368+0000","last_undegraded":"2024-09-11T07:41:02.680368+0000","last_fullsized":"2024-09-11T07:41:02.680368+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:43.087302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:37:43.087302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00405653,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":352,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.718328+0000","last_change":"2024-09-11T07:40:36.708489+0000","last_active":"2024-09-11T07:41:02.718328+0000","last_peered":"2024-09-11T07:41:02.718328+0000","last_clean":"2024-09-11T07:41:02.718328+0000","last_became_active":"2024-09-11T07:40:11.698235+0000","last_became_peered":"2024-09-11T07:40:11.698235+0000","last_unstale":"2024-09-11T07:41:02.718328+0000","last_undegraded":"2024-09-11T07:41:02.718328+0000","last_fullsized":"2024-09-11T07:41:02.718328+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:36.708406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:40:36.708406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:50.355594+0000","scrub_duration":0.0020771840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"324'10267","reported_seq":19325,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T07:41:04.130176+0000","last_change":"2024-09-11T07:40:36.515700+0000","last_active":"2024-09-11T07:41:04.130176+0000","last_peered":"2024-09-11T07:41:04.130176+0000","last_clean":"2024-09-11T07:41:04.130176+0000","last_became_active":"2024-09-11T07:40:36.499497+0000","last_became_peered":"2024-09-11T07:40:36.499497+0000","last_unstale":"2024-09-11T07:41:04.130176+0000","last_undegraded":"2024-09-11T07:41:04.130176+0000","last_fullsized":"2024-09-11T07:41:04.130176+0000","mapping_epoch":307,"log_start":"266'5865","ondisk_log_start":"266'5865","created":259,"last_epoch_clean":308,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4402,"log_dups_size":0,"ondisk_log_size":4402,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36110336,"num_objects":551,"num_object_clones":0,"num_object_copies":1653,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7503,"num_write_kb":257728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4458,"num_bytes_recovered":292159488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":330,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.679008+0000","last_change":"2024-09-11T07:37:34.542508+0000","last_active":"2024-09-11T07:41:02.679008+0000","last_peered":"2024-09-11T07:41:02.679008+0000","last_clean":"2024-09-11T07:41:02.679008+0000","last_became_active":"2024-09-11T07:32:00.516662+0000","last_became_peered":"2024-09-11T07:32:00.516662+0000","last_unstale":"2024-09-11T07:41:02.679008+0000","last_undegraded":"2024-09-11T07:41:02.679008+0000","last_fullsized":"2024-09-11T07:41:02.679008+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:34.542441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:34.542441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015726010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"234'9","reported_seq":364,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.718579+0000","last_change":"2024-09-11T07:39:57.773949+0000","last_active":"2024-09-11T07:41:02.718579+0000","last_peered":"2024-09-11T07:41:02.718579+0000","last_clean":"2024-09-11T07:41:02.718579+0000","last_became_active":"2024-09-11T07:39:57.765606+0000","last_became_peered":"2024-09-11T07:39:57.765606+0000","last_unstale":"2024-09-11T07:41:02.718579+0000","last_undegraded":"2024-09-11T07:41:02.718579+0000","last_fullsized":"2024-09-11T07:41:02.718579+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'9","last_scrub_stamp":"2024-09-11T07:37:37.558036+0000","last_deep_scrub":"82'7","last_deep_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_clean_scrub_stamp":"2024-09-11T07:37:37.558036+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034408770000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":299,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.678173+0000","last_change":"2024-09-11T07:39:32.482789+0000","last_active":"2024-09-11T07:41:02.678173+0000","last_peered":"2024-09-11T07:41:02.678173+0000","last_clean":"2024-09-11T07:41:02.678173+0000","last_became_active":"2024-09-11T07:32:00.513206+0000","last_became_peered":"2024-09-11T07:32:00.513206+0000","last_unstale":"2024-09-11T07:41:02.678173+0000","last_undegraded":"2024-09-11T07:41:02.678173+0000","last_fullsized":"2024-09-11T07:41:02.678173+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:50.332244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.973323+0000","last_clean_scrub_stamp":"2024-09-11T07:35:50.332244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034655860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":58,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.679519+0000","last_change":"2024-09-11T07:40:11.703754+0000","last_active":"2024-09-11T07:41:02.679519+0000","last_peered":"2024-09-11T07:41:02.679519+0000","last_clean":"2024-09-11T07:41:02.679519+0000","last_became_active":"2024-09-11T07:40:11.701560+0000","last_became_peered":"2024-09-11T07:40:11.701560+0000","last_unstale":"2024-09-11T07:41:02.679519+0000","last_undegraded":"2024-09-11T07:41:02.679519+0000","last_fullsized":"2024-09-11T07:41:02.679519+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:36.462794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:45.149916+0000","last_clean_scrub_stamp":"2024-09-11T07:37:36.462794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"324'10200","reported_seq":19191,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T07:41:05.241830+0000","last_change":"2024-09-11T07:40:26.422385+0000","last_active":"2024-09-11T07:41:05.241830+0000","last_peered":"2024-09-11T07:41:05.241830+0000","last_clean":"2024-09-11T07:41:05.241830+0000","last_became_active":"2024-09-11T07:40:26.404009+0000","last_became_peered":"2024-09-11T07:40:26.404009+0000","last_unstale":"2024-09-11T07:41:05.241830+0000","last_undegraded":"2024-09-11T07:41:05.241830+0000","last_fullsized":"2024-09-11T07:41:05.241830+0000","mapping_epoch":301,"log_start":"265'5545","ondisk_log_start":"265'5545","created":259,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4655,"log_dups_size":0,"ondisk_log_size":4655,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30670848,"num_objects":468,"num_object_clones":0,"num_object_copies":1404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7496,"num_write_kb":254848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3243,"num_bytes_recovered":212533248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":295,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.687563+0000","last_change":"2024-09-11T07:37:40.336139+0000","last_active":"2024-09-11T07:41:02.687563+0000","last_peered":"2024-09-11T07:41:02.687563+0000","last_clean":"2024-09-11T07:41:02.687563+0000","last_became_active":"2024-09-11T07:32:00.505709+0000","last_became_peered":"2024-09-11T07:32:00.505709+0000","last_unstale":"2024-09-11T07:41:02.687563+0000","last_undegraded":"2024-09-11T07:41:02.687563+0000","last_fullsized":"2024-09-11T07:41:02.687563+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:40.336050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_clean_scrub_stamp":"2024-09-11T07:37:40.336050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021490649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"234'19","reported_seq":355,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.718649+0000","last_change":"2024-09-11T07:39:39.571267+0000","last_active":"2024-09-11T07:41:02.718649+0000","last_peered":"2024-09-11T07:41:02.718649+0000","last_clean":"2024-09-11T07:41:02.718649+0000","last_became_active":"2024-09-11T07:26:33.544386+0000","last_became_peered":"2024-09-11T07:26:33.544386+0000","last_unstale":"2024-09-11T07:41:02.718649+0000","last_undegraded":"2024-09-11T07:41:02.718649+0000","last_fullsized":"2024-09-11T07:41:02.718649+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'13","last_scrub_stamp":"2024-09-11T07:36:27.746295+0000","last_deep_scrub":"136'13","last_deep_scrub_stamp":"2024-09-11T07:31:08.042198+0000","last_clean_scrub_stamp":"2024-09-11T07:36:27.746295+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0089695690000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":347,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.683795+0000","last_change":"2024-09-11T07:40:11.700127+0000","last_active":"2024-09-11T07:41:02.683795+0000","last_peered":"2024-09-11T07:41:02.683795+0000","last_clean":"2024-09-11T07:41:02.683795+0000","last_became_active":"2024-09-11T07:40:11.699772+0000","last_became_peered":"2024-09-11T07:40:11.699772+0000","last_unstale":"2024-09-11T07:41:02.683795+0000","last_undegraded":"2024-09-11T07:41:02.683795+0000","last_fullsized":"2024-09-11T07:41:02.683795+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:27.145883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:38:27.145883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011677339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":346,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.682847+0000","last_change":"2024-09-11T07:39:51.811562+0000","last_active":"2024-09-11T07:41:02.682847+0000","last_peered":"2024-09-11T07:41:02.682847+0000","last_clean":"2024-09-11T07:41:02.682847+0000","last_became_active":"2024-09-11T07:32:00.530335+0000","last_became_peered":"2024-09-11T07:32:00.530335+0000","last_unstale":"2024-09-11T07:41:02.682847+0000","last_undegraded":"2024-09-11T07:41:02.682847+0000","last_fullsized":"2024-09-11T07:41:02.682847+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:51.811287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:39:51.811287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:21.373975+0000","scrub_duration":0.0069635549999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"324'10103","reported_seq":11385,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T07:41:04.512731+0000","last_change":"2024-09-11T07:41:02.711634+0000","last_active":"2024-09-11T07:41:04.512731+0000","last_peered":"2024-09-11T07:41:04.512731+0000","last_clean":"2024-09-11T07:41:04.512731+0000","last_became_active":"2024-09-11T07:41:02.685660+0000","last_became_peered":"2024-09-11T07:41:02.685660+0000","last_unstale":"2024-09-11T07:41:04.512731+0000","last_undegraded":"2024-09-11T07:41:04.512731+0000","last_fullsized":"2024-09-11T07:41:04.512731+0000","mapping_epoch":322,"log_start":"256'4404","ondisk_log_start":"256'4404","created":247,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":5699,"log_dups_size":0,"ondisk_log_size":5699,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36831232,"num_objects":562,"num_object_clones":0,"num_object_copies":1686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7384,"num_write_kb":254272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":577,"num_bytes_recovered":37814272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"234'31","reported_seq":431,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.719870+0000","last_change":"2024-09-11T07:39:41.579223+0000","last_active":"2024-09-11T07:41:02.719870+0000","last_peered":"2024-09-11T07:41:02.719870+0000","last_clean":"2024-09-11T07:41:02.719870+0000","last_became_active":"2024-09-11T07:29:51.482755+0000","last_became_peered":"2024-09-11T07:29:51.482755+0000","last_unstale":"2024-09-11T07:41:02.719870+0000","last_undegraded":"2024-09-11T07:41:02.719870+0000","last_fullsized":"2024-09-11T07:41:02.719870+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'21","last_scrub_stamp":"2024-09-11T07:36:39.736944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:36:39.736944+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034170279999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":328,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.678485+0000","last_change":"2024-09-11T07:39:44.385573+0000","last_active":"2024-09-11T07:41:02.678485+0000","last_peered":"2024-09-11T07:41:02.678485+0000","last_clean":"2024-09-11T07:41:02.678485+0000","last_became_active":"2024-09-11T07:32:00.507191+0000","last_became_peered":"2024-09-11T07:32:00.507191+0000","last_unstale":"2024-09-11T07:41:02.678485+0000","last_undegraded":"2024-09-11T07:41:02.678485+0000","last_fullsized":"2024-09-11T07:41:02.678485+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:51.418258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:51.418258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001500181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":323,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.680847+0000","last_change":"2024-09-11T07:38:22.027909+0000","last_active":"2024-09-11T07:41:02.680847+0000","last_peered":"2024-09-11T07:41:02.680847+0000","last_clean":"2024-09-11T07:41:02.680847+0000","last_became_active":"2024-09-11T07:26:35.557989+0000","last_became_peered":"2024-09-11T07:26:35.557989+0000","last_unstale":"2024-09-11T07:41:02.680847+0000","last_undegraded":"2024-09-11T07:41:02.680847+0000","last_fullsized":"2024-09-11T07:41:02.680847+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:22.027769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:38:22.027769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060071020000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":342,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.679444+0000","last_change":"2024-09-11T07:39:41.220688+0000","last_active":"2024-09-11T07:41:02.679444+0000","last_peered":"2024-09-11T07:41:02.679444+0000","last_clean":"2024-09-11T07:41:02.679444+0000","last_became_active":"2024-09-11T07:26:37.564554+0000","last_became_peered":"2024-09-11T07:26:37.564554+0000","last_unstale":"2024-09-11T07:41:02.679444+0000","last_undegraded":"2024-09-11T07:41:02.679444+0000","last_fullsized":"2024-09-11T07:41:02.679444+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:41.220589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:39:41.220589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066621019999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"324'9857","reported_seq":13813,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T07:41:04.969795+0000","last_change":"2024-09-11T07:40:10.728235+0000","last_active":"2024-09-11T07:41:04.969795+0000","last_peered":"2024-09-11T07:41:04.969795+0000","last_clean":"2024-09-11T07:41:04.969795+0000","last_became_active":"2024-09-11T07:40:10.693596+0000","last_became_peered":"2024-09-11T07:40:10.693596+0000","last_unstale":"2024-09-11T07:41:04.969795+0000","last_undegraded":"2024-09-11T07:41:04.969795+0000","last_fullsized":"2024-09-11T07:41:04.969795+0000","mapping_epoch":295,"log_start":"260'5363","ondisk_log_start":"260'5363","created":247,"last_epoch_clean":296,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4494,"log_dups_size":0,"ondisk_log_size":4494,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38404096,"num_objects":586,"num_object_clones":0,"num_object_copies":1758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7180,"num_write_kb":248512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1918,"num_bytes_recovered":125698048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"234'25","reported_seq":371,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.718836+0000","last_change":"2024-09-11T07:39:33.667725+0000","last_active":"2024-09-11T07:41:02.718836+0000","last_peered":"2024-09-11T07:41:02.718836+0000","last_clean":"2024-09-11T07:41:02.718836+0000","last_became_active":"2024-09-11T07:26:33.535411+0000","last_became_peered":"2024-09-11T07:26:33.535411+0000","last_unstale":"2024-09-11T07:41:02.718836+0000","last_undegraded":"2024-09-11T07:41:02.718836+0000","last_fullsized":"2024-09-11T07:41:02.718836+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'25","last_scrub_stamp":"2024-09-11T07:39:33.667530+0000","last_deep_scrub":"112'15","last_deep_scrub_stamp":"2024-09-11T07:31:10.005896+0000","last_clean_scrub_stamp":"2024-09-11T07:39:33.667530+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.041705696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":343,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.689314+0000","last_change":"2024-09-11T07:37:36.098229+0000","last_active":"2024-09-11T07:41:02.689314+0000","last_peered":"2024-09-11T07:41:02.689314+0000","last_clean":"2024-09-11T07:41:02.689314+0000","last_became_active":"2024-09-11T07:32:00.506615+0000","last_became_peered":"2024-09-11T07:32:00.506615+0000","last_unstale":"2024-09-11T07:41:02.689314+0000","last_undegraded":"2024-09-11T07:41:02.689314+0000","last_fullsized":"2024-09-11T07:41:02.689314+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:36.098175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:36.098175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017555839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"299'11","reported_seq":549,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.678457+0000","last_change":"2024-09-11T07:40:11.703968+0000","last_active":"2024-09-11T07:41:02.678457+0000","last_peered":"2024-09-11T07:41:02.678457+0000","last_clean":"2024-09-11T07:41:02.678457+0000","last_became_active":"2024-09-11T07:40:11.697568+0000","last_became_peered":"2024-09-11T07:40:11.697568+0000","last_unstale":"2024-09-11T07:41:02.678457+0000","last_undegraded":"2024-09-11T07:41:02.678457+0000","last_fullsized":"2024-09-11T07:41:02.678457+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:37:43.140999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:43.140999+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039651130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":339,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.719544+0000","last_change":"2024-09-11T07:39:57.698127+0000","last_active":"2024-09-11T07:41:02.719544+0000","last_peered":"2024-09-11T07:41:02.719544+0000","last_clean":"2024-09-11T07:41:02.719544+0000","last_became_active":"2024-09-11T07:39:57.697588+0000","last_became_peered":"2024-09-11T07:39:57.697588+0000","last_unstale":"2024-09-11T07:41:02.719544+0000","last_undegraded":"2024-09-11T07:41:02.719544+0000","last_fullsized":"2024-09-11T07:41:02.719544+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:52.605883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:39:52.605883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:21.117617+0000","scrub_duration":0.0013626420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"324'10003","reported_seq":12729,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T07:41:05.063958+0000","last_change":"2024-09-11T07:40:37.540503+0000","last_active":"2024-09-11T07:41:05.063958+0000","last_peered":"2024-09-11T07:41:05.063958+0000","last_clean":"2024-09-11T07:41:05.063958+0000","last_became_active":"2024-09-11T07:40:37.499654+0000","last_became_peered":"2024-09-11T07:40:37.499654+0000","last_unstale":"2024-09-11T07:41:05.063958+0000","last_undegraded":"2024-09-11T07:41:05.063958+0000","last_fullsized":"2024-09-11T07:41:05.063958+0000","mapping_epoch":308,"log_start":"260'5353","ondisk_log_start":"260'5353","created":247,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4650,"log_dups_size":0,"ondisk_log_size":4650,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31129600,"num_objects":475,"num_object_clones":0,"num_object_copies":1425,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7341,"num_write_kb":250112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1302,"num_bytes_recovered":85327872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"234'34","reported_seq":372,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.682129+0000","last_change":"2024-09-11T07:37:16.938565+0000","last_active":"2024-09-11T07:41:02.682129+0000","last_peered":"2024-09-11T07:41:02.682129+0000","last_clean":"2024-09-11T07:41:02.682129+0000","last_became_active":"2024-09-11T07:26:33.549733+0000","last_became_peered":"2024-09-11T07:26:33.549733+0000","last_unstale":"2024-09-11T07:41:02.682129+0000","last_undegraded":"2024-09-11T07:41:02.682129+0000","last_fullsized":"2024-09-11T07:41:02.682129+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'34","last_scrub_stamp":"2024-09-11T07:37:16.938496+0000","last_deep_scrub":"83'22","last_deep_scrub_stamp":"2024-09-11T07:34:04.375457+0000","last_clean_scrub_stamp":"2024-09-11T07:37:16.938496+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028468529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":354,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.679845+0000","last_change":"2024-09-11T07:38:29.939925+0000","last_active":"2024-09-11T07:41:02.679845+0000","last_peered":"2024-09-11T07:41:02.679845+0000","last_clean":"2024-09-11T07:41:02.679845+0000","last_became_active":"2024-09-11T07:32:00.513472+0000","last_became_peered":"2024-09-11T07:32:00.513472+0000","last_unstale":"2024-09-11T07:41:02.679845+0000","last_undegraded":"2024-09-11T07:41:02.679845+0000","last_fullsized":"2024-09-11T07:41:02.679845+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:29.939826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:38:29.939826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017750210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":516,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.719621+0000","last_change":"2024-09-11T07:40:46.534047+0000","last_active":"2024-09-11T07:41:02.719621+0000","last_peered":"2024-09-11T07:41:02.719621+0000","last_clean":"2024-09-11T07:41:02.719621+0000","last_became_active":"2024-09-11T07:26:35.560406+0000","last_became_peered":"2024-09-11T07:26:35.560406+0000","last_unstale":"2024-09-11T07:41:02.719621+0000","last_undegraded":"2024-09-11T07:41:02.719621+0000","last_fullsized":"2024-09-11T07:41:02.719621+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:40:46.533966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:40:46.533966+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:51.243657+0000","scrub_duration":0.0050834449999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":329,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.688465+0000","last_change":"2024-09-11T07:39:57.762607+0000","last_active":"2024-09-11T07:41:02.688465+0000","last_peered":"2024-09-11T07:41:02.688465+0000","last_clean":"2024-09-11T07:41:02.688465+0000","last_became_active":"2024-09-11T07:39:57.762309+0000","last_became_peered":"2024-09-11T07:39:57.762309+0000","last_unstale":"2024-09-11T07:41:02.688465+0000","last_undegraded":"2024-09-11T07:41:02.688465+0000","last_fullsized":"2024-09-11T07:41:02.688465+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:20.369199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:31.200685+0000","last_clean_scrub_stamp":"2024-09-11T07:37:20.369199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001371688,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"324'9872","reported_seq":9970,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T07:41:04.397920+0000","last_change":"2024-09-11T07:40:45.528413+0000","last_active":"2024-09-11T07:41:04.397920+0000","last_peered":"2024-09-11T07:41:04.397920+0000","last_clean":"2024-09-11T07:41:04.397920+0000","last_became_active":"2024-09-11T07:38:27.361246+0000","last_became_peered":"2024-09-11T07:38:27.361246+0000","last_unstale":"2024-09-11T07:41:04.397920+0000","last_undegraded":"2024-09-11T07:41:04.397920+0000","last_fullsized":"2024-09-11T07:41:04.397920+0000","mapping_epoch":247,"log_start":"256'3499","ondisk_log_start":"256'3499","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6373,"log_dups_size":0,"ondisk_log_size":6373,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33652736,"num_objects":513,"num_object_clones":0,"num_object_copies":1539,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7219,"num_write_kb":247456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"234'24","reported_seq":401,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.719644+0000","last_change":"2024-09-11T07:40:38.740191+0000","last_active":"2024-09-11T07:41:02.719644+0000","last_peered":"2024-09-11T07:41:02.719644+0000","last_clean":"2024-09-11T07:41:02.719644+0000","last_became_active":"2024-09-11T07:39:57.766134+0000","last_became_peered":"2024-09-11T07:39:57.766134+0000","last_unstale":"2024-09-11T07:41:02.719644+0000","last_undegraded":"2024-09-11T07:41:02.719644+0000","last_fullsized":"2024-09-11T07:41:02.719644+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'24","last_scrub_stamp":"2024-09-11T07:40:38.740127+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-09-11T07:29:38.112111+0000","last_clean_scrub_stamp":"2024-09-11T07:40:38.740127+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:52.114070+0000","scrub_duration":0.0044589670000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":322,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.677985+0000","last_change":"2024-09-11T07:39:36.451885+0000","last_active":"2024-09-11T07:41:02.677985+0000","last_peered":"2024-09-11T07:41:02.677985+0000","last_clean":"2024-09-11T07:41:02.677985+0000","last_became_active":"2024-09-11T07:32:00.518538+0000","last_became_peered":"2024-09-11T07:32:00.518538+0000","last_unstale":"2024-09-11T07:41:02.677985+0000","last_undegraded":"2024-09-11T07:41:02.677985+0000","last_fullsized":"2024-09-11T07:41:02.677985+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:37.444760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:37.444760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012636722,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":331,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.687929+0000","last_change":"2024-09-11T07:38:24.129487+0000","last_active":"2024-09-11T07:41:02.687929+0000","last_peered":"2024-09-11T07:41:02.687929+0000","last_clean":"2024-09-11T07:41:02.687929+0000","last_became_active":"2024-09-11T07:26:35.569081+0000","last_became_peered":"2024-09-11T07:26:35.569081+0000","last_unstale":"2024-09-11T07:41:02.687929+0000","last_undegraded":"2024-09-11T07:41:02.687929+0000","last_fullsized":"2024-09-11T07:41:02.687929+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:24.129047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:38:24.129047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003571196,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":319,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.686508+0000","last_change":"2024-09-11T07:36:50.920711+0000","last_active":"2024-09-11T07:41:02.686508+0000","last_peered":"2024-09-11T07:41:02.686508+0000","last_clean":"2024-09-11T07:41:02.686508+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:41:02.686508+0000","last_undegraded":"2024-09-11T07:41:02.686508+0000","last_fullsized":"2024-09-11T07:41:02.686508+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:50.920625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:46.853387+0000","last_clean_scrub_stamp":"2024-09-11T07:36:50.920625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012404324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"324'14689","reported_seq":14773,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T07:41:05.040410+0000","last_change":"2024-09-11T07:37:25.855661+0000","last_active":"2024-09-11T07:41:05.040410+0000","last_peered":"2024-09-11T07:41:05.040410+0000","last_clean":"2024-09-11T07:41:05.040410+0000","last_became_active":"2024-09-11T07:37:25.854796+0000","last_became_peered":"2024-09-11T07:37:25.854796+0000","last_unstale":"2024-09-11T07:41:05.040410+0000","last_undegraded":"2024-09-11T07:41:05.040410+0000","last_fullsized":"2024-09-11T07:41:05.040410+0000","mapping_epoch":247,"log_start":"282'10206","ondisk_log_start":"282'10206","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4483,"log_dups_size":0,"ondisk_log_size":4483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64028672,"num_objects":977,"num_object_clones":0,"num_object_copies":2931,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14689,"num_write_kb":501312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"234'28","reported_seq":375,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.719055+0000","last_change":"2024-09-11T07:40:40.689386+0000","last_active":"2024-09-11T07:41:02.719055+0000","last_peered":"2024-09-11T07:41:02.719055+0000","last_clean":"2024-09-11T07:41:02.719055+0000","last_became_active":"2024-09-11T07:26:33.537850+0000","last_became_peered":"2024-09-11T07:26:33.537850+0000","last_unstale":"2024-09-11T07:41:02.719055+0000","last_undegraded":"2024-09-11T07:41:02.719055+0000","last_fullsized":"2024-09-11T07:41:02.719055+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'28","last_scrub_stamp":"2024-09-11T07:40:40.689301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:40:40.689301+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:48.883615+0000","scrub_duration":0.0071558330000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":101,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.679595+0000","last_change":"2024-09-11T07:39:57.768072+0000","last_active":"2024-09-11T07:41:02.679595+0000","last_peered":"2024-09-11T07:41:02.679595+0000","last_clean":"2024-09-11T07:41:02.679595+0000","last_became_active":"2024-09-11T07:39:57.767741+0000","last_became_peered":"2024-09-11T07:39:57.767741+0000","last_unstale":"2024-09-11T07:41:02.679595+0000","last_undegraded":"2024-09-11T07:41:02.679595+0000","last_fullsized":"2024-09-11T07:41:02.679595+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:49.609120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:39:49.609120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032789239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3],"acting":[2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"84'9","reported_seq":505,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T07:41:03.688127+0000","last_change":"2024-09-11T07:37:37.000676+0000","last_active":"2024-09-11T07:41:03.688127+0000","last_peered":"2024-09-11T07:41:03.688127+0000","last_clean":"2024-09-11T07:41:03.688127+0000","last_became_active":"2024-09-11T07:26:35.562997+0000","last_became_peered":"2024-09-11T07:26:35.562997+0000","last_unstale":"2024-09-11T07:41:03.688127+0000","last_undegraded":"2024-09-11T07:41:03.688127+0000","last_fullsized":"2024-09-11T07:41:03.688127+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-11T07:37:37.000619+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-11T07:35:39.832547+0000","last_clean_scrub_stamp":"2024-09-11T07:37:37.000619+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021426219999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":303,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.678542+0000","last_change":"2024-09-11T07:39:35.486304+0000","last_active":"2024-09-11T07:41:02.678542+0000","last_peered":"2024-09-11T07:41:02.678542+0000","last_clean":"2024-09-11T07:41:02.678542+0000","last_became_active":"2024-09-11T07:32:00.511005+0000","last_became_peered":"2024-09-11T07:32:00.511005+0000","last_unstale":"2024-09-11T07:41:02.678542+0000","last_undegraded":"2024-09-11T07:41:02.678542+0000","last_fullsized":"2024-09-11T07:41:02.678542+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:35.486250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:39:35.486250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042980279999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"324'14543","reported_seq":25145,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T07:41:04.365904+0000","last_change":"2024-09-11T07:40:27.473852+0000","last_active":"2024-09-11T07:41:04.365904+0000","last_peered":"2024-09-11T07:41:04.365904+0000","last_clean":"2024-09-11T07:41:04.365904+0000","last_became_active":"2024-09-11T07:40:27.407183+0000","last_became_peered":"2024-09-11T07:40:27.407183+0000","last_unstale":"2024-09-11T07:41:04.365904+0000","last_undegraded":"2024-09-11T07:41:04.365904+0000","last_fullsized":"2024-09-11T07:41:04.365904+0000","mapping_epoch":302,"log_start":"278'9244","ondisk_log_start":"278'9244","created":247,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":5299,"log_dups_size":0,"ondisk_log_size":5299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65929240,"num_objects":1007,"num_object_clones":0,"num_object_copies":3021,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1007,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14543,"num_write_kb":497537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3497,"num_bytes_recovered":229113880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"234'38","reported_seq":371,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.682952+0000","last_change":"2024-09-11T07:39:57.792218+0000","last_active":"2024-09-11T07:41:02.682952+0000","last_peered":"2024-09-11T07:41:02.682952+0000","last_clean":"2024-09-11T07:41:02.682952+0000","last_became_active":"2024-09-11T07:39:57.775541+0000","last_became_peered":"2024-09-11T07:39:57.775541+0000","last_unstale":"2024-09-11T07:41:02.682952+0000","last_undegraded":"2024-09-11T07:41:02.682952+0000","last_fullsized":"2024-09-11T07:41:02.682952+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'38","last_scrub_stamp":"2024-09-11T07:38:24.103612+0000","last_deep_scrub":"87'26","last_deep_scrub_stamp":"2024-09-11T07:35:52.942347+0000","last_clean_scrub_stamp":"2024-09-11T07:38:24.103612+0000","objects_scrubbed":8,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010845960999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":51,"num_read_kb":39,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":40,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.719693+0000","last_change":"2024-09-11T07:40:11.698799+0000","last_active":"2024-09-11T07:41:02.719693+0000","last_peered":"2024-09-11T07:41:02.719693+0000","last_clean":"2024-09-11T07:41:02.719693+0000","last_became_active":"2024-09-11T07:40:11.698558+0000","last_became_peered":"2024-09-11T07:40:11.698558+0000","last_unstale":"2024-09-11T07:41:02.719693+0000","last_undegraded":"2024-09-11T07:41:02.719693+0000","last_fullsized":"2024-09-11T07:41:02.719693+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:33.081815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:33.081815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":351,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.718905+0000","last_change":"2024-09-11T07:39:57.767026+0000","last_active":"2024-09-11T07:41:02.718905+0000","last_peered":"2024-09-11T07:41:02.718905+0000","last_clean":"2024-09-11T07:41:02.718905+0000","last_became_active":"2024-09-11T07:39:57.766717+0000","last_became_peered":"2024-09-11T07:39:57.766717+0000","last_unstale":"2024-09-11T07:41:02.718905+0000","last_undegraded":"2024-09-11T07:41:02.718905+0000","last_fullsized":"2024-09-11T07:41:02.718905+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:36.585153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:27.968579+0000","last_clean_scrub_stamp":"2024-09-11T07:39:36.585153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0067939640000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":339,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.718576+0000","last_change":"2024-09-11T07:39:57.692130+0000","last_active":"2024-09-11T07:41:02.718576+0000","last_peered":"2024-09-11T07:41:02.718576+0000","last_clean":"2024-09-11T07:41:02.718576+0000","last_became_active":"2024-09-11T07:39:57.691831+0000","last_became_peered":"2024-09-11T07:39:57.691831+0000","last_unstale":"2024-09-11T07:41:02.718576+0000","last_undegraded":"2024-09-11T07:41:02.718576+0000","last_fullsized":"2024-09-11T07:41:02.718576+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:51.576754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:29.978798+0000","last_clean_scrub_stamp":"2024-09-11T07:39:51.576754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:03.752909+0000","scrub_duration":0.0036155279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"324'14642","reported_seq":21082,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T07:41:04.130260+0000","last_change":"2024-09-11T07:40:39.907214+0000","last_active":"2024-09-11T07:41:04.130260+0000","last_peered":"2024-09-11T07:41:04.130260+0000","last_clean":"2024-09-11T07:41:04.130260+0000","last_became_active":"2024-09-11T07:40:34.480392+0000","last_became_peered":"2024-09-11T07:40:34.480392+0000","last_unstale":"2024-09-11T07:41:04.130260+0000","last_undegraded":"2024-09-11T07:41:04.130260+0000","last_fullsized":"2024-09-11T07:41:04.130260+0000","mapping_epoch":305,"log_start":"282'10199","ondisk_log_start":"282'10199","created":247,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"310'13316","last_scrub_stamp":"2024-09-11T07:40:39.907152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:40:39.907152+0000","objects_scrubbed":2362,"log_size":4443,"log_dups_size":0,"ondisk_log_size":4443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:05.320495+0000","scrub_duration":0.211466654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67895296,"num_objects":1036,"num_object_clones":0,"num_object_copies":3108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1036,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14642,"num_write_kb":501696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3114,"num_bytes_recovered":204079104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"236'32","reported_seq":394,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.683696+0000","last_change":"2024-09-11T07:36:50.024003+0000","last_active":"2024-09-11T07:41:02.683696+0000","last_peered":"2024-09-11T07:41:02.683696+0000","last_clean":"2024-09-11T07:41:02.683696+0000","last_became_active":"2024-09-11T07:32:00.506270+0000","last_became_peered":"2024-09-11T07:32:00.506270+0000","last_unstale":"2024-09-11T07:41:02.683696+0000","last_undegraded":"2024-09-11T07:41:02.683696+0000","last_fullsized":"2024-09-11T07:41:02.683696+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'32","last_scrub_stamp":"2024-09-11T07:36:50.023909+0000","last_deep_scrub":"236'32","last_deep_scrub_stamp":"2024-09-11T07:36:50.023909+0000","last_clean_scrub_stamp":"2024-09-11T07:36:50.023909+0000","objects_scrubbed":8,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.030217668999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":303,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.678961+0000","last_change":"2024-09-11T07:39:57.281547+0000","last_active":"2024-09-11T07:41:02.678961+0000","last_peered":"2024-09-11T07:41:02.678961+0000","last_clean":"2024-09-11T07:41:02.678961+0000","last_became_active":"2024-09-11T07:32:00.510631+0000","last_became_peered":"2024-09-11T07:32:00.510631+0000","last_unstale":"2024-09-11T07:41:02.678961+0000","last_undegraded":"2024-09-11T07:41:02.678961+0000","last_fullsized":"2024-09-11T07:41:02.678961+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:57.281493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:49.144041+0000","last_clean_scrub_stamp":"2024-09-11T07:39:57.281493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001207484,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":132,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.683657+0000","last_change":"2024-09-11T07:40:45.799295+0000","last_active":"2024-09-11T07:41:02.683657+0000","last_peered":"2024-09-11T07:41:02.683657+0000","last_clean":"2024-09-11T07:41:02.683657+0000","last_became_active":"2024-09-11T07:36:05.225840+0000","last_became_peered":"2024-09-11T07:36:05.225840+0000","last_unstale":"2024-09-11T07:41:02.683657+0000","last_undegraded":"2024-09-11T07:41:02.683657+0000","last_fullsized":"2024-09-11T07:41:02.683657+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:45.799189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:40:45.799189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:54.908201+0000","scrub_duration":0.047262133999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":321,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.679605+0000","last_change":"2024-09-11T07:40:45.844765+0000","last_active":"2024-09-11T07:41:02.679605+0000","last_peered":"2024-09-11T07:41:02.679605+0000","last_clean":"2024-09-11T07:41:02.679605+0000","last_became_active":"2024-09-11T07:26:37.861279+0000","last_became_peered":"2024-09-11T07:26:37.861279+0000","last_unstale":"2024-09-11T07:41:02.679605+0000","last_undegraded":"2024-09-11T07:41:02.679605+0000","last_fullsized":"2024-09-11T07:41:02.679605+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:54.827454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:39.723598+0000","last_clean_scrub_stamp":"2024-09-11T07:36:54.827454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016642015,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.f","version":"324'14561","reported_seq":14644,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T07:41:04.335673+0000","last_change":"2024-09-11T07:37:25.856504+0000","last_active":"2024-09-11T07:41:04.335673+0000","last_peered":"2024-09-11T07:41:04.335673+0000","last_clean":"2024-09-11T07:41:04.335673+0000","last_became_active":"2024-09-11T07:37:25.856214+0000","last_became_peered":"2024-09-11T07:37:25.856214+0000","last_unstale":"2024-09-11T07:41:04.335673+0000","last_undegraded":"2024-09-11T07:41:04.335673+0000","last_fullsized":"2024-09-11T07:41:04.335673+0000","mapping_epoch":247,"log_start":"278'9241","ondisk_log_start":"278'9241","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":5320,"log_dups_size":0,"ondisk_log_size":5320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69926912,"num_objects":1067,"num_object_clones":0,"num_object_copies":3201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14561,"num_write_kb":500096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"234'20","reported_seq":338,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.718502+0000","last_change":"2024-09-11T07:39:58.520372+0000","last_active":"2024-09-11T07:41:02.718502+0000","last_peered":"2024-09-11T07:41:02.718502+0000","last_clean":"2024-09-11T07:41:02.718502+0000","last_became_active":"2024-09-11T07:39:57.692071+0000","last_became_peered":"2024-09-11T07:39:57.692071+0000","last_unstale":"2024-09-11T07:41:02.718502+0000","last_undegraded":"2024-09-11T07:41:02.718502+0000","last_fullsized":"2024-09-11T07:41:02.718502+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'20","last_scrub_stamp":"2024-09-11T07:39:58.520311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:39:58.520311+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:12.044084+0000","scrub_duration":0.0024543820000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2],"acting":[4,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":361,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.680999+0000","last_change":"2024-09-11T07:40:44.863008+0000","last_active":"2024-09-11T07:41:02.680999+0000","last_peered":"2024-09-11T07:41:02.680999+0000","last_clean":"2024-09-11T07:41:02.680999+0000","last_became_active":"2024-09-11T07:32:00.522456+0000","last_became_peered":"2024-09-11T07:32:00.522456+0000","last_unstale":"2024-09-11T07:41:02.680999+0000","last_undegraded":"2024-09-11T07:41:02.680999+0000","last_fullsized":"2024-09-11T07:41:02.680999+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:40:44.862646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:40:44.862646+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:02.689591+0000","scrub_duration":0.005065918,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"290'9","reported_seq":528,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T07:41:03.686959+0000","last_change":"2024-09-11T07:39:57.780023+0000","last_active":"2024-09-11T07:41:03.686959+0000","last_peered":"2024-09-11T07:41:03.686959+0000","last_clean":"2024-09-11T07:41:03.686959+0000","last_became_active":"2024-09-11T07:39:57.768155+0000","last_became_peered":"2024-09-11T07:39:57.768155+0000","last_unstale":"2024-09-11T07:41:03.686959+0000","last_undegraded":"2024-09-11T07:41:03.686959+0000","last_fullsized":"2024-09-11T07:41:03.686959+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:37:26.141447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.141447+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015408170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":329,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.682650+0000","last_change":"2024-09-11T07:36:40.027067+0000","last_active":"2024-09-11T07:41:02.682650+0000","last_peered":"2024-09-11T07:41:02.682650+0000","last_clean":"2024-09-11T07:41:02.682650+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:41:02.682650+0000","last_undegraded":"2024-09-11T07:41:02.682650+0000","last_fullsized":"2024-09-11T07:41:02.682650+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:40.026982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_clean_scrub_stamp":"2024-09-11T07:36:40.026982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090395279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.9","version":"324'10225","reported_seq":10321,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T07:41:05.327977+0000","last_change":"2024-09-11T07:38:27.369273+0000","last_active":"2024-09-11T07:41:05.327977+0000","last_peered":"2024-09-11T07:41:05.327977+0000","last_clean":"2024-09-11T07:41:05.327977+0000","last_became_active":"2024-09-11T07:38:27.364955+0000","last_became_peered":"2024-09-11T07:38:27.364955+0000","last_unstale":"2024-09-11T07:41:05.327977+0000","last_undegraded":"2024-09-11T07:41:05.327977+0000","last_fullsized":"2024-09-11T07:41:05.327977+0000","mapping_epoch":247,"log_start":"266'5826","ondisk_log_start":"266'5826","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4399,"log_dups_size":0,"ondisk_log_size":4399,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32309248,"num_objects":493,"num_object_clones":0,"num_object_copies":1479,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7517,"num_write_kb":256320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"290'42","reported_seq":255,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.678951+0000","last_change":"2024-09-11T07:40:33.449484+0000","last_active":"2024-09-11T07:41:02.678951+0000","last_peered":"2024-09-11T07:41:02.678951+0000","last_clean":"2024-09-11T07:41:02.678951+0000","last_became_active":"2024-09-11T07:39:57.768267+0000","last_became_peered":"2024-09-11T07:39:57.768267+0000","last_unstale":"2024-09-11T07:41:02.678951+0000","last_undegraded":"2024-09-11T07:41:02.678951+0000","last_fullsized":"2024-09-11T07:41:02.678951+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'41","last_scrub_stamp":"2024-09-11T07:36:41.343611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:36:41.343611+0000","objects_scrubbed":10,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0099986009999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":345,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.687727+0000","last_change":"2024-09-11T07:39:57.468860+0000","last_active":"2024-09-11T07:41:02.687727+0000","last_peered":"2024-09-11T07:41:02.687727+0000","last_clean":"2024-09-11T07:41:02.687727+0000","last_became_active":"2024-09-11T07:26:31.424618+0000","last_became_peered":"2024-09-11T07:26:31.424618+0000","last_unstale":"2024-09-11T07:41:02.687727+0000","last_undegraded":"2024-09-11T07:41:02.687727+0000","last_fullsized":"2024-09-11T07:41:02.687727+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:57.468791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:51.308492+0000","last_clean_scrub_stamp":"2024-09-11T07:39:57.468791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:21.538207+0000","scrub_duration":0.0013109230000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":331,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.688137+0000","last_change":"2024-09-11T07:37:33.144930+0000","last_active":"2024-09-11T07:41:02.688137+0000","last_peered":"2024-09-11T07:41:02.688137+0000","last_clean":"2024-09-11T07:41:02.688137+0000","last_became_active":"2024-09-11T07:26:35.560363+0000","last_became_peered":"2024-09-11T07:26:35.560363+0000","last_unstale":"2024-09-11T07:41:02.688137+0000","last_undegraded":"2024-09-11T07:41:02.688137+0000","last_fullsized":"2024-09-11T07:41:02.688137+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:33.144873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:33.144873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001754675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":336,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.688623+0000","last_change":"2024-09-11T07:37:39.053981+0000","last_active":"2024-09-11T07:41:02.688623+0000","last_peered":"2024-09-11T07:41:02.688623+0000","last_clean":"2024-09-11T07:41:02.688623+0000","last_became_active":"2024-09-11T07:32:00.505919+0000","last_became_peered":"2024-09-11T07:32:00.505919+0000","last_unstale":"2024-09-11T07:41:02.688623+0000","last_undegraded":"2024-09-11T07:41:02.688623+0000","last_fullsized":"2024-09-11T07:41:02.688623+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:39.053900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:39.053900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050142590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"324'15007","reported_seq":22903,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T07:41:05.233662+0000","last_change":"2024-09-11T07:40:13.749548+0000","last_active":"2024-09-11T07:41:05.233662+0000","last_peered":"2024-09-11T07:41:05.233662+0000","last_clean":"2024-09-11T07:41:05.233662+0000","last_became_active":"2024-09-11T07:40:13.698098+0000","last_became_peered":"2024-09-11T07:40:13.698098+0000","last_unstale":"2024-09-11T07:41:05.233662+0000","last_undegraded":"2024-09-11T07:41:05.233662+0000","last_fullsized":"2024-09-11T07:41:05.233662+0000","mapping_epoch":298,"log_start":"282'10318","ondisk_log_start":"282'10318","created":247,"last_epoch_clean":299,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4689,"log_dups_size":0,"ondisk_log_size":4689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65601536,"num_objects":1001,"num_object_clones":0,"num_object_copies":3003,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15007,"num_write_kb":512256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3894,"num_bytes_recovered":255197184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"234'22","reported_seq":343,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.688094+0000","last_change":"2024-09-11T07:40:43.269210+0000","last_active":"2024-09-11T07:41:02.688094+0000","last_peered":"2024-09-11T07:41:02.688094+0000","last_clean":"2024-09-11T07:41:02.688094+0000","last_became_active":"2024-09-11T07:40:11.704155+0000","last_became_peered":"2024-09-11T07:40:11.704155+0000","last_unstale":"2024-09-11T07:41:02.688094+0000","last_undegraded":"2024-09-11T07:41:02.688094+0000","last_fullsized":"2024-09-11T07:41:02.688094+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'22","last_scrub_stamp":"2024-09-11T07:37:20.965147+0000","last_deep_scrub":"144'18","last_deep_scrub_stamp":"2024-09-11T07:31:51.603378+0000","last_clean_scrub_stamp":"2024-09-11T07:37:20.965147+0000","objects_scrubbed":5,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006530999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":1207,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T07:41:04.239631+0000","last_change":"2024-09-11T07:37:22.990093+0000","last_active":"2024-09-11T07:41:04.239631+0000","last_peered":"2024-09-11T07:41:04.239631+0000","last_clean":"2024-09-11T07:41:04.239631+0000","last_became_active":"2024-09-11T07:26:31.421276+0000","last_became_peered":"2024-09-11T07:26:31.421276+0000","last_unstale":"2024-09-11T07:41:04.239631+0000","last_undegraded":"2024-09-11T07:41:04.239631+0000","last_fullsized":"2024-09-11T07:41:04.239631+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'5","last_scrub_stamp":"2024-09-11T07:37:22.990033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:22.990033+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001646954,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":865,"num_read_kb":860,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":364,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.678516+0000","last_change":"2024-09-11T07:40:38.385663+0000","last_active":"2024-09-11T07:41:02.678516+0000","last_peered":"2024-09-11T07:41:02.678516+0000","last_clean":"2024-09-11T07:41:02.678516+0000","last_became_active":"2024-09-11T07:26:31.413354+0000","last_became_peered":"2024-09-11T07:26:31.413354+0000","last_unstale":"2024-09-11T07:41:02.678516+0000","last_undegraded":"2024-09-11T07:41:02.678516+0000","last_fullsized":"2024-09-11T07:41:02.678516+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:38.385583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:07.149367+0000","last_clean_scrub_stamp":"2024-09-11T07:40:38.385583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:53.440444+0000","scrub_duration":0.014349707999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":351,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.678758+0000","last_change":"2024-09-11T07:39:45.351040+0000","last_active":"2024-09-11T07:41:02.678758+0000","last_peered":"2024-09-11T07:41:02.678758+0000","last_clean":"2024-09-11T07:41:02.678758+0000","last_became_active":"2024-09-11T07:32:00.508398+0000","last_became_peered":"2024-09-11T07:32:00.508398+0000","last_unstale":"2024-09-11T07:41:02.678758+0000","last_undegraded":"2024-09-11T07:41:02.678758+0000","last_fullsized":"2024-09-11T07:41:02.678758+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:24.876930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:38:24.876930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0095042219999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":329,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.678641+0000","last_change":"2024-09-11T07:39:49.399814+0000","last_active":"2024-09-11T07:41:02.678641+0000","last_peered":"2024-09-11T07:41:02.678641+0000","last_clean":"2024-09-11T07:41:02.678641+0000","last_became_active":"2024-09-11T07:32:00.515745+0000","last_became_peered":"2024-09-11T07:32:00.515745+0000","last_unstale":"2024-09-11T07:41:02.678641+0000","last_undegraded":"2024-09-11T07:41:02.678641+0000","last_fullsized":"2024-09-11T07:41:02.678641+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:49.399708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:56.490940+0000","last_clean_scrub_stamp":"2024-09-11T07:39:49.399708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:16.075069+0000","scrub_duration":0.0053800089999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"324'15056","reported_seq":15141,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T07:41:05.343296+0000","last_change":"2024-09-11T07:37:26.029935+0000","last_active":"2024-09-11T07:41:05.343296+0000","last_peered":"2024-09-11T07:41:05.343296+0000","last_clean":"2024-09-11T07:41:05.343296+0000","last_became_active":"2024-09-11T07:37:26.029608+0000","last_became_peered":"2024-09-11T07:37:26.029608+0000","last_unstale":"2024-09-11T07:41:05.343296+0000","last_undegraded":"2024-09-11T07:41:05.343296+0000","last_fullsized":"2024-09-11T07:41:05.343296+0000","mapping_epoch":247,"log_start":"284'10644","ondisk_log_start":"284'10644","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4412,"log_dups_size":0,"ondisk_log_size":4412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63700992,"num_objects":972,"num_object_clones":0,"num_object_copies":2916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15056,"num_write_kb":512896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"234'30","reported_seq":409,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.683532+0000","last_change":"2024-09-11T07:40:34.716884+0000","last_active":"2024-09-11T07:41:02.683532+0000","last_peered":"2024-09-11T07:41:02.683532+0000","last_clean":"2024-09-11T07:41:02.683532+0000","last_became_active":"2024-09-11T07:28:09.018134+0000","last_became_peered":"2024-09-11T07:28:09.018134+0000","last_unstale":"2024-09-11T07:41:02.683532+0000","last_undegraded":"2024-09-11T07:41:02.683532+0000","last_fullsized":"2024-09-11T07:41:02.683532+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'30","last_scrub_stamp":"2024-09-11T07:40:34.716804+0000","last_deep_scrub":"94'18","last_deep_scrub_stamp":"2024-09-11T07:28:51.437397+0000","last_clean_scrub_stamp":"2024-09-11T07:40:34.716804+0000","objects_scrubbed":6,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:00.562358+0000","scrub_duration":0.0055680800000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":355,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.683212+0000","last_change":"2024-09-11T07:39:57.769695+0000","last_active":"2024-09-11T07:41:02.683212+0000","last_peered":"2024-09-11T07:41:02.683212+0000","last_clean":"2024-09-11T07:41:02.683212+0000","last_became_active":"2024-09-11T07:39:57.758029+0000","last_became_peered":"2024-09-11T07:39:57.758029+0000","last_unstale":"2024-09-11T07:41:02.683212+0000","last_undegraded":"2024-09-11T07:41:02.683212+0000","last_fullsized":"2024-09-11T07:41:02.683212+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-11T07:38:21.385813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:38:21.385813+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060130330000000001,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":340,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.678286+0000","last_change":"2024-09-11T07:39:57.772569+0000","last_active":"2024-09-11T07:41:02.678286+0000","last_peered":"2024-09-11T07:41:02.678286+0000","last_clean":"2024-09-11T07:41:02.678286+0000","last_became_active":"2024-09-11T07:39:57.772244+0000","last_became_peered":"2024-09-11T07:39:57.772244+0000","last_unstale":"2024-09-11T07:41:02.678286+0000","last_undegraded":"2024-09-11T07:41:02.678286+0000","last_fullsized":"2024-09-11T07:41:02.678286+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:26.556972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:16.967492+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.556972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001503206,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":48,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.681923+0000","last_change":"2024-09-11T07:39:57.688960+0000","last_active":"2024-09-11T07:41:02.681923+0000","last_peered":"2024-09-11T07:41:02.681923+0000","last_clean":"2024-09-11T07:41:02.681923+0000","last_became_active":"2024-09-11T07:39:57.688619+0000","last_became_peered":"2024-09-11T07:39:57.688619+0000","last_unstale":"2024-09-11T07:41:02.681923+0000","last_undegraded":"2024-09-11T07:41:02.681923+0000","last_fullsized":"2024-09-11T07:41:02.681923+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:23.100768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:51.187127+0000","last_clean_scrub_stamp":"2024-09-11T07:38:23.100768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":48,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.687764+0000","last_change":"2024-09-11T07:39:57.772562+0000","last_active":"2024-09-11T07:41:02.687764+0000","last_peered":"2024-09-11T07:41:02.687764+0000","last_clean":"2024-09-11T07:41:02.687764+0000","last_became_active":"2024-09-11T07:39:57.772223+0000","last_became_peered":"2024-09-11T07:39:57.772223+0000","last_unstale":"2024-09-11T07:41:02.687764+0000","last_undegraded":"2024-09-11T07:41:02.687764+0000","last_fullsized":"2024-09-11T07:41:02.687764+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:25.907711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:38:25.907711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"324'10132","reported_seq":11910,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T07:41:05.328110+0000","last_change":"2024-09-11T07:41:03.709361+0000","last_active":"2024-09-11T07:41:05.328110+0000","last_peered":"2024-09-11T07:41:05.328110+0000","last_clean":"2024-09-11T07:41:05.328110+0000","last_became_active":"2024-09-11T07:41:03.691512+0000","last_became_peered":"2024-09-11T07:41:03.691512+0000","last_unstale":"2024-09-11T07:41:05.328110+0000","last_undegraded":"2024-09-11T07:41:05.328110+0000","last_fullsized":"2024-09-11T07:41:05.328110+0000","mapping_epoch":323,"log_start":"266'5724","ondisk_log_start":"266'5724","created":247,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4408,"log_dups_size":0,"ondisk_log_size":4408,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28835840,"num_objects":440,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7458,"num_write_kb":252736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":550,"num_bytes_recovered":36044800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"234'26","reported_seq":174,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.719713+0000","last_change":"2024-09-11T07:37:40.649065+0000","last_active":"2024-09-11T07:41:02.719713+0000","last_peered":"2024-09-11T07:41:02.719713+0000","last_clean":"2024-09-11T07:41:02.719713+0000","last_became_active":"2024-09-11T07:36:05.224704+0000","last_became_peered":"2024-09-11T07:36:05.224704+0000","last_unstale":"2024-09-11T07:41:02.719713+0000","last_undegraded":"2024-09-11T07:41:02.719713+0000","last_fullsized":"2024-09-11T07:41:02.719713+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'26","last_scrub_stamp":"2024-09-11T07:37:40.648986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:40.648986+0000","objects_scrubbed":7,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030578620000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'3","reported_seq":347,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.687861+0000","last_change":"2024-09-11T07:40:11.706429+0000","last_active":"2024-09-11T07:41:02.687861+0000","last_peered":"2024-09-11T07:41:02.687861+0000","last_clean":"2024-09-11T07:41:02.687861+0000","last_became_active":"2024-09-11T07:40:11.701416+0000","last_became_peered":"2024-09-11T07:40:11.701416+0000","last_unstale":"2024-09-11T07:41:02.687861+0000","last_undegraded":"2024-09-11T07:41:02.687861+0000","last_fullsized":"2024-09-11T07:41:02.687861+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-09-11T07:36:54.937201+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_clean_scrub_stamp":"2024-09-11T07:36:54.937201+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007045239,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":355,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.687510+0000","last_change":"2024-09-11T07:40:11.706063+0000","last_active":"2024-09-11T07:41:02.687510+0000","last_peered":"2024-09-11T07:41:02.687510+0000","last_clean":"2024-09-11T07:41:02.687510+0000","last_became_active":"2024-09-11T07:40:11.705286+0000","last_became_peered":"2024-09-11T07:40:11.705286+0000","last_unstale":"2024-09-11T07:41:02.687510+0000","last_undegraded":"2024-09-11T07:41:02.687510+0000","last_fullsized":"2024-09-11T07:41:02.687510+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:35.504238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:35.504238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013066169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"87'81","reported_seq":172,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.684711+0000","last_change":"2024-09-11T07:40:38.751275+0000","last_active":"2024-09-11T07:41:02.684711+0000","last_peered":"2024-09-11T07:41:02.684711+0000","last_clean":"2024-09-11T07:41:02.684711+0000","last_became_active":"2024-09-11T07:39:57.761901+0000","last_became_peered":"2024-09-11T07:39:57.761901+0000","last_unstale":"2024-09-11T07:41:02.684711+0000","last_undegraded":"2024-09-11T07:41:02.684711+0000","last_fullsized":"2024-09-11T07:41:02.684711+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:40:38.751193+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:40:38.751193+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:44.042328+0000","scrub_duration":0.010400779000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":151,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.679229+0000","last_change":"2024-09-11T07:39:57.749041+0000","last_active":"2024-09-11T07:41:02.679229+0000","last_peered":"2024-09-11T07:41:02.679229+0000","last_clean":"2024-09-11T07:41:02.679229+0000","last_became_active":"2024-09-11T07:39:57.748658+0000","last_became_peered":"2024-09-11T07:39:57.748658+0000","last_unstale":"2024-09-11T07:41:02.679229+0000","last_undegraded":"2024-09-11T07:41:02.679229+0000","last_fullsized":"2024-09-11T07:41:02.679229+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:41.562322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:41.562322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":322,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.687978+0000","last_change":"2024-09-11T07:40:41.208608+0000","last_active":"2024-09-11T07:41:02.687978+0000","last_peered":"2024-09-11T07:41:02.687978+0000","last_clean":"2024-09-11T07:41:02.687978+0000","last_became_active":"2024-09-11T07:26:37.559920+0000","last_became_peered":"2024-09-11T07:26:37.559920+0000","last_unstale":"2024-09-11T07:41:02.687978+0000","last_undegraded":"2024-09-11T07:41:02.687978+0000","last_fullsized":"2024-09-11T07:41:02.687978+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:23.436012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:23.436012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001213481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.7","version":"324'10005","reported_seq":10122,"reported_epoch":324,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-11T07:41:04.335660+0000","last_change":"2024-09-11T07:40:59.016836+0000","last_active":"2024-09-11T07:41:04.335660+0000","last_peered":"2024-09-11T07:41:04.335660+0000","last_clean":"2024-09-11T07:39:56.662527+0000","last_became_active":"2024-09-11T07:39:58.961246+0000","last_became_peered":"2024-09-11T07:39:58.961246+0000","last_unstale":"2024-09-11T07:41:04.335660+0000","last_undegraded":"2024-09-11T07:41:04.335660+0000","last_fullsized":"2024-09-11T07:41:04.335660+0000","mapping_epoch":289,"log_start":"256'4058","ondisk_log_start":"256'4058","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":5947,"log_dups_size":0,"ondisk_log_size":5947,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35717120,"num_objects":545,"num_object_clones":0,"num_object_copies":1635,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1090,"num_objects_unfound":0,"num_objects_dirty":545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7301,"num_write_kb":251072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,3,7],"avail_no_missing":["5","3","7"],"object_location_counts":[{"shards":"3,5,7","objects":545}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"234'35","reported_seq":218,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.689187+0000","last_change":"2024-09-11T07:39:57.784855+0000","last_active":"2024-09-11T07:41:02.689187+0000","last_peered":"2024-09-11T07:41:02.689187+0000","last_clean":"2024-09-11T07:41:02.689187+0000","last_became_active":"2024-09-11T07:39:57.759856+0000","last_became_peered":"2024-09-11T07:39:57.759856+0000","last_unstale":"2024-09-11T07:41:02.689187+0000","last_undegraded":"2024-09-11T07:41:02.689187+0000","last_fullsized":"2024-09-11T07:41:02.689187+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'35","last_scrub_stamp":"2024-09-11T07:38:23.944683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:38:23.944683+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003078154,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":357,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.679300+0000","last_change":"2024-09-11T07:40:35.376311+0000","last_active":"2024-09-11T07:41:02.679300+0000","last_peered":"2024-09-11T07:41:02.679300+0000","last_clean":"2024-09-11T07:41:02.679300+0000","last_became_active":"2024-09-11T07:26:31.419560+0000","last_became_peered":"2024-09-11T07:26:31.419560+0000","last_unstale":"2024-09-11T07:41:02.679300+0000","last_undegraded":"2024-09-11T07:41:02.679300+0000","last_fullsized":"2024-09-11T07:41:02.679300+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:35.376219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:40:35.376219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:44.257357+0000","scrub_duration":0.003361682,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":354,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.678557+0000","last_change":"2024-09-11T07:40:40.411425+0000","last_active":"2024-09-11T07:41:02.678557+0000","last_peered":"2024-09-11T07:41:02.678557+0000","last_clean":"2024-09-11T07:41:02.678557+0000","last_became_active":"2024-09-11T07:39:57.748953+0000","last_became_peered":"2024-09-11T07:39:57.748953+0000","last_unstale":"2024-09-11T07:41:02.678557+0000","last_undegraded":"2024-09-11T07:41:02.678557+0000","last_fullsized":"2024-09-11T07:41:02.678557+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:40.411364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:40:40.411364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:40.631026+0000","scrub_duration":0.0012767259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":159,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.678792+0000","last_change":"2024-09-11T07:39:57.767088+0000","last_active":"2024-09-11T07:41:02.678792+0000","last_peered":"2024-09-11T07:41:02.678792+0000","last_clean":"2024-09-11T07:41:02.678792+0000","last_became_active":"2024-09-11T07:39:57.766605+0000","last_became_peered":"2024-09-11T07:39:57.766605+0000","last_unstale":"2024-09-11T07:41:02.678792+0000","last_undegraded":"2024-09-11T07:41:02.678792+0000","last_fullsized":"2024-09-11T07:41:02.678792+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:41.428866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:39:41.428866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"324'10370","reported_seq":10467,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T07:41:04.338685+0000","last_change":"2024-09-11T07:40:45.283673+0000","last_active":"2024-09-11T07:41:04.338685+0000","last_peered":"2024-09-11T07:41:04.338685+0000","last_clean":"2024-09-11T07:41:04.338685+0000","last_became_active":"2024-09-11T07:38:27.351200+0000","last_became_peered":"2024-09-11T07:38:27.351200+0000","last_unstale":"2024-09-11T07:41:04.338685+0000","last_undegraded":"2024-09-11T07:41:04.338685+0000","last_fullsized":"2024-09-11T07:41:04.338685+0000","mapping_epoch":247,"log_start":"256'4716","ondisk_log_start":"256'4716","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":5654,"log_dups_size":0,"ondisk_log_size":5654,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":512,"num_object_clones":0,"num_object_copies":1536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7606,"num_write_kb":259776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"234'15","reported_seq":358,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.683621+0000","last_change":"2024-09-11T07:40:46.759444+0000","last_active":"2024-09-11T07:41:02.683621+0000","last_peered":"2024-09-11T07:41:02.683621+0000","last_clean":"2024-09-11T07:41:02.683621+0000","last_became_active":"2024-09-11T07:26:33.543148+0000","last_became_peered":"2024-09-11T07:26:33.543148+0000","last_unstale":"2024-09-11T07:41:02.683621+0000","last_undegraded":"2024-09-11T07:41:02.683621+0000","last_fullsized":"2024-09-11T07:41:02.683621+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:40:46.759189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:40:46.759189+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:55.039295+0000","scrub_duration":0.007573715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":334,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.681404+0000","last_change":"2024-09-11T07:37:26.019680+0000","last_active":"2024-09-11T07:41:02.681404+0000","last_peered":"2024-09-11T07:41:02.681404+0000","last_clean":"2024-09-11T07:41:02.681404+0000","last_became_active":"2024-09-11T07:26:31.423056+0000","last_became_peered":"2024-09-11T07:26:31.423056+0000","last_unstale":"2024-09-11T07:41:02.681404+0000","last_undegraded":"2024-09-11T07:41:02.681404+0000","last_fullsized":"2024-09-11T07:41:02.681404+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:26.019623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.019623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013393599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":350,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.679788+0000","last_change":"2024-09-11T07:39:52.485307+0000","last_active":"2024-09-11T07:41:02.679788+0000","last_peered":"2024-09-11T07:41:02.679788+0000","last_clean":"2024-09-11T07:41:02.679788+0000","last_became_active":"2024-09-11T07:26:35.563182+0000","last_became_peered":"2024-09-11T07:26:35.563182+0000","last_unstale":"2024-09-11T07:41:02.679788+0000","last_undegraded":"2024-09-11T07:41:02.679788+0000","last_fullsized":"2024-09-11T07:41:02.679788+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:52.485217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:39:52.485217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:10.747603+0000","scrub_duration":0.0013231569999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":336,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.683120+0000","last_change":"2024-09-11T07:36:36.023079+0000","last_active":"2024-09-11T07:41:02.683120+0000","last_peered":"2024-09-11T07:41:02.683120+0000","last_clean":"2024-09-11T07:41:02.683120+0000","last_became_active":"2024-09-11T07:32:00.505185+0000","last_became_peered":"2024-09-11T07:32:00.505185+0000","last_unstale":"2024-09-11T07:41:02.683120+0000","last_undegraded":"2024-09-11T07:41:02.683120+0000","last_fullsized":"2024-09-11T07:41:02.683120+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:36.023006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:46.944763+0000","last_clean_scrub_stamp":"2024-09-11T07:36:36.023006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011934019000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.5","version":"324'10261","reported_seq":12156,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T07:41:05.303242+0000","last_change":"2024-09-11T07:41:03.708277+0000","last_active":"2024-09-11T07:41:05.303242+0000","last_peered":"2024-09-11T07:41:05.303242+0000","last_clean":"2024-09-11T07:41:05.303242+0000","last_became_active":"2024-09-11T07:41:03.688690+0000","last_became_peered":"2024-09-11T07:41:03.688690+0000","last_unstale":"2024-09-11T07:41:05.303242+0000","last_undegraded":"2024-09-11T07:41:05.303242+0000","last_fullsized":"2024-09-11T07:41:05.303242+0000","mapping_epoch":323,"log_start":"266'5892","ondisk_log_start":"266'5892","created":247,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4369,"log_dups_size":0,"ondisk_log_size":4369,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32833536,"num_objects":501,"num_object_clones":0,"num_object_copies":1503,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7547,"num_write_kb":257536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":589,"num_bytes_recovered":38600704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"234'27","reported_seq":78,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.718100+0000","last_change":"2024-09-11T07:40:34.735380+0000","last_active":"2024-09-11T07:41:02.718100+0000","last_peered":"2024-09-11T07:41:02.718100+0000","last_clean":"2024-09-11T07:41:02.718100+0000","last_became_active":"2024-09-11T07:39:57.766550+0000","last_became_peered":"2024-09-11T07:39:57.766550+0000","last_unstale":"2024-09-11T07:41:02.718100+0000","last_undegraded":"2024-09-11T07:41:02.718100+0000","last_fullsized":"2024-09-11T07:41:02.718100+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'27","last_scrub_stamp":"2024-09-11T07:40:34.735292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:40:34.735292+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:40.999720+0000","scrub_duration":0.0055354389999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":392,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.678936+0000","last_change":"2024-09-11T07:40:11.702110+0000","last_active":"2024-09-11T07:41:02.678936+0000","last_peered":"2024-09-11T07:41:02.678936+0000","last_clean":"2024-09-11T07:41:02.678936+0000","last_became_active":"2024-09-11T07:40:11.701330+0000","last_became_peered":"2024-09-11T07:40:11.701330+0000","last_unstale":"2024-09-11T07:41:02.678936+0000","last_undegraded":"2024-09-11T07:41:02.678936+0000","last_fullsized":"2024-09-11T07:41:02.678936+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:46.402060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_clean_scrub_stamp":"2024-09-11T07:39:46.402060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:14.503163+0000","scrub_duration":0.0055267119999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"151'10","reported_seq":464,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.678238+0000","last_change":"2024-09-11T07:37:38.544127+0000","last_active":"2024-09-11T07:41:02.678238+0000","last_peered":"2024-09-11T07:41:02.678238+0000","last_clean":"2024-09-11T07:41:02.678238+0000","last_became_active":"2024-09-11T07:32:00.506806+0000","last_became_peered":"2024-09-11T07:32:00.506806+0000","last_unstale":"2024-09-11T07:41:02.678238+0000","last_undegraded":"2024-09-11T07:41:02.678238+0000","last_fullsized":"2024-09-11T07:41:02.678238+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:37:38.544072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:38.544072+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027401579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":322,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.687190+0000","last_change":"2024-09-11T07:40:33.314374+0000","last_active":"2024-09-11T07:41:02.687190+0000","last_peered":"2024-09-11T07:41:02.687190+0000","last_clean":"2024-09-11T07:41:02.687190+0000","last_became_active":"2024-09-11T07:26:37.861134+0000","last_became_peered":"2024-09-11T07:26:37.861134+0000","last_unstale":"2024-09-11T07:41:02.687190+0000","last_undegraded":"2024-09-11T07:41:02.687190+0000","last_fullsized":"2024-09-11T07:41:02.687190+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:44.293653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:44.293653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0074032500000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.4","version":"324'10202","reported_seq":10305,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T07:41:05.248316+0000","last_change":"2024-09-11T07:40:46.843129+0000","last_active":"2024-09-11T07:41:05.248316+0000","last_peered":"2024-09-11T07:41:05.248316+0000","last_clean":"2024-09-11T07:41:05.248316+0000","last_became_active":"2024-09-11T07:38:27.372060+0000","last_became_peered":"2024-09-11T07:38:27.372060+0000","last_unstale":"2024-09-11T07:41:05.248316+0000","last_undegraded":"2024-09-11T07:41:05.248316+0000","last_fullsized":"2024-09-11T07:41:05.248316+0000","mapping_epoch":247,"log_start":"265'5554","ondisk_log_start":"265'5554","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4648,"log_dups_size":0,"ondisk_log_size":4648,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29425664,"num_objects":449,"num_object_clones":0,"num_object_copies":1347,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7515,"num_write_kb":254848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"234'23","reported_seq":392,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.718978+0000","last_change":"2024-09-11T07:39:57.768799+0000","last_active":"2024-09-11T07:41:02.718978+0000","last_peered":"2024-09-11T07:41:02.718978+0000","last_clean":"2024-09-11T07:41:02.718978+0000","last_became_active":"2024-09-11T07:39:57.691713+0000","last_became_peered":"2024-09-11T07:39:57.691713+0000","last_unstale":"2024-09-11T07:41:02.718978+0000","last_undegraded":"2024-09-11T07:41:02.718978+0000","last_fullsized":"2024-09-11T07:41:02.718978+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'23","last_scrub_stamp":"2024-09-11T07:39:55.578519+0000","last_deep_scrub":"234'23","last_deep_scrub_stamp":"2024-09-11T07:37:34.613280+0000","last_clean_scrub_stamp":"2024-09-11T07:39:55.578519+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:15.441265+0000","scrub_duration":0.0030590740000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":358,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.719388+0000","last_change":"2024-09-11T07:39:27.582662+0000","last_active":"2024-09-11T07:41:02.719388+0000","last_peered":"2024-09-11T07:41:02.719388+0000","last_clean":"2024-09-11T07:41:02.719388+0000","last_became_active":"2024-09-11T07:32:00.504344+0000","last_became_peered":"2024-09-11T07:32:00.504344+0000","last_unstale":"2024-09-11T07:41:02.719388+0000","last_undegraded":"2024-09-11T07:41:02.719388+0000","last_fullsized":"2024-09-11T07:41:02.719388+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:35:46.769070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:35:46.769070+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049213720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":350,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.678870+0000","last_change":"2024-09-11T07:39:55.453328+0000","last_active":"2024-09-11T07:41:02.678870+0000","last_peered":"2024-09-11T07:41:02.678870+0000","last_clean":"2024-09-11T07:41:02.678870+0000","last_became_active":"2024-09-11T07:26:35.561967+0000","last_became_peered":"2024-09-11T07:26:35.561967+0000","last_unstale":"2024-09-11T07:41:02.678870+0000","last_undegraded":"2024-09-11T07:41:02.678870+0000","last_fullsized":"2024-09-11T07:41:02.678870+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:55.453221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:44.820354+0000","last_clean_scrub_stamp":"2024-09-11T07:39:55.453221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:11.983136+0000","scrub_duration":0.0014898649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":346,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.686715+0000","last_change":"2024-09-11T07:39:57.776965+0000","last_active":"2024-09-11T07:41:02.686715+0000","last_peered":"2024-09-11T07:41:02.686715+0000","last_clean":"2024-09-11T07:41:02.686715+0000","last_became_active":"2024-09-11T07:39:57.776576+0000","last_became_peered":"2024-09-11T07:39:57.776576+0000","last_unstale":"2024-09-11T07:41:02.686715+0000","last_undegraded":"2024-09-11T07:41:02.686715+0000","last_fullsized":"2024-09-11T07:41:02.686715+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:16.860930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:16.860930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024991000000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"234'15","reported_seq":385,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.683450+0000","last_change":"2024-09-11T07:39:57.786497+0000","last_active":"2024-09-11T07:41:02.683450+0000","last_peered":"2024-09-11T07:41:02.683450+0000","last_clean":"2024-09-11T07:41:02.683450+0000","last_became_active":"2024-09-11T07:39:57.767661+0000","last_became_peered":"2024-09-11T07:39:57.767661+0000","last_unstale":"2024-09-11T07:41:02.683450+0000","last_undegraded":"2024-09-11T07:41:02.683450+0000","last_fullsized":"2024-09-11T07:41:02.683450+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:37:19.096817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:19.096817+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00175015,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":354,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.678999+0000","last_change":"2024-09-11T07:39:57.443485+0000","last_active":"2024-09-11T07:41:02.678999+0000","last_peered":"2024-09-11T07:41:02.678999+0000","last_clean":"2024-09-11T07:41:02.678999+0000","last_became_active":"2024-09-11T07:26:31.415701+0000","last_became_peered":"2024-09-11T07:26:31.415701+0000","last_unstale":"2024-09-11T07:41:02.678999+0000","last_undegraded":"2024-09-11T07:41:02.678999+0000","last_fullsized":"2024-09-11T07:41:02.678999+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:57.443424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.925200+0000","last_clean_scrub_stamp":"2024-09-11T07:39:57.443424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:03.724321+0000","scrub_duration":0.0013832989999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":174,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.682180+0000","last_change":"2024-09-11T07:40:39.893675+0000","last_active":"2024-09-11T07:41:02.682180+0000","last_peered":"2024-09-11T07:41:02.682180+0000","last_clean":"2024-09-11T07:41:02.682180+0000","last_became_active":"2024-09-11T07:40:11.711154+0000","last_became_peered":"2024-09-11T07:40:11.711154+0000","last_unstale":"2024-09-11T07:41:02.682180+0000","last_undegraded":"2024-09-11T07:41:02.682180+0000","last_fullsized":"2024-09-11T07:41:02.682180+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:39.893619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:35.392360+0000","last_clean_scrub_stamp":"2024-09-11T07:40:39.893619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:46.233742+0000","scrub_duration":0.001330475,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":347,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.679469+0000","last_change":"2024-09-11T07:38:29.941071+0000","last_active":"2024-09-11T07:41:02.679469+0000","last_peered":"2024-09-11T07:41:02.679469+0000","last_clean":"2024-09-11T07:41:02.679469+0000","last_became_active":"2024-09-11T07:32:00.506697+0000","last_became_peered":"2024-09-11T07:32:00.506697+0000","last_unstale":"2024-09-11T07:41:02.679469+0000","last_undegraded":"2024-09-11T07:41:02.679469+0000","last_fullsized":"2024-09-11T07:41:02.679469+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:38:29.941008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:38:29.941008+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016886850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":125,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.719125+0000","last_change":"2024-09-11T07:39:46.613784+0000","last_active":"2024-09-11T07:41:02.719125+0000","last_peered":"2024-09-11T07:41:02.719125+0000","last_clean":"2024-09-11T07:41:02.719125+0000","last_became_active":"2024-09-11T07:36:05.225791+0000","last_became_peered":"2024-09-11T07:36:05.225791+0000","last_unstale":"2024-09-11T07:41:02.719125+0000","last_undegraded":"2024-09-11T07:41:02.719125+0000","last_fullsized":"2024-09-11T07:41:02.719125+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:51.617730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:51.617730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001776197,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"234'15","reported_seq":97,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.682055+0000","last_change":"2024-09-11T07:39:57.773581+0000","last_active":"2024-09-11T07:41:02.682055+0000","last_peered":"2024-09-11T07:41:02.682055+0000","last_clean":"2024-09-11T07:41:02.682055+0000","last_became_active":"2024-09-11T07:39:57.691830+0000","last_became_peered":"2024-09-11T07:39:57.691830+0000","last_unstale":"2024-09-11T07:41:02.682055+0000","last_undegraded":"2024-09-11T07:41:02.682055+0000","last_fullsized":"2024-09-11T07:41:02.682055+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:37:26.450122+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.450122+0000","objects_scrubbed":3,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":341,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.687660+0000","last_change":"2024-09-11T07:40:39.267299+0000","last_active":"2024-09-11T07:41:02.687660+0000","last_peered":"2024-09-11T07:41:02.687660+0000","last_clean":"2024-09-11T07:41:02.687660+0000","last_became_active":"2024-09-11T07:26:35.566894+0000","last_became_peered":"2024-09-11T07:26:35.566894+0000","last_unstale":"2024-09-11T07:41:02.687660+0000","last_undegraded":"2024-09-11T07:41:02.687660+0000","last_fullsized":"2024-09-11T07:41:02.687660+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:39.267248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:40:39.267248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:55.182199+0000","scrub_duration":0.0012024570000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":327,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.679065+0000","last_change":"2024-09-11T07:40:37.436683+0000","last_active":"2024-09-11T07:41:02.679065+0000","last_peered":"2024-09-11T07:41:02.679065+0000","last_clean":"2024-09-11T07:41:02.679065+0000","last_became_active":"2024-09-11T07:32:00.514100+0000","last_became_peered":"2024-09-11T07:32:00.514100+0000","last_unstale":"2024-09-11T07:41:02.679065+0000","last_undegraded":"2024-09-11T07:41:02.679065+0000","last_fullsized":"2024-09-11T07:41:02.679065+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:37.436628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:40:37.436628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:37.557030+0000","scrub_duration":0.001208518,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.19","version":"324'10254","reported_seq":22874,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T07:41:04.315751+0000","last_change":"2024-09-11T07:40:09.708552+0000","last_active":"2024-09-11T07:41:04.315751+0000","last_peered":"2024-09-11T07:41:04.315751+0000","last_clean":"2024-09-11T07:41:04.315751+0000","last_became_active":"2024-09-11T07:40:09.688156+0000","last_became_peered":"2024-09-11T07:40:09.688156+0000","last_unstale":"2024-09-11T07:41:04.315751+0000","last_undegraded":"2024-09-11T07:41:04.315751+0000","last_fullsized":"2024-09-11T07:41:04.315751+0000","mapping_epoch":294,"log_start":"256'4582","ondisk_log_start":"256'4582","created":259,"last_epoch_clean":295,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":5672,"log_dups_size":0,"ondisk_log_size":5672,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34209792,"num_objects":522,"num_object_clones":0,"num_object_copies":1566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7546,"num_write_kb":258176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4857,"num_bytes_recovered":318308352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"234'25","reported_seq":397,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.688084+0000","last_change":"2024-09-11T07:39:57.777199+0000","last_active":"2024-09-11T07:41:02.688084+0000","last_peered":"2024-09-11T07:41:02.688084+0000","last_clean":"2024-09-11T07:41:02.688084+0000","last_became_active":"2024-09-11T07:39:57.771009+0000","last_became_peered":"2024-09-11T07:39:57.771009+0000","last_unstale":"2024-09-11T07:41:02.688084+0000","last_undegraded":"2024-09-11T07:41:02.688084+0000","last_fullsized":"2024-09-11T07:41:02.688084+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'25","last_scrub_stamp":"2024-09-11T07:36:55.966595+0000","last_deep_scrub":"86'15","last_deep_scrub_stamp":"2024-09-11T07:28:22.858137+0000","last_clean_scrub_stamp":"2024-09-11T07:36:55.966595+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.032608074000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":355,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.687641+0000","last_change":"2024-09-11T07:39:57.764438+0000","last_active":"2024-09-11T07:41:02.687641+0000","last_peered":"2024-09-11T07:41:02.687641+0000","last_clean":"2024-09-11T07:41:02.687641+0000","last_became_active":"2024-09-11T07:39:57.764063+0000","last_became_peered":"2024-09-11T07:39:57.764063+0000","last_unstale":"2024-09-11T07:41:02.687641+0000","last_undegraded":"2024-09-11T07:41:02.687641+0000","last_fullsized":"2024-09-11T07:41:02.687641+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:54.493614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:39:54.493614+0000","last_clean_scrub_stamp":"2024-09-11T07:39:54.493614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:22.925815+0000","scrub_duration":0.0049403759999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":513,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.688523+0000","last_change":"2024-09-11T07:40:46.321277+0000","last_active":"2024-09-11T07:41:02.688523+0000","last_peered":"2024-09-11T07:41:02.688523+0000","last_clean":"2024-09-11T07:41:02.688523+0000","last_became_active":"2024-09-11T07:26:35.562245+0000","last_became_peered":"2024-09-11T07:26:35.562245+0000","last_unstale":"2024-09-11T07:41:02.688523+0000","last_undegraded":"2024-09-11T07:41:02.688523+0000","last_fullsized":"2024-09-11T07:41:02.688523+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:40:46.321189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:40:46.321189+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:03.756105+0000","scrub_duration":0.024243521000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":356,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.683129+0000","last_change":"2024-09-11T07:39:57.759567+0000","last_active":"2024-09-11T07:41:02.683129+0000","last_peered":"2024-09-11T07:41:02.683129+0000","last_clean":"2024-09-11T07:41:02.683129+0000","last_became_active":"2024-09-11T07:39:57.758650+0000","last_became_peered":"2024-09-11T07:39:57.758650+0000","last_unstale":"2024-09-11T07:41:02.683129+0000","last_undegraded":"2024-09-11T07:41:02.683129+0000","last_fullsized":"2024-09-11T07:41:02.683129+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:42.084305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:42.084305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047035039999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"324'10038","reported_seq":24615,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T07:41:04.999360+0000","last_change":"2024-09-11T07:40:49.712435+0000","last_active":"2024-09-11T07:41:04.999360+0000","last_peered":"2024-09-11T07:41:04.999360+0000","last_clean":"2024-09-11T07:41:04.999360+0000","last_became_active":"2024-09-11T07:40:49.697184+0000","last_became_peered":"2024-09-11T07:40:49.697184+0000","last_unstale":"2024-09-11T07:41:04.999360+0000","last_undegraded":"2024-09-11T07:41:04.999360+0000","last_fullsized":"2024-09-11T07:41:04.999360+0000","mapping_epoch":315,"log_start":"265'5509","ondisk_log_start":"265'5509","created":259,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4529,"log_dups_size":0,"ondisk_log_size":4529,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34865152,"num_objects":532,"num_object_clones":0,"num_object_copies":1596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7364,"num_write_kb":252672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5363,"num_bytes_recovered":351469568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"234'19","reported_seq":329,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.678885+0000","last_change":"2024-09-11T07:39:57.780187+0000","last_active":"2024-09-11T07:41:02.678885+0000","last_peered":"2024-09-11T07:41:02.678885+0000","last_clean":"2024-09-11T07:41:02.678885+0000","last_became_active":"2024-09-11T07:39:57.765149+0000","last_became_peered":"2024-09-11T07:39:57.765149+0000","last_unstale":"2024-09-11T07:41:02.678885+0000","last_undegraded":"2024-09-11T07:41:02.678885+0000","last_fullsized":"2024-09-11T07:41:02.678885+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:37:41.338036+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:37:41.338036+0000","last_clean_scrub_stamp":"2024-09-11T07:37:41.338036+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045362370000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":328,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.683877+0000","last_change":"2024-09-11T07:40:44.709140+0000","last_active":"2024-09-11T07:41:02.683877+0000","last_peered":"2024-09-11T07:41:02.683877+0000","last_clean":"2024-09-11T07:41:02.683877+0000","last_became_active":"2024-09-11T07:26:31.419409+0000","last_became_peered":"2024-09-11T07:26:31.419409+0000","last_unstale":"2024-09-11T07:41:02.683877+0000","last_undegraded":"2024-09-11T07:41:02.683877+0000","last_fullsized":"2024-09-11T07:41:02.683877+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:27.958386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:27.958386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0093493710000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"290'11","reported_seq":255,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.678626+0000","last_change":"2024-09-11T07:39:57.778860+0000","last_active":"2024-09-11T07:41:02.678626+0000","last_peered":"2024-09-11T07:41:02.678626+0000","last_clean":"2024-09-11T07:41:02.678626+0000","last_became_active":"2024-09-11T07:39:57.772790+0000","last_became_peered":"2024-09-11T07:39:57.772790+0000","last_unstale":"2024-09-11T07:41:02.678626+0000","last_undegraded":"2024-09-11T07:41:02.678626+0000","last_fullsized":"2024-09-11T07:41:02.678626+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:38:25.065071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:38:25.065071+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020633370000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":322,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.718999+0000","last_change":"2024-09-11T07:39:50.572103+0000","last_active":"2024-09-11T07:41:02.718999+0000","last_peered":"2024-09-11T07:41:02.718999+0000","last_clean":"2024-09-11T07:41:02.718999+0000","last_became_active":"2024-09-11T07:26:37.565723+0000","last_became_peered":"2024-09-11T07:26:37.565723+0000","last_unstale":"2024-09-11T07:41:02.718999+0000","last_undegraded":"2024-09-11T07:41:02.718999+0000","last_fullsized":"2024-09-11T07:41:02.718999+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:26.659337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.659337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012563839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"236'16","reported_seq":327,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.679870+0000","last_change":"2024-09-11T07:40:29.564753+0000","last_active":"2024-09-11T07:41:02.679870+0000","last_peered":"2024-09-11T07:41:02.679870+0000","last_clean":"2024-09-11T07:41:02.679870+0000","last_became_active":"2024-09-11T07:39:57.689424+0000","last_became_peered":"2024-09-11T07:39:57.689424+0000","last_unstale":"2024-09-11T07:41:02.679870+0000","last_undegraded":"2024-09-11T07:41:02.679870+0000","last_fullsized":"2024-09-11T07:41:02.679870+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'16","last_scrub_stamp":"2024-09-11T07:36:49.809484+0000","last_deep_scrub":"236'16","last_deep_scrub_stamp":"2024-09-11T07:36:49.809484+0000","last_clean_scrub_stamp":"2024-09-11T07:36:49.809484+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011537936,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":131,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.688144+0000","last_change":"2024-09-11T07:39:57.779694+0000","last_active":"2024-09-11T07:41:02.688144+0000","last_peered":"2024-09-11T07:41:02.688144+0000","last_clean":"2024-09-11T07:41:02.688144+0000","last_became_active":"2024-09-11T07:39:57.776841+0000","last_became_peered":"2024-09-11T07:39:57.776841+0000","last_unstale":"2024-09-11T07:41:02.688144+0000","last_undegraded":"2024-09-11T07:41:02.688144+0000","last_fullsized":"2024-09-11T07:41:02.688144+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:22.914781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:09.772000+0000","last_clean_scrub_stamp":"2024-09-11T07:37:22.914781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001446864,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":317,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.686798+0000","last_change":"2024-09-11T07:40:31.747754+0000","last_active":"2024-09-11T07:41:02.686798+0000","last_peered":"2024-09-11T07:41:02.686798+0000","last_clean":"2024-09-11T07:41:02.686798+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:41:02.686798+0000","last_undegraded":"2024-09-11T07:41:02.686798+0000","last_fullsized":"2024-09-11T07:41:02.686798+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:35:51.886407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:35:51.886407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002900241,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":339,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.682577+0000","last_change":"2024-09-11T07:39:57.765790+0000","last_active":"2024-09-11T07:41:02.682577+0000","last_peered":"2024-09-11T07:41:02.682577+0000","last_clean":"2024-09-11T07:41:02.682577+0000","last_became_active":"2024-09-11T07:39:57.761754+0000","last_became_peered":"2024-09-11T07:39:57.761754+0000","last_unstale":"2024-09-11T07:41:02.682577+0000","last_undegraded":"2024-09-11T07:41:02.682577+0000","last_fullsized":"2024-09-11T07:41:02.682577+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:32.186561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:32.186561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014950059999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"234'24","reported_seq":362,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.718655+0000","last_change":"2024-09-11T07:39:42.612220+0000","last_active":"2024-09-11T07:41:02.718655+0000","last_peered":"2024-09-11T07:41:02.718655+0000","last_clean":"2024-09-11T07:41:02.718655+0000","last_became_active":"2024-09-11T07:26:33.540208+0000","last_became_peered":"2024-09-11T07:26:33.540208+0000","last_unstale":"2024-09-11T07:41:02.718655+0000","last_undegraded":"2024-09-11T07:41:02.718655+0000","last_fullsized":"2024-09-11T07:41:02.718655+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'16","last_scrub_stamp":"2024-09-11T07:36:31.789841+0000","last_deep_scrub":"136'16","last_deep_scrub_stamp":"2024-09-11T07:31:06.039023+0000","last_clean_scrub_stamp":"2024-09-11T07:36:31.789841+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048561150000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":337,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.681253+0000","last_change":"2024-09-11T07:39:51.779556+0000","last_active":"2024-09-11T07:41:02.681253+0000","last_peered":"2024-09-11T07:41:02.681253+0000","last_clean":"2024-09-11T07:41:02.681253+0000","last_became_active":"2024-09-11T07:26:31.422671+0000","last_became_peered":"2024-09-11T07:26:31.422671+0000","last_unstale":"2024-09-11T07:41:02.681253+0000","last_undegraded":"2024-09-11T07:41:02.681253+0000","last_fullsized":"2024-09-11T07:41:02.681253+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:51.779468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:39:51.779468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0089535459999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":332,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.681844+0000","last_change":"2024-09-11T07:39:57.761360+0000","last_active":"2024-09-11T07:41:02.681844+0000","last_peered":"2024-09-11T07:41:02.681844+0000","last_clean":"2024-09-11T07:41:02.681844+0000","last_became_active":"2024-09-11T07:39:57.759390+0000","last_became_peered":"2024-09-11T07:39:57.759390+0000","last_unstale":"2024-09-11T07:41:02.681844+0000","last_undegraded":"2024-09-11T07:41:02.681844+0000","last_fullsized":"2024-09-11T07:41:02.681844+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:55.895853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:39:55.895853+0000","last_clean_scrub_stamp":"2024-09-11T07:39:55.895853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:17.232667+0000","scrub_duration":0.011708260999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":58,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.682786+0000","last_change":"2024-09-11T07:40:11.701200+0000","last_active":"2024-09-11T07:41:02.682786+0000","last_peered":"2024-09-11T07:41:02.682786+0000","last_clean":"2024-09-11T07:41:02.682786+0000","last_became_active":"2024-09-11T07:40:11.700676+0000","last_became_peered":"2024-09-11T07:40:11.700676+0000","last_unstale":"2024-09-11T07:41:02.682786+0000","last_undegraded":"2024-09-11T07:41:02.682786+0000","last_fullsized":"2024-09-11T07:41:02.682786+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:43.814370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:43.814370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"234'10","reported_seq":337,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.686834+0000","last_change":"2024-09-11T07:36:46.900027+0000","last_active":"2024-09-11T07:41:02.686834+0000","last_peered":"2024-09-11T07:41:02.686834+0000","last_clean":"2024-09-11T07:41:02.686834+0000","last_became_active":"2024-09-11T07:26:33.545830+0000","last_became_peered":"2024-09-11T07:26:33.545830+0000","last_unstale":"2024-09-11T07:41:02.686834+0000","last_undegraded":"2024-09-11T07:41:02.686834+0000","last_fullsized":"2024-09-11T07:41:02.686834+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'10","last_scrub_stamp":"2024-09-11T07:36:46.899947+0000","last_deep_scrub":"82'6","last_deep_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_clean_scrub_stamp":"2024-09-11T07:36:46.899947+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003412277,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":335,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.687777+0000","last_change":"2024-09-11T07:39:56.466407+0000","last_active":"2024-09-11T07:41:02.687777+0000","last_peered":"2024-09-11T07:41:02.687777+0000","last_clean":"2024-09-11T07:41:02.687777+0000","last_became_active":"2024-09-11T07:26:31.420518+0000","last_became_peered":"2024-09-11T07:26:31.420518+0000","last_unstale":"2024-09-11T07:41:02.687777+0000","last_undegraded":"2024-09-11T07:41:02.687777+0000","last_fullsized":"2024-09-11T07:41:02.687777+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:56.466067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_clean_scrub_stamp":"2024-09-11T07:39:56.466067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030505950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":266,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.718838+0000","last_change":"2024-09-11T07:40:11.700029+0000","last_active":"2024-09-11T07:41:02.718838+0000","last_peered":"2024-09-11T07:41:02.718838+0000","last_clean":"2024-09-11T07:41:02.718838+0000","last_became_active":"2024-09-11T07:40:11.699702+0000","last_became_peered":"2024-09-11T07:40:11.699702+0000","last_unstale":"2024-09-11T07:41:02.718838+0000","last_undegraded":"2024-09-11T07:41:02.718838+0000","last_fullsized":"2024-09-11T07:41:02.718838+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:34.658586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:47.675050+0000","last_clean_scrub_stamp":"2024-09-11T07:39:34.658586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047285640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":329,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.678178+0000","last_change":"2024-09-11T07:39:43.398492+0000","last_active":"2024-09-11T07:41:02.678178+0000","last_peered":"2024-09-11T07:41:02.678178+0000","last_clean":"2024-09-11T07:41:02.678178+0000","last_became_active":"2024-09-11T07:32:00.511407+0000","last_became_peered":"2024-09-11T07:32:00.511407+0000","last_unstale":"2024-09-11T07:41:02.678178+0000","last_undegraded":"2024-09-11T07:41:02.678178+0000","last_fullsized":"2024-09-11T07:41:02.678178+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:43.398419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:39:43.398419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:06.930545+0000","scrub_duration":0.0064759270000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"234'19","reported_seq":383,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.686964+0000","last_change":"2024-09-11T07:39:57.791332+0000","last_active":"2024-09-11T07:41:02.686964+0000","last_peered":"2024-09-11T07:41:02.686964+0000","last_clean":"2024-09-11T07:41:02.686964+0000","last_became_active":"2024-09-11T07:39:57.767928+0000","last_became_peered":"2024-09-11T07:39:57.767928+0000","last_unstale":"2024-09-11T07:41:02.686964+0000","last_undegraded":"2024-09-11T07:41:02.686964+0000","last_fullsized":"2024-09-11T07:41:02.686964+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:38:28.017124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:38:28.017124+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018425620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":121,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.688088+0000","last_change":"2024-09-11T07:39:29.286259+0000","last_active":"2024-09-11T07:41:02.688088+0000","last_peered":"2024-09-11T07:41:02.688088+0000","last_clean":"2024-09-11T07:41:02.688088+0000","last_became_active":"2024-09-11T07:36:05.228360+0000","last_became_peered":"2024-09-11T07:36:05.228360+0000","last_unstale":"2024-09-11T07:41:02.688088+0000","last_undegraded":"2024-09-11T07:41:02.688088+0000","last_fullsized":"2024-09-11T07:41:02.688088+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:29.286202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:48.366931+0000","last_clean_scrub_stamp":"2024-09-11T07:39:29.286202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001125411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":302,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.677991+0000","last_change":"2024-09-11T07:39:56.318341+0000","last_active":"2024-09-11T07:41:02.677991+0000","last_peered":"2024-09-11T07:41:02.677991+0000","last_clean":"2024-09-11T07:41:02.677991+0000","last_became_active":"2024-09-11T07:32:00.520534+0000","last_became_peered":"2024-09-11T07:32:00.520534+0000","last_unstale":"2024-09-11T07:41:02.677991+0000","last_undegraded":"2024-09-11T07:41:02.677991+0000","last_fullsized":"2024-09-11T07:41:02.677991+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:56.318259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:39.021876+0000","last_clean_scrub_stamp":"2024-09-11T07:39:56.318259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:25.586713+0000","scrub_duration":0.0056755970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":309,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.686636+0000","last_change":"2024-09-11T07:36:49.899993+0000","last_active":"2024-09-11T07:41:02.686636+0000","last_peered":"2024-09-11T07:41:02.686636+0000","last_clean":"2024-09-11T07:41:02.686636+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:41:02.686636+0000","last_undegraded":"2024-09-11T07:41:02.686636+0000","last_fullsized":"2024-09-11T07:41:02.686636+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:49.899620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:49.899620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014021288999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"324'9962","reported_seq":22367,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T07:41:04.345738+0000","last_change":"2024-09-11T07:40:27.421619+0000","last_active":"2024-09-11T07:41:04.345738+0000","last_peered":"2024-09-11T07:41:04.345738+0000","last_clean":"2024-09-11T07:41:04.345738+0000","last_became_active":"2024-09-11T07:40:27.404959+0000","last_became_peered":"2024-09-11T07:40:27.404959+0000","last_unstale":"2024-09-11T07:41:04.345738+0000","last_undegraded":"2024-09-11T07:41:04.345738+0000","last_fullsized":"2024-09-11T07:41:04.345738+0000","mapping_epoch":302,"log_start":"256'4095","ondisk_log_start":"256'4095","created":259,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":5867,"log_dups_size":0,"ondisk_log_size":5867,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32735232,"num_objects":500,"num_object_clones":0,"num_object_copies":1500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7310,"num_write_kb":249888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4649,"num_bytes_recovered":304676864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":299,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.678386+0000","last_change":"2024-09-11T07:40:44.505650+0000","last_active":"2024-09-11T07:41:02.678386+0000","last_peered":"2024-09-11T07:41:02.678386+0000","last_clean":"2024-09-11T07:41:02.678386+0000","last_became_active":"2024-09-11T07:32:00.511258+0000","last_became_peered":"2024-09-11T07:32:00.511258+0000","last_unstale":"2024-09-11T07:41:02.678386+0000","last_undegraded":"2024-09-11T07:41:02.678386+0000","last_fullsized":"2024-09-11T07:41:02.678386+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:36:44.447809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:44.447809+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005540555,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":361,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.687829+0000","last_change":"2024-09-11T07:40:32.337500+0000","last_active":"2024-09-11T07:41:02.687829+0000","last_peered":"2024-09-11T07:41:02.687829+0000","last_clean":"2024-09-11T07:41:02.687829+0000","last_became_active":"2024-09-11T07:39:57.696805+0000","last_became_peered":"2024-09-11T07:39:57.696805+0000","last_unstale":"2024-09-11T07:41:02.687829+0000","last_undegraded":"2024-09-11T07:41:02.687829+0000","last_fullsized":"2024-09-11T07:41:02.687829+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:36:31.353426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:31.353426+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0063773889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"234'20","reported_seq":166,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.719430+0000","last_change":"2024-09-11T07:40:47.544520+0000","last_active":"2024-09-11T07:41:02.719430+0000","last_peered":"2024-09-11T07:41:02.719430+0000","last_clean":"2024-09-11T07:41:02.719430+0000","last_became_active":"2024-09-11T07:35:00.561615+0000","last_became_peered":"2024-09-11T07:35:00.561615+0000","last_unstale":"2024-09-11T07:41:02.719430+0000","last_undegraded":"2024-09-11T07:41:02.719430+0000","last_fullsized":"2024-09-11T07:41:02.719430+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'20","last_scrub_stamp":"2024-09-11T07:37:29.661671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:29.661671+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019609900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1f","version":"290'12","reported_seq":219,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.719313+0000","last_change":"2024-09-11T07:39:57.749653+0000","last_active":"2024-09-11T07:41:02.719313+0000","last_peered":"2024-09-11T07:41:02.719313+0000","last_clean":"2024-09-11T07:41:02.719313+0000","last_became_active":"2024-09-11T07:39:57.687809+0000","last_became_peered":"2024-09-11T07:39:57.687809+0000","last_unstale":"2024-09-11T07:41:02.719313+0000","last_undegraded":"2024-09-11T07:41:02.719313+0000","last_fullsized":"2024-09-11T07:41:02.719313+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"225'11","last_scrub_stamp":"2024-09-11T07:38:22.504199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:38:22.504199+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020188300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"324'10002","reported_seq":18403,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T07:41:05.207469+0000","last_change":"2024-09-11T07:41:02.781490+0000","last_active":"2024-09-11T07:41:05.207469+0000","last_peered":"2024-09-11T07:41:05.207469+0000","last_clean":"2024-09-11T07:41:05.207469+0000","last_became_active":"2024-09-11T07:41:02.697692+0000","last_became_peered":"2024-09-11T07:41:02.697692+0000","last_unstale":"2024-09-11T07:41:05.207469+0000","last_undegraded":"2024-09-11T07:41:05.207469+0000","last_fullsized":"2024-09-11T07:41:05.207469+0000","mapping_epoch":322,"log_start":"258'5317","ondisk_log_start":"258'5317","created":259,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4685,"log_dups_size":0,"ondisk_log_size":4685,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30801920,"num_objects":470,"num_object_clones":0,"num_object_copies":1410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7340,"num_write_kb":249920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2760,"num_bytes_recovered":180813824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":328,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.680583+0000","last_change":"2024-09-11T07:38:26.939048+0000","last_active":"2024-09-11T07:41:02.680583+0000","last_peered":"2024-09-11T07:41:02.680583+0000","last_clean":"2024-09-11T07:41:02.680583+0000","last_became_active":"2024-09-11T07:26:37.562501+0000","last_became_peered":"2024-09-11T07:26:37.562501+0000","last_unstale":"2024-09-11T07:41:02.680583+0000","last_undegraded":"2024-09-11T07:41:02.680583+0000","last_fullsized":"2024-09-11T07:41:02.680583+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:26.938987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:53.875300+0000","last_clean_scrub_stamp":"2024-09-11T07:38:26.938987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016308410000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":338,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.687322+0000","last_change":"2024-09-11T07:40:34.645930+0000","last_active":"2024-09-11T07:41:02.687322+0000","last_peered":"2024-09-11T07:41:02.687322+0000","last_clean":"2024-09-11T07:41:02.687322+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:41:02.687322+0000","last_undegraded":"2024-09-11T07:41:02.687322+0000","last_fullsized":"2024-09-11T07:41:02.687322+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:34.645645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:40:34.645645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:53.530666+0000","scrub_duration":0.0077905500000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"234'19","reported_seq":164,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.688039+0000","last_change":"2024-09-11T07:38:25.113379+0000","last_active":"2024-09-11T07:41:02.688039+0000","last_peered":"2024-09-11T07:41:02.688039+0000","last_clean":"2024-09-11T07:41:02.688039+0000","last_became_active":"2024-09-11T07:36:05.228746+0000","last_became_peered":"2024-09-11T07:36:05.228746+0000","last_unstale":"2024-09-11T07:41:02.688039+0000","last_undegraded":"2024-09-11T07:41:02.688039+0000","last_fullsized":"2024-09-11T07:41:02.688039+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:38:25.113300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:38:25.113300+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0063902960000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":338,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.683360+0000","last_change":"2024-09-11T07:37:21.150801+0000","last_active":"2024-09-11T07:41:02.683360+0000","last_peered":"2024-09-11T07:41:02.683360+0000","last_clean":"2024-09-11T07:41:02.683360+0000","last_became_active":"2024-09-11T07:32:00.507050+0000","last_became_peered":"2024-09-11T07:32:00.507050+0000","last_unstale":"2024-09-11T07:41:02.683360+0000","last_undegraded":"2024-09-11T07:41:02.683360+0000","last_fullsized":"2024-09-11T07:41:02.683360+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:21.150732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:24.575210+0000","last_clean_scrub_stamp":"2024-09-11T07:37:21.150732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001360691,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.11","version":"324'10218","reported_seq":10321,"reported_epoch":324,"state":"active+clean","last_fresh":"2024-09-11T07:41:04.414728+0000","last_change":"2024-09-11T07:38:29.349114+0000","last_active":"2024-09-11T07:41:04.414728+0000","last_peered":"2024-09-11T07:41:04.414728+0000","last_clean":"2024-09-11T07:41:04.414728+0000","last_became_active":"2024-09-11T07:38:29.346889+0000","last_became_peered":"2024-09-11T07:38:29.346889+0000","last_unstale":"2024-09-11T07:41:04.414728+0000","last_undegraded":"2024-09-11T07:41:04.414728+0000","last_fullsized":"2024-09-11T07:41:04.414728+0000","mapping_epoch":261,"log_start":"256'4190","ondisk_log_start":"256'4190","created":259,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6028,"log_dups_size":0,"ondisk_log_size":6028,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32178176,"num_objects":491,"num_object_clones":0,"num_object_copies":1473,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7541,"num_write_kb":257024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":332,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.719572+0000","last_change":"2024-09-11T07:39:28.550044+0000","last_active":"2024-09-11T07:41:02.719572+0000","last_peered":"2024-09-11T07:41:02.719572+0000","last_clean":"2024-09-11T07:41:02.719572+0000","last_became_active":"2024-09-11T07:26:37.862373+0000","last_became_peered":"2024-09-11T07:26:37.862373+0000","last_unstale":"2024-09-11T07:41:02.719572+0000","last_undegraded":"2024-09-11T07:41:02.719572+0000","last_fullsized":"2024-09-11T07:41:02.719572+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:28.549952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:39:28.549952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001250695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":270,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.718721+0000","last_change":"2024-09-11T07:39:54.632584+0000","last_active":"2024-09-11T07:41:02.718721+0000","last_peered":"2024-09-11T07:41:02.718721+0000","last_clean":"2024-09-11T07:41:02.718721+0000","last_became_active":"2024-09-11T07:36:05.224085+0000","last_became_peered":"2024-09-11T07:36:05.224085+0000","last_unstale":"2024-09-11T07:41:02.718721+0000","last_undegraded":"2024-09-11T07:41:02.718721+0000","last_fullsized":"2024-09-11T07:41:02.718721+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:54.632499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:39:54.632499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:06.525855+0000","scrub_duration":0.013616645,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"234'27","reported_seq":392,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.679035+0000","last_change":"2024-09-11T07:40:39.395515+0000","last_active":"2024-09-11T07:41:02.679035+0000","last_peered":"2024-09-11T07:41:02.679035+0000","last_clean":"2024-09-11T07:41:02.679035+0000","last_became_active":"2024-09-11T07:26:33.542544+0000","last_became_peered":"2024-09-11T07:26:33.542544+0000","last_unstale":"2024-09-11T07:41:02.679035+0000","last_undegraded":"2024-09-11T07:41:02.679035+0000","last_fullsized":"2024-09-11T07:41:02.679035+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'27","last_scrub_stamp":"2024-09-11T07:40:39.395458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:40:39.395458+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:04.483004+0000","scrub_duration":0.0018350700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":333,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.680821+0000","last_change":"2024-09-11T07:40:38.884375+0000","last_active":"2024-09-11T07:41:02.680821+0000","last_peered":"2024-09-11T07:41:02.680821+0000","last_clean":"2024-09-11T07:41:02.680821+0000","last_became_active":"2024-09-11T07:26:35.555862+0000","last_became_peered":"2024-09-11T07:26:35.555862+0000","last_unstale":"2024-09-11T07:41:02.680821+0000","last_undegraded":"2024-09-11T07:41:02.680821+0000","last_fullsized":"2024-09-11T07:41:02.680821+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:38.884278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:40:38.884278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:53.219499+0000","scrub_duration":0.0017591729999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"324'10200","reported_seq":10324,"reported_epoch":324,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-11T07:41:05.197845+0000","last_change":"2024-09-11T07:40:38.661172+0000","last_active":"2024-09-11T07:41:05.197845+0000","last_peered":"2024-09-11T07:41:05.197845+0000","last_clean":"2024-09-11T07:38:28.331892+0000","last_became_active":"2024-09-11T07:38:30.445788+0000","last_became_peered":"2024-09-11T07:38:30.445788+0000","last_unstale":"2024-09-11T07:41:05.197845+0000","last_undegraded":"2024-09-11T07:41:05.197845+0000","last_fullsized":"2024-09-11T07:41:05.197845+0000","mapping_epoch":262,"log_start":"265'5581","ondisk_log_start":"265'5581","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4619,"log_dups_size":0,"ondisk_log_size":4619,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28770304,"num_objects":439,"num_object_clones":0,"num_object_copies":1317,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1317,"num_objects_unfound":0,"num_objects_dirty":439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7481,"num_write_kb":253440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[1,2,6],"avail_no_missing":["1","2","6"],"object_location_counts":[{"shards":"1,2,6","objects":439}],"blocked_by":[],"up_primary":4,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":329,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.688461+0000","last_change":"2024-09-11T07:39:57.769864+0000","last_active":"2024-09-11T07:41:02.688461+0000","last_peered":"2024-09-11T07:41:02.688461+0000","last_clean":"2024-09-11T07:41:02.688461+0000","last_became_active":"2024-09-11T07:39:57.768994+0000","last_became_peered":"2024-09-11T07:39:57.768994+0000","last_unstale":"2024-09-11T07:41:02.688461+0000","last_undegraded":"2024-09-11T07:41:02.688461+0000","last_fullsized":"2024-09-11T07:41:02.688461+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:32.004125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:32.004125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019485520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":352,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.686984+0000","last_change":"2024-09-11T07:39:57.768623+0000","last_active":"2024-09-11T07:41:02.686984+0000","last_peered":"2024-09-11T07:41:02.686984+0000","last_clean":"2024-09-11T07:41:02.686984+0000","last_became_active":"2024-09-11T07:39:57.766915+0000","last_became_peered":"2024-09-11T07:39:57.766915+0000","last_unstale":"2024-09-11T07:41:02.686984+0000","last_undegraded":"2024-09-11T07:41:02.686984+0000","last_fullsized":"2024-09-11T07:41:02.686984+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:42.078765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:42.078765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0062011560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"234'23","reported_seq":360,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.689218+0000","last_change":"2024-09-11T07:37:43.078969+0000","last_active":"2024-09-11T07:41:02.689218+0000","last_peered":"2024-09-11T07:41:02.689218+0000","last_clean":"2024-09-11T07:41:02.689218+0000","last_became_active":"2024-09-11T07:26:33.542711+0000","last_became_peered":"2024-09-11T07:26:33.542711+0000","last_unstale":"2024-09-11T07:41:02.689218+0000","last_undegraded":"2024-09-11T07:41:02.689218+0000","last_fullsized":"2024-09-11T07:41:02.689218+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'23","last_scrub_stamp":"2024-09-11T07:37:43.078872+0000","last_deep_scrub":"127'17","last_deep_scrub_stamp":"2024-09-11T07:35:43.871918+0000","last_clean_scrub_stamp":"2024-09-11T07:37:43.078872+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051065670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":327,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-09-11T07:41:02.718749+0000","last_change":"2024-09-11T07:39:31.586959+0000","last_active":"2024-09-11T07:41:02.718749+0000","last_peered":"2024-09-11T07:41:02.718749+0000","last_clean":"2024-09-11T07:41:02.718749+0000","last_became_active":"2024-09-11T07:26:35.562037+0000","last_became_peered":"2024-09-11T07:26:35.562037+0000","last_unstale":"2024-09-11T07:41:02.718749+0000","last_undegraded":"2024-09-11T07:41:02.718749+0000","last_fullsized":"2024-09-11T07:41:02.718749+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:37.811009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:36:37.811009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039204729999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":26,"stat_sum":{"num_bytes":1061617688,"num_objects":16200,"num_object_clones":0,"num_object_copies":48600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3551,"num_objects_unfound":0,"num_objects_dirty":16200,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":236978,"num_write_kb":8101633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45568,"num_bytes_recovered":2986147864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4506013696,"data_stored":4505993336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130383,"ondisk_log_size":130383,"up":78,"acting":78,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1118,"num_read_kb":875,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":2916,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":765,"ondisk_log_size":765,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":865,"num_read_kb":860,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":147,"seq":631360192622,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1099648,"kb_used_data":779304,"kb_used_omap":4,"kb_used_meta":320315,"kb_avail":92645504,"statfs":{"total":95995035648,"available":94868996096,"internally_reserved":0,"allocated":798007296,"data_stored":797578635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":328003078},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:40:17 2024","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.375,"15min":2.6030000000000002},"min":{"1min":0.23100000000000001,"5min":0.191,"15min":0.191},"max":{"1min":4.609,"5min":14.086,"15min":49.573},"last":0.24299999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.488,"15min":2.508},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":4.069,"5min":16.841999999999999,"15min":49.881},"last":0.95899999999999996}]},{"osd":1,"last update":"Wed Sep 11 07:40:17 2024","interfaces":[{"interface":"back","average":{"1min":1.677,"5min":1.2050000000000001,"15min":0.84699999999999998},"min":{"1min":0.28299999999999997,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":5.0039999999999996,"5min":10.063000000000001,"15min":10.063000000000001},"last":1.24},{"interface":"front","average":{"1min":1.361,"5min":1.2789999999999999,"15min":0.96999999999999997},"min":{"1min":0.25600000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.5149999999999997,"5min":9.6059999999999999,"15min":9.6059999999999999},"last":0.76500000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:40:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":3.6030000000000002,"15min":1.8400000000000001},"min":{"1min":0.25700000000000001,"5min":0.249,"15min":0.22800000000000001},"max":{"1min":4.282,"5min":227.79300000000001,"15min":227.79300000000001},"last":0.63},{"interface":"front","average":{"1min":1.1779999999999999,"5min":3.7250000000000001,"15min":1.863},"min":{"1min":0.313,"5min":0.246,"15min":0.246},"max":{"1min":4.5529999999999999,"5min":227.22,"15min":227.22},"last":0.78800000000000003}]},{"osd":3,"last update":"Wed Sep 11 07:40:17 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.206,"15min":2.48},"min":{"1min":0.32800000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":4.6349999999999998,"5min":5.181,"15min":49.798999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":2.0680000000000001,"5min":1.339,"15min":2.4860000000000002},"min":{"1min":0.32100000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":7.3369999999999997,"5min":7.3369999999999997,"15min":49.524000000000001},"last":0.91100000000000003}]},{"osd":4,"last update":"Wed Sep 11 07:40:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":0.85799999999999998,"15min":0.58399999999999996},"min":{"1min":0.27300000000000002,"5min":0.25700000000000001,"15min":0.253},"max":{"1min":3.9399999999999999,"5min":6.0220000000000002,"15min":6.0220000000000002},"last":0.996},{"interface":"front","average":{"1min":1.5629999999999999,"5min":0.86699999999999999,"15min":0.69699999999999995},"min":{"1min":0.33700000000000002,"5min":0.247,"15min":0.247},"max":{"1min":6.9000000000000004,"5min":6.9000000000000004,"15min":6.9000000000000004},"last":1.196}]},{"osd":5,"last update":"Wed Sep 11 07:40:17 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.014,"15min":0.66700000000000004},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":5.2320000000000002,"5min":7.2160000000000002,"15min":7.2160000000000002},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.149,"5min":0.91100000000000003,"15min":0.75800000000000001},"min":{"1min":0.36299999999999999,"5min":0.27900000000000003,"15min":0.27600000000000002},"max":{"1min":4.4610000000000003,"5min":4.9969999999999999,"15min":4.9969999999999999},"last":0.60299999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:40:17 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":0.81999999999999995,"15min":0.626},"min":{"1min":0.312,"5min":0.31,"15min":0.29099999999999998},"max":{"1min":5.1909999999999998,"5min":5.1909999999999998,"15min":5.1909999999999998},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.091,"5min":0.92000000000000004,"15min":0.68100000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29099999999999998},"max":{"1min":4.2030000000000003,"5min":4.2030000000000003,"15min":4.2030000000000003},"last":0.97799999999999998}]}]},{"osd":6,"up_from":44,"seq":188978561207,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":730348,"kb_used_data":420508,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93014804,"statfs":{"total":95995035648,"available":95247159296,"internally_reserved":0,"allocated":430600192,"data_stored":430127360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:40:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":1.081,"15min":1.7989999999999999},"min":{"1min":0.223,"5min":0.218,"15min":0.192},"max":{"1min":1.405,"5min":20.675999999999998,"15min":142.29599999999999},"last":0.219},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.2709999999999999,"15min":1.964},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":3.754,"5min":20.663,"15min":142.68799999999999},"last":0.38600000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:40:18 2024","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.9990000000000001,"15min":1.5589999999999999},"min":{"1min":0.29399999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":4.7350000000000003,"5min":39.390000000000001,"15min":127.92100000000001},"last":0.246},{"interface":"front","average":{"1min":0.81200000000000006,"5min":1.853,"15min":1.5389999999999999},"min":{"1min":0.29099999999999998,"5min":0.26900000000000002,"15min":0.23000000000000001},"max":{"1min":4.6859999999999999,"5min":40.186999999999998,"15min":127.746},"last":0.316}]},{"osd":2,"last update":"Wed Sep 11 07:40:18 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.83599999999999997,"15min":0.86099999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":2.1699999999999999,"5min":3.6539999999999999,"15min":26.491},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":1.0169999999999999,"15min":1.048},"min":{"1min":0.313,"5min":0.26800000000000002,"15min":0.26400000000000001},"max":{"1min":2.6960000000000002,"5min":6.0819999999999999,"15min":26.561},"last":0.41199999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:40:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":2.1960000000000002,"15min":1.4550000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":4.0419999999999998,"5min":49.109000000000002,"15min":49.109000000000002},"last":0.28499999999999998},{"interface":"front","average":{"1min":1.8480000000000001,"5min":2.3210000000000002,"15min":1.571},"min":{"1min":0.29299999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":7.0659999999999998,"5min":48.963999999999999,"15min":48.963999999999999},"last":0.34999999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:40:18 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.72699999999999998,"15min":0.57699999999999996},"min":{"1min":0.23899999999999999,"5min":0.223,"15min":0.223},"max":{"1min":2.444,"5min":3.3599999999999999,"15min":4.4720000000000004},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.69999999999999996,"15min":0.52800000000000002},"min":{"1min":0.32100000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":2.597,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":0.39500000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:40:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.395,"15min":0.874},"min":{"1min":0.33500000000000002,"5min":0.30199999999999999,"15min":0.255},"max":{"1min":3.7189999999999999,"5min":57.436,"15min":57.436},"last":0.26400000000000001},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.4159999999999999,"15min":0.88700000000000001},"min":{"1min":0.28899999999999998,"5min":0.28599999999999998,"15min":0.251},"max":{"1min":8.5489999999999995,"5min":56.822000000000003,"15min":56.822000000000003},"last":0.373}]},{"osd":7,"last update":"Wed Sep 11 07:40:18 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":0.84199999999999997,"15min":0.71799999999999997},"min":{"1min":0.373,"5min":0.309,"15min":0.28999999999999998},"max":{"1min":5.4290000000000003,"5min":5.4290000000000003,"15min":5.4290000000000003},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.86199999999999999,"15min":0.754},"min":{"1min":0.308,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.2869999999999999,"5min":6.2869999999999999,"15min":6.2869999999999999},"last":0.40600000000000003}]}]},{"osd":5,"up_from":37,"seq":158913790141,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1308872,"kb_used_data":723072,"kb_used_omap":1,"kb_used_meta":585726,"kb_avail":92436280,"statfs":{"total":95995035648,"available":94654750720,"internally_reserved":0,"allocated":740425728,"data_stored":739980473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":599783768},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:40:55 2024","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.48,"15min":2.319},"min":{"1min":0.249,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":5.3559999999999999,"5min":25.808,"15min":185.68199999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":1.9810000000000001,"5min":1.502,"15min":2.4340000000000002},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":17.635999999999999,"5min":25.722999999999999,"15min":186.12200000000001},"last":0.42199999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:40:55 2024","interfaces":[{"interface":"back","average":{"1min":7.5149999999999997,"5min":2.7749999999999999,"15min":1.5740000000000001},"min":{"1min":0.24299999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":122.738,"5min":122.738,"15min":122.738},"last":0.48499999999999999},{"interface":"front","average":{"1min":7.7350000000000003,"5min":2.6320000000000001,"15min":1.5900000000000001},"min":{"1min":0.24099999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":122.66200000000001,"5min":122.66200000000001,"15min":122.66200000000001},"last":0.5}]},{"osd":2,"last update":"Wed Sep 11 07:40:55 2024","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.0600000000000001,"15min":1.6870000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.22700000000000001},"max":{"1min":5.5839999999999996,"5min":5.726,"15min":187.99199999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.498,"5min":1.2430000000000001,"15min":1.7010000000000001},"min":{"1min":0.29699999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":5.3029999999999999,"5min":13.016,"15min":186.375},"last":0.51500000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:40:55 2024","interfaces":[{"interface":"back","average":{"1min":8.6880000000000006,"5min":2.9529999999999998,"15min":2.899},"min":{"1min":0.30299999999999999,"5min":0.26800000000000002,"15min":0.23599999999999999},"max":{"1min":122.499,"5min":122.499,"15min":188.11500000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":8.8960000000000008,"5min":3.2549999999999999,"15min":2.9630000000000001},"min":{"1min":0.41999999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":122.209,"5min":122.209,"15min":186.44200000000001},"last":0.56299999999999994}]},{"osd":4,"last update":"Wed Sep 11 07:40:55 2024","interfaces":[{"interface":"back","average":{"1min":1.6379999999999999,"5min":0.996,"15min":0.65800000000000003},"min":{"1min":0.28100000000000003,"5min":0.24199999999999999,"15min":0.19900000000000001},"max":{"1min":6.3979999999999997,"5min":11.103,"15min":11.103},"last":0.53000000000000003},{"interface":"front","average":{"1min":1.758,"5min":1.0649999999999999,"15min":0.69199999999999995},"min":{"1min":0.26300000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":8.407,"5min":10.305,"15min":10.305},"last":0.65700000000000003}]},{"osd":6,"last update":"Wed Sep 11 07:40:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":0.873,"15min":0.624},"min":{"1min":0.32900000000000001,"5min":0.29199999999999998,"15min":0.26000000000000001},"max":{"1min":3.5019999999999998,"5min":5.6859999999999999,"15min":5.6859999999999999},"last":0.621},{"interface":"front","average":{"1min":1.1439999999999999,"5min":0.84899999999999998,"15min":0.62},"min":{"1min":0.28799999999999998,"5min":0.26800000000000002,"15min":0.25800000000000001},"max":{"1min":5.415,"5min":10.613,"15min":10.613},"last":0.58799999999999997}]},{"osd":7,"last update":"Wed Sep 11 07:40:15 2024","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.375,"15min":0.879},"min":{"1min":0.42299999999999999,"5min":0.30099999999999999,"15min":0.28000000000000003},"max":{"1min":5.9290000000000003,"5min":50.338999999999999,"15min":50.338999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.369,"15min":0.875},"min":{"1min":0.36199999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":3.5249999999999999,"5min":50.298999999999999,"15min":50.298999999999999},"last":0.64100000000000001}]}]},{"osd":4,"up_from":31,"seq":133143986369,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":729692,"kb_used_data":424012,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93015460,"statfs":{"total":95995035648,"available":95247831040,"internally_reserved":0,"allocated":434188288,"data_stored":433723915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:40:40 2024","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.042,"15min":1.222},"min":{"1min":0.22500000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":4.423,"5min":9.2330000000000005,"15min":35.308},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.89100000000000001,"15min":1.159},"min":{"1min":0.26900000000000002,"5min":0.222,"15min":0.20499999999999999},"max":{"1min":2.6379999999999999,"5min":4.5620000000000003,"15min":35.597999999999999},"last":0.443}]},{"osd":1,"last update":"Wed Sep 11 07:40:40 2024","interfaces":[{"interface":"back","average":{"1min":5.6479999999999997,"5min":2.2629999999999999,"15min":1.484},"min":{"1min":0.35699999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":73.447999999999993,"5min":73.447999999999993,"15min":73.447999999999993},"last":141.78399999999999},{"interface":"front","average":{"1min":5.492,"5min":2.3130000000000002,"15min":1.3460000000000001},"min":{"1min":0.26600000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":73.557000000000002,"5min":73.557000000000002,"15min":73.557000000000002},"last":145.65899999999999}]},{"osd":2,"last update":"Wed Sep 11 07:40:40 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":0.90000000000000002,"15min":0.97099999999999997},"min":{"1min":0.24099999999999999,"5min":0.23699999999999999,"15min":0.223},"max":{"1min":4.1550000000000002,"5min":4.1550000000000002,"15min":19.927},"last":0.34300000000000003},{"interface":"front","average":{"1min":1.2789999999999999,"5min":0.93899999999999995,"15min":0.98099999999999998},"min":{"1min":0.34100000000000003,"5min":0.28899999999999998,"15min":0.24399999999999999},"max":{"1min":4.0730000000000004,"5min":4.101,"15min":20.02},"last":0.36799999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:40:40 2024","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3140000000000001,"15min":1.2390000000000001},"min":{"1min":0.30499999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.0339999999999998,"5min":13.853,"15min":24.029},"last":0.45300000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.236,"15min":1.157},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":2.9020000000000001,"5min":13.651,"15min":24.265000000000001},"last":0.42899999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:40:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":0.80900000000000005,"15min":0.55800000000000005},"min":{"1min":0.224,"5min":0.223,"15min":0.21099999999999999},"max":{"1min":4.1299999999999999,"5min":4.6079999999999997,"15min":4.6079999999999997},"last":0.35599999999999998},{"interface":"front","average":{"1min":1.0660000000000001,"5min":0.82299999999999995,"15min":0.57899999999999996},"min":{"1min":0.28499999999999998,"5min":0.23300000000000001,"15min":0.22700000000000001},"max":{"1min":4.1840000000000002,"5min":5.4560000000000004,"15min":5.7320000000000002},"last":0.41099999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:40:30 2024","interfaces":[{"interface":"back","average":{"1min":1.9059999999999999,"5min":0.85499999999999998,"15min":0.63700000000000001},"min":{"1min":0.28399999999999997,"5min":0.255,"15min":0.20899999999999999},"max":{"1min":9.0579999999999998,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":1.133,"5min":0.77200000000000002,"15min":0.61399999999999999},"min":{"1min":0.33900000000000002,"5min":0.27500000000000002,"15min":0.25800000000000001},"max":{"1min":4.2140000000000004,"5min":4.2140000000000004,"15min":9.3770000000000007},"last":0.48799999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:40:16 2024","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":0.89400000000000002,"15min":0.67300000000000004},"min":{"1min":0.39200000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":8.0129999999999999,"5min":8.0129999999999999,"15min":8.0129999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":1.1919999999999999,"5min":0.80500000000000005,"15min":0.65100000000000002},"min":{"1min":0.312,"5min":0.29799999999999999,"15min":0.29599999999999999},"max":{"1min":3.274,"5min":3.274,"15min":3.274},"last":0.50800000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182598,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1653156,"kb_used_data":741784,"kb_used_omap":1,"kb_used_meta":911358,"kb_avail":92091996,"statfs":{"total":95995035648,"available":94302203904,"internally_reserved":0,"allocated":759586816,"data_stored":759116535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":933230936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:40:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.468,"15min":1.413},"min":{"1min":0.18099999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":9.0790000000000006,"5min":18.704000000000001,"15min":38.432000000000002},"last":2.2759999999999998},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.7230000000000001,"15min":1.8160000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.19500000000000001},"max":{"1min":11.895,"5min":14.749000000000001,"15min":91.876999999999995},"last":2.1720000000000002}]},{"osd":1,"last update":"Wed Sep 11 07:40:23 2024","interfaces":[{"interface":"back","average":{"1min":2.2309999999999999,"5min":3.1509999999999998,"15min":1.8959999999999999},"min":{"1min":0.28499999999999998,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":11.917999999999999,"5min":67.305000000000007,"15min":67.305000000000007},"last":0.36899999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":2.169,"15min":1.4279999999999999},"min":{"1min":0.27500000000000002,"5min":0.23799999999999999,"15min":0.192},"max":{"1min":6.9569999999999999,"5min":44.325000000000003,"15min":44.325000000000003},"last":2.2120000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:40:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.139,"15min":1.214},"min":{"1min":0.246,"5min":0.23699999999999999,"15min":0.23200000000000001},"max":{"1min":4.6799999999999997,"5min":6.6619999999999999,"15min":8.282},"last":2.0459999999999998},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.1619999999999999,"15min":1.147},"min":{"1min":0.29299999999999998,"5min":0.252,"15min":0.20999999999999999},"max":{"1min":5.992,"5min":6.5640000000000001,"15min":8.7110000000000003},"last":1.718}]},{"osd":4,"last update":"Wed Sep 11 07:40:43 2024","interfaces":[{"interface":"back","average":{"1min":11.355,"5min":3.04,"15min":1.746},"min":{"1min":0.33300000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":132.76400000000001,"5min":132.76400000000001,"15min":132.76400000000001},"last":1.746},{"interface":"front","average":{"1min":11.542,"5min":3.1960000000000002,"15min":1.9330000000000001},"min":{"1min":0.311,"5min":0.255,"15min":0.255},"max":{"1min":132.154,"5min":132.154,"15min":132.154},"last":1.431}]},{"osd":5,"last update":"Wed Sep 11 07:40:54 2024","interfaces":[{"interface":"back","average":{"1min":8.2970000000000006,"5min":2.52,"15min":1.635},"min":{"1min":0.41499999999999998,"5min":0.309,"15min":0.29599999999999999},"max":{"1min":137.50299999999999,"5min":137.50299999999999,"15min":137.50299999999999},"last":1.8939999999999999},{"interface":"front","average":{"1min":8.4939999999999998,"5min":2.6080000000000001,"15min":1.742},"min":{"1min":0.33800000000000002,"5min":0.28599999999999998,"15min":0.28299999999999997},"max":{"1min":137.45599999999999,"5min":137.45599999999999,"15min":137.45599999999999},"last":1.4630000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:40:23 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.131,"15min":1.2849999999999999},"min":{"1min":0.36199999999999999,"5min":0.313,"15min":0.27000000000000002},"max":{"1min":6.8680000000000003,"5min":6.8680000000000003,"15min":32.350999999999999},"last":2.0830000000000002},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.069,"15min":1.2410000000000001},"min":{"1min":0.34200000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":4.5199999999999996,"5min":4.5199999999999996,"15min":32.402999999999999},"last":1.776}]},{"osd":7,"last update":"Wed Sep 11 07:40:15 2024","interfaces":[{"interface":"back","average":{"1min":1.673,"5min":1.3580000000000001,"15min":1.345},"min":{"1min":0.35699999999999998,"5min":0.32300000000000001,"15min":0.308},"max":{"1min":11.864000000000001,"5min":11.864000000000001,"15min":11.864000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.2070000000000001,"15min":1.903},"min":{"1min":0.38700000000000001,"5min":0.36599999999999999,"15min":0.36099999999999999},"max":{"1min":7.0300000000000002,"5min":7.0300000000000002,"15min":32.289999999999999},"last":2.1520000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411531,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":569484,"kb_used_data":267964,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93175668,"statfs":{"total":95995035648,"available":95411884032,"internally_reserved":0,"allocated":274395136,"data_stored":273953289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:40:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.859,"15min":1.8169999999999999},"min":{"1min":0.193,"5min":0.17799999999999999,"15min":0.16400000000000001},"max":{"1min":4.9889999999999999,"5min":57.802999999999997,"15min":57.802999999999997},"last":0.27000000000000002},{"interface":"front","average":{"1min":1.7250000000000001,"5min":2.0289999999999999,"15min":1.919},"min":{"1min":0.23899999999999999,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":8.1899999999999995,"5min":57.759,"15min":57.759},"last":0.314}]},{"osd":1,"last update":"Wed Sep 11 07:40:57 2024","interfaces":[{"interface":"back","average":{"1min":1.6200000000000001,"5min":1.7150000000000001,"15min":1.5529999999999999},"min":{"1min":0.31,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":5.7789999999999999,"5min":48.042000000000002,"15min":65.153999999999996},"last":0.23599999999999999},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.9990000000000001,"15min":1.764},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.216},"max":{"1min":4.9630000000000001,"5min":50.201000000000001,"15min":63.311999999999998},"last":0.39200000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:40:23 2024","interfaces":[{"interface":"back","average":{"1min":4.2480000000000002,"5min":2.1909999999999998,"15min":1.7629999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":67.552999999999997,"5min":67.552999999999997,"15min":67.552999999999997},"last":0.26200000000000001},{"interface":"front","average":{"1min":4.3449999999999998,"5min":2.0139999999999998,"15min":1.7589999999999999},"min":{"1min":0.23999999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":67.352999999999994,"5min":67.352999999999994,"15min":67.352999999999994},"last":0.37}]},{"osd":4,"last update":"Wed Sep 11 07:40:41 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.79500000000000004,"15min":1.083},"min":{"1min":0.315,"5min":0.30599999999999999,"15min":0.27500000000000002},"max":{"1min":2.4790000000000001,"5min":3.254,"15min":30.946999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.81200000000000006,"15min":0.98099999999999998},"min":{"1min":0.33900000000000002,"5min":0.313,"15min":0.29999999999999999},"max":{"1min":2.7429999999999999,"5min":3.1669999999999998,"15min":22.434000000000001},"last":0.443}]},{"osd":5,"last update":"Wed Sep 11 07:40:57 2024","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":0.997,"15min":1.141},"min":{"1min":0.376,"5min":0.32100000000000001,"15min":0.31900000000000001},"max":{"1min":3.319,"5min":3.3290000000000002,"15min":31.789000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.405,"5min":1.0620000000000001,"15min":1.2090000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.31},"max":{"1min":8.1690000000000005,"5min":8.1690000000000005,"15min":30.898},"last":0.42499999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:40:23 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.89000000000000001,"15min":1.0820000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":6.891,"5min":6.891,"15min":22.399000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.875,"5min":0.92000000000000004,"15min":1.0860000000000001},"min":{"1min":0.377,"5min":0.311,"15min":0.29999999999999999},"max":{"1min":2.2400000000000002,"5min":4.1230000000000002,"15min":25.739000000000001},"last":0.45700000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:40:11 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.82299999999999995,"15min":1.6359999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.1349999999999998,"5min":2.6600000000000001,"15min":22.484000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.107,"15min":1.962},"min":{"1min":0.46000000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.9809999999999999,"5min":4.1719999999999997,"15min":25.670000000000002},"last":0.49299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575054,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1598488,"kb_used_data":694160,"kb_used_omap":1,"kb_used_meta":904318,"kb_avail":92146664,"statfs":{"total":95995035648,"available":94358183936,"internally_reserved":0,"allocated":710819840,"data_stored":710358583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":926021976},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:40:22 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.6779999999999999,"15min":1.921},"min":{"1min":0.185,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":3.944,"5min":9.2669999999999995,"15min":82.218999999999994},"last":2.2250000000000001},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.7869999999999999,"15min":1.956},"min":{"1min":0.255,"5min":0.186,"15min":0.186},"max":{"1min":5.0789999999999997,"5min":8.0739999999999998,"15min":82.278999999999996},"last":0.63300000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:40:58 2024","interfaces":[{"interface":"back","average":{"1min":2.141,"5min":1.9019999999999999,"15min":1.4870000000000001},"min":{"1min":0.251,"5min":0.22,"15min":0.19800000000000001},"max":{"1min":5.9240000000000004,"5min":9.0969999999999995,"15min":9.0969999999999995},"last":0.56999999999999995},{"interface":"front","average":{"1min":1.6619999999999999,"5min":1.5920000000000001,"15min":1.516},"min":{"1min":0.27700000000000002,"5min":0.191,"15min":0.191},"max":{"1min":5.4400000000000004,"5min":20.518000000000001,"15min":20.518000000000001},"last":0.46600000000000003}]},{"osd":3,"last update":"Wed Sep 11 07:40:19 2024","interfaces":[{"interface":"back","average":{"1min":1.796,"5min":2.7080000000000002,"15min":2.2999999999999998},"min":{"1min":0.33000000000000002,"5min":0.27600000000000002,"15min":0.25800000000000001},"max":{"1min":4.774,"5min":67.120999999999995,"15min":81.882999999999996},"last":0.51400000000000001},{"interface":"front","average":{"1min":1.1200000000000001,"5min":2.6080000000000001,"15min":2.2400000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.20999999999999999},"max":{"1min":4.3639999999999999,"5min":66.888999999999996,"15min":82.400000000000006},"last":2.169}]},{"osd":4,"last update":"Wed Sep 11 07:40:33 2024","interfaces":[{"interface":"back","average":{"1min":2.4940000000000002,"5min":1.8620000000000001,"15min":1.2290000000000001},"min":{"1min":0.30299999999999999,"5min":0.27500000000000002,"15min":0.27400000000000002},"max":{"1min":7.8390000000000004,"5min":7.8390000000000004,"15min":7.8390000000000004},"last":0.60699999999999998},{"interface":"front","average":{"1min":2.2450000000000001,"5min":1.7729999999999999,"15min":1.3360000000000001},"min":{"1min":0.38700000000000001,"5min":0.29999999999999999,"15min":0.26500000000000001},"max":{"1min":7.8239999999999998,"5min":9.2289999999999992,"15min":9.2289999999999992},"last":2.367}]},{"osd":5,"last update":"Wed Sep 11 07:40:58 2024","interfaces":[{"interface":"back","average":{"1min":2.6459999999999999,"5min":1.954,"15min":1.333},"min":{"1min":0.29599999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":8.3680000000000003,"5min":8.3680000000000003,"15min":8.3680000000000003},"last":0.68999999999999995},{"interface":"front","average":{"1min":2.681,"5min":1.869,"15min":1.419},"min":{"1min":0.35099999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":8.6809999999999992,"5min":9.1929999999999996,"15min":9.1929999999999996},"last":0.54000000000000004}]},{"osd":6,"last update":"Wed Sep 11 07:40:19 2024","interfaces":[{"interface":"back","average":{"1min":2.0219999999999998,"5min":1.6639999999999999,"15min":1.3520000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.30099999999999999},"max":{"1min":9.1850000000000005,"5min":9.2569999999999997,"15min":24.018000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.3540000000000001,"15min":1.141},"min":{"1min":0.318,"5min":0.313,"15min":0.29999999999999999},"max":{"1min":4.968,"5min":9.3339999999999996,"15min":23.681000000000001},"last":2.2010000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:40:19 2024","interfaces":[{"interface":"back","average":{"1min":2.6000000000000001,"5min":1.6659999999999999,"15min":1.417},"min":{"1min":0.36099999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":9.0489999999999995,"5min":9.3629999999999995,"15min":9.3629999999999995},"last":2.3260000000000001},{"interface":"front","average":{"1min":2.9340000000000002,"5min":2.0670000000000002,"15min":1.663},"min":{"1min":0.46300000000000002,"5min":0.36899999999999999,"15min":0.313},"max":{"1min":8.2219999999999995,"5min":8.2219999999999995,"15min":8.2219999999999995},"last":1.1140000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738580,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1076444,"kb_used_data":486540,"kb_used_omap":1,"kb_used_meta":589886,"kb_avail":92668708,"statfs":{"total":95995035648,"available":94892756992,"internally_reserved":0,"allocated":498216960,"data_stored":497759368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":604043608},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,16],"upper_bound":2048},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:40:27 2024","interfaces":[{"interface":"back","average":{"1min":2.9449999999999998,"5min":1.6479999999999999,"15min":1.274},"min":{"1min":0.24399999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":15.522,"5min":15.522,"15min":15.522},"last":0.34100000000000003},{"interface":"front","average":{"1min":2.222,"5min":1.72,"15min":1.3009999999999999},"min":{"1min":0.254,"5min":0.20499999999999999,"15min":0.16600000000000001},"max":{"1min":13.988,"5min":13.988,"15min":13.988},"last":0.39600000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:40:53 2024","interfaces":[{"interface":"back","average":{"1min":2.2160000000000002,"5min":3.472,"15min":2.0950000000000002},"min":{"1min":0.23499999999999999,"5min":0.188,"15min":0.188},"max":{"1min":8.8439999999999994,"5min":200.339,"15min":200.339},"last":0.51700000000000002},{"interface":"front","average":{"1min":2.0489999999999999,"5min":3.3620000000000001,"15min":2.0019999999999998},"min":{"1min":0.251,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":15.481,"5min":200.40100000000001,"15min":200.40100000000001},"last":0.377}]},{"osd":3,"last update":"Wed Sep 11 07:40:27 2024","interfaces":[{"interface":"back","average":{"1min":1.99,"5min":1.6619999999999999,"15min":1.9550000000000001},"min":{"1min":0.23300000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":8.8070000000000004,"5min":9.4920000000000009,"15min":45.185000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":2.1789999999999998,"5min":1.6319999999999999,"15min":1.746},"min":{"1min":0.251,"5min":0.24299999999999999,"15min":0.22800000000000001},"max":{"1min":15.252000000000001,"5min":15.252000000000001,"15min":43.228999999999999},"last":0.44600000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:40:36 2024","interfaces":[{"interface":"back","average":{"1min":3.0369999999999999,"5min":1.673,"15min":1.2490000000000001},"min":{"1min":0.311,"5min":0.26800000000000002,"15min":0.26700000000000002},"max":{"1min":30.315000000000001,"5min":30.315000000000001,"15min":30.315000000000001},"last":0.497},{"interface":"front","average":{"1min":3.3159999999999998,"5min":1.6919999999999999,"15min":1.3899999999999999},"min":{"1min":0.36899999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":30.363,"5min":30.363,"15min":30.363},"last":0.629}]},{"osd":5,"last update":"Wed Sep 11 07:40:53 2024","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.359,"15min":1.2749999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":3.9790000000000001,"5min":5.0830000000000002,"15min":19.381},"last":0.53400000000000003},{"interface":"front","average":{"1min":2.2000000000000002,"5min":1.4970000000000001,"15min":1.405},"min":{"1min":0.38700000000000001,"5min":0.36899999999999999,"15min":0.27200000000000002},"max":{"1min":7.202,"5min":7.202,"15min":19.437000000000001},"last":0.57299999999999995}]},{"osd":6,"last update":"Wed Sep 11 07:40:27 2024","interfaces":[{"interface":"back","average":{"1min":1.76,"5min":1.353,"15min":1.331},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.316},"max":{"1min":5.5010000000000003,"5min":9.5990000000000002,"15min":23.190999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.865,"5min":1.198,"15min":1.306},"min":{"1min":0.375,"5min":0.35299999999999998,"15min":0.34200000000000003},"max":{"1min":15.404,"5min":15.404,"15min":23.632000000000001},"last":0.55800000000000005}]},{"osd":7,"last update":"Wed Sep 11 07:40:16 2024","interfaces":[{"interface":"back","average":{"1min":2.9039999999999999,"5min":1.7609999999999999,"15min":1.7030000000000001},"min":{"1min":0.47999999999999998,"5min":0.373,"15min":0.373},"max":{"1min":30.43,"5min":30.43,"15min":30.43},"last":0.65300000000000002},{"interface":"front","average":{"1min":3.3410000000000002,"5min":1.7649999999999999,"15min":1.7450000000000001},"min":{"1min":0.36199999999999999,"5min":0.315,"15min":0.315},"max":{"1min":31.065000000000001,"5min":31.065000000000001,"15min":31.065000000000001},"last":0.59399999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":480641024,"data_stored":480641024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":705040384,"data_stored":705036312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270401536,"data_stored":270401536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":724893696,"data_stored":724893696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":416354304,"data_stored":416350232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":733810688,"data_stored":733806616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":416944128,"data_stored":416940056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":757927936,"data_stored":757923864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:41:09.735 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj -9, currently 1 2024-09-11T07:41:09.760 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:09.781 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:41:09.791 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:41:09.830 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:09.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:09.832 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:09.933 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:09.966 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:09.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:09.968 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:10.069 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:10.108 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:41:09 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:41:10.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:10.109 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:10.109 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:10.210 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:10.216 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:10 smithi022 ceph-mon[29462]: from='client.15720 v1:172.21.15.22:0/1234957704' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:41:10.217 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:10 smithi022 ceph-mon[37330]: from='client.15720 v1:172.21.15.22:0/1234957704' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:41:10.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:10.242 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:10.242 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:10.342 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:10.390 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:10.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:10.391 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:10.467 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:41:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:41:10] "GET /metrics HTTP/1.1" 200 224766 "" "Prometheus/2.43.0" 2024-09-11T07:41:10.492 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:10.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:10 smithi104 ceph-mon[33507]: from='client.15720 v1:172.21.15.22:0/1234957704' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:41:10.532 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:10.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:10.533 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:10.634 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:10.665 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:10.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:10.666 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:10.767 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:10.804 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:10.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:10.805 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:10.906 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:10.942 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:10.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:10.943 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:10.961 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:41:11.044 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:11.096 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:11.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:11.097 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:11.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:11 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:41:11.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:11 smithi022 ceph-mon[29462]: pgmap v676: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 769 MiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 821 op/s; 2779/37455 objects misplaced (7.420%); 12 MiB/s, 185 objects/s recovering 2024-09-11T07:41:11.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:11 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:41:11.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:11 smithi022 ceph-mon[37330]: pgmap v676: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 769 MiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 821 op/s; 2779/37455 objects misplaced (7.420%); 12 MiB/s, 185 objects/s recovering 2024-09-11T07:41:11.198 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:11.241 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:11.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:11.242 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:11.343 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:11.370 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:11.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:11.371 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:11.471 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:11.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:11.506 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:11.506 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:11.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:11 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:41:11.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:11 smithi104 ceph-mon[33507]: pgmap v676: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 769 MiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 821 op/s; 2779/37455 objects misplaced (7.420%); 12 MiB/s, 185 objects/s recovering 2024-09-11T07:41:11.607 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:11.640 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:11.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:11.641 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:11.744 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:11.776 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:11.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:11.777 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:11.878 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:11.910 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:11.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:11.911 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:12.012 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:12.048 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:12.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:12.049 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:12.150 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:12.182 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:12.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:12.183 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:12.284 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:12.356 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:12.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:12.357 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:12.458 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:12.486 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:12.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:12.487 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:12.587 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:12.648 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:12.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:12.650 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:12.750 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:12.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:12.788 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:12.788 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:12.889 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:12.923 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:12.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:12.925 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:13.025 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:13.052 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:13.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:13.053 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:13.102 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:41:13.154 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:13.161 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:41:13.161 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:41:13.208 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:13.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:13.209 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:13.310 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:13.341 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:13.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:13.342 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:13.443 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:13.483 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:13.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:13.484 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:13.580 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:13 smithi022 ceph-mon[29462]: pgmap v677: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 623 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 930 op/s; 2227/30447 objects misplaced (7.314%); 9.6 MiB/s, 154 objects/s recovering 2024-09-11T07:41:13.581 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:13 smithi022 ceph-mon[37330]: pgmap v677: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 623 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 930 op/s; 2227/30447 objects misplaced (7.314%); 9.6 MiB/s, 154 objects/s recovering 2024-09-11T07:41:13.584 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:13.612 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:13.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:13.613 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:13.714 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:13.759 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:13.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:13.760 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:13.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:13 smithi104 ceph-mon[33507]: pgmap v677: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 623 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 930 op/s; 2227/30447 objects misplaced (7.314%); 9.6 MiB/s, 154 objects/s recovering 2024-09-11T07:41:13.861 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:13.900 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:13.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:13.901 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:14.002 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:14.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:14.045 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:14.046 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:14.146 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:14.183 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:14.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:14.184 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:14.285 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:14.328 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:14.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:14.329 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:14.337 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:41:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:41:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:41:14.343 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is unset 2024-09-11T07:41:14.431 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:14.459 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:14.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:14.460 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:14.561 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:14.581 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:14.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:14.582 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:14.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:14 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:41:14.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:14 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:41:14.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:14 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3155935823' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:41:14.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:14 smithi022 ceph-mon[29462]: from='client.25609 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:41:14.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:14 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:41:14.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:14 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:41:14.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:14 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3155935823' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:41:14.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:14 smithi022 ceph-mon[37330]: from='client.25609 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:41:14.683 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:14.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:14.731 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:14.731 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:14.736 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:41:14.736 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:41:14.737 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_2 2024-09-11T07:41:14.737 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:41:14.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:14 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:41:14.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:14 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:41:14.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:14 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3155935823' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:41:14.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:14 smithi104 ceph-mon[33507]: from='client.25609 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:41:14.832 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:14.867 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:14.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:14.868 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:14.969 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:15.004 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:15.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:15.005 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:15.105 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:15.143 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:15.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:15.144 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:15.245 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:15.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:15.276 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:15.276 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:15.377 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:15.424 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:15.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:15.425 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:15.526 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:15.559 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:15.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:15.560 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:15.661 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:15.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:15 smithi022 ceph-mon[29462]: pgmap v678: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 573 MiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 967 B/s rd, 0 B/s wr, 917 op/s; 1827/28068 objects misplaced (6.509%); 8.7 MiB/s, 139 objects/s recovering 2024-09-11T07:41:15.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:15 smithi022 ceph-mon[29462]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:41:15.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:15 smithi022 ceph-mon[29462]: from='client.25609 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:41:15.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:15 smithi022 ceph-mon[29462]: osdmap e325: 8 total, 8 up, 8 in 2024-09-11T07:41:15.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:15 smithi022 ceph-mon[29462]: 6.1 scrub starts 2024-09-11T07:41:15.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:15 smithi022 ceph-mon[29462]: 6.1 scrub ok 2024-09-11T07:41:15.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:15 smithi022 ceph-mon[37330]: pgmap v678: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 573 MiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 967 B/s rd, 0 B/s wr, 917 op/s; 1827/28068 objects misplaced (6.509%); 8.7 MiB/s, 139 objects/s recovering 2024-09-11T07:41:15.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:15 smithi022 ceph-mon[37330]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:41:15.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:15 smithi022 ceph-mon[37330]: from='client.25609 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:41:15.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:15 smithi022 ceph-mon[37330]: osdmap e325: 8 total, 8 up, 8 in 2024-09-11T07:41:15.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:15 smithi022 ceph-mon[37330]: 6.1 scrub starts 2024-09-11T07:41:15.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:15 smithi022 ceph-mon[37330]: 6.1 scrub ok 2024-09-11T07:41:15.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:15.700 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:15.700 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:15.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:15 smithi104 ceph-mon[33507]: pgmap v678: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 573 MiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 967 B/s rd, 0 B/s wr, 917 op/s; 1827/28068 objects misplaced (6.509%); 8.7 MiB/s, 139 objects/s recovering 2024-09-11T07:41:15.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:15 smithi104 ceph-mon[33507]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:41:15.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:15 smithi104 ceph-mon[33507]: from='client.25609 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:41:15.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:15 smithi104 ceph-mon[33507]: osdmap e325: 8 total, 8 up, 8 in 2024-09-11T07:41:15.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:15 smithi104 ceph-mon[33507]: 6.1 scrub starts 2024-09-11T07:41:15.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:15 smithi104 ceph-mon[33507]: 6.1 scrub ok 2024-09-11T07:41:15.801 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:15.828 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:15.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:15.829 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:15.930 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:15.965 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:15.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:15.967 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:16.068 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:16.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:16.105 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:16.106 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:16.206 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:16.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:16.250 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:16.250 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:16.351 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:16.390 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:16.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:16.391 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:16.492 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:16.529 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:16.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:16.530 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:16.630 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:16.665 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:16.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:16.666 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:16.767 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:16.806 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:16.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:16.807 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:16.908 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:16.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:16 smithi022 ceph-mon[29462]: 5.13 scrub starts 2024-09-11T07:41:16.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:16 smithi022 ceph-mon[29462]: 5.13 scrub ok 2024-09-11T07:41:16.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:16 smithi022 ceph-mon[29462]: 5.1a scrub starts 2024-09-11T07:41:16.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:16 smithi022 ceph-mon[29462]: 5.1a scrub ok 2024-09-11T07:41:16.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:16 smithi022 ceph-mon[29462]: osdmap e326: 8 total, 8 up, 8 in 2024-09-11T07:41:16.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:16 smithi022 ceph-mon[29462]: pgmap v681: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 232 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.56k op/s; 803/11694 objects misplaced (6.867%) 2024-09-11T07:41:16.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:16 smithi022 ceph-mon[37330]: 5.13 scrub starts 2024-09-11T07:41:16.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:16 smithi022 ceph-mon[37330]: 5.13 scrub ok 2024-09-11T07:41:16.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:16 smithi022 ceph-mon[37330]: 5.1a scrub starts 2024-09-11T07:41:16.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:16 smithi022 ceph-mon[37330]: 5.1a scrub ok 2024-09-11T07:41:16.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:16 smithi022 ceph-mon[37330]: osdmap e326: 8 total, 8 up, 8 in 2024-09-11T07:41:16.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:16 smithi022 ceph-mon[37330]: pgmap v681: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 232 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.56k op/s; 803/11694 objects misplaced (6.867%) 2024-09-11T07:41:16.929 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:16.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:16.930 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:16 smithi104 ceph-mon[33507]: 5.13 scrub starts 2024-09-11T07:41:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:16 smithi104 ceph-mon[33507]: 5.13 scrub ok 2024-09-11T07:41:17.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:16 smithi104 ceph-mon[33507]: 5.1a scrub starts 2024-09-11T07:41:17.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:16 smithi104 ceph-mon[33507]: 5.1a scrub ok 2024-09-11T07:41:17.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:16 smithi104 ceph-mon[33507]: osdmap e326: 8 total, 8 up, 8 in 2024-09-11T07:41:17.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:16 smithi104 ceph-mon[33507]: pgmap v681: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 232 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.56k op/s; 803/11694 objects misplaced (6.867%) 2024-09-11T07:41:17.031 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:17.071 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:17.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:17.072 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:17.173 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:17.207 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:17.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:17.208 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:17.309 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:17.340 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:17.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:17.341 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:17.441 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:17.464 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:17.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:17.465 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:17.477 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-11T07:41:17.566 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:17.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:17.613 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:17.613 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:17.714 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:17.739 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:17.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:17.740 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:17.841 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:17.875 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:17.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:17.876 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:17.977 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:17.985 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Removed 126588 objects 2024-09-11T07:41:17.985 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Clean up completed and total clean up time :129.493 2024-09-11T07:41:17.989 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:17 smithi022 ceph-mon[29462]: 4.12 scrub starts 2024-09-11T07:41:17.989 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:17 smithi022 ceph-mon[29462]: 4.12 scrub ok 2024-09-11T07:41:17.989 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:17 smithi022 ceph-mon[29462]: 3.2 scrub starts 2024-09-11T07:41:17.989 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:17 smithi022 ceph-mon[29462]: 3.2 scrub ok 2024-09-11T07:41:17.989 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:17 smithi022 ceph-mon[29462]: 2.f scrub starts 2024-09-11T07:41:17.989 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:17 smithi022 ceph-mon[29462]: 2.f scrub ok 2024-09-11T07:41:17.989 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:17 smithi022 ceph-mon[37330]: 4.12 scrub starts 2024-09-11T07:41:17.989 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:17 smithi022 ceph-mon[37330]: 4.12 scrub ok 2024-09-11T07:41:17.989 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:17 smithi022 ceph-mon[37330]: 3.2 scrub starts 2024-09-11T07:41:17.989 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:17 smithi022 ceph-mon[37330]: 3.2 scrub ok 2024-09-11T07:41:17.990 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:17 smithi022 ceph-mon[37330]: 2.f scrub starts 2024-09-11T07:41:17.990 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:17 smithi022 ceph-mon[37330]: 2.f scrub ok 2024-09-11T07:41:18.005 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T07:41:18.006 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:18.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:18.007 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:18.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:17 smithi104 ceph-mon[33507]: 4.12 scrub starts 2024-09-11T07:41:18.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:17 smithi104 ceph-mon[33507]: 4.12 scrub ok 2024-09-11T07:41:18.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:17 smithi104 ceph-mon[33507]: 3.2 scrub starts 2024-09-11T07:41:18.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:17 smithi104 ceph-mon[33507]: 3.2 scrub ok 2024-09-11T07:41:18.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:17 smithi104 ceph-mon[33507]: 2.f scrub starts 2024-09-11T07:41:18.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:17 smithi104 ceph-mon[33507]: 2.f scrub ok 2024-09-11T07:41:18.033 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:41:18.034 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:41:18.034 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:41:18.065 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:41:18.065 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:41:18.065 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:41:18.082 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:41:18.082 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:41:18.099 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:41:18.099 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:41:18.108 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:18.115 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:41:18.115 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:41:18.129 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:18.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:18.130 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:18.131 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:41:18.131 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:41:18.147 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:41:18.147 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:41:18.163 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:41:18.163 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:41:18.231 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:18.253 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:18.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:18.254 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:18.355 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:18.377 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:18.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:18.378 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:18.478 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:18.500 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:18.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:18.501 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:18.602 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:18.626 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:18.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:18.627 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:18.728 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:18.750 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:18.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:18.751 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:18.852 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:18.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:18.874 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:18.874 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:18.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:18 smithi022 ceph-mon[29462]: 2.18 scrub starts 2024-09-11T07:41:18.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:18 smithi022 ceph-mon[29462]: 2.18 scrub ok 2024-09-11T07:41:18.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:18 smithi022 ceph-mon[29462]: 9.4 scrub starts 2024-09-11T07:41:18.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:18 smithi022 ceph-mon[29462]: 9.4 scrub ok 2024-09-11T07:41:18.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:18 smithi022 ceph-mon[29462]: 2.13 scrub starts 2024-09-11T07:41:18.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:18 smithi022 ceph-mon[29462]: 2.13 scrub ok 2024-09-11T07:41:18.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:18 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4002109768' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:41:18.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:18 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:41:18.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:18 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:41:18.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:18 smithi022 ceph-mon[29462]: pgmap v682: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 232 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.07k op/s; 803/11694 objects misplaced (6.867%) 2024-09-11T07:41:18.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:18 smithi022 ceph-mon[37330]: 2.18 scrub starts 2024-09-11T07:41:18.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:18 smithi022 ceph-mon[37330]: 2.18 scrub ok 2024-09-11T07:41:18.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:18 smithi022 ceph-mon[37330]: 9.4 scrub starts 2024-09-11T07:41:18.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:18 smithi022 ceph-mon[37330]: 9.4 scrub ok 2024-09-11T07:41:18.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:18 smithi022 ceph-mon[37330]: 2.13 scrub starts 2024-09-11T07:41:18.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:18 smithi022 ceph-mon[37330]: 2.13 scrub ok 2024-09-11T07:41:18.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:18 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4002109768' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:41:18.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:18 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:41:18.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:18 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:41:18.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:18 smithi022 ceph-mon[37330]: pgmap v682: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 232 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.07k op/s; 803/11694 objects misplaced (6.867%) 2024-09-11T07:41:18.958 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:41:18.976 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:18.998 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:18.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:18.999 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:19.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:18 smithi104 ceph-mon[33507]: 2.18 scrub starts 2024-09-11T07:41:19.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:18 smithi104 ceph-mon[33507]: 2.18 scrub ok 2024-09-11T07:41:19.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:18 smithi104 ceph-mon[33507]: 9.4 scrub starts 2024-09-11T07:41:19.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:18 smithi104 ceph-mon[33507]: 9.4 scrub ok 2024-09-11T07:41:19.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:18 smithi104 ceph-mon[33507]: 2.13 scrub starts 2024-09-11T07:41:19.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:18 smithi104 ceph-mon[33507]: 2.13 scrub ok 2024-09-11T07:41:19.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:18 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4002109768' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:41:19.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:18 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:41:19.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:18 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:41:19.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:18 smithi104 ceph-mon[33507]: pgmap v682: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 232 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.07k op/s; 803/11694 objects misplaced (6.867%) 2024-09-11T07:41:19.100 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:19.122 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:19.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:19.123 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:19.224 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:19.246 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:19.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:19.247 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:19.348 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:19.370 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:19.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:19.371 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:19.472 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:19.495 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:19.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:19.496 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:19.597 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:19.618 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:19.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:19.619 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:19.719 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:19.741 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:19.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:19.742 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:19.843 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:19.844 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:19 smithi022 ceph-mon[29462]: 6.a scrub starts 2024-09-11T07:41:19.844 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:19 smithi022 ceph-mon[29462]: 6.a scrub ok 2024-09-11T07:41:19.844 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:19 smithi022 ceph-mon[29462]: 2.8 scrub starts 2024-09-11T07:41:19.844 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:19 smithi022 ceph-mon[29462]: 2.8 scrub ok 2024-09-11T07:41:19.844 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:19 smithi022 ceph-mon[29462]: osdmap e327: 8 total, 8 up, 8 in 2024-09-11T07:41:19.844 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:19 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:41:19.845 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:19 smithi022 ceph-mon[37330]: 6.a scrub starts 2024-09-11T07:41:19.845 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:19 smithi022 ceph-mon[37330]: 6.a scrub ok 2024-09-11T07:41:19.845 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:19 smithi022 ceph-mon[37330]: 2.8 scrub starts 2024-09-11T07:41:19.845 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:19 smithi022 ceph-mon[37330]: 2.8 scrub ok 2024-09-11T07:41:19.845 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:19 smithi022 ceph-mon[37330]: osdmap e327: 8 total, 8 up, 8 in 2024-09-11T07:41:19.845 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:19 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:41:19.868 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:19.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:19.869 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:19.970 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:19.993 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:19.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:19.994 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:20.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:41:19 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:41:20.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:19 smithi104 ceph-mon[33507]: 6.a scrub starts 2024-09-11T07:41:20.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:19 smithi104 ceph-mon[33507]: 6.a scrub ok 2024-09-11T07:41:20.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:19 smithi104 ceph-mon[33507]: 2.8 scrub starts 2024-09-11T07:41:20.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:19 smithi104 ceph-mon[33507]: 2.8 scrub ok 2024-09-11T07:41:20.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:19 smithi104 ceph-mon[33507]: osdmap e327: 8 total, 8 up, 8 in 2024-09-11T07:41:20.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:19 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:41:20.094 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:20.116 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:20.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:20.117 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:20.218 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:20.240 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:20.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:20.241 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:20.344 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:20.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:20.366 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:20.366 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:20.406 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:41:20 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:41:20] "GET /metrics HTTP/1.1" 200 224766 "" "Prometheus/2.43.0" 2024-09-11T07:41:20.467 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:20.490 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:20.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:20.491 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:20.591 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:20.613 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:20.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:20.614 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:20.715 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:20.737 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:20.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:20.738 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:20.839 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:20.860 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:20.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:20.861 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:20.962 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:20.983 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:20.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:20.984 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:21.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:20 smithi104 ceph-mon[33507]: 5.1c scrub starts 2024-09-11T07:41:21.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:20 smithi104 ceph-mon[33507]: 5.1c scrub ok 2024-09-11T07:41:21.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:20 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:41:21.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:20 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:41:21.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:20 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:41:21.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:20 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:41:21.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:20 smithi104 ceph-mon[33507]: pgmap v684: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 80 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.33k op/s; 279/4422 objects misplaced (6.309%) 2024-09-11T07:41:21.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:20 smithi104 ceph-mon[33507]: osdmap e328: 8 total, 8 up, 8 in 2024-09-11T07:41:21.085 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:21.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:21.109 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:21.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:20 smithi022 ceph-mon[37330]: 5.1c scrub starts 2024-09-11T07:41:21.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:20 smithi022 ceph-mon[37330]: 5.1c scrub ok 2024-09-11T07:41:21.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:20 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:41:21.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:20 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:41:21.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:20 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:41:21.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:20 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:41:21.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:20 smithi022 ceph-mon[37330]: pgmap v684: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 80 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.33k op/s; 279/4422 objects misplaced (6.309%) 2024-09-11T07:41:21.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:20 smithi022 ceph-mon[37330]: osdmap e328: 8 total, 8 up, 8 in 2024-09-11T07:41:21.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:20 smithi022 ceph-mon[29462]: 5.1c scrub starts 2024-09-11T07:41:21.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:20 smithi022 ceph-mon[29462]: 5.1c scrub ok 2024-09-11T07:41:21.170 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:20 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:41:21.170 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:20 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:41:21.170 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:20 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:41:21.170 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:20 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:41:21.170 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:20 smithi022 ceph-mon[29462]: pgmap v684: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 80 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.33k op/s; 279/4422 objects misplaced (6.309%) 2024-09-11T07:41:21.170 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:20 smithi022 ceph-mon[29462]: osdmap e328: 8 total, 8 up, 8 in 2024-09-11T07:41:21.209 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:21.231 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:21.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:21.232 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:21.333 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:21.356 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:21.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:21.358 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:21.458 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:21.482 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:41:21.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:21.483 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:21.484 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:21.584 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:21.607 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:21.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:21.608 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:21.709 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:21.731 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:21.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:21.732 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:21.833 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:21.855 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:21.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:21.856 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:21.957 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:21.978 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:21.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:21.979 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:22.080 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:22.103 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:22.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:22.104 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:22.204 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:22.227 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:22.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:22.228 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:22.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:22 smithi104 ceph-mon[33507]: 6.7 scrub starts 2024-09-11T07:41:22.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:22 smithi104 ceph-mon[33507]: 6.7 scrub ok 2024-09-11T07:41:22.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:22 smithi104 ceph-mon[33507]: 2.b scrub starts 2024-09-11T07:41:22.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:22 smithi104 ceph-mon[33507]: 2.b scrub ok 2024-09-11T07:41:22.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:22 smithi104 ceph-mon[33507]: 6.0 scrub starts 2024-09-11T07:41:22.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:22 smithi104 ceph-mon[33507]: 6.0 scrub ok 2024-09-11T07:41:22.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:22 smithi104 ceph-mon[33507]: 4.17 scrub starts 2024-09-11T07:41:22.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:22 smithi104 ceph-mon[33507]: 4.17 scrub ok 2024-09-11T07:41:22.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:22 smithi104 ceph-mon[33507]: 4.3 scrub starts 2024-09-11T07:41:22.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:22 smithi104 ceph-mon[33507]: 4.3 scrub ok 2024-09-11T07:41:22.329 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:22.352 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:22.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:22.353 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:22.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:22 smithi022 ceph-mon[37330]: 6.7 scrub starts 2024-09-11T07:41:22.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:22 smithi022 ceph-mon[37330]: 6.7 scrub ok 2024-09-11T07:41:22.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:22 smithi022 ceph-mon[37330]: 2.b scrub starts 2024-09-11T07:41:22.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:22 smithi022 ceph-mon[37330]: 2.b scrub ok 2024-09-11T07:41:22.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:22 smithi022 ceph-mon[37330]: 6.0 scrub starts 2024-09-11T07:41:22.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:22 smithi022 ceph-mon[37330]: 6.0 scrub ok 2024-09-11T07:41:22.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:22 smithi022 ceph-mon[37330]: 4.17 scrub starts 2024-09-11T07:41:22.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:22 smithi022 ceph-mon[37330]: 4.17 scrub ok 2024-09-11T07:41:22.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:22 smithi022 ceph-mon[37330]: 4.3 scrub starts 2024-09-11T07:41:22.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:22 smithi022 ceph-mon[37330]: 4.3 scrub ok 2024-09-11T07:41:22.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:22 smithi022 ceph-mon[29462]: 6.7 scrub starts 2024-09-11T07:41:22.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:22 smithi022 ceph-mon[29462]: 6.7 scrub ok 2024-09-11T07:41:22.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:22 smithi022 ceph-mon[29462]: 2.b scrub starts 2024-09-11T07:41:22.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:22 smithi022 ceph-mon[29462]: 2.b scrub ok 2024-09-11T07:41:22.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:22 smithi022 ceph-mon[29462]: 6.0 scrub starts 2024-09-11T07:41:22.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:22 smithi022 ceph-mon[29462]: 6.0 scrub ok 2024-09-11T07:41:22.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:22 smithi022 ceph-mon[29462]: 4.17 scrub starts 2024-09-11T07:41:22.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:22 smithi022 ceph-mon[29462]: 4.17 scrub ok 2024-09-11T07:41:22.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:22 smithi022 ceph-mon[29462]: 4.3 scrub starts 2024-09-11T07:41:22.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:22 smithi022 ceph-mon[29462]: 4.3 scrub ok 2024-09-11T07:41:22.454 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:22.477 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:22.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:22.478 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:22.578 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:22.601 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:22.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:22.602 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:22.703 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:22.730 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:22.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:22.732 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:22.832 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:22.859 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:22.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:22.861 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:22.961 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:22.985 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:22.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:22.986 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:23.086 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:23.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:23.110 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:23.110 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:23.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:23 smithi022 ceph-mon[29462]: 4.18 scrub starts 2024-09-11T07:41:23.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:23 smithi022 ceph-mon[29462]: 4.18 scrub ok 2024-09-11T07:41:23.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:23 smithi022 ceph-mon[29462]: pgmap v686: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 18 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 775 B/s rd, 0 op/s; 132/1416 objects misplaced (9.322%) 2024-09-11T07:41:23.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:23 smithi022 ceph-mon[29462]: 4.4 scrub starts 2024-09-11T07:41:23.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:23 smithi022 ceph-mon[29462]: 4.4 scrub ok 2024-09-11T07:41:23.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:23 smithi022 ceph-mon[37330]: 4.18 scrub starts 2024-09-11T07:41:23.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:23 smithi022 ceph-mon[37330]: 4.18 scrub ok 2024-09-11T07:41:23.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:23 smithi022 ceph-mon[37330]: pgmap v686: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 18 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 775 B/s rd, 0 op/s; 132/1416 objects misplaced (9.322%) 2024-09-11T07:41:23.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:23 smithi022 ceph-mon[37330]: 4.4 scrub starts 2024-09-11T07:41:23.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:23 smithi022 ceph-mon[37330]: 4.4 scrub ok 2024-09-11T07:41:23.211 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:23.237 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:23.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:23.239 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:23.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:23 smithi104 ceph-mon[33507]: 4.18 scrub starts 2024-09-11T07:41:23.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:23 smithi104 ceph-mon[33507]: 4.18 scrub ok 2024-09-11T07:41:23.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:23 smithi104 ceph-mon[33507]: pgmap v686: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 18 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 775 B/s rd, 0 op/s; 132/1416 objects misplaced (9.322%) 2024-09-11T07:41:23.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:23 smithi104 ceph-mon[33507]: 4.4 scrub starts 2024-09-11T07:41:23.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:23 smithi104 ceph-mon[33507]: 4.4 scrub ok 2024-09-11T07:41:23.339 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:23.362 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:23.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:23.364 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:23.465 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:23.492 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:23.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:23.493 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:23.594 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:23.618 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:23.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:23.619 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:23.720 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:23.743 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:23.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:23.744 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:23.845 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:23.868 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:23.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:23.869 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:23.970 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:23.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:23.999 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:24.099 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:24.122 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:24.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:24.123 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:24.224 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:24.251 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:24.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:24.252 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:24.279 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:41:24 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:41:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:41:24.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:24 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:41:24.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:24 smithi104 ceph-mon[33507]: 5.4 scrub starts 2024-09-11T07:41:24.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:24 smithi104 ceph-mon[33507]: 5.4 scrub ok 2024-09-11T07:41:24.352 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:24.380 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:24.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:24.381 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:24.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:24 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:41:24.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:24 smithi022 ceph-mon[29462]: 5.4 scrub starts 2024-09-11T07:41:24.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:24 smithi022 ceph-mon[29462]: 5.4 scrub ok 2024-09-11T07:41:24.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:24 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:41:24.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:24 smithi022 ceph-mon[37330]: 5.4 scrub starts 2024-09-11T07:41:24.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:24 smithi022 ceph-mon[37330]: 5.4 scrub ok 2024-09-11T07:41:24.482 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:24.504 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:24.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:24.505 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:24.606 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:24.626 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:24.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:24.628 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:24.728 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:24.754 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:24.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:24.755 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:24.855 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:24.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:24.882 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:24.883 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:24.983 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:25.146 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:25.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:25.147 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:25.248 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:25.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:25.271 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:25.272 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:25.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:25 smithi104 ceph-mon[33507]: pgmap v687: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 18 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 132/1416 objects misplaced (9.322%) 2024-09-11T07:41:25.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:25 smithi104 ceph-mon[33507]: 2.d scrub starts 2024-09-11T07:41:25.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:25 smithi104 ceph-mon[33507]: 2.d scrub ok 2024-09-11T07:41:25.313 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:25 smithi022 ceph-mon[29462]: pgmap v687: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 18 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 132/1416 objects misplaced (9.322%) 2024-09-11T07:41:25.313 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:25 smithi022 ceph-mon[29462]: 2.d scrub starts 2024-09-11T07:41:25.314 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:25 smithi022 ceph-mon[29462]: 2.d scrub ok 2024-09-11T07:41:25.314 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:25 smithi022 ceph-mon[37330]: pgmap v687: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 18 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 132/1416 objects misplaced (9.322%) 2024-09-11T07:41:25.314 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:25 smithi022 ceph-mon[37330]: 2.d scrub starts 2024-09-11T07:41:25.314 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:25 smithi022 ceph-mon[37330]: 2.d scrub ok 2024-09-11T07:41:25.373 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:25.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:25.398 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:25.399 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:25.500 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:25.521 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:25.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:25.522 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:25.623 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:25.647 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:25.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:25.648 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:25.683 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:41:25.749 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:25.771 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:25.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:25.772 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:25.811 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:41:25.814 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:41:25.873 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:25.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:25.897 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:25.897 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:25.998 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:26.021 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:26.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:26.022 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:26.123 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:26.145 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:26.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:26.146 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:26.247 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:26.269 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:26.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:26.270 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:26.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:26 smithi104 ceph-mon[33507]: 4.1a scrub starts 2024-09-11T07:41:26.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:26 smithi104 ceph-mon[33507]: 4.1a scrub ok 2024-09-11T07:41:26.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:26 smithi104 ceph-mon[33507]: 6.1c scrub starts 2024-09-11T07:41:26.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:26 smithi104 ceph-mon[33507]: 6.1c scrub ok 2024-09-11T07:41:26.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:26 smithi104 ceph-mon[33507]: 4.14 scrub starts 2024-09-11T07:41:26.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:26 smithi104 ceph-mon[33507]: 4.14 scrub ok 2024-09-11T07:41:26.310 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:26 smithi022 ceph-mon[29462]: 4.1a scrub starts 2024-09-11T07:41:26.310 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:26 smithi022 ceph-mon[29462]: 4.1a scrub ok 2024-09-11T07:41:26.310 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:26 smithi022 ceph-mon[29462]: 6.1c scrub starts 2024-09-11T07:41:26.311 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:26 smithi022 ceph-mon[29462]: 6.1c scrub ok 2024-09-11T07:41:26.311 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:26 smithi022 ceph-mon[29462]: 4.14 scrub starts 2024-09-11T07:41:26.311 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:26 smithi022 ceph-mon[29462]: 4.14 scrub ok 2024-09-11T07:41:26.311 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:26 smithi022 ceph-mon[37330]: 4.1a scrub starts 2024-09-11T07:41:26.311 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:26 smithi022 ceph-mon[37330]: 4.1a scrub ok 2024-09-11T07:41:26.311 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:26 smithi022 ceph-mon[37330]: 6.1c scrub starts 2024-09-11T07:41:26.311 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:26 smithi022 ceph-mon[37330]: 6.1c scrub ok 2024-09-11T07:41:26.312 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:26 smithi022 ceph-mon[37330]: 4.14 scrub starts 2024-09-11T07:41:26.312 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:26 smithi022 ceph-mon[37330]: 4.14 scrub ok 2024-09-11T07:41:26.371 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:26.394 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:26.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:26.395 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:26.496 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:26.518 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:26.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:26.520 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:26.620 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:26.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:26.646 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:26.646 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:26.747 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:26.770 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:26.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:26.771 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:26.871 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:26.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:26.894 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:26.895 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:26.995 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:27.018 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:27.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:27.020 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:27.025 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:41:27.025 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 20, 2024-09-11T07:41:27.025 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 600, 2024-09-11T07:41:27.025 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:41:27.025 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.025 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:213455 7.f 7:f4abd973:::benchmark_data_smithi022_77720_object89700:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-11T07:41:27.025 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:00.080874+0000", 2024-09-11T07:41:27.026 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 566.93901896600005, 2024-09-11T07:41:27.026 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9603731720000002, 2024-09-11T07:41:27.026 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:27.026 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:41:27.026 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:27.026 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:41:27.026 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:41:27.026 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213455 2024-09-11T07:41:27.026 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.027 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:27.027 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.027 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:27.027 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.080874+0000", 2024-09-11T07:41:27.027 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.027 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.027 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.027 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:27.027 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.080874+0000", 2024-09-11T07:41:27.027 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.027 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.027 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.027 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:27.028 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.080871+0000", 2024-09-11T07:41:27.028 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:41:27.028 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.028 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.028 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:27.028 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.080880+0000", 2024-09-11T07:41:27.028 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.3540000000000006e-06 2024-09-11T07:41:27.028 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.028 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.028 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:27.028 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.080894+0000", 2024-09-11T07:41:27.028 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3073e-05 2024-09-11T07:41:27.028 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.029 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.029 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:27.029 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.080901+0000", 2024-09-11T07:41:27.029 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0609999999999998e-06 2024-09-11T07:41:27.029 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.029 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.029 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.029 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.080993+0000", 2024-09-11T07:41:27.029 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.2237000000000005e-05 2024-09-11T07:41:27.029 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.029 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.029 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:27.029 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.081121+0000", 2024-09-11T07:41:27.030 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012860000000000001 2024-09-11T07:41:27.030 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.030 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.030 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3", 2024-09-11T07:41:27.030 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.081242+0000", 2024-09-11T07:41:27.030 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000120677 2024-09-11T07:41:27.030 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.030 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.030 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:27.030 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.099147+0000", 2024-09-11T07:41:27.030 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.017904863 2024-09-11T07:41:27.030 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.030 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.030 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.031 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.104850+0000", 2024-09-11T07:41:27.031 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0057033930000000002 2024-09-11T07:41:27.031 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.031 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.031 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.031 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.104859+0000", 2024-09-11T07:41:27.031 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.0180000000000007e-06 2024-09-11T07:41:27.031 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.031 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.031 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.031 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502755+0000", 2024-09-11T07:41:27.031 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.39789565399999999 2024-09-11T07:41:27.031 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.031 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.032 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.032 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502764+0000", 2024-09-11T07:41:27.032 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.9460000000000004e-06 2024-09-11T07:41:27.032 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.032 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.032 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.032 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:01.995532+0000", 2024-09-11T07:41:27.032 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.492767924 2024-09-11T07:41:27.032 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.032 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.032 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for missing object", 2024-09-11T07:41:27.032 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:01.995669+0000", 2024-09-11T07:41:27.032 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013712899999999999 2024-09-11T07:41:27.033 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.033 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.033 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.033 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.041144+0000", 2024-09-11T07:41:27.033 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.045474964 2024-09-11T07:41:27.033 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.033 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.033 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:27.033 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.041247+0000", 2024-09-11T07:41:27.033 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000102835 2024-09-11T07:41:27.033 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.033 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:27.033 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.034 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.034 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.034 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:213456 7.f 7:fe3cd8b0:::benchmark_data_smithi022_77720_object89701:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-11T07:41:27.034 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:00.087849+0000", 2024-09-11T07:41:27.034 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 566.93204320300003, 2024-09-11T07:41:27.034 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9475085029999999, 2024-09-11T07:41:27.034 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:27.034 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:27.034 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:27.034 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:41:27.034 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:41:27.034 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213456 2024-09-11T07:41:27.034 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.035 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:27.035 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.035 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:27.035 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.087849+0000", 2024-09-11T07:41:27.035 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.035 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.035 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.035 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:27.035 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.087849+0000", 2024-09-11T07:41:27.035 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.035 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.036 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.036 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:27.036 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.087847+0000", 2024-09-11T07:41:27.036 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:41:27.036 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.036 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.036 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:27.036 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.087855+0000", 2024-09-11T07:41:27.036 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.2199999999999992e-06 2024-09-11T07:41:27.036 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.036 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.037 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:27.037 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.087868+0000", 2024-09-11T07:41:27.037 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2447e-05 2024-09-11T07:41:27.037 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.037 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.037 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:27.037 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.087875+0000", 2024-09-11T07:41:27.037 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.759e-06 2024-09-11T07:41:27.037 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.037 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.037 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.037 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.107015+0000", 2024-09-11T07:41:27.038 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.019140834999999998 2024-09-11T07:41:27.038 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.038 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.038 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.038 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.107025+0000", 2024-09-11T07:41:27.038 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.064e-06 2024-09-11T07:41:27.038 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.038 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.038 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.038 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502790+0000", 2024-09-11T07:41:27.039 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.39576567200000001 2024-09-11T07:41:27.039 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.039 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.039 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.039 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502797+0000", 2024-09-11T07:41:27.039 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1439999999999997e-06 2024-09-11T07:41:27.039 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.039 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.039 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.039 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:01.995707+0000", 2024-09-11T07:41:27.040 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4929092989999999 2024-09-11T07:41:27.040 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.040 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.040 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:27.040 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:01.995811+0000", 2024-09-11T07:41:27.040 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010403099999999999 2024-09-11T07:41:27.040 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.040 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.040 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:41:27.040 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:01.995891+0000", 2024-09-11T07:41:27.040 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.0668999999999996e-05 2024-09-11T07:41:27.041 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.041 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.041 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:27.041 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.008367+0000", 2024-09-11T07:41:27.041 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.012475966999999999 2024-09-11T07:41:27.041 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.041 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.041 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.041 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.008945+0000", 2024-09-11T07:41:27.041 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00057729099999999996 2024-09-11T07:41:27.042 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.042 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.042 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.042 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.034874+0000", 2024-09-11T07:41:27.042 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.025929285999999999 2024-09-11T07:41:27.042 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.042 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.042 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:27.042 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.034966+0000", 2024-09-11T07:41:27.042 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.2115999999999997e-05 2024-09-11T07:41:27.042 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.042 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.042 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:27.043 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.035358+0000", 2024-09-11T07:41:27.043 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00039192399999999998 2024-09-11T07:41:27.043 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.043 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:27.043 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.043 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.043 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.043 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:213461 7.f 7:f62d3026:::benchmark_data_smithi022_77720_object89706:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-11T07:41:27.043 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:00.102257+0000", 2024-09-11T07:41:27.043 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 566.91763606100005, 2024-09-11T07:41:27.043 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9333541919999999, 2024-09-11T07:41:27.044 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:27.044 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:27.044 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:27.044 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:41:27.044 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:41:27.044 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213461 2024-09-11T07:41:27.044 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.044 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:27.044 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.044 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:27.044 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.102257+0000", 2024-09-11T07:41:27.044 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.045 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.045 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.045 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:27.045 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.102257+0000", 2024-09-11T07:41:27.045 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.045 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.045 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.045 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:27.045 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.102255+0000", 2024-09-11T07:41:27.045 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:41:27.045 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.045 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.046 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:27.046 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.102261+0000", 2024-09-11T07:41:27.046 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5309999999999998e-06 2024-09-11T07:41:27.046 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.046 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.046 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:27.046 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.102271+0000", 2024-09-11T07:41:27.046 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0169e-05 2024-09-11T07:41:27.046 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.047 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.047 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:27.047 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.102292+0000", 2024-09-11T07:41:27.047 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0650999999999999e-05 2024-09-11T07:41:27.047 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.047 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.047 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.047 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.107047+0000", 2024-09-11T07:41:27.047 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0047554040000000004 2024-09-11T07:41:27.047 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.047 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.048 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.048 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.107053+0000", 2024-09-11T07:41:27.048 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0639999999999999e-06 2024-09-11T07:41:27.048 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.048 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.048 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.048 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502817+0000", 2024-09-11T07:41:27.048 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.39576341199999998 2024-09-11T07:41:27.048 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.048 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.048 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.049 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502823+0000", 2024-09-11T07:41:27.049 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.725e-06 2024-09-11T07:41:27.049 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.049 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.049 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.049 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:01.998668+0000", 2024-09-11T07:41:27.049 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4958450189999999 2024-09-11T07:41:27.049 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.049 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.049 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:27.049 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:01.998813+0000", 2024-09-11T07:41:27.050 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014467399999999999 2024-09-11T07:41:27.050 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.050 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.050 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:41:27.050 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:01.998964+0000", 2024-09-11T07:41:27.050 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015080700000000001 2024-09-11T07:41:27.050 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.050 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.050 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:27.050 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.008955+0000", 2024-09-11T07:41:27.050 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0099908700000000007 2024-09-11T07:41:27.051 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.051 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.051 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.051 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.009680+0000", 2024-09-11T07:41:27.051 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00072526700000000003 2024-09-11T07:41:27.051 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.051 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.051 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.051 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.035411+0000", 2024-09-11T07:41:27.051 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.025730755000000001 2024-09-11T07:41:27.052 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.052 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.052 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:27.052 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.035457+0000", 2024-09-11T07:41:27.053 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6254999999999997e-05 2024-09-11T07:41:27.053 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.053 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.053 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:27.053 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.035611+0000", 2024-09-11T07:41:27.053 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000153621 2024-09-11T07:41:27.053 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.053 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:27.053 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.053 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.053 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.054 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:213466 7.6 7:66df2139:::benchmark_data_smithi022_77720_object89711:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-11T07:41:27.054 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:00.108031+0000", 2024-09-11T07:41:27.054 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 566.91186114300001, 2024-09-11T07:41:27.054 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9748235599999999, 2024-09-11T07:41:27.054 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:27.054 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:27.054 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:27.054 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:41:27.055 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:41:27.055 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213466 2024-09-11T07:41:27.055 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.055 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:27.055 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.055 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:27.055 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.108031+0000", 2024-09-11T07:41:27.055 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.055 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.055 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.055 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:27.055 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.108031+0000", 2024-09-11T07:41:27.055 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.056 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.056 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.056 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:27.056 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.108029+0000", 2024-09-11T07:41:27.056 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:41:27.056 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.056 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.056 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:27.056 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.108037+0000", 2024-09-11T07:41:27.056 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.9510000000000006e-06 2024-09-11T07:41:27.056 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.056 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.056 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:27.057 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.108047+0000", 2024-09-11T07:41:27.057 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.7249999999999992e-06 2024-09-11T07:41:27.057 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.057 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.057 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:27.057 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.108053+0000", 2024-09-11T07:41:27.057 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7030000000000003e-06 2024-09-11T07:41:27.057 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.057 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.057 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.057 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.111646+0000", 2024-09-11T07:41:27.057 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0035932659999999999 2024-09-11T07:41:27.057 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.058 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.058 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.058 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.111655+0000", 2024-09-11T07:41:27.058 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.4900000000000006e-06 2024-09-11T07:41:27.058 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.058 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.058 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.058 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502475+0000", 2024-09-11T07:41:27.058 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.39081970199999999 2024-09-11T07:41:27.059 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.059 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.059 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.059 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502486+0000", 2024-09-11T07:41:27.060 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1391e-05 2024-09-11T07:41:27.060 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.060 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.060 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.060 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.055238+0000", 2024-09-11T07:41:27.060 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5527511650000001 2024-09-11T07:41:27.060 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.060 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.060 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:27.060 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.055350+0000", 2024-09-11T07:41:27.060 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000112248 2024-09-11T07:41:27.061 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.061 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.061 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:41:27.061 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.055469+0000", 2024-09-11T07:41:27.061 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000119067 2024-09-11T07:41:27.061 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.061 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.061 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:27.061 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.064868+0000", 2024-09-11T07:41:27.061 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0093990099999999993 2024-09-11T07:41:27.061 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.061 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.061 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.062 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.079028+0000", 2024-09-11T07:41:27.062 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.014160104 2024-09-11T07:41:27.062 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.062 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.062 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.062 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.082289+0000", 2024-09-11T07:41:27.062 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0032610529999999999 2024-09-11T07:41:27.062 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.062 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.062 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:27.062 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.082491+0000", 2024-09-11T07:41:27.062 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00020178899999999999 2024-09-11T07:41:27.062 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.063 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.063 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:27.063 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.082855+0000", 2024-09-11T07:41:27.063 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000364144 2024-09-11T07:41:27.063 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.063 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:27.063 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.063 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.063 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.063 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:213482 7.6 7:68f51978:::benchmark_data_smithi022_77720_object89727:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-11T07:41:27.063 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:00.121732+0000", 2024-09-11T07:41:27.063 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 566.89816092299998, 2024-09-11T07:41:27.063 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9616346930000002, 2024-09-11T07:41:27.063 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:27.064 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:27.064 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:27.064 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:41:27.064 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:41:27.064 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213482 2024-09-11T07:41:27.064 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.064 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:27.064 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.064 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:27.064 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.121732+0000", 2024-09-11T07:41:27.064 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.064 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.065 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.065 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:27.065 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.121732+0000", 2024-09-11T07:41:27.065 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.065 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.065 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.065 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:27.065 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.121731+0000", 2024-09-11T07:41:27.065 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:41:27.065 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.065 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.065 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:27.065 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.121735+0000", 2024-09-11T07:41:27.066 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7999999999999998e-06 2024-09-11T07:41:27.066 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.066 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.066 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:27.066 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.121743+0000", 2024-09-11T07:41:27.066 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.588e-06 2024-09-11T07:41:27.066 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.066 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.066 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:27.066 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.121747+0000", 2024-09-11T07:41:27.066 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2390000000000004e-06 2024-09-11T07:41:27.066 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.066 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.066 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.067 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.122374+0000", 2024-09-11T07:41:27.067 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00062680600000000004 2024-09-11T07:41:27.067 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.067 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.067 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.067 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.122382+0000", 2024-09-11T07:41:27.067 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.2889999999999998e-06 2024-09-11T07:41:27.067 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.067 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.068 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.068 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502524+0000", 2024-09-11T07:41:27.068 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.38014126300000001 2024-09-11T07:41:27.068 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.068 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.068 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.068 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502531+0000", 2024-09-11T07:41:27.069 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1740000000000003e-06 2024-09-11T07:41:27.069 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.069 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.069 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.069 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.058510+0000", 2024-09-11T07:41:27.069 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.555978825 2024-09-11T07:41:27.069 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.069 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.069 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:27.069 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.058651+0000", 2024-09-11T07:41:27.069 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000141395 2024-09-11T07:41:27.069 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.070 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.070 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:41:27.070 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.058837+0000", 2024-09-11T07:41:27.070 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00018597500000000001 2024-09-11T07:41:27.070 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.070 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.070 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:27.070 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.073988+0000", 2024-09-11T07:41:27.070 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.015151009999999999 2024-09-11T07:41:27.070 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.070 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.070 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.070 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.082894+0000", 2024-09-11T07:41:27.070 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0089062640000000005 2024-09-11T07:41:27.071 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.071 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.071 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.071 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.083176+0000", 2024-09-11T07:41:27.071 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00028186800000000001 2024-09-11T07:41:27.071 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.071 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.071 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:27.071 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.083216+0000", 2024-09-11T07:41:27.071 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.019e-05 2024-09-11T07:41:27.071 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.071 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.071 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:27.072 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.083366+0000", 2024-09-11T07:41:27.072 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015016300000000001 2024-09-11T07:41:27.072 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.072 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:27.072 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.072 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.072 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.072 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:213492 7.6 7:63c90940:::benchmark_data_smithi022_77720_object89737:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-11T07:41:27.072 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:00.132840+0000", 2024-09-11T07:41:27.072 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 566.88705243100003, 2024-09-11T07:41:27.072 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.950725611, 2024-09-11T07:41:27.072 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:27.072 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:27.073 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:27.073 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:41:27.073 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:41:27.073 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213492 2024-09-11T07:41:27.073 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.073 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:27.073 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.073 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:27.074 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.132840+0000", 2024-09-11T07:41:27.074 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.074 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.074 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.074 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:27.074 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.132840+0000", 2024-09-11T07:41:27.074 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.074 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.074 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.074 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:27.075 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.132839+0000", 2024-09-11T07:41:27.075 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:41:27.075 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.075 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.075 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:27.075 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.132845+0000", 2024-09-11T07:41:27.075 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.798e-06 2024-09-11T07:41:27.075 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.075 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.075 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:27.075 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.132861+0000", 2024-09-11T07:41:27.075 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.6086000000000002e-05 2024-09-11T07:41:27.076 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is unset 2024-09-11T07:41:27.077 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.077 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.078 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:27.078 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.132868+0000", 2024-09-11T07:41:27.078 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2129999999999996e-06 2024-09-11T07:41:27.078 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.078 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.078 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.078 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.132953+0000", 2024-09-11T07:41:27.078 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.5389999999999994e-05 2024-09-11T07:41:27.078 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.078 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.078 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.078 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.132963+0000", 2024-09-11T07:41:27.079 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.9389999999999992e-06 2024-09-11T07:41:27.079 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.079 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.079 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.079 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502555+0000", 2024-09-11T07:41:27.079 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.36959232199999997 2024-09-11T07:41:27.079 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.079 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.079 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.079 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502562+0000", 2024-09-11T07:41:27.079 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1160000000000001e-06 2024-09-11T07:41:27.079 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.080 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.080 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.080 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.061705+0000", 2024-09-11T07:41:27.080 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5591422750000001 2024-09-11T07:41:27.080 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.080 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.080 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:27.080 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.061818+0000", 2024-09-11T07:41:27.080 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000113824 2024-09-11T07:41:27.080 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.080 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.081 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:41:27.081 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.061942+0000", 2024-09-11T07:41:27.081 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000123125 2024-09-11T07:41:27.081 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.081 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.081 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:27.081 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.074019+0000", 2024-09-11T07:41:27.081 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.012077048999999999 2024-09-11T07:41:27.081 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.081 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.081 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.082 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.082921+0000", 2024-09-11T07:41:27.082 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0089027959999999993 2024-09-11T07:41:27.082 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.082 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.082 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.082 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.083399+0000", 2024-09-11T07:41:27.082 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00047718000000000001 2024-09-11T07:41:27.082 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.082 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.083 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:27.083 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.083434+0000", 2024-09-11T07:41:27.083 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5805e-05 2024-09-11T07:41:27.083 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.083 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.083 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:27.083 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.083566+0000", 2024-09-11T07:41:27.083 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013135999999999999 2024-09-11T07:41:27.083 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.083 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:27.083 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.083 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.083 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.084 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:213495 7.6 7:6bab33dc:::benchmark_data_smithi022_77720_object89740:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-11T07:41:27.084 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:00.133603+0000", 2024-09-11T07:41:27.084 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 566.88628956299999, 2024-09-11T07:41:27.084 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.950173537, 2024-09-11T07:41:27.084 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:27.084 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:27.084 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:27.084 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:41:27.084 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:41:27.084 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213495 2024-09-11T07:41:27.084 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.084 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:27.084 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.085 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:27.085 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.133603+0000", 2024-09-11T07:41:27.085 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.085 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.085 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.085 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:27.085 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.133603+0000", 2024-09-11T07:41:27.085 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.085 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.085 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.085 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:27.085 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.133602+0000", 2024-09-11T07:41:27.086 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:41:27.086 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.086 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.086 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:27.086 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.133608+0000", 2024-09-11T07:41:27.086 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9540000000000001e-06 2024-09-11T07:41:27.086 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.086 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.086 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:27.086 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.133618+0000", 2024-09-11T07:41:27.086 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0414000000000001e-05 2024-09-11T07:41:27.086 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.086 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.087 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:27.087 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.133624+0000", 2024-09-11T07:41:27.087 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8030000000000002e-06 2024-09-11T07:41:27.087 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.087 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.087 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.087 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.133667+0000", 2024-09-11T07:41:27.087 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3158000000000003e-05 2024-09-11T07:41:27.087 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.087 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.088 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.088 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.133693+0000", 2024-09-11T07:41:27.088 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.637e-05 2024-09-11T07:41:27.088 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.088 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.088 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.088 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502586+0000", 2024-09-11T07:41:27.088 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.36889280699999999 2024-09-11T07:41:27.088 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.088 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.088 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.088 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502593+0000", 2024-09-11T07:41:27.088 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.2150000000000004e-06 2024-09-11T07:41:27.089 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.089 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.089 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.089 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.064896+0000", 2024-09-11T07:41:27.089 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5623021349999999 2024-09-11T07:41:27.089 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.089 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.089 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:27.089 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.065019+0000", 2024-09-11T07:41:27.089 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000122958 2024-09-11T07:41:27.090 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.090 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.090 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:41:27.090 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.065150+0000", 2024-09-11T07:41:27.090 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013193999999999999 2024-09-11T07:41:27.090 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.090 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.090 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.090 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.082948+0000", 2024-09-11T07:41:27.090 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.017797149000000002 2024-09-11T07:41:27.091 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.091 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.091 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:27.091 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.083004+0000", 2024-09-11T07:41:27.091 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6184999999999998e-05 2024-09-11T07:41:27.091 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.091 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.091 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.091 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.083598+0000", 2024-09-11T07:41:27.091 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00059438899999999996 2024-09-11T07:41:27.091 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.092 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.092 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:27.092 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.083638+0000", 2024-09-11T07:41:27.092 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9570999999999998e-05 2024-09-11T07:41:27.092 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.092 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.092 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:27.092 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.083777+0000", 2024-09-11T07:41:27.092 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013881399999999999 2024-09-11T07:41:27.092 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.092 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:27.092 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.092 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.093 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.093 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:213498 7.f 7:fce9e756:::benchmark_data_smithi022_77720_object89743:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-11T07:41:27.093 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:00.138124+0000", 2024-09-11T07:41:27.093 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 566.88176836399998, 2024-09-11T07:41:27.093 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8977508329999999, 2024-09-11T07:41:27.093 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:27.093 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:27.093 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:27.093 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:41:27.093 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:41:27.094 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213498 2024-09-11T07:41:27.094 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.094 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:27.094 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.094 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:27.094 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.138124+0000", 2024-09-11T07:41:27.094 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.094 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.094 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.094 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:27.095 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.138124+0000", 2024-09-11T07:41:27.095 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.095 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.095 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.095 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:27.095 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.138122+0000", 2024-09-11T07:41:27.095 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:41:27.095 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.095 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.095 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:27.095 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.138130+0000", 2024-09-11T07:41:27.096 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.8380000000000002e-06 2024-09-11T07:41:27.096 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.096 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.096 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:27.096 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.138142+0000", 2024-09-11T07:41:27.096 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2143999999999999e-05 2024-09-11T07:41:27.096 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.096 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.096 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:27.096 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.138149+0000", 2024-09-11T07:41:27.097 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9709999999999998e-06 2024-09-11T07:41:27.097 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.097 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.097 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.097 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.138199+0000", 2024-09-11T07:41:27.097 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.0308e-05 2024-09-11T07:41:27.097 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.100 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.100 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.100 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.138208+0000", 2024-09-11T07:41:27.100 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.1790000000000008e-06 2024-09-11T07:41:27.100 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.100 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.101 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.101 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502844+0000", 2024-09-11T07:41:27.101 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.36463590600000001 2024-09-11T07:41:27.101 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.101 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.101 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.101 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502851+0000", 2024-09-11T07:41:27.101 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5240000000000003e-06 2024-09-11T07:41:27.101 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.101 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.101 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.101 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.001786+0000", 2024-09-11T07:41:27.102 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.498934945 2024-09-11T07:41:27.102 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.102 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.102 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:27.102 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.001928+0000", 2024-09-11T07:41:27.102 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000142368 2024-09-11T07:41:27.102 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.102 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.102 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:41:27.102 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.002065+0000", 2024-09-11T07:41:27.102 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000136694 2024-09-11T07:41:27.102 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.102 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.103 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:27.103 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.008965+0000", 2024-09-11T07:41:27.103 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0068995339999999997 2024-09-11T07:41:27.103 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.103 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.103 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.103 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.009714+0000", 2024-09-11T07:41:27.103 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00074906300000000001 2024-09-11T07:41:27.103 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.103 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.103 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.103 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.035662+0000", 2024-09-11T07:41:27.103 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.02594861 2024-09-11T07:41:27.103 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.104 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.104 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:27.104 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.035691+0000", 2024-09-11T07:41:27.104 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8841000000000001e-05 2024-09-11T07:41:27.104 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.104 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.104 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:27.104 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.035875+0000", 2024-09-11T07:41:27.104 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000183938 2024-09-11T07:41:27.104 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.104 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:27.104 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.104 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.105 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.105 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:213504 7.f 7:f173a132:::benchmark_data_smithi022_77720_object89749:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-11T07:41:27.105 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:00.147571+0000", 2024-09-11T07:41:27.105 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 566.87232163299996, 2024-09-11T07:41:27.105 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.888562385, 2024-09-11T07:41:27.105 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:27.105 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:27.105 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:27.105 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:41:27.105 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:41:27.105 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213504 2024-09-11T07:41:27.105 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.105 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:27.106 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.106 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:27.106 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.147571+0000", 2024-09-11T07:41:27.106 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.106 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.106 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.106 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:27.106 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.147571+0000", 2024-09-11T07:41:27.106 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.106 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.106 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.106 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:27.106 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.147569+0000", 2024-09-11T07:41:27.107 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:41:27.107 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.107 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.107 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:27.107 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.147575+0000", 2024-09-11T07:41:27.107 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9020000000000004e-06 2024-09-11T07:41:27.107 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.107 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.107 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:27.107 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.147586+0000", 2024-09-11T07:41:27.107 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0256e-05 2024-09-11T07:41:27.107 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.107 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.107 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:27.108 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.147590+0000", 2024-09-11T07:41:27.108 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5750000000000002e-06 2024-09-11T07:41:27.108 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.108 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.108 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.108 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.147633+0000", 2024-09-11T07:41:27.108 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3205999999999999e-05 2024-09-11T07:41:27.108 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.108 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.108 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.108 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.147641+0000", 2024-09-11T07:41:27.108 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.7020000000000001e-06 2024-09-11T07:41:27.108 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.109 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.109 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.109 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502872+0000", 2024-09-11T07:41:27.109 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.35523062300000002 2024-09-11T07:41:27.109 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.109 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.109 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.109 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502879+0000", 2024-09-11T07:41:27.109 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.8900000000000001e-06 2024-09-11T07:41:27.109 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.109 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.110 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.110 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.004748+0000", 2024-09-11T07:41:27.110 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.501869117 2024-09-11T07:41:27.110 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.110 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.110 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:27.110 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.004874+0000", 2024-09-11T07:41:27.110 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012605799999999999 2024-09-11T07:41:27.110 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.111 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.111 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:41:27.111 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.005321+0000", 2024-09-11T07:41:27.111 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00044770500000000001 2024-09-11T07:41:27.111 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.111 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.111 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:27.111 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.009129+0000", 2024-09-11T07:41:27.111 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0038073949999999999 2024-09-11T07:41:27.111 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.111 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.111 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.112 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.009797+0000", 2024-09-11T07:41:27.112 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00066787400000000005 2024-09-11T07:41:27.112 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.112 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.112 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.112 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.035927+0000", 2024-09-11T07:41:27.112 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.026129896999999999 2024-09-11T07:41:27.112 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.112 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.112 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:27.113 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.035974+0000", 2024-09-11T07:41:27.113 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7679999999999998e-05 2024-09-11T07:41:27.113 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.113 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.113 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:27.113 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.036133+0000", 2024-09-11T07:41:27.113 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015902599999999999 2024-09-11T07:41:27.113 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.113 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:27.113 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.114 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.114 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.114 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24778.0:213505 7.6 7:62fc8d5e:::benchmark_data_smithi022_77720_object89750:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-11T07:41:27.114 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:32:00.148250+0000", 2024-09-11T07:41:27.114 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 566.87164250000001, 2024-09-11T07:41:27.114 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9638549599999999, 2024-09-11T07:41:27.114 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:27.114 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:27.114 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:27.114 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24778", 2024-09-11T07:41:27.114 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1066810893", 2024-09-11T07:41:27.114 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213505 2024-09-11T07:41:27.114 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.115 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:27.115 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.115 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:27.115 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.148250+0000", 2024-09-11T07:41:27.115 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.115 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.115 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.115 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:27.115 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.148250+0000", 2024-09-11T07:41:27.115 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.115 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.115 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.115 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:27.115 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.148249+0000", 2024-09-11T07:41:27.116 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:41:27.116 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.116 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.116 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:27.116 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.148254+0000", 2024-09-11T07:41:27.116 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7910000000000004e-06 2024-09-11T07:41:27.116 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.116 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.116 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:27.116 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.148263+0000", 2024-09-11T07:41:27.116 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.7709999999999995e-06 2024-09-11T07:41:27.116 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.116 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.117 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:27.117 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.148266+0000", 2024-09-11T07:41:27.117 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5580000000000001e-06 2024-09-11T07:41:27.117 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.117 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.117 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.117 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.148301+0000", 2024-09-11T07:41:27.117 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5200999999999997e-05 2024-09-11T07:41:27.117 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.117 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.117 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.117 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.148307+0000", 2024-09-11T07:41:27.117 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.4559999999999999e-06 2024-09-11T07:41:27.118 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.118 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.118 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.118 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502614+0000", 2024-09-11T07:41:27.118 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.35430687100000002 2024-09-11T07:41:27.118 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.118 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.118 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.118 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:00.502621+0000", 2024-09-11T07:41:27.118 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.8789999999999997e-06 2024-09-11T07:41:27.118 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.118 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.118 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.119 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.068237+0000", 2024-09-11T07:41:27.119 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5656168400000001 2024-09-11T07:41:27.119 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.119 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.119 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:27.119 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.068353+0000", 2024-09-11T07:41:27.119 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000115969 2024-09-11T07:41:27.119 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.119 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.119 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:41:27.119 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.068475+0000", 2024-09-11T07:41:27.119 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012209699999999999 2024-09-11T07:41:27.119 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.120 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.120 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.120 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.082974+0000", 2024-09-11T07:41:27.120 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.014498176999999999 2024-09-11T07:41:27.120 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.120 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.120 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:27.120 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.083010+0000", 2024-09-11T07:41:27.120 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6448000000000001e-05 2024-09-11T07:41:27.120 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.121 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.121 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.121 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.111738+0000", 2024-09-11T07:41:27.121 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.028728331999999999 2024-09-11T07:41:27.121 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.121 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.121 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:27.121 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.111794+0000", 2024-09-11T07:41:27.121 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.5967000000000001e-05 2024-09-11T07:41:27.121 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.121 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.122 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:27.122 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:32:02.112105+0000", 2024-09-11T07:41:27.122 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000310713 2024-09-11T07:41:27.122 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.122 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:27.122 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.122 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.122 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.122 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:85945 9.15 9:ade5fd86:::benchmark_data_smithi022_133943_object85944:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e259)", 2024-09-11T07:41:27.122 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:26.360905+0000", 2024-09-11T07:41:27.122 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 180.65898775100001, 2024-09-11T07:41:27.122 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.024384057, 2024-09-11T07:41:27.122 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:27.123 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:41:27.123 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:27.123 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:27.123 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:27.123 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 85945 2024-09-11T07:41:27.123 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.123 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:27.123 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.123 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:27.123 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360905+0000", 2024-09-11T07:41:27.123 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.123 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.123 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.123 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:27.124 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360905+0000", 2024-09-11T07:41:27.124 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.124 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.124 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.124 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:27.124 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360904+0000", 2024-09-11T07:41:27.124 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:41:27.124 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.124 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.124 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:27.124 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360927+0000", 2024-09-11T07:41:27.124 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2274999999999999e-05 2024-09-11T07:41:27.125 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.125 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.125 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:27.125 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360932+0000", 2024-09-11T07:41:27.125 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2650000000000004e-06 2024-09-11T07:41:27.125 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.125 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.125 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:27.125 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360935+0000", 2024-09-11T07:41:27.125 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.303e-06 2024-09-11T07:41:27.126 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.126 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.126 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.126 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.461352+0000", 2024-09-11T07:41:27.126 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.10041712 2024-09-11T07:41:27.126 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.126 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.126 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.126 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.461362+0000", 2024-09-11T07:41:27.126 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0285999999999999e-05 2024-09-11T07:41:27.127 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.127 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.127 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.127 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.384927+0000", 2024-09-11T07:41:27.127 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.92356409100000003 2024-09-11T07:41:27.127 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.127 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.127 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:27.128 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.385289+0000", 2024-09-11T07:41:27.128 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00036233199999999999 2024-09-11T07:41:27.128 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.128 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:27.128 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.128 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.128 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.128 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:85949 9.9 9:956e8bba:::benchmark_data_smithi022_133943_object85948:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e259)", 2024-09-11T07:41:27.128 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:26.360938+0000", 2024-09-11T07:41:27.129 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 180.65895436, 2024-09-11T07:41:27.129 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.021308804, 2024-09-11T07:41:27.129 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:27.129 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:27.129 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:27.129 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:27.129 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:27.129 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 85949 2024-09-11T07:41:27.129 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.129 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:27.129 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.129 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:27.130 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360938+0000", 2024-09-11T07:41:27.130 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.130 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.130 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.130 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:27.130 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360938+0000", 2024-09-11T07:41:27.130 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.130 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.130 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.130 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:27.130 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360938+0000", 2024-09-11T07:41:27.130 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:41:27.130 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.131 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.131 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:27.131 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360995+0000", 2024-09-11T07:41:27.131 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7525999999999998e-05 2024-09-11T07:41:27.131 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.131 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.131 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:27.131 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.361001+0000", 2024-09-11T07:41:27.131 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3480000000000003e-06 2024-09-11T07:41:27.131 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.131 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.131 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:27.131 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.361004+0000", 2024-09-11T07:41:27.132 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5370000000000002e-06 2024-09-11T07:41:27.132 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.132 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.132 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.132 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.482968+0000", 2024-09-11T07:41:27.132 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.12196362700000001 2024-09-11T07:41:27.132 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.132 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.132 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.132 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.482978+0000", 2024-09-11T07:41:27.132 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.8260000000000005e-06 2024-09-11T07:41:27.132 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.132 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.133 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.133 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.369380+0000", 2024-09-11T07:41:27.133 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.886402201 2024-09-11T07:41:27.133 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.133 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.133 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:27.133 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.369511+0000", 2024-09-11T07:41:27.133 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013084699999999999 2024-09-11T07:41:27.133 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.133 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.133 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,7", 2024-09-11T07:41:27.133 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.369654+0000", 2024-09-11T07:41:27.133 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014310200000000001 2024-09-11T07:41:27.134 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.134 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.134 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:27.134 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.371458+0000", 2024-09-11T07:41:27.134 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001804436 2024-09-11T07:41:27.134 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.134 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.134 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.134 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.378473+0000", 2024-09-11T07:41:27.134 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0070152299999999999 2024-09-11T07:41:27.134 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.134 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.134 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.135 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.382167+0000", 2024-09-11T07:41:27.135 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.003693694 2024-09-11T07:41:27.135 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.135 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.135 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:27.135 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.382213+0000", 2024-09-11T07:41:27.135 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5469999999999997e-05 2024-09-11T07:41:27.135 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.135 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.135 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:27.135 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.382247+0000", 2024-09-11T07:41:27.135 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4468000000000003e-05 2024-09-11T07:41:27.135 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.136 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:27.136 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.136 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.136 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.136 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87210 9.19 9:9bd7b134:::benchmark_data_smithi022_133943_object87209:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:41:27.136 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.341865+0000", 2024-09-11T07:41:27.136 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 177.67802741200001, 2024-09-11T07:41:27.136 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.225729383, 2024-09-11T07:41:27.136 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:27.137 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:27.137 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:27.137 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:27.137 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:27.137 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87210 2024-09-11T07:41:27.137 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.137 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:27.137 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.137 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:27.137 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341865+0000", 2024-09-11T07:41:27.137 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.138 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.138 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.138 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:27.138 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341865+0000", 2024-09-11T07:41:27.138 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.138 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.138 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.138 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:27.138 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341863+0000", 2024-09-11T07:41:27.138 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:41:27.138 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.139 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.139 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:27.139 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341890+0000", 2024-09-11T07:41:27.139 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7458999999999999e-05 2024-09-11T07:41:27.139 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.139 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.139 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:27.139 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341903+0000", 2024-09-11T07:41:27.139 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3132999999999999e-05 2024-09-11T07:41:27.139 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.139 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.139 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:27.140 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341911+0000", 2024-09-11T07:41:27.140 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.8699999999999992e-06 2024-09-11T07:41:27.140 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.140 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.140 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.140 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342621+0000", 2024-09-11T07:41:27.140 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00070930199999999998 2024-09-11T07:41:27.140 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.140 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.140 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.141 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342627+0000", 2024-09-11T07:41:27.141 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0129999999999999e-06 2024-09-11T07:41:27.141 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.141 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.141 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.141 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.519201+0000", 2024-09-11T07:41:27.141 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.176574606 2024-09-11T07:41:27.141 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.141 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.141 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:27.141 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.519326+0000", 2024-09-11T07:41:27.141 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012495799999999999 2024-09-11T07:41:27.142 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.142 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.142 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,5,6", 2024-09-11T07:41:27.142 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.519682+0000", 2024-09-11T07:41:27.142 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00035572700000000002 2024-09-11T07:41:27.142 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.142 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.142 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.142 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.560173+0000", 2024-09-11T07:41:27.142 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.040491003999999997 2024-09-11T07:41:27.143 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.143 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.143 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.143 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.560894+0000", 2024-09-11T07:41:27.143 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00072137700000000004 2024-09-11T07:41:27.143 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.143 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.143 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.143 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.561062+0000", 2024-09-11T07:41:27.143 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000167658 2024-09-11T07:41:27.144 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.144 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.144 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:27.144 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.564965+0000", 2024-09-11T07:41:27.144 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0039035150000000002 2024-09-11T07:41:27.144 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.144 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.144 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:27.144 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.565018+0000", 2024-09-11T07:41:27.144 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2139999999999999e-05 2024-09-11T07:41:27.144 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.144 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.145 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:27.145 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.567595+0000", 2024-09-11T07:41:27.145 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0025769970000000001 2024-09-11T07:41:27.145 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.145 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:27.145 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.145 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.145 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.145 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87222 9.18 9:190c7e59:::benchmark_data_smithi022_133943_object87221:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:41:27.145 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.341925+0000", 2024-09-11T07:41:27.145 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 177.67796775599999, 2024-09-11T07:41:27.145 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.119733941, 2024-09-11T07:41:27.146 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:27.146 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:27.146 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:27.146 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:27.146 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:27.146 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87222 2024-09-11T07:41:27.146 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.146 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:27.147 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.147 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:27.147 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341925+0000", 2024-09-11T07:41:27.147 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.147 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.147 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.147 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:27.147 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341925+0000", 2024-09-11T07:41:27.147 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.147 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.147 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.147 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:27.147 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341924+0000", 2024-09-11T07:41:27.148 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:41:27.148 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.148 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.148 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:27.148 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341945+0000", 2024-09-11T07:41:27.148 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1024000000000001e-05 2024-09-11T07:41:27.148 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.148 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.148 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:27.148 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341952+0000", 2024-09-11T07:41:27.148 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.545e-06 2024-09-11T07:41:27.148 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.148 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.149 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:27.149 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341956+0000", 2024-09-11T07:41:27.149 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7589999999999997e-06 2024-09-11T07:41:27.149 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.149 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.149 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.149 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342444+0000", 2024-09-11T07:41:27.149 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00048726500000000002 2024-09-11T07:41:27.149 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.149 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.149 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.149 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342452+0000", 2024-09-11T07:41:27.149 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.1810000000000001e-06 2024-09-11T07:41:27.150 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.150 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.150 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.150 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.449925+0000", 2024-09-11T07:41:27.150 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.107473197 2024-09-11T07:41:27.150 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.150 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.150 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:27.150 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.450079+0000", 2024-09-11T07:41:27.150 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000153475 2024-09-11T07:41:27.150 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.150 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.150 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,5,6,7", 2024-09-11T07:41:27.150 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.450248+0000", 2024-09-11T07:41:27.151 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00016988499999999999 2024-09-11T07:41:27.151 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.151 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.151 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.151 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.456414+0000", 2024-09-11T07:41:27.151 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0061653100000000002 2024-09-11T07:41:27.151 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.151 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.151 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.151 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.460051+0000", 2024-09-11T07:41:27.151 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0036376450000000001 2024-09-11T07:41:27.151 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.151 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.152 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.152 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.460085+0000", 2024-09-11T07:41:27.152 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3268000000000001e-05 2024-09-11T07:41:27.152 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.152 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.152 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.152 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.460115+0000", 2024-09-11T07:41:27.152 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0292999999999999e-05 2024-09-11T07:41:27.152 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.152 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.152 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:27.152 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461466+0000", 2024-09-11T07:41:27.152 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0013508859999999999 2024-09-11T07:41:27.152 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.153 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.153 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:27.153 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461524+0000", 2024-09-11T07:41:27.153 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8607000000000003e-05 2024-09-11T07:41:27.153 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.153 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.153 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:27.153 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461659+0000", 2024-09-11T07:41:27.153 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000134358 2024-09-11T07:41:27.153 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.153 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:27.153 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.153 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.154 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.154 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87225 9.19 9:9ba55202:::benchmark_data_smithi022_133943_object87224:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:41:27.154 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.341961+0000", 2024-09-11T07:41:27.154 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 177.67793172500001, 2024-09-11T07:41:27.154 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2429198539999999, 2024-09-11T07:41:27.154 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:27.154 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:27.154 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:27.154 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:27.154 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:27.154 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87225 2024-09-11T07:41:27.154 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.154 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:27.155 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.155 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:27.155 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341961+0000", 2024-09-11T07:41:27.155 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.155 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.155 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.155 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:27.155 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341961+0000", 2024-09-11T07:41:27.155 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.155 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.155 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.155 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:27.155 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341960+0000", 2024-09-11T07:41:27.156 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:41:27.156 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.156 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.156 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:27.156 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341979+0000", 2024-09-11T07:41:27.156 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8606000000000001e-05 2024-09-11T07:41:27.156 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.156 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.156 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:27.156 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341985+0000", 2024-09-11T07:41:27.156 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1319999999999999e-06 2024-09-11T07:41:27.156 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.156 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.157 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:27.157 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341989+0000", 2024-09-11T07:41:27.157 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9469999999999996e-06 2024-09-11T07:41:27.157 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.157 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.157 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.157 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342643+0000", 2024-09-11T07:41:27.157 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00065436200000000002 2024-09-11T07:41:27.157 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.157 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.157 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.157 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342648+0000", 2024-09-11T07:41:27.157 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6759999999999998e-06 2024-09-11T07:41:27.158 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.158 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.158 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.158 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.561081+0000", 2024-09-11T07:41:27.158 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2184332229999999 2024-09-11T07:41:27.158 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.158 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.158 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:27.158 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.561226+0000", 2024-09-11T07:41:27.158 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014486300000000001 2024-09-11T07:41:27.158 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.158 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.158 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,5,6", 2024-09-11T07:41:27.158 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.561922+0000", 2024-09-11T07:41:27.159 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00069608500000000004 2024-09-11T07:41:27.159 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.159 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.159 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.159 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.567646+0000", 2024-09-11T07:41:27.159 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0057242279999999996 2024-09-11T07:41:27.159 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.159 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.159 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.159 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.567751+0000", 2024-09-11T07:41:27.159 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000104591 2024-09-11T07:41:27.159 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.159 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.160 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:27.160 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.574420+0000", 2024-09-11T07:41:27.160 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0066688900000000002 2024-09-11T07:41:27.160 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.160 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.160 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.160 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.582333+0000", 2024-09-11T07:41:27.160 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0079128410000000003 2024-09-11T07:41:27.160 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.160 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.160 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:27.160 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.582396+0000", 2024-09-11T07:41:27.160 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2976000000000001e-05 2024-09-11T07:41:27.161 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.161 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.161 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:27.161 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.584881+0000", 2024-09-11T07:41:27.161 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002485012 2024-09-11T07:41:27.161 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.161 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:27.161 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.161 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.161 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.161 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87240 9.18 9:1d562e34:::benchmark_data_smithi022_133943_object87239:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:41:27.161 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.341993+0000", 2024-09-11T07:41:27.161 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 177.67789962000001, 2024-09-11T07:41:27.161 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.121035673, 2024-09-11T07:41:27.162 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:27.162 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:27.162 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:27.162 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:27.162 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:27.162 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87240 2024-09-11T07:41:27.162 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.162 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:27.162 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.162 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:27.162 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341993+0000", 2024-09-11T07:41:27.162 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.163 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.163 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.163 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:27.163 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341993+0000", 2024-09-11T07:41:27.163 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.163 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.163 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.163 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:27.163 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341992+0000", 2024-09-11T07:41:27.163 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:41:27.163 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.163 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.163 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:27.163 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342012+0000", 2024-09-11T07:41:27.164 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9117999999999999e-05 2024-09-11T07:41:27.164 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.164 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.164 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:27.164 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342017+0000", 2024-09-11T07:41:27.164 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7559999999999996e-06 2024-09-11T07:41:27.164 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.164 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.164 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:27.164 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342021+0000", 2024-09-11T07:41:27.164 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8639999999999998e-06 2024-09-11T07:41:27.164 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.164 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.165 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.165 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342489+0000", 2024-09-11T07:41:27.165 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00046751600000000001 2024-09-11T07:41:27.165 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.165 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.165 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.165 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342495+0000", 2024-09-11T07:41:27.165 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3720000000000001e-06 2024-09-11T07:41:27.165 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.165 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.165 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.165 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.452736+0000", 2024-09-11T07:41:27.165 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.110240694 2024-09-11T07:41:27.166 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.166 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.166 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:27.166 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.452864+0000", 2024-09-11T07:41:27.166 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012804299999999999 2024-09-11T07:41:27.166 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.166 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.166 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,5,6,7", 2024-09-11T07:41:27.166 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.453027+0000", 2024-09-11T07:41:27.166 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00016329100000000001 2024-09-11T07:41:27.166 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.166 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.166 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.166 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.459540+0000", 2024-09-11T07:41:27.167 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0065125319999999997 2024-09-11T07:41:27.167 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.167 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.167 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.167 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.460175+0000", 2024-09-11T07:41:27.167 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00063524399999999998 2024-09-11T07:41:27.167 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.167 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.167 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.167 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.460204+0000", 2024-09-11T07:41:27.168 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8762999999999999e-05 2024-09-11T07:41:27.168 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.168 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.168 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:27.168 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461822+0000", 2024-09-11T07:41:27.168 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0016181539999999999 2024-09-11T07:41:27.168 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.168 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.168 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.168 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.462813+0000", 2024-09-11T07:41:27.168 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00099143499999999993 2024-09-11T07:41:27.168 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.169 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.169 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:27.169 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.462901+0000", 2024-09-11T07:41:27.169 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.7409000000000003e-05 2024-09-11T07:41:27.169 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.169 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.169 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:27.169 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.463029+0000", 2024-09-11T07:41:27.169 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012804 2024-09-11T07:41:27.169 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.169 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:27.169 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.169 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.170 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.170 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235366 9.8 9:1702ff49:::benchmark_data_smithi022_133943_object108775:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e323)", 2024-09-11T07:41:27.170 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:02.747880+0000", 2024-09-11T07:41:27.170 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 24.2720129, 2024-09-11T07:41:27.170 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.96185844799999998, 2024-09-11T07:41:27.170 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:27.170 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:27.170 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:27.170 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:27.170 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:27.170 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235366 2024-09-11T07:41:27.170 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.170 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:27.171 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.171 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:27.171 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.747880+0000", 2024-09-11T07:41:27.171 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.171 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.171 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.171 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:27.171 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.747880+0000", 2024-09-11T07:41:27.171 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.171 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.171 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.171 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:27.171 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.747877+0000", 2024-09-11T07:41:27.171 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:41:27.172 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.172 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.172 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:27.172 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.747887+0000", 2024-09-11T07:41:27.172 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.9960000000000005e-06 2024-09-11T07:41:27.172 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.172 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.172 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:27.172 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.747913+0000", 2024-09-11T07:41:27.172 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.6363000000000002e-05 2024-09-11T07:41:27.172 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.172 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.172 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:27.173 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.747920+0000", 2024-09-11T07:41:27.173 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.2259999999999999e-06 2024-09-11T07:41:27.173 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.173 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.173 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.173 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.747967+0000", 2024-09-11T07:41:27.173 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6034e-05 2024-09-11T07:41:27.173 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.173 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.173 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.173 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.747974+0000", 2024-09-11T07:41:27.173 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.9370000000000004e-06 2024-09-11T07:41:27.174 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.174 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.174 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.174 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.706758+0000", 2024-09-11T07:41:27.174 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.95878366699999995 2024-09-11T07:41:27.174 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.174 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.174 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:27.174 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.706864+0000", 2024-09-11T07:41:27.174 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000105722 2024-09-11T07:41:27.174 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.174 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.174 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,7", 2024-09-11T07:41:27.175 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.706971+0000", 2024-09-11T07:41:27.175 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000107589 2024-09-11T07:41:27.175 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.175 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.175 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.175 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.709589+0000", 2024-09-11T07:41:27.175 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0026178730000000002 2024-09-11T07:41:27.175 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.175 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.175 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:27.175 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.709599+0000", 2024-09-11T07:41:27.175 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0149999999999999e-05 2024-09-11T07:41:27.175 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.176 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.176 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.176 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.709652+0000", 2024-09-11T07:41:27.176 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2627000000000003e-05 2024-09-11T07:41:27.176 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.176 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.176 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:27.176 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.709694+0000", 2024-09-11T07:41:27.176 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1956999999999998e-05 2024-09-11T07:41:27.176 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.176 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.176 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:27.176 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.709738+0000", 2024-09-11T07:41:27.177 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4122000000000002e-05 2024-09-11T07:41:27.177 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.177 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:27.177 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.177 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.177 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.177 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235378 9.8 9:17e72ff8:::benchmark_data_smithi022_133943_object108787:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e323)", 2024-09-11T07:41:27.177 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:02.759690+0000", 2024-09-11T07:41:27.177 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 24.260202884999998, 2024-09-11T07:41:27.177 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.96079267800000001, 2024-09-11T07:41:27.177 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:27.177 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:27.177 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:27.178 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:27.178 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:27.178 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235378 2024-09-11T07:41:27.178 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.178 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:27.178 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.178 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:27.178 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.759690+0000", 2024-09-11T07:41:27.178 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.178 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.178 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.178 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:27.178 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.759690+0000", 2024-09-11T07:41:27.178 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.179 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.179 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.179 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:27.179 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.759688+0000", 2024-09-11T07:41:27.179 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:41:27.179 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.179 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.179 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:27.179 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.759694+0000", 2024-09-11T07:41:27.179 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7400000000000001e-06 2024-09-11T07:41:27.179 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.179 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.179 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:27.179 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.759703+0000", 2024-09-11T07:41:27.179 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.5830000000000007e-06 2024-09-11T07:41:27.180 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.180 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.180 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:27.180 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.759708+0000", 2024-09-11T07:41:27.180 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.4199999999999998e-06 2024-09-11T07:41:27.180 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.180 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.180 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.180 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.759773+0000", 2024-09-11T07:41:27.180 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4772999999999998e-05 2024-09-11T07:41:27.180 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.180 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.181 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.181 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.759780+0000", 2024-09-11T07:41:27.181 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.2729999999999997e-06 2024-09-11T07:41:27.181 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.181 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.181 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.181 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.709785+0000", 2024-09-11T07:41:27.181 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.95000451399999997 2024-09-11T07:41:27.181 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.181 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.181 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:27.181 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.709890+0000", 2024-09-11T07:41:27.181 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010527300000000001 2024-09-11T07:41:27.182 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.182 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.182 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,7", 2024-09-11T07:41:27.182 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.709962+0000", 2024-09-11T07:41:27.182 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1935999999999997e-05 2024-09-11T07:41:27.182 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.182 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.182 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.182 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.714856+0000", 2024-09-11T07:41:27.182 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0048941230000000002 2024-09-11T07:41:27.182 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.182 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.182 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:27.182 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.714864+0000", 2024-09-11T07:41:27.183 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.7330000000000003e-06 2024-09-11T07:41:27.183 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.183 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.183 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.183 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.720399+0000", 2024-09-11T07:41:27.183 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.005534743 2024-09-11T07:41:27.183 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.183 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.183 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:27.183 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.720445+0000", 2024-09-11T07:41:27.183 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5998000000000002e-05 2024-09-11T07:41:27.183 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.183 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.184 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:27.184 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.720482+0000", 2024-09-11T07:41:27.184 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7861999999999998e-05 2024-09-11T07:41:27.184 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.184 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:27.184 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.184 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.184 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.184 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235388 9.8 9:129d98d7:::benchmark_data_smithi022_133943_object108797:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e323)", 2024-09-11T07:41:27.184 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:02.768761+0000", 2024-09-11T07:41:27.184 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 24.251131669999999, 2024-09-11T07:41:27.184 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.95185301700000002, 2024-09-11T07:41:27.184 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:27.185 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:27.185 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:27.185 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:27.185 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:27.185 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235388 2024-09-11T07:41:27.185 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.185 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:27.185 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.185 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:27.185 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768761+0000", 2024-09-11T07:41:27.185 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.185 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.185 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.185 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:27.186 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768761+0000", 2024-09-11T07:41:27.186 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.186 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.186 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.186 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:27.186 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768758+0000", 2024-09-11T07:41:27.186 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:41:27.186 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.186 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.186 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:27.186 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768766+0000", 2024-09-11T07:41:27.186 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5089999999999996e-06 2024-09-11T07:41:27.186 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.187 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.187 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:27.187 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768782+0000", 2024-09-11T07:41:27.187 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5985e-05 2024-09-11T07:41:27.187 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.187 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.187 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:27.187 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768788+0000", 2024-09-11T07:41:27.187 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2559999999999999e-06 2024-09-11T07:41:27.187 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.187 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.187 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.187 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768835+0000", 2024-09-11T07:41:27.187 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6455999999999997e-05 2024-09-11T07:41:27.188 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.188 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.188 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.188 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768843+0000", 2024-09-11T07:41:27.188 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.4209999999999995e-06 2024-09-11T07:41:27.188 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.188 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.188 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.188 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:27.189 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.712304+0000", 2024-09-11T07:41:27.190 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.94346046699999997 2024-09-11T07:41:27.190 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.190 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.190 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:27.190 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.712424+0000", 2024-09-11T07:41:27.190 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000119997 2024-09-11T07:41:27.190 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.190 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.190 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,7", 2024-09-11T07:41:27.190 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.712526+0000", 2024-09-11T07:41:27.190 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000102649 2024-09-11T07:41:27.190 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.190 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.720360+0000", 2024-09-11T07:41:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0078334990000000007 2024-09-11T07:41:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.720519+0000", 2024-09-11T07:41:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015941600000000001 2024-09-11T07:41:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.720547+0000", 2024-09-11T07:41:27.191 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8078000000000001e-05 2024-09-11T07:41:27.192 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.192 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.192 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:27.192 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.720582+0000", 2024-09-11T07:41:27.192 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4860000000000002e-05 2024-09-11T07:41:27.192 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.192 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.192 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:27.192 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.720614+0000", 2024-09-11T07:41:27.192 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.1931000000000003e-05 2024-09-11T07:41:27.192 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.192 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:27.192 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.193 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.193 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.193 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235390 9.8 9:17241c1b:::benchmark_data_smithi022_133943_object108799:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e323)", 2024-09-11T07:41:27.193 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:02.770070+0000", 2024-09-11T07:41:27.193 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 24.249822224999999, 2024-09-11T07:41:27.193 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.95338818199999997, 2024-09-11T07:41:27.193 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:27.193 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:27.193 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:27.193 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:27.193 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:27.193 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235390 2024-09-11T07:41:27.193 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.193 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:27.194 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.194 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:27.194 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.770070+0000", 2024-09-11T07:41:27.194 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.194 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.194 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.194 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:27.194 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.770070+0000", 2024-09-11T07:41:27.194 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:27.194 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.194 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.194 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:27.194 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.770069+0000", 2024-09-11T07:41:27.195 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:41:27.195 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.195 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.195 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:27.195 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.770076+0000", 2024-09-11T07:41:27.195 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0509999999999999e-06 2024-09-11T07:41:27.195 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.195 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.195 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:27.195 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.770099+0000", 2024-09-11T07:41:27.195 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2938e-05 2024-09-11T07:41:27.195 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.195 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.195 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:27.196 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.770105+0000", 2024-09-11T07:41:27.196 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9510000000000003e-06 2024-09-11T07:41:27.196 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.196 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.196 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.196 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.770145+0000", 2024-09-11T07:41:27.196 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0818999999999997e-05 2024-09-11T07:41:27.196 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.196 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.196 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:27.196 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.770152+0000", 2024-09-11T07:41:27.196 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9750000000000001e-06 2024-09-11T07:41:27.196 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.197 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.197 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:27.197 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.714887+0000", 2024-09-11T07:41:27.197 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.94473459699999995 2024-09-11T07:41:27.197 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.197 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.197 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:27.197 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.714990+0000", 2024-09-11T07:41:27.197 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000103251 2024-09-11T07:41:27.197 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.197 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.197 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,7", 2024-09-11T07:41:27.197 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.715109+0000", 2024-09-11T07:41:27.197 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000119255 2024-09-11T07:41:27.198 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.198 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.198 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.198 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.720644+0000", 2024-09-11T07:41:27.198 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0055347640000000002 2024-09-11T07:41:27.198 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.198 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.198 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:27.198 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.720669+0000", 2024-09-11T07:41:27.198 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4468999999999999e-05 2024-09-11T07:41:27.198 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.198 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.198 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:27.199 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.723232+0000", 2024-09-11T07:41:27.199 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0025635329999999998 2024-09-11T07:41:27.199 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.199 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.199 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:27.199 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.723417+0000", 2024-09-11T07:41:27.199 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00018494900000000001 2024-09-11T07:41:27.199 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:27.199 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:27.199 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:27.199 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.723459+0000", 2024-09-11T07:41:27.199 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1399000000000001e-05 2024-09-11T07:41:27.199 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.199 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:27.200 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.200 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:27.200 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:27.200 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:41:27.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:27.222 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:27.223 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:27.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:27 smithi022 ceph-mon[29462]: 4.7 scrub starts 2024-09-11T07:41:27.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:27 smithi022 ceph-mon[29462]: 4.7 scrub ok 2024-09-11T07:41:27.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:27 smithi022 ceph-mon[29462]: from='client.15738 v1:172.21.15.22:0/2532645936' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:41:27.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:27 smithi022 ceph-mon[29462]: 6.9 scrub starts 2024-09-11T07:41:27.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:27 smithi022 ceph-mon[29462]: 6.9 scrub ok 2024-09-11T07:41:27.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:27 smithi022 ceph-mon[29462]: 5.d scrub starts 2024-09-11T07:41:27.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:27 smithi022 ceph-mon[29462]: 5.d scrub ok 2024-09-11T07:41:27.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:27 smithi022 ceph-mon[29462]: pgmap v688: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 18 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 132/1416 objects misplaced (9.322%) 2024-09-11T07:41:27.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:27 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2148667905' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:41:27.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:27 smithi022 ceph-mon[29462]: from='client.25619 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:41:27.251 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:27 smithi022 ceph-mon[37330]: 4.7 scrub starts 2024-09-11T07:41:27.251 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:27 smithi022 ceph-mon[37330]: 4.7 scrub ok 2024-09-11T07:41:27.252 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:27 smithi022 ceph-mon[37330]: from='client.15738 v1:172.21.15.22:0/2532645936' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:41:27.252 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:27 smithi022 ceph-mon[37330]: 6.9 scrub starts 2024-09-11T07:41:27.252 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:27 smithi022 ceph-mon[37330]: 6.9 scrub ok 2024-09-11T07:41:27.252 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:27 smithi022 ceph-mon[37330]: 5.d scrub starts 2024-09-11T07:41:27.252 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:27 smithi022 ceph-mon[37330]: 5.d scrub ok 2024-09-11T07:41:27.252 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:27 smithi022 ceph-mon[37330]: pgmap v688: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 18 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 132/1416 objects misplaced (9.322%) 2024-09-11T07:41:27.252 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:27 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2148667905' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:41:27.252 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:27 smithi022 ceph-mon[37330]: from='client.25619 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:41:27.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:27 smithi104 ceph-mon[33507]: 4.7 scrub starts 2024-09-11T07:41:27.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:27 smithi104 ceph-mon[33507]: 4.7 scrub ok 2024-09-11T07:41:27.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:27 smithi104 ceph-mon[33507]: from='client.15738 v1:172.21.15.22:0/2532645936' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:41:27.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:27 smithi104 ceph-mon[33507]: 6.9 scrub starts 2024-09-11T07:41:27.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:27 smithi104 ceph-mon[33507]: 6.9 scrub ok 2024-09-11T07:41:27.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:27 smithi104 ceph-mon[33507]: 5.d scrub starts 2024-09-11T07:41:27.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:27 smithi104 ceph-mon[33507]: 5.d scrub ok 2024-09-11T07:41:27.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:27 smithi104 ceph-mon[33507]: pgmap v688: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 18 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 132/1416 objects misplaced (9.322%) 2024-09-11T07:41:27.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:27 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2148667905' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:41:27.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:27 smithi104 ceph-mon[33507]: from='client.25619 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:41:27.324 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:27.346 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:27.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:27.347 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:27.448 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:27.476 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:27.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:27.477 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:27.578 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:27.604 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:27.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:27.606 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:27.707 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:27.734 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:27.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:27.735 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:27.836 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:27.858 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:27.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:27.860 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:27.961 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:27.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:27.984 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:27.984 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:28.085 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:28.108 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:28.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:28.109 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:28.210 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:28.214 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":687,"stamp":"2024-09-11T07:41:24.266839+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":18716892,"num_objects":472,"num_object_clones":0,"num_object_copies":1416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":132,"num_objects_unfound":0,"num_objects_dirty":472,"num_whiteouts":0,"num_read":2154,"num_read_kb":2011,"num_write":246022,"num_write_kb":7848456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45703,"num_bytes_recovered":2987201337,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129716,"ondisk_log_size":129716,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":503,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":4828744,"kb_used_data":583404,"kb_used_omap":16,"kb_used_meta":4245231,"kb_avail":745132472,"statfs":{"total":767960285184,"available":763015651328,"internally_reserved":0,"allocated":597405696,"data_stored":593637768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16530,"internal_metadata":4347117422},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-224428056,"num_objects":-3426,"num_object_clones":0,"num_object_copies":-10278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-671,"num_objects_unfound":0,"num_objects_dirty":-3426,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":-4283,"num_write_kb":-254848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":-131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001747"},"pg_stats":[{"pgid":"9.14","version":"326'10610","reported_seq":10739,"reported_epoch":327,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-11T07:41:18.687788+0000","last_change":"2024-09-11T07:41:12.796036+0000","last_active":"2024-09-11T07:41:18.687788+0000","last_peered":"2024-09-11T07:41:18.687788+0000","last_clean":"2024-09-11T07:39:56.654544+0000","last_became_active":"2024-09-11T07:40:12.755113+0000","last_became_peered":"2024-09-11T07:40:12.755113+0000","last_unstale":"2024-09-11T07:41:18.687788+0000","last_undegraded":"2024-09-11T07:41:18.687788+0000","last_fullsized":"2024-09-11T07:41:18.687788+0000","mapping_epoch":297,"log_start":"272'6232","ondisk_log_start":"272'6232","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4378,"log_dups_size":0,"ondisk_log_size":4378,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":66,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7923,"num_write_kb":254592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,2],"acting":[4,1,3],"avail_no_missing":["4","1","3"],"object_location_counts":[{"shards":"1,3,4","objects":33}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":145,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.674510+0000","last_change":"2024-09-11T07:40:39.605681+0000","last_active":"2024-09-11T07:41:15.674510+0000","last_peered":"2024-09-11T07:41:15.674510+0000","last_clean":"2024-09-11T07:41:15.674510+0000","last_became_active":"2024-09-11T07:40:39.605361+0000","last_became_peered":"2024-09-11T07:40:39.605361+0000","last_unstale":"2024-09-11T07:41:15.674510+0000","last_undegraded":"2024-09-11T07:41:15.674510+0000","last_fullsized":"2024-09-11T07:41:15.674510+0000","mapping_epoch":310,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":311,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:29.007411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:29.007411+0000","last_clean_scrub_stamp":"2024-09-11T07:37:29.007411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001472253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":355,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.677304+0000","last_change":"2024-09-11T07:38:27.418908+0000","last_active":"2024-09-11T07:41:15.677304+0000","last_peered":"2024-09-11T07:41:15.677304+0000","last_clean":"2024-09-11T07:41:15.677304+0000","last_became_active":"2024-09-11T07:32:00.508572+0000","last_became_peered":"2024-09-11T07:32:00.508572+0000","last_unstale":"2024-09-11T07:41:15.677304+0000","last_undegraded":"2024-09-11T07:41:15.677304+0000","last_fullsized":"2024-09-11T07:41:15.677304+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:27.418829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:38:27.418829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036972480000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"234'34","reported_seq":379,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.679313+0000","last_change":"2024-09-11T07:40:39.607640+0000","last_active":"2024-09-11T07:41:15.679313+0000","last_peered":"2024-09-11T07:41:15.679313+0000","last_clean":"2024-09-11T07:41:15.679313+0000","last_became_active":"2024-09-11T07:26:33.544627+0000","last_became_peered":"2024-09-11T07:26:33.544627+0000","last_unstale":"2024-09-11T07:41:15.679313+0000","last_undegraded":"2024-09-11T07:41:15.679313+0000","last_fullsized":"2024-09-11T07:41:15.679313+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'34","last_scrub_stamp":"2024-09-11T07:40:39.607579+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:40:39.607579+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:51.954396+0000","scrub_duration":0.0054431660000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":345,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.687409+0000","last_change":"2024-09-11T07:39:56.545948+0000","last_active":"2024-09-11T07:41:18.687409+0000","last_peered":"2024-09-11T07:41:18.687409+0000","last_clean":"2024-09-11T07:41:18.687409+0000","last_became_active":"2024-09-11T07:26:35.556296+0000","last_became_peered":"2024-09-11T07:26:35.556296+0000","last_unstale":"2024-09-11T07:41:18.687409+0000","last_undegraded":"2024-09-11T07:41:18.687409+0000","last_fullsized":"2024-09-11T07:41:18.687409+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:56.545872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:39:56.545872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:24.540468+0000","scrub_duration":0.002240283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.15","version":"326'10683","reported_seq":23171,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:17.962374+0000","last_change":"2024-09-11T07:40:10.796286+0000","last_active":"2024-09-11T07:41:17.962374+0000","last_peered":"2024-09-11T07:41:17.962374+0000","last_clean":"2024-09-11T07:41:17.962374+0000","last_became_active":"2024-09-11T07:40:10.762538+0000","last_became_peered":"2024-09-11T07:40:10.762538+0000","last_unstale":"2024-09-11T07:41:17.962374+0000","last_undegraded":"2024-09-11T07:41:17.962374+0000","last_fullsized":"2024-09-11T07:41:17.962374+0000","mapping_epoch":295,"log_start":"256'4623","ondisk_log_start":"256'4623","created":259,"last_epoch_clean":296,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6060,"log_dups_size":0,"ondisk_log_size":6060,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7969,"num_write_kb":254912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4795,"num_bytes_recovered":314179584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":351,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.680104+0000","last_change":"2024-09-11T07:40:35.645201+0000","last_active":"2024-09-11T07:41:15.680104+0000","last_peered":"2024-09-11T07:41:15.680104+0000","last_clean":"2024-09-11T07:41:15.680104+0000","last_became_active":"2024-09-11T07:39:57.767806+0000","last_became_peered":"2024-09-11T07:39:57.767806+0000","last_unstale":"2024-09-11T07:41:15.680104+0000","last_undegraded":"2024-09-11T07:41:15.680104+0000","last_fullsized":"2024-09-11T07:41:15.680104+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:35.645119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:40:35.645119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:00.870748+0000","scrub_duration":0.0071309060000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"234'19","reported_seq":372,"reported_epoch":328,"state":"active+clean","last_fresh":"2024-09-11T07:41:21.709354+0000","last_change":"2024-09-11T07:41:21.709325+0000","last_active":"2024-09-11T07:41:21.709354+0000","last_peered":"2024-09-11T07:41:21.709354+0000","last_clean":"2024-09-11T07:41:21.709354+0000","last_became_active":"2024-09-11T07:26:33.540732+0000","last_became_peered":"2024-09-11T07:26:33.540732+0000","last_unstale":"2024-09-11T07:41:21.709354+0000","last_undegraded":"2024-09-11T07:41:21.709354+0000","last_fullsized":"2024-09-11T07:41:21.709354+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:41:21.709271+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:36:42.708941+0000","last_clean_scrub_stamp":"2024-09-11T07:41:21.709271+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:28.521264+0000","scrub_duration":0.001672682,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":343,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.673989+0000","last_change":"2024-09-11T07:37:43.087423+0000","last_active":"2024-09-11T07:41:15.673989+0000","last_peered":"2024-09-11T07:41:15.673989+0000","last_clean":"2024-09-11T07:41:15.673989+0000","last_became_active":"2024-09-11T07:32:00.513650+0000","last_became_peered":"2024-09-11T07:32:00.513650+0000","last_unstale":"2024-09-11T07:41:15.673989+0000","last_undegraded":"2024-09-11T07:41:15.673989+0000","last_fullsized":"2024-09-11T07:41:15.673989+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:43.087302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:37:43.087302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00405653,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":356,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.687488+0000","last_change":"2024-09-11T07:40:36.708489+0000","last_active":"2024-09-11T07:41:18.687488+0000","last_peered":"2024-09-11T07:41:18.687488+0000","last_clean":"2024-09-11T07:41:18.687488+0000","last_became_active":"2024-09-11T07:40:11.698235+0000","last_became_peered":"2024-09-11T07:40:11.698235+0000","last_unstale":"2024-09-11T07:41:18.687488+0000","last_undegraded":"2024-09-11T07:41:18.687488+0000","last_fullsized":"2024-09-11T07:41:18.687488+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:36.708406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:40:36.708406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:50.355594+0000","scrub_duration":0.0020771840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"326'10809","reported_seq":19872,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.687874+0000","last_change":"2024-09-11T07:40:36.515700+0000","last_active":"2024-09-11T07:41:18.687874+0000","last_peered":"2024-09-11T07:41:18.687874+0000","last_clean":"2024-09-11T07:41:18.687874+0000","last_became_active":"2024-09-11T07:40:36.499497+0000","last_became_peered":"2024-09-11T07:40:36.499497+0000","last_unstale":"2024-09-11T07:41:18.687874+0000","last_undegraded":"2024-09-11T07:41:18.687874+0000","last_fullsized":"2024-09-11T07:41:18.687874+0000","mapping_epoch":307,"log_start":"272'6365","ondisk_log_start":"272'6365","created":259,"last_epoch_clean":308,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4444,"log_dups_size":0,"ondisk_log_size":4444,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8045,"num_write_kb":257728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4458,"num_bytes_recovered":292159488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":333,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.684380+0000","last_change":"2024-09-11T07:37:34.542508+0000","last_active":"2024-09-11T07:41:15.684380+0000","last_peered":"2024-09-11T07:41:15.684380+0000","last_clean":"2024-09-11T07:41:15.684380+0000","last_became_active":"2024-09-11T07:32:00.516662+0000","last_became_peered":"2024-09-11T07:32:00.516662+0000","last_unstale":"2024-09-11T07:41:15.684380+0000","last_undegraded":"2024-09-11T07:41:15.684380+0000","last_fullsized":"2024-09-11T07:41:15.684380+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:34.542441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:34.542441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015726010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"234'9","reported_seq":368,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.687590+0000","last_change":"2024-09-11T07:39:57.773949+0000","last_active":"2024-09-11T07:41:18.687590+0000","last_peered":"2024-09-11T07:41:18.687590+0000","last_clean":"2024-09-11T07:41:18.687590+0000","last_became_active":"2024-09-11T07:39:57.765606+0000","last_became_peered":"2024-09-11T07:39:57.765606+0000","last_unstale":"2024-09-11T07:41:18.687590+0000","last_undegraded":"2024-09-11T07:41:18.687590+0000","last_fullsized":"2024-09-11T07:41:18.687590+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'9","last_scrub_stamp":"2024-09-11T07:37:37.558036+0000","last_deep_scrub":"82'7","last_deep_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_clean_scrub_stamp":"2024-09-11T07:37:37.558036+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034408770000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":312,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.683381+0000","last_change":"2024-09-11T07:41:15.577924+0000","last_active":"2024-09-11T07:41:15.683381+0000","last_peered":"2024-09-11T07:41:15.683381+0000","last_clean":"2024-09-11T07:41:15.683381+0000","last_became_active":"2024-09-11T07:32:00.513206+0000","last_became_peered":"2024-09-11T07:32:00.513206+0000","last_unstale":"2024-09-11T07:41:15.683381+0000","last_undegraded":"2024-09-11T07:41:15.683381+0000","last_fullsized":"2024-09-11T07:41:15.683381+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:15.577841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.973323+0000","last_clean_scrub_stamp":"2024-09-11T07:41:15.577841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:18.259405+0000","scrub_duration":0.0059899519999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":61,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.672868+0000","last_change":"2024-09-11T07:40:11.703754+0000","last_active":"2024-09-11T07:41:15.672868+0000","last_peered":"2024-09-11T07:41:15.672868+0000","last_clean":"2024-09-11T07:41:15.672868+0000","last_became_active":"2024-09-11T07:40:11.701560+0000","last_became_peered":"2024-09-11T07:40:11.701560+0000","last_unstale":"2024-09-11T07:41:15.672868+0000","last_undegraded":"2024-09-11T07:41:15.672868+0000","last_fullsized":"2024-09-11T07:41:15.672868+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:36.462794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:45.149916+0000","last_clean_scrub_stamp":"2024-09-11T07:37:36.462794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"326'10701","reported_seq":19695,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:17.969839+0000","last_change":"2024-09-11T07:40:26.422385+0000","last_active":"2024-09-11T07:41:17.969839+0000","last_peered":"2024-09-11T07:41:17.969839+0000","last_clean":"2024-09-11T07:41:17.969839+0000","last_became_active":"2024-09-11T07:40:26.404009+0000","last_became_peered":"2024-09-11T07:40:26.404009+0000","last_unstale":"2024-09-11T07:41:17.969839+0000","last_undegraded":"2024-09-11T07:41:17.969839+0000","last_fullsized":"2024-09-11T07:41:17.969839+0000","mapping_epoch":301,"log_start":"268'5945","ondisk_log_start":"268'5945","created":259,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4756,"log_dups_size":0,"ondisk_log_size":4756,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7997,"num_write_kb":254848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3243,"num_bytes_recovered":212533248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":298,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.675750+0000","last_change":"2024-09-11T07:37:40.336139+0000","last_active":"2024-09-11T07:41:15.675750+0000","last_peered":"2024-09-11T07:41:15.675750+0000","last_clean":"2024-09-11T07:41:15.675750+0000","last_became_active":"2024-09-11T07:32:00.505709+0000","last_became_peered":"2024-09-11T07:32:00.505709+0000","last_unstale":"2024-09-11T07:41:15.675750+0000","last_undegraded":"2024-09-11T07:41:15.675750+0000","last_fullsized":"2024-09-11T07:41:15.675750+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:40.336050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_clean_scrub_stamp":"2024-09-11T07:37:40.336050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021490649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"234'19","reported_seq":359,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.687895+0000","last_change":"2024-09-11T07:39:39.571267+0000","last_active":"2024-09-11T07:41:18.687895+0000","last_peered":"2024-09-11T07:41:18.687895+0000","last_clean":"2024-09-11T07:41:18.687895+0000","last_became_active":"2024-09-11T07:26:33.544386+0000","last_became_peered":"2024-09-11T07:26:33.544386+0000","last_unstale":"2024-09-11T07:41:18.687895+0000","last_undegraded":"2024-09-11T07:41:18.687895+0000","last_fullsized":"2024-09-11T07:41:18.687895+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'13","last_scrub_stamp":"2024-09-11T07:36:27.746295+0000","last_deep_scrub":"136'13","last_deep_scrub_stamp":"2024-09-11T07:31:08.042198+0000","last_clean_scrub_stamp":"2024-09-11T07:36:27.746295+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0089695690000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":350,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.675845+0000","last_change":"2024-09-11T07:40:11.700127+0000","last_active":"2024-09-11T07:41:15.675845+0000","last_peered":"2024-09-11T07:41:15.675845+0000","last_clean":"2024-09-11T07:41:15.675845+0000","last_became_active":"2024-09-11T07:40:11.699772+0000","last_became_peered":"2024-09-11T07:40:11.699772+0000","last_unstale":"2024-09-11T07:41:15.675845+0000","last_undegraded":"2024-09-11T07:41:15.675845+0000","last_fullsized":"2024-09-11T07:41:15.675845+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:27.145883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:38:27.145883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011677339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":349,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.675223+0000","last_change":"2024-09-11T07:39:51.811562+0000","last_active":"2024-09-11T07:41:15.675223+0000","last_peered":"2024-09-11T07:41:15.675223+0000","last_clean":"2024-09-11T07:41:15.675223+0000","last_became_active":"2024-09-11T07:32:00.530335+0000","last_became_peered":"2024-09-11T07:32:00.530335+0000","last_unstale":"2024-09-11T07:41:15.675223+0000","last_undegraded":"2024-09-11T07:41:15.675223+0000","last_fullsized":"2024-09-11T07:41:15.675223+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:51.811287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:39:51.811287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:21.373975+0000","scrub_duration":0.0069635549999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"326'10613","reported_seq":11898,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:17.969017+0000","last_change":"2024-09-11T07:41:02.711634+0000","last_active":"2024-09-11T07:41:17.969017+0000","last_peered":"2024-09-11T07:41:17.969017+0000","last_clean":"2024-09-11T07:41:17.969017+0000","last_became_active":"2024-09-11T07:41:02.685660+0000","last_became_peered":"2024-09-11T07:41:02.685660+0000","last_unstale":"2024-09-11T07:41:17.969017+0000","last_undegraded":"2024-09-11T07:41:17.969017+0000","last_fullsized":"2024-09-11T07:41:17.969017+0000","mapping_epoch":322,"log_start":"258'5341","ondisk_log_start":"258'5341","created":247,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":5272,"log_dups_size":0,"ondisk_log_size":5272,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3407872,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7894,"num_write_kb":254272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":577,"num_bytes_recovered":37814272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"234'31","reported_seq":435,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.688083+0000","last_change":"2024-09-11T07:39:41.579223+0000","last_active":"2024-09-11T07:41:18.688083+0000","last_peered":"2024-09-11T07:41:18.688083+0000","last_clean":"2024-09-11T07:41:18.688083+0000","last_became_active":"2024-09-11T07:29:51.482755+0000","last_became_peered":"2024-09-11T07:29:51.482755+0000","last_unstale":"2024-09-11T07:41:18.688083+0000","last_undegraded":"2024-09-11T07:41:18.688083+0000","last_fullsized":"2024-09-11T07:41:18.688083+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'21","last_scrub_stamp":"2024-09-11T07:36:39.736944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:36:39.736944+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034170279999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":331,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.683872+0000","last_change":"2024-09-11T07:39:44.385573+0000","last_active":"2024-09-11T07:41:15.683872+0000","last_peered":"2024-09-11T07:41:15.683872+0000","last_clean":"2024-09-11T07:41:15.683872+0000","last_became_active":"2024-09-11T07:32:00.507191+0000","last_became_peered":"2024-09-11T07:32:00.507191+0000","last_unstale":"2024-09-11T07:41:15.683872+0000","last_undegraded":"2024-09-11T07:41:15.683872+0000","last_fullsized":"2024-09-11T07:41:15.683872+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:51.418258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:51.418258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001500181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":326,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.676115+0000","last_change":"2024-09-11T07:38:22.027909+0000","last_active":"2024-09-11T07:41:15.676115+0000","last_peered":"2024-09-11T07:41:15.676115+0000","last_clean":"2024-09-11T07:41:15.676115+0000","last_became_active":"2024-09-11T07:26:35.557989+0000","last_became_peered":"2024-09-11T07:26:35.557989+0000","last_unstale":"2024-09-11T07:41:15.676115+0000","last_undegraded":"2024-09-11T07:41:15.676115+0000","last_fullsized":"2024-09-11T07:41:15.676115+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:22.027769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:38:22.027769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060071020000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":345,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.688214+0000","last_change":"2024-09-11T07:39:41.220688+0000","last_active":"2024-09-11T07:41:15.688214+0000","last_peered":"2024-09-11T07:41:15.688214+0000","last_clean":"2024-09-11T07:41:15.688214+0000","last_became_active":"2024-09-11T07:26:37.564554+0000","last_became_peered":"2024-09-11T07:26:37.564554+0000","last_unstale":"2024-09-11T07:41:15.688214+0000","last_undegraded":"2024-09-11T07:41:15.688214+0000","last_fullsized":"2024-09-11T07:41:15.688214+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:41.220589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:39:41.220589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066621019999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"326'10351","reported_seq":14310,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:17.940540+0000","last_change":"2024-09-11T07:40:10.728235+0000","last_active":"2024-09-11T07:41:17.940540+0000","last_peered":"2024-09-11T07:41:17.940540+0000","last_clean":"2024-09-11T07:41:17.940540+0000","last_became_active":"2024-09-11T07:40:10.693596+0000","last_became_peered":"2024-09-11T07:40:10.693596+0000","last_unstale":"2024-09-11T07:41:17.940540+0000","last_undegraded":"2024-09-11T07:41:17.940540+0000","last_fullsized":"2024-09-11T07:41:17.940540+0000","mapping_epoch":295,"log_start":"266'5763","ondisk_log_start":"266'5763","created":247,"last_epoch_clean":296,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4588,"log_dups_size":0,"ondisk_log_size":4588,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6029312,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7674,"num_write_kb":248512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1918,"num_bytes_recovered":125698048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"234'25","reported_seq":375,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.688167+0000","last_change":"2024-09-11T07:39:33.667725+0000","last_active":"2024-09-11T07:41:18.688167+0000","last_peered":"2024-09-11T07:41:18.688167+0000","last_clean":"2024-09-11T07:41:18.688167+0000","last_became_active":"2024-09-11T07:26:33.535411+0000","last_became_peered":"2024-09-11T07:26:33.535411+0000","last_unstale":"2024-09-11T07:41:18.688167+0000","last_undegraded":"2024-09-11T07:41:18.688167+0000","last_fullsized":"2024-09-11T07:41:18.688167+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'25","last_scrub_stamp":"2024-09-11T07:39:33.667530+0000","last_deep_scrub":"112'15","last_deep_scrub_stamp":"2024-09-11T07:31:10.005896+0000","last_clean_scrub_stamp":"2024-09-11T07:39:33.667530+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.041705696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":346,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.676170+0000","last_change":"2024-09-11T07:37:36.098229+0000","last_active":"2024-09-11T07:41:15.676170+0000","last_peered":"2024-09-11T07:41:15.676170+0000","last_clean":"2024-09-11T07:41:15.676170+0000","last_became_active":"2024-09-11T07:32:00.506615+0000","last_became_peered":"2024-09-11T07:32:00.506615+0000","last_unstale":"2024-09-11T07:41:15.676170+0000","last_undegraded":"2024-09-11T07:41:15.676170+0000","last_fullsized":"2024-09-11T07:41:15.676170+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:36.098175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:36.098175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017555839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"299'11","reported_seq":555,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.688296+0000","last_change":"2024-09-11T07:40:11.703968+0000","last_active":"2024-09-11T07:41:15.688296+0000","last_peered":"2024-09-11T07:41:15.688296+0000","last_clean":"2024-09-11T07:41:15.688296+0000","last_became_active":"2024-09-11T07:40:11.697568+0000","last_became_peered":"2024-09-11T07:40:11.697568+0000","last_unstale":"2024-09-11T07:41:15.688296+0000","last_undegraded":"2024-09-11T07:41:15.688296+0000","last_fullsized":"2024-09-11T07:41:15.688296+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:37:43.140999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:43.140999+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039651130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":343,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.688131+0000","last_change":"2024-09-11T07:39:57.698127+0000","last_active":"2024-09-11T07:41:18.688131+0000","last_peered":"2024-09-11T07:41:18.688131+0000","last_clean":"2024-09-11T07:41:18.688131+0000","last_became_active":"2024-09-11T07:39:57.697588+0000","last_became_peered":"2024-09-11T07:39:57.697588+0000","last_unstale":"2024-09-11T07:41:18.688131+0000","last_undegraded":"2024-09-11T07:41:18.688131+0000","last_fullsized":"2024-09-11T07:41:18.688131+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:52.605883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:39:52.605883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:21.117617+0000","scrub_duration":0.0013626420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"326'10475","reported_seq":13204,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:17.982261+0000","last_change":"2024-09-11T07:40:37.540503+0000","last_active":"2024-09-11T07:41:17.982261+0000","last_peered":"2024-09-11T07:41:17.982261+0000","last_clean":"2024-09-11T07:41:17.982261+0000","last_became_active":"2024-09-11T07:40:37.499654+0000","last_became_peered":"2024-09-11T07:40:37.499654+0000","last_unstale":"2024-09-11T07:41:17.982261+0000","last_undegraded":"2024-09-11T07:41:17.982261+0000","last_fullsized":"2024-09-11T07:41:17.982261+0000","mapping_epoch":308,"log_start":"266'5753","ondisk_log_start":"266'5753","created":247,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4722,"log_dups_size":0,"ondisk_log_size":4722,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7813,"num_write_kb":250112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1302,"num_bytes_recovered":85327872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"234'34","reported_seq":375,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.677509+0000","last_change":"2024-09-11T07:37:16.938565+0000","last_active":"2024-09-11T07:41:15.677509+0000","last_peered":"2024-09-11T07:41:15.677509+0000","last_clean":"2024-09-11T07:41:15.677509+0000","last_became_active":"2024-09-11T07:26:33.549733+0000","last_became_peered":"2024-09-11T07:26:33.549733+0000","last_unstale":"2024-09-11T07:41:15.677509+0000","last_undegraded":"2024-09-11T07:41:15.677509+0000","last_fullsized":"2024-09-11T07:41:15.677509+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'34","last_scrub_stamp":"2024-09-11T07:37:16.938496+0000","last_deep_scrub":"83'22","last_deep_scrub_stamp":"2024-09-11T07:34:04.375457+0000","last_clean_scrub_stamp":"2024-09-11T07:37:16.938496+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028468529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":357,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.674146+0000","last_change":"2024-09-11T07:38:29.939925+0000","last_active":"2024-09-11T07:41:15.674146+0000","last_peered":"2024-09-11T07:41:15.674146+0000","last_clean":"2024-09-11T07:41:15.674146+0000","last_became_active":"2024-09-11T07:32:00.513472+0000","last_became_peered":"2024-09-11T07:32:00.513472+0000","last_unstale":"2024-09-11T07:41:15.674146+0000","last_undegraded":"2024-09-11T07:41:15.674146+0000","last_fullsized":"2024-09-11T07:41:15.674146+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:29.939826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:38:29.939826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017750210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":524,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:19.685311+0000","last_change":"2024-09-11T07:40:46.534047+0000","last_active":"2024-09-11T07:41:19.685311+0000","last_peered":"2024-09-11T07:41:19.685311+0000","last_clean":"2024-09-11T07:41:19.685311+0000","last_became_active":"2024-09-11T07:26:35.560406+0000","last_became_peered":"2024-09-11T07:26:35.560406+0000","last_unstale":"2024-09-11T07:41:19.685311+0000","last_undegraded":"2024-09-11T07:41:19.685311+0000","last_fullsized":"2024-09-11T07:41:19.685311+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:40:46.533966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:40:46.533966+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:51.243657+0000","scrub_duration":0.0050834449999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":332,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.675719+0000","last_change":"2024-09-11T07:39:57.762607+0000","last_active":"2024-09-11T07:41:15.675719+0000","last_peered":"2024-09-11T07:41:15.675719+0000","last_clean":"2024-09-11T07:41:15.675719+0000","last_became_active":"2024-09-11T07:39:57.762309+0000","last_became_peered":"2024-09-11T07:39:57.762309+0000","last_unstale":"2024-09-11T07:41:15.675719+0000","last_undegraded":"2024-09-11T07:41:15.675719+0000","last_fullsized":"2024-09-11T07:41:15.675719+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:20.369199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:31.200685+0000","last_clean_scrub_stamp":"2024-09-11T07:37:20.369199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001371688,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"326'10375","reported_seq":10476,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:17.924292+0000","last_change":"2024-09-11T07:40:45.528413+0000","last_active":"2024-09-11T07:41:17.924292+0000","last_peered":"2024-09-11T07:41:17.924292+0000","last_clean":"2024-09-11T07:41:17.924292+0000","last_became_active":"2024-09-11T07:38:27.361246+0000","last_became_peered":"2024-09-11T07:38:27.361246+0000","last_unstale":"2024-09-11T07:41:17.924292+0000","last_undegraded":"2024-09-11T07:41:17.924292+0000","last_fullsized":"2024-09-11T07:41:17.924292+0000","mapping_epoch":247,"log_start":"256'4207","ondisk_log_start":"256'4207","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":6168,"log_dups_size":0,"ondisk_log_size":6168,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":688128,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7722,"num_write_kb":247456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"234'24","reported_seq":405,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.688236+0000","last_change":"2024-09-11T07:40:38.740191+0000","last_active":"2024-09-11T07:41:18.688236+0000","last_peered":"2024-09-11T07:41:18.688236+0000","last_clean":"2024-09-11T07:41:18.688236+0000","last_became_active":"2024-09-11T07:39:57.766134+0000","last_became_peered":"2024-09-11T07:39:57.766134+0000","last_unstale":"2024-09-11T07:41:18.688236+0000","last_undegraded":"2024-09-11T07:41:18.688236+0000","last_fullsized":"2024-09-11T07:41:18.688236+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'24","last_scrub_stamp":"2024-09-11T07:40:38.740127+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-09-11T07:29:38.112111+0000","last_clean_scrub_stamp":"2024-09-11T07:40:38.740127+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:52.114070+0000","scrub_duration":0.0044589670000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":335,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.636181+0000","last_change":"2024-09-11T07:41:18.636124+0000","last_active":"2024-09-11T07:41:18.636181+0000","last_peered":"2024-09-11T07:41:18.636181+0000","last_clean":"2024-09-11T07:41:18.636181+0000","last_became_active":"2024-09-11T07:32:00.518538+0000","last_became_peered":"2024-09-11T07:32:00.518538+0000","last_unstale":"2024-09-11T07:41:18.636181+0000","last_undegraded":"2024-09-11T07:41:18.636181+0000","last_fullsized":"2024-09-11T07:41:18.636181+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:18.636068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:41:18.636068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:47.127326+0000","scrub_duration":0.001280811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":334,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.675235+0000","last_change":"2024-09-11T07:38:24.129487+0000","last_active":"2024-09-11T07:41:15.675235+0000","last_peered":"2024-09-11T07:41:15.675235+0000","last_clean":"2024-09-11T07:41:15.675235+0000","last_became_active":"2024-09-11T07:26:35.569081+0000","last_became_peered":"2024-09-11T07:26:35.569081+0000","last_unstale":"2024-09-11T07:41:15.675235+0000","last_undegraded":"2024-09-11T07:41:15.675235+0000","last_fullsized":"2024-09-11T07:41:15.675235+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:24.129047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:38:24.129047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003571196,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":322,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.682768+0000","last_change":"2024-09-11T07:36:50.920711+0000","last_active":"2024-09-11T07:41:15.682768+0000","last_peered":"2024-09-11T07:41:15.682768+0000","last_clean":"2024-09-11T07:41:15.682768+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:41:15.682768+0000","last_undegraded":"2024-09-11T07:41:15.682768+0000","last_fullsized":"2024-09-11T07:41:15.682768+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:50.920625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:46.853387+0000","last_clean_scrub_stamp":"2024-09-11T07:36:50.920625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012404324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"326'15666","reported_seq":15753,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:17.979208+0000","last_change":"2024-09-11T07:37:25.855661+0000","last_active":"2024-09-11T07:41:17.979208+0000","last_peered":"2024-09-11T07:41:17.979208+0000","last_clean":"2024-09-11T07:41:17.979208+0000","last_became_active":"2024-09-11T07:37:25.854796+0000","last_became_peered":"2024-09-11T07:37:25.854796+0000","last_unstale":"2024-09-11T07:41:17.979208+0000","last_undegraded":"2024-09-11T07:41:17.979208+0000","last_fullsized":"2024-09-11T07:41:17.979208+0000","mapping_epoch":247,"log_start":"287'11106","ondisk_log_start":"287'11106","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4560,"log_dups_size":0,"ondisk_log_size":4560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15666,"num_write_kb":501312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"234'28","reported_seq":379,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.687588+0000","last_change":"2024-09-11T07:40:40.689386+0000","last_active":"2024-09-11T07:41:18.687588+0000","last_peered":"2024-09-11T07:41:18.687588+0000","last_clean":"2024-09-11T07:41:18.687588+0000","last_became_active":"2024-09-11T07:26:33.537850+0000","last_became_peered":"2024-09-11T07:26:33.537850+0000","last_unstale":"2024-09-11T07:41:18.687588+0000","last_undegraded":"2024-09-11T07:41:18.687588+0000","last_fullsized":"2024-09-11T07:41:18.687588+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'28","last_scrub_stamp":"2024-09-11T07:40:40.689301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:40:40.689301+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:48.883615+0000","scrub_duration":0.0071558330000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":104,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.688350+0000","last_change":"2024-09-11T07:39:57.768072+0000","last_active":"2024-09-11T07:41:15.688350+0000","last_peered":"2024-09-11T07:41:15.688350+0000","last_clean":"2024-09-11T07:41:15.688350+0000","last_became_active":"2024-09-11T07:39:57.767741+0000","last_became_peered":"2024-09-11T07:39:57.767741+0000","last_unstale":"2024-09-11T07:41:15.688350+0000","last_undegraded":"2024-09-11T07:41:15.688350+0000","last_fullsized":"2024-09-11T07:41:15.688350+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:49.609120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:39:49.609120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032789239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3],"acting":[2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"84'9","reported_seq":511,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:19.306108+0000","last_change":"2024-09-11T07:37:37.000676+0000","last_active":"2024-09-11T07:41:19.306108+0000","last_peered":"2024-09-11T07:41:19.306108+0000","last_clean":"2024-09-11T07:41:19.306108+0000","last_became_active":"2024-09-11T07:26:35.562997+0000","last_became_peered":"2024-09-11T07:26:35.562997+0000","last_unstale":"2024-09-11T07:41:19.306108+0000","last_undegraded":"2024-09-11T07:41:19.306108+0000","last_fullsized":"2024-09-11T07:41:19.306108+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-11T07:37:37.000619+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-11T07:35:39.832547+0000","last_clean_scrub_stamp":"2024-09-11T07:37:37.000619+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021426219999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":306,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.683730+0000","last_change":"2024-09-11T07:39:35.486304+0000","last_active":"2024-09-11T07:41:15.683730+0000","last_peered":"2024-09-11T07:41:15.683730+0000","last_clean":"2024-09-11T07:41:15.683730+0000","last_became_active":"2024-09-11T07:32:00.511005+0000","last_became_peered":"2024-09-11T07:32:00.511005+0000","last_unstale":"2024-09-11T07:41:15.683730+0000","last_undegraded":"2024-09-11T07:41:15.683730+0000","last_fullsized":"2024-09-11T07:41:15.683730+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:35.486250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:39:35.486250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042980279999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"326'15550","reported_seq":26155,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:17.987682+0000","last_change":"2024-09-11T07:40:27.473852+0000","last_active":"2024-09-11T07:41:17.987682+0000","last_peered":"2024-09-11T07:41:17.987682+0000","last_clean":"2024-09-11T07:41:17.987682+0000","last_became_active":"2024-09-11T07:40:27.407183+0000","last_became_peered":"2024-09-11T07:40:27.407183+0000","last_unstale":"2024-09-11T07:41:17.987682+0000","last_undegraded":"2024-09-11T07:41:17.987682+0000","last_fullsized":"2024-09-11T07:41:17.987682+0000","mapping_epoch":302,"log_start":"282'10144","ondisk_log_start":"282'10144","created":247,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":5406,"log_dups_size":0,"ondisk_log_size":5406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15550,"num_write_kb":497537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3497,"num_bytes_recovered":229113880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"234'38","reported_seq":374,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.676101+0000","last_change":"2024-09-11T07:39:57.792218+0000","last_active":"2024-09-11T07:41:15.676101+0000","last_peered":"2024-09-11T07:41:15.676101+0000","last_clean":"2024-09-11T07:41:15.676101+0000","last_became_active":"2024-09-11T07:39:57.775541+0000","last_became_peered":"2024-09-11T07:39:57.775541+0000","last_unstale":"2024-09-11T07:41:15.676101+0000","last_undegraded":"2024-09-11T07:41:15.676101+0000","last_fullsized":"2024-09-11T07:41:15.676101+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'38","last_scrub_stamp":"2024-09-11T07:38:24.103612+0000","last_deep_scrub":"87'26","last_deep_scrub_stamp":"2024-09-11T07:35:52.942347+0000","last_clean_scrub_stamp":"2024-09-11T07:38:24.103612+0000","objects_scrubbed":8,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010845960999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":51,"num_read_kb":39,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":44,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.688296+0000","last_change":"2024-09-11T07:40:11.698799+0000","last_active":"2024-09-11T07:41:18.688296+0000","last_peered":"2024-09-11T07:41:18.688296+0000","last_clean":"2024-09-11T07:41:18.688296+0000","last_became_active":"2024-09-11T07:40:11.698558+0000","last_became_peered":"2024-09-11T07:40:11.698558+0000","last_unstale":"2024-09-11T07:41:18.688296+0000","last_undegraded":"2024-09-11T07:41:18.688296+0000","last_fullsized":"2024-09-11T07:41:18.688296+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:33.081815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:33.081815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":355,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.687509+0000","last_change":"2024-09-11T07:39:57.767026+0000","last_active":"2024-09-11T07:41:18.687509+0000","last_peered":"2024-09-11T07:41:18.687509+0000","last_clean":"2024-09-11T07:41:18.687509+0000","last_became_active":"2024-09-11T07:39:57.766717+0000","last_became_peered":"2024-09-11T07:39:57.766717+0000","last_unstale":"2024-09-11T07:41:18.687509+0000","last_undegraded":"2024-09-11T07:41:18.687509+0000","last_fullsized":"2024-09-11T07:41:18.687509+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:36.585153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:27.968579+0000","last_clean_scrub_stamp":"2024-09-11T07:39:36.585153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0067939640000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":343,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.687692+0000","last_change":"2024-09-11T07:39:57.692130+0000","last_active":"2024-09-11T07:41:18.687692+0000","last_peered":"2024-09-11T07:41:18.687692+0000","last_clean":"2024-09-11T07:41:18.687692+0000","last_became_active":"2024-09-11T07:39:57.691831+0000","last_became_peered":"2024-09-11T07:39:57.691831+0000","last_unstale":"2024-09-11T07:41:18.687692+0000","last_undegraded":"2024-09-11T07:41:18.687692+0000","last_fullsized":"2024-09-11T07:41:18.687692+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:51.576754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:29.978798+0000","last_clean_scrub_stamp":"2024-09-11T07:39:51.576754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036155279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"326'15678","reported_seq":22122,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.687928+0000","last_change":"2024-09-11T07:40:39.907214+0000","last_active":"2024-09-11T07:41:18.687928+0000","last_peered":"2024-09-11T07:41:18.687928+0000","last_clean":"2024-09-11T07:41:18.687928+0000","last_became_active":"2024-09-11T07:40:34.480392+0000","last_became_peered":"2024-09-11T07:40:34.480392+0000","last_unstale":"2024-09-11T07:41:18.687928+0000","last_undegraded":"2024-09-11T07:41:18.687928+0000","last_fullsized":"2024-09-11T07:41:18.687928+0000","mapping_epoch":305,"log_start":"287'11299","ondisk_log_start":"287'11299","created":247,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"310'13316","last_scrub_stamp":"2024-09-11T07:40:39.907152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:40:39.907152+0000","objects_scrubbed":2362,"log_size":4379,"log_dups_size":0,"ondisk_log_size":4379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:05.320495+0000","scrub_duration":0.211466654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15678,"num_write_kb":501696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3114,"num_bytes_recovered":204079104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"236'32","reported_seq":397,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.675763+0000","last_change":"2024-09-11T07:36:50.024003+0000","last_active":"2024-09-11T07:41:15.675763+0000","last_peered":"2024-09-11T07:41:15.675763+0000","last_clean":"2024-09-11T07:41:15.675763+0000","last_became_active":"2024-09-11T07:32:00.506270+0000","last_became_peered":"2024-09-11T07:32:00.506270+0000","last_unstale":"2024-09-11T07:41:15.675763+0000","last_undegraded":"2024-09-11T07:41:15.675763+0000","last_fullsized":"2024-09-11T07:41:15.675763+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'32","last_scrub_stamp":"2024-09-11T07:36:50.023909+0000","last_deep_scrub":"236'32","last_deep_scrub_stamp":"2024-09-11T07:36:50.023909+0000","last_clean_scrub_stamp":"2024-09-11T07:36:50.023909+0000","objects_scrubbed":8,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.030217668999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":306,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.684309+0000","last_change":"2024-09-11T07:39:57.281547+0000","last_active":"2024-09-11T07:41:15.684309+0000","last_peered":"2024-09-11T07:41:15.684309+0000","last_clean":"2024-09-11T07:41:15.684309+0000","last_became_active":"2024-09-11T07:32:00.510631+0000","last_became_peered":"2024-09-11T07:32:00.510631+0000","last_unstale":"2024-09-11T07:41:15.684309+0000","last_undegraded":"2024-09-11T07:41:15.684309+0000","last_fullsized":"2024-09-11T07:41:15.684309+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:57.281493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:49.144041+0000","last_clean_scrub_stamp":"2024-09-11T07:39:57.281493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001207484,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":135,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.675437+0000","last_change":"2024-09-11T07:40:45.799295+0000","last_active":"2024-09-11T07:41:15.675437+0000","last_peered":"2024-09-11T07:41:15.675437+0000","last_clean":"2024-09-11T07:41:15.675437+0000","last_became_active":"2024-09-11T07:36:05.225840+0000","last_became_peered":"2024-09-11T07:36:05.225840+0000","last_unstale":"2024-09-11T07:41:15.675437+0000","last_undegraded":"2024-09-11T07:41:15.675437+0000","last_fullsized":"2024-09-11T07:41:15.675437+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:45.799189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:40:45.799189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:54.908201+0000","scrub_duration":0.047262133999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":334,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.673923+0000","last_change":"2024-09-11T07:41:14.401926+0000","last_active":"2024-09-11T07:41:15.673923+0000","last_peered":"2024-09-11T07:41:15.673923+0000","last_clean":"2024-09-11T07:41:15.673923+0000","last_became_active":"2024-09-11T07:26:37.861279+0000","last_became_peered":"2024-09-11T07:26:37.861279+0000","last_unstale":"2024-09-11T07:41:15.673923+0000","last_undegraded":"2024-09-11T07:41:15.673923+0000","last_fullsized":"2024-09-11T07:41:15.673923+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:14.401843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:39.723598+0000","last_clean_scrub_stamp":"2024-09-11T07:41:14.401843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:27.665632+0000","scrub_duration":0.0063578150000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.f","version":"326'15628","reported_seq":15715,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:17.982052+0000","last_change":"2024-09-11T07:37:25.856504+0000","last_active":"2024-09-11T07:41:17.982052+0000","last_peered":"2024-09-11T07:41:17.982052+0000","last_clean":"2024-09-11T07:41:17.982052+0000","last_became_active":"2024-09-11T07:37:25.856214+0000","last_became_peered":"2024-09-11T07:37:25.856214+0000","last_unstale":"2024-09-11T07:41:17.982052+0000","last_undegraded":"2024-09-11T07:41:17.982052+0000","last_fullsized":"2024-09-11T07:41:17.982052+0000","mapping_epoch":247,"log_start":"283'10241","ondisk_log_start":"283'10241","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":5387,"log_dups_size":0,"ondisk_log_size":5387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15628,"num_write_kb":500096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"234'20","reported_seq":342,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.687644+0000","last_change":"2024-09-11T07:39:58.520372+0000","last_active":"2024-09-11T07:41:18.687644+0000","last_peered":"2024-09-11T07:41:18.687644+0000","last_clean":"2024-09-11T07:41:18.687644+0000","last_became_active":"2024-09-11T07:39:57.692071+0000","last_became_peered":"2024-09-11T07:39:57.692071+0000","last_unstale":"2024-09-11T07:41:18.687644+0000","last_undegraded":"2024-09-11T07:41:18.687644+0000","last_fullsized":"2024-09-11T07:41:18.687644+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'20","last_scrub_stamp":"2024-09-11T07:39:58.520311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:39:58.520311+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024543820000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2],"acting":[4,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":364,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.676197+0000","last_change":"2024-09-11T07:40:44.863008+0000","last_active":"2024-09-11T07:41:15.676197+0000","last_peered":"2024-09-11T07:41:15.676197+0000","last_clean":"2024-09-11T07:41:15.676197+0000","last_became_active":"2024-09-11T07:32:00.522456+0000","last_became_peered":"2024-09-11T07:32:00.522456+0000","last_unstale":"2024-09-11T07:41:15.676197+0000","last_undegraded":"2024-09-11T07:41:15.676197+0000","last_fullsized":"2024-09-11T07:41:15.676197+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:40:44.862646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:40:44.862646+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:02.689591+0000","scrub_duration":0.005065918,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"290'9","reported_seq":534,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.688462+0000","last_change":"2024-09-11T07:39:57.780023+0000","last_active":"2024-09-11T07:41:18.688462+0000","last_peered":"2024-09-11T07:41:18.688462+0000","last_clean":"2024-09-11T07:41:18.688462+0000","last_became_active":"2024-09-11T07:39:57.768155+0000","last_became_peered":"2024-09-11T07:39:57.768155+0000","last_unstale":"2024-09-11T07:41:18.688462+0000","last_undegraded":"2024-09-11T07:41:18.688462+0000","last_fullsized":"2024-09-11T07:41:18.688462+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:37:26.141447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.141447+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015408170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":332,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.675035+0000","last_change":"2024-09-11T07:36:40.027067+0000","last_active":"2024-09-11T07:41:15.675035+0000","last_peered":"2024-09-11T07:41:15.675035+0000","last_clean":"2024-09-11T07:41:15.675035+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:41:15.675035+0000","last_undegraded":"2024-09-11T07:41:15.675035+0000","last_fullsized":"2024-09-11T07:41:15.675035+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:40.026982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_clean_scrub_stamp":"2024-09-11T07:36:40.026982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090395279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.9","version":"326'10707","reported_seq":10806,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:17.983514+0000","last_change":"2024-09-11T07:38:27.369273+0000","last_active":"2024-09-11T07:41:17.983514+0000","last_peered":"2024-09-11T07:41:17.983514+0000","last_clean":"2024-09-11T07:41:17.983514+0000","last_became_active":"2024-09-11T07:38:27.364955+0000","last_became_peered":"2024-09-11T07:38:27.364955+0000","last_unstale":"2024-09-11T07:41:17.983514+0000","last_undegraded":"2024-09-11T07:41:17.983514+0000","last_fullsized":"2024-09-11T07:41:17.983514+0000","mapping_epoch":247,"log_start":"272'6326","ondisk_log_start":"272'6326","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4381,"log_dups_size":0,"ondisk_log_size":4381,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7999,"num_write_kb":256320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"290'42","reported_seq":261,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.690631+0000","last_change":"2024-09-11T07:40:33.449484+0000","last_active":"2024-09-11T07:41:15.690631+0000","last_peered":"2024-09-11T07:41:15.690631+0000","last_clean":"2024-09-11T07:41:15.690631+0000","last_became_active":"2024-09-11T07:39:57.768267+0000","last_became_peered":"2024-09-11T07:39:57.768267+0000","last_unstale":"2024-09-11T07:41:15.690631+0000","last_undegraded":"2024-09-11T07:41:15.690631+0000","last_fullsized":"2024-09-11T07:41:15.690631+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'41","last_scrub_stamp":"2024-09-11T07:36:41.343611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:36:41.343611+0000","objects_scrubbed":10,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0099986009999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":348,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.675791+0000","last_change":"2024-09-11T07:39:57.468860+0000","last_active":"2024-09-11T07:41:15.675791+0000","last_peered":"2024-09-11T07:41:15.675791+0000","last_clean":"2024-09-11T07:41:15.675791+0000","last_became_active":"2024-09-11T07:26:31.424618+0000","last_became_peered":"2024-09-11T07:26:31.424618+0000","last_unstale":"2024-09-11T07:41:15.675791+0000","last_undegraded":"2024-09-11T07:41:15.675791+0000","last_fullsized":"2024-09-11T07:41:15.675791+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:57.468791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:51.308492+0000","last_clean_scrub_stamp":"2024-09-11T07:39:57.468791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:21.538207+0000","scrub_duration":0.0013109230000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":334,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.676779+0000","last_change":"2024-09-11T07:37:33.144930+0000","last_active":"2024-09-11T07:41:15.676779+0000","last_peered":"2024-09-11T07:41:15.676779+0000","last_clean":"2024-09-11T07:41:15.676779+0000","last_became_active":"2024-09-11T07:26:35.560363+0000","last_became_peered":"2024-09-11T07:26:35.560363+0000","last_unstale":"2024-09-11T07:41:15.676779+0000","last_undegraded":"2024-09-11T07:41:15.676779+0000","last_fullsized":"2024-09-11T07:41:15.676779+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:33.144873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:33.144873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001754675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":339,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.679957+0000","last_change":"2024-09-11T07:37:39.053981+0000","last_active":"2024-09-11T07:41:15.679957+0000","last_peered":"2024-09-11T07:41:15.679957+0000","last_clean":"2024-09-11T07:41:15.679957+0000","last_became_active":"2024-09-11T07:32:00.505919+0000","last_became_peered":"2024-09-11T07:32:00.505919+0000","last_unstale":"2024-09-11T07:41:15.679957+0000","last_undegraded":"2024-09-11T07:41:15.679957+0000","last_fullsized":"2024-09-11T07:41:15.679957+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:39.053900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:39.053900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050142590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"326'16008","reported_seq":23907,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:17.982401+0000","last_change":"2024-09-11T07:40:13.749548+0000","last_active":"2024-09-11T07:41:17.982401+0000","last_peered":"2024-09-11T07:41:17.982401+0000","last_clean":"2024-09-11T07:41:17.982401+0000","last_became_active":"2024-09-11T07:40:13.698098+0000","last_became_peered":"2024-09-11T07:40:13.698098+0000","last_unstale":"2024-09-11T07:41:17.982401+0000","last_undegraded":"2024-09-11T07:41:17.982401+0000","last_fullsized":"2024-09-11T07:41:17.982401+0000","mapping_epoch":298,"log_start":"286'11318","ondisk_log_start":"286'11318","created":247,"last_epoch_clean":299,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4690,"log_dups_size":0,"ondisk_log_size":4690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16008,"num_write_kb":512256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3894,"num_bytes_recovered":255197184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"234'22","reported_seq":346,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.676252+0000","last_change":"2024-09-11T07:40:43.269210+0000","last_active":"2024-09-11T07:41:15.676252+0000","last_peered":"2024-09-11T07:41:15.676252+0000","last_clean":"2024-09-11T07:41:15.676252+0000","last_became_active":"2024-09-11T07:40:11.704155+0000","last_became_peered":"2024-09-11T07:40:11.704155+0000","last_unstale":"2024-09-11T07:41:15.676252+0000","last_undegraded":"2024-09-11T07:41:15.676252+0000","last_fullsized":"2024-09-11T07:41:15.676252+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'22","last_scrub_stamp":"2024-09-11T07:37:20.965147+0000","last_deep_scrub":"144'18","last_deep_scrub_stamp":"2024-09-11T07:31:51.603378+0000","last_clean_scrub_stamp":"2024-09-11T07:37:20.965147+0000","objects_scrubbed":5,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006530999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":1225,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:19.265647+0000","last_change":"2024-09-11T07:37:22.990093+0000","last_active":"2024-09-11T07:41:19.265647+0000","last_peered":"2024-09-11T07:41:19.265647+0000","last_clean":"2024-09-11T07:41:19.265647+0000","last_became_active":"2024-09-11T07:26:31.421276+0000","last_became_peered":"2024-09-11T07:26:31.421276+0000","last_unstale":"2024-09-11T07:41:19.265647+0000","last_undegraded":"2024-09-11T07:41:19.265647+0000","last_fullsized":"2024-09-11T07:41:19.265647+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'5","last_scrub_stamp":"2024-09-11T07:37:22.990033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:22.990033+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001646954,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":880,"num_read_kb":875,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":367,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.684632+0000","last_change":"2024-09-11T07:40:38.385663+0000","last_active":"2024-09-11T07:41:15.684632+0000","last_peered":"2024-09-11T07:41:15.684632+0000","last_clean":"2024-09-11T07:41:15.684632+0000","last_became_active":"2024-09-11T07:26:31.413354+0000","last_became_peered":"2024-09-11T07:26:31.413354+0000","last_unstale":"2024-09-11T07:41:15.684632+0000","last_undegraded":"2024-09-11T07:41:15.684632+0000","last_fullsized":"2024-09-11T07:41:15.684632+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:38.385583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:07.149367+0000","last_clean_scrub_stamp":"2024-09-11T07:40:38.385583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:53.440444+0000","scrub_duration":0.014349707999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":354,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.688107+0000","last_change":"2024-09-11T07:39:45.351040+0000","last_active":"2024-09-11T07:41:15.688107+0000","last_peered":"2024-09-11T07:41:15.688107+0000","last_clean":"2024-09-11T07:41:15.688107+0000","last_became_active":"2024-09-11T07:32:00.508398+0000","last_became_peered":"2024-09-11T07:32:00.508398+0000","last_unstale":"2024-09-11T07:41:15.688107+0000","last_undegraded":"2024-09-11T07:41:15.688107+0000","last_fullsized":"2024-09-11T07:41:15.688107+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:24.876930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:38:24.876930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0095042219999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":332,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.684006+0000","last_change":"2024-09-11T07:39:49.399814+0000","last_active":"2024-09-11T07:41:15.684006+0000","last_peered":"2024-09-11T07:41:15.684006+0000","last_clean":"2024-09-11T07:41:15.684006+0000","last_became_active":"2024-09-11T07:32:00.515745+0000","last_became_peered":"2024-09-11T07:32:00.515745+0000","last_unstale":"2024-09-11T07:41:15.684006+0000","last_undegraded":"2024-09-11T07:41:15.684006+0000","last_fullsized":"2024-09-11T07:41:15.684006+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:49.399708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:56.490940+0000","last_clean_scrub_stamp":"2024-09-11T07:39:49.399708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053800089999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"326'16028","reported_seq":16116,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:17.975411+0000","last_change":"2024-09-11T07:37:26.029935+0000","last_active":"2024-09-11T07:41:17.975411+0000","last_peered":"2024-09-11T07:41:17.975411+0000","last_clean":"2024-09-11T07:41:17.975411+0000","last_became_active":"2024-09-11T07:37:26.029608+0000","last_became_peered":"2024-09-11T07:37:26.029608+0000","last_unstale":"2024-09-11T07:41:17.975411+0000","last_undegraded":"2024-09-11T07:41:17.975411+0000","last_fullsized":"2024-09-11T07:41:17.975411+0000","mapping_epoch":247,"log_start":"287'11544","ondisk_log_start":"287'11544","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4484,"log_dups_size":0,"ondisk_log_size":4484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16028,"num_write_kb":512896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"234'30","reported_seq":412,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.676905+0000","last_change":"2024-09-11T07:40:34.716884+0000","last_active":"2024-09-11T07:41:15.676905+0000","last_peered":"2024-09-11T07:41:15.676905+0000","last_clean":"2024-09-11T07:41:15.676905+0000","last_became_active":"2024-09-11T07:28:09.018134+0000","last_became_peered":"2024-09-11T07:28:09.018134+0000","last_unstale":"2024-09-11T07:41:15.676905+0000","last_undegraded":"2024-09-11T07:41:15.676905+0000","last_fullsized":"2024-09-11T07:41:15.676905+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'30","last_scrub_stamp":"2024-09-11T07:40:34.716804+0000","last_deep_scrub":"94'18","last_deep_scrub_stamp":"2024-09-11T07:28:51.437397+0000","last_clean_scrub_stamp":"2024-09-11T07:40:34.716804+0000","objects_scrubbed":6,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:00.562358+0000","scrub_duration":0.0055680800000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":358,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.675624+0000","last_change":"2024-09-11T07:39:57.769695+0000","last_active":"2024-09-11T07:41:15.675624+0000","last_peered":"2024-09-11T07:41:15.675624+0000","last_clean":"2024-09-11T07:41:15.675624+0000","last_became_active":"2024-09-11T07:39:57.758029+0000","last_became_peered":"2024-09-11T07:39:57.758029+0000","last_unstale":"2024-09-11T07:41:15.675624+0000","last_undegraded":"2024-09-11T07:41:15.675624+0000","last_fullsized":"2024-09-11T07:41:15.675624+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-11T07:38:21.385813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:38:21.385813+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060130330000000001,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":343,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.684608+0000","last_change":"2024-09-11T07:39:57.772569+0000","last_active":"2024-09-11T07:41:15.684608+0000","last_peered":"2024-09-11T07:41:15.684608+0000","last_clean":"2024-09-11T07:41:15.684608+0000","last_became_active":"2024-09-11T07:39:57.772244+0000","last_became_peered":"2024-09-11T07:39:57.772244+0000","last_unstale":"2024-09-11T07:41:15.684608+0000","last_undegraded":"2024-09-11T07:41:15.684608+0000","last_fullsized":"2024-09-11T07:41:15.684608+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:26.556972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:16.967492+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.556972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001503206,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":51,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.677171+0000","last_change":"2024-09-11T07:39:57.688960+0000","last_active":"2024-09-11T07:41:15.677171+0000","last_peered":"2024-09-11T07:41:15.677171+0000","last_clean":"2024-09-11T07:41:15.677171+0000","last_became_active":"2024-09-11T07:39:57.688619+0000","last_became_peered":"2024-09-11T07:39:57.688619+0000","last_unstale":"2024-09-11T07:41:15.677171+0000","last_undegraded":"2024-09-11T07:41:15.677171+0000","last_fullsized":"2024-09-11T07:41:15.677171+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:23.100768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:51.187127+0000","last_clean_scrub_stamp":"2024-09-11T07:38:23.100768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":51,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.675598+0000","last_change":"2024-09-11T07:39:57.772562+0000","last_active":"2024-09-11T07:41:15.675598+0000","last_peered":"2024-09-11T07:41:15.675598+0000","last_clean":"2024-09-11T07:41:15.675598+0000","last_became_active":"2024-09-11T07:39:57.772223+0000","last_became_peered":"2024-09-11T07:39:57.772223+0000","last_unstale":"2024-09-11T07:41:15.675598+0000","last_undegraded":"2024-09-11T07:41:15.675598+0000","last_fullsized":"2024-09-11T07:41:15.675598+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:25.907711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:38:25.907711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"326'10602","reported_seq":12383,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:17.980120+0000","last_change":"2024-09-11T07:41:03.709361+0000","last_active":"2024-09-11T07:41:17.980120+0000","last_peered":"2024-09-11T07:41:17.980120+0000","last_clean":"2024-09-11T07:41:17.980120+0000","last_became_active":"2024-09-11T07:41:03.691512+0000","last_became_peered":"2024-09-11T07:41:03.691512+0000","last_unstale":"2024-09-11T07:41:17.980120+0000","last_undegraded":"2024-09-11T07:41:17.980120+0000","last_fullsized":"2024-09-11T07:41:17.980120+0000","mapping_epoch":323,"log_start":"272'6224","ondisk_log_start":"272'6224","created":247,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4378,"log_dups_size":0,"ondisk_log_size":4378,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7928,"num_write_kb":252736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":550,"num_bytes_recovered":36044800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"234'26","reported_seq":178,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.687992+0000","last_change":"2024-09-11T07:37:40.649065+0000","last_active":"2024-09-11T07:41:18.687992+0000","last_peered":"2024-09-11T07:41:18.687992+0000","last_clean":"2024-09-11T07:41:18.687992+0000","last_became_active":"2024-09-11T07:36:05.224704+0000","last_became_peered":"2024-09-11T07:36:05.224704+0000","last_unstale":"2024-09-11T07:41:18.687992+0000","last_undegraded":"2024-09-11T07:41:18.687992+0000","last_fullsized":"2024-09-11T07:41:18.687992+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'26","last_scrub_stamp":"2024-09-11T07:37:40.648986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:40.648986+0000","objects_scrubbed":7,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030578620000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'3","reported_seq":360,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:16.572792+0000","last_change":"2024-09-11T07:41:16.572712+0000","last_active":"2024-09-11T07:41:16.572792+0000","last_peered":"2024-09-11T07:41:16.572792+0000","last_clean":"2024-09-11T07:41:16.572792+0000","last_became_active":"2024-09-11T07:40:11.701416+0000","last_became_peered":"2024-09-11T07:40:11.701416+0000","last_unstale":"2024-09-11T07:41:16.572792+0000","last_undegraded":"2024-09-11T07:41:16.572792+0000","last_fullsized":"2024-09-11T07:41:16.572792+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-09-11T07:41:16.572623+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_clean_scrub_stamp":"2024-09-11T07:41:16.572623+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:31.955260+0000","scrub_duration":0.0033378140000000001,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":358,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.675573+0000","last_change":"2024-09-11T07:40:11.706063+0000","last_active":"2024-09-11T07:41:15.675573+0000","last_peered":"2024-09-11T07:41:15.675573+0000","last_clean":"2024-09-11T07:41:15.675573+0000","last_became_active":"2024-09-11T07:40:11.705286+0000","last_became_peered":"2024-09-11T07:40:11.705286+0000","last_unstale":"2024-09-11T07:41:15.675573+0000","last_undegraded":"2024-09-11T07:41:15.675573+0000","last_fullsized":"2024-09-11T07:41:15.675573+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:35.504238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:35.504238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013066169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"87'81","reported_seq":175,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.676422+0000","last_change":"2024-09-11T07:40:38.751275+0000","last_active":"2024-09-11T07:41:15.676422+0000","last_peered":"2024-09-11T07:41:15.676422+0000","last_clean":"2024-09-11T07:41:15.676422+0000","last_became_active":"2024-09-11T07:39:57.761901+0000","last_became_peered":"2024-09-11T07:39:57.761901+0000","last_unstale":"2024-09-11T07:41:15.676422+0000","last_undegraded":"2024-09-11T07:41:15.676422+0000","last_fullsized":"2024-09-11T07:41:15.676422+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:40:38.751193+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:40:38.751193+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:44.042328+0000","scrub_duration":0.010400779000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":154,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.690917+0000","last_change":"2024-09-11T07:39:57.749041+0000","last_active":"2024-09-11T07:41:15.690917+0000","last_peered":"2024-09-11T07:41:15.690917+0000","last_clean":"2024-09-11T07:41:15.690917+0000","last_became_active":"2024-09-11T07:39:57.748658+0000","last_became_peered":"2024-09-11T07:39:57.748658+0000","last_unstale":"2024-09-11T07:41:15.690917+0000","last_undegraded":"2024-09-11T07:41:15.690917+0000","last_fullsized":"2024-09-11T07:41:15.690917+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:41.562322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:41.562322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":325,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.676092+0000","last_change":"2024-09-11T07:40:41.208608+0000","last_active":"2024-09-11T07:41:15.676092+0000","last_peered":"2024-09-11T07:41:15.676092+0000","last_clean":"2024-09-11T07:41:15.676092+0000","last_became_active":"2024-09-11T07:26:37.559920+0000","last_became_peered":"2024-09-11T07:26:37.559920+0000","last_unstale":"2024-09-11T07:41:15.676092+0000","last_undegraded":"2024-09-11T07:41:15.676092+0000","last_fullsized":"2024-09-11T07:41:15.676092+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:23.436012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:23.436012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001213481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.7","version":"326'10517","reported_seq":10637,"reported_epoch":326,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-11T07:41:17.908905+0000","last_change":"2024-09-11T07:40:59.016836+0000","last_active":"2024-09-11T07:41:17.908905+0000","last_peered":"2024-09-11T07:41:17.908905+0000","last_clean":"2024-09-11T07:39:56.662527+0000","last_became_active":"2024-09-11T07:39:58.961246+0000","last_became_peered":"2024-09-11T07:39:58.961246+0000","last_unstale":"2024-09-11T07:41:17.908905+0000","last_undegraded":"2024-09-11T07:41:17.908905+0000","last_fullsized":"2024-09-11T07:41:17.908905+0000","mapping_epoch":289,"log_start":"256'5007","ondisk_log_start":"256'5007","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":5510,"log_dups_size":0,"ondisk_log_size":5510,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":66,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7813,"num_write_kb":251072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,3,7],"avail_no_missing":["5","3","7"],"object_location_counts":[{"shards":"3,5,7","objects":33}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"234'35","reported_seq":221,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.676251+0000","last_change":"2024-09-11T07:39:57.784855+0000","last_active":"2024-09-11T07:41:15.676251+0000","last_peered":"2024-09-11T07:41:15.676251+0000","last_clean":"2024-09-11T07:41:15.676251+0000","last_became_active":"2024-09-11T07:39:57.759856+0000","last_became_peered":"2024-09-11T07:39:57.759856+0000","last_unstale":"2024-09-11T07:41:15.676251+0000","last_undegraded":"2024-09-11T07:41:15.676251+0000","last_fullsized":"2024-09-11T07:41:15.676251+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'35","last_scrub_stamp":"2024-09-11T07:38:23.944683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:38:23.944683+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003078154,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":360,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.690991+0000","last_change":"2024-09-11T07:40:35.376311+0000","last_active":"2024-09-11T07:41:15.690991+0000","last_peered":"2024-09-11T07:41:15.690991+0000","last_clean":"2024-09-11T07:41:15.690991+0000","last_became_active":"2024-09-11T07:26:31.419560+0000","last_became_peered":"2024-09-11T07:26:31.419560+0000","last_unstale":"2024-09-11T07:41:15.690991+0000","last_undegraded":"2024-09-11T07:41:15.690991+0000","last_fullsized":"2024-09-11T07:41:15.690991+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:35.376219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:40:35.376219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:44.257357+0000","scrub_duration":0.003361682,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":357,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.690106+0000","last_change":"2024-09-11T07:40:40.411425+0000","last_active":"2024-09-11T07:41:15.690106+0000","last_peered":"2024-09-11T07:41:15.690106+0000","last_clean":"2024-09-11T07:41:15.690106+0000","last_became_active":"2024-09-11T07:39:57.748953+0000","last_became_peered":"2024-09-11T07:39:57.748953+0000","last_unstale":"2024-09-11T07:41:15.690106+0000","last_undegraded":"2024-09-11T07:41:15.690106+0000","last_fullsized":"2024-09-11T07:41:15.690106+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:40.411364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:40:40.411364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:40.631026+0000","scrub_duration":0.0012767259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":162,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.696079+0000","last_change":"2024-09-11T07:39:57.767088+0000","last_active":"2024-09-11T07:41:15.696079+0000","last_peered":"2024-09-11T07:41:15.696079+0000","last_clean":"2024-09-11T07:41:15.696079+0000","last_became_active":"2024-09-11T07:39:57.766605+0000","last_became_peered":"2024-09-11T07:39:57.766605+0000","last_unstale":"2024-09-11T07:41:15.696079+0000","last_undegraded":"2024-09-11T07:41:15.696079+0000","last_fullsized":"2024-09-11T07:41:15.696079+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:41.428866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:39:41.428866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"326'10891","reported_seq":10991,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:17.883745+0000","last_change":"2024-09-11T07:40:45.283673+0000","last_active":"2024-09-11T07:41:17.883745+0000","last_peered":"2024-09-11T07:41:17.883745+0000","last_clean":"2024-09-11T07:41:17.883745+0000","last_became_active":"2024-09-11T07:38:27.351200+0000","last_became_peered":"2024-09-11T07:38:27.351200+0000","last_unstale":"2024-09-11T07:41:17.883745+0000","last_undegraded":"2024-09-11T07:41:17.883745+0000","last_fullsized":"2024-09-11T07:41:17.883745+0000","mapping_epoch":247,"log_start":"264'5600","ondisk_log_start":"264'5600","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":5291,"log_dups_size":0,"ondisk_log_size":5291,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8127,"num_write_kb":259776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"234'15","reported_seq":361,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.675665+0000","last_change":"2024-09-11T07:40:46.759444+0000","last_active":"2024-09-11T07:41:15.675665+0000","last_peered":"2024-09-11T07:41:15.675665+0000","last_clean":"2024-09-11T07:41:15.675665+0000","last_became_active":"2024-09-11T07:26:33.543148+0000","last_became_peered":"2024-09-11T07:26:33.543148+0000","last_unstale":"2024-09-11T07:41:15.675665+0000","last_undegraded":"2024-09-11T07:41:15.675665+0000","last_fullsized":"2024-09-11T07:41:15.675665+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:40:46.759189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:40:46.759189+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:55.039295+0000","scrub_duration":0.007573715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":337,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.674322+0000","last_change":"2024-09-11T07:37:26.019680+0000","last_active":"2024-09-11T07:41:15.674322+0000","last_peered":"2024-09-11T07:41:15.674322+0000","last_clean":"2024-09-11T07:41:15.674322+0000","last_became_active":"2024-09-11T07:26:31.423056+0000","last_became_peered":"2024-09-11T07:26:31.423056+0000","last_unstale":"2024-09-11T07:41:15.674322+0000","last_undegraded":"2024-09-11T07:41:15.674322+0000","last_fullsized":"2024-09-11T07:41:15.674322+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:26.019623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.019623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013393599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":353,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.686472+0000","last_change":"2024-09-11T07:39:52.485307+0000","last_active":"2024-09-11T07:41:15.686472+0000","last_peered":"2024-09-11T07:41:15.686472+0000","last_clean":"2024-09-11T07:41:15.686472+0000","last_became_active":"2024-09-11T07:26:35.563182+0000","last_became_peered":"2024-09-11T07:26:35.563182+0000","last_unstale":"2024-09-11T07:41:15.686472+0000","last_undegraded":"2024-09-11T07:41:15.686472+0000","last_fullsized":"2024-09-11T07:41:15.686472+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:52.485217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:39:52.485217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013231569999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":339,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.675540+0000","last_change":"2024-09-11T07:36:36.023079+0000","last_active":"2024-09-11T07:41:15.675540+0000","last_peered":"2024-09-11T07:41:15.675540+0000","last_clean":"2024-09-11T07:41:15.675540+0000","last_became_active":"2024-09-11T07:32:00.505185+0000","last_became_peered":"2024-09-11T07:32:00.505185+0000","last_unstale":"2024-09-11T07:41:15.675540+0000","last_undegraded":"2024-09-11T07:41:15.675540+0000","last_fullsized":"2024-09-11T07:41:15.675540+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:36.023006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:46.944763+0000","last_clean_scrub_stamp":"2024-09-11T07:36:36.023006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011934019000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.5","version":"326'10759","reported_seq":12657,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:17.967244+0000","last_change":"2024-09-11T07:41:03.708277+0000","last_active":"2024-09-11T07:41:17.967244+0000","last_peered":"2024-09-11T07:41:17.967244+0000","last_clean":"2024-09-11T07:41:17.967244+0000","last_became_active":"2024-09-11T07:41:03.688690+0000","last_became_peered":"2024-09-11T07:41:03.688690+0000","last_unstale":"2024-09-11T07:41:17.967244+0000","last_undegraded":"2024-09-11T07:41:17.967244+0000","last_fullsized":"2024-09-11T07:41:17.967244+0000","mapping_epoch":323,"log_start":"272'6292","ondisk_log_start":"272'6292","created":247,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4467,"log_dups_size":0,"ondisk_log_size":4467,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8045,"num_write_kb":257536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":589,"num_bytes_recovered":38600704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"234'27","reported_seq":82,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.687291+0000","last_change":"2024-09-11T07:40:34.735380+0000","last_active":"2024-09-11T07:41:18.687291+0000","last_peered":"2024-09-11T07:41:18.687291+0000","last_clean":"2024-09-11T07:41:18.687291+0000","last_became_active":"2024-09-11T07:39:57.766550+0000","last_became_peered":"2024-09-11T07:39:57.766550+0000","last_unstale":"2024-09-11T07:41:18.687291+0000","last_undegraded":"2024-09-11T07:41:18.687291+0000","last_fullsized":"2024-09-11T07:41:18.687291+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'27","last_scrub_stamp":"2024-09-11T07:40:34.735292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:40:34.735292+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:40.999720+0000","scrub_duration":0.0055354389999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":395,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.688420+0000","last_change":"2024-09-11T07:40:11.702110+0000","last_active":"2024-09-11T07:41:15.688420+0000","last_peered":"2024-09-11T07:41:15.688420+0000","last_clean":"2024-09-11T07:41:15.688420+0000","last_became_active":"2024-09-11T07:40:11.701330+0000","last_became_peered":"2024-09-11T07:40:11.701330+0000","last_unstale":"2024-09-11T07:41:15.688420+0000","last_undegraded":"2024-09-11T07:41:15.688420+0000","last_fullsized":"2024-09-11T07:41:15.688420+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:46.402060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_clean_scrub_stamp":"2024-09-11T07:39:46.402060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0055267119999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"151'10","reported_seq":470,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.684699+0000","last_change":"2024-09-11T07:37:38.544127+0000","last_active":"2024-09-11T07:41:15.684699+0000","last_peered":"2024-09-11T07:41:15.684699+0000","last_clean":"2024-09-11T07:41:15.684699+0000","last_became_active":"2024-09-11T07:32:00.506806+0000","last_became_peered":"2024-09-11T07:32:00.506806+0000","last_unstale":"2024-09-11T07:41:15.684699+0000","last_undegraded":"2024-09-11T07:41:15.684699+0000","last_fullsized":"2024-09-11T07:41:15.684699+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:37:38.544072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:37:38.544072+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027401579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":335,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.267582+0000","last_change":"2024-09-11T07:41:18.267534+0000","last_active":"2024-09-11T07:41:18.267582+0000","last_peered":"2024-09-11T07:41:18.267582+0000","last_clean":"2024-09-11T07:41:18.267582+0000","last_became_active":"2024-09-11T07:26:37.861134+0000","last_became_peered":"2024-09-11T07:26:37.861134+0000","last_unstale":"2024-09-11T07:41:18.267582+0000","last_undegraded":"2024-09-11T07:41:18.267582+0000","last_fullsized":"2024-09-11T07:41:18.267582+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:18.267482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:41:18.267482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:47.403256+0000","scrub_duration":0.0010584349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.4","version":"326'10684","reported_seq":10801,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:17.955253+0000","last_change":"2024-09-11T07:41:17.517381+0000","last_active":"2024-09-11T07:41:17.955253+0000","last_peered":"2024-09-11T07:41:17.955253+0000","last_clean":"2024-09-11T07:41:17.955253+0000","last_became_active":"2024-09-11T07:38:27.372060+0000","last_became_peered":"2024-09-11T07:38:27.372060+0000","last_unstale":"2024-09-11T07:41:17.955253+0000","last_undegraded":"2024-09-11T07:41:17.955253+0000","last_fullsized":"2024-09-11T07:41:17.955253+0000","mapping_epoch":247,"log_start":"268'5954","ondisk_log_start":"268'5954","created":247,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'10672","last_scrub_stamp":"2024-09-11T07:41:17.517138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:41:17.517138+0000","objects_scrubbed":12,"log_size":4730,"log_dups_size":0,"ondisk_log_size":4730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:44.094615+0000","scrub_duration":0.074539657999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"234'23","reported_seq":396,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.687565+0000","last_change":"2024-09-11T07:39:57.768799+0000","last_active":"2024-09-11T07:41:18.687565+0000","last_peered":"2024-09-11T07:41:18.687565+0000","last_clean":"2024-09-11T07:41:18.687565+0000","last_became_active":"2024-09-11T07:39:57.691713+0000","last_became_peered":"2024-09-11T07:39:57.691713+0000","last_unstale":"2024-09-11T07:41:18.687565+0000","last_undegraded":"2024-09-11T07:41:18.687565+0000","last_fullsized":"2024-09-11T07:41:18.687565+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'23","last_scrub_stamp":"2024-09-11T07:39:55.578519+0000","last_deep_scrub":"234'23","last_deep_scrub_stamp":"2024-09-11T07:37:34.613280+0000","last_clean_scrub_stamp":"2024-09-11T07:39:55.578519+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030590740000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":372,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.688441+0000","last_change":"2024-09-11T07:41:16.742565+0000","last_active":"2024-09-11T07:41:18.688441+0000","last_peered":"2024-09-11T07:41:18.688441+0000","last_clean":"2024-09-11T07:41:18.688441+0000","last_became_active":"2024-09-11T07:32:00.504344+0000","last_became_peered":"2024-09-11T07:32:00.504344+0000","last_unstale":"2024-09-11T07:41:18.688441+0000","last_undegraded":"2024-09-11T07:41:18.688441+0000","last_fullsized":"2024-09-11T07:41:18.688441+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:41:16.742481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:41:16.742481+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:34.640553+0000","scrub_duration":0.0043896899999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":353,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.693299+0000","last_change":"2024-09-11T07:39:55.453328+0000","last_active":"2024-09-11T07:41:15.693299+0000","last_peered":"2024-09-11T07:41:15.693299+0000","last_clean":"2024-09-11T07:41:15.693299+0000","last_became_active":"2024-09-11T07:26:35.561967+0000","last_became_peered":"2024-09-11T07:26:35.561967+0000","last_unstale":"2024-09-11T07:41:15.693299+0000","last_undegraded":"2024-09-11T07:41:15.693299+0000","last_fullsized":"2024-09-11T07:41:15.693299+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:55.453221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:44.820354+0000","last_clean_scrub_stamp":"2024-09-11T07:39:55.453221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014898649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":349,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.681894+0000","last_change":"2024-09-11T07:39:57.776965+0000","last_active":"2024-09-11T07:41:15.681894+0000","last_peered":"2024-09-11T07:41:15.681894+0000","last_clean":"2024-09-11T07:41:15.681894+0000","last_became_active":"2024-09-11T07:39:57.776576+0000","last_became_peered":"2024-09-11T07:39:57.776576+0000","last_unstale":"2024-09-11T07:41:15.681894+0000","last_undegraded":"2024-09-11T07:41:15.681894+0000","last_fullsized":"2024-09-11T07:41:15.681894+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:16.860930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:16.860930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024991000000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"234'15","reported_seq":388,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.676826+0000","last_change":"2024-09-11T07:39:57.786497+0000","last_active":"2024-09-11T07:41:15.676826+0000","last_peered":"2024-09-11T07:41:15.676826+0000","last_clean":"2024-09-11T07:41:15.676826+0000","last_became_active":"2024-09-11T07:39:57.767661+0000","last_became_peered":"2024-09-11T07:39:57.767661+0000","last_unstale":"2024-09-11T07:41:15.676826+0000","last_undegraded":"2024-09-11T07:41:15.676826+0000","last_fullsized":"2024-09-11T07:41:15.676826+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:37:19.096817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:19.096817+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00175015,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":357,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.693435+0000","last_change":"2024-09-11T07:39:57.443485+0000","last_active":"2024-09-11T07:41:15.693435+0000","last_peered":"2024-09-11T07:41:15.693435+0000","last_clean":"2024-09-11T07:41:15.693435+0000","last_became_active":"2024-09-11T07:26:31.415701+0000","last_became_peered":"2024-09-11T07:26:31.415701+0000","last_unstale":"2024-09-11T07:41:15.693435+0000","last_undegraded":"2024-09-11T07:41:15.693435+0000","last_fullsized":"2024-09-11T07:41:15.693435+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:57.443424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:48.925200+0000","last_clean_scrub_stamp":"2024-09-11T07:39:57.443424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013832989999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":177,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.677094+0000","last_change":"2024-09-11T07:40:39.893675+0000","last_active":"2024-09-11T07:41:15.677094+0000","last_peered":"2024-09-11T07:41:15.677094+0000","last_clean":"2024-09-11T07:41:15.677094+0000","last_became_active":"2024-09-11T07:40:11.711154+0000","last_became_peered":"2024-09-11T07:40:11.711154+0000","last_unstale":"2024-09-11T07:41:15.677094+0000","last_undegraded":"2024-09-11T07:41:15.677094+0000","last_fullsized":"2024-09-11T07:41:15.677094+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:39.893619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:35.392360+0000","last_clean_scrub_stamp":"2024-09-11T07:40:39.893619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:46.233742+0000","scrub_duration":0.001330475,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":350,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.690422+0000","last_change":"2024-09-11T07:38:29.941071+0000","last_active":"2024-09-11T07:41:15.690422+0000","last_peered":"2024-09-11T07:41:15.690422+0000","last_clean":"2024-09-11T07:41:15.690422+0000","last_became_active":"2024-09-11T07:32:00.506697+0000","last_became_peered":"2024-09-11T07:32:00.506697+0000","last_unstale":"2024-09-11T07:41:15.690422+0000","last_undegraded":"2024-09-11T07:41:15.690422+0000","last_fullsized":"2024-09-11T07:41:15.690422+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:38:29.941008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:38:29.941008+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016886850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":129,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.687651+0000","last_change":"2024-09-11T07:39:46.613784+0000","last_active":"2024-09-11T07:41:18.687651+0000","last_peered":"2024-09-11T07:41:18.687651+0000","last_clean":"2024-09-11T07:41:18.687651+0000","last_became_active":"2024-09-11T07:36:05.225791+0000","last_became_peered":"2024-09-11T07:36:05.225791+0000","last_unstale":"2024-09-11T07:41:18.687651+0000","last_undegraded":"2024-09-11T07:41:18.687651+0000","last_fullsized":"2024-09-11T07:41:18.687651+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:51.617730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:36:51.617730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001776197,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"234'15","reported_seq":100,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.677425+0000","last_change":"2024-09-11T07:39:57.773581+0000","last_active":"2024-09-11T07:41:15.677425+0000","last_peered":"2024-09-11T07:41:15.677425+0000","last_clean":"2024-09-11T07:41:15.677425+0000","last_became_active":"2024-09-11T07:39:57.691830+0000","last_became_peered":"2024-09-11T07:39:57.691830+0000","last_unstale":"2024-09-11T07:41:15.677425+0000","last_undegraded":"2024-09-11T07:41:15.677425+0000","last_fullsized":"2024-09-11T07:41:15.677425+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:37:26.450122+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.450122+0000","objects_scrubbed":3,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":344,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.675906+0000","last_change":"2024-09-11T07:40:39.267299+0000","last_active":"2024-09-11T07:41:15.675906+0000","last_peered":"2024-09-11T07:41:15.675906+0000","last_clean":"2024-09-11T07:41:15.675906+0000","last_became_active":"2024-09-11T07:26:35.566894+0000","last_became_peered":"2024-09-11T07:26:35.566894+0000","last_unstale":"2024-09-11T07:41:15.675906+0000","last_undegraded":"2024-09-11T07:41:15.675906+0000","last_fullsized":"2024-09-11T07:41:15.675906+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:39.267248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:40:39.267248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:55.182199+0000","scrub_duration":0.0012024570000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":330,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.684456+0000","last_change":"2024-09-11T07:40:37.436683+0000","last_active":"2024-09-11T07:41:15.684456+0000","last_peered":"2024-09-11T07:41:15.684456+0000","last_clean":"2024-09-11T07:41:15.684456+0000","last_became_active":"2024-09-11T07:32:00.514100+0000","last_became_peered":"2024-09-11T07:32:00.514100+0000","last_unstale":"2024-09-11T07:41:15.684456+0000","last_undegraded":"2024-09-11T07:41:15.684456+0000","last_fullsized":"2024-09-11T07:41:15.684456+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:37.436628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:40:37.436628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:37.557030+0000","scrub_duration":0.001208518,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.19","version":"326'10787","reported_seq":23410,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:17.983623+0000","last_change":"2024-09-11T07:40:09.708552+0000","last_active":"2024-09-11T07:41:17.983623+0000","last_peered":"2024-09-11T07:41:17.983623+0000","last_clean":"2024-09-11T07:41:17.983623+0000","last_became_active":"2024-09-11T07:40:09.688156+0000","last_became_peered":"2024-09-11T07:40:09.688156+0000","last_unstale":"2024-09-11T07:41:17.983623+0000","last_undegraded":"2024-09-11T07:41:17.983623+0000","last_fullsized":"2024-09-11T07:41:17.983623+0000","mapping_epoch":294,"log_start":"260'5440","ondisk_log_start":"260'5440","created":259,"last_epoch_clean":295,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":5347,"log_dups_size":0,"ondisk_log_size":5347,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8079,"num_write_kb":258176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4857,"num_bytes_recovered":318308352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"234'25","reported_seq":400,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.683185+0000","last_change":"2024-09-11T07:39:57.777199+0000","last_active":"2024-09-11T07:41:15.683185+0000","last_peered":"2024-09-11T07:41:15.683185+0000","last_clean":"2024-09-11T07:41:15.683185+0000","last_became_active":"2024-09-11T07:39:57.771009+0000","last_became_peered":"2024-09-11T07:39:57.771009+0000","last_unstale":"2024-09-11T07:41:15.683185+0000","last_undegraded":"2024-09-11T07:41:15.683185+0000","last_fullsized":"2024-09-11T07:41:15.683185+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'25","last_scrub_stamp":"2024-09-11T07:36:55.966595+0000","last_deep_scrub":"86'15","last_deep_scrub_stamp":"2024-09-11T07:28:22.858137+0000","last_clean_scrub_stamp":"2024-09-11T07:36:55.966595+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.032608074000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":358,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.675727+0000","last_change":"2024-09-11T07:39:57.764438+0000","last_active":"2024-09-11T07:41:15.675727+0000","last_peered":"2024-09-11T07:41:15.675727+0000","last_clean":"2024-09-11T07:41:15.675727+0000","last_became_active":"2024-09-11T07:39:57.764063+0000","last_became_peered":"2024-09-11T07:39:57.764063+0000","last_unstale":"2024-09-11T07:41:15.675727+0000","last_undegraded":"2024-09-11T07:41:15.675727+0000","last_fullsized":"2024-09-11T07:41:15.675727+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:54.493614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:39:54.493614+0000","last_clean_scrub_stamp":"2024-09-11T07:39:54.493614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:22.925815+0000","scrub_duration":0.0049403759999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":519,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.675798+0000","last_change":"2024-09-11T07:40:46.321277+0000","last_active":"2024-09-11T07:41:15.675798+0000","last_peered":"2024-09-11T07:41:15.675798+0000","last_clean":"2024-09-11T07:41:15.675798+0000","last_became_active":"2024-09-11T07:26:35.562245+0000","last_became_peered":"2024-09-11T07:26:35.562245+0000","last_unstale":"2024-09-11T07:41:15.675798+0000","last_undegraded":"2024-09-11T07:41:15.675798+0000","last_fullsized":"2024-09-11T07:41:15.675798+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:40:46.321189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:40:46.321189+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:03.756105+0000","scrub_duration":0.024243521000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":359,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.678349+0000","last_change":"2024-09-11T07:39:57.759567+0000","last_active":"2024-09-11T07:41:15.678349+0000","last_peered":"2024-09-11T07:41:15.678349+0000","last_clean":"2024-09-11T07:41:15.678349+0000","last_became_active":"2024-09-11T07:39:57.758650+0000","last_became_peered":"2024-09-11T07:39:57.758650+0000","last_unstale":"2024-09-11T07:41:15.678349+0000","last_undegraded":"2024-09-11T07:41:15.678349+0000","last_fullsized":"2024-09-11T07:41:15.678349+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:42.084305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:42.084305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047035039999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"326'10540","reported_seq":25120,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:17.967928+0000","last_change":"2024-09-11T07:40:49.712435+0000","last_active":"2024-09-11T07:41:17.967928+0000","last_peered":"2024-09-11T07:41:17.967928+0000","last_clean":"2024-09-11T07:41:17.967928+0000","last_became_active":"2024-09-11T07:40:49.697184+0000","last_became_peered":"2024-09-11T07:40:49.697184+0000","last_unstale":"2024-09-11T07:41:17.967928+0000","last_undegraded":"2024-09-11T07:41:17.967928+0000","last_fullsized":"2024-09-11T07:41:17.967928+0000","mapping_epoch":315,"log_start":"268'5909","ondisk_log_start":"268'5909","created":259,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4631,"log_dups_size":0,"ondisk_log_size":4631,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7866,"num_write_kb":252672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5363,"num_bytes_recovered":351469568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"234'19","reported_seq":332,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.684239+0000","last_change":"2024-09-11T07:39:57.780187+0000","last_active":"2024-09-11T07:41:15.684239+0000","last_peered":"2024-09-11T07:41:15.684239+0000","last_clean":"2024-09-11T07:41:15.684239+0000","last_became_active":"2024-09-11T07:39:57.765149+0000","last_became_peered":"2024-09-11T07:39:57.765149+0000","last_unstale":"2024-09-11T07:41:15.684239+0000","last_undegraded":"2024-09-11T07:41:15.684239+0000","last_fullsized":"2024-09-11T07:41:15.684239+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:37:41.338036+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:37:41.338036+0000","last_clean_scrub_stamp":"2024-09-11T07:37:41.338036+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045362370000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":341,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:17.889619+0000","last_change":"2024-09-11T07:41:17.889562+0000","last_active":"2024-09-11T07:41:17.889619+0000","last_peered":"2024-09-11T07:41:17.889619+0000","last_clean":"2024-09-11T07:41:17.889619+0000","last_became_active":"2024-09-11T07:26:31.419409+0000","last_became_peered":"2024-09-11T07:26:31.419409+0000","last_unstale":"2024-09-11T07:41:17.889619+0000","last_undegraded":"2024-09-11T07:41:17.889619+0000","last_fullsized":"2024-09-11T07:41:17.889619+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:17.889482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:41:17.889482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:27.897684+0000","scrub_duration":0.007281348,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"290'11","reported_seq":261,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.684055+0000","last_change":"2024-09-11T07:39:57.778860+0000","last_active":"2024-09-11T07:41:15.684055+0000","last_peered":"2024-09-11T07:41:15.684055+0000","last_clean":"2024-09-11T07:41:15.684055+0000","last_became_active":"2024-09-11T07:39:57.772790+0000","last_became_peered":"2024-09-11T07:39:57.772790+0000","last_unstale":"2024-09-11T07:41:15.684055+0000","last_undegraded":"2024-09-11T07:41:15.684055+0000","last_fullsized":"2024-09-11T07:41:15.684055+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:38:25.065071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:38:25.065071+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020633370000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":326,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.688064+0000","last_change":"2024-09-11T07:39:50.572103+0000","last_active":"2024-09-11T07:41:18.688064+0000","last_peered":"2024-09-11T07:41:18.688064+0000","last_clean":"2024-09-11T07:41:18.688064+0000","last_became_active":"2024-09-11T07:26:37.565723+0000","last_became_peered":"2024-09-11T07:26:37.565723+0000","last_unstale":"2024-09-11T07:41:18.688064+0000","last_undegraded":"2024-09-11T07:41:18.688064+0000","last_fullsized":"2024-09-11T07:41:18.688064+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:26.659337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:26.659337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012563839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"236'16","reported_seq":340,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:16.349074+0000","last_change":"2024-09-11T07:41:16.349002+0000","last_active":"2024-09-11T07:41:16.349074+0000","last_peered":"2024-09-11T07:41:16.349074+0000","last_clean":"2024-09-11T07:41:16.349074+0000","last_became_active":"2024-09-11T07:39:57.689424+0000","last_became_peered":"2024-09-11T07:39:57.689424+0000","last_unstale":"2024-09-11T07:41:16.349074+0000","last_undegraded":"2024-09-11T07:41:16.349074+0000","last_fullsized":"2024-09-11T07:41:16.349074+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'16","last_scrub_stamp":"2024-09-11T07:41:16.348704+0000","last_deep_scrub":"236'16","last_deep_scrub_stamp":"2024-09-11T07:36:49.809484+0000","last_clean_scrub_stamp":"2024-09-11T07:41:16.348704+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:40.650244+0000","scrub_duration":0.0054645800000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":134,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.683259+0000","last_change":"2024-09-11T07:39:57.779694+0000","last_active":"2024-09-11T07:41:15.683259+0000","last_peered":"2024-09-11T07:41:15.683259+0000","last_clean":"2024-09-11T07:41:15.683259+0000","last_became_active":"2024-09-11T07:39:57.776841+0000","last_became_peered":"2024-09-11T07:39:57.776841+0000","last_unstale":"2024-09-11T07:41:15.683259+0000","last_undegraded":"2024-09-11T07:41:15.683259+0000","last_fullsized":"2024-09-11T07:41:15.683259+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:22.914781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:09.772000+0000","last_clean_scrub_stamp":"2024-09-11T07:37:22.914781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001446864,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":330,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.680680+0000","last_change":"2024-09-11T07:41:15.553229+0000","last_active":"2024-09-11T07:41:15.680680+0000","last_peered":"2024-09-11T07:41:15.680680+0000","last_clean":"2024-09-11T07:41:15.680680+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:41:15.680680+0000","last_undegraded":"2024-09-11T07:41:15.680680+0000","last_fullsized":"2024-09-11T07:41:15.680680+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:15.552917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:41:15.552917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:29.109533+0000","scrub_duration":0.001869838,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":342,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.674923+0000","last_change":"2024-09-11T07:39:57.765790+0000","last_active":"2024-09-11T07:41:15.674923+0000","last_peered":"2024-09-11T07:41:15.674923+0000","last_clean":"2024-09-11T07:41:15.674923+0000","last_became_active":"2024-09-11T07:39:57.761754+0000","last_became_peered":"2024-09-11T07:39:57.761754+0000","last_unstale":"2024-09-11T07:41:15.674923+0000","last_undegraded":"2024-09-11T07:41:15.674923+0000","last_fullsized":"2024-09-11T07:41:15.674923+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:32.186561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:32.186561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014950059999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"234'24","reported_seq":366,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.687682+0000","last_change":"2024-09-11T07:39:42.612220+0000","last_active":"2024-09-11T07:41:18.687682+0000","last_peered":"2024-09-11T07:41:18.687682+0000","last_clean":"2024-09-11T07:41:18.687682+0000","last_became_active":"2024-09-11T07:26:33.540208+0000","last_became_peered":"2024-09-11T07:26:33.540208+0000","last_unstale":"2024-09-11T07:41:18.687682+0000","last_undegraded":"2024-09-11T07:41:18.687682+0000","last_fullsized":"2024-09-11T07:41:18.687682+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'16","last_scrub_stamp":"2024-09-11T07:36:31.789841+0000","last_deep_scrub":"136'16","last_deep_scrub_stamp":"2024-09-11T07:31:06.039023+0000","last_clean_scrub_stamp":"2024-09-11T07:36:31.789841+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048561150000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":340,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.674213+0000","last_change":"2024-09-11T07:39:51.779556+0000","last_active":"2024-09-11T07:41:15.674213+0000","last_peered":"2024-09-11T07:41:15.674213+0000","last_clean":"2024-09-11T07:41:15.674213+0000","last_became_active":"2024-09-11T07:26:31.422671+0000","last_became_peered":"2024-09-11T07:26:31.422671+0000","last_unstale":"2024-09-11T07:41:15.674213+0000","last_undegraded":"2024-09-11T07:41:15.674213+0000","last_fullsized":"2024-09-11T07:41:15.674213+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:51.779468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:39:51.779468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0089535459999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":335,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.676010+0000","last_change":"2024-09-11T07:39:57.761360+0000","last_active":"2024-09-11T07:41:15.676010+0000","last_peered":"2024-09-11T07:41:15.676010+0000","last_clean":"2024-09-11T07:41:15.676010+0000","last_became_active":"2024-09-11T07:39:57.759390+0000","last_became_peered":"2024-09-11T07:39:57.759390+0000","last_unstale":"2024-09-11T07:41:15.676010+0000","last_undegraded":"2024-09-11T07:41:15.676010+0000","last_fullsized":"2024-09-11T07:41:15.676010+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:55.895853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:39:55.895853+0000","last_clean_scrub_stamp":"2024-09-11T07:39:55.895853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011708260999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":61,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.675225+0000","last_change":"2024-09-11T07:40:11.701200+0000","last_active":"2024-09-11T07:41:15.675225+0000","last_peered":"2024-09-11T07:41:15.675225+0000","last_clean":"2024-09-11T07:41:15.675225+0000","last_became_active":"2024-09-11T07:40:11.700676+0000","last_became_peered":"2024-09-11T07:40:11.700676+0000","last_unstale":"2024-09-11T07:41:15.675225+0000","last_undegraded":"2024-09-11T07:41:15.675225+0000","last_fullsized":"2024-09-11T07:41:15.675225+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:43.814370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:43.814370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"234'10","reported_seq":340,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.678859+0000","last_change":"2024-09-11T07:36:46.900027+0000","last_active":"2024-09-11T07:41:15.678859+0000","last_peered":"2024-09-11T07:41:15.678859+0000","last_clean":"2024-09-11T07:41:15.678859+0000","last_became_active":"2024-09-11T07:26:33.545830+0000","last_became_peered":"2024-09-11T07:26:33.545830+0000","last_unstale":"2024-09-11T07:41:15.678859+0000","last_undegraded":"2024-09-11T07:41:15.678859+0000","last_fullsized":"2024-09-11T07:41:15.678859+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'10","last_scrub_stamp":"2024-09-11T07:36:46.899947+0000","last_deep_scrub":"82'6","last_deep_scrub_stamp":"2024-09-11T07:27:47.789465+0000","last_clean_scrub_stamp":"2024-09-11T07:36:46.899947+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003412277,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":338,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.675988+0000","last_change":"2024-09-11T07:39:56.466407+0000","last_active":"2024-09-11T07:41:15.675988+0000","last_peered":"2024-09-11T07:41:15.675988+0000","last_clean":"2024-09-11T07:41:15.675988+0000","last_became_active":"2024-09-11T07:26:31.420518+0000","last_became_peered":"2024-09-11T07:26:31.420518+0000","last_unstale":"2024-09-11T07:41:15.675988+0000","last_undegraded":"2024-09-11T07:41:15.675988+0000","last_fullsized":"2024-09-11T07:41:15.675988+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:56.466067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_clean_scrub_stamp":"2024-09-11T07:39:56.466067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030505950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":270,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.687458+0000","last_change":"2024-09-11T07:40:11.700029+0000","last_active":"2024-09-11T07:41:18.687458+0000","last_peered":"2024-09-11T07:41:18.687458+0000","last_clean":"2024-09-11T07:41:18.687458+0000","last_became_active":"2024-09-11T07:40:11.699702+0000","last_became_peered":"2024-09-11T07:40:11.699702+0000","last_unstale":"2024-09-11T07:41:18.687458+0000","last_undegraded":"2024-09-11T07:41:18.687458+0000","last_fullsized":"2024-09-11T07:41:18.687458+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:34.658586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:47.675050+0000","last_clean_scrub_stamp":"2024-09-11T07:39:34.658586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047285640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":332,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.683471+0000","last_change":"2024-09-11T07:39:43.398492+0000","last_active":"2024-09-11T07:41:15.683471+0000","last_peered":"2024-09-11T07:41:15.683471+0000","last_clean":"2024-09-11T07:41:15.683471+0000","last_became_active":"2024-09-11T07:32:00.511407+0000","last_became_peered":"2024-09-11T07:32:00.511407+0000","last_unstale":"2024-09-11T07:41:15.683471+0000","last_undegraded":"2024-09-11T07:41:15.683471+0000","last_fullsized":"2024-09-11T07:41:15.683471+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:43.398419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:39:43.398419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0064759270000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"234'19","reported_seq":386,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.678552+0000","last_change":"2024-09-11T07:39:57.791332+0000","last_active":"2024-09-11T07:41:15.678552+0000","last_peered":"2024-09-11T07:41:15.678552+0000","last_clean":"2024-09-11T07:41:15.678552+0000","last_became_active":"2024-09-11T07:39:57.767928+0000","last_became_peered":"2024-09-11T07:39:57.767928+0000","last_unstale":"2024-09-11T07:41:15.678552+0000","last_undegraded":"2024-09-11T07:41:15.678552+0000","last_fullsized":"2024-09-11T07:41:15.678552+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:38:28.017124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:38:28.017124+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018425620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":124,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.676288+0000","last_change":"2024-09-11T07:39:29.286259+0000","last_active":"2024-09-11T07:41:15.676288+0000","last_peered":"2024-09-11T07:41:15.676288+0000","last_clean":"2024-09-11T07:41:15.676288+0000","last_became_active":"2024-09-11T07:36:05.228360+0000","last_became_peered":"2024-09-11T07:36:05.228360+0000","last_unstale":"2024-09-11T07:41:15.676288+0000","last_undegraded":"2024-09-11T07:41:15.676288+0000","last_fullsized":"2024-09-11T07:41:15.676288+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:29.286202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:48.366931+0000","last_clean_scrub_stamp":"2024-09-11T07:39:29.286202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001125411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":305,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.683414+0000","last_change":"2024-09-11T07:39:56.318341+0000","last_active":"2024-09-11T07:41:15.683414+0000","last_peered":"2024-09-11T07:41:15.683414+0000","last_clean":"2024-09-11T07:41:15.683414+0000","last_became_active":"2024-09-11T07:32:00.520534+0000","last_became_peered":"2024-09-11T07:32:00.520534+0000","last_unstale":"2024-09-11T07:41:15.683414+0000","last_undegraded":"2024-09-11T07:41:15.683414+0000","last_fullsized":"2024-09-11T07:41:15.683414+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:56.318259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:39.021876+0000","last_clean_scrub_stamp":"2024-09-11T07:39:56.318259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:25.586713+0000","scrub_duration":0.0056755970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":312,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.679596+0000","last_change":"2024-09-11T07:36:49.899993+0000","last_active":"2024-09-11T07:41:15.679596+0000","last_peered":"2024-09-11T07:41:15.679596+0000","last_clean":"2024-09-11T07:41:15.679596+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:41:15.679596+0000","last_undegraded":"2024-09-11T07:41:15.679596+0000","last_fullsized":"2024-09-11T07:41:15.679596+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:36:49.899620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:49.899620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014021288999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"326'10472","reported_seq":22880,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:17.983565+0000","last_change":"2024-09-11T07:40:27.421619+0000","last_active":"2024-09-11T07:41:17.983565+0000","last_peered":"2024-09-11T07:41:17.983565+0000","last_clean":"2024-09-11T07:41:17.983565+0000","last_became_active":"2024-09-11T07:40:27.404959+0000","last_became_peered":"2024-09-11T07:40:27.404959+0000","last_unstale":"2024-09-11T07:41:17.983565+0000","last_undegraded":"2024-09-11T07:41:17.983565+0000","last_fullsized":"2024-09-11T07:41:17.983565+0000","mapping_epoch":302,"log_start":"257'5047","ondisk_log_start":"257'5047","created":259,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":5425,"log_dups_size":0,"ondisk_log_size":5425,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7820,"num_write_kb":249888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4649,"num_bytes_recovered":304676864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":302,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.683857+0000","last_change":"2024-09-11T07:40:44.505650+0000","last_active":"2024-09-11T07:41:15.683857+0000","last_peered":"2024-09-11T07:41:15.683857+0000","last_clean":"2024-09-11T07:41:15.683857+0000","last_became_active":"2024-09-11T07:32:00.511258+0000","last_became_peered":"2024-09-11T07:32:00.511258+0000","last_unstale":"2024-09-11T07:41:15.683857+0000","last_undegraded":"2024-09-11T07:41:15.683857+0000","last_fullsized":"2024-09-11T07:41:15.683857+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:36:44.447809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:36:44.447809+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005540555,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":374,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:17.276211+0000","last_change":"2024-09-11T07:41:17.276152+0000","last_active":"2024-09-11T07:41:17.276211+0000","last_peered":"2024-09-11T07:41:17.276211+0000","last_clean":"2024-09-11T07:41:17.276211+0000","last_became_active":"2024-09-11T07:39:57.696805+0000","last_became_peered":"2024-09-11T07:39:57.696805+0000","last_unstale":"2024-09-11T07:41:17.276211+0000","last_undegraded":"2024-09-11T07:41:17.276211+0000","last_fullsized":"2024-09-11T07:41:17.276211+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:41:17.276067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:41:17.276067+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:34.474209+0000","scrub_duration":0.052791591999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"234'20","reported_seq":170,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.688058+0000","last_change":"2024-09-11T07:40:47.544520+0000","last_active":"2024-09-11T07:41:18.688058+0000","last_peered":"2024-09-11T07:41:18.688058+0000","last_clean":"2024-09-11T07:41:18.688058+0000","last_became_active":"2024-09-11T07:35:00.561615+0000","last_became_peered":"2024-09-11T07:35:00.561615+0000","last_unstale":"2024-09-11T07:41:18.688058+0000","last_undegraded":"2024-09-11T07:41:18.688058+0000","last_fullsized":"2024-09-11T07:41:18.688058+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'20","last_scrub_stamp":"2024-09-11T07:37:29.661671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:37:29.661671+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019609900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1f","version":"290'12","reported_seq":227,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:19.685337+0000","last_change":"2024-09-11T07:39:57.749653+0000","last_active":"2024-09-11T07:41:19.685337+0000","last_peered":"2024-09-11T07:41:19.685337+0000","last_clean":"2024-09-11T07:41:19.685337+0000","last_became_active":"2024-09-11T07:39:57.687809+0000","last_became_peered":"2024-09-11T07:39:57.687809+0000","last_unstale":"2024-09-11T07:41:19.685337+0000","last_undegraded":"2024-09-11T07:41:19.685337+0000","last_fullsized":"2024-09-11T07:41:19.685337+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"225'11","last_scrub_stamp":"2024-09-11T07:38:22.504199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:38:22.504199+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020188300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"326'10475","reported_seq":18879,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:17.972870+0000","last_change":"2024-09-11T07:41:02.781490+0000","last_active":"2024-09-11T07:41:17.972870+0000","last_peered":"2024-09-11T07:41:17.972870+0000","last_clean":"2024-09-11T07:41:17.972870+0000","last_became_active":"2024-09-11T07:41:02.697692+0000","last_became_peered":"2024-09-11T07:41:02.697692+0000","last_unstale":"2024-09-11T07:41:17.972870+0000","last_undegraded":"2024-09-11T07:41:17.972870+0000","last_fullsized":"2024-09-11T07:41:17.972870+0000","mapping_epoch":322,"log_start":"266'5717","ondisk_log_start":"266'5717","created":259,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4758,"log_dups_size":0,"ondisk_log_size":4758,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7813,"num_write_kb":249920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2760,"num_bytes_recovered":180813824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":331,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.673413+0000","last_change":"2024-09-11T07:38:26.939048+0000","last_active":"2024-09-11T07:41:15.673413+0000","last_peered":"2024-09-11T07:41:15.673413+0000","last_clean":"2024-09-11T07:41:15.673413+0000","last_became_active":"2024-09-11T07:26:37.562501+0000","last_became_peered":"2024-09-11T07:26:37.562501+0000","last_unstale":"2024-09-11T07:41:15.673413+0000","last_undegraded":"2024-09-11T07:41:15.673413+0000","last_fullsized":"2024-09-11T07:41:15.673413+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:38:26.938987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:53.875300+0000","last_clean_scrub_stamp":"2024-09-11T07:38:26.938987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016308410000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":341,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.679434+0000","last_change":"2024-09-11T07:40:34.645930+0000","last_active":"2024-09-11T07:41:15.679434+0000","last_peered":"2024-09-11T07:41:15.679434+0000","last_clean":"2024-09-11T07:41:15.679434+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:41:15.679434+0000","last_undegraded":"2024-09-11T07:41:15.679434+0000","last_fullsized":"2024-09-11T07:41:15.679434+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:34.645645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:40:34.645645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:53.530666+0000","scrub_duration":0.0077905500000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"234'19","reported_seq":167,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.676172+0000","last_change":"2024-09-11T07:38:25.113379+0000","last_active":"2024-09-11T07:41:15.676172+0000","last_peered":"2024-09-11T07:41:15.676172+0000","last_clean":"2024-09-11T07:41:15.676172+0000","last_became_active":"2024-09-11T07:36:05.228746+0000","last_became_peered":"2024-09-11T07:36:05.228746+0000","last_unstale":"2024-09-11T07:41:15.676172+0000","last_undegraded":"2024-09-11T07:41:15.676172+0000","last_fullsized":"2024-09-11T07:41:15.676172+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:38:25.113300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:38:25.113300+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0063902960000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":341,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.676748+0000","last_change":"2024-09-11T07:37:21.150801+0000","last_active":"2024-09-11T07:41:15.676748+0000","last_peered":"2024-09-11T07:41:15.676748+0000","last_clean":"2024-09-11T07:41:15.676748+0000","last_became_active":"2024-09-11T07:32:00.507050+0000","last_became_peered":"2024-09-11T07:32:00.507050+0000","last_unstale":"2024-09-11T07:41:15.676748+0000","last_undegraded":"2024-09-11T07:41:15.676748+0000","last_fullsized":"2024-09-11T07:41:15.676748+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:21.150732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:24.575210+0000","last_clean_scrub_stamp":"2024-09-11T07:37:21.150732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001360691,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.11","version":"326'10801","reported_seq":10907,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:17.978918+0000","last_change":"2024-09-11T07:38:29.349114+0000","last_active":"2024-09-11T07:41:17.978918+0000","last_peered":"2024-09-11T07:41:17.978918+0000","last_clean":"2024-09-11T07:41:17.978918+0000","last_became_active":"2024-09-11T07:38:29.346889+0000","last_became_peered":"2024-09-11T07:38:29.346889+0000","last_unstale":"2024-09-11T07:41:17.978918+0000","last_undegraded":"2024-09-11T07:41:17.978918+0000","last_fullsized":"2024-09-11T07:41:17.978918+0000","mapping_epoch":261,"log_start":"257'4946","ondisk_log_start":"257'4946","created":259,"last_epoch_clean":262,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":5855,"log_dups_size":0,"ondisk_log_size":5855,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8124,"num_write_kb":257024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":336,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.688153+0000","last_change":"2024-09-11T07:39:28.550044+0000","last_active":"2024-09-11T07:41:18.688153+0000","last_peered":"2024-09-11T07:41:18.688153+0000","last_clean":"2024-09-11T07:41:18.688153+0000","last_became_active":"2024-09-11T07:26:37.862373+0000","last_became_peered":"2024-09-11T07:26:37.862373+0000","last_unstale":"2024-09-11T07:41:18.688153+0000","last_undegraded":"2024-09-11T07:41:18.688153+0000","last_fullsized":"2024-09-11T07:41:18.688153+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:28.549952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:39:28.549952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001250695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":274,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:18.687798+0000","last_change":"2024-09-11T07:39:54.632584+0000","last_active":"2024-09-11T07:41:18.687798+0000","last_peered":"2024-09-11T07:41:18.687798+0000","last_clean":"2024-09-11T07:41:18.687798+0000","last_became_active":"2024-09-11T07:36:05.224085+0000","last_became_peered":"2024-09-11T07:36:05.224085+0000","last_unstale":"2024-09-11T07:41:18.687798+0000","last_undegraded":"2024-09-11T07:41:18.687798+0000","last_fullsized":"2024-09-11T07:41:18.687798+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:39:54.632499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:39:54.632499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013616645,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"234'27","reported_seq":395,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.690717+0000","last_change":"2024-09-11T07:40:39.395515+0000","last_active":"2024-09-11T07:41:15.690717+0000","last_peered":"2024-09-11T07:41:15.690717+0000","last_clean":"2024-09-11T07:41:15.690717+0000","last_became_active":"2024-09-11T07:26:33.542544+0000","last_became_peered":"2024-09-11T07:26:33.542544+0000","last_unstale":"2024-09-11T07:41:15.690717+0000","last_undegraded":"2024-09-11T07:41:15.690717+0000","last_fullsized":"2024-09-11T07:41:15.690717+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'27","last_scrub_stamp":"2024-09-11T07:40:39.395458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:40:39.395458+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:04.483004+0000","scrub_duration":0.0018350700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":336,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.673570+0000","last_change":"2024-09-11T07:40:38.884375+0000","last_active":"2024-09-11T07:41:15.673570+0000","last_peered":"2024-09-11T07:41:15.673570+0000","last_clean":"2024-09-11T07:41:15.673570+0000","last_became_active":"2024-09-11T07:26:35.555862+0000","last_became_peered":"2024-09-11T07:26:35.555862+0000","last_unstale":"2024-09-11T07:41:15.673570+0000","last_undegraded":"2024-09-11T07:41:15.673570+0000","last_fullsized":"2024-09-11T07:41:15.673570+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:38.884278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:40:38.884278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:53.219499+0000","scrub_duration":0.0017591729999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"326'10691","reported_seq":10822,"reported_epoch":326,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-11T07:41:17.984817+0000","last_change":"2024-09-11T07:41:08.673041+0000","last_active":"2024-09-11T07:41:17.984817+0000","last_peered":"2024-09-11T07:41:17.984817+0000","last_clean":"2024-09-11T07:38:28.331892+0000","last_became_active":"2024-09-11T07:38:30.445788+0000","last_became_peered":"2024-09-11T07:38:30.445788+0000","last_unstale":"2024-09-11T07:41:17.984817+0000","last_undegraded":"2024-09-11T07:41:17.984817+0000","last_fullsized":"2024-09-11T07:41:17.984817+0000","mapping_epoch":262,"log_start":"268'5981","ondisk_log_start":"268'5981","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:24.833068+0000","last_clean_scrub_stamp":"2024-09-11T07:37:24.833068+0000","objects_scrubbed":0,"log_size":4710,"log_dups_size":0,"ondisk_log_size":4710,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7972,"num_write_kb":253440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[1,2,6],"avail_no_missing":["1","0","2","4","6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":332,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.680044+0000","last_change":"2024-09-11T07:39:57.769864+0000","last_active":"2024-09-11T07:41:15.680044+0000","last_peered":"2024-09-11T07:41:15.680044+0000","last_clean":"2024-09-11T07:41:15.680044+0000","last_became_active":"2024-09-11T07:39:57.768994+0000","last_became_peered":"2024-09-11T07:39:57.768994+0000","last_unstale":"2024-09-11T07:41:15.680044+0000","last_undegraded":"2024-09-11T07:41:15.680044+0000","last_fullsized":"2024-09-11T07:41:15.680044+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:32.004125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:37:32.004125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019485520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":355,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.680869+0000","last_change":"2024-09-11T07:39:57.768623+0000","last_active":"2024-09-11T07:41:15.680869+0000","last_peered":"2024-09-11T07:41:15.680869+0000","last_clean":"2024-09-11T07:41:15.680869+0000","last_became_active":"2024-09-11T07:39:57.766915+0000","last_became_peered":"2024-09-11T07:39:57.766915+0000","last_unstale":"2024-09-11T07:41:15.680869+0000","last_undegraded":"2024-09-11T07:41:15.680869+0000","last_fullsized":"2024-09-11T07:41:15.680869+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:37:42.078765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:37:42.078765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0062011560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"234'23","reported_seq":363,"reported_epoch":326,"state":"active+clean","last_fresh":"2024-09-11T07:41:15.680436+0000","last_change":"2024-09-11T07:37:43.078969+0000","last_active":"2024-09-11T07:41:15.680436+0000","last_peered":"2024-09-11T07:41:15.680436+0000","last_clean":"2024-09-11T07:41:15.680436+0000","last_became_active":"2024-09-11T07:26:33.542711+0000","last_became_peered":"2024-09-11T07:26:33.542711+0000","last_unstale":"2024-09-11T07:41:15.680436+0000","last_undegraded":"2024-09-11T07:41:15.680436+0000","last_fullsized":"2024-09-11T07:41:15.680436+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'23","last_scrub_stamp":"2024-09-11T07:37:43.078872+0000","last_deep_scrub":"127'17","last_deep_scrub_stamp":"2024-09-11T07:35:43.871918+0000","last_clean_scrub_stamp":"2024-09-11T07:37:43.078872+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051065670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":341,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-11T07:41:19.724727+0000","last_change":"2024-09-11T07:41:19.724693+0000","last_active":"2024-09-11T07:41:19.724727+0000","last_peered":"2024-09-11T07:41:19.724727+0000","last_clean":"2024-09-11T07:41:19.724727+0000","last_became_active":"2024-09-11T07:26:35.562037+0000","last_became_peered":"2024-09-11T07:26:35.562037+0000","last_unstale":"2024-09-11T07:41:19.724727+0000","last_undegraded":"2024-09-11T07:41:19.724727+0000","last_fullsized":"2024-09-11T07:41:19.724727+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:19.724643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:41:19.724643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:42:47.334990+0000","scrub_duration":0.001322174,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":26,"stat_sum":{"num_bytes":18120704,"num_objects":276,"num_object_clones":0,"num_object_copies":828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":132,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":245193,"num_write_kb":7846785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45566,"num_bytes_recovered":2986016792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569053184,"data_stored":569049112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128777,"ondisk_log_size":128777,"up":78,"acting":78,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1118,"num_read_kb":875,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":2916,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":765,"ondisk_log_size":765,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":880,"num_read_kb":875,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":147,"seq":631360192625,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":514392,"kb_used_data":185740,"kb_used_omap":4,"kb_used_meta":328635,"kb_avail":93230760,"statfs":{"total":95995035648,"available":95468298240,"internally_reserved":0,"allocated":190197760,"data_stored":189753146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":336522758},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:40:17 2024","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.375,"15min":2.6030000000000002},"min":{"1min":0.23100000000000001,"5min":0.191,"15min":0.191},"max":{"1min":4.609,"5min":14.086,"15min":49.573},"last":0.45800000000000002},{"interface":"front","average":{"1min":1.05,"5min":1.488,"15min":2.508},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":4.069,"5min":16.841999999999999,"15min":49.881},"last":0.25700000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:40:17 2024","interfaces":[{"interface":"back","average":{"1min":1.677,"5min":1.2050000000000001,"15min":0.84699999999999998},"min":{"1min":0.28299999999999997,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":5.0039999999999996,"5min":10.063000000000001,"15min":10.063000000000001},"last":0.35999999999999999},{"interface":"front","average":{"1min":1.361,"5min":1.2789999999999999,"15min":0.96999999999999997},"min":{"1min":0.25600000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.5149999999999997,"5min":9.6059999999999999,"15min":9.6059999999999999},"last":1.2909999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:40:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":3.6030000000000002,"15min":1.8400000000000001},"min":{"1min":0.25700000000000001,"5min":0.249,"15min":0.22800000000000001},"max":{"1min":4.282,"5min":227.79300000000001,"15min":227.79300000000001},"last":0.39500000000000002},{"interface":"front","average":{"1min":1.1779999999999999,"5min":3.7250000000000001,"15min":1.863},"min":{"1min":0.313,"5min":0.246,"15min":0.246},"max":{"1min":4.5529999999999999,"5min":227.22,"15min":227.22},"last":0.55700000000000005}]},{"osd":3,"last update":"Wed Sep 11 07:40:17 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.206,"15min":2.48},"min":{"1min":0.32800000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":4.6349999999999998,"5min":5.181,"15min":49.798999999999999},"last":0.375},{"interface":"front","average":{"1min":2.0680000000000001,"5min":1.339,"15min":2.4860000000000002},"min":{"1min":0.32100000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":7.3369999999999997,"5min":7.3369999999999997,"15min":49.524000000000001},"last":0.44600000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:40:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":0.85799999999999998,"15min":0.58399999999999996},"min":{"1min":0.27300000000000002,"5min":0.25700000000000001,"15min":0.253},"max":{"1min":3.9399999999999999,"5min":6.0220000000000002,"15min":6.0220000000000002},"last":0.32900000000000001},{"interface":"front","average":{"1min":1.5629999999999999,"5min":0.86699999999999999,"15min":0.69699999999999995},"min":{"1min":0.33700000000000002,"5min":0.247,"15min":0.247},"max":{"1min":6.9000000000000004,"5min":6.9000000000000004,"15min":6.9000000000000004},"last":0.42299999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:40:17 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.014,"15min":0.66700000000000004},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":5.2320000000000002,"5min":7.2160000000000002,"15min":7.2160000000000002},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.149,"5min":0.91100000000000003,"15min":0.75800000000000001},"min":{"1min":0.36299999999999999,"5min":0.27900000000000003,"15min":0.27600000000000002},"max":{"1min":4.4610000000000003,"5min":4.9969999999999999,"15min":4.9969999999999999},"last":0.48199999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:40:17 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":0.81999999999999995,"15min":0.626},"min":{"1min":0.312,"5min":0.31,"15min":0.29099999999999998},"max":{"1min":5.1909999999999998,"5min":5.1909999999999998,"15min":5.1909999999999998},"last":0.34300000000000003},{"interface":"front","average":{"1min":1.091,"5min":0.92000000000000004,"15min":0.68100000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29099999999999998},"max":{"1min":4.2030000000000003,"5min":4.2030000000000003,"15min":4.2030000000000003},"last":0.40799999999999997}]}]},{"osd":6,"up_from":44,"seq":188978561211,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":303124,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93442028,"statfs":{"total":95995035648,"available":95684636672,"internally_reserved":0,"allocated":1646592,"data_stored":1156584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:41:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0209999999999999,"15min":1.8380000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.192},"max":{"1min":5.141,"5min":20.675999999999998,"15min":142.29599999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.2430000000000001,"15min":2.008},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":8.9139999999999997,"5min":20.663,"15min":142.68799999999999},"last":0.64700000000000002}]},{"osd":1,"last update":"Wed Sep 11 07:41:21 2024","interfaces":[{"interface":"back","average":{"1min":1.617,"5min":1.982,"15min":1.6259999999999999},"min":{"1min":0.246,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":10.249000000000001,"5min":39.390000000000001,"15min":127.92100000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.821,"15min":1.583},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":5.0970000000000004,"5min":40.186999999999998,"15min":127.746},"last":0.61099999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:41:21 2024","interfaces":[{"interface":"back","average":{"1min":1.76,"5min":0.90400000000000003,"15min":0.93500000000000005},"min":{"1min":0.26600000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":15.798999999999999,"5min":15.798999999999999,"15min":26.491},"last":0.62},{"interface":"front","average":{"1min":1.413,"5min":1.0149999999999999,"15min":1.101},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":6.282,"5min":6.282,"15min":26.561},"last":0.58699999999999997}]},{"osd":3,"last update":"Wed Sep 11 07:41:21 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":2.1560000000000001,"15min":1.4950000000000001},"min":{"1min":0.27700000000000002,"5min":0.246,"15min":0.246},"max":{"1min":5.1680000000000001,"5min":49.109000000000002,"15min":49.109000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.90900000000000003,"5min":2.2610000000000001,"15min":1.587},"min":{"1min":0.28699999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":5.024,"5min":48.963999999999999,"15min":48.963999999999999},"last":0.59499999999999997}]},{"osd":4,"last update":"Wed Sep 11 07:41:21 2024","interfaces":[{"interface":"back","average":{"1min":15.026,"5min":3.5299999999999998,"15min":1.5369999999999999},"min":{"1min":0.24199999999999999,"5min":0.223,"15min":0.223},"max":{"1min":181.78800000000001,"5min":181.78800000000001,"15min":181.78800000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":7.0129999999999999,"5min":1.9570000000000001,"15min":0.96699999999999997},"min":{"1min":0.26800000000000002,"5min":0.25900000000000001,"15min":0.24399999999999999},"max":{"1min":128.59299999999999,"5min":128.59299999999999,"15min":128.59299999999999},"last":0.42499999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:41:21 2024","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.518,"15min":0.92700000000000005},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":10.935,"5min":57.436,"15min":57.436},"last":0.441},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4930000000000001,"15min":0.94599999999999995},"min":{"1min":0.253,"5min":0.253,"15min":0.251},"max":{"1min":11.15,"5min":56.822000000000003,"15min":56.822000000000003},"last":0.53600000000000003}]},{"osd":7,"last update":"Wed Sep 11 07:41:21 2024","interfaces":[{"interface":"back","average":{"1min":1.5569999999999999,"5min":0.95299999999999996,"15min":0.77800000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":11.121,"5min":11.121,"15min":11.121},"last":0.497},{"interface":"front","average":{"1min":1.329,"5min":0.92800000000000005,"15min":0.79400000000000004},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":10.962999999999999,"5min":10.962999999999999,"15min":10.962999999999999},"last":0.50800000000000001}]}]},{"osd":5,"up_from":37,"seq":158913790144,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":730844,"kb_used_data":136788,"kb_used_omap":1,"kb_used_meta":594046,"kb_avail":93014308,"statfs":{"total":95995035648,"available":95246651392,"internally_reserved":0,"allocated":140070912,"data_stored":139619105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":608303448},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:40:55 2024","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.48,"15min":2.319},"min":{"1min":0.249,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":5.3559999999999999,"5min":25.808,"15min":185.68199999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.9810000000000001,"5min":1.502,"15min":2.4340000000000002},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":17.635999999999999,"5min":25.722999999999999,"15min":186.12200000000001},"last":1.1519999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:40:55 2024","interfaces":[{"interface":"back","average":{"1min":7.5149999999999997,"5min":2.7749999999999999,"15min":1.5740000000000001},"min":{"1min":0.24299999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":122.738,"5min":122.738,"15min":122.738},"last":0.24199999999999999},{"interface":"front","average":{"1min":7.7350000000000003,"5min":2.6320000000000001,"15min":1.5900000000000001},"min":{"1min":0.24099999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":122.66200000000001,"5min":122.66200000000001,"15min":122.66200000000001},"last":1.222}]},{"osd":2,"last update":"Wed Sep 11 07:40:55 2024","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.0600000000000001,"15min":1.6870000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.22700000000000001},"max":{"1min":5.5839999999999996,"5min":5.726,"15min":187.99199999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":1.498,"5min":1.2430000000000001,"15min":1.7010000000000001},"min":{"1min":0.29699999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":5.3029999999999999,"5min":13.016,"15min":186.375},"last":0.28399999999999997}]},{"osd":3,"last update":"Wed Sep 11 07:40:55 2024","interfaces":[{"interface":"back","average":{"1min":8.6880000000000006,"5min":2.9529999999999998,"15min":2.899},"min":{"1min":0.30299999999999999,"5min":0.26800000000000002,"15min":0.23599999999999999},"max":{"1min":122.499,"5min":122.499,"15min":188.11500000000001},"last":0.436},{"interface":"front","average":{"1min":8.8960000000000008,"5min":3.2549999999999999,"15min":2.9630000000000001},"min":{"1min":0.41999999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":122.209,"5min":122.209,"15min":186.44200000000001},"last":1.5069999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:40:55 2024","interfaces":[{"interface":"back","average":{"1min":1.6379999999999999,"5min":0.996,"15min":0.65800000000000003},"min":{"1min":0.28100000000000003,"5min":0.24199999999999999,"15min":0.19900000000000001},"max":{"1min":6.3979999999999997,"5min":11.103,"15min":11.103},"last":0.313},{"interface":"front","average":{"1min":1.758,"5min":1.0649999999999999,"15min":0.69199999999999995},"min":{"1min":0.26300000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":8.407,"5min":10.305,"15min":10.305},"last":0.39400000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:41:16 2024","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":0.94699999999999995,"15min":0.68700000000000006},"min":{"1min":0.36899999999999999,"5min":0.29199999999999998,"15min":0.26000000000000001},"max":{"1min":5.6840000000000002,"5min":5.6859999999999999,"15min":5.6859999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.002,"15min":0.68400000000000005},"min":{"1min":0.33900000000000002,"5min":0.27800000000000002,"15min":0.25800000000000001},"max":{"1min":5.4589999999999996,"5min":10.613,"15min":10.613},"last":0.34699999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:41:16 2024","interfaces":[{"interface":"back","average":{"1min":1.8100000000000001,"5min":1.569,"15min":0.95999999999999996},"min":{"1min":0.35499999999999998,"5min":0.30099999999999999,"15min":0.28000000000000003},"max":{"1min":6.0499999999999998,"5min":50.338999999999999,"15min":50.338999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.47,"15min":0.93899999999999995},"min":{"1min":0.36099999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":5.7089999999999996,"5min":50.298999999999999,"15min":50.298999999999999},"last":0.41599999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986373,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298384,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446768,"statfs":{"total":95995035648,"available":95689490432,"internally_reserved":0,"allocated":1052672,"data_stored":565794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:40:40 2024","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.042,"15min":1.222},"min":{"1min":0.22500000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":4.423,"5min":9.2330000000000005,"15min":35.308},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.89100000000000001,"15min":1.159},"min":{"1min":0.26900000000000002,"5min":0.222,"15min":0.20499999999999999},"max":{"1min":2.6379999999999999,"5min":4.5620000000000003,"15min":35.597999999999999},"last":0.46200000000000002}]},{"osd":1,"last update":"Wed Sep 11 07:40:40 2024","interfaces":[{"interface":"back","average":{"1min":5.6479999999999997,"5min":2.2629999999999999,"15min":1.484},"min":{"1min":0.35699999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":73.447999999999993,"5min":73.447999999999993,"15min":73.447999999999993},"last":0.41199999999999998},{"interface":"front","average":{"1min":5.492,"5min":2.3130000000000002,"15min":1.3460000000000001},"min":{"1min":0.26600000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":73.557000000000002,"5min":73.557000000000002,"15min":73.557000000000002},"last":0.50900000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:40:40 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":0.90000000000000002,"15min":0.97099999999999997},"min":{"1min":0.24099999999999999,"5min":0.23699999999999999,"15min":0.223},"max":{"1min":4.1550000000000002,"5min":4.1550000000000002,"15min":19.927},"last":0.48499999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":0.93899999999999995,"15min":0.98099999999999998},"min":{"1min":0.34100000000000003,"5min":0.28899999999999998,"15min":0.24399999999999999},"max":{"1min":4.0730000000000004,"5min":4.101,"15min":20.02},"last":0.46899999999999997}]},{"osd":3,"last update":"Wed Sep 11 07:40:40 2024","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3140000000000001,"15min":1.2390000000000001},"min":{"1min":0.30499999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.0339999999999998,"5min":13.853,"15min":24.029},"last":0.44400000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.236,"15min":1.157},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":2.9020000000000001,"5min":13.651,"15min":24.265000000000001},"last":0.45500000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:41:05 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":0.89400000000000002,"15min":0.62},"min":{"1min":0.28899999999999998,"5min":0.223,"15min":0.21099999999999999},"max":{"1min":5.6470000000000002,"5min":5.6470000000000002,"15min":5.6470000000000002},"last":0.308},{"interface":"front","average":{"1min":1.2909999999999999,"5min":0.90400000000000003,"15min":0.64100000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.22700000000000001},"max":{"1min":5.7450000000000001,"5min":5.7450000000000001,"15min":5.7450000000000001},"last":0.42799999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:40:30 2024","interfaces":[{"interface":"back","average":{"1min":1.9059999999999999,"5min":0.85499999999999998,"15min":0.63700000000000001},"min":{"1min":0.28399999999999997,"5min":0.255,"15min":0.20899999999999999},"max":{"1min":9.0579999999999998,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":0.375},{"interface":"front","average":{"1min":1.133,"5min":0.77200000000000002,"15min":0.61399999999999999},"min":{"1min":0.33900000000000002,"5min":0.27500000000000002,"15min":0.25800000000000001},"max":{"1min":4.2140000000000004,"5min":4.2140000000000004,"15min":9.3770000000000007},"last":0.35599999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:41:17 2024","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.05,"15min":0.73299999999999998},"min":{"1min":0.30099999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":5.6639999999999997,"5min":8.0129999999999999,"15min":8.0129999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":1.294,"5min":0.877,"15min":0.70099999999999996},"min":{"1min":0.33700000000000002,"5min":0.29799999999999999,"15min":0.29599999999999999},"max":{"1min":5.5499999999999998,"5min":5.5499999999999998,"15min":5.5499999999999998},"last":0.34799999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182601,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1009160,"kb_used_data":106108,"kb_used_omap":1,"kb_used_meta":903038,"kb_avail":92735992,"statfs":{"total":95995035648,"available":94961655808,"internally_reserved":0,"allocated":108654592,"data_stored":108168400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":924711256},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:40:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.468,"15min":1.413},"min":{"1min":0.18099999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":9.0790000000000006,"5min":18.704000000000001,"15min":38.432000000000002},"last":0.41299999999999998},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.7230000000000001,"15min":1.8160000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.19500000000000001},"max":{"1min":11.895,"5min":14.749000000000001,"15min":91.876999999999995},"last":0.26500000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:40:23 2024","interfaces":[{"interface":"back","average":{"1min":2.2309999999999999,"5min":3.1509999999999998,"15min":1.8959999999999999},"min":{"1min":0.28499999999999998,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":11.917999999999999,"5min":67.305000000000007,"15min":67.305000000000007},"last":0.36099999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":2.169,"15min":1.4279999999999999},"min":{"1min":0.27500000000000002,"5min":0.23799999999999999,"15min":0.192},"max":{"1min":6.9569999999999999,"5min":44.325000000000003,"15min":44.325000000000003},"last":0.58199999999999996}]},{"osd":2,"last update":"Wed Sep 11 07:40:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.139,"15min":1.214},"min":{"1min":0.246,"5min":0.23699999999999999,"15min":0.23200000000000001},"max":{"1min":4.6799999999999997,"5min":6.6619999999999999,"15min":8.282},"last":0.34499999999999997},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.1619999999999999,"15min":1.147},"min":{"1min":0.29299999999999998,"5min":0.252,"15min":0.20999999999999999},"max":{"1min":5.992,"5min":6.5640000000000001,"15min":8.7110000000000003},"last":0.47499999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:40:43 2024","interfaces":[{"interface":"back","average":{"1min":11.355,"5min":3.04,"15min":1.746},"min":{"1min":0.33300000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":132.76400000000001,"5min":132.76400000000001,"15min":132.76400000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":11.542,"5min":3.1960000000000002,"15min":1.9330000000000001},"min":{"1min":0.311,"5min":0.255,"15min":0.255},"max":{"1min":132.154,"5min":132.154,"15min":132.154},"last":0.40300000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:40:54 2024","interfaces":[{"interface":"back","average":{"1min":8.2970000000000006,"5min":2.52,"15min":1.635},"min":{"1min":0.41499999999999998,"5min":0.309,"15min":0.29599999999999999},"max":{"1min":137.50299999999999,"5min":137.50299999999999,"15min":137.50299999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":8.4939999999999998,"5min":2.6080000000000001,"15min":1.742},"min":{"1min":0.33800000000000002,"5min":0.28599999999999998,"15min":0.28299999999999997},"max":{"1min":137.45599999999999,"5min":137.45599999999999,"15min":137.45599999999999},"last":0.42899999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:40:23 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.131,"15min":1.2849999999999999},"min":{"1min":0.36199999999999999,"5min":0.313,"15min":0.27000000000000002},"max":{"1min":6.8680000000000003,"5min":6.8680000000000003,"15min":32.350999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.069,"15min":1.2410000000000001},"min":{"1min":0.34200000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":4.5199999999999996,"5min":4.5199999999999996,"15min":32.402999999999999},"last":0.51000000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:41:15 2024","interfaces":[{"interface":"back","average":{"1min":12.583,"5min":3.5800000000000001,"15min":2.0920000000000001},"min":{"1min":0.41599999999999998,"5min":0.32300000000000001,"15min":0.308},"max":{"1min":132.565,"5min":132.565,"15min":132.565},"last":0.45900000000000002},{"interface":"front","average":{"1min":12.816000000000001,"5min":3.5209999999999999,"15min":2.5899999999999999},"min":{"1min":0.436,"5min":0.38700000000000001,"15min":0.36099999999999999},"max":{"1min":132.202,"5min":132.202,"15min":132.202},"last":0.55200000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411534,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":306668,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93438484,"statfs":{"total":95995035648,"available":95681007616,"internally_reserved":0,"allocated":1015808,"data_stored":558050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:40:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.859,"15min":1.8169999999999999},"min":{"1min":0.193,"5min":0.17799999999999999,"15min":0.16400000000000001},"max":{"1min":4.9889999999999999,"5min":57.802999999999997,"15min":57.802999999999997},"last":0.23400000000000001},{"interface":"front","average":{"1min":1.7250000000000001,"5min":2.0289999999999999,"15min":1.919},"min":{"1min":0.23899999999999999,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":8.1899999999999995,"5min":57.759,"15min":57.759},"last":0.34399999999999997}]},{"osd":1,"last update":"Wed Sep 11 07:40:57 2024","interfaces":[{"interface":"back","average":{"1min":1.6200000000000001,"5min":1.7150000000000001,"15min":1.5529999999999999},"min":{"1min":0.31,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":5.7789999999999999,"5min":48.042000000000002,"15min":65.153999999999996},"last":0.32000000000000001},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.9990000000000001,"15min":1.764},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.216},"max":{"1min":4.9630000000000001,"5min":50.201000000000001,"15min":63.311999999999998},"last":0.309}]},{"osd":3,"last update":"Wed Sep 11 07:40:23 2024","interfaces":[{"interface":"back","average":{"1min":4.2480000000000002,"5min":2.1909999999999998,"15min":1.7629999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":67.552999999999997,"5min":67.552999999999997,"15min":67.552999999999997},"last":0.26800000000000002},{"interface":"front","average":{"1min":4.3449999999999998,"5min":2.0139999999999998,"15min":1.7589999999999999},"min":{"1min":0.23999999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":67.352999999999994,"5min":67.352999999999994,"15min":67.352999999999994},"last":0.35999999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:40:41 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.79500000000000004,"15min":1.083},"min":{"1min":0.315,"5min":0.30599999999999999,"15min":0.27500000000000002},"max":{"1min":2.4790000000000001,"5min":3.254,"15min":30.946999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.81200000000000006,"15min":0.98099999999999998},"min":{"1min":0.33900000000000002,"5min":0.313,"15min":0.29999999999999999},"max":{"1min":2.7429999999999999,"5min":3.1669999999999998,"15min":22.434000000000001},"last":0.61899999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:40:57 2024","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":0.997,"15min":1.141},"min":{"1min":0.376,"5min":0.32100000000000001,"15min":0.31900000000000001},"max":{"1min":3.319,"5min":3.3290000000000002,"15min":31.789000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.405,"5min":1.0620000000000001,"15min":1.2090000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.31},"max":{"1min":8.1690000000000005,"5min":8.1690000000000005,"15min":30.898},"last":0.58499999999999996}]},{"osd":6,"last update":"Wed Sep 11 07:40:23 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.89000000000000001,"15min":1.0820000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":6.891,"5min":6.891,"15min":22.399000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.875,"5min":0.92000000000000004,"15min":1.0860000000000001},"min":{"1min":0.377,"5min":0.311,"15min":0.29999999999999999},"max":{"1min":2.2400000000000002,"5min":4.1230000000000002,"15min":25.739000000000001},"last":0.55900000000000005}]},{"osd":7,"last update":"Wed Sep 11 07:41:11 2024","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":0.93700000000000006,"15min":1.581},"min":{"1min":0.45300000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":5.2199999999999998,"5min":5.2199999999999998,"15min":22.484000000000002},"last":0.56799999999999995},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.1140000000000001,"15min":1.869},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.36699999999999999},"max":{"1min":4.9059999999999997,"5min":4.9059999999999997,"15min":25.670000000000002},"last":0.60599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575057,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1011328,"kb_used_data":94516,"kb_used_omap":1,"kb_used_meta":916798,"kb_avail":92733824,"statfs":{"total":95995035648,"available":94959435776,"internally_reserved":0,"allocated":96784384,"data_stored":96307216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":938801496},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:40:22 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.6779999999999999,"15min":1.921},"min":{"1min":0.185,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":3.944,"5min":9.2669999999999995,"15min":82.218999999999994},"last":0.26600000000000001},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.7869999999999999,"15min":1.956},"min":{"1min":0.255,"5min":0.186,"15min":0.186},"max":{"1min":5.0789999999999997,"5min":8.0739999999999998,"15min":82.278999999999996},"last":0.23799999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:40:58 2024","interfaces":[{"interface":"back","average":{"1min":2.141,"5min":1.9019999999999999,"15min":1.4870000000000001},"min":{"1min":0.251,"5min":0.22,"15min":0.19800000000000001},"max":{"1min":5.9240000000000004,"5min":9.0969999999999995,"15min":9.0969999999999995},"last":0.29099999999999998},{"interface":"front","average":{"1min":1.6619999999999999,"5min":1.5920000000000001,"15min":1.516},"min":{"1min":0.27700000000000002,"5min":0.191,"15min":0.191},"max":{"1min":5.4400000000000004,"5min":20.518000000000001,"15min":20.518000000000001},"last":0.252}]},{"osd":3,"last update":"Wed Sep 11 07:41:20 2024","interfaces":[{"interface":"back","average":{"1min":6.7939999999999996,"5min":3.5129999999999999,"15min":2.7229999999999999},"min":{"1min":0.33000000000000002,"5min":0.27600000000000002,"15min":0.25800000000000001},"max":{"1min":90.766999999999996,"5min":90.766999999999996,"15min":90.766999999999996},"last":0.33000000000000002},{"interface":"front","average":{"1min":6.9690000000000003,"5min":3.4769999999999999,"15min":2.677},"min":{"1min":0.246,"5min":0.22900000000000001,"15min":0.20999999999999999},"max":{"1min":91.513000000000005,"5min":91.513000000000005,"15min":91.513000000000005},"last":0.31}]},{"osd":4,"last update":"Wed Sep 11 07:40:33 2024","interfaces":[{"interface":"back","average":{"1min":2.4940000000000002,"5min":1.8620000000000001,"15min":1.2290000000000001},"min":{"1min":0.30299999999999999,"5min":0.27500000000000002,"15min":0.27400000000000002},"max":{"1min":7.8390000000000004,"5min":7.8390000000000004,"15min":7.8390000000000004},"last":0.53100000000000003},{"interface":"front","average":{"1min":2.2450000000000001,"5min":1.7729999999999999,"15min":1.3360000000000001},"min":{"1min":0.38700000000000001,"5min":0.29999999999999999,"15min":0.26500000000000001},"max":{"1min":7.8239999999999998,"5min":9.2289999999999992,"15min":9.2289999999999992},"last":0.49299999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:40:58 2024","interfaces":[{"interface":"back","average":{"1min":2.6459999999999999,"5min":1.954,"15min":1.333},"min":{"1min":0.29599999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":8.3680000000000003,"5min":8.3680000000000003,"15min":8.3680000000000003},"last":0.48099999999999998},{"interface":"front","average":{"1min":2.681,"5min":1.869,"15min":1.419},"min":{"1min":0.35099999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":8.6809999999999992,"5min":9.1929999999999996,"15min":9.1929999999999996},"last":0.54800000000000004}]},{"osd":6,"last update":"Wed Sep 11 07:41:20 2024","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.5329999999999999,"15min":1.4199999999999999},"min":{"1min":0.33400000000000002,"5min":0.33000000000000002,"15min":0.30099999999999999},"max":{"1min":6.226,"5min":9.1850000000000005,"15min":24.018000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":1.617,"5min":1.49,"15min":1.212},"min":{"1min":0.35799999999999998,"5min":0.318,"15min":0.29999999999999999},"max":{"1min":6.0730000000000004,"5min":9.3339999999999996,"15min":23.681000000000001},"last":0.51000000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:41:20 2024","interfaces":[{"interface":"back","average":{"1min":1.77,"5min":1.8029999999999999,"15min":1.4450000000000001},"min":{"1min":0.373,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":6.1070000000000002,"5min":9.3629999999999995,"15min":9.3629999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":1.796,"5min":2.0790000000000002,"15min":1.6799999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.313},"max":{"1min":6.0090000000000003,"5min":8.2219999999999995,"15min":8.2219999999999995},"last":0.57999999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738583,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":654844,"kb_used_data":56624,"kb_used_omap":1,"kb_used_meta":598206,"kb_avail":93090308,"statfs":{"total":95995035648,"available":95324475392,"internally_reserved":0,"allocated":57982976,"data_stored":57509473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":612563288},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:40:27 2024","interfaces":[{"interface":"back","average":{"1min":2.9449999999999998,"5min":1.6479999999999999,"15min":1.274},"min":{"1min":0.24399999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":15.522,"5min":15.522,"15min":15.522},"last":0.41999999999999998},{"interface":"front","average":{"1min":2.222,"5min":1.72,"15min":1.3009999999999999},"min":{"1min":0.254,"5min":0.20499999999999999,"15min":0.16600000000000001},"max":{"1min":13.988,"5min":13.988,"15min":13.988},"last":0.69399999999999995}]},{"osd":2,"last update":"Wed Sep 11 07:40:53 2024","interfaces":[{"interface":"back","average":{"1min":2.2160000000000002,"5min":3.472,"15min":2.0950000000000002},"min":{"1min":0.23499999999999999,"5min":0.188,"15min":0.188},"max":{"1min":8.8439999999999994,"5min":200.339,"15min":200.339},"last":0.435},{"interface":"front","average":{"1min":2.0489999999999999,"5min":3.3620000000000001,"15min":2.0019999999999998},"min":{"1min":0.251,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":15.481,"5min":200.40100000000001,"15min":200.40100000000001},"last":0.502}]},{"osd":3,"last update":"Wed Sep 11 07:40:27 2024","interfaces":[{"interface":"back","average":{"1min":1.99,"5min":1.6619999999999999,"15min":1.9550000000000001},"min":{"1min":0.23300000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":8.8070000000000004,"5min":9.4920000000000009,"15min":45.185000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":2.1789999999999998,"5min":1.6319999999999999,"15min":1.746},"min":{"1min":0.251,"5min":0.24299999999999999,"15min":0.22800000000000001},"max":{"1min":15.252000000000001,"5min":15.252000000000001,"15min":43.228999999999999},"last":0.56799999999999995}]},{"osd":4,"last update":"Wed Sep 11 07:40:36 2024","interfaces":[{"interface":"back","average":{"1min":3.0369999999999999,"5min":1.673,"15min":1.2490000000000001},"min":{"1min":0.311,"5min":0.26800000000000002,"15min":0.26700000000000002},"max":{"1min":30.315000000000001,"5min":30.315000000000001,"15min":30.315000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":3.3159999999999998,"5min":1.6919999999999999,"15min":1.3899999999999999},"min":{"1min":0.36899999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":30.363,"5min":30.363,"15min":30.363},"last":0.81799999999999995}]},{"osd":5,"last update":"Wed Sep 11 07:40:53 2024","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.359,"15min":1.2749999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":3.9790000000000001,"5min":5.0830000000000002,"15min":19.381},"last":0.90400000000000003},{"interface":"front","average":{"1min":2.2000000000000002,"5min":1.4970000000000001,"15min":1.405},"min":{"1min":0.38700000000000001,"5min":0.36899999999999999,"15min":0.27200000000000002},"max":{"1min":7.202,"5min":7.202,"15min":19.437000000000001},"last":0.92800000000000005}]},{"osd":6,"last update":"Wed Sep 11 07:40:27 2024","interfaces":[{"interface":"back","average":{"1min":1.76,"5min":1.353,"15min":1.331},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.316},"max":{"1min":5.5010000000000003,"5min":9.5990000000000002,"15min":23.190999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.865,"5min":1.198,"15min":1.306},"min":{"1min":0.375,"5min":0.35299999999999998,"15min":0.34200000000000003},"max":{"1min":15.404,"5min":15.404,"15min":23.632000000000001},"last":0.85399999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:41:19 2024","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.74,"15min":1.677},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":5.5309999999999997,"5min":30.43,"15min":30.43},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.502,"5min":1.819,"15min":1.7170000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.315},"max":{"1min":5.6699999999999999,"5min":31.065000000000001,"15min":31.065000000000001},"last":0.75600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":52428800,"data_stored":52428800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90636288,"data_stored":90636288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":103612416,"data_stored":103612416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":139067392,"data_stored":139067392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":183308288,"data_stored":183304216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:41:28.223 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pool set unique_pool_2 pgp_num 26 2024-09-11T07:41:28.235 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:28.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:28.236 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:28.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:28 smithi104 ceph-mon[33507]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:41:28.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:28 smithi104 ceph-mon[33507]: from='client.25619 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:41:28.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:28 smithi104 ceph-mon[33507]: osdmap e329: 8 total, 8 up, 8 in 2024-09-11T07:41:28.337 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:28.360 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:28.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:28.361 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:28.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:28 smithi022 ceph-mon[29462]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:41:28.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:28 smithi022 ceph-mon[29462]: from='client.25619 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:41:28.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:28 smithi022 ceph-mon[29462]: osdmap e329: 8 total, 8 up, 8 in 2024-09-11T07:41:28.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:28 smithi022 ceph-mon[37330]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:41:28.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:28 smithi022 ceph-mon[37330]: from='client.25619 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:41:28.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:28 smithi022 ceph-mon[37330]: osdmap e329: 8 total, 8 up, 8 in 2024-09-11T07:41:28.462 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:28.488 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:28.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:28.489 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:28.590 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:28.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:28.613 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:28.614 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:28.714 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:28.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:28.738 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:28.739 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:28.839 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:28.861 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:28.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:28.862 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:28.963 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:28.988 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:28.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:28.989 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:28.996 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-11T07:41:29.090 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:29.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:29.118 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:29.118 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:29.219 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:29.244 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:29.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:29.245 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:29.346 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:29.368 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:29.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:29.370 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:29.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:29 smithi022 ceph-mon[37330]: 5.6 deep-scrub starts 2024-09-11T07:41:29.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:29 smithi022 ceph-mon[37330]: 5.6 deep-scrub ok 2024-09-11T07:41:29.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:29 smithi022 ceph-mon[37330]: pgmap v690: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 18 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 132/1416 objects misplaced (9.322%) 2024-09-11T07:41:29.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:29 smithi022 ceph-mon[37330]: 9.a scrub starts 2024-09-11T07:41:29.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:29 smithi022 ceph-mon[37330]: 9.a scrub ok 2024-09-11T07:41:29.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:29 smithi022 ceph-mon[37330]: 6.11 scrub starts 2024-09-11T07:41:29.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:29 smithi022 ceph-mon[37330]: 6.11 scrub ok 2024-09-11T07:41:29.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:29 smithi022 ceph-mon[29462]: 5.6 deep-scrub starts 2024-09-11T07:41:29.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:29 smithi022 ceph-mon[29462]: 5.6 deep-scrub ok 2024-09-11T07:41:29.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:29 smithi022 ceph-mon[29462]: pgmap v690: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 18 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 132/1416 objects misplaced (9.322%) 2024-09-11T07:41:29.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:29 smithi022 ceph-mon[29462]: 9.a scrub starts 2024-09-11T07:41:29.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:29 smithi022 ceph-mon[29462]: 9.a scrub ok 2024-09-11T07:41:29.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:29 smithi022 ceph-mon[29462]: 6.11 scrub starts 2024-09-11T07:41:29.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:29 smithi022 ceph-mon[29462]: 6.11 scrub ok 2024-09-11T07:41:29.471 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:29.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:29.498 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:29.498 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:29.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:29 smithi104 ceph-mon[33507]: 5.6 deep-scrub starts 2024-09-11T07:41:29.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:29 smithi104 ceph-mon[33507]: 5.6 deep-scrub ok 2024-09-11T07:41:29.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:29 smithi104 ceph-mon[33507]: pgmap v690: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 18 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 132/1416 objects misplaced (9.322%) 2024-09-11T07:41:29.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:29 smithi104 ceph-mon[33507]: 9.a scrub starts 2024-09-11T07:41:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:29 smithi104 ceph-mon[33507]: 9.a scrub ok 2024-09-11T07:41:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:29 smithi104 ceph-mon[33507]: 6.11 scrub starts 2024-09-11T07:41:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:29 smithi104 ceph-mon[33507]: 6.11 scrub ok 2024-09-11T07:41:29.599 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:29.624 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:29.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:29.625 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:29.726 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:29.749 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:29.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:29.750 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:29.850 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:29.876 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:29.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:29.877 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:29.978 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:30.003 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:30.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:30.004 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:30.105 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:30.129 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:30.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:30.130 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:30.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:30 smithi022 ceph-mon[29462]: 9.6 scrub starts 2024-09-11T07:41:30.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:30 smithi022 ceph-mon[29462]: 9.6 scrub ok 2024-09-11T07:41:30.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:30 smithi022 ceph-mon[29462]: 4.d scrub starts 2024-09-11T07:41:30.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:30 smithi022 ceph-mon[29462]: 4.d scrub ok 2024-09-11T07:41:30.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:30 smithi022 ceph-mon[29462]: 6.14 scrub starts 2024-09-11T07:41:30.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:30 smithi022 ceph-mon[29462]: 6.14 scrub ok 2024-09-11T07:41:30.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:30 smithi022 ceph-mon[29462]: 4.f scrub starts 2024-09-11T07:41:30.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:30 smithi022 ceph-mon[29462]: 4.f scrub ok 2024-09-11T07:41:30.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:30 smithi022 ceph-mon[29462]: 4.10 deep-scrub starts 2024-09-11T07:41:30.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:30 smithi022 ceph-mon[29462]: 4.10 deep-scrub ok 2024-09-11T07:41:30.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:30 smithi022 ceph-mon[37330]: 9.6 scrub starts 2024-09-11T07:41:30.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:30 smithi022 ceph-mon[37330]: 9.6 scrub ok 2024-09-11T07:41:30.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:30 smithi022 ceph-mon[37330]: 4.d scrub starts 2024-09-11T07:41:30.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:30 smithi022 ceph-mon[37330]: 4.d scrub ok 2024-09-11T07:41:30.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:30 smithi022 ceph-mon[37330]: 6.14 scrub starts 2024-09-11T07:41:30.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:30 smithi022 ceph-mon[37330]: 6.14 scrub ok 2024-09-11T07:41:30.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:30 smithi022 ceph-mon[37330]: 4.f scrub starts 2024-09-11T07:41:30.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:30 smithi022 ceph-mon[37330]: 4.f scrub ok 2024-09-11T07:41:30.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:30 smithi022 ceph-mon[37330]: 4.10 deep-scrub starts 2024-09-11T07:41:30.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:30 smithi022 ceph-mon[37330]: 4.10 deep-scrub ok 2024-09-11T07:41:30.231 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:30.255 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:30.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:30.256 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:30.279 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:41:29 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:41:30.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:30 smithi104 ceph-mon[33507]: 9.6 scrub starts 2024-09-11T07:41:30.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:30 smithi104 ceph-mon[33507]: 9.6 scrub ok 2024-09-11T07:41:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:30 smithi104 ceph-mon[33507]: 4.d scrub starts 2024-09-11T07:41:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:30 smithi104 ceph-mon[33507]: 4.d scrub ok 2024-09-11T07:41:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:30 smithi104 ceph-mon[33507]: 6.14 scrub starts 2024-09-11T07:41:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:30 smithi104 ceph-mon[33507]: 6.14 scrub ok 2024-09-11T07:41:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:30 smithi104 ceph-mon[33507]: 4.f scrub starts 2024-09-11T07:41:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:30 smithi104 ceph-mon[33507]: 4.f scrub ok 2024-09-11T07:41:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:30 smithi104 ceph-mon[33507]: 4.10 deep-scrub starts 2024-09-11T07:41:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:30 smithi104 ceph-mon[33507]: 4.10 deep-scrub ok 2024-09-11T07:41:30.358 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:30.382 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:30.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:30.383 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:30.484 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:30.501 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:41:30 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:41:30] "GET /metrics HTTP/1.1" 200 224752 "" "Prometheus/2.43.0" 2024-09-11T07:41:30.508 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:30.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:30.509 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:30.517 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:41:30.610 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:30.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:30.634 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:30.634 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:30.735 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:30.757 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:30.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:30.758 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:30.859 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:30.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:30.883 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:30.883 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:30.895 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:41:30.985 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:31.008 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:31.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:31.009 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:31.109 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:31.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:31.133 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:31.133 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:31.234 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:31.256 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:31.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:31.257 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:31.358 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:31.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:31.386 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:31.386 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:31.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:31 smithi022 ceph-mon[37330]: 9.3 scrub starts 2024-09-11T07:41:31.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:31 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:41:31.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:31 smithi022 ceph-mon[37330]: 9.3 scrub ok 2024-09-11T07:41:31.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:31 smithi022 ceph-mon[37330]: 4.16 scrub starts 2024-09-11T07:41:31.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:31 smithi022 ceph-mon[37330]: 4.16 scrub ok 2024-09-11T07:41:31.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:31 smithi022 ceph-mon[37330]: pgmap v691: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 18 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 533 B/s rd, 0 op/s; 132/1416 objects misplaced (9.322%) 2024-09-11T07:41:31.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:31 smithi022 ceph-mon[37330]: 6.13 deep-scrub starts 2024-09-11T07:41:31.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:31 smithi022 ceph-mon[37330]: 6.13 deep-scrub ok 2024-09-11T07:41:31.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:31 smithi022 ceph-mon[37330]: osdmap e330: 8 total, 8 up, 8 in 2024-09-11T07:41:31.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:31 smithi022 ceph-mon[29462]: 9.3 scrub starts 2024-09-11T07:41:31.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:31 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:41:31.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:31 smithi022 ceph-mon[29462]: 9.3 scrub ok 2024-09-11T07:41:31.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:31 smithi022 ceph-mon[29462]: 4.16 scrub starts 2024-09-11T07:41:31.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:31 smithi022 ceph-mon[29462]: 4.16 scrub ok 2024-09-11T07:41:31.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:31 smithi022 ceph-mon[29462]: pgmap v691: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 18 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 533 B/s rd, 0 op/s; 132/1416 objects misplaced (9.322%) 2024-09-11T07:41:31.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:31 smithi022 ceph-mon[29462]: 6.13 deep-scrub starts 2024-09-11T07:41:31.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:31 smithi022 ceph-mon[29462]: 6.13 deep-scrub ok 2024-09-11T07:41:31.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:31 smithi022 ceph-mon[29462]: osdmap e330: 8 total, 8 up, 8 in 2024-09-11T07:41:31.487 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:31.505 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T07:41:31.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:31.510 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:31.510 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:31.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:31 smithi104 ceph-mon[33507]: 9.3 scrub starts 2024-09-11T07:41:31.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:31 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:41:31.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:31 smithi104 ceph-mon[33507]: 9.3 scrub ok 2024-09-11T07:41:31.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:31 smithi104 ceph-mon[33507]: 4.16 scrub starts 2024-09-11T07:41:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:31 smithi104 ceph-mon[33507]: 4.16 scrub ok 2024-09-11T07:41:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:31 smithi104 ceph-mon[33507]: pgmap v691: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 18 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 533 B/s rd, 0 op/s; 132/1416 objects misplaced (9.322%) 2024-09-11T07:41:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:31 smithi104 ceph-mon[33507]: 6.13 deep-scrub starts 2024-09-11T07:41:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:31 smithi104 ceph-mon[33507]: 6.13 deep-scrub ok 2024-09-11T07:41:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:31 smithi104 ceph-mon[33507]: osdmap e330: 8 total, 8 up, 8 in 2024-09-11T07:41:31.611 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:31.637 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:31.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:31.638 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:31.739 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:31.763 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:31.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:31.764 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:31.865 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:31.888 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:31.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:31.889 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:31.990 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:32.013 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:32.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:32.014 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:32.115 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:32.141 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:32.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:32.142 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:32.243 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:32.265 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:32.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:32.266 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:32.367 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:32.392 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:32.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:32.393 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:32.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:32 smithi022 ceph-mon[29462]: 9.11 scrub starts 2024-09-11T07:41:32.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:32 smithi022 ceph-mon[29462]: 9.11 scrub ok 2024-09-11T07:41:32.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:32 smithi022 ceph-mon[29462]: 6.1b scrub starts 2024-09-11T07:41:32.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:32 smithi022 ceph-mon[29462]: 6.1b scrub ok 2024-09-11T07:41:32.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:32 smithi022 ceph-mon[37330]: 9.11 scrub starts 2024-09-11T07:41:32.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:32 smithi022 ceph-mon[37330]: 9.11 scrub ok 2024-09-11T07:41:32.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:32 smithi022 ceph-mon[37330]: 6.1b scrub starts 2024-09-11T07:41:32.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:32 smithi022 ceph-mon[37330]: 6.1b scrub ok 2024-09-11T07:41:32.494 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:32.517 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:32.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:32.518 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:32.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:32 smithi104 ceph-mon[33507]: 9.11 scrub starts 2024-09-11T07:41:32.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:32 smithi104 ceph-mon[33507]: 9.11 scrub ok 2024-09-11T07:41:32.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:32 smithi104 ceph-mon[33507]: 6.1b scrub starts 2024-09-11T07:41:32.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:32 smithi104 ceph-mon[33507]: 6.1b scrub ok 2024-09-11T07:41:32.619 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:32.641 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:32.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:32.642 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:32.743 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:32.765 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:32.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:32.766 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:32.867 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:32.893 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:32.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:32.894 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:32.995 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:33.020 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:33.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:33.021 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:33.121 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:33.135 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:33 smithi022 ceph-mon[29462]: pgmap v693: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 18 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 132/1416 objects misplaced (9.322%) 2024-09-11T07:41:33.136 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:33 smithi022 ceph-mon[29462]: 6.f deep-scrub starts 2024-09-11T07:41:33.136 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:33 smithi022 ceph-mon[29462]: 6.f deep-scrub ok 2024-09-11T07:41:33.136 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:33 smithi022 ceph-mon[29462]: 4.13 deep-scrub starts 2024-09-11T07:41:33.136 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:33 smithi022 ceph-mon[29462]: 4.13 deep-scrub ok 2024-09-11T07:41:33.136 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:33 smithi022 ceph-mon[37330]: pgmap v693: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 18 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 132/1416 objects misplaced (9.322%) 2024-09-11T07:41:33.136 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:33 smithi022 ceph-mon[37330]: 6.f deep-scrub starts 2024-09-11T07:41:33.136 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:33 smithi022 ceph-mon[37330]: 6.f deep-scrub ok 2024-09-11T07:41:33.136 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:33 smithi022 ceph-mon[37330]: 4.13 deep-scrub starts 2024-09-11T07:41:33.136 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:33 smithi022 ceph-mon[37330]: 4.13 deep-scrub ok 2024-09-11T07:41:33.146 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:33.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:33.147 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:33.247 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:33.272 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:33.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:33.273 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:33.373 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:33.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:33.399 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:33.399 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:33.500 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:33.512 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:41:33.512 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:41:33.512 INFO:teuthology.orchestra.run.smithi022.stdout: "num_ops": 0 2024-09-11T07:41:33.512 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:41:33.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:33.526 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:33.527 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:33.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:33 smithi104 ceph-mon[33507]: pgmap v693: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 18 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 132/1416 objects misplaced (9.322%) 2024-09-11T07:41:33.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:33 smithi104 ceph-mon[33507]: 6.f deep-scrub starts 2024-09-11T07:41:33.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:33 smithi104 ceph-mon[33507]: 6.f deep-scrub ok 2024-09-11T07:41:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:33 smithi104 ceph-mon[33507]: 4.13 deep-scrub starts 2024-09-11T07:41:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:33 smithi104 ceph-mon[33507]: 4.13 deep-scrub ok 2024-09-11T07:41:33.627 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:33.651 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:33.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:33.652 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:33.753 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:33.779 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:33.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:33.780 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:33.881 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:33.907 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:33.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:33.908 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:33.982 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:41:33.982 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:41:33.997 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:41:33.998 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:41:34.008 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:34.012 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:41:34.012 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:41:34.027 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:41:34.027 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:41:34.034 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:34.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:34.035 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:34.042 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:41:34.043 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:41:34.058 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:41:34.059 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:41:34.074 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:41:34.074 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:41:34.091 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:41:34.091 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:41:34.136 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:34.158 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:34 smithi022 ceph-mon[29462]: 2.0 scrub starts 2024-09-11T07:41:34.158 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:34 smithi022 ceph-mon[29462]: 2.0 scrub ok 2024-09-11T07:41:34.158 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:34 smithi022 ceph-mon[29462]: 9.9 scrub starts 2024-09-11T07:41:34.159 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:34 smithi022 ceph-mon[29462]: 9.9 scrub ok 2024-09-11T07:41:34.159 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:34 smithi022 ceph-mon[29462]: 2.7 scrub starts 2024-09-11T07:41:34.159 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:34 smithi022 ceph-mon[29462]: 2.7 scrub ok 2024-09-11T07:41:34.159 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:34 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2494193899' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:41:34.159 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:34 smithi022 ceph-mon[37330]: 2.0 scrub starts 2024-09-11T07:41:34.159 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:34 smithi022 ceph-mon[37330]: 2.0 scrub ok 2024-09-11T07:41:34.159 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:34 smithi022 ceph-mon[37330]: 9.9 scrub starts 2024-09-11T07:41:34.159 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:34 smithi022 ceph-mon[37330]: 9.9 scrub ok 2024-09-11T07:41:34.159 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:34 smithi022 ceph-mon[37330]: 2.7 scrub starts 2024-09-11T07:41:34.159 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:34 smithi022 ceph-mon[37330]: 2.7 scrub ok 2024-09-11T07:41:34.160 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:34 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2494193899' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:41:34.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:34.160 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:34.161 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:34.261 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:34.284 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:34.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:34.285 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:34.385 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:34.408 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:34.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:34.410 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:34.510 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:34.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:41:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:41:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:41:34.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:34 smithi104 ceph-mon[33507]: 2.0 scrub starts 2024-09-11T07:41:34.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:34 smithi104 ceph-mon[33507]: 2.0 scrub ok 2024-09-11T07:41:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:34 smithi104 ceph-mon[33507]: 9.9 scrub starts 2024-09-11T07:41:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:34 smithi104 ceph-mon[33507]: 9.9 scrub ok 2024-09-11T07:41:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:34 smithi104 ceph-mon[33507]: 2.7 scrub starts 2024-09-11T07:41:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:34 smithi104 ceph-mon[33507]: 2.7 scrub ok 2024-09-11T07:41:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:34 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2494193899' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:41:34.533 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:34.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:34.534 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:34.634 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:34.661 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:34.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:34.662 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:34.762 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:34.785 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:34.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:34.786 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:34.887 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:34.913 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:34.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:34.914 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:35.015 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:35.037 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:35.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:35.038 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:35.051 INFO:teuthology.orchestra.run.smithi022.stderr:set pool 9 pgp_num to 26 2024-09-11T07:41:35.138 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:35.161 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:35.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:35.162 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:35.263 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:35.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:35 smithi104 ceph-mon[33507]: 5.3 scrub starts 2024-09-11T07:41:35.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:35 smithi104 ceph-mon[33507]: 5.3 scrub ok 2024-09-11T07:41:35.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:35 smithi104 ceph-mon[33507]: 6.8 scrub starts 2024-09-11T07:41:35.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:35 smithi104 ceph-mon[33507]: 6.8 scrub ok 2024-09-11T07:41:35.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:35 smithi104 ceph-mon[33507]: pgmap v694: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 18 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 132/1416 objects misplaced (9.322%) 2024-09-11T07:41:35.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:35 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3685252305' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "26"}]: dispatch 2024-09-11T07:41:35.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:35 smithi104 ceph-mon[33507]: 3.0 scrub starts 2024-09-11T07:41:35.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:35 smithi104 ceph-mon[33507]: 3.0 scrub ok 2024-09-11T07:41:35.286 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:35.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:35.287 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:35.308 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:35 smithi022 ceph-mon[29462]: 5.3 scrub starts 2024-09-11T07:41:35.309 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:35 smithi022 ceph-mon[29462]: 5.3 scrub ok 2024-09-11T07:41:35.309 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:35 smithi022 ceph-mon[29462]: 6.8 scrub starts 2024-09-11T07:41:35.309 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:35 smithi022 ceph-mon[29462]: 6.8 scrub ok 2024-09-11T07:41:35.309 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:35 smithi022 ceph-mon[29462]: pgmap v694: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 18 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 132/1416 objects misplaced (9.322%) 2024-09-11T07:41:35.309 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:35 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3685252305' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "26"}]: dispatch 2024-09-11T07:41:35.309 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:35 smithi022 ceph-mon[29462]: 3.0 scrub starts 2024-09-11T07:41:35.309 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:35 smithi022 ceph-mon[29462]: 3.0 scrub ok 2024-09-11T07:41:35.309 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:35 smithi022 ceph-mon[37330]: 5.3 scrub starts 2024-09-11T07:41:35.309 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:35 smithi022 ceph-mon[37330]: 5.3 scrub ok 2024-09-11T07:41:35.309 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:35 smithi022 ceph-mon[37330]: 6.8 scrub starts 2024-09-11T07:41:35.310 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:35 smithi022 ceph-mon[37330]: 6.8 scrub ok 2024-09-11T07:41:35.310 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:35 smithi022 ceph-mon[37330]: pgmap v694: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 18 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 132/1416 objects misplaced (9.322%) 2024-09-11T07:41:35.310 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:35 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3685252305' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "26"}]: dispatch 2024-09-11T07:41:35.310 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:35 smithi022 ceph-mon[37330]: 3.0 scrub starts 2024-09-11T07:41:35.310 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:35 smithi022 ceph-mon[37330]: 3.0 scrub ok 2024-09-11T07:41:35.335 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-11T07:41:35.387 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:35.440 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:35.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:35.441 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:35.542 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:35.564 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:35.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:35.565 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:35.599 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:41:35.666 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:35.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:35.693 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:35.693 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:35.794 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:35.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:35.817 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:35.817 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:35.918 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:35.940 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:35.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:35.941 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:36.026 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_2 2024-09-11T07:41:36.027 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pool rm unique_pool_2 unique_pool_2 --yes-i-really-really-mean-it 2024-09-11T07:41:36.041 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:36.043 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:41:36.052 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[37330]: 2.3 scrub starts 2024-09-11T07:41:36.052 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[37330]: 2.3 scrub ok 2024-09-11T07:41:36.052 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[37330]: 5.9 scrub starts 2024-09-11T07:41:36.052 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[37330]: 5.9 scrub ok 2024-09-11T07:41:36.052 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[37330]: 6.10 scrub starts 2024-09-11T07:41:36.052 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[37330]: 6.10 scrub ok 2024-09-11T07:41:36.052 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3685252305' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "26"}]': finished 2024-09-11T07:41:36.052 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[37330]: osdmap e331: 8 total, 8 up, 8 in 2024-09-11T07:41:36.052 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[37330]: 5.19 scrub starts 2024-09-11T07:41:36.052 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[37330]: 5.19 scrub ok 2024-09-11T07:41:36.053 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[29462]: 2.3 scrub starts 2024-09-11T07:41:36.053 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[29462]: 2.3 scrub ok 2024-09-11T07:41:36.053 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[29462]: 5.9 scrub starts 2024-09-11T07:41:36.053 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[29462]: 5.9 scrub ok 2024-09-11T07:41:36.053 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[29462]: 6.10 scrub starts 2024-09-11T07:41:36.053 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[29462]: 6.10 scrub ok 2024-09-11T07:41:36.053 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3685252305' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "26"}]': finished 2024-09-11T07:41:36.053 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[29462]: osdmap e331: 8 total, 8 up, 8 in 2024-09-11T07:41:36.053 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[29462]: 5.19 scrub starts 2024-09-11T07:41:36.053 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[29462]: 5.19 scrub ok 2024-09-11T07:41:36.054 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[29462]: 2.11 scrub starts 2024-09-11T07:41:36.054 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[29462]: 2.11 scrub ok 2024-09-11T07:41:36.054 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[29462]: 6.c scrub starts 2024-09-11T07:41:36.054 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[29462]: 6.c scrub ok 2024-09-11T07:41:36.054 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[29462]: osdmap e332: 8 total, 8 up, 8 in 2024-09-11T07:41:36.063 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:36.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:36.064 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:36.165 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:36.187 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:36.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:36.188 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:36.289 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:36.309 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[37330]: 2.11 scrub starts 2024-09-11T07:41:36.309 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[37330]: 2.11 scrub ok 2024-09-11T07:41:36.309 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[37330]: 6.c scrub starts 2024-09-11T07:41:36.309 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[37330]: 6.c scrub ok 2024-09-11T07:41:36.309 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:36 smithi022 ceph-mon[37330]: osdmap e332: 8 total, 8 up, 8 in 2024-09-11T07:41:36.313 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:36.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:36.315 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:36.416 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:36.439 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:36.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:36.440 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:36.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:36 smithi104 ceph-mon[33507]: 2.3 scrub starts 2024-09-11T07:41:36.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:36 smithi104 ceph-mon[33507]: 2.3 scrub ok 2024-09-11T07:41:36.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:36 smithi104 ceph-mon[33507]: 5.9 scrub starts 2024-09-11T07:41:36.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:36 smithi104 ceph-mon[33507]: 5.9 scrub ok 2024-09-11T07:41:36.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:36 smithi104 ceph-mon[33507]: 6.10 scrub starts 2024-09-11T07:41:36.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:36 smithi104 ceph-mon[33507]: 6.10 scrub ok 2024-09-11T07:41:36.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:36 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3685252305' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "26"}]': finished 2024-09-11T07:41:36.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:36 smithi104 ceph-mon[33507]: osdmap e331: 8 total, 8 up, 8 in 2024-09-11T07:41:36.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:36 smithi104 ceph-mon[33507]: 5.19 scrub starts 2024-09-11T07:41:36.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:36 smithi104 ceph-mon[33507]: 5.19 scrub ok 2024-09-11T07:41:36.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:36 smithi104 ceph-mon[33507]: 2.11 scrub starts 2024-09-11T07:41:36.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:36 smithi104 ceph-mon[33507]: 2.11 scrub ok 2024-09-11T07:41:36.532 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:36 smithi104 ceph-mon[33507]: 6.c scrub starts 2024-09-11T07:41:36.532 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:36 smithi104 ceph-mon[33507]: 6.c scrub ok 2024-09-11T07:41:36.532 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:36 smithi104 ceph-mon[33507]: osdmap e332: 8 total, 8 up, 8 in 2024-09-11T07:41:36.540 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:36.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:36.567 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:36.567 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:36.668 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:36.690 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:36.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:36.692 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:36.792 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:36.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:36.815 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:36.815 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:36.880 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:41:36.916 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:36.941 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:36.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:36.942 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:37.043 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:37.053 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:37 smithi022 ceph-mon[29462]: 9.d scrub starts 2024-09-11T07:41:37.054 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:37 smithi022 ceph-mon[29462]: 9.d scrub ok 2024-09-11T07:41:37.054 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:37 smithi022 ceph-mon[29462]: 9.b deep-scrub starts 2024-09-11T07:41:37.054 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:37 smithi022 ceph-mon[29462]: 9.b deep-scrub ok 2024-09-11T07:41:37.054 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:37 smithi022 ceph-mon[29462]: pgmap v697: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 17 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 132/1353 objects misplaced (9.756%) 2024-09-11T07:41:37.054 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:37 smithi022 ceph-mon[29462]: 2.a scrub starts 2024-09-11T07:41:37.054 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:37 smithi022 ceph-mon[29462]: 2.a scrub ok 2024-09-11T07:41:37.067 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:37.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:37.068 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:37.169 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:37.190 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:37.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:37.192 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:37.292 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:37.315 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:37.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:37.316 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:37.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:37 smithi022 ceph-mon[37330]: 9.d scrub starts 2024-09-11T07:41:37.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:37 smithi022 ceph-mon[37330]: 9.d scrub ok 2024-09-11T07:41:37.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:37 smithi022 ceph-mon[37330]: 9.b deep-scrub starts 2024-09-11T07:41:37.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:37 smithi022 ceph-mon[37330]: 9.b deep-scrub ok 2024-09-11T07:41:37.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:37 smithi022 ceph-mon[37330]: pgmap v697: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 17 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 132/1353 objects misplaced (9.756%) 2024-09-11T07:41:37.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:37 smithi022 ceph-mon[37330]: 2.a scrub starts 2024-09-11T07:41:37.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:37 smithi022 ceph-mon[37330]: 2.a scrub ok 2024-09-11T07:41:37.418 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:37.440 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:37.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:37.441 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:37.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:37 smithi104 ceph-mon[33507]: 9.d scrub starts 2024-09-11T07:41:37.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:37 smithi104 ceph-mon[33507]: 9.d scrub ok 2024-09-11T07:41:37.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:37 smithi104 ceph-mon[33507]: 9.b deep-scrub starts 2024-09-11T07:41:37.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:37 smithi104 ceph-mon[33507]: 9.b deep-scrub ok 2024-09-11T07:41:37.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:37 smithi104 ceph-mon[33507]: pgmap v697: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 17 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 132/1353 objects misplaced (9.756%) 2024-09-11T07:41:37.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:37 smithi104 ceph-mon[33507]: 2.a scrub starts 2024-09-11T07:41:37.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:37 smithi104 ceph-mon[33507]: 2.a scrub ok 2024-09-11T07:41:37.542 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:37.564 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:37.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:37.565 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:37.666 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:37.688 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:37.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:37.689 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:37.790 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:37.812 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:37.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:37.813 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:37.914 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:37.936 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:37.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:37.941 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:38.042 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:38.067 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:38.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:38.068 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:38.169 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:38.195 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:38.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:38.197 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:38.298 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:38.324 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:38.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:38.325 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:38.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:38 smithi022 ceph-mon[29462]: 6.d scrub starts 2024-09-11T07:41:38.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:38 smithi022 ceph-mon[29462]: 6.d scrub ok 2024-09-11T07:41:38.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:38 smithi022 ceph-mon[29462]: 2.1d scrub starts 2024-09-11T07:41:38.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:38 smithi022 ceph-mon[29462]: 2.1d scrub ok 2024-09-11T07:41:38.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:38 smithi022 ceph-mon[29462]: 9.0 scrub starts 2024-09-11T07:41:38.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:38 smithi022 ceph-mon[29462]: 9.0 scrub ok 2024-09-11T07:41:38.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:41:38.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:41:38.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:38 smithi022 ceph-mon[37330]: 6.d scrub starts 2024-09-11T07:41:38.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:38 smithi022 ceph-mon[37330]: 6.d scrub ok 2024-09-11T07:41:38.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:38 smithi022 ceph-mon[37330]: 2.1d scrub starts 2024-09-11T07:41:38.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:38 smithi022 ceph-mon[37330]: 2.1d scrub ok 2024-09-11T07:41:38.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:38 smithi022 ceph-mon[37330]: 9.0 scrub starts 2024-09-11T07:41:38.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:38 smithi022 ceph-mon[37330]: 9.0 scrub ok 2024-09-11T07:41:38.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:41:38.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:41:38.426 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:38.451 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:38.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:38.452 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:38.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:38 smithi104 ceph-mon[33507]: 6.d scrub starts 2024-09-11T07:41:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:38 smithi104 ceph-mon[33507]: 6.d scrub ok 2024-09-11T07:41:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:38 smithi104 ceph-mon[33507]: 2.1d scrub starts 2024-09-11T07:41:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:38 smithi104 ceph-mon[33507]: 2.1d scrub ok 2024-09-11T07:41:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:38 smithi104 ceph-mon[33507]: 9.0 scrub starts 2024-09-11T07:41:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:38 smithi104 ceph-mon[33507]: 9.0 scrub ok 2024-09-11T07:41:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:41:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:41:38.553 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:38.569 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:41:38.578 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:38.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:38.579 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:38.680 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:38.705 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:38.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:38.706 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:38.807 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:38.832 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:38.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:38.833 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:38.934 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:38.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:38.958 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:38.959 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:39.059 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:39.086 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:39.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:39.088 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:39.101 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:39 smithi022 ceph-mon[29462]: pgmap v698: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 17 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 538 B/s rd, 0 op/s; 132/1353 objects misplaced (9.756%) 2024-09-11T07:41:39.101 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:39 smithi022 ceph-mon[29462]: 6.19 scrub starts 2024-09-11T07:41:39.101 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:39 smithi022 ceph-mon[29462]: 6.19 scrub ok 2024-09-11T07:41:39.101 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:39 smithi022 ceph-mon[29462]: 2.14 deep-scrub starts 2024-09-11T07:41:39.101 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:39 smithi022 ceph-mon[29462]: 2.14 deep-scrub ok 2024-09-11T07:41:39.102 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:39 smithi022 ceph-mon[37330]: pgmap v698: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 17 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 538 B/s rd, 0 op/s; 132/1353 objects misplaced (9.756%) 2024-09-11T07:41:39.102 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:39 smithi022 ceph-mon[37330]: 6.19 scrub starts 2024-09-11T07:41:39.102 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:39 smithi022 ceph-mon[37330]: 6.19 scrub ok 2024-09-11T07:41:39.102 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:39 smithi022 ceph-mon[37330]: 2.14 deep-scrub starts 2024-09-11T07:41:39.102 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:39 smithi022 ceph-mon[37330]: 2.14 deep-scrub ok 2024-09-11T07:41:39.126 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:41:39.126 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:41:39.189 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:39.214 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:39.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:39.215 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:39.316 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:39.342 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:39.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:39.343 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:39.444 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:39.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:39.468 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:39.468 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:39.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:39 smithi104 ceph-mon[33507]: pgmap v698: 158 pgs: 3 active+remapped+backfill_toofull, 155 active+clean; 17 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 538 B/s rd, 0 op/s; 132/1353 objects misplaced (9.756%) 2024-09-11T07:41:39.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:39 smithi104 ceph-mon[33507]: 6.19 scrub starts 2024-09-11T07:41:39.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:39 smithi104 ceph-mon[33507]: 6.19 scrub ok 2024-09-11T07:41:39.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:39 smithi104 ceph-mon[33507]: 2.14 deep-scrub starts 2024-09-11T07:41:39.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:39 smithi104 ceph-mon[33507]: 2.14 deep-scrub ok 2024-09-11T07:41:39.569 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:39.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:39.595 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:39.595 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:39.696 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:39.719 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:39.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:39.720 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:39.820 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:39.844 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:39.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:39.845 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:39.936 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-11T07:41:39.946 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:40.002 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:40.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:40.004 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:40.104 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:40.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:40.129 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:40.130 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:40.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:40 smithi022 ceph-mon[29462]: 4.15 scrub starts 2024-09-11T07:41:40.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:40 smithi022 ceph-mon[29462]: 4.15 scrub ok 2024-09-11T07:41:40.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:40 smithi022 ceph-mon[29462]: osdmap e333: 8 total, 8 up, 8 in 2024-09-11T07:41:40.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:40 smithi022 ceph-mon[37330]: 4.15 scrub starts 2024-09-11T07:41:40.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:40 smithi022 ceph-mon[37330]: 4.15 scrub ok 2024-09-11T07:41:40.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:40 smithi022 ceph-mon[37330]: osdmap e333: 8 total, 8 up, 8 in 2024-09-11T07:41:40.231 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:40.254 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:40.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:40.255 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:40.279 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:41:39 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:41:40.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:40 smithi104 ceph-mon[33507]: 4.15 scrub starts 2024-09-11T07:41:40.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:40 smithi104 ceph-mon[33507]: 4.15 scrub ok 2024-09-11T07:41:40.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:40 smithi104 ceph-mon[33507]: osdmap e333: 8 total, 8 up, 8 in 2024-09-11T07:41:40.356 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:40.379 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:40.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:40.381 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:40.386 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:41:40.475 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:41:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:41:40] "GET /metrics HTTP/1.1" 200 224714 "" "Prometheus/2.43.0" 2024-09-11T07:41:40.481 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:40.505 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:40.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:40.506 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:40.607 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:40.629 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:40.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:40.630 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:40.731 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:40.753 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:40.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:40.754 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:40.855 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:40.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:40.882 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:40.882 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:40.982 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:41.008 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:41.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:41.009 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:41.027 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:41:41.028 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:41:41.028 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-11T07:41:41.028 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T07:41:41.028 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T07:41:41.028 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 config set filestore_inject_stall 3 2024-09-11T07:41:41.077 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is set 2024-09-11T07:41:41.110 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:41.134 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:41.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:41.135 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:41.235 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:41.260 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:41.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:41.261 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:41.362 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:41.385 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:41.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:41.386 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:41.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:41 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:41:41.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:41 smithi022 ceph-mon[29462]: osdmap e334: 8 total, 8 up, 8 in 2024-09-11T07:41:41.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:41 smithi022 ceph-mon[29462]: 2.10 deep-scrub starts 2024-09-11T07:41:41.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:41 smithi022 ceph-mon[29462]: 2.10 deep-scrub ok 2024-09-11T07:41:41.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:41 smithi022 ceph-mon[29462]: pgmap v701: 158 pgs: 1 peering, 2 active+remapped+backfill_toofull, 155 active+clean; 13 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 784 B/s rd, 0 op/s; 132/1197 objects misplaced (11.028%) 2024-09-11T07:41:41.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:41 smithi022 ceph-mon[29462]: 5.c scrub starts 2024-09-11T07:41:41.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:41 smithi022 ceph-mon[29462]: 5.c scrub ok 2024-09-11T07:41:41.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:41 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1680407213' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:41:41.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:41 smithi022 ceph-mon[29462]: from='client.25637 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:41:41.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:41 smithi022 ceph-mon[29462]: 5.1e scrub starts 2024-09-11T07:41:41.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:41 smithi022 ceph-mon[29462]: 5.1e scrub ok 2024-09-11T07:41:41.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:41 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:41:41.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:41 smithi022 ceph-mon[37330]: osdmap e334: 8 total, 8 up, 8 in 2024-09-11T07:41:41.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:41 smithi022 ceph-mon[37330]: 2.10 deep-scrub starts 2024-09-11T07:41:41.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:41 smithi022 ceph-mon[37330]: 2.10 deep-scrub ok 2024-09-11T07:41:41.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:41 smithi022 ceph-mon[37330]: pgmap v701: 158 pgs: 1 peering, 2 active+remapped+backfill_toofull, 155 active+clean; 13 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 784 B/s rd, 0 op/s; 132/1197 objects misplaced (11.028%) 2024-09-11T07:41:41.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:41 smithi022 ceph-mon[37330]: 5.c scrub starts 2024-09-11T07:41:41.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:41 smithi022 ceph-mon[37330]: 5.c scrub ok 2024-09-11T07:41:41.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:41 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1680407213' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:41:41.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:41 smithi022 ceph-mon[37330]: from='client.25637 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:41:41.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:41 smithi022 ceph-mon[37330]: 5.1e scrub starts 2024-09-11T07:41:41.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:41 smithi022 ceph-mon[37330]: 5.1e scrub ok 2024-09-11T07:41:41.487 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:41.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:41.511 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:41.511 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:41.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:41 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:41:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:41 smithi104 ceph-mon[33507]: osdmap e334: 8 total, 8 up, 8 in 2024-09-11T07:41:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:41 smithi104 ceph-mon[33507]: 2.10 deep-scrub starts 2024-09-11T07:41:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:41 smithi104 ceph-mon[33507]: 2.10 deep-scrub ok 2024-09-11T07:41:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:41 smithi104 ceph-mon[33507]: pgmap v701: 158 pgs: 1 peering, 2 active+remapped+backfill_toofull, 155 active+clean; 13 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 784 B/s rd, 0 op/s; 132/1197 objects misplaced (11.028%) 2024-09-11T07:41:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:41 smithi104 ceph-mon[33507]: 5.c scrub starts 2024-09-11T07:41:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:41 smithi104 ceph-mon[33507]: 5.c scrub ok 2024-09-11T07:41:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:41 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1680407213' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:41:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:41 smithi104 ceph-mon[33507]: from='client.25637 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:41:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:41 smithi104 ceph-mon[33507]: 5.1e scrub starts 2024-09-11T07:41:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:41 smithi104 ceph-mon[33507]: 5.1e scrub ok 2024-09-11T07:41:41.612 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:41.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:41.638 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:41.639 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:41.739 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:41.764 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:41.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:41.765 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:41.866 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:41.890 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:41.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:41.891 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:41.992 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:42.015 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:42.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:42.016 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:42.117 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:42.139 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:42.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:42.140 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:42.181 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:42 smithi022 ceph-mon[29462]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:41:42.182 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:42 smithi022 ceph-mon[29462]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:41:42.182 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:42 smithi022 ceph-mon[29462]: from='client.25637 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:41:42.182 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:42 smithi022 ceph-mon[29462]: osdmap e335: 8 total, 8 up, 8 in 2024-09-11T07:41:42.182 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:42 smithi022 ceph-mon[37330]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:41:42.182 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:42 smithi022 ceph-mon[37330]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:41:42.182 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:42 smithi022 ceph-mon[37330]: from='client.25637 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:41:42.182 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:42 smithi022 ceph-mon[37330]: osdmap e335: 8 total, 8 up, 8 in 2024-09-11T07:41:42.241 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:42.266 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:42.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:42.267 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:42.368 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:42.390 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:42.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:42.391 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:42.492 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:42.518 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:42.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:42.519 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:42.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:42 smithi104 ceph-mon[33507]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:41:42.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:42 smithi104 ceph-mon[33507]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:41:42.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:42 smithi104 ceph-mon[33507]: from='client.25637 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:41:42.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:42 smithi104 ceph-mon[33507]: osdmap e335: 8 total, 8 up, 8 in 2024-09-11T07:41:42.620 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:42.643 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:42.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:42.644 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:42.745 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:42.767 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:42.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:42.768 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:42.869 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:42.891 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:42.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:42.892 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:42.993 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:43.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:43.017 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:43.017 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:43.118 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:43.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:43.144 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:43.144 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:43.245 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:43.269 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:43.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:43.270 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:43.371 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:43.396 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:43.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:43.397 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:43.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:43 smithi022 ceph-mon[29462]: pgmap v703: 158 pgs: 1 peering, 2 active+remapped+backfill_toofull, 155 active+clean; 13 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 132/1197 objects misplaced (11.028%) 2024-09-11T07:41:43.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:43 smithi022 ceph-mon[37330]: pgmap v703: 158 pgs: 1 peering, 2 active+remapped+backfill_toofull, 155 active+clean; 13 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 132/1197 objects misplaced (11.028%) 2024-09-11T07:41:43.497 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:43.522 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:43.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:43.523 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:43.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:43 smithi104 ceph-mon[33507]: pgmap v703: 158 pgs: 1 peering, 2 active+remapped+backfill_toofull, 155 active+clean; 13 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 132/1197 objects misplaced (11.028%) 2024-09-11T07:41:43.604 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:41:43.624 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:43.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:43.647 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:43.647 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:43.748 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:43.774 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:43.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:43.775 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:43.876 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:43.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:43.899 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:43.946 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T07:41:44.000 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:44.026 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:44.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:44.027 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:44.127 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:44.151 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:44.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:44.152 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:44.253 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:44.277 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:44.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:44.278 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:44.293 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:41:44.294 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:41:44.309 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:41:44.309 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:41:44.325 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:41:44.325 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:41:44.328 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:41:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:41:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:41:44.341 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:41:44.341 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:41:44.357 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:41:44.357 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:41:44.373 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:41:44.373 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:41:44.379 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:44.394 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:41:44.394 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:41:44.402 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:44.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:44.403 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:44.411 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:41:44.411 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:41:44.504 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:44.528 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:44.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:44.529 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:44.629 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:44.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:44.655 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:44.655 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:44.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:44 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1191633106' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:41:44.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:44 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1191633106' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:41:44.756 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:44.779 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:44.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:44 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1191633106' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:41:44.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:44.780 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:44.841 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:41:44.881 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:44.903 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:44.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:44.904 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:45.005 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:45.031 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:45.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:45.032 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:45.133 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:45.158 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:45.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:45.159 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:45.260 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:45.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:45.284 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:45.285 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:45.385 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:45.408 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:45.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:45.409 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:45.509 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:45.536 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:45.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:45.536 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:45.546 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:45 smithi022 ceph-mon[29462]: pgmap v704: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 13 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 132/1197 objects misplaced (11.028%) 2024-09-11T07:41:45.546 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:45 smithi022 ceph-mon[37330]: pgmap v704: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 13 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 132/1197 objects misplaced (11.028%) 2024-09-11T07:41:45.637 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:45.662 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:45.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:45.663 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:45.764 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:45.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:45 smithi104 ceph-mon[33507]: pgmap v704: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 13 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 132/1197 objects misplaced (11.028%) 2024-09-11T07:41:45.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:45.787 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:45.787 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:45.888 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:45.910 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:45.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:45.911 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:46.012 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:46.037 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:46.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:46.038 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:46.139 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:46.161 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:46.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:46.162 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:46.263 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:46.285 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:46.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:46.286 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:46.387 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:46.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:46.412 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:46.412 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:46.513 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:46.539 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:46.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:46.540 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:46.641 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:46.664 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:46.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:46.665 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:46.685 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'unique_pool_2' removed 2024-09-11T07:41:46.762 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:41:46.763 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 20, 2024-09-11T07:41:46.763 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 600, 2024-09-11T07:41:46.763 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:41:46.763 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.763 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:85939 9.14 9:2a2e89c7:::benchmark_data_smithi022_133943_object85938:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e259)", 2024-09-11T07:41:46.763 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:26.333855+0000", 2024-09-11T07:41:46.763 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 200.42435763099999, 2024-09-11T07:41:46.763 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.058456533, 2024-09-11T07:41:46.763 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:46.763 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:41:46.763 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:46.764 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:46.764 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:46.764 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 85939 2024-09-11T07:41:46.764 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.764 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:46.764 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.764 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:46.764 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333855+0000", 2024-09-11T07:41:46.764 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.764 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.765 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.765 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:46.765 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333855+0000", 2024-09-11T07:41:46.765 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.765 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.765 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.765 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:46.765 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333852+0000", 2024-09-11T07:41:46.765 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:41:46.766 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.766 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.766 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:46.766 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333883+0000", 2024-09-11T07:41:46.766 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0558000000000001e-05 2024-09-11T07:41:46.766 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.766 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.766 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:46.766 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333898+0000", 2024-09-11T07:41:46.766 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5258e-05 2024-09-11T07:41:46.766 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.767 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.767 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:46.767 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333906+0000", 2024-09-11T07:41:46.767 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.892e-06 2024-09-11T07:41:46.767 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.767 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.767 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.767 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.562799+0000", 2024-09-11T07:41:46.767 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.228892701 2024-09-11T07:41:46.767 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.767 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.768 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:46.768 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.562805+0000", 2024-09-11T07:41:46.768 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0310000000000004e-06 2024-09-11T07:41:46.768 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.768 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.768 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.768 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.392245+0000", 2024-09-11T07:41:46.768 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.829440124 2024-09-11T07:41:46.768 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.768 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.768 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:46.769 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.392312+0000", 2024-09-11T07:41:46.769 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.6910999999999997e-05 2024-09-11T07:41:46.769 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.769 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:46.769 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.769 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.769 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.769 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:85943 9.4 9:27627e86:::benchmark_data_smithi022_133943_object85942:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e259)", 2024-09-11T07:41:46.769 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:26.333925+0000", 2024-09-11T07:41:46.769 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 200.424287558, 2024-09-11T07:41:46.769 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.04661077, 2024-09-11T07:41:46.770 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:46.770 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:41:46.770 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:46.770 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:46.770 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:46.770 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 85943 2024-09-11T07:41:46.770 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.770 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:46.771 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.771 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:46.771 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333925+0000", 2024-09-11T07:41:46.771 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.771 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.771 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.771 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:46.771 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333925+0000", 2024-09-11T07:41:46.772 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.772 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.772 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.772 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:46.772 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333924+0000", 2024-09-11T07:41:46.772 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:41:46.772 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.772 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.772 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:46.772 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333948+0000", 2024-09-11T07:41:46.772 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3336000000000002e-05 2024-09-11T07:41:46.773 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.773 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.773 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:46.773 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333955+0000", 2024-09-11T07:41:46.773 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.6850000000000001e-06 2024-09-11T07:41:46.773 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.773 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.773 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:46.773 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333960+0000", 2024-09-11T07:41:46.774 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7119999999999999e-06 2024-09-11T07:41:46.774 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.774 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.774 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.774 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.334072+0000", 2024-09-11T07:41:46.774 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011152799999999999 2024-09-11T07:41:46.774 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.774 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.774 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op must wait for map", 2024-09-11T07:41:46.774 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.334077+0000", 2024-09-11T07:41:46.774 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8919999999999997e-06 2024-09-11T07:41:46.775 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.775 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.775 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.775 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.562040+0000", 2024-09-11T07:41:46.775 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.227962259 2024-09-11T07:41:46.775 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.775 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.775 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:46.775 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.562047+0000", 2024-09-11T07:41:46.776 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.537e-06 2024-09-11T07:41:46.776 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.776 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.776 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.776 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.375671+0000", 2024-09-11T07:41:46.776 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.81362374599999998 2024-09-11T07:41:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for degraded object", 2024-09-11T07:41:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.376570+0000", 2024-09-11T07:41:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00089861200000000002 2024-09-11T07:41:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.380412+0000", 2024-09-11T07:41:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.003842322 2024-09-11T07:41:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.777 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.380536+0000", 2024-09-11T07:41:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000123991 2024-09-11T07:41:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:85950 9.4 9:22f2c04f:::benchmark_data_smithi022_133943_object85949:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e259)", 2024-09-11T07:41:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:26.333965+0000", 2024-09-11T07:41:46.778 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 200.424247511, 2024-09-11T07:41:46.779 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0552972730000001, 2024-09-11T07:41:46.779 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:46.779 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:41:46.779 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:46.779 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:46.779 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:46.779 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 85950 2024-09-11T07:41:46.779 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.779 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:46.779 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.779 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:46.780 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333965+0000", 2024-09-11T07:41:46.780 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.780 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.780 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.780 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:46.780 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333965+0000", 2024-09-11T07:41:46.780 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.780 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.780 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.780 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:46.780 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333965+0000", 2024-09-11T07:41:46.780 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:41:46.781 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.781 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.781 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:46.781 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333987+0000", 2024-09-11T07:41:46.781 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1985e-05 2024-09-11T07:41:46.781 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.781 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:46.783 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.783 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:46.783 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333993+0000", 2024-09-11T07:41:46.783 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7150000000000001e-06 2024-09-11T07:41:46.783 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.783 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.783 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:46.783 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333998+0000", 2024-09-11T07:41:46.783 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3649999999999997e-06 2024-09-11T07:41:46.783 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.784 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.784 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.784 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.334119+0000", 2024-09-11T07:41:46.784 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012089 2024-09-11T07:41:46.784 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.784 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.784 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting_for_map not empty", 2024-09-11T07:41:46.784 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.334123+0000", 2024-09-11T07:41:46.784 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1949999999999996e-06 2024-09-11T07:41:46.784 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.785 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.785 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.785 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.562063+0000", 2024-09-11T07:41:46.785 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.22794028499999999 2024-09-11T07:41:46.785 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.785 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.785 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:46.785 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.562068+0000", 2024-09-11T07:41:46.785 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1669999999999998e-06 2024-09-11T07:41:46.785 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.785 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.786 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.786 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.376594+0000", 2024-09-11T07:41:46.786 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.81452606699999996 2024-09-11T07:41:46.786 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.786 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.786 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for degraded object", 2024-09-11T07:41:46.786 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.379280+0000", 2024-09-11T07:41:46.786 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0026856340000000001 2024-09-11T07:41:46.786 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.786 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.787 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.787 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.389179+0000", 2024-09-11T07:41:46.787 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0098992579999999993 2024-09-11T07:41:46.787 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.787 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.787 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:46.787 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.389263+0000", 2024-09-11T07:41:46.787 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3319000000000006e-05 2024-09-11T07:41:46.787 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.787 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:46.787 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.788 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.788 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.788 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87214 9.10 9:08bc93d3:::benchmark_data_smithi022_133943_object87213:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:41:46.788 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.342684+0000", 2024-09-11T07:41:46.788 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 197.41552829400001, 2024-09-11T07:41:46.788 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1203623110000001, 2024-09-11T07:41:46.788 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:46.788 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:46.788 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:46.788 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:46.788 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:46.789 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87214 2024-09-11T07:41:46.789 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.789 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:46.789 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.789 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:46.789 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342684+0000", 2024-09-11T07:41:46.789 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.789 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.789 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.789 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342684+0000", 2024-09-11T07:41:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342682+0000", 2024-09-11T07:41:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:41:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342765+0000", 2024-09-11T07:41:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3146999999999995e-05 2024-09-11T07:41:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.790 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.791 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:46.791 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342777+0000", 2024-09-11T07:41:46.791 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1494999999999999e-05 2024-09-11T07:41:46.791 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.791 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.791 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:46.791 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342784+0000", 2024-09-11T07:41:46.791 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9750000000000001e-06 2024-09-11T07:41:46.791 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.791 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.791 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343439+0000", 2024-09-11T07:41:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00065492600000000001 2024-09-11T07:41:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343443+0000", 2024-09-11T07:41:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.583e-06 2024-09-11T07:41:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.449163+0000", 2024-09-11T07:41:46.792 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.105719841 2024-09-11T07:41:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.449317+0000", 2024-09-11T07:41:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000154358 2024-09-11T07:41:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2,4,6,7", 2024-09-11T07:41:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.450361+0000", 2024-09-11T07:41:46.793 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0010434000000000001 2024-09-11T07:41:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.456929+0000", 2024-09-11T07:41:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0065678389999999998 2024-09-11T07:41:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.457159+0000", 2024-09-11T07:41:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000230518 2024-09-11T07:41:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.794 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461254+0000", 2024-09-11T07:41:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0040944909999999996 2024-09-11T07:41:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461887+0000", 2024-09-11T07:41:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000633455 2024-09-11T07:41:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.795 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461946+0000", 2024-09-11T07:41:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9064999999999998e-05 2024-09-11T07:41:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.462748+0000", 2024-09-11T07:41:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00080188799999999995 2024-09-11T07:41:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.796 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.462836+0000", 2024-09-11T07:41:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.7596999999999995e-05 2024-09-11T07:41:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.463047+0000", 2024-09-11T07:41:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000211196 2024-09-11T07:41:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.797 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87241 9.16 9:6d9f9b4a:::benchmark_data_smithi022_133943_object87240:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:41:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.342793+0000", 2024-09-11T07:41:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 197.41542021999999, 2024-09-11T07:41:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.087127435, 2024-09-11T07:41:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87241 2024-09-11T07:41:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:46.798 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342793+0000", 2024-09-11T07:41:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342793+0000", 2024-09-11T07:41:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:46.799 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342792+0000", 2024-09-11T07:41:46.800 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:41:46.800 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.800 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.800 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:46.800 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342816+0000", 2024-09-11T07:41:46.800 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4511e-05 2024-09-11T07:41:46.800 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.800 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.800 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:46.800 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342823+0000", 2024-09-11T07:41:46.801 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.2189999999999998e-06 2024-09-11T07:41:46.801 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.801 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.801 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:46.801 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342828+0000", 2024-09-11T07:41:46.801 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5950000000000001e-06 2024-09-11T07:41:46.801 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.801 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.801 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.801 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.346264+0000", 2024-09-11T07:41:46.801 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0034359199999999999 2024-09-11T07:41:46.802 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.802 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.802 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:46.802 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.346271+0000", 2024-09-11T07:41:46.802 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9070000000000001e-06 2024-09-11T07:41:46.802 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.802 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.802 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.802 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.416568+0000", 2024-09-11T07:41:46.802 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.07029756 2024-09-11T07:41:46.802 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.802 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.416798+0000", 2024-09-11T07:41:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00022939600000000001 2024-09-11T07:41:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,4,5", 2024-09-11T07:41:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.417111+0000", 2024-09-11T07:41:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000313374 2024-09-11T07:41:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.420512+0000", 2024-09-11T07:41:46.803 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0034004019999999999 2024-09-11T07:41:46.804 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.804 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.804 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.804 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.424231+0000", 2024-09-11T07:41:46.804 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0037196479999999999 2024-09-11T07:41:46.804 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.804 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.804 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.804 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.424639+0000", 2024-09-11T07:41:46.804 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00040762699999999998 2024-09-11T07:41:46.804 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.804 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.804 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.805 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429504+0000", 2024-09-11T07:41:46.805 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0048647830000000001 2024-09-11T07:41:46.805 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.805 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.805 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:46.805 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429776+0000", 2024-09-11T07:41:46.805 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00027279600000000001 2024-09-11T07:41:46.805 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.805 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.805 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:46.805 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429920+0000", 2024-09-11T07:41:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014360299999999999 2024-09-11T07:41:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87242 9.16 9:6d646f5a:::benchmark_data_smithi022_133943_object87241:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:41:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.342889+0000", 2024-09-11T07:41:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 197.41532384600001, 2024-09-11T07:41:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0872590680000001, 2024-09-11T07:41:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:46.806 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:46.807 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:46.807 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:46.807 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87242 2024-09-11T07:41:46.807 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.807 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:46.807 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.807 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:46.807 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342889+0000", 2024-09-11T07:41:46.807 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.807 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.807 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.808 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:46.808 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342889+0000", 2024-09-11T07:41:46.808 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.808 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.808 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.808 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:46.808 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342888+0000", 2024-09-11T07:41:46.808 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:41:46.808 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.808 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.808 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342914+0000", 2024-09-11T07:41:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.6143999999999999e-05 2024-09-11T07:41:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342923+0000", 2024-09-11T07:41:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.7050000000000005e-06 2024-09-11T07:41:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342928+0000", 2024-09-11T07:41:46.809 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.13e-06 2024-09-11T07:41:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.346523+0000", 2024-09-11T07:41:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0035955869999999999 2024-09-11T07:41:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.346533+0000", 2024-09-11T07:41:46.810 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.4290000000000007e-06 2024-09-11T07:41:46.811 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.811 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.811 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.811 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.418238+0000", 2024-09-11T07:41:46.811 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0717048899999999 2024-09-11T07:41:46.811 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.811 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.811 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:46.811 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.418354+0000", 2024-09-11T07:41:46.811 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011647899999999999 2024-09-11T07:41:46.811 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.812 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.812 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,4,5", 2024-09-11T07:41:46.812 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.418586+0000", 2024-09-11T07:41:46.812 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000231618 2024-09-11T07:41:46.812 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.812 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.812 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.812 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.424264+0000", 2024-09-11T07:41:46.812 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0056779370000000001 2024-09-11T07:41:46.812 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.812 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.812 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.813 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.424711+0000", 2024-09-11T07:41:46.813 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00044714499999999998 2024-09-11T07:41:46.813 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.813 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.813 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:46.813 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429400+0000", 2024-09-11T07:41:46.813 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0046887550000000002 2024-09-11T07:41:46.813 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.813 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.813 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.814 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429971+0000", 2024-09-11T07:41:46.814 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00057166200000000002 2024-09-11T07:41:46.814 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.814 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.814 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:46.814 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.430024+0000", 2024-09-11T07:41:46.814 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3009e-05 2024-09-11T07:41:46.814 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.814 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.814 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:46.814 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.430148+0000", 2024-09-11T07:41:46.814 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012362300000000001 2024-09-11T07:41:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87252 9.12 9:4a1febda:::benchmark_data_smithi022_133943_object87251:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:41:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.342933+0000", 2024-09-11T07:41:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 197.41528017499999, 2024-09-11T07:41:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0912417050000001, 2024-09-11T07:41:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:46.815 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:46.816 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:46.816 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87252 2024-09-11T07:41:46.816 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.816 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:46.816 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.816 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:46.816 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342933+0000", 2024-09-11T07:41:46.816 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.816 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.816 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.816 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:46.817 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342933+0000", 2024-09-11T07:41:46.817 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.817 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.817 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.817 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:46.817 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342932+0000", 2024-09-11T07:41:46.817 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:41:46.817 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.817 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.817 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:46.817 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342953+0000", 2024-09-11T07:41:46.818 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0792999999999999e-05 2024-09-11T07:41:46.818 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.818 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.818 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:46.818 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342959+0000", 2024-09-11T07:41:46.818 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4869999999999999e-06 2024-09-11T07:41:46.818 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.818 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.818 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:46.818 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342964+0000", 2024-09-11T07:41:46.818 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4100000000000001e-06 2024-09-11T07:41:46.818 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.819 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.819 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.819 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.344041+0000", 2024-09-11T07:41:46.819 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0010773219999999999 2024-09-11T07:41:46.819 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.819 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.819 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:46.819 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.344048+0000", 2024-09-11T07:41:46.819 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7220000000000002e-06 2024-09-11T07:41:46.819 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.819 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.423752+0000", 2024-09-11T07:41:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.079704123 2024-09-11T07:41:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.423914+0000", 2024-09-11T07:41:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000161972 2024-09-11T07:41:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2,3,5", 2024-09-11T07:41:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.424186+0000", 2024-09-11T07:41:46.820 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000272238 2024-09-11T07:41:46.821 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.821 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.821 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.821 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429530+0000", 2024-09-11T07:41:46.821 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0053444929999999996 2024-09-11T07:41:46.821 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.821 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.821 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.821 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429824+0000", 2024-09-11T07:41:46.821 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00029352499999999999 2024-09-11T07:41:46.821 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.822 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.822 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.822 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.430566+0000", 2024-09-11T07:41:46.822 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00074221400000000005 2024-09-11T07:41:46.822 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.822 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.822 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:46.822 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.431126+0000", 2024-09-11T07:41:46.822 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00055992399999999995 2024-09-11T07:41:46.822 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.822 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.822 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.822 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.433998+0000", 2024-09-11T07:41:46.823 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0028716219999999999 2024-09-11T07:41:46.823 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.823 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.823 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:46.823 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.434055+0000", 2024-09-11T07:41:46.823 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7658999999999998e-05 2024-09-11T07:41:46.823 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.823 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.823 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:46.823 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.434174+0000", 2024-09-11T07:41:46.823 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000118891 2024-09-11T07:41:46.823 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.823 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87253 9.10 9:0ceb58b8:::benchmark_data_smithi022_133943_object87252:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:41:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.342970+0000", 2024-09-11T07:41:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 197.41524272000001, 2024-09-11T07:41:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.123611441, 2024-09-11T07:41:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:46.824 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87253 2024-09-11T07:41:46.825 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.825 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:46.825 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.825 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:46.825 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342970+0000", 2024-09-11T07:41:46.825 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.825 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.825 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.825 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:46.825 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342970+0000", 2024-09-11T07:41:46.825 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.826 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.826 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.826 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:46.826 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342969+0000", 2024-09-11T07:41:46.826 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:41:46.826 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.826 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.826 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:46.826 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342991+0000", 2024-09-11T07:41:46.826 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1855999999999999e-05 2024-09-11T07:41:46.826 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.826 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.827 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:46.827 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342998+0000", 2024-09-11T07:41:46.827 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7769999999999997e-06 2024-09-11T07:41:46.827 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.827 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.827 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:46.827 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343002+0000", 2024-09-11T07:41:46.827 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0860000000000004e-06 2024-09-11T07:41:46.827 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.827 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.827 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.828 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343467+0000", 2024-09-11T07:41:46.828 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00046544499999999999 2024-09-11T07:41:46.828 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.828 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.828 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:46.828 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343475+0000", 2024-09-11T07:41:46.828 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1400000000000002e-06 2024-09-11T07:41:46.828 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.828 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.828 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.828 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.455239+0000", 2024-09-11T07:41:46.828 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1117646999999999 2024-09-11T07:41:46.829 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.829 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.829 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:46.829 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.455378+0000", 2024-09-11T07:41:46.829 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013862299999999999 2024-09-11T07:41:46.829 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.829 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.829 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2,4,6,7", 2024-09-11T07:41:46.829 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.455674+0000", 2024-09-11T07:41:46.829 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00029640200000000003 2024-09-11T07:41:46.829 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.829 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.830 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.830 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.459952+0000", 2024-09-11T07:41:46.830 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0042780409999999998 2024-09-11T07:41:46.830 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.830 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.830 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.830 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461424+0000", 2024-09-11T07:41:46.830 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001471483 2024-09-11T07:41:46.830 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.830 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.830 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.830 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461918+0000", 2024-09-11T07:41:46.831 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00049364099999999996 2024-09-11T07:41:46.831 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.831 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.831 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.831 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.463082+0000", 2024-09-11T07:41:46.832 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00116457 2024-09-11T07:41:46.832 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.833 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.833 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.833 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.464588+0000", 2024-09-11T07:41:46.833 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0015063889999999999 2024-09-11T07:41:46.833 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.833 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.833 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:46.833 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.466379+0000", 2024-09-11T07:41:46.833 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00179065 2024-09-11T07:41:46.833 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.833 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.833 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:46.834 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.466455+0000", 2024-09-11T07:41:46.834 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5956000000000003e-05 2024-09-11T07:41:46.834 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.834 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.834 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:46.834 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.466581+0000", 2024-09-11T07:41:46.834 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012639699999999999 2024-09-11T07:41:46.834 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.834 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:46.834 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.834 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.834 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.834 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87254 9.16 9:6e7c07b3:::benchmark_data_smithi022_133943_object87253:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:41:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.343006+0000", 2024-09-11T07:41:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 197.415206337, 2024-09-11T07:41:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.094570789, 2024-09-11T07:41:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87254 2024-09-11T07:41:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.835 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:46.836 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343006+0000", 2024-09-11T07:41:46.836 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.836 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.836 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.836 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:46.836 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343006+0000", 2024-09-11T07:41:46.836 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.836 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.836 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.836 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:46.836 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343006+0000", 2024-09-11T07:41:46.836 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:41:46.837 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.837 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.837 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:46.837 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343027+0000", 2024-09-11T07:41:46.837 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1491000000000001e-05 2024-09-11T07:41:46.837 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.837 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.837 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:46.837 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343034+0000", 2024-09-11T07:41:46.837 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7009999999999999e-06 2024-09-11T07:41:46.837 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.838 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.838 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:46.838 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343038+0000", 2024-09-11T07:41:46.838 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.98e-06 2024-09-11T07:41:46.838 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.838 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.838 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.838 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.346555+0000", 2024-09-11T07:41:46.838 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00351688 2024-09-11T07:41:46.838 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.838 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.838 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:46.839 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.346561+0000", 2024-09-11T07:41:46.839 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7359999999999998e-06 2024-09-11T07:41:46.839 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.839 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.839 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.839 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.420577+0000", 2024-09-11T07:41:46.839 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0740159579999999 2024-09-11T07:41:46.839 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.839 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.839 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:46.839 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.420669+0000", 2024-09-11T07:41:46.839 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.2723e-05 2024-09-11T07:41:46.840 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.840 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.840 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,4,5", 2024-09-11T07:41:46.840 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.420997+0000", 2024-09-11T07:41:46.840 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00032735499999999999 2024-09-11T07:41:46.840 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.840 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.840 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:46.840 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429409+0000", 2024-09-11T07:41:46.840 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0084126630000000008 2024-09-11T07:41:46.840 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.840 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.841 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.841 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429463+0000", 2024-09-11T07:41:46.841 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3375999999999999e-05 2024-09-11T07:41:46.841 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.841 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.841 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.841 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.430192+0000", 2024-09-11T07:41:46.841 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00072935800000000005 2024-09-11T07:41:46.841 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.841 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.841 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.841 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.437414+0000", 2024-09-11T07:41:46.841 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0072219939999999998 2024-09-11T07:41:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.437450+0000", 2024-09-11T07:41:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5957000000000001e-05 2024-09-11T07:41:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.437577+0000", 2024-09-11T07:41:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012722100000000001 2024-09-11T07:41:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:46.842 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:234463 9.2 9:4445ea71:::benchmark_data_smithi022_133943_object107872:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:41:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:59.743553+0000", 2024-09-11T07:41:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 47.014659661000003, 2024-09-11T07:41:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1848900979999999, 2024-09-11T07:41:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 234463 2024-09-11T07:41:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.843 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:46.844 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.844 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:46.844 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.743553+0000", 2024-09-11T07:41:46.844 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.844 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.844 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.844 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:46.844 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.743553+0000", 2024-09-11T07:41:46.844 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.844 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.844 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.844 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:46.845 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.743552+0000", 2024-09-11T07:41:46.845 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:41:46.845 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.845 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.845 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:46.845 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.743558+0000", 2024-09-11T07:41:46.845 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0440000000000001e-06 2024-09-11T07:41:46.845 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.845 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.845 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:46.845 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.743573+0000", 2024-09-11T07:41:46.845 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5206e-05 2024-09-11T07:41:46.845 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.846 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.846 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:46.846 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.743614+0000", 2024-09-11T07:41:46.846 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1532000000000001e-05 2024-09-11T07:41:46.846 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.846 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.846 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.846 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.270167+0000", 2024-09-11T07:41:46.846 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.52655248300000002 2024-09-11T07:41:46.846 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.846 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.846 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:46.846 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.270264+0000", 2024-09-11T07:41:46.847 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.7097999999999998e-05 2024-09-11T07:41:46.847 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.847 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.847 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,7", 2024-09-11T07:41:46.847 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.270365+0000", 2024-09-11T07:41:46.847 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000100635 2024-09-11T07:41:46.847 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.847 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.847 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:46.847 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274143+0000", 2024-09-11T07:41:46.847 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0037784020000000001 2024-09-11T07:41:46.847 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.848 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.848 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.848 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.277336+0000", 2024-09-11T07:41:46.848 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0031928590000000001 2024-09-11T07:41:46.848 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.848 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.848 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.848 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928350+0000", 2024-09-11T07:41:46.849 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.65101431899999995 2024-09-11T07:41:46.849 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.849 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.849 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:46.849 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928401+0000", 2024-09-11T07:41:46.849 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1303e-05 2024-09-11T07:41:46.849 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.849 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.849 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:46.849 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928443+0000", 2024-09-11T07:41:46.849 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1795999999999998e-05 2024-09-11T07:41:46.850 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.850 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:46.850 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.850 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.850 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.850 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:234477 9.a 9:5131a6fa:::benchmark_data_smithi022_133943_object107886:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:41:46.850 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:59.763716+0000", 2024-09-11T07:41:46.850 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 46.994497086999999, 2024-09-11T07:41:46.850 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1647252889999999, 2024-09-11T07:41:46.850 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:46.850 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:46.850 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 234477 2024-09-11T07:41:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.763716+0000", 2024-09-11T07:41:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:46.851 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.763716+0000", 2024-09-11T07:41:46.852 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.852 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.852 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.852 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:46.852 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.763714+0000", 2024-09-11T07:41:46.852 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:41:46.852 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.852 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.852 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:46.852 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.763729+0000", 2024-09-11T07:41:46.852 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5347999999999999e-05 2024-09-11T07:41:46.852 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.853 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.853 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:46.853 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.763740+0000", 2024-09-11T07:41:46.853 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1165000000000001e-05 2024-09-11T07:41:46.853 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.853 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.853 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:46.853 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.763745+0000", 2024-09-11T07:41:46.853 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1050000000000003e-06 2024-09-11T07:41:46.853 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.269894+0000", 2024-09-11T07:41:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.50614913299999997 2024-09-11T07:41:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.269999+0000", 2024-09-11T07:41:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000105189 2024-09-11T07:41:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:41:46.854 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.270104+0000", 2024-09-11T07:41:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010485899999999999 2024-09-11T07:41:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274132+0000", 2024-09-11T07:41:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0040275689999999999 2024-09-11T07:41:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274138+0000", 2024-09-11T07:41:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5729999999999996e-06 2024-09-11T07:41:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.855 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928373+0000", 2024-09-11T07:41:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.65423462899999996 2024-09-11T07:41:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928412+0000", 2024-09-11T07:41:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9181000000000003e-05 2024-09-11T07:41:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928441+0000", 2024-09-11T07:41:46.856 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8656000000000001e-05 2024-09-11T07:41:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:234480 9.2 9:45c5c2ce:::benchmark_data_smithi022_133943_object107889:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:41:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:59.767350+0000", 2024-09-11T07:41:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 46.990862868999997, 2024-09-11T07:41:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.161190985, 2024-09-11T07:41:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:46.857 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 234480 2024-09-11T07:41:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.767350+0000", 2024-09-11T07:41:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:46.858 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.767350+0000", 2024-09-11T07:41:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.767347+0000", 2024-09-11T07:41:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:41:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.767356+0000", 2024-09-11T07:41:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.6160000000000002e-06 2024-09-11T07:41:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.859 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.767367+0000", 2024-09-11T07:41:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1946999999999999e-05 2024-09-11T07:41:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.767373+0000", 2024-09-11T07:41:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9159999999999998e-06 2024-09-11T07:41:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274356+0000", 2024-09-11T07:41:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.50698302299999998 2024-09-11T07:41:46.860 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.861 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.861 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:46.861 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274462+0000", 2024-09-11T07:41:46.861 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000105349 2024-09-11T07:41:46.861 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.861 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.861 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,7", 2024-09-11T07:41:46.861 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274568+0000", 2024-09-11T07:41:46.861 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010647899999999999 2024-09-11T07:41:46.861 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.861 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.279627+0000", 2024-09-11T07:41:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0050591680000000002 2024-09-11T07:41:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.321730+0000", 2024-09-11T07:41:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.042102300000000002 2024-09-11T07:41:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928475+0000", 2024-09-11T07:41:46.862 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.60674521000000003 2024-09-11T07:41:46.863 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.863 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.863 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:46.863 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928510+0000", 2024-09-11T07:41:46.863 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5036999999999997e-05 2024-09-11T07:41:46.863 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.863 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.863 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:46.863 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928541+0000", 2024-09-11T07:41:46.863 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0879e-05 2024-09-11T07:41:46.863 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.863 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:46.864 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.864 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.864 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.864 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:234484 9.a 9:5db53976:::benchmark_data_smithi022_133943_object107893:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:41:46.864 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:59.776130+0000", 2024-09-11T07:41:46.864 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 46.982082824000003, 2024-09-11T07:41:46.864 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1524430130000001, 2024-09-11T07:41:46.864 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:46.864 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:46.864 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:46.864 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:46.864 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:46.865 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 234484 2024-09-11T07:41:46.865 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.865 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:46.865 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.865 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:46.865 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.776130+0000", 2024-09-11T07:41:46.865 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.865 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.865 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.865 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:46.865 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.776130+0000", 2024-09-11T07:41:46.865 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.866 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.866 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.866 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:46.866 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.776128+0000", 2024-09-11T07:41:46.866 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:41:46.866 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.866 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.866 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:46.866 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.776136+0000", 2024-09-11T07:41:46.866 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3410000000000003e-06 2024-09-11T07:41:46.866 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.866 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.866 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.776165+0000", 2024-09-11T07:41:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.9031e-05 2024-09-11T07:41:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.776171+0000", 2024-09-11T07:41:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1929999999999998e-06 2024-09-11T07:41:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274214+0000", 2024-09-11T07:41:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.49804272399999999 2024-09-11T07:41:46.867 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.868 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.868 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:46.868 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274342+0000", 2024-09-11T07:41:46.868 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012783099999999999 2024-09-11T07:41:46.868 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.868 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.868 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:41:46.868 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274443+0000", 2024-09-11T07:41:46.868 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000101517 2024-09-11T07:41:46.868 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.868 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.869 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.869 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.277315+0000", 2024-09-11T07:41:46.869 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002871958 2024-09-11T07:41:46.869 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.869 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.869 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:46.869 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.277321+0000", 2024-09-11T07:41:46.869 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9569999999999999e-06 2024-09-11T07:41:46.869 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.869 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.869 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928494+0000", 2024-09-11T07:41:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.651173212 2024-09-11T07:41:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928537+0000", 2024-09-11T07:41:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2274999999999998e-05 2024-09-11T07:41:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928573+0000", 2024-09-11T07:41:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6310000000000003e-05 2024-09-11T07:41:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.870 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:46.871 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.871 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.871 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.871 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:234493 9.2 9:43b51550:::benchmark_data_smithi022_133943_object107902:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:41:46.871 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:59.812761+0000", 2024-09-11T07:41:46.871 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 46.945451837, 2024-09-11T07:41:46.871 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1158854009999999, 2024-09-11T07:41:46.871 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:46.871 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:46.871 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:46.871 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 234493 2024-09-11T07:41:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.812761+0000", 2024-09-11T07:41:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.812761+0000", 2024-09-11T07:41:46.872 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.873 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.873 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.873 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:46.873 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.812758+0000", 2024-09-11T07:41:46.873 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:41:46.873 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.873 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.873 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:46.873 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.812767+0000", 2024-09-11T07:41:46.873 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.5229999999999995e-06 2024-09-11T07:41:46.873 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.873 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.812780+0000", 2024-09-11T07:41:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2869e-05 2024-09-11T07:41:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.812786+0000", 2024-09-11T07:41:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5479999999999998e-06 2024-09-11T07:41:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.277362+0000", 2024-09-11T07:41:46.874 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.46457584499999999 2024-09-11T07:41:46.875 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.875 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.875 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:46.875 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.277474+0000", 2024-09-11T07:41:46.875 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011171 2024-09-11T07:41:46.875 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.875 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.875 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,7", 2024-09-11T07:41:46.875 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.277592+0000", 2024-09-11T07:41:46.875 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011786900000000001 2024-09-11T07:41:46.875 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.875 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.875 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:46.876 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.321737+0000", 2024-09-11T07:41:46.876 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.044145292000000003 2024-09-11T07:41:46.876 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.876 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.876 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.876 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.321772+0000", 2024-09-11T07:41:46.876 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4910000000000003e-05 2024-09-11T07:41:46.876 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.876 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.876 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928574+0000", 2024-09-11T07:41:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.60680186700000005 2024-09-11T07:41:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928615+0000", 2024-09-11T07:41:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0728e-05 2024-09-11T07:41:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928646+0000", 2024-09-11T07:41:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.1696999999999998e-05 2024-09-11T07:41:46.878 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.878 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:46.878 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.878 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.878 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.878 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235135 9.12 9:4bac11d3:::benchmark_data_smithi022_133943_object108544:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e321)", 2024-09-11T07:41:46.878 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.671922+0000", 2024-09-11T07:41:46.878 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 45.086290331000001, 2024-09-11T07:41:46.878 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.096036827, 2024-09-11T07:41:46.878 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:46.878 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:41:46.878 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:46.879 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:46.879 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:46.879 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235135 2024-09-11T07:41:46.879 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.879 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:46.879 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.879 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:46.879 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.671922+0000", 2024-09-11T07:41:46.879 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.879 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.879 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.879 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:46.880 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.671922+0000", 2024-09-11T07:41:46.880 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.880 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.880 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.880 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:46.880 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.671921+0000", 2024-09-11T07:41:46.880 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:41:46.880 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.880 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.880 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:46.880 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.671926+0000", 2024-09-11T07:41:46.881 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1359999999999996e-06 2024-09-11T07:41:46.881 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.881 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.881 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:46.881 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.671937+0000", 2024-09-11T07:41:46.881 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0676000000000001e-05 2024-09-11T07:41:46.881 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.881 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.881 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:46.881 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.671942+0000", 2024-09-11T07:41:46.881 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.062e-06 2024-09-11T07:41:46.881 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.882 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.882 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.882 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.684807+0000", 2024-09-11T07:41:46.882 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.012864855 2024-09-11T07:41:46.882 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.882 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.882 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:46.882 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.684939+0000", 2024-09-11T07:41:46.882 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013172800000000001 2024-09-11T07:41:46.882 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.882 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.883 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2,5", 2024-09-11T07:41:46.883 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.685060+0000", 2024-09-11T07:41:46.883 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000121413 2024-09-11T07:41:46.883 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.883 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.883 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.883 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.700013+0000", 2024-09-11T07:41:46.883 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.014952995 2024-09-11T07:41:46.883 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.883 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.883 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:46.883 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.700021+0000", 2024-09-11T07:41:46.883 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.6729999999999992e-06 2024-09-11T07:41:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.722409+0000", 2024-09-11T07:41:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.022387961 2024-09-11T07:41:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for missing object", 2024-09-11T07:41:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.722505+0000", 2024-09-11T07:41:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.6838999999999999e-05 2024-09-11T07:41:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.884 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.885 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.767903+0000", 2024-09-11T07:41:46.885 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.045397237 2024-09-11T07:41:46.885 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.885 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.885 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:46.885 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.767959+0000", 2024-09-11T07:41:46.885 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6588000000000001e-05 2024-09-11T07:41:46.885 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.885 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:46.885 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.885 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.885 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.886 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235140 9.12 9:4a4ad0d1:::benchmark_data_smithi022_133943_object108549:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-11T07:41:46.886 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.685741+0000", 2024-09-11T07:41:46.886 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 45.072471692000001, 2024-09-11T07:41:46.886 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.082396656, 2024-09-11T07:41:46.886 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:46.886 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:46.886 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:46.886 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:46.886 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:46.886 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235140 2024-09-11T07:41:46.886 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.886 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:46.886 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.887 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:46.887 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.685741+0000", 2024-09-11T07:41:46.887 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.887 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.887 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.887 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:46.887 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.685741+0000", 2024-09-11T07:41:46.887 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.887 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.887 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.887 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:46.888 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.685739+0000", 2024-09-11T07:41:46.888 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:41:46.888 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.888 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.888 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:46.888 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.685746+0000", 2024-09-11T07:41:46.888 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0310000000000001e-06 2024-09-11T07:41:46.888 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.888 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.888 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:46.888 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.685757+0000", 2024-09-11T07:41:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0773e-05 2024-09-11T07:41:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.685763+0000", 2024-09-11T07:41:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2400000000000004e-06 2024-09-11T07:41:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.702365+0000", 2024-09-11T07:41:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.016602519 2024-09-11T07:41:46.889 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.702372+0000", 2024-09-11T07:41:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2299999999999996e-06 2024-09-11T07:41:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.722543+0000", 2024-09-11T07:41:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0201711920000001 2024-09-11T07:41:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.890 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.722638+0000", 2024-09-11T07:41:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.4757000000000001e-05 2024-09-11T07:41:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2", 2024-09-11T07:41:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.722761+0000", 2024-09-11T07:41:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012308800000000001 2024-09-11T07:41:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.733739+0000", 2024-09-11T07:41:46.891 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.01097806 2024-09-11T07:41:46.892 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.892 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.892 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.892 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.767979+0000", 2024-09-11T07:41:46.892 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.034240697 2024-09-11T07:41:46.892 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.892 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.892 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.892 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.767996+0000", 2024-09-11T07:41:46.892 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.6189e-05 2024-09-11T07:41:46.892 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.893 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.893 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:46.893 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768036+0000", 2024-09-11T07:41:46.893 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0198999999999999e-05 2024-09-11T07:41:46.893 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.893 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.893 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:46.893 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768138+0000", 2024-09-11T07:41:46.893 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000101866 2024-09-11T07:41:46.893 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.894 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:46.894 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.894 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.894 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.894 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235178 9.12 9:4ef9439e:::benchmark_data_smithi022_133943_object108587:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-11T07:41:46.894 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.722941+0000", 2024-09-11T07:41:46.894 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 45.035271713999997, 2024-09-11T07:41:46.894 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.045418086, 2024-09-11T07:41:46.894 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:46.894 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:46.894 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:46.895 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:46.895 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:46.895 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235178 2024-09-11T07:41:46.895 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.895 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:46.895 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.895 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:46.895 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.722941+0000", 2024-09-11T07:41:46.895 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.895 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.895 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.895 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:46.896 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.722941+0000", 2024-09-11T07:41:46.896 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.896 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.896 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.896 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:46.896 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.722940+0000", 2024-09-11T07:41:46.896 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:41:46.896 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.896 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.896 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:46.896 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.722943+0000", 2024-09-11T07:41:46.897 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8650000000000001e-06 2024-09-11T07:41:46.897 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.897 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.897 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:46.897 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.722948+0000", 2024-09-11T07:41:46.897 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3390000000000003e-06 2024-09-11T07:41:46.897 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.897 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.897 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:46.897 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.722950+0000", 2024-09-11T07:41:46.897 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7609999999999999e-06 2024-09-11T07:41:46.897 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.898 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.898 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.898 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.733542+0000", 2024-09-11T07:41:46.898 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.010591313 2024-09-11T07:41:46.898 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.898 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.898 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:46.898 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.733548+0000", 2024-09-11T07:41:46.898 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7719999999999997e-06 2024-09-11T07:41:46.898 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.898 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.898 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.898 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726822+0000", 2024-09-11T07:41:46.899 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.99327332599999996 2024-09-11T07:41:46.899 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.899 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.899 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:46.899 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726931+0000", 2024-09-11T07:41:46.899 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010914 2024-09-11T07:41:46.899 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.899 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.899 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2", 2024-09-11T07:41:46.899 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.727048+0000", 2024-09-11T07:41:46.899 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011745599999999999 2024-09-11T07:41:46.899 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.900 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.900 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:46.900 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.743419+0000", 2024-09-11T07:41:46.900 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.016370268 2024-09-11T07:41:46.900 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.900 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.900 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.900 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768173+0000", 2024-09-11T07:41:46.900 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.024754438 2024-09-11T07:41:46.901 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.901 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.901 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.901 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768201+0000", 2024-09-11T07:41:46.901 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7946e-05 2024-09-11T07:41:46.901 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.901 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.901 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:46.901 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768246+0000", 2024-09-11T07:41:46.901 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4567999999999997e-05 2024-09-11T07:41:46.901 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.901 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.902 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:46.902 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768359+0000", 2024-09-11T07:41:46.902 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011356 2024-09-11T07:41:46.902 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.902 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:46.902 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.902 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.902 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.902 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235201 9.12 9:4fabe53d:::benchmark_data_smithi022_133943_object108610:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-11T07:41:46.902 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.747159+0000", 2024-09-11T07:41:46.902 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 45.011053711999999, 2024-09-11T07:41:46.902 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.021527321, 2024-09-11T07:41:46.902 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:46.903 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:46.903 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:46.903 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:46.903 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:46.903 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235201 2024-09-11T07:41:46.903 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.903 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:46.903 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.903 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:46.903 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.747159+0000", 2024-09-11T07:41:46.903 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.903 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.903 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.904 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:46.904 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.747159+0000", 2024-09-11T07:41:46.904 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.904 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.904 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.904 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:46.904 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.747157+0000", 2024-09-11T07:41:46.904 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:41:46.904 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.904 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.904 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:46.904 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.747164+0000", 2024-09-11T07:41:46.904 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7360000000000004e-06 2024-09-11T07:41:46.905 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.905 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.905 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:46.905 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.747192+0000", 2024-09-11T07:41:46.905 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8422e-05 2024-09-11T07:41:46.905 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.905 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.905 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:46.905 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.747201+0000", 2024-09-11T07:41:46.905 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.5290000000000008e-06 2024-09-11T07:41:46.905 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.905 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.906 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.906 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.747514+0000", 2024-09-11T07:41:46.906 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00031351600000000001 2024-09-11T07:41:46.906 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.906 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.906 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:46.906 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.747524+0000", 2024-09-11T07:41:46.906 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.533e-06 2024-09-11T07:41:46.906 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.906 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.906 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.906 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.730920+0000", 2024-09-11T07:41:46.907 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.98339653500000002 2024-09-11T07:41:46.907 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.907 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.907 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:46.907 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.731036+0000", 2024-09-11T07:41:46.907 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000115571 2024-09-11T07:41:46.907 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.907 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.907 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2", 2024-09-11T07:41:46.907 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.731157+0000", 2024-09-11T07:41:46.907 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000121109 2024-09-11T07:41:46.907 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.908 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.908 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:46.908 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.743425+0000", 2024-09-11T07:41:46.908 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.012267554999999999 2024-09-11T07:41:46.908 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.908 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.908 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.908 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768397+0000", 2024-09-11T07:41:46.908 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.024972765000000001 2024-09-11T07:41:46.908 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.908 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.909 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.909 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768428+0000", 2024-09-11T07:41:46.909 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0167000000000001e-05 2024-09-11T07:41:46.909 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.909 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.909 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:46.909 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768615+0000", 2024-09-11T07:41:46.909 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00018698900000000001 2024-09-11T07:41:46.909 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.909 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.909 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:46.910 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768686+0000", 2024-09-11T07:41:46.910 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1790000000000002e-05 2024-09-11T07:41:46.910 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.910 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:46.910 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.910 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.910 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.910 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235224 9.12 9:4fcd73f7:::benchmark_data_smithi022_133943_object108633:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-11T07:41:46.910 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.769489+0000", 2024-09-11T07:41:46.910 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 44.988723993999997, 2024-09-11T07:41:46.910 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.99944138900000001, 2024-09-11T07:41:46.911 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:46.911 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:46.911 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:46.911 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:46.911 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:46.911 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235224 2024-09-11T07:41:46.911 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.911 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:46.911 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.911 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:46.911 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.769489+0000", 2024-09-11T07:41:46.912 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.912 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.912 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.912 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:46.912 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.769489+0000", 2024-09-11T07:41:46.912 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.912 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.912 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.912 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:46.912 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.769487+0000", 2024-09-11T07:41:46.912 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:41:46.913 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.913 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.913 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:46.913 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.769494+0000", 2024-09-11T07:41:46.913 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.3649999999999998e-06 2024-09-11T07:41:46.913 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.913 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.913 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:46.913 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.769509+0000", 2024-09-11T07:41:46.913 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5122e-05 2024-09-11T07:41:46.913 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.914 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.914 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:46.914 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.769515+0000", 2024-09-11T07:41:46.914 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1330000000000004e-06 2024-09-11T07:41:46.914 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.914 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.914 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.914 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.771837+0000", 2024-09-11T07:41:46.914 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0023218840000000002 2024-09-11T07:41:46.914 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.914 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.915 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:46.915 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.771845+0000", 2024-09-11T07:41:46.915 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.2239999999999995e-06 2024-09-11T07:41:46.915 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.915 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.915 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.915 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.733765+0000", 2024-09-11T07:41:46.915 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.96191987199999995 2024-09-11T07:41:46.915 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.915 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.915 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:46.915 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.733865+0000", 2024-09-11T07:41:46.915 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010025499999999999 2024-09-11T07:41:46.915 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.916 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.916 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2", 2024-09-11T07:41:46.916 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.734245+0000", 2024-09-11T07:41:46.916 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00037957000000000003 2024-09-11T07:41:46.916 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.916 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.916 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:46.916 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.746336+0000", 2024-09-11T07:41:46.916 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.012091497 2024-09-11T07:41:46.916 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.916 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.917 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.917 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768732+0000", 2024-09-11T07:41:46.917 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.022395854999999999 2024-09-11T07:41:46.917 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.917 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.917 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.917 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768797+0000", 2024-09-11T07:41:46.917 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4886000000000002e-05 2024-09-11T07:41:46.917 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.917 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.917 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:46.917 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768837+0000", 2024-09-11T07:41:46.918 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0203000000000001e-05 2024-09-11T07:41:46.918 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.918 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.918 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:46.918 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768930+0000", 2024-09-11T07:41:46.918 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.2763999999999995e-05 2024-09-11T07:41:46.918 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.918 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:46.918 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.918 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.918 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.919 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235274 9.12 9:48c72601:::benchmark_data_smithi022_133943_object108683:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-11T07:41:46.919 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.815001+0000", 2024-09-11T07:41:46.919 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 44.943211351999999, 2024-09-11T07:41:46.919 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.95419106899999995, 2024-09-11T07:41:46.919 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:41:46.919 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:41:46.919 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:41:46.919 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:41:46.919 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:41:46.919 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235274 2024-09-11T07:41:46.919 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.919 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:41:46.920 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.920 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:41:46.920 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.815001+0000", 2024-09-11T07:41:46.920 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.920 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.920 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.920 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:41:46.920 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.815001+0000", 2024-09-11T07:41:46.920 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:41:46.920 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.920 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.920 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:41:46.921 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.815000+0000", 2024-09-11T07:41:46.921 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:41:46.921 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.921 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.921 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:41:46.921 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.815006+0000", 2024-09-11T07:41:46.921 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.55e-06 2024-09-11T07:41:46.921 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.921 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.921 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:41:46.921 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.815015+0000", 2024-09-11T07:41:46.922 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.4030000000000007e-06 2024-09-11T07:41:46.922 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.922 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.922 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:41:46.922 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.815019+0000", 2024-09-11T07:41:46.922 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1760000000000003e-06 2024-09-11T07:41:46.922 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.922 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.922 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.922 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.815066+0000", 2024-09-11T07:41:46.922 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7092000000000002e-05 2024-09-11T07:41:46.922 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.923 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.923 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:41:46.923 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.815074+0000", 2024-09-11T07:41:46.923 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.7069999999999992e-06 2024-09-11T07:41:46.923 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.923 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.923 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:41:46.923 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.738191+0000", 2024-09-11T07:41:46.923 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.92311757100000003 2024-09-11T07:41:46.923 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.923 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.923 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:41:46.924 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.738316+0000", 2024-09-11T07:41:46.924 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000124573 2024-09-11T07:41:46.924 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.924 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.924 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2", 2024-09-11T07:41:46.924 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.738445+0000", 2024-09-11T07:41:46.924 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012939399999999999 2024-09-11T07:41:46.924 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.924 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.924 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:41:46.924 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.749519+0000", 2024-09-11T07:41:46.924 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.011073697 2024-09-11T07:41:46.925 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.925 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.925 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.925 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768769+0000", 2024-09-11T07:41:46.925 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.019249720000000001 2024-09-11T07:41:46.925 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.925 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.925 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:41:46.925 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.769045+0000", 2024-09-11T07:41:46.925 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000276335 2024-09-11T07:41:46.925 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.925 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.926 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:41:46.926 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.769089+0000", 2024-09-11T07:41:46.926 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4314999999999998e-05 2024-09-11T07:41:46.926 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:41:46.926 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:41:46.926 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:41:46.926 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.769192+0000", 2024-09-11T07:41:46.926 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010329500000000001 2024-09-11T07:41:46.926 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.926 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:46.926 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.927 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:41:46.927 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:41:46.927 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:41:46.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:46.928 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:46.929 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:46.929 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:46 smithi022 ceph-mon[37330]: osdmap e336: 8 total, 8 up, 8 in 2024-09-11T07:41:46.929 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:46 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2242430452' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:41:46.929 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:46 smithi022 ceph-mon[37330]: from='client.25649 ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:41:46.929 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:46 smithi022 ceph-mon[37330]: pgmap v706: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 13 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s; 132/1197 objects misplaced (11.028%) 2024-09-11T07:41:46.929 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:46 smithi022 ceph-mon[29462]: osdmap e336: 8 total, 8 up, 8 in 2024-09-11T07:41:46.929 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:46 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2242430452' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:41:46.930 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:46 smithi022 ceph-mon[29462]: from='client.25649 ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:41:46.930 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:46 smithi022 ceph-mon[29462]: pgmap v706: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 13 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s; 132/1197 objects misplaced (11.028%) 2024-09-11T07:41:47.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:46 smithi104 ceph-mon[33507]: osdmap e336: 8 total, 8 up, 8 in 2024-09-11T07:41:47.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:46 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2242430452' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:41:47.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:46 smithi104 ceph-mon[33507]: from='client.25649 ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:41:47.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:46 smithi104 ceph-mon[33507]: pgmap v706: 158 pgs: 2 active+remapped+backfill_toofull, 156 active+clean; 13 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s; 132/1197 objects misplaced (11.028%) 2024-09-11T07:41:47.030 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:47.052 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:47.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:47.054 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:47.154 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:47.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:47.178 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:47.178 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:47.279 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:47.309 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:47.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:47.310 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:47.411 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:47.434 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:47.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:47.435 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:47.536 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:47.557 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:47.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:47.558 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:47.659 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:47.675 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:47 smithi022 ceph-mon[29462]: from='client.25649 ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T07:41:47.681 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:47.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:47.682 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:47.782 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:47.804 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:47.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:47.805 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:47.906 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:47.927 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:47.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:47.928 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:48.029 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:48.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:47 smithi104 ceph-mon[33507]: from='client.25649 ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T07:41:48.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:47 smithi104 ceph-mon[33507]: osdmap e337: 8 total, 8 up, 8 in 2024-09-11T07:41:48.036 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:47 smithi022 ceph-mon[29462]: osdmap e337: 8 total, 8 up, 8 in 2024-09-11T07:41:48.036 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:47 smithi022 ceph-mon[37330]: from='client.25649 ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T07:41:48.036 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:47 smithi022 ceph-mon[37330]: osdmap e337: 8 total, 8 up, 8 in 2024-09-11T07:41:48.054 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:48.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:48.055 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:48.064 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-11T07:41:48.156 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:48.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:48.179 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:48.179 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:48.280 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:48.302 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:48.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:48.303 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:48.403 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:48.427 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:48.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:48.427 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:48.528 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:48.550 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:48.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:48.551 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:48.652 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:48.674 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:48.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:48.675 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:48.775 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:48.797 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:48.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:48.798 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:48.876 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:48 smithi022 ceph-mon[37330]: pgmap v708: 132 pgs: 132 active+clean; 582 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 568 B/s rd, 0 op/s 2024-09-11T07:41:48.876 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:48 smithi022 ceph-mon[29462]: pgmap v708: 132 pgs: 132 active+clean; 582 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 568 B/s rd, 0 op/s 2024-09-11T07:41:48.899 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-11T07:41:48.900 INFO:tasks.radosbench:Beginning radosbench... 2024-09-11T07:41:48.900 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_3 2024-09-11T07:41:48.901 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pool create unique_pool_3 16 2024-09-11T07:41:48.901 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:48.923 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:48.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:48.924 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:49.025 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:49.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:48 smithi104 ceph-mon[33507]: pgmap v708: 132 pgs: 132 active+clean; 582 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 568 B/s rd, 0 op/s 2024-09-11T07:41:49.047 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:49.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:49.048 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:49.149 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:49.171 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:49.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:49.172 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:49.272 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:49.295 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:49.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:49.296 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:49.397 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:49.420 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:49.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:49.421 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:49.522 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:49.544 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:49.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:49.545 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:49.646 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:49.670 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:49.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:49.671 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:49.772 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:49.794 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:49.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:49.795 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:49.881 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:49 smithi022 ceph-mon[29462]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull) 2024-09-11T07:41:49.882 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:49 smithi022 ceph-mon[37330]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull) 2024-09-11T07:41:49.896 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:49.917 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:41:49.917 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-11T07:41:49.917 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:41:49.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:49.918 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:49.918 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:50.019 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:50.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:49 smithi104 ceph-mon[33507]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull) 2024-09-11T07:41:50.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:41:49 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:41:50.041 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:50.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:50.042 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:50.137 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:41:50.143 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:50.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:50.165 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:50.166 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:50.266 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:50.289 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:50.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:50.290 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:50.390 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:50.412 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:50.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:50.413 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:50.514 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:50.535 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:50.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:50.536 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:50.538 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:41:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:41:50] "GET /metrics HTTP/1.1" 200 224714 "" "Prometheus/2.43.0" 2024-09-11T07:41:50.611 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:41:50.612 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:41:50.638 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:50.660 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:50.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:50.660 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:50.761 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:50.783 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:50.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:50.784 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:50.866 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:50 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:41:50.867 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:50 smithi022 ceph-mon[29462]: pgmap v709: 132 pgs: 132 active+clean; 582 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-11T07:41:50.867 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:50 smithi022 ceph-mon[29462]: osdmap e338: 8 total, 8 up, 8 in 2024-09-11T07:41:50.867 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:50 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:41:50.867 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:50 smithi022 ceph-mon[37330]: pgmap v709: 132 pgs: 132 active+clean; 582 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-11T07:41:50.867 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:50 smithi022 ceph-mon[37330]: osdmap e338: 8 total, 8 up, 8 in 2024-09-11T07:41:50.886 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:50.908 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:50.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:50.909 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:51.010 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:51.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:50 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:41:51.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:50 smithi104 ceph-mon[33507]: pgmap v709: 132 pgs: 132 active+clean; 582 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-11T07:41:51.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:50 smithi104 ceph-mon[33507]: osdmap e338: 8 total, 8 up, 8 in 2024-09-11T07:41:51.032 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:51.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:51.033 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:51.134 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:51.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:51.157 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:51.157 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:51.258 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:51.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:51.281 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:51.282 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:51.332 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:41:51.382 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:51.405 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:51.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:51.406 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:51.507 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:51.531 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:51.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:51.532 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:51.633 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:51.655 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:51.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:51.656 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:51.757 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:51.781 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:51.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:51.782 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:51.883 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:51.906 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:51.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:51.907 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:52.008 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:52.030 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:52.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:52.031 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:52.132 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:52.155 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:52.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:52.156 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:52.257 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:52.283 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:52.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:52.284 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:52.385 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:52.410 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:52.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:52.411 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:52.512 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:52.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:52.538 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:52.538 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:52.639 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:52.662 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:52.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:52.663 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:52.764 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:52.787 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:52.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:52.788 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:52.889 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:52.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:52.915 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:52.915 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:53.016 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:53.042 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:53.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:53.043 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:53.143 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:53.168 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:53.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:53.169 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:53.270 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:53.294 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:53.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:53.295 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:53.396 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:53.419 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:53.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:53.420 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:53.521 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:53.543 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:53.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:53.544 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:53.645 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:53.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:53 smithi022 ceph-mon[29462]: pgmap v711: 132 pgs: 132 active+clean; 582 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 620 B/s rd, 0 op/s 2024-09-11T07:41:53.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:41:53.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:41:53.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:53 smithi022 ceph-mon[37330]: pgmap v711: 132 pgs: 132 active+clean; 582 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 620 B/s rd, 0 op/s 2024-09-11T07:41:53.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:41:53.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:41:53.669 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:53.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:53.670 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:53.771 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:53.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:53 smithi104 ceph-mon[33507]: pgmap v711: 132 pgs: 132 active+clean; 582 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 620 B/s rd, 0 op/s 2024-09-11T07:41:53.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:41:53.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:41:53.793 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:53.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:53.794 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:53.895 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:53.918 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:53.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:53.919 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:54.020 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:54.046 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:54.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:54.047 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:54.148 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:54.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:54.173 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:54.173 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:54.274 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:54.296 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:54.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:54.297 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:54.329 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:41:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:41:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:41:54.336 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is set 2024-09-11T07:41:54.399 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:54.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:54 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1455622431' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:41:54.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:54 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1455622431' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:41:54.425 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:54.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:54.426 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:54.527 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:54.552 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:54.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:54.553 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:54.654 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:54.677 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:54.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:54.678 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:54.779 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:54.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:54 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1455622431' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:41:54.802 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:54.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:54.806 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:54.907 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:54.933 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:54.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:54.934 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:55.035 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:55.061 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:55.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:55.062 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:55.162 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:55.165 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:41:55.165 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:41:55.165 INFO:teuthology.orchestra.run.smithi022.stdout: "num_ops": 0 2024-09-11T07:41:55.166 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:41:55.186 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:55.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:55.187 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:55.288 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:55.310 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:55.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:55.311 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:55.411 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:55.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:55 smithi022 ceph-mon[29462]: pgmap v712: 132 pgs: 132 active+clean; 582 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-11T07:41:55.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:55 smithi022 ceph-mon[29462]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:41:55.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:55 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1455622431' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:41:55.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:55 smithi022 ceph-mon[29462]: osdmap e339: 8 total, 8 up, 8 in 2024-09-11T07:41:55.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:55 smithi022 ceph-mon[37330]: pgmap v712: 132 pgs: 132 active+clean; 582 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-11T07:41:55.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:55 smithi022 ceph-mon[37330]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:41:55.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:55 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1455622431' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:41:55.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:55 smithi022 ceph-mon[37330]: osdmap e339: 8 total, 8 up, 8 in 2024-09-11T07:41:55.438 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:55.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:55.439 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:55.540 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:55.562 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:55.562 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:41:55.562 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:41:55.563 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-11T07:41:55.563 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 0.666057 2024-09-11T07:41:55.563 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 5 0.6660572741904994 2024-09-11T07:41:55.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:55.564 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:55.611 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:41:55.666 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:55.716 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:55.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:55.717 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:55.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:55 smithi104 ceph-mon[33507]: pgmap v712: 132 pgs: 132 active+clean; 582 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-11T07:41:55.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:55 smithi104 ceph-mon[33507]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:41:55.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:55 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1455622431' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:41:55.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:55 smithi104 ceph-mon[33507]: osdmap e339: 8 total, 8 up, 8 in 2024-09-11T07:41:55.818 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:55.843 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:55.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:55.844 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:55.945 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:55.967 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:55.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:55.968 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:56.069 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:56.091 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:56.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:56.092 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:56.193 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:56.195 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-11T07:41:56.219 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:56.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:56.220 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:56.320 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:56.342 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:56.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:56.343 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:56.444 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:56.466 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:56.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:56.467 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:56.568 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:56.585 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:41:56.593 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:56.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:56.594 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:56.689 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'unique_pool_3' created 2024-09-11T07:41:56.695 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:56.718 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:56.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:56.719 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:56.820 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:56.842 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:56.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:56.843 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:56.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:56 smithi022 ceph-mon[29462]: osdmap e340: 8 total, 8 up, 8 in 2024-09-11T07:41:56.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:56 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1357481527' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-09-11T07:41:56.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:56 smithi022 ceph-mon[29462]: from='client.25661 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-09-11T07:41:56.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:56 smithi022 ceph-mon[29462]: pgmap v715: 132 pgs: 132 active+clean; 582 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T07:41:56.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:56 smithi022 ceph-mon[37330]: osdmap e340: 8 total, 8 up, 8 in 2024-09-11T07:41:56.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:56 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1357481527' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-09-11T07:41:56.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:56 smithi022 ceph-mon[37330]: from='client.25661 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-09-11T07:41:56.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:56 smithi022 ceph-mon[37330]: pgmap v715: 132 pgs: 132 active+clean; 582 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T07:41:56.944 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:56.966 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:56.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:56.967 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:57.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:56 smithi104 ceph-mon[33507]: osdmap e340: 8 total, 8 up, 8 in 2024-09-11T07:41:57.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:56 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1357481527' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-09-11T07:41:57.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:56 smithi104 ceph-mon[33507]: from='client.25661 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-09-11T07:41:57.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:56 smithi104 ceph-mon[33507]: pgmap v715: 132 pgs: 132 active+clean; 582 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T07:41:57.068 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:57.093 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:57.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:57.094 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:57.195 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:57.218 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:57.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:57.219 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:57.320 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:57.343 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:57.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:57.344 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:57.444 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:41:57.468 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:57.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:57.469 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:41:57.570 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:57.593 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:57.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:57.594 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:57.695 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:57.721 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:57.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:57.722 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:57.822 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:57.845 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:57.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:57.846 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:57.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:57 smithi022 ceph-mon[29462]: from='client.25661 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-09-11T07:41:57.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:57 smithi022 ceph-mon[29462]: osdmap e341: 8 total, 8 up, 8 in 2024-09-11T07:41:57.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:57 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:41:57.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:57 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:41:57.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:57 smithi022 ceph-mon[37330]: from='client.25661 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-09-11T07:41:57.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:57 smithi022 ceph-mon[37330]: osdmap e341: 8 total, 8 up, 8 in 2024-09-11T07:41:57.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:57 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:41:57.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:57 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:41:57.946 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:57.970 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:57.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:57.971 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:58.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:57 smithi104 ceph-mon[33507]: from='client.25661 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-09-11T07:41:58.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:57 smithi104 ceph-mon[33507]: osdmap e341: 8 total, 8 up, 8 in 2024-09-11T07:41:58.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:57 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:41:58.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:57 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:41:58.073 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:58.096 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:58.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:58.097 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:58.198 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:58.221 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:58.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:58.222 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:58.323 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:58.346 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:58.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:58.347 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:58.448 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:41:58.474 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:58.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:58.475 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:41:58.566 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T07:41:58.576 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:58.631 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:58.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:58.632 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:58.733 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:58.756 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:58.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:58.757 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:58.801 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pool application enable unique_pool_3 rados --yes-i-really-mean-it || true 2024-09-11T07:41:58.849 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:41:58.858 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:58.884 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:58.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:58.885 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:58.888 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:41:58.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:58 smithi022 ceph-mon[29462]: osdmap e342: 8 total, 8 up, 8 in 2024-09-11T07:41:58.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:58 smithi022 ceph-mon[29462]: pgmap v718: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:41:58.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:58 smithi022 ceph-mon[37330]: osdmap e342: 8 total, 8 up, 8 in 2024-09-11T07:41:58.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:58 smithi022 ceph-mon[37330]: pgmap v718: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:41:58.985 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:59.010 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:59.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:59.012 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:59.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:58 smithi104 ceph-mon[33507]: osdmap e342: 8 total, 8 up, 8 in 2024-09-11T07:41:59.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:58 smithi104 ceph-mon[33507]: pgmap v718: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:41:59.113 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:41:59.138 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:59.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:59.139 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:41:59.240 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:59.263 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:59.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:59.264 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:59.365 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:41:59.387 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:59.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:59.387 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:41:59.488 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:41:59.510 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:59.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:59.511 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:41:59.612 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:41:59.636 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:59.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:59.638 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:41:59.739 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:41:59.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:59.765 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:41:59.765 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:41:59.865 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:41:59.890 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:41:59.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:41:59.891 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:41:59.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:59 smithi022 ceph-mon[29462]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:41:59.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:41:59 smithi022 ceph-mon[29462]: osdmap e343: 8 total, 8 up, 8 in 2024-09-11T07:41:59.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:59 smithi022 ceph-mon[37330]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:41:59.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:41:59 smithi022 ceph-mon[37330]: osdmap e343: 8 total, 8 up, 8 in 2024-09-11T07:41:59.992 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:00.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:00.018 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:00.020 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:00.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:41:59 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:42:00.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:59 smithi104 ceph-mon[33507]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:42:00.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:41:59 smithi104 ceph-mon[33507]: osdmap e343: 8 total, 8 up, 8 in 2024-09-11T07:42:00.121 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:00.145 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:00.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:00.146 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:00.246 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:00.270 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:00.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:00.271 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:00.372 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:00.398 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:00.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:00.399 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:00.500 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:00.525 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:00.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:00.526 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:00.627 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:00.649 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:00.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:00.650 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:00.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:42:00 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:42:00] "GET /metrics HTTP/1.1" 200 222317 "" "Prometheus/2.43.0" 2024-09-11T07:42:00.751 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:00.777 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:00.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:00.778 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:00.879 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:00.906 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:00.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:00.907 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:00.978 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:00 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:42:00.979 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:00 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1798006321' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:42:00.979 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:00 smithi022 ceph-mon[37330]: pgmap v720: 148 pgs: 5 unknown, 143 active+clean; 582 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail 2024-09-11T07:42:00.979 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:00 smithi022 ceph-mon[37330]: osdmap e344: 8 total, 8 up, 8 in 2024-09-11T07:42:00.979 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:00 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:42:00.979 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:00 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1798006321' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:42:00.979 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:00 smithi022 ceph-mon[29462]: pgmap v720: 148 pgs: 5 unknown, 143 active+clean; 582 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail 2024-09-11T07:42:00.979 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:00 smithi022 ceph-mon[29462]: osdmap e344: 8 total, 8 up, 8 in 2024-09-11T07:42:01.008 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:01.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:00 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:42:01.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:00 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1798006321' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:42:01.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:00 smithi104 ceph-mon[33507]: pgmap v720: 148 pgs: 5 unknown, 143 active+clean; 582 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail 2024-09-11T07:42:01.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:00 smithi104 ceph-mon[33507]: osdmap e344: 8 total, 8 up, 8 in 2024-09-11T07:42:01.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:01.032 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:01.032 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:01.104 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:42:01.104 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:42:01.104 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:42:01.104 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:42:01.104 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:42:01.104 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:42:01.104 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:42:01.105 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:42:01.105 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:42:01.105 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:42:01.105 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:42:01.105 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:42:01.105 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:42:01.105 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:42:01.105 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:42:01.105 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:42:01.134 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:01.159 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:01.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:01.161 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:01.261 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:01.287 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:01.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:01.288 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:01.389 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:01.414 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:01.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:01.415 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:01.516 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:01.540 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:01.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:01.541 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:01.641 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:01.669 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:01.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:01.670 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:01.770 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:01.796 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:01.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:01.797 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:01.850 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:01 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:42:01.850 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:01 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:42:01.850 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:01 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:42:01.850 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:01 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:42:01.851 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:01 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:42:01.851 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:01 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:42:01.851 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:01 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:42:01.851 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:01 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:42:01.897 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:01.922 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:01.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:01.923 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:02.024 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:02.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:01 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:42:02.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:01 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:42:02.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:01 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:42:02.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:01 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:42:02.048 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:02.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:02.049 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:02.150 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:02.174 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:02.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:02.175 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:02.276 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:02.300 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:02.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:02.301 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:02.402 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:02.426 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:02.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:02.427 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:02.528 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:02.553 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:02.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:02.554 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:02.655 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:02.680 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:02.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:02.681 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:02.782 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:02.805 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:02.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:02.806 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:02.880 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:02 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:42:02.881 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:02 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:42:02.881 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:02 smithi022 ceph-mon[29462]: osdmap e345: 8 total, 8 up, 8 in 2024-09-11T07:42:02.881 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:02 smithi022 ceph-mon[29462]: pgmap v723: 148 pgs: 148 active+clean; 582 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:42:02.881 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:02 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:42:02.881 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:02 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:42:02.881 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:02 smithi022 ceph-mon[37330]: osdmap e345: 8 total, 8 up, 8 in 2024-09-11T07:42:02.881 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:02 smithi022 ceph-mon[37330]: pgmap v723: 148 pgs: 148 active+clean; 582 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:42:02.907 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:02.929 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:02.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:02.930 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:03.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:02 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:42:03.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:02 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:42:03.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:02 smithi104 ceph-mon[33507]: osdmap e345: 8 total, 8 up, 8 in 2024-09-11T07:42:03.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:02 smithi104 ceph-mon[33507]: pgmap v723: 148 pgs: 148 active+clean; 582 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:42:03.031 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:03.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:03.054 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:03.054 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:03.155 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:03.179 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:03.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:03.180 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:03.281 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:03.307 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:03.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:03.309 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:03.410 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:03.435 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:03.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:03.436 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:03.536 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:03.562 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:03.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:03.563 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:03.664 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:03.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:03.690 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:03.690 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:03.718 INFO:teuthology.orchestra.run.smithi022.stderr:set osd.5 primary-affinity to 0.666057 (8436502) 2024-09-11T07:42:03.791 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:03.792 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:42:03.792 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:42:03.817 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:03.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:03.818 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:03.875 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:03 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3314375713' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.6660572741904994}]: dispatch 2024-09-11T07:42:03.875 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:03 smithi022 ceph-mon[29462]: from='client.25673 ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.6660572741904994}]: dispatch 2024-09-11T07:42:03.875 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:03 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3314375713' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.6660572741904994}]: dispatch 2024-09-11T07:42:03.875 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:03 smithi022 ceph-mon[37330]: from='client.25673 ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.6660572741904994}]: dispatch 2024-09-11T07:42:03.919 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:03.942 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:03.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:03.943 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:04.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:03 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3314375713' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.6660572741904994}]: dispatch 2024-09-11T07:42:04.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:03 smithi104 ceph-mon[33507]: from='client.25673 ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.6660572741904994}]: dispatch 2024-09-11T07:42:04.044 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:04.067 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:04.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:04.068 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:04.169 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:04.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:04.191 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:04.191 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:04.292 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:04.299 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:42:04.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:04.317 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:04.317 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:04.419 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:04.442 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:04.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:04.443 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:04.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:42:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:42:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:42:04.544 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:04.570 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:04.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:04.571 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:04.672 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:04.679 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-11T07:42:04.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:04.695 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:04.695 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:04.724 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is unset 2024-09-11T07:42:04.796 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:04.819 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:04.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:04.821 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:04.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:04 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:42:04.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:04 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:42:04.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:04 smithi022 ceph-mon[29462]: from='client.25673 ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.6660572741904994}]': finished 2024-09-11T07:42:04.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:04 smithi022 ceph-mon[29462]: osdmap e346: 8 total, 8 up, 8 in 2024-09-11T07:42:04.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:04 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3872154691' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:42:04.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:04 smithi022 ceph-mon[29462]: pgmap v725: 148 pgs: 1 peering, 147 active+clean; 582 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 917 B/s rd, 0 op/s 2024-09-11T07:42:04.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:04 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:42:04.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:04 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:42:04.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:04 smithi022 ceph-mon[37330]: from='client.25673 ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.6660572741904994}]': finished 2024-09-11T07:42:04.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:04 smithi022 ceph-mon[37330]: osdmap e346: 8 total, 8 up, 8 in 2024-09-11T07:42:04.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:04 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3872154691' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:42:04.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:04 smithi022 ceph-mon[37330]: pgmap v725: 148 pgs: 1 peering, 147 active+clean; 582 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 917 B/s rd, 0 op/s 2024-09-11T07:42:04.921 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:04.943 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:04.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:04.944 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:05.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:04 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:42:05.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:04 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:42:05.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:04 smithi104 ceph-mon[33507]: from='client.25673 ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.6660572741904994}]': finished 2024-09-11T07:42:05.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:04 smithi104 ceph-mon[33507]: osdmap e346: 8 total, 8 up, 8 in 2024-09-11T07:42:05.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:04 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3872154691' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:42:05.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:04 smithi104 ceph-mon[33507]: pgmap v725: 148 pgs: 1 peering, 147 active+clean; 582 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 917 B/s rd, 0 op/s 2024-09-11T07:42:05.045 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:05.070 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:05.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:05.071 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:05.172 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:05.198 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:05.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:05.199 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:05.300 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:05.324 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:05.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:05.325 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:05.425 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:05.449 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:05.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:05.450 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:05.551 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:05.573 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:05.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:05.574 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:05.675 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:05.699 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:05.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:05.700 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:05.801 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:05.823 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:05.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:05.824 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:05.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:05 smithi022 ceph-mon[37330]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-11T07:42:05.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:05 smithi022 ceph-mon[37330]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:42:05.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:05 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3872154691' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:42:05.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:05 smithi022 ceph-mon[37330]: osdmap e347: 8 total, 8 up, 8 in 2024-09-11T07:42:05.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:05 smithi022 ceph-mon[37330]: osdmap e348: 8 total, 8 up, 8 in 2024-09-11T07:42:05.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:05 smithi022 ceph-mon[29462]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-11T07:42:05.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:05 smithi022 ceph-mon[29462]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:42:05.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:05 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3872154691' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:42:05.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:05 smithi022 ceph-mon[29462]: osdmap e347: 8 total, 8 up, 8 in 2024-09-11T07:42:05.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:05 smithi022 ceph-mon[29462]: osdmap e348: 8 total, 8 up, 8 in 2024-09-11T07:42:05.925 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:05.949 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:05.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:05.950 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:06.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:05 smithi104 ceph-mon[33507]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-11T07:42:06.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:05 smithi104 ceph-mon[33507]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:42:06.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:05 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3872154691' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:42:06.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:05 smithi104 ceph-mon[33507]: osdmap e347: 8 total, 8 up, 8 in 2024-09-11T07:42:06.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:05 smithi104 ceph-mon[33507]: osdmap e348: 8 total, 8 up, 8 in 2024-09-11T07:42:06.048 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:42:06.051 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:06.076 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:06.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:06.078 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:06.088 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:42:06.179 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:06.203 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:06.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:06.204 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:06.305 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:06.326 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:06.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:06.327 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:06.428 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:06.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:06.455 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:06.455 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:06.555 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:06.581 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:06.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:06.582 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:06.683 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:06.706 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:06.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:06.707 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:06.808 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:06.834 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:06.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:06.835 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:06 smithi022 ceph-mon[29462]: 5.5 scrub starts 2024-09-11T07:42:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:06 smithi022 ceph-mon[29462]: 5.5 scrub ok 2024-09-11T07:42:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:06 smithi022 ceph-mon[29462]: 5.a scrub starts 2024-09-11T07:42:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:06 smithi022 ceph-mon[29462]: 5.a scrub ok 2024-09-11T07:42:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:06 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:42:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:06 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:42:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:06 smithi022 ceph-mon[29462]: 6.17 scrub starts 2024-09-11T07:42:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:06 smithi022 ceph-mon[29462]: 6.17 scrub ok 2024-09-11T07:42:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:06 smithi022 ceph-mon[29462]: 5.14 scrub starts 2024-09-11T07:42:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:06 smithi022 ceph-mon[29462]: 5.14 scrub ok 2024-09-11T07:42:06.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:06 smithi022 ceph-mon[29462]: pgmap v728: 148 pgs: 1 peering, 147 active+clean; 582 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 895 B/s rd, 0 op/s 2024-09-11T07:42:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:06 smithi022 ceph-mon[37330]: 5.5 scrub starts 2024-09-11T07:42:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:06 smithi022 ceph-mon[37330]: 5.5 scrub ok 2024-09-11T07:42:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:06 smithi022 ceph-mon[37330]: 5.a scrub starts 2024-09-11T07:42:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:06 smithi022 ceph-mon[37330]: 5.a scrub ok 2024-09-11T07:42:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:06 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:42:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:06 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:42:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:06 smithi022 ceph-mon[37330]: 6.17 scrub starts 2024-09-11T07:42:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:06 smithi022 ceph-mon[37330]: 6.17 scrub ok 2024-09-11T07:42:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:06 smithi022 ceph-mon[37330]: 5.14 scrub starts 2024-09-11T07:42:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:06 smithi022 ceph-mon[37330]: 5.14 scrub ok 2024-09-11T07:42:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:06 smithi022 ceph-mon[37330]: pgmap v728: 148 pgs: 1 peering, 147 active+clean; 582 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 895 B/s rd, 0 op/s 2024-09-11T07:42:06.936 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:06.962 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:06.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:06.963 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:07.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:06 smithi104 ceph-mon[33507]: 5.5 scrub starts 2024-09-11T07:42:07.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:06 smithi104 ceph-mon[33507]: 5.5 scrub ok 2024-09-11T07:42:07.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:06 smithi104 ceph-mon[33507]: 5.a scrub starts 2024-09-11T07:42:07.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:06 smithi104 ceph-mon[33507]: 5.a scrub ok 2024-09-11T07:42:07.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:06 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:42:07.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:06 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:42:07.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:06 smithi104 ceph-mon[33507]: 6.17 scrub starts 2024-09-11T07:42:07.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:06 smithi104 ceph-mon[33507]: 6.17 scrub ok 2024-09-11T07:42:07.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:06 smithi104 ceph-mon[33507]: 5.14 scrub starts 2024-09-11T07:42:07.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:06 smithi104 ceph-mon[33507]: 5.14 scrub ok 2024-09-11T07:42:07.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:06 smithi104 ceph-mon[33507]: pgmap v728: 148 pgs: 1 peering, 147 active+clean; 582 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 895 B/s rd, 0 op/s 2024-09-11T07:42:07.064 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:07.088 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:07.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:07.088 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:07.189 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:07.215 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:07.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:07.216 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:07.317 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:07.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:07.345 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:07.345 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:07.446 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:07.472 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:07.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:07.473 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:07.574 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:07.599 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:07.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:07.600 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:07.700 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:07.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:07.725 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:07.726 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:07.826 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:07.851 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:07.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:07.852 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:07.878 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:42:07.953 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:07.979 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:07.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:07.980 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:08.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:07 smithi104 ceph-mon[33507]: 6.1f scrub starts 2024-09-11T07:42:08.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:07 smithi104 ceph-mon[33507]: 6.1f scrub ok 2024-09-11T07:42:08.059 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:07 smithi022 ceph-mon[29462]: 6.1f scrub starts 2024-09-11T07:42:08.059 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:07 smithi022 ceph-mon[29462]: 6.1f scrub ok 2024-09-11T07:42:08.059 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:07 smithi022 ceph-mon[37330]: 6.1f scrub starts 2024-09-11T07:42:08.059 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:07 smithi022 ceph-mon[37330]: 6.1f scrub ok 2024-09-11T07:42:08.073 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T07:42:08.080 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:08.106 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:08.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:08.106 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:08.207 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:08.231 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:08.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:08.232 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:08.332 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:08.355 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:08.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:08.356 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:08.455 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:42:08.456 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:42:08.457 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:08.470 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:42:08.470 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:42:08.482 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:08.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:08.483 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:08.485 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:42:08.485 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:42:08.500 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:42:08.500 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:42:08.517 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:42:08.517 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:42:08.533 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:42:08.533 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:42:08.548 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:42:08.549 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:42:08.565 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:42:08.566 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:42:08.583 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:08.606 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:08.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:08.607 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:08.707 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:08.732 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:08.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:08.733 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:08.834 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:08.858 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:08.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:08.859 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:08.960 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:08.984 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:08.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:08.985 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:09.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:08 smithi104 ceph-mon[33507]: 6.b scrub starts 2024-09-11T07:42:09.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:08 smithi104 ceph-mon[33507]: 6.b scrub ok 2024-09-11T07:42:09.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:42:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:42:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:08 smithi104 ceph-mon[33507]: pgmap v729: 148 pgs: 1 peering, 147 active+clean; 582 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-11T07:42:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:08 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/786383124' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:42:09.086 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:09.110 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:09.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:09.111 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:09.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:08 smithi022 ceph-mon[29462]: 6.b scrub starts 2024-09-11T07:42:09.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:08 smithi022 ceph-mon[29462]: 6.b scrub ok 2024-09-11T07:42:09.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:42:09.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:42:09.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:08 smithi022 ceph-mon[29462]: pgmap v729: 148 pgs: 1 peering, 147 active+clean; 582 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-11T07:42:09.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:08 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/786383124' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:42:09.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:08 smithi022 ceph-mon[37330]: 6.b scrub starts 2024-09-11T07:42:09.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:08 smithi022 ceph-mon[37330]: 6.b scrub ok 2024-09-11T07:42:09.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:42:09.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:42:09.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:08 smithi022 ceph-mon[37330]: pgmap v729: 148 pgs: 1 peering, 147 active+clean; 582 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-11T07:42:09.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:08 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/786383124' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:42:09.212 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:09.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:09.235 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:09.236 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:09.336 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:09.359 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:09.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:09.360 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:09.461 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:09.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:09.487 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:09.488 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:09.588 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:09.611 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:09.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:09.612 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:09.713 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:09.739 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:09.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:09.740 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:09.837 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:09 smithi022 ceph-mon[29462]: 6.16 scrub starts 2024-09-11T07:42:09.837 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:09 smithi022 ceph-mon[29462]: 6.16 scrub ok 2024-09-11T07:42:09.837 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:09 smithi022 ceph-mon[29462]: 5.8 scrub starts 2024-09-11T07:42:09.837 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:09 smithi022 ceph-mon[29462]: 5.8 scrub ok 2024-09-11T07:42:09.837 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:09 smithi022 ceph-mon[29462]: 5.7 scrub starts 2024-09-11T07:42:09.837 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:09 smithi022 ceph-mon[29462]: 5.7 scrub ok 2024-09-11T07:42:09.837 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:09 smithi022 ceph-mon[29462]: 6.6 scrub starts 2024-09-11T07:42:09.837 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:09 smithi022 ceph-mon[29462]: 6.6 scrub ok 2024-09-11T07:42:09.837 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:09 smithi022 ceph-mon[29462]: 6.3 scrub starts 2024-09-11T07:42:09.838 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:09 smithi022 ceph-mon[29462]: 6.3 scrub ok 2024-09-11T07:42:09.838 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:09 smithi022 ceph-mon[37330]: 6.16 scrub starts 2024-09-11T07:42:09.838 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:09 smithi022 ceph-mon[37330]: 6.16 scrub ok 2024-09-11T07:42:09.838 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:09 smithi022 ceph-mon[37330]: 5.8 scrub starts 2024-09-11T07:42:09.838 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:09 smithi022 ceph-mon[37330]: 5.8 scrub ok 2024-09-11T07:42:09.838 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:09 smithi022 ceph-mon[37330]: 5.7 scrub starts 2024-09-11T07:42:09.838 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:09 smithi022 ceph-mon[37330]: 5.7 scrub ok 2024-09-11T07:42:09.838 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:09 smithi022 ceph-mon[37330]: 6.6 scrub starts 2024-09-11T07:42:09.838 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:09 smithi022 ceph-mon[37330]: 6.6 scrub ok 2024-09-11T07:42:09.838 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:09 smithi022 ceph-mon[37330]: 6.3 scrub starts 2024-09-11T07:42:09.839 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:09 smithi022 ceph-mon[37330]: 6.3 scrub ok 2024-09-11T07:42:09.841 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:09.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:09.864 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:09.865 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:09.965 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:09.988 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:09.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:09.988 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:10.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:42:09 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:42:10.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:09 smithi104 ceph-mon[33507]: 6.16 scrub starts 2024-09-11T07:42:10.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:09 smithi104 ceph-mon[33507]: 6.16 scrub ok 2024-09-11T07:42:10.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:09 smithi104 ceph-mon[33507]: 5.8 scrub starts 2024-09-11T07:42:10.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:09 smithi104 ceph-mon[33507]: 5.8 scrub ok 2024-09-11T07:42:10.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:09 smithi104 ceph-mon[33507]: 5.7 scrub starts 2024-09-11T07:42:10.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:09 smithi104 ceph-mon[33507]: 5.7 scrub ok 2024-09-11T07:42:10.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:09 smithi104 ceph-mon[33507]: 6.6 scrub starts 2024-09-11T07:42:10.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:09 smithi104 ceph-mon[33507]: 6.6 scrub ok 2024-09-11T07:42:10.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:09 smithi104 ceph-mon[33507]: 6.3 scrub starts 2024-09-11T07:42:10.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:09 smithi104 ceph-mon[33507]: 6.3 scrub ok 2024-09-11T07:42:10.090 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:10.112 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:10.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:10.113 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:10.214 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:10.236 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:10.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:10.237 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:10.338 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:10.360 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:10.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:10.361 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:10.462 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:10.472 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:42:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:42:10] "GET /metrics HTTP/1.1" 200 224656 "" "Prometheus/2.43.0" 2024-09-11T07:42:10.485 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:10.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:10.486 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:10.561 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:42:10.561 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:42:10.561 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-11T07:42:10.561 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T07:42:10.561 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T07:42:10.562 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.3 config set filestore_inject_stall 3 2024-09-11T07:42:10.587 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:10.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:10.614 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:10.614 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:10.715 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:10.739 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:10.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:10.740 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:10.777 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:42:10.841 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:10.862 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:10.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:10.863 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:10.964 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:10.986 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:10.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:10.987 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:11.005 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:10 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:42:11.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:10 smithi022 ceph-mon[29462]: 2.5 scrub starts 2024-09-11T07:42:11.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:10 smithi022 ceph-mon[29462]: 2.5 scrub ok 2024-09-11T07:42:11.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:10 smithi022 ceph-mon[29462]: pgmap v730: 148 pgs: 148 active+clean; 582 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:42:11.006 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:10 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:42:11.006 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:10 smithi022 ceph-mon[37330]: 2.5 scrub starts 2024-09-11T07:42:11.006 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:10 smithi022 ceph-mon[37330]: 2.5 scrub ok 2024-09-11T07:42:11.006 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:10 smithi022 ceph-mon[37330]: pgmap v730: 148 pgs: 148 active+clean; 582 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:42:11.007 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:42:11.007 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 20, 2024-09-11T07:42:11.007 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 600, 2024-09-11T07:42:11.007 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:42:11.007 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.007 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.15504.0:175714 9.0 e285/259 9:006c31a1:::benchmark_data_smithi022_133943_object49123:head v 285'8211, mlcod=285'8208)", 2024-09-11T07:42:11.007 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:39:49.913145+0000", 2024-09-11T07:42:11.007 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 141.090520578, 2024-09-11T07:42:11.007 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.14059284699999999, 2024-09-11T07:42:11.007 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:11.008 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:11.008 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:11.008 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.008 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:11.008 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:49.913145+0000", 2024-09-11T07:42:11.008 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.008 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.008 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.008 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:11.008 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:49.913145+0000", 2024-09-11T07:42:11.008 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.008 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.008 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.009 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:11.009 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:49.913143+0000", 2024-09-11T07:42:11.009 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:42:11.009 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.009 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.009 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:11.009 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:49.913152+0000", 2024-09-11T07:42:11.009 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.0040000000000005e-06 2024-09-11T07:42:11.009 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.009 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.009 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:11.009 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:49.913163+0000", 2024-09-11T07:42:11.009 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.095e-05 2024-09-11T07:42:11.010 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.010 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.010 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:11.010 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:49.913169+0000", 2024-09-11T07:42:11.010 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6489999999999996e-06 2024-09-11T07:42:11.010 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.010 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.010 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.010 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:49.914942+0000", 2024-09-11T07:42:11.010 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001773447 2024-09-11T07:42:11.010 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.010 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.010 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:11.011 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:49.914960+0000", 2024-09-11T07:42:11.011 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.7915999999999999e-05 2024-09-11T07:42:11.011 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.011 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.011 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:11.011 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:49.917478+0000", 2024-09-11T07:42:11.011 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002517933 2024-09-11T07:42:11.011 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.011 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.011 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:11.011 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:50.053738+0000", 2024-09-11T07:42:11.011 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.136260039 2024-09-11T07:42:11.011 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.012 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:11.012 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.012 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.012 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.012 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.15504.0:175719 9.2 e285/259 9:4755b0aa:::benchmark_data_smithi022_133943_object49128:head v 285'8079, mlcod=285'8077)", 2024-09-11T07:42:11.012 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:39:49.917815+0000", 2024-09-11T07:42:11.012 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 141.08585106500001, 2024-09-11T07:42:11.012 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.14771964900000001, 2024-09-11T07:42:11.012 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:11.012 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:11.012 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:11.012 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.013 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:11.013 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:49.917815+0000", 2024-09-11T07:42:11.013 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.013 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.013 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.013 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:11.013 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:49.917815+0000", 2024-09-11T07:42:11.013 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.013 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.013 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.013 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:11.013 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:49.917813+0000", 2024-09-11T07:42:11.014 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:42:11.014 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.014 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.014 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:11.014 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:49.917823+0000", 2024-09-11T07:42:11.014 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0242e-05 2024-09-11T07:42:11.014 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.014 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.014 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:11.014 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:50.053631+0000", 2024-09-11T07:42:11.014 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.13580754 2024-09-11T07:42:11.014 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.014 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.014 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:11.014 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:50.053636+0000", 2024-09-11T07:42:11.015 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.4410000000000001e-06 2024-09-11T07:42:11.015 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.015 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.015 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.015 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:50.055995+0000", 2024-09-11T07:42:11.015 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002358719 2024-09-11T07:42:11.015 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.015 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.015 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:11.015 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:50.056021+0000", 2024-09-11T07:42:11.015 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5337000000000002e-05 2024-09-11T07:42:11.015 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.015 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.015 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:11.015 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:50.065416+0000", 2024-09-11T07:42:11.016 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0093957929999999995 2024-09-11T07:42:11.016 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.016 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.016 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:11.016 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:39:50.065535+0000", 2024-09-11T07:42:11.016 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011822 2024-09-11T07:42:11.016 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.016 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:11.016 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.016 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.016 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.016 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:233733 9.0 9:0702ffbd:::benchmark_data_smithi022_133943_object107142:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:42:11.016 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:58.855983+0000", 2024-09-11T07:42:11.017 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 72.147683041999997, 2024-09-11T07:42:11.017 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.056325709000000002, 2024-09-11T07:42:11.017 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:11.017 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:11.017 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:11.017 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:11.017 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:11.017 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 233733 2024-09-11T07:42:11.017 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.017 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:11.017 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.017 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:11.017 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:58.855983+0000", 2024-09-11T07:42:11.017 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.018 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.018 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.018 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:11.018 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:58.855983+0000", 2024-09-11T07:42:11.018 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.018 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.018 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.018 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:11.018 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:58.855981+0000", 2024-09-11T07:42:11.018 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:42:11.018 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.018 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.018 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:11.018 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:58.855989+0000", 2024-09-11T07:42:11.019 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.7680000000000008e-06 2024-09-11T07:42:11.019 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.019 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.019 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:11.019 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:58.855999+0000", 2024-09-11T07:42:11.019 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0413e-05 2024-09-11T07:42:11.019 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.019 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.019 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:11.019 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:58.856005+0000", 2024-09-11T07:42:11.019 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9490000000000001e-06 2024-09-11T07:42:11.019 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.019 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.019 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.020 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:58.856058+0000", 2024-09-11T07:42:11.020 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3380000000000001e-05 2024-09-11T07:42:11.020 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.020 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.020 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:11.020 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:58.856186+0000", 2024-09-11T07:42:11.020 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012752600000000001 2024-09-11T07:42:11.020 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.020 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.020 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,1,6", 2024-09-11T07:42:11.020 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:58.856700+0000", 2024-09-11T07:42:11.020 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00051407899999999997 2024-09-11T07:42:11.020 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.020 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.021 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.021 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:58.864247+0000", 2024-09-11T07:42:11.021 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0075470229999999999 2024-09-11T07:42:11.021 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.021 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.021 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.021 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:58.864735+0000", 2024-09-11T07:42:11.022 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00048788899999999998 2024-09-11T07:42:11.022 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.022 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.022 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:11.022 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:58.864795+0000", 2024-09-11T07:42:11.022 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0341999999999999e-05 2024-09-11T07:42:11.022 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.022 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.022 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.022 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:58.911714+0000", 2024-09-11T07:42:11.022 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.046918754 2024-09-11T07:42:11.023 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.023 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.023 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:11.023 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:58.911920+0000", 2024-09-11T07:42:11.023 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00020586500000000001 2024-09-11T07:42:11.023 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.023 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.023 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:11.023 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:58.912309+0000", 2024-09-11T07:42:11.023 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000388847 2024-09-11T07:42:11.023 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.023 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:11.023 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.023 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.024 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.024 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:234252 9.0 9:0367baf5:::benchmark_data_smithi022_133943_object107661:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:42:11.024 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:59.416155+0000", 2024-09-11T07:42:11.024 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 71.587510718999994, 2024-09-11T07:42:11.024 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.096494863, 2024-09-11T07:42:11.024 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:11.024 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:11.024 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:11.024 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:11.024 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:11.024 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 234252 2024-09-11T07:42:11.024 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.025 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:11.025 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.025 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:11.025 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.416155+0000", 2024-09-11T07:42:11.025 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.025 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.025 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.025 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:11.025 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.416155+0000", 2024-09-11T07:42:11.025 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.025 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.025 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.025 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:11.026 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.416152+0000", 2024-09-11T07:42:11.026 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:42:11.026 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.026 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.026 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:11.026 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.416162+0000", 2024-09-11T07:42:11.026 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.4180000000000003e-06 2024-09-11T07:42:11.026 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.026 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.026 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:11.026 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.416174+0000", 2024-09-11T07:42:11.026 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2218000000000001e-05 2024-09-11T07:42:11.026 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.026 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.027 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:11.027 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.416181+0000", 2024-09-11T07:42:11.027 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4389999999999997e-06 2024-09-11T07:42:11.027 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.027 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.027 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.027 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.417541+0000", 2024-09-11T07:42:11.027 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0013609410000000001 2024-09-11T07:42:11.027 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.027 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.027 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:11.027 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.417666+0000", 2024-09-11T07:42:11.027 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000124869 2024-09-11T07:42:11.028 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.028 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.028 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,1,6", 2024-09-11T07:42:11.028 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.418106+0000", 2024-09-11T07:42:11.028 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00043991899999999999 2024-09-11T07:42:11.028 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.028 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.028 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:11.028 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.420865+0000", 2024-09-11T07:42:11.028 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002758665 2024-09-11T07:42:11.028 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.028 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.029 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.029 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.421194+0000", 2024-09-11T07:42:11.029 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00032864299999999999 2024-09-11T07:42:11.029 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.029 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.029 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.029 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.421269+0000", 2024-09-11T07:42:11.029 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5922999999999998e-05 2024-09-11T07:42:11.029 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.029 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.029 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.029 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.512329+0000", 2024-09-11T07:42:11.029 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.091059980999999998 2024-09-11T07:42:11.030 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.030 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.030 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:11.030 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.512386+0000", 2024-09-11T07:42:11.030 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6294e-05 2024-09-11T07:42:11.030 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.030 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.030 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:11.030 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.512650+0000", 2024-09-11T07:42:11.030 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00026433700000000001 2024-09-11T07:42:11.030 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.030 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:11.030 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.031 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.031 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.031 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:234260 9.0 9:052a6504:::benchmark_data_smithi022_133943_object107669:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:42:11.031 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:59.422494+0000", 2024-09-11T07:42:11.031 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 71.581171627000003, 2024-09-11T07:42:11.031 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.091722445999999999, 2024-09-11T07:42:11.031 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:11.031 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:11.031 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:11.031 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:11.031 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:11.031 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 234260 2024-09-11T07:42:11.031 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.032 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:11.032 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.032 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:11.032 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.422494+0000", 2024-09-11T07:42:11.032 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.032 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.032 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.032 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:11.032 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.422494+0000", 2024-09-11T07:42:11.032 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.032 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.032 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.032 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:11.033 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.422492+0000", 2024-09-11T07:42:11.033 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:42:11.033 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.033 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.033 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:11.033 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.422500+0000", 2024-09-11T07:42:11.033 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.0209999999999999e-06 2024-09-11T07:42:11.033 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.033 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.033 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:11.033 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.422516+0000", 2024-09-11T07:42:11.033 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5996999999999999e-05 2024-09-11T07:42:11.033 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.034 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.034 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:11.034 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.422522+0000", 2024-09-11T07:42:11.034 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.5679999999999999e-06 2024-09-11T07:42:11.034 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.034 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.034 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.034 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.422601+0000", 2024-09-11T07:42:11.034 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.9034000000000003e-05 2024-09-11T07:42:11.034 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.034 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.034 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:11.034 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.422741+0000", 2024-09-11T07:42:11.035 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014067100000000001 2024-09-11T07:42:11.035 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.035 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.035 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,1,6", 2024-09-11T07:42:11.035 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.423207+0000", 2024-09-11T07:42:11.035 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00046516199999999998 2024-09-11T07:42:11.035 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.035 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.035 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:11.035 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.425354+0000", 2024-09-11T07:42:11.035 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0021478740000000001 2024-09-11T07:42:11.035 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.035 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.035 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.036 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.425820+0000", 2024-09-11T07:42:11.036 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00046582600000000002 2024-09-11T07:42:11.036 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.036 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.036 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.036 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.512071+0000", 2024-09-11T07:42:11.036 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.086250773000000003 2024-09-11T07:42:11.036 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.036 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.036 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.036 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.514018+0000", 2024-09-11T07:42:11.036 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00194735 2024-09-11T07:42:11.036 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.037 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.037 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:11.037 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.514109+0000", 2024-09-11T07:42:11.037 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.1071999999999998e-05 2024-09-11T07:42:11.037 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.037 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.037 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:11.037 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.514217+0000", 2024-09-11T07:42:11.037 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000107379 2024-09-11T07:42:11.037 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.037 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:11.037 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.038 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.038 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.038 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:234355 9.0 9:07598df1:::benchmark_data_smithi022_133943_object107764:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:42:11.038 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:59.586942+0000", 2024-09-11T07:42:11.038 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 71.416723861999998, 2024-09-11T07:42:11.038 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.085143394999999997, 2024-09-11T07:42:11.038 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:11.038 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:11.038 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:11.038 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:11.038 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:11.038 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 234355 2024-09-11T07:42:11.039 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.039 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:11.039 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.039 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:11.039 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.586942+0000", 2024-09-11T07:42:11.039 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.039 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.039 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.039 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:11.039 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.586942+0000", 2024-09-11T07:42:11.039 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.039 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.039 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.039 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:11.040 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.586940+0000", 2024-09-11T07:42:11.040 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:42:11.040 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.040 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.040 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:11.040 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.586948+0000", 2024-09-11T07:42:11.040 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.0870000000000007e-06 2024-09-11T07:42:11.040 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.040 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.040 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:11.040 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.587157+0000", 2024-09-11T07:42:11.040 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00020979000000000001 2024-09-11T07:42:11.040 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.041 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.041 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:11.041 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.587167+0000", 2024-09-11T07:42:11.041 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.1940000000000004e-06 2024-09-11T07:42:11.041 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.041 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.041 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.041 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.587209+0000", 2024-09-11T07:42:11.041 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2215000000000002e-05 2024-09-11T07:42:11.041 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.041 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.041 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:11.041 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.587324+0000", 2024-09-11T07:42:11.041 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000115497 2024-09-11T07:42:11.042 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.042 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.042 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,1,6", 2024-09-11T07:42:11.042 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.587808+0000", 2024-09-11T07:42:11.042 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000483573 2024-09-11T07:42:11.042 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.042 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.042 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.042 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.592820+0000", 2024-09-11T07:42:11.042 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0050117740000000001 2024-09-11T07:42:11.042 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.042 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.043 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:11.043 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.593156+0000", 2024-09-11T07:42:11.043 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00033644499999999999 2024-09-11T07:42:11.043 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.043 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.043 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.043 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.623125+0000", 2024-09-11T07:42:11.043 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.029968752000000001 2024-09-11T07:42:11.043 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.043 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.043 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.043 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.671438+0000", 2024-09-11T07:42:11.043 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.048313215999999999 2024-09-11T07:42:11.044 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.044 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.044 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:11.044 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.671676+0000", 2024-09-11T07:42:11.044 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000237722 2024-09-11T07:42:11.044 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.044 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.044 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:11.044 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.672085+0000", 2024-09-11T07:42:11.044 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000409718 2024-09-11T07:42:11.044 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.044 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:11.044 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.044 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.045 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.045 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:234371 9.0 9:072a149d:::benchmark_data_smithi022_133943_object107780:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:42:11.045 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:59.600883+0000", 2024-09-11T07:42:11.045 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 71.402782594000001, 2024-09-11T07:42:11.045 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.071491551, 2024-09-11T07:42:11.045 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:11.045 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:11.045 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:11.045 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:11.045 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:11.045 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 234371 2024-09-11T07:42:11.045 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.045 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:11.045 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.046 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:11.046 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.600883+0000", 2024-09-11T07:42:11.046 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.046 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.046 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.046 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:11.046 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.600883+0000", 2024-09-11T07:42:11.046 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.046 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.046 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.046 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:11.046 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.600882+0000", 2024-09-11T07:42:11.046 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:42:11.047 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.047 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.047 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:11.047 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.600888+0000", 2024-09-11T07:42:11.047 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.6279999999999999e-06 2024-09-11T07:42:11.047 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.047 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.047 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:11.047 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.600897+0000", 2024-09-11T07:42:11.047 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.6819999999999992e-06 2024-09-11T07:42:11.047 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.047 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.047 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:11.048 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.600901+0000", 2024-09-11T07:42:11.048 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0799999999999999e-06 2024-09-11T07:42:11.048 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.048 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.048 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.048 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.600951+0000", 2024-09-11T07:42:11.048 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9632000000000002e-05 2024-09-11T07:42:11.048 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.048 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.048 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:11.048 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.601093+0000", 2024-09-11T07:42:11.048 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014222799999999999 2024-09-11T07:42:11.048 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.049 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.049 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,1,6", 2024-09-11T07:42:11.049 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.601469+0000", 2024-09-11T07:42:11.049 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00037605400000000001 2024-09-11T07:42:11.049 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.049 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.049 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:11.049 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.603295+0000", 2024-09-11T07:42:11.049 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0018256419999999999 2024-09-11T07:42:11.049 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.049 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.049 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.050 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.603924+0000", 2024-09-11T07:42:11.050 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00062985500000000002 2024-09-11T07:42:11.050 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.050 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.050 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.050 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.626524+0000", 2024-09-11T07:42:11.050 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.022599175999999999 2024-09-11T07:42:11.050 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.050 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.050 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.050 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.672152+0000", 2024-09-11T07:42:11.050 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.045628613999999998 2024-09-11T07:42:11.050 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.050 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.050 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:11.051 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.672252+0000", 2024-09-11T07:42:11.051 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.9780999999999999e-05 2024-09-11T07:42:11.051 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.051 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.051 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:11.051 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.672375+0000", 2024-09-11T07:42:11.051 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012297999999999999 2024-09-11T07:42:11.051 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.051 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:11.051 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.051 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.051 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.052 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235127 9.0 9:01e55b17:::benchmark_data_smithi022_133943_object108536:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e321)", 2024-09-11T07:42:11.052 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.667941+0000", 2024-09-11T07:42:11.052 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 69.335724502999994, 2024-09-11T07:42:11.052 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0432926819999999, 2024-09-11T07:42:11.052 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:11.052 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:42:11.052 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:11.052 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:11.052 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:11.052 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235127 2024-09-11T07:42:11.052 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.052 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:11.052 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.052 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:11.053 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.667941+0000", 2024-09-11T07:42:11.053 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.053 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.053 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.053 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:11.053 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.667941+0000", 2024-09-11T07:42:11.053 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.053 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.053 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.053 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:11.053 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.667939+0000", 2024-09-11T07:42:11.053 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:42:11.053 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.054 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.054 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:11.054 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.667947+0000", 2024-09-11T07:42:11.054 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3690000000000007e-06 2024-09-11T07:42:11.054 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.054 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.054 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:11.054 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.667963+0000", 2024-09-11T07:42:11.054 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.6305999999999999e-05 2024-09-11T07:42:11.054 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.054 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.054 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:11.055 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.667969+0000", 2024-09-11T07:42:11.055 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8619999999999999e-06 2024-09-11T07:42:11.055 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.055 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.055 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.055 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.668109+0000", 2024-09-11T07:42:11.055 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014013900000000001 2024-09-11T07:42:11.055 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.055 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.055 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:11.055 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.668418+0000", 2024-09-11T07:42:11.055 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000308657 2024-09-11T07:42:11.055 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.055 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.056 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,1,6", 2024-09-11T07:42:11.056 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.669377+0000", 2024-09-11T07:42:11.056 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00095886099999999998 2024-09-11T07:42:11.056 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.056 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.056 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:11.056 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.679949+0000", 2024-09-11T07:42:11.056 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.010572125 2024-09-11T07:42:11.056 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.056 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.056 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.056 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.682612+0000", 2024-09-11T07:42:11.056 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0026625070000000002 2024-09-11T07:42:11.057 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.057 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.057 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:11.057 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.682635+0000", 2024-09-11T07:42:11.057 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3396000000000001e-05 2024-09-11T07:42:11.057 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.057 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.057 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.057 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.711146+0000", 2024-09-11T07:42:11.057 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0285110040000001 2024-09-11T07:42:11.057 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.057 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.058 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:11.058 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.711234+0000", 2024-09-11T07:42:11.058 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.8091999999999996e-05 2024-09-11T07:42:11.058 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.058 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:11.058 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.058 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.058 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.058 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235175 9.0 9:077fcbba:::benchmark_data_smithi022_133943_object108584:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-11T07:42:11.058 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.720021+0000", 2024-09-11T07:42:11.058 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 69.283645321999998, 2024-09-11T07:42:11.058 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.99970432600000003, 2024-09-11T07:42:11.058 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:11.058 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:11.059 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:11.059 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:11.059 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:11.059 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235175 2024-09-11T07:42:11.059 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.059 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:11.059 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.059 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:11.059 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.720021+0000", 2024-09-11T07:42:11.059 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.059 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.059 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.059 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:11.060 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.720021+0000", 2024-09-11T07:42:11.060 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.060 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.060 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.060 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:11.060 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.720018+0000", 2024-09-11T07:42:11.060 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:42:11.060 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.060 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.060 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:11.060 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.720027+0000", 2024-09-11T07:42:11.060 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.4780000000000008e-06 2024-09-11T07:42:11.060 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.060 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.061 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:11.061 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.720044+0000", 2024-09-11T07:42:11.061 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.7180000000000002e-05 2024-09-11T07:42:11.061 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.061 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.061 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:11.061 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.720051+0000", 2024-09-11T07:42:11.061 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.503e-06 2024-09-11T07:42:11.061 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.061 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.061 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.061 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.720097+0000", 2024-09-11T07:42:11.061 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5689000000000003e-05 2024-09-11T07:42:11.061 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.062 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.062 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:11.062 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.720105+0000", 2024-09-11T07:42:11.062 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.9000000000000006e-06 2024-09-11T07:42:11.062 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.062 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.062 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.062 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.711744+0000", 2024-09-11T07:42:11.062 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.99163922400000004 2024-09-11T07:42:11.062 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.062 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.062 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:11.062 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.711839+0000", 2024-09-11T07:42:11.063 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.5249999999999998e-05 2024-09-11T07:42:11.063 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.063 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.063 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:42:11.063 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.712333+0000", 2024-09-11T07:42:11.063 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00049357399999999999 2024-09-11T07:42:11.063 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.063 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.063 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.063 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.719518+0000", 2024-09-11T07:42:11.063 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0071850860000000003 2024-09-11T07:42:11.063 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.063 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.063 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:11.063 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.719600+0000", 2024-09-11T07:42:11.064 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.2018e-05 2024-09-11T07:42:11.064 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.064 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.064 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.064 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.719637+0000", 2024-09-11T07:42:11.064 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6650999999999999e-05 2024-09-11T07:42:11.064 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.064 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.064 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:11.064 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.719684+0000", 2024-09-11T07:42:11.064 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7289000000000001e-05 2024-09-11T07:42:11.064 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.064 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.064 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:11.065 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.719725+0000", 2024-09-11T07:42:11.065 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0952999999999999e-05 2024-09-11T07:42:11.065 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.065 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:11.065 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.065 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.065 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.066 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:10 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:42:11.066 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:10 smithi104 ceph-mon[33507]: 2.5 scrub starts 2024-09-11T07:42:11.066 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:10 smithi104 ceph-mon[33507]: 2.5 scrub ok 2024-09-11T07:42:11.066 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:10 smithi104 ceph-mon[33507]: pgmap v730: 148 pgs: 148 active+clean; 582 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:42:11.067 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235239 9.0 9:01129c16:::benchmark_data_smithi022_133943_object108648:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-11T07:42:11.067 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.783885+0000", 2024-09-11T07:42:11.067 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 69.219780559, 2024-09-11T07:42:11.067 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.94223383800000005, 2024-09-11T07:42:11.067 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:11.067 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:11.067 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:11.067 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:11.067 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:11.068 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235239 2024-09-11T07:42:11.068 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.068 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:11.068 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.068 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:11.068 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.783885+0000", 2024-09-11T07:42:11.068 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.068 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.068 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.068 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:11.068 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.783885+0000", 2024-09-11T07:42:11.068 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.069 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.069 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.069 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:11.069 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.783883+0000", 2024-09-11T07:42:11.069 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:42:11.069 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.069 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.069 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:11.069 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.783892+0000", 2024-09-11T07:42:11.069 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.6150000000000003e-06 2024-09-11T07:42:11.069 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.069 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.069 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:11.069 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.783911+0000", 2024-09-11T07:42:11.070 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8561000000000001e-05 2024-09-11T07:42:11.070 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.070 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.070 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:11.070 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.783920+0000", 2024-09-11T07:42:11.070 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.0110000000000006e-06 2024-09-11T07:42:11.070 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.070 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.070 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.070 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.784064+0000", 2024-09-11T07:42:11.070 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000144346 2024-09-11T07:42:11.070 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.070 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.071 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:11.071 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.784088+0000", 2024-09-11T07:42:11.071 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3830999999999999e-05 2024-09-11T07:42:11.071 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.071 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.071 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.071 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.714582+0000", 2024-09-11T07:42:11.071 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.93049353499999998 2024-09-11T07:42:11.071 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.071 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.071 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:11.071 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.714782+0000", 2024-09-11T07:42:11.071 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00019994000000000001 2024-09-11T07:42:11.071 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.072 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.072 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:42:11.072 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.715188+0000", 2024-09-11T07:42:11.072 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00040689900000000001 2024-09-11T07:42:11.072 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.072 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.072 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:11.072 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.719605+0000", 2024-09-11T07:42:11.072 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0044166520000000001 2024-09-11T07:42:11.072 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.072 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.072 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.072 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.719760+0000", 2024-09-11T07:42:11.073 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015526900000000001 2024-09-11T07:42:11.073 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.073 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.073 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.073 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726015+0000", 2024-09-11T07:42:11.073 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0062551669999999998 2024-09-11T07:42:11.073 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.073 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.073 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:11.073 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726075+0000", 2024-09-11T07:42:11.073 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9857e-05 2024-09-11T07:42:11.074 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.074 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.074 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:11.074 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726119+0000", 2024-09-11T07:42:11.074 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3884000000000002e-05 2024-09-11T07:42:11.074 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.074 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:11.074 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.074 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.074 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.074 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235253 9.0 9:03cfe51f:::benchmark_data_smithi022_133943_object108662:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-11T07:42:11.074 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.795378+0000", 2024-09-11T07:42:11.074 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 69.208288187999997, 2024-09-11T07:42:11.075 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.93095434300000002, 2024-09-11T07:42:11.075 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:11.075 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:11.075 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:11.075 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:11.075 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:11.075 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235253 2024-09-11T07:42:11.075 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.075 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:11.075 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.075 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:11.075 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795378+0000", 2024-09-11T07:42:11.075 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.075 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.076 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.076 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:11.076 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795378+0000", 2024-09-11T07:42:11.076 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.076 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.076 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.076 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:11.076 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795375+0000", 2024-09-11T07:42:11.076 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:42:11.076 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.076 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.076 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:11.077 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795385+0000", 2024-09-11T07:42:11.077 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.9019999999999994e-06 2024-09-11T07:42:11.077 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.077 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.077 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:11.077 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795403+0000", 2024-09-11T07:42:11.077 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8428999999999999e-05 2024-09-11T07:42:11.077 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.077 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.077 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:11.077 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795410+0000", 2024-09-11T07:42:11.077 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.742e-06 2024-09-11T07:42:11.077 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.078 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.078 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.078 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795457+0000", 2024-09-11T07:42:11.078 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7469999999999998e-05 2024-09-11T07:42:11.078 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.078 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.078 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:11.078 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795466+0000", 2024-09-11T07:42:11.078 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.8829999999999995e-06 2024-09-11T07:42:11.078 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.078 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.078 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.078 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.717205+0000", 2024-09-11T07:42:11.079 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.92173829900000004 2024-09-11T07:42:11.079 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.079 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.079 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:11.079 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.717345+0000", 2024-09-11T07:42:11.079 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014056800000000001 2024-09-11T07:42:11.079 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.079 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.079 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:42:11.079 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.717626+0000", 2024-09-11T07:42:11.079 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00028078599999999999 2024-09-11T07:42:11.079 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.079 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.080 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:11.080 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.725938+0000", 2024-09-11T07:42:11.080 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0083124600000000007 2024-09-11T07:42:11.080 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.080 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.080 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.080 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.725977+0000", 2024-09-11T07:42:11.080 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8679e-05 2024-09-11T07:42:11.080 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.080 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.080 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.080 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726219+0000", 2024-09-11T07:42:11.080 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00024190600000000001 2024-09-11T07:42:11.081 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.081 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.081 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:11.081 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726290+0000", 2024-09-11T07:42:11.081 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1551000000000007e-05 2024-09-11T07:42:11.081 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.081 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.081 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:11.081 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726332+0000", 2024-09-11T07:42:11.081 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1641000000000003e-05 2024-09-11T07:42:11.081 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.081 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:11.081 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.082 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.082 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.082 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235255 9.0 9:06894acd:::benchmark_data_smithi022_133943_object108664:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-11T07:42:11.082 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.795871+0000", 2024-09-11T07:42:11.082 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 69.207794809999996, 2024-09-11T07:42:11.082 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.93060655199999998, 2024-09-11T07:42:11.082 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:11.082 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:11.082 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:11.082 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:11.082 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:11.082 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235255 2024-09-11T07:42:11.082 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.083 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:11.083 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.083 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:11.083 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795871+0000", 2024-09-11T07:42:11.083 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.083 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.083 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.083 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:11.083 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795871+0000", 2024-09-11T07:42:11.083 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.083 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.083 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.083 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:11.083 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795870+0000", 2024-09-11T07:42:11.084 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:42:11.084 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.084 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.084 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:11.084 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795876+0000", 2024-09-11T07:42:11.084 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.993e-06 2024-09-11T07:42:11.084 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.084 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.084 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:11.084 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795886+0000", 2024-09-11T07:42:11.084 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0259e-05 2024-09-11T07:42:11.084 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.085 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.085 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:11.085 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795892+0000", 2024-09-11T07:42:11.085 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3110000000000002e-06 2024-09-11T07:42:11.085 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.085 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.085 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.085 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795933+0000", 2024-09-11T07:42:11.085 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0852000000000001e-05 2024-09-11T07:42:11.085 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.085 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.085 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:11.085 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795940+0000", 2024-09-11T07:42:11.085 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0199999999999997e-06 2024-09-11T07:42:11.086 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.086 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.086 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.086 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.719781+0000", 2024-09-11T07:42:11.086 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.92384101299999999 2024-09-11T07:42:11.086 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.086 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.086 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:11.086 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.719886+0000", 2024-09-11T07:42:11.086 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000105339 2024-09-11T07:42:11.086 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.086 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.086 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:42:11.087 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.720315+0000", 2024-09-11T07:42:11.087 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00042835200000000001 2024-09-11T07:42:11.087 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.087 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.087 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:11.087 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.725983+0000", 2024-09-11T07:42:11.087 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0056682360000000001 2024-09-11T07:42:11.087 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.087 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.087 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.087 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726182+0000", 2024-09-11T07:42:11.087 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00019910500000000001 2024-09-11T07:42:11.087 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.088 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.088 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.088 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726408+0000", 2024-09-11T07:42:11.088 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000225644 2024-09-11T07:42:11.088 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.088 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.088 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:11.088 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726446+0000", 2024-09-11T07:42:11.088 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8281000000000002e-05 2024-09-11T07:42:11.088 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.088 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.088 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:11.088 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726478+0000", 2024-09-11T07:42:11.089 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.1622000000000003e-05 2024-09-11T07:42:11.089 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.089 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:11.089 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.089 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.089 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.089 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235283 9.0 9:076f29af:::benchmark_data_smithi022_133943_object108692:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-11T07:42:11.089 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.825762+0000", 2024-09-11T07:42:11.089 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 69.177904007999999, 2024-09-11T07:42:11.089 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.90922861499999996, 2024-09-11T07:42:11.089 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:11.089 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:11.089 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:11.090 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:11.090 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:11.090 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235283 2024-09-11T07:42:11.090 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.090 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:11.090 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.090 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:11.090 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.825762+0000", 2024-09-11T07:42:11.090 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.090 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.090 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.090 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:11.090 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.825762+0000", 2024-09-11T07:42:11.091 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.091 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.091 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.091 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:11.091 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.825759+0000", 2024-09-11T07:42:11.091 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:42:11.091 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.091 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.091 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:11.091 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.825771+0000", 2024-09-11T07:42:11.091 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1622e-05 2024-09-11T07:42:11.091 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.091 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.092 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:11.092 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.825793+0000", 2024-09-11T07:42:11.092 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2328e-05 2024-09-11T07:42:11.092 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.092 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.092 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:11.092 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.825802+0000", 2024-09-11T07:42:11.092 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.3030000000000005e-06 2024-09-11T07:42:11.092 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.092 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.092 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.092 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.825861+0000", 2024-09-11T07:42:11.092 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8440999999999998e-05 2024-09-11T07:42:11.093 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.093 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.093 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:11.093 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.825870+0000", 2024-09-11T07:42:11.093 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.1849999999999993e-06 2024-09-11T07:42:11.093 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.093 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.093 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.093 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.722844+0000", 2024-09-11T07:42:11.093 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.89697426999999996 2024-09-11T07:42:11.093 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.093 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.093 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:11.094 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.722956+0000", 2024-09-11T07:42:11.094 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011223 2024-09-11T07:42:11.094 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.094 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.094 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:42:11.094 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.723413+0000", 2024-09-11T07:42:11.094 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00045686400000000002 2024-09-11T07:42:11.094 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.094 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.094 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.094 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726373+0000", 2024-09-11T07:42:11.094 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0029602420000000001 2024-09-11T07:42:11.094 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.095 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.095 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.095 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.734805+0000", 2024-09-11T07:42:11.095 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0084313219999999998 2024-09-11T07:42:11.095 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.095 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.095 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:11.095 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.734902+0000", 2024-09-11T07:42:11.095 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.7293999999999994e-05 2024-09-11T07:42:11.095 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.095 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.096 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:11.096 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.734949+0000", 2024-09-11T07:42:11.096 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7023e-05 2024-09-11T07:42:11.096 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.096 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.096 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:11.096 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.734991+0000", 2024-09-11T07:42:11.096 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1492000000000002e-05 2024-09-11T07:42:11.096 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.096 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:11.096 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.096 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.096 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.097 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235330 9.0 9:030ff2ac:::benchmark_data_smithi022_133943_object108739:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-11T07:42:11.097 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.882265+0000", 2024-09-11T07:42:11.097 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 69.121400636999994, 2024-09-11T07:42:11.097 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.85279510800000002, 2024-09-11T07:42:11.097 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:11.097 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:11.097 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:11.097 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:11.097 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:11.097 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235330 2024-09-11T07:42:11.097 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.097 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:11.098 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.098 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:11.098 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.882265+0000", 2024-09-11T07:42:11.098 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.098 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.098 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.098 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:11.098 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.882265+0000", 2024-09-11T07:42:11.098 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.098 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.098 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.098 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:11.098 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.882263+0000", 2024-09-11T07:42:11.099 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:42:11.099 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.099 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.099 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:11.099 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.882272+0000", 2024-09-11T07:42:11.099 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.9600000000000006e-06 2024-09-11T07:42:11.099 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.099 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.099 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:11.099 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.882287+0000", 2024-09-11T07:42:11.099 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.539e-05 2024-09-11T07:42:11.099 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.099 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.100 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:11.100 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.882294+0000", 2024-09-11T07:42:11.100 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1199999999999996e-06 2024-09-11T07:42:11.100 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.100 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.100 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.100 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.882347+0000", 2024-09-11T07:42:11.100 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2775999999999998e-05 2024-09-11T07:42:11.100 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.100 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.100 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:11.100 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.882357+0000", 2024-09-11T07:42:11.100 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.5449999999999992e-06 2024-09-11T07:42:11.101 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.101 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.101 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.101 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726505+0000", 2024-09-11T07:42:11.101 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.84414887199999999 2024-09-11T07:42:11.101 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.101 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.101 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:11.101 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726883+0000", 2024-09-11T07:42:11.101 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00037714500000000001 2024-09-11T07:42:11.101 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.101 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.102 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:42:11.102 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.727344+0000", 2024-09-11T07:42:11.102 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000461125 2024-09-11T07:42:11.102 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.102 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.102 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.102 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.734836+0000", 2024-09-11T07:42:11.102 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0074918440000000001 2024-09-11T07:42:11.102 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.102 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.102 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.102 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.734863+0000", 2024-09-11T07:42:11.102 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7881e-05 2024-09-11T07:42:11.102 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.103 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.103 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:11.103 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.734997+0000", 2024-09-11T07:42:11.103 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013358200000000001 2024-09-11T07:42:11.103 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.103 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.103 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:11.103 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.735030+0000", 2024-09-11T07:42:11.103 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3080000000000002e-05 2024-09-11T07:42:11.103 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.103 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.103 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:11.103 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.735060+0000", 2024-09-11T07:42:11.104 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0295e-05 2024-09-11T07:42:11.104 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.104 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:11.104 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.104 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.104 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.104 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235344 9.0 9:00f7bf85:::benchmark_data_smithi022_133943_object108753:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-11T07:42:11.104 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.902276+0000", 2024-09-11T07:42:11.104 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 69.101390421999994, 2024-09-11T07:42:11.104 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.83494863200000002, 2024-09-11T07:42:11.104 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:11.104 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:11.104 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:11.105 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:11.105 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:11.105 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235344 2024-09-11T07:42:11.105 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.105 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:11.105 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.105 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:11.105 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.902276+0000", 2024-09-11T07:42:11.105 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.105 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.105 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.105 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:11.105 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.902276+0000", 2024-09-11T07:42:11.106 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.106 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.106 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.106 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:11.106 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.902273+0000", 2024-09-11T07:42:11.106 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:42:11.106 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.106 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.106 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:11.106 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.902281+0000", 2024-09-11T07:42:11.106 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.977e-06 2024-09-11T07:42:11.106 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.106 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.106 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:11.107 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.902295+0000", 2024-09-11T07:42:11.107 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3726e-05 2024-09-11T07:42:11.107 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.107 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.107 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:11.107 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.902300+0000", 2024-09-11T07:42:11.107 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1549999999999998e-06 2024-09-11T07:42:11.107 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.107 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.107 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.107 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.902458+0000", 2024-09-11T07:42:11.107 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015830900000000001 2024-09-11T07:42:11.107 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.108 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.108 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:11.108 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.902495+0000", 2024-09-11T07:42:11.108 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7129999999999999e-05 2024-09-11T07:42:11.108 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.108 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.108 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.108 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.731580+0000", 2024-09-11T07:42:11.108 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.82908503600000005 2024-09-11T07:42:11.108 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.108 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.108 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:11.108 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.731693+0000", 2024-09-11T07:42:11.109 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000112385 2024-09-11T07:42:11.109 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.109 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.109 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:42:11.109 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.732164+0000", 2024-09-11T07:42:11.109 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00047157600000000003 2024-09-11T07:42:11.109 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.109 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.109 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.109 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.734894+0000", 2024-09-11T07:42:11.109 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0027298420000000001 2024-09-11T07:42:11.109 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.109 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.110 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:11.110 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.735986+0000", 2024-09-11T07:42:11.110 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0010919199999999999 2024-09-11T07:42:11.110 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.110 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.110 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.110 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.736952+0000", 2024-09-11T07:42:11.110 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00096604600000000003 2024-09-11T07:42:11.110 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.110 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.110 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:11.110 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.737178+0000", 2024-09-11T07:42:11.110 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00022557100000000001 2024-09-11T07:42:11.110 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.111 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.111 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:11.111 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.737224+0000", 2024-09-11T07:42:11.111 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6517999999999997e-05 2024-09-11T07:42:11.111 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.111 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:11.111 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.111 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.111 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.111 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.15504.0:238059 9.6 e324/259 9:638fc1da:::benchmark_data_smithi022_133943_object111468:head v 324'10440, mlcod=324'10437)", 2024-09-11T07:42:11.111 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:05.637692+0000", 2024-09-11T07:42:11.111 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 65.365973979000003, 2024-09-11T07:42:11.111 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0271874, 2024-09-11T07:42:11.112 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:11.112 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:11.112 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:11.112 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.112 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:11.112 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:05.637692+0000", 2024-09-11T07:42:11.112 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.112 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.112 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.112 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:11.112 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:05.637692+0000", 2024-09-11T07:42:11.112 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.112 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.112 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.113 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:11.113 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:05.637690+0000", 2024-09-11T07:42:11.113 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:42:11.113 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.113 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.113 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:11.113 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:05.637702+0000", 2024-09-11T07:42:11.113 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2106000000000001e-05 2024-09-11T07:42:11.113 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.113 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.113 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:11.113 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:05.637729+0000", 2024-09-11T07:42:11.113 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.6956000000000001e-05 2024-09-11T07:42:11.114 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.114 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.114 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:11.114 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:05.637737+0000", 2024-09-11T07:42:11.114 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5299999999999999e-06 2024-09-11T07:42:11.114 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.114 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.114 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.114 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:05.637909+0000", 2024-09-11T07:42:11.114 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000172158 2024-09-11T07:42:11.114 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.114 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.114 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:11.115 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:05.637975+0000", 2024-09-11T07:42:11.115 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.6184000000000002e-05 2024-09-11T07:42:11.115 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.115 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.115 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:11.115 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:05.659061+0000", 2024-09-11T07:42:11.115 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.02108581 2024-09-11T07:42:11.115 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.115 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.115 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:11.115 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:05.664879+0000", 2024-09-11T07:42:11.115 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0058185210000000001 2024-09-11T07:42:11.115 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.116 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:11.116 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.116 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.116 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.116 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:238350 9.0 9:018ba3af:::benchmark_data_smithi022_133943_object111759:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e324)", 2024-09-11T07:42:11.116 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:05.869383+0000", 2024-09-11T07:42:11.116 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 65.134283170000003, 2024-09-11T07:42:11.116 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.027329886000000001, 2024-09-11T07:42:11.116 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:11.116 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:11.116 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:11.116 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:11.117 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:11.117 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 238350 2024-09-11T07:42:11.117 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.117 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:11.117 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.117 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:11.117 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:05.869383+0000", 2024-09-11T07:42:11.117 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.117 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.117 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.117 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:11.117 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:05.869383+0000", 2024-09-11T07:42:11.118 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.118 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.118 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.118 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:11.118 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:05.869382+0000", 2024-09-11T07:42:11.118 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:42:11.118 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.118 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.118 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:11.118 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:05.869385+0000", 2024-09-11T07:42:11.118 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.146e-06 2024-09-11T07:42:11.118 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.118 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.118 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:11.119 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:05.869390+0000", 2024-09-11T07:42:11.119 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.967e-06 2024-09-11T07:42:11.119 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.119 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.119 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:11.119 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:05.869394+0000", 2024-09-11T07:42:11.119 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.54e-06 2024-09-11T07:42:11.119 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.119 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.119 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.119 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:05.877559+0000", 2024-09-11T07:42:11.119 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0081650100000000003 2024-09-11T07:42:11.119 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.119 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.120 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:11.120 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:05.877694+0000", 2024-09-11T07:42:11.120 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013519199999999999 2024-09-11T07:42:11.120 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.120 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.120 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:42:11.120 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:05.878292+0000", 2024-09-11T07:42:11.120 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00059780800000000002 2024-09-11T07:42:11.120 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.120 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.120 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:11.120 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:05.882158+0000", 2024-09-11T07:42:11.120 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0038658709999999999 2024-09-11T07:42:11.120 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.121 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.121 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.121 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:05.882391+0000", 2024-09-11T07:42:11.121 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00023337600000000001 2024-09-11T07:42:11.121 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.121 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.121 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:11.121 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:05.896632+0000", 2024-09-11T07:42:11.121 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.014240887000000001 2024-09-11T07:42:11.121 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.121 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.121 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:11.121 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:05.896678+0000", 2024-09-11T07:42:11.122 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5762e-05 2024-09-11T07:42:11.122 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.122 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.122 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:11.122 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:05.896713+0000", 2024-09-11T07:42:11.122 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4937000000000001e-05 2024-09-11T07:42:11.122 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.122 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:11.122 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.122 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.122 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.122 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "replica_scrub(pg: 6.3,from:0'0,to:0'0,epoch:348/147,start:6:c0000000::::0,end:MAX,chunky:1,deep:0,version:9,allow_preemption:1,priority=5)", 2024-09-11T07:42:11.122 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:42:08.910597+0000", 2024-09-11T07:42:11.122 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 2.0930685900000001, 2024-09-11T07:42:11.123 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000240227, 2024-09-11T07:42:11.123 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:11.123 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:42:11.123 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:11.123 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.123 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:11.123 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:08.910597+0000", 2024-09-11T07:42:11.123 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.123 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.123 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.123 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:11.123 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:08.910597+0000", 2024-09-11T07:42:11.123 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.123 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.124 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.124 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:11.124 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:08.910595+0000", 2024-09-11T07:42:11.124 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:42:11.124 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.124 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.124 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:11.124 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:08.910608+0000", 2024-09-11T07:42:11.124 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3156000000000001e-05 2024-09-11T07:42:11.124 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.124 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.124 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:11.124 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:08.910636+0000", 2024-09-11T07:42:11.124 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8374e-05 2024-09-11T07:42:11.125 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.125 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.125 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:11.125 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:08.910649+0000", 2024-09-11T07:42:11.125 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2616000000000001e-05 2024-09-11T07:42:11.125 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.125 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.125 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.125 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:08.910776+0000", 2024-09-11T07:42:11.125 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012740900000000001 2024-09-11T07:42:11.125 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.125 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.125 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:11.126 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:08.910797+0000", 2024-09-11T07:42:11.126 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1120000000000001e-05 2024-09-11T07:42:11.126 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.126 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.126 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:11.126 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:08.910838+0000", 2024-09-11T07:42:11.126 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0108999999999998e-05 2024-09-11T07:42:11.126 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.126 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:11.126 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.126 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.126 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.126 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24353.0:2224 5.d 5:b4812045:::notify.4:head [watch ping cookie 94395805755184 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e346)", 2024-09-11T07:42:11.127 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:42:09.696996+0000", 2024-09-11T07:42:11.127 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 1.3066703340000001, 2024-09-11T07:42:11.127 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00029861900000000002, 2024-09-11T07:42:11.127 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:11.127 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:42:11.127 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:11.127 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24353", 2024-09-11T07:42:11.127 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1860969011", 2024-09-11T07:42:11.127 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 2224 2024-09-11T07:42:11.127 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.127 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:11.127 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.127 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:11.128 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:09.696996+0000", 2024-09-11T07:42:11.128 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.128 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.128 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.128 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:11.128 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:09.696996+0000", 2024-09-11T07:42:11.128 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.128 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.128 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.128 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:11.128 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:09.696994+0000", 2024-09-11T07:42:11.128 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:42:11.128 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.129 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.129 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:11.129 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:09.697000+0000", 2024-09-11T07:42:11.129 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3010000000000003e-06 2024-09-11T07:42:11.129 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.129 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.129 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:11.129 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:09.697012+0000", 2024-09-11T07:42:11.129 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2067e-05 2024-09-11T07:42:11.129 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.129 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.129 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:11.129 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:09.697020+0000", 2024-09-11T07:42:11.129 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.7640000000000005e-06 2024-09-11T07:42:11.130 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.130 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.130 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.130 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:09.697117+0000", 2024-09-11T07:42:11.130 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.6778000000000001e-05 2024-09-11T07:42:11.130 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.130 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.130 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:11.130 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:09.697195+0000", 2024-09-11T07:42:11.130 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.8646000000000006e-05 2024-09-11T07:42:11.130 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.130 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.130 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:11.131 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:09.697294+0000", 2024-09-11T07:42:11.131 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.9061000000000006e-05 2024-09-11T07:42:11.131 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.131 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:11.131 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.131 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.131 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.131 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24353.0:2225 4.4 4:22d26bf9:::data_loggenerations_metadata:head [watch ping cookie 94395805765568 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e346)", 2024-09-11T07:42:11.131 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:42:09.697101+0000", 2024-09-11T07:42:11.131 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 1.3065646390000001, 2024-09-11T07:42:11.131 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00028262300000000002, 2024-09-11T07:42:11.131 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:11.131 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:42:11.132 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:11.132 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24353", 2024-09-11T07:42:11.132 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1860969011", 2024-09-11T07:42:11.132 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 2225 2024-09-11T07:42:11.132 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.132 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:11.132 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.132 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:11.132 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:09.697101+0000", 2024-09-11T07:42:11.132 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.132 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.132 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.132 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:11.133 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:09.697101+0000", 2024-09-11T07:42:11.133 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:11.133 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.133 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.133 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:11.133 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:09.697100+0000", 2024-09-11T07:42:11.133 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:42:11.133 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.133 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.133 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:11.133 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:09.697106+0000", 2024-09-11T07:42:11.133 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9020000000000004e-06 2024-09-11T07:42:11.133 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.134 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.134 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:11.134 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:09.697115+0000", 2024-09-11T07:42:11.134 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3990000000000004e-06 2024-09-11T07:42:11.134 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.134 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.134 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:11.134 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:09.697120+0000", 2024-09-11T07:42:11.134 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1699999999999996e-06 2024-09-11T07:42:11.134 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.134 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.134 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:11.134 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:09.697191+0000", 2024-09-11T07:42:11.134 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1098000000000003e-05 2024-09-11T07:42:11.135 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.135 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.135 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:11.135 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:09.697275+0000", 2024-09-11T07:42:11.135 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3646999999999994e-05 2024-09-11T07:42:11.135 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:11.135 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:11.135 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:11.135 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:09.697384+0000", 2024-09-11T07:42:11.135 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0001094 2024-09-11T07:42:11.135 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.135 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:11.135 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.135 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:11.136 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:11.136 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:42:11.136 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:11.158 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:11.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:11.160 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:11.260 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:11.287 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:11.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:11.288 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:11.389 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:11.411 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:11.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:11.412 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:11.513 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:11.536 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:11.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:11.537 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:11.638 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:11.660 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:11.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:11.661 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:11.762 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:11.784 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:11.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:11.785 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:11.886 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:11.912 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:11.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:11.914 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:11.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:11 smithi022 ceph-mon[29462]: 3.1 scrub starts 2024-09-11T07:42:11.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:11 smithi022 ceph-mon[29462]: 3.1 scrub ok 2024-09-11T07:42:11.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:11 smithi022 ceph-mon[29462]: 2.1c scrub starts 2024-09-11T07:42:11.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:11 smithi022 ceph-mon[29462]: 2.1c scrub ok 2024-09-11T07:42:11.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:11 smithi022 ceph-mon[29462]: 4.a scrub starts 2024-09-11T07:42:11.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:11 smithi022 ceph-mon[29462]: 4.a scrub ok 2024-09-11T07:42:11.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:11 smithi022 ceph-mon[29462]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-11T07:42:11.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:11 smithi022 ceph-mon[37330]: 3.1 scrub starts 2024-09-11T07:42:11.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:11 smithi022 ceph-mon[37330]: 3.1 scrub ok 2024-09-11T07:42:11.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:11 smithi022 ceph-mon[37330]: 2.1c scrub starts 2024-09-11T07:42:11.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:11 smithi022 ceph-mon[37330]: 2.1c scrub ok 2024-09-11T07:42:11.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:11 smithi022 ceph-mon[37330]: 4.a scrub starts 2024-09-11T07:42:11.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:11 smithi022 ceph-mon[37330]: 4.a scrub ok 2024-09-11T07:42:11.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:11 smithi022 ceph-mon[37330]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-11T07:42:12.014 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:12.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:11 smithi104 ceph-mon[33507]: 3.1 scrub starts 2024-09-11T07:42:12.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:11 smithi104 ceph-mon[33507]: 3.1 scrub ok 2024-09-11T07:42:12.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:11 smithi104 ceph-mon[33507]: 2.1c scrub starts 2024-09-11T07:42:12.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:11 smithi104 ceph-mon[33507]: 2.1c scrub ok 2024-09-11T07:42:12.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:11 smithi104 ceph-mon[33507]: 4.a scrub starts 2024-09-11T07:42:12.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:11 smithi104 ceph-mon[33507]: 4.a scrub ok 2024-09-11T07:42:12.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:11 smithi104 ceph-mon[33507]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-11T07:42:12.036 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:12.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:12.037 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:12.138 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:12.160 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:12.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:12.161 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:12.262 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:12.288 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:12.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:12.289 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:12.390 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:12.413 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:12.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:12.414 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:12.515 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:12.541 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:12.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:12.542 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:12.642 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:12.660 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-11T07:42:12.668 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:12.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:12.669 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:12.694 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:42:12.742 INFO:teuthology.orchestra.run.smithi022.stderr:enabled application 'rados' on pool 'unique_pool_3' 2024-09-11T07:42:12.770 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:12.793 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:12.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:12.794 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:12.806 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:42:12.894 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:12.911 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:12 smithi022 ceph-mon[29462]: 2.1f scrub starts 2024-09-11T07:42:12.911 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:12 smithi022 ceph-mon[29462]: 2.1f scrub ok 2024-09-11T07:42:12.911 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:12 smithi022 ceph-mon[29462]: 6.1d scrub starts 2024-09-11T07:42:12.911 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:12 smithi022 ceph-mon[29462]: 6.1d scrub ok 2024-09-11T07:42:12.911 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:12 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2350878699' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:42:12.912 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:12 smithi022 ceph-mon[29462]: pgmap v731: 148 pgs: 148 active+clean; 582 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:42:12.912 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:12 smithi022 ceph-mon[37330]: 2.1f scrub starts 2024-09-11T07:42:12.912 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:12 smithi022 ceph-mon[37330]: 2.1f scrub ok 2024-09-11T07:42:12.912 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:12 smithi022 ceph-mon[37330]: 6.1d scrub starts 2024-09-11T07:42:12.913 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:12 smithi022 ceph-mon[37330]: 6.1d scrub ok 2024-09-11T07:42:12.913 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:12 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2350878699' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:42:12.913 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:12 smithi022 ceph-mon[37330]: pgmap v731: 148 pgs: 148 active+clean; 582 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:42:12.923 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:12.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:12.924 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:13.025 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:13.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:12 smithi104 ceph-mon[33507]: 2.1f scrub starts 2024-09-11T07:42:13.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:12 smithi104 ceph-mon[33507]: 2.1f scrub ok 2024-09-11T07:42:13.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:12 smithi104 ceph-mon[33507]: 6.1d scrub starts 2024-09-11T07:42:13.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:12 smithi104 ceph-mon[33507]: 6.1d scrub ok 2024-09-11T07:42:13.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:12 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2350878699' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:42:13.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:12 smithi104 ceph-mon[33507]: pgmap v731: 148 pgs: 148 active+clean; 582 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:42:13.051 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:13.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:13.052 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:13.152 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:13.175 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:13.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:13.176 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:13.276 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:13.299 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:13.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:13.300 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:13.400 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:13.426 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:13.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:13.427 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:13.528 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:13.554 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:13.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:13.555 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:13.655 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:13.680 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:13.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:13.681 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:13.782 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:13.807 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:13.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:13.808 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:13.909 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:13.916 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:13 smithi022 ceph-mon[37330]: 2.e scrub starts 2024-09-11T07:42:13.916 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:13 smithi022 ceph-mon[37330]: 2.e scrub ok 2024-09-11T07:42:13.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:13 smithi022 ceph-mon[37330]: 6.4 scrub starts 2024-09-11T07:42:13.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:13 smithi022 ceph-mon[37330]: 6.4 scrub ok 2024-09-11T07:42:13.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:13 smithi022 ceph-mon[37330]: 5.1b scrub starts 2024-09-11T07:42:13.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:13 smithi022 ceph-mon[37330]: 5.1b scrub ok 2024-09-11T07:42:13.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:13 smithi022 ceph-mon[37330]: 4.1e scrub starts 2024-09-11T07:42:13.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:13 smithi022 ceph-mon[37330]: 4.1e scrub ok 2024-09-11T07:42:13.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:13 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2350878699' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:42:13.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:13 smithi022 ceph-mon[37330]: osdmap e349: 8 total, 8 up, 8 in 2024-09-11T07:42:13.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:13 smithi022 ceph-mon[37330]: 2.1b scrub starts 2024-09-11T07:42:13.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:13 smithi022 ceph-mon[37330]: 2.1b scrub ok 2024-09-11T07:42:13.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:13 smithi022 ceph-mon[29462]: 2.e scrub starts 2024-09-11T07:42:13.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:13 smithi022 ceph-mon[29462]: 2.e scrub ok 2024-09-11T07:42:13.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:13 smithi022 ceph-mon[29462]: 6.4 scrub starts 2024-09-11T07:42:13.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:13 smithi022 ceph-mon[29462]: 6.4 scrub ok 2024-09-11T07:42:13.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:13 smithi022 ceph-mon[29462]: 5.1b scrub starts 2024-09-11T07:42:13.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:13 smithi022 ceph-mon[29462]: 5.1b scrub ok 2024-09-11T07:42:13.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:13 smithi022 ceph-mon[29462]: 4.1e scrub starts 2024-09-11T07:42:13.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:13 smithi022 ceph-mon[29462]: 4.1e scrub ok 2024-09-11T07:42:13.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:13 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2350878699' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:42:13.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:13 smithi022 ceph-mon[29462]: osdmap e349: 8 total, 8 up, 8 in 2024-09-11T07:42:13.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:13 smithi022 ceph-mon[29462]: 2.1b scrub starts 2024-09-11T07:42:13.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:13 smithi022 ceph-mon[29462]: 2.1b scrub ok 2024-09-11T07:42:13.932 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:13.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:13.933 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:14.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:13 smithi104 ceph-mon[33507]: 2.e scrub starts 2024-09-11T07:42:14.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:13 smithi104 ceph-mon[33507]: 2.e scrub ok 2024-09-11T07:42:14.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:13 smithi104 ceph-mon[33507]: 6.4 scrub starts 2024-09-11T07:42:14.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:13 smithi104 ceph-mon[33507]: 6.4 scrub ok 2024-09-11T07:42:14.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:13 smithi104 ceph-mon[33507]: 5.1b scrub starts 2024-09-11T07:42:14.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:13 smithi104 ceph-mon[33507]: 5.1b scrub ok 2024-09-11T07:42:14.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:13 smithi104 ceph-mon[33507]: 4.1e scrub starts 2024-09-11T07:42:14.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:13 smithi104 ceph-mon[33507]: 4.1e scrub ok 2024-09-11T07:42:14.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:13 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2350878699' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:42:14.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:13 smithi104 ceph-mon[33507]: osdmap e349: 8 total, 8 up, 8 in 2024-09-11T07:42:14.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:13 smithi104 ceph-mon[33507]: 2.1b scrub starts 2024-09-11T07:42:14.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:13 smithi104 ceph-mon[33507]: 2.1b scrub ok 2024-09-11T07:42:14.034 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:14.057 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:14.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:14.058 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:14.159 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:14.182 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:14.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:14.183 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:14.284 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:14.310 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:14.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:14.311 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:14.411 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:14.434 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:14.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:14.435 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:14.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:42:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:42:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:42:14.536 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:14.559 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:14.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:14.561 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:14.661 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:14.684 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:14.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:14.685 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:14.786 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:14.809 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:14.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:14.810 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:14.910 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:14.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:14 smithi022 ceph-mon[29462]: 5.b scrub starts 2024-09-11T07:42:14.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:14 smithi022 ceph-mon[29462]: 5.b scrub ok 2024-09-11T07:42:14.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:14 smithi022 ceph-mon[29462]: 2.17 scrub starts 2024-09-11T07:42:14.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:14 smithi022 ceph-mon[29462]: 2.17 scrub ok 2024-09-11T07:42:14.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:14 smithi022 ceph-mon[29462]: 4.0 scrub starts 2024-09-11T07:42:14.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:14 smithi022 ceph-mon[29462]: 4.0 scrub ok 2024-09-11T07:42:14.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:14 smithi022 ceph-mon[29462]: 4.1f scrub starts 2024-09-11T07:42:14.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:14 smithi022 ceph-mon[29462]: 4.1f scrub ok 2024-09-11T07:42:14.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:14 smithi022 ceph-mon[29462]: 2.6 scrub starts 2024-09-11T07:42:14.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:14 smithi022 ceph-mon[29462]: 2.6 scrub ok 2024-09-11T07:42:14.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:14 smithi022 ceph-mon[29462]: pgmap v733: 148 pgs: 148 active+clean; 582 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T07:42:14.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:14 smithi022 ceph-mon[37330]: 5.b scrub starts 2024-09-11T07:42:14.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:14 smithi022 ceph-mon[37330]: 5.b scrub ok 2024-09-11T07:42:14.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:14 smithi022 ceph-mon[37330]: 2.17 scrub starts 2024-09-11T07:42:14.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:14 smithi022 ceph-mon[37330]: 2.17 scrub ok 2024-09-11T07:42:14.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:14 smithi022 ceph-mon[37330]: 4.0 scrub starts 2024-09-11T07:42:14.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:14 smithi022 ceph-mon[37330]: 4.0 scrub ok 2024-09-11T07:42:14.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:14 smithi022 ceph-mon[37330]: 4.1f scrub starts 2024-09-11T07:42:14.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:14 smithi022 ceph-mon[37330]: 4.1f scrub ok 2024-09-11T07:42:14.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:14 smithi022 ceph-mon[37330]: 2.6 scrub starts 2024-09-11T07:42:14.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:14 smithi022 ceph-mon[37330]: 2.6 scrub ok 2024-09-11T07:42:14.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:14 smithi022 ceph-mon[37330]: pgmap v733: 148 pgs: 148 active+clean; 582 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T07:42:14.933 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:14.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:14.934 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:15.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:14 smithi104 ceph-mon[33507]: 5.b scrub starts 2024-09-11T07:42:15.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:14 smithi104 ceph-mon[33507]: 5.b scrub ok 2024-09-11T07:42:15.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:14 smithi104 ceph-mon[33507]: 2.17 scrub starts 2024-09-11T07:42:15.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:14 smithi104 ceph-mon[33507]: 2.17 scrub ok 2024-09-11T07:42:15.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:14 smithi104 ceph-mon[33507]: 4.0 scrub starts 2024-09-11T07:42:15.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:14 smithi104 ceph-mon[33507]: 4.0 scrub ok 2024-09-11T07:42:15.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:14 smithi104 ceph-mon[33507]: 4.1f scrub starts 2024-09-11T07:42:15.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:14 smithi104 ceph-mon[33507]: 4.1f scrub ok 2024-09-11T07:42:15.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:14 smithi104 ceph-mon[33507]: 2.6 scrub starts 2024-09-11T07:42:15.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:14 smithi104 ceph-mon[33507]: 2.6 scrub ok 2024-09-11T07:42:15.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:14 smithi104 ceph-mon[33507]: pgmap v733: 148 pgs: 148 active+clean; 582 KiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T07:42:15.034 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:15.060 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:15.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:15.061 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:15.162 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:15.184 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:15.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:15.185 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:15.285 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:15.311 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:15.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:15.312 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:15.413 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:15.435 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:15.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:15.436 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:15.537 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:15.560 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:15.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:15.561 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:15.662 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:15.667 DEBUG:teuthology.orchestra.run.smithi022:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_3 bench 90 write' 2024-09-11T07:42:15.707 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:15.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:15.708 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:15.763 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-11T07:42:15.763 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-11T07:42:15.809 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:15.835 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:15.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:15.836 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:15.909 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:15 smithi022 ceph-mon[29462]: 4.1b scrub starts 2024-09-11T07:42:15.909 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:15 smithi022 ceph-mon[29462]: 4.1b scrub ok 2024-09-11T07:42:15.909 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:15 smithi022 ceph-mon[29462]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:42:15.910 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:15 smithi022 ceph-mon[29462]: osdmap e350: 8 total, 8 up, 8 in 2024-09-11T07:42:15.910 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:15 smithi022 ceph-mon[37330]: 4.1b scrub starts 2024-09-11T07:42:15.910 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:15 smithi022 ceph-mon[37330]: 4.1b scrub ok 2024-09-11T07:42:15.910 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:15 smithi022 ceph-mon[37330]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:42:15.910 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:15 smithi022 ceph-mon[37330]: osdmap e350: 8 total, 8 up, 8 in 2024-09-11T07:42:15.916 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:42:15.936 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:15.949 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:hints = 1 2024-09-11T07:42:15.949 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-11T07:42:15.949 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Object prefix: benchmark_data_smithi022_163866 2024-09-11T07:42:15.950 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T07:42:15.950 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 0 0 0 0 0 0 - 0 2024-09-11T07:42:15.976 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:15.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:15.977 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:16.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:15 smithi104 ceph-mon[33507]: 4.1b scrub starts 2024-09-11T07:42:16.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:15 smithi104 ceph-mon[33507]: 4.1b scrub ok 2024-09-11T07:42:16.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:15 smithi104 ceph-mon[33507]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:42:16.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:15 smithi104 ceph-mon[33507]: osdmap e350: 8 total, 8 up, 8 in 2024-09-11T07:42:16.078 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:16.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:16.121 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:16.121 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:16.222 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:16.251 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:16.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:16.252 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:16.353 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:16.383 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:16.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:16.384 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:16.485 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:16.524 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:16.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:16.525 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:16.626 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:16.661 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:16.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:16.662 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:16.763 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:16.764 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is unset 2024-09-11T07:42:16.792 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:42:16.792 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-11T07:42:16.792 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:42:16.805 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:16.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:16.806 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:16.907 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:16.940 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:16.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:16.941 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:16.950 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 1 15 3357 3342 208.864 208.875 0.00352834 0.00476861 2024-09-11T07:42:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:16 smithi104 ceph-mon[33507]: 2.16 scrub starts 2024-09-11T07:42:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:16 smithi104 ceph-mon[33507]: 2.16 scrub ok 2024-09-11T07:42:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:16 smithi104 ceph-mon[33507]: 6.18 scrub starts 2024-09-11T07:42:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:16 smithi104 ceph-mon[33507]: 6.18 scrub ok 2024-09-11T07:42:17.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:16 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1223367177' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:42:17.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:16 smithi104 ceph-mon[33507]: pgmap v735: 148 pgs: 148 active+clean; 582 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:42:17.042 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:17.075 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:17.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:17.076 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:17.077 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:16 smithi022 ceph-mon[29462]: 2.16 scrub starts 2024-09-11T07:42:17.077 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:16 smithi022 ceph-mon[29462]: 2.16 scrub ok 2024-09-11T07:42:17.078 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:16 smithi022 ceph-mon[29462]: 6.18 scrub starts 2024-09-11T07:42:17.078 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:16 smithi022 ceph-mon[29462]: 6.18 scrub ok 2024-09-11T07:42:17.078 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:16 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1223367177' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:42:17.078 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:16 smithi022 ceph-mon[29462]: pgmap v735: 148 pgs: 148 active+clean; 582 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:42:17.078 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:16 smithi022 ceph-mon[37330]: 2.16 scrub starts 2024-09-11T07:42:17.078 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:16 smithi022 ceph-mon[37330]: 2.16 scrub ok 2024-09-11T07:42:17.078 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:16 smithi022 ceph-mon[37330]: 6.18 scrub starts 2024-09-11T07:42:17.078 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:16 smithi022 ceph-mon[37330]: 6.18 scrub ok 2024-09-11T07:42:17.078 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:16 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1223367177' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:42:17.078 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:16 smithi022 ceph-mon[37330]: pgmap v735: 148 pgs: 148 active+clean; 582 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:42:17.177 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:17.200 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:17.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:17.201 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:17.302 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:17.341 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:17.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:17.342 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:17.443 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:17.477 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:17.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:17.478 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:17.578 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:17.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:17.612 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:17.612 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:17.713 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:17.728 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:42:17.742 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:17.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:17.743 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:17.844 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:17.857 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:17 smithi022 ceph-mon[29462]: 2.9 scrub starts 2024-09-11T07:42:17.858 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:17 smithi022 ceph-mon[29462]: 2.9 scrub ok 2024-09-11T07:42:17.858 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:17 smithi022 ceph-mon[29462]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:42:17.858 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:17 smithi022 ceph-mon[29462]: Cluster is now healthy 2024-09-11T07:42:17.858 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:17 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1223367177' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:42:17.858 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:17 smithi022 ceph-mon[29462]: osdmap e351: 8 total, 8 up, 8 in 2024-09-11T07:42:17.858 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:17 smithi022 ceph-mon[29462]: 6.12 scrub starts 2024-09-11T07:42:17.858 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:17 smithi022 ceph-mon[29462]: 6.12 scrub ok 2024-09-11T07:42:17.858 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:17 smithi022 ceph-mon[29462]: 5.0 scrub starts 2024-09-11T07:42:17.858 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:17 smithi022 ceph-mon[29462]: 5.0 scrub ok 2024-09-11T07:42:17.858 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:17 smithi022 ceph-mon[37330]: 2.9 scrub starts 2024-09-11T07:42:17.858 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:17 smithi022 ceph-mon[37330]: 2.9 scrub ok 2024-09-11T07:42:17.859 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:17 smithi022 ceph-mon[37330]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:42:17.859 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:17 smithi022 ceph-mon[37330]: Cluster is now healthy 2024-09-11T07:42:17.859 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:17 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1223367177' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:42:17.859 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:17 smithi022 ceph-mon[37330]: osdmap e351: 8 total, 8 up, 8 in 2024-09-11T07:42:17.859 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:17 smithi022 ceph-mon[37330]: 6.12 scrub starts 2024-09-11T07:42:17.859 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:17 smithi022 ceph-mon[37330]: 6.12 scrub ok 2024-09-11T07:42:17.859 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:17 smithi022 ceph-mon[37330]: 5.0 scrub starts 2024-09-11T07:42:17.859 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:17 smithi022 ceph-mon[37330]: 5.0 scrub ok 2024-09-11T07:42:17.886 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:17.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:17.887 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:17.950 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 2 16 6846 6830 213.422 218 0.00434058 0.00467688 2024-09-11T07:42:17.989 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:18.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:17 smithi104 ceph-mon[33507]: 2.9 scrub starts 2024-09-11T07:42:18.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:17 smithi104 ceph-mon[33507]: 2.9 scrub ok 2024-09-11T07:42:18.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:17 smithi104 ceph-mon[33507]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:42:18.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:17 smithi104 ceph-mon[33507]: Cluster is now healthy 2024-09-11T07:42:18.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:17 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1223367177' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:42:18.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:17 smithi104 ceph-mon[33507]: osdmap e351: 8 total, 8 up, 8 in 2024-09-11T07:42:18.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:17 smithi104 ceph-mon[33507]: 6.12 scrub starts 2024-09-11T07:42:18.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:17 smithi104 ceph-mon[33507]: 6.12 scrub ok 2024-09-11T07:42:18.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:17 smithi104 ceph-mon[33507]: 5.0 scrub starts 2024-09-11T07:42:18.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:17 smithi104 ceph-mon[33507]: 5.0 scrub ok 2024-09-11T07:42:18.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:18.033 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:18.034 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:18.135 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:18.174 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:18.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:18.176 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:18.276 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:18.316 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:18.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:18.318 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:18.418 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:18.449 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:18.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:18.450 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:18.550 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:18.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:18.593 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:18.593 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:18.694 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:18.728 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:18.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:18.729 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:18.829 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:18.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:18.861 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:18.861 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:18.950 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 3 15 10192 10177 212.005 209.188 0.0054699 0.00470822 2024-09-11T07:42:18.962 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:18.993 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:18.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:18.994 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:19.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:18 smithi104 ceph-mon[33507]: 1.0 scrub starts 2024-09-11T07:42:19.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:18 smithi104 ceph-mon[33507]: 1.0 scrub ok 2024-09-11T07:42:19.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:18 smithi104 ceph-mon[33507]: 6.5 scrub starts 2024-09-11T07:42:19.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:18 smithi104 ceph-mon[33507]: 6.5 scrub ok 2024-09-11T07:42:19.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:18 smithi104 ceph-mon[33507]: pgmap v737: 148 pgs: 148 active+clean; 582 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:42:19.095 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:19.134 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:19.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:19.136 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:19.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:18 smithi022 ceph-mon[29462]: 1.0 scrub starts 2024-09-11T07:42:19.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:18 smithi022 ceph-mon[29462]: 1.0 scrub ok 2024-09-11T07:42:19.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:18 smithi022 ceph-mon[29462]: 6.5 scrub starts 2024-09-11T07:42:19.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:18 smithi022 ceph-mon[29462]: 6.5 scrub ok 2024-09-11T07:42:19.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:18 smithi022 ceph-mon[29462]: pgmap v737: 148 pgs: 148 active+clean; 582 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:42:19.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:18 smithi022 ceph-mon[37330]: 1.0 scrub starts 2024-09-11T07:42:19.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:18 smithi022 ceph-mon[37330]: 1.0 scrub ok 2024-09-11T07:42:19.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:18 smithi022 ceph-mon[37330]: 6.5 scrub starts 2024-09-11T07:42:19.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:18 smithi022 ceph-mon[37330]: 6.5 scrub ok 2024-09-11T07:42:19.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:18 smithi022 ceph-mon[37330]: pgmap v737: 148 pgs: 148 active+clean; 582 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:42:19.236 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:19.276 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:19.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:19.277 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:19.378 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:19.409 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:19.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:19.410 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:19.511 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:19.544 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:19.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:19.545 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:19.646 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:19.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:19.684 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:19.684 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:19.785 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:19.809 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:19.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:19.810 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:19.911 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:19.943 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:19.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:19.944 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:19.950 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 4 16 13229 13213 206.437 189.75 0.00481672 0.00483777 2024-09-11T07:42:20.045 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:20.060 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:19 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:42:20.060 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:19 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:42:20.060 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:19 smithi022 ceph-mon[29462]: 2.15 scrub starts 2024-09-11T07:42:20.061 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:19 smithi022 ceph-mon[29462]: 2.15 scrub ok 2024-09-11T07:42:20.061 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:19 smithi022 ceph-mon[29462]: 4.5 scrub starts 2024-09-11T07:42:20.061 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:19 smithi022 ceph-mon[29462]: 4.5 scrub ok 2024-09-11T07:42:20.061 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:19 smithi022 ceph-mon[29462]: osdmap e352: 8 total, 8 up, 8 in 2024-09-11T07:42:20.061 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:19 smithi022 ceph-mon[29462]: 4.1d scrub starts 2024-09-11T07:42:20.061 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:19 smithi022 ceph-mon[29462]: 4.1d scrub ok 2024-09-11T07:42:20.061 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:19 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:42:20.062 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:19 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:42:20.062 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:19 smithi022 ceph-mon[37330]: 2.15 scrub starts 2024-09-11T07:42:20.062 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:19 smithi022 ceph-mon[37330]: 2.15 scrub ok 2024-09-11T07:42:20.062 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:19 smithi022 ceph-mon[37330]: 4.5 scrub starts 2024-09-11T07:42:20.062 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:19 smithi022 ceph-mon[37330]: 4.5 scrub ok 2024-09-11T07:42:20.062 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:19 smithi022 ceph-mon[37330]: osdmap e352: 8 total, 8 up, 8 in 2024-09-11T07:42:20.062 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:19 smithi022 ceph-mon[37330]: 4.1d scrub starts 2024-09-11T07:42:20.062 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:19 smithi022 ceph-mon[37330]: 4.1d scrub ok 2024-09-11T07:42:20.078 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:20.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:20.079 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:20.141 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:42:20.142 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:42:20.142 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.142 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:13706 10.f 10:f641c6e6:::benchmark_data_smithi022_163866_object13705:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-11T07:42:20.142 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:42:20.129831+0000", 2024-09-11T07:42:20.142 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0076940239999999998, 2024-09-11T07:42:20.142 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0077253319999999997, 2024-09-11T07:42:20.142 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:20.142 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "waiting for sub ops", 2024-09-11T07:42:20.142 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:20.142 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:42:20.143 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:42:20.143 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 13706 2024-09-11T07:42:20.143 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.143 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:20.143 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.143 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:20.143 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.129831+0000", 2024-09-11T07:42:20.143 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:20.143 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.143 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.143 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:20.143 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.129831+0000", 2024-09-11T07:42:20.144 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:20.144 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.144 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.144 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:20.144 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.129829+0000", 2024-09-11T07:42:20.144 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:42:20.144 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.144 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.144 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:20.144 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.129855+0000", 2024-09-11T07:42:20.144 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5689000000000001e-05 2024-09-11T07:42:20.144 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.145 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.145 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:20.145 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.129865+0000", 2024-09-11T07:42:20.145 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0137e-05 2024-09-11T07:42:20.145 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.145 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.145 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:20.145 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.129871+0000", 2024-09-11T07:42:20.145 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6559999999999997e-06 2024-09-11T07:42:20.145 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.145 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.146 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:20.146 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.129900+0000", 2024-09-11T07:42:20.146 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.9759000000000001e-05 2024-09-11T07:42:20.146 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.146 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.146 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:20.146 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.130037+0000", 2024-09-11T07:42:20.146 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013634500000000001 2024-09-11T07:42:20.146 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.146 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.146 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:42:20.147 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.130169+0000", 2024-09-11T07:42:20.147 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013243799999999999 2024-09-11T07:42:20.147 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.147 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.147 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:20.147 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.136337+0000", 2024-09-11T07:42:20.147 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0061674099999999999 2024-09-11T07:42:20.147 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:20.147 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:20.147 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:20.147 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.148 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.148 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:13708 10.f 10:fac2a4bf:::benchmark_data_smithi022_163866_object13707:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-11T07:42:20.148 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:42:20.130596+0000", 2024-09-11T07:42:20.148 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0069292379999999999, 2024-09-11T07:42:20.148 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0070546000000000003, 2024-09-11T07:42:20.148 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:20.148 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "waiting for sub ops", 2024-09-11T07:42:20.148 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:20.148 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:42:20.148 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:42:20.149 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 13708 2024-09-11T07:42:20.149 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.149 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:20.149 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.149 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:20.149 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.130596+0000", 2024-09-11T07:42:20.149 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:20.149 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.149 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.149 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:20.150 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.130596+0000", 2024-09-11T07:42:20.150 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:20.150 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.150 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.150 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:20.150 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.130594+0000", 2024-09-11T07:42:20.150 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:42:20.150 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.150 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.150 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:20.150 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.130621+0000", 2024-09-11T07:42:20.151 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.6715000000000001e-05 2024-09-11T07:42:20.151 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.151 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.151 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:20.151 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.130633+0000", 2024-09-11T07:42:20.151 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.171e-05 2024-09-11T07:42:20.151 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.151 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.151 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:20.151 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.130639+0000", 2024-09-11T07:42:20.152 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5309999999999998e-06 2024-09-11T07:42:20.152 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.152 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.152 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:20.152 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.131540+0000", 2024-09-11T07:42:20.152 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00090120100000000002 2024-09-11T07:42:20.152 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.152 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.152 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:20.152 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.133585+0000", 2024-09-11T07:42:20.152 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002044779 2024-09-11T07:42:20.153 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.153 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.153 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:42:20.153 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.133713+0000", 2024-09-11T07:42:20.153 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000128162 2024-09-11T07:42:20.153 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.153 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.153 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:20.153 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.136369+0000", 2024-09-11T07:42:20.153 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0026556190000000001 2024-09-11T07:42:20.153 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:20.154 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:20.154 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:20.154 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.154 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.154 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:13712 10.f 10:f1eaa820:::benchmark_data_smithi022_163866_object13711:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-11T07:42:20.154 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:42:20.130650+0000", 2024-09-11T07:42:20.154 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0068750399999999998, 2024-09-11T07:42:20.154 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0070930669999999998, 2024-09-11T07:42:20.154 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:20.154 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "waiting for sub ops", 2024-09-11T07:42:20.154 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:20.154 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:42:20.155 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:42:20.155 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 13712 2024-09-11T07:42:20.155 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.155 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:20.155 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.155 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:20.155 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.130650+0000", 2024-09-11T07:42:20.155 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:20.155 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.155 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.155 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:20.155 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.130650+0000", 2024-09-11T07:42:20.155 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:20.156 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.156 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.156 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:20.156 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.130649+0000", 2024-09-11T07:42:20.156 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:42:20.156 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.156 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.156 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:20.156 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.130672+0000", 2024-09-11T07:42:20.156 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3110999999999999e-05 2024-09-11T07:42:20.156 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.157 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.157 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:20.157 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.130680+0000", 2024-09-11T07:42:20.157 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.9859999999999994e-06 2024-09-11T07:42:20.157 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.157 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.157 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:20.157 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.130685+0000", 2024-09-11T07:42:20.157 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8629999999999999e-06 2024-09-11T07:42:20.157 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.157 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.158 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:20.158 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.135051+0000", 2024-09-11T07:42:20.158 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0043660590000000003 2024-09-11T07:42:20.158 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.158 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.158 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:20.158 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.135223+0000", 2024-09-11T07:42:20.158 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000171731 2024-09-11T07:42:20.158 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.158 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.158 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:42:20.159 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.135350+0000", 2024-09-11T07:42:20.159 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012705799999999999 2024-09-11T07:42:20.159 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.159 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.159 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:20.159 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.137154+0000", 2024-09-11T07:42:20.159 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001803912 2024-09-11T07:42:20.159 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:20.159 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:20.159 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:20.159 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.160 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.160 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:13716 10.f 10:f5fb63e4:::benchmark_data_smithi022_163866_object13715:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-11T07:42:20.160 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:42:20.132965+0000", 2024-09-11T07:42:20.160 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.0045603600000000003, 2024-09-11T07:42:20.160 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0048650680000000002, 2024-09-11T07:42:20.160 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:20.160 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "waiting for sub ops", 2024-09-11T07:42:20.160 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:20.160 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:42:20.160 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:42:20.160 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 13716 2024-09-11T07:42:20.161 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.161 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:20.161 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.161 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:20.161 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.132965+0000", 2024-09-11T07:42:20.161 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:20.161 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.161 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.161 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:20.161 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.132965+0000", 2024-09-11T07:42:20.162 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:20.162 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.162 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.162 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:20.162 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.132963+0000", 2024-09-11T07:42:20.162 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:42:20.162 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.162 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.162 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:20.162 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.133017+0000", 2024-09-11T07:42:20.162 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.4611999999999998e-05 2024-09-11T07:42:20.162 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.163 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.163 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:20.163 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.133031+0000", 2024-09-11T07:42:20.163 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3718e-05 2024-09-11T07:42:20.163 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.163 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.163 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:20.163 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.133038+0000", 2024-09-11T07:42:20.163 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4860000000000002e-06 2024-09-11T07:42:20.163 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.163 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.163 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:20.164 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.137180+0000", 2024-09-11T07:42:20.164 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0041419999999999998 2024-09-11T07:42:20.164 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.164 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.164 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:20.164 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.137460+0000", 2024-09-11T07:42:20.164 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00028073399999999999 2024-09-11T07:42:20.164 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.164 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.164 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:42:20.164 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.137589+0000", 2024-09-11T07:42:20.165 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012839000000000001 2024-09-11T07:42:20.165 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:20.165 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:20.165 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:20.165 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.165 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.165 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:13718 10.f 10.d3a7e2f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-11T07:42:20.165 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:42:20.136576+0000", 2024-09-11T07:42:20.165 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.00094936900000000004, 2024-09-11T07:42:20.165 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0013285630000000001, 2024-09-11T07:42:20.166 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:20.166 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "queued for pg", 2024-09-11T07:42:20.166 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:20.166 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:42:20.166 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:42:20.166 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 13718 2024-09-11T07:42:20.166 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.166 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:20.166 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.166 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:20.166 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.136576+0000", 2024-09-11T07:42:20.166 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:20.167 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.167 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.167 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:20.167 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.136576+0000", 2024-09-11T07:42:20.167 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:20.167 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.167 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.167 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:20.167 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.136574+0000", 2024-09-11T07:42:20.167 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:42:20.167 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.168 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.168 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:20.168 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.136630+0000", 2024-09-11T07:42:20.168 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.5835000000000003e-05 2024-09-11T07:42:20.168 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.168 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.168 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:20.168 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.136644+0000", 2024-09-11T07:42:20.168 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4739000000000001e-05 2024-09-11T07:42:20.168 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.168 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.168 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:20.169 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.136651+0000", 2024-09-11T07:42:20.169 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1199999999999999e-06 2024-09-11T07:42:20.169 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:20.169 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:20.169 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:20.169 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.169 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.169 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:13722 10.f 10.377e180f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-11T07:42:20.169 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:42:20.136981+0000", 2024-09-11T07:42:20.169 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.00054471599999999999, 2024-09-11T07:42:20.169 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00098139899999999999, 2024-09-11T07:42:20.169 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:20.170 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "queued for pg", 2024-09-11T07:42:20.170 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:20.170 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:42:20.170 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:42:20.170 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 13722 2024-09-11T07:42:20.170 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.170 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:20.170 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.170 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:20.170 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.136981+0000", 2024-09-11T07:42:20.170 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:20.171 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.171 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.171 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:20.171 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.136981+0000", 2024-09-11T07:42:20.171 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:20.171 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.171 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.171 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:20.171 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.136979+0000", 2024-09-11T07:42:20.172 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:42:20.172 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.172 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.172 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:20.172 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.137004+0000", 2024-09-11T07:42:20.172 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4456000000000001e-05 2024-09-11T07:42:20.172 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.172 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.172 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:20.172 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.137020+0000", 2024-09-11T07:42:20.172 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5990000000000001e-05 2024-09-11T07:42:20.173 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.173 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.173 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:20.173 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.137027+0000", 2024-09-11T07:42:20.173 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0450000000000003e-06 2024-09-11T07:42:20.173 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:20.173 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:20.173 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:20.173 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.173 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.173 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop_reply(client.25706.0:13706 10.f e352/341)", 2024-09-11T07:42:20.174 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:42:20.137266+0000", 2024-09-11T07:42:20.174 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.00025954800000000002, 2024-09-11T07:42:20.174 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00075062100000000004, 2024-09-11T07:42:20.174 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:20.174 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "queued for pg", 2024-09-11T07:42:20.174 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:20.174 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.174 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:20.174 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.137266+0000", 2024-09-11T07:42:20.174 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:20.174 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.174 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.175 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:20.175 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.137266+0000", 2024-09-11T07:42:20.175 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:20.175 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.175 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.175 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:20.175 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.137265+0000", 2024-09-11T07:42:20.175 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:42:20.175 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.175 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.175 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:20.176 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.137270+0000", 2024-09-11T07:42:20.176 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8810000000000001e-06 2024-09-11T07:42:20.176 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.176 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.176 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:20.176 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.137279+0000", 2024-09-11T07:42:20.176 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.4910000000000005e-06 2024-09-11T07:42:20.176 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.176 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.176 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:20.176 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.137284+0000", 2024-09-11T07:42:20.177 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7400000000000004e-06 2024-09-11T07:42:20.177 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:20.177 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:20.177 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:20.177 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.177 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.177 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop_reply(client.25706.0:13708 10.f e352/341)", 2024-09-11T07:42:20.177 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:42:20.137288+0000", 2024-09-11T07:42:20.177 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.00023704800000000001, 2024-09-11T07:42:20.177 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00077670400000000002, 2024-09-11T07:42:20.177 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:20.178 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "queued for pg", 2024-09-11T07:42:20.178 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:20.178 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.178 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:20.178 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.137288+0000", 2024-09-11T07:42:20.178 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:20.178 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.178 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.178 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:20.178 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.137288+0000", 2024-09-11T07:42:20.178 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:20.178 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.179 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.179 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:20.179 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.137288+0000", 2024-09-11T07:42:20.179 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:42:20.179 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.179 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.179 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:20.179 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.137291+0000", 2024-09-11T07:42:20.179 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.9009999999999998e-06 2024-09-11T07:42:20.179 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.180 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.180 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:20.180 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.137296+0000", 2024-09-11T07:42:20.180 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9180000000000002e-06 2024-09-11T07:42:20.180 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.180 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.180 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:20.180 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.137299+0000", 2024-09-11T07:42:20.180 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6370000000000001e-06 2024-09-11T07:42:20.180 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:20.180 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:20.181 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:20.181 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.181 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.181 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:13726 10.f 10.bc71df4f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-11T07:42:20.181 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:42:20.137410+0000", 2024-09-11T07:42:20.181 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 0.00011560100000000001, 2024-09-11T07:42:20.181 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000701722, 2024-09-11T07:42:20.181 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:20.181 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "queued for pg", 2024-09-11T07:42:20.181 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:20.181 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:42:20.182 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:42:20.182 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 13726 2024-09-11T07:42:20.182 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.182 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:20.182 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.182 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:20.182 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.137410+0000", 2024-09-11T07:42:20.182 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:20.182 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.182 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.182 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:20.183 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.137410+0000", 2024-09-11T07:42:20.183 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:20.183 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.183 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.183 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:20.183 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.137408+0000", 2024-09-11T07:42:20.183 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:42:20.183 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.183 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.183 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:20.183 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.137432+0000", 2024-09-11T07:42:20.183 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4026999999999999e-05 2024-09-11T07:42:20.184 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.184 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.184 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:20.184 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.137449+0000", 2024-09-11T07:42:20.184 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.6541999999999998e-05 2024-09-11T07:42:20.184 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:20.184 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:20.184 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:20.184 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:20.137454+0000", 2024-09-11T07:42:20.184 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8100000000000003e-06 2024-09-11T07:42:20.184 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:20.184 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:20.185 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:20.185 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:20.185 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:42:20.185 INFO:teuthology.orchestra.run.smithi022.stdout: "num_ops": 9 2024-09-11T07:42:20.185 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:42:20.185 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:20.206 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:20.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:20.207 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:20.279 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:42:19 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:42:20.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:19 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:42:20.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:19 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:42:20.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:19 smithi104 ceph-mon[33507]: 2.15 scrub starts 2024-09-11T07:42:20.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:19 smithi104 ceph-mon[33507]: 2.15 scrub ok 2024-09-11T07:42:20.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:19 smithi104 ceph-mon[33507]: 4.5 scrub starts 2024-09-11T07:42:20.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:19 smithi104 ceph-mon[33507]: 4.5 scrub ok 2024-09-11T07:42:20.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:19 smithi104 ceph-mon[33507]: osdmap e352: 8 total, 8 up, 8 in 2024-09-11T07:42:20.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:19 smithi104 ceph-mon[33507]: 4.1d scrub starts 2024-09-11T07:42:20.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:19 smithi104 ceph-mon[33507]: 4.1d scrub ok 2024-09-11T07:42:20.308 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:20.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:20.340 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:20.340 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:20.441 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:20.483 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:20.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:20.484 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:20.585 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:20.618 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:20.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:20.619 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:20.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:42:20 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:42:20] "GET /metrics HTTP/1.1" 200 224656 "" "Prometheus/2.43.0" 2024-09-11T07:42:20.720 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:20.751 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:20.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:20.753 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:20.853 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:20.885 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:20.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:20.886 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:20.950 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 5 16 15606 15590 194.861 148.562 0.00386332 0.00512564 2024-09-11T07:42:20.987 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:20.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:20 smithi022 ceph-mon[29462]: 5.12 scrub starts 2024-09-11T07:42:20.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:20 smithi022 ceph-mon[29462]: 5.12 scrub ok 2024-09-11T07:42:20.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:20 smithi022 ceph-mon[29462]: 5.f deep-scrub starts 2024-09-11T07:42:20.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:20 smithi022 ceph-mon[29462]: 5.f deep-scrub ok 2024-09-11T07:42:20.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:20 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:42:20.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:20 smithi022 ceph-mon[29462]: 4.11 scrub starts 2024-09-11T07:42:20.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:20 smithi022 ceph-mon[29462]: 4.11 scrub ok 2024-09-11T07:42:20.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:20 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:42:20.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:20 smithi022 ceph-mon[29462]: pgmap v739: 148 pgs: 148 active+clean; 654 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 109 MiB/s wr, 1.74k op/s 2024-09-11T07:42:20.999 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:20 smithi022 ceph-mon[37330]: 5.12 scrub starts 2024-09-11T07:42:20.999 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:20 smithi022 ceph-mon[37330]: 5.12 scrub ok 2024-09-11T07:42:21.000 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:20 smithi022 ceph-mon[37330]: 5.f deep-scrub starts 2024-09-11T07:42:21.000 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:20 smithi022 ceph-mon[37330]: 5.f deep-scrub ok 2024-09-11T07:42:21.000 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:20 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:42:21.000 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:20 smithi022 ceph-mon[37330]: 4.11 scrub starts 2024-09-11T07:42:21.000 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:20 smithi022 ceph-mon[37330]: 4.11 scrub ok 2024-09-11T07:42:21.000 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:20 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:42:21.000 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:20 smithi022 ceph-mon[37330]: pgmap v739: 148 pgs: 148 active+clean; 654 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 109 MiB/s wr, 1.74k op/s 2024-09-11T07:42:21.024 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:21.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:21.027 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:21.127 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:21.160 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:21.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:21.161 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:21.262 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:21.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:20 smithi104 ceph-mon[33507]: 5.12 scrub starts 2024-09-11T07:42:21.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:20 smithi104 ceph-mon[33507]: 5.12 scrub ok 2024-09-11T07:42:21.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:20 smithi104 ceph-mon[33507]: 5.f deep-scrub starts 2024-09-11T07:42:21.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:20 smithi104 ceph-mon[33507]: 5.f deep-scrub ok 2024-09-11T07:42:21.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:20 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:42:21.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:20 smithi104 ceph-mon[33507]: 4.11 scrub starts 2024-09-11T07:42:21.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:20 smithi104 ceph-mon[33507]: 4.11 scrub ok 2024-09-11T07:42:21.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:20 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:42:21.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:20 smithi104 ceph-mon[33507]: pgmap v739: 148 pgs: 148 active+clean; 654 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 109 MiB/s wr, 1.74k op/s 2024-09-11T07:42:21.300 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:21.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:21.301 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:21.402 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:21.426 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T07:42:21.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:21.452 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:21.452 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:21.553 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:21.580 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:21.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:21.581 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:21.682 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:21.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:21.711 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:21.711 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:21.803 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:21 smithi022 ceph-mon[29462]: 2.12 scrub starts 2024-09-11T07:42:21.803 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:21 smithi022 ceph-mon[29462]: 2.12 scrub ok 2024-09-11T07:42:21.803 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:21 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:42:21.804 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:21 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:42:21.804 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:21 smithi022 ceph-mon[29462]: osdmap e353: 8 total, 8 up, 8 in 2024-09-11T07:42:21.804 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:21 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:42:21.804 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:21 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:42:21.804 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:21 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:42:21.812 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:21.857 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:21.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:21.858 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:21.951 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 6 16 18033 18017 187.662 151.688 0.0012707 0.00529762 2024-09-11T07:42:21.959 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:21.980 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:21.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:21.981 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:22.082 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:22.123 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:22.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:22.124 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:22.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:21 smithi022 ceph-mon[37330]: 2.12 scrub starts 2024-09-11T07:42:22.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:21 smithi022 ceph-mon[37330]: 2.12 scrub ok 2024-09-11T07:42:22.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:21 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:42:22.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:21 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:42:22.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:21 smithi022 ceph-mon[37330]: osdmap e353: 8 total, 8 up, 8 in 2024-09-11T07:42:22.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:21 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:42:22.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:21 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:42:22.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:21 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:42:22.225 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:22.258 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:22.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:22.259 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:22.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:21 smithi104 ceph-mon[33507]: 2.12 scrub starts 2024-09-11T07:42:22.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:21 smithi104 ceph-mon[33507]: 2.12 scrub ok 2024-09-11T07:42:22.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:21 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:42:22.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:21 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:42:22.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:21 smithi104 ceph-mon[33507]: osdmap e353: 8 total, 8 up, 8 in 2024-09-11T07:42:22.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:21 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:42:22.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:21 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:42:22.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:21 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:42:22.360 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:22.393 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:22.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:22.394 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:22.495 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:22.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:22.522 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:22.522 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:22.623 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:22.655 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:22.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:22.657 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:22.757 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:22.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:22.779 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:22.779 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:22.879 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:22.901 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:22.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:22.902 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:22.951 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 7 16 19941 19925 177.885 119.25 0.00214989 0.00551957 2024-09-11T07:42:23.002 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:23.024 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:23.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:23.025 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:23.125 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:23.157 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:23.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:23.159 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:23.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:22 smithi022 ceph-mon[29462]: 4.19 scrub starts 2024-09-11T07:42:23.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:22 smithi022 ceph-mon[29462]: 4.19 scrub ok 2024-09-11T07:42:23.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:22 smithi022 ceph-mon[29462]: pgmap v741: 148 pgs: 148 active+clean; 654 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 109 MiB/s wr, 1.74k op/s 2024-09-11T07:42:23.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:22 smithi022 ceph-mon[37330]: 4.19 scrub starts 2024-09-11T07:42:23.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:22 smithi022 ceph-mon[37330]: 4.19 scrub ok 2024-09-11T07:42:23.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:22 smithi022 ceph-mon[37330]: pgmap v741: 148 pgs: 148 active+clean; 654 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 109 MiB/s wr, 1.74k op/s 2024-09-11T07:42:23.259 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:23.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:22 smithi104 ceph-mon[33507]: 4.19 scrub starts 2024-09-11T07:42:23.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:22 smithi104 ceph-mon[33507]: 4.19 scrub ok 2024-09-11T07:42:23.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:22 smithi104 ceph-mon[33507]: pgmap v741: 148 pgs: 148 active+clean; 654 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 109 MiB/s wr, 1.74k op/s 2024-09-11T07:42:23.285 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:23.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:23.286 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:23.387 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:23.425 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:23.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:23.426 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:23.527 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:23.561 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:23.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:23.562 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:23.663 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:23.701 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:23.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:23.703 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:23.803 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:23.806 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:42:23.806 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:42:23.806 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-11T07:42:23.807 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T07:42:23.807 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T07:42:23.807 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.3 config set filestore_inject_stall 3 2024-09-11T07:42:23.867 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-11T07:42:23.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:23.875 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:23.875 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:23.951 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 8 16 22188 22172 173.203 140.438 0.00839182 0.00576641 2024-09-11T07:42:23.977 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:24.035 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:24.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:24.036 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:24.137 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:24.160 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:42:24.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:23 smithi022 ceph-mon[29462]: 2.c scrub starts 2024-09-11T07:42:24.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:23 smithi022 ceph-mon[29462]: 2.c scrub ok 2024-09-11T07:42:24.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:23 smithi022 ceph-mon[29462]: 5.2 deep-scrub starts 2024-09-11T07:42:24.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:23 smithi022 ceph-mon[29462]: 5.2 deep-scrub ok 2024-09-11T07:42:24.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:42:24.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:42:24.173 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:23 smithi022 ceph-mon[37330]: 2.c scrub starts 2024-09-11T07:42:24.173 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:23 smithi022 ceph-mon[37330]: 2.c scrub ok 2024-09-11T07:42:24.173 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:23 smithi022 ceph-mon[37330]: 5.2 deep-scrub starts 2024-09-11T07:42:24.173 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:23 smithi022 ceph-mon[37330]: 5.2 deep-scrub ok 2024-09-11T07:42:24.173 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:42:24.173 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:42:24.180 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:42:24 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:42:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:42:24.180 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:23 smithi104 ceph-mon[33507]: 2.c scrub starts 2024-09-11T07:42:24.181 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:23 smithi104 ceph-mon[33507]: 2.c scrub ok 2024-09-11T07:42:24.181 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:23 smithi104 ceph-mon[33507]: 5.2 deep-scrub starts 2024-09-11T07:42:24.181 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:23 smithi104 ceph-mon[33507]: 5.2 deep-scrub ok 2024-09-11T07:42:24.181 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:42:24.181 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:42:24.188 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:24.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:24.191 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:24.292 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:24.326 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:24.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:24.327 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:24.428 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:24.470 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:24.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:24.471 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:24.572 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:24.601 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:24.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:24.602 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:24.703 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:24.724 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:24.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:24.725 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:24.825 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:24.861 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:24.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:24.862 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:24.951 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 9 16 23751 23735 164.811 97.6875 0.00563002 0.00606179 2024-09-11T07:42:24.962 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:24.999 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:25.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:25.000 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:25.101 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:25.137 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:25.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:25.138 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:25.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:24 smithi022 ceph-mon[29462]: 2.1 scrub starts 2024-09-11T07:42:25.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:24 smithi022 ceph-mon[29462]: 2.1 scrub ok 2024-09-11T07:42:25.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:24 smithi022 ceph-mon[29462]: 5.1f scrub starts 2024-09-11T07:42:25.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:24 smithi022 ceph-mon[29462]: 5.1f scrub ok 2024-09-11T07:42:25.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:24 smithi022 ceph-mon[29462]: 5.10 scrub starts 2024-09-11T07:42:25.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:24 smithi022 ceph-mon[29462]: 5.10 scrub ok 2024-09-11T07:42:25.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:24 smithi022 ceph-mon[29462]: pgmap v742: 148 pgs: 148 active+clean; 1.3 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 1.3 KiB/s rd, 174 MiB/s wr, 2.79k op/s 2024-09-11T07:42:25.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:24 smithi022 ceph-mon[37330]: 2.1 scrub starts 2024-09-11T07:42:25.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:24 smithi022 ceph-mon[37330]: 2.1 scrub ok 2024-09-11T07:42:25.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:24 smithi022 ceph-mon[37330]: 5.1f scrub starts 2024-09-11T07:42:25.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:24 smithi022 ceph-mon[37330]: 5.1f scrub ok 2024-09-11T07:42:25.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:24 smithi022 ceph-mon[37330]: 5.10 scrub starts 2024-09-11T07:42:25.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:24 smithi022 ceph-mon[37330]: 5.10 scrub ok 2024-09-11T07:42:25.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:24 smithi022 ceph-mon[37330]: pgmap v742: 148 pgs: 148 active+clean; 1.3 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 1.3 KiB/s rd, 174 MiB/s wr, 2.79k op/s 2024-09-11T07:42:25.239 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:25.279 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:25.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:24 smithi104 ceph-mon[33507]: 2.1 scrub starts 2024-09-11T07:42:25.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:24 smithi104 ceph-mon[33507]: 2.1 scrub ok 2024-09-11T07:42:25.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:24 smithi104 ceph-mon[33507]: 5.1f scrub starts 2024-09-11T07:42:25.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:24 smithi104 ceph-mon[33507]: 5.1f scrub ok 2024-09-11T07:42:25.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:24 smithi104 ceph-mon[33507]: 5.10 scrub starts 2024-09-11T07:42:25.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:24 smithi104 ceph-mon[33507]: 5.10 scrub ok 2024-09-11T07:42:25.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:24 smithi104 ceph-mon[33507]: pgmap v742: 148 pgs: 148 active+clean; 1.3 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 1.3 KiB/s rd, 174 MiB/s wr, 2.79k op/s 2024-09-11T07:42:25.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:25.280 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:25.381 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:25.413 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:25.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:25.414 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:25.514 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:25.539 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:25.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:25.540 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:25.641 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:25.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:25.679 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:25.679 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:25.779 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:25.819 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:25.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:25.820 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:25.921 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:25.951 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 10 16 25949 25933 162.067 137.375 0.00832178 0.0061644 2024-09-11T07:42:25.954 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:25.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:25.956 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:26.056 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:26.097 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:26.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:26.099 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:26.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:25 smithi022 ceph-mon[29462]: 4.b scrub starts 2024-09-11T07:42:26.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:25 smithi022 ceph-mon[29462]: 4.b scrub ok 2024-09-11T07:42:26.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:25 smithi022 ceph-mon[29462]: 5.11 scrub starts 2024-09-11T07:42:26.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:25 smithi022 ceph-mon[29462]: 5.11 scrub ok 2024-09-11T07:42:26.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:25 smithi022 ceph-mon[29462]: 2.2 scrub starts 2024-09-11T07:42:26.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:25 smithi022 ceph-mon[29462]: 2.2 scrub ok 2024-09-11T07:42:26.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:25 smithi022 ceph-mon[29462]: osdmap e354: 8 total, 8 up, 8 in 2024-09-11T07:42:26.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:25 smithi022 ceph-mon[37330]: 4.b scrub starts 2024-09-11T07:42:26.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:25 smithi022 ceph-mon[37330]: 4.b scrub ok 2024-09-11T07:42:26.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:25 smithi022 ceph-mon[37330]: 5.11 scrub starts 2024-09-11T07:42:26.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:25 smithi022 ceph-mon[37330]: 5.11 scrub ok 2024-09-11T07:42:26.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:25 smithi022 ceph-mon[37330]: 2.2 scrub starts 2024-09-11T07:42:26.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:25 smithi022 ceph-mon[37330]: 2.2 scrub ok 2024-09-11T07:42:26.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:25 smithi022 ceph-mon[37330]: osdmap e354: 8 total, 8 up, 8 in 2024-09-11T07:42:26.200 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:26.228 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:26.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:26.229 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:26.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:25 smithi104 ceph-mon[33507]: 4.b scrub starts 2024-09-11T07:42:26.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:25 smithi104 ceph-mon[33507]: 4.b scrub ok 2024-09-11T07:42:26.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:25 smithi104 ceph-mon[33507]: 5.11 scrub starts 2024-09-11T07:42:26.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:25 smithi104 ceph-mon[33507]: 5.11 scrub ok 2024-09-11T07:42:26.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:25 smithi104 ceph-mon[33507]: 2.2 scrub starts 2024-09-11T07:42:26.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:25 smithi104 ceph-mon[33507]: 2.2 scrub ok 2024-09-11T07:42:26.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:25 smithi104 ceph-mon[33507]: osdmap e354: 8 total, 8 up, 8 in 2024-09-11T07:42:26.330 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:26.361 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:26.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:26.362 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:26.463 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:26.492 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:26.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:26.493 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:26.594 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:26.616 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:26.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:26.617 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:26.718 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:26.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:26.741 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:26.741 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:26.842 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:26.866 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:26.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:26.867 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:26.951 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 11 16 27463 27447 155.935 94.625 0.00867305 0.00640772 2024-09-11T07:42:26.967 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:27.006 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:27.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:27.007 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:27.029 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:42:27.030 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:42:27.030 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:42:27.030 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:42:27.030 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:42:27.030 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:42:27.030 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:42:27.030 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:42:27.030 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:42:27.031 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:42:27.031 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:42:27.031 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:42:27.031 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:42:27.031 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:42:27.051 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:42:27.051 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:42:27.068 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:26 smithi022 ceph-mon[29462]: 4.1c scrub starts 2024-09-11T07:42:27.068 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:26 smithi022 ceph-mon[29462]: 4.1c scrub ok 2024-09-11T07:42:27.068 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:26 smithi022 ceph-mon[29462]: 4.6 scrub starts 2024-09-11T07:42:27.068 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:26 smithi022 ceph-mon[29462]: 4.6 scrub ok 2024-09-11T07:42:27.068 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:26 smithi022 ceph-mon[29462]: 2.19 scrub starts 2024-09-11T07:42:27.069 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:26 smithi022 ceph-mon[29462]: 2.19 scrub ok 2024-09-11T07:42:27.069 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:26 smithi022 ceph-mon[29462]: pgmap v744: 148 pgs: 148 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.3 KiB/s rd, 186 MiB/s wr, 2.97k op/s 2024-09-11T07:42:27.069 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:26 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3145079972' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:42:27.069 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:26 smithi022 ceph-mon[37330]: 4.1c scrub starts 2024-09-11T07:42:27.069 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:26 smithi022 ceph-mon[37330]: 4.1c scrub ok 2024-09-11T07:42:27.069 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:26 smithi022 ceph-mon[37330]: 4.6 scrub starts 2024-09-11T07:42:27.069 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:26 smithi022 ceph-mon[37330]: 4.6 scrub ok 2024-09-11T07:42:27.069 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:26 smithi022 ceph-mon[37330]: 2.19 scrub starts 2024-09-11T07:42:27.069 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:26 smithi022 ceph-mon[37330]: 2.19 scrub ok 2024-09-11T07:42:27.069 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:26 smithi022 ceph-mon[37330]: pgmap v744: 148 pgs: 148 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.3 KiB/s rd, 186 MiB/s wr, 2.97k op/s 2024-09-11T07:42:27.070 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:26 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3145079972' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:42:27.108 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:27.145 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:27.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:27.146 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:27.247 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:27.269 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:27.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:27.270 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:27.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:26 smithi104 ceph-mon[33507]: 4.1c scrub starts 2024-09-11T07:42:27.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:26 smithi104 ceph-mon[33507]: 4.1c scrub ok 2024-09-11T07:42:27.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:26 smithi104 ceph-mon[33507]: 4.6 scrub starts 2024-09-11T07:42:27.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:26 smithi104 ceph-mon[33507]: 4.6 scrub ok 2024-09-11T07:42:27.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:26 smithi104 ceph-mon[33507]: 2.19 scrub starts 2024-09-11T07:42:27.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:26 smithi104 ceph-mon[33507]: 2.19 scrub ok 2024-09-11T07:42:27.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:26 smithi104 ceph-mon[33507]: pgmap v744: 148 pgs: 148 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.3 KiB/s rd, 186 MiB/s wr, 2.97k op/s 2024-09-11T07:42:27.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:26 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3145079972' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:42:27.371 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:27.407 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:27.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:27.408 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:27.509 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:27.540 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:27.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:27.541 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:27.642 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:27.684 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:27.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:27.685 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:27.786 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:27.825 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:27.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:27.827 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:27.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:27 smithi022 ceph-mon[29462]: 5.16 scrub starts 2024-09-11T07:42:27.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:27 smithi022 ceph-mon[29462]: 5.16 scrub ok 2024-09-11T07:42:27.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:27 smithi022 ceph-mon[29462]: 2.1e scrub starts 2024-09-11T07:42:27.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:27 smithi022 ceph-mon[29462]: 2.1e scrub ok 2024-09-11T07:42:27.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:27 smithi022 ceph-mon[37330]: 5.16 scrub starts 2024-09-11T07:42:27.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:27 smithi022 ceph-mon[37330]: 5.16 scrub ok 2024-09-11T07:42:27.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:27 smithi022 ceph-mon[37330]: 2.1e scrub starts 2024-09-11T07:42:27.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:27 smithi022 ceph-mon[37330]: 2.1e scrub ok 2024-09-11T07:42:27.928 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:27.951 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 12 16 29656 29640 154.361 137.062 0.00730711 0.00647217 2024-09-11T07:42:27.962 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:27.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:27.963 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:28.064 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:28.100 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:28.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:28.101 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:28.202 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:28.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:28.240 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:28.240 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:28.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:27 smithi104 ceph-mon[33507]: 5.16 scrub starts 2024-09-11T07:42:28.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:27 smithi104 ceph-mon[33507]: 5.16 scrub ok 2024-09-11T07:42:28.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:27 smithi104 ceph-mon[33507]: 2.1e scrub starts 2024-09-11T07:42:28.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:27 smithi104 ceph-mon[33507]: 2.1e scrub ok 2024-09-11T07:42:28.341 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:28.382 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:28.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:28.383 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:28.484 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:28.523 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:28.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:28.524 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:28.625 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:28.667 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:28.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:28.669 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:28.770 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:28.804 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:28.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:28.806 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:28.906 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:28.934 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:42:28.938 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:28.939 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:42:28.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:28.940 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:28.953 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 13 16 31917 31901 153.338 141.312 0.0101226 0.00651562 2024-09-11T07:42:29.040 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:29.073 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:28 smithi022 ceph-mon[29462]: 5.15 scrub starts 2024-09-11T07:42:29.073 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:28 smithi022 ceph-mon[29462]: 5.15 scrub ok 2024-09-11T07:42:29.073 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:28 smithi022 ceph-mon[29462]: pgmap v745: 148 pgs: 148 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 93 MiB/s wr, 1.48k op/s 2024-09-11T07:42:29.073 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:28 smithi022 ceph-mon[29462]: 5.17 scrub starts 2024-09-11T07:42:29.074 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:28 smithi022 ceph-mon[29462]: 5.17 scrub ok 2024-09-11T07:42:29.074 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:28 smithi022 ceph-mon[37330]: 5.15 scrub starts 2024-09-11T07:42:29.074 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:28 smithi022 ceph-mon[37330]: 5.15 scrub ok 2024-09-11T07:42:29.074 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:28 smithi022 ceph-mon[37330]: pgmap v745: 148 pgs: 148 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 93 MiB/s wr, 1.48k op/s 2024-09-11T07:42:29.074 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:28 smithi022 ceph-mon[37330]: 5.17 scrub starts 2024-09-11T07:42:29.074 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:28 smithi022 ceph-mon[37330]: 5.17 scrub ok 2024-09-11T07:42:29.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:29.083 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:29.083 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:29.184 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:29.217 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:29.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:29.218 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:29.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:28 smithi104 ceph-mon[33507]: 5.15 scrub starts 2024-09-11T07:42:29.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:28 smithi104 ceph-mon[33507]: 5.15 scrub ok 2024-09-11T07:42:29.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:28 smithi104 ceph-mon[33507]: pgmap v745: 148 pgs: 148 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 93 MiB/s wr, 1.48k op/s 2024-09-11T07:42:29.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:28 smithi104 ceph-mon[33507]: 5.17 scrub starts 2024-09-11T07:42:29.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:28 smithi104 ceph-mon[33507]: 5.17 scrub ok 2024-09-11T07:42:29.319 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:29.351 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:29.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:29.353 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:29.453 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:29.488 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:29.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:29.489 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:29.590 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:29.613 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:29.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:29.613 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:29.714 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:29.747 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:29.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:29.748 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:29.849 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:29.879 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:29.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:29.879 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:29.953 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 14 16 33867 33851 151.09 121.875 0.00284069 0.00661264 2024-09-11T07:42:29.980 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:30.013 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:30.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:30.014 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:30.114 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:30.150 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:30.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:30.151 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:30.252 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:30.279 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:42:29 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:42:30.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:30 smithi104 ceph-mon[33507]: 5.13 scrub starts 2024-09-11T07:42:30.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:30 smithi104 ceph-mon[33507]: 5.13 scrub ok 2024-09-11T07:42:30.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:30.288 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:30.288 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:30.373 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:30 smithi022 ceph-mon[29462]: 5.13 scrub starts 2024-09-11T07:42:30.373 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:30 smithi022 ceph-mon[29462]: 5.13 scrub ok 2024-09-11T07:42:30.374 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:30 smithi022 ceph-mon[37330]: 5.13 scrub starts 2024-09-11T07:42:30.374 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:30 smithi022 ceph-mon[37330]: 5.13 scrub ok 2024-09-11T07:42:30.389 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:30.420 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:30.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:30.421 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:30.440 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:42:30.522 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:30.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:30.556 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:30.557 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:30.658 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:30.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:42:30 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:42:30] "GET /metrics HTTP/1.1" 200 224801 "" "Prometheus/2.43.0" 2024-09-11T07:42:30.700 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:30.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:30.701 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:30.802 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:30.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:30.838 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:30.838 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:30.939 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:30.953 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 15 16 35612 35596 148.288 109.062 0.00217369 0.00672414 2024-09-11T07:42:30.959 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:30.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:30.960 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:31.061 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:31.082 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:31.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:31.083 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:31.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:31 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:42:31.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:31 smithi022 ceph-mon[29462]: pgmap v746: 148 pgs: 148 active+clean; 2.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 157 MiB/s wr, 2.52k op/s 2024-09-11T07:42:31.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:31 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:42:31.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:31 smithi022 ceph-mon[37330]: pgmap v746: 148 pgs: 148 active+clean; 2.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 157 MiB/s wr, 2.52k op/s 2024-09-11T07:42:31.183 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:31.213 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:31.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:31.214 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:31.315 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:31.355 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:31.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:31.356 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:31.457 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:31.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:31.484 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:31.485 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:31.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:31 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:42:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:31 smithi104 ceph-mon[33507]: pgmap v746: 148 pgs: 148 active+clean; 2.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 157 MiB/s wr, 2.52k op/s 2024-09-11T07:42:31.585 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:31.623 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:31.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:31.624 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:31.725 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:31.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:31.763 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:31.764 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:31.864 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:31.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:31.901 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:31.901 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:31.953 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 16 16 37017 37001 144.508 87.8125 0.00787703 0.00691472 2024-09-11T07:42:32.003 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:32.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:32.039 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:32.039 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:32.140 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:32.173 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:32.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:32.174 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:32.274 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:32.307 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:32.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:32.308 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:32.366 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:32 smithi022 ceph-mon[29462]: 2.13 scrub starts 2024-09-11T07:42:32.367 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:32 smithi022 ceph-mon[29462]: 2.13 scrub ok 2024-09-11T07:42:32.367 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:32 smithi022 ceph-mon[29462]: 5.1 scrub starts 2024-09-11T07:42:32.367 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:32 smithi022 ceph-mon[29462]: 5.1 scrub ok 2024-09-11T07:42:32.367 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:32 smithi022 ceph-mon[37330]: 2.13 scrub starts 2024-09-11T07:42:32.367 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:32 smithi022 ceph-mon[37330]: 2.13 scrub ok 2024-09-11T07:42:32.367 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:32 smithi022 ceph-mon[37330]: 5.1 scrub starts 2024-09-11T07:42:32.367 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:32 smithi022 ceph-mon[37330]: 5.1 scrub ok 2024-09-11T07:42:32.409 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:32.446 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:32.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:32.447 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:32.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:32 smithi104 ceph-mon[33507]: 2.13 scrub starts 2024-09-11T07:42:32.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:32 smithi104 ceph-mon[33507]: 2.13 scrub ok 2024-09-11T07:42:32.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:32 smithi104 ceph-mon[33507]: 5.1 scrub starts 2024-09-11T07:42:32.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:32 smithi104 ceph-mon[33507]: 5.1 scrub ok 2024-09-11T07:42:32.548 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:32.583 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:32.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:32.585 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:32.686 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:32.722 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:32.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:32.723 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:32.824 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:32.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:32.861 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:32.861 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:32.953 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 17 16 39137 39121 143.801 132.5 0.00559227 0.00694867 2024-09-11T07:42:32.962 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:32.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:32.996 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:32.996 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:33.097 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:33.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:33.150 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:33.150 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:33.251 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:33.255 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:33 smithi022 ceph-mon[29462]: 4.16 scrub starts 2024-09-11T07:42:33.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:33 smithi022 ceph-mon[29462]: 4.16 scrub ok 2024-09-11T07:42:33.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:33 smithi022 ceph-mon[29462]: pgmap v747: 148 pgs: 148 active+clean; 2.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 135 MiB/s wr, 2.16k op/s 2024-09-11T07:42:33.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:33 smithi022 ceph-mon[29462]: 5.1d deep-scrub starts 2024-09-11T07:42:33.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:33 smithi022 ceph-mon[29462]: 5.1d deep-scrub ok 2024-09-11T07:42:33.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:33 smithi022 ceph-mon[29462]: 4.c scrub starts 2024-09-11T07:42:33.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:33 smithi022 ceph-mon[29462]: 4.c scrub ok 2024-09-11T07:42:33.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:33 smithi022 ceph-mon[29462]: 6.13 deep-scrub starts 2024-09-11T07:42:33.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:33 smithi022 ceph-mon[29462]: 6.13 deep-scrub ok 2024-09-11T07:42:33.256 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:33 smithi022 ceph-mon[37330]: 4.16 scrub starts 2024-09-11T07:42:33.256 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:33 smithi022 ceph-mon[37330]: 4.16 scrub ok 2024-09-11T07:42:33.257 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:33 smithi022 ceph-mon[37330]: pgmap v747: 148 pgs: 148 active+clean; 2.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 135 MiB/s wr, 2.16k op/s 2024-09-11T07:42:33.257 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:33 smithi022 ceph-mon[37330]: 5.1d deep-scrub starts 2024-09-11T07:42:33.257 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:33 smithi022 ceph-mon[37330]: 5.1d deep-scrub ok 2024-09-11T07:42:33.257 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:33 smithi022 ceph-mon[37330]: 4.c scrub starts 2024-09-11T07:42:33.257 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:33 smithi022 ceph-mon[37330]: 4.c scrub ok 2024-09-11T07:42:33.257 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:33 smithi022 ceph-mon[37330]: 6.13 deep-scrub starts 2024-09-11T07:42:33.257 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:33 smithi022 ceph-mon[37330]: 6.13 deep-scrub ok 2024-09-11T07:42:33.291 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:33.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:33.292 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:33.392 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:33.429 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:33.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:33.430 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:33.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:33 smithi104 ceph-mon[33507]: 4.16 scrub starts 2024-09-11T07:42:33.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:33 smithi104 ceph-mon[33507]: 4.16 scrub ok 2024-09-11T07:42:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:33 smithi104 ceph-mon[33507]: pgmap v747: 148 pgs: 148 active+clean; 2.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 135 MiB/s wr, 2.16k op/s 2024-09-11T07:42:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:33 smithi104 ceph-mon[33507]: 5.1d deep-scrub starts 2024-09-11T07:42:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:33 smithi104 ceph-mon[33507]: 5.1d deep-scrub ok 2024-09-11T07:42:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:33 smithi104 ceph-mon[33507]: 4.c scrub starts 2024-09-11T07:42:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:33 smithi104 ceph-mon[33507]: 4.c scrub ok 2024-09-11T07:42:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:33 smithi104 ceph-mon[33507]: 6.13 deep-scrub starts 2024-09-11T07:42:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:33 smithi104 ceph-mon[33507]: 6.13 deep-scrub ok 2024-09-11T07:42:33.530 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:33.569 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:33.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:33.570 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:33.671 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:33.703 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:33.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:33.704 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:33.804 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:33.834 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:33.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:33.836 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:33.937 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:33.953 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 18 16 40892 40876 141.905 109.688 0.00502771 0.0070409 2024-09-11T07:42:33.972 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:33.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:33.974 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:34.074 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:34.100 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:34.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:34.101 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:34.201 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:34.236 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:34.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:34.237 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:34.338 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:34.376 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:34.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:34.381 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:34.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:34 smithi022 ceph-mon[29462]: 6.15 scrub starts 2024-09-11T07:42:34.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:34 smithi022 ceph-mon[29462]: 6.15 scrub ok 2024-09-11T07:42:34.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:34 smithi022 ceph-mon[29462]: 2.4 scrub starts 2024-09-11T07:42:34.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:34 smithi022 ceph-mon[29462]: 2.4 scrub ok 2024-09-11T07:42:34.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:34 smithi022 ceph-mon[29462]: 6.1e scrub starts 2024-09-11T07:42:34.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:34 smithi022 ceph-mon[29462]: 6.1e scrub ok 2024-09-11T07:42:34.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:34 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/930466067' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:42:34.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:34 smithi022 ceph-mon[29462]: from='client.25718 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:42:34.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:34 smithi022 ceph-mon[37330]: 6.15 scrub starts 2024-09-11T07:42:34.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:34 smithi022 ceph-mon[37330]: 6.15 scrub ok 2024-09-11T07:42:34.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:34 smithi022 ceph-mon[37330]: 2.4 scrub starts 2024-09-11T07:42:34.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:34 smithi022 ceph-mon[37330]: 2.4 scrub ok 2024-09-11T07:42:34.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:34 smithi022 ceph-mon[37330]: 6.1e scrub starts 2024-09-11T07:42:34.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:34 smithi022 ceph-mon[37330]: 6.1e scrub ok 2024-09-11T07:42:34.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:34 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/930466067' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:42:34.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:34 smithi022 ceph-mon[37330]: from='client.25718 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:42:34.482 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:34.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:34.519 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:34.519 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:34.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:42:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:42:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:42:34.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:34 smithi104 ceph-mon[33507]: 6.15 scrub starts 2024-09-11T07:42:34.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:34 smithi104 ceph-mon[33507]: 6.15 scrub ok 2024-09-11T07:42:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:34 smithi104 ceph-mon[33507]: 2.4 scrub starts 2024-09-11T07:42:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:34 smithi104 ceph-mon[33507]: 2.4 scrub ok 2024-09-11T07:42:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:34 smithi104 ceph-mon[33507]: 6.1e scrub starts 2024-09-11T07:42:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:34 smithi104 ceph-mon[33507]: 6.1e scrub ok 2024-09-11T07:42:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:34 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/930466067' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:42:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:34 smithi104 ceph-mon[33507]: from='client.25718 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:42:34.619 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:34.655 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:34.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:34.656 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:34.757 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:34.794 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:34.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:34.795 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:34.896 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:34.943 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:34.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:34.944 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:34.953 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 19 16 42839 42823 140.84 121.688 0.0119797 0.00709455 2024-09-11T07:42:35.045 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:35.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:35.079 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:35.079 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:35.084 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is set 2024-09-11T07:42:35.180 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:35.219 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:35.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:35.220 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:35.321 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:35.353 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:35.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:35.354 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:35.391 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:35 smithi022 ceph-mon[29462]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:42:35.391 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:35 smithi022 ceph-mon[29462]: from='client.25718 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:42:35.391 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:35 smithi022 ceph-mon[29462]: osdmap e355: 8 total, 8 up, 8 in 2024-09-11T07:42:35.392 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:35 smithi022 ceph-mon[29462]: pgmap v749: 148 pgs: 148 active+clean; 2.4 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 128 MiB/s wr, 2.05k op/s 2024-09-11T07:42:35.392 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:35 smithi022 ceph-mon[37330]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:42:35.392 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:35 smithi022 ceph-mon[37330]: from='client.25718 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:42:35.392 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:35 smithi022 ceph-mon[37330]: osdmap e355: 8 total, 8 up, 8 in 2024-09-11T07:42:35.392 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:35 smithi022 ceph-mon[37330]: pgmap v749: 148 pgs: 148 active+clean; 2.4 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 128 MiB/s wr, 2.05k op/s 2024-09-11T07:42:35.455 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:35.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:35.494 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:35.495 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:35.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:35 smithi104 ceph-mon[33507]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:42:35.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:35 smithi104 ceph-mon[33507]: from='client.25718 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:42:35.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:35 smithi104 ceph-mon[33507]: osdmap e355: 8 total, 8 up, 8 in 2024-09-11T07:42:35.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:35 smithi104 ceph-mon[33507]: pgmap v749: 148 pgs: 148 active+clean; 2.4 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 128 MiB/s wr, 2.05k op/s 2024-09-11T07:42:35.595 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:35.631 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:35.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:35.633 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:35.733 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:35.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:35.778 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:35.778 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:35.879 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:35.923 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:35.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:35.924 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:35.954 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:2024-09-11T07:42:35.953284+0000 min lat: 0.00106648 max lat: 0.324223 avg lat: 0.00714725 2024-09-11T07:42:35.954 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T07:42:35.954 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 20 16 44767 44751 139.823 120.5 0.0128485 0.00714725 2024-09-11T07:42:36.024 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:36.064 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:36.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:36.065 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:36.166 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:36.189 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:42:36.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:36.225 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:36.226 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:36.326 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:36.356 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:36.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:36.357 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:36.458 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:36.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:36.492 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:36.492 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:36.593 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:36.626 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:36.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:36.627 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:36.727 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:36.745 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:36 smithi022 ceph-mon[29462]: pgmap v750: 148 pgs: 148 active+clean; 2.5 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 117 MiB/s wr, 1.87k op/s 2024-09-11T07:42:36.746 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:36 smithi022 ceph-mon[37330]: pgmap v750: 148 pgs: 148 active+clean; 2.5 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 117 MiB/s wr, 1.87k op/s 2024-09-11T07:42:36.765 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:36.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:36.766 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:36.866 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:36.895 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:36.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:36.896 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:36.954 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 21 16 46491 46475 138.295 107.75 0.00837066 0.00722593 2024-09-11T07:42:36.997 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:37.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:36 smithi104 ceph-mon[33507]: pgmap v750: 148 pgs: 148 active+clean; 2.5 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 117 MiB/s wr, 1.87k op/s 2024-09-11T07:42:37.030 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:37.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:37.031 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:37.132 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:37.172 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:37.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:37.173 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:37.181 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:42:37.182 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "" 2024-09-11T07:42:37.182 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:42:37.267 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:42:37.267 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:42:37.275 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:37.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:37.309 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:37.309 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:37.410 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:37.445 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:37.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:37.446 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:37.547 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:37.579 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:37.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:37.580 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:37.681 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:37.720 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:37.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:37.721 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:37.750 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T07:42:37.823 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:37.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:37.853 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:37.853 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:37.954 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 22 16 48193 48177 136.844 106.375 0.01007 0.00730161 2024-09-11T07:42:37.954 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:38.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:38.007 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:38.007 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:38.108 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:38.131 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:38.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:38.132 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:38.233 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:38.271 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:38.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:38.272 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:38.373 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:38.409 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:38.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:38.410 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:38.510 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:38.532 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:38.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:38.533 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:38.634 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:38.655 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:38.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:38.656 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:38.756 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:38.795 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:38.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:38.796 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:38.897 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:38.929 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:38.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:38.930 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:38.953 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 23 16 49540 49524 134.555 84.1875 0.00668578 0.00742707 2024-09-11T07:42:39.031 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:39.059 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:39.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:39.061 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:39.161 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:39.196 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:39.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:39.197 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:39.298 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:39.334 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:39.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:39.335 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:39.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:39 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:42:39.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:39 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:42:39.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:39 smithi022 ceph-mon[37330]: pgmap v751: 148 pgs: 148 active+clean; 2.5 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 117 MiB/s wr, 1.87k op/s 2024-09-11T07:42:39.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:39 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:42:39.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:39 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:42:39.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:39 smithi022 ceph-mon[29462]: pgmap v751: 148 pgs: 148 active+clean; 2.5 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 117 MiB/s wr, 1.87k op/s 2024-09-11T07:42:39.436 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:39.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:39.459 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:39.459 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:39.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:39 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:42:39.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:39 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:42:39.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:39 smithi104 ceph-mon[33507]: pgmap v751: 148 pgs: 148 active+clean; 2.5 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 117 MiB/s wr, 1.87k op/s 2024-09-11T07:42:39.560 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:39.592 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:39.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:39.593 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:39.693 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:39.731 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:39.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:39.732 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:39.833 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:39.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:39.869 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:39.869 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:39.954 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 24 16 50703 50687 131.977 72.6875 0.00217847 0.00754299 2024-09-11T07:42:39.970 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:40.001 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:40.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:40.002 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:40.103 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:40.143 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:40.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:40.144 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:40.245 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:40.271 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:40.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:40.272 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:40.279 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:42:39 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:42:40.373 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:40.398 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:40.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:40.399 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:40.500 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:40.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:40.525 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:40.525 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:40.626 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:40.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:40.664 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:40.664 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:40.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:42:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:42:40] "GET /metrics HTTP/1.1" 200 224897 "" "Prometheus/2.43.0" 2024-09-11T07:42:40.765 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:40.807 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:40.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:40.808 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:40.909 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:40.945 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:40.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:40.946 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:40.954 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 25 16 52113 52097 130.222 88.125 0.0197956 0.00767389 2024-09-11T07:42:41.047 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:41.084 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:41.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:41.085 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:41.186 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:41.219 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:41.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:41.220 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:41.321 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:41.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:41.358 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:41.358 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:41.459 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:41.492 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:41.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:41.493 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:41.528 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-11T07:42:41.594 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:41.611 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:41 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:42:41.611 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:41 smithi022 ceph-mon[29462]: pgmap v752: 148 pgs: 148 active+clean; 3.0 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 110 MiB/s wr, 1.75k op/s 2024-09-11T07:42:41.611 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:41 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:42:41.611 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:41 smithi022 ceph-mon[37330]: pgmap v752: 148 pgs: 148 active+clean; 3.0 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 110 MiB/s wr, 1.75k op/s 2024-09-11T07:42:41.622 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:41.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:41.623 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:41.724 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:41.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:41.774 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:41.775 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:41.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:41 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:42:41.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:41 smithi104 ceph-mon[33507]: pgmap v752: 148 pgs: 148 active+clean; 3.0 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 110 MiB/s wr, 1.75k op/s 2024-09-11T07:42:41.876 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:41.929 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:41.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:41.930 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:41.954 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 26 16 53740 53724 129.125 101.688 0.00741507 0.0077398 2024-09-11T07:42:42.031 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:42.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:42.074 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:42.074 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:42.175 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:42.203 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:42.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:42.204 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:42.305 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:42.339 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:42.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:42.340 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:42.440 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:42.474 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:42.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:42.476 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:42.576 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:42.612 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:42.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:42.613 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:42.714 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:42.756 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:42.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:42.758 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:42.856 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:42:42.859 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:42.925 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:42.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:42.926 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:42.954 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 27 16 55367 55351 128.108 101.688 0.00507643 0.00780117 2024-09-11T07:42:43.027 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:43.057 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:43.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:43.058 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:43.159 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:43.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:43.198 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:43.198 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:43.299 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:43.325 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:43.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:43.326 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:43.427 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:43.462 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:43.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:43.463 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:43.526 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:43 smithi022 ceph-mon[29462]: pgmap v753: 148 pgs: 148 active+clean; 3.0 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 110 MiB/s wr, 1.75k op/s 2024-09-11T07:42:43.527 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:43 smithi022 ceph-mon[37330]: pgmap v753: 148 pgs: 148 active+clean; 3.0 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 110 MiB/s wr, 1.75k op/s 2024-09-11T07:42:43.564 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:43.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:43.608 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:43.608 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:43.709 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:43.740 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:43.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:43.741 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:43.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:43 smithi104 ceph-mon[33507]: pgmap v753: 148 pgs: 148 active+clean; 3.0 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 110 MiB/s wr, 1.75k op/s 2024-09-11T07:42:43.842 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:43.874 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:43.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:43.875 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:43.954 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 28 16 56879 56863 126.908 94.5 0.00368669 0.00787456 2024-09-11T07:42:43.976 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:44.016 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:44.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:44.017 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:44.118 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:44.149 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:44.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:44.149 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:44.225 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:42:44.251 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:44.276 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:44.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:44.277 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:44.293 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:42:44.293 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:42:44.326 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:42:44.327 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:42:44.357 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:42:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:42:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:42:44.378 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:44.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:44.401 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:44.401 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:44.432 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:42:44.432 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:42:44.465 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:42:44.465 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:42:44.496 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:42:44.497 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:42:44.502 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:44.527 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:42:44.527 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:42:44.544 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:44.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:44.545 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:44.558 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:42:44.558 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:42:44.589 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:42:44.590 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:42:44.646 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:44.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:44 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/307679393' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:42:44.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:44 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/307679393' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:42:44.678 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:44.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:44.679 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:44.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:44 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/307679393' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:42:44.781 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:44.801 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:44.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:44.802 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:44.903 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:44.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:44.939 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:44.939 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:44.954 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 29 16 58184 58168 125.344 81.5625 0.0137817 0.00797278 2024-09-11T07:42:45.040 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:45.078 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:45.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:45.079 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:45.180 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:45.210 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:45.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:45.211 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:45.312 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:45.345 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:45.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:45.346 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:45.446 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:45.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:45.479 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:45.479 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:45.580 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:45.620 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:45.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:45.621 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:45.659 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:45 smithi022 ceph-mon[29462]: pgmap v754: 148 pgs: 148 active+clean; 3.4 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1003 B/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-11T07:42:45.660 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:45 smithi022 ceph-mon[37330]: pgmap v754: 148 pgs: 148 active+clean; 3.4 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1003 B/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-11T07:42:45.722 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:45.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:45.745 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:45.745 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:45.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:45 smithi104 ceph-mon[33507]: pgmap v754: 148 pgs: 148 active+clean; 3.4 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1003 B/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-11T07:42:45.846 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:45.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:45.874 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:45.874 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:45.955 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 30 16 59404 59388 123.707 76.25 0.00299202 0.00807715 2024-09-11T07:42:45.975 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:46.006 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:46.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:46.007 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:46.107 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:46.143 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:46.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:46.145 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:46.246 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:46.278 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:46.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:46.279 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:46.380 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:46.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:46.415 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:46.415 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:46.516 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:46.551 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:46.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:46.552 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:46.652 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:46.698 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:46.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:46.700 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:46.801 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:46.837 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:46.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:46.838 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:46.939 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:46.955 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 31 16 61014 60998 122.959 100.625 0.0080792 0.00812689 2024-09-11T07:42:46.978 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:46.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:46.979 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:47.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:46 smithi104 ceph-mon[33507]: pgmap v755: 148 pgs: 148 active+clean; 3.5 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 89 MiB/s wr, 1.42k op/s 2024-09-11T07:42:47.037 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:46 smithi022 ceph-mon[29462]: pgmap v755: 148 pgs: 148 active+clean; 3.5 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 89 MiB/s wr, 1.42k op/s 2024-09-11T07:42:47.037 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:46 smithi022 ceph-mon[37330]: pgmap v755: 148 pgs: 148 active+clean; 3.5 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 89 MiB/s wr, 1.42k op/s 2024-09-11T07:42:47.080 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:47.114 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:47.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:47.115 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:47.215 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:47.236 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:47.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:47.237 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:47.338 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:47.360 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:47.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:47.361 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:47.462 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:47.502 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:47.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:47.504 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:47.604 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:47.643 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:47.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:47.644 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:47.745 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:47.778 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:47.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:47.779 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:47.880 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:47.912 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:42:47.912 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:42:47.912 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-11T07:42:47.912 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-11T07:42:47.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:47.916 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:47.917 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:47.955 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 32 15 62224 62209 121.482 75.6875 0.0175109 0.00822665 2024-09-11T07:42:48.018 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:48.051 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:48.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:48.052 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:48.153 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:48.196 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:48.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:48.198 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:48.298 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:48.329 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:48.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:48.330 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:48.431 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:48.463 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:48.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:48.464 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:48.565 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:48.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:48.604 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:48.604 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:48.704 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:48.739 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:48.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:48.740 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:48.841 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:48.877 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:48.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:48.878 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:48.955 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 33 16 63769 63753 120.724 96.5 0.00509309 0.00827819 2024-09-11T07:42:48.979 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:49.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:49.021 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:49.021 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:49.122 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:49.150 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:49.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:49.151 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:49.252 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:49.285 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:49.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:49.286 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:49.387 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:49.414 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:49.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:49.415 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:49.515 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:49.557 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:49.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:49.558 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:49.613 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:49 smithi022 ceph-mon[29462]: pgmap v756: 148 pgs: 148 active+clean; 3.5 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-11T07:42:49.614 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:49 smithi022 ceph-mon[37330]: pgmap v756: 148 pgs: 148 active+clean; 3.5 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-11T07:42:49.658 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:49.694 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:49.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:49.695 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:49.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:49 smithi104 ceph-mon[33507]: pgmap v756: 148 pgs: 148 active+clean; 3.5 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-11T07:42:49.795 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:49.826 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:49.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:49.827 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:49.927 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:49.948 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:49.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:49.949 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:49.955 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 34 16 65183 65167 119.773 88.375 0.0128235 0.00833647 2024-09-11T07:42:50.050 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:50.093 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:50.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:50.094 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:50.195 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:50.236 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:50.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:50.237 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:50.279 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:42:49 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:42:50.338 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:50.372 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:50.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:50.376 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:50.477 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:50.497 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:50.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:50.498 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:50.599 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:50.603 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:42:50.603 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 20, 2024-09-11T07:42:50.603 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 600, 2024-09-11T07:42:50.603 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:42:50.603 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.604 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:85936 9.1 9:848eddcb:::benchmark_data_smithi022_133943_object85935:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e259)", 2024-09-11T07:42:50.604 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:26.388789+0000", 2024-09-11T07:42:50.604 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 263.58146551099998, 2024-09-11T07:42:50.604 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.97977777300000002, 2024-09-11T07:42:50.604 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:50.604 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:42:50.604 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:50.604 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:50.604 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:50.604 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 85936 2024-09-11T07:42:50.604 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.604 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:50.604 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.604 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:50.605 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.388789+0000", 2024-09-11T07:42:50.605 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.605 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.605 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.605 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:50.605 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.388789+0000", 2024-09-11T07:42:50.605 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.605 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.605 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.605 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:50.605 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.388786+0000", 2024-09-11T07:42:50.605 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:42:50.605 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.606 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.606 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:50.606 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.388884+0000", 2024-09-11T07:42:50.606 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.7833999999999999e-05 2024-09-11T07:42:50.606 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.606 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.606 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:50.606 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.388897+0000", 2024-09-11T07:42:50.606 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2887999999999999e-05 2024-09-11T07:42:50.606 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.606 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.606 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:50.606 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.388901+0000", 2024-09-11T07:42:50.606 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5759999999999999e-06 2024-09-11T07:42:50.606 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.607 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.607 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.607 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.582158+0000", 2024-09-11T07:42:50.607 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.19325661499999999 2024-09-11T07:42:50.607 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.607 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.607 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.607 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.582171+0000", 2024-09-11T07:42:50.607 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3159000000000001e-05 2024-09-11T07:42:50.607 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.607 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.607 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.607 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.368040+0000", 2024-09-11T07:42:50.607 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.78586916600000001 2024-09-11T07:42:50.608 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.608 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.608 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:50.608 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.368567+0000", 2024-09-11T07:42:50.608 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00052641600000000004 2024-09-11T07:42:50.608 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.608 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:50.608 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.608 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.608 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.608 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:85940 9.12 9:4bfb6d0e:::benchmark_data_smithi022_133943_object85939:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e259)", 2024-09-11T07:42:50.608 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:26.400764+0000", 2024-09-11T07:42:50.608 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 263.56949080300001, 2024-09-11T07:42:50.608 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.97914990700000004, 2024-09-11T07:42:50.608 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:50.609 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:42:50.609 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:50.609 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:50.609 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:50.609 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 85940 2024-09-11T07:42:50.609 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.609 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:50.609 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.609 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:50.609 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.400764+0000", 2024-09-11T07:42:50.609 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.609 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.609 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.610 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:50.610 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.400764+0000", 2024-09-11T07:42:50.610 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.610 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.610 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.610 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:50.610 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.400761+0000", 2024-09-11T07:42:50.610 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:42:50.610 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.610 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.610 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:50.610 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.400814+0000", 2024-09-11T07:42:50.610 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3137999999999998e-05 2024-09-11T07:42:50.610 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.610 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.611 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:50.611 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.400834+0000", 2024-09-11T07:42:50.611 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0551999999999999e-05 2024-09-11T07:42:50.611 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.611 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.611 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:50.611 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.400845+0000", 2024-09-11T07:42:50.611 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0105000000000001e-05 2024-09-11T07:42:50.611 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.611 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.611 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.611 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.617276+0000", 2024-09-11T07:42:50.611 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.21643145 2024-09-11T07:42:50.611 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.611 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.611 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.612 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.617284+0000", 2024-09-11T07:42:50.612 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5020000000000003e-06 2024-09-11T07:42:50.612 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.612 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.612 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.612 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.379482+0000", 2024-09-11T07:42:50.612 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.76219837300000004 2024-09-11T07:42:50.612 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.612 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.612 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:50.612 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.379913+0000", 2024-09-11T07:42:50.612 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00043153499999999998 2024-09-11T07:42:50.612 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.612 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:50.612 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.613 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.613 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.613 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87210 9.19 9.2c8debd9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:42:50.613 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:28.354454+0000", 2024-09-11T07:42:50.613 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 261.61580080099998, 2024-09-11T07:42:50.613 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.98994377300000003, 2024-09-11T07:42:50.613 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:50.613 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:42:50.613 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:50.613 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:50.613 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:50.613 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87210 2024-09-11T07:42:50.613 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.613 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:50.614 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.614 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:50.614 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:28.354454+0000", 2024-09-11T07:42:50.614 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.614 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.614 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.614 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:50.614 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:28.354454+0000", 2024-09-11T07:42:50.614 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.614 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.614 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.614 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:50.614 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:28.354451+0000", 2024-09-11T07:42:50.614 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:42:50.615 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.615 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.615 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:50.615 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:28.354476+0000", 2024-09-11T07:42:50.615 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5184000000000001e-05 2024-09-11T07:42:50.615 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.615 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.615 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:50.615 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:28.354488+0000", 2024-09-11T07:42:50.615 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1610000000000001e-05 2024-09-11T07:42:50.615 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.615 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.615 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:50.615 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:28.354494+0000", 2024-09-11T07:42:50.615 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3400000000000003e-06 2024-09-11T07:42:50.615 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.616 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.616 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.616 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:28.354574+0000", 2024-09-11T07:42:50.616 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.9917000000000001e-05 2024-09-11T07:42:50.616 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.616 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.616 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.616 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:28.354581+0000", 2024-09-11T07:42:50.616 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.8850000000000001e-06 2024-09-11T07:42:50.616 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.616 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.616 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.616 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.344385+0000", 2024-09-11T07:42:50.616 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.98980429199999997 2024-09-11T07:42:50.617 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.617 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.617 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:50.617 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.344397+0000", 2024-09-11T07:42:50.617 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2126999999999999e-05 2024-09-11T07:42:50.617 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.617 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:50.617 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.617 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.617 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.617 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87225 9.19 9.404aa5d9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:42:50.617 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:28.363028+0000", 2024-09-11T07:42:50.617 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 261.60722679000003, 2024-09-11T07:42:50.618 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.98140373199999997, 2024-09-11T07:42:50.618 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:50.618 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:42:50.618 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:50.618 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:50.618 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:50.618 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87225 2024-09-11T07:42:50.618 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.618 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:50.618 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.618 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:50.618 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:28.363028+0000", 2024-09-11T07:42:50.618 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.618 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.618 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.618 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:50.619 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:28.363028+0000", 2024-09-11T07:42:50.619 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.619 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.619 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.619 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:50.619 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:28.363027+0000", 2024-09-11T07:42:50.619 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:42:50.619 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.619 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.619 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:50.619 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:28.363049+0000", 2024-09-11T07:42:50.619 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2416999999999999e-05 2024-09-11T07:42:50.619 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.619 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.620 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:50.620 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:28.363058+0000", 2024-09-11T07:42:50.620 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.2379999999999997e-06 2024-09-11T07:42:50.620 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.620 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.620 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:50.620 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:28.363062+0000", 2024-09-11T07:42:50.620 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2189999999999998e-06 2024-09-11T07:42:50.620 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.620 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.620 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.620 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:28.363374+0000", 2024-09-11T07:42:50.620 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00031244699999999999 2024-09-11T07:42:50.621 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.621 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.621 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.621 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:28.363381+0000", 2024-09-11T07:42:50.621 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1199999999999996e-06 2024-09-11T07:42:50.621 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.621 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.621 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.621 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.344422+0000", 2024-09-11T07:42:50.621 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.98104041600000003 2024-09-11T07:42:50.621 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.621 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.621 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:50.621 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.344431+0000", 2024-09-11T07:42:50.621 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.5289999999999997e-06 2024-09-11T07:42:50.622 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.622 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:50.622 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.622 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.622 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.622 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:213692 9.18 9:1bf9b530:::benchmark_data_smithi022_133943_object87101:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:42:50.622 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:38.520588+0000", 2024-09-11T07:42:50.622 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 131.44966588, 2024-09-11T07:42:50.622 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1203804499999999, 2024-09-11T07:42:50.622 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:50.622 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:42:50.622 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:50.622 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:50.622 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:50.623 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213692 2024-09-11T07:42:50.623 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.623 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:50.623 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.623 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:50.623 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.520588+0000", 2024-09-11T07:42:50.623 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.623 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.623 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.623 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:50.623 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.520588+0000", 2024-09-11T07:42:50.623 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.623 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.623 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.623 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:50.623 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.520586+0000", 2024-09-11T07:42:50.624 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:42:50.624 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.624 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.624 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:50.624 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.520594+0000", 2024-09-11T07:42:50.624 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.9899999999999997e-06 2024-09-11T07:42:50.624 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.624 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.624 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:50.624 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.520605+0000", 2024-09-11T07:42:50.624 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0292000000000001e-05 2024-09-11T07:42:50.624 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.624 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.624 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:50.625 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.520610+0000", 2024-09-11T07:42:50.625 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3700000000000003e-06 2024-09-11T07:42:50.625 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.625 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.625 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.625 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.523617+0000", 2024-09-11T07:42:50.625 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0030071709999999999 2024-09-11T07:42:50.625 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.625 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.625 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op must wait for map", 2024-09-11T07:42:50.625 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.523623+0000", 2024-09-11T07:42:50.625 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3690000000000003e-06 2024-09-11T07:42:50.625 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.625 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.625 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.626 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.526958+0000", 2024-09-11T07:42:50.626 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0033348179999999998 2024-09-11T07:42:50.626 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.626 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.626 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.626 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.526967+0000", 2024-09-11T07:42:50.626 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3000000000000002e-06 2024-09-11T07:42:50.626 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.626 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.626 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.626 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.515377+0000", 2024-09-11T07:42:50.626 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.98841051400000002 2024-09-11T07:42:50.626 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.626 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.626 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.626 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.515389+0000", 2024-09-11T07:42:50.627 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1469e-05 2024-09-11T07:42:50.627 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.627 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.627 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.627 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.640883+0000", 2024-09-11T07:42:50.627 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.125494005 2024-09-11T07:42:50.627 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.627 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.627 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:50.627 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.640969+0000", 2024-09-11T07:42:50.627 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.6353999999999993e-05 2024-09-11T07:42:50.627 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.627 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:50.627 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.627 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.628 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.628 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:213706 9.18 9:1b47dd8a:::benchmark_data_smithi022_133943_object87115:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:42:50.628 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:38.520616+0000", 2024-09-11T07:42:50.628 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 131.449638472, 2024-09-11T07:42:50.628 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.120655272, 2024-09-11T07:42:50.628 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:50.628 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:42:50.628 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:50.628 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:50.628 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:50.628 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213706 2024-09-11T07:42:50.628 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.628 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:50.629 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.629 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:50.629 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.520616+0000", 2024-09-11T07:42:50.629 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.629 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.629 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.629 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:50.629 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.520616+0000", 2024-09-11T07:42:50.629 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.629 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.629 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.629 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:50.629 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.520615+0000", 2024-09-11T07:42:50.630 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:42:50.630 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.630 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.630 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:50.630 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.520619+0000", 2024-09-11T07:42:50.630 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6320000000000001e-06 2024-09-11T07:42:50.630 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.630 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.630 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:50.630 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.520626+0000", 2024-09-11T07:42:50.630 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9670000000000004e-06 2024-09-11T07:42:50.630 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.630 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.630 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:50.631 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.520630+0000", 2024-09-11T07:42:50.631 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9530000000000001e-06 2024-09-11T07:42:50.631 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.631 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.631 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.631 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.523648+0000", 2024-09-11T07:42:50.631 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0030183860000000001 2024-09-11T07:42:50.631 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.631 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.631 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting_for_map not empty", 2024-09-11T07:42:50.631 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.523653+0000", 2024-09-11T07:42:50.631 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6650000000000002e-06 2024-09-11T07:42:50.631 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.631 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.631 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.632 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.527883+0000", 2024-09-11T07:42:50.632 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0042298759999999996 2024-09-11T07:42:50.632 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.632 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.632 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.632 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.527891+0000", 2024-09-11T07:42:50.632 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.1370000000000002e-06 2024-09-11T07:42:50.632 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.632 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.632 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.632 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.515440+0000", 2024-09-11T07:42:50.632 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.98754916299999995 2024-09-11T07:42:50.632 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.633 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.633 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.633 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.515447+0000", 2024-09-11T07:42:50.633 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5020000000000003e-06 2024-09-11T07:42:50.633 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.633 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.633 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.633 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.641221+0000", 2024-09-11T07:42:50.633 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.12577382 2024-09-11T07:42:50.633 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.633 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.633 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:50.633 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.641271+0000", 2024-09-11T07:42:50.633 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9789000000000001e-05 2024-09-11T07:42:50.633 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.634 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:50.634 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.634 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.634 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.634 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:213711 9.18 9:1bd0a1c2:::benchmark_data_smithi022_133943_object87120:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:42:50.634 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:38.521665+0000", 2024-09-11T07:42:50.634 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 131.44858910900001, 2024-09-11T07:42:50.634 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.130706253, 2024-09-11T07:42:50.634 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:50.634 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:50.634 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:50.634 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:50.634 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:50.634 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213711 2024-09-11T07:42:50.634 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.635 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:50.635 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.635 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:50.635 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.521665+0000", 2024-09-11T07:42:50.635 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.635 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.635 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.635 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:50.635 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.521665+0000", 2024-09-11T07:42:50.635 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.635 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.635 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.635 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:50.635 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.521665+0000", 2024-09-11T07:42:50.635 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:42:50.636 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.636 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.636 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:50.636 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.521668+0000", 2024-09-11T07:42:50.636 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2629999999999999e-06 2024-09-11T07:42:50.636 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.636 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.636 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:50.636 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.521673+0000", 2024-09-11T07:42:50.636 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1939999999999997e-06 2024-09-11T07:42:50.636 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.636 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.636 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:50.636 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.521677+0000", 2024-09-11T07:42:50.636 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4939999999999999e-06 2024-09-11T07:42:50.637 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.637 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.637 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.637 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.523673+0000", 2024-09-11T07:42:50.637 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0019969559999999998 2024-09-11T07:42:50.637 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.637 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.637 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting_for_map not empty", 2024-09-11T07:42:50.637 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.523678+0000", 2024-09-11T07:42:50.637 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5430000000000004e-06 2024-09-11T07:42:50.637 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.637 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.637 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.638 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.527916+0000", 2024-09-11T07:42:50.638 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.004237663 2024-09-11T07:42:50.638 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.638 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.638 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.638 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.527922+0000", 2024-09-11T07:42:50.638 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7329999999999997e-06 2024-09-11T07:42:50.638 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.638 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.638 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.638 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.515500+0000", 2024-09-11T07:42:50.638 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.98757713400000002 2024-09-11T07:42:50.638 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.638 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.639 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.639 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.515509+0000", 2024-09-11T07:42:50.639 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.8910000000000007e-06 2024-09-11T07:42:50.639 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.639 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.639 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.639 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.641303+0000", 2024-09-11T07:42:50.639 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.125793807 2024-09-11T07:42:50.639 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.639 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.639 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:50.639 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.641401+0000", 2024-09-11T07:42:50.639 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.7643e-05 2024-09-11T07:42:50.639 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.640 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.640 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:42:50.640 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.641682+0000", 2024-09-11T07:42:50.640 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00028099700000000001 2024-09-11T07:42:50.640 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.640 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.640 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:50.640 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.649986+0000", 2024-09-11T07:42:50.640 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0083042949999999997 2024-09-11T07:42:50.640 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.640 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.640 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.640 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.650042+0000", 2024-09-11T07:42:50.640 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6295999999999998e-05 2024-09-11T07:42:50.641 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.641 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.641 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.641 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652201+0000", 2024-09-11T07:42:50.641 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0021582160000000001 2024-09-11T07:42:50.641 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.641 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.641 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.641 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652272+0000", 2024-09-11T07:42:50.641 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1070999999999998e-05 2024-09-11T07:42:50.641 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.641 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.641 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:50.642 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652304+0000", 2024-09-11T07:42:50.642 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2710999999999999e-05 2024-09-11T07:42:50.642 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.642 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.642 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:50.642 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652371+0000", 2024-09-11T07:42:50.642 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.6966999999999998e-05 2024-09-11T07:42:50.642 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.642 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:50.642 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.642 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.642 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.642 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:213773 9.18 9:19914c12:::benchmark_data_smithi022_133943_object87182:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:42:50.642 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:38.576545+0000", 2024-09-11T07:42:50.643 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 131.39370951000001, 2024-09-11T07:42:50.643 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0760134770000001, 2024-09-11T07:42:50.643 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:50.643 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:50.643 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:50.643 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:50.643 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:50.643 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213773 2024-09-11T07:42:50.643 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.643 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:50.643 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.643 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:50.643 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.576545+0000", 2024-09-11T07:42:50.644 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.644 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.644 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.644 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:50.644 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.576545+0000", 2024-09-11T07:42:50.644 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.644 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.644 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.644 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:50.644 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.576542+0000", 2024-09-11T07:42:50.644 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999967 2024-09-11T07:42:50.644 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.644 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.644 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:50.645 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.576552+0000", 2024-09-11T07:42:50.645 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0701e-05 2024-09-11T07:42:50.645 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.645 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.645 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:50.645 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.576573+0000", 2024-09-11T07:42:50.645 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0358999999999999e-05 2024-09-11T07:42:50.645 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.645 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.645 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:50.645 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.576580+0000", 2024-09-11T07:42:50.645 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.8070000000000002e-06 2024-09-11T07:42:50.645 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.646 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.646 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.646 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.576639+0000", 2024-09-11T07:42:50.646 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9517e-05 2024-09-11T07:42:50.646 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.646 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.646 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.646 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.576650+0000", 2024-09-11T07:42:50.646 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0750999999999999e-05 2024-09-11T07:42:50.646 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.646 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.646 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.646 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516367+0000", 2024-09-11T07:42:50.646 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.93971757 2024-09-11T07:42:50.647 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.647 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.647 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.647 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516376+0000", 2024-09-11T07:42:50.647 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3100000000000001e-06 2024-09-11T07:42:50.647 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.647 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.647 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.647 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.648174+0000", 2024-09-11T07:42:50.647 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1317987030000001 2024-09-11T07:42:50.647 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.647 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.647 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:50.647 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.648313+0000", 2024-09-11T07:42:50.648 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000138932 2024-09-11T07:42:50.648 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.648 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.648 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:42:50.648 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.648596+0000", 2024-09-11T07:42:50.648 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00028308700000000001 2024-09-11T07:42:50.648 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.648 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.648 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:50.648 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.651981+0000", 2024-09-11T07:42:50.648 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.003384827 2024-09-11T07:42:50.648 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.648 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.649 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.649 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652038+0000", 2024-09-11T07:42:50.649 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6925000000000001e-05 2024-09-11T07:42:50.649 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.649 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.649 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.649 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652418+0000", 2024-09-11T07:42:50.649 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00038032300000000001 2024-09-11T07:42:50.649 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.649 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.649 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.649 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652463+0000", 2024-09-11T07:42:50.649 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4298000000000002e-05 2024-09-11T07:42:50.649 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.650 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.650 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:50.650 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652489+0000", 2024-09-11T07:42:50.650 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.6219000000000001e-05 2024-09-11T07:42:50.650 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.650 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.650 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:50.650 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652558+0000", 2024-09-11T07:42:50.650 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9294000000000004e-05 2024-09-11T07:42:50.650 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.650 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:50.650 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.650 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.650 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.650 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:213812 9.18 9:190c7e59:::benchmark_data_smithi022_133943_object87221:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:42:50.651 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:38.613884+0000", 2024-09-11T07:42:50.651 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 131.35637072099999, 2024-09-11T07:42:50.651 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0389224910000001, 2024-09-11T07:42:50.651 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:50.651 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:50.651 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:50.651 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:50.651 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:50.651 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213812 2024-09-11T07:42:50.651 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.651 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:50.651 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.651 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:50.651 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.613884+0000", 2024-09-11T07:42:50.652 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.652 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.652 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.652 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:50.652 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.613884+0000", 2024-09-11T07:42:50.652 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.652 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.652 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.652 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:50.652 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.613881+0000", 2024-09-11T07:42:50.652 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:42:50.652 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.652 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.653 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:50.653 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.613891+0000", 2024-09-11T07:42:50.653 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.8889999999999997e-06 2024-09-11T07:42:50.653 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.653 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.653 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:50.653 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.613908+0000", 2024-09-11T07:42:50.653 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.7595e-05 2024-09-11T07:42:50.653 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.653 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.653 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:50.653 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.613916+0000", 2024-09-11T07:42:50.653 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.824e-06 2024-09-11T07:42:50.653 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.654 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.654 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.654 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.613975+0000", 2024-09-11T07:42:50.654 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9441999999999998e-05 2024-09-11T07:42:50.654 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.654 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.654 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.654 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.613986+0000", 2024-09-11T07:42:50.654 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0107e-05 2024-09-11T07:42:50.654 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.654 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.654 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.654 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516401+0000", 2024-09-11T07:42:50.655 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.90241526999999999 2024-09-11T07:42:50.655 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.655 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.655 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.655 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516407+0000", 2024-09-11T07:42:50.655 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5930000000000002e-06 2024-09-11T07:42:50.655 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.655 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.655 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.655 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.650082+0000", 2024-09-11T07:42:50.655 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1336741699999999 2024-09-11T07:42:50.655 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.655 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.655 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:50.655 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.650188+0000", 2024-09-11T07:42:50.656 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000106847 2024-09-11T07:42:50.656 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.656 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.656 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:42:50.656 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.650406+0000", 2024-09-11T07:42:50.656 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000217395 2024-09-11T07:42:50.656 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.656 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.656 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:50.656 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652381+0000", 2024-09-11T07:42:50.656 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0019755599999999999 2024-09-11T07:42:50.656 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.656 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.657 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.657 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652594+0000", 2024-09-11T07:42:50.657 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000212568 2024-09-11T07:42:50.657 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.657 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.657 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.657 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652626+0000", 2024-09-11T07:42:50.657 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2514000000000001e-05 2024-09-11T07:42:50.657 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.657 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.657 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.657 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652672+0000", 2024-09-11T07:42:50.657 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5481e-05 2024-09-11T07:42:50.658 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.658 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.658 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:50.658 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652700+0000", 2024-09-11T07:42:50.658 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7827e-05 2024-09-11T07:42:50.658 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.658 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.658 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:50.658 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652806+0000", 2024-09-11T07:42:50.658 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000106327 2024-09-11T07:42:50.658 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.658 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:50.658 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.658 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.659 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.659 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:213830 9.18 9:1d562e34:::benchmark_data_smithi022_133943_object87239:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:42:50.659 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:38.628512+0000", 2024-09-11T07:42:50.659 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 131.34174245599999, 2024-09-11T07:42:50.659 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0332906839999998, 2024-09-11T07:42:50.659 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:50.659 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:50.659 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:50.659 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:50.659 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:50.659 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213830 2024-09-11T07:42:50.659 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.659 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:50.660 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.660 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:50.660 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.628512+0000", 2024-09-11T07:42:50.660 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.660 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.660 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.660 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:50.660 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.628512+0000", 2024-09-11T07:42:50.660 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.660 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.660 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.660 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:50.660 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.628510+0000", 2024-09-11T07:42:50.660 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:42:50.661 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.661 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.661 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:50.661 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.628517+0000", 2024-09-11T07:42:50.661 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.588e-06 2024-09-11T07:42:50.661 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.661 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.661 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:50.661 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.628529+0000", 2024-09-11T07:42:50.661 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1737e-05 2024-09-11T07:42:50.661 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.661 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.661 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:50.662 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.628535+0000", 2024-09-11T07:42:50.662 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.5860000000000004e-06 2024-09-11T07:42:50.662 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.662 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.662 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.662 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.628585+0000", 2024-09-11T07:42:50.662 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.0519000000000002e-05 2024-09-11T07:42:50.662 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.662 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.662 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.662 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.628594+0000", 2024-09-11T07:42:50.662 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.7199999999999995e-06 2024-09-11T07:42:50.662 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.662 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.663 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.663 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516430+0000", 2024-09-11T07:42:50.663 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.88783586599999997 2024-09-11T07:42:50.663 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.663 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.663 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.663 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516437+0000", 2024-09-11T07:42:50.663 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9280000000000004e-06 2024-09-11T07:42:50.663 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.663 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.663 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.663 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652844+0000", 2024-09-11T07:42:50.664 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.136407495 2024-09-11T07:42:50.664 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.664 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.664 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:50.664 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652953+0000", 2024-09-11T07:42:50.664 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010859 2024-09-11T07:42:50.664 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.664 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.664 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:42:50.664 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.653209+0000", 2024-09-11T07:42:50.664 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00025661 2024-09-11T07:42:50.664 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.664 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.664 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.665 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.657770+0000", 2024-09-11T07:42:50.665 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0045608970000000004 2024-09-11T07:42:50.665 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.665 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.665 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:50.665 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.657841+0000", 2024-09-11T07:42:50.665 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0417e-05 2024-09-11T07:42:50.665 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.665 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.665 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.665 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.657898+0000", 2024-09-11T07:42:50.665 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7083000000000002e-05 2024-09-11T07:42:50.665 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.665 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.666 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.666 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.661631+0000", 2024-09-11T07:42:50.666 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0037329630000000002 2024-09-11T07:42:50.666 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.666 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.666 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:50.666 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.661665+0000", 2024-09-11T07:42:50.666 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4257000000000001e-05 2024-09-11T07:42:50.666 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.666 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.666 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:50.666 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.661803+0000", 2024-09-11T07:42:50.666 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000137637 2024-09-11T07:42:50.666 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.667 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:50.667 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.667 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.667 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.667 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:213855 9.18 9:1c5e2bfb:::benchmark_data_smithi022_133943_object87264:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:42:50.667 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:38.650487+0000", 2024-09-11T07:42:50.667 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 131.31976741400001, 2024-09-11T07:42:50.667 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0237156920000001, 2024-09-11T07:42:50.667 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:50.667 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:50.667 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:50.668 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:50.668 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:50.668 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213855 2024-09-11T07:42:50.668 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.668 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:50.668 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.668 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:50.668 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.650487+0000", 2024-09-11T07:42:50.668 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.668 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.668 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.668 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:50.668 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.650487+0000", 2024-09-11T07:42:50.669 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.669 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.669 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.669 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:50.669 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.650485+0000", 2024-09-11T07:42:50.669 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:42:50.669 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.669 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.669 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:50.669 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.650492+0000", 2024-09-11T07:42:50.669 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9630000000000001e-06 2024-09-11T07:42:50.669 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.669 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.669 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:50.670 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.650505+0000", 2024-09-11T07:42:50.670 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3114e-05 2024-09-11T07:42:50.670 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.670 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.670 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:50.670 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.650511+0000", 2024-09-11T07:42:50.670 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1190000000000002e-06 2024-09-11T07:42:50.670 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.670 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.670 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.670 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.650568+0000", 2024-09-11T07:42:50.670 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6511999999999997e-05 2024-09-11T07:42:50.670 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.670 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.670 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.671 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.650578+0000", 2024-09-11T07:42:50.671 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0631e-05 2024-09-11T07:42:50.671 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.671 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.671 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.671 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516460+0000", 2024-09-11T07:42:50.671 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.86588189000000004 2024-09-11T07:42:50.671 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.671 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.671 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.671 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516467+0000", 2024-09-11T07:42:50.671 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.421e-06 2024-09-11T07:42:50.671 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.671 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.672 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.672 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.654928+0000", 2024-09-11T07:42:50.672 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1384612590000001 2024-09-11T07:42:50.672 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.672 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.672 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:50.672 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.655059+0000", 2024-09-11T07:42:50.672 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013165200000000001 2024-09-11T07:42:50.672 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.672 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.672 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:42:50.672 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.655322+0000", 2024-09-11T07:42:50.672 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00026282699999999998 2024-09-11T07:42:50.673 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.673 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.673 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.673 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.657828+0000", 2024-09-11T07:42:50.673 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0025059990000000001 2024-09-11T07:42:50.673 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.673 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.673 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:50.673 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.657850+0000", 2024-09-11T07:42:50.673 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1801e-05 2024-09-11T07:42:50.674 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.674 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.674 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.674 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.665730+0000", 2024-09-11T07:42:50.674 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0078795240000000006 2024-09-11T07:42:50.674 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.674 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.674 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.674 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.673993+0000", 2024-09-11T07:42:50.674 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0082633740000000004 2024-09-11T07:42:50.674 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.674 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.674 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:50.674 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.674059+0000", 2024-09-11T07:42:50.675 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.6138999999999998e-05 2024-09-11T07:42:50.675 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.675 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.675 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:50.675 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.674203+0000", 2024-09-11T07:42:50.675 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014348299999999999 2024-09-11T07:42:50.675 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.675 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:50.675 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.675 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.675 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.675 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:213989 9.18 9:19156234:::benchmark_data_smithi022_133943_object87398:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:42:50.675 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:38.779547+0000", 2024-09-11T07:42:50.675 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 131.190707675, 2024-09-11T07:42:50.675 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.894937614, 2024-09-11T07:42:50.676 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:50.676 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:50.676 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:50.676 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:50.676 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:50.676 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213989 2024-09-11T07:42:50.676 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.676 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:50.676 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.676 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:50.676 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.779547+0000", 2024-09-11T07:42:50.676 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.676 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.676 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.677 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:50.677 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.779547+0000", 2024-09-11T07:42:50.677 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.677 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.677 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.677 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:50.677 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.779545+0000", 2024-09-11T07:42:50.677 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:42:50.677 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.677 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.677 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:50.677 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.779552+0000", 2024-09-11T07:42:50.677 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.6969999999999993e-06 2024-09-11T07:42:50.678 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.678 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.678 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:50.678 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.779596+0000", 2024-09-11T07:42:50.678 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3779000000000002e-05 2024-09-11T07:42:50.678 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.678 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.678 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:50.678 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.779603+0000", 2024-09-11T07:42:50.678 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9800000000000001e-06 2024-09-11T07:42:50.678 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.678 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.678 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.678 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.779656+0000", 2024-09-11T07:42:50.678 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2692000000000003e-05 2024-09-11T07:42:50.679 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.679 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.679 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.679 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.779665+0000", 2024-09-11T07:42:50.679 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.0180000000000007e-06 2024-09-11T07:42:50.679 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.679 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.679 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.679 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516490+0000", 2024-09-11T07:42:50.679 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.736824967 2024-09-11T07:42:50.679 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.679 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.679 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.679 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516496+0000", 2024-09-11T07:42:50.680 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5050000000000004e-06 2024-09-11T07:42:50.680 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.680 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.680 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.680 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.656449+0000", 2024-09-11T07:42:50.680 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1399522580000001 2024-09-11T07:42:50.680 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.680 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.680 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:50.680 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.656542+0000", 2024-09-11T07:42:50.680 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.3465e-05 2024-09-11T07:42:50.680 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.680 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.681 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:42:50.681 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.656765+0000", 2024-09-11T07:42:50.681 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00022321900000000001 2024-09-11T07:42:50.681 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.681 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.681 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:50.681 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.659429+0000", 2024-09-11T07:42:50.681 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0026633440000000002 2024-09-11T07:42:50.681 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.681 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.681 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.681 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.661547+0000", 2024-09-11T07:42:50.681 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0021182459999999998 2024-09-11T07:42:50.681 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.681 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.682 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.682 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.665762+0000", 2024-09-11T07:42:50.682 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0042147069999999998 2024-09-11T07:42:50.682 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.682 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.682 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.682 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.674298+0000", 2024-09-11T07:42:50.682 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0085361289999999999 2024-09-11T07:42:50.682 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.682 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.682 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:50.682 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.674337+0000", 2024-09-11T07:42:50.682 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9708e-05 2024-09-11T07:42:50.682 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.683 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.683 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:50.683 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.674484+0000", 2024-09-11T07:42:50.683 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014686500000000001 2024-09-11T07:42:50.683 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.683 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:50.683 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.683 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.683 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.683 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:214018 9.18 9:1a17d8cd:::benchmark_data_smithi022_133943_object87427:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:42:50.683 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:38.807290+0000", 2024-09-11T07:42:50.683 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 131.16296438500001, 2024-09-11T07:42:50.684 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.868875066, 2024-09-11T07:42:50.684 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:50.684 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:50.684 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:50.684 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:50.684 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:50.684 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 214018 2024-09-11T07:42:50.684 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.684 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:50.684 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.684 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:50.684 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.807290+0000", 2024-09-11T07:42:50.684 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.684 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.685 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.685 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:50.685 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.807290+0000", 2024-09-11T07:42:50.685 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.685 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.685 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.685 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:50.686 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:42:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:42:50] "GET /metrics HTTP/1.1" 200 224897 "" "Prometheus/2.43.0" 2024-09-11T07:42:50.686 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.807288+0000", 2024-09-11T07:42:50.686 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:42:50.686 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.686 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.686 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:50.686 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.807296+0000", 2024-09-11T07:42:50.686 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.5150000000000006e-06 2024-09-11T07:42:50.686 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.687 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.687 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:50.687 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.807312+0000", 2024-09-11T07:42:50.687 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5974999999999999e-05 2024-09-11T07:42:50.687 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.687 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.687 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:50.687 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.807318+0000", 2024-09-11T07:42:50.687 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1210000000000003e-06 2024-09-11T07:42:50.687 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.687 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.687 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.687 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.807371+0000", 2024-09-11T07:42:50.687 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2958999999999999e-05 2024-09-11T07:42:50.688 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.688 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.688 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.688 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.807380+0000", 2024-09-11T07:42:50.688 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.7059999999999993e-06 2024-09-11T07:42:50.688 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.688 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.688 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.688 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516519+0000", 2024-09-11T07:42:50.688 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.709139301 2024-09-11T07:42:50.688 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.688 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.688 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.689 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516526+0000", 2024-09-11T07:42:50.689 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4130000000000003e-06 2024-09-11T07:42:50.689 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.689 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.689 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.689 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.657921+0000", 2024-09-11T07:42:50.689 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.141395846 2024-09-11T07:42:50.689 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.689 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.689 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:50.689 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.657991+0000", 2024-09-11T07:42:50.689 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9283999999999996e-05 2024-09-11T07:42:50.689 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.690 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.690 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:42:50.690 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.658156+0000", 2024-09-11T07:42:50.690 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000165234 2024-09-11T07:42:50.690 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.690 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.690 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.690 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.661591+0000", 2024-09-11T07:42:50.690 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0034355660000000001 2024-09-11T07:42:50.690 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.690 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.690 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:50.690 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.661812+0000", 2024-09-11T07:42:50.690 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000220662 2024-09-11T07:42:50.691 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.691 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.691 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.691 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.666688+0000", 2024-09-11T07:42:50.691 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0048756729999999996 2024-09-11T07:42:50.691 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.691 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.691 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.691 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.675122+0000", 2024-09-11T07:42:50.691 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0084342429999999993 2024-09-11T07:42:50.691 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.691 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.691 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:50.691 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.675921+0000", 2024-09-11T07:42:50.692 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00079866599999999996 2024-09-11T07:42:50.692 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.692 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.692 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:50.692 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676165+0000", 2024-09-11T07:42:50.692 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00024431099999999999 2024-09-11T07:42:50.692 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.692 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:50.692 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.692 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.692 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.692 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:214026 9.18 9:1d1707cd:::benchmark_data_smithi022_133943_object87435:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:42:50.693 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:38.815450+0000", 2024-09-11T07:42:50.693 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 131.154804391, 2024-09-11T07:42:50.693 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8610872550000002, 2024-09-11T07:42:50.693 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:50.693 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:50.693 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:50.693 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:50.693 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:50.693 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 214026 2024-09-11T07:42:50.693 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.693 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:50.693 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.693 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:50.694 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.815450+0000", 2024-09-11T07:42:50.694 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.694 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.694 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.694 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:50.694 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.815450+0000", 2024-09-11T07:42:50.694 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.694 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.694 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.694 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:50.694 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.815448+0000", 2024-09-11T07:42:50.694 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:42:50.694 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.694 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.695 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:50.695 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.815455+0000", 2024-09-11T07:42:50.695 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4690000000000002e-06 2024-09-11T07:42:50.695 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.695 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.695 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:50.695 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.815462+0000", 2024-09-11T07:42:50.695 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0419999999999997e-06 2024-09-11T07:42:50.695 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.695 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.695 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:50.695 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.815466+0000", 2024-09-11T07:42:50.695 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8039999999999999e-06 2024-09-11T07:42:50.695 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.696 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.696 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.696 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.815510+0000", 2024-09-11T07:42:50.696 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4657999999999998e-05 2024-09-11T07:42:50.696 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.696 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.696 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.696 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.815519+0000", 2024-09-11T07:42:50.696 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.704e-06 2024-09-11T07:42:50.696 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.696 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.697 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.697 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516550+0000", 2024-09-11T07:42:50.697 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.70103092099999997 2024-09-11T07:42:50.697 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.697 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.697 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.697 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516556+0000", 2024-09-11T07:42:50.697 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4470000000000003e-06 2024-09-11T07:42:50.697 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.697 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.697 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.698 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.659474+0000", 2024-09-11T07:42:50.698 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1429177239999999 2024-09-11T07:42:50.698 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.698 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.698 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:50.698 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.659598+0000", 2024-09-11T07:42:50.698 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012443 2024-09-11T07:42:50.698 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.698 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.698 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:42:50.698 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.659846+0000", 2024-09-11T07:42:50.698 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000247259 2024-09-11T07:42:50.698 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.698 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.699 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.699 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.661864+0000", 2024-09-11T07:42:50.699 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002018527 2024-09-11T07:42:50.699 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.699 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.699 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:50.699 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.663437+0000", 2024-09-11T07:42:50.699 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0015723569999999999 2024-09-11T07:42:50.699 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.699 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.699 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.699 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.669061+0000", 2024-09-11T07:42:50.699 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.005624352 2024-09-11T07:42:50.699 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.699 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.700 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.700 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676315+0000", 2024-09-11T07:42:50.700 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0072540979999999996 2024-09-11T07:42:50.700 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.700 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.700 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:50.700 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676369+0000", 2024-09-11T07:42:50.700 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.4094000000000001e-05 2024-09-11T07:42:50.700 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.700 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.700 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:50.700 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676537+0000", 2024-09-11T07:42:50.700 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000167994 2024-09-11T07:42:50.700 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.701 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:50.701 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.701 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.701 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.701 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:214088 9.18 9:1b56ff78:::benchmark_data_smithi022_133943_object87497:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:42:50.701 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:38.875033+0000", 2024-09-11T07:42:50.701 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 131.09522111199999, 2024-09-11T07:42:50.701 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8017833599999999, 2024-09-11T07:42:50.701 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:50.701 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:50.701 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:50.701 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:50.701 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:50.701 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 214088 2024-09-11T07:42:50.702 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.702 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:50.702 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.702 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:50.702 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.875033+0000", 2024-09-11T07:42:50.702 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.702 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.702 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.702 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:50.702 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.875033+0000", 2024-09-11T07:42:50.702 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.702 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.702 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.702 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:50.703 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.875031+0000", 2024-09-11T07:42:50.703 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:42:50.703 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.703 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.703 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:50.703 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.875039+0000", 2024-09-11T07:42:50.703 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1790000000000002e-06 2024-09-11T07:42:50.703 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.703 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.703 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:50.703 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.875051+0000", 2024-09-11T07:42:50.703 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2592000000000001e-05 2024-09-11T07:42:50.703 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.703 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.704 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:50.704 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.875058+0000", 2024-09-11T07:42:50.704 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7700000000000004e-06 2024-09-11T07:42:50.704 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.704 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.704 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.704 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.875107+0000", 2024-09-11T07:42:50.704 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9285e-05 2024-09-11T07:42:50.704 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.704 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.704 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.704 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.875114+0000", 2024-09-11T07:42:50.704 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2840000000000003e-06 2024-09-11T07:42:50.704 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.705 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.705 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.705 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516578+0000", 2024-09-11T07:42:50.705 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.64146455499999999 2024-09-11T07:42:50.705 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.705 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.705 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.705 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516585+0000", 2024-09-11T07:42:50.705 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7290000000000003e-06 2024-09-11T07:42:50.705 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.705 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.705 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.705 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.661910+0000", 2024-09-11T07:42:50.705 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1453256519999999 2024-09-11T07:42:50.705 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.706 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.706 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:50.706 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.662040+0000", 2024-09-11T07:42:50.706 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012906899999999999 2024-09-11T07:42:50.706 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.706 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.706 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:42:50.706 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.662277+0000", 2024-09-11T07:42:50.706 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00023776400000000001 2024-09-11T07:42:50.706 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.706 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.706 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.706 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.665805+0000", 2024-09-11T07:42:50.706 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0035281079999999999 2024-09-11T07:42:50.707 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.707 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.707 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:50.707 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.669022+0000", 2024-09-11T07:42:50.707 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0032164060000000002 2024-09-11T07:42:50.707 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.707 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.707 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.707 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.673957+0000", 2024-09-11T07:42:50.707 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0049350879999999998 2024-09-11T07:42:50.707 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.707 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.707 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.707 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676648+0000", 2024-09-11T07:42:50.707 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0026909709999999999 2024-09-11T07:42:50.707 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.708 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.708 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:50.708 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676696+0000", 2024-09-11T07:42:50.708 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8321e-05 2024-09-11T07:42:50.708 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.708 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.708 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:50.708 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676817+0000", 2024-09-11T07:42:50.708 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000120359 2024-09-11T07:42:50.708 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.708 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:50.708 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.708 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.708 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.709 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:214109 9.18 9:1f0f4b12:::benchmark_data_smithi022_133943_object87518:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:42:50.709 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:38.901408+0000", 2024-09-11T07:42:50.709 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 131.06884593000001, 2024-09-11T07:42:50.709 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.775759847, 2024-09-11T07:42:50.709 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:50.709 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:50.709 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:50.709 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:50.709 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:50.709 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 214109 2024-09-11T07:42:50.709 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.709 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:50.709 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.709 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:50.710 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.901408+0000", 2024-09-11T07:42:50.710 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.710 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.710 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.710 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:50.710 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.901408+0000", 2024-09-11T07:42:50.710 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.710 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.710 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.710 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:50.710 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.901406+0000", 2024-09-11T07:42:50.710 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:42:50.710 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.710 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.710 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:50.711 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.901414+0000", 2024-09-11T07:42:50.711 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.6860000000000006e-06 2024-09-11T07:42:50.711 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.711 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.711 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:50.711 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.901429+0000", 2024-09-11T07:42:50.711 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4842e-05 2024-09-11T07:42:50.711 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.711 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.711 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:50.711 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.901436+0000", 2024-09-11T07:42:50.711 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7789999999999998e-06 2024-09-11T07:42:50.711 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.711 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.711 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.711 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.901484+0000", 2024-09-11T07:42:50.712 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7979999999999998e-05 2024-09-11T07:42:50.712 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.712 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.712 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.712 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.901490+0000", 2024-09-11T07:42:50.712 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.139e-06 2024-09-11T07:42:50.712 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.712 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.712 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.712 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516608+0000", 2024-09-11T07:42:50.712 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.61511833400000004 2024-09-11T07:42:50.712 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.712 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.712 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.713 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516616+0000", 2024-09-11T07:42:50.713 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5490000000000001e-06 2024-09-11T07:42:50.713 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.713 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.713 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.713 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.663475+0000", 2024-09-11T07:42:50.713 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1468589119999999 2024-09-11T07:42:50.713 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.713 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.713 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:50.713 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.663568+0000", 2024-09-11T07:42:50.714 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:50 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3601309742' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:42:50.714 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:50 smithi022 ceph-mon[37330]: from='client.25738 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:42:50.714 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.3740999999999996e-05 2024-09-11T07:42:50.714 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.714 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.714 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:42:50.714 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.663812+0000", 2024-09-11T07:42:50.715 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00024389 2024-09-11T07:42:50.715 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.715 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.715 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:50.715 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.669028+0000", 2024-09-11T07:42:50.715 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0052160720000000004 2024-09-11T07:42:50.715 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.715 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.715 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.715 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.674240+0000", 2024-09-11T07:42:50.715 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0052113469999999999 2024-09-11T07:42:50.715 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.715 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.716 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.716 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.674270+0000", 2024-09-11T07:42:50.716 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0156000000000001e-05 2024-09-11T07:42:50.716 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.716 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.716 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.716 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676998+0000", 2024-09-11T07:42:50.716 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0027284900000000001 2024-09-11T07:42:50.716 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.716 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.716 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:50.716 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.677045+0000", 2024-09-11T07:42:50.716 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7055999999999998e-05 2024-09-11T07:42:50.717 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.717 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.717 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:50.717 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.677168+0000", 2024-09-11T07:42:50.717 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012286400000000001 2024-09-11T07:42:50.717 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.717 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:50.717 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.717 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.717 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.717 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:214203 9.18 9:1e3a93c6:::benchmark_data_smithi022_133943_object87612:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:42:50.717 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:39.007710+0000", 2024-09-11T07:42:50.717 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 130.96254415000001, 2024-09-11T07:42:50.718 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.681745552, 2024-09-11T07:42:50.718 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:50.718 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:50.718 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:50.718 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:50.718 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:50.718 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 214203 2024-09-11T07:42:50.718 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.718 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:50.718 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.718 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:50.718 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.007710+0000", 2024-09-11T07:42:50.718 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.719 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.719 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.719 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:50.719 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.007710+0000", 2024-09-11T07:42:50.719 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.719 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.719 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.719 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:50.719 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.007708+0000", 2024-09-11T07:42:50.719 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:42:50.719 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.719 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.720 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:50.720 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.007715+0000", 2024-09-11T07:42:50.720 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1860000000000003e-06 2024-09-11T07:42:50.720 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.720 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.720 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:50.720 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.007739+0000", 2024-09-11T07:42:50.720 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3646999999999999e-05 2024-09-11T07:42:50.720 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.720 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.720 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:50.720 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.007745+0000", 2024-09-11T07:42:50.720 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5270000000000004e-06 2024-09-11T07:42:50.721 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.721 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.721 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.721 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.007805+0000", 2024-09-11T07:42:50.721 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9833000000000001e-05 2024-09-11T07:42:50.721 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.721 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.721 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.721 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.007812+0000", 2024-09-11T07:42:50.721 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5080000000000002e-06 2024-09-11T07:42:50.721 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.721 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.721 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.722 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516640+0000", 2024-09-11T07:42:50.722 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.50882864999999999 2024-09-11T07:42:50.722 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.722 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.722 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.722 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516647+0000", 2024-09-11T07:42:50.722 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3459999999999999e-06 2024-09-11T07:42:50.722 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.722 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.722 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.722 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.666712+0000", 2024-09-11T07:42:50.722 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.150065023 2024-09-11T07:42:50.722 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.722 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.722 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:50.723 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.666829+0000", 2024-09-11T07:42:50.723 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000117748 2024-09-11T07:42:50.723 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.723 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.723 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:42:50.723 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.667169+0000", 2024-09-11T07:42:50.723 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00033908499999999998 2024-09-11T07:42:50.723 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.723 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.723 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:50.723 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.673874+0000", 2024-09-11T07:42:50.723 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0067059199999999998 2024-09-11T07:42:50.723 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.723 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.724 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.724 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.675054+0000", 2024-09-11T07:42:50.724 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001179865 2024-09-11T07:42:50.724 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.724 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.724 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.724 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.675092+0000", 2024-09-11T07:42:50.724 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7827000000000002e-05 2024-09-11T07:42:50.724 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.724 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.724 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.724 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.689279+0000", 2024-09-11T07:42:50.724 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.014186772 2024-09-11T07:42:50.724 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.725 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.725 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:50.725 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.689331+0000", 2024-09-11T07:42:50.725 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2410999999999997e-05 2024-09-11T07:42:50.725 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.725 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.725 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:50.725 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.689456+0000", 2024-09-11T07:42:50.725 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012438999999999999 2024-09-11T07:42:50.725 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.725 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:50.725 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.725 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.725 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.726 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:214212 9.18 9:1a025e7e:::benchmark_data_smithi022_133943_object87621:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:42:50.726 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:39.021035+0000", 2024-09-11T07:42:50.726 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 130.94921883999999, 2024-09-11T07:42:50.726 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.6686958949999999, 2024-09-11T07:42:50.726 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:50.726 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:50.726 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:50.726 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:50.726 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:50.726 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 214212 2024-09-11T07:42:50.726 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.726 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:50.726 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.726 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:50.726 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.021035+0000", 2024-09-11T07:42:50.727 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.727 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.727 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.727 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:50.727 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.021035+0000", 2024-09-11T07:42:50.727 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.727 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.727 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.727 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:50.727 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.021032+0000", 2024-09-11T07:42:50.727 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999967 2024-09-11T07:42:50.727 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.727 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.727 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:50.727 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.021043+0000", 2024-09-11T07:42:50.728 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0825999999999999e-05 2024-09-11T07:42:50.728 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.728 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.728 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:50.728 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.021064+0000", 2024-09-11T07:42:50.728 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0899000000000001e-05 2024-09-11T07:42:50.728 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.728 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.728 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:50.728 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.021070+0000", 2024-09-11T07:42:50.728 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2559999999999999e-06 2024-09-11T07:42:50.728 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.728 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.729 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.729 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.021141+0000", 2024-09-11T07:42:50.729 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0778000000000006e-05 2024-09-11T07:42:50.729 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.729 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.729 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.729 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.021154+0000", 2024-09-11T07:42:50.729 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3176000000000001e-05 2024-09-11T07:42:50.729 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.729 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.729 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.729 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516671+0000", 2024-09-11T07:42:50.729 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.49551719500000002 2024-09-11T07:42:50.730 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.730 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.730 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.730 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516678+0000", 2024-09-11T07:42:50.730 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0740000000000004e-06 2024-09-11T07:42:50.730 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.730 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.730 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.730 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.669082+0000", 2024-09-11T07:42:50.730 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.152403734 2024-09-11T07:42:50.730 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.730 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.730 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:50.730 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.669197+0000", 2024-09-11T07:42:50.731 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000115133 2024-09-11T07:42:50.731 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.731 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.731 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:42:50.731 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.669451+0000", 2024-09-11T07:42:50.731 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00025389500000000002 2024-09-11T07:42:50.731 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.731 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.731 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.731 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676214+0000", 2024-09-11T07:42:50.731 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0067624369999999996 2024-09-11T07:42:50.731 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.731 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.731 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.731 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676265+0000", 2024-09-11T07:42:50.731 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1180999999999997e-05 2024-09-11T07:42:50.732 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.732 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.732 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:50.732 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676274+0000", 2024-09-11T07:42:50.732 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.5489999999999995e-06 2024-09-11T07:42:50.732 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.732 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.732 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.732 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.689519+0000", 2024-09-11T07:42:50.732 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.013244211000000001 2024-09-11T07:42:50.732 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.732 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.732 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:50.732 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.689599+0000", 2024-09-11T07:42:50.733 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.0069999999999997e-05 2024-09-11T07:42:50.733 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.733 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.733 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:50.733 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.689731+0000", 2024-09-11T07:42:50.733 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013274399999999999 2024-09-11T07:42:50.733 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.733 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:50.733 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.733 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.733 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.733 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:214216 9.18 9:182c3ab1:::benchmark_data_smithi022_133943_object87625:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:42:50.734 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:39.025544+0000", 2024-09-11T07:42:50.734 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 130.944710337, 2024-09-11T07:42:50.734 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.664438707, 2024-09-11T07:42:50.734 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:50.734 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:50.734 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:50.734 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:50.734 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:50.734 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 214216 2024-09-11T07:42:50.734 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.734 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:50.734 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.734 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:50.734 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.025544+0000", 2024-09-11T07:42:50.735 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.735 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.735 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.735 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:50.735 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.025544+0000", 2024-09-11T07:42:50.735 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.735 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.735 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.735 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:50.735 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.025542+0000", 2024-09-11T07:42:50.735 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:42:50.735 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.735 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.735 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:50.736 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.025551+0000", 2024-09-11T07:42:50.736 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.0429999999999996e-06 2024-09-11T07:42:50.736 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.736 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.736 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:50.736 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.025563+0000", 2024-09-11T07:42:50.736 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1949e-05 2024-09-11T07:42:50.736 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.736 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.736 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:50.736 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.025570+0000", 2024-09-11T07:42:50.736 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.7419999999999997e-06 2024-09-11T07:42:50.736 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.736 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.736 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.737 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.025627+0000", 2024-09-11T07:42:50.737 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6320999999999998e-05 2024-09-11T07:42:50.737 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.737 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.737 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.737 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.025639+0000", 2024-09-11T07:42:50.737 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2086000000000001e-05 2024-09-11T07:42:50.737 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.737 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.737 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.737 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516706+0000", 2024-09-11T07:42:50.737 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.49106744200000002 2024-09-11T07:42:50.737 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.737 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.738 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.738 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516713+0000", 2024-09-11T07:42:50.738 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.686e-06 2024-09-11T07:42:50.738 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.738 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.738 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.738 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.671438+0000", 2024-09-11T07:42:50.738 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1547249019999999 2024-09-11T07:42:50.738 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.738 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.738 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:50.738 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.671509+0000", 2024-09-11T07:42:50.738 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1224999999999997e-05 2024-09-11T07:42:50.738 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.739 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.739 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:42:50.739 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.671839+0000", 2024-09-11T07:42:50.739 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00032957899999999998 2024-09-11T07:42:50.739 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.739 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.739 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:50.739 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676280+0000", 2024-09-11T07:42:50.739 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0044416059999999999 2024-09-11T07:42:50.739 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.739 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.739 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.739 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676581+0000", 2024-09-11T07:42:50.739 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00030059999999999999 2024-09-11T07:42:50.739 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.739 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.740 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.740 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676617+0000", 2024-09-11T07:42:50.740 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6337000000000001e-05 2024-09-11T07:42:50.740 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.740 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.740 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.740 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.689767+0000", 2024-09-11T07:42:50.740 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.013149793999999999 2024-09-11T07:42:50.740 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.740 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.740 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:50.740 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.689851+0000", 2024-09-11T07:42:50.740 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3857999999999996e-05 2024-09-11T07:42:50.740 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.741 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.741 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:50.741 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.689983+0000", 2024-09-11T07:42:50.741 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013197799999999999 2024-09-11T07:42:50.741 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.741 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:50.741 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.741 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.741 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.741 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:214246 9.18 9:1b25dba9:::benchmark_data_smithi022_133943_object87655:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:42:50.741 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:39.125384+0000", 2024-09-11T07:42:50.741 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 130.844870601, 2024-09-11T07:42:50.742 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5648045229999998, 2024-09-11T07:42:50.742 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:42:50.742 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:50.742 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:42:50.742 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:42:50.742 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:42:50.742 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 214246 2024-09-11T07:42:50.742 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.742 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:42:50.742 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.742 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:42:50.742 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.125384+0000", 2024-09-11T07:42:50.742 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.742 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.742 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.743 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:42:50.743 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.125384+0000", 2024-09-11T07:42:50.743 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:42:50.743 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.743 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.743 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:42:50.743 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.125382+0000", 2024-09-11T07:42:50.743 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:42:50.743 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.743 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.743 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:42:50.743 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.125389+0000", 2024-09-11T07:42:50.743 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.2760000000000003e-06 2024-09-11T07:42:50.743 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.744 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.744 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:42:50.744 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.125401+0000", 2024-09-11T07:42:50.744 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2174000000000001e-05 2024-09-11T07:42:50.744 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.744 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.744 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:42:50.744 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.125406+0000", 2024-09-11T07:42:50.744 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8550000000000001e-06 2024-09-11T07:42:50.744 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.744 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.744 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.744 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.125456+0000", 2024-09-11T07:42:50.744 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9475000000000003e-05 2024-09-11T07:42:50.745 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.745 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.745 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.745 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.125463+0000", 2024-09-11T07:42:50.745 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.8129999999999996e-06 2024-09-11T07:42:50.745 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.745 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.745 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.745 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516885+0000", 2024-09-11T07:42:50.745 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.39142164699999998 2024-09-11T07:42:50.745 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.745 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.745 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:42:50.746 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516894+0000", 2024-09-11T07:42:50.746 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.3929999999999997e-06 2024-09-11T07:42:50.746 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.746 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.746 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:42:50.746 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.677196+0000", 2024-09-11T07:42:50.746 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.160301123 2024-09-11T07:42:50.746 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.746 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.746 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:42:50.746 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.677304+0000", 2024-09-11T07:42:50.746 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010870399999999999 2024-09-11T07:42:50.746 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.746 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.746 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:42:50.747 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.677637+0000", 2024-09-11T07:42:50.747 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00033315200000000002 2024-09-11T07:42:50.747 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.747 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.747 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.747 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.689127+0000", 2024-09-11T07:42:50.747 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.011489432000000001 2024-09-11T07:42:50.747 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.747 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.747 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:42:50.747 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.689182+0000", 2024-09-11T07:42:50.747 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.4849000000000003e-05 2024-09-11T07:42:50.747 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.747 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.748 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.748 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.689250+0000", 2024-09-11T07:42:50.748 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.8209999999999999e-05 2024-09-11T07:42:50.748 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.748 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.748 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:50.748 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.690018+0000", 2024-09-11T07:42:50.748 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00076781699999999996 2024-09-11T07:42:50.748 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.748 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.748 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:42:50.748 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.690069+0000", 2024-09-11T07:42:50.748 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1017999999999998e-05 2024-09-11T07:42:50.748 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:50.749 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:50.749 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:42:50.749 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.690188+0000", 2024-09-11T07:42:50.749 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000119518 2024-09-11T07:42:50.749 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.749 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:50.749 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.749 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:50.749 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:50.749 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:42:50.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:50 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3601309742' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:42:50.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:50 smithi022 ceph-mon[29462]: from='client.25738 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:42:50.750 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:42:50.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:50.751 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:50.751 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:50.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:50 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3601309742' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:42:50.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:50 smithi104 ceph-mon[33507]: from='client.25738 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:42:50.853 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:50.891 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:50.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:50.892 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:50.955 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 35 16 66395 66379 118.515 75.75 0.00494306 0.00843251 2024-09-11T07:42:50.993 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:51.057 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:51.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:51.058 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:51.159 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:51.186 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:51.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:51.187 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:51.288 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:51.323 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:51.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:51.324 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:51.425 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:51.467 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:51.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:51.468 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:51.569 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:51.609 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:51.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:51.612 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:51.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:51 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:42:51.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:51 smithi022 ceph-mon[29462]: pgmap v757: 148 pgs: 148 active+clean; 3.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 119 MiB/s wr, 1.90k op/s 2024-09-11T07:42:51.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:51 smithi022 ceph-mon[29462]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:42:51.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:51 smithi022 ceph-mon[29462]: from='client.25738 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:42:51.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:51 smithi022 ceph-mon[29462]: osdmap e356: 8 total, 8 up, 8 in 2024-09-11T07:42:51.672 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:51 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:42:51.672 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:51 smithi022 ceph-mon[37330]: pgmap v757: 148 pgs: 148 active+clean; 3.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 119 MiB/s wr, 1.90k op/s 2024-09-11T07:42:51.672 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:51 smithi022 ceph-mon[37330]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:42:51.672 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:51 smithi022 ceph-mon[37330]: from='client.25738 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:42:51.672 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:51 smithi022 ceph-mon[37330]: osdmap e356: 8 total, 8 up, 8 in 2024-09-11T07:42:51.713 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:51.751 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:51.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:51.753 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:51.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:51 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:42:51.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:51 smithi104 ceph-mon[33507]: pgmap v757: 148 pgs: 148 active+clean; 3.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 119 MiB/s wr, 1.90k op/s 2024-09-11T07:42:51.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:51 smithi104 ceph-mon[33507]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:42:51.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:51 smithi104 ceph-mon[33507]: from='client.25738 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:42:51.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:51 smithi104 ceph-mon[33507]: osdmap e356: 8 total, 8 up, 8 in 2024-09-11T07:42:51.844 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is set 2024-09-11T07:42:51.853 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:51.891 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:51.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:51.892 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:51.956 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 36 16 67721 67705 117.525 82.875 0.0141327 0.00850408 2024-09-11T07:42:51.992 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:52.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:52.027 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:52.027 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:52.128 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:52.163 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:52.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:52.164 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:52.265 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:52.300 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:52.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:52.301 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:52.402 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:52.438 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:52.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:52.439 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:52.540 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:52.583 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:52.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:52.584 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:52.684 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:52.725 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:52.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:52.726 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:52.787 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-11T07:42:52.828 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:52.867 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:52.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:52.868 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:52.956 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 37 16 69206 69190 116.857 92.8125 0.00901218 0.00855236 2024-09-11T07:42:52.969 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:53.008 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:53.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:53.010 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:53.111 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:53.148 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:53.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:53.149 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:53.239 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:42:53.251 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:53.282 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:42:53.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:53.304 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:53.305 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:53.360 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:42:53.405 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:53.438 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:53.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:53.439 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:53.540 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:53.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:53.585 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:53.585 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:53.596 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:53 smithi022 ceph-mon[29462]: pgmap v759: 148 pgs: 148 active+clean; 3.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-11T07:42:53.596 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:42:53.597 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:53 smithi022 ceph-mon[37330]: pgmap v759: 148 pgs: 148 active+clean; 3.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-11T07:42:53.597 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:42:53.687 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:53.726 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:53.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:53.727 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:53.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:53 smithi104 ceph-mon[33507]: pgmap v759: 148 pgs: 148 active+clean; 3.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-11T07:42:53.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:42:53.828 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:53.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:53.859 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:53.859 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:53.956 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 38 16 70381 70365 115.714 73.4375 0.167693 0.00861346 2024-09-11T07:42:53.960 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:53.999 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:53.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:54.000 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:54.101 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:54.140 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:54.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:54.141 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:54.242 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:54.282 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:54.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:54.283 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:54.384 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:54.419 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:54.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:54.420 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:54.521 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:54.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:42:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:42:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:42:54.529 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:42:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=sqlstore.transactions t=2024-09-11T07:42:54.250639307Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T07:42:54.558 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:54.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:54.559 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:54.660 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:54.698 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:54.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:54.699 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:54.800 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:54.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:54.848 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:54.848 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:54.949 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:42:54.956 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 39 16 71793 71777 115.01 88.25 0.00979278 0.00869017 2024-09-11T07:42:54.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:54.995 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:54.996 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:42:55.096 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:55.137 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:55.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:55.138 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:55.239 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:55.294 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:55.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:55.295 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:55.327 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T07:42:55.397 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:55.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:55.433 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:55.433 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:55.534 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:55.561 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:55.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:55.562 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:55.663 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:55.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:55 smithi022 ceph-mon[29462]: pgmap v760: 148 pgs: 148 active+clean; 4.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 86 MiB/s wr, 1.37k op/s 2024-09-11T07:42:55.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:55 smithi022 ceph-mon[37330]: pgmap v760: 148 pgs: 148 active+clean; 4.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 86 MiB/s wr, 1.37k op/s 2024-09-11T07:42:55.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:55 smithi104 ceph-mon[33507]: pgmap v760: 148 pgs: 148 active+clean; 4.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 86 MiB/s wr, 1.37k op/s 2024-09-11T07:42:55.696 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:55.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:55.697 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:55.798 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:55.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:55.841 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:55.842 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:55.942 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:55.956 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:2024-09-11T07:42:55.955955+0000 min lat: 0.00106648 max lat: 0.324223 avg lat: 0.0087552 2024-09-11T07:42:55.956 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T07:42:55.956 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 40 16 73084 73068 114.151 80.6875 0.0100325 0.0087552 2024-09-11T07:42:55.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:55.990 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:55.990 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:56.091 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:56.125 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:56.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:56.126 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:56.227 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:56.270 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:56.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:56.271 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:56.372 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:56.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:56.408 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:56.408 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:56.509 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:56.548 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:56.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:56.549 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:56.650 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:56.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:56.681 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:56.682 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:56.783 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:56.812 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:42:56.814 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:42:56.814 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 356, 2024-09-11T07:42:56.814 INFO:teuthology.orchestra.run.smithi022.stdout: "fsid": "12a285d8-700e-11ef-bcea-c7b262605968", 2024-09-11T07:42:56.814 INFO:teuthology.orchestra.run.smithi022.stdout: "created": "2024-09-11T07:21:18.695332+0000", 2024-09-11T07:42:56.814 INFO:teuthology.orchestra.run.smithi022.stdout: "modified": "2024-09-11T07:42:50.345391+0000", 2024-09-11T07:42:56.814 INFO:teuthology.orchestra.run.smithi022.stdout: "last_up_change": "2024-09-11T07:32:00.070825+0000", 2024-09-11T07:42:56.814 INFO:teuthology.orchestra.run.smithi022.stdout: "last_in_change": "2024-09-11T07:25:58.551813+0000", 2024-09-11T07:42:56.815 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-11T07:42:56.815 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_num": 5806080, 2024-09-11T07:42:56.815 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_set": [ 2024-09-11T07:42:56.815 INFO:teuthology.orchestra.run.smithi022.stdout: "nodeep-scrub", 2024-09-11T07:42:56.815 INFO:teuthology.orchestra.run.smithi022.stdout: "noscrub", 2024-09-11T07:42:56.815 INFO:teuthology.orchestra.run.smithi022.stdout: "pglog_hardlimit", 2024-09-11T07:42:56.815 INFO:teuthology.orchestra.run.smithi022.stdout: "purged_snapdirs", 2024-09-11T07:42:56.815 INFO:teuthology.orchestra.run.smithi022.stdout: "recovery_deletes", 2024-09-11T07:42:56.815 INFO:teuthology.orchestra.run.smithi022.stdout: "sortbitwise" 2024-09-11T07:42:56.815 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:42:56.815 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_version": 19, 2024-09-11T07:42:56.815 INFO:teuthology.orchestra.run.smithi022.stdout: "full_ratio": 0.94999998807907104, 2024-09-11T07:42:56.815 INFO:teuthology.orchestra.run.smithi022.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-11T07:42:56.815 INFO:teuthology.orchestra.run.smithi022.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-11T07:42:56.816 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_snapshot": "", 2024-09-11T07:42:56.816 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_max": 10, 2024-09-11T07:42:56.816 INFO:teuthology.orchestra.run.smithi022.stdout: "max_osd": 8, 2024-09-11T07:42:56.816 INFO:teuthology.orchestra.run.smithi022.stdout: "require_min_compat_client": "luminous", 2024-09-11T07:42:56.816 INFO:teuthology.orchestra.run.smithi022.stdout: "min_compat_client": "luminous", 2024-09-11T07:42:56.816 INFO:teuthology.orchestra.run.smithi022.stdout: "require_osd_release": "quincy", 2024-09-11T07:42:56.816 INFO:teuthology.orchestra.run.smithi022.stdout: "pools": [ 2024-09-11T07:42:56.816 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.816 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 1, 2024-09-11T07:42:56.816 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": ".mgr", 2024-09-11T07:42:56.816 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:24:21.772426+0000", 2024-09-11T07:42:56.816 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:42:56.816 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:42:56.816 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:42:56.817 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:42:56.817 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:42:56.817 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:42:56.817 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:42:56.817 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:42:56.817 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:42:56.817 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:42:56.817 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:42:56.817 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:42:56.817 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 1, 2024-09-11T07:42:56.817 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 1, 2024-09-11T07:42:56.817 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 1, 2024-09-11T07:42:56.818 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 1, 2024-09-11T07:42:56.818 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 1, 2024-09-11T07:42:56.818 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:42:56.818 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:42:56.818 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:42:56.818 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:42:56.818 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:42:56.818 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:42:56.818 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:42:56.818 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.818 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "21", 2024-09-11T07:42:56.818 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:42:56.818 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:42:56.819 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:42:56.819 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:42:56.819 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:42:56.819 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:42:56.819 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:42:56.819 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:42:56.819 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:42:56.819 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:42:56.819 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:42:56.819 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:42:56.819 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:42:56.819 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:42:56.819 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:42:56.820 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:42:56.820 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:42:56.820 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:42:56.820 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:42:56.820 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:42:56.820 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:42:56.820 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:42:56.820 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:42:56.820 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:42:56.820 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:42:56.820 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:42:56.820 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.820 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:42:56.821 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:42:56.821 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:42:56.821 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:42:56.821 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:42:56.821 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:42:56.821 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:42:56.821 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:42:56.821 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:42:56.821 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:42:56.821 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:42:56.821 INFO:teuthology.orchestra.run.smithi022.stdout: "options": { 2024-09-11T07:42:56.821 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_max": 32, 2024-09-11T07:42:56.821 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_min": 1 2024-09-11T07:42:56.822 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.822 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:42:56.822 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": {} 2024-09-11T07:42:56.822 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.822 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.822 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.822 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 2, 2024-09-11T07:42:56.822 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": ".rgw.root", 2024-09-11T07:42:56.822 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:29.941354+0000", 2024-09-11T07:42:56.822 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:42:56.822 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:42:56.822 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:42:56.822 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:42:56.823 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:42:56.823 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:42:56.823 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:42:56.823 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:42:56.823 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:42:56.823 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:42:56.823 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:42:56.823 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:42:56.823 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:42:56.823 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:42:56.823 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:42:56.823 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:42:56.823 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:42:56.824 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:42:56.824 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:42:56.824 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:42:56.824 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:42:56.824 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:42:56.824 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:42:56.824 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:42:56.824 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.824 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "58", 2024-09-11T07:42:56.824 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:42:56.824 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:42:56.825 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:42:56.825 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:42:56.825 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:42:56.825 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:42:56.825 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:42:56.825 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:42:56.825 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:42:56.825 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:42:56.825 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:42:56.825 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:42:56.825 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:42:56.825 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:42:56.825 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:42:56.826 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:42:56.826 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:42:56.826 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:42:56.826 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:42:56.826 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:42:56.826 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:42:56.826 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:42:56.826 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:42:56.826 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:42:56.826 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:42:56.826 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:42:56.826 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.826 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:42:56.827 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:42:56.827 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:42:56.827 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:42:56.827 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:42:56.827 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:42:56.827 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:42:56.827 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:42:56.827 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:42:56.827 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:42:56.827 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:42:56.827 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:42:56.827 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:42:56.827 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:42:56.827 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.828 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.828 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.828 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 3, 2024-09-11T07:42:56.828 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "datapool", 2024-09-11T07:42:56.828 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:30.396859+0000", 2024-09-11T07:42:56.828 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 8193, 2024-09-11T07:42:56.828 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T07:42:56.828 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:42:56.828 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:42:56.828 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:42:56.828 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:42:56.828 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:42:56.828 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:42:56.829 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:42:56.829 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:42:56.829 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:42:56.829 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:42:56.829 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 3, 2024-09-11T07:42:56.829 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 3, 2024-09-11T07:42:56.829 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 3, 2024-09-11T07:42:56.829 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 3, 2024-09-11T07:42:56.829 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 3, 2024-09-11T07:42:56.829 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:42:56.829 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:42:56.829 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:42:56.829 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:42:56.830 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:42:56.830 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:42:56.830 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:42:56.830 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.830 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "61", 2024-09-11T07:42:56.830 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:42:56.830 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:42:56.830 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:42:56.830 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:42:56.830 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:42:56.830 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 3, 2024-09-11T07:42:56.830 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 61, 2024-09-11T07:42:56.830 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:42:56.830 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:42:56.831 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:42:56.831 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:42:56.831 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:42:56.831 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:42:56.831 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:42:56.831 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:42:56.831 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:42:56.831 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:42:56.831 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:42:56.831 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:42:56.831 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:42:56.831 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:42:56.831 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:42:56.832 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:42:56.832 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:42:56.832 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:42:56.832 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:42:56.833 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.833 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:42:56.833 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:42:56.833 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:42:56.833 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:42:56.833 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:42:56.833 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:42:56.833 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:42:56.833 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:42:56.834 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:42:56.834 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:42:56.834 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:42:56.834 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:42:56.834 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:42:56.834 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd": {} 2024-09-11T07:42:56.834 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.834 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.834 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.834 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 4, 2024-09-11T07:42:56.834 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.log", 2024-09-11T07:42:56.834 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:31.922926+0000", 2024-09-11T07:42:56.834 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:42:56.834 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:42:56.835 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:42:56.835 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:42:56.835 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:42:56.835 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:42:56.835 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:42:56.835 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:42:56.835 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:42:56.835 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:42:56.835 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:42:56.835 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:42:56.835 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:42:56.835 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:42:56.835 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:42:56.835 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:42:56.835 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:42:56.836 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:42:56.836 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:42:56.836 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:42:56.836 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:42:56.836 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:42:56.836 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:42:56.836 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:42:56.836 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.836 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "60", 2024-09-11T07:42:56.836 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:42:56.836 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:42:56.836 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:42:56.836 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:42:56.836 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:42:56.837 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:42:56.837 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:42:56.837 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:42:56.837 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:42:56.837 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:42:56.837 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:42:56.837 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:42:56.837 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:42:56.837 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:42:56.837 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:42:56.837 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:42:56.837 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:42:56.837 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:42:56.837 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:42:56.838 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:42:56.838 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:42:56.838 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:42:56.838 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:42:56.838 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:42:56.838 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:42:56.838 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:42:56.838 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.838 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:42:56.838 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:42:56.838 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:42:56.838 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:42:56.838 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:42:56.838 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:42:56.839 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:42:56.839 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:42:56.839 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:42:56.839 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:42:56.839 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:42:56.839 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:42:56.839 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:42:56.839 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:42:56.839 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.839 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.839 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.839 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 5, 2024-09-11T07:42:56.839 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.control", 2024-09-11T07:42:56.839 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:34.007746+0000", 2024-09-11T07:42:56.840 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:42:56.840 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:42:56.840 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:42:56.840 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:42:56.840 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:42:56.840 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:42:56.840 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:42:56.840 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:42:56.840 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:42:56.840 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:42:56.840 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:42:56.840 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:42:56.840 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:42:56.841 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:42:56.841 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:42:56.841 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:42:56.841 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:42:56.841 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:42:56.841 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:42:56.841 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:42:56.841 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:42:56.841 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:42:56.841 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:42:56.841 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:42:56.841 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.841 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "62", 2024-09-11T07:42:56.841 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:42:56.841 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:42:56.842 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:42:56.842 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:42:56.842 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:42:56.842 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:42:56.842 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:42:56.842 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:42:56.842 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:42:56.842 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:42:56.842 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:42:56.842 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:42:56.842 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:42:56.842 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:42:56.842 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:42:56.842 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:42:56.843 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:42:56.843 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:42:56.843 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:42:56.843 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:42:56.843 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:42:56.843 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:42:56.843 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:42:56.843 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:42:56.843 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:42:56.843 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:42:56.843 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.843 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:42:56.843 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:42:56.843 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:42:56.843 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:42:56.844 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:42:56.844 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:42:56.844 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:42:56.844 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:42:56.844 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:42:56.844 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:42:56.844 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:42:56.844 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:42:56.844 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:42:56.844 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:42:56.844 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.844 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.844 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.844 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 6, 2024-09-11T07:42:56.845 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.meta", 2024-09-11T07:42:56.845 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:36.077215+0000", 2024-09-11T07:42:56.845 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:42:56.845 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:42:56.845 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:42:56.845 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:42:56.845 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:42:56.845 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:42:56.845 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:42:56.845 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:42:56.845 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:42:56.845 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:42:56.845 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:42:56.845 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:42:56.846 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:42:56.846 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:42:56.846 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:42:56.846 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:42:56.846 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:42:56.846 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:42:56.846 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:42:56.846 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:42:56.846 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:42:56.846 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:42:56.846 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:42:56.846 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:42:56.846 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.846 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "64", 2024-09-11T07:42:56.847 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:42:56.847 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:42:56.847 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:42:56.847 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:42:56.847 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:42:56.847 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:42:56.847 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:42:56.847 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:42:56.847 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:42:56.847 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:42:56.847 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:42:56.847 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:42:56.847 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:42:56.847 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:42:56.847 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:42:56.847 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:42:56.848 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:42:56.848 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:42:56.848 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:42:56.848 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:42:56.848 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:42:56.848 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:42:56.848 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:42:56.848 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:42:56.848 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:42:56.848 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:42:56.848 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.848 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:42:56.848 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:42:56.848 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:42:56.848 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:42:56.849 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:42:56.849 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:42:56.849 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:42:56.849 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:42:56.849 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:42:56.849 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:42:56.849 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:42:56.849 INFO:teuthology.orchestra.run.smithi022.stdout: "options": { 2024-09-11T07:42:56.849 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_bias": 4 2024-09-11T07:42:56.849 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.849 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:42:56.849 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:42:56.849 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.849 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.850 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.850 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 10, 2024-09-11T07:42:56.850 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "unique_pool_3", 2024-09-11T07:42:56.850 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:41:56.278857+0000", 2024-09-11T07:42:56.850 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:42:56.850 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:42:56.850 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:42:56.850 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:42:56.850 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:42:56.850 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:42:56.850 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:42:56.850 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:42:56.850 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:42:56.850 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:42:56.850 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:42:56.851 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:42:56.851 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 16, 2024-09-11T07:42:56.851 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 16, 2024-09-11T07:42:56.851 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 16, 2024-09-11T07:42:56.851 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 16, 2024-09-11T07:42:56.851 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 16, 2024-09-11T07:42:56.851 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:42:56.851 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:42:56.851 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:42:56.851 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:42:56.851 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:42:56.851 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:42:56.851 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:42:56.851 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.851 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "349", 2024-09-11T07:42:56.851 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:42:56.852 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:42:56.852 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:42:56.852 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:42:56.852 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:42:56.852 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:42:56.852 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:42:56.852 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:42:56.852 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:42:56.852 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:42:56.852 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:42:56.852 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:42:56.852 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:42:56.852 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:42:56.852 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:42:56.852 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:42:56.853 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:42:56.853 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:42:56.853 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:42:56.853 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:42:56.853 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:42:56.853 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:42:56.853 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:42:56.853 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:42:56.853 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:42:56.853 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:42:56.853 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.853 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:42:56.853 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:42:56.853 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:42:56.853 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:42:56.853 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:42:56.854 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:42:56.854 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:42:56.854 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:42:56.854 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:42:56.854 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:42:56.854 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:42:56.854 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:42:56.854 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:42:56.854 INFO:teuthology.orchestra.run.smithi022.stdout: "rados": {} 2024-09-11T07:42:56.854 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.854 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.854 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:42:56.854 INFO:teuthology.orchestra.run.smithi022.stdout: "osds": [ 2024-09-11T07:42:56.854 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.854 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 0, 2024-09-11T07:42:56.855 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "09075f16-32f4-4f88-a116-c3d7f977a0e0", 2024-09-11T07:42:56.855 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:42:56.855 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:42:56.855 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:42:56.855 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:42:56.855 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:42:56.855 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:42:56.855 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 8, 2024-09-11T07:42:56.855 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 346, 2024-09-11T07:42:56.855 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:42:56.855 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:42:56.855 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:42:56.855 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.855 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.855 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.856 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6801", 2024-09-11T07:42:56.856 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:42:56.856 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.856 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.856 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.856 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:42:56.856 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.856 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.856 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.856 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6802", 2024-09-11T07:42:56.856 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:42:56.856 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.856 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.856 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.857 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:42:56.857 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.857 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.857 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.857 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6804", 2024-09-11T07:42:56.857 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:42:56.857 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.857 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.857 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.857 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:42:56.857 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.857 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.857 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.857 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6803", 2024-09-11T07:42:56.857 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:42:56.858 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.858 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.858 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.858 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6801/2821493591", 2024-09-11T07:42:56.858 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6802/2821493591", 2024-09-11T07:42:56.858 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6804/2821493591", 2024-09-11T07:42:56.858 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6803/2821493591", 2024-09-11T07:42:56.858 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:42:56.858 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:42:56.858 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:42:56.858 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.858 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.858 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.858 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 1, 2024-09-11T07:42:56.858 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "52cc1e06-9506-48c0-9f7b-d4270dff4b9f", 2024-09-11T07:42:56.859 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:42:56.859 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:42:56.859 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 0.64752197265625, 2024-09-11T07:42:56.859 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:42:56.859 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:42:56.859 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:42:56.859 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 13, 2024-09-11T07:42:56.859 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 346, 2024-09-11T07:42:56.859 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:42:56.859 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:42:56.859 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:42:56.859 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.859 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.859 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.859 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6805", 2024-09-11T07:42:56.860 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:42:56.860 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.860 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.860 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.860 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:42:56.860 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.860 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.860 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.860 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6806", 2024-09-11T07:42:56.860 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:42:56.860 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.860 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.860 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.860 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:42:56.860 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.860 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.861 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.861 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6808", 2024-09-11T07:42:56.861 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:42:56.861 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.861 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.861 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.861 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:42:56.861 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.861 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.861 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.861 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6807", 2024-09-11T07:42:56.861 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:42:56.861 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.861 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.862 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.862 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6805/4037911317", 2024-09-11T07:42:56.862 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6806/4037911317", 2024-09-11T07:42:56.862 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6808/4037911317", 2024-09-11T07:42:56.862 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6807/4037911317", 2024-09-11T07:42:56.862 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:42:56.862 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:42:56.862 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:42:56.862 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.862 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.862 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.862 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 2, 2024-09-11T07:42:56.862 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "fa2b2730-2c1d-45d0-be8a-494a60dc6b9b", 2024-09-11T07:42:56.862 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:42:56.862 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:42:56.862 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:42:56.863 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:42:56.863 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:42:56.863 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:42:56.863 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 18, 2024-09-11T07:42:56.863 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 341, 2024-09-11T07:42:56.863 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:42:56.863 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:42:56.863 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:42:56.863 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.863 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.863 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.863 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6809", 2024-09-11T07:42:56.863 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:42:56.863 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.863 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.863 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.864 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:42:56.864 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.864 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.864 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.864 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6810", 2024-09-11T07:42:56.864 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:42:56.864 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.864 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.864 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.864 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:42:56.864 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.864 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.864 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.864 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6812", 2024-09-11T07:42:56.864 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:42:56.864 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.865 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.865 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.865 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:42:56.865 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.865 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.865 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.865 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6811", 2024-09-11T07:42:56.865 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:42:56.865 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.865 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.865 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.865 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6809/1675799666", 2024-09-11T07:42:56.865 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6810/1675799666", 2024-09-11T07:42:56.865 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6812/1675799666", 2024-09-11T07:42:56.865 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6811/1675799666", 2024-09-11T07:42:56.866 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:42:56.866 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:42:56.866 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:42:56.866 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.866 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:56.867 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:56.867 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:56.868 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.868 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 3, 2024-09-11T07:42:56.868 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "8d53d3fc-e401-47df-b70b-4b4b1f4539df", 2024-09-11T07:42:56.868 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:42:56.868 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:42:56.868 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 0.686370849609375, 2024-09-11T07:42:56.868 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:42:56.868 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:42:56.868 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:42:56.868 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 25, 2024-09-11T07:42:56.868 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 341, 2024-09-11T07:42:56.868 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:42:56.868 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:42:56.869 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:42:56.869 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.869 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.869 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.869 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6813", 2024-09-11T07:42:56.869 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:42:56.869 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.869 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.869 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.869 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:42:56.869 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.869 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.869 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.870 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6814", 2024-09-11T07:42:56.870 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:42:56.870 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.870 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.870 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.870 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:42:56.870 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.870 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.870 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.870 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6816", 2024-09-11T07:42:56.870 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:42:56.870 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.870 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.871 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.871 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:42:56.871 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.871 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.871 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.871 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6815", 2024-09-11T07:42:56.871 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:42:56.871 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.871 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.871 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.871 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6813/1501962752", 2024-09-11T07:42:56.871 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6814/1501962752", 2024-09-11T07:42:56.871 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6816/1501962752", 2024-09-11T07:42:56.872 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6815/1501962752", 2024-09-11T07:42:56.872 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:42:56.872 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:42:56.872 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:42:56.872 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.872 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.872 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.872 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 4, 2024-09-11T07:42:56.872 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "d3207fe0-8b97-4067-a755-ba6d449c53c7", 2024-09-11T07:42:56.872 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:42:56.872 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:42:56.872 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:42:56.872 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:42:56.873 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:42:56.873 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:42:56.873 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 31, 2024-09-11T07:42:56.873 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 346, 2024-09-11T07:42:56.873 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:42:56.873 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:42:56.873 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:42:56.873 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.873 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.873 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.873 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6800", 2024-09-11T07:42:56.873 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:42:56.873 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.873 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.873 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.874 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:42:56.874 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.874 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.874 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.874 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6801", 2024-09-11T07:42:56.874 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:42:56.874 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.874 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.874 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.874 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:42:56.874 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.874 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.874 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.874 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6803", 2024-09-11T07:42:56.874 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:42:56.875 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.875 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.875 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.875 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:42:56.875 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.875 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.875 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.875 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6802", 2024-09-11T07:42:56.875 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:42:56.875 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.875 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.875 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.875 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6800/1210229443", 2024-09-11T07:42:56.875 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6801/1210229443", 2024-09-11T07:42:56.875 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6803/1210229443", 2024-09-11T07:42:56.876 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6802/1210229443", 2024-09-11T07:42:56.876 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:42:56.876 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:42:56.876 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:42:56.876 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.876 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.876 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.876 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 5, 2024-09-11T07:42:56.876 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "89c9352b-30b2-48c6-869e-047d07a23a43", 2024-09-11T07:42:56.876 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:42:56.876 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:42:56.876 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 0.6509552001953125, 2024-09-11T07:42:56.876 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 0.666046142578125, 2024-09-11T07:42:56.877 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:42:56.877 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:42:56.877 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 37, 2024-09-11T07:42:56.877 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 341, 2024-09-11T07:42:56.877 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:42:56.877 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:42:56.877 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:42:56.877 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.877 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.877 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.877 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6804", 2024-09-11T07:42:56.877 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:42:56.877 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.877 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.878 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.878 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:42:56.878 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.878 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.878 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.878 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6805", 2024-09-11T07:42:56.878 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:42:56.878 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.878 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.878 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.878 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:42:56.878 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.878 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.878 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.878 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6807", 2024-09-11T07:42:56.878 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:42:56.879 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.879 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.879 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.879 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:42:56.879 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.879 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.879 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.879 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6806", 2024-09-11T07:42:56.879 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:42:56.879 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.879 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.879 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.879 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6804/2735328732", 2024-09-11T07:42:56.879 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6805/2735328732", 2024-09-11T07:42:56.880 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6807/2735328732", 2024-09-11T07:42:56.880 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6806/2735328732", 2024-09-11T07:42:56.880 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:42:56.880 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:42:56.880 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:42:56.880 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.880 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.880 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.880 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 6, 2024-09-11T07:42:56.880 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "704563de-d88f-480a-b469-720135704d42", 2024-09-11T07:42:56.880 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:42:56.880 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:42:56.880 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:42:56.880 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 0, 2024-09-11T07:42:56.880 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:42:56.881 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:42:56.881 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 44, 2024-09-11T07:42:56.881 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 177, 2024-09-11T07:42:56.881 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:42:56.881 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:42:56.881 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:42:56.881 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.881 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.881 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.881 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6808", 2024-09-11T07:42:56.881 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:42:56.881 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.881 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.882 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.882 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:42:56.882 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.882 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.882 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.882 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6809", 2024-09-11T07:42:56.882 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:42:56.882 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.882 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.882 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.882 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:42:56.882 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.882 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.883 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.883 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6811", 2024-09-11T07:42:56.883 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:42:56.883 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.883 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.883 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.883 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:42:56.883 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.883 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.884 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.884 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6810", 2024-09-11T07:42:56.884 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:42:56.884 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.884 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.884 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.884 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6808/721493010", 2024-09-11T07:42:56.884 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6809/721493010", 2024-09-11T07:42:56.884 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6811/721493010", 2024-09-11T07:42:56.884 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6810/721493010", 2024-09-11T07:42:56.884 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:42:56.884 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:42:56.884 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:42:56.884 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.885 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.885 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.885 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 7, 2024-09-11T07:42:56.885 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "18199e47-0052-49f0-b45f-0fb201e0d0f4", 2024-09-11T07:42:56.885 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:42:56.885 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:42:56.885 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 0.6657562255859375, 2024-09-11T07:42:56.885 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:42:56.885 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 52, 2024-09-11T07:42:56.885 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 133, 2024-09-11T07:42:56.885 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 147, 2024-09-11T07:42:56.885 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 341, 2024-09-11T07:42:56.885 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 134, 2024-09-11T07:42:56.886 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:42:56.886 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:42:56.886 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.886 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.886 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.886 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6812", 2024-09-11T07:42:56.886 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2218863853 2024-09-11T07:42:56.886 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.886 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.886 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.886 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:42:56.886 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.886 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.886 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.887 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6813", 2024-09-11T07:42:56.887 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2218863853 2024-09-11T07:42:56.887 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.887 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.887 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.887 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:42:56.887 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.887 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.887 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.887 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6815", 2024-09-11T07:42:56.887 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2218863853 2024-09-11T07:42:56.887 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.887 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.887 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.888 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:42:56.888 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:42:56.888 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.888 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:42:56.888 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6814", 2024-09-11T07:42:56.888 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2218863853 2024-09-11T07:42:56.888 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.888 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.888 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.888 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6812/2218863853", 2024-09-11T07:42:56.888 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6813/2218863853", 2024-09-11T07:42:56.888 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6815/2218863853", 2024-09-11T07:42:56.888 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6814/2218863853", 2024-09-11T07:42:56.889 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:42:56.889 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:42:56.889 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:42:56.889 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.889 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.889 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:42:56.889 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_xinfo": [ 2024-09-11T07:42:56.889 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.889 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 0, 2024-09-11T07:42:56.889 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:42:56.889 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:42:56.889 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:42:56.889 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:42:56.890 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:42:56.890 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:42:05.640226+0000", 2024-09-11T07:42:56.890 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:42:56.890 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.890 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.890 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 1, 2024-09-11T07:42:56.890 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:42:56.890 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:42:56.890 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:42:56.890 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:42:56.890 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:42:56.890 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:42:00.726790+0000", 2024-09-11T07:42:56.890 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:42:56.890 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.890 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.891 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 2, 2024-09-11T07:42:56.891 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:42:56.891 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:42:56.891 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:42:56.891 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:42:56.891 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:42:56.891 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:42:00.226145+0000", 2024-09-11T07:42:56.891 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:42:56.891 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.891 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.891 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 3, 2024-09-11T07:42:56.891 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:42:56.891 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:42:56.891 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:42:56.892 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:42:56.892 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:42:56.892 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:42:18.279268+0000", 2024-09-11T07:42:56.892 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:42:56.892 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.892 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.892 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 4, 2024-09-11T07:42:56.892 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:42:56.892 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:42:56.892 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:42:56.892 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:42:56.892 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:42:56.892 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:42:03.255095+0000", 2024-09-11T07:42:56.893 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:42:56.893 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.893 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.893 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 5, 2024-09-11T07:42:56.893 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:42:56.893 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:42:56.893 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:42:56.893 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:42:56.893 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:42:56.893 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:42:01.276730+0000", 2024-09-11T07:42:56.893 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:42:56.893 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.893 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.893 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 6, 2024-09-11T07:42:56.894 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:42:56.894 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:42:56.894 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:42:56.894 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:42:56.894 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:42:56.894 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:41:56.711182+0000", 2024-09-11T07:42:56.894 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:42:56.894 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.894 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.894 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 7, 2024-09-11T07:42:56.894 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:42:56.894 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:42:56.894 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:42:56.894 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:42:56.894 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:42:56.895 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:42:20.744248+0000", 2024-09-11T07:42:56.895 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 133 2024-09-11T07:42:56.895 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.895 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:42:56.895 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_upmap": [ 2024-09-11T07:42:56.895 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.895 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "2.1c", 2024-09-11T07:42:56.895 INFO:teuthology.orchestra.run.smithi022.stdout: "osds": [ 2024-09-11T07:42:56.895 INFO:teuthology.orchestra.run.smithi022.stdout: 5, 2024-09-11T07:42:56.895 INFO:teuthology.orchestra.run.smithi022.stdout: 2, 2024-09-11T07:42:56.895 INFO:teuthology.orchestra.run.smithi022.stdout: 7 2024-09-11T07:42:56.895 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.895 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.895 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.895 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.1e", 2024-09-11T07:42:56.895 INFO:teuthology.orchestra.run.smithi022.stdout: "osds": [ 2024-09-11T07:42:56.896 INFO:teuthology.orchestra.run.smithi022.stdout: 4, 2024-09-11T07:42:56.896 INFO:teuthology.orchestra.run.smithi022.stdout: 5, 2024-09-11T07:42:56.896 INFO:teuthology.orchestra.run.smithi022.stdout: 3 2024-09-11T07:42:56.896 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.896 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.896 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:42:56.896 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_upmap_items": [ 2024-09-11T07:42:56.896 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.896 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.6", 2024-09-11T07:42:56.896 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:42:56.896 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:42:56.896 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 1, 2024-09-11T07:42:56.896 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 5 2024-09-11T07:42:56.897 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.897 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:42:56.897 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.897 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:42:56.897 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_temp": [], 2024-09-11T07:42:56.897 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_temp": [], 2024-09-11T07:42:56.897 INFO:teuthology.orchestra.run.smithi022.stdout: "blocklist": { 2024-09-11T07:42:56.897 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/240999923": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:42:56.897 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/4016369460": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:42:56.897 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/585929238": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:42:56.897 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/544031024": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:42:56.897 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/2504196132": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:42:56.897 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/1451388349": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:42:56.898 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/639336081": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:42:56.898 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2104069730": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:42:56.898 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/4236450731": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:42:56.898 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/3573193776": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:42:56.898 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2661764422": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:42:56.898 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2197488530": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:42:56.898 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2968034970": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:42:56.898 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/599318175": "2024-09-12T07:27:07.919998+0000" 2024-09-11T07:42:56.898 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.898 INFO:teuthology.orchestra.run.smithi022.stdout: "range_blocklist": {}, 2024-09-11T07:42:56.898 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profiles": { 2024-09-11T07:42:56.898 INFO:teuthology.orchestra.run.smithi022.stdout: "default": { 2024-09-11T07:42:56.898 INFO:teuthology.orchestra.run.smithi022.stdout: "crush-failure-domain": "osd", 2024-09-11T07:42:56.898 INFO:teuthology.orchestra.run.smithi022.stdout: "k": "2", 2024-09-11T07:42:56.898 INFO:teuthology.orchestra.run.smithi022.stdout: "m": "1", 2024-09-11T07:42:56.899 INFO:teuthology.orchestra.run.smithi022.stdout: "plugin": "jerasure", 2024-09-11T07:42:56.899 INFO:teuthology.orchestra.run.smithi022.stdout: "technique": "reed_sol_van" 2024-09-11T07:42:56.899 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.899 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:42:56.899 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps_queue": [], 2024-09-11T07:42:56.899 INFO:teuthology.orchestra.run.smithi022.stdout: "new_removed_snaps": [], 2024-09-11T07:42:56.899 INFO:teuthology.orchestra.run.smithi022.stdout: "new_purged_snaps": [], 2024-09-11T07:42:56.899 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_node_flags": {}, 2024-09-11T07:42:56.899 INFO:teuthology.orchestra.run.smithi022.stdout: "device_class_flags": {}, 2024-09-11T07:42:56.899 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode": { 2024-09-11T07:42:56.899 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode_enabled": false, 2024-09-11T07:42:56.899 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_bucket_count": 0, 2024-09-11T07:42:56.899 INFO:teuthology.orchestra.run.smithi022.stdout: "degraded_stretch_mode": 0, 2024-09-11T07:42:56.899 INFO:teuthology.orchestra.run.smithi022.stdout: "recovering_stretch_mode": 0, 2024-09-11T07:42:56.899 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode_bucket": 0 2024-09-11T07:42:56.899 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:42:56.900 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:42:56.956 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 41 16 74295 74279 113.213 75.6875 0.0109982 0.0088282 2024-09-11T07:42:56.968 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:57.006 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:57.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:57.007 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:57.025 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:56 smithi022 ceph-mon[29462]: pgmap v761: 148 pgs: 148 active+clean; 4.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 85 MiB/s wr, 1.35k op/s 2024-09-11T07:42:57.026 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:56 smithi022 ceph-mon[37330]: pgmap v761: 148 pgs: 148 active+clean; 4.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 85 MiB/s wr, 1.35k op/s 2024-09-11T07:42:57.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:56 smithi104 ceph-mon[33507]: pgmap v761: 148 pgs: 148 active+clean; 4.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 85 MiB/s wr, 1.35k op/s 2024-09-11T07:42:57.108 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:57.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:57.148 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:57.148 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:57.249 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:57.281 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:57.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:57.282 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:57.383 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:57.424 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:57.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:57.425 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:57.526 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:57.566 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:57.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:57.567 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:57.668 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:57.702 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:57.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:57.703 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:57.804 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:57.839 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:57.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:57.840 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:57.895 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:42:57.896 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2024-09-11T07:42:57.896 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.896 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:75524 10.3 e356/341 10:cfa88a1b:::benchmark_data_smithi022_163866_object75523:head v 356'4638, mlcod=356'4636)", 2024-09-11T07:42:57.896 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:57.890730+0000", 2024-09-11T07:42:57.896 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.0014446540000000001, 2024-09-11T07:42:57.896 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0014724969999999999, 2024-09-11T07:42:57.896 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:42:57.896 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2024-09-11T07:42:57.896 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:42:57.896 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.896 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:42:57.896 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.890730+0000", 2024-09-11T07:42:57.896 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:42:57.896 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.897 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.897 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:42:57.897 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.890730+0000", 2024-09-11T07:42:57.897 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:42:57.897 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.897 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.897 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:42:57.897 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.890729+0000", 2024-09-11T07:42:57.897 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:42:57.897 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.897 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.897 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:42:57.897 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.890813+0000", 2024-09-11T07:42:57.897 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3912000000000006e-05 2024-09-11T07:42:57.898 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.898 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.898 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:42:57.898 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.890831+0000", 2024-09-11T07:42:57.898 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7872000000000001e-05 2024-09-11T07:42:57.898 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.898 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.898 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:42:57.898 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.890836+0000", 2024-09-11T07:42:57.898 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2959999999999998e-06 2024-09-11T07:42:57.898 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.898 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.898 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:42:57.898 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.890881+0000", 2024-09-11T07:42:57.898 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5176000000000003e-05 2024-09-11T07:42:57.899 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.899 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.899 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:42:57.899 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.890899+0000", 2024-09-11T07:42:57.899 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7343e-05 2024-09-11T07:42:57.899 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:42:57.899 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:42:57.899 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:42:57.899 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.899 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.899 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:75526 10.0 e356/341 10:094ccb79:::benchmark_data_smithi022_163866_object75525:head v 356'4663, mlcod=356'4661)", 2024-09-11T07:42:57.899 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:57.891315+0000", 2024-09-11T07:42:57.899 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.00085956800000000001, 2024-09-11T07:42:57.899 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00095929999999999995, 2024-09-11T07:42:57.900 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:42:57.900 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2024-09-11T07:42:57.900 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:42:57.900 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.900 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:42:57.900 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.891315+0000", 2024-09-11T07:42:57.900 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:42:57.900 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.900 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.900 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:42:57.900 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.891315+0000", 2024-09-11T07:42:57.900 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:42:57.900 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.901 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.901 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:42:57.901 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.891314+0000", 2024-09-11T07:42:57.901 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:42:57.901 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.901 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.901 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:42:57.901 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.891384+0000", 2024-09-11T07:42:57.901 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0131999999999999e-05 2024-09-11T07:42:57.901 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.901 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.901 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:42:57.901 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.891395+0000", 2024-09-11T07:42:57.901 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0808999999999999e-05 2024-09-11T07:42:57.902 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.902 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.902 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:42:57.902 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.891400+0000", 2024-09-11T07:42:57.902 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8019999999999999e-06 2024-09-11T07:42:57.902 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.902 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.902 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:42:57.902 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.891447+0000", 2024-09-11T07:42:57.902 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6856e-05 2024-09-11T07:42:57.902 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.902 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.902 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:42:57.903 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.891467+0000", 2024-09-11T07:42:57.903 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9660000000000002e-05 2024-09-11T07:42:57.903 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:42:57.903 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:42:57.903 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:42:57.903 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.903 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.903 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:75525 10.c e356/341 10:3835b2a9:::benchmark_data_smithi022_163866_object75524:head v 356'4800, mlcod=356'4797)", 2024-09-11T07:42:57.903 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:57.891546+0000", 2024-09-11T07:42:57.903 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.00062892099999999995, 2024-09-11T07:42:57.903 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00078171999999999996, 2024-09-11T07:42:57.903 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:42:57.903 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2024-09-11T07:42:57.903 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:42:57.904 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.904 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:42:57.904 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.891546+0000", 2024-09-11T07:42:57.904 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:42:57.904 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.904 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.904 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:42:57.904 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.891546+0000", 2024-09-11T07:42:57.904 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:42:57.904 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.904 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.904 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:42:57.904 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.891545+0000", 2024-09-11T07:42:57.904 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:42:57.904 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.905 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.905 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:42:57.905 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.891568+0000", 2024-09-11T07:42:57.905 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.2918999999999999e-05 2024-09-11T07:42:57.905 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.905 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.905 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:42:57.905 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.891576+0000", 2024-09-11T07:42:57.905 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1340000000000004e-06 2024-09-11T07:42:57.905 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.905 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.905 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:42:57.905 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.891581+0000", 2024-09-11T07:42:57.905 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.746e-06 2024-09-11T07:42:57.906 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.906 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.906 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:42:57.906 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.891661+0000", 2024-09-11T07:42:57.906 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9950000000000005e-05 2024-09-11T07:42:57.906 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.906 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.906 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:42:57.906 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.891677+0000", 2024-09-11T07:42:57.907 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6509999999999999e-05 2024-09-11T07:42:57.907 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:42:57.907 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:42:57.907 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:42:57.907 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.907 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.907 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:75521 10.9 e356/341 10:905ae8e9:::benchmark_data_smithi022_163866_object75520:head v 356'4702, mlcod=356'4697)", 2024-09-11T07:42:57.907 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:57.887184+0000", 2024-09-11T07:42:57.907 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.0049909469999999999, 2024-09-11T07:42:57.907 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0051966240000000004, 2024-09-11T07:42:57.907 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:42:57.907 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:42:57.907 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:42:57.907 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.907 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:42:57.908 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.887184+0000", 2024-09-11T07:42:57.908 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:42:57.908 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.908 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.908 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:42:57.908 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.887184+0000", 2024-09-11T07:42:57.908 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:42:57.908 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.908 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.908 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:42:57.908 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.887183+0000", 2024-09-11T07:42:57.908 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:42:57.908 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.909 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.909 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:42:57.909 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.887214+0000", 2024-09-11T07:42:57.909 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.0545e-05 2024-09-11T07:42:57.909 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.909 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.909 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:42:57.909 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.887220+0000", 2024-09-11T07:42:57.909 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9089999999999997e-06 2024-09-11T07:42:57.909 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.909 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.909 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:42:57.909 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.887223+0000", 2024-09-11T07:42:57.909 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.608e-06 2024-09-11T07:42:57.910 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.910 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.910 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:42:57.910 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.889466+0000", 2024-09-11T07:42:57.910 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002242427 2024-09-11T07:42:57.910 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.910 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.910 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:42:57.910 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.889484+0000", 2024-09-11T07:42:57.910 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8428e-05 2024-09-11T07:42:57.910 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.910 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.910 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:42:57.910 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.892376+0000", 2024-09-11T07:42:57.911 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002891294 2024-09-11T07:42:57.911 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:42:57.911 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:42:57.911 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:42:57.911 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.911 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.911 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25706.0:75522 10.e 10:706a2710:::benchmark_data_smithi022_163866_object75521:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e356)", 2024-09-11T07:42:57.911 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:57.887848+0000", 2024-09-11T07:42:57.911 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.0043269780000000004, 2024-09-11T07:42:57.911 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0045969280000000001, 2024-09-11T07:42:57.911 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:42:57.911 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2024-09-11T07:42:57.911 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:42:57.911 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25706", 2024-09-11T07:42:57.911 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:42:57.912 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 75522 2024-09-11T07:42:57.912 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.912 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:42:57.912 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.912 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:42:57.912 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.887848+0000", 2024-09-11T07:42:57.912 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:42:57.912 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.912 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.912 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:42:57.912 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.887848+0000", 2024-09-11T07:42:57.912 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:42:57.912 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.913 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.913 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:42:57.913 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.887846+0000", 2024-09-11T07:42:57.913 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:42:57.913 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.913 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.913 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:42:57.913 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.887903+0000", 2024-09-11T07:42:57.913 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7102999999999999e-05 2024-09-11T07:42:57.913 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.913 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.913 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:42:57.914 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.887916+0000", 2024-09-11T07:42:57.914 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2608e-05 2024-09-11T07:42:57.914 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.914 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.914 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:42:57.914 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.887921+0000", 2024-09-11T07:42:57.914 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3689999999999998e-06 2024-09-11T07:42:57.914 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.914 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.914 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:42:57.914 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.887977+0000", 2024-09-11T07:42:57.914 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6511000000000002e-05 2024-09-11T07:42:57.914 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.914 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.914 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:42:57.915 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.888100+0000", 2024-09-11T07:42:57.915 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012249 2024-09-11T07:42:57.915 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.915 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.915 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:42:57.915 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.888240+0000", 2024-09-11T07:42:57.915 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014049900000000001 2024-09-11T07:42:57.915 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.915 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.915 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:57.915 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.891904+0000", 2024-09-11T07:42:57.915 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0036632460000000002 2024-09-11T07:42:57.915 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:42:57.915 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:42:57.916 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:42:57.916 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:57.891931+0000", 2024-09-11T07:42:57.916 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.6908999999999999e-05 2024-09-11T07:42:57.916 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:42:57.916 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:42:57.916 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:42:57.916 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:42:57.916 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2024-09-11T07:42:57.916 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 5 2024-09-11T07:42:57.916 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:42:57.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:57 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1376861050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:42:57.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:57 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1376861050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:42:57.941 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:57.956 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 42 16 75606 75590 112.468 81.9375 0.0128599 0.00888624 2024-09-11T07:42:57.975 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:57.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:57.976 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:58.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:57 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1376861050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:42:58.078 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:58.116 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:58.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:58.118 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:58.159 INFO:tasks.thrashosds.thrasher:j is {'epoch': 356, 'fsid': '12a285d8-700e-11ef-bcea-c7b262605968', 'created': '2024-09-11T07:21:18.695332+0000', 'modified': '2024-09-11T07:42:50.345391+0000', 'last_up_change': '2024-09-11T07:32:00.070825+0000', 'last_in_change': '2024-09-11T07:25:58.551813+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 19, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 10, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T07:24:21.772426+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': '.rgw.root', 'create_time': '2024-09-11T07:26:29.941354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 3, 'pool_name': 'datapool', 'create_time': '2024-09-11T07:26:30.396859+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-11T07:26:31.922926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-11T07:26:34.007746+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-11T07:26:36.077215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}, {'pool': 10, 'pool_name': 'unique_pool_3', 'create_time': '2024-09-11T07:41:56.278857+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '349', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '09075f16-32f4-4f88-a116-c3d7f977a0e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 346, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6801', 'nonce': 2821493591}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6802', 'nonce': 2821493591}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6804', 'nonce': 2821493591}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6803', 'nonce': 2821493591}]}, 'public_addr': '172.21.15.22:6801/2821493591', 'cluster_addr': '172.21.15.22:6802/2821493591', 'heartbeat_back_addr': '172.21.15.22:6804/2821493591', 'heartbeat_front_addr': '172.21.15.22:6803/2821493591', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '52cc1e06-9506-48c0-9f7b-d4270dff4b9f', 'up': 1, 'in': 1, 'weight': 0.64752197265625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 346, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6805', 'nonce': 4037911317}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6806', 'nonce': 4037911317}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6808', 'nonce': 4037911317}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6807', 'nonce': 4037911317}]}, 'public_addr': '172.21.15.22:6805/4037911317', 'cluster_addr': '172.21.15.22:6806/4037911317', 'heartbeat_back_addr': '172.21.15.22:6808/4037911317', 'heartbeat_front_addr': '172.21.15.22:6807/4037911317', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'fa2b2730-2c1d-45d0-be8a-494a60dc6b9b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 341, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6809', 'nonce': 1675799666}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6810', 'nonce': 1675799666}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6812', 'nonce': 1675799666}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6811', 'nonce': 1675799666}]}, 'public_addr': '172.21.15.22:6809/1675799666', 'cluster_addr': '172.21.15.22:6810/1675799666', 'heartbeat_back_addr': '172.21.15.22:6812/1675799666', 'heartbeat_front_addr': '172.21.15.22:6811/1675799666', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '8d53d3fc-e401-47df-b70b-4b4b1f4539df', 'up': 1, 'in': 1, 'weight': 0.686370849609375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 25, 'up_thru': 341, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6813', 'nonce': 1501962752}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6814', 'nonce': 1501962752}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6816', 'nonce': 1501962752}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6815', 'nonce': 1501962752}]}, 'public_addr': '172.21.15.22:6813/1501962752', 'cluster_addr': '172.21.15.22:6814/1501962752', 'heartbeat_back_addr': '172.21.15.22:6816/1501962752', 'heartbeat_front_addr': '172.21.15.22:6815/1501962752', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'd3207fe0-8b97-4067-a755-ba6d449c53c7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 346, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6800', 'nonce': 1210229443}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6801', 'nonce': 1210229443}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6803', 'nonce': 1210229443}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6802', 'nonce': 1210229443}]}, 'public_addr': '172.21.15.104:6800/1210229443', 'cluster_addr': '172.21.15.104:6801/1210229443', 'heartbeat_back_addr': '172.21.15.104:6803/1210229443', 'heartbeat_front_addr': '172.21.15.104:6802/1210229443', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '89c9352b-30b2-48c6-869e-047d07a23a43', 'up': 1, 'in': 1, 'weight': 0.6509552001953125, 'primary_affinity': 0.666046142578125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 37, 'up_thru': 341, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6804', 'nonce': 2735328732}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6805', 'nonce': 2735328732}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6807', 'nonce': 2735328732}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6806', 'nonce': 2735328732}]}, 'public_addr': '172.21.15.104:6804/2735328732', 'cluster_addr': '172.21.15.104:6805/2735328732', 'heartbeat_back_addr': '172.21.15.104:6807/2735328732', 'heartbeat_front_addr': '172.21.15.104:6806/2735328732', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '704563de-d88f-480a-b469-720135704d42', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 44, 'up_thru': 177, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6808', 'nonce': 721493010}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6809', 'nonce': 721493010}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6811', 'nonce': 721493010}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6810', 'nonce': 721493010}]}, 'public_addr': '172.21.15.104:6808/721493010', 'cluster_addr': '172.21.15.104:6809/721493010', 'heartbeat_back_addr': '172.21.15.104:6811/721493010', 'heartbeat_front_addr': '172.21.15.104:6810/721493010', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '18199e47-0052-49f0-b45f-0fb201e0d0f4', 'up': 1, 'in': 1, 'weight': 0.6657562255859375, 'primary_affinity': 1, 'last_clean_begin': 52, 'last_clean_end': 133, 'up_from': 147, 'up_thru': 341, 'down_at': 134, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6812', 'nonce': 2218863853}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6813', 'nonce': 2218863853}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6815', 'nonce': 2218863853}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6814', 'nonce': 2218863853}]}, 'public_addr': '172.21.15.104:6812/2218863853', 'cluster_addr': '172.21.15.104:6813/2218863853', 'heartbeat_back_addr': '172.21.15.104:6815/2218863853', 'heartbeat_front_addr': '172.21.15.104:6814/2218863853', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:42:05.640226+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:42:00.726790+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:42:00.226145+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:42:18.279268+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:42:03.255095+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:42:01.276730+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:41:56.711182+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:42:20.744248+0000', 'dead_epoch': 133}], 'pg_upmap': [{'pgid': '2.1c', 'osds': [5, 2, 7]}, {'pgid': '4.1e', 'osds': [4, 5, 3]}], 'pg_upmap_items': [{'pgid': '4.6', 'mappings': [{'from': 1, 'to': 5}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.22:0/240999923': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:6800/4016369460': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/585929238': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/544031024': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:6800/2504196132': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/1451388349': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/639336081': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/2104069730': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/4236450731': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:6800/3573193776': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/2661764422': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/2197488530': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/2968034970': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/599318175': '2024-09-12T07:27:07.919998+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-11T07:42:58.159 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 4.6 2024-09-11T07:42:58.160 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd rm-pg-upmap-items 4.6 2024-09-11T07:42:58.218 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:58.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:58.258 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:58.258 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:58.359 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:58.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:58.389 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:58.389 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:58.436 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:42:58.484 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:42:58.484 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:42:58.490 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:58.518 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:42:58.518 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:42:58.525 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:58.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:58.526 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:58.547 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:42:58.548 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:42:58.575 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:42:58.575 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:42:58.601 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:42:58.601 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:42:58.617 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:42:58.617 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:42:58.627 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:42:58.632 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:42:58.632 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:42:58.652 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:58.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:58.653 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:42:58.655 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-11T07:42:58.656 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:42:58.657 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:42:58.754 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:58.788 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:58.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:58.790 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:58.890 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:58.931 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:58.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:58.932 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:58.956 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 43 15 76687 76672 111.425 67.625 0.0168141 0.00896937 2024-09-11T07:42:59.033 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:42:59.038 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:58 smithi104 ceph-mon[33507]: pgmap v762: 148 pgs: 148 active+clean; 4.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 85 MiB/s wr, 1.35k op/s 2024-09-11T07:42:59.038 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:42:58 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1291136636' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:42:59.039 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:58 smithi022 ceph-mon[29462]: pgmap v762: 148 pgs: 148 active+clean; 4.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 85 MiB/s wr, 1.35k op/s 2024-09-11T07:42:59.039 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:42:58 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1291136636' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:42:59.040 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:58 smithi022 ceph-mon[37330]: pgmap v762: 148 pgs: 148 active+clean; 4.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 85 MiB/s wr, 1.35k op/s 2024-09-11T07:42:59.040 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:42:58 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1291136636' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:42:59.083 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:59.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:59.084 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:42:59.120 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:42:59.185 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:59.222 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:59.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:59.223 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:59.323 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:42:59.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:59.368 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:59.368 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:42:59.469 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:42:59.517 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:59.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:59.518 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:42:59.619 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:42:59.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:59.656 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:59.656 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:42:59.757 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:42:59.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:59.786 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:42:59.786 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:42:59.887 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:42:59.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:42:59.932 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:42:59.933 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:42:59.958 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 44 16 77965 77949 110.702 79.8125 0.0121596 0.00902842 2024-09-11T07:43:00.034 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:00.077 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:00.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:00.078 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:00.179 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:00.217 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:00.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:00.219 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:00.279 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:42:59 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:43:00.320 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:00.355 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:00.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:00.357 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:00.457 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:00.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:00.501 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:00.602 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:00.638 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:00.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:00.639 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:00.654 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:43:00 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:43:00] "GET /metrics HTTP/1.1" 200 224890 "" "Prometheus/2.43.0" 2024-09-11T07:43:00.740 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:00.773 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:00.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:00.775 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:00.875 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:00.910 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:00.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:00.912 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:00.959 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 45 16 79338 79322 110.148 85.8125 0.0156495 0.00907378 2024-09-11T07:43:01.012 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:43:01.071 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:01.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:01.073 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:01.173 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:01.221 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:01.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:01.222 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:01.323 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:01.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:01.371 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:01.372 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:01.376 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:43:01.376 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:43:01.473 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:01.504 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:01.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:01.505 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:01.549 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:01 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:43:01.550 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:01 smithi022 ceph-mon[29462]: pgmap v763: 148 pgs: 148 active+clean; 4.7 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 81 MiB/s wr, 1.29k op/s 2024-09-11T07:43:01.550 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:01 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:43:01.550 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:01 smithi022 ceph-mon[37330]: pgmap v763: 148 pgs: 148 active+clean; 4.7 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 81 MiB/s wr, 1.29k op/s 2024-09-11T07:43:01.605 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:01.637 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:01.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:01.638 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:01.739 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:01.779 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:01.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:01 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:43:01.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:01 smithi104 ceph-mon[33507]: pgmap v763: 148 pgs: 148 active+clean; 4.7 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 81 MiB/s wr, 1.29k op/s 2024-09-11T07:43:01.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:01.780 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:01.881 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:01.913 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:01.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:01.914 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:01.959 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 46 16 80530 80514 109.373 74.5 0.0121318 0.00913822 2024-09-11T07:43:02.016 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:02.038 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:02.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:02.039 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:02.140 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:02.162 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:02.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:02.163 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:02.264 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:02.317 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:02.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:02.318 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:02.419 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:02.455 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:02.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:02.456 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:02.539 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:43:02.557 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:02.629 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:02.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:02.631 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:02.731 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:02.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:02.774 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:02.774 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:02.797 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-11T07:43:02.875 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:02.917 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:02.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:02.918 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:02.959 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 47 15 81459 81444 108.282 58.125 0.0175024 0.00922976 2024-09-11T07:43:03.020 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:03.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:03.058 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:03.059 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:03.159 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:03.179 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:43:03.185 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:03.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:03.186 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:03.287 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:03.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:03.326 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:03.326 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:03.373 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is unset 2024-09-11T07:43:03.427 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:03.466 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:03.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:03.467 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:03.568 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:03.603 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:03 smithi104 ceph-mon[33507]: pgmap v764: 148 pgs: 148 active+clean; 4.7 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 857 B/s rd, 68 MiB/s wr, 1.08k op/s 2024-09-11T07:43:03.603 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:03 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3170371253' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:43:03.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:03.604 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:03.604 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:03.635 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:03 smithi022 ceph-mon[29462]: pgmap v764: 148 pgs: 148 active+clean; 4.7 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 857 B/s rd, 68 MiB/s wr, 1.08k op/s 2024-09-11T07:43:03.636 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:03 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3170371253' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:43:03.636 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:03 smithi022 ceph-mon[37330]: pgmap v764: 148 pgs: 148 active+clean; 4.7 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 857 B/s rd, 68 MiB/s wr, 1.08k op/s 2024-09-11T07:43:03.636 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:03 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3170371253' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:43:03.704 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:03.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:03.749 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:03.850 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:03.881 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:03.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:03.882 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:03.959 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 48 16 82671 82655 107.603 75.6875 0.0218444 0.00928855 2024-09-11T07:43:03.982 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:04.020 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:04.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:04.021 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:04.123 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:04.161 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:04.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:04.162 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:04.169 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:43:04.262 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:04.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:04.298 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:04.298 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:04.354 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:43:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:43:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:43:04.385 INFO:teuthology.orchestra.run.smithi022.stderr:clear 4.6 pg_upmap_items mapping 2024-09-11T07:43:04.398 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:04.420 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:04.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:04.421 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:04.522 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:04.548 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:04.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:04.550 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:04.650 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:04.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:04 smithi022 ceph-mon[29462]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:43:04.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:04 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3170371253' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:43:04.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:04 smithi022 ceph-mon[29462]: osdmap e357: 8 total, 8 up, 8 in 2024-09-11T07:43:04.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:04 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2430247169' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.6"}]: dispatch 2024-09-11T07:43:04.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:04 smithi022 ceph-mon[37330]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:43:04.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:04 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3170371253' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:43:04.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:04 smithi022 ceph-mon[37330]: osdmap e357: 8 total, 8 up, 8 in 2024-09-11T07:43:04.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:04 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2430247169' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.6"}]: dispatch 2024-09-11T07:43:04.673 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:04.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:04.674 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:04.711 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:04 smithi104 ceph-mon[33507]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:43:04.712 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:04 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3170371253' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:43:04.712 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:04 smithi104 ceph-mon[33507]: osdmap e357: 8 total, 8 up, 8 in 2024-09-11T07:43:04.712 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:04 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2430247169' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.6"}]: dispatch 2024-09-11T07:43:04.775 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:04.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:04.796 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:04.797 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:04.897 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:04.933 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:04.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:04.934 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:04.945 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:43:04.945 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:43:04.959 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 49 16 83386 83370 106.319 44.6875 0.0201789 0.00940044 2024-09-11T07:43:05.035 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:05.067 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:05.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:05.068 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:05.169 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:05.195 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:05.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:05.197 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:05.298 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:05.341 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:05.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:05.342 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:05.443 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:05.481 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:05.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:05.483 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:05.584 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:05.609 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:05 smithi104 ceph-mon[33507]: 5.1a scrub starts 2024-09-11T07:43:05.609 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:05 smithi104 ceph-mon[33507]: 5.1a scrub ok 2024-09-11T07:43:05.609 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:05 smithi104 ceph-mon[33507]: pgmap v766: 148 pgs: 148 active+clean; 5.0 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-11T07:43:05.609 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:05 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2430247169' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.6"}]': finished 2024-09-11T07:43:05.609 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:05 smithi104 ceph-mon[33507]: osdmap e358: 8 total, 8 up, 8 in 2024-09-11T07:43:05.621 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:05.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:05.622 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:05.649 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-11T07:43:05.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:05 smithi022 ceph-mon[29462]: 5.1a scrub starts 2024-09-11T07:43:05.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:05 smithi022 ceph-mon[29462]: 5.1a scrub ok 2024-09-11T07:43:05.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:05 smithi022 ceph-mon[29462]: pgmap v766: 148 pgs: 148 active+clean; 5.0 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-11T07:43:05.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:05 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2430247169' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.6"}]': finished 2024-09-11T07:43:05.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:05 smithi022 ceph-mon[29462]: osdmap e358: 8 total, 8 up, 8 in 2024-09-11T07:43:05.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:05 smithi022 ceph-mon[37330]: 5.1a scrub starts 2024-09-11T07:43:05.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:05 smithi022 ceph-mon[37330]: 5.1a scrub ok 2024-09-11T07:43:05.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:05 smithi022 ceph-mon[37330]: pgmap v766: 148 pgs: 148 active+clean; 5.0 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-11T07:43:05.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:05 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2430247169' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "4.6"}]': finished 2024-09-11T07:43:05.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:05 smithi022 ceph-mon[37330]: osdmap e358: 8 total, 8 up, 8 in 2024-09-11T07:43:05.723 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:05.762 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:05.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:05.763 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:05.864 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:05.886 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:05.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:05.887 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:05.959 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 50 16 84551 84535 105.649 72.8125 0.00883389 0.00946027 2024-09-11T07:43:05.988 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:06.025 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:43:06.041 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:06.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:06.042 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:06.095 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T07:43:06.143 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:06.164 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:06.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:06.165 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:06.266 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:06.305 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:06.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:06.306 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:06.407 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:06.439 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:06.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:06.440 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:06.488 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:06 smithi104 ceph-mon[33507]: 2.b scrub starts 2024-09-11T07:43:06.488 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:06 smithi104 ceph-mon[33507]: 2.b scrub ok 2024-09-11T07:43:06.488 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:06 smithi104 ceph-mon[33507]: 5.4 scrub starts 2024-09-11T07:43:06.488 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:06 smithi104 ceph-mon[33507]: 5.4 scrub ok 2024-09-11T07:43:06.488 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:06 smithi104 ceph-mon[33507]: 3.2 scrub starts 2024-09-11T07:43:06.488 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:06 smithi104 ceph-mon[33507]: 3.2 scrub ok 2024-09-11T07:43:06.488 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:06 smithi104 ceph-mon[33507]: 5.3 scrub starts 2024-09-11T07:43:06.488 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:06 smithi104 ceph-mon[33507]: 5.3 scrub ok 2024-09-11T07:43:06.489 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:06 smithi104 ceph-mon[33507]: osdmap e359: 8 total, 8 up, 8 in 2024-09-11T07:43:06.541 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:06.578 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:06.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:06.579 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:06.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:06 smithi022 ceph-mon[29462]: 2.b scrub starts 2024-09-11T07:43:06.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:06 smithi022 ceph-mon[29462]: 2.b scrub ok 2024-09-11T07:43:06.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:06 smithi022 ceph-mon[29462]: 5.4 scrub starts 2024-09-11T07:43:06.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:06 smithi022 ceph-mon[29462]: 5.4 scrub ok 2024-09-11T07:43:06.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:06 smithi022 ceph-mon[29462]: 3.2 scrub starts 2024-09-11T07:43:06.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:06 smithi022 ceph-mon[29462]: 3.2 scrub ok 2024-09-11T07:43:06.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:06 smithi022 ceph-mon[29462]: 5.3 scrub starts 2024-09-11T07:43:06.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:06 smithi022 ceph-mon[29462]: 5.3 scrub ok 2024-09-11T07:43:06.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:06 smithi022 ceph-mon[29462]: osdmap e359: 8 total, 8 up, 8 in 2024-09-11T07:43:06.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:06 smithi022 ceph-mon[37330]: 2.b scrub starts 2024-09-11T07:43:06.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:06 smithi022 ceph-mon[37330]: 2.b scrub ok 2024-09-11T07:43:06.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:06 smithi022 ceph-mon[37330]: 5.4 scrub starts 2024-09-11T07:43:06.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:06 smithi022 ceph-mon[37330]: 5.4 scrub ok 2024-09-11T07:43:06.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:06 smithi022 ceph-mon[37330]: 3.2 scrub starts 2024-09-11T07:43:06.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:06 smithi022 ceph-mon[37330]: 3.2 scrub ok 2024-09-11T07:43:06.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:06 smithi022 ceph-mon[37330]: 5.3 scrub starts 2024-09-11T07:43:06.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:06 smithi022 ceph-mon[37330]: 5.3 scrub ok 2024-09-11T07:43:06.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:06 smithi022 ceph-mon[37330]: osdmap e359: 8 total, 8 up, 8 in 2024-09-11T07:43:06.679 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:06.716 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:06.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:06.717 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:06.817 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:06.860 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:06.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:06.861 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:06.959 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 51 16 85747 85731 105.043 74.75 0.013026 0.00951441 2024-09-11T07:43:06.962 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:07.000 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:07.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:07.001 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:07.102 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:07.140 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:07.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:07.141 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:07.242 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:07.274 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:07.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:07.275 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:07.375 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:07.423 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:07.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:07.424 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:07.525 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:07.562 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:07.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:07.563 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:07.664 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:07.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:07 smithi022 ceph-mon[29462]: pgmap v769: 148 pgs: 148 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 62 MiB/s wr, 996 op/s 2024-09-11T07:43:07.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:07 smithi022 ceph-mon[29462]: 6.11 scrub starts 2024-09-11T07:43:07.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:07 smithi022 ceph-mon[29462]: 6.11 scrub ok 2024-09-11T07:43:07.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:07 smithi022 ceph-mon[37330]: pgmap v769: 148 pgs: 148 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 62 MiB/s wr, 996 op/s 2024-09-11T07:43:07.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:07 smithi022 ceph-mon[37330]: 6.11 scrub starts 2024-09-11T07:43:07.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:07 smithi022 ceph-mon[37330]: 6.11 scrub ok 2024-09-11T07:43:07.699 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:07.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:07.700 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:07.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:07 smithi104 ceph-mon[33507]: pgmap v769: 148 pgs: 148 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 62 MiB/s wr, 996 op/s 2024-09-11T07:43:07.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:07 smithi104 ceph-mon[33507]: 6.11 scrub starts 2024-09-11T07:43:07.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:07 smithi104 ceph-mon[33507]: 6.11 scrub ok 2024-09-11T07:43:07.801 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:07.833 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:07.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:07.834 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:07.935 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:07.959 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 52 15 87070 87055 104.614 82.75 0.0114046 0.00955411 2024-09-11T07:43:07.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:07.975 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:07.975 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:08.076 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:08.115 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:08.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:08.116 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:08.217 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:08.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:08.257 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:08.258 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:08.358 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:08.387 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:08.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:08.388 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:08.489 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:08.524 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:08.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:08.525 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:08.626 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:08.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:08.653 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:08.654 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:08.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:08 smithi022 ceph-mon[29462]: 6.2 scrub starts 2024-09-11T07:43:08.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:08 smithi022 ceph-mon[29462]: 6.2 scrub ok 2024-09-11T07:43:08.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:43:08.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:08 smithi022 ceph-mon[37330]: 6.2 scrub starts 2024-09-11T07:43:08.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:08 smithi022 ceph-mon[37330]: 6.2 scrub ok 2024-09-11T07:43:08.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:43:08.742 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:08 smithi104 ceph-mon[33507]: 6.2 scrub starts 2024-09-11T07:43:08.742 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:08 smithi104 ceph-mon[33507]: 6.2 scrub ok 2024-09-11T07:43:08.742 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:43:08.754 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:08.792 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:08.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:08.794 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:08.894 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:08.932 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:08.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:08.934 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:08.959 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 53 16 88310 88294 104.102 77.4375 0.00763727 0.00960157 2024-09-11T07:43:09.034 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:09.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:09.072 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:09.072 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:09.138 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:43:09.138 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:43:09.173 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:09.210 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:09.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:09.211 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:09.312 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:09.346 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:09.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:09.347 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:09.448 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:09.476 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:09.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:09.477 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:09.512 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:43:09.578 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:09.618 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:09.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:09.619 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:09.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:09 smithi022 ceph-mon[29462]: 6.1c scrub starts 2024-09-11T07:43:09.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:09 smithi022 ceph-mon[29462]: 6.1c scrub ok 2024-09-11T07:43:09.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:09 smithi022 ceph-mon[29462]: pgmap v770: 148 pgs: 148 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 62 MiB/s wr, 996 op/s 2024-09-11T07:43:09.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:09 smithi022 ceph-mon[29462]: 6.0 scrub starts 2024-09-11T07:43:09.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:09 smithi022 ceph-mon[29462]: 6.0 scrub ok 2024-09-11T07:43:09.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:09 smithi022 ceph-mon[29462]: 2.1a scrub starts 2024-09-11T07:43:09.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:09 smithi022 ceph-mon[29462]: 2.1a scrub ok 2024-09-11T07:43:09.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:09 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [3, 7]}]: dispatch 2024-09-11T07:43:09.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:09 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [3, 0]}]: dispatch 2024-09-11T07:43:09.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:09 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [3, 7]}]: dispatch 2024-09-11T07:43:09.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:09 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.c", "id": [1, 2]}]: dispatch 2024-09-11T07:43:09.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:09 smithi022 ceph-mon[37330]: 6.1c scrub starts 2024-09-11T07:43:09.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:09 smithi022 ceph-mon[37330]: 6.1c scrub ok 2024-09-11T07:43:09.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:09 smithi022 ceph-mon[37330]: pgmap v770: 148 pgs: 148 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 62 MiB/s wr, 996 op/s 2024-09-11T07:43:09.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:09 smithi022 ceph-mon[37330]: 6.0 scrub starts 2024-09-11T07:43:09.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:09 smithi022 ceph-mon[37330]: 6.0 scrub ok 2024-09-11T07:43:09.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:09 smithi022 ceph-mon[37330]: 2.1a scrub starts 2024-09-11T07:43:09.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:09 smithi022 ceph-mon[37330]: 2.1a scrub ok 2024-09-11T07:43:09.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:09 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [3, 7]}]: dispatch 2024-09-11T07:43:09.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:09 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [3, 0]}]: dispatch 2024-09-11T07:43:09.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:09 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [3, 7]}]: dispatch 2024-09-11T07:43:09.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:09 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.c", "id": [1, 2]}]: dispatch 2024-09-11T07:43:09.720 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:09.762 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:09.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:09.763 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:09.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:09 smithi104 ceph-mon[33507]: 6.1c scrub starts 2024-09-11T07:43:09.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:09 smithi104 ceph-mon[33507]: 6.1c scrub ok 2024-09-11T07:43:09.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:09 smithi104 ceph-mon[33507]: pgmap v770: 148 pgs: 148 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 62 MiB/s wr, 996 op/s 2024-09-11T07:43:09.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:09 smithi104 ceph-mon[33507]: 6.0 scrub starts 2024-09-11T07:43:09.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:09 smithi104 ceph-mon[33507]: 6.0 scrub ok 2024-09-11T07:43:09.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:09 smithi104 ceph-mon[33507]: 2.1a scrub starts 2024-09-11T07:43:09.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:09 smithi104 ceph-mon[33507]: 2.1a scrub ok 2024-09-11T07:43:09.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:09 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [3, 7]}]: dispatch 2024-09-11T07:43:09.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:09 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [3, 0]}]: dispatch 2024-09-11T07:43:09.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:09 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [3, 7]}]: dispatch 2024-09-11T07:43:09.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:09 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.c", "id": [1, 2]}]: dispatch 2024-09-11T07:43:09.857 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-11T07:43:09.891 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:09.923 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:09.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:09.924 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:09.960 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 54 16 89513 89497 103.566 75.1875 0.00318317 0.00964692 2024-09-11T07:43:10.024 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:10.066 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:10.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:10.067 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:10.168 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:10.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:10.206 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:10.207 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:10.281 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:43:09 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:43:10.291 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:43:10.308 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:10.350 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:10.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:10.351 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:10.451 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:10.478 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:10.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:10.479 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:10 smithi022 ceph-mon[29462]: 4.12 scrub starts 2024-09-11T07:43:10.532 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:10 smithi022 ceph-mon[29462]: 4.12 scrub ok 2024-09-11T07:43:10.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:10 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [3, 7]}]': finished 2024-09-11T07:43:10.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:10 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [3, 0]}]': finished 2024-09-11T07:43:10.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:10 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [3, 7]}]': finished 2024-09-11T07:43:10.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:10 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.c", "id": [1, 2]}]': finished 2024-09-11T07:43:10.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:10 smithi022 ceph-mon[29462]: osdmap e360: 8 total, 8 up, 8 in 2024-09-11T07:43:10.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:10 smithi022 ceph-mon[29462]: 4.17 scrub starts 2024-09-11T07:43:10.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:10 smithi022 ceph-mon[29462]: 4.17 scrub ok 2024-09-11T07:43:10.533 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:10 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/800226616' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:43:10.533 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:43:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:43:10] "GET /metrics HTTP/1.1" 200 224882 "" "Prometheus/2.43.0" 2024-09-11T07:43:10.534 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:10 smithi022 ceph-mon[37330]: 4.12 scrub starts 2024-09-11T07:43:10.534 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:10 smithi022 ceph-mon[37330]: 4.12 scrub ok 2024-09-11T07:43:10.534 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:10 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [3, 7]}]': finished 2024-09-11T07:43:10.534 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:10 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [3, 0]}]': finished 2024-09-11T07:43:10.534 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:10 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [3, 7]}]': finished 2024-09-11T07:43:10.534 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:10 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.c", "id": [1, 2]}]': finished 2024-09-11T07:43:10.534 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:10 smithi022 ceph-mon[37330]: osdmap e360: 8 total, 8 up, 8 in 2024-09-11T07:43:10.534 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:10 smithi022 ceph-mon[37330]: 4.17 scrub starts 2024-09-11T07:43:10.534 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:10 smithi022 ceph-mon[37330]: 4.17 scrub ok 2024-09-11T07:43:10.534 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:10 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/800226616' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:43:10.580 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:10.615 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:10.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:10.616 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:10.717 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:10.733 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:10 smithi104 ceph-mon[33507]: 4.12 scrub starts 2024-09-11T07:43:10.733 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:10 smithi104 ceph-mon[33507]: 4.12 scrub ok 2024-09-11T07:43:10.734 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:10 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [3, 7]}]': finished 2024-09-11T07:43:10.734 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:10 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [3, 0]}]': finished 2024-09-11T07:43:10.734 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:10 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [3, 7]}]': finished 2024-09-11T07:43:10.734 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:10 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.c", "id": [1, 2]}]': finished 2024-09-11T07:43:10.734 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:10 smithi104 ceph-mon[33507]: osdmap e360: 8 total, 8 up, 8 in 2024-09-11T07:43:10.734 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:10 smithi104 ceph-mon[33507]: 4.17 scrub starts 2024-09-11T07:43:10.734 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:10 smithi104 ceph-mon[33507]: 4.17 scrub ok 2024-09-11T07:43:10.734 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:10 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/800226616' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:43:10.755 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:10.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:10.756 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:10.857 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:10.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:10.894 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:10.894 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:10.960 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 55 16 90681 90665 103.01 73 0.0089651 0.00970228 2024-09-11T07:43:10.996 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:11.041 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:11.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:11.042 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:11.142 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:11.197 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:11.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:11.198 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:11.299 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:11.326 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:11.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:11.327 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:11.410 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:43:11.411 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:43:11.411 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:43:11.411 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:43:11.411 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:43:11.411 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:43:11.411 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:43:11.411 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:43:11.411 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:43:11.411 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:43:11.411 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:43:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:43:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:43:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:43:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:43:11.412 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:43:11.428 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:11.461 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:11.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:11.462 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:11.563 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:11.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:11.594 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:11.594 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:11.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:11 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:43:11.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:11 smithi022 ceph-mon[29462]: pgmap v772: 148 pgs: 1 peering, 147 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 67 MiB/s wr, 1.07k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:43:11.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:11 smithi022 ceph-mon[29462]: 4.10 scrub starts 2024-09-11T07:43:11.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:11 smithi022 ceph-mon[29462]: 4.10 scrub ok 2024-09-11T07:43:11.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:11 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:43:11.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:11 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:43:11.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:11 smithi022 ceph-mon[29462]: osdmap e361: 8 total, 8 up, 8 in 2024-09-11T07:43:11.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:11 smithi022 ceph-mon[29462]: 6.1 scrub starts 2024-09-11T07:43:11.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:11 smithi022 ceph-mon[29462]: 6.1 scrub ok 2024-09-11T07:43:11.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:11 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:43:11.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:11 smithi022 ceph-mon[37330]: pgmap v772: 148 pgs: 1 peering, 147 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 67 MiB/s wr, 1.07k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:43:11.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:11 smithi022 ceph-mon[37330]: 4.10 scrub starts 2024-09-11T07:43:11.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:11 smithi022 ceph-mon[37330]: 4.10 scrub ok 2024-09-11T07:43:11.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:11 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:43:11.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:11 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:43:11.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:11 smithi022 ceph-mon[37330]: osdmap e361: 8 total, 8 up, 8 in 2024-09-11T07:43:11.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:11 smithi022 ceph-mon[37330]: 6.1 scrub starts 2024-09-11T07:43:11.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:11 smithi022 ceph-mon[37330]: 6.1 scrub ok 2024-09-11T07:43:11.695 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:11.726 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:11.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:11.727 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:11.739 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:11 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:43:11.739 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:11 smithi104 ceph-mon[33507]: pgmap v772: 148 pgs: 1 peering, 147 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 67 MiB/s wr, 1.07k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:43:11.739 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:11 smithi104 ceph-mon[33507]: 4.10 scrub starts 2024-09-11T07:43:11.739 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:11 smithi104 ceph-mon[33507]: 4.10 scrub ok 2024-09-11T07:43:11.739 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:11 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:43:11.739 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:11 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:43:11.739 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:11 smithi104 ceph-mon[33507]: osdmap e361: 8 total, 8 up, 8 in 2024-09-11T07:43:11.739 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:11 smithi104 ceph-mon[33507]: 6.1 scrub starts 2024-09-11T07:43:11.739 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:11 smithi104 ceph-mon[33507]: 6.1 scrub ok 2024-09-11T07:43:11.828 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:11.856 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:11.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:11.857 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:11.958 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:11.960 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 56 16 91904 91888 102.536 76.4375 0.0107596 0.00974792 2024-09-11T07:43:11.997 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:11.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:11.998 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:12.098 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:12.137 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:12.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:12.138 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:12.239 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:12.240 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:43:12.240 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:43:12.241 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2024-09-11T07:43:12.241 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2024-09-11T07:43:12.241 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:43:12.280 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:12.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:12.282 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:12.382 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:12.419 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:12.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:12.420 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:12.521 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:12.560 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:12.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:12.561 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:12.662 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:12.693 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:12.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:12.694 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:12.714 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:12 smithi022 ceph-mon[29462]: 5.6 scrub starts 2024-09-11T07:43:12.715 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:12 smithi022 ceph-mon[29462]: 5.6 scrub ok 2024-09-11T07:43:12.715 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:12 smithi022 ceph-mon[29462]: 4.14 scrub starts 2024-09-11T07:43:12.715 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:12 smithi022 ceph-mon[29462]: 4.14 scrub ok 2024-09-11T07:43:12.715 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:12 smithi022 ceph-mon[37330]: 5.6 scrub starts 2024-09-11T07:43:12.715 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:12 smithi022 ceph-mon[37330]: 5.6 scrub ok 2024-09-11T07:43:12.715 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:12 smithi022 ceph-mon[37330]: 4.14 scrub starts 2024-09-11T07:43:12.715 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:12 smithi022 ceph-mon[37330]: 4.14 scrub ok 2024-09-11T07:43:12.758 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:12 smithi104 ceph-mon[33507]: 5.6 scrub starts 2024-09-11T07:43:12.759 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:12 smithi104 ceph-mon[33507]: 5.6 scrub ok 2024-09-11T07:43:12.759 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:12 smithi104 ceph-mon[33507]: 4.14 scrub starts 2024-09-11T07:43:12.759 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:12 smithi104 ceph-mon[33507]: 4.14 scrub ok 2024-09-11T07:43:12.795 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:12.837 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:12.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:12.838 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:12.939 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:12.960 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 57 16 93100 93084 102.048 74.75 0.0102464 0.00979433 2024-09-11T07:43:12.967 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:12.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:12.968 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:13.069 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:13.097 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:13.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:13.098 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:13.133 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-11T07:43:13.199 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:13.239 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:13.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:13.240 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:13.341 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:13.376 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:13.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:13.378 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:13.478 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:13.505 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is unset 2024-09-11T07:43:13.520 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:13.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:13.521 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:13.574 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:43:13.622 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:13.671 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:13.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:13.672 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:13.760 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:13 smithi104 ceph-mon[33507]: 4.3 scrub starts 2024-09-11T07:43:13.761 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:13 smithi104 ceph-mon[33507]: 4.3 scrub ok 2024-09-11T07:43:13.761 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:13 smithi104 ceph-mon[33507]: pgmap v774: 148 pgs: 1 peering, 147 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 742 B/s rd, 51 MiB/s wr, 820 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:43:13.761 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:13 smithi104 ceph-mon[33507]: 4.2 scrub starts 2024-09-11T07:43:13.761 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:13 smithi104 ceph-mon[33507]: 4.2 scrub ok 2024-09-11T07:43:13.761 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:13 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3518600409' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:43:13.772 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:13.808 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:13.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:13.809 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:13.886 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:13 smithi022 ceph-mon[29462]: 4.3 scrub starts 2024-09-11T07:43:13.886 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:13 smithi022 ceph-mon[29462]: 4.3 scrub ok 2024-09-11T07:43:13.886 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:13 smithi022 ceph-mon[29462]: pgmap v774: 148 pgs: 1 peering, 147 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 742 B/s rd, 51 MiB/s wr, 820 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:43:13.886 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:13 smithi022 ceph-mon[29462]: 4.2 scrub starts 2024-09-11T07:43:13.886 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:13 smithi022 ceph-mon[29462]: 4.2 scrub ok 2024-09-11T07:43:13.886 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:13 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3518600409' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:43:13.887 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:13 smithi022 ceph-mon[37330]: 4.3 scrub starts 2024-09-11T07:43:13.887 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:13 smithi022 ceph-mon[37330]: 4.3 scrub ok 2024-09-11T07:43:13.887 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:13 smithi022 ceph-mon[37330]: pgmap v774: 148 pgs: 1 peering, 147 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 742 B/s rd, 51 MiB/s wr, 820 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:43:13.887 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:13 smithi022 ceph-mon[37330]: 4.2 scrub starts 2024-09-11T07:43:13.887 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:13 smithi022 ceph-mon[37330]: 4.2 scrub ok 2024-09-11T07:43:13.887 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:13 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3518600409' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:43:13.909 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:13.947 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:13.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:13.948 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:13.960 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 58 16 94260 94244 101.539 72.5 0.00676322 0.00984076 2024-09-11T07:43:13.986 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:43:13.986 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:43:13.986 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-11T07:43:13.987 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-11T07:43:14.050 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:14.078 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:14.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:14.079 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:14.170 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:43:14.180 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:14.207 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:14.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:14.208 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:14.309 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:14.348 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:14.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:14.349 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:14.450 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:14.474 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:43:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:43:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:43:14.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:14.495 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:14.495 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:14.596 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:14.629 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:14.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:14.630 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:14.730 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:14.751 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:14.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:14.752 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:14.763 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:14 smithi104 ceph-mon[33507]: 6.c scrub starts 2024-09-11T07:43:14.763 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:14 smithi104 ceph-mon[33507]: 6.c scrub ok 2024-09-11T07:43:14.763 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:14 smithi104 ceph-mon[33507]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:43:14.763 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:14 smithi104 ceph-mon[33507]: Cluster is now healthy 2024-09-11T07:43:14.763 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:14 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3518600409' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:43:14.764 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:14 smithi104 ceph-mon[33507]: osdmap e362: 8 total, 8 up, 8 in 2024-09-11T07:43:14.853 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:14.890 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:14.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:14.891 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:14.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:14 smithi022 ceph-mon[37330]: 6.c scrub starts 2024-09-11T07:43:14.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:14 smithi022 ceph-mon[37330]: 6.c scrub ok 2024-09-11T07:43:14.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:14 smithi022 ceph-mon[37330]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:43:14.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:14 smithi022 ceph-mon[37330]: Cluster is now healthy 2024-09-11T07:43:14.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:14 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3518600409' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:43:14.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:14 smithi022 ceph-mon[37330]: osdmap e362: 8 total, 8 up, 8 in 2024-09-11T07:43:14.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:14 smithi022 ceph-mon[29462]: 6.c scrub starts 2024-09-11T07:43:14.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:14 smithi022 ceph-mon[29462]: 6.c scrub ok 2024-09-11T07:43:14.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:14 smithi022 ceph-mon[29462]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:43:14.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:14 smithi022 ceph-mon[29462]: Cluster is now healthy 2024-09-11T07:43:14.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:14 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3518600409' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:43:14.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:14 smithi022 ceph-mon[29462]: osdmap e362: 8 total, 8 up, 8 in 2024-09-11T07:43:14.960 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 59 16 95251 95235 100.867 61.9375 0.00870007 0.00990952 2024-09-11T07:43:14.992 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:15.030 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:15.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:15.031 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:15.132 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:15.166 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:15.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:15.167 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:15.268 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:15.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:15.295 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:15.296 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:15.396 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:15.418 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:15.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:15.419 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:15.518 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:43:15.519 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2024-09-11T07:43:15.519 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2024-09-11T07:43:15.519 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2024-09-11T07:43:15.519 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.519 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:85934 9.f 9:fdbe5b3c:::benchmark_data_smithi022_133943_object85933:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e258)", 2024-09-11T07:43:15.519 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:26.311544+0000", 2024-09-11T07:43:15.519 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 289.20234419799999, 2024-09-11T07:43:15.519 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.30120598399999998, 2024-09-11T07:43:15.519 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:15.519 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:15.519 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:15.519 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:15.519 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:15.520 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 85934 2024-09-11T07:43:15.520 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.520 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:15.520 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.520 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:15.520 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.311544+0000", 2024-09-11T07:43:15.520 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.520 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.521 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.521 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:15.521 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.311544+0000", 2024-09-11T07:43:15.521 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.521 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.521 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.521 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:15.521 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.311542+0000", 2024-09-11T07:43:15.522 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:43:15.522 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.522 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.522 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:15.522 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.311634+0000", 2024-09-11T07:43:15.522 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.1664999999999997e-05 2024-09-11T07:43:15.522 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.522 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.522 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:15.522 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.311653+0000", 2024-09-11T07:43:15.523 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8987e-05 2024-09-11T07:43:15.523 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.523 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.523 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:15.523 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.311665+0000", 2024-09-11T07:43:15.523 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1657e-05 2024-09-11T07:43:15.523 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.523 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.523 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:15.523 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.317571+0000", 2024-09-11T07:43:15.523 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0059061249999999999 2024-09-11T07:43:15.523 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.524 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.524 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:15.524 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.317705+0000", 2024-09-11T07:43:15.524 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013474 2024-09-11T07:43:15.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.524 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.524 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:43:15.524 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.317854+0000", 2024-09-11T07:43:15.524 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000148627 2024-09-11T07:43:15.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.524 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.525 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.525 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.325240+0000", 2024-09-11T07:43:15.525 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0073861480000000004 2024-09-11T07:43:15.525 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.525 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.525 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:15.525 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.325252+0000", 2024-09-11T07:43:15.525 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1678000000000001e-05 2024-09-11T07:43:15.525 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.525 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.526 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.526 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.612633+0000", 2024-09-11T07:43:15.526 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.28738107099999999 2024-09-11T07:43:15.526 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.526 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.526 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:15.526 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.612702+0000", 2024-09-11T07:43:15.526 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9142999999999999e-05 2024-09-11T07:43:15.526 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.526 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.526 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:15.526 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.612750+0000", 2024-09-11T07:43:15.527 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7559000000000003e-05 2024-09-11T07:43:15.527 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.527 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:15.527 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.527 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.527 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.527 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:85942 9.f 9:fc70c519:::benchmark_data_smithi022_133943_object85941:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e258)", 2024-09-11T07:43:15.527 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:26.316250+0000", 2024-09-11T07:43:15.527 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 289.197637568, 2024-09-11T07:43:15.527 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.29663829400000002, 2024-09-11T07:43:15.528 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:15.528 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:15.528 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:15.528 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:15.528 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:15.528 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 85942 2024-09-11T07:43:15.528 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.528 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:15.528 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.528 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:15.528 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.316250+0000", 2024-09-11T07:43:15.528 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.529 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.529 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.529 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:15.529 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.316250+0000", 2024-09-11T07:43:15.529 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.529 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.529 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.529 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:15.529 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.316249+0000", 2024-09-11T07:43:15.530 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:43:15.530 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.530 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.530 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:15.530 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.316331+0000", 2024-09-11T07:43:15.530 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2365000000000002e-05 2024-09-11T07:43:15.530 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.530 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.530 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:15.530 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.316343+0000", 2024-09-11T07:43:15.530 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1596e-05 2024-09-11T07:43:15.530 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.530 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.530 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:15.531 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.316348+0000", 2024-09-11T07:43:15.531 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6470000000000003e-06 2024-09-11T07:43:15.531 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.531 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.531 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:15.531 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.322428+0000", 2024-09-11T07:43:15.531 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0060797029999999997 2024-09-11T07:43:15.531 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.531 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.531 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:15.531 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.322544+0000", 2024-09-11T07:43:15.531 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000116296 2024-09-11T07:43:15.531 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.531 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.532 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:43:15.532 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.322682+0000", 2024-09-11T07:43:15.532 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013749500000000001 2024-09-11T07:43:15.532 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.532 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.532 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:15.532 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.325339+0000", 2024-09-11T07:43:15.532 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0026568189999999999 2024-09-11T07:43:15.532 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.532 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.532 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.532 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.328284+0000", 2024-09-11T07:43:15.532 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0029453029999999998 2024-09-11T07:43:15.533 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.533 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.533 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.533 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.612802+0000", 2024-09-11T07:43:15.533 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.28451834599999998 2024-09-11T07:43:15.533 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.533 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.533 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:15.533 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.612852+0000", 2024-09-11T07:43:15.533 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0139999999999998e-05 2024-09-11T07:43:15.533 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.533 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.533 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:15.533 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.612889+0000", 2024-09-11T07:43:15.534 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6168999999999999e-05 2024-09-11T07:43:15.534 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.534 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:15.534 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.534 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.534 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.534 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:85953 9.f 9:fc848a32:::benchmark_data_smithi022_133943_object85952:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e258)", 2024-09-11T07:43:15.534 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:26.331015+0000", 2024-09-11T07:43:15.534 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 289.18287281300002, 2024-09-11T07:43:15.534 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.28199429199999998, 2024-09-11T07:43:15.534 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:15.534 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:15.534 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:15.534 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:15.535 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:15.535 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 85953 2024-09-11T07:43:15.535 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.535 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:15.535 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.535 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:15.535 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.331015+0000", 2024-09-11T07:43:15.535 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.535 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.535 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.535 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:15.535 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.331015+0000", 2024-09-11T07:43:15.535 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.536 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.536 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.536 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:15.536 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.331013+0000", 2024-09-11T07:43:15.536 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:43:15.536 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.536 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.536 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:15.536 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.331046+0000", 2024-09-11T07:43:15.536 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3189999999999999e-05 2024-09-11T07:43:15.536 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.536 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.536 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:15.537 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.331057+0000", 2024-09-11T07:43:15.537 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0703999999999999e-05 2024-09-11T07:43:15.537 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.537 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.537 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:15.537 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.331062+0000", 2024-09-11T07:43:15.537 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5439999999999998e-06 2024-09-11T07:43:15.537 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.537 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.537 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:15.537 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.332439+0000", 2024-09-11T07:43:15.537 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0013765870000000001 2024-09-11T07:43:15.537 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.537 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.538 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:15.538 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.332578+0000", 2024-09-11T07:43:15.538 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013949000000000001 2024-09-11T07:43:15.538 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.538 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.538 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:43:15.538 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.332730+0000", 2024-09-11T07:43:15.538 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015161100000000001 2024-09-11T07:43:15.538 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.538 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.538 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.538 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.454340+0000", 2024-09-11T07:43:15.538 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.12160982100000001 2024-09-11T07:43:15.538 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.539 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.539 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:15.539 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.454786+0000", 2024-09-11T07:43:15.539 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00044578 2024-09-11T07:43:15.539 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.539 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.539 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.539 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.612928+0000", 2024-09-11T07:43:15.539 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.15814265899999999 2024-09-11T07:43:15.539 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.539 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.539 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:15.540 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.612973+0000", 2024-09-11T07:43:15.540 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4904999999999998e-05 2024-09-11T07:43:15.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.540 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:15.540 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.613009+0000", 2024-09-11T07:43:15.540 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6177000000000003e-05 2024-09-11T07:43:15.540 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.540 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:15.540 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.540 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:85932 9.7 9:e6d503e6:::benchmark_data_smithi022_133943_object85931:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e259)", 2024-09-11T07:43:15.541 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:26.333730+0000", 2024-09-11T07:43:15.541 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 289.18015800900002, 2024-09-11T07:43:15.541 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.025048288, 2024-09-11T07:43:15.541 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:15.541 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:43:15.541 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:15.541 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:15.541 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:15.541 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 85932 2024-09-11T07:43:15.541 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.541 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:15.541 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.541 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:15.542 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333730+0000", 2024-09-11T07:43:15.542 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.542 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.542 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.542 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:15.542 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333730+0000", 2024-09-11T07:43:15.543 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.543 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.543 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.543 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:15.543 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333727+0000", 2024-09-11T07:43:15.543 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999967 2024-09-11T07:43:15.543 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.543 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.543 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:15.543 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333773+0000", 2024-09-11T07:43:15.543 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6940999999999997e-05 2024-09-11T07:43:15.543 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.543 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.544 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:15.544 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333787+0000", 2024-09-11T07:43:15.544 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3107e-05 2024-09-11T07:43:15.544 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.544 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.544 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:15.544 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333794+0000", 2024-09-11T07:43:15.544 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4059999999999999e-06 2024-09-11T07:43:15.544 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.544 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.544 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:15.544 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.454708+0000", 2024-09-11T07:43:15.544 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.120914359 2024-09-11T07:43:15.544 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.545 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:43:15.545 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.454717+0000", 2024-09-11T07:43:15.545 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2830000000000001e-06 2024-09-11T07:43:15.545 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.545 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:15.545 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.358693+0000", 2024-09-11T07:43:15.545 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.90397602300000002 2024-09-11T07:43:15.545 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.545 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:15.545 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.358778+0000", 2024-09-11T07:43:15.545 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5457999999999994e-05 2024-09-11T07:43:15.546 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.546 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:15.546 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.546 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.546 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.546 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:85948 9.6 9:6409403d:::benchmark_data_smithi022_133943_object85947:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e259)", 2024-09-11T07:43:15.546 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:26.333803+0000", 2024-09-11T07:43:15.546 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 289.18008518900001, 2024-09-11T07:43:15.546 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0201702269999999, 2024-09-11T07:43:15.546 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:15.546 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:43:15.546 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:15.546 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:15.546 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:15.547 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 85948 2024-09-11T07:43:15.547 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.547 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:15.547 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.547 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:15.547 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333803+0000", 2024-09-11T07:43:15.547 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.547 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.547 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.547 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:15.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333803+0000", 2024-09-11T07:43:15.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.548 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.548 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:15.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333802+0000", 2024-09-11T07:43:15.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:43:15.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.548 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.548 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:15.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333842+0000", 2024-09-11T07:43:15.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0456e-05 2024-09-11T07:43:15.549 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.549 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:15.549 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333849+0000", 2024-09-11T07:43:15.549 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.511e-06 2024-09-11T07:43:15.549 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.549 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:15.549 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333853+0000", 2024-09-11T07:43:15.549 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.963e-06 2024-09-11T07:43:15.549 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.549 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:15.549 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.469490+0000", 2024-09-11T07:43:15.550 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13563673200000001 2024-09-11T07:43:15.550 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.550 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:15.551 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.551 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:43:15.551 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.469495+0000", 2024-09-11T07:43:15.552 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3410000000000002e-06 2024-09-11T07:43:15.552 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.552 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.552 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:15.552 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.353902+0000", 2024-09-11T07:43:15.552 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.88440743099999997 2024-09-11T07:43:15.552 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.552 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.552 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:15.552 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.353973+0000", 2024-09-11T07:43:15.552 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0479000000000001e-05 2024-09-11T07:43:15.552 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.552 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:15.552 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.553 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.553 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.553 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:86232 9.6 9:6062b21b:::benchmark_data_smithi022_133943_object86231:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:43:15.553 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:27.532402+0000", 2024-09-11T07:43:15.553 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 287.981485437, 2024-09-11T07:43:15.553 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.180620107, 2024-09-11T07:43:15.553 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:15.553 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:15.553 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:15.553 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:15.553 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:15.553 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 86232 2024-09-11T07:43:15.554 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.554 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:15.554 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.554 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:15.554 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532402+0000", 2024-09-11T07:43:15.554 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.554 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.554 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.554 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:15.554 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532402+0000", 2024-09-11T07:43:15.554 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.554 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.554 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.555 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:15.555 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532400+0000", 2024-09-11T07:43:15.555 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:43:15.555 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.555 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.555 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:15.555 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532456+0000", 2024-09-11T07:43:15.555 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5791000000000002e-05 2024-09-11T07:43:15.555 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.555 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.555 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:15.555 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532470+0000", 2024-09-11T07:43:15.555 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4280999999999999e-05 2024-09-11T07:43:15.555 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.556 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.556 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:15.556 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532477+0000", 2024-09-11T07:43:15.556 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3350000000000003e-06 2024-09-11T07:43:15.556 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.556 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.556 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:15.556 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532524+0000", 2024-09-11T07:43:15.556 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7128e-05 2024-09-11T07:43:15.556 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.556 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.556 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:15.556 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532627+0000", 2024-09-11T07:43:15.557 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010340099999999999 2024-09-11T07:43:15.557 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.557 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.557 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:43:15.557 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532919+0000", 2024-09-11T07:43:15.557 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00029145000000000001 2024-09-11T07:43:15.557 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.557 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.557 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:15.557 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.534532+0000", 2024-09-11T07:43:15.557 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0016128519999999999 2024-09-11T07:43:15.557 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.558 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.558 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.558 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.537151+0000", 2024-09-11T07:43:15.558 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0026190900000000001 2024-09-11T07:43:15.558 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.558 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.558 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.558 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.712780+0000", 2024-09-11T07:43:15.558 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.175629435 2024-09-11T07:43:15.558 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.558 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.558 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:15.558 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.712894+0000", 2024-09-11T07:43:15.559 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000113725 2024-09-11T07:43:15.559 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.559 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.559 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:15.559 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.713022+0000", 2024-09-11T07:43:15.559 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012873499999999999 2024-09-11T07:43:15.559 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.559 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:15.559 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.559 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.559 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.559 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:86234 9.6 9:6677583e:::benchmark_data_smithi022_133943_object86233:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:43:15.559 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:27.532659+0000", 2024-09-11T07:43:15.559 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 287.98122915699997, 2024-09-11T07:43:15.560 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18307527700000001, 2024-09-11T07:43:15.560 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:15.560 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:15.560 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:15.560 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:15.560 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:15.560 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 86234 2024-09-11T07:43:15.560 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.560 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:15.560 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.560 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:15.560 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532659+0000", 2024-09-11T07:43:15.560 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.561 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.561 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.561 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:15.561 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532659+0000", 2024-09-11T07:43:15.561 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.561 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.561 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.561 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:15.561 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532657+0000", 2024-09-11T07:43:15.561 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:43:15.561 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.561 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.561 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:15.562 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532714+0000", 2024-09-11T07:43:15.562 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6298999999999997e-05 2024-09-11T07:43:15.562 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.562 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.562 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:15.562 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532725+0000", 2024-09-11T07:43:15.562 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1216000000000001e-05 2024-09-11T07:43:15.562 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.562 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.562 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:15.562 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532730+0000", 2024-09-11T07:43:15.562 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5849999999999999e-06 2024-09-11T07:43:15.562 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.563 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.563 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:15.563 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.535694+0000", 2024-09-11T07:43:15.563 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0029634069999999999 2024-09-11T07:43:15.563 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.563 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.563 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:15.563 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.535830+0000", 2024-09-11T07:43:15.563 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013649300000000001 2024-09-11T07:43:15.563 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.563 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.563 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:43:15.563 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536159+0000", 2024-09-11T07:43:15.564 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00032905400000000001 2024-09-11T07:43:15.564 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.564 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.564 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:15.564 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.537278+0000", 2024-09-11T07:43:15.564 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0011182830000000001 2024-09-11T07:43:15.564 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.564 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.564 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.564 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.537813+0000", 2024-09-11T07:43:15.564 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00053518700000000001 2024-09-11T07:43:15.564 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.564 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.564 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.568 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.715656+0000", 2024-09-11T07:43:15.568 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17784266000000001 2024-09-11T07:43:15.568 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.568 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.568 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:15.568 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.715700+0000", 2024-09-11T07:43:15.568 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4508000000000001e-05 2024-09-11T07:43:15.568 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.568 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.568 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:15.569 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.715734+0000", 2024-09-11T07:43:15.569 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3877000000000001e-05 2024-09-11T07:43:15.569 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.569 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:15.569 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.569 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.569 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.569 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:86244 9.a 9:5743f4ed:::benchmark_data_smithi022_133943_object86243:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:43:15.569 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:27.540150+0000", 2024-09-11T07:43:15.569 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 287.97373824900001, 2024-09-11T07:43:15.569 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.177488494, 2024-09-11T07:43:15.569 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:15.570 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:15.570 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:15.570 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:15.570 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:15.570 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 86244 2024-09-11T07:43:15.570 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.570 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:15.570 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.570 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:15.570 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540150+0000", 2024-09-11T07:43:15.570 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.570 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.570 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.571 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:15.571 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540150+0000", 2024-09-11T07:43:15.571 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.571 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.571 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.571 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:15.571 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540148+0000", 2024-09-11T07:43:15.571 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:43:15.571 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.571 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.571 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:15.571 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540199+0000", 2024-09-11T07:43:15.571 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1332999999999998e-05 2024-09-11T07:43:15.571 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.572 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.572 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:15.572 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540210+0000", 2024-09-11T07:43:15.572 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1323e-05 2024-09-11T07:43:15.572 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.572 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.572 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:15.572 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540215+0000", 2024-09-11T07:43:15.572 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6280000000000004e-06 2024-09-11T07:43:15.572 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.572 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.572 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:15.573 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540253+0000", 2024-09-11T07:43:15.573 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8096000000000003e-05 2024-09-11T07:43:15.573 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.573 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.573 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:15.573 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540335+0000", 2024-09-11T07:43:15.573 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1703e-05 2024-09-11T07:43:15.573 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.573 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.574 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,7", 2024-09-11T07:43:15.574 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540427+0000", 2024-09-11T07:43:15.574 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2069000000000002e-05 2024-09-11T07:43:15.574 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.574 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.574 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:15.574 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.542279+0000", 2024-09-11T07:43:15.574 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001852245 2024-09-11T07:43:15.574 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.574 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.574 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.574 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.542406+0000", 2024-09-11T07:43:15.574 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012750700000000001 2024-09-11T07:43:15.575 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.575 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.575 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.575 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.717564+0000", 2024-09-11T07:43:15.575 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17515773600000001 2024-09-11T07:43:15.575 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.575 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.575 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:15.575 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.717600+0000", 2024-09-11T07:43:15.575 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5410000000000001e-05 2024-09-11T07:43:15.575 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.575 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.575 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:15.576 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.717638+0000", 2024-09-11T07:43:15.576 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8473999999999998e-05 2024-09-11T07:43:15.576 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.576 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:15.576 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.576 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.576 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.576 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87252 9.12 9.5bd7f852 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:43:15.576 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.378195+0000", 2024-09-11T07:43:15.576 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 287.13569270099998, 2024-09-11T07:43:15.576 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.96275559600000005, 2024-09-11T07:43:15.576 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:15.576 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:43:15.577 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:15.577 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:15.577 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:15.577 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87252 2024-09-11T07:43:15.577 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.577 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:15.577 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.577 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:15.577 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378195+0000", 2024-09-11T07:43:15.577 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.577 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.577 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.577 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:15.577 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378195+0000", 2024-09-11T07:43:15.578 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.578 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.578 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.578 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:15.578 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378193+0000", 2024-09-11T07:43:15.578 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:43:15.578 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.578 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.578 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:15.578 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378254+0000", 2024-09-11T07:43:15.578 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0699999999999998e-05 2024-09-11T07:43:15.578 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.578 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.579 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:15.579 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378263+0000", 2024-09-11T07:43:15.579 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7579999999999998e-06 2024-09-11T07:43:15.579 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.579 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.579 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:15.579 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378268+0000", 2024-09-11T07:43:15.579 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9830000000000004e-06 2024-09-11T07:43:15.579 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.579 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.579 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:15.579 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378303+0000", 2024-09-11T07:43:15.579 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5503999999999998e-05 2024-09-11T07:43:15.579 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.580 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.580 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:43:15.580 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378308+0000", 2024-09-11T07:43:15.580 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.33e-06 2024-09-11T07:43:15.580 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.580 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.580 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:15.580 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.340939+0000", 2024-09-11T07:43:15.580 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.96263122800000001 2024-09-11T07:43:15.580 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.580 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.580 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:15.580 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.340951+0000", 2024-09-11T07:43:15.580 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1800999999999999e-05 2024-09-11T07:43:15.581 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.581 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:15.581 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.581 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.581 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.581 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196432 9.d 9:b43c5e98:::benchmark_data_smithi022_133943_object69841:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:43:15.581 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.509114+0000", 2024-09-11T07:43:15.581 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 178.004773473, 2024-09-11T07:43:15.581 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.152507894, 2024-09-11T07:43:15.581 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:15.581 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:15.581 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:15.581 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:15.582 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:15.582 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196432 2024-09-11T07:43:15.582 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.582 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:15.582 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.582 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:15.582 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509114+0000", 2024-09-11T07:43:15.582 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.582 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.582 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.582 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:15.582 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509114+0000", 2024-09-11T07:43:15.582 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.582 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.583 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.583 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:15.583 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509112+0000", 2024-09-11T07:43:15.583 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:43:15.583 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.583 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.583 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:15.583 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509119+0000", 2024-09-11T07:43:15.583 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3439999999999997e-06 2024-09-11T07:43:15.583 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.583 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.583 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:15.583 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509130+0000", 2024-09-11T07:43:15.583 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1066999999999999e-05 2024-09-11T07:43:15.583 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.584 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.584 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:15.584 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509136+0000", 2024-09-11T07:43:15.584 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0460000000000002e-06 2024-09-11T07:43:15.584 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.584 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.584 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:15.584 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509237+0000", 2024-09-11T07:43:15.584 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000101338 2024-09-11T07:43:15.584 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.584 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.584 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:15.585 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509361+0000", 2024-09-11T07:43:15.585 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000123287 2024-09-11T07:43:15.585 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.585 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.585 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,4,6,7", 2024-09-11T07:43:15.585 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509470+0000", 2024-09-11T07:43:15.585 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000109986 2024-09-11T07:43:15.585 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.585 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.585 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.585 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.515270+0000", 2024-09-11T07:43:15.585 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0057995310000000001 2024-09-11T07:43:15.585 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.585 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.586 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.586 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.515306+0000", 2024-09-11T07:43:15.586 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5920999999999997e-05 2024-09-11T07:43:15.586 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.586 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.586 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:15.586 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.515314+0000", 2024-09-11T07:43:15.586 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7680000000000008e-06 2024-09-11T07:43:15.586 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.586 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.586 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.586 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.659682+0000", 2024-09-11T07:43:15.587 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14436806299999999 2024-09-11T07:43:15.587 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.587 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.587 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.587 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.661408+0000", 2024-09-11T07:43:15.587 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0017266499999999999 2024-09-11T07:43:15.587 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.587 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.587 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:15.587 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.661490+0000", 2024-09-11T07:43:15.587 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1951000000000002e-05 2024-09-11T07:43:15.587 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.587 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.588 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:15.588 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.661622+0000", 2024-09-11T07:43:15.588 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013185900000000001 2024-09-11T07:43:15.588 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.588 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:15.588 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.588 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.588 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.588 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196436 9.f 9:f28fa10b:::benchmark_data_smithi022_133943_object69845:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:43:15.588 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.509149+0000", 2024-09-11T07:43:15.588 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 178.00473929, 2024-09-11T07:43:15.588 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.15385757, 2024-09-11T07:43:15.588 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:15.588 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:15.589 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:15.589 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:15.589 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:15.589 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196436 2024-09-11T07:43:15.589 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.589 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:15.589 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.589 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:15.589 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509149+0000", 2024-09-11T07:43:15.589 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.589 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.589 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.589 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:15.590 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509149+0000", 2024-09-11T07:43:15.590 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.590 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.590 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.590 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:15.590 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509148+0000", 2024-09-11T07:43:15.590 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:43:15.590 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.590 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.590 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:15.590 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509152+0000", 2024-09-11T07:43:15.590 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3309999999999997e-06 2024-09-11T07:43:15.590 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.590 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.591 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:15.591 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509158+0000", 2024-09-11T07:43:15.591 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0090000000000004e-06 2024-09-11T07:43:15.591 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.591 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.591 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:15.591 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509162+0000", 2024-09-11T07:43:15.591 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9149999999999998e-06 2024-09-11T07:43:15.591 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.591 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.591 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:15.591 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509226+0000", 2024-09-11T07:43:15.591 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3969999999999999e-05 2024-09-11T07:43:15.592 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.592 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.592 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:15.592 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509327+0000", 2024-09-11T07:43:15.592 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000101217 2024-09-11T07:43:15.592 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.592 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.592 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:43:15.592 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509426+0000", 2024-09-11T07:43:15.592 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8525999999999998e-05 2024-09-11T07:43:15.592 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.592 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.592 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:15.592 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.512876+0000", 2024-09-11T07:43:15.593 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003449939 2024-09-11T07:43:15.593 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.593 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.593 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.593 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.658382+0000", 2024-09-11T07:43:15.593 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.145506042 2024-09-11T07:43:15.593 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.593 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.593 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.593 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.662896+0000", 2024-09-11T07:43:15.593 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0045144510000000001 2024-09-11T07:43:15.593 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.594 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.594 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:15.594 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.662971+0000", 2024-09-11T07:43:15.594 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4895999999999996e-05 2024-09-11T07:43:15.594 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.594 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.594 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:15.594 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.663006+0000", 2024-09-11T07:43:15.594 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5099999999999999e-05 2024-09-11T07:43:15.594 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.594 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:15.594 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.594 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.594 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.594 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196438 9.7 9:e639c272:::benchmark_data_smithi022_133943_object69847:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:43:15.595 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.509172+0000", 2024-09-11T07:43:15.595 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 178.00471614099999, 2024-09-11T07:43:15.595 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.154085528, 2024-09-11T07:43:15.595 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:15.595 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:15.595 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:15.595 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:15.595 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:15.595 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196438 2024-09-11T07:43:15.595 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.595 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:15.595 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.595 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:15.595 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509172+0000", 2024-09-11T07:43:15.595 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.596 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.596 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.596 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:15.596 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509172+0000", 2024-09-11T07:43:15.596 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.596 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.596 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.596 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:15.596 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509171+0000", 2024-09-11T07:43:15.596 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:43:15.596 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.596 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.596 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:15.596 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509174+0000", 2024-09-11T07:43:15.596 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2679999999999999e-06 2024-09-11T07:43:15.597 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.597 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.597 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:15.597 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509179+0000", 2024-09-11T07:43:15.597 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9119999999999997e-06 2024-09-11T07:43:15.597 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.597 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.597 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:15.597 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509183+0000", 2024-09-11T07:43:15.597 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4410000000000002e-06 2024-09-11T07:43:15.597 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.597 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:15.599 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:15.599 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.510817+0000", 2024-09-11T07:43:15.599 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0016344020000000001 2024-09-11T07:43:15.599 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.600 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.600 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:15.600 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.510945+0000", 2024-09-11T07:43:15.600 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012800999999999999 2024-09-11T07:43:15.600 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.600 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.600 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2,3,7", 2024-09-11T07:43:15.600 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.511375+0000", 2024-09-11T07:43:15.600 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00042995300000000001 2024-09-11T07:43:15.600 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.600 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.600 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:15.600 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.659619+0000", 2024-09-11T07:43:15.601 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.148244129 2024-09-11T07:43:15.601 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.601 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.601 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.601 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.659674+0000", 2024-09-11T07:43:15.601 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4605000000000003e-05 2024-09-11T07:43:15.601 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.601 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.601 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.601 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.659715+0000", 2024-09-11T07:43:15.601 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0935e-05 2024-09-11T07:43:15.601 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.601 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.602 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.602 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.662817+0000", 2024-09-11T07:43:15.602 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0031017319999999998 2024-09-11T07:43:15.602 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.602 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.602 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.602 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.663041+0000", 2024-09-11T07:43:15.602 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000224303 2024-09-11T07:43:15.602 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.602 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.602 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:15.602 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.663089+0000", 2024-09-11T07:43:15.602 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8464000000000002e-05 2024-09-11T07:43:15.603 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.603 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.603 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:15.603 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.663257+0000", 2024-09-11T07:43:15.603 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016791399999999999 2024-09-11T07:43:15.603 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.603 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:15.603 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.603 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.603 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.603 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196563 9.6 9:63ebaa83:::benchmark_data_smithi022_133943_object69972:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:43:15.603 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.771425+0000", 2024-09-11T07:43:15.603 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 177.74246232499999, 2024-09-11T07:43:15.603 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.153416993, 2024-09-11T07:43:15.603 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:15.604 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:15.604 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:15.604 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:15.604 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:15.604 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196563 2024-09-11T07:43:15.604 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.604 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:15.604 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.604 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:15.604 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771425+0000", 2024-09-11T07:43:15.604 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.604 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.604 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.604 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:15.604 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771425+0000", 2024-09-11T07:43:15.605 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.605 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.605 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.605 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:15.605 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771424+0000", 2024-09-11T07:43:15.605 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:43:15.605 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.605 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.605 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:15.605 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771430+0000", 2024-09-11T07:43:15.605 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4540000000000004e-06 2024-09-11T07:43:15.605 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.606 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.606 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:15.606 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771438+0000", 2024-09-11T07:43:15.606 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.418e-06 2024-09-11T07:43:15.606 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.606 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.606 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:15.606 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771442+0000", 2024-09-11T07:43:15.606 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.622e-06 2024-09-11T07:43:15.606 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.606 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.606 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:15.606 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771560+0000", 2024-09-11T07:43:15.606 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000117298 2024-09-11T07:43:15.606 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.607 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.607 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:15.607 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771669+0000", 2024-09-11T07:43:15.607 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000109386 2024-09-11T07:43:15.607 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.607 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.607 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:43:15.607 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.772085+0000", 2024-09-11T07:43:15.607 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00041578699999999998 2024-09-11T07:43:15.607 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.607 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.607 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:15.607 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.776041+0000", 2024-09-11T07:43:15.608 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0039559069999999998 2024-09-11T07:43:15.608 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.608 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.608 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.608 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.782755+0000", 2024-09-11T07:43:15.608 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0067147760000000004 2024-09-11T07:43:15.608 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.608 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.608 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.608 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924754+0000", 2024-09-11T07:43:15.608 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14199824599999999 2024-09-11T07:43:15.608 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.608 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.608 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:15.609 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924797+0000", 2024-09-11T07:43:15.609 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2988999999999998e-05 2024-09-11T07:43:15.609 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.609 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.609 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:15.609 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924842+0000", 2024-09-11T07:43:15.609 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5884000000000003e-05 2024-09-11T07:43:15.609 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.609 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:15.609 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.609 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.609 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.609 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196577 9.13 9:c894ba9e:::benchmark_data_smithi022_133943_object69986:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:43:15.609 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.785754+0000", 2024-09-11T07:43:15.610 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 177.72813420599999, 2024-09-11T07:43:15.610 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.167572637, 2024-09-11T07:43:15.610 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:15.610 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:15.610 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:15.610 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:15.610 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:15.610 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196577 2024-09-11T07:43:15.610 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.610 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:15.610 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.610 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:15.610 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785754+0000", 2024-09-11T07:43:15.610 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.611 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.611 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.611 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:15.611 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785754+0000", 2024-09-11T07:43:15.611 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.611 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.611 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.611 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:15.611 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785752+0000", 2024-09-11T07:43:15.611 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:43:15.611 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.611 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.611 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:15.612 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785757+0000", 2024-09-11T07:43:15.612 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0010000000000001e-06 2024-09-11T07:43:15.612 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.612 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.612 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:15.612 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785765+0000", 2024-09-11T07:43:15.612 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2670000000000006e-06 2024-09-11T07:43:15.612 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.612 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.613 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:15.613 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785771+0000", 2024-09-11T07:43:15.613 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.242e-06 2024-09-11T07:43:15.613 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.613 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.613 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:15.613 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924515+0000", 2024-09-11T07:43:15.613 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13874467300000001 2024-09-11T07:43:15.613 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.613 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.613 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:15.613 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924668+0000", 2024-09-11T07:43:15.614 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015245999999999999 2024-09-11T07:43:15.614 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.614 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.614 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,6,7", 2024-09-11T07:43:15.614 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.925194+0000", 2024-09-11T07:43:15.614 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00052637400000000003 2024-09-11T07:43:15.614 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.614 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.614 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.614 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.930777+0000", 2024-09-11T07:43:15.614 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00558319 2024-09-11T07:43:15.614 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.614 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.615 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.615 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.930807+0000", 2024-09-11T07:43:15.615 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9363999999999999e-05 2024-09-11T07:43:15.615 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.615 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.615 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:15.615 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.931376+0000", 2024-09-11T07:43:15.615 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00056952599999999999 2024-09-11T07:43:15.615 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.615 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.615 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.615 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.953159+0000", 2024-09-11T07:43:15.615 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.021782632999999999 2024-09-11T07:43:15.615 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.616 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.616 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:15.616 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.953206+0000", 2024-09-11T07:43:15.616 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7624999999999999e-05 2024-09-11T07:43:15.616 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.616 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.616 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:15.616 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.953326+0000", 2024-09-11T07:43:15.616 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000119883 2024-09-11T07:43:15.616 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.616 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:15.616 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.617 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.617 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.617 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:231567 9.6 9:62bf3667:::benchmark_data_smithi022_133943_object104976:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:43:15.617 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:56.613955+0000", 2024-09-11T07:43:15.617 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 138.89993295299999, 2024-09-11T07:43:15.617 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14854477399999999, 2024-09-11T07:43:15.617 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:15.617 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:15.617 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:15.617 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:15.617 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:15.617 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 231567 2024-09-11T07:43:15.618 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.618 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:15.618 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.618 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:15.618 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.613955+0000", 2024-09-11T07:43:15.618 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.618 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.618 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.618 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:15.618 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.613955+0000", 2024-09-11T07:43:15.618 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.618 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.618 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.619 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:15.619 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.613953+0000", 2024-09-11T07:43:15.619 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:43:15.619 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.619 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.619 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:15.619 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.613960+0000", 2024-09-11T07:43:15.619 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3200000000000002e-06 2024-09-11T07:43:15.619 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.619 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.619 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:15.619 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.613988+0000", 2024-09-11T07:43:15.619 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.745e-05 2024-09-11T07:43:15.619 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.620 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.620 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:15.620 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.613994+0000", 2024-09-11T07:43:15.620 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0270000000000001e-06 2024-09-11T07:43:15.620 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.620 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.620 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:15.620 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.614040+0000", 2024-09-11T07:43:15.620 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6403999999999998e-05 2024-09-11T07:43:15.620 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.620 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.620 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:15.620 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.614209+0000", 2024-09-11T07:43:15.621 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016869000000000001 2024-09-11T07:43:15.621 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.621 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.621 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:43:15.621 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.614696+0000", 2024-09-11T07:43:15.621 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00048757100000000001 2024-09-11T07:43:15.621 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.621 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.621 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:15.621 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.617583+0000", 2024-09-11T07:43:15.621 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0028868129999999998 2024-09-11T07:43:15.621 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.621 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.621 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.622 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.618363+0000", 2024-09-11T07:43:15.622 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00078008600000000004 2024-09-11T07:43:15.622 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.622 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.622 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.622 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.762435+0000", 2024-09-11T07:43:15.622 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.144071592 2024-09-11T07:43:15.622 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.622 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.622 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:15.622 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.762464+0000", 2024-09-11T07:43:15.622 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9068999999999999e-05 2024-09-11T07:43:15.622 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.623 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.623 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:15.623 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.762500+0000", 2024-09-11T07:43:15.623 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5751999999999999e-05 2024-09-11T07:43:15.623 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.623 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:15.623 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.623 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.623 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.623 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:232544 9.13 9:cb4ef5ab:::benchmark_data_smithi022_133943_object105953:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:43:15.623 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:57.689887+0000", 2024-09-11T07:43:15.623 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 137.82400084299999, 2024-09-11T07:43:15.623 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.186799876, 2024-09-11T07:43:15.623 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:15.624 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:15.624 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:15.624 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:15.624 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:15.624 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 232544 2024-09-11T07:43:15.624 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.624 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:15.624 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.624 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:15.624 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689887+0000", 2024-09-11T07:43:15.624 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.624 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.624 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.625 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:15.625 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689887+0000", 2024-09-11T07:43:15.625 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.625 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.625 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.625 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:15.625 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689885+0000", 2024-09-11T07:43:15.625 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:43:15.625 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.625 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.625 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:15.625 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689891+0000", 2024-09-11T07:43:15.625 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0310000000000004e-06 2024-09-11T07:43:15.626 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.626 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.626 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:15.626 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689903+0000", 2024-09-11T07:43:15.626 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2167000000000001e-05 2024-09-11T07:43:15.626 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.626 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.626 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:15.626 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689909+0000", 2024-09-11T07:43:15.626 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7620000000000001e-06 2024-09-11T07:43:15.626 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.626 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.626 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:15.626 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689947+0000", 2024-09-11T07:43:15.627 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7960999999999998e-05 2024-09-11T07:43:15.627 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.627 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.627 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:15.627 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.690025+0000", 2024-09-11T07:43:15.627 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7885000000000004e-05 2024-09-11T07:43:15.627 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.627 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.627 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:43:15.627 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.690473+0000", 2024-09-11T07:43:15.627 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00044783300000000002 2024-09-11T07:43:15.627 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.627 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.627 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:15.627 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.692445+0000", 2024-09-11T07:43:15.628 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001972461 2024-09-11T07:43:15.628 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.628 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.628 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.628 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.692900+0000", 2024-09-11T07:43:15.628 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00045435300000000001 2024-09-11T07:43:15.628 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.628 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.628 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.628 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.876606+0000", 2024-09-11T07:43:15.628 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18370584700000001 2024-09-11T07:43:15.628 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.628 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.628 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:15.629 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.876647+0000", 2024-09-11T07:43:15.629 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1140999999999998e-05 2024-09-11T07:43:15.629 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.629 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.629 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:15.629 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.876687+0000", 2024-09-11T07:43:15.629 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0153e-05 2024-09-11T07:43:15.629 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.629 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:15.629 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.629 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.629 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.629 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:234546 9.6 9:65687430:::benchmark_data_smithi022_133943_object107955:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e320)", 2024-09-11T07:43:15.629 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:41:00.355245+0000", 2024-09-11T07:43:15.630 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 135.15864260999999, 2024-09-11T07:43:15.630 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17097563599999999, 2024-09-11T07:43:15.630 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:15.630 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:15.630 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:15.630 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:15.630 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:15.630 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 234546 2024-09-11T07:43:15.630 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.630 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:15.630 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.630 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:15.630 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355245+0000", 2024-09-11T07:43:15.631 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.631 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.631 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.631 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:15.631 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355245+0000", 2024-09-11T07:43:15.631 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.631 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.631 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.631 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:15.631 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355244+0000", 2024-09-11T07:43:15.631 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:43:15.631 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.631 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.631 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:15.632 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355248+0000", 2024-09-11T07:43:15.632 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.529e-06 2024-09-11T07:43:15.632 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.632 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.632 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:15.632 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355255+0000", 2024-09-11T07:43:15.632 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1289999999999998e-06 2024-09-11T07:43:15.632 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.632 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.632 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:15.632 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355259+0000", 2024-09-11T07:43:15.632 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7149999999999997e-06 2024-09-11T07:43:15.632 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.633 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.633 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:15.633 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355306+0000", 2024-09-11T07:43:15.633 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6631999999999997e-05 2024-09-11T07:43:15.633 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.633 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.633 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:15.633 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355383+0000", 2024-09-11T07:43:15.633 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6509999999999998e-05 2024-09-11T07:43:15.633 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.633 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.633 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:43:15.633 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355796+0000", 2024-09-11T07:43:15.633 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00041318000000000003 2024-09-11T07:43:15.634 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.634 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.634 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:15.634 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.357852+0000", 2024-09-11T07:43:15.634 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0020561059999999998 2024-09-11T07:43:15.634 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.634 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.634 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.634 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.359012+0000", 2024-09-11T07:43:15.634 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001160295 2024-09-11T07:43:15.634 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.634 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.634 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:15.634 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.526128+0000", 2024-09-11T07:43:15.634 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.167115923 2024-09-11T07:43:15.635 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.635 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.635 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:15.635 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.526176+0000", 2024-09-11T07:43:15.635 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8310000000000003e-05 2024-09-11T07:43:15.635 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.635 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.635 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:15.635 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.526221+0000", 2024-09-11T07:43:15.635 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4447000000000003e-05 2024-09-11T07:43:15.635 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.635 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:15.635 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.635 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.635 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.636 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:70288 10.6 e356/346 10:6714b47c:::benchmark_data_smithi022_163866_object70287:head v 356'4440, mlcod=356'4437)", 2024-09-11T07:43:15.636 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:53.786739+0000", 2024-09-11T07:43:15.636 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 21.727149170000001, 2024-09-11T07:43:15.636 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.164540297, 2024-09-11T07:43:15.636 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:15.636 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:15.636 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:15.636 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.636 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:15.636 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786739+0000", 2024-09-11T07:43:15.636 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.636 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.636 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.636 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:15.637 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786739+0000", 2024-09-11T07:43:15.637 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.637 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.637 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.637 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:15.637 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786737+0000", 2024-09-11T07:43:15.637 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:43:15.637 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.637 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.637 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:15.637 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786793+0000", 2024-09-11T07:43:15.637 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.516e-05 2024-09-11T07:43:15.637 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.637 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.638 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:15.638 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786802+0000", 2024-09-11T07:43:15.638 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0149999999999992e-06 2024-09-11T07:43:15.638 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.638 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.638 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:15.638 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786807+0000", 2024-09-11T07:43:15.638 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3900000000000001e-06 2024-09-11T07:43:15.638 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.638 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.638 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:15.638 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786852+0000", 2024-09-11T07:43:15.638 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5426000000000002e-05 2024-09-11T07:43:15.639 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.639 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.639 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:15.639 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786869+0000", 2024-09-11T07:43:15.639 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6477000000000002e-05 2024-09-11T07:43:15.639 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.639 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.639 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:15.639 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951240+0000", 2024-09-11T07:43:15.639 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16437090500000001 2024-09-11T07:43:15.639 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.639 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.639 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:15.639 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951279+0000", 2024-09-11T07:43:15.639 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9072999999999997e-05 2024-09-11T07:43:15.639 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.640 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:15.640 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.640 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.640 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.640 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:70292 10.a e356/341 10:547c94c8:::benchmark_data_smithi022_163866_object70291:head v 356'4331, mlcod=356'4329)", 2024-09-11T07:43:15.640 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:53.788575+0000", 2024-09-11T07:43:15.640 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 21.725312461000001, 2024-09-11T07:43:15.640 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16920853899999999, 2024-09-11T07:43:15.640 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:15.640 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:15.640 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:15.640 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.641 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:15.641 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.788575+0000", 2024-09-11T07:43:15.641 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.641 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.641 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.641 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:15.641 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.788575+0000", 2024-09-11T07:43:15.641 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.641 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.641 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.641 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:15.641 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.788574+0000", 2024-09-11T07:43:15.641 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:43:15.642 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.642 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.642 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:15.642 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.788630+0000", 2024-09-11T07:43:15.642 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6097999999999997e-05 2024-09-11T07:43:15.642 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.642 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.642 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:15.642 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951213+0000", 2024-09-11T07:43:15.642 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16258271199999999 2024-09-11T07:43:15.642 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.642 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.642 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:15.642 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951228+0000", 2024-09-11T07:43:15.643 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4562999999999999e-05 2024-09-11T07:43:15.643 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.643 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.643 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:15.643 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951332+0000", 2024-09-11T07:43:15.643 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000104002 2024-09-11T07:43:15.643 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.643 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.643 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:15.643 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951352+0000", 2024-09-11T07:43:15.643 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0638000000000001e-05 2024-09-11T07:43:15.643 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.643 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.643 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:15.644 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.957755+0000", 2024-09-11T07:43:15.644 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.006402451 2024-09-11T07:43:15.644 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.644 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.644 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:15.644 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.957784+0000", 2024-09-11T07:43:15.644 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9269e-05 2024-09-11T07:43:15.644 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.644 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:15.644 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.644 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.644 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.644 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:70291 10.3 e356/341 10:cdbcfc7c:::benchmark_data_smithi022_163866_object70290:head v 356'4299, mlcod=356'4297)", 2024-09-11T07:43:15.644 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:53.789090+0000", 2024-09-11T07:43:15.645 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 21.724797621, 2024-09-11T07:43:15.645 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16609167599999999, 2024-09-11T07:43:15.645 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:15.645 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:15.645 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:15.645 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.645 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:15.645 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.789090+0000", 2024-09-11T07:43:15.645 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.645 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.645 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.645 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:15.645 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.789090+0000", 2024-09-11T07:43:15.646 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:15.646 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.646 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.646 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:15.646 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.789089+0000", 2024-09-11T07:43:15.646 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:43:15.646 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.646 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.646 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:15.646 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.789132+0000", 2024-09-11T07:43:15.646 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2901000000000001e-05 2024-09-11T07:43:15.646 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.646 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.646 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:15.647 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951213+0000", 2024-09-11T07:43:15.647 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16208165599999999 2024-09-11T07:43:15.647 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.647 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.647 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:15.647 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951228+0000", 2024-09-11T07:43:15.647 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4430000000000001e-05 2024-09-11T07:43:15.647 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.647 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.647 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:15.647 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951350+0000", 2024-09-11T07:43:15.647 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000122213 2024-09-11T07:43:15.647 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.647 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.647 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:15.648 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951369+0000", 2024-09-11T07:43:15.648 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8819000000000001e-05 2024-09-11T07:43:15.648 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.648 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.648 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:15.648 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.955146+0000", 2024-09-11T07:43:15.648 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003776916 2024-09-11T07:43:15.648 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:15.648 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:15.649 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:15.649 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.955182+0000", 2024-09-11T07:43:15.649 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6123e-05 2024-09-11T07:43:15.649 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.649 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:15.649 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.649 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:15.649 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:15.649 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:43:15.650 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:15.650 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:15.751 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:15.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:15.777 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:15.777 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:15.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:15 smithi104 ceph-mon[33507]: pgmap v776: 148 pgs: 1 peering, 147 active+clean; 5.7 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 113 MiB/s wr, 1.81k op/s; 18 B/s, 1 objects/s recovering 2024-09-11T07:43:15.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:15 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:43:15.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:15 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:43:15.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:15 smithi104 ceph-mon[33507]: osdmap e363: 8 total, 8 up, 8 in 2024-09-11T07:43:15.878 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:15.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:15.913 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:15.913 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:15.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:15 smithi022 ceph-mon[29462]: pgmap v776: 148 pgs: 1 peering, 147 active+clean; 5.7 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 113 MiB/s wr, 1.81k op/s; 18 B/s, 1 objects/s recovering 2024-09-11T07:43:15.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:15 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:43:15.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:15 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:43:15.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:15 smithi022 ceph-mon[29462]: osdmap e363: 8 total, 8 up, 8 in 2024-09-11T07:43:15.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:15 smithi022 ceph-mon[37330]: pgmap v776: 148 pgs: 1 peering, 147 active+clean; 5.7 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 113 MiB/s wr, 1.81k op/s; 18 B/s, 1 objects/s recovering 2024-09-11T07:43:15.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:15 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:43:15.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:15 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:43:15.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:15 smithi022 ceph-mon[37330]: osdmap e363: 8 total, 8 up, 8 in 2024-09-11T07:43:15.960 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:2024-09-11T07:43:15.959997+0000 min lat: 0.00106648 max lat: 0.408634 avg lat: 0.00997504 2024-09-11T07:43:15.960 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T07:43:15.960 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 60 16 96140 96124 100.112 55.5625 0.00440425 0.00997504 2024-09-11T07:43:16.014 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:16.051 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:16.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:16.052 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:16.153 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:16.185 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:16.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:16.186 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:16.287 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:16.309 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:16.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:16.310 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:16.411 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:16.434 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:16.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:16.435 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:16.494 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-11T07:43:16.535 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:16.561 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:16.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:16.563 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:16.663 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:16.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:16.710 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:16.711 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:16.812 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:16.825 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:43:16.844 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:43:16.845 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:43:16.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:16.882 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:16.882 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:16.960 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 61 16 97027 97011 99.3796 55.4375 0.00527488 0.0100568 2024-09-11T07:43:16.963 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:16 smithi022 ceph-mon[29462]: 4.9 scrub starts 2024-09-11T07:43:16.964 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:16 smithi022 ceph-mon[29462]: 4.9 scrub ok 2024-09-11T07:43:16.964 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:16 smithi022 ceph-mon[29462]: 2.14 scrub starts 2024-09-11T07:43:16.964 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:16 smithi022 ceph-mon[29462]: 2.14 scrub ok 2024-09-11T07:43:16.964 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:16 smithi022 ceph-mon[29462]: 3.0 scrub starts 2024-09-11T07:43:16.964 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:16 smithi022 ceph-mon[29462]: 3.0 scrub ok 2024-09-11T07:43:16.964 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:16 smithi022 ceph-mon[29462]: pgmap v778: 148 pgs: 148 active+clean; 5.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 852 B/s rd, 61 MiB/s wr, 983 op/s; 91 B/s, 1 objects/s recovering 2024-09-11T07:43:16.964 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:16 smithi022 ceph-mon[37330]: 4.9 scrub starts 2024-09-11T07:43:16.964 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:16 smithi022 ceph-mon[37330]: 4.9 scrub ok 2024-09-11T07:43:16.964 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:16 smithi022 ceph-mon[37330]: 2.14 scrub starts 2024-09-11T07:43:16.965 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:16 smithi022 ceph-mon[37330]: 2.14 scrub ok 2024-09-11T07:43:16.965 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:16 smithi022 ceph-mon[37330]: 3.0 scrub starts 2024-09-11T07:43:16.965 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:16 smithi022 ceph-mon[37330]: 3.0 scrub ok 2024-09-11T07:43:16.965 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:16 smithi022 ceph-mon[37330]: pgmap v778: 148 pgs: 148 active+clean; 5.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 852 B/s rd, 61 MiB/s wr, 983 op/s; 91 B/s, 1 objects/s recovering 2024-09-11T07:43:16.972 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:16 smithi104 ceph-mon[33507]: 4.9 scrub starts 2024-09-11T07:43:16.972 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:16 smithi104 ceph-mon[33507]: 4.9 scrub ok 2024-09-11T07:43:16.972 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:16 smithi104 ceph-mon[33507]: 2.14 scrub starts 2024-09-11T07:43:16.972 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:16 smithi104 ceph-mon[33507]: 2.14 scrub ok 2024-09-11T07:43:16.972 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:16 smithi104 ceph-mon[33507]: 3.0 scrub starts 2024-09-11T07:43:16.973 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:16 smithi104 ceph-mon[33507]: 3.0 scrub ok 2024-09-11T07:43:16.973 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:16 smithi104 ceph-mon[33507]: pgmap v778: 148 pgs: 148 active+clean; 5.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 852 B/s rd, 61 MiB/s wr, 983 op/s; 91 B/s, 1 objects/s recovering 2024-09-11T07:43:16.983 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:17.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:17.022 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:17.022 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:17.123 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:17.168 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:17.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:17.169 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:17.270 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:17.307 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:17.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:17.308 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:17.409 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:17.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:17.446 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:17.447 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:17.548 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:17.584 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:17.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:17.585 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:17.686 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:17.723 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:17.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:17.724 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:17.824 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:17.865 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:17.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:17.866 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:17 smithi022 ceph-mon[29462]: 6.9 deep-scrub starts 2024-09-11T07:43:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:17 smithi022 ceph-mon[29462]: 6.e scrub starts 2024-09-11T07:43:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:17 smithi022 ceph-mon[29462]: 6.14 scrub starts 2024-09-11T07:43:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:17 smithi022 ceph-mon[29462]: 6.14 scrub ok 2024-09-11T07:43:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:17 smithi022 ceph-mon[29462]: 6.e scrub ok 2024-09-11T07:43:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:17 smithi022 ceph-mon[29462]: 6.9 deep-scrub ok 2024-09-11T07:43:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:17 smithi022 ceph-mon[29462]: 6.b scrub starts 2024-09-11T07:43:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:17 smithi022 ceph-mon[29462]: 6.b scrub ok 2024-09-11T07:43:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:17 smithi022 ceph-mon[29462]: 2.d scrub starts 2024-09-11T07:43:17.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:17 smithi022 ceph-mon[29462]: 2.d scrub ok 2024-09-11T07:43:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:17 smithi022 ceph-mon[37330]: 6.9 deep-scrub starts 2024-09-11T07:43:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:17 smithi022 ceph-mon[37330]: 6.e scrub starts 2024-09-11T07:43:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:17 smithi022 ceph-mon[37330]: 6.14 scrub starts 2024-09-11T07:43:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:17 smithi022 ceph-mon[37330]: 6.14 scrub ok 2024-09-11T07:43:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:17 smithi022 ceph-mon[37330]: 6.e scrub ok 2024-09-11T07:43:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:17 smithi022 ceph-mon[37330]: 6.9 deep-scrub ok 2024-09-11T07:43:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:17 smithi022 ceph-mon[37330]: 6.b scrub starts 2024-09-11T07:43:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:17 smithi022 ceph-mon[37330]: 6.b scrub ok 2024-09-11T07:43:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:17 smithi022 ceph-mon[37330]: 2.d scrub starts 2024-09-11T07:43:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:17 smithi022 ceph-mon[37330]: 2.d scrub ok 2024-09-11T07:43:17.960 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 62 16 98335 98319 99.0952 81.75 0.00472605 0.0100868 2024-09-11T07:43:17.966 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:17.997 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:17.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:17.998 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:18.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:17 smithi104 ceph-mon[33507]: 6.9 deep-scrub starts 2024-09-11T07:43:18.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:17 smithi104 ceph-mon[33507]: 6.e scrub starts 2024-09-11T07:43:18.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:17 smithi104 ceph-mon[33507]: 6.14 scrub starts 2024-09-11T07:43:18.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:17 smithi104 ceph-mon[33507]: 6.14 scrub ok 2024-09-11T07:43:18.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:17 smithi104 ceph-mon[33507]: 6.e scrub ok 2024-09-11T07:43:18.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:17 smithi104 ceph-mon[33507]: 6.9 deep-scrub ok 2024-09-11T07:43:18.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:17 smithi104 ceph-mon[33507]: 6.b scrub starts 2024-09-11T07:43:18.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:17 smithi104 ceph-mon[33507]: 6.b scrub ok 2024-09-11T07:43:18.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:17 smithi104 ceph-mon[33507]: 2.d scrub starts 2024-09-11T07:43:18.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:17 smithi104 ceph-mon[33507]: 2.d scrub ok 2024-09-11T07:43:18.099 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:18.130 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:18.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:18.131 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:18.232 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:18.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:18.271 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:18.271 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:18.372 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:18.409 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:18.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:18.410 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:18.511 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:18.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:18.550 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:18.550 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:18.651 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:18.690 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:18.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:18.691 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:18.733 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T07:43:18.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:18 smithi104 ceph-mon[33507]: 6.7 scrub starts 2024-09-11T07:43:18.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:18 smithi104 ceph-mon[33507]: 6.7 scrub ok 2024-09-11T07:43:18.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:18 smithi104 ceph-mon[33507]: 6.a scrub starts 2024-09-11T07:43:18.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:18 smithi104 ceph-mon[33507]: 6.a scrub ok 2024-09-11T07:43:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:18 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:43:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:18 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:43:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:18 smithi104 ceph-mon[33507]: pgmap v779: 148 pgs: 148 active+clean; 5.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 651 B/s rd, 47 MiB/s wr, 750 op/s; 69 B/s, 1 objects/s recovering 2024-09-11T07:43:18.792 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:18.820 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:18.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:18.821 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:18.922 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:18.961 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 63 16 99553 99537 98.73 76.125 0.0237501 0.0101233 2024-09-11T07:43:18.962 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:18.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:18.963 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:19.017 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:18 smithi022 ceph-mon[29462]: 6.7 scrub starts 2024-09-11T07:43:19.017 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:18 smithi022 ceph-mon[29462]: 6.7 scrub ok 2024-09-11T07:43:19.017 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:18 smithi022 ceph-mon[29462]: 6.a scrub starts 2024-09-11T07:43:19.017 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:18 smithi022 ceph-mon[29462]: 6.a scrub ok 2024-09-11T07:43:19.017 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:18 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:43:19.017 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:18 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:43:19.017 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:18 smithi022 ceph-mon[29462]: pgmap v779: 148 pgs: 148 active+clean; 5.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 651 B/s rd, 47 MiB/s wr, 750 op/s; 69 B/s, 1 objects/s recovering 2024-09-11T07:43:19.018 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:18 smithi022 ceph-mon[37330]: 6.7 scrub starts 2024-09-11T07:43:19.018 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:18 smithi022 ceph-mon[37330]: 6.7 scrub ok 2024-09-11T07:43:19.018 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:18 smithi022 ceph-mon[37330]: 6.a scrub starts 2024-09-11T07:43:19.018 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:18 smithi022 ceph-mon[37330]: 6.a scrub ok 2024-09-11T07:43:19.018 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:18 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:43:19.018 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:18 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:43:19.018 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:18 smithi022 ceph-mon[37330]: pgmap v779: 148 pgs: 148 active+clean; 5.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 651 B/s rd, 47 MiB/s wr, 750 op/s; 69 B/s, 1 objects/s recovering 2024-09-11T07:43:19.064 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:19.106 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:19.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:19.107 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:19.208 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:19.219 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:43:19.219 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2024-09-11T07:43:19.219 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:19.219 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:99838 10.f e364/341 10:fca6c31d:::benchmark_data_smithi022_163866_object99837:head v 364'6148, mlcod=364'6146)", 2024-09-11T07:43:19.219 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:43:19.214272+0000", 2024-09-11T07:43:19.219 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.0031658929999999999, 2024-09-11T07:43:19.219 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003194756, 2024-09-11T07:43:19.219 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:19.219 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2024-09-11T07:43:19.220 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:19.220 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:19.220 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:19.220 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:19.214272+0000", 2024-09-11T07:43:19.220 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:19.220 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:19.220 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:19.220 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:19.220 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:19.214272+0000", 2024-09-11T07:43:19.220 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:19.220 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:19.220 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:19.220 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:19.221 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:19.214270+0000", 2024-09-11T07:43:19.221 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:43:19.221 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:19.221 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:19.221 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:19.221 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:19.214317+0000", 2024-09-11T07:43:19.221 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6966999999999999e-05 2024-09-11T07:43:19.221 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:19.221 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:19.221 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:19.221 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:19.215494+0000", 2024-09-11T07:43:19.221 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00117727 2024-09-11T07:43:19.222 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:19.222 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:19.222 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:19.222 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:19.215504+0000", 2024-09-11T07:43:19.222 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8209999999999997e-06 2024-09-11T07:43:19.222 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:19.222 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:19.222 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:19.222 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:19.215570+0000", 2024-09-11T07:43:19.222 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5778999999999995e-05 2024-09-11T07:43:19.222 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:19.222 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:19.222 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:19.222 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:19.215590+0000", 2024-09-11T07:43:19.223 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9669000000000001e-05 2024-09-11T07:43:19.223 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:19.223 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:19.223 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:19.223 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:19.223 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:19.223 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:99839 10.0 e364/341 10:08d5789c:::benchmark_data_smithi022_163866_object99838:head v 364'6133, mlcod=364'6131)", 2024-09-11T07:43:19.223 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:43:19.215718+0000", 2024-09-11T07:43:19.223 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.001719503, 2024-09-11T07:43:19.223 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001808696, 2024-09-11T07:43:19.223 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:19.223 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2024-09-11T07:43:19.224 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:19.224 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:19.224 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:19.224 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:19.215718+0000", 2024-09-11T07:43:19.224 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:19.224 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:19.224 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:19.224 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:19.224 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:19.215718+0000", 2024-09-11T07:43:19.224 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:19.224 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:19.224 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:19.224 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:19.225 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:19.215717+0000", 2024-09-11T07:43:19.225 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:43:19.225 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:19.225 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:19.225 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:19.225 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:19.215755+0000", 2024-09-11T07:43:19.225 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8039e-05 2024-09-11T07:43:19.225 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:19.225 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:19.225 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:19.225 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:19.215764+0000", 2024-09-11T07:43:19.225 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7860000000000008e-06 2024-09-11T07:43:19.225 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:19.226 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:19.226 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:19.226 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:19.215769+0000", 2024-09-11T07:43:19.226 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8840000000000002e-06 2024-09-11T07:43:19.226 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:19.226 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:19.226 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:19.226 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:19.215839+0000", 2024-09-11T07:43:19.226 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0115000000000002e-05 2024-09-11T07:43:19.226 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:19.226 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:19.226 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:19.226 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:19.215856+0000", 2024-09-11T07:43:19.226 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6501e-05 2024-09-11T07:43:19.227 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:19.227 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:19.227 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:19.227 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:19.227 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2024-09-11T07:43:19.227 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 2 2024-09-11T07:43:19.227 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:43:19.242 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:19.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:19.243 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:19.325 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:43:19.344 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:19.392 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:19.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:19.393 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:19.494 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:19.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:19.529 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:19.530 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:19.630 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:19.668 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:19.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:19.669 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:19.770 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:19.808 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:19.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:19.810 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:19.904 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:19 smithi104 ceph-mon[33507]: 6.10 scrub starts 2024-09-11T07:43:19.904 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:19 smithi104 ceph-mon[33507]: 6.10 scrub ok 2024-09-11T07:43:19.904 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:19 smithi104 ceph-mon[33507]: 10.f scrub starts 2024-09-11T07:43:19.904 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:19 smithi104 ceph-mon[33507]: osdmap e364: 8 total, 8 up, 8 in 2024-09-11T07:43:19.911 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:19.931 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:19 smithi022 ceph-mon[29462]: 6.10 scrub starts 2024-09-11T07:43:19.931 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:19 smithi022 ceph-mon[29462]: 6.10 scrub ok 2024-09-11T07:43:19.931 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:19 smithi022 ceph-mon[29462]: 10.f scrub starts 2024-09-11T07:43:19.932 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:19 smithi022 ceph-mon[29462]: osdmap e364: 8 total, 8 up, 8 in 2024-09-11T07:43:19.932 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:19 smithi022 ceph-mon[37330]: 6.10 scrub starts 2024-09-11T07:43:19.932 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:19 smithi022 ceph-mon[37330]: 6.10 scrub ok 2024-09-11T07:43:19.932 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:19 smithi022 ceph-mon[37330]: 10.f scrub starts 2024-09-11T07:43:19.932 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:19 smithi022 ceph-mon[37330]: osdmap e364: 8 total, 8 up, 8 in 2024-09-11T07:43:19.944 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-11T07:43:19.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:19.946 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:19.946 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:19.961 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 64 16 100642 100626 98.2509 68.0625 0.00906853 0.010173 2024-09-11T07:43:20.048 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:20.089 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:20.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:20.091 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:20.192 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:20.239 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:20.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:20.241 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:20.279 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:43:19 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:43:20.342 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:20.376 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:43:20.395 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:20.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:20.396 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:20.497 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:20.530 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:20.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:20.532 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:20.632 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:20.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:43:20 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:43:20] "GET /metrics HTTP/1.1" 200 224882 "" "Prometheus/2.43.0" 2024-09-11T07:43:20.669 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:20.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:20.670 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:20.771 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:20.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:20 smithi104 ceph-mon[33507]: 2.18 scrub starts 2024-09-11T07:43:20.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:20 smithi104 ceph-mon[33507]: 2.18 scrub ok 2024-09-11T07:43:20.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:20 smithi104 ceph-mon[33507]: 5.19 scrub starts 2024-09-11T07:43:20.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:20 smithi104 ceph-mon[33507]: 5.19 scrub ok 2024-09-11T07:43:20.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:20 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:43:20.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:20 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:43:20.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:20 smithi104 ceph-mon[33507]: osdmap e365: 8 total, 8 up, 8 in 2024-09-11T07:43:20.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:20 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:43:20.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:20 smithi104 ceph-mon[33507]: pgmap v782: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 853 B/s rd, 63 MiB/s wr, 1.01k op/s; 73 B/s, 1 objects/s recovering 2024-09-11T07:43:20.812 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:20.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:20.813 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:20.914 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:20.937 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:20.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:20.938 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:20.961 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 65 16 101632 101616 97.6912 61.875 0.0893191 0.0102277 2024-09-11T07:43:21.023 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:20 smithi022 ceph-mon[29462]: 2.18 scrub starts 2024-09-11T07:43:21.023 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:20 smithi022 ceph-mon[29462]: 2.18 scrub ok 2024-09-11T07:43:21.023 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:20 smithi022 ceph-mon[29462]: 5.19 scrub starts 2024-09-11T07:43:21.023 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:20 smithi022 ceph-mon[29462]: 5.19 scrub ok 2024-09-11T07:43:21.023 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:20 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:43:21.023 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:20 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:43:21.023 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:20 smithi022 ceph-mon[29462]: osdmap e365: 8 total, 8 up, 8 in 2024-09-11T07:43:21.023 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:20 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:43:21.023 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:20 smithi022 ceph-mon[29462]: pgmap v782: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 853 B/s rd, 63 MiB/s wr, 1.01k op/s; 73 B/s, 1 objects/s recovering 2024-09-11T07:43:21.024 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:20 smithi022 ceph-mon[37330]: 2.18 scrub starts 2024-09-11T07:43:21.024 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:20 smithi022 ceph-mon[37330]: 2.18 scrub ok 2024-09-11T07:43:21.024 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:20 smithi022 ceph-mon[37330]: 5.19 scrub starts 2024-09-11T07:43:21.024 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:20 smithi022 ceph-mon[37330]: 5.19 scrub ok 2024-09-11T07:43:21.024 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:20 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:43:21.024 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:20 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:43:21.024 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:20 smithi022 ceph-mon[37330]: osdmap e365: 8 total, 8 up, 8 in 2024-09-11T07:43:21.024 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:20 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:43:21.024 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:20 smithi022 ceph-mon[37330]: pgmap v782: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 853 B/s rd, 63 MiB/s wr, 1.01k op/s; 73 B/s, 1 objects/s recovering 2024-09-11T07:43:21.039 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:21.069 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:21.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:21.070 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:21.170 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:21.192 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:21.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:21.193 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:21.293 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:21.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:21.323 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:21.323 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:21.424 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:21.454 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:21.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:21.455 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:21.555 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:21.591 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:21.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:21.592 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:21.606 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:43:21.607 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:43:21.607 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:43:21.607 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:43:21.607 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:43:21.607 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:43:21.607 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:43:21.607 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:43:21.607 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:43:21.607 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:43:21.607 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:43:21.607 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:43:21.607 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:43:21.608 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:43:21.608 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:43:21.608 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:43:21.692 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:21.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:21.738 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:21.738 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:21.838 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:21.871 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:21.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:21.872 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:21.916 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:21 smithi022 ceph-mon[37330]: 4.7 scrub starts 2024-09-11T07:43:21.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:21 smithi022 ceph-mon[37330]: 4.7 scrub ok 2024-09-11T07:43:21.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:21 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3458744453' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:43:21.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:21 smithi022 ceph-mon[29462]: 4.7 scrub starts 2024-09-11T07:43:21.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:21 smithi022 ceph-mon[29462]: 4.7 scrub ok 2024-09-11T07:43:21.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:21 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3458744453' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:43:21.961 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 66 16 102527 102511 97.0586 55.9375 0.0117618 0.0102984 2024-09-11T07:43:21.972 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:22.015 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:22.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:22.016 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:22.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:21 smithi104 ceph-mon[33507]: 4.7 scrub starts 2024-09-11T07:43:22.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:21 smithi104 ceph-mon[33507]: 4.7 scrub ok 2024-09-11T07:43:22.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:21 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3458744453' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:43:22.117 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:22.138 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:22.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:22.139 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:22.240 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:22.261 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:22.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:22.262 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:22.363 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:22.398 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:22.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:22.399 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:22.500 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:22.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:22.534 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:22.534 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:22.542 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:43:22.542 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:43:22.543 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 365, 2024-09-11T07:43:22.543 INFO:teuthology.orchestra.run.smithi022.stdout: "fsid": "12a285d8-700e-11ef-bcea-c7b262605968", 2024-09-11T07:43:22.543 INFO:teuthology.orchestra.run.smithi022.stdout: "created": "2024-09-11T07:21:18.695332+0000", 2024-09-11T07:43:22.543 INFO:teuthology.orchestra.run.smithi022.stdout: "modified": "2024-09-11T07:43:19.736984+0000", 2024-09-11T07:43:22.543 INFO:teuthology.orchestra.run.smithi022.stdout: "last_up_change": "2024-09-11T07:32:00.070825+0000", 2024-09-11T07:43:22.543 INFO:teuthology.orchestra.run.smithi022.stdout: "last_in_change": "2024-09-11T07:25:58.551813+0000", 2024-09-11T07:43:22.543 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-11T07:43:22.543 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_num": 5799936, 2024-09-11T07:43:22.543 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_set": [ 2024-09-11T07:43:22.543 INFO:teuthology.orchestra.run.smithi022.stdout: "pglog_hardlimit", 2024-09-11T07:43:22.543 INFO:teuthology.orchestra.run.smithi022.stdout: "purged_snapdirs", 2024-09-11T07:43:22.543 INFO:teuthology.orchestra.run.smithi022.stdout: "recovery_deletes", 2024-09-11T07:43:22.543 INFO:teuthology.orchestra.run.smithi022.stdout: "sortbitwise" 2024-09-11T07:43:22.543 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:43:22.544 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_version": 19, 2024-09-11T07:43:22.544 INFO:teuthology.orchestra.run.smithi022.stdout: "full_ratio": 0.94999998807907104, 2024-09-11T07:43:22.544 INFO:teuthology.orchestra.run.smithi022.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-11T07:43:22.544 INFO:teuthology.orchestra.run.smithi022.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-11T07:43:22.544 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_snapshot": "", 2024-09-11T07:43:22.544 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_max": 10, 2024-09-11T07:43:22.544 INFO:teuthology.orchestra.run.smithi022.stdout: "max_osd": 8, 2024-09-11T07:43:22.544 INFO:teuthology.orchestra.run.smithi022.stdout: "require_min_compat_client": "luminous", 2024-09-11T07:43:22.544 INFO:teuthology.orchestra.run.smithi022.stdout: "min_compat_client": "luminous", 2024-09-11T07:43:22.544 INFO:teuthology.orchestra.run.smithi022.stdout: "require_osd_release": "quincy", 2024-09-11T07:43:22.544 INFO:teuthology.orchestra.run.smithi022.stdout: "pools": [ 2024-09-11T07:43:22.544 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.544 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 1, 2024-09-11T07:43:22.545 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": ".mgr", 2024-09-11T07:43:22.545 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:24:21.772426+0000", 2024-09-11T07:43:22.545 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:43:22.545 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:43:22.545 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:43:22.545 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:43:22.545 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:43:22.545 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:43:22.545 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:43:22.545 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:43:22.545 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:43:22.545 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:43:22.545 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:43:22.545 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:43:22.545 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 1, 2024-09-11T07:43:22.546 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 1, 2024-09-11T07:43:22.546 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 1, 2024-09-11T07:43:22.546 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 1, 2024-09-11T07:43:22.546 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 1, 2024-09-11T07:43:22.546 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:43:22.546 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:43:22.546 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:43:22.546 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:43:22.546 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:43:22.546 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:43:22.546 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:43:22.546 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.546 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "21", 2024-09-11T07:43:22.546 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:43:22.547 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:43:22.547 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:43:22.547 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:43:22.547 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:43:22.547 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:43:22.547 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:43:22.547 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:43:22.547 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:43:22.547 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:43:22.547 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:43:22.547 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:43:22.547 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:43:22.547 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:43:22.547 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:43:22.547 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:43:22.548 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:43:22.548 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:43:22.548 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:43:22.548 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:43:22.548 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:43:22.548 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:43:22.548 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:43:22.548 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:43:22.548 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:43:22.548 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:43:22.548 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.548 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:43:22.548 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:43:22.548 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:43:22.549 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:43:22.549 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:43:22.549 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:43:22.549 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:43:22.549 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:43:22.549 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:43:22.549 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:43:22.549 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:43:22.549 INFO:teuthology.orchestra.run.smithi022.stdout: "options": { 2024-09-11T07:43:22.549 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_max": 32, 2024-09-11T07:43:22.549 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_min": 1 2024-09-11T07:43:22.549 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.549 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:43:22.549 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": {} 2024-09-11T07:43:22.550 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.550 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.550 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.550 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 2, 2024-09-11T07:43:22.550 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": ".rgw.root", 2024-09-11T07:43:22.550 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:29.941354+0000", 2024-09-11T07:43:22.550 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:43:22.550 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:43:22.550 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:43:22.550 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:43:22.550 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:43:22.550 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:43:22.550 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:43:22.550 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:43:22.551 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:43:22.551 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:43:22.551 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:43:22.551 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:43:22.551 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:43:22.551 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:43:22.551 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:43:22.551 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:43:22.551 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:43:22.551 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:43:22.551 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:43:22.551 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:43:22.551 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:43:22.552 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:43:22.552 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:43:22.552 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:43:22.552 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.552 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "58", 2024-09-11T07:43:22.552 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:43:22.552 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:43:22.552 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:43:22.552 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:43:22.552 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:43:22.552 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:43:22.552 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:43:22.552 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:43:22.552 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:43:22.553 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:43:22.553 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:43:22.553 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:43:22.553 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:43:22.553 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:43:22.553 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:43:22.553 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:43:22.553 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:43:22.553 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:43:22.553 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:43:22.553 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:43:22.553 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:43:22.553 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:43:22.553 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:43:22.554 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:43:22.554 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:43:22.554 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:43:22.554 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.554 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:43:22.554 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:43:22.554 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:43:22.554 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:43:22.554 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:43:22.554 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:43:22.554 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:43:22.554 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:43:22.554 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:43:22.554 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:43:22.554 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:43:22.555 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:43:22.555 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:43:22.555 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:43:22.555 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.555 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.555 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.555 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 3, 2024-09-11T07:43:22.555 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "datapool", 2024-09-11T07:43:22.555 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:30.396859+0000", 2024-09-11T07:43:22.555 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 8193, 2024-09-11T07:43:22.555 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T07:43:22.555 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:43:22.555 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:43:22.555 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:43:22.556 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:43:22.556 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:43:22.556 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:43:22.556 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:43:22.556 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:43:22.556 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:43:22.556 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:43:22.556 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 3, 2024-09-11T07:43:22.556 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 3, 2024-09-11T07:43:22.556 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 3, 2024-09-11T07:43:22.556 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 3, 2024-09-11T07:43:22.556 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 3, 2024-09-11T07:43:22.556 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:43:22.557 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:43:22.557 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:43:22.557 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:43:22.557 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:43:22.557 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:43:22.557 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:43:22.557 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.557 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "61", 2024-09-11T07:43:22.557 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:43:22.557 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:43:22.557 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:43:22.557 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:43:22.557 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:43:22.557 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 3, 2024-09-11T07:43:22.558 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 61, 2024-09-11T07:43:22.558 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:43:22.558 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:43:22.558 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:43:22.558 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:43:22.558 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:43:22.558 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:43:22.558 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:43:22.558 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:43:22.558 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:43:22.558 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:43:22.558 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:43:22.558 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:43:22.558 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:43:22.558 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:43:22.559 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:43:22.559 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:43:22.559 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:43:22.559 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:43:22.559 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:43:22.559 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.559 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:43:22.559 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:43:22.559 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:43:22.559 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:43:22.559 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:43:22.559 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:43:22.559 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:43:22.559 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:43:22.560 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:43:22.560 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:43:22.560 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:43:22.560 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:43:22.560 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:43:22.560 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd": {} 2024-09-11T07:43:22.560 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.560 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.560 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.560 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 4, 2024-09-11T07:43:22.560 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.log", 2024-09-11T07:43:22.560 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:31.922926+0000", 2024-09-11T07:43:22.560 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:43:22.561 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:43:22.561 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:43:22.561 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:43:22.561 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:43:22.561 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:43:22.561 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:43:22.561 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:43:22.561 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:43:22.561 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:43:22.561 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:43:22.561 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:43:22.561 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:43:22.561 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:43:22.561 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:43:22.562 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:43:22.562 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:43:22.562 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:43:22.562 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:43:22.562 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:43:22.562 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:43:22.562 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:43:22.562 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:43:22.562 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:43:22.562 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.562 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "60", 2024-09-11T07:43:22.562 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:43:22.562 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:43:22.563 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:43:22.563 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:43:22.563 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:43:22.563 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:43:22.563 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:43:22.563 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:43:22.563 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:43:22.563 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:43:22.563 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:43:22.563 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:43:22.563 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:43:22.563 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:43:22.564 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:43:22.564 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:43:22.564 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:43:22.564 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:43:22.564 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:43:22.564 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:43:22.564 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:43:22.564 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:43:22.564 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:43:22.564 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:43:22.564 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:43:22.564 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:43:22.564 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.565 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:43:22.565 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:43:22.565 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:43:22.565 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:43:22.565 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:43:22.565 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:43:22.565 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:43:22.565 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:43:22.565 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:43:22.565 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:43:22.565 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:43:22.565 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:43:22.565 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:43:22.565 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:43:22.565 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.566 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.566 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.566 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 5, 2024-09-11T07:43:22.566 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.control", 2024-09-11T07:43:22.566 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:34.007746+0000", 2024-09-11T07:43:22.566 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:43:22.566 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:43:22.566 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:43:22.566 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:43:22.566 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:43:22.566 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:43:22.566 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:43:22.566 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:43:22.567 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:43:22.567 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:43:22.567 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:43:22.567 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:43:22.567 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:43:22.567 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:43:22.567 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:43:22.567 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:43:22.567 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:43:22.567 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:43:22.567 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:43:22.567 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:43:22.567 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:43:22.567 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:43:22.567 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:43:22.568 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:43:22.568 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.568 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "62", 2024-09-11T07:43:22.568 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:43:22.568 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:43:22.568 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:43:22.568 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:43:22.568 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:43:22.568 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:43:22.568 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:43:22.568 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:43:22.568 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:43:22.568 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:43:22.568 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:43:22.569 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:43:22.569 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:43:22.569 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:43:22.569 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:43:22.569 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:43:22.569 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:43:22.569 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:43:22.569 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:43:22.569 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:43:22.569 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:43:22.569 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:43:22.569 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:43:22.569 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:43:22.569 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:43:22.570 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:43:22.570 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.570 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:43:22.570 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:43:22.570 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:43:22.570 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:43:22.570 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:43:22.570 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:43:22.570 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:43:22.570 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:43:22.570 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:43:22.570 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:43:22.570 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:43:22.570 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:43:22.571 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:43:22.571 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:43:22.571 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.571 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.571 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.571 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 6, 2024-09-11T07:43:22.571 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.meta", 2024-09-11T07:43:22.571 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:36.077215+0000", 2024-09-11T07:43:22.571 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:43:22.571 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:43:22.571 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:43:22.571 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:43:22.571 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:43:22.572 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:43:22.572 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:43:22.572 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:43:22.572 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:43:22.572 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:43:22.572 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:43:22.572 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:43:22.572 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:43:22.572 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:43:22.572 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:43:22.572 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:43:22.572 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:43:22.573 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:43:22.573 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:43:22.573 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:43:22.573 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:43:22.573 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:43:22.573 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:43:22.573 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:43:22.573 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.573 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "64", 2024-09-11T07:43:22.573 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:43:22.573 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:43:22.573 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:43:22.573 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:43:22.573 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:43:22.574 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:43:22.574 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:43:22.574 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:43:22.574 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:43:22.574 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:43:22.574 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:43:22.574 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:43:22.574 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:43:22.574 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:43:22.574 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:43:22.574 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:43:22.574 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:43:22.575 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:43:22.575 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:43:22.575 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:43:22.575 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:43:22.575 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:43:22.575 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:43:22.575 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:43:22.575 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:43:22.575 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:43:22.575 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.575 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:43:22.575 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:43:22.575 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:43:22.575 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:43:22.576 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:43:22.576 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:43:22.576 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:43:22.576 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:43:22.576 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:43:22.576 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:43:22.576 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:43:22.576 INFO:teuthology.orchestra.run.smithi022.stdout: "options": { 2024-09-11T07:43:22.576 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_bias": 4 2024-09-11T07:43:22.576 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.576 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:43:22.576 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:43:22.577 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.577 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.577 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.577 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 10, 2024-09-11T07:43:22.577 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "unique_pool_3", 2024-09-11T07:43:22.577 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:41:56.278857+0000", 2024-09-11T07:43:22.577 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:43:22.577 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:43:22.577 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:43:22.577 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:43:22.577 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:43:22.577 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:43:22.578 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:43:22.578 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:43:22.578 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:43:22.578 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:43:22.578 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:43:22.578 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:43:22.578 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 16, 2024-09-11T07:43:22.578 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 16, 2024-09-11T07:43:22.578 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 16, 2024-09-11T07:43:22.578 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 16, 2024-09-11T07:43:22.578 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 16, 2024-09-11T07:43:22.578 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:43:22.578 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:43:22.579 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:43:22.579 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:43:22.579 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:43:22.579 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:43:22.579 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:43:22.579 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.579 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "349", 2024-09-11T07:43:22.579 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:43:22.579 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:43:22.579 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:43:22.579 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:43:22.579 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:43:22.579 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:43:22.579 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:43:22.580 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:43:22.580 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:43:22.580 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:43:22.580 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:43:22.580 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:43:22.580 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:43:22.580 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:43:22.580 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:43:22.580 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:43:22.580 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:43:22.580 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:43:22.580 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:43:22.580 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:43:22.581 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:43:22.581 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:43:22.581 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:43:22.581 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:43:22.581 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:43:22.581 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:43:22.581 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.581 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:43:22.581 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:43:22.581 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:43:22.581 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:43:22.581 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:43:22.581 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:43:22.582 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:43:22.582 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:43:22.582 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:43:22.582 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:43:22.582 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:43:22.582 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:43:22.582 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:43:22.582 INFO:teuthology.orchestra.run.smithi022.stdout: "rados": {} 2024-09-11T07:43:22.582 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.582 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.582 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:43:22.582 INFO:teuthology.orchestra.run.smithi022.stdout: "osds": [ 2024-09-11T07:43:22.582 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.582 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 0, 2024-09-11T07:43:22.583 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "09075f16-32f4-4f88-a116-c3d7f977a0e0", 2024-09-11T07:43:22.583 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:43:22.583 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:43:22.583 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:43:22.583 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:43:22.583 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:43:22.583 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:43:22.583 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 8, 2024-09-11T07:43:22.583 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 358, 2024-09-11T07:43:22.583 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:43:22.583 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:43:22.583 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:43:22.583 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.583 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.584 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.584 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6801", 2024-09-11T07:43:22.584 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:43:22.584 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.584 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.584 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.584 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:43:22.584 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.584 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.584 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.584 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6802", 2024-09-11T07:43:22.584 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:43:22.584 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.585 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.585 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.585 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:43:22.585 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.585 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.585 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.585 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6804", 2024-09-11T07:43:22.585 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:43:22.585 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.585 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.585 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.585 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:43:22.585 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.586 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.586 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.586 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6803", 2024-09-11T07:43:22.586 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:43:22.586 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.586 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.586 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.586 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6801/2821493591", 2024-09-11T07:43:22.586 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6802/2821493591", 2024-09-11T07:43:22.586 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6804/2821493591", 2024-09-11T07:43:22.586 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6803/2821493591", 2024-09-11T07:43:22.586 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:43:22.586 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:43:22.587 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:43:22.587 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.587 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.587 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.587 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 1, 2024-09-11T07:43:22.587 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "52cc1e06-9506-48c0-9f7b-d4270dff4b9f", 2024-09-11T07:43:22.587 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:43:22.587 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:43:22.587 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 0.64752197265625, 2024-09-11T07:43:22.587 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:43:22.587 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:43:22.587 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:43:22.587 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 13, 2024-09-11T07:43:22.587 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 346, 2024-09-11T07:43:22.588 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:43:22.588 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:43:22.588 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:43:22.588 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.588 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.588 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.588 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6805", 2024-09-11T07:43:22.588 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:43:22.588 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.588 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.588 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.588 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:43:22.589 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.589 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.589 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.589 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6806", 2024-09-11T07:43:22.589 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:43:22.589 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.589 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.589 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.589 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:43:22.589 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.589 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.589 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.589 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6808", 2024-09-11T07:43:22.590 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:43:22.590 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.590 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.590 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.590 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:43:22.590 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.590 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.590 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.590 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6807", 2024-09-11T07:43:22.590 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:43:22.590 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.590 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.590 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.591 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6805/4037911317", 2024-09-11T07:43:22.591 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6806/4037911317", 2024-09-11T07:43:22.591 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6808/4037911317", 2024-09-11T07:43:22.591 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6807/4037911317", 2024-09-11T07:43:22.591 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:43:22.591 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:43:22.591 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:43:22.591 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.591 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.591 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.591 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 2, 2024-09-11T07:43:22.591 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "fa2b2730-2c1d-45d0-be8a-494a60dc6b9b", 2024-09-11T07:43:22.591 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:43:22.591 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:43:22.592 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:43:22.592 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:43:22.592 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:43:22.592 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:43:22.592 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 18, 2024-09-11T07:43:22.592 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 360, 2024-09-11T07:43:22.592 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:43:22.592 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:43:22.592 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:43:22.592 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.592 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.592 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.592 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6809", 2024-09-11T07:43:22.592 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:43:22.593 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.593 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.593 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.593 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:43:22.593 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.593 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.593 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.593 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6810", 2024-09-11T07:43:22.593 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:43:22.593 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.593 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.593 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.593 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:43:22.594 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.594 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.594 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.594 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6812", 2024-09-11T07:43:22.594 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:43:22.594 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.594 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.594 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.594 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:43:22.594 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.594 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.594 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.594 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6811", 2024-09-11T07:43:22.595 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:43:22.595 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.595 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.595 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.595 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6809/1675799666", 2024-09-11T07:43:22.595 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6810/1675799666", 2024-09-11T07:43:22.595 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6812/1675799666", 2024-09-11T07:43:22.595 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6811/1675799666", 2024-09-11T07:43:22.595 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:43:22.595 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:43:22.595 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:43:22.595 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.595 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.595 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.595 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 3, 2024-09-11T07:43:22.596 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "8d53d3fc-e401-47df-b70b-4b4b1f4539df", 2024-09-11T07:43:22.596 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:43:22.596 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:43:22.596 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 0.686370849609375, 2024-09-11T07:43:22.596 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:43:22.596 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:43:22.596 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:43:22.596 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 25, 2024-09-11T07:43:22.596 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 341, 2024-09-11T07:43:22.596 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:43:22.596 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:43:22.596 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:43:22.596 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.597 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.597 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.597 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6813", 2024-09-11T07:43:22.597 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:43:22.597 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.597 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.597 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.597 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:43:22.597 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.597 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.597 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.597 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6814", 2024-09-11T07:43:22.597 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:43:22.597 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.597 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.598 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.598 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:43:22.598 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.598 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.598 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.598 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6816", 2024-09-11T07:43:22.598 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:43:22.598 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.598 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.598 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.598 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:43:22.598 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.598 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.598 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.599 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6815", 2024-09-11T07:43:22.599 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:43:22.599 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.599 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.599 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.599 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6813/1501962752", 2024-09-11T07:43:22.599 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6814/1501962752", 2024-09-11T07:43:22.599 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6816/1501962752", 2024-09-11T07:43:22.599 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6815/1501962752", 2024-09-11T07:43:22.599 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:43:22.599 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:43:22.599 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:43:22.599 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.600 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.600 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.600 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 4, 2024-09-11T07:43:22.600 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "d3207fe0-8b97-4067-a755-ba6d449c53c7", 2024-09-11T07:43:22.600 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:43:22.600 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:43:22.600 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:43:22.600 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:43:22.600 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:43:22.600 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:43:22.600 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 31, 2024-09-11T07:43:22.600 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 360, 2024-09-11T07:43:22.600 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:43:22.601 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:43:22.601 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:43:22.601 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.601 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.601 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.601 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6800", 2024-09-11T07:43:22.601 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:43:22.601 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.601 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.601 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.601 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:43:22.601 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.601 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.602 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.602 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6801", 2024-09-11T07:43:22.602 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:43:22.602 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.602 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.602 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.602 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:43:22.602 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.602 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.602 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.602 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6803", 2024-09-11T07:43:22.602 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:43:22.602 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.603 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.603 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.603 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:43:22.603 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.603 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.603 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.603 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6802", 2024-09-11T07:43:22.603 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:43:22.603 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.603 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.603 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.603 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6800/1210229443", 2024-09-11T07:43:22.603 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6801/1210229443", 2024-09-11T07:43:22.604 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6803/1210229443", 2024-09-11T07:43:22.604 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6802/1210229443", 2024-09-11T07:43:22.604 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:43:22.604 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:43:22.604 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:43:22.604 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.604 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.604 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.604 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 5, 2024-09-11T07:43:22.604 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "89c9352b-30b2-48c6-869e-047d07a23a43", 2024-09-11T07:43:22.604 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:43:22.604 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:43:22.604 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 0.6509552001953125, 2024-09-11T07:43:22.604 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 0.666046142578125, 2024-09-11T07:43:22.605 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:43:22.605 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:43:22.605 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 37, 2024-09-11T07:43:22.605 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 360, 2024-09-11T07:43:22.605 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:43:22.605 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:43:22.605 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:43:22.605 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.605 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.605 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.605 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6804", 2024-09-11T07:43:22.605 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:43:22.605 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.606 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.606 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.606 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:43:22.606 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.606 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.606 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.606 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6805", 2024-09-11T07:43:22.606 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:43:22.606 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.606 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.606 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.606 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:43:22.606 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.607 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.607 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.607 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6807", 2024-09-11T07:43:22.607 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:43:22.607 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.607 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.607 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.607 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:43:22.607 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.607 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.607 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.607 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6806", 2024-09-11T07:43:22.607 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:43:22.608 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.608 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.608 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.608 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6804/2735328732", 2024-09-11T07:43:22.608 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6805/2735328732", 2024-09-11T07:43:22.608 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6807/2735328732", 2024-09-11T07:43:22.608 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6806/2735328732", 2024-09-11T07:43:22.608 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:43:22.608 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:43:22.608 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:43:22.608 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.608 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.608 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.609 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 6, 2024-09-11T07:43:22.609 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "704563de-d88f-480a-b469-720135704d42", 2024-09-11T07:43:22.609 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:43:22.609 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:43:22.609 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:43:22.609 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 0, 2024-09-11T07:43:22.609 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:43:22.609 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:43:22.609 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 44, 2024-09-11T07:43:22.609 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 177, 2024-09-11T07:43:22.609 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:43:22.609 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:43:22.609 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:43:22.610 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.610 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.610 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.610 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6808", 2024-09-11T07:43:22.610 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:43:22.610 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.610 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.610 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.610 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:43:22.610 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.610 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.610 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.610 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6809", 2024-09-11T07:43:22.610 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:43:22.611 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.611 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.611 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.611 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:43:22.611 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.611 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.611 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.611 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6811", 2024-09-11T07:43:22.611 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:43:22.611 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.611 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.611 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.611 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:43:22.612 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.612 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.612 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.612 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6810", 2024-09-11T07:43:22.612 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:43:22.612 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.612 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.612 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.612 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6808/721493010", 2024-09-11T07:43:22.612 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6809/721493010", 2024-09-11T07:43:22.612 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6811/721493010", 2024-09-11T07:43:22.612 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6810/721493010", 2024-09-11T07:43:22.613 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:43:22.613 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:43:22.613 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:43:22.613 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.613 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.613 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.613 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 7, 2024-09-11T07:43:22.613 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "18199e47-0052-49f0-b45f-0fb201e0d0f4", 2024-09-11T07:43:22.613 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:43:22.613 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:43:22.613 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 0.6657562255859375, 2024-09-11T07:43:22.613 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:43:22.613 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 52, 2024-09-11T07:43:22.614 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 133, 2024-09-11T07:43:22.614 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 147, 2024-09-11T07:43:22.614 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 360, 2024-09-11T07:43:22.614 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 134, 2024-09-11T07:43:22.614 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:43:22.614 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:43:22.614 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.614 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.614 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.614 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6812", 2024-09-11T07:43:22.614 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2218863853 2024-09-11T07:43:22.614 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.614 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.614 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.615 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:43:22.615 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.615 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.615 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.615 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6813", 2024-09-11T07:43:22.615 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2218863853 2024-09-11T07:43:22.615 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.615 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.615 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.615 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:43:22.615 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.615 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.615 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.616 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6815", 2024-09-11T07:43:22.616 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2218863853 2024-09-11T07:43:22.616 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.616 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.616 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.616 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:43:22.616 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:22.616 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.616 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:22.616 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6814", 2024-09-11T07:43:22.616 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2218863853 2024-09-11T07:43:22.616 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.616 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.617 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.617 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6812/2218863853", 2024-09-11T07:43:22.617 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6813/2218863853", 2024-09-11T07:43:22.617 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6815/2218863853", 2024-09-11T07:43:22.617 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6814/2218863853", 2024-09-11T07:43:22.617 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:43:22.617 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:43:22.617 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:43:22.617 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.618 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.618 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:43:22.618 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_xinfo": [ 2024-09-11T07:43:22.618 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.618 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 0, 2024-09-11T07:43:22.618 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:43:22.618 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:43:22.618 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:43:22.618 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:43:22.618 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:43:22.618 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:42:05.640226+0000", 2024-09-11T07:43:22.618 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:43:22.618 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.619 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.619 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 1, 2024-09-11T07:43:22.619 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:43:22.619 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:43:22.619 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:43:22.619 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:43:22.619 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:43:22.619 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:43:17.772622+0000", 2024-09-11T07:43:22.619 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:43:22.619 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.619 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.619 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 2, 2024-09-11T07:43:22.619 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:43:22.620 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:43:22.620 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:43:22.620 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:43:22.620 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:43:22.620 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:42:00.226145+0000", 2024-09-11T07:43:22.620 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:43:22.620 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.620 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.620 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 3, 2024-09-11T07:43:22.620 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:43:22.620 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:43:22.620 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:43:22.620 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:43:22.621 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:43:22.621 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:42:18.279268+0000", 2024-09-11T07:43:22.621 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:43:22.621 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.621 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.621 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 4, 2024-09-11T07:43:22.621 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:43:22.621 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:43:22.621 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:43:22.621 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:43:22.621 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:43:22.621 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:43:10.409639+0000", 2024-09-11T07:43:22.622 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:43:22.622 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.622 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.622 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 5, 2024-09-11T07:43:22.622 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:43:22.622 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:43:22.622 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:43:22.622 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:43:22.622 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:43:22.622 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:43:14.427937+0000", 2024-09-11T07:43:22.622 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:43:22.622 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.622 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.622 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 6, 2024-09-11T07:43:22.623 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:43:22.623 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:43:22.623 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:43:22.623 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:43:22.623 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:43:22.623 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:43:19.684540+0000", 2024-09-11T07:43:22.623 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:43:22.623 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.623 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.623 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 7, 2024-09-11T07:43:22.623 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:43:22.623 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:43:22.623 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:43:22.623 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:43:22.624 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:43:22.624 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:42:20.744248+0000", 2024-09-11T07:43:22.624 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 133 2024-09-11T07:43:22.624 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.624 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:43:22.624 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_upmap": [ 2024-09-11T07:43:22.624 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.624 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "2.1c", 2024-09-11T07:43:22.624 INFO:teuthology.orchestra.run.smithi022.stdout: "osds": [ 2024-09-11T07:43:22.624 INFO:teuthology.orchestra.run.smithi022.stdout: 5, 2024-09-11T07:43:22.624 INFO:teuthology.orchestra.run.smithi022.stdout: 2, 2024-09-11T07:43:22.624 INFO:teuthology.orchestra.run.smithi022.stdout: 7 2024-09-11T07:43:22.624 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.625 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.625 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.625 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.1e", 2024-09-11T07:43:22.625 INFO:teuthology.orchestra.run.smithi022.stdout: "osds": [ 2024-09-11T07:43:22.625 INFO:teuthology.orchestra.run.smithi022.stdout: 4, 2024-09-11T07:43:22.625 INFO:teuthology.orchestra.run.smithi022.stdout: 5, 2024-09-11T07:43:22.625 INFO:teuthology.orchestra.run.smithi022.stdout: 3 2024-09-11T07:43:22.625 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.625 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.625 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:43:22.625 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_upmap_items": [ 2024-09-11T07:43:22.625 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.625 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.7", 2024-09-11T07:43:22.625 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:43:22.626 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.626 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 3, 2024-09-11T07:43:22.626 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 7 2024-09-11T07:43:22.626 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.626 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.626 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.626 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.626 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.15", 2024-09-11T07:43:22.626 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:43:22.626 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.626 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 3, 2024-09-11T07:43:22.626 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 0 2024-09-11T07:43:22.626 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.626 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.626 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.627 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.627 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.1b", 2024-09-11T07:43:22.627 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:43:22.627 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.627 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 3, 2024-09-11T07:43:22.627 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 7 2024-09-11T07:43:22.627 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.627 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.627 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.627 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.627 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "5.c", 2024-09-11T07:43:22.627 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:43:22.627 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:22.627 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 1, 2024-09-11T07:43:22.628 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 2 2024-09-11T07:43:22.628 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.628 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:22.628 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.628 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:43:22.628 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_temp": [], 2024-09-11T07:43:22.628 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_temp": [], 2024-09-11T07:43:22.628 INFO:teuthology.orchestra.run.smithi022.stdout: "blocklist": { 2024-09-11T07:43:22.628 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/240999923": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:43:22.628 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/4016369460": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:43:22.628 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/585929238": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:43:22.628 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/544031024": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:43:22.628 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/2504196132": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:43:22.629 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/1451388349": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:43:22.629 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/639336081": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:43:22.629 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2104069730": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:43:22.629 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/4236450731": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:43:22.629 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/3573193776": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:43:22.629 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2661764422": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:43:22.629 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2197488530": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:43:22.629 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2968034970": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:43:22.629 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/599318175": "2024-09-12T07:27:07.919998+0000" 2024-09-11T07:43:22.629 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.629 INFO:teuthology.orchestra.run.smithi022.stdout: "range_blocklist": {}, 2024-09-11T07:43:22.629 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profiles": { 2024-09-11T07:43:22.629 INFO:teuthology.orchestra.run.smithi022.stdout: "default": { 2024-09-11T07:43:22.629 INFO:teuthology.orchestra.run.smithi022.stdout: "crush-failure-domain": "osd", 2024-09-11T07:43:22.630 INFO:teuthology.orchestra.run.smithi022.stdout: "k": "2", 2024-09-11T07:43:22.630 INFO:teuthology.orchestra.run.smithi022.stdout: "m": "1", 2024-09-11T07:43:22.630 INFO:teuthology.orchestra.run.smithi022.stdout: "plugin": "jerasure", 2024-09-11T07:43:22.630 INFO:teuthology.orchestra.run.smithi022.stdout: "technique": "reed_sol_van" 2024-09-11T07:43:22.630 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.630 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:22.630 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps_queue": [], 2024-09-11T07:43:22.630 INFO:teuthology.orchestra.run.smithi022.stdout: "new_removed_snaps": [], 2024-09-11T07:43:22.630 INFO:teuthology.orchestra.run.smithi022.stdout: "new_purged_snaps": [], 2024-09-11T07:43:22.630 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_node_flags": {}, 2024-09-11T07:43:22.630 INFO:teuthology.orchestra.run.smithi022.stdout: "device_class_flags": {}, 2024-09-11T07:43:22.630 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode": { 2024-09-11T07:43:22.630 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode_enabled": false, 2024-09-11T07:43:22.630 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_bucket_count": 0, 2024-09-11T07:43:22.631 INFO:teuthology.orchestra.run.smithi022.stdout: "degraded_stretch_mode": 0, 2024-09-11T07:43:22.631 INFO:teuthology.orchestra.run.smithi022.stdout: "recovering_stretch_mode": 0, 2024-09-11T07:43:22.631 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode_bucket": 0 2024-09-11T07:43:22.631 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:22.631 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:43:22.634 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:22.682 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:22.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:22.683 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:22.784 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:22.815 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:22.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:22.816 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:22.916 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:22.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[29462]: 6.d scrub starts 2024-09-11T07:43:22.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[29462]: 5.18 scrub starts 2024-09-11T07:43:22.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[29462]: 5.18 scrub ok 2024-09-11T07:43:22.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[29462]: 6.d scrub ok 2024-09-11T07:43:22.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[29462]: 5.d scrub starts 2024-09-11T07:43:22.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[29462]: 5.d scrub ok 2024-09-11T07:43:22.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[29462]: 10.f scrub ok 2024-09-11T07:43:22.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[29462]: 5.9 scrub starts 2024-09-11T07:43:22.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[29462]: 5.9 scrub ok 2024-09-11T07:43:22.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:43:22.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3209373341' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:43:22.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[29462]: pgmap v783: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 656 B/s rd, 46 MiB/s wr, 736 op/s 2024-09-11T07:43:22.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:43:22.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:43:22.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[37330]: 6.d scrub starts 2024-09-11T07:43:22.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[37330]: 5.18 scrub starts 2024-09-11T07:43:22.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[37330]: 5.18 scrub ok 2024-09-11T07:43:22.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[37330]: 6.d scrub ok 2024-09-11T07:43:22.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[37330]: 5.d scrub starts 2024-09-11T07:43:22.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[37330]: 5.d scrub ok 2024-09-11T07:43:22.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[37330]: 10.f scrub ok 2024-09-11T07:43:22.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[37330]: 5.9 scrub starts 2024-09-11T07:43:22.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[37330]: 5.9 scrub ok 2024-09-11T07:43:22.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:43:22.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3209373341' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:43:22.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[37330]: pgmap v783: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 656 B/s rd, 46 MiB/s wr, 736 op/s 2024-09-11T07:43:22.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:43:22.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:22 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:43:22.936 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:22 smithi104 ceph-mon[33507]: 6.d scrub starts 2024-09-11T07:43:22.936 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:22 smithi104 ceph-mon[33507]: 5.18 scrub starts 2024-09-11T07:43:22.936 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:22 smithi104 ceph-mon[33507]: 5.18 scrub ok 2024-09-11T07:43:22.936 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:22 smithi104 ceph-mon[33507]: 6.d scrub ok 2024-09-11T07:43:22.936 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:22 smithi104 ceph-mon[33507]: 5.d scrub starts 2024-09-11T07:43:22.937 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:22 smithi104 ceph-mon[33507]: 5.d scrub ok 2024-09-11T07:43:22.937 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:22 smithi104 ceph-mon[33507]: 10.f scrub ok 2024-09-11T07:43:22.937 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:22 smithi104 ceph-mon[33507]: 5.9 scrub starts 2024-09-11T07:43:22.937 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:22 smithi104 ceph-mon[33507]: 5.9 scrub ok 2024-09-11T07:43:22.937 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:22 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:43:22.937 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:22 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3209373341' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:43:22.937 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:22 smithi104 ceph-mon[33507]: pgmap v783: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 656 B/s rd, 46 MiB/s wr, 736 op/s 2024-09-11T07:43:22.937 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:22 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:43:22.937 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:22 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:43:22.962 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 67 16 103337 103321 96.3643 50.625 0.0239698 0.0103726 2024-09-11T07:43:22.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:22.968 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:22.968 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:22.971 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:43:22.971 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:43:23.070 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:23.097 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:23.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:23.098 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:23.199 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:23.233 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:23.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:23.234 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:23.335 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:23.366 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:23.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:23.367 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:23.468 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:23.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:23.515 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:23.515 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:23.616 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:23.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:23.665 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:23.665 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:23.671 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-11T07:43:23.765 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:23.810 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:23.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:23.811 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:23.912 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:23.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:23.962 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:23.962 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:23.963 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 68 16 104483 104467 96.0004 71.625 0.0103535 0.0104117 2024-09-11T07:43:24.063 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:24.117 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:43:24.121 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:24.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:24.122 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:24.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[29462]: 4.8 scrub starts 2024-09-11T07:43:24.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[29462]: 4.8 scrub ok 2024-09-11T07:43:24.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[29462]: 2.3 scrub starts 2024-09-11T07:43:24.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[29462]: 2.3 scrub ok 2024-09-11T07:43:24.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[29462]: 4.4 scrub starts 2024-09-11T07:43:24.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[29462]: 4.4 scrub ok 2024-09-11T07:43:24.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:43:24.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:43:24.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:43:24.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:43:24.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:43:24.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:43:24.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:43:24.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:43:24.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:43:24.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[37330]: 4.8 scrub starts 2024-09-11T07:43:24.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[37330]: 4.8 scrub ok 2024-09-11T07:43:24.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[37330]: 2.3 scrub starts 2024-09-11T07:43:24.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[37330]: 2.3 scrub ok 2024-09-11T07:43:24.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[37330]: 4.4 scrub starts 2024-09-11T07:43:24.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[37330]: 4.4 scrub ok 2024-09-11T07:43:24.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:43:24.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:43:24.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:43:24.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:43:24.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:43:24.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:43:24.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:43:24.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:43:24.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:24 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:43:24.223 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:24.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:24.261 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:24.261 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:24.309 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:43:24 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:43:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:43:24.309 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:24 smithi104 ceph-mon[33507]: 4.8 scrub starts 2024-09-11T07:43:24.309 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:24 smithi104 ceph-mon[33507]: 4.8 scrub ok 2024-09-11T07:43:24.310 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:24 smithi104 ceph-mon[33507]: 2.3 scrub starts 2024-09-11T07:43:24.310 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:24 smithi104 ceph-mon[33507]: 2.3 scrub ok 2024-09-11T07:43:24.310 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:24 smithi104 ceph-mon[33507]: 4.4 scrub starts 2024-09-11T07:43:24.310 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:24 smithi104 ceph-mon[33507]: 4.4 scrub ok 2024-09-11T07:43:24.310 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:24 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:43:24.310 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:24 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:43:24.310 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:24 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:43:24.310 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:24 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:43:24.310 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:24 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:43:24.310 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:24 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:43:24.310 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:24 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:43:24.310 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:24 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:43:24.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:24 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:43:24.362 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:24.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:24.400 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:24.400 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:24.501 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:24.543 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:24.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:24.544 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:24.645 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:24.686 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:24.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:24.687 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:24.730 INFO:tasks.thrashosds.thrasher:j is {'epoch': 365, 'fsid': '12a285d8-700e-11ef-bcea-c7b262605968', 'created': '2024-09-11T07:21:18.695332+0000', 'modified': '2024-09-11T07:43:19.736984+0000', 'last_up_change': '2024-09-11T07:32:00.070825+0000', 'last_in_change': '2024-09-11T07:25:58.551813+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 19, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 10, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T07:24:21.772426+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': '.rgw.root', 'create_time': '2024-09-11T07:26:29.941354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 3, 'pool_name': 'datapool', 'create_time': '2024-09-11T07:26:30.396859+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-11T07:26:31.922926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-11T07:26:34.007746+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-11T07:26:36.077215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}, {'pool': 10, 'pool_name': 'unique_pool_3', 'create_time': '2024-09-11T07:41:56.278857+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '349', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '09075f16-32f4-4f88-a116-c3d7f977a0e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 358, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6801', 'nonce': 2821493591}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6802', 'nonce': 2821493591}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6804', 'nonce': 2821493591}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6803', 'nonce': 2821493591}]}, 'public_addr': '172.21.15.22:6801/2821493591', 'cluster_addr': '172.21.15.22:6802/2821493591', 'heartbeat_back_addr': '172.21.15.22:6804/2821493591', 'heartbeat_front_addr': '172.21.15.22:6803/2821493591', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '52cc1e06-9506-48c0-9f7b-d4270dff4b9f', 'up': 1, 'in': 1, 'weight': 0.64752197265625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 346, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6805', 'nonce': 4037911317}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6806', 'nonce': 4037911317}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6808', 'nonce': 4037911317}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6807', 'nonce': 4037911317}]}, 'public_addr': '172.21.15.22:6805/4037911317', 'cluster_addr': '172.21.15.22:6806/4037911317', 'heartbeat_back_addr': '172.21.15.22:6808/4037911317', 'heartbeat_front_addr': '172.21.15.22:6807/4037911317', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'fa2b2730-2c1d-45d0-be8a-494a60dc6b9b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 360, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6809', 'nonce': 1675799666}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6810', 'nonce': 1675799666}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6812', 'nonce': 1675799666}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6811', 'nonce': 1675799666}]}, 'public_addr': '172.21.15.22:6809/1675799666', 'cluster_addr': '172.21.15.22:6810/1675799666', 'heartbeat_back_addr': '172.21.15.22:6812/1675799666', 'heartbeat_front_addr': '172.21.15.22:6811/1675799666', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '8d53d3fc-e401-47df-b70b-4b4b1f4539df', 'up': 1, 'in': 1, 'weight': 0.686370849609375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 25, 'up_thru': 341, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6813', 'nonce': 1501962752}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6814', 'nonce': 1501962752}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6816', 'nonce': 1501962752}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6815', 'nonce': 1501962752}]}, 'public_addr': '172.21.15.22:6813/1501962752', 'cluster_addr': '172.21.15.22:6814/1501962752', 'heartbeat_back_addr': '172.21.15.22:6816/1501962752', 'heartbeat_front_addr': '172.21.15.22:6815/1501962752', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'd3207fe0-8b97-4067-a755-ba6d449c53c7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 360, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6800', 'nonce': 1210229443}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6801', 'nonce': 1210229443}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6803', 'nonce': 1210229443}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6802', 'nonce': 1210229443}]}, 'public_addr': '172.21.15.104:6800/1210229443', 'cluster_addr': '172.21.15.104:6801/1210229443', 'heartbeat_back_addr': '172.21.15.104:6803/1210229443', 'heartbeat_front_addr': '172.21.15.104:6802/1210229443', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '89c9352b-30b2-48c6-869e-047d07a23a43', 'up': 1, 'in': 1, 'weight': 0.6509552001953125, 'primary_affinity': 0.666046142578125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 37, 'up_thru': 360, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6804', 'nonce': 2735328732}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6805', 'nonce': 2735328732}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6807', 'nonce': 2735328732}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6806', 'nonce': 2735328732}]}, 'public_addr': '172.21.15.104:6804/2735328732', 'cluster_addr': '172.21.15.104:6805/2735328732', 'heartbeat_back_addr': '172.21.15.104:6807/2735328732', 'heartbeat_front_addr': '172.21.15.104:6806/2735328732', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '704563de-d88f-480a-b469-720135704d42', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 44, 'up_thru': 177, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6808', 'nonce': 721493010}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6809', 'nonce': 721493010}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6811', 'nonce': 721493010}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6810', 'nonce': 721493010}]}, 'public_addr': '172.21.15.104:6808/721493010', 'cluster_addr': '172.21.15.104:6809/721493010', 'heartbeat_back_addr': '172.21.15.104:6811/721493010', 'heartbeat_front_addr': '172.21.15.104:6810/721493010', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '18199e47-0052-49f0-b45f-0fb201e0d0f4', 'up': 1, 'in': 1, 'weight': 0.6657562255859375, 'primary_affinity': 1, 'last_clean_begin': 52, 'last_clean_end': 133, 'up_from': 147, 'up_thru': 360, 'down_at': 134, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6812', 'nonce': 2218863853}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6813', 'nonce': 2218863853}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6815', 'nonce': 2218863853}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6814', 'nonce': 2218863853}]}, 'public_addr': '172.21.15.104:6812/2218863853', 'cluster_addr': '172.21.15.104:6813/2218863853', 'heartbeat_back_addr': '172.21.15.104:6815/2218863853', 'heartbeat_front_addr': '172.21.15.104:6814/2218863853', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:42:05.640226+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:43:17.772622+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:42:00.226145+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:42:18.279268+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:43:10.409639+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:43:14.427937+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:43:19.684540+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:42:20.744248+0000', 'dead_epoch': 133}], 'pg_upmap': [{'pgid': '2.1c', 'osds': [5, 2, 7]}, {'pgid': '4.1e', 'osds': [4, 5, 3]}], 'pg_upmap_items': [{'pgid': '4.7', 'mappings': [{'from': 3, 'to': 7}]}, {'pgid': '4.15', 'mappings': [{'from': 3, 'to': 0}]}, {'pgid': '4.1b', 'mappings': [{'from': 3, 'to': 7}]}, {'pgid': '5.c', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.22:0/240999923': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:6800/4016369460': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/585929238': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/544031024': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:6800/2504196132': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/1451388349': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/639336081': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/2104069730': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/4236450731': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:6800/3573193776': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/2661764422': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/2197488530': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/2968034970': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/599318175': '2024-09-12T07:27:07.919998+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-11T07:43:24.730 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:43:24.788 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:24.854 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:24.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:24.855 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:24.956 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:24.962 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 69 16 105705 105689 95.7159 76.375 0.0190538 0.0104424 2024-09-11T07:43:24.986 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:24.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:24.987 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:25.088 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:25.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:25.118 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:25.118 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:25.219 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:25.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:25.268 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:25.268 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:25.369 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:25.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:25.393 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:25.393 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:25.416 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:25 smithi104 ceph-mon[33507]: 5.c scrub starts 2024-09-11T07:43:25.416 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:25 smithi104 ceph-mon[33507]: 5.c scrub ok 2024-09-11T07:43:25.416 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:25 smithi104 ceph-mon[33507]: 6.6 scrub starts 2024-09-11T07:43:25.416 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:25 smithi104 ceph-mon[33507]: 6.6 scrub ok 2024-09-11T07:43:25.416 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:25 smithi104 ceph-mon[33507]: osdmap e366: 8 total, 8 up, 8 in 2024-09-11T07:43:25.416 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:25 smithi104 ceph-mon[33507]: pgmap v785: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.7 KiB/s rd, 101 MiB/s wr, 1.62k op/s 2024-09-11T07:43:25.416 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:25 smithi104 ceph-mon[33507]: 4.1 scrub starts 2024-09-11T07:43:25.416 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:25 smithi104 ceph-mon[33507]: 2.7 scrub starts 2024-09-11T07:43:25.416 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:25 smithi104 ceph-mon[33507]: 2.7 scrub ok 2024-09-11T07:43:25.416 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:25 smithi104 ceph-mon[33507]: 4.1 scrub ok 2024-09-11T07:43:25.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:25 smithi022 ceph-mon[29462]: 5.c scrub starts 2024-09-11T07:43:25.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:25 smithi022 ceph-mon[29462]: 5.c scrub ok 2024-09-11T07:43:25.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:25 smithi022 ceph-mon[29462]: 6.6 scrub starts 2024-09-11T07:43:25.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:25 smithi022 ceph-mon[29462]: 6.6 scrub ok 2024-09-11T07:43:25.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:25 smithi022 ceph-mon[29462]: osdmap e366: 8 total, 8 up, 8 in 2024-09-11T07:43:25.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:25 smithi022 ceph-mon[29462]: pgmap v785: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.7 KiB/s rd, 101 MiB/s wr, 1.62k op/s 2024-09-11T07:43:25.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:25 smithi022 ceph-mon[29462]: 4.1 scrub starts 2024-09-11T07:43:25.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:25 smithi022 ceph-mon[29462]: 2.7 scrub starts 2024-09-11T07:43:25.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:25 smithi022 ceph-mon[29462]: 2.7 scrub ok 2024-09-11T07:43:25.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:25 smithi022 ceph-mon[29462]: 4.1 scrub ok 2024-09-11T07:43:25.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:25 smithi022 ceph-mon[37330]: 5.c scrub starts 2024-09-11T07:43:25.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:25 smithi022 ceph-mon[37330]: 5.c scrub ok 2024-09-11T07:43:25.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:25 smithi022 ceph-mon[37330]: 6.6 scrub starts 2024-09-11T07:43:25.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:25 smithi022 ceph-mon[37330]: 6.6 scrub ok 2024-09-11T07:43:25.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:25 smithi022 ceph-mon[37330]: osdmap e366: 8 total, 8 up, 8 in 2024-09-11T07:43:25.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:25 smithi022 ceph-mon[37330]: pgmap v785: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.7 KiB/s rd, 101 MiB/s wr, 1.62k op/s 2024-09-11T07:43:25.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:25 smithi022 ceph-mon[37330]: 4.1 scrub starts 2024-09-11T07:43:25.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:25 smithi022 ceph-mon[37330]: 2.7 scrub starts 2024-09-11T07:43:25.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:25 smithi022 ceph-mon[37330]: 2.7 scrub ok 2024-09-11T07:43:25.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:25 smithi022 ceph-mon[37330]: 4.1 scrub ok 2024-09-11T07:43:25.494 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:25.531 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:25.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:25.532 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:25.568 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:43:25.633 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:25.711 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:25.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:25.712 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:25.813 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:25.845 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:25.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:25.846 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:25.857 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:43:25.946 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:25.963 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 70 16 106625 106609 95.17 57.5 0.0148705 0.0105025 2024-09-11T07:43:25.986 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:25.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:25.987 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:26.088 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:26.109 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:26.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:26.110 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:26.211 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:26.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:26.233 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:26.233 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:26.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:26 smithi104 ceph-mon[33507]: 5.1e scrub starts 2024-09-11T07:43:26.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:26 smithi104 ceph-mon[33507]: 5.1e scrub ok 2024-09-11T07:43:26.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:26 smithi104 ceph-mon[33507]: 2.8 deep-scrub starts 2024-09-11T07:43:26.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:26 smithi104 ceph-mon[33507]: 2.8 deep-scrub ok 2024-09-11T07:43:26.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:26 smithi104 ceph-mon[33507]: 6.1a scrub starts 2024-09-11T07:43:26.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:26 smithi104 ceph-mon[33507]: 6.1a scrub ok 2024-09-11T07:43:26.333 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:26.356 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:26.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:26.357 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:26.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:26 smithi022 ceph-mon[29462]: 5.1e scrub starts 2024-09-11T07:43:26.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:26 smithi022 ceph-mon[29462]: 5.1e scrub ok 2024-09-11T07:43:26.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:26 smithi022 ceph-mon[29462]: 2.8 deep-scrub starts 2024-09-11T07:43:26.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:26 smithi022 ceph-mon[29462]: 2.8 deep-scrub ok 2024-09-11T07:43:26.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:26 smithi022 ceph-mon[29462]: 6.1a scrub starts 2024-09-11T07:43:26.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:26 smithi022 ceph-mon[29462]: 6.1a scrub ok 2024-09-11T07:43:26.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:26 smithi022 ceph-mon[37330]: 5.1e scrub starts 2024-09-11T07:43:26.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:26 smithi022 ceph-mon[37330]: 5.1e scrub ok 2024-09-11T07:43:26.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:26 smithi022 ceph-mon[37330]: 2.8 deep-scrub starts 2024-09-11T07:43:26.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:26 smithi022 ceph-mon[37330]: 2.8 deep-scrub ok 2024-09-11T07:43:26.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:26 smithi022 ceph-mon[37330]: 6.1a scrub starts 2024-09-11T07:43:26.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:26 smithi022 ceph-mon[37330]: 6.1a scrub ok 2024-09-11T07:43:26.458 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:26.480 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:26.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:26.482 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:26.582 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:26.621 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:26.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:26.623 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:26.724 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:26.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:26.766 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:26.766 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:26.867 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:26.888 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:26.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:26.889 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:26.962 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 71 16 107469 107453 94.5725 52.75 0.0135814 0.010569 2024-09-11T07:43:26.990 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:27.029 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:27.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:27.032 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:27.118 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is set 2024-09-11T07:43:27.133 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:27.158 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:27.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:27.159 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:27.260 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:27.280 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:27.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:27.281 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:27.367 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:27 smithi022 ceph-mon[29462]: 4.1d deep-scrub starts 2024-09-11T07:43:27.367 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:27 smithi022 ceph-mon[29462]: 4.1d deep-scrub ok 2024-09-11T07:43:27.367 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:27 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/32139923' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:43:27.367 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:27 smithi022 ceph-mon[29462]: pgmap v786: 148 pgs: 148 active+clean; 6.4 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.3 KiB/s rd, 85 MiB/s wr, 1.37k op/s 2024-09-11T07:43:27.367 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:27 smithi022 ceph-mon[29462]: 5.e deep-scrub starts 2024-09-11T07:43:27.368 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:27 smithi022 ceph-mon[29462]: 5.e deep-scrub ok 2024-09-11T07:43:27.368 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:27 smithi022 ceph-mon[29462]: 2.10 deep-scrub starts 2024-09-11T07:43:27.368 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:27 smithi022 ceph-mon[29462]: 2.10 deep-scrub ok 2024-09-11T07:43:27.368 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:27 smithi022 ceph-mon[37330]: 4.1d deep-scrub starts 2024-09-11T07:43:27.368 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:27 smithi022 ceph-mon[37330]: 4.1d deep-scrub ok 2024-09-11T07:43:27.368 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:27 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/32139923' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:43:27.368 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:27 smithi022 ceph-mon[37330]: pgmap v786: 148 pgs: 148 active+clean; 6.4 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.3 KiB/s rd, 85 MiB/s wr, 1.37k op/s 2024-09-11T07:43:27.368 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:27 smithi022 ceph-mon[37330]: 5.e deep-scrub starts 2024-09-11T07:43:27.368 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:27 smithi022 ceph-mon[37330]: 5.e deep-scrub ok 2024-09-11T07:43:27.368 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:27 smithi022 ceph-mon[37330]: 2.10 deep-scrub starts 2024-09-11T07:43:27.368 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:27 smithi022 ceph-mon[37330]: 2.10 deep-scrub ok 2024-09-11T07:43:27.382 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:27.414 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:27.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:27.415 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:27.516 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:27.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:27 smithi104 ceph-mon[33507]: 4.1d deep-scrub starts 2024-09-11T07:43:27.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:27 smithi104 ceph-mon[33507]: 4.1d deep-scrub ok 2024-09-11T07:43:27.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:27 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/32139923' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:43:27.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:27 smithi104 ceph-mon[33507]: pgmap v786: 148 pgs: 148 active+clean; 6.4 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.3 KiB/s rd, 85 MiB/s wr, 1.37k op/s 2024-09-11T07:43:27.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:27 smithi104 ceph-mon[33507]: 5.e deep-scrub starts 2024-09-11T07:43:27.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:27 smithi104 ceph-mon[33507]: 5.e deep-scrub ok 2024-09-11T07:43:27.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:27 smithi104 ceph-mon[33507]: 2.10 deep-scrub starts 2024-09-11T07:43:27.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:27 smithi104 ceph-mon[33507]: 2.10 deep-scrub ok 2024-09-11T07:43:27.552 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:27.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:27.553 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:27.654 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:27.697 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:27.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:27.698 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:27.799 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:27.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:27.821 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:27.822 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:27.922 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:27.954 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:27.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:27.955 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:27.962 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 72 16 108506 108490 94.1591 64.8125 0.00678203 0.0106154 2024-09-11T07:43:28.056 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:28.100 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:28.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:28.102 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:28.105 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:28 smithi022 ceph-mon[29462]: 2.1d scrub starts 2024-09-11T07:43:28.105 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:28 smithi022 ceph-mon[29462]: 2.1d scrub ok 2024-09-11T07:43:28.105 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:28 smithi022 ceph-mon[29462]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:43:28.105 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:28 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/32139923' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:43:28.105 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:28 smithi022 ceph-mon[29462]: osdmap e367: 8 total, 8 up, 8 in 2024-09-11T07:43:28.105 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:28 smithi022 ceph-mon[37330]: 2.1d scrub starts 2024-09-11T07:43:28.105 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:28 smithi022 ceph-mon[37330]: 2.1d scrub ok 2024-09-11T07:43:28.105 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:28 smithi022 ceph-mon[37330]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:43:28.105 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:28 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/32139923' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:43:28.105 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:28 smithi022 ceph-mon[37330]: osdmap e367: 8 total, 8 up, 8 in 2024-09-11T07:43:28.202 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:28.241 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:28.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:28.242 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:28.343 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:28.383 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:28.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:28.384 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:28.439 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:28 smithi104 ceph-mon[33507]: 2.1d scrub starts 2024-09-11T07:43:28.439 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:28 smithi104 ceph-mon[33507]: 2.1d scrub ok 2024-09-11T07:43:28.440 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:28 smithi104 ceph-mon[33507]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:43:28.440 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:28 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/32139923' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:43:28.440 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:28 smithi104 ceph-mon[33507]: osdmap e367: 8 total, 8 up, 8 in 2024-09-11T07:43:28.484 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:28.524 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:28.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:28.525 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:28.625 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:28.664 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:28.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:28.665 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:28.765 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:28.812 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:28.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:28.813 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:28.913 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:28.930 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:43:28.930 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:43:28.953 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:28.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:28.955 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:28.962 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 73 15 109759 109744 93.9428 78.375 0.0177945 0.0106398 2024-09-11T07:43:29.056 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:29.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:29.095 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:29.096 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:29.142 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:43:29.197 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:29.242 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:29.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:29.244 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:29.344 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:29.382 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:29.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:29.384 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:29.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:29 smithi022 ceph-mon[29462]: pgmap v788: 148 pgs: 148 active+clean; 6.4 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 639 B/s rd, 39 MiB/s wr, 617 op/s 2024-09-11T07:43:29.409 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:29 smithi022 ceph-mon[37330]: pgmap v788: 148 pgs: 148 active+clean; 6.4 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 639 B/s rd, 39 MiB/s wr, 617 op/s 2024-09-11T07:43:29.484 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:29.493 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:29 smithi104 ceph-mon[33507]: pgmap v788: 148 pgs: 148 active+clean; 6.4 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 639 B/s rd, 39 MiB/s wr, 617 op/s 2024-09-11T07:43:29.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:29.521 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:29.521 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:29.622 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:29.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:29.661 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:29.661 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:29.728 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-11T07:43:29.762 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:29.803 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:29.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:29.804 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:29.905 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:29.944 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:29.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:29.945 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:29.963 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 74 16 110913 110897 93.6471 72.0625 0.00865805 0.0106736 2024-09-11T07:43:30.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:43:29 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:43:30.047 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:30.080 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:30.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:30.082 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:30.133 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:43:30.145 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:43:30.149 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:43:30.182 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:30.223 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:30.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:30.224 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:30.325 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:30.361 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:30.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:30.362 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:30.462 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:30.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:30.501 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:30.501 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:30.602 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:30.633 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T07:43:30.634 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:30.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:30.635 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:30.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:43:30 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:43:30] "GET /metrics HTTP/1.1" 200 224961 "" "Prometheus/2.43.0" 2024-09-11T07:43:30.737 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:30.765 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:30.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:30.767 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:30.867 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:30.901 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:30.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:30.903 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:30.963 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 75 16 112135 112119 93.4167 76.375 0.0172453 0.0107003 2024-09-11T07:43:31.004 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:31.051 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:31.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:31.052 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:31.153 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:31.177 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":788,"stamp":"2024-09-11T07:43:28.302713+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6859659484,"num_objects":104857,"num_object_clones":0,"num_object_copies":314571,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104857,"num_whiteouts":0,"num_read":2275,"num_read_kb":2132,"num_write":105490,"num_write_kb":6699975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":1185095,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83027,"ondisk_log_size":83027,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":25946820,"kb_used_data":19979508,"kb_used_omap":16,"kb_used_meta":5966959,"kb_avail":724014396,"statfs":{"total":767960285184,"available":741390741504,"internally_reserved":0,"allocated":20459016192,"data_stored":20455827768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16530,"internal_metadata":6110166894},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,0,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":323616768,"num_objects":4938,"num_object_clones":0,"num_object_copies":14814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4938,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4938,"num_write_kb":316032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002179"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":193,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.739918+0000","last_change":"2024-09-11T07:41:31.455086+0000","last_active":"2024-09-11T07:43:18.739918+0000","last_peered":"2024-09-11T07:43:18.739918+0000","last_clean":"2024-09-11T07:43:18.739918+0000","last_became_active":"2024-09-11T07:40:39.605361+0000","last_became_peered":"2024-09-11T07:40:39.605361+0000","last_unstale":"2024-09-11T07:43:18.739918+0000","last_undegraded":"2024-09-11T07:43:18.739918+0000","last_fullsized":"2024-09-11T07:43:18.739918+0000","mapping_epoch":310,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":311,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:31.455034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:29.007411+0000","last_clean_scrub_stamp":"2024-09-11T07:41:31.455034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014648129999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":403,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.746274+0000","last_change":"2024-09-11T07:42:11.150484+0000","last_active":"2024-09-11T07:43:18.746274+0000","last_peered":"2024-09-11T07:43:18.746274+0000","last_clean":"2024-09-11T07:43:18.746274+0000","last_became_active":"2024-09-11T07:32:00.508572+0000","last_became_peered":"2024-09-11T07:32:00.508572+0000","last_unstale":"2024-09-11T07:43:18.746274+0000","last_undegraded":"2024-09-11T07:43:18.746274+0000","last_fullsized":"2024-09-11T07:43:18.746274+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:11.150417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:42:11.150417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:30.546879+0000","scrub_duration":0.0022784350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"234'34","reported_seq":428,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.758863+0000","last_change":"2024-09-11T07:42:21.994403+0000","last_active":"2024-09-11T07:43:19.758863+0000","last_peered":"2024-09-11T07:43:19.758863+0000","last_clean":"2024-09-11T07:43:19.758863+0000","last_became_active":"2024-09-11T07:26:33.544627+0000","last_became_peered":"2024-09-11T07:26:33.544627+0000","last_unstale":"2024-09-11T07:43:19.758863+0000","last_undegraded":"2024-09-11T07:43:19.758863+0000","last_fullsized":"2024-09-11T07:43:19.758863+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'34","last_scrub_stamp":"2024-09-11T07:42:21.994239+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:42:21.994239+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:32.497793+0000","scrub_duration":0.014537408999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":392,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:21.291916+0000","last_change":"2024-09-11T07:43:21.291883+0000","last_active":"2024-09-11T07:43:21.291916+0000","last_peered":"2024-09-11T07:43:21.291916+0000","last_clean":"2024-09-11T07:43:21.291916+0000","last_became_active":"2024-09-11T07:26:35.556296+0000","last_became_peered":"2024-09-11T07:26:35.556296+0000","last_unstale":"2024-09-11T07:43:21.291916+0000","last_undegraded":"2024-09-11T07:43:21.291916+0000","last_fullsized":"2024-09-11T07:43:21.291916+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:21.291787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:43:21.291787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:46.989515+0000","scrub_duration":0.003794969,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":400,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.755552+0000","last_change":"2024-09-11T07:42:26.890346+0000","last_active":"2024-09-11T07:43:19.755552+0000","last_peered":"2024-09-11T07:43:19.755552+0000","last_clean":"2024-09-11T07:43:19.755552+0000","last_became_active":"2024-09-11T07:39:57.767806+0000","last_became_peered":"2024-09-11T07:39:57.767806+0000","last_unstale":"2024-09-11T07:43:19.755552+0000","last_undegraded":"2024-09-11T07:43:19.755552+0000","last_fullsized":"2024-09-11T07:43:19.755552+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:26.890251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:42:26.890251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:46.907530+0000","scrub_duration":0.011003028999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"234'19","reported_seq":409,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.746383+0000","last_change":"2024-09-11T07:41:21.709325+0000","last_active":"2024-09-11T07:43:18.746383+0000","last_peered":"2024-09-11T07:43:18.746383+0000","last_clean":"2024-09-11T07:43:18.746383+0000","last_became_active":"2024-09-11T07:26:33.540732+0000","last_became_peered":"2024-09-11T07:26:33.540732+0000","last_unstale":"2024-09-11T07:43:18.746383+0000","last_undegraded":"2024-09-11T07:43:18.746383+0000","last_fullsized":"2024-09-11T07:43:18.746383+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:41:21.709271+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:36:42.708941+0000","last_clean_scrub_stamp":"2024-09-11T07:41:21.709271+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001672682,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":401,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.656624+0000","last_change":"2024-09-11T07:43:19.656563+0000","last_active":"2024-09-11T07:43:19.656624+0000","last_peered":"2024-09-11T07:43:19.656624+0000","last_clean":"2024-09-11T07:43:19.656624+0000","last_became_active":"2024-09-11T07:32:00.513650+0000","last_became_peered":"2024-09-11T07:32:00.513650+0000","last_unstale":"2024-09-11T07:43:19.656624+0000","last_undegraded":"2024-09-11T07:43:19.656624+0000","last_fullsized":"2024-09-11T07:43:19.656624+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:19.656481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:43:19.656481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:37.657925+0000","scrub_duration":0.0032999000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":393,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.744845+0000","last_change":"2024-09-11T07:40:36.708489+0000","last_active":"2024-09-11T07:43:18.744845+0000","last_peered":"2024-09-11T07:43:18.744845+0000","last_clean":"2024-09-11T07:43:18.744845+0000","last_became_active":"2024-09-11T07:40:11.698235+0000","last_became_peered":"2024-09-11T07:40:11.698235+0000","last_unstale":"2024-09-11T07:43:18.744845+0000","last_undegraded":"2024-09-11T07:43:18.744845+0000","last_fullsized":"2024-09-11T07:43:18.744845+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:40:36.708406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:40:36.708406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020771840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":382,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.755664+0000","last_change":"2024-09-11T07:41:36.750495+0000","last_active":"2024-09-11T07:43:19.755664+0000","last_peered":"2024-09-11T07:43:19.755664+0000","last_clean":"2024-09-11T07:43:19.755664+0000","last_became_active":"2024-09-11T07:32:00.516662+0000","last_became_peered":"2024-09-11T07:32:00.516662+0000","last_unstale":"2024-09-11T07:43:19.755664+0000","last_undegraded":"2024-09-11T07:43:19.755664+0000","last_fullsized":"2024-09-11T07:43:19.755664+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:36.750440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:41:36.750440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014695050000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"234'9","reported_seq":431,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.744246+0000","last_change":"2024-09-11T07:43:10.453615+0000","last_active":"2024-09-11T07:43:18.744246+0000","last_peered":"2024-09-11T07:43:18.744246+0000","last_clean":"2024-09-11T07:43:18.744246+0000","last_became_active":"2024-09-11T07:43:10.446549+0000","last_became_peered":"2024-09-11T07:43:10.446549+0000","last_unstale":"2024-09-11T07:43:18.744246+0000","last_undegraded":"2024-09-11T07:43:18.744246+0000","last_fullsized":"2024-09-11T07:43:18.744246+0000","mapping_epoch":360,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":361,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'9","last_scrub_stamp":"2024-09-11T07:42:14.618430+0000","last_deep_scrub":"82'7","last_deep_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_clean_scrub_stamp":"2024-09-11T07:42:14.618430+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:32.072640+0000","scrub_duration":0.001734221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":361,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.755222+0000","last_change":"2024-09-11T07:43:03.593232+0000","last_active":"2024-09-11T07:43:19.755222+0000","last_peered":"2024-09-11T07:43:19.755222+0000","last_clean":"2024-09-11T07:43:19.755222+0000","last_became_active":"2024-09-11T07:32:00.513206+0000","last_became_peered":"2024-09-11T07:32:00.513206+0000","last_unstale":"2024-09-11T07:43:19.755222+0000","last_undegraded":"2024-09-11T07:43:19.755222+0000","last_fullsized":"2024-09-11T07:43:19.755222+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:03.593136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.973323+0000","last_clean_scrub_stamp":"2024-09-11T07:43:03.593136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:12.876322+0000","scrub_duration":0.0028732409999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":109,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.739557+0000","last_change":"2024-09-11T07:41:38.462561+0000","last_active":"2024-09-11T07:43:18.739557+0000","last_peered":"2024-09-11T07:43:18.739557+0000","last_clean":"2024-09-11T07:43:18.739557+0000","last_became_active":"2024-09-11T07:40:11.701560+0000","last_became_peered":"2024-09-11T07:40:11.701560+0000","last_unstale":"2024-09-11T07:43:18.739557+0000","last_undegraded":"2024-09-11T07:43:18.739557+0000","last_fullsized":"2024-09-11T07:43:18.739557+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:38.462496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:45.149916+0000","last_clean_scrub_stamp":"2024-09-11T07:41:38.462496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012557239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":347,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.762040+0000","last_change":"2024-09-11T07:42:10.187899+0000","last_active":"2024-09-11T07:43:19.762040+0000","last_peered":"2024-09-11T07:43:19.762040+0000","last_clean":"2024-09-11T07:43:19.762040+0000","last_became_active":"2024-09-11T07:32:00.505709+0000","last_became_peered":"2024-09-11T07:32:00.505709+0000","last_unstale":"2024-09-11T07:43:19.762040+0000","last_undegraded":"2024-09-11T07:43:19.762040+0000","last_fullsized":"2024-09-11T07:43:19.762040+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:10.187808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_clean_scrub_stamp":"2024-09-11T07:42:10.187808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001432266,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"234'19","reported_seq":406,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.745093+0000","last_change":"2024-09-11T07:41:24.659232+0000","last_active":"2024-09-11T07:43:18.745093+0000","last_peered":"2024-09-11T07:43:18.745093+0000","last_clean":"2024-09-11T07:43:18.745093+0000","last_became_active":"2024-09-11T07:26:33.544386+0000","last_became_peered":"2024-09-11T07:26:33.544386+0000","last_unstale":"2024-09-11T07:43:18.745093+0000","last_undegraded":"2024-09-11T07:43:18.745093+0000","last_fullsized":"2024-09-11T07:43:18.745093+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:41:24.659175+0000","last_deep_scrub":"136'13","last_deep_scrub_stamp":"2024-09-11T07:31:08.042198+0000","last_clean_scrub_stamp":"2024-09-11T07:41:24.659175+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001886731,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":398,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.748335+0000","last_change":"2024-09-11T07:42:12.197810+0000","last_active":"2024-09-11T07:43:18.748335+0000","last_peered":"2024-09-11T07:43:18.748335+0000","last_clean":"2024-09-11T07:43:18.748335+0000","last_became_active":"2024-09-11T07:40:11.699772+0000","last_became_peered":"2024-09-11T07:40:11.699772+0000","last_unstale":"2024-09-11T07:43:18.748335+0000","last_undegraded":"2024-09-11T07:43:18.748335+0000","last_fullsized":"2024-09-11T07:43:18.748335+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:12.197750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:42:12.197750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:34.084233+0000","scrub_duration":0.001336708,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":397,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.745971+0000","last_change":"2024-09-11T07:42:15.258247+0000","last_active":"2024-09-11T07:43:18.745971+0000","last_peered":"2024-09-11T07:43:18.745971+0000","last_clean":"2024-09-11T07:43:18.745971+0000","last_became_active":"2024-09-11T07:32:00.530335+0000","last_became_peered":"2024-09-11T07:32:00.530335+0000","last_unstale":"2024-09-11T07:43:18.745971+0000","last_undegraded":"2024-09-11T07:43:18.745971+0000","last_fullsized":"2024-09-11T07:43:18.745971+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:15.258189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:42:15.258189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:24.536319+0000","scrub_duration":0.0016177299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3","version":"365'6542","reported_seq":6573,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:24.034813+0000","last_change":"2024-09-11T07:41:57.697743+0000","last_active":"2024-09-11T07:43:24.034813+0000","last_peered":"2024-09-11T07:43:24.034813+0000","last_clean":"2024-09-11T07:43:24.034813+0000","last_became_active":"2024-09-11T07:41:57.697479+0000","last_became_peered":"2024-09-11T07:41:57.697479+0000","last_unstale":"2024-09-11T07:43:24.034813+0000","last_undegraded":"2024-09-11T07:43:24.034813+0000","last_fullsized":"2024-09-11T07:43:24.034813+0000","mapping_epoch":341,"log_start":"353'1158","ondisk_log_start":"353'1158","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:41:56.678995+0000","objects_scrubbed":0,"log_size":5384,"log_dups_size":0,"ondisk_log_size":5384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":428736512,"num_objects":6542,"num_object_clones":0,"num_object_copies":19626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6542,"num_write_kb":418688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"234'31","reported_seq":482,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.745347+0000","last_change":"2024-09-11T07:41:28.644798+0000","last_active":"2024-09-11T07:43:18.745347+0000","last_peered":"2024-09-11T07:43:18.745347+0000","last_clean":"2024-09-11T07:43:18.745347+0000","last_became_active":"2024-09-11T07:29:51.482755+0000","last_became_peered":"2024-09-11T07:29:51.482755+0000","last_unstale":"2024-09-11T07:43:18.745347+0000","last_undegraded":"2024-09-11T07:43:18.745347+0000","last_fullsized":"2024-09-11T07:43:18.745347+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'31","last_scrub_stamp":"2024-09-11T07:41:28.644723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:41:28.644723+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003343843,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":390,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.755472+0000","last_change":"2024-09-11T07:43:04.601186+0000","last_active":"2024-09-11T07:43:19.755472+0000","last_peered":"2024-09-11T07:43:19.755472+0000","last_clean":"2024-09-11T07:43:19.755472+0000","last_became_active":"2024-09-11T07:32:00.507191+0000","last_became_peered":"2024-09-11T07:32:00.507191+0000","last_unstale":"2024-09-11T07:43:19.755472+0000","last_undegraded":"2024-09-11T07:43:19.755472+0000","last_fullsized":"2024-09-11T07:43:19.755472+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:04.601133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:43:04.601133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:29.966043+0000","scrub_duration":0.0012288259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":24,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:23.359517+0000","last_change":"2024-09-11T07:43:23.359474+0000","last_active":"2024-09-11T07:43:23.359517+0000","last_peered":"2024-09-11T07:43:23.359517+0000","last_clean":"2024-09-11T07:43:23.359517+0000","last_became_active":"2024-09-11T07:43:10.467439+0000","last_became_peered":"2024-09-11T07:43:10.467439+0000","last_unstale":"2024-09-11T07:43:23.359517+0000","last_undegraded":"2024-09-11T07:43:23.359517+0000","last_fullsized":"2024-09-11T07:43:23.359517+0000","mapping_epoch":360,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":361,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:23.359400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:43:23.359400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:23.648005+0000","scrub_duration":0.002772437,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":393,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.755605+0000","last_change":"2024-09-11T07:41:32.281945+0000","last_active":"2024-09-11T07:43:18.755605+0000","last_peered":"2024-09-11T07:43:18.755605+0000","last_clean":"2024-09-11T07:43:18.755605+0000","last_became_active":"2024-09-11T07:26:37.564554+0000","last_became_peered":"2024-09-11T07:26:37.564554+0000","last_unstale":"2024-09-11T07:43:18.755605+0000","last_undegraded":"2024-09-11T07:43:18.755605+0000","last_fullsized":"2024-09-11T07:43:18.755605+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:32.281886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:32.281886+0000","last_clean_scrub_stamp":"2024-09-11T07:41:32.281886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018140579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"365'6474","reported_seq":6504,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:23.956456+0000","last_change":"2024-09-11T07:41:57.975075+0000","last_active":"2024-09-11T07:43:23.956456+0000","last_peered":"2024-09-11T07:43:23.956456+0000","last_clean":"2024-09-11T07:43:23.956456+0000","last_became_active":"2024-09-11T07:41:57.974623+0000","last_became_peered":"2024-09-11T07:41:57.974623+0000","last_unstale":"2024-09-11T07:43:23.956456+0000","last_undegraded":"2024-09-11T07:43:23.956456+0000","last_fullsized":"2024-09-11T07:43:23.956456+0000","mapping_epoch":341,"log_start":"354'2100","ondisk_log_start":"354'2100","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:41:56.678995+0000","objects_scrubbed":0,"log_size":4374,"log_dups_size":0,"ondisk_log_size":4374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":424280064,"num_objects":6474,"num_object_clones":0,"num_object_copies":19422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6474,"num_write_kb":414336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.c","version":"234'25","reported_seq":422,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.744936+0000","last_change":"2024-09-11T07:42:32.557611+0000","last_active":"2024-09-11T07:43:18.744936+0000","last_peered":"2024-09-11T07:43:18.744936+0000","last_clean":"2024-09-11T07:43:18.744936+0000","last_became_active":"2024-09-11T07:26:33.535411+0000","last_became_peered":"2024-09-11T07:26:33.535411+0000","last_unstale":"2024-09-11T07:43:18.744936+0000","last_undegraded":"2024-09-11T07:43:18.744936+0000","last_fullsized":"2024-09-11T07:43:18.744936+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'25","last_scrub_stamp":"2024-09-11T07:42:32.557306+0000","last_deep_scrub":"112'15","last_deep_scrub_stamp":"2024-09-11T07:31:10.005896+0000","last_clean_scrub_stamp":"2024-09-11T07:42:32.557306+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:41.957637+0000","scrub_duration":0.0042575720000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":394,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.740151+0000","last_change":"2024-09-11T07:41:36.437274+0000","last_active":"2024-09-11T07:43:18.740151+0000","last_peered":"2024-09-11T07:43:18.740151+0000","last_clean":"2024-09-11T07:43:18.740151+0000","last_became_active":"2024-09-11T07:32:00.506615+0000","last_became_peered":"2024-09-11T07:32:00.506615+0000","last_unstale":"2024-09-11T07:43:18.740151+0000","last_undegraded":"2024-09-11T07:43:18.740151+0000","last_fullsized":"2024-09-11T07:43:18.740151+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:36.437219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:41:36.437219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001620036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"299'11","reported_seq":638,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:21.382017+0000","last_change":"2024-09-11T07:43:21.381901+0000","last_active":"2024-09-11T07:43:21.382017+0000","last_peered":"2024-09-11T07:43:21.382017+0000","last_clean":"2024-09-11T07:43:21.382017+0000","last_became_active":"2024-09-11T07:40:11.697568+0000","last_became_peered":"2024-09-11T07:40:11.697568+0000","last_unstale":"2024-09-11T07:43:21.382017+0000","last_undegraded":"2024-09-11T07:43:21.382017+0000","last_fullsized":"2024-09-11T07:43:21.382017+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"299'11","last_scrub_stamp":"2024-09-11T07:43:21.381642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:43:21.381642+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:40.682699+0000","scrub_duration":0.0023952790000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":390,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.746491+0000","last_change":"2024-09-11T07:43:16.342459+0000","last_active":"2024-09-11T07:43:18.746491+0000","last_peered":"2024-09-11T07:43:18.746491+0000","last_clean":"2024-09-11T07:43:18.746491+0000","last_became_active":"2024-09-11T07:39:57.697588+0000","last_became_peered":"2024-09-11T07:39:57.697588+0000","last_unstale":"2024-09-11T07:43:18.746491+0000","last_undegraded":"2024-09-11T07:43:18.746491+0000","last_fullsized":"2024-09-11T07:43:18.746491+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:16.342336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:43:16.342336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:18.166654+0000","scrub_duration":0.063116406999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1","version":"365'6587","reported_seq":6619,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:23.938452+0000","last_change":"2024-09-11T07:41:57.700507+0000","last_active":"2024-09-11T07:43:23.938452+0000","last_peered":"2024-09-11T07:43:23.938452+0000","last_clean":"2024-09-11T07:43:23.938452+0000","last_became_active":"2024-09-11T07:41:57.700179+0000","last_became_peered":"2024-09-11T07:41:57.700179+0000","last_unstale":"2024-09-11T07:43:23.938452+0000","last_undegraded":"2024-09-11T07:43:23.938452+0000","last_fullsized":"2024-09-11T07:43:23.938452+0000","mapping_epoch":341,"log_start":"354'2300","ondisk_log_start":"354'2300","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:41:56.678995+0000","objects_scrubbed":0,"log_size":4287,"log_dups_size":0,"ondisk_log_size":4287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:24.303550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":431685632,"num_objects":6587,"num_object_clones":0,"num_object_copies":19761,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6587,"num_write_kb":421568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"234'34","reported_seq":423,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.742204+0000","last_change":"2024-09-11T07:41:29.498764+0000","last_active":"2024-09-11T07:43:18.742204+0000","last_peered":"2024-09-11T07:43:18.742204+0000","last_clean":"2024-09-11T07:43:18.742204+0000","last_became_active":"2024-09-11T07:26:33.549733+0000","last_became_peered":"2024-09-11T07:26:33.549733+0000","last_unstale":"2024-09-11T07:43:18.742204+0000","last_undegraded":"2024-09-11T07:43:18.742204+0000","last_fullsized":"2024-09-11T07:43:18.742204+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'34","last_scrub_stamp":"2024-09-11T07:41:29.498696+0000","last_deep_scrub":"83'22","last_deep_scrub_stamp":"2024-09-11T07:34:04.375457+0000","last_clean_scrub_stamp":"2024-09-11T07:41:29.498696+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018507770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":405,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.740338+0000","last_change":"2024-09-11T07:42:16.458616+0000","last_active":"2024-09-11T07:43:18.740338+0000","last_peered":"2024-09-11T07:43:18.740338+0000","last_clean":"2024-09-11T07:43:18.740338+0000","last_became_active":"2024-09-11T07:32:00.513472+0000","last_became_peered":"2024-09-11T07:32:00.513472+0000","last_unstale":"2024-09-11T07:43:18.740338+0000","last_undegraded":"2024-09-11T07:43:18.740338+0000","last_fullsized":"2024-09-11T07:43:18.740338+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:16.458519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:42:16.458519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:46.120193+0000","scrub_duration":0.002746727,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":585,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.712070+0000","last_change":"2024-09-11T07:40:46.534047+0000","last_active":"2024-09-11T07:43:19.712070+0000","last_peered":"2024-09-11T07:43:19.712070+0000","last_clean":"2024-09-11T07:43:19.712070+0000","last_became_active":"2024-09-11T07:26:35.560406+0000","last_became_peered":"2024-09-11T07:26:35.560406+0000","last_unstale":"2024-09-11T07:43:19.712070+0000","last_undegraded":"2024-09-11T07:43:19.712070+0000","last_fullsized":"2024-09-11T07:43:19.712070+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:40:46.533966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:40:46.533966+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050834449999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":391,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:21.294044+0000","last_change":"2024-09-11T07:43:21.293977+0000","last_active":"2024-09-11T07:43:21.294044+0000","last_peered":"2024-09-11T07:43:21.294044+0000","last_clean":"2024-09-11T07:43:21.294044+0000","last_became_active":"2024-09-11T07:39:57.762309+0000","last_became_peered":"2024-09-11T07:39:57.762309+0000","last_unstale":"2024-09-11T07:43:21.294044+0000","last_undegraded":"2024-09-11T07:43:21.294044+0000","last_fullsized":"2024-09-11T07:43:21.294044+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:21.293894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:31.200685+0000","last_clean_scrub_stamp":"2024-09-11T07:43:21.293894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:45.902198+0000","scrub_duration":0.0065006049999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"365'6411","reported_seq":6443,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:23.921757+0000","last_change":"2024-09-11T07:41:57.974484+0000","last_active":"2024-09-11T07:43:23.921757+0000","last_peered":"2024-09-11T07:43:23.921757+0000","last_clean":"2024-09-11T07:43:23.921757+0000","last_became_active":"2024-09-11T07:41:57.974162+0000","last_became_peered":"2024-09-11T07:41:57.974162+0000","last_unstale":"2024-09-11T07:43:23.921757+0000","last_undegraded":"2024-09-11T07:43:23.921757+0000","last_fullsized":"2024-09-11T07:43:23.921757+0000","mapping_epoch":341,"log_start":"351'200","ondisk_log_start":"351'200","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:41:56.678995+0000","objects_scrubbed":0,"log_size":6211,"log_dups_size":0,"ondisk_log_size":6211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420151296,"num_objects":6411,"num_object_clones":0,"num_object_copies":19233,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6411,"num_write_kb":410304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"234'24","reported_seq":442,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.746642+0000","last_change":"2024-09-11T07:40:38.740191+0000","last_active":"2024-09-11T07:43:18.746642+0000","last_peered":"2024-09-11T07:43:18.746642+0000","last_clean":"2024-09-11T07:43:18.746642+0000","last_became_active":"2024-09-11T07:39:57.766134+0000","last_became_peered":"2024-09-11T07:39:57.766134+0000","last_unstale":"2024-09-11T07:43:18.746642+0000","last_undegraded":"2024-09-11T07:43:18.746642+0000","last_fullsized":"2024-09-11T07:43:18.746642+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'24","last_scrub_stamp":"2024-09-11T07:40:38.740127+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-09-11T07:29:38.112111+0000","last_clean_scrub_stamp":"2024-09-11T07:40:38.740127+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044589670000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":374,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.755113+0000","last_change":"2024-09-11T07:41:18.636124+0000","last_active":"2024-09-11T07:43:19.755113+0000","last_peered":"2024-09-11T07:43:19.755113+0000","last_clean":"2024-09-11T07:43:19.755113+0000","last_became_active":"2024-09-11T07:32:00.518538+0000","last_became_peered":"2024-09-11T07:32:00.518538+0000","last_unstale":"2024-09-11T07:43:19.755113+0000","last_undegraded":"2024-09-11T07:43:19.755113+0000","last_fullsized":"2024-09-11T07:43:19.755113+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:18.636068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:41:18.636068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001280811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":41,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.746238+0000","last_change":"2024-09-11T07:42:19.714646+0000","last_active":"2024-09-11T07:43:18.746238+0000","last_peered":"2024-09-11T07:43:18.746238+0000","last_clean":"2024-09-11T07:43:18.746238+0000","last_became_active":"2024-09-11T07:42:04.729291+0000","last_became_peered":"2024-09-11T07:42:04.729291+0000","last_unstale":"2024-09-11T07:43:18.746238+0000","last_undegraded":"2024-09-11T07:43:18.746238+0000","last_fullsized":"2024-09-11T07:43:18.746238+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:19.714568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:19.714568+0000","last_clean_scrub_stamp":"2024-09-11T07:42:19.714568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:31.240795+0000","scrub_duration":0.0025615109999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":381,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.755014+0000","last_change":"2024-09-11T07:43:13.372948+0000","last_active":"2024-09-11T07:43:19.755014+0000","last_peered":"2024-09-11T07:43:19.755014+0000","last_clean":"2024-09-11T07:43:19.755014+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:43:19.755014+0000","last_undegraded":"2024-09-11T07:43:19.755014+0000","last_fullsized":"2024-09-11T07:43:19.755014+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:13.372485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:46.853387+0000","last_clean_scrub_stamp":"2024-09-11T07:43:13.372485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:42.160965+0000","scrub_duration":0.0038403320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"366'6536","reported_seq":6827,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-09-11T07:43:25.041551+0000","last_change":"2024-09-11T07:43:21.477070+0000","last_active":"2024-09-11T07:43:25.041551+0000","last_peered":"2024-09-11T07:43:25.041551+0000","last_clean":"2024-09-11T07:43:25.041551+0000","last_became_active":"2024-09-11T07:41:57.973267+0000","last_became_peered":"2024-09-11T07:41:57.973267+0000","last_unstale":"2024-09-11T07:43:25.041551+0000","last_undegraded":"2024-09-11T07:43:25.041551+0000","last_fullsized":"2024-09-11T07:43:25.041551+0000","mapping_epoch":341,"log_start":"351'500","ondisk_log_start":"351'500","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"365'6295","last_scrub_stamp":"2024-09-11T07:43:21.477005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:43:21.477005+0000","objects_scrubbed":6192,"log_size":6036,"log_dups_size":0,"ondisk_log_size":6036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:37.393380+0000","scrub_duration":2.9790974430000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":428343296,"num_objects":6536,"num_object_clones":0,"num_object_copies":19608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6536,"num_write_kb":418304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"234'28","reported_seq":416,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.745337+0000","last_change":"2024-09-11T07:40:40.689386+0000","last_active":"2024-09-11T07:43:18.745337+0000","last_peered":"2024-09-11T07:43:18.745337+0000","last_clean":"2024-09-11T07:43:18.745337+0000","last_became_active":"2024-09-11T07:26:33.537850+0000","last_became_peered":"2024-09-11T07:26:33.537850+0000","last_unstale":"2024-09-11T07:43:18.745337+0000","last_undegraded":"2024-09-11T07:43:18.745337+0000","last_fullsized":"2024-09-11T07:43:18.745337+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'28","last_scrub_stamp":"2024-09-11T07:40:40.689301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:40:40.689301+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071558330000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":152,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.755850+0000","last_change":"2024-09-11T07:41:33.261416+0000","last_active":"2024-09-11T07:43:18.755850+0000","last_peered":"2024-09-11T07:43:18.755850+0000","last_clean":"2024-09-11T07:43:18.755850+0000","last_became_active":"2024-09-11T07:39:57.767741+0000","last_became_peered":"2024-09-11T07:39:57.767741+0000","last_unstale":"2024-09-11T07:43:18.755850+0000","last_undegraded":"2024-09-11T07:43:18.755850+0000","last_fullsized":"2024-09-11T07:43:18.755850+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:33.261361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:41:33.261361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014665889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3],"acting":[2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"84'9","reported_seq":585,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-09-11T07:43:24.093239+0000","last_change":"2024-09-11T07:42:16.937638+0000","last_active":"2024-09-11T07:43:24.093239+0000","last_peered":"2024-09-11T07:43:24.093239+0000","last_clean":"2024-09-11T07:43:24.093239+0000","last_became_active":"2024-09-11T07:26:35.562997+0000","last_became_peered":"2024-09-11T07:26:35.562997+0000","last_unstale":"2024-09-11T07:43:24.093239+0000","last_undegraded":"2024-09-11T07:43:24.093239+0000","last_fullsized":"2024-09-11T07:43:24.093239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-11T07:42:16.937549+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-11T07:35:39.832547+0000","last_clean_scrub_stamp":"2024-09-11T07:42:16.937549+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:26.077746+0000","scrub_duration":0.0040836179999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":355,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.755331+0000","last_change":"2024-09-11T07:42:08.911482+0000","last_active":"2024-09-11T07:43:19.755331+0000","last_peered":"2024-09-11T07:43:19.755331+0000","last_clean":"2024-09-11T07:43:19.755331+0000","last_became_active":"2024-09-11T07:32:00.511005+0000","last_became_peered":"2024-09-11T07:32:00.511005+0000","last_unstale":"2024-09-11T07:43:19.755331+0000","last_undegraded":"2024-09-11T07:43:19.755331+0000","last_fullsized":"2024-09-11T07:43:19.755331+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:08.911420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:42:08.911420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:25.818418+0000","scrub_duration":0.0016919649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"365'6486","reported_seq":6518,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:23.967770+0000","last_change":"2024-09-11T07:41:57.973797+0000","last_active":"2024-09-11T07:43:23.967770+0000","last_peered":"2024-09-11T07:43:23.967770+0000","last_clean":"2024-09-11T07:43:23.967770+0000","last_became_active":"2024-09-11T07:41:57.973510+0000","last_became_peered":"2024-09-11T07:41:57.973510+0000","last_unstale":"2024-09-11T07:43:23.967770+0000","last_undegraded":"2024-09-11T07:43:23.967770+0000","last_fullsized":"2024-09-11T07:43:23.967770+0000","mapping_epoch":341,"log_start":"354'2200","ondisk_log_start":"354'2200","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:41:56.678995+0000","objects_scrubbed":0,"log_size":4286,"log_dups_size":0,"ondisk_log_size":4286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":425066496,"num_objects":6486,"num_object_clones":0,"num_object_copies":19458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6486,"num_write_kb":415104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"234'38","reported_seq":422,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.746051+0000","last_change":"2024-09-11T07:42:13.225893+0000","last_active":"2024-09-11T07:43:18.746051+0000","last_peered":"2024-09-11T07:43:18.746051+0000","last_clean":"2024-09-11T07:43:18.746051+0000","last_became_active":"2024-09-11T07:39:57.775541+0000","last_became_peered":"2024-09-11T07:39:57.775541+0000","last_unstale":"2024-09-11T07:43:18.746051+0000","last_undegraded":"2024-09-11T07:43:18.746051+0000","last_fullsized":"2024-09-11T07:43:18.746051+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'38","last_scrub_stamp":"2024-09-11T07:42:13.225787+0000","last_deep_scrub":"87'26","last_deep_scrub_stamp":"2024-09-11T07:35:52.942347+0000","last_clean_scrub_stamp":"2024-09-11T07:42:13.225787+0000","objects_scrubbed":8,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022801570000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":51,"num_read_kb":39,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":91,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.746687+0000","last_change":"2024-09-11T07:42:13.577814+0000","last_active":"2024-09-11T07:43:18.746687+0000","last_peered":"2024-09-11T07:43:18.746687+0000","last_clean":"2024-09-11T07:43:18.746687+0000","last_became_active":"2024-09-11T07:40:11.698558+0000","last_became_peered":"2024-09-11T07:40:11.698558+0000","last_unstale":"2024-09-11T07:43:18.746687+0000","last_undegraded":"2024-09-11T07:43:18.746687+0000","last_fullsized":"2024-09-11T07:43:18.746687+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:13.577750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:42:13.577750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017170639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":402,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.745172+0000","last_change":"2024-09-11T07:42:31.509275+0000","last_active":"2024-09-11T07:43:18.745172+0000","last_peered":"2024-09-11T07:43:18.745172+0000","last_clean":"2024-09-11T07:43:18.745172+0000","last_became_active":"2024-09-11T07:39:57.766717+0000","last_became_peered":"2024-09-11T07:39:57.766717+0000","last_unstale":"2024-09-11T07:43:18.745172+0000","last_undegraded":"2024-09-11T07:43:18.745172+0000","last_fullsized":"2024-09-11T07:43:18.745172+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:31.509199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:27.968579+0000","last_clean_scrub_stamp":"2024-09-11T07:42:31.509199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:47.703283+0000","scrub_duration":0.0025090170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":390,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.745065+0000","last_change":"2024-09-11T07:43:07.335284+0000","last_active":"2024-09-11T07:43:18.745065+0000","last_peered":"2024-09-11T07:43:18.745065+0000","last_clean":"2024-09-11T07:43:18.745065+0000","last_became_active":"2024-09-11T07:39:57.691831+0000","last_became_peered":"2024-09-11T07:39:57.691831+0000","last_unstale":"2024-09-11T07:43:18.745065+0000","last_undegraded":"2024-09-11T07:43:18.745065+0000","last_fullsized":"2024-09-11T07:43:18.745065+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:07.335214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:29.978798+0000","last_clean_scrub_stamp":"2024-09-11T07:43:07.335214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:28.421093+0000","scrub_duration":0.0025693080000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.d","version":"365'6452","reported_seq":6484,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:23.945786+0000","last_change":"2024-09-11T07:41:57.973609+0000","last_active":"2024-09-11T07:43:23.945786+0000","last_peered":"2024-09-11T07:43:23.945786+0000","last_clean":"2024-09-11T07:43:23.945786+0000","last_became_active":"2024-09-11T07:41:57.973299+0000","last_became_peered":"2024-09-11T07:41:57.973299+0000","last_unstale":"2024-09-11T07:43:23.945786+0000","last_undegraded":"2024-09-11T07:43:23.945786+0000","last_fullsized":"2024-09-11T07:43:23.945786+0000","mapping_epoch":341,"log_start":"354'2100","ondisk_log_start":"354'2100","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:41:56.678995+0000","objects_scrubbed":0,"log_size":4352,"log_dups_size":0,"ondisk_log_size":4352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":422838272,"num_objects":6452,"num_object_clones":0,"num_object_copies":19356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6452,"num_write_kb":412928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"236'32","reported_seq":455,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.748271+0000","last_change":"2024-09-11T07:43:12.244522+0000","last_active":"2024-09-11T07:43:18.748271+0000","last_peered":"2024-09-11T07:43:18.748271+0000","last_clean":"2024-09-11T07:43:18.748271+0000","last_became_active":"2024-09-11T07:32:00.506270+0000","last_became_peered":"2024-09-11T07:32:00.506270+0000","last_unstale":"2024-09-11T07:43:18.748271+0000","last_undegraded":"2024-09-11T07:43:18.748271+0000","last_fullsized":"2024-09-11T07:43:18.748271+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'32","last_scrub_stamp":"2024-09-11T07:43:12.244424+0000","last_deep_scrub":"236'32","last_deep_scrub_stamp":"2024-09-11T07:36:50.023909+0000","last_clean_scrub_stamp":"2024-09-11T07:43:12.244424+0000","objects_scrubbed":8,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:15.856735+0000","scrub_duration":0.013251099000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":355,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.755594+0000","last_change":"2024-09-11T07:42:09.915233+0000","last_active":"2024-09-11T07:43:19.755594+0000","last_peered":"2024-09-11T07:43:19.755594+0000","last_clean":"2024-09-11T07:43:19.755594+0000","last_became_active":"2024-09-11T07:32:00.510631+0000","last_became_peered":"2024-09-11T07:32:00.510631+0000","last_unstale":"2024-09-11T07:43:19.755594+0000","last_undegraded":"2024-09-11T07:43:19.755594+0000","last_fullsized":"2024-09-11T07:43:19.755594+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:09.915136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:49.144041+0000","last_clean_scrub_stamp":"2024-09-11T07:42:09.915136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:29.299092+0000","scrub_duration":0.001863003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":183,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.747273+0000","last_change":"2024-09-11T07:42:22.285910+0000","last_active":"2024-09-11T07:43:18.747273+0000","last_peered":"2024-09-11T07:43:18.747273+0000","last_clean":"2024-09-11T07:43:18.747273+0000","last_became_active":"2024-09-11T07:36:05.225840+0000","last_became_peered":"2024-09-11T07:36:05.225840+0000","last_unstale":"2024-09-11T07:43:18.747273+0000","last_undegraded":"2024-09-11T07:43:18.747273+0000","last_fullsized":"2024-09-11T07:43:18.747273+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:22.285763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:22.285763+0000","last_clean_scrub_stamp":"2024-09-11T07:42:22.285763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:30.253362+0000","scrub_duration":0.0093495249999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":382,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.739637+0000","last_change":"2024-09-11T07:43:10.524018+0000","last_active":"2024-09-11T07:43:18.739637+0000","last_peered":"2024-09-11T07:43:18.739637+0000","last_clean":"2024-09-11T07:43:18.739637+0000","last_became_active":"2024-09-11T07:26:37.861279+0000","last_became_peered":"2024-09-11T07:26:37.861279+0000","last_unstale":"2024-09-11T07:43:18.739637+0000","last_undegraded":"2024-09-11T07:43:18.739637+0000","last_fullsized":"2024-09-11T07:43:18.739637+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:10.523919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:39.723598+0000","last_clean_scrub_stamp":"2024-09-11T07:43:10.523919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:11.966449+0000","scrub_duration":0.0047615210000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"366'6647","reported_seq":6679,"reported_epoch":366,"state":"active+clean","last_fresh":"2024-09-11T07:43:24.471777+0000","last_change":"2024-09-11T07:41:57.699216+0000","last_active":"2024-09-11T07:43:24.471777+0000","last_peered":"2024-09-11T07:43:24.471777+0000","last_clean":"2024-09-11T07:43:24.471777+0000","last_became_active":"2024-09-11T07:41:57.698900+0000","last_became_peered":"2024-09-11T07:41:57.698900+0000","last_unstale":"2024-09-11T07:43:24.471777+0000","last_undegraded":"2024-09-11T07:43:24.471777+0000","last_fullsized":"2024-09-11T07:43:24.471777+0000","mapping_epoch":341,"log_start":"0'0","ondisk_log_start":"0'0","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:41:56.678995+0000","objects_scrubbed":0,"log_size":6647,"log_dups_size":0,"ondisk_log_size":6647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":435617792,"num_objects":6647,"num_object_clones":0,"num_object_copies":19941,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6647,"num_write_kb":425408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"234'20","reported_seq":389,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.745015+0000","last_change":"2024-09-11T07:43:12.364633+0000","last_active":"2024-09-11T07:43:18.745015+0000","last_peered":"2024-09-11T07:43:18.745015+0000","last_clean":"2024-09-11T07:43:18.745015+0000","last_became_active":"2024-09-11T07:39:57.692071+0000","last_became_peered":"2024-09-11T07:39:57.692071+0000","last_unstale":"2024-09-11T07:43:18.745015+0000","last_undegraded":"2024-09-11T07:43:18.745015+0000","last_fullsized":"2024-09-11T07:43:18.745015+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'20","last_scrub_stamp":"2024-09-11T07:43:12.364487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:43:12.364487+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:38.109742+0000","scrub_duration":0.0077528839999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2],"acting":[4,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":412,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.740608+0000","last_change":"2024-09-11T07:42:33.461705+0000","last_active":"2024-09-11T07:43:18.740608+0000","last_peered":"2024-09-11T07:43:18.740608+0000","last_clean":"2024-09-11T07:43:18.740608+0000","last_became_active":"2024-09-11T07:32:00.522456+0000","last_became_peered":"2024-09-11T07:32:00.522456+0000","last_unstale":"2024-09-11T07:43:18.740608+0000","last_undegraded":"2024-09-11T07:43:18.740608+0000","last_fullsized":"2024-09-11T07:43:18.740608+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:42:33.461606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:42:33.461606+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:53.605542+0000","scrub_duration":0.0041044219999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"290'9","reported_seq":616,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.713241+0000","last_change":"2024-09-11T07:43:05.319705+0000","last_active":"2024-09-11T07:43:19.713241+0000","last_peered":"2024-09-11T07:43:19.713241+0000","last_clean":"2024-09-11T07:43:19.713241+0000","last_became_active":"2024-09-11T07:39:57.768155+0000","last_became_peered":"2024-09-11T07:39:57.768155+0000","last_unstale":"2024-09-11T07:43:19.713241+0000","last_undegraded":"2024-09-11T07:43:19.713241+0000","last_fullsized":"2024-09-11T07:43:19.713241+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"290'9","last_scrub_stamp":"2024-09-11T07:43:05.319506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:43:05.319506+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:15.923377+0000","scrub_duration":0.0081677780000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":390,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.745839+0000","last_change":"2024-09-11T07:43:08.315384+0000","last_active":"2024-09-11T07:43:18.745839+0000","last_peered":"2024-09-11T07:43:18.745839+0000","last_clean":"2024-09-11T07:43:18.745839+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:43:18.745839+0000","last_undegraded":"2024-09-11T07:43:18.745839+0000","last_fullsized":"2024-09-11T07:43:18.745839+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:08.315222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_clean_scrub_stamp":"2024-09-11T07:43:08.315222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:36.641725+0000","scrub_duration":0.010657583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.a","version":"365'6489","reported_seq":6521,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:24.000389+0000","last_change":"2024-09-11T07:41:57.973674+0000","last_active":"2024-09-11T07:43:24.000389+0000","last_peered":"2024-09-11T07:43:24.000389+0000","last_clean":"2024-09-11T07:43:24.000389+0000","last_became_active":"2024-09-11T07:41:57.973322+0000","last_became_peered":"2024-09-11T07:41:57.973322+0000","last_unstale":"2024-09-11T07:43:24.000389+0000","last_undegraded":"2024-09-11T07:43:24.000389+0000","last_fullsized":"2024-09-11T07:43:24.000389+0000","mapping_epoch":341,"log_start":"354'2000","ondisk_log_start":"354'2000","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:41:56.678995+0000","objects_scrubbed":0,"log_size":4489,"log_dups_size":0,"ondisk_log_size":4489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:26.063600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":425263104,"num_objects":6489,"num_object_clones":0,"num_object_copies":19467,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6489,"num_write_kb":415296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"290'42","reported_seq":344,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:22.389421+0000","last_change":"2024-09-11T07:43:22.389378+0000","last_active":"2024-09-11T07:43:22.389421+0000","last_peered":"2024-09-11T07:43:22.389421+0000","last_clean":"2024-09-11T07:43:22.389421+0000","last_became_active":"2024-09-11T07:39:57.768267+0000","last_became_peered":"2024-09-11T07:39:57.768267+0000","last_unstale":"2024-09-11T07:43:22.389421+0000","last_undegraded":"2024-09-11T07:43:22.389421+0000","last_fullsized":"2024-09-11T07:43:22.389421+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"290'42","last_scrub_stamp":"2024-09-11T07:43:22.389278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:43:22.389278+0000","objects_scrubbed":10,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:33.624676+0000","scrub_duration":0.0085721119999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":41,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.740087+0000","last_change":"2024-09-11T07:42:25.444786+0000","last_active":"2024-09-11T07:43:18.740087+0000","last_peered":"2024-09-11T07:43:18.740087+0000","last_clean":"2024-09-11T07:43:18.740087+0000","last_became_active":"2024-09-11T07:42:04.727660+0000","last_became_peered":"2024-09-11T07:42:04.727660+0000","last_unstale":"2024-09-11T07:43:18.740087+0000","last_undegraded":"2024-09-11T07:43:18.740087+0000","last_fullsized":"2024-09-11T07:43:18.740087+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:25.444690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:51.308492+0000","last_clean_scrub_stamp":"2024-09-11T07:42:25.444690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:34.471025+0000","scrub_duration":0.0024959560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":382,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.746823+0000","last_change":"2024-09-11T07:42:05.078548+0000","last_active":"2024-09-11T07:43:18.746823+0000","last_peered":"2024-09-11T07:43:18.746823+0000","last_clean":"2024-09-11T07:43:18.746823+0000","last_became_active":"2024-09-11T07:26:35.560363+0000","last_became_peered":"2024-09-11T07:26:35.560363+0000","last_unstale":"2024-09-11T07:43:18.746823+0000","last_undegraded":"2024-09-11T07:43:18.746823+0000","last_fullsized":"2024-09-11T07:43:18.746823+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:05.078430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:42:05.078430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:30.390190+0000","scrub_duration":0.001770787,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":398,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:23.541555+0000","last_change":"2024-09-11T07:43:23.540793+0000","last_active":"2024-09-11T07:43:23.541555+0000","last_peered":"2024-09-11T07:43:23.541555+0000","last_clean":"2024-09-11T07:43:23.541555+0000","last_became_active":"2024-09-11T07:32:00.505919+0000","last_became_peered":"2024-09-11T07:32:00.505919+0000","last_unstale":"2024-09-11T07:43:23.541555+0000","last_undegraded":"2024-09-11T07:43:23.541555+0000","last_fullsized":"2024-09-11T07:43:23.541555+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:23.540701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:43:23.540701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:41.959569+0000","scrub_duration":0.0022189290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"365'6413","reported_seq":6445,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:24.022919+0000","last_change":"2024-09-11T07:41:57.697196+0000","last_active":"2024-09-11T07:43:24.022919+0000","last_peered":"2024-09-11T07:43:24.022919+0000","last_clean":"2024-09-11T07:43:24.022919+0000","last_became_active":"2024-09-11T07:41:57.696935+0000","last_became_peered":"2024-09-11T07:41:57.696935+0000","last_unstale":"2024-09-11T07:43:24.022919+0000","last_undegraded":"2024-09-11T07:43:24.022919+0000","last_fullsized":"2024-09-11T07:43:24.022919+0000","mapping_epoch":341,"log_start":"352'1045","ondisk_log_start":"352'1045","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:41:56.678995+0000","objects_scrubbed":0,"log_size":5368,"log_dups_size":0,"ondisk_log_size":5368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420282368,"num_objects":6413,"num_object_clones":0,"num_object_copies":19239,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6413,"num_write_kb":410432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"234'22","reported_seq":427,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:20.300703+0000","last_change":"2024-09-11T07:43:20.300627+0000","last_active":"2024-09-11T07:43:20.300703+0000","last_peered":"2024-09-11T07:43:20.300703+0000","last_clean":"2024-09-11T07:43:20.300703+0000","last_became_active":"2024-09-11T07:43:10.450225+0000","last_became_peered":"2024-09-11T07:43:10.450225+0000","last_unstale":"2024-09-11T07:43:20.300703+0000","last_undegraded":"2024-09-11T07:43:20.300703+0000","last_fullsized":"2024-09-11T07:43:20.300703+0000","mapping_epoch":360,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":361,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'22","last_scrub_stamp":"2024-09-11T07:43:20.300528+0000","last_deep_scrub":"144'18","last_deep_scrub_stamp":"2024-09-11T07:31:51.603378+0000","last_clean_scrub_stamp":"2024-09-11T07:43:20.300528+0000","objects_scrubbed":5,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:43.772925+0000","scrub_duration":0.004705435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":1406,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:22.646996+0000","last_change":"2024-09-11T07:43:15.545760+0000","last_active":"2024-09-11T07:43:22.646996+0000","last_peered":"2024-09-11T07:43:22.646996+0000","last_clean":"2024-09-11T07:43:22.646996+0000","last_became_active":"2024-09-11T07:26:31.421276+0000","last_became_peered":"2024-09-11T07:26:31.421276+0000","last_unstale":"2024-09-11T07:43:22.646996+0000","last_undegraded":"2024-09-11T07:43:22.646996+0000","last_fullsized":"2024-09-11T07:43:22.646996+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'5","last_scrub_stamp":"2024-09-11T07:43:15.545534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:43:15.545534+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:17.185625+0000","scrub_duration":0.0036066779999999999,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1003,"num_read_kb":998,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":415,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.751688+0000","last_change":"2024-09-11T07:42:23.099823+0000","last_active":"2024-09-11T07:43:18.751688+0000","last_peered":"2024-09-11T07:43:18.751688+0000","last_clean":"2024-09-11T07:43:18.751688+0000","last_became_active":"2024-09-11T07:26:31.413354+0000","last_became_peered":"2024-09-11T07:26:31.413354+0000","last_unstale":"2024-09-11T07:43:18.751688+0000","last_undegraded":"2024-09-11T07:43:18.751688+0000","last_fullsized":"2024-09-11T07:43:18.751688+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:23.099709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:07.149367+0000","last_clean_scrub_stamp":"2024-09-11T07:42:23.099709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:36.459656+0000","scrub_duration":0.0034093029999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":412,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.751627+0000","last_change":"2024-09-11T07:43:11.249241+0000","last_active":"2024-09-11T07:43:18.751627+0000","last_peered":"2024-09-11T07:43:18.751627+0000","last_clean":"2024-09-11T07:43:18.751627+0000","last_became_active":"2024-09-11T07:32:00.508398+0000","last_became_peered":"2024-09-11T07:32:00.508398+0000","last_unstale":"2024-09-11T07:43:18.751627+0000","last_undegraded":"2024-09-11T07:43:18.751627+0000","last_fullsized":"2024-09-11T07:43:18.751627+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:11.242643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:28.246601+0000","last_clean_scrub_stamp":"2024-09-11T07:43:11.242643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:36.220620+0000","scrub_duration":0.0056937530000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":381,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.755491+0000","last_change":"2024-09-11T07:42:17.790734+0000","last_active":"2024-09-11T07:43:19.755491+0000","last_peered":"2024-09-11T07:43:19.755491+0000","last_clean":"2024-09-11T07:43:19.755491+0000","last_became_active":"2024-09-11T07:32:00.515745+0000","last_became_peered":"2024-09-11T07:32:00.515745+0000","last_unstale":"2024-09-11T07:43:19.755491+0000","last_undegraded":"2024-09-11T07:43:19.755491+0000","last_fullsized":"2024-09-11T07:43:19.755491+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:17.790650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:56.490940+0000","last_clean_scrub_stamp":"2024-09-11T07:42:17.790650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:34.418486+0000","scrub_duration":0.0017167840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"365'6712","reported_seq":6744,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:24.065038+0000","last_change":"2024-09-11T07:41:57.697011+0000","last_active":"2024-09-11T07:43:24.065038+0000","last_peered":"2024-09-11T07:43:24.065038+0000","last_clean":"2024-09-11T07:43:24.065038+0000","last_became_active":"2024-09-11T07:41:57.696712+0000","last_became_peered":"2024-09-11T07:41:57.696712+0000","last_unstale":"2024-09-11T07:43:24.065038+0000","last_undegraded":"2024-09-11T07:43:24.065038+0000","last_fullsized":"2024-09-11T07:43:24.065038+0000","mapping_epoch":341,"log_start":"354'2200","ondisk_log_start":"354'2200","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:41:56.678995+0000","objects_scrubbed":0,"log_size":4512,"log_dups_size":0,"ondisk_log_size":4512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":439877632,"num_objects":6712,"num_object_clones":0,"num_object_copies":20136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6712,"num_write_kb":429568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"234'30","reported_seq":484,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.749138+0000","last_change":"2024-09-11T07:43:05.434867+0000","last_active":"2024-09-11T07:43:18.749138+0000","last_peered":"2024-09-11T07:43:18.749138+0000","last_clean":"2024-09-11T07:43:18.749138+0000","last_became_active":"2024-09-11T07:43:05.417321+0000","last_became_peered":"2024-09-11T07:43:05.417321+0000","last_unstale":"2024-09-11T07:43:18.749138+0000","last_undegraded":"2024-09-11T07:43:18.749138+0000","last_fullsized":"2024-09-11T07:43:18.749138+0000","mapping_epoch":358,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":359,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'30","last_scrub_stamp":"2024-09-11T07:42:25.300445+0000","last_deep_scrub":"94'18","last_deep_scrub_stamp":"2024-09-11T07:28:51.437397+0000","last_clean_scrub_stamp":"2024-09-11T07:42:25.300445+0000","objects_scrubbed":6,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:35.765735+0000","scrub_duration":0.001696031,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":406,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.747642+0000","last_change":"2024-09-11T07:42:10.142081+0000","last_active":"2024-09-11T07:43:18.747642+0000","last_peered":"2024-09-11T07:43:18.747642+0000","last_clean":"2024-09-11T07:43:18.747642+0000","last_became_active":"2024-09-11T07:39:57.758029+0000","last_became_peered":"2024-09-11T07:39:57.758029+0000","last_unstale":"2024-09-11T07:43:18.747642+0000","last_undegraded":"2024-09-11T07:43:18.747642+0000","last_fullsized":"2024-09-11T07:43:18.747642+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-11T07:42:10.141973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:42:10.141973+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:20.675716+0000","scrub_duration":0.001587679,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":392,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.755293+0000","last_change":"2024-09-11T07:41:32.749695+0000","last_active":"2024-09-11T07:43:19.755293+0000","last_peered":"2024-09-11T07:43:19.755293+0000","last_clean":"2024-09-11T07:43:19.755293+0000","last_became_active":"2024-09-11T07:39:57.772244+0000","last_became_peered":"2024-09-11T07:39:57.772244+0000","last_unstale":"2024-09-11T07:43:19.755293+0000","last_undegraded":"2024-09-11T07:43:19.755293+0000","last_fullsized":"2024-09-11T07:43:19.755293+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:32.749630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:16.967492+0000","last_clean_scrub_stamp":"2024-09-11T07:41:32.749630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00168944,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":99,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.742042+0000","last_change":"2024-09-11T07:42:08.449947+0000","last_active":"2024-09-11T07:43:18.742042+0000","last_peered":"2024-09-11T07:43:18.742042+0000","last_clean":"2024-09-11T07:43:18.742042+0000","last_became_active":"2024-09-11T07:39:57.688619+0000","last_became_peered":"2024-09-11T07:39:57.688619+0000","last_unstale":"2024-09-11T07:43:18.742042+0000","last_undegraded":"2024-09-11T07:43:18.742042+0000","last_fullsized":"2024-09-11T07:43:18.742042+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:08.449887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:51.187127+0000","last_clean_scrub_stamp":"2024-09-11T07:42:08.449887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:28.587711+0000","scrub_duration":0.0014126049999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":100,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.762393+0000","last_change":"2024-09-11T07:42:12.188869+0000","last_active":"2024-09-11T07:43:19.762393+0000","last_peered":"2024-09-11T07:43:19.762393+0000","last_clean":"2024-09-11T07:43:19.762393+0000","last_became_active":"2024-09-11T07:39:57.772223+0000","last_became_peered":"2024-09-11T07:39:57.772223+0000","last_unstale":"2024-09-11T07:43:19.762393+0000","last_undegraded":"2024-09-11T07:43:19.762393+0000","last_fullsized":"2024-09-11T07:43:19.762393+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:12.188785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:42:12.188785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017329800000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"365'6592","reported_seq":6624,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:23.944683+0000","last_change":"2024-09-11T07:41:57.973731+0000","last_active":"2024-09-11T07:43:23.944683+0000","last_peered":"2024-09-11T07:43:23.944683+0000","last_clean":"2024-09-11T07:43:23.944683+0000","last_became_active":"2024-09-11T07:41:57.973347+0000","last_became_peered":"2024-09-11T07:41:57.973347+0000","last_unstale":"2024-09-11T07:43:23.944683+0000","last_undegraded":"2024-09-11T07:43:23.944683+0000","last_fullsized":"2024-09-11T07:43:23.944683+0000","mapping_epoch":341,"log_start":"351'400","ondisk_log_start":"351'400","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:41:56.678995+0000","objects_scrubbed":0,"log_size":6192,"log_dups_size":0,"ondisk_log_size":6192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":432013312,"num_objects":6592,"num_object_clones":0,"num_object_copies":19776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6592,"num_write_kb":421888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"234'26","reported_seq":225,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.745221+0000","last_change":"2024-09-11T07:42:18.689554+0000","last_active":"2024-09-11T07:43:18.745221+0000","last_peered":"2024-09-11T07:43:18.745221+0000","last_clean":"2024-09-11T07:43:18.745221+0000","last_became_active":"2024-09-11T07:36:05.224704+0000","last_became_peered":"2024-09-11T07:36:05.224704+0000","last_unstale":"2024-09-11T07:43:18.745221+0000","last_undegraded":"2024-09-11T07:43:18.745221+0000","last_fullsized":"2024-09-11T07:43:18.745221+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'26","last_scrub_stamp":"2024-09-11T07:42:18.689474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:42:18.689474+0000","objects_scrubbed":7,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:35.336584+0000","scrub_duration":0.0022199279999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'3","reported_seq":409,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.754796+0000","last_change":"2024-09-11T07:43:05.245890+0000","last_active":"2024-09-11T07:43:19.754796+0000","last_peered":"2024-09-11T07:43:19.754796+0000","last_clean":"2024-09-11T07:43:19.754796+0000","last_became_active":"2024-09-11T07:40:11.701416+0000","last_became_peered":"2024-09-11T07:40:11.701416+0000","last_unstale":"2024-09-11T07:43:19.754796+0000","last_undegraded":"2024-09-11T07:43:19.754796+0000","last_fullsized":"2024-09-11T07:43:19.754796+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-09-11T07:43:05.245801+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_clean_scrub_stamp":"2024-09-11T07:43:05.245801+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:29.326580+0000","scrub_duration":0.0046496159999999996,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":417,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:22.360621+0000","last_change":"2024-09-11T07:43:22.360424+0000","last_active":"2024-09-11T07:43:22.360621+0000","last_peered":"2024-09-11T07:43:22.360621+0000","last_clean":"2024-09-11T07:43:22.360621+0000","last_became_active":"2024-09-11T07:40:11.705286+0000","last_became_peered":"2024-09-11T07:40:11.705286+0000","last_unstale":"2024-09-11T07:43:22.360621+0000","last_undegraded":"2024-09-11T07:43:22.360621+0000","last_fullsized":"2024-09-11T07:43:22.360621+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:22.360187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:43:22.360187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:43.812678+0000","scrub_duration":0.046795017000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"87'81","reported_seq":223,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.746322+0000","last_change":"2024-09-11T07:42:17.252703+0000","last_active":"2024-09-11T07:43:18.746322+0000","last_peered":"2024-09-11T07:43:18.746322+0000","last_clean":"2024-09-11T07:43:18.746322+0000","last_became_active":"2024-09-11T07:39:57.761901+0000","last_became_peered":"2024-09-11T07:39:57.761901+0000","last_unstale":"2024-09-11T07:43:18.746322+0000","last_undegraded":"2024-09-11T07:43:18.746322+0000","last_fullsized":"2024-09-11T07:43:18.746322+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:42:17.252610+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:42:17.252610+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:31.903304+0000","scrub_duration":0.0046082149999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":212,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.751783+0000","last_change":"2024-09-11T07:43:05.156258+0000","last_active":"2024-09-11T07:43:18.751783+0000","last_peered":"2024-09-11T07:43:18.751783+0000","last_clean":"2024-09-11T07:43:18.751783+0000","last_became_active":"2024-09-11T07:39:57.748658+0000","last_became_peered":"2024-09-11T07:39:57.748658+0000","last_unstale":"2024-09-11T07:43:18.751783+0000","last_undegraded":"2024-09-11T07:43:18.751783+0000","last_fullsized":"2024-09-11T07:43:18.751783+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:05.156156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:43:05.156156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:30.331262+0000","scrub_duration":0.0032562279999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":384,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.762654+0000","last_change":"2024-09-11T07:43:17.268650+0000","last_active":"2024-09-11T07:43:19.762654+0000","last_peered":"2024-09-11T07:43:19.762654+0000","last_clean":"2024-09-11T07:43:19.762654+0000","last_became_active":"2024-09-11T07:26:37.559920+0000","last_became_peered":"2024-09-11T07:26:37.559920+0000","last_unstale":"2024-09-11T07:43:19.762654+0000","last_undegraded":"2024-09-11T07:43:19.762654+0000","last_fullsized":"2024-09-11T07:43:19.762654+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:17.268558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:43:17.268558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:41.996245+0000","scrub_duration":0.043293015999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"365'6510","reported_seq":6542,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:24.069401+0000","last_change":"2024-09-11T07:41:57.699086+0000","last_active":"2024-09-11T07:43:24.069401+0000","last_peered":"2024-09-11T07:43:24.069401+0000","last_clean":"2024-09-11T07:43:24.069401+0000","last_became_active":"2024-09-11T07:41:57.698691+0000","last_became_peered":"2024-09-11T07:41:57.698691+0000","last_unstale":"2024-09-11T07:43:24.069401+0000","last_undegraded":"2024-09-11T07:43:24.069401+0000","last_fullsized":"2024-09-11T07:43:24.069401+0000","mapping_epoch":341,"log_start":"354'1952","ondisk_log_start":"354'1952","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:41:56.678995+0000","objects_scrubbed":0,"log_size":4558,"log_dups_size":0,"ondisk_log_size":4558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":426639360,"num_objects":6510,"num_object_clones":0,"num_object_copies":19530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6510,"num_write_kb":416640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"234'35","reported_seq":269,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.740013+0000","last_change":"2024-09-11T07:42:10.432178+0000","last_active":"2024-09-11T07:43:18.740013+0000","last_peered":"2024-09-11T07:43:18.740013+0000","last_clean":"2024-09-11T07:43:18.740013+0000","last_became_active":"2024-09-11T07:39:57.759856+0000","last_became_peered":"2024-09-11T07:39:57.759856+0000","last_unstale":"2024-09-11T07:43:18.740013+0000","last_undegraded":"2024-09-11T07:43:18.740013+0000","last_fullsized":"2024-09-11T07:43:18.740013+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'35","last_scrub_stamp":"2024-09-11T07:42:10.432110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:42:10.432110+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:33.461314+0000","scrub_duration":0.0024153680000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":408,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.751979+0000","last_change":"2024-09-11T07:42:22.091245+0000","last_active":"2024-09-11T07:43:18.751979+0000","last_peered":"2024-09-11T07:43:18.751979+0000","last_clean":"2024-09-11T07:43:18.751979+0000","last_became_active":"2024-09-11T07:26:31.419560+0000","last_became_peered":"2024-09-11T07:26:31.419560+0000","last_unstale":"2024-09-11T07:43:18.751979+0000","last_undegraded":"2024-09-11T07:43:18.751979+0000","last_fullsized":"2024-09-11T07:43:18.751979+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:22.091155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:42:22.091155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:43.161196+0000","scrub_duration":0.003208518,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":405,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.751282+0000","last_change":"2024-09-11T07:42:13.043015+0000","last_active":"2024-09-11T07:43:18.751282+0000","last_peered":"2024-09-11T07:43:18.751282+0000","last_clean":"2024-09-11T07:43:18.751282+0000","last_became_active":"2024-09-11T07:39:57.748953+0000","last_became_peered":"2024-09-11T07:39:57.748953+0000","last_unstale":"2024-09-11T07:43:18.751282+0000","last_undegraded":"2024-09-11T07:43:18.751282+0000","last_fullsized":"2024-09-11T07:43:18.751282+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:13.042959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:42:13.042959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013387900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":210,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.753153+0000","last_change":"2024-09-11T07:41:34.244244+0000","last_active":"2024-09-11T07:43:18.753153+0000","last_peered":"2024-09-11T07:43:18.753153+0000","last_clean":"2024-09-11T07:43:18.753153+0000","last_became_active":"2024-09-11T07:39:57.766605+0000","last_became_peered":"2024-09-11T07:39:57.766605+0000","last_unstale":"2024-09-11T07:43:18.753153+0000","last_undegraded":"2024-09-11T07:43:18.753153+0000","last_fullsized":"2024-09-11T07:43:18.753153+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:34.244143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:41:34.244143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016665180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"365'6675","reported_seq":6705,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:24.010261+0000","last_change":"2024-09-11T07:41:57.696448+0000","last_active":"2024-09-11T07:43:24.010261+0000","last_peered":"2024-09-11T07:43:24.010261+0000","last_clean":"2024-09-11T07:43:24.010261+0000","last_became_active":"2024-09-11T07:41:57.696197+0000","last_became_peered":"2024-09-11T07:41:57.696197+0000","last_unstale":"2024-09-11T07:43:24.010261+0000","last_undegraded":"2024-09-11T07:43:24.010261+0000","last_fullsized":"2024-09-11T07:43:24.010261+0000","mapping_epoch":341,"log_start":"353'1233","ondisk_log_start":"353'1233","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:41:56.678995+0000","objects_scrubbed":0,"log_size":5442,"log_dups_size":0,"ondisk_log_size":5442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437452800,"num_objects":6675,"num_object_clones":0,"num_object_copies":20025,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6675,"num_write_kb":427200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"234'15","reported_seq":409,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.748205+0000","last_change":"2024-09-11T07:42:24.281479+0000","last_active":"2024-09-11T07:43:18.748205+0000","last_peered":"2024-09-11T07:43:18.748205+0000","last_clean":"2024-09-11T07:43:18.748205+0000","last_became_active":"2024-09-11T07:26:33.543148+0000","last_became_peered":"2024-09-11T07:26:33.543148+0000","last_unstale":"2024-09-11T07:43:18.748205+0000","last_undegraded":"2024-09-11T07:43:18.748205+0000","last_fullsized":"2024-09-11T07:43:18.748205+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:42:24.281381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:42:24.281381+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:45.573959+0000","scrub_duration":0.0060122939999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":395,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.742492+0000","last_change":"2024-09-11T07:43:16.532652+0000","last_active":"2024-09-11T07:43:18.742492+0000","last_peered":"2024-09-11T07:43:18.742492+0000","last_clean":"2024-09-11T07:43:18.742492+0000","last_became_active":"2024-09-11T07:26:31.423056+0000","last_became_peered":"2024-09-11T07:26:31.423056+0000","last_unstale":"2024-09-11T07:43:18.742492+0000","last_undegraded":"2024-09-11T07:43:18.742492+0000","last_fullsized":"2024-09-11T07:43:18.742492+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:16.532601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:43:16.532601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:17.345657+0000","scrub_duration":0.0012610080000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":401,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.751498+0000","last_change":"2024-09-11T07:42:05.151258+0000","last_active":"2024-09-11T07:43:18.751498+0000","last_peered":"2024-09-11T07:43:18.751498+0000","last_clean":"2024-09-11T07:43:18.751498+0000","last_became_active":"2024-09-11T07:26:35.563182+0000","last_became_peered":"2024-09-11T07:26:35.563182+0000","last_unstale":"2024-09-11T07:43:18.751498+0000","last_undegraded":"2024-09-11T07:43:18.751498+0000","last_fullsized":"2024-09-11T07:43:18.751498+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:05.151200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:42:05.151200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016491069999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":397,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.747566+0000","last_change":"2024-09-11T07:43:16.343875+0000","last_active":"2024-09-11T07:43:18.747566+0000","last_peered":"2024-09-11T07:43:18.747566+0000","last_clean":"2024-09-11T07:43:18.747566+0000","last_became_active":"2024-09-11T07:32:00.505185+0000","last_became_peered":"2024-09-11T07:32:00.505185+0000","last_unstale":"2024-09-11T07:43:18.747566+0000","last_undegraded":"2024-09-11T07:43:18.747566+0000","last_fullsized":"2024-09-11T07:43:18.747566+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:16.343786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:43:16.343786+0000","last_clean_scrub_stamp":"2024-09-11T07:43:16.343786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:20.217616+0000","scrub_duration":0.083487128999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"365'6599","reported_seq":6630,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:24.072346+0000","last_change":"2024-09-11T07:42:04.729255+0000","last_active":"2024-09-11T07:43:24.072346+0000","last_peered":"2024-09-11T07:43:24.072346+0000","last_clean":"2024-09-11T07:43:24.072346+0000","last_became_active":"2024-09-11T07:42:04.728945+0000","last_became_peered":"2024-09-11T07:42:04.728945+0000","last_unstale":"2024-09-11T07:43:24.072346+0000","last_undegraded":"2024-09-11T07:43:24.072346+0000","last_fullsized":"2024-09-11T07:43:24.072346+0000","mapping_epoch":346,"log_start":"354'2100","ondisk_log_start":"354'2100","created":341,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:41:56.678995+0000","objects_scrubbed":0,"log_size":4499,"log_dups_size":0,"ondisk_log_size":4499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":432472064,"num_objects":6599,"num_object_clones":0,"num_object_copies":19797,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6599,"num_write_kb":422336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4],"acting":[0,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"234'27","reported_seq":129,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:22.301061+0000","last_change":"2024-09-11T07:43:22.301033+0000","last_active":"2024-09-11T07:43:22.301061+0000","last_peered":"2024-09-11T07:43:22.301061+0000","last_clean":"2024-09-11T07:43:22.301061+0000","last_became_active":"2024-09-11T07:39:57.766550+0000","last_became_peered":"2024-09-11T07:39:57.766550+0000","last_unstale":"2024-09-11T07:43:22.301061+0000","last_undegraded":"2024-09-11T07:43:22.301061+0000","last_fullsized":"2024-09-11T07:43:22.301061+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'27","last_scrub_stamp":"2024-09-11T07:43:22.300979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:43:22.300979+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:35.812123+0000","scrub_duration":0.0017609329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":443,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.752593+0000","last_change":"2024-09-11T07:42:12.069079+0000","last_active":"2024-09-11T07:43:18.752593+0000","last_peered":"2024-09-11T07:43:18.752593+0000","last_clean":"2024-09-11T07:43:18.752593+0000","last_became_active":"2024-09-11T07:40:11.701330+0000","last_became_peered":"2024-09-11T07:40:11.701330+0000","last_unstale":"2024-09-11T07:43:18.752593+0000","last_undegraded":"2024-09-11T07:43:18.752593+0000","last_fullsized":"2024-09-11T07:43:18.752593+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:12.069024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_clean_scrub_stamp":"2024-09-11T07:42:12.069024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:23.621610+0000","scrub_duration":0.001696812,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"151'10","reported_seq":554,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:21.597231+0000","last_change":"2024-09-11T07:43:21.597166+0000","last_active":"2024-09-11T07:43:21.597231+0000","last_peered":"2024-09-11T07:43:21.597231+0000","last_clean":"2024-09-11T07:43:21.597231+0000","last_became_active":"2024-09-11T07:32:00.506806+0000","last_became_peered":"2024-09-11T07:32:00.506806+0000","last_unstale":"2024-09-11T07:43:21.597231+0000","last_undegraded":"2024-09-11T07:43:21.597231+0000","last_fullsized":"2024-09-11T07:43:21.597231+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:43:21.597088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:43:21.597088+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:45.690910+0000","scrub_duration":0.0079746980000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":41,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.747199+0000","last_change":"2024-09-11T07:43:17.286114+0000","last_active":"2024-09-11T07:43:18.747199+0000","last_peered":"2024-09-11T07:43:18.747199+0000","last_clean":"2024-09-11T07:43:18.747199+0000","last_became_active":"2024-09-11T07:42:04.729103+0000","last_became_peered":"2024-09-11T07:42:04.729103+0000","last_unstale":"2024-09-11T07:43:18.747199+0000","last_undegraded":"2024-09-11T07:43:18.747199+0000","last_fullsized":"2024-09-11T07:43:18.747199+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:17.285919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:43:17.285919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:26.472883+0000","scrub_duration":0.010493728000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"365'6536","reported_seq":6568,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:24.027910+0000","last_change":"2024-09-11T07:41:57.974511+0000","last_active":"2024-09-11T07:43:24.027910+0000","last_peered":"2024-09-11T07:43:24.027910+0000","last_clean":"2024-09-11T07:43:24.027910+0000","last_became_active":"2024-09-11T07:41:57.974221+0000","last_became_peered":"2024-09-11T07:41:57.974221+0000","last_unstale":"2024-09-11T07:43:24.027910+0000","last_undegraded":"2024-09-11T07:43:24.027910+0000","last_fullsized":"2024-09-11T07:43:24.027910+0000","mapping_epoch":341,"log_start":"352'1085","ondisk_log_start":"352'1085","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:41:56.678995+0000","objects_scrubbed":0,"log_size":5451,"log_dups_size":0,"ondisk_log_size":5451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:24.525870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":428343296,"num_objects":6536,"num_object_clones":0,"num_object_copies":19608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6536,"num_write_kb":418304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"234'23","reported_seq":443,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.745255+0000","last_change":"2024-09-11T07:43:15.318622+0000","last_active":"2024-09-11T07:43:18.745255+0000","last_peered":"2024-09-11T07:43:18.745255+0000","last_clean":"2024-09-11T07:43:18.745255+0000","last_became_active":"2024-09-11T07:39:57.691713+0000","last_became_peered":"2024-09-11T07:39:57.691713+0000","last_unstale":"2024-09-11T07:43:18.745255+0000","last_undegraded":"2024-09-11T07:43:18.745255+0000","last_fullsized":"2024-09-11T07:43:18.745255+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'23","last_scrub_stamp":"2024-09-11T07:43:15.318542+0000","last_deep_scrub":"234'23","last_deep_scrub_stamp":"2024-09-11T07:37:34.613280+0000","last_clean_scrub_stamp":"2024-09-11T07:43:15.318542+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:29.657765+0000","scrub_duration":0.0097875570000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":409,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.746603+0000","last_change":"2024-09-11T07:41:16.742565+0000","last_active":"2024-09-11T07:43:18.746603+0000","last_peered":"2024-09-11T07:43:18.746603+0000","last_clean":"2024-09-11T07:43:18.746603+0000","last_became_active":"2024-09-11T07:32:00.504344+0000","last_became_peered":"2024-09-11T07:32:00.504344+0000","last_unstale":"2024-09-11T07:43:18.746603+0000","last_undegraded":"2024-09-11T07:43:18.746603+0000","last_fullsized":"2024-09-11T07:43:18.746603+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:41:16.742481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:41:16.742481+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043896899999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":401,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.753162+0000","last_change":"2024-09-11T07:42:08.200025+0000","last_active":"2024-09-11T07:43:18.753162+0000","last_peered":"2024-09-11T07:43:18.753162+0000","last_clean":"2024-09-11T07:43:18.753162+0000","last_became_active":"2024-09-11T07:26:35.561967+0000","last_became_peered":"2024-09-11T07:26:35.561967+0000","last_unstale":"2024-09-11T07:43:18.753162+0000","last_undegraded":"2024-09-11T07:43:18.753162+0000","last_fullsized":"2024-09-11T07:43:18.753162+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:08.199968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:44.820354+0000","last_clean_scrub_stamp":"2024-09-11T07:42:08.199968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:29.465973+0000","scrub_duration":0.0013036600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":408,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.755614+0000","last_change":"2024-09-11T07:43:16.417085+0000","last_active":"2024-09-11T07:43:19.755614+0000","last_peered":"2024-09-11T07:43:19.755614+0000","last_clean":"2024-09-11T07:43:19.755614+0000","last_became_active":"2024-09-11T07:39:57.776576+0000","last_became_peered":"2024-09-11T07:39:57.776576+0000","last_unstale":"2024-09-11T07:43:19.755614+0000","last_undegraded":"2024-09-11T07:43:19.755614+0000","last_fullsized":"2024-09-11T07:43:19.755614+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:16.416995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:43:16.416995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:23.781897+0000","scrub_duration":0.0015454920000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"234'15","reported_seq":446,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.749059+0000","last_change":"2024-09-11T07:42:31.480040+0000","last_active":"2024-09-11T07:43:18.749059+0000","last_peered":"2024-09-11T07:43:18.749059+0000","last_clean":"2024-09-11T07:43:18.749059+0000","last_became_active":"2024-09-11T07:39:57.767661+0000","last_became_peered":"2024-09-11T07:39:57.767661+0000","last_unstale":"2024-09-11T07:43:18.749059+0000","last_undegraded":"2024-09-11T07:43:18.749059+0000","last_fullsized":"2024-09-11T07:43:18.749059+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:42:31.479941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:42:31.479941+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:37.673995+0000","scrub_duration":0.004151208,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":405,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.753304+0000","last_change":"2024-09-11T07:41:40.270685+0000","last_active":"2024-09-11T07:43:18.753304+0000","last_peered":"2024-09-11T07:43:18.753304+0000","last_clean":"2024-09-11T07:43:18.753304+0000","last_became_active":"2024-09-11T07:26:31.415701+0000","last_became_peered":"2024-09-11T07:26:31.415701+0000","last_unstale":"2024-09-11T07:43:18.753304+0000","last_undegraded":"2024-09-11T07:43:18.753304+0000","last_fullsized":"2024-09-11T07:43:18.753304+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:40.270638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:40.270638+0000","last_clean_scrub_stamp":"2024-09-11T07:41:40.270638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020516589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":225,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.741963+0000","last_change":"2024-09-11T07:42:28.471359+0000","last_active":"2024-09-11T07:43:18.741963+0000","last_peered":"2024-09-11T07:43:18.741963+0000","last_clean":"2024-09-11T07:43:18.741963+0000","last_became_active":"2024-09-11T07:40:11.711154+0000","last_became_peered":"2024-09-11T07:40:11.711154+0000","last_unstale":"2024-09-11T07:43:18.741963+0000","last_undegraded":"2024-09-11T07:43:18.741963+0000","last_fullsized":"2024-09-11T07:43:18.741963+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:28.471265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:35.392360+0000","last_clean_scrub_stamp":"2024-09-11T07:42:28.471265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:34.294025+0000","scrub_duration":0.0030406719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":408,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.751199+0000","last_change":"2024-09-11T07:43:16.338643+0000","last_active":"2024-09-11T07:43:18.751199+0000","last_peered":"2024-09-11T07:43:18.751199+0000","last_clean":"2024-09-11T07:43:18.751199+0000","last_became_active":"2024-09-11T07:32:00.506697+0000","last_became_peered":"2024-09-11T07:32:00.506697+0000","last_unstale":"2024-09-11T07:43:18.751199+0000","last_undegraded":"2024-09-11T07:43:18.751199+0000","last_fullsized":"2024-09-11T07:43:18.751199+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:43:16.338554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:43:16.338554+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:18.265724+0000","scrub_duration":0.002991315,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":176,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.745404+0000","last_change":"2024-09-11T07:41:35.646908+0000","last_active":"2024-09-11T07:43:18.745404+0000","last_peered":"2024-09-11T07:43:18.745404+0000","last_clean":"2024-09-11T07:43:18.745404+0000","last_became_active":"2024-09-11T07:36:05.225791+0000","last_became_peered":"2024-09-11T07:36:05.225791+0000","last_unstale":"2024-09-11T07:43:18.745404+0000","last_undegraded":"2024-09-11T07:43:18.745404+0000","last_fullsized":"2024-09-11T07:43:18.745404+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:35.646855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:41:35.646855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001374842,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"234'15","reported_seq":158,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.742123+0000","last_change":"2024-09-11T07:43:09.477665+0000","last_active":"2024-09-11T07:43:18.742123+0000","last_peered":"2024-09-11T07:43:18.742123+0000","last_clean":"2024-09-11T07:43:18.742123+0000","last_became_active":"2024-09-11T07:39:57.691830+0000","last_became_peered":"2024-09-11T07:39:57.691830+0000","last_unstale":"2024-09-11T07:43:18.742123+0000","last_undegraded":"2024-09-11T07:43:18.742123+0000","last_fullsized":"2024-09-11T07:43:18.742123+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:43:09.477571+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:43:09.477571+0000","objects_scrubbed":3,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:16.078234+0000","scrub_duration":0.0035276859999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":393,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.762633+0000","last_change":"2024-09-11T07:42:26.151088+0000","last_active":"2024-09-11T07:43:19.762633+0000","last_peered":"2024-09-11T07:43:19.762633+0000","last_clean":"2024-09-11T07:43:19.762633+0000","last_became_active":"2024-09-11T07:26:35.566894+0000","last_became_peered":"2024-09-11T07:26:35.566894+0000","last_unstale":"2024-09-11T07:43:19.762633+0000","last_undegraded":"2024-09-11T07:43:19.762633+0000","last_fullsized":"2024-09-11T07:43:19.762633+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:26.151015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:42:26.151015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:28.535761+0000","scrub_duration":0.0046939240000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":379,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.755698+0000","last_change":"2024-09-11T07:42:32.822795+0000","last_active":"2024-09-11T07:43:19.755698+0000","last_peered":"2024-09-11T07:43:19.755698+0000","last_clean":"2024-09-11T07:43:19.755698+0000","last_became_active":"2024-09-11T07:32:00.514100+0000","last_became_peered":"2024-09-11T07:32:00.514100+0000","last_unstale":"2024-09-11T07:43:19.755698+0000","last_undegraded":"2024-09-11T07:43:19.755698+0000","last_fullsized":"2024-09-11T07:43:19.755698+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:32.822745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:42:32.822745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:37.966679+0000","scrub_duration":0.001459213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"234'25","reported_seq":459,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.755320+0000","last_change":"2024-09-11T07:43:11.344672+0000","last_active":"2024-09-11T07:43:19.755320+0000","last_peered":"2024-09-11T07:43:19.755320+0000","last_clean":"2024-09-11T07:43:19.755320+0000","last_became_active":"2024-09-11T07:39:57.771009+0000","last_became_peered":"2024-09-11T07:39:57.771009+0000","last_unstale":"2024-09-11T07:43:19.755320+0000","last_undegraded":"2024-09-11T07:43:19.755320+0000","last_fullsized":"2024-09-11T07:43:19.755320+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'25","last_scrub_stamp":"2024-09-11T07:43:11.344584+0000","last_deep_scrub":"86'15","last_deep_scrub_stamp":"2024-09-11T07:28:22.858137+0000","last_clean_scrub_stamp":"2024-09-11T07:43:11.344584+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:24.084550+0000","scrub_duration":0.0061577919999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":407,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.762159+0000","last_change":"2024-09-11T07:42:20.158863+0000","last_active":"2024-09-11T07:43:19.762159+0000","last_peered":"2024-09-11T07:43:19.762159+0000","last_clean":"2024-09-11T07:43:19.762159+0000","last_became_active":"2024-09-11T07:39:57.764063+0000","last_became_peered":"2024-09-11T07:39:57.764063+0000","last_unstale":"2024-09-11T07:43:19.762159+0000","last_undegraded":"2024-09-11T07:43:19.762159+0000","last_fullsized":"2024-09-11T07:43:19.762159+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:20.158768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:39:54.493614+0000","last_clean_scrub_stamp":"2024-09-11T07:42:20.158768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:25.741985+0000","scrub_duration":0.003468259,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":593,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.762307+0000","last_change":"2024-09-11T07:42:27.156341+0000","last_active":"2024-09-11T07:43:19.762307+0000","last_peered":"2024-09-11T07:43:19.762307+0000","last_clean":"2024-09-11T07:43:19.762307+0000","last_became_active":"2024-09-11T07:26:35.562245+0000","last_became_peered":"2024-09-11T07:26:35.562245+0000","last_unstale":"2024-09-11T07:43:19.762307+0000","last_undegraded":"2024-09-11T07:43:19.762307+0000","last_fullsized":"2024-09-11T07:43:19.762307+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:42:27.156250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:42:27.156250+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:39.667484+0000","scrub_duration":0.0030775300000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":407,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.746658+0000","last_change":"2024-09-11T07:42:08.141912+0000","last_active":"2024-09-11T07:43:18.746658+0000","last_peered":"2024-09-11T07:43:18.746658+0000","last_clean":"2024-09-11T07:43:18.746658+0000","last_became_active":"2024-09-11T07:39:57.758650+0000","last_became_peered":"2024-09-11T07:39:57.758650+0000","last_unstale":"2024-09-11T07:43:18.746658+0000","last_undegraded":"2024-09-11T07:43:18.746658+0000","last_fullsized":"2024-09-11T07:43:18.746658+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:08.141858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:42:08.141858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:23.524475+0000","scrub_duration":0.001432214,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"234'19","reported_seq":401,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.755609+0000","last_change":"2024-09-11T07:43:10.499834+0000","last_active":"2024-09-11T07:43:19.755609+0000","last_peered":"2024-09-11T07:43:19.755609+0000","last_clean":"2024-09-11T07:43:19.755609+0000","last_became_active":"2024-09-11T07:43:10.448494+0000","last_became_peered":"2024-09-11T07:43:10.448494+0000","last_unstale":"2024-09-11T07:43:19.755609+0000","last_undegraded":"2024-09-11T07:43:19.755609+0000","last_fullsized":"2024-09-11T07:43:19.755609+0000","mapping_epoch":360,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":361,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:41:38.707118+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:37:41.338036+0000","last_clean_scrub_stamp":"2024-09-11T07:41:38.707118+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021100030000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":389,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.748393+0000","last_change":"2024-09-11T07:42:30.415501+0000","last_active":"2024-09-11T07:43:18.748393+0000","last_peered":"2024-09-11T07:43:18.748393+0000","last_clean":"2024-09-11T07:43:18.748393+0000","last_became_active":"2024-09-11T07:26:31.419409+0000","last_became_peered":"2024-09-11T07:26:31.419409+0000","last_unstale":"2024-09-11T07:43:18.748393+0000","last_undegraded":"2024-09-11T07:43:18.748393+0000","last_fullsized":"2024-09-11T07:43:18.748393+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:30.415425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:42:30.415425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:42.560763+0000","scrub_duration":0.0061185789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"290'11","reported_seq":335,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.755648+0000","last_change":"2024-09-11T07:42:05.858328+0000","last_active":"2024-09-11T07:43:19.755648+0000","last_peered":"2024-09-11T07:43:19.755648+0000","last_clean":"2024-09-11T07:43:19.755648+0000","last_became_active":"2024-09-11T07:39:57.772790+0000","last_became_peered":"2024-09-11T07:39:57.772790+0000","last_unstale":"2024-09-11T07:43:19.755648+0000","last_undegraded":"2024-09-11T07:43:19.755648+0000","last_fullsized":"2024-09-11T07:43:19.755648+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"290'11","last_scrub_stamp":"2024-09-11T07:42:05.858268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:42:05.858268+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022026849999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":373,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.746116+0000","last_change":"2024-09-11T07:42:05.675208+0000","last_active":"2024-09-11T07:43:18.746116+0000","last_peered":"2024-09-11T07:43:18.746116+0000","last_clean":"2024-09-11T07:43:18.746116+0000","last_became_active":"2024-09-11T07:26:37.565723+0000","last_became_peered":"2024-09-11T07:26:37.565723+0000","last_unstale":"2024-09-11T07:43:18.746116+0000","last_undegraded":"2024-09-11T07:43:18.746116+0000","last_fullsized":"2024-09-11T07:43:18.746116+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:05.675136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:42:05.675136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:23.277312+0000","scrub_duration":0.0014991150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"236'16","reported_seq":388,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.751415+0000","last_change":"2024-09-11T07:43:09.251084+0000","last_active":"2024-09-11T07:43:18.751415+0000","last_peered":"2024-09-11T07:43:18.751415+0000","last_clean":"2024-09-11T07:43:18.751415+0000","last_became_active":"2024-09-11T07:39:57.689424+0000","last_became_peered":"2024-09-11T07:39:57.689424+0000","last_unstale":"2024-09-11T07:43:18.751415+0000","last_undegraded":"2024-09-11T07:43:18.751415+0000","last_fullsized":"2024-09-11T07:43:18.751415+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'16","last_scrub_stamp":"2024-09-11T07:43:09.250993+0000","last_deep_scrub":"236'16","last_deep_scrub_stamp":"2024-09-11T07:36:49.809484+0000","last_clean_scrub_stamp":"2024-09-11T07:43:09.250993+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:12.401780+0000","scrub_duration":0.013487453999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":193,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.755391+0000","last_change":"2024-09-11T07:43:15.385301+0000","last_active":"2024-09-11T07:43:19.755391+0000","last_peered":"2024-09-11T07:43:19.755391+0000","last_clean":"2024-09-11T07:43:19.755391+0000","last_became_active":"2024-09-11T07:39:57.776841+0000","last_became_peered":"2024-09-11T07:39:57.776841+0000","last_unstale":"2024-09-11T07:43:19.755391+0000","last_undegraded":"2024-09-11T07:43:19.755391+0000","last_fullsized":"2024-09-11T07:43:19.755391+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:15.385106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:38.721118+0000","last_clean_scrub_stamp":"2024-09-11T07:43:15.385106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:34.858608+0000","scrub_duration":0.0024763319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":379,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.755088+0000","last_change":"2024-09-11T07:42:29.915287+0000","last_active":"2024-09-11T07:43:19.755088+0000","last_peered":"2024-09-11T07:43:19.755088+0000","last_clean":"2024-09-11T07:43:19.755088+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:43:19.755088+0000","last_undegraded":"2024-09-11T07:43:19.755088+0000","last_fullsized":"2024-09-11T07:43:19.755088+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:29.915237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:42:29.915237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:53.778315+0000","scrub_duration":0.043343591000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":400,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.745754+0000","last_change":"2024-09-11T07:43:18.311031+0000","last_active":"2024-09-11T07:43:18.745754+0000","last_peered":"2024-09-11T07:43:18.745754+0000","last_clean":"2024-09-11T07:43:18.745754+0000","last_became_active":"2024-09-11T07:39:57.761754+0000","last_became_peered":"2024-09-11T07:39:57.761754+0000","last_unstale":"2024-09-11T07:43:18.745754+0000","last_undegraded":"2024-09-11T07:43:18.745754+0000","last_fullsized":"2024-09-11T07:43:18.745754+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:18.310936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:43:18.310936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:41.754185+0000","scrub_duration":0.0094815590000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"234'24","reported_seq":413,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.744425+0000","last_change":"2024-09-11T07:41:32.694601+0000","last_active":"2024-09-11T07:43:18.744425+0000","last_peered":"2024-09-11T07:43:18.744425+0000","last_clean":"2024-09-11T07:43:18.744425+0000","last_became_active":"2024-09-11T07:26:33.540208+0000","last_became_peered":"2024-09-11T07:26:33.540208+0000","last_unstale":"2024-09-11T07:43:18.744425+0000","last_undegraded":"2024-09-11T07:43:18.744425+0000","last_fullsized":"2024-09-11T07:43:18.744425+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'24","last_scrub_stamp":"2024-09-11T07:41:32.694510+0000","last_deep_scrub":"234'24","last_deep_scrub_stamp":"2024-09-11T07:41:32.694510+0000","last_clean_scrub_stamp":"2024-09-11T07:41:32.694510+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021937689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":388,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.742476+0000","last_change":"2024-09-11T07:42:18.471016+0000","last_active":"2024-09-11T07:43:18.742476+0000","last_peered":"2024-09-11T07:43:18.742476+0000","last_clean":"2024-09-11T07:43:18.742476+0000","last_became_active":"2024-09-11T07:26:31.422671+0000","last_became_peered":"2024-09-11T07:26:31.422671+0000","last_unstale":"2024-09-11T07:43:18.742476+0000","last_undegraded":"2024-09-11T07:43:18.742476+0000","last_fullsized":"2024-09-11T07:43:18.742476+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:18.470914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:42:18.470914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:30.125116+0000","scrub_duration":0.0018254269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":383,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.739924+0000","last_change":"2024-09-11T07:42:19.496604+0000","last_active":"2024-09-11T07:43:18.739924+0000","last_peered":"2024-09-11T07:43:18.739924+0000","last_clean":"2024-09-11T07:43:18.739924+0000","last_became_active":"2024-09-11T07:39:57.759390+0000","last_became_peered":"2024-09-11T07:39:57.759390+0000","last_unstale":"2024-09-11T07:43:18.739924+0000","last_undegraded":"2024-09-11T07:43:18.739924+0000","last_fullsized":"2024-09-11T07:43:18.739924+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:19.496507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:39:55.895853+0000","last_clean_scrub_stamp":"2024-09-11T07:42:19.496507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:25.800447+0000","scrub_duration":0.0021717680000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":119,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.747353+0000","last_change":"2024-09-11T07:43:06.316844+0000","last_active":"2024-09-11T07:43:18.747353+0000","last_peered":"2024-09-11T07:43:18.747353+0000","last_clean":"2024-09-11T07:43:18.747353+0000","last_became_active":"2024-09-11T07:40:11.700676+0000","last_became_peered":"2024-09-11T07:40:11.700676+0000","last_unstale":"2024-09-11T07:43:18.747353+0000","last_undegraded":"2024-09-11T07:43:18.747353+0000","last_fullsized":"2024-09-11T07:43:18.747353+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:06.316756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:43:06.316756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:23.228212+0000","scrub_duration":0.0096085179999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"234'10","reported_seq":399,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.754796+0000","last_change":"2024-09-11T07:43:10.326858+0000","last_active":"2024-09-11T07:43:19.754796+0000","last_peered":"2024-09-11T07:43:19.754796+0000","last_clean":"2024-09-11T07:43:19.754796+0000","last_became_active":"2024-09-11T07:26:33.545830+0000","last_became_peered":"2024-09-11T07:26:33.545830+0000","last_unstale":"2024-09-11T07:43:19.754796+0000","last_undegraded":"2024-09-11T07:43:19.754796+0000","last_fullsized":"2024-09-11T07:43:19.754796+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'10","last_scrub_stamp":"2024-09-11T07:43:10.326752+0000","last_deep_scrub":"234'10","last_deep_scrub_stamp":"2024-09-11T07:41:29.585284+0000","last_clean_scrub_stamp":"2024-09-11T07:43:10.326752+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:12.662755+0000","scrub_duration":0.0083240369999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":387,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.762693+0000","last_change":"2024-09-11T07:42:15.174369+0000","last_active":"2024-09-11T07:43:19.762693+0000","last_peered":"2024-09-11T07:43:19.762693+0000","last_clean":"2024-09-11T07:43:19.762693+0000","last_became_active":"2024-09-11T07:26:31.420518+0000","last_became_peered":"2024-09-11T07:26:31.420518+0000","last_unstale":"2024-09-11T07:43:19.762693+0000","last_undegraded":"2024-09-11T07:43:19.762693+0000","last_fullsized":"2024-09-11T07:43:19.762693+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:15.174300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_clean_scrub_stamp":"2024-09-11T07:42:15.174300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014157970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":317,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.743884+0000","last_change":"2024-09-11T07:42:24.675003+0000","last_active":"2024-09-11T07:43:18.743884+0000","last_peered":"2024-09-11T07:43:18.743884+0000","last_clean":"2024-09-11T07:43:18.743884+0000","last_became_active":"2024-09-11T07:40:11.699702+0000","last_became_peered":"2024-09-11T07:40:11.699702+0000","last_unstale":"2024-09-11T07:43:18.743884+0000","last_undegraded":"2024-09-11T07:43:18.743884+0000","last_fullsized":"2024-09-11T07:43:18.743884+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:24.674952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:47.675050+0000","last_clean_scrub_stamp":"2024-09-11T07:42:24.674952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:42.817227+0000","scrub_duration":0.001252188,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":381,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.755300+0000","last_change":"2024-09-11T07:42:16.804633+0000","last_active":"2024-09-11T07:43:19.755300+0000","last_peered":"2024-09-11T07:43:19.755300+0000","last_clean":"2024-09-11T07:43:19.755300+0000","last_became_active":"2024-09-11T07:32:00.511407+0000","last_became_peered":"2024-09-11T07:32:00.511407+0000","last_unstale":"2024-09-11T07:43:19.755300+0000","last_undegraded":"2024-09-11T07:43:19.755300+0000","last_fullsized":"2024-09-11T07:43:19.755300+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:16.804545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:42:16.804545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004611403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"234'19","reported_seq":435,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.758740+0000","last_change":"2024-09-11T07:42:19.993500+0000","last_active":"2024-09-11T07:43:19.758740+0000","last_peered":"2024-09-11T07:43:19.758740+0000","last_clean":"2024-09-11T07:43:19.758740+0000","last_became_active":"2024-09-11T07:39:57.767928+0000","last_became_peered":"2024-09-11T07:39:57.767928+0000","last_unstale":"2024-09-11T07:43:19.758740+0000","last_undegraded":"2024-09-11T07:43:19.758740+0000","last_fullsized":"2024-09-11T07:43:19.758740+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:42:19.993390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:42:19.993390+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:37.626592+0000","scrub_duration":0.002852528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":173,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.762844+0000","last_change":"2024-09-11T07:42:13.160716+0000","last_active":"2024-09-11T07:43:19.762844+0000","last_peered":"2024-09-11T07:43:19.762844+0000","last_clean":"2024-09-11T07:43:19.762844+0000","last_became_active":"2024-09-11T07:36:05.228360+0000","last_became_peered":"2024-09-11T07:36:05.228360+0000","last_unstale":"2024-09-11T07:43:19.762844+0000","last_undegraded":"2024-09-11T07:43:19.762844+0000","last_fullsized":"2024-09-11T07:43:19.762844+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:13.160652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:48.366931+0000","last_clean_scrub_stamp":"2024-09-11T07:42:13.160652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:36.932041+0000","scrub_duration":0.001446506,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":354,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.755186+0000","last_change":"2024-09-11T07:42:23.794853+0000","last_active":"2024-09-11T07:43:19.755186+0000","last_peered":"2024-09-11T07:43:19.755186+0000","last_clean":"2024-09-11T07:43:19.755186+0000","last_became_active":"2024-09-11T07:32:00.520534+0000","last_became_peered":"2024-09-11T07:32:00.520534+0000","last_unstale":"2024-09-11T07:43:19.755186+0000","last_undegraded":"2024-09-11T07:43:19.755186+0000","last_fullsized":"2024-09-11T07:43:19.755186+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:23.794780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:39.021876+0000","last_clean_scrub_stamp":"2024-09-11T07:42:23.794780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:29.264407+0000","scrub_duration":0.0029093309999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":371,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.754992+0000","last_change":"2024-09-11T07:42:32.904092+0000","last_active":"2024-09-11T07:43:19.754992+0000","last_peered":"2024-09-11T07:43:19.754992+0000","last_clean":"2024-09-11T07:43:19.754992+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:43:19.754992+0000","last_undegraded":"2024-09-11T07:43:19.754992+0000","last_fullsized":"2024-09-11T07:43:19.754992+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:32.904006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:32.904006+0000","last_clean_scrub_stamp":"2024-09-11T07:42:32.904006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:40.824608+0000","scrub_duration":0.0017632870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":361,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.755496+0000","last_change":"2024-09-11T07:43:07.677815+0000","last_active":"2024-09-11T07:43:19.755496+0000","last_peered":"2024-09-11T07:43:19.755496+0000","last_clean":"2024-09-11T07:43:19.755496+0000","last_became_active":"2024-09-11T07:32:00.511258+0000","last_became_peered":"2024-09-11T07:32:00.511258+0000","last_unstale":"2024-09-11T07:43:19.755496+0000","last_undegraded":"2024-09-11T07:43:19.755496+0000","last_fullsized":"2024-09-11T07:43:19.755496+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:43:07.675565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:43:07.675565+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:34.471182+0000","scrub_duration":0.012571676,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":423,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.761603+0000","last_change":"2024-09-11T07:43:19.305329+0000","last_active":"2024-09-11T07:43:19.761603+0000","last_peered":"2024-09-11T07:43:19.761603+0000","last_clean":"2024-09-11T07:43:19.761603+0000","last_became_active":"2024-09-11T07:39:57.696805+0000","last_became_peered":"2024-09-11T07:39:57.696805+0000","last_unstale":"2024-09-11T07:43:19.761603+0000","last_undegraded":"2024-09-11T07:43:19.761603+0000","last_fullsized":"2024-09-11T07:43:19.761603+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:43:19.305239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:43:19.305239+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:21.828558+0000","scrub_duration":0.0020446459999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"234'20","reported_seq":217,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.746349+0000","last_change":"2024-09-11T07:42:12.618989+0000","last_active":"2024-09-11T07:43:18.746349+0000","last_peered":"2024-09-11T07:43:18.746349+0000","last_clean":"2024-09-11T07:43:18.746349+0000","last_became_active":"2024-09-11T07:35:00.561615+0000","last_became_peered":"2024-09-11T07:35:00.561615+0000","last_unstale":"2024-09-11T07:43:18.746349+0000","last_undegraded":"2024-09-11T07:43:18.746349+0000","last_fullsized":"2024-09-11T07:43:18.746349+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'20","last_scrub_stamp":"2024-09-11T07:42:12.618921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:42:12.618921+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:29.354325+0000","scrub_duration":0.002152784,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1f","version":"290'12","reported_seq":298,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.712054+0000","last_change":"2024-09-11T07:42:23.666873+0000","last_active":"2024-09-11T07:43:19.712054+0000","last_peered":"2024-09-11T07:43:19.712054+0000","last_clean":"2024-09-11T07:43:19.712054+0000","last_became_active":"2024-09-11T07:39:57.687809+0000","last_became_peered":"2024-09-11T07:39:57.687809+0000","last_unstale":"2024-09-11T07:43:19.712054+0000","last_undegraded":"2024-09-11T07:43:19.712054+0000","last_fullsized":"2024-09-11T07:43:19.712054+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"290'12","last_scrub_stamp":"2024-09-11T07:42:23.666786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:42:23.666786+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:26.005785+0000","scrub_duration":0.0043065259999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":379,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.742338+0000","last_change":"2024-09-11T07:42:11.460740+0000","last_active":"2024-09-11T07:43:18.742338+0000","last_peered":"2024-09-11T07:43:18.742338+0000","last_clean":"2024-09-11T07:43:18.742338+0000","last_became_active":"2024-09-11T07:26:37.562501+0000","last_became_peered":"2024-09-11T07:26:37.562501+0000","last_unstale":"2024-09-11T07:43:18.742338+0000","last_undegraded":"2024-09-11T07:43:18.742338+0000","last_fullsized":"2024-09-11T07:43:18.742338+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:11.460677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:53.875300+0000","last_clean_scrub_stamp":"2024-09-11T07:42:11.460677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:31.119910+0000","scrub_duration":0.0015970139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":390,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.758973+0000","last_change":"2024-09-11T07:42:25.918593+0000","last_active":"2024-09-11T07:43:19.758973+0000","last_peered":"2024-09-11T07:43:19.758973+0000","last_clean":"2024-09-11T07:43:19.758973+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:43:19.758973+0000","last_undegraded":"2024-09-11T07:43:19.758973+0000","last_fullsized":"2024-09-11T07:43:19.758973+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:25.918507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:42:25.918507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:29.856961+0000","scrub_duration":0.0021642969999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"234'19","reported_seq":104,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.741885+0000","last_change":"2024-09-11T07:42:13.441031+0000","last_active":"2024-09-11T07:43:18.741885+0000","last_peered":"2024-09-11T07:43:18.741885+0000","last_clean":"2024-09-11T07:43:18.741885+0000","last_became_active":"2024-09-11T07:42:04.729191+0000","last_became_peered":"2024-09-11T07:42:04.729191+0000","last_unstale":"2024-09-11T07:43:18.741885+0000","last_undegraded":"2024-09-11T07:43:18.741885+0000","last_fullsized":"2024-09-11T07:43:18.741885+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:42:13.440976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:42:13.440976+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001959527,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":389,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.748975+0000","last_change":"2024-09-11T07:41:40.892143+0000","last_active":"2024-09-11T07:43:18.748975+0000","last_peered":"2024-09-11T07:43:18.748975+0000","last_clean":"2024-09-11T07:43:18.748975+0000","last_became_active":"2024-09-11T07:32:00.507050+0000","last_became_peered":"2024-09-11T07:32:00.507050+0000","last_unstale":"2024-09-11T07:43:18.748975+0000","last_undegraded":"2024-09-11T07:43:18.748975+0000","last_fullsized":"2024-09-11T07:43:18.748975+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:40.892054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:24.575210+0000","last_clean_scrub_stamp":"2024-09-11T07:41:40.892054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015644179999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":383,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.746551+0000","last_change":"2024-09-11T07:42:33.597664+0000","last_active":"2024-09-11T07:43:18.746551+0000","last_peered":"2024-09-11T07:43:18.746551+0000","last_clean":"2024-09-11T07:43:18.746551+0000","last_became_active":"2024-09-11T07:26:37.862373+0000","last_became_peered":"2024-09-11T07:26:37.862373+0000","last_unstale":"2024-09-11T07:43:18.746551+0000","last_undegraded":"2024-09-11T07:43:18.746551+0000","last_fullsized":"2024-09-11T07:43:18.746551+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:33.597580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:42:33.597580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:34.716004+0000","scrub_duration":0.002999842,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":321,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.745135+0000","last_change":"2024-09-11T07:43:08.381231+0000","last_active":"2024-09-11T07:43:18.745135+0000","last_peered":"2024-09-11T07:43:18.745135+0000","last_clean":"2024-09-11T07:43:18.745135+0000","last_became_active":"2024-09-11T07:36:05.224085+0000","last_became_peered":"2024-09-11T07:36:05.224085+0000","last_unstale":"2024-09-11T07:43:18.745135+0000","last_undegraded":"2024-09-11T07:43:18.745135+0000","last_fullsized":"2024-09-11T07:43:18.745135+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:08.381158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:43:08.381158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:11.367426+0000","scrub_duration":0.01040922,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"234'27","reported_seq":443,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.751559+0000","last_change":"2024-09-11T07:42:25.209195+0000","last_active":"2024-09-11T07:43:18.751559+0000","last_peered":"2024-09-11T07:43:18.751559+0000","last_clean":"2024-09-11T07:43:18.751559+0000","last_became_active":"2024-09-11T07:26:33.542544+0000","last_became_peered":"2024-09-11T07:26:33.542544+0000","last_unstale":"2024-09-11T07:43:18.751559+0000","last_undegraded":"2024-09-11T07:43:18.751559+0000","last_fullsized":"2024-09-11T07:43:18.751559+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'27","last_scrub_stamp":"2024-09-11T07:42:25.209063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:42:25.209063+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:44.356148+0000","scrub_duration":0.010336984,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":384,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.742405+0000","last_change":"2024-09-11T07:42:32.456953+0000","last_active":"2024-09-11T07:43:18.742405+0000","last_peered":"2024-09-11T07:43:18.742405+0000","last_clean":"2024-09-11T07:43:18.742405+0000","last_became_active":"2024-09-11T07:26:35.555862+0000","last_became_peered":"2024-09-11T07:26:35.555862+0000","last_unstale":"2024-09-11T07:43:18.742405+0000","last_undegraded":"2024-09-11T07:43:18.742405+0000","last_fullsized":"2024-09-11T07:43:18.742405+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:32.456852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:32.456852+0000","last_clean_scrub_stamp":"2024-09-11T07:42:32.456852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:34.248006+0000","scrub_duration":0.0036784880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":381,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.754891+0000","last_change":"2024-09-11T07:42:06.687713+0000","last_active":"2024-09-11T07:43:19.754891+0000","last_peered":"2024-09-11T07:43:19.754891+0000","last_clean":"2024-09-11T07:43:19.754891+0000","last_became_active":"2024-09-11T07:39:57.768994+0000","last_became_peered":"2024-09-11T07:39:57.768994+0000","last_unstale":"2024-09-11T07:43:19.754891+0000","last_undegraded":"2024-09-11T07:43:19.754891+0000","last_fullsized":"2024-09-11T07:43:19.754891+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:06.687650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:42:06.687650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:27.920723+0000","scrub_duration":0.001674313,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":404,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.755200+0000","last_change":"2024-09-11T07:42:12.887480+0000","last_active":"2024-09-11T07:43:19.755200+0000","last_peered":"2024-09-11T07:43:19.755200+0000","last_clean":"2024-09-11T07:43:19.755200+0000","last_became_active":"2024-09-11T07:39:57.766915+0000","last_became_peered":"2024-09-11T07:39:57.766915+0000","last_unstale":"2024-09-11T07:43:19.755200+0000","last_undegraded":"2024-09-11T07:43:19.755200+0000","last_fullsized":"2024-09-11T07:43:19.755200+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:12.887419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:42:12.887419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:43:25.417873+0000","scrub_duration":0.0015796460000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"234'23","reported_seq":412,"reported_epoch":365,"state":"active+clean","last_fresh":"2024-09-11T07:43:19.755526+0000","last_change":"2024-09-11T07:42:18.971596+0000","last_active":"2024-09-11T07:43:19.755526+0000","last_peered":"2024-09-11T07:43:19.755526+0000","last_clean":"2024-09-11T07:43:19.755526+0000","last_became_active":"2024-09-11T07:26:33.542711+0000","last_became_peered":"2024-09-11T07:26:33.542711+0000","last_unstale":"2024-09-11T07:43:19.755526+0000","last_undegraded":"2024-09-11T07:43:19.755526+0000","last_fullsized":"2024-09-11T07:43:19.755526+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'23","last_scrub_stamp":"2024-09-11T07:42:18.971492+0000","last_deep_scrub":"127'17","last_deep_scrub_stamp":"2024-09-11T07:35:43.871918+0000","last_clean_scrub_stamp":"2024-09-11T07:42:18.971492+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043193449999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":378,"reported_epoch":364,"state":"active+clean","last_fresh":"2024-09-11T07:43:18.744866+0000","last_change":"2024-09-11T07:41:19.724693+0000","last_active":"2024-09-11T07:43:18.744866+0000","last_peered":"2024-09-11T07:43:18.744866+0000","last_clean":"2024-09-11T07:43:18.744866+0000","last_became_active":"2024-09-11T07:26:35.562037+0000","last_became_peered":"2024-09-11T07:26:35.562037+0000","last_unstale":"2024-09-11T07:43:18.744866+0000","last_undegraded":"2024-09-11T07:43:18.744866+0000","last_fullsized":"2024-09-11T07:43:18.744866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:19.724643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:41:19.724643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001322174,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":6859063296,"num_objects":104661,"num_object_clones":0,"num_object_copies":313983,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104661,"num_write_kb":6698304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20598554624,"data_stored":20598554624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82088,"ondisk_log_size":82088,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1117,"num_read_kb":874,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":3466,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":765,"ondisk_log_size":765,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1003,"num_read_kb":998,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":147,"seq":631360192650,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2298300,"kb_used_data":1677108,"kb_used_omap":4,"kb_used_meta":621179,"kb_avail":91446852,"statfs":{"total":95995035648,"available":93641576448,"internally_reserved":0,"allocated":1717358592,"data_stored":1716955416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":636087814},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:42:26 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":1.7529999999999999,"15min":2.456},"min":{"1min":0.26800000000000002,"5min":0.191,"15min":0.191},"max":{"1min":7.1680000000000001,"5min":75.677000000000007,"15min":75.677000000000007},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.60399999999999998,"5min":1.952,"15min":2.4209999999999998},"min":{"1min":0.33700000000000002,"5min":0.22,"15min":0.22},"max":{"1min":1.4399999999999999,"5min":75.638999999999996,"15min":75.638999999999996},"last":1.5229999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:42:26 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":1.7549999999999999,"15min":1.1120000000000001},"min":{"1min":0.247,"5min":0.247,"15min":0.22500000000000001},"max":{"1min":0.71299999999999997,"5min":75.790000000000006,"15min":75.790000000000006},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":1.611,"15min":1.2070000000000001},"min":{"1min":0.33800000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.4359999999999999,"5min":76.363,"15min":76.363},"last":0.435}]},{"osd":2,"last update":"Wed Sep 11 07:42:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":3.4350000000000001,"15min":1.829},"min":{"1min":0.313,"5min":0.25700000000000001,"15min":0.22800000000000001},"max":{"1min":0.80700000000000005,"5min":227.79300000000001,"15min":227.79300000000001},"last":0.255},{"interface":"front","average":{"1min":0.53400000000000003,"5min":3.383,"15min":1.8320000000000001},"min":{"1min":0.27500000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.70599999999999996,"5min":227.22,"15min":227.22},"last":1.4490000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:42:26 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":1.119,"15min":2.1259999999999999},"min":{"1min":0.34799999999999998,"5min":0.307,"15min":0.27400000000000002},"max":{"1min":1.351,"5min":5.181,"15min":49.798999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":1.1910000000000001,"15min":2.125},"min":{"1min":0.40899999999999997,"5min":0.28899999999999998,"15min":0.27200000000000002},"max":{"1min":1.0229999999999999,"5min":7.3369999999999997,"15min":49.524000000000001},"last":1.371}]},{"osd":4,"last update":"Wed Sep 11 07:42:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.80800000000000005,"15min":0.623},"min":{"1min":0.34200000000000003,"5min":0.25700000000000001,"15min":0.253},"max":{"1min":1.0469999999999999,"5min":3.9399999999999999,"15min":6.0220000000000002},"last":3.5230000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.83399999999999996,"15min":0.70099999999999996},"min":{"1min":0.32200000000000001,"5min":0.26900000000000002,"15min":0.247},"max":{"1min":0.83099999999999996,"5min":6.9000000000000004,"15min":6.9000000000000004},"last":2.7959999999999998}]},{"osd":5,"last update":"Wed Sep 11 07:42:26 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.96399999999999997,"15min":0.70399999999999996},"min":{"1min":0.29199999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.96999999999999997,"5min":7.2160000000000002,"15min":7.2160000000000002},"last":1.498},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.877,"15min":0.77000000000000002},"min":{"1min":0.311,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.86799999999999999,"5min":6.7649999999999997,"15min":6.7649999999999997},"last":0.38300000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:42:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.749,"15min":0.64200000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29099999999999998},"max":{"1min":0.77700000000000002,"5min":5.1909999999999998,"15min":5.1909999999999998},"last":1.4850000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.85499999999999998,"15min":0.70499999999999996},"min":{"1min":0.33400000000000002,"5min":0.29899999999999999,"15min":0.29099999999999998},"max":{"1min":1.1040000000000001,"5min":4.2030000000000003,"15min":4.2030000000000003},"last":1.4710000000000001}]}]},{"osd":6,"up_from":44,"seq":188978561235,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3897172,"kb_used_data":3311436,"kb_used_omap":1,"kb_used_meta":585726,"kb_avail":89847980,"statfs":{"total":95995035648,"available":92004331520,"internally_reserved":0,"allocated":3390910464,"data_stored":3390484390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":599783768},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:42:25 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.73099999999999998,"15min":1.8400000000000001},"min":{"1min":0.32000000000000001,"5min":0.219,"15min":0.192},"max":{"1min":1.4099999999999999,"5min":5.141,"15min":142.29599999999999},"last":1.794},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.95999999999999996,"15min":2.0009999999999999},"min":{"1min":0.33600000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.81899999999999995,"5min":8.9139999999999997,"15min":142.68799999999999},"last":2.0600000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:42:25 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":1.49,"15min":1.619},"min":{"1min":0.32700000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.67600000000000005,"5min":37.350999999999999,"15min":127.92100000000001},"last":2.1150000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":1.3140000000000001,"15min":1.5860000000000001},"min":{"1min":0.32000000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.458,"5min":39.262999999999998,"15min":127.746},"last":0.90000000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:42:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.84899999999999998,"15min":0.92700000000000005},"min":{"1min":0.31,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.79300000000000004,"5min":15.798999999999999,"15min":26.491},"last":1.9770000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.84599999999999997,"15min":1.095},"min":{"1min":0.35699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.72399999999999998,"5min":6.282,"15min":26.561},"last":6.6559999999999997}]},{"osd":3,"last update":"Wed Sep 11 07:42:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":1.8129999999999999,"15min":1.4970000000000001},"min":{"1min":0.41099999999999998,"5min":0.246,"15min":0.246},"max":{"1min":1.4370000000000001,"5min":49.109000000000002,"15min":49.109000000000002},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":1.9550000000000001,"15min":1.5800000000000001},"min":{"1min":0.40899999999999997,"5min":0.28699999999999998,"15min":0.26700000000000002},"max":{"1min":0.79400000000000004,"5min":48.963999999999999,"15min":48.963999999999999},"last":1.9379999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:42:25 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":3.5409999999999999,"15min":1.53},"min":{"1min":0.29099999999999998,"5min":0.23899999999999999,"15min":0.223},"max":{"1min":2.0939999999999999,"5min":181.78800000000001,"15min":181.78800000000001},"last":6.7510000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":1.954,"15min":0.97599999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.24399999999999999},"max":{"1min":1.804,"5min":128.59299999999999,"15min":128.59299999999999},"last":4.3890000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:42:25 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.89500000000000002,"15min":0.91400000000000003},"min":{"1min":0.27800000000000002,"5min":0.246,"15min":0.246},"max":{"1min":1.2649999999999999,"5min":10.935,"15min":57.436},"last":2.0350000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.93600000000000005,"15min":0.94899999999999995},"min":{"1min":0.29299999999999998,"5min":0.253,"15min":0.251},"max":{"1min":2.0209999999999999,"5min":11.15,"15min":56.822000000000003},"last":2.0920000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:42:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.97799999999999998,"15min":0.77300000000000002},"min":{"1min":0.33900000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":2.0619999999999998,"5min":11.121,"15min":11.121},"last":2.2509999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.89800000000000002,"15min":0.78400000000000003},"min":{"1min":0.31,"5min":0.215,"15min":0.215},"max":{"1min":0.97799999999999998,"5min":10.962999999999999,"15min":10.962999999999999},"last":2.0059999999999998}]}]},{"osd":5,"up_from":37,"seq":158913790169,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2150024,"kb_used_data":1262008,"kb_used_omap":1,"kb_used_meta":887870,"kb_avail":91595128,"statfs":{"total":95995035648,"available":93793411072,"internally_reserved":0,"allocated":1292296192,"data_stored":1291948937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":909179224},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":1.1759999999999999,"15min":2.3420000000000001},"min":{"1min":0.28999999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":2.2269999999999999,"5min":10.65,"15min":185.68199999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.1299999999999999,"15min":2.484},"min":{"1min":0.20899999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":4.4649999999999999,"5min":17.635999999999999,"15min":186.12200000000001},"last":0.41699999999999998}]},{"osd":1,"last update":"Wed Sep 11 07:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":2.6040000000000001,"15min":1.5960000000000001},"min":{"1min":0.252,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":4.4829999999999997,"5min":122.738,"15min":122.738},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.624,"5min":2.556,"15min":1.627},"min":{"1min":0.27700000000000002,"5min":0.24099999999999999,"15min":0.22700000000000001},"max":{"1min":2.1520000000000001,"5min":122.66200000000001,"15min":122.66200000000001},"last":0.35599999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:42:57 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.105,"15min":1.754},"min":{"1min":0.316,"5min":0.28299999999999997,"15min":0.22700000000000001},"max":{"1min":4.2569999999999997,"5min":5.726,"15min":187.99199999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":1.0369999999999999,"15min":1.712},"min":{"1min":0.28199999999999997,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.8859999999999999,"5min":13.016,"15min":186.375},"last":2.1219999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:42:57 2024","interfaces":[{"interface":"back","average":{"1min":8.9000000000000004,"5min":4.2469999999999999,"15min":3.4620000000000002},"min":{"1min":0.34100000000000003,"5min":0.26900000000000002,"15min":0.23599999999999999},"max":{"1min":132.715,"5min":132.715,"15min":188.11500000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":8.4290000000000003,"5min":4.4569999999999999,"15min":3.5110000000000001},"min":{"1min":0.33600000000000002,"5min":0.28100000000000003,"15min":0.25700000000000001},"max":{"1min":133.01599999999999,"5min":133.01599999999999,"15min":186.44200000000001},"last":0.46000000000000002}]},{"osd":4,"last update":"Wed Sep 11 07:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1.1080000000000001,"15min":0.71599999999999997},"min":{"1min":0.33700000000000002,"5min":0.24199999999999999,"15min":0.19900000000000001},"max":{"1min":4.5090000000000003,"5min":11.103,"15min":11.103},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":1.0920000000000001,"15min":0.72599999999999998},"min":{"1min":0.30299999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":3.7959999999999998,"5min":10.305,"15min":10.305},"last":0.497}]},{"osd":6,"last update":"Wed Sep 11 07:43:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.92000000000000004,"15min":0.70799999999999996},"min":{"1min":0.32700000000000001,"5min":0.29199999999999998,"15min":0.26000000000000001},"max":{"1min":2.4100000000000001,"5min":5.6859999999999999,"15min":5.6859999999999999},"last":1.018},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.94099999999999995,"15min":0.69899999999999995},"min":{"1min":0.315,"5min":0.27800000000000002,"15min":0.25800000000000001},"max":{"1min":1.5680000000000001,"5min":10.613,"15min":10.613},"last":0.372}]},{"osd":7,"last update":"Wed Sep 11 07:43:20 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":1.5529999999999999,"15min":0.97299999999999998},"min":{"1min":0.34300000000000003,"5min":0.308,"15min":0.28000000000000003},"max":{"1min":2.8460000000000001,"5min":50.338999999999999,"15min":50.338999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":1.4299999999999999,"15min":0.95099999999999996},"min":{"1min":0.36299999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.7149999999999999,"5min":50.298999999999999,"15min":50.298999999999999},"last":0.47099999999999997}]}]},{"osd":4,"up_from":31,"seq":133143986397,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5147088,"kb_used_data":4548872,"kb_used_omap":1,"kb_used_meta":598206,"kb_avail":88598064,"statfs":{"total":95995035648,"available":90724417536,"internally_reserved":0,"allocated":4658044928,"data_stored":4657621984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":612563288},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,0,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:42:47 2024","interfaces":[{"interface":"back","average":{"1min":6.3609999999999998,"5min":2.097,"15min":1.635},"min":{"1min":0.25900000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":87.623000000000005,"5min":87.623000000000005,"15min":87.623000000000005},"last":0.86499999999999999},{"interface":"front","average":{"1min":7.2450000000000001,"5min":2.1699999999999999,"15min":1.643},"min":{"1min":0.34200000000000003,"5min":0.222,"15min":0.20499999999999999},"max":{"1min":87.682000000000002,"5min":87.682000000000002,"15min":87.682000000000002},"last":3.52}]},{"osd":1,"last update":"Wed Sep 11 07:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":3.6219999999999999,"15min":2.0169999999999999},"min":{"1min":0.253,"5min":0.22900000000000001,"15min":0.22700000000000001},"max":{"1min":2.5950000000000002,"5min":141.78399999999999,"15min":141.78399999999999},"last":0.437},{"interface":"front","average":{"1min":0.90400000000000003,"5min":3.6259999999999999,"15min":1.8640000000000001},"min":{"1min":0.35099999999999998,"5min":0.224,"15min":0.224},"max":{"1min":2.919,"5min":145.65899999999999,"15min":145.65899999999999},"last":3.4460000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:42:47 2024","interfaces":[{"interface":"back","average":{"1min":1.8959999999999999,"5min":1.0389999999999999,"15min":1.091},"min":{"1min":0.307,"5min":0.24099999999999999,"15min":0.223},"max":{"1min":14.262,"5min":14.262,"15min":19.927},"last":3.4700000000000002},{"interface":"front","average":{"1min":1.887,"5min":1.0669999999999999,"15min":1.1000000000000001},"min":{"1min":0.41499999999999998,"5min":0.28399999999999997,"15min":0.24399999999999999},"max":{"1min":14.273999999999999,"5min":14.273999999999999,"15min":20.02},"last":3.214}]},{"osd":3,"last update":"Wed Sep 11 07:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.1879999999999999,"15min":1.2889999999999999},"min":{"1min":0.373,"5min":0.28100000000000003,"15min":0.26200000000000001},"max":{"1min":3.6040000000000001,"5min":13.853,"15min":24.029},"last":3.2469999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":1.1379999999999999,"15min":1.204},"min":{"1min":0.33300000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":2.048,"5min":13.651,"15min":24.265000000000001},"last":0.57799999999999996}]},{"osd":5,"last update":"Wed Sep 11 07:43:08 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.84599999999999997,"15min":0.65800000000000003},"min":{"1min":0.26300000000000001,"5min":0.224,"15min":0.21099999999999999},"max":{"1min":1.9470000000000001,"5min":5.6470000000000002,"15min":5.6470000000000002},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.90500000000000003,"15min":0.68300000000000005},"min":{"1min":0.29099999999999998,"5min":0.23000000000000001,"15min":0.22700000000000001},"max":{"1min":2.9860000000000002,"5min":5.7450000000000001,"15min":5.7450000000000001},"last":3.4969999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:42:36 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.94299999999999995,"15min":0.70299999999999996},"min":{"1min":0.35199999999999998,"5min":0.27000000000000002,"15min":0.20899999999999999},"max":{"1min":1.899,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":3.27},{"interface":"front","average":{"1min":0.5,"5min":0.71899999999999997,"15min":0.63500000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.25800000000000001},"max":{"1min":0.71199999999999997,"5min":4.2140000000000004,"15min":9.3770000000000007},"last":3.5459999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:42:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":1,"15min":0.73199999999999998},"min":{"1min":0.39600000000000002,"5min":0.30099999999999999,"15min":0.29399999999999998},"max":{"1min":0.76300000000000001,"5min":8.0129999999999999,"15min":8.0129999999999999},"last":3.4129999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.84199999999999997,"15min":0.69899999999999995},"min":{"1min":0.34799999999999998,"5min":0.312,"15min":0.29599999999999999},"max":{"1min":0.72499999999999998,"5min":5.5499999999999998,"15min":5.5499999999999998},"last":0.39600000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182626,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2164240,"kb_used_data":1265216,"kb_used_omap":1,"kb_used_meta":898878,"kb_avail":91580912,"statfs":{"total":95995035648,"available":93778853888,"internally_reserved":0,"allocated":1295581184,"data_stored":1295224761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":920451416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:42:28 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.5660000000000001,"15min":1.5840000000000001},"min":{"1min":0.19800000000000001,"5min":0.17199999999999999,"15min":0.16600000000000001},"max":{"1min":14.909000000000001,"5min":26.864999999999998,"15min":38.432000000000002},"last":5.4199999999999999},{"interface":"front","average":{"1min":1.03,"5min":1.8009999999999999,"15min":2.0009999999999999},"min":{"1min":0.25,"5min":0.20899999999999999,"15min":0.20300000000000001},"max":{"1min":15.037000000000001,"5min":26.657,"15min":91.876999999999995},"last":0.79400000000000004}]},{"osd":1,"last update":"Wed Sep 11 07:42:28 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":4.5960000000000001,"15min":2.7829999999999999},"min":{"1min":0.245,"5min":0.218,"15min":0.19500000000000001},"max":{"1min":0.84599999999999997,"5min":135.41999999999999,"15min":135.41999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":4.5579999999999998,"15min":2.399},"min":{"1min":0.23699999999999999,"5min":0.23499999999999999,"15min":0.192},"max":{"1min":1.052,"5min":136.126,"15min":136.126},"last":0.59499999999999997}]},{"osd":2,"last update":"Wed Sep 11 07:42:28 2024","interfaces":[{"interface":"back","average":{"1min":7.9119999999999999,"5min":2.5139999999999998,"15min":1.782},"min":{"1min":0.26900000000000002,"5min":0.24099999999999999,"15min":0.23699999999999999},"max":{"1min":164.02600000000001,"5min":164.02600000000001,"15min":164.02600000000001},"last":2.0579999999999998},{"interface":"front","average":{"1min":7.7939999999999996,"5min":2.4169999999999998,"15min":1.6799999999999999},"min":{"1min":0.24199999999999999,"5min":0.222,"15min":0.21199999999999999},"max":{"1min":163.37799999999999,"5min":163.37799999999999,"15min":163.37799999999999},"last":5.2190000000000003}]},{"osd":4,"last update":"Wed Sep 11 07:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":3.0259999999999998,"15min":1.714},"min":{"1min":0.36199999999999999,"5min":0.312,"15min":0.27500000000000002},"max":{"1min":2.7389999999999999,"5min":132.76400000000001,"15min":132.76400000000001},"last":3.9620000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":3.1179999999999999,"15min":1.903},"min":{"1min":0.374,"5min":0.311,"15min":0.255},"max":{"1min":2.044,"5min":132.154,"15min":132.154},"last":0.83899999999999997}]},{"osd":5,"last update":"Wed Sep 11 07:42:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":2.3959999999999999,"15min":1.651},"min":{"1min":0.36899999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.7050000000000001,"5min":137.50299999999999,"15min":137.50299999999999},"last":3.8889999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":2.496,"15min":1.7010000000000001},"min":{"1min":0.35099999999999998,"5min":0.29199999999999998,"15min":0.28299999999999997},"max":{"1min":1.1739999999999999,"5min":137.45599999999999,"15min":137.45599999999999},"last":0.81499999999999995}]},{"osd":6,"last update":"Wed Sep 11 07:42:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":1.0149999999999999,"15min":1.1890000000000001},"min":{"1min":0.35599999999999998,"5min":0.313,"15min":0.27000000000000002},"max":{"1min":1.109,"5min":6.8680000000000003,"15min":32.350999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":1.034,"15min":1.165},"min":{"1min":0.40400000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":3.0059999999999998,"5min":4.5199999999999996,"15min":32.402999999999999},"last":3.9380000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:43:16 2024","interfaces":[{"interface":"back","average":{"1min":9.7330000000000005,"5min":5.2530000000000001,"15min":2.597},"min":{"1min":0.44500000000000001,"5min":0.35699999999999998,"15min":0.308},"max":{"1min":162.42699999999999,"5min":162.42699999999999,"15min":162.42699999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":9.6370000000000005,"5min":5.1909999999999998,"15min":2.9420000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":162.39099999999999,"5min":162.39099999999999,"15min":162.39099999999999},"last":0.56000000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411559,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3496824,"kb_used_data":2911088,"kb_used_omap":1,"kb_used_meta":585726,"kb_avail":90248328,"statfs":{"total":95995035648,"available":92414287872,"internally_reserved":0,"allocated":2980954112,"data_stored":2980554951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":599783768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:43:01 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.587,"15min":1.871},"min":{"1min":0.26900000000000002,"5min":0.17799999999999999,"15min":0.16400000000000001},"max":{"1min":5.1580000000000004,"5min":57.802999999999997,"15min":57.802999999999997},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.496,"5min":1.748,"15min":2},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":6.4539999999999997,"5min":57.759,"15min":57.759},"last":1.8129999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":1.617,"15min":1.573},"min":{"1min":0.28399999999999997,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":2.3769999999999998,"5min":48.042000000000002,"15min":65.153999999999996},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.88200000000000001,"5min":1.7350000000000001,"15min":1.8049999999999999},"min":{"1min":0.249,"5min":0.23699999999999999,"15min":0.216},"max":{"1min":5.1200000000000001,"5min":50.201000000000001,"15min":63.311999999999998},"last":0.36299999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:42:27 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":1.988,"15min":1.8380000000000001},"min":{"1min":0.28799999999999998,"5min":0.224,"15min":0.224},"max":{"1min":2.395,"5min":67.552999999999997,"15min":67.552999999999997},"last":1.7},{"interface":"front","average":{"1min":0.441,"5min":1.8680000000000001,"15min":1.821},"min":{"1min":0.311,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.83899999999999997,"5min":67.352999999999994,"15min":67.352999999999994},"last":1.327}]},{"osd":4,"last update":"Wed Sep 11 07:42:46 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.80300000000000005,"15min":1.1180000000000001},"min":{"1min":0.41399999999999998,"5min":0.315,"15min":0.27500000000000002},"max":{"1min":1.569,"5min":4.625,"15min":30.946999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.83199999999999996,"15min":1.024},"min":{"1min":0.46300000000000002,"5min":0.313,"15min":0.29999999999999999},"max":{"1min":1.7909999999999999,"5min":4.0510000000000002,"15min":22.434000000000001},"last":0.746}]},{"osd":5,"last update":"Wed Sep 11 07:43:01 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.94199999999999995,"15min":1.1679999999999999},"min":{"1min":0.44400000000000001,"5min":0.32100000000000001,"15min":0.31900000000000001},"max":{"1min":2.698,"5min":3.319,"15min":31.789000000000001},"last":1.6240000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.93899999999999995,"15min":1.2170000000000001},"min":{"1min":0.41199999999999998,"5min":0.34499999999999997,"15min":0.31},"max":{"1min":1.6339999999999999,"5min":8.1690000000000005,"15min":30.898},"last":0.436}]},{"osd":6,"last update":"Wed Sep 11 07:42:27 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.94199999999999995,"15min":1.1479999999999999},"min":{"1min":0.40999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":4.298,"5min":6.891,"15min":22.399000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.88500000000000001,"15min":1.1319999999999999},"min":{"1min":0.375,"5min":0.35699999999999998,"15min":0.29999999999999999},"max":{"1min":1.1339999999999999,"5min":4.867,"15min":25.739000000000001},"last":0.45700000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:43:17 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":0.94899999999999995,"15min":1.3799999999999999},"min":{"1min":0.46000000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.2890000000000001,"5min":5.2199999999999998,"15min":22.484000000000002},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0860000000000001,"15min":1.6180000000000001},"min":{"1min":0.39300000000000002,"5min":0.373,"15min":0.36699999999999999},"max":{"1min":2.6819999999999999,"5min":4.9059999999999997,"15min":25.670000000000002},"last":0.47999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575082,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1731400,"kb_used_data":831232,"kb_used_omap":1,"kb_used_meta":900158,"kb_avail":92013752,"statfs":{"total":95995035648,"available":94222082048,"internally_reserved":0,"allocated":851181568,"data_stored":850767069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":921762136},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:42:26 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":1.411,"15min":2.0419999999999998},"min":{"1min":0.223,"5min":0.17299999999999999,"15min":0.16900000000000001},"max":{"1min":5.7350000000000003,"5min":9.2669999999999995,"15min":82.218999999999994},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":1.4419999999999999,"15min":2.0649999999999999},"min":{"1min":0.23899999999999999,"5min":0.186,"15min":0.186},"max":{"1min":3.0659999999999998,"5min":6.9880000000000004,"15min":82.278999999999996},"last":1.113}]},{"osd":2,"last update":"Wed Sep 11 07:43:04 2024","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.4770000000000001,"15min":1.571},"min":{"1min":0.34300000000000003,"5min":0.251,"15min":0.19800000000000001},"max":{"1min":5.0469999999999997,"5min":6.7480000000000002,"15min":9.0969999999999995},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.244,"5min":1.5029999999999999,"15min":1.613},"min":{"1min":0.24199999999999999,"5min":0.20499999999999999,"15min":0.191},"max":{"1min":3.7650000000000001,"5min":20.518000000000001,"15min":20.518000000000001},"last":0.59499999999999997}]},{"osd":3,"last update":"Wed Sep 11 07:43:23 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":2.5840000000000001,"15min":2.7909999999999999},"min":{"1min":0.28899999999999998,"5min":0.25900000000000001,"15min":0.25800000000000001},"max":{"1min":4.5819999999999999,"5min":90.766999999999996,"15min":90.766999999999996},"last":3.504},{"interface":"front","average":{"1min":0.97099999999999997,"5min":2.4790000000000001,"15min":2.718},"min":{"1min":0.27300000000000002,"5min":0.22900000000000001,"15min":0.20999999999999999},"max":{"1min":3.7080000000000002,"5min":91.513000000000005,"15min":91.513000000000005},"last":0.92500000000000004}]},{"osd":4,"last update":"Wed Sep 11 07:42:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":1.6299999999999999,"15min":1.3049999999999999},"min":{"1min":0.40600000000000003,"5min":0.30299999999999999,"15min":0.27400000000000002},"max":{"1min":1.8109999999999999,"5min":7.8390000000000004,"15min":7.8390000000000004},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.86099999999999999,"5min":1.53,"15min":1.4019999999999999},"min":{"1min":0.441,"5min":0.29999999999999999,"15min":0.26500000000000001},"max":{"1min":2.9340000000000002,"5min":7.8239999999999998,"15min":9.2289999999999992},"last":0.33700000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:43:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":1.5820000000000001,"15min":1.359},"min":{"1min":0.373,"5min":0.27500000000000002,"15min":0.26600000000000001},"max":{"1min":2.0649999999999999,"5min":8.3680000000000003,"15min":8.3680000000000003},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":1.5740000000000001,"15min":1.4490000000000001},"min":{"1min":0.32700000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.391,"5min":8.6809999999999992,"15min":9.1929999999999996},"last":0.36599999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:43:23 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.393,"15min":1.454},"min":{"1min":0.35299999999999998,"5min":0.33000000000000002,"15min":0.30099999999999999},"max":{"1min":3.1160000000000001,"5min":9.1850000000000005,"15min":24.018000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":1.2709999999999999,"15min":1.2250000000000001},"min":{"1min":0.40000000000000002,"5min":0.318,"15min":0.29999999999999999},"max":{"1min":1.6060000000000001,"5min":9.3339999999999996,"15min":23.681000000000001},"last":0.45100000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:43:23 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":1.5680000000000001,"15min":1.361},"min":{"1min":0.42899999999999999,"5min":0.36099999999999999,"15min":0.32200000000000001},"max":{"1min":1.569,"5min":9.3629999999999995,"15min":9.3629999999999995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.875,"5min":1.714,"15min":1.575},"min":{"1min":0.38400000000000001,"5min":0.33400000000000002,"15min":0.313},"max":{"1min":2.9849999999999999,"5min":8.2219999999999995,"15min":8.2219999999999995},"last":0.46899999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738608,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5061772,"kb_used_data":4172548,"kb_used_omap":1,"kb_used_meta":889214,"kb_avail":88683380,"statfs":{"total":95995035648,"available":90811781120,"internally_reserved":0,"allocated":4272689152,"data_stored":4272270260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":910555480},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":1.379,"15min":1.3240000000000001},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":1.5289999999999999,"5min":15.522,"15min":15.522},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":1.409,"15min":1.365},"min":{"1min":0.20799999999999999,"5min":0.20499999999999999,"15min":0.16600000000000001},"max":{"1min":6.6289999999999996,"5min":13.988,"15min":13.988},"last":0.29399999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:43:00 2024","interfaces":[{"interface":"back","average":{"1min":4.7469999999999999,"5min":3.96,"15min":2.3929999999999998},"min":{"1min":0.27400000000000002,"5min":0.188,"15min":0.188},"max":{"1min":68,"5min":200.339,"15min":200.339},"last":1.3300000000000001},{"interface":"front","average":{"1min":4.5469999999999997,"5min":3.8490000000000002,"15min":2.2919999999999998},"min":{"1min":0.23999999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":67.625,"5min":200.40100000000001,"15min":200.40100000000001},"last":1.262}]},{"osd":3,"last update":"Wed Sep 11 07:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":1.3580000000000001,"15min":2.028},"min":{"1min":0.25,"5min":0.23300000000000001,"15min":0.20799999999999999},"max":{"1min":2.484,"5min":8.8070000000000004,"15min":45.185000000000002},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":1.472,"15min":1.8260000000000001},"min":{"1min":0.28699999999999998,"5min":0.24299999999999999,"15min":0.22800000000000001},"max":{"1min":1.5700000000000001,"5min":15.252000000000001,"15min":43.228999999999999},"last":1.054}]},{"osd":4,"last update":"Wed Sep 11 07:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":1.4219999999999999,"15min":1.282},"min":{"1min":0.31,"5min":0.29299999999999998,"15min":0.26700000000000002},"max":{"1min":2.9220000000000002,"5min":30.315000000000001,"15min":30.315000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.659,"15min":1.486},"min":{"1min":0.35199999999999998,"5min":0.28599999999999998,"15min":0.27000000000000002},"max":{"1min":6.5439999999999996,"5min":30.363,"15min":30.363},"last":0.56899999999999995}]},{"osd":5,"last update":"Wed Sep 11 07:43:00 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":1.1839999999999999,"15min":1.3049999999999999},"min":{"1min":0.40100000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.976,"5min":4.9749999999999996,"15min":19.381},"last":0.51900000000000002},{"interface":"front","average":{"1min":1.357,"5min":1.454,"15min":1.464},"min":{"1min":0.378,"5min":0.36399999999999999,"15min":0.27200000000000002},"max":{"1min":6.46,"5min":7.202,"15min":19.437000000000001},"last":0.433}]},{"osd":6,"last update":"Wed Sep 11 07:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":1.3200000000000001,"15min":1.393},"min":{"1min":0.39300000000000002,"5min":0.33900000000000002,"15min":0.316},"max":{"1min":2.6840000000000002,"5min":5.843,"15min":23.190999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":1.2450000000000001,"15min":1.373},"min":{"1min":0.38300000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.8129999999999999,"5min":15.404,"15min":23.632000000000001},"last":1.014}]},{"osd":7,"last update":"Wed Sep 11 07:43:22 2024","interfaces":[{"interface":"back","average":{"1min":1.637,"5min":1.758,"15min":1.5860000000000001},"min":{"1min":0.378,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":6.4939999999999998,"5min":30.43,"15min":30.43},"last":0.69899999999999995},{"interface":"front","average":{"1min":1.661,"5min":1.857,"15min":1.617},"min":{"1min":0.47899999999999998,"5min":0.35999999999999999,"15min":0.315},"max":{"1min":4.9080000000000004,"5min":31.065000000000001,"15min":31.065000000000001},"last":1.0860000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4308271104,"data_stored":4308271104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":852361216,"data_stored":852361216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3002990592,"data_stored":3002990592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1303248896,"data_stored":1303248896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4691656704,"data_stored":4691656704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1292500992,"data_stored":1292500992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3412590592,"data_stored":3412590592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1734934528,"data_stored":1734934528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:43:31.188 INFO:tasks.thrashosds.thrasher:Setting 4.f to [1, 3, 4] 2024-09-11T07:43:31.188 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '4.f', '1', '3', '4'] 2024-09-11T07:43:31.189 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pg-upmap 4.f 1 3 4 2024-09-11T07:43:31.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:31.191 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:31.192 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:31.293 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:31.337 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:31.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:31.342 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:31.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:31 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:43:31.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:31 smithi022 ceph-mon[29462]: from='client.25781 v1:172.21.15.22:0/1718328787' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:43:31.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:31 smithi022 ceph-mon[29462]: pgmap v789: 148 pgs: 148 active+clean; 6.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-11T07:43:31.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:31 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:43:31.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:31 smithi022 ceph-mon[37330]: from='client.25781 v1:172.21.15.22:0/1718328787' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:43:31.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:31 smithi022 ceph-mon[37330]: pgmap v789: 148 pgs: 148 active+clean; 6.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-11T07:43:31.443 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:31.490 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:31.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:31.491 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:31.592 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:31.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:31 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:43:31.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:31 smithi104 ceph-mon[33507]: from='client.25781 v1:172.21.15.22:0/1718328787' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:43:31.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:31 smithi104 ceph-mon[33507]: pgmap v789: 148 pgs: 148 active+clean; 6.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-11T07:43:31.628 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:31.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:31.629 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:31.730 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:31.792 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:43:31.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:31.808 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:31.809 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:31.909 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:31.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:31.953 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:31.953 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:31.963 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 76 16 113321 113305 93.1628 74.125 0.0319169 0.0107294 2024-09-11T07:43:32.054 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:32.094 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:32.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:32.095 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:32.097 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:43:32.097 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:43:32.195 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:32.221 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:32.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:32.222 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:32.323 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:32.359 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:32.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:32.361 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:32.461 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:32.499 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:43:32.499 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:43:32.501 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:32.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:32.502 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:32.510 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:43:32.529 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:43:32.530 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:43:32.560 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:43:32.560 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:43:32.589 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:43:32.589 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:43:32.603 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:32.605 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:43:32.606 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:43:32.628 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:43:32.628 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:43:32.630 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:32.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:32.631 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:32.662 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:43:32.662 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:43:32.693 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:43:32.693 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:43:32.732 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:32.767 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:32.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:32.769 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:32.796 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-11T07:43:32.870 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:32.917 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:32.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:32.918 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:32.963 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 77 16 114434 114418 92.8562 69.5625 0.0150554 0.0107642 2024-09-11T07:43:33.019 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:33.068 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:33.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:33.069 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:33.170 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:33.206 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:33.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:33.207 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:33.257 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:43:33.307 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:33.350 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:33.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:33.351 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:33.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:33 smithi022 ceph-mon[29462]: pgmap v790: 148 pgs: 148 active+clean; 6.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 623 B/s rd, 45 MiB/s wr, 717 op/s 2024-09-11T07:43:33.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:33 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1845310411' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:43:33.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:33 smithi022 ceph-mon[37330]: pgmap v790: 148 pgs: 148 active+clean; 6.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 623 B/s rd, 45 MiB/s wr, 717 op/s 2024-09-11T07:43:33.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:33 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1845310411' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:43:33.451 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:33.485 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:33.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:33.486 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:33.587 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:33.621 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:33.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:33.623 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:33.723 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:33.756 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:33.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:33.757 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:33.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:33 smithi104 ceph-mon[33507]: pgmap v790: 148 pgs: 148 active+clean; 6.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 623 B/s rd, 45 MiB/s wr, 717 op/s 2024-09-11T07:43:33.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:33 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1845310411' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:43:33.858 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:33.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:33.898 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:33.898 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:33.963 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 78 16 115620 115604 92.616 74.125 0.0287797 0.0107927 2024-09-11T07:43:34.000 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:34.040 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:34.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:34.041 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:34.142 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:34.180 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:34.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:34.181 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:34.282 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:34.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:34.316 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:34.316 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:34.348 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:43:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:43:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:43:34.418 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:34.449 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:34.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:34.450 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:34.551 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:34.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:34.596 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:34.596 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:34.697 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:34.736 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:34.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:34.737 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:34.838 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:34.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:34.886 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:34.886 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:34.963 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 79 16 116916 116900 92.4689 81 0.00918605 0.01081 2024-09-11T07:43:34.986 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:35.025 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:35.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:35.027 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:35.127 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:35.164 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:35.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:35.165 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:35.266 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:35.312 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:35.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:35.314 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:35.415 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:35.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:35 smithi022 ceph-mon[29462]: pgmap v791: 148 pgs: 148 active+clean; 7.0 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1023 B/s rd, 69 MiB/s wr, 1.11k op/s 2024-09-11T07:43:35.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:35 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:43:35.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:35 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:43:35.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:35 smithi022 ceph-mon[37330]: pgmap v791: 148 pgs: 148 active+clean; 7.0 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1023 B/s rd, 69 MiB/s wr, 1.11k op/s 2024-09-11T07:43:35.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:35 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:43:35.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:35 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:43:35.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:35.450 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:35.450 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:35.513 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:35 smithi104 ceph-mon[33507]: pgmap v791: 148 pgs: 148 active+clean; 7.0 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1023 B/s rd, 69 MiB/s wr, 1.11k op/s 2024-09-11T07:43:35.513 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:35 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:43:35.513 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:35 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:43:35.551 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:35.584 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:35.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:35.586 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:35.686 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:35.725 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:35.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:35.727 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:35.820 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:43:35.821 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:43:35.821 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2024-09-11T07:43:35.821 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2024-09-11T07:43:35.821 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:43:35.827 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:35.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:35.873 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:35.873 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:35.963 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:2024-09-11T07:43:35.962992+0000 min lat: 0.00106648 max lat: 0.408634 avg lat: 0.0108349 2024-09-11T07:43:35.963 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T07:43:35.963 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 80 16 118119 118103 92.2528 75.1875 0.014444 0.0108349 2024-09-11T07:43:35.974 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:36.007 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:36.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:36.008 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:36.109 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:36.150 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:36.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:36.151 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:36.252 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:36.285 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:36.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:36.286 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:36.314 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:43:36.387 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:36.415 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:36.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:36.416 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:36.517 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:36.520 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:36 smithi022 ceph-mon[29462]: osdmap e368: 8 total, 8 up, 8 in 2024-09-11T07:43:36.521 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:36 smithi022 ceph-mon[37330]: osdmap e368: 8 total, 8 up, 8 in 2024-09-11T07:43:36.546 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:36.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:36.547 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:36.648 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:36.660 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:36 smithi104 ceph-mon[33507]: osdmap e368: 8 total, 8 up, 8 in 2024-09-11T07:43:36.663 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-11T07:43:36.681 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:36.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:36.682 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:36.782 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:36.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:36.826 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:36.826 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:36.927 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:36.963 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 81 16 119223 119207 91.9657 69 0.0105684 0.0108689 2024-09-11T07:43:36.965 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:36.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:36.966 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:37.066 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:37.080 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:43:37.104 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:37.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:37.105 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:37.206 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:37.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:37.243 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:37.244 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:37.344 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:37.389 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:37.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:37.390 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:37.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:37 smithi022 ceph-mon[29462]: pgmap v793: 148 pgs: 148 active+clean; 7.1 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.1 KiB/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-11T07:43:37.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:37 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2176947424' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "4.f", "id": [1, 3, 4]}]: dispatch 2024-09-11T07:43:37.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:37 smithi022 ceph-mon[29462]: from='client.25787 ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "4.f", "id": [1, 3, 4]}]: dispatch 2024-09-11T07:43:37.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:37 smithi022 ceph-mon[37330]: pgmap v793: 148 pgs: 148 active+clean; 7.1 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.1 KiB/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-11T07:43:37.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:37 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2176947424' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "4.f", "id": [1, 3, 4]}]: dispatch 2024-09-11T07:43:37.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:37 smithi022 ceph-mon[37330]: from='client.25787 ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "4.f", "id": [1, 3, 4]}]: dispatch 2024-09-11T07:43:37.491 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:37.520 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:37.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:37.521 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:37.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:37 smithi104 ceph-mon[33507]: pgmap v793: 148 pgs: 148 active+clean; 7.1 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.1 KiB/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-11T07:43:37.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:37 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2176947424' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "4.f", "id": [1, 3, 4]}]: dispatch 2024-09-11T07:43:37.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:37 smithi104 ceph-mon[33507]: from='client.25787 ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "4.f", "id": [1, 3, 4]}]: dispatch 2024-09-11T07:43:37.623 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:37.650 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:43:37.652 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:37.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:37.653 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:37.723 INFO:teuthology.orchestra.run.smithi022.stderr:set 4.f pg_upmap mapping to [1,3,4] 2024-09-11T07:43:37.754 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:37.780 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:37.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:37.781 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:37.882 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:37.926 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:37.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:37.927 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:37.963 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 82 16 120402 120386 91.7427 73.6875 0.0215091 0.0108957 2024-09-11T07:43:38.028 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:38.071 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:38.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:38.072 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:38.173 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:38.209 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:38.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:38.210 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:38.311 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:38.344 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:38.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:38.345 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:38.446 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:38.479 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:38.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:38.480 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:38.581 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:38.623 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:38.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:38.624 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:38.725 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:38.725 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is set 2024-09-11T07:43:38.757 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:38.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:38.758 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:38.858 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:38.888 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:38.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:38.890 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:38.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:38 smithi104 ceph-mon[33507]: from='client.25787 ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "4.f", "id": [1, 3, 4]}]': finished 2024-09-11T07:43:38.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:38 smithi104 ceph-mon[33507]: osdmap e369: 8 total, 8 up, 8 in 2024-09-11T07:43:38.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:38 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3976604641' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:43:38.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:43:38.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:43:38.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:38 smithi104 ceph-mon[33507]: pgmap v795: 148 pgs: 148 active+clean; 7.1 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 639 B/s rd, 46 MiB/s wr, 739 op/s 2024-09-11T07:43:38.963 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 83 16 121636 121620 91.5665 77.125 0.0150992 0.0109164 2024-09-11T07:43:38.991 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:39.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:39.028 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:39.028 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:39.112 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:43:39.113 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2024-09-11T07:43:39.113 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2024-09-11T07:43:39.113 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2024-09-11T07:43:39.113 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.113 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:86227 9.e 9:74b4edf6:::benchmark_data_smithi022_133943_object86226:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:43:39.113 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:27.528103+0000", 2024-09-11T07:43:39.113 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 311.57920803799999, 2024-09-11T07:43:39.113 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18483545900000001, 2024-09-11T07:43:39.113 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:39.113 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:39.113 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:39.114 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:39.114 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:39.114 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 86227 2024-09-11T07:43:39.114 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.114 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:39.114 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.114 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:39.114 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.528103+0000", 2024-09-11T07:43:39.114 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.114 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.114 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.114 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:39.114 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.528103+0000", 2024-09-11T07:43:39.115 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.115 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.115 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.115 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:39.115 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.528101+0000", 2024-09-11T07:43:39.115 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:43:39.115 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.115 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.115 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:39.115 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.528241+0000", 2024-09-11T07:43:39.115 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014016300000000001 2024-09-11T07:43:39.115 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.115 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.115 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:39.115 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.528250+0000", 2024-09-11T07:43:39.116 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8279999999999992e-06 2024-09-11T07:43:39.116 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.116 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.116 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:39.116 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.528255+0000", 2024-09-11T07:43:39.116 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0200000000000002e-06 2024-09-11T07:43:39.116 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.116 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.116 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.116 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.528294+0000", 2024-09-11T07:43:39.117 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8683000000000002e-05 2024-09-11T07:43:39.117 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.117 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.117 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:39.117 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.528397+0000", 2024-09-11T07:43:39.117 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010292400000000001 2024-09-11T07:43:39.117 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.117 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.117 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,4", 2024-09-11T07:43:39.117 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.528530+0000", 2024-09-11T07:43:39.117 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000132729 2024-09-11T07:43:39.117 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.117 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.118 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:39.118 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.531084+0000", 2024-09-11T07:43:39.118 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0025538969999999999 2024-09-11T07:43:39.118 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.118 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.118 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.118 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.531301+0000", 2024-09-11T07:43:39.118 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00021777799999999999 2024-09-11T07:43:39.118 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.118 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.118 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.118 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.712803+0000", 2024-09-11T07:43:39.118 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18150128800000001 2024-09-11T07:43:39.118 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.118 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.119 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:39.119 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.712872+0000", 2024-09-11T07:43:39.119 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9064000000000001e-05 2024-09-11T07:43:39.119 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.119 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.119 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:39.119 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.712938+0000", 2024-09-11T07:43:39.119 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6431000000000001e-05 2024-09-11T07:43:39.119 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.119 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:39.119 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.119 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.119 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.120 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:86231 9.3 9:c7d2fbd6:::benchmark_data_smithi022_133943_object86230:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:43:39.120 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:27.531792+0000", 2024-09-11T07:43:39.120 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 311.57551858199997, 2024-09-11T07:43:39.120 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.24124353800000001, 2024-09-11T07:43:39.120 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:39.120 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:39.120 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:39.120 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:39.120 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:39.120 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 86231 2024-09-11T07:43:39.120 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.120 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:39.120 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.120 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:39.120 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.531792+0000", 2024-09-11T07:43:39.121 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.121 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.121 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.121 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:39.121 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.531792+0000", 2024-09-11T07:43:39.121 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.121 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.121 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.121 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:39.121 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.531790+0000", 2024-09-11T07:43:39.121 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:43:39.121 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.121 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.122 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:39.122 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.531895+0000", 2024-09-11T07:43:39.122 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000105342 2024-09-11T07:43:39.124 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.124 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.124 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:39.124 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.531909+0000", 2024-09-11T07:43:39.124 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3998e-05 2024-09-11T07:43:39.124 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.124 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.124 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:39.124 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.531915+0000", 2024-09-11T07:43:39.124 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9800000000000003e-06 2024-09-11T07:43:39.125 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.125 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.125 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.125 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.531955+0000", 2024-09-11T07:43:39.125 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9527999999999998e-05 2024-09-11T07:43:39.125 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.125 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.125 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:39.125 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532091+0000", 2024-09-11T07:43:39.125 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013630399999999999 2024-09-11T07:43:39.125 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.125 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.125 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:43:39.125 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532398+0000", 2024-09-11T07:43:39.126 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00030691399999999998 2024-09-11T07:43:39.126 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.126 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.126 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.126 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536109+0000", 2024-09-11T07:43:39.126 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0037104379999999999 2024-09-11T07:43:39.126 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.126 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.126 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:39.126 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.537537+0000", 2024-09-11T07:43:39.126 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001428594 2024-09-11T07:43:39.126 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.126 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.126 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.127 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.772927+0000", 2024-09-11T07:43:39.127 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.235389549 2024-09-11T07:43:39.127 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.127 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.127 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:39.127 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.772991+0000", 2024-09-11T07:43:39.127 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4746999999999995e-05 2024-09-11T07:43:39.127 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.127 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.127 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:39.127 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.773036+0000", 2024-09-11T07:43:39.127 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4199000000000002e-05 2024-09-11T07:43:39.127 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.128 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:39.128 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.128 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.128 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.128 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:86233 9.e 9:746045db:::benchmark_data_smithi022_133943_object86232:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:43:39.128 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:27.532483+0000", 2024-09-11T07:43:39.128 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 311.57482722899999, 2024-09-11T07:43:39.128 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18061364699999999, 2024-09-11T07:43:39.128 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:39.128 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:39.128 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:39.128 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:39.128 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:39.129 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 86233 2024-09-11T07:43:39.129 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.129 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:39.129 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.129 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:39.129 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532483+0000", 2024-09-11T07:43:39.129 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.129 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.129 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.129 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:39.129 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532483+0000", 2024-09-11T07:43:39.129 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.129 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.130 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.130 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:39.130 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532482+0000", 2024-09-11T07:43:39.130 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:43:39.130 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.130 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.130 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:39.130 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532896+0000", 2024-09-11T07:43:39.130 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000413557 2024-09-11T07:43:39.130 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.130 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.130 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:39.130 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532910+0000", 2024-09-11T07:43:39.131 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4083999999999999e-05 2024-09-11T07:43:39.131 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.131 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.131 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:39.131 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532916+0000", 2024-09-11T07:43:39.131 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9939999999999997e-06 2024-09-11T07:43:39.131 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.131 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.131 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.131 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532971+0000", 2024-09-11T07:43:39.131 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5179999999999997e-05 2024-09-11T07:43:39.131 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.131 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.132 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:39.132 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.533088+0000", 2024-09-11T07:43:39.132 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000116618 2024-09-11T07:43:39.132 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.132 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.132 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,4", 2024-09-11T07:43:39.132 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.533232+0000", 2024-09-11T07:43:39.132 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000144191 2024-09-11T07:43:39.132 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.133 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.133 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:39.133 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.535865+0000", 2024-09-11T07:43:39.133 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0026336250000000001 2024-09-11T07:43:39.133 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.133 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.133 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.133 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.538553+0000", 2024-09-11T07:43:39.133 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0026877300000000002 2024-09-11T07:43:39.133 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.133 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.133 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.134 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.713002+0000", 2024-09-11T07:43:39.134 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17444870100000001 2024-09-11T07:43:39.134 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.134 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.134 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:39.134 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.713065+0000", 2024-09-11T07:43:39.134 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3140999999999997e-05 2024-09-11T07:43:39.134 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.134 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.134 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:39.134 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.713097+0000", 2024-09-11T07:43:39.134 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2076000000000003e-05 2024-09-11T07:43:39.134 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.135 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:39.135 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.135 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.135 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.135 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:86237 9.3 9:c1b2182b:::benchmark_data_smithi022_133943_object86236:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:43:39.135 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:27.535933+0000", 2024-09-11T07:43:39.135 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 311.57137760199998, 2024-09-11T07:43:39.135 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.23897332099999999, 2024-09-11T07:43:39.135 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:39.135 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:39.135 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:39.135 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:39.135 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:39.135 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 86237 2024-09-11T07:43:39.135 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.135 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:39.136 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.136 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:39.136 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.535933+0000", 2024-09-11T07:43:39.136 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.136 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.136 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.136 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:39.136 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.535933+0000", 2024-09-11T07:43:39.136 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.136 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.136 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.136 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:39.136 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.535931+0000", 2024-09-11T07:43:39.136 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:43:39.137 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.137 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.137 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:39.137 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.535972+0000", 2024-09-11T07:43:39.137 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0861000000000001e-05 2024-09-11T07:43:39.137 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.137 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.137 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:39.137 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.535985+0000", 2024-09-11T07:43:39.137 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2336e-05 2024-09-11T07:43:39.137 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.137 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.137 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:39.137 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.535990+0000", 2024-09-11T07:43:39.137 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.772e-06 2024-09-11T07:43:39.138 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.138 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.138 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.138 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536127+0000", 2024-09-11T07:43:39.138 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013698900000000001 2024-09-11T07:43:39.138 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.138 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.138 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:39.138 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536211+0000", 2024-09-11T07:43:39.138 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3901999999999997e-05 2024-09-11T07:43:39.138 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.138 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.138 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:43:39.138 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536455+0000", 2024-09-11T07:43:39.139 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00024371199999999999 2024-09-11T07:43:39.139 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.139 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.139 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:39.139 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.537667+0000", 2024-09-11T07:43:39.139 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00121188 2024-09-11T07:43:39.139 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.139 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.139 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.139 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.715691+0000", 2024-09-11T07:43:39.139 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.178024185 2024-09-11T07:43:39.139 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.139 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.139 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.140 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.774831+0000", 2024-09-11T07:43:39.140 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.059139518000000002 2024-09-11T07:43:39.140 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.140 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.140 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:39.140 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.774874+0000", 2024-09-11T07:43:39.140 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3080000000000001e-05 2024-09-11T07:43:39.140 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.140 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.140 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:39.140 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.774906+0000", 2024-09-11T07:43:39.140 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2827000000000003e-05 2024-09-11T07:43:39.140 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.141 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:39.141 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.141 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.141 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.141 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:86238 9.e 9:76c6605d:::benchmark_data_smithi022_133943_object86237:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:43:39.141 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:27.536029+0000", 2024-09-11T07:43:39.141 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 311.571281484, 2024-09-11T07:43:39.141 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18003709600000001, 2024-09-11T07:43:39.141 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:39.141 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:39.141 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:39.141 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:39.141 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:39.141 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 86238 2024-09-11T07:43:39.142 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.142 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:39.142 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.142 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:39.142 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536029+0000", 2024-09-11T07:43:39.142 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.142 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.142 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.142 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:39.142 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536029+0000", 2024-09-11T07:43:39.142 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.142 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.142 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.143 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:39.143 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536028+0000", 2024-09-11T07:43:39.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:39.144 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:43:39.144 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.144 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.145 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:39.145 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536060+0000", 2024-09-11T07:43:39.145 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1294000000000002e-05 2024-09-11T07:43:39.145 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.145 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.145 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:39.145 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536072+0000", 2024-09-11T07:43:39.145 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2031000000000001e-05 2024-09-11T07:43:39.145 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.145 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.145 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:39.145 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536077+0000", 2024-09-11T07:43:39.145 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0939999999999998e-06 2024-09-11T07:43:39.146 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.146 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.146 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.146 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536120+0000", 2024-09-11T07:43:39.146 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3315000000000001e-05 2024-09-11T07:43:39.146 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.146 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.146 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:39.146 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536227+0000", 2024-09-11T07:43:39.146 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010718 2024-09-11T07:43:39.146 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.146 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.146 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,4", 2024-09-11T07:43:39.147 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536331+0000", 2024-09-11T07:43:39.147 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000104227 2024-09-11T07:43:39.147 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.147 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.147 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:39.147 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.538631+0000", 2024-09-11T07:43:39.147 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002299784 2024-09-11T07:43:39.147 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.147 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.147 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.147 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.538870+0000", 2024-09-11T07:43:39.147 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00023881600000000001 2024-09-11T07:43:39.147 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.147 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.147 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.148 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.715974+0000", 2024-09-11T07:43:39.148 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17710394400000001 2024-09-11T07:43:39.148 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.148 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.148 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:39.148 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.716028+0000", 2024-09-11T07:43:39.148 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3665999999999997e-05 2024-09-11T07:43:39.148 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.148 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.148 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:39.149 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.716066+0000", 2024-09-11T07:43:39.149 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8702000000000003e-05 2024-09-11T07:43:39.149 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.149 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:39.149 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.149 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.149 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.149 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:86241 9.13 9:cc909b6c:::benchmark_data_smithi022_133943_object86240:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:43:39.149 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:27.536369+0000", 2024-09-11T07:43:39.149 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 311.57094199800002, 2024-09-11T07:43:39.149 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.23673264699999999, 2024-09-11T07:43:39.149 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:39.149 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:39.150 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:39.150 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:39.150 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:39.150 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 86241 2024-09-11T07:43:39.150 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.150 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:39.150 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.150 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:39.150 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536369+0000", 2024-09-11T07:43:39.150 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.150 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.150 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.150 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:39.150 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536369+0000", 2024-09-11T07:43:39.151 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.151 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.151 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.151 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:39.151 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536367+0000", 2024-09-11T07:43:39.151 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:43:39.151 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.151 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.151 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:39.151 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536426+0000", 2024-09-11T07:43:39.151 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8866999999999997e-05 2024-09-11T07:43:39.151 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.151 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.151 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:39.152 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536434+0000", 2024-09-11T07:43:39.152 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9389999999999997e-06 2024-09-11T07:43:39.152 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.152 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.152 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:39.152 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536440+0000", 2024-09-11T07:43:39.152 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2769999999999996e-06 2024-09-11T07:43:39.152 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.152 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.152 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.152 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.537555+0000", 2024-09-11T07:43:39.152 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0011153420000000001 2024-09-11T07:43:39.152 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.152 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.153 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:39.153 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.537650+0000", 2024-09-11T07:43:39.153 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4883999999999995e-05 2024-09-11T07:43:39.153 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.153 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.153 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:43:39.153 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.537882+0000", 2024-09-11T07:43:39.153 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000231952 2024-09-11T07:43:39.153 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.153 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.153 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:39.153 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.538889+0000", 2024-09-11T07:43:39.153 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0010068939999999999 2024-09-11T07:43:39.153 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.154 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.154 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.154 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.715709+0000", 2024-09-11T07:43:39.154 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17682025700000001 2024-09-11T07:43:39.154 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.154 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.154 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.154 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.772956+0000", 2024-09-11T07:43:39.154 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.057247167000000002 2024-09-11T07:43:39.154 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.154 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.154 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:39.154 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.773061+0000", 2024-09-11T07:43:39.154 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010540299999999999 2024-09-11T07:43:39.154 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.155 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.155 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:39.155 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.773101+0000", 2024-09-11T07:43:39.155 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9910999999999998e-05 2024-09-11T07:43:39.155 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.155 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:39.155 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.155 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.155 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.155 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87203 9.13 9:cc4de378:::benchmark_data_smithi022_133943_object87202:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:43:39.155 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.339440+0000", 2024-09-11T07:43:39.155 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 310.76787038499998, 2024-09-11T07:43:39.155 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1017081860000002, 2024-09-11T07:43:39.155 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:39.155 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:43:39.156 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:39.156 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:39.156 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:39.156 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87203 2024-09-11T07:43:39.156 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.156 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:39.156 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.156 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:39.156 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339440+0000", 2024-09-11T07:43:39.156 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.156 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.156 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.156 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:39.157 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339440+0000", 2024-09-11T07:43:39.157 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.157 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.157 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.157 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:39.157 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339440+0000", 2024-09-11T07:43:39.157 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:43:39.157 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.157 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.157 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:39.157 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339474+0000", 2024-09-11T07:43:39.157 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4149000000000001e-05 2024-09-11T07:43:39.157 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.157 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.158 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:39.158 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339480+0000", 2024-09-11T07:43:39.158 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3550000000000001e-06 2024-09-11T07:43:39.158 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.158 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.158 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:39.158 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339484+0000", 2024-09-11T07:43:39.158 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1239999999999998e-06 2024-09-11T07:43:39.158 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.158 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.158 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.158 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339567+0000", 2024-09-11T07:43:39.158 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3145000000000004e-05 2024-09-11T07:43:39.159 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.159 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.159 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:39.159 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339673+0000", 2024-09-11T07:43:39.159 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010586699999999999 2024-09-11T07:43:39.159 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.159 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.159 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:43:39.159 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339947+0000", 2024-09-11T07:43:39.159 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00027409800000000002 2024-09-11T07:43:39.159 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.159 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.159 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:39.159 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.341767+0000", 2024-09-11T07:43:39.160 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001819883 2024-09-11T07:43:39.160 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.160 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.160 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.160 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.342624+0000", 2024-09-11T07:43:39.160 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00085697099999999999 2024-09-11T07:43:39.160 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.160 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.160 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.160 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.347306+0000", 2024-09-11T07:43:39.160 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0046816230000000002 2024-09-11T07:43:39.160 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.160 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.161 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:43:39.161 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.347315+0000", 2024-09-11T07:43:39.161 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.1600000000000004e-06 2024-09-11T07:43:39.161 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.161 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.161 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.161 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.346303+0000", 2024-09-11T07:43:39.161 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.99898775399999995 2024-09-11T07:43:39.161 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.161 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.161 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:43:39.161 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.346312+0000", 2024-09-11T07:43:39.161 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.6889999999999999e-06 2024-09-11T07:43:39.162 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.162 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.162 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.162 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.436501+0000", 2024-09-11T07:43:39.162 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.090188951 2024-09-11T07:43:39.162 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.162 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.162 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for degraded object", 2024-09-11T07:43:39.162 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.437198+0000", 2024-09-11T07:43:39.162 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00069633100000000001 2024-09-11T07:43:39.162 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.162 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.162 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.162 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.441056+0000", 2024-09-11T07:43:39.163 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0038578359999999999 2024-09-11T07:43:39.163 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.163 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.163 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:39.163 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.441149+0000", 2024-09-11T07:43:39.163 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2905000000000006e-05 2024-09-11T07:43:39.163 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.163 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:39.163 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.163 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.163 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.164 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87197 9.17 9:ede1dd36:::benchmark_data_smithi022_133943_object87196:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:43:39.164 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.345260+0000", 2024-09-11T07:43:39.164 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 310.76205043599998, 2024-09-11T07:43:39.164 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0776407649999999, 2024-09-11T07:43:39.164 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:39.164 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:43:39.164 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:39.164 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:39.164 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:39.164 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87197 2024-09-11T07:43:39.164 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.165 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:39.165 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.165 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:39.165 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.345260+0000", 2024-09-11T07:43:39.165 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.165 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.165 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.165 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:39.165 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.345260+0000", 2024-09-11T07:43:39.165 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.165 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.165 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.165 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:39.166 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.345258+0000", 2024-09-11T07:43:39.166 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:43:39.166 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.166 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.166 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:39.166 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.345346+0000", 2024-09-11T07:43:39.166 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7859e-05 2024-09-11T07:43:39.166 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.166 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.166 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:39.166 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.345356+0000", 2024-09-11T07:43:39.166 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0205e-05 2024-09-11T07:43:39.166 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.166 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.166 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:39.167 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.345361+0000", 2024-09-11T07:43:39.167 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4199999999999998e-06 2024-09-11T07:43:39.167 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.167 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.167 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.167 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.347877+0000", 2024-09-11T07:43:39.167 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0025154270000000002 2024-09-11T07:43:39.167 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.167 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.167 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:43:39.167 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.347884+0000", 2024-09-11T07:43:39.167 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4699999999999996e-06 2024-09-11T07:43:39.167 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.167 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.167 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.167 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.344661+0000", 2024-09-11T07:43:39.168 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.99677682199999995 2024-09-11T07:43:39.168 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.168 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.168 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:43:39.168 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.344670+0000", 2024-09-11T07:43:39.168 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3449999999999995e-06 2024-09-11T07:43:39.168 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.168 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.168 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.168 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.422818+0000", 2024-09-11T07:43:39.168 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0781474900000001 2024-09-11T07:43:39.168 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.168 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.168 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:39.169 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.422901+0000", 2024-09-11T07:43:39.169 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3071000000000005e-05 2024-09-11T07:43:39.169 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.169 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:39.169 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.169 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.169 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.169 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87222 9.18 9.9a7e3098 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:43:39.169 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.362550+0000", 2024-09-11T07:43:39.169 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 310.74476022099998, 2024-09-11T07:43:39.169 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.98180590199999995, 2024-09-11T07:43:39.169 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:39.169 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:43:39.169 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:39.170 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:39.170 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:39.170 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87222 2024-09-11T07:43:39.170 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.170 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:39.170 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.170 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:39.170 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.362550+0000", 2024-09-11T07:43:39.170 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.170 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.170 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.170 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:39.170 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.362550+0000", 2024-09-11T07:43:39.170 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.171 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.171 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.171 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:39.171 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.362549+0000", 2024-09-11T07:43:39.171 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:43:39.171 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.171 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.171 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:39.171 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.362658+0000", 2024-09-11T07:43:39.171 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000108529 2024-09-11T07:43:39.171 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.171 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.171 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:39.171 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.362667+0000", 2024-09-11T07:43:39.172 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9020000000000005e-06 2024-09-11T07:43:39.172 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.172 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.172 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:39.172 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.362670+0000", 2024-09-11T07:43:39.172 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4340000000000001e-06 2024-09-11T07:43:39.172 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.172 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.172 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.172 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.362714+0000", 2024-09-11T07:43:39.172 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3615000000000002e-05 2024-09-11T07:43:39.172 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.172 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.172 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:43:39.173 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.362719+0000", 2024-09-11T07:43:39.173 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0350000000000001e-06 2024-09-11T07:43:39.173 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.173 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.173 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.173 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.344348+0000", 2024-09-11T07:43:39.173 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.98162881999999996 2024-09-11T07:43:39.173 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.173 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.173 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:39.173 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.344356+0000", 2024-09-11T07:43:39.173 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6580000000000008e-06 2024-09-11T07:43:39.173 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.173 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:39.174 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.174 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.174 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.174 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87236 9.15 9:aa3be195:::benchmark_data_smithi022_133943_object87235:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:43:39.174 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.369488+0000", 2024-09-11T07:43:39.174 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 310.73782314300001, 2024-09-11T07:43:39.174 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.156802254, 2024-09-11T07:43:39.174 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:39.174 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:39.174 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:39.174 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:39.174 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:39.174 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87236 2024-09-11T07:43:39.175 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.175 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:39.175 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.175 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:39.175 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369488+0000", 2024-09-11T07:43:39.175 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.175 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.175 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.175 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:39.175 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369488+0000", 2024-09-11T07:43:39.175 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.175 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.175 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.175 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:39.176 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369486+0000", 2024-09-11T07:43:39.176 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:43:39.176 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.176 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.176 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:39.176 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369534+0000", 2024-09-11T07:43:39.176 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8127000000000002e-05 2024-09-11T07:43:39.176 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.176 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.176 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:39.176 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369545+0000", 2024-09-11T07:43:39.176 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0920999999999999e-05 2024-09-11T07:43:39.176 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.176 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.177 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:39.177 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369549+0000", 2024-09-11T07:43:39.177 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4580000000000003e-06 2024-09-11T07:43:39.177 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.177 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.177 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.177 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369610+0000", 2024-09-11T07:43:39.177 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0631000000000003e-05 2024-09-11T07:43:39.177 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.177 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.177 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:43:39.177 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369618+0000", 2024-09-11T07:43:39.177 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7420000000000003e-06 2024-09-11T07:43:39.177 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.178 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.178 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.178 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.345692+0000", 2024-09-11T07:43:39.178 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.97607389 2024-09-11T07:43:39.178 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.178 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.178 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:43:39.178 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.345702+0000", 2024-09-11T07:43:39.178 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0086e-05 2024-09-11T07:43:39.178 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.178 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.178 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.178 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.510602+0000", 2024-09-11T07:43:39.179 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1648999250000001 2024-09-11T07:43:39.179 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.179 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.179 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:39.179 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.510767+0000", 2024-09-11T07:43:39.179 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000164656 2024-09-11T07:43:39.179 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.179 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.179 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:43:39.179 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.511002+0000", 2024-09-11T07:43:39.179 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000234699 2024-09-11T07:43:39.179 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.179 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.179 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:39.180 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.514166+0000", 2024-09-11T07:43:39.180 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0031640069999999999 2024-09-11T07:43:39.180 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.180 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.180 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.180 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.523959+0000", 2024-09-11T07:43:39.180 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0097934619999999993 2024-09-11T07:43:39.180 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.180 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.180 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.180 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.523981+0000", 2024-09-11T07:43:39.180 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.181e-05 2024-09-11T07:43:39.180 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.180 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.181 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:39.181 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.524012+0000", 2024-09-11T07:43:39.181 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1241000000000001e-05 2024-09-11T07:43:39.181 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.181 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.181 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:39.181 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.526290+0000", 2024-09-11T07:43:39.181 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0022775819999999998 2024-09-11T07:43:39.181 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.181 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:39.181 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.181 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.181 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.182 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87239 9.17 9:eb730478:::benchmark_data_smithi022_133943_object87238:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:43:39.182 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.369713+0000", 2024-09-11T07:43:39.182 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 310.73759802799998, 2024-09-11T07:43:39.182 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0664777230000002, 2024-09-11T07:43:39.182 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:39.182 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:39.182 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:39.182 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:39.182 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:39.182 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87239 2024-09-11T07:43:39.182 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.182 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:39.182 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.183 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:39.183 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369713+0000", 2024-09-11T07:43:39.183 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.183 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.183 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.183 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:39.183 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369713+0000", 2024-09-11T07:43:39.183 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.183 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.183 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.183 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:39.183 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369712+0000", 2024-09-11T07:43:39.183 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:43:39.183 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.183 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.184 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:39.184 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369781+0000", 2024-09-11T07:43:39.184 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9408999999999999e-05 2024-09-11T07:43:39.184 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.184 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.184 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:39.184 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369796+0000", 2024-09-11T07:43:39.184 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4355999999999999e-05 2024-09-11T07:43:39.184 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.184 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.184 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:39.184 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369801+0000", 2024-09-11T07:43:39.184 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1410000000000004e-06 2024-09-11T07:43:39.185 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.185 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.185 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.185 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369852+0000", 2024-09-11T07:43:39.185 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0983000000000003e-05 2024-09-11T07:43:39.185 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.185 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.185 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:43:39.185 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369860+0000", 2024-09-11T07:43:39.185 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6200000000000005e-06 2024-09-11T07:43:39.185 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.185 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.185 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.185 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.344696+0000", 2024-09-11T07:43:39.186 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.97483578000000004 2024-09-11T07:43:39.186 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.186 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.186 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:43:39.187 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:38 smithi022 ceph-mon[29462]: from='client.25787 ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "4.f", "id": [1, 3, 4]}]': finished 2024-09-11T07:43:39.187 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:38 smithi022 ceph-mon[29462]: osdmap e369: 8 total, 8 up, 8 in 2024-09-11T07:43:39.187 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:38 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3976604641' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:43:39.188 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:43:39.188 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:43:39.188 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:38 smithi022 ceph-mon[29462]: pgmap v795: 148 pgs: 148 active+clean; 7.1 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 639 B/s rd, 46 MiB/s wr, 739 op/s 2024-09-11T07:43:39.188 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:38 smithi022 ceph-mon[37330]: from='client.25787 ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "4.f", "id": [1, 3, 4]}]': finished 2024-09-11T07:43:39.188 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:38 smithi022 ceph-mon[37330]: osdmap e369: 8 total, 8 up, 8 in 2024-09-11T07:43:39.188 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:38 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3976604641' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:43:39.188 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:43:39.189 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:43:39.189 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:38 smithi022 ceph-mon[37330]: pgmap v795: 148 pgs: 148 active+clean; 7.1 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 639 B/s rd, 46 MiB/s wr, 739 op/s 2024-09-11T07:43:39.189 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.344703+0000", 2024-09-11T07:43:39.189 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4240000000000004e-06 2024-09-11T07:43:39.189 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.189 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.189 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.189 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.422922+0000", 2024-09-11T07:43:39.189 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0782185689999999 2024-09-11T07:43:39.189 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.190 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.190 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:39.190 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.423049+0000", 2024-09-11T07:43:39.190 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012664800000000001 2024-09-11T07:43:39.190 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.190 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.190 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,1,3,5", 2024-09-11T07:43:39.190 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.423321+0000", 2024-09-11T07:43:39.190 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000272355 2024-09-11T07:43:39.190 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.190 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.190 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.190 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.428425+0000", 2024-09-11T07:43:39.191 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0051037970000000002 2024-09-11T07:43:39.191 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.191 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.191 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.191 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.429833+0000", 2024-09-11T07:43:39.191 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0014079769999999999 2024-09-11T07:43:39.191 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.191 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.191 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.191 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.435362+0000", 2024-09-11T07:43:39.191 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0055289069999999996 2024-09-11T07:43:39.191 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.191 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.191 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.192 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.435951+0000", 2024-09-11T07:43:39.192 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000589158 2024-09-11T07:43:39.192 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.192 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.192 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:39.192 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.435960+0000", 2024-09-11T07:43:39.192 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9600000000000006e-06 2024-09-11T07:43:39.192 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.192 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.192 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:39.192 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.436059+0000", 2024-09-11T07:43:39.192 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8784999999999997e-05 2024-09-11T07:43:39.192 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.193 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.193 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:39.193 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.436190+0000", 2024-09-11T07:43:39.193 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000131772 2024-09-11T07:43:39.193 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.193 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:39.193 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.193 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.193 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.193 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87240 9.18 9.2c746ab8 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:43:39.193 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.372326+0000", 2024-09-11T07:43:39.193 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 310.73498463300001, 2024-09-11T07:43:39.194 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.97205600599999997, 2024-09-11T07:43:39.194 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:39.194 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:43:39.194 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:39.194 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:39.194 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:39.194 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87240 2024-09-11T07:43:39.194 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.194 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:39.194 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.194 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:39.194 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372326+0000", 2024-09-11T07:43:39.194 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.194 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.195 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.195 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:39.195 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372326+0000", 2024-09-11T07:43:39.195 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.195 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.195 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.195 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:39.195 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372324+0000", 2024-09-11T07:43:39.195 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:43:39.195 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.195 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.195 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:39.195 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372381+0000", 2024-09-11T07:43:39.195 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7188000000000002e-05 2024-09-11T07:43:39.196 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.196 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.196 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:39.196 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372394+0000", 2024-09-11T07:43:39.196 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2901999999999999e-05 2024-09-11T07:43:39.196 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.196 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.196 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:39.196 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372400+0000", 2024-09-11T07:43:39.196 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.699e-06 2024-09-11T07:43:39.196 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.196 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.196 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.197 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372439+0000", 2024-09-11T07:43:39.197 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9839000000000003e-05 2024-09-11T07:43:39.197 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.197 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.197 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:43:39.197 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372445+0000", 2024-09-11T07:43:39.197 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9490000000000001e-06 2024-09-11T07:43:39.197 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.197 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.197 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.197 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.344376+0000", 2024-09-11T07:43:39.197 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.97193081100000001 2024-09-11T07:43:39.197 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.198 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.198 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:39.198 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.344382+0000", 2024-09-11T07:43:39.198 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9070000000000004e-06 2024-09-11T07:43:39.198 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.198 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:39.198 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.198 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.198 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.198 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87250 9.15 9:ac3c2ae6:::benchmark_data_smithi022_133943_object87249:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:43:39.198 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.377199+0000", 2024-09-11T07:43:39.198 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 310.73011220500001, 2024-09-11T07:43:39.198 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1512879869999999, 2024-09-11T07:43:39.198 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:39.199 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:39.199 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:39.199 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:39.199 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:39.199 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87250 2024-09-11T07:43:39.199 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.199 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:39.199 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.199 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:39.199 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.377199+0000", 2024-09-11T07:43:39.199 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.199 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.199 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.199 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:39.199 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.377199+0000", 2024-09-11T07:43:39.200 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.200 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.200 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.200 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:39.200 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.377197+0000", 2024-09-11T07:43:39.200 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:43:39.200 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.200 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.200 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:39.200 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.377241+0000", 2024-09-11T07:43:39.200 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3498999999999998e-05 2024-09-11T07:43:39.200 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.200 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.201 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:39.201 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.377249+0000", 2024-09-11T07:43:39.201 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0989999999999999e-06 2024-09-11T07:43:39.201 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.201 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.201 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:39.201 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.377254+0000", 2024-09-11T07:43:39.201 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1220000000000003e-06 2024-09-11T07:43:39.201 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.201 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.201 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.201 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.377302+0000", 2024-09-11T07:43:39.201 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7479999999999999e-05 2024-09-11T07:43:39.201 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.202 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.202 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:43:39.202 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.377311+0000", 2024-09-11T07:43:39.202 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2359999999999993e-06 2024-09-11T07:43:39.202 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.202 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.202 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.202 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.345770+0000", 2024-09-11T07:43:39.202 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.968458763 2024-09-11T07:43:39.202 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.202 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.202 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:43:39.202 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.345779+0000", 2024-09-11T07:43:39.202 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0869999999999996e-06 2024-09-11T07:43:39.203 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.203 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.203 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.203 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.512718+0000", 2024-09-11T07:43:39.203 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.166939323 2024-09-11T07:43:39.203 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.203 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.203 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:39.203 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.512902+0000", 2024-09-11T07:43:39.203 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018403899999999999 2024-09-11T07:43:39.203 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.203 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.203 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:43:39.204 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.513149+0000", 2024-09-11T07:43:39.204 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00024737899999999999 2024-09-11T07:43:39.204 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.204 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.204 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:39.204 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.519581+0000", 2024-09-11T07:43:39.204 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0064314819999999997 2024-09-11T07:43:39.204 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.204 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.204 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.204 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.526384+0000", 2024-09-11T07:43:39.204 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0068032070000000004 2024-09-11T07:43:39.204 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.204 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.205 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.205 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.526419+0000", 2024-09-11T07:43:39.205 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5036000000000001e-05 2024-09-11T07:43:39.205 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.205 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.205 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:39.205 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.526478+0000", 2024-09-11T07:43:39.205 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8462999999999999e-05 2024-09-11T07:43:39.205 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.205 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.205 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:39.205 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.528486+0000", 2024-09-11T07:43:39.205 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0020089330000000001 2024-09-11T07:43:39.205 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.206 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:39.206 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.206 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.206 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.206 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87257 9.13 9:ca17476d:::benchmark_data_smithi022_133943_object87256:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:43:39.206 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.384938+0000", 2024-09-11T07:43:39.206 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 310.722373103, 2024-09-11T07:43:39.206 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0584020930000002, 2024-09-11T07:43:39.206 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:39.206 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:39.206 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:39.206 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:39.206 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:39.206 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87257 2024-09-11T07:43:39.207 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.207 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:39.207 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.207 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:39.207 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.384938+0000", 2024-09-11T07:43:39.207 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.207 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.207 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.207 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:39.207 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.384938+0000", 2024-09-11T07:43:39.208 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.208 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.208 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.208 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:39.208 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.384936+0000", 2024-09-11T07:43:39.208 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:43:39.208 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.208 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.208 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:39.208 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.384998+0000", 2024-09-11T07:43:39.208 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1383000000000005e-05 2024-09-11T07:43:39.208 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.208 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.209 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:39.209 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.385007+0000", 2024-09-11T07:43:39.209 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3370000000000006e-06 2024-09-11T07:43:39.209 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.209 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.209 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:39.209 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.385011+0000", 2024-09-11T07:43:39.209 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9840000000000003e-06 2024-09-11T07:43:39.209 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.209 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.209 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.209 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.385067+0000", 2024-09-11T07:43:39.209 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6425999999999998e-05 2024-09-11T07:43:39.209 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.210 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.210 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:43:39.210 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.385079+0000", 2024-09-11T07:43:39.210 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.155e-05 2024-09-11T07:43:39.210 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.210 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.210 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.210 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.346702+0000", 2024-09-11T07:43:39.210 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.96162294299999995 2024-09-11T07:43:39.210 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.210 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.210 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:43:39.210 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.346713+0000", 2024-09-11T07:43:39.210 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0688e-05 2024-09-11T07:43:39.211 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.211 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.211 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.211 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.437750+0000", 2024-09-11T07:43:39.211 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0910372879999999 2024-09-11T07:43:39.211 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.211 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.211 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:39.211 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.437940+0000", 2024-09-11T07:43:39.211 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019046499999999999 2024-09-11T07:43:39.211 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.211 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.211 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3,5,6", 2024-09-11T07:43:39.211 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.438830+0000", 2024-09-11T07:43:39.211 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00088921499999999995 2024-09-11T07:43:39.212 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.212 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.212 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:39.212 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.440627+0000", 2024-09-11T07:43:39.212 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001797873 2024-09-11T07:43:39.212 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.212 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.212 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.212 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.441186+0000", 2024-09-11T07:43:39.212 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00055897300000000004 2024-09-11T07:43:39.212 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.212 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.212 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.213 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.441752+0000", 2024-09-11T07:43:39.213 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00056608299999999999 2024-09-11T07:43:39.213 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.213 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.213 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.213 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.442310+0000", 2024-09-11T07:43:39.213 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00055787000000000002 2024-09-11T07:43:39.213 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.213 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.213 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.213 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.443120+0000", 2024-09-11T07:43:39.213 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00080973899999999997 2024-09-11T07:43:39.213 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.213 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.214 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:39.214 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.443167+0000", 2024-09-11T07:43:39.214 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7302999999999998e-05 2024-09-11T07:43:39.214 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.214 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.214 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:39.214 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.443340+0000", 2024-09-11T07:43:39.214 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017236000000000001 2024-09-11T07:43:39.214 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.214 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:39.214 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.214 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.214 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.214 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196445 9.17 9:edf5eae4:::benchmark_data_smithi022_133943_object69854:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:43:39.214 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.519449+0000", 2024-09-11T07:43:39.215 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 201.58786185100001, 2024-09-11T07:43:39.215 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.161483352, 2024-09-11T07:43:39.215 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:39.215 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:39.215 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:39.215 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:43:39.215 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:43:39.215 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196445 2024-09-11T07:43:39.215 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.215 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:39.215 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.215 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:39.215 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.519449+0000", 2024-09-11T07:43:39.215 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.216 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.216 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.216 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:39.216 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.519449+0000", 2024-09-11T07:43:39.216 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.216 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.216 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.216 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:39.216 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.519447+0000", 2024-09-11T07:43:39.216 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:43:39.216 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.217 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.217 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:39.217 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.519453+0000", 2024-09-11T07:43:39.217 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0619999999999997e-06 2024-09-11T07:43:39.217 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.217 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.217 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:39.217 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.519462+0000", 2024-09-11T07:43:39.217 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.1009999999999998e-06 2024-09-11T07:43:39.217 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.217 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.217 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:39.217 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.519468+0000", 2024-09-11T07:43:39.218 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7950000000000004e-06 2024-09-11T07:43:39.218 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.218 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.218 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.218 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.522372+0000", 2024-09-11T07:43:39.218 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0029042389999999999 2024-09-11T07:43:39.218 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.218 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.218 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:39.218 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.522451+0000", 2024-09-11T07:43:39.218 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9005000000000007e-05 2024-09-11T07:43:39.218 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.218 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.218 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,1,2,3,5", 2024-09-11T07:43:39.218 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.658722+0000", 2024-09-11T07:43:39.219 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.136270488 2024-09-11T07:43:39.219 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.219 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.219 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.219 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.665102+0000", 2024-09-11T07:43:39.219 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0063799490000000002 2024-09-11T07:43:39.219 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.219 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.219 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.219 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.665138+0000", 2024-09-11T07:43:39.219 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6230999999999999e-05 2024-09-11T07:43:39.219 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.219 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.219 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.220 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.665177+0000", 2024-09-11T07:43:39.220 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9316000000000001e-05 2024-09-11T07:43:39.220 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.220 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.220 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.220 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.670335+0000", 2024-09-11T07:43:39.220 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0051581659999999996 2024-09-11T07:43:39.220 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.220 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.220 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:39.220 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.670342+0000", 2024-09-11T07:43:39.220 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0600000000000002e-06 2024-09-11T07:43:39.220 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.220 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.221 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.221 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.680708+0000", 2024-09-11T07:43:39.221 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.010365592 2024-09-11T07:43:39.221 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.221 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.221 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:39.221 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.680765+0000", 2024-09-11T07:43:39.221 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6641999999999997e-05 2024-09-11T07:43:39.221 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.221 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.221 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:39.221 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.680932+0000", 2024-09-11T07:43:39.221 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016753800000000001 2024-09-11T07:43:39.221 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.222 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:39.222 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.222 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.222 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.222 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.15504.0:196449 9.c e299/247 9:3c3917ff:::benchmark_data_smithi022_133943_object69858:head v 299'12118, mlcod=299'12114)", 2024-09-11T07:43:39.222 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.523914+0000", 2024-09-11T07:43:39.222 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 201.58339685199999, 2024-09-11T07:43:39.222 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.143445717, 2024-09-11T07:43:39.222 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:39.222 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:39.222 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:39.222 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.222 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:39.223 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.523914+0000", 2024-09-11T07:43:39.223 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.223 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.223 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.223 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:39.223 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.523914+0000", 2024-09-11T07:43:39.223 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.223 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.223 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.223 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:39.223 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.523912+0000", 2024-09-11T07:43:39.223 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:43:39.223 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.223 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.224 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:39.224 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.523923+0000", 2024-09-11T07:43:39.224 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1362999999999999e-05 2024-09-11T07:43:39.224 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.224 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.224 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:39.224 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.658139+0000", 2024-09-11T07:43:39.224 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13421586299999999 2024-09-11T07:43:39.224 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.224 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.224 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:39.224 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.658149+0000", 2024-09-11T07:43:39.224 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4360000000000008e-06 2024-09-11T07:43:39.224 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.225 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.225 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.225 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.660255+0000", 2024-09-11T07:43:39.225 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0021059709999999999 2024-09-11T07:43:39.225 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.225 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.225 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:39.225 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.660276+0000", 2024-09-11T07:43:39.225 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0928e-05 2024-09-11T07:43:39.225 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.225 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.225 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:39.225 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.667249+0000", 2024-09-11T07:43:39.225 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0069737710000000001 2024-09-11T07:43:39.226 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.226 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.226 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:39.226 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.667360+0000", 2024-09-11T07:43:39.226 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000110208 2024-09-11T07:43:39.226 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.226 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:39.226 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.226 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.226 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.226 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25706.0:27077 10.c 10:3438f62c:::benchmark_data_smithi022_163866_object27076:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e354)", 2024-09-11T07:43:39.226 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:26.457049+0000", 2024-09-11T07:43:39.226 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 72.650261873000005, 2024-09-11T07:43:39.226 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.31959205600000001, 2024-09-11T07:43:39.227 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:39.227 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:39.227 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:39.227 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25706", 2024-09-11T07:43:39.227 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:43:39.227 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 27077 2024-09-11T07:43:39.227 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.227 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:39.227 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.227 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:39.227 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:26.457049+0000", 2024-09-11T07:43:39.227 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.227 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.227 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.228 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:39.228 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:26.457049+0000", 2024-09-11T07:43:39.228 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.228 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.228 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.228 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:39.228 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:26.457046+0000", 2024-09-11T07:43:39.228 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:43:39.228 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.228 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.228 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:39.228 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:26.457102+0000", 2024-09-11T07:43:39.228 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5671999999999998e-05 2024-09-11T07:43:39.229 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.229 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.229 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:39.229 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:26.457116+0000", 2024-09-11T07:43:39.229 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4021e-05 2024-09-11T07:43:39.229 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.229 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.229 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:39.229 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:26.457122+0000", 2024-09-11T07:43:39.229 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2559999999999999e-06 2024-09-11T07:43:39.229 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.229 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.229 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.229 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:26.457170+0000", 2024-09-11T07:43:39.230 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7766000000000003e-05 2024-09-11T07:43:39.230 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.230 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.230 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:39.230 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:26.457308+0000", 2024-09-11T07:43:39.230 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013831600000000001 2024-09-11T07:43:39.230 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.230 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.230 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,4", 2024-09-11T07:43:39.230 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:26.457452+0000", 2024-09-11T07:43:39.230 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000143876 2024-09-11T07:43:39.230 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.230 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.231 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:39.231 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:26.458544+0000", 2024-09-11T07:43:39.231 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0010920909999999999 2024-09-11T07:43:39.231 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.231 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.231 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.231 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:26.458703+0000", 2024-09-11T07:43:39.231 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015906500000000001 2024-09-11T07:43:39.231 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.231 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.231 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.231 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:26.776536+0000", 2024-09-11T07:43:39.231 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.31783249800000002 2024-09-11T07:43:39.232 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.232 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.232 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:39.232 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:26.776598+0000", 2024-09-11T07:43:39.232 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2163999999999996e-05 2024-09-11T07:43:39.232 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.232 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.232 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:39.232 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:26.776641+0000", 2024-09-11T07:43:39.232 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2778000000000003e-05 2024-09-11T07:43:39.232 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.232 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:39.232 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.233 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.233 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.233 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25706.0:58355 10.c 10:3da0e5a9:::benchmark_data_smithi022_163866_object58354:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e355)", 2024-09-11T07:43:39.233 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:45.068457+0000", 2024-09-11T07:43:39.233 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 54.038853359000001, 2024-09-11T07:43:39.233 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14367881599999999, 2024-09-11T07:43:39.233 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:39.233 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:39.233 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:39.233 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25706", 2024-09-11T07:43:39.233 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:43:39.233 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 58355 2024-09-11T07:43:39.233 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.233 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:39.234 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.234 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:39.234 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:45.068457+0000", 2024-09-11T07:43:39.234 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.234 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.234 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.234 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:39.234 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:45.068457+0000", 2024-09-11T07:43:39.234 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.234 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.234 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.234 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:39.234 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:45.068455+0000", 2024-09-11T07:43:39.235 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:43:39.235 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.235 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.235 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:39.235 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:45.068496+0000", 2024-09-11T07:43:39.235 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1355e-05 2024-09-11T07:43:39.235 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.235 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.235 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:39.235 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:45.068514+0000", 2024-09-11T07:43:39.235 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8125999999999999e-05 2024-09-11T07:43:39.235 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.235 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.235 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:39.236 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:45.068521+0000", 2024-09-11T07:43:39.236 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9600000000000003e-06 2024-09-11T07:43:39.236 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.236 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.236 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.236 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:45.068582+0000", 2024-09-11T07:43:39.236 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0315000000000001e-05 2024-09-11T07:43:39.236 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.236 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.236 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:39.236 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:45.068740+0000", 2024-09-11T07:43:39.236 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015821300000000001 2024-09-11T07:43:39.236 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.236 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.237 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,4", 2024-09-11T07:43:39.237 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:45.068901+0000", 2024-09-11T07:43:39.237 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000160915 2024-09-11T07:43:39.237 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.237 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.237 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:39.237 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:45.070976+0000", 2024-09-11T07:43:39.237 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002074827 2024-09-11T07:43:39.237 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.237 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.237 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.237 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:45.071704+0000", 2024-09-11T07:43:39.237 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00072795300000000004 2024-09-11T07:43:39.238 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.238 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.238 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.238 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:45.212055+0000", 2024-09-11T07:43:39.238 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14035097999999999 2024-09-11T07:43:39.238 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.238 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.238 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:39.238 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:45.212099+0000", 2024-09-11T07:43:39.238 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.422e-05 2024-09-11T07:43:39.238 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.238 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.238 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:39.238 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:45.212136+0000", 2024-09-11T07:43:39.239 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7298000000000001e-05 2024-09-11T07:43:39.239 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.239 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:39.239 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.239 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.239 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.239 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25706.0:96549 10.c 10:319e8ce6:::benchmark_data_smithi022_163866_object96548:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e363)", 2024-09-11T07:43:39.239 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:43:16.359817+0000", 2024-09-11T07:43:39.239 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 22.747493394999999, 2024-09-11T07:43:39.239 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.20600390700000001, 2024-09-11T07:43:39.239 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:39.239 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:39.239 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:39.239 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25706", 2024-09-11T07:43:39.240 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:43:39.240 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 96549 2024-09-11T07:43:39.240 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.240 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:39.240 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.240 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:39.240 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.359817+0000", 2024-09-11T07:43:39.240 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.240 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.240 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.240 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:39.240 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.359817+0000", 2024-09-11T07:43:39.240 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.241 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.241 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.241 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:39.241 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.359815+0000", 2024-09-11T07:43:39.241 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:43:39.241 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.241 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.241 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:39.241 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.359868+0000", 2024-09-11T07:43:39.241 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2349999999999999e-05 2024-09-11T07:43:39.241 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.241 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.241 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:39.242 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.359882+0000", 2024-09-11T07:43:39.242 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4620999999999999e-05 2024-09-11T07:43:39.242 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.242 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.242 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:39.242 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.359888+0000", 2024-09-11T07:43:39.242 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4480000000000002e-06 2024-09-11T07:43:39.242 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.242 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.242 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.242 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.359938+0000", 2024-09-11T07:43:39.242 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0871000000000001e-05 2024-09-11T07:43:39.242 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.242 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.243 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:39.243 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.360063+0000", 2024-09-11T07:43:39.243 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012440899999999999 2024-09-11T07:43:39.243 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.243 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.243 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,4", 2024-09-11T07:43:39.243 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.360176+0000", 2024-09-11T07:43:39.243 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000113237 2024-09-11T07:43:39.243 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.243 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.243 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:39.243 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.366265+0000", 2024-09-11T07:43:39.243 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0060891219999999998 2024-09-11T07:43:39.243 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.244 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.244 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.244 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.366682+0000", 2024-09-11T07:43:39.244 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000417045 2024-09-11T07:43:39.244 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.244 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.244 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.244 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.565705+0000", 2024-09-11T07:43:39.244 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.19902228899999999 2024-09-11T07:43:39.244 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.244 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.244 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:39.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:39.246 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:39.246 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:39.248 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.565778+0000", 2024-09-11T07:43:39.248 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3826999999999996e-05 2024-09-11T07:43:39.248 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.248 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.248 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:39.248 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.565821+0000", 2024-09-11T07:43:39.248 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2833000000000001e-05 2024-09-11T07:43:39.248 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.249 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:39.249 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.249 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.249 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.249 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25706.0:96550 10.c 10:306cfa14:::benchmark_data_smithi022_163866_object96549:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e363)", 2024-09-11T07:43:39.249 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:43:16.359908+0000", 2024-09-11T07:43:39.249 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 22.747402908000002, 2024-09-11T07:43:39.249 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.20611137299999999, 2024-09-11T07:43:39.249 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:43:39.249 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:43:39.249 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:43:39.249 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25706", 2024-09-11T07:43:39.249 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:43:39.250 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 96550 2024-09-11T07:43:39.250 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.250 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:43:39.250 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.250 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:43:39.250 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.359908+0000", 2024-09-11T07:43:39.250 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.250 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.250 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.250 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:43:39.250 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.359908+0000", 2024-09-11T07:43:39.250 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:43:39.250 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.251 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.251 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:43:39.251 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.359907+0000", 2024-09-11T07:43:39.251 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:43:39.251 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.251 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.251 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:43:39.251 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.359956+0000", 2024-09-11T07:43:39.251 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8612999999999997e-05 2024-09-11T07:43:39.251 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.251 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.251 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:43:39.251 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.359962+0000", 2024-09-11T07:43:39.251 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2519999999999996e-06 2024-09-11T07:43:39.252 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.252 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.252 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:43:39.252 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.359965+0000", 2024-09-11T07:43:39.252 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.0400000000000001e-06 2024-09-11T07:43:39.252 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.252 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.252 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:43:39.252 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.363381+0000", 2024-09-11T07:43:39.252 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003416414 2024-09-11T07:43:39.252 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.252 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.252 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:43:39.253 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.363530+0000", 2024-09-11T07:43:39.253 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014878100000000001 2024-09-11T07:43:39.253 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.253 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.253 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,4", 2024-09-11T07:43:39.253 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.363686+0000", 2024-09-11T07:43:39.253 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015559600000000001 2024-09-11T07:43:39.253 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.253 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.253 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:43:39.253 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.366392+0000", 2024-09-11T07:43:39.253 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002705954 2024-09-11T07:43:39.253 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.254 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.254 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.254 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.366815+0000", 2024-09-11T07:43:39.254 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00042378000000000001 2024-09-11T07:43:39.254 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.254 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.254 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:43:39.254 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.565881+0000", 2024-09-11T07:43:39.254 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.199065929 2024-09-11T07:43:39.254 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.254 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.254 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:43:39.254 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.565983+0000", 2024-09-11T07:43:39.254 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000101859 2024-09-11T07:43:39.255 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:43:39.255 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:43:39.255 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:43:39.255 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:16.566019+0000", 2024-09-11T07:43:39.255 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5920999999999997e-05 2024-09-11T07:43:39.255 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.255 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:39.255 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.255 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:43:39.255 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:43:39.255 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:43:39.347 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:39.388 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:39.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:39.389 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:39.490 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:39.527 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:39.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:39.528 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:39.629 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:39.668 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:39.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:39.669 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:39.770 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:39.803 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:39.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:39.804 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:39.905 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:39.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:39 smithi022 ceph-mon[29462]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:43:39.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:39 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3976604641' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:43:39.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:39 smithi022 ceph-mon[29462]: osdmap e370: 8 total, 8 up, 8 in 2024-09-11T07:43:39.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:39 smithi022 ceph-mon[37330]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:43:39.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:39 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3976604641' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:43:39.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:39 smithi022 ceph-mon[37330]: osdmap e370: 8 total, 8 up, 8 in 2024-09-11T07:43:39.947 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:39.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:39.949 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:39.963 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 84 16 122837 122821 91.37 75.0625 0.00816079 0.0109389 2024-09-11T07:43:39.996 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:39 smithi104 ceph-mon[33507]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:43:39.996 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:39 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3976604641' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:43:39.996 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:39 smithi104 ceph-mon[33507]: osdmap e370: 8 total, 8 up, 8 in 2024-09-11T07:43:40.049 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:40.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:40.085 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:40.085 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:40.186 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:40.227 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:40.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:40.228 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:40.279 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:43:39 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:43:40.328 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:40.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:40.367 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:40.367 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:40.468 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:40.496 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-11T07:43:40.512 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:40.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:40.513 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:40.547 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:43:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:43:40] "GET /metrics HTTP/1.1" 200 224958 "" "Prometheus/2.43.0" 2024-09-11T07:43:40.614 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:40.727 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:40.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:40.728 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:40.829 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:40.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:40.886 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:40.886 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:40.964 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 85 16 123996 123980 91.1472 72.4375 0.0150526 0.0109641 2024-09-11T07:43:40.987 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:41.033 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:41.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:41.034 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:41.134 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:41.147 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:41 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:43:41.147 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:41 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:43:41.147 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:41 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:43:41.147 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:41 smithi022 ceph-mon[29462]: osdmap e371: 8 total, 8 up, 8 in 2024-09-11T07:43:41.147 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:41 smithi022 ceph-mon[29462]: pgmap v798: 148 pgs: 148 active+clean; 7.4 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.0 KiB/s rd, 76 MiB/s wr, 1.21k op/s 2024-09-11T07:43:41.147 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:41 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:43:41.147 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:41 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:43:41.148 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:41 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:43:41.148 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:41 smithi022 ceph-mon[37330]: osdmap e371: 8 total, 8 up, 8 in 2024-09-11T07:43:41.148 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:41 smithi022 ceph-mon[37330]: pgmap v798: 148 pgs: 148 active+clean; 7.4 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.0 KiB/s rd, 76 MiB/s wr, 1.21k op/s 2024-09-11T07:43:41.174 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:41.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:41.176 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:41.224 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:43:41.246 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:43:41.246 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:43:41.277 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:41.281 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:43:41.281 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:43:41.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:41.307 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:41.307 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:41.311 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:43:41.312 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:43:41.344 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:43:41.344 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:43:41.375 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:43:41.376 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:43:41.408 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:41.410 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:43:41.410 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:43:41.439 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:43:41.439 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:43:41.445 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:41.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:41.446 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:41.476 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:43:41.477 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:43:41.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:41 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:43:41.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:41 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:43:41.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:41 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:43:41.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:41 smithi104 ceph-mon[33507]: osdmap e371: 8 total, 8 up, 8 in 2024-09-11T07:43:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:41 smithi104 ceph-mon[33507]: pgmap v798: 148 pgs: 148 active+clean; 7.4 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.0 KiB/s rd, 76 MiB/s wr, 1.21k op/s 2024-09-11T07:43:41.547 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:41.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:41.585 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:41.585 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:41.686 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:41.731 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:41.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:41.733 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:41.833 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:41.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:41.867 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:41.868 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:41.964 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 86 16 125064 125048 90.8634 66.75 0.00474727 0.0109956 2024-09-11T07:43:41.968 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:41.973 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T07:43:42.012 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:42.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:42.015 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:42.116 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:42.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:42.163 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:42.164 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:42.264 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:42.300 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:42.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:42.301 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:42.402 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:42.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:42 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:43:42.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:42 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:43:42.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:42 smithi022 ceph-mon[29462]: osdmap e372: 8 total, 8 up, 8 in 2024-09-11T07:43:42.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:42 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3180700981' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:43:42.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:42 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:43:42.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:42 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:43:42.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:42 smithi022 ceph-mon[37330]: osdmap e372: 8 total, 8 up, 8 in 2024-09-11T07:43:42.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:42 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3180700981' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:43:42.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:42.447 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:42.448 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:42.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:42 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:43:42.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:42 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:43:42.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:42 smithi104 ceph-mon[33507]: osdmap e372: 8 total, 8 up, 8 in 2024-09-11T07:43:42.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:42 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3180700981' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:43:42.548 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:42.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:42.585 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:42.585 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:42.686 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:42.725 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:42.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:42.726 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:42.827 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:42.860 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:42.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:42.861 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:42.961 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:42.964 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 87 15 126188 126173 90.6271 70.3125 0.0188022 0.0110293 2024-09-11T07:43:42.994 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:42.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:42.995 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:43.096 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:43.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:43.139 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:43.140 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:43.240 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:43.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:43.268 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:43.268 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:43.369 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:43.391 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:43.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:43.392 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:43.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:43 smithi022 ceph-mon[29462]: pgmap v800: 148 pgs: 148 active+clean; 7.4 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.1 KiB/s rd, 82 MiB/s wr, 1.30k op/s 2024-09-11T07:43:43.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:43 smithi022 ceph-mon[37330]: pgmap v800: 148 pgs: 148 active+clean; 7.4 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.1 KiB/s rd, 82 MiB/s wr, 1.30k op/s 2024-09-11T07:43:43.493 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:43.523 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:43.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:43.524 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:43.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:43 smithi104 ceph-mon[33507]: pgmap v800: 148 pgs: 148 active+clean; 7.4 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.1 KiB/s rd, 82 MiB/s wr, 1.30k op/s 2024-09-11T07:43:43.624 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:43.662 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:43.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:43.664 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:43.764 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:43.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:43.793 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:43.793 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:43.894 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:43.934 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:43.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:43.935 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:43.964 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 88 16 127050 127034 90.2088 53.8125 0.011575 0.0110811 2024-09-11T07:43:44.037 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:44.068 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:44.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:44.069 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:44.170 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:44.206 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:44.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:44.207 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:44.248 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:43:44.248 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:43:44.249 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-11T07:43:44.249 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-11T07:43:44.307 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:44.339 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:44.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:44.340 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:44.441 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:44.476 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:44.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:44.477 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:44.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:43:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:43:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:43:44.578 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:44.606 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:44.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:44.607 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:44.708 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:44.745 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:44.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:44.746 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:44.847 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:44.887 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:44.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:44.888 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:44.964 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 89 16 128199 128183 90.002 71.8125 0.00580772 0.0111054 2024-09-11T07:43:44.989 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:45.029 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:45.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:45.030 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:45.131 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:45.158 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:45.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:45.159 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:45.261 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:45.299 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:45.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:45.301 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:45.402 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:45.434 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:45.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:45.435 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:45.536 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:45.559 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:45.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:45.561 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:45.664 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:45.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:45 smithi022 ceph-mon[29462]: pgmap v801: 148 pgs: 148 active+clean; 7.7 GiB data, 29 GiB used, 687 GiB / 715 GiB avail; 1.7 KiB/s rd, 112 MiB/s wr, 1.79k op/s 2024-09-11T07:43:45.666 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:45 smithi022 ceph-mon[37330]: pgmap v801: 148 pgs: 148 active+clean; 7.7 GiB data, 29 GiB used, 687 GiB / 715 GiB avail; 1.7 KiB/s rd, 112 MiB/s wr, 1.79k op/s 2024-09-11T07:43:45.707 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:45.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:45.708 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:45.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:45 smithi104 ceph-mon[33507]: pgmap v801: 148 pgs: 148 active+clean; 7.7 GiB data, 29 GiB used, 687 GiB / 715 GiB avail; 1.7 KiB/s rd, 112 MiB/s wr, 1.79k op/s 2024-09-11T07:43:45.809 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:45.846 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:45.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:45.848 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:45.948 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:45.964 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Total time run: 90.0078 2024-09-11T07:43:45.964 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Total writes made: 129423 2024-09-11T07:43:45.964 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Write size: 65536 2024-09-11T07:43:45.964 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Object size: 65536 2024-09-11T07:43:45.964 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Bandwidth (MB/sec): 89.8693 2024-09-11T07:43:45.964 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Stddev Bandwidth: 34.3245 2024-09-11T07:43:45.964 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Max bandwidth (MB/sec): 218 2024-09-11T07:43:45.965 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Min bandwidth (MB/sec): 44.6875 2024-09-11T07:43:45.965 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Average IOPS: 1437 2024-09-11T07:43:45.965 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Stddev IOPS: 549.192 2024-09-11T07:43:45.965 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Max IOPS: 3488 2024-09-11T07:43:45.965 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Min IOPS: 715 2024-09-11T07:43:45.965 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Average Latency(s): 0.0111231 2024-09-11T07:43:45.965 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Stddev Latency(s): 0.0137719 2024-09-11T07:43:45.965 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Max latency(s): 0.408634 2024-09-11T07:43:45.965 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Min latency(s): 0.00106648 2024-09-11T07:43:45.970 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Cleaning up (deleting benchmark objects) 2024-09-11T07:43:45.974 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:45.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:45.975 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:46.076 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:46.106 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:46.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:46.107 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:46.208 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:46.239 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:46.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:46.240 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:46.341 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:46.376 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:46.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:46.377 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:46.478 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:46.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:46.513 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:46.513 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:46.614 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:46.642 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:46.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:46.643 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:46.744 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:46.778 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:46.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:46.779 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:46.880 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:46.918 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:46.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:46.919 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:47.019 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:47.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:46 smithi104 ceph-mon[33507]: pgmap v802: 148 pgs: 148 active+clean; 7.8 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1.3 KiB/s rd, 94 MiB/s wr, 1.50k op/s 2024-09-11T07:43:47.048 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:47.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:47.049 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:47.149 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:47.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:46 smithi022 ceph-mon[29462]: pgmap v802: 148 pgs: 148 active+clean; 7.8 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1.3 KiB/s rd, 94 MiB/s wr, 1.50k op/s 2024-09-11T07:43:47.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:46 smithi022 ceph-mon[37330]: pgmap v802: 148 pgs: 148 active+clean; 7.8 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1.3 KiB/s rd, 94 MiB/s wr, 1.50k op/s 2024-09-11T07:43:47.188 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:47.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:47.189 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:47.290 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:47.321 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:47.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:47.322 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:47.423 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:47.468 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:47.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:47.469 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:47.569 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:47.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:47.601 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:47.601 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:47.702 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:47.737 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:47.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:47.738 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:47.839 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:47.878 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:47.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:47.879 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:47.952 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:43:47.953 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:43:47.980 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:48.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:48.040 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:48.040 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:48.141 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:48.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:48.189 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:48.190 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:48.290 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:48.323 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:48.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:48.325 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:48.426 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:48.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:48.466 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:48.466 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:48.567 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:48.603 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:48.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:48.605 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:48.705 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:48.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:48.749 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:48.749 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:48.850 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:48.886 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:48.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:48.887 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:48.988 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:49.022 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:49.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:49.024 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:49.124 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:49.163 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:49.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:49.165 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:49.265 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:49.280 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:43:49.295 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:49.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:49.297 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:49.398 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:49.448 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:49.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:49.449 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:49.552 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:49.599 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:49.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:49.600 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:49.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:49 smithi022 ceph-mon[29462]: pgmap v803: 148 pgs: 148 active+clean; 7.8 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 619 B/s rd, 43 MiB/s wr, 696 op/s 2024-09-11T07:43:49.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:49 smithi022 ceph-mon[37330]: pgmap v803: 148 pgs: 148 active+clean; 7.8 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 619 B/s rd, 43 MiB/s wr, 696 op/s 2024-09-11T07:43:49.702 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:49.742 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:49.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:49.743 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:49.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:49 smithi104 ceph-mon[33507]: pgmap v803: 148 pgs: 148 active+clean; 7.8 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 619 B/s rd, 43 MiB/s wr, 696 op/s 2024-09-11T07:43:49.844 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:49.880 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:49.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:49.881 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:49.982 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:50.010 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:50.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:50.011 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:50.112 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:50.150 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:50.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:50.152 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:50.253 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:50.279 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:43:50 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:43:50.282 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:50.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:50.283 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:50.384 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:50.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:50.417 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:50.418 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:50.518 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:50.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:50.577 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:50.577 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:50.665 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:43:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:43:50] "GET /metrics HTTP/1.1" 200 224958 "" "Prometheus/2.43.0" 2024-09-11T07:43:50.681 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:50.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:50.713 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:50.713 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:50.814 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:50.861 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:50.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:50.862 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:50.963 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:50.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:50.999 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:50.999 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:51.100 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:51.136 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:51.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:51.138 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:51.238 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:51.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:51.271 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:51.272 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:51.373 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:51.413 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:51.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:51.414 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:51.475 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:43:51.475 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:43:51.515 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:51.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:51.555 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:51.556 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:51.656 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:51.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:51 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:43:51.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:51 smithi022 ceph-mon[29462]: pgmap v804: 148 pgs: 148 active+clean; 7.7 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1.1 KiB/s rd, 47 MiB/s wr, 1.16k op/s 2024-09-11T07:43:51.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:51 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:43:51.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:51 smithi022 ceph-mon[37330]: pgmap v804: 148 pgs: 148 active+clean; 7.7 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1.1 KiB/s rd, 47 MiB/s wr, 1.16k op/s 2024-09-11T07:43:51.695 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:51.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:51.697 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:51.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:51 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:43:51.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:51 smithi104 ceph-mon[33507]: pgmap v804: 148 pgs: 148 active+clean; 7.7 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1.1 KiB/s rd, 47 MiB/s wr, 1.16k op/s 2024-09-11T07:43:51.798 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:51.833 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:51.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:51.834 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:51.935 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:51.970 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:51.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:51.972 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:51.983 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:43:52.073 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:52.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:52.116 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:52.117 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:52.217 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:52.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:52.265 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:52.266 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:52.312 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:43:52.312 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:43:52.312 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 372, 2024-09-11T07:43:52.312 INFO:teuthology.orchestra.run.smithi022.stdout: "fsid": "12a285d8-700e-11ef-bcea-c7b262605968", 2024-09-11T07:43:52.312 INFO:teuthology.orchestra.run.smithi022.stdout: "created": "2024-09-11T07:21:18.695332+0000", 2024-09-11T07:43:52.313 INFO:teuthology.orchestra.run.smithi022.stdout: "modified": "2024-09-11T07:43:41.052119+0000", 2024-09-11T07:43:52.313 INFO:teuthology.orchestra.run.smithi022.stdout: "last_up_change": "2024-09-11T07:32:00.070825+0000", 2024-09-11T07:43:52.313 INFO:teuthology.orchestra.run.smithi022.stdout: "last_in_change": "2024-09-11T07:25:58.551813+0000", 2024-09-11T07:43:52.313 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-11T07:43:52.313 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_num": 5806080, 2024-09-11T07:43:52.313 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_set": [ 2024-09-11T07:43:52.313 INFO:teuthology.orchestra.run.smithi022.stdout: "nodeep-scrub", 2024-09-11T07:43:52.313 INFO:teuthology.orchestra.run.smithi022.stdout: "noscrub", 2024-09-11T07:43:52.313 INFO:teuthology.orchestra.run.smithi022.stdout: "pglog_hardlimit", 2024-09-11T07:43:52.313 INFO:teuthology.orchestra.run.smithi022.stdout: "purged_snapdirs", 2024-09-11T07:43:52.313 INFO:teuthology.orchestra.run.smithi022.stdout: "recovery_deletes", 2024-09-11T07:43:52.314 INFO:teuthology.orchestra.run.smithi022.stdout: "sortbitwise" 2024-09-11T07:43:52.314 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:43:52.314 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_version": 19, 2024-09-11T07:43:52.314 INFO:teuthology.orchestra.run.smithi022.stdout: "full_ratio": 0.94999998807907104, 2024-09-11T07:43:52.314 INFO:teuthology.orchestra.run.smithi022.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-11T07:43:52.314 INFO:teuthology.orchestra.run.smithi022.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-11T07:43:52.314 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_snapshot": "", 2024-09-11T07:43:52.314 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_max": 10, 2024-09-11T07:43:52.314 INFO:teuthology.orchestra.run.smithi022.stdout: "max_osd": 8, 2024-09-11T07:43:52.314 INFO:teuthology.orchestra.run.smithi022.stdout: "require_min_compat_client": "luminous", 2024-09-11T07:43:52.314 INFO:teuthology.orchestra.run.smithi022.stdout: "min_compat_client": "luminous", 2024-09-11T07:43:52.314 INFO:teuthology.orchestra.run.smithi022.stdout: "require_osd_release": "quincy", 2024-09-11T07:43:52.315 INFO:teuthology.orchestra.run.smithi022.stdout: "pools": [ 2024-09-11T07:43:52.315 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.315 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 1, 2024-09-11T07:43:52.315 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": ".mgr", 2024-09-11T07:43:52.315 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:24:21.772426+0000", 2024-09-11T07:43:52.315 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:43:52.315 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:43:52.315 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:43:52.315 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:43:52.315 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:43:52.315 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:43:52.315 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:43:52.315 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:43:52.315 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:43:52.316 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:43:52.316 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:43:52.316 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:43:52.316 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 1, 2024-09-11T07:43:52.316 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 1, 2024-09-11T07:43:52.316 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 1, 2024-09-11T07:43:52.316 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 1, 2024-09-11T07:43:52.316 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 1, 2024-09-11T07:43:52.316 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:43:52.316 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:43:52.316 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:43:52.316 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:43:52.317 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:43:52.317 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:43:52.317 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:43:52.317 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.317 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "21", 2024-09-11T07:43:52.317 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:43:52.318 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:43:52.318 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:43:52.318 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:43:52.318 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:43:52.318 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:43:52.318 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:43:52.318 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:43:52.318 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:43:52.318 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:43:52.319 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:43:52.319 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:43:52.319 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:43:52.319 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:43:52.319 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:43:52.319 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:43:52.319 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:43:52.319 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:43:52.319 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:43:52.319 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:43:52.319 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:43:52.319 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:43:52.319 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:43:52.320 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:43:52.320 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:43:52.320 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:43:52.320 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.320 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:43:52.320 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:43:52.320 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:43:52.320 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:43:52.320 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:43:52.320 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:43:52.320 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:43:52.320 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:43:52.321 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:43:52.321 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:43:52.321 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:43:52.321 INFO:teuthology.orchestra.run.smithi022.stdout: "options": { 2024-09-11T07:43:52.321 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_max": 32, 2024-09-11T07:43:52.321 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_min": 1 2024-09-11T07:43:52.321 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.322 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:43:52.322 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": {} 2024-09-11T07:43:52.322 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.322 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.322 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.322 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 2, 2024-09-11T07:43:52.322 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": ".rgw.root", 2024-09-11T07:43:52.322 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:29.941354+0000", 2024-09-11T07:43:52.322 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:43:52.322 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:43:52.322 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:43:52.322 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:43:52.323 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:43:52.323 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:43:52.323 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:43:52.323 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:43:52.323 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:43:52.323 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:43:52.323 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:43:52.323 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:43:52.323 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:43:52.323 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:43:52.323 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:43:52.323 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:43:52.323 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:43:52.324 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:43:52.324 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:43:52.324 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:43:52.324 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:43:52.324 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:43:52.324 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:43:52.324 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:43:52.324 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.324 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "58", 2024-09-11T07:43:52.324 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:43:52.324 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:43:52.324 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:43:52.324 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:43:52.325 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:43:52.325 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:43:52.325 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:43:52.325 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:43:52.325 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:43:52.325 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:43:52.325 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:43:52.325 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:43:52.325 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:43:52.325 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:43:52.325 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:43:52.326 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:43:52.326 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:43:52.326 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:43:52.326 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:43:52.326 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:43:52.326 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:43:52.326 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:43:52.326 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:43:52.326 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:43:52.326 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:43:52.326 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:43:52.326 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.326 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:43:52.326 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:43:52.327 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:43:52.327 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:43:52.327 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:43:52.327 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:43:52.327 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:43:52.327 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:43:52.327 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:43:52.327 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:43:52.327 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:43:52.327 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:43:52.327 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:43:52.327 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:43:52.328 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.328 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.328 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.328 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 3, 2024-09-11T07:43:52.328 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "datapool", 2024-09-11T07:43:52.328 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:30.396859+0000", 2024-09-11T07:43:52.328 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 8193, 2024-09-11T07:43:52.328 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T07:43:52.328 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:43:52.328 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:43:52.328 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:43:52.328 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:43:52.328 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:43:52.328 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:43:52.329 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:43:52.329 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:43:52.329 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:43:52.329 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:43:52.329 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 3, 2024-09-11T07:43:52.329 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 3, 2024-09-11T07:43:52.329 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 3, 2024-09-11T07:43:52.329 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 3, 2024-09-11T07:43:52.329 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 3, 2024-09-11T07:43:52.329 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:43:52.329 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:43:52.329 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:43:52.329 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:43:52.329 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:43:52.330 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:43:52.330 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:43:52.330 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.330 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "61", 2024-09-11T07:43:52.330 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:43:52.330 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:43:52.330 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:43:52.330 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:43:52.330 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:43:52.330 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 3, 2024-09-11T07:43:52.330 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 61, 2024-09-11T07:43:52.330 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:43:52.330 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:43:52.330 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:43:52.331 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:43:52.331 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:43:52.331 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:43:52.331 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:43:52.331 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:43:52.331 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:43:52.331 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:43:52.331 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:43:52.331 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:43:52.331 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:43:52.331 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:43:52.331 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:43:52.331 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:43:52.332 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:43:52.332 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:43:52.332 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:43:52.332 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.332 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:43:52.332 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:43:52.332 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:43:52.332 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:43:52.333 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:43:52.333 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:43:52.333 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:43:52.333 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:43:52.333 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:43:52.333 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:43:52.333 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:43:52.333 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:43:52.333 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:43:52.333 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd": {} 2024-09-11T07:43:52.333 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.333 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.334 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.334 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 4, 2024-09-11T07:43:52.334 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.log", 2024-09-11T07:43:52.334 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:31.922926+0000", 2024-09-11T07:43:52.334 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:43:52.334 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:43:52.334 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:43:52.334 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:43:52.334 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:43:52.334 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:43:52.334 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:43:52.334 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:43:52.334 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:43:52.334 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:43:52.335 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:43:52.335 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:43:52.335 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:43:52.335 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:43:52.335 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:43:52.335 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:43:52.335 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:43:52.335 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:43:52.335 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:43:52.335 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:43:52.335 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:43:52.335 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:43:52.336 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:43:52.336 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:43:52.336 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.336 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "60", 2024-09-11T07:43:52.336 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:43:52.336 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:43:52.336 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:43:52.336 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:43:52.336 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:43:52.336 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:43:52.336 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:43:52.337 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:43:52.337 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:43:52.337 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:43:52.337 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:43:52.337 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:43:52.337 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:43:52.337 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:43:52.337 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:43:52.337 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:43:52.337 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:43:52.337 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:43:52.337 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:43:52.337 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:43:52.338 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:43:52.338 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:43:52.338 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:43:52.338 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:43:52.338 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:43:52.338 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:43:52.338 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.338 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:43:52.338 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:43:52.338 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:43:52.338 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:43:52.338 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:43:52.338 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:43:52.339 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:43:52.339 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:43:52.339 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:43:52.339 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:43:52.339 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:43:52.339 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:43:52.339 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:43:52.339 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:43:52.339 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.339 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.339 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.339 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 5, 2024-09-11T07:43:52.340 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.control", 2024-09-11T07:43:52.340 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:34.007746+0000", 2024-09-11T07:43:52.340 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:43:52.340 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:43:52.340 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:43:52.340 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:43:52.340 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:43:52.340 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:43:52.340 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:43:52.340 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:43:52.340 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:43:52.340 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:43:52.341 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:43:52.341 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:43:52.341 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:43:52.341 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:43:52.341 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:43:52.341 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:43:52.341 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:43:52.341 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:43:52.341 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:43:52.341 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:43:52.341 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:43:52.341 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:43:52.341 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:43:52.342 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:43:52.342 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.342 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "62", 2024-09-11T07:43:52.342 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:43:52.342 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:43:52.342 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:43:52.342 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:43:52.342 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:43:52.342 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:43:52.342 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:43:52.342 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:43:52.342 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:43:52.342 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:43:52.343 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:43:52.343 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:43:52.343 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:43:52.343 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:43:52.343 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:43:52.343 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:43:52.343 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:43:52.343 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:43:52.343 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:43:52.343 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:43:52.343 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:43:52.343 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:43:52.343 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:43:52.344 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:43:52.344 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:43:52.344 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:43:52.344 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.344 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:43:52.344 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:43:52.344 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:43:52.344 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:43:52.344 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:43:52.344 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:43:52.344 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:43:52.344 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:43:52.345 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:43:52.345 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:43:52.345 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:43:52.345 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:43:52.345 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:43:52.345 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:43:52.345 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.345 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.345 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.345 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 6, 2024-09-11T07:43:52.345 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.meta", 2024-09-11T07:43:52.346 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:36.077215+0000", 2024-09-11T07:43:52.346 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:43:52.346 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:43:52.346 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:43:52.346 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:43:52.346 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:43:52.346 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:43:52.346 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:43:52.346 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:43:52.346 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:43:52.346 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:43:52.346 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:43:52.346 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:43:52.347 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:43:52.347 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:43:52.347 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:43:52.347 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:43:52.347 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:43:52.347 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:43:52.347 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:43:52.347 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:43:52.347 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:43:52.347 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:43:52.347 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:43:52.347 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:43:52.348 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.348 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "64", 2024-09-11T07:43:52.348 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:43:52.348 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:43:52.348 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:43:52.348 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:43:52.348 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:43:52.348 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:43:52.348 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:43:52.348 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:43:52.348 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:43:52.348 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:43:52.348 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:43:52.349 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:43:52.349 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:43:52.349 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:43:52.349 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:43:52.349 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:43:52.349 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:43:52.349 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:43:52.349 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:43:52.349 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:43:52.349 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:43:52.349 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:43:52.349 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:43:52.349 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:43:52.349 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:43:52.350 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:43:52.350 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.350 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:43:52.350 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:43:52.350 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:43:52.350 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:43:52.350 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:43:52.350 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:43:52.350 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:43:52.350 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:43:52.350 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:43:52.350 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:43:52.350 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:43:52.351 INFO:teuthology.orchestra.run.smithi022.stdout: "options": { 2024-09-11T07:43:52.351 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_bias": 4 2024-09-11T07:43:52.351 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.351 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:43:52.351 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:43:52.352 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.352 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.352 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.352 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 10, 2024-09-11T07:43:52.352 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "unique_pool_3", 2024-09-11T07:43:52.352 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:41:56.278857+0000", 2024-09-11T07:43:52.352 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:43:52.352 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:43:52.352 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:43:52.352 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:43:52.352 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:43:52.352 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:43:52.352 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:43:52.353 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:43:52.353 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:43:52.353 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:43:52.353 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:43:52.353 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:43:52.353 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 16, 2024-09-11T07:43:52.353 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 16, 2024-09-11T07:43:52.353 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 16, 2024-09-11T07:43:52.353 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 16, 2024-09-11T07:43:52.353 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 16, 2024-09-11T07:43:52.354 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:43:52.354 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:43:52.354 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:43:52.354 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:43:52.354 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:43:52.354 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:43:52.354 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:43:52.354 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.354 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "349", 2024-09-11T07:43:52.354 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:43:52.354 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:43:52.354 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:43:52.354 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:43:52.355 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:43:52.355 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:43:52.355 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:43:52.355 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:43:52.355 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:43:52.355 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:43:52.355 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:43:52.355 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:43:52.355 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:43:52.355 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:43:52.355 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:43:52.355 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:43:52.355 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:43:52.356 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:43:52.356 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:43:52.356 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:43:52.356 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:43:52.356 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:43:52.356 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:43:52.356 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:43:52.356 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:43:52.356 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:43:52.356 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.356 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:43:52.356 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:43:52.356 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:43:52.356 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:43:52.357 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:43:52.357 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:43:52.357 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:43:52.357 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:43:52.357 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:43:52.357 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:43:52.357 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:43:52.357 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:43:52.357 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:43:52.357 INFO:teuthology.orchestra.run.smithi022.stdout: "rados": {} 2024-09-11T07:43:52.357 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.357 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.357 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:43:52.358 INFO:teuthology.orchestra.run.smithi022.stdout: "osds": [ 2024-09-11T07:43:52.358 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.358 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 0, 2024-09-11T07:43:52.358 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "09075f16-32f4-4f88-a116-c3d7f977a0e0", 2024-09-11T07:43:52.358 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:43:52.358 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:43:52.358 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:43:52.358 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:43:52.358 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:43:52.358 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:43:52.358 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 8, 2024-09-11T07:43:52.358 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 358, 2024-09-11T07:43:52.359 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:43:52.359 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:43:52.359 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:43:52.359 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.359 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.359 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.359 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6801", 2024-09-11T07:43:52.359 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:43:52.359 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.359 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.359 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.359 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:43:52.359 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.360 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.360 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.360 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6802", 2024-09-11T07:43:52.360 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:43:52.360 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.360 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.360 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.360 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:43:52.360 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.360 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.360 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.360 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6804", 2024-09-11T07:43:52.360 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:43:52.360 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.361 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.361 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.361 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:43:52.361 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.361 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.361 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.361 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6803", 2024-09-11T07:43:52.361 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:43:52.361 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.361 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.361 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.361 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6801/2821493591", 2024-09-11T07:43:52.361 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6802/2821493591", 2024-09-11T07:43:52.362 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6804/2821493591", 2024-09-11T07:43:52.362 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6803/2821493591", 2024-09-11T07:43:52.362 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:43:52.362 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:43:52.362 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:43:52.362 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.362 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.362 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.362 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 1, 2024-09-11T07:43:52.362 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "52cc1e06-9506-48c0-9f7b-d4270dff4b9f", 2024-09-11T07:43:52.362 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:43:52.362 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:43:52.362 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 0.64752197265625, 2024-09-11T07:43:52.362 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:43:52.362 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:43:52.363 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:43:52.363 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 13, 2024-09-11T07:43:52.363 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 346, 2024-09-11T07:43:52.363 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:43:52.363 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:43:52.363 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:43:52.363 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.363 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.363 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.363 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6805", 2024-09-11T07:43:52.363 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:43:52.363 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.363 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.363 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.364 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:43:52.364 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.364 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.364 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.364 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6806", 2024-09-11T07:43:52.364 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:43:52.364 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.364 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.364 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.364 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:43:52.364 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.364 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.364 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.364 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6808", 2024-09-11T07:43:52.365 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:43:52.365 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.365 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.365 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.365 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:43:52.365 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.365 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.365 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.365 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6807", 2024-09-11T07:43:52.365 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:43:52.365 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.365 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.365 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.365 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6805/4037911317", 2024-09-11T07:43:52.366 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6806/4037911317", 2024-09-11T07:43:52.366 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6808/4037911317", 2024-09-11T07:43:52.366 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6807/4037911317", 2024-09-11T07:43:52.366 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:43:52.366 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:43:52.366 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:43:52.366 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.366 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.366 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.366 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 2, 2024-09-11T07:43:52.366 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "fa2b2730-2c1d-45d0-be8a-494a60dc6b9b", 2024-09-11T07:43:52.366 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:43:52.366 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:43:52.366 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:43:52.367 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:43:52.367 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:43:52.367 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:43:52.367 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 18, 2024-09-11T07:43:52.367 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 360, 2024-09-11T07:43:52.367 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:43:52.367 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:43:52.367 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:43:52.367 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.367 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.367 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.367 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6809", 2024-09-11T07:43:52.367 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:43:52.367 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.367 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.368 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.368 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:43:52.368 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.368 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.368 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.368 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6810", 2024-09-11T07:43:52.368 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:43:52.368 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.368 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.368 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.368 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:43:52.368 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.368 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.368 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.369 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6812", 2024-09-11T07:43:52.369 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:43:52.369 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.369 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.369 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.369 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:43:52.369 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.369 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.369 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.369 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6811", 2024-09-11T07:43:52.369 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:43:52.369 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.369 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.370 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.370 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6809/1675799666", 2024-09-11T07:43:52.370 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6810/1675799666", 2024-09-11T07:43:52.370 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6812/1675799666", 2024-09-11T07:43:52.370 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6811/1675799666", 2024-09-11T07:43:52.370 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:43:52.370 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:43:52.370 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:43:52.370 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.370 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.370 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.370 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 3, 2024-09-11T07:43:52.370 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "8d53d3fc-e401-47df-b70b-4b4b1f4539df", 2024-09-11T07:43:52.371 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:43:52.371 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:43:52.371 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 0.686370849609375, 2024-09-11T07:43:52.371 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:43:52.371 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:43:52.371 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:43:52.371 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 25, 2024-09-11T07:43:52.371 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 341, 2024-09-11T07:43:52.371 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:43:52.371 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:43:52.371 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:43:52.371 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.372 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.372 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.372 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6813", 2024-09-11T07:43:52.372 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:43:52.372 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.372 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.372 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.372 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:43:52.372 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.372 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.372 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.372 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6814", 2024-09-11T07:43:52.372 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:43:52.373 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.373 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.373 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.373 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:43:52.373 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.373 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.373 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.373 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6816", 2024-09-11T07:43:52.373 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:43:52.373 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.373 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.373 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.373 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:43:52.374 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.374 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.374 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.374 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6815", 2024-09-11T07:43:52.374 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:43:52.374 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.374 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.374 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.374 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6813/1501962752", 2024-09-11T07:43:52.374 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6814/1501962752", 2024-09-11T07:43:52.374 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6816/1501962752", 2024-09-11T07:43:52.374 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6815/1501962752", 2024-09-11T07:43:52.374 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:43:52.375 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:43:52.375 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:43:52.375 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.375 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.375 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.375 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 4, 2024-09-11T07:43:52.375 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "d3207fe0-8b97-4067-a755-ba6d449c53c7", 2024-09-11T07:43:52.375 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:43:52.375 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:43:52.375 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:43:52.375 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:43:52.375 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:43:52.375 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:43:52.376 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 31, 2024-09-11T07:43:52.376 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 360, 2024-09-11T07:43:52.376 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:43:52.376 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:43:52.376 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:43:52.376 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.376 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.376 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.376 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6800", 2024-09-11T07:43:52.376 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:43:52.376 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.376 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.376 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.377 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:43:52.377 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.377 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.377 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.377 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6801", 2024-09-11T07:43:52.377 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:43:52.377 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.377 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.377 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.377 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:43:52.377 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.377 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.377 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.378 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6803", 2024-09-11T07:43:52.378 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:43:52.378 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.378 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.378 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.378 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:43:52.378 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.378 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.378 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.378 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6802", 2024-09-11T07:43:52.378 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:43:52.378 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.378 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.378 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.379 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6800/1210229443", 2024-09-11T07:43:52.379 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6801/1210229443", 2024-09-11T07:43:52.379 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6803/1210229443", 2024-09-11T07:43:52.379 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6802/1210229443", 2024-09-11T07:43:52.379 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:43:52.379 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:43:52.379 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:43:52.379 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.379 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.379 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.379 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 5, 2024-09-11T07:43:52.379 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "89c9352b-30b2-48c6-869e-047d07a23a43", 2024-09-11T07:43:52.379 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:43:52.379 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:43:52.379 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 0.6509552001953125, 2024-09-11T07:43:52.380 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 0.666046142578125, 2024-09-11T07:43:52.380 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:43:52.380 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:43:52.380 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 37, 2024-09-11T07:43:52.380 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 360, 2024-09-11T07:43:52.380 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:43:52.380 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:43:52.380 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:43:52.380 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.380 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.380 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.380 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6804", 2024-09-11T07:43:52.380 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:43:52.381 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.381 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.381 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.381 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:43:52.381 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.381 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.381 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.381 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6805", 2024-09-11T07:43:52.381 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:43:52.381 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.381 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.381 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.381 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:43:52.382 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.382 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.382 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.382 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6807", 2024-09-11T07:43:52.382 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:43:52.382 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.382 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.382 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.382 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:43:52.382 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.382 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.382 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.382 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6806", 2024-09-11T07:43:52.383 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:43:52.383 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.383 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.383 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.383 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6804/2735328732", 2024-09-11T07:43:52.383 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6805/2735328732", 2024-09-11T07:43:52.383 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6807/2735328732", 2024-09-11T07:43:52.383 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6806/2735328732", 2024-09-11T07:43:52.383 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:43:52.383 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:43:52.383 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:43:52.383 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.383 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.383 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.383 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 6, 2024-09-11T07:43:52.384 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "704563de-d88f-480a-b469-720135704d42", 2024-09-11T07:43:52.384 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:43:52.384 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:43:52.384 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:43:52.384 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 0, 2024-09-11T07:43:52.384 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:43:52.384 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:43:52.384 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 44, 2024-09-11T07:43:52.384 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 177, 2024-09-11T07:43:52.384 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:43:52.384 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:43:52.384 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:43:52.384 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.384 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.385 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.385 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6808", 2024-09-11T07:43:52.385 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:43:52.385 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.385 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.385 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.385 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:43:52.385 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.385 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.385 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.385 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6809", 2024-09-11T07:43:52.385 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:43:52.385 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.385 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.386 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.386 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:43:52.386 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.386 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.386 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.386 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6811", 2024-09-11T07:43:52.386 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:43:52.386 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.386 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.386 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.386 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:43:52.386 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.386 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.386 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.387 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6810", 2024-09-11T07:43:52.387 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:43:52.387 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.387 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.387 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.387 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6808/721493010", 2024-09-11T07:43:52.387 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6809/721493010", 2024-09-11T07:43:52.387 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6811/721493010", 2024-09-11T07:43:52.387 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6810/721493010", 2024-09-11T07:43:52.387 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:43:52.387 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:43:52.387 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:43:52.387 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.387 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.387 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.388 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 7, 2024-09-11T07:43:52.388 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "18199e47-0052-49f0-b45f-0fb201e0d0f4", 2024-09-11T07:43:52.388 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:43:52.388 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:43:52.388 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 0.6657562255859375, 2024-09-11T07:43:52.388 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:43:52.388 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 52, 2024-09-11T07:43:52.388 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 133, 2024-09-11T07:43:52.388 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 147, 2024-09-11T07:43:52.388 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 360, 2024-09-11T07:43:52.388 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 134, 2024-09-11T07:43:52.388 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:43:52.389 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:43:52.389 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.389 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.389 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.389 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6812", 2024-09-11T07:43:52.389 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2218863853 2024-09-11T07:43:52.389 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.389 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.389 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.389 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:43:52.389 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.389 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.389 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.389 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6813", 2024-09-11T07:43:52.390 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:52.391 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2218863853 2024-09-11T07:43:52.391 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.391 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.391 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.391 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:43:52.391 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.391 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.391 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.391 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6815", 2024-09-11T07:43:52.391 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2218863853 2024-09-11T07:43:52.391 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.392 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.392 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.392 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:43:52.392 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:43:52.392 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.392 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:43:52.392 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6814", 2024-09-11T07:43:52.392 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2218863853 2024-09-11T07:43:52.392 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.392 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.392 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.392 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6812/2218863853", 2024-09-11T07:43:52.392 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6813/2218863853", 2024-09-11T07:43:52.392 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6815/2218863853", 2024-09-11T07:43:52.393 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6814/2218863853", 2024-09-11T07:43:52.393 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:43:52.393 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:43:52.393 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:43:52.393 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.393 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.393 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:43:52.393 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_xinfo": [ 2024-09-11T07:43:52.393 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.393 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 0, 2024-09-11T07:43:52.393 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:43:52.393 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:43:52.393 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:43:52.393 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:43:52.394 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:43:52.394 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:43:34.976279+0000", 2024-09-11T07:43:52.394 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:43:52.394 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.394 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.394 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 1, 2024-09-11T07:43:52.394 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:43:52.394 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:43:52.394 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:43:52.394 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:43:52.394 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:43:52.394 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:43:17.772622+0000", 2024-09-11T07:43:52.394 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:43:52.394 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.394 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.395 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 2, 2024-09-11T07:43:52.395 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:43:52.395 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:43:52.395 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:43:52.395 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:43:52.395 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:43:52.395 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:43:23.345678+0000", 2024-09-11T07:43:52.395 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:43:52.395 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.395 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.395 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 3, 2024-09-11T07:43:52.395 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:43:52.395 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:43:52.395 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:43:52.395 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:43:52.396 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:43:52.396 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:43:39.466292+0000", 2024-09-11T07:43:52.396 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:43:52.396 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.396 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.396 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 4, 2024-09-11T07:43:52.396 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:43:52.396 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:43:52.396 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:43:52.396 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:43:52.396 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:43:52.396 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:43:10.409639+0000", 2024-09-11T07:43:52.396 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:43:52.396 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.397 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.397 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 5, 2024-09-11T07:43:52.397 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:43:52.397 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:43:52.397 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:43:52.397 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:43:52.397 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:43:52.397 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:43:14.427937+0000", 2024-09-11T07:43:52.397 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:43:52.397 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.397 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.397 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 6, 2024-09-11T07:43:52.397 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:43:52.398 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:43:52.398 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:43:52.398 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:43:52.398 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:43:52.398 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:43:19.684540+0000", 2024-09-11T07:43:52.398 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:43:52.398 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.398 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.398 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 7, 2024-09-11T07:43:52.398 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:43:52.398 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:43:52.398 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:43:52.398 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:43:52.398 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:43:52.399 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:43:40.591961+0000", 2024-09-11T07:43:52.399 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 133 2024-09-11T07:43:52.399 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.399 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:43:52.399 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_upmap": [ 2024-09-11T07:43:52.399 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.399 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "2.1c", 2024-09-11T07:43:52.399 INFO:teuthology.orchestra.run.smithi022.stdout: "osds": [ 2024-09-11T07:43:52.399 INFO:teuthology.orchestra.run.smithi022.stdout: 5, 2024-09-11T07:43:52.399 INFO:teuthology.orchestra.run.smithi022.stdout: 2, 2024-09-11T07:43:52.399 INFO:teuthology.orchestra.run.smithi022.stdout: 7 2024-09-11T07:43:52.399 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.399 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.399 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.400 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.1e", 2024-09-11T07:43:52.400 INFO:teuthology.orchestra.run.smithi022.stdout: "osds": [ 2024-09-11T07:43:52.400 INFO:teuthology.orchestra.run.smithi022.stdout: 4, 2024-09-11T07:43:52.400 INFO:teuthology.orchestra.run.smithi022.stdout: 5, 2024-09-11T07:43:52.400 INFO:teuthology.orchestra.run.smithi022.stdout: 3 2024-09-11T07:43:52.400 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.400 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.400 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:43:52.400 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_upmap_items": [ 2024-09-11T07:43:52.400 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.400 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.7", 2024-09-11T07:43:52.400 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:43:52.400 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.400 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 3, 2024-09-11T07:43:52.401 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 7 2024-09-11T07:43:52.401 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.401 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.401 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.401 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.401 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.15", 2024-09-11T07:43:52.401 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:43:52.401 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.401 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 3, 2024-09-11T07:43:52.401 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 0 2024-09-11T07:43:52.401 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.401 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.401 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.402 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.402 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.1b", 2024-09-11T07:43:52.402 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:43:52.402 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.402 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 3, 2024-09-11T07:43:52.402 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 7 2024-09-11T07:43:52.402 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.402 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.402 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.402 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.402 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "5.c", 2024-09-11T07:43:52.402 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:43:52.402 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:43:52.403 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 1, 2024-09-11T07:43:52.403 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 2 2024-09-11T07:43:52.403 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.403 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:43:52.403 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.403 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:43:52.403 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_temp": [], 2024-09-11T07:43:52.403 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_temp": [], 2024-09-11T07:43:52.403 INFO:teuthology.orchestra.run.smithi022.stdout: "blocklist": { 2024-09-11T07:43:52.403 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/240999923": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:43:52.403 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/4016369460": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:43:52.404 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/585929238": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:43:52.404 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/544031024": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:43:52.404 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/2504196132": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:43:52.404 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/1451388349": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:43:52.404 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/639336081": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:43:52.404 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2104069730": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:43:52.404 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/4236450731": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:43:52.404 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/3573193776": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:43:52.404 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2661764422": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:43:52.404 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2197488530": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:43:52.404 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2968034970": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:43:52.404 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/599318175": "2024-09-12T07:27:07.919998+0000" 2024-09-11T07:43:52.405 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.405 INFO:teuthology.orchestra.run.smithi022.stdout: "range_blocklist": {}, 2024-09-11T07:43:52.405 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profiles": { 2024-09-11T07:43:52.405 INFO:teuthology.orchestra.run.smithi022.stdout: "default": { 2024-09-11T07:43:52.405 INFO:teuthology.orchestra.run.smithi022.stdout: "crush-failure-domain": "osd", 2024-09-11T07:43:52.405 INFO:teuthology.orchestra.run.smithi022.stdout: "k": "2", 2024-09-11T07:43:52.405 INFO:teuthology.orchestra.run.smithi022.stdout: "m": "1", 2024-09-11T07:43:52.405 INFO:teuthology.orchestra.run.smithi022.stdout: "plugin": "jerasure", 2024-09-11T07:43:52.405 INFO:teuthology.orchestra.run.smithi022.stdout: "technique": "reed_sol_van" 2024-09-11T07:43:52.405 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.405 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:43:52.405 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps_queue": [], 2024-09-11T07:43:52.405 INFO:teuthology.orchestra.run.smithi022.stdout: "new_removed_snaps": [], 2024-09-11T07:43:52.406 INFO:teuthology.orchestra.run.smithi022.stdout: "new_purged_snaps": [], 2024-09-11T07:43:52.406 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_node_flags": {}, 2024-09-11T07:43:52.406 INFO:teuthology.orchestra.run.smithi022.stdout: "device_class_flags": {}, 2024-09-11T07:43:52.406 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode": { 2024-09-11T07:43:52.406 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode_enabled": false, 2024-09-11T07:43:52.406 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_bucket_count": 0, 2024-09-11T07:43:52.406 INFO:teuthology.orchestra.run.smithi022.stdout: "degraded_stretch_mode": 0, 2024-09-11T07:43:52.406 INFO:teuthology.orchestra.run.smithi022.stdout: "recovering_stretch_mode": 0, 2024-09-11T07:43:52.406 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode_bucket": 0 2024-09-11T07:43:52.406 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:43:52.406 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:43:52.439 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:52.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:52.440 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:52.541 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:52.585 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:52.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:52.586 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:52.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:52 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3229312943' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:43:52.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:52 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3229312943' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:43:52.687 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:52.726 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:52.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:52.727 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:52.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:52 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3229312943' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:43:52.829 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:52.854 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:52.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:52.855 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:52.956 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:52.995 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:52.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:52.996 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:53.097 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:53.137 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:53.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:53.138 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:53.239 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:53.275 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:53.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:53.276 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:53.377 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:53.411 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:53.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:53.412 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:53.514 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:53.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:53.552 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:53.552 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:53.653 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:53.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:53 smithi022 ceph-mon[29462]: pgmap v805: 148 pgs: 148 active+clean; 7.7 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1001 B/s rd, 42 MiB/s wr, 1.03k op/s 2024-09-11T07:43:53.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:43:53.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:53 smithi022 ceph-mon[37330]: pgmap v805: 148 pgs: 148 active+clean; 7.7 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1001 B/s rd, 42 MiB/s wr, 1.03k op/s 2024-09-11T07:43:53.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:43:53.692 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:53.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:53.693 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:53.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:53 smithi104 ceph-mon[33507]: pgmap v805: 148 pgs: 148 active+clean; 7.7 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1001 B/s rd, 42 MiB/s wr, 1.03k op/s 2024-09-11T07:43:53.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:43:53.794 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:53.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:53.833 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:53.833 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:53.934 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:53.969 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:53.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:53.970 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:54.071 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:54.104 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:54.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:54.105 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:54.206 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:54.245 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:54.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:54.246 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:54.333 INFO:tasks.thrashosds.thrasher:j is {'epoch': 372, 'fsid': '12a285d8-700e-11ef-bcea-c7b262605968', 'created': '2024-09-11T07:21:18.695332+0000', 'modified': '2024-09-11T07:43:41.052119+0000', 'last_up_change': '2024-09-11T07:32:00.070825+0000', 'last_in_change': '2024-09-11T07:25:58.551813+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 19, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 10, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T07:24:21.772426+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': '.rgw.root', 'create_time': '2024-09-11T07:26:29.941354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 3, 'pool_name': 'datapool', 'create_time': '2024-09-11T07:26:30.396859+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-11T07:26:31.922926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-11T07:26:34.007746+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-11T07:26:36.077215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}, {'pool': 10, 'pool_name': 'unique_pool_3', 'create_time': '2024-09-11T07:41:56.278857+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '349', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '09075f16-32f4-4f88-a116-c3d7f977a0e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 358, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6801', 'nonce': 2821493591}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6802', 'nonce': 2821493591}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6804', 'nonce': 2821493591}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6803', 'nonce': 2821493591}]}, 'public_addr': '172.21.15.22:6801/2821493591', 'cluster_addr': '172.21.15.22:6802/2821493591', 'heartbeat_back_addr': '172.21.15.22:6804/2821493591', 'heartbeat_front_addr': '172.21.15.22:6803/2821493591', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '52cc1e06-9506-48c0-9f7b-d4270dff4b9f', 'up': 1, 'in': 1, 'weight': 0.64752197265625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 346, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6805', 'nonce': 4037911317}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6806', 'nonce': 4037911317}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6808', 'nonce': 4037911317}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6807', 'nonce': 4037911317}]}, 'public_addr': '172.21.15.22:6805/4037911317', 'cluster_addr': '172.21.15.22:6806/4037911317', 'heartbeat_back_addr': '172.21.15.22:6808/4037911317', 'heartbeat_front_addr': '172.21.15.22:6807/4037911317', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'fa2b2730-2c1d-45d0-be8a-494a60dc6b9b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 360, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6809', 'nonce': 1675799666}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6810', 'nonce': 1675799666}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6812', 'nonce': 1675799666}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6811', 'nonce': 1675799666}]}, 'public_addr': '172.21.15.22:6809/1675799666', 'cluster_addr': '172.21.15.22:6810/1675799666', 'heartbeat_back_addr': '172.21.15.22:6812/1675799666', 'heartbeat_front_addr': '172.21.15.22:6811/1675799666', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '8d53d3fc-e401-47df-b70b-4b4b1f4539df', 'up': 1, 'in': 1, 'weight': 0.686370849609375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 25, 'up_thru': 341, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6813', 'nonce': 1501962752}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6814', 'nonce': 1501962752}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6816', 'nonce': 1501962752}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6815', 'nonce': 1501962752}]}, 'public_addr': '172.21.15.22:6813/1501962752', 'cluster_addr': '172.21.15.22:6814/1501962752', 'heartbeat_back_addr': '172.21.15.22:6816/1501962752', 'heartbeat_front_addr': '172.21.15.22:6815/1501962752', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'd3207fe0-8b97-4067-a755-ba6d449c53c7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 360, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6800', 'nonce': 1210229443}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6801', 'nonce': 1210229443}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6803', 'nonce': 1210229443}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6802', 'nonce': 1210229443}]}, 'public_addr': '172.21.15.104:6800/1210229443', 'cluster_addr': '172.21.15.104:6801/1210229443', 'heartbeat_back_addr': '172.21.15.104:6803/1210229443', 'heartbeat_front_addr': '172.21.15.104:6802/1210229443', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '89c9352b-30b2-48c6-869e-047d07a23a43', 'up': 1, 'in': 1, 'weight': 0.6509552001953125, 'primary_affinity': 0.666046142578125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 37, 'up_thru': 360, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6804', 'nonce': 2735328732}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6805', 'nonce': 2735328732}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6807', 'nonce': 2735328732}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6806', 'nonce': 2735328732}]}, 'public_addr': '172.21.15.104:6804/2735328732', 'cluster_addr': '172.21.15.104:6805/2735328732', 'heartbeat_back_addr': '172.21.15.104:6807/2735328732', 'heartbeat_front_addr': '172.21.15.104:6806/2735328732', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '704563de-d88f-480a-b469-720135704d42', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 44, 'up_thru': 177, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6808', 'nonce': 721493010}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6809', 'nonce': 721493010}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6811', 'nonce': 721493010}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6810', 'nonce': 721493010}]}, 'public_addr': '172.21.15.104:6808/721493010', 'cluster_addr': '172.21.15.104:6809/721493010', 'heartbeat_back_addr': '172.21.15.104:6811/721493010', 'heartbeat_front_addr': '172.21.15.104:6810/721493010', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '18199e47-0052-49f0-b45f-0fb201e0d0f4', 'up': 1, 'in': 1, 'weight': 0.6657562255859375, 'primary_affinity': 1, 'last_clean_begin': 52, 'last_clean_end': 133, 'up_from': 147, 'up_thru': 360, 'down_at': 134, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6812', 'nonce': 2218863853}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6813', 'nonce': 2218863853}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6815', 'nonce': 2218863853}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6814', 'nonce': 2218863853}]}, 'public_addr': '172.21.15.104:6812/2218863853', 'cluster_addr': '172.21.15.104:6813/2218863853', 'heartbeat_back_addr': '172.21.15.104:6815/2218863853', 'heartbeat_front_addr': '172.21.15.104:6814/2218863853', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:43:34.976279+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:43:17.772622+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:43:23.345678+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:43:39.466292+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:43:10.409639+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:43:14.427937+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:43:19.684540+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:43:40.591961+0000', 'dead_epoch': 133}], 'pg_upmap': [{'pgid': '2.1c', 'osds': [5, 2, 7]}, {'pgid': '4.1e', 'osds': [4, 5, 3]}], 'pg_upmap_items': [{'pgid': '4.7', 'mappings': [{'from': 3, 'to': 7}]}, {'pgid': '4.15', 'mappings': [{'from': 3, 'to': 0}]}, {'pgid': '4.1b', 'mappings': [{'from': 3, 'to': 7}]}, {'pgid': '5.c', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.22:0/240999923': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:6800/4016369460': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/585929238': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/544031024': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:6800/2504196132': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/1451388349': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/639336081': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/2104069730': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/4236450731': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:6800/3573193776': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/2661764422': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/2197488530': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/2968034970': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/599318175': '2024-09-12T07:27:07.919998+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-11T07:43:54.333 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 5.c 2024-09-11T07:43:54.333 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd rm-pg-upmap-items 5.c 2024-09-11T07:43:54.346 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:54.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:54.375 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:54.375 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:54.410 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:43:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:43:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:43:54.420 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is unset 2024-09-11T07:43:54.476 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:54.518 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:54.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:54.519 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:54.620 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:54.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:54.664 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:54.664 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:54.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:54 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3127287703' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:43:54.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:54 smithi022 ceph-mon[29462]: from='client.25816 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:43:54.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:54 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3127287703' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:43:54.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:54 smithi022 ceph-mon[37330]: from='client.25816 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:43:54.765 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:54.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:54 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3127287703' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:43:54.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:54 smithi104 ceph-mon[33507]: from='client.25816 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:43:54.798 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:54.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:54.799 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:54.900 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:54.932 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:54.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:54.933 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:55.034 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:55.069 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:55.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:55.070 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:55.171 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:55.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:55.205 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:55.205 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:55.253 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-11T07:43:55.306 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:55.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:55.338 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:55.338 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:55.439 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:55.472 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:55.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:55.473 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:55.574 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:55 smithi022 ceph-mon[29462]: pgmap v806: 148 pgs: 148 active+clean; 7.3 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.3 KiB/s rd, 39 MiB/s wr, 1.44k op/s 2024-09-11T07:43:55.574 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:55 smithi022 ceph-mon[29462]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:43:55.574 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:55 smithi022 ceph-mon[29462]: from='client.25816 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:43:55.574 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:55 smithi022 ceph-mon[29462]: osdmap e373: 8 total, 8 up, 8 in 2024-09-11T07:43:55.574 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:55 smithi022 ceph-mon[37330]: pgmap v806: 148 pgs: 148 active+clean; 7.3 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.3 KiB/s rd, 39 MiB/s wr, 1.44k op/s 2024-09-11T07:43:55.574 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:55 smithi022 ceph-mon[37330]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:43:55.574 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:55 smithi022 ceph-mon[37330]: from='client.25816 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:43:55.575 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:55 smithi022 ceph-mon[37330]: osdmap e373: 8 total, 8 up, 8 in 2024-09-11T07:43:55.575 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:55.606 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:55.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:55.607 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:55.691 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:55 smithi104 ceph-mon[33507]: pgmap v806: 148 pgs: 148 active+clean; 7.3 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.3 KiB/s rd, 39 MiB/s wr, 1.44k op/s 2024-09-11T07:43:55.691 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:55 smithi104 ceph-mon[33507]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:43:55.691 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:55 smithi104 ceph-mon[33507]: from='client.25816 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:43:55.691 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:55 smithi104 ceph-mon[33507]: osdmap e373: 8 total, 8 up, 8 in 2024-09-11T07:43:55.708 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:55.744 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:55.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:55.745 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:55.846 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:55.879 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:55.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:55.880 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:55.981 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:56.024 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:56.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:56.025 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:56.127 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:56.159 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:56.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:56.160 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:56.261 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:56.299 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:56.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:56.300 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:56.401 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:56.441 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:56.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:56.442 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:56.508 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:43:56.542 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:56.573 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:56.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:56.574 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:56.675 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:56.721 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:56.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:56.722 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:56.823 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:56.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:56.857 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:56.857 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:56.958 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:56.959 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:56 smithi022 ceph-mon[37330]: 2.a scrub starts 2024-09-11T07:43:56.959 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:56 smithi022 ceph-mon[37330]: 2.a scrub ok 2024-09-11T07:43:56.959 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:56 smithi022 ceph-mon[37330]: 2.16 scrub starts 2024-09-11T07:43:56.959 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:56 smithi022 ceph-mon[37330]: 2.16 scrub ok 2024-09-11T07:43:56.959 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:56 smithi022 ceph-mon[37330]: 4.e scrub starts 2024-09-11T07:43:56.959 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:56 smithi022 ceph-mon[37330]: 4.e scrub ok 2024-09-11T07:43:56.960 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:56 smithi022 ceph-mon[37330]: 10.4 scrub starts 2024-09-11T07:43:56.960 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:56 smithi022 ceph-mon[37330]: 2.1b scrub starts 2024-09-11T07:43:56.960 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:56 smithi022 ceph-mon[37330]: 2.1b scrub ok 2024-09-11T07:43:56.960 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:56 smithi022 ceph-mon[37330]: pgmap v808: 148 pgs: 148 active+clean; 7.3 GiB data, 28 GiB used, 688 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 MiB/s wr, 1.26k op/s 2024-09-11T07:43:56.960 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:56 smithi022 ceph-mon[37330]: 2.0 scrub starts 2024-09-11T07:43:56.960 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:56 smithi022 ceph-mon[37330]: 2.0 scrub ok 2024-09-11T07:43:56.960 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:56 smithi022 ceph-mon[29462]: 2.a scrub starts 2024-09-11T07:43:56.960 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:56 smithi022 ceph-mon[29462]: 2.a scrub ok 2024-09-11T07:43:56.960 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:56 smithi022 ceph-mon[29462]: 2.16 scrub starts 2024-09-11T07:43:56.960 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:56 smithi022 ceph-mon[29462]: 2.16 scrub ok 2024-09-11T07:43:56.960 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:56 smithi022 ceph-mon[29462]: 4.e scrub starts 2024-09-11T07:43:56.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:56 smithi022 ceph-mon[29462]: 4.e scrub ok 2024-09-11T07:43:56.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:56 smithi022 ceph-mon[29462]: 10.4 scrub starts 2024-09-11T07:43:56.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:56 smithi022 ceph-mon[29462]: 2.1b scrub starts 2024-09-11T07:43:56.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:56 smithi022 ceph-mon[29462]: 2.1b scrub ok 2024-09-11T07:43:56.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:56 smithi022 ceph-mon[29462]: pgmap v808: 148 pgs: 148 active+clean; 7.3 GiB data, 28 GiB used, 688 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 MiB/s wr, 1.26k op/s 2024-09-11T07:43:56.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:56 smithi022 ceph-mon[29462]: 2.0 scrub starts 2024-09-11T07:43:56.961 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:56 smithi022 ceph-mon[29462]: 2.0 scrub ok 2024-09-11T07:43:57.003 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:57.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:57.004 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:57.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:56 smithi104 ceph-mon[33507]: 2.a scrub starts 2024-09-11T07:43:57.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:56 smithi104 ceph-mon[33507]: 2.a scrub ok 2024-09-11T07:43:57.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:56 smithi104 ceph-mon[33507]: 2.16 scrub starts 2024-09-11T07:43:57.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:56 smithi104 ceph-mon[33507]: 2.16 scrub ok 2024-09-11T07:43:57.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:56 smithi104 ceph-mon[33507]: 4.e scrub starts 2024-09-11T07:43:57.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:56 smithi104 ceph-mon[33507]: 4.e scrub ok 2024-09-11T07:43:57.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:56 smithi104 ceph-mon[33507]: 10.4 scrub starts 2024-09-11T07:43:57.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:56 smithi104 ceph-mon[33507]: 2.1b scrub starts 2024-09-11T07:43:57.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:56 smithi104 ceph-mon[33507]: 2.1b scrub ok 2024-09-11T07:43:57.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:56 smithi104 ceph-mon[33507]: pgmap v808: 148 pgs: 148 active+clean; 7.3 GiB data, 28 GiB used, 688 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 MiB/s wr, 1.26k op/s 2024-09-11T07:43:57.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:56 smithi104 ceph-mon[33507]: 2.0 scrub starts 2024-09-11T07:43:57.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:56 smithi104 ceph-mon[33507]: 2.0 scrub ok 2024-09-11T07:43:57.105 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:57.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:57.146 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:57.146 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:57.247 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:57.291 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:57.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:57.292 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:57.393 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:57.431 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:57.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:57.432 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:57.533 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:57.557 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:57.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:57.558 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:57.659 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:57.693 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:57.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:57.695 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:57.795 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:57.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:57.840 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:57.840 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:57.941 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:57.979 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:57.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:57.980 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:58.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:57 smithi104 ceph-mon[33507]: 2.1c scrub starts 2024-09-11T07:43:58.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:57 smithi104 ceph-mon[33507]: 2.1c scrub ok 2024-09-11T07:43:58.049 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T07:43:58.081 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:58.081 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:57 smithi022 ceph-mon[29462]: 2.1c scrub starts 2024-09-11T07:43:58.082 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:57 smithi022 ceph-mon[29462]: 2.1c scrub ok 2024-09-11T07:43:58.082 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:57 smithi022 ceph-mon[37330]: 2.1c scrub starts 2024-09-11T07:43:58.082 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:57 smithi022 ceph-mon[37330]: 2.1c scrub ok 2024-09-11T07:43:58.115 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:58.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:58.116 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:58.217 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:43:58.248 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:58.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:58.249 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:43:58.350 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:58.389 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:58.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:58.390 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:58.491 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:58.534 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:58.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:58.535 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:58.636 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:43:58.689 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:58.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:58.690 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:43:58.791 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:43:58.823 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:58.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:58.824 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:43:58.925 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:58.948 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:58.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:58.950 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:59.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:58 smithi104 ceph-mon[33507]: 4.1a scrub starts 2024-09-11T07:43:59.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:58 smithi104 ceph-mon[33507]: 4.1a scrub ok 2024-09-11T07:43:59.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:58 smithi104 ceph-mon[33507]: 6.f scrub starts 2024-09-11T07:43:59.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:58 smithi104 ceph-mon[33507]: 6.f scrub ok 2024-09-11T07:43:59.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:58 smithi104 ceph-mon[33507]: 5.0 scrub starts 2024-09-11T07:43:59.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:58 smithi104 ceph-mon[33507]: 5.0 scrub ok 2024-09-11T07:43:59.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:58 smithi104 ceph-mon[33507]: pgmap v809: 148 pgs: 148 active+clean; 7.3 GiB data, 28 GiB used, 688 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 MiB/s wr, 1.26k op/s 2024-09-11T07:43:59.050 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:43:59.074 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:59.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:59.075 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:43:59.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:58 smithi022 ceph-mon[37330]: 4.1a scrub starts 2024-09-11T07:43:59.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:58 smithi022 ceph-mon[37330]: 4.1a scrub ok 2024-09-11T07:43:59.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:58 smithi022 ceph-mon[37330]: 6.f scrub starts 2024-09-11T07:43:59.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:58 smithi022 ceph-mon[37330]: 6.f scrub ok 2024-09-11T07:43:59.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:58 smithi022 ceph-mon[37330]: 5.0 scrub starts 2024-09-11T07:43:59.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:58 smithi022 ceph-mon[37330]: 5.0 scrub ok 2024-09-11T07:43:59.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:58 smithi022 ceph-mon[37330]: pgmap v809: 148 pgs: 148 active+clean; 7.3 GiB data, 28 GiB used, 688 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 MiB/s wr, 1.26k op/s 2024-09-11T07:43:59.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:58 smithi022 ceph-mon[29462]: 4.1a scrub starts 2024-09-11T07:43:59.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:58 smithi022 ceph-mon[29462]: 4.1a scrub ok 2024-09-11T07:43:59.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:58 smithi022 ceph-mon[29462]: 6.f scrub starts 2024-09-11T07:43:59.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:58 smithi022 ceph-mon[29462]: 6.f scrub ok 2024-09-11T07:43:59.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:58 smithi022 ceph-mon[29462]: 5.0 scrub starts 2024-09-11T07:43:59.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:58 smithi022 ceph-mon[29462]: 5.0 scrub ok 2024-09-11T07:43:59.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:58 smithi022 ceph-mon[29462]: pgmap v809: 148 pgs: 148 active+clean; 7.3 GiB data, 28 GiB used, 688 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 MiB/s wr, 1.26k op/s 2024-09-11T07:43:59.175 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:59.213 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:59.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:59.214 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:59.314 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:59.355 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:59.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:59.356 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:43:59.457 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:43:59.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:59.485 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:59.485 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:43:59.586 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:43:59.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:59.623 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:43:59.623 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:43:59.724 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:43:59.769 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:59.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:59.770 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:43:59.871 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:43:59.910 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:43:59.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:43:59.910 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:00.011 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:00.016 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:59 smithi104 ceph-mon[33507]: 4.f scrub starts 2024-09-11T07:44:00.016 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:59 smithi104 ceph-mon[33507]: 4.f scrub ok 2024-09-11T07:44:00.017 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:59 smithi104 ceph-mon[33507]: 6.4 scrub starts 2024-09-11T07:44:00.017 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:59 smithi104 ceph-mon[33507]: 6.4 scrub ok 2024-09-11T07:44:00.017 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:59 smithi104 ceph-mon[33507]: 10.8 scrub starts 2024-09-11T07:44:00.017 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:43:59 smithi104 ceph-mon[33507]: 10.4 scrub ok 2024-09-11T07:44:00.050 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:00.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:00.052 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:00.152 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:00.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:59 smithi022 ceph-mon[29462]: 4.f scrub starts 2024-09-11T07:44:00.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:59 smithi022 ceph-mon[29462]: 4.f scrub ok 2024-09-11T07:44:00.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:59 smithi022 ceph-mon[29462]: 6.4 scrub starts 2024-09-11T07:44:00.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:59 smithi022 ceph-mon[29462]: 6.4 scrub ok 2024-09-11T07:44:00.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:59 smithi022 ceph-mon[29462]: 10.8 scrub starts 2024-09-11T07:44:00.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:43:59 smithi022 ceph-mon[29462]: 10.4 scrub ok 2024-09-11T07:44:00.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:59 smithi022 ceph-mon[37330]: 4.f scrub starts 2024-09-11T07:44:00.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:59 smithi022 ceph-mon[37330]: 4.f scrub ok 2024-09-11T07:44:00.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:59 smithi022 ceph-mon[37330]: 6.4 scrub starts 2024-09-11T07:44:00.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:59 smithi022 ceph-mon[37330]: 6.4 scrub ok 2024-09-11T07:44:00.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:59 smithi022 ceph-mon[37330]: 10.8 scrub starts 2024-09-11T07:44:00.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:43:59 smithi022 ceph-mon[37330]: 10.4 scrub ok 2024-09-11T07:44:00.180 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:00.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:00.181 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:00.279 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:44:00 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:44:00.283 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:00.323 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:00.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:00.324 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:00.425 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:00.460 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:00.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:00.461 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:00.562 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:00.593 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:00.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:00.594 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:00.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:44:00 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:44:00] "GET /metrics HTTP/1.1" 200 224948 "" "Prometheus/2.43.0" 2024-09-11T07:44:00.695 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:00.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:00.748 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:00.748 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:00.849 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:00.884 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:00.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:00.885 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:00.986 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:01.010 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:00 smithi022 ceph-mon[29462]: 4.18 scrub starts 2024-09-11T07:44:01.011 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:00 smithi022 ceph-mon[29462]: 4.18 scrub ok 2024-09-11T07:44:01.011 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:00 smithi022 ceph-mon[29462]: 4.0 scrub starts 2024-09-11T07:44:01.011 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:00 smithi022 ceph-mon[29462]: 4.0 scrub ok 2024-09-11T07:44:01.011 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:00 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:44:01.011 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:00 smithi022 ceph-mon[29462]: pgmap v810: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.9 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T07:44:01.011 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:00 smithi022 ceph-mon[29462]: 4.15 scrub starts 2024-09-11T07:44:01.011 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:00 smithi022 ceph-mon[29462]: 4.15 scrub ok 2024-09-11T07:44:01.011 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:00 smithi022 ceph-mon[37330]: 4.18 scrub starts 2024-09-11T07:44:01.011 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:00 smithi022 ceph-mon[37330]: 4.18 scrub ok 2024-09-11T07:44:01.011 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:00 smithi022 ceph-mon[37330]: 4.0 scrub starts 2024-09-11T07:44:01.011 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:00 smithi022 ceph-mon[37330]: 4.0 scrub ok 2024-09-11T07:44:01.012 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:00 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:44:01.012 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:00 smithi022 ceph-mon[37330]: pgmap v810: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.9 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T07:44:01.012 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:00 smithi022 ceph-mon[37330]: 4.15 scrub starts 2024-09-11T07:44:01.012 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:00 smithi022 ceph-mon[37330]: 4.15 scrub ok 2024-09-11T07:44:01.019 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:01.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:01.020 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:01.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:00 smithi104 ceph-mon[33507]: 4.18 scrub starts 2024-09-11T07:44:01.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:00 smithi104 ceph-mon[33507]: 4.18 scrub ok 2024-09-11T07:44:01.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:00 smithi104 ceph-mon[33507]: 4.0 scrub starts 2024-09-11T07:44:01.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:00 smithi104 ceph-mon[33507]: 4.0 scrub ok 2024-09-11T07:44:01.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:00 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:44:01.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:00 smithi104 ceph-mon[33507]: pgmap v810: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.9 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T07:44:01.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:00 smithi104 ceph-mon[33507]: 4.15 scrub starts 2024-09-11T07:44:01.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:00 smithi104 ceph-mon[33507]: 4.15 scrub ok 2024-09-11T07:44:01.121 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:01.133 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:44:01.133 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:44:01.133 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:44:01.133 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:44:01.133 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:44:01.133 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:44:01.133 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:44:01.133 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:44:01.141 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:44:01.141 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:44:01.149 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:01.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:01.150 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:01.151 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:44:01.174 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:44:01.174 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:44:01.205 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:44:01.205 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:44:01.234 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:44:01.234 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:44:01.250 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:01.278 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:01.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:01.280 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:01.381 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:01.422 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:01.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:01.423 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:01.524 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:01.560 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:01.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:01.562 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:01.663 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:01.705 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:01.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:01.706 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:01.807 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:01.839 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:01.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:01.840 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:01.941 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:01.978 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:01.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:01.979 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:02.024 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:01 smithi022 ceph-mon[37330]: 6.1b scrub starts 2024-09-11T07:44:02.025 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:01 smithi022 ceph-mon[37330]: 6.1b scrub ok 2024-09-11T07:44:02.025 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:01 smithi022 ceph-mon[37330]: 6.8 scrub starts 2024-09-11T07:44:02.025 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:01 smithi022 ceph-mon[37330]: 6.8 scrub ok 2024-09-11T07:44:02.025 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:01 smithi022 ceph-mon[37330]: 6.1f scrub starts 2024-09-11T07:44:02.025 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:01 smithi022 ceph-mon[37330]: 6.1f scrub ok 2024-09-11T07:44:02.025 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:01 smithi022 ceph-mon[37330]: 10.8 scrub ok 2024-09-11T07:44:02.025 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:01 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/144696463' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:44:02.025 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:01 smithi022 ceph-mon[29462]: 6.1b scrub starts 2024-09-11T07:44:02.025 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:01 smithi022 ceph-mon[29462]: 6.1b scrub ok 2024-09-11T07:44:02.025 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:01 smithi022 ceph-mon[29462]: 6.8 scrub starts 2024-09-11T07:44:02.025 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:01 smithi022 ceph-mon[29462]: 6.8 scrub ok 2024-09-11T07:44:02.025 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:01 smithi022 ceph-mon[29462]: 6.1f scrub starts 2024-09-11T07:44:02.026 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:01 smithi022 ceph-mon[29462]: 6.1f scrub ok 2024-09-11T07:44:02.026 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:01 smithi022 ceph-mon[29462]: 10.8 scrub ok 2024-09-11T07:44:02.026 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:01 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/144696463' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:44:02.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:01 smithi104 ceph-mon[33507]: 6.1b scrub starts 2024-09-11T07:44:02.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:01 smithi104 ceph-mon[33507]: 6.1b scrub ok 2024-09-11T07:44:02.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:01 smithi104 ceph-mon[33507]: 6.8 scrub starts 2024-09-11T07:44:02.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:01 smithi104 ceph-mon[33507]: 6.8 scrub ok 2024-09-11T07:44:02.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:01 smithi104 ceph-mon[33507]: 6.1f scrub starts 2024-09-11T07:44:02.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:01 smithi104 ceph-mon[33507]: 6.1f scrub ok 2024-09-11T07:44:02.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:01 smithi104 ceph-mon[33507]: 10.8 scrub ok 2024-09-11T07:44:02.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:01 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/144696463' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:44:02.080 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:02.111 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:02.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:02.113 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:02.213 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:02.247 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:02.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:02.248 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:02.349 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:02.385 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:02.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:02.386 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:02.487 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:02.531 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:02.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:02.532 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:02.632 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:02.669 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:02.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:02.670 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:02.771 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:02.777 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:44:02.779 INFO:teuthology.orchestra.run.smithi022.stderr:clear 5.c pg_upmap_items mapping 2024-09-11T07:44:02.826 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:02.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:02.828 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:02.929 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:02.970 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:02.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:02.971 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:03.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:02 smithi104 ceph-mon[33507]: 2.12 scrub starts 2024-09-11T07:44:03.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:02 smithi104 ceph-mon[33507]: 2.12 scrub ok 2024-09-11T07:44:03.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:02 smithi104 ceph-mon[33507]: 10.6 scrub starts 2024-09-11T07:44:03.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:02 smithi104 ceph-mon[33507]: 2.19 scrub starts 2024-09-11T07:44:03.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:02 smithi104 ceph-mon[33507]: 2.19 scrub ok 2024-09-11T07:44:03.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:02 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/56991822' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "5.c"}]: dispatch 2024-09-11T07:44:03.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:02 smithi104 ceph-mon[33507]: pgmap v811: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.9 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T07:44:03.072 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:03.111 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:02 smithi022 ceph-mon[29462]: 2.12 scrub starts 2024-09-11T07:44:03.111 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:02 smithi022 ceph-mon[29462]: 2.12 scrub ok 2024-09-11T07:44:03.111 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:02 smithi022 ceph-mon[29462]: 10.6 scrub starts 2024-09-11T07:44:03.111 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:02 smithi022 ceph-mon[29462]: 2.19 scrub starts 2024-09-11T07:44:03.112 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:02 smithi022 ceph-mon[29462]: 2.19 scrub ok 2024-09-11T07:44:03.112 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:02 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/56991822' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "5.c"}]: dispatch 2024-09-11T07:44:03.112 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:02 smithi022 ceph-mon[29462]: pgmap v811: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.9 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T07:44:03.112 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:03.112 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:02 smithi022 ceph-mon[37330]: 2.12 scrub starts 2024-09-11T07:44:03.113 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:02 smithi022 ceph-mon[37330]: 2.12 scrub ok 2024-09-11T07:44:03.113 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:02 smithi022 ceph-mon[37330]: 10.6 scrub starts 2024-09-11T07:44:03.113 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:02 smithi022 ceph-mon[37330]: 2.19 scrub starts 2024-09-11T07:44:03.113 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:02 smithi022 ceph-mon[37330]: 2.19 scrub ok 2024-09-11T07:44:03.113 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:02 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/56991822' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "5.c"}]: dispatch 2024-09-11T07:44:03.113 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:02 smithi022 ceph-mon[37330]: pgmap v811: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.9 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-11T07:44:03.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:03.114 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:03.215 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:03.248 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:03.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:03.249 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:03.350 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:03.381 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:03.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:03.382 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:03.483 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:03.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:03.521 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:03.522 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:03.622 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:03.657 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:03.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:03.658 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:03.759 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:03.800 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:03.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:03.801 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:03.902 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:03.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:03 smithi022 ceph-mon[29462]: 3.1 scrub starts 2024-09-11T07:44:03.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:03 smithi022 ceph-mon[29462]: 3.1 scrub ok 2024-09-11T07:44:03.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:03 smithi022 ceph-mon[29462]: 4.19 scrub starts 2024-09-11T07:44:03.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:03 smithi022 ceph-mon[29462]: 4.19 scrub ok 2024-09-11T07:44:03.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:03 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/56991822' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "5.c"}]': finished 2024-09-11T07:44:03.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:03 smithi022 ceph-mon[29462]: osdmap e374: 8 total, 8 up, 8 in 2024-09-11T07:44:03.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:03 smithi022 ceph-mon[37330]: 3.1 scrub starts 2024-09-11T07:44:03.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:03 smithi022 ceph-mon[37330]: 3.1 scrub ok 2024-09-11T07:44:03.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:03 smithi022 ceph-mon[37330]: 4.19 scrub starts 2024-09-11T07:44:03.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:03 smithi022 ceph-mon[37330]: 4.19 scrub ok 2024-09-11T07:44:03.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:03 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/56991822' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "5.c"}]': finished 2024-09-11T07:44:03.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:03 smithi022 ceph-mon[37330]: osdmap e374: 8 total, 8 up, 8 in 2024-09-11T07:44:03.926 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:44:03.945 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:03.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:03.946 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:04.043 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:03 smithi104 ceph-mon[33507]: 3.1 scrub starts 2024-09-11T07:44:04.043 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:03 smithi104 ceph-mon[33507]: 3.1 scrub ok 2024-09-11T07:44:04.043 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:03 smithi104 ceph-mon[33507]: 4.19 scrub starts 2024-09-11T07:44:04.043 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:03 smithi104 ceph-mon[33507]: 4.19 scrub ok 2024-09-11T07:44:04.043 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:03 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/56991822' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "5.c"}]': finished 2024-09-11T07:44:04.043 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:03 smithi104 ceph-mon[33507]: osdmap e374: 8 total, 8 up, 8 in 2024-09-11T07:44:04.047 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:04.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:04.078 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:04.078 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:04.179 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:04.214 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:04.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:04.215 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:04.316 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:04.343 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:04.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:04.344 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:04.445 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:04.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:04.483 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:04.483 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:04.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:44:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:44:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:44:04.585 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:04.620 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:04.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:04.622 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:04.723 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:04.762 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:04.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:04.763 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:04.864 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:04.895 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:04.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:04.896 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:04.997 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:05.017 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:05.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:05.018 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:05.119 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:05.156 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:05.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:05.157 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:05.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:04 smithi022 ceph-mon[29462]: 6.16 scrub starts 2024-09-11T07:44:05.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:04 smithi022 ceph-mon[29462]: 6.16 scrub ok 2024-09-11T07:44:05.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:04 smithi022 ceph-mon[29462]: 10.3 scrub starts 2024-09-11T07:44:05.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:04 smithi022 ceph-mon[29462]: 4.11 scrub starts 2024-09-11T07:44:05.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:04 smithi022 ceph-mon[29462]: 4.11 scrub ok 2024-09-11T07:44:05.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:04 smithi022 ceph-mon[29462]: osdmap e375: 8 total, 8 up, 8 in 2024-09-11T07:44:05.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:04 smithi022 ceph-mon[29462]: 10.6 scrub ok 2024-09-11T07:44:05.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:04 smithi022 ceph-mon[29462]: pgmap v814: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.6 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.41k op/s 2024-09-11T07:44:05.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:04 smithi022 ceph-mon[37330]: 6.16 scrub starts 2024-09-11T07:44:05.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:04 smithi022 ceph-mon[37330]: 6.16 scrub ok 2024-09-11T07:44:05.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:04 smithi022 ceph-mon[37330]: 10.3 scrub starts 2024-09-11T07:44:05.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:04 smithi022 ceph-mon[37330]: 4.11 scrub starts 2024-09-11T07:44:05.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:04 smithi022 ceph-mon[37330]: 4.11 scrub ok 2024-09-11T07:44:05.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:04 smithi022 ceph-mon[37330]: osdmap e375: 8 total, 8 up, 8 in 2024-09-11T07:44:05.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:04 smithi022 ceph-mon[37330]: 10.6 scrub ok 2024-09-11T07:44:05.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:04 smithi022 ceph-mon[37330]: pgmap v814: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.6 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.41k op/s 2024-09-11T07:44:05.257 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:05.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:04 smithi104 ceph-mon[33507]: 6.16 scrub starts 2024-09-11T07:44:05.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:04 smithi104 ceph-mon[33507]: 6.16 scrub ok 2024-09-11T07:44:05.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:04 smithi104 ceph-mon[33507]: 10.3 scrub starts 2024-09-11T07:44:05.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:04 smithi104 ceph-mon[33507]: 4.11 scrub starts 2024-09-11T07:44:05.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:04 smithi104 ceph-mon[33507]: 4.11 scrub ok 2024-09-11T07:44:05.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:04 smithi104 ceph-mon[33507]: osdmap e375: 8 total, 8 up, 8 in 2024-09-11T07:44:05.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:04 smithi104 ceph-mon[33507]: 10.6 scrub ok 2024-09-11T07:44:05.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:04 smithi104 ceph-mon[33507]: pgmap v814: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.6 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.41k op/s 2024-09-11T07:44:05.290 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:05.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:05.291 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:05.392 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:05.424 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:05.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:05.425 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:05.526 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:05.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:05.569 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:05.569 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:05.670 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:05.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:05.709 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:05.709 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:05.810 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:05.839 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:05.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:05.840 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:05.941 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:05.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:05.974 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:05.975 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:06.075 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:06.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:05 smithi104 ceph-mon[33507]: 6.19 scrub starts 2024-09-11T07:44:06.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:05 smithi104 ceph-mon[33507]: 6.19 scrub ok 2024-09-11T07:44:06.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:05 smithi104 ceph-mon[33507]: 5.16 scrub starts 2024-09-11T07:44:06.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:05 smithi104 ceph-mon[33507]: 6.18 scrub starts 2024-09-11T07:44:06.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:05 smithi104 ceph-mon[33507]: 5.16 scrub ok 2024-09-11T07:44:06.106 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:05 smithi104 ceph-mon[33507]: 6.18 scrub ok 2024-09-11T07:44:06.119 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:06.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:06.120 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:06.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:05 smithi022 ceph-mon[29462]: 6.19 scrub starts 2024-09-11T07:44:06.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:05 smithi022 ceph-mon[29462]: 6.19 scrub ok 2024-09-11T07:44:06.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:05 smithi022 ceph-mon[29462]: 5.16 scrub starts 2024-09-11T07:44:06.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:05 smithi022 ceph-mon[29462]: 6.18 scrub starts 2024-09-11T07:44:06.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:05 smithi022 ceph-mon[29462]: 5.16 scrub ok 2024-09-11T07:44:06.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:05 smithi022 ceph-mon[29462]: 6.18 scrub ok 2024-09-11T07:44:06.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:05 smithi022 ceph-mon[37330]: 6.19 scrub starts 2024-09-11T07:44:06.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:05 smithi022 ceph-mon[37330]: 6.19 scrub ok 2024-09-11T07:44:06.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:05 smithi022 ceph-mon[37330]: 5.16 scrub starts 2024-09-11T07:44:06.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:05 smithi022 ceph-mon[37330]: 6.18 scrub starts 2024-09-11T07:44:06.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:05 smithi022 ceph-mon[37330]: 5.16 scrub ok 2024-09-11T07:44:06.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:05 smithi022 ceph-mon[37330]: 6.18 scrub ok 2024-09-11T07:44:06.221 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:06.245 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:06.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:06.246 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:06.347 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:06.384 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:06.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:06.385 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:06.486 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:06.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:06.522 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:06.522 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:06.623 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:06.657 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:06.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:06.658 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:06.759 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:06.786 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:06.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:06.787 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:06.888 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:06.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:06.923 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:06.923 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:07.024 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:07.063 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:07.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:07.064 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:07.165 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:07.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:06 smithi022 ceph-mon[29462]: 10.a scrub starts 2024-09-11T07:44:07.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:06 smithi022 ceph-mon[29462]: 10.3 scrub ok 2024-09-11T07:44:07.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:06 smithi022 ceph-mon[29462]: pgmap v815: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.5 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.50k op/s 2024-09-11T07:44:07.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:06 smithi022 ceph-mon[37330]: 10.a scrub starts 2024-09-11T07:44:07.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:06 smithi022 ceph-mon[37330]: 10.3 scrub ok 2024-09-11T07:44:07.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:06 smithi022 ceph-mon[37330]: pgmap v815: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.5 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.50k op/s 2024-09-11T07:44:07.204 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:07.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:07.205 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:07.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:06 smithi104 ceph-mon[33507]: 10.a scrub starts 2024-09-11T07:44:07.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:06 smithi104 ceph-mon[33507]: 10.3 scrub ok 2024-09-11T07:44:07.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:06 smithi104 ceph-mon[33507]: pgmap v815: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.5 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.50k op/s 2024-09-11T07:44:07.306 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:07.340 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:07.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:07.341 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:07.442 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:07.477 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:07.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:07.478 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:07.579 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:07.625 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:07.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:07.626 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:07.727 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:07.748 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:07.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:07.749 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:07.850 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:07.872 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:07.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:07.873 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:07.973 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:08.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:08.008 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:08.009 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:08.109 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:08.144 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:08.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:08.145 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:08.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:07 smithi022 ceph-mon[29462]: 4.d scrub starts 2024-09-11T07:44:08.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:07 smithi022 ceph-mon[29462]: 4.d scrub ok 2024-09-11T07:44:08.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:07 smithi022 ceph-mon[29462]: 10.5 scrub starts 2024-09-11T07:44:08.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:07 smithi022 ceph-mon[37330]: 4.d scrub starts 2024-09-11T07:44:08.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:07 smithi022 ceph-mon[37330]: 4.d scrub ok 2024-09-11T07:44:08.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:07 smithi022 ceph-mon[37330]: 10.5 scrub starts 2024-09-11T07:44:08.246 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:08.267 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:08.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:08.268 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:08.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:07 smithi104 ceph-mon[33507]: 4.d scrub starts 2024-09-11T07:44:08.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:07 smithi104 ceph-mon[33507]: 4.d scrub ok 2024-09-11T07:44:08.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:07 smithi104 ceph-mon[33507]: 10.5 scrub starts 2024-09-11T07:44:08.369 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:08.400 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:08.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:08.402 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:08.502 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:08.534 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:08.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:08.535 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:08.636 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:08.671 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:08.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:08.672 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:08.773 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:08.806 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:08.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:08.807 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:08.907 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:08.938 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:08.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:08.939 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:09.039 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:44:09.039 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:44:09.039 INFO:teuthology.orchestra.run.smithi022.stdout: "complaint_time": 30, 2024-09-11T07:44:09.039 INFO:teuthology.orchestra.run.smithi022.stdout: "num_blocked_ops": 0 2024-09-11T07:44:09.039 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:44:09.041 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:09.048 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:44:09.048 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:44:09.048 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-11T07:44:09.048 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.000000 2024-09-11T07:44:09.048 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 1 0 2024-09-11T07:44:09.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:09.073 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:09.073 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:09.084 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:08 smithi022 ceph-mon[29462]: 10.a scrub ok 2024-09-11T07:44:09.084 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:44:09.084 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:08 smithi022 ceph-mon[29462]: pgmap v816: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.5 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 765 op/s 2024-09-11T07:44:09.084 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:08 smithi022 ceph-mon[29462]: 10.c scrub starts 2024-09-11T07:44:09.084 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:08 smithi022 ceph-mon[29462]: 5.a scrub starts 2024-09-11T07:44:09.085 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:08 smithi022 ceph-mon[29462]: 5.a scrub ok 2024-09-11T07:44:09.085 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:08 smithi022 ceph-mon[37330]: 10.a scrub ok 2024-09-11T07:44:09.085 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:44:09.085 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:08 smithi022 ceph-mon[37330]: pgmap v816: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.5 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 765 op/s 2024-09-11T07:44:09.085 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:08 smithi022 ceph-mon[37330]: 10.c scrub starts 2024-09-11T07:44:09.085 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:08 smithi022 ceph-mon[37330]: 5.a scrub starts 2024-09-11T07:44:09.085 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:08 smithi022 ceph-mon[37330]: 5.a scrub ok 2024-09-11T07:44:09.174 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:09.207 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:09.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:09.208 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:09.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:08 smithi104 ceph-mon[33507]: 10.a scrub ok 2024-09-11T07:44:09.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:44:09.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:08 smithi104 ceph-mon[33507]: pgmap v816: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.5 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 765 op/s 2024-09-11T07:44:09.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:08 smithi104 ceph-mon[33507]: 10.c scrub starts 2024-09-11T07:44:09.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:08 smithi104 ceph-mon[33507]: 5.a scrub starts 2024-09-11T07:44:09.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:08 smithi104 ceph-mon[33507]: 5.a scrub ok 2024-09-11T07:44:09.309 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:09.336 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:09.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:09.337 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:09.437 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:09.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:09.459 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:09.460 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:09.561 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:09.587 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:09.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:09.589 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:09.689 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:09.728 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:09.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:09.729 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:09.830 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:09.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:09.871 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:09.871 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:09.972 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:10.018 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:10.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:10.019 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:10.120 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:10.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:10.157 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:10.158 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:10.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:09 smithi022 ceph-mon[29462]: 10.b scrub starts 2024-09-11T07:44:10.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:09 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.7", "id": [1, 2]}]: dispatch 2024-09-11T07:44:10.166 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:09 smithi022 ceph-mon[37330]: 10.b scrub starts 2024-09-11T07:44:10.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:09 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.7", "id": [1, 2]}]: dispatch 2024-09-11T07:44:10.258 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:10.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:09 smithi104 ceph-mon[33507]: 10.b scrub starts 2024-09-11T07:44:10.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:09 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.7", "id": [1, 2]}]: dispatch 2024-09-11T07:44:10.279 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:44:10 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:44:10.293 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:10.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:10.294 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:10.395 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:10.427 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:10.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:10.428 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:10.490 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-11T07:44:10.529 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:10.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:10.579 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:10.641 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:44:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:44:10] "GET /metrics HTTP/1.1" 200 224931 "" "Prometheus/2.43.0" 2024-09-11T07:44:10.680 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:10.728 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:10.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:10.729 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:10.830 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:10.890 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:10.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:10.891 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:10.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:10 smithi022 ceph-mon[37330]: 2.f scrub starts 2024-09-11T07:44:10.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:10 smithi022 ceph-mon[37330]: 6.13 scrub starts 2024-09-11T07:44:10.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:10 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.7", "id": [1, 2]}]': finished 2024-09-11T07:44:10.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:10 smithi022 ceph-mon[37330]: osdmap e376: 8 total, 8 up, 8 in 2024-09-11T07:44:10.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:10 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:44:10.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:10 smithi022 ceph-mon[37330]: pgmap v818: 148 pgs: 3 active+clean+scrubbing, 145 active+clean; 6.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.51k op/s 2024-09-11T07:44:10.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:10 smithi022 ceph-mon[37330]: 5.14 scrub starts 2024-09-11T07:44:10.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:10 smithi022 ceph-mon[37330]: 5.14 scrub ok 2024-09-11T07:44:10.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:10 smithi022 ceph-mon[29462]: 2.f scrub starts 2024-09-11T07:44:10.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:10 smithi022 ceph-mon[29462]: 6.13 scrub starts 2024-09-11T07:44:10.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:10 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.7", "id": [1, 2]}]': finished 2024-09-11T07:44:10.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:10 smithi022 ceph-mon[29462]: osdmap e376: 8 total, 8 up, 8 in 2024-09-11T07:44:10.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:10 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:44:10.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:10 smithi022 ceph-mon[29462]: pgmap v818: 148 pgs: 3 active+clean+scrubbing, 145 active+clean; 6.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.51k op/s 2024-09-11T07:44:10.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:10 smithi022 ceph-mon[29462]: 5.14 scrub starts 2024-09-11T07:44:10.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:10 smithi022 ceph-mon[29462]: 5.14 scrub ok 2024-09-11T07:44:10.992 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:11.019 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:11.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:11.020 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:11.088 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:44:11.121 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:11.145 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:44:11.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:11.199 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:11.199 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:11.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:10 smithi104 ceph-mon[33507]: 2.f scrub starts 2024-09-11T07:44:11.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:10 smithi104 ceph-mon[33507]: 6.13 scrub starts 2024-09-11T07:44:11.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:10 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.7", "id": [1, 2]}]': finished 2024-09-11T07:44:11.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:10 smithi104 ceph-mon[33507]: osdmap e376: 8 total, 8 up, 8 in 2024-09-11T07:44:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:10 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:44:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:10 smithi104 ceph-mon[33507]: pgmap v818: 148 pgs: 3 active+clean+scrubbing, 145 active+clean; 6.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.51k op/s 2024-09-11T07:44:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:10 smithi104 ceph-mon[33507]: 5.14 scrub starts 2024-09-11T07:44:11.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:10 smithi104 ceph-mon[33507]: 5.14 scrub ok 2024-09-11T07:44:11.300 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:11.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:11.366 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:11.366 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:11.467 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:11.510 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:11.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:11.511 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:11.612 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:11.660 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:11.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:11.661 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:11.762 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:11.799 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:11.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:11.800 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:11.901 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:11.945 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:11.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:11.947 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:12.047 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:12.082 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:12.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:12.083 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:12.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:11 smithi022 ceph-mon[29462]: 2.1e scrub starts 2024-09-11T07:44:12.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:11 smithi022 ceph-mon[29462]: osdmap e377: 8 total, 8 up, 8 in 2024-09-11T07:44:12.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:11 smithi022 ceph-mon[29462]: 10.b scrub ok 2024-09-11T07:44:12.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:11 smithi022 ceph-mon[29462]: 10.5 scrub ok 2024-09-11T07:44:12.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:11 smithi022 ceph-mon[37330]: 2.1e scrub starts 2024-09-11T07:44:12.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:11 smithi022 ceph-mon[37330]: osdmap e377: 8 total, 8 up, 8 in 2024-09-11T07:44:12.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:11 smithi022 ceph-mon[37330]: 10.b scrub ok 2024-09-11T07:44:12.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:11 smithi022 ceph-mon[37330]: 10.5 scrub ok 2024-09-11T07:44:12.184 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:12.219 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:12.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:12.220 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:12.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:11 smithi104 ceph-mon[33507]: 2.1e scrub starts 2024-09-11T07:44:12.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:11 smithi104 ceph-mon[33507]: osdmap e377: 8 total, 8 up, 8 in 2024-09-11T07:44:12.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:11 smithi104 ceph-mon[33507]: 10.b scrub ok 2024-09-11T07:44:12.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:11 smithi104 ceph-mon[33507]: 10.5 scrub ok 2024-09-11T07:44:12.321 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:12.365 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:12.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:12.366 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:12.467 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:12.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:12.510 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:12.511 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:12.598 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:44:12.611 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:12.639 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:12.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:12.640 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:12.741 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:12.778 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:12.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:12.780 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:12.880 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:12.917 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:12.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:12.918 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:13.019 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:13.048 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:13.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:13.050 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:13.128 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:12 smithi022 ceph-mon[37330]: 2.17 scrub starts 2024-09-11T07:44:13.128 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:12 smithi022 ceph-mon[37330]: 2.17 scrub ok 2024-09-11T07:44:13.129 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:12 smithi022 ceph-mon[37330]: 10.c scrub ok 2024-09-11T07:44:13.129 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:12 smithi022 ceph-mon[37330]: 4.1f scrub starts 2024-09-11T07:44:13.129 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:12 smithi022 ceph-mon[37330]: 4.1f scrub ok 2024-09-11T07:44:13.129 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:12 smithi022 ceph-mon[37330]: 5.2 scrub starts 2024-09-11T07:44:13.129 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:12 smithi022 ceph-mon[37330]: 5.2 scrub ok 2024-09-11T07:44:13.129 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:12 smithi022 ceph-mon[37330]: pgmap v820: 148 pgs: 3 active+clean+scrubbing, 145 active+clean; 6.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 755 op/s 2024-09-11T07:44:13.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:12 smithi022 ceph-mon[29462]: 2.17 scrub starts 2024-09-11T07:44:13.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:12 smithi022 ceph-mon[29462]: 2.17 scrub ok 2024-09-11T07:44:13.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:12 smithi022 ceph-mon[29462]: 10.c scrub ok 2024-09-11T07:44:13.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:12 smithi022 ceph-mon[29462]: 4.1f scrub starts 2024-09-11T07:44:13.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:12 smithi022 ceph-mon[29462]: 4.1f scrub ok 2024-09-11T07:44:13.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:12 smithi022 ceph-mon[29462]: 5.2 scrub starts 2024-09-11T07:44:13.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:12 smithi022 ceph-mon[29462]: 5.2 scrub ok 2024-09-11T07:44:13.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:12 smithi022 ceph-mon[29462]: pgmap v820: 148 pgs: 3 active+clean+scrubbing, 145 active+clean; 6.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 755 op/s 2024-09-11T07:44:13.151 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:13.200 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:13.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:13.201 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:13.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:12 smithi104 ceph-mon[33507]: 2.17 scrub starts 2024-09-11T07:44:13.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:12 smithi104 ceph-mon[33507]: 2.17 scrub ok 2024-09-11T07:44:13.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:12 smithi104 ceph-mon[33507]: 10.c scrub ok 2024-09-11T07:44:13.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:12 smithi104 ceph-mon[33507]: 4.1f scrub starts 2024-09-11T07:44:13.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:12 smithi104 ceph-mon[33507]: 4.1f scrub ok 2024-09-11T07:44:13.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:12 smithi104 ceph-mon[33507]: 5.2 scrub starts 2024-09-11T07:44:13.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:12 smithi104 ceph-mon[33507]: 5.2 scrub ok 2024-09-11T07:44:13.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:12 smithi104 ceph-mon[33507]: pgmap v820: 148 pgs: 3 active+clean+scrubbing, 145 active+clean; 6.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 755 op/s 2024-09-11T07:44:13.302 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:13.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:13.337 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:13.337 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:13.438 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:13.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:13.474 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:13.475 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:13.575 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:13.621 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:13.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:13.622 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:13.723 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:13.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:13.759 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:13.759 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:13.860 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:13.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:13.897 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:13.897 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:13.998 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:14.033 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:13 smithi022 ceph-mon[37330]: 6.12 scrub starts 2024-09-11T07:44:14.033 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:13 smithi022 ceph-mon[37330]: 6.12 scrub ok 2024-09-11T07:44:14.036 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:13 smithi022 ceph-mon[29462]: 6.12 scrub starts 2024-09-11T07:44:14.036 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:13 smithi022 ceph-mon[29462]: 6.12 scrub ok 2024-09-11T07:44:14.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:14.053 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:14.054 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:14.155 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:14.200 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:14.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:14.201 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:14.280 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:44:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:44:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:44:14.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:13 smithi104 ceph-mon[33507]: 6.12 scrub starts 2024-09-11T07:44:14.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:13 smithi104 ceph-mon[33507]: 6.12 scrub ok 2024-09-11T07:44:14.302 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:14.345 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:14.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:14.346 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:14.447 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:14.481 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:14.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:14.482 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:14.583 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:14.630 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:14.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:14.631 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:14.700 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:44:14.700 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 20, 2024-09-11T07:44:14.700 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 600, 2024-09-11T07:44:14.701 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:44:14.701 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.701 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:85945 9.15 9:ade5fd86:::benchmark_data_smithi022_133943_object85944:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e259)", 2024-09-11T07:44:14.701 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:26.360905+0000", 2024-09-11T07:44:14.701 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 348.33450904400001, 2024-09-11T07:44:14.701 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.024384057, 2024-09-11T07:44:14.701 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:44:14.701 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:44:14.701 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:44:14.701 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:44:14.701 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:44:14.701 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 85945 2024-09-11T07:44:14.702 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.702 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:44:14.702 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.702 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:44:14.702 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360905+0000", 2024-09-11T07:44:14.702 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.702 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.702 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.702 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:44:14.702 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360905+0000", 2024-09-11T07:44:14.702 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.702 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.702 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.702 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:44:14.703 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360904+0000", 2024-09-11T07:44:14.703 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:44:14.703 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.703 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.703 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:44:14.703 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360927+0000", 2024-09-11T07:44:14.703 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2274999999999999e-05 2024-09-11T07:44:14.703 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.703 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.703 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:44:14.703 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360932+0000", 2024-09-11T07:44:14.703 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2650000000000004e-06 2024-09-11T07:44:14.703 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.704 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.704 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:44:14.704 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360935+0000", 2024-09-11T07:44:14.704 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.303e-06 2024-09-11T07:44:14.704 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.704 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.704 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.704 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.461352+0000", 2024-09-11T07:44:14.704 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.10041712 2024-09-11T07:44:14.704 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.704 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.704 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:44:14.704 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.461362+0000", 2024-09-11T07:44:14.705 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0285999999999999e-05 2024-09-11T07:44:14.705 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.705 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.705 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.705 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.384927+0000", 2024-09-11T07:44:14.705 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.92356409100000003 2024-09-11T07:44:14.705 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.705 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.705 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:44:14.705 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.385289+0000", 2024-09-11T07:44:14.705 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00036233199999999999 2024-09-11T07:44:14.705 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.705 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:44:14.706 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.706 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.706 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.706 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:85949 9.9 9:956e8bba:::benchmark_data_smithi022_133943_object85948:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e259)", 2024-09-11T07:44:14.706 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:26.360938+0000", 2024-09-11T07:44:14.706 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 348.33447565300003, 2024-09-11T07:44:14.706 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.021308804, 2024-09-11T07:44:14.706 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:44:14.706 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:44:14.706 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:44:14.707 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:44:14.707 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:44:14.707 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 85949 2024-09-11T07:44:14.707 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.707 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:44:14.707 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.707 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:44:14.707 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360938+0000", 2024-09-11T07:44:14.707 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.707 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.707 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.707 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:44:14.707 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360938+0000", 2024-09-11T07:44:14.708 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.708 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.708 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.708 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:44:14.708 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360938+0000", 2024-09-11T07:44:14.708 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:44:14.708 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.708 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.708 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:44:14.708 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360995+0000", 2024-09-11T07:44:14.708 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7525999999999998e-05 2024-09-11T07:44:14.708 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.708 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.709 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:44:14.709 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.361001+0000", 2024-09-11T07:44:14.709 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3480000000000003e-06 2024-09-11T07:44:14.709 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.709 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.709 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:44:14.709 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.361004+0000", 2024-09-11T07:44:14.709 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5370000000000002e-06 2024-09-11T07:44:14.709 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.709 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.709 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.709 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.482968+0000", 2024-09-11T07:44:14.709 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.12196362700000001 2024-09-11T07:44:14.709 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.710 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.710 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:44:14.710 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.482978+0000", 2024-09-11T07:44:14.710 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.8260000000000005e-06 2024-09-11T07:44:14.710 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.710 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.710 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.710 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.369380+0000", 2024-09-11T07:44:14.710 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.886402201 2024-09-11T07:44:14.710 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.710 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.710 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:44:14.710 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.369511+0000", 2024-09-11T07:44:14.711 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013084699999999999 2024-09-11T07:44:14.711 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.711 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.711 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,7", 2024-09-11T07:44:14.711 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.369654+0000", 2024-09-11T07:44:14.711 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014310200000000001 2024-09-11T07:44:14.711 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.711 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.711 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:44:14.711 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.371458+0000", 2024-09-11T07:44:14.711 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001804436 2024-09-11T07:44:14.711 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.711 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.711 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.712 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.378473+0000", 2024-09-11T07:44:14.712 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0070152299999999999 2024-09-11T07:44:14.712 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.712 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.712 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.712 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.382167+0000", 2024-09-11T07:44:14.712 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.003693694 2024-09-11T07:44:14.712 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.712 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.712 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:44:14.712 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.382213+0000", 2024-09-11T07:44:14.712 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5469999999999997e-05 2024-09-11T07:44:14.713 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.713 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.713 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:44:14.713 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.382247+0000", 2024-09-11T07:44:14.713 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4468000000000003e-05 2024-09-11T07:44:14.713 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.713 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:44:14.713 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.713 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.713 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.713 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87210 9.19 9:9bd7b134:::benchmark_data_smithi022_133943_object87209:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:44:14.713 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.341865+0000", 2024-09-11T07:44:14.713 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 345.35354870499998, 2024-09-11T07:44:14.713 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.225729383, 2024-09-11T07:44:14.714 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:44:14.714 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:44:14.714 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:44:14.714 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:44:14.714 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:44:14.714 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87210 2024-09-11T07:44:14.714 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.714 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:44:14.714 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.714 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:44:14.714 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341865+0000", 2024-09-11T07:44:14.714 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.715 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.715 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.715 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:44:14.715 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341865+0000", 2024-09-11T07:44:14.715 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.715 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.715 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.716 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:44:14.716 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341863+0000", 2024-09-11T07:44:14.716 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:44:14.716 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.716 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.716 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:44:14.716 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341890+0000", 2024-09-11T07:44:14.716 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7458999999999999e-05 2024-09-11T07:44:14.716 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.716 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.716 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:44:14.716 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341903+0000", 2024-09-11T07:44:14.716 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3132999999999999e-05 2024-09-11T07:44:14.717 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.717 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.717 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:44:14.717 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341911+0000", 2024-09-11T07:44:14.717 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.8699999999999992e-06 2024-09-11T07:44:14.717 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.717 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.717 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.717 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342621+0000", 2024-09-11T07:44:14.717 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00070930199999999998 2024-09-11T07:44:14.717 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.717 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.717 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:44:14.718 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342627+0000", 2024-09-11T07:44:14.718 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0129999999999999e-06 2024-09-11T07:44:14.718 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.718 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.718 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.718 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.519201+0000", 2024-09-11T07:44:14.718 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.176574606 2024-09-11T07:44:14.718 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.718 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.718 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:44:14.718 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.519326+0000", 2024-09-11T07:44:14.718 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012495799999999999 2024-09-11T07:44:14.718 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.718 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.719 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,5,6", 2024-09-11T07:44:14.719 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.519682+0000", 2024-09-11T07:44:14.719 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00035572700000000002 2024-09-11T07:44:14.719 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.719 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.719 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.719 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.560173+0000", 2024-09-11T07:44:14.719 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.040491003999999997 2024-09-11T07:44:14.719 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.719 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.719 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.719 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.560894+0000", 2024-09-11T07:44:14.719 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00072137700000000004 2024-09-11T07:44:14.720 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.720 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.720 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.720 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.561062+0000", 2024-09-11T07:44:14.720 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000167658 2024-09-11T07:44:14.720 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.720 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.720 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:44:14.720 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.564965+0000", 2024-09-11T07:44:14.720 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0039035150000000002 2024-09-11T07:44:14.720 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.720 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.721 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:44:14.721 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.565018+0000", 2024-09-11T07:44:14.721 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2139999999999999e-05 2024-09-11T07:44:14.721 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.721 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.721 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:44:14.721 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.567595+0000", 2024-09-11T07:44:14.721 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0025769970000000001 2024-09-11T07:44:14.721 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.721 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:44:14.721 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.721 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.721 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.722 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87222 9.18 9:190c7e59:::benchmark_data_smithi022_133943_object87221:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:44:14.722 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.341925+0000", 2024-09-11T07:44:14.722 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 345.35348904900002, 2024-09-11T07:44:14.722 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.119733941, 2024-09-11T07:44:14.722 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:44:14.722 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:44:14.722 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:44:14.722 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:44:14.722 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:44:14.722 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87222 2024-09-11T07:44:14.722 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.722 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:44:14.722 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.723 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:44:14.723 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341925+0000", 2024-09-11T07:44:14.723 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.723 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.723 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.723 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:44:14.723 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341925+0000", 2024-09-11T07:44:14.723 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.723 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.723 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.723 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:44:14.723 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341924+0000", 2024-09-11T07:44:14.723 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:44:14.724 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.724 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.724 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:44:14.724 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341945+0000", 2024-09-11T07:44:14.724 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1024000000000001e-05 2024-09-11T07:44:14.724 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.724 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.724 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:44:14.724 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341952+0000", 2024-09-11T07:44:14.724 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.545e-06 2024-09-11T07:44:14.724 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.724 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.724 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:44:14.725 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341956+0000", 2024-09-11T07:44:14.725 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7589999999999997e-06 2024-09-11T07:44:14.725 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.725 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.725 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.725 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342444+0000", 2024-09-11T07:44:14.725 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00048726500000000002 2024-09-11T07:44:14.725 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.725 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.725 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:44:14.725 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342452+0000", 2024-09-11T07:44:14.725 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.1810000000000001e-06 2024-09-11T07:44:14.725 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.725 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.726 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.726 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.449925+0000", 2024-09-11T07:44:14.726 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.107473197 2024-09-11T07:44:14.726 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.726 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.726 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:44:14.726 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.450079+0000", 2024-09-11T07:44:14.726 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000153475 2024-09-11T07:44:14.726 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.726 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.726 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,5,6,7", 2024-09-11T07:44:14.726 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.450248+0000", 2024-09-11T07:44:14.726 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00016988499999999999 2024-09-11T07:44:14.727 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.727 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.727 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.727 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.456414+0000", 2024-09-11T07:44:14.727 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0061653100000000002 2024-09-11T07:44:14.727 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.727 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.727 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.727 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.460051+0000", 2024-09-11T07:44:14.727 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0036376450000000001 2024-09-11T07:44:14.727 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.727 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.727 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.728 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.460085+0000", 2024-09-11T07:44:14.728 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3268000000000001e-05 2024-09-11T07:44:14.728 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.728 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.728 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.728 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.460115+0000", 2024-09-11T07:44:14.728 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0292999999999999e-05 2024-09-11T07:44:14.728 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.728 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.728 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:44:14.728 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461466+0000", 2024-09-11T07:44:14.728 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0013508859999999999 2024-09-11T07:44:14.728 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.728 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.729 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:44:14.729 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461524+0000", 2024-09-11T07:44:14.729 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8607000000000003e-05 2024-09-11T07:44:14.729 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.729 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.729 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:44:14.729 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461659+0000", 2024-09-11T07:44:14.729 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000134358 2024-09-11T07:44:14.729 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.729 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:44:14.729 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.729 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.729 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.730 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87225 9.19 9:9ba55202:::benchmark_data_smithi022_133943_object87224:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:44:14.730 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.341961+0000", 2024-09-11T07:44:14.730 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 345.35345301799998, 2024-09-11T07:44:14.730 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2429198539999999, 2024-09-11T07:44:14.730 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:44:14.730 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:44:14.730 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:44:14.730 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:44:14.730 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:44:14.730 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87225 2024-09-11T07:44:14.730 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.730 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:44:14.730 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.731 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:44:14.731 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341961+0000", 2024-09-11T07:44:14.731 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.731 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.731 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.731 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:44:14.731 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341961+0000", 2024-09-11T07:44:14.731 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.731 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.731 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.731 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:44:14.731 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341960+0000", 2024-09-11T07:44:14.731 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:44:14.732 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.732 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.732 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:44:14.732 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341979+0000", 2024-09-11T07:44:14.732 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8606000000000001e-05 2024-09-11T07:44:14.732 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.732 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.732 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:44:14.732 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341985+0000", 2024-09-11T07:44:14.732 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1319999999999999e-06 2024-09-11T07:44:14.732 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.732 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.732 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:44:14.733 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341989+0000", 2024-09-11T07:44:14.733 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9469999999999996e-06 2024-09-11T07:44:14.733 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.733 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.733 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.733 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342643+0000", 2024-09-11T07:44:14.733 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00065436200000000002 2024-09-11T07:44:14.733 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.733 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.733 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:44:14.733 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342648+0000", 2024-09-11T07:44:14.733 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6759999999999998e-06 2024-09-11T07:44:14.733 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.561081+0000", 2024-09-11T07:44:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2184332229999999 2024-09-11T07:44:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:44:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.561226+0000", 2024-09-11T07:44:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014486300000000001 2024-09-11T07:44:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,5,6", 2024-09-11T07:44:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.561922+0000", 2024-09-11T07:44:14.734 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00069608500000000004 2024-09-11T07:44:14.735 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.735 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.735 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.735 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.567646+0000", 2024-09-11T07:44:14.735 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0057242279999999996 2024-09-11T07:44:14.735 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.735 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.735 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.735 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.567751+0000", 2024-09-11T07:44:14.735 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000104591 2024-09-11T07:44:14.735 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.735 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.735 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:44:14.736 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.574420+0000", 2024-09-11T07:44:14.736 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0066688900000000002 2024-09-11T07:44:14.736 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.736 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.736 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.736 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.582333+0000", 2024-09-11T07:44:14.736 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0079128410000000003 2024-09-11T07:44:14.736 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.736 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.736 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:44:14.736 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.582396+0000", 2024-09-11T07:44:14.736 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2976000000000001e-05 2024-09-11T07:44:14.736 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.737 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.737 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:44:14.737 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.584881+0000", 2024-09-11T07:44:14.737 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002485012 2024-09-11T07:44:14.737 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.737 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:44:14.737 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.737 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.737 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.737 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87240 9.18 9:1d562e34:::benchmark_data_smithi022_133943_object87239:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:44:14.737 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.341993+0000", 2024-09-11T07:44:14.737 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 345.35342091299998, 2024-09-11T07:44:14.737 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.121035673, 2024-09-11T07:44:14.738 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:44:14.738 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:44:14.738 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:44:14.738 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:44:14.738 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:44:14.738 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87240 2024-09-11T07:44:14.738 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.738 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:44:14.738 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.738 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:44:14.739 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341993+0000", 2024-09-11T07:44:14.739 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.739 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.739 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.739 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:44:14.739 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341993+0000", 2024-09-11T07:44:14.739 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.739 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.739 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.739 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:44:14.739 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341992+0000", 2024-09-11T07:44:14.739 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:44:14.739 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.740 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.740 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:44:14.740 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342012+0000", 2024-09-11T07:44:14.740 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9117999999999999e-05 2024-09-11T07:44:14.740 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.740 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.740 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:44:14.740 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342017+0000", 2024-09-11T07:44:14.740 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7559999999999996e-06 2024-09-11T07:44:14.740 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.740 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.740 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:44:14.740 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342021+0000", 2024-09-11T07:44:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8639999999999998e-06 2024-09-11T07:44:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342489+0000", 2024-09-11T07:44:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00046751600000000001 2024-09-11T07:44:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:44:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342495+0000", 2024-09-11T07:44:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3720000000000001e-06 2024-09-11T07:44:14.741 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.452736+0000", 2024-09-11T07:44:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.110240694 2024-09-11T07:44:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:44:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.452864+0000", 2024-09-11T07:44:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012804299999999999 2024-09-11T07:44:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,5,6,7", 2024-09-11T07:44:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.453027+0000", 2024-09-11T07:44:14.742 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00016329100000000001 2024-09-11T07:44:14.743 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.743 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.743 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.743 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.459540+0000", 2024-09-11T07:44:14.743 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0065125319999999997 2024-09-11T07:44:14.743 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.743 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.743 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.743 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.460175+0000", 2024-09-11T07:44:14.743 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00063524399999999998 2024-09-11T07:44:14.743 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.743 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.743 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.744 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.460204+0000", 2024-09-11T07:44:14.744 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8762999999999999e-05 2024-09-11T07:44:14.744 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.744 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.744 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:44:14.744 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:14.745 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461822+0000", 2024-09-11T07:44:14.745 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0016181539999999999 2024-09-11T07:44:14.745 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.745 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.745 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.462813+0000", 2024-09-11T07:44:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00099143499999999993 2024-09-11T07:44:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:44:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.462901+0000", 2024-09-11T07:44:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.7409000000000003e-05 2024-09-11T07:44:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:44:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.463029+0000", 2024-09-11T07:44:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012804 2024-09-11T07:44:14.746 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.747 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:44:14.747 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.747 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.747 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.747 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235366 9.8 9:1702ff49:::benchmark_data_smithi022_133943_object108775:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e323)", 2024-09-11T07:44:14.747 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:02.747880+0000", 2024-09-11T07:44:14.747 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 191.947534193, 2024-09-11T07:44:14.747 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.96185844799999998, 2024-09-11T07:44:14.747 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:44:14.747 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:44:14.747 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:44:14.747 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:44:14.747 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:44:14.748 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235366 2024-09-11T07:44:14.748 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.748 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:44:14.748 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.748 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:44:14.748 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.747880+0000", 2024-09-11T07:44:14.748 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.748 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.748 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.748 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:44:14.748 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.747880+0000", 2024-09-11T07:44:14.748 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.748 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.748 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.749 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:44:14.749 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.747877+0000", 2024-09-11T07:44:14.749 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:44:14.749 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.749 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.749 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:44:14.749 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.747887+0000", 2024-09-11T07:44:14.749 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.9960000000000005e-06 2024-09-11T07:44:14.749 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.749 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.749 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:44:14.749 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.747913+0000", 2024-09-11T07:44:14.749 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.6363000000000002e-05 2024-09-11T07:44:14.750 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.750 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.750 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:44:14.750 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.747920+0000", 2024-09-11T07:44:14.750 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.2259999999999999e-06 2024-09-11T07:44:14.750 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.750 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.751 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.751 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.747967+0000", 2024-09-11T07:44:14.752 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6034e-05 2024-09-11T07:44:14.752 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.752 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.752 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:44:14.752 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.747974+0000", 2024-09-11T07:44:14.752 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.9370000000000004e-06 2024-09-11T07:44:14.752 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.752 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.752 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.752 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.706758+0000", 2024-09-11T07:44:14.752 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.95878366699999995 2024-09-11T07:44:14.752 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.752 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.753 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:44:14.753 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.706864+0000", 2024-09-11T07:44:14.753 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000105722 2024-09-11T07:44:14.753 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.753 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.753 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,7", 2024-09-11T07:44:14.753 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.706971+0000", 2024-09-11T07:44:14.753 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000107589 2024-09-11T07:44:14.753 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.753 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.753 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.753 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.709589+0000", 2024-09-11T07:44:14.753 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0026178730000000002 2024-09-11T07:44:14.753 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.754 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.754 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:44:14.754 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.709599+0000", 2024-09-11T07:44:14.754 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0149999999999999e-05 2024-09-11T07:44:14.754 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.754 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.754 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.754 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.709652+0000", 2024-09-11T07:44:14.754 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2627000000000003e-05 2024-09-11T07:44:14.754 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.754 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.754 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:44:14.754 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.709694+0000", 2024-09-11T07:44:14.754 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1956999999999998e-05 2024-09-11T07:44:14.755 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.755 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.755 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:44:14.755 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.709738+0000", 2024-09-11T07:44:14.755 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4122000000000002e-05 2024-09-11T07:44:14.755 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.755 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:44:14.755 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.755 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.755 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.755 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235378 9.8 9:17e72ff8:::benchmark_data_smithi022_133943_object108787:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e323)", 2024-09-11T07:44:14.755 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:02.759690+0000", 2024-09-11T07:44:14.755 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 191.93572417799999, 2024-09-11T07:44:14.756 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.96079267800000001, 2024-09-11T07:44:14.756 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:44:14.756 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:44:14.756 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:44:14.756 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:44:14.756 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:44:14.756 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235378 2024-09-11T07:44:14.756 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.756 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:44:14.756 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.756 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:44:14.756 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.759690+0000", 2024-09-11T07:44:14.756 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.757 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.757 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.757 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:44:14.757 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.759690+0000", 2024-09-11T07:44:14.757 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.757 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.757 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.757 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:44:14.757 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.759688+0000", 2024-09-11T07:44:14.757 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:44:14.757 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.758 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.758 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:44:14.758 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.759694+0000", 2024-09-11T07:44:14.758 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7400000000000001e-06 2024-09-11T07:44:14.758 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.758 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.758 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:44:14.758 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.759703+0000", 2024-09-11T07:44:14.758 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.5830000000000007e-06 2024-09-11T07:44:14.758 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.758 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.758 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:44:14.758 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.759708+0000", 2024-09-11T07:44:14.758 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.4199999999999998e-06 2024-09-11T07:44:14.759 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.759 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.759 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.759 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.759773+0000", 2024-09-11T07:44:14.759 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4772999999999998e-05 2024-09-11T07:44:14.759 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.759 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.759 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:44:14.759 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.759780+0000", 2024-09-11T07:44:14.759 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.2729999999999997e-06 2024-09-11T07:44:14.759 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.759 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.759 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.760 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.709785+0000", 2024-09-11T07:44:14.760 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.95000451399999997 2024-09-11T07:44:14.760 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.760 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.760 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:44:14.760 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.709890+0000", 2024-09-11T07:44:14.760 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010527300000000001 2024-09-11T07:44:14.760 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.760 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.760 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,7", 2024-09-11T07:44:14.760 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.709962+0000", 2024-09-11T07:44:14.760 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1935999999999997e-05 2024-09-11T07:44:14.760 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.761 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.761 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.761 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.714856+0000", 2024-09-11T07:44:14.761 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0048941230000000002 2024-09-11T07:44:14.761 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.761 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.761 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:44:14.761 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.714864+0000", 2024-09-11T07:44:14.761 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.7330000000000003e-06 2024-09-11T07:44:14.761 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.761 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.761 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.762 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.720399+0000", 2024-09-11T07:44:14.762 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.005534743 2024-09-11T07:44:14.762 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.762 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.762 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:44:14.762 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.720445+0000", 2024-09-11T07:44:14.762 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5998000000000002e-05 2024-09-11T07:44:14.762 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.762 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.762 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:44:14.762 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.720482+0000", 2024-09-11T07:44:14.762 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7861999999999998e-05 2024-09-11T07:44:14.762 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.762 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:44:14.763 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.763 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.763 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.763 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235388 9.8 9:129d98d7:::benchmark_data_smithi022_133943_object108797:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e323)", 2024-09-11T07:44:14.763 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:02.768761+0000", 2024-09-11T07:44:14.763 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 191.92665296300001, 2024-09-11T07:44:14.763 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.95185301700000002, 2024-09-11T07:44:14.763 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:44:14.763 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:44:14.763 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:44:14.763 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:44:14.763 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:44:14.763 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235388 2024-09-11T07:44:14.763 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.764 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:44:14.764 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.764 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:44:14.764 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768761+0000", 2024-09-11T07:44:14.764 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.764 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.764 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.764 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:44:14.764 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768761+0000", 2024-09-11T07:44:14.764 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.764 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.764 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.764 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:44:14.765 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768758+0000", 2024-09-11T07:44:14.765 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:44:14.765 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.765 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.765 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:44:14.765 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768766+0000", 2024-09-11T07:44:14.765 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5089999999999996e-06 2024-09-11T07:44:14.765 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.765 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.765 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:44:14.765 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768782+0000", 2024-09-11T07:44:14.765 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5985e-05 2024-09-11T07:44:14.765 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.766 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.766 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:44:14.766 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768788+0000", 2024-09-11T07:44:14.766 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2559999999999999e-06 2024-09-11T07:44:14.766 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.766 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.766 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.766 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768835+0000", 2024-09-11T07:44:14.766 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6455999999999997e-05 2024-09-11T07:44:14.766 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.766 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.766 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:44:14.766 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768843+0000", 2024-09-11T07:44:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.4209999999999995e-06 2024-09-11T07:44:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.712304+0000", 2024-09-11T07:44:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.94346046699999997 2024-09-11T07:44:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:44:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.712424+0000", 2024-09-11T07:44:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000119997 2024-09-11T07:44:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.767 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,7", 2024-09-11T07:44:14.768 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.712526+0000", 2024-09-11T07:44:14.768 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000102649 2024-09-11T07:44:14.768 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.768 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.768 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:44:14.768 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.720360+0000", 2024-09-11T07:44:14.768 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0078334990000000007 2024-09-11T07:44:14.768 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.768 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.768 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.768 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.720519+0000", 2024-09-11T07:44:14.768 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015941600000000001 2024-09-11T07:44:14.768 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.769 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.769 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.769 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.720547+0000", 2024-09-11T07:44:14.769 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8078000000000001e-05 2024-09-11T07:44:14.769 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.769 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.769 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:44:14.769 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.720582+0000", 2024-09-11T07:44:14.769 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4860000000000002e-05 2024-09-11T07:44:14.769 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.769 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.769 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:44:14.769 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.720614+0000", 2024-09-11T07:44:14.770 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.1931000000000003e-05 2024-09-11T07:44:14.770 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.770 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:44:14.770 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.770 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.770 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.770 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235390 9.8 9:17241c1b:::benchmark_data_smithi022_133943_object108799:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e323)", 2024-09-11T07:44:14.770 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:02.770070+0000", 2024-09-11T07:44:14.770 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 191.92534351800001, 2024-09-11T07:44:14.770 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.95338818199999997, 2024-09-11T07:44:14.770 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:44:14.770 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:44:14.770 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:44:14.770 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:44:14.771 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:44:14.771 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235390 2024-09-11T07:44:14.771 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.771 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:44:14.771 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.771 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:44:14.771 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.770070+0000", 2024-09-11T07:44:14.771 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.771 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.771 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.771 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:44:14.771 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.770070+0000", 2024-09-11T07:44:14.771 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.771 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.772 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.772 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:44:14.772 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.770069+0000", 2024-09-11T07:44:14.772 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:44:14.772 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.772 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.772 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:44:14.772 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.770076+0000", 2024-09-11T07:44:14.772 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0509999999999999e-06 2024-09-11T07:44:14.772 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.772 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.772 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:44:14.773 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.770099+0000", 2024-09-11T07:44:14.773 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2938e-05 2024-09-11T07:44:14.773 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.773 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.773 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:44:14.773 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.770105+0000", 2024-09-11T07:44:14.773 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9510000000000003e-06 2024-09-11T07:44:14.773 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.773 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.773 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.773 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.770145+0000", 2024-09-11T07:44:14.773 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0818999999999997e-05 2024-09-11T07:44:14.773 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.773 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.774 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:44:14.774 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.770152+0000", 2024-09-11T07:44:14.774 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9750000000000001e-06 2024-09-11T07:44:14.774 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.774 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.774 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.774 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.714887+0000", 2024-09-11T07:44:14.774 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.94473459699999995 2024-09-11T07:44:14.774 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.774 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.774 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:44:14.774 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.714990+0000", 2024-09-11T07:44:14.774 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000103251 2024-09-11T07:44:14.774 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.774 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.775 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,7", 2024-09-11T07:44:14.775 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.715109+0000", 2024-09-11T07:44:14.775 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000119255 2024-09-11T07:44:14.775 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.775 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.775 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.775 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.720644+0000", 2024-09-11T07:44:14.775 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0055347640000000002 2024-09-11T07:44:14.775 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.775 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.775 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.775 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.720669+0000", 2024-09-11T07:44:14.775 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4468999999999999e-05 2024-09-11T07:44:14.776 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.776 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.776 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:44:14.776 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.723232+0000", 2024-09-11T07:44:14.776 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0025635329999999998 2024-09-11T07:44:14.776 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.776 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.776 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:44:14.776 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.723417+0000", 2024-09-11T07:44:14.776 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00018494900000000001 2024-09-11T07:44:14.776 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.776 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.776 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:44:14.776 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.723459+0000", 2024-09-11T07:44:14.777 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1399000000000001e-05 2024-09-11T07:44:14.777 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.777 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:44:14.777 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.777 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.777 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.777 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:27072 10.8 10:158b6afe:::benchmark_data_smithi022_163866_object27071:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e354)", 2024-09-11T07:44:14.777 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:42:26.456153+0000", 2024-09-11T07:44:14.777 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 108.23926050599999, 2024-09-11T07:44:14.777 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.32160622500000002, 2024-09-11T07:44:14.777 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:44:14.777 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:44:14.777 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:44:14.778 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:44:14.778 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:44:14.778 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 27072 2024-09-11T07:44:14.778 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.778 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:44:14.778 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.778 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:44:14.778 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.456153+0000", 2024-09-11T07:44:14.778 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.778 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.778 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.778 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:44:14.778 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.456153+0000", 2024-09-11T07:44:14.778 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.779 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.779 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.779 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:44:14.779 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.456152+0000", 2024-09-11T07:44:14.779 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:44:14.779 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.779 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.779 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:44:14.779 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.456178+0000", 2024-09-11T07:44:14.779 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5446e-05 2024-09-11T07:44:14.779 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.779 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.779 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:44:14.780 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.456186+0000", 2024-09-11T07:44:14.780 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.1030000000000002e-06 2024-09-11T07:44:14.780 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.780 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.780 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:44:14.780 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.456190+0000", 2024-09-11T07:44:14.780 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4610000000000001e-06 2024-09-11T07:44:14.780 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.780 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.780 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.780 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.481472+0000", 2024-09-11T07:44:14.780 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.025281527000000002 2024-09-11T07:44:14.780 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:44:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.481602+0000", 2024-09-11T07:44:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000129939 2024-09-11T07:44:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,7", 2024-09-11T07:44:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.481758+0000", 2024-09-11T07:44:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015594900000000001 2024-09-11T07:44:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:44:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.484199+0000", 2024-09-11T07:44:14.781 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0024416540000000001 2024-09-11T07:44:14.782 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.782 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.782 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.782 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.486941+0000", 2024-09-11T07:44:14.782 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0027419670000000001 2024-09-11T07:44:14.782 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.782 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.782 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.782 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.777669+0000", 2024-09-11T07:44:14.782 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.29072745799999999 2024-09-11T07:44:14.782 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.782 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.782 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:44:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.777713+0000", 2024-09-11T07:44:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3708000000000002e-05 2024-09-11T07:44:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:44:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.777760+0000", 2024-09-11T07:44:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6995e-05 2024-09-11T07:44:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:44:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:27073 10.8 10:1fb5fa90:::benchmark_data_smithi022_163866_object27072:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e354)", 2024-09-11T07:44:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:42:26.456202+0000", 2024-09-11T07:44:14.783 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 108.23921186600001, 2024-09-11T07:44:14.784 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.32168576799999998, 2024-09-11T07:44:14.784 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:44:14.784 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:44:14.784 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:44:14.784 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:44:14.784 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:44:14.784 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 27073 2024-09-11T07:44:14.784 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.784 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:44:14.784 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.784 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:44:14.784 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.456202+0000", 2024-09-11T07:44:14.784 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.785 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.785 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.785 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:44:14.785 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.456202+0000", 2024-09-11T07:44:14.785 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.785 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.785 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.785 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:44:14.785 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.456201+0000", 2024-09-11T07:44:14.786 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:44:14.786 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.786 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.786 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:44:14.786 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.456223+0000", 2024-09-11T07:44:14.786 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1197999999999999e-05 2024-09-11T07:44:14.786 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.786 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.786 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:44:14.786 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.456229+0000", 2024-09-11T07:44:14.786 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0460000000000002e-06 2024-09-11T07:44:14.786 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.786 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:44:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.456232+0000", 2024-09-11T07:44:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8689999999999997e-06 2024-09-11T07:44:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.482939+0000", 2024-09-11T07:44:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.026706912999999999 2024-09-11T07:44:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:44:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.483064+0000", 2024-09-11T07:44:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012448 2024-09-11T07:44:14.787 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.788 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.788 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,7", 2024-09-11T07:44:14.788 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.483203+0000", 2024-09-11T07:44:14.788 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000138889 2024-09-11T07:44:14.788 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.788 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.788 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.788 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.486973+0000", 2024-09-11T07:44:14.788 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.003770261 2024-09-11T07:44:14.788 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.788 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.788 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:44:14.788 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.777770+0000", 2024-09-11T07:44:14.789 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.29079714099999998 2024-09-11T07:44:14.789 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.789 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.789 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.789 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.777814+0000", 2024-09-11T07:44:14.789 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3585999999999999e-05 2024-09-11T07:44:14.789 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.789 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.789 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:44:14.790 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.777854+0000", 2024-09-11T07:44:14.790 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0132999999999997e-05 2024-09-11T07:44:14.790 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.791 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.791 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:44:14.791 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.777888+0000", 2024-09-11T07:44:14.791 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3908000000000001e-05 2024-09-11T07:44:14.791 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.791 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:44:14.791 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.791 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.791 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.791 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(client.25706.0:27077 10.c e354/341 10:3438f62c:::benchmark_data_smithi022_163866_object27076:head v 354'1763, mlcod=354'1761)", 2024-09-11T07:44:14.791 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:42:26.457596+0000", 2024-09-11T07:44:14.791 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 108.237817537, 2024-09-11T07:44:14.792 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.31859894900000002, 2024-09-11T07:44:14.792 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:44:14.792 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:44:14.792 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:44:14.792 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.792 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:44:14.792 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.457596+0000", 2024-09-11T07:44:14.792 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.792 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.792 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.792 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:44:14.792 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.457596+0000", 2024-09-11T07:44:14.792 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.793 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.793 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.793 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:44:14.793 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.457595+0000", 2024-09-11T07:44:14.793 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:44:14.793 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.793 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.793 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:44:14.793 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.457657+0000", 2024-09-11T07:44:14.793 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.224e-05 2024-09-11T07:44:14.793 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.793 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.793 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:44:14.794 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.480584+0000", 2024-09-11T07:44:14.794 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.022926683 2024-09-11T07:44:14.794 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.794 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.794 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:44:14.794 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.480606+0000", 2024-09-11T07:44:14.794 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2164999999999999e-05 2024-09-11T07:44:14.794 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.794 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.794 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.794 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.484022+0000", 2024-09-11T07:44:14.794 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.003415375 2024-09-11T07:44:14.794 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.794 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.795 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:44:14.795 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.484040+0000", 2024-09-11T07:44:14.795 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8162e-05 2024-09-11T07:44:14.795 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.795 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.795 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:44:14.795 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.776162+0000", 2024-09-11T07:44:14.795 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.292122359 2024-09-11T07:44:14.795 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.795 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.795 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:44:14.795 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.776195+0000", 2024-09-11T07:44:14.795 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3129000000000001e-05 2024-09-11T07:44:14.796 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.796 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:44:14.796 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.796 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.796 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.796 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:27080 10.a 10:5720d1fa:::benchmark_data_smithi022_163866_object27079:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e354)", 2024-09-11T07:44:14.796 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:42:26.481290+0000", 2024-09-11T07:44:14.796 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 108.214123481, 2024-09-11T07:44:14.796 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.29499214400000001, 2024-09-11T07:44:14.796 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:44:14.796 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:44:14.796 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:44:14.796 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:44:14.796 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:44:14.797 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 27080 2024-09-11T07:44:14.797 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.797 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:44:14.797 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.797 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:44:14.797 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.481290+0000", 2024-09-11T07:44:14.797 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.797 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.797 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.797 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:44:14.797 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.481290+0000", 2024-09-11T07:44:14.797 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.797 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.798 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.798 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:44:14.798 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.481288+0000", 2024-09-11T07:44:14.798 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:44:14.798 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.798 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.798 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:44:14.798 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.481316+0000", 2024-09-11T07:44:14.798 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7931000000000001e-05 2024-09-11T07:44:14.798 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.798 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.798 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:44:14.798 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.481328+0000", 2024-09-11T07:44:14.798 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2347000000000001e-05 2024-09-11T07:44:14.799 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.799 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.799 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:44:14.799 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.481334+0000", 2024-09-11T07:44:14.799 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2160000000000003e-06 2024-09-11T07:44:14.799 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.799 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.799 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.799 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.481371+0000", 2024-09-11T07:44:14.799 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6210000000000001e-05 2024-09-11T07:44:14.799 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.799 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.799 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:44:14.800 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.481586+0000", 2024-09-11T07:44:14.800 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00021575300000000001 2024-09-11T07:44:14.800 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.800 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.800 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:44:14.800 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.481742+0000", 2024-09-11T07:44:14.800 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015520399999999999 2024-09-11T07:44:14.800 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.800 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.800 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.800 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.486089+0000", 2024-09-11T07:44:14.800 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0043476490000000003 2024-09-11T07:44:14.800 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.801 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.801 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.801 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.486120+0000", 2024-09-11T07:44:14.801 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0812999999999998e-05 2024-09-11T07:44:14.801 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.801 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.801 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:44:14.801 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.776175+0000", 2024-09-11T07:44:14.801 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.29005491 2024-09-11T07:44:14.801 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.801 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.801 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:44:14.801 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.776235+0000", 2024-09-11T07:44:14.802 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0015000000000001e-05 2024-09-11T07:44:14.802 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.802 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.802 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:44:14.802 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.776283+0000", 2024-09-11T07:44:14.802 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7605000000000002e-05 2024-09-11T07:44:14.802 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.802 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:44:14.802 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.802 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.802 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.802 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:27083 10.8 10:17b4a9da:::benchmark_data_smithi022_163866_object27082:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e354)", 2024-09-11T07:44:14.802 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:42:26.482826+0000", 2024-09-11T07:44:14.802 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 108.212588354, 2024-09-11T07:44:14.803 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.29536549499999998, 2024-09-11T07:44:14.803 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:44:14.803 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:44:14.803 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:44:14.803 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:44:14.803 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:44:14.803 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 27083 2024-09-11T07:44:14.803 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.803 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:44:14.803 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.803 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:44:14.803 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.482826+0000", 2024-09-11T07:44:14.803 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:44:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.482826+0000", 2024-09-11T07:44:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:44:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.482824+0000", 2024-09-11T07:44:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:44:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:44:14.804 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.482886+0000", 2024-09-11T07:44:14.805 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1938000000000002e-05 2024-09-11T07:44:14.805 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.805 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.805 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:44:14.805 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.482898+0000", 2024-09-11T07:44:14.805 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2483999999999999e-05 2024-09-11T07:44:14.805 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.805 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.805 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:44:14.805 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.482904+0000", 2024-09-11T07:44:14.805 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1700000000000002e-06 2024-09-11T07:44:14.805 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.805 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.806 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.806 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.484327+0000", 2024-09-11T07:44:14.806 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0014230060000000001 2024-09-11T07:44:14.806 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.806 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.806 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:44:14.806 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.484432+0000", 2024-09-11T07:44:14.806 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000104498 2024-09-11T07:44:14.806 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.806 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,7", 2024-09-11T07:44:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.484555+0000", 2024-09-11T07:44:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012350800000000001 2024-09-11T07:44:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.487003+0000", 2024-09-11T07:44:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0024477169999999999 2024-09-11T07:44:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:44:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.777775+0000", 2024-09-11T07:44:14.807 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.29077188199999998 2024-09-11T07:44:14.808 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.808 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.808 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.808 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.778118+0000", 2024-09-11T07:44:14.808 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00034284400000000003 2024-09-11T07:44:14.808 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.808 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.808 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:44:14.808 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.778157+0000", 2024-09-11T07:44:14.808 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.93e-05 2024-09-11T07:44:14.808 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.808 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.808 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:44:14.809 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.778191+0000", 2024-09-11T07:44:14.809 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3887999999999998e-05 2024-09-11T07:44:14.809 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.809 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:44:14.809 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.809 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.809 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.809 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:27084 10.6 10:68e763ec:::benchmark_data_smithi022_163866_object27083:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e354)", 2024-09-11T07:44:14.809 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:42:26.482910+0000", 2024-09-11T07:44:14.809 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 108.212503728, 2024-09-11T07:44:14.809 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.29590309300000001, 2024-09-11T07:44:14.809 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:44:14.809 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:44:14.809 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:44:14.810 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:44:14.810 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:44:14.810 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 27084 2024-09-11T07:44:14.810 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.810 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:44:14.810 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.810 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:44:14.810 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.482910+0000", 2024-09-11T07:44:14.810 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.810 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.810 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.810 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:44:14.810 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.482910+0000", 2024-09-11T07:44:14.811 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.811 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.811 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.811 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:44:14.811 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.482909+0000", 2024-09-11T07:44:14.811 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:44:14.811 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.811 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.811 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:44:14.811 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.482932+0000", 2024-09-11T07:44:14.811 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2578999999999998e-05 2024-09-11T07:44:14.811 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.812 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.812 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:44:14.812 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.482940+0000", 2024-09-11T07:44:14.812 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3450000000000006e-06 2024-09-11T07:44:14.812 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.812 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.812 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:44:14.812 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.482945+0000", 2024-09-11T07:44:14.812 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7400000000000004e-06 2024-09-11T07:44:14.812 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.812 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.812 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.812 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.483537+0000", 2024-09-11T07:44:14.813 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00059203900000000004 2024-09-11T07:44:14.813 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.813 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.813 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:44:14.813 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.483648+0000", 2024-09-11T07:44:14.813 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000111035 2024-09-11T07:44:14.813 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.813 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.813 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 4,5", 2024-09-11T07:44:14.813 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.485344+0000", 2024-09-11T07:44:14.813 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0016954470000000001 2024-09-11T07:44:14.813 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.813 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.813 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.814 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.778024+0000", 2024-09-11T07:44:14.814 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.29268009099999998 2024-09-11T07:44:14.814 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.814 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.814 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.814 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.778056+0000", 2024-09-11T07:44:14.814 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.1912000000000002e-05 2024-09-11T07:44:14.814 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.814 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.814 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:44:14.814 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.778653+0000", 2024-09-11T07:44:14.814 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00059757899999999999 2024-09-11T07:44:14.814 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:44:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.778774+0000", 2024-09-11T07:44:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012095300000000001 2024-09-11T07:44:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:44:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.778813+0000", 2024-09-11T07:44:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9156999999999998e-05 2024-09-11T07:44:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:44:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.815 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.816 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:27086 10.4 10:2b9b42ed:::benchmark_data_smithi022_163866_object27085:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e354)", 2024-09-11T07:44:14.816 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:42:26.484821+0000", 2024-09-11T07:44:14.816 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 108.210592812, 2024-09-11T07:44:14.816 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.29146080299999999, 2024-09-11T07:44:14.816 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:44:14.816 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:44:14.816 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:44:14.816 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:44:14.816 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:44:14.816 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 27086 2024-09-11T07:44:14.816 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.816 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:44:14.816 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.816 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:44:14.817 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.484821+0000", 2024-09-11T07:44:14.817 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.817 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.817 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.817 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:44:14.817 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.484821+0000", 2024-09-11T07:44:14.817 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.817 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.817 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.817 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:44:14.817 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.484819+0000", 2024-09-11T07:44:14.817 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:44:14.817 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:44:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.484846+0000", 2024-09-11T07:44:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7228000000000001e-05 2024-09-11T07:44:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:44:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.484858+0000", 2024-09-11T07:44:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1678999999999999e-05 2024-09-11T07:44:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:44:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.484864+0000", 2024-09-11T07:44:14.818 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0000000000000002e-06 2024-09-11T07:44:14.819 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.819 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.819 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.819 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.486456+0000", 2024-09-11T07:44:14.819 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0015918530000000001 2024-09-11T07:44:14.819 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.819 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.819 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:44:14.819 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.486582+0000", 2024-09-11T07:44:14.819 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012578700000000001 2024-09-11T07:44:14.819 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.819 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.819 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,4", 2024-09-11T07:44:14.819 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.486714+0000", 2024-09-11T07:44:14.820 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000132278 2024-09-11T07:44:14.820 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.820 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.820 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.820 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.776149+0000", 2024-09-11T07:44:14.820 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.28943537000000003 2024-09-11T07:44:14.820 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.820 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.820 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.820 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.776193+0000", 2024-09-11T07:44:14.820 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4159999999999997e-05 2024-09-11T07:44:14.820 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.820 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.821 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:44:14.821 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.776203+0000", 2024-09-11T07:44:14.821 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.1770000000000004e-06 2024-09-11T07:44:14.821 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.821 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.821 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:44:14.821 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.776243+0000", 2024-09-11T07:44:14.821 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0088e-05 2024-09-11T07:44:14.821 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.821 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.821 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:44:14.821 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.776282+0000", 2024-09-11T07:44:14.822 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9122999999999998e-05 2024-09-11T07:44:14.822 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.822 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:44:14.822 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.822 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.822 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.822 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:27087 10.6 10:6d4b00e5:::benchmark_data_smithi022_163866_object27086:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e354)", 2024-09-11T07:44:14.822 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:42:26.485538+0000", 2024-09-11T07:44:14.822 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 108.20987621899999, 2024-09-11T07:44:14.822 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.29345342499999999, 2024-09-11T07:44:14.822 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:44:14.822 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:44:14.822 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:44:14.822 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:44:14.823 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:44:14.823 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 27087 2024-09-11T07:44:14.823 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.823 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:44:14.823 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.823 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:44:14.823 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.485538+0000", 2024-09-11T07:44:14.823 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.823 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.823 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.823 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:44:14.823 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.485538+0000", 2024-09-11T07:44:14.823 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.823 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.823 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.824 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:44:14.824 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.485536+0000", 2024-09-11T07:44:14.824 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:44:14.824 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.824 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.824 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:44:14.824 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.485562+0000", 2024-09-11T07:44:14.824 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5829999999999998e-05 2024-09-11T07:44:14.824 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.824 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.824 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:44:14.824 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.485573+0000", 2024-09-11T07:44:14.824 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0995999999999999e-05 2024-09-11T07:44:14.824 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.825 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.825 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:44:14.825 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.485579+0000", 2024-09-11T07:44:14.825 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3749999999999999e-06 2024-09-11T07:44:14.825 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.825 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.825 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.825 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.486817+0000", 2024-09-11T07:44:14.825 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001238086 2024-09-11T07:44:14.825 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.825 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.825 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:44:14.825 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.486923+0000", 2024-09-11T07:44:14.825 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010522 2024-09-11T07:44:14.826 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.826 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.826 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 4,5", 2024-09-11T07:44:14.826 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.487042+0000", 2024-09-11T07:44:14.826 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000118998 2024-09-11T07:44:14.826 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.826 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.826 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.826 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.778084+0000", 2024-09-11T07:44:14.826 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.29104269199999999 2024-09-11T07:44:14.826 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.826 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.826 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.826 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.778622+0000", 2024-09-11T07:44:14.826 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00053805300000000003 2024-09-11T07:44:14.827 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.827 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.827 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:44:14.827 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.778875+0000", 2024-09-11T07:44:14.827 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00025230199999999998 2024-09-11T07:44:14.827 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.827 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.827 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:44:14.827 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.778958+0000", 2024-09-11T07:44:14.827 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3014000000000002e-05 2024-09-11T07:44:14.827 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.827 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.827 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:44:14.828 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.778991+0000", 2024-09-11T07:44:14.828 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3521e-05 2024-09-11T07:44:14.828 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.828 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:44:14.828 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.828 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.828 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.828 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:27091 10.8 10:1ad8b1f5:::benchmark_data_smithi022_163866_object27090:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e354)", 2024-09-11T07:44:14.828 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:42:26.487436+0000", 2024-09-11T07:44:14.828 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 108.20797803799999, 2024-09-11T07:44:14.828 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.29237684000000003, 2024-09-11T07:44:14.828 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:44:14.828 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:44:14.828 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:44:14.828 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:44:14.829 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:44:14.829 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 27091 2024-09-11T07:44:14.829 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.829 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:44:14.829 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.829 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:44:14.829 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.487436+0000", 2024-09-11T07:44:14.829 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.829 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.829 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.829 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:44:14.829 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.487436+0000", 2024-09-11T07:44:14.829 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.829 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.829 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.830 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:44:14.830 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.487434+0000", 2024-09-11T07:44:14.830 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:44:14.830 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.830 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.830 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:44:14.830 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.487559+0000", 2024-09-11T07:44:14.830 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012506299999999999 2024-09-11T07:44:14.830 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.830 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.830 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:44:14.830 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.487570+0000", 2024-09-11T07:44:14.830 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1083999999999999e-05 2024-09-11T07:44:14.830 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.831 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.831 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:44:14.831 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.487594+0000", 2024-09-11T07:44:14.831 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.404e-05 2024-09-11T07:44:14.831 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.831 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.831 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.831 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.487796+0000", 2024-09-11T07:44:14.831 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000202179 2024-09-11T07:44:14.831 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.831 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.831 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:44:14.831 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.487920+0000", 2024-09-11T07:44:14.831 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012406799999999999 2024-09-11T07:44:14.832 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.832 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.832 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,7", 2024-09-11T07:44:14.832 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.488056+0000", 2024-09-11T07:44:14.832 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013563799999999999 2024-09-11T07:44:14.832 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.832 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.832 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:44:14.832 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.777779+0000", 2024-09-11T07:44:14.833 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.28972340099999999 2024-09-11T07:44:14.833 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.833 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.833 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.834 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.778404+0000", 2024-09-11T07:44:14.834 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00062436299999999996 2024-09-11T07:44:14.834 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.834 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.834 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.834 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.779733+0000", 2024-09-11T07:44:14.834 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00132975 2024-09-11T07:44:14.834 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.834 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.834 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:44:14.834 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.779774+0000", 2024-09-11T07:44:14.834 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0419e-05 2024-09-11T07:44:14.834 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.834 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:44:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.779813+0000", 2024-09-11T07:44:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8804000000000003e-05 2024-09-11T07:44:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:44:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:27092 10.6 10:6788775a:::benchmark_data_smithi022_163866_object27091:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e354)", 2024-09-11T07:44:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:42:26.487705+0000", 2024-09-11T07:44:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 108.20770939099999, 2024-09-11T07:44:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.29179041, 2024-09-11T07:44:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:44:14.835 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:44:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:44:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:44:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:44:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 27092 2024-09-11T07:44:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:44:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:44:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.487705+0000", 2024-09-11T07:44:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:44:14.836 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.487705+0000", 2024-09-11T07:44:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:44:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:44:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.487703+0000", 2024-09-11T07:44:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:44:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:44:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.487741+0000", 2024-09-11T07:44:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7437000000000001e-05 2024-09-11T07:44:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.837 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:44:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.487750+0000", 2024-09-11T07:44:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.3009999999999995e-06 2024-09-11T07:44:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:44:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.487756+0000", 2024-09-11T07:44:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9039999999999997e-06 2024-09-11T07:44:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:44:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.489496+0000", 2024-09-11T07:44:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001739991 2024-09-11T07:44:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.838 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:44:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.489618+0000", 2024-09-11T07:44:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000122099 2024-09-11T07:44:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 4,5", 2024-09-11T07:44:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.776220+0000", 2024-09-11T07:44:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.28660193099999998 2024-09-11T07:44:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.778648+0000", 2024-09-11T07:44:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0024280759999999999 2024-09-11T07:44:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:44:14.839 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.779003+0000", 2024-09-11T07:44:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00035487099999999998 2024-09-11T07:44:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:44:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.779398+0000", 2024-09-11T07:44:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00039547999999999997 2024-09-11T07:44:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:44:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.779463+0000", 2024-09-11T07:44:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4559000000000003e-05 2024-09-11T07:44:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:44:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:44:14.840 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:44:14.841 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:42:26.779495+0000", 2024-09-11T07:44:14.841 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2004000000000001e-05 2024-09-11T07:44:14.841 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.841 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:44:14.841 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.841 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:44:14.841 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:44:14.841 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:44:14.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:14.842 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:14.842 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:14.890 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:14 smithi022 ceph-mon[37330]: 5.15 scrub starts 2024-09-11T07:44:14.890 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:14 smithi022 ceph-mon[37330]: 5.15 scrub ok 2024-09-11T07:44:14.890 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:14 smithi022 ceph-mon[37330]: pgmap v821: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.9 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-11T07:44:14.890 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:14 smithi022 ceph-mon[29462]: 5.15 scrub starts 2024-09-11T07:44:14.890 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:14 smithi022 ceph-mon[29462]: 5.15 scrub ok 2024-09-11T07:44:14.890 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:14 smithi022 ceph-mon[29462]: pgmap v821: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.9 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-11T07:44:14.943 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:14.964 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:14.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:14.965 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:15.066 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:15.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:15.111 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:15.111 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:15.212 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:15.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:15.238 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:15.238 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:15.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:14 smithi104 ceph-mon[33507]: 5.15 scrub starts 2024-09-11T07:44:15.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:14 smithi104 ceph-mon[33507]: 5.15 scrub ok 2024-09-11T07:44:15.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:14 smithi104 ceph-mon[33507]: pgmap v821: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.9 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-11T07:44:15.338 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:15.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:15.370 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:15.370 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:15.471 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:15.507 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:15.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:15.508 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:15.609 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:15.634 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:15.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:15.635 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:15.736 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:15.778 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:15.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:15.779 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:15.879 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:15.914 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:15.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:15.915 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:16.016 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:16.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:16.052 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:16.052 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:16.153 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:16.191 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:16.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:16.192 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:16.293 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:16.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:16.328 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:16.329 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:16.430 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:16.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:16.470 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:16.471 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:16.572 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:16.609 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:16.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:16.610 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:16.711 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:16.738 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:16.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:16.739 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:16.840 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:16.872 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:16.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:16.873 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:16.973 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:16.995 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:16 smithi022 ceph-mon[29462]: 2.11 scrub starts 2024-09-11T07:44:16.995 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:16 smithi022 ceph-mon[29462]: 2.11 scrub ok 2024-09-11T07:44:16.995 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:16 smithi022 ceph-mon[29462]: 5.13 scrub starts 2024-09-11T07:44:16.995 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:16 smithi022 ceph-mon[29462]: 5.13 scrub ok 2024-09-11T07:44:16.995 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:16 smithi022 ceph-mon[29462]: pgmap v822: 148 pgs: 148 active+clean; 5.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-09-11T07:44:16.996 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:16 smithi022 ceph-mon[37330]: 2.11 scrub starts 2024-09-11T07:44:16.996 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:16 smithi022 ceph-mon[37330]: 2.11 scrub ok 2024-09-11T07:44:16.996 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:16 smithi022 ceph-mon[37330]: 5.13 scrub starts 2024-09-11T07:44:16.996 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:16 smithi022 ceph-mon[37330]: 5.13 scrub ok 2024-09-11T07:44:16.996 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:16 smithi022 ceph-mon[37330]: pgmap v822: 148 pgs: 148 active+clean; 5.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-09-11T07:44:17.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:17.013 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:17.014 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:16 smithi104 ceph-mon[33507]: 2.11 scrub starts 2024-09-11T07:44:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:16 smithi104 ceph-mon[33507]: 2.11 scrub ok 2024-09-11T07:44:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:16 smithi104 ceph-mon[33507]: 5.13 scrub starts 2024-09-11T07:44:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:16 smithi104 ceph-mon[33507]: 5.13 scrub ok 2024-09-11T07:44:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:16 smithi104 ceph-mon[33507]: pgmap v822: 148 pgs: 148 active+clean; 5.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-09-11T07:44:17.114 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:17.150 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:17.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:17.151 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:17.252 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:17.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:17.284 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:17.285 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:17.386 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:17.424 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:17.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:17.425 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:17.526 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:17.560 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:17.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:17.561 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:17.663 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-11T07:44:17.663 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:17.703 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:17.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:17.704 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:17.805 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:17.848 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:17.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:17.849 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:17 smithi022 ceph-mon[29462]: 4.10 scrub starts 2024-09-11T07:44:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:17 smithi022 ceph-mon[29462]: 4.10 scrub ok 2024-09-11T07:44:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:17 smithi022 ceph-mon[29462]: 6.3 scrub starts 2024-09-11T07:44:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:17 smithi022 ceph-mon[29462]: 6.3 scrub ok 2024-09-11T07:44:17.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:17 smithi022 ceph-mon[37330]: 4.10 scrub starts 2024-09-11T07:44:17.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:17 smithi022 ceph-mon[37330]: 4.10 scrub ok 2024-09-11T07:44:17.921 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:17 smithi022 ceph-mon[37330]: 6.3 scrub starts 2024-09-11T07:44:17.921 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:17 smithi022 ceph-mon[37330]: 6.3 scrub ok 2024-09-11T07:44:17.950 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:17.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:17.985 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:17.986 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:18.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:17 smithi104 ceph-mon[33507]: 4.10 scrub starts 2024-09-11T07:44:18.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:17 smithi104 ceph-mon[33507]: 4.10 scrub ok 2024-09-11T07:44:18.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:17 smithi104 ceph-mon[33507]: 6.3 scrub starts 2024-09-11T07:44:18.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:17 smithi104 ceph-mon[33507]: 6.3 scrub ok 2024-09-11T07:44:18.087 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:18.117 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:18.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:18.118 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:18.219 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:18.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:18.258 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:18.258 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:18.359 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:18.398 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:18.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:18.399 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:18.500 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:18.541 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:18.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:18.542 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:18.643 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:18.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:18.678 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:18.678 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:18.743 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is unset 2024-09-11T07:44:18.780 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:18.809 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:18.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:18.811 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:18.912 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:18.916 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:44:18.916 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:44:18.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:18.943 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:18.943 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:18.949 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:44:18.950 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:44:18.985 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:44:18.985 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:44:18.985 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:18 smithi022 ceph-mon[29462]: 5.1c scrub starts 2024-09-11T07:44:18.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:18 smithi022 ceph-mon[29462]: 5.1c scrub ok 2024-09-11T07:44:18.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:18 smithi022 ceph-mon[29462]: 6.13 scrub starts 2024-09-11T07:44:18.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:18 smithi022 ceph-mon[29462]: 6.13 scrub ok 2024-09-11T07:44:18.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:18 smithi022 ceph-mon[29462]: pgmap v823: 148 pgs: 148 active+clean; 5.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 601 B/s rd, 0 B/s wr, 715 op/s 2024-09-11T07:44:18.986 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:18 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1397741937' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:44:18.986 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:18 smithi022 ceph-mon[37330]: 5.1c scrub starts 2024-09-11T07:44:18.986 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:18 smithi022 ceph-mon[37330]: 5.1c scrub ok 2024-09-11T07:44:18.986 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:18 smithi022 ceph-mon[37330]: 6.13 scrub starts 2024-09-11T07:44:18.986 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:18 smithi022 ceph-mon[37330]: 6.13 scrub ok 2024-09-11T07:44:18.986 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:18 smithi022 ceph-mon[37330]: pgmap v823: 148 pgs: 148 active+clean; 5.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 601 B/s rd, 0 B/s wr, 715 op/s 2024-09-11T07:44:18.986 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:18 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1397741937' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:44:19.019 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:44:19.019 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:44:19.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:18 smithi104 ceph-mon[33507]: 5.1c scrub starts 2024-09-11T07:44:19.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:18 smithi104 ceph-mon[33507]: 5.1c scrub ok 2024-09-11T07:44:19.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:18 smithi104 ceph-mon[33507]: 6.13 scrub starts 2024-09-11T07:44:19.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:18 smithi104 ceph-mon[33507]: 6.13 scrub ok 2024-09-11T07:44:19.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:18 smithi104 ceph-mon[33507]: pgmap v823: 148 pgs: 148 active+clean; 5.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 601 B/s rd, 0 B/s wr, 715 op/s 2024-09-11T07:44:19.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:18 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1397741937' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:44:19.044 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:19.053 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:44:19.053 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:44:19.080 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:19.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:19.081 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:19.084 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:44:19.084 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:44:19.118 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:44:19.119 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:44:19.149 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:44:19.150 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:44:19.182 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:19.221 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:19.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:19.222 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:19.323 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:19.356 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:19.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:19.357 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:19.458 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:19.497 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:19.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:19.498 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:19.599 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:19.632 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:19.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:19.634 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:19.735 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:19.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:19.769 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:19.769 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:19.870 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:19.904 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:19.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:19.905 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:20.006 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:20.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:19 smithi104 ceph-mon[33507]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:44:20.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:19 smithi104 ceph-mon[33507]: Cluster is now healthy 2024-09-11T07:44:20.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:19 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1397741937' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:44:20.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:19 smithi104 ceph-mon[33507]: osdmap e378: 8 total, 8 up, 8 in 2024-09-11T07:44:20.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:19 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2909173795' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:44:20.038 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:20.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:20.039 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:20.140 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:20.160 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:19 smithi022 ceph-mon[29462]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:44:20.160 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:19 smithi022 ceph-mon[29462]: Cluster is now healthy 2024-09-11T07:44:20.160 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:19 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1397741937' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:44:20.160 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:19 smithi022 ceph-mon[29462]: osdmap e378: 8 total, 8 up, 8 in 2024-09-11T07:44:20.160 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:19 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2909173795' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:44:20.161 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:19 smithi022 ceph-mon[37330]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:44:20.161 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:19 smithi022 ceph-mon[37330]: Cluster is now healthy 2024-09-11T07:44:20.161 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:19 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1397741937' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:44:20.161 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:19 smithi022 ceph-mon[37330]: osdmap e378: 8 total, 8 up, 8 in 2024-09-11T07:44:20.161 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:19 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2909173795' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:44:20.171 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:20.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:20.172 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:20.273 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:20.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:20.317 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:20.317 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:20.416 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:44:20 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:44:20] "GET /metrics HTTP/1.1" 200 224931 "" "Prometheus/2.43.0" 2024-09-11T07:44:20.418 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:20.460 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:20.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:20.461 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:20.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:44:20 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:44:20.563 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:20.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:20.604 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:20.604 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:20.705 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:20.745 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:20.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:20.746 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:20.847 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:20.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:20.885 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:20.885 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:20.900 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:20 smithi022 ceph-mon[29462]: 2.1e deep-scrub starts 2024-09-11T07:44:20.900 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:20 smithi022 ceph-mon[29462]: 2.1e deep-scrub ok 2024-09-11T07:44:20.901 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:20 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:44:20.901 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:20 smithi022 ceph-mon[29462]: pgmap v825: 148 pgs: 148 active+clean; 5.5 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T07:44:20.901 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:20 smithi022 ceph-mon[29462]: 5.10 scrub starts 2024-09-11T07:44:20.901 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:20 smithi022 ceph-mon[29462]: 5.10 scrub ok 2024-09-11T07:44:20.901 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:20 smithi022 ceph-mon[37330]: 2.1e deep-scrub starts 2024-09-11T07:44:20.901 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:20 smithi022 ceph-mon[37330]: 2.1e deep-scrub ok 2024-09-11T07:44:20.901 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:20 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:44:20.901 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:20 smithi022 ceph-mon[37330]: pgmap v825: 148 pgs: 148 active+clean; 5.5 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T07:44:20.901 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:20 smithi022 ceph-mon[37330]: 5.10 scrub starts 2024-09-11T07:44:20.901 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:20 smithi022 ceph-mon[37330]: 5.10 scrub ok 2024-09-11T07:44:20.986 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:21.018 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:21.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:21.019 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:21.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:20 smithi104 ceph-mon[33507]: 2.1e deep-scrub starts 2024-09-11T07:44:21.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:20 smithi104 ceph-mon[33507]: 2.1e deep-scrub ok 2024-09-11T07:44:21.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:20 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:44:21.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:20 smithi104 ceph-mon[33507]: pgmap v825: 148 pgs: 148 active+clean; 5.5 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-11T07:44:21.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:20 smithi104 ceph-mon[33507]: 5.10 scrub starts 2024-09-11T07:44:21.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:20 smithi104 ceph-mon[33507]: 5.10 scrub ok 2024-09-11T07:44:21.121 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:21.153 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:21.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:21.154 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:21.254 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:21.291 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:21.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:21.293 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:21.393 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:21.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:21.435 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:21.435 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:21.536 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:21.572 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:21.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:21.573 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:21.674 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:21.708 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:21.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:21.709 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:21.810 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:21.850 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:44:21.852 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:21.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:21.853 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:21.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:21 smithi022 ceph-mon[29462]: 5.5 scrub starts 2024-09-11T07:44:21.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:21 smithi022 ceph-mon[29462]: 5.5 scrub ok 2024-09-11T07:44:21.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:21 smithi022 ceph-mon[37330]: 5.5 scrub starts 2024-09-11T07:44:21.921 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:21 smithi022 ceph-mon[37330]: 5.5 scrub ok 2024-09-11T07:44:21.954 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:21.991 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:21.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:21.992 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:22.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:21 smithi104 ceph-mon[33507]: 5.5 scrub starts 2024-09-11T07:44:22.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:21 smithi104 ceph-mon[33507]: 5.5 scrub ok 2024-09-11T07:44:22.094 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:22.136 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:22.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:22.137 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:22.238 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:22.268 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:22.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:22.269 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:22.370 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:22.414 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:22.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:22.415 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:22.516 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:22.556 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:22.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:22.557 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:22.648 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:44:22.658 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:22.695 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:22.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:22.696 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:22.765 INFO:teuthology.orchestra.run.smithi022.stderr:set osd.1 primary-affinity to 0 (802) 2024-09-11T07:44:22.797 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:22.841 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:22.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:22.842 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:22.945 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:22.967 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:22.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:22.968 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:23.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:22 smithi104 ceph-mon[33507]: 10.2 scrub starts 2024-09-11T07:44:23.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:22 smithi104 ceph-mon[33507]: 2.1f deep-scrub starts 2024-09-11T07:44:23.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:22 smithi104 ceph-mon[33507]: 2.1f deep-scrub ok 2024-09-11T07:44:23.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:22 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3925667420' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.0}]: dispatch 2024-09-11T07:44:23.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:22 smithi104 ceph-mon[33507]: pgmap v826: 148 pgs: 148 active+clean; 5.5 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.20k op/s 2024-09-11T07:44:23.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:22 smithi104 ceph-mon[33507]: 5.7 scrub starts 2024-09-11T07:44:23.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:22 smithi104 ceph-mon[33507]: 5.7 scrub ok 2024-09-11T07:44:23.060 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:22 smithi022 ceph-mon[29462]: 10.2 scrub starts 2024-09-11T07:44:23.061 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:22 smithi022 ceph-mon[29462]: 2.1f deep-scrub starts 2024-09-11T07:44:23.061 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:22 smithi022 ceph-mon[29462]: 2.1f deep-scrub ok 2024-09-11T07:44:23.061 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:22 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3925667420' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.0}]: dispatch 2024-09-11T07:44:23.061 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:22 smithi022 ceph-mon[29462]: pgmap v826: 148 pgs: 148 active+clean; 5.5 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.20k op/s 2024-09-11T07:44:23.061 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:22 smithi022 ceph-mon[29462]: 5.7 scrub starts 2024-09-11T07:44:23.061 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:22 smithi022 ceph-mon[29462]: 5.7 scrub ok 2024-09-11T07:44:23.061 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:22 smithi022 ceph-mon[37330]: 10.2 scrub starts 2024-09-11T07:44:23.061 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:22 smithi022 ceph-mon[37330]: 2.1f deep-scrub starts 2024-09-11T07:44:23.062 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:22 smithi022 ceph-mon[37330]: 2.1f deep-scrub ok 2024-09-11T07:44:23.062 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:22 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3925667420' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.0}]: dispatch 2024-09-11T07:44:23.062 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:22 smithi022 ceph-mon[37330]: pgmap v826: 148 pgs: 148 active+clean; 5.5 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.20k op/s 2024-09-11T07:44:23.062 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:22 smithi022 ceph-mon[37330]: 5.7 scrub starts 2024-09-11T07:44:23.062 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:22 smithi022 ceph-mon[37330]: 5.7 scrub ok 2024-09-11T07:44:23.069 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:23.091 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:23.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:23.092 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:23.193 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:23.216 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:23.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:23.217 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:23.318 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:23.342 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:23.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:23.343 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:23.444 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:23.470 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:23.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:23.471 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:23.572 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:23.594 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:23.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:23.595 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:23.637 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T07:44:23.696 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:23.722 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:23.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:23.723 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:23.823 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:23.855 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:23.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:23.856 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:23.957 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:23.988 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:23.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:23.989 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:24.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:23 smithi104 ceph-mon[33507]: 10.0 scrub starts 2024-09-11T07:44:24.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:23 smithi104 ceph-mon[33507]: 2.18 scrub starts 2024-09-11T07:44:24.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:23 smithi104 ceph-mon[33507]: 2.18 scrub ok 2024-09-11T07:44:24.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:23 smithi104 ceph-mon[33507]: 1.0 scrub starts 2024-09-11T07:44:24.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:23 smithi104 ceph-mon[33507]: 1.0 scrub ok 2024-09-11T07:44:24.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:23 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3925667420' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.0}]': finished 2024-09-11T07:44:24.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:23 smithi104 ceph-mon[33507]: osdmap e379: 8 total, 8 up, 8 in 2024-09-11T07:44:24.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:23 smithi104 ceph-mon[33507]: 10.2 scrub ok 2024-09-11T07:44:24.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:44:24.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:44:24.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:23 smithi104 ceph-mon[33507]: 5.b scrub starts 2024-09-11T07:44:24.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:23 smithi104 ceph-mon[33507]: 5.b scrub ok 2024-09-11T07:44:24.090 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:24.113 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:24.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:24.115 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:24.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:23 smithi022 ceph-mon[29462]: 10.0 scrub starts 2024-09-11T07:44:24.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:23 smithi022 ceph-mon[29462]: 2.18 scrub starts 2024-09-11T07:44:24.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:23 smithi022 ceph-mon[29462]: 2.18 scrub ok 2024-09-11T07:44:24.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:23 smithi022 ceph-mon[29462]: 1.0 scrub starts 2024-09-11T07:44:24.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:23 smithi022 ceph-mon[29462]: 1.0 scrub ok 2024-09-11T07:44:24.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:23 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3925667420' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.0}]': finished 2024-09-11T07:44:24.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:23 smithi022 ceph-mon[29462]: osdmap e379: 8 total, 8 up, 8 in 2024-09-11T07:44:24.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:23 smithi022 ceph-mon[29462]: 10.2 scrub ok 2024-09-11T07:44:24.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:44:24.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:44:24.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:23 smithi022 ceph-mon[29462]: 5.b scrub starts 2024-09-11T07:44:24.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:23 smithi022 ceph-mon[29462]: 5.b scrub ok 2024-09-11T07:44:24.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:23 smithi022 ceph-mon[37330]: 10.0 scrub starts 2024-09-11T07:44:24.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:23 smithi022 ceph-mon[37330]: 2.18 scrub starts 2024-09-11T07:44:24.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:23 smithi022 ceph-mon[37330]: 2.18 scrub ok 2024-09-11T07:44:24.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:23 smithi022 ceph-mon[37330]: 1.0 scrub starts 2024-09-11T07:44:24.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:23 smithi022 ceph-mon[37330]: 1.0 scrub ok 2024-09-11T07:44:24.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:23 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3925667420' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.0}]': finished 2024-09-11T07:44:24.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:23 smithi022 ceph-mon[37330]: osdmap e379: 8 total, 8 up, 8 in 2024-09-11T07:44:24.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:23 smithi022 ceph-mon[37330]: 10.2 scrub ok 2024-09-11T07:44:24.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:44:24.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:44:24.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:23 smithi022 ceph-mon[37330]: 5.b scrub starts 2024-09-11T07:44:24.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:23 smithi022 ceph-mon[37330]: 5.b scrub ok 2024-09-11T07:44:24.216 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:24.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:24.257 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:24.257 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:24.358 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:24.380 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:44:24.380 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:44:24.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:24.403 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:24.404 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:24.505 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:24.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:44:24 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:44:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:44:24.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:24.541 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:24.541 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:24.642 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:24.678 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:24.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:24.679 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:24.779 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:24.811 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:24.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:24.812 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:24.912 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:24.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:24.951 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:24.952 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:24.978 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:24 smithi104 ceph-mon[33507]: osdmap e380: 8 total, 8 up, 8 in 2024-09-11T07:44:24.978 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:24 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:44:24.978 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:24 smithi104 ceph-mon[33507]: pgmap v829: 148 pgs: 148 active+clean; 5.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 0 B/s wr, 1.76k op/s 2024-09-11T07:44:24.978 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:24 smithi104 ceph-mon[33507]: 2.5 deep-scrub starts 2024-09-11T07:44:24.978 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:24 smithi104 ceph-mon[33507]: 10.9 scrub starts 2024-09-11T07:44:24.978 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:24 smithi104 ceph-mon[33507]: 2.5 deep-scrub ok 2024-09-11T07:44:25.052 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:25.078 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:25.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:25.079 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:25.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:24 smithi022 ceph-mon[29462]: osdmap e380: 8 total, 8 up, 8 in 2024-09-11T07:44:25.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:24 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:44:25.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:24 smithi022 ceph-mon[29462]: pgmap v829: 148 pgs: 148 active+clean; 5.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 0 B/s wr, 1.76k op/s 2024-09-11T07:44:25.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:24 smithi022 ceph-mon[29462]: 2.5 deep-scrub starts 2024-09-11T07:44:25.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:24 smithi022 ceph-mon[29462]: 10.9 scrub starts 2024-09-11T07:44:25.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:24 smithi022 ceph-mon[29462]: 2.5 deep-scrub ok 2024-09-11T07:44:25.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:24 smithi022 ceph-mon[37330]: osdmap e380: 8 total, 8 up, 8 in 2024-09-11T07:44:25.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:24 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:44:25.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:24 smithi022 ceph-mon[37330]: pgmap v829: 148 pgs: 148 active+clean; 5.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 0 B/s wr, 1.76k op/s 2024-09-11T07:44:25.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:24 smithi022 ceph-mon[37330]: 2.5 deep-scrub starts 2024-09-11T07:44:25.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:24 smithi022 ceph-mon[37330]: 10.9 scrub starts 2024-09-11T07:44:25.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:24 smithi022 ceph-mon[37330]: 2.5 deep-scrub ok 2024-09-11T07:44:25.180 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:25.211 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:25.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:25.212 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:25.313 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:25.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:25.354 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:25.355 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:25.456 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:25.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:25.497 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:25.497 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:25.598 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:25.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:25.633 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:25.633 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:25.734 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:25.774 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:25.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:25.775 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:25.876 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:25.896 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:25.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:25.897 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:25.998 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:26.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:25 smithi104 ceph-mon[33507]: 5.1b scrub starts 2024-09-11T07:44:26.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:25 smithi104 ceph-mon[33507]: 5.1b scrub ok 2024-09-11T07:44:26.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:25 smithi104 ceph-mon[33507]: 6.5 deep-scrub starts 2024-09-11T07:44:26.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:25 smithi104 ceph-mon[33507]: 6.5 deep-scrub ok 2024-09-11T07:44:26.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:25 smithi104 ceph-mon[33507]: 2.e scrub starts 2024-09-11T07:44:26.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:25 smithi104 ceph-mon[33507]: 2.e scrub ok 2024-09-11T07:44:26.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:26.031 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:26.032 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:26.132 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:26.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:25 smithi022 ceph-mon[29462]: 5.1b scrub starts 2024-09-11T07:44:26.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:25 smithi022 ceph-mon[29462]: 5.1b scrub ok 2024-09-11T07:44:26.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:25 smithi022 ceph-mon[29462]: 6.5 deep-scrub starts 2024-09-11T07:44:26.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:25 smithi022 ceph-mon[29462]: 6.5 deep-scrub ok 2024-09-11T07:44:26.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:25 smithi022 ceph-mon[29462]: 2.e scrub starts 2024-09-11T07:44:26.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:25 smithi022 ceph-mon[29462]: 2.e scrub ok 2024-09-11T07:44:26.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:25 smithi022 ceph-mon[37330]: 5.1b scrub starts 2024-09-11T07:44:26.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:25 smithi022 ceph-mon[37330]: 5.1b scrub ok 2024-09-11T07:44:26.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:25 smithi022 ceph-mon[37330]: 6.5 deep-scrub starts 2024-09-11T07:44:26.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:25 smithi022 ceph-mon[37330]: 6.5 deep-scrub ok 2024-09-11T07:44:26.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:25 smithi022 ceph-mon[37330]: 2.e scrub starts 2024-09-11T07:44:26.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:25 smithi022 ceph-mon[37330]: 2.e scrub ok 2024-09-11T07:44:26.173 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:26.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:26.174 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:26.275 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:26.305 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:26.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:26.306 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:26.407 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:26.430 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:26.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:26.431 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:26.531 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:26.562 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:26.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:26.563 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:26.664 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:26.698 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:26.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:26.700 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:26.800 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:26.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:26.838 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:26.838 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:26.939 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:26.977 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:26.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:26.978 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:27.079 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:27.111 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:27.111 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:44:27.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:27.112 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:27.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:26 smithi022 ceph-mon[29462]: 10.9 scrub ok 2024-09-11T07:44:27.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:26 smithi022 ceph-mon[29462]: pgmap v830: 148 pgs: 148 active+clean; 5.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 0 B/s wr, 1.47k op/s 2024-09-11T07:44:27.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:26 smithi022 ceph-mon[29462]: 6.15 scrub starts 2024-09-11T07:44:27.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:26 smithi022 ceph-mon[29462]: 6.15 scrub ok 2024-09-11T07:44:27.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:26 smithi022 ceph-mon[29462]: 10.7 scrub starts 2024-09-11T07:44:27.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:26 smithi022 ceph-mon[37330]: 10.9 scrub ok 2024-09-11T07:44:27.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:26 smithi022 ceph-mon[37330]: pgmap v830: 148 pgs: 148 active+clean; 5.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 0 B/s wr, 1.47k op/s 2024-09-11T07:44:27.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:26 smithi022 ceph-mon[37330]: 6.15 scrub starts 2024-09-11T07:44:27.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:26 smithi022 ceph-mon[37330]: 6.15 scrub ok 2024-09-11T07:44:27.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:26 smithi022 ceph-mon[37330]: 10.7 scrub starts 2024-09-11T07:44:27.213 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:27.243 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:27.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:27.244 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:27.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:26 smithi104 ceph-mon[33507]: 10.9 scrub ok 2024-09-11T07:44:27.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:26 smithi104 ceph-mon[33507]: pgmap v830: 148 pgs: 148 active+clean; 5.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 0 B/s wr, 1.47k op/s 2024-09-11T07:44:27.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:26 smithi104 ceph-mon[33507]: 6.15 scrub starts 2024-09-11T07:44:27.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:26 smithi104 ceph-mon[33507]: 6.15 scrub ok 2024-09-11T07:44:27.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:26 smithi104 ceph-mon[33507]: 10.7 scrub starts 2024-09-11T07:44:27.345 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:27.375 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:27.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:27.376 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:27.477 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:27.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:27.513 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:27.513 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:27.614 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:27.651 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:27.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:27.652 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:27.752 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:27.788 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:27.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:27.789 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:27.890 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:27.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:27 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:44:27.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:27 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:44:27.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:27 smithi022 ceph-mon[29462]: 2.9 deep-scrub starts 2024-09-11T07:44:27.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:27 smithi022 ceph-mon[29462]: 2.9 deep-scrub ok 2024-09-11T07:44:27.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:27 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:44:27.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:27 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:44:27.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:27 smithi022 ceph-mon[37330]: 2.9 deep-scrub starts 2024-09-11T07:44:27.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:27 smithi022 ceph-mon[37330]: 2.9 deep-scrub ok 2024-09-11T07:44:27.927 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:27.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:27.928 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:28.028 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:28.066 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:28.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:28.067 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:28.167 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:28.221 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:28.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:28.222 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:28.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:27 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:44:28.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:27 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:44:28.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:27 smithi104 ceph-mon[33507]: 2.9 deep-scrub starts 2024-09-11T07:44:28.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:27 smithi104 ceph-mon[33507]: 2.9 deep-scrub ok 2024-09-11T07:44:28.323 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:28.363 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:28.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:28.364 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:28.465 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:28.501 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:28.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:28.502 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:28.603 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:28.629 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-11T07:44:28.630 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:28.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:28.631 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:28.656 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:44:28.732 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:28.793 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:28.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:28.795 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:28.897 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:28.903 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:28 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:44:28.903 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:28 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:44:28.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:28 smithi022 ceph-mon[29462]: 6.b deep-scrub starts 2024-09-11T07:44:28.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:28 smithi022 ceph-mon[29462]: 6.b deep-scrub ok 2024-09-11T07:44:28.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:28 smithi022 ceph-mon[29462]: osdmap e381: 8 total, 8 up, 8 in 2024-09-11T07:44:28.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:28 smithi022 ceph-mon[29462]: 4.6 scrub starts 2024-09-11T07:44:28.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:28 smithi022 ceph-mon[29462]: 4.6 scrub ok 2024-09-11T07:44:28.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:28 smithi022 ceph-mon[29462]: pgmap v832: 148 pgs: 148 active+clean; 5.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 0 B/s wr, 836 op/s 2024-09-11T07:44:28.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:28 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:44:28.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:28 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:44:28.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:28 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:44:28.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:28 smithi022 ceph-mon[29462]: 5.8 scrub starts 2024-09-11T07:44:28.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:28 smithi022 ceph-mon[29462]: 5.8 scrub ok 2024-09-11T07:44:28.905 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:28 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:44:28.905 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:28 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:44:28.905 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:28 smithi022 ceph-mon[37330]: 6.b deep-scrub starts 2024-09-11T07:44:28.905 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:28 smithi022 ceph-mon[37330]: 6.b deep-scrub ok 2024-09-11T07:44:28.905 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:28 smithi022 ceph-mon[37330]: osdmap e381: 8 total, 8 up, 8 in 2024-09-11T07:44:28.905 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:28 smithi022 ceph-mon[37330]: 4.6 scrub starts 2024-09-11T07:44:28.905 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:28 smithi022 ceph-mon[37330]: 4.6 scrub ok 2024-09-11T07:44:28.905 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:28 smithi022 ceph-mon[37330]: pgmap v832: 148 pgs: 148 active+clean; 5.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 0 B/s wr, 836 op/s 2024-09-11T07:44:28.905 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:28 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:44:28.905 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:28 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:44:28.905 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:28 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:44:28.905 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:28 smithi022 ceph-mon[37330]: 5.8 scrub starts 2024-09-11T07:44:28.905 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:28 smithi022 ceph-mon[37330]: 5.8 scrub ok 2024-09-11T07:44:28.929 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:28.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:28.930 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:29.031 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:29.069 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:29.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:29.070 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:29.171 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:29.208 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:29.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:29.209 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:29.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:28 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:44:29.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:28 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:44:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:28 smithi104 ceph-mon[33507]: 6.b deep-scrub starts 2024-09-11T07:44:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:28 smithi104 ceph-mon[33507]: 6.b deep-scrub ok 2024-09-11T07:44:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:28 smithi104 ceph-mon[33507]: osdmap e381: 8 total, 8 up, 8 in 2024-09-11T07:44:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:28 smithi104 ceph-mon[33507]: 4.6 scrub starts 2024-09-11T07:44:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:28 smithi104 ceph-mon[33507]: 4.6 scrub ok 2024-09-11T07:44:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:28 smithi104 ceph-mon[33507]: pgmap v832: 148 pgs: 148 active+clean; 5.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 0 B/s wr, 836 op/s 2024-09-11T07:44:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:28 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:44:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:28 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:44:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:28 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:44:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:28 smithi104 ceph-mon[33507]: 5.8 scrub starts 2024-09-11T07:44:29.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:28 smithi104 ceph-mon[33507]: 5.8 scrub ok 2024-09-11T07:44:29.310 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:29.352 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:29.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:29.353 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:29.454 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:29.484 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:29.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:29.485 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:29.586 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:29.623 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:29.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:29.625 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:29.726 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:29.766 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:29.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:29.767 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:29.868 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:29.899 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:29.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:29.900 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:30.001 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:30.034 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:30.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:30.036 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:30.137 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:30.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:29 smithi022 ceph-mon[29462]: 6.1d scrub starts 2024-09-11T07:44:30.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:29 smithi022 ceph-mon[29462]: 6.1d scrub ok 2024-09-11T07:44:30.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:29 smithi022 ceph-mon[29462]: 10.7 scrub ok 2024-09-11T07:44:30.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:29 smithi022 ceph-mon[29462]: 5.17 scrub starts 2024-09-11T07:44:30.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:29 smithi022 ceph-mon[29462]: 5.17 scrub ok 2024-09-11T07:44:30.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:29 smithi022 ceph-mon[37330]: 6.1d scrub starts 2024-09-11T07:44:30.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:29 smithi022 ceph-mon[37330]: 6.1d scrub ok 2024-09-11T07:44:30.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:29 smithi022 ceph-mon[37330]: 10.7 scrub ok 2024-09-11T07:44:30.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:29 smithi022 ceph-mon[37330]: 5.17 scrub starts 2024-09-11T07:44:30.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:29 smithi022 ceph-mon[37330]: 5.17 scrub ok 2024-09-11T07:44:30.175 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:30.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:30.176 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:30.277 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:29 smithi104 ceph-mon[33507]: 6.1d scrub starts 2024-09-11T07:44:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:29 smithi104 ceph-mon[33507]: 6.1d scrub ok 2024-09-11T07:44:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:29 smithi104 ceph-mon[33507]: 10.7 scrub ok 2024-09-11T07:44:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:29 smithi104 ceph-mon[33507]: 5.17 scrub starts 2024-09-11T07:44:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:29 smithi104 ceph-mon[33507]: 5.17 scrub ok 2024-09-11T07:44:30.280 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:44:30 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:44:30.312 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:30.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:30.313 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:30.414 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:30.454 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:30.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:30.455 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:30.556 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:30.593 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:30.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:30.595 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:30.637 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:44:30 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:44:30] "GET /metrics HTTP/1.1" 200 224952 "" "Prometheus/2.43.0" 2024-09-11T07:44:30.696 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:30.727 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:30.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:30.728 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:30.829 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:30.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:30.863 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:30.864 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:30.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:30 smithi022 ceph-mon[29462]: 4.14 deep-scrub starts 2024-09-11T07:44:30.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:30 smithi022 ceph-mon[29462]: 4.14 deep-scrub ok 2024-09-11T07:44:30.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:30 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:44:30.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:30 smithi022 ceph-mon[29462]: 4.16 deep-scrub starts 2024-09-11T07:44:30.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:30 smithi022 ceph-mon[29462]: 4.16 deep-scrub ok 2024-09-11T07:44:30.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:30 smithi022 ceph-mon[29462]: pgmap v833: 148 pgs: 148 active+clean; 4.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 0 B/s wr, 1.41k op/s 2024-09-11T07:44:30.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:30 smithi022 ceph-mon[37330]: 4.14 deep-scrub starts 2024-09-11T07:44:30.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:30 smithi022 ceph-mon[37330]: 4.14 deep-scrub ok 2024-09-11T07:44:30.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:30 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:44:30.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:30 smithi022 ceph-mon[37330]: 4.16 deep-scrub starts 2024-09-11T07:44:30.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:30 smithi022 ceph-mon[37330]: 4.16 deep-scrub ok 2024-09-11T07:44:30.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:30 smithi022 ceph-mon[37330]: pgmap v833: 148 pgs: 148 active+clean; 4.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 0 B/s wr, 1.41k op/s 2024-09-11T07:44:30.964 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:31.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:31.002 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:31.002 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:31.103 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:31.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:31.146 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:31.146 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:31.247 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:31.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:30 smithi104 ceph-mon[33507]: 4.14 deep-scrub starts 2024-09-11T07:44:31.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:30 smithi104 ceph-mon[33507]: 4.14 deep-scrub ok 2024-09-11T07:44:31.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:30 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:44:31.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:30 smithi104 ceph-mon[33507]: 4.16 deep-scrub starts 2024-09-11T07:44:31.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:30 smithi104 ceph-mon[33507]: 4.16 deep-scrub ok 2024-09-11T07:44:31.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:30 smithi104 ceph-mon[33507]: pgmap v833: 148 pgs: 148 active+clean; 4.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 0 B/s wr, 1.41k op/s 2024-09-11T07:44:31.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:31.287 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:31.287 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:31.388 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:31.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:31.427 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:31.427 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:31.528 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:31.559 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:31.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:31.560 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:31.661 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:31.696 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:31.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:31.697 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:31.798 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:31.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:31.832 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:31.833 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:31.933 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:31.974 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:31.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:31.975 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:32.042 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:31 smithi022 ceph-mon[29462]: 3.2 scrub starts 2024-09-11T07:44:32.043 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:31 smithi022 ceph-mon[29462]: 3.2 scrub ok 2024-09-11T07:44:32.043 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:31 smithi022 ceph-mon[37330]: 3.2 scrub starts 2024-09-11T07:44:32.043 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:31 smithi022 ceph-mon[37330]: 3.2 scrub ok 2024-09-11T07:44:32.077 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:32.123 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:32.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:32.124 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:32.177 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:44:32.177 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:44:32.177 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:44:32.177 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:44:32.206 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:44:32.206 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:44:32.225 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:32.245 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:44:32.245 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:44:32.251 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:32.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:32.252 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:32.276 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:44:32.279 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:44:32.279 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:44:32.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:31 smithi104 ceph-mon[33507]: 3.2 scrub starts 2024-09-11T07:44:32.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:31 smithi104 ceph-mon[33507]: 3.2 scrub ok 2024-09-11T07:44:32.313 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:44:32.313 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:44:32.347 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:44:32.347 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:44:32.353 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:32.384 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:44:32.384 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:44:32.387 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:32.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:32.388 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:32.489 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:32.531 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:32.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:32.532 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:32.633 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:32.665 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:32.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:32.666 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:32.767 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:32.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:32.805 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:32.805 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:32.866 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is set 2024-09-11T07:44:32.907 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:32.950 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:32.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:32.951 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:33.052 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:33.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:33.093 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:33.093 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:33.127 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:44:33.127 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:44:33.128 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2024-09-11T07:44:33.128 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:44:33.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:32 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3590124752' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:44:33.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:32 smithi022 ceph-mon[29462]: pgmap v834: 148 pgs: 148 active+clean; 4.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 0 B/s wr, 1.25k op/s 2024-09-11T07:44:33.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:32 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4243843584' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:44:33.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:32 smithi022 ceph-mon[29462]: from='client.25835 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:44:33.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:32 smithi022 ceph-mon[29462]: 5.1a scrub starts 2024-09-11T07:44:33.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:32 smithi022 ceph-mon[29462]: 5.1a scrub ok 2024-09-11T07:44:33.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:32 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3590124752' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:44:33.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:32 smithi022 ceph-mon[37330]: pgmap v834: 148 pgs: 148 active+clean; 4.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 0 B/s wr, 1.25k op/s 2024-09-11T07:44:33.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:32 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4243843584' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:44:33.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:32 smithi022 ceph-mon[37330]: from='client.25835 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:44:33.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:32 smithi022 ceph-mon[37330]: 5.1a scrub starts 2024-09-11T07:44:33.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:32 smithi022 ceph-mon[37330]: 5.1a scrub ok 2024-09-11T07:44:33.195 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:33.250 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:33.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:33.251 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:33.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:32 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3590124752' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:44:33.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:32 smithi104 ceph-mon[33507]: pgmap v834: 148 pgs: 148 active+clean; 4.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 0 B/s wr, 1.25k op/s 2024-09-11T07:44:33.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:32 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4243843584' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:44:33.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:32 smithi104 ceph-mon[33507]: from='client.25835 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:44:33.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:32 smithi104 ceph-mon[33507]: 5.1a scrub starts 2024-09-11T07:44:33.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:32 smithi104 ceph-mon[33507]: 5.1a scrub ok 2024-09-11T07:44:33.352 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:33.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:33.412 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:33.412 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:33.513 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:33.552 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:33.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:33.553 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:33.654 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:33.691 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:33.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:33.692 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:33.793 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:33.829 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:33.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:33.830 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:33.931 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:33.972 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:33.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:33.973 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:34.012 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:33 smithi022 ceph-mon[29462]: 4.13 scrub starts 2024-09-11T07:44:34.012 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:33 smithi022 ceph-mon[29462]: 4.13 scrub ok 2024-09-11T07:44:34.012 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:33 smithi022 ceph-mon[29462]: 4.a scrub starts 2024-09-11T07:44:34.012 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:33 smithi022 ceph-mon[29462]: 4.a scrub ok 2024-09-11T07:44:34.012 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:33 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:44:34.013 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:33 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:44:34.013 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:33 smithi022 ceph-mon[29462]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:44:34.013 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:33 smithi022 ceph-mon[29462]: from='client.25835 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:44:34.013 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:33 smithi022 ceph-mon[29462]: osdmap e382: 8 total, 8 up, 8 in 2024-09-11T07:44:34.013 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:33 smithi022 ceph-mon[37330]: 4.13 scrub starts 2024-09-11T07:44:34.013 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:33 smithi022 ceph-mon[37330]: 4.13 scrub ok 2024-09-11T07:44:34.013 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:33 smithi022 ceph-mon[37330]: 4.a scrub starts 2024-09-11T07:44:34.013 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:33 smithi022 ceph-mon[37330]: 4.a scrub ok 2024-09-11T07:44:34.013 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:33 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:44:34.013 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:33 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:44:34.014 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:33 smithi022 ceph-mon[37330]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:44:34.014 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:33 smithi022 ceph-mon[37330]: from='client.25835 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:44:34.014 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:33 smithi022 ceph-mon[37330]: osdmap e382: 8 total, 8 up, 8 in 2024-09-11T07:44:34.074 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:34.110 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:34.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:34.111 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:34.212 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:34.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:34.249 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:34.249 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:34.279 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:44:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:44:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:44:34.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:33 smithi104 ceph-mon[33507]: 4.13 scrub starts 2024-09-11T07:44:34.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:33 smithi104 ceph-mon[33507]: 4.13 scrub ok 2024-09-11T07:44:34.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:33 smithi104 ceph-mon[33507]: 4.a scrub starts 2024-09-11T07:44:34.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:33 smithi104 ceph-mon[33507]: 4.a scrub ok 2024-09-11T07:44:34.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:33 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:44:34.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:33 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:44:34.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:33 smithi104 ceph-mon[33507]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:44:34.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:33 smithi104 ceph-mon[33507]: from='client.25835 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:44:34.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:33 smithi104 ceph-mon[33507]: osdmap e382: 8 total, 8 up, 8 in 2024-09-11T07:44:34.350 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:34.390 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:34.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:34.391 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:34.493 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:34.533 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:34.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:34.535 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:34.635 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:34.670 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:34.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:34.671 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:34.771 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:34.807 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:34.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:34.809 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:34.909 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:34.932 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:44:34.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:34.980 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:34.980 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:35.077 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:34 smithi022 ceph-mon[29462]: pgmap v836: 148 pgs: 148 active+clean; 4.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.44k op/s 2024-09-11T07:44:35.077 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:34 smithi022 ceph-mon[37330]: pgmap v836: 148 pgs: 148 active+clean; 4.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.44k op/s 2024-09-11T07:44:35.081 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:35.109 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:35.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:35.113 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:35.213 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:35.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:35.253 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:35.253 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:35.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:34 smithi104 ceph-mon[33507]: pgmap v836: 148 pgs: 148 active+clean; 4.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.44k op/s 2024-09-11T07:44:35.284 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:44:35.354 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:35.399 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:35.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:35.400 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:35.501 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:35.534 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:35.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:35.535 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:35.636 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:35.674 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:44:35.674 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:44:35.674 INFO:teuthology.orchestra.run.smithi022.stdout: "complaint_time": 30, 2024-09-11T07:44:35.674 INFO:teuthology.orchestra.run.smithi022.stdout: "num_blocked_ops": 0 2024-09-11T07:44:35.674 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:44:35.684 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:35.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:35.685 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:35.785 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:35.826 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:35.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:35.827 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:35.928 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:35.950 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:35.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:35.951 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:36.051 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:36.086 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:36.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:36.087 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:36.188 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:36.223 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:36.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:36.224 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:36.325 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:36.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:36.358 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:36.359 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:36.427 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T07:44:36.459 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:36.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:36.494 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:36.494 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:36.595 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:36.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:36.629 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:36.629 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:36.730 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:36.781 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:36.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:36.783 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:36.883 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:36.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:36 smithi022 ceph-mon[29462]: pgmap v837: 148 pgs: 148 active+clean; 4.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.45k op/s 2024-09-11T07:44:36.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:36 smithi022 ceph-mon[37330]: pgmap v837: 148 pgs: 148 active+clean; 4.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.45k op/s 2024-09-11T07:44:36.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:36.927 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:36.927 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:37.028 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:37.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:36 smithi104 ceph-mon[33507]: pgmap v837: 148 pgs: 148 active+clean; 4.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.45k op/s 2024-09-11T07:44:37.064 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:37.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:37.065 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:37.166 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:37.204 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:37.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:37.205 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:37.306 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:37.335 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:37.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:37.337 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:37.437 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:37.474 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:37.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:37.476 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:37.576 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:37.608 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:37.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:37.610 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:37.662 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-11T07:44:37.711 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:37.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:37.769 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:37.769 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:37.870 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:37.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:37.912 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:37.913 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:38.013 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:38.047 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:38.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:38.048 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:38.149 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:38.150 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:44:38.175 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:38.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:38.176 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:38.206 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:44:38.277 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:38.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:38.322 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:38.322 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:38.423 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:38.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:38.457 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:38.458 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:38.558 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:38.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:38.601 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:38.601 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:38.702 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:38.734 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:38.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:38.735 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:38.836 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:38.872 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:38.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:38.873 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:38.974 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:39.014 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:39.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:39.015 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:39.116 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:39.144 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:39.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:39.145 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:39.246 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:39.286 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:39.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:39.287 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:39.388 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:39.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:39 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:44:39.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:39 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:44:39.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:39 smithi022 ceph-mon[29462]: pgmap v838: 148 pgs: 148 active+clean; 4.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-11T07:44:39.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:39 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:44:39.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:39 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:44:39.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:39 smithi022 ceph-mon[37330]: pgmap v838: 148 pgs: 148 active+clean; 4.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-11T07:44:39.423 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:39.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:39.424 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:39.525 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:39.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:39 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:44:39.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:39 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:44:39.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:39 smithi104 ceph-mon[33507]: pgmap v838: 148 pgs: 148 active+clean; 4.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-11T07:44:39.557 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:39.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:39.558 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:39.658 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:39.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:39.701 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:39.702 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:39.802 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:39.841 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:39.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:39.843 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:39.944 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:39.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:39.985 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:39.985 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:40.007 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:44:40.010 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:44:40.087 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:40.123 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:40.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:40.124 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:40.225 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:40.262 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:40.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:40.263 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:40.364 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:40.395 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:40.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:40.397 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:40.498 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:40.525 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:40.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:40.526 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:40.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:44:40 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:44:40.628 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:40.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:40.665 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:40.665 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:40.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:44:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:44:40] "GET /metrics HTTP/1.1" 200 224974 "" "Prometheus/2.43.0" 2024-09-11T07:44:40.767 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:40.809 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:40.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:40.810 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:40.911 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:40.931 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:40.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:40.932 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:41.033 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:41.076 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:41.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:41.077 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:41.178 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:41.210 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:41.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:41.211 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:41.312 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:41.349 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:41.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:41.350 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:41.451 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:41.491 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:41.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:41.492 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:41.592 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:41.632 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:41.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:41.633 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:41 smithi022 ceph-mon[29462]: from='client.25864 v1:172.21.15.22:0/223840889' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:44:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:41 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:44:41.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:41 smithi022 ceph-mon[29462]: pgmap v839: 148 pgs: 148 active+clean; 4.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.31k op/s 2024-09-11T07:44:41.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:41 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:44:41.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:41 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:44:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:41 smithi022 ceph-mon[37330]: from='client.25864 v1:172.21.15.22:0/223840889' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:44:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:41 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:44:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:41 smithi022 ceph-mon[37330]: pgmap v839: 148 pgs: 148 active+clean; 4.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.31k op/s 2024-09-11T07:44:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:41 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:44:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:41 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:44:41.734 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:41.761 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:41.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:41.762 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:41.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:41 smithi104 ceph-mon[33507]: from='client.25864 v1:172.21.15.22:0/223840889' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:44:41.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:41 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:44:41.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:41 smithi104 ceph-mon[33507]: pgmap v839: 148 pgs: 148 active+clean; 4.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.31k op/s 2024-09-11T07:44:41.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:41 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:44:41.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:41 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:44:41.863 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:41.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:41.888 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:41.888 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:41.989 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:42.025 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:42.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:42.026 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:42.127 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:42.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:42.163 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:42.163 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:42.264 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:42.300 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:42.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:42.301 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:42.402 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:42.425 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:42.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:42.426 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:42.527 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:42.564 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:42.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:42.565 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:42.666 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:42.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:42.702 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:42.702 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:42.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:42 smithi104 ceph-mon[33507]: osdmap e383: 8 total, 8 up, 8 in 2024-09-11T07:44:42.803 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:42.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:42.838 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:42.838 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:42.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:42 smithi022 ceph-mon[29462]: osdmap e383: 8 total, 8 up, 8 in 2024-09-11T07:44:42.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:42 smithi022 ceph-mon[37330]: osdmap e383: 8 total, 8 up, 8 in 2024-09-11T07:44:42.938 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:42.973 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:42.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:42.974 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:43.075 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:43.112 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:43.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:43.113 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:43.214 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:43.243 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:43.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:43.244 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:43.315 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:44:43.344 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:43.373 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:43.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:43.374 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:43.475 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:43.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:43.500 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:43.500 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:43.601 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:43.635 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:43.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:43.636 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:43.737 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:43.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:43 smithi104 ceph-mon[33507]: pgmap v841: 148 pgs: 148 active+clean; 4.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.38k op/s 2024-09-11T07:44:43.781 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:43.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:43.782 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:43.883 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:43.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:43.916 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:43.916 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:43.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:43 smithi022 ceph-mon[29462]: pgmap v841: 148 pgs: 148 active+clean; 4.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.38k op/s 2024-09-11T07:44:43.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:43 smithi022 ceph-mon[37330]: pgmap v841: 148 pgs: 148 active+clean; 4.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.38k op/s 2024-09-11T07:44:44.017 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:44.045 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:44.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:44.046 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:44.147 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:44.169 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:44.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:44.170 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:44.271 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:44.306 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:44.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:44.307 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:44.408 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:44.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:44.437 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:44.438 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:44.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:44:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:44:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:44:44.539 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:44.569 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:44.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:44.570 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:44.671 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:44.708 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:44.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:44.709 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:44.810 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:44.840 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:44.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:44.841 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:44.942 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:44.979 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:44.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:44.980 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:45.080 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:45.115 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:45.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:45.116 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:45.217 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:45.250 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:45.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:45.251 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:45.351 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:45.390 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:45.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:45.391 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:45.492 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:45.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:45.525 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:45.525 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:45.626 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:45.663 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:45.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:45.664 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:45.765 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:45.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:45 smithi104 ceph-mon[33507]: pgmap v842: 148 pgs: 148 active+clean; 3.7 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.29k op/s 2024-09-11T07:44:45.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:45 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:44:45.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:45 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:44:45.798 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:45.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:45.799 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:45.900 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:45.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:45 smithi022 ceph-mon[29462]: pgmap v842: 148 pgs: 148 active+clean; 3.7 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.29k op/s 2024-09-11T07:44:45.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:45 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:44:45.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:45 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:44:45.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:45 smithi022 ceph-mon[37330]: pgmap v842: 148 pgs: 148 active+clean; 3.7 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.29k op/s 2024-09-11T07:44:45.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:45 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:44:45.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:45 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:44:45.935 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:45.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:45.937 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:46.037 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:46.072 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:46.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:46.074 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:46.175 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:46.211 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:46.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:46.212 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:46.313 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:46.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:46.337 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:46.337 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:46.437 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:46.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:46.473 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:46.473 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:46.574 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:46.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:46.612 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:46.612 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:46.713 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:46.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:46.749 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:46.749 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:46.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:46 smithi104 ceph-mon[33507]: osdmap e384: 8 total, 8 up, 8 in 2024-09-11T07:44:46.849 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:46.885 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:46.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:46.886 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:46.916 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:46 smithi022 ceph-mon[37330]: osdmap e384: 8 total, 8 up, 8 in 2024-09-11T07:44:46.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:46 smithi022 ceph-mon[29462]: osdmap e384: 8 total, 8 up, 8 in 2024-09-11T07:44:46.986 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:47.021 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:47.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:47.022 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:47.123 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:47.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:47.162 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:47.162 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:47.263 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:47.298 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:47.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:47.299 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:47.360 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":838,"stamp":"2024-09-11T07:44:38.348046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4739438836,"num_objects":72506,"num_object_clones":0,"num_object_copies":217518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72506,"num_whiteouts":0,"num_read":1283,"num_read_kb":1140,"num_write":187367,"num_write_kb":8284744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":1184985,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79959,"ondisk_log_size":79959,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":20680388,"kb_used_data":14049008,"kb_used_omap":16,"kb_used_meta":6630703,"kb_avail":729280828,"statfs":{"total":767960285184,"available":746783567872,"internally_reserved":0,"allocated":14386184192,"data_stored":14383025500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16530,"internal_metadata":6789840750},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,2,4,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-810614784,"num_objects":-12369,"num_object_clones":0,"num_object_copies":-37107,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-12369,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":12369,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.004552"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":135,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.802304+0000","last_change":"2024-09-11T07:44:23.770638+0000","last_active":"2024-09-11T07:44:27.802304+0000","last_peered":"2024-09-11T07:44:27.802304+0000","last_clean":"2024-09-11T07:44:27.802304+0000","last_became_active":"2024-09-11T07:44:23.769058+0000","last_became_peered":"2024-09-11T07:44:23.769058+0000","last_unstale":"2024-09-11T07:44:27.802304+0000","last_undegraded":"2024-09-11T07:44:27.802304+0000","last_fullsized":"2024-09-11T07:44:27.802304+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:00.048124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:29.007411+0000","last_clean_scrub_stamp":"2024-09-11T07:44:00.048124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:21.082122+0000","scrub_duration":0.001472253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":430,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.803175+0000","last_change":"2024-09-11T07:44:21.375151+0000","last_active":"2024-09-11T07:44:27.803175+0000","last_peered":"2024-09-11T07:44:27.803175+0000","last_clean":"2024-09-11T07:44:27.803175+0000","last_became_active":"2024-09-11T07:32:00.508572+0000","last_became_peered":"2024-09-11T07:32:00.508572+0000","last_unstale":"2024-09-11T07:44:27.803175+0000","last_undegraded":"2024-09-11T07:44:27.803175+0000","last_fullsized":"2024-09-11T07:44:27.803175+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:21.375076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:21.375076+0000","last_clean_scrub_stamp":"2024-09-11T07:44:21.375076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:50.804301+0000","scrub_duration":0.0068686850000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"234'34","reported_seq":454,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.792569+0000","last_change":"2024-09-11T07:44:02.682805+0000","last_active":"2024-09-11T07:44:27.792569+0000","last_peered":"2024-09-11T07:44:27.792569+0000","last_clean":"2024-09-11T07:44:27.792569+0000","last_became_active":"2024-09-11T07:26:33.544627+0000","last_became_peered":"2024-09-11T07:26:33.544627+0000","last_unstale":"2024-09-11T07:44:27.792569+0000","last_undegraded":"2024-09-11T07:44:27.792569+0000","last_fullsized":"2024-09-11T07:44:27.792569+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'34","last_scrub_stamp":"2024-09-11T07:44:02.682692+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:44:02.682692+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:25.577522+0000","scrub_duration":0.0056034049999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":409,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.794480+0000","last_change":"2024-09-11T07:43:21.291883+0000","last_active":"2024-09-11T07:44:27.794480+0000","last_peered":"2024-09-11T07:44:27.794480+0000","last_clean":"2024-09-11T07:44:27.794480+0000","last_became_active":"2024-09-11T07:26:35.556296+0000","last_became_peered":"2024-09-11T07:26:35.556296+0000","last_unstale":"2024-09-11T07:44:27.794480+0000","last_undegraded":"2024-09-11T07:44:27.794480+0000","last_fullsized":"2024-09-11T07:44:27.794480+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:21.291787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:43:21.291787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:46.989515+0000","scrub_duration":0.003794969,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":429,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.794026+0000","last_change":"2024-09-11T07:44:19.655482+0000","last_active":"2024-09-11T07:44:27.794026+0000","last_peered":"2024-09-11T07:44:27.794026+0000","last_clean":"2024-09-11T07:44:27.794026+0000","last_became_active":"2024-09-11T07:39:57.767806+0000","last_became_peered":"2024-09-11T07:39:57.767806+0000","last_unstale":"2024-09-11T07:44:27.794026+0000","last_undegraded":"2024-09-11T07:44:27.794026+0000","last_fullsized":"2024-09-11T07:44:27.794026+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:19.655401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:19.655401+0000","last_clean_scrub_stamp":"2024-09-11T07:44:19.655401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:19.877337+0000","scrub_duration":0.0025632099999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"234'19","reported_seq":436,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.794556+0000","last_change":"2024-09-11T07:43:59.245247+0000","last_active":"2024-09-11T07:44:27.794556+0000","last_peered":"2024-09-11T07:44:27.794556+0000","last_clean":"2024-09-11T07:44:27.794556+0000","last_became_active":"2024-09-11T07:26:33.540732+0000","last_became_peered":"2024-09-11T07:26:33.540732+0000","last_unstale":"2024-09-11T07:44:27.794556+0000","last_undegraded":"2024-09-11T07:44:27.794556+0000","last_fullsized":"2024-09-11T07:44:27.794556+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:43:59.245081+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:36:42.708941+0000","last_clean_scrub_stamp":"2024-09-11T07:43:59.245081+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:20.943226+0000","scrub_duration":0.0024691890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":139,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.800731+0000","last_change":"2024-09-11T07:44:23.769817+0000","last_active":"2024-09-11T07:44:27.800731+0000","last_peered":"2024-09-11T07:44:27.800731+0000","last_clean":"2024-09-11T07:44:27.800731+0000","last_became_active":"2024-09-11T07:44:23.769502+0000","last_became_peered":"2024-09-11T07:44:23.769502+0000","last_unstale":"2024-09-11T07:44:27.800731+0000","last_undegraded":"2024-09-11T07:44:27.800731+0000","last_fullsized":"2024-09-11T07:44:27.800731+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:19.656481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:43:19.656481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:34.225080+0000","scrub_duration":0.0063588289999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":420,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.794460+0000","last_change":"2024-09-11T07:43:25.450161+0000","last_active":"2024-09-11T07:44:27.794460+0000","last_peered":"2024-09-11T07:44:27.794460+0000","last_clean":"2024-09-11T07:44:27.794460+0000","last_became_active":"2024-09-11T07:40:11.698235+0000","last_became_peered":"2024-09-11T07:40:11.698235+0000","last_unstale":"2024-09-11T07:44:27.794460+0000","last_undegraded":"2024-09-11T07:44:27.794460+0000","last_fullsized":"2024-09-11T07:44:27.794460+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:25.450076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:43:25.450076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:35.537752+0000","scrub_duration":0.15257026000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":408,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.801227+0000","last_change":"2024-09-11T07:43:26.559203+0000","last_active":"2024-09-11T07:44:27.801227+0000","last_peered":"2024-09-11T07:44:27.801227+0000","last_clean":"2024-09-11T07:44:27.801227+0000","last_became_active":"2024-09-11T07:32:00.516662+0000","last_became_peered":"2024-09-11T07:32:00.516662+0000","last_unstale":"2024-09-11T07:44:27.801227+0000","last_undegraded":"2024-09-11T07:44:27.801227+0000","last_fullsized":"2024-09-11T07:44:27.801227+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:26.559109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:43:26.559109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:50.733895+0000","scrub_duration":0.0044217739999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"234'9","reported_seq":448,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.794710+0000","last_change":"2024-09-11T07:43:10.453615+0000","last_active":"2024-09-11T07:44:27.794710+0000","last_peered":"2024-09-11T07:44:27.794710+0000","last_clean":"2024-09-11T07:44:27.794710+0000","last_became_active":"2024-09-11T07:43:10.446549+0000","last_became_peered":"2024-09-11T07:43:10.446549+0000","last_unstale":"2024-09-11T07:44:27.794710+0000","last_undegraded":"2024-09-11T07:44:27.794710+0000","last_fullsized":"2024-09-11T07:44:27.794710+0000","mapping_epoch":360,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":361,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'9","last_scrub_stamp":"2024-09-11T07:42:14.618430+0000","last_deep_scrub":"82'7","last_deep_scrub_stamp":"2024-09-11T07:27:52.097071+0000","last_clean_scrub_stamp":"2024-09-11T07:42:14.618430+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001734221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":387,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:32.398526+0000","last_change":"2024-09-11T07:44:32.398474+0000","last_active":"2024-09-11T07:44:32.398526+0000","last_peered":"2024-09-11T07:44:32.398526+0000","last_clean":"2024-09-11T07:44:32.398526+0000","last_became_active":"2024-09-11T07:32:00.513206+0000","last_became_peered":"2024-09-11T07:32:00.513206+0000","last_unstale":"2024-09-11T07:44:32.398526+0000","last_undegraded":"2024-09-11T07:44:32.398526+0000","last_fullsized":"2024-09-11T07:44:32.398526+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:32.398398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.973323+0000","last_clean_scrub_stamp":"2024-09-11T07:44:32.398398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:01.259799+0000","scrub_duration":0.002203782,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":37,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.800024+0000","last_change":"2024-09-11T07:44:23.769584+0000","last_active":"2024-09-11T07:44:27.800024+0000","last_peered":"2024-09-11T07:44:27.800024+0000","last_clean":"2024-09-11T07:44:27.800024+0000","last_became_active":"2024-09-11T07:44:23.769187+0000","last_became_peered":"2024-09-11T07:44:23.769187+0000","last_unstale":"2024-09-11T07:44:27.800024+0000","last_undegraded":"2024-09-11T07:44:27.800024+0000","last_fullsized":"2024-09-11T07:44:27.800024+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:04.074067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:45.149916+0000","last_clean_scrub_stamp":"2024-09-11T07:44:04.074067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:31.054128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":373,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.806151+0000","last_change":"2024-09-11T07:43:56.219460+0000","last_active":"2024-09-11T07:44:27.806151+0000","last_peered":"2024-09-11T07:44:27.806151+0000","last_clean":"2024-09-11T07:44:27.806151+0000","last_became_active":"2024-09-11T07:32:00.505709+0000","last_became_peered":"2024-09-11T07:32:00.505709+0000","last_unstale":"2024-09-11T07:44:27.806151+0000","last_undegraded":"2024-09-11T07:44:27.806151+0000","last_fullsized":"2024-09-11T07:44:27.806151+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:56.219368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_clean_scrub_stamp":"2024-09-11T07:43:56.219368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:04.957165+0000","scrub_duration":0.0023725780000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"234'19","reported_seq":433,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.794736+0000","last_change":"2024-09-11T07:43:57.284409+0000","last_active":"2024-09-11T07:44:27.794736+0000","last_peered":"2024-09-11T07:44:27.794736+0000","last_clean":"2024-09-11T07:44:27.794736+0000","last_became_active":"2024-09-11T07:26:33.544386+0000","last_became_peered":"2024-09-11T07:26:33.544386+0000","last_unstale":"2024-09-11T07:44:27.794736+0000","last_undegraded":"2024-09-11T07:44:27.794736+0000","last_fullsized":"2024-09-11T07:44:27.794736+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:43:57.284262+0000","last_deep_scrub":"136'13","last_deep_scrub_stamp":"2024-09-11T07:31:08.042198+0000","last_clean_scrub_stamp":"2024-09-11T07:43:57.284262+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:20.074875+0000","scrub_duration":0.010992422,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":425,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.802856+0000","last_change":"2024-09-11T07:44:24.340103+0000","last_active":"2024-09-11T07:44:27.802856+0000","last_peered":"2024-09-11T07:44:27.802856+0000","last_clean":"2024-09-11T07:44:27.802856+0000","last_became_active":"2024-09-11T07:40:11.699772+0000","last_became_peered":"2024-09-11T07:40:11.699772+0000","last_unstale":"2024-09-11T07:44:27.802856+0000","last_undegraded":"2024-09-11T07:44:27.802856+0000","last_fullsized":"2024-09-11T07:44:27.802856+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:24.340025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:44:24.340025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:26.742277+0000","scrub_duration":0.002461366,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":424,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.801068+0000","last_change":"2024-09-11T07:44:04.287572+0000","last_active":"2024-09-11T07:44:27.801068+0000","last_peered":"2024-09-11T07:44:27.801068+0000","last_clean":"2024-09-11T07:44:27.801068+0000","last_became_active":"2024-09-11T07:32:00.530335+0000","last_became_peered":"2024-09-11T07:32:00.530335+0000","last_unstale":"2024-09-11T07:44:27.801068+0000","last_undegraded":"2024-09-11T07:44:27.801068+0000","last_fullsized":"2024-09-11T07:44:27.801068+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:04.287469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:44:04.287469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:08.494521+0000","scrub_duration":0.011609975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3","version":"382'11599","reported_seq":11925,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-09-11T07:44:34.132214+0000","last_change":"2024-09-11T07:44:06.274133+0000","last_active":"2024-09-11T07:44:34.132214+0000","last_peered":"2024-09-11T07:44:34.132214+0000","last_clean":"2024-09-11T07:44:34.132214+0000","last_became_active":"2024-09-11T07:41:57.697479+0000","last_became_peered":"2024-09-11T07:41:57.697479+0000","last_unstale":"2024-09-11T07:44:34.132214+0000","last_undegraded":"2024-09-11T07:44:34.132214+0000","last_fullsized":"2024-09-11T07:44:34.132214+0000","mapping_epoch":341,"log_start":"364'6183","ondisk_log_start":"364'6183","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'9612","last_scrub_stamp":"2024-09-11T07:44:06.273902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:44:06.273902+0000","objects_scrubbed":6703,"log_size":5416,"log_dups_size":0,"ondisk_log_size":5416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:18.284286+0000","scrub_duration":2.7623221249999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":304021504,"num_objects":4639,"num_object_clones":0,"num_object_copies":13917,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11599,"num_write_kb":519616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"234'31","reported_seq":509,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.796089+0000","last_change":"2024-09-11T07:44:06.132784+0000","last_active":"2024-09-11T07:44:27.796089+0000","last_peered":"2024-09-11T07:44:27.796089+0000","last_clean":"2024-09-11T07:44:27.796089+0000","last_became_active":"2024-09-11T07:29:51.482755+0000","last_became_peered":"2024-09-11T07:29:51.482755+0000","last_unstale":"2024-09-11T07:44:27.796089+0000","last_undegraded":"2024-09-11T07:44:27.796089+0000","last_fullsized":"2024-09-11T07:44:27.796089+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'31","last_scrub_stamp":"2024-09-11T07:44:06.132709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:44:06.132709+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:33.086546+0000","scrub_duration":0.012657417000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":406,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.800841+0000","last_change":"2024-09-11T07:43:04.601186+0000","last_active":"2024-09-11T07:44:27.800841+0000","last_peered":"2024-09-11T07:44:27.800841+0000","last_clean":"2024-09-11T07:44:27.800841+0000","last_became_active":"2024-09-11T07:32:00.507191+0000","last_became_peered":"2024-09-11T07:32:00.507191+0000","last_unstale":"2024-09-11T07:44:27.800841+0000","last_undegraded":"2024-09-11T07:44:27.800841+0000","last_fullsized":"2024-09-11T07:44:27.800841+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:04.601133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:43:04.601133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012288259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":16,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.796915+0000","last_change":"2024-09-11T07:44:23.770079+0000","last_active":"2024-09-11T07:44:27.796915+0000","last_peered":"2024-09-11T07:44:27.796915+0000","last_clean":"2024-09-11T07:44:27.796915+0000","last_became_active":"2024-09-11T07:44:23.769605+0000","last_became_peered":"2024-09-11T07:44:23.769605+0000","last_unstale":"2024-09-11T07:44:27.796915+0000","last_undegraded":"2024-09-11T07:44:27.796915+0000","last_fullsized":"2024-09-11T07:44:27.796915+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:23.359400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:43:23.359400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:49.668355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":420,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.801808+0000","last_change":"2024-09-11T07:43:57.645751+0000","last_active":"2024-09-11T07:44:27.801808+0000","last_peered":"2024-09-11T07:44:27.801808+0000","last_clean":"2024-09-11T07:44:27.801808+0000","last_became_active":"2024-09-11T07:26:37.564554+0000","last_became_peered":"2024-09-11T07:26:37.564554+0000","last_unstale":"2024-09-11T07:44:27.801808+0000","last_undegraded":"2024-09-11T07:44:27.801808+0000","last_fullsized":"2024-09-11T07:44:27.801808+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:57.645536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:32.281886+0000","last_clean_scrub_stamp":"2024-09-11T07:43:57.645536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:07.247192+0000","scrub_duration":0.0018691090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"382'11637","reported_seq":11908,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-09-11T07:44:34.032150+0000","last_change":"2024-09-11T07:44:22.932534+0000","last_active":"2024-09-11T07:44:34.032150+0000","last_peered":"2024-09-11T07:44:34.032150+0000","last_clean":"2024-09-11T07:44:34.032150+0000","last_became_active":"2024-09-11T07:41:57.974623+0000","last_became_peered":"2024-09-11T07:41:57.974623+0000","last_unstale":"2024-09-11T07:44:34.032150+0000","last_undegraded":"2024-09-11T07:44:34.032150+0000","last_fullsized":"2024-09-11T07:44:34.032150+0000","mapping_epoch":341,"log_start":"367'7300","ondisk_log_start":"367'7300","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"379'10838","last_scrub_stamp":"2024-09-11T07:44:22.932448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.932448+0000","objects_scrubbed":5343,"log_size":4337,"log_dups_size":0,"ondisk_log_size":4337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:34.639031+0000","scrub_duration":1.827374225,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":293797888,"num_objects":4483,"num_object_clones":0,"num_object_copies":13449,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11637,"num_write_kb":515840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.c","version":"234'25","reported_seq":439,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.797014+0000","last_change":"2024-09-11T07:42:32.557611+0000","last_active":"2024-09-11T07:44:27.797014+0000","last_peered":"2024-09-11T07:44:27.797014+0000","last_clean":"2024-09-11T07:44:27.797014+0000","last_became_active":"2024-09-11T07:26:33.535411+0000","last_became_peered":"2024-09-11T07:26:33.535411+0000","last_unstale":"2024-09-11T07:44:27.797014+0000","last_undegraded":"2024-09-11T07:44:27.797014+0000","last_fullsized":"2024-09-11T07:44:27.797014+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'25","last_scrub_stamp":"2024-09-11T07:42:32.557306+0000","last_deep_scrub":"112'15","last_deep_scrub_stamp":"2024-09-11T07:31:10.005896+0000","last_clean_scrub_stamp":"2024-09-11T07:42:32.557306+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042575720000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":143,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.793423+0000","last_change":"2024-09-11T07:44:23.768467+0000","last_active":"2024-09-11T07:44:27.793423+0000","last_peered":"2024-09-11T07:44:27.793423+0000","last_clean":"2024-09-11T07:44:27.793423+0000","last_became_active":"2024-09-11T07:44:23.768172+0000","last_became_peered":"2024-09-11T07:44:23.768172+0000","last_unstale":"2024-09-11T07:44:27.793423+0000","last_undegraded":"2024-09-11T07:44:27.793423+0000","last_fullsized":"2024-09-11T07:44:27.793423+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:54.910214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:43:54.910214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:03.045744+0000","scrub_duration":0.015981232000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"299'11","reported_seq":669,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:29.732994+0000","last_change":"2024-09-11T07:43:21.381901+0000","last_active":"2024-09-11T07:44:29.732994+0000","last_peered":"2024-09-11T07:44:29.732994+0000","last_clean":"2024-09-11T07:44:29.732994+0000","last_became_active":"2024-09-11T07:40:11.697568+0000","last_became_peered":"2024-09-11T07:40:11.697568+0000","last_unstale":"2024-09-11T07:44:29.732994+0000","last_undegraded":"2024-09-11T07:44:29.732994+0000","last_fullsized":"2024-09-11T07:44:29.732994+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"299'11","last_scrub_stamp":"2024-09-11T07:43:21.381642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:43:21.381642+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:40.682699+0000","scrub_duration":0.0023952790000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":407,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.795864+0000","last_change":"2024-09-11T07:43:16.342459+0000","last_active":"2024-09-11T07:44:27.795864+0000","last_peered":"2024-09-11T07:44:27.795864+0000","last_clean":"2024-09-11T07:44:27.795864+0000","last_became_active":"2024-09-11T07:39:57.697588+0000","last_became_peered":"2024-09-11T07:39:57.697588+0000","last_unstale":"2024-09-11T07:44:27.795864+0000","last_undegraded":"2024-09-11T07:44:27.795864+0000","last_fullsized":"2024-09-11T07:44:27.795864+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:16.342336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:43:16.342336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.063116406999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1","version":"382'11725","reported_seq":11772,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-09-11T07:44:34.041276+0000","last_change":"2024-09-11T07:41:57.700507+0000","last_active":"2024-09-11T07:44:34.041276+0000","last_peered":"2024-09-11T07:44:34.041276+0000","last_clean":"2024-09-11T07:44:34.041276+0000","last_became_active":"2024-09-11T07:41:57.700179+0000","last_became_peered":"2024-09-11T07:41:57.700179+0000","last_unstale":"2024-09-11T07:44:34.041276+0000","last_undegraded":"2024-09-11T07:44:34.041276+0000","last_fullsized":"2024-09-11T07:44:34.041276+0000","mapping_epoch":341,"log_start":"367'7400","ondisk_log_start":"367'7400","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:41:56.678995+0000","objects_scrubbed":0,"log_size":4325,"log_dups_size":0,"ondisk_log_size":4325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":296026112,"num_objects":4517,"num_object_clones":0,"num_object_copies":13551,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11725,"num_write_kb":519744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"234'34","reported_seq":285,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.794579+0000","last_change":"2024-09-11T07:44:23.770978+0000","last_active":"2024-09-11T07:44:27.794579+0000","last_peered":"2024-09-11T07:44:27.794579+0000","last_clean":"2024-09-11T07:44:27.794579+0000","last_became_active":"2024-09-11T07:44:23.769306+0000","last_became_peered":"2024-09-11T07:44:23.769306+0000","last_unstale":"2024-09-11T07:44:27.794579+0000","last_undegraded":"2024-09-11T07:44:27.794579+0000","last_fullsized":"2024-09-11T07:44:27.794579+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'34","last_scrub_stamp":"2024-09-11T07:43:57.973282+0000","last_deep_scrub":"83'22","last_deep_scrub_stamp":"2024-09-11T07:34:04.375457+0000","last_clean_scrub_stamp":"2024-09-11T07:43:57.973282+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:14.982270+0000","scrub_duration":0.0086024310000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":163,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.800617+0000","last_change":"2024-09-11T07:44:27.416260+0000","last_active":"2024-09-11T07:44:27.800617+0000","last_peered":"2024-09-11T07:44:27.800617+0000","last_clean":"2024-09-11T07:44:27.800617+0000","last_became_active":"2024-09-11T07:44:23.770343+0000","last_became_peered":"2024-09-11T07:44:23.770343+0000","last_unstale":"2024-09-11T07:44:27.800617+0000","last_undegraded":"2024-09-11T07:44:27.800617+0000","last_fullsized":"2024-09-11T07:44:27.800617+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:27.416207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:27.416207+0000","last_clean_scrub_stamp":"2024-09-11T07:44:27.416207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:35.967986+0000","scrub_duration":0.001230096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"366'9","reported_seq":627,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:29.729631+0000","last_change":"2024-09-11T07:43:26.334831+0000","last_active":"2024-09-11T07:44:29.729631+0000","last_peered":"2024-09-11T07:44:29.729631+0000","last_clean":"2024-09-11T07:44:29.729631+0000","last_became_active":"2024-09-11T07:26:35.560406+0000","last_became_peered":"2024-09-11T07:26:35.560406+0000","last_unstale":"2024-09-11T07:44:29.729631+0000","last_undegraded":"2024-09-11T07:44:29.729631+0000","last_fullsized":"2024-09-11T07:44:29.729631+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"366'9","last_scrub_stamp":"2024-09-11T07:43:26.334501+0000","last_deep_scrub":"366'9","last_deep_scrub_stamp":"2024-09-11T07:43:26.334501+0000","last_clean_scrub_stamp":"2024-09-11T07:43:26.334501+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:48.224289+0000","scrub_duration":0.0045954409999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":407,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.805859+0000","last_change":"2024-09-11T07:43:21.293977+0000","last_active":"2024-09-11T07:44:27.805859+0000","last_peered":"2024-09-11T07:44:27.805859+0000","last_clean":"2024-09-11T07:44:27.805859+0000","last_became_active":"2024-09-11T07:39:57.762309+0000","last_became_peered":"2024-09-11T07:39:57.762309+0000","last_unstale":"2024-09-11T07:44:27.805859+0000","last_undegraded":"2024-09-11T07:44:27.805859+0000","last_fullsized":"2024-09-11T07:44:27.805859+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:21.293894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:31.200685+0000","last_clean_scrub_stamp":"2024-09-11T07:43:21.293894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:45.902198+0000","scrub_duration":0.0065006049999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"382'11491","reported_seq":11644,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-09-11T07:44:34.006532+0000","last_change":"2024-09-11T07:44:23.808380+0000","last_active":"2024-09-11T07:44:34.006532+0000","last_peered":"2024-09-11T07:44:34.006532+0000","last_clean":"2024-09-11T07:44:34.006532+0000","last_became_active":"2024-09-11T07:44:23.768079+0000","last_became_peered":"2024-09-11T07:44:23.768079+0000","last_unstale":"2024-09-11T07:44:34.006532+0000","last_undegraded":"2024-09-11T07:44:34.006532+0000","last_fullsized":"2024-09-11T07:44:34.006532+0000","mapping_epoch":379,"log_start":"367'7172","ondisk_log_start":"367'7172","created":341,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:41:56.678995+0000","objects_scrubbed":2261,"log_size":4319,"log_dups_size":0,"ondisk_log_size":4319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":292880384,"num_objects":4469,"num_object_clones":0,"num_object_copies":13407,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11491,"num_write_kb":510720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"234'24","reported_seq":469,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.796042+0000","last_change":"2024-09-11T07:43:55.283287+0000","last_active":"2024-09-11T07:44:27.796042+0000","last_peered":"2024-09-11T07:44:27.796042+0000","last_clean":"2024-09-11T07:44:27.796042+0000","last_became_active":"2024-09-11T07:39:57.766134+0000","last_became_peered":"2024-09-11T07:39:57.766134+0000","last_unstale":"2024-09-11T07:44:27.796042+0000","last_undegraded":"2024-09-11T07:44:27.796042+0000","last_fullsized":"2024-09-11T07:44:27.796042+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'24","last_scrub_stamp":"2024-09-11T07:43:55.283206+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-09-11T07:29:38.112111+0000","last_clean_scrub_stamp":"2024-09-11T07:43:55.283206+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:16.903591+0000","scrub_duration":0.003457725,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":400,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.800323+0000","last_change":"2024-09-11T07:43:24.643754+0000","last_active":"2024-09-11T07:44:27.800323+0000","last_peered":"2024-09-11T07:44:27.800323+0000","last_clean":"2024-09-11T07:44:27.800323+0000","last_became_active":"2024-09-11T07:32:00.518538+0000","last_became_peered":"2024-09-11T07:32:00.518538+0000","last_unstale":"2024-09-11T07:44:27.800323+0000","last_undegraded":"2024-09-11T07:44:27.800323+0000","last_fullsized":"2024-09-11T07:44:27.800323+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:24.643676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:43:24.643676+0000","last_clean_scrub_stamp":"2024-09-11T07:43:24.643676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:37.543604+0000","scrub_duration":0.0039936859999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":58,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.794872+0000","last_change":"2024-09-11T07:42:19.714646+0000","last_active":"2024-09-11T07:44:27.794872+0000","last_peered":"2024-09-11T07:44:27.794872+0000","last_clean":"2024-09-11T07:44:27.794872+0000","last_became_active":"2024-09-11T07:42:04.729291+0000","last_became_peered":"2024-09-11T07:42:04.729291+0000","last_unstale":"2024-09-11T07:44:27.794872+0000","last_undegraded":"2024-09-11T07:44:27.794872+0000","last_fullsized":"2024-09-11T07:44:27.794872+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:19.714568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:19.714568+0000","last_clean_scrub_stamp":"2024-09-11T07:42:19.714568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025615109999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":397,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.792450+0000","last_change":"2024-09-11T07:43:13.372948+0000","last_active":"2024-09-11T07:44:27.792450+0000","last_peered":"2024-09-11T07:44:27.792450+0000","last_clean":"2024-09-11T07:44:27.792450+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:44:27.792450+0000","last_undegraded":"2024-09-11T07:44:27.792450+0000","last_fullsized":"2024-09-11T07:44:27.792450+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:13.372485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:46.853387+0000","last_clean_scrub_stamp":"2024-09-11T07:43:13.372485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:42.160965+0000","scrub_duration":0.0038403320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"382'11545","reported_seq":11852,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-09-11T07:44:35.141962+0000","last_change":"2024-09-11T07:43:21.477070+0000","last_active":"2024-09-11T07:44:35.141962+0000","last_peered":"2024-09-11T07:44:35.141962+0000","last_clean":"2024-09-11T07:44:35.141962+0000","last_became_active":"2024-09-11T07:41:57.973267+0000","last_became_peered":"2024-09-11T07:41:57.973267+0000","last_unstale":"2024-09-11T07:44:35.141962+0000","last_undegraded":"2024-09-11T07:44:35.141962+0000","last_fullsized":"2024-09-11T07:44:35.141962+0000","mapping_epoch":341,"log_start":"360'5500","ondisk_log_start":"360'5500","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"365'6295","last_scrub_stamp":"2024-09-11T07:43:21.477005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:43:21.477005+0000","objects_scrubbed":6192,"log_size":6045,"log_dups_size":0,"ondisk_log_size":6045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:37.393380+0000","scrub_duration":2.9790974430000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":285933568,"num_objects":4363,"num_object_clones":0,"num_object_copies":13089,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11545,"num_write_kb":509056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"234'28","reported_seq":443,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.795526+0000","last_change":"2024-09-11T07:43:24.327838+0000","last_active":"2024-09-11T07:44:27.795526+0000","last_peered":"2024-09-11T07:44:27.795526+0000","last_clean":"2024-09-11T07:44:27.795526+0000","last_became_active":"2024-09-11T07:26:33.537850+0000","last_became_peered":"2024-09-11T07:26:33.537850+0000","last_unstale":"2024-09-11T07:44:27.795526+0000","last_undegraded":"2024-09-11T07:44:27.795526+0000","last_fullsized":"2024-09-11T07:44:27.795526+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'28","last_scrub_stamp":"2024-09-11T07:43:24.327746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:43:24.327746+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:45.802197+0000","scrub_duration":0.0045852430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":179,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.802040+0000","last_change":"2024-09-11T07:43:24.326702+0000","last_active":"2024-09-11T07:44:27.802040+0000","last_peered":"2024-09-11T07:44:27.802040+0000","last_clean":"2024-09-11T07:44:27.802040+0000","last_became_active":"2024-09-11T07:39:57.767741+0000","last_became_peered":"2024-09-11T07:39:57.767741+0000","last_unstale":"2024-09-11T07:44:27.802040+0000","last_undegraded":"2024-09-11T07:44:27.802040+0000","last_fullsized":"2024-09-11T07:44:27.802040+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:24.326341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:43:24.326341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:34.151707+0000","scrub_duration":0.0027901060000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3],"acting":[2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"84'9","reported_seq":625,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-09-11T07:44:32.868575+0000","last_change":"2024-09-11T07:43:57.666749+0000","last_active":"2024-09-11T07:44:32.868575+0000","last_peered":"2024-09-11T07:44:32.868575+0000","last_clean":"2024-09-11T07:44:32.868575+0000","last_became_active":"2024-09-11T07:26:35.562997+0000","last_became_peered":"2024-09-11T07:26:35.562997+0000","last_unstale":"2024-09-11T07:44:32.868575+0000","last_undegraded":"2024-09-11T07:44:32.868575+0000","last_fullsized":"2024-09-11T07:44:32.868575+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-11T07:43:57.666646+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-11T07:35:39.832547+0000","last_clean_scrub_stamp":"2024-09-11T07:43:57.666646+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:20.191077+0000","scrub_duration":0.0034760120000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":381,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.800765+0000","last_change":"2024-09-11T07:44:17.483184+0000","last_active":"2024-09-11T07:44:27.800765+0000","last_peered":"2024-09-11T07:44:27.800765+0000","last_clean":"2024-09-11T07:44:27.800765+0000","last_became_active":"2024-09-11T07:32:00.511005+0000","last_became_peered":"2024-09-11T07:32:00.511005+0000","last_unstale":"2024-09-11T07:44:27.800765+0000","last_undegraded":"2024-09-11T07:44:27.800765+0000","last_fullsized":"2024-09-11T07:44:27.800765+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:17.483090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:44:17.483090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:36.992770+0000","scrub_duration":0.0057481720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"382'11669","reported_seq":11718,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-09-11T07:44:34.010466+0000","last_change":"2024-09-11T07:41:57.973797+0000","last_active":"2024-09-11T07:44:34.010466+0000","last_peered":"2024-09-11T07:44:34.010466+0000","last_clean":"2024-09-11T07:44:34.010466+0000","last_became_active":"2024-09-11T07:41:57.973510+0000","last_became_peered":"2024-09-11T07:41:57.973510+0000","last_unstale":"2024-09-11T07:44:34.010466+0000","last_undegraded":"2024-09-11T07:44:34.010466+0000","last_fullsized":"2024-09-11T07:44:34.010466+0000","mapping_epoch":341,"log_start":"368'7300","ondisk_log_start":"368'7300","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:41:56.678995+0000","objects_scrubbed":0,"log_size":4369,"log_dups_size":0,"ondisk_log_size":4369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":291962880,"num_objects":4455,"num_object_clones":0,"num_object_copies":13365,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11669,"num_write_kb":515968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"234'38","reported_seq":449,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.801161+0000","last_change":"2024-09-11T07:43:59.351755+0000","last_active":"2024-09-11T07:44:27.801161+0000","last_peered":"2024-09-11T07:44:27.801161+0000","last_clean":"2024-09-11T07:44:27.801161+0000","last_became_active":"2024-09-11T07:39:57.775541+0000","last_became_peered":"2024-09-11T07:39:57.775541+0000","last_unstale":"2024-09-11T07:44:27.801161+0000","last_undegraded":"2024-09-11T07:44:27.801161+0000","last_fullsized":"2024-09-11T07:44:27.801161+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'38","last_scrub_stamp":"2024-09-11T07:43:59.351585+0000","last_deep_scrub":"87'26","last_deep_scrub_stamp":"2024-09-11T07:35:52.942347+0000","last_clean_scrub_stamp":"2024-09-11T07:43:59.351585+0000","objects_scrubbed":8,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:07.845180+0000","scrub_duration":0.0056500760000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":51,"num_read_kb":39,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":108,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.795520+0000","last_change":"2024-09-11T07:42:13.577814+0000","last_active":"2024-09-11T07:44:27.795520+0000","last_peered":"2024-09-11T07:44:27.795520+0000","last_clean":"2024-09-11T07:44:27.795520+0000","last_became_active":"2024-09-11T07:40:11.698558+0000","last_became_peered":"2024-09-11T07:40:11.698558+0000","last_unstale":"2024-09-11T07:44:27.795520+0000","last_undegraded":"2024-09-11T07:44:27.795520+0000","last_fullsized":"2024-09-11T07:44:27.795520+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:13.577750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:42:13.577750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017170639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":419,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.795376+0000","last_change":"2024-09-11T07:42:31.509275+0000","last_active":"2024-09-11T07:44:27.795376+0000","last_peered":"2024-09-11T07:44:27.795376+0000","last_clean":"2024-09-11T07:44:27.795376+0000","last_became_active":"2024-09-11T07:39:57.766717+0000","last_became_peered":"2024-09-11T07:39:57.766717+0000","last_unstale":"2024-09-11T07:44:27.795376+0000","last_undegraded":"2024-09-11T07:44:27.795376+0000","last_fullsized":"2024-09-11T07:44:27.795376+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:31.509199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:27.968579+0000","last_clean_scrub_stamp":"2024-09-11T07:42:31.509199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025090170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":407,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.794658+0000","last_change":"2024-09-11T07:43:07.335284+0000","last_active":"2024-09-11T07:44:27.794658+0000","last_peered":"2024-09-11T07:44:27.794658+0000","last_clean":"2024-09-11T07:44:27.794658+0000","last_became_active":"2024-09-11T07:39:57.691831+0000","last_became_peered":"2024-09-11T07:39:57.691831+0000","last_unstale":"2024-09-11T07:44:27.794658+0000","last_undegraded":"2024-09-11T07:44:27.794658+0000","last_fullsized":"2024-09-11T07:44:27.794658+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:07.335214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:29.978798+0000","last_clean_scrub_stamp":"2024-09-11T07:43:07.335214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025693080000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.d","version":"382'11495","reported_seq":11545,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-09-11T07:44:34.024087+0000","last_change":"2024-09-11T07:41:57.973609+0000","last_active":"2024-09-11T07:44:34.024087+0000","last_peered":"2024-09-11T07:44:34.024087+0000","last_clean":"2024-09-11T07:44:34.024087+0000","last_became_active":"2024-09-11T07:41:57.973299+0000","last_became_peered":"2024-09-11T07:41:57.973299+0000","last_unstale":"2024-09-11T07:44:34.024087+0000","last_undegraded":"2024-09-11T07:44:34.024087+0000","last_fullsized":"2024-09-11T07:44:34.024087+0000","mapping_epoch":341,"log_start":"367'7200","ondisk_log_start":"367'7200","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:41:56.678995+0000","objects_scrubbed":0,"log_size":4295,"log_dups_size":0,"ondisk_log_size":4295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":295305240,"num_objects":4507,"num_object_clones":0,"num_object_copies":13521,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4507,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11495,"num_write_kb":512001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"236'32","reported_seq":472,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.802780+0000","last_change":"2024-09-11T07:43:12.244522+0000","last_active":"2024-09-11T07:44:27.802780+0000","last_peered":"2024-09-11T07:44:27.802780+0000","last_clean":"2024-09-11T07:44:27.802780+0000","last_became_active":"2024-09-11T07:32:00.506270+0000","last_became_peered":"2024-09-11T07:32:00.506270+0000","last_unstale":"2024-09-11T07:44:27.802780+0000","last_undegraded":"2024-09-11T07:44:27.802780+0000","last_fullsized":"2024-09-11T07:44:27.802780+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'32","last_scrub_stamp":"2024-09-11T07:43:12.244424+0000","last_deep_scrub":"236'32","last_deep_scrub_stamp":"2024-09-11T07:36:50.023909+0000","last_clean_scrub_stamp":"2024-09-11T07:43:12.244424+0000","objects_scrubbed":8,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013251099000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":381,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.801153+0000","last_change":"2024-09-11T07:44:24.470679+0000","last_active":"2024-09-11T07:44:27.801153+0000","last_peered":"2024-09-11T07:44:27.801153+0000","last_clean":"2024-09-11T07:44:27.801153+0000","last_became_active":"2024-09-11T07:32:00.510631+0000","last_became_peered":"2024-09-11T07:32:00.510631+0000","last_unstale":"2024-09-11T07:44:27.801153+0000","last_undegraded":"2024-09-11T07:44:27.801153+0000","last_fullsized":"2024-09-11T07:44:27.801153+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:24.470602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:24.470602+0000","last_clean_scrub_stamp":"2024-09-11T07:44:24.470602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:40.322706+0000","scrub_duration":0.040471685,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":210,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.804949+0000","last_change":"2024-09-11T07:44:12.312130+0000","last_active":"2024-09-11T07:44:27.804949+0000","last_peered":"2024-09-11T07:44:27.804949+0000","last_clean":"2024-09-11T07:44:27.804949+0000","last_became_active":"2024-09-11T07:36:05.225840+0000","last_became_peered":"2024-09-11T07:36:05.225840+0000","last_unstale":"2024-09-11T07:44:27.804949+0000","last_undegraded":"2024-09-11T07:44:27.804949+0000","last_fullsized":"2024-09-11T07:44:27.804949+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:12.312068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:22.285763+0000","last_clean_scrub_stamp":"2024-09-11T07:44:12.312068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:35.686692+0000","scrub_duration":0.01309188,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.800145+0000","last_change":"2024-09-11T07:44:23.771617+0000","last_active":"2024-09-11T07:44:27.800145+0000","last_peered":"2024-09-11T07:44:27.800145+0000","last_clean":"2024-09-11T07:44:27.800145+0000","last_became_active":"2024-09-11T07:44:23.769063+0000","last_became_peered":"2024-09-11T07:44:23.769063+0000","last_unstale":"2024-09-11T07:44:27.800145+0000","last_undegraded":"2024-09-11T07:44:27.800145+0000","last_fullsized":"2024-09-11T07:44:27.800145+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:10.523919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:39.723598+0000","last_clean_scrub_stamp":"2024-09-11T07:43:10.523919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"382'11929","reported_seq":12243,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-09-11T07:44:34.498561+0000","last_change":"2024-09-11T07:44:11.626142+0000","last_active":"2024-09-11T07:44:34.498561+0000","last_peered":"2024-09-11T07:44:34.498561+0000","last_clean":"2024-09-11T07:44:34.498561+0000","last_became_active":"2024-09-11T07:41:57.698900+0000","last_became_peered":"2024-09-11T07:41:57.698900+0000","last_unstale":"2024-09-11T07:44:34.498561+0000","last_undegraded":"2024-09-11T07:44:34.498561+0000","last_fullsized":"2024-09-11T07:44:34.498561+0000","mapping_epoch":341,"log_start":"356'4900","ondisk_log_start":"356'4900","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"377'10198","last_scrub_stamp":"2024-09-11T07:44:11.626088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:44:11.626088+0000","objects_scrubbed":6336,"log_size":7029,"log_dups_size":0,"ondisk_log_size":7029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:27.822884+0000","scrub_duration":3.2058600770000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":294322176,"num_objects":4491,"num_object_clones":0,"num_object_copies":13473,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11929,"num_write_kb":525440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"234'20","reported_seq":406,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.794881+0000","last_change":"2024-09-11T07:43:12.364633+0000","last_active":"2024-09-11T07:44:27.794881+0000","last_peered":"2024-09-11T07:44:27.794881+0000","last_clean":"2024-09-11T07:44:27.794881+0000","last_became_active":"2024-09-11T07:39:57.692071+0000","last_became_peered":"2024-09-11T07:39:57.692071+0000","last_unstale":"2024-09-11T07:44:27.794881+0000","last_undegraded":"2024-09-11T07:44:27.794881+0000","last_fullsized":"2024-09-11T07:44:27.794881+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'20","last_scrub_stamp":"2024-09-11T07:43:12.364487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:43:12.364487+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:38.109742+0000","scrub_duration":0.0077528839999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2],"acting":[4,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":23,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.802154+0000","last_change":"2024-09-11T07:44:23.771391+0000","last_active":"2024-09-11T07:44:27.802154+0000","last_peered":"2024-09-11T07:44:27.802154+0000","last_clean":"2024-09-11T07:44:27.802154+0000","last_became_active":"2024-09-11T07:44:23.771022+0000","last_became_peered":"2024-09-11T07:44:23.771022+0000","last_unstale":"2024-09-11T07:44:27.802154+0000","last_undegraded":"2024-09-11T07:44:27.802154+0000","last_fullsized":"2024-09-11T07:44:27.802154+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:42:33.461606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:42:33.461606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"290'9","reported_seq":647,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:29.731847+0000","last_change":"2024-09-11T07:43:05.319705+0000","last_active":"2024-09-11T07:44:29.731847+0000","last_peered":"2024-09-11T07:44:29.731847+0000","last_clean":"2024-09-11T07:44:29.731847+0000","last_became_active":"2024-09-11T07:39:57.768155+0000","last_became_peered":"2024-09-11T07:39:57.768155+0000","last_unstale":"2024-09-11T07:44:29.731847+0000","last_undegraded":"2024-09-11T07:44:29.731847+0000","last_fullsized":"2024-09-11T07:44:29.731847+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"290'9","last_scrub_stamp":"2024-09-11T07:43:05.319506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:43:05.319506+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0081677780000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":407,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.800935+0000","last_change":"2024-09-11T07:43:08.315384+0000","last_active":"2024-09-11T07:44:27.800935+0000","last_peered":"2024-09-11T07:44:27.800935+0000","last_clean":"2024-09-11T07:44:27.800935+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:44:27.800935+0000","last_undegraded":"2024-09-11T07:44:27.800935+0000","last_fullsized":"2024-09-11T07:44:27.800935+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:08.315222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_clean_scrub_stamp":"2024-09-11T07:43:08.315222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:36.641725+0000","scrub_duration":0.010657583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.a","version":"382'11546","reported_seq":11862,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-09-11T07:44:34.198815+0000","last_change":"2024-09-11T07:44:07.892985+0000","last_active":"2024-09-11T07:44:34.198815+0000","last_peered":"2024-09-11T07:44:34.198815+0000","last_clean":"2024-09-11T07:44:34.198815+0000","last_became_active":"2024-09-11T07:41:57.973322+0000","last_became_peered":"2024-09-11T07:41:57.973322+0000","last_unstale":"2024-09-11T07:44:34.198815+0000","last_undegraded":"2024-09-11T07:44:34.198815+0000","last_fullsized":"2024-09-11T07:44:34.198815+0000","mapping_epoch":341,"log_start":"367'7000","ondisk_log_start":"367'7000","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'9648","last_scrub_stamp":"2024-09-11T07:44:07.892915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:44:07.892915+0000","objects_scrubbed":6441,"log_size":4546,"log_dups_size":0,"ondisk_log_size":4546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:08.645661+0000","scrub_duration":2.6293975650000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":293732352,"num_objects":4482,"num_object_clones":0,"num_object_copies":13446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11546,"num_write_kb":512896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"290'42","reported_seq":375,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:29.730898+0000","last_change":"2024-09-11T07:43:22.389378+0000","last_active":"2024-09-11T07:44:29.730898+0000","last_peered":"2024-09-11T07:44:29.730898+0000","last_clean":"2024-09-11T07:44:29.730898+0000","last_became_active":"2024-09-11T07:39:57.768267+0000","last_became_peered":"2024-09-11T07:39:57.768267+0000","last_unstale":"2024-09-11T07:44:29.730898+0000","last_undegraded":"2024-09-11T07:44:29.730898+0000","last_fullsized":"2024-09-11T07:44:29.730898+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"290'42","last_scrub_stamp":"2024-09-11T07:43:22.389278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:43:22.389278+0000","objects_scrubbed":10,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:33.624676+0000","scrub_duration":0.0085721119999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":380,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.806420+0000","last_change":"2024-09-11T07:44:23.773043+0000","last_active":"2024-09-11T07:44:27.806420+0000","last_peered":"2024-09-11T07:44:27.806420+0000","last_clean":"2024-09-11T07:44:27.806420+0000","last_became_active":"2024-09-11T07:44:23.772612+0000","last_became_peered":"2024-09-11T07:44:23.772612+0000","last_unstale":"2024-09-11T07:44:27.806420+0000","last_undegraded":"2024-09-11T07:44:27.806420+0000","last_fullsized":"2024-09-11T07:44:27.806420+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:25.444690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:51.308492+0000","last_clean_scrub_stamp":"2024-09-11T07:42:25.444690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":409,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.809743+0000","last_change":"2024-09-11T07:44:20.382255+0000","last_active":"2024-09-11T07:44:27.809743+0000","last_peered":"2024-09-11T07:44:27.809743+0000","last_clean":"2024-09-11T07:44:27.809743+0000","last_became_active":"2024-09-11T07:26:35.560363+0000","last_became_peered":"2024-09-11T07:26:35.560363+0000","last_unstale":"2024-09-11T07:44:27.809743+0000","last_undegraded":"2024-09-11T07:44:27.809743+0000","last_fullsized":"2024-09-11T07:44:27.809743+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:20.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:44:20.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:26.077834+0000","scrub_duration":0.0075336659999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":414,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.793901+0000","last_change":"2024-09-11T07:43:23.540793+0000","last_active":"2024-09-11T07:44:27.793901+0000","last_peered":"2024-09-11T07:44:27.793901+0000","last_clean":"2024-09-11T07:44:27.793901+0000","last_became_active":"2024-09-11T07:32:00.505919+0000","last_became_peered":"2024-09-11T07:32:00.505919+0000","last_unstale":"2024-09-11T07:44:27.793901+0000","last_undegraded":"2024-09-11T07:44:27.793901+0000","last_fullsized":"2024-09-11T07:44:27.793901+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:23.540701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:43:23.540701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:41.959569+0000","scrub_duration":0.0022189290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"382'11507","reported_seq":11771,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-09-11T07:44:34.128099+0000","last_change":"2024-09-11T07:44:26.161825+0000","last_active":"2024-09-11T07:44:34.128099+0000","last_peered":"2024-09-11T07:44:34.128099+0000","last_clean":"2024-09-11T07:44:34.128099+0000","last_became_active":"2024-09-11T07:41:57.696935+0000","last_became_peered":"2024-09-11T07:41:57.696935+0000","last_unstale":"2024-09-11T07:44:34.128099+0000","last_undegraded":"2024-09-11T07:44:34.128099+0000","last_fullsized":"2024-09-11T07:44:34.128099+0000","mapping_epoch":341,"log_start":"364'6113","ondisk_log_start":"364'6113","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"380'10878","last_scrub_stamp":"2024-09-11T07:44:26.161716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:44:26.161716+0000","objects_scrubbed":5131,"log_size":5394,"log_dups_size":0,"ondisk_log_size":5394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:26.179914+0000","scrub_duration":1.6972763500000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":291700736,"num_objects":4451,"num_object_clones":0,"num_object_copies":13353,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11507,"num_write_kb":510656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"234'22","reported_seq":443,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.806185+0000","last_change":"2024-09-11T07:43:20.300627+0000","last_active":"2024-09-11T07:44:27.806185+0000","last_peered":"2024-09-11T07:44:27.806185+0000","last_clean":"2024-09-11T07:44:27.806185+0000","last_became_active":"2024-09-11T07:43:10.450225+0000","last_became_peered":"2024-09-11T07:43:10.450225+0000","last_unstale":"2024-09-11T07:44:27.806185+0000","last_undegraded":"2024-09-11T07:44:27.806185+0000","last_fullsized":"2024-09-11T07:44:27.806185+0000","mapping_epoch":360,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":361,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'22","last_scrub_stamp":"2024-09-11T07:43:20.300528+0000","last_deep_scrub":"144'18","last_deep_scrub_stamp":"2024-09-11T07:31:51.603378+0000","last_clean_scrub_stamp":"2024-09-11T07:43:20.300528+0000","objects_scrubbed":5,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:43.772925+0000","scrub_duration":0.004705435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":54,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-09-11T07:44:32.864357+0000","last_change":"2024-09-11T07:44:23.769639+0000","last_active":"2024-09-11T07:44:32.864357+0000","last_peered":"2024-09-11T07:44:32.864357+0000","last_clean":"2024-09-11T07:44:32.864357+0000","last_became_active":"2024-09-11T07:44:23.769220+0000","last_became_peered":"2024-09-11T07:44:23.769220+0000","last_unstale":"2024-09-11T07:44:32.864357+0000","last_undegraded":"2024-09-11T07:44:32.864357+0000","last_fullsized":"2024-09-11T07:44:32.864357+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'5","last_scrub_stamp":"2024-09-11T07:43:15.545534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:43:15.545534+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":17,"num_read_kb":12,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":432,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.800374+0000","last_change":"2024-09-11T07:42:23.099823+0000","last_active":"2024-09-11T07:44:27.800374+0000","last_peered":"2024-09-11T07:44:27.800374+0000","last_clean":"2024-09-11T07:44:27.800374+0000","last_became_active":"2024-09-11T07:26:31.413354+0000","last_became_peered":"2024-09-11T07:26:31.413354+0000","last_unstale":"2024-09-11T07:44:27.800374+0000","last_undegraded":"2024-09-11T07:44:27.800374+0000","last_fullsized":"2024-09-11T07:44:27.800374+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:23.099709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:07.149367+0000","last_clean_scrub_stamp":"2024-09-11T07:42:23.099709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034093029999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":429,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.800690+0000","last_change":"2024-09-11T07:43:11.249241+0000","last_active":"2024-09-11T07:44:27.800690+0000","last_peered":"2024-09-11T07:44:27.800690+0000","last_clean":"2024-09-11T07:44:27.800690+0000","last_became_active":"2024-09-11T07:32:00.508398+0000","last_became_peered":"2024-09-11T07:32:00.508398+0000","last_unstale":"2024-09-11T07:44:27.800690+0000","last_undegraded":"2024-09-11T07:44:27.800690+0000","last_fullsized":"2024-09-11T07:44:27.800690+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:11.242643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:28.246601+0000","last_clean_scrub_stamp":"2024-09-11T07:43:11.242643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:36.220620+0000","scrub_duration":0.0056937530000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":407,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.800843+0000","last_change":"2024-09-11T07:44:25.382614+0000","last_active":"2024-09-11T07:44:27.800843+0000","last_peered":"2024-09-11T07:44:27.800843+0000","last_clean":"2024-09-11T07:44:27.800843+0000","last_became_active":"2024-09-11T07:32:00.515745+0000","last_became_peered":"2024-09-11T07:32:00.515745+0000","last_unstale":"2024-09-11T07:44:27.800843+0000","last_undegraded":"2024-09-11T07:44:27.800843+0000","last_fullsized":"2024-09-11T07:44:27.800843+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:25.382533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:25.382533+0000","last_clean_scrub_stamp":"2024-09-11T07:44:25.382533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:42.386047+0000","scrub_duration":0.0018079610000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"382'11843","reported_seq":12186,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-09-11T07:44:34.205857+0000","last_change":"2024-09-11T07:44:00.908110+0000","last_active":"2024-09-11T07:44:34.205857+0000","last_peered":"2024-09-11T07:44:34.205857+0000","last_clean":"2024-09-11T07:44:34.205857+0000","last_became_active":"2024-09-11T07:41:57.696712+0000","last_became_peered":"2024-09-11T07:41:57.696712+0000","last_unstale":"2024-09-11T07:44:34.205857+0000","last_undegraded":"2024-09-11T07:44:34.205857+0000","last_fullsized":"2024-09-11T07:44:34.205857+0000","mapping_epoch":341,"log_start":"367'7300","ondisk_log_start":"367'7300","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"373'9480","last_scrub_stamp":"2024-09-11T07:44:00.908017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:44:00.908017+0000","objects_scrubbed":7131,"log_size":4543,"log_dups_size":0,"ondisk_log_size":4543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:28.011815+0000","scrub_duration":2.553471906,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306118656,"num_objects":4671,"num_object_clones":0,"num_object_copies":14013,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11843,"num_write_kb":528448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"234'30","reported_seq":511,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:28.273485+0000","last_change":"2024-09-11T07:44:28.272807+0000","last_active":"2024-09-11T07:44:28.273485+0000","last_peered":"2024-09-11T07:44:28.273485+0000","last_clean":"2024-09-11T07:44:28.273485+0000","last_became_active":"2024-09-11T07:43:05.417321+0000","last_became_peered":"2024-09-11T07:43:05.417321+0000","last_unstale":"2024-09-11T07:44:28.273485+0000","last_undegraded":"2024-09-11T07:44:28.273485+0000","last_fullsized":"2024-09-11T07:44:28.273485+0000","mapping_epoch":358,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":359,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'30","last_scrub_stamp":"2024-09-11T07:44:28.272498+0000","last_deep_scrub":"94'18","last_deep_scrub_stamp":"2024-09-11T07:28:51.437397+0000","last_clean_scrub_stamp":"2024-09-11T07:44:28.272498+0000","objects_scrubbed":6,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:53.890906+0000","scrub_duration":0.006216643,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":433,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.802427+0000","last_change":"2024-09-11T07:44:02.361487+0000","last_active":"2024-09-11T07:44:27.802427+0000","last_peered":"2024-09-11T07:44:27.802427+0000","last_clean":"2024-09-11T07:44:27.802427+0000","last_became_active":"2024-09-11T07:39:57.758029+0000","last_became_peered":"2024-09-11T07:39:57.758029+0000","last_unstale":"2024-09-11T07:44:27.802427+0000","last_undegraded":"2024-09-11T07:44:27.802427+0000","last_fullsized":"2024-09-11T07:44:27.802427+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-11T07:44:02.361409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:44:02.361409+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:13.016364+0000","scrub_duration":0.010533394,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":418,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.800712+0000","last_change":"2024-09-11T07:43:56.441264+0000","last_active":"2024-09-11T07:44:27.800712+0000","last_peered":"2024-09-11T07:44:27.800712+0000","last_clean":"2024-09-11T07:44:27.800712+0000","last_became_active":"2024-09-11T07:39:57.772244+0000","last_became_peered":"2024-09-11T07:39:57.772244+0000","last_unstale":"2024-09-11T07:44:27.800712+0000","last_undegraded":"2024-09-11T07:44:27.800712+0000","last_fullsized":"2024-09-11T07:44:27.800712+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:56.441184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:16.967492+0000","last_clean_scrub_stamp":"2024-09-11T07:43:56.441184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:58.173384+0000","scrub_duration":0.0047728199999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":25,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.801248+0000","last_change":"2024-09-11T07:44:22.519600+0000","last_active":"2024-09-11T07:44:27.801248+0000","last_peered":"2024-09-11T07:44:27.801248+0000","last_clean":"2024-09-11T07:44:27.801248+0000","last_became_active":"2024-09-11T07:44:10.869822+0000","last_became_peered":"2024-09-11T07:44:10.869822+0000","last_unstale":"2024-09-11T07:44:27.801248+0000","last_undegraded":"2024-09-11T07:44:27.801248+0000","last_fullsized":"2024-09-11T07:44:27.801248+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:22.519491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:51.187127+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.519491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:32.967006+0000","scrub_duration":0.024174253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":126,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.805513+0000","last_change":"2024-09-11T07:43:58.208814+0000","last_active":"2024-09-11T07:44:27.805513+0000","last_peered":"2024-09-11T07:44:27.805513+0000","last_clean":"2024-09-11T07:44:27.805513+0000","last_became_active":"2024-09-11T07:39:57.772223+0000","last_became_peered":"2024-09-11T07:39:57.772223+0000","last_unstale":"2024-09-11T07:44:27.805513+0000","last_undegraded":"2024-09-11T07:44:27.805513+0000","last_fullsized":"2024-09-11T07:44:27.805513+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:58.208726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:43:58.208726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:21.097092+0000","scrub_duration":0.002370193,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"382'11668","reported_seq":11993,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-09-11T07:44:34.195689+0000","last_change":"2024-09-11T07:44:23.791267+0000","last_active":"2024-09-11T07:44:34.195689+0000","last_peered":"2024-09-11T07:44:34.195689+0000","last_clean":"2024-09-11T07:44:34.195689+0000","last_became_active":"2024-09-11T07:44:23.769488+0000","last_became_peered":"2024-09-11T07:44:23.769488+0000","last_unstale":"2024-09-11T07:44:34.195689+0000","last_undegraded":"2024-09-11T07:44:34.195689+0000","last_fullsized":"2024-09-11T07:44:34.195689+0000","mapping_epoch":379,"log_start":"367'7122","ondisk_log_start":"367'7122","created":341,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"377'9947","last_scrub_stamp":"2024-09-11T07:44:10.920459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:44:10.920459+0000","objects_scrubbed":6450,"log_size":4546,"log_dups_size":0,"ondisk_log_size":4546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:31.774957+0000","scrub_duration":2.802560342,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":305135616,"num_objects":4656,"num_object_clones":0,"num_object_copies":13968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11668,"num_write_kb":522368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"234'26","reported_seq":242,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.795629+0000","last_change":"2024-09-11T07:42:18.689554+0000","last_active":"2024-09-11T07:44:27.795629+0000","last_peered":"2024-09-11T07:44:27.795629+0000","last_clean":"2024-09-11T07:44:27.795629+0000","last_became_active":"2024-09-11T07:36:05.224704+0000","last_became_peered":"2024-09-11T07:36:05.224704+0000","last_unstale":"2024-09-11T07:44:27.795629+0000","last_undegraded":"2024-09-11T07:44:27.795629+0000","last_fullsized":"2024-09-11T07:44:27.795629+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'26","last_scrub_stamp":"2024-09-11T07:42:18.689474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:42:18.689474+0000","objects_scrubbed":7,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022199279999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":32,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'3","reported_seq":435,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:30.572286+0000","last_change":"2024-09-11T07:44:30.572204+0000","last_active":"2024-09-11T07:44:30.572286+0000","last_peered":"2024-09-11T07:44:30.572286+0000","last_clean":"2024-09-11T07:44:30.572286+0000","last_became_active":"2024-09-11T07:40:11.701416+0000","last_became_peered":"2024-09-11T07:40:11.701416+0000","last_unstale":"2024-09-11T07:44:30.572286+0000","last_undegraded":"2024-09-11T07:44:30.572286+0000","last_fullsized":"2024-09-11T07:44:30.572286+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-09-11T07:44:30.572101+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_clean_scrub_stamp":"2024-09-11T07:44:30.572101+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:44.821982+0000","scrub_duration":0.003584968,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":433,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.805703+0000","last_change":"2024-09-11T07:43:22.360424+0000","last_active":"2024-09-11T07:44:27.805703+0000","last_peered":"2024-09-11T07:44:27.805703+0000","last_clean":"2024-09-11T07:44:27.805703+0000","last_became_active":"2024-09-11T07:40:11.705286+0000","last_became_peered":"2024-09-11T07:40:11.705286+0000","last_unstale":"2024-09-11T07:44:27.805703+0000","last_undegraded":"2024-09-11T07:44:27.805703+0000","last_fullsized":"2024-09-11T07:44:27.805703+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:22.360187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:43:22.360187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:43.812678+0000","scrub_duration":0.046795017000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"87'81","reported_seq":250,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.801416+0000","last_change":"2024-09-11T07:44:22.387613+0000","last_active":"2024-09-11T07:44:27.801416+0000","last_peered":"2024-09-11T07:44:27.801416+0000","last_clean":"2024-09-11T07:44:27.801416+0000","last_became_active":"2024-09-11T07:39:57.761901+0000","last_became_peered":"2024-09-11T07:39:57.761901+0000","last_unstale":"2024-09-11T07:44:27.801416+0000","last_undegraded":"2024-09-11T07:44:27.801416+0000","last_fullsized":"2024-09-11T07:44:27.801416+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:44:22.387527+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.387527+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:28.175443+0000","scrub_duration":0.015232676000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":229,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.800894+0000","last_change":"2024-09-11T07:43:05.156258+0000","last_active":"2024-09-11T07:44:27.800894+0000","last_peered":"2024-09-11T07:44:27.800894+0000","last_clean":"2024-09-11T07:44:27.800894+0000","last_became_active":"2024-09-11T07:39:57.748658+0000","last_became_peered":"2024-09-11T07:39:57.748658+0000","last_unstale":"2024-09-11T07:44:27.800894+0000","last_undegraded":"2024-09-11T07:44:27.800894+0000","last_fullsized":"2024-09-11T07:44:27.800894+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:05.156156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:43:05.156156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032562279999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":400,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.805848+0000","last_change":"2024-09-11T07:43:17.268650+0000","last_active":"2024-09-11T07:44:27.805848+0000","last_peered":"2024-09-11T07:44:27.805848+0000","last_clean":"2024-09-11T07:44:27.805848+0000","last_became_active":"2024-09-11T07:26:37.559920+0000","last_became_peered":"2024-09-11T07:26:37.559920+0000","last_unstale":"2024-09-11T07:44:27.805848+0000","last_undegraded":"2024-09-11T07:44:27.805848+0000","last_fullsized":"2024-09-11T07:44:27.805848+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:17.268558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:43:17.268558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:41.996245+0000","scrub_duration":0.043293015999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"382'11748","reported_seq":12096,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-09-11T07:44:34.215001+0000","last_change":"2024-09-11T07:43:59.168117+0000","last_active":"2024-09-11T07:44:34.215001+0000","last_peered":"2024-09-11T07:44:34.215001+0000","last_clean":"2024-09-11T07:44:34.215001+0000","last_became_active":"2024-09-11T07:41:57.698691+0000","last_became_peered":"2024-09-11T07:41:57.698691+0000","last_unstale":"2024-09-11T07:44:34.215001+0000","last_undegraded":"2024-09-11T07:44:34.215001+0000","last_fullsized":"2024-09-11T07:44:34.215001+0000","mapping_epoch":341,"log_start":"367'7252","ondisk_log_start":"367'7252","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"373'9106","last_scrub_stamp":"2024-09-11T07:43:59.167871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:43:59.167871+0000","objects_scrubbed":7229,"log_size":4496,"log_dups_size":0,"ondisk_log_size":4496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:20.967083+0000","scrub_duration":3.782704351,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":290586624,"num_objects":4434,"num_object_clones":0,"num_object_copies":13302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11748,"num_write_kb":517824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"234'35","reported_seq":180,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:32.236182+0000","last_change":"2024-09-11T07:44:32.236043+0000","last_active":"2024-09-11T07:44:32.236182+0000","last_peered":"2024-09-11T07:44:32.236182+0000","last_clean":"2024-09-11T07:44:32.236182+0000","last_became_active":"2024-09-11T07:44:23.768296+0000","last_became_peered":"2024-09-11T07:44:23.768296+0000","last_unstale":"2024-09-11T07:44:32.236182+0000","last_undegraded":"2024-09-11T07:44:32.236182+0000","last_fullsized":"2024-09-11T07:44:32.236182+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'35","last_scrub_stamp":"2024-09-11T07:44:32.235984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:44:32.235984+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:58.576773+0000","scrub_duration":0.0066145329999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":425,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.800966+0000","last_change":"2024-09-11T07:42:22.091245+0000","last_active":"2024-09-11T07:44:27.800966+0000","last_peered":"2024-09-11T07:44:27.800966+0000","last_clean":"2024-09-11T07:44:27.800966+0000","last_became_active":"2024-09-11T07:26:31.419560+0000","last_became_peered":"2024-09-11T07:26:31.419560+0000","last_unstale":"2024-09-11T07:44:27.800966+0000","last_undegraded":"2024-09-11T07:44:27.800966+0000","last_fullsized":"2024-09-11T07:44:27.800966+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:22.091155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:42:22.091155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003208518,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":432,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.800845+0000","last_change":"2024-09-11T07:44:23.485237+0000","last_active":"2024-09-11T07:44:27.800845+0000","last_peered":"2024-09-11T07:44:27.800845+0000","last_clean":"2024-09-11T07:44:27.800845+0000","last_became_active":"2024-09-11T07:39:57.748953+0000","last_became_peered":"2024-09-11T07:39:57.748953+0000","last_unstale":"2024-09-11T07:44:27.800845+0000","last_undegraded":"2024-09-11T07:44:27.800845+0000","last_fullsized":"2024-09-11T07:44:27.800845+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:23.485176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:44:23.485176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:27.860899+0000","scrub_duration":0.0015945569999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":237,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.804942+0000","last_change":"2024-09-11T07:44:00.601823+0000","last_active":"2024-09-11T07:44:27.804942+0000","last_peered":"2024-09-11T07:44:27.804942+0000","last_clean":"2024-09-11T07:44:27.804942+0000","last_became_active":"2024-09-11T07:39:57.766605+0000","last_became_peered":"2024-09-11T07:39:57.766605+0000","last_unstale":"2024-09-11T07:44:27.804942+0000","last_undegraded":"2024-09-11T07:44:27.804942+0000","last_fullsized":"2024-09-11T07:44:27.804942+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:00.601321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:44:00.601321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:12.743038+0000","scrub_duration":0.009846334,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"382'11836","reported_seq":12153,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-09-11T07:44:34.130397+0000","last_change":"2024-09-11T07:44:11.212244+0000","last_active":"2024-09-11T07:44:34.130397+0000","last_peered":"2024-09-11T07:44:34.130397+0000","last_clean":"2024-09-11T07:44:34.130397+0000","last_became_active":"2024-09-11T07:41:57.696197+0000","last_became_peered":"2024-09-11T07:41:57.696197+0000","last_unstale":"2024-09-11T07:44:34.130397+0000","last_undegraded":"2024-09-11T07:44:34.130397+0000","last_fullsized":"2024-09-11T07:44:34.130397+0000","mapping_epoch":341,"log_start":"365'6471","ondisk_log_start":"365'6471","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"377'10116","last_scrub_stamp":"2024-09-11T07:44:11.212157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:44:11.212157+0000","objects_scrubbed":6441,"log_size":5365,"log_dups_size":0,"ondisk_log_size":5365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:31.231609+0000","scrub_duration":3.7851406029999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":301596672,"num_objects":4602,"num_object_clones":0,"num_object_copies":13806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11836,"num_write_kb":526016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"234'15","reported_seq":426,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.802694+0000","last_change":"2024-09-11T07:42:24.281479+0000","last_active":"2024-09-11T07:44:27.802694+0000","last_peered":"2024-09-11T07:44:27.802694+0000","last_clean":"2024-09-11T07:44:27.802694+0000","last_became_active":"2024-09-11T07:26:33.543148+0000","last_became_peered":"2024-09-11T07:26:33.543148+0000","last_unstale":"2024-09-11T07:44:27.802694+0000","last_undegraded":"2024-09-11T07:44:27.802694+0000","last_fullsized":"2024-09-11T07:44:27.802694+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:42:24.281381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:42:24.281381+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060122939999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":15,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.795817+0000","last_change":"2024-09-11T07:44:23.774581+0000","last_active":"2024-09-11T07:44:27.795817+0000","last_peered":"2024-09-11T07:44:27.795817+0000","last_clean":"2024-09-11T07:44:27.795817+0000","last_became_active":"2024-09-11T07:44:23.774250+0000","last_became_peered":"2024-09-11T07:44:23.774250+0000","last_unstale":"2024-09-11T07:44:27.795817+0000","last_undegraded":"2024-09-11T07:44:27.795817+0000","last_fullsized":"2024-09-11T07:44:27.795817+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:16.532601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:43:16.532601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:39.028131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":428,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.800873+0000","last_change":"2024-09-11T07:44:08.436638+0000","last_active":"2024-09-11T07:44:27.800873+0000","last_peered":"2024-09-11T07:44:27.800873+0000","last_clean":"2024-09-11T07:44:27.800873+0000","last_became_active":"2024-09-11T07:26:35.563182+0000","last_became_peered":"2024-09-11T07:26:35.563182+0000","last_unstale":"2024-09-11T07:44:27.800873+0000","last_undegraded":"2024-09-11T07:44:27.800873+0000","last_fullsized":"2024-09-11T07:44:27.800873+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:08.436552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:44:08.436552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:11.017974+0000","scrub_duration":0.0064625339999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":414,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.802347+0000","last_change":"2024-09-11T07:43:16.343875+0000","last_active":"2024-09-11T07:44:27.802347+0000","last_peered":"2024-09-11T07:44:27.802347+0000","last_clean":"2024-09-11T07:44:27.802347+0000","last_became_active":"2024-09-11T07:32:00.505185+0000","last_became_peered":"2024-09-11T07:32:00.505185+0000","last_unstale":"2024-09-11T07:44:27.802347+0000","last_undegraded":"2024-09-11T07:44:27.802347+0000","last_fullsized":"2024-09-11T07:44:27.802347+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:16.343786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:43:16.343786+0000","last_clean_scrub_stamp":"2024-09-11T07:43:16.343786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.083487128999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"382'11657","reported_seq":11989,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-09-11T07:44:34.188604+0000","last_change":"2024-09-11T07:44:04.182564+0000","last_active":"2024-09-11T07:44:34.188604+0000","last_peered":"2024-09-11T07:44:34.188604+0000","last_clean":"2024-09-11T07:44:34.188604+0000","last_became_active":"2024-09-11T07:42:04.728945+0000","last_became_peered":"2024-09-11T07:42:04.728945+0000","last_unstale":"2024-09-11T07:44:34.188604+0000","last_undegraded":"2024-09-11T07:44:34.188604+0000","last_fullsized":"2024-09-11T07:44:34.188604+0000","mapping_epoch":346,"log_start":"367'7100","ondisk_log_start":"367'7100","created":341,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"375'9484","last_scrub_stamp":"2024-09-11T07:44:04.182488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:44:04.182488+0000","objects_scrubbed":6854,"log_size":4557,"log_dups_size":0,"ondisk_log_size":4557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:12.533477+0000","scrub_duration":2.8141656209999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297730048,"num_objects":4543,"num_object_clones":0,"num_object_copies":13629,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11657,"num_write_kb":518400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4],"acting":[0,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"234'27","reported_seq":146,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.794289+0000","last_change":"2024-09-11T07:43:22.301033+0000","last_active":"2024-09-11T07:44:27.794289+0000","last_peered":"2024-09-11T07:44:27.794289+0000","last_clean":"2024-09-11T07:44:27.794289+0000","last_became_active":"2024-09-11T07:39:57.766550+0000","last_became_peered":"2024-09-11T07:39:57.766550+0000","last_unstale":"2024-09-11T07:44:27.794289+0000","last_undegraded":"2024-09-11T07:44:27.794289+0000","last_fullsized":"2024-09-11T07:44:27.794289+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'27","last_scrub_stamp":"2024-09-11T07:43:22.300979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:43:22.300979+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:35.812123+0000","scrub_duration":0.0017609329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":470,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.800953+0000","last_change":"2024-09-11T07:44:25.455385+0000","last_active":"2024-09-11T07:44:27.800953+0000","last_peered":"2024-09-11T07:44:27.800953+0000","last_clean":"2024-09-11T07:44:27.800953+0000","last_became_active":"2024-09-11T07:40:11.701330+0000","last_became_peered":"2024-09-11T07:40:11.701330+0000","last_unstale":"2024-09-11T07:44:27.800953+0000","last_undegraded":"2024-09-11T07:44:27.800953+0000","last_fullsized":"2024-09-11T07:44:27.800953+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:25.455028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_clean_scrub_stamp":"2024-09-11T07:44:25.455028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:31.276465+0000","scrub_duration":0.0044567610000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"151'10","reported_seq":584,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:29.729931+0000","last_change":"2024-09-11T07:43:21.597166+0000","last_active":"2024-09-11T07:44:29.729931+0000","last_peered":"2024-09-11T07:44:29.729931+0000","last_clean":"2024-09-11T07:44:29.729931+0000","last_became_active":"2024-09-11T07:32:00.506806+0000","last_became_peered":"2024-09-11T07:32:00.506806+0000","last_unstale":"2024-09-11T07:44:29.729931+0000","last_undegraded":"2024-09-11T07:44:29.729931+0000","last_fullsized":"2024-09-11T07:44:29.729931+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:43:21.597088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:43:21.597088+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:45.690910+0000","scrub_duration":0.0079746980000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":58,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.804879+0000","last_change":"2024-09-11T07:43:17.286114+0000","last_active":"2024-09-11T07:44:27.804879+0000","last_peered":"2024-09-11T07:44:27.804879+0000","last_clean":"2024-09-11T07:44:27.804879+0000","last_became_active":"2024-09-11T07:42:04.729103+0000","last_became_peered":"2024-09-11T07:42:04.729103+0000","last_unstale":"2024-09-11T07:44:27.804879+0000","last_undegraded":"2024-09-11T07:44:27.804879+0000","last_fullsized":"2024-09-11T07:44:27.804879+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:17.285919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:43:17.285919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010493728000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"382'11643","reported_seq":11907,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-09-11T07:44:34.121952+0000","last_change":"2024-09-11T07:44:28.744021+0000","last_active":"2024-09-11T07:44:34.121952+0000","last_peered":"2024-09-11T07:44:34.121952+0000","last_clean":"2024-09-11T07:44:34.121952+0000","last_became_active":"2024-09-11T07:41:57.974221+0000","last_became_peered":"2024-09-11T07:41:57.974221+0000","last_unstale":"2024-09-11T07:44:34.121952+0000","last_undegraded":"2024-09-11T07:44:34.121952+0000","last_fullsized":"2024-09-11T07:44:34.121952+0000","mapping_epoch":341,"log_start":"363'6210","ondisk_log_start":"363'6210","created":341,"last_epoch_clean":342,"parent":"0.0","parent_split_bits":0,"last_scrub":"381'11163","last_scrub_stamp":"2024-09-11T07:44:28.743943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:56.678995+0000","last_clean_scrub_stamp":"2024-09-11T07:44:28.743943+0000","objects_scrubbed":5118,"log_size":5433,"log_dups_size":0,"ondisk_log_size":5433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:30.575295+0000","scrub_duration":2.2074303899999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297992192,"num_objects":4547,"num_object_clones":0,"num_object_copies":13641,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11643,"num_write_kb":518080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"234'23","reported_seq":460,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.795452+0000","last_change":"2024-09-11T07:43:15.318622+0000","last_active":"2024-09-11T07:44:27.795452+0000","last_peered":"2024-09-11T07:44:27.795452+0000","last_clean":"2024-09-11T07:44:27.795452+0000","last_became_active":"2024-09-11T07:39:57.691713+0000","last_became_peered":"2024-09-11T07:39:57.691713+0000","last_unstale":"2024-09-11T07:44:27.795452+0000","last_undegraded":"2024-09-11T07:44:27.795452+0000","last_fullsized":"2024-09-11T07:44:27.795452+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'23","last_scrub_stamp":"2024-09-11T07:43:15.318542+0000","last_deep_scrub":"234'23","last_deep_scrub_stamp":"2024-09-11T07:37:34.613280+0000","last_clean_scrub_stamp":"2024-09-11T07:43:15.318542+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0097875570000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":429,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.795161+0000","last_change":"2024-09-11T07:44:09.114183+0000","last_active":"2024-09-11T07:44:27.795161+0000","last_peered":"2024-09-11T07:44:27.795161+0000","last_clean":"2024-09-11T07:44:27.795161+0000","last_became_active":"2024-09-11T07:32:00.504344+0000","last_became_peered":"2024-09-11T07:32:00.504344+0000","last_unstale":"2024-09-11T07:44:27.795161+0000","last_undegraded":"2024-09-11T07:44:27.795161+0000","last_fullsized":"2024-09-11T07:44:27.795161+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:41:16.742481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:41:16.742481+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043896899999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":428,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:28.572741+0000","last_change":"2024-09-11T07:44:28.572130+0000","last_active":"2024-09-11T07:44:28.572741+0000","last_peered":"2024-09-11T07:44:28.572741+0000","last_clean":"2024-09-11T07:44:28.572741+0000","last_became_active":"2024-09-11T07:26:35.561967+0000","last_became_peered":"2024-09-11T07:26:35.561967+0000","last_unstale":"2024-09-11T07:44:28.572741+0000","last_undegraded":"2024-09-11T07:44:28.572741+0000","last_fullsized":"2024-09-11T07:44:28.572741+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:28.572059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:44.820354+0000","last_clean_scrub_stamp":"2024-09-11T07:44:28.572059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:48.252572+0000","scrub_duration":0.003042159,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":434,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.792952+0000","last_change":"2024-09-11T07:44:27.626846+0000","last_active":"2024-09-11T07:44:27.792952+0000","last_peered":"2024-09-11T07:44:27.792952+0000","last_clean":"2024-09-11T07:44:27.792952+0000","last_became_active":"2024-09-11T07:39:57.776576+0000","last_became_peered":"2024-09-11T07:39:57.776576+0000","last_unstale":"2024-09-11T07:44:27.792952+0000","last_undegraded":"2024-09-11T07:44:27.792952+0000","last_fullsized":"2024-09-11T07:44:27.792952+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:27.626761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:27.626761+0000","last_clean_scrub_stamp":"2024-09-11T07:44:27.626761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:54.465574+0000","scrub_duration":0.0025844079999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"381'19","reported_seq":477,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:30.282784+0000","last_change":"2024-09-11T07:44:30.282613+0000","last_active":"2024-09-11T07:44:30.282784+0000","last_peered":"2024-09-11T07:44:30.282784+0000","last_clean":"2024-09-11T07:44:30.282784+0000","last_became_active":"2024-09-11T07:39:57.767661+0000","last_became_peered":"2024-09-11T07:39:57.767661+0000","last_unstale":"2024-09-11T07:44:30.282784+0000","last_undegraded":"2024-09-11T07:44:30.282784+0000","last_fullsized":"2024-09-11T07:44:30.282784+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"381'19","last_scrub_stamp":"2024-09-11T07:44:30.282534+0000","last_deep_scrub":"381'19","last_deep_scrub_stamp":"2024-09-11T07:44:30.282534+0000","last_clean_scrub_stamp":"2024-09-11T07:44:30.282534+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:41.064757+0000","scrub_duration":0.017437478999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":432,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.805199+0000","last_change":"2024-09-11T07:43:26.338301+0000","last_active":"2024-09-11T07:44:27.805199+0000","last_peered":"2024-09-11T07:44:27.805199+0000","last_clean":"2024-09-11T07:44:27.805199+0000","last_became_active":"2024-09-11T07:26:31.415701+0000","last_became_peered":"2024-09-11T07:26:31.415701+0000","last_unstale":"2024-09-11T07:44:27.805199+0000","last_undegraded":"2024-09-11T07:44:27.805199+0000","last_fullsized":"2024-09-11T07:44:27.805199+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:26.338227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:43:26.338227+0000","last_clean_scrub_stamp":"2024-09-11T07:43:26.338227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001220877,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":153,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:29.396144+0000","last_change":"2024-09-11T07:44:29.396082+0000","last_active":"2024-09-11T07:44:29.396144+0000","last_peered":"2024-09-11T07:44:29.396144+0000","last_clean":"2024-09-11T07:44:29.396144+0000","last_became_active":"2024-09-11T07:44:23.768703+0000","last_became_peered":"2024-09-11T07:44:23.768703+0000","last_unstale":"2024-09-11T07:44:29.396144+0000","last_undegraded":"2024-09-11T07:44:29.396144+0000","last_fullsized":"2024-09-11T07:44:29.396144+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:29.395994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:35.392360+0000","last_clean_scrub_stamp":"2024-09-11T07:44:29.395994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:46.055934+0000","scrub_duration":0.0045575399999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":425,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.800359+0000","last_change":"2024-09-11T07:43:16.338643+0000","last_active":"2024-09-11T07:44:27.800359+0000","last_peered":"2024-09-11T07:44:27.800359+0000","last_clean":"2024-09-11T07:44:27.800359+0000","last_became_active":"2024-09-11T07:32:00.506697+0000","last_became_peered":"2024-09-11T07:32:00.506697+0000","last_unstale":"2024-09-11T07:44:27.800359+0000","last_undegraded":"2024-09-11T07:44:27.800359+0000","last_fullsized":"2024-09-11T07:44:27.800359+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:43:16.338554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:43:16.338554+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002991315,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":203,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.795595+0000","last_change":"2024-09-11T07:44:15.127430+0000","last_active":"2024-09-11T07:44:27.795595+0000","last_peered":"2024-09-11T07:44:27.795595+0000","last_clean":"2024-09-11T07:44:27.795595+0000","last_became_active":"2024-09-11T07:36:05.225791+0000","last_became_peered":"2024-09-11T07:36:05.225791+0000","last_unstale":"2024-09-11T07:44:27.795595+0000","last_undegraded":"2024-09-11T07:44:27.795595+0000","last_fullsized":"2024-09-11T07:44:27.795595+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:15.127360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:44:15.127360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:38.208765+0000","scrub_duration":0.001575344,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"234'15","reported_seq":57,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.801661+0000","last_change":"2024-09-11T07:44:23.770309+0000","last_active":"2024-09-11T07:44:27.801661+0000","last_peered":"2024-09-11T07:44:27.801661+0000","last_clean":"2024-09-11T07:44:27.801661+0000","last_became_active":"2024-09-11T07:44:23.769896+0000","last_became_peered":"2024-09-11T07:44:23.769896+0000","last_unstale":"2024-09-11T07:44:27.801661+0000","last_undegraded":"2024-09-11T07:44:27.801661+0000","last_fullsized":"2024-09-11T07:44:27.801661+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:43:09.477571+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:43:09.477571+0000","objects_scrubbed":3,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":419,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.806094+0000","last_change":"2024-09-11T07:44:04.279671+0000","last_active":"2024-09-11T07:44:27.806094+0000","last_peered":"2024-09-11T07:44:27.806094+0000","last_clean":"2024-09-11T07:44:27.806094+0000","last_became_active":"2024-09-11T07:26:35.566894+0000","last_became_peered":"2024-09-11T07:26:35.566894+0000","last_unstale":"2024-09-11T07:44:27.806094+0000","last_undegraded":"2024-09-11T07:44:27.806094+0000","last_fullsized":"2024-09-11T07:44:27.806094+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:04.279588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:44:04.279588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:22.867679+0000","scrub_duration":0.0069828010000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":405,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.801294+0000","last_change":"2024-09-11T07:44:26.403265+0000","last_active":"2024-09-11T07:44:27.801294+0000","last_peered":"2024-09-11T07:44:27.801294+0000","last_clean":"2024-09-11T07:44:27.801294+0000","last_became_active":"2024-09-11T07:32:00.514100+0000","last_became_peered":"2024-09-11T07:32:00.514100+0000","last_unstale":"2024-09-11T07:44:27.801294+0000","last_undegraded":"2024-09-11T07:44:27.801294+0000","last_fullsized":"2024-09-11T07:44:27.801294+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:26.403212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:44:26.403212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:52.522883+0000","scrub_duration":0.001222218,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"234'25","reported_seq":485,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:29.610707+0000","last_change":"2024-09-11T07:44:29.610618+0000","last_active":"2024-09-11T07:44:29.610707+0000","last_peered":"2024-09-11T07:44:29.610707+0000","last_clean":"2024-09-11T07:44:29.610707+0000","last_became_active":"2024-09-11T07:39:57.771009+0000","last_became_peered":"2024-09-11T07:39:57.771009+0000","last_unstale":"2024-09-11T07:44:29.610707+0000","last_undegraded":"2024-09-11T07:44:29.610707+0000","last_fullsized":"2024-09-11T07:44:29.610707+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'25","last_scrub_stamp":"2024-09-11T07:44:29.610392+0000","last_deep_scrub":"234'25","last_deep_scrub_stamp":"2024-09-11T07:44:29.610392+0000","last_clean_scrub_stamp":"2024-09-11T07:44:29.610392+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:52.509786+0000","scrub_duration":0.005704598,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":433,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.806345+0000","last_change":"2024-09-11T07:44:01.220549+0000","last_active":"2024-09-11T07:44:27.806345+0000","last_peered":"2024-09-11T07:44:27.806345+0000","last_clean":"2024-09-11T07:44:27.806345+0000","last_became_active":"2024-09-11T07:39:57.764063+0000","last_became_peered":"2024-09-11T07:39:57.764063+0000","last_unstale":"2024-09-11T07:44:27.806345+0000","last_undegraded":"2024-09-11T07:44:27.806345+0000","last_fullsized":"2024-09-11T07:44:27.806345+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:01.220444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:39:54.493614+0000","last_clean_scrub_stamp":"2024-09-11T07:44:01.220444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:14.200749+0000","scrub_duration":0.001943183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":633,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:29.727909+0000","last_change":"2024-09-11T07:44:13.339230+0000","last_active":"2024-09-11T07:44:29.727909+0000","last_peered":"2024-09-11T07:44:29.727909+0000","last_clean":"2024-09-11T07:44:29.727909+0000","last_became_active":"2024-09-11T07:26:35.562245+0000","last_became_peered":"2024-09-11T07:26:35.562245+0000","last_unstale":"2024-09-11T07:44:29.727909+0000","last_undegraded":"2024-09-11T07:44:29.727909+0000","last_fullsized":"2024-09-11T07:44:29.727909+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:44:13.339137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:44:13.339137+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:40.240052+0000","scrub_duration":0.0023674870000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":434,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.804388+0000","last_change":"2024-09-11T07:44:03.325601+0000","last_active":"2024-09-11T07:44:27.804388+0000","last_peered":"2024-09-11T07:44:27.804388+0000","last_clean":"2024-09-11T07:44:27.804388+0000","last_became_active":"2024-09-11T07:39:57.758650+0000","last_became_peered":"2024-09-11T07:39:57.758650+0000","last_unstale":"2024-09-11T07:44:27.804388+0000","last_undegraded":"2024-09-11T07:44:27.804388+0000","last_fullsized":"2024-09-11T07:44:27.804388+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:03.324782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:44:03.324782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:22.036320+0000","scrub_duration":0.0059257859999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"234'19","reported_seq":427,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.801019+0000","last_change":"2024-09-11T07:44:00.397238+0000","last_active":"2024-09-11T07:44:27.801019+0000","last_peered":"2024-09-11T07:44:27.801019+0000","last_clean":"2024-09-11T07:44:27.801019+0000","last_became_active":"2024-09-11T07:43:10.448494+0000","last_became_peered":"2024-09-11T07:43:10.448494+0000","last_unstale":"2024-09-11T07:44:27.801019+0000","last_undegraded":"2024-09-11T07:44:27.801019+0000","last_fullsized":"2024-09-11T07:44:27.801019+0000","mapping_epoch":360,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":361,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:44:00.397157+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:37:41.338036+0000","last_clean_scrub_stamp":"2024-09-11T07:44:00.397157+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:14.009003+0000","scrub_duration":0.0072446960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":406,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.802929+0000","last_change":"2024-09-11T07:42:30.415501+0000","last_active":"2024-09-11T07:44:27.802929+0000","last_peered":"2024-09-11T07:44:27.802929+0000","last_clean":"2024-09-11T07:44:27.802929+0000","last_became_active":"2024-09-11T07:26:31.419409+0000","last_became_peered":"2024-09-11T07:26:31.419409+0000","last_unstale":"2024-09-11T07:44:27.802929+0000","last_undegraded":"2024-09-11T07:44:27.802929+0000","last_fullsized":"2024-09-11T07:44:27.802929+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:30.415425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:42:30.415425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061185789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"290'11","reported_seq":375,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:29.730074+0000","last_change":"2024-09-11T07:44:10.410699+0000","last_active":"2024-09-11T07:44:29.730074+0000","last_peered":"2024-09-11T07:44:29.730074+0000","last_clean":"2024-09-11T07:44:29.730074+0000","last_became_active":"2024-09-11T07:39:57.772790+0000","last_became_peered":"2024-09-11T07:39:57.772790+0000","last_unstale":"2024-09-11T07:44:29.730074+0000","last_undegraded":"2024-09-11T07:44:29.730074+0000","last_fullsized":"2024-09-11T07:44:29.730074+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"290'11","last_scrub_stamp":"2024-09-11T07:44:10.410600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:44:10.410600+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:38.281690+0000","scrub_duration":0.0090484429999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":390,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.794743+0000","last_change":"2024-09-11T07:42:05.675208+0000","last_active":"2024-09-11T07:44:27.794743+0000","last_peered":"2024-09-11T07:44:27.794743+0000","last_clean":"2024-09-11T07:44:27.794743+0000","last_became_active":"2024-09-11T07:26:37.565723+0000","last_became_peered":"2024-09-11T07:26:37.565723+0000","last_unstale":"2024-09-11T07:44:27.794743+0000","last_undegraded":"2024-09-11T07:44:27.794743+0000","last_fullsized":"2024-09-11T07:44:27.794743+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:05.675136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:42:05.675136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014991150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"236'16","reported_seq":405,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.800954+0000","last_change":"2024-09-11T07:43:09.251084+0000","last_active":"2024-09-11T07:44:27.800954+0000","last_peered":"2024-09-11T07:44:27.800954+0000","last_clean":"2024-09-11T07:44:27.800954+0000","last_became_active":"2024-09-11T07:39:57.689424+0000","last_became_peered":"2024-09-11T07:39:57.689424+0000","last_unstale":"2024-09-11T07:44:27.800954+0000","last_undegraded":"2024-09-11T07:44:27.800954+0000","last_fullsized":"2024-09-11T07:44:27.800954+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'16","last_scrub_stamp":"2024-09-11T07:43:09.250993+0000","last_deep_scrub":"236'16","last_deep_scrub_stamp":"2024-09-11T07:36:49.809484+0000","last_clean_scrub_stamp":"2024-09-11T07:43:09.250993+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013487453999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":209,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.795016+0000","last_change":"2024-09-11T07:43:15.385301+0000","last_active":"2024-09-11T07:44:27.795016+0000","last_peered":"2024-09-11T07:44:27.795016+0000","last_clean":"2024-09-11T07:44:27.795016+0000","last_became_active":"2024-09-11T07:39:57.776841+0000","last_became_peered":"2024-09-11T07:39:57.776841+0000","last_unstale":"2024-09-11T07:44:27.795016+0000","last_undegraded":"2024-09-11T07:44:27.795016+0000","last_fullsized":"2024-09-11T07:44:27.795016+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:15.385106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:38.721118+0000","last_clean_scrub_stamp":"2024-09-11T07:43:15.385106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:34.858608+0000","scrub_duration":0.0024763319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":405,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.793027+0000","last_change":"2024-09-11T07:44:15.612662+0000","last_active":"2024-09-11T07:44:27.793027+0000","last_peered":"2024-09-11T07:44:27.793027+0000","last_clean":"2024-09-11T07:44:27.793027+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:44:27.793027+0000","last_undegraded":"2024-09-11T07:44:27.793027+0000","last_fullsized":"2024-09-11T07:44:27.793027+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:15.612580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:44:15.612580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:25.693520+0000","scrub_duration":0.001412515,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":417,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.800855+0000","last_change":"2024-09-11T07:43:18.311031+0000","last_active":"2024-09-11T07:44:27.800855+0000","last_peered":"2024-09-11T07:44:27.800855+0000","last_clean":"2024-09-11T07:44:27.800855+0000","last_became_active":"2024-09-11T07:39:57.761754+0000","last_became_peered":"2024-09-11T07:39:57.761754+0000","last_unstale":"2024-09-11T07:44:27.800855+0000","last_undegraded":"2024-09-11T07:44:27.800855+0000","last_fullsized":"2024-09-11T07:44:27.800855+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:18.310936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:43:18.310936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:41.754185+0000","scrub_duration":0.0094815590000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"234'24","reported_seq":440,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:32.228039+0000","last_change":"2024-09-11T07:44:32.227965+0000","last_active":"2024-09-11T07:44:32.228039+0000","last_peered":"2024-09-11T07:44:32.228039+0000","last_clean":"2024-09-11T07:44:32.228039+0000","last_became_active":"2024-09-11T07:26:33.540208+0000","last_became_peered":"2024-09-11T07:26:33.540208+0000","last_unstale":"2024-09-11T07:44:32.228039+0000","last_undegraded":"2024-09-11T07:44:32.228039+0000","last_fullsized":"2024-09-11T07:44:32.228039+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'24","last_scrub_stamp":"2024-09-11T07:44:32.227871+0000","last_deep_scrub":"234'24","last_deep_scrub_stamp":"2024-09-11T07:41:32.694510+0000","last_clean_scrub_stamp":"2024-09-11T07:44:32.227871+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:51.004601+0000","scrub_duration":0.0040623300000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":15,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.806010+0000","last_change":"2024-09-11T07:44:23.772996+0000","last_active":"2024-09-11T07:44:27.806010+0000","last_peered":"2024-09-11T07:44:27.806010+0000","last_clean":"2024-09-11T07:44:27.806010+0000","last_became_active":"2024-09-11T07:44:23.772554+0000","last_became_peered":"2024-09-11T07:44:23.772554+0000","last_unstale":"2024-09-11T07:44:27.806010+0000","last_undegraded":"2024-09-11T07:44:27.806010+0000","last_fullsized":"2024-09-11T07:44:27.806010+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:18.470914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:42:18.470914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":304,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.805911+0000","last_change":"2024-09-11T07:44:23.768705+0000","last_active":"2024-09-11T07:44:27.805911+0000","last_peered":"2024-09-11T07:44:27.805911+0000","last_clean":"2024-09-11T07:44:27.805911+0000","last_became_active":"2024-09-11T07:44:23.768376+0000","last_became_peered":"2024-09-11T07:44:23.768376+0000","last_unstale":"2024-09-11T07:44:27.805911+0000","last_undegraded":"2024-09-11T07:44:27.805911+0000","last_fullsized":"2024-09-11T07:44:27.805911+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:19.496507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:39:55.895853+0000","last_clean_scrub_stamp":"2024-09-11T07:42:19.496507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011708260999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":136,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.804711+0000","last_change":"2024-09-11T07:43:06.316844+0000","last_active":"2024-09-11T07:44:27.804711+0000","last_peered":"2024-09-11T07:44:27.804711+0000","last_clean":"2024-09-11T07:44:27.804711+0000","last_became_active":"2024-09-11T07:40:11.700676+0000","last_became_peered":"2024-09-11T07:40:11.700676+0000","last_unstale":"2024-09-11T07:44:27.804711+0000","last_undegraded":"2024-09-11T07:44:27.804711+0000","last_fullsized":"2024-09-11T07:44:27.804711+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:06.316756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:43:06.316756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0096085179999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"234'10","reported_seq":425,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.793187+0000","last_change":"2024-09-11T07:44:16.603894+0000","last_active":"2024-09-11T07:44:27.793187+0000","last_peered":"2024-09-11T07:44:27.793187+0000","last_clean":"2024-09-11T07:44:27.793187+0000","last_became_active":"2024-09-11T07:26:33.545830+0000","last_became_peered":"2024-09-11T07:26:33.545830+0000","last_unstale":"2024-09-11T07:44:27.793187+0000","last_undegraded":"2024-09-11T07:44:27.793187+0000","last_fullsized":"2024-09-11T07:44:27.793187+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'10","last_scrub_stamp":"2024-09-11T07:44:16.603800+0000","last_deep_scrub":"234'10","last_deep_scrub_stamp":"2024-09-11T07:41:29.585284+0000","last_clean_scrub_stamp":"2024-09-11T07:44:16.603800+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:29.777779+0000","scrub_duration":0.0047267960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":413,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.806163+0000","last_change":"2024-09-11T07:43:55.188214+0000","last_active":"2024-09-11T07:44:27.806163+0000","last_peered":"2024-09-11T07:44:27.806163+0000","last_clean":"2024-09-11T07:44:27.806163+0000","last_became_active":"2024-09-11T07:26:31.420518+0000","last_became_peered":"2024-09-11T07:26:31.420518+0000","last_unstale":"2024-09-11T07:44:27.806163+0000","last_undegraded":"2024-09-11T07:44:27.806163+0000","last_fullsized":"2024-09-11T07:44:27.806163+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:55.185252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_clean_scrub_stamp":"2024-09-11T07:43:55.185252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:00.047910+0000","scrub_duration":0.0055867790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":334,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.795305+0000","last_change":"2024-09-11T07:42:24.675003+0000","last_active":"2024-09-11T07:44:27.795305+0000","last_peered":"2024-09-11T07:44:27.795305+0000","last_clean":"2024-09-11T07:44:27.795305+0000","last_became_active":"2024-09-11T07:40:11.699702+0000","last_became_peered":"2024-09-11T07:40:11.699702+0000","last_unstale":"2024-09-11T07:44:27.795305+0000","last_undegraded":"2024-09-11T07:44:27.795305+0000","last_fullsized":"2024-09-11T07:44:27.795305+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:24.674952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:47.675050+0000","last_clean_scrub_stamp":"2024-09-11T07:42:24.674952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001252188,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":407,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.800626+0000","last_change":"2024-09-11T07:44:13.460998+0000","last_active":"2024-09-11T07:44:27.800626+0000","last_peered":"2024-09-11T07:44:27.800626+0000","last_clean":"2024-09-11T07:44:27.800626+0000","last_became_active":"2024-09-11T07:32:00.511407+0000","last_became_peered":"2024-09-11T07:32:00.511407+0000","last_unstale":"2024-09-11T07:44:27.800626+0000","last_undegraded":"2024-09-11T07:44:27.800626+0000","last_fullsized":"2024-09-11T07:44:27.800626+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:13.460946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:44:13.460946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:41.442372+0000","scrub_duration":0.0014486880000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"234'19","reported_seq":461,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.792442+0000","last_change":"2024-09-11T07:44:03.649070+0000","last_active":"2024-09-11T07:44:27.792442+0000","last_peered":"2024-09-11T07:44:27.792442+0000","last_clean":"2024-09-11T07:44:27.792442+0000","last_became_active":"2024-09-11T07:39:57.767928+0000","last_became_peered":"2024-09-11T07:39:57.767928+0000","last_unstale":"2024-09-11T07:44:27.792442+0000","last_undegraded":"2024-09-11T07:44:27.792442+0000","last_fullsized":"2024-09-11T07:44:27.792442+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:44:03.648975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:44:03.648975+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:29.019437+0000","scrub_duration":0.003981989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":199,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.806244+0000","last_change":"2024-09-11T07:44:11.315656+0000","last_active":"2024-09-11T07:44:27.806244+0000","last_peered":"2024-09-11T07:44:27.806244+0000","last_clean":"2024-09-11T07:44:27.806244+0000","last_became_active":"2024-09-11T07:36:05.228360+0000","last_became_peered":"2024-09-11T07:36:05.228360+0000","last_unstale":"2024-09-11T07:44:27.806244+0000","last_undegraded":"2024-09-11T07:44:27.806244+0000","last_fullsized":"2024-09-11T07:44:27.806244+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:11.315570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:48.366931+0000","last_clean_scrub_stamp":"2024-09-11T07:44:11.315570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:25.419714+0000","scrub_duration":0.0063871090000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":380,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.800490+0000","last_change":"2024-09-11T07:44:20.480008+0000","last_active":"2024-09-11T07:44:27.800490+0000","last_peered":"2024-09-11T07:44:27.800490+0000","last_clean":"2024-09-11T07:44:27.800490+0000","last_became_active":"2024-09-11T07:32:00.520534+0000","last_became_peered":"2024-09-11T07:32:00.520534+0000","last_unstale":"2024-09-11T07:44:27.800490+0000","last_undegraded":"2024-09-11T07:44:27.800490+0000","last_fullsized":"2024-09-11T07:44:27.800490+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:20.479924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:39.021876+0000","last_clean_scrub_stamp":"2024-09-11T07:44:20.479924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:39.051860+0000","scrub_duration":0.001690261,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":400,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.792873+0000","last_change":"2024-09-11T07:44:17.592717+0000","last_active":"2024-09-11T07:44:27.792873+0000","last_peered":"2024-09-11T07:44:27.792873+0000","last_clean":"2024-09-11T07:44:27.792873+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:44:27.792873+0000","last_undegraded":"2024-09-11T07:44:27.792873+0000","last_fullsized":"2024-09-11T07:44:27.792873+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:17.592633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:32.904006+0000","last_clean_scrub_stamp":"2024-09-11T07:44:17.592633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:30.866975+0000","scrub_duration":0.0044439140000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":377,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.800906+0000","last_change":"2024-09-11T07:43:07.677815+0000","last_active":"2024-09-11T07:44:27.800906+0000","last_peered":"2024-09-11T07:44:27.800906+0000","last_clean":"2024-09-11T07:44:27.800906+0000","last_became_active":"2024-09-11T07:32:00.511258+0000","last_became_peered":"2024-09-11T07:32:00.511258+0000","last_unstale":"2024-09-11T07:44:27.800906+0000","last_undegraded":"2024-09-11T07:44:27.800906+0000","last_fullsized":"2024-09-11T07:44:27.800906+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:43:07.675565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:43:07.675565+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:34.471182+0000","scrub_duration":0.012571676,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":449,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.805479+0000","last_change":"2024-09-11T07:44:22.301052+0000","last_active":"2024-09-11T07:44:27.805479+0000","last_peered":"2024-09-11T07:44:27.805479+0000","last_clean":"2024-09-11T07:44:27.805479+0000","last_became_active":"2024-09-11T07:39:57.696805+0000","last_became_peered":"2024-09-11T07:39:57.696805+0000","last_unstale":"2024-09-11T07:44:27.805479+0000","last_undegraded":"2024-09-11T07:44:27.805479+0000","last_fullsized":"2024-09-11T07:44:27.805479+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:44:22.300957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.300957+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:45.255902+0000","scrub_duration":0.0068790429999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"234'20","reported_seq":234,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.795768+0000","last_change":"2024-09-11T07:42:12.618989+0000","last_active":"2024-09-11T07:44:27.795768+0000","last_peered":"2024-09-11T07:44:27.795768+0000","last_clean":"2024-09-11T07:44:27.795768+0000","last_became_active":"2024-09-11T07:35:00.561615+0000","last_became_peered":"2024-09-11T07:35:00.561615+0000","last_unstale":"2024-09-11T07:44:27.795768+0000","last_undegraded":"2024-09-11T07:44:27.795768+0000","last_fullsized":"2024-09-11T07:44:27.795768+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'20","last_scrub_stamp":"2024-09-11T07:42:12.618921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:42:12.618921+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002152784,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1f","version":"290'12","reported_seq":329,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:29.729593+0000","last_change":"2024-09-11T07:42:23.666873+0000","last_active":"2024-09-11T07:44:29.729593+0000","last_peered":"2024-09-11T07:44:29.729593+0000","last_clean":"2024-09-11T07:44:29.729593+0000","last_became_active":"2024-09-11T07:39:57.687809+0000","last_became_peered":"2024-09-11T07:39:57.687809+0000","last_unstale":"2024-09-11T07:44:29.729593+0000","last_undegraded":"2024-09-11T07:44:29.729593+0000","last_fullsized":"2024-09-11T07:44:29.729593+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"290'12","last_scrub_stamp":"2024-09-11T07:42:23.666786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:42:23.666786+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043065259999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":25,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:28.236681+0000","last_change":"2024-09-11T07:44:28.236628+0000","last_active":"2024-09-11T07:44:28.236681+0000","last_peered":"2024-09-11T07:44:28.236681+0000","last_clean":"2024-09-11T07:44:28.236681+0000","last_became_active":"2024-09-11T07:44:23.772800+0000","last_became_peered":"2024-09-11T07:44:23.772800+0000","last_unstale":"2024-09-11T07:44:28.236681+0000","last_undegraded":"2024-09-11T07:44:28.236681+0000","last_fullsized":"2024-09-11T07:44:28.236681+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:28.236557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:53.875300+0000","last_clean_scrub_stamp":"2024-09-11T07:44:28.236557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:31.845132+0000","scrub_duration":0.001550165,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":416,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.792675+0000","last_change":"2024-09-11T07:44:01.714689+0000","last_active":"2024-09-11T07:44:27.792675+0000","last_peered":"2024-09-11T07:44:27.792675+0000","last_clean":"2024-09-11T07:44:27.792675+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:44:27.792675+0000","last_undegraded":"2024-09-11T07:44:27.792675+0000","last_fullsized":"2024-09-11T07:44:27.792675+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:01.714607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:44:01.714607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:10.246305+0000","scrub_duration":0.0079815600000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"234'19","reported_seq":78,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.800065+0000","last_change":"2024-09-11T07:44:23.774242+0000","last_active":"2024-09-11T07:44:27.800065+0000","last_peered":"2024-09-11T07:44:27.800065+0000","last_clean":"2024-09-11T07:44:27.800065+0000","last_became_active":"2024-09-11T07:44:23.773794+0000","last_became_peered":"2024-09-11T07:44:23.773794+0000","last_unstale":"2024-09-11T07:44:27.800065+0000","last_undegraded":"2024-09-11T07:44:27.800065+0000","last_fullsized":"2024-09-11T07:44:27.800065+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:44:12.144423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:44:12.144423+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:23.188701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":416,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.804477+0000","last_change":"2024-09-11T07:43:24.330744+0000","last_active":"2024-09-11T07:44:27.804477+0000","last_peered":"2024-09-11T07:44:27.804477+0000","last_clean":"2024-09-11T07:44:27.804477+0000","last_became_active":"2024-09-11T07:32:00.507050+0000","last_became_peered":"2024-09-11T07:32:00.507050+0000","last_unstale":"2024-09-11T07:44:27.804477+0000","last_undegraded":"2024-09-11T07:44:27.804477+0000","last_fullsized":"2024-09-11T07:44:27.804477+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:24.330520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:24.575210+0000","last_clean_scrub_stamp":"2024-09-11T07:43:24.330520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:35.312624+0000","scrub_duration":0.0031611009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":400,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.795919+0000","last_change":"2024-09-11T07:42:33.597664+0000","last_active":"2024-09-11T07:44:27.795919+0000","last_peered":"2024-09-11T07:44:27.795919+0000","last_clean":"2024-09-11T07:44:27.795919+0000","last_became_active":"2024-09-11T07:26:37.862373+0000","last_became_peered":"2024-09-11T07:26:37.862373+0000","last_unstale":"2024-09-11T07:44:27.795919+0000","last_undegraded":"2024-09-11T07:44:27.795919+0000","last_fullsized":"2024-09-11T07:44:27.795919+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:33.597580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:42:33.597580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002999842,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":338,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.794807+0000","last_change":"2024-09-11T07:43:08.381231+0000","last_active":"2024-09-11T07:44:27.794807+0000","last_peered":"2024-09-11T07:44:27.794807+0000","last_clean":"2024-09-11T07:44:27.794807+0000","last_became_active":"2024-09-11T07:36:05.224085+0000","last_became_peered":"2024-09-11T07:36:05.224085+0000","last_unstale":"2024-09-11T07:44:27.794807+0000","last_undegraded":"2024-09-11T07:44:27.794807+0000","last_fullsized":"2024-09-11T07:44:27.794807+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:08.381158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:43:08.381158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01040922,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"234'27","reported_seq":460,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.800695+0000","last_change":"2024-09-11T07:42:25.209195+0000","last_active":"2024-09-11T07:44:27.800695+0000","last_peered":"2024-09-11T07:44:27.800695+0000","last_clean":"2024-09-11T07:44:27.800695+0000","last_became_active":"2024-09-11T07:26:33.542544+0000","last_became_peered":"2024-09-11T07:26:33.542544+0000","last_unstale":"2024-09-11T07:44:27.800695+0000","last_undegraded":"2024-09-11T07:44:27.800695+0000","last_fullsized":"2024-09-11T07:44:27.800695+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'27","last_scrub_stamp":"2024-09-11T07:42:25.209063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:42:25.209063+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010336984,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.795503+0000","last_change":"2024-09-11T07:44:23.774755+0000","last_active":"2024-09-11T07:44:27.795503+0000","last_peered":"2024-09-11T07:44:27.795503+0000","last_clean":"2024-09-11T07:44:27.795503+0000","last_became_active":"2024-09-11T07:44:23.774452+0000","last_became_peered":"2024-09-11T07:44:23.774452+0000","last_unstale":"2024-09-11T07:44:27.795503+0000","last_undegraded":"2024-09-11T07:44:27.795503+0000","last_fullsized":"2024-09-11T07:44:27.795503+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:32.456852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:32.456852+0000","last_clean_scrub_stamp":"2024-09-11T07:42:32.456852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":407,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.794416+0000","last_change":"2024-09-11T07:44:00.677726+0000","last_active":"2024-09-11T07:44:27.794416+0000","last_peered":"2024-09-11T07:44:27.794416+0000","last_clean":"2024-09-11T07:44:27.794416+0000","last_became_active":"2024-09-11T07:39:57.768994+0000","last_became_peered":"2024-09-11T07:39:57.768994+0000","last_unstale":"2024-09-11T07:44:27.794416+0000","last_undegraded":"2024-09-11T07:44:27.794416+0000","last_fullsized":"2024-09-11T07:44:27.794416+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:00.677444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:44:00.677444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:22.289331+0000","scrub_duration":0.0030734109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":430,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.793144+0000","last_change":"2024-09-11T07:43:55.705340+0000","last_active":"2024-09-11T07:44:27.793144+0000","last_peered":"2024-09-11T07:44:27.793144+0000","last_clean":"2024-09-11T07:44:27.793144+0000","last_became_active":"2024-09-11T07:39:57.766915+0000","last_became_peered":"2024-09-11T07:39:57.766915+0000","last_unstale":"2024-09-11T07:44:27.793144+0000","last_undegraded":"2024-09-11T07:44:27.793144+0000","last_fullsized":"2024-09-11T07:44:27.793144+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:55.705240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:43:55.705240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:14.405511+0000","scrub_duration":0.0048801449999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"234'23","reported_seq":438,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.794164+0000","last_change":"2024-09-11T07:43:25.530935+0000","last_active":"2024-09-11T07:44:27.794164+0000","last_peered":"2024-09-11T07:44:27.794164+0000","last_clean":"2024-09-11T07:44:27.794164+0000","last_became_active":"2024-09-11T07:26:33.542711+0000","last_became_peered":"2024-09-11T07:26:33.542711+0000","last_unstale":"2024-09-11T07:44:27.794164+0000","last_undegraded":"2024-09-11T07:44:27.794164+0000","last_fullsized":"2024-09-11T07:44:27.794164+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'23","last_scrub_stamp":"2024-09-11T07:43:25.530826+0000","last_deep_scrub":"234'23","last_deep_scrub_stamp":"2024-09-11T07:43:25.530826+0000","last_clean_scrub_stamp":"2024-09-11T07:43:25.530826+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:44:51.522413+0000","scrub_duration":0.0052726639999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":405,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-09-11T07:44:27.796940+0000","last_change":"2024-09-11T07:44:17.114372+0000","last_active":"2024-09-11T07:44:27.796940+0000","last_peered":"2024-09-11T07:44:27.796940+0000","last_clean":"2024-09-11T07:44:27.796940+0000","last_became_active":"2024-09-11T07:26:35.562037+0000","last_became_peered":"2024-09-11T07:26:35.562037+0000","last_unstale":"2024-09-11T07:44:27.796940+0000","last_undegraded":"2024-09-11T07:44:27.796940+0000","last_fullsized":"2024-09-11T07:44:27.796940+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:17.114312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:44:17.114312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:20.091839+0000","scrub_duration":0.00559912,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":4738842648,"num_objects":72310,"num_object_clones":0,"num_object_copies":216930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72310,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":186538,"num_write_kb":8283073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":14199894016,"data_stored":14199881800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79015,"ondisk_log_size":79015,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1116,"num_read_kb":873,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":3356,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":769,"ondisk_log_size":769,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":12,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":147,"seq":631360192664,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2078668,"kb_used_data":1155076,"kb_used_omap":4,"kb_used_meta":923515,"kb_avail":91666484,"statfs":{"total":95995035648,"available":93866479616,"internally_reserved":0,"allocated":1182797824,"data_stored":1182412702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":945679878},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:44:32 2024","interfaces":[{"interface":"back","average":{"1min":3.0430000000000001,"5min":2.1960000000000002,"15min":2.234},"min":{"1min":0.21099999999999999,"5min":0.193,"15min":0.191},"max":{"1min":20.832999999999998,"5min":75.677000000000007,"15min":75.677000000000007},"last":0.49399999999999999},{"interface":"front","average":{"1min":2.988,"5min":2.1659999999999999,"15min":2.2330000000000001},"min":{"1min":0.25700000000000001,"5min":0.22,"15min":0.22},"max":{"1min":20.937000000000001,"5min":75.638999999999996,"15min":75.638999999999996},"last":1.821}]},{"osd":1,"last update":"Wed Sep 11 07:44:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":1.5840000000000001,"15min":1.1040000000000001},"min":{"1min":0.26300000000000001,"5min":0.247,"15min":0.22500000000000001},"max":{"1min":1.7889999999999999,"5min":75.790000000000006,"15min":75.790000000000006},"last":2.681},{"interface":"front","average":{"1min":0.66800000000000004,"5min":1.591,"15min":1.1950000000000001},"min":{"1min":0.27800000000000002,"5min":0.25600000000000001,"15min":0.22800000000000001},"max":{"1min":2.2629999999999999,"5min":76.363,"15min":76.363},"last":0.34499999999999997}]},{"osd":2,"last update":"Wed Sep 11 07:44:32 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":0.95299999999999996,"15min":1.8169999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.22800000000000001},"max":{"1min":6.4820000000000002,"5min":7.0289999999999999,"15min":227.79300000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":0.91900000000000004,"15min":1.8460000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.246},"max":{"1min":6.6600000000000001,"5min":6.6600000000000001,"15min":227.22},"last":0.40300000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:44:32 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.94799999999999995,"15min":1.7450000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.9610000000000001,"5min":4.6349999999999998,"15min":49.798999999999999},"last":2.5750000000000002},{"interface":"front","average":{"1min":0.622,"5min":1.016,"15min":1.7629999999999999},"min":{"1min":0.316,"5min":0.28899999999999998,"15min":0.27200000000000002},"max":{"1min":2.6829999999999998,"5min":7.3369999999999997,"15min":49.524000000000001},"last":2.2999999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:44:32 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.80100000000000005,"15min":0.66200000000000003},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.97099999999999997,"5min":3.9399999999999999,"15min":6.0220000000000002},"last":1.496},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.874,"15min":0.72399999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":2.355,"5min":6.9000000000000004,"15min":6.9000000000000004},"last":1.607}]},{"osd":5,"last update":"Wed Sep 11 07:44:32 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.91100000000000003,"15min":0.76500000000000001},"min":{"1min":0.27600000000000002,"5min":0.25,"15min":0.25},"max":{"1min":1.2370000000000001,"5min":9.9339999999999993,"15min":9.9339999999999993},"last":1.1759999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.83299999999999996,"15min":0.78400000000000003},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.8660000000000001,"5min":10.183999999999999,"15min":10.183999999999999},"last":0.317}]},{"osd":6,"last update":"Wed Sep 11 07:44:32 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.73199999999999998,"15min":0.65100000000000002},"min":{"1min":0.30399999999999999,"5min":0.29499999999999998,"15min":0.29099999999999998},"max":{"1min":1.216,"5min":5.1909999999999998,"15min":5.1909999999999998},"last":0.37},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.76100000000000001,"15min":0.71799999999999997},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.028,"5min":4.2030000000000003,"15min":4.2030000000000003},"last":1.1439999999999999}]}]},{"osd":6,"up_from":44,"seq":188978561249,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2914856,"kb_used_data":2328928,"kb_used_omap":1,"kb_used_meta":585726,"kb_avail":90830296,"statfs":{"total":95995035648,"available":93010223104,"internally_reserved":0,"allocated":2384822272,"data_stored":2384410180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":599783768},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:44:26 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":2.0449999999999999,"15min":2.319},"min":{"1min":0.32000000000000001,"5min":0.219,"15min":0.192},"max":{"1min":3.8319999999999999,"5min":130.62299999999999,"15min":142.29599999999999},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.135,"5min":2.2930000000000001,"15min":2.5129999999999999},"min":{"1min":0.35399999999999998,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":4.6950000000000003,"5min":130.86199999999999,"15min":142.68799999999999},"last":0.33900000000000002}]},{"osd":1,"last update":"Wed Sep 11 07:44:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":3.2570000000000001,"15min":2.3980000000000001},"min":{"1min":0.32100000000000001,"5min":0.246,"15min":0.23599999999999999},"max":{"1min":4.0659999999999998,"5min":131.42500000000001,"15min":131.42500000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":1.1459999999999999,"5min":3.149,"15min":2.4009999999999998},"min":{"1min":0.27500000000000002,"5min":0.22,"15min":0.22},"max":{"1min":3.71,"5min":132.39599999999999,"15min":132.39599999999999},"last":0.47099999999999997}]},{"osd":2,"last update":"Wed Sep 11 07:44:26 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.2190000000000001,"15min":1.0669999999999999},"min":{"1min":0.27600000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":7.7779999999999996,"5min":15.798999999999999,"15min":26.491},"last":0.52300000000000002},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.0329999999999999,"15min":1.1870000000000001},"min":{"1min":0.28699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":4.0279999999999996,"5min":6.6559999999999997,"15min":26.561},"last":1.46}]},{"osd":3,"last update":"Wed Sep 11 07:44:26 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":1.4239999999999999,"15min":1.712},"min":{"1min":0.308,"5min":0.246,"15min":0.246},"max":{"1min":3.8799999999999999,"5min":39.308999999999997,"15min":49.109000000000002},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.544,"15min":1.8029999999999999},"min":{"1min":0.38100000000000001,"5min":0.246,"15min":0.246},"max":{"1min":4.6429999999999998,"5min":41.149999999999999,"15min":48.963999999999999},"last":0.44}]},{"osd":4,"last update":"Wed Sep 11 07:44:26 2024","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":3.8220000000000001,"15min":1.653},"min":{"1min":0.39500000000000002,"5min":0.23899999999999999,"15min":0.223},"max":{"1min":6.7389999999999999,"5min":181.78800000000001,"15min":181.78800000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":1.048,"5min":2.222,"15min":1.099},"min":{"1min":0.25,"5min":0.25,"15min":0.24399999999999999},"max":{"1min":3.7389999999999999,"5min":128.59299999999999,"15min":128.59299999999999},"last":0.499}]},{"osd":5,"last update":"Wed Sep 11 07:44:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.095,"15min":1.014},"min":{"1min":0.29999999999999999,"5min":0.246,"15min":0.246},"max":{"1min":4.6219999999999999,"5min":10.935,"15min":57.436},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.171,"5min":1.038,"15min":1.01},"min":{"1min":0.35399999999999998,"5min":0.253,"15min":0.251},"max":{"1min":4.1319999999999997,"5min":11.15,"15min":56.822000000000003},"last":0.27500000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:44:26 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.135,"15min":0.84699999999999998},"min":{"1min":0.33000000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":4.0910000000000002,"5min":11.121,"15min":11.121},"last":0.32300000000000001},{"interface":"front","average":{"1min":1.125,"5min":1.0960000000000001,"15min":0.86299999999999999},"min":{"1min":0.40999999999999998,"5min":0.215,"15min":0.215},"max":{"1min":4.6719999999999997,"5min":10.962999999999999,"15min":10.962999999999999},"last":0.372}]}]},{"osd":5,"up_from":37,"seq":158913790183,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1775948,"kb_used_data":875584,"kb_used_omap":1,"kb_used_meta":900350,"kb_avail":91969204,"statfs":{"total":95995035648,"available":94176464896,"internally_reserved":0,"allocated":896598016,"data_stored":896217573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":921958744},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:44:00 2024","interfaces":[{"interface":"back","average":{"1min":1.899,"5min":1.266,"15min":2.4369999999999998},"min":{"1min":0.23100000000000001,"5min":0.216,"15min":0.20200000000000001},"max":{"1min":20.391999999999999,"5min":20.391999999999999,"15min":185.68199999999999},"last":2.3319999999999999},{"interface":"front","average":{"1min":2.1240000000000001,"5min":1.3799999999999999,"15min":2.5950000000000002},"min":{"1min":0.248,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":20.001000000000001,"5min":20.001000000000001,"15min":186.12200000000001},"last":2.5640000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:44:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":2.254,"15min":1.617},"min":{"1min":0.26100000000000001,"5min":0.24199999999999999,"15min":0.20799999999999999},"max":{"1min":2.1309999999999998,"5min":122.738,"15min":122.738},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":2.496,"15min":1.6679999999999999},"min":{"1min":0.27700000000000002,"5min":0.24099999999999999,"15min":0.22700000000000001},"max":{"1min":3.476,"5min":122.66200000000001,"15min":122.66200000000001},"last":0.59999999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:44:00 2024","interfaces":[{"interface":"back","average":{"1min":2.0129999999999999,"5min":1.2589999999999999,"15min":1.8540000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.22700000000000001},"max":{"1min":20.718,"5min":20.718,"15min":187.99199999999999},"last":3.6930000000000001},{"interface":"front","average":{"1min":2.0059999999999998,"5min":1.159,"15min":1.8120000000000001},"min":{"1min":0.32200000000000001,"5min":0.28199999999999997,"15min":0.24399999999999999},"max":{"1min":19.988,"5min":19.988,"15min":186.375},"last":0.79800000000000004}]},{"osd":3,"last update":"Wed Sep 11 07:44:00 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":4.1779999999999999,"15min":3.4830000000000001},"min":{"1min":0.29399999999999998,"5min":0.28699999999999998,"15min":0.23599999999999999},"max":{"1min":2.0550000000000002,"5min":132.715,"15min":188.11500000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":4.1550000000000002,"15min":3.5289999999999999},"min":{"1min":0.34899999999999998,"5min":0.28100000000000003,"15min":0.25700000000000001},"max":{"1min":3.4260000000000002,"5min":133.01599999999999,"15min":186.44200000000001},"last":0.68999999999999995}]},{"osd":4,"last update":"Wed Sep 11 07:44:00 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.149,"15min":0.75700000000000001},"min":{"1min":0.32300000000000001,"5min":0.24199999999999999,"15min":0.19900000000000001},"max":{"1min":2.581,"5min":11.103,"15min":11.103},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.879,"5min":1.1259999999999999,"15min":0.75900000000000001},"min":{"1min":0.30499999999999999,"5min":0.26300000000000001,"15min":0.22500000000000001},"max":{"1min":3.6259999999999999,"5min":10.305,"15min":10.305},"last":0.73399999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.85099999999999998,"15min":0.71599999999999997},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.26000000000000001},"max":{"1min":1.5600000000000001,"5min":5.6840000000000002,"15min":5.6859999999999999},"last":0.879},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.88600000000000001,"15min":0.72099999999999997},"min":{"1min":0.32700000000000001,"5min":0.28799999999999998,"15min":0.25800000000000001},"max":{"1min":3.0990000000000002,"5min":5.4589999999999996,"15min":10.613},"last":0.87}]},{"osd":7,"last update":"Wed Sep 11 07:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":1.109,"15min":0.98499999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.28000000000000003},"max":{"1min":3.8090000000000002,"5min":6.0499999999999998,"15min":50.338999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.91900000000000004,"15min":0.94599999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.26800000000000002},"max":{"1min":1.476,"5min":5.7089999999999996,"15min":50.298999999999999},"last":0.67500000000000004}]}]},{"osd":4,"up_from":31,"seq":133143986411,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4133352,"kb_used_data":3227740,"kb_used_omap":1,"kb_used_meta":905534,"kb_avail":89611800,"statfs":{"total":95995035648,"available":91762483200,"internally_reserved":0,"allocated":3305205760,"data_stored":3304796798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":927267160},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:43:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":2.0760000000000001,"15min":1.671},"min":{"1min":0.249,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":3.085,"5min":87.623000000000005,"15min":87.623000000000005},"last":1.9450000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":2.1869999999999998,"15min":1.6679999999999999},"min":{"1min":0.23200000000000001,"5min":0.222,"15min":0.20499999999999999},"max":{"1min":3.52,"5min":87.682000000000002,"15min":87.682000000000002},"last":1.9650000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:43:52 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":3.4049999999999998,"15min":2.0430000000000001},"min":{"1min":0.28399999999999997,"5min":0.22900000000000001,"15min":0.22700000000000001},"max":{"1min":3.1030000000000002,"5min":141.78399999999999,"15min":141.78399999999999},"last":1.8280000000000001},{"interface":"front","average":{"1min":0.95299999999999996,"5min":3.3700000000000001,"15min":1.891},"min":{"1min":0.27200000000000002,"5min":0.224,"15min":0.224},"max":{"1min":3.4460000000000002,"5min":145.65899999999999,"15min":145.65899999999999},"last":0.497}]},{"osd":2,"last update":"Wed Sep 11 07:43:52 2024","interfaces":[{"interface":"back","average":{"1min":2.5840000000000001,"5min":1.4370000000000001,"15min":1.2270000000000001},"min":{"1min":0.34599999999999997,"5min":0.24099999999999999,"15min":0.223},"max":{"1min":21.978999999999999,"5min":21.978999999999999,"15min":21.978999999999999},"last":1.931},{"interface":"front","average":{"1min":1.75,"5min":1.2989999999999999,"15min":1.1839999999999999},"min":{"1min":0.39700000000000002,"5min":0.28399999999999997,"15min":0.24399999999999999},"max":{"1min":6.2939999999999996,"5min":14.273999999999999,"15min":20.02},"last":1.722}]},{"osd":3,"last update":"Wed Sep 11 07:43:52 2024","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.111,"15min":1.3440000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.9809999999999999,"5min":4.5410000000000004,"15min":24.029},"last":0.56599999999999995},{"interface":"front","average":{"1min":1.165,"5min":1.0489999999999999,"15min":1.2450000000000001},"min":{"1min":0.317,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":3.7970000000000002,"5min":3.7970000000000002,"15min":24.265000000000001},"last":1.806}]},{"osd":5,"last update":"Wed Sep 11 07:44:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.85499999999999998,"15min":0.66500000000000004},"min":{"1min":0.28399999999999997,"5min":0.224,"15min":0.21099999999999999},"max":{"1min":1.649,"5min":5.6470000000000002,"15min":5.6470000000000002},"last":1.7370000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.93999999999999995,"15min":0.71499999999999997},"min":{"1min":0.29599999999999999,"5min":0.23000000000000001,"15min":0.22700000000000001},"max":{"1min":3.4969999999999999,"5min":5.7450000000000001,"15min":5.7450000000000001},"last":0.42999999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:43:38 2024","interfaces":[{"interface":"back","average":{"1min":3.117,"5min":1.472,"15min":0.88100000000000001},"min":{"1min":0.28899999999999998,"5min":0.27000000000000002,"15min":0.20899999999999999},"max":{"1min":35.777999999999999,"5min":35.777999999999999,"15min":35.777999999999999},"last":1.919},{"interface":"front","average":{"1min":3.3220000000000001,"5min":1.278,"15min":0.82399999999999995},"min":{"1min":0.39300000000000002,"5min":0.28199999999999997,"15min":0.25800000000000001},"max":{"1min":35.691000000000003,"5min":35.691000000000003,"15min":35.691000000000003},"last":0.45700000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:44:30 2024","interfaces":[{"interface":"back","average":{"1min":6.3040000000000003,"5min":2.1760000000000002,"15min":1.151},"min":{"1min":0.39500000000000002,"5min":0.30099999999999999,"15min":0.29399999999999998},"max":{"1min":102.258,"5min":102.258,"15min":102.258},"last":0.47699999999999998},{"interface":"front","average":{"1min":6.0800000000000001,"5min":2.016,"15min":1.097},"min":{"1min":0.313,"5min":0.312,"15min":0.29599999999999999},"max":{"1min":101.886,"5min":101.886,"15min":101.886},"last":1.7829999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182640,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1768980,"kb_used_data":857608,"kb_used_omap":1,"kb_used_meta":911358,"kb_avail":91976172,"statfs":{"total":95995035648,"available":94183600128,"internally_reserved":0,"allocated":878190592,"data_stored":877800981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":933230936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:44:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.478,"15min":1.7110000000000001},"min":{"1min":0.218,"5min":0.18099999999999999,"15min":0.16600000000000001},"max":{"1min":5.8680000000000003,"5min":26.864999999999998,"15min":38.432000000000002},"last":0.34399999999999997},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.5129999999999999,"15min":2.1200000000000001},"min":{"1min":0.33800000000000002,"5min":0.183,"15min":0.183},"max":{"1min":4.718,"5min":26.657,"15min":91.876999999999995},"last":1.3799999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:44:31 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":3.7759999999999998,"15min":2.8959999999999999},"min":{"1min":0.22700000000000001,"5min":0.218,"15min":0.19500000000000001},"max":{"1min":2.2770000000000001,"5min":135.41999999999999,"15min":135.41999999999999},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.84299999999999997,"5min":3.867,"15min":2.5350000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.192},"max":{"1min":2.3079999999999998,"5min":136.126,"15min":136.126},"last":0.65900000000000003}]},{"osd":2,"last update":"Wed Sep 11 07:44:31 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":2.7690000000000001,"15min":1.9399999999999999},"min":{"1min":0.253,"5min":0.246,"15min":0.23699999999999999},"max":{"1min":3.6419999999999999,"5min":164.02600000000001,"15min":164.02600000000001},"last":4.4400000000000004},{"interface":"front","average":{"1min":1.296,"5min":2.6150000000000002,"15min":1.8109999999999999},"min":{"1min":0.35399999999999998,"5min":0.222,"15min":0.21199999999999999},"max":{"1min":5.8280000000000003,"5min":163.37799999999999,"15min":163.37799999999999},"last":0.40899999999999997}]},{"osd":4,"last update":"Wed Sep 11 07:43:49 2024","interfaces":[{"interface":"back","average":{"1min":2.2010000000000001,"5min":3.2269999999999999,"15min":1.8240000000000001},"min":{"1min":0.34200000000000003,"5min":0.312,"15min":0.27500000000000002},"max":{"1min":8.6479999999999997,"5min":132.76400000000001,"15min":132.76400000000001},"last":0.317},{"interface":"front","average":{"1min":2.1949999999999998,"5min":3.222,"15min":1.9530000000000001},"min":{"1min":0.45800000000000002,"5min":0.311,"15min":0.255},"max":{"1min":21.965,"5min":132.154,"15min":132.154},"last":1.1699999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:43:58 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":2.3450000000000002,"15min":1.6739999999999999},"min":{"1min":0.32500000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":3.8889999999999998,"5min":137.50299999999999,"15min":137.50299999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":1.1180000000000001,"5min":2.411,"15min":1.738},"min":{"1min":0.42099999999999999,"5min":0.309,"15min":0.28299999999999997},"max":{"1min":3.649,"5min":137.45599999999999,"15min":137.45599999999999},"last":1.202}]},{"osd":6,"last update":"Wed Sep 11 07:44:31 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":3.7170000000000001,"15min":2.1629999999999998},"min":{"1min":0.35899999999999999,"5min":0.35599999999999998,"15min":0.27000000000000002},"max":{"1min":3.4910000000000001,"5min":233.98500000000001,"15min":233.98500000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.98799999999999999,"5min":3.8450000000000002,"15min":2.1579999999999999},"min":{"1min":0.38,"5min":0.33800000000000002,"15min":0.30399999999999999},"max":{"1min":3.109,"5min":234.05199999999999,"15min":234.05199999999999},"last":0.44}]},{"osd":7,"last update":"Wed Sep 11 07:44:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":5.1710000000000003,"15min":2.5870000000000002},"min":{"1min":0.441,"5min":0.35699999999999998,"15min":0.308},"max":{"1min":4.891,"5min":162.42699999999999,"15min":162.42699999999999},"last":1.23},{"interface":"front","average":{"1min":1.304,"5min":5.149,"15min":2.8620000000000001},"min":{"1min":0.44900000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":7.1989999999999998,"5min":162.39099999999999,"15min":162.39099999999999},"last":0.69099999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411573,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2656204,"kb_used_data":2057984,"kb_used_omap":1,"kb_used_meta":598206,"kb_avail":91088948,"statfs":{"total":95995035648,"available":93275082752,"internally_reserved":0,"allocated":2107375616,"data_stored":2106994509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":612563288},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:44:05 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":0.995,"15min":1.921},"min":{"1min":0.188,"5min":0.188,"15min":0.16400000000000001},"max":{"1min":3.6930000000000001,"5min":5.1580000000000004,"15min":57.802999999999997},"last":1.1819999999999999},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.1060000000000001,"15min":2.0379999999999998},"min":{"1min":0.219,"5min":0.19,"15min":0.19},"max":{"1min":5.6109999999999998,"5min":8.1899999999999995,"15min":57.759},"last":0.318}]},{"osd":1,"last update":"Wed Sep 11 07:44:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.92300000000000004,"15min":1.6120000000000001},"min":{"1min":0.308,"5min":0.23000000000000001,"15min":0.22800000000000001},"max":{"1min":5.3570000000000002,"5min":5.7789999999999999,"15min":65.153999999999996},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.997,"15min":1.8320000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.216},"max":{"1min":3.0960000000000001,"5min":5.1200000000000001,"15min":63.311999999999998},"last":0.29199999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:44:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.7450000000000001,"15min":1.944},"min":{"1min":0.22500000000000001,"5min":0.224,"15min":0.224},"max":{"1min":4.1790000000000003,"5min":67.552999999999997,"15min":67.552999999999997},"last":0.33900000000000002},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.7729999999999999,"15min":1.9399999999999999},"min":{"1min":0.32200000000000001,"5min":0.23899999999999999,"15min":0.22700000000000001},"max":{"1min":9.4670000000000005,"5min":67.352999999999994,"15min":67.352999999999994},"last":0.371}]},{"osd":4,"last update":"Wed Sep 11 07:43:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":0.90300000000000002,"15min":1.171},"min":{"1min":0.377,"5min":0.315,"15min":0.27500000000000002},"max":{"1min":7.1719999999999997,"5min":7.1719999999999997,"15min":30.946999999999999},"last":2.0449999999999999},{"interface":"front","average":{"1min":1.097,"5min":0.90300000000000002,"15min":1.0589999999999999},"min":{"1min":0.45900000000000002,"5min":0.313,"15min":0.29999999999999999},"max":{"1min":3.657,"5min":4.0510000000000002,"15min":22.434000000000001},"last":0.45600000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:44:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.95699999999999996,"15min":1.1910000000000001},"min":{"1min":0.374,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":2.1829999999999998,"5min":3.319,"15min":31.789000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":1.0620000000000001,"5min":0.98799999999999999,"15min":1.248},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.31},"max":{"1min":3.4809999999999999,"5min":8.1690000000000005,"15min":30.898},"last":2.9780000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:44:34 2024","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.171,"15min":1.242},"min":{"1min":0.45800000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":2.7250000000000001,"5min":6.891,"15min":22.399000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":1.175,"5min":1.071,"15min":1.2190000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.29999999999999999},"max":{"1min":4.5540000000000003,"5min":4.867,"15min":25.739000000000001},"last":1.248}]},{"osd":7,"last update":"Wed Sep 11 07:44:20 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.0289999999999999,"15min":1.2989999999999999},"min":{"1min":0.42399999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.5259999999999998,"5min":5.2199999999999998,"15min":22.484000000000002},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.0569999999999999,"15min":1.5},"min":{"1min":0.39000000000000001,"5min":0.373,"15min":0.36699999999999999},"max":{"1min":4.5220000000000002,"5min":4.9059999999999997,"15min":25.670000000000002},"last":1.2789999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575096,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1497048,"kb_used_data":588560,"kb_used_omap":1,"kb_used_meta":908478,"kb_avail":92248104,"statfs":{"total":95995035648,"available":94462058496,"internally_reserved":0,"allocated":602685440,"data_stored":602288995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":930281816},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:44:30 2024","interfaces":[{"interface":"back","average":{"1min":2.7029999999999998,"5min":1.4390000000000001,"15min":2.2389999999999999},"min":{"1min":0.23100000000000001,"5min":0.17299999999999999,"15min":0.16900000000000001},"max":{"1min":24.561,"5min":24.561,"15min":82.218999999999994},"last":0.30499999999999999},{"interface":"front","average":{"1min":2.2959999999999998,"5min":1.534,"15min":2.2610000000000001},"min":{"1min":0.223,"5min":0.20499999999999999,"15min":0.186},"max":{"1min":24.343,"5min":24.343,"15min":82.278999999999996},"last":0.39200000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:44:06 2024","interfaces":[{"interface":"back","average":{"1min":2.6150000000000002,"5min":1.6339999999999999,"15min":1.7210000000000001},"min":{"1min":0.25900000000000001,"5min":0.251,"15min":0.19800000000000001},"max":{"1min":25.440000000000001,"5min":25.440000000000001,"15min":25.440000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":2.5369999999999999,"5min":1.5660000000000001,"15min":1.7569999999999999},"min":{"1min":0.23899999999999999,"5min":0.20499999999999999,"15min":0.191},"max":{"1min":24.581,"5min":24.581,"15min":24.581},"last":0.497}]},{"osd":3,"last update":"Wed Sep 11 07:44:24 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":2.2839999999999998,"15min":2.8279999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":3.9249999999999998,"5min":90.766999999999996,"15min":90.766999999999996},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.94099999999999995,"5min":2.0920000000000001,"15min":2.754},"min":{"1min":0.247,"5min":0.22900000000000001,"15min":0.22600000000000001},"max":{"1min":3.3149999999999999,"5min":91.513000000000005,"15min":91.513000000000005},"last":0.42199999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:43:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.607,"15min":1.3540000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.27400000000000002},"max":{"1min":6.9580000000000002,"5min":7.8390000000000004,"15min":7.8390000000000004},"last":0.59899999999999998},{"interface":"front","average":{"1min":1.6859999999999999,"5min":1.6080000000000001,"15min":1.4770000000000001},"min":{"1min":0.33700000000000002,"5min":0.29999999999999999,"15min":0.26500000000000001},"max":{"1min":6.7469999999999999,"5min":7.8239999999999998,"15min":9.2289999999999992},"last":0.52800000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:44:06 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.349,"15min":1.387},"min":{"1min":0.33500000000000002,"5min":0.27500000000000002,"15min":0.26600000000000001},"max":{"1min":3.786,"5min":8.3680000000000003,"15min":8.3680000000000003},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.5329999999999999,"15min":1.506},"min":{"1min":0.35499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":4.258,"5min":8.6809999999999992,"15min":9.1929999999999996},"last":0.55200000000000005}]},{"osd":6,"last update":"Wed Sep 11 07:44:24 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.2649999999999999,"15min":1.488},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30099999999999999},"max":{"1min":4.2889999999999997,"5min":9.1850000000000005,"15min":24.018000000000001},"last":1.022},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.1279999999999999,"15min":1.254},"min":{"1min":0.32700000000000001,"5min":0.318,"15min":0.29999999999999999},"max":{"1min":3.6680000000000001,"5min":6.0730000000000004,"15min":23.681000000000001},"last":0.96899999999999997}]},{"osd":7,"last update":"Wed Sep 11 07:44:24 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.3540000000000001,"15min":1.335},"min":{"1min":0.36499999999999999,"5min":0.36099999999999999,"15min":0.32200000000000001},"max":{"1min":3.2690000000000001,"5min":9.0489999999999995,"15min":9.3629999999999995},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.038,"5min":1.452,"15min":1.542},"min":{"1min":0.38200000000000001,"5min":0.33400000000000002,"15min":0.313},"max":{"1min":4.5149999999999997,"5min":8.2219999999999995,"15min":8.2219999999999995},"last":0.746}]}]},{"osd":0,"up_from":8,"seq":34359738622,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3855332,"kb_used_data":2957528,"kb_used_omap":1,"kb_used_meta":897534,"kb_avail":89889820,"statfs":{"total":95995035648,"available":92047175680,"internally_reserved":0,"allocated":3028508672,"data_stored":3028103762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":919075160},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:44:31 2024","interfaces":[{"interface":"back","average":{"1min":4.7389999999999999,"5min":2.04,"15min":1.6519999999999999},"min":{"1min":0.19700000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":72.418000000000006,"5min":72.418000000000006,"15min":72.418000000000006},"last":2.6779999999999999},{"interface":"front","average":{"1min":4.4829999999999997,"5min":1.8680000000000001,"15min":1.675},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.16600000000000001},"max":{"1min":71.703999999999994,"5min":71.703999999999994,"15min":71.703999999999994},"last":0.52300000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:44:02 2024","interfaces":[{"interface":"back","average":{"1min":1.512,"5min":2.0099999999999998,"15min":2.4700000000000002},"min":{"1min":0.25900000000000001,"5min":0.21199999999999999,"15min":0.188},"max":{"1min":4.1900000000000004,"5min":68,"15min":200.339},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.583,"5min":1.923,"15min":2.3740000000000001},"min":{"1min":0.252,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":7.6820000000000004,"5min":67.625,"15min":200.40100000000001},"last":0.33100000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:44:31 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.2609999999999999,"15min":2.129},"min":{"1min":0.29199999999999998,"5min":0.23300000000000001,"15min":0.20799999999999999},"max":{"1min":3.1499999999999999,"5min":8.8070000000000004,"15min":45.185000000000002},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.3999999999999999,"15min":1.9530000000000001},"min":{"1min":0.253,"5min":0.251,"15min":0.22800000000000001},"max":{"1min":3.2879999999999998,"5min":15.252000000000001,"15min":43.228999999999999},"last":0.27900000000000003}]},{"osd":4,"last update":"Wed Sep 11 07:43:44 2024","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.536,"15min":1.343},"min":{"1min":0.38600000000000001,"5min":0.29299999999999998,"15min":0.26700000000000002},"max":{"1min":7.0519999999999996,"5min":30.315000000000001,"15min":30.315000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.71,"15min":1.5269999999999999},"min":{"1min":0.311,"5min":0.28599999999999998,"15min":0.27000000000000002},"max":{"1min":3.6400000000000001,"5min":30.363,"15min":30.363},"last":0.84199999999999997}]},{"osd":5,"last update":"Wed Sep 11 07:44:02 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":1.0609999999999999,"15min":1.3129999999999999},"min":{"1min":0.40300000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.151,"5min":3.9790000000000001,"15min":19.381},"last":0.55500000000000005},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.403,"15min":1.516},"min":{"1min":0.433,"5min":0.36399999999999999,"15min":0.27200000000000002},"max":{"1min":3.6150000000000002,"5min":7.202,"15min":19.437000000000001},"last":0.88700000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:44:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.3260000000000001,"15min":1.502},"min":{"1min":0.35699999999999998,"5min":0.33900000000000002,"15min":0.316},"max":{"1min":3.7440000000000002,"5min":6.5190000000000001,"15min":23.190999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.369,"5min":1.3799999999999999,"15min":1.4850000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":3.4319999999999999,"5min":15.404,"15min":23.632000000000001},"last":0.60899999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:44:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.5649999999999999,"15min":1.5389999999999999},"min":{"1min":0.434,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":3.4900000000000002,"5min":30.43,"15min":30.43},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.702,"15min":1.5760000000000001},"min":{"1min":0.40100000000000002,"5min":0.35999999999999999,"15min":0.315},"max":{"1min":3.5640000000000001,"5min":31.065000000000001,"15min":31.065000000000001},"last":0.58199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2972192768,"data_stored":2972188696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":599523328,"data_stored":599523328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2084634624,"data_stored":2084634624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":867958784,"data_stored":867958784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3253276672,"data_stored":3253272600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":890699776,"data_stored":890699776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2350190592,"data_stored":2350186520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1181417472,"data_stored":1181417472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:44:47.369 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-11T07:44:47.369 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pool set .mgr pg_num 11 2024-09-11T07:44:47.401 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:47.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:47.465 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:47.466 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:47.567 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:47.597 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:47.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:47.598 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:47.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:47 smithi022 ceph-mon[37330]: pgmap v844: 148 pgs: 148 active+clean; 3.6 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.59k op/s 2024-09-11T07:44:47.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:47 smithi022 ceph-mon[29462]: pgmap v844: 148 pgs: 148 active+clean; 3.6 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.59k op/s 2024-09-11T07:44:47.698 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:47.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:47.744 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:47.744 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:47.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:47 smithi104 ceph-mon[33507]: pgmap v844: 148 pgs: 148 active+clean; 3.6 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.59k op/s 2024-09-11T07:44:47.845 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:47.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:47.883 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:47.883 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:47.985 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:48.014 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:48.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:48.015 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:48.116 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:48.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:48.161 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:48.162 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:48.262 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:48.300 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:48.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:48.301 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:48.403 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:48.440 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:48.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:48.443 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:48.544 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:48.576 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:48.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:48.577 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:48.678 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:48.715 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:48.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:48.716 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:48.817 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:48.846 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:48.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:48.847 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:48.947 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:48.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:48.978 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:48.978 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:49.079 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:49.120 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:49.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:49.122 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:49.223 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:49.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:49.257 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:49.257 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:49.357 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:49.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:49.392 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:49.393 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:49.493 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:49.520 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:49.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:49.521 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:49.622 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:49.662 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:49.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:49.663 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:49.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:49 smithi022 ceph-mon[29462]: pgmap v845: 148 pgs: 148 active+clean; 3.6 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 789 op/s 2024-09-11T07:44:49.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:49 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:44:49.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:49 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:44:49.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:49 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2881394484' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:44:49.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:49 smithi022 ceph-mon[37330]: pgmap v845: 148 pgs: 148 active+clean; 3.6 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 789 op/s 2024-09-11T07:44:49.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:49 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:44:49.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:49 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:44:49.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:49 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2881394484' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:44:49.764 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:49.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:49 smithi104 ceph-mon[33507]: pgmap v845: 148 pgs: 148 active+clean; 3.6 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 789 op/s 2024-09-11T07:44:49.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:49 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:44:49.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:49 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:44:49.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:49 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2881394484' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:44:49.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:49.792 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:49.792 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:49.878 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is set 2024-09-11T07:44:49.893 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:49.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:49.933 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:49.934 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:50.034 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:50.079 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:50.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:50.080 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:50.181 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:50.216 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:50.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:50.217 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:50.318 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:50.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:50.358 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:50.358 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:50.452 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:44:50 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:44:50.460 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:50.490 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:50.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:50.492 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:50.593 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:50.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:50.628 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:50.628 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:50.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:44:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:44:50] "GET /metrics HTTP/1.1" 200 224974 "" "Prometheus/2.43.0" 2024-09-11T07:44:50.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:50 smithi022 ceph-mon[29462]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:44:50.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:50 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2881394484' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:44:50.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:50 smithi022 ceph-mon[29462]: osdmap e385: 8 total, 8 up, 8 in 2024-09-11T07:44:50.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:50 smithi022 ceph-mon[37330]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:44:50.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:50 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2881394484' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:44:50.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:50 smithi022 ceph-mon[37330]: osdmap e385: 8 total, 8 up, 8 in 2024-09-11T07:44:50.729 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:50.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:50.774 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:50.775 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:50.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:50 smithi104 ceph-mon[33507]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:44:50.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:50 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2881394484' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:44:50.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:50 smithi104 ceph-mon[33507]: osdmap e385: 8 total, 8 up, 8 in 2024-09-11T07:44:50.876 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:50.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:50.915 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:50.915 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:51.015 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:51.050 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:51.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:51.051 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:51.099 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:44:51.152 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:51.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:51.192 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:51.192 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:51.293 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:51.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:51.327 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:51.327 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:51.428 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:51.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:51.467 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:51.468 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:51.569 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:51.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:51.600 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:51.600 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:51.629 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:51 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:44:51.629 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:51 smithi022 ceph-mon[29462]: pgmap v847: 148 pgs: 148 active+clean; 3.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.59k op/s 2024-09-11T07:44:51.629 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:51 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:44:51.629 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:51 smithi022 ceph-mon[37330]: pgmap v847: 148 pgs: 148 active+clean; 3.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.59k op/s 2024-09-11T07:44:51.702 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:51.731 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:51.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:51.733 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:51.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:51 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:44:51.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:51 smithi104 ceph-mon[33507]: pgmap v847: 148 pgs: 148 active+clean; 3.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.59k op/s 2024-09-11T07:44:51.784 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:44:51.784 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:44:51.834 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:51.854 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:44:51.854 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:44:51.854 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:44:51.854 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:44:51.854 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:44:51.855 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:44:51.855 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:44:51.855 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:44:51.855 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:44:51.856 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:44:51.871 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:51.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:51.872 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:51.878 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:44:51.878 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:44:51.915 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:44:51.915 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:44:51.949 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:44:51.949 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:44:51.974 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:52.014 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:52.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:52.015 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:52.116 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:52.152 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:52.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:52.153 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:52.254 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:52.294 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:52.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:52.295 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:52.395 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:52.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:52.427 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:52.427 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:52.528 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:52.560 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:52.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:52.561 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:52.662 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:52.699 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:52.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:52.700 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:52.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:52 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2347202984' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:44:52.801 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:52.836 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:52.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:52.837 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:52.904 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:52 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2347202984' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:44:52.905 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:52 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2347202984' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:44:52.937 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:52.947 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T07:44:52.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:52.979 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:52.979 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:53.080 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:53.110 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:53.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:53.111 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:53.212 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:53.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:53.262 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:53.262 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:53.363 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:53.394 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:53.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:53.395 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:53.496 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:53.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:53.536 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:53.536 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:53.637 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:53.666 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:53.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:53.667 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:53.715 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-11T07:44:53.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:53 smithi022 ceph-mon[29462]: pgmap v848: 148 pgs: 148 active+clean; 3.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 887 op/s 2024-09-11T07:44:53.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:44:53.751 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:53 smithi022 ceph-mon[37330]: pgmap v848: 148 pgs: 148 active+clean; 3.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 887 op/s 2024-09-11T07:44:53.751 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:44:53.768 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:53.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:53 smithi104 ceph-mon[33507]: pgmap v848: 148 pgs: 148 active+clean; 3.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 887 op/s 2024-09-11T07:44:53.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:44:53.817 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:53.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:53.818 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:53.919 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:53.965 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:53.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:53.967 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:54.067 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:54.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:54.109 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:54.109 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:54.210 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:54.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:54.250 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:54.250 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:54.351 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:54.392 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:54.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:54.393 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:54.494 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:54.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:44:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:44:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:44:54.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:54.534 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:54.534 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:54.635 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:54.682 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:54.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:54.683 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:54.784 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:54.813 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:44:54.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:54.850 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:54.850 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:54.949 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:44:54.951 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:54.980 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:54.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:54.981 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:55.077 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:44:55.082 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:55.141 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:55.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:55.144 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:55.245 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:55.299 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:55.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:55.300 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:55.401 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:55.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:55.439 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:55.439 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:55.540 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:55.576 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:55.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:55.578 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:55.666 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:55 smithi022 ceph-mon[37330]: pgmap v849: 148 pgs: 148 active+clean; 2.9 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.37k op/s 2024-09-11T07:44:55.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:55 smithi022 ceph-mon[29462]: pgmap v849: 148 pgs: 148 active+clean; 2.9 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.37k op/s 2024-09-11T07:44:55.678 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:55.717 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:55.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:55.718 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:55.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:55 smithi104 ceph-mon[33507]: pgmap v849: 148 pgs: 148 active+clean; 2.9 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.37k op/s 2024-09-11T07:44:55.819 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:55.861 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:55.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:55.862 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:55.963 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:56.002 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:56.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:56.004 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:56.104 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:44:56.166 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:56.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:56.167 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:44:56.268 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:56.294 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:56.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:56.295 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:56.395 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:56.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:56.433 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:56.433 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:56.534 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:56.567 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:56.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:56.568 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:56.669 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:56.707 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:56.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:56.708 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:56.746 INFO:teuthology.orchestra.run.smithi022.stderr:set pool 1 pg_num to 11 2024-09-11T07:44:56.809 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:56.840 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:56.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:56.841 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:56.942 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:56.977 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:56.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:56.978 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:57.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:56 smithi104 ceph-mon[33507]: pgmap v850: 148 pgs: 148 active+clean; 2.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.30k op/s 2024-09-11T07:44:57.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:56 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1688870697' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]: dispatch 2024-09-11T07:44:57.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:56 smithi104 ceph-mon[33507]: from='client.25879 ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]: dispatch 2024-09-11T07:44:57.079 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:57.113 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:57.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:57.114 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:57.166 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:56 smithi022 ceph-mon[37330]: pgmap v850: 148 pgs: 148 active+clean; 2.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.30k op/s 2024-09-11T07:44:57.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:56 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1688870697' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]: dispatch 2024-09-11T07:44:57.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:56 smithi022 ceph-mon[37330]: from='client.25879 ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]: dispatch 2024-09-11T07:44:57.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:56 smithi022 ceph-mon[29462]: pgmap v850: 148 pgs: 148 active+clean; 2.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.30k op/s 2024-09-11T07:44:57.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:56 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1688870697' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]: dispatch 2024-09-11T07:44:57.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:56 smithi022 ceph-mon[29462]: from='client.25879 ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]: dispatch 2024-09-11T07:44:57.215 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:57.254 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:57.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:57.255 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:57.356 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:57.397 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:57.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:57.398 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:57.499 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:57.566 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:57.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:57.567 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:57.667 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:57.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:57.707 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:57.707 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:57.808 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:57.844 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:57 smithi022 ceph-mon[37330]: from='client.25879 ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-11T07:44:57.844 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:57 smithi022 ceph-mon[37330]: osdmap e386: 8 total, 8 up, 8 in 2024-09-11T07:44:57.844 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:57 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:44:57.844 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:57 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:44:57.845 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:57 smithi022 ceph-mon[29462]: from='client.25879 ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-11T07:44:57.845 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:57 smithi022 ceph-mon[29462]: osdmap e386: 8 total, 8 up, 8 in 2024-09-11T07:44:57.845 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:57 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:44:57.845 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:57 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:44:57.846 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:57.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:57.847 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:57.948 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:44:57.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:57.979 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:57.979 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:44:58.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:57 smithi104 ceph-mon[33507]: from='client.25879 ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-11T07:44:58.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:57 smithi104 ceph-mon[33507]: osdmap e386: 8 total, 8 up, 8 in 2024-09-11T07:44:58.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:57 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:44:58.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:57 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:44:58.080 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:58.113 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:58.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:58.114 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:58.215 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:58.254 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:58.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:58.255 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:58.356 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:58.391 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:58.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:58.392 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:58.493 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:58.533 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:58.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:58.534 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:58.575 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:44:58.576 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:44:58.635 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:58.669 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:58.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:58.671 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:58.772 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:58.804 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:58.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:58.805 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:58.806 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:44:58.906 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:44:58.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:58 smithi022 ceph-mon[29462]: osdmap e387: 8 total, 8 up, 8 in 2024-09-11T07:44:58.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:58 smithi022 ceph-mon[29462]: pgmap v853: 148 pgs: 148 active+clean; 2.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 830 op/s 2024-09-11T07:44:58.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:58 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-09-11T07:44:58.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:58 smithi022 ceph-mon[37330]: osdmap e387: 8 total, 8 up, 8 in 2024-09-11T07:44:58.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:58 smithi022 ceph-mon[37330]: pgmap v853: 148 pgs: 148 active+clean; 2.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 830 op/s 2024-09-11T07:44:58.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:58 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-09-11T07:44:58.945 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:58.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:58.946 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:44:59.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:58 smithi104 ceph-mon[33507]: osdmap e387: 8 total, 8 up, 8 in 2024-09-11T07:44:59.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:58 smithi104 ceph-mon[33507]: pgmap v853: 148 pgs: 148 active+clean; 2.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 830 op/s 2024-09-11T07:44:59.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:58 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-09-11T07:44:59.047 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:44:59.081 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:59.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:59.082 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:44:59.183 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:59.220 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:59.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:59.221 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:59.266 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-11T07:44:59.322 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:44:59.354 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:59.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:59.355 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:44:59.456 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:59.488 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:59.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:59.489 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:59.590 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:44:59.629 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:59.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:59.631 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:44:59.732 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:44:59.771 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:44:59.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:59.772 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:44:59.873 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:44:59.899 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:44:59.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:44:59.900 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:44:59.916 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:59 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-11T07:44:59.916 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:44:59 smithi022 ceph-mon[37330]: osdmap e388: 8 total, 8 up, 8 in 2024-09-11T07:44:59.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:59 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-11T07:44:59.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:44:59 smithi022 ceph-mon[29462]: osdmap e388: 8 total, 8 up, 8 in 2024-09-11T07:45:00.001 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:00.024 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:00.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:00.025 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:00.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:59 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-11T07:45:00.032 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:44:59 smithi104 ceph-mon[33507]: osdmap e388: 8 total, 8 up, 8 in 2024-09-11T07:45:00.126 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:00.148 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:00.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:00.150 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:00.250 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:00.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:00.273 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:00.273 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:00.374 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:00.421 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:00.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:00.422 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:00.523 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:00.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:45:00 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:45:00.563 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:00.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:00.564 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:00.618 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:45:00 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:45:00] "GET /metrics HTTP/1.1" 200 224999 "" "Prometheus/2.43.0" 2024-09-11T07:45:00.665 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:00.702 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:00.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:00.703 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:00.804 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:00.837 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:00.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:00.838 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:00.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:00 smithi022 ceph-mon[37330]: osdmap e389: 8 total, 8 up, 8 in 2024-09-11T07:45:00.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:00 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:45:00.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:00 smithi022 ceph-mon[37330]: pgmap v856: 158 pgs: 1 peering, 10 unknown, 147 active+clean; 2.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 0 B/s wr, 1.61k op/s 2024-09-11T07:45:00.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:00 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-09-11T07:45:00.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:00 smithi022 ceph-mon[29462]: osdmap e389: 8 total, 8 up, 8 in 2024-09-11T07:45:00.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:00 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:45:00.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:00 smithi022 ceph-mon[29462]: pgmap v856: 158 pgs: 1 peering, 10 unknown, 147 active+clean; 2.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 0 B/s wr, 1.61k op/s 2024-09-11T07:45:00.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:00 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-09-11T07:45:00.939 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:00.977 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:00.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:00.978 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:01.079 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:01.101 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:01.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:01.102 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:01.203 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:01.239 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:01.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:01.240 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:01.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:00 smithi104 ceph-mon[33507]: osdmap e389: 8 total, 8 up, 8 in 2024-09-11T07:45:01.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:00 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:45:01.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:00 smithi104 ceph-mon[33507]: pgmap v856: 158 pgs: 1 peering, 10 unknown, 147 active+clean; 2.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 0 B/s wr, 1.61k op/s 2024-09-11T07:45:01.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:00 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-09-11T07:45:01.341 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:01.378 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:01.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:01.379 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:01.480 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:01.506 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:01.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:01.507 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:01.608 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:01.643 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:01.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:01.644 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:01.745 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:01.766 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:01.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:01.767 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:01.868 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:01.902 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:01.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:01.904 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:02.004 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:02.028 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:02.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:02.029 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:02.130 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:02.166 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:02.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:01 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-11T07:45:02.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:01 smithi022 ceph-mon[29462]: osdmap e390: 8 total, 8 up, 8 in 2024-09-11T07:45:02.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:01 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:45:02.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:01 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:45:02.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:02.167 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:02.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:01 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-11T07:45:02.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:01 smithi022 ceph-mon[37330]: osdmap e390: 8 total, 8 up, 8 in 2024-09-11T07:45:02.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:01 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:45:02.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:01 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:45:02.269 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:02.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:01 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-11T07:45:02.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:01 smithi104 ceph-mon[33507]: osdmap e390: 8 total, 8 up, 8 in 2024-09-11T07:45:02.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:01 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:45:02.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:01 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:45:02.301 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:02.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:02.302 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:02.403 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:02.439 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:02.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:02.441 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:02.541 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:02.568 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:02.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:02.569 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:02.670 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:02.703 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:02.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:02.705 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:02.805 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:02.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:02.840 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:02.840 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:02.941 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:02.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:02.982 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:02.982 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:03.083 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:03.111 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:03.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:03.112 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:03.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:02 smithi022 ceph-mon[29462]: osdmap e391: 8 total, 8 up, 8 in 2024-09-11T07:45:03.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:02 smithi022 ceph-mon[29462]: pgmap v859: 158 pgs: 1 peering, 10 unknown, 147 active+clean; 2.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 0 B/s wr, 1.61k op/s 2024-09-11T07:45:03.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:02 smithi022 ceph-mon[37330]: osdmap e391: 8 total, 8 up, 8 in 2024-09-11T07:45:03.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:02 smithi022 ceph-mon[37330]: pgmap v859: 158 pgs: 1 peering, 10 unknown, 147 active+clean; 2.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 0 B/s wr, 1.61k op/s 2024-09-11T07:45:03.213 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:03.248 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:03.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:03.249 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:03.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:02 smithi104 ceph-mon[33507]: osdmap e391: 8 total, 8 up, 8 in 2024-09-11T07:45:03.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:02 smithi104 ceph-mon[33507]: pgmap v859: 158 pgs: 1 peering, 10 unknown, 147 active+clean; 2.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 0 B/s wr, 1.61k op/s 2024-09-11T07:45:03.350 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:03.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:03.371 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:03.371 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:03.472 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:03.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:03.504 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:03.504 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:03.605 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:03.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:03.638 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:03.639 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:03.740 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:03.769 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:03.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:03.771 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:03.871 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:03.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:03.910 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:03.910 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:04.011 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:04.040 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:04.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:04.041 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:04.141 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:04.174 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:04.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:04.176 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:04.277 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:04.308 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:04.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:04.309 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:04.410 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:04.441 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:04.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:04.442 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:04.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:45:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:45:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:45:04.543 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:04.577 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:04.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:04.578 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:04.678 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:04.715 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:04.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:04.716 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:04.817 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:04.838 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:04.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:04.839 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:04.939 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:04.970 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:04.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:04.971 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:05.071 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:05.089 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:45:05.090 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:45:05.090 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-11T07:45:05.090 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T07:45:05.090 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T07:45:05.090 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 config set filestore_inject_stall 3 2024-09-11T07:45:05.108 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:05.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:05.109 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:05.210 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:05.252 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:05.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:05.254 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:05.354 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:05.389 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:05.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:05.390 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:05.491 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:05.526 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:05.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:05.527 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:05.535 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:45:05.628 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:05.665 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:05.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:05.666 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:05.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:05 smithi104 ceph-mon[33507]: pgmap v860: 158 pgs: 3 unknown, 155 active+clean; 2.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 0 B/s wr, 2.08k op/s 2024-09-11T07:45:05.708 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:05 smithi022 ceph-mon[29462]: pgmap v860: 158 pgs: 3 unknown, 155 active+clean; 2.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 0 B/s wr, 2.08k op/s 2024-09-11T07:45:05.708 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:05 smithi022 ceph-mon[37330]: pgmap v860: 158 pgs: 3 unknown, 155 active+clean; 2.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 0 B/s wr, 2.08k op/s 2024-09-11T07:45:05.767 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:05.821 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:05.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:05.822 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:05.923 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:05.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:05.962 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:05.963 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:06.063 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:06.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:06.101 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:06.101 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:06.202 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:06.236 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:06.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:06.237 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:06.338 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:06.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:06.381 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:06.381 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:06.482 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:06.522 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:06.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:06.523 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:06.624 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:06.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:06.663 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:06.663 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:06.734 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:45:06.760 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is unset 2024-09-11T07:45:06.764 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:06.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:06.794 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:06.794 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:06.797 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:45:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:45:06.832 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:45:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:45:06.860 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:45:06.860 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:45:06.890 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:45:06.890 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:45:06.895 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:06.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:06.931 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:06.931 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:06.932 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:45:06.932 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:45:06.958 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:45:06.958 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:45:06.993 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:45:06.993 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:45:06.996 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:06 smithi022 ceph-mon[29462]: pgmap v861: 158 pgs: 158 active+clean; 2.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 33 KiB/s rd, 225 KiB/s wr, 991 op/s; 247 KiB/s, 0 objects/s recovering 2024-09-11T07:45:06.996 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:06 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3447229997' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:45:06.996 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:06 smithi022 ceph-mon[37330]: pgmap v861: 158 pgs: 158 active+clean; 2.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 33 KiB/s rd, 225 KiB/s wr, 991 op/s; 247 KiB/s, 0 objects/s recovering 2024-09-11T07:45:06.996 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:06 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3447229997' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:45:07.026 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:45:07.026 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:45:07.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:06 smithi104 ceph-mon[33507]: pgmap v861: 158 pgs: 158 active+clean; 2.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 33 KiB/s rd, 225 KiB/s wr, 991 op/s; 247 KiB/s, 0 objects/s recovering 2024-09-11T07:45:07.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:06 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3447229997' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:45:07.032 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:07.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:07.078 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:07.078 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:07.179 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:07.219 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:07.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:07.220 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:07.321 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:07.374 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:07.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:07.375 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:07.476 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:07.513 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:07.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:07.514 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:07.598 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:45:07.598 INFO:teuthology.orchestra.run.smithi104.stdout: "success": "" 2024-09-11T07:45:07.598 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:45:07.614 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:07.647 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:07.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:07.649 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:07.750 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:07.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:07.784 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:07.784 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:07.885 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:07.899 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:07 smithi022 ceph-mon[29462]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:45:07.899 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:07 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3447229997' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:45:07.899 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:07 smithi022 ceph-mon[29462]: osdmap e392: 8 total, 8 up, 8 in 2024-09-11T07:45:07.899 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:07 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2347362391' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:45:07.899 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:07 smithi022 ceph-mon[37330]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:45:07.900 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:07 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3447229997' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:45:07.900 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:07 smithi022 ceph-mon[37330]: osdmap e392: 8 total, 8 up, 8 in 2024-09-11T07:45:07.900 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:07 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2347362391' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:45:07.925 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:07.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:07.926 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:08.027 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:08.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:07 smithi104 ceph-mon[33507]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:45:08.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:07 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3447229997' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:45:08.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:07 smithi104 ceph-mon[33507]: osdmap e392: 8 total, 8 up, 8 in 2024-09-11T07:45:08.031 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:07 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2347362391' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:45:08.066 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:08.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:08.067 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:08.168 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:08.203 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:08.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:08.204 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:08.305 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:08.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:08.339 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:08.340 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:08.441 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:08.474 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:08.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:08.475 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:08.576 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:08.614 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:08.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:08.615 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:08.716 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:08.751 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:08.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:08.752 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:08.853 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:08.891 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:08.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:08.892 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:08.992 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:09.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:08 smithi104 ceph-mon[33507]: 5.12 scrub starts 2024-09-11T07:45:09.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:08 smithi104 ceph-mon[33507]: 5.12 scrub ok 2024-09-11T07:45:09.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:08 smithi104 ceph-mon[33507]: 2.1 scrub starts 2024-09-11T07:45:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:08 smithi104 ceph-mon[33507]: 2.1 scrub ok 2024-09-11T07:45:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:45:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:08 smithi104 ceph-mon[33507]: 2.f scrub starts 2024-09-11T07:45:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:08 smithi104 ceph-mon[33507]: 2.f scrub ok 2024-09-11T07:45:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:08 smithi104 ceph-mon[33507]: pgmap v863: 158 pgs: 158 active+clean; 2.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 28 KiB/s rd, 195 KiB/s wr, 807 op/s; 214 KiB/s, 0 objects/s recovering 2024-09-11T07:45:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:08 smithi104 ceph-mon[33507]: 2.b scrub starts 2024-09-11T07:45:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:08 smithi104 ceph-mon[33507]: 2.b scrub ok 2024-09-11T07:45:09.031 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:09.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:09.032 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:09.077 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:08 smithi022 ceph-mon[29462]: 5.12 scrub starts 2024-09-11T07:45:09.077 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:08 smithi022 ceph-mon[29462]: 5.12 scrub ok 2024-09-11T07:45:09.077 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:08 smithi022 ceph-mon[29462]: 2.1 scrub starts 2024-09-11T07:45:09.077 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:08 smithi022 ceph-mon[29462]: 2.1 scrub ok 2024-09-11T07:45:09.077 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:45:09.077 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:08 smithi022 ceph-mon[29462]: 2.f scrub starts 2024-09-11T07:45:09.077 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:08 smithi022 ceph-mon[29462]: 2.f scrub ok 2024-09-11T07:45:09.077 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:08 smithi022 ceph-mon[29462]: pgmap v863: 158 pgs: 158 active+clean; 2.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 28 KiB/s rd, 195 KiB/s wr, 807 op/s; 214 KiB/s, 0 objects/s recovering 2024-09-11T07:45:09.077 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:08 smithi022 ceph-mon[29462]: 2.b scrub starts 2024-09-11T07:45:09.078 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:08 smithi022 ceph-mon[29462]: 2.b scrub ok 2024-09-11T07:45:09.078 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:08 smithi022 ceph-mon[37330]: 5.12 scrub starts 2024-09-11T07:45:09.078 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:08 smithi022 ceph-mon[37330]: 5.12 scrub ok 2024-09-11T07:45:09.078 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:08 smithi022 ceph-mon[37330]: 2.1 scrub starts 2024-09-11T07:45:09.078 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:08 smithi022 ceph-mon[37330]: 2.1 scrub ok 2024-09-11T07:45:09.078 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:45:09.078 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:08 smithi022 ceph-mon[37330]: 2.f scrub starts 2024-09-11T07:45:09.078 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:08 smithi022 ceph-mon[37330]: 2.f scrub ok 2024-09-11T07:45:09.078 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:08 smithi022 ceph-mon[37330]: pgmap v863: 158 pgs: 158 active+clean; 2.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 28 KiB/s rd, 195 KiB/s wr, 807 op/s; 214 KiB/s, 0 objects/s recovering 2024-09-11T07:45:09.078 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:08 smithi022 ceph-mon[37330]: 2.b scrub starts 2024-09-11T07:45:09.078 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:08 smithi022 ceph-mon[37330]: 2.b scrub ok 2024-09-11T07:45:09.132 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:09.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:09.177 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:09.177 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:09.278 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:09.314 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:09.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:09.315 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:09.416 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:09.453 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:09.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:09.454 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:09.555 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:09.594 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:09.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:09.595 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:09.696 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:09.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:09.733 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:09.733 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:09.833 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T07:45:09.834 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:09.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:09.882 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:09.882 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:09.983 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:10.014 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:10.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:10.015 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:10.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:09 smithi104 ceph-mon[33507]: 2.2 scrub starts 2024-09-11T07:45:10.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:09 smithi104 ceph-mon[33507]: 2.2 scrub ok 2024-09-11T07:45:10.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:09 smithi104 ceph-mon[33507]: 2.14 scrub starts 2024-09-11T07:45:10.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:09 smithi104 ceph-mon[33507]: 2.14 scrub ok 2024-09-11T07:45:10.082 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:09 smithi022 ceph-mon[29462]: 2.2 scrub starts 2024-09-11T07:45:10.083 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:09 smithi022 ceph-mon[29462]: 2.2 scrub ok 2024-09-11T07:45:10.083 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:09 smithi022 ceph-mon[29462]: 2.14 scrub starts 2024-09-11T07:45:10.083 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:09 smithi022 ceph-mon[29462]: 2.14 scrub ok 2024-09-11T07:45:10.084 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:09 smithi022 ceph-mon[37330]: 2.2 scrub starts 2024-09-11T07:45:10.084 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:09 smithi022 ceph-mon[37330]: 2.2 scrub ok 2024-09-11T07:45:10.084 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:09 smithi022 ceph-mon[37330]: 2.14 scrub starts 2024-09-11T07:45:10.084 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:09 smithi022 ceph-mon[37330]: 2.14 scrub ok 2024-09-11T07:45:10.116 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:10.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:10.178 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:10.178 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:10.279 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:10.320 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:10.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:10.321 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:10.422 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:10.464 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:10.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:10.465 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:10.474 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:45:10.474 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:45:10.474 INFO:teuthology.orchestra.run.smithi022.stdout: "complaint_time": 30, 2024-09-11T07:45:10.474 INFO:teuthology.orchestra.run.smithi022.stdout: "num_blocked_ops": 0 2024-09-11T07:45:10.475 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:45:10.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:45:10 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:45:10.566 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:10.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:10.600 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:10.600 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:10.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:45:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:45:10] "GET /metrics HTTP/1.1" 200 225009 "" "Prometheus/2.43.0" 2024-09-11T07:45:10.701 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:10.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:10.730 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:10.730 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:10.831 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:10.867 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:10.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:10.868 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:10.969 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:11.007 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:11.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:11.008 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:11.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:10 smithi104 ceph-mon[33507]: 2.15 scrub starts 2024-09-11T07:45:11.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:10 smithi104 ceph-mon[33507]: 2.15 scrub ok 2024-09-11T07:45:11.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:10 smithi104 ceph-mon[33507]: 2.13 scrub starts 2024-09-11T07:45:11.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:10 smithi104 ceph-mon[33507]: 2.13 scrub ok 2024-09-11T07:45:11.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:10 smithi104 ceph-mon[33507]: 10.f scrub starts 2024-09-11T07:45:11.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:10 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:45:11.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:10 smithi104 ceph-mon[33507]: pgmap v864: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 1.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 26 KiB/s rd, 173 KiB/s wr, 1.49k op/s; 190 KiB/s, 0 objects/s recovering 2024-09-11T07:45:11.109 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:11.120 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:45:11.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:11.154 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:11.154 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:11.155 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:10 smithi022 ceph-mon[29462]: 2.15 scrub starts 2024-09-11T07:45:11.155 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:10 smithi022 ceph-mon[29462]: 2.15 scrub ok 2024-09-11T07:45:11.155 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:10 smithi022 ceph-mon[29462]: 2.13 scrub starts 2024-09-11T07:45:11.155 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:10 smithi022 ceph-mon[29462]: 2.13 scrub ok 2024-09-11T07:45:11.155 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:10 smithi022 ceph-mon[29462]: 10.f scrub starts 2024-09-11T07:45:11.155 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:10 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:45:11.155 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:10 smithi022 ceph-mon[29462]: pgmap v864: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 1.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 26 KiB/s rd, 173 KiB/s wr, 1.49k op/s; 190 KiB/s, 0 objects/s recovering 2024-09-11T07:45:11.156 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:10 smithi022 ceph-mon[37330]: 2.15 scrub starts 2024-09-11T07:45:11.156 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:10 smithi022 ceph-mon[37330]: 2.15 scrub ok 2024-09-11T07:45:11.156 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:10 smithi022 ceph-mon[37330]: 2.13 scrub starts 2024-09-11T07:45:11.156 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:10 smithi022 ceph-mon[37330]: 2.13 scrub ok 2024-09-11T07:45:11.156 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:10 smithi022 ceph-mon[37330]: 10.f scrub starts 2024-09-11T07:45:11.156 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:10 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:45:11.156 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:10 smithi022 ceph-mon[37330]: pgmap v864: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 1.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 26 KiB/s rd, 173 KiB/s wr, 1.49k op/s; 190 KiB/s, 0 objects/s recovering 2024-09-11T07:45:11.255 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:11.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:11.284 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:11.285 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:11.386 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:11.413 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:11.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:11.414 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:11.465 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-11T07:45:11.515 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:11.541 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:11.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:11.542 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:11.561 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:45:11.610 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:45:11.643 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:11.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:11.701 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:11.701 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:11.802 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:11.830 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:11.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:11.831 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:11.932 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:11.966 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:11.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:11.967 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:12.062 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:11 smithi022 ceph-mon[29462]: 10.f scrub ok 2024-09-11T07:45:12.062 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:11 smithi022 ceph-mon[29462]: 2.c scrub starts 2024-09-11T07:45:12.062 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:11 smithi022 ceph-mon[29462]: 2.c scrub ok 2024-09-11T07:45:12.063 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:11 smithi022 ceph-mon[29462]: 10.e scrub starts 2024-09-11T07:45:12.063 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:11 smithi022 ceph-mon[37330]: 10.f scrub ok 2024-09-11T07:45:12.063 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:11 smithi022 ceph-mon[37330]: 2.c scrub starts 2024-09-11T07:45:12.063 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:11 smithi022 ceph-mon[37330]: 2.c scrub ok 2024-09-11T07:45:12.063 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:11 smithi022 ceph-mon[37330]: 10.e scrub starts 2024-09-11T07:45:12.067 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:12.101 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:12.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:12.102 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:12.203 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:12.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:12.238 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:12.238 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:12.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:11 smithi104 ceph-mon[33507]: 10.f scrub ok 2024-09-11T07:45:12.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:11 smithi104 ceph-mon[33507]: 2.c scrub starts 2024-09-11T07:45:12.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:11 smithi104 ceph-mon[33507]: 2.c scrub ok 2024-09-11T07:45:12.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:11 smithi104 ceph-mon[33507]: 10.e scrub starts 2024-09-11T07:45:12.339 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:12.369 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:12.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:12.370 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:12.471 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:12.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:12.503 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:12.503 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:12.604 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:12.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:12.646 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:12.646 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:12.747 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:12.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:12.780 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:12.780 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:12.881 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:12.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:12.918 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:12.918 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:13.019 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:13.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:13.053 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:13.053 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:13.154 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:13.162 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:12 smithi022 ceph-mon[37330]: 4.b scrub starts 2024-09-11T07:45:13.162 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:12 smithi022 ceph-mon[37330]: 4.b scrub ok 2024-09-11T07:45:13.162 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:12 smithi022 ceph-mon[37330]: 10.e scrub ok 2024-09-11T07:45:13.162 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:12 smithi022 ceph-mon[37330]: pgmap v865: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 1.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 22 KiB/s rd, 148 KiB/s wr, 1.28k op/s; 162 KiB/s, 0 objects/s recovering 2024-09-11T07:45:13.163 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:12 smithi022 ceph-mon[37330]: 5.19 scrub starts 2024-09-11T07:45:13.163 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:12 smithi022 ceph-mon[37330]: 5.19 scrub ok 2024-09-11T07:45:13.163 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:12 smithi022 ceph-mon[29462]: 4.b scrub starts 2024-09-11T07:45:13.163 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:12 smithi022 ceph-mon[29462]: 4.b scrub ok 2024-09-11T07:45:13.163 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:12 smithi022 ceph-mon[29462]: 10.e scrub ok 2024-09-11T07:45:13.163 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:12 smithi022 ceph-mon[29462]: pgmap v865: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 1.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 22 KiB/s rd, 148 KiB/s wr, 1.28k op/s; 162 KiB/s, 0 objects/s recovering 2024-09-11T07:45:13.163 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:12 smithi022 ceph-mon[29462]: 5.19 scrub starts 2024-09-11T07:45:13.163 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:12 smithi022 ceph-mon[29462]: 5.19 scrub ok 2024-09-11T07:45:13.185 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:13.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:13.186 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:13.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:12 smithi104 ceph-mon[33507]: 4.b scrub starts 2024-09-11T07:45:13.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:12 smithi104 ceph-mon[33507]: 4.b scrub ok 2024-09-11T07:45:13.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:12 smithi104 ceph-mon[33507]: 10.e scrub ok 2024-09-11T07:45:13.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:12 smithi104 ceph-mon[33507]: pgmap v865: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 1.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 22 KiB/s rd, 148 KiB/s wr, 1.28k op/s; 162 KiB/s, 0 objects/s recovering 2024-09-11T07:45:13.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:12 smithi104 ceph-mon[33507]: 5.19 scrub starts 2024-09-11T07:45:13.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:12 smithi104 ceph-mon[33507]: 5.19 scrub ok 2024-09-11T07:45:13.287 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:13.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:13.320 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:13.321 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:13.421 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:13.423 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:45:13.424 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:45:13.424 INFO:tasks.thrashosds.thrasher:Removing osd 4, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:45:13.424 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd out 4 2024-09-11T07:45:13.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:13.462 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:13.462 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:13.563 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:13.596 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:13.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:13.597 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:13.698 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:13.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:13.728 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:13.729 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:13.830 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:13.872 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:13.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:13.873 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:13.974 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:14.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:14.017 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:14.017 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:14.042 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:13 smithi104 ceph-mon[33507]: 2.4 scrub starts 2024-09-11T07:45:14.042 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:13 smithi104 ceph-mon[33507]: 2.4 scrub ok 2024-09-11T07:45:14.061 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:13 smithi022 ceph-mon[29462]: 2.4 scrub starts 2024-09-11T07:45:14.061 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:13 smithi022 ceph-mon[29462]: 2.4 scrub ok 2024-09-11T07:45:14.061 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:13 smithi022 ceph-mon[37330]: 2.4 scrub starts 2024-09-11T07:45:14.061 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:13 smithi022 ceph-mon[37330]: 2.4 scrub ok 2024-09-11T07:45:14.118 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:14.155 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:14.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:14.156 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:14.257 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:14.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:14.295 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:14.295 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:14.396 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:14.432 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:14.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:14.433 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:14.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:45:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:45:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:45:14.534 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:14.559 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:14.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:14.560 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:14.661 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:14.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:14.703 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:14.704 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:14.804 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:14.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:14.844 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:14.844 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:14.945 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:14.988 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:14.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:14.989 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:15.090 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:15.119 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:15.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:15.120 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:15.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:14 smithi022 ceph-mon[29462]: 6.6 scrub starts 2024-09-11T07:45:15.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:14 smithi022 ceph-mon[29462]: 6.6 scrub ok 2024-09-11T07:45:15.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:14 smithi022 ceph-mon[29462]: 4.3 scrub starts 2024-09-11T07:45:15.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:14 smithi022 ceph-mon[29462]: 4.3 scrub ok 2024-09-11T07:45:15.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:14 smithi022 ceph-mon[29462]: 4.1c scrub starts 2024-09-11T07:45:15.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:14 smithi022 ceph-mon[29462]: 4.1c scrub ok 2024-09-11T07:45:15.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:14 smithi022 ceph-mon[29462]: 10.d scrub starts 2024-09-11T07:45:15.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:14 smithi022 ceph-mon[29462]: pgmap v866: 158 pgs: 2 active+clean+scrubbing, 156 active+clean; 1.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 7.3 KiB/s rd, 44 KiB/s wr, 1.32k op/s; 63 KiB/s, 0 objects/s recovering 2024-09-11T07:45:15.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:14 smithi022 ceph-mon[37330]: 6.6 scrub starts 2024-09-11T07:45:15.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:14 smithi022 ceph-mon[37330]: 6.6 scrub ok 2024-09-11T07:45:15.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:14 smithi022 ceph-mon[37330]: 4.3 scrub starts 2024-09-11T07:45:15.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:14 smithi022 ceph-mon[37330]: 4.3 scrub ok 2024-09-11T07:45:15.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:14 smithi022 ceph-mon[37330]: 4.1c scrub starts 2024-09-11T07:45:15.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:14 smithi022 ceph-mon[37330]: 4.1c scrub ok 2024-09-11T07:45:15.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:14 smithi022 ceph-mon[37330]: 10.d scrub starts 2024-09-11T07:45:15.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:14 smithi022 ceph-mon[37330]: pgmap v866: 158 pgs: 2 active+clean+scrubbing, 156 active+clean; 1.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 7.3 KiB/s rd, 44 KiB/s wr, 1.32k op/s; 63 KiB/s, 0 objects/s recovering 2024-09-11T07:45:15.221 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:15.256 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:15.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:15.259 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:15.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:14 smithi104 ceph-mon[33507]: 6.6 scrub starts 2024-09-11T07:45:15.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:14 smithi104 ceph-mon[33507]: 6.6 scrub ok 2024-09-11T07:45:15.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:14 smithi104 ceph-mon[33507]: 4.3 scrub starts 2024-09-11T07:45:15.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:14 smithi104 ceph-mon[33507]: 4.3 scrub ok 2024-09-11T07:45:15.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:14 smithi104 ceph-mon[33507]: 4.1c scrub starts 2024-09-11T07:45:15.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:14 smithi104 ceph-mon[33507]: 4.1c scrub ok 2024-09-11T07:45:15.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:14 smithi104 ceph-mon[33507]: 10.d scrub starts 2024-09-11T07:45:15.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:14 smithi104 ceph-mon[33507]: pgmap v866: 158 pgs: 2 active+clean+scrubbing, 156 active+clean; 1.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 7.3 KiB/s rd, 44 KiB/s wr, 1.32k op/s; 63 KiB/s, 0 objects/s recovering 2024-09-11T07:45:15.360 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:15.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:15.416 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:15.416 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:15.517 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:15.556 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:15.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:15.557 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:15.565 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:45:15.658 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:15.690 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:15.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:15.691 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:15.792 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:15.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:15.835 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:15.835 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:15.936 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:15.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:15.975 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:15.976 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:16.076 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:16.098 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:15 smithi022 ceph-mon[29462]: 6.7 scrub starts 2024-09-11T07:45:16.098 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:15 smithi022 ceph-mon[29462]: 6.7 scrub ok 2024-09-11T07:45:16.098 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:15 smithi022 ceph-mon[29462]: 10.d scrub ok 2024-09-11T07:45:16.099 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:15 smithi022 ceph-mon[29462]: 4.12 scrub starts 2024-09-11T07:45:16.099 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:15 smithi022 ceph-mon[29462]: 4.12 scrub ok 2024-09-11T07:45:16.099 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:15 smithi022 ceph-mon[37330]: 6.7 scrub starts 2024-09-11T07:45:16.099 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:15 smithi022 ceph-mon[37330]: 6.7 scrub ok 2024-09-11T07:45:16.099 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:15 smithi022 ceph-mon[37330]: 10.d scrub ok 2024-09-11T07:45:16.099 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:15 smithi022 ceph-mon[37330]: 4.12 scrub starts 2024-09-11T07:45:16.099 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:15 smithi022 ceph-mon[37330]: 4.12 scrub ok 2024-09-11T07:45:16.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:16.109 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:16.109 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:16.210 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:16.218 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:45:16.219 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:45:16.219 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:45:16.219 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:45:16.219 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:45:16.219 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:45:16.242 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:16.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:16.243 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:16.247 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:45:16.247 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:45:16.279 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:45:16.280 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:45:16.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:15 smithi104 ceph-mon[33507]: 6.7 scrub starts 2024-09-11T07:45:16.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:15 smithi104 ceph-mon[33507]: 6.7 scrub ok 2024-09-11T07:45:16.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:15 smithi104 ceph-mon[33507]: 10.d scrub ok 2024-09-11T07:45:16.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:15 smithi104 ceph-mon[33507]: 4.12 scrub starts 2024-09-11T07:45:16.281 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:15 smithi104 ceph-mon[33507]: 4.12 scrub ok 2024-09-11T07:45:16.308 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:45:16.308 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:45:16.342 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:45:16.342 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:45:16.343 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:16.374 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:16.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:16.375 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:16.379 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:45:16.380 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:45:16.475 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:16.514 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:16.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:16.515 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:16.616 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:16.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:16.653 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:16.653 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:16.754 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:16.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:16.791 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:16.791 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:16.892 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:16.926 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:16.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:16.927 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:17.028 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:17.063 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:17.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:17.064 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:17.165 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:17.166 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:16 smithi022 ceph-mon[37330]: 2.6 scrub starts 2024-09-11T07:45:17.166 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:16 smithi022 ceph-mon[37330]: 2.6 scrub ok 2024-09-11T07:45:17.166 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:16 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1115006598' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:45:17.166 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:16 smithi022 ceph-mon[37330]: pgmap v867: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 1.3 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T07:45:17.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:16 smithi022 ceph-mon[29462]: 2.6 scrub starts 2024-09-11T07:45:17.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:16 smithi022 ceph-mon[29462]: 2.6 scrub ok 2024-09-11T07:45:17.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:16 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1115006598' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:45:17.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:16 smithi022 ceph-mon[29462]: pgmap v867: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 1.3 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T07:45:17.202 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:17.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:17.203 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:17.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:16 smithi104 ceph-mon[33507]: 2.6 scrub starts 2024-09-11T07:45:17.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:16 smithi104 ceph-mon[33507]: 2.6 scrub ok 2024-09-11T07:45:17.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:16 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1115006598' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:45:17.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:16 smithi104 ceph-mon[33507]: pgmap v867: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 1.3 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T07:45:17.304 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:17.333 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:17.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:17.334 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:17.435 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:17.469 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:17.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:17.470 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:17.571 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:17.595 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:17.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:17.596 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:17.697 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:17.732 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:17.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:17.733 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:17.836 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:17.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:17.876 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:17.876 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:17.977 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:17.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:18.000 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:18.000 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:18.100 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:18.134 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:18.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:18.135 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:18.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:17 smithi022 ceph-mon[29462]: 10.0 scrub starts 2024-09-11T07:45:18.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:17 smithi022 ceph-mon[37330]: 10.0 scrub starts 2024-09-11T07:45:18.236 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:18.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:18.273 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:18.273 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:18.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:17 smithi104 ceph-mon[33507]: 10.0 scrub starts 2024-09-11T07:45:18.374 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:18.399 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:18.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:18.400 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:18.501 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:18.536 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:18.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:18.538 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:18.638 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:18.659 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:18.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:18.660 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:18.761 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:18.791 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:18.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:18.792 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:18.893 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:18.933 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:18.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:18.934 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:19.035 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:19.078 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:19.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:19.079 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:19.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:18 smithi022 ceph-mon[29462]: 4.7 scrub starts 2024-09-11T07:45:19.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:18 smithi022 ceph-mon[29462]: 4.7 scrub ok 2024-09-11T07:45:19.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:18 smithi022 ceph-mon[29462]: 10.0 scrub ok 2024-09-11T07:45:19.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:18 smithi022 ceph-mon[29462]: 6.c scrub starts 2024-09-11T07:45:19.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:18 smithi022 ceph-mon[29462]: 6.c scrub ok 2024-09-11T07:45:19.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:18 smithi022 ceph-mon[29462]: pgmap v868: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 1.3 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 881 B/s rd, 0 B/s wr, 1.13k op/s 2024-09-11T07:45:19.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:18 smithi022 ceph-mon[29462]: 6.14 scrub starts 2024-09-11T07:45:19.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:18 smithi022 ceph-mon[29462]: 6.14 scrub ok 2024-09-11T07:45:19.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:18 smithi022 ceph-mon[37330]: 4.7 scrub starts 2024-09-11T07:45:19.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:18 smithi022 ceph-mon[37330]: 4.7 scrub ok 2024-09-11T07:45:19.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:18 smithi022 ceph-mon[37330]: 10.0 scrub ok 2024-09-11T07:45:19.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:18 smithi022 ceph-mon[37330]: 6.c scrub starts 2024-09-11T07:45:19.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:18 smithi022 ceph-mon[37330]: 6.c scrub ok 2024-09-11T07:45:19.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:18 smithi022 ceph-mon[37330]: pgmap v868: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 1.3 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 881 B/s rd, 0 B/s wr, 1.13k op/s 2024-09-11T07:45:19.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:18 smithi022 ceph-mon[37330]: 6.14 scrub starts 2024-09-11T07:45:19.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:18 smithi022 ceph-mon[37330]: 6.14 scrub ok 2024-09-11T07:45:19.179 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:19.205 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:19.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:19.206 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:19.216 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:18 smithi104 ceph-mon[33507]: 4.7 scrub starts 2024-09-11T07:45:19.216 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:18 smithi104 ceph-mon[33507]: 4.7 scrub ok 2024-09-11T07:45:19.217 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:18 smithi104 ceph-mon[33507]: 10.0 scrub ok 2024-09-11T07:45:19.217 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:18 smithi104 ceph-mon[33507]: 6.c scrub starts 2024-09-11T07:45:19.217 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:18 smithi104 ceph-mon[33507]: 6.c scrub ok 2024-09-11T07:45:19.217 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:18 smithi104 ceph-mon[33507]: pgmap v868: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 1.3 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 881 B/s rd, 0 B/s wr, 1.13k op/s 2024-09-11T07:45:19.217 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:18 smithi104 ceph-mon[33507]: 6.14 scrub starts 2024-09-11T07:45:19.217 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:18 smithi104 ceph-mon[33507]: 6.14 scrub ok 2024-09-11T07:45:19.308 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:19.341 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:19.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:19.342 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:19.443 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:19.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:19.470 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:19.470 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:19.570 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:19.592 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:19.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:19.593 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:19.694 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:19.731 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:19.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:19.732 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:19.833 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:19.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:19.877 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:19.877 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:19.978 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:20.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:20.018 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:20.018 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:20.082 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:19 smithi104 ceph-mon[33507]: 2.3 scrub starts 2024-09-11T07:45:20.082 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:19 smithi104 ceph-mon[33507]: 2.3 scrub ok 2024-09-11T07:45:20.082 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:19 smithi104 ceph-mon[33507]: 4.17 scrub starts 2024-09-11T07:45:20.082 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:19 smithi104 ceph-mon[33507]: 4.17 scrub ok 2024-09-11T07:45:20.119 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:20.152 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:20.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:20.153 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:20.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:19 smithi022 ceph-mon[29462]: 2.3 scrub starts 2024-09-11T07:45:20.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:19 smithi022 ceph-mon[29462]: 2.3 scrub ok 2024-09-11T07:45:20.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:19 smithi022 ceph-mon[29462]: 4.17 scrub starts 2024-09-11T07:45:20.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:19 smithi022 ceph-mon[29462]: 4.17 scrub ok 2024-09-11T07:45:20.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:19 smithi022 ceph-mon[37330]: 2.3 scrub starts 2024-09-11T07:45:20.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:19 smithi022 ceph-mon[37330]: 2.3 scrub ok 2024-09-11T07:45:20.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:19 smithi022 ceph-mon[37330]: 4.17 scrub starts 2024-09-11T07:45:20.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:19 smithi022 ceph-mon[37330]: 4.17 scrub ok 2024-09-11T07:45:20.254 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:20.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:20.301 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:20.301 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:20.402 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:20.433 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:20.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:20.434 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:20.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:45:20 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:45:20.536 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:20.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:20.574 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:20.574 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:20.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:45:20 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:45:20] "GET /metrics HTTP/1.1" 200 225009 "" "Prometheus/2.43.0" 2024-09-11T07:45:20.675 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:20.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:20.718 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:20.718 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:20.818 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:20.862 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:20.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:20.863 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:20.964 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:20.992 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:20.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:20.993 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:21.094 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:21.121 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:21.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:21.122 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:21.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:20 smithi022 ceph-mon[29462]: 6.d scrub starts 2024-09-11T07:45:21.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:20 smithi022 ceph-mon[29462]: 6.d scrub ok 2024-09-11T07:45:21.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:20 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:45:21.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:20 smithi022 ceph-mon[29462]: pgmap v869: 158 pgs: 158 active+clean; 900 MiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.64k op/s 2024-09-11T07:45:21.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:20 smithi022 ceph-mon[29462]: 3.0 scrub starts 2024-09-11T07:45:21.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:20 smithi022 ceph-mon[29462]: 3.0 scrub ok 2024-09-11T07:45:21.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:20 smithi022 ceph-mon[37330]: 6.d scrub starts 2024-09-11T07:45:21.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:20 smithi022 ceph-mon[37330]: 6.d scrub ok 2024-09-11T07:45:21.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:20 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:45:21.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:20 smithi022 ceph-mon[37330]: pgmap v869: 158 pgs: 158 active+clean; 900 MiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.64k op/s 2024-09-11T07:45:21.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:20 smithi022 ceph-mon[37330]: 3.0 scrub starts 2024-09-11T07:45:21.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:20 smithi022 ceph-mon[37330]: 3.0 scrub ok 2024-09-11T07:45:21.222 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:21.260 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:21.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:21.261 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:21.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:20 smithi104 ceph-mon[33507]: 6.d scrub starts 2024-09-11T07:45:21.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:20 smithi104 ceph-mon[33507]: 6.d scrub ok 2024-09-11T07:45:21.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:20 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:45:21.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:20 smithi104 ceph-mon[33507]: pgmap v869: 158 pgs: 158 active+clean; 900 MiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.64k op/s 2024-09-11T07:45:21.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:20 smithi104 ceph-mon[33507]: 3.0 scrub starts 2024-09-11T07:45:21.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:20 smithi104 ceph-mon[33507]: 3.0 scrub ok 2024-09-11T07:45:21.362 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:21.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:21.399 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:21.399 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:21.500 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:21.525 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:45:21.529 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:21.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:21.530 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:21.631 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:21.668 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:21.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:21.669 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:21.770 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:21.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:21.808 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:21.808 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:21.864 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is unset 2024-09-11T07:45:21.909 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:21.948 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:21.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:21.949 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:22.050 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:22.053 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:21 smithi022 ceph-mon[29462]: 6.17 scrub starts 2024-09-11T07:45:22.053 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:21 smithi022 ceph-mon[29462]: 6.17 scrub ok 2024-09-11T07:45:22.054 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:21 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4143731362' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:45:22.054 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:21 smithi022 ceph-mon[37330]: 6.17 scrub starts 2024-09-11T07:45:22.054 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:21 smithi022 ceph-mon[37330]: 6.17 scrub ok 2024-09-11T07:45:22.054 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:21 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4143731362' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:45:22.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:22.085 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:22.085 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:22.186 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:22.227 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:22.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:22.228 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:22.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:21 smithi104 ceph-mon[33507]: 6.17 scrub starts 2024-09-11T07:45:22.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:21 smithi104 ceph-mon[33507]: 6.17 scrub ok 2024-09-11T07:45:22.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:21 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4143731362' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:45:22.329 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:22.365 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:22.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:22.366 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:22.467 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:22.500 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:22.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:22.501 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:22.602 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:22.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:22.624 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:22.625 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:22.725 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:22.738 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:45:22.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:22.760 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:22.761 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:22.861 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:22.890 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:22.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:22.891 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:22.992 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:23.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:23.031 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:23.031 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:23.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:22 smithi022 ceph-mon[29462]: 5.3 scrub starts 2024-09-11T07:45:23.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:22 smithi022 ceph-mon[29462]: 5.3 scrub ok 2024-09-11T07:45:23.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:22 smithi022 ceph-mon[29462]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:45:23.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:22 smithi022 ceph-mon[29462]: Cluster is now healthy 2024-09-11T07:45:23.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:22 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4143731362' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:45:23.035 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:22 smithi022 ceph-mon[29462]: osdmap e393: 8 total, 8 up, 8 in 2024-09-11T07:45:23.035 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:22 smithi022 ceph-mon[29462]: 10.1 deep-scrub starts 2024-09-11T07:45:23.035 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:22 smithi022 ceph-mon[29462]: pgmap v871: 158 pgs: 158 active+clean; 900 MiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.30k op/s 2024-09-11T07:45:23.035 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:22 smithi022 ceph-mon[29462]: 5.4 scrub starts 2024-09-11T07:45:23.035 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:22 smithi022 ceph-mon[29462]: 5.4 scrub ok 2024-09-11T07:45:23.035 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:22 smithi022 ceph-mon[37330]: 5.3 scrub starts 2024-09-11T07:45:23.035 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:22 smithi022 ceph-mon[37330]: 5.3 scrub ok 2024-09-11T07:45:23.035 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:22 smithi022 ceph-mon[37330]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:45:23.035 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:22 smithi022 ceph-mon[37330]: Cluster is now healthy 2024-09-11T07:45:23.035 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:22 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4143731362' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:45:23.035 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:22 smithi022 ceph-mon[37330]: osdmap e393: 8 total, 8 up, 8 in 2024-09-11T07:45:23.035 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:22 smithi022 ceph-mon[37330]: 10.1 deep-scrub starts 2024-09-11T07:45:23.036 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:22 smithi022 ceph-mon[37330]: pgmap v871: 158 pgs: 158 active+clean; 900 MiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.30k op/s 2024-09-11T07:45:23.036 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:22 smithi022 ceph-mon[37330]: 5.4 scrub starts 2024-09-11T07:45:23.036 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:22 smithi022 ceph-mon[37330]: 5.4 scrub ok 2024-09-11T07:45:23.132 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:23.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:23.174 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:23.174 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:23.275 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:23.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:22 smithi104 ceph-mon[33507]: 5.3 scrub starts 2024-09-11T07:45:23.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:22 smithi104 ceph-mon[33507]: 5.3 scrub ok 2024-09-11T07:45:23.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:22 smithi104 ceph-mon[33507]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:45:23.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:22 smithi104 ceph-mon[33507]: Cluster is now healthy 2024-09-11T07:45:23.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:22 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4143731362' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:45:23.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:22 smithi104 ceph-mon[33507]: osdmap e393: 8 total, 8 up, 8 in 2024-09-11T07:45:23.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:22 smithi104 ceph-mon[33507]: 10.1 deep-scrub starts 2024-09-11T07:45:23.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:22 smithi104 ceph-mon[33507]: pgmap v871: 158 pgs: 158 active+clean; 900 MiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.30k op/s 2024-09-11T07:45:23.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:22 smithi104 ceph-mon[33507]: 5.4 scrub starts 2024-09-11T07:45:23.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:22 smithi104 ceph-mon[33507]: 5.4 scrub ok 2024-09-11T07:45:23.306 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:23.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:23.307 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:23.408 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:23.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:23.446 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:23.446 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:23.547 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:23.578 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:23.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:23.579 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:23.680 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:23.717 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:23.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:23.718 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:23.819 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:23.855 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:23.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:23.856 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:23.957 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:23.984 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:23.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:23.985 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:24.086 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:24.124 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:24.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:24.125 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:24.139 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:24 smithi022 ceph-mon[37330]: 6.9 deep-scrub starts 2024-09-11T07:45:24.140 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:24 smithi022 ceph-mon[37330]: 6.9 deep-scrub ok 2024-09-11T07:45:24.140 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:24 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:45:24.140 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:24 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:45:24.140 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:24 smithi022 ceph-mon[37330]: 4.1f scrub starts 2024-09-11T07:45:24.140 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:24 smithi022 ceph-mon[37330]: 4.1f scrub ok 2024-09-11T07:45:24.140 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:24 smithi022 ceph-mon[37330]: 6.1 scrub starts 2024-09-11T07:45:24.140 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:24 smithi022 ceph-mon[37330]: 6.1 scrub ok 2024-09-11T07:45:24.140 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:24 smithi022 ceph-mon[29462]: 6.9 deep-scrub starts 2024-09-11T07:45:24.140 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:24 smithi022 ceph-mon[29462]: 6.9 deep-scrub ok 2024-09-11T07:45:24.141 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:24 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:45:24.141 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:24 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:45:24.141 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:24 smithi022 ceph-mon[29462]: 4.1f scrub starts 2024-09-11T07:45:24.141 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:24 smithi022 ceph-mon[29462]: 4.1f scrub ok 2024-09-11T07:45:24.141 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:24 smithi022 ceph-mon[29462]: 6.1 scrub starts 2024-09-11T07:45:24.141 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:24 smithi022 ceph-mon[29462]: 6.1 scrub ok 2024-09-11T07:45:24.190 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:45:24.190 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:45:24.226 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:24.271 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:24.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:24.272 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:24.373 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:24.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:24.411 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:24.411 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:24.512 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:24.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:45:24 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:45:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:45:24.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:24 smithi104 ceph-mon[33507]: 6.9 deep-scrub starts 2024-09-11T07:45:24.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:24 smithi104 ceph-mon[33507]: 6.9 deep-scrub ok 2024-09-11T07:45:24.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:24 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:45:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:24 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:45:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:24 smithi104 ceph-mon[33507]: 4.1f scrub starts 2024-09-11T07:45:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:24 smithi104 ceph-mon[33507]: 4.1f scrub ok 2024-09-11T07:45:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:24 smithi104 ceph-mon[33507]: 6.1 scrub starts 2024-09-11T07:45:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:24 smithi104 ceph-mon[33507]: 6.1 scrub ok 2024-09-11T07:45:24.546 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:24.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:24.547 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:24.647 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:24.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:24.680 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:24.680 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:24.780 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:24.809 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:24.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:24.810 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:24.911 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:24.943 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:24.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:24.944 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:25.045 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:25.077 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:25.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:25.078 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:25.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:25 smithi022 ceph-mon[29462]: pgmap v872: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 559 MiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.28k op/s 2024-09-11T07:45:25.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:25 smithi022 ceph-mon[37330]: pgmap v872: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 559 MiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.28k op/s 2024-09-11T07:45:25.180 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:25.215 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:25.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:25.216 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:25.317 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:25.347 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:25.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:25.348 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:25.374 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T07:45:25.449 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:25.478 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:25.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:25.479 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:25.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:25 smithi104 ceph-mon[33507]: pgmap v872: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 559 MiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.28k op/s 2024-09-11T07:45:25.548 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:45:25.571 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-11T07:45:25.580 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:25.636 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:25 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-a[29458]: 2024-09-11T07:45:25.327+0000 7fde5d00f640 -1 mon.a@0(leader).osd e393 definitely_dead 0 2024-09-11T07:45:25.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:25.677 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:25.677 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:25.778 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:25.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:25.805 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:25.806 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:25.906 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:25.935 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:25.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:25.936 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:26.037 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:26.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:26.088 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:26.089 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:26.089 INFO:teuthology.orchestra.run.smithi022.stderr:marked out osd.4. 2024-09-11T07:45:26.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:26 smithi022 ceph-mon[29462]: 4.1d scrub starts 2024-09-11T07:45:26.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:26 smithi022 ceph-mon[29462]: 4.1d scrub ok 2024-09-11T07:45:26.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:26 smithi022 ceph-mon[29462]: 6.11 scrub starts 2024-09-11T07:45:26.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:26 smithi022 ceph-mon[29462]: 6.11 scrub ok 2024-09-11T07:45:26.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:26 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2970794656' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["4"]}]: dispatch 2024-09-11T07:45:26.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:26 smithi022 ceph-mon[29462]: from='client.25906 ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["4"]}]: dispatch 2024-09-11T07:45:26.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:26 smithi022 ceph-mon[29462]: Client client.admin marked osd.4 out, while it was still marked up 2024-09-11T07:45:26.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:26 smithi022 ceph-mon[37330]: 4.1d scrub starts 2024-09-11T07:45:26.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:26 smithi022 ceph-mon[37330]: 4.1d scrub ok 2024-09-11T07:45:26.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:26 smithi022 ceph-mon[37330]: 6.11 scrub starts 2024-09-11T07:45:26.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:26 smithi022 ceph-mon[37330]: 6.11 scrub ok 2024-09-11T07:45:26.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:26 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2970794656' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["4"]}]: dispatch 2024-09-11T07:45:26.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:26 smithi022 ceph-mon[37330]: from='client.25906 ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["4"]}]: dispatch 2024-09-11T07:45:26.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:26 smithi022 ceph-mon[37330]: Client client.admin marked osd.4 out, while it was still marked up 2024-09-11T07:45:26.189 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:26.210 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:26.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:26.211 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:26.312 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:26.333 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:26.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:26.334 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:26.435 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:26.458 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:26.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:26.459 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:26.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:26 smithi104 ceph-mon[33507]: 4.1d scrub starts 2024-09-11T07:45:26.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:26 smithi104 ceph-mon[33507]: 4.1d scrub ok 2024-09-11T07:45:26.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:26 smithi104 ceph-mon[33507]: 6.11 scrub starts 2024-09-11T07:45:26.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:26 smithi104 ceph-mon[33507]: 6.11 scrub ok 2024-09-11T07:45:26.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:26 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2970794656' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["4"]}]: dispatch 2024-09-11T07:45:26.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:26 smithi104 ceph-mon[33507]: from='client.25906 ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["4"]}]: dispatch 2024-09-11T07:45:26.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:26 smithi104 ceph-mon[33507]: Client client.admin marked osd.4 out, while it was still marked up 2024-09-11T07:45:26.560 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:26.582 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:26.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:26.583 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:26.684 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:26.706 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:26.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:26.707 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:26.808 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:26.833 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:26.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:26.834 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:26.935 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:26.958 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:26.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:26.959 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:27.060 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:27.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:27.090 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:27.091 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:27.191 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:27.213 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:27.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:27.214 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:27.315 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:27.338 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:27.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:27.339 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:27.381 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:27 smithi022 ceph-mon[29462]: 10.1 deep-scrub ok 2024-09-11T07:45:27.381 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:27 smithi022 ceph-mon[29462]: from='client.25906 ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["4"]}]': finished 2024-09-11T07:45:27.381 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:27 smithi022 ceph-mon[29462]: osdmap e394: 8 total, 8 up, 7 in 2024-09-11T07:45:27.381 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:27 smithi022 ceph-mon[29462]: pgmap v874: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 512 MiB data, 7.3 GiB used, 619 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.59k op/s 2024-09-11T07:45:27.381 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:27 smithi022 ceph-mon[37330]: 10.1 deep-scrub ok 2024-09-11T07:45:27.381 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:27 smithi022 ceph-mon[37330]: from='client.25906 ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["4"]}]': finished 2024-09-11T07:45:27.382 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:27 smithi022 ceph-mon[37330]: osdmap e394: 8 total, 8 up, 7 in 2024-09-11T07:45:27.382 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:27 smithi022 ceph-mon[37330]: pgmap v874: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 512 MiB data, 7.3 GiB used, 619 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.59k op/s 2024-09-11T07:45:27.439 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:27.463 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:27.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:27.464 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:27.471 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:45:27.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:27 smithi104 ceph-mon[33507]: 10.1 deep-scrub ok 2024-09-11T07:45:27.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:27 smithi104 ceph-mon[33507]: from='client.25906 ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["4"]}]': finished 2024-09-11T07:45:27.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:27 smithi104 ceph-mon[33507]: osdmap e394: 8 total, 8 up, 7 in 2024-09-11T07:45:27.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:27 smithi104 ceph-mon[33507]: pgmap v874: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 512 MiB data, 7.3 GiB used, 619 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.59k op/s 2024-09-11T07:45:27.565 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:27.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:27.590 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:27.590 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:27.691 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:27.713 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:27.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:27.714 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:27.815 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:27.837 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:27.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:27.838 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:27.939 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:27.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:27.963 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:27.964 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:28.064 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:28.094 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:28.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:28.095 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:28.196 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:28.252 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:28.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:28.253 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:28.354 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:28.386 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:28.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:28.388 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:28.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:28 smithi022 ceph-mon[29462]: 6.a scrub starts 2024-09-11T07:45:28.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:28 smithi022 ceph-mon[29462]: 6.a scrub ok 2024-09-11T07:45:28.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:28 smithi022 ceph-mon[29462]: osdmap e395: 8 total, 8 up, 7 in 2024-09-11T07:45:28.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:28 smithi022 ceph-mon[29462]: osdmap e396: 8 total, 8 up, 7 in 2024-09-11T07:45:28.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:28 smithi022 ceph-mon[37330]: 6.a scrub starts 2024-09-11T07:45:28.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:28 smithi022 ceph-mon[37330]: 6.a scrub ok 2024-09-11T07:45:28.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:28 smithi022 ceph-mon[37330]: osdmap e395: 8 total, 8 up, 7 in 2024-09-11T07:45:28.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:28 smithi022 ceph-mon[37330]: osdmap e396: 8 total, 8 up, 7 in 2024-09-11T07:45:28.489 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:28.526 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:28.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:28.528 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:28.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:28 smithi104 ceph-mon[33507]: 6.a scrub starts 2024-09-11T07:45:28.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:28 smithi104 ceph-mon[33507]: 6.a scrub ok 2024-09-11T07:45:28.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:28 smithi104 ceph-mon[33507]: osdmap e395: 8 total, 8 up, 7 in 2024-09-11T07:45:28.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:28 smithi104 ceph-mon[33507]: osdmap e396: 8 total, 8 up, 7 in 2024-09-11T07:45:28.629 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:28.666 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:28.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:28.667 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:28.768 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:28.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:28.831 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:28.832 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:28.855 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:45:28.856 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:45:28.888 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:45:28.889 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:45:28.929 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:45:28.930 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:45:28.933 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:28.961 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:45:28.961 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:45:28.975 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:28.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:28.977 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:28.997 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:45:28.997 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:45:29.031 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:45:29.031 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:45:29.066 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:45:29.067 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:45:29.078 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:29.105 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:45:29.105 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:45:29.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:29.126 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:29.126 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:29.227 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:29.261 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:29.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:29.263 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:29.363 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:29.383 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[29462]: 10.2 starting backfill to osd.7 from (0'0,0'0] MAX to 393'15742 2024-09-11T07:45:29.383 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[29462]: 10.4 starting backfill to osd.1 from (0'0,0'0] MAX to 393'15791 2024-09-11T07:45:29.384 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[29462]: 10.1 starting backfill to osd.1 from (0'0,0'0] MAX to 393'16420 2024-09-11T07:45:29.384 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[29462]: 10.c starting backfill to osd.6 from (0'0,0'0] MAX to 393'16036 2024-09-11T07:45:29.384 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[29462]: 10.d starting backfill to osd.3 from (0'0,0'0] MAX to 393'15612 2024-09-11T07:45:29.384 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[29462]: 10.0 starting backfill to osd.7 from (0'0,0'0] MAX to 393'15591 2024-09-11T07:45:29.384 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[29462]: 10.6 starting backfill to osd.1 from (0'0,0'0] MAX to 393'15809 2024-09-11T07:45:29.384 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[29462]: 10.3 starting backfill to osd.0 from (0'0,0'0] MAX to 393'15841 2024-09-11T07:45:29.384 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[29462]: 10.9 starting backfill to osd.3 from (0'0,0'0] MAX to 393'15569 2024-09-11T07:45:29.385 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[29462]: 10.e starting backfill to osd.5 from (0'0,0'0] MAX to 393'15731 2024-09-11T07:45:29.385 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[29462]: 10.7 starting backfill to osd.0 from (0'0,0'0] MAX to 393'15780 2024-09-11T07:45:29.385 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[29462]: pgmap v877: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 512 MiB data, 7.3 GiB used, 619 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-11T07:45:29.385 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:45:29.385 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/647364414' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:45:29.387 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[37330]: 10.2 starting backfill to osd.7 from (0'0,0'0] MAX to 393'15742 2024-09-11T07:45:29.387 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[37330]: 10.4 starting backfill to osd.1 from (0'0,0'0] MAX to 393'15791 2024-09-11T07:45:29.387 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[37330]: 10.1 starting backfill to osd.1 from (0'0,0'0] MAX to 393'16420 2024-09-11T07:45:29.387 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[37330]: 10.c starting backfill to osd.6 from (0'0,0'0] MAX to 393'16036 2024-09-11T07:45:29.387 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[37330]: 10.d starting backfill to osd.3 from (0'0,0'0] MAX to 393'15612 2024-09-11T07:45:29.388 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[37330]: 10.0 starting backfill to osd.7 from (0'0,0'0] MAX to 393'15591 2024-09-11T07:45:29.388 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[37330]: 10.6 starting backfill to osd.1 from (0'0,0'0] MAX to 393'15809 2024-09-11T07:45:29.388 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[37330]: 10.3 starting backfill to osd.0 from (0'0,0'0] MAX to 393'15841 2024-09-11T07:45:29.388 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[37330]: 10.9 starting backfill to osd.3 from (0'0,0'0] MAX to 393'15569 2024-09-11T07:45:29.388 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[37330]: 10.e starting backfill to osd.5 from (0'0,0'0] MAX to 393'15731 2024-09-11T07:45:29.389 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[37330]: 10.7 starting backfill to osd.0 from (0'0,0'0] MAX to 393'15780 2024-09-11T07:45:29.389 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[37330]: pgmap v877: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 512 MiB data, 7.3 GiB used, 619 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-11T07:45:29.389 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:45:29.389 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:29 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/647364414' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:45:29.393 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:45:29.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:29.428 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:29.428 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:29.529 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:29 smithi104 ceph-mon[33507]: 10.2 starting backfill to osd.7 from (0'0,0'0] MAX to 393'15742 2024-09-11T07:45:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:29 smithi104 ceph-mon[33507]: 10.4 starting backfill to osd.1 from (0'0,0'0] MAX to 393'15791 2024-09-11T07:45:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:29 smithi104 ceph-mon[33507]: 10.1 starting backfill to osd.1 from (0'0,0'0] MAX to 393'16420 2024-09-11T07:45:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:29 smithi104 ceph-mon[33507]: 10.c starting backfill to osd.6 from (0'0,0'0] MAX to 393'16036 2024-09-11T07:45:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:29 smithi104 ceph-mon[33507]: 10.d starting backfill to osd.3 from (0'0,0'0] MAX to 393'15612 2024-09-11T07:45:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:29 smithi104 ceph-mon[33507]: 10.0 starting backfill to osd.7 from (0'0,0'0] MAX to 393'15591 2024-09-11T07:45:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:29 smithi104 ceph-mon[33507]: 10.6 starting backfill to osd.1 from (0'0,0'0] MAX to 393'15809 2024-09-11T07:45:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:29 smithi104 ceph-mon[33507]: 10.3 starting backfill to osd.0 from (0'0,0'0] MAX to 393'15841 2024-09-11T07:45:29.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:29 smithi104 ceph-mon[33507]: 10.9 starting backfill to osd.3 from (0'0,0'0] MAX to 393'15569 2024-09-11T07:45:29.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:29 smithi104 ceph-mon[33507]: 10.e starting backfill to osd.5 from (0'0,0'0] MAX to 393'15731 2024-09-11T07:45:29.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:29 smithi104 ceph-mon[33507]: 10.7 starting backfill to osd.0 from (0'0,0'0] MAX to 393'15780 2024-09-11T07:45:29.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:29 smithi104 ceph-mon[33507]: pgmap v877: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 512 MiB data, 7.3 GiB used, 619 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-11T07:45:29.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:29 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:45:29.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:29 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/647364414' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:45:29.559 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:29.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:29.561 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:29.662 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:29.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:29.688 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:29.688 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:29.789 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:29.827 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:29.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:29.828 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:29.929 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:29.958 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:29.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:29.959 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:30.059 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:30.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:30.090 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:30.090 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:30.192 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:30.227 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:30.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:30.228 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:30.329 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:30.356 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:30.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:30.358 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:30.416 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:45:30 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:45:30] "GET /metrics HTTP/1.1" 200 224957 "" "Prometheus/2.43.0" 2024-09-11T07:45:30.459 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:30.481 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:30.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:30.482 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:30.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:45:30 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:45:30.584 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:30.606 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:30.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:30.607 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:30.678 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:45:30.678 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:45:30.678 INFO:teuthology.orchestra.run.smithi022.stdout: "num_ops": 0 2024-09-11T07:45:30.678 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:45:30.708 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:30.729 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:30.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:30.731 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:30.831 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:30.854 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:30.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:30.855 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:30.955 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:30.977 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:30.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:30.978 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:31.078 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:31.100 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:31.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:31.101 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:31.202 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:31.223 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:31.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:31.224 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:31.325 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:31.348 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:31.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:31.349 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:31.381 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:31 smithi022 ceph-mon[29462]: osdmap e397: 8 total, 8 up, 7 in 2024-09-11T07:45:31.381 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:31 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:45:31.381 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:31 smithi022 ceph-mon[29462]: pgmap v879: 158 pgs: 1 active+clean+remapped, 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 147 active+clean; 323 MiB data, 6.8 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 704 op/s; 2902/16053 objects misplaced (18.078%); 11 MiB/s, 191 objects/s recovering 2024-09-11T07:45:31.381 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:31 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:45:31.382 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:31 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:45:31.382 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:31 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:45:31.382 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:31 smithi022 ceph-mon[29462]: osdmap e398: 8 total, 8 up, 7 in 2024-09-11T07:45:31.382 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:31 smithi022 ceph-mon[37330]: osdmap e397: 8 total, 8 up, 7 in 2024-09-11T07:45:31.382 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:31 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:45:31.382 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:31 smithi022 ceph-mon[37330]: pgmap v879: 158 pgs: 1 active+clean+remapped, 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 147 active+clean; 323 MiB data, 6.8 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 704 op/s; 2902/16053 objects misplaced (18.078%); 11 MiB/s, 191 objects/s recovering 2024-09-11T07:45:31.383 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:31 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:45:31.383 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:31 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:45:31.383 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:31 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:45:31.383 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:31 smithi022 ceph-mon[37330]: osdmap e398: 8 total, 8 up, 7 in 2024-09-11T07:45:31.451 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-11T07:45:31.452 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:31.514 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:31.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:31.515 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:31.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:31 smithi104 ceph-mon[33507]: osdmap e397: 8 total, 8 up, 7 in 2024-09-11T07:45:31.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:31 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:45:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:31 smithi104 ceph-mon[33507]: pgmap v879: 158 pgs: 1 active+clean+remapped, 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 147 active+clean; 323 MiB data, 6.8 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 704 op/s; 2902/16053 objects misplaced (18.078%); 11 MiB/s, 191 objects/s recovering 2024-09-11T07:45:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:31 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:45:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:31 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:45:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:31 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:45:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:31 smithi104 ceph-mon[33507]: osdmap e398: 8 total, 8 up, 7 in 2024-09-11T07:45:31.615 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:31.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:31.638 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:31.739 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:31.767 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:31.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:31.768 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:31.847 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:45:31.847 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:45:31.847 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-11T07:45:31.847 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 0.603115 2024-09-11T07:45:31.847 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 5 0.6031152587241752 2024-09-11T07:45:31.869 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:31.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:31.952 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:31.952 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:32.053 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:32.104 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:32.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:32.106 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:32.197 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:45:32.206 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:32.288 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:32.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:32.289 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:32.390 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:32.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:32 smithi022 ceph-mon[29462]: 4.1e scrub starts 2024-09-11T07:45:32.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:32 smithi022 ceph-mon[29462]: 4.1e scrub ok 2024-09-11T07:45:32.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:32 smithi022 ceph-mon[29462]: 5.f scrub starts 2024-09-11T07:45:32.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:32 smithi022 ceph-mon[29462]: 5.f scrub ok 2024-09-11T07:45:32.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:32 smithi022 ceph-mon[29462]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:45:32.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:32 smithi022 ceph-mon[29462]: 5.1f scrub starts 2024-09-11T07:45:32.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:32 smithi022 ceph-mon[29462]: 5.1f scrub ok 2024-09-11T07:45:32.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:32 smithi022 ceph-mon[29462]: osdmap e399: 8 total, 8 up, 7 in 2024-09-11T07:45:32.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:32 smithi022 ceph-mon[37330]: 4.1e scrub starts 2024-09-11T07:45:32.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:32 smithi022 ceph-mon[37330]: 4.1e scrub ok 2024-09-11T07:45:32.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:32 smithi022 ceph-mon[37330]: 5.f scrub starts 2024-09-11T07:45:32.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:32 smithi022 ceph-mon[37330]: 5.f scrub ok 2024-09-11T07:45:32.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:32 smithi022 ceph-mon[37330]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:45:32.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:32 smithi022 ceph-mon[37330]: 5.1f scrub starts 2024-09-11T07:45:32.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:32 smithi022 ceph-mon[37330]: 5.1f scrub ok 2024-09-11T07:45:32.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:32 smithi022 ceph-mon[37330]: osdmap e399: 8 total, 8 up, 7 in 2024-09-11T07:45:32.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:32.419 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:32.419 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:32.520 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:32.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:32 smithi104 ceph-mon[33507]: 4.1e scrub starts 2024-09-11T07:45:32.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:32 smithi104 ceph-mon[33507]: 4.1e scrub ok 2024-09-11T07:45:32.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:32 smithi104 ceph-mon[33507]: 5.f scrub starts 2024-09-11T07:45:32.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:32 smithi104 ceph-mon[33507]: 5.f scrub ok 2024-09-11T07:45:32.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:32 smithi104 ceph-mon[33507]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:45:32.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:32 smithi104 ceph-mon[33507]: 5.1f scrub starts 2024-09-11T07:45:32.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:32 smithi104 ceph-mon[33507]: 5.1f scrub ok 2024-09-11T07:45:32.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:32 smithi104 ceph-mon[33507]: osdmap e399: 8 total, 8 up, 7 in 2024-09-11T07:45:32.544 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:32.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:32.545 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:32.560 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:45:32.646 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:32.669 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:32.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:32.670 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:32.770 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:32.816 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:32.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:32.817 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:32.918 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:32.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:32.974 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:32.974 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:33.075 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:33.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:33.119 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:33.119 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:33.219 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:33.245 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:33.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:33.246 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:33.316 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:33 smithi022 ceph-mon[37330]: 5.1 scrub starts 2024-09-11T07:45:33.316 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:33 smithi022 ceph-mon[37330]: 5.1 scrub ok 2024-09-11T07:45:33.316 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:33 smithi022 ceph-mon[37330]: 5.11 deep-scrub starts 2024-09-11T07:45:33.316 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:33 smithi022 ceph-mon[37330]: 5.11 deep-scrub ok 2024-09-11T07:45:33.316 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:33 smithi022 ceph-mon[37330]: pgmap v882: 158 pgs: 1 active+clean+remapped, 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 147 active+clean; 323 MiB data, 6.8 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 704 op/s; 2902/16053 objects misplaced (18.078%); 11 MiB/s, 191 objects/s recovering 2024-09-11T07:45:33.316 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:33 smithi022 ceph-mon[37330]: 4.5 deep-scrub starts 2024-09-11T07:45:33.316 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:33 smithi022 ceph-mon[37330]: 4.5 deep-scrub ok 2024-09-11T07:45:33.317 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:33 smithi022 ceph-mon[29462]: 5.1 scrub starts 2024-09-11T07:45:33.317 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:33 smithi022 ceph-mon[29462]: 5.1 scrub ok 2024-09-11T07:45:33.317 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:33 smithi022 ceph-mon[29462]: 5.11 deep-scrub starts 2024-09-11T07:45:33.317 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:33 smithi022 ceph-mon[29462]: 5.11 deep-scrub ok 2024-09-11T07:45:33.317 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:33 smithi022 ceph-mon[29462]: pgmap v882: 158 pgs: 1 active+clean+remapped, 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 147 active+clean; 323 MiB data, 6.8 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 704 op/s; 2902/16053 objects misplaced (18.078%); 11 MiB/s, 191 objects/s recovering 2024-09-11T07:45:33.317 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:33 smithi022 ceph-mon[29462]: 4.5 deep-scrub starts 2024-09-11T07:45:33.317 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:33 smithi022 ceph-mon[29462]: 4.5 deep-scrub ok 2024-09-11T07:45:33.347 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:33.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:33.378 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:33.378 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:33.478 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:33.510 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:33.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:33.511 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:33.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:33 smithi104 ceph-mon[33507]: 5.1 scrub starts 2024-09-11T07:45:33.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:33 smithi104 ceph-mon[33507]: 5.1 scrub ok 2024-09-11T07:45:33.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:33 smithi104 ceph-mon[33507]: 5.11 deep-scrub starts 2024-09-11T07:45:33.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:33 smithi104 ceph-mon[33507]: 5.11 deep-scrub ok 2024-09-11T07:45:33.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:33 smithi104 ceph-mon[33507]: pgmap v882: 158 pgs: 1 active+clean+remapped, 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 147 active+clean; 323 MiB data, 6.8 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 704 op/s; 2902/16053 objects misplaced (18.078%); 11 MiB/s, 191 objects/s recovering 2024-09-11T07:45:33.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:33 smithi104 ceph-mon[33507]: 4.5 deep-scrub starts 2024-09-11T07:45:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:33 smithi104 ceph-mon[33507]: 4.5 deep-scrub ok 2024-09-11T07:45:33.612 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:33.662 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:33.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:33.664 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:33.689 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:45:33.765 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:33.817 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:33.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:33.818 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:33.919 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:33.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:33.958 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:33.958 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:34.059 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:34.100 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:34.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:34.101 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:34.202 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:34.232 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:34.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:34.234 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:34.334 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:34.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:34.362 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:34.362 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:34.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:34 smithi022 ceph-mon[29462]: 4.1b deep-scrub starts 2024-09-11T07:45:34.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:34 smithi022 ceph-mon[29462]: 4.1b deep-scrub ok 2024-09-11T07:45:34.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:34 smithi022 ceph-mon[37330]: 4.1b deep-scrub starts 2024-09-11T07:45:34.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:34 smithi022 ceph-mon[37330]: 4.1b deep-scrub ok 2024-09-11T07:45:34.463 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:34.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:34.494 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:34.494 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:34.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:45:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:45:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:45:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:34 smithi104 ceph-mon[33507]: 4.1b deep-scrub starts 2024-09-11T07:45:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:34 smithi104 ceph-mon[33507]: 4.1b deep-scrub ok 2024-09-11T07:45:34.596 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:34.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:34.642 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:34.642 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:34.743 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:34.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:34.771 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:34.771 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:34.872 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:34.903 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:34.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:34.905 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:35.006 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:35.035 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:45:35.043 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:35.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:35.044 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:35.135 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is set 2024-09-11T07:45:35.145 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:35.160 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Removed 129423 objects 2024-09-11T07:45:35.161 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Clean up completed and total clean up time :109.19 2024-09-11T07:45:35.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:35.171 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:35.172 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:35.181 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:35 smithi022 ceph-mon[29462]: 4.c scrub starts 2024-09-11T07:45:35.181 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:35 smithi022 ceph-mon[29462]: 4.c scrub ok 2024-09-11T07:45:35.182 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:35 smithi022 ceph-mon[29462]: pgmap v883: 158 pgs: 1 active+clean+remapped, 1 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 150 active+clean; 122 MiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 1.04k op/s; 811/6408 objects misplaced (12.656%); 14 MiB/s, 240 objects/s recovering 2024-09-11T07:45:35.182 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:35 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1539625334' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:45:35.182 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:35 smithi022 ceph-mon[29462]: from='client.25895 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:45:35.182 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:35 smithi022 ceph-mon[37330]: 4.c scrub starts 2024-09-11T07:45:35.183 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:35 smithi022 ceph-mon[37330]: 4.c scrub ok 2024-09-11T07:45:35.183 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:35 smithi022 ceph-mon[37330]: pgmap v883: 158 pgs: 1 active+clean+remapped, 1 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 150 active+clean; 122 MiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 1.04k op/s; 811/6408 objects misplaced (12.656%); 14 MiB/s, 240 objects/s recovering 2024-09-11T07:45:35.183 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:35 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1539625334' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:45:35.183 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:35 smithi022 ceph-mon[37330]: from='client.25895 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:45:35.273 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:35.295 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:35.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:35.296 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:35.397 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:35.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:35.419 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:35.419 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:35.520 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:35.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:35 smithi104 ceph-mon[33507]: 4.c scrub starts 2024-09-11T07:45:35.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:35 smithi104 ceph-mon[33507]: 4.c scrub ok 2024-09-11T07:45:35.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:35 smithi104 ceph-mon[33507]: pgmap v883: 158 pgs: 1 active+clean+remapped, 1 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 150 active+clean; 122 MiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 1.04k op/s; 811/6408 objects misplaced (12.656%); 14 MiB/s, 240 objects/s recovering 2024-09-11T07:45:35.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:35 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1539625334' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:45:35.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:35 smithi104 ceph-mon[33507]: from='client.25895 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:45:35.541 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:35.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:35.542 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:35.642 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:35.665 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:35.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:35.666 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:35.767 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:35.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:35.789 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:35.789 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:35.890 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:35.911 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:35.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:35.912 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:36.013 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:36.036 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:36.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:36.037 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:36.137 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:36.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:36.159 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:36.160 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:36.261 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:36.284 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:36.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:36.285 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:36.385 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:36.407 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:36.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:36.408 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:36.416 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:36 smithi022 ceph-mon[37330]: 5.1d scrub starts 2024-09-11T07:45:36.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:36 smithi022 ceph-mon[37330]: 5.1d scrub ok 2024-09-11T07:45:36.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:36 smithi022 ceph-mon[37330]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:45:36.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:36 smithi022 ceph-mon[37330]: from='client.25895 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:45:36.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:36 smithi022 ceph-mon[37330]: osdmap e400: 8 total, 8 up, 7 in 2024-09-11T07:45:36.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:36 smithi022 ceph-mon[37330]: osdmap e401: 8 total, 8 up, 7 in 2024-09-11T07:45:36.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:36 smithi022 ceph-mon[29462]: 5.1d scrub starts 2024-09-11T07:45:36.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:36 smithi022 ceph-mon[29462]: 5.1d scrub ok 2024-09-11T07:45:36.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:36 smithi022 ceph-mon[29462]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:45:36.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:36 smithi022 ceph-mon[29462]: from='client.25895 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:45:36.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:36 smithi022 ceph-mon[29462]: osdmap e400: 8 total, 8 up, 7 in 2024-09-11T07:45:36.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:36 smithi022 ceph-mon[29462]: osdmap e401: 8 total, 8 up, 7 in 2024-09-11T07:45:36.509 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:36.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:36 smithi104 ceph-mon[33507]: 5.1d scrub starts 2024-09-11T07:45:36.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:36 smithi104 ceph-mon[33507]: 5.1d scrub ok 2024-09-11T07:45:36.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:36 smithi104 ceph-mon[33507]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:45:36.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:36 smithi104 ceph-mon[33507]: from='client.25895 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:45:36.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:36 smithi104 ceph-mon[33507]: osdmap e400: 8 total, 8 up, 7 in 2024-09-11T07:45:36.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:36 smithi104 ceph-mon[33507]: osdmap e401: 8 total, 8 up, 7 in 2024-09-11T07:45:36.532 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:36.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:36.533 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:36.634 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:36.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:36.657 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:36.657 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:36.758 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:36.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:36.780 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:36.780 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:36.881 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:36.904 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:36.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:36.905 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:37.006 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:37.029 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:37.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:37.030 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:37.107 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_3 2024-09-11T07:45:37.108 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pool rm unique_pool_3 unique_pool_3 --yes-i-really-really-mean-it 2024-09-11T07:45:37.131 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:37.173 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:37.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:37.174 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:37.275 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:37.299 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:37.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:37.300 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:37.401 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:37.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:37 smithi022 ceph-mon[29462]: pgmap v886: 158 pgs: 6 active+remapped+backfill_toofull, 152 active+clean; 54 MiB data, 6.3 GiB used, 620 GiB / 626 GiB avail; 907 B/s rd, 0 B/s wr, 762 op/s; 349/3150 objects misplaced (11.079%); 8.8 MiB/s, 141 objects/s recovering 2024-09-11T07:45:37.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:37 smithi022 ceph-mon[37330]: pgmap v886: 158 pgs: 6 active+remapped+backfill_toofull, 152 active+clean; 54 MiB data, 6.3 GiB used, 620 GiB / 626 GiB avail; 907 B/s rd, 0 B/s wr, 762 op/s; 349/3150 objects misplaced (11.079%); 8.8 MiB/s, 141 objects/s recovering 2024-09-11T07:45:37.423 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:37.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:37.424 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:37.525 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:37.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:37 smithi104 ceph-mon[33507]: pgmap v886: 158 pgs: 6 active+remapped+backfill_toofull, 152 active+clean; 54 MiB data, 6.3 GiB used, 620 GiB / 626 GiB avail; 907 B/s rd, 0 B/s wr, 762 op/s; 349/3150 objects misplaced (11.079%); 8.8 MiB/s, 141 objects/s recovering 2024-09-11T07:45:37.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:37.549 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:37.549 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:37.650 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:37.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:37.674 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:37.775 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:37.799 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:37.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:37.800 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:37.901 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:37.927 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:37.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:37.928 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:38.029 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:38.053 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:38.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:38.054 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:38.155 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:38.179 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:38.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:38.180 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:38.281 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:38.308 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:38.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:38.310 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:38.389 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:45:38.389 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:45:38.389 INFO:teuthology.orchestra.run.smithi022.stdout: "complaint_time": 30, 2024-09-11T07:45:38.389 INFO:teuthology.orchestra.run.smithi022.stdout: "num_blocked_ops": 0 2024-09-11T07:45:38.390 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:45:38.410 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:38.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:38.437 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:38.437 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:38.538 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:38.561 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:38.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:38.564 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:38.664 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:38.690 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:38.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:38.691 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:38.792 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:38.814 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:38.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:38.815 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:38.916 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:38.942 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:38.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:38.943 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:39.044 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:39.059 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:45:39.069 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:39.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:39.070 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:39.087 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-11T07:45:39.170 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:39.193 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:39.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:39.194 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:39.294 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:39.318 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:39.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:39.319 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:39.321 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:39 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:45:39.321 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:39 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:45:39.321 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:39 smithi022 ceph-mon[37330]: pgmap v887: 158 pgs: 6 active+remapped+backfill_toofull, 152 active+clean; 54 MiB data, 6.3 GiB used, 620 GiB / 626 GiB avail; 770 B/s rd, 0 B/s wr, 648 op/s; 349/3150 objects misplaced (11.079%); 7.5 MiB/s, 119 objects/s recovering 2024-09-11T07:45:39.321 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:39 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:45:39.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:39 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:45:39.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:39 smithi022 ceph-mon[29462]: pgmap v887: 158 pgs: 6 active+remapped+backfill_toofull, 152 active+clean; 54 MiB data, 6.3 GiB used, 620 GiB / 626 GiB avail; 770 B/s rd, 0 B/s wr, 648 op/s; 349/3150 objects misplaced (11.079%); 7.5 MiB/s, 119 objects/s recovering 2024-09-11T07:45:39.420 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:39.446 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:39.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:39.447 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:39.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:39 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:45:39.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:39 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:45:39.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:39 smithi104 ceph-mon[33507]: pgmap v887: 158 pgs: 6 active+remapped+backfill_toofull, 152 active+clean; 54 MiB data, 6.3 GiB used, 620 GiB / 626 GiB avail; 770 B/s rd, 0 B/s wr, 648 op/s; 349/3150 objects misplaced (11.079%); 7.5 MiB/s, 119 objects/s recovering 2024-09-11T07:45:39.548 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:39.573 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:39.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:39.574 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:39.675 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:39.699 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:39.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:39.701 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:39.802 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:39.806 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T07:45:39.826 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:39.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:39.827 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:39.928 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:39.951 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:39.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:39.952 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:40.053 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:40.077 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:40.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:40.078 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:40.179 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:40.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:40.205 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:40.205 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:40.306 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:40.330 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:40.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:40.331 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:40.432 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:40.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:40.458 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:40.458 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:40.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:45:40 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:45:40.559 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:40.583 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:40.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:40.584 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:40.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:45:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:45:40] "GET /metrics HTTP/1.1" 200 225028 "" "Prometheus/2.43.0" 2024-09-11T07:45:40.685 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:40.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:40.710 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:40.710 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:40.811 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:40.835 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:40.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:40.836 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:40.937 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:40.960 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:40.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:40.961 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:41.062 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:41.088 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:41.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:41.089 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:41.190 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:41.214 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:41.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:41.215 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:41.316 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:41.339 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:41.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:41.340 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:41.441 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:41.466 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:41.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:41.467 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:41.568 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:41.593 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:41.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:41.594 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:41.695 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:41.718 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:41.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:41.719 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:41.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:41 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:45:41.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:41 smithi104 ceph-mon[33507]: pgmap v888: 158 pgs: 6 active+remapped+backfill_toofull, 152 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 645 op/s; 6.2 MiB/s, 99 objects/s recovering 2024-09-11T07:45:41.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:41 smithi104 ceph-mon[33507]: osdmap e402: 8 total, 8 up, 7 in 2024-09-11T07:45:41.820 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:41.843 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:41.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:41.844 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:41.916 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:41 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:45:41.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:41 smithi022 ceph-mon[37330]: pgmap v888: 158 pgs: 6 active+remapped+backfill_toofull, 152 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 645 op/s; 6.2 MiB/s, 99 objects/s recovering 2024-09-11T07:45:41.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:41 smithi022 ceph-mon[37330]: osdmap e402: 8 total, 8 up, 7 in 2024-09-11T07:45:41.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:41 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:45:41.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:41 smithi022 ceph-mon[29462]: pgmap v888: 158 pgs: 6 active+remapped+backfill_toofull, 152 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 645 op/s; 6.2 MiB/s, 99 objects/s recovering 2024-09-11T07:45:41.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:41 smithi022 ceph-mon[29462]: osdmap e402: 8 total, 8 up, 7 in 2024-09-11T07:45:41.945 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:41.968 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:41.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:41.969 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:42.070 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:42.094 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:42.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:42.094 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:42.195 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:42.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:42.219 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:42.219 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:42.320 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:42.345 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:42.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:42.346 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:42.447 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:42.472 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:42.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:42.473 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:42.574 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:42.598 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:42.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:42.599 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:42.700 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:42.723 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:42.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:42.724 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:42.825 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:42.846 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:42.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:42.848 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:42.948 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:42.970 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:42.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:42.971 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:43.072 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:43.094 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:43.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:43.095 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:43.196 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:43.221 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:43.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:43.223 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:43.323 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:43.346 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:43.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:43.347 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:43.448 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:43.473 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:43.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:43.474 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:43.575 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:43.602 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:43.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:43.604 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:43.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:43 smithi022 ceph-mon[29462]: pgmap v890: 158 pgs: 6 active+remapped+backfill_toofull, 152 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 705 B/s rd, 0 B/s wr, 155 op/s 2024-09-11T07:45:43.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:43 smithi022 ceph-mon[37330]: pgmap v890: 158 pgs: 6 active+remapped+backfill_toofull, 152 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 705 B/s rd, 0 B/s wr, 155 op/s 2024-09-11T07:45:43.705 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:43.731 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:43.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:43.732 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:43.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:43 smithi104 ceph-mon[33507]: pgmap v890: 158 pgs: 6 active+remapped+backfill_toofull, 152 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 705 B/s rd, 0 B/s wr, 155 op/s 2024-09-11T07:45:43.833 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:43.859 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:43.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:43.860 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:43.961 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:43.988 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:43.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:43.989 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:44.090 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:44.116 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:44.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:44.117 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:44.218 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:44.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:44.245 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:44.245 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:44.346 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:44.372 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:44.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:44.373 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:44.450 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:45:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:45:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:45:44.474 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:44.499 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:44.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:44.500 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:44.600 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:44.624 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:44.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:44.625 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:44.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:44 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:45:44.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:44 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:45:44.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:44 smithi022 ceph-mon[29462]: osdmap e403: 8 total, 8 up, 7 in 2024-09-11T07:45:44.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:44 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:45:44.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:44 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:45:44.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:44 smithi022 ceph-mon[37330]: osdmap e403: 8 total, 8 up, 7 in 2024-09-11T07:45:44.726 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:44.753 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:44.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:44.754 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:44.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:44 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:45:44.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:44 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:45:44.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:44 smithi104 ceph-mon[33507]: osdmap e403: 8 total, 8 up, 7 in 2024-09-11T07:45:44.855 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:44.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:44.881 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:44.881 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:44.982 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:45.005 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:45.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:45.006 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:45.107 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:45.133 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:45.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:45.134 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:45.169 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:45:45.170 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:45:45.185 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:45:45.185 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:45:45.200 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:45:45.200 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:45:45.215 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:45:45.215 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:45:45.231 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:45:45.231 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:45:45.234 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:45.248 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:45:45.249 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:45:45.259 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:45.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:45.260 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:45.262 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:45:45.263 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:45:45.278 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:45:45.279 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:45:45.361 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:45.384 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:45.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:45.385 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:45.486 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:45.511 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:45.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:45.512 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:45.613 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:45.637 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:45.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:45.638 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:45.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:45 smithi022 ceph-mon[29462]: pgmap v892: 158 pgs: 6 active+remapped+backfill_toofull, 152 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 107 op/s 2024-09-11T07:45:45.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:45 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:45:45.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:45 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:45:45.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:45 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/180148923' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:45:45.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:45 smithi022 ceph-mon[37330]: pgmap v892: 158 pgs: 6 active+remapped+backfill_toofull, 152 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 107 op/s 2024-09-11T07:45:45.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:45 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:45:45.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:45 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:45:45.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:45 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/180148923' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:45:45.739 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:45.749 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'unique_pool_3' removed 2024-09-11T07:45:45.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:45.774 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:45.774 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:45.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:45 smithi104 ceph-mon[33507]: pgmap v892: 158 pgs: 6 active+remapped+backfill_toofull, 152 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 107 op/s 2024-09-11T07:45:45.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:45 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:45:45.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:45 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:45:45.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:45 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/180148923' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:45:45.875 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:45.897 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:45.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:45.898 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:45.999 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:46.021 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:46.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:46.022 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:46.122 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:46.146 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:46.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:46.147 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:46.248 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:46.271 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:46.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:46.272 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:46.373 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:46.394 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:46.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:46.395 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:46.496 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:46.517 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:46.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:46.518 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:46.619 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:46.640 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:46.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:46.641 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:46.742 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:46.763 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:46.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:46.764 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:46.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:46 smithi104 ceph-mon[33507]: osdmap e404: 8 total, 8 up, 7 in 2024-09-11T07:45:46.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:46 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3317359973' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:45:46.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:46 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3317359973' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T07:45:46.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:46 smithi104 ceph-mon[33507]: osdmap e405: 8 total, 8 up, 7 in 2024-09-11T07:45:46.833 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:46 smithi022 ceph-mon[29462]: osdmap e404: 8 total, 8 up, 7 in 2024-09-11T07:45:46.833 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:46 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3317359973' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:45:46.833 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:46 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3317359973' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T07:45:46.834 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:46 smithi022 ceph-mon[29462]: osdmap e405: 8 total, 8 up, 7 in 2024-09-11T07:45:46.834 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:46 smithi022 ceph-mon[37330]: osdmap e404: 8 total, 8 up, 7 in 2024-09-11T07:45:46.834 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:46 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3317359973' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:45:46.834 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:46 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3317359973' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T07:45:46.834 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:46 smithi022 ceph-mon[37330]: osdmap e405: 8 total, 8 up, 7 in 2024-09-11T07:45:46.864 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-11T07:45:46.865 INFO:tasks.radosbench:Beginning radosbench... 2024-09-11T07:45:46.865 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_4 2024-09-11T07:45:46.865 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pool create unique_pool_4 16 2024-09-11T07:45:46.866 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:46.870 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:45:46.871 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:45:46.887 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:46.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:46.888 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:46.989 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:47.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:47.012 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:47.012 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:47.113 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:47.136 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:47.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:47.137 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:47.238 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:47.261 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:47.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:47.262 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:47.363 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:47.386 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:47.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:47.387 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:47.488 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:47.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:47.511 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:47.511 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:47.612 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:47.634 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:47.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:47.635 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:47.736 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:47.759 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:47.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:47.760 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:47.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:47 smithi104 ceph-mon[33507]: pgmap v895: 142 pgs: 142 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 725 B/s rd, 0 op/s 2024-09-11T07:45:47.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:47 smithi104 ceph-mon[33507]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull) 2024-09-11T07:45:47.861 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:47.885 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:47.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:47.886 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:47.909 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:47 smithi022 ceph-mon[29462]: pgmap v895: 142 pgs: 142 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 725 B/s rd, 0 op/s 2024-09-11T07:45:47.909 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:47 smithi022 ceph-mon[29462]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull) 2024-09-11T07:45:47.910 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:47 smithi022 ceph-mon[37330]: pgmap v895: 142 pgs: 142 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 725 B/s rd, 0 op/s 2024-09-11T07:45:47.910 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:47 smithi022 ceph-mon[37330]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull) 2024-09-11T07:45:47.987 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:48.011 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:48.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:48.013 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:48.113 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:48.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:48.141 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:48.141 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:48.157 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:45:48.242 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:48.265 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:48.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:48.266 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:48.367 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:48.391 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:48.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:48.393 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:48.493 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:48.517 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:48.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:48.518 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:48.619 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:48.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:48.644 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:48.644 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:48.745 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:48.751 INFO:teuthology.orchestra.run.smithi022.stderr:set osd.5 primary-affinity to 0.603115 (8395252) 2024-09-11T07:45:48.771 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:48.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:48.772 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:48.873 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:48.897 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:48.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:48.898 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:48.999 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:49.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:49.025 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:49.026 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:49.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:48 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:45:49.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:48 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:45:49.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:48 smithi104 ceph-mon[33507]: osdmap e406: 8 total, 8 up, 7 in 2024-09-11T07:45:49.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:48 smithi104 ceph-mon[33507]: pgmap v897: 142 pgs: 142 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 828 B/s rd, 0 op/s 2024-09-11T07:45:49.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:48 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4189009252' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.6031152587241752}]: dispatch 2024-09-11T07:45:49.068 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:48 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:45:49.068 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:48 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:45:49.068 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:48 smithi022 ceph-mon[37330]: osdmap e406: 8 total, 8 up, 7 in 2024-09-11T07:45:49.068 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:48 smithi022 ceph-mon[37330]: pgmap v897: 142 pgs: 142 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 828 B/s rd, 0 op/s 2024-09-11T07:45:49.068 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:48 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4189009252' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.6031152587241752}]: dispatch 2024-09-11T07:45:49.069 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:48 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:45:49.069 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:48 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:45:49.069 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:48 smithi022 ceph-mon[29462]: osdmap e406: 8 total, 8 up, 7 in 2024-09-11T07:45:49.069 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:48 smithi022 ceph-mon[29462]: pgmap v897: 142 pgs: 142 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 828 B/s rd, 0 op/s 2024-09-11T07:45:49.069 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:48 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4189009252' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.6031152587241752}]: dispatch 2024-09-11T07:45:49.127 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:49.152 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:49.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:49.153 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:49.254 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:49.281 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:49.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:49.282 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:49.382 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:49.411 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:49.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:49.412 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:49.513 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:49.538 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:49.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:49.539 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:49.640 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:49.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:49.669 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:49.670 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:49.771 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:49.795 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:49.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:49.796 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:49.897 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:49.915 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:49 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4189009252' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.6031152587241752}]': finished 2024-09-11T07:45:49.915 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:49 smithi022 ceph-mon[29462]: osdmap e407: 8 total, 8 up, 7 in 2024-09-11T07:45:49.915 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:49 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4189009252' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.6031152587241752}]': finished 2024-09-11T07:45:49.916 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:49 smithi022 ceph-mon[37330]: osdmap e407: 8 total, 8 up, 7 in 2024-09-11T07:45:49.922 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:49.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:49.923 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:50.024 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:50.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:49 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4189009252' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.6031152587241752}]': finished 2024-09-11T07:45:50.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:49 smithi104 ceph-mon[33507]: osdmap e407: 8 total, 8 up, 7 in 2024-09-11T07:45:50.050 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:50.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:50.051 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:50.151 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:50.177 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:50.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:50.178 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:50.278 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:50.304 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:50.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:50.305 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:50.406 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:50.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:50.432 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:50.433 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:50.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:45:50 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:45:50.534 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:50.541 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:45:50.541 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:45:50.559 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:50.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:50.560 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:50.661 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:50.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:45:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:45:50] "GET /metrics HTTP/1.1" 200 225028 "" "Prometheus/2.43.0" 2024-09-11T07:45:50.687 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:50.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:50.688 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:50.789 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:50.815 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:50.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:50.816 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:50.904 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:45:50.916 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:50.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:50.941 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:50.941 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:51.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:50 smithi104 ceph-mon[33507]: osdmap e408: 8 total, 8 up, 7 in 2024-09-11T07:45:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:50 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:45:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:50 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:45:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:50 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:45:51.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:50 smithi104 ceph-mon[33507]: pgmap v900: 142 pgs: 3 peering, 139 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:45:51.042 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:51.067 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:51.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:51.068 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:51.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:50 smithi022 ceph-mon[29462]: osdmap e408: 8 total, 8 up, 7 in 2024-09-11T07:45:51.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:50 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:45:51.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:50 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:45:51.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:50 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:45:51.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:50 smithi022 ceph-mon[29462]: pgmap v900: 142 pgs: 3 peering, 139 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:45:51.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:50 smithi022 ceph-mon[37330]: osdmap e408: 8 total, 8 up, 7 in 2024-09-11T07:45:51.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:50 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:45:51.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:50 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:45:51.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:50 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:45:51.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:50 smithi022 ceph-mon[37330]: pgmap v900: 142 pgs: 3 peering, 139 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:45:51.170 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:51.192 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:51.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:51.193 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:51.293 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:51.316 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:51.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:51.317 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:51.418 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:51.442 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:51.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:51.443 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:51.544 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:51.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:51.567 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:51.567 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:51.668 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:51.694 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:51.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:51.695 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:51.797 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:51.820 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:51.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:51.821 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:51.909 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-11T07:45:51.922 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:51.974 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:45:51.976 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:51.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:51.977 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:52.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:51 smithi104 ceph-mon[33507]: Health check failed: Reduced data availability: 2 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-09-11T07:45:52.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:51 smithi104 ceph-mon[33507]: osdmap e409: 8 total, 8 up, 7 in 2024-09-11T07:45:52.065 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:51 smithi022 ceph-mon[29462]: Health check failed: Reduced data availability: 2 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-09-11T07:45:52.066 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:51 smithi022 ceph-mon[29462]: osdmap e409: 8 total, 8 up, 7 in 2024-09-11T07:45:52.066 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:51 smithi022 ceph-mon[37330]: Health check failed: Reduced data availability: 2 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-09-11T07:45:52.066 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:51 smithi022 ceph-mon[37330]: osdmap e409: 8 total, 8 up, 7 in 2024-09-11T07:45:52.077 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:52.100 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:52.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:52.101 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:52.168 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:45:52.202 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:52.223 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:52.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:52.224 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:52.325 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:52.348 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:52.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:52.349 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:52.450 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:52.472 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:52.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:52.473 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:52.574 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:52.597 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:52.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:52.598 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:52.698 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:52.721 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:52.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:52.722 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:52.822 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:52.845 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:52.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:52.846 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:52.916 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:52 smithi022 ceph-mon[37330]: pgmap v902: 142 pgs: 3 peering, 139 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:45:52.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:52 smithi022 ceph-mon[29462]: pgmap v902: 142 pgs: 3 peering, 139 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:45:52.947 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:52.969 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:52.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:52.970 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:53.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:52 smithi104 ceph-mon[33507]: pgmap v902: 142 pgs: 3 peering, 139 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:45:53.072 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:53.093 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:53.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:53.094 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:53.195 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:53.216 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:53.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:53.217 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:53.318 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:53.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:53.340 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:53.340 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:53.441 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:53.463 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:53.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:53.464 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:53.565 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:53.587 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:53.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:53.588 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:53.688 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:53.711 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:53.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:53.712 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:53.784 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is set 2024-09-11T07:45:53.813 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:53.835 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:53.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:53.836 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:53.848 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:45:53.848 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:45:53.898 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:53 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:45:53.898 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:53 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:45:53.898 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:53 smithi022 ceph-mon[29462]: osdmap e410: 8 total, 8 up, 7 in 2024-09-11T07:45:53.898 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:53 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3217961205' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:45:53.898 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:53 smithi022 ceph-mon[29462]: from='client.25939 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:45:53.899 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:45:53.899 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:45:53.899 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:53 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:45:53.899 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:53 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:45:53.899 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:53 smithi022 ceph-mon[37330]: osdmap e410: 8 total, 8 up, 7 in 2024-09-11T07:45:53.899 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:53 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3217961205' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:45:53.899 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:53 smithi022 ceph-mon[37330]: from='client.25939 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:45:53.899 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:45:53.899 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:45:53.937 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:53.958 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:53.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:53.959 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:54.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:53 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:45:54.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:53 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:45:54.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:53 smithi104 ceph-mon[33507]: osdmap e410: 8 total, 8 up, 7 in 2024-09-11T07:45:54.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:53 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3217961205' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:45:54.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:53 smithi104 ceph-mon[33507]: from='client.25939 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:45:54.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:45:54.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:45:54.060 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:54.084 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:54.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:54.085 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:54.186 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:54.208 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:54.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:54.209 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:54.308 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:45:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:45:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:45:54.310 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:54.331 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:54.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:54.332 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:54.430 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:45:54.430 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:45:54.430 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-11T07:45:54.431 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-11T07:45:54.433 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:54.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:54.456 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:54.456 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:54.556 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:54.569 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-11T07:45:54.578 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:54.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:54.578 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:54.680 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:54.704 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:54.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:54.705 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:54.805 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:54.819 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:45:54.827 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:54.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:54.828 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:54.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:54 smithi022 ceph-mon[29462]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:45:54.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:54 smithi022 ceph-mon[29462]: from='client.25939 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:45:54.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:54 smithi022 ceph-mon[29462]: osdmap e411: 8 total, 8 up, 7 in 2024-09-11T07:45:54.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:54 smithi022 ceph-mon[29462]: pgmap v905: 142 pgs: 142 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:45:54.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:54 smithi022 ceph-mon[37330]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:45:54.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:54 smithi022 ceph-mon[37330]: from='client.25939 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:45:54.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:54 smithi022 ceph-mon[37330]: osdmap e411: 8 total, 8 up, 7 in 2024-09-11T07:45:54.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:54 smithi022 ceph-mon[37330]: pgmap v905: 142 pgs: 142 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:45:54.928 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:54.940 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:54 smithi104 ceph-mon[33507]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:45:54.940 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:54 smithi104 ceph-mon[33507]: from='client.25939 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:45:54.940 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:54 smithi104 ceph-mon[33507]: osdmap e411: 8 total, 8 up, 7 in 2024-09-11T07:45:54.940 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:54 smithi104 ceph-mon[33507]: pgmap v905: 142 pgs: 142 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:45:54.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:54.950 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:54.950 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:55.051 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:55.072 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:55.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:55.073 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:55.174 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:55.198 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:55.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:55.199 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:55.299 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:55.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:55.322 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:55.322 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:55.384 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:45:55.424 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:55.446 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:55.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:55.448 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:55.548 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:55.571 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:55.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:55.572 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:55.673 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:55.695 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:55.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:55.696 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:55.754 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'unique_pool_4' created 2024-09-11T07:45:55.797 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:55.818 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:55.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:55.819 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:55.916 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:55 smithi022 ceph-mon[37330]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 3 pgs peering) 2024-09-11T07:45:55.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:55 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/332532495' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-11T07:45:55.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:55 smithi022 ceph-mon[37330]: from='client.25945 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-11T07:45:55.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:55 smithi022 ceph-mon[37330]: from='client.25945 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-09-11T07:45:55.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:55 smithi022 ceph-mon[37330]: osdmap e412: 8 total, 8 up, 7 in 2024-09-11T07:45:55.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:55 smithi022 ceph-mon[29462]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 3 pgs peering) 2024-09-11T07:45:55.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:55 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/332532495' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-11T07:45:55.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:55 smithi022 ceph-mon[29462]: from='client.25945 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-11T07:45:55.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:55 smithi022 ceph-mon[29462]: from='client.25945 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-09-11T07:45:55.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:55 smithi022 ceph-mon[29462]: osdmap e412: 8 total, 8 up, 7 in 2024-09-11T07:45:55.920 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:55.940 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:55.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:55.941 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:56.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:55 smithi104 ceph-mon[33507]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 3 pgs peering) 2024-09-11T07:45:56.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:55 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/332532495' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-11T07:45:56.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:55 smithi104 ceph-mon[33507]: from='client.25945 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-11T07:45:56.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:55 smithi104 ceph-mon[33507]: from='client.25945 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-09-11T07:45:56.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:55 smithi104 ceph-mon[33507]: osdmap e412: 8 total, 8 up, 7 in 2024-09-11T07:45:56.042 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:56.062 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:56.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:56.063 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:56.163 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:56.186 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:56.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:56.187 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:56.288 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:56.309 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:56.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:56.310 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:56.411 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:56.433 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:56.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:56.434 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:56.524 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T07:45:56.535 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:56.590 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:56.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:56.591 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:56.692 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:56.694 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pool application enable unique_pool_4 rados --yes-i-really-mean-it || true 2024-09-11T07:45:56.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:56.766 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:56.766 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:56.867 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:56.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:56.890 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:56.890 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:56.991 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:57.013 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:57.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:57.014 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:57.115 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:57.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:56 smithi022 ceph-mon[29462]: pgmap v907: 158 pgs: 16 unknown, 142 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 911 B/s rd, 0 op/s 2024-09-11T07:45:57.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:56 smithi022 ceph-mon[29462]: osdmap e413: 8 total, 8 up, 7 in 2024-09-11T07:45:57.121 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:56 smithi022 ceph-mon[37330]: pgmap v907: 158 pgs: 16 unknown, 142 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 911 B/s rd, 0 op/s 2024-09-11T07:45:57.121 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:56 smithi022 ceph-mon[37330]: osdmap e413: 8 total, 8 up, 7 in 2024-09-11T07:45:57.142 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:57.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:57.143 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:57.237 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:56 smithi104 ceph-mon[33507]: pgmap v907: 158 pgs: 16 unknown, 142 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 911 B/s rd, 0 op/s 2024-09-11T07:45:57.237 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:56 smithi104 ceph-mon[33507]: osdmap e413: 8 total, 8 up, 7 in 2024-09-11T07:45:57.244 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:45:57.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:57.267 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:57.267 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:45:57.368 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:57.391 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:57.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:57.392 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:57.493 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:57.515 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:57.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:57.516 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:57.617 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:57.638 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:57.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:57.639 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:57.740 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:57.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:57.767 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:57.777 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:45:57.777 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:45:57.866 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:57 smithi022 ceph-mon[29462]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:45:57.866 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:57 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:45:57.866 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:57 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:45:57.866 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:57 smithi022 ceph-mon[29462]: osdmap e414: 8 total, 8 up, 7 in 2024-09-11T07:45:57.866 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:57 smithi022 ceph-mon[37330]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:45:57.867 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:57 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:45:57.867 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:57 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:45:57.867 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:57 smithi022 ceph-mon[37330]: osdmap e414: 8 total, 8 up, 7 in 2024-09-11T07:45:57.868 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:45:57.890 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:57.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:57.891 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:45:57.992 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:58.015 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:58.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:58.016 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:58.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:57 smithi104 ceph-mon[33507]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:45:58.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:57 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:45:58.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:57 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:45:58.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:57 smithi104 ceph-mon[33507]: osdmap e414: 8 total, 8 up, 7 in 2024-09-11T07:45:58.117 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:45:58.138 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:58.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:58.139 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:45:58.240 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:58.264 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:58.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:58.265 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:58.348 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:45:58.348 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:45:58.352 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:45:58.365 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:45:58.366 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:45:58.366 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:58.382 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:45:58.383 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:45:58.392 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:58.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:58.393 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:58.402 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:45:58.403 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:45:58.418 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:45:58.418 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:45:58.434 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:45:58.434 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:45:58.449 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:45:58.449 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:45:58.464 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:45:58.464 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:45:58.493 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-11T07:45:58.494 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:58.514 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:58.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:58.515 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:58.616 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:45:58.638 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:58.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:58.639 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:45:58.736 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:45:58.740 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:45:58.761 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:58.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:58.762 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:45:58.800 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:45:58.800 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:45:58.800 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 414, 2024-09-11T07:45:58.800 INFO:teuthology.orchestra.run.smithi022.stdout: "fsid": "12a285d8-700e-11ef-bcea-c7b262605968", 2024-09-11T07:45:58.800 INFO:teuthology.orchestra.run.smithi022.stdout: "created": "2024-09-11T07:21:18.695332+0000", 2024-09-11T07:45:58.800 INFO:teuthology.orchestra.run.smithi022.stdout: "modified": "2024-09-11T07:45:57.747027+0000", 2024-09-11T07:45:58.800 INFO:teuthology.orchestra.run.smithi022.stdout: "last_up_change": "2024-09-11T07:32:00.070825+0000", 2024-09-11T07:45:58.800 INFO:teuthology.orchestra.run.smithi022.stdout: "last_in_change": "2024-09-11T07:45:26.061430+0000", 2024-09-11T07:45:58.800 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-11T07:45:58.801 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_num": 5806080, 2024-09-11T07:45:58.801 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_set": [ 2024-09-11T07:45:58.801 INFO:teuthology.orchestra.run.smithi022.stdout: "nodeep-scrub", 2024-09-11T07:45:58.801 INFO:teuthology.orchestra.run.smithi022.stdout: "noscrub", 2024-09-11T07:45:58.801 INFO:teuthology.orchestra.run.smithi022.stdout: "pglog_hardlimit", 2024-09-11T07:45:58.801 INFO:teuthology.orchestra.run.smithi022.stdout: "purged_snapdirs", 2024-09-11T07:45:58.801 INFO:teuthology.orchestra.run.smithi022.stdout: "recovery_deletes", 2024-09-11T07:45:58.801 INFO:teuthology.orchestra.run.smithi022.stdout: "sortbitwise" 2024-09-11T07:45:58.801 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:45:58.801 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_version": 19, 2024-09-11T07:45:58.801 INFO:teuthology.orchestra.run.smithi022.stdout: "full_ratio": 0.94999998807907104, 2024-09-11T07:45:58.801 INFO:teuthology.orchestra.run.smithi022.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-11T07:45:58.801 INFO:teuthology.orchestra.run.smithi022.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-11T07:45:58.801 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_snapshot": "", 2024-09-11T07:45:58.802 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_max": 11, 2024-09-11T07:45:58.802 INFO:teuthology.orchestra.run.smithi022.stdout: "max_osd": 8, 2024-09-11T07:45:58.802 INFO:teuthology.orchestra.run.smithi022.stdout: "require_min_compat_client": "luminous", 2024-09-11T07:45:58.802 INFO:teuthology.orchestra.run.smithi022.stdout: "min_compat_client": "luminous", 2024-09-11T07:45:58.802 INFO:teuthology.orchestra.run.smithi022.stdout: "require_osd_release": "quincy", 2024-09-11T07:45:58.802 INFO:teuthology.orchestra.run.smithi022.stdout: "pools": [ 2024-09-11T07:45:58.802 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.802 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 1, 2024-09-11T07:45:58.802 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": ".mgr", 2024-09-11T07:45:58.802 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:24:21.772426+0000", 2024-09-11T07:45:58.802 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:45:58.802 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:45:58.802 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:45:58.803 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:45:58.803 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:45:58.803 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:45:58.803 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:45:58.803 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:45:58.803 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:45:58.803 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:45:58.803 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:45:58.803 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:45:58.803 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 11, 2024-09-11T07:45:58.803 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 11, 2024-09-11T07:45:58.803 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 11, 2024-09-11T07:45:58.804 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 11, 2024-09-11T07:45:58.804 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 11, 2024-09-11T07:45:58.804 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:45:58.804 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:45:58.804 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:45:58.804 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:45:58.804 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:45:58.804 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:45:58.804 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:45:58.804 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.804 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "390", 2024-09-11T07:45:58.804 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:45:58.804 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:45:58.804 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "388", 2024-09-11T07:45:58.805 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:45:58.805 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:45:58.805 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:45:58.805 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:45:58.805 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:45:58.805 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:45:58.805 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:45:58.805 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:45:58.805 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:45:58.805 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:45:58.805 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:45:58.805 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:45:58.806 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:45:58.806 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:45:58.806 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:45:58.806 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:45:58.806 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:45:58.806 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:45:58.806 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:45:58.806 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:45:58.806 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:45:58.806 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:45:58.806 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:45:58.806 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.807 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:45:58.807 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:45:58.807 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:45:58.807 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:45:58.807 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:45:58.807 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:45:58.807 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:45:58.807 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:45:58.807 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:45:58.807 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:45:58.807 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:45:58.807 INFO:teuthology.orchestra.run.smithi022.stdout: "options": { 2024-09-11T07:45:58.807 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_max": 32, 2024-09-11T07:45:58.807 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_min": 1 2024-09-11T07:45:58.808 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.808 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:45:58.808 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": {} 2024-09-11T07:45:58.808 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.808 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.808 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.808 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 2, 2024-09-11T07:45:58.808 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": ".rgw.root", 2024-09-11T07:45:58.808 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:29.941354+0000", 2024-09-11T07:45:58.808 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:45:58.808 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:45:58.808 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:45:58.808 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:45:58.809 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:45:58.809 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:45:58.809 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:45:58.809 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:45:58.809 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:45:58.809 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:45:58.809 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:45:58.809 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:45:58.809 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:45:58.809 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:45:58.809 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:45:58.809 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:45:58.809 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:45:58.809 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:45:58.809 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:45:58.810 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:45:58.810 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:45:58.810 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:45:58.810 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:45:58.810 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:45:58.810 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.810 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "58", 2024-09-11T07:45:58.810 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:45:58.810 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:45:58.810 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:45:58.810 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:45:58.810 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:45:58.810 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:45:58.810 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:45:58.810 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:45:58.811 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:45:58.811 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:45:58.811 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:45:58.811 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:45:58.811 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:45:58.811 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:45:58.811 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:45:58.811 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:45:58.811 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:45:58.811 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:45:58.811 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:45:58.811 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:45:58.811 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:45:58.811 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:45:58.812 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:45:58.812 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:45:58.812 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:45:58.812 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:45:58.812 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.812 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:45:58.812 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:45:58.812 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:45:58.812 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:45:58.812 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:45:58.812 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:45:58.812 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:45:58.812 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:45:58.812 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:45:58.813 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:45:58.813 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:45:58.813 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:45:58.813 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:45:58.813 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:45:58.813 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.813 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.813 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.813 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 3, 2024-09-11T07:45:58.813 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "datapool", 2024-09-11T07:45:58.813 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:30.396859+0000", 2024-09-11T07:45:58.813 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 8193, 2024-09-11T07:45:58.814 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T07:45:58.814 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:45:58.814 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:45:58.814 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:45:58.814 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:45:58.814 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:45:58.814 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:45:58.814 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:45:58.814 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:45:58.814 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:45:58.814 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:45:58.814 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 3, 2024-09-11T07:45:58.814 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 3, 2024-09-11T07:45:58.815 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 3, 2024-09-11T07:45:58.815 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 3, 2024-09-11T07:45:58.815 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 3, 2024-09-11T07:45:58.815 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:45:58.815 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:45:58.815 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:45:58.815 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:45:58.815 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:45:58.815 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:45:58.815 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:45:58.815 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.815 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "61", 2024-09-11T07:45:58.816 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:45:58.816 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:45:58.816 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:45:58.816 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:45:58.816 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:45:58.816 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 3, 2024-09-11T07:45:58.816 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 61, 2024-09-11T07:45:58.816 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:45:58.816 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:45:58.816 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:45:58.816 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:45:58.817 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:45:58.817 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:45:58.817 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:45:58.817 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:45:58.817 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:45:58.817 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:45:58.817 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:45:58.817 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:45:58.817 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:45:58.817 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:45:58.817 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:45:58.817 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:45:58.818 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:45:58.818 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:45:58.818 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:45:58.818 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.818 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:45:58.818 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:45:58.818 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:45:58.818 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:45:58.818 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:45:58.818 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:45:58.818 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:45:58.818 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:45:58.819 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:45:58.819 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:45:58.819 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:45:58.819 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:45:58.819 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:45:58.819 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd": {} 2024-09-11T07:45:58.819 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.819 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.819 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.819 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 4, 2024-09-11T07:45:58.819 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.log", 2024-09-11T07:45:58.819 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:31.922926+0000", 2024-09-11T07:45:58.819 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:45:58.819 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:45:58.820 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:45:58.820 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:45:58.820 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:45:58.820 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:45:58.820 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:45:58.820 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:45:58.820 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:45:58.820 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:45:58.820 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:45:58.820 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:45:58.820 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:45:58.821 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:45:58.821 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:45:58.821 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:45:58.821 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:45:58.821 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:45:58.821 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:45:58.821 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:45:58.821 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:45:58.821 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:45:58.821 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:45:58.821 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:45:58.821 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.821 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "60", 2024-09-11T07:45:58.822 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:45:58.822 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:45:58.822 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:45:58.822 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:45:58.822 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:45:58.822 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:45:58.822 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:45:58.822 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:45:58.822 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:45:58.822 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:45:58.822 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:45:58.822 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:45:58.822 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:45:58.823 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:45:58.823 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:45:58.823 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:45:58.823 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:45:58.823 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:45:58.823 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:45:58.823 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:45:58.823 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:45:58.823 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:45:58.823 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:45:58.823 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:45:58.823 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:45:58.823 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:45:58.823 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.823 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:45:58.823 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:45:58.824 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:45:58.824 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:45:58.824 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:45:58.824 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:45:58.824 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:45:58.824 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:45:58.824 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:45:58.824 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:45:58.824 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:45:58.824 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:45:58.824 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:45:58.824 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:45:58.824 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.824 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.825 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.825 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 5, 2024-09-11T07:45:58.825 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.control", 2024-09-11T07:45:58.825 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:34.007746+0000", 2024-09-11T07:45:58.825 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:45:58.825 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:45:58.825 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:45:58.825 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:45:58.825 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:45:58.825 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:45:58.825 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:45:58.825 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:45:58.825 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:45:58.826 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:45:58.826 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:45:58.826 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:45:58.826 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:45:58.826 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:45:58.826 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:45:58.826 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:45:58.826 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:45:58.826 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:45:58.826 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:45:58.826 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:45:58.827 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:45:58.827 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:45:58.827 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:45:58.827 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:45:58.827 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.827 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "62", 2024-09-11T07:45:58.827 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:45:58.827 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:45:58.827 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:45:58.827 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:45:58.827 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:45:58.827 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:45:58.827 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:45:58.827 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:45:58.828 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:45:58.828 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:45:58.828 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:45:58.828 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:45:58.828 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:45:58.828 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:45:58.828 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:45:58.828 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:45:58.828 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:45:58.828 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:45:58.828 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:45:58.828 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:45:58.828 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:45:58.828 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:45:58.829 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:45:58.829 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:45:58.829 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:45:58.829 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:45:58.829 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.829 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:45:58.829 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:45:58.829 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:45:58.829 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:45:58.829 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:45:58.829 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:45:58.829 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:45:58.830 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:45:58.830 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:45:58.830 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:45:58.830 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:45:58.830 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:45:58.830 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:45:58.830 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:45:58.830 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.830 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.830 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.830 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 6, 2024-09-11T07:45:58.830 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "default.rgw.meta", 2024-09-11T07:45:58.830 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:26:36.077215+0000", 2024-09-11T07:45:58.830 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:45:58.831 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:45:58.831 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:45:58.831 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:45:58.831 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:45:58.831 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:45:58.831 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:45:58.831 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:45:58.831 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:45:58.831 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:45:58.831 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:45:58.831 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:45:58.831 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 32, 2024-09-11T07:45:58.831 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 32, 2024-09-11T07:45:58.831 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 32, 2024-09-11T07:45:58.832 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 32, 2024-09-11T07:45:58.832 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 32, 2024-09-11T07:45:58.832 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:45:58.832 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:45:58.832 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:45:58.832 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:45:58.832 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:45:58.832 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:45:58.832 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:45:58.832 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.832 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "64", 2024-09-11T07:45:58.832 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:45:58.832 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:45:58.832 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:45:58.833 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:45:58.833 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:45:58.833 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:45:58.833 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:45:58.833 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:45:58.833 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:45:58.833 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:45:58.833 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:45:58.833 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:45:58.833 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:45:58.833 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:45:58.833 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:45:58.833 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:45:58.834 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:45:58.834 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:45:58.834 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:45:58.834 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:45:58.834 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:45:58.834 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:45:58.834 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:45:58.834 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:45:58.834 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:45:58.834 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:45:58.834 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.834 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:45:58.834 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:45:58.834 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:45:58.834 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:45:58.835 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:45:58.835 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:45:58.835 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:45:58.835 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:45:58.835 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:45:58.835 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:45:58.835 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:45:58.835 INFO:teuthology.orchestra.run.smithi022.stdout: "options": { 2024-09-11T07:45:58.835 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_bias": 4 2024-09-11T07:45:58.835 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.835 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": { 2024-09-11T07:45:58.835 INFO:teuthology.orchestra.run.smithi022.stdout: "rgw": {} 2024-09-11T07:45:58.835 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.835 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.836 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.836 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 11, 2024-09-11T07:45:58.836 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": "unique_pool_4", 2024-09-11T07:45:58.836 INFO:teuthology.orchestra.run.smithi022.stdout: "create_time": "2024-09-11T07:45:55.403826+0000", 2024-09-11T07:45:58.836 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2024-09-11T07:45:58.836 INFO:teuthology.orchestra.run.smithi022.stdout: "flags_names": "hashpspool", 2024-09-11T07:45:58.836 INFO:teuthology.orchestra.run.smithi022.stdout: "type": 1, 2024-09-11T07:45:58.836 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 3, 2024-09-11T07:45:58.836 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": 2, 2024-09-11T07:45:58.836 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_rule": 0, 2024-09-11T07:45:58.836 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_count": 0, 2024-09-11T07:45:58.836 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_target": 0, 2024-09-11T07:45:58.836 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T07:45:58.836 INFO:teuthology.orchestra.run.smithi022.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T07:45:58.837 INFO:teuthology.orchestra.run.smithi022.stdout: "object_hash": 2, 2024-09-11T07:45:58.837 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscale_mode": "off", 2024-09-11T07:45:58.837 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": 16, 2024-09-11T07:45:58.837 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num": 16, 2024-09-11T07:45:58.837 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_placement_num_target": 16, 2024-09-11T07:45:58.837 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_target": 16, 2024-09-11T07:45:58.837 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num_pending": 16, 2024-09-11T07:45:58.837 INFO:teuthology.orchestra.run.smithi022.stdout: "last_pg_merge_meta": { 2024-09-11T07:45:58.837 INFO:teuthology.orchestra.run.smithi022.stdout: "source_pgid": "0.0", 2024-09-11T07:45:58.837 INFO:teuthology.orchestra.run.smithi022.stdout: "ready_epoch": 0, 2024-09-11T07:45:58.837 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_started": 0, 2024-09-11T07:45:58.837 INFO:teuthology.orchestra.run.smithi022.stdout: "last_epoch_clean": 0, 2024-09-11T07:45:58.837 INFO:teuthology.orchestra.run.smithi022.stdout: "source_version": "0'0", 2024-09-11T07:45:58.837 INFO:teuthology.orchestra.run.smithi022.stdout: "target_version": "0'0" 2024-09-11T07:45:58.838 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.838 INFO:teuthology.orchestra.run.smithi022.stdout: "last_change": "414", 2024-09-11T07:45:58.838 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend": "0", 2024-09-11T07:45:58.838 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T07:45:58.838 INFO:teuthology.orchestra.run.smithi022.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T07:45:58.838 INFO:teuthology.orchestra.run.smithi022.stdout: "auid": 0, 2024-09-11T07:45:58.838 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_mode": "selfmanaged", 2024-09-11T07:45:58.838 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_seq": 0, 2024-09-11T07:45:58.838 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_epoch": 0, 2024-09-11T07:45:58.838 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_snaps": [], 2024-09-11T07:45:58.838 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps": "[]", 2024-09-11T07:45:58.838 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_bytes": 0, 2024-09-11T07:45:58.838 INFO:teuthology.orchestra.run.smithi022.stdout: "quota_max_objects": 0, 2024-09-11T07:45:58.838 INFO:teuthology.orchestra.run.smithi022.stdout: "tiers": [], 2024-09-11T07:45:58.839 INFO:teuthology.orchestra.run.smithi022.stdout: "tier_of": -1, 2024-09-11T07:45:58.839 INFO:teuthology.orchestra.run.smithi022.stdout: "read_tier": -1, 2024-09-11T07:45:58.839 INFO:teuthology.orchestra.run.smithi022.stdout: "write_tier": -1, 2024-09-11T07:45:58.839 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_mode": "none", 2024-09-11T07:45:58.839 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_bytes": 0, 2024-09-11T07:45:58.839 INFO:teuthology.orchestra.run.smithi022.stdout: "target_max_objects": 0, 2024-09-11T07:45:58.839 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T07:45:58.839 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T07:45:58.839 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T07:45:58.839 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_flush_age": 0, 2024-09-11T07:45:58.839 INFO:teuthology.orchestra.run.smithi022.stdout: "cache_min_evict_age": 0, 2024-09-11T07:45:58.839 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profile": "", 2024-09-11T07:45:58.839 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_params": { 2024-09-11T07:45:58.839 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "none" 2024-09-11T07:45:58.840 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.840 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_period": 0, 2024-09-11T07:45:58.840 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_count": 0, 2024-09-11T07:45:58.840 INFO:teuthology.orchestra.run.smithi022.stdout: "use_gmt_hitset": true, 2024-09-11T07:45:58.840 INFO:teuthology.orchestra.run.smithi022.stdout: "min_read_recency_for_promote": 0, 2024-09-11T07:45:58.840 INFO:teuthology.orchestra.run.smithi022.stdout: "min_write_recency_for_promote": 0, 2024-09-11T07:45:58.840 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T07:45:58.840 INFO:teuthology.orchestra.run.smithi022.stdout: "hit_set_search_last_n": 0, 2024-09-11T07:45:58.840 INFO:teuthology.orchestra.run.smithi022.stdout: "grade_table": [], 2024-09-11T07:45:58.840 INFO:teuthology.orchestra.run.smithi022.stdout: "stripe_width": 0, 2024-09-11T07:45:58.840 INFO:teuthology.orchestra.run.smithi022.stdout: "expected_num_objects": 0, 2024-09-11T07:45:58.840 INFO:teuthology.orchestra.run.smithi022.stdout: "fast_read": false, 2024-09-11T07:45:58.840 INFO:teuthology.orchestra.run.smithi022.stdout: "options": {}, 2024-09-11T07:45:58.841 INFO:teuthology.orchestra.run.smithi022.stdout: "application_metadata": {} 2024-09-11T07:45:58.841 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.841 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:45:58.841 INFO:teuthology.orchestra.run.smithi022.stdout: "osds": [ 2024-09-11T07:45:58.841 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.841 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 0, 2024-09-11T07:45:58.841 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "09075f16-32f4-4f88-a116-c3d7f977a0e0", 2024-09-11T07:45:58.841 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:45:58.841 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:45:58.841 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:45:58.841 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:45:58.841 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:45:58.841 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:45:58.841 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 8, 2024-09-11T07:45:58.842 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 412, 2024-09-11T07:45:58.842 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:45:58.842 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:45:58.842 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:45:58.842 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.842 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.842 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.842 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6801", 2024-09-11T07:45:58.842 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:45:58.842 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.842 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.842 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.842 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:45:58.842 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.843 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.843 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.843 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6802", 2024-09-11T07:45:58.843 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:45:58.843 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.843 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.843 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.843 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:45:58.843 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.843 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.843 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.843 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6804", 2024-09-11T07:45:58.843 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:45:58.843 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.843 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.844 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.844 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:45:58.844 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.844 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.844 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.844 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6803", 2024-09-11T07:45:58.844 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2821493591 2024-09-11T07:45:58.844 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.844 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.844 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.844 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6801/2821493591", 2024-09-11T07:45:58.844 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6802/2821493591", 2024-09-11T07:45:58.844 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6804/2821493591", 2024-09-11T07:45:58.844 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6803/2821493591", 2024-09-11T07:45:58.845 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:45:58.845 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:45:58.845 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:45:58.845 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.845 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.845 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.845 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 1, 2024-09-11T07:45:58.845 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "52cc1e06-9506-48c0-9f7b-d4270dff4b9f", 2024-09-11T07:45:58.845 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:45:58.845 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:45:58.845 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 0.64752197265625, 2024-09-11T07:45:58.845 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 0, 2024-09-11T07:45:58.845 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:45:58.845 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:45:58.846 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 13, 2024-09-11T07:45:58.846 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 396, 2024-09-11T07:45:58.846 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:45:58.846 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:45:58.846 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:45:58.846 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.846 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.846 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.846 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6805", 2024-09-11T07:45:58.846 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:45:58.846 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.846 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.846 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.846 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:45:58.846 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.847 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.847 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.847 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6806", 2024-09-11T07:45:58.847 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:45:58.847 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.847 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.847 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.847 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:45:58.847 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.847 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.847 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.847 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6808", 2024-09-11T07:45:58.847 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:45:58.847 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.848 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.848 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.848 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:45:58.848 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.848 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.848 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.848 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6807", 2024-09-11T07:45:58.848 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4037911317 2024-09-11T07:45:58.848 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.848 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.848 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.848 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6805/4037911317", 2024-09-11T07:45:58.848 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6806/4037911317", 2024-09-11T07:45:58.848 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6808/4037911317", 2024-09-11T07:45:58.848 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6807/4037911317", 2024-09-11T07:45:58.849 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:45:58.849 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:45:58.849 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:45:58.849 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.849 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.849 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.849 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 2, 2024-09-11T07:45:58.849 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "fa2b2730-2c1d-45d0-be8a-494a60dc6b9b", 2024-09-11T07:45:58.918 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:45:58.918 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:45:58.918 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:45:58.918 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:45:58.918 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:45:58.918 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:45:58.918 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 18, 2024-09-11T07:45:58.918 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 412, 2024-09-11T07:45:58.918 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:45:58.919 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:45:58.919 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:45:58.919 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.919 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.919 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.919 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6809", 2024-09-11T07:45:58.919 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:45:58.919 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.919 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.919 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.919 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:45:58.919 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.919 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.919 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.919 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6810", 2024-09-11T07:45:58.920 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:45:58.920 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.920 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.920 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.920 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:45:58.920 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.920 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.920 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.920 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6812", 2024-09-11T07:45:58.920 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:45:58.920 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.920 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.920 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.920 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:45:58.920 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.921 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.921 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.921 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6811", 2024-09-11T07:45:58.921 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1675799666 2024-09-11T07:45:58.921 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.921 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.921 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.921 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6809/1675799666", 2024-09-11T07:45:58.921 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6810/1675799666", 2024-09-11T07:45:58.921 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6812/1675799666", 2024-09-11T07:45:58.921 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6811/1675799666", 2024-09-11T07:45:58.921 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:45:58.921 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:45:58.922 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:45:58.922 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.922 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.922 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.922 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 3, 2024-09-11T07:45:58.922 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "8d53d3fc-e401-47df-b70b-4b4b1f4539df", 2024-09-11T07:45:58.922 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:45:58.922 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:45:58.922 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 0.686370849609375, 2024-09-11T07:45:58.922 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:45:58.922 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:45:58.922 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:45:58.922 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 25, 2024-09-11T07:45:58.923 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 412, 2024-09-11T07:45:58.923 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:45:58.923 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:45:58.923 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:45:58.923 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.923 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.923 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.923 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6813", 2024-09-11T07:45:58.923 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:45:58.923 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.923 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.923 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.923 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:45:58.923 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.924 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.924 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.924 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6814", 2024-09-11T07:45:58.924 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:45:58.924 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.924 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.924 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.924 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:45:58.924 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.924 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.924 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.924 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6816", 2024-09-11T07:45:58.924 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:45:58.924 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.925 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.925 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.925 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:45:58.925 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.925 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.925 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.925 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6815", 2024-09-11T07:45:58.925 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1501962752 2024-09-11T07:45:58.925 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.925 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.925 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.925 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.22:6813/1501962752", 2024-09-11T07:45:58.925 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.22:6814/1501962752", 2024-09-11T07:45:58.925 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.22:6816/1501962752", 2024-09-11T07:45:58.926 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.22:6815/1501962752", 2024-09-11T07:45:58.926 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:45:58.926 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:45:58.926 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:45:58.926 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.926 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.926 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.926 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 4, 2024-09-11T07:45:58.926 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "d3207fe0-8b97-4067-a755-ba6d449c53c7", 2024-09-11T07:45:58.926 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:45:58.926 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 0, 2024-09-11T07:45:58.926 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 0, 2024-09-11T07:45:58.927 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:45:58.927 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:45:58.927 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:45:58.927 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 31, 2024-09-11T07:45:58.927 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 390, 2024-09-11T07:45:58.927 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:45:58.927 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:45:58.927 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:45:58.927 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.927 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.927 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.927 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6800", 2024-09-11T07:45:58.927 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:45:58.927 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.928 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.928 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.928 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:45:58.928 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.928 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.928 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.928 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6801", 2024-09-11T07:45:58.928 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:45:58.928 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.928 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.928 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.928 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:45:58.928 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.929 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.929 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.929 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6803", 2024-09-11T07:45:58.929 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:45:58.929 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.929 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.929 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.929 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:45:58.929 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.929 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.929 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.929 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6802", 2024-09-11T07:45:58.929 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1210229443 2024-09-11T07:45:58.930 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.930 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.930 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.930 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6800/1210229443", 2024-09-11T07:45:58.930 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6801/1210229443", 2024-09-11T07:45:58.930 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6803/1210229443", 2024-09-11T07:45:58.930 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6802/1210229443", 2024-09-11T07:45:58.930 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:45:58.930 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:45:58.930 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:45:58.930 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.930 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.930 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.930 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 5, 2024-09-11T07:45:58.931 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "89c9352b-30b2-48c6-869e-047d07a23a43", 2024-09-11T07:45:58.931 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:45:58.931 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:45:58.931 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 0.6509552001953125, 2024-09-11T07:45:58.931 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 0.6031036376953125, 2024-09-11T07:45:58.931 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:45:58.931 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:45:58.931 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 37, 2024-09-11T07:45:58.931 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 412, 2024-09-11T07:45:58.931 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:45:58.931 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:45:58.931 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:45:58.931 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.931 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.932 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.932 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6804", 2024-09-11T07:45:58.932 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:45:58.932 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.932 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.932 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.932 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:45:58.932 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.932 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.932 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.932 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6805", 2024-09-11T07:45:58.932 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:45:58.932 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.932 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.933 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.933 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:45:58.933 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.933 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.933 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.933 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6807", 2024-09-11T07:45:58.933 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:45:58.933 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.933 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.934 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.934 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:45:58.934 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.934 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.934 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.934 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6806", 2024-09-11T07:45:58.934 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2735328732 2024-09-11T07:45:58.934 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.934 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.934 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.934 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6804/2735328732", 2024-09-11T07:45:58.934 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6805/2735328732", 2024-09-11T07:45:58.934 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6807/2735328732", 2024-09-11T07:45:58.934 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6806/2735328732", 2024-09-11T07:45:58.934 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:45:58.935 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:45:58.935 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:45:58.935 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.935 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.935 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.935 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 6, 2024-09-11T07:45:58.935 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "704563de-d88f-480a-b469-720135704d42", 2024-09-11T07:45:58.935 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:45:58.935 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:45:58.935 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 1, 2024-09-11T07:45:58.935 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 0, 2024-09-11T07:45:58.935 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 0, 2024-09-11T07:45:58.936 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 0, 2024-09-11T07:45:58.936 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 44, 2024-09-11T07:45:58.936 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 394, 2024-09-11T07:45:58.936 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 0, 2024-09-11T07:45:58.936 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:45:58.936 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:45:58.936 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.936 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.936 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.936 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6808", 2024-09-11T07:45:58.936 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:45:58.936 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.936 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.936 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.937 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:45:58.937 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.937 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.937 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.937 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6809", 2024-09-11T07:45:58.937 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:45:58.937 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.937 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.937 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.937 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:45:58.937 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.937 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.937 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.937 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6811", 2024-09-11T07:45:58.938 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:45:58.938 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.938 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.938 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.938 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:45:58.938 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.938 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.938 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.938 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6810", 2024-09-11T07:45:58.938 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 721493010 2024-09-11T07:45:58.938 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.938 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.938 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.938 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6808/721493010", 2024-09-11T07:45:58.938 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6809/721493010", 2024-09-11T07:45:58.939 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6811/721493010", 2024-09-11T07:45:58.939 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6810/721493010", 2024-09-11T07:45:58.939 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:45:58.939 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:45:58.939 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:45:58.939 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.939 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.939 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.939 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 7, 2024-09-11T07:45:58.939 INFO:teuthology.orchestra.run.smithi022.stdout: "uuid": "18199e47-0052-49f0-b45f-0fb201e0d0f4", 2024-09-11T07:45:58.939 INFO:teuthology.orchestra.run.smithi022.stdout: "up": 1, 2024-09-11T07:45:58.939 INFO:teuthology.orchestra.run.smithi022.stdout: "in": 1, 2024-09-11T07:45:58.939 INFO:teuthology.orchestra.run.smithi022.stdout: "weight": 0.6657562255859375, 2024-09-11T07:45:58.939 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_affinity": 1, 2024-09-11T07:45:58.940 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_begin": 52, 2024-09-11T07:45:58.940 INFO:teuthology.orchestra.run.smithi022.stdout: "last_clean_end": 133, 2024-09-11T07:45:58.940 INFO:teuthology.orchestra.run.smithi022.stdout: "up_from": 147, 2024-09-11T07:45:58.940 INFO:teuthology.orchestra.run.smithi022.stdout: "up_thru": 412, 2024-09-11T07:45:58.940 INFO:teuthology.orchestra.run.smithi022.stdout: "down_at": 134, 2024-09-11T07:45:58.940 INFO:teuthology.orchestra.run.smithi022.stdout: "lost_at": 0, 2024-09-11T07:45:58.940 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addrs": { 2024-09-11T07:45:58.940 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.940 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.940 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.940 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6812", 2024-09-11T07:45:58.940 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2218863853 2024-09-11T07:45:58.940 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.940 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.940 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.941 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addrs": { 2024-09-11T07:45:58.941 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.941 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.941 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.941 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6813", 2024-09-11T07:45:58.941 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2218863853 2024-09-11T07:45:58.941 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.941 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.941 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.941 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addrs": { 2024-09-11T07:45:58.941 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.941 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.941 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.942 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6815", 2024-09-11T07:45:58.942 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2218863853 2024-09-11T07:45:58.942 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.942 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.942 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.942 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addrs": { 2024-09-11T07:45:58.942 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2024-09-11T07:45:58.942 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.942 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2024-09-11T07:45:58.942 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.104:6814", 2024-09-11T07:45:58.942 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2218863853 2024-09-11T07:45:58.942 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.942 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.942 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.942 INFO:teuthology.orchestra.run.smithi022.stdout: "public_addr": "172.21.15.104:6812/2218863853", 2024-09-11T07:45:58.943 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_addr": "172.21.15.104:6813/2218863853", 2024-09-11T07:45:58.943 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_back_addr": "172.21.15.104:6815/2218863853", 2024-09-11T07:45:58.943 INFO:teuthology.orchestra.run.smithi022.stdout: "heartbeat_front_addr": "172.21.15.104:6814/2218863853", 2024-09-11T07:45:58.943 INFO:teuthology.orchestra.run.smithi022.stdout: "state": [ 2024-09-11T07:45:58.943 INFO:teuthology.orchestra.run.smithi022.stdout: "exists", 2024-09-11T07:45:58.943 INFO:teuthology.orchestra.run.smithi022.stdout: "up" 2024-09-11T07:45:58.943 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.943 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.943 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:45:58.943 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_xinfo": [ 2024-09-11T07:45:58.943 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.943 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 0, 2024-09-11T07:45:58.943 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:45:58.943 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:45:58.943 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:45:58.943 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:45:58.944 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:45:58.944 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:45:00.943950+0000", 2024-09-11T07:45:58.944 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:45:58.944 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.944 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.944 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 1, 2024-09-11T07:45:58.944 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:45:58.944 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:45:58.944 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:45:58.944 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:45:58.944 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:45:58.944 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:45:43.172433+0000", 2024-09-11T07:45:58.944 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:45:58.944 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.945 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.945 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 2, 2024-09-11T07:45:58.945 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:45:58.945 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:45:58.945 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:45:58.945 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:45:58.945 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:45:58.945 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:45:50.217864+0000", 2024-09-11T07:45:58.945 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:45:58.945 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.945 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.945 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 3, 2024-09-11T07:45:58.945 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:45:58.945 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:45:58.946 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:45:58.946 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:45:58.946 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:45:58.946 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:44:49.215826+0000", 2024-09-11T07:45:58.946 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:45:58.946 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.946 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.946 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 4, 2024-09-11T07:45:58.946 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:45:58.946 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:45:58.946 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:45:58.946 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:45:58.946 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 65536, 2024-09-11T07:45:58.946 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:45:52.350229+0000", 2024-09-11T07:45:58.946 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:45:58.947 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.947 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.947 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 5, 2024-09-11T07:45:58.947 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:45:58.947 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:45:58.947 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:45:58.947 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:45:58.947 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:45:58.947 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:45:45.036502+0000", 2024-09-11T07:45:58.947 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:45:58.947 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.947 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.947 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 6, 2024-09-11T07:45:58.947 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:45:58.947 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:45:58.948 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:45:58.948 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:45:58.948 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:45:58.948 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:45:47.044101+0000", 2024-09-11T07:45:58.948 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 0 2024-09-11T07:45:58.948 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.948 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.948 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": 7, 2024-09-11T07:45:58.948 INFO:teuthology.orchestra.run.smithi022.stdout: "down_stamp": "0.000000", 2024-09-11T07:45:58.948 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_probability": 0, 2024-09-11T07:45:58.948 INFO:teuthology.orchestra.run.smithi022.stdout: "laggy_interval": 0, 2024-09-11T07:45:58.948 INFO:teuthology.orchestra.run.smithi022.stdout: "features": 4540138320759226367, 2024-09-11T07:45:58.948 INFO:teuthology.orchestra.run.smithi022.stdout: "old_weight": 0, 2024-09-11T07:45:58.948 INFO:teuthology.orchestra.run.smithi022.stdout: "last_purged_snaps_scrub": "2024-09-11T07:45:57.135385+0000", 2024-09-11T07:45:58.948 INFO:teuthology.orchestra.run.smithi022.stdout: "dead_epoch": 133 2024-09-11T07:45:58.949 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.949 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:45:58.949 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_upmap": [ 2024-09-11T07:45:58.949 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.949 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "2.1c", 2024-09-11T07:45:58.949 INFO:teuthology.orchestra.run.smithi022.stdout: "osds": [ 2024-09-11T07:45:58.949 INFO:teuthology.orchestra.run.smithi022.stdout: 5, 2024-09-11T07:45:58.949 INFO:teuthology.orchestra.run.smithi022.stdout: 2, 2024-09-11T07:45:58.949 INFO:teuthology.orchestra.run.smithi022.stdout: 7 2024-09-11T07:45:58.949 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.949 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.949 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.949 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.1e", 2024-09-11T07:45:58.949 INFO:teuthology.orchestra.run.smithi022.stdout: "osds": [ 2024-09-11T07:45:58.949 INFO:teuthology.orchestra.run.smithi022.stdout: 4, 2024-09-11T07:45:58.950 INFO:teuthology.orchestra.run.smithi022.stdout: 5, 2024-09-11T07:45:58.950 INFO:teuthology.orchestra.run.smithi022.stdout: 3 2024-09-11T07:45:58.950 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.950 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.950 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:45:58.950 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_upmap_items": [ 2024-09-11T07:45:58.950 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.950 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.7", 2024-09-11T07:45:58.950 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:45:58.950 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.950 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 3, 2024-09-11T07:45:58.950 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 7 2024-09-11T07:45:58.950 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.950 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.950 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.950 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.951 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.15", 2024-09-11T07:45:58.951 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:45:58.951 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.951 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 3, 2024-09-11T07:45:58.951 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 0 2024-09-11T07:45:58.951 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.951 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.951 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.951 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.951 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "4.1b", 2024-09-11T07:45:58.951 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:45:58.951 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.951 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 3, 2024-09-11T07:45:58.951 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 7 2024-09-11T07:45:58.951 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.951 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.952 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.952 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.952 INFO:teuthology.orchestra.run.smithi022.stdout: "pgid": "5.7", 2024-09-11T07:45:58.952 INFO:teuthology.orchestra.run.smithi022.stdout: "mappings": [ 2024-09-11T07:45:58.952 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:45:58.952 INFO:teuthology.orchestra.run.smithi022.stdout: "from": 1, 2024-09-11T07:45:58.952 INFO:teuthology.orchestra.run.smithi022.stdout: "to": 2 2024-09-11T07:45:58.952 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.952 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:45:58.952 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.952 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-11T07:45:58.952 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_temp": [], 2024-09-11T07:45:58.952 INFO:teuthology.orchestra.run.smithi022.stdout: "primary_temp": [], 2024-09-11T07:45:58.953 INFO:teuthology.orchestra.run.smithi022.stdout: "blocklist": { 2024-09-11T07:45:58.953 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/240999923": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:45:58.953 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/4016369460": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:45:58.953 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/585929238": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:45:58.953 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/544031024": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:45:58.953 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/2504196132": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:45:58.953 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/1451388349": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:45:58.953 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/639336081": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:45:58.953 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2104069730": "2024-09-12T07:27:07.919998+0000", 2024-09-11T07:45:58.953 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/4236450731": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:45:58.953 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:6800/3573193776": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:45:58.953 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2661764422": "2024-09-12T07:21:48.438371+0000", 2024-09-11T07:45:58.953 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2197488530": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:45:58.953 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/2968034970": "2024-09-12T07:22:11.238618+0000", 2024-09-11T07:45:58.953 INFO:teuthology.orchestra.run.smithi022.stdout: "172.21.15.22:0/599318175": "2024-09-12T07:27:07.919998+0000" 2024-09-11T07:45:58.953 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.954 INFO:teuthology.orchestra.run.smithi022.stdout: "range_blocklist": {}, 2024-09-11T07:45:58.954 INFO:teuthology.orchestra.run.smithi022.stdout: "erasure_code_profiles": { 2024-09-11T07:45:58.954 INFO:teuthology.orchestra.run.smithi022.stdout: "default": { 2024-09-11T07:45:58.954 INFO:teuthology.orchestra.run.smithi022.stdout: "crush-failure-domain": "osd", 2024-09-11T07:45:58.954 INFO:teuthology.orchestra.run.smithi022.stdout: "k": "2", 2024-09-11T07:45:58.954 INFO:teuthology.orchestra.run.smithi022.stdout: "m": "1", 2024-09-11T07:45:58.954 INFO:teuthology.orchestra.run.smithi022.stdout: "plugin": "jerasure", 2024-09-11T07:45:58.954 INFO:teuthology.orchestra.run.smithi022.stdout: "technique": "reed_sol_van" 2024-09-11T07:45:58.954 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.954 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:45:58.954 INFO:teuthology.orchestra.run.smithi022.stdout: "removed_snaps_queue": [], 2024-09-11T07:45:58.954 INFO:teuthology.orchestra.run.smithi022.stdout: "new_removed_snaps": [], 2024-09-11T07:45:58.954 INFO:teuthology.orchestra.run.smithi022.stdout: "new_purged_snaps": [], 2024-09-11T07:45:58.954 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_node_flags": {}, 2024-09-11T07:45:58.955 INFO:teuthology.orchestra.run.smithi022.stdout: "device_class_flags": {}, 2024-09-11T07:45:58.955 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode": { 2024-09-11T07:45:58.955 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode_enabled": false, 2024-09-11T07:45:58.955 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_bucket_count": 0, 2024-09-11T07:45:58.955 INFO:teuthology.orchestra.run.smithi022.stdout: "degraded_stretch_mode": 0, 2024-09-11T07:45:58.955 INFO:teuthology.orchestra.run.smithi022.stdout: "recovering_stretch_mode": 0, 2024-09-11T07:45:58.955 INFO:teuthology.orchestra.run.smithi022.stdout: "stretch_mode_bucket": 0 2024-09-11T07:45:58.955 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:45:58.955 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:45:58.955 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:58.980 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:58.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:58.981 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:59.077 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:58 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1798475810' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:45:59.077 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:58 smithi022 ceph-mon[29462]: pgmap v910: 158 pgs: 16 unknown, 142 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:45:59.077 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:45:58 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3594856458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:45:59.078 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:58 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1798475810' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:45:59.078 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:58 smithi022 ceph-mon[37330]: pgmap v910: 158 pgs: 16 unknown, 142 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:45:59.078 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:45:58 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3594856458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:45:59.081 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:59.097 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:58 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1798475810' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:45:59.097 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:58 smithi104 ceph-mon[33507]: pgmap v910: 158 pgs: 16 unknown, 142 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:45:59.097 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:45:58 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3594856458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-11T07:45:59.105 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:59.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:59.106 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:59.207 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:59.228 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:59.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:59.229 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:59.329 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:59.350 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:59.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:59.351 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:59.452 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:59.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:59.474 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:59.475 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:59.565 INFO:tasks.thrashosds.thrasher:j is {'epoch': 414, 'fsid': '12a285d8-700e-11ef-bcea-c7b262605968', 'created': '2024-09-11T07:21:18.695332+0000', 'modified': '2024-09-11T07:45:57.747027+0000', 'last_up_change': '2024-09-11T07:32:00.070825+0000', 'last_in_change': '2024-09-11T07:45:26.061430+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 19, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 11, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T07:24:21.772426+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 11, 'pg_placement_num': 11, 'pg_placement_num_target': 11, 'pg_num_target': 11, 'pg_num_pending': 11, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '390', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '388', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': '.rgw.root', 'create_time': '2024-09-11T07:26:29.941354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 3, 'pool_name': 'datapool', 'create_time': '2024-09-11T07:26:30.396859+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-11T07:26:31.922926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-11T07:26:34.007746+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-11T07:26:36.077215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}, {'pool': 11, 'pool_name': 'unique_pool_4', 'create_time': '2024-09-11T07:45:55.403826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '414', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}}], 'osds': [{'osd': 0, 'uuid': '09075f16-32f4-4f88-a116-c3d7f977a0e0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 412, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6801', 'nonce': 2821493591}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6802', 'nonce': 2821493591}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6804', 'nonce': 2821493591}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6803', 'nonce': 2821493591}]}, 'public_addr': '172.21.15.22:6801/2821493591', 'cluster_addr': '172.21.15.22:6802/2821493591', 'heartbeat_back_addr': '172.21.15.22:6804/2821493591', 'heartbeat_front_addr': '172.21.15.22:6803/2821493591', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '52cc1e06-9506-48c0-9f7b-d4270dff4b9f', 'up': 1, 'in': 1, 'weight': 0.64752197265625, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 396, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6805', 'nonce': 4037911317}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6806', 'nonce': 4037911317}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6808', 'nonce': 4037911317}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6807', 'nonce': 4037911317}]}, 'public_addr': '172.21.15.22:6805/4037911317', 'cluster_addr': '172.21.15.22:6806/4037911317', 'heartbeat_back_addr': '172.21.15.22:6808/4037911317', 'heartbeat_front_addr': '172.21.15.22:6807/4037911317', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'fa2b2730-2c1d-45d0-be8a-494a60dc6b9b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 412, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6809', 'nonce': 1675799666}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6810', 'nonce': 1675799666}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6812', 'nonce': 1675799666}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6811', 'nonce': 1675799666}]}, 'public_addr': '172.21.15.22:6809/1675799666', 'cluster_addr': '172.21.15.22:6810/1675799666', 'heartbeat_back_addr': '172.21.15.22:6812/1675799666', 'heartbeat_front_addr': '172.21.15.22:6811/1675799666', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '8d53d3fc-e401-47df-b70b-4b4b1f4539df', 'up': 1, 'in': 1, 'weight': 0.686370849609375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 25, 'up_thru': 412, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6813', 'nonce': 1501962752}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6814', 'nonce': 1501962752}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6816', 'nonce': 1501962752}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.22:6815', 'nonce': 1501962752}]}, 'public_addr': '172.21.15.22:6813/1501962752', 'cluster_addr': '172.21.15.22:6814/1501962752', 'heartbeat_back_addr': '172.21.15.22:6816/1501962752', 'heartbeat_front_addr': '172.21.15.22:6815/1501962752', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'd3207fe0-8b97-4067-a755-ba6d449c53c7', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 390, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6800', 'nonce': 1210229443}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6801', 'nonce': 1210229443}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6803', 'nonce': 1210229443}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6802', 'nonce': 1210229443}]}, 'public_addr': '172.21.15.104:6800/1210229443', 'cluster_addr': '172.21.15.104:6801/1210229443', 'heartbeat_back_addr': '172.21.15.104:6803/1210229443', 'heartbeat_front_addr': '172.21.15.104:6802/1210229443', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '89c9352b-30b2-48c6-869e-047d07a23a43', 'up': 1, 'in': 1, 'weight': 0.6509552001953125, 'primary_affinity': 0.6031036376953125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 37, 'up_thru': 412, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6804', 'nonce': 2735328732}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6805', 'nonce': 2735328732}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6807', 'nonce': 2735328732}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6806', 'nonce': 2735328732}]}, 'public_addr': '172.21.15.104:6804/2735328732', 'cluster_addr': '172.21.15.104:6805/2735328732', 'heartbeat_back_addr': '172.21.15.104:6807/2735328732', 'heartbeat_front_addr': '172.21.15.104:6806/2735328732', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '704563de-d88f-480a-b469-720135704d42', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 44, 'up_thru': 394, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6808', 'nonce': 721493010}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6809', 'nonce': 721493010}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6811', 'nonce': 721493010}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6810', 'nonce': 721493010}]}, 'public_addr': '172.21.15.104:6808/721493010', 'cluster_addr': '172.21.15.104:6809/721493010', 'heartbeat_back_addr': '172.21.15.104:6811/721493010', 'heartbeat_front_addr': '172.21.15.104:6810/721493010', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '18199e47-0052-49f0-b45f-0fb201e0d0f4', 'up': 1, 'in': 1, 'weight': 0.6657562255859375, 'primary_affinity': 1, 'last_clean_begin': 52, 'last_clean_end': 133, 'up_from': 147, 'up_thru': 412, 'down_at': 134, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6812', 'nonce': 2218863853}]}, 'cluster_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6813', 'nonce': 2218863853}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6815', 'nonce': 2218863853}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.104:6814', 'nonce': 2218863853}]}, 'public_addr': '172.21.15.104:6812/2218863853', 'cluster_addr': '172.21.15.104:6813/2218863853', 'heartbeat_back_addr': '172.21.15.104:6815/2218863853', 'heartbeat_front_addr': '172.21.15.104:6814/2218863853', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:45:00.943950+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:45:43.172433+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:45:50.217864+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:44:49.215826+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-11T07:45:52.350229+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:45:45.036502+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:45:47.044101+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T07:45:57.135385+0000', 'dead_epoch': 133}], 'pg_upmap': [{'pgid': '2.1c', 'osds': [5, 2, 7]}, {'pgid': '4.1e', 'osds': [4, 5, 3]}], 'pg_upmap_items': [{'pgid': '4.7', 'mappings': [{'from': 3, 'to': 7}]}, {'pgid': '4.15', 'mappings': [{'from': 3, 'to': 0}]}, {'pgid': '4.1b', 'mappings': [{'from': 3, 'to': 7}]}, {'pgid': '5.7', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.22:0/240999923': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:6800/4016369460': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/585929238': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/544031024': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:6800/2504196132': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/1451388349': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/639336081': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/2104069730': '2024-09-12T07:27:07.919998+0000', '172.21.15.22:0/4236450731': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:6800/3573193776': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/2661764422': '2024-09-12T07:21:48.438371+0000', '172.21.15.22:0/2197488530': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/2968034970': '2024-09-12T07:22:11.238618+0000', '172.21.15.22:0/599318175': '2024-09-12T07:27:07.919998+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-11T07:45:59.565 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:45:59.575 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:45:59.630 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:45:59.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:59.631 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:45:59.732 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:45:59.753 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:59.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:59.754 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:45:59.855 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:45:59.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:59.878 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:45:59.878 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:45:59.979 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:00.002 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:00.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:00.003 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:00.103 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:00.125 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:00.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:00.126 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:00.181 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:46:00.226 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:00.248 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:00.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:00.249 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:00.349 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:00.372 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:00.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:00.373 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:00.455 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:46:00 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:46:00.474 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:00.476 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:46:00.476 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2024-09-11T07:46:00.476 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2024-09-11T07:46:00.477 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2024-09-11T07:46:00.477 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.477 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87214 9.10 9.cbc93d10 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:46:00.477 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.358568+0000", 2024-09-11T07:46:00.477 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 452.11403282700002, 2024-09-11T07:46:00.477 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.98231133800000003, 2024-09-11T07:46:00.477 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:00.477 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:46:00.477 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:00.477 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:00.477 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:00.477 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87214 2024-09-11T07:46:00.477 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.478 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:00.478 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.478 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:00.478 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.358568+0000", 2024-09-11T07:46:00.478 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.478 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.478 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.478 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:00.478 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.358568+0000", 2024-09-11T07:46:00.478 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.478 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.478 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.478 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:00.479 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.358565+0000", 2024-09-11T07:46:00.479 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999971 2024-09-11T07:46:00.479 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.479 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.479 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:00.479 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.358689+0000", 2024-09-11T07:46:00.479 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012315399999999999 2024-09-11T07:46:00.479 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.479 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.479 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:00.479 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.358704+0000", 2024-09-11T07:46:00.479 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5931e-05 2024-09-11T07:46:00.479 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.479 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.480 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:00.480 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.358711+0000", 2024-09-11T07:46:00.480 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7410000000000003e-06 2024-09-11T07:46:00.480 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.480 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.480 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.480 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.358765+0000", 2024-09-11T07:46:00.480 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3458000000000002e-05 2024-09-11T07:46:00.480 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.480 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.480 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:00.480 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.358773+0000", 2024-09-11T07:46:00.480 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.892e-06 2024-09-11T07:46:00.480 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.481 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.481 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.481 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.340867+0000", 2024-09-11T07:46:00.481 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.98209449400000004 2024-09-11T07:46:00.481 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.481 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.481 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:00.481 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.340880+0000", 2024-09-11T07:46:00.481 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2479e-05 2024-09-11T07:46:00.481 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.481 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:00.481 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.481 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.481 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.482 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87241 9.16 9.52d9f9b6 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:46:00.482 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.372509+0000", 2024-09-11T07:46:00.482 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 452.10009239499999, 2024-09-11T07:46:00.482 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.97069455699999996, 2024-09-11T07:46:00.482 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:00.482 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:46:00.482 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:00.482 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:00.482 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:00.482 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87241 2024-09-11T07:46:00.482 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.482 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:00.482 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.482 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:00.483 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372509+0000", 2024-09-11T07:46:00.483 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.483 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.483 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.483 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:00.483 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372509+0000", 2024-09-11T07:46:00.483 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.483 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.483 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.483 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:00.483 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372506+0000", 2024-09-11T07:46:00.483 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999971 2024-09-11T07:46:00.483 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.483 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.484 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:00.484 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372572+0000", 2024-09-11T07:46:00.484 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6177e-05 2024-09-11T07:46:00.484 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.484 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.484 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:00.484 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372586+0000", 2024-09-11T07:46:00.484 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3866000000000001e-05 2024-09-11T07:46:00.484 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.484 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.484 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:00.484 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372592+0000", 2024-09-11T07:46:00.484 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8490000000000002e-06 2024-09-11T07:46:00.485 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.485 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.485 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.485 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372635+0000", 2024-09-11T07:46:00.485 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3566999999999998e-05 2024-09-11T07:46:00.485 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.485 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.485 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:00.485 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372644+0000", 2024-09-11T07:46:00.485 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4020000000000002e-06 2024-09-11T07:46:00.485 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.485 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.485 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.485 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.343195+0000", 2024-09-11T07:46:00.486 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.97055083499999995 2024-09-11T07:46:00.486 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.486 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.486 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:00.486 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.343203+0000", 2024-09-11T07:46:00.486 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6880000000000005e-06 2024-09-11T07:46:00.486 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.486 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:00.486 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.486 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.486 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.486 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87242 9.16 9.5af626b6 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:46:00.486 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.372676+0000", 2024-09-11T07:46:00.486 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 452.099924902, 2024-09-11T07:46:00.486 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.970557585, 2024-09-11T07:46:00.486 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:00.487 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:46:00.487 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:00.487 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:00.487 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:00.487 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87242 2024-09-11T07:46:00.487 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.487 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:00.487 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.487 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:00.487 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372676+0000", 2024-09-11T07:46:00.487 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.487 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.487 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.487 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:00.488 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372676+0000", 2024-09-11T07:46:00.488 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.488 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.488 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.488 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:00.488 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372675+0000", 2024-09-11T07:46:00.488 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:46:00.488 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.488 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.488 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:00.488 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372801+0000", 2024-09-11T07:46:00.488 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000126692 2024-09-11T07:46:00.488 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.488 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.489 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:00.489 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372815+0000", 2024-09-11T07:46:00.489 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4235e-05 2024-09-11T07:46:00.489 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.489 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.489 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:00.489 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372829+0000", 2024-09-11T07:46:00.489 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3794999999999999e-05 2024-09-11T07:46:00.489 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.489 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.489 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.489 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372894+0000", 2024-09-11T07:46:00.489 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5038999999999999e-05 2024-09-11T07:46:00.489 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.490 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.490 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:00.490 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372902+0000", 2024-09-11T07:46:00.490 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5909999999999998e-06 2024-09-11T07:46:00.490 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.490 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.490 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.490 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.343226+0000", 2024-09-11T07:46:00.490 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.97032444799999995 2024-09-11T07:46:00.490 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.490 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.490 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:00.490 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.343234+0000", 2024-09-11T07:46:00.490 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4150000000000002e-06 2024-09-11T07:46:00.491 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.491 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:00.491 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.491 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.491 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.491 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87253 9.10 9.1d1ad730 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:46:00.491 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.378537+0000", 2024-09-11T07:46:00.491 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 452.09406396200001, 2024-09-11T07:46:00.491 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.96243146199999996, 2024-09-11T07:46:00.491 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:00.491 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:46:00.491 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:00.491 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:00.491 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:00.491 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87253 2024-09-11T07:46:00.492 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.492 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:00.492 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.492 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:00.492 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378537+0000", 2024-09-11T07:46:00.492 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.492 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.492 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.492 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:00.492 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378537+0000", 2024-09-11T07:46:00.492 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.492 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.492 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.492 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:00.492 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378534+0000", 2024-09-11T07:46:00.493 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999971 2024-09-11T07:46:00.493 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.493 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.493 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:00.493 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378616+0000", 2024-09-11T07:46:00.493 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1242999999999994e-05 2024-09-11T07:46:00.493 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.493 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.493 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:00.493 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378630+0000", 2024-09-11T07:46:00.493 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4463e-05 2024-09-11T07:46:00.493 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.493 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.494 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:00.494 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378637+0000", 2024-09-11T07:46:00.494 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5300000000000002e-06 2024-09-11T07:46:00.494 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.494 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.494 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.494 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378682+0000", 2024-09-11T07:46:00.494 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5768e-05 2024-09-11T07:46:00.494 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.494 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.494 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:00.494 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378688+0000", 2024-09-11T07:46:00.494 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.259e-06 2024-09-11T07:46:00.494 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.494 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.495 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.495 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.340958+0000", 2024-09-11T07:46:00.495 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.96227069300000001 2024-09-11T07:46:00.495 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.495 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.495 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:00.495 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.340969+0000", 2024-09-11T07:46:00.495 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0175e-05 2024-09-11T07:46:00.495 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.495 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:00.495 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.495 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.495 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.495 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87254 9.16 9.cde03e76 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:46:00.495 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.379589+0000", 2024-09-11T07:46:00.496 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 452.09301242499998, 2024-09-11T07:46:00.496 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.96368716799999998, 2024-09-11T07:46:00.496 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:00.496 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:46:00.496 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:00.496 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:00.496 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:00.496 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87254 2024-09-11T07:46:00.496 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.496 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:00.496 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.496 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:00.496 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.379589+0000", 2024-09-11T07:46:00.497 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.497 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.497 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.497 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:00.497 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.379589+0000", 2024-09-11T07:46:00.497 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.497 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.497 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.497 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:00.497 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.379587+0000", 2024-09-11T07:46:00.497 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:46:00.497 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.497 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.497 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:00.498 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.379677+0000", 2024-09-11T07:46:00.498 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9827000000000006e-05 2024-09-11T07:46:00.498 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.498 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.498 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:00.498 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.379690+0000", 2024-09-11T07:46:00.498 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3224000000000001e-05 2024-09-11T07:46:00.498 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.498 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.498 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:00.498 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.379696+0000", 2024-09-11T07:46:00.498 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.288e-06 2024-09-11T07:46:00.498 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.498 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.499 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.499 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.379728+0000", 2024-09-11T07:46:00.499 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2771000000000002e-05 2024-09-11T07:46:00.499 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.499 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.499 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:00.499 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.379734+0000", 2024-09-11T07:46:00.499 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2000000000000002e-06 2024-09-11T07:46:00.499 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.499 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.499 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.499 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.343266+0000", 2024-09-11T07:46:00.499 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.96353272199999995 2024-09-11T07:46:00.499 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.499 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.500 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:00.500 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.343276+0000", 2024-09-11T07:46:00.500 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4029999999999996e-06 2024-09-11T07:46:00.500 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.500 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:00.500 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.500 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.500 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.500 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211568 9.16 9:6b0caae0:::benchmark_data_smithi022_133943_object84977:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e306)", 2024-09-11T07:46:00.500 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.491978+0000", 2024-09-11T07:46:00.500 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 324.98062314399999, 2024-09-11T07:46:00.500 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0266594309999999, 2024-09-11T07:46:00.500 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:00.500 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:00.501 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:00.501 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:00.501 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:00.501 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211568 2024-09-11T07:46:00.501 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.501 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:00.501 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.501 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:00.501 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.491978+0000", 2024-09-11T07:46:00.501 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.501 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.501 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.501 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:00.501 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.491978+0000", 2024-09-11T07:46:00.502 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.502 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.502 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.502 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:00.502 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.491976+0000", 2024-09-11T07:46:00.502 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:46:00.502 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.502 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.502 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:00.502 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.491984+0000", 2024-09-11T07:46:00.502 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9879999999999993e-06 2024-09-11T07:46:00.502 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.502 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.502 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:00.503 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.493497+0000", 2024-09-11T07:46:00.503 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001512572 2024-09-11T07:46:00.503 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.503 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.503 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:00.503 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.493505+0000", 2024-09-11T07:46:00.503 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.8169999999999999e-06 2024-09-11T07:46:00.503 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.503 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.503 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.503 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.494096+0000", 2024-09-11T07:46:00.503 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00059062700000000004 2024-09-11T07:46:00.503 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.503 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.503 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:00.504 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.494105+0000", 2024-09-11T07:46:00.504 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8050000000000001e-06 2024-09-11T07:46:00.504 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.504 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.504 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.504 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.513691+0000", 2024-09-11T07:46:00.504 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.019586165 2024-09-11T07:46:00.504 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.504 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.505 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:00.505 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.513793+0000", 2024-09-11T07:46:00.505 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000101745 2024-09-11T07:46:00.505 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.505 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.506 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:46:00.506 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.514096+0000", 2024-09-11T07:46:00.506 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00030249600000000003 2024-09-11T07:46:00.506 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.506 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.506 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:00.506 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518426+0000", 2024-09-11T07:46:00.506 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0043307750000000002 2024-09-11T07:46:00.506 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.506 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.506 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.506 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518448+0000", 2024-09-11T07:46:00.506 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1321000000000001e-05 2024-09-11T07:46:00.506 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.507 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.507 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.507 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518570+0000", 2024-09-11T07:46:00.507 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012255 2024-09-11T07:46:00.507 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.507 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.507 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:00.507 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518602+0000", 2024-09-11T07:46:00.507 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1735e-05 2024-09-11T07:46:00.507 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.507 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.507 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:00.507 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518637+0000", 2024-09-11T07:46:00.507 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5238999999999999e-05 2024-09-11T07:46:00.508 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.508 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:00.508 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.508 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.508 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.508 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211583 9.16 9:6baf4737:::benchmark_data_smithi022_133943_object84992:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:46:00.508 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.510888+0000", 2024-09-11T07:46:00.508 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 324.96171314600002, 2024-09-11T07:46:00.508 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0079589120000001, 2024-09-11T07:46:00.508 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:00.508 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:00.508 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:00.508 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:00.509 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:00.509 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211583 2024-09-11T07:46:00.509 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.509 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:00.509 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.509 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:00.509 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.510888+0000", 2024-09-11T07:46:00.509 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.509 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.509 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.509 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:00.509 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.510888+0000", 2024-09-11T07:46:00.509 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.510 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.510 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.510 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:00.510 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.510885+0000", 2024-09-11T07:46:00.510 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:46:00.510 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.510 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.510 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:00.510 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.510895+0000", 2024-09-11T07:46:00.510 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2399999999999996e-06 2024-09-11T07:46:00.510 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.510 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.510 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:00.510 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.510914+0000", 2024-09-11T07:46:00.510 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9018999999999999e-05 2024-09-11T07:46:00.511 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.511 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.511 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:00.511 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.510921+0000", 2024-09-11T07:46:00.511 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5530000000000004e-06 2024-09-11T07:46:00.511 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.511 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.511 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.511 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.510974+0000", 2024-09-11T07:46:00.511 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2683999999999999e-05 2024-09-11T07:46:00.511 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.511 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.511 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:00.512 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.510983+0000", 2024-09-11T07:46:00.512 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7290000000000006e-06 2024-09-11T07:46:00.512 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.512 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.512 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.512 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.515778+0000", 2024-09-11T07:46:00.512 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.004795729 2024-09-11T07:46:00.512 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.512 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.512 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:00.512 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.515882+0000", 2024-09-11T07:46:00.512 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010367899999999999 2024-09-11T07:46:00.512 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.512 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.513 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:46:00.513 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.516114+0000", 2024-09-11T07:46:00.513 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000232394 2024-09-11T07:46:00.513 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.513 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.513 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:00.513 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518531+0000", 2024-09-11T07:46:00.513 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0024165269999999999 2024-09-11T07:46:00.513 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.513 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.513 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.513 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518699+0000", 2024-09-11T07:46:00.513 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016786500000000001 2024-09-11T07:46:00.514 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.514 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.514 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.514 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518758+0000", 2024-09-11T07:46:00.514 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9435000000000003e-05 2024-09-11T07:46:00.514 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.514 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.514 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:00.514 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518800+0000", 2024-09-11T07:46:00.514 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1912000000000001e-05 2024-09-11T07:46:00.514 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.514 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.514 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:00.514 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518847+0000", 2024-09-11T07:46:00.514 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6686e-05 2024-09-11T07:46:00.515 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.515 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:00.515 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.515 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.515 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.515 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211591 9.16 9:6c004eb6:::benchmark_data_smithi022_133943_object85000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:46:00.515 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.519215+0000", 2024-09-11T07:46:00.515 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 324.95338653599998, 2024-09-11T07:46:00.515 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.001420948, 2024-09-11T07:46:00.515 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:00.515 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:00.515 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:00.515 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:00.515 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:00.515 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211591 2024-09-11T07:46:00.516 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.516 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:00.516 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.516 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:00.516 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.519215+0000", 2024-09-11T07:46:00.516 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.516 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.516 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.516 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:00.516 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.519215+0000", 2024-09-11T07:46:00.516 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.516 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.516 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.516 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:00.516 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.519212+0000", 2024-09-11T07:46:00.517 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999971 2024-09-11T07:46:00.517 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.517 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.517 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:00.517 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.519220+0000", 2024-09-11T07:46:00.517 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1920000000000005e-06 2024-09-11T07:46:00.517 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.517 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.517 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:00.517 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.519235+0000", 2024-09-11T07:46:00.517 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.483e-05 2024-09-11T07:46:00.517 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.517 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.517 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:00.518 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.519241+0000", 2024-09-11T07:46:00.518 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9150000000000001e-06 2024-09-11T07:46:00.518 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.518 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.518 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.518 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.519281+0000", 2024-09-11T07:46:00.518 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0568999999999997e-05 2024-09-11T07:46:00.518 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.518 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.518 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:00.518 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.519289+0000", 2024-09-11T07:46:00.518 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3699999999999997e-06 2024-09-11T07:46:00.518 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.518 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.519 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.519 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.517249+0000", 2024-09-11T07:46:00.519 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.99796017800000003 2024-09-11T07:46:00.519 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.519 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.519 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:00.519 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.517317+0000", 2024-09-11T07:46:00.519 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.8004999999999997e-05 2024-09-11T07:46:00.519 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.519 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.519 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:46:00.519 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.517499+0000", 2024-09-11T07:46:00.519 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018260999999999999 2024-09-11T07:46:00.520 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.520 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.520 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.520 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518719+0000", 2024-09-11T07:46:00.520 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001219588 2024-09-11T07:46:00.520 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.520 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.520 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:00.520 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518857+0000", 2024-09-11T07:46:00.520 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013752 2024-09-11T07:46:00.520 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.520 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.520 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.520 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.520551+0000", 2024-09-11T07:46:00.521 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001694688 2024-09-11T07:46:00.521 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.521 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.521 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:00.521 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.520599+0000", 2024-09-11T07:46:00.521 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7874999999999998e-05 2024-09-11T07:46:00.521 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.521 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.521 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:00.521 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.520635+0000", 2024-09-11T07:46:00.521 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6313000000000003e-05 2024-09-11T07:46:00.521 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.521 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:00.521 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.522 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.522 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.522 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211637 9.16 9:6a9b2a50:::benchmark_data_smithi022_133943_object85046:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:46:00.522 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.555357+0000", 2024-09-11T07:46:00.522 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 324.91724355700001, 2024-09-11T07:46:00.522 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.96961262199999998, 2024-09-11T07:46:00.522 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:00.522 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:00.522 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:00.522 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:00.522 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:00.522 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211637 2024-09-11T07:46:00.522 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.522 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:00.523 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.523 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:00.523 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.555357+0000", 2024-09-11T07:46:00.523 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.523 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.523 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.523 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:00.523 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.555357+0000", 2024-09-11T07:46:00.523 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.523 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.523 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.523 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:00.523 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.555355+0000", 2024-09-11T07:46:00.523 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999971 2024-09-11T07:46:00.523 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.524 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.524 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:00.524 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.555363+0000", 2024-09-11T07:46:00.524 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0679999999999997e-06 2024-09-11T07:46:00.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.524 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.524 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:00.524 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.555375+0000", 2024-09-11T07:46:00.524 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1653e-05 2024-09-11T07:46:00.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.524 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.524 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:00.524 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.555380+0000", 2024-09-11T07:46:00.524 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7679999999999997e-06 2024-09-11T07:46:00.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.525 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.525 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.525 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.555426+0000", 2024-09-11T07:46:00.525 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5615999999999998e-05 2024-09-11T07:46:00.525 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.525 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.525 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:00.525 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.555434+0000", 2024-09-11T07:46:00.525 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6159999999999996e-06 2024-09-11T07:46:00.525 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.525 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.525 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.525 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518872+0000", 2024-09-11T07:46:00.526 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.96343850799999997 2024-09-11T07:46:00.526 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.526 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.526 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:00.526 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518941+0000", 2024-09-11T07:46:00.526 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9207000000000003e-05 2024-09-11T07:46:00.526 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.526 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.526 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:46:00.526 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.519150+0000", 2024-09-11T07:46:00.526 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00020839900000000001 2024-09-11T07:46:00.526 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.526 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.526 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.526 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.524769+0000", 2024-09-11T07:46:00.526 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0056189810000000003 2024-09-11T07:46:00.527 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.527 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.527 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:00.527 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.524776+0000", 2024-09-11T07:46:00.527 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7670000000000003e-06 2024-09-11T07:46:00.527 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.527 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.527 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.527 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.524884+0000", 2024-09-11T07:46:00.527 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000107384 2024-09-11T07:46:00.527 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.527 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.527 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:00.527 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.524930+0000", 2024-09-11T07:46:00.528 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6008999999999999e-05 2024-09-11T07:46:00.528 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.528 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.528 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:00.528 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.524970+0000", 2024-09-11T07:46:00.528 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0268999999999997e-05 2024-09-11T07:46:00.528 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.528 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:00.528 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.528 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.528 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.528 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211686 9.16 9:69c3c680:::benchmark_data_smithi022_133943_object85095:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:46:00.528 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.594485+0000", 2024-09-11T07:46:00.529 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 324.87811640299998, 2024-09-11T07:46:00.529 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.93446916000000002, 2024-09-11T07:46:00.529 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:00.529 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:00.529 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:00.529 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:00.529 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:00.529 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211686 2024-09-11T07:46:00.529 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.529 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:00.529 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.529 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:00.529 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.594485+0000", 2024-09-11T07:46:00.530 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.530 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.530 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.530 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:00.530 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.594485+0000", 2024-09-11T07:46:00.530 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.530 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.530 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.530 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:00.530 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.594483+0000", 2024-09-11T07:46:00.530 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:46:00.530 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.530 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.530 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:00.531 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.594489+0000", 2024-09-11T07:46:00.531 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7710000000000001e-06 2024-09-11T07:46:00.531 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.531 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.531 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:00.531 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.594502+0000", 2024-09-11T07:46:00.531 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2847000000000001e-05 2024-09-11T07:46:00.531 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.531 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.531 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:00.531 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.594508+0000", 2024-09-11T07:46:00.531 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7350000000000001e-06 2024-09-11T07:46:00.531 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.532 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.532 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.532 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.594572+0000", 2024-09-11T07:46:00.532 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4414999999999999e-05 2024-09-11T07:46:00.532 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.532 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.532 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:00.532 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.594579+0000", 2024-09-11T07:46:00.532 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.041e-06 2024-09-11T07:46:00.532 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.533 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.533 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.533 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.520664+0000", 2024-09-11T07:46:00.533 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.92608433199999995 2024-09-11T07:46:00.533 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.533 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.533 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:00.533 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.520762+0000", 2024-09-11T07:46:00.533 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7971000000000001e-05 2024-09-11T07:46:00.533 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.533 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.533 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:46:00.533 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.521031+0000", 2024-09-11T07:46:00.534 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00026898699999999999 2024-09-11T07:46:00.534 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.534 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.534 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.534 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.524805+0000", 2024-09-11T07:46:00.534 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0037746870000000001 2024-09-11T07:46:00.534 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.534 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.534 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:00.534 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.524981+0000", 2024-09-11T07:46:00.534 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017573400000000001 2024-09-11T07:46:00.534 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.534 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.534 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.535 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.528877+0000", 2024-09-11T07:46:00.535 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0038956770000000002 2024-09-11T07:46:00.535 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.535 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.535 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:00.535 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.528918+0000", 2024-09-11T07:46:00.535 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1600999999999997e-05 2024-09-11T07:46:00.535 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.535 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.535 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:00.535 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.528954+0000", 2024-09-11T07:46:00.535 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5330000000000002e-05 2024-09-11T07:46:00.535 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.536 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:00.536 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.536 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.536 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.536 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211710 9.16 9:6b0c937f:::benchmark_data_smithi022_133943_object85119:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:46:00.536 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.612884+0000", 2024-09-11T07:46:00.536 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 324.85971672599999, 2024-09-11T07:46:00.536 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.92170397500000001, 2024-09-11T07:46:00.536 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:00.536 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:00.536 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:00.536 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:00.536 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:00.536 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211710 2024-09-11T07:46:00.537 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.537 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:00.537 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.537 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:00.537 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.612884+0000", 2024-09-11T07:46:00.537 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.537 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.537 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.537 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:00.537 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.612884+0000", 2024-09-11T07:46:00.537 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.537 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.537 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.538 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:00.538 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.612882+0000", 2024-09-11T07:46:00.538 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:46:00.538 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.538 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.538 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:00.538 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.612892+0000", 2024-09-11T07:46:00.538 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4010000000000003e-06 2024-09-11T07:46:00.538 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.538 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.538 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:00.538 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.612905+0000", 2024-09-11T07:46:00.538 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3653e-05 2024-09-11T07:46:00.538 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.538 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.539 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:00.539 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.612912+0000", 2024-09-11T07:46:00.539 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3380000000000001e-06 2024-09-11T07:46:00.539 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.539 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.539 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.539 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.613011+0000", 2024-09-11T07:46:00.539 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.9469000000000006e-05 2024-09-11T07:46:00.539 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.539 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.539 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:00.539 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.613018+0000", 2024-09-11T07:46:00.539 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0679999999999999e-06 2024-09-11T07:46:00.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.540 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.540 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.522738+0000", 2024-09-11T07:46:00.540 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.90972005199999995 2024-09-11T07:46:00.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.540 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:00.540 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.522865+0000", 2024-09-11T07:46:00.540 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000127093 2024-09-11T07:46:00.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.540 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:46:00.541 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.523141+0000", 2024-09-11T07:46:00.541 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00027508200000000001 2024-09-11T07:46:00.541 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.541 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.541 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:00.541 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.528797+0000", 2024-09-11T07:46:00.541 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0056566009999999998 2024-09-11T07:46:00.541 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.541 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.541 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.541 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.528839+0000", 2024-09-11T07:46:00.541 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1924e-05 2024-09-11T07:46:00.541 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.542 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.542 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.542 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.534494+0000", 2024-09-11T07:46:00.542 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0056551739999999998 2024-09-11T07:46:00.542 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.542 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.542 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:00.542 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.534544+0000", 2024-09-11T07:46:00.542 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9911999999999999e-05 2024-09-11T07:46:00.542 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.543 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.543 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:00.543 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.534588+0000", 2024-09-11T07:46:00.543 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4122000000000002e-05 2024-09-11T07:46:00.543 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.543 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:00.543 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.543 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.543 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.543 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211745 9.16 9:694b4fca:::benchmark_data_smithi022_133943_object85154:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:46:00.543 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.639786+0000", 2024-09-11T07:46:00.543 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 324.832814756, 2024-09-11T07:46:00.543 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.90253554000000002, 2024-09-11T07:46:00.543 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:00.543 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:00.544 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:00.544 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:00.544 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:00.544 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211745 2024-09-11T07:46:00.544 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.544 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:00.544 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.544 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:00.544 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.639786+0000", 2024-09-11T07:46:00.544 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.544 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.544 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.544 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:00.545 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.639786+0000", 2024-09-11T07:46:00.545 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.545 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.545 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:00.545 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.639784+0000", 2024-09-11T07:46:00.545 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:46:00.545 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.545 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:00.545 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.639791+0000", 2024-09-11T07:46:00.545 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.2980000000000003e-06 2024-09-11T07:46:00.545 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.546 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.546 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:00.546 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.639804+0000", 2024-09-11T07:46:00.546 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.207e-05 2024-09-11T07:46:00.546 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.546 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.546 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:00.546 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.639809+0000", 2024-09-11T07:46:00.546 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7429999999999999e-06 2024-09-11T07:46:00.546 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.546 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.546 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.546 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.639861+0000", 2024-09-11T07:46:00.546 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1946e-05 2024-09-11T07:46:00.547 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.547 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.547 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:00.547 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.639871+0000", 2024-09-11T07:46:00.547 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3379999999999994e-06 2024-09-11T07:46:00.547 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.547 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.547 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.547 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.525002+0000", 2024-09-11T07:46:00.547 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.88513150799999996 2024-09-11T07:46:00.547 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.547 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.547 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:00.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.525118+0000", 2024-09-11T07:46:00.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000116166 2024-09-11T07:46:00.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.548 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.548 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:46:00.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.525429+0000", 2024-09-11T07:46:00.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00031055399999999999 2024-09-11T07:46:00.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.548 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.548 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:00.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.528849+0000", 2024-09-11T07:46:00.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003419872 2024-09-11T07:46:00.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.549 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.549 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.529004+0000", 2024-09-11T07:46:00.549 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000155663 2024-09-11T07:46:00.549 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.549 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.549 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542237+0000", 2024-09-11T07:46:00.549 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.013232961 2024-09-11T07:46:00.549 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.549 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:00.549 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542285+0000", 2024-09-11T07:46:00.549 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7598000000000001e-05 2024-09-11T07:46:00.550 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.550 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.550 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:00.550 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542322+0000", 2024-09-11T07:46:00.550 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6925000000000003e-05 2024-09-11T07:46:00.550 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.550 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:00.550 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.550 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.550 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.550 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211775 9.16 9:68b77f6d:::benchmark_data_smithi022_133943_object85184:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:46:00.550 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.676391+0000", 2024-09-11T07:46:00.550 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 324.79620960099999, 2024-09-11T07:46:00.550 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.86609124999999998, 2024-09-11T07:46:00.551 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:00.551 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:00.551 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:00.551 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:00.551 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:00.551 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211775 2024-09-11T07:46:00.551 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.551 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:00.551 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.551 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:00.551 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.676391+0000", 2024-09-11T07:46:00.551 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.551 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.551 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.552 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:00.552 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.676391+0000", 2024-09-11T07:46:00.552 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.552 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.552 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.552 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:00.552 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.676389+0000", 2024-09-11T07:46:00.552 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:46:00.552 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.552 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.552 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:00.552 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.676399+0000", 2024-09-11T07:46:00.552 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.6020000000000006e-06 2024-09-11T07:46:00.552 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.553 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.553 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:00.553 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.676414+0000", 2024-09-11T07:46:00.553 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5577999999999999e-05 2024-09-11T07:46:00.553 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.553 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.553 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:00.553 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.676422+0000", 2024-09-11T07:46:00.553 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9310000000000008e-06 2024-09-11T07:46:00.553 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.553 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.553 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.553 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.676471+0000", 2024-09-11T07:46:00.554 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8769e-05 2024-09-11T07:46:00.554 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.554 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.554 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:00.554 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.676481+0000", 2024-09-11T07:46:00.554 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8490000000000001e-06 2024-09-11T07:46:00.554 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.554 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.554 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.554 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.527191+0000", 2024-09-11T07:46:00.554 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.85070976600000003 2024-09-11T07:46:00.554 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.554 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.554 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:00.555 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.527309+0000", 2024-09-11T07:46:00.555 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000118206 2024-09-11T07:46:00.555 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.555 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.555 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:46:00.555 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.527534+0000", 2024-09-11T07:46:00.555 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00022490499999999999 2024-09-11T07:46:00.555 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.555 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.555 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.555 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.529042+0000", 2024-09-11T07:46:00.555 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0015080779999999999 2024-09-11T07:46:00.555 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.556 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.556 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:00.556 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.529048+0000", 2024-09-11T07:46:00.556 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2219999999999999e-06 2024-09-11T07:46:00.556 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.556 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.556 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.556 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542405+0000", 2024-09-11T07:46:00.556 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.013356632 2024-09-11T07:46:00.556 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.556 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.556 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:00.556 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542450+0000", 2024-09-11T07:46:00.556 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5164999999999999e-05 2024-09-11T07:46:00.556 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.557 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.557 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:00.557 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542483+0000", 2024-09-11T07:46:00.557 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2945999999999999e-05 2024-09-11T07:46:00.557 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.557 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:00.557 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.557 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.557 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.557 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211789 9.16 9:6d58905b:::benchmark_data_smithi022_133943_object85198:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:46:00.557 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.692735+0000", 2024-09-11T07:46:00.557 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 324.77986645599998, 2024-09-11T07:46:00.557 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.84991873100000004, 2024-09-11T07:46:00.557 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:00.558 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:00.558 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:00.558 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:00.558 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:00.558 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211789 2024-09-11T07:46:00.558 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.558 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:00.558 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.558 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:00.558 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.692735+0000", 2024-09-11T07:46:00.558 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.558 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.558 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.558 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:00.559 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.692735+0000", 2024-09-11T07:46:00.559 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.559 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.559 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.559 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:00.559 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.692732+0000", 2024-09-11T07:46:00.559 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:46:00.559 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.559 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.559 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:00.559 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.692740+0000", 2024-09-11T07:46:00.559 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9589999999999995e-06 2024-09-11T07:46:00.559 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.559 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.560 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:00.560 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.692755+0000", 2024-09-11T07:46:00.560 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4649e-05 2024-09-11T07:46:00.560 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.560 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.560 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:00.560 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.692761+0000", 2024-09-11T07:46:00.560 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5010000000000001e-06 2024-09-11T07:46:00.560 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.560 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.560 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.560 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.692809+0000", 2024-09-11T07:46:00.560 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7410999999999997e-05 2024-09-11T07:46:00.560 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.560 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:00.562 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:00.562 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:00.563 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:00.563 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.692817+0000", 2024-09-11T07:46:00.563 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4440000000000008e-06 2024-09-11T07:46:00.563 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.563 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.563 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.563 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.529069+0000", 2024-09-11T07:46:00.563 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.83625205700000005 2024-09-11T07:46:00.563 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.563 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.563 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:00.563 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.529140+0000", 2024-09-11T07:46:00.563 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0500000000000006e-05 2024-09-11T07:46:00.563 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.564 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.564 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:46:00.564 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.529333+0000", 2024-09-11T07:46:00.564 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000193396 2024-09-11T07:46:00.564 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.564 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.564 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:00.564 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.534390+0000", 2024-09-11T07:46:00.564 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0050565050000000002 2024-09-11T07:46:00.564 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.564 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.564 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.564 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.534425+0000", 2024-09-11T07:46:00.565 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5138999999999997e-05 2024-09-11T07:46:00.565 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.565 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.565 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.565 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542573+0000", 2024-09-11T07:46:00.565 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0081485240000000007 2024-09-11T07:46:00.565 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.565 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.565 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:00.565 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542622+0000", 2024-09-11T07:46:00.565 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8279000000000003e-05 2024-09-11T07:46:00.565 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.565 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.566 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:00.566 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542653+0000", 2024-09-11T07:46:00.566 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1745000000000002e-05 2024-09-11T07:46:00.566 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.566 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:00.566 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.566 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.566 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.566 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211797 9.16 9:6ee58f2c:::benchmark_data_smithi022_133943_object85206:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:46:00.566 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.698284+0000", 2024-09-11T07:46:00.566 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 324.774317225, 2024-09-11T07:46:00.566 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.84449932100000002, 2024-09-11T07:46:00.566 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:00.567 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:00.567 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:00.567 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:00.567 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:00.567 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211797 2024-09-11T07:46:00.567 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.567 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:00.567 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.567 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:00.567 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.698284+0000", 2024-09-11T07:46:00.567 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.567 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.567 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.567 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:00.568 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.698284+0000", 2024-09-11T07:46:00.568 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.568 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.568 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.568 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:00.568 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.698282+0000", 2024-09-11T07:46:00.568 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:46:00.568 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.568 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.568 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:00.568 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.698288+0000", 2024-09-11T07:46:00.568 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6819999999999997e-06 2024-09-11T07:46:00.568 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.569 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.569 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:00.569 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.698299+0000", 2024-09-11T07:46:00.569 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0668e-05 2024-09-11T07:46:00.569 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.569 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.569 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:00.569 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.698305+0000", 2024-09-11T07:46:00.569 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6969999999999998e-06 2024-09-11T07:46:00.569 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.569 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.569 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.569 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.698345+0000", 2024-09-11T07:46:00.569 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.036e-05 2024-09-11T07:46:00.570 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.570 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.570 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:00.570 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.698352+0000", 2024-09-11T07:46:00.570 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.8680000000000001e-06 2024-09-11T07:46:00.570 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.570 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.570 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.570 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.530667+0000", 2024-09-11T07:46:00.570 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.83231449499999999 2024-09-11T07:46:00.570 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.570 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.570 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:00.570 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.530772+0000", 2024-09-11T07:46:00.570 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010552699999999999 2024-09-11T07:46:00.571 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.571 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.571 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:46:00.571 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.531034+0000", 2024-09-11T07:46:00.571 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000262439 2024-09-11T07:46:00.571 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.571 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.571 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:00.571 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.534430+0000", 2024-09-11T07:46:00.571 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0033959440000000001 2024-09-11T07:46:00.571 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.571 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.571 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.571 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.534458+0000", 2024-09-11T07:46:00.572 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.7622000000000001e-05 2024-09-11T07:46:00.572 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.572 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.572 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.572 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542705+0000", 2024-09-11T07:46:00.572 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0082472069999999995 2024-09-11T07:46:00.572 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.572 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.572 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:00.572 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542756+0000", 2024-09-11T07:46:00.572 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0600999999999999e-05 2024-09-11T07:46:00.572 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.572 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.573 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:00.573 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542783+0000", 2024-09-11T07:46:00.573 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.7300999999999998e-05 2024-09-11T07:46:00.573 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.573 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:00.573 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.573 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.573 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.573 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211806 9.16 9:6a26adda:::benchmark_data_smithi022_133943_object85215:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:46:00.573 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.708768+0000", 2024-09-11T07:46:00.573 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 324.76383323900001, 2024-09-11T07:46:00.573 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.83413179299999995, 2024-09-11T07:46:00.573 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:00.573 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:00.573 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:00.574 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:00.574 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:00.574 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211806 2024-09-11T07:46:00.574 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.574 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:00.574 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.574 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:00.574 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.708768+0000", 2024-09-11T07:46:00.574 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.574 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.574 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.574 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:00.574 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.708768+0000", 2024-09-11T07:46:00.574 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.574 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.575 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.575 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:00.575 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.708765+0000", 2024-09-11T07:46:00.575 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999971 2024-09-11T07:46:00.575 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.575 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.575 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:00.575 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.708774+0000", 2024-09-11T07:46:00.575 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2869999999999993e-06 2024-09-11T07:46:00.575 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.575 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.575 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:00.575 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.708789+0000", 2024-09-11T07:46:00.576 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4391e-05 2024-09-11T07:46:00.576 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.576 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.576 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:00.576 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.708796+0000", 2024-09-11T07:46:00.576 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0530000000000001e-06 2024-09-11T07:46:00.576 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.576 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.576 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.576 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.708860+0000", 2024-09-11T07:46:00.576 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4565000000000003e-05 2024-09-11T07:46:00.576 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.576 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.577 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:00.577 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.708867+0000", 2024-09-11T07:46:00.577 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1739999999999997e-06 2024-09-11T07:46:00.577 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.577 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.577 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.577 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.532429+0000", 2024-09-11T07:46:00.577 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.82356275800000001 2024-09-11T07:46:00.577 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.577 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.577 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:00.577 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.532517+0000", 2024-09-11T07:46:00.577 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7460999999999995e-05 2024-09-11T07:46:00.578 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.578 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.578 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:46:00.578 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.532755+0000", 2024-09-11T07:46:00.578 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000237941 2024-09-11T07:46:00.578 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.578 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.578 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:00.578 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.536455+0000", 2024-09-11T07:46:00.578 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0036998270000000002 2024-09-11T07:46:00.578 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.578 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.578 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.578 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542204+0000", 2024-09-11T07:46:00.579 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0057493090000000002 2024-09-11T07:46:00.579 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.579 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.579 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.579 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542818+0000", 2024-09-11T07:46:00.579 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00061378599999999995 2024-09-11T07:46:00.579 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.579 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.579 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:00.579 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542856+0000", 2024-09-11T07:46:00.579 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8791999999999997e-05 2024-09-11T07:46:00.579 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.580 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.580 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:00.580 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542900+0000", 2024-09-11T07:46:00.580 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3170999999999997e-05 2024-09-11T07:46:00.580 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.580 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:00.580 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.580 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.580 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.580 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211852 9.16 9:6ab70b3a:::benchmark_data_smithi022_133943_object85261:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:46:00.580 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.765778+0000", 2024-09-11T07:46:00.580 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 324.70682309599999, 2024-09-11T07:46:00.580 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.78150204000000001, 2024-09-11T07:46:00.581 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:00.581 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:00.581 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:00.581 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:00.581 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:00.581 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211852 2024-09-11T07:46:00.581 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.581 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:00.581 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.581 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:00.581 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.765778+0000", 2024-09-11T07:46:00.581 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.581 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.582 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.582 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:00.582 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.765778+0000", 2024-09-11T07:46:00.582 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.582 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.582 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.582 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:00.582 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.765775+0000", 2024-09-11T07:46:00.582 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999971 2024-09-11T07:46:00.582 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.582 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.582 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:00.582 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.765784+0000", 2024-09-11T07:46:00.582 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3919999999999992e-06 2024-09-11T07:46:00.583 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.583 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.583 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:00.583 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.765801+0000", 2024-09-11T07:46:00.583 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6969e-05 2024-09-11T07:46:00.583 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.583 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.583 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:00.583 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.765809+0000", 2024-09-11T07:46:00.583 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4070000000000004e-06 2024-09-11T07:46:00.583 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.583 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.583 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.583 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.765876+0000", 2024-09-11T07:46:00.584 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7101999999999996e-05 2024-09-11T07:46:00.584 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.584 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.584 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:00.584 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.765884+0000", 2024-09-11T07:46:00.584 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4880000000000007e-06 2024-09-11T07:46:00.584 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.584 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.584 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.584 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.534626+0000", 2024-09-11T07:46:00.584 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.76874199799999998 2024-09-11T07:46:00.584 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.584 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.584 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:00.585 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.534739+0000", 2024-09-11T07:46:00.585 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000112904 2024-09-11T07:46:00.585 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.585 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.585 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:46:00.585 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.535029+0000", 2024-09-11T07:46:00.585 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00028969500000000002 2024-09-11T07:46:00.585 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.585 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.585 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:00.585 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.540522+0000", 2024-09-11T07:46:00.585 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.005493118 2024-09-11T07:46:00.585 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.586 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.586 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.586 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542355+0000", 2024-09-11T07:46:00.586 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0018325609999999999 2024-09-11T07:46:00.586 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.586 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.586 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.586 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.547211+0000", 2024-09-11T07:46:00.586 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0048561899999999998 2024-09-11T07:46:00.586 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.586 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.586 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:00.586 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.547256+0000", 2024-09-11T07:46:00.586 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4821999999999999e-05 2024-09-11T07:46:00.586 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.587 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.587 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:00.587 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.547280+0000", 2024-09-11T07:46:00.587 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.4431999999999999e-05 2024-09-11T07:46:00.587 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.587 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:00.587 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.587 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.587 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.587 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211923 9.16 9:68ffa8c1:::benchmark_data_smithi022_133943_object85332:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:46:00.587 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.884034+0000", 2024-09-11T07:46:00.587 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 324.58856739999999, 2024-09-11T07:46:00.587 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.66333196100000003, 2024-09-11T07:46:00.587 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:00.588 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:00.588 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:00.588 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:00.588 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:00.588 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211923 2024-09-11T07:46:00.588 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.588 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:00.588 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.588 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:00.588 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.884034+0000", 2024-09-11T07:46:00.588 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.588 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.588 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.588 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:00.589 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.884034+0000", 2024-09-11T07:46:00.589 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.589 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.589 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.589 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:00.589 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.884032+0000", 2024-09-11T07:46:00.589 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:46:00.589 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.589 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.589 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:00.589 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.884042+0000", 2024-09-11T07:46:00.589 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.9299999999999998e-06 2024-09-11T07:46:00.589 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.590 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.590 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:00.590 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.884051+0000", 2024-09-11T07:46:00.590 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0159999999999997e-06 2024-09-11T07:46:00.590 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.590 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.590 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:00.590 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.884055+0000", 2024-09-11T07:46:00.590 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.053e-06 2024-09-11T07:46:00.590 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.590 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.590 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.590 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.884086+0000", 2024-09-11T07:46:00.590 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.0858999999999997e-05 2024-09-11T07:46:00.591 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.591 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.591 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:00.591 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.884092+0000", 2024-09-11T07:46:00.591 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0379999999999996e-06 2024-09-11T07:46:00.591 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.591 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.591 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.591 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.536480+0000", 2024-09-11T07:46:00.591 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.65238803999999995 2024-09-11T07:46:00.591 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.591 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.591 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:00.591 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.536577+0000", 2024-09-11T07:46:00.592 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.6854999999999993e-05 2024-09-11T07:46:00.592 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.592 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.592 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:46:00.592 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.536851+0000", 2024-09-11T07:46:00.592 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00027332099999999998 2024-09-11T07:46:00.592 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.592 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.592 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:00.592 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542171+0000", 2024-09-11T07:46:00.592 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0053199620000000001 2024-09-11T07:46:00.592 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.592 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.593 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.593 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542539+0000", 2024-09-11T07:46:00.593 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00036858700000000003 2024-09-11T07:46:00.593 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.593 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.593 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.593 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.547314+0000", 2024-09-11T07:46:00.593 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0047751470000000004 2024-09-11T07:46:00.593 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.593 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.593 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:00.593 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.547342+0000", 2024-09-11T07:46:00.593 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.8113999999999999e-05 2024-09-11T07:46:00.594 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.594 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.594 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:00.594 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.547366+0000", 2024-09-11T07:46:00.594 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3238e-05 2024-09-11T07:46:00.594 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.594 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:00.594 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.594 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.594 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.594 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211949 9.16 9:6827f352:::benchmark_data_smithi022_133943_object85358:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:46:00.594 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.917915+0000", 2024-09-11T07:46:00.594 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 324.55468600699999, 2024-09-11T07:46:00.595 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.62951596099999996, 2024-09-11T07:46:00.595 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:00.595 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:00.595 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:00.595 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:00.595 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:00.595 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211949 2024-09-11T07:46:00.595 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.595 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:00.595 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.595 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:00.595 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.917915+0000", 2024-09-11T07:46:00.595 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.595 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.596 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.596 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:00.596 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.917915+0000", 2024-09-11T07:46:00.596 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.596 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.596 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.596 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:00.596 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.917914+0000", 2024-09-11T07:46:00.596 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:46:00.596 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.596 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.596 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:00.596 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.917918+0000", 2024-09-11T07:46:00.596 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2899999999999996e-06 2024-09-11T07:46:00.597 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.597 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.597 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:00.597 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.917925+0000", 2024-09-11T07:46:00.597 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3740000000000003e-06 2024-09-11T07:46:00.597 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.597 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.597 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:00.597 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.917929+0000", 2024-09-11T07:46:00.597 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4139999999999996e-06 2024-09-11T07:46:00.597 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.597 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.597 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.598 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.917960+0000", 2024-09-11T07:46:00.598 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.0925e-05 2024-09-11T07:46:00.598 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.598 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.598 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:00.598 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.917966+0000", 2024-09-11T07:46:00.598 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3990000000000001e-06 2024-09-11T07:46:00.598 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.598 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.598 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.598 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.538684+0000", 2024-09-11T07:46:00.598 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.62071715599999999 2024-09-11T07:46:00.598 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.599 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.599 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:00.599 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.538783+0000", 2024-09-11T07:46:00.599 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.9036000000000005e-05 2024-09-11T07:46:00.599 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.599 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.599 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:46:00.599 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.539075+0000", 2024-09-11T07:46:00.599 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00029267600000000001 2024-09-11T07:46:00.599 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.599 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.599 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:00.599 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542210+0000", 2024-09-11T07:46:00.599 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0031345840000000002 2024-09-11T07:46:00.599 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.600 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.600 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.600 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.545296+0000", 2024-09-11T07:46:00.600 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0030863309999999999 2024-09-11T07:46:00.600 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.600 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.600 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.600 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.547390+0000", 2024-09-11T07:46:00.600 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0020940630000000002 2024-09-11T07:46:00.600 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.600 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.600 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:00.600 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.547413+0000", 2024-09-11T07:46:00.601 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.2399999999999999e-05 2024-09-11T07:46:00.601 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.601 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.601 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:00.601 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.547431+0000", 2024-09-11T07:46:00.601 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8263000000000001e-05 2024-09-11T07:46:00.601 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.601 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:00.601 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.601 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.601 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.601 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211951 9.16 9:6974843e:::benchmark_data_smithi022_133943_object85360:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:46:00.601 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.921562+0000", 2024-09-11T07:46:00.602 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 324.551039371, 2024-09-11T07:46:00.602 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.62686241200000004, 2024-09-11T07:46:00.602 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:00.602 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:00.602 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:00.602 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:00.602 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:00.602 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211951 2024-09-11T07:46:00.602 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.602 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:00.602 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.602 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:00.602 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.921562+0000", 2024-09-11T07:46:00.603 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.603 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.603 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.603 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:00.603 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.921562+0000", 2024-09-11T07:46:00.603 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:00.603 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.603 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.603 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:00.603 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.921560+0000", 2024-09-11T07:46:00.603 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:46:00.603 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.603 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.603 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:00.603 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.921565+0000", 2024-09-11T07:46:00.604 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0690000000000001e-06 2024-09-11T07:46:00.604 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.604 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.604 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:00.604 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.921574+0000", 2024-09-11T07:46:00.604 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2810000000000008e-06 2024-09-11T07:46:00.604 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.604 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.604 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:00.604 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.921578+0000", 2024-09-11T07:46:00.604 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3189999999999996e-06 2024-09-11T07:46:00.604 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.604 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.604 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.605 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.921606+0000", 2024-09-11T07:46:00.605 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.7716e-05 2024-09-11T07:46:00.605 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.605 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.605 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:00.605 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.921610+0000", 2024-09-11T07:46:00.605 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.408e-06 2024-09-11T07:46:00.605 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.605 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.605 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:00.605 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.540550+0000", 2024-09-11T07:46:00.605 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.61893973300000005 2024-09-11T07:46:00.605 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.606 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.606 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:00.606 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.540657+0000", 2024-09-11T07:46:00.606 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000106907 2024-09-11T07:46:00.606 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.606 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.606 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:46:00.606 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.540893+0000", 2024-09-11T07:46:00.606 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00023581900000000001 2024-09-11T07:46:00.606 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.606 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.606 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:00.606 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542361+0000", 2024-09-11T07:46:00.606 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0014681570000000001 2024-09-11T07:46:00.606 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.606 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.607 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.607 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.545374+0000", 2024-09-11T07:46:00.607 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0030128239999999999 2024-09-11T07:46:00.607 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.607 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.607 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:00.607 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.548342+0000", 2024-09-11T07:46:00.607 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002968428 2024-09-11T07:46:00.607 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.607 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.607 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:00.607 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.548388+0000", 2024-09-11T07:46:00.607 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6269e-05 2024-09-11T07:46:00.608 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:00.608 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:00.608 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:00.608 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.548424+0000", 2024-09-11T07:46:00.608 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5775999999999998e-05 2024-09-11T07:46:00.608 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.608 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:00.608 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.608 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:00.608 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:00.608 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:46:00.663 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:00.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:46:00 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:46:00] "GET /metrics HTTP/1.1" 200 222568 "" "Prometheus/2.43.0" 2024-09-11T07:46:00.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:00.685 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:00.685 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:00.786 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:00.808 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:00.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:00.810 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:00.910 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:00.932 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:00.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:00.933 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:01.034 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:01.055 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:01.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:01.056 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:01.156 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:01.177 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:01.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:01.178 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:01.268 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-11T07:46:01.279 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:01.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:01.333 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:01.333 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:01.434 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:01.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:01.461 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:01.461 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:01.513 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:46:01.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:01 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:46:01.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:01 smithi104 ceph-mon[33507]: pgmap v911: 158 pgs: 158 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:46:01.561 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:01.570 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:01 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:46:01.570 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:01 smithi022 ceph-mon[29462]: pgmap v911: 158 pgs: 158 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:46:01.570 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:01 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:46:01.570 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:01 smithi022 ceph-mon[37330]: pgmap v911: 158 pgs: 158 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:46:01.586 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:01.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:01.587 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:01.635 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:46:01.689 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:01.711 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:01.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:01.712 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:01.813 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:01.836 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:01.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:01.837 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:01.937 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:01.987 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:01.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:01.988 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:02.089 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:02.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:02.111 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:02.111 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:02.212 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:02.235 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:02.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:02.236 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:02.336 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:02.358 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:02.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:02.359 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:02.449 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is unset 2024-09-11T07:46:02.459 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:02.481 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:02.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:02.482 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:02.552 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:02 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:46:02.552 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:02 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:46:02.552 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:02 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1512839604' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:46:02.583 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:02.607 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:02.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:02.608 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:02.661 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:02 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:46:02.661 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:02 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:46:02.661 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:02 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1512839604' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:46:02.661 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:02 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:46:02.661 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:02 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:46:02.661 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:02 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1512839604' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:46:02.709 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:02.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:02.733 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:02.733 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:02.784 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:46:02.834 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:02.856 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:02.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:02.857 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:02.958 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:02.982 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:02.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:02.984 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:03.080 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:46:03.080 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:46:03.080 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 0 2024-09-11T07:46:03.080 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:46:03.084 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:03.107 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:03.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:03.108 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:03.209 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:03.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:03.231 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:03.232 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:03.332 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:03.360 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:03.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:03.361 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:03.462 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:03.484 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:03.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:03.485 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:03.586 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:03.592 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:03 smithi022 ceph-mon[29462]: pgmap v912: 158 pgs: 158 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 770 B/s rd, 0 op/s 2024-09-11T07:46:03.592 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:03 smithi022 ceph-mon[29462]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:46:03.592 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:03 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1512839604' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:46:03.592 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:03 smithi022 ceph-mon[29462]: osdmap e415: 8 total, 8 up, 7 in 2024-09-11T07:46:03.593 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:03 smithi022 ceph-mon[37330]: pgmap v912: 158 pgs: 158 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 770 B/s rd, 0 op/s 2024-09-11T07:46:03.593 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:03 smithi022 ceph-mon[37330]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:46:03.593 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:03 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1512839604' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:46:03.593 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:03 smithi022 ceph-mon[37330]: osdmap e415: 8 total, 8 up, 7 in 2024-09-11T07:46:03.608 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:03.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:03.609 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:03.633 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:03 smithi104 ceph-mon[33507]: pgmap v912: 158 pgs: 158 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 770 B/s rd, 0 op/s 2024-09-11T07:46:03.633 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:03 smithi104 ceph-mon[33507]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:46:03.633 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:03 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1512839604' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:46:03.633 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:03 smithi104 ceph-mon[33507]: osdmap e415: 8 total, 8 up, 7 in 2024-09-11T07:46:03.657 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-11T07:46:03.710 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:03.761 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:03.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:03.762 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:03.863 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:03.885 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:03.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:03.886 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:03.941 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:46:03.976 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:46:03.979 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:46:03.987 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:04.010 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:04.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:04.011 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:04.112 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:04.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:04.134 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:04.134 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:04.235 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:04.274 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:04.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:04.275 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:04.328 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:46:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:46:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:46:04.376 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:04.400 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:04.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:04.401 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:04.502 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:04.524 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:04.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:04.525 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:04.626 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:04.652 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:04.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:04.653 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:04.666 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:04 smithi022 ceph-mon[37330]: 6.2 scrub starts 2024-09-11T07:46:04.666 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:04 smithi022 ceph-mon[37330]: 6.2 scrub ok 2024-09-11T07:46:04.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:04 smithi022 ceph-mon[29462]: 6.2 scrub starts 2024-09-11T07:46:04.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:04 smithi022 ceph-mon[29462]: 6.2 scrub ok 2024-09-11T07:46:04.674 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:46:04.753 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:04.777 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:04.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:04.778 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:04.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:04 smithi104 ceph-mon[33507]: 6.2 scrub starts 2024-09-11T07:46:04.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:04 smithi104 ceph-mon[33507]: 6.2 scrub ok 2024-09-11T07:46:04.879 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:04.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:04.901 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:04.901 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:05.002 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:05.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:05.027 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:05.027 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:05.105 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":913,"stamp":"2024-09-11T07:46:02.444406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1343,"num_read_kb":1200,"num_write":829,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":2050295,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1765,"ondisk_log_size":1765,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":456,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":6302744,"kb_used_data":9132,"kb_used_omap":14,"kb_used_meta":6293489,"kb_avail":649913320,"statfs":{"total":671965249536,"available":665511239680,"internally_reserved":0,"allocated":9351168,"data_stored":5713471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14826,"internal_metadata":6444533270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.058223"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":167,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.753313+0000","last_change":"2024-09-11T07:44:23.770638+0000","last_active":"2024-09-11T07:45:56.753313+0000","last_peered":"2024-09-11T07:45:56.753313+0000","last_clean":"2024-09-11T07:45:56.753313+0000","last_became_active":"2024-09-11T07:44:23.769058+0000","last_became_peered":"2024-09-11T07:44:23.769058+0000","last_unstale":"2024-09-11T07:45:56.753313+0000","last_undegraded":"2024-09-11T07:45:56.753313+0000","last_fullsized":"2024-09-11T07:45:56.753313+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:00.048124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:37:29.007411+0000","last_clean_scrub_stamp":"2024-09-11T07:44:00.048124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001472253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":472,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.758112+0000","last_change":"2024-09-11T07:45:27.108301+0000","last_active":"2024-09-11T07:45:56.758112+0000","last_peered":"2024-09-11T07:45:56.758112+0000","last_clean":"2024-09-11T07:45:56.758112+0000","last_became_active":"2024-09-11T07:45:27.107996+0000","last_became_peered":"2024-09-11T07:45:27.107996+0000","last_unstale":"2024-09-11T07:45:56.758112+0000","last_undegraded":"2024-09-11T07:45:56.758112+0000","last_fullsized":"2024-09-11T07:45:56.758112+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:21.375076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:21.375076+0000","last_clean_scrub_stamp":"2024-09-11T07:44:21.375076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0068686850000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"234'34","reported_seq":486,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.752935+0000","last_change":"2024-09-11T07:44:02.682805+0000","last_active":"2024-09-11T07:45:56.752935+0000","last_peered":"2024-09-11T07:45:56.752935+0000","last_clean":"2024-09-11T07:45:56.752935+0000","last_became_active":"2024-09-11T07:26:33.544627+0000","last_became_peered":"2024-09-11T07:26:33.544627+0000","last_unstale":"2024-09-11T07:45:56.752935+0000","last_undegraded":"2024-09-11T07:45:56.752935+0000","last_fullsized":"2024-09-11T07:45:56.752935+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'34","last_scrub_stamp":"2024-09-11T07:44:02.682692+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:44:02.682692+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056034049999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":29,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.760082+0000","last_change":"2024-09-11T07:45:27.100020+0000","last_active":"2024-09-11T07:45:56.760082+0000","last_peered":"2024-09-11T07:45:56.760082+0000","last_clean":"2024-09-11T07:45:56.760082+0000","last_became_active":"2024-09-11T07:45:27.095499+0000","last_became_peered":"2024-09-11T07:45:27.095499+0000","last_unstale":"2024-09-11T07:45:56.760082+0000","last_undegraded":"2024-09-11T07:45:56.760082+0000","last_fullsized":"2024-09-11T07:45:56.760082+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:21.291787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:43:21.291787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":461,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.755421+0000","last_change":"2024-09-11T07:44:19.655482+0000","last_active":"2024-09-11T07:45:56.755421+0000","last_peered":"2024-09-11T07:45:56.755421+0000","last_clean":"2024-09-11T07:45:56.755421+0000","last_became_active":"2024-09-11T07:39:57.767806+0000","last_became_peered":"2024-09-11T07:39:57.767806+0000","last_unstale":"2024-09-11T07:45:56.755421+0000","last_undegraded":"2024-09-11T07:45:56.755421+0000","last_fullsized":"2024-09-11T07:45:56.755421+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:19.655401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:19.655401+0000","last_clean_scrub_stamp":"2024-09-11T07:44:19.655401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025632099999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"234'19","reported_seq":39,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.761386+0000","last_change":"2024-09-11T07:45:27.144397+0000","last_active":"2024-09-11T07:45:56.761386+0000","last_peered":"2024-09-11T07:45:56.761386+0000","last_clean":"2024-09-11T07:45:56.761386+0000","last_became_active":"2024-09-11T07:45:27.095974+0000","last_became_peered":"2024-09-11T07:45:27.095974+0000","last_unstale":"2024-09-11T07:45:56.761386+0000","last_undegraded":"2024-09-11T07:45:56.761386+0000","last_fullsized":"2024-09-11T07:45:56.761386+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:43:59.245081+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:36:42.708941+0000","last_clean_scrub_stamp":"2024-09-11T07:43:59.245081+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040986499999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":181,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.751487+0000","last_change":"2024-09-11T07:45:12.417072+0000","last_active":"2024-09-11T07:45:56.751487+0000","last_peered":"2024-09-11T07:45:56.751487+0000","last_clean":"2024-09-11T07:45:56.751487+0000","last_became_active":"2024-09-11T07:44:23.769502+0000","last_became_peered":"2024-09-11T07:44:23.769502+0000","last_unstale":"2024-09-11T07:45:56.751487+0000","last_undegraded":"2024-09-11T07:45:56.751487+0000","last_fullsized":"2024-09-11T07:45:56.751487+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:12.416990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:45:12.416990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:27.805987+0000","scrub_duration":0.001315529,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":338,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.754396+0000","last_change":"2024-09-11T07:45:27.104266+0000","last_active":"2024-09-11T07:45:56.754396+0000","last_peered":"2024-09-11T07:45:56.754396+0000","last_clean":"2024-09-11T07:45:56.754396+0000","last_became_active":"2024-09-11T07:45:27.103947+0000","last_became_peered":"2024-09-11T07:45:27.103947+0000","last_unstale":"2024-09-11T07:45:56.754396+0000","last_undegraded":"2024-09-11T07:45:56.754396+0000","last_fullsized":"2024-09-11T07:45:56.754396+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:25.450076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:43:25.450076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058146650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":440,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.751960+0000","last_change":"2024-09-11T07:43:26.559203+0000","last_active":"2024-09-11T07:45:56.751960+0000","last_peered":"2024-09-11T07:45:56.751960+0000","last_clean":"2024-09-11T07:45:56.751960+0000","last_became_active":"2024-09-11T07:32:00.516662+0000","last_became_peered":"2024-09-11T07:32:00.516662+0000","last_unstale":"2024-09-11T07:45:56.751960+0000","last_undegraded":"2024-09-11T07:45:56.751960+0000","last_fullsized":"2024-09-11T07:45:56.751960+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:26.559109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:43:26.559109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044217739999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"234'9","reported_seq":45,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.757279+0000","last_change":"2024-09-11T07:45:32.375248+0000","last_active":"2024-09-11T07:45:56.757279+0000","last_peered":"2024-09-11T07:45:56.757279+0000","last_clean":"2024-09-11T07:45:56.757279+0000","last_became_active":"2024-09-11T07:45:27.096974+0000","last_became_peered":"2024-09-11T07:45:27.096974+0000","last_unstale":"2024-09-11T07:45:56.757279+0000","last_undegraded":"2024-09-11T07:45:56.757279+0000","last_fullsized":"2024-09-11T07:45:56.757279+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'9","last_scrub_stamp":"2024-09-11T07:45:32.375156+0000","last_deep_scrub":"234'9","last_deep_scrub_stamp":"2024-09-11T07:45:32.375156+0000","last_clean_scrub_stamp":"2024-09-11T07:45:32.375156+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:52.079908+0000","scrub_duration":0.011478199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":429,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.751422+0000","last_change":"2024-09-11T07:45:27.113031+0000","last_active":"2024-09-11T07:45:56.751422+0000","last_peered":"2024-09-11T07:45:56.751422+0000","last_clean":"2024-09-11T07:45:56.751422+0000","last_became_active":"2024-09-11T07:45:27.112681+0000","last_became_peered":"2024-09-11T07:45:27.112681+0000","last_unstale":"2024-09-11T07:45:56.751422+0000","last_undegraded":"2024-09-11T07:45:56.751422+0000","last_fullsized":"2024-09-11T07:45:56.751422+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:32.398398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.973323+0000","last_clean_scrub_stamp":"2024-09-11T07:44:32.398398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002203782,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":69,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.758622+0000","last_change":"2024-09-11T07:44:23.769584+0000","last_active":"2024-09-11T07:45:56.758622+0000","last_peered":"2024-09-11T07:45:56.758622+0000","last_clean":"2024-09-11T07:45:56.758622+0000","last_became_active":"2024-09-11T07:44:23.769187+0000","last_became_peered":"2024-09-11T07:44:23.769187+0000","last_unstale":"2024-09-11T07:45:56.758622+0000","last_undegraded":"2024-09-11T07:45:56.758622+0000","last_fullsized":"2024-09-11T07:45:56.758622+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:04.074067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:45.149916+0000","last_clean_scrub_stamp":"2024-09-11T07:44:04.074067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":97,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.762696+0000","last_change":"2024-09-11T07:45:49.765879+0000","last_active":"2024-09-11T07:45:56.762696+0000","last_peered":"2024-09-11T07:45:56.762696+0000","last_clean":"2024-09-11T07:45:56.762696+0000","last_became_active":"2024-09-11T07:45:49.765541+0000","last_became_peered":"2024-09-11T07:45:49.765541+0000","last_unstale":"2024-09-11T07:45:56.762696+0000","last_undegraded":"2024-09-11T07:45:56.762696+0000","last_fullsized":"2024-09-11T07:45:56.762696+0000","mapping_epoch":407,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":408,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:56.219368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_clean_scrub_stamp":"2024-09-11T07:43:56.219368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032973189999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"234'19","reported_seq":302,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.752423+0000","last_change":"2024-09-11T07:45:27.158944+0000","last_active":"2024-09-11T07:45:56.752423+0000","last_peered":"2024-09-11T07:45:56.752423+0000","last_clean":"2024-09-11T07:45:56.752423+0000","last_became_active":"2024-09-11T07:45:27.112765+0000","last_became_peered":"2024-09-11T07:45:27.112765+0000","last_unstale":"2024-09-11T07:45:56.752423+0000","last_undegraded":"2024-09-11T07:45:56.752423+0000","last_fullsized":"2024-09-11T07:45:56.752423+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:43:57.284262+0000","last_deep_scrub":"136'13","last_deep_scrub_stamp":"2024-09-11T07:31:08.042198+0000","last_clean_scrub_stamp":"2024-09-11T07:43:57.284262+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0089695690000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":467,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.753941+0000","last_change":"2024-09-11T07:45:27.115012+0000","last_active":"2024-09-11T07:45:56.753941+0000","last_peered":"2024-09-11T07:45:56.753941+0000","last_clean":"2024-09-11T07:45:56.753941+0000","last_became_active":"2024-09-11T07:45:27.114162+0000","last_became_peered":"2024-09-11T07:45:27.114162+0000","last_unstale":"2024-09-11T07:45:56.753941+0000","last_undegraded":"2024-09-11T07:45:56.753941+0000","last_fullsized":"2024-09-11T07:45:56.753941+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:24.340025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:44:24.340025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002461366,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":456,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.754941+0000","last_change":"2024-09-11T07:44:04.287572+0000","last_active":"2024-09-11T07:45:56.754941+0000","last_peered":"2024-09-11T07:45:56.754941+0000","last_clean":"2024-09-11T07:45:56.754941+0000","last_became_active":"2024-09-11T07:32:00.530335+0000","last_became_peered":"2024-09-11T07:32:00.530335+0000","last_unstale":"2024-09-11T07:45:56.754941+0000","last_undegraded":"2024-09-11T07:45:56.754941+0000","last_fullsized":"2024-09-11T07:45:56.754941+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:04.287469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:44:04.287469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011609975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.765536+0000","last_change":"2024-09-11T07:45:56.765116+0000","last_active":"2024-09-11T07:45:56.765536+0000","last_peered":"2024-09-11T07:45:56.765536+0000","last_clean":"2024-09-11T07:45:56.765536+0000","last_became_active":"2024-09-11T07:45:56.763997+0000","last_became_peered":"2024-09-11T07:45:56.763997+0000","last_unstale":"2024-09-11T07:45:56.765536+0000","last_undegraded":"2024-09-11T07:45:56.765536+0000","last_fullsized":"2024-09-11T07:45:56.765536+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:08.078880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"87'81","reported_seq":38,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.759651+0000","last_change":"2024-09-11T07:45:01.833611+0000","last_active":"2024-09-11T07:45:56.759651+0000","last_peered":"2024-09-11T07:45:56.759651+0000","last_clean":"2024-09-11T07:45:56.759651+0000","last_became_active":"2024-09-11T07:45:01.808023+0000","last_became_peered":"2024-09-11T07:45:01.808023+0000","last_unstale":"2024-09-11T07:45:56.759651+0000","last_undegraded":"2024-09-11T07:45:56.759651+0000","last_fullsized":"2024-09-11T07:45:56.759651+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:44:22.387527+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.387527+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015232676000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":370027,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"234'31","reported_seq":386,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.760680+0000","last_change":"2024-09-11T07:45:27.125496+0000","last_active":"2024-09-11T07:45:56.760680+0000","last_peered":"2024-09-11T07:45:56.760680+0000","last_clean":"2024-09-11T07:45:56.760680+0000","last_became_active":"2024-09-11T07:45:27.094411+0000","last_became_peered":"2024-09-11T07:45:27.094411+0000","last_unstale":"2024-09-11T07:45:56.760680+0000","last_undegraded":"2024-09-11T07:45:56.760680+0000","last_fullsized":"2024-09-11T07:45:56.760680+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'31","last_scrub_stamp":"2024-09-11T07:44:06.132709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:44:06.132709+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034170279999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":458,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.751606+0000","last_change":"2024-09-11T07:45:27.100579+0000","last_active":"2024-09-11T07:45:56.751606+0000","last_peered":"2024-09-11T07:45:56.751606+0000","last_clean":"2024-09-11T07:45:56.751606+0000","last_became_active":"2024-09-11T07:45:27.100238+0000","last_became_peered":"2024-09-11T07:45:27.100238+0000","last_unstale":"2024-09-11T07:45:56.751606+0000","last_undegraded":"2024-09-11T07:45:56.751606+0000","last_fullsized":"2024-09-11T07:45:56.751606+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:08.501392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:08.501392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:32.180280+0000","scrub_duration":0.004339763,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":37,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.753018+0000","last_change":"2024-09-11T07:45:27.117697+0000","last_active":"2024-09-11T07:45:56.753018+0000","last_peered":"2024-09-11T07:45:56.753018+0000","last_clean":"2024-09-11T07:45:56.753018+0000","last_became_active":"2024-09-11T07:45:27.117371+0000","last_became_peered":"2024-09-11T07:45:27.117371+0000","last_unstale":"2024-09-11T07:45:56.753018+0000","last_undegraded":"2024-09-11T07:45:56.753018+0000","last_fullsized":"2024-09-11T07:45:56.753018+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:23.359400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:43:23.359400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":462,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.762337+0000","last_change":"2024-09-11T07:45:27.104861+0000","last_active":"2024-09-11T07:45:56.762337+0000","last_peered":"2024-09-11T07:45:56.762337+0000","last_clean":"2024-09-11T07:45:56.762337+0000","last_became_active":"2024-09-11T07:45:27.104519+0000","last_became_peered":"2024-09-11T07:45:27.104519+0000","last_unstale":"2024-09-11T07:45:56.762337+0000","last_undegraded":"2024-09-11T07:45:56.762337+0000","last_fullsized":"2024-09-11T07:45:56.762337+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:57.645536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:32.281886+0000","last_clean_scrub_stamp":"2024-09-11T07:43:57.645536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018691090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:57.201262+0000","last_change":"2024-09-11T07:45:57.200575+0000","last_active":"2024-09-11T07:45:57.201262+0000","last_peered":"2024-09-11T07:45:57.201262+0000","last_clean":"2024-09-11T07:45:57.201262+0000","last_became_active":"2024-09-11T07:45:57.200271+0000","last_became_peered":"2024-09-11T07:45:57.200271+0000","last_unstale":"2024-09-11T07:45:57.201262+0000","last_undegraded":"2024-09-11T07:45:57.201262+0000","last_fullsized":"2024-09-11T07:45:57.201262+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:15.892665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.9","version":"87'81","reported_seq":33,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.751245+0000","last_change":"2024-09-11T07:45:01.812611+0000","last_active":"2024-09-11T07:45:56.751245+0000","last_peered":"2024-09-11T07:45:56.751245+0000","last_clean":"2024-09-11T07:45:56.751245+0000","last_became_active":"2024-09-11T07:45:01.808923+0000","last_became_peered":"2024-09-11T07:45:01.808923+0000","last_unstale":"2024-09-11T07:45:56.751245+0000","last_undegraded":"2024-09-11T07:45:56.751245+0000","last_fullsized":"2024-09-11T07:45:56.751245+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:44:22.387527+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.387527+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015232676000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"234'25","reported_seq":310,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.755985+0000","last_change":"2024-09-11T07:45:33.404225+0000","last_active":"2024-09-11T07:45:56.755985+0000","last_peered":"2024-09-11T07:45:56.755985+0000","last_clean":"2024-09-11T07:45:56.755985+0000","last_became_active":"2024-09-11T07:45:27.097522+0000","last_became_peered":"2024-09-11T07:45:27.097522+0000","last_unstale":"2024-09-11T07:45:56.755985+0000","last_undegraded":"2024-09-11T07:45:56.755985+0000","last_fullsized":"2024-09-11T07:45:56.755985+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'25","last_scrub_stamp":"2024-09-11T07:45:33.403950+0000","last_deep_scrub":"112'15","last_deep_scrub_stamp":"2024-09-11T07:31:10.005896+0000","last_clean_scrub_stamp":"2024-09-11T07:45:33.403950+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:59.506481+0000","scrub_duration":0.0044240010000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":175,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.752560+0000","last_change":"2024-09-11T07:44:23.768467+0000","last_active":"2024-09-11T07:45:56.752560+0000","last_peered":"2024-09-11T07:45:56.752560+0000","last_clean":"2024-09-11T07:45:56.752560+0000","last_became_active":"2024-09-11T07:44:23.768172+0000","last_became_peered":"2024-09-11T07:44:23.768172+0000","last_unstale":"2024-09-11T07:45:56.752560+0000","last_undegraded":"2024-09-11T07:45:56.752560+0000","last_fullsized":"2024-09-11T07:45:56.752560+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:54.910214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:43:54.910214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015981232000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"299'11","reported_seq":718,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.761327+0000","last_change":"2024-09-11T07:43:21.381901+0000","last_active":"2024-09-11T07:45:56.761327+0000","last_peered":"2024-09-11T07:45:56.761327+0000","last_clean":"2024-09-11T07:45:56.761327+0000","last_became_active":"2024-09-11T07:40:11.697568+0000","last_became_peered":"2024-09-11T07:40:11.697568+0000","last_unstale":"2024-09-11T07:45:56.761327+0000","last_undegraded":"2024-09-11T07:45:56.761327+0000","last_fullsized":"2024-09-11T07:45:56.761327+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"299'11","last_scrub_stamp":"2024-09-11T07:43:21.381642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:43:21.381642+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023952790000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":327,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.760525+0000","last_change":"2024-09-11T07:45:27.099766+0000","last_active":"2024-09-11T07:45:56.760525+0000","last_peered":"2024-09-11T07:45:56.760525+0000","last_clean":"2024-09-11T07:45:56.760525+0000","last_became_active":"2024-09-11T07:45:27.099434+0000","last_became_peered":"2024-09-11T07:45:27.099434+0000","last_unstale":"2024-09-11T07:45:56.760525+0000","last_undegraded":"2024-09-11T07:45:56.760525+0000","last_fullsized":"2024-09-11T07:45:56.760525+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:16.342336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:43:16.342336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013626420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.759658+0000","last_change":"2024-09-11T07:45:56.758475+0000","last_active":"2024-09-11T07:45:56.759658+0000","last_peered":"2024-09-11T07:45:56.759658+0000","last_clean":"2024-09-11T07:45:56.759658+0000","last_became_active":"2024-09-11T07:45:56.758196+0000","last_became_peered":"2024-09-11T07:45:56.758196+0000","last_unstale":"2024-09-11T07:45:56.759658+0000","last_undegraded":"2024-09-11T07:45:56.759658+0000","last_fullsized":"2024-09-11T07:45:56.759658+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:20.308690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"87'81","reported_seq":33,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.757710+0000","last_change":"2024-09-11T07:45:01.807167+0000","last_active":"2024-09-11T07:45:56.757710+0000","last_peered":"2024-09-11T07:45:56.757710+0000","last_clean":"2024-09-11T07:45:56.757710+0000","last_became_active":"2024-09-11T07:45:01.805300+0000","last_became_peered":"2024-09-11T07:45:01.805300+0000","last_unstale":"2024-09-11T07:45:56.757710+0000","last_undegraded":"2024-09-11T07:45:56.757710+0000","last_fullsized":"2024-09-11T07:45:56.757710+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:44:22.387527+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.387527+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015232676000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53669,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"234'34","reported_seq":343,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.757686+0000","last_change":"2024-09-11T07:45:27.129184+0000","last_active":"2024-09-11T07:45:56.757686+0000","last_peered":"2024-09-11T07:45:56.757686+0000","last_clean":"2024-09-11T07:45:56.757686+0000","last_became_active":"2024-09-11T07:45:27.095634+0000","last_became_peered":"2024-09-11T07:45:27.095634+0000","last_unstale":"2024-09-11T07:45:56.757686+0000","last_undegraded":"2024-09-11T07:45:56.757686+0000","last_fullsized":"2024-09-11T07:45:56.757686+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'34","last_scrub_stamp":"2024-09-11T07:43:57.973282+0000","last_deep_scrub":"83'22","last_deep_scrub_stamp":"2024-09-11T07:34:04.375457+0000","last_clean_scrub_stamp":"2024-09-11T07:43:57.973282+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086024310000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":195,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.751263+0000","last_change":"2024-09-11T07:44:27.416260+0000","last_active":"2024-09-11T07:45:56.751263+0000","last_peered":"2024-09-11T07:45:56.751263+0000","last_clean":"2024-09-11T07:45:56.751263+0000","last_became_active":"2024-09-11T07:44:23.770343+0000","last_became_peered":"2024-09-11T07:44:23.770343+0000","last_unstale":"2024-09-11T07:45:56.751263+0000","last_undegraded":"2024-09-11T07:45:56.751263+0000","last_fullsized":"2024-09-11T07:45:56.751263+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:27.416207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:27.416207+0000","last_clean_scrub_stamp":"2024-09-11T07:44:27.416207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001230096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"395'10","reported_seq":634,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.753291+0000","last_change":"2024-09-11T07:45:27.148760+0000","last_active":"2024-09-11T07:45:56.753291+0000","last_peered":"2024-09-11T07:45:56.753291+0000","last_clean":"2024-09-11T07:45:56.753291+0000","last_became_active":"2024-09-11T07:45:27.122326+0000","last_became_peered":"2024-09-11T07:45:27.122326+0000","last_unstale":"2024-09-11T07:45:56.753291+0000","last_undegraded":"2024-09-11T07:45:56.753291+0000","last_fullsized":"2024-09-11T07:45:56.753291+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"366'9","last_scrub_stamp":"2024-09-11T07:43:26.334501+0000","last_deep_scrub":"366'9","last_deep_scrub_stamp":"2024-09-11T07:43:26.334501+0000","last_clean_scrub_stamp":"2024-09-11T07:43:26.334501+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050834449999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":304,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.757028+0000","last_change":"2024-09-11T07:45:49.765810+0000","last_active":"2024-09-11T07:45:56.757028+0000","last_peered":"2024-09-11T07:45:56.757028+0000","last_clean":"2024-09-11T07:45:56.757028+0000","last_became_active":"2024-09-11T07:45:49.765504+0000","last_became_peered":"2024-09-11T07:45:49.765504+0000","last_unstale":"2024-09-11T07:45:56.757028+0000","last_undegraded":"2024-09-11T07:45:56.757028+0000","last_fullsized":"2024-09-11T07:45:56.757028+0000","mapping_epoch":407,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":408,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:19.336454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:31.200685+0000","last_clean_scrub_stamp":"2024-09-11T07:45:19.336454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:23.234525+0000","scrub_duration":0.001371688,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:57.199693+0000","last_change":"2024-09-11T07:45:57.199288+0000","last_active":"2024-09-11T07:45:57.199693+0000","last_peered":"2024-09-11T07:45:57.199693+0000","last_clean":"2024-09-11T07:45:57.199693+0000","last_became_active":"2024-09-11T07:45:57.198828+0000","last_became_peered":"2024-09-11T07:45:57.198828+0000","last_unstale":"2024-09-11T07:45:57.199693+0000","last_undegraded":"2024-09-11T07:45:57.199693+0000","last_fullsized":"2024-09-11T07:45:57.199693+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:59.923306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"234'24","reported_seq":379,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.755160+0000","last_change":"2024-09-11T07:45:27.157398+0000","last_active":"2024-09-11T07:45:56.755160+0000","last_peered":"2024-09-11T07:45:56.755160+0000","last_clean":"2024-09-11T07:45:56.755160+0000","last_became_active":"2024-09-11T07:45:27.102164+0000","last_became_peered":"2024-09-11T07:45:27.102164+0000","last_unstale":"2024-09-11T07:45:56.755160+0000","last_undegraded":"2024-09-11T07:45:56.755160+0000","last_fullsized":"2024-09-11T07:45:56.755160+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'24","last_scrub_stamp":"2024-09-11T07:43:55.283206+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-09-11T07:29:38.112111+0000","last_clean_scrub_stamp":"2024-09-11T07:43:55.283206+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001886759,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":432,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.751216+0000","last_change":"2024-09-11T07:43:24.643754+0000","last_active":"2024-09-11T07:45:56.751216+0000","last_peered":"2024-09-11T07:45:56.751216+0000","last_clean":"2024-09-11T07:45:56.751216+0000","last_became_active":"2024-09-11T07:32:00.518538+0000","last_became_peered":"2024-09-11T07:32:00.518538+0000","last_unstale":"2024-09-11T07:45:56.751216+0000","last_undegraded":"2024-09-11T07:45:56.751216+0000","last_fullsized":"2024-09-11T07:45:56.751216+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:24.643676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:43:24.643676+0000","last_clean_scrub_stamp":"2024-09-11T07:43:24.643676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039936859999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":39,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.761880+0000","last_change":"2024-09-11T07:45:30.682670+0000","last_active":"2024-09-11T07:45:56.761880+0000","last_peered":"2024-09-11T07:45:56.761880+0000","last_clean":"2024-09-11T07:45:56.761880+0000","last_became_active":"2024-09-11T07:45:27.094025+0000","last_became_peered":"2024-09-11T07:45:27.094025+0000","last_unstale":"2024-09-11T07:45:56.761880+0000","last_undegraded":"2024-09-11T07:45:56.761880+0000","last_fullsized":"2024-09-11T07:45:56.761880+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:30.682614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:19.714568+0000","last_clean_scrub_stamp":"2024-09-11T07:45:30.682614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:36.638911+0000","scrub_duration":0.0015960729999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":439,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.755675+0000","last_change":"2024-09-11T07:45:17.470823+0000","last_active":"2024-09-11T07:45:56.755675+0000","last_peered":"2024-09-11T07:45:56.755675+0000","last_clean":"2024-09-11T07:45:56.755675+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:45:56.755675+0000","last_undegraded":"2024-09-11T07:45:56.755675+0000","last_fullsized":"2024-09-11T07:45:56.755675+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:17.470730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:46.853387+0000","last_clean_scrub_stamp":"2024-09-11T07:45:17.470730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:25.293620+0000","scrub_duration":0.002761412,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.762404+0000","last_change":"2024-09-11T07:45:56.762206+0000","last_active":"2024-09-11T07:45:56.762404+0000","last_peered":"2024-09-11T07:45:56.762404+0000","last_clean":"2024-09-11T07:45:56.762404+0000","last_became_active":"2024-09-11T07:45:56.761971+0000","last_became_peered":"2024-09-11T07:45:56.761971+0000","last_unstale":"2024-09-11T07:45:56.762404+0000","last_undegraded":"2024-09-11T07:45:56.762404+0000","last_fullsized":"2024-09-11T07:45:56.762404+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:02.947203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"87'81","reported_seq":52,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.755526+0000","last_change":"2024-09-11T07:45:27.141030+0000","last_active":"2024-09-11T07:45:56.755526+0000","last_peered":"2024-09-11T07:45:56.755526+0000","last_clean":"2024-09-11T07:45:56.755526+0000","last_became_active":"2024-09-11T07:45:27.097022+0000","last_became_peered":"2024-09-11T07:45:27.097022+0000","last_unstale":"2024-09-11T07:45:56.755526+0000","last_undegraded":"2024-09-11T07:45:56.755526+0000","last_fullsized":"2024-09-11T07:45:56.755526+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:44:22.387527+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.387527+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015232676000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":762700,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"234'28","reported_seq":311,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.754028+0000","last_change":"2024-09-11T07:45:27.137984+0000","last_active":"2024-09-11T07:45:56.754028+0000","last_peered":"2024-09-11T07:45:56.754028+0000","last_clean":"2024-09-11T07:45:56.754028+0000","last_became_active":"2024-09-11T07:45:27.120120+0000","last_became_peered":"2024-09-11T07:45:27.120120+0000","last_unstale":"2024-09-11T07:45:56.754028+0000","last_undegraded":"2024-09-11T07:45:56.754028+0000","last_fullsized":"2024-09-11T07:45:56.754028+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'28","last_scrub_stamp":"2024-09-11T07:43:24.327746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:43:24.327746+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012975469999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":211,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.762444+0000","last_change":"2024-09-11T07:43:24.326702+0000","last_active":"2024-09-11T07:45:56.762444+0000","last_peered":"2024-09-11T07:45:56.762444+0000","last_clean":"2024-09-11T07:45:56.762444+0000","last_became_active":"2024-09-11T07:39:57.767741+0000","last_became_peered":"2024-09-11T07:39:57.767741+0000","last_unstale":"2024-09-11T07:45:56.762444+0000","last_undegraded":"2024-09-11T07:45:56.762444+0000","last_fullsized":"2024-09-11T07:45:56.762444+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:24.326341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:43:24.326341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027901060000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3],"acting":[2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"395'10","reported_seq":689,"reported_epoch":414,"state":"active+clean","last_fresh":"2024-09-11T07:45:57.903780+0000","last_change":"2024-09-11T07:45:27.151129+0000","last_active":"2024-09-11T07:45:57.903780+0000","last_peered":"2024-09-11T07:45:57.903780+0000","last_clean":"2024-09-11T07:45:57.903780+0000","last_became_active":"2024-09-11T07:45:27.098861+0000","last_became_peered":"2024-09-11T07:45:27.098861+0000","last_unstale":"2024-09-11T07:45:57.903780+0000","last_undegraded":"2024-09-11T07:45:57.903780+0000","last_fullsized":"2024-09-11T07:45:57.903780+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-11T07:43:57.666646+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-11T07:35:39.832547+0000","last_clean_scrub_stamp":"2024-09-11T07:43:57.666646+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034760120000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":413,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.751496+0000","last_change":"2024-09-11T07:44:17.483184+0000","last_active":"2024-09-11T07:45:56.751496+0000","last_peered":"2024-09-11T07:45:56.751496+0000","last_clean":"2024-09-11T07:45:56.751496+0000","last_became_active":"2024-09-11T07:32:00.511005+0000","last_became_peered":"2024-09-11T07:32:00.511005+0000","last_unstale":"2024-09-11T07:45:56.751496+0000","last_undegraded":"2024-09-11T07:45:56.751496+0000","last_fullsized":"2024-09-11T07:45:56.751496+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:17.483090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:44:17.483090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057481720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.765155+0000","last_change":"2024-09-11T07:45:56.764982+0000","last_active":"2024-09-11T07:45:56.765155+0000","last_peered":"2024-09-11T07:45:56.765155+0000","last_clean":"2024-09-11T07:45:56.765155+0000","last_became_active":"2024-09-11T07:45:56.764754+0000","last_became_peered":"2024-09-11T07:45:56.764754+0000","last_unstale":"2024-09-11T07:45:56.765155+0000","last_undegraded":"2024-09-11T07:45:56.765155+0000","last_fullsized":"2024-09-11T07:45:56.765155+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:25.447509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"87'81","reported_seq":33,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.759976+0000","last_change":"2024-09-11T07:45:27.100289+0000","last_active":"2024-09-11T07:45:56.759976+0000","last_peered":"2024-09-11T07:45:56.759976+0000","last_clean":"2024-09-11T07:45:56.759976+0000","last_became_active":"2024-09-11T07:45:27.099677+0000","last_became_peered":"2024-09-11T07:45:27.099677+0000","last_unstale":"2024-09-11T07:45:56.759976+0000","last_undegraded":"2024-09-11T07:45:56.759976+0000","last_fullsized":"2024-09-11T07:45:56.759976+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:44:22.387527+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.387527+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015232676000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"234'38","reported_seq":481,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.755024+0000","last_change":"2024-09-11T07:43:59.351755+0000","last_active":"2024-09-11T07:45:56.755024+0000","last_peered":"2024-09-11T07:45:56.755024+0000","last_clean":"2024-09-11T07:45:56.755024+0000","last_became_active":"2024-09-11T07:39:57.775541+0000","last_became_peered":"2024-09-11T07:39:57.775541+0000","last_unstale":"2024-09-11T07:45:56.755024+0000","last_undegraded":"2024-09-11T07:45:56.755024+0000","last_fullsized":"2024-09-11T07:45:56.755024+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'38","last_scrub_stamp":"2024-09-11T07:43:59.351585+0000","last_deep_scrub":"87'26","last_deep_scrub_stamp":"2024-09-11T07:35:52.942347+0000","last_clean_scrub_stamp":"2024-09-11T07:43:59.351585+0000","objects_scrubbed":8,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056500760000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":51,"num_read_kb":39,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":36,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.755076+0000","last_change":"2024-09-11T07:45:27.113078+0000","last_active":"2024-09-11T07:45:56.755076+0000","last_peered":"2024-09-11T07:45:56.755076+0000","last_clean":"2024-09-11T07:45:56.755076+0000","last_became_active":"2024-09-11T07:45:27.112250+0000","last_became_peered":"2024-09-11T07:45:27.112250+0000","last_unstale":"2024-09-11T07:45:56.755076+0000","last_undegraded":"2024-09-11T07:45:56.755076+0000","last_fullsized":"2024-09-11T07:45:56.755076+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:16.022013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:16.022013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:33.793740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":349,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.752644+0000","last_change":"2024-09-11T07:45:31.356987+0000","last_active":"2024-09-11T07:45:56.752644+0000","last_peered":"2024-09-11T07:45:56.752644+0000","last_clean":"2024-09-11T07:45:56.752644+0000","last_became_active":"2024-09-11T07:45:27.094619+0000","last_became_peered":"2024-09-11T07:45:27.094619+0000","last_unstale":"2024-09-11T07:45:56.752644+0000","last_undegraded":"2024-09-11T07:45:56.752644+0000","last_fullsized":"2024-09-11T07:45:56.752644+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:31.356921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:27.968579+0000","last_clean_scrub_stamp":"2024-09-11T07:45:31.356921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:44.701341+0000","scrub_duration":0.001320955,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":327,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.757922+0000","last_change":"2024-09-11T07:45:27.099015+0000","last_active":"2024-09-11T07:45:56.757922+0000","last_peered":"2024-09-11T07:45:56.757922+0000","last_clean":"2024-09-11T07:45:56.757922+0000","last_became_active":"2024-09-11T07:45:27.098502+0000","last_became_peered":"2024-09-11T07:45:27.098502+0000","last_unstale":"2024-09-11T07:45:56.757922+0000","last_undegraded":"2024-09-11T07:45:56.757922+0000","last_fullsized":"2024-09-11T07:45:56.757922+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:07.335214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:29.978798+0000","last_clean_scrub_stamp":"2024-09-11T07:43:07.335214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036155279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.765443+0000","last_change":"2024-09-11T07:45:56.764201+0000","last_active":"2024-09-11T07:45:56.765443+0000","last_peered":"2024-09-11T07:45:56.765443+0000","last_clean":"2024-09-11T07:45:56.765443+0000","last_became_active":"2024-09-11T07:45:56.763888+0000","last_became_peered":"2024-09-11T07:45:56.763888+0000","last_unstale":"2024-09-11T07:45:56.765443+0000","last_undegraded":"2024-09-11T07:45:56.765443+0000","last_fullsized":"2024-09-11T07:45:56.765443+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:01.979288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,7],"acting":[2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"87'81","reported_seq":312,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.753124+0000","last_change":"2024-09-11T07:45:27.114430+0000","last_active":"2024-09-11T07:45:56.753124+0000","last_peered":"2024-09-11T07:45:56.753124+0000","last_clean":"2024-09-11T07:45:56.753124+0000","last_became_active":"2024-09-11T07:45:27.111145+0000","last_became_peered":"2024-09-11T07:45:27.111145+0000","last_unstale":"2024-09-11T07:45:56.753124+0000","last_undegraded":"2024-09-11T07:45:56.753124+0000","last_fullsized":"2024-09-11T07:45:56.753124+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:44:22.387527+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.387527+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015232676000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"236'32","reported_seq":514,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.753860+0000","last_change":"2024-09-11T07:45:13.572062+0000","last_active":"2024-09-11T07:45:56.753860+0000","last_peered":"2024-09-11T07:45:56.753860+0000","last_clean":"2024-09-11T07:45:56.753860+0000","last_became_active":"2024-09-11T07:32:00.506270+0000","last_became_peered":"2024-09-11T07:32:00.506270+0000","last_unstale":"2024-09-11T07:45:56.753860+0000","last_undegraded":"2024-09-11T07:45:56.753860+0000","last_fullsized":"2024-09-11T07:45:56.753860+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'32","last_scrub_stamp":"2024-09-11T07:45:13.571949+0000","last_deep_scrub":"236'32","last_deep_scrub_stamp":"2024-09-11T07:36:50.023909+0000","last_clean_scrub_stamp":"2024-09-11T07:45:13.571949+0000","objects_scrubbed":8,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:19.249665+0000","scrub_duration":0.020494009000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":423,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.752226+0000","last_change":"2024-09-11T07:45:27.114174+0000","last_active":"2024-09-11T07:45:56.752226+0000","last_peered":"2024-09-11T07:45:56.752226+0000","last_clean":"2024-09-11T07:45:56.752226+0000","last_became_active":"2024-09-11T07:45:27.113680+0000","last_became_peered":"2024-09-11T07:45:27.113680+0000","last_unstale":"2024-09-11T07:45:56.752226+0000","last_undegraded":"2024-09-11T07:45:56.752226+0000","last_fullsized":"2024-09-11T07:45:56.752226+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:24.470602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:24.470602+0000","last_clean_scrub_stamp":"2024-09-11T07:44:24.470602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.040471685,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":242,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.758472+0000","last_change":"2024-09-11T07:44:12.312130+0000","last_active":"2024-09-11T07:45:56.758472+0000","last_peered":"2024-09-11T07:45:56.758472+0000","last_clean":"2024-09-11T07:45:56.758472+0000","last_became_active":"2024-09-11T07:36:05.225840+0000","last_became_peered":"2024-09-11T07:36:05.225840+0000","last_unstale":"2024-09-11T07:45:56.758472+0000","last_undegraded":"2024-09-11T07:45:56.758472+0000","last_fullsized":"2024-09-11T07:45:56.758472+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:12.312068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:22.285763+0000","last_clean_scrub_stamp":"2024-09-11T07:44:12.312068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01309188,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":57,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.758759+0000","last_change":"2024-09-11T07:45:23.663301+0000","last_active":"2024-09-11T07:45:56.758759+0000","last_peered":"2024-09-11T07:45:56.758759+0000","last_clean":"2024-09-11T07:45:56.758759+0000","last_became_active":"2024-09-11T07:44:23.769063+0000","last_became_peered":"2024-09-11T07:44:23.769063+0000","last_unstale":"2024-09-11T07:45:56.758759+0000","last_undegraded":"2024-09-11T07:45:56.758759+0000","last_fullsized":"2024-09-11T07:45:56.758759+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:23.663218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:39.723598+0000","last_clean_scrub_stamp":"2024-09-11T07:45:23.663218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:51.865688+0000","scrub_duration":0.006627272,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:57.202191+0000","last_change":"2024-09-11T07:45:57.201789+0000","last_active":"2024-09-11T07:45:57.202191+0000","last_peered":"2024-09-11T07:45:57.202191+0000","last_clean":"2024-09-11T07:45:57.202191+0000","last_became_active":"2024-09-11T07:45:57.201368+0000","last_became_peered":"2024-09-11T07:45:57.201368+0000","last_unstale":"2024-09-11T07:45:57.202191+0000","last_undegraded":"2024-09-11T07:45:57.202191+0000","last_fullsized":"2024-09-11T07:45:57.202191+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:58.453865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"87'81","reported_seq":33,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.754725+0000","last_change":"2024-09-11T07:45:01.818367+0000","last_active":"2024-09-11T07:45:56.754725+0000","last_peered":"2024-09-11T07:45:56.754725+0000","last_clean":"2024-09-11T07:45:56.754725+0000","last_became_active":"2024-09-11T07:45:01.817896+0000","last_became_peered":"2024-09-11T07:45:01.817896+0000","last_unstale":"2024-09-11T07:45:56.754725+0000","last_undegraded":"2024-09-11T07:45:56.754725+0000","last_fullsized":"2024-09-11T07:45:56.754725+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:44:22.387527+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.387527+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015232676000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"234'20","reported_seq":311,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.753840+0000","last_change":"2024-09-11T07:45:27.133766+0000","last_active":"2024-09-11T07:45:56.753840+0000","last_peered":"2024-09-11T07:45:56.753840+0000","last_clean":"2024-09-11T07:45:56.753840+0000","last_became_active":"2024-09-11T07:45:27.119509+0000","last_became_peered":"2024-09-11T07:45:27.119509+0000","last_unstale":"2024-09-11T07:45:56.753840+0000","last_undegraded":"2024-09-11T07:45:56.753840+0000","last_fullsized":"2024-09-11T07:45:56.753840+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'20","last_scrub_stamp":"2024-09-11T07:43:12.364487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:43:12.364487+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01198222,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":65,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.753102+0000","last_change":"2024-09-11T07:45:12.553757+0000","last_active":"2024-09-11T07:45:56.753102+0000","last_peered":"2024-09-11T07:45:56.753102+0000","last_clean":"2024-09-11T07:45:56.753102+0000","last_became_active":"2024-09-11T07:44:23.771022+0000","last_became_peered":"2024-09-11T07:44:23.771022+0000","last_unstale":"2024-09-11T07:45:56.753102+0000","last_undegraded":"2024-09-11T07:45:56.753102+0000","last_fullsized":"2024-09-11T07:45:56.753102+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:45:12.551633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:12.551633+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:19.283285+0000","scrub_duration":0.014607669,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"290'9","reported_seq":706,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.753580+0000","last_change":"2024-09-11T07:45:21.492545+0000","last_active":"2024-09-11T07:45:56.753580+0000","last_peered":"2024-09-11T07:45:56.753580+0000","last_clean":"2024-09-11T07:45:56.753580+0000","last_became_active":"2024-09-11T07:39:57.768155+0000","last_became_peered":"2024-09-11T07:39:57.768155+0000","last_unstale":"2024-09-11T07:45:56.753580+0000","last_undegraded":"2024-09-11T07:45:56.753580+0000","last_fullsized":"2024-09-11T07:45:56.753580+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"290'9","last_scrub_stamp":"2024-09-11T07:45:21.492460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:45:21.492460+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:48.304323+0000","scrub_duration":0.0074254259999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":439,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.754826+0000","last_change":"2024-09-11T07:43:08.315384+0000","last_active":"2024-09-11T07:45:56.754826+0000","last_peered":"2024-09-11T07:45:56.754826+0000","last_clean":"2024-09-11T07:45:56.754826+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:45:56.754826+0000","last_undegraded":"2024-09-11T07:45:56.754826+0000","last_fullsized":"2024-09-11T07:45:56.754826+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:08.315222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_clean_scrub_stamp":"2024-09-11T07:43:08.315222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010657583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:57.199952+0000","last_change":"2024-09-11T07:45:57.199626+0000","last_active":"2024-09-11T07:45:57.199952+0000","last_peered":"2024-09-11T07:45:57.199952+0000","last_clean":"2024-09-11T07:45:57.199952+0000","last_became_active":"2024-09-11T07:45:57.199305+0000","last_became_peered":"2024-09-11T07:45:57.199305+0000","last_unstale":"2024-09-11T07:45:57.199952+0000","last_undegraded":"2024-09-11T07:45:57.199952+0000","last_fullsized":"2024-09-11T07:45:57.199952+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:18.422653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"87'81","reported_seq":301,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.758833+0000","last_change":"2024-09-11T07:45:01.821266+0000","last_active":"2024-09-11T07:45:56.758833+0000","last_peered":"2024-09-11T07:45:56.758833+0000","last_clean":"2024-09-11T07:45:56.758833+0000","last_became_active":"2024-09-11T07:45:01.820927+0000","last_became_peered":"2024-09-11T07:45:01.820927+0000","last_unstale":"2024-09-11T07:45:56.758833+0000","last_undegraded":"2024-09-11T07:45:56.758833+0000","last_fullsized":"2024-09-11T07:45:56.758833+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:44:22.387527+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.387527+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015232676000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":21,"num_write":11,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":107340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"290'42","reported_seq":424,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.762484+0000","last_change":"2024-09-11T07:43:22.389378+0000","last_active":"2024-09-11T07:45:56.762484+0000","last_peered":"2024-09-11T07:45:56.762484+0000","last_clean":"2024-09-11T07:45:56.762484+0000","last_became_active":"2024-09-11T07:39:57.768267+0000","last_became_peered":"2024-09-11T07:39:57.768267+0000","last_unstale":"2024-09-11T07:45:56.762484+0000","last_undegraded":"2024-09-11T07:45:56.762484+0000","last_fullsized":"2024-09-11T07:45:56.762484+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"290'42","last_scrub_stamp":"2024-09-11T07:43:22.389278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:43:22.389278+0000","objects_scrubbed":10,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0085721119999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":422,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.754968+0000","last_change":"2024-09-11T07:45:08.282066+0000","last_active":"2024-09-11T07:45:56.754968+0000","last_peered":"2024-09-11T07:45:56.754968+0000","last_clean":"2024-09-11T07:45:56.754968+0000","last_became_active":"2024-09-11T07:44:23.772612+0000","last_became_peered":"2024-09-11T07:44:23.772612+0000","last_unstale":"2024-09-11T07:45:56.754968+0000","last_undegraded":"2024-09-11T07:45:56.754968+0000","last_fullsized":"2024-09-11T07:45:56.754968+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:08.281957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:51.308492+0000","last_clean_scrub_stamp":"2024-09-11T07:45:08.281957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:36.150157+0000","scrub_duration":0.012237031000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":451,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.757167+0000","last_change":"2024-09-11T07:45:27.120609+0000","last_active":"2024-09-11T07:45:56.757167+0000","last_peered":"2024-09-11T07:45:56.757167+0000","last_clean":"2024-09-11T07:45:56.757167+0000","last_became_active":"2024-09-11T07:45:27.120021+0000","last_became_peered":"2024-09-11T07:45:27.120021+0000","last_unstale":"2024-09-11T07:45:56.757167+0000","last_undegraded":"2024-09-11T07:45:56.757167+0000","last_fullsized":"2024-09-11T07:45:56.757167+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:20.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:44:20.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075336659999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":466,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.755349+0000","last_change":"2024-09-11T07:45:27.103089+0000","last_active":"2024-09-11T07:45:56.755349+0000","last_peered":"2024-09-11T07:45:56.755349+0000","last_clean":"2024-09-11T07:45:56.755349+0000","last_became_active":"2024-09-11T07:45:27.102600+0000","last_became_peered":"2024-09-11T07:45:27.102600+0000","last_unstale":"2024-09-11T07:45:56.755349+0000","last_undegraded":"2024-09-11T07:45:56.755349+0000","last_fullsized":"2024-09-11T07:45:56.755349+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:13.504354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:45:13.504354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:25.240970+0000","scrub_duration":0.0018451749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:57.199664+0000","last_change":"2024-09-11T07:45:57.198992+0000","last_active":"2024-09-11T07:45:57.199664+0000","last_peered":"2024-09-11T07:45:57.199664+0000","last_clean":"2024-09-11T07:45:57.199664+0000","last_became_active":"2024-09-11T07:45:57.198709+0000","last_became_peered":"2024-09-11T07:45:57.198709+0000","last_unstale":"2024-09-11T07:45:57.199664+0000","last_undegraded":"2024-09-11T07:45:57.199664+0000","last_fullsized":"2024-09-11T07:45:57.199664+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:59.340606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"87'81","reported_seq":312,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.753694+0000","last_change":"2024-09-11T07:45:27.117544+0000","last_active":"2024-09-11T07:45:56.753694+0000","last_peered":"2024-09-11T07:45:56.753694+0000","last_clean":"2024-09-11T07:45:56.753694+0000","last_became_active":"2024-09-11T07:45:27.117020+0000","last_became_peered":"2024-09-11T07:45:27.117020+0000","last_unstale":"2024-09-11T07:45:56.753694+0000","last_undegraded":"2024-09-11T07:45:56.753694+0000","last_fullsized":"2024-09-11T07:45:56.753694+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:44:22.387527+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.387527+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015232676000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":21,"num_write":11,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":107340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,5],"acting":[0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"234'22","reported_seq":277,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.758041+0000","last_change":"2024-09-11T07:45:49.765866+0000","last_active":"2024-09-11T07:45:56.758041+0000","last_peered":"2024-09-11T07:45:56.758041+0000","last_clean":"2024-09-11T07:45:56.758041+0000","last_became_active":"2024-09-11T07:45:49.765519+0000","last_became_peered":"2024-09-11T07:45:49.765519+0000","last_unstale":"2024-09-11T07:45:56.758041+0000","last_undegraded":"2024-09-11T07:45:56.758041+0000","last_fullsized":"2024-09-11T07:45:56.758041+0000","mapping_epoch":407,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":408,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'22","last_scrub_stamp":"2024-09-11T07:45:17.380652+0000","last_deep_scrub":"144'18","last_deep_scrub_stamp":"2024-09-11T07:31:51.603378+0000","last_clean_scrub_stamp":"2024-09-11T07:45:17.380652+0000","objects_scrubbed":5,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:45.277530+0000","scrub_duration":0.006530999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":181,"reported_epoch":414,"state":"active+clean","last_fresh":"2024-09-11T07:45:57.992355+0000","last_change":"2024-09-11T07:45:20.637602+0000","last_active":"2024-09-11T07:45:57.992355+0000","last_peered":"2024-09-11T07:45:57.992355+0000","last_clean":"2024-09-11T07:45:57.992355+0000","last_became_active":"2024-09-11T07:44:23.769220+0000","last_became_peered":"2024-09-11T07:44:23.769220+0000","last_unstale":"2024-09-11T07:45:57.992355+0000","last_undegraded":"2024-09-11T07:45:57.992355+0000","last_fullsized":"2024-09-11T07:45:57.992355+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'5","last_scrub_stamp":"2024-09-11T07:45:20.637535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:20.637535+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:26.191736+0000","scrub_duration":0.0059316919999999997,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":102,"num_read_kb":97,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":474,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.758898+0000","last_change":"2024-09-11T07:45:07.727921+0000","last_active":"2024-09-11T07:45:56.758898+0000","last_peered":"2024-09-11T07:45:56.758898+0000","last_clean":"2024-09-11T07:45:56.758898+0000","last_became_active":"2024-09-11T07:26:31.413354+0000","last_became_peered":"2024-09-11T07:26:31.413354+0000","last_unstale":"2024-09-11T07:45:56.758898+0000","last_undegraded":"2024-09-11T07:45:56.758898+0000","last_fullsized":"2024-09-11T07:45:56.758898+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:07.727847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:07.149367+0000","last_clean_scrub_stamp":"2024-09-11T07:45:07.727847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:26.945366+0000","scrub_duration":0.0053654319999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":461,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.760638+0000","last_change":"2024-09-11T07:43:11.249241+0000","last_active":"2024-09-11T07:45:56.760638+0000","last_peered":"2024-09-11T07:45:56.760638+0000","last_clean":"2024-09-11T07:45:56.760638+0000","last_became_active":"2024-09-11T07:32:00.508398+0000","last_became_peered":"2024-09-11T07:32:00.508398+0000","last_unstale":"2024-09-11T07:45:56.760638+0000","last_undegraded":"2024-09-11T07:45:56.760638+0000","last_fullsized":"2024-09-11T07:45:56.760638+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:11.242643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:28.246601+0000","last_clean_scrub_stamp":"2024-09-11T07:43:11.242643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056937530000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":439,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.751726+0000","last_change":"2024-09-11T07:44:25.382614+0000","last_active":"2024-09-11T07:45:56.751726+0000","last_peered":"2024-09-11T07:45:56.751726+0000","last_clean":"2024-09-11T07:45:56.751726+0000","last_became_active":"2024-09-11T07:32:00.515745+0000","last_became_peered":"2024-09-11T07:32:00.515745+0000","last_unstale":"2024-09-11T07:45:56.751726+0000","last_undegraded":"2024-09-11T07:45:56.751726+0000","last_fullsized":"2024-09-11T07:45:56.751726+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:25.382533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:25.382533+0000","last_clean_scrub_stamp":"2024-09-11T07:44:25.382533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018079610000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:57.202332+0000","last_change":"2024-09-11T07:45:57.202010+0000","last_active":"2024-09-11T07:45:57.202332+0000","last_peered":"2024-09-11T07:45:57.202332+0000","last_clean":"2024-09-11T07:45:57.202332+0000","last_became_active":"2024-09-11T07:45:57.201607+0000","last_became_peered":"2024-09-11T07:45:57.201607+0000","last_unstale":"2024-09-11T07:45:57.202332+0000","last_undegraded":"2024-09-11T07:45:57.202332+0000","last_fullsized":"2024-09-11T07:45:57.202332+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:58.302447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"87'81","reported_seq":29,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.753677+0000","last_change":"2024-09-11T07:45:27.115249+0000","last_active":"2024-09-11T07:45:56.753677+0000","last_peered":"2024-09-11T07:45:56.753677+0000","last_clean":"2024-09-11T07:45:56.753677+0000","last_became_active":"2024-09-11T07:45:27.114518+0000","last_became_peered":"2024-09-11T07:45:27.114518+0000","last_unstale":"2024-09-11T07:45:56.753677+0000","last_undegraded":"2024-09-11T07:45:56.753677+0000","last_fullsized":"2024-09-11T07:45:56.753677+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:44:22.387527+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.387527+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015639180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":21,"num_write":11,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":53670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"234'30","reported_seq":543,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.754096+0000","last_change":"2024-09-11T07:44:28.272807+0000","last_active":"2024-09-11T07:45:56.754096+0000","last_peered":"2024-09-11T07:45:56.754096+0000","last_clean":"2024-09-11T07:45:56.754096+0000","last_became_active":"2024-09-11T07:43:05.417321+0000","last_became_peered":"2024-09-11T07:43:05.417321+0000","last_unstale":"2024-09-11T07:45:56.754096+0000","last_undegraded":"2024-09-11T07:45:56.754096+0000","last_fullsized":"2024-09-11T07:45:56.754096+0000","mapping_epoch":358,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":359,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'30","last_scrub_stamp":"2024-09-11T07:44:28.272498+0000","last_deep_scrub":"94'18","last_deep_scrub_stamp":"2024-09-11T07:28:51.437397+0000","last_clean_scrub_stamp":"2024-09-11T07:44:28.272498+0000","objects_scrubbed":6,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006216643,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":479,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.758556+0000","last_change":"2024-09-11T07:45:27.124568+0000","last_active":"2024-09-11T07:45:56.758556+0000","last_peered":"2024-09-11T07:45:56.758556+0000","last_clean":"2024-09-11T07:45:56.758556+0000","last_became_active":"2024-09-11T07:45:27.115139+0000","last_became_peered":"2024-09-11T07:45:27.115139+0000","last_unstale":"2024-09-11T07:45:56.758556+0000","last_undegraded":"2024-09-11T07:45:56.758556+0000","last_fullsized":"2024-09-11T07:45:56.758556+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-11T07:44:02.361409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:44:02.361409+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010533394,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":450,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.751311+0000","last_change":"2024-09-11T07:43:56.441264+0000","last_active":"2024-09-11T07:45:56.751311+0000","last_peered":"2024-09-11T07:45:56.751311+0000","last_clean":"2024-09-11T07:45:56.751311+0000","last_became_active":"2024-09-11T07:39:57.772244+0000","last_became_peered":"2024-09-11T07:39:57.772244+0000","last_unstale":"2024-09-11T07:45:56.751311+0000","last_undegraded":"2024-09-11T07:45:56.751311+0000","last_fullsized":"2024-09-11T07:45:56.751311+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:56.441184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:16.967492+0000","last_clean_scrub_stamp":"2024-09-11T07:43:56.441184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047728199999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":67,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.761977+0000","last_change":"2024-09-11T07:45:27.102656+0000","last_active":"2024-09-11T07:45:56.761977+0000","last_peered":"2024-09-11T07:45:56.761977+0000","last_clean":"2024-09-11T07:45:56.761977+0000","last_became_active":"2024-09-11T07:45:27.101352+0000","last_became_peered":"2024-09-11T07:45:27.101352+0000","last_unstale":"2024-09-11T07:45:56.761977+0000","last_undegraded":"2024-09-11T07:45:56.761977+0000","last_fullsized":"2024-09-11T07:45:56.761977+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:22.519491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:51.187127+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.519491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.024174253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":158,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.754424+0000","last_change":"2024-09-11T07:43:58.208814+0000","last_active":"2024-09-11T07:45:56.754424+0000","last_peered":"2024-09-11T07:45:56.754424+0000","last_clean":"2024-09-11T07:45:56.754424+0000","last_became_active":"2024-09-11T07:39:57.772223+0000","last_became_peered":"2024-09-11T07:39:57.772223+0000","last_unstale":"2024-09-11T07:45:56.754424+0000","last_undegraded":"2024-09-11T07:45:56.754424+0000","last_fullsized":"2024-09-11T07:45:56.754424+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:58.208726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:43:58.208726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002370193,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:57.199693+0000","last_change":"2024-09-11T07:45:57.199407+0000","last_active":"2024-09-11T07:45:57.199693+0000","last_peered":"2024-09-11T07:45:57.199693+0000","last_clean":"2024-09-11T07:45:57.199693+0000","last_became_active":"2024-09-11T07:45:57.199102+0000","last_became_peered":"2024-09-11T07:45:57.199102+0000","last_unstale":"2024-09-11T07:45:57.199693+0000","last_undegraded":"2024-09-11T07:45:57.199693+0000","last_fullsized":"2024-09-11T07:45:57.199693+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:17.282566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"399'33","reported_seq":137,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.752144+0000","last_change":"2024-09-11T07:45:32.442239+0000","last_active":"2024-09-11T07:45:56.752144+0000","last_peered":"2024-09-11T07:45:56.752144+0000","last_clean":"2024-09-11T07:45:56.752144+0000","last_became_active":"2024-09-11T07:45:27.113188+0000","last_became_peered":"2024-09-11T07:45:27.113188+0000","last_unstale":"2024-09-11T07:45:56.752144+0000","last_undegraded":"2024-09-11T07:45:56.752144+0000","last_fullsized":"2024-09-11T07:45:56.752144+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"399'33","last_scrub_stamp":"2024-09-11T07:45:32.442160+0000","last_deep_scrub":"399'33","last_deep_scrub_stamp":"2024-09-11T07:45:32.442160+0000","last_clean_scrub_stamp":"2024-09-11T07:45:32.442160+0000","objects_scrubbed":7,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:52.336506+0000","scrub_duration":0.015982277,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"83'3","reported_seq":481,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.752985+0000","last_change":"2024-09-11T07:45:27.153261+0000","last_active":"2024-09-11T07:45:56.752985+0000","last_peered":"2024-09-11T07:45:56.752985+0000","last_clean":"2024-09-11T07:45:56.752985+0000","last_became_active":"2024-09-11T07:45:27.099666+0000","last_became_peered":"2024-09-11T07:45:27.099666+0000","last_unstale":"2024-09-11T07:45:56.752985+0000","last_undegraded":"2024-09-11T07:45:56.752985+0000","last_fullsized":"2024-09-11T07:45:56.752985+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-09-11T07:44:30.572101+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_clean_scrub_stamp":"2024-09-11T07:44:30.572101+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003584968,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":475,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.754651+0000","last_change":"2024-09-11T07:45:18.381847+0000","last_active":"2024-09-11T07:45:56.754651+0000","last_peered":"2024-09-11T07:45:56.754651+0000","last_clean":"2024-09-11T07:45:56.754651+0000","last_became_active":"2024-09-11T07:40:11.705286+0000","last_became_peered":"2024-09-11T07:40:11.705286+0000","last_unstale":"2024-09-11T07:45:56.754651+0000","last_undegraded":"2024-09-11T07:45:56.754651+0000","last_fullsized":"2024-09-11T07:45:56.754651+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:18.381749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:18.381749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:26.985723+0000","scrub_duration":0.025310684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"87'81","reported_seq":293,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.755323+0000","last_change":"2024-09-11T07:44:59.849392+0000","last_active":"2024-09-11T07:45:56.755323+0000","last_peered":"2024-09-11T07:45:56.755323+0000","last_clean":"2024-09-11T07:45:56.755323+0000","last_became_active":"2024-09-11T07:44:59.848971+0000","last_became_peered":"2024-09-11T07:44:59.848971+0000","last_unstale":"2024-09-11T07:45:56.755323+0000","last_undegraded":"2024-09-11T07:45:56.755323+0000","last_fullsized":"2024-09-11T07:45:56.755323+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:44:22.387527+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.387527+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015232676000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53669,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":271,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.762777+0000","last_change":"2024-09-11T07:45:22.667614+0000","last_active":"2024-09-11T07:45:56.762777+0000","last_peered":"2024-09-11T07:45:56.762777+0000","last_clean":"2024-09-11T07:45:56.762777+0000","last_became_active":"2024-09-11T07:39:57.748658+0000","last_became_peered":"2024-09-11T07:39:57.748658+0000","last_unstale":"2024-09-11T07:45:56.762777+0000","last_undegraded":"2024-09-11T07:45:56.762777+0000","last_fullsized":"2024-09-11T07:45:56.762777+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:22.667519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:45:22.667519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:36.493380+0000","scrub_duration":0.0046246309999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":452,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.754900+0000","last_change":"2024-09-11T07:45:27.103851+0000","last_active":"2024-09-11T07:45:56.754900+0000","last_peered":"2024-09-11T07:45:56.754900+0000","last_clean":"2024-09-11T07:45:56.754900+0000","last_became_active":"2024-09-11T07:45:27.103334+0000","last_became_peered":"2024-09-11T07:45:27.103334+0000","last_unstale":"2024-09-11T07:45:56.754900+0000","last_undegraded":"2024-09-11T07:45:56.754900+0000","last_fullsized":"2024-09-11T07:45:56.754900+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:14.389468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:45:14.389468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:34.657343+0000","scrub_duration":0.005315012,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:57.199700+0000","last_change":"2024-09-11T07:45:57.199301+0000","last_active":"2024-09-11T07:45:57.199700+0000","last_peered":"2024-09-11T07:45:57.199700+0000","last_clean":"2024-09-11T07:45:57.199700+0000","last_became_active":"2024-09-11T07:45:57.199035+0000","last_became_peered":"2024-09-11T07:45:57.199035+0000","last_unstale":"2024-09-11T07:45:57.199700+0000","last_undegraded":"2024-09-11T07:45:57.199700+0000","last_fullsized":"2024-09-11T07:45:57.199700+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:09.238194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"234'35","reported_seq":212,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.754973+0000","last_change":"2024-09-11T07:44:32.236043+0000","last_active":"2024-09-11T07:45:56.754973+0000","last_peered":"2024-09-11T07:45:56.754973+0000","last_clean":"2024-09-11T07:45:56.754973+0000","last_became_active":"2024-09-11T07:44:23.768296+0000","last_became_peered":"2024-09-11T07:44:23.768296+0000","last_unstale":"2024-09-11T07:45:56.754973+0000","last_undegraded":"2024-09-11T07:45:56.754973+0000","last_fullsized":"2024-09-11T07:45:56.754973+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'35","last_scrub_stamp":"2024-09-11T07:44:32.235984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:44:32.235984+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:45:58.576773+0000","scrub_duration":0.0066145329999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":467,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.762843+0000","last_change":"2024-09-11T07:45:10.747839+0000","last_active":"2024-09-11T07:45:56.762843+0000","last_peered":"2024-09-11T07:45:56.762843+0000","last_clean":"2024-09-11T07:45:56.762843+0000","last_became_active":"2024-09-11T07:26:31.419560+0000","last_became_peered":"2024-09-11T07:26:31.419560+0000","last_unstale":"2024-09-11T07:45:56.762843+0000","last_undegraded":"2024-09-11T07:45:56.762843+0000","last_fullsized":"2024-09-11T07:45:56.762843+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:10.747787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:10.747787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:39.300259+0000","scrub_duration":0.0014756929999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":464,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.759702+0000","last_change":"2024-09-11T07:44:23.485237+0000","last_active":"2024-09-11T07:45:56.759702+0000","last_peered":"2024-09-11T07:45:56.759702+0000","last_clean":"2024-09-11T07:45:56.759702+0000","last_became_active":"2024-09-11T07:39:57.748953+0000","last_became_peered":"2024-09-11T07:39:57.748953+0000","last_unstale":"2024-09-11T07:45:56.759702+0000","last_undegraded":"2024-09-11T07:45:56.759702+0000","last_fullsized":"2024-09-11T07:45:56.759702+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:23.485176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:44:23.485176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015945569999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":269,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.759862+0000","last_change":"2024-09-11T07:44:00.601823+0000","last_active":"2024-09-11T07:45:56.759862+0000","last_peered":"2024-09-11T07:45:56.759862+0000","last_clean":"2024-09-11T07:45:56.759862+0000","last_became_active":"2024-09-11T07:39:57.766605+0000","last_became_peered":"2024-09-11T07:39:57.766605+0000","last_unstale":"2024-09-11T07:45:56.759862+0000","last_undegraded":"2024-09-11T07:45:56.759862+0000","last_fullsized":"2024-09-11T07:45:56.759862+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:00.601321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:44:00.601321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.009846334,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:57.201792+0000","last_change":"2024-09-11T07:45:57.201461+0000","last_active":"2024-09-11T07:45:57.201792+0000","last_peered":"2024-09-11T07:45:57.201792+0000","last_clean":"2024-09-11T07:45:57.201792+0000","last_became_active":"2024-09-11T07:45:57.201054+0000","last_became_peered":"2024-09-11T07:45:57.201054+0000","last_unstale":"2024-09-11T07:45:57.201792+0000","last_undegraded":"2024-09-11T07:45:57.201792+0000","last_fullsized":"2024-09-11T07:45:57.201792+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:01.222410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"234'15","reported_seq":488,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.753774+0000","last_change":"2024-09-11T07:45:27.145068+0000","last_active":"2024-09-11T07:45:56.753774+0000","last_peered":"2024-09-11T07:45:56.753774+0000","last_clean":"2024-09-11T07:45:56.753774+0000","last_became_active":"2024-09-11T07:45:27.114646+0000","last_became_peered":"2024-09-11T07:45:27.114646+0000","last_unstale":"2024-09-11T07:45:56.753774+0000","last_undegraded":"2024-09-11T07:45:56.753774+0000","last_fullsized":"2024-09-11T07:45:56.753774+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:45:11.521203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:45:11.521203+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:24.236351+0000","scrub_duration":0.015327073,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":36,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.753279+0000","last_change":"2024-09-11T07:45:27.102638+0000","last_active":"2024-09-11T07:45:56.753279+0000","last_peered":"2024-09-11T07:45:56.753279+0000","last_clean":"2024-09-11T07:45:56.753279+0000","last_became_active":"2024-09-11T07:45:27.102350+0000","last_became_peered":"2024-09-11T07:45:27.102350+0000","last_unstale":"2024-09-11T07:45:56.753279+0000","last_undegraded":"2024-09-11T07:45:56.753279+0000","last_fullsized":"2024-09-11T07:45:56.753279+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:16.532601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:43:16.532601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":470,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.758120+0000","last_change":"2024-09-11T07:45:27.123255+0000","last_active":"2024-09-11T07:45:56.758120+0000","last_peered":"2024-09-11T07:45:56.758120+0000","last_clean":"2024-09-11T07:45:56.758120+0000","last_became_active":"2024-09-11T07:45:27.122533+0000","last_became_peered":"2024-09-11T07:45:27.122533+0000","last_unstale":"2024-09-11T07:45:56.758120+0000","last_undegraded":"2024-09-11T07:45:56.758120+0000","last_fullsized":"2024-09-11T07:45:56.758120+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:08.436552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:44:08.436552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0064625339999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":456,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.756087+0000","last_change":"2024-09-11T07:45:22.489678+0000","last_active":"2024-09-11T07:45:56.756087+0000","last_peered":"2024-09-11T07:45:56.756087+0000","last_clean":"2024-09-11T07:45:56.756087+0000","last_became_active":"2024-09-11T07:32:00.505185+0000","last_became_peered":"2024-09-11T07:32:00.505185+0000","last_unstale":"2024-09-11T07:45:56.756087+0000","last_undegraded":"2024-09-11T07:45:56.756087+0000","last_fullsized":"2024-09-11T07:45:56.756087+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:22.489595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:22.489595+0000","last_clean_scrub_stamp":"2024-09-11T07:45:22.489595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:26.324373+0000","scrub_duration":0.045984707999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.765321+0000","last_change":"2024-09-11T07:45:56.764280+0000","last_active":"2024-09-11T07:45:56.765321+0000","last_peered":"2024-09-11T07:45:56.765321+0000","last_clean":"2024-09-11T07:45:56.765321+0000","last_became_active":"2024-09-11T07:45:56.763089+0000","last_became_peered":"2024-09-11T07:45:56.763089+0000","last_unstale":"2024-09-11T07:45:56.765321+0000","last_undegraded":"2024-09-11T07:45:56.765321+0000","last_fullsized":"2024-09-11T07:45:56.765321+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:14.646257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"234'27","reported_seq":49,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.755699+0000","last_change":"2024-09-11T07:45:27.160003+0000","last_active":"2024-09-11T07:45:56.755699+0000","last_peered":"2024-09-11T07:45:56.755699+0000","last_clean":"2024-09-11T07:45:56.755699+0000","last_became_active":"2024-09-11T07:45:27.099653+0000","last_became_peered":"2024-09-11T07:45:27.099653+0000","last_unstale":"2024-09-11T07:45:56.755699+0000","last_undegraded":"2024-09-11T07:45:56.755699+0000","last_fullsized":"2024-09-11T07:45:56.755699+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'27","last_scrub_stamp":"2024-09-11T07:43:22.300979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:43:22.300979+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043984430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":502,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.760857+0000","last_change":"2024-09-11T07:44:25.455385+0000","last_active":"2024-09-11T07:45:56.760857+0000","last_peered":"2024-09-11T07:45:56.760857+0000","last_clean":"2024-09-11T07:45:56.760857+0000","last_became_active":"2024-09-11T07:40:11.701330+0000","last_became_peered":"2024-09-11T07:40:11.701330+0000","last_unstale":"2024-09-11T07:45:56.760857+0000","last_undegraded":"2024-09-11T07:45:56.760857+0000","last_fullsized":"2024-09-11T07:45:56.760857+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:25.455028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_clean_scrub_stamp":"2024-09-11T07:44:25.455028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044567610000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"395'11","reported_seq":648,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.751392+0000","last_change":"2024-09-11T07:45:27.105103+0000","last_active":"2024-09-11T07:45:56.751392+0000","last_peered":"2024-09-11T07:45:56.751392+0000","last_clean":"2024-09-11T07:45:56.751392+0000","last_became_active":"2024-09-11T07:45:27.093333+0000","last_became_peered":"2024-09-11T07:45:27.093333+0000","last_unstale":"2024-09-11T07:45:56.751392+0000","last_undegraded":"2024-09-11T07:45:56.751392+0000","last_fullsized":"2024-09-11T07:45:56.751392+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'10","last_scrub_stamp":"2024-09-11T07:43:21.597088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:43:21.597088+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079746980000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":100,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.755475+0000","last_change":"2024-09-11T07:45:26.475689+0000","last_active":"2024-09-11T07:45:56.755475+0000","last_peered":"2024-09-11T07:45:56.755475+0000","last_clean":"2024-09-11T07:45:56.755475+0000","last_became_active":"2024-09-11T07:42:04.729103+0000","last_became_peered":"2024-09-11T07:42:04.729103+0000","last_unstale":"2024-09-11T07:45:56.755475+0000","last_undegraded":"2024-09-11T07:45:56.755475+0000","last_fullsized":"2024-09-11T07:45:56.755475+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:26.475614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:45:26.475614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:44.947478+0000","scrub_duration":0.001709363,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:57.199767+0000","last_change":"2024-09-11T07:45:57.199112+0000","last_active":"2024-09-11T07:45:57.199767+0000","last_peered":"2024-09-11T07:45:57.199767+0000","last_clean":"2024-09-11T07:45:57.199767+0000","last_became_active":"2024-09-11T07:45:57.198860+0000","last_became_peered":"2024-09-11T07:45:57.198860+0000","last_unstale":"2024-09-11T07:45:57.199767+0000","last_undegraded":"2024-09-11T07:45:57.199767+0000","last_fullsized":"2024-09-11T07:45:57.199767+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:24.428389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"234'23","reported_seq":41,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.758938+0000","last_change":"2024-09-11T07:45:27.150010+0000","last_active":"2024-09-11T07:45:56.758938+0000","last_peered":"2024-09-11T07:45:56.758938+0000","last_clean":"2024-09-11T07:45:56.758938+0000","last_became_active":"2024-09-11T07:45:27.118858+0000","last_became_peered":"2024-09-11T07:45:27.118858+0000","last_unstale":"2024-09-11T07:45:56.758938+0000","last_undegraded":"2024-09-11T07:45:56.758938+0000","last_fullsized":"2024-09-11T07:45:56.758938+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'23","last_scrub_stamp":"2024-09-11T07:43:15.318542+0000","last_deep_scrub":"234'23","last_deep_scrub_stamp":"2024-09-11T07:37:34.613280+0000","last_clean_scrub_stamp":"2024-09-11T07:43:15.318542+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054521589999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":47,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.757960+0000","last_change":"2024-09-11T07:45:27.125454+0000","last_active":"2024-09-11T07:45:56.757960+0000","last_peered":"2024-09-11T07:45:56.757960+0000","last_clean":"2024-09-11T07:45:56.757960+0000","last_became_active":"2024-09-11T07:45:27.112394+0000","last_became_peered":"2024-09-11T07:45:27.112394+0000","last_unstale":"2024-09-11T07:45:56.757960+0000","last_undegraded":"2024-09-11T07:45:56.757960+0000","last_fullsized":"2024-09-11T07:45:56.757960+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:45:08.089525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:08.089525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:19.206584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":460,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.760004+0000","last_change":"2024-09-11T07:44:28.572130+0000","last_active":"2024-09-11T07:45:56.760004+0000","last_peered":"2024-09-11T07:45:56.760004+0000","last_clean":"2024-09-11T07:45:56.760004+0000","last_became_active":"2024-09-11T07:26:35.561967+0000","last_became_peered":"2024-09-11T07:26:35.561967+0000","last_unstale":"2024-09-11T07:45:56.760004+0000","last_undegraded":"2024-09-11T07:45:56.760004+0000","last_fullsized":"2024-09-11T07:45:56.760004+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:28.572059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:44.820354+0000","last_clean_scrub_stamp":"2024-09-11T07:44:28.572059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003042159,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":466,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.756781+0000","last_change":"2024-09-11T07:44:27.626846+0000","last_active":"2024-09-11T07:45:56.756781+0000","last_peered":"2024-09-11T07:45:56.756781+0000","last_clean":"2024-09-11T07:45:56.756781+0000","last_became_active":"2024-09-11T07:39:57.776576+0000","last_became_peered":"2024-09-11T07:39:57.776576+0000","last_unstale":"2024-09-11T07:45:56.756781+0000","last_undegraded":"2024-09-11T07:45:56.756781+0000","last_fullsized":"2024-09-11T07:45:56.756781+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:27.626761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:27.626761+0000","last_clean_scrub_stamp":"2024-09-11T07:44:27.626761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025844079999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"381'19","reported_seq":509,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.754012+0000","last_change":"2024-09-11T07:44:30.282613+0000","last_active":"2024-09-11T07:45:56.754012+0000","last_peered":"2024-09-11T07:45:56.754012+0000","last_clean":"2024-09-11T07:45:56.754012+0000","last_became_active":"2024-09-11T07:39:57.767661+0000","last_became_peered":"2024-09-11T07:39:57.767661+0000","last_unstale":"2024-09-11T07:45:56.754012+0000","last_undegraded":"2024-09-11T07:45:56.754012+0000","last_fullsized":"2024-09-11T07:45:56.754012+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"381'19","last_scrub_stamp":"2024-09-11T07:44:30.282534+0000","last_deep_scrub":"381'19","last_deep_scrub_stamp":"2024-09-11T07:44:30.282534+0000","last_clean_scrub_stamp":"2024-09-11T07:44:30.282534+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017437478999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":464,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.760570+0000","last_change":"2024-09-11T07:43:26.338301+0000","last_active":"2024-09-11T07:45:56.760570+0000","last_peered":"2024-09-11T07:45:56.760570+0000","last_clean":"2024-09-11T07:45:56.760570+0000","last_became_active":"2024-09-11T07:26:31.415701+0000","last_became_peered":"2024-09-11T07:26:31.415701+0000","last_unstale":"2024-09-11T07:45:56.760570+0000","last_undegraded":"2024-09-11T07:45:56.760570+0000","last_fullsized":"2024-09-11T07:45:56.760570+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:26.338227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:43:26.338227+0000","last_clean_scrub_stamp":"2024-09-11T07:43:26.338227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001220877,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":185,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.752201+0000","last_change":"2024-09-11T07:44:29.396082+0000","last_active":"2024-09-11T07:45:56.752201+0000","last_peered":"2024-09-11T07:45:56.752201+0000","last_clean":"2024-09-11T07:45:56.752201+0000","last_became_active":"2024-09-11T07:44:23.768703+0000","last_became_peered":"2024-09-11T07:44:23.768703+0000","last_unstale":"2024-09-11T07:45:56.752201+0000","last_undegraded":"2024-09-11T07:45:56.752201+0000","last_fullsized":"2024-09-11T07:45:56.752201+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:29.395994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:35.392360+0000","last_clean_scrub_stamp":"2024-09-11T07:44:29.395994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045575399999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":481,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.762262+0000","last_change":"2024-09-11T07:45:27.129839+0000","last_active":"2024-09-11T07:45:56.762262+0000","last_peered":"2024-09-11T07:45:56.762262+0000","last_clean":"2024-09-11T07:45:56.762262+0000","last_became_active":"2024-09-11T07:45:27.124181+0000","last_became_peered":"2024-09-11T07:45:27.124181+0000","last_unstale":"2024-09-11T07:45:56.762262+0000","last_undegraded":"2024-09-11T07:45:56.762262+0000","last_fullsized":"2024-09-11T07:45:56.762262+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:45:18.615900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:45:18.615900+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:35.505236+0000","scrub_duration":0.005478391,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":256,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:57.199222+0000","last_change":"2024-09-11T07:45:27.091379+0000","last_active":"2024-09-11T07:45:57.199222+0000","last_peered":"2024-09-11T07:45:57.199222+0000","last_clean":"2024-09-11T07:45:57.199222+0000","last_became_active":"2024-09-11T07:45:27.091114+0000","last_became_peered":"2024-09-11T07:45:27.091114+0000","last_unstale":"2024-09-11T07:45:57.199222+0000","last_undegraded":"2024-09-11T07:45:57.199222+0000","last_fullsized":"2024-09-11T07:45:57.199222+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:15.127360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:44:15.127360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"234'15","reported_seq":99,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.762373+0000","last_change":"2024-09-11T07:45:19.660354+0000","last_active":"2024-09-11T07:45:56.762373+0000","last_peered":"2024-09-11T07:45:56.762373+0000","last_clean":"2024-09-11T07:45:56.762373+0000","last_became_active":"2024-09-11T07:44:23.769896+0000","last_became_peered":"2024-09-11T07:44:23.769896+0000","last_unstale":"2024-09-11T07:45:56.762373+0000","last_undegraded":"2024-09-11T07:45:56.762373+0000","last_fullsized":"2024-09-11T07:45:56.762373+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:45:19.660273+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:45:19.660273+0000","objects_scrubbed":3,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:30.218706+0000","scrub_duration":0.0032335910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":451,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.754801+0000","last_change":"2024-09-11T07:44:04.279671+0000","last_active":"2024-09-11T07:45:56.754801+0000","last_peered":"2024-09-11T07:45:56.754801+0000","last_clean":"2024-09-11T07:45:56.754801+0000","last_became_active":"2024-09-11T07:26:35.566894+0000","last_became_peered":"2024-09-11T07:26:35.566894+0000","last_unstale":"2024-09-11T07:45:56.754801+0000","last_undegraded":"2024-09-11T07:45:56.754801+0000","last_fullsized":"2024-09-11T07:45:56.754801+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:04.279588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:44:04.279588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069828010000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":447,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.752053+0000","last_change":"2024-09-11T07:45:27.113998+0000","last_active":"2024-09-11T07:45:56.752053+0000","last_peered":"2024-09-11T07:45:56.752053+0000","last_clean":"2024-09-11T07:45:56.752053+0000","last_became_active":"2024-09-11T07:45:27.112711+0000","last_became_peered":"2024-09-11T07:45:27.112711+0000","last_unstale":"2024-09-11T07:45:56.752053+0000","last_undegraded":"2024-09-11T07:45:56.752053+0000","last_fullsized":"2024-09-11T07:45:56.752053+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:26.403212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:44:26.403212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001222218,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"234'25","reported_seq":517,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.756067+0000","last_change":"2024-09-11T07:44:29.610618+0000","last_active":"2024-09-11T07:45:56.756067+0000","last_peered":"2024-09-11T07:45:56.756067+0000","last_clean":"2024-09-11T07:45:56.756067+0000","last_became_active":"2024-09-11T07:39:57.771009+0000","last_became_peered":"2024-09-11T07:39:57.771009+0000","last_unstale":"2024-09-11T07:45:56.756067+0000","last_undegraded":"2024-09-11T07:45:56.756067+0000","last_fullsized":"2024-09-11T07:45:56.756067+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'25","last_scrub_stamp":"2024-09-11T07:44:29.610392+0000","last_deep_scrub":"234'25","last_deep_scrub_stamp":"2024-09-11T07:44:29.610392+0000","last_clean_scrub_stamp":"2024-09-11T07:44:29.610392+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005704598,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":465,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.754893+0000","last_change":"2024-09-11T07:44:01.220549+0000","last_active":"2024-09-11T07:45:56.754893+0000","last_peered":"2024-09-11T07:45:56.754893+0000","last_clean":"2024-09-11T07:45:56.754893+0000","last_became_active":"2024-09-11T07:39:57.764063+0000","last_became_peered":"2024-09-11T07:39:57.764063+0000","last_unstale":"2024-09-11T07:45:56.754893+0000","last_undegraded":"2024-09-11T07:45:56.754893+0000","last_fullsized":"2024-09-11T07:45:56.754893+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:01.220444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:39:54.493614+0000","last_clean_scrub_stamp":"2024-09-11T07:44:01.220444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001943183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":682,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.754612+0000","last_change":"2024-09-11T07:44:13.339230+0000","last_active":"2024-09-11T07:45:56.754612+0000","last_peered":"2024-09-11T07:45:56.754612+0000","last_clean":"2024-09-11T07:45:56.754612+0000","last_became_active":"2024-09-11T07:26:35.562245+0000","last_became_peered":"2024-09-11T07:26:35.562245+0000","last_unstale":"2024-09-11T07:45:56.754612+0000","last_undegraded":"2024-09-11T07:45:56.754612+0000","last_fullsized":"2024-09-11T07:45:56.754612+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:44:13.339137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:44:13.339137+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023674870000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":476,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.753207+0000","last_change":"2024-09-11T07:45:27.124686+0000","last_active":"2024-09-11T07:45:56.753207+0000","last_peered":"2024-09-11T07:45:56.753207+0000","last_clean":"2024-09-11T07:45:56.753207+0000","last_became_active":"2024-09-11T07:45:27.123325+0000","last_became_peered":"2024-09-11T07:45:27.123325+0000","last_unstale":"2024-09-11T07:45:56.753207+0000","last_undegraded":"2024-09-11T07:45:56.753207+0000","last_fullsized":"2024-09-11T07:45:56.753207+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:03.324782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:44:03.324782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059257859999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"234'19","reported_seq":459,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.751835+0000","last_change":"2024-09-11T07:44:00.397238+0000","last_active":"2024-09-11T07:45:56.751835+0000","last_peered":"2024-09-11T07:45:56.751835+0000","last_clean":"2024-09-11T07:45:56.751835+0000","last_became_active":"2024-09-11T07:43:10.448494+0000","last_became_peered":"2024-09-11T07:43:10.448494+0000","last_unstale":"2024-09-11T07:45:56.751835+0000","last_undegraded":"2024-09-11T07:45:56.751835+0000","last_fullsized":"2024-09-11T07:45:56.751835+0000","mapping_epoch":360,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":361,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:44:00.397157+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:37:41.338036+0000","last_clean_scrub_stamp":"2024-09-11T07:44:00.397157+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072446960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":458,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.754020+0000","last_change":"2024-09-11T07:45:27.117986+0000","last_active":"2024-09-11T07:45:56.754020+0000","last_peered":"2024-09-11T07:45:56.754020+0000","last_clean":"2024-09-11T07:45:56.754020+0000","last_became_active":"2024-09-11T07:45:27.114097+0000","last_became_peered":"2024-09-11T07:45:27.114097+0000","last_unstale":"2024-09-11T07:45:56.754020+0000","last_undegraded":"2024-09-11T07:45:56.754020+0000","last_fullsized":"2024-09-11T07:45:56.754020+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:09.537798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:09.537798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:18.428529+0000","scrub_duration":0.003422685,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"290'11","reported_seq":424,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.751931+0000","last_change":"2024-09-11T07:44:10.410699+0000","last_active":"2024-09-11T07:45:56.751931+0000","last_peered":"2024-09-11T07:45:56.751931+0000","last_clean":"2024-09-11T07:45:56.751931+0000","last_became_active":"2024-09-11T07:39:57.772790+0000","last_became_peered":"2024-09-11T07:39:57.772790+0000","last_unstale":"2024-09-11T07:45:56.751931+0000","last_undegraded":"2024-09-11T07:45:56.751931+0000","last_fullsized":"2024-09-11T07:45:56.751931+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"290'11","last_scrub_stamp":"2024-09-11T07:44:10.410600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:44:10.410600+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090484429999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":32,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.762253+0000","last_change":"2024-09-11T07:45:27.104143+0000","last_active":"2024-09-11T07:45:56.762253+0000","last_peered":"2024-09-11T07:45:56.762253+0000","last_clean":"2024-09-11T07:45:56.762253+0000","last_became_active":"2024-09-11T07:45:27.101913+0000","last_became_peered":"2024-09-11T07:45:27.101913+0000","last_unstale":"2024-09-11T07:45:56.762253+0000","last_undegraded":"2024-09-11T07:45:56.762253+0000","last_fullsized":"2024-09-11T07:45:56.762253+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:21.026931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:45:21.026931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:22.807067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"236'16","reported_seq":447,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.758218+0000","last_change":"2024-09-11T07:45:14.682735+0000","last_active":"2024-09-11T07:45:56.758218+0000","last_peered":"2024-09-11T07:45:56.758218+0000","last_clean":"2024-09-11T07:45:56.758218+0000","last_became_active":"2024-09-11T07:39:57.689424+0000","last_became_peered":"2024-09-11T07:39:57.689424+0000","last_unstale":"2024-09-11T07:45:56.758218+0000","last_undegraded":"2024-09-11T07:45:56.758218+0000","last_fullsized":"2024-09-11T07:45:56.758218+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'16","last_scrub_stamp":"2024-09-11T07:45:14.682638+0000","last_deep_scrub":"236'16","last_deep_scrub_stamp":"2024-09-11T07:36:49.809484+0000","last_clean_scrub_stamp":"2024-09-11T07:45:14.682638+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:22.685967+0000","scrub_duration":0.014584720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":251,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.756150+0000","last_change":"2024-09-11T07:45:08.577358+0000","last_active":"2024-09-11T07:45:56.756150+0000","last_peered":"2024-09-11T07:45:56.756150+0000","last_clean":"2024-09-11T07:45:56.756150+0000","last_became_active":"2024-09-11T07:39:57.776841+0000","last_became_peered":"2024-09-11T07:39:57.776841+0000","last_unstale":"2024-09-11T07:45:56.756150+0000","last_undegraded":"2024-09-11T07:45:56.756150+0000","last_fullsized":"2024-09-11T07:45:56.756150+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:08.577262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:38.721118+0000","last_clean_scrub_stamp":"2024-09-11T07:45:08.577262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:32.896318+0000","scrub_duration":0.0073986299999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":437,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.756905+0000","last_change":"2024-09-11T07:44:15.612662+0000","last_active":"2024-09-11T07:45:56.756905+0000","last_peered":"2024-09-11T07:45:56.756905+0000","last_clean":"2024-09-11T07:45:56.756905+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:45:56.756905+0000","last_undegraded":"2024-09-11T07:45:56.756905+0000","last_fullsized":"2024-09-11T07:45:56.756905+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:15.612580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:44:15.612580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001412515,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":449,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.754692+0000","last_change":"2024-09-11T07:43:18.311031+0000","last_active":"2024-09-11T07:45:56.754692+0000","last_peered":"2024-09-11T07:45:56.754692+0000","last_clean":"2024-09-11T07:45:56.754692+0000","last_became_active":"2024-09-11T07:39:57.761754+0000","last_became_peered":"2024-09-11T07:39:57.761754+0000","last_unstale":"2024-09-11T07:45:56.754692+0000","last_undegraded":"2024-09-11T07:45:56.754692+0000","last_fullsized":"2024-09-11T07:45:56.754692+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:18.310936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:43:18.310936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0094815590000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"234'24","reported_seq":41,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.754400+0000","last_change":"2024-09-11T07:45:27.115850+0000","last_active":"2024-09-11T07:45:56.754400+0000","last_peered":"2024-09-11T07:45:56.754400+0000","last_clean":"2024-09-11T07:45:56.754400+0000","last_became_active":"2024-09-11T07:45:27.096521+0000","last_became_peered":"2024-09-11T07:45:27.096521+0000","last_unstale":"2024-09-11T07:45:56.754400+0000","last_undegraded":"2024-09-11T07:45:56.754400+0000","last_fullsized":"2024-09-11T07:45:56.754400+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'24","last_scrub_stamp":"2024-09-11T07:44:32.227871+0000","last_deep_scrub":"234'24","last_deep_scrub_stamp":"2024-09-11T07:41:32.694510+0000","last_clean_scrub_stamp":"2024-09-11T07:44:32.227871+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048561150000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":57,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.754703+0000","last_change":"2024-09-11T07:45:09.333401+0000","last_active":"2024-09-11T07:45:56.754703+0000","last_peered":"2024-09-11T07:45:56.754703+0000","last_clean":"2024-09-11T07:45:56.754703+0000","last_became_active":"2024-09-11T07:44:23.772554+0000","last_became_peered":"2024-09-11T07:44:23.772554+0000","last_unstale":"2024-09-11T07:45:56.754703+0000","last_undegraded":"2024-09-11T07:45:56.754703+0000","last_fullsized":"2024-09-11T07:45:56.754703+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:09.333312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:45:09.333312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:30.544490+0000","scrub_duration":0.015903309000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":346,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.754731+0000","last_change":"2024-09-11T07:45:07.272118+0000","last_active":"2024-09-11T07:45:56.754731+0000","last_peered":"2024-09-11T07:45:56.754731+0000","last_clean":"2024-09-11T07:45:56.754731+0000","last_became_active":"2024-09-11T07:44:23.768376+0000","last_became_peered":"2024-09-11T07:44:23.768376+0000","last_unstale":"2024-09-11T07:45:56.754731+0000","last_undegraded":"2024-09-11T07:45:56.754731+0000","last_fullsized":"2024-09-11T07:45:56.754731+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:07.272030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:39:55.895853+0000","last_clean_scrub_stamp":"2024-09-11T07:45:07.272030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:10.586979+0000","scrub_duration":0.013154004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":178,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.756260+0000","last_change":"2024-09-11T07:45:24.519416+0000","last_active":"2024-09-11T07:45:56.756260+0000","last_peered":"2024-09-11T07:45:56.756260+0000","last_clean":"2024-09-11T07:45:56.756260+0000","last_became_active":"2024-09-11T07:40:11.700676+0000","last_became_peered":"2024-09-11T07:40:11.700676+0000","last_unstale":"2024-09-11T07:45:56.756260+0000","last_undegraded":"2024-09-11T07:45:56.756260+0000","last_fullsized":"2024-09-11T07:45:56.756260+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:24.519137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:45:24.519137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:24.555737+0000","scrub_duration":0.0077895680000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"234'10","reported_seq":457,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.755988+0000","last_change":"2024-09-11T07:44:16.603894+0000","last_active":"2024-09-11T07:45:56.755988+0000","last_peered":"2024-09-11T07:45:56.755988+0000","last_clean":"2024-09-11T07:45:56.755988+0000","last_became_active":"2024-09-11T07:26:33.545830+0000","last_became_peered":"2024-09-11T07:26:33.545830+0000","last_unstale":"2024-09-11T07:45:56.755988+0000","last_undegraded":"2024-09-11T07:45:56.755988+0000","last_fullsized":"2024-09-11T07:45:56.755988+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'10","last_scrub_stamp":"2024-09-11T07:44:16.603800+0000","last_deep_scrub":"234'10","last_deep_scrub_stamp":"2024-09-11T07:41:29.585284+0000","last_clean_scrub_stamp":"2024-09-11T07:44:16.603800+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047267960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":445,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.754806+0000","last_change":"2024-09-11T07:43:55.188214+0000","last_active":"2024-09-11T07:45:56.754806+0000","last_peered":"2024-09-11T07:45:56.754806+0000","last_clean":"2024-09-11T07:45:56.754806+0000","last_became_active":"2024-09-11T07:26:31.420518+0000","last_became_peered":"2024-09-11T07:26:31.420518+0000","last_unstale":"2024-09-11T07:45:56.754806+0000","last_undegraded":"2024-09-11T07:45:56.754806+0000","last_fullsized":"2024-09-11T07:45:56.754806+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:55.185252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_clean_scrub_stamp":"2024-09-11T07:43:55.185252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0055867790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":272,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.758452+0000","last_change":"2024-09-11T07:45:31.477617+0000","last_active":"2024-09-11T07:45:56.758452+0000","last_peered":"2024-09-11T07:45:56.758452+0000","last_clean":"2024-09-11T07:45:56.758452+0000","last_became_active":"2024-09-11T07:45:27.118170+0000","last_became_peered":"2024-09-11T07:45:27.118170+0000","last_unstale":"2024-09-11T07:45:56.758452+0000","last_undegraded":"2024-09-11T07:45:56.758452+0000","last_fullsized":"2024-09-11T07:45:56.758452+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:31.477552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:31.477552+0000","last_clean_scrub_stamp":"2024-09-11T07:45:31.477552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:50.533989+0000","scrub_duration":0.001479164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":449,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.751416+0000","last_change":"2024-09-11T07:45:27.098514+0000","last_active":"2024-09-11T07:45:56.751416+0000","last_peered":"2024-09-11T07:45:56.751416+0000","last_clean":"2024-09-11T07:45:56.751416+0000","last_became_active":"2024-09-11T07:45:27.097259+0000","last_became_peered":"2024-09-11T07:45:27.097259+0000","last_unstale":"2024-09-11T07:45:56.751416+0000","last_undegraded":"2024-09-11T07:45:56.751416+0000","last_fullsized":"2024-09-11T07:45:56.751416+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:13.460946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:44:13.460946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014486880000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"234'19","reported_seq":493,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.752801+0000","last_change":"2024-09-11T07:44:03.649070+0000","last_active":"2024-09-11T07:45:56.752801+0000","last_peered":"2024-09-11T07:45:56.752801+0000","last_clean":"2024-09-11T07:45:56.752801+0000","last_became_active":"2024-09-11T07:39:57.767928+0000","last_became_peered":"2024-09-11T07:39:57.767928+0000","last_unstale":"2024-09-11T07:45:56.752801+0000","last_undegraded":"2024-09-11T07:45:56.752801+0000","last_fullsized":"2024-09-11T07:45:56.752801+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:44:03.648975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:44:03.648975+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003981989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":231,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.755045+0000","last_change":"2024-09-11T07:44:11.315656+0000","last_active":"2024-09-11T07:45:56.755045+0000","last_peered":"2024-09-11T07:45:56.755045+0000","last_clean":"2024-09-11T07:45:56.755045+0000","last_became_active":"2024-09-11T07:36:05.228360+0000","last_became_peered":"2024-09-11T07:36:05.228360+0000","last_unstale":"2024-09-11T07:45:56.755045+0000","last_undegraded":"2024-09-11T07:45:56.755045+0000","last_fullsized":"2024-09-11T07:45:56.755045+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:11.315570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:48.366931+0000","last_clean_scrub_stamp":"2024-09-11T07:44:11.315570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0063871090000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":422,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.751216+0000","last_change":"2024-09-11T07:45:27.101549+0000","last_active":"2024-09-11T07:45:56.751216+0000","last_peered":"2024-09-11T07:45:56.751216+0000","last_clean":"2024-09-11T07:45:56.751216+0000","last_became_active":"2024-09-11T07:45:27.101251+0000","last_became_peered":"2024-09-11T07:45:27.101251+0000","last_unstale":"2024-09-11T07:45:56.751216+0000","last_undegraded":"2024-09-11T07:45:56.751216+0000","last_fullsized":"2024-09-11T07:45:56.751216+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:20.479924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:39.021876+0000","last_clean_scrub_stamp":"2024-09-11T07:44:20.479924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001690261,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":432,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.756369+0000","last_change":"2024-09-11T07:44:17.592717+0000","last_active":"2024-09-11T07:45:56.756369+0000","last_peered":"2024-09-11T07:45:56.756369+0000","last_clean":"2024-09-11T07:45:56.756369+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:45:56.756369+0000","last_undegraded":"2024-09-11T07:45:56.756369+0000","last_fullsized":"2024-09-11T07:45:56.756369+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:17.592633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:32.904006+0000","last_clean_scrub_stamp":"2024-09-11T07:44:17.592633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044439140000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":409,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.751770+0000","last_change":"2024-09-11T07:43:07.677815+0000","last_active":"2024-09-11T07:45:56.751770+0000","last_peered":"2024-09-11T07:45:56.751770+0000","last_clean":"2024-09-11T07:45:56.751770+0000","last_became_active":"2024-09-11T07:32:00.511258+0000","last_became_peered":"2024-09-11T07:32:00.511258+0000","last_unstale":"2024-09-11T07:45:56.751770+0000","last_undegraded":"2024-09-11T07:45:56.751770+0000","last_fullsized":"2024-09-11T07:45:56.751770+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:43:07.675565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:43:07.675565+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012571676,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":495,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.753928+0000","last_change":"2024-09-11T07:45:27.121627+0000","last_active":"2024-09-11T07:45:56.753928+0000","last_peered":"2024-09-11T07:45:56.753928+0000","last_clean":"2024-09-11T07:45:56.753928+0000","last_became_active":"2024-09-11T07:45:27.098795+0000","last_became_peered":"2024-09-11T07:45:27.098795+0000","last_unstale":"2024-09-11T07:45:56.753928+0000","last_undegraded":"2024-09-11T07:45:56.753928+0000","last_fullsized":"2024-09-11T07:45:56.753928+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:44:22.300957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.300957+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0068790429999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1748,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"234'20","reported_seq":56,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.753028+0000","last_change":"2024-09-11T07:45:30.511577+0000","last_active":"2024-09-11T07:45:56.753028+0000","last_peered":"2024-09-11T07:45:56.753028+0000","last_clean":"2024-09-11T07:45:56.753028+0000","last_became_active":"2024-09-11T07:45:27.122864+0000","last_became_peered":"2024-09-11T07:45:27.122864+0000","last_unstale":"2024-09-11T07:45:56.753028+0000","last_undegraded":"2024-09-11T07:45:56.753028+0000","last_fullsized":"2024-09-11T07:45:56.753028+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'20","last_scrub_stamp":"2024-09-11T07:45:30.511477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:45:30.511477+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:43.153986+0000","scrub_duration":0.001786167,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"395'13","reported_seq":211,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.762143+0000","last_change":"2024-09-11T07:45:31.645251+0000","last_active":"2024-09-11T07:45:56.762143+0000","last_peered":"2024-09-11T07:45:56.762143+0000","last_clean":"2024-09-11T07:45:56.762143+0000","last_became_active":"2024-09-11T07:45:27.105351+0000","last_became_peered":"2024-09-11T07:45:27.105351+0000","last_unstale":"2024-09-11T07:45:56.762143+0000","last_undegraded":"2024-09-11T07:45:56.762143+0000","last_fullsized":"2024-09-11T07:45:56.762143+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"395'13","last_scrub_stamp":"2024-09-11T07:45:31.645192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:45:31.645192+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:49.609737+0000","scrub_duration":0.0016838580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3],"acting":[2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":67,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.754654+0000","last_change":"2024-09-11T07:45:27.105050+0000","last_active":"2024-09-11T07:45:56.754654+0000","last_peered":"2024-09-11T07:45:56.754654+0000","last_clean":"2024-09-11T07:45:56.754654+0000","last_became_active":"2024-09-11T07:45:27.104499+0000","last_became_peered":"2024-09-11T07:45:27.104499+0000","last_unstale":"2024-09-11T07:45:56.754654+0000","last_undegraded":"2024-09-11T07:45:56.754654+0000","last_fullsized":"2024-09-11T07:45:56.754654+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:28.236557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:53.875300+0000","last_clean_scrub_stamp":"2024-09-11T07:44:28.236557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001550165,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":448,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.753060+0000","last_change":"2024-09-11T07:44:01.714689+0000","last_active":"2024-09-11T07:45:56.753060+0000","last_peered":"2024-09-11T07:45:56.753060+0000","last_clean":"2024-09-11T07:45:56.753060+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:45:56.753060+0000","last_undegraded":"2024-09-11T07:45:56.753060+0000","last_fullsized":"2024-09-11T07:45:56.753060+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:01.714607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:44:01.714607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079815600000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"234'19","reported_seq":120,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:57.200222+0000","last_change":"2024-09-11T07:45:23.537130+0000","last_active":"2024-09-11T07:45:57.200222+0000","last_peered":"2024-09-11T07:45:57.200222+0000","last_clean":"2024-09-11T07:45:57.200222+0000","last_became_active":"2024-09-11T07:44:23.773794+0000","last_became_peered":"2024-09-11T07:44:23.773794+0000","last_unstale":"2024-09-11T07:45:57.200222+0000","last_undegraded":"2024-09-11T07:45:57.200222+0000","last_fullsized":"2024-09-11T07:45:57.200222+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:45:23.537025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:45:23.537025+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:33.535474+0000","scrub_duration":0.005335724,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":23,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":448,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.753368+0000","last_change":"2024-09-11T07:43:24.330744+0000","last_active":"2024-09-11T07:45:56.753368+0000","last_peered":"2024-09-11T07:45:56.753368+0000","last_clean":"2024-09-11T07:45:56.753368+0000","last_became_active":"2024-09-11T07:32:00.507050+0000","last_became_peered":"2024-09-11T07:32:00.507050+0000","last_unstale":"2024-09-11T07:45:56.753368+0000","last_undegraded":"2024-09-11T07:45:56.753368+0000","last_fullsized":"2024-09-11T07:45:56.753368+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:24.330520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:24.575210+0000","last_clean_scrub_stamp":"2024-09-11T07:43:24.330520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031611009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":29,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.754994+0000","last_change":"2024-09-11T07:45:27.101508+0000","last_active":"2024-09-11T07:45:56.754994+0000","last_peered":"2024-09-11T07:45:56.754994+0000","last_clean":"2024-09-11T07:45:56.754994+0000","last_became_active":"2024-09-11T07:45:27.100903+0000","last_became_peered":"2024-09-11T07:45:27.100903+0000","last_unstale":"2024-09-11T07:45:56.754994+0000","last_undegraded":"2024-09-11T07:45:56.754994+0000","last_fullsized":"2024-09-11T07:45:56.754994+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:42:33.597580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:42:33.597580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002999842,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":174,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.751520+0000","last_change":"2024-09-11T07:45:27.098312+0000","last_active":"2024-09-11T07:45:56.751520+0000","last_peered":"2024-09-11T07:45:56.751520+0000","last_clean":"2024-09-11T07:45:56.751520+0000","last_became_active":"2024-09-11T07:45:27.096591+0000","last_became_peered":"2024-09-11T07:45:27.096591+0000","last_unstale":"2024-09-11T07:45:56.751520+0000","last_undegraded":"2024-09-11T07:45:56.751520+0000","last_fullsized":"2024-09-11T07:45:56.751520+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:08.381158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:43:08.381158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001295526,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1c","version":"234'27","reported_seq":502,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.762561+0000","last_change":"2024-09-11T07:45:13.715138+0000","last_active":"2024-09-11T07:45:56.762561+0000","last_peered":"2024-09-11T07:45:56.762561+0000","last_clean":"2024-09-11T07:45:56.762561+0000","last_became_active":"2024-09-11T07:26:33.542544+0000","last_became_peered":"2024-09-11T07:26:33.542544+0000","last_unstale":"2024-09-11T07:45:56.762561+0000","last_undegraded":"2024-09-11T07:45:56.762561+0000","last_fullsized":"2024-09-11T07:45:56.762561+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'27","last_scrub_stamp":"2024-09-11T07:45:13.715050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:45:13.715050+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:35.546034+0000","scrub_duration":0.0047085019999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":46,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.757098+0000","last_change":"2024-09-11T07:45:34.587277+0000","last_active":"2024-09-11T07:45:56.757098+0000","last_peered":"2024-09-11T07:45:56.757098+0000","last_clean":"2024-09-11T07:45:56.757098+0000","last_became_active":"2024-09-11T07:45:27.119977+0000","last_became_peered":"2024-09-11T07:45:27.119977+0000","last_unstale":"2024-09-11T07:45:56.757098+0000","last_undegraded":"2024-09-11T07:45:56.757098+0000","last_fullsized":"2024-09-11T07:45:56.757098+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:34.587193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:32.456852+0000","last_clean_scrub_stamp":"2024-09-11T07:45:34.587193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:01.619800+0000","scrub_duration":0.0049811509999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":439,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.753662+0000","last_change":"2024-09-11T07:44:00.677726+0000","last_active":"2024-09-11T07:45:56.753662+0000","last_peered":"2024-09-11T07:45:56.753662+0000","last_clean":"2024-09-11T07:45:56.753662+0000","last_became_active":"2024-09-11T07:39:57.768994+0000","last_became_peered":"2024-09-11T07:39:57.768994+0000","last_unstale":"2024-09-11T07:45:56.753662+0000","last_undegraded":"2024-09-11T07:45:56.753662+0000","last_fullsized":"2024-09-11T07:45:56.753662+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:00.677444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:44:00.677444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030734109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":472,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.757023+0000","last_change":"2024-09-11T07:45:27.102339+0000","last_active":"2024-09-11T07:45:56.757023+0000","last_peered":"2024-09-11T07:45:56.757023+0000","last_clean":"2024-09-11T07:45:56.757023+0000","last_became_active":"2024-09-11T07:45:27.101964+0000","last_became_peered":"2024-09-11T07:45:27.101964+0000","last_unstale":"2024-09-11T07:45:56.757023+0000","last_undegraded":"2024-09-11T07:45:56.757023+0000","last_fullsized":"2024-09-11T07:45:56.757023+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:55.705240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:43:55.705240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048801449999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"234'23","reported_seq":502,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.753201+0000","last_change":"2024-09-11T07:45:27.139564+0000","last_active":"2024-09-11T07:45:56.753201+0000","last_peered":"2024-09-11T07:45:56.753201+0000","last_clean":"2024-09-11T07:45:56.753201+0000","last_became_active":"2024-09-11T07:45:27.095974+0000","last_became_peered":"2024-09-11T07:45:27.095974+0000","last_unstale":"2024-09-11T07:45:56.753201+0000","last_undegraded":"2024-09-11T07:45:56.753201+0000","last_fullsized":"2024-09-11T07:45:56.753201+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'23","last_scrub_stamp":"2024-09-11T07:45:24.422401+0000","last_deep_scrub":"234'23","last_deep_scrub_stamp":"2024-09-11T07:43:25.530826+0000","last_clean_scrub_stamp":"2024-09-11T07:45:24.422401+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:50.733770+0000","scrub_duration":0.0068140780000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":32,"reported_epoch":413,"state":"active+clean","last_fresh":"2024-09-11T07:45:56.755781+0000","last_change":"2024-09-11T07:45:27.104898+0000","last_active":"2024-09-11T07:45:56.755781+0000","last_peered":"2024-09-11T07:45:56.755781+0000","last_clean":"2024-09-11T07:45:56.755781+0000","last_became_active":"2024-09-11T07:45:27.104493+0000","last_became_peered":"2024-09-11T07:45:27.104493+0000","last_unstale":"2024-09-11T07:45:56.755781+0000","last_undegraded":"2024-09-11T07:45:56.755781+0000","last_fullsized":"2024-09-11T07:45:56.755781+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:17.114312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:44:17.114312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":48,"acting":48,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1107,"num_read_kb":864,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":3290,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":475136,"data_stored":12690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":776,"ondisk_log_size":776,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":97,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":2045114,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2109440,"data_stored":2098784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":891,"ondisk_log_size":891,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":147,"seq":631360192681,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":924584,"kb_used_data":1048,"kb_used_omap":4,"kb_used_meta":923515,"kb_avail":92820568,"statfs":{"total":95995035648,"available":95048261632,"internally_reserved":0,"allocated":1073152,"data_stored":562791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":945679878},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:45:35 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":2.194,"15min":2.0619999999999998},"min":{"1min":0.23999999999999999,"5min":0.193,"15min":0.191},"max":{"1min":4.992,"5min":75.677000000000007,"15min":75.677000000000007},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":2.149,"15min":2.0649999999999999},"min":{"1min":0.29399999999999998,"5min":0.248,"15min":0.22},"max":{"1min":2.387,"5min":75.638999999999996,"15min":75.638999999999996},"last":0.36899999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:45:35 2024","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.5169999999999999,"15min":1.153},"min":{"1min":0.30199999999999999,"5min":0.247,"15min":0.22500000000000001},"max":{"1min":6.0339999999999998,"5min":75.790000000000006,"15min":75.790000000000006},"last":0.501},{"interface":"front","average":{"1min":0.77300000000000002,"5min":1.4730000000000001,"15min":1.1930000000000001},"min":{"1min":0.26500000000000001,"5min":0.26300000000000001,"15min":0.22800000000000001},"max":{"1min":3.2679999999999998,"5min":76.363,"15min":76.363},"last":0.41399999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:45:35 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":0.93799999999999994,"15min":1.823},"min":{"1min":0.27100000000000002,"5min":0.23699999999999999,"15min":0.22800000000000001},"max":{"1min":2.9950000000000001,"5min":7.0289999999999999,"15min":227.79300000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.83899999999999997,"15min":1.837},"min":{"1min":0.254,"5min":0.254,"15min":0.246},"max":{"1min":2.173,"5min":6.6600000000000001,"15min":227.22},"last":0.48999999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:45:35 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.86199999999999999,"15min":1.5680000000000001},"min":{"1min":0.40400000000000003,"5min":0.252,"15min":0.252},"max":{"1min":2.5750000000000002,"5min":4.5919999999999996,"15min":49.798999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":1.1399999999999999,"5min":0.83099999999999996,"15min":1.6080000000000001},"min":{"1min":0.29499999999999998,"5min":0.28899999999999998,"15min":0.27200000000000002},"max":{"1min":6,"5min":6,"15min":49.524000000000001},"last":0.44}]},{"osd":4,"last update":"Wed Sep 11 07:45:35 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.76200000000000001,"15min":0.69999999999999996},"min":{"1min":0.30299999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":2.3100000000000001,"5min":3.8319999999999999,"15min":6.0220000000000002},"last":0.51200000000000001},{"interface":"front","average":{"1min":1.079,"5min":0.77700000000000002,"15min":0.75600000000000001},"min":{"1min":0.28199999999999997,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":2.5470000000000002,"5min":3.3170000000000002,"15min":6.9000000000000004},"last":0.57299999999999995}]},{"osd":5,"last update":"Wed Sep 11 07:45:35 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.79000000000000004,"15min":0.78400000000000003},"min":{"1min":0.33100000000000002,"5min":0.27600000000000002,"15min":0.25},"max":{"1min":2.3410000000000002,"5min":9.9339999999999993,"15min":9.9339999999999993},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.73199999999999998,"15min":0.78300000000000003},"min":{"1min":0.29499999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":2.786,"5min":10.183999999999999,"15min":10.183999999999999},"last":0.59699999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:45:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.624,"15min":0.66400000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.1190000000000002,"5min":4.5439999999999996,"15min":5.1909999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.88,"5min":0.71799999999999997,"15min":0.74099999999999999},"min":{"1min":0.38800000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":2.4079999999999999,"5min":3.4449999999999998,"15min":4.2030000000000003},"last":0.53400000000000003}]}]},{"osd":6,"up_from":44,"seq":188978561266,"num_pgs":84,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":903104,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":901438,"kb_avail":92842048,"statfs":{"total":95995035648,"available":95070257152,"internally_reserved":0,"allocated":1691648,"data_stored":1154403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":923072856},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:45:28 2024","interfaces":[{"interface":"back","average":{"1min":6.8899999999999997,"5min":3.3279999999999998,"15min":2.5310000000000001},"min":{"1min":0.40300000000000002,"5min":0.219,"15min":0.192},"max":{"1min":84.194000000000003,"5min":130.62299999999999,"15min":142.29599999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":6.3639999999999999,"5min":3.3809999999999998,"15min":2.6819999999999999},"min":{"1min":0.30199999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":83.783000000000001,"5min":130.86199999999999,"15min":142.68799999999999},"last":0.49099999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:45:28 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":3.2240000000000002,"15min":2.423},"min":{"1min":0.35999999999999999,"5min":0.246,"15min":0.23599999999999999},"max":{"1min":7.4020000000000001,"5min":131.42500000000001,"15min":131.42500000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":3.1240000000000001,"15min":2.3980000000000001},"min":{"1min":0.315,"5min":0.22,"15min":0.22},"max":{"1min":1.4550000000000001,"5min":132.39599999999999,"15min":132.39599999999999},"last":0.46800000000000003}]},{"osd":2,"last update":"Wed Sep 11 07:45:28 2024","interfaces":[{"interface":"back","average":{"1min":6.577,"5min":2.395,"15min":1.4510000000000001},"min":{"1min":0.27100000000000002,"5min":0.26600000000000001,"15min":0.24299999999999999},"max":{"1min":83.915999999999997,"5min":83.915999999999997,"15min":83.915999999999997},"last":0.42399999999999999},{"interface":"front","average":{"1min":7.5010000000000003,"5min":2.3719999999999999,"15min":1.6299999999999999},"min":{"1min":0.38900000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":84.097999999999999,"5min":84.097999999999999,"15min":84.097999999999999},"last":0.48099999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:45:28 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":1.339,"15min":1.657},"min":{"1min":0.34200000000000003,"5min":0.27700000000000002,"15min":0.246},"max":{"1min":2.2610000000000001,"5min":39.308999999999997,"15min":49.109000000000002},"last":0.435},{"interface":"front","average":{"1min":0.93799999999999994,"5min":1.3620000000000001,"15min":1.74},"min":{"1min":0.32600000000000001,"5min":0.246,"15min":0.246},"max":{"1min":3.2999999999999998,"5min":41.149999999999999,"15min":48.963999999999999},"last":0.45600000000000002}]},{"osd":4,"last update":"Wed Sep 11 07:45:28 2024","interfaces":[{"interface":"back","average":{"1min":2.1190000000000002,"5min":4.0780000000000003,"15min":1.7629999999999999},"min":{"1min":0.35999999999999999,"5min":0.24199999999999999,"15min":0.223},"max":{"1min":12.257999999999999,"5min":181.78800000000001,"15min":181.78800000000001},"last":0.33200000000000002},{"interface":"front","average":{"1min":1.633,"5min":2.383,"15min":1.1799999999999999},"min":{"1min":0.28100000000000003,"5min":0.25,"15min":0.24399999999999999},"max":{"1min":12.102,"5min":128.59299999999999,"15min":128.59299999999999},"last":0.315}]},{"osd":5,"last update":"Wed Sep 11 07:45:28 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.97599999999999998,"15min":1.014},"min":{"1min":0.29199999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.79600000000000004,"5min":10.935,"15min":57.436},"last":0.28599999999999998},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.0389999999999999,"15min":1.0580000000000001},"min":{"1min":0.27500000000000002,"5min":0.253,"15min":0.251},"max":{"1min":7.4420000000000002,"5min":11.15,"15min":56.822000000000003},"last":0.44700000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:45:28 2024","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.206,"15min":0.90100000000000002},"min":{"1min":0.32300000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":7.4699999999999998,"5min":11.121,"15min":11.121},"last":0.373},{"interface":"front","average":{"1min":0.47399999999999998,"5min":1.002,"15min":0.84699999999999998},"min":{"1min":0.34799999999999998,"5min":0.215,"15min":0.215},"max":{"1min":0.72099999999999997,"5min":10.962999999999999,"15min":10.962999999999999},"last":0.35399999999999998}]}]},{"osd":5,"up_from":37,"seq":158913790200,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":888936,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":887870,"kb_avail":92856216,"statfs":{"total":95995035648,"available":95084765184,"internally_reserved":0,"allocated":1073152,"data_stored":563356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":909179224},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:45:06 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.1719999999999999,"15min":2.4649999999999999},"min":{"1min":0.309,"5min":0.23100000000000001,"15min":0.20200000000000001},"max":{"1min":3.4830000000000001,"5min":20.391999999999999,"15min":185.68199999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.89300000000000002,"5min":1.3560000000000001,"15min":2.605},"min":{"1min":0.23400000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":4.2069999999999999,"5min":20.001000000000001,"15min":186.12200000000001},"last":0.501}]},{"osd":1,"last update":"Wed Sep 11 07:45:06 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":2.1400000000000001,"15min":1.6459999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.20799999999999999},"max":{"1min":7.2510000000000003,"5min":122.738,"15min":122.738},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.81699999999999995,"5min":2.2429999999999999,"15min":1.6759999999999999},"min":{"1min":0.26400000000000001,"5min":0.24099999999999999,"15min":0.22700000000000001},"max":{"1min":3.1669999999999998,"5min":122.66200000000001,"15min":122.66200000000001},"last":0.30499999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:45:06 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.3400000000000001,"15min":1.8899999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.22700000000000001},"max":{"1min":5.4820000000000002,"5min":20.718,"15min":187.99199999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":1.131,"5min":1.175,"15min":1.841},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.24399999999999999},"max":{"1min":3.6890000000000001,"5min":19.988,"15min":186.375},"last":0.48699999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:45:06 2024","interfaces":[{"interface":"back","average":{"1min":3.3290000000000002,"5min":4.468,"15min":3.621},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.23599999999999999},"max":{"1min":53.490000000000002,"5min":132.715,"15min":188.11500000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":3.0659999999999998,"5min":4.407,"15min":3.6680000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.25700000000000001},"max":{"1min":53.636000000000003,"5min":133.01599999999999,"15min":186.44200000000001},"last":0.41499999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.97499999999999998,"15min":0.77100000000000002},"min":{"1min":0.26900000000000002,"5min":0.246,"15min":0.19900000000000001},"max":{"1min":3.198,"5min":6.3979999999999997,"15min":11.103},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.93400000000000005,"15min":0.77400000000000002},"min":{"1min":0.27300000000000002,"5min":0.26300000000000001,"15min":0.22500000000000001},"max":{"1min":2.742,"5min":8.407,"15min":10.305},"last":0.54500000000000004}]},{"osd":6,"last update":"Wed Sep 11 07:45:22 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.749,"15min":0.72599999999999998},"min":{"1min":0.32100000000000001,"5min":0.27400000000000002,"15min":0.27200000000000002},"max":{"1min":1.6359999999999999,"5min":5.6840000000000002,"15min":5.6859999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.79000000000000004,"15min":0.73599999999999999},"min":{"1min":0.33800000000000002,"5min":0.315,"15min":0.25800000000000001},"max":{"1min":2.1400000000000001,"5min":5.4589999999999996,"15min":10.613},"last":0.51500000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:45:22 2024","interfaces":[{"interface":"back","average":{"1min":2.4860000000000002,"5min":1.288,"15min":1.1120000000000001},"min":{"1min":0.308,"5min":0.29299999999999998,"15min":0.28000000000000003},"max":{"1min":37.857999999999997,"5min":37.857999999999997,"15min":50.338999999999999},"last":0.439},{"interface":"front","average":{"1min":2.4820000000000002,"5min":1.1919999999999999,"15min":1.071},"min":{"1min":0.37,"5min":0.34399999999999997,"15min":0.26800000000000002},"max":{"1min":37.759999999999998,"5min":37.759999999999998,"15min":50.298999999999999},"last":0.52300000000000002}]}]},{"osd":4,"up_from":0,"seq":133143986428,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":25,"seq":107374182657,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":904688,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":903038,"kb_avail":92840464,"statfs":{"total":95995035648,"available":95068635136,"internally_reserved":0,"allocated":1671168,"data_stored":1152766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":924711256},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:45:32 2024","interfaces":[{"interface":"back","average":{"1min":2.0819999999999999,"5min":1.607,"15min":1.752},"min":{"1min":0.24399999999999999,"5min":0.19800000000000001,"15min":0.16600000000000001},"max":{"1min":27.655999999999999,"5min":27.655999999999999,"15min":38.432000000000002},"last":0.39500000000000002},{"interface":"front","average":{"1min":2.125,"5min":1.655,"15min":2.145},"min":{"1min":0.29999999999999999,"5min":0.183,"15min":0.183},"max":{"1min":26.759,"5min":26.759,"15min":91.876999999999995},"last":0.54100000000000004}]},{"osd":1,"last update":"Wed Sep 11 07:45:32 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":3.4870000000000001,"15min":2.8559999999999999},"min":{"1min":0.26200000000000001,"5min":0.218,"15min":0.19500000000000001},"max":{"1min":3.8799999999999999,"5min":135.41999999999999,"15min":135.41999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.79500000000000004,"5min":3.7890000000000001,"15min":2.4950000000000001},"min":{"1min":0.25,"5min":0.221,"15min":0.192},"max":{"1min":4.4240000000000004,"5min":136.126,"15min":136.126},"last":0.42099999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:45:32 2024","interfaces":[{"interface":"back","average":{"1min":2.29,"5min":2.976,"15min":1.964},"min":{"1min":0.33200000000000002,"5min":0.253,"15min":0.23699999999999999},"max":{"1min":17.204000000000001,"5min":164.02600000000001,"15min":164.02600000000001},"last":0.495},{"interface":"front","average":{"1min":1.49,"5min":2.633,"15min":1.8260000000000001},"min":{"1min":0.32300000000000001,"5min":0.222,"15min":0.21199999999999999},"max":{"1min":17.265000000000001,"5min":163.37799999999999,"15min":163.37799999999999},"last":0.61799999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:45:53 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.155,"15min":1.8280000000000001},"min":{"1min":0.438,"5min":0.314,"15min":0.27500000000000002},"max":{"1min":3.48,"5min":8.6479999999999997,"15min":132.76400000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":1.038,"5min":1.1539999999999999,"15min":1.911},"min":{"1min":0.36499999999999999,"5min":0.317,"15min":0.255},"max":{"1min":3.907,"5min":21.965,"15min":132.154},"last":0.74099999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":2.2709999999999999,"15min":1.669},"min":{"1min":0.34200000000000003,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":3.085,"5min":137.50299999999999,"15min":137.50299999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":2.3159999999999998,"15min":1.7110000000000001},"min":{"1min":0.35099999999999998,"5min":0.33800000000000002,"15min":0.28299999999999997},"max":{"1min":2.1549999999999998,"5min":137.45599999999999,"15min":137.45599999999999},"last":0.71799999999999997}]},{"osd":6,"last update":"Wed Sep 11 07:45:32 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":3.6880000000000002,"15min":2.1360000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.27000000000000002},"max":{"1min":4.2729999999999997,"5min":233.98500000000001,"15min":233.98500000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.89200000000000002,"5min":3.762,"15min":2.1469999999999998},"min":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.30399999999999999},"max":{"1min":2.7669999999999999,"5min":234.05199999999999,"15min":234.05199999999999},"last":0.56000000000000005}]},{"osd":7,"last update":"Wed Sep 11 07:45:20 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":5.0419999999999998,"15min":2.5630000000000002},"min":{"1min":0.41199999999999998,"5min":0.39400000000000002,"15min":0.308},"max":{"1min":3.8500000000000001,"5min":162.42699999999999,"15min":162.42699999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":5.0629999999999997,"15min":2.7549999999999999},"min":{"1min":0.36699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":4.306,"5min":162.39099999999999,"15min":162.39099999999999},"last":0.64300000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411590,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":889580,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":888254,"kb_avail":92855572,"statfs":{"total":95995035648,"available":95084105728,"internally_reserved":0,"allocated":1339392,"data_stored":826272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":909572440},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:45:08 2024","interfaces":[{"interface":"back","average":{"1min":2.8170000000000002,"5min":1.397,"15min":2.032},"min":{"1min":0.20999999999999999,"5min":0.188,"15min":0.16400000000000001},"max":{"1min":19.907,"5min":19.907,"15min":57.802999999999997},"last":0.30399999999999999},{"interface":"front","average":{"1min":2.6150000000000002,"5min":1.46,"15min":2.1469999999999998},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":16.039999999999999,"5min":16.039999999999999,"15min":57.759},"last":0.28499999999999998}]},{"osd":1,"last update":"Wed Sep 11 07:45:08 2024","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":0.97199999999999998,"15min":1.643},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22800000000000001},"max":{"1min":2.6389999999999998,"5min":5.7789999999999999,"15min":65.153999999999996},"last":0.23300000000000001},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.96299999999999997,"15min":1.8420000000000001},"min":{"1min":0.28599999999999998,"5min":0.224,"15min":0.216},"max":{"1min":4.4109999999999996,"5min":5.1200000000000001,"15min":63.311999999999998},"last":0.35699999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:45:37 2024","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.131,"15min":1.911},"min":{"1min":0.29299999999999998,"5min":0.22500000000000001,"15min":0.224},"max":{"1min":3.8700000000000001,"5min":5.2149999999999999,"15min":67.552999999999997},"last":0.372},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.133,"15min":1.895},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":3.6890000000000001,"5min":9.4670000000000005,"15min":67.352999999999994},"last":0.40799999999999997}]},{"osd":4,"last update":"Wed Sep 11 07:45:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.099,"15min":1.0860000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.6629999999999998,"5min":7.1719999999999997,"15min":25.643000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":1,"5min":1.0069999999999999,"15min":1.0449999999999999},"min":{"1min":0.38,"5min":0.36599999999999999,"15min":0.29999999999999999},"max":{"1min":3.702,"5min":4.0510000000000002,"15min":22.434000000000001},"last":0.55800000000000005}]},{"osd":5,"last update":"Wed Sep 11 07:45:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":0.98999999999999999,"15min":1.22},"min":{"1min":0.48499999999999999,"5min":0.374,"15min":0.32100000000000001},"max":{"1min":2.7130000000000001,"5min":3.319,"15min":31.789000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.0629999999999999,"15min":1.2889999999999999},"min":{"1min":0.47999999999999998,"5min":0.32100000000000001,"15min":0.31},"max":{"1min":4.6029999999999998,"5min":8.1690000000000005,"15min":30.898},"last":0.59499999999999997}]},{"osd":6,"last update":"Wed Sep 11 07:45:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.2030000000000001,"15min":1.21},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.33400000000000002},"max":{"1min":3.7400000000000002,"5min":4.5359999999999996,"15min":22.399000000000001},"last":0.435},{"interface":"front","average":{"1min":1.296,"5min":1.155,"15min":1.1819999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.29999999999999999},"max":{"1min":5.1639999999999997,"5min":5.1639999999999997,"15min":25.739000000000001},"last":0.623}]},{"osd":7,"last update":"Wed Sep 11 07:45:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.117,"15min":1.2310000000000001},"min":{"1min":0.41699999999999998,"5min":0.38400000000000001,"15min":0.34300000000000003},"max":{"1min":4.6950000000000003,"5min":5.2199999999999998,"15min":22.484000000000002},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.0329999999999999,"5min":0.999,"15min":1.387},"min":{"1min":0.45800000000000002,"5min":0.373,"15min":0.36699999999999999},"max":{"1min":2.944,"5min":4.9059999999999997,"15min":25.670000000000002},"last":0.5}]}]},{"osd":1,"up_from":13,"seq":55834575113,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":905716,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":904318,"kb_avail":92839436,"statfs":{"total":95995035648,"available":95067582464,"internally_reserved":0,"allocated":1409024,"data_stored":890801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":926021976},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:45:33 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":1.3999999999999999,"15min":2.1600000000000001},"min":{"1min":0.20799999999999999,"5min":0.17299999999999999,"15min":0.16900000000000001},"max":{"1min":4.3639999999999999,"5min":24.561,"15min":82.218999999999994},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.395,"15min":2.1869999999999998},"min":{"1min":0.26500000000000001,"5min":0.20499999999999999,"15min":0.186},"max":{"1min":3.9750000000000001,"5min":24.343,"15min":82.278999999999996},"last":0.376}]},{"osd":2,"last update":"Wed Sep 11 07:45:08 2024","interfaces":[{"interface":"back","average":{"1min":4.8440000000000003,"5min":2.3420000000000001,"15min":1.996},"min":{"1min":0.36599999999999999,"5min":0.251,"15min":0.19800000000000001},"max":{"1min":71.025999999999996,"5min":71.025999999999996,"15min":71.025999999999996},"last":0.34699999999999998},{"interface":"front","average":{"1min":4.569,"5min":2.2149999999999999,"15min":1.9950000000000001},"min":{"1min":0.27700000000000002,"5min":0.20499999999999999,"15min":0.191},"max":{"1min":71.238,"5min":71.238,"15min":71.238},"last":0.35999999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":2.0699999999999998,"15min":2.7589999999999999},"min":{"1min":0.28599999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":4.3049999999999997,"5min":90.766999999999996,"15min":90.766999999999996},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":1.984,"15min":2.6669999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.22600000000000001},"max":{"1min":1.877,"5min":91.513000000000005,"15min":91.513000000000005},"last":0.247}]},{"osd":4,"last update":"Wed Sep 11 07:45:44 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.129,"15min":1.395},"min":{"1min":0.311,"5min":0.30299999999999999,"15min":0.27400000000000002},"max":{"1min":3.6589999999999998,"5min":6.9580000000000002,"15min":7.8390000000000004},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":1.143,"15min":1.4570000000000001},"min":{"1min":0.33700000000000002,"5min":0.30099999999999999,"15min":0.26500000000000001},"max":{"1min":1.8919999999999999,"5min":7.1280000000000001,"15min":9.2289999999999992},"last":0.51200000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:45:08 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":1.177,"15min":1.391},"min":{"1min":0.32000000000000001,"5min":0.27500000000000002,"15min":0.26600000000000001},"max":{"1min":4.5330000000000004,"5min":8.3680000000000003,"15min":8.3680000000000003},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":1.266,"15min":1.4890000000000001},"min":{"1min":0.374,"5min":0.32700000000000001,"15min":0.27200000000000002},"max":{"1min":3.6819999999999999,"5min":8.6809999999999992,"15min":9.1929999999999996},"last":0.56399999999999995}]},{"osd":6,"last update":"Wed Sep 11 07:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":1.0389999999999999,"15min":1.381},"min":{"1min":0.379,"5min":0.30599999999999999,"15min":0.30099999999999999},"max":{"1min":2.2360000000000002,"5min":6.226,"15min":9.2569999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.96399999999999997,"15min":1.1539999999999999},"min":{"1min":0.376,"5min":0.32700000000000001,"15min":0.30499999999999999},"max":{"1min":2.3319999999999999,"5min":6.0730000000000004,"15min":9.3339999999999996},"last":0.53700000000000003}]},{"osd":7,"last update":"Wed Sep 11 07:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.96699999999999997,"15min":1.2889999999999999},"min":{"1min":0.40200000000000002,"5min":0.36499999999999999,"15min":0.32200000000000001},"max":{"1min":2.02,"5min":6.1070000000000002,"15min":9.3629999999999995},"last":0.59399999999999997},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.083,"15min":1.5129999999999999},"min":{"1min":0.437,"5min":0.33400000000000002,"15min":0.313},"max":{"1min":4.8739999999999997,"5min":6.0090000000000003,"15min":8.2219999999999995},"last":0.67600000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738639,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":886136,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":885054,"kb_avail":92859016,"statfs":{"total":95995035648,"available":95087632384,"internally_reserved":0,"allocated":1093632,"data_stored":563082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":906295640},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:45:36 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":1.627,"15min":1.607},"min":{"1min":0.20599999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":3.944,"5min":72.418000000000006,"15min":72.418000000000006},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.745,"5min":1.573,"15min":1.5800000000000001},"min":{"1min":0.26900000000000002,"5min":0.17599999999999999,"15min":0.16600000000000001},"max":{"1min":3.0019999999999998,"5min":71.703999999999994,"15min":71.703999999999994},"last":0.45900000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:45:02 2024","interfaces":[{"interface":"back","average":{"1min":9.9179999999999993,"5min":3.7839999999999998,"15min":3.073},"min":{"1min":0.29699999999999999,"5min":0.23499999999999999,"15min":0.188},"max":{"1min":164.821,"5min":164.821,"15min":200.339},"last":0.26100000000000001},{"interface":"front","average":{"1min":11.220000000000001,"5min":4.0049999999999999,"15min":3.0430000000000001},"min":{"1min":0.32500000000000001,"5min":0.23999999999999999,"15min":0.20699999999999999},"max":{"1min":167.68799999999999,"5min":167.68799999999999,"15min":200.40100000000001},"last":0.29099999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:45:36 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.0780000000000001,"15min":1.8300000000000001},"min":{"1min":0.255,"5min":0.25,"15min":0.20799999999999999},"max":{"1min":4.173,"5min":6.2450000000000001,"15min":45.185000000000002},"last":0.47399999999999998},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.1699999999999999,"15min":1.7230000000000001},"min":{"1min":0.31900000000000001,"5min":0.253,"15min":0.22800000000000001},"max":{"1min":3.254,"5min":5.7309999999999999,"15min":43.228999999999999},"last":0.41999999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:45:53 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":1.0529999999999999,"15min":1.3480000000000001},"min":{"1min":0.33400000000000002,"5min":0.30399999999999999,"15min":0.26700000000000002},"max":{"1min":2.9700000000000002,"5min":7.0519999999999996,"15min":30.315000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.236,"5min":1.23,"15min":1.5149999999999999},"min":{"1min":0.379,"5min":0.28599999999999998,"15min":0.27000000000000002},"max":{"1min":4.2119999999999997,"5min":6.5439999999999996,"15min":30.363},"last":0.441}]},{"osd":5,"last update":"Wed Sep 11 07:45:02 2024","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.028,"15min":1.319},"min":{"1min":0.313,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":3.3479999999999999,"5min":3.9790000000000001,"15min":19.381},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":1.276,"15min":1.5089999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.27200000000000002},"max":{"1min":1.7669999999999999,"5min":7.202,"15min":19.437000000000001},"last":0.503}]},{"osd":6,"last update":"Wed Sep 11 07:45:36 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.2050000000000001,"15min":1.458},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.316},"max":{"1min":4.2359999999999998,"5min":6.5190000000000001,"15min":23.190999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.99299999999999999,"5min":1.2050000000000001,"15min":1.454},"min":{"1min":0.38,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":2.004,"5min":4.6920000000000002,"15min":23.632000000000001},"last":0.57899999999999996}]},{"osd":7,"last update":"Wed Sep 11 07:45:27 2024","interfaces":[{"interface":"back","average":{"1min":3.359,"5min":1.6559999999999999,"15min":1.6399999999999999},"min":{"1min":0.375,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":34.704999999999998,"5min":34.704999999999998,"15min":34.704999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":3.1840000000000002,"5min":1.671,"15min":1.6599999999999999},"min":{"1min":0.38800000000000001,"5min":0.35999999999999999,"15min":0.315},"max":{"1min":34.930999999999997,"5min":34.930999999999997,"15min":34.930999999999997},"last":0.48899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:46:05.112 INFO:tasks.thrashosds.thrasher:Setting 4.1f to [3, 5, 6] 2024-09-11T07:46:05.112 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '4.1f', '3', '5', '6'] 2024-09-11T07:46:05.113 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pg-upmap 4.1f 3 5 6 2024-09-11T07:46:05.127 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:05.149 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:05.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:05.150 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:05.250 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:05.272 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:05.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:05.273 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:05.372 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T07:46:05.373 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:05.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:05.396 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:05.396 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:05.497 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:05.518 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:05.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:05.520 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:05.620 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:05.637 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:46:05.637 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:46:05.637 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2024-09-11T07:46:05.637 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2024-09-11T07:46:05.638 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:46:05.638 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:05 smithi104 ceph-mon[33507]: 2.1a scrub starts 2024-09-11T07:46:05.638 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:05 smithi104 ceph-mon[33507]: 2.1a scrub ok 2024-09-11T07:46:05.638 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:05 smithi104 ceph-mon[33507]: from='client.16083 v1:172.21.15.22:0/3264993718' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:46:05.638 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:05 smithi104 ceph-mon[33507]: pgmap v914: 158 pgs: 158 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T07:46:05.639 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:05 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:46:05.639 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:05 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:46:05.643 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:05.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:05.644 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:05.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:05 smithi022 ceph-mon[29462]: 2.1a scrub starts 2024-09-11T07:46:05.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:05 smithi022 ceph-mon[29462]: 2.1a scrub ok 2024-09-11T07:46:05.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:05 smithi022 ceph-mon[29462]: from='client.16083 v1:172.21.15.22:0/3264993718' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:46:05.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:05 smithi022 ceph-mon[29462]: pgmap v914: 158 pgs: 158 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T07:46:05.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:05 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:46:05.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:05 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:46:05.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:05 smithi022 ceph-mon[37330]: 2.1a scrub starts 2024-09-11T07:46:05.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:05 smithi022 ceph-mon[37330]: 2.1a scrub ok 2024-09-11T07:46:05.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:05 smithi022 ceph-mon[37330]: from='client.16083 v1:172.21.15.22:0/3264993718' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:46:05.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:05 smithi022 ceph-mon[37330]: pgmap v914: 158 pgs: 158 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T07:46:05.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:05 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:46:05.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:05 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:46:05.745 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:05.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:05.771 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:05.771 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:05.872 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:05.894 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:05.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:05.895 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:05.996 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:06.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:06.019 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:06.019 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:06.120 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:06.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:06.142 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:06.143 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:06.244 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:06.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:06.266 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:06.266 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:06.288 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-11T07:46:06.367 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:06.395 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:06.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:06.396 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:06.497 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:06.519 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:06.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:06.520 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:06.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:06 smithi104 ceph-mon[33507]: 5.18 scrub starts 2024-09-11T07:46:06.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:06 smithi104 ceph-mon[33507]: 5.18 scrub ok 2024-09-11T07:46:06.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:06 smithi104 ceph-mon[33507]: 2.11 scrub starts 2024-09-11T07:46:06.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:06 smithi104 ceph-mon[33507]: 2.11 scrub ok 2024-09-11T07:46:06.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:06 smithi104 ceph-mon[33507]: 6.1a scrub starts 2024-09-11T07:46:06.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:06 smithi104 ceph-mon[33507]: 6.1a scrub ok 2024-09-11T07:46:06.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:06 smithi104 ceph-mon[33507]: 6.1e scrub starts 2024-09-11T07:46:06.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:06 smithi104 ceph-mon[33507]: 6.1e scrub ok 2024-09-11T07:46:06.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:06 smithi104 ceph-mon[33507]: osdmap e416: 8 total, 8 up, 7 in 2024-09-11T07:46:06.582 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:46:06.621 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:06.642 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:06.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:06.643 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:06.724 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:06 smithi022 ceph-mon[29462]: 5.18 scrub starts 2024-09-11T07:46:06.724 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:06 smithi022 ceph-mon[29462]: 5.18 scrub ok 2024-09-11T07:46:06.724 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:06 smithi022 ceph-mon[29462]: 2.11 scrub starts 2024-09-11T07:46:06.725 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:06 smithi022 ceph-mon[29462]: 2.11 scrub ok 2024-09-11T07:46:06.725 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:06 smithi022 ceph-mon[29462]: 6.1a scrub starts 2024-09-11T07:46:06.725 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:06 smithi022 ceph-mon[29462]: 6.1a scrub ok 2024-09-11T07:46:06.725 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:06 smithi022 ceph-mon[29462]: 6.1e scrub starts 2024-09-11T07:46:06.725 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:06 smithi022 ceph-mon[29462]: 6.1e scrub ok 2024-09-11T07:46:06.725 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:06 smithi022 ceph-mon[29462]: osdmap e416: 8 total, 8 up, 7 in 2024-09-11T07:46:06.725 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:06 smithi022 ceph-mon[37330]: 5.18 scrub starts 2024-09-11T07:46:06.725 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:06 smithi022 ceph-mon[37330]: 5.18 scrub ok 2024-09-11T07:46:06.725 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:06 smithi022 ceph-mon[37330]: 2.11 scrub starts 2024-09-11T07:46:06.725 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:06 smithi022 ceph-mon[37330]: 2.11 scrub ok 2024-09-11T07:46:06.726 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:06 smithi022 ceph-mon[37330]: 6.1a scrub starts 2024-09-11T07:46:06.726 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:06 smithi022 ceph-mon[37330]: 6.1a scrub ok 2024-09-11T07:46:06.726 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:06 smithi022 ceph-mon[37330]: 6.1e scrub starts 2024-09-11T07:46:06.726 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:06 smithi022 ceph-mon[37330]: 6.1e scrub ok 2024-09-11T07:46:06.726 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:06 smithi022 ceph-mon[37330]: osdmap e416: 8 total, 8 up, 7 in 2024-09-11T07:46:06.744 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:06.769 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:06.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:06.770 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:46:06.871 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:06.893 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:06.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:06.894 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:06.995 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:07.017 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:07.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:07.018 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:07.119 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:07.141 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:07.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:07.142 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:07.242 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:07.265 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:07.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:07.266 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:07.367 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:07.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:07.390 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:07.391 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:07.491 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:07.516 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:07.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:07.517 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:07.589 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:07 smithi022 ceph-mon[29462]: pgmap v916: 158 pgs: 158 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:46:07.589 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:07 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2828515501' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:46:07.589 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:07 smithi022 ceph-mon[29462]: 4.9 scrub starts 2024-09-11T07:46:07.589 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:07 smithi022 ceph-mon[29462]: 4.9 scrub ok 2024-09-11T07:46:07.589 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:07 smithi022 ceph-mon[37330]: pgmap v916: 158 pgs: 158 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:46:07.589 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:07 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2828515501' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:46:07.590 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:07 smithi022 ceph-mon[37330]: 4.9 scrub starts 2024-09-11T07:46:07.590 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:07 smithi022 ceph-mon[37330]: 4.9 scrub ok 2024-09-11T07:46:07.618 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:07.640 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:07.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:07.642 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:07.732 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:07 smithi104 ceph-mon[33507]: pgmap v916: 158 pgs: 158 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:46:07.733 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:07 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2828515501' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:46:07.733 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:07 smithi104 ceph-mon[33507]: 4.9 scrub starts 2024-09-11T07:46:07.733 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:07 smithi104 ceph-mon[33507]: 4.9 scrub ok 2024-09-11T07:46:07.742 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:07.755 INFO:teuthology.orchestra.run.smithi022.stderr:enabled application 'rados' on pool 'unique_pool_4' 2024-09-11T07:46:07.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:07.769 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:07.769 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:07.870 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:07.892 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:07.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:07.893 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:07.994 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:08.016 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:08.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:08.017 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:08.118 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:08.141 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:08.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:08.142 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:08.243 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:08.248 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:46:08.248 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2024-09-11T07:46:08.248 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2024-09-11T07:46:08.248 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2024-09-11T07:46:08.248 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.248 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:85934 9.f 9:fdbe5b3c:::benchmark_data_smithi022_133943_object85933:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e258)", 2024-09-11T07:46:08.248 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:26.311544+0000", 2024-09-11T07:46:08.248 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 461.93416120199998, 2024-09-11T07:46:08.248 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.30120598399999998, 2024-09-11T07:46:08.248 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 85934 2024-09-11T07:46:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.311544+0000", 2024-09-11T07:46:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.249 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.311544+0000", 2024-09-11T07:46:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.311542+0000", 2024-09-11T07:46:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:46:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.311634+0000", 2024-09-11T07:46:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.1664999999999997e-05 2024-09-11T07:46:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.311653+0000", 2024-09-11T07:46:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8987e-05 2024-09-11T07:46:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.311665+0000", 2024-09-11T07:46:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1657e-05 2024-09-11T07:46:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.317571+0000", 2024-09-11T07:46:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0059061249999999999 2024-09-11T07:46:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.317705+0000", 2024-09-11T07:46:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013474 2024-09-11T07:46:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:46:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.317854+0000", 2024-09-11T07:46:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000148627 2024-09-11T07:46:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.325240+0000", 2024-09-11T07:46:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0073861480000000004 2024-09-11T07:46:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.253 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.253 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:08.253 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.325252+0000", 2024-09-11T07:46:08.253 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1678000000000001e-05 2024-09-11T07:46:08.253 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.253 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.253 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.253 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.612633+0000", 2024-09-11T07:46:08.253 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.28738107099999999 2024-09-11T07:46:08.253 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.253 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.253 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:08.253 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.612702+0000", 2024-09-11T07:46:08.253 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9142999999999999e-05 2024-09-11T07:46:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.612750+0000", 2024-09-11T07:46:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7559000000000003e-05 2024-09-11T07:46:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:85942 9.f 9:fc70c519:::benchmark_data_smithi022_133943_object85941:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e258)", 2024-09-11T07:46:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:26.316250+0000", 2024-09-11T07:46:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 461.929454572, 2024-09-11T07:46:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.29663829400000002, 2024-09-11T07:46:08.254 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 85942 2024-09-11T07:46:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.316250+0000", 2024-09-11T07:46:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.255 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.316250+0000", 2024-09-11T07:46:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.316249+0000", 2024-09-11T07:46:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:46:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.316331+0000", 2024-09-11T07:46:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2365000000000002e-05 2024-09-11T07:46:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.256 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.316343+0000", 2024-09-11T07:46:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1596e-05 2024-09-11T07:46:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.316348+0000", 2024-09-11T07:46:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6470000000000003e-06 2024-09-11T07:46:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.322428+0000", 2024-09-11T07:46:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0060797029999999997 2024-09-11T07:46:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.257 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.322544+0000", 2024-09-11T07:46:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000116296 2024-09-11T07:46:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:46:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.322682+0000", 2024-09-11T07:46:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013749500000000001 2024-09-11T07:46:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.325339+0000", 2024-09-11T07:46:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0026568189999999999 2024-09-11T07:46:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.258 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.259 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.259 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.328284+0000", 2024-09-11T07:46:08.259 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0029453029999999998 2024-09-11T07:46:08.259 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.259 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.259 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.259 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.612802+0000", 2024-09-11T07:46:08.259 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.28451834599999998 2024-09-11T07:46:08.259 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.259 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.259 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:08.259 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.612852+0000", 2024-09-11T07:46:08.259 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0139999999999998e-05 2024-09-11T07:46:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.612889+0000", 2024-09-11T07:46:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6168999999999999e-05 2024-09-11T07:46:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:85953 9.f 9:fc848a32:::benchmark_data_smithi022_133943_object85952:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e258)", 2024-09-11T07:46:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:26.331015+0000", 2024-09-11T07:46:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 461.91468981700001, 2024-09-11T07:46:08.260 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.28199429199999998, 2024-09-11T07:46:08.261 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:08.261 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:08.261 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:08.261 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:08.262 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:08.262 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 85953 2024-09-11T07:46:08.262 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.262 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:08.262 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.262 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:08.262 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.331015+0000", 2024-09-11T07:46:08.262 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.262 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.262 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.262 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:08.262 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.331015+0000", 2024-09-11T07:46:08.263 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.263 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.263 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.263 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:08.263 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.331013+0000", 2024-09-11T07:46:08.263 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:46:08.263 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.263 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.263 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:08.263 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.331046+0000", 2024-09-11T07:46:08.263 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3189999999999999e-05 2024-09-11T07:46:08.263 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.263 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.263 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:08.264 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.331057+0000", 2024-09-11T07:46:08.264 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0703999999999999e-05 2024-09-11T07:46:08.264 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.264 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.264 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:08.264 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.331062+0000", 2024-09-11T07:46:08.264 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5439999999999998e-06 2024-09-11T07:46:08.264 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.264 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.264 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:08.264 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.332439+0000", 2024-09-11T07:46:08.264 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0013765870000000001 2024-09-11T07:46:08.264 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.264 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.265 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:08.265 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.332578+0000", 2024-09-11T07:46:08.265 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013949000000000001 2024-09-11T07:46:08.265 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.265 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.265 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:46:08.265 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.332730+0000", 2024-09-11T07:46:08.265 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015161100000000001 2024-09-11T07:46:08.265 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.265 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.265 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.265 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.454340+0000", 2024-09-11T07:46:08.265 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.12160982100000001 2024-09-11T07:46:08.266 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.266 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.266 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:08.266 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.454786+0000", 2024-09-11T07:46:08.266 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00044578 2024-09-11T07:46:08.266 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.266 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.266 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.266 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.612928+0000", 2024-09-11T07:46:08.266 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.15814265899999999 2024-09-11T07:46:08.266 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.266 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.266 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:08.266 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.612973+0000", 2024-09-11T07:46:08.267 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4904999999999998e-05 2024-09-11T07:46:08.267 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.267 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.267 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:08.267 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.613009+0000", 2024-09-11T07:46:08.267 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6177000000000003e-05 2024-09-11T07:46:08.267 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.267 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:08.267 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.267 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.267 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.267 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:85932 9.7 9:e6d503e6:::benchmark_data_smithi022_133943_object85931:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e259)", 2024-09-11T07:46:08.267 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:26.333730+0000", 2024-09-11T07:46:08.267 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 461.91197501300002, 2024-09-11T07:46:08.268 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.025048288, 2024-09-11T07:46:08.268 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:08.268 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:46:08.268 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:08.268 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:08.268 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:08.268 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 85932 2024-09-11T07:46:08.268 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.268 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:08.268 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.268 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:08.268 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333730+0000", 2024-09-11T07:46:08.268 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.268 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.269 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.269 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:08.269 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333730+0000", 2024-09-11T07:46:08.269 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.269 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.269 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.269 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:08.269 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333727+0000", 2024-09-11T07:46:08.269 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999967 2024-09-11T07:46:08.269 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.269 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.269 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:08.269 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333773+0000", 2024-09-11T07:46:08.270 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6940999999999997e-05 2024-09-11T07:46:08.270 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.270 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.270 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:08.270 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333787+0000", 2024-09-11T07:46:08.270 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3107e-05 2024-09-11T07:46:08.270 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.270 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.270 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:08.270 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333794+0000", 2024-09-11T07:46:08.270 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4059999999999999e-06 2024-09-11T07:46:08.270 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.270 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.270 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:08.271 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.454708+0000", 2024-09-11T07:46:08.271 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.120914359 2024-09-11T07:46:08.271 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.271 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.271 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:08.271 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.454717+0000", 2024-09-11T07:46:08.271 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2830000000000001e-06 2024-09-11T07:46:08.271 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.271 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.271 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:08.271 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.358693+0000", 2024-09-11T07:46:08.271 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.90397602300000002 2024-09-11T07:46:08.271 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.271 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.272 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:08.272 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.358778+0000", 2024-09-11T07:46:08.272 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5457999999999994e-05 2024-09-11T07:46:08.272 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.272 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:08.272 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.272 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.272 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.272 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:85948 9.6 9:6409403d:::benchmark_data_smithi022_133943_object85947:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e259)", 2024-09-11T07:46:08.272 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:26.333803+0000", 2024-09-11T07:46:08.272 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 461.911902193, 2024-09-11T07:46:08.272 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0201702269999999, 2024-09-11T07:46:08.272 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:08.273 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:46:08.273 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:08.273 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:08.273 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:08.273 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 85948 2024-09-11T07:46:08.273 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.273 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:08.273 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.273 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:08.273 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333803+0000", 2024-09-11T07:46:08.273 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.273 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.273 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.273 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:08.273 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333803+0000", 2024-09-11T07:46:08.274 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.274 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.274 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.274 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:08.274 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333802+0000", 2024-09-11T07:46:08.274 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:46:08.274 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.274 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.274 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:08.274 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333842+0000", 2024-09-11T07:46:08.274 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0456e-05 2024-09-11T07:46:08.274 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.274 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.274 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:08.275 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333849+0000", 2024-09-11T07:46:08.275 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.511e-06 2024-09-11T07:46:08.275 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.275 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.275 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:08.275 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.333853+0000", 2024-09-11T07:46:08.275 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.963e-06 2024-09-11T07:46:08.275 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.275 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.275 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:08.275 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.469490+0000", 2024-09-11T07:46:08.275 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13563673200000001 2024-09-11T07:46:08.275 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.275 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.275 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:08.276 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:26.469495+0000", 2024-09-11T07:46:08.276 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3410000000000002e-06 2024-09-11T07:46:08.276 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.276 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.276 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:08.276 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.353902+0000", 2024-09-11T07:46:08.276 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.88440743099999997 2024-09-11T07:46:08.276 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.276 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.276 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:08.276 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.353973+0000", 2024-09-11T07:46:08.276 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0479000000000001e-05 2024-09-11T07:46:08.276 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.276 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:08.277 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.277 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.277 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.277 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:86232 9.6 9:6062b21b:::benchmark_data_smithi022_133943_object86231:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:46:08.277 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:27.532402+0000", 2024-09-11T07:46:08.277 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 460.713302441, 2024-09-11T07:46:08.277 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.180620107, 2024-09-11T07:46:08.277 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:08.277 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:08.277 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:08.277 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:08.277 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:08.277 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 86232 2024-09-11T07:46:08.277 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.278 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:08.278 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.278 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:08.278 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532402+0000", 2024-09-11T07:46:08.278 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.278 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.278 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.278 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:08.278 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532402+0000", 2024-09-11T07:46:08.278 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.278 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.278 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.278 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:08.278 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532400+0000", 2024-09-11T07:46:08.278 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:46:08.279 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.279 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.279 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:08.279 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532456+0000", 2024-09-11T07:46:08.279 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5791000000000002e-05 2024-09-11T07:46:08.279 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.279 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.279 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:08.279 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532470+0000", 2024-09-11T07:46:08.279 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4280999999999999e-05 2024-09-11T07:46:08.279 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.279 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.279 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:08.279 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532477+0000", 2024-09-11T07:46:08.280 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3350000000000003e-06 2024-09-11T07:46:08.280 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.280 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.280 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:08.280 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532524+0000", 2024-09-11T07:46:08.280 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7128e-05 2024-09-11T07:46:08.280 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.280 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.280 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:08.280 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532627+0000", 2024-09-11T07:46:08.280 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010340099999999999 2024-09-11T07:46:08.280 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.280 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.280 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:46:08.281 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532919+0000", 2024-09-11T07:46:08.281 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00029145000000000001 2024-09-11T07:46:08.281 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.281 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.281 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:08.281 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.534532+0000", 2024-09-11T07:46:08.281 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0016128519999999999 2024-09-11T07:46:08.281 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.281 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.281 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.281 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.537151+0000", 2024-09-11T07:46:08.281 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0026190900000000001 2024-09-11T07:46:08.281 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.281 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.282 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.282 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.712780+0000", 2024-09-11T07:46:08.282 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.175629435 2024-09-11T07:46:08.282 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.282 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.282 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:08.282 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.712894+0000", 2024-09-11T07:46:08.282 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000113725 2024-09-11T07:46:08.282 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.282 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.282 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:08.282 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.713022+0000", 2024-09-11T07:46:08.282 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012873499999999999 2024-09-11T07:46:08.282 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.283 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:08.283 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.283 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.283 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.283 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:86234 9.6 9:6677583e:::benchmark_data_smithi022_133943_object86233:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:46:08.283 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:27.532659+0000", 2024-09-11T07:46:08.283 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 460.71304616100002, 2024-09-11T07:46:08.283 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18307527700000001, 2024-09-11T07:46:08.283 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:08.283 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:08.283 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:08.283 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:08.283 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:08.283 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 86234 2024-09-11T07:46:08.284 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.284 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:08.284 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.284 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:08.284 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532659+0000", 2024-09-11T07:46:08.284 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.284 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.284 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.284 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:08.284 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532659+0000", 2024-09-11T07:46:08.284 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.284 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.284 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.284 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:08.285 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532657+0000", 2024-09-11T07:46:08.285 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:46:08.285 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.285 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.285 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:08.285 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532714+0000", 2024-09-11T07:46:08.285 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6298999999999997e-05 2024-09-11T07:46:08.285 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.285 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.285 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:08.285 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532725+0000", 2024-09-11T07:46:08.285 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1216000000000001e-05 2024-09-11T07:46:08.285 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.285 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.286 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:08.286 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.532730+0000", 2024-09-11T07:46:08.286 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5849999999999999e-06 2024-09-11T07:46:08.286 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.286 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.286 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:08.286 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.535694+0000", 2024-09-11T07:46:08.286 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0029634069999999999 2024-09-11T07:46:08.286 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.286 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.286 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:08.286 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.535830+0000", 2024-09-11T07:46:08.287 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013649300000000001 2024-09-11T07:46:08.287 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.287 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.287 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:46:08.287 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.536159+0000", 2024-09-11T07:46:08.287 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00032905400000000001 2024-09-11T07:46:08.287 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.287 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.287 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:08.287 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.537278+0000", 2024-09-11T07:46:08.287 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0011182830000000001 2024-09-11T07:46:08.287 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.287 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.287 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.288 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.537813+0000", 2024-09-11T07:46:08.288 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00053518700000000001 2024-09-11T07:46:08.288 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.288 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.288 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.288 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.715656+0000", 2024-09-11T07:46:08.288 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17784266000000001 2024-09-11T07:46:08.288 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.288 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.288 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:08.288 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.715700+0000", 2024-09-11T07:46:08.288 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4508000000000001e-05 2024-09-11T07:46:08.288 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.288 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.289 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:08.289 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.715734+0000", 2024-09-11T07:46:08.289 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3877000000000001e-05 2024-09-11T07:46:08.289 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.289 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:08.289 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.289 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.289 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.289 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:86244 9.a 9:5743f4ed:::benchmark_data_smithi022_133943_object86243:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:46:08.289 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:27.540150+0000", 2024-09-11T07:46:08.289 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 460.705555253, 2024-09-11T07:46:08.289 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.177488494, 2024-09-11T07:46:08.289 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:08.290 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:08.290 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:08.290 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:08.290 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:08.290 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 86244 2024-09-11T07:46:08.290 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.290 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:08.290 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.290 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:08.290 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540150+0000", 2024-09-11T07:46:08.290 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.290 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.290 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.290 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:08.291 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540150+0000", 2024-09-11T07:46:08.291 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.291 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.291 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.291 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:08.291 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540148+0000", 2024-09-11T07:46:08.291 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:46:08.291 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.291 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.291 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:08.291 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540199+0000", 2024-09-11T07:46:08.291 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1332999999999998e-05 2024-09-11T07:46:08.291 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.292 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.292 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:08.292 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540210+0000", 2024-09-11T07:46:08.292 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1323e-05 2024-09-11T07:46:08.292 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.292 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.292 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:08.292 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540215+0000", 2024-09-11T07:46:08.292 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6280000000000004e-06 2024-09-11T07:46:08.292 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.292 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.292 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:08.292 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540253+0000", 2024-09-11T07:46:08.292 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8096000000000003e-05 2024-09-11T07:46:08.293 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.293 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.293 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:08.293 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540335+0000", 2024-09-11T07:46:08.293 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1703e-05 2024-09-11T07:46:08.293 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.293 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.293 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,7", 2024-09-11T07:46:08.293 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.540427+0000", 2024-09-11T07:46:08.293 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2069000000000002e-05 2024-09-11T07:46:08.293 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.293 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.293 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:08.293 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.542279+0000", 2024-09-11T07:46:08.294 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001852245 2024-09-11T07:46:08.294 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.294 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.294 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.294 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.542406+0000", 2024-09-11T07:46:08.294 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012750700000000001 2024-09-11T07:46:08.294 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.294 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.294 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.294 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.717564+0000", 2024-09-11T07:46:08.294 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17515773600000001 2024-09-11T07:46:08.294 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.294 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.294 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:08.294 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.717600+0000", 2024-09-11T07:46:08.295 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5410000000000001e-05 2024-09-11T07:46:08.295 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.295 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.295 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:08.295 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:27.717638+0000", 2024-09-11T07:46:08.295 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8473999999999998e-05 2024-09-11T07:46:08.295 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.295 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:08.295 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.295 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.295 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.295 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87252 9.12 9.5bd7f852 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:46:08.295 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.378195+0000", 2024-09-11T07:46:08.295 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 459.86750970499997, 2024-09-11T07:46:08.296 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.96275559600000005, 2024-09-11T07:46:08.296 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:08.296 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:46:08.296 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:08.296 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:08.296 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:08.296 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87252 2024-09-11T07:46:08.296 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.296 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:08.296 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.296 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:08.296 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378195+0000", 2024-09-11T07:46:08.296 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.297 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.297 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.297 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:08.297 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378195+0000", 2024-09-11T07:46:08.297 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.297 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.297 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.297 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:08.297 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378193+0000", 2024-09-11T07:46:08.297 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:46:08.297 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.297 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.297 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:08.297 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378254+0000", 2024-09-11T07:46:08.298 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0699999999999998e-05 2024-09-11T07:46:08.298 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.298 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.298 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:08.298 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378263+0000", 2024-09-11T07:46:08.298 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7579999999999998e-06 2024-09-11T07:46:08.298 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.298 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.298 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:08.298 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378268+0000", 2024-09-11T07:46:08.298 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9830000000000004e-06 2024-09-11T07:46:08.298 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.298 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.298 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:08.299 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378303+0000", 2024-09-11T07:46:08.299 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5503999999999998e-05 2024-09-11T07:46:08.299 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.299 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.299 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:08.299 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.378308+0000", 2024-09-11T07:46:08.299 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.33e-06 2024-09-11T07:46:08.299 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.299 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.299 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:08.299 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.340939+0000", 2024-09-11T07:46:08.299 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.96263122800000001 2024-09-11T07:46:08.299 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.299 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.300 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:08.300 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.340951+0000", 2024-09-11T07:46:08.300 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1800999999999999e-05 2024-09-11T07:46:08.300 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.300 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:08.300 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.300 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.300 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.300 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196432 9.d 9:b43c5e98:::benchmark_data_smithi022_133943_object69841:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:46:08.300 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.509114+0000", 2024-09-11T07:46:08.300 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 350.73659047699999, 2024-09-11T07:46:08.300 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.152507894, 2024-09-11T07:46:08.300 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:08.301 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:08.301 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:08.301 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:08.301 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:08.301 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196432 2024-09-11T07:46:08.301 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.301 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:08.301 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.301 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:08.301 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509114+0000", 2024-09-11T07:46:08.301 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.301 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.301 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.301 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:08.302 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509114+0000", 2024-09-11T07:46:08.302 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.302 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.302 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.302 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:08.302 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509112+0000", 2024-09-11T07:46:08.302 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:46:08.302 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.302 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.302 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:08.302 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509119+0000", 2024-09-11T07:46:08.302 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3439999999999997e-06 2024-09-11T07:46:08.302 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.302 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.303 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:08.303 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509130+0000", 2024-09-11T07:46:08.303 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1066999999999999e-05 2024-09-11T07:46:08.303 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.303 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.303 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:08.303 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509136+0000", 2024-09-11T07:46:08.303 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0460000000000002e-06 2024-09-11T07:46:08.303 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.303 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.303 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:08.303 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509237+0000", 2024-09-11T07:46:08.303 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000101338 2024-09-11T07:46:08.304 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.304 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.304 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:08.304 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509361+0000", 2024-09-11T07:46:08.304 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000123287 2024-09-11T07:46:08.304 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.304 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.304 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,4,6,7", 2024-09-11T07:46:08.304 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509470+0000", 2024-09-11T07:46:08.304 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000109986 2024-09-11T07:46:08.304 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.304 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.304 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.304 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.515270+0000", 2024-09-11T07:46:08.305 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0057995310000000001 2024-09-11T07:46:08.305 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.305 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.305 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.305 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.515306+0000", 2024-09-11T07:46:08.305 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5920999999999997e-05 2024-09-11T07:46:08.305 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.305 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.305 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:08.305 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.515314+0000", 2024-09-11T07:46:08.305 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7680000000000008e-06 2024-09-11T07:46:08.305 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.305 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.305 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.306 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.659682+0000", 2024-09-11T07:46:08.306 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14436806299999999 2024-09-11T07:46:08.306 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.306 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.306 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.306 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.661408+0000", 2024-09-11T07:46:08.306 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0017266499999999999 2024-09-11T07:46:08.306 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.306 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.306 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:08.306 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.661490+0000", 2024-09-11T07:46:08.306 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1951000000000002e-05 2024-09-11T07:46:08.306 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.306 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.307 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:08.307 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.661622+0000", 2024-09-11T07:46:08.307 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013185900000000001 2024-09-11T07:46:08.307 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.307 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:08.307 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.307 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.307 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.307 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196436 9.f 9:f28fa10b:::benchmark_data_smithi022_133943_object69845:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:46:08.307 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.509149+0000", 2024-09-11T07:46:08.307 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 350.73655629400002, 2024-09-11T07:46:08.307 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.15385757, 2024-09-11T07:46:08.307 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:08.307 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:08.308 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:08.308 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:08.308 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:08.308 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196436 2024-09-11T07:46:08.308 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.308 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:08.308 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.308 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:08.308 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509149+0000", 2024-09-11T07:46:08.308 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.308 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.308 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.308 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:08.308 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509149+0000", 2024-09-11T07:46:08.309 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.309 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.309 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.309 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:08.309 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509148+0000", 2024-09-11T07:46:08.309 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:46:08.309 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.309 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.309 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:08.309 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509152+0000", 2024-09-11T07:46:08.309 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3309999999999997e-06 2024-09-11T07:46:08.309 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.309 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.309 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:08.310 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509158+0000", 2024-09-11T07:46:08.310 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0090000000000004e-06 2024-09-11T07:46:08.310 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.310 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.310 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:08.310 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509162+0000", 2024-09-11T07:46:08.310 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9149999999999998e-06 2024-09-11T07:46:08.310 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.310 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.310 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:08.310 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509226+0000", 2024-09-11T07:46:08.310 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3969999999999999e-05 2024-09-11T07:46:08.310 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.310 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.311 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:08.311 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509327+0000", 2024-09-11T07:46:08.311 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000101217 2024-09-11T07:46:08.311 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.311 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.311 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:46:08.311 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509426+0000", 2024-09-11T07:46:08.311 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8525999999999998e-05 2024-09-11T07:46:08.311 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.311 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.311 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:08.311 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.512876+0000", 2024-09-11T07:46:08.311 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003449939 2024-09-11T07:46:08.311 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.312 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.312 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.312 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.658382+0000", 2024-09-11T07:46:08.312 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.145506042 2024-09-11T07:46:08.312 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.312 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.312 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.312 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.662896+0000", 2024-09-11T07:46:08.312 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0045144510000000001 2024-09-11T07:46:08.312 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.312 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.312 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:08.312 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.662971+0000", 2024-09-11T07:46:08.313 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4895999999999996e-05 2024-09-11T07:46:08.313 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.313 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.313 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:08.313 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.663006+0000", 2024-09-11T07:46:08.313 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5099999999999999e-05 2024-09-11T07:46:08.313 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.313 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:08.313 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.313 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.313 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.313 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196438 9.7 9:e639c272:::benchmark_data_smithi022_133943_object69847:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:46:08.313 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.509172+0000", 2024-09-11T07:46:08.314 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 350.73653314500001, 2024-09-11T07:46:08.314 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.154085528, 2024-09-11T07:46:08.314 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:08.314 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:08.314 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:08.314 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:08.314 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:08.314 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196438 2024-09-11T07:46:08.314 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.314 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:08.314 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.314 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:08.314 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509172+0000", 2024-09-11T07:46:08.314 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.314 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.315 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.315 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:08.315 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509172+0000", 2024-09-11T07:46:08.315 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.315 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.315 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.315 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:08.315 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509171+0000", 2024-09-11T07:46:08.315 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:46:08.315 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.315 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.315 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:08.315 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509174+0000", 2024-09-11T07:46:08.315 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2679999999999999e-06 2024-09-11T07:46:08.315 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.316 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.316 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:08.316 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509179+0000", 2024-09-11T07:46:08.316 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9119999999999997e-06 2024-09-11T07:46:08.316 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.316 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.316 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:08.316 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509183+0000", 2024-09-11T07:46:08.316 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4410000000000002e-06 2024-09-11T07:46:08.316 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.316 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:08.318 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:08.318 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.510817+0000", 2024-09-11T07:46:08.318 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0016344020000000001 2024-09-11T07:46:08.318 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.318 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.318 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:08.318 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.510945+0000", 2024-09-11T07:46:08.318 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012800999999999999 2024-09-11T07:46:08.318 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.318 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.319 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2,3,7", 2024-09-11T07:46:08.319 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.511375+0000", 2024-09-11T07:46:08.319 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00042995300000000001 2024-09-11T07:46:08.319 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.319 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.319 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:08.319 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.659619+0000", 2024-09-11T07:46:08.319 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.148244129 2024-09-11T07:46:08.319 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.319 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.319 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.319 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.659674+0000", 2024-09-11T07:46:08.319 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4605000000000003e-05 2024-09-11T07:46:08.319 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.320 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.320 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.320 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.659715+0000", 2024-09-11T07:46:08.320 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0935e-05 2024-09-11T07:46:08.320 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.320 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.320 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.320 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.662817+0000", 2024-09-11T07:46:08.320 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0031017319999999998 2024-09-11T07:46:08.320 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.320 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.320 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.320 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.663041+0000", 2024-09-11T07:46:08.320 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000224303 2024-09-11T07:46:08.321 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.321 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.321 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:08.321 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.663089+0000", 2024-09-11T07:46:08.321 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8464000000000002e-05 2024-09-11T07:46:08.321 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.321 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.321 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:08.321 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.663257+0000", 2024-09-11T07:46:08.321 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016791399999999999 2024-09-11T07:46:08.321 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.321 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:08.321 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.322 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.322 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.322 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196563 9.6 9:63ebaa83:::benchmark_data_smithi022_133943_object69972:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:46:08.322 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.771425+0000", 2024-09-11T07:46:08.322 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 350.47427932900001, 2024-09-11T07:46:08.322 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.153416993, 2024-09-11T07:46:08.322 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:08.322 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:08.322 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:08.322 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:08.322 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:08.322 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196563 2024-09-11T07:46:08.322 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.322 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:08.322 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.323 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:08.323 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771425+0000", 2024-09-11T07:46:08.323 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.323 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.323 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.323 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:08.323 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771425+0000", 2024-09-11T07:46:08.323 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.323 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.323 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.323 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:08.323 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771424+0000", 2024-09-11T07:46:08.323 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:46:08.323 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.324 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.324 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:08.324 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771430+0000", 2024-09-11T07:46:08.324 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4540000000000004e-06 2024-09-11T07:46:08.324 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.324 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.324 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:08.324 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771438+0000", 2024-09-11T07:46:08.324 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.418e-06 2024-09-11T07:46:08.324 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.324 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.324 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:08.324 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771442+0000", 2024-09-11T07:46:08.324 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.622e-06 2024-09-11T07:46:08.325 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.325 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.325 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:08.325 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771560+0000", 2024-09-11T07:46:08.325 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000117298 2024-09-11T07:46:08.325 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.325 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.325 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:08.325 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771669+0000", 2024-09-11T07:46:08.325 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000109386 2024-09-11T07:46:08.325 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.325 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.325 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:46:08.325 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.772085+0000", 2024-09-11T07:46:08.326 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00041578699999999998 2024-09-11T07:46:08.326 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.326 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.326 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:08.326 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.776041+0000", 2024-09-11T07:46:08.326 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0039559069999999998 2024-09-11T07:46:08.326 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.326 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.326 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.326 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.782755+0000", 2024-09-11T07:46:08.326 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0067147760000000004 2024-09-11T07:46:08.326 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.326 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.326 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.327 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924754+0000", 2024-09-11T07:46:08.327 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14199824599999999 2024-09-11T07:46:08.327 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.327 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.327 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:08.327 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924797+0000", 2024-09-11T07:46:08.327 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2988999999999998e-05 2024-09-11T07:46:08.327 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.327 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.327 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:08.327 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924842+0000", 2024-09-11T07:46:08.327 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5884000000000003e-05 2024-09-11T07:46:08.327 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.328 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:08.328 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.328 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.328 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.328 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196577 9.13 9:c894ba9e:::benchmark_data_smithi022_133943_object69986:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:46:08.328 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.785754+0000", 2024-09-11T07:46:08.328 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 350.45995120999999, 2024-09-11T07:46:08.328 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.167572637, 2024-09-11T07:46:08.328 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:08.328 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:08.328 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:08.328 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:08.328 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:08.328 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196577 2024-09-11T07:46:08.329 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.329 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:08.329 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.329 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:08.329 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785754+0000", 2024-09-11T07:46:08.329 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.329 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.329 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.329 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:08.329 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785754+0000", 2024-09-11T07:46:08.329 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.329 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.329 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.329 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:08.330 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785752+0000", 2024-09-11T07:46:08.330 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:46:08.330 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.330 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.330 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:08.330 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785757+0000", 2024-09-11T07:46:08.330 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0010000000000001e-06 2024-09-11T07:46:08.330 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.330 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.330 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:08.330 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785765+0000", 2024-09-11T07:46:08.330 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2670000000000006e-06 2024-09-11T07:46:08.330 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.330 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.331 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:08.331 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785771+0000", 2024-09-11T07:46:08.331 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.242e-06 2024-09-11T07:46:08.331 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.331 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.331 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:08.331 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924515+0000", 2024-09-11T07:46:08.331 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13874467300000001 2024-09-11T07:46:08.331 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.331 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.331 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:08.331 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924668+0000", 2024-09-11T07:46:08.331 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015245999999999999 2024-09-11T07:46:08.332 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.332 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.332 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,6,7", 2024-09-11T07:46:08.332 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.925194+0000", 2024-09-11T07:46:08.332 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00052637400000000003 2024-09-11T07:46:08.332 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.332 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.332 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.332 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.930777+0000", 2024-09-11T07:46:08.332 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00558319 2024-09-11T07:46:08.332 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.332 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.332 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.332 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.930807+0000", 2024-09-11T07:46:08.333 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9363999999999999e-05 2024-09-11T07:46:08.333 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.333 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.333 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:08.333 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.931376+0000", 2024-09-11T07:46:08.333 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00056952599999999999 2024-09-11T07:46:08.333 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.333 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.333 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.333 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.953159+0000", 2024-09-11T07:46:08.333 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.021782632999999999 2024-09-11T07:46:08.333 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.333 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.333 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:08.334 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.953206+0000", 2024-09-11T07:46:08.334 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7624999999999999e-05 2024-09-11T07:46:08.334 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.334 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.334 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:08.334 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.953326+0000", 2024-09-11T07:46:08.334 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000119883 2024-09-11T07:46:08.334 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.334 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:08.334 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.334 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.334 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.334 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:231567 9.6 9:62bf3667:::benchmark_data_smithi022_133943_object104976:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:46:08.334 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:56.613955+0000", 2024-09-11T07:46:08.335 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 311.63174995700001, 2024-09-11T07:46:08.335 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14854477399999999, 2024-09-11T07:46:08.335 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:08.335 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:08.335 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:08.335 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:08.335 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:08.335 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 231567 2024-09-11T07:46:08.335 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.335 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:08.335 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.335 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:08.335 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.613955+0000", 2024-09-11T07:46:08.335 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.335 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.336 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.336 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:08.336 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.613955+0000", 2024-09-11T07:46:08.336 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.336 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.336 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.336 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:08.336 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.613953+0000", 2024-09-11T07:46:08.336 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:46:08.336 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.336 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.336 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:08.336 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.613960+0000", 2024-09-11T07:46:08.337 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3200000000000002e-06 2024-09-11T07:46:08.337 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.337 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.337 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:08.337 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.613988+0000", 2024-09-11T07:46:08.337 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.745e-05 2024-09-11T07:46:08.337 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.337 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.337 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:08.337 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.613994+0000", 2024-09-11T07:46:08.337 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0270000000000001e-06 2024-09-11T07:46:08.337 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.337 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.337 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:08.338 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.614040+0000", 2024-09-11T07:46:08.338 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6403999999999998e-05 2024-09-11T07:46:08.338 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.338 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.338 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:08.338 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.614209+0000", 2024-09-11T07:46:08.338 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016869000000000001 2024-09-11T07:46:08.338 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.338 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.338 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:46:08.338 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.614696+0000", 2024-09-11T07:46:08.338 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00048757100000000001 2024-09-11T07:46:08.338 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.338 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.338 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:08.339 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.617583+0000", 2024-09-11T07:46:08.339 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0028868129999999998 2024-09-11T07:46:08.339 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.339 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.339 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.339 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.618363+0000", 2024-09-11T07:46:08.339 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00078008600000000004 2024-09-11T07:46:08.339 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.339 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.339 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.339 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.762435+0000", 2024-09-11T07:46:08.339 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.144071592 2024-09-11T07:46:08.339 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.339 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.340 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:08.340 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.762464+0000", 2024-09-11T07:46:08.340 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9068999999999999e-05 2024-09-11T07:46:08.340 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.340 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.340 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:08.340 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.762500+0000", 2024-09-11T07:46:08.340 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5751999999999999e-05 2024-09-11T07:46:08.340 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.340 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:08.340 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.340 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.340 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.341 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:232544 9.13 9:cb4ef5ab:::benchmark_data_smithi022_133943_object105953:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:46:08.341 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:57.689887+0000", 2024-09-11T07:46:08.341 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 310.55581784700001, 2024-09-11T07:46:08.341 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.186799876, 2024-09-11T07:46:08.341 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:08.341 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:08.341 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:08.341 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:08.341 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:08.341 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 232544 2024-09-11T07:46:08.341 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.341 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:08.341 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.341 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:08.342 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689887+0000", 2024-09-11T07:46:08.342 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.342 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.342 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.342 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:08.342 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689887+0000", 2024-09-11T07:46:08.342 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.342 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.342 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.342 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:08.342 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689885+0000", 2024-09-11T07:46:08.342 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:46:08.342 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.342 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.343 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:08.343 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689891+0000", 2024-09-11T07:46:08.343 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0310000000000004e-06 2024-09-11T07:46:08.343 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.343 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.343 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:08.343 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689903+0000", 2024-09-11T07:46:08.343 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2167000000000001e-05 2024-09-11T07:46:08.343 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.343 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.343 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:08.343 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689909+0000", 2024-09-11T07:46:08.343 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7620000000000001e-06 2024-09-11T07:46:08.343 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.344 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.344 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:08.344 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689947+0000", 2024-09-11T07:46:08.344 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7960999999999998e-05 2024-09-11T07:46:08.344 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.344 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.344 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:08.344 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.690025+0000", 2024-09-11T07:46:08.344 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7885000000000004e-05 2024-09-11T07:46:08.344 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.344 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.344 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:46:08.344 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.690473+0000", 2024-09-11T07:46:08.344 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00044783300000000002 2024-09-11T07:46:08.345 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.345 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.345 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:08.345 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.692445+0000", 2024-09-11T07:46:08.345 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001972461 2024-09-11T07:46:08.345 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.345 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.345 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.345 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.692900+0000", 2024-09-11T07:46:08.345 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00045435300000000001 2024-09-11T07:46:08.345 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.345 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.345 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.345 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.876606+0000", 2024-09-11T07:46:08.346 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18370584700000001 2024-09-11T07:46:08.346 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.346 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.346 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:08.346 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.876647+0000", 2024-09-11T07:46:08.346 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1140999999999998e-05 2024-09-11T07:46:08.346 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.346 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.346 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:08.346 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.876687+0000", 2024-09-11T07:46:08.346 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0153e-05 2024-09-11T07:46:08.346 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.346 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:08.346 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.347 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.347 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.347 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:234546 9.6 9:65687430:::benchmark_data_smithi022_133943_object107955:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e320)", 2024-09-11T07:46:08.347 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:41:00.355245+0000", 2024-09-11T07:46:08.347 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 307.89045961400001, 2024-09-11T07:46:08.347 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17097563599999999, 2024-09-11T07:46:08.347 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:08.347 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:08.347 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:08.347 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:08.347 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:08.347 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 234546 2024-09-11T07:46:08.347 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.347 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:08.348 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.348 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:08.348 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355245+0000", 2024-09-11T07:46:08.348 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.348 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.348 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.348 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:08.348 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355245+0000", 2024-09-11T07:46:08.348 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.348 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.348 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.348 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:08.348 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355244+0000", 2024-09-11T07:46:08.348 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:46:08.349 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.349 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.349 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:08.349 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355248+0000", 2024-09-11T07:46:08.349 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.529e-06 2024-09-11T07:46:08.349 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.349 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.349 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:08.349 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355255+0000", 2024-09-11T07:46:08.349 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1289999999999998e-06 2024-09-11T07:46:08.349 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.349 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.349 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:08.349 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355259+0000", 2024-09-11T07:46:08.350 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7149999999999997e-06 2024-09-11T07:46:08.350 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.350 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.350 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:08.350 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355306+0000", 2024-09-11T07:46:08.350 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6631999999999997e-05 2024-09-11T07:46:08.350 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.350 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.350 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:08.350 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355383+0000", 2024-09-11T07:46:08.350 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6509999999999998e-05 2024-09-11T07:46:08.350 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.350 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.350 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:46:08.351 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355796+0000", 2024-09-11T07:46:08.351 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00041318000000000003 2024-09-11T07:46:08.351 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.351 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.351 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:08.351 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.357852+0000", 2024-09-11T07:46:08.351 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0020561059999999998 2024-09-11T07:46:08.351 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.351 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.351 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.351 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.359012+0000", 2024-09-11T07:46:08.351 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001160295 2024-09-11T07:46:08.351 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.351 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.352 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:08.352 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.526128+0000", 2024-09-11T07:46:08.352 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.167115923 2024-09-11T07:46:08.352 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.352 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.352 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:08.352 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.526176+0000", 2024-09-11T07:46:08.352 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8310000000000003e-05 2024-09-11T07:46:08.352 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.352 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.352 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:08.352 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.526221+0000", 2024-09-11T07:46:08.352 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4447000000000003e-05 2024-09-11T07:46:08.352 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.353 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:08.353 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.353 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.353 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.353 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:70288 10.6 e356/346 10:6714b47c:::benchmark_data_smithi022_163866_object70287:head v 356'4440, mlcod=356'4437)", 2024-09-11T07:46:08.353 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:53.786739+0000", 2024-09-11T07:46:08.353 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 194.45896617400001, 2024-09-11T07:46:08.353 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.164540297, 2024-09-11T07:46:08.353 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:08.353 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:08.353 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:08.353 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.353 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:08.354 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786739+0000", 2024-09-11T07:46:08.354 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.354 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.354 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.354 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:08.354 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786739+0000", 2024-09-11T07:46:08.354 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.354 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.354 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.354 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:08.354 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786737+0000", 2024-09-11T07:46:08.354 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:46:08.354 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.354 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.355 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:08.355 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786793+0000", 2024-09-11T07:46:08.355 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.516e-05 2024-09-11T07:46:08.355 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.355 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.355 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:08.355 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786802+0000", 2024-09-11T07:46:08.355 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0149999999999992e-06 2024-09-11T07:46:08.355 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.355 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.355 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:08.355 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786807+0000", 2024-09-11T07:46:08.355 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3900000000000001e-06 2024-09-11T07:46:08.355 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.356 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.356 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:08.356 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786852+0000", 2024-09-11T07:46:08.356 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5426000000000002e-05 2024-09-11T07:46:08.356 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.356 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.356 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:08.356 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786869+0000", 2024-09-11T07:46:08.356 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6477000000000002e-05 2024-09-11T07:46:08.356 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.356 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.356 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:08.356 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951240+0000", 2024-09-11T07:46:08.356 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16437090500000001 2024-09-11T07:46:08.357 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.357 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.357 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:08.357 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951279+0000", 2024-09-11T07:46:08.357 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9072999999999997e-05 2024-09-11T07:46:08.357 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.357 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:08.357 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.357 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.357 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.357 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:70292 10.a e356/341 10:547c94c8:::benchmark_data_smithi022_163866_object70291:head v 356'4331, mlcod=356'4329)", 2024-09-11T07:46:08.357 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:53.788575+0000", 2024-09-11T07:46:08.357 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 194.45712946500001, 2024-09-11T07:46:08.357 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16920853899999999, 2024-09-11T07:46:08.358 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:08.358 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:08.358 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:08.358 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.358 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:08.358 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.788575+0000", 2024-09-11T07:46:08.358 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.358 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.358 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.358 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:08.358 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.788575+0000", 2024-09-11T07:46:08.358 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.358 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.358 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.359 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:08.359 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.788574+0000", 2024-09-11T07:46:08.359 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:46:08.359 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.359 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.359 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:08.359 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.788630+0000", 2024-09-11T07:46:08.359 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6097999999999997e-05 2024-09-11T07:46:08.359 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.359 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.359 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:08.359 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951213+0000", 2024-09-11T07:46:08.359 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16258271199999999 2024-09-11T07:46:08.359 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.359 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.360 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:08.360 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951228+0000", 2024-09-11T07:46:08.360 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4562999999999999e-05 2024-09-11T07:46:08.360 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.360 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.360 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:08.360 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951332+0000", 2024-09-11T07:46:08.360 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000104002 2024-09-11T07:46:08.360 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.360 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.360 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:08.360 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951352+0000", 2024-09-11T07:46:08.360 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0638000000000001e-05 2024-09-11T07:46:08.360 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.361 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.361 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:08.361 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.957755+0000", 2024-09-11T07:46:08.361 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.006402451 2024-09-11T07:46:08.361 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.361 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.361 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:08.361 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.957784+0000", 2024-09-11T07:46:08.361 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9269e-05 2024-09-11T07:46:08.361 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.361 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:08.361 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.361 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.362 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.362 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:70291 10.3 e356/341 10:cdbcfc7c:::benchmark_data_smithi022_163866_object70290:head v 356'4299, mlcod=356'4297)", 2024-09-11T07:46:08.362 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:53.789090+0000", 2024-09-11T07:46:08.362 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 194.45661462499999, 2024-09-11T07:46:08.362 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16609167599999999, 2024-09-11T07:46:08.362 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:08.362 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:08.362 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:08.362 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.362 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:08.362 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.789090+0000", 2024-09-11T07:46:08.362 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.362 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.362 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.362 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:08.363 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.789090+0000", 2024-09-11T07:46:08.363 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:08.363 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.363 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.363 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:08.363 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.789089+0000", 2024-09-11T07:46:08.363 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:46:08.363 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.363 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.363 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:08.363 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.789132+0000", 2024-09-11T07:46:08.363 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2901000000000001e-05 2024-09-11T07:46:08.363 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.363 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.364 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:08.364 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951213+0000", 2024-09-11T07:46:08.364 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16208165599999999 2024-09-11T07:46:08.364 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.364 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.364 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:08.364 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951228+0000", 2024-09-11T07:46:08.364 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4430000000000001e-05 2024-09-11T07:46:08.364 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.364 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.364 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:08.364 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951350+0000", 2024-09-11T07:46:08.364 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000122213 2024-09-11T07:46:08.365 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.365 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.365 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:08.365 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951369+0000", 2024-09-11T07:46:08.365 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8819000000000001e-05 2024-09-11T07:46:08.365 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.365 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.365 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:08.365 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.955146+0000", 2024-09-11T07:46:08.365 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003776916 2024-09-11T07:46:08.365 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:08.365 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:08.365 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:08.365 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.955182+0000", 2024-09-11T07:46:08.366 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6123e-05 2024-09-11T07:46:08.366 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.366 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:08.366 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.366 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:08.366 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:08.366 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:46:08.366 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:08.367 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:08.380 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:46:08.380 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:46:08.396 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:46:08.396 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:46:08.414 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:46:08.415 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:46:08.429 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:46:08.430 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:46:08.446 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:46:08.446 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:46:08.461 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:46:08.461 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:46:08.468 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:08.476 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:46:08.476 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:46:08.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:08.491 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:08.491 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:08.492 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:46:08.493 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:46:08.592 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:08.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:08.616 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:08.616 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:08.717 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:08.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:08.740 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:08.740 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:08.841 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:08.863 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:08.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:08.864 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:08.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:08 smithi022 ceph-mon[29462]: 5.9 scrub starts 2024-09-11T07:46:08.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:08 smithi022 ceph-mon[29462]: 5.9 scrub ok 2024-09-11T07:46:08.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:08 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2828515501' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:46:08.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:08 smithi022 ceph-mon[29462]: osdmap e417: 8 total, 8 up, 7 in 2024-09-11T07:46:08.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:46:08.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:46:08.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:08 smithi022 ceph-mon[29462]: 4.2 scrub starts 2024-09-11T07:46:08.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:08 smithi022 ceph-mon[29462]: 4.2 scrub ok 2024-09-11T07:46:08.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:08 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3221378418' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:46:08.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:08 smithi022 ceph-mon[29462]: pgmap v918: 158 pgs: 158 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:46:08.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:08 smithi022 ceph-mon[37330]: 5.9 scrub starts 2024-09-11T07:46:08.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:08 smithi022 ceph-mon[37330]: 5.9 scrub ok 2024-09-11T07:46:08.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:08 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2828515501' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:46:08.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:08 smithi022 ceph-mon[37330]: osdmap e417: 8 total, 8 up, 7 in 2024-09-11T07:46:08.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:46:08.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:46:08.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:08 smithi022 ceph-mon[37330]: 4.2 scrub starts 2024-09-11T07:46:08.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:08 smithi022 ceph-mon[37330]: 4.2 scrub ok 2024-09-11T07:46:08.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:08 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3221378418' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:46:08.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:08 smithi022 ceph-mon[37330]: pgmap v918: 158 pgs: 158 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:46:08.965 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:08.987 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:08.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:08.988 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:09.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:08 smithi104 ceph-mon[33507]: 5.9 scrub starts 2024-09-11T07:46:09.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:08 smithi104 ceph-mon[33507]: 5.9 scrub ok 2024-09-11T07:46:09.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:08 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2828515501' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:46:09.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:08 smithi104 ceph-mon[33507]: osdmap e417: 8 total, 8 up, 7 in 2024-09-11T07:46:09.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:46:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:46:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:08 smithi104 ceph-mon[33507]: 4.2 scrub starts 2024-09-11T07:46:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:08 smithi104 ceph-mon[33507]: 4.2 scrub ok 2024-09-11T07:46:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:08 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3221378418' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:46:09.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:08 smithi104 ceph-mon[33507]: pgmap v918: 158 pgs: 158 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:46:09.089 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:09.112 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:09.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:09.113 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:09.135 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-11T07:46:09.213 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:09.236 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:09.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:09.237 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:09.338 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:09.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:09.361 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:09.362 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:09.380 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:46:09.463 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:09.484 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:09.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:09.485 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:09.586 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:09.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:09.609 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:09.609 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:09.710 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:09.726 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:46:09.734 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:09.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:09.735 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:09.752 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:09 smithi104 ceph-mon[33507]: 4.1 scrub starts 2024-09-11T07:46:09.752 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:09 smithi104 ceph-mon[33507]: 4.1 scrub ok 2024-09-11T07:46:09.752 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:09 smithi104 ceph-mon[33507]: 6.1c scrub starts 2024-09-11T07:46:09.753 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:09 smithi104 ceph-mon[33507]: 6.1c scrub ok 2024-09-11T07:46:09.753 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:09 smithi104 ceph-mon[33507]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:46:09.753 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:09 smithi104 ceph-mon[33507]: 5.c scrub starts 2024-09-11T07:46:09.753 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:09 smithi104 ceph-mon[33507]: 5.c scrub ok 2024-09-11T07:46:09.836 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:09.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:09.859 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:09.859 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:09.960 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:09.982 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:09.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:09.983 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:09.996 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:09 smithi022 ceph-mon[29462]: 4.1 scrub starts 2024-09-11T07:46:09.996 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:09 smithi022 ceph-mon[29462]: 4.1 scrub ok 2024-09-11T07:46:09.996 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:09 smithi022 ceph-mon[29462]: 6.1c scrub starts 2024-09-11T07:46:09.996 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:09 smithi022 ceph-mon[29462]: 6.1c scrub ok 2024-09-11T07:46:09.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:09 smithi022 ceph-mon[29462]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:46:09.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:09 smithi022 ceph-mon[29462]: 5.c scrub starts 2024-09-11T07:46:09.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:09 smithi022 ceph-mon[29462]: 5.c scrub ok 2024-09-11T07:46:09.997 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:09 smithi022 ceph-mon[37330]: 4.1 scrub starts 2024-09-11T07:46:09.997 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:09 smithi022 ceph-mon[37330]: 4.1 scrub ok 2024-09-11T07:46:09.997 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:09 smithi022 ceph-mon[37330]: 6.1c scrub starts 2024-09-11T07:46:09.997 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:09 smithi022 ceph-mon[37330]: 6.1c scrub ok 2024-09-11T07:46:09.997 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:09 smithi022 ceph-mon[37330]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:46:09.997 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:09 smithi022 ceph-mon[37330]: 5.c scrub starts 2024-09-11T07:46:09.997 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:09 smithi022 ceph-mon[37330]: 5.c scrub ok 2024-09-11T07:46:10.084 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:10.107 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:10.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:10.108 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:10.209 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:10.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:10.232 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:10.232 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:10.333 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:10.355 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:10.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:10.356 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:10.457 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:10.474 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:46:10 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:46:10.479 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:10.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:10.480 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:10.545 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:46:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:46:10] "GET /metrics HTTP/1.1" 200 224902 "" "Prometheus/2.43.0" 2024-09-11T07:46:10.581 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:10.603 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:10.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:10.604 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:10.705 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:10.725 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:10.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:10.726 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:10.827 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:10.849 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:10.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:10.850 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:10.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:10 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.0", "id": [1, 2]}]: dispatch 2024-09-11T07:46:10.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:10 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.a", "id": [0, 6]}]: dispatch 2024-09-11T07:46:10.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:10 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.16", "id": [1, 6]}]: dispatch 2024-09-11T07:46:10.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:10 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:46:10.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:10 smithi022 ceph-mon[29462]: 5.e scrub starts 2024-09-11T07:46:10.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:10 smithi022 ceph-mon[29462]: 5.e scrub ok 2024-09-11T07:46:10.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:10 smithi022 ceph-mon[29462]: pgmap v919: 158 pgs: 158 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T07:46:10.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:10 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.0", "id": [1, 2]}]: dispatch 2024-09-11T07:46:10.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:10 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.a", "id": [0, 6]}]: dispatch 2024-09-11T07:46:10.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:10 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.16", "id": [1, 6]}]: dispatch 2024-09-11T07:46:10.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:10 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:46:10.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:10 smithi022 ceph-mon[37330]: 5.e scrub starts 2024-09-11T07:46:10.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:10 smithi022 ceph-mon[37330]: 5.e scrub ok 2024-09-11T07:46:10.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:10 smithi022 ceph-mon[37330]: pgmap v919: 158 pgs: 158 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T07:46:10.950 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:10.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:10.977 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:10.977 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:11.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:10 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.0", "id": [1, 2]}]: dispatch 2024-09-11T07:46:11.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:10 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.a", "id": [0, 6]}]: dispatch 2024-09-11T07:46:11.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:10 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.16", "id": [1, 6]}]: dispatch 2024-09-11T07:46:11.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:10 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:46:11.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:10 smithi104 ceph-mon[33507]: 5.e scrub starts 2024-09-11T07:46:11.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:10 smithi104 ceph-mon[33507]: 5.e scrub ok 2024-09-11T07:46:11.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:10 smithi104 ceph-mon[33507]: pgmap v919: 158 pgs: 158 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T07:46:11.078 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:11.095 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:46:11.095 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:46:11.100 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:11.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:11.101 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:11.202 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:11.225 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:11.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:11.226 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:11.326 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:11.348 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:11.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:11.349 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:11.362 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:46:11.450 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:11.472 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:11.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:11.473 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:11.574 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:11.596 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:11.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:11.598 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:11.699 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:11.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:11.723 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:11.723 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:11.733 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-11T07:46:11.823 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:11.845 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:11.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:11.846 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:11.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:11 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.0", "id": [1, 2]}]': finished 2024-09-11T07:46:11.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:11 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.a", "id": [0, 6]}]': finished 2024-09-11T07:46:11.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:11 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.16", "id": [1, 6]}]': finished 2024-09-11T07:46:11.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:11 smithi022 ceph-mon[29462]: osdmap e418: 8 total, 8 up, 7 in 2024-09-11T07:46:11.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:11 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.0", "id": [1, 2]}]': finished 2024-09-11T07:46:11.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:11 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.a", "id": [0, 6]}]': finished 2024-09-11T07:46:11.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:11 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.16", "id": [1, 6]}]': finished 2024-09-11T07:46:11.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:11 smithi022 ceph-mon[37330]: osdmap e418: 8 total, 8 up, 7 in 2024-09-11T07:46:11.947 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:11.968 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:11.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:11.969 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:11.980 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:46:11.998 DEBUG:teuthology.orchestra.run.smithi022:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_4 bench 90 write' 2024-09-11T07:46:12.001 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-11T07:46:12.001 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-11T07:46:12.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:11 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.0", "id": [1, 2]}]': finished 2024-09-11T07:46:12.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:11 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.a", "id": [0, 6]}]': finished 2024-09-11T07:46:12.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:11 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.16", "id": [1, 6]}]': finished 2024-09-11T07:46:12.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:11 smithi104 ceph-mon[33507]: osdmap e418: 8 total, 8 up, 7 in 2024-09-11T07:46:12.071 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:12.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:12.094 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:12.094 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:12.098 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:hints = 1 2024-09-11T07:46:12.098 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-11T07:46:12.099 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Object prefix: benchmark_data_smithi022_188626 2024-09-11T07:46:12.099 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T07:46:12.099 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 0 0 0 0 0 0 - 0 2024-09-11T07:46:12.195 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:12.223 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:12.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:12.224 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:12.325 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:12.354 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:12.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:12.355 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:12.456 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:12.483 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:12.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:12.484 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:12.503 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:46:12.585 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:12.628 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:12.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:12.629 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:12.730 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:12.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:12.769 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:12.770 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:12.780 INFO:teuthology.orchestra.run.smithi022.stderr:set 4.1f pg_upmap mapping to [3,5,6] 2024-09-11T07:46:12.871 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:12.897 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:12.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:12.898 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:12.999 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:13.026 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:13.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:13.027 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:13.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:12 smithi104 ceph-mon[33507]: osdmap e419: 8 total, 8 up, 7 in 2024-09-11T07:46:13.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:12 smithi104 ceph-mon[33507]: pgmap v922: 158 pgs: 158 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:46:13.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:12 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1892986378' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "4.1f", "id": [3, 5, 6]}]: dispatch 2024-09-11T07:46:13.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:12 smithi104 ceph-mon[33507]: from='client.25976 ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "4.1f", "id": [3, 5, 6]}]: dispatch 2024-09-11T07:46:13.099 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 1 16 2905 2889 180.552 180.562 0.00236179 0.00552324 2024-09-11T07:46:13.129 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:13.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:12 smithi022 ceph-mon[29462]: osdmap e419: 8 total, 8 up, 7 in 2024-09-11T07:46:13.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:12 smithi022 ceph-mon[29462]: pgmap v922: 158 pgs: 158 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:46:13.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:12 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1892986378' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "4.1f", "id": [3, 5, 6]}]: dispatch 2024-09-11T07:46:13.130 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:12 smithi022 ceph-mon[29462]: from='client.25976 ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "4.1f", "id": [3, 5, 6]}]: dispatch 2024-09-11T07:46:13.130 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:12 smithi022 ceph-mon[37330]: osdmap e419: 8 total, 8 up, 7 in 2024-09-11T07:46:13.131 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:12 smithi022 ceph-mon[37330]: pgmap v922: 158 pgs: 158 active+clean; 582 KiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:46:13.131 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:12 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1892986378' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "4.1f", "id": [3, 5, 6]}]: dispatch 2024-09-11T07:46:13.131 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:12 smithi022 ceph-mon[37330]: from='client.25976 ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "4.1f", "id": [3, 5, 6]}]: dispatch 2024-09-11T07:46:13.155 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:13.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:13.157 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:13.257 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:13.289 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:13.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:13.291 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:13.392 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:13.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:13.435 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:13.435 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:13.536 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:13.571 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:13.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:13.572 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:13.673 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:13.706 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:13.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:13.707 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:13.782 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is unset 2024-09-11T07:46:13.808 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:13.817 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:46:13.817 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:46:13.838 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:13.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:13.839 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:13.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:13 smithi022 ceph-mon[29462]: from='client.25976 ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "4.1f", "id": [3, 5, 6]}]': finished 2024-09-11T07:46:13.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:13 smithi022 ceph-mon[29462]: osdmap e420: 8 total, 8 up, 7 in 2024-09-11T07:46:13.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:13 smithi022 ceph-mon[29462]: 2.8 scrub starts 2024-09-11T07:46:13.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:13 smithi022 ceph-mon[29462]: 2.8 scrub ok 2024-09-11T07:46:13.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:13 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2900490858' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:46:13.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:13 smithi022 ceph-mon[29462]: from='client.26002 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:46:13.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:13 smithi022 ceph-mon[37330]: from='client.25976 ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "4.1f", "id": [3, 5, 6]}]': finished 2024-09-11T07:46:13.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:13 smithi022 ceph-mon[37330]: osdmap e420: 8 total, 8 up, 7 in 2024-09-11T07:46:13.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:13 smithi022 ceph-mon[37330]: 2.8 scrub starts 2024-09-11T07:46:13.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:13 smithi022 ceph-mon[37330]: 2.8 scrub ok 2024-09-11T07:46:13.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:13 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2900490858' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:46:13.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:13 smithi022 ceph-mon[37330]: from='client.26002 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:46:13.939 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:13.983 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:13.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:13.984 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:14.043 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:13 smithi104 ceph-mon[33507]: from='client.25976 ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "4.1f", "id": [3, 5, 6]}]': finished 2024-09-11T07:46:14.043 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:13 smithi104 ceph-mon[33507]: osdmap e420: 8 total, 8 up, 7 in 2024-09-11T07:46:14.044 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:13 smithi104 ceph-mon[33507]: 2.8 scrub starts 2024-09-11T07:46:14.044 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:13 smithi104 ceph-mon[33507]: 2.8 scrub ok 2024-09-11T07:46:14.044 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:13 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2900490858' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:46:14.044 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:13 smithi104 ceph-mon[33507]: from='client.26002 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:46:14.085 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:14.099 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 2 16 5943 5927 185.205 189.875 0.00426318 0.0053844 2024-09-11T07:46:14.113 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:14.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:14.114 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:14.215 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:14.254 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:14.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:14.256 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:14.356 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:14.385 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:14.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:14.386 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:14.469 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:46:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:46:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:46:14.487 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:14.505 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-11T07:46:14.525 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:14.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:14.526 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:14.627 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:14.657 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:14.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:14.658 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:14.759 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:14.802 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:14.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:14.803 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:14.858 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:46:14.903 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:14.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:14 smithi022 ceph-mon[29462]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:46:14.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:14 smithi022 ceph-mon[29462]: Cluster is now healthy 2024-09-11T07:46:14.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:14 smithi022 ceph-mon[29462]: from='client.26002 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:46:14.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:14 smithi022 ceph-mon[29462]: osdmap e421: 8 total, 8 up, 7 in 2024-09-11T07:46:14.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:14 smithi022 ceph-mon[29462]: 2.10 scrub starts 2024-09-11T07:46:14.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:14 smithi022 ceph-mon[29462]: 2.10 scrub ok 2024-09-11T07:46:14.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:14 smithi022 ceph-mon[29462]: pgmap v925: 158 pgs: 158 active+clean; 202 MiB data, 6.5 GiB used, 619 GiB / 626 GiB avail; 1.2 KiB/s rd, 50 MiB/s wr, 805 op/s; 54 B/s, 1 objects/s recovering 2024-09-11T07:46:14.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:14 smithi022 ceph-mon[37330]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:46:14.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:14 smithi022 ceph-mon[37330]: Cluster is now healthy 2024-09-11T07:46:14.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:14 smithi022 ceph-mon[37330]: from='client.26002 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:46:14.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:14 smithi022 ceph-mon[37330]: osdmap e421: 8 total, 8 up, 7 in 2024-09-11T07:46:14.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:14 smithi022 ceph-mon[37330]: 2.10 scrub starts 2024-09-11T07:46:14.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:14 smithi022 ceph-mon[37330]: 2.10 scrub ok 2024-09-11T07:46:14.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:14 smithi022 ceph-mon[37330]: pgmap v925: 158 pgs: 158 active+clean; 202 MiB data, 6.5 GiB used, 619 GiB / 626 GiB avail; 1.2 KiB/s rd, 50 MiB/s wr, 805 op/s; 54 B/s, 1 objects/s recovering 2024-09-11T07:46:14.942 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:14.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:14.943 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:15.044 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:15.082 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:15.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:15.082 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:15.099 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 3 16 8911 8895 185.298 185.5 0.00556135 0.00538443 2024-09-11T07:46:15.121 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:14 smithi104 ceph-mon[33507]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:46:15.122 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:14 smithi104 ceph-mon[33507]: Cluster is now healthy 2024-09-11T07:46:15.122 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:14 smithi104 ceph-mon[33507]: from='client.26002 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:46:15.122 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:14 smithi104 ceph-mon[33507]: osdmap e421: 8 total, 8 up, 7 in 2024-09-11T07:46:15.122 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:14 smithi104 ceph-mon[33507]: 2.10 scrub starts 2024-09-11T07:46:15.122 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:14 smithi104 ceph-mon[33507]: 2.10 scrub ok 2024-09-11T07:46:15.122 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:14 smithi104 ceph-mon[33507]: pgmap v925: 158 pgs: 158 active+clean; 202 MiB data, 6.5 GiB used, 619 GiB / 626 GiB avail; 1.2 KiB/s rd, 50 MiB/s wr, 805 op/s; 54 B/s, 1 objects/s recovering 2024-09-11T07:46:15.183 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:15.220 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:15.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:15.221 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:15.322 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:15.344 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:15.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:15.345 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:15.445 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:15.472 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:15.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:15.473 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:15.574 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:15.596 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:15.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:15.597 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:15.698 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:15.737 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:15.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:15.738 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:15.839 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:15.868 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:15.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:15.869 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:15.970 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:16.010 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:16.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:16.011 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:16.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:15 smithi104 ceph-mon[33507]: 4.8 deep-scrub starts 2024-09-11T07:46:16.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:15 smithi104 ceph-mon[33507]: 4.8 deep-scrub ok 2024-09-11T07:46:16.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:15 smithi104 ceph-mon[33507]: 6.e scrub starts 2024-09-11T07:46:16.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:15 smithi104 ceph-mon[33507]: 6.e scrub ok 2024-09-11T07:46:16.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:15 smithi104 ceph-mon[33507]: 5.1e scrub starts 2024-09-11T07:46:16.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:15 smithi104 ceph-mon[33507]: 5.1e scrub ok 2024-09-11T07:46:16.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:15 smithi104 ceph-mon[33507]: osdmap e422: 8 total, 8 up, 7 in 2024-09-11T07:46:16.099 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 4 15 11241 11226 175.392 145.688 0.00413692 0.00569338 2024-09-11T07:46:16.112 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:16.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:16.158 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:16.158 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:16.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:15 smithi022 ceph-mon[37330]: 4.8 deep-scrub starts 2024-09-11T07:46:16.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:15 smithi022 ceph-mon[37330]: 4.8 deep-scrub ok 2024-09-11T07:46:16.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:15 smithi022 ceph-mon[37330]: 6.e scrub starts 2024-09-11T07:46:16.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:15 smithi022 ceph-mon[37330]: 6.e scrub ok 2024-09-11T07:46:16.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:15 smithi022 ceph-mon[37330]: 5.1e scrub starts 2024-09-11T07:46:16.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:15 smithi022 ceph-mon[37330]: 5.1e scrub ok 2024-09-11T07:46:16.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:15 smithi022 ceph-mon[37330]: osdmap e422: 8 total, 8 up, 7 in 2024-09-11T07:46:16.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:15 smithi022 ceph-mon[29462]: 4.8 deep-scrub starts 2024-09-11T07:46:16.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:15 smithi022 ceph-mon[29462]: 4.8 deep-scrub ok 2024-09-11T07:46:16.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:15 smithi022 ceph-mon[29462]: 6.e scrub starts 2024-09-11T07:46:16.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:15 smithi022 ceph-mon[29462]: 6.e scrub ok 2024-09-11T07:46:16.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:15 smithi022 ceph-mon[29462]: 5.1e scrub starts 2024-09-11T07:46:16.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:15 smithi022 ceph-mon[29462]: 5.1e scrub ok 2024-09-11T07:46:16.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:15 smithi022 ceph-mon[29462]: osdmap e422: 8 total, 8 up, 7 in 2024-09-11T07:46:16.259 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:16.298 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:16.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:16.300 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:16.400 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:16.425 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:16.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:16.426 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:16.527 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:16.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:16.559 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:16.559 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:16.659 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:16.689 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:16.690 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:46:16.690 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:46:16.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:16.691 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:16.792 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:16.818 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:16.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:16.819 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:16.920 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:16.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:16.966 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:16.966 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:16 smithi104 ceph-mon[33507]: pgmap v927: 158 pgs: 158 active+clean; 459 MiB data, 7.0 GiB used, 619 GiB / 626 GiB avail; 1.1 KiB/s rd, 99 MiB/s wr, 1.58k op/s; 71 B/s, 2 objects/s recovering 2024-09-11T07:46:17.066 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:17.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:17.094 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:17.094 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:17.099 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 5 16 13582 13566 169.56 146.25 0.00145518 0.00586071 2024-09-11T07:46:17.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:16 smithi022 ceph-mon[29462]: pgmap v927: 158 pgs: 158 active+clean; 459 MiB data, 7.0 GiB used, 619 GiB / 626 GiB avail; 1.1 KiB/s rd, 99 MiB/s wr, 1.58k op/s; 71 B/s, 2 objects/s recovering 2024-09-11T07:46:17.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:16 smithi022 ceph-mon[37330]: pgmap v927: 158 pgs: 158 active+clean; 459 MiB data, 7.0 GiB used, 619 GiB / 626 GiB avail; 1.1 KiB/s rd, 99 MiB/s wr, 1.58k op/s; 71 B/s, 2 objects/s recovering 2024-09-11T07:46:17.195 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:17.233 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:17.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:17.234 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:17.335 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:17.362 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:17.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:17.363 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:17.464 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:17.492 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:17.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:17.493 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:17.594 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:17.622 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:17.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:17.622 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:17.724 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:17.751 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:17.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:17.752 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:17.853 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:17.879 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:17.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:17.880 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:17.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:17 smithi022 ceph-mon[29462]: 2.d scrub starts 2024-09-11T07:46:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:17 smithi022 ceph-mon[29462]: 2.d scrub ok 2024-09-11T07:46:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:17 smithi022 ceph-mon[29462]: 4.4 scrub starts 2024-09-11T07:46:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:17 smithi022 ceph-mon[29462]: 4.4 scrub ok 2024-09-11T07:46:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:17 smithi022 ceph-mon[29462]: 2.16 scrub starts 2024-09-11T07:46:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:17 smithi022 ceph-mon[29462]: 2.16 scrub ok 2024-09-11T07:46:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:17 smithi022 ceph-mon[29462]: 6.0 scrub starts 2024-09-11T07:46:17.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:17 smithi022 ceph-mon[29462]: 6.0 scrub ok 2024-09-11T07:46:17.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:17 smithi022 ceph-mon[37330]: 2.d scrub starts 2024-09-11T07:46:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:17 smithi022 ceph-mon[37330]: 2.d scrub ok 2024-09-11T07:46:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:17 smithi022 ceph-mon[37330]: 4.4 scrub starts 2024-09-11T07:46:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:17 smithi022 ceph-mon[37330]: 4.4 scrub ok 2024-09-11T07:46:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:17 smithi022 ceph-mon[37330]: 2.16 scrub starts 2024-09-11T07:46:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:17 smithi022 ceph-mon[37330]: 2.16 scrub ok 2024-09-11T07:46:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:17 smithi022 ceph-mon[37330]: 6.0 scrub starts 2024-09-11T07:46:17.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:17 smithi022 ceph-mon[37330]: 6.0 scrub ok 2024-09-11T07:46:17.981 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:18.007 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:18.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:18.008 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:18.100 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 6 16 16157 16141 168.121 160.938 0.00507465 0.00594008 2024-09-11T07:46:18.108 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:18.137 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:18.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:18.138 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:18.164 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:17 smithi104 ceph-mon[33507]: 2.d scrub starts 2024-09-11T07:46:18.164 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:17 smithi104 ceph-mon[33507]: 2.d scrub ok 2024-09-11T07:46:18.164 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:17 smithi104 ceph-mon[33507]: 4.4 scrub starts 2024-09-11T07:46:18.164 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:17 smithi104 ceph-mon[33507]: 4.4 scrub ok 2024-09-11T07:46:18.164 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:17 smithi104 ceph-mon[33507]: 2.16 scrub starts 2024-09-11T07:46:18.164 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:17 smithi104 ceph-mon[33507]: 2.16 scrub ok 2024-09-11T07:46:18.164 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:17 smithi104 ceph-mon[33507]: 6.0 scrub starts 2024-09-11T07:46:18.164 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:17 smithi104 ceph-mon[33507]: 6.0 scrub ok 2024-09-11T07:46:18.239 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:18.281 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:18.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:18.282 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:18.329 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:46:18.329 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:46:18.360 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:46:18.361 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:46:18.383 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:18.393 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:46:18.393 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:46:18.423 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:18.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:18.424 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:18.427 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:46:18.427 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:46:18.460 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:46:18.460 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:46:18.493 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:46:18.493 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:46:18.525 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:18.526 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:46:18.526 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:46:18.527 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-11T07:46:18.558 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:46:18.558 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:46:18.564 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:18.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:18.566 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:18.667 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:18.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:18.696 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:18.696 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:18.797 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:18.836 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:18.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:18.837 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:18.858 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:46:18.938 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:18.965 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:18.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:18.966 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:19.067 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:19.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:19.093 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:19.093 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:19.093 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:46:19.094 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:46:19.094 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-11T07:46:19.094 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 0.000000 2024-09-11T07:46:19.094 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 5 0 2024-09-11T07:46:19.100 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 7 16 18190 18174 162.253 127.062 0.00392361 0.0061346 2024-09-11T07:46:19.121 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:18 smithi022 ceph-mon[29462]: 1.7 scrub starts 2024-09-11T07:46:19.121 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:18 smithi022 ceph-mon[29462]: 1.7 scrub ok 2024-09-11T07:46:19.121 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:18 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3205104693' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:46:19.122 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:18 smithi022 ceph-mon[29462]: pgmap v928: 158 pgs: 158 active+clean; 459 MiB data, 7.0 GiB used, 619 GiB / 626 GiB avail; 853 B/s rd, 76 MiB/s wr, 1.22k op/s; 54 B/s, 2 objects/s recovering 2024-09-11T07:46:19.122 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:18 smithi022 ceph-mon[37330]: 1.7 scrub starts 2024-09-11T07:46:19.122 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:18 smithi022 ceph-mon[37330]: 1.7 scrub ok 2024-09-11T07:46:19.122 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:18 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3205104693' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:46:19.122 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:18 smithi022 ceph-mon[37330]: pgmap v928: 158 pgs: 158 active+clean; 459 MiB data, 7.0 GiB used, 619 GiB / 626 GiB avail; 853 B/s rd, 76 MiB/s wr, 1.22k op/s; 54 B/s, 2 objects/s recovering 2024-09-11T07:46:19.126 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:18 smithi104 ceph-mon[33507]: 1.7 scrub starts 2024-09-11T07:46:19.126 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:18 smithi104 ceph-mon[33507]: 1.7 scrub ok 2024-09-11T07:46:19.126 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:18 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3205104693' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:46:19.126 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:18 smithi104 ceph-mon[33507]: pgmap v928: 158 pgs: 158 active+clean; 459 MiB data, 7.0 GiB used, 619 GiB / 626 GiB avail; 853 B/s rd, 76 MiB/s wr, 1.22k op/s; 54 B/s, 2 objects/s recovering 2024-09-11T07:46:19.194 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:19.221 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:19.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:19.223 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:19.324 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:19.369 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:19.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:19.370 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:19.471 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:19.499 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:19.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:19.500 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:19.601 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:19.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:19.642 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:19.642 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:19.688 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T07:46:19.744 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:19.773 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:19.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:19.775 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:19.875 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:19.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:19.923 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:19.923 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:20.025 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:20.069 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:20.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:20.070 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:20.100 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 8 16 20101 20085 156.9 119.438 0.00163944 0.0063569 2024-09-11T07:46:20.129 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:19 smithi104 ceph-mon[33507]: 2.12 scrub starts 2024-09-11T07:46:20.130 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:19 smithi104 ceph-mon[33507]: 2.12 scrub ok 2024-09-11T07:46:20.130 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:19 smithi104 ceph-mon[33507]: 6.10 scrub starts 2024-09-11T07:46:20.130 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:19 smithi104 ceph-mon[33507]: 6.10 scrub ok 2024-09-11T07:46:20.166 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:19 smithi022 ceph-mon[37330]: 2.12 scrub starts 2024-09-11T07:46:20.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:19 smithi022 ceph-mon[37330]: 2.12 scrub ok 2024-09-11T07:46:20.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:19 smithi022 ceph-mon[37330]: 6.10 scrub starts 2024-09-11T07:46:20.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:19 smithi022 ceph-mon[37330]: 6.10 scrub ok 2024-09-11T07:46:20.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:19 smithi022 ceph-mon[29462]: 2.12 scrub starts 2024-09-11T07:46:20.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:19 smithi022 ceph-mon[29462]: 2.12 scrub ok 2024-09-11T07:46:20.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:19 smithi022 ceph-mon[29462]: 6.10 scrub starts 2024-09-11T07:46:20.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:19 smithi022 ceph-mon[29462]: 6.10 scrub ok 2024-09-11T07:46:20.171 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:20.211 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:20.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:20.212 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:20.313 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:20.342 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:20.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:20.344 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:20.388 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:46:20 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:46:20.444 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:20.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:20.480 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:20.480 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:20.581 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:20.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:20.604 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:20.604 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:20.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:46:20 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:46:20] "GET /metrics HTTP/1.1" 200 224902 "" "Prometheus/2.43.0" 2024-09-11T07:46:20.705 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:20.732 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:20.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:20.733 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:20.834 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:20.875 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:20.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:20.876 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:20.977 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:20.987 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:46:20.987 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2024-09-11T07:46:20.988 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:20.988 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:21023 11.4 11:2846e557:::benchmark_data_smithi022_188626_object21022:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e422)", 2024-09-11T07:46:20.988 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:46:20.546362+0000", 2024-09-11T07:46:20.988 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 0.082150940000000006, 2024-09-11T07:46:20.988 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.082180460999999996, 2024-09-11T07:46:20.988 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:20.988 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "waiting for sub ops", 2024-09-11T07:46:20.988 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:20.988 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:46:20.988 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:46:20.988 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 21023 2024-09-11T07:46:20.988 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:20.989 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:20.989 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:20.989 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:20.989 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:20.546362+0000", 2024-09-11T07:46:20.990 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:20.990 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:20.990 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:20.990 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:20.990 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:20.546362+0000", 2024-09-11T07:46:20.990 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:20.991 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:20.991 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:20.991 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:20.991 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:20.546361+0000", 2024-09-11T07:46:20.991 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:46:20.991 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:20.991 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:20.992 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:20.992 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:20.546420+0000", 2024-09-11T07:46:20.992 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9011999999999997e-05 2024-09-11T07:46:20.992 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:20.992 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:20.992 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:20.993 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:20.546428+0000", 2024-09-11T07:46:20.993 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4579999999999993e-06 2024-09-11T07:46:20.993 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:20.993 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:20.993 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:20.993 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:20.546432+0000", 2024-09-11T07:46:20.993 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8500000000000004e-06 2024-09-11T07:46:20.994 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:20.994 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:20.994 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:20.994 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:20.546468+0000", 2024-09-11T07:46:20.994 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5661000000000003e-05 2024-09-11T07:46:20.994 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:20.995 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:20.995 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:20.995 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:20.546577+0000", 2024-09-11T07:46:20.995 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000109178 2024-09-11T07:46:20.995 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:20.995 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:20.996 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:46:20.996 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:20.546714+0000", 2024-09-11T07:46:20.996 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013684 2024-09-11T07:46:20.996 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:20.996 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:20.996 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:20.996 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:20.548227+0000", 2024-09-11T07:46:20.997 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001513503 2024-09-11T07:46:20.997 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:20.997 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:20.997 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:20.997 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:20.548269+0000", 2024-09-11T07:46:20.997 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1977999999999997e-05 2024-09-11T07:46:20.997 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:20.997 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:20.997 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:20.998 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:20.998 INFO:teuthology.orchestra.run.smithi104.stdout: ], 2024-09-11T07:46:20.998 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 1 2024-09-11T07:46:20.998 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:46:21.005 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:21.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:21.006 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:21.100 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 9 16 21863 21847 151.701 110.125 0.0128977 0.0065848 2024-09-11T07:46:21.107 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:21.135 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:21.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:21.136 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:21.166 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:20 smithi022 ceph-mon[37330]: 4.1a scrub starts 2024-09-11T07:46:21.166 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:20 smithi022 ceph-mon[37330]: 4.1a scrub ok 2024-09-11T07:46:21.166 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:20 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:46:21.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:20 smithi022 ceph-mon[37330]: 1.0 scrub starts 2024-09-11T07:46:21.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:20 smithi022 ceph-mon[37330]: 1.0 scrub ok 2024-09-11T07:46:21.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:20 smithi022 ceph-mon[37330]: pgmap v929: 158 pgs: 158 active+clean; 1.2 GiB data, 9.3 GiB used, 617 GiB / 626 GiB avail; 156 MiB/s wr, 2.50k op/s; 0 B/s, 1 objects/s recovering 2024-09-11T07:46:21.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:20 smithi022 ceph-mon[29462]: 4.1a scrub starts 2024-09-11T07:46:21.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:20 smithi022 ceph-mon[29462]: 4.1a scrub ok 2024-09-11T07:46:21.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:20 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:46:21.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:20 smithi022 ceph-mon[29462]: 1.0 scrub starts 2024-09-11T07:46:21.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:20 smithi022 ceph-mon[29462]: 1.0 scrub ok 2024-09-11T07:46:21.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:20 smithi022 ceph-mon[29462]: pgmap v929: 158 pgs: 158 active+clean; 1.2 GiB data, 9.3 GiB used, 617 GiB / 626 GiB avail; 156 MiB/s wr, 2.50k op/s; 0 B/s, 1 objects/s recovering 2024-09-11T07:46:21.237 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:21.264 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:21.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:21.266 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:21.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:20 smithi104 ceph-mon[33507]: 4.1a scrub starts 2024-09-11T07:46:21.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:20 smithi104 ceph-mon[33507]: 4.1a scrub ok 2024-09-11T07:46:21.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:20 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:46:21.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:20 smithi104 ceph-mon[33507]: 1.0 scrub starts 2024-09-11T07:46:21.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:20 smithi104 ceph-mon[33507]: 1.0 scrub ok 2024-09-11T07:46:21.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:20 smithi104 ceph-mon[33507]: pgmap v929: 158 pgs: 158 active+clean; 1.2 GiB data, 9.3 GiB used, 617 GiB / 626 GiB avail; 156 MiB/s wr, 2.50k op/s; 0 B/s, 1 objects/s recovering 2024-09-11T07:46:21.367 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:21.393 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:21.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:21.394 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:21.494 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:21.522 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:21.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:21.523 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:21.624 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:21.663 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:21.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:21.665 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:21.765 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:21.805 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:21.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:21.806 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:21.907 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:21.952 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:21.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:21.953 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:22.053 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:22.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:22.093 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:22.093 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:22.100 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 10 16 23621 23605 147.517 109.875 0.0030149 0.00677189 2024-09-11T07:46:22.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:21 smithi022 ceph-mon[29462]: 6.4 scrub starts 2024-09-11T07:46:22.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:21 smithi022 ceph-mon[29462]: 6.4 scrub ok 2024-09-11T07:46:22.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:21 smithi022 ceph-mon[29462]: 1.1 scrub starts 2024-09-11T07:46:22.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:21 smithi022 ceph-mon[29462]: 1.1 scrub ok 2024-09-11T07:46:22.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:21 smithi022 ceph-mon[37330]: 6.4 scrub starts 2024-09-11T07:46:22.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:21 smithi022 ceph-mon[37330]: 6.4 scrub ok 2024-09-11T07:46:22.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:21 smithi022 ceph-mon[37330]: 1.1 scrub starts 2024-09-11T07:46:22.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:21 smithi022 ceph-mon[37330]: 1.1 scrub ok 2024-09-11T07:46:22.194 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:22.221 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:22.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:22.222 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:22.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:21 smithi104 ceph-mon[33507]: 6.4 scrub starts 2024-09-11T07:46:22.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:21 smithi104 ceph-mon[33507]: 6.4 scrub ok 2024-09-11T07:46:22.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:21 smithi104 ceph-mon[33507]: 1.1 scrub starts 2024-09-11T07:46:22.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:21 smithi104 ceph-mon[33507]: 1.1 scrub ok 2024-09-11T07:46:22.323 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:22.350 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:22.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:22.351 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:22.452 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:22.487 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:22.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:22.488 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:22.589 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:22.614 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:22.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:22.616 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:22.716 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:22.758 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:22.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:22.759 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:22.860 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:22.896 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:22.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:22.897 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:22.998 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:23.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:22 smithi104 ceph-mon[33507]: 2.1d scrub starts 2024-09-11T07:46:23.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:22 smithi104 ceph-mon[33507]: 2.1d scrub ok 2024-09-11T07:46:23.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:22 smithi104 ceph-mon[33507]: 3.1 deep-scrub starts 2024-09-11T07:46:23.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:22 smithi104 ceph-mon[33507]: 3.1 deep-scrub ok 2024-09-11T07:46:23.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:22 smithi104 ceph-mon[33507]: pgmap v930: 158 pgs: 158 active+clean; 1.2 GiB data, 9.3 GiB used, 617 GiB / 626 GiB avail; 138 MiB/s wr, 2.21k op/s; 0 B/s, 1 objects/s recovering 2024-09-11T07:46:23.040 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:23.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:23.041 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:23.087 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-11T07:46:23.100 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 11 16 25609 25593 145.401 124.25 0.00562261 0.00687062 2024-09-11T07:46:23.141 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:23.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:22 smithi022 ceph-mon[37330]: 2.1d scrub starts 2024-09-11T07:46:23.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:22 smithi022 ceph-mon[37330]: 2.1d scrub ok 2024-09-11T07:46:23.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:22 smithi022 ceph-mon[37330]: 3.1 deep-scrub starts 2024-09-11T07:46:23.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:22 smithi022 ceph-mon[37330]: 3.1 deep-scrub ok 2024-09-11T07:46:23.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:22 smithi022 ceph-mon[37330]: pgmap v930: 158 pgs: 158 active+clean; 1.2 GiB data, 9.3 GiB used, 617 GiB / 626 GiB avail; 138 MiB/s wr, 2.21k op/s; 0 B/s, 1 objects/s recovering 2024-09-11T07:46:23.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:22 smithi022 ceph-mon[29462]: 2.1d scrub starts 2024-09-11T07:46:23.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:22 smithi022 ceph-mon[29462]: 2.1d scrub ok 2024-09-11T07:46:23.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:22 smithi022 ceph-mon[29462]: 3.1 deep-scrub starts 2024-09-11T07:46:23.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:22 smithi022 ceph-mon[29462]: 3.1 deep-scrub ok 2024-09-11T07:46:23.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:22 smithi022 ceph-mon[29462]: pgmap v930: 158 pgs: 158 active+clean; 1.2 GiB data, 9.3 GiB used, 617 GiB / 626 GiB avail; 138 MiB/s wr, 2.21k op/s; 0 B/s, 1 objects/s recovering 2024-09-11T07:46:23.170 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:23.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:23.171 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:23.272 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:23.299 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:23.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:23.300 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:23.364 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:46:23.400 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:23.421 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:46:23.452 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:23.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:23.454 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:23.555 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:23.595 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:23.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:23.597 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:23.698 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:23.728 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:23.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:23.729 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:23.830 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:23.860 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:23.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:23.861 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:23.962 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:23.990 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:23.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:23.991 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:24.091 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:24.100 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 12 16 27654 27638 143.934 127.812 0.00481653 0.00694134 2024-09-11T07:46:24.119 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:46:24.126 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:24.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:24.128 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:24.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:24 smithi022 ceph-mon[37330]: 2.1b scrub starts 2024-09-11T07:46:24.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:24 smithi022 ceph-mon[37330]: 2.1b scrub ok 2024-09-11T07:46:24.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:24 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:46:24.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:24 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:46:24.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:24 smithi022 ceph-mon[29462]: 2.1b scrub starts 2024-09-11T07:46:24.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:24 smithi022 ceph-mon[29462]: 2.1b scrub ok 2024-09-11T07:46:24.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:24 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:46:24.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:24 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:46:24.228 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:24.277 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:24.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:24.278 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:24.279 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:46:24 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:46:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:46:24.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:24 smithi104 ceph-mon[33507]: 2.1b scrub starts 2024-09-11T07:46:24.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:24 smithi104 ceph-mon[33507]: 2.1b scrub ok 2024-09-11T07:46:24.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:24 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:46:24.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:24 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:46:24.379 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:24.418 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:24.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:24.419 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:24.520 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:24.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:24.542 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:24.542 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:24.643 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:24.683 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:24.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:24.684 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:24.785 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:24.813 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:24.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:24.814 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:24.915 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:24.941 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:24.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:24.942 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:25.043 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:25.071 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:25.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:25.072 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:25.100 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 13 15 29467 29452 141.583 113.375 0.0082232 0.00705773 2024-09-11T07:46:25.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:25 smithi022 ceph-mon[29462]: 2.a scrub starts 2024-09-11T07:46:25.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:25 smithi022 ceph-mon[29462]: 2.a scrub ok 2024-09-11T07:46:25.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:25 smithi022 ceph-mon[29462]: 5.16 scrub starts 2024-09-11T07:46:25.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:25 smithi022 ceph-mon[29462]: 5.16 scrub ok 2024-09-11T07:46:25.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:25 smithi022 ceph-mon[29462]: 6.18 scrub starts 2024-09-11T07:46:25.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:25 smithi022 ceph-mon[29462]: 6.18 scrub ok 2024-09-11T07:46:25.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:25 smithi022 ceph-mon[29462]: pgmap v931: 158 pgs: 158 active+clean; 1.5 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 129 MiB/s wr, 2.06k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:46:25.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:25 smithi022 ceph-mon[37330]: 2.a scrub starts 2024-09-11T07:46:25.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:25 smithi022 ceph-mon[37330]: 2.a scrub ok 2024-09-11T07:46:25.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:25 smithi022 ceph-mon[37330]: 5.16 scrub starts 2024-09-11T07:46:25.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:25 smithi022 ceph-mon[37330]: 5.16 scrub ok 2024-09-11T07:46:25.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:25 smithi022 ceph-mon[37330]: 6.18 scrub starts 2024-09-11T07:46:25.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:25 smithi022 ceph-mon[37330]: 6.18 scrub ok 2024-09-11T07:46:25.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:25 smithi022 ceph-mon[37330]: pgmap v931: 158 pgs: 158 active+clean; 1.5 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 129 MiB/s wr, 2.06k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:46:25.173 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:25.214 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:25.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:25.216 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:25.311 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:46:25.317 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:25.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:25.395 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:25.395 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:25.399 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:25 smithi104 ceph-mon[33507]: 2.a scrub starts 2024-09-11T07:46:25.400 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:25 smithi104 ceph-mon[33507]: 2.a scrub ok 2024-09-11T07:46:25.400 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:25 smithi104 ceph-mon[33507]: 5.16 scrub starts 2024-09-11T07:46:25.400 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:25 smithi104 ceph-mon[33507]: 5.16 scrub ok 2024-09-11T07:46:25.400 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:25 smithi104 ceph-mon[33507]: 6.18 scrub starts 2024-09-11T07:46:25.400 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:25 smithi104 ceph-mon[33507]: 6.18 scrub ok 2024-09-11T07:46:25.400 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:25 smithi104 ceph-mon[33507]: pgmap v931: 158 pgs: 158 active+clean; 1.5 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 129 MiB/s wr, 2.06k op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:46:25.497 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:25.553 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:25.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:25.554 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:25.655 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:25.696 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:25.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:25.697 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:25.798 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:25.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:25.821 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:25.821 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:25.922 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:25.961 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:25.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:25.962 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:26.063 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:26.089 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:26.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:26.090 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:26.100 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 14 16 31154 31138 138.996 105.375 0.00632672 0.00718862 2024-09-11T07:46:26.190 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:26.213 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:26.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:26.214 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:26.315 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:26.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:26.348 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:26.348 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:26.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:26 smithi022 ceph-mon[29462]: 4.e scrub starts 2024-09-11T07:46:26.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:26 smithi022 ceph-mon[29462]: 4.e scrub ok 2024-09-11T07:46:26.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:26 smithi022 ceph-mon[37330]: 4.e scrub starts 2024-09-11T07:46:26.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:26 smithi022 ceph-mon[37330]: 4.e scrub ok 2024-09-11T07:46:26.427 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:26 smithi104 ceph-mon[33507]: 4.e scrub starts 2024-09-11T07:46:26.427 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:26 smithi104 ceph-mon[33507]: 4.e scrub ok 2024-09-11T07:46:26.449 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:26.472 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:26.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:26.473 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:26.574 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:26.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:26.597 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:26.597 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:26.697 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:26.702 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:46:26.702 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:46:26.702 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2024-09-11T07:46:26.702 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2024-09-11T07:46:26.702 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:46:26.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:26.737 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:26.737 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:26.838 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:26.871 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:26.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:26.872 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:26.973 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:27.005 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:27.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:27.006 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:27.100 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 15 16 32239 32223 134.25 67.8125 0.00888698 0.00744296 2024-09-11T07:46:27.107 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:27.147 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:27.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:27.148 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:27.249 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:27.279 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:27.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:27.280 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:27.357 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:27 smithi104 ceph-mon[33507]: 4.0 deep-scrub starts 2024-09-11T07:46:27.357 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:27 smithi104 ceph-mon[33507]: 4.0 deep-scrub ok 2024-09-11T07:46:27.357 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:27 smithi104 ceph-mon[33507]: pgmap v932: 158 pgs: 158 active+clean; 1.7 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 125 MiB/s wr, 1.99k op/s 2024-09-11T07:46:27.381 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-11T07:46:27.381 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:27.413 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:27.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:27.414 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:27.416 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:27 smithi022 ceph-mon[37330]: 4.0 deep-scrub starts 2024-09-11T07:46:27.416 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:27 smithi022 ceph-mon[37330]: 4.0 deep-scrub ok 2024-09-11T07:46:27.416 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:27 smithi022 ceph-mon[37330]: pgmap v932: 158 pgs: 158 active+clean; 1.7 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 125 MiB/s wr, 1.99k op/s 2024-09-11T07:46:27.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:27 smithi022 ceph-mon[29462]: 4.0 deep-scrub starts 2024-09-11T07:46:27.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:27 smithi022 ceph-mon[29462]: 4.0 deep-scrub ok 2024-09-11T07:46:27.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:27 smithi022 ceph-mon[29462]: pgmap v932: 158 pgs: 158 active+clean; 1.7 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 125 MiB/s wr, 1.99k op/s 2024-09-11T07:46:27.515 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:27.536 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:27.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:27.537 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:27.638 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:27.654 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:46:27.678 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:27.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:27.679 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:27.780 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:27.823 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:27.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:27.824 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:27.925 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:27.951 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:27.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:27.953 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:28.054 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:28.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:28.096 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:28.096 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:28.101 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 16 16 33576 33560 131.079 83.5625 0.00389738 0.00762234 2024-09-11T07:46:28.197 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:28.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:28.247 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:28.247 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:28.348 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:28.389 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:28.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:28.391 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:28.492 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:28.532 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:28.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:28.533 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:28.634 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:28.672 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:28.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:28.673 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:28.774 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:28.798 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:28.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:28.799 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:28.822 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:46:28.899 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:28.952 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:28.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:28.953 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:29.054 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:29.098 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:29.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:29.099 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:29.101 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 17 16 35051 35035 128.79 92.1875 0.0179224 0.00775819 2024-09-11T07:46:29.200 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:29.232 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:29.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:29.233 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:29.333 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:29.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:29.377 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:29.377 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:29.461 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is set 2024-09-11T07:46:29.478 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:29.496 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:46:29.496 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2024-09-11T07:46:29.496 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2024-09-11T07:46:29.496 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2024-09-11T07:46:29.496 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.496 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87203 9.13 9:cc4de378:::benchmark_data_smithi022_133943_object87202:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e260)", 2024-09-11T07:46:29.496 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.339440+0000", 2024-09-11T07:46:29.496 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 481.149115397, 2024-09-11T07:46:29.497 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1017081860000002, 2024-09-11T07:46:29.497 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:29.497 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:46:29.497 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:29.497 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:29.497 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:29.497 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87203 2024-09-11T07:46:29.497 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.497 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:29.497 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.497 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:29.497 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339440+0000", 2024-09-11T07:46:29.497 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.498 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.498 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.498 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:29.498 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339440+0000", 2024-09-11T07:46:29.498 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.498 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.498 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.498 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:29.498 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339440+0000", 2024-09-11T07:46:29.498 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:46:29.498 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.498 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.498 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:29.499 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339474+0000", 2024-09-11T07:46:29.499 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4149000000000001e-05 2024-09-11T07:46:29.499 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.499 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.499 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:29.499 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339480+0000", 2024-09-11T07:46:29.499 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3550000000000001e-06 2024-09-11T07:46:29.499 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.499 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.499 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:29.499 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339484+0000", 2024-09-11T07:46:29.499 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1239999999999998e-06 2024-09-11T07:46:29.499 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.499 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.500 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.500 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339567+0000", 2024-09-11T07:46:29.500 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3145000000000004e-05 2024-09-11T07:46:29.500 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.500 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.500 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:29.500 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339673+0000", 2024-09-11T07:46:29.500 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010586699999999999 2024-09-11T07:46:29.500 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.500 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.500 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:46:29.500 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.339947+0000", 2024-09-11T07:46:29.500 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00027409800000000002 2024-09-11T07:46:29.501 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.501 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.501 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:29.501 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.341767+0000", 2024-09-11T07:46:29.501 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001819883 2024-09-11T07:46:29.501 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.501 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.501 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.501 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.342624+0000", 2024-09-11T07:46:29.501 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00085697099999999999 2024-09-11T07:46:29.501 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.501 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.501 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.502 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.347306+0000", 2024-09-11T07:46:29.502 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0046816230000000002 2024-09-11T07:46:29.502 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.502 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.502 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:29.502 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.347315+0000", 2024-09-11T07:46:29.502 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.1600000000000004e-06 2024-09-11T07:46:29.502 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.502 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.502 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.502 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.346303+0000", 2024-09-11T07:46:29.503 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.99898775399999995 2024-09-11T07:46:29.503 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.503 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.503 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:29.503 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.346312+0000", 2024-09-11T07:46:29.503 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.6889999999999999e-06 2024-09-11T07:46:29.503 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.503 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.503 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.503 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.436501+0000", 2024-09-11T07:46:29.503 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.090188951 2024-09-11T07:46:29.503 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.503 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.503 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for degraded object", 2024-09-11T07:46:29.504 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.437198+0000", 2024-09-11T07:46:29.504 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00069633100000000001 2024-09-11T07:46:29.504 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.504 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.504 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.504 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.441056+0000", 2024-09-11T07:46:29.504 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0038578359999999999 2024-09-11T07:46:29.504 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.504 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.504 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:29.504 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.441149+0000", 2024-09-11T07:46:29.505 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2905000000000006e-05 2024-09-11T07:46:29.505 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.505 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:29.505 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.505 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.505 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.505 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87197 9.17 9:ede1dd36:::benchmark_data_smithi022_133943_object87196:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:46:29.505 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.345260+0000", 2024-09-11T07:46:29.505 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 481.143295448, 2024-09-11T07:46:29.505 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0776407649999999, 2024-09-11T07:46:29.505 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:29.505 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:46:29.505 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:29.505 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:29.506 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:29.506 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87197 2024-09-11T07:46:29.506 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.506 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:29.506 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.506 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:29.506 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.345260+0000", 2024-09-11T07:46:29.506 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.506 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.506 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.506 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:29.506 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.345260+0000", 2024-09-11T07:46:29.506 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.507 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.507 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.507 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:29.507 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.345258+0000", 2024-09-11T07:46:29.507 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:46:29.507 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.507 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.507 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:29.507 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.345346+0000", 2024-09-11T07:46:29.507 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7859e-05 2024-09-11T07:46:29.507 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.507 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.507 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:29.507 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.345356+0000", 2024-09-11T07:46:29.508 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0205e-05 2024-09-11T07:46:29.508 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.508 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.508 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:29.508 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.345361+0000", 2024-09-11T07:46:29.508 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4199999999999998e-06 2024-09-11T07:46:29.508 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.508 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.508 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.508 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.347877+0000", 2024-09-11T07:46:29.508 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0025154270000000002 2024-09-11T07:46:29.508 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.508 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.508 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:29.508 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.347884+0000", 2024-09-11T07:46:29.509 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4699999999999996e-06 2024-09-11T07:46:29.509 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.509 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.509 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.509 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.344661+0000", 2024-09-11T07:46:29.509 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.99677682199999995 2024-09-11T07:46:29.509 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.509 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.509 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:29.509 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.344670+0000", 2024-09-11T07:46:29.509 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3449999999999995e-06 2024-09-11T07:46:29.509 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.509 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.510 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.510 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.422818+0000", 2024-09-11T07:46:29.510 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0781474900000001 2024-09-11T07:46:29.510 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.510 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.510 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:29.510 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.422901+0000", 2024-09-11T07:46:29.510 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3071000000000005e-05 2024-09-11T07:46:29.510 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.510 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:29.510 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.510 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.510 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.510 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87222 9.18 9.9a7e3098 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:46:29.511 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.362550+0000", 2024-09-11T07:46:29.511 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 481.126005233, 2024-09-11T07:46:29.511 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.98180590199999995, 2024-09-11T07:46:29.511 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:29.511 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:46:29.511 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:29.511 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:29.511 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:29.511 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87222 2024-09-11T07:46:29.511 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.511 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:29.511 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.512 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:29.512 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.362550+0000", 2024-09-11T07:46:29.512 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.512 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.512 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.512 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:29.512 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.362550+0000", 2024-09-11T07:46:29.512 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.512 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.512 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.512 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:29.512 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.362549+0000", 2024-09-11T07:46:29.512 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:46:29.512 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.513 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.513 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:29.513 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.362658+0000", 2024-09-11T07:46:29.513 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000108529 2024-09-11T07:46:29.513 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.513 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.513 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:29.513 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.362667+0000", 2024-09-11T07:46:29.513 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9020000000000005e-06 2024-09-11T07:46:29.513 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.513 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.513 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:29.514 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.362670+0000", 2024-09-11T07:46:29.514 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4340000000000001e-06 2024-09-11T07:46:29.514 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.514 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.514 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.514 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.362714+0000", 2024-09-11T07:46:29.514 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3615000000000002e-05 2024-09-11T07:46:29.514 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.514 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.514 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:29.514 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.362719+0000", 2024-09-11T07:46:29.514 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0350000000000001e-06 2024-09-11T07:46:29.514 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.515 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.515 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.515 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.344348+0000", 2024-09-11T07:46:29.515 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.98162881999999996 2024-09-11T07:46:29.515 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.515 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.515 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:29.515 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.344356+0000", 2024-09-11T07:46:29.515 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6580000000000008e-06 2024-09-11T07:46:29.516 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.516 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:29.516 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.516 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.516 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.516 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87236 9.15 9:aa3be195:::benchmark_data_smithi022_133943_object87235:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:46:29.516 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.369488+0000", 2024-09-11T07:46:29.516 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 481.11906815499998, 2024-09-11T07:46:29.516 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.156802254, 2024-09-11T07:46:29.516 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:29.516 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:29.516 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:29.517 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:29.517 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:29.517 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87236 2024-09-11T07:46:29.517 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.517 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:29.517 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.517 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:29.517 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369488+0000", 2024-09-11T07:46:29.517 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.517 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.517 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.517 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:29.517 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369488+0000", 2024-09-11T07:46:29.517 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.517 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.518 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.518 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:29.518 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369486+0000", 2024-09-11T07:46:29.518 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:46:29.518 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.518 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.518 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:29.518 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369534+0000", 2024-09-11T07:46:29.518 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8127000000000002e-05 2024-09-11T07:46:29.518 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.518 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.518 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:29.518 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369545+0000", 2024-09-11T07:46:29.518 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0920999999999999e-05 2024-09-11T07:46:29.519 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.519 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.519 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:29.519 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369549+0000", 2024-09-11T07:46:29.519 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4580000000000003e-06 2024-09-11T07:46:29.519 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.519 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.519 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.519 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369610+0000", 2024-09-11T07:46:29.519 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0631000000000003e-05 2024-09-11T07:46:29.519 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.519 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.519 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:29.520 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369618+0000", 2024-09-11T07:46:29.520 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7420000000000003e-06 2024-09-11T07:46:29.520 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.520 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.520 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.520 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.345692+0000", 2024-09-11T07:46:29.520 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.97607389 2024-09-11T07:46:29.520 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.520 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.520 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:29.520 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.345702+0000", 2024-09-11T07:46:29.520 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0086e-05 2024-09-11T07:46:29.520 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.521 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.521 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.521 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.510602+0000", 2024-09-11T07:46:29.521 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1648999250000001 2024-09-11T07:46:29.521 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.521 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.521 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:29.521 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.510767+0000", 2024-09-11T07:46:29.521 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000164656 2024-09-11T07:46:29.521 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.521 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.521 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:46:29.521 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.511002+0000", 2024-09-11T07:46:29.522 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000234699 2024-09-11T07:46:29.522 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.522 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.522 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:29.522 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.514166+0000", 2024-09-11T07:46:29.522 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0031640069999999999 2024-09-11T07:46:29.522 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.522 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.522 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.522 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.523959+0000", 2024-09-11T07:46:29.522 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0097934619999999993 2024-09-11T07:46:29.522 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.522 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.523 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.523 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.523981+0000", 2024-09-11T07:46:29.523 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.181e-05 2024-09-11T07:46:29.523 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.523 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.523 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:29.523 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.524012+0000", 2024-09-11T07:46:29.523 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1241000000000001e-05 2024-09-11T07:46:29.523 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.523 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.523 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:29.523 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.526290+0000", 2024-09-11T07:46:29.523 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0022775819999999998 2024-09-11T07:46:29.523 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.524 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:29.524 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.524 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.524 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87239 9.17 9:eb730478:::benchmark_data_smithi022_133943_object87238:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:46:29.524 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.369713+0000", 2024-09-11T07:46:29.524 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 481.11884304, 2024-09-11T07:46:29.524 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0664777230000002, 2024-09-11T07:46:29.524 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:29.524 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:29.524 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:29.524 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:29.524 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:29.524 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87239 2024-09-11T07:46:29.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.525 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:29.525 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.525 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:29.525 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369713+0000", 2024-09-11T07:46:29.525 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.525 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.525 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.525 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:29.525 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369713+0000", 2024-09-11T07:46:29.525 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.525 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.525 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.525 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:29.525 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369712+0000", 2024-09-11T07:46:29.526 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:46:29.526 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.526 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.526 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:29.526 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369781+0000", 2024-09-11T07:46:29.526 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9408999999999999e-05 2024-09-11T07:46:29.526 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.526 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.526 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:29.526 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369796+0000", 2024-09-11T07:46:29.526 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4355999999999999e-05 2024-09-11T07:46:29.526 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.526 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.526 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:29.526 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369801+0000", 2024-09-11T07:46:29.527 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1410000000000004e-06 2024-09-11T07:46:29.527 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.527 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.527 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.527 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369852+0000", 2024-09-11T07:46:29.527 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0983000000000003e-05 2024-09-11T07:46:29.527 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.527 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.527 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:29.527 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.369860+0000", 2024-09-11T07:46:29.527 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6200000000000005e-06 2024-09-11T07:46:29.527 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.527 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.527 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.527 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.344696+0000", 2024-09-11T07:46:29.528 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.97483578000000004 2024-09-11T07:46:29.528 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.528 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.528 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:29.528 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.344703+0000", 2024-09-11T07:46:29.528 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4240000000000004e-06 2024-09-11T07:46:29.528 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.528 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.528 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.528 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.422922+0000", 2024-09-11T07:46:29.528 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0782185689999999 2024-09-11T07:46:29.528 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.528 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.528 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:29.529 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.423049+0000", 2024-09-11T07:46:29.529 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012664800000000001 2024-09-11T07:46:29.529 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.529 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.529 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,1,3,5", 2024-09-11T07:46:29.529 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.423321+0000", 2024-09-11T07:46:29.529 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000272355 2024-09-11T07:46:29.529 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.529 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.529 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.529 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.428425+0000", 2024-09-11T07:46:29.529 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0051037970000000002 2024-09-11T07:46:29.529 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.529 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.530 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.530 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.429833+0000", 2024-09-11T07:46:29.530 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0014079769999999999 2024-09-11T07:46:29.530 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.530 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.530 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.530 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.435362+0000", 2024-09-11T07:46:29.530 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0055289069999999996 2024-09-11T07:46:29.530 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.530 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.530 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.530 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.435951+0000", 2024-09-11T07:46:29.530 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000589158 2024-09-11T07:46:29.531 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.531 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.531 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:29.531 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.435960+0000", 2024-09-11T07:46:29.531 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9600000000000006e-06 2024-09-11T07:46:29.531 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.531 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.531 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:29.531 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.436059+0000", 2024-09-11T07:46:29.531 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8784999999999997e-05 2024-09-11T07:46:29.531 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.531 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.531 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:29.532 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.436190+0000", 2024-09-11T07:46:29.532 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000131772 2024-09-11T07:46:29.532 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.532 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:29.532 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.532 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.532 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.532 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87240 9.18 9.2c746ab8 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:46:29.532 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.372326+0000", 2024-09-11T07:46:29.532 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 481.11622964499998, 2024-09-11T07:46:29.532 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.97205600599999997, 2024-09-11T07:46:29.532 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:29.532 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:46:29.532 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:29.533 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:29.533 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:29.533 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87240 2024-09-11T07:46:29.533 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.533 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:29.533 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.533 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:29.533 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372326+0000", 2024-09-11T07:46:29.533 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.533 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.533 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.533 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:29.533 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372326+0000", 2024-09-11T07:46:29.533 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.534 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.534 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.534 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:29.534 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372324+0000", 2024-09-11T07:46:29.534 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:46:29.534 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.534 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.534 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:29.534 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372381+0000", 2024-09-11T07:46:29.534 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7188000000000002e-05 2024-09-11T07:46:29.534 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.534 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.534 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:29.534 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372394+0000", 2024-09-11T07:46:29.534 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2901999999999999e-05 2024-09-11T07:46:29.535 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.535 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.535 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:29.535 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372400+0000", 2024-09-11T07:46:29.535 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.699e-06 2024-09-11T07:46:29.535 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.535 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.535 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.535 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372439+0000", 2024-09-11T07:46:29.535 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9839000000000003e-05 2024-09-11T07:46:29.535 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.535 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.535 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:29.536 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.372445+0000", 2024-09-11T07:46:29.536 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9490000000000001e-06 2024-09-11T07:46:29.536 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.536 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.536 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.536 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.344376+0000", 2024-09-11T07:46:29.536 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.97193081100000001 2024-09-11T07:46:29.536 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.536 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.536 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:29.536 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.344382+0000", 2024-09-11T07:46:29.536 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9070000000000004e-06 2024-09-11T07:46:29.536 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.536 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:29.536 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.536 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.537 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.537 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87250 9.15 9:ac3c2ae6:::benchmark_data_smithi022_133943_object87249:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:46:29.537 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.377199+0000", 2024-09-11T07:46:29.537 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 481.11135721699998, 2024-09-11T07:46:29.537 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1512879869999999, 2024-09-11T07:46:29.537 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:29.537 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:29.537 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:29.537 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:29.537 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:29.537 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87250 2024-09-11T07:46:29.537 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.537 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:29.537 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.538 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:29.538 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.377199+0000", 2024-09-11T07:46:29.538 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.538 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.538 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.538 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:29.538 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.377199+0000", 2024-09-11T07:46:29.538 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.538 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.539 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.539 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:29.539 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.377197+0000", 2024-09-11T07:46:29.539 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:46:29.539 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.539 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.539 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:29.539 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.377241+0000", 2024-09-11T07:46:29.539 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3498999999999998e-05 2024-09-11T07:46:29.539 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.540 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:29.540 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.377249+0000", 2024-09-11T07:46:29.540 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0989999999999999e-06 2024-09-11T07:46:29.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.540 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:29.540 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.377254+0000", 2024-09-11T07:46:29.540 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1220000000000003e-06 2024-09-11T07:46:29.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.540 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.540 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.377302+0000", 2024-09-11T07:46:29.540 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7479999999999999e-05 2024-09-11T07:46:29.541 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.541 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.541 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:29.541 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.377311+0000", 2024-09-11T07:46:29.541 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2359999999999993e-06 2024-09-11T07:46:29.541 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.541 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.541 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.541 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.345770+0000", 2024-09-11T07:46:29.541 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.968458763 2024-09-11T07:46:29.541 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.541 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.542 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:29.542 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.345779+0000", 2024-09-11T07:46:29.542 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0869999999999996e-06 2024-09-11T07:46:29.542 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.542 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.542 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.542 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.512718+0000", 2024-09-11T07:46:29.542 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.166939323 2024-09-11T07:46:29.542 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.542 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.542 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:29.542 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.512902+0000", 2024-09-11T07:46:29.542 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018403899999999999 2024-09-11T07:46:29.543 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.543 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.543 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:46:29.543 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.513149+0000", 2024-09-11T07:46:29.543 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00024737899999999999 2024-09-11T07:46:29.543 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.543 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.543 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:29.543 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.519581+0000", 2024-09-11T07:46:29.543 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0064314819999999997 2024-09-11T07:46:29.543 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.543 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.543 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.544 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.526384+0000", 2024-09-11T07:46:29.544 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0068032070000000004 2024-09-11T07:46:29.544 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.544 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.544 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.544 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.526419+0000", 2024-09-11T07:46:29.544 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5036000000000001e-05 2024-09-11T07:46:29.544 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.544 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.544 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:29.544 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.526478+0000", 2024-09-11T07:46:29.544 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8462999999999999e-05 2024-09-11T07:46:29.544 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.545 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:29.545 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.528486+0000", 2024-09-11T07:46:29.545 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0020089330000000001 2024-09-11T07:46:29.545 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.545 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:29.545 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.545 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.545 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:87257 9.13 9:ca17476d:::benchmark_data_smithi022_133943_object87256:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e261)", 2024-09-11T07:46:29.545 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:38:28.384938+0000", 2024-09-11T07:46:29.545 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 481.10361811500002, 2024-09-11T07:46:29.545 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0584020930000002, 2024-09-11T07:46:29.545 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:29.545 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:29.546 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:29.546 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:46:29.546 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:29.546 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 87257 2024-09-11T07:46:29.546 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.546 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:29.546 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.546 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:29.546 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.384938+0000", 2024-09-11T07:46:29.546 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.546 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.546 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.546 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:29.546 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.384938+0000", 2024-09-11T07:46:29.547 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.547 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.547 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.547 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:29.547 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.384936+0000", 2024-09-11T07:46:29.547 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:46:29.547 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.547 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.547 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:29.547 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.384998+0000", 2024-09-11T07:46:29.547 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1383000000000005e-05 2024-09-11T07:46:29.547 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.547 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.547 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:29.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.385007+0000", 2024-09-11T07:46:29.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3370000000000006e-06 2024-09-11T07:46:29.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.548 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.548 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:29.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.385011+0000", 2024-09-11T07:46:29.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9840000000000003e-06 2024-09-11T07:46:29.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.548 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.548 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.385067+0000", 2024-09-11T07:46:29.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6425999999999998e-05 2024-09-11T07:46:29.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.549 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:29.549 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:28.385079+0000", 2024-09-11T07:46:29.549 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.155e-05 2024-09-11T07:46:29.549 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.549 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.549 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.346702+0000", 2024-09-11T07:46:29.549 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.96162294299999995 2024-09-11T07:46:29.549 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.549 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:29.549 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:29.346713+0000", 2024-09-11T07:46:29.550 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0688e-05 2024-09-11T07:46:29.550 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.550 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.550 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.550 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.437750+0000", 2024-09-11T07:46:29.550 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0910372879999999 2024-09-11T07:46:29.550 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.550 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.550 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:29.550 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.437940+0000", 2024-09-11T07:46:29.550 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019046499999999999 2024-09-11T07:46:29.550 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.550 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.551 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3,5,6", 2024-09-11T07:46:29.551 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.438830+0000", 2024-09-11T07:46:29.551 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00088921499999999995 2024-09-11T07:46:29.551 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.551 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.551 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:29.551 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.440627+0000", 2024-09-11T07:46:29.551 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001797873 2024-09-11T07:46:29.551 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.551 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.551 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.551 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.441186+0000", 2024-09-11T07:46:29.551 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00055897300000000004 2024-09-11T07:46:29.552 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.552 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.552 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.552 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.441752+0000", 2024-09-11T07:46:29.552 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00056608299999999999 2024-09-11T07:46:29.552 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.552 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.552 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.552 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.442310+0000", 2024-09-11T07:46:29.552 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00055787000000000002 2024-09-11T07:46:29.552 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.552 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.552 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.553 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.443120+0000", 2024-09-11T07:46:29.553 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00080973899999999997 2024-09-11T07:46:29.553 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.553 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:29.554 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:29.554 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:29.555 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:29.555 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.443167+0000", 2024-09-11T07:46:29.555 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7302999999999998e-05 2024-09-11T07:46:29.555 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.555 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.556 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:29.556 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:38:30.443340+0000", 2024-09-11T07:46:29.556 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017236000000000001 2024-09-11T07:46:29.556 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.556 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:29.556 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.556 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.556 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.556 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25706.0:254594 10.0 10:0bd5f10b:::benchmark_data_smithi022_163866_object125168:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e397)", 2024-09-11T07:46:29.556 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:45:30.132927+0000", 2024-09-11T07:46:29.556 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 59.355629206000003, 2024-09-11T07:46:29.556 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.64135062300000001, 2024-09-11T07:46:29.556 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:29.556 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:29.557 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:29.557 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25706", 2024-09-11T07:46:29.557 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:46:29.557 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 254594 2024-09-11T07:46:29.557 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.557 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:29.557 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.557 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:29.557 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.132927+0000", 2024-09-11T07:46:29.557 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.557 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.557 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.557 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:29.558 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.132927+0000", 2024-09-11T07:46:29.558 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.558 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.558 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.558 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:29.558 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.132925+0000", 2024-09-11T07:46:29.558 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:46:29.558 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.558 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.558 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:29.558 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.132930+0000", 2024-09-11T07:46:29.558 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6930000000000004e-06 2024-09-11T07:46:29.558 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.558 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.558 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:29.558 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.132940+0000", 2024-09-11T07:46:29.559 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8700000000000004e-06 2024-09-11T07:46:29.559 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.559 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.559 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:29.559 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.132946+0000", 2024-09-11T07:46:29.559 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7289999999999997e-06 2024-09-11T07:46:29.559 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.559 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.559 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.559 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.133007+0000", 2024-09-11T07:46:29.559 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1167000000000006e-05 2024-09-11T07:46:29.559 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.559 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.559 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:29.559 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.133015+0000", 2024-09-11T07:46:29.560 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0290000000000005e-06 2024-09-11T07:46:29.560 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.560 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.560 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.560 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.769704+0000", 2024-09-11T07:46:29.560 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.63668900399999995 2024-09-11T07:46:29.560 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.560 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.560 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:29.560 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.769873+0000", 2024-09-11T07:46:29.560 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016917000000000001 2024-09-11T07:46:29.560 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.560 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.560 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,6", 2024-09-11T07:46:29.561 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.769995+0000", 2024-09-11T07:46:29.561 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000121511 2024-09-11T07:46:29.561 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.561 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.561 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.561 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.774147+0000", 2024-09-11T07:46:29.561 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0041521700000000002 2024-09-11T07:46:29.561 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.561 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.561 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:29.561 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.774153+0000", 2024-09-11T07:46:29.561 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3509999999999998e-06 2024-09-11T07:46:29.561 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.561 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.562 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.562 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.774193+0000", 2024-09-11T07:46:29.562 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9557000000000001e-05 2024-09-11T07:46:29.562 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.562 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.562 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:29.562 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.774241+0000", 2024-09-11T07:46:29.562 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8396999999999998e-05 2024-09-11T07:46:29.562 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.562 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.562 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:29.562 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.774277+0000", 2024-09-11T07:46:29.562 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5763999999999998e-05 2024-09-11T07:46:29.562 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.562 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:29.563 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.563 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.563 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.563 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25706.0:254598 10.0 10:0a549420:::benchmark_data_smithi022_163866_object125172:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e397)", 2024-09-11T07:46:29.563 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:45:30.137856+0000", 2024-09-11T07:46:29.563 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 59.350699540999997, 2024-09-11T07:46:29.563 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.63836637600000001, 2024-09-11T07:46:29.563 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:29.563 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:29.563 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:29.563 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25706", 2024-09-11T07:46:29.563 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:46:29.563 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 254598 2024-09-11T07:46:29.564 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.564 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:29.564 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.564 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:29.564 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.137856+0000", 2024-09-11T07:46:29.564 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.564 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.564 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.564 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:29.564 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.137856+0000", 2024-09-11T07:46:29.564 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.564 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.564 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.565 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:29.565 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.137855+0000", 2024-09-11T07:46:29.565 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:46:29.565 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.565 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.565 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:29.565 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.137862+0000", 2024-09-11T07:46:29.565 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1069999999999999e-06 2024-09-11T07:46:29.565 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.565 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.600 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:29.600 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.137872+0000", 2024-09-11T07:46:29.600 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7820000000000005e-06 2024-09-11T07:46:29.600 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.601 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.601 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:29.601 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.137877+0000", 2024-09-11T07:46:29.601 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2780000000000001e-06 2024-09-11T07:46:29.601 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.601 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.601 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.601 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.137920+0000", 2024-09-11T07:46:29.601 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2500000000000003e-05 2024-09-11T07:46:29.601 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.601 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.601 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:29.601 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.137928+0000", 2024-09-11T07:46:29.602 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3909999999999998e-06 2024-09-11T07:46:29.602 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.602 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.602 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.602 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.772388+0000", 2024-09-11T07:46:29.602 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.63446054100000004 2024-09-11T07:46:29.602 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.602 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.602 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:29.602 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.772478+0000", 2024-09-11T07:46:29.603 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9903999999999998e-05 2024-09-11T07:46:29.603 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.603 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.603 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,6", 2024-09-11T07:46:29.603 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.772574+0000", 2024-09-11T07:46:29.603 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5779000000000006e-05 2024-09-11T07:46:29.603 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.603 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.603 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.603 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.774323+0000", 2024-09-11T07:46:29.603 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0017492639999999999 2024-09-11T07:46:29.603 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.603 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.604 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.604 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.776136+0000", 2024-09-11T07:46:29.604 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0018123359999999999 2024-09-11T07:46:29.604 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.604 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.604 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:29.604 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.776144+0000", 2024-09-11T07:46:29.604 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.3180000000000007e-06 2024-09-11T07:46:29.604 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.604 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.604 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:29.604 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.776187+0000", 2024-09-11T07:46:29.605 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2889999999999998e-05 2024-09-11T07:46:29.605 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.605 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.605 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:29.605 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.776223+0000", 2024-09-11T07:46:29.605 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5627999999999998e-05 2024-09-11T07:46:29.605 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.605 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:29.605 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.605 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.605 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.605 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25706.0:254599 10.c 10:34692d4f:::benchmark_data_smithi022_163866_object125173:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e397)", 2024-09-11T07:46:29.605 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:45:30.138400+0000", 2024-09-11T07:46:29.606 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 59.350155264000001, 2024-09-11T07:46:29.606 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.65205146899999999, 2024-09-11T07:46:29.606 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:29.606 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:29.606 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:29.606 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25706", 2024-09-11T07:46:29.606 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:46:29.606 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 254599 2024-09-11T07:46:29.606 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.606 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:29.606 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.607 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:29.607 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.138400+0000", 2024-09-11T07:46:29.607 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.607 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.607 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.607 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:29.607 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.138400+0000", 2024-09-11T07:46:29.607 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.607 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.607 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.607 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:29.607 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.138399+0000", 2024-09-11T07:46:29.607 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:46:29.608 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.608 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.608 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:29.608 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.138405+0000", 2024-09-11T07:46:29.608 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8680000000000004e-06 2024-09-11T07:46:29.608 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.608 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.608 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:29.608 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.138413+0000", 2024-09-11T07:46:29.608 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6699999999999994e-06 2024-09-11T07:46:29.608 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.608 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.608 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:29.609 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.138416+0000", 2024-09-11T07:46:29.609 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5260000000000002e-06 2024-09-11T07:46:29.609 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.609 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.609 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.609 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.138453+0000", 2024-09-11T07:46:29.609 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6770000000000002e-05 2024-09-11T07:46:29.609 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.609 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.609 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:29.609 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.138458+0000", 2024-09-11T07:46:29.609 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6199999999999998e-06 2024-09-11T07:46:29.609 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.609 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.610 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.610 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.770416+0000", 2024-09-11T07:46:29.610 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.63195844899999998 2024-09-11T07:46:29.610 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.610 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.610 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:29.610 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.770487+0000", 2024-09-11T07:46:29.610 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0949000000000001e-05 2024-09-11T07:46:29.610 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.610 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.610 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:46:29.610 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.770553+0000", 2024-09-11T07:46:29.611 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6304999999999996e-05 2024-09-11T07:46:29.611 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.611 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.611 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:29.611 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.783705+0000", 2024-09-11T07:46:29.611 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.013151685 2024-09-11T07:46:29.611 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.611 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.611 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.611 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790322+0000", 2024-09-11T07:46:29.611 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0066173789999999996 2024-09-11T07:46:29.612 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.612 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.612 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.612 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790373+0000", 2024-09-11T07:46:29.612 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0693999999999999e-05 2024-09-11T07:46:29.612 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.612 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.612 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:29.612 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790417+0000", 2024-09-11T07:46:29.612 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3501000000000002e-05 2024-09-11T07:46:29.612 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.612 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.613 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:29.613 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790452+0000", 2024-09-11T07:46:29.613 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5377999999999999e-05 2024-09-11T07:46:29.613 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.613 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:29.613 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.613 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.613 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.613 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25706.0:254607 10.c 10:33e52c3b:::benchmark_data_smithi022_163866_object125181:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e397)", 2024-09-11T07:46:29.613 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:45:30.142279+0000", 2024-09-11T07:46:29.613 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 59.346277067999999, 2024-09-11T07:46:29.613 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.64834728900000005, 2024-09-11T07:46:29.613 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:29.614 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:29.614 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:29.614 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25706", 2024-09-11T07:46:29.614 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:46:29.614 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 254607 2024-09-11T07:46:29.614 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.614 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:29.614 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.614 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:29.614 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.142279+0000", 2024-09-11T07:46:29.614 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.614 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.615 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.615 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:29.615 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.142279+0000", 2024-09-11T07:46:29.615 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.615 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.615 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.615 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:29.615 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.142277+0000", 2024-09-11T07:46:29.615 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:46:29.615 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.616 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.616 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:29.616 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.142283+0000", 2024-09-11T07:46:29.616 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1020000000000002e-06 2024-09-11T07:46:29.616 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.616 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.616 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:29.616 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.142295+0000", 2024-09-11T07:46:29.616 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.222e-05 2024-09-11T07:46:29.616 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.616 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.616 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:29.617 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.142301+0000", 2024-09-11T07:46:29.617 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5509999999999999e-06 2024-09-11T07:46:29.617 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.617 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.617 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.617 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.142344+0000", 2024-09-11T07:46:29.617 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3316999999999999e-05 2024-09-11T07:46:29.617 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.617 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.617 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:29.617 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.142351+0000", 2024-09-11T07:46:29.617 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7750000000000004e-06 2024-09-11T07:46:29.617 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.618 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.618 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.618 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.773044+0000", 2024-09-11T07:46:29.618 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.63069285200000003 2024-09-11T07:46:29.618 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.618 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.618 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:29.618 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.773111+0000", 2024-09-11T07:46:29.618 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7602999999999996e-05 2024-09-11T07:46:29.618 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.618 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.618 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:46:29.618 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.773176+0000", 2024-09-11T07:46:29.618 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4419999999999996e-05 2024-09-11T07:46:29.619 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.619 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.619 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:29.619 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790332+0000", 2024-09-11T07:46:29.619 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.017156062 2024-09-11T07:46:29.619 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.619 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.619 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.619 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790512+0000", 2024-09-11T07:46:29.619 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018059900000000001 2024-09-11T07:46:29.619 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.619 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.619 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.619 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790551+0000", 2024-09-11T07:46:29.620 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8346000000000002e-05 2024-09-11T07:46:29.620 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.620 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.620 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:29.620 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790591+0000", 2024-09-11T07:46:29.620 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0272999999999999e-05 2024-09-11T07:46:29.620 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.620 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.620 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:29.620 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790626+0000", 2024-09-11T07:46:29.620 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4898999999999999e-05 2024-09-11T07:46:29.620 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.620 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:29.621 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.621 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.621 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.621 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25706.0:254613 10.0 10:00ac1752:::benchmark_data_smithi022_163866_object125187:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e397)", 2024-09-11T07:46:29.621 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:45:30.146473+0000", 2024-09-11T07:46:29.621 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 59.342083125999999, 2024-09-11T07:46:29.621 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.63342851099999997, 2024-09-11T07:46:29.621 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:29.621 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:29.621 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:29.621 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25706", 2024-09-11T07:46:29.621 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:46:29.621 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 254613 2024-09-11T07:46:29.622 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.622 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:29.622 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.622 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:29.622 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.146473+0000", 2024-09-11T07:46:29.622 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.622 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.622 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.622 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:29.622 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.146473+0000", 2024-09-11T07:46:29.622 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.622 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.622 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.623 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:29.623 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.146471+0000", 2024-09-11T07:46:29.623 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:46:29.623 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.623 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.623 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:29.623 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.146477+0000", 2024-09-11T07:46:29.623 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6529999999999999e-06 2024-09-11T07:46:29.623 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.623 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.623 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:29.623 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.146487+0000", 2024-09-11T07:46:29.623 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0206e-05 2024-09-11T07:46:29.624 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.624 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.624 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:29.624 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.146491+0000", 2024-09-11T07:46:29.624 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0940000000000001e-06 2024-09-11T07:46:29.624 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.624 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.624 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.624 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.146532+0000", 2024-09-11T07:46:29.624 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0997000000000001e-05 2024-09-11T07:46:29.624 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.624 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.624 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:29.624 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.146539+0000", 2024-09-11T07:46:29.625 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0890000000000002e-06 2024-09-11T07:46:29.625 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.625 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.625 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.625 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.774402+0000", 2024-09-11T07:46:29.625 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.62786251400000004 2024-09-11T07:46:29.625 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.625 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.625 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:29.625 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.774499+0000", 2024-09-11T07:46:29.625 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7490000000000003e-05 2024-09-11T07:46:29.625 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.625 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.626 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,6", 2024-09-11T07:46:29.626 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.774603+0000", 2024-09-11T07:46:29.626 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000103552 2024-09-11T07:46:29.626 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.626 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.626 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.626 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.776274+0000", 2024-09-11T07:46:29.626 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0016714830000000001 2024-09-11T07:46:29.626 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.626 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.626 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:29.626 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.778184+0000", 2024-09-11T07:46:29.626 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0019098679999999999 2024-09-11T07:46:29.626 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.627 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.627 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.627 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.779812+0000", 2024-09-11T07:46:29.627 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0016285239999999999 2024-09-11T07:46:29.627 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.627 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.627 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:29.627 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.779863+0000", 2024-09-11T07:46:29.627 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0645e-05 2024-09-11T07:46:29.627 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.627 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.627 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:29.627 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.779901+0000", 2024-09-11T07:46:29.628 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8016999999999999e-05 2024-09-11T07:46:29.628 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.628 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:29.628 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.628 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.628 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.628 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25706.0:254623 10.0 10:07d7aa64:::benchmark_data_smithi022_163866_object125197:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e397)", 2024-09-11T07:46:29.628 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:45:30.156266+0000", 2024-09-11T07:46:29.628 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 59.332290215, 2024-09-11T07:46:29.628 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.62372887399999999, 2024-09-11T07:46:29.628 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:29.628 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:29.628 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:29.629 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25706", 2024-09-11T07:46:29.629 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:46:29.629 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 254623 2024-09-11T07:46:29.629 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.629 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:29.629 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.629 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:29.629 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.156266+0000", 2024-09-11T07:46:29.629 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.629 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.629 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.629 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:29.629 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.156266+0000", 2024-09-11T07:46:29.629 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.630 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.630 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.630 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:29.630 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.156265+0000", 2024-09-11T07:46:29.630 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:46:29.630 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.630 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.630 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:29.630 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.156268+0000", 2024-09-11T07:46:29.630 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5659999999999998e-06 2024-09-11T07:46:29.630 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.630 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.631 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:29.631 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.156275+0000", 2024-09-11T07:46:29.631 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.917e-06 2024-09-11T07:46:29.631 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.631 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.631 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:29.631 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.156279+0000", 2024-09-11T07:46:29.631 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1030000000000004e-06 2024-09-11T07:46:29.631 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.631 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.631 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.631 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.156323+0000", 2024-09-11T07:46:29.632 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3260999999999998e-05 2024-09-11T07:46:29.632 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.632 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.632 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:29.632 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.156330+0000", 2024-09-11T07:46:29.632 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7859999999999997e-06 2024-09-11T07:46:29.632 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.632 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.632 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.632 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.776310+0000", 2024-09-11T07:46:29.632 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.61997906000000003 2024-09-11T07:46:29.632 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.633 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.633 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:29.633 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.776422+0000", 2024-09-11T07:46:29.633 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000112073 2024-09-11T07:46:29.633 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.633 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.633 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,6", 2024-09-11T07:46:29.633 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.776493+0000", 2024-09-11T07:46:29.633 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1768999999999997e-05 2024-09-11T07:46:29.633 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.633 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.633 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.633 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.779770+0000", 2024-09-11T07:46:29.634 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0032763890000000002 2024-09-11T07:46:29.634 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.634 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.634 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.634 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.779939+0000", 2024-09-11T07:46:29.634 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000169517 2024-09-11T07:46:29.634 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.634 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.634 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:29.634 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.779944+0000", 2024-09-11T07:46:29.634 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3950000000000003e-06 2024-09-11T07:46:29.634 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.634 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.634 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:29.635 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.779971+0000", 2024-09-11T07:46:29.635 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.7183999999999999e-05 2024-09-11T07:46:29.635 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.635 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.635 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:29.635 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.779994+0000", 2024-09-11T07:46:29.635 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3499999999999999e-05 2024-09-11T07:46:29.635 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.635 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:29.635 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.635 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.635 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.635 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25706.0:254630 10.c 10:313fa01e:::benchmark_data_smithi022_163866_object125204:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e397)", 2024-09-11T07:46:29.636 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:45:30.162527+0000", 2024-09-11T07:46:29.636 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 59.326028888000003, 2024-09-11T07:46:29.636 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.62831541899999999, 2024-09-11T07:46:29.636 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:29.636 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:29.636 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:29.636 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25706", 2024-09-11T07:46:29.636 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:46:29.636 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 254630 2024-09-11T07:46:29.636 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.636 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:29.636 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.636 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:29.636 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.162527+0000", 2024-09-11T07:46:29.637 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.637 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.637 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.637 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:29.637 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.162527+0000", 2024-09-11T07:46:29.637 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.637 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.637 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.637 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:29.637 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.162524+0000", 2024-09-11T07:46:29.637 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:46:29.637 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.637 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.638 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:29.638 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.162532+0000", 2024-09-11T07:46:29.638 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5599999999999996e-06 2024-09-11T07:46:29.638 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.638 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.638 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:29.638 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.162542+0000", 2024-09-11T07:46:29.638 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8630000000000003e-06 2024-09-11T07:46:29.638 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.638 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.638 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:29.638 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.162548+0000", 2024-09-11T07:46:29.638 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4320000000000004e-06 2024-09-11T07:46:29.638 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.639 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.639 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.639 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.162608+0000", 2024-09-11T07:46:29.639 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9791999999999999e-05 2024-09-11T07:46:29.639 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.639 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.639 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:29.639 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.162616+0000", 2024-09-11T07:46:29.639 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7789999999999996e-06 2024-09-11T07:46:29.639 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.639 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.639 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.640 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.775714+0000", 2024-09-11T07:46:29.640 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.61309788799999998 2024-09-11T07:46:29.640 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.640 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.640 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:29.640 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.775809+0000", 2024-09-11T07:46:29.640 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4990999999999999e-05 2024-09-11T07:46:29.640 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.640 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.640 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:46:29.640 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.775925+0000", 2024-09-11T07:46:29.640 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000116647 2024-09-11T07:46:29.640 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.641 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.641 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:29.641 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790337+0000", 2024-09-11T07:46:29.641 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.014412059 2024-09-11T07:46:29.641 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.641 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.641 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.641 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790725+0000", 2024-09-11T07:46:29.641 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00038779199999999998 2024-09-11T07:46:29.641 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.641 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.641 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.641 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790760+0000", 2024-09-11T07:46:29.641 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4962000000000002e-05 2024-09-11T07:46:29.642 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.642 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.642 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:29.642 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790801+0000", 2024-09-11T07:46:29.642 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1056999999999997e-05 2024-09-11T07:46:29.642 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.642 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.642 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:29.642 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790842+0000", 2024-09-11T07:46:29.642 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0992000000000003e-05 2024-09-11T07:46:29.642 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.642 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:29.642 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.643 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.643 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.643 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25706.0:254640 10.0 10:0d123341:::benchmark_data_smithi022_163866_object125214:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e397)", 2024-09-11T07:46:29.643 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:45:30.171832+0000", 2024-09-11T07:46:29.643 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 59.316723279000001, 2024-09-11T07:46:29.643 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.611698729, 2024-09-11T07:46:29.643 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:29.643 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:29.643 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:29.643 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25706", 2024-09-11T07:46:29.643 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:46:29.643 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 254640 2024-09-11T07:46:29.643 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.644 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:29.644 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.644 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:29.644 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.171832+0000", 2024-09-11T07:46:29.644 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.644 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.644 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.644 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:29.644 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.171832+0000", 2024-09-11T07:46:29.644 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.644 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.644 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.645 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:29.645 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.171831+0000", 2024-09-11T07:46:29.645 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:46:29.645 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.645 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.645 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:29.645 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.171837+0000", 2024-09-11T07:46:29.645 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8649999999999998e-06 2024-09-11T07:46:29.645 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.645 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.645 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:29.645 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.171848+0000", 2024-09-11T07:46:29.646 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1804000000000001e-05 2024-09-11T07:46:29.646 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.646 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.646 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:29.646 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.171852+0000", 2024-09-11T07:46:29.646 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9020000000000001e-06 2024-09-11T07:46:29.646 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.646 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.646 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.646 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.171886+0000", 2024-09-11T07:46:29.646 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3470999999999999e-05 2024-09-11T07:46:29.646 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.646 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.647 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:29.647 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.171893+0000", 2024-09-11T07:46:29.647 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0310000000000001e-06 2024-09-11T07:46:29.647 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.647 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.647 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.647 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.778229+0000", 2024-09-11T07:46:29.647 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.60633658499999998 2024-09-11T07:46:29.647 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.647 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.647 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:29.647 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.778340+0000", 2024-09-11T07:46:29.647 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000110685 2024-09-11T07:46:29.647 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.648 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.648 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,6", 2024-09-11T07:46:29.648 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.778452+0000", 2024-09-11T07:46:29.648 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000112088 2024-09-11T07:46:29.648 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.648 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.648 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.648 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.783446+0000", 2024-09-11T07:46:29.648 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0049940729999999999 2024-09-11T07:46:29.648 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.648 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.648 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:29.648 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.783452+0000", 2024-09-11T07:46:29.648 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5720000000000002e-06 2024-09-11T07:46:29.649 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.649 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.649 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.649 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.783479+0000", 2024-09-11T07:46:29.649 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.7725999999999998e-05 2024-09-11T07:46:29.649 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.649 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.649 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:29.649 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.783508+0000", 2024-09-11T07:46:29.649 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.8106000000000002e-05 2024-09-11T07:46:29.649 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.649 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.649 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:29.650 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.783531+0000", 2024-09-11T07:46:29.652 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3614000000000001e-05 2024-09-11T07:46:29.652 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.652 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:29.652 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.653 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.653 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.653 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25706.0:254645 10.c 10:36dbc198:::benchmark_data_smithi022_163866_object125219:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e397)", 2024-09-11T07:46:29.653 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:45:30.178343+0000", 2024-09-11T07:46:29.653 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 59.310212544999999, 2024-09-11T07:46:29.653 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.612674206, 2024-09-11T07:46:29.653 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:29.653 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:29.653 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:29.653 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25706", 2024-09-11T07:46:29.653 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:46:29.653 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 254645 2024-09-11T07:46:29.653 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.654 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:29.654 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.654 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:29.654 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.178343+0000", 2024-09-11T07:46:29.654 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.654 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.654 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.654 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:29.654 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.178343+0000", 2024-09-11T07:46:29.654 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.654 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.654 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.654 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:29.654 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.178341+0000", 2024-09-11T07:46:29.655 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:46:29.655 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.655 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.655 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:29.655 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.178349+0000", 2024-09-11T07:46:29.655 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6389999999999992e-06 2024-09-11T07:46:29.655 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.655 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.655 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:29.655 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.178362+0000", 2024-09-11T07:46:29.655 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3557000000000001e-05 2024-09-11T07:46:29.655 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.655 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.656 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:29.656 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.178368+0000", 2024-09-11T07:46:29.656 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.215e-06 2024-09-11T07:46:29.656 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.656 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.656 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.656 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.178421+0000", 2024-09-11T07:46:29.656 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3758999999999998e-05 2024-09-11T07:46:29.656 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.656 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.656 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:29.657 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.178431+0000", 2024-09-11T07:46:29.657 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.1589999999999999e-06 2024-09-11T07:46:29.657 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.657 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.657 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.657 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.778452+0000", 2024-09-11T07:46:29.657 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.600021833 2024-09-11T07:46:29.657 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.657 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.657 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:29.657 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.778529+0000", 2024-09-11T07:46:29.657 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6391999999999996e-05 2024-09-11T07:46:29.657 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.658 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.658 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:46:29.658 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.778643+0000", 2024-09-11T07:46:29.658 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000113832 2024-09-11T07:46:29.658 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.658 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.658 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:29.658 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790342+0000", 2024-09-11T07:46:29.658 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.011699620000000001 2024-09-11T07:46:29.658 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.658 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.658 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.658 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790897+0000", 2024-09-11T07:46:29.659 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00055455900000000004 2024-09-11T07:46:29.659 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.659 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.659 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.659 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790942+0000", 2024-09-11T07:46:29.659 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4991999999999999e-05 2024-09-11T07:46:29.659 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.659 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.659 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:29.659 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790983+0000", 2024-09-11T07:46:29.659 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1446999999999998e-05 2024-09-11T07:46:29.659 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.659 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.659 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:29.660 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.791017+0000", 2024-09-11T07:46:29.660 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4189000000000001e-05 2024-09-11T07:46:29.660 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.660 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:29.660 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.660 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.660 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.660 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25706.0:254646 10.c 10:34b24ed9:::benchmark_data_smithi022_163866_object125220:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e397)", 2024-09-11T07:46:29.660 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:45:30.178934+0000", 2024-09-11T07:46:29.660 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 59.309622009999998, 2024-09-11T07:46:29.660 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.61233413599999997, 2024-09-11T07:46:29.660 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:29.660 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:29.661 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:29.661 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25706", 2024-09-11T07:46:29.661 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:46:29.661 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 254646 2024-09-11T07:46:29.661 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.661 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:29.661 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.661 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:29.661 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.178934+0000", 2024-09-11T07:46:29.661 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.661 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.661 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.661 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:29.662 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.178934+0000", 2024-09-11T07:46:29.662 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.662 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.662 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.662 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:29.662 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.178932+0000", 2024-09-11T07:46:29.662 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:46:29.662 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.662 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.662 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:29.662 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.178938+0000", 2024-09-11T07:46:29.662 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8649999999999998e-06 2024-09-11T07:46:29.662 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.663 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.663 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:29.663 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.178947+0000", 2024-09-11T07:46:29.663 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3070000000000008e-06 2024-09-11T07:46:29.663 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.663 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.663 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:29.663 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.178951+0000", 2024-09-11T07:46:29.663 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0319999999999997e-06 2024-09-11T07:46:29.663 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.663 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.663 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.663 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.178987+0000", 2024-09-11T07:46:29.664 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5945999999999998e-05 2024-09-11T07:46:29.664 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.664 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.664 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:29.664 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.178994+0000", 2024-09-11T07:46:29.664 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4309999999999999e-06 2024-09-11T07:46:29.664 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.664 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.664 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.664 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.781252+0000", 2024-09-11T07:46:29.664 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.60225796300000001 2024-09-11T07:46:29.664 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.664 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.664 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:29.665 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.781318+0000", 2024-09-11T07:46:29.665 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5988999999999994e-05 2024-09-11T07:46:29.665 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.665 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.665 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:46:29.665 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.781397+0000", 2024-09-11T07:46:29.665 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9511000000000005e-05 2024-09-11T07:46:29.665 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.665 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.665 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:29.665 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790463+0000", 2024-09-11T07:46:29.665 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0090655100000000006 2024-09-11T07:46:29.666 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.666 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.666 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.666 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.791058+0000", 2024-09-11T07:46:29.666 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000595631 2024-09-11T07:46:29.666 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.666 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.666 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.666 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.791119+0000", 2024-09-11T07:46:29.666 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0637000000000002e-05 2024-09-11T07:46:29.666 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.666 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.666 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:29.667 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.791232+0000", 2024-09-11T07:46:29.667 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000112748 2024-09-11T07:46:29.667 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.667 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.667 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:29.667 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.791268+0000", 2024-09-11T07:46:29.667 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6037000000000001e-05 2024-09-11T07:46:29.667 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.667 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:29.667 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.667 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.667 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.667 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25706.0:254653 10.0 10:0fbe1c80:::benchmark_data_smithi022_163866_object125227:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e397)", 2024-09-11T07:46:29.668 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:45:30.191390+0000", 2024-09-11T07:46:29.668 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 59.297165935999999, 2024-09-11T07:46:29.668 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.59533172499999998, 2024-09-11T07:46:29.668 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:29.668 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:29.668 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:29.668 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25706", 2024-09-11T07:46:29.668 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:46:29.668 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 254653 2024-09-11T07:46:29.668 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.668 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:29.668 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.668 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:29.669 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.191390+0000", 2024-09-11T07:46:29.669 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.669 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.669 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.669 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:29.669 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.191390+0000", 2024-09-11T07:46:29.669 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.669 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.669 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.669 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:29.669 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.191388+0000", 2024-09-11T07:46:29.669 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:46:29.669 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.669 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.670 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:29.670 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.191394+0000", 2024-09-11T07:46:29.670 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2700000000000004e-06 2024-09-11T07:46:29.670 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.670 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.670 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:29.670 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.191403+0000", 2024-09-11T07:46:29.670 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8300000000000008e-06 2024-09-11T07:46:29.670 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.670 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.670 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:29.670 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.191408+0000", 2024-09-11T07:46:29.670 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5129999999999998e-06 2024-09-11T07:46:29.670 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.671 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.671 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.671 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.191437+0000", 2024-09-11T07:46:29.671 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9141000000000001e-05 2024-09-11T07:46:29.671 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.671 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.671 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:29.671 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.191443+0000", 2024-09-11T07:46:29.671 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5600000000000001e-06 2024-09-11T07:46:29.671 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.671 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.671 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.671 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.780026+0000", 2024-09-11T07:46:29.671 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.58858348100000002 2024-09-11T07:46:29.672 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.672 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.672 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:29.672 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.780125+0000", 2024-09-11T07:46:29.672 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.904e-05 2024-09-11T07:46:29.672 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.672 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.672 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,6", 2024-09-11T07:46:29.672 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.780230+0000", 2024-09-11T07:46:29.672 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000104573 2024-09-11T07:46:29.672 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.672 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.672 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.672 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.783568+0000", 2024-09-11T07:46:29.672 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00333878 2024-09-11T07:46:29.673 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.673 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.673 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:29.673 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.783573+0000", 2024-09-11T07:46:29.673 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6199999999999998e-06 2024-09-11T07:46:29.673 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.673 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.673 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.673 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.786637+0000", 2024-09-11T07:46:29.673 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003064008 2024-09-11T07:46:29.673 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.673 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.673 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:29.673 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.786678+0000", 2024-09-11T07:46:29.674 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1028000000000001e-05 2024-09-11T07:46:29.674 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.674 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.674 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:29.674 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.786722+0000", 2024-09-11T07:46:29.674 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3409999999999999e-05 2024-09-11T07:46:29.674 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.674 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:29.674 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.674 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.674 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.674 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25706.0:254659 10.c 10:3b7cc1a9:::benchmark_data_smithi022_163866_object125233:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e397)", 2024-09-11T07:46:29.674 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:45:30.196468+0000", 2024-09-11T07:46:29.674 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 59.292087262000003, 2024-09-11T07:46:29.674 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.59493246700000002, 2024-09-11T07:46:29.675 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:46:29.675 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:29.675 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:46:29.675 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25706", 2024-09-11T07:46:29.675 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:46:29.675 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 254659 2024-09-11T07:46:29.675 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.675 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:46:29.675 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.675 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:46:29.675 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.196468+0000", 2024-09-11T07:46:29.675 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.675 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.675 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.675 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:46:29.676 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.196468+0000", 2024-09-11T07:46:29.676 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:46:29.676 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.676 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.676 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:46:29.676 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.196467+0000", 2024-09-11T07:46:29.676 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:46:29.676 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.676 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.676 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:46:29.676 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.196473+0000", 2024-09-11T07:46:29.676 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1129999999999997e-06 2024-09-11T07:46:29.676 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.677 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.677 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:46:29.677 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.196486+0000", 2024-09-11T07:46:29.677 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3278000000000001e-05 2024-09-11T07:46:29.677 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.677 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.677 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:46:29.677 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.196493+0000", 2024-09-11T07:46:29.677 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4620000000000001e-06 2024-09-11T07:46:29.677 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.677 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.677 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.677 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.196532+0000", 2024-09-11T07:46:29.677 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9536999999999998e-05 2024-09-11T07:46:29.677 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.678 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.678 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:46:29.678 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.196540+0000", 2024-09-11T07:46:29.678 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6249999999999998e-06 2024-09-11T07:46:29.678 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.678 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.678 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:46:29.678 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.783734+0000", 2024-09-11T07:46:29.678 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.58719373200000002 2024-09-11T07:46:29.678 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.678 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.678 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:46:29.678 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.783812+0000", 2024-09-11T07:46:29.679 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7966000000000005e-05 2024-09-11T07:46:29.679 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.679 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.679 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:46:29.679 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.783905+0000", 2024-09-11T07:46:29.679 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3505999999999995e-05 2024-09-11T07:46:29.679 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.679 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.679 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:46:29.679 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790470+0000", 2024-09-11T07:46:29.679 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.006564304 2024-09-11T07:46:29.679 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.679 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.680 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.680 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.791088+0000", 2024-09-11T07:46:29.680 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00061853099999999996 2024-09-11T07:46:29.680 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.680 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.680 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:29.680 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.791302+0000", 2024-09-11T07:46:29.680 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000213453 2024-09-11T07:46:29.680 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.680 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.680 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:46:29.680 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.791368+0000", 2024-09-11T07:46:29.681 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6582999999999996e-05 2024-09-11T07:46:29.681 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:46:29.681 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:46:29.681 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:46:29.681 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.791401+0000", 2024-09-11T07:46:29.681 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2747999999999999e-05 2024-09-11T07:46:29.681 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.681 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:29.681 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.681 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:46:29.681 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:46:29.681 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:46:29.682 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:29.707 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:29.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:29.708 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:29.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:29 smithi104 ceph-mon[33507]: 2.0 scrub starts 2024-09-11T07:46:29.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:29 smithi104 ceph-mon[33507]: 2.0 scrub ok 2024-09-11T07:46:29.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:29 smithi104 ceph-mon[33507]: 6.1b deep-scrub starts 2024-09-11T07:46:29.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:29 smithi104 ceph-mon[33507]: 6.1b deep-scrub ok 2024-09-11T07:46:29.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:29 smithi104 ceph-mon[33507]: pgmap v933: 158 pgs: 158 active+clean; 1.7 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 111 MiB/s wr, 1.77k op/s 2024-09-11T07:46:29.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:29 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3279008534' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:46:29.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:29 smithi104 ceph-mon[33507]: from='client.26014 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:46:29.809 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:29.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:29.851 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:29.851 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:29.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:29 smithi022 ceph-mon[29462]: 2.0 scrub starts 2024-09-11T07:46:29.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:29 smithi022 ceph-mon[29462]: 2.0 scrub ok 2024-09-11T07:46:29.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:29 smithi022 ceph-mon[29462]: 6.1b deep-scrub starts 2024-09-11T07:46:29.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:29 smithi022 ceph-mon[29462]: 6.1b deep-scrub ok 2024-09-11T07:46:29.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:29 smithi022 ceph-mon[29462]: pgmap v933: 158 pgs: 158 active+clean; 1.7 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 111 MiB/s wr, 1.77k op/s 2024-09-11T07:46:29.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:29 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3279008534' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:46:29.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:29 smithi022 ceph-mon[29462]: from='client.26014 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:46:29.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:29 smithi022 ceph-mon[37330]: 2.0 scrub starts 2024-09-11T07:46:29.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:29 smithi022 ceph-mon[37330]: 2.0 scrub ok 2024-09-11T07:46:29.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:29 smithi022 ceph-mon[37330]: 6.1b deep-scrub starts 2024-09-11T07:46:29.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:29 smithi022 ceph-mon[37330]: 6.1b deep-scrub ok 2024-09-11T07:46:29.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:29 smithi022 ceph-mon[37330]: pgmap v933: 158 pgs: 158 active+clean; 1.7 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 111 MiB/s wr, 1.77k op/s 2024-09-11T07:46:29.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:29 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3279008534' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:46:29.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:29 smithi022 ceph-mon[37330]: from='client.26014 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:46:29.952 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:30.007 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:30.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:30.009 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:30.101 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 18 16 36461 36445 126.531 88.125 0.00350731 0.00789185 2024-09-11T07:46:30.109 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:30.131 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:30.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:30.132 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:30.233 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:30.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:30.276 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:30.279 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:46:30 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:46:30.377 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:30.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:30.416 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:30.416 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:30.468 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-11T07:46:30.499 INFO:teuthology.orchestra.run.smithi022.stderr:set osd.5 primary-affinity to 0 (802) 2024-09-11T07:46:30.516 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:30.541 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:30.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:30.542 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:30.643 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:30.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:46:30 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:46:30] "GET /metrics HTTP/1.1" 200 225100 "" "Prometheus/2.43.0" 2024-09-11T07:46:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:30 smithi022 ceph-mon[29462]: 1.2 scrub starts 2024-09-11T07:46:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:30 smithi022 ceph-mon[29462]: 1.2 scrub ok 2024-09-11T07:46:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:30 smithi022 ceph-mon[29462]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:46:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:30 smithi022 ceph-mon[29462]: from='client.26014 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:46:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:30 smithi022 ceph-mon[29462]: osdmap e423: 8 total, 8 up, 7 in 2024-09-11T07:46:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:30 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1690419135' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.0}]: dispatch 2024-09-11T07:46:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:30 smithi022 ceph-mon[29462]: from='client.25994 ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.0}]: dispatch 2024-09-11T07:46:30.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:30 smithi022 ceph-mon[37330]: 1.2 scrub starts 2024-09-11T07:46:30.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:30 smithi022 ceph-mon[37330]: 1.2 scrub ok 2024-09-11T07:46:30.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:30 smithi022 ceph-mon[37330]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:46:30.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:30 smithi022 ceph-mon[37330]: from='client.26014 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:46:30.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:30 smithi022 ceph-mon[37330]: osdmap e423: 8 total, 8 up, 7 in 2024-09-11T07:46:30.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:30 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1690419135' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.0}]: dispatch 2024-09-11T07:46:30.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:30 smithi022 ceph-mon[37330]: from='client.25994 ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.0}]: dispatch 2024-09-11T07:46:30.679 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:30.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:30.680 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:30.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:30 smithi104 ceph-mon[33507]: 1.2 scrub starts 2024-09-11T07:46:30.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:30 smithi104 ceph-mon[33507]: 1.2 scrub ok 2024-09-11T07:46:30.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:30 smithi104 ceph-mon[33507]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:46:30.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:30 smithi104 ceph-mon[33507]: from='client.26014 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:46:30.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:30 smithi104 ceph-mon[33507]: osdmap e423: 8 total, 8 up, 7 in 2024-09-11T07:46:30.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:30 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1690419135' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.0}]: dispatch 2024-09-11T07:46:30.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:30 smithi104 ceph-mon[33507]: from='client.25994 ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.0}]: dispatch 2024-09-11T07:46:30.781 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:30.803 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:30.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:30.804 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:30.906 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:30.927 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:30.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:30.928 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:31.029 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:31.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:31.051 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:31.052 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:31.101 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 19 16 37241 37225 122.437 48.75 0.00244524 0.00794399 2024-09-11T07:46:31.152 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:31.178 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:31.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:31.179 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:31.280 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:31.301 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:31.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:31.302 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:31.403 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:31.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:31.426 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:31.427 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:31.528 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:31.569 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:31.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:31.571 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:31.672 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:31.712 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:31.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:31.713 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:31.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:31 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:46:31.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:31 smithi104 ceph-mon[33507]: pgmap v935: 158 pgs: 158 active+clean; 2.2 GiB data, 12 GiB used, 613 GiB / 626 GiB avail; 104 MiB/s wr, 1.66k op/s 2024-09-11T07:46:31.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:31 smithi104 ceph-mon[33507]: from='client.25994 ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.0}]': finished 2024-09-11T07:46:31.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:31 smithi104 ceph-mon[33507]: osdmap e424: 8 total, 8 up, 7 in 2024-09-11T07:46:31.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:31 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:46:31.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:31 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:46:31.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:31 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:46:31.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:31 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:46:31.814 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:31.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:31.854 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:31.854 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:31.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:31 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:46:31.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:31 smithi022 ceph-mon[29462]: pgmap v935: 158 pgs: 158 active+clean; 2.2 GiB data, 12 GiB used, 613 GiB / 626 GiB avail; 104 MiB/s wr, 1.66k op/s 2024-09-11T07:46:31.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:31 smithi022 ceph-mon[29462]: from='client.25994 ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.0}]': finished 2024-09-11T07:46:31.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:31 smithi022 ceph-mon[29462]: osdmap e424: 8 total, 8 up, 7 in 2024-09-11T07:46:31.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:31 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:46:31.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:31 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:46:31.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:31 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:46:31.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:31 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:46:31.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:31 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:46:31.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:31 smithi022 ceph-mon[37330]: pgmap v935: 158 pgs: 158 active+clean; 2.2 GiB data, 12 GiB used, 613 GiB / 626 GiB avail; 104 MiB/s wr, 1.66k op/s 2024-09-11T07:46:31.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:31 smithi022 ceph-mon[37330]: from='client.25994 ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 0.0}]': finished 2024-09-11T07:46:31.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:31 smithi022 ceph-mon[37330]: osdmap e424: 8 total, 8 up, 7 in 2024-09-11T07:46:31.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:31 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:46:31.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:31 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:46:31.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:31 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:46:31.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:31 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:46:31.955 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:31.982 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:31.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:31.983 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:32.052 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:46:32.084 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:32.102 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:2024-09-11T07:46:32.101538+0000 min lat: 0.00115663 max lat: 1.02411 avg lat: 0.0083592 2024-09-11T07:46:32.102 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T07:46:32.102 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 20 16 38269 38253 119.525 64.25 0.0141928 0.0083592 2024-09-11T07:46:32.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:32.145 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:32.146 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:32.246 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:32.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:32.288 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:32.288 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:32.389 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:32.427 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:32.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:32.428 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:32.529 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:32.556 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:32.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:32.557 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:32.658 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:32.684 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:32.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:32.685 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:32.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:32 smithi104 ceph-mon[33507]: osdmap e425: 8 total, 8 up, 7 in 2024-09-11T07:46:32.787 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:32.812 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:32.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:32.813 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:32.857 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T07:46:32.869 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:32 smithi022 ceph-mon[29462]: osdmap e425: 8 total, 8 up, 7 in 2024-09-11T07:46:32.869 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:32 smithi022 ceph-mon[37330]: osdmap e425: 8 total, 8 up, 7 in 2024-09-11T07:46:32.914 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:32.962 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:32.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:32.963 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:33.064 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:33.090 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:33.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:33.091 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:33.102 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 21 16 39972 39956 118.902 106.438 0.00665075 0.00840447 2024-09-11T07:46:33.192 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:33.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:33.217 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:33.218 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:33.318 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:33.343 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:33.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:33.344 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:33.445 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:33.471 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:33.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:33.472 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:33.573 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:33.599 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:33.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:33.601 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:33.701 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:33.744 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:33.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:33.745 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:33.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:33 smithi104 ceph-mon[33507]: pgmap v938: 158 pgs: 158 active+clean; 2.2 GiB data, 12 GiB used, 613 GiB / 626 GiB avail; 75 MiB/s wr, 1.19k op/s 2024-09-11T07:46:33.846 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:33.847 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:33 smithi022 ceph-mon[37330]: pgmap v938: 158 pgs: 158 active+clean; 2.2 GiB data, 12 GiB used, 613 GiB / 626 GiB avail; 75 MiB/s wr, 1.19k op/s 2024-09-11T07:46:33.847 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:33 smithi022 ceph-mon[29462]: pgmap v938: 158 pgs: 158 active+clean; 2.2 GiB data, 12 GiB used, 613 GiB / 626 GiB avail; 75 MiB/s wr, 1.19k op/s 2024-09-11T07:46:33.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:33.896 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:33.897 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:33.998 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:34.024 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:34.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:34.025 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:34.034 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:46:34.034 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:46:34.072 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:46:34.072 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:46:34.102 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 22 16 41604 41588 118.133 102 0.0110756 0.00845932 2024-09-11T07:46:34.110 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:46:34.111 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:46:34.125 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:34.145 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:46:34.145 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:46:34.150 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:34.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:34.151 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:34.181 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:46:34.182 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:46:34.213 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:46:34.214 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:46:34.248 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:46:34.248 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:46:34.252 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:34.280 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:46:34.280 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:46:34.291 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:34.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:34.292 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:34.393 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:34.435 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:34.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:34.436 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:34.520 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:46:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:46:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:46:34.537 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:34.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:34.565 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:34.565 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:34.666 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:34.721 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:34.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:34.722 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:34.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:34 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3034691780' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:46:34.823 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:34.829 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:34 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3034691780' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:46:34.831 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:34 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3034691780' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:46:34.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:34.876 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:34.876 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:34.977 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:35.012 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:46:35.034 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:35.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:35.035 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:35.104 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 23 16 43185 43169 117.284 98.8125 0.00840626 0.00852076 2024-09-11T07:46:35.136 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:35.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:35.177 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:35.178 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:35.278 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:35.300 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:35.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:35.300 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:35.402 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:35.445 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:35.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:35.446 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:35.547 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:35.574 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:35.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:35.575 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:35.676 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:35.714 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:35.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:35.715 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:35.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:35 smithi104 ceph-mon[33507]: pgmap v939: 158 pgs: 158 active+clean; 2.4 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.7 KiB/s rd, 113 MiB/s wr, 1.80k op/s 2024-09-11T07:46:35.815 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:35.837 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:35.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:35.839 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:35.916 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:35 smithi022 ceph-mon[37330]: pgmap v939: 158 pgs: 158 active+clean; 2.4 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.7 KiB/s rd, 113 MiB/s wr, 1.80k op/s 2024-09-11T07:46:35.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:35 smithi022 ceph-mon[29462]: pgmap v939: 158 pgs: 158 active+clean; 2.4 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.7 KiB/s rd, 113 MiB/s wr, 1.80k op/s 2024-09-11T07:46:35.939 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:35.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:35.962 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:35.963 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:36.063 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:36.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:36.086 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:36.086 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:36.103 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 24 16 44118 44102 114.827 58.3125 0.00211377 0.00858783 2024-09-11T07:46:36.186 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:36.213 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:36.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:36.214 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:36.315 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:36.353 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:36.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:36.355 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:36.455 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:36.494 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:36.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:36.495 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:36.596 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:36.621 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:36.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:36.622 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:36.635 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:46:36.635 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:46:36.723 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:36.765 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:36.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:36.766 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:36.867 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:36.892 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:36.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:36.893 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:36.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:36 smithi022 ceph-mon[29462]: osdmap e426: 8 total, 8 up, 7 in 2024-09-11T07:46:36.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:36 smithi022 ceph-mon[29462]: pgmap v941: 158 pgs: 158 active+clean; 2.6 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 4.2 KiB/s rd, 69 MiB/s wr, 1.10k op/s; 17 KiB/s, 0 objects/s recovering 2024-09-11T07:46:36.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:36 smithi022 ceph-mon[37330]: osdmap e426: 8 total, 8 up, 7 in 2024-09-11T07:46:36.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:36 smithi022 ceph-mon[37330]: pgmap v941: 158 pgs: 158 active+clean; 2.6 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 4.2 KiB/s rd, 69 MiB/s wr, 1.10k op/s; 17 KiB/s, 0 objects/s recovering 2024-09-11T07:46:36.994 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:37.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:36 smithi104 ceph-mon[33507]: osdmap e426: 8 total, 8 up, 7 in 2024-09-11T07:46:37.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:36 smithi104 ceph-mon[33507]: pgmap v941: 158 pgs: 158 active+clean; 2.6 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 4.2 KiB/s rd, 69 MiB/s wr, 1.10k op/s; 17 KiB/s, 0 objects/s recovering 2024-09-11T07:46:37.035 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:37.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:37.036 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:37.062 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:46:37.062 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:46:37.062 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_4 2024-09-11T07:46:37.062 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool unique_pool_4 by 10 min 8 2024-09-11T07:46:37.063 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:46:37.105 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 25 16 45625 45609 113.996 94.1875 0.00659664 0.00876704 2024-09-11T07:46:37.138 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:37.162 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:37.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:37.163 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:37.264 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:37.289 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:37.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:37.290 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:37.390 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:37.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:37.444 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:37.444 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:37.545 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:37.586 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:37.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:37.587 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:37.688 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:37.725 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:37.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:37.726 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:37.827 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:37.853 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:37.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:37.854 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:37.955 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:37.982 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:37.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:37.982 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:38.083 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:38.105 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 26 16 46983 46967 112.875 84.875 0.00270032 0.00884698 2024-09-11T07:46:38.111 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:38.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:38.112 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:38.213 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:38.252 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:38.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:38.253 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:38.354 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:38.400 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:38.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:38.401 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:38.502 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:38.541 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:38.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:38.542 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:38.643 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:38.664 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:38.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:38.665 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:38.766 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:38.807 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:38.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:38.809 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:38.909 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:38.956 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:38.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:38.957 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:39.058 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:39.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:39.101 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:39.101 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:39.105 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 27 16 48138 48122 111.368 72.1875 0.00620439 0.00897346 2024-09-11T07:46:39.203 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:39.227 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:39.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:39.228 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:39.329 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:39.348 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:39 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:46:39.348 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:39 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:46:39.348 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:39 smithi022 ceph-mon[29462]: pgmap v942: 158 pgs: 158 active+clean; 2.6 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 3.2 KiB/s rd, 52 MiB/s wr, 838 op/s; 13 KiB/s, 0 objects/s recovering 2024-09-11T07:46:39.348 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:39 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:46:39.348 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:39 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:46:39.348 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:39 smithi022 ceph-mon[37330]: pgmap v942: 158 pgs: 158 active+clean; 2.6 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 3.2 KiB/s rd, 52 MiB/s wr, 838 op/s; 13 KiB/s, 0 objects/s recovering 2024-09-11T07:46:39.387 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:39.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:39.388 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:39.489 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:39.515 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:39.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:39.516 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:39.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:39 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:46:39.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:39 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:46:39.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:39 smithi104 ceph-mon[33507]: pgmap v942: 158 pgs: 158 active+clean; 2.6 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 3.2 KiB/s rd, 52 MiB/s wr, 838 op/s; 13 KiB/s, 0 objects/s recovering 2024-09-11T07:46:39.617 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:39.654 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:39.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:39.655 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:39.756 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:39.784 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:39.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:39.785 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:39.886 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:39.914 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:39.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:39.915 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:40.016 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:40.059 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:40.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:40.060 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:40.105 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 28 16 49517 49501 110.469 86.1875 0.00649023 0.00904611 2024-09-11T07:46:40.162 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:40.205 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:40.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:40.206 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:40.307 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:40.350 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:40.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:40.351 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:40.415 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-11T07:46:40.452 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:40.507 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:40.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:40.508 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:40.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:46:40 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:46:40.610 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:40.658 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:40.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:40.660 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:40.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:46:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:46:40] "GET /metrics HTTP/1.1" 200 225126 "" "Prometheus/2.43.0" 2024-09-11T07:46:40.761 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:40.804 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:40.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:40.805 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:40.906 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:40.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:40.951 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:40.951 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:41.052 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:41.097 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:41.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:41.098 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:41.105 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 29 16 50904 50888 109.649 86.6875 0.00760998 0.00911454 2024-09-11T07:46:41.199 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:41.225 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:41.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:41.227 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:41.327 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:41.355 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:41.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:41.356 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:41.456 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:41.457 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:41 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:46:41.457 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:41 smithi022 ceph-mon[29462]: pgmap v943: 158 pgs: 158 active+clean; 3.0 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 23 KiB/s rd, 91 MiB/s wr, 1.50k op/s; 12 KiB/s, 0 objects/s recovering 2024-09-11T07:46:41.507 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:41.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:41.508 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:41.609 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:41.637 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:41.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:41.638 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:41.739 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:41.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:41.767 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:41.767 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:41.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:41 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:46:41.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:41 smithi104 ceph-mon[33507]: pgmap v943: 158 pgs: 158 active+clean; 3.0 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 23 KiB/s rd, 91 MiB/s wr, 1.50k op/s; 12 KiB/s, 0 objects/s recovering 2024-09-11T07:46:41.869 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:41.894 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:41.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:41.896 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:41.916 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:41 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:46:41.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:41 smithi022 ceph-mon[37330]: pgmap v943: 158 pgs: 158 active+clean; 3.0 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 23 KiB/s rd, 91 MiB/s wr, 1.50k op/s; 12 KiB/s, 0 objects/s recovering 2024-09-11T07:46:41.996 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:41.999 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:46:42.007 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:46:42.021 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:42.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:42.022 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:42.106 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 30 16 52373 52357 109.051 91.8125 0.0114169 0.009165 2024-09-11T07:46:42.123 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:42.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:42.175 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:42.175 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:42.276 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:42.317 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:42.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:42.318 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:42.419 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:42.445 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:42.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:42.447 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:42.548 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:42.573 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:42.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:42.574 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:42.675 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:42.709 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:42.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:42.710 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:42.811 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:42.837 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:42.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:42.838 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:42.894 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:46:42.939 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:42.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:42.987 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:42.987 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:43.088 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:43.106 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 31 16 53718 53702 108.245 84.0625 0.00400218 0.00921416 2024-09-11T07:46:43.109 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:43.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:43.110 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:43.211 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:43.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:43.258 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:43.258 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:43.360 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:43.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:43.405 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:43.405 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:43.506 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:43.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:43.537 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:43.537 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:43.638 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:43.666 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:43.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:43.667 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:43.768 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:43.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:43 smithi104 ceph-mon[33507]: pgmap v944: 158 pgs: 158 active+clean; 3.0 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 21 KiB/s rd, 82 MiB/s wr, 1.34k op/s; 10 KiB/s, 0 objects/s recovering 2024-09-11T07:46:43.790 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:43.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:43.791 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:43.891 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:43.915 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:43.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:43.916 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:43.916 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:43 smithi022 ceph-mon[37330]: pgmap v944: 158 pgs: 158 active+clean; 3.0 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 21 KiB/s rd, 82 MiB/s wr, 1.34k op/s; 10 KiB/s, 0 objects/s recovering 2024-09-11T07:46:43.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:43 smithi022 ceph-mon[29462]: pgmap v944: 158 pgs: 158 active+clean; 3.0 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 21 KiB/s rd, 82 MiB/s wr, 1.34k op/s; 10 KiB/s, 0 objects/s recovering 2024-09-11T07:46:44.017 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:44.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:44.056 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:44.056 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:44.106 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 32 15 54788 54773 106.954 66.9375 0.00613403 0.00934433 2024-09-11T07:46:44.157 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:44.199 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:44.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:44.200 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:44.301 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:44.322 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:44.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:44.323 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:44.424 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:44.466 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:44.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:44.468 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:44.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:46:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:46:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:46:44.569 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:44.610 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:44.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:44.611 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:44.712 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:44.746 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:44.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:44.748 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:44.849 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:44.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:44.892 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:44.893 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:44.994 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:45.016 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:45.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:45.017 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:45.106 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 33 16 55765 55749 105.561 61 0.00237733 0.00945609 2024-09-11T07:46:45.118 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:45.154 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:45.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:45.156 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:45.256 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:45.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:45.283 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:45.283 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:45.384 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:45.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:45.407 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:45.407 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:45.508 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:45.548 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:45.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:45.549 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:45.650 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:45.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:45 smithi022 ceph-mon[29462]: pgmap v945: 158 pgs: 158 active+clean; 3.2 GiB data, 15 GiB used, 610 GiB / 626 GiB avail; 33 KiB/s rd, 81 MiB/s wr, 1.36k op/s 2024-09-11T07:46:45.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:45 smithi022 ceph-mon[37330]: pgmap v945: 158 pgs: 158 active+clean; 3.2 GiB data, 15 GiB used, 610 GiB / 626 GiB avail; 33 KiB/s rd, 81 MiB/s wr, 1.36k op/s 2024-09-11T07:46:45.674 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:45.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:45.676 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:45.777 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:45.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:45 smithi104 ceph-mon[33507]: pgmap v945: 158 pgs: 158 active+clean; 3.2 GiB data, 15 GiB used, 610 GiB / 626 GiB avail; 33 KiB/s rd, 81 MiB/s wr, 1.36k op/s 2024-09-11T07:46:45.804 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:45.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:45.805 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:45.906 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:45.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:45.957 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:45.957 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:46.058 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:46.082 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:46.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:46.083 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:46.109 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 34 16 57056 57040 104.823 80.6875 0.0120932 0.00953443 2024-09-11T07:46:46.184 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:46.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:46.237 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:46.237 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:46.338 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:46.381 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:46.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:46.382 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:46.483 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:46.521 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:46.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:46.522 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:46.623 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:46.673 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:46.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:46.675 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:46.776 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:46.802 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:46.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:46.803 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:46.904 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:46.931 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:46.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:46.932 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:47.032 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:47.060 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:47.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:47.061 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:47.109 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 35 16 58449 58433 104.316 87.0625 0.0167503 0.0095809 2024-09-11T07:46:47.161 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:47.212 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:47.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:47.213 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:47.314 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:47.369 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:47.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:47.370 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:47.470 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:47.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:47.514 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:47.515 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:47.579 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:47 smithi022 ceph-mon[37330]: pgmap v946: 158 pgs: 158 active+clean; 3.4 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 41 KiB/s rd, 76 MiB/s wr, 1.28k op/s 2024-09-11T07:46:47.579 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:47 smithi022 ceph-mon[29462]: pgmap v946: 158 pgs: 158 active+clean; 3.4 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 41 KiB/s rd, 76 MiB/s wr, 1.28k op/s 2024-09-11T07:46:47.616 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:47.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:47.678 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:47.678 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:47.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:47 smithi104 ceph-mon[33507]: pgmap v946: 158 pgs: 158 active+clean; 3.4 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 41 KiB/s rd, 76 MiB/s wr, 1.28k op/s 2024-09-11T07:46:47.779 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:47.803 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:47.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:47.804 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:47.863 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:46:47.905 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:47.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:47.957 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:47.957 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:48.058 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:48.102 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:48.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:48.104 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:48.109 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 36 16 59613 59597 103.438 72.75 0.0191375 0.00966157 2024-09-11T07:46:48.204 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:48.244 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:46:48.250 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:48.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:48.253 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:48.257 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:46:48.354 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:48.379 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:48.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:48.380 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:48.481 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:48.487 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is set 2024-09-11T07:46:48.508 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:48.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:48.509 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:48.610 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:48.637 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:48.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:48.638 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:48.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:48 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/840033155' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:46:48.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:48 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/840033155' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:46:48.738 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:48.765 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:48.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:48.766 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:48.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:48 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/840033155' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:46:48.867 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:48.912 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:48.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:48.913 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:49.014 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:49.040 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":946,"stamp":"2024-09-11T07:46:46.401416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3621168107,"num_objects":55442,"num_object_clones":0,"num_object_copies":166326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55442,"num_whiteouts":0,"num_read":1781,"num_read_kb":1610,"num_write":56331,"num_write_kb":3537099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":1727622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57308,"ondisk_log_size":57308,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":474,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":16820372,"kb_used_data":10506012,"kb_used_omap":14,"kb_used_meta":6314289,"kb_avail":639395692,"statfs":{"total":671965249536,"available":654741188608,"internally_reserved":0,"allocated":10758156288,"data_stored":10754080671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14826,"internal_metadata":6465832470},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":845611008,"num_objects":12903,"num_object_clones":0,"num_object_copies":38709,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12903,"num_whiteouts":0,"num_read":439,"num_read_kb":439,"num_write":13189,"num_write_kb":825792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.645006"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":189,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.488162+0000","last_change":"2024-09-11T07:46:28.323440+0000","last_active":"2024-09-11T07:46:31.488162+0000","last_peered":"2024-09-11T07:46:31.488162+0000","last_clean":"2024-09-11T07:46:31.488162+0000","last_became_active":"2024-09-11T07:44:23.769058+0000","last_became_peered":"2024-09-11T07:44:23.769058+0000","last_unstale":"2024-09-11T07:46:31.488162+0000","last_undegraded":"2024-09-11T07:46:31.488162+0000","last_fullsized":"2024-09-11T07:46:31.488162+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:28.323351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:46:28.323351+0000","last_clean_scrub_stamp":"2024-09-11T07:46:28.323351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:48.922438+0000","scrub_duration":0.0094746069999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":484,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.489577+0000","last_change":"2024-09-11T07:45:27.108301+0000","last_active":"2024-09-11T07:46:31.489577+0000","last_peered":"2024-09-11T07:46:31.489577+0000","last_clean":"2024-09-11T07:46:31.489577+0000","last_became_active":"2024-09-11T07:45:27.107996+0000","last_became_peered":"2024-09-11T07:45:27.107996+0000","last_unstale":"2024-09-11T07:46:31.489577+0000","last_undegraded":"2024-09-11T07:46:31.489577+0000","last_fullsized":"2024-09-11T07:46:31.489577+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:21.375076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:21.375076+0000","last_clean_scrub_stamp":"2024-09-11T07:44:21.375076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0068686850000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"426'46","reported_seq":516,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.799291+0000","last_change":"2024-09-11T07:44:02.682805+0000","last_active":"2024-09-11T07:46:40.799291+0000","last_peered":"2024-09-11T07:46:40.799291+0000","last_clean":"2024-09-11T07:46:40.799291+0000","last_became_active":"2024-09-11T07:26:33.544627+0000","last_became_peered":"2024-09-11T07:26:33.544627+0000","last_unstale":"2024-09-11T07:46:40.799291+0000","last_undegraded":"2024-09-11T07:46:40.799291+0000","last_fullsized":"2024-09-11T07:46:40.799291+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'34","last_scrub_stamp":"2024-09-11T07:44:02.682692+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:44:02.682692+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056034049999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":51,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.493071+0000","last_change":"2024-09-11T07:46:04.878844+0000","last_active":"2024-09-11T07:46:31.493071+0000","last_peered":"2024-09-11T07:46:31.493071+0000","last_clean":"2024-09-11T07:46:31.493071+0000","last_became_active":"2024-09-11T07:45:27.095499+0000","last_became_peered":"2024-09-11T07:45:27.095499+0000","last_unstale":"2024-09-11T07:46:31.493071+0000","last_undegraded":"2024-09-11T07:46:31.493071+0000","last_fullsized":"2024-09-11T07:46:31.493071+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:04.878787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:46:04.878787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:10.897341+0000","scrub_duration":0.0013075770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":473,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.484944+0000","last_change":"2024-09-11T07:44:19.655482+0000","last_active":"2024-09-11T07:46:31.484944+0000","last_peered":"2024-09-11T07:46:31.484944+0000","last_clean":"2024-09-11T07:46:31.484944+0000","last_became_active":"2024-09-11T07:39:57.767806+0000","last_became_peered":"2024-09-11T07:39:57.767806+0000","last_unstale":"2024-09-11T07:46:31.484944+0000","last_undegraded":"2024-09-11T07:46:31.484944+0000","last_fullsized":"2024-09-11T07:46:31.484944+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:19.655401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:19.655401+0000","last_clean_scrub_stamp":"2024-09-11T07:44:19.655401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025632099999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"426'25","reported_seq":60,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:39.417397+0000","last_change":"2024-09-11T07:45:27.144397+0000","last_active":"2024-09-11T07:46:39.417397+0000","last_peered":"2024-09-11T07:46:39.417397+0000","last_clean":"2024-09-11T07:46:39.417397+0000","last_became_active":"2024-09-11T07:45:27.095974+0000","last_became_peered":"2024-09-11T07:45:27.095974+0000","last_unstale":"2024-09-11T07:46:39.417397+0000","last_undegraded":"2024-09-11T07:46:39.417397+0000","last_fullsized":"2024-09-11T07:46:39.417397+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:43:59.245081+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:36:42.708941+0000","last_clean_scrub_stamp":"2024-09-11T07:43:59.245081+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040986499999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":193,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.508498+0000","last_change":"2024-09-11T07:45:12.417072+0000","last_active":"2024-09-11T07:46:31.508498+0000","last_peered":"2024-09-11T07:46:31.508498+0000","last_clean":"2024-09-11T07:46:31.508498+0000","last_became_active":"2024-09-11T07:44:23.769502+0000","last_became_peered":"2024-09-11T07:44:23.769502+0000","last_unstale":"2024-09-11T07:46:31.508498+0000","last_undegraded":"2024-09-11T07:46:31.508498+0000","last_fullsized":"2024-09-11T07:46:31.508498+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:12.416990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:45:12.416990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001315529,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":324,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.505516+0000","last_change":"2024-09-11T07:46:31.505181+0000","last_active":"2024-09-11T07:46:31.505516+0000","last_peered":"2024-09-11T07:46:31.505516+0000","last_clean":"2024-09-11T07:46:31.505516+0000","last_became_active":"2024-09-11T07:46:31.504850+0000","last_became_peered":"2024-09-11T07:46:31.504850+0000","last_unstale":"2024-09-11T07:46:31.505516+0000","last_undegraded":"2024-09-11T07:46:31.505516+0000","last_fullsized":"2024-09-11T07:46:31.505516+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:05.327186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:46:05.327186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:14.919617+0000","scrub_duration":0.0058146650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":462,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.508910+0000","last_change":"2024-09-11T07:46:21.898482+0000","last_active":"2024-09-11T07:46:31.508910+0000","last_peered":"2024-09-11T07:46:31.508910+0000","last_clean":"2024-09-11T07:46:31.508910+0000","last_became_active":"2024-09-11T07:32:00.516662+0000","last_became_peered":"2024-09-11T07:32:00.516662+0000","last_unstale":"2024-09-11T07:46:31.508910+0000","last_undegraded":"2024-09-11T07:46:31.508910+0000","last_fullsized":"2024-09-11T07:46:31.508910+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:21.898400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:46:21.898400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:40.556920+0000","scrub_duration":0.0063092759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"426'11","reported_seq":60,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:39.967942+0000","last_change":"2024-09-11T07:45:32.375248+0000","last_active":"2024-09-11T07:46:39.967942+0000","last_peered":"2024-09-11T07:46:39.967942+0000","last_clean":"2024-09-11T07:46:39.967942+0000","last_became_active":"2024-09-11T07:45:27.096974+0000","last_became_peered":"2024-09-11T07:45:27.096974+0000","last_unstale":"2024-09-11T07:46:39.967942+0000","last_undegraded":"2024-09-11T07:46:39.967942+0000","last_fullsized":"2024-09-11T07:46:39.967942+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'9","last_scrub_stamp":"2024-09-11T07:45:32.375156+0000","last_deep_scrub":"234'9","last_deep_scrub_stamp":"2024-09-11T07:45:32.375156+0000","last_clean_scrub_stamp":"2024-09-11T07:45:32.375156+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:52.079908+0000","scrub_duration":0.011478199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":13,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":441,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.508137+0000","last_change":"2024-09-11T07:45:27.113031+0000","last_active":"2024-09-11T07:46:31.508137+0000","last_peered":"2024-09-11T07:46:31.508137+0000","last_clean":"2024-09-11T07:46:31.508137+0000","last_became_active":"2024-09-11T07:45:27.112681+0000","last_became_peered":"2024-09-11T07:45:27.112681+0000","last_unstale":"2024-09-11T07:46:31.508137+0000","last_undegraded":"2024-09-11T07:46:31.508137+0000","last_fullsized":"2024-09-11T07:46:31.508137+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:32.398398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.973323+0000","last_clean_scrub_stamp":"2024-09-11T07:44:32.398398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002203782,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":81,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.488671+0000","last_change":"2024-09-11T07:44:23.769584+0000","last_active":"2024-09-11T07:46:31.488671+0000","last_peered":"2024-09-11T07:46:31.488671+0000","last_clean":"2024-09-11T07:46:31.488671+0000","last_became_active":"2024-09-11T07:44:23.769187+0000","last_became_peered":"2024-09-11T07:44:23.769187+0000","last_unstale":"2024-09-11T07:46:31.488671+0000","last_undegraded":"2024-09-11T07:46:31.488671+0000","last_fullsized":"2024-09-11T07:46:31.488671+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:04.074067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:45.149916+0000","last_clean_scrub_stamp":"2024-09-11T07:44:04.074067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":109,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.496161+0000","last_change":"2024-09-11T07:45:49.765879+0000","last_active":"2024-09-11T07:46:31.496161+0000","last_peered":"2024-09-11T07:46:31.496161+0000","last_clean":"2024-09-11T07:46:31.496161+0000","last_became_active":"2024-09-11T07:45:49.765541+0000","last_became_peered":"2024-09-11T07:45:49.765541+0000","last_unstale":"2024-09-11T07:46:31.496161+0000","last_undegraded":"2024-09-11T07:46:31.496161+0000","last_fullsized":"2024-09-11T07:46:31.496161+0000","mapping_epoch":407,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":408,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:56.219368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:52.057959+0000","last_clean_scrub_stamp":"2024-09-11T07:43:56.219368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032973189999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"426'25","reported_seq":333,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.137491+0000","last_change":"2024-09-11T07:46:19.457329+0000","last_active":"2024-09-11T07:46:40.137491+0000","last_peered":"2024-09-11T07:46:40.137491+0000","last_clean":"2024-09-11T07:46:40.137491+0000","last_became_active":"2024-09-11T07:45:27.112765+0000","last_became_peered":"2024-09-11T07:45:27.112765+0000","last_unstale":"2024-09-11T07:46:40.137491+0000","last_undegraded":"2024-09-11T07:46:40.137491+0000","last_fullsized":"2024-09-11T07:46:40.137491+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:46:19.457213+0000","last_deep_scrub":"136'13","last_deep_scrub_stamp":"2024-09-11T07:31:08.042198+0000","last_clean_scrub_stamp":"2024-09-11T07:46:19.457213+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:40.530051+0000","scrub_duration":0.0046268419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":479,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.491072+0000","last_change":"2024-09-11T07:45:27.115012+0000","last_active":"2024-09-11T07:46:31.491072+0000","last_peered":"2024-09-11T07:46:31.491072+0000","last_clean":"2024-09-11T07:46:31.491072+0000","last_became_active":"2024-09-11T07:45:27.114162+0000","last_became_peered":"2024-09-11T07:45:27.114162+0000","last_unstale":"2024-09-11T07:46:31.491072+0000","last_undegraded":"2024-09-11T07:46:31.491072+0000","last_fullsized":"2024-09-11T07:46:31.491072+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:24.340025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:44:24.340025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002461366,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":478,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.487650+0000","last_change":"2024-09-11T07:46:24.272062+0000","last_active":"2024-09-11T07:46:31.487650+0000","last_peered":"2024-09-11T07:46:31.487650+0000","last_clean":"2024-09-11T07:46:31.487650+0000","last_became_active":"2024-09-11T07:32:00.530335+0000","last_became_peered":"2024-09-11T07:32:00.530335+0000","last_unstale":"2024-09-11T07:46:31.487650+0000","last_undegraded":"2024-09-11T07:46:31.487650+0000","last_fullsized":"2024-09-11T07:46:31.487650+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:24.271967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:46:24.271967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:38.913016+0000","scrub_duration":0.0042967989999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"426'3448","reported_seq":3470,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:44.201514+0000","last_change":"2024-09-11T07:45:56.765116+0000","last_active":"2024-09-11T07:46:44.201514+0000","last_peered":"2024-09-11T07:46:44.201514+0000","last_clean":"2024-09-11T07:46:44.201514+0000","last_became_active":"2024-09-11T07:45:56.763997+0000","last_became_peered":"2024-09-11T07:45:56.763997+0000","last_unstale":"2024-09-11T07:46:44.201514+0000","last_undegraded":"2024-09-11T07:46:44.201514+0000","last_fullsized":"2024-09-11T07:46:44.201514+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3448,"log_dups_size":0,"ondisk_log_size":3448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:08.078880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225968128,"num_objects":3448,"num_object_clones":0,"num_object_copies":10344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3448,"num_write_kb":220672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"87'81","reported_seq":50,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.490660+0000","last_change":"2024-09-11T07:45:01.833611+0000","last_active":"2024-09-11T07:46:31.490660+0000","last_peered":"2024-09-11T07:46:31.490660+0000","last_clean":"2024-09-11T07:46:31.490660+0000","last_became_active":"2024-09-11T07:45:01.808023+0000","last_became_peered":"2024-09-11T07:45:01.808023+0000","last_unstale":"2024-09-11T07:46:31.490660+0000","last_undegraded":"2024-09-11T07:46:31.490660+0000","last_fullsized":"2024-09-11T07:46:31.490660+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:44:22.387527+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.387527+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015232676000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":370027,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"426'45","reported_seq":419,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.727419+0000","last_change":"2024-09-11T07:45:27.125496+0000","last_active":"2024-09-11T07:46:40.727419+0000","last_peered":"2024-09-11T07:46:40.727419+0000","last_clean":"2024-09-11T07:46:40.727419+0000","last_became_active":"2024-09-11T07:45:27.094411+0000","last_became_peered":"2024-09-11T07:45:27.094411+0000","last_unstale":"2024-09-11T07:46:40.727419+0000","last_undegraded":"2024-09-11T07:46:40.727419+0000","last_fullsized":"2024-09-11T07:46:40.727419+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'31","last_scrub_stamp":"2024-09-11T07:44:06.132709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:44:06.132709+0000","objects_scrubbed":8,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034170279999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":60,"num_write":46,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":470,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.508272+0000","last_change":"2024-09-11T07:45:27.100579+0000","last_active":"2024-09-11T07:46:31.508272+0000","last_peered":"2024-09-11T07:46:31.508272+0000","last_clean":"2024-09-11T07:46:31.508272+0000","last_became_active":"2024-09-11T07:45:27.100238+0000","last_became_peered":"2024-09-11T07:45:27.100238+0000","last_unstale":"2024-09-11T07:46:31.508272+0000","last_undegraded":"2024-09-11T07:46:31.508272+0000","last_fullsized":"2024-09-11T07:46:31.508272+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:08.501392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:08.501392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004339763,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":59,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.487135+0000","last_change":"2024-09-11T07:46:09.235485+0000","last_active":"2024-09-11T07:46:31.487135+0000","last_peered":"2024-09-11T07:46:31.487135+0000","last_clean":"2024-09-11T07:46:31.487135+0000","last_became_active":"2024-09-11T07:45:27.117371+0000","last_became_peered":"2024-09-11T07:45:27.117371+0000","last_unstale":"2024-09-11T07:46:31.487135+0000","last_undegraded":"2024-09-11T07:46:31.487135+0000","last_fullsized":"2024-09-11T07:46:31.487135+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:09.235371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:46:09.235371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:19.605064+0000","scrub_duration":0.0016833869999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":474,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.492023+0000","last_change":"2024-09-11T07:45:27.104861+0000","last_active":"2024-09-11T07:46:31.492023+0000","last_peered":"2024-09-11T07:46:31.492023+0000","last_clean":"2024-09-11T07:46:31.492023+0000","last_became_active":"2024-09-11T07:45:27.104519+0000","last_became_peered":"2024-09-11T07:45:27.104519+0000","last_unstale":"2024-09-11T07:46:31.492023+0000","last_undegraded":"2024-09-11T07:46:31.492023+0000","last_fullsized":"2024-09-11T07:46:31.492023+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:57.645536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:32.281886+0000","last_clean_scrub_stamp":"2024-09-11T07:43:57.645536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018691090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"426'3437","reported_seq":3468,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:44.198946+0000","last_change":"2024-09-11T07:46:31.506264+0000","last_active":"2024-09-11T07:46:44.198946+0000","last_peered":"2024-09-11T07:46:44.198946+0000","last_clean":"2024-09-11T07:46:44.198946+0000","last_became_active":"2024-09-11T07:46:31.502938+0000","last_became_peered":"2024-09-11T07:46:31.502938+0000","last_unstale":"2024-09-11T07:46:44.198946+0000","last_undegraded":"2024-09-11T07:46:44.198946+0000","last_fullsized":"2024-09-11T07:46:44.198946+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3437,"log_dups_size":0,"ondisk_log_size":3437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:06.760564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225247232,"num_objects":3437,"num_object_clones":0,"num_object_copies":10311,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3437,"num_write_kb":219968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"87'81","reported_seq":45,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.508067+0000","last_change":"2024-09-11T07:45:01.812611+0000","last_active":"2024-09-11T07:46:31.508067+0000","last_peered":"2024-09-11T07:46:31.508067+0000","last_clean":"2024-09-11T07:46:31.508067+0000","last_became_active":"2024-09-11T07:45:01.808923+0000","last_became_peered":"2024-09-11T07:45:01.808923+0000","last_unstale":"2024-09-11T07:46:31.508067+0000","last_undegraded":"2024-09-11T07:46:31.508067+0000","last_fullsized":"2024-09-11T07:46:31.508067+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:44:22.387527+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.387527+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015232676000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"426'35","reported_seq":337,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.452253+0000","last_change":"2024-09-11T07:45:33.404225+0000","last_active":"2024-09-11T07:46:40.452253+0000","last_peered":"2024-09-11T07:46:40.452253+0000","last_clean":"2024-09-11T07:46:40.452253+0000","last_became_active":"2024-09-11T07:45:27.097522+0000","last_became_peered":"2024-09-11T07:45:27.097522+0000","last_unstale":"2024-09-11T07:46:40.452253+0000","last_undegraded":"2024-09-11T07:46:40.452253+0000","last_fullsized":"2024-09-11T07:46:40.452253+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'25","last_scrub_stamp":"2024-09-11T07:45:33.403950+0000","last_deep_scrub":"112'15","last_deep_scrub_stamp":"2024-09-11T07:31:10.005896+0000","last_clean_scrub_stamp":"2024-09-11T07:45:33.403950+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:59.506481+0000","scrub_duration":0.0044240010000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":197,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.486223+0000","last_change":"2024-09-11T07:46:23.435408+0000","last_active":"2024-09-11T07:46:31.486223+0000","last_peered":"2024-09-11T07:46:31.486223+0000","last_clean":"2024-09-11T07:46:31.486223+0000","last_became_active":"2024-09-11T07:44:23.768172+0000","last_became_peered":"2024-09-11T07:44:23.768172+0000","last_unstale":"2024-09-11T07:46:31.486223+0000","last_undegraded":"2024-09-11T07:46:31.486223+0000","last_fullsized":"2024-09-11T07:46:31.486223+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:23.435037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:46:23.435037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:36.807613+0000","scrub_duration":0.0049327829999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"299'11","reported_seq":739,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:35.762389+0000","last_change":"2024-09-11T07:43:21.381901+0000","last_active":"2024-09-11T07:46:35.762389+0000","last_peered":"2024-09-11T07:46:35.762389+0000","last_clean":"2024-09-11T07:46:35.762389+0000","last_became_active":"2024-09-11T07:40:11.697568+0000","last_became_peered":"2024-09-11T07:40:11.697568+0000","last_unstale":"2024-09-11T07:46:35.762389+0000","last_undegraded":"2024-09-11T07:46:35.762389+0000","last_fullsized":"2024-09-11T07:46:35.762389+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"299'11","last_scrub_stamp":"2024-09-11T07:43:21.381642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:43:21.381642+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023952790000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":349,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.491234+0000","last_change":"2024-09-11T07:46:14.955001+0000","last_active":"2024-09-11T07:46:31.491234+0000","last_peered":"2024-09-11T07:46:31.491234+0000","last_clean":"2024-09-11T07:46:31.491234+0000","last_became_active":"2024-09-11T07:45:27.099434+0000","last_became_peered":"2024-09-11T07:45:27.099434+0000","last_unstale":"2024-09-11T07:46:31.491234+0000","last_undegraded":"2024-09-11T07:46:31.491234+0000","last_fullsized":"2024-09-11T07:46:31.491234+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:14.954917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:46:14.954917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:26.237334+0000","scrub_duration":0.0052986090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.0","version":"426'3533","reported_seq":3555,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:45.181827+0000","last_change":"2024-09-11T07:45:56.758475+0000","last_active":"2024-09-11T07:46:45.181827+0000","last_peered":"2024-09-11T07:46:45.181827+0000","last_clean":"2024-09-11T07:46:45.181827+0000","last_became_active":"2024-09-11T07:45:56.758196+0000","last_became_peered":"2024-09-11T07:45:56.758196+0000","last_unstale":"2024-09-11T07:46:45.181827+0000","last_undegraded":"2024-09-11T07:46:45.181827+0000","last_fullsized":"2024-09-11T07:46:45.181827+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3533,"log_dups_size":0,"ondisk_log_size":3533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:20.308690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":231538688,"num_objects":3533,"num_object_clones":0,"num_object_copies":10599,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3533,"num_write_kb":226112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"87'81","reported_seq":45,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.489485+0000","last_change":"2024-09-11T07:45:01.807167+0000","last_active":"2024-09-11T07:46:31.489485+0000","last_peered":"2024-09-11T07:46:31.489485+0000","last_clean":"2024-09-11T07:46:31.489485+0000","last_became_active":"2024-09-11T07:45:01.805300+0000","last_became_peered":"2024-09-11T07:45:01.805300+0000","last_unstale":"2024-09-11T07:46:31.489485+0000","last_undegraded":"2024-09-11T07:46:31.489485+0000","last_fullsized":"2024-09-11T07:46:31.489485+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:44:22.387527+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.387527+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015232676000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53669,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"426'46","reported_seq":373,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.846915+0000","last_change":"2024-09-11T07:45:27.129184+0000","last_active":"2024-09-11T07:46:40.846915+0000","last_peered":"2024-09-11T07:46:40.846915+0000","last_clean":"2024-09-11T07:46:40.846915+0000","last_became_active":"2024-09-11T07:45:27.095634+0000","last_became_peered":"2024-09-11T07:45:27.095634+0000","last_unstale":"2024-09-11T07:46:40.846915+0000","last_undegraded":"2024-09-11T07:46:40.846915+0000","last_fullsized":"2024-09-11T07:46:40.846915+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'34","last_scrub_stamp":"2024-09-11T07:43:57.973282+0000","last_deep_scrub":"83'22","last_deep_scrub_stamp":"2024-09-11T07:34:04.375457+0000","last_clean_scrub_stamp":"2024-09-11T07:43:57.973282+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086024310000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":207,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.508139+0000","last_change":"2024-09-11T07:44:27.416260+0000","last_active":"2024-09-11T07:46:31.508139+0000","last_peered":"2024-09-11T07:46:31.508139+0000","last_clean":"2024-09-11T07:46:31.508139+0000","last_became_active":"2024-09-11T07:44:23.770343+0000","last_became_peered":"2024-09-11T07:44:23.770343+0000","last_unstale":"2024-09-11T07:46:31.508139+0000","last_undegraded":"2024-09-11T07:46:31.508139+0000","last_fullsized":"2024-09-11T07:46:31.508139+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:27.416207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:27.416207+0000","last_clean_scrub_stamp":"2024-09-11T07:44:27.416207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001230096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"395'10","reported_seq":665,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:35.775490+0000","last_change":"2024-09-11T07:46:10.279342+0000","last_active":"2024-09-11T07:46:35.775490+0000","last_peered":"2024-09-11T07:46:35.775490+0000","last_clean":"2024-09-11T07:46:35.775490+0000","last_became_active":"2024-09-11T07:45:27.122326+0000","last_became_peered":"2024-09-11T07:45:27.122326+0000","last_unstale":"2024-09-11T07:46:35.775490+0000","last_undegraded":"2024-09-11T07:46:35.775490+0000","last_fullsized":"2024-09-11T07:46:35.775490+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"395'10","last_scrub_stamp":"2024-09-11T07:46:10.279284+0000","last_deep_scrub":"366'9","last_deep_scrub_stamp":"2024-09-11T07:43:26.334501+0000","last_clean_scrub_stamp":"2024-09-11T07:46:10.279284+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:25.963185+0000","scrub_duration":0.001636265,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":316,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.488993+0000","last_change":"2024-09-11T07:45:49.765810+0000","last_active":"2024-09-11T07:46:31.488993+0000","last_peered":"2024-09-11T07:46:31.488993+0000","last_clean":"2024-09-11T07:46:31.488993+0000","last_became_active":"2024-09-11T07:45:49.765504+0000","last_became_peered":"2024-09-11T07:45:49.765504+0000","last_unstale":"2024-09-11T07:46:31.488993+0000","last_undegraded":"2024-09-11T07:46:31.488993+0000","last_fullsized":"2024-09-11T07:46:31.488993+0000","mapping_epoch":407,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":408,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:19.336454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:31.200685+0000","last_clean_scrub_stamp":"2024-09-11T07:45:19.336454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001371688,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"426'3376","reported_seq":3398,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:44.349846+0000","last_change":"2024-09-11T07:45:57.199288+0000","last_active":"2024-09-11T07:46:44.349846+0000","last_peered":"2024-09-11T07:46:44.349846+0000","last_clean":"2024-09-11T07:46:44.349846+0000","last_became_active":"2024-09-11T07:45:57.198828+0000","last_became_peered":"2024-09-11T07:45:57.198828+0000","last_unstale":"2024-09-11T07:46:44.349846+0000","last_undegraded":"2024-09-11T07:46:44.349846+0000","last_fullsized":"2024-09-11T07:46:44.349846+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3376,"log_dups_size":0,"ondisk_log_size":3376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:59.923306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":221249536,"num_objects":3376,"num_object_clones":0,"num_object_copies":10128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3376,"num_write_kb":216064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"426'32","reported_seq":413,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.742284+0000","last_change":"2024-09-11T07:46:24.388932+0000","last_active":"2024-09-11T07:46:40.742284+0000","last_peered":"2024-09-11T07:46:40.742284+0000","last_clean":"2024-09-11T07:46:40.742284+0000","last_became_active":"2024-09-11T07:45:27.102164+0000","last_became_peered":"2024-09-11T07:45:27.102164+0000","last_unstale":"2024-09-11T07:46:40.742284+0000","last_undegraded":"2024-09-11T07:46:40.742284+0000","last_fullsized":"2024-09-11T07:46:40.742284+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'24","last_scrub_stamp":"2024-09-11T07:46:24.388838+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-09-11T07:29:38.112111+0000","last_clean_scrub_stamp":"2024-09-11T07:46:24.388838+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:39.786424+0000","scrub_duration":0.0055826570000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":454,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.507798+0000","last_change":"2024-09-11T07:46:12.925725+0000","last_active":"2024-09-11T07:46:31.507798+0000","last_peered":"2024-09-11T07:46:31.507798+0000","last_clean":"2024-09-11T07:46:31.507798+0000","last_became_active":"2024-09-11T07:32:00.518538+0000","last_became_peered":"2024-09-11T07:32:00.518538+0000","last_unstale":"2024-09-11T07:46:31.507798+0000","last_undegraded":"2024-09-11T07:46:31.507798+0000","last_fullsized":"2024-09-11T07:46:31.507798+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:12.925664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:43:24.643676+0000","last_clean_scrub_stamp":"2024-09-11T07:46:12.925664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:24.378872+0000","scrub_duration":0.0026407430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":51,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.489188+0000","last_change":"2024-09-11T07:45:30.682670+0000","last_active":"2024-09-11T07:46:31.489188+0000","last_peered":"2024-09-11T07:46:31.489188+0000","last_clean":"2024-09-11T07:46:31.489188+0000","last_became_active":"2024-09-11T07:45:27.094025+0000","last_became_peered":"2024-09-11T07:45:27.094025+0000","last_unstale":"2024-09-11T07:46:31.489188+0000","last_undegraded":"2024-09-11T07:46:31.489188+0000","last_fullsized":"2024-09-11T07:46:31.489188+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:30.682614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:19.714568+0000","last_clean_scrub_stamp":"2024-09-11T07:45:30.682614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:36.638911+0000","scrub_duration":0.0015960729999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":451,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.484021+0000","last_change":"2024-09-11T07:45:17.470823+0000","last_active":"2024-09-11T07:46:31.484021+0000","last_peered":"2024-09-11T07:46:31.484021+0000","last_clean":"2024-09-11T07:46:31.484021+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:46:31.484021+0000","last_undegraded":"2024-09-11T07:46:31.484021+0000","last_fullsized":"2024-09-11T07:46:31.484021+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:17.470730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:46.853387+0000","last_clean_scrub_stamp":"2024-09-11T07:45:17.470730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002761412,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.e","version":"426'3511","reported_seq":3533,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:44.522508+0000","last_change":"2024-09-11T07:45:56.762206+0000","last_active":"2024-09-11T07:46:44.522508+0000","last_peered":"2024-09-11T07:46:44.522508+0000","last_clean":"2024-09-11T07:46:44.522508+0000","last_became_active":"2024-09-11T07:45:56.761971+0000","last_became_peered":"2024-09-11T07:45:56.761971+0000","last_unstale":"2024-09-11T07:46:44.522508+0000","last_undegraded":"2024-09-11T07:46:44.522508+0000","last_fullsized":"2024-09-11T07:46:44.522508+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3511,"log_dups_size":0,"ondisk_log_size":3511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:02.947203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230096896,"num_objects":3511,"num_object_clones":0,"num_object_copies":10533,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3511,"num_write_kb":224704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"87'81","reported_seq":64,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.483715+0000","last_change":"2024-09-11T07:45:27.141030+0000","last_active":"2024-09-11T07:46:31.483715+0000","last_peered":"2024-09-11T07:46:31.483715+0000","last_clean":"2024-09-11T07:46:31.483715+0000","last_became_active":"2024-09-11T07:45:27.097022+0000","last_became_peered":"2024-09-11T07:45:27.097022+0000","last_unstale":"2024-09-11T07:46:31.483715+0000","last_undegraded":"2024-09-11T07:46:31.483715+0000","last_fullsized":"2024-09-11T07:46:31.483715+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:44:22.387527+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.387527+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015232676000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":762700,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"426'42","reported_seq":302,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.757135+0000","last_change":"2024-09-11T07:46:31.496469+0000","last_active":"2024-09-11T07:46:40.757135+0000","last_peered":"2024-09-11T07:46:40.757135+0000","last_clean":"2024-09-11T07:46:40.757135+0000","last_became_active":"2024-09-11T07:46:31.496001+0000","last_became_peered":"2024-09-11T07:46:31.496001+0000","last_unstale":"2024-09-11T07:46:40.757135+0000","last_undegraded":"2024-09-11T07:46:40.757135+0000","last_fullsized":"2024-09-11T07:46:40.757135+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'28","last_scrub_stamp":"2024-09-11T07:46:08.280072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:46:08.280072+0000","objects_scrubbed":7,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:22.368030+0000","scrub_duration":0.012975469999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":55,"num_write":42,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":223,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.496018+0000","last_change":"2024-09-11T07:43:24.326702+0000","last_active":"2024-09-11T07:46:31.496018+0000","last_peered":"2024-09-11T07:46:31.496018+0000","last_clean":"2024-09-11T07:46:31.496018+0000","last_became_active":"2024-09-11T07:39:57.767741+0000","last_became_peered":"2024-09-11T07:39:57.767741+0000","last_unstale":"2024-09-11T07:46:31.496018+0000","last_undegraded":"2024-09-11T07:46:31.496018+0000","last_fullsized":"2024-09-11T07:46:31.496018+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:24.326341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:43:24.326341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027901060000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3],"acting":[2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"395'10","reported_seq":710,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:39.752256+0000","last_change":"2024-09-11T07:45:27.151129+0000","last_active":"2024-09-11T07:46:39.752256+0000","last_peered":"2024-09-11T07:46:39.752256+0000","last_clean":"2024-09-11T07:46:39.752256+0000","last_became_active":"2024-09-11T07:45:27.098861+0000","last_became_peered":"2024-09-11T07:45:27.098861+0000","last_unstale":"2024-09-11T07:46:39.752256+0000","last_undegraded":"2024-09-11T07:46:39.752256+0000","last_fullsized":"2024-09-11T07:46:39.752256+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'9","last_scrub_stamp":"2024-09-11T07:43:57.666646+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-11T07:35:39.832547+0000","last_clean_scrub_stamp":"2024-09-11T07:43:57.666646+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034760120000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":425,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.508199+0000","last_change":"2024-09-11T07:44:17.483184+0000","last_active":"2024-09-11T07:46:31.508199+0000","last_peered":"2024-09-11T07:46:31.508199+0000","last_clean":"2024-09-11T07:46:31.508199+0000","last_became_active":"2024-09-11T07:32:00.511005+0000","last_became_peered":"2024-09-11T07:32:00.511005+0000","last_unstale":"2024-09-11T07:46:31.508199+0000","last_undegraded":"2024-09-11T07:46:31.508199+0000","last_fullsized":"2024-09-11T07:46:31.508199+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:17.483090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:44:17.483090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057481720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"426'3394","reported_seq":3415,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:44.512076+0000","last_change":"2024-09-11T07:45:56.764982+0000","last_active":"2024-09-11T07:46:44.512076+0000","last_peered":"2024-09-11T07:46:44.512076+0000","last_clean":"2024-09-11T07:46:44.512076+0000","last_became_active":"2024-09-11T07:45:56.764754+0000","last_became_peered":"2024-09-11T07:45:56.764754+0000","last_unstale":"2024-09-11T07:46:44.512076+0000","last_undegraded":"2024-09-11T07:46:44.512076+0000","last_fullsized":"2024-09-11T07:46:44.512076+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3394,"log_dups_size":0,"ondisk_log_size":3394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:25.447509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222429184,"num_objects":3394,"num_object_clones":0,"num_object_copies":10182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3394,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3394,"num_write_kb":217216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"87'81","reported_seq":45,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.494778+0000","last_change":"2024-09-11T07:45:27.100289+0000","last_active":"2024-09-11T07:46:31.494778+0000","last_peered":"2024-09-11T07:46:31.494778+0000","last_clean":"2024-09-11T07:46:31.494778+0000","last_became_active":"2024-09-11T07:45:27.099677+0000","last_became_peered":"2024-09-11T07:45:27.099677+0000","last_unstale":"2024-09-11T07:46:31.494778+0000","last_undegraded":"2024-09-11T07:46:31.494778+0000","last_fullsized":"2024-09-11T07:46:31.494778+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:44:22.387527+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.387527+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015232676000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"426'50","reported_seq":549,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.570494+0000","last_change":"2024-09-11T07:46:26.243185+0000","last_active":"2024-09-11T07:46:40.570494+0000","last_peered":"2024-09-11T07:46:40.570494+0000","last_clean":"2024-09-11T07:46:40.570494+0000","last_became_active":"2024-09-11T07:46:11.762572+0000","last_became_peered":"2024-09-11T07:46:11.762572+0000","last_unstale":"2024-09-11T07:46:40.570494+0000","last_undegraded":"2024-09-11T07:46:40.570494+0000","last_fullsized":"2024-09-11T07:46:40.570494+0000","mapping_epoch":418,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":419,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'38","last_scrub_stamp":"2024-09-11T07:46:26.243126+0000","last_deep_scrub":"234'38","last_deep_scrub_stamp":"2024-09-11T07:46:26.243126+0000","last_clean_scrub_stamp":"2024-09-11T07:46:26.243126+0000","objects_scrubbed":8,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:34.336031+0000","scrub_duration":0.0022410799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":69,"num_read_kb":57,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,5],"acting":[0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":48,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.484476+0000","last_change":"2024-09-11T07:45:27.113078+0000","last_active":"2024-09-11T07:46:31.484476+0000","last_peered":"2024-09-11T07:46:31.484476+0000","last_clean":"2024-09-11T07:46:31.484476+0000","last_became_active":"2024-09-11T07:45:27.112250+0000","last_became_peered":"2024-09-11T07:45:27.112250+0000","last_unstale":"2024-09-11T07:46:31.484476+0000","last_undegraded":"2024-09-11T07:46:31.484476+0000","last_fullsized":"2024-09-11T07:46:31.484476+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:16.022013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:16.022013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":361,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.483846+0000","last_change":"2024-09-11T07:45:31.356987+0000","last_active":"2024-09-11T07:46:31.483846+0000","last_peered":"2024-09-11T07:46:31.483846+0000","last_clean":"2024-09-11T07:46:31.483846+0000","last_became_active":"2024-09-11T07:45:27.094619+0000","last_became_peered":"2024-09-11T07:45:27.094619+0000","last_unstale":"2024-09-11T07:46:31.483846+0000","last_undegraded":"2024-09-11T07:46:31.483846+0000","last_fullsized":"2024-09-11T07:46:31.483846+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:31.356921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:27.968579+0000","last_clean_scrub_stamp":"2024-09-11T07:45:31.356921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:44.701341+0000","scrub_duration":0.001320955,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":349,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.490317+0000","last_change":"2024-09-11T07:46:02.831398+0000","last_active":"2024-09-11T07:46:31.490317+0000","last_peered":"2024-09-11T07:46:31.490317+0000","last_clean":"2024-09-11T07:46:31.490317+0000","last_became_active":"2024-09-11T07:45:27.098502+0000","last_became_peered":"2024-09-11T07:45:27.098502+0000","last_unstale":"2024-09-11T07:46:31.490317+0000","last_undegraded":"2024-09-11T07:46:31.490317+0000","last_fullsized":"2024-09-11T07:46:31.490317+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:02.831341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:29.978798+0000","last_clean_scrub_stamp":"2024-09-11T07:46:02.831341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:09.740392+0000","scrub_duration":0.0012920449999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.c","version":"426'3445","reported_seq":3467,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:44.332100+0000","last_change":"2024-09-11T07:45:56.764201+0000","last_active":"2024-09-11T07:46:44.332100+0000","last_peered":"2024-09-11T07:46:44.332100+0000","last_clean":"2024-09-11T07:46:44.332100+0000","last_became_active":"2024-09-11T07:45:56.763888+0000","last_became_peered":"2024-09-11T07:45:56.763888+0000","last_unstale":"2024-09-11T07:46:44.332100+0000","last_undegraded":"2024-09-11T07:46:44.332100+0000","last_fullsized":"2024-09-11T07:46:44.332100+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3445,"log_dups_size":0,"ondisk_log_size":3445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:01.979288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225771520,"num_objects":3445,"num_object_clones":0,"num_object_copies":10335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3445,"num_write_kb":220480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,7],"acting":[2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"87'81","reported_seq":324,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.490007+0000","last_change":"2024-09-11T07:45:27.114430+0000","last_active":"2024-09-11T07:46:31.490007+0000","last_peered":"2024-09-11T07:46:31.490007+0000","last_clean":"2024-09-11T07:46:31.490007+0000","last_became_active":"2024-09-11T07:45:27.111145+0000","last_became_peered":"2024-09-11T07:45:27.111145+0000","last_unstale":"2024-09-11T07:46:31.490007+0000","last_undegraded":"2024-09-11T07:46:31.490007+0000","last_fullsized":"2024-09-11T07:46:31.490007+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:44:22.387527+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.387527+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015232676000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"426'42","reported_seq":541,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.638242+0000","last_change":"2024-09-11T07:45:13.572062+0000","last_active":"2024-09-11T07:46:40.638242+0000","last_peered":"2024-09-11T07:46:40.638242+0000","last_clean":"2024-09-11T07:46:40.638242+0000","last_became_active":"2024-09-11T07:32:00.506270+0000","last_became_peered":"2024-09-11T07:32:00.506270+0000","last_unstale":"2024-09-11T07:46:40.638242+0000","last_undegraded":"2024-09-11T07:46:40.638242+0000","last_fullsized":"2024-09-11T07:46:40.638242+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'32","last_scrub_stamp":"2024-09-11T07:45:13.571949+0000","last_deep_scrub":"236'32","last_deep_scrub_stamp":"2024-09-11T07:36:50.023909+0000","last_clean_scrub_stamp":"2024-09-11T07:45:13.571949+0000","objects_scrubbed":8,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020494009000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":69,"num_read_kb":55,"num_write":42,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":435,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.508831+0000","last_change":"2024-09-11T07:45:27.114174+0000","last_active":"2024-09-11T07:46:31.508831+0000","last_peered":"2024-09-11T07:46:31.508831+0000","last_clean":"2024-09-11T07:46:31.508831+0000","last_became_active":"2024-09-11T07:45:27.113680+0000","last_became_peered":"2024-09-11T07:45:27.113680+0000","last_unstale":"2024-09-11T07:46:31.508831+0000","last_undegraded":"2024-09-11T07:46:31.508831+0000","last_fullsized":"2024-09-11T07:46:31.508831+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:24.470602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:24.470602+0000","last_clean_scrub_stamp":"2024-09-11T07:44:24.470602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.040471685,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":254,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.487922+0000","last_change":"2024-09-11T07:44:12.312130+0000","last_active":"2024-09-11T07:46:31.487922+0000","last_peered":"2024-09-11T07:46:31.487922+0000","last_clean":"2024-09-11T07:46:31.487922+0000","last_became_active":"2024-09-11T07:36:05.225840+0000","last_became_peered":"2024-09-11T07:36:05.225840+0000","last_unstale":"2024-09-11T07:46:31.487922+0000","last_undegraded":"2024-09-11T07:46:31.487922+0000","last_fullsized":"2024-09-11T07:46:31.487922+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:12.312068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:22.285763+0000","last_clean_scrub_stamp":"2024-09-11T07:44:12.312068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01309188,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":69,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.488761+0000","last_change":"2024-09-11T07:45:23.663301+0000","last_active":"2024-09-11T07:46:31.488761+0000","last_peered":"2024-09-11T07:46:31.488761+0000","last_clean":"2024-09-11T07:46:31.488761+0000","last_became_active":"2024-09-11T07:44:23.769063+0000","last_became_peered":"2024-09-11T07:44:23.769063+0000","last_unstale":"2024-09-11T07:46:31.488761+0000","last_undegraded":"2024-09-11T07:46:31.488761+0000","last_fullsized":"2024-09-11T07:46:31.488761+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:23.663218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:39.723598+0000","last_clean_scrub_stamp":"2024-09-11T07:45:23.663218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:51.865688+0000","scrub_duration":0.006627272,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"426'3347","reported_seq":3369,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:44.162020+0000","last_change":"2024-09-11T07:45:57.201789+0000","last_active":"2024-09-11T07:46:44.162020+0000","last_peered":"2024-09-11T07:46:44.162020+0000","last_clean":"2024-09-11T07:46:44.162020+0000","last_became_active":"2024-09-11T07:45:57.201368+0000","last_became_peered":"2024-09-11T07:45:57.201368+0000","last_unstale":"2024-09-11T07:46:44.162020+0000","last_undegraded":"2024-09-11T07:46:44.162020+0000","last_fullsized":"2024-09-11T07:46:44.162020+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3347,"log_dups_size":0,"ondisk_log_size":3347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:58.453865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":219348992,"num_objects":3347,"num_object_clones":0,"num_object_copies":10041,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3347,"num_write_kb":214208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"87'81","reported_seq":15,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.504793+0000","last_change":"2024-09-11T07:46:31.504183+0000","last_active":"2024-09-11T07:46:31.504793+0000","last_peered":"2024-09-11T07:46:31.504793+0000","last_clean":"2024-09-11T07:46:31.504793+0000","last_became_active":"2024-09-11T07:46:31.503868+0000","last_became_peered":"2024-09-11T07:46:31.503868+0000","last_unstale":"2024-09-11T07:46:31.504793+0000","last_undegraded":"2024-09-11T07:46:31.504793+0000","last_fullsized":"2024-09-11T07:46:31.504793+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:46:18.111609+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:46:18.111609+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015232676000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"426'30","reported_seq":348,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.675715+0000","last_change":"2024-09-11T07:46:08.202429+0000","last_active":"2024-09-11T07:46:40.675715+0000","last_peered":"2024-09-11T07:46:40.675715+0000","last_clean":"2024-09-11T07:46:40.675715+0000","last_became_active":"2024-09-11T07:45:27.119509+0000","last_became_peered":"2024-09-11T07:45:27.119509+0000","last_unstale":"2024-09-11T07:46:40.675715+0000","last_undegraded":"2024-09-11T07:46:40.675715+0000","last_fullsized":"2024-09-11T07:46:40.675715+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'20","last_scrub_stamp":"2024-09-11T07:46:08.202366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:46:08.202366+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:15.502719+0000","scrub_duration":0.0024018009999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":77,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.487167+0000","last_change":"2024-09-11T07:45:12.553757+0000","last_active":"2024-09-11T07:46:31.487167+0000","last_peered":"2024-09-11T07:46:31.487167+0000","last_clean":"2024-09-11T07:46:31.487167+0000","last_became_active":"2024-09-11T07:44:23.771022+0000","last_became_peered":"2024-09-11T07:44:23.771022+0000","last_unstale":"2024-09-11T07:46:31.487167+0000","last_undegraded":"2024-09-11T07:46:31.487167+0000","last_fullsized":"2024-09-11T07:46:31.487167+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:45:12.551633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:12.551633+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014607669,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"290'9","reported_seq":727,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:35.773287+0000","last_change":"2024-09-11T07:45:21.492545+0000","last_active":"2024-09-11T07:46:35.773287+0000","last_peered":"2024-09-11T07:46:35.773287+0000","last_clean":"2024-09-11T07:46:35.773287+0000","last_became_active":"2024-09-11T07:39:57.768155+0000","last_became_peered":"2024-09-11T07:39:57.768155+0000","last_unstale":"2024-09-11T07:46:35.773287+0000","last_undegraded":"2024-09-11T07:46:35.773287+0000","last_fullsized":"2024-09-11T07:46:35.773287+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"290'9","last_scrub_stamp":"2024-09-11T07:45:21.492460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:45:21.492460+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:48.304323+0000","scrub_duration":0.0074254259999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":461,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.487525+0000","last_change":"2024-09-11T07:46:17.190867+0000","last_active":"2024-09-11T07:46:31.487525+0000","last_peered":"2024-09-11T07:46:31.487525+0000","last_clean":"2024-09-11T07:46:31.487525+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:46:31.487525+0000","last_undegraded":"2024-09-11T07:46:31.487525+0000","last_fullsized":"2024-09-11T07:46:31.487525+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:17.190770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_clean_scrub_stamp":"2024-09-11T07:46:17.190770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:29.410697+0000","scrub_duration":0.007916569,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"426'3485","reported_seq":3507,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:45.185429+0000","last_change":"2024-09-11T07:45:57.199626+0000","last_active":"2024-09-11T07:46:45.185429+0000","last_peered":"2024-09-11T07:46:45.185429+0000","last_clean":"2024-09-11T07:46:45.185429+0000","last_became_active":"2024-09-11T07:45:57.199305+0000","last_became_peered":"2024-09-11T07:45:57.199305+0000","last_unstale":"2024-09-11T07:46:45.185429+0000","last_undegraded":"2024-09-11T07:46:45.185429+0000","last_fullsized":"2024-09-11T07:46:45.185429+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3485,"log_dups_size":0,"ondisk_log_size":3485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:18.422653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":228392960,"num_objects":3485,"num_object_clones":0,"num_object_copies":10455,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3485,"num_write_kb":223040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"87'81","reported_seq":323,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.492040+0000","last_change":"2024-09-11T07:46:21.225100+0000","last_active":"2024-09-11T07:46:31.492040+0000","last_peered":"2024-09-11T07:46:31.492040+0000","last_clean":"2024-09-11T07:46:31.492040+0000","last_became_active":"2024-09-11T07:45:01.820927+0000","last_became_peered":"2024-09-11T07:45:01.820927+0000","last_unstale":"2024-09-11T07:46:31.492040+0000","last_undegraded":"2024-09-11T07:46:31.492040+0000","last_fullsized":"2024-09-11T07:46:31.492040+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:46:21.225015+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:46:21.225015+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:25.231800+0000","scrub_duration":0.0050456870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"426'54","reported_seq":473,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.693210+0000","last_change":"2024-09-11T07:46:17.005728+0000","last_active":"2024-09-11T07:46:40.693210+0000","last_peered":"2024-09-11T07:46:40.693210+0000","last_clean":"2024-09-11T07:46:40.693210+0000","last_became_active":"2024-09-11T07:39:57.768267+0000","last_became_peered":"2024-09-11T07:39:57.768267+0000","last_unstale":"2024-09-11T07:46:40.693210+0000","last_undegraded":"2024-09-11T07:46:40.693210+0000","last_fullsized":"2024-09-11T07:46:40.693210+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"290'42","last_scrub_stamp":"2024-09-11T07:46:17.005627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:46:17.005627+0000","objects_scrubbed":10,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:35.160644+0000","scrub_duration":0.0098261949999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":83,"num_read_kb":68,"num_write":50,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":434,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.488886+0000","last_change":"2024-09-11T07:45:08.282066+0000","last_active":"2024-09-11T07:46:31.488886+0000","last_peered":"2024-09-11T07:46:31.488886+0000","last_clean":"2024-09-11T07:46:31.488886+0000","last_became_active":"2024-09-11T07:44:23.772612+0000","last_became_peered":"2024-09-11T07:44:23.772612+0000","last_unstale":"2024-09-11T07:46:31.488886+0000","last_undegraded":"2024-09-11T07:46:31.488886+0000","last_fullsized":"2024-09-11T07:46:31.488886+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:08.281957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:51.308492+0000","last_clean_scrub_stamp":"2024-09-11T07:45:08.281957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:36.150157+0000","scrub_duration":0.012237031000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":463,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.489141+0000","last_change":"2024-09-11T07:45:27.120609+0000","last_active":"2024-09-11T07:46:31.489141+0000","last_peered":"2024-09-11T07:46:31.489141+0000","last_clean":"2024-09-11T07:46:31.489141+0000","last_became_active":"2024-09-11T07:45:27.120021+0000","last_became_peered":"2024-09-11T07:45:27.120021+0000","last_unstale":"2024-09-11T07:46:31.489141+0000","last_undegraded":"2024-09-11T07:46:31.489141+0000","last_fullsized":"2024-09-11T07:46:31.489141+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:20.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:44:20.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075336659999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":478,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.484870+0000","last_change":"2024-09-11T07:45:27.103089+0000","last_active":"2024-09-11T07:46:31.484870+0000","last_peered":"2024-09-11T07:46:31.484870+0000","last_clean":"2024-09-11T07:46:31.484870+0000","last_became_active":"2024-09-11T07:45:27.102600+0000","last_became_peered":"2024-09-11T07:45:27.102600+0000","last_unstale":"2024-09-11T07:46:31.484870+0000","last_undegraded":"2024-09-11T07:46:31.484870+0000","last_fullsized":"2024-09-11T07:46:31.484870+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:13.504354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:45:13.504354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018451749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"426'3465","reported_seq":3487,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:44.353962+0000","last_change":"2024-09-11T07:45:57.198992+0000","last_active":"2024-09-11T07:46:44.353962+0000","last_peered":"2024-09-11T07:46:44.353962+0000","last_clean":"2024-09-11T07:46:44.353962+0000","last_became_active":"2024-09-11T07:45:57.198709+0000","last_became_peered":"2024-09-11T07:45:57.198709+0000","last_unstale":"2024-09-11T07:46:44.353962+0000","last_undegraded":"2024-09-11T07:46:44.353962+0000","last_fullsized":"2024-09-11T07:46:44.353962+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3465,"log_dups_size":0,"ondisk_log_size":3465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:59.340606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":227082240,"num_objects":3465,"num_object_clones":0,"num_object_copies":10395,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3465,"num_write_kb":221760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"87'81","reported_seq":334,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.487348+0000","last_change":"2024-09-11T07:46:29.293393+0000","last_active":"2024-09-11T07:46:31.487348+0000","last_peered":"2024-09-11T07:46:31.487348+0000","last_clean":"2024-09-11T07:46:31.487348+0000","last_became_active":"2024-09-11T07:45:27.117020+0000","last_became_peered":"2024-09-11T07:45:27.117020+0000","last_unstale":"2024-09-11T07:46:31.487348+0000","last_undegraded":"2024-09-11T07:46:31.487348+0000","last_fullsized":"2024-09-11T07:46:31.487348+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:46:29.293202+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:46:29.293202+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:54.265960+0000","scrub_duration":0.0030387169999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,5],"acting":[0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"426'26","reported_seq":295,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.274581+0000","last_change":"2024-09-11T07:45:49.765866+0000","last_active":"2024-09-11T07:46:40.274581+0000","last_peered":"2024-09-11T07:46:40.274581+0000","last_clean":"2024-09-11T07:46:40.274581+0000","last_became_active":"2024-09-11T07:45:49.765519+0000","last_became_peered":"2024-09-11T07:45:49.765519+0000","last_unstale":"2024-09-11T07:46:40.274581+0000","last_undegraded":"2024-09-11T07:46:40.274581+0000","last_fullsized":"2024-09-11T07:46:40.274581+0000","mapping_epoch":407,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":408,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'22","last_scrub_stamp":"2024-09-11T07:45:17.380652+0000","last_deep_scrub":"144'18","last_deep_scrub_stamp":"2024-09-11T07:31:51.603378+0000","last_clean_scrub_stamp":"2024-09-11T07:45:17.380652+0000","objects_scrubbed":5,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:45.277530+0000","scrub_duration":0.006530999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":41,"num_read_kb":30,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":238,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:43.165665+0000","last_change":"2024-09-11T07:45:20.637602+0000","last_active":"2024-09-11T07:46:43.165665+0000","last_peered":"2024-09-11T07:46:43.165665+0000","last_clean":"2024-09-11T07:46:43.165665+0000","last_became_active":"2024-09-11T07:44:23.769220+0000","last_became_peered":"2024-09-11T07:44:23.769220+0000","last_unstale":"2024-09-11T07:46:43.165665+0000","last_undegraded":"2024-09-11T07:46:43.165665+0000","last_fullsized":"2024-09-11T07:46:43.165665+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'5","last_scrub_stamp":"2024-09-11T07:45:20.637535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:20.637535+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059316919999999997,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":147,"num_read_kb":142,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":486,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.488853+0000","last_change":"2024-09-11T07:45:07.727921+0000","last_active":"2024-09-11T07:46:31.488853+0000","last_peered":"2024-09-11T07:46:31.488853+0000","last_clean":"2024-09-11T07:46:31.488853+0000","last_became_active":"2024-09-11T07:26:31.413354+0000","last_became_peered":"2024-09-11T07:26:31.413354+0000","last_unstale":"2024-09-11T07:46:31.488853+0000","last_undegraded":"2024-09-11T07:46:31.488853+0000","last_fullsized":"2024-09-11T07:46:31.488853+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:07.727847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:07.149367+0000","last_clean_scrub_stamp":"2024-09-11T07:45:07.727847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053654319999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":473,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.491348+0000","last_change":"2024-09-11T07:43:11.249241+0000","last_active":"2024-09-11T07:46:31.491348+0000","last_peered":"2024-09-11T07:46:31.491348+0000","last_clean":"2024-09-11T07:46:31.491348+0000","last_became_active":"2024-09-11T07:32:00.508398+0000","last_became_peered":"2024-09-11T07:32:00.508398+0000","last_unstale":"2024-09-11T07:46:31.491348+0000","last_undegraded":"2024-09-11T07:46:31.491348+0000","last_fullsized":"2024-09-11T07:46:31.491348+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:43:11.242643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:28.246601+0000","last_clean_scrub_stamp":"2024-09-11T07:43:11.242643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056937530000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":451,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.508418+0000","last_change":"2024-09-11T07:44:25.382614+0000","last_active":"2024-09-11T07:46:31.508418+0000","last_peered":"2024-09-11T07:46:31.508418+0000","last_clean":"2024-09-11T07:46:31.508418+0000","last_became_active":"2024-09-11T07:32:00.515745+0000","last_became_peered":"2024-09-11T07:32:00.515745+0000","last_unstale":"2024-09-11T07:46:31.508418+0000","last_undegraded":"2024-09-11T07:46:31.508418+0000","last_fullsized":"2024-09-11T07:46:31.508418+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:25.382533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:25.382533+0000","last_clean_scrub_stamp":"2024-09-11T07:44:25.382533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018079610000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"426'3509","reported_seq":3530,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:45.188904+0000","last_change":"2024-09-11T07:45:57.202010+0000","last_active":"2024-09-11T07:46:45.188904+0000","last_peered":"2024-09-11T07:46:45.188904+0000","last_clean":"2024-09-11T07:46:45.188904+0000","last_became_active":"2024-09-11T07:45:57.201607+0000","last_became_peered":"2024-09-11T07:45:57.201607+0000","last_unstale":"2024-09-11T07:46:45.188904+0000","last_undegraded":"2024-09-11T07:46:45.188904+0000","last_fullsized":"2024-09-11T07:46:45.188904+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3509,"log_dups_size":0,"ondisk_log_size":3509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:58.302447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":229965824,"num_objects":3509,"num_object_clones":0,"num_object_copies":10527,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3509,"num_write_kb":224576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"87'81","reported_seq":41,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.490838+0000","last_change":"2024-09-11T07:45:27.115249+0000","last_active":"2024-09-11T07:46:31.490838+0000","last_peered":"2024-09-11T07:46:31.490838+0000","last_clean":"2024-09-11T07:46:31.490838+0000","last_became_active":"2024-09-11T07:45:27.114518+0000","last_became_peered":"2024-09-11T07:45:27.114518+0000","last_unstale":"2024-09-11T07:46:31.490838+0000","last_undegraded":"2024-09-11T07:46:31.490838+0000","last_fullsized":"2024-09-11T07:46:31.490838+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:44:22.387527+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.387527+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015639180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":21,"num_write":11,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":53670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"426'42","reported_seq":573,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.343425+0000","last_change":"2024-09-11T07:44:28.272807+0000","last_active":"2024-09-11T07:46:40.343425+0000","last_peered":"2024-09-11T07:46:40.343425+0000","last_clean":"2024-09-11T07:46:40.343425+0000","last_became_active":"2024-09-11T07:43:05.417321+0000","last_became_peered":"2024-09-11T07:43:05.417321+0000","last_unstale":"2024-09-11T07:46:40.343425+0000","last_undegraded":"2024-09-11T07:46:40.343425+0000","last_fullsized":"2024-09-11T07:46:40.343425+0000","mapping_epoch":358,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":359,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'30","last_scrub_stamp":"2024-09-11T07:44:28.272498+0000","last_deep_scrub":"94'18","last_deep_scrub_stamp":"2024-09-11T07:28:51.437397+0000","last_clean_scrub_stamp":"2024-09-11T07:44:28.272498+0000","objects_scrubbed":6,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006216643,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":54,"num_read_kb":48,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"422'2","reported_seq":502,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.490394+0000","last_change":"2024-09-11T07:46:22.213763+0000","last_active":"2024-09-11T07:46:31.490394+0000","last_peered":"2024-09-11T07:46:31.490394+0000","last_clean":"2024-09-11T07:46:31.490394+0000","last_became_active":"2024-09-11T07:45:27.115139+0000","last_became_peered":"2024-09-11T07:45:27.115139+0000","last_unstale":"2024-09-11T07:46:31.490394+0000","last_undegraded":"2024-09-11T07:46:31.490394+0000","last_fullsized":"2024-09-11T07:46:31.490394+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"422'2","last_scrub_stamp":"2024-09-11T07:46:22.213676+0000","last_deep_scrub":"422'2","last_deep_scrub_stamp":"2024-09-11T07:46:22.213676+0000","last_clean_scrub_stamp":"2024-09-11T07:46:22.213676+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:31.909700+0000","scrub_duration":0.0077027509999999999,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":472,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.507888+0000","last_change":"2024-09-11T07:46:27.815229+0000","last_active":"2024-09-11T07:46:31.507888+0000","last_peered":"2024-09-11T07:46:31.507888+0000","last_clean":"2024-09-11T07:46:31.507888+0000","last_became_active":"2024-09-11T07:39:57.772244+0000","last_became_peered":"2024-09-11T07:39:57.772244+0000","last_unstale":"2024-09-11T07:46:31.507888+0000","last_undegraded":"2024-09-11T07:46:31.507888+0000","last_fullsized":"2024-09-11T07:46:31.507888+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:27.815152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:16.967492+0000","last_clean_scrub_stamp":"2024-09-11T07:46:27.815152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:41.545249+0000","scrub_duration":0.0047261389999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":79,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.489286+0000","last_change":"2024-09-11T07:45:27.102656+0000","last_active":"2024-09-11T07:46:31.489286+0000","last_peered":"2024-09-11T07:46:31.489286+0000","last_clean":"2024-09-11T07:46:31.489286+0000","last_became_active":"2024-09-11T07:45:27.101352+0000","last_became_peered":"2024-09-11T07:45:27.101352+0000","last_unstale":"2024-09-11T07:46:31.489286+0000","last_undegraded":"2024-09-11T07:46:31.489286+0000","last_fullsized":"2024-09-11T07:46:31.489286+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:22.519491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:51.187127+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.519491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.024174253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":18,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.493819+0000","last_change":"2024-09-11T07:46:31.493189+0000","last_active":"2024-09-11T07:46:31.493819+0000","last_peered":"2024-09-11T07:46:31.493819+0000","last_clean":"2024-09-11T07:46:31.493819+0000","last_became_active":"2024-09-11T07:46:31.492869+0000","last_became_peered":"2024-09-11T07:46:31.492869+0000","last_unstale":"2024-09-11T07:46:31.493819+0000","last_undegraded":"2024-09-11T07:46:31.493819+0000","last_fullsized":"2024-09-11T07:46:31.493819+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:21.014717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:46:21.014717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:43.295895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"426'3490","reported_seq":3512,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:45.172343+0000","last_change":"2024-09-11T07:45:57.199407+0000","last_active":"2024-09-11T07:46:45.172343+0000","last_peered":"2024-09-11T07:46:45.172343+0000","last_clean":"2024-09-11T07:46:45.172343+0000","last_became_active":"2024-09-11T07:45:57.199102+0000","last_became_peered":"2024-09-11T07:45:57.199102+0000","last_unstale":"2024-09-11T07:46:45.172343+0000","last_undegraded":"2024-09-11T07:46:45.172343+0000","last_fullsized":"2024-09-11T07:46:45.172343+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3490,"log_dups_size":0,"ondisk_log_size":3490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:17.282566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":228720640,"num_objects":3490,"num_object_clones":0,"num_object_copies":10470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3490,"num_write_kb":223360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"426'45","reported_seq":167,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.712833+0000","last_change":"2024-09-11T07:45:32.442239+0000","last_active":"2024-09-11T07:46:40.712833+0000","last_peered":"2024-09-11T07:46:40.712833+0000","last_clean":"2024-09-11T07:46:40.712833+0000","last_became_active":"2024-09-11T07:45:27.113188+0000","last_became_peered":"2024-09-11T07:45:27.113188+0000","last_unstale":"2024-09-11T07:46:40.712833+0000","last_undegraded":"2024-09-11T07:46:40.712833+0000","last_fullsized":"2024-09-11T07:46:40.712833+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"399'33","last_scrub_stamp":"2024-09-11T07:45:32.442160+0000","last_deep_scrub":"399'33","last_deep_scrub_stamp":"2024-09-11T07:45:32.442160+0000","last_clean_scrub_stamp":"2024-09-11T07:45:32.442160+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:52.336506+0000","scrub_duration":0.015982277,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":59,"num_read_kb":49,"num_write":37,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"83'3","reported_seq":493,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.487037+0000","last_change":"2024-09-11T07:45:27.153261+0000","last_active":"2024-09-11T07:46:31.487037+0000","last_peered":"2024-09-11T07:46:31.487037+0000","last_clean":"2024-09-11T07:46:31.487037+0000","last_became_active":"2024-09-11T07:45:27.099666+0000","last_became_peered":"2024-09-11T07:45:27.099666+0000","last_unstale":"2024-09-11T07:46:31.487037+0000","last_undegraded":"2024-09-11T07:46:31.487037+0000","last_fullsized":"2024-09-11T07:46:31.487037+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-09-11T07:44:30.572101+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_clean_scrub_stamp":"2024-09-11T07:44:30.572101+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003584968,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":333,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.502387+0000","last_change":"2024-09-11T07:46:31.501022+0000","last_active":"2024-09-11T07:46:31.502387+0000","last_peered":"2024-09-11T07:46:31.502387+0000","last_clean":"2024-09-11T07:46:31.502387+0000","last_became_active":"2024-09-11T07:46:31.500613+0000","last_became_peered":"2024-09-11T07:46:31.500613+0000","last_unstale":"2024-09-11T07:46:31.502387+0000","last_undegraded":"2024-09-11T07:46:31.502387+0000","last_fullsized":"2024-09-11T07:46:31.502387+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:18.381749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:18.381749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013066169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"87'81","reported_seq":315,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.488047+0000","last_change":"2024-09-11T07:46:20.190045+0000","last_active":"2024-09-11T07:46:31.488047+0000","last_peered":"2024-09-11T07:46:31.488047+0000","last_clean":"2024-09-11T07:46:31.488047+0000","last_became_active":"2024-09-11T07:44:59.848971+0000","last_became_peered":"2024-09-11T07:44:59.848971+0000","last_unstale":"2024-09-11T07:46:31.488047+0000","last_undegraded":"2024-09-11T07:46:31.488047+0000","last_fullsized":"2024-09-11T07:46:31.488047+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:46:20.189949+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:46:20.189949+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:20.387224+0000","scrub_duration":0.0063253600000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":283,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.496239+0000","last_change":"2024-09-11T07:45:22.667614+0000","last_active":"2024-09-11T07:46:31.496239+0000","last_peered":"2024-09-11T07:46:31.496239+0000","last_clean":"2024-09-11T07:46:31.496239+0000","last_became_active":"2024-09-11T07:39:57.748658+0000","last_became_peered":"2024-09-11T07:39:57.748658+0000","last_unstale":"2024-09-11T07:46:31.496239+0000","last_undegraded":"2024-09-11T07:46:31.496239+0000","last_fullsized":"2024-09-11T07:46:31.496239+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:22.667519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:45:22.667519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:36.493380+0000","scrub_duration":0.0046246309999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":438,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.509480+0000","last_change":"2024-09-11T07:46:31.509326+0000","last_active":"2024-09-11T07:46:31.509480+0000","last_peered":"2024-09-11T07:46:31.509480+0000","last_clean":"2024-09-11T07:46:31.509480+0000","last_became_active":"2024-09-11T07:46:31.509054+0000","last_became_peered":"2024-09-11T07:46:31.509054+0000","last_unstale":"2024-09-11T07:46:31.509480+0000","last_undegraded":"2024-09-11T07:46:31.509480+0000","last_fullsized":"2024-09-11T07:46:31.509480+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:14.389468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:45:14.389468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005315012,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.5","version":"426'3440","reported_seq":3461,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:44.373421+0000","last_change":"2024-09-11T07:45:57.199301+0000","last_active":"2024-09-11T07:46:44.373421+0000","last_peered":"2024-09-11T07:46:44.373421+0000","last_clean":"2024-09-11T07:46:44.373421+0000","last_became_active":"2024-09-11T07:45:57.199035+0000","last_became_peered":"2024-09-11T07:45:57.199035+0000","last_unstale":"2024-09-11T07:46:44.373421+0000","last_undegraded":"2024-09-11T07:46:44.373421+0000","last_fullsized":"2024-09-11T07:46:44.373421+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3440,"log_dups_size":0,"ondisk_log_size":3440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:09.238194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225443840,"num_objects":3440,"num_object_clones":0,"num_object_copies":10320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3440,"num_write_kb":220160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"426'51","reported_seq":125,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.835278+0000","last_change":"2024-09-11T07:46:31.495428+0000","last_active":"2024-09-11T07:46:40.835278+0000","last_peered":"2024-09-11T07:46:40.835278+0000","last_clean":"2024-09-11T07:46:40.835278+0000","last_became_active":"2024-09-11T07:46:31.494953+0000","last_became_peered":"2024-09-11T07:46:31.494953+0000","last_unstale":"2024-09-11T07:46:40.835278+0000","last_undegraded":"2024-09-11T07:46:40.835278+0000","last_fullsized":"2024-09-11T07:46:40.835278+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'35","last_scrub_stamp":"2024-09-11T07:44:32.235984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:44:32.235984+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":78,"num_read_kb":67,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":479,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.496312+0000","last_change":"2024-09-11T07:45:10.747839+0000","last_active":"2024-09-11T07:46:31.496312+0000","last_peered":"2024-09-11T07:46:31.496312+0000","last_clean":"2024-09-11T07:46:31.496312+0000","last_became_active":"2024-09-11T07:26:31.419560+0000","last_became_peered":"2024-09-11T07:26:31.419560+0000","last_unstale":"2024-09-11T07:46:31.496312+0000","last_undegraded":"2024-09-11T07:46:31.496312+0000","last_fullsized":"2024-09-11T07:46:31.496312+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:10.747787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:10.747787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:39.300259+0000","scrub_duration":0.0014756929999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":476,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.488846+0000","last_change":"2024-09-11T07:44:23.485237+0000","last_active":"2024-09-11T07:46:31.488846+0000","last_peered":"2024-09-11T07:46:31.488846+0000","last_clean":"2024-09-11T07:46:31.488846+0000","last_became_active":"2024-09-11T07:39:57.748953+0000","last_became_peered":"2024-09-11T07:39:57.748953+0000","last_unstale":"2024-09-11T07:46:31.488846+0000","last_undegraded":"2024-09-11T07:46:31.488846+0000","last_fullsized":"2024-09-11T07:46:31.488846+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:23.485176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:44:23.485176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015945569999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":281,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.492448+0000","last_change":"2024-09-11T07:44:00.601823+0000","last_active":"2024-09-11T07:46:31.492448+0000","last_peered":"2024-09-11T07:46:31.492448+0000","last_clean":"2024-09-11T07:46:31.492448+0000","last_became_active":"2024-09-11T07:39:57.766605+0000","last_became_peered":"2024-09-11T07:39:57.766605+0000","last_unstale":"2024-09-11T07:46:31.492448+0000","last_undegraded":"2024-09-11T07:46:31.492448+0000","last_fullsized":"2024-09-11T07:46:31.492448+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:00.601321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:44:00.601321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.009846334,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.4","version":"426'3536","reported_seq":3558,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:44.507462+0000","last_change":"2024-09-11T07:45:57.201461+0000","last_active":"2024-09-11T07:46:44.507462+0000","last_peered":"2024-09-11T07:46:44.507462+0000","last_clean":"2024-09-11T07:46:44.507462+0000","last_became_active":"2024-09-11T07:45:57.201054+0000","last_became_peered":"2024-09-11T07:45:57.201054+0000","last_unstale":"2024-09-11T07:46:44.507462+0000","last_undegraded":"2024-09-11T07:46:44.507462+0000","last_fullsized":"2024-09-11T07:46:44.507462+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3536,"log_dups_size":0,"ondisk_log_size":3536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:01.222410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":231735296,"num_objects":3536,"num_object_clones":0,"num_object_copies":10608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3536,"num_write_kb":226304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"426'21","reported_seq":509,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.616593+0000","last_change":"2024-09-11T07:45:27.145068+0000","last_active":"2024-09-11T07:46:40.616593+0000","last_peered":"2024-09-11T07:46:40.616593+0000","last_clean":"2024-09-11T07:46:40.616593+0000","last_became_active":"2024-09-11T07:45:27.114646+0000","last_became_peered":"2024-09-11T07:45:27.114646+0000","last_unstale":"2024-09-11T07:46:40.616593+0000","last_undegraded":"2024-09-11T07:46:40.616593+0000","last_fullsized":"2024-09-11T07:46:40.616593+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:45:11.521203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:45:11.521203+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015327073,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":58,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.484388+0000","last_change":"2024-09-11T07:46:16.505169+0000","last_active":"2024-09-11T07:46:31.484388+0000","last_peered":"2024-09-11T07:46:31.484388+0000","last_clean":"2024-09-11T07:46:31.484388+0000","last_became_active":"2024-09-11T07:45:27.102350+0000","last_became_peered":"2024-09-11T07:45:27.102350+0000","last_unstale":"2024-09-11T07:46:31.484388+0000","last_undegraded":"2024-09-11T07:46:31.484388+0000","last_fullsized":"2024-09-11T07:46:31.484388+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:16.505062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:46:16.505062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:40.975683+0000","scrub_duration":0.037745665999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":492,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.490798+0000","last_change":"2024-09-11T07:46:12.107858+0000","last_active":"2024-09-11T07:46:31.490798+0000","last_peered":"2024-09-11T07:46:31.490798+0000","last_clean":"2024-09-11T07:46:31.490798+0000","last_became_active":"2024-09-11T07:46:12.107538+0000","last_became_peered":"2024-09-11T07:46:12.107538+0000","last_unstale":"2024-09-11T07:46:31.490798+0000","last_undegraded":"2024-09-11T07:46:31.490798+0000","last_fullsized":"2024-09-11T07:46:31.490798+0000","mapping_epoch":418,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":419,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:08.436552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:44:08.436552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0064625339999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":468,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.489872+0000","last_change":"2024-09-11T07:45:22.489678+0000","last_active":"2024-09-11T07:46:31.489872+0000","last_peered":"2024-09-11T07:46:31.489872+0000","last_clean":"2024-09-11T07:46:31.489872+0000","last_became_active":"2024-09-11T07:32:00.505185+0000","last_became_peered":"2024-09-11T07:32:00.505185+0000","last_unstale":"2024-09-11T07:46:31.489872+0000","last_undegraded":"2024-09-11T07:46:31.489872+0000","last_fullsized":"2024-09-11T07:46:31.489872+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:22.489595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:22.489595+0000","last_clean_scrub_stamp":"2024-09-11T07:45:22.489595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.045984707999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"426'3424","reported_seq":3446,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:44.383654+0000","last_change":"2024-09-11T07:45:56.764280+0000","last_active":"2024-09-11T07:46:44.383654+0000","last_peered":"2024-09-11T07:46:44.383654+0000","last_clean":"2024-09-11T07:46:44.383654+0000","last_became_active":"2024-09-11T07:45:56.763089+0000","last_became_peered":"2024-09-11T07:45:56.763089+0000","last_unstale":"2024-09-11T07:46:44.383654+0000","last_undegraded":"2024-09-11T07:46:44.383654+0000","last_fullsized":"2024-09-11T07:46:44.383654+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3424,"log_dups_size":0,"ondisk_log_size":3424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:14.646257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":224395264,"num_objects":3424,"num_object_clones":0,"num_object_copies":10272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3424,"num_write_kb":219136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"426'46","reported_seq":96,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.442548+0000","last_change":"2024-09-11T07:46:14.474268+0000","last_active":"2024-09-11T07:46:40.442548+0000","last_peered":"2024-09-11T07:46:40.442548+0000","last_clean":"2024-09-11T07:46:40.442548+0000","last_became_active":"2024-09-11T07:45:27.099653+0000","last_became_peered":"2024-09-11T07:45:27.099653+0000","last_unstale":"2024-09-11T07:46:40.442548+0000","last_undegraded":"2024-09-11T07:46:40.442548+0000","last_fullsized":"2024-09-11T07:46:40.442548+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"421'34","last_scrub_stamp":"2024-09-11T07:46:14.474174+0000","last_deep_scrub":"421'34","last_deep_scrub_stamp":"2024-09-11T07:46:14.474174+0000","last_clean_scrub_stamp":"2024-09-11T07:46:14.474174+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:41.275098+0000","scrub_duration":0.01054072,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":514,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.491693+0000","last_change":"2024-09-11T07:44:25.455385+0000","last_active":"2024-09-11T07:46:31.491693+0000","last_peered":"2024-09-11T07:46:31.491693+0000","last_clean":"2024-09-11T07:46:31.491693+0000","last_became_active":"2024-09-11T07:40:11.701330+0000","last_became_peered":"2024-09-11T07:40:11.701330+0000","last_unstale":"2024-09-11T07:46:31.491693+0000","last_undegraded":"2024-09-11T07:46:31.491693+0000","last_fullsized":"2024-09-11T07:46:31.491693+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:25.455028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_clean_scrub_stamp":"2024-09-11T07:44:25.455028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044567610000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"395'11","reported_seq":679,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:35.761465+0000","last_change":"2024-09-11T07:46:07.749662+0000","last_active":"2024-09-11T07:46:35.761465+0000","last_peered":"2024-09-11T07:46:35.761465+0000","last_clean":"2024-09-11T07:46:35.761465+0000","last_became_active":"2024-09-11T07:45:27.093333+0000","last_became_peered":"2024-09-11T07:45:27.093333+0000","last_unstale":"2024-09-11T07:46:35.761465+0000","last_undegraded":"2024-09-11T07:46:35.761465+0000","last_fullsized":"2024-09-11T07:46:35.761465+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"395'11","last_scrub_stamp":"2024-09-11T07:46:07.749598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:46:07.749598+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:22.604660+0000","scrub_duration":0.0017864059999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":112,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.487851+0000","last_change":"2024-09-11T07:45:26.475689+0000","last_active":"2024-09-11T07:46:31.487851+0000","last_peered":"2024-09-11T07:46:31.487851+0000","last_clean":"2024-09-11T07:46:31.487851+0000","last_became_active":"2024-09-11T07:42:04.729103+0000","last_became_peered":"2024-09-11T07:42:04.729103+0000","last_unstale":"2024-09-11T07:46:31.487851+0000","last_undegraded":"2024-09-11T07:46:31.487851+0000","last_fullsized":"2024-09-11T07:46:31.487851+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:26.475614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:45:26.475614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:44.947478+0000","scrub_duration":0.001709363,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"426'3408","reported_seq":3430,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:44.353967+0000","last_change":"2024-09-11T07:45:57.199112+0000","last_active":"2024-09-11T07:46:44.353967+0000","last_peered":"2024-09-11T07:46:44.353967+0000","last_clean":"2024-09-11T07:46:44.353967+0000","last_became_active":"2024-09-11T07:45:57.198860+0000","last_became_peered":"2024-09-11T07:45:57.198860+0000","last_unstale":"2024-09-11T07:46:44.353967+0000","last_undegraded":"2024-09-11T07:46:44.353967+0000","last_fullsized":"2024-09-11T07:46:44.353967+0000","mapping_epoch":412,"log_start":"0'0","ondisk_log_start":"0'0","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3408,"log_dups_size":0,"ondisk_log_size":3408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:24.428389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":223346688,"num_objects":3408,"num_object_clones":0,"num_object_copies":10224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3408,"num_write_kb":218112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"426'29","reported_seq":72,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:39.516349+0000","last_change":"2024-09-11T07:46:07.191286+0000","last_active":"2024-09-11T07:46:39.516349+0000","last_peered":"2024-09-11T07:46:39.516349+0000","last_clean":"2024-09-11T07:46:39.516349+0000","last_became_active":"2024-09-11T07:45:27.118858+0000","last_became_peered":"2024-09-11T07:45:27.118858+0000","last_unstale":"2024-09-11T07:46:39.516349+0000","last_undegraded":"2024-09-11T07:46:39.516349+0000","last_fullsized":"2024-09-11T07:46:39.516349+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'23","last_scrub_stamp":"2024-09-11T07:46:07.191227+0000","last_deep_scrub":"234'23","last_deep_scrub_stamp":"2024-09-11T07:37:34.613280+0000","last_clean_scrub_stamp":"2024-09-11T07:46:07.191227+0000","objects_scrubbed":5,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:20.944254+0000","scrub_duration":0.002042533,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":59,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.489308+0000","last_change":"2024-09-11T07:45:27.125454+0000","last_active":"2024-09-11T07:46:31.489308+0000","last_peered":"2024-09-11T07:46:31.489308+0000","last_clean":"2024-09-11T07:46:31.489308+0000","last_became_active":"2024-09-11T07:45:27.112394+0000","last_became_peered":"2024-09-11T07:45:27.112394+0000","last_unstale":"2024-09-11T07:46:31.489308+0000","last_undegraded":"2024-09-11T07:46:31.489308+0000","last_fullsized":"2024-09-11T07:46:31.489308+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:45:08.089525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:08.089525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":472,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.492920+0000","last_change":"2024-09-11T07:44:28.572130+0000","last_active":"2024-09-11T07:46:31.492920+0000","last_peered":"2024-09-11T07:46:31.492920+0000","last_clean":"2024-09-11T07:46:31.492920+0000","last_became_active":"2024-09-11T07:26:35.561967+0000","last_became_peered":"2024-09-11T07:26:35.561967+0000","last_unstale":"2024-09-11T07:46:31.492920+0000","last_undegraded":"2024-09-11T07:46:31.492920+0000","last_fullsized":"2024-09-11T07:46:31.492920+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:28.572059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:44.820354+0000","last_clean_scrub_stamp":"2024-09-11T07:44:28.572059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003042159,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":478,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.486752+0000","last_change":"2024-09-11T07:44:27.626846+0000","last_active":"2024-09-11T07:46:31.486752+0000","last_peered":"2024-09-11T07:46:31.486752+0000","last_clean":"2024-09-11T07:46:31.486752+0000","last_became_active":"2024-09-11T07:39:57.776576+0000","last_became_peered":"2024-09-11T07:39:57.776576+0000","last_unstale":"2024-09-11T07:46:31.486752+0000","last_undegraded":"2024-09-11T07:46:31.486752+0000","last_fullsized":"2024-09-11T07:46:31.486752+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:27.626761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:27.626761+0000","last_clean_scrub_stamp":"2024-09-11T07:44:27.626761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025844079999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"426'25","reported_seq":530,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:39.669808+0000","last_change":"2024-09-11T07:44:30.282613+0000","last_active":"2024-09-11T07:46:39.669808+0000","last_peered":"2024-09-11T07:46:39.669808+0000","last_clean":"2024-09-11T07:46:39.669808+0000","last_became_active":"2024-09-11T07:39:57.767661+0000","last_became_peered":"2024-09-11T07:39:57.767661+0000","last_unstale":"2024-09-11T07:46:39.669808+0000","last_undegraded":"2024-09-11T07:46:39.669808+0000","last_fullsized":"2024-09-11T07:46:39.669808+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"381'19","last_scrub_stamp":"2024-09-11T07:44:30.282534+0000","last_deep_scrub":"381'19","last_deep_scrub_stamp":"2024-09-11T07:44:30.282534+0000","last_clean_scrub_stamp":"2024-09-11T07:44:30.282534+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017437478999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":486,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.493233+0000","last_change":"2024-09-11T07:46:13.910021+0000","last_active":"2024-09-11T07:46:31.493233+0000","last_peered":"2024-09-11T07:46:31.493233+0000","last_clean":"2024-09-11T07:46:31.493233+0000","last_became_active":"2024-09-11T07:26:31.415701+0000","last_became_peered":"2024-09-11T07:26:31.415701+0000","last_unstale":"2024-09-11T07:46:31.493233+0000","last_undegraded":"2024-09-11T07:46:31.493233+0000","last_fullsized":"2024-09-11T07:46:31.493233+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:13.909932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:43:26.338227+0000","last_clean_scrub_stamp":"2024-09-11T07:46:13.909932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:42.316268+0000","scrub_duration":0.0075390129999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":197,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.508835+0000","last_change":"2024-09-11T07:44:29.396082+0000","last_active":"2024-09-11T07:46:31.508835+0000","last_peered":"2024-09-11T07:46:31.508835+0000","last_clean":"2024-09-11T07:46:31.508835+0000","last_became_active":"2024-09-11T07:44:23.768703+0000","last_became_peered":"2024-09-11T07:44:23.768703+0000","last_unstale":"2024-09-11T07:46:31.508835+0000","last_undegraded":"2024-09-11T07:46:31.508835+0000","last_fullsized":"2024-09-11T07:46:31.508835+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:29.395994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:35.392360+0000","last_clean_scrub_stamp":"2024-09-11T07:44:29.395994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045575399999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":493,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.494586+0000","last_change":"2024-09-11T07:45:27.129839+0000","last_active":"2024-09-11T07:46:31.494586+0000","last_peered":"2024-09-11T07:46:31.494586+0000","last_clean":"2024-09-11T07:46:31.494586+0000","last_became_active":"2024-09-11T07:45:27.124181+0000","last_became_peered":"2024-09-11T07:45:27.124181+0000","last_unstale":"2024-09-11T07:46:31.494586+0000","last_undegraded":"2024-09-11T07:46:31.494586+0000","last_fullsized":"2024-09-11T07:46:31.494586+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:45:18.615900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:45:18.615900+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005478391,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":278,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.490926+0000","last_change":"2024-09-11T07:46:05.308376+0000","last_active":"2024-09-11T07:46:31.490926+0000","last_peered":"2024-09-11T07:46:31.490926+0000","last_clean":"2024-09-11T07:46:31.490926+0000","last_became_active":"2024-09-11T07:45:27.091114+0000","last_became_peered":"2024-09-11T07:45:27.091114+0000","last_unstale":"2024-09-11T07:46:31.490926+0000","last_undegraded":"2024-09-11T07:46:31.490926+0000","last_fullsized":"2024-09-11T07:46:31.490926+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:05.308321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:46:05.308321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:10.635332+0000","scrub_duration":0.001462395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"426'21","reported_seq":120,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.534535+0000","last_change":"2024-09-11T07:45:19.660354+0000","last_active":"2024-09-11T07:46:40.534535+0000","last_peered":"2024-09-11T07:46:40.534535+0000","last_clean":"2024-09-11T07:46:40.534535+0000","last_became_active":"2024-09-11T07:44:23.769896+0000","last_became_peered":"2024-09-11T07:44:23.769896+0000","last_unstale":"2024-09-11T07:46:40.534535+0000","last_undegraded":"2024-09-11T07:46:40.534535+0000","last_fullsized":"2024-09-11T07:46:40.534535+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:45:19.660273+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:45:19.660273+0000","objects_scrubbed":3,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032335910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":23,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":471,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.493739+0000","last_change":"2024-09-11T07:46:31.493250+0000","last_active":"2024-09-11T07:46:31.493739+0000","last_peered":"2024-09-11T07:46:31.493739+0000","last_clean":"2024-09-11T07:46:31.493739+0000","last_became_active":"2024-09-11T07:46:31.492884+0000","last_became_peered":"2024-09-11T07:46:31.492884+0000","last_unstale":"2024-09-11T07:46:31.493739+0000","last_undegraded":"2024-09-11T07:46:31.493739+0000","last_fullsized":"2024-09-11T07:46:31.493739+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:24.015330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:46:24.015330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:35.180268+0000","scrub_duration":0.0069828010000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":459,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.509557+0000","last_change":"2024-09-11T07:45:27.113998+0000","last_active":"2024-09-11T07:46:31.509557+0000","last_peered":"2024-09-11T07:46:31.509557+0000","last_clean":"2024-09-11T07:46:31.509557+0000","last_became_active":"2024-09-11T07:45:27.112711+0000","last_became_peered":"2024-09-11T07:45:27.112711+0000","last_unstale":"2024-09-11T07:46:31.509557+0000","last_undegraded":"2024-09-11T07:46:31.509557+0000","last_fullsized":"2024-09-11T07:46:31.509557+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:26.403212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:44:26.403212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001222218,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"426'35","reported_seq":544,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.646568+0000","last_change":"2024-09-11T07:44:29.610618+0000","last_active":"2024-09-11T07:46:40.646568+0000","last_peered":"2024-09-11T07:46:40.646568+0000","last_clean":"2024-09-11T07:46:40.646568+0000","last_became_active":"2024-09-11T07:39:57.771009+0000","last_became_peered":"2024-09-11T07:39:57.771009+0000","last_unstale":"2024-09-11T07:46:40.646568+0000","last_undegraded":"2024-09-11T07:46:40.646568+0000","last_fullsized":"2024-09-11T07:46:40.646568+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'25","last_scrub_stamp":"2024-09-11T07:44:29.610392+0000","last_deep_scrub":"234'25","last_deep_scrub_stamp":"2024-09-11T07:44:29.610392+0000","last_clean_scrub_stamp":"2024-09-11T07:44:29.610392+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005704598,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":325,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.497749+0000","last_change":"2024-09-11T07:46:31.497548+0000","last_active":"2024-09-11T07:46:31.497749+0000","last_peered":"2024-09-11T07:46:31.497749+0000","last_clean":"2024-09-11T07:46:31.497749+0000","last_became_active":"2024-09-11T07:46:31.497105+0000","last_became_peered":"2024-09-11T07:46:31.497105+0000","last_unstale":"2024-09-11T07:46:31.497749+0000","last_undegraded":"2024-09-11T07:46:31.497749+0000","last_fullsized":"2024-09-11T07:46:31.497749+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:19.063698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:39:54.493614+0000","last_clean_scrub_stamp":"2024-09-11T07:46:19.063698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:44.292280+0000","scrub_duration":0.0049403759999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"425'9","reported_seq":36,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:35.773491+0000","last_change":"2024-09-11T07:46:31.499944+0000","last_active":"2024-09-11T07:46:35.773491+0000","last_peered":"2024-09-11T07:46:35.773491+0000","last_clean":"2024-09-11T07:46:35.773491+0000","last_became_active":"2024-09-11T07:46:31.499246+0000","last_became_peered":"2024-09-11T07:46:31.499246+0000","last_unstale":"2024-09-11T07:46:35.773491+0000","last_undegraded":"2024-09-11T07:46:35.773491+0000","last_fullsized":"2024-09-11T07:46:35.773491+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-11T07:44:13.339137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:44:13.339137+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":488,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.490089+0000","last_change":"2024-09-11T07:45:27.124686+0000","last_active":"2024-09-11T07:46:31.490089+0000","last_peered":"2024-09-11T07:46:31.490089+0000","last_clean":"2024-09-11T07:46:31.490089+0000","last_became_active":"2024-09-11T07:45:27.123325+0000","last_became_peered":"2024-09-11T07:45:27.123325+0000","last_unstale":"2024-09-11T07:46:31.490089+0000","last_undegraded":"2024-09-11T07:46:31.490089+0000","last_fullsized":"2024-09-11T07:46:31.490089+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:03.324782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:44:03.324782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059257859999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"426'25","reported_seq":480,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.660465+0000","last_change":"2024-09-11T07:44:00.397238+0000","last_active":"2024-09-11T07:46:40.660465+0000","last_peered":"2024-09-11T07:46:40.660465+0000","last_clean":"2024-09-11T07:46:40.660465+0000","last_became_active":"2024-09-11T07:43:10.448494+0000","last_became_peered":"2024-09-11T07:43:10.448494+0000","last_unstale":"2024-09-11T07:46:40.660465+0000","last_undegraded":"2024-09-11T07:46:40.660465+0000","last_fullsized":"2024-09-11T07:46:40.660465+0000","mapping_epoch":360,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":361,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:44:00.397157+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:37:41.338036+0000","last_clean_scrub_stamp":"2024-09-11T07:44:00.397157+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072446960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":470,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.491145+0000","last_change":"2024-09-11T07:45:27.117986+0000","last_active":"2024-09-11T07:46:31.491145+0000","last_peered":"2024-09-11T07:46:31.491145+0000","last_clean":"2024-09-11T07:46:31.491145+0000","last_became_active":"2024-09-11T07:45:27.114097+0000","last_became_peered":"2024-09-11T07:45:27.114097+0000","last_unstale":"2024-09-11T07:46:31.491145+0000","last_undegraded":"2024-09-11T07:46:31.491145+0000","last_fullsized":"2024-09-11T07:46:31.491145+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:09.537798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:09.537798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003422685,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"290'11","reported_seq":445,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:35.763911+0000","last_change":"2024-09-11T07:44:10.410699+0000","last_active":"2024-09-11T07:46:35.763911+0000","last_peered":"2024-09-11T07:46:35.763911+0000","last_clean":"2024-09-11T07:46:35.763911+0000","last_became_active":"2024-09-11T07:39:57.772790+0000","last_became_peered":"2024-09-11T07:39:57.772790+0000","last_unstale":"2024-09-11T07:46:35.763911+0000","last_undegraded":"2024-09-11T07:46:35.763911+0000","last_fullsized":"2024-09-11T07:46:35.763911+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"290'11","last_scrub_stamp":"2024-09-11T07:44:10.410600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:44:10.410600+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090484429999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":44,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.491947+0000","last_change":"2024-09-11T07:45:27.104143+0000","last_active":"2024-09-11T07:46:31.491947+0000","last_peered":"2024-09-11T07:46:31.491947+0000","last_clean":"2024-09-11T07:46:31.491947+0000","last_became_active":"2024-09-11T07:45:27.101913+0000","last_became_peered":"2024-09-11T07:45:27.101913+0000","last_unstale":"2024-09-11T07:46:31.491947+0000","last_undegraded":"2024-09-11T07:46:31.491947+0000","last_fullsized":"2024-09-11T07:46:31.491947+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:21.026931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:45:21.026931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"426'22","reported_seq":468,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.096625+0000","last_change":"2024-09-11T07:45:14.682735+0000","last_active":"2024-09-11T07:46:40.096625+0000","last_peered":"2024-09-11T07:46:40.096625+0000","last_clean":"2024-09-11T07:46:40.096625+0000","last_became_active":"2024-09-11T07:39:57.689424+0000","last_became_peered":"2024-09-11T07:39:57.689424+0000","last_unstale":"2024-09-11T07:46:40.096625+0000","last_undegraded":"2024-09-11T07:46:40.096625+0000","last_fullsized":"2024-09-11T07:46:40.096625+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'16","last_scrub_stamp":"2024-09-11T07:45:14.682638+0000","last_deep_scrub":"236'16","last_deep_scrub_stamp":"2024-09-11T07:36:49.809484+0000","last_clean_scrub_stamp":"2024-09-11T07:45:14.682638+0000","objects_scrubbed":4,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014584720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":263,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.484806+0000","last_change":"2024-09-11T07:45:08.577358+0000","last_active":"2024-09-11T07:46:31.484806+0000","last_peered":"2024-09-11T07:46:31.484806+0000","last_clean":"2024-09-11T07:46:31.484806+0000","last_became_active":"2024-09-11T07:39:57.776841+0000","last_became_peered":"2024-09-11T07:39:57.776841+0000","last_unstale":"2024-09-11T07:46:31.484806+0000","last_undegraded":"2024-09-11T07:46:31.484806+0000","last_fullsized":"2024-09-11T07:46:31.484806+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:08.577262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:38.721118+0000","last_clean_scrub_stamp":"2024-09-11T07:45:08.577262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0073986299999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":449,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.486874+0000","last_change":"2024-09-11T07:44:15.612662+0000","last_active":"2024-09-11T07:46:31.486874+0000","last_peered":"2024-09-11T07:46:31.486874+0000","last_clean":"2024-09-11T07:46:31.486874+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:46:31.486874+0000","last_undegraded":"2024-09-11T07:46:31.486874+0000","last_fullsized":"2024-09-11T07:46:31.486874+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:15.612580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:44:15.612580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001412515,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":471,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.487453+0000","last_change":"2024-09-11T07:46:19.170608+0000","last_active":"2024-09-11T07:46:31.487453+0000","last_peered":"2024-09-11T07:46:31.487453+0000","last_clean":"2024-09-11T07:46:31.487453+0000","last_became_active":"2024-09-11T07:39:57.761754+0000","last_became_peered":"2024-09-11T07:39:57.761754+0000","last_unstale":"2024-09-11T07:46:31.487453+0000","last_undegraded":"2024-09-11T07:46:31.487453+0000","last_fullsized":"2024-09-11T07:46:31.487453+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:19.170527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:46:19.170527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:21.133015+0000","scrub_duration":0.0099471420000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"426'32","reported_seq":293,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.590366+0000","last_change":"2024-09-11T07:46:31.495336+0000","last_active":"2024-09-11T07:46:40.590366+0000","last_peered":"2024-09-11T07:46:40.590366+0000","last_clean":"2024-09-11T07:46:40.590366+0000","last_became_active":"2024-09-11T07:46:31.494995+0000","last_became_peered":"2024-09-11T07:46:31.494995+0000","last_unstale":"2024-09-11T07:46:40.590366+0000","last_undegraded":"2024-09-11T07:46:40.590366+0000","last_fullsized":"2024-09-11T07:46:40.590366+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'24","last_scrub_stamp":"2024-09-11T07:44:32.227871+0000","last_deep_scrub":"234'24","last_deep_scrub_stamp":"2024-09-11T07:41:32.694510+0000","last_clean_scrub_stamp":"2024-09-11T07:44:32.227871+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048561150000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":36,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":18,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.501512+0000","last_change":"2024-09-11T07:46:31.500716+0000","last_active":"2024-09-11T07:46:31.501512+0000","last_peered":"2024-09-11T07:46:31.501512+0000","last_clean":"2024-09-11T07:46:31.501512+0000","last_became_active":"2024-09-11T07:46:31.500431+0000","last_became_peered":"2024-09-11T07:46:31.500431+0000","last_unstale":"2024-09-11T07:46:31.501512+0000","last_undegraded":"2024-09-11T07:46:31.501512+0000","last_fullsized":"2024-09-11T07:46:31.501512+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:09.333312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:45:09.333312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":409,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.498527+0000","last_change":"2024-09-11T07:46:31.495545+0000","last_active":"2024-09-11T07:46:31.498527+0000","last_peered":"2024-09-11T07:46:31.498527+0000","last_clean":"2024-09-11T07:46:31.498527+0000","last_became_active":"2024-09-11T07:46:31.495137+0000","last_became_peered":"2024-09-11T07:46:31.495137+0000","last_unstale":"2024-09-11T07:46:31.498527+0000","last_undegraded":"2024-09-11T07:46:31.498527+0000","last_fullsized":"2024-09-11T07:46:31.498527+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:07.272030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:39:55.895853+0000","last_clean_scrub_stamp":"2024-09-11T07:45:07.272030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011708260999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":190,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.490045+0000","last_change":"2024-09-11T07:45:24.519416+0000","last_active":"2024-09-11T07:46:31.490045+0000","last_peered":"2024-09-11T07:46:31.490045+0000","last_clean":"2024-09-11T07:46:31.490045+0000","last_became_active":"2024-09-11T07:40:11.700676+0000","last_became_peered":"2024-09-11T07:40:11.700676+0000","last_unstale":"2024-09-11T07:46:31.490045+0000","last_undegraded":"2024-09-11T07:46:31.490045+0000","last_fullsized":"2024-09-11T07:46:31.490045+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:24.519137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:45:24.519137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077895680000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"426'14","reported_seq":475,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:39.530102+0000","last_change":"2024-09-11T07:44:16.603894+0000","last_active":"2024-09-11T07:46:39.530102+0000","last_peered":"2024-09-11T07:46:39.530102+0000","last_clean":"2024-09-11T07:46:39.530102+0000","last_became_active":"2024-09-11T07:26:33.545830+0000","last_became_peered":"2024-09-11T07:26:33.545830+0000","last_unstale":"2024-09-11T07:46:39.530102+0000","last_undegraded":"2024-09-11T07:46:39.530102+0000","last_fullsized":"2024-09-11T07:46:39.530102+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'10","last_scrub_stamp":"2024-09-11T07:44:16.603800+0000","last_deep_scrub":"234'10","last_deep_scrub_stamp":"2024-09-11T07:41:29.585284+0000","last_clean_scrub_stamp":"2024-09-11T07:44:16.603800+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047267960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":16,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.504798+0000","last_change":"2024-09-11T07:46:31.503627+0000","last_active":"2024-09-11T07:46:31.504798+0000","last_peered":"2024-09-11T07:46:31.504798+0000","last_clean":"2024-09-11T07:46:31.504798+0000","last_became_active":"2024-09-11T07:46:31.503399+0000","last_became_peered":"2024-09-11T07:46:31.503399+0000","last_unstale":"2024-09-11T07:46:31.504798+0000","last_undegraded":"2024-09-11T07:46:31.504798+0000","last_fullsized":"2024-09-11T07:46:31.504798+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:17.132174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_clean_scrub_stamp":"2024-09-11T07:46:17.132174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:25.612606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":284,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.490254+0000","last_change":"2024-09-11T07:45:31.477617+0000","last_active":"2024-09-11T07:46:31.490254+0000","last_peered":"2024-09-11T07:46:31.490254+0000","last_clean":"2024-09-11T07:46:31.490254+0000","last_became_active":"2024-09-11T07:45:27.118170+0000","last_became_peered":"2024-09-11T07:45:27.118170+0000","last_unstale":"2024-09-11T07:46:31.490254+0000","last_undegraded":"2024-09-11T07:46:31.490254+0000","last_fullsized":"2024-09-11T07:46:31.490254+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:31.477552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:31.477552+0000","last_clean_scrub_stamp":"2024-09-11T07:45:31.477552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:50.533989+0000","scrub_duration":0.001479164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":461,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.508208+0000","last_change":"2024-09-11T07:45:27.098514+0000","last_active":"2024-09-11T07:46:31.508208+0000","last_peered":"2024-09-11T07:46:31.508208+0000","last_clean":"2024-09-11T07:46:31.508208+0000","last_became_active":"2024-09-11T07:45:27.097259+0000","last_became_peered":"2024-09-11T07:45:27.097259+0000","last_unstale":"2024-09-11T07:46:31.508208+0000","last_undegraded":"2024-09-11T07:46:31.508208+0000","last_fullsized":"2024-09-11T07:46:31.508208+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:13.460946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:44:13.460946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014486880000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"426'27","reported_seq":517,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.763591+0000","last_change":"2024-09-11T07:44:03.649070+0000","last_active":"2024-09-11T07:46:40.763591+0000","last_peered":"2024-09-11T07:46:40.763591+0000","last_clean":"2024-09-11T07:46:40.763591+0000","last_became_active":"2024-09-11T07:39:57.767928+0000","last_became_peered":"2024-09-11T07:39:57.767928+0000","last_unstale":"2024-09-11T07:46:40.763591+0000","last_undegraded":"2024-09-11T07:46:40.763591+0000","last_fullsized":"2024-09-11T07:46:40.763591+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:44:03.648975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:44:03.648975+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003981989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.504809+0000","last_change":"2024-09-11T07:46:31.504241+0000","last_active":"2024-09-11T07:46:31.504809+0000","last_peered":"2024-09-11T07:46:31.504809+0000","last_clean":"2024-09-11T07:46:31.504809+0000","last_became_active":"2024-09-11T07:46:31.503969+0000","last_became_peered":"2024-09-11T07:46:31.503969+0000","last_unstale":"2024-09-11T07:46:31.504809+0000","last_undegraded":"2024-09-11T07:46:31.504809+0000","last_fullsized":"2024-09-11T07:46:31.504809+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:11.315570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:48.366931+0000","last_clean_scrub_stamp":"2024-09-11T07:44:11.315570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":434,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.507855+0000","last_change":"2024-09-11T07:45:27.101549+0000","last_active":"2024-09-11T07:46:31.507855+0000","last_peered":"2024-09-11T07:46:31.507855+0000","last_clean":"2024-09-11T07:46:31.507855+0000","last_became_active":"2024-09-11T07:45:27.101251+0000","last_became_peered":"2024-09-11T07:45:27.101251+0000","last_unstale":"2024-09-11T07:46:31.507855+0000","last_undegraded":"2024-09-11T07:46:31.507855+0000","last_fullsized":"2024-09-11T07:46:31.507855+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:20.479924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:39.021876+0000","last_clean_scrub_stamp":"2024-09-11T07:44:20.479924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001690261,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":444,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.486586+0000","last_change":"2024-09-11T07:44:17.592717+0000","last_active":"2024-09-11T07:46:31.486586+0000","last_peered":"2024-09-11T07:46:31.486586+0000","last_clean":"2024-09-11T07:46:31.486586+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:46:31.486586+0000","last_undegraded":"2024-09-11T07:46:31.486586+0000","last_fullsized":"2024-09-11T07:46:31.486586+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:17.592633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:32.904006+0000","last_clean_scrub_stamp":"2024-09-11T07:44:17.592633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044439140000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":431,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.508706+0000","last_change":"2024-09-11T07:46:08.795262+0000","last_active":"2024-09-11T07:46:31.508706+0000","last_peered":"2024-09-11T07:46:31.508706+0000","last_clean":"2024-09-11T07:46:31.508706+0000","last_became_active":"2024-09-11T07:32:00.511258+0000","last_became_peered":"2024-09-11T07:32:00.511258+0000","last_unstale":"2024-09-11T07:46:31.508706+0000","last_undegraded":"2024-09-11T07:46:31.508706+0000","last_fullsized":"2024-09-11T07:46:31.508706+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:46:08.795209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:46:08.795209+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:22.767524+0000","scrub_duration":0.0014785359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":477,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.504780+0000","last_change":"2024-09-11T07:46:31.503438+0000","last_active":"2024-09-11T07:46:31.504780+0000","last_peered":"2024-09-11T07:46:31.504780+0000","last_clean":"2024-09-11T07:46:31.504780+0000","last_became_active":"2024-09-11T07:46:31.503050+0000","last_became_peered":"2024-09-11T07:46:31.503050+0000","last_unstale":"2024-09-11T07:46:31.504780+0000","last_undegraded":"2024-09-11T07:46:31.504780+0000","last_fullsized":"2024-09-11T07:46:31.504780+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:44:22.300957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.300957+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0068790429999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"426'30","reported_seq":83,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.750967+0000","last_change":"2024-09-11T07:45:30.511577+0000","last_active":"2024-09-11T07:46:40.750967+0000","last_peered":"2024-09-11T07:46:40.750967+0000","last_clean":"2024-09-11T07:46:40.750967+0000","last_became_active":"2024-09-11T07:45:27.122864+0000","last_became_peered":"2024-09-11T07:45:27.122864+0000","last_unstale":"2024-09-11T07:46:40.750967+0000","last_undegraded":"2024-09-11T07:46:40.750967+0000","last_fullsized":"2024-09-11T07:46:40.750967+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'20","last_scrub_stamp":"2024-09-11T07:45:30.511477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:45:30.511477+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:43.153986+0000","scrub_duration":0.001786167,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"395'13","reported_seq":232,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:35.759647+0000","last_change":"2024-09-11T07:45:31.645251+0000","last_active":"2024-09-11T07:46:35.759647+0000","last_peered":"2024-09-11T07:46:35.759647+0000","last_clean":"2024-09-11T07:46:35.759647+0000","last_became_active":"2024-09-11T07:45:27.105351+0000","last_became_peered":"2024-09-11T07:45:27.105351+0000","last_unstale":"2024-09-11T07:46:35.759647+0000","last_undegraded":"2024-09-11T07:46:35.759647+0000","last_fullsized":"2024-09-11T07:46:35.759647+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"395'13","last_scrub_stamp":"2024-09-11T07:45:31.645192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:45:31.645192+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:49.609737+0000","scrub_duration":0.0016838580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3],"acting":[2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":53,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.502553+0000","last_change":"2024-09-11T07:46:31.502229+0000","last_active":"2024-09-11T07:46:31.502553+0000","last_peered":"2024-09-11T07:46:31.502553+0000","last_clean":"2024-09-11T07:46:31.502553+0000","last_became_active":"2024-09-11T07:46:31.501945+0000","last_became_peered":"2024-09-11T07:46:31.501945+0000","last_unstale":"2024-09-11T07:46:31.502553+0000","last_undegraded":"2024-09-11T07:46:31.502553+0000","last_fullsized":"2024-09-11T07:46:31.502553+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:28.236557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:53.875300+0000","last_clean_scrub_stamp":"2024-09-11T07:44:28.236557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001550165,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":460,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.484170+0000","last_change":"2024-09-11T07:44:01.714689+0000","last_active":"2024-09-11T07:46:31.484170+0000","last_peered":"2024-09-11T07:46:31.484170+0000","last_clean":"2024-09-11T07:46:31.484170+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:46:31.484170+0000","last_undegraded":"2024-09-11T07:46:31.484170+0000","last_fullsized":"2024-09-11T07:46:31.484170+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:01.714607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:44:01.714607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079815600000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"426'27","reported_seq":39,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.771392+0000","last_change":"2024-09-11T07:46:13.812491+0000","last_active":"2024-09-11T07:46:40.771392+0000","last_peered":"2024-09-11T07:46:40.771392+0000","last_clean":"2024-09-11T07:46:40.771392+0000","last_became_active":"2024-09-11T07:46:13.790166+0000","last_became_peered":"2024-09-11T07:46:13.790166+0000","last_unstale":"2024-09-11T07:46:40.771392+0000","last_undegraded":"2024-09-11T07:46:40.771392+0000","last_fullsized":"2024-09-11T07:46:40.771392+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:45:23.537025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:45:23.537025+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033255630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":470,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.490256+0000","last_change":"2024-09-11T07:46:15.189821+0000","last_active":"2024-09-11T07:46:31.490256+0000","last_peered":"2024-09-11T07:46:31.490256+0000","last_clean":"2024-09-11T07:46:31.490256+0000","last_became_active":"2024-09-11T07:32:00.507050+0000","last_became_peered":"2024-09-11T07:32:00.507050+0000","last_unstale":"2024-09-11T07:46:31.490256+0000","last_undegraded":"2024-09-11T07:46:31.490256+0000","last_fullsized":"2024-09-11T07:46:31.490256+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:15.189711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:24.575210+0000","last_clean_scrub_stamp":"2024-09-11T07:46:15.189711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:18.958013+0000","scrub_duration":0.0066152470000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":51,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.484400+0000","last_change":"2024-09-11T07:46:05.410699+0000","last_active":"2024-09-11T07:46:31.484400+0000","last_peered":"2024-09-11T07:46:31.484400+0000","last_clean":"2024-09-11T07:46:31.484400+0000","last_became_active":"2024-09-11T07:45:27.100903+0000","last_became_peered":"2024-09-11T07:45:27.100903+0000","last_unstale":"2024-09-11T07:46:31.484400+0000","last_undegraded":"2024-09-11T07:46:31.484400+0000","last_fullsized":"2024-09-11T07:46:31.484400+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:05.410607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:46:05.410607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:16.917476+0000","scrub_duration":0.0016366040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":196,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.508325+0000","last_change":"2024-09-11T07:46:03.709201+0000","last_active":"2024-09-11T07:46:31.508325+0000","last_peered":"2024-09-11T07:46:31.508325+0000","last_clean":"2024-09-11T07:46:31.508325+0000","last_became_active":"2024-09-11T07:45:27.096591+0000","last_became_peered":"2024-09-11T07:45:27.096591+0000","last_unstale":"2024-09-11T07:46:31.508325+0000","last_undegraded":"2024-09-11T07:46:31.508325+0000","last_fullsized":"2024-09-11T07:46:31.508325+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:03.709095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:46:03.709095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:11.467089+0000","scrub_duration":0.001464166,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1c","version":"426'39","reported_seq":532,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.392029+0000","last_change":"2024-09-11T07:45:13.715138+0000","last_active":"2024-09-11T07:46:40.392029+0000","last_peered":"2024-09-11T07:46:40.392029+0000","last_clean":"2024-09-11T07:46:40.392029+0000","last_became_active":"2024-09-11T07:26:33.542544+0000","last_became_peered":"2024-09-11T07:26:33.542544+0000","last_unstale":"2024-09-11T07:46:40.392029+0000","last_undegraded":"2024-09-11T07:46:40.392029+0000","last_fullsized":"2024-09-11T07:46:40.392029+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'27","last_scrub_stamp":"2024-09-11T07:45:13.715050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:45:13.715050+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047085019999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":58,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.489072+0000","last_change":"2024-09-11T07:45:34.587277+0000","last_active":"2024-09-11T07:46:31.489072+0000","last_peered":"2024-09-11T07:46:31.489072+0000","last_clean":"2024-09-11T07:46:31.489072+0000","last_became_active":"2024-09-11T07:45:27.119977+0000","last_became_peered":"2024-09-11T07:45:27.119977+0000","last_unstale":"2024-09-11T07:46:31.489072+0000","last_undegraded":"2024-09-11T07:46:31.489072+0000","last_fullsized":"2024-09-11T07:46:31.489072+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:34.587193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:32.456852+0000","last_clean_scrub_stamp":"2024-09-11T07:45:34.587193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:01.619800+0000","scrub_duration":0.0049811509999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":451,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.484545+0000","last_change":"2024-09-11T07:44:00.677726+0000","last_active":"2024-09-11T07:46:31.484545+0000","last_peered":"2024-09-11T07:46:31.484545+0000","last_clean":"2024-09-11T07:46:31.484545+0000","last_became_active":"2024-09-11T07:39:57.768994+0000","last_became_peered":"2024-09-11T07:39:57.768994+0000","last_unstale":"2024-09-11T07:46:31.484545+0000","last_undegraded":"2024-09-11T07:46:31.484545+0000","last_fullsized":"2024-09-11T07:46:31.484545+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:00.677444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:44:00.677444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030734109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":494,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.486993+0000","last_change":"2024-09-11T07:46:22.481200+0000","last_active":"2024-09-11T07:46:31.486993+0000","last_peered":"2024-09-11T07:46:31.486993+0000","last_clean":"2024-09-11T07:46:31.486993+0000","last_became_active":"2024-09-11T07:45:27.101964+0000","last_became_peered":"2024-09-11T07:45:27.101964+0000","last_unstale":"2024-09-11T07:46:31.486993+0000","last_undegraded":"2024-09-11T07:46:31.486993+0000","last_fullsized":"2024-09-11T07:46:31.486993+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:22.481110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:46:22.481110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:44.619663+0000","scrub_duration":0.005622022,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"426'29","reported_seq":523,"reported_epoch":426,"state":"active+clean","last_fresh":"2024-09-11T07:46:40.811188+0000","last_change":"2024-09-11T07:45:27.139564+0000","last_active":"2024-09-11T07:46:40.811188+0000","last_peered":"2024-09-11T07:46:40.811188+0000","last_clean":"2024-09-11T07:46:40.811188+0000","last_became_active":"2024-09-11T07:45:27.095974+0000","last_became_peered":"2024-09-11T07:45:27.095974+0000","last_unstale":"2024-09-11T07:46:40.811188+0000","last_undegraded":"2024-09-11T07:46:40.811188+0000","last_fullsized":"2024-09-11T07:46:40.811188+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'23","last_scrub_stamp":"2024-09-11T07:45:24.422401+0000","last_deep_scrub":"234'23","last_deep_scrub_stamp":"2024-09-11T07:43:25.530826+0000","last_clean_scrub_stamp":"2024-09-11T07:45:24.422401+0000","objects_scrubbed":5,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:50.733770+0000","scrub_duration":0.0068140780000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":44,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-11T07:46:31.484097+0000","last_change":"2024-09-11T07:45:27.104898+0000","last_active":"2024-09-11T07:46:31.484097+0000","last_peered":"2024-09-11T07:46:31.484097+0000","last_clean":"2024-09-11T07:46:31.484097+0000","last_became_active":"2024-09-11T07:45:27.104493+0000","last_became_peered":"2024-09-11T07:45:27.104493+0000","last_unstale":"2024-09-11T07:46:31.484097+0000","last_undegraded":"2024-09-11T07:46:31.484097+0000","last_fullsized":"2024-09-11T07:46:31.484097+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:17.114312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:44:17.114312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":3620732928,"num_objects":55248,"num_object_clones":0,"num_object_copies":165744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55248,"num_write_kb":3535872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10824384512,"data_stored":10824384512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55248,"ondisk_log_size":55248,"up":48,"acting":48,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84,"ondisk_log_size":84,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1535,"num_read_kb":1292,"num_write":989,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":3510,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":475136,"data_stored":12690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1069,"ondisk_log_size":1069,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":147,"num_read_kb":142,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":429359,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":91,"num_read_kb":168,"num_write":81,"num_write_kb":1184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1723095,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2109440,"data_stored":2098784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":891,"ondisk_log_size":891,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":147,"seq":631360192690,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2450992,"kb_used_data":1531624,"kb_used_omap":4,"kb_used_meta":919355,"kb_avail":91294160,"statfs":{"total":95995035648,"available":93485219840,"internally_reserved":0,"allocated":1568382976,"data_stored":1567810055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":941420038},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":1.4370000000000001,"15min":1.851},"min":{"1min":0.22800000000000001,"5min":0.21099999999999999,"15min":0.191},"max":{"1min":1.5940000000000001,"5min":20.832999999999998,"15min":75.677000000000007},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.3380000000000001,"15min":1.899},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":4.9710000000000001,"5min":20.937000000000001,"15min":75.638999999999996},"last":1.4299999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:46:36 2024","interfaces":[{"interface":"back","average":{"1min":18.690999999999999,"5min":4.3159999999999998,"15min":2.3599999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.22500000000000001},"max":{"1min":327.06099999999998,"5min":327.06099999999998,"15min":327.06099999999998},"last":0.27000000000000002},{"interface":"front","average":{"1min":18.693999999999999,"5min":4.3070000000000004,"15min":2.3860000000000001},"min":{"1min":0.313,"5min":0.26300000000000001,"15min":0.22800000000000001},"max":{"1min":327.07499999999999,"5min":327.07499999999999,"15min":327.07499999999999},"last":0.41499999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.79000000000000004,"15min":1.792},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.22800000000000001},"max":{"1min":0.95599999999999996,"5min":6.4820000000000002,"15min":227.79300000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.78100000000000003,"15min":1.8120000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.246},"max":{"1min":1.1839999999999999,"5min":6.6600000000000001,"15min":227.22},"last":0.48299999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.73799999999999999,"15min":1.3740000000000001},"min":{"1min":0.33200000000000002,"5min":0.252,"15min":0.252},"max":{"1min":1.7450000000000001,"5min":3.399,"15min":49.798999999999999},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.79800000000000004,"15min":1.4279999999999999},"min":{"1min":0.33400000000000002,"5min":0.29499999999999998,"15min":0.27200000000000002},"max":{"1min":2.097,"5min":6,"15min":49.524000000000001},"last":0.36799999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.67500000000000004,"15min":0.70099999999999996},"min":{"1min":0.30199999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.71599999999999997,"5min":3.8319999999999999,"15min":6.0220000000000002},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.70799999999999996,"15min":0.74399999999999999},"min":{"1min":0.28499999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.65000000000000002,"5min":3.3170000000000002,"15min":6.9000000000000004},"last":0.32800000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.69599999999999995,"15min":0.78200000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.25},"max":{"1min":0.67800000000000005,"5min":9.9339999999999993,"15min":9.9339999999999993},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.61699999999999999,"15min":0.76800000000000002},"min":{"1min":0.312,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.59699999999999998,"5min":10.183999999999999,"15min":10.183999999999999},"last":0.49099999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.57599999999999996,"15min":0.66300000000000003},"min":{"1min":0.316,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.302,"5min":2.1190000000000002,"15min":5.1909999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.65600000000000003,"15min":0.73999999999999999},"min":{"1min":0.33400000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.341,"5min":2.6970000000000001,"15min":4.2030000000000003},"last":0.56100000000000005}]}]},{"osd":6,"up_from":44,"seq":188978561275,"num_pgs":86,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3099020,"kb_used_data":2197572,"kb_used_omap":1,"kb_used_meta":901438,"kb_avail":90646132,"statfs":{"total":95995035648,"available":92821639168,"internally_reserved":0,"allocated":2250313728,"data_stored":2249713923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":923072856},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:46:29 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":3.238,"15min":2.399},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.192},"max":{"1min":2.4449999999999998,"5min":130.62299999999999,"15min":142.29599999999999},"last":1.155},{"interface":"front","average":{"1min":0.54500000000000004,"5min":3.2309999999999999,"15min":2.5310000000000001},"min":{"1min":0.29699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.91400000000000003,"5min":130.86199999999999,"15min":142.68799999999999},"last":0.36499999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:46:29 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":3,"15min":2.3860000000000001},"min":{"1min":0.33900000000000002,"5min":0.28000000000000003,"15min":0.23599999999999999},"max":{"1min":0.81999999999999995,"5min":131.42500000000001,"15min":131.42500000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.46100000000000002,"5min":2.9849999999999999,"15min":2.2709999999999999},"min":{"1min":0.28299999999999997,"5min":0.22,"15min":0.22},"max":{"1min":0.628,"5min":132.39599999999999,"15min":132.39599999999999},"last":0.32300000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:46:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":2.1549999999999998,"15min":1.3240000000000001},"min":{"1min":0.26800000000000002,"5min":0.26700000000000002,"15min":0.24299999999999999},"max":{"1min":1.109,"5min":83.915999999999997,"15min":83.915999999999997},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":2.1949999999999998,"15min":1.4810000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.80600000000000005,"5min":84.097999999999999,"15min":84.097999999999999},"last":0.47799999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:46:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":1.226,"15min":1.5149999999999999},"min":{"1min":0.309,"5min":0.30099999999999999,"15min":0.246},"max":{"1min":1.554,"5min":39.308999999999997,"15min":49.109000000000002},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":1.2989999999999999,"15min":1.5840000000000001},"min":{"1min":0.28299999999999997,"5min":0.246,"15min":0.246},"max":{"1min":1.726,"5min":41.149999999999999,"15min":48.963999999999999},"last":0.48799999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:46:29 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":1.1599999999999999,"15min":1.764},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.223},"max":{"1min":0.66700000000000004,"5min":12.257999999999999,"15min":181.78800000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":1.0649999999999999,"15min":1.1799999999999999},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.61499999999999999,"5min":12.102,"15min":128.59299999999999},"last":0.38400000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:46:29 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.749,"15min":1.008},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.57799999999999996,"5min":7.4219999999999997,"15min":57.436},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.85099999999999998,"15min":1.0589999999999999},"min":{"1min":0.32000000000000001,"5min":0.254,"15min":0.251},"max":{"1min":0.629,"5min":7.4420000000000002,"15min":56.822000000000003},"last":0.60599999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:46:29 2024","interfaces":[{"interface":"back","average":{"1min":3.6680000000000001,"5min":1.6279999999999999,"15min":1.1020000000000001},"min":{"1min":0.32700000000000001,"5min":0.32300000000000001,"15min":0.27800000000000002},"max":{"1min":60.279000000000003,"5min":60.279000000000003,"15min":60.279000000000003},"last":0.63200000000000001},{"interface":"front","average":{"1min":3.6499999999999999,"5min":1.466,"15min":1.042},"min":{"1min":0.311,"5min":0.31,"15min":0.215},"max":{"1min":60.767000000000003,"5min":60.767000000000003,"15min":60.767000000000003},"last":0.50800000000000001}]}]},{"osd":5,"up_from":37,"seq":158913790209,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1324080,"kb_used_data":440360,"kb_used_omap":1,"kb_used_meta":883710,"kb_avail":92421072,"statfs":{"total":95995035648,"available":94639177728,"internally_reserved":0,"allocated":450928640,"data_stored":450356284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":904919384},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.98699999999999999,"15min":1.7},"min":{"1min":0.28999999999999998,"5min":0.23100000000000001,"15min":0.20200000000000001},"max":{"1min":2.3069999999999999,"5min":20.391999999999999,"15min":118.086},"last":1.8620000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":1.1240000000000001,"15min":1.863},"min":{"1min":0.26400000000000001,"5min":0.20899999999999999,"15min":0.20200000000000001},"max":{"1min":5.6539999999999999,"5min":20.001000000000001,"15min":118.004},"last":2.7080000000000002}]},{"osd":1,"last update":"Wed Sep 11 07:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.73999999999999999,"15min":1.5980000000000001},"min":{"1min":0.23599999999999999,"5min":0.23300000000000001,"15min":0.20799999999999999},"max":{"1min":1.4610000000000001,"5min":7.2510000000000003,"15min":122.738},"last":1.7609999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.82799999999999996,"15min":1.657},"min":{"1min":0.30499999999999999,"5min":0.26400000000000001,"15min":0.22700000000000001},"max":{"1min":2.5619999999999998,"5min":3.6110000000000002,"15min":122.66200000000001},"last":2.7839999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":1.1799999999999999,"15min":1.1240000000000001},"min":{"1min":0.26200000000000001,"5min":0.25600000000000001,"15min":0.22700000000000001},"max":{"1min":1.389,"5min":20.718,"15min":20.718},"last":1.895},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.99399999999999999,"15min":1.0609999999999999},"min":{"1min":0.34999999999999998,"5min":0.28000000000000003,"15min":0.24399999999999999},"max":{"1min":1.2849999999999999,"5min":19.988,"15min":19.988},"last":0.247}]},{"osd":3,"last update":"Wed Sep 11 07:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":2.835,"15min":2.8109999999999999},"min":{"1min":0.34300000000000003,"5min":0.28899999999999998,"15min":0.26400000000000001},"max":{"1min":0.89300000000000002,"5min":132.715,"15min":132.715},"last":3.1269999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":2.7360000000000002,"15min":2.8820000000000001},"min":{"1min":0.374,"5min":0.33400000000000002,"15min":0.25700000000000001},"max":{"1min":1.0620000000000001,"5min":133.01599999999999,"15min":133.01599999999999},"last":2.9710000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.76700000000000002,"15min":0.77700000000000002},"min":{"1min":0.33000000000000002,"5min":0.246,"15min":0.19900000000000001},"max":{"1min":3.2130000000000001,"5min":4.5090000000000003,"15min":11.103},"last":0.313},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.70199999999999996,"15min":0.78700000000000003},"min":{"1min":0.30499999999999999,"5min":0.27300000000000002,"15min":0.22500000000000001},"max":{"1min":1.4350000000000001,"5min":3.7959999999999998,"15min":10.305},"last":0.30299999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:46:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.57799999999999996,"15min":0.72999999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79800000000000004,"5min":2.4100000000000001,"15min":5.6859999999999999},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.61599999999999999,"15min":0.73799999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.26800000000000002},"max":{"1min":0.92000000000000004,"5min":3.0990000000000002,"15min":10.613},"last":0.34699999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:46:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":1.032,"15min":1.1080000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.28000000000000003},"max":{"1min":1.5600000000000001,"5min":37.857999999999997,"15min":50.338999999999999},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.98599999999999999,"15min":1.0660000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.26800000000000002},"max":{"1min":1.262,"5min":37.759999999999998,"15min":50.298999999999999},"last":0.33900000000000002}]}]},{"osd":4,"up_from":0,"seq":133143986437,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":25,"seq":107374182666,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1790652,"kb_used_data":887604,"kb_used_omap":1,"kb_used_meta":903038,"kb_avail":91954500,"statfs":{"total":95995035648,"available":94161408000,"internally_reserved":0,"allocated":908906496,"data_stored":908321548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":924711256},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:46:36 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":1.2370000000000001,"15min":1.597},"min":{"1min":0.23300000000000001,"5min":0.19800000000000001,"15min":0.16600000000000001},"max":{"1min":1.71,"5min":27.655999999999999,"15min":38.432000000000002},"last":3.7349999999999999},{"interface":"front","average":{"1min":0.626,"5min":1.264,"15min":1.9930000000000001},"min":{"1min":0.28999999999999998,"5min":0.183,"15min":0.183},"max":{"1min":2.206,"5min":26.759,"15min":91.876999999999995},"last":2.5}]},{"osd":1,"last update":"Wed Sep 11 07:46:36 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.85499999999999998,"15min":2.6960000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.19500000000000001},"max":{"1min":2.2349999999999999,"5min":19.753,"15min":135.41999999999999},"last":2.6669999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.93000000000000005,"15min":2.403},"min":{"1min":0.251,"5min":0.221,"15min":0.192},"max":{"1min":3.6970000000000001,"5min":19.484999999999999,"15min":136.126},"last":0.88900000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:46:36 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":2.7690000000000001,"15min":1.821},"min":{"1min":0.30599999999999999,"5min":0.253,"15min":0.23699999999999999},"max":{"1min":0.65000000000000002,"5min":164.02600000000001,"15min":164.02600000000001},"last":3.359},{"interface":"front","average":{"1min":0.71499999999999997,"5min":2.556,"15min":1.6919999999999999},"min":{"1min":0.26800000000000002,"5min":0.24199999999999999,"15min":0.222},"max":{"1min":3.5430000000000001,"5min":163.37799999999999,"15min":163.37799999999999},"last":3.7690000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:45:53 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.155,"15min":1.8280000000000001},"min":{"1min":0.438,"5min":0.314,"15min":0.27500000000000002},"max":{"1min":3.48,"5min":8.6479999999999997,"15min":132.76400000000001},"last":0.873},{"interface":"front","average":{"1min":1.038,"5min":1.1539999999999999,"15min":1.911},"min":{"1min":0.36499999999999999,"5min":0.317,"15min":0.255},"max":{"1min":3.907,"5min":21.965,"15min":132.154},"last":2.532}]},{"osd":5,"last update":"Wed Sep 11 07:46:03 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76500000000000001,"15min":1.621},"min":{"1min":0.39300000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.8180000000000001,"5min":3.8889999999999998,"15min":137.50299999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.80500000000000005,"15min":1.637},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.28299999999999997},"max":{"1min":3.7639999999999998,"5min":3.7639999999999998,"15min":137.45599999999999},"last":2.6120000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:46:36 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":3.6040000000000001,"15min":2.056},"min":{"1min":0.38400000000000001,"5min":0.33700000000000002,"15min":0.27000000000000002},"max":{"1min":0.753,"5min":233.98500000000001,"15min":233.98500000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":3.6720000000000002,"15min":2.073},"min":{"1min":0.45100000000000001,"5min":0.35599999999999998,"15min":0.30399999999999999},"max":{"1min":1.7889999999999999,"5min":234.05199999999999,"15min":234.05199999999999},"last":1.018}]},{"osd":7,"last update":"Wed Sep 11 07:46:22 2024","interfaces":[{"interface":"back","average":{"1min":3.347,"5min":3.1949999999999998,"15min":2.6930000000000001},"min":{"1min":0.46300000000000002,"5min":0.39400000000000002,"15min":0.308},"max":{"1min":50.542000000000002,"5min":162.42699999999999,"15min":162.42699999999999},"last":2.6440000000000001},{"interface":"front","average":{"1min":3.419,"5min":3.1840000000000002,"15min":2.8159999999999998},"min":{"1min":0.379,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":51.555,"5min":162.39099999999999,"15min":162.39099999999999},"last":0.999}]}]},{"osd":2,"up_from":18,"seq":77309411599,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3082364,"kb_used_data":2177460,"kb_used_omap":1,"kb_used_meta":904894,"kb_avail":90662788,"statfs":{"total":95995035648,"available":92838694912,"internally_reserved":0,"allocated":2229719040,"data_stored":2229135388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":926611800},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:46:08 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":1.26,"15min":1.9359999999999999},"min":{"1min":0.23499999999999999,"5min":0.188,"15min":0.16400000000000001},"max":{"1min":3.802,"5min":19.907,"15min":57.802999999999997},"last":1.204},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.391,"15min":2.0649999999999999},"min":{"1min":0.23799999999999999,"5min":0.188,"15min":0.188},"max":{"1min":9.5090000000000003,"5min":16.039999999999999,"15min":57.759},"last":3.6819999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:46:08 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.79200000000000004,"15min":1.5680000000000001},"min":{"1min":0.23300000000000001,"5min":0.22900000000000001,"15min":0.22800000000000001},"max":{"1min":3.6030000000000002,"5min":5.3570000000000002,"15min":65.153999999999996},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.79900000000000004,"15min":1.764},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":3.625,"5min":5.1200000000000001,"15min":63.311999999999998},"last":0.65000000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:46:42 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.999,"15min":1.802},"min":{"1min":0.28399999999999997,"5min":0.22500000000000001,"15min":0.224},"max":{"1min":2.9239999999999999,"5min":5.2149999999999999,"15min":67.552999999999997},"last":0.30299999999999999},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.1080000000000001,"15min":1.8009999999999999},"min":{"1min":0.309,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":4.4779999999999998,"5min":9.4670000000000005,"15min":67.352999999999994},"last":0.52900000000000003}]},{"osd":4,"last update":"Wed Sep 11 07:45:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.099,"15min":1.0860000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.6629999999999998,"5min":7.1719999999999997,"15min":25.643000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":1,"5min":1.0069999999999999,"15min":1.0449999999999999},"min":{"1min":0.38,"5min":0.36599999999999999,"15min":0.29999999999999999},"max":{"1min":3.702,"5min":4.0510000000000002,"15min":22.434000000000001},"last":0.41399999999999998}]},{"osd":5,"last update":"Wed Sep 11 07:46:08 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.88600000000000001,"15min":1.05},"min":{"1min":0.371,"5min":0.371,"15min":0.32100000000000001},"max":{"1min":3.8340000000000001,"5min":3.8340000000000001,"15min":22.460000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.94799999999999995,"15min":1.1220000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.31},"max":{"1min":3.7130000000000001,"5min":4.6029999999999998,"15min":25.696999999999999},"last":0.56999999999999995}]},{"osd":6,"last update":"Wed Sep 11 07:46:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":1.0760000000000001,"15min":1.139},"min":{"1min":0.373,"5min":0.373,"15min":0.33400000000000002},"max":{"1min":2.4580000000000002,"5min":4.298,"15min":22.399000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.1659999999999999,"15min":1.1879999999999999},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.29999999999999999},"max":{"1min":4.4119999999999999,"5min":5.1639999999999997,"15min":25.739000000000001},"last":0.61199999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:46:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.95199999999999996,"15min":1.1200000000000001},"min":{"1min":0.40600000000000003,"5min":0.38400000000000001,"15min":0.34300000000000003},"max":{"1min":1.5740000000000001,"5min":4.6950000000000003,"15min":22.484000000000002},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.0900000000000001,"5min":0.95199999999999996,"15min":1.2789999999999999},"min":{"1min":0.44500000000000001,"5min":0.373,"15min":0.36699999999999999},"max":{"1min":3.7789999999999999,"5min":4.5220000000000002,"15min":25.670000000000002},"last":0.626}]}]},{"osd":1,"up_from":13,"seq":55834575122,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2202668,"kb_used_data":1294180,"kb_used_omap":1,"kb_used_meta":908478,"kb_avail":91542484,"statfs":{"total":95995035648,"available":93739503616,"internally_reserved":0,"allocated":1325240320,"data_stored":1324671495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":930281816},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:46:34 2024","interfaces":[{"interface":"back","average":{"1min":1.5900000000000001,"5min":1.349,"15min":1.8200000000000001},"min":{"1min":0.23499999999999999,"5min":0.20799999999999999,"15min":0.16900000000000001},"max":{"1min":18.754000000000001,"5min":24.561,"15min":82.218999999999994},"last":2.7389999999999999},{"interface":"front","average":{"1min":1.956,"5min":1.415,"15min":1.9350000000000001},"min":{"1min":0.246,"5min":0.223,"15min":0.186},"max":{"1min":19.445,"5min":24.343,"15min":82.278999999999996},"last":3.2829999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:46:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":2.0499999999999998,"15min":1.8919999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.19800000000000001},"max":{"1min":3.4199999999999999,"5min":71.025999999999996,"15min":71.025999999999996},"last":2.4340000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":1.994,"15min":1.8859999999999999},"min":{"1min":0.30099999999999999,"5min":0.20499999999999999,"15min":0.191},"max":{"1min":1.883,"5min":71.238,"15min":71.238},"last":2.6920000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:46:28 2024","interfaces":[{"interface":"back","average":{"1min":1.712,"5min":1.0529999999999999,"15min":2.4609999999999999},"min":{"1min":0.25600000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":18.815999999999999,"5min":18.815999999999999,"15min":90.766999999999996},"last":2.6619999999999999},{"interface":"front","average":{"1min":1.448,"5min":0.88,"15min":2.3639999999999999},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":18.338000000000001,"5min":18.338000000000001,"15min":91.513000000000005},"last":1.119}]},{"osd":4,"last update":"Wed Sep 11 07:45:44 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.129,"15min":1.395},"min":{"1min":0.311,"5min":0.30299999999999999,"15min":0.27400000000000002},"max":{"1min":3.6589999999999998,"5min":6.9580000000000002,"15min":7.8390000000000004},"last":1.3380000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":1.143,"15min":1.4570000000000001},"min":{"1min":0.33700000000000002,"5min":0.30099999999999999,"15min":0.26500000000000001},"max":{"1min":1.8919999999999999,"5min":7.1280000000000001,"15min":9.2289999999999992},"last":2.5649999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:46:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.78500000000000003,"15min":1.3600000000000001},"min":{"1min":0.33600000000000002,"5min":0.27500000000000002,"15min":0.26600000000000001},"max":{"1min":3.9199999999999999,"5min":4.5330000000000004,"15min":8.3680000000000003},"last":1.371},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.85299999999999998,"15min":1.3999999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.27200000000000002},"max":{"1min":1.4490000000000001,"5min":6.1189999999999998,"15min":9.1929999999999996},"last":2.6280000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:46:28 2024","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":0.98899999999999999,"15min":1.341},"min":{"1min":0.40600000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":13.981999999999999,"5min":13.981999999999999,"15min":13.981999999999999},"last":2.601},{"interface":"front","average":{"1min":1.3660000000000001,"5min":0.91400000000000003,"15min":1.1180000000000001},"min":{"1min":0.34399999999999997,"5min":0.32700000000000001,"15min":0.30499999999999999},"max":{"1min":13.928000000000001,"5min":13.928000000000001,"15min":13.928000000000001},"last":0.60899999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:46:28 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.73899999999999999,"15min":1.2410000000000001},"min":{"1min":0.379,"5min":0.36499999999999999,"15min":0.32200000000000001},"max":{"1min":1.5660000000000001,"5min":3.2690000000000001,"15min":9.3629999999999995},"last":1.089},{"interface":"front","average":{"1min":1.7849999999999999,"5min":1.081,"15min":1.53},"min":{"1min":0.372,"5min":0.372,"15min":0.313},"max":{"1min":13.715,"5min":13.715,"15min":13.715},"last":1.3959999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738648,"num_pgs":81,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2870596,"kb_used_data":1977212,"kb_used_omap":1,"kb_used_meta":893374,"kb_avail":90874556,"statfs":{"total":95995035648,"available":93055545344,"internally_reserved":0,"allocated":2024665088,"data_stored":2024071978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":914815320},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:46:36 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":1.484,"15min":1.5189999999999999},"min":{"1min":0.19600000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.79100000000000004,"5min":72.418000000000006,"15min":72.418000000000006},"last":1.456},{"interface":"front","average":{"1min":0.69199999999999995,"5min":1.504,"15min":1.49},"min":{"1min":0.253,"5min":0.17599999999999999,"15min":0.16600000000000001},"max":{"1min":2.77,"5min":71.703999999999994,"15min":71.703999999999994},"last":2.903}]},{"osd":2,"last update":"Wed Sep 11 07:46:04 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":3.5219999999999998,"15min":2.9950000000000001},"min":{"1min":0.26100000000000001,"5min":0.25900000000000001,"15min":0.188},"max":{"1min":4.0890000000000004,"5min":164.821,"15min":200.339},"last":1.6930000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":3.6920000000000002,"15min":2.984},"min":{"1min":0.254,"5min":0.23999999999999999,"15min":0.20699999999999999},"max":{"1min":1.3360000000000001,"5min":167.68799999999999,"15min":200.40100000000001},"last":1.3560000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:46:36 2024","interfaces":[{"interface":"back","average":{"1min":3.3450000000000002,"5min":1.468,"15min":1.72},"min":{"1min":0.32900000000000001,"5min":0.25,"15min":0.20799999999999999},"max":{"1min":37.143000000000001,"5min":37.143000000000001,"15min":37.143000000000001},"last":2.391},{"interface":"front","average":{"1min":3.3580000000000001,"5min":1.52,"15min":1.6579999999999999},"min":{"1min":0.30499999999999999,"5min":0.253,"15min":0.22800000000000001},"max":{"1min":38.198,"5min":38.198,"15min":38.198},"last":2.7679999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:45:53 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":1.0529999999999999,"15min":1.3480000000000001},"min":{"1min":0.33400000000000002,"5min":0.30399999999999999,"15min":0.26700000000000002},"max":{"1min":2.9700000000000002,"5min":7.0519999999999996,"15min":30.315000000000001},"last":1.8380000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.23,"15min":1.5149999999999999},"min":{"1min":0.379,"5min":0.28599999999999998,"15min":0.27000000000000002},"max":{"1min":4.2119999999999997,"5min":6.5439999999999996,"15min":30.363},"last":0.37}]},{"osd":5,"last update":"Wed Sep 11 07:46:04 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.88700000000000001,"15min":1.2949999999999999},"min":{"1min":0.37,"5min":0.313,"15min":0.26800000000000002},"max":{"1min":5.0119999999999996,"5min":5.0119999999999996,"15min":19.381},"last":1.8759999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.075,"15min":1.4510000000000001},"min":{"1min":0.40100000000000002,"5min":0.35299999999999998,"15min":0.27200000000000002},"max":{"1min":4.0659999999999998,"5min":6.46,"15min":19.437000000000001},"last":0.34200000000000003}]},{"osd":6,"last update":"Wed Sep 11 07:46:36 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.1299999999999999,"15min":1.4330000000000001},"min":{"1min":0.42499999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":3.1059999999999999,"5min":6.5190000000000001,"15min":23.190999999999999},"last":0.312},{"interface":"front","average":{"1min":0.70299999999999996,"5min":1.0580000000000001,"15min":1.387},"min":{"1min":0.38200000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.925,"5min":4.4640000000000004,"15min":23.632000000000001},"last":1.3879999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:46:32 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.5449999999999999,"15min":1.577},"min":{"1min":0.44700000000000001,"5min":0.375,"15min":0.34399999999999997},"max":{"1min":2.7309999999999999,"5min":34.704999999999998,"15min":34.704999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":1.5069999999999999,"15min":1.577},"min":{"1min":0.40400000000000003,"5min":0.38800000000000001,"15min":0.315},"max":{"1min":1.9810000000000001,"5min":34.930999999999997,"15min":34.930999999999997},"last":1.425}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2028077056,"data_stored":2028077056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1339883520,"data_stored":1339883520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2244280320,"data_stored":2244280320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":918814720,"data_stored":918814720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":450953216,"data_stored":450953216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2250571776,"data_stored":2250571776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1591803904,"data_stored":1591803904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:46:49.049 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj 6, currently 16 2024-09-11T07:46:49.061 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:49.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:49.062 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:49.109 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 37 16 60898 60882 102.813 80.3125 0.0118719 0.00972041 2024-09-11T07:46:49.163 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:49.188 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:49.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:49.189 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:49.251 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:46:49.252 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:46:49.290 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:49.315 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:49.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:49.316 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:49.417 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:49.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:49.470 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:49.470 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:49.571 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:49.599 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:49.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:49.600 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:49.701 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:49.728 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:49.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:49.729 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:49.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:49 smithi104 ceph-mon[33507]: from='client.26035 v1:172.21.15.22:0/4074083393' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:46:49.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:49 smithi104 ceph-mon[33507]: pgmap v947: 158 pgs: 158 active+clean; 3.4 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 37 KiB/s rd, 67 MiB/s wr, 1.14k op/s 2024-09-11T07:46:49.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:49 smithi104 ceph-mon[33507]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:46:49.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:49 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/840033155' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:46:49.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:49 smithi104 ceph-mon[33507]: osdmap e427: 8 total, 8 up, 7 in 2024-09-11T07:46:49.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:49 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:46:49.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:49 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:46:49.830 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:49.838 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:49 smithi022 ceph-mon[29462]: from='client.26035 v1:172.21.15.22:0/4074083393' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:46:49.838 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:49 smithi022 ceph-mon[29462]: pgmap v947: 158 pgs: 158 active+clean; 3.4 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 37 KiB/s rd, 67 MiB/s wr, 1.14k op/s 2024-09-11T07:46:49.838 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:49 smithi022 ceph-mon[29462]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:46:49.839 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:49 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/840033155' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:46:49.839 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:49 smithi022 ceph-mon[29462]: osdmap e427: 8 total, 8 up, 7 in 2024-09-11T07:46:49.839 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:49 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:46:49.839 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:49 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:46:49.839 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:49 smithi022 ceph-mon[37330]: from='client.26035 v1:172.21.15.22:0/4074083393' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:46:49.840 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:49 smithi022 ceph-mon[37330]: pgmap v947: 158 pgs: 158 active+clean; 3.4 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 37 KiB/s rd, 67 MiB/s wr, 1.14k op/s 2024-09-11T07:46:49.840 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:49 smithi022 ceph-mon[37330]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:46:49.840 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:49 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/840033155' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:46:49.840 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:49 smithi022 ceph-mon[37330]: osdmap e427: 8 total, 8 up, 7 in 2024-09-11T07:46:49.840 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:49 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:46:49.841 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:49 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:46:49.858 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:49.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:49.859 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:49.960 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:49.990 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:49.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:49.991 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:50.092 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:50.178 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 38 16 62221 62205 102.283 82.6875 0.0175415 0.00977193 2024-09-11T07:46:50.203 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:50.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:50.204 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:50.305 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:50.344 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:50.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:50.345 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:50.446 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:50.473 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:50.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:50.474 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:50.488 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:46:50 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:46:50.574 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:50.626 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:50.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:50.628 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:50.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:50 smithi022 ceph-mon[29462]: osdmap e428: 8 total, 8 up, 7 in 2024-09-11T07:46:50.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:46:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:46:50] "GET /metrics HTTP/1.1" 200 225126 "" "Prometheus/2.43.0" 2024-09-11T07:46:50.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:50 smithi022 ceph-mon[37330]: osdmap e428: 8 total, 8 up, 7 in 2024-09-11T07:46:50.729 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:50.753 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:50.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:50.754 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:50.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:50 smithi104 ceph-mon[33507]: osdmap e428: 8 total, 8 up, 7 in 2024-09-11T07:46:50.855 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:50.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:50.896 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:50.897 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:50.997 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:51.036 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:51.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:51.037 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:51.084 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-11T07:46:51.109 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 39 16 63570 63554 101.822 84.3125 0.00965826 0.00981605 2024-09-11T07:46:51.139 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:51.164 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:51.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:51.165 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:51.265 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:51.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:51.306 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:51.307 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:51.407 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:51.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:51.465 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:51.466 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:51.566 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:51.592 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:51.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:51.593 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:51.694 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:51.717 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:51.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:51.718 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:51.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:51 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:46:51.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:51 smithi104 ceph-mon[33507]: pgmap v950: 158 pgs: 158 active+clean; 3.8 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 33 KiB/s rd, 100 MiB/s wr, 1.65k op/s 2024-09-11T07:46:51.819 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:51.882 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:51.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:51.884 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:51.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:51 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:46:51.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:51 smithi022 ceph-mon[29462]: pgmap v950: 158 pgs: 158 active+clean; 3.8 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 33 KiB/s rd, 100 MiB/s wr, 1.65k op/s 2024-09-11T07:46:51.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:51 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:46:51.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:51 smithi022 ceph-mon[37330]: pgmap v950: 158 pgs: 158 active+clean; 3.8 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 33 KiB/s rd, 100 MiB/s wr, 1.65k op/s 2024-09-11T07:46:51.985 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:52.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:52.013 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:52.013 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:52.109 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:2024-09-11T07:46:52.109416+0000 min lat: 0.00115663 max lat: 1.02411 avg lat: 0.00987336 2024-09-11T07:46:52.110 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T07:46:52.110 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 40 16 64812 64796 101.217 77.625 0.0167488 0.00987336 2024-09-11T07:46:52.113 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:52.167 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:52.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:52.168 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:52.269 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:52.307 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:52.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:52.308 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:52.358 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:46:52.409 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:52.430 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:46:52.430 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:46:52.453 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:52.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:52.454 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:52.463 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:46:52.463 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:46:52.492 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:46:52.493 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:46:52.527 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:46:52.527 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:46:52.555 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:52.562 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:46:52.562 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:46:52.586 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T07:46:52.597 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:46:52.597 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:46:52.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:52.603 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:52.603 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:52.628 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:46:52.629 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:46:52.665 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:46:52.665 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:46:52.704 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:52.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:52.748 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:52.748 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:52.778 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:52 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/483418238' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:46:52.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:52 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/483418238' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:46:52.779 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:52 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/483418238' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:46:52.849 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:52.874 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:52.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:52.875 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:52.976 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:53.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:53.023 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:53.023 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:53.110 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 41 16 65956 65940 100.492 71.5 0.00477001 0.00994566 2024-09-11T07:46:53.124 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:53.181 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:53.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:53.182 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:53.283 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:53.326 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:53.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:53.328 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:53.429 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:53.456 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:53.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:53.457 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:53.558 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:53.598 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:53.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:53.599 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:53.700 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:53.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:53.741 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:53.742 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:53.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:53 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:46:53.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:53 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:46:53.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:53 smithi104 ceph-mon[33507]: pgmap v951: 158 pgs: 158 active+clean; 3.8 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 14 KiB/s rd, 71 MiB/s wr, 1.16k op/s 2024-09-11T07:46:53.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:53 smithi104 ceph-mon[33507]: osdmap e429: 8 total, 8 up, 7 in 2024-09-11T07:46:53.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:46:53.842 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:53.849 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:53 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:46:53.849 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:53 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:46:53.849 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:53 smithi022 ceph-mon[29462]: pgmap v951: 158 pgs: 158 active+clean; 3.8 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 14 KiB/s rd, 71 MiB/s wr, 1.16k op/s 2024-09-11T07:46:53.849 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:53 smithi022 ceph-mon[29462]: osdmap e429: 8 total, 8 up, 7 in 2024-09-11T07:46:53.849 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:46:53.849 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:53 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:46:53.849 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:53 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:46:53.849 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:53 smithi022 ceph-mon[37330]: pgmap v951: 158 pgs: 158 active+clean; 3.8 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 14 KiB/s rd, 71 MiB/s wr, 1.16k op/s 2024-09-11T07:46:53.849 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:53 smithi022 ceph-mon[37330]: osdmap e429: 8 total, 8 up, 7 in 2024-09-11T07:46:53.849 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:46:53.897 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:53.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:53.898 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:53.999 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:54.043 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:54.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:54.044 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:54.049 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:46:54.049 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:46:54.049 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-11T07:46:54.049 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T07:46:54.049 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T07:46:54.050 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 config set filestore_inject_stall 3 2024-09-11T07:46:54.110 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 42 16 67256 67240 100.034 81.25 0.0106615 0.00999168 2024-09-11T07:46:54.144 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:54.171 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:54.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:54.172 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:54.273 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:54.302 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:54.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:54.303 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:54.404 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:54.405 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:46:54.434 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:54.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:54.436 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:54.502 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:46:54.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:46:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:46:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:46:54.537 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:46:54.596 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:54.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:54.597 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:46:54.698 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:54.742 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:54.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:54.744 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:54.845 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:54.877 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:54.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:54.878 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:54.979 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:55.021 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:55.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:55.022 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:55.110 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 43 16 68516 68500 99.5392 78.75 0.0176216 0.0100404 2024-09-11T07:46:55.123 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:55.153 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:55.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:55.154 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:55.254 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:55.299 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:55.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:55.301 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:55.401 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:55.444 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:55.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:55.445 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:55.546 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:55.586 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:55.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:55.587 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:55.629 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:55 smithi104 ceph-mon[33507]: pgmap v953: 158 pgs: 158 active+clean; 3.9 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.7 KiB/s rd, 82 MiB/s wr, 1.31k op/s 2024-09-11T07:46:55.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:55 smithi022 ceph-mon[29462]: pgmap v953: 158 pgs: 158 active+clean; 3.9 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.7 KiB/s rd, 82 MiB/s wr, 1.31k op/s 2024-09-11T07:46:55.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:55 smithi022 ceph-mon[37330]: pgmap v953: 158 pgs: 158 active+clean; 3.9 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.7 KiB/s rd, 82 MiB/s wr, 1.31k op/s 2024-09-11T07:46:55.688 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:55.720 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:55.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:55.721 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:55.821 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:55.851 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:55.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:55.852 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:55.854 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:46:55.854 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 20, 2024-09-11T07:46:55.854 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 600, 2024-09-11T07:46:55.854 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:46:55.854 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.855 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:85945 9.15 9:ade5fd86:::benchmark_data_smithi022_133943_object85944:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e259)", 2024-09-11T07:46:55.855 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:26.360905+0000", 2024-09-11T07:46:55.855 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 509.48875836299999, 2024-09-11T07:46:55.855 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.024384057, 2024-09-11T07:46:55.855 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:46:55.855 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:46:55.855 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:46:55.856 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:46:55.856 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:55.856 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 85945 2024-09-11T07:46:55.856 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.856 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:46:55.856 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.856 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:46:55.857 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360905+0000", 2024-09-11T07:46:55.857 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.857 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.857 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.857 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:46:55.857 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360905+0000", 2024-09-11T07:46:55.858 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.858 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.858 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.858 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:46:55.858 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360904+0000", 2024-09-11T07:46:55.858 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:46:55.859 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.859 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.859 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:46:55.859 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360927+0000", 2024-09-11T07:46:55.859 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2274999999999999e-05 2024-09-11T07:46:55.859 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.859 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.859 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:46:55.860 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360932+0000", 2024-09-11T07:46:55.860 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2650000000000004e-06 2024-09-11T07:46:55.860 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.860 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.860 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:46:55.860 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360935+0000", 2024-09-11T07:46:55.860 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.303e-06 2024-09-11T07:46:55.860 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.860 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.860 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.860 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.461352+0000", 2024-09-11T07:46:55.861 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.10041712 2024-09-11T07:46:55.861 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.861 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.861 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:46:55.861 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.461362+0000", 2024-09-11T07:46:55.861 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0285999999999999e-05 2024-09-11T07:46:55.861 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.861 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.861 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.861 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.384927+0000", 2024-09-11T07:46:55.861 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.92356409100000003 2024-09-11T07:46:55.861 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.862 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.862 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:46:55.862 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.385289+0000", 2024-09-11T07:46:55.862 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00036233199999999999 2024-09-11T07:46:55.862 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.862 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:46:55.862 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.862 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.862 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.862 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:85949 9.9 9:956e8bba:::benchmark_data_smithi022_133943_object85948:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e259)", 2024-09-11T07:46:55.862 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:26.360938+0000", 2024-09-11T07:46:55.862 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 509.488724972, 2024-09-11T07:46:55.862 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.021308804, 2024-09-11T07:46:55.862 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:46:55.863 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:55.863 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:46:55.863 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:46:55.863 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:55.863 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 85949 2024-09-11T07:46:55.863 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.863 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:46:55.863 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.863 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:46:55.863 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360938+0000", 2024-09-11T07:46:55.863 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.863 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.864 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.864 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:46:55.864 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360938+0000", 2024-09-11T07:46:55.864 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.864 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.864 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.864 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:46:55.864 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360938+0000", 2024-09-11T07:46:55.864 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:46:55.864 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.864 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.864 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:46:55.865 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.360995+0000", 2024-09-11T07:46:55.865 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7525999999999998e-05 2024-09-11T07:46:55.865 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.865 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.865 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:46:55.865 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.361001+0000", 2024-09-11T07:46:55.865 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3480000000000003e-06 2024-09-11T07:46:55.865 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.865 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.865 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:46:55.865 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.361004+0000", 2024-09-11T07:46:55.865 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5370000000000002e-06 2024-09-11T07:46:55.865 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.865 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.866 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.866 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.482968+0000", 2024-09-11T07:46:55.866 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.12196362700000001 2024-09-11T07:46:55.866 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.866 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.866 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:46:55.866 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.482978+0000", 2024-09-11T07:46:55.866 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.8260000000000005e-06 2024-09-11T07:46:55.866 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.866 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.866 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.866 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.369380+0000", 2024-09-11T07:46:55.866 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.886402201 2024-09-11T07:46:55.866 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.867 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.867 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:46:55.867 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.369511+0000", 2024-09-11T07:46:55.867 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013084699999999999 2024-09-11T07:46:55.867 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.867 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.867 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,7", 2024-09-11T07:46:55.867 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.369654+0000", 2024-09-11T07:46:55.867 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014310200000000001 2024-09-11T07:46:55.867 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.867 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.867 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:46:55.867 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.371458+0000", 2024-09-11T07:46:55.867 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001804436 2024-09-11T07:46:55.868 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.868 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.868 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.868 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.378473+0000", 2024-09-11T07:46:55.868 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0070152299999999999 2024-09-11T07:46:55.868 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.868 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.868 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.868 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.382167+0000", 2024-09-11T07:46:55.868 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.003693694 2024-09-11T07:46:55.868 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.868 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.868 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:46:55.868 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.382213+0000", 2024-09-11T07:46:55.869 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5469999999999997e-05 2024-09-11T07:46:55.869 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.869 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.869 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:46:55.869 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.382247+0000", 2024-09-11T07:46:55.869 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4468000000000003e-05 2024-09-11T07:46:55.869 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.869 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:46:55.869 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.869 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.869 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.869 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87210 9.19 9:9bd7b134:::benchmark_data_smithi022_133943_object87209:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:46:55.869 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.341865+0000", 2024-09-11T07:46:55.869 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 506.50779802400001, 2024-09-11T07:46:55.870 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.225729383, 2024-09-11T07:46:55.870 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:46:55.870 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:55.870 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:46:55.870 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:46:55.870 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:55.870 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87210 2024-09-11T07:46:55.870 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.870 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:46:55.870 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.870 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:46:55.871 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341865+0000", 2024-09-11T07:46:55.871 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.871 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.871 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.871 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:46:55.872 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341865+0000", 2024-09-11T07:46:55.872 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.873 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.873 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.873 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:46:55.873 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341863+0000", 2024-09-11T07:46:55.873 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:46:55.873 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.874 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.874 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:46:55.874 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341890+0000", 2024-09-11T07:46:55.874 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7458999999999999e-05 2024-09-11T07:46:55.874 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.875 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.875 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:46:55.875 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341903+0000", 2024-09-11T07:46:55.875 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3132999999999999e-05 2024-09-11T07:46:55.875 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.875 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.875 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:46:55.876 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341911+0000", 2024-09-11T07:46:55.876 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.8699999999999992e-06 2024-09-11T07:46:55.876 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.876 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.876 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.876 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342621+0000", 2024-09-11T07:46:55.876 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00070930199999999998 2024-09-11T07:46:55.876 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.877 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.877 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:46:55.877 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342627+0000", 2024-09-11T07:46:55.877 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0129999999999999e-06 2024-09-11T07:46:55.877 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.877 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.877 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.877 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.519201+0000", 2024-09-11T07:46:55.878 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.176574606 2024-09-11T07:46:55.878 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.878 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.878 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:46:55.878 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.519326+0000", 2024-09-11T07:46:55.878 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012495799999999999 2024-09-11T07:46:55.878 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.878 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.879 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,5,6", 2024-09-11T07:46:55.879 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.519682+0000", 2024-09-11T07:46:55.879 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00035572700000000002 2024-09-11T07:46:55.879 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.879 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.879 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.879 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.560173+0000", 2024-09-11T07:46:55.879 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.040491003999999997 2024-09-11T07:46:55.880 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.880 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.880 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.880 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.560894+0000", 2024-09-11T07:46:55.880 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00072137700000000004 2024-09-11T07:46:55.880 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.880 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.880 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.880 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.561062+0000", 2024-09-11T07:46:55.880 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000167658 2024-09-11T07:46:55.881 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.881 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.881 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:46:55.881 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.564965+0000", 2024-09-11T07:46:55.881 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0039035150000000002 2024-09-11T07:46:55.881 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.881 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.881 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:46:55.881 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.565018+0000", 2024-09-11T07:46:55.881 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2139999999999999e-05 2024-09-11T07:46:55.881 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.882 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.882 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:46:55.882 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.567595+0000", 2024-09-11T07:46:55.882 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0025769970000000001 2024-09-11T07:46:55.882 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.882 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:46:55.882 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.882 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.882 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.882 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87222 9.18 9:190c7e59:::benchmark_data_smithi022_133943_object87221:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:46:55.882 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.341925+0000", 2024-09-11T07:46:55.882 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 506.50773836799999, 2024-09-11T07:46:55.882 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.119733941, 2024-09-11T07:46:55.882 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:46:55.883 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:55.883 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:46:55.883 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:46:55.883 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:55.883 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87222 2024-09-11T07:46:55.883 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.883 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:46:55.884 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.884 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:46:55.884 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341925+0000", 2024-09-11T07:46:55.884 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.884 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.884 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.884 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:46:55.884 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341925+0000", 2024-09-11T07:46:55.884 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.884 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.884 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.885 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:46:55.885 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341924+0000", 2024-09-11T07:46:55.885 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:46:55.885 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.885 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.885 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:46:55.885 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341945+0000", 2024-09-11T07:46:55.885 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1024000000000001e-05 2024-09-11T07:46:55.885 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.885 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.885 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:46:55.886 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341952+0000", 2024-09-11T07:46:55.886 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.545e-06 2024-09-11T07:46:55.886 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.886 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.886 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:46:55.886 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341956+0000", 2024-09-11T07:46:55.886 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7589999999999997e-06 2024-09-11T07:46:55.886 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.886 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.886 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.886 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342444+0000", 2024-09-11T07:46:55.886 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00048726500000000002 2024-09-11T07:46:55.886 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.887 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.887 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:46:55.887 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342452+0000", 2024-09-11T07:46:55.887 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.1810000000000001e-06 2024-09-11T07:46:55.887 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.887 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.887 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.887 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.449925+0000", 2024-09-11T07:46:55.887 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.107473197 2024-09-11T07:46:55.887 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.887 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.887 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:46:55.887 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.450079+0000", 2024-09-11T07:46:55.887 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000153475 2024-09-11T07:46:55.887 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.888 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.888 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,5,6,7", 2024-09-11T07:46:55.888 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.450248+0000", 2024-09-11T07:46:55.888 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00016988499999999999 2024-09-11T07:46:55.888 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.888 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.888 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.888 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.456414+0000", 2024-09-11T07:46:55.888 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0061653100000000002 2024-09-11T07:46:55.888 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.888 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.888 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.888 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.460051+0000", 2024-09-11T07:46:55.889 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0036376450000000001 2024-09-11T07:46:55.889 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.889 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.889 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.889 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.460085+0000", 2024-09-11T07:46:55.889 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3268000000000001e-05 2024-09-11T07:46:55.889 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.889 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.889 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.889 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.460115+0000", 2024-09-11T07:46:55.889 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0292999999999999e-05 2024-09-11T07:46:55.889 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.889 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.889 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:46:55.890 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461466+0000", 2024-09-11T07:46:55.890 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0013508859999999999 2024-09-11T07:46:55.890 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.890 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.890 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:46:55.890 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461524+0000", 2024-09-11T07:46:55.890 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8607000000000003e-05 2024-09-11T07:46:55.890 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.890 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.890 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:46:55.890 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461659+0000", 2024-09-11T07:46:55.890 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000134358 2024-09-11T07:46:55.890 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.890 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:46:55.891 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.891 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.891 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.891 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87225 9.19 9:9ba55202:::benchmark_data_smithi022_133943_object87224:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:46:55.891 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.341961+0000", 2024-09-11T07:46:55.891 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 506.50770233700001, 2024-09-11T07:46:55.891 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2429198539999999, 2024-09-11T07:46:55.891 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:46:55.891 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:55.891 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:46:55.891 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:46:55.891 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:55.891 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87225 2024-09-11T07:46:55.892 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.892 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:46:55.892 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.892 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:46:55.892 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341961+0000", 2024-09-11T07:46:55.892 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.892 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.892 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.892 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:46:55.892 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341961+0000", 2024-09-11T07:46:55.892 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.892 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.892 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.893 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:46:55.893 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341960+0000", 2024-09-11T07:46:55.893 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:46:55.893 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.893 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.893 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:46:55.893 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341979+0000", 2024-09-11T07:46:55.893 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8606000000000001e-05 2024-09-11T07:46:55.893 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.893 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.894 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:46:55.894 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341985+0000", 2024-09-11T07:46:55.894 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1319999999999999e-06 2024-09-11T07:46:55.894 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.894 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.894 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:46:55.894 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341989+0000", 2024-09-11T07:46:55.894 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9469999999999996e-06 2024-09-11T07:46:55.894 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.894 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.894 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.894 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342643+0000", 2024-09-11T07:46:55.894 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00065436200000000002 2024-09-11T07:46:55.895 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.895 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.895 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:46:55.895 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342648+0000", 2024-09-11T07:46:55.895 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6759999999999998e-06 2024-09-11T07:46:55.895 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.895 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.895 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.895 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.561081+0000", 2024-09-11T07:46:55.895 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2184332229999999 2024-09-11T07:46:55.895 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.895 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.895 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:46:55.895 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.561226+0000", 2024-09-11T07:46:55.896 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014486300000000001 2024-09-11T07:46:55.896 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.896 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.896 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,5,6", 2024-09-11T07:46:55.896 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.561922+0000", 2024-09-11T07:46:55.896 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00069608500000000004 2024-09-11T07:46:55.896 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.896 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.896 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.896 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.567646+0000", 2024-09-11T07:46:55.896 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0057242279999999996 2024-09-11T07:46:55.896 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.896 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.896 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.897 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.567751+0000", 2024-09-11T07:46:55.897 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000104591 2024-09-11T07:46:55.897 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.897 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.897 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:46:55.897 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.574420+0000", 2024-09-11T07:46:55.897 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0066688900000000002 2024-09-11T07:46:55.897 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.897 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.897 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.897 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.582333+0000", 2024-09-11T07:46:55.897 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0079128410000000003 2024-09-11T07:46:55.897 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.898 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.898 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:46:55.898 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.582396+0000", 2024-09-11T07:46:55.898 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2976000000000001e-05 2024-09-11T07:46:55.898 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.898 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.898 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:46:55.898 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.584881+0000", 2024-09-11T07:46:55.898 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002485012 2024-09-11T07:46:55.898 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.898 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:46:55.898 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.898 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.898 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.899 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87240 9.18 9:1d562e34:::benchmark_data_smithi022_133943_object87239:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:46:55.899 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.341993+0000", 2024-09-11T07:46:55.899 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 506.50767023200001, 2024-09-11T07:46:55.899 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.121035673, 2024-09-11T07:46:55.899 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:46:55.899 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:55.899 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:46:55.899 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:46:55.899 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:55.899 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87240 2024-09-11T07:46:55.899 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.899 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:46:55.899 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.899 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:46:55.900 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341993+0000", 2024-09-11T07:46:55.900 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.900 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.900 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.900 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:46:55.900 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341993+0000", 2024-09-11T07:46:55.900 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.900 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.900 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.900 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:46:55.900 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.341992+0000", 2024-09-11T07:46:55.900 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:46:55.900 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.900 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.901 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:46:55.901 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342012+0000", 2024-09-11T07:46:55.901 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9117999999999999e-05 2024-09-11T07:46:55.901 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.901 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.901 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:46:55.901 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342017+0000", 2024-09-11T07:46:55.901 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7559999999999996e-06 2024-09-11T07:46:55.901 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.901 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.901 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:46:55.902 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342021+0000", 2024-09-11T07:46:55.902 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8639999999999998e-06 2024-09-11T07:46:55.902 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.902 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.902 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.902 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342489+0000", 2024-09-11T07:46:55.902 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00046751600000000001 2024-09-11T07:46:55.902 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.902 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.902 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:46:55.902 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342495+0000", 2024-09-11T07:46:55.902 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3720000000000001e-06 2024-09-11T07:46:55.902 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.902 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.903 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.903 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.452736+0000", 2024-09-11T07:46:55.903 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.110240694 2024-09-11T07:46:55.903 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.903 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.903 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:46:55.903 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.452864+0000", 2024-09-11T07:46:55.903 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012804299999999999 2024-09-11T07:46:55.903 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.904 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.904 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,5,6,7", 2024-09-11T07:46:55.904 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.453027+0000", 2024-09-11T07:46:55.904 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00016329100000000001 2024-09-11T07:46:55.904 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.904 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.904 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.904 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.459540+0000", 2024-09-11T07:46:55.904 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0065125319999999997 2024-09-11T07:46:55.904 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.904 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.904 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.904 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.460175+0000", 2024-09-11T07:46:55.904 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00063524399999999998 2024-09-11T07:46:55.905 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.905 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.905 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.905 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.460204+0000", 2024-09-11T07:46:55.905 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8762999999999999e-05 2024-09-11T07:46:55.905 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.905 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.905 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:46:55.905 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461822+0000", 2024-09-11T07:46:55.905 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0016181539999999999 2024-09-11T07:46:55.906 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.906 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.906 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.906 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.462813+0000", 2024-09-11T07:46:55.906 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00099143499999999993 2024-09-11T07:46:55.906 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.906 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.906 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:46:55.906 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.462901+0000", 2024-09-11T07:46:55.906 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.7409000000000003e-05 2024-09-11T07:46:55.906 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.907 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.907 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:46:55.907 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.463029+0000", 2024-09-11T07:46:55.907 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012804 2024-09-11T07:46:55.907 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.907 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:46:55.907 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.907 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.907 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.908 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235366 9.8 9:1702ff49:::benchmark_data_smithi022_133943_object108775:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e323)", 2024-09-11T07:46:55.908 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:02.747880+0000", 2024-09-11T07:46:55.908 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 353.101783512, 2024-09-11T07:46:55.908 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.96185844799999998, 2024-09-11T07:46:55.908 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:46:55.908 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:55.908 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:46:55.908 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:46:55.908 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:55.908 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235366 2024-09-11T07:46:55.909 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.909 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:46:55.909 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.909 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:46:55.909 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.747880+0000", 2024-09-11T07:46:55.909 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.909 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.909 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.909 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:46:55.909 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.747880+0000", 2024-09-11T07:46:55.909 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.909 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.910 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.910 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:46:55.910 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.747877+0000", 2024-09-11T07:46:55.910 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:46:55.910 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.910 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.910 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:46:55.910 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.747887+0000", 2024-09-11T07:46:55.910 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.9960000000000005e-06 2024-09-11T07:46:55.910 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.910 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.910 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:46:55.911 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.747913+0000", 2024-09-11T07:46:55.911 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.6363000000000002e-05 2024-09-11T07:46:55.911 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.911 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.911 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:46:55.911 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.747920+0000", 2024-09-11T07:46:55.911 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.2259999999999999e-06 2024-09-11T07:46:55.911 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.911 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.911 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.912 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.747967+0000", 2024-09-11T07:46:55.912 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6034e-05 2024-09-11T07:46:55.912 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.912 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.912 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:46:55.912 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.747974+0000", 2024-09-11T07:46:55.912 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.9370000000000004e-06 2024-09-11T07:46:55.912 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.912 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.912 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.913 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.706758+0000", 2024-09-11T07:46:55.913 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.95878366699999995 2024-09-11T07:46:55.913 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.913 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.913 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:46:55.913 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.706864+0000", 2024-09-11T07:46:55.913 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000105722 2024-09-11T07:46:55.913 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.914 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.914 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,7", 2024-09-11T07:46:55.914 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.706971+0000", 2024-09-11T07:46:55.914 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000107589 2024-09-11T07:46:55.914 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.914 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.914 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.915 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.709589+0000", 2024-09-11T07:46:55.915 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0026178730000000002 2024-09-11T07:46:55.915 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.915 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.915 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:46:55.915 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.709599+0000", 2024-09-11T07:46:55.915 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0149999999999999e-05 2024-09-11T07:46:55.915 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.915 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.916 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.916 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.709652+0000", 2024-09-11T07:46:55.916 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2627000000000003e-05 2024-09-11T07:46:55.916 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.916 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.916 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:46:55.916 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.709694+0000", 2024-09-11T07:46:55.916 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1956999999999998e-05 2024-09-11T07:46:55.916 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.916 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.916 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:46:55.917 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.709738+0000", 2024-09-11T07:46:55.917 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4122000000000002e-05 2024-09-11T07:46:55.917 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.917 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:46:55.917 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.917 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.917 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.917 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235378 9.8 9:17e72ff8:::benchmark_data_smithi022_133943_object108787:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e323)", 2024-09-11T07:46:55.917 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:02.759690+0000", 2024-09-11T07:46:55.917 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 353.08997349700002, 2024-09-11T07:46:55.917 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.96079267800000001, 2024-09-11T07:46:55.917 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:46:55.917 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:55.917 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:46:55.918 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:46:55.918 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:55.918 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235378 2024-09-11T07:46:55.918 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.918 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:46:55.918 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.918 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:46:55.918 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.759690+0000", 2024-09-11T07:46:55.918 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.918 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.918 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.918 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:46:55.918 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.759690+0000", 2024-09-11T07:46:55.919 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.919 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.919 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.919 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:46:55.919 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.759688+0000", 2024-09-11T07:46:55.919 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:46:55.919 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.919 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.919 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:46:55.919 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.759694+0000", 2024-09-11T07:46:55.919 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7400000000000001e-06 2024-09-11T07:46:55.919 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.919 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.919 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:46:55.920 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.759703+0000", 2024-09-11T07:46:55.920 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.5830000000000007e-06 2024-09-11T07:46:55.920 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.920 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.920 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:46:55.920 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.759708+0000", 2024-09-11T07:46:55.920 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.4199999999999998e-06 2024-09-11T07:46:55.920 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.920 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.920 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.920 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.759773+0000", 2024-09-11T07:46:55.920 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4772999999999998e-05 2024-09-11T07:46:55.920 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.921 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.921 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:46:55.921 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.759780+0000", 2024-09-11T07:46:55.921 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.2729999999999997e-06 2024-09-11T07:46:55.921 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.921 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.921 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.921 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.709785+0000", 2024-09-11T07:46:55.921 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.95000451399999997 2024-09-11T07:46:55.922 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.922 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.922 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:46:55.922 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.709890+0000", 2024-09-11T07:46:55.922 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010527300000000001 2024-09-11T07:46:55.922 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.922 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.922 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,7", 2024-09-11T07:46:55.922 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.709962+0000", 2024-09-11T07:46:55.922 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1935999999999997e-05 2024-09-11T07:46:55.922 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.922 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.922 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.923 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.714856+0000", 2024-09-11T07:46:55.923 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:46:55.924 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0048941230000000002 2024-09-11T07:46:55.924 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.924 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.924 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:46:55.924 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.714864+0000", 2024-09-11T07:46:55.925 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.7330000000000003e-06 2024-09-11T07:46:55.925 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.925 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.925 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.925 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.720399+0000", 2024-09-11T07:46:55.926 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.005534743 2024-09-11T07:46:55.926 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.926 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.926 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:46:55.926 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.720445+0000", 2024-09-11T07:46:55.926 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5998000000000002e-05 2024-09-11T07:46:55.927 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.927 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.927 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:46:55.927 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.720482+0000", 2024-09-11T07:46:55.927 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7861999999999998e-05 2024-09-11T07:46:55.927 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.928 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:46:55.928 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.928 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.928 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.928 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235390 9.8 9:17241c1b:::benchmark_data_smithi022_133943_object108799:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e323)", 2024-09-11T07:46:55.928 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:02.770070+0000", 2024-09-11T07:46:55.929 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 353.07959283700001, 2024-09-11T07:46:55.929 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.95338818199999997, 2024-09-11T07:46:55.929 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:46:55.929 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:55.929 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:46:55.929 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:46:55.930 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:46:55.930 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235390 2024-09-11T07:46:55.930 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.930 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:46:55.930 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.930 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:46:55.930 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.770070+0000", 2024-09-11T07:46:55.931 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.931 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.931 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.931 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:46:55.931 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.770070+0000", 2024-09-11T07:46:55.931 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.932 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.932 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.932 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:46:55.932 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.770069+0000", 2024-09-11T07:46:55.933 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:46:55.933 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.933 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.933 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:46:55.933 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.770076+0000", 2024-09-11T07:46:55.933 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0509999999999999e-06 2024-09-11T07:46:55.933 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.933 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.933 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:46:55.934 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.770099+0000", 2024-09-11T07:46:55.934 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2938e-05 2024-09-11T07:46:55.934 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.934 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.934 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:46:55.934 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.770105+0000", 2024-09-11T07:46:55.934 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9510000000000003e-06 2024-09-11T07:46:55.934 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.934 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.934 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.934 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.770145+0000", 2024-09-11T07:46:55.935 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0818999999999997e-05 2024-09-11T07:46:55.935 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.935 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.935 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:46:55.935 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.770152+0000", 2024-09-11T07:46:55.935 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9750000000000001e-06 2024-09-11T07:46:55.935 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.935 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.935 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.935 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.714887+0000", 2024-09-11T07:46:55.935 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.94473459699999995 2024-09-11T07:46:55.936 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.936 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.936 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:46:55.936 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.714990+0000", 2024-09-11T07:46:55.936 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000103251 2024-09-11T07:46:55.936 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.936 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.936 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,7", 2024-09-11T07:46:55.936 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.715109+0000", 2024-09-11T07:46:55.936 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000119255 2024-09-11T07:46:55.936 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.936 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.936 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.937 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.720644+0000", 2024-09-11T07:46:55.937 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0055347640000000002 2024-09-11T07:46:55.937 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.937 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.937 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.937 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.720669+0000", 2024-09-11T07:46:55.937 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4468999999999999e-05 2024-09-11T07:46:55.937 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.937 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.937 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:46:55.937 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.723232+0000", 2024-09-11T07:46:55.937 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0025635329999999998 2024-09-11T07:46:55.937 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.937 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.938 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:46:55.938 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.723417+0000", 2024-09-11T07:46:55.938 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00018494900000000001 2024-09-11T07:46:55.938 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.938 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.938 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:46:55.938 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:03.723459+0000", 2024-09-11T07:46:55.938 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1399000000000001e-05 2024-09-11T07:46:55.938 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.938 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:46:55.938 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.938 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.938 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.938 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:254719 10.2 10:41ac8189:::benchmark_data_smithi022_163866_object125293:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e398)", 2024-09-11T07:46:55.939 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:45:30.779849+0000", 2024-09-11T07:46:55.939 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 85.069814683999994, 2024-09-11T07:46:55.939 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0017689890000001, 2024-09-11T07:46:55.939 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:46:55.939 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:55.939 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:46:55.939 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:46:55.939 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:46:55.939 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 254719 2024-09-11T07:46:55.939 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.939 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:46:55.939 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.939 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:46:55.939 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.779849+0000", 2024-09-11T07:46:55.940 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.940 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.940 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.940 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:46:55.940 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.779849+0000", 2024-09-11T07:46:55.940 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.940 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.940 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.940 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:46:55.940 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.779847+0000", 2024-09-11T07:46:55.940 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:46:55.940 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.940 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.940 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:46:55.941 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.779853+0000", 2024-09-11T07:46:55.941 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7449999999999998e-06 2024-09-11T07:46:55.941 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.941 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.941 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:46:55.941 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.779864+0000", 2024-09-11T07:46:55.941 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1060000000000001e-05 2024-09-11T07:46:55.941 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.941 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.941 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:46:55.941 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.779870+0000", 2024-09-11T07:46:55.941 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7119999999999997e-06 2024-09-11T07:46:55.941 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.941 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.942 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.942 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.779911+0000", 2024-09-11T07:46:55.942 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0420000000000003e-05 2024-09-11T07:46:55.942 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.942 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.942 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:46:55.942 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.779915+0000", 2024-09-11T07:46:55.942 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8099999999999997e-06 2024-09-11T07:46:55.942 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.943 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.943 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.943 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.774391+0000", 2024-09-11T07:46:55.943 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.99447601500000005 2024-09-11T07:46:55.943 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.943 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.943 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:46:55.943 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.774488+0000", 2024-09-11T07:46:55.943 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.6154000000000001e-05 2024-09-11T07:46:55.943 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.943 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.943 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 6,7", 2024-09-11T07:46:55.943 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.774590+0000", 2024-09-11T07:46:55.943 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000102106 2024-09-11T07:46:55.943 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.944 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.944 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:46:55.944 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.779195+0000", 2024-09-11T07:46:55.944 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0046050070000000004 2024-09-11T07:46:55.944 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.944 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.944 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.944 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.781508+0000", 2024-09-11T07:46:55.944 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0023138009999999999 2024-09-11T07:46:55.944 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.944 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.944 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.944 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.781540+0000", 2024-09-11T07:46:55.945 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.1745000000000002e-05 2024-09-11T07:46:55.945 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.945 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.945 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:46:55.945 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.781584+0000", 2024-09-11T07:46:55.945 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3934000000000003e-05 2024-09-11T07:46:55.945 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.945 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.945 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:46:55.945 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.781618+0000", 2024-09-11T07:46:55.945 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3349999999999997e-05 2024-09-11T07:46:55.945 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.945 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:46:55.945 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.946 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.946 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.946 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:254723 10.2 10:40f50ed4:::benchmark_data_smithi022_163866_object125297:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e398)", 2024-09-11T07:46:55.946 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:45:30.783849+0000", 2024-09-11T07:46:55.946 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 85.065814098000004, 2024-09-11T07:46:55.946 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0043386169999999, 2024-09-11T07:46:55.946 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:46:55.946 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:55.946 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:46:55.946 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:46:55.946 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:46:55.946 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 254723 2024-09-11T07:46:55.946 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.946 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:46:55.947 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.947 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:46:55.947 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.783849+0000", 2024-09-11T07:46:55.947 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.947 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.947 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.947 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:46:55.947 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.783849+0000", 2024-09-11T07:46:55.947 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.947 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.947 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.947 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:46:55.947 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.783847+0000", 2024-09-11T07:46:55.947 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:46:55.947 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.948 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.948 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:46:55.948 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.783855+0000", 2024-09-11T07:46:55.948 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.1880000000000002e-06 2024-09-11T07:46:55.948 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.948 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.948 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:46:55.948 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.783865+0000", 2024-09-11T07:46:55.948 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0044000000000001e-05 2024-09-11T07:46:55.948 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.948 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.948 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:46:55.948 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.783871+0000", 2024-09-11T07:46:55.948 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.4029999999999998e-06 2024-09-11T07:46:55.949 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.949 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.949 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.949 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.783919+0000", 2024-09-11T07:46:55.949 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8718999999999999e-05 2024-09-11T07:46:55.949 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.949 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.949 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:46:55.950 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.783927+0000", 2024-09-11T07:46:55.950 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5680000000000002e-06 2024-09-11T07:46:55.950 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.950 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.950 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.950 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.777001+0000", 2024-09-11T07:46:55.951 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.99307440800000002 2024-09-11T07:46:55.951 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.951 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.951 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:46:55.951 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.777106+0000", 2024-09-11T07:46:55.951 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000104292 2024-09-11T07:46:55.951 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.951 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.951 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 6,7", 2024-09-11T07:46:55.951 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.777169+0000", 2024-09-11T07:46:55.951 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3366000000000003e-05 2024-09-11T07:46:55.951 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.952 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.952 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:46:55.952 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.781474+0000", 2024-09-11T07:46:55.952 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0043051069999999999 2024-09-11T07:46:55.952 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.952 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.952 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.952 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.787983+0000", 2024-09-11T07:46:55.952 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0065085480000000003 2024-09-11T07:46:55.952 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.952 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.952 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.952 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.788026+0000", 2024-09-11T07:46:55.953 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3883e-05 2024-09-11T07:46:55.953 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.953 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.953 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:46:55.953 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.788118+0000", 2024-09-11T07:46:55.953 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.1235000000000003e-05 2024-09-11T07:46:55.953 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.953 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.953 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:46:55.953 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.788188+0000", 2024-09-11T07:46:55.953 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0068999999999996e-05 2024-09-11T07:46:55.954 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.954 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:46:55.954 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.954 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.954 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.954 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:254746 10.4 10:2e7f9d42:::benchmark_data_smithi022_163866_object125320:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e398)", 2024-09-11T07:46:55.954 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:45:30.801228+0000", 2024-09-11T07:46:55.954 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 85.048434983000007, 2024-09-11T07:46:55.954 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.96725645800000004, 2024-09-11T07:46:55.954 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:46:55.954 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:55.954 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:46:55.954 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:46:55.954 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:46:55.954 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 254746 2024-09-11T07:46:55.955 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.955 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:46:55.955 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.955 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:46:55.955 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.801228+0000", 2024-09-11T07:46:55.955 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.955 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.955 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.955 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:46:55.955 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.801228+0000", 2024-09-11T07:46:55.955 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.955 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.955 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.955 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:46:55.956 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.801227+0000", 2024-09-11T07:46:55.956 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:46:55.956 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.956 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.956 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:46:55.956 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.801232+0000", 2024-09-11T07:46:55.956 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8609999999999997e-06 2024-09-11T07:46:55.956 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.956 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.956 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:46:55.956 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.801240+0000", 2024-09-11T07:46:55.956 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.8679999999999999e-06 2024-09-11T07:46:55.956 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.956 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.957 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:46:55.957 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.801244+0000", 2024-09-11T07:46:55.957 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2379999999999999e-06 2024-09-11T07:46:55.957 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.957 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.957 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.957 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.801282+0000", 2024-09-11T07:46:55.957 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8185000000000001e-05 2024-09-11T07:46:55.957 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.957 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.957 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:46:55.957 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.801288+0000", 2024-09-11T07:46:55.957 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6489999999999996e-06 2024-09-11T07:46:55.957 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.958 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.958 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.958 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.762908+0000", 2024-09-11T07:46:55.958 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.96162007900000002 2024-09-11T07:46:55.958 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.958 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.958 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:46:55.958 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.763067+0000", 2024-09-11T07:46:55.958 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000158858 2024-09-11T07:46:55.958 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.958 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.958 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:46:55.958 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.763198+0000", 2024-09-11T07:46:55.958 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013084199999999999 2024-09-11T07:46:55.959 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.959 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.959 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.959 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.768279+0000", 2024-09-11T07:46:55.959 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.005080981 2024-09-11T07:46:55.959 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.959 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.959 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.959 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.768369+0000", 2024-09-11T07:46:55.959 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.0305999999999999e-05 2024-09-11T07:46:55.959 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.959 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.959 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:46:55.959 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.768381+0000", 2024-09-11T07:46:55.960 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1792e-05 2024-09-11T07:46:55.960 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.960 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.960 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:46:55.960 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.768443+0000", 2024-09-11T07:46:55.960 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2125000000000006e-05 2024-09-11T07:46:55.960 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.960 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.960 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:46:55.960 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.768485+0000", 2024-09-11T07:46:55.960 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1740999999999999e-05 2024-09-11T07:46:55.960 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.960 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:46:55.960 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.961 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.961 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.961 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:254747 10.2 10:47edeaf9:::benchmark_data_smithi022_163866_object125321:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e398)", 2024-09-11T07:46:55.961 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:45:30.802679+0000", 2024-09-11T07:46:55.961 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 85.046984017, 2024-09-11T07:46:55.961 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.98566537399999998, 2024-09-11T07:46:55.961 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:46:55.961 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:55.961 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:46:55.961 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:46:55.961 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:46:55.961 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 254747 2024-09-11T07:46:55.961 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.961 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:46:55.962 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.962 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:46:55.962 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.802679+0000", 2024-09-11T07:46:55.962 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.962 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.962 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.962 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:46:55.962 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.802679+0000", 2024-09-11T07:46:55.962 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.962 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.962 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.962 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:46:55.962 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.802678+0000", 2024-09-11T07:46:55.962 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:46:55.962 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.963 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.963 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:46:55.963 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.802684+0000", 2024-09-11T07:46:55.963 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0299999999999999e-06 2024-09-11T07:46:55.963 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.963 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.963 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:46:55.963 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.802693+0000", 2024-09-11T07:46:55.963 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.5910000000000002e-06 2024-09-11T07:46:55.964 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.964 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.964 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:46:55.964 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.802699+0000", 2024-09-11T07:46:55.964 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7629999999999997e-06 2024-09-11T07:46:55.964 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.964 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.964 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.964 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.802944+0000", 2024-09-11T07:46:55.964 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00024512399999999999 2024-09-11T07:46:55.964 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.964 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.965 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:46:55.965 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.802952+0000", 2024-09-11T07:46:55.965 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.7959999999999996e-06 2024-09-11T07:46:55.965 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.965 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.965 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.965 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.779224+0000", 2024-09-11T07:46:55.965 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.97627190500000005 2024-09-11T07:46:55.965 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.965 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.965 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:46:55.965 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.779333+0000", 2024-09-11T07:46:55.966 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000109104 2024-09-11T07:46:55.966 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.966 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.966 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 6,7", 2024-09-11T07:46:55.966 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.779395+0000", 2024-09-11T07:46:55.966 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2119000000000006e-05 2024-09-11T07:46:55.966 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.966 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.966 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:46:55.966 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.787994+0000", 2024-09-11T07:46:55.966 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0085991620000000005 2024-09-11T07:46:55.966 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.966 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.966 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.967 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.788237+0000", 2024-09-11T07:46:55.967 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000243176 2024-09-11T07:46:55.967 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.967 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.967 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.967 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.788271+0000", 2024-09-11T07:46:55.967 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3691e-05 2024-09-11T07:46:55.967 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.967 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.967 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:46:55.967 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.788301+0000", 2024-09-11T07:46:55.967 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.9751e-05 2024-09-11T07:46:55.967 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.967 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.967 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:46:55.968 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.788345+0000", 2024-09-11T07:46:55.968 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3637999999999998e-05 2024-09-11T07:46:55.968 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.968 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:46:55.968 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.968 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.968 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.968 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:254757 10.4 10:2bd04c6d:::benchmark_data_smithi022_163866_object125331:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e398)", 2024-09-11T07:46:55.968 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:45:30.807462+0000", 2024-09-11T07:46:55.968 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 85.042200906999994, 2024-09-11T07:46:55.968 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.96554583500000002, 2024-09-11T07:46:55.968 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:46:55.968 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:55.968 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:46:55.969 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:46:55.969 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:46:55.969 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 254757 2024-09-11T07:46:55.969 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.969 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:46:55.969 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.969 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:46:55.969 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.807462+0000", 2024-09-11T07:46:55.969 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.969 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.969 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.969 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:46:55.969 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.807462+0000", 2024-09-11T07:46:55.969 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.970 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.970 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.970 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:46:55.970 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.807461+0000", 2024-09-11T07:46:55.970 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:46:55.970 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.970 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.970 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:46:55.970 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.807466+0000", 2024-09-11T07:46:55.970 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3800000000000002e-06 2024-09-11T07:46:55.970 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.970 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.970 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:46:55.970 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.807486+0000", 2024-09-11T07:46:55.971 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9653999999999999e-05 2024-09-11T07:46:55.971 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.971 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.971 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:46:55.971 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.807491+0000", 2024-09-11T07:46:55.971 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.5049999999999998e-06 2024-09-11T07:46:55.971 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.971 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.971 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.971 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.807583+0000", 2024-09-11T07:46:55.971 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.1738999999999997e-05 2024-09-11T07:46:55.971 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.971 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.971 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:46:55.971 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.807590+0000", 2024-09-11T07:46:55.972 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.652e-06 2024-09-11T07:46:55.972 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.972 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.972 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.972 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.765823+0000", 2024-09-11T07:46:55.972 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.95823342199999995 2024-09-11T07:46:55.972 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.972 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.972 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:46:55.972 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.766064+0000", 2024-09-11T07:46:55.972 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00024126 2024-09-11T07:46:55.972 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.972 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.973 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:55.974 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:46:55.974 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.766190+0000", 2024-09-11T07:46:55.974 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000125401 2024-09-11T07:46:55.975 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.975 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.975 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.975 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.768646+0000", 2024-09-11T07:46:55.975 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0024568020000000001 2024-09-11T07:46:55.975 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.975 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.975 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:46:55.975 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.768652+0000", 2024-09-11T07:46:55.975 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8599999999999998e-06 2024-09-11T07:46:55.975 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.975 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.975 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.976 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.772694+0000", 2024-09-11T07:46:55.976 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0040419150000000001 2024-09-11T07:46:55.976 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.976 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.976 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:46:55.977 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.772755+0000", 2024-09-11T07:46:55.977 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0683999999999997e-05 2024-09-11T07:46:55.977 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.977 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.977 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:46:55.977 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.773008+0000", 2024-09-11T07:46:55.977 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000253243 2024-09-11T07:46:55.977 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.977 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:46:55.977 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.978 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.978 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.978 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:254759 10.4 10:2fcc4b3f:::benchmark_data_smithi022_163866_object125333:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e398)", 2024-09-11T07:46:55.978 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:45:30.810285+0000", 2024-09-11T07:46:55.978 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 85.039377940999998, 2024-09-11T07:46:55.978 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.96322548500000005, 2024-09-11T07:46:55.978 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:46:55.978 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:55.978 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:46:55.978 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:46:55.978 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:46:55.978 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 254759 2024-09-11T07:46:55.978 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.978 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:46:55.979 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.979 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:46:55.979 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.810285+0000", 2024-09-11T07:46:55.979 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.979 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.979 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.979 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:46:55.979 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.810285+0000", 2024-09-11T07:46:55.979 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.979 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.979 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.979 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:46:55.979 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.810283+0000", 2024-09-11T07:46:55.979 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:46:55.980 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.980 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.980 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:46:55.980 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.810290+0000", 2024-09-11T07:46:55.980 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4899999999999997e-06 2024-09-11T07:46:55.980 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.980 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.980 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:46:55.980 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.810305+0000", 2024-09-11T07:46:55.980 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4918e-05 2024-09-11T07:46:55.981 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.981 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.981 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:46:55.981 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.810311+0000", 2024-09-11T07:46:55.981 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0059999999999998e-06 2024-09-11T07:46:55.981 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.981 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.981 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.981 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.810391+0000", 2024-09-11T07:46:55.981 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.0245000000000002e-05 2024-09-11T07:46:55.981 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.981 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.981 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:46:55.982 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.810399+0000", 2024-09-11T07:46:55.982 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.6879999999999999e-06 2024-09-11T07:46:55.982 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.982 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.982 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.982 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.768674+0000", 2024-09-11T07:46:55.982 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.95827523800000003 2024-09-11T07:46:55.982 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.982 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.982 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:46:55.982 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.768792+0000", 2024-09-11T07:46:55.982 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000118384 2024-09-11T07:46:55.982 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.982 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.983 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:46:55.983 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.768918+0000", 2024-09-11T07:46:55.983 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012557900000000001 2024-09-11T07:46:55.983 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.983 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.983 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.983 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.773064+0000", 2024-09-11T07:46:55.983 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0041461240000000002 2024-09-11T07:46:55.983 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.983 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.983 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.983 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.773396+0000", 2024-09-11T07:46:55.983 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00033213300000000003 2024-09-11T07:46:55.983 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.984 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.984 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:46:55.984 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.773434+0000", 2024-09-11T07:46:55.984 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8305e-05 2024-09-11T07:46:55.984 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.984 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.984 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:46:55.984 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.773474+0000", 2024-09-11T07:46:55.984 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9539999999999998e-05 2024-09-11T07:46:55.985 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.985 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.985 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:46:55.985 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.773511+0000", 2024-09-11T07:46:55.985 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6783999999999999e-05 2024-09-11T07:46:55.985 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.985 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:46:55.985 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.985 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.985 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.985 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:254763 10.2 10:49d0590c:::benchmark_data_smithi022_163866_object125337:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e398)", 2024-09-11T07:46:55.985 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:45:30.811598+0000", 2024-09-11T07:46:55.985 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 85.038065516000003, 2024-09-11T07:46:55.986 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.97688686599999996, 2024-09-11T07:46:55.986 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:46:55.986 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:55.986 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:46:55.986 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:46:55.986 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:46:55.986 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 254763 2024-09-11T07:46:55.986 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.986 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:46:55.986 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.986 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:46:55.986 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.811598+0000", 2024-09-11T07:46:55.986 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.987 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.987 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.987 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:46:55.987 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.811598+0000", 2024-09-11T07:46:55.987 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.987 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.987 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.988 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:46:55.988 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.811596+0000", 2024-09-11T07:46:55.988 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:46:55.988 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.988 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.989 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:46:55.989 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.811602+0000", 2024-09-11T07:46:55.989 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0100000000000001e-06 2024-09-11T07:46:55.989 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.989 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.989 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:46:55.990 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.811612+0000", 2024-09-11T07:46:55.990 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.448e-06 2024-09-11T07:46:55.990 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.990 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.990 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:46:55.990 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.811617+0000", 2024-09-11T07:46:55.991 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.5419999999999996e-06 2024-09-11T07:46:55.991 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.991 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.991 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.991 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.811659+0000", 2024-09-11T07:46:55.991 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2154999999999999e-05 2024-09-11T07:46:55.991 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.991 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.991 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:46:55.991 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.811667+0000", 2024-09-11T07:46:55.991 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.2359999999999998e-06 2024-09-11T07:46:55.991 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.991 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.991 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.992 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.781641+0000", 2024-09-11T07:46:55.992 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.96997469300000005 2024-09-11T07:46:55.992 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.992 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.992 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:46:55.992 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.781852+0000", 2024-09-11T07:46:55.992 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000210657 2024-09-11T07:46:55.992 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.992 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.992 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 6,7", 2024-09-11T07:46:55.992 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.781952+0000", 2024-09-11T07:46:55.992 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000100221 2024-09-11T07:46:55.992 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.993 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.993 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:46:55.993 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.788197+0000", 2024-09-11T07:46:55.993 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0062450800000000001 2024-09-11T07:46:55.993 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.993 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.993 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.993 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.788381+0000", 2024-09-11T07:46:55.993 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000183389 2024-09-11T07:46:55.993 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.993 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.994 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:55.994 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.788414+0000", 2024-09-11T07:46:55.994 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3059000000000003e-05 2024-09-11T07:46:55.994 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.994 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.994 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:46:55.994 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.788452+0000", 2024-09-11T07:46:55.994 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8040999999999997e-05 2024-09-11T07:46:55.994 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.994 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.994 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:46:55.994 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.788485+0000", 2024-09-11T07:46:55.995 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2811000000000001e-05 2024-09-11T07:46:55.995 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.995 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:46:55.995 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:55.995 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.995 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.995 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:254769 10.4 10:253985a6:::benchmark_data_smithi022_163866_object125343:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e398)", 2024-09-11T07:46:55.995 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:45:30.815852+0000", 2024-09-11T07:46:55.995 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 85.033811688, 2024-09-11T07:46:55.995 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.95775579499999997, 2024-09-11T07:46:55.995 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:46:55.995 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:55.996 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:46:55.996 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:46:55.996 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:46:55.996 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 254769 2024-09-11T07:46:55.996 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.996 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:46:55.996 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.996 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:46:55.996 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.815852+0000", 2024-09-11T07:46:55.996 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.996 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.996 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.996 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:46:55.996 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.815852+0000", 2024-09-11T07:46:55.997 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:55.997 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.997 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.997 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:46:55.997 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.815850+0000", 2024-09-11T07:46:55.997 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:46:55.997 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.997 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.997 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:46:55.997 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.815855+0000", 2024-09-11T07:46:55.997 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9479999999999999e-06 2024-09-11T07:46:55.997 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.997 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.998 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:46:55.998 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.815870+0000", 2024-09-11T07:46:55.998 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4489e-05 2024-09-11T07:46:55.998 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.998 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.998 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:46:55.998 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.815876+0000", 2024-09-11T07:46:55.998 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2920000000000001e-06 2024-09-11T07:46:55.998 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.998 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.998 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.999 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.815925+0000", 2024-09-11T07:46:55.999 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9282e-05 2024-09-11T07:46:55.999 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.999 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.999 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:46:55.999 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.815935+0000", 2024-09-11T07:46:55.999 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.6690000000000001e-06 2024-09-11T07:46:55.999 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.999 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.999 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:55.999 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.770921+0000", 2024-09-11T07:46:55.999 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.95498606200000002 2024-09-11T07:46:55.999 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:55.999 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:55.999 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:46:56.000 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.771031+0000", 2024-09-11T07:46:56.000 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000109491 2024-09-11T07:46:56.000 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.000 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.000 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:46:56.000 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.771120+0000", 2024-09-11T07:46:56.000 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.9833000000000005e-05 2024-09-11T07:46:56.000 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.000 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.000 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:56.000 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.773428+0000", 2024-09-11T07:46:56.000 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002307847 2024-09-11T07:46:56.001 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.001 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.001 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:46:56.001 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.773516+0000", 2024-09-11T07:46:56.001 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.8144000000000002e-05 2024-09-11T07:46:56.001 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.001 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.001 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:56.001 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.773543+0000", 2024-09-11T07:46:56.001 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.6216000000000001e-05 2024-09-11T07:46:56.002 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.002 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.002 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:46:56.002 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.773577+0000", 2024-09-11T07:46:56.002 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4663000000000003e-05 2024-09-11T07:46:56.002 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.002 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.002 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:46:56.002 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.773607+0000", 2024-09-11T07:46:56.002 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.9980000000000001e-05 2024-09-11T07:46:56.002 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:56.002 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:46:56.002 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:56.002 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.003 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.003 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:254777 10.2 10:46a6228e:::benchmark_data_smithi022_163866_object125351:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e398)", 2024-09-11T07:46:56.003 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:45:30.822147+0000", 2024-09-11T07:46:56.003 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 85.027516579999997, 2024-09-11T07:46:56.003 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.96944138400000002, 2024-09-11T07:46:56.003 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:46:56.003 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:56.003 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:46:56.003 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:46:56.003 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:46:56.004 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 254777 2024-09-11T07:46:56.004 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.004 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:46:56.004 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.004 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:46:56.004 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.822147+0000", 2024-09-11T07:46:56.005 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:56.005 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.005 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.005 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:46:56.005 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.822147+0000", 2024-09-11T07:46:56.005 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:56.005 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.006 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.006 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:46:56.006 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.822145+0000", 2024-09-11T07:46:56.006 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:46:56.006 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.006 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.006 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:46:56.006 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.822151+0000", 2024-09-11T07:46:56.006 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7180000000000001e-06 2024-09-11T07:46:56.006 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.007 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.007 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:46:56.007 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.822161+0000", 2024-09-11T07:46:56.007 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0349999999999999e-05 2024-09-11T07:46:56.007 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.007 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.007 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:46:56.007 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.822167+0000", 2024-09-11T07:46:56.007 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7620000000000001e-06 2024-09-11T07:46:56.007 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.007 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.007 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:56.008 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.822208+0000", 2024-09-11T07:46:56.008 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0605999999999997e-05 2024-09-11T07:46:56.008 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.008 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.008 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:46:56.008 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.822212+0000", 2024-09-11T07:46:56.008 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7330000000000003e-06 2024-09-11T07:46:56.008 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.008 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.008 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:56.009 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.785493+0000", 2024-09-11T07:46:56.009 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.96328040800000003 2024-09-11T07:46:56.009 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.009 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.009 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:46:56.009 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.785591+0000", 2024-09-11T07:46:56.009 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.8103999999999994e-05 2024-09-11T07:46:56.009 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.010 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.010 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 6,7", 2024-09-11T07:46:56.010 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.785746+0000", 2024-09-11T07:46:56.010 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015488899999999999 2024-09-11T07:46:56.010 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.010 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.010 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:46:56.011 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.788203+0000", 2024-09-11T07:46:56.011 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0024568110000000002 2024-09-11T07:46:56.011 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.011 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.011 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:56.011 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.791112+0000", 2024-09-11T07:46:56.011 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0029093209999999999 2024-09-11T07:46:56.012 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.012 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.012 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:56.012 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.791141+0000", 2024-09-11T07:46:56.012 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8617000000000001e-05 2024-09-11T07:46:56.012 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.013 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.013 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:46:56.013 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.791549+0000", 2024-09-11T07:46:56.013 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00040818799999999998 2024-09-11T07:46:56.013 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.013 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.014 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:46:56.014 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.791588+0000", 2024-09-11T07:46:56.014 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9232e-05 2024-09-11T07:46:56.014 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:56.014 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:46:56.014 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:56.014 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.015 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.015 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:254780 10.2 10:4ca99afb:::benchmark_data_smithi022_163866_object125354:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e398)", 2024-09-11T07:46:56.015 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:45:30.823982+0000", 2024-09-11T07:46:56.015 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 85.025681606000006, 2024-09-11T07:46:56.015 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.972432348, 2024-09-11T07:46:56.016 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:46:56.016 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:46:56.016 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:46:56.016 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:46:56.016 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:46:56.016 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 254780 2024-09-11T07:46:56.017 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.017 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:46:56.017 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.017 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:46:56.017 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.823982+0000", 2024-09-11T07:46:56.017 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:56.018 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.018 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.018 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:46:56.018 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.823982+0000", 2024-09-11T07:46:56.018 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:56.018 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.019 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.019 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:46:56.019 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.823980+0000", 2024-09-11T07:46:56.019 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:46:56.019 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.020 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.020 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:46:56.020 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.823985+0000", 2024-09-11T07:46:56.020 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7910000000000004e-06 2024-09-11T07:46:56.020 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.020 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.021 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:46:56.021 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.823994+0000", 2024-09-11T07:46:56.021 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.0299999999999999e-06 2024-09-11T07:46:56.021 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.021 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.021 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:46:56.022 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.824000+0000", 2024-09-11T07:46:56.022 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.947e-06 2024-09-11T07:46:56.022 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.022 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.022 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:56.022 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.824044+0000", 2024-09-11T07:46:56.023 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3815e-05 2024-09-11T07:46:56.023 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.023 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.023 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:46:56.023 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.824055+0000", 2024-09-11T07:46:56.023 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1396999999999999e-05 2024-09-11T07:46:56.023 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.024 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.024 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:56.024 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.788927+0000", 2024-09-11T07:46:56.024 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.96487158100000003 2024-09-11T07:46:56.024 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.025 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.025 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:46:56.025 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.789031+0000", 2024-09-11T07:46:56.025 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000103801 2024-09-11T07:46:56.025 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.025 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.025 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 6,7", 2024-09-11T07:46:56.026 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.789114+0000", 2024-09-11T07:46:56.026 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3516999999999993e-05 2024-09-11T07:46:56.026 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.026 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.026 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:56.026 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.796154+0000", 2024-09-11T07:46:56.027 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0070394630000000001 2024-09-11T07:46:56.027 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.027 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.027 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:46:56.027 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.796182+0000", 2024-09-11T07:46:56.028 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8430000000000001e-05 2024-09-11T07:46:56.028 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.028 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.028 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:46:56.028 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.796330+0000", 2024-09-11T07:46:56.028 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000147885 2024-09-11T07:46:56.029 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.029 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.029 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:46:56.029 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.796376+0000", 2024-09-11T07:46:56.029 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6037e-05 2024-09-11T07:46:56.029 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.030 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.030 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:46:56.030 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.796414+0000", 2024-09-11T07:46:56.030 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7877999999999999e-05 2024-09-11T07:46:56.030 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:56.030 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:46:56.031 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:56.031 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.031 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.031 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24353.0:2704 5.15 5:a93a5511:::notify.2:head [watch reconnect cookie 94395805749008 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e424)", 2024-09-11T07:46:56.031 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:46:30.482628+0000", 2024-09-11T07:46:56.031 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 25.367035427000001, 2024-09-11T07:46:56.032 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.021040468, 2024-09-11T07:46:56.032 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:46:56.032 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:46:56.032 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:46:56.032 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24353", 2024-09-11T07:46:56.032 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1860969011", 2024-09-11T07:46:56.033 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 2704 2024-09-11T07:46:56.033 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.033 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:46:56.033 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.033 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:46:56.034 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.482628+0000", 2024-09-11T07:46:56.034 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:56.034 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.034 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.034 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:46:56.034 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.482628+0000", 2024-09-11T07:46:56.035 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:46:56.035 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.035 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.035 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:46:56.035 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.482626+0000", 2024-09-11T07:46:56.035 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:46:56.036 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.036 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.036 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:46:56.036 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.482633+0000", 2024-09-11T07:46:56.036 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.8550000000000002e-06 2024-09-11T07:46:56.036 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.037 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.037 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:46:56.037 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.482648+0000", 2024-09-11T07:46:56.037 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4663e-05 2024-09-11T07:46:56.037 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.038 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.038 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:46:56.038 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.482654+0000", 2024-09-11T07:46:56.038 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.5559999999999998e-06 2024-09-11T07:46:56.038 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.038 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.039 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:56.039 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.482693+0000", 2024-09-11T07:46:56.039 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9372000000000002e-05 2024-09-11T07:46:56.039 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.039 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.040 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op must wait for map", 2024-09-11T07:46:56.040 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.482700+0000", 2024-09-11T07:46:56.040 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1080000000000004e-06 2024-09-11T07:46:56.040 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.040 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.040 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:56.041 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.515417+0000", 2024-09-11T07:46:56.041 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.032716687000000001 2024-09-11T07:46:56.041 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.041 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.041 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:46:56.041 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.515426+0000", 2024-09-11T07:46:56.042 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.772e-06 2024-09-11T07:46:56.042 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.042 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.042 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:46:56.042 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.500221+0000", 2024-09-11T07:46:56.042 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.98479507499999996 2024-09-11T07:46:56.042 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.042 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.043 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:46:56.043 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.500376+0000", 2024-09-11T07:46:56.043 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015580499999999999 2024-09-11T07:46:56.043 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:46:56.043 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:46:56.043 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:46:56.043 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.503668+0000", 2024-09-11T07:46:56.043 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0032917950000000001 2024-09-11T07:46:56.044 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:56.044 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:46:56.044 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:56.044 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:46:56.044 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:46:56.044 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:46:56.087 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:56.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:56.088 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:56.110 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 44 16 69692 69676 98.9473 73.5 0.00857691 0.0101006 2024-09-11T07:46:56.190 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:56.237 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:56.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:56.238 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:56.339 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:56.372 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:56.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:56.373 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:56.474 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:46:56.520 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:56.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:56.521 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:46:56.550 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:46:56.551 INFO:teuthology.orchestra.run.smithi104.stdout: "success": "" 2024-09-11T07:46:56.551 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:46:56.622 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:56.650 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:56.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:56.651 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:56.751 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:56.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:56.779 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:56.779 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:56.880 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:56.921 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:56.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:56.922 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:57.023 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:57.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:56 smithi104 ceph-mon[33507]: osdmap e430: 8 total, 8 up, 7 in 2024-09-11T07:46:57.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:56 smithi104 ceph-mon[33507]: pgmap v955: 158 pgs: 158 active+clean; 4.1 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 739 B/s rd, 83 MiB/s wr, 1.32k op/s 2024-09-11T07:46:57.050 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:57.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:57.051 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:57.110 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 45 15 70851 70836 98.3595 72.5 0.0140763 0.0101619 2024-09-11T07:46:57.152 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:57.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:56 smithi022 ceph-mon[29462]: osdmap e430: 8 total, 8 up, 7 in 2024-09-11T07:46:57.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:56 smithi022 ceph-mon[29462]: pgmap v955: 158 pgs: 158 active+clean; 4.1 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 739 B/s rd, 83 MiB/s wr, 1.32k op/s 2024-09-11T07:46:57.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:56 smithi022 ceph-mon[37330]: osdmap e430: 8 total, 8 up, 7 in 2024-09-11T07:46:57.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:56 smithi022 ceph-mon[37330]: pgmap v955: 158 pgs: 158 active+clean; 4.1 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 739 B/s rd, 83 MiB/s wr, 1.32k op/s 2024-09-11T07:46:57.179 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:57.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:57.180 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:57.281 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:57.305 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:57.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:57.306 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:57.407 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:57.436 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:57.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:57.437 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:57.538 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:57.580 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:57.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:57.581 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:57.682 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:57.709 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:57.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:57.710 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:57.811 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:57.850 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:57.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:57.851 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:57.952 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:57.979 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:57.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:57.980 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:58.081 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:58.108 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:58.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:58.109 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:58.110 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 46 16 72116 72100 97.9386 79 0.0066063 0.0102051 2024-09-11T07:46:58.209 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:58.235 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:58.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:58.236 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:58.240 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-11T07:46:58.337 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:58.362 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:58.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:58.363 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:58.464 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:58.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:58.491 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:58.491 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:58.592 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:58.650 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:58.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:58.651 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:58.752 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:58.778 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:58.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:58.779 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:58.866 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:46:58.879 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:58.903 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:58.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:58.904 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:59.004 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:46:59.031 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:59.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:59.032 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:46:59.110 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 47 16 73220 73204 97.3228 69 0.0163668 0.0102703 2024-09-11T07:46:59.133 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:46:59.180 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:59.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:59.182 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:46:59.283 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:59.308 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:59.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:59.310 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:59.411 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:46:59.437 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:59.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:59.438 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:46:59.470 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is unset 2024-09-11T07:46:59.479 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:59 smithi022 ceph-mon[29462]: pgmap v956: 158 pgs: 158 active+clean; 4.1 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 639 B/s rd, 50 MiB/s wr, 798 op/s 2024-09-11T07:46:59.479 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:59 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2657502647' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:46:59.479 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:46:59 smithi022 ceph-mon[29462]: from='client.26047 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:46:59.538 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:46:59.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:59.582 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:46:59.582 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:46:59.683 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:46:59.708 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:59.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:59.710 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:46:59.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:59 smithi104 ceph-mon[33507]: pgmap v956: 158 pgs: 158 active+clean; 4.1 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 639 B/s rd, 50 MiB/s wr, 798 op/s 2024-09-11T07:46:59.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:59 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2657502647' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:46:59.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:46:59 smithi104 ceph-mon[33507]: from='client.26047 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:46:59.789 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:59 smithi022 ceph-mon[37330]: pgmap v956: 158 pgs: 158 active+clean; 4.1 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 639 B/s rd, 50 MiB/s wr, 798 op/s 2024-09-11T07:46:59.789 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:59 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2657502647' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:46:59.789 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:46:59 smithi022 ceph-mon[37330]: from='client.26047 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:46:59.811 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:59.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:59.839 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:59.839 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:46:59.940 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:46:59.966 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:46:59.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:46:59.967 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:00.067 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:00.094 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:00.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:00.095 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:00.110 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 48 16 74354 74338 96.7718 70.875 0.0110874 0.0103282 2024-09-11T07:47:00.196 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:00.236 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:00.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:00.237 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:00.338 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:00.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:00.366 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:00.367 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:00.468 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:00.475 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:47:00 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:47:00.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:00.494 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:00.494 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:00.595 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:00.624 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:00.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:00.626 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:00.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:47:00 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:47:00] "GET /metrics HTTP/1.1" 200 225146 "" "Prometheus/2.43.0" 2024-09-11T07:47:00.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:00 smithi022 ceph-mon[29462]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:47:00.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:00 smithi022 ceph-mon[29462]: from='client.26047 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:47:00.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:00 smithi022 ceph-mon[29462]: osdmap e431: 8 total, 8 up, 7 in 2024-09-11T07:47:00.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:00 smithi022 ceph-mon[29462]: 2.19 scrub starts 2024-09-11T07:47:00.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:00 smithi022 ceph-mon[29462]: 2.19 scrub ok 2024-09-11T07:47:00.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:00 smithi022 ceph-mon[37330]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:47:00.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:00 smithi022 ceph-mon[37330]: from='client.26047 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:47:00.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:00 smithi022 ceph-mon[37330]: osdmap e431: 8 total, 8 up, 7 in 2024-09-11T07:47:00.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:00 smithi022 ceph-mon[37330]: 2.19 scrub starts 2024-09-11T07:47:00.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:00 smithi022 ceph-mon[37330]: 2.19 scrub ok 2024-09-11T07:47:00.726 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:00.771 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:00.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:00.772 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:00.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:00 smithi104 ceph-mon[33507]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:47:00.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:00 smithi104 ceph-mon[33507]: from='client.26047 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:47:00.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:00 smithi104 ceph-mon[33507]: osdmap e431: 8 total, 8 up, 7 in 2024-09-11T07:47:00.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:00 smithi104 ceph-mon[33507]: 2.19 scrub starts 2024-09-11T07:47:00.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:00 smithi104 ceph-mon[33507]: 2.19 scrub ok 2024-09-11T07:47:00.873 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:00.897 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:00.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:00.898 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:00.999 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:01.024 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:01.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:01.026 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:01.101 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:47:01.113 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 49 16 75474 75458 96.2221 70 0.014586 0.0103869 2024-09-11T07:47:01.126 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:01.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:01.175 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:01.176 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:01.277 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:01.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:01.329 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:01.329 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:01.430 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:01.469 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:01.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:01.470 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:01.571 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:01.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:01.599 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:01.599 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:01.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:01 smithi022 ceph-mon[37330]: 1.9 scrub starts 2024-09-11T07:47:01.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:01 smithi022 ceph-mon[37330]: 1.9 scrub ok 2024-09-11T07:47:01.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:01 smithi022 ceph-mon[37330]: 5.12 scrub starts 2024-09-11T07:47:01.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:01 smithi022 ceph-mon[37330]: 5.12 scrub ok 2024-09-11T07:47:01.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:01 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:47:01.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:01 smithi022 ceph-mon[37330]: pgmap v958: 158 pgs: 158 active+clean; 4.5 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 1.3 KiB/s rd, 97 MiB/s wr, 1.55k op/s 2024-09-11T07:47:01.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:01 smithi022 ceph-mon[37330]: 4.13 scrub starts 2024-09-11T07:47:01.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:01 smithi022 ceph-mon[37330]: 4.13 scrub ok 2024-09-11T07:47:01.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:01 smithi022 ceph-mon[37330]: 2.1e scrub starts 2024-09-11T07:47:01.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:01 smithi022 ceph-mon[37330]: 2.1e scrub ok 2024-09-11T07:47:01.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:01 smithi022 ceph-mon[37330]: osdmap e432: 8 total, 8 up, 7 in 2024-09-11T07:47:01.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:01 smithi022 ceph-mon[29462]: 1.9 scrub starts 2024-09-11T07:47:01.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:01 smithi022 ceph-mon[29462]: 1.9 scrub ok 2024-09-11T07:47:01.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:01 smithi022 ceph-mon[29462]: 5.12 scrub starts 2024-09-11T07:47:01.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:01 smithi022 ceph-mon[29462]: 5.12 scrub ok 2024-09-11T07:47:01.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:01 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:47:01.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:01 smithi022 ceph-mon[29462]: pgmap v958: 158 pgs: 158 active+clean; 4.5 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 1.3 KiB/s rd, 97 MiB/s wr, 1.55k op/s 2024-09-11T07:47:01.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:01 smithi022 ceph-mon[29462]: 4.13 scrub starts 2024-09-11T07:47:01.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:01 smithi022 ceph-mon[29462]: 4.13 scrub ok 2024-09-11T07:47:01.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:01 smithi022 ceph-mon[29462]: 2.1e scrub starts 2024-09-11T07:47:01.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:01 smithi022 ceph-mon[29462]: 2.1e scrub ok 2024-09-11T07:47:01.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:01 smithi022 ceph-mon[29462]: osdmap e432: 8 total, 8 up, 7 in 2024-09-11T07:47:01.700 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:01.725 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:01.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:01.726 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:01.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:01 smithi104 ceph-mon[33507]: 1.9 scrub starts 2024-09-11T07:47:01.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:01 smithi104 ceph-mon[33507]: 1.9 scrub ok 2024-09-11T07:47:01.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:01 smithi104 ceph-mon[33507]: 5.12 scrub starts 2024-09-11T07:47:01.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:01 smithi104 ceph-mon[33507]: 5.12 scrub ok 2024-09-11T07:47:01.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:01 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:47:01.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:01 smithi104 ceph-mon[33507]: pgmap v958: 158 pgs: 158 active+clean; 4.5 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 1.3 KiB/s rd, 97 MiB/s wr, 1.55k op/s 2024-09-11T07:47:01.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:01 smithi104 ceph-mon[33507]: 4.13 scrub starts 2024-09-11T07:47:01.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:01 smithi104 ceph-mon[33507]: 4.13 scrub ok 2024-09-11T07:47:01.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:01 smithi104 ceph-mon[33507]: 2.1e scrub starts 2024-09-11T07:47:01.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:01 smithi104 ceph-mon[33507]: 2.1e scrub ok 2024-09-11T07:47:01.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:01 smithi104 ceph-mon[33507]: osdmap e432: 8 total, 8 up, 7 in 2024-09-11T07:47:01.827 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:01.868 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:01.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:01.869 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:01.970 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:01.997 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:01.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:01.998 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:02.099 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:02.112 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 50 16 76635 76619 95.7489 72.5625 0.00344863 0.0104384 2024-09-11T07:47:02.121 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:02.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:02.122 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:02.223 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:02.248 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:02.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:02.249 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:02.350 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:02.395 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:02.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:02.396 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:02.497 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:02.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:02.519 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:02.520 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:02.563 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T07:47:02.620 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:02.645 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:02.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:02.646 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:02.745 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:02 smithi022 ceph-mon[29462]: 4.15 scrub starts 2024-09-11T07:47:02.745 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:02 smithi022 ceph-mon[29462]: 4.15 scrub ok 2024-09-11T07:47:02.745 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:02 smithi022 ceph-mon[29462]: 2.7 scrub starts 2024-09-11T07:47:02.745 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:02 smithi022 ceph-mon[29462]: 2.7 scrub ok 2024-09-11T07:47:02.746 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:02 smithi022 ceph-mon[29462]: 2.2 scrub starts 2024-09-11T07:47:02.746 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:02 smithi022 ceph-mon[29462]: 2.2 scrub ok 2024-09-11T07:47:02.746 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:02 smithi022 ceph-mon[29462]: 4.a scrub starts 2024-09-11T07:47:02.746 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:02 smithi022 ceph-mon[29462]: 4.a scrub ok 2024-09-11T07:47:02.746 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:02 smithi022 ceph-mon[37330]: 4.15 scrub starts 2024-09-11T07:47:02.746 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:02 smithi022 ceph-mon[37330]: 4.15 scrub ok 2024-09-11T07:47:02.746 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:02 smithi022 ceph-mon[37330]: 2.7 scrub starts 2024-09-11T07:47:02.746 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:02 smithi022 ceph-mon[37330]: 2.7 scrub ok 2024-09-11T07:47:02.746 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:02 smithi022 ceph-mon[37330]: 2.2 scrub starts 2024-09-11T07:47:02.746 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:02 smithi022 ceph-mon[37330]: 2.2 scrub ok 2024-09-11T07:47:02.746 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:02 smithi022 ceph-mon[37330]: 4.a scrub starts 2024-09-11T07:47:02.746 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:02 smithi022 ceph-mon[37330]: 4.a scrub ok 2024-09-11T07:47:02.747 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:02.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:02 smithi104 ceph-mon[33507]: 4.15 scrub starts 2024-09-11T07:47:02.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:02 smithi104 ceph-mon[33507]: 4.15 scrub ok 2024-09-11T07:47:02.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:02 smithi104 ceph-mon[33507]: 2.7 scrub starts 2024-09-11T07:47:02.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:02 smithi104 ceph-mon[33507]: 2.7 scrub ok 2024-09-11T07:47:02.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:02 smithi104 ceph-mon[33507]: 2.2 scrub starts 2024-09-11T07:47:02.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:02 smithi104 ceph-mon[33507]: 2.2 scrub ok 2024-09-11T07:47:02.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:02 smithi104 ceph-mon[33507]: 4.a scrub starts 2024-09-11T07:47:02.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:02 smithi104 ceph-mon[33507]: 4.a scrub ok 2024-09-11T07:47:02.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:02.810 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:02.811 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:02.824 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:47:02.824 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:47:02.824 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-11T07:47:02.824 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T07:47:02.824 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T07:47:02.825 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 config set filestore_inject_stall 3 2024-09-11T07:47:02.912 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:02.935 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:02.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:02.936 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:03.037 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:03.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:03.082 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:03.082 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:03.112 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 51 16 77506 77490 94.938 54.4375 0.0213179 0.0105276 2024-09-11T07:47:03.183 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:03.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:03.205 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:03.205 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:03.306 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:03.331 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:03.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:03.333 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:03.434 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:03.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:03.481 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:03.481 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:03.582 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:03.609 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:03.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:03.610 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:03.711 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:03.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:03.733 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:03.733 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:03.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:03 smithi104 ceph-mon[33507]: 5.6 scrub starts 2024-09-11T07:47:03.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:03 smithi104 ceph-mon[33507]: 5.6 scrub ok 2024-09-11T07:47:03.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:03 smithi104 ceph-mon[33507]: pgmap v960: 158 pgs: 158 active+clean; 4.5 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 770 B/s rd, 55 MiB/s wr, 880 op/s 2024-09-11T07:47:03.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:03 smithi104 ceph-mon[33507]: 6.1f scrub starts 2024-09-11T07:47:03.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:03 smithi104 ceph-mon[33507]: 6.1f scrub ok 2024-09-11T07:47:03.834 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:03.855 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:03.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:03.856 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:03.884 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:03 smithi022 ceph-mon[29462]: 5.6 scrub starts 2024-09-11T07:47:03.884 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:03 smithi022 ceph-mon[29462]: 5.6 scrub ok 2024-09-11T07:47:03.884 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:03 smithi022 ceph-mon[29462]: pgmap v960: 158 pgs: 158 active+clean; 4.5 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 770 B/s rd, 55 MiB/s wr, 880 op/s 2024-09-11T07:47:03.884 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:03 smithi022 ceph-mon[29462]: 6.1f scrub starts 2024-09-11T07:47:03.884 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:03 smithi022 ceph-mon[29462]: 6.1f scrub ok 2024-09-11T07:47:03.885 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:03 smithi022 ceph-mon[37330]: 5.6 scrub starts 2024-09-11T07:47:03.885 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:03 smithi022 ceph-mon[37330]: 5.6 scrub ok 2024-09-11T07:47:03.885 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:03 smithi022 ceph-mon[37330]: pgmap v960: 158 pgs: 158 active+clean; 4.5 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 770 B/s rd, 55 MiB/s wr, 880 op/s 2024-09-11T07:47:03.885 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:03 smithi022 ceph-mon[37330]: 6.1f scrub starts 2024-09-11T07:47:03.885 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:03 smithi022 ceph-mon[37330]: 6.1f scrub ok 2024-09-11T07:47:03.957 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:03.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:03.989 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:03.990 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:04.090 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:04.112 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 52 16 78398 78382 94.1845 55.75 0.0256683 0.0106129 2024-09-11T07:47:04.115 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:04.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:04.116 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:04.216 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:04.238 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:04.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:04.239 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:04.339 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:04.364 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:04.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:04.365 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:04.466 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:04.505 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:04.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:04.506 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:04.521 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:47:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:47:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:47:04.607 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:04.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:04.649 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:04.649 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:04.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:04 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:47:04.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:04 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:47:04.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:04 smithi022 ceph-mon[29462]: osdmap e433: 8 total, 8 up, 7 in 2024-09-11T07:47:04.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:04 smithi022 ceph-mon[29462]: 4.19 scrub starts 2024-09-11T07:47:04.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:04 smithi022 ceph-mon[29462]: 4.19 scrub ok 2024-09-11T07:47:04.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:04 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2779864631' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:47:04.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:04 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:47:04.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:04 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:47:04.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:04 smithi022 ceph-mon[37330]: osdmap e433: 8 total, 8 up, 7 in 2024-09-11T07:47:04.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:04 smithi022 ceph-mon[37330]: 4.19 scrub starts 2024-09-11T07:47:04.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:04 smithi022 ceph-mon[37330]: 4.19 scrub ok 2024-09-11T07:47:04.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:04 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2779864631' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:47:04.750 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:04.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:04 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:47:04.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:04 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:47:04.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:04 smithi104 ceph-mon[33507]: osdmap e433: 8 total, 8 up, 7 in 2024-09-11T07:47:04.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:04 smithi104 ceph-mon[33507]: 4.19 scrub starts 2024-09-11T07:47:04.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:04 smithi104 ceph-mon[33507]: 4.19 scrub ok 2024-09-11T07:47:04.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:04 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2779864631' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:47:04.791 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:04.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:04.792 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:04.893 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:04.915 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:04.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:04.916 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:05.017 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:05.043 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:05.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:05.044 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:05.085 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:47:05.085 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:47:05.085 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:47:05.085 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:47:05.086 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:47:05.086 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:47:05.086 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:47:05.086 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:47:05.086 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:47:05.086 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:47:05.086 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:47:05.086 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:47:05.086 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:47:05.086 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:47:05.086 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:47:05.087 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:47:05.113 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 53 16 79303 79287 93.4746 56.5625 0.0124024 0.0106934 2024-09-11T07:47:05.145 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:05.168 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:05.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:05.170 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:05.270 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:05.318 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:05.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:05.320 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:05.420 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:05.483 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:05.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:05.485 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:05.585 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:05.612 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:05.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:05.613 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:05.714 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:05.739 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:05.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:05.740 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:05.840 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:05.862 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:05.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:05.863 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:05.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:05 smithi022 ceph-mon[29462]: 1.6 scrub starts 2024-09-11T07:47:05.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:05 smithi022 ceph-mon[29462]: 1.6 scrub ok 2024-09-11T07:47:05.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:05 smithi022 ceph-mon[29462]: pgmap v962: 158 pgs: 158 active+clean; 4.6 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.7 KiB/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-11T07:47:05.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:05 smithi022 ceph-mon[37330]: 1.6 scrub starts 2024-09-11T07:47:05.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:05 smithi022 ceph-mon[37330]: 1.6 scrub ok 2024-09-11T07:47:05.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:05 smithi022 ceph-mon[37330]: pgmap v962: 158 pgs: 158 active+clean; 4.6 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.7 KiB/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-11T07:47:05.965 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:06.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:06.007 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:06.007 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:06.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:05 smithi104 ceph-mon[33507]: 1.6 scrub starts 2024-09-11T07:47:06.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:05 smithi104 ceph-mon[33507]: 1.6 scrub ok 2024-09-11T07:47:06.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:05 smithi104 ceph-mon[33507]: pgmap v962: 158 pgs: 158 active+clean; 4.6 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.7 KiB/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-11T07:47:06.108 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:06.113 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 54 16 80396 80380 93.0086 68.3125 0.0128173 0.010747 2024-09-11T07:47:06.148 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:06.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:06.149 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:06.249 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:06.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:06.289 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:06.289 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:06.390 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:06.416 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:06.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:06.417 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:06.518 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:06.564 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:06.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:06.565 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:06.666 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:06.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:06.706 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:06.707 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:06.807 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:06.832 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:06.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:06.833 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:06.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:06 smithi022 ceph-mon[29462]: 6.15 scrub starts 2024-09-11T07:47:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:06 smithi022 ceph-mon[29462]: 6.15 scrub ok 2024-09-11T07:47:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:06 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:47:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:06 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:47:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:06 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:47:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:06 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:47:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:06 smithi022 ceph-mon[29462]: osdmap e434: 8 total, 8 up, 7 in 2024-09-11T07:47:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:06 smithi022 ceph-mon[29462]: 5.13 scrub starts 2024-09-11T07:47:06.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:06 smithi022 ceph-mon[29462]: 5.13 scrub ok 2024-09-11T07:47:06.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:06 smithi022 ceph-mon[37330]: 6.15 scrub starts 2024-09-11T07:47:06.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:06 smithi022 ceph-mon[37330]: 6.15 scrub ok 2024-09-11T07:47:06.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:06 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:47:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:06 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:47:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:06 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:47:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:06 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:47:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:06 smithi022 ceph-mon[37330]: osdmap e434: 8 total, 8 up, 7 in 2024-09-11T07:47:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:06 smithi022 ceph-mon[37330]: 5.13 scrub starts 2024-09-11T07:47:06.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:06 smithi022 ceph-mon[37330]: 5.13 scrub ok 2024-09-11T07:47:06.933 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:06.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:06.971 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:06.971 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:07.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:06 smithi104 ceph-mon[33507]: 6.15 scrub starts 2024-09-11T07:47:07.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:06 smithi104 ceph-mon[33507]: 6.15 scrub ok 2024-09-11T07:47:07.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:06 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:47:07.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:06 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:47:07.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:06 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:47:07.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:06 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:47:07.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:06 smithi104 ceph-mon[33507]: osdmap e434: 8 total, 8 up, 7 in 2024-09-11T07:47:07.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:06 smithi104 ceph-mon[33507]: 5.13 scrub starts 2024-09-11T07:47:07.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:06 smithi104 ceph-mon[33507]: 5.13 scrub ok 2024-09-11T07:47:07.072 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:07.096 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:07.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:07.097 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:07.113 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 55 15 81395 81380 92.4539 62.5 0.0108963 0.0108114 2024-09-11T07:47:07.198 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:07.238 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:07.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:07.239 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:07.340 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:07.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:07.373 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:07.374 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:07.474 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:07.499 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:07.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:07.500 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:07.601 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:07.624 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:07.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:07.625 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:07.726 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:07.764 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:07.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:07.765 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:07.866 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:07.894 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:07.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:07.895 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:07.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:07 smithi022 ceph-mon[29462]: 2.5 scrub starts 2024-09-11T07:47:07.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:07 smithi022 ceph-mon[29462]: 2.5 scrub ok 2024-09-11T07:47:07.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:07 smithi022 ceph-mon[29462]: pgmap v964: 158 pgs: 158 active+clean; 4.8 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 53 MiB/s wr, 841 op/s 2024-09-11T07:47:07.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:07 smithi022 ceph-mon[29462]: 1.4 scrub starts 2024-09-11T07:47:07.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:07 smithi022 ceph-mon[29462]: 1.4 scrub ok 2024-09-11T07:47:07.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:07 smithi022 ceph-mon[37330]: 2.5 scrub starts 2024-09-11T07:47:07.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:07 smithi022 ceph-mon[37330]: 2.5 scrub ok 2024-09-11T07:47:07.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:07 smithi022 ceph-mon[37330]: pgmap v964: 158 pgs: 158 active+clean; 4.8 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 53 MiB/s wr, 841 op/s 2024-09-11T07:47:07.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:07 smithi022 ceph-mon[37330]: 1.4 scrub starts 2024-09-11T07:47:07.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:07 smithi022 ceph-mon[37330]: 1.4 scrub ok 2024-09-11T07:47:07.995 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:08.022 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:08.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:08.024 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:08.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:07 smithi104 ceph-mon[33507]: 2.5 scrub starts 2024-09-11T07:47:08.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:07 smithi104 ceph-mon[33507]: 2.5 scrub ok 2024-09-11T07:47:08.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:07 smithi104 ceph-mon[33507]: pgmap v964: 158 pgs: 158 active+clean; 4.8 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 53 MiB/s wr, 841 op/s 2024-09-11T07:47:08.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:07 smithi104 ceph-mon[33507]: 1.4 scrub starts 2024-09-11T07:47:08.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:07 smithi104 ceph-mon[33507]: 1.4 scrub ok 2024-09-11T07:47:08.113 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 56 16 82525 82509 92.063 70.5625 0.00830611 0.0108576 2024-09-11T07:47:08.124 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:08.161 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:08.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:08.162 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:08.263 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:08.311 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:08.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:08.312 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:08.413 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:08.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:08.455 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:08.456 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:08.556 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:08.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:08.594 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:08.594 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:08.695 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:08.723 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:08.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:08.724 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:08.825 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:08.851 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:08.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:08.852 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:08.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:08 smithi022 ceph-mon[29462]: 5.d scrub starts 2024-09-11T07:47:08.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:08 smithi022 ceph-mon[29462]: 5.d scrub ok 2024-09-11T07:47:08.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:47:08.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:08 smithi022 ceph-mon[37330]: 5.d scrub starts 2024-09-11T07:47:08.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:08 smithi022 ceph-mon[37330]: 5.d scrub ok 2024-09-11T07:47:08.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:47:08.953 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:08.990 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:08.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:08.992 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:09.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:08 smithi104 ceph-mon[33507]: 5.d scrub starts 2024-09-11T07:47:09.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:08 smithi104 ceph-mon[33507]: 5.d scrub ok 2024-09-11T07:47:09.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:47:09.093 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:09.113 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 57 16 83589 83573 91.6144 66.5 0.0733351 0.0108992 2024-09-11T07:47:09.115 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:09.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:09.116 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:09.217 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:09.260 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:09.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:09.261 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:09.362 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:09.405 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:09.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:09.406 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:09.507 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:09.549 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:09.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:09.550 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:09.651 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:09.692 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:09.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:09.693 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:09.793 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:09.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:09.821 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:09.821 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:09.893 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:09 smithi022 ceph-mon[29462]: 6.f scrub starts 2024-09-11T07:47:09.894 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:09 smithi022 ceph-mon[29462]: 6.f scrub ok 2024-09-11T07:47:09.894 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:09 smithi022 ceph-mon[29462]: pgmap v965: 158 pgs: 158 active+clean; 4.8 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 41 MiB/s wr, 661 op/s 2024-09-11T07:47:09.894 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:09 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:47:09.894 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:09 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:47:09.894 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:09 smithi022 ceph-mon[37330]: 6.f scrub starts 2024-09-11T07:47:09.894 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:09 smithi022 ceph-mon[37330]: 6.f scrub ok 2024-09-11T07:47:09.894 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:09 smithi022 ceph-mon[37330]: pgmap v965: 158 pgs: 158 active+clean; 4.8 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 41 MiB/s wr, 661 op/s 2024-09-11T07:47:09.894 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:09 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:47:09.894 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:09 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:47:09.922 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:09.964 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:09.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:09.965 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:10.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:09 smithi104 ceph-mon[33507]: 6.f scrub starts 2024-09-11T07:47:10.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:09 smithi104 ceph-mon[33507]: 6.f scrub ok 2024-09-11T07:47:10.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:09 smithi104 ceph-mon[33507]: pgmap v965: 158 pgs: 158 active+clean; 4.8 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 41 MiB/s wr, 661 op/s 2024-09-11T07:47:10.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:09 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:47:10.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:09 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:47:10.066 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:10.117 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 58 16 84790 84774 91.3234 75.0625 0.0126735 0.0109438 2024-09-11T07:47:10.125 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:10.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:10.129 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:10.230 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:10.255 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:10.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:10.256 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:10.357 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:10.397 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:10.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:10.398 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:10.499 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:10.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:47:10 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:47:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:10.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:10.548 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:10.649 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:10.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:47:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:47:10] "GET /metrics HTTP/1.1" 200 225176 "" "Prometheus/2.43.0" 2024-09-11T07:47:10.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:10 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:47:10.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:10 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:47:10.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:10 smithi022 ceph-mon[29462]: 1.a scrub starts 2024-09-11T07:47:10.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:10 smithi022 ceph-mon[29462]: 1.a scrub ok 2024-09-11T07:47:10.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:10 smithi022 ceph-mon[29462]: osdmap e435: 8 total, 8 up, 7 in 2024-09-11T07:47:10.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:10 smithi022 ceph-mon[29462]: 2.9 scrub starts 2024-09-11T07:47:10.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:10 smithi022 ceph-mon[29462]: 2.9 scrub ok 2024-09-11T07:47:10.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:10 smithi022 ceph-mon[29462]: 1.3 scrub starts 2024-09-11T07:47:10.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:10 smithi022 ceph-mon[29462]: 1.3 scrub ok 2024-09-11T07:47:10.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:10 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:47:10.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:10 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:47:10.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:10 smithi022 ceph-mon[37330]: 1.a scrub starts 2024-09-11T07:47:10.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:10 smithi022 ceph-mon[37330]: 1.a scrub ok 2024-09-11T07:47:10.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:10 smithi022 ceph-mon[37330]: osdmap e435: 8 total, 8 up, 7 in 2024-09-11T07:47:10.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:10 smithi022 ceph-mon[37330]: 2.9 scrub starts 2024-09-11T07:47:10.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:10 smithi022 ceph-mon[37330]: 2.9 scrub ok 2024-09-11T07:47:10.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:10 smithi022 ceph-mon[37330]: 1.3 scrub starts 2024-09-11T07:47:10.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:10 smithi022 ceph-mon[37330]: 1.3 scrub ok 2024-09-11T07:47:10.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:10.690 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:10.690 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:10.791 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:10.815 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:10.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:10.817 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:10.918 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:10.944 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:10.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:10.945 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:11.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:10 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:47:11.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:10 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:47:11.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:10 smithi104 ceph-mon[33507]: 1.a scrub starts 2024-09-11T07:47:11.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:10 smithi104 ceph-mon[33507]: 1.a scrub ok 2024-09-11T07:47:11.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:10 smithi104 ceph-mon[33507]: osdmap e435: 8 total, 8 up, 7 in 2024-09-11T07:47:11.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:10 smithi104 ceph-mon[33507]: 2.9 scrub starts 2024-09-11T07:47:11.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:10 smithi104 ceph-mon[33507]: 2.9 scrub ok 2024-09-11T07:47:11.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:10 smithi104 ceph-mon[33507]: 1.3 scrub starts 2024-09-11T07:47:11.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:10 smithi104 ceph-mon[33507]: 1.3 scrub ok 2024-09-11T07:47:11.046 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:11.089 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:11.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:11.090 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:11.117 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 59 16 85937 85921 90.9905 71.6875 0.0222542 0.010984 2024-09-11T07:47:11.121 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:47:11.191 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:11.197 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:47:11.198 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:47:11.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:11.243 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:11.244 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:11.344 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:11.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:11.376 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:11.376 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:11.477 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:11.504 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:11.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:11.505 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:11.605 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:11.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:11.646 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:11.646 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:11.747 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:11.773 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:11.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:11.773 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:11.874 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:11.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:11 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:47:11.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:11 smithi022 ceph-mon[29462]: pgmap v967: 158 pgs: 158 active+clean; 5.1 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 94 MiB/s wr, 1.50k op/s 2024-09-11T07:47:11.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:11 smithi022 ceph-mon[29462]: 6.3 scrub starts 2024-09-11T07:47:11.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:11 smithi022 ceph-mon[29462]: 6.3 scrub ok 2024-09-11T07:47:11.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:11 smithi022 ceph-mon[29462]: osdmap e436: 8 total, 8 up, 7 in 2024-09-11T07:47:11.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:11 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:47:11.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:11 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:47:11.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:11 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:47:11.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:11 smithi022 ceph-mon[37330]: pgmap v967: 158 pgs: 158 active+clean; 5.1 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 94 MiB/s wr, 1.50k op/s 2024-09-11T07:47:11.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:11 smithi022 ceph-mon[37330]: 6.3 scrub starts 2024-09-11T07:47:11.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:11 smithi022 ceph-mon[37330]: 6.3 scrub ok 2024-09-11T07:47:11.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:11 smithi022 ceph-mon[37330]: osdmap e436: 8 total, 8 up, 7 in 2024-09-11T07:47:11.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:11 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:47:11.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:11 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:47:11.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:11.919 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:11.919 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:12.020 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:12.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:11 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:47:12.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:11 smithi104 ceph-mon[33507]: pgmap v967: 158 pgs: 158 active+clean; 5.1 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 94 MiB/s wr, 1.50k op/s 2024-09-11T07:47:12.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:11 smithi104 ceph-mon[33507]: 6.3 scrub starts 2024-09-11T07:47:12.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:11 smithi104 ceph-mon[33507]: 6.3 scrub ok 2024-09-11T07:47:12.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:11 smithi104 ceph-mon[33507]: osdmap e436: 8 total, 8 up, 7 in 2024-09-11T07:47:12.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:11 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:47:12.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:11 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:47:12.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:12.047 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:12.047 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:12.117 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:2024-09-11T07:47:12.116922+0000 min lat: 0.00115663 max lat: 1.02411 avg lat: 0.0110167 2024-09-11T07:47:12.117 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T07:47:12.117 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 60 16 87139 87123 90.726 75.125 0.0129544 0.0110167 2024-09-11T07:47:12.137 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-11T07:47:12.148 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:12.174 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:12.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:12.175 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:12.276 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:12.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:12.325 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:12.326 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:12.426 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:12.450 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:12.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:12.452 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:12.553 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:12.589 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:47:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:12.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:12.607 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:12.708 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:12.734 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:12.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:12.735 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:12.836 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:12.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:12.881 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:12.882 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:12.982 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:13.021 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:13.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:13.022 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:13.117 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 61 16 88263 88247 90.3903 70.25 0.011225 0.0110586 2024-09-11T07:47:13.124 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:13.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:12 smithi022 ceph-mon[29462]: osdmap e437: 8 total, 8 up, 7 in 2024-09-11T07:47:13.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:12 smithi022 ceph-mon[29462]: 1.8 scrub starts 2024-09-11T07:47:13.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:12 smithi022 ceph-mon[29462]: 1.8 scrub ok 2024-09-11T07:47:13.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:12 smithi022 ceph-mon[29462]: pgmap v970: 158 pgs: 158 active+clean; 5.1 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 56 MiB/s wr, 889 op/s 2024-09-11T07:47:13.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:12 smithi022 ceph-mon[37330]: osdmap e437: 8 total, 8 up, 7 in 2024-09-11T07:47:13.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:12 smithi022 ceph-mon[37330]: 1.8 scrub starts 2024-09-11T07:47:13.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:12 smithi022 ceph-mon[37330]: 1.8 scrub ok 2024-09-11T07:47:13.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:12 smithi022 ceph-mon[37330]: pgmap v970: 158 pgs: 158 active+clean; 5.1 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 56 MiB/s wr, 889 op/s 2024-09-11T07:47:13.178 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:13.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:13.179 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:13.275 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:47:13.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:12 smithi104 ceph-mon[33507]: osdmap e437: 8 total, 8 up, 7 in 2024-09-11T07:47:13.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:12 smithi104 ceph-mon[33507]: 1.8 scrub starts 2024-09-11T07:47:13.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:12 smithi104 ceph-mon[33507]: 1.8 scrub ok 2024-09-11T07:47:13.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:12 smithi104 ceph-mon[33507]: pgmap v970: 158 pgs: 158 active+clean; 5.1 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 56 MiB/s wr, 889 op/s 2024-09-11T07:47:13.279 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:13.303 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:13.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:13.304 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:13.404 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:13.449 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:13.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:13.451 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:13.551 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:13.599 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:13.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:13.600 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:13.701 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:13.727 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:13.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:13.728 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:13.829 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:13.854 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:13.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:13.856 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:13.956 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:13.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:13.994 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:13.994 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:14.095 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:14.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:14.117 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:14.117 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:14.117 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 62 16 89383 89367 90.0613 70 0.0140977 0.0110883 2024-09-11T07:47:14.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:13 smithi022 ceph-mon[29462]: 2.11 scrub starts 2024-09-11T07:47:14.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:13 smithi022 ceph-mon[29462]: 2.11 scrub ok 2024-09-11T07:47:14.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:13 smithi022 ceph-mon[29462]: 5.14 scrub starts 2024-09-11T07:47:14.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:13 smithi022 ceph-mon[29462]: 5.14 scrub ok 2024-09-11T07:47:14.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:13 smithi022 ceph-mon[29462]: 4.18 scrub starts 2024-09-11T07:47:14.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:13 smithi022 ceph-mon[29462]: 4.18 scrub ok 2024-09-11T07:47:14.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:13 smithi022 ceph-mon[37330]: 2.11 scrub starts 2024-09-11T07:47:14.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:13 smithi022 ceph-mon[37330]: 2.11 scrub ok 2024-09-11T07:47:14.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:13 smithi022 ceph-mon[37330]: 5.14 scrub starts 2024-09-11T07:47:14.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:13 smithi022 ceph-mon[37330]: 5.14 scrub ok 2024-09-11T07:47:14.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:13 smithi022 ceph-mon[37330]: 4.18 scrub starts 2024-09-11T07:47:14.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:13 smithi022 ceph-mon[37330]: 4.18 scrub ok 2024-09-11T07:47:14.218 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:14.259 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:14.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:14.260 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:14.280 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:47:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:47:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:47:14.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:13 smithi104 ceph-mon[33507]: 2.11 scrub starts 2024-09-11T07:47:14.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:13 smithi104 ceph-mon[33507]: 2.11 scrub ok 2024-09-11T07:47:14.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:13 smithi104 ceph-mon[33507]: 5.14 scrub starts 2024-09-11T07:47:14.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:13 smithi104 ceph-mon[33507]: 5.14 scrub ok 2024-09-11T07:47:14.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:13 smithi104 ceph-mon[33507]: 4.18 scrub starts 2024-09-11T07:47:14.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:13 smithi104 ceph-mon[33507]: 4.18 scrub ok 2024-09-11T07:47:14.361 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:14.387 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:14.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:14.388 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:14.489 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:14.529 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:14.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:14.530 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:14.631 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:14.655 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:14.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:14.656 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:14.757 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:14.795 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:14.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:14.796 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:14.897 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:14.939 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:14.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:14.940 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:15.041 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:15.085 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:15.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:15.086 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:15.117 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 63 16 90584 90568 89.8231 75.0625 0.0126272 0.0111279 2024-09-11T07:47:15.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:14 smithi022 ceph-mon[29462]: 5.0 scrub starts 2024-09-11T07:47:15.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:14 smithi022 ceph-mon[29462]: 5.0 scrub ok 2024-09-11T07:47:15.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:14 smithi022 ceph-mon[29462]: 6.16 scrub starts 2024-09-11T07:47:15.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:14 smithi022 ceph-mon[29462]: 6.16 scrub ok 2024-09-11T07:47:15.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:14 smithi022 ceph-mon[29462]: pgmap v971: 158 pgs: 158 active+clean; 5.2 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 71 MiB/s wr, 1.13k op/s 2024-09-11T07:47:15.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:14 smithi022 ceph-mon[37330]: 5.0 scrub starts 2024-09-11T07:47:15.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:14 smithi022 ceph-mon[37330]: 5.0 scrub ok 2024-09-11T07:47:15.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:14 smithi022 ceph-mon[37330]: 6.16 scrub starts 2024-09-11T07:47:15.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:14 smithi022 ceph-mon[37330]: 6.16 scrub ok 2024-09-11T07:47:15.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:14 smithi022 ceph-mon[37330]: pgmap v971: 158 pgs: 158 active+clean; 5.2 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 71 MiB/s wr, 1.13k op/s 2024-09-11T07:47:15.187 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:15.229 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:15.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:15.230 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:15.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:14 smithi104 ceph-mon[33507]: 5.0 scrub starts 2024-09-11T07:47:15.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:14 smithi104 ceph-mon[33507]: 5.0 scrub ok 2024-09-11T07:47:15.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:14 smithi104 ceph-mon[33507]: 6.16 scrub starts 2024-09-11T07:47:15.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:14 smithi104 ceph-mon[33507]: 6.16 scrub ok 2024-09-11T07:47:15.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:14 smithi104 ceph-mon[33507]: pgmap v971: 158 pgs: 158 active+clean; 5.2 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 71 MiB/s wr, 1.13k op/s 2024-09-11T07:47:15.331 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:15.358 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:15.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:15.359 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:15.459 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:15.485 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:15.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:15.486 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:15.586 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:15.627 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:15.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:15.628 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:15.729 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:15.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:15.777 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:15.778 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:15.878 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:15.910 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:47:15.921 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:47:15.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:15.935 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:15.936 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:16.036 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:16.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:16.087 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:16.087 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:16.119 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 64 16 91729 91713 89.5373 71.5625 0.0188684 0.0111629 2024-09-11T07:47:16.188 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:16.243 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:16.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:16.244 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:16.345 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:16.370 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:16.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:16.371 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:16.472 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:16.513 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:16.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:16.514 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:16.615 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:16.657 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:16.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:16.658 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:16.758 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:16.803 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:16.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:16.804 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:16.905 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:16.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:16 smithi022 ceph-mon[29462]: 4.f scrub starts 2024-09-11T07:47:16.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:16 smithi022 ceph-mon[29462]: 4.f scrub ok 2024-09-11T07:47:16.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:16 smithi022 ceph-mon[29462]: osdmap e438: 8 total, 8 up, 7 in 2024-09-11T07:47:16.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:16 smithi022 ceph-mon[29462]: pgmap v973: 158 pgs: 158 active+clean; 5.5 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 60 MiB/s wr, 962 op/s 2024-09-11T07:47:16.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:16 smithi022 ceph-mon[37330]: 4.f scrub starts 2024-09-11T07:47:16.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:16 smithi022 ceph-mon[37330]: 4.f scrub ok 2024-09-11T07:47:16.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:16 smithi022 ceph-mon[37330]: osdmap e438: 8 total, 8 up, 7 in 2024-09-11T07:47:16.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:16 smithi022 ceph-mon[37330]: pgmap v973: 158 pgs: 158 active+clean; 5.5 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 60 MiB/s wr, 962 op/s 2024-09-11T07:47:16.948 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:16.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:16.949 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:16 smithi104 ceph-mon[33507]: 4.f scrub starts 2024-09-11T07:47:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:16 smithi104 ceph-mon[33507]: 4.f scrub ok 2024-09-11T07:47:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:16 smithi104 ceph-mon[33507]: osdmap e438: 8 total, 8 up, 7 in 2024-09-11T07:47:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:16 smithi104 ceph-mon[33507]: pgmap v973: 158 pgs: 158 active+clean; 5.5 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 60 MiB/s wr, 962 op/s 2024-09-11T07:47:17.050 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:17.092 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:17.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:17.093 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:17.118 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 65 16 92821 92805 89.2096 68.25 0.00929399 0.0112045 2024-09-11T07:47:17.195 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:17.234 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:17.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:17.236 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:17.336 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:17.359 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:17.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:17.360 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:17.461 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:17.486 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:17.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:17.487 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:17.588 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:17.626 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:17.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:17.627 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:17.728 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:17.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:17.771 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:17.771 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:17.794 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is unset 2024-09-11T07:47:17.871 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:17.899 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:17.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:17.900 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:17.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:17 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/127807228' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:47:17.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:17 smithi022 ceph-mon[29462]: from='client.26056 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:47:17.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:17 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/127807228' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:47:17.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:17 smithi022 ceph-mon[37330]: from='client.26056 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:47:18.001 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:18.041 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:18.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:18.042 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:18.118 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 66 16 93987 93971 88.9621 72.875 0.0119042 0.0112359 2024-09-11T07:47:18.144 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:18.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:18.185 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:18.185 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:18.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:17 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/127807228' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:47:18.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:17 smithi104 ceph-mon[33507]: from='client.26056 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:47:18.286 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:18.328 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:18.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:18.329 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:18.430 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:18.468 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:18.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:18.469 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:18.570 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:18.610 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:18.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:18.611 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:18.712 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:18.754 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:18.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:18.755 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:18.855 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:18.892 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:18.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:18.893 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:18.994 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:19.018 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:19.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:19.019 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:19.118 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 67 16 95056 95040 88.6315 66.8125 0.00639989 0.0112773 2024-09-11T07:47:19.119 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:19.144 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:19.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:19.145 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:19.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:18 smithi022 ceph-mon[37330]: 4.11 scrub starts 2024-09-11T07:47:19.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:18 smithi022 ceph-mon[37330]: 4.11 scrub ok 2024-09-11T07:47:19.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:18 smithi022 ceph-mon[37330]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:47:19.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:18 smithi022 ceph-mon[37330]: Cluster is now healthy 2024-09-11T07:47:19.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:18 smithi022 ceph-mon[37330]: from='client.26056 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:47:19.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:18 smithi022 ceph-mon[37330]: osdmap e439: 8 total, 8 up, 7 in 2024-09-11T07:47:19.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:18 smithi022 ceph-mon[37330]: 6.12 scrub starts 2024-09-11T07:47:19.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:18 smithi022 ceph-mon[37330]: 6.12 scrub ok 2024-09-11T07:47:19.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:18 smithi022 ceph-mon[37330]: 6.8 scrub starts 2024-09-11T07:47:19.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:18 smithi022 ceph-mon[37330]: 6.8 scrub ok 2024-09-11T07:47:19.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:18 smithi022 ceph-mon[37330]: 5.2 scrub starts 2024-09-11T07:47:19.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:18 smithi022 ceph-mon[37330]: 5.2 scrub ok 2024-09-11T07:47:19.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:18 smithi022 ceph-mon[37330]: pgmap v975: 158 pgs: 158 active+clean; 5.5 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 55 MiB/s wr, 870 op/s 2024-09-11T07:47:19.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:18 smithi022 ceph-mon[29462]: 4.11 scrub starts 2024-09-11T07:47:19.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:18 smithi022 ceph-mon[29462]: 4.11 scrub ok 2024-09-11T07:47:19.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:18 smithi022 ceph-mon[29462]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:47:19.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:18 smithi022 ceph-mon[29462]: Cluster is now healthy 2024-09-11T07:47:19.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:18 smithi022 ceph-mon[29462]: from='client.26056 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:47:19.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:18 smithi022 ceph-mon[29462]: osdmap e439: 8 total, 8 up, 7 in 2024-09-11T07:47:19.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:18 smithi022 ceph-mon[29462]: 6.12 scrub starts 2024-09-11T07:47:19.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:18 smithi022 ceph-mon[29462]: 6.12 scrub ok 2024-09-11T07:47:19.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:18 smithi022 ceph-mon[29462]: 6.8 scrub starts 2024-09-11T07:47:19.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:18 smithi022 ceph-mon[29462]: 6.8 scrub ok 2024-09-11T07:47:19.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:18 smithi022 ceph-mon[29462]: 5.2 scrub starts 2024-09-11T07:47:19.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:18 smithi022 ceph-mon[29462]: 5.2 scrub ok 2024-09-11T07:47:19.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:18 smithi022 ceph-mon[29462]: pgmap v975: 158 pgs: 158 active+clean; 5.5 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 55 MiB/s wr, 870 op/s 2024-09-11T07:47:19.246 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:19.273 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:19.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:19.274 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:19.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:18 smithi104 ceph-mon[33507]: 4.11 scrub starts 2024-09-11T07:47:19.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:18 smithi104 ceph-mon[33507]: 4.11 scrub ok 2024-09-11T07:47:19.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:18 smithi104 ceph-mon[33507]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:47:19.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:18 smithi104 ceph-mon[33507]: Cluster is now healthy 2024-09-11T07:47:19.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:18 smithi104 ceph-mon[33507]: from='client.26056 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:47:19.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:18 smithi104 ceph-mon[33507]: osdmap e439: 8 total, 8 up, 7 in 2024-09-11T07:47:19.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:18 smithi104 ceph-mon[33507]: 6.12 scrub starts 2024-09-11T07:47:19.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:18 smithi104 ceph-mon[33507]: 6.12 scrub ok 2024-09-11T07:47:19.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:18 smithi104 ceph-mon[33507]: 6.8 scrub starts 2024-09-11T07:47:19.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:18 smithi104 ceph-mon[33507]: 6.8 scrub ok 2024-09-11T07:47:19.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:18 smithi104 ceph-mon[33507]: 5.2 scrub starts 2024-09-11T07:47:19.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:18 smithi104 ceph-mon[33507]: 5.2 scrub ok 2024-09-11T07:47:19.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:18 smithi104 ceph-mon[33507]: pgmap v975: 158 pgs: 158 active+clean; 5.5 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 55 MiB/s wr, 870 op/s 2024-09-11T07:47:19.375 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:19.412 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:19.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:19.414 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:19.515 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:19.566 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:19.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:19.567 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:19.668 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:19.695 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:19.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:19.696 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:19.797 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:19.823 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:19.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:19.824 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:19.925 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:19.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:19.971 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:19.971 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:20.066 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:19 smithi022 ceph-mon[37330]: 2.1c deep-scrub starts 2024-09-11T07:47:20.066 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:19 smithi022 ceph-mon[37330]: 2.1c deep-scrub ok 2024-09-11T07:47:20.066 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:19 smithi022 ceph-mon[29462]: 2.1c deep-scrub starts 2024-09-11T07:47:20.067 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:19 smithi022 ceph-mon[29462]: 2.1c deep-scrub ok 2024-09-11T07:47:20.072 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:20.119 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 68 16 96240 96224 88.4147 74 0.0140797 0.0113048 2024-09-11T07:47:20.120 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:20.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:20.121 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:20.216 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:19 smithi104 ceph-mon[33507]: 2.1c deep-scrub starts 2024-09-11T07:47:20.216 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:19 smithi104 ceph-mon[33507]: 2.1c deep-scrub ok 2024-09-11T07:47:20.222 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:20.246 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:20.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:20.247 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:20.348 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:20.374 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:20.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:20.375 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:20.476 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:20.523 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:20.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:20.524 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:20.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:47:20 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:47:20.626 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:20.652 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:20.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:20.653 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:20.664 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:47:20.664 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "" 2024-09-11T07:47:20.664 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:47:20.664 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:47:20 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:47:20] "GET /metrics HTTP/1.1" 200 225176 "" "Prometheus/2.43.0" 2024-09-11T07:47:20.754 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:20.799 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:20.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:20.800 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:20.900 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:20.916 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:20 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:47:20.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:20 smithi022 ceph-mon[37330]: pgmap v976: 158 pgs: 158 active+clean; 5.8 GiB data, 23 GiB used, 602 GiB / 626 GiB avail; 90 MiB/s wr, 1.44k op/s 2024-09-11T07:47:20.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:20 smithi022 ceph-mon[37330]: osdmap e440: 8 total, 8 up, 7 in 2024-09-11T07:47:20.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:20 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:47:20.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:20 smithi022 ceph-mon[29462]: pgmap v976: 158 pgs: 158 active+clean; 5.8 GiB data, 23 GiB used, 602 GiB / 626 GiB avail; 90 MiB/s wr, 1.44k op/s 2024-09-11T07:47:20.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:20 smithi022 ceph-mon[29462]: osdmap e440: 8 total, 8 up, 7 in 2024-09-11T07:47:20.926 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:20.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:20.927 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:21.028 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:21.053 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:21.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:21.054 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:21.119 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 69 16 97448 97432 88.2274 75.5 0.0116171 0.0113291 2024-09-11T07:47:21.155 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:21.179 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:21.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:21.180 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:21.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:20 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:47:21.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:20 smithi104 ceph-mon[33507]: pgmap v976: 158 pgs: 158 active+clean; 5.8 GiB data, 23 GiB used, 602 GiB / 626 GiB avail; 90 MiB/s wr, 1.44k op/s 2024-09-11T07:47:21.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:20 smithi104 ceph-mon[33507]: osdmap e440: 8 total, 8 up, 7 in 2024-09-11T07:47:21.282 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:21.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:21.327 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:21.328 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:21.428 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:21.457 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:21.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:21.458 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:21.559 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:21.597 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:21.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:21.598 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:21.698 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:21.723 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:21.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:21.724 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:21.825 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:21.851 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:21.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:21.852 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:21.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:21 smithi022 ceph-mon[29462]: 4.10 scrub starts 2024-09-11T07:47:21.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:21 smithi022 ceph-mon[29462]: 4.10 scrub ok 2024-09-11T07:47:21.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:21 smithi022 ceph-mon[29462]: 5.15 scrub starts 2024-09-11T07:47:21.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:21 smithi022 ceph-mon[29462]: 5.15 scrub ok 2024-09-11T07:47:21.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:21 smithi022 ceph-mon[37330]: 4.10 scrub starts 2024-09-11T07:47:21.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:21 smithi022 ceph-mon[37330]: 4.10 scrub ok 2024-09-11T07:47:21.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:21 smithi022 ceph-mon[37330]: 5.15 scrub starts 2024-09-11T07:47:21.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:21 smithi022 ceph-mon[37330]: 5.15 scrub ok 2024-09-11T07:47:21.953 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:21.979 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:21.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:21.980 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:22.081 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:22.109 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:22.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:22.110 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:22.119 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 70 16 98672 98656 88.0598 76.5 0.00987422 0.0113512 2024-09-11T07:47:22.211 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:22.255 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:22.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:22.256 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:22.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:21 smithi104 ceph-mon[33507]: 4.10 scrub starts 2024-09-11T07:47:22.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:21 smithi104 ceph-mon[33507]: 4.10 scrub ok 2024-09-11T07:47:22.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:21 smithi104 ceph-mon[33507]: 5.15 scrub starts 2024-09-11T07:47:22.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:21 smithi104 ceph-mon[33507]: 5.15 scrub ok 2024-09-11T07:47:22.357 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:22.394 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:22.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:22.395 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:22.496 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:22.537 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:22.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:22.539 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:22.639 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:22.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:22.683 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:22.683 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:22.784 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:22.824 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:22.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:22.825 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:22.926 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:22.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:22.951 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:22.951 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:23.052 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:23.093 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:23.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:23.094 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:23.119 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 71 16 99879 99863 87.882 75.4375 0.0142612 0.0113742 2024-09-11T07:47:23.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:22 smithi022 ceph-mon[29462]: 1.5 scrub starts 2024-09-11T07:47:23.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:22 smithi022 ceph-mon[29462]: 1.5 scrub ok 2024-09-11T07:47:23.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:22 smithi022 ceph-mon[29462]: 4.16 deep-scrub starts 2024-09-11T07:47:23.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:22 smithi022 ceph-mon[29462]: 4.16 deep-scrub ok 2024-09-11T07:47:23.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:22 smithi022 ceph-mon[29462]: pgmap v978: 158 pgs: 158 active+clean; 5.8 GiB data, 23 GiB used, 602 GiB / 626 GiB avail; 770 B/s rd, 54 MiB/s wr, 859 op/s 2024-09-11T07:47:23.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:22 smithi022 ceph-mon[37330]: 1.5 scrub starts 2024-09-11T07:47:23.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:22 smithi022 ceph-mon[37330]: 1.5 scrub ok 2024-09-11T07:47:23.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:22 smithi022 ceph-mon[37330]: 4.16 deep-scrub starts 2024-09-11T07:47:23.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:22 smithi022 ceph-mon[37330]: 4.16 deep-scrub ok 2024-09-11T07:47:23.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:22 smithi022 ceph-mon[37330]: pgmap v978: 158 pgs: 158 active+clean; 5.8 GiB data, 23 GiB used, 602 GiB / 626 GiB avail; 770 B/s rd, 54 MiB/s wr, 859 op/s 2024-09-11T07:47:23.195 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:23.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:23.222 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:23.222 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:23.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:22 smithi104 ceph-mon[33507]: 1.5 scrub starts 2024-09-11T07:47:23.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:22 smithi104 ceph-mon[33507]: 1.5 scrub ok 2024-09-11T07:47:23.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:22 smithi104 ceph-mon[33507]: 4.16 deep-scrub starts 2024-09-11T07:47:23.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:22 smithi104 ceph-mon[33507]: 4.16 deep-scrub ok 2024-09-11T07:47:23.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:22 smithi104 ceph-mon[33507]: pgmap v978: 158 pgs: 158 active+clean; 5.8 GiB data, 23 GiB used, 602 GiB / 626 GiB avail; 770 B/s rd, 54 MiB/s wr, 859 op/s 2024-09-11T07:47:23.323 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:23.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:23.393 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:23.393 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:23.494 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:23.521 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:23.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:23.522 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:23.623 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:23.642 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T07:47:23.648 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:23.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:23.649 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:23.750 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:23.801 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:23.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:23.802 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:23.903 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:23.960 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:23.961 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:47:23.961 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:47:23.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:23.962 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:24.063 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:24.090 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:24.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:24.091 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:24.120 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 72 16 100964 100948 87.6032 67.8125 0.0169316 0.0114104 2024-09-11T07:47:24.161 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:24 smithi022 ceph-mon[29462]: 6.5 scrub starts 2024-09-11T07:47:24.162 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:24 smithi022 ceph-mon[29462]: 6.5 scrub ok 2024-09-11T07:47:24.162 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:24 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:47:24.162 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:24 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:47:24.162 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:24 smithi022 ceph-mon[29462]: 4.d scrub starts 2024-09-11T07:47:24.162 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:24 smithi022 ceph-mon[29462]: 4.d scrub ok 2024-09-11T07:47:24.162 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:24 smithi022 ceph-mon[37330]: 6.5 scrub starts 2024-09-11T07:47:24.162 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:24 smithi022 ceph-mon[37330]: 6.5 scrub ok 2024-09-11T07:47:24.163 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:24 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:47:24.163 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:24 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:47:24.163 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:24 smithi022 ceph-mon[37330]: 4.d scrub starts 2024-09-11T07:47:24.163 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:24 smithi022 ceph-mon[37330]: 4.d scrub ok 2024-09-11T07:47:24.192 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:24.219 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:24.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:24.220 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:24.321 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:24.347 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:24.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:24.348 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:24.449 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:24.493 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:24.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:24.494 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:24.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:47:24 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:47:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:47:24.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:24 smithi104 ceph-mon[33507]: 6.5 scrub starts 2024-09-11T07:47:24.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:24 smithi104 ceph-mon[33507]: 6.5 scrub ok 2024-09-11T07:47:24.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:24 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:47:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:24 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:47:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:24 smithi104 ceph-mon[33507]: 4.d scrub starts 2024-09-11T07:47:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:24 smithi104 ceph-mon[33507]: 4.d scrub ok 2024-09-11T07:47:24.595 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:24.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:24.637 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:24.637 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:24.729 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:47:24.729 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:47:24.738 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:24.766 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:47:24.766 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:47:24.779 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:24.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:24.780 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:24.800 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:47:24.800 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:47:24.829 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:47:24.829 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:47:24.861 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:47:24.861 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:47:24.880 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:24.895 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:47:24.895 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:47:24.905 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:24.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:24.906 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:24.927 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:47:24.927 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:47:24.957 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:47:24.957 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:47:25.007 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:25.051 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:25.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:25.053 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:25.121 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 73 15 102083 102068 87.362 70 0.0140097 0.011442 2024-09-11T07:47:25.153 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:25.177 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:25.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:25.179 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:25.182 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-11T07:47:25.280 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:25.305 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:25.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:25.306 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:25.407 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:25.416 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:25 smithi022 ceph-mon[37330]: 6.13 scrub starts 2024-09-11T07:47:25.416 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:25 smithi022 ceph-mon[37330]: 6.13 scrub ok 2024-09-11T07:47:25.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:25 smithi022 ceph-mon[37330]: 2.1f deep-scrub starts 2024-09-11T07:47:25.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:25 smithi022 ceph-mon[37330]: 2.1f deep-scrub ok 2024-09-11T07:47:25.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:25 smithi022 ceph-mon[37330]: pgmap v979: 158 pgs: 158 active+clean; 5.9 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 56 MiB/s wr, 901 op/s 2024-09-11T07:47:25.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:25 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1781614211' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:47:25.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:25 smithi022 ceph-mon[29462]: 6.13 scrub starts 2024-09-11T07:47:25.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:25 smithi022 ceph-mon[29462]: 6.13 scrub ok 2024-09-11T07:47:25.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:25 smithi022 ceph-mon[29462]: 2.1f deep-scrub starts 2024-09-11T07:47:25.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:25 smithi022 ceph-mon[29462]: 2.1f deep-scrub ok 2024-09-11T07:47:25.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:25 smithi022 ceph-mon[29462]: pgmap v979: 158 pgs: 158 active+clean; 5.9 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 56 MiB/s wr, 901 op/s 2024-09-11T07:47:25.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:25 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1781614211' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:47:25.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:25.455 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:25.455 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:25.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:25 smithi104 ceph-mon[33507]: 6.13 scrub starts 2024-09-11T07:47:25.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:25 smithi104 ceph-mon[33507]: 6.13 scrub ok 2024-09-11T07:47:25.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:25 smithi104 ceph-mon[33507]: 2.1f deep-scrub starts 2024-09-11T07:47:25.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:25 smithi104 ceph-mon[33507]: 2.1f deep-scrub ok 2024-09-11T07:47:25.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:25 smithi104 ceph-mon[33507]: pgmap v979: 158 pgs: 158 active+clean; 5.9 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 56 MiB/s wr, 901 op/s 2024-09-11T07:47:25.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:25 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1781614211' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:47:25.556 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:25.583 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:25.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:25.584 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:25.684 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:25.722 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:25.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:25.723 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:25.824 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:25.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:25.920 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:25.921 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:25.921 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:47:25.937 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:47:26.021 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:26.047 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:26.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:26.048 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:26.120 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 74 16 103164 103148 87.0936 67.5 0.00709602 0.0114774 2024-09-11T07:47:26.149 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:26.203 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:26.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:26.204 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:26.305 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:26.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:26 smithi022 ceph-mon[29462]: 3.2 scrub starts 2024-09-11T07:47:26.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:26 smithi022 ceph-mon[29462]: 3.2 scrub ok 2024-09-11T07:47:26.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:26 smithi022 ceph-mon[29462]: 5.17 scrub starts 2024-09-11T07:47:26.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:26 smithi022 ceph-mon[29462]: 5.17 scrub ok 2024-09-11T07:47:26.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:26 smithi022 ceph-mon[37330]: 3.2 scrub starts 2024-09-11T07:47:26.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:26 smithi022 ceph-mon[37330]: 3.2 scrub ok 2024-09-11T07:47:26.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:26 smithi022 ceph-mon[37330]: 5.17 scrub starts 2024-09-11T07:47:26.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:26 smithi022 ceph-mon[37330]: 5.17 scrub ok 2024-09-11T07:47:26.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:26.353 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:26.353 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:26.454 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:26.479 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:26.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:26.480 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:26.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:26 smithi104 ceph-mon[33507]: 3.2 scrub starts 2024-09-11T07:47:26.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:26 smithi104 ceph-mon[33507]: 3.2 scrub ok 2024-09-11T07:47:26.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:26 smithi104 ceph-mon[33507]: 5.17 scrub starts 2024-09-11T07:47:26.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:26 smithi104 ceph-mon[33507]: 5.17 scrub ok 2024-09-11T07:47:26.581 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:26.605 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:26.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:26.606 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:26.706 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:26.745 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:26.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:26.746 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:26.847 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:26.892 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:26.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:26.894 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:26.994 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:27.038 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:27.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:27.039 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:27.120 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 75 16 104342 104326 86.914 73.625 0.0110775 0.0115 2024-09-11T07:47:27.140 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:27.167 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:27.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:27.167 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:27.268 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:27.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:27.295 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:27.295 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:27.372 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:47:27.373 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:47:27.373 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_4 2024-09-11T07:47:27.373 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:47:27.395 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:27.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:27 smithi022 ceph-mon[37330]: 5.1c scrub starts 2024-09-11T07:47:27.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:27 smithi022 ceph-mon[37330]: 5.1c scrub ok 2024-09-11T07:47:27.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:27 smithi022 ceph-mon[37330]: 5.10 scrub starts 2024-09-11T07:47:27.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:27 smithi022 ceph-mon[37330]: 5.10 scrub ok 2024-09-11T07:47:27.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:27 smithi022 ceph-mon[37330]: 5.b scrub starts 2024-09-11T07:47:27.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:27 smithi022 ceph-mon[37330]: 5.b scrub ok 2024-09-11T07:47:27.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:27 smithi022 ceph-mon[37330]: 5.5 scrub starts 2024-09-11T07:47:27.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:27 smithi022 ceph-mon[37330]: 5.5 scrub ok 2024-09-11T07:47:27.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:27 smithi022 ceph-mon[37330]: pgmap v980: 158 pgs: 158 active+clean; 6.2 GiB data, 24 GiB used, 601 GiB / 626 GiB avail; 84 MiB/s wr, 1.34k op/s 2024-09-11T07:47:27.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:27 smithi022 ceph-mon[29462]: 5.1c scrub starts 2024-09-11T07:47:27.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:27 smithi022 ceph-mon[29462]: 5.1c scrub ok 2024-09-11T07:47:27.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:27 smithi022 ceph-mon[29462]: 5.10 scrub starts 2024-09-11T07:47:27.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:27 smithi022 ceph-mon[29462]: 5.10 scrub ok 2024-09-11T07:47:27.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:27 smithi022 ceph-mon[29462]: 5.b scrub starts 2024-09-11T07:47:27.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:27 smithi022 ceph-mon[29462]: 5.b scrub ok 2024-09-11T07:47:27.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:27 smithi022 ceph-mon[29462]: 5.5 scrub starts 2024-09-11T07:47:27.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:27 smithi022 ceph-mon[29462]: 5.5 scrub ok 2024-09-11T07:47:27.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:27 smithi022 ceph-mon[29462]: pgmap v980: 158 pgs: 158 active+clean; 6.2 GiB data, 24 GiB used, 601 GiB / 626 GiB avail; 84 MiB/s wr, 1.34k op/s 2024-09-11T07:47:27.421 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:27.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:27.422 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:27.523 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:27.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:27 smithi104 ceph-mon[33507]: 5.1c scrub starts 2024-09-11T07:47:27.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:27 smithi104 ceph-mon[33507]: 5.1c scrub ok 2024-09-11T07:47:27.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:27 smithi104 ceph-mon[33507]: 5.10 scrub starts 2024-09-11T07:47:27.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:27 smithi104 ceph-mon[33507]: 5.10 scrub ok 2024-09-11T07:47:27.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:27 smithi104 ceph-mon[33507]: 5.b scrub starts 2024-09-11T07:47:27.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:27 smithi104 ceph-mon[33507]: 5.b scrub ok 2024-09-11T07:47:27.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:27 smithi104 ceph-mon[33507]: 5.5 scrub starts 2024-09-11T07:47:27.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:27 smithi104 ceph-mon[33507]: 5.5 scrub ok 2024-09-11T07:47:27.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:27 smithi104 ceph-mon[33507]: pgmap v980: 158 pgs: 158 active+clean; 6.2 GiB data, 24 GiB used, 601 GiB / 626 GiB avail; 84 MiB/s wr, 1.34k op/s 2024-09-11T07:47:27.569 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:27.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:27.570 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:27.671 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:27.725 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:27.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:27.726 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:27.807 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:47:27.827 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:27.898 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:27.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:27.899 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:28.000 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:28.027 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:28.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:28.028 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:28.125 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 76 16 105455 105439 86.6815 69.5625 0.0206133 0.0115305 2024-09-11T07:47:28.129 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:28.153 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:28.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:28.154 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:28.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:28 smithi022 ceph-mon[29462]: 4.14 scrub starts 2024-09-11T07:47:28.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:28 smithi022 ceph-mon[29462]: 4.14 scrub ok 2024-09-11T07:47:28.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:28 smithi022 ceph-mon[29462]: 5.1a scrub starts 2024-09-11T07:47:28.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:28 smithi022 ceph-mon[29462]: 5.1a scrub ok 2024-09-11T07:47:28.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:28 smithi022 ceph-mon[29462]: 5.1b scrub starts 2024-09-11T07:47:28.169 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:28 smithi022 ceph-mon[29462]: 5.1b scrub ok 2024-09-11T07:47:28.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:28 smithi022 ceph-mon[37330]: 4.14 scrub starts 2024-09-11T07:47:28.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:28 smithi022 ceph-mon[37330]: 4.14 scrub ok 2024-09-11T07:47:28.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:28 smithi022 ceph-mon[37330]: 5.1a scrub starts 2024-09-11T07:47:28.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:28 smithi022 ceph-mon[37330]: 5.1a scrub ok 2024-09-11T07:47:28.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:28 smithi022 ceph-mon[37330]: 5.1b scrub starts 2024-09-11T07:47:28.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:28 smithi022 ceph-mon[37330]: 5.1b scrub ok 2024-09-11T07:47:28.255 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:28.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:28.281 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:28.282 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:28.382 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:28.410 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:28.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:28.411 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:28.512 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:28.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:28 smithi104 ceph-mon[33507]: 4.14 scrub starts 2024-09-11T07:47:28.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:28 smithi104 ceph-mon[33507]: 4.14 scrub ok 2024-09-11T07:47:28.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:28 smithi104 ceph-mon[33507]: 5.1a scrub starts 2024-09-11T07:47:28.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:28 smithi104 ceph-mon[33507]: 5.1a scrub ok 2024-09-11T07:47:28.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:28 smithi104 ceph-mon[33507]: 5.1b scrub starts 2024-09-11T07:47:28.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:28 smithi104 ceph-mon[33507]: 5.1b scrub ok 2024-09-11T07:47:28.538 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:28.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:28.539 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:28.640 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:28.662 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:28.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:28.663 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:28.764 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:28.789 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:28.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:28.790 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:28.891 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:28.916 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:28.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:28.917 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:29.020 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:29.046 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:29.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:29.047 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:29.124 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 77 16 106603 106587 86.4875 71.75 0.0104184 0.0115577 2024-09-11T07:47:29.147 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:29.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:29.184 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:29.184 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:29.284 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:29.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:29.329 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:29.330 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:29.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:29 smithi022 ceph-mon[29462]: 6.19 scrub starts 2024-09-11T07:47:29.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:29 smithi022 ceph-mon[29462]: 6.19 scrub ok 2024-09-11T07:47:29.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:29 smithi022 ceph-mon[29462]: pgmap v981: 158 pgs: 158 active+clean; 6.2 GiB data, 24 GiB used, 601 GiB / 626 GiB avail; 72 MiB/s wr, 1.15k op/s 2024-09-11T07:47:29.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:29 smithi022 ceph-mon[37330]: 6.19 scrub starts 2024-09-11T07:47:29.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:29 smithi022 ceph-mon[37330]: 6.19 scrub ok 2024-09-11T07:47:29.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:29 smithi022 ceph-mon[37330]: pgmap v981: 158 pgs: 158 active+clean; 6.2 GiB data, 24 GiB used, 601 GiB / 626 GiB avail; 72 MiB/s wr, 1.15k op/s 2024-09-11T07:47:29.431 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:29.468 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:29.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:29.469 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:29.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:29 smithi104 ceph-mon[33507]: 6.19 scrub starts 2024-09-11T07:47:29.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:29 smithi104 ceph-mon[33507]: 6.19 scrub ok 2024-09-11T07:47:29.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:29 smithi104 ceph-mon[33507]: pgmap v981: 158 pgs: 158 active+clean; 6.2 GiB data, 24 GiB used, 601 GiB / 626 GiB avail; 72 MiB/s wr, 1.15k op/s 2024-09-11T07:47:29.570 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:29.596 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:29.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:29.597 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:29.698 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:29.750 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:29.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:29.751 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:29.852 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:29.910 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:29.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:29.911 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:30.012 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:30.065 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:30.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:30.066 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:30.124 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 78 16 107751 107735 86.2985 71.75 0.0057246 0.0115826 2024-09-11T07:47:30.167 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:30.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:30.212 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:30.212 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:30.295 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:47:30.313 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:30.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:30.380 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:30.380 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:30.416 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:47:30 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:47:30] "GET /metrics HTTP/1.1" 200 225190 "" "Prometheus/2.43.0" 2024-09-11T07:47:30.482 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:30.502 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:30.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:30.503 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:30.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:47:30 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:47:30.604 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:30.627 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:30.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:30.628 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:30.729 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:30.751 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:30.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:30.752 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:30.853 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:30.875 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:30.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:30.876 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:30.977 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:31.003 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:31.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:31.004 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:31.105 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:31.124 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 79 16 108297 108281 85.6382 34.125 0.0388892 0.0116715 2024-09-11T07:47:31.139 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:31.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:31.140 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:31.241 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:31.266 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:31.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:31.267 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:31.367 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:31.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:31.410 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:31.410 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:31.487 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is set 2024-09-11T07:47:31.511 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:31.581 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:31.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:31.582 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:31.683 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:31.726 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:31.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:31.728 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:31.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:31 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:47:31.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:31 smithi104 ceph-mon[33507]: pgmap v982: 158 pgs: 158 active+clean; 6.5 GiB data, 25 GiB used, 600 GiB / 626 GiB avail; 72 MiB/s wr, 1.15k op/s 2024-09-11T07:47:31.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:31 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3869535390' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:47:31.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:31 smithi104 ceph-mon[33507]: from='client.26068 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:47:31.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:31 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:47:31.829 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:31.854 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:31.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:31.856 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:31.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:31 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:47:31.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:31 smithi022 ceph-mon[29462]: pgmap v982: 158 pgs: 158 active+clean; 6.5 GiB data, 25 GiB used, 600 GiB / 626 GiB avail; 72 MiB/s wr, 1.15k op/s 2024-09-11T07:47:31.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:31 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3869535390' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:47:31.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:31 smithi022 ceph-mon[29462]: from='client.26068 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:47:31.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:31 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:47:31.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:31 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:47:31.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:31 smithi022 ceph-mon[37330]: pgmap v982: 158 pgs: 158 active+clean; 6.5 GiB data, 25 GiB used, 600 GiB / 626 GiB avail; 72 MiB/s wr, 1.15k op/s 2024-09-11T07:47:31.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:31 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3869535390' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:47:31.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:31 smithi022 ceph-mon[37330]: from='client.26068 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:47:31.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:31 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:47:31.956 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:31.981 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:31.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:31.982 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:32.085 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:32.112 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:32.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:32.113 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:32.124 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:2024-09-11T07:47:32.124092+0000 min lat: 0.00115663 max lat: 1.02411 avg lat: 0.0117295 2024-09-11T07:47:32.124 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T07:47:32.124 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 80 16 109134 109118 85.2217 52.3125 0.0212173 0.0117295 2024-09-11T07:47:32.214 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:32.242 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:32.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:32.243 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:32.344 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:32.369 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:32.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:32.370 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:32.471 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:32.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:32.499 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:32.499 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:32.599 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:32.623 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:32.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:32.624 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:32.725 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:32.747 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:32.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:32.748 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:32.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:32 smithi022 ceph-mon[29462]: 2.e scrub starts 2024-09-11T07:47:32.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:32 smithi022 ceph-mon[29462]: 2.e scrub ok 2024-09-11T07:47:32.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:32 smithi022 ceph-mon[29462]: 4.6 deep-scrub starts 2024-09-11T07:47:32.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:32 smithi022 ceph-mon[29462]: 4.6 deep-scrub ok 2024-09-11T07:47:32.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:32 smithi022 ceph-mon[29462]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:47:32.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:32 smithi022 ceph-mon[29462]: from='client.26068 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:47:32.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:32 smithi022 ceph-mon[29462]: osdmap e441: 8 total, 8 up, 7 in 2024-09-11T07:47:32.749 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:32 smithi022 ceph-mon[37330]: 2.e scrub starts 2024-09-11T07:47:32.749 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:32 smithi022 ceph-mon[37330]: 2.e scrub ok 2024-09-11T07:47:32.750 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:32 smithi022 ceph-mon[37330]: 4.6 deep-scrub starts 2024-09-11T07:47:32.750 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:32 smithi022 ceph-mon[37330]: 4.6 deep-scrub ok 2024-09-11T07:47:32.750 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:32 smithi022 ceph-mon[37330]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:47:32.750 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:32 smithi022 ceph-mon[37330]: from='client.26068 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:47:32.750 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:32 smithi022 ceph-mon[37330]: osdmap e441: 8 total, 8 up, 7 in 2024-09-11T07:47:32.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:32 smithi104 ceph-mon[33507]: 2.e scrub starts 2024-09-11T07:47:32.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:32 smithi104 ceph-mon[33507]: 2.e scrub ok 2024-09-11T07:47:32.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:32 smithi104 ceph-mon[33507]: 4.6 deep-scrub starts 2024-09-11T07:47:32.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:32 smithi104 ceph-mon[33507]: 4.6 deep-scrub ok 2024-09-11T07:47:32.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:32 smithi104 ceph-mon[33507]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:47:32.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:32 smithi104 ceph-mon[33507]: from='client.26068 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:47:32.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:32 smithi104 ceph-mon[33507]: osdmap e441: 8 total, 8 up, 7 in 2024-09-11T07:47:32.850 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:32.877 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:32.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:32.878 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:32.979 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:33.014 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:33.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:33.015 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:33.116 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:33.126 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 81 16 110158 110142 84.9581 64 0.0130113 0.011766 2024-09-11T07:47:33.150 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:33.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:33.152 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:33.252 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:33.278 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:33.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:33.279 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:33.380 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:33.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:33.424 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:33.424 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:33.525 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:33.551 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:33.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:33.553 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:33.653 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:33.693 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:33.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:33.695 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:33.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:33 smithi104 ceph-mon[33507]: pgmap v984: 158 pgs: 158 active+clean; 6.5 GiB data, 25 GiB used, 600 GiB / 626 GiB avail; 72 MiB/s wr, 1.15k op/s 2024-09-11T07:47:33.795 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:33.835 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:33.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:33.837 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:33.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:33 smithi022 ceph-mon[29462]: pgmap v984: 158 pgs: 158 active+clean; 6.5 GiB data, 25 GiB used, 600 GiB / 626 GiB avail; 72 MiB/s wr, 1.15k op/s 2024-09-11T07:47:33.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:33 smithi022 ceph-mon[37330]: pgmap v984: 158 pgs: 158 active+clean; 6.5 GiB data, 25 GiB used, 600 GiB / 626 GiB avail; 72 MiB/s wr, 1.15k op/s 2024-09-11T07:47:33.938 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:33.966 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:33.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:33.967 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:34.068 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:34.107 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:34.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:34.108 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:34.126 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 82 16 111279 111263 84.7764 70.0625 0.020342 0.0117908 2024-09-11T07:47:34.208 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:34.237 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:34.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:34.238 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:34.339 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:34.364 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:34.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:34.365 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:34.466 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:34.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:34.506 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:34.506 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:34.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:47:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:47:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:47:34.607 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:34.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:34.630 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:34.631 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:34.731 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:34.755 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:34.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:34.756 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:34.857 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:34.903 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:34.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:34.905 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:35.006 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:35.031 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:35.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:35.032 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:35.126 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 83 16 112233 112217 84.4734 59.625 0.00489369 0.0118325 2024-09-11T07:47:35.132 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:35.154 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:35.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:35.155 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:35.256 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:35.282 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:35.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:35.283 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:35.383 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:35.408 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:35.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:35.410 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:35.510 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:35.548 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:35.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:35.550 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:35.591 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:47:35.591 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 20, 2024-09-11T07:47:35.591 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 600, 2024-09-11T07:47:35.591 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:47:35.591 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.591 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:85939 9.14 9:2a2e89c7:::benchmark_data_smithi022_133943_object85938:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e259)", 2024-09-11T07:47:35.591 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:26.333855+0000", 2024-09-11T07:47:35.591 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 549.25302180699998, 2024-09-11T07:47:35.592 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.058456533, 2024-09-11T07:47:35.592 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:47:35.592 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:47:35.592 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:47:35.592 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:47:35.592 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:47:35.592 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 85939 2024-09-11T07:47:35.592 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.592 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:47:35.592 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.593 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:47:35.593 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333855+0000", 2024-09-11T07:47:35.593 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.593 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.593 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.593 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:47:35.593 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333855+0000", 2024-09-11T07:47:35.593 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.593 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.593 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.593 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:47:35.593 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333852+0000", 2024-09-11T07:47:35.593 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:47:35.594 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.594 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.594 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:47:35.594 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333883+0000", 2024-09-11T07:47:35.594 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0558000000000001e-05 2024-09-11T07:47:35.594 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.594 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.594 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:47:35.594 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333898+0000", 2024-09-11T07:47:35.594 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5258e-05 2024-09-11T07:47:35.594 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.594 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.594 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:47:35.595 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333906+0000", 2024-09-11T07:47:35.595 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.892e-06 2024-09-11T07:47:35.595 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.595 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.595 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.595 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.562799+0000", 2024-09-11T07:47:35.595 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.228892701 2024-09-11T07:47:35.595 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.595 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.595 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:47:35.595 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.562805+0000", 2024-09-11T07:47:35.595 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0310000000000004e-06 2024-09-11T07:47:35.595 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.595 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.596 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.596 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.392245+0000", 2024-09-11T07:47:35.596 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.829440124 2024-09-11T07:47:35.596 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.596 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.596 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:47:35.596 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.392312+0000", 2024-09-11T07:47:35.596 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.6910999999999997e-05 2024-09-11T07:47:35.596 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.596 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:47:35.596 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.596 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.596 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.596 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:85943 9.4 9:27627e86:::benchmark_data_smithi022_133943_object85942:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e259)", 2024-09-11T07:47:35.597 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:26.333925+0000", 2024-09-11T07:47:35.597 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 549.25295173400002, 2024-09-11T07:47:35.597 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.04661077, 2024-09-11T07:47:35.597 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:47:35.597 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:47:35.597 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:47:35.597 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:47:35.597 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:47:35.597 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 85943 2024-09-11T07:47:35.597 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.597 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:47:35.597 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.598 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:47:35.598 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333925+0000", 2024-09-11T07:47:35.598 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.598 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.598 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.598 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:47:35.598 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333925+0000", 2024-09-11T07:47:35.598 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.598 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.598 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.598 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:47:35.598 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333924+0000", 2024-09-11T07:47:35.598 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:47:35.598 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.599 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.599 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:47:35.599 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333948+0000", 2024-09-11T07:47:35.599 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3336000000000002e-05 2024-09-11T07:47:35.599 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.599 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.599 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:47:35.599 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333955+0000", 2024-09-11T07:47:35.599 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.6850000000000001e-06 2024-09-11T07:47:35.599 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.599 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.599 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:47:35.599 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333960+0000", 2024-09-11T07:47:35.599 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7119999999999999e-06 2024-09-11T07:47:35.600 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.600 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.600 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.600 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.334072+0000", 2024-09-11T07:47:35.600 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011152799999999999 2024-09-11T07:47:35.600 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.600 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.600 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op must wait for map", 2024-09-11T07:47:35.600 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.334077+0000", 2024-09-11T07:47:35.600 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8919999999999997e-06 2024-09-11T07:47:35.600 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.600 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.600 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.601 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.562040+0000", 2024-09-11T07:47:35.601 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.227962259 2024-09-11T07:47:35.601 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.601 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.601 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:47:35.601 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.562047+0000", 2024-09-11T07:47:35.601 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.537e-06 2024-09-11T07:47:35.601 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.601 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.601 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.601 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.375671+0000", 2024-09-11T07:47:35.601 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.81362374599999998 2024-09-11T07:47:35.601 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.602 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.602 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for degraded object", 2024-09-11T07:47:35.602 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.376570+0000", 2024-09-11T07:47:35.602 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00089861200000000002 2024-09-11T07:47:35.602 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.602 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.602 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.602 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.380412+0000", 2024-09-11T07:47:35.602 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.003842322 2024-09-11T07:47:35.602 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.602 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.602 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:47:35.602 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.380536+0000", 2024-09-11T07:47:35.602 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000123991 2024-09-11T07:47:35.603 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.603 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:47:35.603 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.603 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.603 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.603 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:85950 9.4 9:22f2c04f:::benchmark_data_smithi022_133943_object85949:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e259)", 2024-09-11T07:47:35.603 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:26.333965+0000", 2024-09-11T07:47:35.603 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 549.25291168700005, 2024-09-11T07:47:35.603 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0552972730000001, 2024-09-11T07:47:35.603 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:47:35.603 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:47:35.603 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:47:35.603 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:47:35.603 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:47:35.604 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 85950 2024-09-11T07:47:35.604 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.604 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:47:35.604 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.604 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:47:35.604 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333965+0000", 2024-09-11T07:47:35.604 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.604 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.604 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.604 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:47:35.604 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333965+0000", 2024-09-11T07:47:35.604 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.605 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.605 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.605 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:47:35.605 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333965+0000", 2024-09-11T07:47:35.605 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:47:35.605 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.605 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.605 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:47:35.605 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333987+0000", 2024-09-11T07:47:35.605 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1985e-05 2024-09-11T07:47:35.605 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.606 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.606 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:47:35.606 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333993+0000", 2024-09-11T07:47:35.606 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7150000000000001e-06 2024-09-11T07:47:35.606 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.606 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.606 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:47:35.606 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.333998+0000", 2024-09-11T07:47:35.607 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3649999999999997e-06 2024-09-11T07:47:35.607 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.607 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.607 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.607 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.334119+0000", 2024-09-11T07:47:35.607 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012089 2024-09-11T07:47:35.607 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.607 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.608 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting_for_map not empty", 2024-09-11T07:47:35.608 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.334123+0000", 2024-09-11T07:47:35.608 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1949999999999996e-06 2024-09-11T07:47:35.608 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.608 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.608 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.608 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.562063+0000", 2024-09-11T07:47:35.608 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.22794028499999999 2024-09-11T07:47:35.608 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.608 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.608 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:47:35.608 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:26.562068+0000", 2024-09-11T07:47:35.608 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1669999999999998e-06 2024-09-11T07:47:35.609 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.609 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.609 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.609 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.376594+0000", 2024-09-11T07:47:35.609 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.81452606699999996 2024-09-11T07:47:35.609 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.609 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.609 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for degraded object", 2024-09-11T07:47:35.609 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.379280+0000", 2024-09-11T07:47:35.609 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0026856340000000001 2024-09-11T07:47:35.609 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.609 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.609 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.609 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.389179+0000", 2024-09-11T07:47:35.610 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0098992579999999993 2024-09-11T07:47:35.610 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.610 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.610 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:47:35.610 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:27.389263+0000", 2024-09-11T07:47:35.610 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3319000000000006e-05 2024-09-11T07:47:35.610 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.610 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:47:35.610 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.610 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.610 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.610 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87214 9.10 9:08bc93d3:::benchmark_data_smithi022_133943_object87213:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:47:35.610 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.342684+0000", 2024-09-11T07:47:35.610 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 546.24419247000003, 2024-09-11T07:47:35.611 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1203623110000001, 2024-09-11T07:47:35.611 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:47:35.611 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:47:35.611 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:47:35.611 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:47:35.611 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:47:35.611 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87214 2024-09-11T07:47:35.611 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.611 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:47:35.611 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.611 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:47:35.611 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342684+0000", 2024-09-11T07:47:35.611 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.611 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.612 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.612 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:47:35.612 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342684+0000", 2024-09-11T07:47:35.612 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.612 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.612 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.612 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:47:35.612 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342682+0000", 2024-09-11T07:47:35.612 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:47:35.612 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.612 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.612 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:47:35.612 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342765+0000", 2024-09-11T07:47:35.612 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3146999999999995e-05 2024-09-11T07:47:35.613 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.613 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.613 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:47:35.613 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342777+0000", 2024-09-11T07:47:35.613 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1494999999999999e-05 2024-09-11T07:47:35.613 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.613 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.613 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:47:35.613 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342784+0000", 2024-09-11T07:47:35.613 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9750000000000001e-06 2024-09-11T07:47:35.613 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.613 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.613 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.614 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343439+0000", 2024-09-11T07:47:35.614 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00065492600000000001 2024-09-11T07:47:35.614 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.614 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.614 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:47:35.614 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343443+0000", 2024-09-11T07:47:35.614 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.583e-06 2024-09-11T07:47:35.614 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.614 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.614 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.614 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.449163+0000", 2024-09-11T07:47:35.614 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.105719841 2024-09-11T07:47:35.614 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.615 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.615 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:47:35.615 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.449317+0000", 2024-09-11T07:47:35.615 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000154358 2024-09-11T07:47:35.615 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.615 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.615 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2,4,6,7", 2024-09-11T07:47:35.615 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.450361+0000", 2024-09-11T07:47:35.615 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0010434000000000001 2024-09-11T07:47:35.615 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.615 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.615 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:47:35.615 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.456929+0000", 2024-09-11T07:47:35.616 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0065678389999999998 2024-09-11T07:47:35.616 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.616 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.616 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.616 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.457159+0000", 2024-09-11T07:47:35.616 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000230518 2024-09-11T07:47:35.616 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.616 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.616 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.616 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461254+0000", 2024-09-11T07:47:35.616 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0040944909999999996 2024-09-11T07:47:35.616 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.616 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.617 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.617 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461887+0000", 2024-09-11T07:47:35.617 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000633455 2024-09-11T07:47:35.617 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.617 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.617 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.617 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461946+0000", 2024-09-11T07:47:35.617 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9064999999999998e-05 2024-09-11T07:47:35.617 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.617 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.617 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.617 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.462748+0000", 2024-09-11T07:47:35.618 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00080188799999999995 2024-09-11T07:47:35.618 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.618 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.618 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:47:35.618 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.462836+0000", 2024-09-11T07:47:35.618 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.7596999999999995e-05 2024-09-11T07:47:35.618 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.618 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.618 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:47:35.618 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.463047+0000", 2024-09-11T07:47:35.618 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000211196 2024-09-11T07:47:35.618 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.618 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:47:35.618 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.619 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.619 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.619 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87241 9.16 9:6d9f9b4a:::benchmark_data_smithi022_133943_object87240:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:47:35.619 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.342793+0000", 2024-09-11T07:47:35.619 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 546.24408439599995, 2024-09-11T07:47:35.619 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.087127435, 2024-09-11T07:47:35.619 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:47:35.619 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:47:35.619 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:47:35.619 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:47:35.619 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:47:35.619 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87241 2024-09-11T07:47:35.619 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.620 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:47:35.620 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.620 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:47:35.620 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342793+0000", 2024-09-11T07:47:35.620 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.620 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.620 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.620 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:47:35.620 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342793+0000", 2024-09-11T07:47:35.620 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.620 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.620 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.621 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:47:35.621 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342792+0000", 2024-09-11T07:47:35.621 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:47:35.621 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.621 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.621 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:47:35.621 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342816+0000", 2024-09-11T07:47:35.621 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4511e-05 2024-09-11T07:47:35.621 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.621 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.621 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:47:35.621 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342823+0000", 2024-09-11T07:47:35.621 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.2189999999999998e-06 2024-09-11T07:47:35.622 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.622 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.622 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:47:35.622 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342828+0000", 2024-09-11T07:47:35.622 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5950000000000001e-06 2024-09-11T07:47:35.622 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.622 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.622 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.622 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.346264+0000", 2024-09-11T07:47:35.622 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0034359199999999999 2024-09-11T07:47:35.622 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.622 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.622 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:47:35.622 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.346271+0000", 2024-09-11T07:47:35.623 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9070000000000001e-06 2024-09-11T07:47:35.623 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.623 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.623 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.623 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.416568+0000", 2024-09-11T07:47:35.623 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.07029756 2024-09-11T07:47:35.623 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.623 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.623 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:47:35.623 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.416798+0000", 2024-09-11T07:47:35.623 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00022939600000000001 2024-09-11T07:47:35.623 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.623 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.623 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,4,5", 2024-09-11T07:47:35.624 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.417111+0000", 2024-09-11T07:47:35.624 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000313374 2024-09-11T07:47:35.624 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.624 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.624 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:47:35.624 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.420512+0000", 2024-09-11T07:47:35.624 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0034004019999999999 2024-09-11T07:47:35.624 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.624 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.624 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.624 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.424231+0000", 2024-09-11T07:47:35.624 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0037196479999999999 2024-09-11T07:47:35.624 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.624 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.625 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.625 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.424639+0000", 2024-09-11T07:47:35.625 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00040762699999999998 2024-09-11T07:47:35.625 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.625 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.625 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.625 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429504+0000", 2024-09-11T07:47:35.625 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0048647830000000001 2024-09-11T07:47:35.625 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.625 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.625 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:47:35.625 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429776+0000", 2024-09-11T07:47:35.625 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00027279600000000001 2024-09-11T07:47:35.626 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.626 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.626 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:47:35.626 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429920+0000", 2024-09-11T07:47:35.626 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014360299999999999 2024-09-11T07:47:35.626 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.626 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:47:35.626 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.626 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.626 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.626 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87242 9.16 9:6d646f5a:::benchmark_data_smithi022_133943_object87241:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:47:35.626 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.342889+0000", 2024-09-11T07:47:35.626 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 546.24398802200005, 2024-09-11T07:47:35.627 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0872590680000001, 2024-09-11T07:47:35.627 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:47:35.627 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:47:35.627 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:47:35.627 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:47:35.627 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:47:35.627 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87242 2024-09-11T07:47:35.627 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.627 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:47:35.627 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.627 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:47:35.627 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342889+0000", 2024-09-11T07:47:35.627 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.628 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.628 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.628 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:47:35.628 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342889+0000", 2024-09-11T07:47:35.628 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.628 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.628 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.628 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:47:35.628 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342888+0000", 2024-09-11T07:47:35.629 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:47:35.629 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.629 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.629 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:47:35.629 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342914+0000", 2024-09-11T07:47:35.629 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.6143999999999999e-05 2024-09-11T07:47:35.629 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.629 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.629 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:47:35.629 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342923+0000", 2024-09-11T07:47:35.629 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.7050000000000005e-06 2024-09-11T07:47:35.629 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.629 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.630 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:47:35.630 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342928+0000", 2024-09-11T07:47:35.630 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.13e-06 2024-09-11T07:47:35.630 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.630 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.630 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.630 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.346523+0000", 2024-09-11T07:47:35.630 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0035955869999999999 2024-09-11T07:47:35.630 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.630 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.630 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:47:35.630 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.346533+0000", 2024-09-11T07:47:35.630 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.4290000000000007e-06 2024-09-11T07:47:35.630 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.631 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.631 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.631 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.418238+0000", 2024-09-11T07:47:35.631 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0717048899999999 2024-09-11T07:47:35.631 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.631 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.631 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:47:35.631 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.418354+0000", 2024-09-11T07:47:35.631 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011647899999999999 2024-09-11T07:47:35.631 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.631 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.632 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,4,5", 2024-09-11T07:47:35.632 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.418586+0000", 2024-09-11T07:47:35.632 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000231618 2024-09-11T07:47:35.632 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.632 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.632 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.632 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.424264+0000", 2024-09-11T07:47:35.632 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0056779370000000001 2024-09-11T07:47:35.632 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.632 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.632 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.632 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.424711+0000", 2024-09-11T07:47:35.632 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00044714499999999998 2024-09-11T07:47:35.632 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.632 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.633 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:47:35.633 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429400+0000", 2024-09-11T07:47:35.633 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0046887550000000002 2024-09-11T07:47:35.633 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.633 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.633 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.633 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429971+0000", 2024-09-11T07:47:35.633 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00057166200000000002 2024-09-11T07:47:35.633 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.633 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.633 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:47:35.633 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.430024+0000", 2024-09-11T07:47:35.633 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3009e-05 2024-09-11T07:47:35.634 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.634 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.634 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:47:35.634 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.430148+0000", 2024-09-11T07:47:35.634 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012362300000000001 2024-09-11T07:47:35.634 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.634 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:47:35.634 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.634 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.634 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.634 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87252 9.12 9:4a1febda:::benchmark_data_smithi022_133943_object87251:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:47:35.634 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.342933+0000", 2024-09-11T07:47:35.635 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 546.24394435099998, 2024-09-11T07:47:35.635 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0912417050000001, 2024-09-11T07:47:35.636 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:47:35.636 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:47:35.636 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:47:35.636 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:47:35.636 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:47:35.636 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87252 2024-09-11T07:47:35.636 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.636 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:47:35.636 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.636 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:47:35.636 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342933+0000", 2024-09-11T07:47:35.636 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.637 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.637 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.637 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:47:35.637 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342933+0000", 2024-09-11T07:47:35.637 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.637 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.637 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.637 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:47:35.637 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342932+0000", 2024-09-11T07:47:35.637 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:47:35.637 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.637 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.637 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:47:35.637 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342953+0000", 2024-09-11T07:47:35.638 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0792999999999999e-05 2024-09-11T07:47:35.638 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.638 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.638 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:47:35.638 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342959+0000", 2024-09-11T07:47:35.638 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4869999999999999e-06 2024-09-11T07:47:35.638 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.638 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.638 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:47:35.638 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342964+0000", 2024-09-11T07:47:35.638 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4100000000000001e-06 2024-09-11T07:47:35.638 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.638 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.638 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.638 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.344041+0000", 2024-09-11T07:47:35.639 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0010773219999999999 2024-09-11T07:47:35.639 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.639 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.639 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:47:35.639 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.344048+0000", 2024-09-11T07:47:35.639 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7220000000000002e-06 2024-09-11T07:47:35.639 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.639 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.639 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.640 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.423752+0000", 2024-09-11T07:47:35.640 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.079704123 2024-09-11T07:47:35.640 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.640 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.640 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:47:35.640 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.423914+0000", 2024-09-11T07:47:35.640 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000161972 2024-09-11T07:47:35.640 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.640 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.640 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2,3,5", 2024-09-11T07:47:35.640 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.424186+0000", 2024-09-11T07:47:35.640 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000272238 2024-09-11T07:47:35.640 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.640 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.641 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.641 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429530+0000", 2024-09-11T07:47:35.641 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0053444929999999996 2024-09-11T07:47:35.641 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.641 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.641 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.641 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429824+0000", 2024-09-11T07:47:35.641 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00029352499999999999 2024-09-11T07:47:35.641 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.641 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.641 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.641 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.430566+0000", 2024-09-11T07:47:35.641 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00074221400000000005 2024-09-11T07:47:35.642 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.642 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.642 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:47:35.642 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.431126+0000", 2024-09-11T07:47:35.642 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00055992399999999995 2024-09-11T07:47:35.642 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.642 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.642 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.642 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.433998+0000", 2024-09-11T07:47:35.642 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0028716219999999999 2024-09-11T07:47:35.642 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.642 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.642 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:47:35.642 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.434055+0000", 2024-09-11T07:47:35.642 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7658999999999998e-05 2024-09-11T07:47:35.643 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.643 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.643 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:47:35.643 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.434174+0000", 2024-09-11T07:47:35.643 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000118891 2024-09-11T07:47:35.643 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.643 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:47:35.643 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.643 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.643 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.643 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87253 9.10 9:0ceb58b8:::benchmark_data_smithi022_133943_object87252:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:47:35.643 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.342970+0000", 2024-09-11T07:47:35.643 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 546.243906896, 2024-09-11T07:47:35.643 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.123611441, 2024-09-11T07:47:35.644 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:47:35.644 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:47:35.644 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:47:35.644 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:47:35.644 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:47:35.644 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87253 2024-09-11T07:47:35.644 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.644 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:47:35.644 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.644 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:47:35.644 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342970+0000", 2024-09-11T07:47:35.644 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.644 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.644 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.644 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:47:35.645 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342970+0000", 2024-09-11T07:47:35.645 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.645 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.645 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.645 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:47:35.645 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342969+0000", 2024-09-11T07:47:35.645 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:47:35.645 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.645 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.645 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:47:35.645 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342991+0000", 2024-09-11T07:47:35.645 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1855999999999999e-05 2024-09-11T07:47:35.645 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.645 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.646 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:47:35.646 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.342998+0000", 2024-09-11T07:47:35.646 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7769999999999997e-06 2024-09-11T07:47:35.646 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.646 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.646 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:47:35.646 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343002+0000", 2024-09-11T07:47:35.646 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0860000000000004e-06 2024-09-11T07:47:35.646 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.646 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.646 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.647 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343467+0000", 2024-09-11T07:47:35.647 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00046544499999999999 2024-09-11T07:47:35.647 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.647 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.647 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:47:35.647 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343475+0000", 2024-09-11T07:47:35.647 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1400000000000002e-06 2024-09-11T07:47:35.647 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.647 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.647 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.647 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.455239+0000", 2024-09-11T07:47:35.647 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1117646999999999 2024-09-11T07:47:35.648 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.648 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.648 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:47:35.648 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.455378+0000", 2024-09-11T07:47:35.648 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013862299999999999 2024-09-11T07:47:35.648 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.648 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.648 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2,4,6,7", 2024-09-11T07:47:35.648 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.455674+0000", 2024-09-11T07:47:35.648 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00029640200000000003 2024-09-11T07:47:35.648 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.648 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.648 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.648 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.459952+0000", 2024-09-11T07:47:35.649 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0042780409999999998 2024-09-11T07:47:35.649 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.649 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.649 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.649 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461424+0000", 2024-09-11T07:47:35.649 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001471483 2024-09-11T07:47:35.649 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.649 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.649 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.649 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.461918+0000", 2024-09-11T07:47:35.649 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00049364099999999996 2024-09-11T07:47:35.649 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.649 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.649 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.649 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.463082+0000", 2024-09-11T07:47:35.650 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00116457 2024-09-11T07:47:35.650 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.650 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.650 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.650 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.464588+0000", 2024-09-11T07:47:35.650 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0015063889999999999 2024-09-11T07:47:35.650 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.650 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.650 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:47:35.650 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.466379+0000", 2024-09-11T07:47:35.650 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00179065 2024-09-11T07:47:35.650 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.650 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.650 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:47:35.650 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.466455+0000", 2024-09-11T07:47:35.651 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5956000000000003e-05 2024-09-11T07:47:35.651 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.651 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.651 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:47:35.651 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.466581+0000", 2024-09-11T07:47:35.651 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012639699999999999 2024-09-11T07:47:35.651 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.651 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:47:35.651 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.651 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.651 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.651 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:87254 9.16 9:6e7c07b3:::benchmark_data_smithi022_133943_object87253:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e262)", 2024-09-11T07:47:35.651 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:38:29.343006+0000", 2024-09-11T07:47:35.651 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 546.24387051300005, 2024-09-11T07:47:35.651 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.094570789, 2024-09-11T07:47:35.652 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:47:35.652 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:47:35.652 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:47:35.652 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:47:35.652 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:47:35.652 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 87254 2024-09-11T07:47:35.652 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.652 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:47:35.652 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.652 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:47:35.652 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343006+0000", 2024-09-11T07:47:35.652 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.652 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.652 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.653 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:47:35.653 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343006+0000", 2024-09-11T07:47:35.653 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.653 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.653 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.653 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:47:35.653 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343006+0000", 2024-09-11T07:47:35.653 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:47:35.653 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.653 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.653 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:47:35.653 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343027+0000", 2024-09-11T07:47:35.653 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1491000000000001e-05 2024-09-11T07:47:35.653 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.654 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.654 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:47:35.654 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343034+0000", 2024-09-11T07:47:35.654 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7009999999999999e-06 2024-09-11T07:47:35.654 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.654 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.654 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:47:35.654 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.343038+0000", 2024-09-11T07:47:35.654 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.98e-06 2024-09-11T07:47:35.654 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.654 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.654 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.654 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.346555+0000", 2024-09-11T07:47:35.654 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00351688 2024-09-11T07:47:35.654 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.655 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.655 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:47:35.655 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:29.346561+0000", 2024-09-11T07:47:35.655 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7359999999999998e-06 2024-09-11T07:47:35.655 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.655 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.655 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.655 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.420577+0000", 2024-09-11T07:47:35.655 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0740159579999999 2024-09-11T07:47:35.655 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.655 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.655 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:47:35.656 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.420669+0000", 2024-09-11T07:47:35.656 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.2723e-05 2024-09-11T07:47:35.656 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.656 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.656 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,4,5", 2024-09-11T07:47:35.656 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.420997+0000", 2024-09-11T07:47:35.656 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00032735499999999999 2024-09-11T07:47:35.656 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.656 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.656 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:47:35.656 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429409+0000", 2024-09-11T07:47:35.656 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0084126630000000008 2024-09-11T07:47:35.656 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.657 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.657 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.657 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.429463+0000", 2024-09-11T07:47:35.657 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3375999999999999e-05 2024-09-11T07:47:35.657 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.657 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.657 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.657 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.430192+0000", 2024-09-11T07:47:35.657 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00072935800000000005 2024-09-11T07:47:35.657 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.657 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.657 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.657 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.437414+0000", 2024-09-11T07:47:35.657 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0072219939999999998 2024-09-11T07:47:35.658 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.658 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.658 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:47:35.658 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.437450+0000", 2024-09-11T07:47:35.658 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5957000000000001e-05 2024-09-11T07:47:35.658 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.658 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.658 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:47:35.658 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:38:30.437577+0000", 2024-09-11T07:47:35.658 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012722100000000001 2024-09-11T07:47:35.658 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.658 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:47:35.658 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.658 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.658 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.658 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:234463 9.2 9:4445ea71:::benchmark_data_smithi022_133943_object107872:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:47:35.659 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:59.743553+0000", 2024-09-11T07:47:35.659 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 395.84332383700001, 2024-09-11T07:47:35.659 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1848900979999999, 2024-09-11T07:47:35.659 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:47:35.659 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:47:35.659 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:47:35.659 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:47:35.659 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:47:35.659 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 234463 2024-09-11T07:47:35.659 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.659 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:47:35.659 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.659 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:47:35.659 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.743553+0000", 2024-09-11T07:47:35.659 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.660 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.660 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.660 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:47:35.660 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.743553+0000", 2024-09-11T07:47:35.660 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.660 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.660 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.660 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:47:35.660 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.743552+0000", 2024-09-11T07:47:35.660 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:47:35.660 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.660 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.660 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:47:35.661 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.743558+0000", 2024-09-11T07:47:35.661 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0440000000000001e-06 2024-09-11T07:47:35.661 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.661 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.661 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:47:35.661 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.743573+0000", 2024-09-11T07:47:35.661 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5206e-05 2024-09-11T07:47:35.661 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.661 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.661 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:47:35.661 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.743614+0000", 2024-09-11T07:47:35.661 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1532000000000001e-05 2024-09-11T07:47:35.661 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.661 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.662 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.662 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.270167+0000", 2024-09-11T07:47:35.662 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.52655248300000002 2024-09-11T07:47:35.662 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.662 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.662 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:47:35.662 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.270264+0000", 2024-09-11T07:47:35.662 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.7097999999999998e-05 2024-09-11T07:47:35.662 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.662 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.662 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,7", 2024-09-11T07:47:35.663 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.270365+0000", 2024-09-11T07:47:35.663 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000100635 2024-09-11T07:47:35.663 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.663 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.663 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:47:35.663 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274143+0000", 2024-09-11T07:47:35.663 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0037784020000000001 2024-09-11T07:47:35.663 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.663 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.663 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.663 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.277336+0000", 2024-09-11T07:47:35.663 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0031928590000000001 2024-09-11T07:47:35.663 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.663 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.664 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.664 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928350+0000", 2024-09-11T07:47:35.664 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.65101431899999995 2024-09-11T07:47:35.664 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.664 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.664 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:47:35.664 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928401+0000", 2024-09-11T07:47:35.664 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1303e-05 2024-09-11T07:47:35.664 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.664 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.664 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:47:35.665 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928443+0000", 2024-09-11T07:47:35.665 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1795999999999998e-05 2024-09-11T07:47:35.665 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.665 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:47:35.665 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.665 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.665 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.665 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:234477 9.a 9:5131a6fa:::benchmark_data_smithi022_133943_object107886:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:47:35.665 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:59.763716+0000", 2024-09-11T07:47:35.665 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 395.82316126299997, 2024-09-11T07:47:35.665 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1647252889999999, 2024-09-11T07:47:35.666 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:47:35.666 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:47:35.666 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:47:35.666 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:47:35.666 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:47:35.666 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 234477 2024-09-11T07:47:35.666 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.666 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:47:35.666 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.666 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:47:35.666 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.763716+0000", 2024-09-11T07:47:35.666 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.667 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.667 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.667 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:47:35.667 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.763716+0000", 2024-09-11T07:47:35.667 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.667 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.667 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.667 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:47:35.667 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.763714+0000", 2024-09-11T07:47:35.667 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:47:35.667 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.667 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.667 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:47:35.667 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.763729+0000", 2024-09-11T07:47:35.668 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5347999999999999e-05 2024-09-11T07:47:35.668 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.668 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.668 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:47:35.668 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.763740+0000", 2024-09-11T07:47:35.668 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1165000000000001e-05 2024-09-11T07:47:35.668 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.668 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.668 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:47:35.668 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.763745+0000", 2024-09-11T07:47:35.668 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1050000000000003e-06 2024-09-11T07:47:35.668 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.668 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.668 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.669 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.269894+0000", 2024-09-11T07:47:35.669 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.50614913299999997 2024-09-11T07:47:35.669 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.669 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.669 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:47:35.669 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.269999+0000", 2024-09-11T07:47:35.669 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000105189 2024-09-11T07:47:35.669 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.669 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.669 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:47:35.669 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.270104+0000", 2024-09-11T07:47:35.669 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010485899999999999 2024-09-11T07:47:35.669 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.669 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.670 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.670 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274132+0000", 2024-09-11T07:47:35.670 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0040275689999999999 2024-09-11T07:47:35.670 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.670 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.670 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:47:35.670 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274138+0000", 2024-09-11T07:47:35.670 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5729999999999996e-06 2024-09-11T07:47:35.670 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.670 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.670 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.670 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928373+0000", 2024-09-11T07:47:35.670 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.65423462899999996 2024-09-11T07:47:35.670 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.670 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.671 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:47:35.671 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928412+0000", 2024-09-11T07:47:35.671 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9181000000000003e-05 2024-09-11T07:47:35.671 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.671 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.671 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:47:35.671 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928441+0000", 2024-09-11T07:47:35.671 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8656000000000001e-05 2024-09-11T07:47:35.671 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.671 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:47:35.671 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.671 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.671 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.671 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:234480 9.2 9:45c5c2ce:::benchmark_data_smithi022_133943_object107889:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:47:35.672 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:59.767350+0000", 2024-09-11T07:47:35.672 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 395.81952704499997, 2024-09-11T07:47:35.672 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.161190985, 2024-09-11T07:47:35.672 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:47:35.672 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:47:35.672 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:47:35.672 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:47:35.672 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:47:35.672 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 234480 2024-09-11T07:47:35.672 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.672 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:47:35.672 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.673 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:47:35.673 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.767350+0000", 2024-09-11T07:47:35.673 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.673 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.673 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.673 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:47:35.673 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.767350+0000", 2024-09-11T07:47:35.673 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.673 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.673 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.673 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:47:35.673 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.767347+0000", 2024-09-11T07:47:35.673 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:47:35.673 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.674 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.674 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:47:35.674 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.767356+0000", 2024-09-11T07:47:35.674 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.6160000000000002e-06 2024-09-11T07:47:35.674 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.674 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.674 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:47:35.674 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.767367+0000", 2024-09-11T07:47:35.674 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1946999999999999e-05 2024-09-11T07:47:35.674 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.674 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.674 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:47:35.674 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.767373+0000", 2024-09-11T07:47:35.674 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9159999999999998e-06 2024-09-11T07:47:35.675 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.675 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.675 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.675 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274356+0000", 2024-09-11T07:47:35.675 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.50698302299999998 2024-09-11T07:47:35.675 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.675 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.675 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:47:35.675 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274462+0000", 2024-09-11T07:47:35.675 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000105349 2024-09-11T07:47:35.675 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.675 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.675 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,7", 2024-09-11T07:47:35.676 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274568+0000", 2024-09-11T07:47:35.676 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010647899999999999 2024-09-11T07:47:35.676 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.676 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.676 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:47:35.676 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.279627+0000", 2024-09-11T07:47:35.676 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0050591680000000002 2024-09-11T07:47:35.676 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.676 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.676 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.676 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.321730+0000", 2024-09-11T07:47:35.676 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.042102300000000002 2024-09-11T07:47:35.676 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.676 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.677 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.677 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928475+0000", 2024-09-11T07:47:35.677 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.60674521000000003 2024-09-11T07:47:35.677 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.677 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.677 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:47:35.677 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928510+0000", 2024-09-11T07:47:35.678 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5036999999999997e-05 2024-09-11T07:47:35.678 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.678 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.678 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:47:35.679 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:35.680 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928541+0000", 2024-09-11T07:47:35.680 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0879e-05 2024-09-11T07:47:35.680 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.680 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:47:35.680 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.680 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.680 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.680 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:234484 9.a 9:5db53976:::benchmark_data_smithi022_133943_object107893:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:47:35.680 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:59.776130+0000", 2024-09-11T07:47:35.680 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 395.81074699999999, 2024-09-11T07:47:35.680 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1524430130000001, 2024-09-11T07:47:35.681 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:47:35.681 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:47:35.681 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:47:35.681 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:47:35.681 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:47:35.681 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 234484 2024-09-11T07:47:35.681 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.681 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:47:35.681 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.681 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:47:35.681 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.776130+0000", 2024-09-11T07:47:35.681 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.681 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.682 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.682 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:47:35.682 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.776130+0000", 2024-09-11T07:47:35.682 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.682 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.682 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.682 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:47:35.682 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.776128+0000", 2024-09-11T07:47:35.682 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:47:35.682 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.682 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.682 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:47:35.682 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.776136+0000", 2024-09-11T07:47:35.682 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3410000000000003e-06 2024-09-11T07:47:35.683 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.683 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.683 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:47:35.683 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.776165+0000", 2024-09-11T07:47:35.683 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.9031e-05 2024-09-11T07:47:35.683 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.683 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.683 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:47:35.683 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.776171+0000", 2024-09-11T07:47:35.683 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1929999999999998e-06 2024-09-11T07:47:35.683 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.683 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.683 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.684 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274214+0000", 2024-09-11T07:47:35.684 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.49804272399999999 2024-09-11T07:47:35.684 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.684 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.684 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:47:35.684 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274342+0000", 2024-09-11T07:47:35.684 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012783099999999999 2024-09-11T07:47:35.684 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.684 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.684 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:47:35.684 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274443+0000", 2024-09-11T07:47:35.684 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000101517 2024-09-11T07:47:35.684 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.684 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.684 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.685 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.277315+0000", 2024-09-11T07:47:35.685 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002871958 2024-09-11T07:47:35.685 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.685 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.685 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:47:35.685 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.277321+0000", 2024-09-11T07:47:35.685 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9569999999999999e-06 2024-09-11T07:47:35.685 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.685 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.685 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.685 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928494+0000", 2024-09-11T07:47:35.685 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.651173212 2024-09-11T07:47:35.685 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.685 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.686 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:47:35.686 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928537+0000", 2024-09-11T07:47:35.686 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2274999999999998e-05 2024-09-11T07:47:35.686 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.686 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.686 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:47:35.686 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928573+0000", 2024-09-11T07:47:35.686 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6310000000000003e-05 2024-09-11T07:47:35.686 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.686 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:47:35.686 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.686 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.686 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.686 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:234493 9.2 9:43b51550:::benchmark_data_smithi022_133943_object107902:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:47:35.686 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:59.812761+0000", 2024-09-11T07:47:35.687 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 395.77411601300003, 2024-09-11T07:47:35.687 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1158854009999999, 2024-09-11T07:47:35.687 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:47:35.687 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:47:35.687 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:47:35.687 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:47:35.687 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:47:35.687 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 234493 2024-09-11T07:47:35.687 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.687 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:47:35.687 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.687 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:47:35.687 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.812761+0000", 2024-09-11T07:47:35.688 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.688 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.688 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.688 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:47:35.688 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.812761+0000", 2024-09-11T07:47:35.688 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.688 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.688 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.688 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:47:35.688 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.812758+0000", 2024-09-11T07:47:35.688 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:47:35.688 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.688 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.688 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:47:35.688 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.812767+0000", 2024-09-11T07:47:35.689 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.5229999999999995e-06 2024-09-11T07:47:35.689 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.689 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.689 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:47:35.689 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.812780+0000", 2024-09-11T07:47:35.689 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2869e-05 2024-09-11T07:47:35.689 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.689 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.689 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:47:35.689 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.812786+0000", 2024-09-11T07:47:35.689 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5479999999999998e-06 2024-09-11T07:47:35.689 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.689 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.689 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.690 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.277362+0000", 2024-09-11T07:47:35.690 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.46457584499999999 2024-09-11T07:47:35.690 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.690 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.690 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:47:35.690 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.277474+0000", 2024-09-11T07:47:35.690 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011171 2024-09-11T07:47:35.690 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.690 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.690 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,7", 2024-09-11T07:47:35.690 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.277592+0000", 2024-09-11T07:47:35.690 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011786900000000001 2024-09-11T07:47:35.690 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.690 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.690 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:47:35.691 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.321737+0000", 2024-09-11T07:47:35.691 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.044145292000000003 2024-09-11T07:47:35.691 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.691 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.691 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.691 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.321772+0000", 2024-09-11T07:47:35.691 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4910000000000003e-05 2024-09-11T07:47:35.691 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.691 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.691 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.691 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928574+0000", 2024-09-11T07:47:35.691 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.60680186700000005 2024-09-11T07:47:35.691 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.691 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.691 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:47:35.692 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928615+0000", 2024-09-11T07:47:35.692 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0728e-05 2024-09-11T07:47:35.692 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.692 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.692 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:47:35.692 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928646+0000", 2024-09-11T07:47:35.692 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.1696999999999998e-05 2024-09-11T07:47:35.692 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.692 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:47:35.692 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.692 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.692 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.692 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235135 9.12 9:4bac11d3:::benchmark_data_smithi022_133943_object108544:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e321)", 2024-09-11T07:47:35.692 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.671922+0000", 2024-09-11T07:47:35.693 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 393.914954507, 2024-09-11T07:47:35.693 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.096036827, 2024-09-11T07:47:35.693 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:47:35.693 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:47:35.693 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:47:35.693 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:47:35.693 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:47:35.693 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235135 2024-09-11T07:47:35.693 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.693 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:47:35.693 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.693 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:47:35.693 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.671922+0000", 2024-09-11T07:47:35.693 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.693 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.694 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.694 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:47:35.694 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.671922+0000", 2024-09-11T07:47:35.694 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.694 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.694 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.694 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:47:35.694 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.671921+0000", 2024-09-11T07:47:35.694 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:47:35.694 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.694 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.694 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:47:35.695 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.671926+0000", 2024-09-11T07:47:35.695 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1359999999999996e-06 2024-09-11T07:47:35.695 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.695 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.695 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:47:35.695 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.671937+0000", 2024-09-11T07:47:35.695 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0676000000000001e-05 2024-09-11T07:47:35.695 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.695 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.695 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:47:35.695 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.671942+0000", 2024-09-11T07:47:35.695 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.062e-06 2024-09-11T07:47:35.695 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.695 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.696 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.696 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.684807+0000", 2024-09-11T07:47:35.696 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.012864855 2024-09-11T07:47:35.696 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.696 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.696 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:47:35.696 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.684939+0000", 2024-09-11T07:47:35.696 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013172800000000001 2024-09-11T07:47:35.696 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.696 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.696 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2,5", 2024-09-11T07:47:35.696 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.685060+0000", 2024-09-11T07:47:35.696 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000121413 2024-09-11T07:47:35.696 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.697 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.697 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.697 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.700013+0000", 2024-09-11T07:47:35.697 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.014952995 2024-09-11T07:47:35.697 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.697 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.697 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:47:35.697 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.700021+0000", 2024-09-11T07:47:35.697 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.6729999999999992e-06 2024-09-11T07:47:35.697 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.697 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.697 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.698 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.722409+0000", 2024-09-11T07:47:35.698 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.022387961 2024-09-11T07:47:35.698 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.698 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.698 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for missing object", 2024-09-11T07:47:35.698 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.722505+0000", 2024-09-11T07:47:35.698 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.6838999999999999e-05 2024-09-11T07:47:35.698 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.698 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.698 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.698 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.767903+0000", 2024-09-11T07:47:35.698 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.045397237 2024-09-11T07:47:35.698 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.698 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.698 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:47:35.699 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.767959+0000", 2024-09-11T07:47:35.699 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6588000000000001e-05 2024-09-11T07:47:35.699 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.699 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:47:35.699 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.699 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.699 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.699 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235140 9.12 9:4a4ad0d1:::benchmark_data_smithi022_133943_object108549:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-11T07:47:35.699 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.685741+0000", 2024-09-11T07:47:35.699 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 393.90113586799998, 2024-09-11T07:47:35.699 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.082396656, 2024-09-11T07:47:35.699 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:47:35.699 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:47:35.699 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:47:35.699 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:47:35.700 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:47:35.700 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235140 2024-09-11T07:47:35.700 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.700 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:47:35.700 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.700 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:47:35.700 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.685741+0000", 2024-09-11T07:47:35.700 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.700 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.700 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.700 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:47:35.700 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.685741+0000", 2024-09-11T07:47:35.700 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.700 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.700 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.701 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:47:35.701 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.685739+0000", 2024-09-11T07:47:35.701 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:47:35.701 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.701 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.701 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:47:35.701 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.685746+0000", 2024-09-11T07:47:35.701 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0310000000000001e-06 2024-09-11T07:47:35.701 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.701 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.701 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:47:35.701 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.685757+0000", 2024-09-11T07:47:35.701 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0773e-05 2024-09-11T07:47:35.701 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.701 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.702 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:47:35.702 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.685763+0000", 2024-09-11T07:47:35.702 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2400000000000004e-06 2024-09-11T07:47:35.702 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.702 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.702 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.702 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.702365+0000", 2024-09-11T07:47:35.702 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.016602519 2024-09-11T07:47:35.702 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.702 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.702 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:47:35.702 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.702372+0000", 2024-09-11T07:47:35.702 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2299999999999996e-06 2024-09-11T07:47:35.702 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.702 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.703 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.703 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.722543+0000", 2024-09-11T07:47:35.703 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0201711920000001 2024-09-11T07:47:35.703 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.703 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.703 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:47:35.703 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.722638+0000", 2024-09-11T07:47:35.703 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.4757000000000001e-05 2024-09-11T07:47:35.703 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.703 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.703 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2", 2024-09-11T07:47:35.703 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.722761+0000", 2024-09-11T07:47:35.703 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012308800000000001 2024-09-11T07:47:35.703 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.704 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.704 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:47:35.704 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.733739+0000", 2024-09-11T07:47:35.704 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.01097806 2024-09-11T07:47:35.704 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.704 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.704 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.704 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.767979+0000", 2024-09-11T07:47:35.704 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.034240697 2024-09-11T07:47:35.704 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.704 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.704 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.704 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.767996+0000", 2024-09-11T07:47:35.704 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.6189e-05 2024-09-11T07:47:35.705 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.705 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.705 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:47:35.705 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768036+0000", 2024-09-11T07:47:35.705 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0198999999999999e-05 2024-09-11T07:47:35.705 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.705 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.705 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:47:35.705 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768138+0000", 2024-09-11T07:47:35.705 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000101866 2024-09-11T07:47:35.705 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.705 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:47:35.705 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.706 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.706 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.706 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235178 9.12 9:4ef9439e:::benchmark_data_smithi022_133943_object108587:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-11T07:47:35.706 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.722941+0000", 2024-09-11T07:47:35.706 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 393.86393588999999, 2024-09-11T07:47:35.706 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.045418086, 2024-09-11T07:47:35.706 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:47:35.706 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:47:35.706 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:47:35.706 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:47:35.706 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:47:35.706 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235178 2024-09-11T07:47:35.706 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.706 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:47:35.706 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.706 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:47:35.707 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.722941+0000", 2024-09-11T07:47:35.707 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.707 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:35 smithi022 ceph-mon[29462]: pgmap v985: 158 pgs: 158 active+clean; 6.6 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1023 B/s rd, 69 MiB/s wr, 1.11k op/s 2024-09-11T07:47:35.707 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:35 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:47:35.707 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:35 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:47:35.707 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:35 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:47:35.707 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:35 smithi022 ceph-mon[37330]: pgmap v985: 158 pgs: 158 active+clean; 6.6 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1023 B/s rd, 69 MiB/s wr, 1.11k op/s 2024-09-11T07:47:35.707 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:35 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:47:35.708 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:35 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:47:35.708 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:35 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:47:35.708 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.708 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.708 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:47:35.708 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.722941+0000", 2024-09-11T07:47:35.708 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.709 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.709 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.709 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:47:35.709 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.722940+0000", 2024-09-11T07:47:35.709 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:47:35.709 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.709 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.709 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:47:35.709 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.722943+0000", 2024-09-11T07:47:35.709 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8650000000000001e-06 2024-09-11T07:47:35.709 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.709 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.709 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:47:35.709 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.722948+0000", 2024-09-11T07:47:35.710 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3390000000000003e-06 2024-09-11T07:47:35.710 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.710 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.710 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:47:35.710 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.722950+0000", 2024-09-11T07:47:35.710 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7609999999999999e-06 2024-09-11T07:47:35.710 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.710 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.710 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.710 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.733542+0000", 2024-09-11T07:47:35.710 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.010591313 2024-09-11T07:47:35.710 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.710 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.711 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:47:35.711 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.733548+0000", 2024-09-11T07:47:35.711 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7719999999999997e-06 2024-09-11T07:47:35.711 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.711 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.711 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.711 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726822+0000", 2024-09-11T07:47:35.711 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.99327332599999996 2024-09-11T07:47:35.711 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.711 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.711 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:47:35.711 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726931+0000", 2024-09-11T07:47:35.711 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010914 2024-09-11T07:47:35.711 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.712 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.712 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2", 2024-09-11T07:47:35.712 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.727048+0000", 2024-09-11T07:47:35.712 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011745599999999999 2024-09-11T07:47:35.712 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.712 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.712 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:47:35.712 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.743419+0000", 2024-09-11T07:47:35.712 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.016370268 2024-09-11T07:47:35.712 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.712 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.712 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.712 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768173+0000", 2024-09-11T07:47:35.712 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.024754438 2024-09-11T07:47:35.712 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.713 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.713 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.713 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768201+0000", 2024-09-11T07:47:35.713 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7946e-05 2024-09-11T07:47:35.713 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.713 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.713 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:47:35.713 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768246+0000", 2024-09-11T07:47:35.713 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4567999999999997e-05 2024-09-11T07:47:35.713 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.713 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.713 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:47:35.713 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768359+0000", 2024-09-11T07:47:35.714 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011356 2024-09-11T07:47:35.714 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.714 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:47:35.714 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.714 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.714 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.714 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235201 9.12 9:4fabe53d:::benchmark_data_smithi022_133943_object108610:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-11T07:47:35.714 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.747159+0000", 2024-09-11T07:47:35.714 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 393.839717888, 2024-09-11T07:47:35.714 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.021527321, 2024-09-11T07:47:35.714 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:47:35.714 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:47:35.714 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:47:35.715 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:47:35.715 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:47:35.715 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235201 2024-09-11T07:47:35.715 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.715 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:47:35.715 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.715 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:47:35.715 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.747159+0000", 2024-09-11T07:47:35.715 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.715 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.715 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.715 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:47:35.715 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.747159+0000", 2024-09-11T07:47:35.716 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.716 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.716 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.716 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:47:35.716 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.747157+0000", 2024-09-11T07:47:35.716 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:47:35.716 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.716 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.716 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:47:35.716 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.747164+0000", 2024-09-11T07:47:35.716 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7360000000000004e-06 2024-09-11T07:47:35.716 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.717 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.717 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:47:35.717 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.747192+0000", 2024-09-11T07:47:35.717 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8422e-05 2024-09-11T07:47:35.717 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.717 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.717 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:47:35.717 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.747201+0000", 2024-09-11T07:47:35.717 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.5290000000000008e-06 2024-09-11T07:47:35.717 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.717 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.717 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.717 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.747514+0000", 2024-09-11T07:47:35.717 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00031351600000000001 2024-09-11T07:47:35.718 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.718 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.718 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:47:35.718 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.747524+0000", 2024-09-11T07:47:35.718 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.533e-06 2024-09-11T07:47:35.718 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.718 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.718 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.718 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.730920+0000", 2024-09-11T07:47:35.718 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.98339653500000002 2024-09-11T07:47:35.718 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.718 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.718 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:47:35.718 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.731036+0000", 2024-09-11T07:47:35.718 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000115571 2024-09-11T07:47:35.719 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.719 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.719 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2", 2024-09-11T07:47:35.719 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.731157+0000", 2024-09-11T07:47:35.719 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000121109 2024-09-11T07:47:35.719 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.719 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.719 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:47:35.719 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.743425+0000", 2024-09-11T07:47:35.719 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.012267554999999999 2024-09-11T07:47:35.719 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.719 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.719 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.719 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768397+0000", 2024-09-11T07:47:35.719 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.024972765000000001 2024-09-11T07:47:35.720 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.720 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.720 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.720 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768428+0000", 2024-09-11T07:47:35.720 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0167000000000001e-05 2024-09-11T07:47:35.720 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.720 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.720 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:47:35.720 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768615+0000", 2024-09-11T07:47:35.720 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00018698900000000001 2024-09-11T07:47:35.720 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.720 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.720 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:47:35.720 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768686+0000", 2024-09-11T07:47:35.720 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1790000000000002e-05 2024-09-11T07:47:35.721 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.721 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:47:35.721 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.721 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.721 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.721 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235224 9.12 9:4fcd73f7:::benchmark_data_smithi022_133943_object108633:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-11T07:47:35.721 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.769489+0000", 2024-09-11T07:47:35.721 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 393.81738817000002, 2024-09-11T07:47:35.721 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.99944138900000001, 2024-09-11T07:47:35.721 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:47:35.721 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:47:35.721 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:47:35.721 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:47:35.722 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:47:35.722 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235224 2024-09-11T07:47:35.722 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.722 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:47:35.722 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.722 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:47:35.722 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.769489+0000", 2024-09-11T07:47:35.722 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.722 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.722 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.722 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:47:35.722 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.769489+0000", 2024-09-11T07:47:35.722 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.722 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.722 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.723 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:47:35.723 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.769487+0000", 2024-09-11T07:47:35.723 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:47:35.723 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.723 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.723 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:47:35.723 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.769494+0000", 2024-09-11T07:47:35.723 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.3649999999999998e-06 2024-09-11T07:47:35.723 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.723 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.723 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:47:35.723 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.769509+0000", 2024-09-11T07:47:35.723 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5122e-05 2024-09-11T07:47:35.723 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.723 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.724 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:47:35.724 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.769515+0000", 2024-09-11T07:47:35.724 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1330000000000004e-06 2024-09-11T07:47:35.724 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.724 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.724 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.724 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.771837+0000", 2024-09-11T07:47:35.724 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0023218840000000002 2024-09-11T07:47:35.724 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.724 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.724 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:47:35.724 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.771845+0000", 2024-09-11T07:47:35.724 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.2239999999999995e-06 2024-09-11T07:47:35.724 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.724 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.725 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.725 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.733765+0000", 2024-09-11T07:47:35.725 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.96191987199999995 2024-09-11T07:47:35.725 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.725 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.725 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:47:35.725 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.733865+0000", 2024-09-11T07:47:35.725 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010025499999999999 2024-09-11T07:47:35.725 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.725 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.725 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2", 2024-09-11T07:47:35.725 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.734245+0000", 2024-09-11T07:47:35.725 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00037957000000000003 2024-09-11T07:47:35.725 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.725 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.726 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:47:35.726 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.746336+0000", 2024-09-11T07:47:35.726 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.012091497 2024-09-11T07:47:35.726 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.726 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.726 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.726 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768732+0000", 2024-09-11T07:47:35.726 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.022395854999999999 2024-09-11T07:47:35.726 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.726 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.726 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.726 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768797+0000", 2024-09-11T07:47:35.726 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4886000000000002e-05 2024-09-11T07:47:35.726 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.727 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.727 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:47:35.727 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768837+0000", 2024-09-11T07:47:35.727 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0203000000000001e-05 2024-09-11T07:47:35.727 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.727 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.727 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:47:35.727 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768930+0000", 2024-09-11T07:47:35.727 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.2763999999999995e-05 2024-09-11T07:47:35.727 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.727 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:47:35.727 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.728 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.728 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.728 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235274 9.12 9:48c72601:::benchmark_data_smithi022_133943_object108683:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-11T07:47:35.728 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.815001+0000", 2024-09-11T07:47:35.728 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 393.77187552800001, 2024-09-11T07:47:35.728 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.95419106899999995, 2024-09-11T07:47:35.728 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:47:35.728 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:47:35.728 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:47:35.728 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:47:35.729 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:47:35.729 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235274 2024-09-11T07:47:35.729 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.729 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:47:35.729 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.729 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:47:35.729 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.815001+0000", 2024-09-11T07:47:35.729 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.729 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.729 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.729 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:47:35.729 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.815001+0000", 2024-09-11T07:47:35.729 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:47:35.730 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.730 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.730 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:47:35.730 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.815000+0000", 2024-09-11T07:47:35.730 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:47:35.730 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.730 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.730 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:47:35.730 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.815006+0000", 2024-09-11T07:47:35.730 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.55e-06 2024-09-11T07:47:35.730 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.730 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.730 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:47:35.731 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.815015+0000", 2024-09-11T07:47:35.731 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.4030000000000007e-06 2024-09-11T07:47:35.731 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.731 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.731 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:47:35.731 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.815019+0000", 2024-09-11T07:47:35.731 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1760000000000003e-06 2024-09-11T07:47:35.731 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.731 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.731 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.731 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.815066+0000", 2024-09-11T07:47:35.731 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7092000000000002e-05 2024-09-11T07:47:35.731 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.731 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.732 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:47:35.732 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.815074+0000", 2024-09-11T07:47:35.732 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.7069999999999992e-06 2024-09-11T07:47:35.732 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.732 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.732 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:47:35.732 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.738191+0000", 2024-09-11T07:47:35.732 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.92311757100000003 2024-09-11T07:47:35.732 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.732 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.732 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:47:35.732 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.738316+0000", 2024-09-11T07:47:35.732 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000124573 2024-09-11T07:47:35.732 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.732 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.733 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2", 2024-09-11T07:47:35.733 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.738445+0000", 2024-09-11T07:47:35.733 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012939399999999999 2024-09-11T07:47:35.733 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.733 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.733 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:47:35.733 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.749519+0000", 2024-09-11T07:47:35.733 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.011073697 2024-09-11T07:47:35.733 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.733 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.733 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.733 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768769+0000", 2024-09-11T07:47:35.733 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.019249720000000001 2024-09-11T07:47:35.733 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.733 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.734 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:47:35.734 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.769045+0000", 2024-09-11T07:47:35.734 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000276335 2024-09-11T07:47:35.734 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.734 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.734 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:47:35.734 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.769089+0000", 2024-09-11T07:47:35.734 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4314999999999998e-05 2024-09-11T07:47:35.734 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:47:35.734 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:47:35.734 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:47:35.734 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.769192+0000", 2024-09-11T07:47:35.734 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010329500000000001 2024-09-11T07:47:35.734 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.734 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:47:35.735 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.735 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:47:35.735 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:47:35.735 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:47:35.749 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:35.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:35.750 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:35.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:35 smithi104 ceph-mon[33507]: pgmap v985: 158 pgs: 158 active+clean; 6.6 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1023 B/s rd, 69 MiB/s wr, 1.11k op/s 2024-09-11T07:47:35.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:35 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:47:35.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:35 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:47:35.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:35 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:47:35.851 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:35.893 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:35.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:35.894 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:35.995 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:36.020 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:36.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:36.021 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:36.122 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:36.126 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 84 16 113231 113215 84.2103 62.375 0.0096361 0.0118704 2024-09-11T07:47:36.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:36.167 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:36.167 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:36.268 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:36.306 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:36.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:36.307 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:36.408 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:36.433 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:36.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:36.435 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:36.535 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:36.562 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:36.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:36.563 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:36.664 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:36.711 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:36.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:36.713 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:36.813 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:36.838 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:36.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:36.840 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:36.940 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:36.983 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:36.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:36.984 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:37.081 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:36 smithi022 ceph-mon[29462]: pgmap v986: 158 pgs: 158 active+clean; 6.8 GiB data, 26 GiB used, 599 GiB / 626 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-11T07:47:37.082 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:36 smithi022 ceph-mon[37330]: pgmap v986: 158 pgs: 158 active+clean; 6.8 GiB data, 26 GiB used, 599 GiB / 626 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-11T07:47:37.085 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:37.109 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:37.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:37.110 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:37.126 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 85 16 114403 114387 84.0813 73.25 0.0088973 0.0118879 2024-09-11T07:47:37.210 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:37.250 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:37.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:37.251 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:37.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:36 smithi104 ceph-mon[33507]: pgmap v986: 158 pgs: 158 active+clean; 6.8 GiB data, 26 GiB used, 599 GiB / 626 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-11T07:47:37.352 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:37.352 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:47:37.376 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:37.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:37.377 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:37.478 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:37.527 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:37.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:37.528 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:37.629 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:37.653 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:37.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:37.654 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:37.755 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:37.779 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:37.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:37.781 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:37.881 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:37.927 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:37.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:37.928 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:38.029 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:38.045 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-11T07:47:38.053 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:38.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:38.054 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:38.126 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 86 16 115573 115557 83.9538 73.125 0.0183966 0.0119066 2024-09-11T07:47:38.155 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:38.179 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:38.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:38.181 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:38.281 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:38.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:38.329 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:38.330 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:38.430 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:38.455 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:38.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:38.456 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:38.557 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:38.582 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:38.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:38.583 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:38.683 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:38.707 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:38.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:38.708 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:38.809 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:38.835 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:38.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:38.836 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:38.936 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:38.962 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:38.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:38.963 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:39.064 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:39.089 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:39.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:39.091 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:39.127 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 87 16 116714 116698 83.8085 71.3125 0.0124291 0.0119271 2024-09-11T07:47:39.127 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T07:47:39.192 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:39.217 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:39.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:39.218 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:39.318 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:39.344 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:39.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:39.345 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:39.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:39 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:47:39.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:39 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:47:39.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:39 smithi022 ceph-mon[29462]: pgmap v987: 158 pgs: 158 active+clean; 6.8 GiB data, 26 GiB used, 599 GiB / 626 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-11T07:47:39.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:39 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:47:39.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:39 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:47:39.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:39 smithi022 ceph-mon[37330]: pgmap v987: 158 pgs: 158 active+clean; 6.8 GiB data, 26 GiB used, 599 GiB / 626 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-11T07:47:39.446 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:39.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:39.497 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:39.498 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:39.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:39 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:47:39.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:39 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:47:39.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:39 smithi104 ceph-mon[33507]: pgmap v987: 158 pgs: 158 active+clean; 6.8 GiB data, 26 GiB used, 599 GiB / 626 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-11T07:47:39.598 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:39.624 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:39.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:39.625 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:39.726 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:39.765 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:39.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:39.767 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:39.868 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:39.893 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:39.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:39.894 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:39.995 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:40.019 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:40.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:40.021 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:40.121 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:40.127 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 88 16 117848 117832 83.6607 70.875 0.0073623 0.0119479 2024-09-11T07:47:40.160 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:40.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:40.161 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:40.262 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:40.284 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:40.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:40.285 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:40.386 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:40.413 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:40.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:40.414 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:40.515 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:40.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:47:40 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:47:40.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:40.541 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:40.541 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:40.642 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:40.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:47:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:47:40] "GET /metrics HTTP/1.1" 200 225165 "" "Prometheus/2.43.0" 2024-09-11T07:47:40.682 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:40.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:40.683 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:40.784 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:40.812 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:40.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:40.813 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:40.914 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:40.938 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:40.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:40.939 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:41.040 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:41.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:41.092 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:41.093 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:41.127 INFO:tasks.radosbench.radosbench.0.smithi022.stdout: 89 16 118878 118862 83.444 64.375 0.00305295 0.0119765 2024-09-11T07:47:41.194 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:41.219 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:41.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:41.219 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:41.320 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:41.343 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:41.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:41.344 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:41.444 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:41.489 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:41.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:41.490 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:41.591 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:41.619 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:41.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:41.620 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:41.720 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:41.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:41.756 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:41.756 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:41.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:41 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:47:41.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:41 smithi104 ceph-mon[33507]: pgmap v988: 158 pgs: 158 active+clean; 7.2 GiB data, 27 GiB used, 598 GiB / 626 GiB avail; 1023 B/s rd, 63 MiB/s wr, 1.01k op/s 2024-09-11T07:47:41.858 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:41.883 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:41.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:41.884 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:41.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:41 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:47:41.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:41 smithi022 ceph-mon[29462]: pgmap v988: 158 pgs: 158 active+clean; 7.2 GiB data, 27 GiB used, 598 GiB / 626 GiB avail; 1023 B/s rd, 63 MiB/s wr, 1.01k op/s 2024-09-11T07:47:41.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:41 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:47:41.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:41 smithi022 ceph-mon[37330]: pgmap v988: 158 pgs: 158 active+clean; 7.2 GiB data, 27 GiB used, 598 GiB / 626 GiB avail; 1023 B/s rd, 63 MiB/s wr, 1.01k op/s 2024-09-11T07:47:41.985 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:42.027 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:42.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:42.028 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:42.128 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Total time run: 90.0065 2024-09-11T07:47:42.128 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Total writes made: 119805 2024-09-11T07:47:42.128 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Write size: 65536 2024-09-11T07:47:42.128 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Object size: 65536 2024-09-11T07:47:42.128 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Bandwidth (MB/sec): 83.1919 2024-09-11T07:47:42.128 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Stddev Bandwidth: 28.7834 2024-09-11T07:47:42.128 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Max bandwidth (MB/sec): 189.875 2024-09-11T07:47:42.128 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Min bandwidth (MB/sec): 34.125 2024-09-11T07:47:42.128 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Average IOPS: 1331 2024-09-11T07:47:42.128 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Stddev IOPS: 460.534 2024-09-11T07:47:42.128 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Max IOPS: 3038 2024-09-11T07:47:42.129 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Min IOPS: 546 2024-09-11T07:47:42.129 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Average Latency(s): 0.0120165 2024-09-11T07:47:42.129 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Stddev Latency(s): 0.0163057 2024-09-11T07:47:42.129 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Max latency(s): 1.02411 2024-09-11T07:47:42.129 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Min latency(s): 0.00115663 2024-09-11T07:47:42.129 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:42.130 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Cleaning up (deleting benchmark objects) 2024-09-11T07:47:42.154 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:42.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:42.155 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:42.256 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:42.299 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:42.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:42.300 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:42.401 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:42.432 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:47:42.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:42.440 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:42.440 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:42.441 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:47:42.541 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:42.565 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:42.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:42.566 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:42.667 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:42.714 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:47:42.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:42.715 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:42.715 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:42.816 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:42.865 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:42.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:42.867 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:42.967 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:43.008 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:43.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:43.009 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:43.110 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:43.138 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:43.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:43.138 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:43.214 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":988,"stamp":"2024-09-11T07:47:40.426650+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7688088194,"num_objects":117497,"num_object_clones":0,"num_object_copies":352491,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117497,"num_whiteouts":0,"num_read":1756,"num_read_kb":1518,"num_write":118313,"num_write_kb":7507455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":111867,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72661,"ondisk_log_size":72661,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":474,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":28685736,"kb_used_data":22396336,"kb_used_omap":14,"kb_used_meta":6289329,"kb_avail":627530328,"statfs":{"total":671965249536,"available":642591055872,"internally_reserved":0,"allocated":22933848064,"data_stored":22930361763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14826,"internal_metadata":6440273430},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,9,3,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":659423232,"num_objects":10062,"num_object_clones":0,"num_object_copies":30186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10062,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10062,"num_write_kb":643968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003799"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":204,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.794427+0000","last_change":"2024-09-11T07:46:28.323440+0000","last_active":"2024-09-11T07:47:20.794427+0000","last_peered":"2024-09-11T07:47:20.794427+0000","last_clean":"2024-09-11T07:47:20.794427+0000","last_became_active":"2024-09-11T07:44:23.769058+0000","last_became_peered":"2024-09-11T07:44:23.769058+0000","last_unstale":"2024-09-11T07:47:20.794427+0000","last_undegraded":"2024-09-11T07:47:20.794427+0000","last_fullsized":"2024-09-11T07:47:20.794427+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:28.323351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:46:28.323351+0000","last_clean_scrub_stamp":"2024-09-11T07:46:28.323351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:48.922438+0000","scrub_duration":0.0094746069999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":509,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:24.201256+0000","last_change":"2024-09-11T07:47:24.201113+0000","last_active":"2024-09-11T07:47:24.201256+0000","last_peered":"2024-09-11T07:47:24.201256+0000","last_clean":"2024-09-11T07:47:24.201256+0000","last_became_active":"2024-09-11T07:45:27.107996+0000","last_became_peered":"2024-09-11T07:45:27.107996+0000","last_unstale":"2024-09-11T07:47:24.201256+0000","last_undegraded":"2024-09-11T07:47:24.201256+0000","last_fullsized":"2024-09-11T07:47:24.201256+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:24.201007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:47:24.201007+0000","last_clean_scrub_stamp":"2024-09-11T07:47:24.201007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:34.114509+0000","scrub_duration":0.006129532,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"426'46","reported_seq":541,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.790576+0000","last_change":"2024-09-11T07:47:03.746508+0000","last_active":"2024-09-11T07:47:20.790576+0000","last_peered":"2024-09-11T07:47:20.790576+0000","last_clean":"2024-09-11T07:47:20.790576+0000","last_became_active":"2024-09-11T07:26:33.544627+0000","last_became_peered":"2024-09-11T07:26:33.544627+0000","last_unstale":"2024-09-11T07:47:20.790576+0000","last_undegraded":"2024-09-11T07:47:20.790576+0000","last_fullsized":"2024-09-11T07:47:20.790576+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'46","last_scrub_stamp":"2024-09-11T07:47:03.746404+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:47:03.746404+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:06.049251+0000","scrub_duration":0.0054538659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":66,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.781861+0000","last_change":"2024-09-11T07:46:04.878844+0000","last_active":"2024-09-11T07:47:20.781861+0000","last_peered":"2024-09-11T07:47:20.781861+0000","last_clean":"2024-09-11T07:47:20.781861+0000","last_became_active":"2024-09-11T07:45:27.095499+0000","last_became_peered":"2024-09-11T07:45:27.095499+0000","last_unstale":"2024-09-11T07:47:20.781861+0000","last_undegraded":"2024-09-11T07:47:20.781861+0000","last_fullsized":"2024-09-11T07:47:20.781861+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:04.878787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:46:04.878787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013075770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":498,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.789913+0000","last_change":"2024-09-11T07:47:00.693301+0000","last_active":"2024-09-11T07:47:20.789913+0000","last_peered":"2024-09-11T07:47:20.789913+0000","last_clean":"2024-09-11T07:47:20.789913+0000","last_became_active":"2024-09-11T07:39:57.767806+0000","last_became_peered":"2024-09-11T07:39:57.767806+0000","last_unstale":"2024-09-11T07:47:20.789913+0000","last_undegraded":"2024-09-11T07:47:20.789913+0000","last_fullsized":"2024-09-11T07:47:20.789913+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:00.693208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:19.655401+0000","last_clean_scrub_stamp":"2024-09-11T07:47:00.693208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:30.503436+0000","scrub_duration":0.0093827379999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"426'25","reported_seq":85,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.781407+0000","last_change":"2024-09-11T07:47:13.067514+0000","last_active":"2024-09-11T07:47:20.781407+0000","last_peered":"2024-09-11T07:47:20.781407+0000","last_clean":"2024-09-11T07:47:20.781407+0000","last_became_active":"2024-09-11T07:45:27.095974+0000","last_became_peered":"2024-09-11T07:45:27.095974+0000","last_unstale":"2024-09-11T07:47:20.781407+0000","last_undegraded":"2024-09-11T07:47:20.781407+0000","last_fullsized":"2024-09-11T07:47:20.781407+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'25","last_scrub_stamp":"2024-09-11T07:47:13.067271+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:36:42.708941+0000","last_clean_scrub_stamp":"2024-09-11T07:47:13.067271+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:16.797264+0000","scrub_duration":0.012124374,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":208,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.783928+0000","last_change":"2024-09-11T07:45:12.417072+0000","last_active":"2024-09-11T07:47:20.783928+0000","last_peered":"2024-09-11T07:47:20.783928+0000","last_clean":"2024-09-11T07:47:20.783928+0000","last_became_active":"2024-09-11T07:44:23.769502+0000","last_became_peered":"2024-09-11T07:44:23.769502+0000","last_unstale":"2024-09-11T07:47:20.783928+0000","last_undegraded":"2024-09-11T07:47:20.783928+0000","last_fullsized":"2024-09-11T07:47:20.783928+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:12.416990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:45:12.416990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001315529,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":339,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.783208+0000","last_change":"2024-09-11T07:46:31.505181+0000","last_active":"2024-09-11T07:47:20.783208+0000","last_peered":"2024-09-11T07:47:20.783208+0000","last_clean":"2024-09-11T07:47:20.783208+0000","last_became_active":"2024-09-11T07:46:31.504850+0000","last_became_peered":"2024-09-11T07:46:31.504850+0000","last_unstale":"2024-09-11T07:47:20.783208+0000","last_undegraded":"2024-09-11T07:47:20.783208+0000","last_fullsized":"2024-09-11T07:47:20.783208+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:05.327186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:46:05.327186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058146650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":477,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.784176+0000","last_change":"2024-09-11T07:46:21.898482+0000","last_active":"2024-09-11T07:47:20.784176+0000","last_peered":"2024-09-11T07:47:20.784176+0000","last_clean":"2024-09-11T07:47:20.784176+0000","last_became_active":"2024-09-11T07:32:00.516662+0000","last_became_peered":"2024-09-11T07:32:00.516662+0000","last_unstale":"2024-09-11T07:47:20.784176+0000","last_undegraded":"2024-09-11T07:47:20.784176+0000","last_fullsized":"2024-09-11T07:47:20.784176+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:21.898400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:46:21.898400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:40.556920+0000","scrub_duration":0.0063092759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"426'11","reported_seq":75,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.794151+0000","last_change":"2024-09-11T07:45:32.375248+0000","last_active":"2024-09-11T07:47:20.794151+0000","last_peered":"2024-09-11T07:47:20.794151+0000","last_clean":"2024-09-11T07:47:20.794151+0000","last_became_active":"2024-09-11T07:45:27.096974+0000","last_became_peered":"2024-09-11T07:45:27.096974+0000","last_unstale":"2024-09-11T07:47:20.794151+0000","last_undegraded":"2024-09-11T07:47:20.794151+0000","last_fullsized":"2024-09-11T07:47:20.794151+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'9","last_scrub_stamp":"2024-09-11T07:45:32.375156+0000","last_deep_scrub":"234'9","last_deep_scrub_stamp":"2024-09-11T07:45:32.375156+0000","last_clean_scrub_stamp":"2024-09-11T07:45:32.375156+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011478199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":13,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":466,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:26.812250+0000","last_change":"2024-09-11T07:47:26.812154+0000","last_active":"2024-09-11T07:47:26.812250+0000","last_peered":"2024-09-11T07:47:26.812250+0000","last_clean":"2024-09-11T07:47:26.812250+0000","last_became_active":"2024-09-11T07:45:27.112681+0000","last_became_peered":"2024-09-11T07:45:27.112681+0000","last_unstale":"2024-09-11T07:47:26.812250+0000","last_undegraded":"2024-09-11T07:47:26.812250+0000","last_fullsized":"2024-09-11T07:47:26.812250+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:26.812037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.973323+0000","last_clean_scrub_stamp":"2024-09-11T07:47:26.812037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:42.862948+0000","scrub_duration":0.0039630519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":106,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:28.114677+0000","last_change":"2024-09-11T07:47:28.114615+0000","last_active":"2024-09-11T07:47:28.114677+0000","last_peered":"2024-09-11T07:47:28.114677+0000","last_clean":"2024-09-11T07:47:28.114677+0000","last_became_active":"2024-09-11T07:44:23.769187+0000","last_became_peered":"2024-09-11T07:44:23.769187+0000","last_unstale":"2024-09-11T07:47:28.114677+0000","last_undegraded":"2024-09-11T07:47:28.114677+0000","last_fullsized":"2024-09-11T07:47:28.114677+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:28.114522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:45.149916+0000","last_clean_scrub_stamp":"2024-09-11T07:47:28.114522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:47.277277+0000","scrub_duration":0.010263939999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":134,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.790553+0000","last_change":"2024-09-11T07:47:19.164653+0000","last_active":"2024-09-11T07:47:20.790553+0000","last_peered":"2024-09-11T07:47:20.790553+0000","last_clean":"2024-09-11T07:47:20.790553+0000","last_became_active":"2024-09-11T07:45:49.765541+0000","last_became_peered":"2024-09-11T07:45:49.765541+0000","last_unstale":"2024-09-11T07:47:20.790553+0000","last_undegraded":"2024-09-11T07:47:20.790553+0000","last_fullsized":"2024-09-11T07:47:20.790553+0000","mapping_epoch":407,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":408,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:19.164579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:47:19.164579+0000","last_clean_scrub_stamp":"2024-09-11T07:47:19.164579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:20.904858+0000","scrub_duration":0.0028400880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"426'25","reported_seq":348,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.788761+0000","last_change":"2024-09-11T07:46:19.457329+0000","last_active":"2024-09-11T07:47:20.788761+0000","last_peered":"2024-09-11T07:47:20.788761+0000","last_clean":"2024-09-11T07:47:20.788761+0000","last_became_active":"2024-09-11T07:45:27.112765+0000","last_became_peered":"2024-09-11T07:45:27.112765+0000","last_unstale":"2024-09-11T07:47:20.788761+0000","last_undegraded":"2024-09-11T07:47:20.788761+0000","last_fullsized":"2024-09-11T07:47:20.788761+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:46:19.457213+0000","last_deep_scrub":"136'13","last_deep_scrub_stamp":"2024-09-11T07:31:08.042198+0000","last_clean_scrub_stamp":"2024-09-11T07:46:19.457213+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:40.530051+0000","scrub_duration":0.0046268419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":504,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:27.201817+0000","last_change":"2024-09-11T07:47:27.201757+0000","last_active":"2024-09-11T07:47:27.201817+0000","last_peered":"2024-09-11T07:47:27.201817+0000","last_clean":"2024-09-11T07:47:27.201817+0000","last_became_active":"2024-09-11T07:45:27.114162+0000","last_became_peered":"2024-09-11T07:45:27.114162+0000","last_unstale":"2024-09-11T07:47:27.201817+0000","last_undegraded":"2024-09-11T07:47:27.201817+0000","last_fullsized":"2024-09-11T07:47:27.201817+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:27.201533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:47:27.201533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:47.512050+0000","scrub_duration":0.015994092000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":493,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.795670+0000","last_change":"2024-09-11T07:46:24.272062+0000","last_active":"2024-09-11T07:47:20.795670+0000","last_peered":"2024-09-11T07:47:20.795670+0000","last_clean":"2024-09-11T07:47:20.795670+0000","last_became_active":"2024-09-11T07:32:00.530335+0000","last_became_peered":"2024-09-11T07:32:00.530335+0000","last_unstale":"2024-09-11T07:47:20.795670+0000","last_undegraded":"2024-09-11T07:47:20.795670+0000","last_fullsized":"2024-09-11T07:47:20.795670+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:24.271967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:46:24.271967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:38.913016+0000","scrub_duration":0.0042967989999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"441'7295","reported_seq":7332,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-11T07:47:39.371954+0000","last_change":"2024-09-11T07:45:56.765116+0000","last_active":"2024-09-11T07:47:39.371954+0000","last_peered":"2024-09-11T07:47:39.371954+0000","last_clean":"2024-09-11T07:47:39.371954+0000","last_became_active":"2024-09-11T07:45:56.763997+0000","last_became_peered":"2024-09-11T07:45:56.763997+0000","last_unstale":"2024-09-11T07:47:39.371954+0000","last_undegraded":"2024-09-11T07:47:39.371954+0000","last_fullsized":"2024-09-11T07:47:39.371954+0000","mapping_epoch":412,"log_start":"426'3400","ondisk_log_start":"426'3400","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3895,"log_dups_size":0,"ondisk_log_size":3895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":478085120,"num_objects":7295,"num_object_clones":0,"num_object_copies":21885,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7295,"num_write_kb":466880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"87'81","reported_seq":75,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.781329+0000","last_change":"2024-09-11T07:47:12.008908+0000","last_active":"2024-09-11T07:47:20.781329+0000","last_peered":"2024-09-11T07:47:20.781329+0000","last_clean":"2024-09-11T07:47:20.781329+0000","last_became_active":"2024-09-11T07:45:01.808023+0000","last_became_peered":"2024-09-11T07:45:01.808023+0000","last_unstale":"2024-09-11T07:47:20.781329+0000","last_undegraded":"2024-09-11T07:47:20.781329+0000","last_fullsized":"2024-09-11T07:47:20.781329+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:47:12.008803+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:47:12.008803+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:30.856293+0000","scrub_duration":0.004197966,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"426'45","reported_seq":444,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:23.170632+0000","last_change":"2024-09-11T07:47:23.170550+0000","last_active":"2024-09-11T07:47:23.170632+0000","last_peered":"2024-09-11T07:47:23.170632+0000","last_clean":"2024-09-11T07:47:23.170632+0000","last_became_active":"2024-09-11T07:45:27.094411+0000","last_became_peered":"2024-09-11T07:45:27.094411+0000","last_unstale":"2024-09-11T07:47:23.170632+0000","last_undegraded":"2024-09-11T07:47:23.170632+0000","last_fullsized":"2024-09-11T07:47:23.170632+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'45","last_scrub_stamp":"2024-09-11T07:47:23.170462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:47:23.170462+0000","objects_scrubbed":8,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:38.322774+0000","scrub_duration":0.030572766000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":60,"num_write":46,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":485,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.783832+0000","last_change":"2024-09-11T07:45:27.100579+0000","last_active":"2024-09-11T07:47:20.783832+0000","last_peered":"2024-09-11T07:47:20.783832+0000","last_clean":"2024-09-11T07:47:20.783832+0000","last_became_active":"2024-09-11T07:45:27.100238+0000","last_became_peered":"2024-09-11T07:45:27.100238+0000","last_unstale":"2024-09-11T07:47:20.783832+0000","last_undegraded":"2024-09-11T07:47:20.783832+0000","last_fullsized":"2024-09-11T07:47:20.783832+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:08.501392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:08.501392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004339763,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":74,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.794715+0000","last_change":"2024-09-11T07:46:09.235485+0000","last_active":"2024-09-11T07:47:20.794715+0000","last_peered":"2024-09-11T07:47:20.794715+0000","last_clean":"2024-09-11T07:47:20.794715+0000","last_became_active":"2024-09-11T07:45:27.117371+0000","last_became_peered":"2024-09-11T07:45:27.117371+0000","last_unstale":"2024-09-11T07:47:20.794715+0000","last_undegraded":"2024-09-11T07:47:20.794715+0000","last_fullsized":"2024-09-11T07:47:20.794715+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:09.235371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:46:09.235371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016833869999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":499,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.783562+0000","last_change":"2024-09-11T07:47:07.991465+0000","last_active":"2024-09-11T07:47:20.783562+0000","last_peered":"2024-09-11T07:47:20.783562+0000","last_clean":"2024-09-11T07:47:20.783562+0000","last_became_active":"2024-09-11T07:45:27.104519+0000","last_became_peered":"2024-09-11T07:45:27.104519+0000","last_unstale":"2024-09-11T07:47:20.783562+0000","last_undegraded":"2024-09-11T07:47:20.783562+0000","last_fullsized":"2024-09-11T07:47:20.783562+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:07.991385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:32.281886+0000","last_clean_scrub_stamp":"2024-09-11T07:47:07.991385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:30.036131+0000","scrub_duration":0.0036103490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"441'7258","reported_seq":7304,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-11T07:47:39.342433+0000","last_change":"2024-09-11T07:46:31.506264+0000","last_active":"2024-09-11T07:47:39.342433+0000","last_peered":"2024-09-11T07:47:39.342433+0000","last_clean":"2024-09-11T07:47:39.342433+0000","last_became_active":"2024-09-11T07:46:31.502938+0000","last_became_peered":"2024-09-11T07:46:31.502938+0000","last_unstale":"2024-09-11T07:47:39.342433+0000","last_undegraded":"2024-09-11T07:47:39.342433+0000","last_fullsized":"2024-09-11T07:47:39.342433+0000","mapping_epoch":424,"log_start":"426'3400","ondisk_log_start":"426'3400","created":412,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3858,"log_dups_size":0,"ondisk_log_size":3858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":475660288,"num_objects":7258,"num_object_clones":0,"num_object_copies":21774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7258,"num_write_kb":464512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"87'81","reported_seq":70,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.783620+0000","last_change":"2024-09-11T07:46:59.664935+0000","last_active":"2024-09-11T07:47:20.783620+0000","last_peered":"2024-09-11T07:47:20.783620+0000","last_clean":"2024-09-11T07:47:20.783620+0000","last_became_active":"2024-09-11T07:45:01.808923+0000","last_became_peered":"2024-09-11T07:45:01.808923+0000","last_unstale":"2024-09-11T07:47:20.783620+0000","last_undegraded":"2024-09-11T07:47:20.783620+0000","last_fullsized":"2024-09-11T07:47:20.783620+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:46:59.664846+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:46:59.664846+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:14.929909+0000","scrub_duration":0.0074640790000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"426'35","reported_seq":352,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.788886+0000","last_change":"2024-09-11T07:45:33.404225+0000","last_active":"2024-09-11T07:47:20.788886+0000","last_peered":"2024-09-11T07:47:20.788886+0000","last_clean":"2024-09-11T07:47:20.788886+0000","last_became_active":"2024-09-11T07:45:27.097522+0000","last_became_peered":"2024-09-11T07:45:27.097522+0000","last_unstale":"2024-09-11T07:47:20.788886+0000","last_undegraded":"2024-09-11T07:47:20.788886+0000","last_fullsized":"2024-09-11T07:47:20.788886+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'25","last_scrub_stamp":"2024-09-11T07:45:33.403950+0000","last_deep_scrub":"112'15","last_deep_scrub_stamp":"2024-09-11T07:31:10.005896+0000","last_clean_scrub_stamp":"2024-09-11T07:45:33.403950+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044240010000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":212,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.788910+0000","last_change":"2024-09-11T07:46:23.435408+0000","last_active":"2024-09-11T07:47:20.788910+0000","last_peered":"2024-09-11T07:47:20.788910+0000","last_clean":"2024-09-11T07:47:20.788910+0000","last_became_active":"2024-09-11T07:44:23.768172+0000","last_became_peered":"2024-09-11T07:44:23.768172+0000","last_unstale":"2024-09-11T07:47:20.788910+0000","last_undegraded":"2024-09-11T07:47:20.788910+0000","last_fullsized":"2024-09-11T07:47:20.788910+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:23.435037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:46:23.435037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:36.807613+0000","scrub_duration":0.0049327829999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"299'11","reported_seq":775,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-11T07:47:31.508987+0000","last_change":"2024-09-11T07:47:07.004835+0000","last_active":"2024-09-11T07:47:31.508987+0000","last_peered":"2024-09-11T07:47:31.508987+0000","last_clean":"2024-09-11T07:47:31.508987+0000","last_became_active":"2024-09-11T07:40:11.697568+0000","last_became_peered":"2024-09-11T07:40:11.697568+0000","last_unstale":"2024-09-11T07:47:31.508987+0000","last_undegraded":"2024-09-11T07:47:31.508987+0000","last_fullsized":"2024-09-11T07:47:31.508987+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"299'11","last_scrub_stamp":"2024-09-11T07:47:07.004756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:47:07.004756+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:18.785247+0000","scrub_duration":0.013171522,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":364,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.788024+0000","last_change":"2024-09-11T07:46:14.955001+0000","last_active":"2024-09-11T07:47:20.788024+0000","last_peered":"2024-09-11T07:47:20.788024+0000","last_clean":"2024-09-11T07:47:20.788024+0000","last_became_active":"2024-09-11T07:45:27.099434+0000","last_became_peered":"2024-09-11T07:45:27.099434+0000","last_unstale":"2024-09-11T07:47:20.788024+0000","last_undegraded":"2024-09-11T07:47:20.788024+0000","last_fullsized":"2024-09-11T07:47:20.788024+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:14.954917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:46:14.954917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052986090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.0","version":"441'7397","reported_seq":7434,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-11T07:47:40.229587+0000","last_change":"2024-09-11T07:45:56.758475+0000","last_active":"2024-09-11T07:47:40.229587+0000","last_peered":"2024-09-11T07:47:40.229587+0000","last_clean":"2024-09-11T07:47:40.229587+0000","last_became_active":"2024-09-11T07:45:56.758196+0000","last_became_peered":"2024-09-11T07:45:56.758196+0000","last_unstale":"2024-09-11T07:47:40.229587+0000","last_undegraded":"2024-09-11T07:47:40.229587+0000","last_fullsized":"2024-09-11T07:47:40.229587+0000","mapping_epoch":412,"log_start":"425'2400","ondisk_log_start":"425'2400","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":4997,"log_dups_size":0,"ondisk_log_size":4997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":484769792,"num_objects":7397,"num_object_clones":0,"num_object_copies":22191,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7397,"num_write_kb":473408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"87'81","reported_seq":70,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.782848+0000","last_change":"2024-09-11T07:47:09.033253+0000","last_active":"2024-09-11T07:47:20.782848+0000","last_peered":"2024-09-11T07:47:20.782848+0000","last_clean":"2024-09-11T07:47:20.782848+0000","last_became_active":"2024-09-11T07:45:01.805300+0000","last_became_peered":"2024-09-11T07:45:01.805300+0000","last_unstale":"2024-09-11T07:47:20.782848+0000","last_undegraded":"2024-09-11T07:47:20.782848+0000","last_fullsized":"2024-09-11T07:47:20.782848+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:47:09.033169+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:47:09.033169+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:09.236120+0000","scrub_duration":0.001752342,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"426'46","reported_seq":398,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.790408+0000","last_change":"2024-09-11T07:47:15.673799+0000","last_active":"2024-09-11T07:47:20.790408+0000","last_peered":"2024-09-11T07:47:20.790408+0000","last_clean":"2024-09-11T07:47:20.790408+0000","last_became_active":"2024-09-11T07:45:27.095634+0000","last_became_peered":"2024-09-11T07:45:27.095634+0000","last_unstale":"2024-09-11T07:47:20.790408+0000","last_undegraded":"2024-09-11T07:47:20.790408+0000","last_fullsized":"2024-09-11T07:47:20.790408+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'46","last_scrub_stamp":"2024-09-11T07:47:15.673697+0000","last_deep_scrub":"83'22","last_deep_scrub_stamp":"2024-09-11T07:34:04.375457+0000","last_clean_scrub_stamp":"2024-09-11T07:47:15.673697+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:26.577269+0000","scrub_duration":0.011487635,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":232,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.783622+0000","last_change":"2024-09-11T07:47:09.781123+0000","last_active":"2024-09-11T07:47:20.783622+0000","last_peered":"2024-09-11T07:47:20.783622+0000","last_clean":"2024-09-11T07:47:20.783622+0000","last_became_active":"2024-09-11T07:44:23.770343+0000","last_became_peered":"2024-09-11T07:44:23.770343+0000","last_unstale":"2024-09-11T07:47:20.783622+0000","last_undegraded":"2024-09-11T07:47:20.783622+0000","last_fullsized":"2024-09-11T07:47:20.783622+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:09.781047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:27.416207+0000","last_clean_scrub_stamp":"2024-09-11T07:47:09.781047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:16.887151+0000","scrub_duration":0.0044366759999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"395'10","reported_seq":691,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-11T07:47:31.520659+0000","last_change":"2024-09-11T07:46:10.279342+0000","last_active":"2024-09-11T07:47:31.520659+0000","last_peered":"2024-09-11T07:47:31.520659+0000","last_clean":"2024-09-11T07:47:31.520659+0000","last_became_active":"2024-09-11T07:45:27.122326+0000","last_became_peered":"2024-09-11T07:45:27.122326+0000","last_unstale":"2024-09-11T07:47:31.520659+0000","last_undegraded":"2024-09-11T07:47:31.520659+0000","last_fullsized":"2024-09-11T07:47:31.520659+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"395'10","last_scrub_stamp":"2024-09-11T07:46:10.279284+0000","last_deep_scrub":"366'9","last_deep_scrub_stamp":"2024-09-11T07:43:26.334501+0000","last_clean_scrub_stamp":"2024-09-11T07:46:10.279284+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001636265,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":331,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.795800+0000","last_change":"2024-09-11T07:45:49.765810+0000","last_active":"2024-09-11T07:47:20.795800+0000","last_peered":"2024-09-11T07:47:20.795800+0000","last_clean":"2024-09-11T07:47:20.795800+0000","last_became_active":"2024-09-11T07:45:49.765504+0000","last_became_peered":"2024-09-11T07:45:49.765504+0000","last_unstale":"2024-09-11T07:47:20.795800+0000","last_undegraded":"2024-09-11T07:47:20.795800+0000","last_fullsized":"2024-09-11T07:47:20.795800+0000","mapping_epoch":407,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":408,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:19.336454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:31.200685+0000","last_clean_scrub_stamp":"2024-09-11T07:45:19.336454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001371688,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"441'7311","reported_seq":7347,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-11T07:47:39.488822+0000","last_change":"2024-09-11T07:45:57.199288+0000","last_active":"2024-09-11T07:47:39.488822+0000","last_peered":"2024-09-11T07:47:39.488822+0000","last_clean":"2024-09-11T07:47:39.488822+0000","last_became_active":"2024-09-11T07:45:57.198828+0000","last_became_peered":"2024-09-11T07:45:57.198828+0000","last_unstale":"2024-09-11T07:47:39.488822+0000","last_undegraded":"2024-09-11T07:47:39.488822+0000","last_fullsized":"2024-09-11T07:47:39.488822+0000","mapping_epoch":412,"log_start":"426'3600","ondisk_log_start":"426'3600","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3711,"log_dups_size":0,"ondisk_log_size":3711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":479133696,"num_objects":7311,"num_object_clones":0,"num_object_copies":21933,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7311,"num_write_kb":467904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"426'32","reported_seq":428,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.789623+0000","last_change":"2024-09-11T07:46:24.388932+0000","last_active":"2024-09-11T07:47:20.789623+0000","last_peered":"2024-09-11T07:47:20.789623+0000","last_clean":"2024-09-11T07:47:20.789623+0000","last_became_active":"2024-09-11T07:45:27.102164+0000","last_became_peered":"2024-09-11T07:45:27.102164+0000","last_unstale":"2024-09-11T07:47:20.789623+0000","last_undegraded":"2024-09-11T07:47:20.789623+0000","last_fullsized":"2024-09-11T07:47:20.789623+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'24","last_scrub_stamp":"2024-09-11T07:46:24.388838+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-09-11T07:29:38.112111+0000","last_clean_scrub_stamp":"2024-09-11T07:46:24.388838+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:39.786424+0000","scrub_duration":0.0055826570000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":469,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.783564+0000","last_change":"2024-09-11T07:46:12.925725+0000","last_active":"2024-09-11T07:47:20.783564+0000","last_peered":"2024-09-11T07:47:20.783564+0000","last_clean":"2024-09-11T07:47:20.783564+0000","last_became_active":"2024-09-11T07:32:00.518538+0000","last_became_peered":"2024-09-11T07:32:00.518538+0000","last_unstale":"2024-09-11T07:47:20.783564+0000","last_undegraded":"2024-09-11T07:47:20.783564+0000","last_fullsized":"2024-09-11T07:47:20.783564+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:12.925664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:43:24.643676+0000","last_clean_scrub_stamp":"2024-09-11T07:46:12.925664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026407430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":66,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.782993+0000","last_change":"2024-09-11T07:45:30.682670+0000","last_active":"2024-09-11T07:47:20.782993+0000","last_peered":"2024-09-11T07:47:20.782993+0000","last_clean":"2024-09-11T07:47:20.782993+0000","last_became_active":"2024-09-11T07:45:27.094025+0000","last_became_peered":"2024-09-11T07:45:27.094025+0000","last_unstale":"2024-09-11T07:47:20.782993+0000","last_undegraded":"2024-09-11T07:47:20.782993+0000","last_fullsized":"2024-09-11T07:47:20.782993+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:30.682614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:19.714568+0000","last_clean_scrub_stamp":"2024-09-11T07:45:30.682614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015960729999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":466,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.788644+0000","last_change":"2024-09-11T07:45:17.470823+0000","last_active":"2024-09-11T07:47:20.788644+0000","last_peered":"2024-09-11T07:47:20.788644+0000","last_clean":"2024-09-11T07:47:20.788644+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:47:20.788644+0000","last_undegraded":"2024-09-11T07:47:20.788644+0000","last_fullsized":"2024-09-11T07:47:20.788644+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:17.470730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:46.853387+0000","last_clean_scrub_stamp":"2024-09-11T07:45:17.470730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002761412,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.e","version":"441'7406","reported_seq":7443,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-11T07:47:39.563022+0000","last_change":"2024-09-11T07:45:56.762206+0000","last_active":"2024-09-11T07:47:39.563022+0000","last_peered":"2024-09-11T07:47:39.563022+0000","last_clean":"2024-09-11T07:47:39.563022+0000","last_became_active":"2024-09-11T07:45:56.761971+0000","last_became_peered":"2024-09-11T07:45:56.761971+0000","last_unstale":"2024-09-11T07:47:39.563022+0000","last_undegraded":"2024-09-11T07:47:39.563022+0000","last_fullsized":"2024-09-11T07:47:39.563022+0000","mapping_epoch":412,"log_start":"422'1900","ondisk_log_start":"422'1900","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":5506,"log_dups_size":0,"ondisk_log_size":5506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":485359616,"num_objects":7406,"num_object_clones":0,"num_object_copies":22218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7406,"num_write_kb":473984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"87'81","reported_seq":89,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.788468+0000","last_change":"2024-09-11T07:47:06.691785+0000","last_active":"2024-09-11T07:47:20.788468+0000","last_peered":"2024-09-11T07:47:20.788468+0000","last_clean":"2024-09-11T07:47:20.788468+0000","last_became_active":"2024-09-11T07:45:27.097022+0000","last_became_peered":"2024-09-11T07:45:27.097022+0000","last_unstale":"2024-09-11T07:47:20.788468+0000","last_undegraded":"2024-09-11T07:47:20.788468+0000","last_fullsized":"2024-09-11T07:47:20.788468+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:47:06.691689+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:47:06.691689+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:18.242055+0000","scrub_duration":0.0033647320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"426'42","reported_seq":317,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.797640+0000","last_change":"2024-09-11T07:46:31.496469+0000","last_active":"2024-09-11T07:47:20.797640+0000","last_peered":"2024-09-11T07:47:20.797640+0000","last_clean":"2024-09-11T07:47:20.797640+0000","last_became_active":"2024-09-11T07:46:31.496001+0000","last_became_peered":"2024-09-11T07:46:31.496001+0000","last_unstale":"2024-09-11T07:47:20.797640+0000","last_undegraded":"2024-09-11T07:47:20.797640+0000","last_fullsized":"2024-09-11T07:47:20.797640+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'28","last_scrub_stamp":"2024-09-11T07:46:08.280072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:46:08.280072+0000","objects_scrubbed":7,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012975469999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":55,"num_write":42,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":248,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.783685+0000","last_change":"2024-09-11T07:47:00.850246+0000","last_active":"2024-09-11T07:47:20.783685+0000","last_peered":"2024-09-11T07:47:20.783685+0000","last_clean":"2024-09-11T07:47:20.783685+0000","last_became_active":"2024-09-11T07:39:57.767741+0000","last_became_peered":"2024-09-11T07:39:57.767741+0000","last_unstale":"2024-09-11T07:47:20.783685+0000","last_undegraded":"2024-09-11T07:47:20.783685+0000","last_fullsized":"2024-09-11T07:47:20.783685+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:00.850169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:47:00.850169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:11.876461+0000","scrub_duration":0.0023451930000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3],"acting":[2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"395'10","reported_seq":746,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-11T07:47:34.764419+0000","last_change":"2024-09-11T07:47:13.601869+0000","last_active":"2024-09-11T07:47:34.764419+0000","last_peered":"2024-09-11T07:47:34.764419+0000","last_clean":"2024-09-11T07:47:34.764419+0000","last_became_active":"2024-09-11T07:45:27.098861+0000","last_became_peered":"2024-09-11T07:45:27.098861+0000","last_unstale":"2024-09-11T07:47:34.764419+0000","last_undegraded":"2024-09-11T07:47:34.764419+0000","last_fullsized":"2024-09-11T07:47:34.764419+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"395'10","last_scrub_stamp":"2024-09-11T07:47:13.601667+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-11T07:35:39.832547+0000","last_clean_scrub_stamp":"2024-09-11T07:47:13.601667+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:39.044308+0000","scrub_duration":0.0076782939999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":450,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.783751+0000","last_change":"2024-09-11T07:47:10.766460+0000","last_active":"2024-09-11T07:47:20.783751+0000","last_peered":"2024-09-11T07:47:20.783751+0000","last_clean":"2024-09-11T07:47:20.783751+0000","last_became_active":"2024-09-11T07:32:00.511005+0000","last_became_peered":"2024-09-11T07:32:00.511005+0000","last_unstale":"2024-09-11T07:47:20.783751+0000","last_undegraded":"2024-09-11T07:47:20.783751+0000","last_fullsized":"2024-09-11T07:47:20.783751+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:10.766409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:47:10.766409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:27.174979+0000","scrub_duration":0.0017833829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"441'7253","reported_seq":7290,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-11T07:47:39.566179+0000","last_change":"2024-09-11T07:45:56.764982+0000","last_active":"2024-09-11T07:47:39.566179+0000","last_peered":"2024-09-11T07:47:39.566179+0000","last_clean":"2024-09-11T07:47:39.566179+0000","last_became_active":"2024-09-11T07:45:56.764754+0000","last_became_peered":"2024-09-11T07:45:56.764754+0000","last_unstale":"2024-09-11T07:47:39.566179+0000","last_undegraded":"2024-09-11T07:47:39.566179+0000","last_fullsized":"2024-09-11T07:47:39.566179+0000","mapping_epoch":412,"log_start":"422'1700","ondisk_log_start":"422'1700","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":5553,"log_dups_size":0,"ondisk_log_size":5553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":475332608,"num_objects":7253,"num_object_clones":0,"num_object_copies":21759,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7253,"num_write_kb":464192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"87'81","reported_seq":70,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:22.146414+0000","last_change":"2024-09-11T07:47:22.146275+0000","last_active":"2024-09-11T07:47:22.146414+0000","last_peered":"2024-09-11T07:47:22.146414+0000","last_clean":"2024-09-11T07:47:22.146414+0000","last_became_active":"2024-09-11T07:45:27.099677+0000","last_became_peered":"2024-09-11T07:45:27.099677+0000","last_unstale":"2024-09-11T07:47:22.146414+0000","last_undegraded":"2024-09-11T07:47:22.146414+0000","last_fullsized":"2024-09-11T07:47:22.146414+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:47:22.145799+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:47:22.145799+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:25.836936+0000","scrub_duration":0.0049845239999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"426'50","reported_seq":564,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.795767+0000","last_change":"2024-09-11T07:46:26.243185+0000","last_active":"2024-09-11T07:47:20.795767+0000","last_peered":"2024-09-11T07:47:20.795767+0000","last_clean":"2024-09-11T07:47:20.795767+0000","last_became_active":"2024-09-11T07:46:11.762572+0000","last_became_peered":"2024-09-11T07:46:11.762572+0000","last_unstale":"2024-09-11T07:47:20.795767+0000","last_undegraded":"2024-09-11T07:47:20.795767+0000","last_fullsized":"2024-09-11T07:47:20.795767+0000","mapping_epoch":418,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":419,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'38","last_scrub_stamp":"2024-09-11T07:46:26.243126+0000","last_deep_scrub":"234'38","last_deep_scrub_stamp":"2024-09-11T07:46:26.243126+0000","last_clean_scrub_stamp":"2024-09-11T07:46:26.243126+0000","objects_scrubbed":8,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:34.336031+0000","scrub_duration":0.0022410799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":69,"num_read_kb":57,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,5],"acting":[0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":63,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.789536+0000","last_change":"2024-09-11T07:45:27.113078+0000","last_active":"2024-09-11T07:47:20.789536+0000","last_peered":"2024-09-11T07:47:20.789536+0000","last_clean":"2024-09-11T07:47:20.789536+0000","last_became_active":"2024-09-11T07:45:27.112250+0000","last_became_peered":"2024-09-11T07:45:27.112250+0000","last_unstale":"2024-09-11T07:47:20.789536+0000","last_undegraded":"2024-09-11T07:47:20.789536+0000","last_fullsized":"2024-09-11T07:47:20.789536+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:16.022013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:16.022013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":376,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.790371+0000","last_change":"2024-09-11T07:45:31.356987+0000","last_active":"2024-09-11T07:47:20.790371+0000","last_peered":"2024-09-11T07:47:20.790371+0000","last_clean":"2024-09-11T07:47:20.790371+0000","last_became_active":"2024-09-11T07:45:27.094619+0000","last_became_peered":"2024-09-11T07:45:27.094619+0000","last_unstale":"2024-09-11T07:47:20.790371+0000","last_undegraded":"2024-09-11T07:47:20.790371+0000","last_fullsized":"2024-09-11T07:47:20.790371+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:31.356921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:27.968579+0000","last_clean_scrub_stamp":"2024-09-11T07:45:31.356921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001320955,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":364,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.783069+0000","last_change":"2024-09-11T07:46:02.831398+0000","last_active":"2024-09-11T07:47:20.783069+0000","last_peered":"2024-09-11T07:47:20.783069+0000","last_clean":"2024-09-11T07:47:20.783069+0000","last_became_active":"2024-09-11T07:45:27.098502+0000","last_became_peered":"2024-09-11T07:45:27.098502+0000","last_unstale":"2024-09-11T07:47:20.783069+0000","last_undegraded":"2024-09-11T07:47:20.783069+0000","last_fullsized":"2024-09-11T07:47:20.783069+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:02.831341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:29.978798+0000","last_clean_scrub_stamp":"2024-09-11T07:46:02.831341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012920449999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.c","version":"441'7187","reported_seq":7224,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-11T07:47:39.344092+0000","last_change":"2024-09-11T07:45:56.764201+0000","last_active":"2024-09-11T07:47:39.344092+0000","last_peered":"2024-09-11T07:47:39.344092+0000","last_clean":"2024-09-11T07:47:39.344092+0000","last_became_active":"2024-09-11T07:45:56.763888+0000","last_became_peered":"2024-09-11T07:45:56.763888+0000","last_unstale":"2024-09-11T07:47:39.344092+0000","last_undegraded":"2024-09-11T07:47:39.344092+0000","last_fullsized":"2024-09-11T07:47:39.344092+0000","mapping_epoch":412,"log_start":"426'3300","ondisk_log_start":"426'3300","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3887,"log_dups_size":0,"ondisk_log_size":3887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":471007232,"num_objects":7187,"num_object_clones":0,"num_object_copies":21561,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7187,"num_write_kb":459968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,7],"acting":[2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"87'81","reported_seq":349,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.796295+0000","last_change":"2024-09-11T07:47:04.284321+0000","last_active":"2024-09-11T07:47:20.796295+0000","last_peered":"2024-09-11T07:47:20.796295+0000","last_clean":"2024-09-11T07:47:20.796295+0000","last_became_active":"2024-09-11T07:45:27.111145+0000","last_became_peered":"2024-09-11T07:45:27.111145+0000","last_unstale":"2024-09-11T07:47:20.796295+0000","last_undegraded":"2024-09-11T07:47:20.796295+0000","last_fullsized":"2024-09-11T07:47:20.796295+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:47:04.284107+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:47:04.284107+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:29.575056+0000","scrub_duration":0.011865242,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"426'42","reported_seq":556,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.794939+0000","last_change":"2024-09-11T07:45:13.572062+0000","last_active":"2024-09-11T07:47:20.794939+0000","last_peered":"2024-09-11T07:47:20.794939+0000","last_clean":"2024-09-11T07:47:20.794939+0000","last_became_active":"2024-09-11T07:32:00.506270+0000","last_became_peered":"2024-09-11T07:32:00.506270+0000","last_unstale":"2024-09-11T07:47:20.794939+0000","last_undegraded":"2024-09-11T07:47:20.794939+0000","last_fullsized":"2024-09-11T07:47:20.794939+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'32","last_scrub_stamp":"2024-09-11T07:45:13.571949+0000","last_deep_scrub":"236'32","last_deep_scrub_stamp":"2024-09-11T07:36:50.023909+0000","last_clean_scrub_stamp":"2024-09-11T07:45:13.571949+0000","objects_scrubbed":8,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020494009000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":69,"num_read_kb":55,"num_write":42,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":460,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.784102+0000","last_change":"2024-09-11T07:47:05.762167+0000","last_active":"2024-09-11T07:47:20.784102+0000","last_peered":"2024-09-11T07:47:20.784102+0000","last_clean":"2024-09-11T07:47:20.784102+0000","last_became_active":"2024-09-11T07:45:27.113680+0000","last_became_peered":"2024-09-11T07:45:27.113680+0000","last_unstale":"2024-09-11T07:47:20.784102+0000","last_undegraded":"2024-09-11T07:47:20.784102+0000","last_fullsized":"2024-09-11T07:47:20.784102+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:05.762110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:24.470602+0000","last_clean_scrub_stamp":"2024-09-11T07:47:05.762110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:28.143244+0000","scrub_duration":0.0077140580000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":279,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.794520+0000","last_change":"2024-09-11T07:47:18.269182+0000","last_active":"2024-09-11T07:47:20.794520+0000","last_peered":"2024-09-11T07:47:20.794520+0000","last_clean":"2024-09-11T07:47:20.794520+0000","last_became_active":"2024-09-11T07:36:05.225840+0000","last_became_peered":"2024-09-11T07:36:05.225840+0000","last_unstale":"2024-09-11T07:47:20.794520+0000","last_undegraded":"2024-09-11T07:47:20.794520+0000","last_fullsized":"2024-09-11T07:47:20.794520+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:18.269109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:22.285763+0000","last_clean_scrub_stamp":"2024-09-11T07:47:18.269109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:20.056073+0000","scrub_duration":0.0050633400000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":84,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.785262+0000","last_change":"2024-09-11T07:45:23.663301+0000","last_active":"2024-09-11T07:47:20.785262+0000","last_peered":"2024-09-11T07:47:20.785262+0000","last_clean":"2024-09-11T07:47:20.785262+0000","last_became_active":"2024-09-11T07:44:23.769063+0000","last_became_peered":"2024-09-11T07:44:23.769063+0000","last_unstale":"2024-09-11T07:47:20.785262+0000","last_undegraded":"2024-09-11T07:47:20.785262+0000","last_fullsized":"2024-09-11T07:47:20.785262+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:23.663218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:39.723598+0000","last_clean_scrub_stamp":"2024-09-11T07:45:23.663218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006627272,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"441'7267","reported_seq":7304,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-11T07:47:39.386516+0000","last_change":"2024-09-11T07:45:57.201789+0000","last_active":"2024-09-11T07:47:39.386516+0000","last_peered":"2024-09-11T07:47:39.386516+0000","last_clean":"2024-09-11T07:47:39.386516+0000","last_became_active":"2024-09-11T07:45:57.201368+0000","last_became_peered":"2024-09-11T07:45:57.201368+0000","last_unstale":"2024-09-11T07:47:39.386516+0000","last_undegraded":"2024-09-11T07:47:39.386516+0000","last_fullsized":"2024-09-11T07:47:39.386516+0000","mapping_epoch":412,"log_start":"426'3400","ondisk_log_start":"426'3400","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3867,"log_dups_size":0,"ondisk_log_size":3867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":476250112,"num_objects":7267,"num_object_clones":0,"num_object_copies":21801,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7267,"num_write_kb":465088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"87'81","reported_seq":30,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.783288+0000","last_change":"2024-09-11T07:46:31.504183+0000","last_active":"2024-09-11T07:47:20.783288+0000","last_peered":"2024-09-11T07:47:20.783288+0000","last_clean":"2024-09-11T07:47:20.783288+0000","last_became_active":"2024-09-11T07:46:31.503868+0000","last_became_peered":"2024-09-11T07:46:31.503868+0000","last_unstale":"2024-09-11T07:47:20.783288+0000","last_undegraded":"2024-09-11T07:47:20.783288+0000","last_fullsized":"2024-09-11T07:47:20.783288+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:46:18.111609+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:46:18.111609+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015232676000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"426'30","reported_seq":363,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.794962+0000","last_change":"2024-09-11T07:46:08.202429+0000","last_active":"2024-09-11T07:47:20.794962+0000","last_peered":"2024-09-11T07:47:20.794962+0000","last_clean":"2024-09-11T07:47:20.794962+0000","last_became_active":"2024-09-11T07:45:27.119509+0000","last_became_peered":"2024-09-11T07:45:27.119509+0000","last_unstale":"2024-09-11T07:47:20.794962+0000","last_undegraded":"2024-09-11T07:47:20.794962+0000","last_fullsized":"2024-09-11T07:47:20.794962+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'20","last_scrub_stamp":"2024-09-11T07:46:08.202366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:46:08.202366+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024018009999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":92,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.794800+0000","last_change":"2024-09-11T07:45:12.553757+0000","last_active":"2024-09-11T07:47:20.794800+0000","last_peered":"2024-09-11T07:47:20.794800+0000","last_clean":"2024-09-11T07:47:20.794800+0000","last_became_active":"2024-09-11T07:44:23.771022+0000","last_became_peered":"2024-09-11T07:44:23.771022+0000","last_unstale":"2024-09-11T07:47:20.794800+0000","last_undegraded":"2024-09-11T07:47:20.794800+0000","last_fullsized":"2024-09-11T07:47:20.794800+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:45:12.551633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:12.551633+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014607669,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"290'9","reported_seq":753,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-11T07:47:31.514469+0000","last_change":"2024-09-11T07:45:21.492545+0000","last_active":"2024-09-11T07:47:31.514469+0000","last_peered":"2024-09-11T07:47:31.514469+0000","last_clean":"2024-09-11T07:47:31.514469+0000","last_became_active":"2024-09-11T07:39:57.768155+0000","last_became_peered":"2024-09-11T07:39:57.768155+0000","last_unstale":"2024-09-11T07:47:31.514469+0000","last_undegraded":"2024-09-11T07:47:31.514469+0000","last_fullsized":"2024-09-11T07:47:31.514469+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"290'9","last_scrub_stamp":"2024-09-11T07:45:21.492460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:45:21.492460+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0074254259999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":476,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.795550+0000","last_change":"2024-09-11T07:46:17.190867+0000","last_active":"2024-09-11T07:47:20.795550+0000","last_peered":"2024-09-11T07:47:20.795550+0000","last_clean":"2024-09-11T07:47:20.795550+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:47:20.795550+0000","last_undegraded":"2024-09-11T07:47:20.795550+0000","last_fullsized":"2024-09-11T07:47:20.795550+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:17.190770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_clean_scrub_stamp":"2024-09-11T07:46:17.190770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007916569,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"441'7438","reported_seq":7474,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-11T07:47:40.246135+0000","last_change":"2024-09-11T07:45:57.199626+0000","last_active":"2024-09-11T07:47:40.246135+0000","last_peered":"2024-09-11T07:47:40.246135+0000","last_clean":"2024-09-11T07:47:40.246135+0000","last_became_active":"2024-09-11T07:45:57.199305+0000","last_became_peered":"2024-09-11T07:45:57.199305+0000","last_unstale":"2024-09-11T07:47:40.246135+0000","last_undegraded":"2024-09-11T07:47:40.246135+0000","last_fullsized":"2024-09-11T07:47:40.246135+0000","mapping_epoch":412,"log_start":"425'2500","ondisk_log_start":"425'2500","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":4938,"log_dups_size":0,"ondisk_log_size":4938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":487456768,"num_objects":7438,"num_object_clones":0,"num_object_copies":22314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7438,"num_write_kb":476032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"87'81","reported_seq":338,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.797811+0000","last_change":"2024-09-11T07:46:21.225100+0000","last_active":"2024-09-11T07:47:20.797811+0000","last_peered":"2024-09-11T07:47:20.797811+0000","last_clean":"2024-09-11T07:47:20.797811+0000","last_became_active":"2024-09-11T07:45:01.820927+0000","last_became_peered":"2024-09-11T07:45:01.820927+0000","last_unstale":"2024-09-11T07:47:20.797811+0000","last_undegraded":"2024-09-11T07:47:20.797811+0000","last_fullsized":"2024-09-11T07:47:20.797811+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:46:21.225015+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:46:21.225015+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050456870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"426'54","reported_seq":499,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-11T07:47:31.527505+0000","last_change":"2024-09-11T07:46:17.005728+0000","last_active":"2024-09-11T07:47:31.527505+0000","last_peered":"2024-09-11T07:47:31.527505+0000","last_clean":"2024-09-11T07:47:31.527505+0000","last_became_active":"2024-09-11T07:39:57.768267+0000","last_became_peered":"2024-09-11T07:39:57.768267+0000","last_unstale":"2024-09-11T07:47:31.527505+0000","last_undegraded":"2024-09-11T07:47:31.527505+0000","last_fullsized":"2024-09-11T07:47:31.527505+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"290'42","last_scrub_stamp":"2024-09-11T07:46:17.005627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:46:17.005627+0000","objects_scrubbed":10,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:35.160644+0000","scrub_duration":0.0098261949999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":83,"num_read_kb":68,"num_write":50,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":459,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.772744+0000","last_change":"2024-09-11T07:47:01.254434+0000","last_active":"2024-09-11T07:47:20.772744+0000","last_peered":"2024-09-11T07:47:20.772744+0000","last_clean":"2024-09-11T07:47:20.772744+0000","last_became_active":"2024-09-11T07:44:23.772612+0000","last_became_peered":"2024-09-11T07:44:23.772612+0000","last_unstale":"2024-09-11T07:47:20.772744+0000","last_undegraded":"2024-09-11T07:47:20.772744+0000","last_fullsized":"2024-09-11T07:47:20.772744+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:01.254358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:51.308492+0000","last_clean_scrub_stamp":"2024-09-11T07:47:01.254358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:09.261035+0000","scrub_duration":0.0038796210000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":488,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:26.192772+0000","last_change":"2024-09-11T07:47:26.192696+0000","last_active":"2024-09-11T07:47:26.192772+0000","last_peered":"2024-09-11T07:47:26.192772+0000","last_clean":"2024-09-11T07:47:26.192772+0000","last_became_active":"2024-09-11T07:45:27.120021+0000","last_became_peered":"2024-09-11T07:45:27.120021+0000","last_unstale":"2024-09-11T07:47:26.192772+0000","last_undegraded":"2024-09-11T07:47:26.192772+0000","last_fullsized":"2024-09-11T07:47:26.192772+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:26.192608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:47:26.192608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:38.745195+0000","scrub_duration":0.004060386,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":493,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.789843+0000","last_change":"2024-09-11T07:45:27.103089+0000","last_active":"2024-09-11T07:47:20.789843+0000","last_peered":"2024-09-11T07:47:20.789843+0000","last_clean":"2024-09-11T07:47:20.789843+0000","last_became_active":"2024-09-11T07:45:27.102600+0000","last_became_peered":"2024-09-11T07:45:27.102600+0000","last_unstale":"2024-09-11T07:47:20.789843+0000","last_undegraded":"2024-09-11T07:47:20.789843+0000","last_fullsized":"2024-09-11T07:47:20.789843+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:13.504354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:45:13.504354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018451749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"441'7382","reported_seq":7419,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-11T07:47:39.477848+0000","last_change":"2024-09-11T07:45:57.198992+0000","last_active":"2024-09-11T07:47:39.477848+0000","last_peered":"2024-09-11T07:47:39.477848+0000","last_clean":"2024-09-11T07:47:39.477848+0000","last_became_active":"2024-09-11T07:45:57.198709+0000","last_became_peered":"2024-09-11T07:45:57.198709+0000","last_unstale":"2024-09-11T07:47:39.477848+0000","last_undegraded":"2024-09-11T07:47:39.477848+0000","last_fullsized":"2024-09-11T07:47:39.477848+0000","mapping_epoch":412,"log_start":"426'3600","ondisk_log_start":"426'3600","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3782,"log_dups_size":0,"ondisk_log_size":3782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483786752,"num_objects":7382,"num_object_clones":0,"num_object_copies":22146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7382,"num_write_kb":472448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"87'81","reported_seq":349,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.794119+0000","last_change":"2024-09-11T07:46:29.293393+0000","last_active":"2024-09-11T07:47:20.794119+0000","last_peered":"2024-09-11T07:47:20.794119+0000","last_clean":"2024-09-11T07:47:20.794119+0000","last_became_active":"2024-09-11T07:45:27.117020+0000","last_became_peered":"2024-09-11T07:45:27.117020+0000","last_unstale":"2024-09-11T07:47:20.794119+0000","last_undegraded":"2024-09-11T07:47:20.794119+0000","last_fullsized":"2024-09-11T07:47:20.794119+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:46:29.293202+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:46:29.293202+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:54.265960+0000","scrub_duration":0.0030387169999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,5],"acting":[0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"426'26","reported_seq":310,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.797471+0000","last_change":"2024-09-11T07:45:49.765866+0000","last_active":"2024-09-11T07:47:20.797471+0000","last_peered":"2024-09-11T07:47:20.797471+0000","last_clean":"2024-09-11T07:47:20.797471+0000","last_became_active":"2024-09-11T07:45:49.765519+0000","last_became_peered":"2024-09-11T07:45:49.765519+0000","last_unstale":"2024-09-11T07:47:20.797471+0000","last_undegraded":"2024-09-11T07:47:20.797471+0000","last_fullsized":"2024-09-11T07:47:20.797471+0000","mapping_epoch":407,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":408,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'22","last_scrub_stamp":"2024-09-11T07:45:17.380652+0000","last_deep_scrub":"144'18","last_deep_scrub_stamp":"2024-09-11T07:31:51.603378+0000","last_clean_scrub_stamp":"2024-09-11T07:45:17.380652+0000","objects_scrubbed":5,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006530999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":41,"num_read_kb":30,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":308,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-11T07:47:38.383245+0000","last_change":"2024-09-11T07:45:20.637602+0000","last_active":"2024-09-11T07:47:38.383245+0000","last_peered":"2024-09-11T07:47:38.383245+0000","last_clean":"2024-09-11T07:47:38.383245+0000","last_became_active":"2024-09-11T07:44:23.769220+0000","last_became_peered":"2024-09-11T07:44:23.769220+0000","last_unstale":"2024-09-11T07:47:38.383245+0000","last_undegraded":"2024-09-11T07:47:38.383245+0000","last_fullsized":"2024-09-11T07:47:38.383245+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'5","last_scrub_stamp":"2024-09-11T07:45:20.637535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:20.637535+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059316919999999997,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":202,"num_read_kb":197,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":501,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.785382+0000","last_change":"2024-09-11T07:45:07.727921+0000","last_active":"2024-09-11T07:47:20.785382+0000","last_peered":"2024-09-11T07:47:20.785382+0000","last_clean":"2024-09-11T07:47:20.785382+0000","last_became_active":"2024-09-11T07:26:31.413354+0000","last_became_peered":"2024-09-11T07:26:31.413354+0000","last_unstale":"2024-09-11T07:47:20.785382+0000","last_undegraded":"2024-09-11T07:47:20.785382+0000","last_fullsized":"2024-09-11T07:47:20.785382+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:07.727847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:07.149367+0000","last_clean_scrub_stamp":"2024-09-11T07:45:07.727847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053654319999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":498,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.788142+0000","last_change":"2024-09-11T07:47:01.841921+0000","last_active":"2024-09-11T07:47:20.788142+0000","last_peered":"2024-09-11T07:47:20.788142+0000","last_clean":"2024-09-11T07:47:20.788142+0000","last_became_active":"2024-09-11T07:32:00.508398+0000","last_became_peered":"2024-09-11T07:32:00.508398+0000","last_unstale":"2024-09-11T07:47:20.788142+0000","last_undegraded":"2024-09-11T07:47:20.788142+0000","last_fullsized":"2024-09-11T07:47:20.788142+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:01.841824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:28.246601+0000","last_clean_scrub_stamp":"2024-09-11T07:47:01.841824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:14.043657+0000","scrub_duration":0.039998764999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":476,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:22.781614+0000","last_change":"2024-09-11T07:47:22.781568+0000","last_active":"2024-09-11T07:47:22.781614+0000","last_peered":"2024-09-11T07:47:22.781614+0000","last_clean":"2024-09-11T07:47:22.781614+0000","last_became_active":"2024-09-11T07:32:00.515745+0000","last_became_peered":"2024-09-11T07:32:00.515745+0000","last_unstale":"2024-09-11T07:47:22.781614+0000","last_undegraded":"2024-09-11T07:47:22.781614+0000","last_fullsized":"2024-09-11T07:47:22.781614+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:22.781517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:25.382533+0000","last_clean_scrub_stamp":"2024-09-11T07:47:22.781517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:25.208018+0000","scrub_duration":0.0017426900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"441'7383","reported_seq":7420,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-11T07:47:40.250765+0000","last_change":"2024-09-11T07:45:57.202010+0000","last_active":"2024-09-11T07:47:40.250765+0000","last_peered":"2024-09-11T07:47:40.250765+0000","last_clean":"2024-09-11T07:47:40.250765+0000","last_became_active":"2024-09-11T07:45:57.201607+0000","last_became_peered":"2024-09-11T07:45:57.201607+0000","last_unstale":"2024-09-11T07:47:40.250765+0000","last_undegraded":"2024-09-11T07:47:40.250765+0000","last_fullsized":"2024-09-11T07:47:40.250765+0000","mapping_epoch":412,"log_start":"425'2400","ondisk_log_start":"425'2400","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":4983,"log_dups_size":0,"ondisk_log_size":4983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483852288,"num_objects":7383,"num_object_clones":0,"num_object_copies":22149,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7383,"num_write_kb":472512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"87'81","reported_seq":66,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.794775+0000","last_change":"2024-09-11T07:47:10.225175+0000","last_active":"2024-09-11T07:47:20.794775+0000","last_peered":"2024-09-11T07:47:20.794775+0000","last_clean":"2024-09-11T07:47:20.794775+0000","last_became_active":"2024-09-11T07:45:27.114518+0000","last_became_peered":"2024-09-11T07:45:27.114518+0000","last_unstale":"2024-09-11T07:47:20.794775+0000","last_undegraded":"2024-09-11T07:47:20.794775+0000","last_fullsized":"2024-09-11T07:47:20.794775+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:47:10.225048+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:47:10.225048+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:33.482173+0000","scrub_duration":0.005009131,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"426'42","reported_seq":598,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:31.370814+0000","last_change":"2024-09-11T07:47:31.370736+0000","last_active":"2024-09-11T07:47:31.370814+0000","last_peered":"2024-09-11T07:47:31.370814+0000","last_clean":"2024-09-11T07:47:31.370814+0000","last_became_active":"2024-09-11T07:43:05.417321+0000","last_became_peered":"2024-09-11T07:43:05.417321+0000","last_unstale":"2024-09-11T07:47:31.370814+0000","last_undegraded":"2024-09-11T07:47:31.370814+0000","last_fullsized":"2024-09-11T07:47:31.370814+0000","mapping_epoch":358,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":359,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'42","last_scrub_stamp":"2024-09-11T07:47:31.370619+0000","last_deep_scrub":"426'42","last_deep_scrub_stamp":"2024-09-11T07:47:31.370619+0000","last_clean_scrub_stamp":"2024-09-11T07:47:31.370619+0000","objects_scrubbed":6,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:33.046778+0000","scrub_duration":0.039486856000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":54,"num_read_kb":48,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"422'2","reported_seq":517,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.797695+0000","last_change":"2024-09-11T07:46:22.213763+0000","last_active":"2024-09-11T07:47:20.797695+0000","last_peered":"2024-09-11T07:47:20.797695+0000","last_clean":"2024-09-11T07:47:20.797695+0000","last_became_active":"2024-09-11T07:45:27.115139+0000","last_became_peered":"2024-09-11T07:45:27.115139+0000","last_unstale":"2024-09-11T07:47:20.797695+0000","last_undegraded":"2024-09-11T07:47:20.797695+0000","last_fullsized":"2024-09-11T07:47:20.797695+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"422'2","last_scrub_stamp":"2024-09-11T07:46:22.213676+0000","last_deep_scrub":"422'2","last_deep_scrub_stamp":"2024-09-11T07:46:22.213676+0000","last_clean_scrub_stamp":"2024-09-11T07:46:22.213676+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:31.909700+0000","scrub_duration":0.0077027509999999999,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":487,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.783674+0000","last_change":"2024-09-11T07:46:27.815229+0000","last_active":"2024-09-11T07:47:20.783674+0000","last_peered":"2024-09-11T07:47:20.783674+0000","last_clean":"2024-09-11T07:47:20.783674+0000","last_became_active":"2024-09-11T07:39:57.772244+0000","last_became_peered":"2024-09-11T07:39:57.772244+0000","last_unstale":"2024-09-11T07:47:20.783674+0000","last_undegraded":"2024-09-11T07:47:20.783674+0000","last_fullsized":"2024-09-11T07:47:20.783674+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:27.815152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:16.967492+0000","last_clean_scrub_stamp":"2024-09-11T07:46:27.815152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:41.545249+0000","scrub_duration":0.0047261389999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":94,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.783070+0000","last_change":"2024-09-11T07:45:27.102656+0000","last_active":"2024-09-11T07:47:20.783070+0000","last_peered":"2024-09-11T07:47:20.783070+0000","last_clean":"2024-09-11T07:47:20.783070+0000","last_became_active":"2024-09-11T07:45:27.101352+0000","last_became_peered":"2024-09-11T07:45:27.101352+0000","last_unstale":"2024-09-11T07:47:20.783070+0000","last_undegraded":"2024-09-11T07:47:20.783070+0000","last_fullsized":"2024-09-11T07:47:20.783070+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:22.519491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:51.187127+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.519491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.024174253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":33,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.788563+0000","last_change":"2024-09-11T07:46:31.493189+0000","last_active":"2024-09-11T07:47:20.788563+0000","last_peered":"2024-09-11T07:47:20.788563+0000","last_clean":"2024-09-11T07:47:20.788563+0000","last_became_active":"2024-09-11T07:46:31.492869+0000","last_became_peered":"2024-09-11T07:46:31.492869+0000","last_unstale":"2024-09-11T07:47:20.788563+0000","last_undegraded":"2024-09-11T07:47:20.788563+0000","last_fullsized":"2024-09-11T07:47:20.788563+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:21.014717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:46:21.014717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:43.295895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"441'7521","reported_seq":7558,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-11T07:47:40.249336+0000","last_change":"2024-09-11T07:45:57.199407+0000","last_active":"2024-09-11T07:47:40.249336+0000","last_peered":"2024-09-11T07:47:40.249336+0000","last_clean":"2024-09-11T07:47:40.249336+0000","last_became_active":"2024-09-11T07:45:57.199102+0000","last_became_peered":"2024-09-11T07:45:57.199102+0000","last_unstale":"2024-09-11T07:47:40.249336+0000","last_undegraded":"2024-09-11T07:47:40.249336+0000","last_fullsized":"2024-09-11T07:47:40.249336+0000","mapping_epoch":412,"log_start":"425'2600","ondisk_log_start":"425'2600","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":4921,"log_dups_size":0,"ondisk_log_size":4921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":492896256,"num_objects":7521,"num_object_clones":0,"num_object_copies":22563,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7521,"num_write_kb":481344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"426'45","reported_seq":182,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.784250+0000","last_change":"2024-09-11T07:45:32.442239+0000","last_active":"2024-09-11T07:47:20.784250+0000","last_peered":"2024-09-11T07:47:20.784250+0000","last_clean":"2024-09-11T07:47:20.784250+0000","last_became_active":"2024-09-11T07:45:27.113188+0000","last_became_peered":"2024-09-11T07:45:27.113188+0000","last_unstale":"2024-09-11T07:47:20.784250+0000","last_undegraded":"2024-09-11T07:47:20.784250+0000","last_fullsized":"2024-09-11T07:47:20.784250+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"399'33","last_scrub_stamp":"2024-09-11T07:45:32.442160+0000","last_deep_scrub":"399'33","last_deep_scrub_stamp":"2024-09-11T07:45:32.442160+0000","last_clean_scrub_stamp":"2024-09-11T07:45:32.442160+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015982277,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":59,"num_read_kb":49,"num_write":37,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"83'3","reported_seq":518,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:24.650855+0000","last_change":"2024-09-11T07:47:24.650786+0000","last_active":"2024-09-11T07:47:24.650855+0000","last_peered":"2024-09-11T07:47:24.650855+0000","last_clean":"2024-09-11T07:47:24.650855+0000","last_became_active":"2024-09-11T07:45:27.099666+0000","last_became_peered":"2024-09-11T07:45:27.099666+0000","last_unstale":"2024-09-11T07:47:24.650855+0000","last_undegraded":"2024-09-11T07:47:24.650855+0000","last_fullsized":"2024-09-11T07:47:24.650855+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-09-11T07:47:24.650681+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_clean_scrub_stamp":"2024-09-11T07:47:24.650681+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:45.847927+0000","scrub_duration":0.0033112549999999999,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":348,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.784097+0000","last_change":"2024-09-11T07:46:31.501022+0000","last_active":"2024-09-11T07:47:20.784097+0000","last_peered":"2024-09-11T07:47:20.784097+0000","last_clean":"2024-09-11T07:47:20.784097+0000","last_became_active":"2024-09-11T07:46:31.500613+0000","last_became_peered":"2024-09-11T07:46:31.500613+0000","last_unstale":"2024-09-11T07:47:20.784097+0000","last_undegraded":"2024-09-11T07:47:20.784097+0000","last_fullsized":"2024-09-11T07:47:20.784097+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:18.381749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:18.381749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013066169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"87'81","reported_seq":330,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.796049+0000","last_change":"2024-09-11T07:46:20.190045+0000","last_active":"2024-09-11T07:47:20.796049+0000","last_peered":"2024-09-11T07:47:20.796049+0000","last_clean":"2024-09-11T07:47:20.796049+0000","last_became_active":"2024-09-11T07:44:59.848971+0000","last_became_peered":"2024-09-11T07:44:59.848971+0000","last_unstale":"2024-09-11T07:47:20.796049+0000","last_undegraded":"2024-09-11T07:47:20.796049+0000","last_fullsized":"2024-09-11T07:47:20.796049+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:46:20.189949+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:46:20.189949+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0063253600000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":298,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.790616+0000","last_change":"2024-09-11T07:45:22.667614+0000","last_active":"2024-09-11T07:47:20.790616+0000","last_peered":"2024-09-11T07:47:20.790616+0000","last_clean":"2024-09-11T07:47:20.790616+0000","last_became_active":"2024-09-11T07:39:57.748658+0000","last_became_peered":"2024-09-11T07:39:57.748658+0000","last_unstale":"2024-09-11T07:47:20.790616+0000","last_undegraded":"2024-09-11T07:47:20.790616+0000","last_fullsized":"2024-09-11T07:47:20.790616+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:22.667519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:45:22.667519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046246309999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":453,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.790147+0000","last_change":"2024-09-11T07:46:31.509326+0000","last_active":"2024-09-11T07:47:20.790147+0000","last_peered":"2024-09-11T07:47:20.790147+0000","last_clean":"2024-09-11T07:47:20.790147+0000","last_became_active":"2024-09-11T07:46:31.509054+0000","last_became_peered":"2024-09-11T07:46:31.509054+0000","last_unstale":"2024-09-11T07:47:20.790147+0000","last_undegraded":"2024-09-11T07:47:20.790147+0000","last_fullsized":"2024-09-11T07:47:20.790147+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:14.389468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:45:14.389468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005315012,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.5","version":"441'7305","reported_seq":7341,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-11T07:47:39.472549+0000","last_change":"2024-09-11T07:45:57.199301+0000","last_active":"2024-09-11T07:47:39.472549+0000","last_peered":"2024-09-11T07:47:39.472549+0000","last_clean":"2024-09-11T07:47:39.472549+0000","last_became_active":"2024-09-11T07:45:57.199035+0000","last_became_peered":"2024-09-11T07:45:57.199035+0000","last_unstale":"2024-09-11T07:47:39.472549+0000","last_undegraded":"2024-09-11T07:47:39.472549+0000","last_fullsized":"2024-09-11T07:47:39.472549+0000","mapping_epoch":412,"log_start":"426'3600","ondisk_log_start":"426'3600","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3705,"log_dups_size":0,"ondisk_log_size":3705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":478740480,"num_objects":7305,"num_object_clones":0,"num_object_copies":21915,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7305,"num_write_kb":467520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"426'51","reported_seq":150,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.796002+0000","last_change":"2024-09-11T07:47:01.594395+0000","last_active":"2024-09-11T07:47:20.796002+0000","last_peered":"2024-09-11T07:47:20.796002+0000","last_clean":"2024-09-11T07:47:20.796002+0000","last_became_active":"2024-09-11T07:46:31.494953+0000","last_became_peered":"2024-09-11T07:46:31.494953+0000","last_unstale":"2024-09-11T07:47:20.796002+0000","last_undegraded":"2024-09-11T07:47:20.796002+0000","last_fullsized":"2024-09-11T07:47:20.796002+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'51","last_scrub_stamp":"2024-09-11T07:47:01.594299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:47:01.594299+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:05.515667+0000","scrub_duration":0.0032010720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":78,"num_read_kb":67,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":494,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.790682+0000","last_change":"2024-09-11T07:45:10.747839+0000","last_active":"2024-09-11T07:47:20.790682+0000","last_peered":"2024-09-11T07:47:20.790682+0000","last_clean":"2024-09-11T07:47:20.790682+0000","last_became_active":"2024-09-11T07:26:31.419560+0000","last_became_peered":"2024-09-11T07:26:31.419560+0000","last_unstale":"2024-09-11T07:47:20.790682+0000","last_undegraded":"2024-09-11T07:47:20.790682+0000","last_fullsized":"2024-09-11T07:47:20.790682+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:10.747787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:10.747787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014756929999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":501,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:26.126155+0000","last_change":"2024-09-11T07:47:26.126014+0000","last_active":"2024-09-11T07:47:26.126155+0000","last_peered":"2024-09-11T07:47:26.126155+0000","last_clean":"2024-09-11T07:47:26.126155+0000","last_became_active":"2024-09-11T07:39:57.748953+0000","last_became_peered":"2024-09-11T07:39:57.748953+0000","last_unstale":"2024-09-11T07:47:26.126155+0000","last_undegraded":"2024-09-11T07:47:26.126155+0000","last_fullsized":"2024-09-11T07:47:26.126155+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:26.125938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:47:26.125938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:45.197855+0000","scrub_duration":0.0088559729999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":306,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.781528+0000","last_change":"2024-09-11T07:47:18.117007+0000","last_active":"2024-09-11T07:47:20.781528+0000","last_peered":"2024-09-11T07:47:20.781528+0000","last_clean":"2024-09-11T07:47:20.781528+0000","last_became_active":"2024-09-11T07:39:57.766605+0000","last_became_peered":"2024-09-11T07:39:57.766605+0000","last_unstale":"2024-09-11T07:47:20.781528+0000","last_undegraded":"2024-09-11T07:47:20.781528+0000","last_fullsized":"2024-09-11T07:47:20.781528+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:18.116773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:47:18.116773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:27.291773+0000","scrub_duration":0.004032056,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.4","version":"441'7457","reported_seq":7494,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-11T07:47:39.556370+0000","last_change":"2024-09-11T07:45:57.201461+0000","last_active":"2024-09-11T07:47:39.556370+0000","last_peered":"2024-09-11T07:47:39.556370+0000","last_clean":"2024-09-11T07:47:39.556370+0000","last_became_active":"2024-09-11T07:45:57.201054+0000","last_became_peered":"2024-09-11T07:45:57.201054+0000","last_unstale":"2024-09-11T07:47:39.556370+0000","last_undegraded":"2024-09-11T07:47:39.556370+0000","last_fullsized":"2024-09-11T07:47:39.556370+0000","mapping_epoch":412,"log_start":"422'2000","ondisk_log_start":"422'2000","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":5457,"log_dups_size":0,"ondisk_log_size":5457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":488701952,"num_objects":7457,"num_object_clones":0,"num_object_copies":22371,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7457,"num_write_kb":477248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"426'21","reported_seq":524,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.794862+0000","last_change":"2024-09-11T07:45:27.145068+0000","last_active":"2024-09-11T07:47:20.794862+0000","last_peered":"2024-09-11T07:47:20.794862+0000","last_clean":"2024-09-11T07:47:20.794862+0000","last_became_active":"2024-09-11T07:45:27.114646+0000","last_became_peered":"2024-09-11T07:45:27.114646+0000","last_unstale":"2024-09-11T07:47:20.794862+0000","last_undegraded":"2024-09-11T07:47:20.794862+0000","last_fullsized":"2024-09-11T07:47:20.794862+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:45:11.521203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:45:11.521203+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015327073,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":73,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.789367+0000","last_change":"2024-09-11T07:46:16.505169+0000","last_active":"2024-09-11T07:47:20.789367+0000","last_peered":"2024-09-11T07:47:20.789367+0000","last_clean":"2024-09-11T07:47:20.789367+0000","last_became_active":"2024-09-11T07:45:27.102350+0000","last_became_peered":"2024-09-11T07:45:27.102350+0000","last_unstale":"2024-09-11T07:47:20.789367+0000","last_undegraded":"2024-09-11T07:47:20.789367+0000","last_fullsized":"2024-09-11T07:47:20.789367+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:16.505062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:46:16.505062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:40.975683+0000","scrub_duration":0.037745665999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":507,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.783283+0000","last_change":"2024-09-11T07:46:12.107858+0000","last_active":"2024-09-11T07:47:20.783283+0000","last_peered":"2024-09-11T07:47:20.783283+0000","last_clean":"2024-09-11T07:47:20.783283+0000","last_became_active":"2024-09-11T07:46:12.107538+0000","last_became_peered":"2024-09-11T07:46:12.107538+0000","last_unstale":"2024-09-11T07:47:20.783283+0000","last_undegraded":"2024-09-11T07:47:20.783283+0000","last_fullsized":"2024-09-11T07:47:20.783283+0000","mapping_epoch":418,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":419,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:08.436552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:44:08.436552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0064625339999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":483,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.796248+0000","last_change":"2024-09-11T07:45:22.489678+0000","last_active":"2024-09-11T07:47:20.796248+0000","last_peered":"2024-09-11T07:47:20.796248+0000","last_clean":"2024-09-11T07:47:20.796248+0000","last_became_active":"2024-09-11T07:32:00.505185+0000","last_became_peered":"2024-09-11T07:32:00.505185+0000","last_unstale":"2024-09-11T07:47:20.796248+0000","last_undegraded":"2024-09-11T07:47:20.796248+0000","last_fullsized":"2024-09-11T07:47:20.796248+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:22.489595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:22.489595+0000","last_clean_scrub_stamp":"2024-09-11T07:45:22.489595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.045984707999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"441'7258","reported_seq":7295,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-11T07:47:39.458971+0000","last_change":"2024-09-11T07:45:56.764280+0000","last_active":"2024-09-11T07:47:39.458971+0000","last_peered":"2024-09-11T07:47:39.458971+0000","last_clean":"2024-09-11T07:47:39.458971+0000","last_became_active":"2024-09-11T07:45:56.763089+0000","last_became_peered":"2024-09-11T07:45:56.763089+0000","last_unstale":"2024-09-11T07:47:39.458971+0000","last_undegraded":"2024-09-11T07:47:39.458971+0000","last_fullsized":"2024-09-11T07:47:39.458971+0000","mapping_epoch":412,"log_start":"426'3500","ondisk_log_start":"426'3500","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3758,"log_dups_size":0,"ondisk_log_size":3758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":475660288,"num_objects":7258,"num_object_clones":0,"num_object_copies":21774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7258,"num_write_kb":464512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"426'46","reported_seq":111,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.788996+0000","last_change":"2024-09-11T07:46:14.474268+0000","last_active":"2024-09-11T07:47:20.788996+0000","last_peered":"2024-09-11T07:47:20.788996+0000","last_clean":"2024-09-11T07:47:20.788996+0000","last_became_active":"2024-09-11T07:45:27.099653+0000","last_became_peered":"2024-09-11T07:45:27.099653+0000","last_unstale":"2024-09-11T07:47:20.788996+0000","last_undegraded":"2024-09-11T07:47:20.788996+0000","last_fullsized":"2024-09-11T07:47:20.788996+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"421'34","last_scrub_stamp":"2024-09-11T07:46:14.474174+0000","last_deep_scrub":"421'34","last_deep_scrub_stamp":"2024-09-11T07:46:14.474174+0000","last_clean_scrub_stamp":"2024-09-11T07:46:14.474174+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:41.275098+0000","scrub_duration":0.01054072,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":539,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:31.140237+0000","last_change":"2024-09-11T07:47:31.140172+0000","last_active":"2024-09-11T07:47:31.140237+0000","last_peered":"2024-09-11T07:47:31.140237+0000","last_clean":"2024-09-11T07:47:31.140237+0000","last_became_active":"2024-09-11T07:40:11.701330+0000","last_became_peered":"2024-09-11T07:40:11.701330+0000","last_unstale":"2024-09-11T07:47:31.140237+0000","last_undegraded":"2024-09-11T07:47:31.140237+0000","last_fullsized":"2024-09-11T07:47:31.140237+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:31.140083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_clean_scrub_stamp":"2024-09-11T07:47:31.140083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:43.623267+0000","scrub_duration":0.010667803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"395'11","reported_seq":705,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-11T07:47:31.505016+0000","last_change":"2024-09-11T07:46:07.749662+0000","last_active":"2024-09-11T07:47:31.505016+0000","last_peered":"2024-09-11T07:47:31.505016+0000","last_clean":"2024-09-11T07:47:31.505016+0000","last_became_active":"2024-09-11T07:45:27.093333+0000","last_became_peered":"2024-09-11T07:45:27.093333+0000","last_unstale":"2024-09-11T07:47:31.505016+0000","last_undegraded":"2024-09-11T07:47:31.505016+0000","last_fullsized":"2024-09-11T07:47:31.505016+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"395'11","last_scrub_stamp":"2024-09-11T07:46:07.749598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:46:07.749598+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017864059999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":127,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.794452+0000","last_change":"2024-09-11T07:45:26.475689+0000","last_active":"2024-09-11T07:47:20.794452+0000","last_peered":"2024-09-11T07:47:20.794452+0000","last_clean":"2024-09-11T07:47:20.794452+0000","last_became_active":"2024-09-11T07:42:04.729103+0000","last_became_peered":"2024-09-11T07:42:04.729103+0000","last_unstale":"2024-09-11T07:47:20.794452+0000","last_undegraded":"2024-09-11T07:47:20.794452+0000","last_fullsized":"2024-09-11T07:47:20.794452+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:26.475614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:45:26.475614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001709363,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"441'7183","reported_seq":7220,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-11T07:47:39.458278+0000","last_change":"2024-09-11T07:45:57.199112+0000","last_active":"2024-09-11T07:47:39.458278+0000","last_peered":"2024-09-11T07:47:39.458278+0000","last_clean":"2024-09-11T07:47:39.458278+0000","last_became_active":"2024-09-11T07:45:57.198860+0000","last_became_peered":"2024-09-11T07:45:57.198860+0000","last_unstale":"2024-09-11T07:47:39.458278+0000","last_undegraded":"2024-09-11T07:47:39.458278+0000","last_fullsized":"2024-09-11T07:47:39.458278+0000","mapping_epoch":412,"log_start":"426'3400","ondisk_log_start":"426'3400","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3783,"log_dups_size":0,"ondisk_log_size":3783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470745088,"num_objects":7183,"num_object_clones":0,"num_object_copies":21549,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7183,"num_write_kb":459712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"426'29","reported_seq":87,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.798020+0000","last_change":"2024-09-11T07:46:07.191286+0000","last_active":"2024-09-11T07:47:20.798020+0000","last_peered":"2024-09-11T07:47:20.798020+0000","last_clean":"2024-09-11T07:47:20.798020+0000","last_became_active":"2024-09-11T07:45:27.118858+0000","last_became_peered":"2024-09-11T07:45:27.118858+0000","last_unstale":"2024-09-11T07:47:20.798020+0000","last_undegraded":"2024-09-11T07:47:20.798020+0000","last_fullsized":"2024-09-11T07:47:20.798020+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'23","last_scrub_stamp":"2024-09-11T07:46:07.191227+0000","last_deep_scrub":"234'23","last_deep_scrub_stamp":"2024-09-11T07:37:34.613280+0000","last_clean_scrub_stamp":"2024-09-11T07:46:07.191227+0000","objects_scrubbed":5,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002042533,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":74,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.797397+0000","last_change":"2024-09-11T07:45:27.125454+0000","last_active":"2024-09-11T07:47:20.797397+0000","last_peered":"2024-09-11T07:47:20.797397+0000","last_clean":"2024-09-11T07:47:20.797397+0000","last_became_active":"2024-09-11T07:45:27.112394+0000","last_became_peered":"2024-09-11T07:45:27.112394+0000","last_unstale":"2024-09-11T07:47:20.797397+0000","last_undegraded":"2024-09-11T07:47:20.797397+0000","last_fullsized":"2024-09-11T07:47:20.797397+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:45:08.089525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:08.089525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":487,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.781664+0000","last_change":"2024-09-11T07:44:28.572130+0000","last_active":"2024-09-11T07:47:20.781664+0000","last_peered":"2024-09-11T07:47:20.781664+0000","last_clean":"2024-09-11T07:47:20.781664+0000","last_became_active":"2024-09-11T07:26:35.561967+0000","last_became_peered":"2024-09-11T07:26:35.561967+0000","last_unstale":"2024-09-11T07:47:20.781664+0000","last_undegraded":"2024-09-11T07:47:20.781664+0000","last_fullsized":"2024-09-11T07:47:20.781664+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:28.572059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:44.820354+0000","last_clean_scrub_stamp":"2024-09-11T07:44:28.572059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003042159,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":493,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.794524+0000","last_change":"2024-09-11T07:44:27.626846+0000","last_active":"2024-09-11T07:47:20.794524+0000","last_peered":"2024-09-11T07:47:20.794524+0000","last_clean":"2024-09-11T07:47:20.794524+0000","last_became_active":"2024-09-11T07:39:57.776576+0000","last_became_peered":"2024-09-11T07:39:57.776576+0000","last_unstale":"2024-09-11T07:47:20.794524+0000","last_undegraded":"2024-09-11T07:47:20.794524+0000","last_fullsized":"2024-09-11T07:47:20.794524+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:27.626761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:27.626761+0000","last_clean_scrub_stamp":"2024-09-11T07:44:27.626761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025844079999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"426'25","reported_seq":555,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:22.197328+0000","last_change":"2024-09-11T07:47:22.197270+0000","last_active":"2024-09-11T07:47:22.197328+0000","last_peered":"2024-09-11T07:47:22.197328+0000","last_clean":"2024-09-11T07:47:22.197328+0000","last_became_active":"2024-09-11T07:39:57.767661+0000","last_became_peered":"2024-09-11T07:39:57.767661+0000","last_unstale":"2024-09-11T07:47:22.197328+0000","last_undegraded":"2024-09-11T07:47:22.197328+0000","last_fullsized":"2024-09-11T07:47:22.197328+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'25","last_scrub_stamp":"2024-09-11T07:47:22.197145+0000","last_deep_scrub":"426'25","last_deep_scrub_stamp":"2024-09-11T07:47:22.197145+0000","last_clean_scrub_stamp":"2024-09-11T07:47:22.197145+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:42.553008+0000","scrub_duration":0.012355245000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":501,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.782018+0000","last_change":"2024-09-11T07:46:13.910021+0000","last_active":"2024-09-11T07:47:20.782018+0000","last_peered":"2024-09-11T07:47:20.782018+0000","last_clean":"2024-09-11T07:47:20.782018+0000","last_became_active":"2024-09-11T07:26:31.415701+0000","last_became_peered":"2024-09-11T07:26:31.415701+0000","last_unstale":"2024-09-11T07:47:20.782018+0000","last_undegraded":"2024-09-11T07:47:20.782018+0000","last_fullsized":"2024-09-11T07:47:20.782018+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:13.909932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:43:26.338227+0000","last_clean_scrub_stamp":"2024-09-11T07:46:13.909932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:42.316268+0000","scrub_duration":0.0075390129999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":222,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:24.810186+0000","last_change":"2024-09-11T07:47:24.810130+0000","last_active":"2024-09-11T07:47:24.810186+0000","last_peered":"2024-09-11T07:47:24.810186+0000","last_clean":"2024-09-11T07:47:24.810186+0000","last_became_active":"2024-09-11T07:44:23.768703+0000","last_became_peered":"2024-09-11T07:44:23.768703+0000","last_unstale":"2024-09-11T07:47:24.810186+0000","last_undegraded":"2024-09-11T07:47:24.810186+0000","last_fullsized":"2024-09-11T07:47:24.810186+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:24.810051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:35.392360+0000","last_clean_scrub_stamp":"2024-09-11T07:47:24.810051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:43.535963+0000","scrub_duration":0.018560989999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":508,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.789936+0000","last_change":"2024-09-11T07:45:27.129839+0000","last_active":"2024-09-11T07:47:20.789936+0000","last_peered":"2024-09-11T07:47:20.789936+0000","last_clean":"2024-09-11T07:47:20.789936+0000","last_became_active":"2024-09-11T07:45:27.124181+0000","last_became_peered":"2024-09-11T07:45:27.124181+0000","last_unstale":"2024-09-11T07:47:20.789936+0000","last_undegraded":"2024-09-11T07:47:20.789936+0000","last_fullsized":"2024-09-11T07:47:20.789936+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:45:18.615900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:45:18.615900+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005478391,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":303,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.795981+0000","last_change":"2024-09-11T07:47:12.474748+0000","last_active":"2024-09-11T07:47:20.795981+0000","last_peered":"2024-09-11T07:47:20.795981+0000","last_clean":"2024-09-11T07:47:20.795981+0000","last_became_active":"2024-09-11T07:45:27.091114+0000","last_became_peered":"2024-09-11T07:45:27.091114+0000","last_unstale":"2024-09-11T07:47:20.795981+0000","last_undegraded":"2024-09-11T07:47:20.795981+0000","last_fullsized":"2024-09-11T07:47:20.795981+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:12.474673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:47:12.474673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:25.229073+0000","scrub_duration":0.0027412030000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"426'21","reported_seq":135,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.783369+0000","last_change":"2024-09-11T07:45:19.660354+0000","last_active":"2024-09-11T07:47:20.783369+0000","last_peered":"2024-09-11T07:47:20.783369+0000","last_clean":"2024-09-11T07:47:20.783369+0000","last_became_active":"2024-09-11T07:44:23.769896+0000","last_became_peered":"2024-09-11T07:44:23.769896+0000","last_unstale":"2024-09-11T07:47:20.783369+0000","last_undegraded":"2024-09-11T07:47:20.783369+0000","last_fullsized":"2024-09-11T07:47:20.783369+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:45:19.660273+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:45:19.660273+0000","objects_scrubbed":3,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032335910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":23,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":486,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.789698+0000","last_change":"2024-09-11T07:46:31.493250+0000","last_active":"2024-09-11T07:47:20.789698+0000","last_peered":"2024-09-11T07:47:20.789698+0000","last_clean":"2024-09-11T07:47:20.789698+0000","last_became_active":"2024-09-11T07:46:31.492884+0000","last_became_peered":"2024-09-11T07:46:31.492884+0000","last_unstale":"2024-09-11T07:47:20.789698+0000","last_undegraded":"2024-09-11T07:47:20.789698+0000","last_fullsized":"2024-09-11T07:47:20.789698+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:24.015330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:46:24.015330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:35.180268+0000","scrub_duration":0.0069828010000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":484,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.784182+0000","last_change":"2024-09-11T07:47:04.742683+0000","last_active":"2024-09-11T07:47:20.784182+0000","last_peered":"2024-09-11T07:47:20.784182+0000","last_clean":"2024-09-11T07:47:20.784182+0000","last_became_active":"2024-09-11T07:45:27.112711+0000","last_became_peered":"2024-09-11T07:45:27.112711+0000","last_unstale":"2024-09-11T07:47:20.784182+0000","last_undegraded":"2024-09-11T07:47:20.784182+0000","last_fullsized":"2024-09-11T07:47:20.784182+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:04.742597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:47:04.742597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:19.493791+0000","scrub_duration":0.0056709050000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"426'35","reported_seq":569,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:26.690126+0000","last_change":"2024-09-11T07:47:26.690054+0000","last_active":"2024-09-11T07:47:26.690126+0000","last_peered":"2024-09-11T07:47:26.690126+0000","last_clean":"2024-09-11T07:47:26.690126+0000","last_became_active":"2024-09-11T07:39:57.771009+0000","last_became_peered":"2024-09-11T07:39:57.771009+0000","last_unstale":"2024-09-11T07:47:26.690126+0000","last_undegraded":"2024-09-11T07:47:26.690126+0000","last_fullsized":"2024-09-11T07:47:26.690126+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'35","last_scrub_stamp":"2024-09-11T07:47:26.689929+0000","last_deep_scrub":"234'25","last_deep_scrub_stamp":"2024-09-11T07:44:29.610392+0000","last_clean_scrub_stamp":"2024-09-11T07:47:26.689929+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:28.374571+0000","scrub_duration":0.0049798070000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":340,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.788839+0000","last_change":"2024-09-11T07:46:31.497548+0000","last_active":"2024-09-11T07:47:20.788839+0000","last_peered":"2024-09-11T07:47:20.788839+0000","last_clean":"2024-09-11T07:47:20.788839+0000","last_became_active":"2024-09-11T07:46:31.497105+0000","last_became_peered":"2024-09-11T07:46:31.497105+0000","last_unstale":"2024-09-11T07:47:20.788839+0000","last_undegraded":"2024-09-11T07:47:20.788839+0000","last_fullsized":"2024-09-11T07:47:20.788839+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:19.063698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:39:54.493614+0000","last_clean_scrub_stamp":"2024-09-11T07:46:19.063698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:44.292280+0000","scrub_duration":0.0049403759999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"425'9","reported_seq":72,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-11T07:47:31.515877+0000","last_change":"2024-09-11T07:47:21.208866+0000","last_active":"2024-09-11T07:47:31.515877+0000","last_peered":"2024-09-11T07:47:31.515877+0000","last_clean":"2024-09-11T07:47:31.515877+0000","last_became_active":"2024-09-11T07:46:31.499246+0000","last_became_peered":"2024-09-11T07:46:31.499246+0000","last_unstale":"2024-09-11T07:47:31.515877+0000","last_undegraded":"2024-09-11T07:47:31.515877+0000","last_fullsized":"2024-09-11T07:47:31.515877+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"425'9","last_scrub_stamp":"2024-09-11T07:47:21.208768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:47:21.208768+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:32.846447+0000","scrub_duration":0.0052892979999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":513,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.796387+0000","last_change":"2024-09-11T07:47:14.261937+0000","last_active":"2024-09-11T07:47:20.796387+0000","last_peered":"2024-09-11T07:47:20.796387+0000","last_clean":"2024-09-11T07:47:20.796387+0000","last_became_active":"2024-09-11T07:45:27.123325+0000","last_became_peered":"2024-09-11T07:45:27.123325+0000","last_unstale":"2024-09-11T07:47:20.796387+0000","last_undegraded":"2024-09-11T07:47:20.796387+0000","last_fullsized":"2024-09-11T07:47:20.796387+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:14.259694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:47:14.259694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:21.828842+0000","scrub_duration":0.010207048999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"426'25","reported_seq":505,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.784107+0000","last_change":"2024-09-11T07:47:00.685544+0000","last_active":"2024-09-11T07:47:20.784107+0000","last_peered":"2024-09-11T07:47:20.784107+0000","last_clean":"2024-09-11T07:47:20.784107+0000","last_became_active":"2024-09-11T07:43:10.448494+0000","last_became_peered":"2024-09-11T07:43:10.448494+0000","last_unstale":"2024-09-11T07:47:20.784107+0000","last_undegraded":"2024-09-11T07:47:20.784107+0000","last_fullsized":"2024-09-11T07:47:20.784107+0000","mapping_epoch":360,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":361,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'25","last_scrub_stamp":"2024-09-11T07:47:00.685485+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:37:41.338036+0000","last_clean_scrub_stamp":"2024-09-11T07:47:00.685485+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:17.529636+0000","scrub_duration":0.0065472430000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":485,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.796051+0000","last_change":"2024-09-11T07:45:27.117986+0000","last_active":"2024-09-11T07:47:20.796051+0000","last_peered":"2024-09-11T07:47:20.796051+0000","last_clean":"2024-09-11T07:47:20.796051+0000","last_became_active":"2024-09-11T07:45:27.114097+0000","last_became_peered":"2024-09-11T07:45:27.114097+0000","last_unstale":"2024-09-11T07:47:20.796051+0000","last_undegraded":"2024-09-11T07:47:20.796051+0000","last_fullsized":"2024-09-11T07:47:20.796051+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:09.537798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:09.537798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003422685,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"290'11","reported_seq":481,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-11T07:47:31.505616+0000","last_change":"2024-09-11T07:47:12.818421+0000","last_active":"2024-09-11T07:47:31.505616+0000","last_peered":"2024-09-11T07:47:31.505616+0000","last_clean":"2024-09-11T07:47:31.505616+0000","last_became_active":"2024-09-11T07:39:57.772790+0000","last_became_peered":"2024-09-11T07:39:57.772790+0000","last_unstale":"2024-09-11T07:47:31.505616+0000","last_undegraded":"2024-09-11T07:47:31.505616+0000","last_fullsized":"2024-09-11T07:47:31.505616+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"290'11","last_scrub_stamp":"2024-09-11T07:47:12.818344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:47:12.818344+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:41.751212+0000","scrub_duration":0.022994837000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":59,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.783496+0000","last_change":"2024-09-11T07:45:27.104143+0000","last_active":"2024-09-11T07:47:20.783496+0000","last_peered":"2024-09-11T07:47:20.783496+0000","last_clean":"2024-09-11T07:47:20.783496+0000","last_became_active":"2024-09-11T07:45:27.101913+0000","last_became_peered":"2024-09-11T07:45:27.101913+0000","last_unstale":"2024-09-11T07:47:20.783496+0000","last_undegraded":"2024-09-11T07:47:20.783496+0000","last_fullsized":"2024-09-11T07:47:20.783496+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:21.026931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:45:21.026931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"426'22","reported_seq":483,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.783442+0000","last_change":"2024-09-11T07:45:14.682735+0000","last_active":"2024-09-11T07:47:20.783442+0000","last_peered":"2024-09-11T07:47:20.783442+0000","last_clean":"2024-09-11T07:47:20.783442+0000","last_became_active":"2024-09-11T07:39:57.689424+0000","last_became_peered":"2024-09-11T07:39:57.689424+0000","last_unstale":"2024-09-11T07:47:20.783442+0000","last_undegraded":"2024-09-11T07:47:20.783442+0000","last_fullsized":"2024-09-11T07:47:20.783442+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'16","last_scrub_stamp":"2024-09-11T07:45:14.682638+0000","last_deep_scrub":"236'16","last_deep_scrub_stamp":"2024-09-11T07:36:49.809484+0000","last_clean_scrub_stamp":"2024-09-11T07:45:14.682638+0000","objects_scrubbed":4,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014584720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":278,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.789046+0000","last_change":"2024-09-11T07:45:08.577358+0000","last_active":"2024-09-11T07:47:20.789046+0000","last_peered":"2024-09-11T07:47:20.789046+0000","last_clean":"2024-09-11T07:47:20.789046+0000","last_became_active":"2024-09-11T07:39:57.776841+0000","last_became_peered":"2024-09-11T07:39:57.776841+0000","last_unstale":"2024-09-11T07:47:20.789046+0000","last_undegraded":"2024-09-11T07:47:20.789046+0000","last_fullsized":"2024-09-11T07:47:20.789046+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:08.577262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:38.721118+0000","last_clean_scrub_stamp":"2024-09-11T07:45:08.577262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0073986299999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":474,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.794638+0000","last_change":"2024-09-11T07:47:05.721515+0000","last_active":"2024-09-11T07:47:20.794638+0000","last_peered":"2024-09-11T07:47:20.794638+0000","last_clean":"2024-09-11T07:47:20.794638+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:47:20.794638+0000","last_undegraded":"2024-09-11T07:47:20.794638+0000","last_fullsized":"2024-09-11T07:47:20.794638+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:05.721116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:47:05.721116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:08.290839+0000","scrub_duration":0.0069049100000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":486,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.795477+0000","last_change":"2024-09-11T07:46:19.170608+0000","last_active":"2024-09-11T07:47:20.795477+0000","last_peered":"2024-09-11T07:47:20.795477+0000","last_clean":"2024-09-11T07:47:20.795477+0000","last_became_active":"2024-09-11T07:39:57.761754+0000","last_became_peered":"2024-09-11T07:39:57.761754+0000","last_unstale":"2024-09-11T07:47:20.795477+0000","last_undegraded":"2024-09-11T07:47:20.795477+0000","last_fullsized":"2024-09-11T07:47:20.795477+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:19.170527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:46:19.170527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0099471420000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"426'32","reported_seq":318,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.796998+0000","last_change":"2024-09-11T07:47:00.575038+0000","last_active":"2024-09-11T07:47:20.796998+0000","last_peered":"2024-09-11T07:47:20.796998+0000","last_clean":"2024-09-11T07:47:20.796998+0000","last_became_active":"2024-09-11T07:46:31.494995+0000","last_became_peered":"2024-09-11T07:46:31.494995+0000","last_unstale":"2024-09-11T07:47:20.796998+0000","last_undegraded":"2024-09-11T07:47:20.796998+0000","last_fullsized":"2024-09-11T07:47:20.796998+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'32","last_scrub_stamp":"2024-09-11T07:47:00.574980+0000","last_deep_scrub":"234'24","last_deep_scrub_stamp":"2024-09-11T07:41:32.694510+0000","last_clean_scrub_stamp":"2024-09-11T07:47:00.574980+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:20.310086+0000","scrub_duration":0.001940443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":36,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":33,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.796381+0000","last_change":"2024-09-11T07:46:31.500716+0000","last_active":"2024-09-11T07:47:20.796381+0000","last_peered":"2024-09-11T07:47:20.796381+0000","last_clean":"2024-09-11T07:47:20.796381+0000","last_became_active":"2024-09-11T07:46:31.500431+0000","last_became_peered":"2024-09-11T07:46:31.500431+0000","last_unstale":"2024-09-11T07:47:20.796381+0000","last_undegraded":"2024-09-11T07:47:20.796381+0000","last_fullsized":"2024-09-11T07:47:20.796381+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:09.333312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:45:09.333312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":434,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.775523+0000","last_change":"2024-09-11T07:47:00.186555+0000","last_active":"2024-09-11T07:47:20.775523+0000","last_peered":"2024-09-11T07:47:20.775523+0000","last_clean":"2024-09-11T07:47:20.775523+0000","last_became_active":"2024-09-11T07:46:31.495137+0000","last_became_peered":"2024-09-11T07:46:31.495137+0000","last_unstale":"2024-09-11T07:47:20.775523+0000","last_undegraded":"2024-09-11T07:47:20.775523+0000","last_fullsized":"2024-09-11T07:47:20.775523+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:00.186475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:39:55.895853+0000","last_clean_scrub_stamp":"2024-09-11T07:47:00.186475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:11.814942+0000","scrub_duration":0.0053751830000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":205,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.796455+0000","last_change":"2024-09-11T07:45:24.519416+0000","last_active":"2024-09-11T07:47:20.796455+0000","last_peered":"2024-09-11T07:47:20.796455+0000","last_clean":"2024-09-11T07:47:20.796455+0000","last_became_active":"2024-09-11T07:40:11.700676+0000","last_became_peered":"2024-09-11T07:40:11.700676+0000","last_unstale":"2024-09-11T07:47:20.796455+0000","last_undegraded":"2024-09-11T07:47:20.796455+0000","last_fullsized":"2024-09-11T07:47:20.796455+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:24.519137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:45:24.519137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077895680000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"426'14","reported_seq":500,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.789313+0000","last_change":"2024-09-11T07:47:20.651223+0000","last_active":"2024-09-11T07:47:20.789313+0000","last_peered":"2024-09-11T07:47:20.789313+0000","last_clean":"2024-09-11T07:47:20.789313+0000","last_became_active":"2024-09-11T07:26:33.545830+0000","last_became_peered":"2024-09-11T07:26:33.545830+0000","last_unstale":"2024-09-11T07:47:20.789313+0000","last_undegraded":"2024-09-11T07:47:20.789313+0000","last_fullsized":"2024-09-11T07:47:20.789313+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'14","last_scrub_stamp":"2024-09-11T07:47:20.650947+0000","last_deep_scrub":"234'10","last_deep_scrub_stamp":"2024-09-11T07:41:29.585284+0000","last_clean_scrub_stamp":"2024-09-11T07:47:20.650947+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:23.593209+0000","scrub_duration":0.022730186999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":16,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":29,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.788325+0000","last_change":"2024-09-11T07:46:31.503627+0000","last_active":"2024-09-11T07:47:20.788325+0000","last_peered":"2024-09-11T07:47:20.788325+0000","last_clean":"2024-09-11T07:47:20.788325+0000","last_became_active":"2024-09-11T07:46:31.503399+0000","last_became_peered":"2024-09-11T07:46:31.503399+0000","last_unstale":"2024-09-11T07:47:20.788325+0000","last_undegraded":"2024-09-11T07:47:20.788325+0000","last_fullsized":"2024-09-11T07:47:20.788325+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:17.132174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_clean_scrub_stamp":"2024-09-11T07:46:17.132174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":299,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.796698+0000","last_change":"2024-09-11T07:45:31.477617+0000","last_active":"2024-09-11T07:47:20.796698+0000","last_peered":"2024-09-11T07:47:20.796698+0000","last_clean":"2024-09-11T07:47:20.796698+0000","last_became_active":"2024-09-11T07:45:27.118170+0000","last_became_peered":"2024-09-11T07:45:27.118170+0000","last_unstale":"2024-09-11T07:47:20.796698+0000","last_undegraded":"2024-09-11T07:47:20.796698+0000","last_fullsized":"2024-09-11T07:47:20.796698+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:31.477552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:31.477552+0000","last_clean_scrub_stamp":"2024-09-11T07:45:31.477552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001479164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":486,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.783913+0000","last_change":"2024-09-11T07:47:17.866202+0000","last_active":"2024-09-11T07:47:20.783913+0000","last_peered":"2024-09-11T07:47:20.783913+0000","last_clean":"2024-09-11T07:47:20.783913+0000","last_became_active":"2024-09-11T07:45:27.097259+0000","last_became_peered":"2024-09-11T07:45:27.097259+0000","last_unstale":"2024-09-11T07:47:20.783913+0000","last_undegraded":"2024-09-11T07:47:20.783913+0000","last_fullsized":"2024-09-11T07:47:20.783913+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:17.866113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:47:17.866113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:36.838418+0000","scrub_duration":0.0064356079999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"426'27","reported_seq":542,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.790503+0000","last_change":"2024-09-11T07:47:17.662976+0000","last_active":"2024-09-11T07:47:20.790503+0000","last_peered":"2024-09-11T07:47:20.790503+0000","last_clean":"2024-09-11T07:47:20.790503+0000","last_became_active":"2024-09-11T07:39:57.767928+0000","last_became_peered":"2024-09-11T07:39:57.767928+0000","last_unstale":"2024-09-11T07:47:20.790503+0000","last_undegraded":"2024-09-11T07:47:20.790503+0000","last_fullsized":"2024-09-11T07:47:20.790503+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'27","last_scrub_stamp":"2024-09-11T07:47:17.662725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:47:17.662725+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:32.813407+0000","scrub_duration":0.011257233,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":33,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.783623+0000","last_change":"2024-09-11T07:46:31.504241+0000","last_active":"2024-09-11T07:47:20.783623+0000","last_peered":"2024-09-11T07:47:20.783623+0000","last_clean":"2024-09-11T07:47:20.783623+0000","last_became_active":"2024-09-11T07:46:31.503969+0000","last_became_peered":"2024-09-11T07:46:31.503969+0000","last_unstale":"2024-09-11T07:47:20.783623+0000","last_undegraded":"2024-09-11T07:47:20.783623+0000","last_fullsized":"2024-09-11T07:47:20.783623+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:11.315570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:48.366931+0000","last_clean_scrub_stamp":"2024-09-11T07:44:11.315570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":459,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:25.782213+0000","last_change":"2024-09-11T07:47:25.782162+0000","last_active":"2024-09-11T07:47:25.782213+0000","last_peered":"2024-09-11T07:47:25.782213+0000","last_clean":"2024-09-11T07:47:25.782213+0000","last_became_active":"2024-09-11T07:45:27.101251+0000","last_became_peered":"2024-09-11T07:45:27.101251+0000","last_unstale":"2024-09-11T07:47:25.782213+0000","last_undegraded":"2024-09-11T07:47:25.782213+0000","last_fullsized":"2024-09-11T07:47:25.782213+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:25.782104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:39.021876+0000","last_clean_scrub_stamp":"2024-09-11T07:47:25.782104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:43.730866+0000","scrub_duration":0.0017487259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":469,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:23.723314+0000","last_change":"2024-09-11T07:47:23.723242+0000","last_active":"2024-09-11T07:47:23.723314+0000","last_peered":"2024-09-11T07:47:23.723314+0000","last_clean":"2024-09-11T07:47:23.723314+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:47:23.723314+0000","last_undegraded":"2024-09-11T07:47:23.723314+0000","last_fullsized":"2024-09-11T07:47:23.723314+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:23.723143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:32.904006+0000","last_clean_scrub_stamp":"2024-09-11T07:47:23.723143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:25.416093+0000","scrub_duration":0.032919889000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":446,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.784050+0000","last_change":"2024-09-11T07:46:08.795262+0000","last_active":"2024-09-11T07:47:20.784050+0000","last_peered":"2024-09-11T07:47:20.784050+0000","last_clean":"2024-09-11T07:47:20.784050+0000","last_became_active":"2024-09-11T07:32:00.511258+0000","last_became_peered":"2024-09-11T07:32:00.511258+0000","last_unstale":"2024-09-11T07:47:20.784050+0000","last_undegraded":"2024-09-11T07:47:20.784050+0000","last_fullsized":"2024-09-11T07:47:20.784050+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:46:08.795209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:46:08.795209+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014785359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":492,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.781600+0000","last_change":"2024-09-11T07:46:31.503438+0000","last_active":"2024-09-11T07:47:20.781600+0000","last_peered":"2024-09-11T07:47:20.781600+0000","last_clean":"2024-09-11T07:47:20.781600+0000","last_became_active":"2024-09-11T07:46:31.503050+0000","last_became_peered":"2024-09-11T07:46:31.503050+0000","last_unstale":"2024-09-11T07:47:20.781600+0000","last_undegraded":"2024-09-11T07:47:20.781600+0000","last_fullsized":"2024-09-11T07:47:20.781600+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:44:22.300957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.300957+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0068790429999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"426'30","reported_seq":98,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.797922+0000","last_change":"2024-09-11T07:45:30.511577+0000","last_active":"2024-09-11T07:47:20.797922+0000","last_peered":"2024-09-11T07:47:20.797922+0000","last_clean":"2024-09-11T07:47:20.797922+0000","last_became_active":"2024-09-11T07:45:27.122864+0000","last_became_peered":"2024-09-11T07:45:27.122864+0000","last_unstale":"2024-09-11T07:47:20.797922+0000","last_undegraded":"2024-09-11T07:47:20.797922+0000","last_fullsized":"2024-09-11T07:47:20.797922+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'20","last_scrub_stamp":"2024-09-11T07:45:30.511477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:45:30.511477+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001786167,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"395'13","reported_seq":258,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-11T07:47:31.513149+0000","last_change":"2024-09-11T07:45:31.645251+0000","last_active":"2024-09-11T07:47:31.513149+0000","last_peered":"2024-09-11T07:47:31.513149+0000","last_clean":"2024-09-11T07:47:31.513149+0000","last_became_active":"2024-09-11T07:45:27.105351+0000","last_became_peered":"2024-09-11T07:45:27.105351+0000","last_unstale":"2024-09-11T07:47:31.513149+0000","last_undegraded":"2024-09-11T07:47:31.513149+0000","last_fullsized":"2024-09-11T07:47:31.513149+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"395'13","last_scrub_stamp":"2024-09-11T07:45:31.645192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:45:31.645192+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016838580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3],"acting":[2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":68,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.785337+0000","last_change":"2024-09-11T07:46:31.502229+0000","last_active":"2024-09-11T07:47:20.785337+0000","last_peered":"2024-09-11T07:47:20.785337+0000","last_clean":"2024-09-11T07:47:20.785337+0000","last_became_active":"2024-09-11T07:46:31.501945+0000","last_became_peered":"2024-09-11T07:46:31.501945+0000","last_unstale":"2024-09-11T07:47:20.785337+0000","last_undegraded":"2024-09-11T07:47:20.785337+0000","last_fullsized":"2024-09-11T07:47:20.785337+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:28.236557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:53.875300+0000","last_clean_scrub_stamp":"2024-09-11T07:44:28.236557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001550165,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":485,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.790654+0000","last_change":"2024-09-11T07:46:59.680297+0000","last_active":"2024-09-11T07:47:20.790654+0000","last_peered":"2024-09-11T07:47:20.790654+0000","last_clean":"2024-09-11T07:47:20.790654+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:47:20.790654+0000","last_undegraded":"2024-09-11T07:47:20.790654+0000","last_fullsized":"2024-09-11T07:47:20.790654+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:59.680093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:46:59.680093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:14.357775+0000","scrub_duration":0.014445909,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"426'27","reported_seq":54,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.790069+0000","last_change":"2024-09-11T07:46:13.812491+0000","last_active":"2024-09-11T07:47:20.790069+0000","last_peered":"2024-09-11T07:47:20.790069+0000","last_clean":"2024-09-11T07:47:20.790069+0000","last_became_active":"2024-09-11T07:46:13.790166+0000","last_became_peered":"2024-09-11T07:46:13.790166+0000","last_unstale":"2024-09-11T07:47:20.790069+0000","last_undegraded":"2024-09-11T07:47:20.790069+0000","last_fullsized":"2024-09-11T07:47:20.790069+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:45:23.537025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:45:23.537025+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033255630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":485,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.796577+0000","last_change":"2024-09-11T07:46:15.189821+0000","last_active":"2024-09-11T07:47:20.796577+0000","last_peered":"2024-09-11T07:47:20.796577+0000","last_clean":"2024-09-11T07:47:20.796577+0000","last_became_active":"2024-09-11T07:32:00.507050+0000","last_became_peered":"2024-09-11T07:32:00.507050+0000","last_unstale":"2024-09-11T07:47:20.796577+0000","last_undegraded":"2024-09-11T07:47:20.796577+0000","last_fullsized":"2024-09-11T07:47:20.796577+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:15.189711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:24.575210+0000","last_clean_scrub_stamp":"2024-09-11T07:46:15.189711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066152470000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":66,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.789463+0000","last_change":"2024-09-11T07:46:05.410699+0000","last_active":"2024-09-11T07:47:20.789463+0000","last_peered":"2024-09-11T07:47:20.789463+0000","last_clean":"2024-09-11T07:47:20.789463+0000","last_became_active":"2024-09-11T07:45:27.100903+0000","last_became_peered":"2024-09-11T07:45:27.100903+0000","last_unstale":"2024-09-11T07:47:20.789463+0000","last_undegraded":"2024-09-11T07:47:20.789463+0000","last_fullsized":"2024-09-11T07:47:20.789463+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:05.410607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:46:05.410607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016366040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":211,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.783973+0000","last_change":"2024-09-11T07:46:03.709201+0000","last_active":"2024-09-11T07:47:20.783973+0000","last_peered":"2024-09-11T07:47:20.783973+0000","last_clean":"2024-09-11T07:47:20.783973+0000","last_became_active":"2024-09-11T07:45:27.096591+0000","last_became_peered":"2024-09-11T07:45:27.096591+0000","last_unstale":"2024-09-11T07:47:20.783973+0000","last_undegraded":"2024-09-11T07:47:20.783973+0000","last_fullsized":"2024-09-11T07:47:20.783973+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:03.709095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:46:03.709095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001464166,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1c","version":"426'39","reported_seq":547,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.790437+0000","last_change":"2024-09-11T07:45:13.715138+0000","last_active":"2024-09-11T07:47:20.790437+0000","last_peered":"2024-09-11T07:47:20.790437+0000","last_clean":"2024-09-11T07:47:20.790437+0000","last_became_active":"2024-09-11T07:26:33.542544+0000","last_became_peered":"2024-09-11T07:26:33.542544+0000","last_unstale":"2024-09-11T07:47:20.790437+0000","last_undegraded":"2024-09-11T07:47:20.790437+0000","last_fullsized":"2024-09-11T07:47:20.790437+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'27","last_scrub_stamp":"2024-09-11T07:45:13.715050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:45:13.715050+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047085019999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":73,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.795880+0000","last_change":"2024-09-11T07:45:34.587277+0000","last_active":"2024-09-11T07:47:20.795880+0000","last_peered":"2024-09-11T07:47:20.795880+0000","last_clean":"2024-09-11T07:47:20.795880+0000","last_became_active":"2024-09-11T07:45:27.119977+0000","last_became_peered":"2024-09-11T07:45:27.119977+0000","last_unstale":"2024-09-11T07:47:20.795880+0000","last_undegraded":"2024-09-11T07:47:20.795880+0000","last_fullsized":"2024-09-11T07:47:20.795880+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:34.587193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:32.456852+0000","last_clean_scrub_stamp":"2024-09-11T07:45:34.587193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049811509999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":476,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.790279+0000","last_change":"2024-09-11T07:47:02.756784+0000","last_active":"2024-09-11T07:47:20.790279+0000","last_peered":"2024-09-11T07:47:20.790279+0000","last_clean":"2024-09-11T07:47:20.790279+0000","last_became_active":"2024-09-11T07:39:57.768994+0000","last_became_peered":"2024-09-11T07:39:57.768994+0000","last_unstale":"2024-09-11T07:47:20.790279+0000","last_undegraded":"2024-09-11T07:47:20.790279+0000","last_fullsized":"2024-09-11T07:47:20.790279+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:02.756683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:47:02.756683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:09.869084+0000","scrub_duration":0.0090198880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":509,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.794791+0000","last_change":"2024-09-11T07:46:22.481200+0000","last_active":"2024-09-11T07:47:20.794791+0000","last_peered":"2024-09-11T07:47:20.794791+0000","last_clean":"2024-09-11T07:47:20.794791+0000","last_became_active":"2024-09-11T07:45:27.101964+0000","last_became_peered":"2024-09-11T07:45:27.101964+0000","last_unstale":"2024-09-11T07:47:20.794791+0000","last_undegraded":"2024-09-11T07:47:20.794791+0000","last_fullsized":"2024-09-11T07:47:20.794791+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:22.481110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:46:22.481110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:47:44.619663+0000","scrub_duration":0.005622022,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"426'29","reported_seq":538,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:20.789298+0000","last_change":"2024-09-11T07:45:27.139564+0000","last_active":"2024-09-11T07:47:20.789298+0000","last_peered":"2024-09-11T07:47:20.789298+0000","last_clean":"2024-09-11T07:47:20.789298+0000","last_became_active":"2024-09-11T07:45:27.095974+0000","last_became_peered":"2024-09-11T07:45:27.095974+0000","last_unstale":"2024-09-11T07:47:20.789298+0000","last_undegraded":"2024-09-11T07:47:20.789298+0000","last_fullsized":"2024-09-11T07:47:20.789298+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'23","last_scrub_stamp":"2024-09-11T07:45:24.422401+0000","last_deep_scrub":"234'23","last_deep_scrub_stamp":"2024-09-11T07:43:25.530826+0000","last_clean_scrub_stamp":"2024-09-11T07:45:24.422401+0000","objects_scrubbed":5,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0068140780000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":69,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-11T07:47:25.649293+0000","last_change":"2024-09-11T07:47:25.649232+0000","last_active":"2024-09-11T07:47:25.649293+0000","last_peered":"2024-09-11T07:47:25.649293+0000","last_clean":"2024-09-11T07:47:25.649293+0000","last_became_active":"2024-09-11T07:45:27.104493+0000","last_became_peered":"2024-09-11T07:45:27.104493+0000","last_unstale":"2024-09-11T07:47:25.649293+0000","last_undegraded":"2024-09-11T07:47:25.649293+0000","last_fullsized":"2024-09-11T07:47:25.649293+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:25.649135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:47:25.649135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:54.433620+0000","scrub_duration":0.0087859719999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":7687438336,"num_objects":117301,"num_object_clones":0,"num_object_copies":351903,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117301,"num_write_kb":7507264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":23017881600,"data_stored":23017881600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70601,"ondisk_log_size":70601,"up":48,"acting":48,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84,"ondisk_log_size":84,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1535,"num_read_kb":1292,"num_write":989,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":3510,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":475136,"data_stored":12690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1069,"ondisk_log_size":1069,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":202,"num_read_kb":197,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":644038,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2109440,"data_stored":2098784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":891,"ondisk_log_size":891,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":147,"seq":631360192701,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4163708,"kb_used_data":3252660,"kb_used_omap":4,"kb_used_meta":911035,"kb_avail":89581444,"statfs":{"total":95995035648,"available":91731398656,"internally_reserved":0,"allocated":3330723840,"data_stored":3330235107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":932900358},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":1.4370000000000001,"15min":1.851},"min":{"1min":0.22800000000000001,"5min":0.21099999999999999,"15min":0.191},"max":{"1min":1.5940000000000001,"5min":20.832999999999998,"15min":75.677000000000007},"last":2},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.3380000000000001,"15min":1.899},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":4.9710000000000001,"5min":20.937000000000001,"15min":75.638999999999996},"last":0.25900000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:46:36 2024","interfaces":[{"interface":"back","average":{"1min":18.690999999999999,"5min":4.3159999999999998,"15min":2.3599999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.22500000000000001},"max":{"1min":327.06099999999998,"5min":327.06099999999998,"15min":327.06099999999998},"last":2.1080000000000001},{"interface":"front","average":{"1min":18.693999999999999,"5min":4.3070000000000004,"15min":2.3860000000000001},"min":{"1min":0.313,"5min":0.26300000000000001,"15min":0.22800000000000001},"max":{"1min":327.07499999999999,"5min":327.07499999999999,"15min":327.07499999999999},"last":0.28699999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.79000000000000004,"15min":1.792},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.22800000000000001},"max":{"1min":0.95599999999999996,"5min":6.4820000000000002,"15min":227.79300000000001},"last":0.245},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.78100000000000003,"15min":1.8120000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.246},"max":{"1min":1.1839999999999999,"5min":6.6600000000000001,"15min":227.22},"last":0.31900000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.73799999999999999,"15min":1.3740000000000001},"min":{"1min":0.33200000000000002,"5min":0.252,"15min":0.252},"max":{"1min":1.7450000000000001,"5min":3.399,"15min":49.798999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.79800000000000004,"15min":1.4279999999999999},"min":{"1min":0.33400000000000002,"5min":0.29499999999999998,"15min":0.27200000000000002},"max":{"1min":2.097,"5min":6,"15min":49.524000000000001},"last":4.9550000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.67500000000000004,"15min":0.70099999999999996},"min":{"1min":0.30199999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.71599999999999997,"5min":3.8319999999999999,"15min":6.0220000000000002},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.70799999999999996,"15min":0.74399999999999999},"min":{"1min":0.28499999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.65000000000000002,"5min":3.3170000000000002,"15min":6.9000000000000004},"last":0.32700000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.69599999999999995,"15min":0.78200000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.25},"max":{"1min":0.67800000000000005,"5min":9.9339999999999993,"15min":9.9339999999999993},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.61699999999999999,"15min":0.76800000000000002},"min":{"1min":0.312,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.59699999999999998,"5min":10.183999999999999,"15min":10.183999999999999},"last":0.27700000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.57599999999999996,"15min":0.66300000000000003},"min":{"1min":0.316,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.302,"5min":2.1190000000000002,"15min":5.1909999999999998},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.65600000000000003,"15min":0.73999999999999999},"min":{"1min":0.33400000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.341,"5min":2.6970000000000001,"15min":4.2030000000000003},"last":0.41699999999999998}]}]},{"osd":6,"up_from":44,"seq":188978561286,"num_pgs":86,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5544088,"kb_used_data":4655120,"kb_used_omap":1,"kb_used_meta":888958,"kb_avail":88201064,"statfs":{"total":95995035648,"available":90317889536,"internally_reserved":0,"allocated":4766842880,"data_stored":4766327263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":910293336},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:47:31 2024","interfaces":[{"interface":"back","average":{"1min":1.73,"5min":3.4820000000000002,"15min":2.399},"min":{"1min":0.23100000000000001,"5min":0.22900000000000001,"15min":0.20699999999999999},"max":{"1min":9.6869999999999994,"5min":130.62299999999999,"15min":142.29599999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":1.224,"5min":3.3690000000000002,"15min":2.5430000000000001},"min":{"1min":0.216,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":4.6680000000000001,"5min":130.86199999999999,"15min":142.68799999999999},"last":0.47899999999999998}]},{"osd":1,"last update":"Wed Sep 11 07:47:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":3.1110000000000002,"15min":2.415},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.23599999999999999},"max":{"1min":2.7970000000000002,"5min":131.42500000000001,"15min":131.42500000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":1.117,"5min":3.0990000000000002,"15min":2.2970000000000002},"min":{"1min":0.30199999999999999,"5min":0.22,"15min":0.22},"max":{"1min":5.4189999999999996,"5min":132.39599999999999,"15min":132.39599999999999},"last":3.6539999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:47:31 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":2.2570000000000001,"15min":1.3440000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.24299999999999999},"max":{"1min":5.2309999999999999,"5min":83.915999999999997,"15min":83.915999999999997},"last":1.925},{"interface":"front","average":{"1min":1.3979999999999999,"5min":2.371,"15min":1.472},"min":{"1min":0.372,"5min":0.254,"15min":0.254},"max":{"1min":2.9830000000000001,"5min":84.097999999999999,"15min":84.097999999999999},"last":1.8620000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:47:31 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3660000000000001,"15min":1.5509999999999999},"min":{"1min":0.315,"5min":0.30099999999999999,"15min":0.246},"max":{"1min":4.2119999999999997,"5min":39.308999999999997,"15min":49.109000000000002},"last":0.372},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.4319999999999999,"15min":1.5669999999999999},"min":{"1min":0.28299999999999997,"5min":0.246,"15min":0.246},"max":{"1min":4.2889999999999997,"5min":41.149999999999999,"15min":48.963999999999999},"last":3.907}]},{"osd":4,"last update":"Wed Sep 11 07:47:31 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":1.135,"15min":1.768},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.223},"max":{"1min":0.52000000000000002,"5min":12.257999999999999,"15min":181.78800000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.39000000000000001,"5min":1.034,"15min":1.1830000000000001},"min":{"1min":0.309,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55200000000000005,"5min":12.102,"15min":128.59299999999999},"last":0.34000000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:47:31 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.71799999999999997,"15min":1.01},"min":{"1min":0.27400000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.59099999999999997,"5min":7.4219999999999997,"15min":57.436},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.82599999999999996,"15min":1.0640000000000001},"min":{"1min":0.30299999999999999,"5min":0.254,"15min":0.251},"max":{"1min":0.66700000000000004,"5min":7.4420000000000002,"15min":56.822000000000003},"last":0.497}]},{"osd":7,"last update":"Wed Sep 11 07:47:31 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":1.613,"15min":1.0920000000000001},"min":{"1min":0.372,"5min":0.32300000000000001,"15min":0.27800000000000002},"max":{"1min":0.79100000000000004,"5min":60.279000000000003,"15min":60.279000000000003},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.436,"5min":1.4390000000000001,"15min":1.0229999999999999},"min":{"1min":0.318,"5min":0.311,"15min":0.215},"max":{"1min":0.77100000000000002,"5min":60.767000000000003,"15min":60.767000000000003},"last":0.38800000000000001}]}]},{"osd":5,"up_from":37,"seq":158913790220,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1825340,"kb_used_data":941620,"kb_used_omap":1,"kb_used_meta":883710,"kb_avail":91919812,"statfs":{"total":95995035648,"available":94125887488,"internally_reserved":0,"allocated":964218880,"data_stored":963730712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":904919384},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:47:09 2024","interfaces":[{"interface":"back","average":{"1min":3.4969999999999999,"5min":1.5660000000000001,"15min":1.8580000000000001},"min":{"1min":0.25900000000000001,"5min":0.23100000000000001,"15min":0.20200000000000001},"max":{"1min":40.313000000000002,"5min":40.313000000000002,"15min":118.086},"last":2.7080000000000002},{"interface":"front","average":{"1min":3.5049999999999999,"5min":1.669,"15min":2.0150000000000001},"min":{"1min":0.29599999999999999,"5min":0.20899999999999999,"15min":0.20200000000000001},"max":{"1min":40.235999999999997,"5min":40.235999999999997,"15min":118.004},"last":1.925}]},{"osd":1,"last update":"Wed Sep 11 07:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.80200000000000005,"15min":1.5900000000000001},"min":{"1min":0.26800000000000002,"5min":0.23300000000000001,"15min":0.20799999999999999},"max":{"1min":4.2110000000000003,"5min":7.2510000000000003,"15min":122.738},"last":1.163},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.755,"15min":1.617},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.22700000000000001},"max":{"1min":2.7839999999999998,"5min":3.476,"15min":122.66200000000001},"last":0.59799999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:47:09 2024","interfaces":[{"interface":"back","average":{"1min":2.73,"5min":1.5609999999999999,"15min":1.242},"min":{"1min":0.26700000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":26.707000000000001,"5min":26.707000000000001,"15min":26.707000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":3.0840000000000001,"5min":1.484,"15min":1.1950000000000001},"min":{"1min":0.247,"5min":0.247,"15min":0.24399999999999999},"max":{"1min":26.666,"5min":26.666,"15min":26.666},"last":0.30199999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":2.8719999999999999,"15min":2.7879999999999998},"min":{"1min":0.30599999999999999,"5min":0.28899999999999998,"15min":0.26400000000000001},"max":{"1min":3.1269999999999998,"5min":132.715,"15min":132.715},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.76500000000000001,"5min":2.7160000000000002,"15min":2.8570000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.25700000000000001},"max":{"1min":2.9710000000000001,"5min":133.01599999999999,"15min":133.01599999999999},"last":0.36099999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.69099999999999995,"15min":0.77600000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.24199999999999999},"max":{"1min":0.51700000000000002,"5min":4.5090000000000003,"15min":11.103},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.64800000000000002,"15min":0.78600000000000003},"min":{"1min":0.30299999999999999,"5min":0.27300000000000002,"15min":0.22500000000000001},"max":{"1min":0.48999999999999999,"5min":3.7959999999999998,"15min":10.305},"last":0.34499999999999997}]},{"osd":6,"last update":"Wed Sep 11 07:47:27 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.55200000000000005,"15min":0.72899999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.61099999999999999,"5min":2.4100000000000001,"15min":5.6859999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.59999999999999998,"15min":0.73799999999999999},"min":{"1min":0.33100000000000002,"5min":0.30299999999999999,"15min":0.26800000000000002},"max":{"1min":1.3700000000000001,"5min":3.0990000000000002,"15min":10.613},"last":0.33800000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:47:27 2024","interfaces":[{"interface":"back","average":{"1min":7.2080000000000002,"5min":2.3700000000000001,"15min":1.5489999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28000000000000003},"max":{"1min":115.44499999999999,"5min":115.44499999999999,"15min":115.44499999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":7.1890000000000001,"5min":2.3119999999999998,"15min":1.506},"min":{"1min":0.33900000000000002,"5min":0.29799999999999999,"15min":0.26800000000000002},"max":{"1min":115.071,"5min":115.071,"15min":115.071},"last":0.314}]}]},{"osd":4,"up_from":0,"seq":133143986448,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":25,"seq":107374182677,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2807304,"kb_used_data":1895936,"kb_used_omap":1,"kb_used_meta":911358,"kb_avail":90937848,"statfs":{"total":95995035648,"available":93120356352,"internally_reserved":0,"allocated":1941438464,"data_stored":1940937704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":933230936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,4,2,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:47:37 2024","interfaces":[{"interface":"back","average":{"1min":1.8360000000000001,"5min":1.411,"15min":1.536},"min":{"1min":0.184,"5min":0.184,"15min":0.16600000000000001},"max":{"1min":3.8839999999999999,"5min":27.655999999999999,"15min":38.432000000000002},"last":3.8839999999999999},{"interface":"front","average":{"1min":2.036,"5min":1.4650000000000001,"15min":1.661},"min":{"1min":0.23400000000000001,"5min":0.183,"15min":0.183},"max":{"1min":4.5759999999999996,"5min":26.759,"15min":33.426000000000002},"last":3.9350000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:47:37 2024","interfaces":[{"interface":"back","average":{"1min":2.2589999999999999,"5min":1.2270000000000001,"15min":2.6899999999999999},"min":{"1min":0.30599999999999999,"5min":0.22500000000000001,"15min":0.19500000000000001},"max":{"1min":16.971,"5min":19.753,"15min":135.41999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":2.1400000000000001,"5min":1.2729999999999999,"15min":2.4380000000000002},"min":{"1min":0.222,"5min":0.221,"15min":0.192},"max":{"1min":18.277000000000001,"5min":19.484999999999999,"15min":136.126},"last":3.4580000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:47:37 2024","interfaces":[{"interface":"back","average":{"1min":1.7969999999999999,"5min":1.546,"15min":1.8160000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":4.5979999999999999,"5min":21.158000000000001,"15min":164.02600000000001},"last":3.5619999999999998},{"interface":"front","average":{"1min":2.593,"5min":1.5149999999999999,"15min":1.7330000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.222},"max":{"1min":10.752000000000001,"5min":17.265000000000001,"15min":163.37799999999999},"last":4.0039999999999996}]},{"osd":4,"last update":"Wed Sep 11 07:46:56 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":1.1200000000000001,"15min":1.7729999999999999},"min":{"1min":0.32000000000000001,"5min":0.314,"15min":0.27500000000000002},"max":{"1min":1.369,"5min":8.6479999999999997,"15min":132.76400000000001},"last":3.9119999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":1.131,"15min":1.8300000000000001},"min":{"1min":0.34200000000000003,"5min":0.317,"15min":0.255},"max":{"1min":2.532,"5min":21.965,"15min":132.154},"last":0.85499999999999998}]},{"osd":5,"last update":"Wed Sep 11 07:47:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77700000000000002,"15min":1.579},"min":{"1min":0.33500000000000002,"5min":0.32500000000000001,"15min":0.28599999999999998},"max":{"1min":1.411,"5min":3.8889999999999998,"15min":137.50299999999999},"last":3.9510000000000001},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83899999999999997,"15min":1.5760000000000001},"min":{"1min":0.35399999999999998,"5min":0.33000000000000002,"15min":0.28299999999999997},"max":{"1min":2.6120000000000001,"5min":3.7639999999999998,"15min":137.45599999999999},"last":0.82399999999999995}]},{"osd":6,"last update":"Wed Sep 11 07:47:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":3.6960000000000002,"15min":2.028},"min":{"1min":0.35099999999999998,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":3.5019999999999998,"5min":233.98500000000001,"15min":233.98500000000001},"last":3.5019999999999998},{"interface":"front","average":{"1min":1.294,"5min":3.7890000000000001,"15min":2.0489999999999999},"min":{"1min":0.41299999999999998,"5min":0.35599999999999998,"15min":0.30399999999999999},"max":{"1min":3.98,"5min":234.05199999999999,"15min":234.05199999999999},"last":3.98}]},{"osd":7,"last update":"Wed Sep 11 07:47:24 2024","interfaces":[{"interface":"back","average":{"1min":2.0249999999999999,"5min":3.4750000000000001,"15min":2.7360000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.308},"max":{"1min":13.935,"5min":162.42699999999999,"15min":162.42699999999999},"last":0.878},{"interface":"front","average":{"1min":1.276,"5min":3.3079999999999998,"15min":2.734},"min":{"1min":0.40400000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.9380000000000002,"5min":162.39099999999999,"15min":162.39099999999999},"last":3.5299999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411610,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5545352,"kb_used_data":4652928,"kb_used_omap":1,"kb_used_meta":892414,"kb_avail":88199800,"statfs":{"total":95995035648,"available":90316595200,"internally_reserved":0,"allocated":4764598272,"data_stored":4764098808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":913832280},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:47:12 2024","interfaces":[{"interface":"back","average":{"1min":2.8570000000000002,"5min":1.754,"15min":2.012},"min":{"1min":0.25,"5min":0.188,"15min":0.16400000000000001},"max":{"1min":6.5970000000000004,"5min":19.907,"15min":57.802999999999997},"last":0.215},{"interface":"front","average":{"1min":2.0590000000000002,"5min":1.6990000000000001,"15min":2.0670000000000002},"min":{"1min":0.249,"5min":0.188,"15min":0.188},"max":{"1min":5.7460000000000004,"5min":16.039999999999999,"15min":57.759},"last":6.0720000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:47:12 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":0.93600000000000005,"15min":1.4850000000000001},"min":{"1min":0.26500000000000001,"5min":0.22900000000000001,"15min":0.22800000000000001},"max":{"1min":3.6360000000000001,"5min":5.3570000000000002,"15min":65.153999999999996},"last":2.6800000000000002},{"interface":"front","average":{"1min":2.2210000000000001,"5min":1.1200000000000001,"15min":1.77},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":8.9589999999999996,"5min":8.9589999999999996,"15min":63.311999999999998},"last":2.8639999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:46:42 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.999,"15min":1.802},"min":{"1min":0.28399999999999997,"5min":0.22500000000000001,"15min":0.224},"max":{"1min":2.9239999999999999,"5min":5.2149999999999999,"15min":67.552999999999997},"last":3.1869999999999998},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.1080000000000001,"15min":1.8009999999999999},"min":{"1min":0.309,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":4.4779999999999998,"5min":9.4670000000000005,"15min":67.352999999999994},"last":2.831}]},{"osd":4,"last update":"Wed Sep 11 07:46:53 2024","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.198,"15min":1.079},"min":{"1min":0.36199999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.77,"5min":7.1719999999999997,"15min":25.643000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.97299999999999998,"15min":0.97999999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.31},"max":{"1min":2.4329999999999998,"5min":3.702,"15min":22.434000000000001},"last":2.6520000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:47:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.96999999999999997,"15min":1.0209999999999999},"min":{"1min":0.38100000000000001,"5min":0.371,"15min":0.32100000000000001},"max":{"1min":3.5630000000000002,"5min":3.8340000000000001,"15min":22.460000000000001},"last":2.5840000000000001},{"interface":"front","average":{"1min":1.821,"5min":1.1819999999999999,"15min":1.159},"min":{"1min":0.47499999999999998,"5min":0.313,"15min":0.313},"max":{"1min":6.6310000000000002,"5min":6.6310000000000002,"15min":25.696999999999999},"last":0.372}]},{"osd":6,"last update":"Wed Sep 11 07:46:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":1.0760000000000001,"15min":1.139},"min":{"1min":0.373,"5min":0.373,"15min":0.33400000000000002},"max":{"1min":2.4580000000000002,"5min":4.298,"15min":22.399000000000001},"last":2.5379999999999998},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.1659999999999999,"15min":1.1879999999999999},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.29999999999999999},"max":{"1min":4.4119999999999999,"5min":5.1639999999999997,"15min":25.739000000000001},"last":0.34999999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:47:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.1060000000000001,"15min":1.0549999999999999},"min":{"1min":0.43099999999999999,"5min":0.40600000000000003,"15min":0.34300000000000003},"max":{"1min":3.536,"5min":4.6950000000000003,"15min":22.484000000000002},"last":2.6179999999999999},{"interface":"front","average":{"1min":2.5230000000000001,"5min":1.3260000000000001,"15min":1.2649999999999999},"min":{"1min":0.52200000000000002,"5min":0.39000000000000001,"15min":0.36699999999999999},"max":{"1min":6.6760000000000002,"5min":6.6760000000000002,"15min":25.670000000000002},"last":0.39400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575133,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3699832,"kb_used_data":2787184,"kb_used_omap":1,"kb_used_meta":912638,"kb_avail":90045320,"statfs":{"total":95995035648,"available":92206407680,"internally_reserved":0,"allocated":2854076416,"data_stored":2853591779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":934541656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:47:38 2024","interfaces":[{"interface":"back","average":{"1min":7.4249999999999998,"5min":2.694,"15min":1.9079999999999999},"min":{"1min":0.312,"5min":0.20799999999999999,"15min":0.16900000000000001},"max":{"1min":88.762,"5min":88.762,"15min":88.762},"last":2.8159999999999998},{"interface":"front","average":{"1min":6.8200000000000003,"5min":2.6680000000000001,"15min":1.952},"min":{"1min":0.24299999999999999,"5min":0.223,"15min":0.186},"max":{"1min":84.926000000000002,"5min":84.926000000000002,"15min":84.926000000000002},"last":0.27700000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:47:12 2024","interfaces":[{"interface":"back","average":{"1min":6.4480000000000004,"5min":3.2000000000000002,"15min":2.1779999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.19800000000000001},"max":{"1min":84.584000000000003,"5min":84.584000000000003,"15min":84.584000000000003},"last":1.5489999999999999},{"interface":"front","average":{"1min":6.3289999999999997,"5min":3.0470000000000002,"15min":2.109},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.191},"max":{"1min":84.656999999999996,"5min":84.656999999999996,"15min":84.656999999999996},"last":1.667}]},{"osd":3,"last update":"Wed Sep 11 07:47:30 2024","interfaces":[{"interface":"back","average":{"1min":23.048999999999999,"5min":5.5789999999999997,"15min":3.5379999999999998},"min":{"1min":0.32000000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":276.24900000000002,"5min":276.24900000000002,"15min":276.24900000000002},"last":0.248},{"interface":"front","average":{"1min":22.138999999999999,"5min":5.2149999999999999,"15min":3.4060000000000001},"min":{"1min":0.33300000000000002,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":274.98000000000002,"5min":274.98000000000002,"15min":274.98000000000002},"last":2.9289999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:46:47 2024","interfaces":[{"interface":"back","average":{"1min":1.631,"5min":1.153,"15min":1.4419999999999999},"min":{"1min":0.36599999999999999,"5min":0.30299999999999999,"15min":0.27400000000000002},"max":{"1min":13.82,"5min":13.82,"15min":13.82},"last":1.528},{"interface":"front","average":{"1min":1.409,"5min":1.1779999999999999,"15min":1.4079999999999999},"min":{"1min":0.34999999999999998,"5min":0.30099999999999999,"15min":0.28499999999999998},"max":{"1min":13.801,"5min":13.801,"15min":13.801},"last":1.5960000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:47:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":0.84799999999999998,"15min":1.333},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.26600000000000001},"max":{"1min":5.3109999999999999,"5min":5.3109999999999999,"15min":8.3680000000000003},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.343,"5min":0.94099999999999995,"15min":1.355},"min":{"1min":0.318,"5min":0.315,"15min":0.27200000000000002},"max":{"1min":4.1879999999999997,"5min":4.258,"15min":9.1929999999999996},"last":1.623}]},{"osd":6,"last update":"Wed Sep 11 07:47:30 2024","interfaces":[{"interface":"back","average":{"1min":1.6970000000000001,"5min":1.2070000000000001,"15min":1.331},"min":{"1min":0.318,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":4.0389999999999997,"5min":13.981999999999999,"15min":13.981999999999999},"last":1.6439999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.93500000000000005,"15min":1.105},"min":{"1min":0.38700000000000001,"5min":0.32700000000000001,"15min":0.313},"max":{"1min":1.8169999999999999,"5min":13.928000000000001,"15min":13.928000000000001},"last":2.9039999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:47:30 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76100000000000001,"15min":1.2},"min":{"1min":0.41499999999999998,"5min":0.36499999999999999,"15min":0.32200000000000001},"max":{"1min":1.7889999999999999,"5min":3.2690000000000001,"15min":9.3629999999999995},"last":2.875},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.153,"15min":1.4930000000000001},"min":{"1min":0.375,"5min":0.372,"15min":0.313},"max":{"1min":2.766,"5min":13.715,"15min":13.715},"last":1.71}]}]},{"osd":0,"up_from":8,"seq":34359738659,"num_pgs":81,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5100112,"kb_used_data":4210888,"kb_used_omap":1,"kb_used_meta":889214,"kb_avail":88645040,"statfs":{"total":95995035648,"available":90772520960,"internally_reserved":0,"allocated":4311949312,"data_stored":4311440390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":910555480},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:47:37 2024","interfaces":[{"interface":"back","average":{"1min":1.946,"5min":1.7829999999999999,"15min":1.4770000000000001},"min":{"1min":0.35199999999999998,"5min":0.19600000000000001,"15min":0.17899999999999999},"max":{"1min":6.79,"5min":72.418000000000006,"15min":72.418000000000006},"last":4.617},{"interface":"front","average":{"1min":2.0779999999999998,"5min":1.7829999999999999,"15min":1.508},"min":{"1min":0.377,"5min":0.17599999999999999,"15min":0.16600000000000001},"max":{"1min":6.3949999999999996,"5min":71.703999999999994,"15min":71.703999999999994},"last":0.38100000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:47:09 2024","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":3.718,"15min":2.883},"min":{"1min":0.26500000000000001,"5min":0.25900000000000001,"15min":0.188},"max":{"1min":4.3109999999999999,"5min":164.821,"15min":200.339},"last":3.3279999999999998},{"interface":"front","average":{"1min":1.696,"5min":3.9060000000000001,"15min":2.8380000000000001},"min":{"1min":0.30099999999999999,"5min":0.23999999999999999,"15min":0.20699999999999999},"max":{"1min":4.6500000000000004,"5min":167.68799999999999,"15min":200.40100000000001},"last":3.661}]},{"osd":3,"last update":"Wed Sep 11 07:47:37 2024","interfaces":[{"interface":"back","average":{"1min":1.8500000000000001,"5min":1.718,"15min":1.542},"min":{"1min":0.36799999999999999,"5min":0.255,"15min":0.20799999999999999},"max":{"1min":7.6130000000000004,"5min":37.143000000000001,"15min":37.143000000000001},"last":3.1779999999999999},{"interface":"front","average":{"1min":2.1859999999999999,"5min":1.853,"15min":1.569},"min":{"1min":0.36299999999999999,"5min":0.253,"15min":0.22800000000000001},"max":{"1min":4.7039999999999997,"5min":38.198,"15min":38.198},"last":6.5410000000000004}]},{"osd":4,"last update":"Wed Sep 11 07:46:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.1100000000000001,"15min":1.353},"min":{"1min":0.34200000000000003,"5min":0.31,"15min":0.26700000000000002},"max":{"1min":2.8340000000000001,"5min":7.0519999999999996,"15min":30.315000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.1679999999999999,"15min":1.4730000000000001},"min":{"1min":0.33000000000000002,"5min":0.311,"15min":0.27000000000000002},"max":{"1min":3.069,"5min":6.5439999999999996,"15min":30.363},"last":3.2959999999999998}]},{"osd":5,"last update":"Wed Sep 11 07:47:09 2024","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.034,"15min":1.173},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.26800000000000002},"max":{"1min":4.2789999999999999,"5min":5.0119999999999996,"15min":6.9400000000000004},"last":0.42699999999999999},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.141,"15min":1.3169999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.27200000000000002},"max":{"1min":3.141,"5min":6.46,"15min":9.6289999999999996},"last":3.2109999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:47:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.214,"15min":1.29},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.6829999999999998,"5min":6.5190000000000001,"15min":23.190999999999999},"last":3.2719999999999998},{"interface":"front","average":{"1min":1.615,"5min":1.224,"15min":1.3009999999999999},"min":{"1min":0.38700000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":4.6310000000000002,"5min":4.6310000000000002,"15min":23.632000000000001},"last":3.5649999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:47:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.6459999999999999,"15min":1.534},"min":{"1min":0.39600000000000002,"5min":0.375,"15min":0.34399999999999997},"max":{"1min":2.7959999999999998,"5min":34.704999999999998,"15min":34.704999999999998},"last":3.2469999999999999},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.6819999999999999,"15min":1.5529999999999999},"min":{"1min":0.41299999999999998,"5min":0.38800000000000001,"15min":0.315},"max":{"1min":3.8580000000000001,"5min":34.930999999999997,"15min":34.930999999999997},"last":3.536}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4315742208,"data_stored":4315742208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2853306368,"data_stored":2853306368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4774363136,"data_stored":4774363136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1949106176,"data_stored":1949106176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":964689920,"data_stored":964689920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4803198976,"data_stored":4803198976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3357474816,"data_stored":3357474816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:47:43.222 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pool set unique_pool_4 pgp_num 16 2024-09-11T07:47:43.239 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:43.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:43.316 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:43.316 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:43.417 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:43.441 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:43.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:43.442 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:43.543 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:43.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:43.569 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:43.569 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:43.670 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:43.695 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:43.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:43.697 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:43.719 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:43 smithi022 ceph-mon[29462]: from='client.26074 v1:172.21.15.22:0/1172863056' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:47:43.719 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:43 smithi022 ceph-mon[29462]: pgmap v989: 158 pgs: 158 active+clean; 7.2 GiB data, 27 GiB used, 598 GiB / 626 GiB avail; 935 B/s rd, 57 MiB/s wr, 920 op/s 2024-09-11T07:47:43.719 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:43 smithi022 ceph-mon[37330]: from='client.26074 v1:172.21.15.22:0/1172863056' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:47:43.719 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:43 smithi022 ceph-mon[37330]: pgmap v989: 158 pgs: 158 active+clean; 7.2 GiB data, 27 GiB used, 598 GiB / 626 GiB avail; 935 B/s rd, 57 MiB/s wr, 920 op/s 2024-09-11T07:47:43.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:43 smithi104 ceph-mon[33507]: from='client.26074 v1:172.21.15.22:0/1172863056' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:47:43.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:43 smithi104 ceph-mon[33507]: pgmap v989: 158 pgs: 158 active+clean; 7.2 GiB data, 27 GiB used, 598 GiB / 626 GiB avail; 935 B/s rd, 57 MiB/s wr, 920 op/s 2024-09-11T07:47:43.798 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:43.823 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:43.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:43.824 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:43.925 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:43.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:43.952 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:43.952 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:44.053 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:44.077 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:44.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:44.078 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:44.153 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:47:44.153 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:47:44.179 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:44.183 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:47:44.183 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:47:44.201 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:47:44.201 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:47:44.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:44.202 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:44.202 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:44.278 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:47:44.278 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:47:44.303 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:44.307 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:47:44.308 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:47:44.327 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:44.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:44.328 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:44.339 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:47:44.339 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:47:44.370 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:47:44.370 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:47:44.404 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:47:44.404 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:47:44.428 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:44.452 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:44.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:44.453 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:44.499 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:47:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:47:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:47:44.554 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:44.574 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:44 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1704843038' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:47:44.574 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:44 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1704843038' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:47:44.580 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:44.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:44.581 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:44.682 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:44.708 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:44.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:44.709 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:44.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:44 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1704843038' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:47:44.811 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:44.838 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:44.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:44.839 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:44.940 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:44.963 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:44.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:44.965 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:45.065 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:45.091 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:45.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:45.092 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:45.192 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:45.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:45.245 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:45.245 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:45.346 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:45.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:45.393 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:45.495 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:45.530 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:45.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:45.531 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:45.632 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:45.657 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:45.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:45.658 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:45.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:45 smithi022 ceph-mon[29462]: pgmap v990: 158 pgs: 158 active+clean; 7.2 GiB data, 27 GiB used, 598 GiB / 626 GiB avail; 1.3 KiB/s rd, 56 MiB/s wr, 953 op/s 2024-09-11T07:47:45.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:45 smithi022 ceph-mon[37330]: pgmap v990: 158 pgs: 158 active+clean; 7.2 GiB data, 27 GiB used, 598 GiB / 626 GiB avail; 1.3 KiB/s rd, 56 MiB/s wr, 953 op/s 2024-09-11T07:47:45.759 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:45.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:45 smithi104 ceph-mon[33507]: pgmap v990: 158 pgs: 158 active+clean; 7.2 GiB data, 27 GiB used, 598 GiB / 626 GiB avail; 1.3 KiB/s rd, 56 MiB/s wr, 953 op/s 2024-09-11T07:47:45.800 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:45.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:45.801 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:45.902 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:45.939 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:45.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:45.940 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:46.040 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:46.066 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:46.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:46.067 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:46.168 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:46.193 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:46.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:46.194 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:46.210 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:47:46.273 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:47:46.294 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:46.344 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:46.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:46.345 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:46.446 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:46.486 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:46.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:46.487 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:46.587 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:46.613 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:46.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:46.614 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:46.715 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:46.739 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:46.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:46.740 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:46.841 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:46.866 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:46.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:46.867 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:46.968 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:46.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:46.993 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:46.993 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:47.094 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:47.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:47.130 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:47.131 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:47.231 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:47.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:47.277 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:47.278 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:47.378 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:47.429 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:47.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:47.430 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:47.531 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:47.537 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:47:47.537 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:47:47.578 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:47.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:47.579 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:47.605 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:47 smithi022 ceph-mon[29462]: pgmap v991: 158 pgs: 158 active+clean; 7.1 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 938 B/s rd, 60 MiB/s wr, 1.21k op/s 2024-09-11T07:47:47.605 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:47 smithi022 ceph-mon[37330]: pgmap v991: 158 pgs: 158 active+clean; 7.1 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 938 B/s rd, 60 MiB/s wr, 1.21k op/s 2024-09-11T07:47:47.679 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:47.721 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:47.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:47.722 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:47.735 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:47:47.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:47 smithi104 ceph-mon[33507]: pgmap v991: 158 pgs: 158 active+clean; 7.1 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 938 B/s rd, 60 MiB/s wr, 1.21k op/s 2024-09-11T07:47:47.822 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:47.848 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:47.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:47.849 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:47.949 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:47.971 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:47.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:47.972 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:48.072 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:48.100 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:48.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:48.101 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:48.202 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:48.239 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:48.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:48.241 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:48.341 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:48.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:48.389 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:48.389 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:48.489 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:48.529 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:48.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:48.530 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:48.631 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:48.656 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:48.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:48.657 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:48.758 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:48.802 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:48.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:48.803 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:48.903 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:48.947 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:48.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:48.948 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:49.049 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:49.072 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:49.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:49.073 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:49.174 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:49.199 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:49.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:49.200 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:49.301 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:49.324 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:49.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:49.325 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:49.426 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:49.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:49.448 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:49.448 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:49.549 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:49.576 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:49.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:49.577 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:49.678 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:49.719 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:49.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:49.720 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:49.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:49 smithi104 ceph-mon[33507]: pgmap v992: 158 pgs: 158 active+clean; 7.1 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 938 B/s rd, 42 MiB/s wr, 917 op/s 2024-09-11T07:47:49.821 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:49.845 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:49.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:49.846 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:49.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:49 smithi022 ceph-mon[29462]: pgmap v992: 158 pgs: 158 active+clean; 7.1 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 938 B/s rd, 42 MiB/s wr, 917 op/s 2024-09-11T07:47:49.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:49 smithi022 ceph-mon[37330]: pgmap v992: 158 pgs: 158 active+clean; 7.1 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 938 B/s rd, 42 MiB/s wr, 917 op/s 2024-09-11T07:47:49.947 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:49.973 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:49.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:49.974 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:50.074 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:50.111 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:50.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:50.112 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:50.213 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:50.238 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:50.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:50.239 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:50.340 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:50.366 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:50.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:50.367 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:50.467 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:50.488 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:50.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:50.489 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:50.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:47:50 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:47:50.591 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:50.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:50.632 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:50.632 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:50.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:47:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:47:50] "GET /metrics HTTP/1.1" 200 225165 "" "Prometheus/2.43.0" 2024-09-11T07:47:50.733 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:50.761 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:50.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:50.762 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:50.862 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:50.888 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:50.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:50.889 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:50.990 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:51.016 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:51.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:51.017 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:51.117 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:51.143 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:51.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:51.144 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:51.245 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:51.284 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:51.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:51.285 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:51.386 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:51.411 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:51.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:51.412 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:51.512 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:51.538 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:51.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:51.541 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:51.641 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:51.667 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:51.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:51.668 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:51.768 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:51.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:51 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:47:51.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:51 smithi104 ceph-mon[33507]: pgmap v993: 158 pgs: 158 active+clean; 6.8 GiB data, 26 GiB used, 599 GiB / 626 GiB avail; 1.2 KiB/s rd, 42 MiB/s wr, 1.41k op/s 2024-09-11T07:47:51.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:51.797 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:51.797 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:51.808 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:51 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:47:51.808 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:51 smithi022 ceph-mon[29462]: pgmap v993: 158 pgs: 158 active+clean; 6.8 GiB data, 26 GiB used, 599 GiB / 626 GiB avail; 1.2 KiB/s rd, 42 MiB/s wr, 1.41k op/s 2024-09-11T07:47:51.808 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:51 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:47:51.808 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:51 smithi022 ceph-mon[37330]: pgmap v993: 158 pgs: 158 active+clean; 6.8 GiB data, 26 GiB used, 599 GiB / 626 GiB avail; 1.2 KiB/s rd, 42 MiB/s wr, 1.41k op/s 2024-09-11T07:47:51.898 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:51.924 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:51.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:51.925 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:51.979 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-11T07:47:52.026 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:52.049 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:52.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:52.051 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:52.151 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:52.175 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:52.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:52.176 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:52.277 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:52.339 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:52.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:52.340 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:52.441 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:52.466 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:52.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:52.467 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:52.568 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:52.591 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:52.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:52.593 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:52.680 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:47:52.693 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:52.717 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:52.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:52.718 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:52.819 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:52.845 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:52.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:52.846 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:52.921 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:47:52.947 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:53.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:53.002 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:53.002 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:53.103 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:53.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:53.129 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:53.130 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:53.230 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:53.256 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:53.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:53.257 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:53.358 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:53.408 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:53.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:53.409 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:53.510 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:53.535 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:53.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:53.536 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:53.637 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:53.673 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:53.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:53.674 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:53.775 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:53.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:53 smithi104 ceph-mon[33507]: pgmap v994: 158 pgs: 158 active+clean; 6.8 GiB data, 26 GiB used, 599 GiB / 626 GiB avail; 852 B/s rd, 13 MiB/s wr, 953 op/s 2024-09-11T07:47:53.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:47:53.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:53.801 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:53.802 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:53.816 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:53 smithi022 ceph-mon[29462]: pgmap v994: 158 pgs: 158 active+clean; 6.8 GiB data, 26 GiB used, 599 GiB / 626 GiB avail; 852 B/s rd, 13 MiB/s wr, 953 op/s 2024-09-11T07:47:53.817 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:47:53.817 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:53 smithi022 ceph-mon[37330]: pgmap v994: 158 pgs: 158 active+clean; 6.8 GiB data, 26 GiB used, 599 GiB / 626 GiB avail; 852 B/s rd, 13 MiB/s wr, 953 op/s 2024-09-11T07:47:53.817 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:47:53.902 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:53.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:53.935 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:53.936 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:54.036 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:54.059 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:54.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:54.060 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:54.160 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:54.182 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:54.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:54.183 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:54.283 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:54.310 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:54.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:54.311 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:54.412 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:54.453 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:54.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:54.454 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:54.524 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is set 2024-09-11T07:47:54.527 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:47:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:47:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:47:54.555 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:54.577 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:54.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:54.578 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:54.679 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:54.702 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:54.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:54.703 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:54.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:54 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2047957291' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:47:54.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:54 smithi104 ceph-mon[33507]: from='client.26083 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:47:54.804 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:54.830 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:54.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:54.831 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:54.916 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:54 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2047957291' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:47:54.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:54 smithi022 ceph-mon[37330]: from='client.26083 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:47:54.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:54 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2047957291' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:47:54.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:54 smithi022 ceph-mon[29462]: from='client.26083 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:47:54.932 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:54.970 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:54.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:54.972 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:55.072 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:55.098 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:55.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:55.099 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:55.200 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:55.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:55.226 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:55.227 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:55.328 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:55.366 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:55.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:55.367 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:55.468 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:55.492 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:55.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:55.493 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:55.528 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:47:55 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a[63641]: logger=cleanup t=2024-09-11T07:47:55.061533925Z level=info msg="Completed cleanup jobs" duration=13.839152ms 2024-09-11T07:47:55.593 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:55.617 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:55.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:55.618 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:55.719 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:55.744 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:55.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:55.745 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:55.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:55 smithi104 ceph-mon[33507]: pgmap v995: 158 pgs: 158 active+clean; 6.8 GiB data, 26 GiB used, 599 GiB / 626 GiB avail; 853 B/s rd, 13 MiB/s wr, 953 op/s 2024-09-11T07:47:55.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:55 smithi104 ceph-mon[33507]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:47:55.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:55 smithi104 ceph-mon[33507]: from='client.26083 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:47:55.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:55 smithi104 ceph-mon[33507]: osdmap e442: 8 total, 8 up, 7 in 2024-09-11T07:47:55.846 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:55.852 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:55 smithi022 ceph-mon[37330]: pgmap v995: 158 pgs: 158 active+clean; 6.8 GiB data, 26 GiB used, 599 GiB / 626 GiB avail; 853 B/s rd, 13 MiB/s wr, 953 op/s 2024-09-11T07:47:55.852 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:55 smithi022 ceph-mon[37330]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:47:55.852 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:55 smithi022 ceph-mon[37330]: from='client.26083 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:47:55.852 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:55 smithi022 ceph-mon[37330]: osdmap e442: 8 total, 8 up, 7 in 2024-09-11T07:47:55.852 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:55 smithi022 ceph-mon[29462]: pgmap v995: 158 pgs: 158 active+clean; 6.8 GiB data, 26 GiB used, 599 GiB / 626 GiB avail; 853 B/s rd, 13 MiB/s wr, 953 op/s 2024-09-11T07:47:55.852 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:55 smithi022 ceph-mon[29462]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:47:55.852 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:55 smithi022 ceph-mon[29462]: from='client.26083 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:47:55.852 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:55 smithi022 ceph-mon[29462]: osdmap e442: 8 total, 8 up, 7 in 2024-09-11T07:47:55.871 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:55.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:55.872 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:55.973 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:56.017 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:56.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:56.018 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:56.119 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:56.159 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:56.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:56.160 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:56.260 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:56.298 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:56.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:56.299 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:56.400 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:56.423 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:56.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:56.424 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:56.525 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:56.569 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:56.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:56.570 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:56.670 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:56.707 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:56.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:56.709 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:56.803 INFO:teuthology.orchestra.run.smithi022.stderr:set pool 11 pgp_num to 16 2024-09-11T07:47:56.810 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:56.859 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:56.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:56.860 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:56.961 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:56.986 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:56.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:56.988 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:57.089 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:57.130 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:57.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:57.131 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:57.166 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:56 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3155103150' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-11T07:47:57.166 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:56 smithi022 ceph-mon[37330]: from='client.26086 ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-11T07:47:57.166 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:56 smithi022 ceph-mon[37330]: pgmap v997: 158 pgs: 158 active+clean; 6.4 GiB data, 25 GiB used, 600 GiB / 626 GiB avail; 921 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-11T07:47:57.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:56 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3155103150' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-11T07:47:57.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:56 smithi022 ceph-mon[29462]: from='client.26086 ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-11T07:47:57.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:56 smithi022 ceph-mon[29462]: pgmap v997: 158 pgs: 158 active+clean; 6.4 GiB data, 25 GiB used, 600 GiB / 626 GiB avail; 921 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-11T07:47:57.231 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:57.272 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:57.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:57.273 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:57.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:56 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3155103150' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-11T07:47:57.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:56 smithi104 ceph-mon[33507]: from='client.26086 ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-11T07:47:57.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:56 smithi104 ceph-mon[33507]: pgmap v997: 158 pgs: 158 active+clean; 6.4 GiB data, 25 GiB used, 600 GiB / 626 GiB avail; 921 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-11T07:47:57.374 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:57.424 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:57.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:57.426 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:57.527 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:57.552 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:57.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:57.553 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:57.654 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:57.678 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:57.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:57.679 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:57.780 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:47:57.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:57.822 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:57.822 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:47:57.923 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:57.968 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:57.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:57.969 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:58.070 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:58.093 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:58.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:58.094 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:58.107 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:57 smithi022 ceph-mon[37330]: from='client.26086 ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num", "val": "16"}]': finished 2024-09-11T07:47:58.107 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:57 smithi022 ceph-mon[37330]: osdmap e443: 8 total, 8 up, 7 in 2024-09-11T07:47:58.107 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:57 smithi022 ceph-mon[29462]: from='client.26086 ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num", "val": "16"}]': finished 2024-09-11T07:47:58.108 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:57 smithi022 ceph-mon[29462]: osdmap e443: 8 total, 8 up, 7 in 2024-09-11T07:47:58.195 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:47:58.220 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:58.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:58.221 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:47:58.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:57 smithi104 ceph-mon[33507]: from='client.26086 ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num", "val": "16"}]': finished 2024-09-11T07:47:58.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:57 smithi104 ceph-mon[33507]: osdmap e443: 8 total, 8 up, 7 in 2024-09-11T07:47:58.323 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:58.348 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:58.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:58.350 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:58.417 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:47:58.417 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:47:58.450 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:58.474 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:58.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:58.475 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:58.575 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:47:58.601 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:58.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:58.602 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:47:58.703 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:58.744 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:58.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:58.745 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:47:58.846 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:58.871 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:58.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:58.872 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:58.973 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:59.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:59.013 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:59.013 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:59.114 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:59.139 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:59.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:59.141 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:59.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:58 smithi022 ceph-mon[29462]: pgmap v999: 158 pgs: 158 active+clean; 6.4 GiB data, 25 GiB used, 600 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 721 op/s 2024-09-11T07:47:59.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:58 smithi022 ceph-mon[37330]: pgmap v999: 158 pgs: 158 active+clean; 6.4 GiB data, 25 GiB used, 600 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 721 op/s 2024-09-11T07:47:59.241 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:47:59.265 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:59.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:59.266 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:47:59.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:58 smithi104 ceph-mon[33507]: pgmap v999: 158 pgs: 158 active+clean; 6.4 GiB data, 25 GiB used, 600 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 721 op/s 2024-09-11T07:47:59.367 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:47:59.415 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:59.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:59.416 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:47:59.517 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:59.542 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:59.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:59.543 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:59.644 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:47:59.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:59.689 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:59.689 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:47:59.790 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:47:59.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:59.816 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:47:59.817 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:47:59.917 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:47:59.963 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:47:59.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:47:59.964 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:00.065 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:00.104 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:00.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:00.105 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:00.118 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:59 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:48:00.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:59 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:48:00.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:47:59 smithi022 ceph-mon[29462]: osdmap e444: 8 total, 8 up, 7 in 2024-09-11T07:48:00.119 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:59 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:48:00.119 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:59 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:48:00.119 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:47:59 smithi022 ceph-mon[37330]: osdmap e444: 8 total, 8 up, 7 in 2024-09-11T07:48:00.206 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:00.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:00.234 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:00.234 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:00.257 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:59 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:48:00.258 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:59 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:48:00.258 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:47:59 smithi104 ceph-mon[33507]: osdmap e444: 8 total, 8 up, 7 in 2024-09-11T07:48:00.334 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:00.360 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:00.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:00.361 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:00.462 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:00.500 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:00.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:00.501 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:00.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:48:00 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:48:00.603 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:00.641 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:00.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:00.642 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:00.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:48:00 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:48:00] "GET /metrics HTTP/1.1" 200 225101 "" "Prometheus/2.43.0" 2024-09-11T07:48:00.743 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:00.767 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:00.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:00.768 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:00.869 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:00.916 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:00.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:00.917 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:01.018 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:01.027 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:00 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:48:01.027 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:00 smithi022 ceph-mon[29462]: pgmap v1001: 158 pgs: 158 active+clean; 6.0 GiB data, 24 GiB used, 601 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.98k op/s 2024-09-11T07:48:01.027 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:00 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:48:01.027 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:00 smithi022 ceph-mon[37330]: pgmap v1001: 158 pgs: 158 active+clean; 6.0 GiB data, 24 GiB used, 601 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.98k op/s 2024-09-11T07:48:01.042 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:01.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:01.043 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:01.071 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:48:01.072 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:48:01.107 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:48:01.108 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:48:01.143 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:48:01.144 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:48:01.144 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:01.167 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:01.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:01.168 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:01.184 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:48:01.184 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:48:01.217 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:48:01.218 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:48:01.235 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:48:01.236 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:48:01.251 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:48:01.251 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:48:01.266 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:48:01.267 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:48:01.268 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:01.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:00 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:48:01.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:00 smithi104 ceph-mon[33507]: pgmap v1001: 158 pgs: 158 active+clean; 6.0 GiB data, 24 GiB used, 601 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.98k op/s 2024-09-11T07:48:01.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:01.294 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:01.294 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:01.395 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:01.436 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:01.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:01.437 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:01.538 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:01.577 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:01.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:01.578 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:01.678 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:01.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:01.704 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:01.704 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:01.805 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:01.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:01.845 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:01.845 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:01.946 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:01.982 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-11T07:48:01.994 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:01.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:01.995 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:02.096 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:02.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:02.133 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:02.133 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:02.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:01 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1415520054' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:48:02.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:01 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1415520054' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:48:02.167 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T07:48:02.234 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:02.257 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:02.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:02.258 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:02.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:01 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1415520054' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:48:02.358 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:02.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:02.381 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:02.381 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:02.482 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:02.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:02.508 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:02.509 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:02.610 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:02.636 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:02.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:02.637 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:02.738 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:02.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:02.776 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:02.776 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:02.868 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:48:02.868 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:48:02.868 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2024-09-11T07:48:02.869 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:48:02.877 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:02.900 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:02.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:02.901 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:03.002 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:03.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:03.026 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:03.027 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:03.127 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:03.148 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:03.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:03.149 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:03.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:02 smithi022 ceph-mon[29462]: pgmap v1002: 158 pgs: 158 active+clean; 6.0 GiB data, 24 GiB used, 601 GiB / 626 GiB avail; 646 B/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T07:48:03.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:02 smithi022 ceph-mon[37330]: pgmap v1002: 158 pgs: 158 active+clean; 6.0 GiB data, 24 GiB used, 601 GiB / 626 GiB avail; 646 B/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T07:48:03.250 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:03.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:02 smithi104 ceph-mon[33507]: pgmap v1002: 158 pgs: 158 active+clean; 6.0 GiB data, 24 GiB used, 601 GiB / 626 GiB avail; 646 B/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T07:48:03.292 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:03.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:03.293 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:03.394 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:03.441 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:03.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:03.442 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:03.543 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:03.577 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:03.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:03.578 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:03.679 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:03.709 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:03.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:03.710 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:03.811 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:03.835 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:03.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:03.836 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:03.937 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:03.961 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:03.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:03.962 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:04.063 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:04.103 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:04.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:04.104 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:04.204 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:04.230 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:04.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:04.231 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:04.331 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:04.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:04.360 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:04.360 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:04.461 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:04.498 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:04.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:04.499 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:04.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:48:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:48:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:48:04.599 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:04.624 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:04.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:04.625 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:04.726 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:04.769 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:04.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:04.770 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:04.871 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:04.893 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:04.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:04.894 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:04.994 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:05.017 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:05.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:05.018 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:05.119 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:05.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:05.154 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:05.154 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:05.255 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:05.281 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:05.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:05.282 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:05.383 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:05.406 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:05.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:05.407 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:05.508 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:05.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:05.552 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:05.653 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:05.733 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:05.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:05.734 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:05.779 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:48:05.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:05 smithi104 ceph-mon[33507]: pgmap v1003: 158 pgs: 158 active+clean; 6.0 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 764 op/s 2024-09-11T07:48:05.792 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:48:05.796 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:48:05.835 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:05.858 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:05.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:05.859 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:05.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:05 smithi022 ceph-mon[29462]: pgmap v1003: 158 pgs: 158 active+clean; 6.0 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 764 op/s 2024-09-11T07:48:05.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:05 smithi022 ceph-mon[37330]: pgmap v1003: 158 pgs: 158 active+clean; 6.0 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 764 op/s 2024-09-11T07:48:05.960 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:06.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:06.009 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:06.009 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:06.110 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:06.134 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:06.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:06.135 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:06.236 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:06.280 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:06.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:06.281 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:06.382 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:06.405 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:06.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:06.406 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:06.506 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:06.546 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:06.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:06.547 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:06.648 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:06.687 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:06.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:06.688 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:06.789 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:06.814 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:06.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:06.815 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:06.916 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:06.941 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:06.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:06.942 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:07.042 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:07.070 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:07.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:07.071 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:07.172 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:07.220 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:07.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:07.221 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:07.322 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:07.346 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:07.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:07.347 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:07.448 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:07.474 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:07.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:07.476 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:07.531 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:48:07.577 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:07.626 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:07.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:07.628 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:07.728 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:07.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:07.761 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:07.762 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:07.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:07 smithi104 ceph-mon[33507]: pgmap v1004: 158 pgs: 158 active+clean; 5.7 GiB data, 23 GiB used, 602 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-09-11T07:48:07.862 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:07.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:07.884 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:07.885 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:07.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:07 smithi022 ceph-mon[29462]: pgmap v1004: 158 pgs: 158 active+clean; 5.7 GiB data, 23 GiB used, 602 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-09-11T07:48:07.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:07 smithi022 ceph-mon[37330]: pgmap v1004: 158 pgs: 158 active+clean; 5.7 GiB data, 23 GiB used, 602 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-09-11T07:48:07.985 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:08.007 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:08.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:08.008 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:08.109 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:08.133 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:08.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:08.134 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:08.235 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:08.260 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:08.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:08.261 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:08.362 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:08.383 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:08.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:08.384 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:08.485 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:08.506 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:08.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:08.507 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:08.608 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:08.635 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:08.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:08.636 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:08.737 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:08.759 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:08.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:08.760 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:08.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:48:08.861 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:08.886 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:08.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:08.887 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:08.912 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:48:08.912 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:48:08.987 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:09.012 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:09.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:09.013 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:09.113 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:09.154 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:09.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:09.155 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:09.256 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:09.296 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:09.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:09.298 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:09.398 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:09.467 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:09.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:09.468 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:09.521 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:09 smithi022 ceph-mon[29462]: pgmap v1005: 158 pgs: 158 active+clean; 5.7 GiB data, 23 GiB used, 602 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-11T07:48:09.569 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:09.611 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:09.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:09.612 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:09.713 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:09.738 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:09.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:09.739 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:09.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:09 smithi104 ceph-mon[33507]: pgmap v1005: 158 pgs: 158 active+clean; 5.7 GiB data, 23 GiB used, 602 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-11T07:48:09.836 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:09 smithi022 ceph-mon[37330]: pgmap v1005: 158 pgs: 158 active+clean; 5.7 GiB data, 23 GiB used, 602 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-11T07:48:09.840 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:09.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:09.876 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:09.876 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:09.977 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:10.021 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:10.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:10.022 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:10.122 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:10.147 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:10.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:10.148 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:10.248 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:10.273 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:10.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:10.274 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:10.375 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:10.417 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:48:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:48:10] "GET /metrics HTTP/1.1" 200 225121 "" "Prometheus/2.43.0" 2024-09-11T07:48:10.420 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:10.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:10.421 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:10.522 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:10.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:48:10 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:48:10.549 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:10.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:10.550 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:10.651 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:10.689 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:10.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:10.690 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:10.791 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:10.816 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:10.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:10.817 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:10.918 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:10.943 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:10.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:10.944 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:11.045 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:11.068 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:11.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:11.069 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:11.170 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:11.194 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:11.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:11.195 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:11.296 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:11.322 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:11.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:11.323 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:11.424 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:11.455 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:11.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:11.456 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:11.557 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:11.578 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:11.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:11.579 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:11.679 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:11.707 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:11.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:11.708 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:11.809 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:11.848 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:11.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:11.849 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:11.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:11 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:48:11.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:11 smithi022 ceph-mon[29462]: pgmap v1006: 158 pgs: 158 active+clean; 5.4 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.42k op/s 2024-09-11T07:48:11.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:11 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:48:11.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:11 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:48:11.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:11 smithi022 ceph-mon[29462]: osdmap e445: 8 total, 8 up, 7 in 2024-09-11T07:48:11.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:11 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:48:11.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:11 smithi022 ceph-mon[37330]: pgmap v1006: 158 pgs: 158 active+clean; 5.4 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.42k op/s 2024-09-11T07:48:11.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:11 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:48:11.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:11 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:48:11.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:11 smithi022 ceph-mon[37330]: osdmap e445: 8 total, 8 up, 7 in 2024-09-11T07:48:11.951 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:11.990 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:11.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:11.991 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:12.028 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:48:12.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:11 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:48:12.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:11 smithi104 ceph-mon[33507]: pgmap v1006: 158 pgs: 158 active+clean; 5.4 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.42k op/s 2024-09-11T07:48:12.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:11 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:48:12.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:11 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:48:12.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:11 smithi104 ceph-mon[33507]: osdmap e445: 8 total, 8 up, 7 in 2024-09-11T07:48:12.093 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:12.142 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:12.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:12.144 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:12.244 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:12.271 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:12.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:12.273 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:12.373 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:12.408 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:12.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:12.409 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:12.510 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:12.549 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:12.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:12.550 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:12.551 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is unset 2024-09-11T07:48:12.651 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:12.677 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:12.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:12.678 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:12.778 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:12.818 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:12.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:12.820 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:12.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:12 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/146985714' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:48:12.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:12 smithi022 ceph-mon[29462]: from='client.26098 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:48:12.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:12 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/146985714' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:48:12.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:12 smithi022 ceph-mon[37330]: from='client.26098 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:48:12.922 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:12.946 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:12.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:12.947 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:12.963 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:48:12.967 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:48:13.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:12 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/146985714' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:48:13.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:12 smithi104 ceph-mon[33507]: from='client.26098 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:48:13.049 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:13.073 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:13.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:13.074 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:13.175 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:13.216 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:13.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:13.218 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:13.319 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:13.361 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:13.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:13.362 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:13.463 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:13.505 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:13.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:13.507 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:13.523 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:48:13.523 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 20, 2024-09-11T07:48:13.523 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 600, 2024-09-11T07:48:13.523 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:48:13.523 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.523 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235127 9.0 9:01e55b17:::benchmark_data_smithi022_133943_object108536:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e321)", 2024-09-11T07:48:13.523 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.667941+0000", 2024-09-11T07:48:13.523 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 431.85105737599997, 2024-09-11T07:48:13.523 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0432926819999999, 2024-09-11T07:48:13.523 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:48:13.524 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:48:13.524 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:48:13.524 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:48:13.524 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:48:13.524 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235127 2024-09-11T07:48:13.524 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.524 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:48:13.524 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.524 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:48:13.524 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.667941+0000", 2024-09-11T07:48:13.524 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.525 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.525 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.525 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:48:13.525 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.667941+0000", 2024-09-11T07:48:13.525 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.525 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.525 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.525 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:48:13.525 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.667939+0000", 2024-09-11T07:48:13.525 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:48:13.525 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.525 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.525 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:48:13.525 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.667947+0000", 2024-09-11T07:48:13.526 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3690000000000007e-06 2024-09-11T07:48:13.526 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.526 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.526 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:48:13.526 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.667963+0000", 2024-09-11T07:48:13.526 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.6305999999999999e-05 2024-09-11T07:48:13.526 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.526 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.527 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:48:13.527 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.667969+0000", 2024-09-11T07:48:13.527 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8619999999999999e-06 2024-09-11T07:48:13.527 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.527 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.527 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.527 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.668109+0000", 2024-09-11T07:48:13.527 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014013900000000001 2024-09-11T07:48:13.527 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.527 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.527 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:48:13.527 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.668418+0000", 2024-09-11T07:48:13.528 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000308657 2024-09-11T07:48:13.528 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.528 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.528 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,1,6", 2024-09-11T07:48:13.528 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.669377+0000", 2024-09-11T07:48:13.528 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00095886099999999998 2024-09-11T07:48:13.528 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.528 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.528 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:48:13.528 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.679949+0000", 2024-09-11T07:48:13.529 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.010572125 2024-09-11T07:48:13.529 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.529 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.529 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.529 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.682612+0000", 2024-09-11T07:48:13.529 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0026625070000000002 2024-09-11T07:48:13.529 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.529 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.529 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:48:13.529 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.682635+0000", 2024-09-11T07:48:13.529 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3396000000000001e-05 2024-09-11T07:48:13.529 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.529 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.530 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.530 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.711146+0000", 2024-09-11T07:48:13.530 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0285110040000001 2024-09-11T07:48:13.530 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.530 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.530 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:48:13.530 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.711234+0000", 2024-09-11T07:48:13.530 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.8091999999999996e-05 2024-09-11T07:48:13.530 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.530 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:48:13.530 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.530 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.531 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.531 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235175 9.0 9:077fcbba:::benchmark_data_smithi022_133943_object108584:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-11T07:48:13.531 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.720021+0000", 2024-09-11T07:48:13.531 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 431.79897819500002, 2024-09-11T07:48:13.531 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.99970432600000003, 2024-09-11T07:48:13.531 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:48:13.531 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:48:13.531 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:48:13.531 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:48:13.531 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:48:13.531 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235175 2024-09-11T07:48:13.531 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.531 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:48:13.531 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.532 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:48:13.532 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.720021+0000", 2024-09-11T07:48:13.532 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.532 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.532 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.532 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:48:13.532 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.720021+0000", 2024-09-11T07:48:13.532 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.532 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.532 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.532 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:48:13.533 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.720018+0000", 2024-09-11T07:48:13.533 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:48:13.533 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.533 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.533 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:48:13.533 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.720027+0000", 2024-09-11T07:48:13.533 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.4780000000000008e-06 2024-09-11T07:48:13.533 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.533 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.533 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:48:13.534 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.720044+0000", 2024-09-11T07:48:13.534 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.7180000000000002e-05 2024-09-11T07:48:13.534 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.534 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.534 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:48:13.534 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.720051+0000", 2024-09-11T07:48:13.534 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.503e-06 2024-09-11T07:48:13.534 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.534 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.534 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.534 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.720097+0000", 2024-09-11T07:48:13.534 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5689000000000003e-05 2024-09-11T07:48:13.534 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.534 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.535 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:48:13.535 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.720105+0000", 2024-09-11T07:48:13.535 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.9000000000000006e-06 2024-09-11T07:48:13.535 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.535 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.535 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.535 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.711744+0000", 2024-09-11T07:48:13.535 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.99163922400000004 2024-09-11T07:48:13.535 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.535 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.535 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:48:13.535 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.711839+0000", 2024-09-11T07:48:13.535 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.5249999999999998e-05 2024-09-11T07:48:13.535 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.536 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.536 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:48:13.536 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.712333+0000", 2024-09-11T07:48:13.536 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00049357399999999999 2024-09-11T07:48:13.536 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.536 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.536 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.536 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.719518+0000", 2024-09-11T07:48:13.536 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0071850860000000003 2024-09-11T07:48:13.536 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.536 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.536 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:48:13.537 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.719600+0000", 2024-09-11T07:48:13.537 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.2018e-05 2024-09-11T07:48:13.537 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.537 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.537 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.537 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.719637+0000", 2024-09-11T07:48:13.537 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6650999999999999e-05 2024-09-11T07:48:13.537 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.537 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.537 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:48:13.537 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.719684+0000", 2024-09-11T07:48:13.537 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7289000000000001e-05 2024-09-11T07:48:13.537 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.537 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.538 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:48:13.538 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.719725+0000", 2024-09-11T07:48:13.538 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0952999999999999e-05 2024-09-11T07:48:13.538 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.538 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:48:13.538 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.538 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.538 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.538 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235239 9.0 9:01129c16:::benchmark_data_smithi022_133943_object108648:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-11T07:48:13.538 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.783885+0000", 2024-09-11T07:48:13.539 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 431.73511343199999, 2024-09-11T07:48:13.539 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.94223383800000005, 2024-09-11T07:48:13.539 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:48:13.539 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:48:13.539 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:48:13.539 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:48:13.539 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:48:13.539 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235239 2024-09-11T07:48:13.539 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.540 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:48:13.540 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.540 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:48:13.540 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.783885+0000", 2024-09-11T07:48:13.540 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.540 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.540 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.540 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:48:13.540 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.783885+0000", 2024-09-11T07:48:13.540 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.540 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.540 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.541 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:48:13.541 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.783883+0000", 2024-09-11T07:48:13.541 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:48:13.541 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.541 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.541 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:48:13.541 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.783892+0000", 2024-09-11T07:48:13.541 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.6150000000000003e-06 2024-09-11T07:48:13.541 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.541 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.541 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:48:13.541 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.783911+0000", 2024-09-11T07:48:13.541 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8561000000000001e-05 2024-09-11T07:48:13.542 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.542 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.542 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:48:13.542 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.783920+0000", 2024-09-11T07:48:13.542 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.0110000000000006e-06 2024-09-11T07:48:13.542 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.542 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.542 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.542 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.784064+0000", 2024-09-11T07:48:13.542 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000144346 2024-09-11T07:48:13.542 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.542 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.542 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:48:13.543 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.784088+0000", 2024-09-11T07:48:13.543 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3830999999999999e-05 2024-09-11T07:48:13.543 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.543 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.543 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.543 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.714582+0000", 2024-09-11T07:48:13.543 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.93049353499999998 2024-09-11T07:48:13.543 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.543 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.543 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:48:13.543 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.714782+0000", 2024-09-11T07:48:13.543 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00019994000000000001 2024-09-11T07:48:13.543 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.544 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.544 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:48:13.544 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.715188+0000", 2024-09-11T07:48:13.544 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00040689900000000001 2024-09-11T07:48:13.544 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.544 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.544 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:48:13.544 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.719605+0000", 2024-09-11T07:48:13.544 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0044166520000000001 2024-09-11T07:48:13.544 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.544 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.544 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.544 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.719760+0000", 2024-09-11T07:48:13.545 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015526900000000001 2024-09-11T07:48:13.545 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.545 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.545 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.545 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726015+0000", 2024-09-11T07:48:13.545 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0062551669999999998 2024-09-11T07:48:13.545 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.545 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.545 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:48:13.545 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726075+0000", 2024-09-11T07:48:13.545 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9857e-05 2024-09-11T07:48:13.545 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.545 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.545 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:48:13.546 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726119+0000", 2024-09-11T07:48:13.546 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3884000000000002e-05 2024-09-11T07:48:13.546 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.546 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:48:13.546 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.546 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.546 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.546 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235253 9.0 9:03cfe51f:::benchmark_data_smithi022_133943_object108662:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-11T07:48:13.546 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.795378+0000", 2024-09-11T07:48:13.546 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 431.72362106100002, 2024-09-11T07:48:13.546 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.93095434300000002, 2024-09-11T07:48:13.546 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:48:13.546 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:48:13.546 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:48:13.547 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:48:13.547 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:48:13.547 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235253 2024-09-11T07:48:13.547 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.547 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:48:13.547 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.547 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:48:13.547 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795378+0000", 2024-09-11T07:48:13.547 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.547 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.547 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.547 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:48:13.547 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795378+0000", 2024-09-11T07:48:13.548 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.548 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.548 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.548 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:48:13.548 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795375+0000", 2024-09-11T07:48:13.548 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:48:13.548 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.548 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.548 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:48:13.549 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795385+0000", 2024-09-11T07:48:13.549 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.9019999999999994e-06 2024-09-11T07:48:13.549 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.549 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.549 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:48:13.549 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795403+0000", 2024-09-11T07:48:13.549 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8428999999999999e-05 2024-09-11T07:48:13.549 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.549 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.549 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:48:13.549 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795410+0000", 2024-09-11T07:48:13.549 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.742e-06 2024-09-11T07:48:13.549 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.550 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.550 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.550 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795457+0000", 2024-09-11T07:48:13.550 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7469999999999998e-05 2024-09-11T07:48:13.550 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.550 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.550 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:48:13.550 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795466+0000", 2024-09-11T07:48:13.550 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.8829999999999995e-06 2024-09-11T07:48:13.550 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.550 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.550 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.550 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.717205+0000", 2024-09-11T07:48:13.551 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.92173829900000004 2024-09-11T07:48:13.551 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.551 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.551 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:48:13.551 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.717345+0000", 2024-09-11T07:48:13.551 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014056800000000001 2024-09-11T07:48:13.551 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.551 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.551 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:48:13.551 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.717626+0000", 2024-09-11T07:48:13.551 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00028078599999999999 2024-09-11T07:48:13.551 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.551 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.552 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:48:13.552 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.725938+0000", 2024-09-11T07:48:13.552 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0083124600000000007 2024-09-11T07:48:13.552 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.552 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.552 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.552 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.725977+0000", 2024-09-11T07:48:13.552 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8679e-05 2024-09-11T07:48:13.552 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.552 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.552 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.552 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726219+0000", 2024-09-11T07:48:13.552 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00024190600000000001 2024-09-11T07:48:13.553 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.553 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.553 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:48:13.553 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726290+0000", 2024-09-11T07:48:13.553 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1551000000000007e-05 2024-09-11T07:48:13.553 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.553 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.553 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:48:13.553 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726332+0000", 2024-09-11T07:48:13.553 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1641000000000003e-05 2024-09-11T07:48:13.553 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.553 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:48:13.553 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.554 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.554 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.554 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235255 9.0 9:06894acd:::benchmark_data_smithi022_133943_object108664:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-11T07:48:13.554 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.795871+0000", 2024-09-11T07:48:13.554 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 431.72312768299997, 2024-09-11T07:48:13.554 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.93060655199999998, 2024-09-11T07:48:13.554 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:48:13.554 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:48:13.554 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:48:13.554 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:48:13.554 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:48:13.554 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235255 2024-09-11T07:48:13.554 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.555 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:48:13.555 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.555 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:48:13.555 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795871+0000", 2024-09-11T07:48:13.555 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.555 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.555 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.555 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:48:13.555 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795871+0000", 2024-09-11T07:48:13.555 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.555 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.555 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.555 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:48:13.556 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795870+0000", 2024-09-11T07:48:13.556 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:48:13.556 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.556 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.556 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:48:13.556 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795876+0000", 2024-09-11T07:48:13.556 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.993e-06 2024-09-11T07:48:13.556 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.556 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.556 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:48:13.556 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795886+0000", 2024-09-11T07:48:13.556 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0259e-05 2024-09-11T07:48:13.556 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.557 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.557 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:48:13.557 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795892+0000", 2024-09-11T07:48:13.557 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3110000000000002e-06 2024-09-11T07:48:13.557 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.557 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.557 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.557 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795933+0000", 2024-09-11T07:48:13.557 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0852000000000001e-05 2024-09-11T07:48:13.557 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.557 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.557 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:48:13.557 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.795940+0000", 2024-09-11T07:48:13.557 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0199999999999997e-06 2024-09-11T07:48:13.558 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.558 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.558 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.558 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.719781+0000", 2024-09-11T07:48:13.558 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.92384101299999999 2024-09-11T07:48:13.558 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.558 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.558 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:48:13.558 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.719886+0000", 2024-09-11T07:48:13.558 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000105339 2024-09-11T07:48:13.558 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.558 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.558 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:48:13.559 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.720315+0000", 2024-09-11T07:48:13.559 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00042835200000000001 2024-09-11T07:48:13.559 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.559 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.559 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:48:13.559 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.725983+0000", 2024-09-11T07:48:13.559 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0056682360000000001 2024-09-11T07:48:13.559 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.559 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.559 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.559 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726182+0000", 2024-09-11T07:48:13.559 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00019910500000000001 2024-09-11T07:48:13.560 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.560 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.560 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.560 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726408+0000", 2024-09-11T07:48:13.560 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000225644 2024-09-11T07:48:13.560 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.560 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.560 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:48:13.560 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726446+0000", 2024-09-11T07:48:13.560 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8281000000000002e-05 2024-09-11T07:48:13.560 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.560 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.560 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:48:13.560 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726478+0000", 2024-09-11T07:48:13.561 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.1622000000000003e-05 2024-09-11T07:48:13.561 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.561 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:48:13.561 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.561 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.561 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.561 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:254717 10.7 10:e4a41305:::benchmark_data_smithi022_163866_object125291:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e398)", 2024-09-11T07:48:13.561 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:45:30.774561+0000", 2024-09-11T07:48:13.561 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 162.74443822399999, 2024-09-11T07:48:13.561 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0181760129999999, 2024-09-11T07:48:13.561 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:48:13.561 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:48:13.561 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:48:13.562 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:48:13.562 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:48:13.562 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 254717 2024-09-11T07:48:13.562 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.562 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:48:13.562 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.562 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:48:13.562 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.774561+0000", 2024-09-11T07:48:13.562 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.562 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.562 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.562 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:48:13.562 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.774561+0000", 2024-09-11T07:48:13.562 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.563 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.563 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.563 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:48:13.563 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.774558+0000", 2024-09-11T07:48:13.563 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:48:13.563 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.563 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.563 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:48:13.563 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.774568+0000", 2024-09-11T07:48:13.563 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.7000000000000003e-06 2024-09-11T07:48:13.563 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.563 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.563 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:48:13.563 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.774585+0000", 2024-09-11T07:48:13.564 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.6657e-05 2024-09-11T07:48:13.564 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.564 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.564 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:48:13.564 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.774591+0000", 2024-09-11T07:48:13.564 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2310000000000001e-06 2024-09-11T07:48:13.564 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.564 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.564 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.564 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.774642+0000", 2024-09-11T07:48:13.564 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.0704000000000001e-05 2024-09-11T07:48:13.564 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.564 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.565 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:48:13.565 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.774650+0000", 2024-09-11T07:48:13.565 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.0360000000000006e-06 2024-09-11T07:48:13.565 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.565 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.565 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.565 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.778135+0000", 2024-09-11T07:48:13.565 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.003485151 2024-09-11T07:48:13.565 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.565 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.565 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:48:13.565 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.778229+0000", 2024-09-11T07:48:13.566 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.4659000000000003e-05 2024-09-11T07:48:13.566 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.566 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.566 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:48:13.566 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.778331+0000", 2024-09-11T07:48:13.566 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010186799999999999 2024-09-11T07:48:13.566 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.566 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.566 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:48:13.566 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.783112+0000", 2024-09-11T07:48:13.566 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0047804299999999996 2024-09-11T07:48:13.566 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.566 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.567 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.567 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.783142+0000", 2024-09-11T07:48:13.567 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0808e-05 2024-09-11T07:48:13.567 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.567 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.567 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.567 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.792653+0000", 2024-09-11T07:48:13.567 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0095102869999999992 2024-09-11T07:48:13.567 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.567 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.568 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:48:13.568 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.792694+0000", 2024-09-11T07:48:13.568 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1473000000000001e-05 2024-09-11T07:48:13.568 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.568 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.568 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:48:13.568 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.792737+0000", 2024-09-11T07:48:13.568 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2401999999999998e-05 2024-09-11T07:48:13.568 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.568 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:48:13.568 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.568 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.569 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.569 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:254745 10.7 10:e5c45d39:::benchmark_data_smithi022_163866_object125319:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e398)", 2024-09-11T07:48:13.569 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:45:30.801138+0000", 2024-09-11T07:48:13.569 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 162.717860589, 2024-09-11T07:48:13.570 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.99176827000000001, 2024-09-11T07:48:13.570 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:48:13.570 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:48:13.570 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:48:13.570 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:48:13.570 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:48:13.570 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 254745 2024-09-11T07:48:13.570 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.570 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:48:13.570 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.570 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:48:13.570 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.801138+0000", 2024-09-11T07:48:13.570 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.571 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.571 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.571 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:48:13.571 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.801138+0000", 2024-09-11T07:48:13.571 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.571 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.571 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.571 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:48:13.571 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.801137+0000", 2024-09-11T07:48:13.571 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:48:13.571 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.571 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.571 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:48:13.572 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.801142+0000", 2024-09-11T07:48:13.572 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.5010000000000004e-06 2024-09-11T07:48:13.572 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.572 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.572 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:48:13.572 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.801150+0000", 2024-09-11T07:48:13.572 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3070000000000003e-06 2024-09-11T07:48:13.572 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.572 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.572 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:48:13.572 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.801155+0000", 2024-09-11T07:48:13.572 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0990000000000001e-06 2024-09-11T07:48:13.573 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.573 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.573 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.573 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.801190+0000", 2024-09-11T07:48:13.573 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5883000000000002e-05 2024-09-11T07:48:13.573 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.573 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.573 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:48:13.573 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.801196+0000", 2024-09-11T07:48:13.573 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8660000000000002e-06 2024-09-11T07:48:13.573 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.574 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.574 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.574 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.780912+0000", 2024-09-11T07:48:13.574 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.97971564499999997 2024-09-11T07:48:13.574 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.574 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.574 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:48:13.574 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.780999+0000", 2024-09-11T07:48:13.574 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.7535999999999997e-05 2024-09-11T07:48:13.574 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.574 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.574 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:48:13.574 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.781098+0000", 2024-09-11T07:48:13.575 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.8868000000000003e-05 2024-09-11T07:48:13.575 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.575 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.575 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.575 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.792573+0000", 2024-09-11T07:48:13.575 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.011474362 2024-09-11T07:48:13.575 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.575 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.575 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:48:13.575 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.792579+0000", 2024-09-11T07:48:13.575 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2400000000000004e-06 2024-09-11T07:48:13.575 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.575 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.576 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.576 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.792823+0000", 2024-09-11T07:48:13.576 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000243599 2024-09-11T07:48:13.576 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.576 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.576 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:48:13.576 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.792873+0000", 2024-09-11T07:48:13.576 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.0408999999999998e-05 2024-09-11T07:48:13.576 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.576 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.576 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:48:13.576 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.792907+0000", 2024-09-11T07:48:13.577 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3544999999999998e-05 2024-09-11T07:48:13.577 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.577 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:48:13.577 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.577 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.577 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.577 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:254775 10.7 10:e5042b3f:::benchmark_data_smithi022_163866_object125349:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e398)", 2024-09-11T07:48:13.577 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:45:30.821882+0000", 2024-09-11T07:48:13.577 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 162.69711697700001, 2024-09-11T07:48:13.577 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.97116269, 2024-09-11T07:48:13.578 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:48:13.578 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:48:13.578 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:48:13.578 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:48:13.578 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:48:13.578 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 254775 2024-09-11T07:48:13.578 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.578 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:48:13.578 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.578 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:48:13.578 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.821882+0000", 2024-09-11T07:48:13.578 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.578 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.579 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.579 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:48:13.579 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.821882+0000", 2024-09-11T07:48:13.579 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.579 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.579 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.579 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:48:13.579 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.821880+0000", 2024-09-11T07:48:13.579 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:48:13.579 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.579 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.579 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:48:13.579 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.821887+0000", 2024-09-11T07:48:13.580 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1210000000000001e-06 2024-09-11T07:48:13.580 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.580 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.580 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:48:13.580 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.821899+0000", 2024-09-11T07:48:13.580 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1467e-05 2024-09-11T07:48:13.580 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.580 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.580 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:48:13.580 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.821904+0000", 2024-09-11T07:48:13.580 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.541e-06 2024-09-11T07:48:13.580 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.580 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.581 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.581 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.821942+0000", 2024-09-11T07:48:13.581 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8328000000000003e-05 2024-09-11T07:48:13.581 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.581 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.581 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:48:13.581 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.821970+0000", 2024-09-11T07:48:13.581 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7250000000000002e-05 2024-09-11T07:48:13.581 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.581 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.581 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.582 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.783291+0000", 2024-09-11T07:48:13.582 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.96132167199999996 2024-09-11T07:48:13.582 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.582 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.582 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:48:13.582 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.783386+0000", 2024-09-11T07:48:13.582 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.4310999999999999e-05 2024-09-11T07:48:13.582 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.582 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.582 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:48:13.582 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.783482+0000", 2024-09-11T07:48:13.582 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.6467999999999999e-05 2024-09-11T07:48:13.582 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.583 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.583 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:48:13.583 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.792613+0000", 2024-09-11T07:48:13.583 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0091308440000000008 2024-09-11T07:48:13.583 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.583 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.583 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.583 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.792772+0000", 2024-09-11T07:48:13.583 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015932500000000001 2024-09-11T07:48:13.583 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.583 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.584 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.584 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.792972+0000", 2024-09-11T07:48:13.584 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00019977600000000001 2024-09-11T07:48:13.584 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.584 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.584 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:48:13.584 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.793014+0000", 2024-09-11T07:48:13.584 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2042000000000002e-05 2024-09-11T07:48:13.584 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.584 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.584 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:48:13.585 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.793045+0000", 2024-09-11T07:48:13.585 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0465000000000001e-05 2024-09-11T07:48:13.585 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.585 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:48:13.585 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.585 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.585 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.585 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:254792 10.7 10:e0187e1a:::benchmark_data_smithi022_163866_object125366:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e398)", 2024-09-11T07:48:13.585 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:45:30.836152+0000", 2024-09-11T07:48:13.585 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 162.68284641400001, 2024-09-11T07:48:13.585 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.95702569299999996, 2024-09-11T07:48:13.585 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:48:13.586 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:48:13.586 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:48:13.586 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:48:13.586 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:48:13.586 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 254792 2024-09-11T07:48:13.586 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.586 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:48:13.586 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.586 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:48:13.586 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.836152+0000", 2024-09-11T07:48:13.586 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.586 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.586 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.587 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:48:13.587 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.836152+0000", 2024-09-11T07:48:13.587 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.587 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.587 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.587 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:48:13.587 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.836151+0000", 2024-09-11T07:48:13.587 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:48:13.587 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.587 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.587 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:48:13.587 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.836158+0000", 2024-09-11T07:48:13.587 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.8379999999999996e-06 2024-09-11T07:48:13.587 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.588 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.588 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:48:13.588 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.836168+0000", 2024-09-11T07:48:13.588 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0210999999999999e-05 2024-09-11T07:48:13.588 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.588 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.588 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:48:13.588 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.836174+0000", 2024-09-11T07:48:13.588 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3350000000000003e-06 2024-09-11T07:48:13.588 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.588 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.588 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.589 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.836225+0000", 2024-09-11T07:48:13.589 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.0569999999999999e-05 2024-09-11T07:48:13.589 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.589 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.589 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:48:13.589 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.836234+0000", 2024-09-11T07:48:13.589 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.2760000000000006e-06 2024-09-11T07:48:13.589 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.589 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.589 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.589 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.785583+0000", 2024-09-11T07:48:13.589 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.94934929999999995 2024-09-11T07:48:13.589 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.590 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.590 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:48:13.590 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.785681+0000", 2024-09-11T07:48:13.590 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.7553000000000006e-05 2024-09-11T07:48:13.590 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.590 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.590 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:48:13.590 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.785790+0000", 2024-09-11T07:48:13.590 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010938999999999999 2024-09-11T07:48:13.590 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.590 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.590 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:48:13.590 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.792618+0000", 2024-09-11T07:48:13.590 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0068281590000000003 2024-09-11T07:48:13.591 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.591 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.591 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.591 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.792945+0000", 2024-09-11T07:48:13.591 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000326905 2024-09-11T07:48:13.591 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.591 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.591 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.591 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.793111+0000", 2024-09-11T07:48:13.591 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00016544800000000001 2024-09-11T07:48:13.591 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.591 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.591 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:48:13.592 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.793149+0000", 2024-09-11T07:48:13.592 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7948000000000003e-05 2024-09-11T07:48:13.592 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.592 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.592 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:48:13.592 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.793178+0000", 2024-09-11T07:48:13.592 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.9505e-05 2024-09-11T07:48:13.592 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.592 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:48:13.592 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.592 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.592 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.593 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25706.0:254807 10.7 10:e2b08f30:::benchmark_data_smithi022_163866_object125381:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e398)", 2024-09-11T07:48:13.593 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:45:30.872564+0000", 2024-09-11T07:48:13.593 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 162.64643506199999, 2024-09-11T07:48:13.593 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.92166003200000002, 2024-09-11T07:48:13.593 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:48:13.593 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:48:13.594 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:48:13.594 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25706", 2024-09-11T07:48:13.594 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:48:13.594 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 254807 2024-09-11T07:48:13.594 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.594 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:48:13.594 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.594 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:48:13.594 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.872564+0000", 2024-09-11T07:48:13.594 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.594 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.595 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.595 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:48:13.595 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.872564+0000", 2024-09-11T07:48:13.595 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.595 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.595 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.595 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:48:13.595 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.872562+0000", 2024-09-11T07:48:13.595 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:48:13.595 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.595 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.595 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:48:13.595 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.872568+0000", 2024-09-11T07:48:13.595 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7919999999999998e-06 2024-09-11T07:48:13.596 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.596 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.596 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:48:13.596 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.872578+0000", 2024-09-11T07:48:13.596 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0295999999999999e-05 2024-09-11T07:48:13.596 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.596 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.596 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:48:13.596 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.872585+0000", 2024-09-11T07:48:13.596 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5930000000000002e-06 2024-09-11T07:48:13.596 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.596 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.596 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.597 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.872620+0000", 2024-09-11T07:48:13.597 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5441000000000002e-05 2024-09-11T07:48:13.597 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.597 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.597 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:48:13.597 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:30.872626+0000", 2024-09-11T07:48:13.597 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3589999999999999e-06 2024-09-11T07:48:13.597 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.597 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.597 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.597 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.787828+0000", 2024-09-11T07:48:13.597 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.91520183399999999 2024-09-11T07:48:13.597 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.598 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.598 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:48:13.598 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.787939+0000", 2024-09-11T07:48:13.598 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000111372 2024-09-11T07:48:13.598 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.598 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.598 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:48:13.598 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.788047+0000", 2024-09-11T07:48:13.598 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000108393 2024-09-11T07:48:13.598 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.598 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.599 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:48:13.599 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.792778+0000", 2024-09-11T07:48:13.599 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.004730878 2024-09-11T07:48:13.599 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.599 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.599 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.599 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.793081+0000", 2024-09-11T07:48:13.599 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00030250699999999998 2024-09-11T07:48:13.599 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.599 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.599 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.599 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.793338+0000", 2024-09-11T07:48:13.599 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00025696300000000003 2024-09-11T07:48:13.599 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.600 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.600 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:48:13.600 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.793377+0000", 2024-09-11T07:48:13.600 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9308999999999999e-05 2024-09-11T07:48:13.600 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.600 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.600 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:48:13.600 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:45:31.794224+0000", 2024-09-11T07:48:13.600 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00084683399999999998 2024-09-11T07:48:13.600 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.601 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:48:13.601 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.601 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.601 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.601 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:36997 11.3 11:c38c9363:::benchmark_data_smithi022_188626_object36996:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e424)", 2024-09-11T07:48:13.601 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:46:30.506555+0000", 2024-09-11T07:48:13.601 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 103.012443969, 2024-09-11T07:48:13.601 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.99941670299999996, 2024-09-11T07:48:13.601 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:48:13.601 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:48:13.601 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:48:13.601 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:48:13.601 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:48:13.602 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 36997 2024-09-11T07:48:13.602 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.602 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:48:13.602 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.602 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:48:13.602 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.506555+0000", 2024-09-11T07:48:13.602 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.602 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.602 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.602 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:48:13.602 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.506555+0000", 2024-09-11T07:48:13.602 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.602 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.603 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.603 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:48:13.603 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.506553+0000", 2024-09-11T07:48:13.603 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:48:13.603 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.603 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.603 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:48:13.603 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.506581+0000", 2024-09-11T07:48:13.603 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8445999999999998e-05 2024-09-11T07:48:13.603 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.603 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.603 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:48:13.603 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.506591+0000", 2024-09-11T07:48:13.603 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0069e-05 2024-09-11T07:48:13.604 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.604 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.604 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:48:13.604 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.506597+0000", 2024-09-11T07:48:13.604 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3709999999999999e-06 2024-09-11T07:48:13.604 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.604 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.604 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.604 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.507556+0000", 2024-09-11T07:48:13.604 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00095968699999999997 2024-09-11T07:48:13.604 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.604 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.604 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:48:13.605 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.507566+0000", 2024-09-11T07:48:13.605 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.9799999999999993e-06 2024-09-11T07:48:13.605 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.605 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.605 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.605 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.505418+0000", 2024-09-11T07:48:13.605 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.99785206900000001 2024-09-11T07:48:13.605 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.605 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.605 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:48:13.605 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.505972+0000", 2024-09-11T07:48:13.606 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000553124 2024-09-11T07:48:13.606 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.606 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:48:13.606 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.606 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.606 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.606 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:37025 11.3 11:c3d7bb5e:::benchmark_data_smithi022_188626_object37024:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e424)", 2024-09-11T07:48:13.607 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:46:30.515144+0000", 2024-09-11T07:48:13.607 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 103.003855286, 2024-09-11T07:48:13.607 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.99553335200000004, 2024-09-11T07:48:13.607 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:48:13.607 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:48:13.607 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:48:13.607 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:48:13.607 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:48:13.607 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 37025 2024-09-11T07:48:13.608 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.608 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:48:13.608 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.608 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:48:13.608 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.515144+0000", 2024-09-11T07:48:13.608 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.608 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.608 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.608 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:48:13.608 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.515144+0000", 2024-09-11T07:48:13.609 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.609 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.609 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.609 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:48:13.609 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.515142+0000", 2024-09-11T07:48:13.609 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:48:13.609 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.609 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.610 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:48:13.610 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.515168+0000", 2024-09-11T07:48:13.610 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5829999999999998e-05 2024-09-11T07:48:13.610 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.610 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.610 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:48:13.610 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.515177+0000", 2024-09-11T07:48:13.610 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.1460000000000002e-06 2024-09-11T07:48:13.610 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.610 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.610 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:48:13.611 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.515183+0000", 2024-09-11T07:48:13.611 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.032e-06 2024-09-11T07:48:13.611 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.611 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.611 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.611 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.515501+0000", 2024-09-11T07:48:13.611 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00031839800000000001 2024-09-11T07:48:13.611 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.611 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.611 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:48:13.611 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.515521+0000", 2024-09-11T07:48:13.611 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9570999999999999e-05 2024-09-11T07:48:13.612 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.612 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.612 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.612 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.506355+0000", 2024-09-11T07:48:13.612 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.99083417500000004 2024-09-11T07:48:13.612 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.612 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.612 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:48:13.612 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.506513+0000", 2024-09-11T07:48:13.612 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000158176 2024-09-11T07:48:13.612 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.612 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.612 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:48:13.613 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.506616+0000", 2024-09-11T07:48:13.613 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010257899999999999 2024-09-11T07:48:13.613 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.613 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.613 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:48:13.613 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.509696+0000", 2024-09-11T07:48:13.613 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0030806259999999999 2024-09-11T07:48:13.613 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.613 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.613 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.613 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.510548+0000", 2024-09-11T07:48:13.613 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00085145100000000003 2024-09-11T07:48:13.613 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.614 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.614 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.614 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.510595+0000", 2024-09-11T07:48:13.614 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7091e-05 2024-09-11T07:48:13.614 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.614 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.614 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:48:13.614 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.510641+0000", 2024-09-11T07:48:13.614 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5612999999999998e-05 2024-09-11T07:48:13.615 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.615 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.615 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:48:13.615 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.510677+0000", 2024-09-11T07:48:13.615 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6279000000000003e-05 2024-09-11T07:48:13.615 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.615 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:48:13.615 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.615 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.615 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.615 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:37028 11.3 11:c8b2d7ef:::benchmark_data_smithi022_188626_object37027:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e424)", 2024-09-11T07:48:13.615 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:46:30.522592+0000", 2024-09-11T07:48:13.615 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 102.99640678599999, 2024-09-11T07:48:13.615 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.99021616599999995, 2024-09-11T07:48:13.616 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:48:13.616 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:48:13.616 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:48:13.616 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:48:13.616 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:48:13.616 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 37028 2024-09-11T07:48:13.616 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.616 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:48:13.616 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.616 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:48:13.616 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.522592+0000", 2024-09-11T07:48:13.617 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:13.618 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.618 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.618 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.618 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:48:13.618 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.522592+0000", 2024-09-11T07:48:13.618 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.618 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.618 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.619 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:48:13.619 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.522590+0000", 2024-09-11T07:48:13.619 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:48:13.619 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.619 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.619 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:48:13.619 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.522623+0000", 2024-09-11T07:48:13.619 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3442999999999998e-05 2024-09-11T07:48:13.619 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.619 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.619 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:48:13.619 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.522635+0000", 2024-09-11T07:48:13.619 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1663e-05 2024-09-11T07:48:13.619 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.619 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.620 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:48:13.620 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.522641+0000", 2024-09-11T07:48:13.620 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7389999999999996e-06 2024-09-11T07:48:13.620 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.620 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.620 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.620 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.522767+0000", 2024-09-11T07:48:13.620 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012648 2024-09-11T07:48:13.620 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.620 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.620 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:48:13.620 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.522777+0000", 2024-09-11T07:48:13.620 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.7070000000000004e-06 2024-09-11T07:48:13.621 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.621 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.621 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.621 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.507779+0000", 2024-09-11T07:48:13.621 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.98500231999999999 2024-09-11T07:48:13.621 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.621 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.621 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:48:13.621 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.507865+0000", 2024-09-11T07:48:13.621 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.5869000000000001e-05 2024-09-11T07:48:13.621 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.621 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.621 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:48:13.622 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.507942+0000", 2024-09-11T07:48:13.622 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.6852000000000002e-05 2024-09-11T07:48:13.622 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.622 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.622 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:48:13.622 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.510554+0000", 2024-09-11T07:48:13.622 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0026118790000000001 2024-09-11T07:48:13.622 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.622 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.622 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.622 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.510726+0000", 2024-09-11T07:48:13.622 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00017185900000000001 2024-09-11T07:48:13.622 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.623 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.623 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.623 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.512737+0000", 2024-09-11T07:48:13.623 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0020117410000000001 2024-09-11T07:48:13.623 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.623 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.623 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:48:13.623 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.512773+0000", 2024-09-11T07:48:13.623 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5948000000000002e-05 2024-09-11T07:48:13.623 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.623 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.623 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:48:13.623 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.512808+0000", 2024-09-11T07:48:13.624 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4839999999999998e-05 2024-09-11T07:48:13.624 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.624 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:48:13.624 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.624 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.624 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.625 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:37043 11.3 11:c38f77c4:::benchmark_data_smithi022_188626_object37042:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e424)", 2024-09-11T07:48:13.625 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:46:30.531969+0000", 2024-09-11T07:48:13.625 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 102.987029482, 2024-09-11T07:48:13.625 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.980975822, 2024-09-11T07:48:13.625 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:48:13.625 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:48:13.626 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:48:13.626 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:48:13.626 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:48:13.626 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 37043 2024-09-11T07:48:13.626 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.626 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:48:13.627 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.627 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:48:13.627 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.531969+0000", 2024-09-11T07:48:13.627 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.627 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.627 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.627 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:48:13.628 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.531969+0000", 2024-09-11T07:48:13.628 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.628 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.628 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.628 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:48:13.628 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.531967+0000", 2024-09-11T07:48:13.628 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:48:13.629 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.629 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.629 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:48:13.629 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.531999+0000", 2024-09-11T07:48:13.629 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2200999999999999e-05 2024-09-11T07:48:13.629 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.630 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.630 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:48:13.630 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.532015+0000", 2024-09-11T07:48:13.630 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.6327000000000001e-05 2024-09-11T07:48:13.630 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.630 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.631 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:48:13.631 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.532022+0000", 2024-09-11T07:48:13.631 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.241e-06 2024-09-11T07:48:13.631 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.631 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.631 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.632 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.532123+0000", 2024-09-11T07:48:13.632 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000101906 2024-09-11T07:48:13.632 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.632 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.632 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:48:13.632 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.532133+0000", 2024-09-11T07:48:13.632 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.8099999999999992e-06 2024-09-11T07:48:13.632 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.633 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.633 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.633 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.508784+0000", 2024-09-11T07:48:13.633 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.976651092 2024-09-11T07:48:13.633 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.633 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.633 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:48:13.633 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.508876+0000", 2024-09-11T07:48:13.633 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.1300999999999999e-05 2024-09-11T07:48:13.633 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.634 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.634 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:48:13.634 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.508959+0000", 2024-09-11T07:48:13.634 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3679999999999998e-05 2024-09-11T07:48:13.634 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.634 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.634 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.634 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.510756+0000", 2024-09-11T07:48:13.634 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0017969150000000001 2024-09-11T07:48:13.634 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.634 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.634 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:48:13.635 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.512691+0000", 2024-09-11T07:48:13.635 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0019346529999999999 2024-09-11T07:48:13.635 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.635 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.635 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.635 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.512891+0000", 2024-09-11T07:48:13.635 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000200566 2024-09-11T07:48:13.635 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.635 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.635 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:48:13.635 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.512920+0000", 2024-09-11T07:48:13.635 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8442999999999999e-05 2024-09-11T07:48:13.635 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.635 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.635 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:48:13.636 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.512945+0000", 2024-09-11T07:48:13.636 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5259e-05 2024-09-11T07:48:13.636 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.636 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:48:13.636 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.636 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.636 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.636 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:37062 11.3 11:ce8632bb:::benchmark_data_smithi022_188626_object37061:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e424)", 2024-09-11T07:48:13.636 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:46:30.545915+0000", 2024-09-11T07:48:13.636 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 102.973083359, 2024-09-11T07:48:13.636 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.96721774299999996, 2024-09-11T07:48:13.636 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:48:13.636 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:48:13.636 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:48:13.637 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:48:13.637 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:48:13.637 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 37062 2024-09-11T07:48:13.637 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.637 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:48:13.637 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.637 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:48:13.637 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.545915+0000", 2024-09-11T07:48:13.638 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.638 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.638 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.638 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:48:13.638 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.545915+0000", 2024-09-11T07:48:13.638 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.638 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.638 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.638 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:48:13.639 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.545914+0000", 2024-09-11T07:48:13.639 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:48:13.639 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.639 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.639 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:48:13.639 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.545940+0000", 2024-09-11T07:48:13.639 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.6729000000000001e-05 2024-09-11T07:48:13.639 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.639 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.639 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:48:13.640 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.545955+0000", 2024-09-11T07:48:13.640 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4637999999999999e-05 2024-09-11T07:48:13.640 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.640 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.640 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:48:13.640 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.545962+0000", 2024-09-11T07:48:13.640 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7789999999999998e-06 2024-09-11T07:48:13.640 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.640 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.641 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.641 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.546138+0000", 2024-09-11T07:48:13.641 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000176248 2024-09-11T07:48:13.641 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.641 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.641 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:48:13.642 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.546149+0000", 2024-09-11T07:48:13.642 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1498000000000001e-05 2024-09-11T07:48:13.642 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.642 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.642 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.642 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.509747+0000", 2024-09-11T07:48:13.642 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.96359715199999996 2024-09-11T07:48:13.642 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.642 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.642 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:48:13.642 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.509829+0000", 2024-09-11T07:48:13.642 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.2924999999999996e-05 2024-09-11T07:48:13.643 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.643 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.643 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:48:13.643 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.509920+0000", 2024-09-11T07:48:13.643 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.0919000000000001e-05 2024-09-11T07:48:13.643 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.643 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.643 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.643 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.512858+0000", 2024-09-11T07:48:13.643 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0029379190000000002 2024-09-11T07:48:13.643 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.643 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.643 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:48:13.644 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.512869+0000", 2024-09-11T07:48:13.644 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0363999999999999e-05 2024-09-11T07:48:13.644 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.644 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.644 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.644 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.513039+0000", 2024-09-11T07:48:13.644 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00017021399999999999 2024-09-11T07:48:13.644 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.644 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.644 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:48:13.645 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.513097+0000", 2024-09-11T07:48:13.645 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8257999999999997e-05 2024-09-11T07:48:13.645 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.645 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.645 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:48:13.645 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.513133+0000", 2024-09-11T07:48:13.645 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6044999999999998e-05 2024-09-11T07:48:13.645 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.645 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:48:13.645 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.646 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.646 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.646 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:37091 11.3 11:c22ec356:::benchmark_data_smithi022_188626_object37090:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e424)", 2024-09-11T07:48:13.646 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:46:30.564447+0000", 2024-09-11T07:48:13.646 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 102.95455183599999, 2024-09-11T07:48:13.646 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.95103048499999998, 2024-09-11T07:48:13.646 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:48:13.646 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:48:13.646 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:48:13.646 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:48:13.646 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:48:13.647 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 37091 2024-09-11T07:48:13.647 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.647 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:48:13.647 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.647 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:48:13.647 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.564447+0000", 2024-09-11T07:48:13.647 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.647 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.647 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.648 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:48:13.648 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.564447+0000", 2024-09-11T07:48:13.648 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.648 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.648 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.648 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:48:13.648 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.564445+0000", 2024-09-11T07:48:13.648 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:48:13.648 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.648 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.649 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:48:13.649 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.564473+0000", 2024-09-11T07:48:13.649 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7858e-05 2024-09-11T07:48:13.649 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.649 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.649 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:48:13.649 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.564492+0000", 2024-09-11T07:48:13.649 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9191e-05 2024-09-11T07:48:13.649 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.649 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.650 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:48:13.650 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.564498+0000", 2024-09-11T07:48:13.650 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.6850000000000004e-06 2024-09-11T07:48:13.650 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.650 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.650 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.650 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.566279+0000", 2024-09-11T07:48:13.650 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001780666 2024-09-11T07:48:13.650 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.650 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.651 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:48:13.651 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.566289+0000", 2024-09-11T07:48:13.651 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.5710000000000004e-06 2024-09-11T07:48:13.651 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.651 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.651 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.651 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.510778+0000", 2024-09-11T07:48:13.651 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.94448944000000001 2024-09-11T07:48:13.651 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.651 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.652 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:48:13.652 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.510895+0000", 2024-09-11T07:48:13.652 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011733 2024-09-11T07:48:13.652 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.652 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.652 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:48:13.652 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.511029+0000", 2024-09-11T07:48:13.652 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013334999999999999 2024-09-11T07:48:13.652 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.652 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.653 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:48:13.653 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.512872+0000", 2024-09-11T07:48:13.653 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0018434910000000001 2024-09-11T07:48:13.653 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.653 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.653 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.653 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.512992+0000", 2024-09-11T07:48:13.653 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011989699999999999 2024-09-11T07:48:13.653 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.654 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.654 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.654 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.515374+0000", 2024-09-11T07:48:13.654 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0023813189999999998 2024-09-11T07:48:13.654 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.654 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.654 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:48:13.654 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.515438+0000", 2024-09-11T07:48:13.654 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4541000000000004e-05 2024-09-11T07:48:13.654 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.655 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.655 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:48:13.655 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.515477+0000", 2024-09-11T07:48:13.655 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9419000000000003e-05 2024-09-11T07:48:13.655 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.655 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:48:13.655 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.655 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.655 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.655 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:37095 11.3 11:c7169744:::benchmark_data_smithi022_188626_object37094:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e424)", 2024-09-11T07:48:13.655 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:46:30.566182+0000", 2024-09-11T07:48:13.656 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 102.952816845, 2024-09-11T07:48:13.656 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.94942647599999996, 2024-09-11T07:48:13.656 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:48:13.656 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:48:13.656 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:48:13.656 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:48:13.656 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:48:13.656 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 37095 2024-09-11T07:48:13.656 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.656 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:48:13.656 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.656 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:48:13.657 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.566182+0000", 2024-09-11T07:48:13.657 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.657 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.657 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.657 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:48:13.657 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.566182+0000", 2024-09-11T07:48:13.657 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.657 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.657 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.657 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:48:13.658 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.566180+0000", 2024-09-11T07:48:13.658 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:48:13.658 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.658 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.658 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:48:13.658 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.566205+0000", 2024-09-11T07:48:13.658 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5250999999999999e-05 2024-09-11T07:48:13.658 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.658 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.658 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:48:13.658 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.566216+0000", 2024-09-11T07:48:13.658 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0838e-05 2024-09-11T07:48:13.659 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.659 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.659 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:48:13.659 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.566222+0000", 2024-09-11T07:48:13.659 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3029999999999999e-06 2024-09-11T07:48:13.659 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.659 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.659 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.659 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.566308+0000", 2024-09-11T07:48:13.659 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.6658999999999998e-05 2024-09-11T07:48:13.659 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.660 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.660 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:48:13.660 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.566315+0000", 2024-09-11T07:48:13.660 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.455e-06 2024-09-11T07:48:13.660 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.660 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.660 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.660 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.511881+0000", 2024-09-11T07:48:13.660 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.94556666300000003 2024-09-11T07:48:13.660 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.660 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.661 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:48:13.661 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.511962+0000", 2024-09-11T07:48:13.661 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.0918e-05 2024-09-11T07:48:13.661 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.661 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.661 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:48:13.661 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.512048+0000", 2024-09-11T07:48:13.661 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.5258000000000003e-05 2024-09-11T07:48:13.661 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.661 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.662 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:48:13.662 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.513000+0000", 2024-09-11T07:48:13.662 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00095237900000000005 2024-09-11T07:48:13.662 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.662 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.662 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.662 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.515513+0000", 2024-09-11T07:48:13.662 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0025131609999999999 2024-09-11T07:48:13.662 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.662 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.662 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.663 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.515539+0000", 2024-09-11T07:48:13.663 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.5624000000000001e-05 2024-09-11T07:48:13.663 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.663 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.663 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:48:13.663 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.515578+0000", 2024-09-11T07:48:13.663 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9373999999999999e-05 2024-09-11T07:48:13.663 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.663 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.663 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:48:13.664 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.515608+0000", 2024-09-11T07:48:13.664 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0343999999999999e-05 2024-09-11T07:48:13.664 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.664 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:48:13.664 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.664 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.664 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.664 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:37105 11.3 11:c3a497ec:::benchmark_data_smithi022_188626_object37104:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e424)", 2024-09-11T07:48:13.664 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:46:30.572171+0000", 2024-09-11T07:48:13.664 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 102.946827351, 2024-09-11T07:48:13.664 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.948209258, 2024-09-11T07:48:13.664 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:48:13.665 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:48:13.665 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:48:13.665 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:48:13.665 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:48:13.665 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 37105 2024-09-11T07:48:13.665 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.665 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:48:13.665 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.665 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:48:13.665 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.572171+0000", 2024-09-11T07:48:13.665 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.666 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.666 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.666 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:48:13.666 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.572171+0000", 2024-09-11T07:48:13.666 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.666 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.666 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.666 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:48:13.666 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.572170+0000", 2024-09-11T07:48:13.666 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:48:13.666 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.666 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.667 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:48:13.667 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.572200+0000", 2024-09-11T07:48:13.667 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0151e-05 2024-09-11T07:48:13.667 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.667 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.667 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:48:13.667 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.572220+0000", 2024-09-11T07:48:13.667 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0565e-05 2024-09-11T07:48:13.667 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.667 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.667 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:48:13.667 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.572228+0000", 2024-09-11T07:48:13.667 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.4320000000000002e-06 2024-09-11T07:48:13.668 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.668 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.668 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.668 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.572539+0000", 2024-09-11T07:48:13.668 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000311298 2024-09-11T07:48:13.668 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.668 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.668 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:48:13.668 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.572554+0000", 2024-09-11T07:48:13.668 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5021000000000001e-05 2024-09-11T07:48:13.668 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.668 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.669 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.669 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.513166+0000", 2024-09-11T07:48:13.669 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.94061227800000002 2024-09-11T07:48:13.669 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.669 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.669 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:48:13.669 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.513289+0000", 2024-09-11T07:48:13.669 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012253000000000001 2024-09-11T07:48:13.669 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.669 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.669 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:48:13.670 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.513429+0000", 2024-09-11T07:48:13.670 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013959199999999999 2024-09-11T07:48:13.670 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.670 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.670 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.670 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.515639+0000", 2024-09-11T07:48:13.670 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0022102340000000002 2024-09-11T07:48:13.670 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.670 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.670 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.670 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.518650+0000", 2024-09-11T07:48:13.671 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0030108729999999998 2024-09-11T07:48:13.671 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.671 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.671 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:48:13.671 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.520301+0000", 2024-09-11T07:48:13.671 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001651268 2024-09-11T07:48:13.671 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.671 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.671 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:48:13.671 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.520345+0000", 2024-09-11T07:48:13.671 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3575000000000002e-05 2024-09-11T07:48:13.671 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.672 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.672 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:48:13.672 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.520381+0000", 2024-09-11T07:48:13.672 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6170000000000001e-05 2024-09-11T07:48:13.672 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.672 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:48:13.672 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.672 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.672 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.672 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:37110 11.3 11:c3bbb465:::benchmark_data_smithi022_188626_object37109:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e424)", 2024-09-11T07:48:13.673 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:13 smithi022 ceph-mon[37330]: pgmap v1008: 158 pgs: 158 active+clean; 5.4 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-11T07:48:13.673 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:13 smithi022 ceph-mon[37330]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:48:13.673 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:13 smithi022 ceph-mon[37330]: from='client.26098 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:48:13.673 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:13 smithi022 ceph-mon[37330]: osdmap e446: 8 total, 8 up, 7 in 2024-09-11T07:48:13.673 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:13 smithi022 ceph-mon[37330]: 4.a scrub starts 2024-09-11T07:48:13.673 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:13 smithi022 ceph-mon[37330]: 4.a scrub ok 2024-09-11T07:48:13.673 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:13 smithi022 ceph-mon[37330]: 2.2 scrub starts 2024-09-11T07:48:13.674 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:13 smithi022 ceph-mon[37330]: 2.2 scrub ok 2024-09-11T07:48:13.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:13 smithi022 ceph-mon[29462]: pgmap v1008: 158 pgs: 158 active+clean; 5.4 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-11T07:48:13.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:13 smithi022 ceph-mon[29462]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:48:13.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:13 smithi022 ceph-mon[29462]: from='client.26098 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:48:13.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:13 smithi022 ceph-mon[29462]: osdmap e446: 8 total, 8 up, 7 in 2024-09-11T07:48:13.674 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:13 smithi022 ceph-mon[29462]: 4.a scrub starts 2024-09-11T07:48:13.675 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:13 smithi022 ceph-mon[29462]: 4.a scrub ok 2024-09-11T07:48:13.675 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:13 smithi022 ceph-mon[29462]: 2.2 scrub starts 2024-09-11T07:48:13.675 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:13 smithi022 ceph-mon[29462]: 2.2 scrub ok 2024-09-11T07:48:13.675 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:46:30.577188+0000", 2024-09-11T07:48:13.675 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 102.941811164, 2024-09-11T07:48:13.675 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.94326748100000002, 2024-09-11T07:48:13.675 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:48:13.676 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:48:13.676 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:48:13.676 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:48:13.676 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:48:13.676 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 37110 2024-09-11T07:48:13.676 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.676 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:48:13.676 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.676 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:48:13.676 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.577188+0000", 2024-09-11T07:48:13.677 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.677 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.677 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.677 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:48:13.677 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.577188+0000", 2024-09-11T07:48:13.677 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.677 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.677 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.677 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:48:13.677 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.577185+0000", 2024-09-11T07:48:13.677 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:48:13.677 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.678 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.678 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:48:13.678 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.577213+0000", 2024-09-11T07:48:13.678 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7937999999999999e-05 2024-09-11T07:48:13.678 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.678 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.678 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:48:13.678 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.577230+0000", 2024-09-11T07:48:13.678 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.6937000000000001e-05 2024-09-11T07:48:13.678 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.678 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.678 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:48:13.678 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.579299+0000", 2024-09-11T07:48:13.679 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0020687990000000001 2024-09-11T07:48:13.679 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.679 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.679 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.679 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.579362+0000", 2024-09-11T07:48:13.679 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2722e-05 2024-09-11T07:48:13.679 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.679 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.679 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:48:13.679 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.579372+0000", 2024-09-11T07:48:13.679 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0237e-05 2024-09-11T07:48:13.679 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.680 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.680 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.680 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.515659+0000", 2024-09-11T07:48:13.680 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.93628713299999999 2024-09-11T07:48:13.680 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.680 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.680 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:48:13.680 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.515784+0000", 2024-09-11T07:48:13.680 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012458799999999999 2024-09-11T07:48:13.680 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.680 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.680 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:48:13.680 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.515939+0000", 2024-09-11T07:48:13.681 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015536900000000001 2024-09-11T07:48:13.681 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.681 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.681 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.681 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.518682+0000", 2024-09-11T07:48:13.681 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0027427670000000001 2024-09-11T07:48:13.681 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.681 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.681 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.681 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.518733+0000", 2024-09-11T07:48:13.681 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1276000000000002e-05 2024-09-11T07:48:13.682 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.682 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.682 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:48:13.682 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.520388+0000", 2024-09-11T07:48:13.682 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0016551389999999999 2024-09-11T07:48:13.682 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.682 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.682 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:48:13.682 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.520427+0000", 2024-09-11T07:48:13.682 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9127e-05 2024-09-11T07:48:13.682 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.683 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.683 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:48:13.683 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.520455+0000", 2024-09-11T07:48:13.683 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7937999999999999e-05 2024-09-11T07:48:13.683 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.683 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:48:13.683 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.683 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.683 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.683 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:37140 11.3 11:caf90087:::benchmark_data_smithi022_188626_object37139:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e424)", 2024-09-11T07:48:13.683 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:46:30.600255+0000", 2024-09-11T07:48:13.683 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 102.918743345, 2024-09-11T07:48:13.683 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.92032909299999999, 2024-09-11T07:48:13.684 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:48:13.684 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:48:13.684 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:48:13.684 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:48:13.684 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:48:13.684 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 37140 2024-09-11T07:48:13.684 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.684 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:48:13.684 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.684 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:48:13.684 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.600255+0000", 2024-09-11T07:48:13.684 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.684 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.685 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.685 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:48:13.685 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.600255+0000", 2024-09-11T07:48:13.685 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:48:13.685 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.685 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.685 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:48:13.685 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.600253+0000", 2024-09-11T07:48:13.685 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:48:13.685 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.685 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.685 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:48:13.686 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.600284+0000", 2024-09-11T07:48:13.686 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.1046000000000001e-05 2024-09-11T07:48:13.686 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.686 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.686 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:48:13.686 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.600296+0000", 2024-09-11T07:48:13.686 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1549000000000001e-05 2024-09-11T07:48:13.686 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.686 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.686 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:48:13.686 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.600301+0000", 2024-09-11T07:48:13.687 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9370000000000001e-06 2024-09-11T07:48:13.687 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.687 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.687 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.687 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.600408+0000", 2024-09-11T07:48:13.687 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000106764 2024-09-11T07:48:13.687 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.687 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.687 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:48:13.687 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:30.600419+0000", 2024-09-11T07:48:13.687 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0268000000000001e-05 2024-09-11T07:48:13.688 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.688 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.688 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:48:13.688 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.517242+0000", 2024-09-11T07:48:13.688 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.91682344699999996 2024-09-11T07:48:13.688 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.688 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.688 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:48:13.688 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.517357+0000", 2024-09-11T07:48:13.688 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000114644 2024-09-11T07:48:13.688 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.689 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.689 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:48:13.689 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.517459+0000", 2024-09-11T07:48:13.689 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000102154 2024-09-11T07:48:13.689 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.689 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.689 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.689 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.520489+0000", 2024-09-11T07:48:13.689 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0030298769999999998 2024-09-11T07:48:13.689 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.689 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.689 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:48:13.690 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.520494+0000", 2024-09-11T07:48:13.690 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.665e-06 2024-09-11T07:48:13.690 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.690 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.690 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:48:13.690 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.520517+0000", 2024-09-11T07:48:13.690 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2980999999999999e-05 2024-09-11T07:48:13.690 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.690 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.690 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:48:13.690 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.520556+0000", 2024-09-11T07:48:13.690 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8738000000000001e-05 2024-09-11T07:48:13.691 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:48:13.691 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:48:13.691 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:48:13.691 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:46:31.520585+0000", 2024-09-11T07:48:13.691 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8574000000000001e-05 2024-09-11T07:48:13.691 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.691 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:48:13.691 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.691 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:48:13.691 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:48:13.691 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:48:13.716 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:13.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:13.718 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:13.819 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:13.845 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:13.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:13.846 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:13.947 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:13.989 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:13.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:13.990 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:14.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:13 smithi104 ceph-mon[33507]: pgmap v1008: 158 pgs: 158 active+clean; 5.4 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-11T07:48:14.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:13 smithi104 ceph-mon[33507]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:48:14.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:13 smithi104 ceph-mon[33507]: from='client.26098 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:48:14.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:13 smithi104 ceph-mon[33507]: osdmap e446: 8 total, 8 up, 7 in 2024-09-11T07:48:14.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:13 smithi104 ceph-mon[33507]: 4.a scrub starts 2024-09-11T07:48:14.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:13 smithi104 ceph-mon[33507]: 4.a scrub ok 2024-09-11T07:48:14.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:13 smithi104 ceph-mon[33507]: 2.2 scrub starts 2024-09-11T07:48:14.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:13 smithi104 ceph-mon[33507]: 2.2 scrub ok 2024-09-11T07:48:14.090 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:14.116 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:14.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:14.117 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:14.218 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:14.255 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:14.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:14.256 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:14.357 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:14.398 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:14.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:14.400 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:14.500 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:14.525 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:14.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:14.526 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:14.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:48:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:48:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:48:14.627 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:14.649 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:14.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:14.650 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:14.750 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:14.779 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:14.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:14.780 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:14.880 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:14.916 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:14 smithi022 ceph-mon[37330]: 5.19 scrub starts 2024-09-11T07:48:14.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:14 smithi022 ceph-mon[37330]: 5.19 scrub ok 2024-09-11T07:48:14.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:14 smithi022 ceph-mon[37330]: from='client.16203 v1:172.21.15.22:0/1055879478' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:48:14.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:14 smithi022 ceph-mon[37330]: 2.13 scrub starts 2024-09-11T07:48:14.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:14 smithi022 ceph-mon[37330]: 2.13 scrub ok 2024-09-11T07:48:14.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:14 smithi022 ceph-mon[37330]: 5.12 scrub starts 2024-09-11T07:48:14.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:14 smithi022 ceph-mon[37330]: 5.12 scrub ok 2024-09-11T07:48:14.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:14 smithi022 ceph-mon[29462]: 5.19 scrub starts 2024-09-11T07:48:14.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:14 smithi022 ceph-mon[29462]: 5.19 scrub ok 2024-09-11T07:48:14.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:14 smithi022 ceph-mon[29462]: from='client.16203 v1:172.21.15.22:0/1055879478' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:48:14.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:14 smithi022 ceph-mon[29462]: 2.13 scrub starts 2024-09-11T07:48:14.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:14 smithi022 ceph-mon[29462]: 2.13 scrub ok 2024-09-11T07:48:14.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:14 smithi022 ceph-mon[29462]: 5.12 scrub starts 2024-09-11T07:48:14.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:14 smithi022 ceph-mon[29462]: 5.12 scrub ok 2024-09-11T07:48:14.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:14.919 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:14.920 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:15.021 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:15.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:14 smithi104 ceph-mon[33507]: 5.19 scrub starts 2024-09-11T07:48:15.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:14 smithi104 ceph-mon[33507]: 5.19 scrub ok 2024-09-11T07:48:15.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:14 smithi104 ceph-mon[33507]: from='client.16203 v1:172.21.15.22:0/1055879478' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:48:15.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:14 smithi104 ceph-mon[33507]: 2.13 scrub starts 2024-09-11T07:48:15.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:14 smithi104 ceph-mon[33507]: 2.13 scrub ok 2024-09-11T07:48:15.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:14 smithi104 ceph-mon[33507]: 5.12 scrub starts 2024-09-11T07:48:15.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:14 smithi104 ceph-mon[33507]: 5.12 scrub ok 2024-09-11T07:48:15.058 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:15.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:15.059 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:15.160 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:15.186 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:15.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:15.187 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:15.288 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:15.313 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:15.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:15.314 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:15.415 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:15.437 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:15.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:15.438 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:15.539 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:15.564 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:15.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:15.565 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:15.666 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:15.693 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:15.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:15.694 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:15.795 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:15.813 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:15 smithi022 ceph-mon[29462]: 2.b scrub starts 2024-09-11T07:48:15.813 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:15 smithi022 ceph-mon[29462]: 2.b scrub ok 2024-09-11T07:48:15.813 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:15 smithi022 ceph-mon[29462]: pgmap v1010: 158 pgs: 158 active+clean; 5.4 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 653 op/s 2024-09-11T07:48:15.814 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:15 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:48:15.814 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:15 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:48:15.814 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:15 smithi022 ceph-mon[37330]: 2.b scrub starts 2024-09-11T07:48:15.815 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:15 smithi022 ceph-mon[37330]: 2.b scrub ok 2024-09-11T07:48:15.815 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:15 smithi022 ceph-mon[37330]: pgmap v1010: 158 pgs: 158 active+clean; 5.4 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 653 op/s 2024-09-11T07:48:15.815 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:15 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:48:15.815 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:15 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:48:15.820 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:15.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:15.821 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:15.922 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:15.963 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:15.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:15.964 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:16.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:15 smithi104 ceph-mon[33507]: 2.b scrub starts 2024-09-11T07:48:16.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:15 smithi104 ceph-mon[33507]: 2.b scrub ok 2024-09-11T07:48:16.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:15 smithi104 ceph-mon[33507]: pgmap v1010: 158 pgs: 158 active+clean; 5.4 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 653 op/s 2024-09-11T07:48:16.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:15 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:48:16.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:15 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:48:16.065 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:16.087 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:16.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:16.088 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:16.189 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:16.225 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:16.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:16.226 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:16.327 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:16.352 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:16.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:16.353 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:16.454 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:16.479 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:16.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:16.480 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:16.581 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:16.622 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:16.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:16.623 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:16.724 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:16.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:16.766 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:16.766 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:16.866 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:16.893 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:16.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:16.894 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:16.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:16 smithi022 ceph-mon[29462]: 4.5 scrub starts 2024-09-11T07:48:16.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:16 smithi022 ceph-mon[29462]: 4.5 scrub ok 2024-09-11T07:48:16.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:16 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:48:16.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:16 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:48:16.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:16 smithi022 ceph-mon[29462]: osdmap e447: 8 total, 8 up, 7 in 2024-09-11T07:48:16.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:16 smithi022 ceph-mon[29462]: 2.f scrub starts 2024-09-11T07:48:16.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:16 smithi022 ceph-mon[29462]: 2.f scrub ok 2024-09-11T07:48:16.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:16 smithi022 ceph-mon[29462]: 6.b scrub starts 2024-09-11T07:48:16.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:16 smithi022 ceph-mon[29462]: 6.b scrub ok 2024-09-11T07:48:16.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:16 smithi022 ceph-mon[37330]: 4.5 scrub starts 2024-09-11T07:48:16.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:16 smithi022 ceph-mon[37330]: 4.5 scrub ok 2024-09-11T07:48:16.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:16 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:48:16.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:16 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:48:16.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:16 smithi022 ceph-mon[37330]: osdmap e447: 8 total, 8 up, 7 in 2024-09-11T07:48:16.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:16 smithi022 ceph-mon[37330]: 2.f scrub starts 2024-09-11T07:48:16.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:16 smithi022 ceph-mon[37330]: 2.f scrub ok 2024-09-11T07:48:16.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:16 smithi022 ceph-mon[37330]: 6.b scrub starts 2024-09-11T07:48:16.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:16 smithi022 ceph-mon[37330]: 6.b scrub ok 2024-09-11T07:48:16.995 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:16 smithi104 ceph-mon[33507]: 4.5 scrub starts 2024-09-11T07:48:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:16 smithi104 ceph-mon[33507]: 4.5 scrub ok 2024-09-11T07:48:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:16 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:48:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:16 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:48:17.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:16 smithi104 ceph-mon[33507]: osdmap e447: 8 total, 8 up, 7 in 2024-09-11T07:48:17.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:16 smithi104 ceph-mon[33507]: 2.f scrub starts 2024-09-11T07:48:17.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:16 smithi104 ceph-mon[33507]: 2.f scrub ok 2024-09-11T07:48:17.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:16 smithi104 ceph-mon[33507]: 6.b scrub starts 2024-09-11T07:48:17.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:16 smithi104 ceph-mon[33507]: 6.b scrub ok 2024-09-11T07:48:17.031 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:17.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:17.032 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:17.133 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:17.273 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:17.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:17.274 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:17.375 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:17.400 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:17.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:17.401 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:17.502 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:17.553 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:17.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:17.554 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:17.654 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:17.679 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:17.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:17.680 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:17.781 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:17.815 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:17.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:17.817 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:17.903 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:17 smithi022 ceph-mon[29462]: pgmap v1012: 158 pgs: 158 active+clean; 5.1 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 962 op/s 2024-09-11T07:48:17.903 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:17 smithi022 ceph-mon[29462]: 6.7 scrub starts 2024-09-11T07:48:17.903 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:17 smithi022 ceph-mon[29462]: 6.7 scrub ok 2024-09-11T07:48:17.903 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:17 smithi022 ceph-mon[37330]: pgmap v1012: 158 pgs: 158 active+clean; 5.1 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 962 op/s 2024-09-11T07:48:17.903 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:17 smithi022 ceph-mon[37330]: 6.7 scrub starts 2024-09-11T07:48:17.903 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:17 smithi022 ceph-mon[37330]: 6.7 scrub ok 2024-09-11T07:48:17.917 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:17.947 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":1009,"stamp":"2024-09-11T07:48:12.546785+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5811399322,"num_objects":88862,"num_object_clones":0,"num_object_copies":266586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88862,"num_whiteouts":0,"num_read":1786,"num_read_kb":1548,"num_write":151958,"num_write_kb":7667712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":111867,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72606,"ondisk_log_size":72606,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":474,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":23499644,"kb_used_data":17205892,"kb_used_omap":14,"kb_used_meta":6293489,"kb_avail":632716420,"statfs":{"total":671965249536,"available":647901614080,"internally_reserved":0,"allocated":17618833408,"data_stored":17616153800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14826,"internal_metadata":6444533270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,5,1,6,2,0,0,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-676593664,"num_objects":-10324,"num_object_clones":0,"num_object_copies":-30972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-10324,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10324,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.110991"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":207,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.809760+0000","last_change":"2024-09-11T07:46:28.323440+0000","last_active":"2024-09-11T07:47:56.809760+0000","last_peered":"2024-09-11T07:47:56.809760+0000","last_clean":"2024-09-11T07:47:56.809760+0000","last_became_active":"2024-09-11T07:44:23.769058+0000","last_became_peered":"2024-09-11T07:44:23.769058+0000","last_unstale":"2024-09-11T07:47:56.809760+0000","last_undegraded":"2024-09-11T07:47:56.809760+0000","last_fullsized":"2024-09-11T07:47:56.809760+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:28.323351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:46:28.323351+0000","last_clean_scrub_stamp":"2024-09-11T07:46:28.323351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0094746069999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":512,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.822706+0000","last_change":"2024-09-11T07:47:24.201113+0000","last_active":"2024-09-11T07:47:56.822706+0000","last_peered":"2024-09-11T07:47:56.822706+0000","last_clean":"2024-09-11T07:47:56.822706+0000","last_became_active":"2024-09-11T07:45:27.107996+0000","last_became_peered":"2024-09-11T07:45:27.107996+0000","last_unstale":"2024-09-11T07:47:56.822706+0000","last_undegraded":"2024-09-11T07:47:56.822706+0000","last_fullsized":"2024-09-11T07:47:56.822706+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:24.201007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:47:24.201007+0000","last_clean_scrub_stamp":"2024-09-11T07:47:24.201007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:34.114509+0000","scrub_duration":0.006129532,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"426'46","reported_seq":544,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.801631+0000","last_change":"2024-09-11T07:47:03.746508+0000","last_active":"2024-09-11T07:47:56.801631+0000","last_peered":"2024-09-11T07:47:56.801631+0000","last_clean":"2024-09-11T07:47:56.801631+0000","last_became_active":"2024-09-11T07:26:33.544627+0000","last_became_peered":"2024-09-11T07:26:33.544627+0000","last_unstale":"2024-09-11T07:47:56.801631+0000","last_undegraded":"2024-09-11T07:47:56.801631+0000","last_fullsized":"2024-09-11T07:47:56.801631+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'46","last_scrub_stamp":"2024-09-11T07:47:03.746404+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:47:03.746404+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:06.049251+0000","scrub_duration":0.0054538659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":69,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.808794+0000","last_change":"2024-09-11T07:46:04.878844+0000","last_active":"2024-09-11T07:47:56.808794+0000","last_peered":"2024-09-11T07:47:56.808794+0000","last_clean":"2024-09-11T07:47:56.808794+0000","last_became_active":"2024-09-11T07:45:27.095499+0000","last_became_peered":"2024-09-11T07:45:27.095499+0000","last_unstale":"2024-09-11T07:47:56.808794+0000","last_undegraded":"2024-09-11T07:47:56.808794+0000","last_fullsized":"2024-09-11T07:47:56.808794+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:04.878787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:46:04.878787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013075770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":501,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.804315+0000","last_change":"2024-09-11T07:47:00.693301+0000","last_active":"2024-09-11T07:47:56.804315+0000","last_peered":"2024-09-11T07:47:56.804315+0000","last_clean":"2024-09-11T07:47:56.804315+0000","last_became_active":"2024-09-11T07:39:57.767806+0000","last_became_peered":"2024-09-11T07:39:57.767806+0000","last_unstale":"2024-09-11T07:47:56.804315+0000","last_undegraded":"2024-09-11T07:47:56.804315+0000","last_fullsized":"2024-09-11T07:47:56.804315+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:00.693208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:19.655401+0000","last_clean_scrub_stamp":"2024-09-11T07:47:00.693208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:30.503436+0000","scrub_duration":0.0093827379999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"426'25","reported_seq":88,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.808344+0000","last_change":"2024-09-11T07:47:13.067514+0000","last_active":"2024-09-11T07:47:56.808344+0000","last_peered":"2024-09-11T07:47:56.808344+0000","last_clean":"2024-09-11T07:47:56.808344+0000","last_became_active":"2024-09-11T07:45:27.095974+0000","last_became_peered":"2024-09-11T07:45:27.095974+0000","last_unstale":"2024-09-11T07:47:56.808344+0000","last_undegraded":"2024-09-11T07:47:56.808344+0000","last_fullsized":"2024-09-11T07:47:56.808344+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'25","last_scrub_stamp":"2024-09-11T07:47:13.067271+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:36:42.708941+0000","last_clean_scrub_stamp":"2024-09-11T07:47:13.067271+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:16.797264+0000","scrub_duration":0.012124374,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":211,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.814178+0000","last_change":"2024-09-11T07:45:12.417072+0000","last_active":"2024-09-11T07:47:56.814178+0000","last_peered":"2024-09-11T07:47:56.814178+0000","last_clean":"2024-09-11T07:47:56.814178+0000","last_became_active":"2024-09-11T07:44:23.769502+0000","last_became_peered":"2024-09-11T07:44:23.769502+0000","last_unstale":"2024-09-11T07:47:56.814178+0000","last_undegraded":"2024-09-11T07:47:56.814178+0000","last_fullsized":"2024-09-11T07:47:56.814178+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:12.416990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:45:12.416990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001315529,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":342,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.809248+0000","last_change":"2024-09-11T07:46:31.505181+0000","last_active":"2024-09-11T07:47:56.809248+0000","last_peered":"2024-09-11T07:47:56.809248+0000","last_clean":"2024-09-11T07:47:56.809248+0000","last_became_active":"2024-09-11T07:46:31.504850+0000","last_became_peered":"2024-09-11T07:46:31.504850+0000","last_unstale":"2024-09-11T07:47:56.809248+0000","last_undegraded":"2024-09-11T07:47:56.809248+0000","last_fullsized":"2024-09-11T07:47:56.809248+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:05.327186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:46:05.327186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058146650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":480,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.815268+0000","last_change":"2024-09-11T07:46:21.898482+0000","last_active":"2024-09-11T07:47:56.815268+0000","last_peered":"2024-09-11T07:47:56.815268+0000","last_clean":"2024-09-11T07:47:56.815268+0000","last_became_active":"2024-09-11T07:32:00.516662+0000","last_became_peered":"2024-09-11T07:32:00.516662+0000","last_unstale":"2024-09-11T07:47:56.815268+0000","last_undegraded":"2024-09-11T07:47:56.815268+0000","last_fullsized":"2024-09-11T07:47:56.815268+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:21.898400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:46:21.898400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0063092759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"426'11","reported_seq":78,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.804395+0000","last_change":"2024-09-11T07:45:32.375248+0000","last_active":"2024-09-11T07:47:56.804395+0000","last_peered":"2024-09-11T07:47:56.804395+0000","last_clean":"2024-09-11T07:47:56.804395+0000","last_became_active":"2024-09-11T07:45:27.096974+0000","last_became_peered":"2024-09-11T07:45:27.096974+0000","last_unstale":"2024-09-11T07:47:56.804395+0000","last_undegraded":"2024-09-11T07:47:56.804395+0000","last_fullsized":"2024-09-11T07:47:56.804395+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'9","last_scrub_stamp":"2024-09-11T07:45:32.375156+0000","last_deep_scrub":"234'9","last_deep_scrub_stamp":"2024-09-11T07:45:32.375156+0000","last_clean_scrub_stamp":"2024-09-11T07:45:32.375156+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011478199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":13,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":469,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.814201+0000","last_change":"2024-09-11T07:47:26.812154+0000","last_active":"2024-09-11T07:47:56.814201+0000","last_peered":"2024-09-11T07:47:56.814201+0000","last_clean":"2024-09-11T07:47:56.814201+0000","last_became_active":"2024-09-11T07:45:27.112681+0000","last_became_peered":"2024-09-11T07:45:27.112681+0000","last_unstale":"2024-09-11T07:47:56.814201+0000","last_undegraded":"2024-09-11T07:47:56.814201+0000","last_fullsized":"2024-09-11T07:47:56.814201+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:26.812037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:30.973323+0000","last_clean_scrub_stamp":"2024-09-11T07:47:26.812037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:42.862948+0000","scrub_duration":0.0039630519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":109,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.808855+0000","last_change":"2024-09-11T07:47:28.114615+0000","last_active":"2024-09-11T07:47:56.808855+0000","last_peered":"2024-09-11T07:47:56.808855+0000","last_clean":"2024-09-11T07:47:56.808855+0000","last_became_active":"2024-09-11T07:44:23.769187+0000","last_became_peered":"2024-09-11T07:44:23.769187+0000","last_unstale":"2024-09-11T07:47:56.808855+0000","last_undegraded":"2024-09-11T07:47:56.808855+0000","last_fullsized":"2024-09-11T07:47:56.808855+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:28.114522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:45.149916+0000","last_clean_scrub_stamp":"2024-09-11T07:47:28.114522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:47.277277+0000","scrub_duration":0.010263939999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":137,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.811128+0000","last_change":"2024-09-11T07:47:19.164653+0000","last_active":"2024-09-11T07:47:56.811128+0000","last_peered":"2024-09-11T07:47:56.811128+0000","last_clean":"2024-09-11T07:47:56.811128+0000","last_became_active":"2024-09-11T07:45:49.765541+0000","last_became_peered":"2024-09-11T07:45:49.765541+0000","last_unstale":"2024-09-11T07:47:56.811128+0000","last_undegraded":"2024-09-11T07:47:56.811128+0000","last_fullsized":"2024-09-11T07:47:56.811128+0000","mapping_epoch":407,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":408,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:19.164579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:47:19.164579+0000","last_clean_scrub_stamp":"2024-09-11T07:47:19.164579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:20.904858+0000","scrub_duration":0.0028400880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"426'25","reported_seq":351,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.808233+0000","last_change":"2024-09-11T07:46:19.457329+0000","last_active":"2024-09-11T07:47:56.808233+0000","last_peered":"2024-09-11T07:47:56.808233+0000","last_clean":"2024-09-11T07:47:56.808233+0000","last_became_active":"2024-09-11T07:45:27.112765+0000","last_became_peered":"2024-09-11T07:45:27.112765+0000","last_unstale":"2024-09-11T07:47:56.808233+0000","last_undegraded":"2024-09-11T07:47:56.808233+0000","last_fullsized":"2024-09-11T07:47:56.808233+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:46:19.457213+0000","last_deep_scrub":"136'13","last_deep_scrub_stamp":"2024-09-11T07:31:08.042198+0000","last_clean_scrub_stamp":"2024-09-11T07:46:19.457213+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046268419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":507,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.810329+0000","last_change":"2024-09-11T07:47:27.201757+0000","last_active":"2024-09-11T07:47:56.810329+0000","last_peered":"2024-09-11T07:47:56.810329+0000","last_clean":"2024-09-11T07:47:56.810329+0000","last_became_active":"2024-09-11T07:45:27.114162+0000","last_became_peered":"2024-09-11T07:45:27.114162+0000","last_unstale":"2024-09-11T07:47:56.810329+0000","last_undegraded":"2024-09-11T07:47:56.810329+0000","last_fullsized":"2024-09-11T07:47:56.810329+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:27.201533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:47:27.201533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:47.512050+0000","scrub_duration":0.015994092000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":496,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.810115+0000","last_change":"2024-09-11T07:46:24.272062+0000","last_active":"2024-09-11T07:47:56.810115+0000","last_peered":"2024-09-11T07:47:56.810115+0000","last_clean":"2024-09-11T07:47:56.810115+0000","last_became_active":"2024-09-11T07:32:00.530335+0000","last_became_peered":"2024-09-11T07:32:00.530335+0000","last_unstale":"2024-09-11T07:47:56.810115+0000","last_undegraded":"2024-09-11T07:47:56.810115+0000","last_fullsized":"2024-09-11T07:47:56.810115+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:24.271967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:46:24.271967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042967989999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"444'9404","reported_seq":9444,"reported_epoch":444,"state":"active+clean","last_fresh":"2024-09-11T07:48:09.426251+0000","last_change":"2024-09-11T07:45:56.765116+0000","last_active":"2024-09-11T07:48:09.426251+0000","last_peered":"2024-09-11T07:48:09.426251+0000","last_clean":"2024-09-11T07:48:09.426251+0000","last_became_active":"2024-09-11T07:45:56.763997+0000","last_became_peered":"2024-09-11T07:45:56.763997+0000","last_unstale":"2024-09-11T07:48:09.426251+0000","last_undegraded":"2024-09-11T07:48:09.426251+0000","last_fullsized":"2024-09-11T07:48:09.426251+0000","mapping_epoch":412,"log_start":"437'5600","ondisk_log_start":"437'5600","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3804,"log_dups_size":0,"ondisk_log_size":3804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":365953024,"num_objects":5584,"num_object_clones":0,"num_object_copies":16752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9404,"num_write_kb":479616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"87'81","reported_seq":78,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.808254+0000","last_change":"2024-09-11T07:47:12.008908+0000","last_active":"2024-09-11T07:47:56.808254+0000","last_peered":"2024-09-11T07:47:56.808254+0000","last_clean":"2024-09-11T07:47:56.808254+0000","last_became_active":"2024-09-11T07:45:01.808023+0000","last_became_peered":"2024-09-11T07:45:01.808023+0000","last_unstale":"2024-09-11T07:47:56.808254+0000","last_undegraded":"2024-09-11T07:47:56.808254+0000","last_fullsized":"2024-09-11T07:47:56.808254+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:47:12.008803+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:47:12.008803+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:30.856293+0000","scrub_duration":0.004197966,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"426'45","reported_seq":447,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.811115+0000","last_change":"2024-09-11T07:47:23.170550+0000","last_active":"2024-09-11T07:47:56.811115+0000","last_peered":"2024-09-11T07:47:56.811115+0000","last_clean":"2024-09-11T07:47:56.811115+0000","last_became_active":"2024-09-11T07:45:27.094411+0000","last_became_peered":"2024-09-11T07:45:27.094411+0000","last_unstale":"2024-09-11T07:47:56.811115+0000","last_undegraded":"2024-09-11T07:47:56.811115+0000","last_fullsized":"2024-09-11T07:47:56.811115+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'45","last_scrub_stamp":"2024-09-11T07:47:23.170462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:47:23.170462+0000","objects_scrubbed":8,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:38.322774+0000","scrub_duration":0.030572766000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":60,"num_write":46,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":488,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.814031+0000","last_change":"2024-09-11T07:45:27.100579+0000","last_active":"2024-09-11T07:47:56.814031+0000","last_peered":"2024-09-11T07:47:56.814031+0000","last_clean":"2024-09-11T07:47:56.814031+0000","last_became_active":"2024-09-11T07:45:27.100238+0000","last_became_peered":"2024-09-11T07:45:27.100238+0000","last_unstale":"2024-09-11T07:47:56.814031+0000","last_undegraded":"2024-09-11T07:47:56.814031+0000","last_fullsized":"2024-09-11T07:47:56.814031+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:08.501392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:08.501392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004339763,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":77,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.807027+0000","last_change":"2024-09-11T07:46:09.235485+0000","last_active":"2024-09-11T07:47:56.807027+0000","last_peered":"2024-09-11T07:47:56.807027+0000","last_clean":"2024-09-11T07:47:56.807027+0000","last_became_active":"2024-09-11T07:45:27.117371+0000","last_became_peered":"2024-09-11T07:45:27.117371+0000","last_unstale":"2024-09-11T07:47:56.807027+0000","last_undegraded":"2024-09-11T07:47:56.807027+0000","last_fullsized":"2024-09-11T07:47:56.807027+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:09.235371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:46:09.235371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016833869999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":502,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.812951+0000","last_change":"2024-09-11T07:47:07.991465+0000","last_active":"2024-09-11T07:47:56.812951+0000","last_peered":"2024-09-11T07:47:56.812951+0000","last_clean":"2024-09-11T07:47:56.812951+0000","last_became_active":"2024-09-11T07:45:27.104519+0000","last_became_peered":"2024-09-11T07:45:27.104519+0000","last_unstale":"2024-09-11T07:47:56.812951+0000","last_undegraded":"2024-09-11T07:47:56.812951+0000","last_fullsized":"2024-09-11T07:47:56.812951+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:07.991385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:32.281886+0000","last_clean_scrub_stamp":"2024-09-11T07:47:07.991385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:30.036131+0000","scrub_duration":0.0036103490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"444'9368","reported_seq":9417,"reported_epoch":444,"state":"active+clean","last_fresh":"2024-09-11T07:48:09.451724+0000","last_change":"2024-09-11T07:46:31.506264+0000","last_active":"2024-09-11T07:48:09.451724+0000","last_peered":"2024-09-11T07:48:09.451724+0000","last_clean":"2024-09-11T07:48:09.451724+0000","last_became_active":"2024-09-11T07:46:31.502938+0000","last_became_peered":"2024-09-11T07:46:31.502938+0000","last_unstale":"2024-09-11T07:48:09.451724+0000","last_undegraded":"2024-09-11T07:48:09.451724+0000","last_fullsized":"2024-09-11T07:48:09.451724+0000","mapping_epoch":424,"log_start":"437'5500","ondisk_log_start":"437'5500","created":412,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3868,"log_dups_size":0,"ondisk_log_size":3868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":361496576,"num_objects":5516,"num_object_clones":0,"num_object_copies":16548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9368,"num_write_kb":476288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"87'81","reported_seq":73,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.813864+0000","last_change":"2024-09-11T07:46:59.664935+0000","last_active":"2024-09-11T07:47:56.813864+0000","last_peered":"2024-09-11T07:47:56.813864+0000","last_clean":"2024-09-11T07:47:56.813864+0000","last_became_active":"2024-09-11T07:45:01.808923+0000","last_became_peered":"2024-09-11T07:45:01.808923+0000","last_unstale":"2024-09-11T07:47:56.813864+0000","last_undegraded":"2024-09-11T07:47:56.813864+0000","last_fullsized":"2024-09-11T07:47:56.813864+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:46:59.664846+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:46:59.664846+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:14.929909+0000","scrub_duration":0.0074640790000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"426'35","reported_seq":355,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.805117+0000","last_change":"2024-09-11T07:45:33.404225+0000","last_active":"2024-09-11T07:47:56.805117+0000","last_peered":"2024-09-11T07:47:56.805117+0000","last_clean":"2024-09-11T07:47:56.805117+0000","last_became_active":"2024-09-11T07:45:27.097522+0000","last_became_peered":"2024-09-11T07:45:27.097522+0000","last_unstale":"2024-09-11T07:47:56.805117+0000","last_undegraded":"2024-09-11T07:47:56.805117+0000","last_fullsized":"2024-09-11T07:47:56.805117+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'25","last_scrub_stamp":"2024-09-11T07:45:33.403950+0000","last_deep_scrub":"112'15","last_deep_scrub_stamp":"2024-09-11T07:31:10.005896+0000","last_clean_scrub_stamp":"2024-09-11T07:45:33.403950+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044240010000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":215,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.808400+0000","last_change":"2024-09-11T07:46:23.435408+0000","last_active":"2024-09-11T07:47:56.808400+0000","last_peered":"2024-09-11T07:47:56.808400+0000","last_clean":"2024-09-11T07:47:56.808400+0000","last_became_active":"2024-09-11T07:44:23.768172+0000","last_became_peered":"2024-09-11T07:44:23.768172+0000","last_unstale":"2024-09-11T07:47:56.808400+0000","last_undegraded":"2024-09-11T07:47:56.808400+0000","last_fullsized":"2024-09-11T07:47:56.808400+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:23.435037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:46:23.435037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049327829999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"299'11","reported_seq":784,"reported_epoch":444,"state":"active+clean","last_fresh":"2024-09-11T07:47:59.770968+0000","last_change":"2024-09-11T07:47:07.004835+0000","last_active":"2024-09-11T07:47:59.770968+0000","last_peered":"2024-09-11T07:47:59.770968+0000","last_clean":"2024-09-11T07:47:59.770968+0000","last_became_active":"2024-09-11T07:40:11.697568+0000","last_became_peered":"2024-09-11T07:40:11.697568+0000","last_unstale":"2024-09-11T07:47:59.770968+0000","last_undegraded":"2024-09-11T07:47:59.770968+0000","last_fullsized":"2024-09-11T07:47:59.770968+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"299'11","last_scrub_stamp":"2024-09-11T07:47:07.004756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:47:07.004756+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:18.785247+0000","scrub_duration":0.013171522,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":367,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.808765+0000","last_change":"2024-09-11T07:46:14.955001+0000","last_active":"2024-09-11T07:47:56.808765+0000","last_peered":"2024-09-11T07:47:56.808765+0000","last_clean":"2024-09-11T07:47:56.808765+0000","last_became_active":"2024-09-11T07:45:27.099434+0000","last_became_peered":"2024-09-11T07:45:27.099434+0000","last_unstale":"2024-09-11T07:47:56.808765+0000","last_undegraded":"2024-09-11T07:47:56.808765+0000","last_fullsized":"2024-09-11T07:47:56.808765+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:14.954917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:46:14.954917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052986090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.0","version":"444'9525","reported_seq":9565,"reported_epoch":444,"state":"active+clean","last_fresh":"2024-09-11T07:48:10.280613+0000","last_change":"2024-09-11T07:45:56.758475+0000","last_active":"2024-09-11T07:48:10.280613+0000","last_peered":"2024-09-11T07:48:10.280613+0000","last_clean":"2024-09-11T07:48:10.280613+0000","last_became_active":"2024-09-11T07:45:56.758196+0000","last_became_peered":"2024-09-11T07:45:56.758196+0000","last_unstale":"2024-09-11T07:48:10.280613+0000","last_undegraded":"2024-09-11T07:48:10.280613+0000","last_fullsized":"2024-09-11T07:48:10.280613+0000","mapping_epoch":412,"log_start":"430'4600","ondisk_log_start":"430'4600","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":4925,"log_dups_size":0,"ondisk_log_size":4925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359464960,"num_objects":5485,"num_object_clones":0,"num_object_copies":16455,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9525,"num_write_kb":480320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"87'81","reported_seq":73,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.808392+0000","last_change":"2024-09-11T07:47:09.033253+0000","last_active":"2024-09-11T07:47:56.808392+0000","last_peered":"2024-09-11T07:47:56.808392+0000","last_clean":"2024-09-11T07:47:56.808392+0000","last_became_active":"2024-09-11T07:45:01.805300+0000","last_became_peered":"2024-09-11T07:45:01.805300+0000","last_unstale":"2024-09-11T07:47:56.808392+0000","last_undegraded":"2024-09-11T07:47:56.808392+0000","last_fullsized":"2024-09-11T07:47:56.808392+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:47:09.033169+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:47:09.033169+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:09.236120+0000","scrub_duration":0.001752342,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"426'46","reported_seq":401,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.804954+0000","last_change":"2024-09-11T07:47:15.673799+0000","last_active":"2024-09-11T07:47:56.804954+0000","last_peered":"2024-09-11T07:47:56.804954+0000","last_clean":"2024-09-11T07:47:56.804954+0000","last_became_active":"2024-09-11T07:45:27.095634+0000","last_became_peered":"2024-09-11T07:45:27.095634+0000","last_unstale":"2024-09-11T07:47:56.804954+0000","last_undegraded":"2024-09-11T07:47:56.804954+0000","last_fullsized":"2024-09-11T07:47:56.804954+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'46","last_scrub_stamp":"2024-09-11T07:47:15.673697+0000","last_deep_scrub":"83'22","last_deep_scrub_stamp":"2024-09-11T07:34:04.375457+0000","last_clean_scrub_stamp":"2024-09-11T07:47:15.673697+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:26.577269+0000","scrub_duration":0.011487635,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":235,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.814067+0000","last_change":"2024-09-11T07:47:09.781123+0000","last_active":"2024-09-11T07:47:56.814067+0000","last_peered":"2024-09-11T07:47:56.814067+0000","last_clean":"2024-09-11T07:47:56.814067+0000","last_became_active":"2024-09-11T07:44:23.770343+0000","last_became_peered":"2024-09-11T07:44:23.770343+0000","last_unstale":"2024-09-11T07:47:56.814067+0000","last_undegraded":"2024-09-11T07:47:56.814067+0000","last_fullsized":"2024-09-11T07:47:56.814067+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:09.781047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:27.416207+0000","last_clean_scrub_stamp":"2024-09-11T07:47:09.781047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:16.887151+0000","scrub_duration":0.0044366759999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"395'10","reported_seq":700,"reported_epoch":444,"state":"active+clean","last_fresh":"2024-09-11T07:47:59.775373+0000","last_change":"2024-09-11T07:46:10.279342+0000","last_active":"2024-09-11T07:47:59.775373+0000","last_peered":"2024-09-11T07:47:59.775373+0000","last_clean":"2024-09-11T07:47:59.775373+0000","last_became_active":"2024-09-11T07:45:27.122326+0000","last_became_peered":"2024-09-11T07:45:27.122326+0000","last_unstale":"2024-09-11T07:47:59.775373+0000","last_undegraded":"2024-09-11T07:47:59.775373+0000","last_fullsized":"2024-09-11T07:47:59.775373+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"395'10","last_scrub_stamp":"2024-09-11T07:46:10.279284+0000","last_deep_scrub":"366'9","last_deep_scrub_stamp":"2024-09-11T07:43:26.334501+0000","last_clean_scrub_stamp":"2024-09-11T07:46:10.279284+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001636265,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":334,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.809013+0000","last_change":"2024-09-11T07:45:49.765810+0000","last_active":"2024-09-11T07:47:56.809013+0000","last_peered":"2024-09-11T07:47:56.809013+0000","last_clean":"2024-09-11T07:47:56.809013+0000","last_became_active":"2024-09-11T07:45:49.765504+0000","last_became_peered":"2024-09-11T07:45:49.765504+0000","last_unstale":"2024-09-11T07:47:56.809013+0000","last_undegraded":"2024-09-11T07:47:56.809013+0000","last_fullsized":"2024-09-11T07:47:56.809013+0000","mapping_epoch":407,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":408,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:19.336454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:31.200685+0000","last_clean_scrub_stamp":"2024-09-11T07:45:19.336454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001371688,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"444'9408","reported_seq":9448,"reported_epoch":444,"state":"active+clean","last_fresh":"2024-09-11T07:48:09.536351+0000","last_change":"2024-09-11T07:45:57.199288+0000","last_active":"2024-09-11T07:48:09.536351+0000","last_peered":"2024-09-11T07:48:09.536351+0000","last_clean":"2024-09-11T07:48:09.536351+0000","last_became_active":"2024-09-11T07:45:57.198828+0000","last_became_peered":"2024-09-11T07:45:57.198828+0000","last_unstale":"2024-09-11T07:48:09.536351+0000","last_undegraded":"2024-09-11T07:48:09.536351+0000","last_fullsized":"2024-09-11T07:48:09.536351+0000","mapping_epoch":412,"log_start":"437'5700","ondisk_log_start":"437'5700","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3708,"log_dups_size":0,"ondisk_log_size":3708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":361889792,"num_objects":5522,"num_object_clones":0,"num_object_copies":16566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9408,"num_write_kb":477760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"426'32","reported_seq":431,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.804028+0000","last_change":"2024-09-11T07:46:24.388932+0000","last_active":"2024-09-11T07:47:56.804028+0000","last_peered":"2024-09-11T07:47:56.804028+0000","last_clean":"2024-09-11T07:47:56.804028+0000","last_became_active":"2024-09-11T07:45:27.102164+0000","last_became_peered":"2024-09-11T07:45:27.102164+0000","last_unstale":"2024-09-11T07:47:56.804028+0000","last_undegraded":"2024-09-11T07:47:56.804028+0000","last_fullsized":"2024-09-11T07:47:56.804028+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'24","last_scrub_stamp":"2024-09-11T07:46:24.388838+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-09-11T07:29:38.112111+0000","last_clean_scrub_stamp":"2024-09-11T07:46:24.388838+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0055826570000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":472,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.813893+0000","last_change":"2024-09-11T07:46:12.925725+0000","last_active":"2024-09-11T07:47:56.813893+0000","last_peered":"2024-09-11T07:47:56.813893+0000","last_clean":"2024-09-11T07:47:56.813893+0000","last_became_active":"2024-09-11T07:32:00.518538+0000","last_became_peered":"2024-09-11T07:32:00.518538+0000","last_unstale":"2024-09-11T07:47:56.813893+0000","last_undegraded":"2024-09-11T07:47:56.813893+0000","last_fullsized":"2024-09-11T07:47:56.813893+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:12.925664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:43:24.643676+0000","last_clean_scrub_stamp":"2024-09-11T07:46:12.925664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026407430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":69,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.808622+0000","last_change":"2024-09-11T07:45:30.682670+0000","last_active":"2024-09-11T07:47:56.808622+0000","last_peered":"2024-09-11T07:47:56.808622+0000","last_clean":"2024-09-11T07:47:56.808622+0000","last_became_active":"2024-09-11T07:45:27.094025+0000","last_became_peered":"2024-09-11T07:45:27.094025+0000","last_unstale":"2024-09-11T07:47:56.808622+0000","last_undegraded":"2024-09-11T07:47:56.808622+0000","last_fullsized":"2024-09-11T07:47:56.808622+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:30.682614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:19.714568+0000","last_clean_scrub_stamp":"2024-09-11T07:45:30.682614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015960729999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":469,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.803158+0000","last_change":"2024-09-11T07:45:17.470823+0000","last_active":"2024-09-11T07:47:56.803158+0000","last_peered":"2024-09-11T07:47:56.803158+0000","last_clean":"2024-09-11T07:47:56.803158+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:47:56.803158+0000","last_undegraded":"2024-09-11T07:47:56.803158+0000","last_fullsized":"2024-09-11T07:47:56.803158+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:17.470730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:46.853387+0000","last_clean_scrub_stamp":"2024-09-11T07:45:17.470730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002761412,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.e","version":"444'9517","reported_seq":9557,"reported_epoch":444,"state":"active+clean","last_fresh":"2024-09-11T07:48:09.571488+0000","last_change":"2024-09-11T07:45:56.762206+0000","last_active":"2024-09-11T07:48:09.571488+0000","last_peered":"2024-09-11T07:48:09.571488+0000","last_clean":"2024-09-11T07:48:09.571488+0000","last_became_active":"2024-09-11T07:45:56.761971+0000","last_became_peered":"2024-09-11T07:45:56.761971+0000","last_unstale":"2024-09-11T07:48:09.571488+0000","last_undegraded":"2024-09-11T07:48:09.571488+0000","last_fullsized":"2024-09-11T07:48:09.571488+0000","mapping_epoch":412,"log_start":"428'4000","ondisk_log_start":"428'4000","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":5517,"log_dups_size":0,"ondisk_log_size":5517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":368115712,"num_objects":5617,"num_object_clones":0,"num_object_copies":16851,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9517,"num_write_kb":484288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"87'81","reported_seq":92,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.804417+0000","last_change":"2024-09-11T07:47:06.691785+0000","last_active":"2024-09-11T07:47:56.804417+0000","last_peered":"2024-09-11T07:47:56.804417+0000","last_clean":"2024-09-11T07:47:56.804417+0000","last_became_active":"2024-09-11T07:45:27.097022+0000","last_became_peered":"2024-09-11T07:45:27.097022+0000","last_unstale":"2024-09-11T07:47:56.804417+0000","last_undegraded":"2024-09-11T07:47:56.804417+0000","last_fullsized":"2024-09-11T07:47:56.804417+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:47:06.691689+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:47:06.691689+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:18.242055+0000","scrub_duration":0.0033647320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"426'42","reported_seq":320,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.822536+0000","last_change":"2024-09-11T07:46:31.496469+0000","last_active":"2024-09-11T07:47:56.822536+0000","last_peered":"2024-09-11T07:47:56.822536+0000","last_clean":"2024-09-11T07:47:56.822536+0000","last_became_active":"2024-09-11T07:46:31.496001+0000","last_became_peered":"2024-09-11T07:46:31.496001+0000","last_unstale":"2024-09-11T07:47:56.822536+0000","last_undegraded":"2024-09-11T07:47:56.822536+0000","last_fullsized":"2024-09-11T07:47:56.822536+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'28","last_scrub_stamp":"2024-09-11T07:46:08.280072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:46:08.280072+0000","objects_scrubbed":7,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012975469999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":55,"num_write":42,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":251,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.813091+0000","last_change":"2024-09-11T07:47:00.850246+0000","last_active":"2024-09-11T07:47:56.813091+0000","last_peered":"2024-09-11T07:47:56.813091+0000","last_clean":"2024-09-11T07:47:56.813091+0000","last_became_active":"2024-09-11T07:39:57.767741+0000","last_became_peered":"2024-09-11T07:39:57.767741+0000","last_unstale":"2024-09-11T07:47:56.813091+0000","last_undegraded":"2024-09-11T07:47:56.813091+0000","last_fullsized":"2024-09-11T07:47:56.813091+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:00.850169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:47:00.850169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:11.876461+0000","scrub_duration":0.0023451930000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3],"acting":[2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"395'10","reported_seq":755,"reported_epoch":444,"state":"active+clean","last_fresh":"2024-09-11T07:48:04.776385+0000","last_change":"2024-09-11T07:47:13.601869+0000","last_active":"2024-09-11T07:48:04.776385+0000","last_peered":"2024-09-11T07:48:04.776385+0000","last_clean":"2024-09-11T07:48:04.776385+0000","last_became_active":"2024-09-11T07:45:27.098861+0000","last_became_peered":"2024-09-11T07:45:27.098861+0000","last_unstale":"2024-09-11T07:48:04.776385+0000","last_undegraded":"2024-09-11T07:48:04.776385+0000","last_fullsized":"2024-09-11T07:48:04.776385+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"395'10","last_scrub_stamp":"2024-09-11T07:47:13.601667+0000","last_deep_scrub":"84'9","last_deep_scrub_stamp":"2024-09-11T07:35:39.832547+0000","last_clean_scrub_stamp":"2024-09-11T07:47:13.601667+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:39.044308+0000","scrub_duration":0.0076782939999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":453,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.813941+0000","last_change":"2024-09-11T07:47:10.766460+0000","last_active":"2024-09-11T07:47:56.813941+0000","last_peered":"2024-09-11T07:47:56.813941+0000","last_clean":"2024-09-11T07:47:56.813941+0000","last_became_active":"2024-09-11T07:32:00.511005+0000","last_became_peered":"2024-09-11T07:32:00.511005+0000","last_unstale":"2024-09-11T07:47:56.813941+0000","last_undegraded":"2024-09-11T07:47:56.813941+0000","last_fullsized":"2024-09-11T07:47:56.813941+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:10.766409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:47:10.766409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:27.174979+0000","scrub_duration":0.0017833829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"444'9297","reported_seq":9337,"reported_epoch":444,"state":"active+clean","last_fresh":"2024-09-11T07:48:09.571421+0000","last_change":"2024-09-11T07:45:56.764982+0000","last_active":"2024-09-11T07:48:09.571421+0000","last_peered":"2024-09-11T07:48:09.571421+0000","last_clean":"2024-09-11T07:48:09.571421+0000","last_became_active":"2024-09-11T07:45:56.764754+0000","last_became_peered":"2024-09-11T07:45:56.764754+0000","last_unstale":"2024-09-11T07:48:09.571421+0000","last_undegraded":"2024-09-11T07:48:09.571421+0000","last_fullsized":"2024-09-11T07:48:09.571421+0000","mapping_epoch":412,"log_start":"428'3800","ondisk_log_start":"428'3800","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":5497,"log_dups_size":0,"ondisk_log_size":5497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":361693184,"num_objects":5519,"num_object_clones":0,"num_object_copies":16557,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9297,"num_write_kb":474112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"87'81","reported_seq":73,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.818081+0000","last_change":"2024-09-11T07:47:22.146275+0000","last_active":"2024-09-11T07:47:56.818081+0000","last_peered":"2024-09-11T07:47:56.818081+0000","last_clean":"2024-09-11T07:47:56.818081+0000","last_became_active":"2024-09-11T07:45:27.099677+0000","last_became_peered":"2024-09-11T07:45:27.099677+0000","last_unstale":"2024-09-11T07:47:56.818081+0000","last_undegraded":"2024-09-11T07:47:56.818081+0000","last_fullsized":"2024-09-11T07:47:56.818081+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:47:22.145799+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:47:22.145799+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:25.836936+0000","scrub_duration":0.0049845239999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"426'50","reported_seq":567,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.810213+0000","last_change":"2024-09-11T07:46:26.243185+0000","last_active":"2024-09-11T07:47:56.810213+0000","last_peered":"2024-09-11T07:47:56.810213+0000","last_clean":"2024-09-11T07:47:56.810213+0000","last_became_active":"2024-09-11T07:46:11.762572+0000","last_became_peered":"2024-09-11T07:46:11.762572+0000","last_unstale":"2024-09-11T07:47:56.810213+0000","last_undegraded":"2024-09-11T07:47:56.810213+0000","last_fullsized":"2024-09-11T07:47:56.810213+0000","mapping_epoch":418,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":419,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'38","last_scrub_stamp":"2024-09-11T07:46:26.243126+0000","last_deep_scrub":"234'38","last_deep_scrub_stamp":"2024-09-11T07:46:26.243126+0000","last_clean_scrub_stamp":"2024-09-11T07:46:26.243126+0000","objects_scrubbed":8,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022410799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":69,"num_read_kb":57,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,5],"acting":[0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":66,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.803935+0000","last_change":"2024-09-11T07:45:27.113078+0000","last_active":"2024-09-11T07:47:56.803935+0000","last_peered":"2024-09-11T07:47:56.803935+0000","last_clean":"2024-09-11T07:47:56.803935+0000","last_became_active":"2024-09-11T07:45:27.112250+0000","last_became_peered":"2024-09-11T07:45:27.112250+0000","last_unstale":"2024-09-11T07:47:56.803935+0000","last_undegraded":"2024-09-11T07:47:56.803935+0000","last_fullsized":"2024-09-11T07:47:56.803935+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:16.022013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:16.022013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":379,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.801323+0000","last_change":"2024-09-11T07:45:31.356987+0000","last_active":"2024-09-11T07:47:56.801323+0000","last_peered":"2024-09-11T07:47:56.801323+0000","last_clean":"2024-09-11T07:47:56.801323+0000","last_became_active":"2024-09-11T07:45:27.094619+0000","last_became_peered":"2024-09-11T07:45:27.094619+0000","last_unstale":"2024-09-11T07:47:56.801323+0000","last_undegraded":"2024-09-11T07:47:56.801323+0000","last_fullsized":"2024-09-11T07:47:56.801323+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:31.356921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:27.968579+0000","last_clean_scrub_stamp":"2024-09-11T07:45:31.356921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001320955,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":367,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.808573+0000","last_change":"2024-09-11T07:46:02.831398+0000","last_active":"2024-09-11T07:47:56.808573+0000","last_peered":"2024-09-11T07:47:56.808573+0000","last_clean":"2024-09-11T07:47:56.808573+0000","last_became_active":"2024-09-11T07:45:27.098502+0000","last_became_peered":"2024-09-11T07:45:27.098502+0000","last_unstale":"2024-09-11T07:47:56.808573+0000","last_undegraded":"2024-09-11T07:47:56.808573+0000","last_fullsized":"2024-09-11T07:47:56.808573+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:02.831341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:29.978798+0000","last_clean_scrub_stamp":"2024-09-11T07:46:02.831341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012920449999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.c","version":"444'9262","reported_seq":9302,"reported_epoch":444,"state":"active+clean","last_fresh":"2024-09-11T07:48:09.420670+0000","last_change":"2024-09-11T07:45:56.764201+0000","last_active":"2024-09-11T07:48:09.420670+0000","last_peered":"2024-09-11T07:48:09.420670+0000","last_clean":"2024-09-11T07:48:09.420670+0000","last_became_active":"2024-09-11T07:45:56.763888+0000","last_became_peered":"2024-09-11T07:45:56.763888+0000","last_unstale":"2024-09-11T07:48:09.420670+0000","last_undegraded":"2024-09-11T07:48:09.420670+0000","last_fullsized":"2024-09-11T07:48:09.420670+0000","mapping_epoch":412,"log_start":"437'5400","ondisk_log_start":"437'5400","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3862,"log_dups_size":0,"ondisk_log_size":3862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":357957632,"num_objects":5462,"num_object_clones":0,"num_object_copies":16386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9262,"num_write_kb":471168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,7],"acting":[2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"87'81","reported_seq":352,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.822388+0000","last_change":"2024-09-11T07:47:04.284321+0000","last_active":"2024-09-11T07:47:56.822388+0000","last_peered":"2024-09-11T07:47:56.822388+0000","last_clean":"2024-09-11T07:47:56.822388+0000","last_became_active":"2024-09-11T07:45:27.111145+0000","last_became_peered":"2024-09-11T07:45:27.111145+0000","last_unstale":"2024-09-11T07:47:56.822388+0000","last_undegraded":"2024-09-11T07:47:56.822388+0000","last_fullsized":"2024-09-11T07:47:56.822388+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:47:04.284107+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:47:04.284107+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:29.575056+0000","scrub_duration":0.011865242,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"426'42","reported_seq":559,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.810254+0000","last_change":"2024-09-11T07:45:13.572062+0000","last_active":"2024-09-11T07:47:56.810254+0000","last_peered":"2024-09-11T07:47:56.810254+0000","last_clean":"2024-09-11T07:47:56.810254+0000","last_became_active":"2024-09-11T07:32:00.506270+0000","last_became_peered":"2024-09-11T07:32:00.506270+0000","last_unstale":"2024-09-11T07:47:56.810254+0000","last_undegraded":"2024-09-11T07:47:56.810254+0000","last_fullsized":"2024-09-11T07:47:56.810254+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'32","last_scrub_stamp":"2024-09-11T07:45:13.571949+0000","last_deep_scrub":"236'32","last_deep_scrub_stamp":"2024-09-11T07:36:50.023909+0000","last_clean_scrub_stamp":"2024-09-11T07:45:13.571949+0000","objects_scrubbed":8,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020494009000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":69,"num_read_kb":55,"num_write":42,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":463,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.815471+0000","last_change":"2024-09-11T07:47:05.762167+0000","last_active":"2024-09-11T07:47:56.815471+0000","last_peered":"2024-09-11T07:47:56.815471+0000","last_clean":"2024-09-11T07:47:56.815471+0000","last_became_active":"2024-09-11T07:45:27.113680+0000","last_became_peered":"2024-09-11T07:45:27.113680+0000","last_unstale":"2024-09-11T07:47:56.815471+0000","last_undegraded":"2024-09-11T07:47:56.815471+0000","last_fullsized":"2024-09-11T07:47:56.815471+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:05.762110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:24.470602+0000","last_clean_scrub_stamp":"2024-09-11T07:47:05.762110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:28.143244+0000","scrub_duration":0.0077140580000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":282,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.809968+0000","last_change":"2024-09-11T07:47:18.269182+0000","last_active":"2024-09-11T07:47:56.809968+0000","last_peered":"2024-09-11T07:47:56.809968+0000","last_clean":"2024-09-11T07:47:56.809968+0000","last_became_active":"2024-09-11T07:36:05.225840+0000","last_became_peered":"2024-09-11T07:36:05.225840+0000","last_unstale":"2024-09-11T07:47:56.809968+0000","last_undegraded":"2024-09-11T07:47:56.809968+0000","last_fullsized":"2024-09-11T07:47:56.809968+0000","mapping_epoch":224,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:18.269109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:22.285763+0000","last_clean_scrub_stamp":"2024-09-11T07:47:18.269109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:20.056073+0000","scrub_duration":0.0050633400000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":87,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.809068+0000","last_change":"2024-09-11T07:45:23.663301+0000","last_active":"2024-09-11T07:47:56.809068+0000","last_peered":"2024-09-11T07:47:56.809068+0000","last_clean":"2024-09-11T07:47:56.809068+0000","last_became_active":"2024-09-11T07:44:23.769063+0000","last_became_peered":"2024-09-11T07:44:23.769063+0000","last_unstale":"2024-09-11T07:47:56.809068+0000","last_undegraded":"2024-09-11T07:47:56.809068+0000","last_fullsized":"2024-09-11T07:47:56.809068+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:23.663218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:39.723598+0000","last_clean_scrub_stamp":"2024-09-11T07:45:23.663218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006627272,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"444'9325","reported_seq":9366,"reported_epoch":444,"state":"active+clean","last_fresh":"2024-09-11T07:48:09.449370+0000","last_change":"2024-09-11T07:45:57.201789+0000","last_active":"2024-09-11T07:48:09.449370+0000","last_peered":"2024-09-11T07:48:09.449370+0000","last_clean":"2024-09-11T07:48:09.449370+0000","last_became_active":"2024-09-11T07:45:57.201368+0000","last_became_peered":"2024-09-11T07:45:57.201368+0000","last_unstale":"2024-09-11T07:48:09.449370+0000","last_undegraded":"2024-09-11T07:48:09.449370+0000","last_fullsized":"2024-09-11T07:48:09.449370+0000","mapping_epoch":412,"log_start":"437'5500","ondisk_log_start":"437'5500","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3825,"log_dups_size":0,"ondisk_log_size":3825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":364773400,"num_objects":5567,"num_object_clones":0,"num_object_copies":16701,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5567,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9325,"num_write_kb":476481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"87'81","reported_seq":33,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.812549+0000","last_change":"2024-09-11T07:46:31.504183+0000","last_active":"2024-09-11T07:47:56.812549+0000","last_peered":"2024-09-11T07:47:56.812549+0000","last_clean":"2024-09-11T07:47:56.812549+0000","last_became_active":"2024-09-11T07:46:31.503868+0000","last_became_peered":"2024-09-11T07:46:31.503868+0000","last_unstale":"2024-09-11T07:47:56.812549+0000","last_undegraded":"2024-09-11T07:47:56.812549+0000","last_fullsized":"2024-09-11T07:47:56.812549+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:46:18.111609+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:46:18.111609+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015232676000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"426'30","reported_seq":366,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.809642+0000","last_change":"2024-09-11T07:46:08.202429+0000","last_active":"2024-09-11T07:47:56.809642+0000","last_peered":"2024-09-11T07:47:56.809642+0000","last_clean":"2024-09-11T07:47:56.809642+0000","last_became_active":"2024-09-11T07:45:27.119509+0000","last_became_peered":"2024-09-11T07:45:27.119509+0000","last_unstale":"2024-09-11T07:47:56.809642+0000","last_undegraded":"2024-09-11T07:47:56.809642+0000","last_fullsized":"2024-09-11T07:47:56.809642+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'20","last_scrub_stamp":"2024-09-11T07:46:08.202366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:46:08.202366+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024018009999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":95,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.807107+0000","last_change":"2024-09-11T07:45:12.553757+0000","last_active":"2024-09-11T07:47:56.807107+0000","last_peered":"2024-09-11T07:47:56.807107+0000","last_clean":"2024-09-11T07:47:56.807107+0000","last_became_active":"2024-09-11T07:44:23.771022+0000","last_became_peered":"2024-09-11T07:44:23.771022+0000","last_unstale":"2024-09-11T07:47:56.807107+0000","last_undegraded":"2024-09-11T07:47:56.807107+0000","last_fullsized":"2024-09-11T07:47:56.807107+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:45:12.551633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:12.551633+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014607669,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"290'9","reported_seq":762,"reported_epoch":444,"state":"active+clean","last_fresh":"2024-09-11T07:47:59.771407+0000","last_change":"2024-09-11T07:45:21.492545+0000","last_active":"2024-09-11T07:47:59.771407+0000","last_peered":"2024-09-11T07:47:59.771407+0000","last_clean":"2024-09-11T07:47:59.771407+0000","last_became_active":"2024-09-11T07:39:57.768155+0000","last_became_peered":"2024-09-11T07:39:57.768155+0000","last_unstale":"2024-09-11T07:47:59.771407+0000","last_undegraded":"2024-09-11T07:47:59.771407+0000","last_fullsized":"2024-09-11T07:47:59.771407+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"290'9","last_scrub_stamp":"2024-09-11T07:45:21.492460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:45:21.492460+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0074254259999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":479,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.807273+0000","last_change":"2024-09-11T07:46:17.190867+0000","last_active":"2024-09-11T07:47:56.807273+0000","last_peered":"2024-09-11T07:47:56.807273+0000","last_clean":"2024-09-11T07:47:56.807273+0000","last_became_active":"2024-09-11T07:26:37.567706+0000","last_became_peered":"2024-09-11T07:26:37.567706+0000","last_unstale":"2024-09-11T07:47:56.807273+0000","last_undegraded":"2024-09-11T07:47:56.807273+0000","last_fullsized":"2024-09-11T07:47:56.807273+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:17.190770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:54.148534+0000","last_clean_scrub_stamp":"2024-09-11T07:46:17.190770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007916569,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"444'9549","reported_seq":9589,"reported_epoch":444,"state":"active+clean","last_fresh":"2024-09-11T07:48:10.296319+0000","last_change":"2024-09-11T07:45:57.199626+0000","last_active":"2024-09-11T07:48:10.296319+0000","last_peered":"2024-09-11T07:48:10.296319+0000","last_clean":"2024-09-11T07:48:10.296319+0000","last_became_active":"2024-09-11T07:45:57.199305+0000","last_became_peered":"2024-09-11T07:45:57.199305+0000","last_unstale":"2024-09-11T07:48:10.296319+0000","last_undegraded":"2024-09-11T07:48:10.296319+0000","last_fullsized":"2024-09-11T07:48:10.296319+0000","mapping_epoch":412,"log_start":"430'4600","ondisk_log_start":"430'4600","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":4949,"log_dups_size":0,"ondisk_log_size":4949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":364838912,"num_objects":5567,"num_object_clones":0,"num_object_copies":16701,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9549,"num_write_kb":483712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"87'81","reported_seq":341,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.825469+0000","last_change":"2024-09-11T07:46:21.225100+0000","last_active":"2024-09-11T07:47:56.825469+0000","last_peered":"2024-09-11T07:47:56.825469+0000","last_clean":"2024-09-11T07:47:56.825469+0000","last_became_active":"2024-09-11T07:45:01.820927+0000","last_became_peered":"2024-09-11T07:45:01.820927+0000","last_unstale":"2024-09-11T07:47:56.825469+0000","last_undegraded":"2024-09-11T07:47:56.825469+0000","last_fullsized":"2024-09-11T07:47:56.825469+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:46:21.225015+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:46:21.225015+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050456870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"426'54","reported_seq":508,"reported_epoch":444,"state":"active+clean","last_fresh":"2024-09-11T07:47:59.778287+0000","last_change":"2024-09-11T07:46:17.005728+0000","last_active":"2024-09-11T07:47:59.778287+0000","last_peered":"2024-09-11T07:47:59.778287+0000","last_clean":"2024-09-11T07:47:59.778287+0000","last_became_active":"2024-09-11T07:39:57.768267+0000","last_became_peered":"2024-09-11T07:39:57.768267+0000","last_unstale":"2024-09-11T07:47:59.778287+0000","last_undegraded":"2024-09-11T07:47:59.778287+0000","last_fullsized":"2024-09-11T07:47:59.778287+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"290'42","last_scrub_stamp":"2024-09-11T07:46:17.005627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:46:17.005627+0000","objects_scrubbed":10,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0098261949999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":83,"num_read_kb":68,"num_write":50,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":462,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.815949+0000","last_change":"2024-09-11T07:47:01.254434+0000","last_active":"2024-09-11T07:47:56.815949+0000","last_peered":"2024-09-11T07:47:56.815949+0000","last_clean":"2024-09-11T07:47:56.815949+0000","last_became_active":"2024-09-11T07:44:23.772612+0000","last_became_peered":"2024-09-11T07:44:23.772612+0000","last_unstale":"2024-09-11T07:47:56.815949+0000","last_undegraded":"2024-09-11T07:47:56.815949+0000","last_fullsized":"2024-09-11T07:47:56.815949+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:01.254358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:36:51.308492+0000","last_clean_scrub_stamp":"2024-09-11T07:47:01.254358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:09.261035+0000","scrub_duration":0.0038796210000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":491,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.809092+0000","last_change":"2024-09-11T07:47:26.192696+0000","last_active":"2024-09-11T07:47:56.809092+0000","last_peered":"2024-09-11T07:47:56.809092+0000","last_clean":"2024-09-11T07:47:56.809092+0000","last_became_active":"2024-09-11T07:45:27.120021+0000","last_became_peered":"2024-09-11T07:45:27.120021+0000","last_unstale":"2024-09-11T07:47:56.809092+0000","last_undegraded":"2024-09-11T07:47:56.809092+0000","last_fullsized":"2024-09-11T07:47:56.809092+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:26.192608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:47:26.192608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:38.745195+0000","scrub_duration":0.004060386,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":496,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.804243+0000","last_change":"2024-09-11T07:45:27.103089+0000","last_active":"2024-09-11T07:47:56.804243+0000","last_peered":"2024-09-11T07:47:56.804243+0000","last_clean":"2024-09-11T07:47:56.804243+0000","last_became_active":"2024-09-11T07:45:27.102600+0000","last_became_peered":"2024-09-11T07:45:27.102600+0000","last_unstale":"2024-09-11T07:47:56.804243+0000","last_undegraded":"2024-09-11T07:47:56.804243+0000","last_fullsized":"2024-09-11T07:47:56.804243+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:13.504354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:45:13.504354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018451749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"444'9496","reported_seq":9536,"reported_epoch":444,"state":"active+clean","last_fresh":"2024-09-11T07:48:09.509358+0000","last_change":"2024-09-11T07:45:57.198992+0000","last_active":"2024-09-11T07:48:09.509358+0000","last_peered":"2024-09-11T07:48:09.509358+0000","last_clean":"2024-09-11T07:48:09.509358+0000","last_became_active":"2024-09-11T07:45:57.198709+0000","last_became_peered":"2024-09-11T07:45:57.198709+0000","last_unstale":"2024-09-11T07:48:09.509358+0000","last_undegraded":"2024-09-11T07:48:09.509358+0000","last_fullsized":"2024-09-11T07:48:09.509358+0000","mapping_epoch":412,"log_start":"437'5700","ondisk_log_start":"437'5700","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3796,"log_dups_size":0,"ondisk_log_size":3796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":367001600,"num_objects":5600,"num_object_clones":0,"num_object_copies":16800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9496,"num_write_kb":483072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"87'81","reported_seq":352,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.807343+0000","last_change":"2024-09-11T07:46:29.293393+0000","last_active":"2024-09-11T07:47:56.807343+0000","last_peered":"2024-09-11T07:47:56.807343+0000","last_clean":"2024-09-11T07:47:56.807343+0000","last_became_active":"2024-09-11T07:45:27.117020+0000","last_became_peered":"2024-09-11T07:45:27.117020+0000","last_unstale":"2024-09-11T07:47:56.807343+0000","last_undegraded":"2024-09-11T07:47:56.807343+0000","last_fullsized":"2024-09-11T07:47:56.807343+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:46:29.293202+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:46:29.293202+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030387169999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,5],"acting":[0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"426'26","reported_seq":313,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.822805+0000","last_change":"2024-09-11T07:45:49.765866+0000","last_active":"2024-09-11T07:47:56.822805+0000","last_peered":"2024-09-11T07:47:56.822805+0000","last_clean":"2024-09-11T07:47:56.822805+0000","last_became_active":"2024-09-11T07:45:49.765519+0000","last_became_peered":"2024-09-11T07:45:49.765519+0000","last_unstale":"2024-09-11T07:47:56.822805+0000","last_undegraded":"2024-09-11T07:47:56.822805+0000","last_fullsized":"2024-09-11T07:47:56.822805+0000","mapping_epoch":407,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":408,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'22","last_scrub_stamp":"2024-09-11T07:45:17.380652+0000","last_deep_scrub":"144'18","last_deep_scrub_stamp":"2024-09-11T07:31:51.603378+0000","last_clean_scrub_stamp":"2024-09-11T07:45:17.380652+0000","objects_scrubbed":5,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006530999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":41,"num_read_kb":30,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":340,"reported_epoch":444,"state":"active+clean","last_fresh":"2024-09-11T07:48:07.590285+0000","last_change":"2024-09-11T07:45:20.637602+0000","last_active":"2024-09-11T07:48:07.590285+0000","last_peered":"2024-09-11T07:48:07.590285+0000","last_clean":"2024-09-11T07:48:07.590285+0000","last_became_active":"2024-09-11T07:44:23.769220+0000","last_became_peered":"2024-09-11T07:44:23.769220+0000","last_unstale":"2024-09-11T07:48:07.590285+0000","last_undegraded":"2024-09-11T07:48:07.590285+0000","last_fullsized":"2024-09-11T07:48:07.590285+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'5","last_scrub_stamp":"2024-09-11T07:45:20.637535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:20.637535+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059316919999999997,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":231,"num_read_kb":226,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":504,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.809898+0000","last_change":"2024-09-11T07:45:07.727921+0000","last_active":"2024-09-11T07:47:56.809898+0000","last_peered":"2024-09-11T07:47:56.809898+0000","last_clean":"2024-09-11T07:47:56.809898+0000","last_became_active":"2024-09-11T07:26:31.413354+0000","last_became_peered":"2024-09-11T07:26:31.413354+0000","last_unstale":"2024-09-11T07:47:56.809898+0000","last_undegraded":"2024-09-11T07:47:56.809898+0000","last_fullsized":"2024-09-11T07:47:56.809898+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:07.727847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:07.149367+0000","last_clean_scrub_stamp":"2024-09-11T07:45:07.727847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053654319999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":501,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.808792+0000","last_change":"2024-09-11T07:47:01.841921+0000","last_active":"2024-09-11T07:47:56.808792+0000","last_peered":"2024-09-11T07:47:56.808792+0000","last_clean":"2024-09-11T07:47:56.808792+0000","last_became_active":"2024-09-11T07:32:00.508398+0000","last_became_peered":"2024-09-11T07:32:00.508398+0000","last_unstale":"2024-09-11T07:47:56.808792+0000","last_undegraded":"2024-09-11T07:47:56.808792+0000","last_fullsized":"2024-09-11T07:47:56.808792+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:01.841824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:28.246601+0000","last_clean_scrub_stamp":"2024-09-11T07:47:01.841824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:14.043657+0000","scrub_duration":0.039998764999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":479,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.814926+0000","last_change":"2024-09-11T07:47:22.781568+0000","last_active":"2024-09-11T07:47:56.814926+0000","last_peered":"2024-09-11T07:47:56.814926+0000","last_clean":"2024-09-11T07:47:56.814926+0000","last_became_active":"2024-09-11T07:32:00.515745+0000","last_became_peered":"2024-09-11T07:32:00.515745+0000","last_unstale":"2024-09-11T07:47:56.814926+0000","last_undegraded":"2024-09-11T07:47:56.814926+0000","last_fullsized":"2024-09-11T07:47:56.814926+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:22.781517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:25.382533+0000","last_clean_scrub_stamp":"2024-09-11T07:47:22.781517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:25.208018+0000","scrub_duration":0.0017426900000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"444'9512","reported_seq":9552,"reported_epoch":444,"state":"active+clean","last_fresh":"2024-09-11T07:48:10.292975+0000","last_change":"2024-09-11T07:45:57.202010+0000","last_active":"2024-09-11T07:48:10.292975+0000","last_peered":"2024-09-11T07:48:10.292975+0000","last_clean":"2024-09-11T07:48:10.292975+0000","last_became_active":"2024-09-11T07:45:57.201607+0000","last_became_peered":"2024-09-11T07:45:57.201607+0000","last_unstale":"2024-09-11T07:48:10.292975+0000","last_undegraded":"2024-09-11T07:48:10.292975+0000","last_fullsized":"2024-09-11T07:48:10.292975+0000","mapping_epoch":412,"log_start":"430'4500","ondisk_log_start":"430'4500","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":5012,"log_dups_size":0,"ondisk_log_size":5012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359792640,"num_objects":5490,"num_object_clones":0,"num_object_copies":16470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9512,"num_write_kb":480064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"87'81","reported_seq":69,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.810128+0000","last_change":"2024-09-11T07:47:10.225175+0000","last_active":"2024-09-11T07:47:56.810128+0000","last_peered":"2024-09-11T07:47:56.810128+0000","last_clean":"2024-09-11T07:47:56.810128+0000","last_became_active":"2024-09-11T07:45:27.114518+0000","last_became_peered":"2024-09-11T07:45:27.114518+0000","last_unstale":"2024-09-11T07:47:56.810128+0000","last_undegraded":"2024-09-11T07:47:56.810128+0000","last_fullsized":"2024-09-11T07:47:56.810128+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:47:10.225048+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:47:10.225048+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:33.482173+0000","scrub_duration":0.005009131,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"426'42","reported_seq":601,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.823029+0000","last_change":"2024-09-11T07:47:31.370736+0000","last_active":"2024-09-11T07:47:56.823029+0000","last_peered":"2024-09-11T07:47:56.823029+0000","last_clean":"2024-09-11T07:47:56.823029+0000","last_became_active":"2024-09-11T07:43:05.417321+0000","last_became_peered":"2024-09-11T07:43:05.417321+0000","last_unstale":"2024-09-11T07:47:56.823029+0000","last_undegraded":"2024-09-11T07:47:56.823029+0000","last_fullsized":"2024-09-11T07:47:56.823029+0000","mapping_epoch":358,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":359,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'42","last_scrub_stamp":"2024-09-11T07:47:31.370619+0000","last_deep_scrub":"426'42","last_deep_scrub_stamp":"2024-09-11T07:47:31.370619+0000","last_clean_scrub_stamp":"2024-09-11T07:47:31.370619+0000","objects_scrubbed":6,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:33.046778+0000","scrub_duration":0.039486856000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":54,"num_read_kb":48,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"422'2","reported_seq":520,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.822378+0000","last_change":"2024-09-11T07:46:22.213763+0000","last_active":"2024-09-11T07:47:56.822378+0000","last_peered":"2024-09-11T07:47:56.822378+0000","last_clean":"2024-09-11T07:47:56.822378+0000","last_became_active":"2024-09-11T07:45:27.115139+0000","last_became_peered":"2024-09-11T07:45:27.115139+0000","last_unstale":"2024-09-11T07:47:56.822378+0000","last_undegraded":"2024-09-11T07:47:56.822378+0000","last_fullsized":"2024-09-11T07:47:56.822378+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"422'2","last_scrub_stamp":"2024-09-11T07:46:22.213676+0000","last_deep_scrub":"422'2","last_deep_scrub_stamp":"2024-09-11T07:46:22.213676+0000","last_clean_scrub_stamp":"2024-09-11T07:46:22.213676+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077027509999999999,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":490,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.814108+0000","last_change":"2024-09-11T07:46:27.815229+0000","last_active":"2024-09-11T07:47:56.814108+0000","last_peered":"2024-09-11T07:47:56.814108+0000","last_clean":"2024-09-11T07:47:56.814108+0000","last_became_active":"2024-09-11T07:39:57.772244+0000","last_became_peered":"2024-09-11T07:39:57.772244+0000","last_unstale":"2024-09-11T07:47:56.814108+0000","last_undegraded":"2024-09-11T07:47:56.814108+0000","last_fullsized":"2024-09-11T07:47:56.814108+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:27.815152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:16.967492+0000","last_clean_scrub_stamp":"2024-09-11T07:46:27.815152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047261389999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":97,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.808706+0000","last_change":"2024-09-11T07:45:27.102656+0000","last_active":"2024-09-11T07:47:56.808706+0000","last_peered":"2024-09-11T07:47:56.808706+0000","last_clean":"2024-09-11T07:47:56.808706+0000","last_became_active":"2024-09-11T07:45:27.101352+0000","last_became_peered":"2024-09-11T07:45:27.101352+0000","last_unstale":"2024-09-11T07:47:56.808706+0000","last_undegraded":"2024-09-11T07:47:56.808706+0000","last_fullsized":"2024-09-11T07:47:56.808706+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:22.519491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:51.187127+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.519491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.024174253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":36,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.803073+0000","last_change":"2024-09-11T07:46:31.493189+0000","last_active":"2024-09-11T07:47:56.803073+0000","last_peered":"2024-09-11T07:47:56.803073+0000","last_clean":"2024-09-11T07:47:56.803073+0000","last_became_active":"2024-09-11T07:46:31.492869+0000","last_became_peered":"2024-09-11T07:46:31.492869+0000","last_unstale":"2024-09-11T07:47:56.803073+0000","last_undegraded":"2024-09-11T07:47:56.803073+0000","last_fullsized":"2024-09-11T07:47:56.803073+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:21.014717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:46:21.014717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"444'9657","reported_seq":9697,"reported_epoch":444,"state":"active+clean","last_fresh":"2024-09-11T07:48:10.294863+0000","last_change":"2024-09-11T07:45:57.199407+0000","last_active":"2024-09-11T07:48:10.294863+0000","last_peered":"2024-09-11T07:48:10.294863+0000","last_clean":"2024-09-11T07:48:10.294863+0000","last_became_active":"2024-09-11T07:45:57.199102+0000","last_became_peered":"2024-09-11T07:45:57.199102+0000","last_unstale":"2024-09-11T07:48:10.294863+0000","last_undegraded":"2024-09-11T07:48:10.294863+0000","last_fullsized":"2024-09-11T07:48:10.294863+0000","mapping_epoch":412,"log_start":"432'4700","ondisk_log_start":"432'4700","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":4957,"log_dups_size":0,"ondisk_log_size":4957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":368508928,"num_objects":5623,"num_object_clones":0,"num_object_copies":16869,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9657,"num_write_kb":488960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"426'45","reported_seq":185,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.815430+0000","last_change":"2024-09-11T07:45:32.442239+0000","last_active":"2024-09-11T07:47:56.815430+0000","last_peered":"2024-09-11T07:47:56.815430+0000","last_clean":"2024-09-11T07:47:56.815430+0000","last_became_active":"2024-09-11T07:45:27.113188+0000","last_became_peered":"2024-09-11T07:45:27.113188+0000","last_unstale":"2024-09-11T07:47:56.815430+0000","last_undegraded":"2024-09-11T07:47:56.815430+0000","last_fullsized":"2024-09-11T07:47:56.815430+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"399'33","last_scrub_stamp":"2024-09-11T07:45:32.442160+0000","last_deep_scrub":"399'33","last_deep_scrub_stamp":"2024-09-11T07:45:32.442160+0000","last_clean_scrub_stamp":"2024-09-11T07:45:32.442160+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015982277,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":59,"num_read_kb":49,"num_write":37,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"83'3","reported_seq":521,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.808581+0000","last_change":"2024-09-11T07:47:24.650786+0000","last_active":"2024-09-11T07:47:56.808581+0000","last_peered":"2024-09-11T07:47:56.808581+0000","last_clean":"2024-09-11T07:47:56.808581+0000","last_became_active":"2024-09-11T07:45:27.099666+0000","last_became_peered":"2024-09-11T07:45:27.099666+0000","last_unstale":"2024-09-11T07:47:56.808581+0000","last_undegraded":"2024-09-11T07:47:56.808581+0000","last_fullsized":"2024-09-11T07:47:56.808581+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-09-11T07:47:24.650681+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-09-11T07:28:01.711241+0000","last_clean_scrub_stamp":"2024-09-11T07:47:24.650681+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:45.847927+0000","scrub_duration":0.0033112549999999999,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":351,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.811705+0000","last_change":"2024-09-11T07:46:31.501022+0000","last_active":"2024-09-11T07:47:56.811705+0000","last_peered":"2024-09-11T07:47:56.811705+0000","last_clean":"2024-09-11T07:47:56.811705+0000","last_became_active":"2024-09-11T07:46:31.500613+0000","last_became_peered":"2024-09-11T07:46:31.500613+0000","last_unstale":"2024-09-11T07:47:56.811705+0000","last_undegraded":"2024-09-11T07:47:56.811705+0000","last_fullsized":"2024-09-11T07:47:56.811705+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:18.381749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:18.381749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013066169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"87'81","reported_seq":333,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.810448+0000","last_change":"2024-09-11T07:46:20.190045+0000","last_active":"2024-09-11T07:47:56.810448+0000","last_peered":"2024-09-11T07:47:56.810448+0000","last_clean":"2024-09-11T07:47:56.810448+0000","last_became_active":"2024-09-11T07:44:59.848971+0000","last_became_peered":"2024-09-11T07:44:59.848971+0000","last_unstale":"2024-09-11T07:47:56.810448+0000","last_undegraded":"2024-09-11T07:47:56.810448+0000","last_fullsized":"2024-09-11T07:47:56.810448+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:46:20.189949+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:46:20.189949+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0063253600000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":301,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.811384+0000","last_change":"2024-09-11T07:45:22.667614+0000","last_active":"2024-09-11T07:47:56.811384+0000","last_peered":"2024-09-11T07:47:56.811384+0000","last_clean":"2024-09-11T07:47:56.811384+0000","last_became_active":"2024-09-11T07:39:57.748658+0000","last_became_peered":"2024-09-11T07:39:57.748658+0000","last_unstale":"2024-09-11T07:47:56.811384+0000","last_undegraded":"2024-09-11T07:47:56.811384+0000","last_fullsized":"2024-09-11T07:47:56.811384+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:22.667519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:45:22.667519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046246309999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":456,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.804688+0000","last_change":"2024-09-11T07:46:31.509326+0000","last_active":"2024-09-11T07:47:56.804688+0000","last_peered":"2024-09-11T07:47:56.804688+0000","last_clean":"2024-09-11T07:47:56.804688+0000","last_became_active":"2024-09-11T07:46:31.509054+0000","last_became_peered":"2024-09-11T07:46:31.509054+0000","last_unstale":"2024-09-11T07:47:56.804688+0000","last_undegraded":"2024-09-11T07:47:56.804688+0000","last_fullsized":"2024-09-11T07:47:56.804688+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:14.389468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:45:14.389468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005315012,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.5","version":"444'9387","reported_seq":9427,"reported_epoch":444,"state":"active+clean","last_fresh":"2024-09-11T07:48:09.490093+0000","last_change":"2024-09-11T07:45:57.199301+0000","last_active":"2024-09-11T07:48:09.490093+0000","last_peered":"2024-09-11T07:48:09.490093+0000","last_clean":"2024-09-11T07:48:09.490093+0000","last_became_active":"2024-09-11T07:45:57.199035+0000","last_became_peered":"2024-09-11T07:45:57.199035+0000","last_unstale":"2024-09-11T07:48:09.490093+0000","last_undegraded":"2024-09-11T07:48:09.490093+0000","last_fullsized":"2024-09-11T07:48:09.490093+0000","mapping_epoch":412,"log_start":"437'5600","ondisk_log_start":"437'5600","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3787,"log_dups_size":0,"ondisk_log_size":3787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":363790336,"num_objects":5551,"num_object_clones":0,"num_object_copies":16653,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9387,"num_write_kb":478016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"426'51","reported_seq":153,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.816113+0000","last_change":"2024-09-11T07:47:01.594395+0000","last_active":"2024-09-11T07:47:56.816113+0000","last_peered":"2024-09-11T07:47:56.816113+0000","last_clean":"2024-09-11T07:47:56.816113+0000","last_became_active":"2024-09-11T07:46:31.494953+0000","last_became_peered":"2024-09-11T07:46:31.494953+0000","last_unstale":"2024-09-11T07:47:56.816113+0000","last_undegraded":"2024-09-11T07:47:56.816113+0000","last_fullsized":"2024-09-11T07:47:56.816113+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'51","last_scrub_stamp":"2024-09-11T07:47:01.594299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:47:01.594299+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:05.515667+0000","scrub_duration":0.0032010720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":78,"num_read_kb":67,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":497,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.811391+0000","last_change":"2024-09-11T07:45:10.747839+0000","last_active":"2024-09-11T07:47:56.811391+0000","last_peered":"2024-09-11T07:47:56.811391+0000","last_clean":"2024-09-11T07:47:56.811391+0000","last_became_active":"2024-09-11T07:26:31.419560+0000","last_became_peered":"2024-09-11T07:26:31.419560+0000","last_unstale":"2024-09-11T07:47:56.811391+0000","last_undegraded":"2024-09-11T07:47:56.811391+0000","last_fullsized":"2024-09-11T07:47:56.811391+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:10.747787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:10.747787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014756929999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":504,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.811639+0000","last_change":"2024-09-11T07:47:26.126014+0000","last_active":"2024-09-11T07:47:56.811639+0000","last_peered":"2024-09-11T07:47:56.811639+0000","last_clean":"2024-09-11T07:47:56.811639+0000","last_became_active":"2024-09-11T07:39:57.748953+0000","last_became_peered":"2024-09-11T07:39:57.748953+0000","last_unstale":"2024-09-11T07:47:56.811639+0000","last_undegraded":"2024-09-11T07:47:56.811639+0000","last_fullsized":"2024-09-11T07:47:56.811639+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:26.125938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:47:26.125938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:45.197855+0000","scrub_duration":0.0088559729999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":309,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.808490+0000","last_change":"2024-09-11T07:47:18.117007+0000","last_active":"2024-09-11T07:47:56.808490+0000","last_peered":"2024-09-11T07:47:56.808490+0000","last_clean":"2024-09-11T07:47:56.808490+0000","last_became_active":"2024-09-11T07:39:57.766605+0000","last_became_peered":"2024-09-11T07:39:57.766605+0000","last_unstale":"2024-09-11T07:47:56.808490+0000","last_undegraded":"2024-09-11T07:47:56.808490+0000","last_fullsized":"2024-09-11T07:47:56.808490+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:18.116773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:47:18.116773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:27.291773+0000","scrub_duration":0.004032056,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.4","version":"444'9579","reported_seq":9619,"reported_epoch":444,"state":"active+clean","last_fresh":"2024-09-11T07:48:09.536641+0000","last_change":"2024-09-11T07:45:57.201461+0000","last_active":"2024-09-11T07:48:09.536641+0000","last_peered":"2024-09-11T07:48:09.536641+0000","last_clean":"2024-09-11T07:48:09.536641+0000","last_became_active":"2024-09-11T07:45:57.201054+0000","last_became_peered":"2024-09-11T07:45:57.201054+0000","last_unstale":"2024-09-11T07:48:09.536641+0000","last_undegraded":"2024-09-11T07:48:09.536641+0000","last_fullsized":"2024-09-11T07:48:09.536641+0000","mapping_epoch":412,"log_start":"428'4100","ondisk_log_start":"428'4100","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":5479,"log_dups_size":0,"ondisk_log_size":5479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":370999296,"num_objects":5661,"num_object_clones":0,"num_object_copies":16983,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9579,"num_write_kb":487680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"426'21","reported_seq":527,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.810215+0000","last_change":"2024-09-11T07:45:27.145068+0000","last_active":"2024-09-11T07:47:56.810215+0000","last_peered":"2024-09-11T07:47:56.810215+0000","last_clean":"2024-09-11T07:47:56.810215+0000","last_became_active":"2024-09-11T07:45:27.114646+0000","last_became_peered":"2024-09-11T07:45:27.114646+0000","last_unstale":"2024-09-11T07:47:56.810215+0000","last_undegraded":"2024-09-11T07:47:56.810215+0000","last_fullsized":"2024-09-11T07:47:56.810215+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:45:11.521203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:45:11.521203+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015327073,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":76,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.802400+0000","last_change":"2024-09-11T07:46:16.505169+0000","last_active":"2024-09-11T07:47:56.802400+0000","last_peered":"2024-09-11T07:47:56.802400+0000","last_clean":"2024-09-11T07:47:56.802400+0000","last_became_active":"2024-09-11T07:45:27.102350+0000","last_became_peered":"2024-09-11T07:45:27.102350+0000","last_unstale":"2024-09-11T07:47:56.802400+0000","last_undegraded":"2024-09-11T07:47:56.802400+0000","last_fullsized":"2024-09-11T07:47:56.802400+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:16.505062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:46:16.505062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.037745665999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":510,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.808790+0000","last_change":"2024-09-11T07:46:12.107858+0000","last_active":"2024-09-11T07:47:56.808790+0000","last_peered":"2024-09-11T07:47:56.808790+0000","last_clean":"2024-09-11T07:47:56.808790+0000","last_became_active":"2024-09-11T07:46:12.107538+0000","last_became_peered":"2024-09-11T07:46:12.107538+0000","last_unstale":"2024-09-11T07:47:56.808790+0000","last_undegraded":"2024-09-11T07:47:56.808790+0000","last_fullsized":"2024-09-11T07:47:56.808790+0000","mapping_epoch":418,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":419,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:08.436552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:44:08.436552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0064625339999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":486,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.810870+0000","last_change":"2024-09-11T07:45:22.489678+0000","last_active":"2024-09-11T07:47:56.810870+0000","last_peered":"2024-09-11T07:47:56.810870+0000","last_clean":"2024-09-11T07:47:56.810870+0000","last_became_active":"2024-09-11T07:32:00.505185+0000","last_became_peered":"2024-09-11T07:32:00.505185+0000","last_unstale":"2024-09-11T07:47:56.810870+0000","last_undegraded":"2024-09-11T07:47:56.810870+0000","last_fullsized":"2024-09-11T07:47:56.810870+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:22.489595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:22.489595+0000","last_clean_scrub_stamp":"2024-09-11T07:45:22.489595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.045984707999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"444'9379","reported_seq":9419,"reported_epoch":444,"state":"active+clean","last_fresh":"2024-09-11T07:48:09.522258+0000","last_change":"2024-09-11T07:45:56.764280+0000","last_active":"2024-09-11T07:48:09.522258+0000","last_peered":"2024-09-11T07:48:09.522258+0000","last_clean":"2024-09-11T07:48:09.522258+0000","last_became_active":"2024-09-11T07:45:56.763089+0000","last_became_peered":"2024-09-11T07:45:56.763089+0000","last_unstale":"2024-09-11T07:48:09.522258+0000","last_undegraded":"2024-09-11T07:48:09.522258+0000","last_fullsized":"2024-09-11T07:48:09.522258+0000","mapping_epoch":412,"log_start":"437'5600","ondisk_log_start":"437'5600","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3779,"log_dups_size":0,"ondisk_log_size":3779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359071744,"num_objects":5479,"num_object_clones":0,"num_object_copies":16437,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9379,"num_write_kb":475456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"426'46","reported_seq":114,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.801738+0000","last_change":"2024-09-11T07:46:14.474268+0000","last_active":"2024-09-11T07:47:56.801738+0000","last_peered":"2024-09-11T07:47:56.801738+0000","last_clean":"2024-09-11T07:47:56.801738+0000","last_became_active":"2024-09-11T07:45:27.099653+0000","last_became_peered":"2024-09-11T07:45:27.099653+0000","last_unstale":"2024-09-11T07:47:56.801738+0000","last_undegraded":"2024-09-11T07:47:56.801738+0000","last_fullsized":"2024-09-11T07:47:56.801738+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"421'34","last_scrub_stamp":"2024-09-11T07:46:14.474174+0000","last_deep_scrub":"421'34","last_deep_scrub_stamp":"2024-09-11T07:46:14.474174+0000","last_clean_scrub_stamp":"2024-09-11T07:46:14.474174+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01054072,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":542,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.809061+0000","last_change":"2024-09-11T07:47:31.140172+0000","last_active":"2024-09-11T07:47:56.809061+0000","last_peered":"2024-09-11T07:47:56.809061+0000","last_clean":"2024-09-11T07:47:56.809061+0000","last_became_active":"2024-09-11T07:40:11.701330+0000","last_became_peered":"2024-09-11T07:40:11.701330+0000","last_unstale":"2024-09-11T07:47:56.809061+0000","last_undegraded":"2024-09-11T07:47:56.809061+0000","last_fullsized":"2024-09-11T07:47:56.809061+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:31.140083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.132552+0000","last_clean_scrub_stamp":"2024-09-11T07:47:31.140083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:43.623267+0000","scrub_duration":0.010667803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"395'11","reported_seq":714,"reported_epoch":444,"state":"active+clean","last_fresh":"2024-09-11T07:47:59.770285+0000","last_change":"2024-09-11T07:46:07.749662+0000","last_active":"2024-09-11T07:47:59.770285+0000","last_peered":"2024-09-11T07:47:59.770285+0000","last_clean":"2024-09-11T07:47:59.770285+0000","last_became_active":"2024-09-11T07:45:27.093333+0000","last_became_peered":"2024-09-11T07:45:27.093333+0000","last_unstale":"2024-09-11T07:47:59.770285+0000","last_undegraded":"2024-09-11T07:47:59.770285+0000","last_fullsized":"2024-09-11T07:47:59.770285+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"395'11","last_scrub_stamp":"2024-09-11T07:46:07.749598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:46:07.749598+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017864059999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":130,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.809884+0000","last_change":"2024-09-11T07:45:26.475689+0000","last_active":"2024-09-11T07:47:56.809884+0000","last_peered":"2024-09-11T07:47:56.809884+0000","last_clean":"2024-09-11T07:47:56.809884+0000","last_became_active":"2024-09-11T07:42:04.729103+0000","last_became_peered":"2024-09-11T07:42:04.729103+0000","last_unstale":"2024-09-11T07:47:56.809884+0000","last_undegraded":"2024-09-11T07:47:56.809884+0000","last_fullsized":"2024-09-11T07:47:56.809884+0000","mapping_epoch":346,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:26.475614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:45:26.475614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001709363,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"444'9281","reported_seq":9321,"reported_epoch":444,"state":"active+clean","last_fresh":"2024-09-11T07:48:09.536732+0000","last_change":"2024-09-11T07:45:57.199112+0000","last_active":"2024-09-11T07:48:09.536732+0000","last_peered":"2024-09-11T07:48:09.536732+0000","last_clean":"2024-09-11T07:48:09.536732+0000","last_became_active":"2024-09-11T07:45:57.198860+0000","last_became_peered":"2024-09-11T07:45:57.198860+0000","last_unstale":"2024-09-11T07:48:09.536732+0000","last_undegraded":"2024-09-11T07:48:09.536732+0000","last_fullsized":"2024-09-11T07:48:09.536732+0000","mapping_epoch":412,"log_start":"437'5500","ondisk_log_start":"437'5500","created":412,"last_epoch_clean":413,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:55.743218+0000","last_clean_scrub_stamp":"2024-09-11T07:45:55.743218+0000","objects_scrubbed":0,"log_size":3781,"log_dups_size":0,"ondisk_log_size":3781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":355401728,"num_objects":5423,"num_object_clones":0,"num_object_copies":16269,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9281,"num_write_kb":470528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"426'29","reported_seq":90,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.822737+0000","last_change":"2024-09-11T07:46:07.191286+0000","last_active":"2024-09-11T07:47:56.822737+0000","last_peered":"2024-09-11T07:47:56.822737+0000","last_clean":"2024-09-11T07:47:56.822737+0000","last_became_active":"2024-09-11T07:45:27.118858+0000","last_became_peered":"2024-09-11T07:45:27.118858+0000","last_unstale":"2024-09-11T07:47:56.822737+0000","last_undegraded":"2024-09-11T07:47:56.822737+0000","last_fullsized":"2024-09-11T07:47:56.822737+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'23","last_scrub_stamp":"2024-09-11T07:46:07.191227+0000","last_deep_scrub":"234'23","last_deep_scrub_stamp":"2024-09-11T07:37:34.613280+0000","last_clean_scrub_stamp":"2024-09-11T07:46:07.191227+0000","objects_scrubbed":5,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002042533,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":77,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.822556+0000","last_change":"2024-09-11T07:45:27.125454+0000","last_active":"2024-09-11T07:47:56.822556+0000","last_peered":"2024-09-11T07:47:56.822556+0000","last_clean":"2024-09-11T07:47:56.822556+0000","last_became_active":"2024-09-11T07:45:27.112394+0000","last_became_peered":"2024-09-11T07:45:27.112394+0000","last_unstale":"2024-09-11T07:47:56.822556+0000","last_undegraded":"2024-09-11T07:47:56.822556+0000","last_fullsized":"2024-09-11T07:47:56.822556+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:45:08.089525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:08.089525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":490,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.808657+0000","last_change":"2024-09-11T07:44:28.572130+0000","last_active":"2024-09-11T07:47:56.808657+0000","last_peered":"2024-09-11T07:47:56.808657+0000","last_clean":"2024-09-11T07:47:56.808657+0000","last_became_active":"2024-09-11T07:26:35.561967+0000","last_became_peered":"2024-09-11T07:26:35.561967+0000","last_unstale":"2024-09-11T07:47:56.808657+0000","last_undegraded":"2024-09-11T07:47:56.808657+0000","last_fullsized":"2024-09-11T07:47:56.808657+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:28.572059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:44.820354+0000","last_clean_scrub_stamp":"2024-09-11T07:44:28.572059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003042159,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":496,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.804706+0000","last_change":"2024-09-11T07:44:27.626846+0000","last_active":"2024-09-11T07:47:56.804706+0000","last_peered":"2024-09-11T07:47:56.804706+0000","last_clean":"2024-09-11T07:47:56.804706+0000","last_became_active":"2024-09-11T07:39:57.776576+0000","last_became_peered":"2024-09-11T07:39:57.776576+0000","last_unstale":"2024-09-11T07:47:56.804706+0000","last_undegraded":"2024-09-11T07:47:56.804706+0000","last_fullsized":"2024-09-11T07:47:56.804706+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:27.626761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:27.626761+0000","last_clean_scrub_stamp":"2024-09-11T07:44:27.626761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025844079999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"426'25","reported_seq":558,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.822940+0000","last_change":"2024-09-11T07:47:22.197270+0000","last_active":"2024-09-11T07:47:56.822940+0000","last_peered":"2024-09-11T07:47:56.822940+0000","last_clean":"2024-09-11T07:47:56.822940+0000","last_became_active":"2024-09-11T07:39:57.767661+0000","last_became_peered":"2024-09-11T07:39:57.767661+0000","last_unstale":"2024-09-11T07:47:56.822940+0000","last_undegraded":"2024-09-11T07:47:56.822940+0000","last_fullsized":"2024-09-11T07:47:56.822940+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'25","last_scrub_stamp":"2024-09-11T07:47:22.197145+0000","last_deep_scrub":"426'25","last_deep_scrub_stamp":"2024-09-11T07:47:22.197145+0000","last_clean_scrub_stamp":"2024-09-11T07:47:22.197145+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:42.553008+0000","scrub_duration":0.012355245000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":504,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.808950+0000","last_change":"2024-09-11T07:46:13.910021+0000","last_active":"2024-09-11T07:47:56.808950+0000","last_peered":"2024-09-11T07:47:56.808950+0000","last_clean":"2024-09-11T07:47:56.808950+0000","last_became_active":"2024-09-11T07:26:31.415701+0000","last_became_peered":"2024-09-11T07:26:31.415701+0000","last_unstale":"2024-09-11T07:47:56.808950+0000","last_undegraded":"2024-09-11T07:47:56.808950+0000","last_fullsized":"2024-09-11T07:47:56.808950+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:13.909932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:43:26.338227+0000","last_clean_scrub_stamp":"2024-09-11T07:46:13.909932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075390129999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":225,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.816252+0000","last_change":"2024-09-11T07:47:24.810130+0000","last_active":"2024-09-11T07:47:56.816252+0000","last_peered":"2024-09-11T07:47:56.816252+0000","last_clean":"2024-09-11T07:47:56.816252+0000","last_became_active":"2024-09-11T07:44:23.768703+0000","last_became_peered":"2024-09-11T07:44:23.768703+0000","last_unstale":"2024-09-11T07:47:56.816252+0000","last_undegraded":"2024-09-11T07:47:56.816252+0000","last_fullsized":"2024-09-11T07:47:56.816252+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:24.810051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:35.392360+0000","last_clean_scrub_stamp":"2024-09-11T07:47:24.810051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:43.535963+0000","scrub_duration":0.018560989999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":511,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.810360+0000","last_change":"2024-09-11T07:45:27.129839+0000","last_active":"2024-09-11T07:47:56.810360+0000","last_peered":"2024-09-11T07:47:56.810360+0000","last_clean":"2024-09-11T07:47:56.810360+0000","last_became_active":"2024-09-11T07:45:27.124181+0000","last_became_peered":"2024-09-11T07:45:27.124181+0000","last_unstale":"2024-09-11T07:47:56.810360+0000","last_undegraded":"2024-09-11T07:47:56.810360+0000","last_fullsized":"2024-09-11T07:47:56.810360+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:45:18.615900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:45:18.615900+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005478391,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":306,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.816516+0000","last_change":"2024-09-11T07:47:12.474748+0000","last_active":"2024-09-11T07:47:56.816516+0000","last_peered":"2024-09-11T07:47:56.816516+0000","last_clean":"2024-09-11T07:47:56.816516+0000","last_became_active":"2024-09-11T07:45:27.091114+0000","last_became_peered":"2024-09-11T07:45:27.091114+0000","last_unstale":"2024-09-11T07:47:56.816516+0000","last_undegraded":"2024-09-11T07:47:56.816516+0000","last_fullsized":"2024-09-11T07:47:56.816516+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:12.474673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:47:12.474673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:25.229073+0000","scrub_duration":0.0027412030000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"426'21","reported_seq":138,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.812637+0000","last_change":"2024-09-11T07:45:19.660354+0000","last_active":"2024-09-11T07:47:56.812637+0000","last_peered":"2024-09-11T07:47:56.812637+0000","last_clean":"2024-09-11T07:47:56.812637+0000","last_became_active":"2024-09-11T07:44:23.769896+0000","last_became_peered":"2024-09-11T07:44:23.769896+0000","last_unstale":"2024-09-11T07:47:56.812637+0000","last_undegraded":"2024-09-11T07:47:56.812637+0000","last_fullsized":"2024-09-11T07:47:56.812637+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'15","last_scrub_stamp":"2024-09-11T07:45:19.660273+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:45:19.660273+0000","objects_scrubbed":3,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032335910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":23,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":489,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.804099+0000","last_change":"2024-09-11T07:46:31.493250+0000","last_active":"2024-09-11T07:47:56.804099+0000","last_peered":"2024-09-11T07:47:56.804099+0000","last_clean":"2024-09-11T07:47:56.804099+0000","last_became_active":"2024-09-11T07:46:31.492884+0000","last_became_peered":"2024-09-11T07:46:31.492884+0000","last_unstale":"2024-09-11T07:47:56.804099+0000","last_undegraded":"2024-09-11T07:47:56.804099+0000","last_fullsized":"2024-09-11T07:47:56.804099+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:24.015330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:46:24.015330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069828010000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":487,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.815350+0000","last_change":"2024-09-11T07:47:04.742683+0000","last_active":"2024-09-11T07:47:56.815350+0000","last_peered":"2024-09-11T07:47:56.815350+0000","last_clean":"2024-09-11T07:47:56.815350+0000","last_became_active":"2024-09-11T07:45:27.112711+0000","last_became_peered":"2024-09-11T07:45:27.112711+0000","last_unstale":"2024-09-11T07:47:56.815350+0000","last_undegraded":"2024-09-11T07:47:56.815350+0000","last_fullsized":"2024-09-11T07:47:56.815350+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:04.742597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:47:04.742597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:19.493791+0000","scrub_duration":0.0056709050000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"426'35","reported_seq":572,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.804501+0000","last_change":"2024-09-11T07:47:26.690054+0000","last_active":"2024-09-11T07:47:56.804501+0000","last_peered":"2024-09-11T07:47:56.804501+0000","last_clean":"2024-09-11T07:47:56.804501+0000","last_became_active":"2024-09-11T07:39:57.771009+0000","last_became_peered":"2024-09-11T07:39:57.771009+0000","last_unstale":"2024-09-11T07:47:56.804501+0000","last_undegraded":"2024-09-11T07:47:56.804501+0000","last_fullsized":"2024-09-11T07:47:56.804501+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'35","last_scrub_stamp":"2024-09-11T07:47:26.689929+0000","last_deep_scrub":"234'25","last_deep_scrub_stamp":"2024-09-11T07:44:29.610392+0000","last_clean_scrub_stamp":"2024-09-11T07:47:26.689929+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:28.374571+0000","scrub_duration":0.0049798070000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":343,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.808319+0000","last_change":"2024-09-11T07:46:31.497548+0000","last_active":"2024-09-11T07:47:56.808319+0000","last_peered":"2024-09-11T07:47:56.808319+0000","last_clean":"2024-09-11T07:47:56.808319+0000","last_became_active":"2024-09-11T07:46:31.497105+0000","last_became_peered":"2024-09-11T07:46:31.497105+0000","last_unstale":"2024-09-11T07:47:56.808319+0000","last_undegraded":"2024-09-11T07:47:56.808319+0000","last_fullsized":"2024-09-11T07:47:56.808319+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:19.063698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:39:54.493614+0000","last_clean_scrub_stamp":"2024-09-11T07:46:19.063698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049403759999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"425'9","reported_seq":81,"reported_epoch":444,"state":"active+clean","last_fresh":"2024-09-11T07:47:59.774028+0000","last_change":"2024-09-11T07:47:21.208866+0000","last_active":"2024-09-11T07:47:59.774028+0000","last_peered":"2024-09-11T07:47:59.774028+0000","last_clean":"2024-09-11T07:47:59.774028+0000","last_became_active":"2024-09-11T07:46:31.499246+0000","last_became_peered":"2024-09-11T07:46:31.499246+0000","last_unstale":"2024-09-11T07:47:59.774028+0000","last_undegraded":"2024-09-11T07:47:59.774028+0000","last_fullsized":"2024-09-11T07:47:59.774028+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"425'9","last_scrub_stamp":"2024-09-11T07:47:21.208768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:47:21.208768+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:32.846447+0000","scrub_duration":0.0052892979999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":516,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.822480+0000","last_change":"2024-09-11T07:47:14.261937+0000","last_active":"2024-09-11T07:47:56.822480+0000","last_peered":"2024-09-11T07:47:56.822480+0000","last_clean":"2024-09-11T07:47:56.822480+0000","last_became_active":"2024-09-11T07:45:27.123325+0000","last_became_peered":"2024-09-11T07:45:27.123325+0000","last_unstale":"2024-09-11T07:47:56.822480+0000","last_undegraded":"2024-09-11T07:47:56.822480+0000","last_fullsized":"2024-09-11T07:47:56.822480+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:14.259694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:47:14.259694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:21.828842+0000","scrub_duration":0.010207048999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"426'25","reported_seq":508,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.815066+0000","last_change":"2024-09-11T07:47:00.685544+0000","last_active":"2024-09-11T07:47:56.815066+0000","last_peered":"2024-09-11T07:47:56.815066+0000","last_clean":"2024-09-11T07:47:56.815066+0000","last_became_active":"2024-09-11T07:43:10.448494+0000","last_became_peered":"2024-09-11T07:43:10.448494+0000","last_unstale":"2024-09-11T07:47:56.815066+0000","last_undegraded":"2024-09-11T07:47:56.815066+0000","last_fullsized":"2024-09-11T07:47:56.815066+0000","mapping_epoch":360,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":361,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'25","last_scrub_stamp":"2024-09-11T07:47:00.685485+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:37:41.338036+0000","last_clean_scrub_stamp":"2024-09-11T07:47:00.685485+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:17.529636+0000","scrub_duration":0.0065472430000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":488,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.810355+0000","last_change":"2024-09-11T07:45:27.117986+0000","last_active":"2024-09-11T07:47:56.810355+0000","last_peered":"2024-09-11T07:47:56.810355+0000","last_clean":"2024-09-11T07:47:56.810355+0000","last_became_active":"2024-09-11T07:45:27.114097+0000","last_became_peered":"2024-09-11T07:45:27.114097+0000","last_unstale":"2024-09-11T07:47:56.810355+0000","last_undegraded":"2024-09-11T07:47:56.810355+0000","last_fullsized":"2024-09-11T07:47:56.810355+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:09.537798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:45:09.537798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003422685,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"290'11","reported_seq":490,"reported_epoch":444,"state":"active+clean","last_fresh":"2024-09-11T07:47:59.770211+0000","last_change":"2024-09-11T07:47:12.818421+0000","last_active":"2024-09-11T07:47:59.770211+0000","last_peered":"2024-09-11T07:47:59.770211+0000","last_clean":"2024-09-11T07:47:59.770211+0000","last_became_active":"2024-09-11T07:39:57.772790+0000","last_became_peered":"2024-09-11T07:39:57.772790+0000","last_unstale":"2024-09-11T07:47:59.770211+0000","last_undegraded":"2024-09-11T07:47:59.770211+0000","last_fullsized":"2024-09-11T07:47:59.770211+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"290'11","last_scrub_stamp":"2024-09-11T07:47:12.818344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:47:12.818344+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:41.751212+0000","scrub_duration":0.022994837000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":62,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.812872+0000","last_change":"2024-09-11T07:45:27.104143+0000","last_active":"2024-09-11T07:47:56.812872+0000","last_peered":"2024-09-11T07:47:56.812872+0000","last_clean":"2024-09-11T07:47:56.812872+0000","last_became_active":"2024-09-11T07:45:27.101913+0000","last_became_peered":"2024-09-11T07:45:27.101913+0000","last_unstale":"2024-09-11T07:47:56.812872+0000","last_undegraded":"2024-09-11T07:47:56.812872+0000","last_fullsized":"2024-09-11T07:47:56.812872+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:21.026931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:45:21.026931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"426'22","reported_seq":486,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.809335+0000","last_change":"2024-09-11T07:45:14.682735+0000","last_active":"2024-09-11T07:47:56.809335+0000","last_peered":"2024-09-11T07:47:56.809335+0000","last_clean":"2024-09-11T07:47:56.809335+0000","last_became_active":"2024-09-11T07:39:57.689424+0000","last_became_peered":"2024-09-11T07:39:57.689424+0000","last_unstale":"2024-09-11T07:47:56.809335+0000","last_undegraded":"2024-09-11T07:47:56.809335+0000","last_fullsized":"2024-09-11T07:47:56.809335+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'16","last_scrub_stamp":"2024-09-11T07:45:14.682638+0000","last_deep_scrub":"236'16","last_deep_scrub_stamp":"2024-09-11T07:36:49.809484+0000","last_clean_scrub_stamp":"2024-09-11T07:45:14.682638+0000","objects_scrubbed":4,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014584720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":281,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.804577+0000","last_change":"2024-09-11T07:45:08.577358+0000","last_active":"2024-09-11T07:47:56.804577+0000","last_peered":"2024-09-11T07:47:56.804577+0000","last_clean":"2024-09-11T07:47:56.804577+0000","last_became_active":"2024-09-11T07:39:57.776841+0000","last_became_peered":"2024-09-11T07:39:57.776841+0000","last_unstale":"2024-09-11T07:47:56.804577+0000","last_undegraded":"2024-09-11T07:47:56.804577+0000","last_fullsized":"2024-09-11T07:47:56.804577+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:08.577262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:38.721118+0000","last_clean_scrub_stamp":"2024-09-11T07:45:08.577262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0073986299999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":477,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.804847+0000","last_change":"2024-09-11T07:47:05.721515+0000","last_active":"2024-09-11T07:47:56.804847+0000","last_peered":"2024-09-11T07:47:56.804847+0000","last_clean":"2024-09-11T07:47:56.804847+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:47:56.804847+0000","last_undegraded":"2024-09-11T07:47:56.804847+0000","last_fullsized":"2024-09-11T07:47:56.804847+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:05.721116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:47:05.721116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:08.290839+0000","scrub_duration":0.0069049100000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":489,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.807186+0000","last_change":"2024-09-11T07:46:19.170608+0000","last_active":"2024-09-11T07:47:56.807186+0000","last_peered":"2024-09-11T07:47:56.807186+0000","last_clean":"2024-09-11T07:47:56.807186+0000","last_became_active":"2024-09-11T07:39:57.761754+0000","last_became_peered":"2024-09-11T07:39:57.761754+0000","last_unstale":"2024-09-11T07:47:56.807186+0000","last_undegraded":"2024-09-11T07:47:56.807186+0000","last_fullsized":"2024-09-11T07:47:56.807186+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:19.170527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:46:19.170527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0099471420000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"426'32","reported_seq":321,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.815062+0000","last_change":"2024-09-11T07:47:00.575038+0000","last_active":"2024-09-11T07:47:56.815062+0000","last_peered":"2024-09-11T07:47:56.815062+0000","last_clean":"2024-09-11T07:47:56.815062+0000","last_became_active":"2024-09-11T07:46:31.494995+0000","last_became_peered":"2024-09-11T07:46:31.494995+0000","last_unstale":"2024-09-11T07:47:56.815062+0000","last_undegraded":"2024-09-11T07:47:56.815062+0000","last_fullsized":"2024-09-11T07:47:56.815062+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'32","last_scrub_stamp":"2024-09-11T07:47:00.574980+0000","last_deep_scrub":"234'24","last_deep_scrub_stamp":"2024-09-11T07:41:32.694510+0000","last_clean_scrub_stamp":"2024-09-11T07:47:00.574980+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:20.310086+0000","scrub_duration":0.001940443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":36,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":36,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.809211+0000","last_change":"2024-09-11T07:46:31.500716+0000","last_active":"2024-09-11T07:47:56.809211+0000","last_peered":"2024-09-11T07:47:56.809211+0000","last_clean":"2024-09-11T07:47:56.809211+0000","last_became_active":"2024-09-11T07:46:31.500431+0000","last_became_peered":"2024-09-11T07:46:31.500431+0000","last_unstale":"2024-09-11T07:47:56.809211+0000","last_undegraded":"2024-09-11T07:47:56.809211+0000","last_fullsized":"2024-09-11T07:47:56.809211+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:09.333312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:45:09.333312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":437,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.809507+0000","last_change":"2024-09-11T07:47:00.186555+0000","last_active":"2024-09-11T07:47:56.809507+0000","last_peered":"2024-09-11T07:47:56.809507+0000","last_clean":"2024-09-11T07:47:56.809507+0000","last_became_active":"2024-09-11T07:46:31.495137+0000","last_became_peered":"2024-09-11T07:46:31.495137+0000","last_unstale":"2024-09-11T07:47:56.809507+0000","last_undegraded":"2024-09-11T07:47:56.809507+0000","last_fullsized":"2024-09-11T07:47:56.809507+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:00.186475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:39:55.895853+0000","last_clean_scrub_stamp":"2024-09-11T07:47:00.186475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:11.814942+0000","scrub_duration":0.0053751830000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":208,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.811080+0000","last_change":"2024-09-11T07:45:24.519416+0000","last_active":"2024-09-11T07:47:56.811080+0000","last_peered":"2024-09-11T07:47:56.811080+0000","last_clean":"2024-09-11T07:47:56.811080+0000","last_became_active":"2024-09-11T07:40:11.700676+0000","last_became_peered":"2024-09-11T07:40:11.700676+0000","last_unstale":"2024-09-11T07:47:56.811080+0000","last_undegraded":"2024-09-11T07:47:56.811080+0000","last_fullsized":"2024-09-11T07:47:56.811080+0000","mapping_epoch":296,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":297,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:24.519137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:45:24.519137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077895680000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"426'14","reported_seq":503,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.802000+0000","last_change":"2024-09-11T07:47:20.651223+0000","last_active":"2024-09-11T07:47:56.802000+0000","last_peered":"2024-09-11T07:47:56.802000+0000","last_clean":"2024-09-11T07:47:56.802000+0000","last_became_active":"2024-09-11T07:26:33.545830+0000","last_became_peered":"2024-09-11T07:26:33.545830+0000","last_unstale":"2024-09-11T07:47:56.802000+0000","last_undegraded":"2024-09-11T07:47:56.802000+0000","last_fullsized":"2024-09-11T07:47:56.802000+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'14","last_scrub_stamp":"2024-09-11T07:47:20.650947+0000","last_deep_scrub":"234'10","last_deep_scrub_stamp":"2024-09-11T07:41:29.585284+0000","last_clean_scrub_stamp":"2024-09-11T07:47:20.650947+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:23.593209+0000","scrub_duration":0.022730186999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":16,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":32,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.809009+0000","last_change":"2024-09-11T07:46:31.503627+0000","last_active":"2024-09-11T07:47:56.809009+0000","last_peered":"2024-09-11T07:47:56.809009+0000","last_clean":"2024-09-11T07:47:56.809009+0000","last_became_active":"2024-09-11T07:46:31.503399+0000","last_became_peered":"2024-09-11T07:46:31.503399+0000","last_unstale":"2024-09-11T07:47:56.809009+0000","last_undegraded":"2024-09-11T07:47:56.809009+0000","last_fullsized":"2024-09-11T07:47:56.809009+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:17.132174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_clean_scrub_stamp":"2024-09-11T07:46:17.132174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":302,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.822207+0000","last_change":"2024-09-11T07:45:31.477617+0000","last_active":"2024-09-11T07:47:56.822207+0000","last_peered":"2024-09-11T07:47:56.822207+0000","last_clean":"2024-09-11T07:47:56.822207+0000","last_became_active":"2024-09-11T07:45:27.118170+0000","last_became_peered":"2024-09-11T07:45:27.118170+0000","last_unstale":"2024-09-11T07:47:56.822207+0000","last_undegraded":"2024-09-11T07:47:56.822207+0000","last_fullsized":"2024-09-11T07:47:56.822207+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:31.477552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:31.477552+0000","last_clean_scrub_stamp":"2024-09-11T07:45:31.477552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001479164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":489,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.814372+0000","last_change":"2024-09-11T07:47:17.866202+0000","last_active":"2024-09-11T07:47:56.814372+0000","last_peered":"2024-09-11T07:47:56.814372+0000","last_clean":"2024-09-11T07:47:56.814372+0000","last_became_active":"2024-09-11T07:45:27.097259+0000","last_became_peered":"2024-09-11T07:45:27.097259+0000","last_unstale":"2024-09-11T07:47:56.814372+0000","last_undegraded":"2024-09-11T07:47:56.814372+0000","last_fullsized":"2024-09-11T07:47:56.814372+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:17.866113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:47:17.866113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:36.838418+0000","scrub_duration":0.0064356079999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"426'27","reported_seq":545,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.801480+0000","last_change":"2024-09-11T07:47:17.662976+0000","last_active":"2024-09-11T07:47:56.801480+0000","last_peered":"2024-09-11T07:47:56.801480+0000","last_clean":"2024-09-11T07:47:56.801480+0000","last_became_active":"2024-09-11T07:39:57.767928+0000","last_became_peered":"2024-09-11T07:39:57.767928+0000","last_unstale":"2024-09-11T07:47:56.801480+0000","last_undegraded":"2024-09-11T07:47:56.801480+0000","last_fullsized":"2024-09-11T07:47:56.801480+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'27","last_scrub_stamp":"2024-09-11T07:47:17.662725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:47:17.662725+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:32.813407+0000","scrub_duration":0.011257233,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":36,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.813022+0000","last_change":"2024-09-11T07:46:31.504241+0000","last_active":"2024-09-11T07:47:56.813022+0000","last_peered":"2024-09-11T07:47:56.813022+0000","last_clean":"2024-09-11T07:47:56.813022+0000","last_became_active":"2024-09-11T07:46:31.503969+0000","last_became_peered":"2024-09-11T07:46:31.503969+0000","last_unstale":"2024-09-11T07:47:56.813022+0000","last_undegraded":"2024-09-11T07:47:56.813022+0000","last_fullsized":"2024-09-11T07:47:56.813022+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:11.315570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:48.366931+0000","last_clean_scrub_stamp":"2024-09-11T07:44:11.315570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":462,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.813972+0000","last_change":"2024-09-11T07:47:25.782162+0000","last_active":"2024-09-11T07:47:56.813972+0000","last_peered":"2024-09-11T07:47:56.813972+0000","last_clean":"2024-09-11T07:47:56.813972+0000","last_became_active":"2024-09-11T07:45:27.101251+0000","last_became_peered":"2024-09-11T07:45:27.101251+0000","last_unstale":"2024-09-11T07:47:56.813972+0000","last_undegraded":"2024-09-11T07:47:56.813972+0000","last_fullsized":"2024-09-11T07:47:56.813972+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:25.782104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:39.021876+0000","last_clean_scrub_stamp":"2024-09-11T07:47:25.782104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:43.730866+0000","scrub_duration":0.0017487259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":472,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.804580+0000","last_change":"2024-09-11T07:47:23.723242+0000","last_active":"2024-09-11T07:47:56.804580+0000","last_peered":"2024-09-11T07:47:56.804580+0000","last_clean":"2024-09-11T07:47:56.804580+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:47:56.804580+0000","last_undegraded":"2024-09-11T07:47:56.804580+0000","last_fullsized":"2024-09-11T07:47:56.804580+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:23.723143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:32.904006+0000","last_clean_scrub_stamp":"2024-09-11T07:47:23.723143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:25.416093+0000","scrub_duration":0.032919889000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":449,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.815094+0000","last_change":"2024-09-11T07:46:08.795262+0000","last_active":"2024-09-11T07:47:56.815094+0000","last_peered":"2024-09-11T07:47:56.815094+0000","last_clean":"2024-09-11T07:47:56.815094+0000","last_became_active":"2024-09-11T07:32:00.511258+0000","last_became_peered":"2024-09-11T07:32:00.511258+0000","last_unstale":"2024-09-11T07:47:56.815094+0000","last_undegraded":"2024-09-11T07:47:56.815094+0000","last_fullsized":"2024-09-11T07:47:56.815094+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:46:08.795209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:46:08.795209+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014785359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":495,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.808580+0000","last_change":"2024-09-11T07:46:31.503438+0000","last_active":"2024-09-11T07:47:56.808580+0000","last_peered":"2024-09-11T07:47:56.808580+0000","last_clean":"2024-09-11T07:47:56.808580+0000","last_became_active":"2024-09-11T07:46:31.503050+0000","last_became_peered":"2024-09-11T07:46:31.503050+0000","last_unstale":"2024-09-11T07:47:56.808580+0000","last_undegraded":"2024-09-11T07:47:56.808580+0000","last_fullsized":"2024-09-11T07:47:56.808580+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:44:22.300957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:44:22.300957+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0068790429999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"426'30","reported_seq":101,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.822295+0000","last_change":"2024-09-11T07:45:30.511577+0000","last_active":"2024-09-11T07:47:56.822295+0000","last_peered":"2024-09-11T07:47:56.822295+0000","last_clean":"2024-09-11T07:47:56.822295+0000","last_became_active":"2024-09-11T07:45:27.122864+0000","last_became_peered":"2024-09-11T07:45:27.122864+0000","last_unstale":"2024-09-11T07:47:56.822295+0000","last_undegraded":"2024-09-11T07:47:56.822295+0000","last_fullsized":"2024-09-11T07:47:56.822295+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'20","last_scrub_stamp":"2024-09-11T07:45:30.511477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:45:30.511477+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001786167,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"395'13","reported_seq":267,"reported_epoch":444,"state":"active+clean","last_fresh":"2024-09-11T07:47:59.770896+0000","last_change":"2024-09-11T07:45:31.645251+0000","last_active":"2024-09-11T07:47:59.770896+0000","last_peered":"2024-09-11T07:47:59.770896+0000","last_clean":"2024-09-11T07:47:59.770896+0000","last_became_active":"2024-09-11T07:45:27.105351+0000","last_became_peered":"2024-09-11T07:45:27.105351+0000","last_unstale":"2024-09-11T07:47:59.770896+0000","last_undegraded":"2024-09-11T07:47:59.770896+0000","last_fullsized":"2024-09-11T07:47:59.770896+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"395'13","last_scrub_stamp":"2024-09-11T07:45:31.645192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:45:31.645192+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016838580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3],"acting":[2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":71,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.810590+0000","last_change":"2024-09-11T07:46:31.502229+0000","last_active":"2024-09-11T07:47:56.810590+0000","last_peered":"2024-09-11T07:47:56.810590+0000","last_clean":"2024-09-11T07:47:56.810590+0000","last_became_active":"2024-09-11T07:46:31.501945+0000","last_became_peered":"2024-09-11T07:46:31.501945+0000","last_unstale":"2024-09-11T07:47:56.810590+0000","last_undegraded":"2024-09-11T07:47:56.810590+0000","last_fullsized":"2024-09-11T07:47:56.810590+0000","mapping_epoch":424,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":425,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:44:28.236557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:53.875300+0000","last_clean_scrub_stamp":"2024-09-11T07:44:28.236557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001550165,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":488,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.801758+0000","last_change":"2024-09-11T07:46:59.680297+0000","last_active":"2024-09-11T07:47:56.801758+0000","last_peered":"2024-09-11T07:47:56.801758+0000","last_clean":"2024-09-11T07:47:56.801758+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:47:56.801758+0000","last_undegraded":"2024-09-11T07:47:56.801758+0000","last_fullsized":"2024-09-11T07:47:56.801758+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:59.680093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:46:59.680093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:14.357775+0000","scrub_duration":0.014445909,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"426'27","reported_seq":57,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.804606+0000","last_change":"2024-09-11T07:46:13.812491+0000","last_active":"2024-09-11T07:47:56.804606+0000","last_peered":"2024-09-11T07:47:56.804606+0000","last_clean":"2024-09-11T07:47:56.804606+0000","last_became_active":"2024-09-11T07:46:13.790166+0000","last_became_peered":"2024-09-11T07:46:13.790166+0000","last_unstale":"2024-09-11T07:47:56.804606+0000","last_undegraded":"2024-09-11T07:47:56.804606+0000","last_fullsized":"2024-09-11T07:47:56.804606+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'19","last_scrub_stamp":"2024-09-11T07:45:23.537025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:45:23.537025+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033255630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":488,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.823120+0000","last_change":"2024-09-11T07:46:15.189821+0000","last_active":"2024-09-11T07:47:56.823120+0000","last_peered":"2024-09-11T07:47:56.823120+0000","last_clean":"2024-09-11T07:47:56.823120+0000","last_became_active":"2024-09-11T07:32:00.507050+0000","last_became_peered":"2024-09-11T07:32:00.507050+0000","last_unstale":"2024-09-11T07:47:56.823120+0000","last_undegraded":"2024-09-11T07:47:56.823120+0000","last_fullsized":"2024-09-11T07:47:56.823120+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:15.189711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:24.575210+0000","last_clean_scrub_stamp":"2024-09-11T07:46:15.189711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066152470000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":69,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.804364+0000","last_change":"2024-09-11T07:46:05.410699+0000","last_active":"2024-09-11T07:47:56.804364+0000","last_peered":"2024-09-11T07:47:56.804364+0000","last_clean":"2024-09-11T07:47:56.804364+0000","last_became_active":"2024-09-11T07:45:27.100903+0000","last_became_peered":"2024-09-11T07:45:27.100903+0000","last_unstale":"2024-09-11T07:47:56.804364+0000","last_undegraded":"2024-09-11T07:47:56.804364+0000","last_fullsized":"2024-09-11T07:47:56.804364+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:05.410607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:46:05.410607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016366040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":214,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.814313+0000","last_change":"2024-09-11T07:46:03.709201+0000","last_active":"2024-09-11T07:47:56.814313+0000","last_peered":"2024-09-11T07:47:56.814313+0000","last_clean":"2024-09-11T07:47:56.814313+0000","last_became_active":"2024-09-11T07:45:27.096591+0000","last_became_peered":"2024-09-11T07:45:27.096591+0000","last_unstale":"2024-09-11T07:47:56.814313+0000","last_undegraded":"2024-09-11T07:47:56.814313+0000","last_fullsized":"2024-09-11T07:47:56.814313+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:03.709095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:46:03.709095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001464166,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1c","version":"426'39","reported_seq":550,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.810717+0000","last_change":"2024-09-11T07:45:13.715138+0000","last_active":"2024-09-11T07:47:56.810717+0000","last_peered":"2024-09-11T07:47:56.810717+0000","last_clean":"2024-09-11T07:47:56.810717+0000","last_became_active":"2024-09-11T07:26:33.542544+0000","last_became_peered":"2024-09-11T07:26:33.542544+0000","last_unstale":"2024-09-11T07:47:56.810717+0000","last_undegraded":"2024-09-11T07:47:56.810717+0000","last_fullsized":"2024-09-11T07:47:56.810717+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'27","last_scrub_stamp":"2024-09-11T07:45:13.715050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:45:13.715050+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047085019999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":76,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.809047+0000","last_change":"2024-09-11T07:45:34.587277+0000","last_active":"2024-09-11T07:47:56.809047+0000","last_peered":"2024-09-11T07:47:56.809047+0000","last_clean":"2024-09-11T07:47:56.809047+0000","last_became_active":"2024-09-11T07:45:27.119977+0000","last_became_peered":"2024-09-11T07:45:27.119977+0000","last_unstale":"2024-09-11T07:47:56.809047+0000","last_undegraded":"2024-09-11T07:47:56.809047+0000","last_fullsized":"2024-09-11T07:47:56.809047+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:45:34.587193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:32.456852+0000","last_clean_scrub_stamp":"2024-09-11T07:45:34.587193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049811509999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":479,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.804821+0000","last_change":"2024-09-11T07:47:02.756784+0000","last_active":"2024-09-11T07:47:56.804821+0000","last_peered":"2024-09-11T07:47:56.804821+0000","last_clean":"2024-09-11T07:47:56.804821+0000","last_became_active":"2024-09-11T07:39:57.768994+0000","last_became_peered":"2024-09-11T07:39:57.768994+0000","last_unstale":"2024-09-11T07:47:56.804821+0000","last_undegraded":"2024-09-11T07:47:56.804821+0000","last_fullsized":"2024-09-11T07:47:56.804821+0000","mapping_epoch":288,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":289,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:02.756683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:47:02.756683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:09.869084+0000","scrub_duration":0.0090198880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":512,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.804960+0000","last_change":"2024-09-11T07:46:22.481200+0000","last_active":"2024-09-11T07:47:56.804960+0000","last_peered":"2024-09-11T07:47:56.804960+0000","last_clean":"2024-09-11T07:47:56.804960+0000","last_became_active":"2024-09-11T07:45:27.101964+0000","last_became_peered":"2024-09-11T07:45:27.101964+0000","last_unstale":"2024-09-11T07:47:56.804960+0000","last_undegraded":"2024-09-11T07:47:56.804960+0000","last_fullsized":"2024-09-11T07:47:56.804960+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:46:22.481110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:46:22.481110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005622022,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"426'29","reported_seq":541,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.802325+0000","last_change":"2024-09-11T07:45:27.139564+0000","last_active":"2024-09-11T07:47:56.802325+0000","last_peered":"2024-09-11T07:47:56.802325+0000","last_clean":"2024-09-11T07:47:56.802325+0000","last_became_active":"2024-09-11T07:45:27.095974+0000","last_became_peered":"2024-09-11T07:45:27.095974+0000","last_unstale":"2024-09-11T07:47:56.802325+0000","last_undegraded":"2024-09-11T07:47:56.802325+0000","last_fullsized":"2024-09-11T07:47:56.802325+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"234'23","last_scrub_stamp":"2024-09-11T07:45:24.422401+0000","last_deep_scrub":"234'23","last_deep_scrub_stamp":"2024-09-11T07:43:25.530826+0000","last_clean_scrub_stamp":"2024-09-11T07:45:24.422401+0000","objects_scrubbed":5,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0068140780000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":72,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-11T07:47:56.803232+0000","last_change":"2024-09-11T07:47:25.649232+0000","last_active":"2024-09-11T07:47:56.803232+0000","last_peered":"2024-09-11T07:47:56.803232+0000","last_clean":"2024-09-11T07:47:56.803232+0000","last_became_active":"2024-09-11T07:45:27.104493+0000","last_became_peered":"2024-09-11T07:45:27.104493+0000","last_unstale":"2024-09-11T07:47:56.803232+0000","last_undegraded":"2024-09-11T07:47:56.803232+0000","last_fullsized":"2024-09-11T07:47:56.803232+0000","mapping_epoch":394,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":395,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:47:25.649135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:47:25.649135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:48:54.433620+0000","scrub_duration":0.0087859719999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":5810749464,"num_objects":88666,"num_object_clones":0,"num_object_copies":265998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88666,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":150946,"num_write_kb":7667521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17478463488,"data_stored":17478451272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70546,"ondisk_log_size":70546,"up":48,"acting":48,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84,"ondisk_log_size":84,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1535,"num_read_kb":1292,"num_write":989,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":3510,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":475136,"data_stored":12690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1069,"ondisk_log_size":1069,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":231,"num_read_kb":226,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":644038,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2109440,"data_stored":2098784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":891,"ondisk_log_size":891,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":147,"seq":631360192707,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3432564,"kb_used_data":2513196,"kb_used_omap":4,"kb_used_meta":919355,"kb_avail":90312588,"statfs":{"total":95995035648,"available":92480090112,"internally_reserved":0,"allocated":2573512704,"data_stored":2573140958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":941420038},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:47:41 2024","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.532,"15min":1.6950000000000001},"min":{"1min":0.22800000000000001,"5min":0.21099999999999999,"15min":0.191},"max":{"1min":5.0999999999999996,"5min":20.832999999999998,"15min":75.677000000000007},"last":0.40799999999999997},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.452,"15min":1.7450000000000001},"min":{"1min":0.22,"5min":0.214,"15min":0.214},"max":{"1min":4.2560000000000002,"5min":20.937000000000001,"15min":75.638999999999996},"last":0.246}]},{"osd":1,"last update":"Wed Sep 11 07:47:41 2024","interfaces":[{"interface":"back","average":{"1min":5.1950000000000003,"5min":5.258,"15min":2.6659999999999999},"min":{"1min":0.27000000000000002,"5min":0.245,"15min":0.22500000000000001},"max":{"1min":91.602999999999994,"5min":327.06099999999998,"15min":327.06099999999998},"last":0.29299999999999998},{"interface":"front","average":{"1min":5.0039999999999996,"5min":5.1799999999999997,"15min":2.6659999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.22800000000000001},"max":{"1min":91.655000000000001,"5min":327.07499999999999,"15min":327.07499999999999},"last":0.35199999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:47:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":0.91000000000000003,"15min":1.8029999999999999},"min":{"1min":0.245,"5min":0.23699999999999999,"15min":0.22800000000000001},"max":{"1min":5.4329999999999998,"5min":6.4820000000000002,"15min":227.79300000000001},"last":0.373},{"interface":"front","average":{"1min":1.657,"5min":1.006,"15min":1.8620000000000001},"min":{"1min":0.28699999999999998,"5min":0.25,"15min":0.246},"max":{"1min":4.8040000000000003,"5min":6.6600000000000001,"15min":227.22},"last":0.38300000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:47:41 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":0.83899999999999997,"15min":1.2070000000000001},"min":{"1min":0.28100000000000003,"5min":0.252,"15min":0.252},"max":{"1min":4.2869999999999999,"5min":4.2869999999999999,"15min":49.798999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.84299999999999997,"15min":1.252},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":4.9550000000000001,"5min":6,"15min":49.524000000000001},"last":0.46999999999999997}]},{"osd":4,"last update":"Wed Sep 11 07:47:41 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.65600000000000003,"15min":0.70299999999999996},"min":{"1min":0.27800000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.90300000000000002,"5min":3.8319999999999999,"15min":6.0220000000000002},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.70599999999999996,"15min":0.73599999999999999},"min":{"1min":0.27700000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.359,"5min":3.3170000000000002,"15min":6.9000000000000004},"last":0.42499999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:47:41 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.67300000000000004,"15min":0.78200000000000003},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25},"max":{"1min":1.147,"5min":9.9339999999999993,"15min":9.9339999999999993},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.623,"15min":0.75800000000000001},"min":{"1min":0.27700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.327,"5min":10.183999999999999,"15min":10.183999999999999},"last":0.32500000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:47:41 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.56899999999999995,"15min":0.66200000000000003},"min":{"1min":0.318,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.2609999999999999,"5min":2.1190000000000002,"15min":5.1909999999999998},"last":0.439},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.64500000000000002,"15min":0.74099999999999999},"min":{"1min":0.32600000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.2370000000000001,"5min":2.6970000000000001,"15min":4.2030000000000003},"last":0.41599999999999998}]}]},{"osd":6,"up_from":44,"seq":188978561292,"num_pgs":86,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4491220,"kb_used_data":3602252,"kb_used_omap":1,"kb_used_meta":888958,"kb_avail":89253932,"statfs":{"total":95995035648,"available":91396026368,"internally_reserved":0,"allocated":3688706048,"data_stored":3688303346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":910293336},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:47:31 2024","interfaces":[{"interface":"back","average":{"1min":1.73,"5min":3.4820000000000002,"15min":2.399},"min":{"1min":0.23100000000000001,"5min":0.22900000000000001,"15min":0.20699999999999999},"max":{"1min":9.6869999999999994,"5min":130.62299999999999,"15min":142.29599999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":1.224,"5min":3.3690000000000002,"15min":2.5430000000000001},"min":{"1min":0.216,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":4.6680000000000001,"5min":130.86199999999999,"15min":142.68799999999999},"last":0.27200000000000002}]},{"osd":1,"last update":"Wed Sep 11 07:47:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":3.1110000000000002,"15min":2.415},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.23599999999999999},"max":{"1min":2.7970000000000002,"5min":131.42500000000001,"15min":131.42500000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":1.117,"5min":3.0990000000000002,"15min":2.2970000000000002},"min":{"1min":0.30199999999999999,"5min":0.22,"15min":0.22},"max":{"1min":5.4189999999999996,"5min":132.39599999999999,"15min":132.39599999999999},"last":0.42499999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:47:31 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":2.2570000000000001,"15min":1.3440000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.24299999999999999},"max":{"1min":5.2309999999999999,"5min":83.915999999999997,"15min":83.915999999999997},"last":13.429},{"interface":"front","average":{"1min":1.3979999999999999,"5min":2.371,"15min":1.472},"min":{"1min":0.372,"5min":0.254,"15min":0.254},"max":{"1min":2.9830000000000001,"5min":84.097999999999999,"15min":84.097999999999999},"last":10.763}]},{"osd":3,"last update":"Wed Sep 11 07:47:31 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3660000000000001,"15min":1.5509999999999999},"min":{"1min":0.315,"5min":0.30099999999999999,"15min":0.246},"max":{"1min":4.2119999999999997,"5min":39.308999999999997,"15min":49.109000000000002},"last":0.38600000000000001},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.4319999999999999,"15min":1.5669999999999999},"min":{"1min":0.28299999999999997,"5min":0.246,"15min":0.246},"max":{"1min":4.2889999999999997,"5min":41.149999999999999,"15min":48.963999999999999},"last":0.372}]},{"osd":4,"last update":"Wed Sep 11 07:47:31 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":1.135,"15min":1.768},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.223},"max":{"1min":0.52000000000000002,"5min":12.257999999999999,"15min":181.78800000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.39000000000000001,"5min":1.034,"15min":1.1830000000000001},"min":{"1min":0.309,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55200000000000005,"5min":12.102,"15min":128.59299999999999},"last":0.45100000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:47:31 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.71799999999999997,"15min":1.01},"min":{"1min":0.27400000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.59099999999999997,"5min":7.4219999999999997,"15min":57.436},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.82599999999999996,"15min":1.0640000000000001},"min":{"1min":0.30299999999999999,"5min":0.254,"15min":0.251},"max":{"1min":0.66700000000000004,"5min":7.4420000000000002,"15min":56.822000000000003},"last":0.436}]},{"osd":7,"last update":"Wed Sep 11 07:47:31 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":1.613,"15min":1.0920000000000001},"min":{"1min":0.372,"5min":0.32300000000000001,"15min":0.27800000000000002},"max":{"1min":0.79100000000000004,"5min":60.279000000000003,"15min":60.279000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.436,"5min":1.4390000000000001,"15min":1.0229999999999999},"min":{"1min":0.318,"5min":0.311,"15min":0.215},"max":{"1min":0.77100000000000002,"5min":60.767000000000003,"15min":60.767000000000003},"last":0.41299999999999998}]}]},{"osd":5,"up_from":37,"seq":158913790226,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1606900,"kb_used_data":723180,"kb_used_omap":1,"kb_used_meta":883710,"kb_avail":92138252,"statfs":{"total":95995035648,"available":94349570048,"internally_reserved":0,"allocated":740536320,"data_stored":740165139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":904919384},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:47:09 2024","interfaces":[{"interface":"back","average":{"1min":3.4969999999999999,"5min":1.5660000000000001,"15min":1.8580000000000001},"min":{"1min":0.25900000000000001,"5min":0.23100000000000001,"15min":0.20200000000000001},"max":{"1min":40.313000000000002,"5min":40.313000000000002,"15min":118.086},"last":1.0649999999999999},{"interface":"front","average":{"1min":3.5049999999999999,"5min":1.669,"15min":2.0150000000000001},"min":{"1min":0.29599999999999999,"5min":0.20899999999999999,"15min":0.20200000000000001},"max":{"1min":40.235999999999997,"5min":40.235999999999997,"15min":118.004},"last":0.315}]},{"osd":1,"last update":"Wed Sep 11 07:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.80200000000000005,"15min":1.5900000000000001},"min":{"1min":0.26800000000000002,"5min":0.23300000000000001,"15min":0.20799999999999999},"max":{"1min":4.2110000000000003,"5min":7.2510000000000003,"15min":122.738},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.755,"15min":1.617},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.22700000000000001},"max":{"1min":2.7839999999999998,"5min":3.476,"15min":122.66200000000001},"last":0.27100000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:47:09 2024","interfaces":[{"interface":"back","average":{"1min":2.73,"5min":1.5609999999999999,"15min":1.242},"min":{"1min":0.26700000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":26.707000000000001,"5min":26.707000000000001,"15min":26.707000000000001},"last":2.1499999999999999},{"interface":"front","average":{"1min":3.0840000000000001,"5min":1.484,"15min":1.1950000000000001},"min":{"1min":0.247,"5min":0.247,"15min":0.24399999999999999},"max":{"1min":26.666,"5min":26.666,"15min":26.666},"last":1.4730000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":2.8719999999999999,"15min":2.7879999999999998},"min":{"1min":0.30599999999999999,"5min":0.28899999999999998,"15min":0.26400000000000001},"max":{"1min":3.1269999999999998,"5min":132.715,"15min":132.715},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":2.7160000000000002,"15min":2.8570000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.25700000000000001},"max":{"1min":2.9710000000000001,"5min":133.01599999999999,"15min":133.01599999999999},"last":0.46600000000000003}]},{"osd":4,"last update":"Wed Sep 11 07:47:09 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.69099999999999995,"15min":0.77600000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.24199999999999999},"max":{"1min":0.51700000000000002,"5min":4.5090000000000003,"15min":11.103},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.64800000000000002,"15min":0.78600000000000003},"min":{"1min":0.30299999999999999,"5min":0.27300000000000002,"15min":0.22500000000000001},"max":{"1min":0.48999999999999999,"5min":3.7959999999999998,"15min":10.305},"last":0.51100000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:47:27 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.55200000000000005,"15min":0.72899999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.61099999999999999,"5min":2.4100000000000001,"15min":5.6859999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.59999999999999998,"15min":0.73799999999999999},"min":{"1min":0.33100000000000002,"5min":0.30299999999999999,"15min":0.26800000000000002},"max":{"1min":1.3700000000000001,"5min":3.0990000000000002,"15min":10.613},"last":0.443}]},{"osd":7,"last update":"Wed Sep 11 07:47:27 2024","interfaces":[{"interface":"back","average":{"1min":7.2080000000000002,"5min":2.3700000000000001,"15min":1.5489999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28000000000000003},"max":{"1min":115.44499999999999,"5min":115.44499999999999,"15min":115.44499999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":7.1890000000000001,"5min":2.3119999999999998,"15min":1.506},"min":{"1min":0.33900000000000002,"5min":0.29799999999999999,"15min":0.26800000000000002},"max":{"1min":115.071,"5min":115.071,"15min":115.071},"last":0.48599999999999999}]}]},{"osd":4,"up_from":0,"seq":133143986454,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":25,"seq":107374182683,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2334656,"kb_used_data":1423160,"kb_used_omap":1,"kb_used_meta":911358,"kb_avail":91410496,"statfs":{"total":95995035648,"available":93604347904,"internally_reserved":0,"allocated":1457315840,"data_stored":1456932067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":933230936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:47:37 2024","interfaces":[{"interface":"back","average":{"1min":1.8360000000000001,"5min":1.411,"15min":1.536},"min":{"1min":0.184,"5min":0.184,"15min":0.16600000000000001},"max":{"1min":3.8839999999999999,"5min":27.655999999999999,"15min":38.432000000000002},"last":0.95699999999999996},{"interface":"front","average":{"1min":2.036,"5min":1.4650000000000001,"15min":1.661},"min":{"1min":0.23400000000000001,"5min":0.183,"15min":0.183},"max":{"1min":4.5759999999999996,"5min":26.759,"15min":33.426000000000002},"last":2.3980000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:47:37 2024","interfaces":[{"interface":"back","average":{"1min":2.2589999999999999,"5min":1.2270000000000001,"15min":2.6899999999999999},"min":{"1min":0.30599999999999999,"5min":0.22500000000000001,"15min":0.19500000000000001},"max":{"1min":16.971,"5min":19.753,"15min":135.41999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":2.1400000000000001,"5min":1.2729999999999999,"15min":2.4380000000000002},"min":{"1min":0.222,"5min":0.221,"15min":0.192},"max":{"1min":18.277000000000001,"5min":19.484999999999999,"15min":136.126},"last":0.64400000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:47:37 2024","interfaces":[{"interface":"back","average":{"1min":1.7969999999999999,"5min":1.546,"15min":1.8160000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":4.5979999999999999,"5min":21.158000000000001,"15min":164.02600000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":2.593,"5min":1.5149999999999999,"15min":1.7330000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.222},"max":{"1min":10.752000000000001,"5min":17.265000000000001,"15min":163.37799999999999},"last":2.2029999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:47:56 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.1859999999999999,"15min":1.734},"min":{"1min":0.34100000000000003,"5min":0.314,"15min":0.27500000000000002},"max":{"1min":3.9119999999999999,"5min":8.6479999999999997,"15min":132.76400000000001},"last":0.441},{"interface":"front","average":{"1min":1.446,"5min":1.288,"15min":1.804},"min":{"1min":0.376,"5min":0.317,"15min":0.255},"max":{"1min":4.181,"5min":21.965,"15min":132.154},"last":0.47799999999999998}]},{"osd":5,"last update":"Wed Sep 11 07:48:08 2024","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":0.92300000000000004,"15min":1.45},"min":{"1min":0.36599999999999999,"5min":0.32500000000000001,"15min":0.28599999999999998},"max":{"1min":4.0279999999999996,"5min":4.0279999999999996,"15min":137.50299999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.0009999999999999,"15min":1.5469999999999999},"min":{"1min":0.40799999999999997,"5min":0.33000000000000002,"15min":0.28599999999999998},"max":{"1min":4.1139999999999999,"5min":4.1139999999999999,"15min":137.45599999999999},"last":0.51000000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:47:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":3.6960000000000002,"15min":2.028},"min":{"1min":0.35099999999999998,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":3.5019999999999998,"5min":233.98500000000001,"15min":233.98500000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":1.294,"5min":3.7890000000000001,"15min":2.0489999999999999},"min":{"1min":0.41299999999999998,"5min":0.35599999999999998,"15min":0.30399999999999999},"max":{"1min":3.98,"5min":234.05199999999999,"15min":234.05199999999999},"last":0.49399999999999999}]},{"osd":7,"last update":"Wed Sep 11 07:47:24 2024","interfaces":[{"interface":"back","average":{"1min":2.0249999999999999,"5min":3.4750000000000001,"15min":2.7360000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.308},"max":{"1min":13.935,"5min":162.42699999999999,"15min":162.42699999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":1.276,"5min":3.3079999999999998,"15min":2.734},"min":{"1min":0.40400000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.9380000000000002,"5min":162.39099999999999,"15min":162.39099999999999},"last":0.52400000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411616,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4464260,"kb_used_data":3567612,"kb_used_omap":1,"kb_used_meta":896574,"kb_avail":89280892,"statfs":{"total":95995035648,"available":91423633408,"internally_reserved":0,"allocated":3653234688,"data_stored":3652848139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":918092120},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:47:12 2024","interfaces":[{"interface":"back","average":{"1min":2.8570000000000002,"5min":1.754,"15min":2.012},"min":{"1min":0.25,"5min":0.188,"15min":0.16400000000000001},"max":{"1min":6.5970000000000004,"5min":19.907,"15min":57.802999999999997},"last":0.34100000000000003},{"interface":"front","average":{"1min":2.0590000000000002,"5min":1.6990000000000001,"15min":2.0670000000000002},"min":{"1min":0.249,"5min":0.188,"15min":0.188},"max":{"1min":5.7460000000000004,"5min":16.039999999999999,"15min":57.759},"last":0.35899999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:47:12 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":0.93600000000000005,"15min":1.4850000000000001},"min":{"1min":0.26500000000000001,"5min":0.22900000000000001,"15min":0.22800000000000001},"max":{"1min":3.6360000000000001,"5min":5.3570000000000002,"15min":65.153999999999996},"last":186.12899999999999},{"interface":"front","average":{"1min":2.2210000000000001,"5min":1.1200000000000001,"15min":1.77},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":8.9589999999999996,"5min":8.9589999999999996,"15min":63.311999999999998},"last":186.28999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:47:43 2024","interfaces":[{"interface":"back","average":{"1min":2.1389999999999998,"5min":1.3069999999999999,"15min":1.786},"min":{"1min":0.33900000000000002,"5min":0.22500000000000001,"15min":0.224},"max":{"1min":7.3760000000000003,"5min":7.3760000000000003,"15min":67.552999999999997},"last":0.38400000000000001},{"interface":"front","average":{"1min":2.9689999999999999,"5min":1.6140000000000001,"15min":1.8360000000000001},"min":{"1min":0.27200000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":7.3600000000000003,"5min":9.4670000000000005,"15min":67.352999999999994},"last":0.42899999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:47:53 2024","interfaces":[{"interface":"back","average":{"1min":2.254,"5min":1.498,"15min":1.0680000000000001},"min":{"1min":0.32100000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":6.5759999999999996,"5min":7.1719999999999997,"15min":7.1719999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":1.7869999999999999,"5min":1.1619999999999999,"15min":0.95399999999999996},"min":{"1min":0.41599999999999998,"5min":0.33800000000000002,"15min":0.313},"max":{"1min":7.2930000000000001,"5min":7.2930000000000001,"15min":7.2930000000000001},"last":0.754}]},{"osd":5,"last update":"Wed Sep 11 07:47:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.96999999999999997,"15min":1.0209999999999999},"min":{"1min":0.38100000000000001,"5min":0.371,"15min":0.32100000000000001},"max":{"1min":3.5630000000000002,"5min":3.8340000000000001,"15min":22.460000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.821,"5min":1.1819999999999999,"15min":1.159},"min":{"1min":0.47499999999999998,"5min":0.313,"15min":0.313},"max":{"1min":6.6310000000000002,"5min":6.6310000000000002,"15min":25.696999999999999},"last":0.90300000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:47:43 2024","interfaces":[{"interface":"back","average":{"1min":1.7430000000000001,"5min":1.25,"15min":1.149},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":7.2160000000000002,"5min":7.2160000000000002,"15min":22.399000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":2.2320000000000002,"5min":1.4810000000000001,"15min":1.25},"min":{"1min":0.34999999999999998,"5min":0.33500000000000002,"15min":0.29999999999999999},"max":{"1min":6.7060000000000004,"5min":6.7060000000000004,"15min":25.739000000000001},"last":0.83399999999999996}]},{"osd":7,"last update":"Wed Sep 11 07:47:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.1060000000000001,"15min":1.0549999999999999},"min":{"1min":0.43099999999999999,"5min":0.40600000000000003,"15min":0.34300000000000003},"max":{"1min":3.536,"5min":4.6950000000000003,"15min":22.484000000000002},"last":0.71399999999999997},{"interface":"front","average":{"1min":2.5230000000000001,"5min":1.3260000000000001,"15min":1.2649999999999999},"min":{"1min":0.52200000000000002,"5min":0.39000000000000001,"15min":0.36699999999999999},"max":{"1min":6.6760000000000002,"5min":6.6760000000000002,"15min":25.670000000000002},"last":0.80300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575139,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3047028,"kb_used_data":2138540,"kb_used_omap":1,"kb_used_meta":908478,"kb_avail":90698124,"statfs":{"total":95995035648,"available":92874878976,"internally_reserved":0,"allocated":2189864960,"data_stored":2189493238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":930281816},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:47:38 2024","interfaces":[{"interface":"back","average":{"1min":7.4249999999999998,"5min":2.694,"15min":1.9079999999999999},"min":{"1min":0.312,"5min":0.20799999999999999,"15min":0.16900000000000001},"max":{"1min":88.762,"5min":88.762,"15min":88.762},"last":0.47099999999999997},{"interface":"front","average":{"1min":6.8200000000000003,"5min":2.6680000000000001,"15min":1.952},"min":{"1min":0.24299999999999999,"5min":0.223,"15min":0.186},"max":{"1min":84.926000000000002,"5min":84.926000000000002,"15min":84.926000000000002},"last":0.376}]},{"osd":2,"last update":"Wed Sep 11 07:47:12 2024","interfaces":[{"interface":"back","average":{"1min":6.4480000000000004,"5min":3.2000000000000002,"15min":2.1779999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.19800000000000001},"max":{"1min":84.584000000000003,"5min":84.584000000000003,"15min":84.584000000000003},"last":5.4480000000000004},{"interface":"front","average":{"1min":6.3289999999999997,"5min":3.0470000000000002,"15min":2.109},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.191},"max":{"1min":84.656999999999996,"5min":84.656999999999996,"15min":84.656999999999996},"last":5.5099999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:47:30 2024","interfaces":[{"interface":"back","average":{"1min":23.048999999999999,"5min":5.5789999999999997,"15min":3.5379999999999998},"min":{"1min":0.32000000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":276.24900000000002,"5min":276.24900000000002,"15min":276.24900000000002},"last":3.0129999999999999},{"interface":"front","average":{"1min":22.138999999999999,"5min":5.2149999999999999,"15min":3.4060000000000001},"min":{"1min":0.33300000000000002,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":274.98000000000002,"5min":274.98000000000002,"15min":274.98000000000002},"last":0.55600000000000005}]},{"osd":4,"last update":"Wed Sep 11 07:47:48 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":1.1739999999999999,"15min":1.3779999999999999},"min":{"1min":0.32500000000000001,"5min":0.30299999999999999,"15min":0.27400000000000002},"max":{"1min":2.504,"5min":13.82,"15min":13.82},"last":0.30499999999999999},{"interface":"front","average":{"1min":1.589,"5min":1.323,"15min":1.397},"min":{"1min":0.35599999999999998,"5min":0.30099999999999999,"15min":0.29999999999999999},"max":{"1min":4.1520000000000001,"5min":13.801,"15min":13.801},"last":3.1589999999999998}]},{"osd":5,"last update":"Wed Sep 11 07:47:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":0.84799999999999998,"15min":1.333},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.26600000000000001},"max":{"1min":5.3109999999999999,"5min":5.3109999999999999,"15min":8.3680000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":1.343,"5min":0.94099999999999995,"15min":1.355},"min":{"1min":0.318,"5min":0.315,"15min":0.27200000000000002},"max":{"1min":4.1879999999999997,"5min":4.258,"15min":9.1929999999999996},"last":3.0990000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:47:30 2024","interfaces":[{"interface":"back","average":{"1min":1.6970000000000001,"5min":1.2070000000000001,"15min":1.331},"min":{"1min":0.318,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":4.0389999999999997,"5min":13.981999999999999,"15min":13.981999999999999},"last":3.1269999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.93500000000000005,"15min":1.105},"min":{"1min":0.38700000000000001,"5min":0.32700000000000001,"15min":0.313},"max":{"1min":1.8169999999999999,"5min":13.928000000000001,"15min":13.928000000000001},"last":0.51100000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:47:30 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76100000000000001,"15min":1.2},"min":{"1min":0.41499999999999998,"5min":0.36499999999999999,"15min":0.32200000000000001},"max":{"1min":1.7889999999999999,"5min":3.2690000000000001,"15min":9.3629999999999995},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.153,"15min":1.4930000000000001},"min":{"1min":0.375,"5min":0.372,"15min":0.313},"max":{"1min":2.766,"5min":13.715,"15min":13.715},"last":0.48899999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738665,"num_pgs":81,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4123016,"kb_used_data":3237952,"kb_used_omap":1,"kb_used_meta":885054,"kb_avail":89622136,"statfs":{"total":95995035648,"available":91773067264,"internally_reserved":0,"allocated":3315662848,"data_stored":3315270913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":906295640},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:47:37 2024","interfaces":[{"interface":"back","average":{"1min":1.946,"5min":1.7829999999999999,"15min":1.4770000000000001},"min":{"1min":0.35199999999999998,"5min":0.19600000000000001,"15min":0.17899999999999999},"max":{"1min":6.79,"5min":72.418000000000006,"15min":72.418000000000006},"last":2.8279999999999998},{"interface":"front","average":{"1min":2.0779999999999998,"5min":1.7829999999999999,"15min":1.508},"min":{"1min":0.377,"5min":0.17599999999999999,"15min":0.16600000000000001},"max":{"1min":6.3949999999999996,"5min":71.703999999999994,"15min":71.703999999999994},"last":0.88200000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:47:09 2024","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":3.718,"15min":2.883},"min":{"1min":0.26500000000000001,"5min":0.25900000000000001,"15min":0.188},"max":{"1min":4.3109999999999999,"5min":164.821,"15min":200.339},"last":2.6629999999999998},{"interface":"front","average":{"1min":1.696,"5min":3.9060000000000001,"15min":2.8380000000000001},"min":{"1min":0.30099999999999999,"5min":0.23999999999999999,"15min":0.20699999999999999},"max":{"1min":4.6500000000000004,"5min":167.68799999999999,"15min":200.40100000000001},"last":2.855}]},{"osd":3,"last update":"Wed Sep 11 07:47:37 2024","interfaces":[{"interface":"back","average":{"1min":1.8500000000000001,"5min":1.718,"15min":1.542},"min":{"1min":0.36799999999999999,"5min":0.255,"15min":0.20799999999999999},"max":{"1min":7.6130000000000004,"5min":37.143000000000001,"15min":37.143000000000001},"last":2.641},{"interface":"front","average":{"1min":2.1859999999999999,"5min":1.853,"15min":1.569},"min":{"1min":0.36299999999999999,"5min":0.253,"15min":0.22800000000000001},"max":{"1min":4.7039999999999997,"5min":38.198,"15min":38.198},"last":2.8879999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:47:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.2450000000000001,"15min":1.2949999999999999},"min":{"1min":0.42499999999999999,"5min":0.33400000000000002,"15min":0.26800000000000002},"max":{"1min":5.1420000000000003,"5min":7.0519999999999996,"15min":30.315000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.202,"15min":1.3520000000000001},"min":{"1min":0.36899999999999999,"5min":0.311,"15min":0.27000000000000002},"max":{"1min":4.5960000000000001,"5min":4.5960000000000001,"15min":30.363},"last":2.5529999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:47:09 2024","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.034,"15min":1.173},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.26800000000000002},"max":{"1min":4.2789999999999999,"5min":5.0119999999999996,"15min":6.9400000000000004},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.141,"15min":1.3169999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.27200000000000002},"max":{"1min":3.141,"5min":6.46,"15min":9.6289999999999996},"last":0.54100000000000004}]},{"osd":6,"last update":"Wed Sep 11 07:47:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.214,"15min":1.29},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.6829999999999998,"5min":6.5190000000000001,"15min":23.190999999999999},"last":2.5779999999999998},{"interface":"front","average":{"1min":1.615,"5min":1.224,"15min":1.3009999999999999},"min":{"1min":0.38700000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":4.6310000000000002,"5min":4.6310000000000002,"15min":23.632000000000001},"last":0.438}]},{"osd":7,"last update":"Wed Sep 11 07:47:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.6459999999999999,"15min":1.534},"min":{"1min":0.39600000000000002,"5min":0.375,"15min":0.34399999999999997},"max":{"1min":2.7959999999999998,"5min":34.704999999999998,"15min":34.704999999999998},"last":2.5019999999999998},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.6819999999999999,"15min":1.5529999999999999},"min":{"1min":0.41299999999999998,"5min":0.38800000000000001,"15min":0.315},"max":{"1min":3.8580000000000001,"5min":34.930999999999997,"15min":34.930999999999997},"last":0.47699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3273588736,"data_stored":3273588736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2186088448,"data_stored":2186084376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3633254400,"data_stored":3633250328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1452343296,"data_stored":1452343296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":734396416,"data_stored":734396416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3653832704,"data_stored":3653828632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2544959488,"data_stored":2544959488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:48:17.955 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pool set .mgr pgp_num 11 2024-09-11T07:48:17.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:17.956 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:17.956 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:18.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:17 smithi104 ceph-mon[33507]: pgmap v1012: 158 pgs: 158 active+clean; 5.1 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 962 op/s 2024-09-11T07:48:18.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:17 smithi104 ceph-mon[33507]: 6.7 scrub starts 2024-09-11T07:48:18.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:17 smithi104 ceph-mon[33507]: 6.7 scrub ok 2024-09-11T07:48:18.058 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:18.081 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:18.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:18.083 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:18.183 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:18.207 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:18.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:18.208 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:18.309 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:18.333 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:18.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:18.336 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:18.393 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-11T07:48:18.437 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:18.461 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:18.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:18.462 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:18.562 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:18.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:18.608 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:18.608 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:18.709 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:18.732 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:18.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:18.733 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:18.834 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:18.859 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:18.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:18.864 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:18.965 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:18.990 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:18.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:18.991 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:19.047 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:48:19.092 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:19.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:19.157 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:19.157 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:19.257 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:19.301 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:19.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:19.302 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:19.403 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:19.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:19.454 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:19.455 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:19.536 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T07:48:19.556 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:19.582 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:19.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:19.583 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:19.684 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:19.731 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:48:19.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:19.763 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:19.764 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:19.795 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:48:19.796 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:48:19.832 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:48:19.833 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:48:19.833 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:19 smithi022 ceph-mon[29462]: 5.a scrub starts 2024-09-11T07:48:19.833 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:19 smithi022 ceph-mon[29462]: 5.a scrub ok 2024-09-11T07:48:19.833 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:19 smithi022 ceph-mon[29462]: pgmap v1013: 158 pgs: 158 active+clean; 5.1 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 647 B/s rd, 0 B/s wr, 730 op/s 2024-09-11T07:48:19.833 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:19 smithi022 ceph-mon[29462]: 4.3 scrub starts 2024-09-11T07:48:19.833 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:19 smithi022 ceph-mon[29462]: 4.3 scrub ok 2024-09-11T07:48:19.834 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:19 smithi022 ceph-mon[37330]: 5.a scrub starts 2024-09-11T07:48:19.834 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:19 smithi022 ceph-mon[37330]: 5.a scrub ok 2024-09-11T07:48:19.834 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:19 smithi022 ceph-mon[37330]: pgmap v1013: 158 pgs: 158 active+clean; 5.1 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 647 B/s rd, 0 B/s wr, 730 op/s 2024-09-11T07:48:19.834 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:19 smithi022 ceph-mon[37330]: 4.3 scrub starts 2024-09-11T07:48:19.834 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:19 smithi022 ceph-mon[37330]: 4.3 scrub ok 2024-09-11T07:48:19.864 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:19.876 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:48:19.877 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:48:19.914 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:48:19.914 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:48:19.926 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:19.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:19.927 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:19.938 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:48:19.939 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:48:19.970 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:48:19.970 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:48:20.006 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:48:20.006 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:48:20.028 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:20.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:19 smithi104 ceph-mon[33507]: 5.a scrub starts 2024-09-11T07:48:20.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:19 smithi104 ceph-mon[33507]: 5.a scrub ok 2024-09-11T07:48:20.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:19 smithi104 ceph-mon[33507]: pgmap v1013: 158 pgs: 158 active+clean; 5.1 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 647 B/s rd, 0 B/s wr, 730 op/s 2024-09-11T07:48:20.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:19 smithi104 ceph-mon[33507]: 4.3 scrub starts 2024-09-11T07:48:20.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:19 smithi104 ceph-mon[33507]: 4.3 scrub ok 2024-09-11T07:48:20.044 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:48:20.044 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:48:20.097 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:20.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:20.098 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:20.199 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:20.228 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:20.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:20.229 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:20.330 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:20.357 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:20.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:20.358 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:20.458 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:20.485 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:20.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:20.486 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:20.582 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:48:20 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:48:20.587 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:20.614 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:20.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:20.615 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:20.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:48:20 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:48:20] "GET /metrics HTTP/1.1" 200 225121 "" "Prometheus/2.43.0" 2024-09-11T07:48:20.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:20 smithi022 ceph-mon[29462]: 11.4 scrub starts 2024-09-11T07:48:20.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:20 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/207329120' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:48:20.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:20 smithi022 ceph-mon[29462]: 11.e scrub starts 2024-09-11T07:48:20.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:20 smithi022 ceph-mon[37330]: 11.4 scrub starts 2024-09-11T07:48:20.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:20 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/207329120' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:48:20.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:20 smithi022 ceph-mon[37330]: 11.e scrub starts 2024-09-11T07:48:20.716 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:20.742 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:20.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:20.743 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:20.843 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:20.869 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:20.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:20.870 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:20.971 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:21.008 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:21.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:21.009 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:21.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:20 smithi104 ceph-mon[33507]: 11.4 scrub starts 2024-09-11T07:48:21.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:20 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/207329120' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:48:21.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:20 smithi104 ceph-mon[33507]: 11.e scrub starts 2024-09-11T07:48:21.110 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:21.143 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:21.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:21.144 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:21.245 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:21.269 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:21.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:21.270 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:21.371 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:21.396 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:21.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:21.397 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:21.498 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:21.525 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:21.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:21.526 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:21.627 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:21.652 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:21.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:21.653 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:21.754 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:21.779 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:21.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:21.780 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:21.881 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:21.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:21.907 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:21.908 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:21.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:21 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:48:21.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:21 smithi022 ceph-mon[29462]: pgmap v1014: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 4.7 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.42k op/s 2024-09-11T07:48:21.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:21 smithi022 ceph-mon[29462]: 11.4 scrub ok 2024-09-11T07:48:21.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:21 smithi022 ceph-mon[29462]: 2.1a scrub starts 2024-09-11T07:48:21.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:21 smithi022 ceph-mon[29462]: 2.1a scrub ok 2024-09-11T07:48:21.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:21 smithi022 ceph-mon[29462]: 4.1f scrub starts 2024-09-11T07:48:21.919 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:21 smithi022 ceph-mon[29462]: 4.1f scrub ok 2024-09-11T07:48:21.919 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:21 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:48:21.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:21 smithi022 ceph-mon[37330]: pgmap v1014: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 4.7 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.42k op/s 2024-09-11T07:48:21.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:21 smithi022 ceph-mon[37330]: 11.4 scrub ok 2024-09-11T07:48:21.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:21 smithi022 ceph-mon[37330]: 2.1a scrub starts 2024-09-11T07:48:21.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:21 smithi022 ceph-mon[37330]: 2.1a scrub ok 2024-09-11T07:48:21.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:21 smithi022 ceph-mon[37330]: 4.1f scrub starts 2024-09-11T07:48:21.920 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:21 smithi022 ceph-mon[37330]: 4.1f scrub ok 2024-09-11T07:48:22.009 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:22.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:21 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:48:22.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:21 smithi104 ceph-mon[33507]: pgmap v1014: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 4.7 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.42k op/s 2024-09-11T07:48:22.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:21 smithi104 ceph-mon[33507]: 11.4 scrub ok 2024-09-11T07:48:22.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:21 smithi104 ceph-mon[33507]: 2.1a scrub starts 2024-09-11T07:48:22.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:21 smithi104 ceph-mon[33507]: 2.1a scrub ok 2024-09-11T07:48:22.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:21 smithi104 ceph-mon[33507]: 4.1f scrub starts 2024-09-11T07:48:22.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:21 smithi104 ceph-mon[33507]: 4.1f scrub ok 2024-09-11T07:48:22.051 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:22.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:22.052 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:22.153 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:22.175 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:22.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:22.176 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:22.277 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:22.303 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:22.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:22.304 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:22.405 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:22.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:22.434 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:22.435 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:22.535 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:22.559 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:22.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:22.560 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:22.661 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:22.685 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:22.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:22.686 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:22.787 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:22.813 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:22.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:22.814 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:22.915 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:22.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:22 smithi022 ceph-mon[29462]: 2.4 scrub starts 2024-09-11T07:48:22.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:22 smithi022 ceph-mon[29462]: 2.4 scrub ok 2024-09-11T07:48:22.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:22 smithi022 ceph-mon[29462]: 11.e scrub ok 2024-09-11T07:48:22.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:22 smithi022 ceph-mon[29462]: 2.15 scrub starts 2024-09-11T07:48:22.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:22 smithi022 ceph-mon[29462]: 2.15 scrub ok 2024-09-11T07:48:22.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:22 smithi022 ceph-mon[29462]: 4.13 scrub starts 2024-09-11T07:48:22.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:22 smithi022 ceph-mon[29462]: 4.13 scrub ok 2024-09-11T07:48:22.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:22 smithi022 ceph-mon[37330]: 2.4 scrub starts 2024-09-11T07:48:22.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:22 smithi022 ceph-mon[37330]: 2.4 scrub ok 2024-09-11T07:48:22.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:22 smithi022 ceph-mon[37330]: 11.e scrub ok 2024-09-11T07:48:22.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:22 smithi022 ceph-mon[37330]: 2.15 scrub starts 2024-09-11T07:48:22.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:22 smithi022 ceph-mon[37330]: 2.15 scrub ok 2024-09-11T07:48:22.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:22 smithi022 ceph-mon[37330]: 4.13 scrub starts 2024-09-11T07:48:22.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:22 smithi022 ceph-mon[37330]: 4.13 scrub ok 2024-09-11T07:48:22.939 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:22.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:22.941 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:23.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:22 smithi104 ceph-mon[33507]: 2.4 scrub starts 2024-09-11T07:48:23.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:22 smithi104 ceph-mon[33507]: 2.4 scrub ok 2024-09-11T07:48:23.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:22 smithi104 ceph-mon[33507]: 11.e scrub ok 2024-09-11T07:48:23.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:22 smithi104 ceph-mon[33507]: 2.15 scrub starts 2024-09-11T07:48:23.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:22 smithi104 ceph-mon[33507]: 2.15 scrub ok 2024-09-11T07:48:23.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:22 smithi104 ceph-mon[33507]: 4.13 scrub starts 2024-09-11T07:48:23.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:22 smithi104 ceph-mon[33507]: 4.13 scrub ok 2024-09-11T07:48:23.041 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:23.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:23.093 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:23.094 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:23.195 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:23.218 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:23.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:23.220 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:23.320 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:23.359 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:23.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:23.360 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:23.461 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:23.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:23.499 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:23.499 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:23.598 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:23 smithi022 ceph-mon[37330]: pgmap v1015: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 4.7 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.15k op/s 2024-09-11T07:48:23.598 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:23 smithi022 ceph-mon[37330]: 6.d scrub starts 2024-09-11T07:48:23.598 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:23 smithi022 ceph-mon[37330]: 6.d scrub ok 2024-09-11T07:48:23.599 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:48:23.600 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:23.626 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:23.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:23.627 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:23.728 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:23.771 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:23.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:23.772 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:23.873 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:23.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:23.914 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:23.914 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:23.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:23 smithi022 ceph-mon[29462]: pgmap v1015: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 4.7 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.15k op/s 2024-09-11T07:48:23.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:23 smithi022 ceph-mon[29462]: 6.d scrub starts 2024-09-11T07:48:23.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:23 smithi022 ceph-mon[29462]: 6.d scrub ok 2024-09-11T07:48:23.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:48:24.015 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:24.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:23 smithi104 ceph-mon[33507]: pgmap v1015: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 4.7 GiB data, 20 GiB used, 605 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.15k op/s 2024-09-11T07:48:24.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:23 smithi104 ceph-mon[33507]: 6.d scrub starts 2024-09-11T07:48:24.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:23 smithi104 ceph-mon[33507]: 6.d scrub ok 2024-09-11T07:48:24.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:48:24.054 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:24.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:24.055 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:24.156 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:24.169 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:48:24.212 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:24.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:24.213 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:24.313 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:24.338 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:24.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:24.339 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:24.440 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:24.484 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:24.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:24.486 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:24.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:48:24 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:48:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:48:24.554 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:48:24.587 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:24.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:24.649 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:24.649 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:24.750 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:24.773 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:24.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:24.774 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:24.875 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:24.898 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:24.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:24.899 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:24.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:24 smithi022 ceph-mon[29462]: 2.18 scrub starts 2024-09-11T07:48:24.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:24 smithi022 ceph-mon[29462]: 2.18 scrub ok 2024-09-11T07:48:24.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:24 smithi022 ceph-mon[29462]: 4.b scrub starts 2024-09-11T07:48:24.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:24 smithi022 ceph-mon[29462]: 4.b scrub ok 2024-09-11T07:48:24.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:24 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:48:24.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:24 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:48:24.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:24 smithi022 ceph-mon[37330]: 2.18 scrub starts 2024-09-11T07:48:24.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:24 smithi022 ceph-mon[37330]: 2.18 scrub ok 2024-09-11T07:48:24.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:24 smithi022 ceph-mon[37330]: 4.b scrub starts 2024-09-11T07:48:24.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:24 smithi022 ceph-mon[37330]: 4.b scrub ok 2024-09-11T07:48:24.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:24 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:48:24.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:24 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:48:25.000 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:25.026 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:25.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:25.027 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:25.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:24 smithi104 ceph-mon[33507]: 2.18 scrub starts 2024-09-11T07:48:25.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:24 smithi104 ceph-mon[33507]: 2.18 scrub ok 2024-09-11T07:48:25.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:24 smithi104 ceph-mon[33507]: 4.b scrub starts 2024-09-11T07:48:25.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:24 smithi104 ceph-mon[33507]: 4.b scrub ok 2024-09-11T07:48:25.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:24 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:48:25.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:24 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:48:25.128 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:25.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:25.169 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:25.169 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:25.270 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:25.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:25.318 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:25.419 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:25.446 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:25.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:25.448 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:25.548 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:25.600 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:25.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:25.601 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:25.702 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:25.729 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:25.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:25.730 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:25.831 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:25.876 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:25.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:25.877 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:25.878 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:25 smithi022 ceph-mon[37330]: pgmap v1016: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 4.7 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.13k op/s 2024-09-11T07:48:25.879 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:25 smithi022 ceph-mon[37330]: osdmap e448: 8 total, 8 up, 7 in 2024-09-11T07:48:25.879 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:25 smithi022 ceph-mon[29462]: pgmap v1016: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 4.7 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.13k op/s 2024-09-11T07:48:25.879 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:25 smithi022 ceph-mon[29462]: osdmap e448: 8 total, 8 up, 7 in 2024-09-11T07:48:25.979 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:26.025 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:26.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:26.028 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:26.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:25 smithi104 ceph-mon[33507]: pgmap v1016: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 4.7 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.13k op/s 2024-09-11T07:48:26.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:25 smithi104 ceph-mon[33507]: osdmap e448: 8 total, 8 up, 7 in 2024-09-11T07:48:26.128 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:26.152 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:26.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:26.153 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:26.167 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:48:26.167 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:48:26.254 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:26.278 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:26.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:26.279 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:26.380 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:26.406 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:26.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:26.407 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:26.508 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:26.533 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:26.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:26.534 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:26.635 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:26.659 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:26.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:26.660 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:26.761 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:26.786 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:26.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:26.788 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:26.888 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:26.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:26 smithi022 ceph-mon[29462]: 6.11 scrub starts 2024-09-11T07:48:26.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:26 smithi022 ceph-mon[29462]: 6.11 scrub ok 2024-09-11T07:48:26.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:26 smithi022 ceph-mon[29462]: 5.9 scrub starts 2024-09-11T07:48:26.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:26 smithi022 ceph-mon[29462]: 5.9 scrub ok 2024-09-11T07:48:26.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:26 smithi022 ceph-mon[29462]: pgmap v1018: 158 pgs: 158 active+clean; 4.4 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.15k op/s 2024-09-11T07:48:26.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:26 smithi022 ceph-mon[37330]: 6.11 scrub starts 2024-09-11T07:48:26.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:26 smithi022 ceph-mon[37330]: 6.11 scrub ok 2024-09-11T07:48:26.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:26 smithi022 ceph-mon[37330]: 5.9 scrub starts 2024-09-11T07:48:26.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:26 smithi022 ceph-mon[37330]: 5.9 scrub ok 2024-09-11T07:48:26.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:26 smithi022 ceph-mon[37330]: pgmap v1018: 158 pgs: 158 active+clean; 4.4 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.15k op/s 2024-09-11T07:48:26.927 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:26.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:26.929 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:27.029 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:27.055 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:27.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:27.055 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:27.156 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:27.198 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:27.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:27.200 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:27.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:26 smithi104 ceph-mon[33507]: 6.11 scrub starts 2024-09-11T07:48:27.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:26 smithi104 ceph-mon[33507]: 6.11 scrub ok 2024-09-11T07:48:27.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:26 smithi104 ceph-mon[33507]: 5.9 scrub starts 2024-09-11T07:48:27.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:26 smithi104 ceph-mon[33507]: 5.9 scrub ok 2024-09-11T07:48:27.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:26 smithi104 ceph-mon[33507]: pgmap v1018: 158 pgs: 158 active+clean; 4.4 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.15k op/s 2024-09-11T07:48:27.301 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:27.326 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:27.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:27.328 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:27.429 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:27.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:27.456 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:27.457 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:27.557 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:27.583 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:27.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:27.584 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:27.685 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:27.709 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:27.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:27.710 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:27.811 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:27.849 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:27.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:27.850 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:27.951 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:27.976 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:27.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:27.977 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:27.988 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:27 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:48:27.988 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:27 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:48:27.988 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:27 smithi022 ceph-mon[37330]: osdmap e449: 8 total, 8 up, 7 in 2024-09-11T07:48:27.988 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:27 smithi022 ceph-mon[37330]: 2.11 scrub starts 2024-09-11T07:48:27.988 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:27 smithi022 ceph-mon[37330]: 2.11 scrub ok 2024-09-11T07:48:27.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:27 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:48:27.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:27 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:48:27.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:27 smithi022 ceph-mon[29462]: osdmap e449: 8 total, 8 up, 7 in 2024-09-11T07:48:27.988 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:27 smithi022 ceph-mon[29462]: 2.11 scrub starts 2024-09-11T07:48:27.989 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:27 smithi022 ceph-mon[29462]: 2.11 scrub ok 2024-09-11T07:48:28.078 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:28.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:28.102 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:28.102 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:28.203 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:28.228 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:28.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:28.229 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:28.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:27 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:48:28.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:27 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:48:28.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:27 smithi104 ceph-mon[33507]: osdmap e449: 8 total, 8 up, 7 in 2024-09-11T07:48:28.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:27 smithi104 ceph-mon[33507]: 2.11 scrub starts 2024-09-11T07:48:28.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:27 smithi104 ceph-mon[33507]: 2.11 scrub ok 2024-09-11T07:48:28.330 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:28.356 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:28.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:28.357 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:28.458 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:28.500 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:28.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:28.501 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:28.602 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:28.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:28.646 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:28.647 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:28.747 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:28.785 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:28.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:28.786 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:28.826 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is unset 2024-09-11T07:48:28.829 INFO:teuthology.orchestra.run.smithi022.stderr:set pool 1 pgp_num to 11 2024-09-11T07:48:28.851 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-11T07:48:28.887 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:28.937 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:28.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:28.938 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:29.039 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:29.063 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:29.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:29.064 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:29.087 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:28 smithi022 ceph-mon[29462]: 2.17 scrub starts 2024-09-11T07:48:29.088 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:28 smithi022 ceph-mon[29462]: 2.17 scrub ok 2024-09-11T07:48:29.088 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:28 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/4221750131' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]: dispatch 2024-09-11T07:48:29.088 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:28 smithi022 ceph-mon[29462]: from='client.26110 ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]: dispatch 2024-09-11T07:48:29.088 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:28 smithi022 ceph-mon[29462]: pgmap v1020: 158 pgs: 158 active+clean; 4.4 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 741 op/s 2024-09-11T07:48:29.088 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:28 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:48:29.088 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:28 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:48:29.088 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:28 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3988558414' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:48:29.088 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:28 smithi022 ceph-mon[29462]: from='client.26116 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:48:29.089 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:28 smithi022 ceph-mon[37330]: 2.17 scrub starts 2024-09-11T07:48:29.089 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:28 smithi022 ceph-mon[37330]: 2.17 scrub ok 2024-09-11T07:48:29.089 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:28 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/4221750131' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]: dispatch 2024-09-11T07:48:29.089 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:28 smithi022 ceph-mon[37330]: from='client.26110 ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]: dispatch 2024-09-11T07:48:29.089 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:28 smithi022 ceph-mon[37330]: pgmap v1020: 158 pgs: 158 active+clean; 4.4 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 741 op/s 2024-09-11T07:48:29.089 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:28 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:48:29.089 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:28 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:48:29.089 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:28 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3988558414' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:48:29.089 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:28 smithi022 ceph-mon[37330]: from='client.26116 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:48:29.165 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:29.213 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:29.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:29.214 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:29.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:28 smithi104 ceph-mon[33507]: 2.17 scrub starts 2024-09-11T07:48:29.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:28 smithi104 ceph-mon[33507]: 2.17 scrub ok 2024-09-11T07:48:29.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:28 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/4221750131' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]: dispatch 2024-09-11T07:48:29.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:28 smithi104 ceph-mon[33507]: from='client.26110 ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]: dispatch 2024-09-11T07:48:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:28 smithi104 ceph-mon[33507]: pgmap v1020: 158 pgs: 158 active+clean; 4.4 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 741 op/s 2024-09-11T07:48:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:28 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:48:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:28 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:48:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:28 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3988558414' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:48:29.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:28 smithi104 ceph-mon[33507]: from='client.26116 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:48:29.314 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:29.362 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:29.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:29.363 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:29.464 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:29.490 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:29.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:29.491 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:29.591 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:29.619 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:29.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:29.620 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:29.721 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:29.763 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:29.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:29.764 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:29.865 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:29.891 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:29.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:29.892 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:29.993 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:30.034 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:30.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:30.035 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:30.118 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:29 smithi022 ceph-mon[37330]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:48:30.118 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:29 smithi022 ceph-mon[37330]: Cluster is now healthy 2024-09-11T07:48:30.119 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:29 smithi022 ceph-mon[37330]: from='client.26110 ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]': finished 2024-09-11T07:48:30.119 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:29 smithi022 ceph-mon[37330]: from='client.26116 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:48:30.119 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:29 smithi022 ceph-mon[37330]: osdmap e450: 8 total, 8 up, 7 in 2024-09-11T07:48:30.119 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:29 smithi022 ceph-mon[37330]: 6.6 scrub starts 2024-09-11T07:48:30.119 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:29 smithi022 ceph-mon[37330]: 6.6 scrub ok 2024-09-11T07:48:30.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:29 smithi022 ceph-mon[29462]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:48:30.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:29 smithi022 ceph-mon[29462]: Cluster is now healthy 2024-09-11T07:48:30.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:29 smithi022 ceph-mon[29462]: from='client.26110 ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]': finished 2024-09-11T07:48:30.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:29 smithi022 ceph-mon[29462]: from='client.26116 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:48:30.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:29 smithi022 ceph-mon[29462]: osdmap e450: 8 total, 8 up, 7 in 2024-09-11T07:48:30.119 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:29 smithi022 ceph-mon[29462]: 6.6 scrub starts 2024-09-11T07:48:30.120 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:29 smithi022 ceph-mon[29462]: 6.6 scrub ok 2024-09-11T07:48:30.136 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:30.163 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:48:30.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:30.198 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:30.199 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:30.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:29 smithi104 ceph-mon[33507]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:48:30.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:29 smithi104 ceph-mon[33507]: Cluster is now healthy 2024-09-11T07:48:30.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:29 smithi104 ceph-mon[33507]: from='client.26110 ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]': finished 2024-09-11T07:48:30.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:29 smithi104 ceph-mon[33507]: from='client.26116 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:48:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:29 smithi104 ceph-mon[33507]: osdmap e450: 8 total, 8 up, 7 in 2024-09-11T07:48:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:29 smithi104 ceph-mon[33507]: 6.6 scrub starts 2024-09-11T07:48:30.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:29 smithi104 ceph-mon[33507]: 6.6 scrub ok 2024-09-11T07:48:30.299 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:30.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:30.355 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:30.355 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:30.456 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:30.496 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:30.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:30.497 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:30.598 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:30.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:30.635 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:30.635 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:30.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:48:30 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:48:30] "GET /metrics HTTP/1.1" 200 225152 "" "Prometheus/2.43.0" 2024-09-11T07:48:30.736 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:30.775 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:30.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:30.776 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:30.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:48:30 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:48:30.876 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:30.904 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:30.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:30.905 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:31.005 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:31.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:31.031 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:31.031 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:31.132 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:31.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:30 smithi022 ceph-mon[29462]: 6.1c deep-scrub starts 2024-09-11T07:48:31.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:30 smithi022 ceph-mon[29462]: 6.1c deep-scrub ok 2024-09-11T07:48:31.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:30 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:48:31.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:30 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:48:31.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:30 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:48:31.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:30 smithi022 ceph-mon[29462]: pgmap v1022: 158 pgs: 158 active+clean; 4.0 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.96k op/s 2024-09-11T07:48:31.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:30 smithi022 ceph-mon[37330]: 6.1c deep-scrub starts 2024-09-11T07:48:31.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:30 smithi022 ceph-mon[37330]: 6.1c deep-scrub ok 2024-09-11T07:48:31.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:30 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:48:31.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:30 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:48:31.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:30 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:48:31.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:30 smithi022 ceph-mon[37330]: pgmap v1022: 158 pgs: 158 active+clean; 4.0 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.96k op/s 2024-09-11T07:48:31.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:31.169 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:31.169 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:31.270 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:31.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:30 smithi104 ceph-mon[33507]: 6.1c deep-scrub starts 2024-09-11T07:48:31.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:30 smithi104 ceph-mon[33507]: 6.1c deep-scrub ok 2024-09-11T07:48:31.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:30 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:48:31.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:30 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:48:31.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:30 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:48:31.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:30 smithi104 ceph-mon[33507]: pgmap v1022: 158 pgs: 158 active+clean; 4.0 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.96k op/s 2024-09-11T07:48:31.297 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:31.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:31.298 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:31.399 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:31.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:31.423 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:31.423 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:31.524 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:31.550 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:31.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:31.551 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:31.652 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:31.688 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:31.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:31.689 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:31.762 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T07:48:31.790 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:31.858 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:31.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:31.859 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:31.960 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:31.984 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:31.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:31.986 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:32.086 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:32.112 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:32.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:32.113 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:32.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:31 smithi022 ceph-mon[29462]: 5.8 deep-scrub starts 2024-09-11T07:48:32.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:31 smithi022 ceph-mon[29462]: 5.8 deep-scrub ok 2024-09-11T07:48:32.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:31 smithi022 ceph-mon[29462]: osdmap e451: 8 total, 8 up, 7 in 2024-09-11T07:48:32.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:31 smithi022 ceph-mon[29462]: 6.c deep-scrub starts 2024-09-11T07:48:32.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:31 smithi022 ceph-mon[29462]: 6.c deep-scrub ok 2024-09-11T07:48:32.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:31 smithi022 ceph-mon[37330]: 5.8 deep-scrub starts 2024-09-11T07:48:32.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:31 smithi022 ceph-mon[37330]: 5.8 deep-scrub ok 2024-09-11T07:48:32.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:31 smithi022 ceph-mon[37330]: osdmap e451: 8 total, 8 up, 7 in 2024-09-11T07:48:32.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:31 smithi022 ceph-mon[37330]: 6.c deep-scrub starts 2024-09-11T07:48:32.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:31 smithi022 ceph-mon[37330]: 6.c deep-scrub ok 2024-09-11T07:48:32.214 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:32.239 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:32.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:32.240 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:32.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:31 smithi104 ceph-mon[33507]: 5.8 deep-scrub starts 2024-09-11T07:48:32.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:31 smithi104 ceph-mon[33507]: 5.8 deep-scrub ok 2024-09-11T07:48:32.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:31 smithi104 ceph-mon[33507]: osdmap e451: 8 total, 8 up, 7 in 2024-09-11T07:48:32.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:31 smithi104 ceph-mon[33507]: 6.c deep-scrub starts 2024-09-11T07:48:32.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:31 smithi104 ceph-mon[33507]: 6.c deep-scrub ok 2024-09-11T07:48:32.341 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:32.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:32.384 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:32.384 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:32.485 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:32.510 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:32.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:32.511 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:32.612 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:32.639 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:32.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:32.640 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:32.740 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:32.767 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:32.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:32.768 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:32.869 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:32.896 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:32.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:32.897 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:32.997 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:33.022 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:33.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:33.024 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:33.124 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:33.148 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:33.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:33.149 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:33.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:32 smithi022 ceph-mon[29462]: 2.8 scrub starts 2024-09-11T07:48:33.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:32 smithi022 ceph-mon[29462]: 2.8 scrub ok 2024-09-11T07:48:33.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:32 smithi022 ceph-mon[29462]: 6.1d scrub starts 2024-09-11T07:48:33.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:32 smithi022 ceph-mon[29462]: pgmap v1024: 158 pgs: 158 active+clean; 4.0 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 973 op/s 2024-09-11T07:48:33.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:32 smithi022 ceph-mon[37330]: 2.8 scrub starts 2024-09-11T07:48:33.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:32 smithi022 ceph-mon[37330]: 2.8 scrub ok 2024-09-11T07:48:33.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:32 smithi022 ceph-mon[37330]: 6.1d scrub starts 2024-09-11T07:48:33.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:32 smithi022 ceph-mon[37330]: pgmap v1024: 158 pgs: 158 active+clean; 4.0 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 973 op/s 2024-09-11T07:48:33.250 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:33.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:32 smithi104 ceph-mon[33507]: 2.8 scrub starts 2024-09-11T07:48:33.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:32 smithi104 ceph-mon[33507]: 2.8 scrub ok 2024-09-11T07:48:33.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:32 smithi104 ceph-mon[33507]: 6.1d scrub starts 2024-09-11T07:48:33.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:32 smithi104 ceph-mon[33507]: pgmap v1024: 158 pgs: 158 active+clean; 4.0 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 973 op/s 2024-09-11T07:48:33.300 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:33.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:33.301 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:33.402 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:33.441 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:33.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:33.442 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:33.543 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:33.583 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:33.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:33.584 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:33.685 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:33.715 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:33.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:33.716 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:33.817 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:33.857 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:33.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:33.858 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:33.916 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:33 smithi022 ceph-mon[37330]: 6.1d scrub ok 2024-09-11T07:48:33.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:33 smithi022 ceph-mon[37330]: 11.f scrub starts 2024-09-11T07:48:33.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:33 smithi022 ceph-mon[37330]: 6.9 scrub starts 2024-09-11T07:48:33.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:33 smithi022 ceph-mon[37330]: 6.9 scrub ok 2024-09-11T07:48:33.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:33 smithi022 ceph-mon[29462]: 6.1d scrub ok 2024-09-11T07:48:33.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:33 smithi022 ceph-mon[29462]: 11.f scrub starts 2024-09-11T07:48:33.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:33 smithi022 ceph-mon[29462]: 6.9 scrub starts 2024-09-11T07:48:33.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:33 smithi022 ceph-mon[29462]: 6.9 scrub ok 2024-09-11T07:48:33.959 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:33.993 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:33.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:33.994 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:34.095 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:34.122 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:34.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:34.123 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:34.223 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:34.244 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:34.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:34.245 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:34.279 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:48:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:48:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:48:34.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:33 smithi104 ceph-mon[33507]: 6.1d scrub ok 2024-09-11T07:48:34.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:33 smithi104 ceph-mon[33507]: 11.f scrub starts 2024-09-11T07:48:34.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:33 smithi104 ceph-mon[33507]: 6.9 scrub starts 2024-09-11T07:48:34.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:33 smithi104 ceph-mon[33507]: 6.9 scrub ok 2024-09-11T07:48:34.346 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:34.378 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:34.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:34.379 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:34.480 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:34.501 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:34.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:34.502 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:34.603 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:34.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:34.641 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:34.641 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:34.742 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:34.783 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:34.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:34.784 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:34.885 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:34.925 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:34.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:34.926 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:35.026 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:35.051 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:35.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:35.052 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:35.153 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:35.164 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:48:35.165 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:48:35.165 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [0, 1, 2, 3, 5, 6, 7] 2024-09-11T07:48:35.165 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd out 2 2024-09-11T07:48:35.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:34 smithi022 ceph-mon[29462]: 11.f scrub ok 2024-09-11T07:48:35.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:34 smithi022 ceph-mon[29462]: pgmap v1025: 158 pgs: 158 active+clean; 4.0 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 670 B/s rd, 0 B/s wr, 764 op/s 2024-09-11T07:48:35.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:34 smithi022 ceph-mon[37330]: 11.f scrub ok 2024-09-11T07:48:35.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:34 smithi022 ceph-mon[37330]: pgmap v1025: 158 pgs: 158 active+clean; 4.0 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 670 B/s rd, 0 B/s wr, 764 op/s 2024-09-11T07:48:35.200 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:35.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:35.201 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:35.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:34 smithi104 ceph-mon[33507]: 11.f scrub ok 2024-09-11T07:48:35.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:34 smithi104 ceph-mon[33507]: pgmap v1025: 158 pgs: 158 active+clean; 4.0 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 670 B/s rd, 0 B/s wr, 764 op/s 2024-09-11T07:48:35.302 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:35.326 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:35.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:35.327 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:35.421 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:48:35.428 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:35.449 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:35.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:35.451 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:35.551 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:35.573 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:35.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:35.574 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:35.675 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:35.700 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:35.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:35.701 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:35.801 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:35.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:35.823 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:35.824 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:35.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:35 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:48:35.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:35 smithi022 ceph-mon[29462]: 2.14 scrub starts 2024-09-11T07:48:35.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:35 smithi022 ceph-mon[29462]: 2.14 scrub ok 2024-09-11T07:48:35.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:35 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:48:35.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:35 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:48:35.907 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:35 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:48:35.908 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:35 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:48:35.908 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:35 smithi022 ceph-mon[37330]: 2.14 scrub starts 2024-09-11T07:48:35.908 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:35 smithi022 ceph-mon[37330]: 2.14 scrub ok 2024-09-11T07:48:35.908 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:35 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:48:35.908 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:35 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:48:35.908 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:35 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:48:35.925 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:35.949 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:35.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:35.950 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:36.051 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:36.075 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:36.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:36.076 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:36.177 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:36.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:36.226 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:36.226 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:36.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:35 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:48:36.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:35 smithi104 ceph-mon[33507]: 2.14 scrub starts 2024-09-11T07:48:36.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:35 smithi104 ceph-mon[33507]: 2.14 scrub ok 2024-09-11T07:48:36.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:35 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:48:36.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:35 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:48:36.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:35 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:48:36.327 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:36.354 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:36.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:36.355 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:36.456 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:36.492 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:36.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:36.493 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:36.499 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:48:36.595 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:36.641 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:36.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:36.642 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:36.742 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:36.784 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:36.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:36.785 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:36.885 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:36.911 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:36.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:36.912 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:37.013 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:37.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:37.038 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:37.039 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:37.139 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:37.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:36 smithi022 ceph-mon[29462]: 2.6 scrub starts 2024-09-11T07:48:37.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:36 smithi022 ceph-mon[29462]: 2.6 scrub ok 2024-09-11T07:48:37.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:36 smithi022 ceph-mon[29462]: pgmap v1026: 158 pgs: 158 active+clean; 3.7 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.42k op/s 2024-09-11T07:48:37.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:36 smithi022 ceph-mon[37330]: 2.6 scrub starts 2024-09-11T07:48:37.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:36 smithi022 ceph-mon[37330]: 2.6 scrub ok 2024-09-11T07:48:37.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:36 smithi022 ceph-mon[37330]: pgmap v1026: 158 pgs: 158 active+clean; 3.7 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.42k op/s 2024-09-11T07:48:37.185 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:37.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:37.186 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:37.237 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:48:37.238 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:48:37.238 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:48:37.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:36 smithi104 ceph-mon[33507]: 2.6 scrub starts 2024-09-11T07:48:37.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:36 smithi104 ceph-mon[33507]: 2.6 scrub ok 2024-09-11T07:48:37.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:36 smithi104 ceph-mon[33507]: pgmap v1026: 158 pgs: 158 active+clean; 3.7 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.42k op/s 2024-09-11T07:48:37.281 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:48:37.281 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:48:37.287 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:37.312 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:37.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:37.313 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:37.314 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:48:37.314 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:48:37.349 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:48:37.349 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:48:37.382 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:48:37.382 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:48:37.413 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:48:37.413 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:48:37.413 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:37.440 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:37.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:37.441 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:37.443 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:48:37.444 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:48:37.481 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:48:37.481 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:48:37.542 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:37.588 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:37.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:37.589 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:37.689 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:37.731 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:37.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:37.732 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:37.833 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:37.871 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:37.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:37.872 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:37.973 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:38.000 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:38.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:38.001 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:38.074 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:37 smithi022 ceph-mon[29462]: 4.1e scrub starts 2024-09-11T07:48:38.074 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:37 smithi022 ceph-mon[29462]: 4.1e scrub ok 2024-09-11T07:48:38.074 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:37 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3840312281' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:48:38.075 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:37 smithi022 ceph-mon[37330]: 4.1e scrub starts 2024-09-11T07:48:38.075 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:37 smithi022 ceph-mon[37330]: 4.1e scrub ok 2024-09-11T07:48:38.075 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:37 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3840312281' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:48:38.102 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:38.145 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:38.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:38.146 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:38.246 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:38.271 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:38.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:38.272 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:38.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:37 smithi104 ceph-mon[33507]: 4.1e scrub starts 2024-09-11T07:48:38.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:37 smithi104 ceph-mon[33507]: 4.1e scrub ok 2024-09-11T07:48:38.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:37 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3840312281' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:48:38.373 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:38.413 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:38.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:38.414 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:38.515 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:38.540 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:38.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:38.541 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:38.642 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:38.666 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:38.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:38.667 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:38.716 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:48:38.716 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:48:38.716 INFO:teuthology.orchestra.run.smithi022.stdout: "complaint_time": 30, 2024-09-11T07:48:38.716 INFO:teuthology.orchestra.run.smithi022.stdout: "num_blocked_ops": 0 2024-09-11T07:48:38.716 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:48:38.767 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:38.792 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:38.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:38.793 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:38.894 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:38.936 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:38.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:38.937 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:39.038 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:39.064 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:39.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:39.065 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:39.166 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:39.204 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:39.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:39.205 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:39.209 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:39 smithi022 ceph-mon[29462]: 5.1 scrub starts 2024-09-11T07:48:39.209 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:39 smithi022 ceph-mon[29462]: 5.1 scrub ok 2024-09-11T07:48:39.209 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:39 smithi022 ceph-mon[29462]: 2.1d deep-scrub starts 2024-09-11T07:48:39.209 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:39 smithi022 ceph-mon[29462]: 2.1d deep-scrub ok 2024-09-11T07:48:39.209 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:39 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:48:39.209 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:39 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:48:39.209 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:39 smithi022 ceph-mon[29462]: pgmap v1027: 158 pgs: 158 active+clean; 3.7 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.18k op/s 2024-09-11T07:48:39.209 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:39 smithi022 ceph-mon[37330]: 5.1 scrub starts 2024-09-11T07:48:39.210 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:39 smithi022 ceph-mon[37330]: 5.1 scrub ok 2024-09-11T07:48:39.210 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:39 smithi022 ceph-mon[37330]: 2.1d deep-scrub starts 2024-09-11T07:48:39.210 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:39 smithi022 ceph-mon[37330]: 2.1d deep-scrub ok 2024-09-11T07:48:39.210 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:39 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:48:39.210 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:39 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:48:39.210 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:39 smithi022 ceph-mon[37330]: pgmap v1027: 158 pgs: 158 active+clean; 3.7 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.18k op/s 2024-09-11T07:48:39.306 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:39.348 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:39.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:39.349 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:39.450 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:39.474 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:39.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:39.475 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:39.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:39 smithi104 ceph-mon[33507]: 5.1 scrub starts 2024-09-11T07:48:39.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:39 smithi104 ceph-mon[33507]: 5.1 scrub ok 2024-09-11T07:48:39.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:39 smithi104 ceph-mon[33507]: 2.1d deep-scrub starts 2024-09-11T07:48:39.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:39 smithi104 ceph-mon[33507]: 2.1d deep-scrub ok 2024-09-11T07:48:39.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:39 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:48:39.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:39 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:48:39.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:39 smithi104 ceph-mon[33507]: pgmap v1027: 158 pgs: 158 active+clean; 3.7 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.18k op/s 2024-09-11T07:48:39.575 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:39.604 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:39.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:39.605 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:39.705 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:39.728 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:39.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:39.729 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:39.830 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:39.863 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:39.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:39.864 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:39.965 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:39.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:39.993 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:39.994 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:40.094 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:40.120 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:40.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:40.122 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:40.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:40 smithi022 ceph-mon[29462]: 4.1d scrub starts 2024-09-11T07:48:40.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:40 smithi022 ceph-mon[29462]: 4.1d scrub ok 2024-09-11T07:48:40.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:40 smithi022 ceph-mon[37330]: 4.1d scrub starts 2024-09-11T07:48:40.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:40 smithi022 ceph-mon[37330]: 4.1d scrub ok 2024-09-11T07:48:40.222 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:40.247 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:40.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:40.248 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:40.349 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:40.374 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:40.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:40.375 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:40.402 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-11T07:48:40.475 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:40.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:48:40 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:48:40.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:40 smithi104 ceph-mon[33507]: 4.1d scrub starts 2024-09-11T07:48:40.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:40 smithi104 ceph-mon[33507]: 4.1d scrub ok 2024-09-11T07:48:40.537 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:40.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:40.538 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:40.639 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:40.640 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:48:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:48:40] "GET /metrics HTTP/1.1" 200 225176 "" "Prometheus/2.43.0" 2024-09-11T07:48:40.664 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:40.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:40.665 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:40.765 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:40.819 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:40.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:40.820 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:40.921 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:40.947 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:40.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:40.948 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:41.048 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:41.093 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:41.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:41.094 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:41.195 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:41.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:41.247 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:41.247 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:41.257 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:48:41.308 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:41 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:48:41.308 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:41 smithi022 ceph-mon[29462]: pgmap v1028: 158 pgs: 158 active+clean; 3.3 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.14k op/s 2024-09-11T07:48:41.308 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:41 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:48:41.309 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:41 smithi022 ceph-mon[37330]: pgmap v1028: 158 pgs: 158 active+clean; 3.3 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.14k op/s 2024-09-11T07:48:41.348 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:41.372 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:41.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:41.373 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:41.474 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:41.497 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:41.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:41.498 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:41.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:41 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:48:41.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:41 smithi104 ceph-mon[33507]: pgmap v1028: 158 pgs: 158 active+clean; 3.3 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.14k op/s 2024-09-11T07:48:41.600 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:41.651 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:41.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:41.652 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:41.753 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:41.778 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:41.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:41.779 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:41.879 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:41.919 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:41.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:41.920 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:42.021 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:42.060 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:42.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:42.061 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:42.161 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:42.186 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:42.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:42.187 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:42.288 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:42.314 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:42.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:42.315 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:42.416 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:42.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:42 smithi022 ceph-mon[29462]: 2.0 scrub starts 2024-09-11T07:48:42.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:42 smithi022 ceph-mon[29462]: 2.0 scrub ok 2024-09-11T07:48:42.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:42 smithi022 ceph-mon[29462]: 4.1b scrub starts 2024-09-11T07:48:42.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:42 smithi022 ceph-mon[37330]: 2.0 scrub starts 2024-09-11T07:48:42.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:42 smithi022 ceph-mon[37330]: 2.0 scrub ok 2024-09-11T07:48:42.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:42 smithi022 ceph-mon[37330]: 4.1b scrub starts 2024-09-11T07:48:42.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:42.445 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:42.445 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:42.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:42 smithi104 ceph-mon[33507]: 2.0 scrub starts 2024-09-11T07:48:42.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:42 smithi104 ceph-mon[33507]: 2.0 scrub ok 2024-09-11T07:48:42.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:42 smithi104 ceph-mon[33507]: 4.1b scrub starts 2024-09-11T07:48:42.546 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:42.580 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:42.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:42.581 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:42.682 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:42.707 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:42.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:42.708 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:42.809 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:42.856 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:42.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:42.857 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:42.957 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:42.983 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:42.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:42.984 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:43.085 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:43.109 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:43.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:43.110 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:43.210 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:43.265 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:43.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:43.267 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:43.287 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:43 smithi022 ceph-mon[29462]: 4.1b scrub ok 2024-09-11T07:48:43.287 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:43 smithi022 ceph-mon[29462]: pgmap v1029: 158 pgs: 158 active+clean; 3.3 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 881 B/s rd, 0 B/s wr, 981 op/s 2024-09-11T07:48:43.288 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:43 smithi022 ceph-mon[37330]: 4.1b scrub ok 2024-09-11T07:48:43.288 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:43 smithi022 ceph-mon[37330]: pgmap v1029: 158 pgs: 158 active+clean; 3.3 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 881 B/s rd, 0 B/s wr, 981 op/s 2024-09-11T07:48:43.367 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:43.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:43.410 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:43.410 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:43.511 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:43.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:43 smithi104 ceph-mon[33507]: 4.1b scrub ok 2024-09-11T07:48:43.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:43 smithi104 ceph-mon[33507]: pgmap v1029: 158 pgs: 158 active+clean; 3.3 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 881 B/s rd, 0 B/s wr, 981 op/s 2024-09-11T07:48:43.534 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:43.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:43.536 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:43.636 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:43.673 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:43.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:43.674 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:43.774 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:43.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:43.799 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:43.799 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:43.900 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:43.921 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:43.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:43.922 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:44.023 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:44.062 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:44.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:44.063 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:44.164 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:44.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:44 smithi022 ceph-mon[29462]: 1.9 scrub starts 2024-09-11T07:48:44.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:44 smithi022 ceph-mon[29462]: 1.9 scrub ok 2024-09-11T07:48:44.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:44 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3303728292' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:48:44.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:44 smithi022 ceph-mon[37330]: 1.9 scrub starts 2024-09-11T07:48:44.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:44 smithi022 ceph-mon[37330]: 1.9 scrub ok 2024-09-11T07:48:44.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:44 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3303728292' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:48:44.189 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:44.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:44.190 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:44.291 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:44.307 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is set 2024-09-11T07:48:44.315 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:44.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:44.316 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:44.417 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:44.441 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:44.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:44.443 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:44.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:48:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:48:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:48:44.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:44 smithi104 ceph-mon[33507]: 1.9 scrub starts 2024-09-11T07:48:44.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:44 smithi104 ceph-mon[33507]: 1.9 scrub ok 2024-09-11T07:48:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:44 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3303728292' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:48:44.543 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:44.583 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:44.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:44.585 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:44.686 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:44.723 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:44.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:44.724 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:44.795 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:48:44.825 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:44.848 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:44.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:44.849 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:44.950 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:44.993 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:44.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:44.996 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:45.097 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:45.140 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:45.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:45.141 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:45.182 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:45 smithi022 ceph-mon[29462]: 11.9 deep-scrub starts 2024-09-11T07:48:45.182 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:45 smithi022 ceph-mon[29462]: 6.a deep-scrub starts 2024-09-11T07:48:45.182 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:45 smithi022 ceph-mon[29462]: 6.a deep-scrub ok 2024-09-11T07:48:45.182 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:45 smithi022 ceph-mon[29462]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:48:45.182 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:45 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3303728292' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:48:45.182 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:45 smithi022 ceph-mon[29462]: osdmap e452: 8 total, 8 up, 7 in 2024-09-11T07:48:45.183 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:45 smithi022 ceph-mon[29462]: pgmap v1031: 158 pgs: 158 active+clean; 3.3 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.14k op/s 2024-09-11T07:48:45.183 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:45 smithi022 ceph-mon[37330]: 11.9 deep-scrub starts 2024-09-11T07:48:45.183 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:45 smithi022 ceph-mon[37330]: 6.a deep-scrub starts 2024-09-11T07:48:45.183 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:45 smithi022 ceph-mon[37330]: 6.a deep-scrub ok 2024-09-11T07:48:45.183 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:45 smithi022 ceph-mon[37330]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:48:45.183 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:45 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3303728292' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:48:45.183 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:45 smithi022 ceph-mon[37330]: osdmap e452: 8 total, 8 up, 7 in 2024-09-11T07:48:45.183 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:45 smithi022 ceph-mon[37330]: pgmap v1031: 158 pgs: 158 active+clean; 3.3 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.14k op/s 2024-09-11T07:48:45.243 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:45.265 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:45.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:45.266 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:45.298 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:48:45.367 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:45.392 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:45.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:45.393 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:45.434 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:45 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-a[29458]: 2024-09-11T07:48:45.179+0000 7fde5d00f640 -1 mon.a@0(leader).osd e452 definitely_dead 0 2024-09-11T07:48:45.493 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:45.517 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:45.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:45.518 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:45 smithi104 ceph-mon[33507]: 11.9 deep-scrub starts 2024-09-11T07:48:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:45 smithi104 ceph-mon[33507]: 6.a deep-scrub starts 2024-09-11T07:48:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:45 smithi104 ceph-mon[33507]: 6.a deep-scrub ok 2024-09-11T07:48:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:45 smithi104 ceph-mon[33507]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:48:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:45 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3303728292' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:48:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:45 smithi104 ceph-mon[33507]: osdmap e452: 8 total, 8 up, 7 in 2024-09-11T07:48:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:45 smithi104 ceph-mon[33507]: pgmap v1031: 158 pgs: 158 active+clean; 3.3 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.14k op/s 2024-09-11T07:48:45.619 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:45.644 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:45.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:45.645 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:45.746 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:45.769 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:45.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:45.770 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:45.871 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:45.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:45.899 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:45.899 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:46.000 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:46.043 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:46.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:46.044 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:46.129 INFO:teuthology.orchestra.run.smithi022.stderr:marked out osd.2. 2024-09-11T07:48:46.145 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:46.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:46.194 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:46.194 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:46.295 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:46.319 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:46.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:46.320 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:46.369 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:46 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1948304462' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["2"]}]: dispatch 2024-09-11T07:48:46.369 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:46 smithi022 ceph-mon[29462]: from='client.26131 ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["2"]}]: dispatch 2024-09-11T07:48:46.369 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:46 smithi022 ceph-mon[29462]: Client client.admin marked osd.2 out, while it was still marked up 2024-09-11T07:48:46.369 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:46 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1948304462' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["2"]}]: dispatch 2024-09-11T07:48:46.369 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:46 smithi022 ceph-mon[37330]: from='client.26131 ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["2"]}]: dispatch 2024-09-11T07:48:46.369 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:46 smithi022 ceph-mon[37330]: Client client.admin marked osd.2 out, while it was still marked up 2024-09-11T07:48:46.421 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:46.445 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:46.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:46.446 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:46.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:46 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1948304462' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["2"]}]: dispatch 2024-09-11T07:48:46.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:46 smithi104 ceph-mon[33507]: from='client.26131 ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["2"]}]: dispatch 2024-09-11T07:48:46.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:46 smithi104 ceph-mon[33507]: Client client.admin marked osd.2 out, while it was still marked up 2024-09-11T07:48:46.547 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:46.572 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:46.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:46.573 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:46.674 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:46.700 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:46.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:46.701 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:46.802 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:46.825 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:46.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:46.826 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:46.927 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:46.949 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:46.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:46.953 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:47.054 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:47.077 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:47.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:47.078 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:47.178 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:47.202 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:47.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:47.203 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:47.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:47 smithi022 ceph-mon[29462]: from='client.26131 ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["2"]}]': finished 2024-09-11T07:48:47.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:47 smithi022 ceph-mon[29462]: osdmap e453: 8 total, 8 up, 6 in 2024-09-11T07:48:47.234 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:47 smithi022 ceph-mon[29462]: pgmap v1033: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 3.0 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.46k op/s 2024-09-11T07:48:47.234 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:47 smithi022 ceph-mon[37330]: from='client.26131 ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["2"]}]': finished 2024-09-11T07:48:47.234 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:47 smithi022 ceph-mon[37330]: osdmap e453: 8 total, 8 up, 6 in 2024-09-11T07:48:47.235 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:47 smithi022 ceph-mon[37330]: pgmap v1033: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 3.0 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.46k op/s 2024-09-11T07:48:47.293 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T07:48:47.304 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:47.358 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:47.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:47.359 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:47.460 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:47.482 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:47.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:47.483 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:47.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:47 smithi104 ceph-mon[33507]: from='client.26131 ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["2"]}]': finished 2024-09-11T07:48:47.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:47 smithi104 ceph-mon[33507]: osdmap e453: 8 total, 8 up, 6 in 2024-09-11T07:48:47.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:47 smithi104 ceph-mon[33507]: pgmap v1033: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 3.0 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.46k op/s 2024-09-11T07:48:47.584 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:47.605 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:47.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:47.606 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:47.707 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:47.731 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:47.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:47.732 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:47.833 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:47.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:47.876 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:47.876 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:47.977 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:47.999 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:47.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:48.000 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:48.100 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:48.125 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:48.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:48.126 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:48.226 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:48.250 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:48.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:48.251 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:48.352 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:48.381 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:48.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:48.384 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:48.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:48 smithi022 ceph-mon[29462]: osdmap e454: 8 total, 8 up, 6 in 2024-09-11T07:48:48.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:48 smithi022 ceph-mon[37330]: osdmap e454: 8 total, 8 up, 6 in 2024-09-11T07:48:48.485 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:48.517 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:48.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:48.518 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:48.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:48 smithi104 ceph-mon[33507]: osdmap e454: 8 total, 8 up, 6 in 2024-09-11T07:48:48.619 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:48.648 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:48.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:48.649 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:48.750 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:48.794 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:48.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:48.795 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:48.896 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:48.924 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:48.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:48.925 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:49.025 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:49.076 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:49.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:49.077 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:49.178 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:49.206 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:49.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:49.207 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:49.308 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:49.338 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:49.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:49.339 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:49.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:49 smithi022 ceph-mon[29462]: 11.3 starting backfill to osd.3 from (0'0,0'0] MAX to 452'11973 2024-09-11T07:48:49.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:49 smithi022 ceph-mon[29462]: osdmap e455: 8 total, 8 up, 6 in 2024-09-11T07:48:49.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:49 smithi022 ceph-mon[29462]: 11.4 starting backfill to osd.1 from (0'0,0'0] MAX to 452'12300 2024-09-11T07:48:49.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:49 smithi022 ceph-mon[29462]: 11.c starting backfill to osd.5 from (0'0,0'0] MAX to 452'11856 2024-09-11T07:48:49.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:49 smithi022 ceph-mon[29462]: 11.f starting backfill to osd.0 from (0'0,0'0] MAX to 452'11899 2024-09-11T07:48:49.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:49 smithi022 ceph-mon[29462]: 11.d starting backfill to osd.5 from (0'0,0'0] MAX to 452'11930 2024-09-11T07:48:49.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:49 smithi022 ceph-mon[29462]: 11.8 starting backfill to osd.1 from (0'0,0'0] MAX to 452'12112 2024-09-11T07:48:49.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:49 smithi022 ceph-mon[29462]: 11.1 starting backfill to osd.1 from (0'0,0'0] MAX to 452'11963 2024-09-11T07:48:49.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:49 smithi022 ceph-mon[29462]: 11.2 starting backfill to osd.1 from (0'0,0'0] MAX to 452'12060 2024-09-11T07:48:49.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:49 smithi022 ceph-mon[29462]: 11.6 starting backfill to osd.3 from (0'0,0'0] MAX to 452'11853 2024-09-11T07:48:49.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:49 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2950673136' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:48:49.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:49 smithi022 ceph-mon[29462]: pgmap v1036: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 3.0 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T07:48:49.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:49 smithi022 ceph-mon[37330]: 11.3 starting backfill to osd.3 from (0'0,0'0] MAX to 452'11973 2024-09-11T07:48:49.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:49 smithi022 ceph-mon[37330]: osdmap e455: 8 total, 8 up, 6 in 2024-09-11T07:48:49.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:49 smithi022 ceph-mon[37330]: 11.4 starting backfill to osd.1 from (0'0,0'0] MAX to 452'12300 2024-09-11T07:48:49.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:49 smithi022 ceph-mon[37330]: 11.c starting backfill to osd.5 from (0'0,0'0] MAX to 452'11856 2024-09-11T07:48:49.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:49 smithi022 ceph-mon[37330]: 11.f starting backfill to osd.0 from (0'0,0'0] MAX to 452'11899 2024-09-11T07:48:49.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:49 smithi022 ceph-mon[37330]: 11.d starting backfill to osd.5 from (0'0,0'0] MAX to 452'11930 2024-09-11T07:48:49.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:49 smithi022 ceph-mon[37330]: 11.8 starting backfill to osd.1 from (0'0,0'0] MAX to 452'12112 2024-09-11T07:48:49.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:49 smithi022 ceph-mon[37330]: 11.1 starting backfill to osd.1 from (0'0,0'0] MAX to 452'11963 2024-09-11T07:48:49.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:49 smithi022 ceph-mon[37330]: 11.2 starting backfill to osd.1 from (0'0,0'0] MAX to 452'12060 2024-09-11T07:48:49.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:49 smithi022 ceph-mon[37330]: 11.6 starting backfill to osd.3 from (0'0,0'0] MAX to 452'11853 2024-09-11T07:48:49.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:49 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2950673136' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:48:49.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:49 smithi022 ceph-mon[37330]: pgmap v1036: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 3.0 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T07:48:49.440 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:49.467 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:49.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:49.468 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:49.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:49 smithi104 ceph-mon[33507]: 11.3 starting backfill to osd.3 from (0'0,0'0] MAX to 452'11973 2024-09-11T07:48:49.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:49 smithi104 ceph-mon[33507]: osdmap e455: 8 total, 8 up, 6 in 2024-09-11T07:48:49.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:49 smithi104 ceph-mon[33507]: 11.4 starting backfill to osd.1 from (0'0,0'0] MAX to 452'12300 2024-09-11T07:48:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:49 smithi104 ceph-mon[33507]: 11.c starting backfill to osd.5 from (0'0,0'0] MAX to 452'11856 2024-09-11T07:48:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:49 smithi104 ceph-mon[33507]: 11.f starting backfill to osd.0 from (0'0,0'0] MAX to 452'11899 2024-09-11T07:48:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:49 smithi104 ceph-mon[33507]: 11.d starting backfill to osd.5 from (0'0,0'0] MAX to 452'11930 2024-09-11T07:48:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:49 smithi104 ceph-mon[33507]: 11.8 starting backfill to osd.1 from (0'0,0'0] MAX to 452'12112 2024-09-11T07:48:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:49 smithi104 ceph-mon[33507]: 11.1 starting backfill to osd.1 from (0'0,0'0] MAX to 452'11963 2024-09-11T07:48:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:49 smithi104 ceph-mon[33507]: 11.2 starting backfill to osd.1 from (0'0,0'0] MAX to 452'12060 2024-09-11T07:48:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:49 smithi104 ceph-mon[33507]: 11.6 starting backfill to osd.3 from (0'0,0'0] MAX to 452'11853 2024-09-11T07:48:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:49 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2950673136' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:48:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:49 smithi104 ceph-mon[33507]: pgmap v1036: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 3.0 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-11T07:48:49.569 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:49.600 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:49.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:49.602 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:49.702 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:49.721 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:48:49.721 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:48:49.721 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:48:49.721 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:48:49.722 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:48:49.722 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:48:49.722 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:48:49.722 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:48:49.722 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:48:49.722 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:48:49.722 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:48:49.722 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:48:49.723 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:48:49.723 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:48:49.723 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:48:49.723 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:48:49.746 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:49.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:49.748 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:49.849 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:49.876 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:49.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:49.878 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:49.979 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:50.004 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:50.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:50.005 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:50.105 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:50.146 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:50.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:50.147 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:50.248 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:50.275 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:50.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:50.276 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:50.377 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:50.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:50.408 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:50.408 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:50.509 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:50.531 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:50.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:50.532 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:50.633 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:50.656 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:50.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:50.657 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:50.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:50 smithi022 ceph-mon[29462]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:48:50.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:48:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:48:50] "GET /metrics HTTP/1.1" 200 225176 "" "Prometheus/2.43.0" 2024-09-11T07:48:50.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:50 smithi022 ceph-mon[37330]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:48:50.758 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:50.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:48:50 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:48:50.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:50 smithi104 ceph-mon[33507]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:48:50.788 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:50.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:50.789 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:50.890 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:50.918 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:50.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:50.919 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:51.020 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:51.061 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:51.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:51.062 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:51.163 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:51.190 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:51.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:51.192 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:51.292 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:51.322 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:51.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:51.323 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:51.424 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:51.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:51.462 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:51.462 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:51.563 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:51.606 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:51.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:51.607 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:51.708 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:51.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:51.753 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:51.753 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:51.854 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:51.901 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:51.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:51.902 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:51.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:51 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:48:51.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:51 smithi022 ceph-mon[29462]: pgmap v1037: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+remapped, 6 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 147 active+clean; 2.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 136 KiB/s rd, 25 KiB/s wr, 1.49k op/s; 27033/138723 objects misplaced (19.487%); 17 MiB/s, 288 objects/s recovering 2024-09-11T07:48:51.916 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:51 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:48:51.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:51 smithi022 ceph-mon[37330]: pgmap v1037: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+remapped, 6 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 147 active+clean; 2.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 136 KiB/s rd, 25 KiB/s wr, 1.49k op/s; 27033/138723 objects misplaced (19.487%); 17 MiB/s, 288 objects/s recovering 2024-09-11T07:48:52.003 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:52.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:51 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:48:52.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:51 smithi104 ceph-mon[33507]: pgmap v1037: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+remapped, 6 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 147 active+clean; 2.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 136 KiB/s rd, 25 KiB/s wr, 1.49k op/s; 27033/138723 objects misplaced (19.487%); 17 MiB/s, 288 objects/s recovering 2024-09-11T07:48:52.038 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:52.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:52.039 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:52.140 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:52.177 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:52.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:52.178 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:52.279 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:52.317 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:52.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:52.318 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:52.419 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:52.460 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:52.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:52.461 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:52.561 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:52.605 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:52.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:52.607 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:52.707 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:52.751 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:52.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:52.753 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:52.854 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:52.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:52.883 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:52.883 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:52.984 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:53.026 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:53.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:53.027 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:53.128 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:53.169 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:53.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:53.170 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:53.270 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:53.299 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:53.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:53.300 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:53.400 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:53.430 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:53.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:53.431 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:53.531 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:53.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:53.563 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:53.564 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:53.664 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:53.689 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:53.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:53.690 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:53.791 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:53.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:53.822 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:53.823 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:53.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:53 smithi022 ceph-mon[29462]: pgmap v1038: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+remapped, 6 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 147 active+clean; 2.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 128 KiB/s rd, 23 KiB/s wr, 506 op/s; 27033/138723 objects misplaced (19.487%); 16 MiB/s, 273 objects/s recovering 2024-09-11T07:48:53.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:48:53.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:48:53.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:53 smithi022 ceph-mon[37330]: pgmap v1038: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+remapped, 6 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 147 active+clean; 2.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 128 KiB/s rd, 23 KiB/s wr, 506 op/s; 27033/138723 objects misplaced (19.487%); 16 MiB/s, 273 objects/s recovering 2024-09-11T07:48:53.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:48:53.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:48:53.923 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:53.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:53.952 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:53.952 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:54.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:53 smithi104 ceph-mon[33507]: pgmap v1038: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+remapped, 6 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 147 active+clean; 2.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 128 KiB/s rd, 23 KiB/s wr, 506 op/s; 27033/138723 objects misplaced (19.487%); 16 MiB/s, 273 objects/s recovering 2024-09-11T07:48:54.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:48:54.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:48:54.053 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:54.098 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:54.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:54.099 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:54.200 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:54.228 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:54.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:54.228 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:54.329 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:54.371 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:54.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:54.372 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:54.473 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:54.501 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:54.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:54.502 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:54.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:48:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:48:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:48:54.603 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:54.646 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:54.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:54.647 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:54.748 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:54.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:54.777 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:54.777 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:54.878 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:54.945 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:54.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:54.951 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:54.967 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:48:55.052 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:55.078 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:55.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:55.079 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:55.180 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:55.205 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:55.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:55.206 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:55.307 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:55.329 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:55.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:55.330 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:55.431 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:55.475 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:55.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:55.476 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:55.577 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:55.603 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:55.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:55.603 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:55.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:55 smithi022 ceph-mon[29462]: pgmap v1039: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+remapped, 6 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 147 active+clean; 2.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 101 KiB/s rd, 18 KiB/s wr, 400 op/s; 27033/138723 objects misplaced (19.487%); 13 MiB/s, 216 objects/s recovering 2024-09-11T07:48:55.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:55 smithi022 ceph-mon[37330]: pgmap v1039: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+remapped, 6 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 147 active+clean; 2.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 101 KiB/s rd, 18 KiB/s wr, 400 op/s; 27033/138723 objects misplaced (19.487%); 13 MiB/s, 216 objects/s recovering 2024-09-11T07:48:55.705 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:55.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:55.744 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:55.744 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:55.761 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:48:55.842 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:48:55.842 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:48:55.846 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:55.882 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:55.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:55.884 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:55.985 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:56.006 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:56.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:56.007 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:56.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:55 smithi104 ceph-mon[33507]: pgmap v1039: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+remapped, 6 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 147 active+clean; 2.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 101 KiB/s rd, 18 KiB/s wr, 400 op/s; 27033/138723 objects misplaced (19.487%); 13 MiB/s, 216 objects/s recovering 2024-09-11T07:48:56.108 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:56.129 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:56.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:56.130 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:56.230 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:56.263 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:56.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:56.264 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:56.365 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:56.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:56.400 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:56.400 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:56.501 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:56.522 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:56.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:56.523 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:56.623 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:56.647 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:56.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:56.648 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:56.749 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:56.785 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:56.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:56.786 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:56.887 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:56.913 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:56.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:56.914 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:56.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:56 smithi022 ceph-mon[29462]: osdmap e456: 8 total, 8 up, 6 in 2024-09-11T07:48:56.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:56 smithi022 ceph-mon[37330]: osdmap e456: 8 total, 8 up, 6 in 2024-09-11T07:48:56.963 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-11T07:48:56.987 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:56 smithi104 ceph-mon[33507]: osdmap e456: 8 total, 8 up, 6 in 2024-09-11T07:48:57.015 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:57.065 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:57.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:57.066 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:57.167 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:57.199 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:57.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:57.201 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:57.286 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:48:57.302 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:57.334 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:57.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:57.336 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:57.436 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:57.462 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:57.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:57.463 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:57.564 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:57.589 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:57.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:57.590 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:57.631 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:48:57.691 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:57.718 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:57.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:57.719 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:57.755 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:57 smithi104 ceph-mon[33507]: pgmap v1041: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+remapped, 6 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 147 active+clean; 2.6 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 98 KiB/s rd, 18 KiB/s wr, 839 op/s; 18037/127404 objects misplaced (14.157%); 66 MiB/s, 1.06k objects/s recovering 2024-09-11T07:48:57.756 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:57 smithi104 ceph-mon[33507]: 11.9 deep-scrub ok 2024-09-11T07:48:57.756 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:57 smithi104 ceph-mon[33507]: osdmap e457: 8 total, 8 up, 6 in 2024-09-11T07:48:57.820 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:57.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:57.869 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:57.869 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:57.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:57 smithi022 ceph-mon[29462]: pgmap v1041: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+remapped, 6 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 147 active+clean; 2.6 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 98 KiB/s rd, 18 KiB/s wr, 839 op/s; 18037/127404 objects misplaced (14.157%); 66 MiB/s, 1.06k objects/s recovering 2024-09-11T07:48:57.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:57 smithi022 ceph-mon[29462]: 11.9 deep-scrub ok 2024-09-11T07:48:57.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:57 smithi022 ceph-mon[29462]: osdmap e457: 8 total, 8 up, 6 in 2024-09-11T07:48:57.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:57 smithi022 ceph-mon[37330]: pgmap v1041: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+remapped, 6 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 147 active+clean; 2.6 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 98 KiB/s rd, 18 KiB/s wr, 839 op/s; 18037/127404 objects misplaced (14.157%); 66 MiB/s, 1.06k objects/s recovering 2024-09-11T07:48:57.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:57 smithi022 ceph-mon[37330]: 11.9 deep-scrub ok 2024-09-11T07:48:57.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:57 smithi022 ceph-mon[37330]: osdmap e457: 8 total, 8 up, 6 in 2024-09-11T07:48:57.970 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:57.995 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:57.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:57.996 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:58.097 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:58.148 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:58.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:58.150 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:58.251 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:58.274 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:58.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:58.276 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:58.376 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:58.402 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:58.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:58.403 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:58.503 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:48:58.546 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:58.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:58.547 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:48:58.648 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:58.677 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:58.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:58.678 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:58.779 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:48:58.821 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:58.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:58.822 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:48:58.923 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:58.947 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:58.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:58.948 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:59.049 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:59.074 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:59.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:59.075 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:59.176 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:48:59.207 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:59.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:59.208 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:48:59.309 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:48:59.351 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:59.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:59.352 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:48:59.438 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 5, 6, 7] out_osds: [4, 2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:48:59.438 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:48:59.439 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-11T07:48:59.439 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T07:48:59.439 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T07:48:59.439 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.1 config set filestore_inject_stall 3 2024-09-11T07:48:59.452 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:48:59.477 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:59.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:59.478 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:48:59.579 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:48:59.605 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:59.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:59.606 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:48:59.624 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is set 2024-09-11T07:48:59.707 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:48:59.708 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:59 smithi022 ceph-mon[29462]: pgmap v1043: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+remapped, 6 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 147 active+clean; 2.6 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 472 op/s; 18037/127404 objects misplaced (14.157%); 56 MiB/s, 890 objects/s recovering 2024-09-11T07:48:59.708 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:59 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3595704550' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:48:59.708 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:48:59 smithi022 ceph-mon[29462]: from='client.26140 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:48:59.709 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:59 smithi022 ceph-mon[37330]: pgmap v1043: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+remapped, 6 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 147 active+clean; 2.6 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 472 op/s; 18037/127404 objects misplaced (14.157%); 56 MiB/s, 890 objects/s recovering 2024-09-11T07:48:59.709 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:59 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3595704550' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:48:59.709 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:48:59 smithi022 ceph-mon[37330]: from='client.26140 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:48:59.772 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:48:59.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:59.773 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:48:59.874 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:48:59.907 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:48:59.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:59.908 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:48:59.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:59 smithi104 ceph-mon[33507]: pgmap v1043: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+remapped, 6 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 147 active+clean; 2.6 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 472 op/s; 18037/127404 objects misplaced (14.157%); 56 MiB/s, 890 objects/s recovering 2024-09-11T07:48:59.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:59 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3595704550' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:48:59.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:48:59 smithi104 ceph-mon[33507]: from='client.26140 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:49:00.009 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:00.041 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:00.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:00.042 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:00.143 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:00.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:00.171 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:00.171 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:00.272 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:00.304 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:00.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:00.305 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:00.406 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:00.429 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:00.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:00.430 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:00.479 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:49:00.479 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:49:00.520 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:49:00.520 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:49:00.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:49:00 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:49:00.531 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:00.554 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:49:00.554 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:49:00.575 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:00.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:00.576 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:00.589 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:49:00.589 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:49:00.621 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:49:00.621 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:49:00.653 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:49:00.653 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:49:00.654 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:00 smithi022 ceph-mon[29462]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:49:00.654 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:00 smithi022 ceph-mon[29462]: from='client.26140 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:49:00.654 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:00 smithi022 ceph-mon[29462]: osdmap e458: 8 total, 8 up, 6 in 2024-09-11T07:49:00.654 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:00 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2448308898' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:49:00.654 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:49:00 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:49:00] "GET /metrics HTTP/1.1" 200 225226 "" "Prometheus/2.43.0" 2024-09-11T07:49:00.655 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:00 smithi022 ceph-mon[37330]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:49:00.655 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:00 smithi022 ceph-mon[37330]: from='client.26140 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:49:00.655 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:00 smithi022 ceph-mon[37330]: osdmap e458: 8 total, 8 up, 6 in 2024-09-11T07:49:00.655 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:00 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2448308898' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:49:00.678 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:00.686 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:49:00.686 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:49:00.705 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:00.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:00.706 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:00.719 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:49:00.720 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:49:00.807 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:00.832 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:00.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:00.833 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:00.934 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:00.977 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:00.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:00.978 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:01.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:00 smithi104 ceph-mon[33507]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:49:01.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:00 smithi104 ceph-mon[33507]: from='client.26140 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:49:01.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:00 smithi104 ceph-mon[33507]: osdmap e458: 8 total, 8 up, 6 in 2024-09-11T07:49:01.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:00 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2448308898' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:49:01.079 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:01.102 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:49:01.102 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:49:01.106 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:01.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:01.107 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:01.208 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:01.234 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:01.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:01.235 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:01.336 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:01.380 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:01.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:01.381 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:01.482 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:01.524 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:01.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:01.525 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:01.593 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:49:01.626 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:01.689 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:01.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:01.692 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:01.783 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:01 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:49:01.784 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:01 smithi022 ceph-mon[29462]: pgmap v1045: 158 pgs: 1 active+clean+remapped, 1 active+clean+scrubbing+deep, 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 149 active+clean; 2.4 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.15k op/s; 13243/118119 objects misplaced (11.212%); 108 MiB/s, 1.73k objects/s recovering 2024-09-11T07:49:01.784 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:01 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:49:01.784 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:01 smithi022 ceph-mon[37330]: pgmap v1045: 158 pgs: 1 active+clean+remapped, 1 active+clean+scrubbing+deep, 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 149 active+clean; 2.4 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.15k op/s; 13243/118119 objects misplaced (11.212%); 108 MiB/s, 1.73k objects/s recovering 2024-09-11T07:49:01.792 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:01.819 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:01.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:01.820 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:01.921 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:01.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:01.971 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:01.971 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:02.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:01 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:49:02.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:01 smithi104 ceph-mon[33507]: pgmap v1045: 158 pgs: 1 active+clean+remapped, 1 active+clean+scrubbing+deep, 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 149 active+clean; 2.4 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.15k op/s; 13243/118119 objects misplaced (11.212%); 108 MiB/s, 1.73k objects/s recovering 2024-09-11T07:49:02.072 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:02.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:02.114 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:02.114 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:02.216 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:02.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:02.258 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:02.259 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:02.360 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:02.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:02.401 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:02.401 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:02.502 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:02.530 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:02.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:02.531 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:02.631 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:02.656 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:02.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:02.658 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:02.758 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:02.780 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T07:49:02.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:02.785 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:02.785 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:02.886 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:02.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:02.914 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:02.914 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:03.015 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:03.038 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:03.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:03.039 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:03.140 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:03.166 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:03.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:03.167 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:03.268 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:03.300 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:03.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:03.301 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:03.305 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-11T07:49:03.402 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:03.460 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:03.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:03.461 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:03.562 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:03.573 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:49:03.598 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:03.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:03.600 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:03.659 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:49:03.701 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:03.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:03.749 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:03.830 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:49:03.830 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "" 2024-09-11T07:49:03.830 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:49:03.830 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:49:03.850 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:03.875 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:03.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:03.876 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:03.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:03 smithi022 ceph-mon[29462]: pgmap v1046: 158 pgs: 1 active+clean+remapped, 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 2.3 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 744 B/s rd, 0 B/s wr, 597 op/s; 13243/115089 objects misplaced (11.507%); 30 MiB/s, 473 objects/s recovering 2024-09-11T07:49:03.917 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:03 smithi104 ceph-mon[33507]: pgmap v1046: 158 pgs: 1 active+clean+remapped, 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 2.3 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 744 B/s rd, 0 B/s wr, 597 op/s; 13243/115089 objects misplaced (11.507%); 30 MiB/s, 473 objects/s recovering 2024-09-11T07:49:03.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:03 smithi022 ceph-mon[37330]: pgmap v1046: 158 pgs: 1 active+clean+remapped, 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 2.3 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 744 B/s rd, 0 B/s wr, 597 op/s; 13243/115089 objects misplaced (11.507%); 30 MiB/s, 473 objects/s recovering 2024-09-11T07:49:03.976 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:04.028 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:04.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:04.031 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:04.132 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:04.174 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:04.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:04.175 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:04.276 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:04.279 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:49:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:49:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:49:04.302 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:04.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:04.303 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:04.404 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:04.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:04.430 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:04.430 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:04.531 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:04.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:04.554 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:04.554 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:04.655 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:04.682 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:04.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:04.683 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:04.784 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:04.810 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:04.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:04.811 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:04.911 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:04.938 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:04.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:04.939 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:05.040 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:05.066 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:05.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:05.067 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:05.168 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:05.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:05.213 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:05.213 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:05.314 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:05.355 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:05.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:05.357 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:05.445 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:49:05.445 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:49:05.445 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2024-09-11T07:49:05.446 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2024-09-11T07:49:05.446 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:49:05.457 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:05.491 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:05.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:05.492 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:05.593 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:05.618 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:05.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:05.619 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:05.720 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:05.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:05 smithi104 ceph-mon[33507]: pgmap v1047: 158 pgs: 1 active+clean+remapped, 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 2.3 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 513 op/s; 13243/115089 objects misplaced (11.507%); 25 MiB/s, 406 objects/s recovering 2024-09-11T07:49:05.793 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:05.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:05.794 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:05.895 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:05.916 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:05 smithi022 ceph-mon[37330]: pgmap v1047: 158 pgs: 1 active+clean+remapped, 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 2.3 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 513 op/s; 13243/115089 objects misplaced (11.507%); 25 MiB/s, 406 objects/s recovering 2024-09-11T07:49:05.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:05 smithi022 ceph-mon[29462]: pgmap v1047: 158 pgs: 1 active+clean+remapped, 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 2.3 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 513 op/s; 13243/115089 objects misplaced (11.507%); 25 MiB/s, 406 objects/s recovering 2024-09-11T07:49:05.923 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:05.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:05.925 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:06.025 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:06.073 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:06.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:06.075 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:06.175 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:06.191 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-11T07:49:06.218 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:06.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:06.219 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:06.319 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:06.365 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:06.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:06.366 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:06.467 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:06.503 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:49:06.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:06.504 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:06.504 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:06.605 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:06.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:06.639 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:06.639 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:06.740 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:06.767 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:06.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:06.768 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:06.869 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:06.893 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:06.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:06.894 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:06.922 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:06 smithi104 ceph-mon[33507]: pgmap v1048: 158 pgs: 1 active+clean+remapped, 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 2.0 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 930 op/s; 9548/99879 objects misplaced (9.560%); 34 MiB/s, 541 objects/s recovering 2024-09-11T07:49:06.995 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:07.016 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:07.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:07.017 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:07.118 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:07.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:07.140 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:07.140 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:07.166 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:06 smithi022 ceph-mon[37330]: pgmap v1048: 158 pgs: 1 active+clean+remapped, 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 2.0 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 930 op/s; 9548/99879 objects misplaced (9.560%); 34 MiB/s, 541 objects/s recovering 2024-09-11T07:49:07.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:06 smithi022 ceph-mon[29462]: pgmap v1048: 158 pgs: 1 active+clean+remapped, 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 2.0 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 930 op/s; 9548/99879 objects misplaced (9.560%); 34 MiB/s, 541 objects/s recovering 2024-09-11T07:49:07.241 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:07.262 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:07.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:07.263 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:07.364 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:07.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:07.388 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:07.388 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:07.489 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:07.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:07.511 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:07.511 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:07.612 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:07.633 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:07.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:07.634 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:07.735 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:07.761 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:07.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:07.762 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:07.863 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:07.887 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:07.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:07.888 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:07.989 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:08.011 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:08.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:08.012 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:08.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:07 smithi104 ceph-mon[33507]: osdmap e459: 8 total, 8 up, 6 in 2024-09-11T07:49:08.113 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:08.135 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:08.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:08.136 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:08.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:07 smithi022 ceph-mon[29462]: osdmap e459: 8 total, 8 up, 6 in 2024-09-11T07:49:08.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:07 smithi022 ceph-mon[37330]: osdmap e459: 8 total, 8 up, 6 in 2024-09-11T07:49:08.237 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:08.260 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:08.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:08.262 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:08.362 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:08.386 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:08.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:08.387 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:08.488 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:08.500 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:49:08.500 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2024-09-11T07:49:08.500 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2024-09-11T07:49:08.500 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2024-09-11T07:49:08.501 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.501 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211568 9.16 9:6b0caae0:::benchmark_data_smithi022_133943_object84977:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e306)", 2024-09-11T07:49:08.501 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.491978+0000", 2024-09-11T07:49:08.501 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 513.00569996199999, 2024-09-11T07:49:08.501 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0266594309999999, 2024-09-11T07:49:08.501 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:08.501 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:08.501 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:08.501 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:49:08.501 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:49:08.501 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211568 2024-09-11T07:49:08.501 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.501 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:08.502 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.502 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:08.502 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.491978+0000", 2024-09-11T07:49:08.502 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.502 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.502 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.502 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:08.502 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.491978+0000", 2024-09-11T07:49:08.502 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.502 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.502 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.502 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:08.502 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.491976+0000", 2024-09-11T07:49:08.503 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:49:08.503 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.503 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.503 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:08.503 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.491984+0000", 2024-09-11T07:49:08.503 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9879999999999993e-06 2024-09-11T07:49:08.503 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.503 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.503 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:08.503 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.493497+0000", 2024-09-11T07:49:08.503 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001512572 2024-09-11T07:49:08.503 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.503 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.503 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:08.504 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.493505+0000", 2024-09-11T07:49:08.504 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.8169999999999999e-06 2024-09-11T07:49:08.504 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.504 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.504 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.504 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.494096+0000", 2024-09-11T07:49:08.504 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00059062700000000004 2024-09-11T07:49:08.504 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.504 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.504 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:08.504 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.494105+0000", 2024-09-11T07:49:08.504 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8050000000000001e-06 2024-09-11T07:49:08.504 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.504 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.505 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.505 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.513691+0000", 2024-09-11T07:49:08.505 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.019586165 2024-09-11T07:49:08.505 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.505 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.505 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:08.506 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.513793+0000", 2024-09-11T07:49:08.506 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000101745 2024-09-11T07:49:08.506 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.506 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.506 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:49:08.506 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.514096+0000", 2024-09-11T07:49:08.506 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00030249600000000003 2024-09-11T07:49:08.506 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.506 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.506 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:08.506 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518426+0000", 2024-09-11T07:49:08.506 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0043307750000000002 2024-09-11T07:49:08.506 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.506 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.506 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.507 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518448+0000", 2024-09-11T07:49:08.507 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1321000000000001e-05 2024-09-11T07:49:08.507 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.507 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.507 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.507 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518570+0000", 2024-09-11T07:49:08.507 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012255 2024-09-11T07:49:08.507 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.507 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.507 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:08.507 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518602+0000", 2024-09-11T07:49:08.507 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1735e-05 2024-09-11T07:49:08.507 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.507 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.508 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:08.508 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518637+0000", 2024-09-11T07:49:08.508 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5238999999999999e-05 2024-09-11T07:49:08.508 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.508 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:08.508 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.508 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.508 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.508 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211583 9.16 9:6baf4737:::benchmark_data_smithi022_133943_object84992:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:49:08.508 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.510888+0000", 2024-09-11T07:49:08.508 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 512.98678996399997, 2024-09-11T07:49:08.508 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0079589120000001, 2024-09-11T07:49:08.508 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:08.508 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:08.508 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:08.509 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:49:08.509 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:49:08.509 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211583 2024-09-11T07:49:08.509 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.509 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:08.509 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.509 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:08.509 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.510888+0000", 2024-09-11T07:49:08.509 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.509 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.509 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.509 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:08.509 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.510888+0000", 2024-09-11T07:49:08.509 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.509 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.510 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.510 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:08.510 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.510885+0000", 2024-09-11T07:49:08.510 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:49:08.510 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.510 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.510 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:08.510 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.510895+0000", 2024-09-11T07:49:08.510 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2399999999999996e-06 2024-09-11T07:49:08.510 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.510 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.510 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:08.510 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.510914+0000", 2024-09-11T07:49:08.510 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9018999999999999e-05 2024-09-11T07:49:08.511 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.511 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.511 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:08.511 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.510921+0000", 2024-09-11T07:49:08.511 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5530000000000004e-06 2024-09-11T07:49:08.511 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.511 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.511 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.511 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.510974+0000", 2024-09-11T07:49:08.511 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2683999999999999e-05 2024-09-11T07:49:08.511 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.511 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.511 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:08.511 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.510983+0000", 2024-09-11T07:49:08.511 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7290000000000006e-06 2024-09-11T07:49:08.511 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.512 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.512 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.512 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.515778+0000", 2024-09-11T07:49:08.512 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.004795729 2024-09-11T07:49:08.512 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.512 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.512 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:08.512 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.515882+0000", 2024-09-11T07:49:08.512 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010367899999999999 2024-09-11T07:49:08.512 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.512 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.512 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:49:08.512 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.516114+0000", 2024-09-11T07:49:08.513 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000232394 2024-09-11T07:49:08.513 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.513 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.513 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:08.513 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518531+0000", 2024-09-11T07:49:08.513 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0024165269999999999 2024-09-11T07:49:08.513 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.513 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.513 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.513 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518699+0000", 2024-09-11T07:49:08.513 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016786500000000001 2024-09-11T07:49:08.513 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.513 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.513 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.514 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518758+0000", 2024-09-11T07:49:08.514 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9435000000000003e-05 2024-09-11T07:49:08.514 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.514 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.514 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:08.514 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518800+0000", 2024-09-11T07:49:08.514 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1912000000000001e-05 2024-09-11T07:49:08.514 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.514 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.514 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:08.515 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518847+0000", 2024-09-11T07:49:08.515 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6686e-05 2024-09-11T07:49:08.515 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.516 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:08.516 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.516 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.516 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.516 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211591 9.16 9:6c004eb6:::benchmark_data_smithi022_133943_object85000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:49:08.516 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.519215+0000", 2024-09-11T07:49:08.516 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 512.97846335400004, 2024-09-11T07:49:08.516 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.001420948, 2024-09-11T07:49:08.516 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:08.516 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:08.516 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:08.516 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:49:08.516 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:49:08.516 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211591 2024-09-11T07:49:08.516 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.517 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:08.517 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.517 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:08.517 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.519215+0000", 2024-09-11T07:49:08.517 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.517 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.517 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.517 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:08.517 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.519215+0000", 2024-09-11T07:49:08.517 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.517 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.517 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.517 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:08.518 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.519212+0000", 2024-09-11T07:49:08.518 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999971 2024-09-11T07:49:08.518 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.518 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.518 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:08.518 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.519220+0000", 2024-09-11T07:49:08.518 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1920000000000005e-06 2024-09-11T07:49:08.518 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.518 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.518 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:08.518 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.519235+0000", 2024-09-11T07:49:08.518 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.483e-05 2024-09-11T07:49:08.518 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.518 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.519 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:08.519 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.519241+0000", 2024-09-11T07:49:08.519 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9150000000000001e-06 2024-09-11T07:49:08.519 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.519 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.519 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.519 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.519281+0000", 2024-09-11T07:49:08.519 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0568999999999997e-05 2024-09-11T07:49:08.519 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.519 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.519 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:08.519 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.519289+0000", 2024-09-11T07:49:08.519 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3699999999999997e-06 2024-09-11T07:49:08.519 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.519 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.520 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.520 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.517249+0000", 2024-09-11T07:49:08.520 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.99796017800000003 2024-09-11T07:49:08.520 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.520 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.520 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:08.520 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.517317+0000", 2024-09-11T07:49:08.520 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.8004999999999997e-05 2024-09-11T07:49:08.520 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.520 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.520 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:49:08.520 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.517499+0000", 2024-09-11T07:49:08.520 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018260999999999999 2024-09-11T07:49:08.520 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.520 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.521 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.521 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518719+0000", 2024-09-11T07:49:08.521 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001219588 2024-09-11T07:49:08.521 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.521 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.521 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:08.521 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518857+0000", 2024-09-11T07:49:08.521 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013752 2024-09-11T07:49:08.521 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.521 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.521 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.521 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.520551+0000", 2024-09-11T07:49:08.521 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001694688 2024-09-11T07:49:08.522 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.522 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.522 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:08.522 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.520599+0000", 2024-09-11T07:49:08.522 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7874999999999998e-05 2024-09-11T07:49:08.522 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.522 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.522 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:08.522 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.520635+0000", 2024-09-11T07:49:08.522 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6313000000000003e-05 2024-09-11T07:49:08.522 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.522 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:08.522 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.522 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.522 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.523 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211637 9.16 9:6a9b2a50:::benchmark_data_smithi022_133943_object85046:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:49:08.523 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.555357+0000", 2024-09-11T07:49:08.523 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 512.94232037500001, 2024-09-11T07:49:08.523 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.96961262199999998, 2024-09-11T07:49:08.523 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:08.523 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:08.523 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:08.523 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:49:08.523 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:49:08.523 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211637 2024-09-11T07:49:08.523 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.523 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:08.523 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.524 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:08.524 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.555357+0000", 2024-09-11T07:49:08.524 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.524 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.524 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:08.524 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.555357+0000", 2024-09-11T07:49:08.524 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.524 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.524 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:08.524 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.555355+0000", 2024-09-11T07:49:08.524 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999971 2024-09-11T07:49:08.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.525 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.525 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:08.525 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.555363+0000", 2024-09-11T07:49:08.525 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0679999999999997e-06 2024-09-11T07:49:08.525 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.525 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.525 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:08.525 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.555375+0000", 2024-09-11T07:49:08.525 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1653e-05 2024-09-11T07:49:08.525 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.525 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.525 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:08.525 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.555380+0000", 2024-09-11T07:49:08.525 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7679999999999997e-06 2024-09-11T07:49:08.525 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.526 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.526 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.526 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.555426+0000", 2024-09-11T07:49:08.526 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5615999999999998e-05 2024-09-11T07:49:08.526 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.526 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.526 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:08.526 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.555434+0000", 2024-09-11T07:49:08.526 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6159999999999996e-06 2024-09-11T07:49:08.526 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.526 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.526 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.526 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518872+0000", 2024-09-11T07:49:08.526 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.96343850799999997 2024-09-11T07:49:08.527 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.527 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.527 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:08.527 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.518941+0000", 2024-09-11T07:49:08.527 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9207000000000003e-05 2024-09-11T07:49:08.527 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.527 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.527 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:49:08.527 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.519150+0000", 2024-09-11T07:49:08.527 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00020839900000000001 2024-09-11T07:49:08.527 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.527 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.527 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.527 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.524769+0000", 2024-09-11T07:49:08.528 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0056189810000000003 2024-09-11T07:49:08.528 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.528 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.528 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:08.528 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.524776+0000", 2024-09-11T07:49:08.528 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7670000000000003e-06 2024-09-11T07:49:08.528 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.528 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.528 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.528 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.524884+0000", 2024-09-11T07:49:08.528 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000107384 2024-09-11T07:49:08.528 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.528 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.529 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:08.529 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.524930+0000", 2024-09-11T07:49:08.529 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6008999999999999e-05 2024-09-11T07:49:08.529 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.529 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.529 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:08.529 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.524970+0000", 2024-09-11T07:49:08.529 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0268999999999997e-05 2024-09-11T07:49:08.529 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.529 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:08.529 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.529 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.529 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.529 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211686 9.16 9:69c3c680:::benchmark_data_smithi022_133943_object85095:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:49:08.530 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.594485+0000", 2024-09-11T07:49:08.530 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 512.90319322100004, 2024-09-11T07:49:08.530 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.93446916000000002, 2024-09-11T07:49:08.530 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:08.530 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:08.530 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:08.530 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:49:08.530 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:49:08.530 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211686 2024-09-11T07:49:08.530 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.530 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:08.530 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.530 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:08.530 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.594485+0000", 2024-09-11T07:49:08.530 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.531 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.531 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.531 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:08.531 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.594485+0000", 2024-09-11T07:49:08.531 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.531 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.531 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.531 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:08.531 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.594483+0000", 2024-09-11T07:49:08.531 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:49:08.531 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.531 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.531 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:08.531 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.594489+0000", 2024-09-11T07:49:08.531 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7710000000000001e-06 2024-09-11T07:49:08.532 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.532 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.532 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:08.532 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.594502+0000", 2024-09-11T07:49:08.532 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2847000000000001e-05 2024-09-11T07:49:08.532 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.532 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.532 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:08.532 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.594508+0000", 2024-09-11T07:49:08.532 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7350000000000001e-06 2024-09-11T07:49:08.532 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.532 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.532 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.533 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.594572+0000", 2024-09-11T07:49:08.533 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4414999999999999e-05 2024-09-11T07:49:08.533 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.533 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.533 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:08.533 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.594579+0000", 2024-09-11T07:49:08.533 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.041e-06 2024-09-11T07:49:08.533 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.533 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.533 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.533 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.520664+0000", 2024-09-11T07:49:08.533 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.92608433199999995 2024-09-11T07:49:08.533 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.534 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.534 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:08.534 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.520762+0000", 2024-09-11T07:49:08.534 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7971000000000001e-05 2024-09-11T07:49:08.534 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.534 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.534 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:49:08.534 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.521031+0000", 2024-09-11T07:49:08.534 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00026898699999999999 2024-09-11T07:49:08.534 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.534 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.534 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.534 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.524805+0000", 2024-09-11T07:49:08.534 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0037746870000000001 2024-09-11T07:49:08.535 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.535 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.535 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:08.535 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.524981+0000", 2024-09-11T07:49:08.535 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017573400000000001 2024-09-11T07:49:08.535 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.535 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.535 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.535 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.528877+0000", 2024-09-11T07:49:08.535 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0038956770000000002 2024-09-11T07:49:08.535 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.535 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.535 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:08.535 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.528918+0000", 2024-09-11T07:49:08.535 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1600999999999997e-05 2024-09-11T07:49:08.536 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.536 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.536 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:08.536 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.528954+0000", 2024-09-11T07:49:08.536 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5330000000000002e-05 2024-09-11T07:49:08.536 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.536 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:08.536 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.536 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.536 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.536 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211710 9.16 9:6b0c937f:::benchmark_data_smithi022_133943_object85119:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:49:08.536 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.612884+0000", 2024-09-11T07:49:08.536 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 512.88479354399999, 2024-09-11T07:49:08.537 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.92170397500000001, 2024-09-11T07:49:08.537 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:08.537 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:08.537 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:08.537 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:49:08.537 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:49:08.537 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211710 2024-09-11T07:49:08.537 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.537 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:08.537 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.537 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:08.537 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.612884+0000", 2024-09-11T07:49:08.537 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.537 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.538 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.538 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:08.538 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.612884+0000", 2024-09-11T07:49:08.538 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.538 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.538 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.538 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:08.538 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.612882+0000", 2024-09-11T07:49:08.538 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:49:08.538 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.538 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.538 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:08.538 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.612892+0000", 2024-09-11T07:49:08.539 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4010000000000003e-06 2024-09-11T07:49:08.539 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.539 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.539 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:08.539 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.612905+0000", 2024-09-11T07:49:08.539 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3653e-05 2024-09-11T07:49:08.539 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.539 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.539 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:08.539 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.612912+0000", 2024-09-11T07:49:08.539 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3380000000000001e-06 2024-09-11T07:49:08.539 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.539 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.539 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.539 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.613011+0000", 2024-09-11T07:49:08.539 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.9469000000000006e-05 2024-09-11T07:49:08.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.540 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:08.540 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.613018+0000", 2024-09-11T07:49:08.540 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0679999999999999e-06 2024-09-11T07:49:08.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.540 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.540 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.522738+0000", 2024-09-11T07:49:08.540 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.90972005199999995 2024-09-11T07:49:08.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.540 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:08.540 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.522865+0000", 2024-09-11T07:49:08.541 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000127093 2024-09-11T07:49:08.541 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.541 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.541 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:49:08.541 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.523141+0000", 2024-09-11T07:49:08.541 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00027508200000000001 2024-09-11T07:49:08.541 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.541 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.541 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:08.541 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.528797+0000", 2024-09-11T07:49:08.541 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0056566009999999998 2024-09-11T07:49:08.541 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.541 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.542 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.542 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.528839+0000", 2024-09-11T07:49:08.542 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1924e-05 2024-09-11T07:49:08.542 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.542 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.542 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.542 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.534494+0000", 2024-09-11T07:49:08.542 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0056551739999999998 2024-09-11T07:49:08.542 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.542 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.542 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:08.542 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.534544+0000", 2024-09-11T07:49:08.542 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9911999999999999e-05 2024-09-11T07:49:08.542 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.542 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.543 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:08.543 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.534588+0000", 2024-09-11T07:49:08.543 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4122000000000002e-05 2024-09-11T07:49:08.543 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.543 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:08.543 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.543 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.543 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.543 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211745 9.16 9:694b4fca:::benchmark_data_smithi022_133943_object85154:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:49:08.543 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.639786+0000", 2024-09-11T07:49:08.543 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 512.85789157399995, 2024-09-11T07:49:08.543 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.90253554000000002, 2024-09-11T07:49:08.543 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:08.543 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:08.543 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:08.544 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:49:08.544 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:49:08.544 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211745 2024-09-11T07:49:08.544 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.544 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:08.544 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.544 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:08.544 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.639786+0000", 2024-09-11T07:49:08.544 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.544 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.544 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.544 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:08.544 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.639786+0000", 2024-09-11T07:49:08.544 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.545 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.545 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:08.545 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.639784+0000", 2024-09-11T07:49:08.545 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:49:08.545 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.545 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:08.545 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.639791+0000", 2024-09-11T07:49:08.545 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.2980000000000003e-06 2024-09-11T07:49:08.545 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.545 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:08.546 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.639804+0000", 2024-09-11T07:49:08.546 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.207e-05 2024-09-11T07:49:08.546 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.546 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.546 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:08.546 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.639809+0000", 2024-09-11T07:49:08.546 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7429999999999999e-06 2024-09-11T07:49:08.546 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.546 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.546 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.546 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.639861+0000", 2024-09-11T07:49:08.546 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1946e-05 2024-09-11T07:49:08.546 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.546 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.547 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:08.547 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.639871+0000", 2024-09-11T07:49:08.547 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3379999999999994e-06 2024-09-11T07:49:08.547 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.547 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.547 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.547 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.525002+0000", 2024-09-11T07:49:08.547 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.88513150799999996 2024-09-11T07:49:08.547 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.547 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.547 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:08.547 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.525118+0000", 2024-09-11T07:49:08.547 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000116166 2024-09-11T07:49:08.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.548 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.548 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:49:08.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.525429+0000", 2024-09-11T07:49:08.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00031055399999999999 2024-09-11T07:49:08.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.548 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.548 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:08.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.528849+0000", 2024-09-11T07:49:08.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003419872 2024-09-11T07:49:08.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.548 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.548 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.529004+0000", 2024-09-11T07:49:08.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000155663 2024-09-11T07:49:08.549 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.549 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.549 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542237+0000", 2024-09-11T07:49:08.549 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.013232961 2024-09-11T07:49:08.549 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.549 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:08.549 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542285+0000", 2024-09-11T07:49:08.549 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7598000000000001e-05 2024-09-11T07:49:08.549 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.549 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:08.550 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542322+0000", 2024-09-11T07:49:08.550 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6925000000000003e-05 2024-09-11T07:49:08.550 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.550 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:08.550 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.550 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.550 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.550 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211775 9.16 9:68b77f6d:::benchmark_data_smithi022_133943_object85184:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:49:08.550 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.676391+0000", 2024-09-11T07:49:08.550 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 512.82128641899999, 2024-09-11T07:49:08.550 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.86609124999999998, 2024-09-11T07:49:08.550 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:08.550 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:08.551 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:08.551 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:49:08.551 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:49:08.551 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211775 2024-09-11T07:49:08.551 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.551 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:08.551 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.551 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:08.551 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.676391+0000", 2024-09-11T07:49:08.551 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.551 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.551 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.551 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:08.552 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.676391+0000", 2024-09-11T07:49:08.552 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.552 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.552 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.552 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:08.552 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.676389+0000", 2024-09-11T07:49:08.552 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:49:08.552 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.552 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.552 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:08.552 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.676399+0000", 2024-09-11T07:49:08.552 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.6020000000000006e-06 2024-09-11T07:49:08.552 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.552 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.552 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:08.552 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.676414+0000", 2024-09-11T07:49:08.553 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5577999999999999e-05 2024-09-11T07:49:08.553 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.553 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.553 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:08.553 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.676422+0000", 2024-09-11T07:49:08.553 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9310000000000008e-06 2024-09-11T07:49:08.553 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.553 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.553 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.553 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.676471+0000", 2024-09-11T07:49:08.553 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8769e-05 2024-09-11T07:49:08.553 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.553 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.553 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:08.553 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.676481+0000", 2024-09-11T07:49:08.554 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8490000000000001e-06 2024-09-11T07:49:08.554 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.554 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.554 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.554 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.527191+0000", 2024-09-11T07:49:08.554 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.85070976600000003 2024-09-11T07:49:08.554 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.554 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.554 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:08.554 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.527309+0000", 2024-09-11T07:49:08.554 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000118206 2024-09-11T07:49:08.554 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.554 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.554 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:49:08.555 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.527534+0000", 2024-09-11T07:49:08.555 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00022490499999999999 2024-09-11T07:49:08.555 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.555 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.555 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.555 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.529042+0000", 2024-09-11T07:49:08.555 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0015080779999999999 2024-09-11T07:49:08.555 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.555 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.555 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:08.555 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.529048+0000", 2024-09-11T07:49:08.555 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2219999999999999e-06 2024-09-11T07:49:08.555 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.556 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.556 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.556 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542405+0000", 2024-09-11T07:49:08.556 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.013356632 2024-09-11T07:49:08.556 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:08.557 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:08.557 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:08.558 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.558 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:08.558 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542450+0000", 2024-09-11T07:49:08.558 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5164999999999999e-05 2024-09-11T07:49:08.558 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.558 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.558 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:08.558 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542483+0000", 2024-09-11T07:49:08.559 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2945999999999999e-05 2024-09-11T07:49:08.559 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.559 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:08.559 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.559 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.559 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.559 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211789 9.16 9:6d58905b:::benchmark_data_smithi022_133943_object85198:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:49:08.559 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.692735+0000", 2024-09-11T07:49:08.559 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 512.80494327400004, 2024-09-11T07:49:08.559 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.84991873100000004, 2024-09-11T07:49:08.559 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:08.559 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:08.559 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:08.560 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:49:08.560 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:49:08.560 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211789 2024-09-11T07:49:08.560 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.560 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:08.560 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.560 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:08.560 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.692735+0000", 2024-09-11T07:49:08.560 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.560 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.560 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.560 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:08.560 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.692735+0000", 2024-09-11T07:49:08.560 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.560 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.561 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.561 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:08.561 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.692732+0000", 2024-09-11T07:49:08.561 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:49:08.561 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.561 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.561 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:08.561 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.692740+0000", 2024-09-11T07:49:08.561 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9589999999999995e-06 2024-09-11T07:49:08.561 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.561 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.561 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:08.561 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.692755+0000", 2024-09-11T07:49:08.561 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4649e-05 2024-09-11T07:49:08.562 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.562 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.562 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:08.562 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.692761+0000", 2024-09-11T07:49:08.562 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5010000000000001e-06 2024-09-11T07:49:08.562 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.562 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.562 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.562 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.692809+0000", 2024-09-11T07:49:08.562 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7410999999999997e-05 2024-09-11T07:49:08.562 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.562 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.562 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:08.562 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.692817+0000", 2024-09-11T07:49:08.563 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4440000000000008e-06 2024-09-11T07:49:08.563 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.563 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.563 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.563 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.529069+0000", 2024-09-11T07:49:08.563 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.83625205700000005 2024-09-11T07:49:08.563 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.563 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.563 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:08.563 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.529140+0000", 2024-09-11T07:49:08.563 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0500000000000006e-05 2024-09-11T07:49:08.563 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.563 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.563 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:49:08.564 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.529333+0000", 2024-09-11T07:49:08.564 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000193396 2024-09-11T07:49:08.564 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.564 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.564 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:08.564 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.534390+0000", 2024-09-11T07:49:08.564 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0050565050000000002 2024-09-11T07:49:08.564 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.564 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.564 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.564 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.534425+0000", 2024-09-11T07:49:08.564 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5138999999999997e-05 2024-09-11T07:49:08.564 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.564 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.565 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.565 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542573+0000", 2024-09-11T07:49:08.565 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0081485240000000007 2024-09-11T07:49:08.565 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.565 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.565 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:08.565 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542622+0000", 2024-09-11T07:49:08.565 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8279000000000003e-05 2024-09-11T07:49:08.565 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.565 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.565 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:08.565 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542653+0000", 2024-09-11T07:49:08.565 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1745000000000002e-05 2024-09-11T07:49:08.566 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.566 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:08.566 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.566 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.566 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.566 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211797 9.16 9:6ee58f2c:::benchmark_data_smithi022_133943_object85206:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:49:08.566 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.698284+0000", 2024-09-11T07:49:08.566 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 512.79939404300001, 2024-09-11T07:49:08.566 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.84449932100000002, 2024-09-11T07:49:08.566 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:08.566 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:08.566 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:08.566 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:49:08.566 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:49:08.566 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211797 2024-09-11T07:49:08.566 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.567 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:08.567 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.567 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:08.567 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.698284+0000", 2024-09-11T07:49:08.567 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.567 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.567 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.567 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:08.567 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.698284+0000", 2024-09-11T07:49:08.567 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.567 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.567 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.567 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:08.567 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.698282+0000", 2024-09-11T07:49:08.568 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:49:08.568 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.568 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.568 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:08.568 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.698288+0000", 2024-09-11T07:49:08.568 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6819999999999997e-06 2024-09-11T07:49:08.568 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.568 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.568 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:08.568 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.698299+0000", 2024-09-11T07:49:08.568 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0668e-05 2024-09-11T07:49:08.568 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.568 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.568 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:08.569 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.698305+0000", 2024-09-11T07:49:08.569 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6969999999999998e-06 2024-09-11T07:49:08.569 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.569 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.569 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.569 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.698345+0000", 2024-09-11T07:49:08.569 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.036e-05 2024-09-11T07:49:08.569 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.569 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.569 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:08.569 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.698352+0000", 2024-09-11T07:49:08.569 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.8680000000000001e-06 2024-09-11T07:49:08.569 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.569 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.569 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.570 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.530667+0000", 2024-09-11T07:49:08.570 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.83231449499999999 2024-09-11T07:49:08.570 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.570 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.570 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:08.570 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.530772+0000", 2024-09-11T07:49:08.570 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010552699999999999 2024-09-11T07:49:08.570 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.570 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.570 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:49:08.570 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.531034+0000", 2024-09-11T07:49:08.570 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000262439 2024-09-11T07:49:08.570 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.570 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.571 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:08.571 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.534430+0000", 2024-09-11T07:49:08.571 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0033959440000000001 2024-09-11T07:49:08.571 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.571 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.571 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.571 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.534458+0000", 2024-09-11T07:49:08.571 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.7622000000000001e-05 2024-09-11T07:49:08.571 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.571 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.571 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.571 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542705+0000", 2024-09-11T07:49:08.571 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0082472069999999995 2024-09-11T07:49:08.571 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.571 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.572 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:08.572 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542756+0000", 2024-09-11T07:49:08.572 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0600999999999999e-05 2024-09-11T07:49:08.572 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.572 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.572 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:08.572 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542783+0000", 2024-09-11T07:49:08.572 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.7300999999999998e-05 2024-09-11T07:49:08.572 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.572 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:08.572 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.572 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.572 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.572 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211806 9.16 9:6a26adda:::benchmark_data_smithi022_133943_object85215:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:49:08.572 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.708768+0000", 2024-09-11T07:49:08.573 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 512.78891005699995, 2024-09-11T07:49:08.573 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.83413179299999995, 2024-09-11T07:49:08.573 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:08.573 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:08.573 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:08.573 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:49:08.573 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:49:08.573 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211806 2024-09-11T07:49:08.573 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.573 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:08.573 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.573 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:08.573 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.708768+0000", 2024-09-11T07:49:08.573 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.574 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.574 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.574 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:08.574 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.708768+0000", 2024-09-11T07:49:08.574 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.574 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.574 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.574 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:08.574 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.708765+0000", 2024-09-11T07:49:08.574 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999971 2024-09-11T07:49:08.574 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.574 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.574 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:08.574 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.708774+0000", 2024-09-11T07:49:08.574 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2869999999999993e-06 2024-09-11T07:49:08.574 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.575 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.575 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:08.575 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.708789+0000", 2024-09-11T07:49:08.575 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4391e-05 2024-09-11T07:49:08.575 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.575 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.575 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:08.575 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.708796+0000", 2024-09-11T07:49:08.575 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0530000000000001e-06 2024-09-11T07:49:08.575 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.575 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.575 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.575 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.708860+0000", 2024-09-11T07:49:08.575 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4565000000000003e-05 2024-09-11T07:49:08.575 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.576 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.576 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:08.576 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.708867+0000", 2024-09-11T07:49:08.576 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1739999999999997e-06 2024-09-11T07:49:08.576 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.576 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.576 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.576 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.532429+0000", 2024-09-11T07:49:08.576 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.82356275800000001 2024-09-11T07:49:08.576 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.576 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.576 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:08.576 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.532517+0000", 2024-09-11T07:49:08.576 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7460999999999995e-05 2024-09-11T07:49:08.577 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.577 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.577 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:49:08.577 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.532755+0000", 2024-09-11T07:49:08.577 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000237941 2024-09-11T07:49:08.577 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.577 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.577 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:08.577 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.536455+0000", 2024-09-11T07:49:08.577 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0036998270000000002 2024-09-11T07:49:08.577 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.577 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.577 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.577 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542204+0000", 2024-09-11T07:49:08.577 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0057493090000000002 2024-09-11T07:49:08.578 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.578 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.578 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.578 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542818+0000", 2024-09-11T07:49:08.578 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00061378599999999995 2024-09-11T07:49:08.578 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.578 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.578 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:08.578 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542856+0000", 2024-09-11T07:49:08.578 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8791999999999997e-05 2024-09-11T07:49:08.578 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.578 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.578 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:08.578 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542900+0000", 2024-09-11T07:49:08.578 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3170999999999997e-05 2024-09-11T07:49:08.578 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.579 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:08.579 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.579 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.579 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.579 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211852 9.16 9:6ab70b3a:::benchmark_data_smithi022_133943_object85261:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:49:08.579 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.765778+0000", 2024-09-11T07:49:08.579 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 512.731899914, 2024-09-11T07:49:08.579 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.78150204000000001, 2024-09-11T07:49:08.579 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:08.579 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:08.579 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:08.579 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:49:08.579 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:49:08.580 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211852 2024-09-11T07:49:08.580 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.580 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:08.580 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.580 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:08.580 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.765778+0000", 2024-09-11T07:49:08.580 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.580 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.580 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.580 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:08.580 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.765778+0000", 2024-09-11T07:49:08.580 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.580 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.580 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.580 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:08.580 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.765775+0000", 2024-09-11T07:49:08.581 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999971 2024-09-11T07:49:08.581 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.581 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.581 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:08.581 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.765784+0000", 2024-09-11T07:49:08.581 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3919999999999992e-06 2024-09-11T07:49:08.581 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.581 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.581 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:08.581 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.765801+0000", 2024-09-11T07:49:08.581 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6969e-05 2024-09-11T07:49:08.581 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.581 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.581 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:08.582 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.765809+0000", 2024-09-11T07:49:08.582 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4070000000000004e-06 2024-09-11T07:49:08.582 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.582 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.582 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.582 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.765876+0000", 2024-09-11T07:49:08.582 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7101999999999996e-05 2024-09-11T07:49:08.582 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.582 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.582 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:08.582 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.765884+0000", 2024-09-11T07:49:08.582 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4880000000000007e-06 2024-09-11T07:49:08.582 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.582 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.582 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.583 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.534626+0000", 2024-09-11T07:49:08.583 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.76874199799999998 2024-09-11T07:49:08.583 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.583 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.583 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:08.583 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.534739+0000", 2024-09-11T07:49:08.583 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000112904 2024-09-11T07:49:08.583 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.583 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.583 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:49:08.583 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.535029+0000", 2024-09-11T07:49:08.583 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00028969500000000002 2024-09-11T07:49:08.583 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.583 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.583 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:08.583 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.540522+0000", 2024-09-11T07:49:08.584 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.005493118 2024-09-11T07:49:08.584 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.584 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.584 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.584 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542355+0000", 2024-09-11T07:49:08.584 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0018325609999999999 2024-09-11T07:49:08.584 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.584 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.584 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.584 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.547211+0000", 2024-09-11T07:49:08.584 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0048561899999999998 2024-09-11T07:49:08.584 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.584 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.584 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:08.584 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.547256+0000", 2024-09-11T07:49:08.585 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4821999999999999e-05 2024-09-11T07:49:08.585 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.585 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.585 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:08.585 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.547280+0000", 2024-09-11T07:49:08.585 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.4431999999999999e-05 2024-09-11T07:49:08.585 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.585 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:08.585 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.585 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.585 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.585 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211923 9.16 9:68ffa8c1:::benchmark_data_smithi022_133943_object85332:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:49:08.585 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.884034+0000", 2024-09-11T07:49:08.585 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 512.61364421799999, 2024-09-11T07:49:08.586 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.66333196100000003, 2024-09-11T07:49:08.586 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:08.586 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:08.586 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:08.586 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:49:08.586 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:49:08.586 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211923 2024-09-11T07:49:08.586 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.586 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:08.586 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.586 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:08.586 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.884034+0000", 2024-09-11T07:49:08.586 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.586 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.586 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.586 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:08.587 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.884034+0000", 2024-09-11T07:49:08.587 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.587 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.587 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.587 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:08.587 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.884032+0000", 2024-09-11T07:49:08.587 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:49:08.587 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.587 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.587 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:08.587 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.884042+0000", 2024-09-11T07:49:08.587 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.9299999999999998e-06 2024-09-11T07:49:08.587 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.587 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.587 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:08.587 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.884051+0000", 2024-09-11T07:49:08.588 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0159999999999997e-06 2024-09-11T07:49:08.588 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.588 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.588 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:08.588 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.884055+0000", 2024-09-11T07:49:08.588 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.053e-06 2024-09-11T07:49:08.588 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.588 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.588 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.588 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.884086+0000", 2024-09-11T07:49:08.588 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.0858999999999997e-05 2024-09-11T07:49:08.588 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.588 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.588 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:08.589 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.884092+0000", 2024-09-11T07:49:08.589 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0379999999999996e-06 2024-09-11T07:49:08.589 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.589 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.589 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.589 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.536480+0000", 2024-09-11T07:49:08.589 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.65238803999999995 2024-09-11T07:49:08.589 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.589 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.589 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:08.589 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.536577+0000", 2024-09-11T07:49:08.589 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.6854999999999993e-05 2024-09-11T07:49:08.589 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.589 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.590 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:49:08.590 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.536851+0000", 2024-09-11T07:49:08.590 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00027332099999999998 2024-09-11T07:49:08.590 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.590 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.590 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:08.590 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542171+0000", 2024-09-11T07:49:08.590 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0053199620000000001 2024-09-11T07:49:08.590 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.590 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.590 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.590 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542539+0000", 2024-09-11T07:49:08.590 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00036858700000000003 2024-09-11T07:49:08.590 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.590 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.591 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.591 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.547314+0000", 2024-09-11T07:49:08.591 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0047751470000000004 2024-09-11T07:49:08.591 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.591 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.591 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:08.591 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.547342+0000", 2024-09-11T07:49:08.591 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.8113999999999999e-05 2024-09-11T07:49:08.591 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.591 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.591 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:08.591 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.547366+0000", 2024-09-11T07:49:08.591 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3238e-05 2024-09-11T07:49:08.591 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.591 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:08.591 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.592 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.592 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.592 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211949 9.16 9:6827f352:::benchmark_data_smithi022_133943_object85358:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:49:08.592 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.917915+0000", 2024-09-11T07:49:08.592 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 512.57976282499999, 2024-09-11T07:49:08.592 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.62951596099999996, 2024-09-11T07:49:08.592 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:08.592 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:08.592 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:08.592 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:49:08.592 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:49:08.592 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211949 2024-09-11T07:49:08.592 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.592 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:08.592 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.593 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:08.593 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.917915+0000", 2024-09-11T07:49:08.593 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.593 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.593 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.593 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:08.593 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.917915+0000", 2024-09-11T07:49:08.593 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.593 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.593 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.593 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:08.593 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.917914+0000", 2024-09-11T07:49:08.593 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:49:08.593 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.593 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.594 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:08.594 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.917918+0000", 2024-09-11T07:49:08.594 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2899999999999996e-06 2024-09-11T07:49:08.594 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.594 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.594 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:08.594 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.917925+0000", 2024-09-11T07:49:08.594 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3740000000000003e-06 2024-09-11T07:49:08.594 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.594 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.594 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:08.594 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.917929+0000", 2024-09-11T07:49:08.594 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4139999999999996e-06 2024-09-11T07:49:08.594 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.594 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.594 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.595 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.917960+0000", 2024-09-11T07:49:08.595 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.0925e-05 2024-09-11T07:49:08.595 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.595 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.595 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:08.595 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.917966+0000", 2024-09-11T07:49:08.595 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3990000000000001e-06 2024-09-11T07:49:08.595 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.595 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.595 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.595 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.538684+0000", 2024-09-11T07:49:08.595 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.62071715599999999 2024-09-11T07:49:08.595 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.595 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.595 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:08.596 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.538783+0000", 2024-09-11T07:49:08.596 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.9036000000000005e-05 2024-09-11T07:49:08.596 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.596 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.596 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:49:08.596 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.539075+0000", 2024-09-11T07:49:08.596 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00029267600000000001 2024-09-11T07:49:08.596 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.596 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.596 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:08.596 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542210+0000", 2024-09-11T07:49:08.596 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0031345840000000002 2024-09-11T07:49:08.596 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.596 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.597 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.597 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.545296+0000", 2024-09-11T07:49:08.597 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0030863309999999999 2024-09-11T07:49:08.597 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.597 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.597 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.597 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.547390+0000", 2024-09-11T07:49:08.597 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0020940630000000002 2024-09-11T07:49:08.597 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.597 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.597 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:08.597 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.547413+0000", 2024-09-11T07:49:08.597 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.2399999999999999e-05 2024-09-11T07:49:08.597 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.597 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.598 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:08.598 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.547431+0000", 2024-09-11T07:49:08.598 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8263000000000001e-05 2024-09-11T07:49:08.598 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.598 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:08.598 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.598 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.598 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.598 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:211951 9.16 9:6974843e:::benchmark_data_smithi022_133943_object85360:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e307)", 2024-09-11T07:49:08.598 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:35.921562+0000", 2024-09-11T07:49:08.598 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 512.576116189, 2024-09-11T07:49:08.598 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.62686241200000004, 2024-09-11T07:49:08.598 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:08.598 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:08.598 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:08.598 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:49:08.599 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:49:08.599 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 211951 2024-09-11T07:49:08.599 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.599 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:08.599 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.599 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:08.599 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.921562+0000", 2024-09-11T07:49:08.599 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.599 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.599 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.599 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:08.599 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.921562+0000", 2024-09-11T07:49:08.599 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:08.599 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.599 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.599 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:08.600 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.921560+0000", 2024-09-11T07:49:08.600 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:49:08.600 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.600 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.600 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:08.600 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.921565+0000", 2024-09-11T07:49:08.600 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0690000000000001e-06 2024-09-11T07:49:08.600 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.600 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.600 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:08.600 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.921574+0000", 2024-09-11T07:49:08.600 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2810000000000008e-06 2024-09-11T07:49:08.600 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.600 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.600 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:08.601 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.921578+0000", 2024-09-11T07:49:08.601 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3189999999999996e-06 2024-09-11T07:49:08.601 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.601 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.601 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.601 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.921606+0000", 2024-09-11T07:49:08.601 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.7716e-05 2024-09-11T07:49:08.601 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.601 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.601 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:08.601 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:35.921610+0000", 2024-09-11T07:49:08.601 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.408e-06 2024-09-11T07:49:08.601 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.601 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.601 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:08.601 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.540550+0000", 2024-09-11T07:49:08.602 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.61893973300000005 2024-09-11T07:49:08.602 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.602 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.602 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:08.602 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.540657+0000", 2024-09-11T07:49:08.602 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000106907 2024-09-11T07:49:08.602 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.602 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.602 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:49:08.602 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.540893+0000", 2024-09-11T07:49:08.602 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00023581900000000001 2024-09-11T07:49:08.602 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.602 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.602 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:08.602 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.542361+0000", 2024-09-11T07:49:08.603 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0014681570000000001 2024-09-11T07:49:08.603 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.603 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.603 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.603 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.545374+0000", 2024-09-11T07:49:08.603 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0030128239999999999 2024-09-11T07:49:08.603 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.603 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.603 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:08.603 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.548342+0000", 2024-09-11T07:49:08.603 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002968428 2024-09-11T07:49:08.603 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.603 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.603 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:08.603 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.548388+0000", 2024-09-11T07:49:08.603 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6269e-05 2024-09-11T07:49:08.604 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:08.604 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:08.604 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:08.604 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:36.548424+0000", 2024-09-11T07:49:08.604 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5775999999999998e-05 2024-09-11T07:49:08.604 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.604 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:08.604 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.604 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:08.604 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:08.604 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:49:08.659 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:08.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:08.684 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:08.684 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:08.785 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:08.812 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:08.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:08.813 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:08.913 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:08.961 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:08.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:08.962 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:09.063 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:09.092 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:09.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:09.093 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:09.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:08 smithi022 ceph-mon[29462]: osdmap e460: 8 total, 8 up, 6 in 2024-09-11T07:49:09.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:49:09.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:08 smithi022 ceph-mon[29462]: pgmap v1051: 158 pgs: 1 active+clean+remapped, 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 2.0 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 760 op/s; 9548/99879 objects misplaced (9.560%); 16 MiB/s, 261 objects/s recovering 2024-09-11T07:49:09.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:08 smithi022 ceph-mon[37330]: osdmap e460: 8 total, 8 up, 6 in 2024-09-11T07:49:09.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:49:09.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:08 smithi022 ceph-mon[37330]: pgmap v1051: 158 pgs: 1 active+clean+remapped, 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 2.0 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 760 op/s; 9548/99879 objects misplaced (9.560%); 16 MiB/s, 261 objects/s recovering 2024-09-11T07:49:09.173 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:08 smithi104 ceph-mon[33507]: osdmap e460: 8 total, 8 up, 6 in 2024-09-11T07:49:09.173 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:49:09.173 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:08 smithi104 ceph-mon[33507]: pgmap v1051: 158 pgs: 1 active+clean+remapped, 4 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 150 active+clean; 2.0 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 760 op/s; 9548/99879 objects misplaced (9.560%); 16 MiB/s, 261 objects/s recovering 2024-09-11T07:49:09.194 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:09.240 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:09.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:09.241 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:09.274 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:49:09.342 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:09.370 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:09.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:09.371 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:09.438 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-11T07:49:09.473 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:09.516 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:09.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:09.517 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:09.618 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:09.657 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:09.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:09.658 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:09.759 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:09.793 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:09.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:09.794 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:09.804 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:49:09.896 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:09.924 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:09.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:09.925 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:09.949 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:09 smithi022 ceph-mon[29462]: osdmap e461: 8 total, 8 up, 6 in 2024-09-11T07:49:09.950 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:09 smithi022 ceph-mon[37330]: osdmap e461: 8 total, 8 up, 6 in 2024-09-11T07:49:10.025 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:10.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:10.067 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:10.067 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:10.082 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:09 smithi104 ceph-mon[33507]: osdmap e461: 8 total, 8 up, 6 in 2024-09-11T07:49:10.168 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:10.208 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:10.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:10.209 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:10.310 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:10.357 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:10.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:10.359 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:10.459 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:10.475 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:49:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:49:10] "GET /metrics HTTP/1.1" 200 225248 "" "Prometheus/2.43.0" 2024-09-11T07:49:10.484 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:10.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:10.485 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:10.586 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:10.618 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:10.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:10.619 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:10.683 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:49:10 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:49:10.720 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:10.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:10.749 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:10.749 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:10.850 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:10.887 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:10.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:10.888 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:10.989 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:10.990 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:10 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:49:10.990 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:10 smithi104 ceph-mon[33507]: pgmap v1053: 158 pgs: 1 active+clean+remapped, 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 153 active+clean; 1.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.25k op/s; 7606/92610 objects misplaced (8.213%); 32 MiB/s, 510 objects/s recovering 2024-09-11T07:49:11.021 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:11.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:11.022 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:11.123 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:11.158 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:11.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:11.159 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:11.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:10 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:49:11.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:10 smithi022 ceph-mon[29462]: pgmap v1053: 158 pgs: 1 active+clean+remapped, 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 153 active+clean; 1.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.25k op/s; 7606/92610 objects misplaced (8.213%); 32 MiB/s, 510 objects/s recovering 2024-09-11T07:49:11.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:10 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:49:11.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:10 smithi022 ceph-mon[37330]: pgmap v1053: 158 pgs: 1 active+clean+remapped, 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 153 active+clean; 1.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.25k op/s; 7606/92610 objects misplaced (8.213%); 32 MiB/s, 510 objects/s recovering 2024-09-11T07:49:11.261 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:11.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:11.303 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:11.303 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:11.404 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:11.443 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:11.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:11.444 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:11.545 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:11.578 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:11.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:11.579 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:11.608 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:49:11.608 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:49:11.608 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 0 2024-09-11T07:49:11.608 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:49:11.680 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:11.715 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:11.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:11.716 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:11.817 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:11.853 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:11.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:11.854 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:11.865 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is unset 2024-09-11T07:49:11.955 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:11.983 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:11.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:11.984 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:12.085 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:12.113 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:12.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:12.114 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:12.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:11 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/797874008' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:49:12.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:11 smithi022 ceph-mon[29462]: from='client.26152 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:49:12.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:11 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/797874008' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:49:12.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:11 smithi022 ceph-mon[37330]: from='client.26152 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:49:12.214 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:12.253 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:12.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:12.254 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:12.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:11 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/797874008' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:49:12.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:11 smithi104 ceph-mon[33507]: from='client.26152 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:49:12.353 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-11T07:49:12.355 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:12.383 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:12.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:12.384 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:12.485 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:12.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:12.525 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:12.526 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:12.626 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:12.667 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:12.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:12.668 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:12.707 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:49:12.769 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:12.806 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:12.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:12.807 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:12.908 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:12.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:12 smithi022 ceph-mon[29462]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:49:12.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:12 smithi022 ceph-mon[29462]: from='client.26152 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:49:12.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:12 smithi022 ceph-mon[29462]: osdmap e462: 8 total, 8 up, 6 in 2024-09-11T07:49:12.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:12 smithi022 ceph-mon[29462]: pgmap v1055: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 1.8 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 907 B/s rd, 0 B/s wr, 657 op/s; 7345/88764 objects misplaced (8.275%); 14 MiB/s, 216 objects/s recovering 2024-09-11T07:49:12.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:12 smithi022 ceph-mon[37330]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:49:12.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:12 smithi022 ceph-mon[37330]: from='client.26152 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:49:12.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:12 smithi022 ceph-mon[37330]: osdmap e462: 8 total, 8 up, 6 in 2024-09-11T07:49:12.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:12 smithi022 ceph-mon[37330]: pgmap v1055: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 1.8 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 907 B/s rd, 0 B/s wr, 657 op/s; 7345/88764 objects misplaced (8.275%); 14 MiB/s, 216 objects/s recovering 2024-09-11T07:49:12.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:12.934 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:12.935 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:13.036 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:13.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:13.077 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:13.078 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:13.121 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:12 smithi104 ceph-mon[33507]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:49:13.122 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:12 smithi104 ceph-mon[33507]: from='client.26152 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:49:13.122 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:12 smithi104 ceph-mon[33507]: osdmap e462: 8 total, 8 up, 6 in 2024-09-11T07:49:13.122 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:12 smithi104 ceph-mon[33507]: pgmap v1055: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 1.8 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 907 B/s rd, 0 B/s wr, 657 op/s; 7345/88764 objects misplaced (8.275%); 14 MiB/s, 216 objects/s recovering 2024-09-11T07:49:13.179 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:13.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:13.210 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:13.210 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:13.311 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:13.348 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:13.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:13.349 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:13.450 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:13.478 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:13.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:13.479 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:13.557 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 5, 6, 7] out_osds: [4, 2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:49:13.557 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:49:13.558 INFO:tasks.thrashosds.thrasher:Removing osd 7, in_osds are: [0, 1, 3, 5, 6, 7] 2024-09-11T07:49:13.558 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd out 7 2024-09-11T07:49:13.581 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:13.602 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:13.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:13.603 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:13.704 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:13.746 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:13.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:13.747 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:13.848 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:13.877 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:13.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:13.878 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:13.978 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:14.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:14.018 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:14.018 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:14.113 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:13 smithi104 ceph-mon[33507]: 5.7 scrub starts 2024-09-11T07:49:14.113 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:13 smithi104 ceph-mon[33507]: 5.7 scrub ok 2024-09-11T07:49:14.113 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:13 smithi104 ceph-mon[33507]: 6.14 scrub starts 2024-09-11T07:49:14.113 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:13 smithi104 ceph-mon[33507]: 6.14 scrub ok 2024-09-11T07:49:14.113 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:49:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:49:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:49:14.120 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:14.156 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:14.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:14.157 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:14.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:13 smithi022 ceph-mon[29462]: 5.7 scrub starts 2024-09-11T07:49:14.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:13 smithi022 ceph-mon[29462]: 5.7 scrub ok 2024-09-11T07:49:14.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:13 smithi022 ceph-mon[29462]: 6.14 scrub starts 2024-09-11T07:49:14.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:13 smithi022 ceph-mon[29462]: 6.14 scrub ok 2024-09-11T07:49:14.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:13 smithi022 ceph-mon[37330]: 5.7 scrub starts 2024-09-11T07:49:14.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:13 smithi022 ceph-mon[37330]: 5.7 scrub ok 2024-09-11T07:49:14.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:13 smithi022 ceph-mon[37330]: 6.14 scrub starts 2024-09-11T07:49:14.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:13 smithi022 ceph-mon[37330]: 6.14 scrub ok 2024-09-11T07:49:14.258 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:14.294 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:14.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:14.295 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:14.396 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:14.418 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:14.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:14.419 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:14.520 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:14.559 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:49:14.559 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:49:14.559 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2024-09-11T07:49:14.559 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2024-09-11T07:49:14.559 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:49:14.564 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:14.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:14.565 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:14.666 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:14.694 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:14.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:14.695 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:14.796 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:14.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:14.835 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:14.835 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:14.935 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:14.965 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:14.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:14.966 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:15.067 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:15.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:15.109 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:15.110 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:15.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:14 smithi022 ceph-mon[29462]: 2.c scrub starts 2024-09-11T07:49:15.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:14 smithi022 ceph-mon[29462]: 2.c scrub ok 2024-09-11T07:49:15.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:14 smithi022 ceph-mon[29462]: 4.1c scrub starts 2024-09-11T07:49:15.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:14 smithi022 ceph-mon[29462]: 4.1c scrub ok 2024-09-11T07:49:15.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:14 smithi022 ceph-mon[29462]: pgmap v1056: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 1.8 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 770 B/s rd, 0 B/s wr, 558 op/s; 7345/88764 objects misplaced (8.275%); 11 MiB/s, 183 objects/s recovering 2024-09-11T07:49:15.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:14 smithi022 ceph-mon[37330]: 2.c scrub starts 2024-09-11T07:49:15.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:14 smithi022 ceph-mon[37330]: 2.c scrub ok 2024-09-11T07:49:15.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:14 smithi022 ceph-mon[37330]: 4.1c scrub starts 2024-09-11T07:49:15.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:14 smithi022 ceph-mon[37330]: 4.1c scrub ok 2024-09-11T07:49:15.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:14 smithi022 ceph-mon[37330]: pgmap v1056: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 1.8 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 770 B/s rd, 0 B/s wr, 558 op/s; 7345/88764 objects misplaced (8.275%); 11 MiB/s, 183 objects/s recovering 2024-09-11T07:49:15.211 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:15.233 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:15.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:15.234 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:15.270 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:14 smithi104 ceph-mon[33507]: 2.c scrub starts 2024-09-11T07:49:15.270 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:14 smithi104 ceph-mon[33507]: 2.c scrub ok 2024-09-11T07:49:15.270 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:14 smithi104 ceph-mon[33507]: 4.1c scrub starts 2024-09-11T07:49:15.270 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:14 smithi104 ceph-mon[33507]: 4.1c scrub ok 2024-09-11T07:49:15.270 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:14 smithi104 ceph-mon[33507]: pgmap v1056: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 1.8 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 770 B/s rd, 0 B/s wr, 558 op/s; 7345/88764 objects misplaced (8.275%); 11 MiB/s, 183 objects/s recovering 2024-09-11T07:49:15.307 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-11T07:49:15.335 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:15.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:15.357 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:15.358 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:15.458 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:15.480 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:15.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:15.482 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:15.583 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:15.589 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:49:15.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:15.625 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:15.625 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:15.726 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:15.766 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:15.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:15.768 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:15.869 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:15.902 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:15.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:15.903 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:15.988 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:15 smithi104 ceph-mon[33507]: 11.d scrub starts 2024-09-11T07:49:15.988 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:15 smithi104 ceph-mon[33507]: 2.1 scrub starts 2024-09-11T07:49:15.988 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:15 smithi104 ceph-mon[33507]: 2.1 scrub ok 2024-09-11T07:49:15.988 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:15 smithi104 ceph-mon[33507]: 2.16 scrub starts 2024-09-11T07:49:15.988 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:15 smithi104 ceph-mon[33507]: 2.16 scrub ok 2024-09-11T07:49:16.004 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:16.050 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:16.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:16.052 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:16.152 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:16.166 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:15 smithi022 ceph-mon[37330]: 11.d scrub starts 2024-09-11T07:49:16.166 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:15 smithi022 ceph-mon[37330]: 2.1 scrub starts 2024-09-11T07:49:16.166 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:15 smithi022 ceph-mon[37330]: 2.1 scrub ok 2024-09-11T07:49:16.166 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:15 smithi022 ceph-mon[37330]: 2.16 scrub starts 2024-09-11T07:49:16.166 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:15 smithi022 ceph-mon[37330]: 2.16 scrub ok 2024-09-11T07:49:16.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:15 smithi022 ceph-mon[29462]: 11.d scrub starts 2024-09-11T07:49:16.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:15 smithi022 ceph-mon[29462]: 2.1 scrub starts 2024-09-11T07:49:16.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:15 smithi022 ceph-mon[29462]: 2.1 scrub ok 2024-09-11T07:49:16.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:15 smithi022 ceph-mon[29462]: 2.16 scrub starts 2024-09-11T07:49:16.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:15 smithi022 ceph-mon[29462]: 2.16 scrub ok 2024-09-11T07:49:16.197 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:16.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:16.198 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:16.299 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:16.327 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:16.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:16.328 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:16.429 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:16.455 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:16.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:16.456 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:16.556 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:16.597 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:16.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:16.598 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:16.698 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:16.724 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:16.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:16.725 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:16.825 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:16.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:16.857 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:16.858 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:16.958 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:17.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:17.002 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:17.002 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:16 smithi104 ceph-mon[33507]: 11.d scrub ok 2024-09-11T07:49:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:16 smithi104 ceph-mon[33507]: 4.a scrub starts 2024-09-11T07:49:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:16 smithi104 ceph-mon[33507]: 4.a scrub ok 2024-09-11T07:49:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:16 smithi104 ceph-mon[33507]: 2.3 scrub starts 2024-09-11T07:49:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:16 smithi104 ceph-mon[33507]: 2.3 scrub ok 2024-09-11T07:49:17.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:16 smithi104 ceph-mon[33507]: pgmap v1057: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 1.4 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.21k op/s; 5838/70869 objects misplaced (8.238%); 9.5 MiB/s, 152 objects/s recovering 2024-09-11T07:49:17.103 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:17.143 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:17.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:17.144 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:17.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:16 smithi022 ceph-mon[29462]: 11.d scrub ok 2024-09-11T07:49:17.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:16 smithi022 ceph-mon[29462]: 4.a scrub starts 2024-09-11T07:49:17.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:16 smithi022 ceph-mon[29462]: 4.a scrub ok 2024-09-11T07:49:17.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:16 smithi022 ceph-mon[29462]: 2.3 scrub starts 2024-09-11T07:49:17.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:16 smithi022 ceph-mon[29462]: 2.3 scrub ok 2024-09-11T07:49:17.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:16 smithi022 ceph-mon[29462]: pgmap v1057: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 1.4 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.21k op/s; 5838/70869 objects misplaced (8.238%); 9.5 MiB/s, 152 objects/s recovering 2024-09-11T07:49:17.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:16 smithi022 ceph-mon[37330]: 11.d scrub ok 2024-09-11T07:49:17.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:16 smithi022 ceph-mon[37330]: 4.a scrub starts 2024-09-11T07:49:17.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:16 smithi022 ceph-mon[37330]: 4.a scrub ok 2024-09-11T07:49:17.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:16 smithi022 ceph-mon[37330]: 2.3 scrub starts 2024-09-11T07:49:17.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:16 smithi022 ceph-mon[37330]: 2.3 scrub ok 2024-09-11T07:49:17.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:16 smithi022 ceph-mon[37330]: pgmap v1057: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 1.4 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.21k op/s; 5838/70869 objects misplaced (8.238%); 9.5 MiB/s, 152 objects/s recovering 2024-09-11T07:49:17.245 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:17.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:17.276 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:17.276 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:17.377 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:17.421 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:17.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:17.422 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:17.425 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:49:17.425 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2024-09-11T07:49:17.425 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2024-09-11T07:49:17.425 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2024-09-11T07:49:17.425 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.425 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196432 9.d 9:b43c5e98:::benchmark_data_smithi022_133943_object69841:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:49:17.425 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.509114+0000", 2024-09-11T07:49:17.425 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 539.91194027100005, 2024-09-11T07:49:17.425 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.152507894, 2024-09-11T07:49:17.425 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:17.425 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:17.425 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:17.426 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:49:17.426 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:49:17.426 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196432 2024-09-11T07:49:17.426 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.426 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:17.426 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.426 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:17.426 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509114+0000", 2024-09-11T07:49:17.426 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.426 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.426 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.426 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:17.426 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509114+0000", 2024-09-11T07:49:17.426 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.427 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.427 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.427 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:17.427 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509112+0000", 2024-09-11T07:49:17.427 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:49:17.427 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.427 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.427 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:17.427 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509119+0000", 2024-09-11T07:49:17.427 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3439999999999997e-06 2024-09-11T07:49:17.427 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.427 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.427 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:17.427 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509130+0000", 2024-09-11T07:49:17.428 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1066999999999999e-05 2024-09-11T07:49:17.428 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.428 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.428 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:17.428 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509136+0000", 2024-09-11T07:49:17.428 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0460000000000002e-06 2024-09-11T07:49:17.428 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.428 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.428 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:17.428 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509237+0000", 2024-09-11T07:49:17.428 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000101338 2024-09-11T07:49:17.428 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.428 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.428 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:17.429 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509361+0000", 2024-09-11T07:49:17.429 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000123287 2024-09-11T07:49:17.429 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.429 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.429 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,4,6,7", 2024-09-11T07:49:17.429 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509470+0000", 2024-09-11T07:49:17.429 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000109986 2024-09-11T07:49:17.429 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.429 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.429 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:17.429 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.515270+0000", 2024-09-11T07:49:17.429 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0057995310000000001 2024-09-11T07:49:17.429 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.429 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.430 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:17.430 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.515306+0000", 2024-09-11T07:49:17.430 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5920999999999997e-05 2024-09-11T07:49:17.430 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.430 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.430 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:17.430 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.515314+0000", 2024-09-11T07:49:17.430 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7680000000000008e-06 2024-09-11T07:49:17.430 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.430 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.430 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:17.430 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.659682+0000", 2024-09-11T07:49:17.430 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14436806299999999 2024-09-11T07:49:17.430 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.430 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.431 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:17.431 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.661408+0000", 2024-09-11T07:49:17.431 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0017266499999999999 2024-09-11T07:49:17.431 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.431 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.431 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:17.431 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.661490+0000", 2024-09-11T07:49:17.431 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1951000000000002e-05 2024-09-11T07:49:17.431 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.431 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.431 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:17.431 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.661622+0000", 2024-09-11T07:49:17.431 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013185900000000001 2024-09-11T07:49:17.431 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.432 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:17.432 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.432 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.432 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.432 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196436 9.f 9:f28fa10b:::benchmark_data_smithi022_133943_object69845:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:49:17.432 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.509149+0000", 2024-09-11T07:49:17.432 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 539.91190608800002, 2024-09-11T07:49:17.432 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.15385757, 2024-09-11T07:49:17.432 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:17.432 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:17.432 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:17.432 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:49:17.432 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:49:17.432 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196436 2024-09-11T07:49:17.433 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.433 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:17.433 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.433 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:17.433 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509149+0000", 2024-09-11T07:49:17.433 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.433 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.433 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.433 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:17.433 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509149+0000", 2024-09-11T07:49:17.433 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.433 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.433 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.434 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:17.434 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509148+0000", 2024-09-11T07:49:17.434 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:49:17.434 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.434 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.434 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:17.434 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509152+0000", 2024-09-11T07:49:17.434 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3309999999999997e-06 2024-09-11T07:49:17.434 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.434 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.434 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:17.434 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509158+0000", 2024-09-11T07:49:17.434 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0090000000000004e-06 2024-09-11T07:49:17.434 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.435 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.435 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:17.435 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509162+0000", 2024-09-11T07:49:17.435 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9149999999999998e-06 2024-09-11T07:49:17.435 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.435 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.435 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:17.435 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509226+0000", 2024-09-11T07:49:17.435 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3969999999999999e-05 2024-09-11T07:49:17.435 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.435 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.435 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:17.435 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509327+0000", 2024-09-11T07:49:17.435 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000101217 2024-09-11T07:49:17.436 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.436 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.436 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:49:17.436 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509426+0000", 2024-09-11T07:49:17.436 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8525999999999998e-05 2024-09-11T07:49:17.436 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.436 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.436 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:17.436 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.512876+0000", 2024-09-11T07:49:17.436 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003449939 2024-09-11T07:49:17.436 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.436 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.436 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:17.436 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.658382+0000", 2024-09-11T07:49:17.437 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.145506042 2024-09-11T07:49:17.437 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.437 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.437 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:17.437 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.662896+0000", 2024-09-11T07:49:17.437 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0045144510000000001 2024-09-11T07:49:17.437 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.437 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.437 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:17.438 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.662971+0000", 2024-09-11T07:49:17.438 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4895999999999996e-05 2024-09-11T07:49:17.438 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.438 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.438 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:17.438 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.663006+0000", 2024-09-11T07:49:17.438 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5099999999999999e-05 2024-09-11T07:49:17.438 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.438 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:17.438 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.438 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.438 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.438 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196438 9.7 9:e639c272:::benchmark_data_smithi022_133943_object69847:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:49:17.439 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.509172+0000", 2024-09-11T07:49:17.439 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 539.91188293899995, 2024-09-11T07:49:17.439 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.154085528, 2024-09-11T07:49:17.439 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:17.439 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:49:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:49:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196438 2024-09-11T07:49:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509172+0000", 2024-09-11T07:49:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509172+0000", 2024-09-11T07:49:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509171+0000", 2024-09-11T07:49:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:49:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509174+0000", 2024-09-11T07:49:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2679999999999999e-06 2024-09-11T07:49:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509179+0000", 2024-09-11T07:49:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9119999999999997e-06 2024-09-11T07:49:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.509183+0000", 2024-09-11T07:49:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4410000000000002e-06 2024-09-11T07:49:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.510817+0000", 2024-09-11T07:49:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0016344020000000001 2024-09-11T07:49:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.510945+0000", 2024-09-11T07:49:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012800999999999999 2024-09-11T07:49:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2,3,7", 2024-09-11T07:49:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.511375+0000", 2024-09-11T07:49:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00042995300000000001 2024-09-11T07:49:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.659619+0000", 2024-09-11T07:49:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.148244129 2024-09-11T07:49:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.659674+0000", 2024-09-11T07:49:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4605000000000003e-05 2024-09-11T07:49:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.659715+0000", 2024-09-11T07:49:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0935e-05 2024-09-11T07:49:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.662817+0000", 2024-09-11T07:49:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0031017319999999998 2024-09-11T07:49:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.663041+0000", 2024-09-11T07:49:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000224303 2024-09-11T07:49:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.663089+0000", 2024-09-11T07:49:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8464000000000002e-05 2024-09-11T07:49:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.663257+0000", 2024-09-11T07:49:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016791399999999999 2024-09-11T07:49:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196563 9.6 9:63ebaa83:::benchmark_data_smithi022_133943_object69972:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:49:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.771425+0000", 2024-09-11T07:49:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 539.64962912299995, 2024-09-11T07:49:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.153416993, 2024-09-11T07:49:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:49:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:49:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196563 2024-09-11T07:49:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771425+0000", 2024-09-11T07:49:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.448 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:17.448 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771425+0000", 2024-09-11T07:49:17.448 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.448 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.448 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.448 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:17.448 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771424+0000", 2024-09-11T07:49:17.448 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:49:17.448 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.448 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.448 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:17.448 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771430+0000", 2024-09-11T07:49:17.448 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4540000000000004e-06 2024-09-11T07:49:17.448 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.449 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.449 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:17.449 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771438+0000", 2024-09-11T07:49:17.449 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.418e-06 2024-09-11T07:49:17.449 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.449 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.449 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:17.449 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771442+0000", 2024-09-11T07:49:17.449 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.622e-06 2024-09-11T07:49:17.449 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.449 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.449 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:17.449 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771560+0000", 2024-09-11T07:49:17.449 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000117298 2024-09-11T07:49:17.449 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.450 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.450 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:17.450 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.771669+0000", 2024-09-11T07:49:17.450 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000109386 2024-09-11T07:49:17.450 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.450 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.450 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:49:17.450 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.772085+0000", 2024-09-11T07:49:17.450 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00041578699999999998 2024-09-11T07:49:17.450 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.450 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.450 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:17.450 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.776041+0000", 2024-09-11T07:49:17.451 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0039559069999999998 2024-09-11T07:49:17.451 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.451 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.451 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:17.451 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.782755+0000", 2024-09-11T07:49:17.451 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0067147760000000004 2024-09-11T07:49:17.451 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.451 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.451 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:17.451 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924754+0000", 2024-09-11T07:49:17.451 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14199824599999999 2024-09-11T07:49:17.451 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.451 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.451 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:17.451 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924797+0000", 2024-09-11T07:49:17.452 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2988999999999998e-05 2024-09-11T07:49:17.452 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.452 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.452 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:17.452 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924842+0000", 2024-09-11T07:49:17.452 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5884000000000003e-05 2024-09-11T07:49:17.452 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.452 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:17.452 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.452 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.452 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.452 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:196577 9.13 9:c894ba9e:::benchmark_data_smithi022_133943_object69986:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e299)", 2024-09-11T07:49:17.452 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:17.785754+0000", 2024-09-11T07:49:17.453 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 539.63530100399998, 2024-09-11T07:49:17.453 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.167572637, 2024-09-11T07:49:17.453 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:17.453 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:17.453 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:17.453 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:49:17.453 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:49:17.453 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 196577 2024-09-11T07:49:17.453 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.453 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:17.453 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.453 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:17.453 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785754+0000", 2024-09-11T07:49:17.453 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.454 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.454 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.454 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:17.454 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785754+0000", 2024-09-11T07:49:17.454 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.454 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.454 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.454 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:17.454 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785752+0000", 2024-09-11T07:49:17.454 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:49:17.454 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.454 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.454 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:17.454 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785757+0000", 2024-09-11T07:49:17.455 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0010000000000001e-06 2024-09-11T07:49:17.455 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.455 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.455 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:17.455 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785765+0000", 2024-09-11T07:49:17.455 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2670000000000006e-06 2024-09-11T07:49:17.455 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.455 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.455 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:17.455 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.785771+0000", 2024-09-11T07:49:17.455 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.242e-06 2024-09-11T07:49:17.455 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.455 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.455 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:17.456 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924515+0000", 2024-09-11T07:49:17.456 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13874467300000001 2024-09-11T07:49:17.456 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.456 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.456 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:17.456 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.924668+0000", 2024-09-11T07:49:17.456 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00015245999999999999 2024-09-11T07:49:17.456 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.456 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.456 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,6,7", 2024-09-11T07:49:17.456 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.925194+0000", 2024-09-11T07:49:17.456 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00052637400000000003 2024-09-11T07:49:17.456 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.456 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.457 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:17.457 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.930777+0000", 2024-09-11T07:49:17.457 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00558319 2024-09-11T07:49:17.457 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.457 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.457 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:17.457 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.930807+0000", 2024-09-11T07:49:17.457 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9363999999999999e-05 2024-09-11T07:49:17.457 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.457 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.457 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:17.457 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.931376+0000", 2024-09-11T07:49:17.457 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00056952599999999999 2024-09-11T07:49:17.457 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.458 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.458 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:17.458 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.953159+0000", 2024-09-11T07:49:17.458 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.021782632999999999 2024-09-11T07:49:17.458 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.458 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.458 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:17.458 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.953206+0000", 2024-09-11T07:49:17.458 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7624999999999999e-05 2024-09-11T07:49:17.458 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.458 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.458 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:17.458 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:17.953326+0000", 2024-09-11T07:49:17.458 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000119883 2024-09-11T07:49:17.459 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.459 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:17.459 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.459 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.459 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.459 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:231567 9.6 9:62bf3667:::benchmark_data_smithi022_133943_object104976:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:49:17.459 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:56.613955+0000", 2024-09-11T07:49:17.459 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 500.80709975100001, 2024-09-11T07:49:17.459 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14854477399999999, 2024-09-11T07:49:17.459 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:17.459 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:17.459 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:17.459 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:49:17.459 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:49:17.460 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 231567 2024-09-11T07:49:17.460 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.460 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:17.460 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.460 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:17.460 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.613955+0000", 2024-09-11T07:49:17.460 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.460 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.460 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.460 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:17.460 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.613955+0000", 2024-09-11T07:49:17.460 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.460 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.460 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.460 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:17.461 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.613953+0000", 2024-09-11T07:49:17.461 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:49:17.461 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.461 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.461 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:17.461 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.613960+0000", 2024-09-11T07:49:17.461 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3200000000000002e-06 2024-09-11T07:49:17.461 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.461 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.461 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:17.461 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.613988+0000", 2024-09-11T07:49:17.461 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.745e-05 2024-09-11T07:49:17.461 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.461 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.461 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:17.462 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.613994+0000", 2024-09-11T07:49:17.462 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0270000000000001e-06 2024-09-11T07:49:17.462 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.462 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.462 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:17.462 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.614040+0000", 2024-09-11T07:49:17.462 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6403999999999998e-05 2024-09-11T07:49:17.462 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.462 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.462 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:17.462 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.614209+0000", 2024-09-11T07:49:17.462 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016869000000000001 2024-09-11T07:49:17.462 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.462 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.462 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:49:17.463 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.614696+0000", 2024-09-11T07:49:17.463 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00048757100000000001 2024-09-11T07:49:17.463 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.463 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.463 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:17.463 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.617583+0000", 2024-09-11T07:49:17.463 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0028868129999999998 2024-09-11T07:49:17.463 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.463 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.463 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:17.463 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.618363+0000", 2024-09-11T07:49:17.463 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00078008600000000004 2024-09-11T07:49:17.463 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.463 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.463 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:17.464 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.762435+0000", 2024-09-11T07:49:17.464 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.144071592 2024-09-11T07:49:17.464 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.464 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.464 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:17.464 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.762464+0000", 2024-09-11T07:49:17.464 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9068999999999999e-05 2024-09-11T07:49:17.464 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.464 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.464 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:17.464 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.762500+0000", 2024-09-11T07:49:17.464 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5751999999999999e-05 2024-09-11T07:49:17.464 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.464 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:17.465 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.465 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.465 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.465 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:232544 9.13 9:cb4ef5ab:::benchmark_data_smithi022_133943_object105953:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:49:17.465 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:57.689887+0000", 2024-09-11T07:49:17.465 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 499.73116764100001, 2024-09-11T07:49:17.465 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.186799876, 2024-09-11T07:49:17.465 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:17.465 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:17.465 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:17.465 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:49:17.465 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:49:17.465 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 232544 2024-09-11T07:49:17.465 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.466 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:17.466 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.466 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:17.466 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689887+0000", 2024-09-11T07:49:17.466 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.466 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.466 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.466 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:17.466 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689887+0000", 2024-09-11T07:49:17.466 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.466 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.466 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.466 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:17.467 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689885+0000", 2024-09-11T07:49:17.467 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:49:17.467 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.467 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.467 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:17.467 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689891+0000", 2024-09-11T07:49:17.467 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0310000000000004e-06 2024-09-11T07:49:17.467 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.467 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.467 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:17.467 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689903+0000", 2024-09-11T07:49:17.467 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2167000000000001e-05 2024-09-11T07:49:17.467 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.467 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.467 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:17.468 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689909+0000", 2024-09-11T07:49:17.468 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7620000000000001e-06 2024-09-11T07:49:17.468 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.468 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.468 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:17.468 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689947+0000", 2024-09-11T07:49:17.468 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7960999999999998e-05 2024-09-11T07:49:17.468 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.468 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.468 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:17.468 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.690025+0000", 2024-09-11T07:49:17.468 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7885000000000004e-05 2024-09-11T07:49:17.468 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.468 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.469 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:49:17.469 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.690473+0000", 2024-09-11T07:49:17.469 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00044783300000000002 2024-09-11T07:49:17.469 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.469 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.469 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:17.469 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.692445+0000", 2024-09-11T07:49:17.469 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001972461 2024-09-11T07:49:17.469 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.469 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.469 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:17.469 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.692900+0000", 2024-09-11T07:49:17.469 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00045435300000000001 2024-09-11T07:49:17.470 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.470 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.470 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:17.470 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.876606+0000", 2024-09-11T07:49:17.470 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18370584700000001 2024-09-11T07:49:17.470 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.470 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.470 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:17.470 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.876647+0000", 2024-09-11T07:49:17.470 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1140999999999998e-05 2024-09-11T07:49:17.470 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.470 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.470 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:17.470 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.876687+0000", 2024-09-11T07:49:17.471 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0153e-05 2024-09-11T07:49:17.471 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.471 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:17.471 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.471 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.471 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.471 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:234546 9.6 9:65687430:::benchmark_data_smithi022_133943_object107955:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e320)", 2024-09-11T07:49:17.471 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:41:00.355245+0000", 2024-09-11T07:49:17.471 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 497.06580940800001, 2024-09-11T07:49:17.471 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17097563599999999, 2024-09-11T07:49:17.471 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:17.471 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:17.471 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:17.471 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:49:17.472 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:49:17.472 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 234546 2024-09-11T07:49:17.472 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.472 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:17.472 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.472 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:17.472 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355245+0000", 2024-09-11T07:49:17.472 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.472 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.472 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.472 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:17.472 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355245+0000", 2024-09-11T07:49:17.472 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.473 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.473 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.473 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:17.473 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355244+0000", 2024-09-11T07:49:17.473 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:49:17.473 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.473 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.473 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:17.473 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355248+0000", 2024-09-11T07:49:17.473 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.529e-06 2024-09-11T07:49:17.473 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.473 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.473 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:17.473 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355255+0000", 2024-09-11T07:49:17.474 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1289999999999998e-06 2024-09-11T07:49:17.474 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.474 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.474 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:17.474 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355259+0000", 2024-09-11T07:49:17.474 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7149999999999997e-06 2024-09-11T07:49:17.474 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.474 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.474 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:17.474 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355306+0000", 2024-09-11T07:49:17.474 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6631999999999997e-05 2024-09-11T07:49:17.474 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.474 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.474 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:17.475 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355383+0000", 2024-09-11T07:49:17.475 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6509999999999998e-05 2024-09-11T07:49:17.475 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.475 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.475 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:49:17.475 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355796+0000", 2024-09-11T07:49:17.475 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00041318000000000003 2024-09-11T07:49:17.475 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.475 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.475 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:17.475 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.357852+0000", 2024-09-11T07:49:17.475 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0020561059999999998 2024-09-11T07:49:17.475 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.475 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.476 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:17.476 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.359012+0000", 2024-09-11T07:49:17.476 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001160295 2024-09-11T07:49:17.476 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.476 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.476 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:17.476 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.526128+0000", 2024-09-11T07:49:17.476 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.167115923 2024-09-11T07:49:17.476 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.476 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.476 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:17.476 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.526176+0000", 2024-09-11T07:49:17.476 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8310000000000003e-05 2024-09-11T07:49:17.476 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.477 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.477 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:17.477 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.526221+0000", 2024-09-11T07:49:17.477 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4447000000000003e-05 2024-09-11T07:49:17.477 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.477 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:17.477 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.477 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.477 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.477 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:70288 10.6 e356/346 10:6714b47c:::benchmark_data_smithi022_163866_object70287:head v 356'4440, mlcod=356'4437)", 2024-09-11T07:49:17.477 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:53.786739+0000", 2024-09-11T07:49:17.477 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 383.63431596800001, 2024-09-11T07:49:17.477 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.164540297, 2024-09-11T07:49:17.478 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:17.478 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:17.478 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:17.478 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.478 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:17.478 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786739+0000", 2024-09-11T07:49:17.478 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.478 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.478 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.478 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:17.478 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786739+0000", 2024-09-11T07:49:17.478 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.478 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.478 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.478 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:17.479 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786737+0000", 2024-09-11T07:49:17.479 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:49:17.479 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.479 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.479 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:17.479 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786793+0000", 2024-09-11T07:49:17.479 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.516e-05 2024-09-11T07:49:17.479 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.479 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.479 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:17.479 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786802+0000", 2024-09-11T07:49:17.479 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0149999999999992e-06 2024-09-11T07:49:17.479 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.479 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.480 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:17.480 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786807+0000", 2024-09-11T07:49:17.480 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3900000000000001e-06 2024-09-11T07:49:17.480 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.480 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.480 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:17.480 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786852+0000", 2024-09-11T07:49:17.480 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5426000000000002e-05 2024-09-11T07:49:17.480 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.480 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.481 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:17.481 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786869+0000", 2024-09-11T07:49:17.481 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6477000000000002e-05 2024-09-11T07:49:17.481 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.481 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.481 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:17.482 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951240+0000", 2024-09-11T07:49:17.482 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16437090500000001 2024-09-11T07:49:17.482 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.482 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.482 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:17.482 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951279+0000", 2024-09-11T07:49:17.482 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9072999999999997e-05 2024-09-11T07:49:17.482 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.482 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:17.482 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.482 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.482 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.482 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:70292 10.a e356/341 10:547c94c8:::benchmark_data_smithi022_163866_object70291:head v 356'4331, mlcod=356'4329)", 2024-09-11T07:49:17.482 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:53.788575+0000", 2024-09-11T07:49:17.483 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 383.63247925899998, 2024-09-11T07:49:17.483 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16920853899999999, 2024-09-11T07:49:17.483 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:17.483 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:17.483 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:17.483 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.483 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:17.483 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.788575+0000", 2024-09-11T07:49:17.483 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.483 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.483 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.483 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:17.483 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.788575+0000", 2024-09-11T07:49:17.484 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.484 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.484 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.484 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:17.484 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.788574+0000", 2024-09-11T07:49:17.484 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:49:17.484 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.484 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.484 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:17.484 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.788630+0000", 2024-09-11T07:49:17.484 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6097999999999997e-05 2024-09-11T07:49:17.484 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.484 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.484 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:17.485 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951213+0000", 2024-09-11T07:49:17.485 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16258271199999999 2024-09-11T07:49:17.485 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.485 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.485 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:17.485 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951228+0000", 2024-09-11T07:49:17.485 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4562999999999999e-05 2024-09-11T07:49:17.485 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.485 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.485 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:17.485 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951332+0000", 2024-09-11T07:49:17.485 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000104002 2024-09-11T07:49:17.485 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.485 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.486 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:17.486 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951352+0000", 2024-09-11T07:49:17.486 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0638000000000001e-05 2024-09-11T07:49:17.486 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.486 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.486 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:17.486 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.957755+0000", 2024-09-11T07:49:17.486 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.006402451 2024-09-11T07:49:17.486 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.486 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.486 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:17.486 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.957784+0000", 2024-09-11T07:49:17.486 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9269e-05 2024-09-11T07:49:17.486 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.487 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:17.487 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.487 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.487 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.487 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:70291 10.3 e356/341 10:cdbcfc7c:::benchmark_data_smithi022_163866_object70290:head v 356'4299, mlcod=356'4297)", 2024-09-11T07:49:17.487 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:53.789090+0000", 2024-09-11T07:49:17.487 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 383.63196441899998, 2024-09-11T07:49:17.487 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16609167599999999, 2024-09-11T07:49:17.487 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:17.487 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:17.487 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:17.487 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.487 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:17.487 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.789090+0000", 2024-09-11T07:49:17.488 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.488 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.488 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.488 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:17.488 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.789090+0000", 2024-09-11T07:49:17.488 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.488 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.488 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.488 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:17.488 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.789089+0000", 2024-09-11T07:49:17.489 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:49:17.489 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.489 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.489 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:17.489 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.789132+0000", 2024-09-11T07:49:17.489 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2901000000000001e-05 2024-09-11T07:49:17.489 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.489 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.489 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:17.489 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951213+0000", 2024-09-11T07:49:17.489 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16208165599999999 2024-09-11T07:49:17.489 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.489 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.489 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:17.490 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951228+0000", 2024-09-11T07:49:17.490 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4430000000000001e-05 2024-09-11T07:49:17.490 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.490 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.490 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:17.490 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951350+0000", 2024-09-11T07:49:17.490 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000122213 2024-09-11T07:49:17.490 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.490 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.490 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:17.490 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951369+0000", 2024-09-11T07:49:17.490 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8819000000000001e-05 2024-09-11T07:49:17.490 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.491 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.491 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:17.491 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.955146+0000", 2024-09-11T07:49:17.491 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003776916 2024-09-11T07:49:17.491 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.491 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.491 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:17.491 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.955182+0000", 2024-09-11T07:49:17.491 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6123e-05 2024-09-11T07:49:17.491 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.491 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:17.491 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.491 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.491 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.492 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25990.0:192507 11.d e455/454 11:be361c01:::benchmark_data_smithi022_188626_object72699:head v 455'11931, mlcod=0'0)", 2024-09-11T07:49:17.492 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:48:48.300725+0000", 2024-09-11T07:49:17.492 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 29.120329363, 2024-09-11T07:49:17.492 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0080738870000000001, 2024-09-11T07:49:17.492 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:17.492 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:17.492 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:17.492 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.492 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:17.492 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.300725+0000", 2024-09-11T07:49:17.492 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.492 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.492 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.492 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:17.493 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.300725+0000", 2024-09-11T07:49:17.493 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.493 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.493 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.493 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:17.493 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.300724+0000", 2024-09-11T07:49:17.493 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:49:17.493 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.493 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.493 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:17.493 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.300732+0000", 2024-09-11T07:49:17.493 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.841e-06 2024-09-11T07:49:17.493 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.493 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.494 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:17.494 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.300742+0000", 2024-09-11T07:49:17.494 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.7960000000000007e-06 2024-09-11T07:49:17.494 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.494 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.494 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:17.494 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.300747+0000", 2024-09-11T07:49:17.494 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4759999999999997e-06 2024-09-11T07:49:17.494 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.494 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.494 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:17.494 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.300785+0000", 2024-09-11T07:49:17.494 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7528999999999999e-05 2024-09-11T07:49:17.494 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.494 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.495 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:17.495 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.300806+0000", 2024-09-11T07:49:17.495 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.124e-05 2024-09-11T07:49:17.495 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.495 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.495 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:17.495 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.308767+0000", 2024-09-11T07:49:17.495 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0079615869999999991 2024-09-11T07:49:17.495 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.495 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.495 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:17.495 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.308799+0000", 2024-09-11T07:49:17.495 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1646000000000002e-05 2024-09-11T07:49:17.495 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.496 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:17.496 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.496 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.496 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.496 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25990.0:203844 11.a e458/412 11:5dde9a9c:::benchmark_data_smithi022_188626_object84036:head v 458'12896, mlcod=458'12894)", 2024-09-11T07:49:17.496 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:02.188444+0000", 2024-09-11T07:49:17.496 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 15.232610489000001, 2024-09-11T07:49:17.496 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0082231930000000002, 2024-09-11T07:49:17.496 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:17.496 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:17.496 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:17.496 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.496 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:17.497 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:02.188444+0000", 2024-09-11T07:49:17.497 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.497 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.497 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.497 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:17.497 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:02.188444+0000", 2024-09-11T07:49:17.497 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.497 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.497 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.497 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:17.497 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:02.188443+0000", 2024-09-11T07:49:17.497 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:49:17.497 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.497 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.498 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:17.498 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:02.188453+0000", 2024-09-11T07:49:17.498 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8800000000000003e-06 2024-09-11T07:49:17.498 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.498 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.498 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:17.498 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:02.188462+0000", 2024-09-11T07:49:17.498 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2699999999999993e-06 2024-09-11T07:49:17.498 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.498 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.498 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:17.498 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:02.188466+0000", 2024-09-11T07:49:17.498 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8829999999999999e-06 2024-09-11T07:49:17.498 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.499 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.499 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:17.499 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:02.188508+0000", 2024-09-11T07:49:17.499 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1868e-05 2024-09-11T07:49:17.499 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.499 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.499 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:17.499 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:02.188526+0000", 2024-09-11T07:49:17.499 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7830999999999999e-05 2024-09-11T07:49:17.499 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.499 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.499 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:17.499 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:02.196632+0000", 2024-09-11T07:49:17.499 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0081067489999999999 2024-09-11T07:49:17.500 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.500 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.500 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:17.500 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:02.196667+0000", 2024-09-11T07:49:17.500 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4925999999999997e-05 2024-09-11T07:49:17.500 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.500 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:17.500 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.500 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.500 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.500 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25990.0:209545 11.a e461/412 11:5e0df399:::benchmark_data_smithi022_188626_object89737:head v 461'13257, mlcod=461'13254)", 2024-09-11T07:49:17.500 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:09.412707+0000", 2024-09-11T07:49:17.500 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 8.0083478509999999, 2024-09-11T07:49:17.501 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0093356299999999993, 2024-09-11T07:49:17.501 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:17.501 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:17.501 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:17.501 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.501 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:17.501 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.412707+0000", 2024-09-11T07:49:17.501 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.501 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.501 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.501 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:17.501 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.412707+0000", 2024-09-11T07:49:17.501 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.501 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.502 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.502 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:17.502 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.412706+0000", 2024-09-11T07:49:17.502 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:49:17.502 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.502 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.502 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:17.502 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.412715+0000", 2024-09-11T07:49:17.502 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8700000000000004e-06 2024-09-11T07:49:17.502 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.502 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.502 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:17.502 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.412726+0000", 2024-09-11T07:49:17.502 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0971999999999999e-05 2024-09-11T07:49:17.503 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.503 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.503 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:17.503 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.412732+0000", 2024-09-11T07:49:17.503 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.665e-06 2024-09-11T07:49:17.503 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.503 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.503 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:17.503 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.414962+0000", 2024-09-11T07:49:17.503 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002230311 2024-09-11T07:49:17.503 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.503 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.503 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:17.503 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.414982+0000", 2024-09-11T07:49:17.504 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0077000000000001e-05 2024-09-11T07:49:17.504 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.504 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.504 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:17.504 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.422007+0000", 2024-09-11T07:49:17.504 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0070242799999999999 2024-09-11T07:49:17.504 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.504 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.504 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:17.504 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.422042+0000", 2024-09-11T07:49:17.504 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5645999999999997e-05 2024-09-11T07:49:17.504 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.504 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:17.504 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.505 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.505 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.505 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25990.0:209798 11.c e461/456 11:30ebe109:::benchmark_data_smithi022_188626_object89990:head v 461'12908, mlcod=461'12905)", 2024-09-11T07:49:17.505 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:09.587780+0000", 2024-09-11T07:49:17.505 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 7.833274769, 2024-09-11T07:49:17.505 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0079268700000000008, 2024-09-11T07:49:17.505 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:17.505 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:17.505 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:17.505 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.505 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:17.505 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.587780+0000", 2024-09-11T07:49:17.505 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.505 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.506 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.506 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:17.506 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.587780+0000", 2024-09-11T07:49:17.506 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.506 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.506 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.506 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:17.506 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.587778+0000", 2024-09-11T07:49:17.506 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:49:17.506 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.506 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.506 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:17.506 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.587787+0000", 2024-09-11T07:49:17.506 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1240000000000005e-06 2024-09-11T07:49:17.507 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.507 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.507 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:17.507 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.587798+0000", 2024-09-11T07:49:17.507 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1657999999999999e-05 2024-09-11T07:49:17.507 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.507 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.507 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:17.507 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.587804+0000", 2024-09-11T07:49:17.507 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9020000000000004e-06 2024-09-11T07:49:17.507 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.507 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.507 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:17.507 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.587864+0000", 2024-09-11T07:49:17.508 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9658999999999999e-05 2024-09-11T07:49:17.508 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.508 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.508 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:17.508 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.587886+0000", 2024-09-11T07:49:17.508 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1801e-05 2024-09-11T07:49:17.508 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.508 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.508 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:17.508 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.595610+0000", 2024-09-11T07:49:17.508 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0077240950000000003 2024-09-11T07:49:17.508 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.508 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.508 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:17.509 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.595707+0000", 2024-09-11T07:49:17.509 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.6984000000000005e-05 2024-09-11T07:49:17.509 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.509 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:17.509 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.509 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.509 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.509 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25990.0:209808 11.a e461/412 11:57e6a670:::benchmark_data_smithi022_188626_object90000:head v 461'13276, mlcod=461'13274)", 2024-09-11T07:49:17.509 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:09.589606+0000", 2024-09-11T07:49:17.509 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 7.8314489790000001, 2024-09-11T07:49:17.509 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0078248359999999999, 2024-09-11T07:49:17.509 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:17.509 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:17.510 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:17.510 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.510 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:17.510 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.589606+0000", 2024-09-11T07:49:17.510 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.510 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.510 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.510 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:17.510 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.589606+0000", 2024-09-11T07:49:17.510 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.510 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.510 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.510 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:17.510 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.589604+0000", 2024-09-11T07:49:17.511 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:49:17.511 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.511 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.511 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:17.511 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.589615+0000", 2024-09-11T07:49:17.511 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0550999999999999e-05 2024-09-11T07:49:17.511 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.511 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.511 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:17.511 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.589630+0000", 2024-09-11T07:49:17.511 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5444999999999999e-05 2024-09-11T07:49:17.511 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.511 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.511 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:17.512 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.589637+0000", 2024-09-11T07:49:17.512 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.8499999999999996e-06 2024-09-11T07:49:17.512 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.512 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.512 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:17.512 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.589679+0000", 2024-09-11T07:49:17.512 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2024999999999998e-05 2024-09-11T07:49:17.512 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.512 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.512 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:17.512 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.589703+0000", 2024-09-11T07:49:17.512 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.4111e-05 2024-09-11T07:49:17.512 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.513 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.513 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:17.513 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.597389+0000", 2024-09-11T07:49:17.513 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0076861029999999997 2024-09-11T07:49:17.513 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.513 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.513 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:17.513 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.597430+0000", 2024-09-11T07:49:17.513 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1403000000000003e-05 2024-09-11T07:49:17.513 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.513 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:17.513 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.513 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.513 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.514 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25990.0:211016 11.3 e461/460 11:c248ee08:::benchmark_data_smithi022_188626_object91208:head v 461'13123, mlcod=461'13120)", 2024-09-11T07:49:17.514 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:10.505712+0000", 2024-09-11T07:49:17.514 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 6.9153430829999998, 2024-09-11T07:49:17.514 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.018404352999999998, 2024-09-11T07:49:17.514 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:17.514 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:17.514 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:17.514 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.514 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:17.514 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:10.505712+0000", 2024-09-11T07:49:17.514 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.514 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.514 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.515 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:17.515 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:10.505712+0000", 2024-09-11T07:49:17.515 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.515 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.515 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.515 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:17.515 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:10.505711+0000", 2024-09-11T07:49:17.515 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:49:17.515 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.515 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.515 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:17.515 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:10.517805+0000", 2024-09-11T07:49:17.515 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.012093497999999999 2024-09-11T07:49:17.515 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.515 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.516 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:17.516 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:10.517849+0000", 2024-09-11T07:49:17.516 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4833000000000003e-05 2024-09-11T07:49:17.516 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.516 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.516 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:17.516 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:10.517866+0000", 2024-09-11T07:49:17.516 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6209999999999999e-05 2024-09-11T07:49:17.516 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.516 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.516 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:17.516 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:10.518049+0000", 2024-09-11T07:49:17.516 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018378000000000001 2024-09-11T07:49:17.517 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.517 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.517 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:17.517 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:10.518068+0000", 2024-09-11T07:49:17.517 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.861e-05 2024-09-11T07:49:17.517 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.517 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.517 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:17.517 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:10.524086+0000", 2024-09-11T07:49:17.517 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0060181619999999996 2024-09-11T07:49:17.517 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.517 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.517 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:17.517 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:10.524116+0000", 2024-09-11T07:49:17.518 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9668000000000001e-05 2024-09-11T07:49:17.518 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.518 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:17.518 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.518 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.518 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.518 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25990.0:213326 11.a e462/412 11:5a597884:::benchmark_data_smithi022_188626_object93518:head v 462'13505, mlcod=462'13503)", 2024-09-11T07:49:17.518 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:12.339170+0000", 2024-09-11T07:49:17.518 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 5.0818849530000003, 2024-09-11T07:49:17.518 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0089150870000000004, 2024-09-11T07:49:17.518 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:17.518 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:17.518 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:17.519 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.519 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:17.519 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.339170+0000", 2024-09-11T07:49:17.519 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.519 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.519 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.519 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:17.519 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.339170+0000", 2024-09-11T07:49:17.519 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.519 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.519 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.519 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:17.519 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.339168+0000", 2024-09-11T07:49:17.519 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:49:17.519 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.520 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.520 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:17.520 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.339177+0000", 2024-09-11T07:49:17.520 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2300000000000008e-06 2024-09-11T07:49:17.520 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.520 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.520 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:17.520 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.339191+0000", 2024-09-11T07:49:17.520 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.454e-05 2024-09-11T07:49:17.520 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.520 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.520 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:17.520 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.339197+0000", 2024-09-11T07:49:17.521 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8030000000000002e-06 2024-09-11T07:49:17.521 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.521 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.521 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:17.521 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.339246+0000", 2024-09-11T07:49:17.521 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8819999999999997e-05 2024-09-11T07:49:17.521 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.521 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.521 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:17.521 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.339264+0000", 2024-09-11T07:49:17.521 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8020999999999999e-05 2024-09-11T07:49:17.521 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.521 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.521 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:17.522 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.348018+0000", 2024-09-11T07:49:17.522 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.008754207 2024-09-11T07:49:17.522 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.522 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.522 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:17.522 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.348085+0000", 2024-09-11T07:49:17.522 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6837999999999999e-05 2024-09-11T07:49:17.522 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.522 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:17.522 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.522 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.522 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.522 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25990.0:213752 11.a e462/412 11:5e279c3d:::benchmark_data_smithi022_188626_object93944:head v 462'13540, mlcod=462'13538)", 2024-09-11T07:49:17.522 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:12.641672+0000", 2024-09-11T07:49:17.523 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 4.7793824090000001, 2024-09-11T07:49:17.523 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.007924113, 2024-09-11T07:49:17.523 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:17.523 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:17.523 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:17.523 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.523 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:17.523 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.641672+0000", 2024-09-11T07:49:17.523 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.523 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.523 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.523 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:17.523 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.641672+0000", 2024-09-11T07:49:17.523 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.524 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.524 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:17.524 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.641670+0000", 2024-09-11T07:49:17.524 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:49:17.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.524 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.524 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:17.524 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.641684+0000", 2024-09-11T07:49:17.524 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3373999999999999e-05 2024-09-11T07:49:17.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.524 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.524 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:17.524 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.641700+0000", 2024-09-11T07:49:17.525 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6399e-05 2024-09-11T07:49:17.525 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.525 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.525 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:17.525 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.641708+0000", 2024-09-11T07:49:17.525 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7060000000000004e-06 2024-09-11T07:49:17.525 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.525 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.525 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:17.525 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.641780+0000", 2024-09-11T07:49:17.525 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1866999999999995e-05 2024-09-11T07:49:17.525 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.525 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.525 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:17.526 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.641799+0000", 2024-09-11T07:49:17.526 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9656999999999998e-05 2024-09-11T07:49:17.526 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.526 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.526 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:17.526 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.649566+0000", 2024-09-11T07:49:17.526 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0077667439999999999 2024-09-11T07:49:17.526 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.526 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.526 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:17.526 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.649596+0000", 2024-09-11T07:49:17.526 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.0275000000000001e-05 2024-09-11T07:49:17.526 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.527 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:17.527 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.527 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.527 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.527 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25990.0:213753 11.a e462/412 11:53b426d3:::benchmark_data_smithi022_188626_object93945:head v 462'13541, mlcod=462'13538)", 2024-09-11T07:49:17.527 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:12.644596+0000", 2024-09-11T07:49:17.527 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 4.7764587570000003, 2024-09-11T07:49:17.527 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0086852509999999997, 2024-09-11T07:49:17.527 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:17.527 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:17.527 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:17.527 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.527 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:17.527 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.644596+0000", 2024-09-11T07:49:17.528 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.528 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.528 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.528 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:17.528 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.644596+0000", 2024-09-11T07:49:17.528 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:17.528 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.528 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.528 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:17.528 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.644595+0000", 2024-09-11T07:49:17.528 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:49:17.528 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.528 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.528 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:17.529 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.644603+0000", 2024-09-11T07:49:17.529 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2269999999999993e-06 2024-09-11T07:49:17.529 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.529 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.529 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:17.529 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.644611+0000", 2024-09-11T07:49:17.529 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1559999999999995e-06 2024-09-11T07:49:17.529 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.529 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.529 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:17.529 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.644616+0000", 2024-09-11T07:49:17.529 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.293e-06 2024-09-11T07:49:17.529 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.529 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.529 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:17.530 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.646603+0000", 2024-09-11T07:49:17.530 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0019864520000000001 2024-09-11T07:49:17.530 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.530 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.530 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:17.530 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.646622+0000", 2024-09-11T07:49:17.530 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9154e-05 2024-09-11T07:49:17.530 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.530 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.530 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:17.530 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.653215+0000", 2024-09-11T07:49:17.530 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0065931460000000002 2024-09-11T07:49:17.530 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:17.530 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:17.530 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:17.531 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.653281+0000", 2024-09-11T07:49:17.531 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5884999999999997e-05 2024-09-11T07:49:17.531 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.531 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:17.531 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.531 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:17.531 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:17.531 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:49:17.531 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:17.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:17.575 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:17.575 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:17.676 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:17.704 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:17.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:17.705 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:17.806 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:17.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:17.843 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:17.843 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:17.944 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:17.983 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:17.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:17.984 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:18.085 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:18.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:18.123 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:18.123 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:18.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:17 smithi022 ceph-mon[29462]: 6.2 scrub starts 2024-09-11T07:49:18.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:17 smithi022 ceph-mon[29462]: 6.2 scrub ok 2024-09-11T07:49:18.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:17 smithi022 ceph-mon[29462]: 5.f scrub starts 2024-09-11T07:49:18.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:17 smithi022 ceph-mon[29462]: 5.f scrub ok 2024-09-11T07:49:18.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:17 smithi022 ceph-mon[37330]: 6.2 scrub starts 2024-09-11T07:49:18.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:17 smithi022 ceph-mon[37330]: 6.2 scrub ok 2024-09-11T07:49:18.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:17 smithi022 ceph-mon[37330]: 5.f scrub starts 2024-09-11T07:49:18.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:17 smithi022 ceph-mon[37330]: 5.f scrub ok 2024-09-11T07:49:18.224 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:18.262 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:18.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:18.263 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:18.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:17 smithi104 ceph-mon[33507]: 6.2 scrub starts 2024-09-11T07:49:18.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:17 smithi104 ceph-mon[33507]: 6.2 scrub ok 2024-09-11T07:49:18.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:17 smithi104 ceph-mon[33507]: 5.f scrub starts 2024-09-11T07:49:18.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:17 smithi104 ceph-mon[33507]: 5.f scrub ok 2024-09-11T07:49:18.364 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:18.395 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:18.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:18.396 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:18.497 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:18.525 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:18.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:18.526 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:18.627 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:18.651 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:18.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:18.652 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:18.753 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:18.775 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:18.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:18.776 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:18.877 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:18.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:18.906 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:18.906 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:19.006 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:19.034 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:19.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:19.035 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:19.136 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:19.164 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:19.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:19.165 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:19.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:18 smithi022 ceph-mon[29462]: 4.17 scrub starts 2024-09-11T07:49:19.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:18 smithi022 ceph-mon[29462]: 4.17 scrub ok 2024-09-11T07:49:19.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:18 smithi022 ceph-mon[29462]: 5.1f scrub starts 2024-09-11T07:49:19.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:18 smithi022 ceph-mon[29462]: 5.1f scrub ok 2024-09-11T07:49:19.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:18 smithi022 ceph-mon[29462]: pgmap v1058: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 1.4 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.00k op/s; 5838/70869 objects misplaced (8.238%); 7.9 MiB/s, 126 objects/s recovering 2024-09-11T07:49:19.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:18 smithi022 ceph-mon[37330]: 4.17 scrub starts 2024-09-11T07:49:19.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:18 smithi022 ceph-mon[37330]: 4.17 scrub ok 2024-09-11T07:49:19.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:18 smithi022 ceph-mon[37330]: 5.1f scrub starts 2024-09-11T07:49:19.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:18 smithi022 ceph-mon[37330]: 5.1f scrub ok 2024-09-11T07:49:19.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:18 smithi022 ceph-mon[37330]: pgmap v1058: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 1.4 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.00k op/s; 5838/70869 objects misplaced (8.238%); 7.9 MiB/s, 126 objects/s recovering 2024-09-11T07:49:19.266 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:19.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:18 smithi104 ceph-mon[33507]: 4.17 scrub starts 2024-09-11T07:49:19.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:18 smithi104 ceph-mon[33507]: 4.17 scrub ok 2024-09-11T07:49:19.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:18 smithi104 ceph-mon[33507]: 5.1f scrub starts 2024-09-11T07:49:19.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:18 smithi104 ceph-mon[33507]: 5.1f scrub ok 2024-09-11T07:49:19.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:18 smithi104 ceph-mon[33507]: pgmap v1058: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 1.4 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.00k op/s; 5838/70869 objects misplaced (8.238%); 7.9 MiB/s, 126 objects/s recovering 2024-09-11T07:49:19.292 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:19.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:19.294 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:19.395 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:19.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:19.424 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:19.424 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:19.525 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:19.564 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:19.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:19.565 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:19.666 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:19.696 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:19.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:19.697 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:19.764 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-11T07:49:19.798 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:19.840 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:19.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:19.841 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:19.942 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:19.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:19.982 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:19.982 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:20.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:19 smithi104 ceph-mon[33507]: 6.1 scrub starts 2024-09-11T07:49:20.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:19 smithi104 ceph-mon[33507]: 6.1 scrub ok 2024-09-11T07:49:20.083 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:20.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:20.117 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:20.117 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:20.128 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:49:20.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:19 smithi022 ceph-mon[29462]: 6.1 scrub starts 2024-09-11T07:49:20.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:19 smithi022 ceph-mon[29462]: 6.1 scrub ok 2024-09-11T07:49:20.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:19 smithi022 ceph-mon[37330]: 6.1 scrub starts 2024-09-11T07:49:20.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:19 smithi022 ceph-mon[37330]: 6.1 scrub ok 2024-09-11T07:49:20.218 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:20.248 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:20.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:20.249 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:20.350 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:20.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:20.382 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:20.382 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:20.483 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:20.511 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:20.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:20.512 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:20.525 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:49:20 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:49:20.613 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:20.654 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:20.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:20.655 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:20.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:49:20 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:49:20] "GET /metrics HTTP/1.1" 200 225248 "" "Prometheus/2.43.0" 2024-09-11T07:49:20.756 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:20.797 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:20.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:20.798 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:20.900 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:20.927 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:20.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:20.928 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:21.029 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:21.056 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:21.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:21.058 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:21.116 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:20 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:49:21.116 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:20 smithi022 ceph-mon[29462]: pgmap v1059: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 1.2 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.14k op/s; 4329/58290 objects misplaced (7.427%); 1.6 MiB/s, 25 objects/s recovering 2024-09-11T07:49:21.116 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:20 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:49:21.116 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:20 smithi022 ceph-mon[37330]: pgmap v1059: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 1.2 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.14k op/s; 4329/58290 objects misplaced (7.427%); 1.6 MiB/s, 25 objects/s recovering 2024-09-11T07:49:21.158 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:21.207 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:21.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:21.209 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:21.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:20 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:49:21.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:20 smithi104 ceph-mon[33507]: pgmap v1059: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 1.2 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.14k op/s; 4329/58290 objects misplaced (7.427%); 1.6 MiB/s, 25 objects/s recovering 2024-09-11T07:49:21.310 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:21.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:21.350 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:21.350 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:21.451 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:21.495 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:21.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:21.496 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:21.513 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:49:21.523 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:49:21.523 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:49:21.556 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:49:21.557 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:49:21.590 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:49:21.591 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:49:21.597 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:21.622 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:49:21.622 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:49:21.642 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:21.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:21.643 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:21.653 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:49:21.653 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:49:21.683 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:49:21.683 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:49:21.715 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:49:21.715 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:49:21.744 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:21.745 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:49:21.745 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:49:21.772 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:21.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:21.773 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:21.874 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:21.905 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:21.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:21.906 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:21.943 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:49:21.943 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:49:22.008 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:22.047 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:22.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:22.049 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:22.092 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:21 smithi022 ceph-mon[37330]: 4.c scrub starts 2024-09-11T07:49:22.092 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:21 smithi022 ceph-mon[37330]: 4.c scrub ok 2024-09-11T07:49:22.093 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:21 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/811911033' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:49:22.093 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:21 smithi022 ceph-mon[29462]: 4.c scrub starts 2024-09-11T07:49:22.093 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:21 smithi022 ceph-mon[29462]: 4.c scrub ok 2024-09-11T07:49:22.093 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:21 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/811911033' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:49:22.151 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:22.191 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:22.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:22.192 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:22.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:21 smithi104 ceph-mon[33507]: 4.c scrub starts 2024-09-11T07:49:22.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:21 smithi104 ceph-mon[33507]: 4.c scrub ok 2024-09-11T07:49:22.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:21 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/811911033' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:49:22.293 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:22.331 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:22.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:22.332 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:22.433 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:22.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:22.456 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:22.456 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:22.557 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:22.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:22.598 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:22.599 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:22.659 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-11T07:49:22.700 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:22.740 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:22.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:22.742 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:22.842 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:22.870 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:22.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:22.871 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:22.875 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T07:49:22.972 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:23.002 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:23.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:23.003 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:23.009 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:49:23.104 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:23.155 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:23.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:23.156 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:23.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:22 smithi022 ceph-mon[29462]: 6.1a scrub starts 2024-09-11T07:49:23.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:22 smithi022 ceph-mon[29462]: 6.1a scrub ok 2024-09-11T07:49:23.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:22 smithi022 ceph-mon[29462]: pgmap v1060: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 1.1 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 965 B/s rd, 0 B/s wr, 1.15k op/s; 4329/52353 objects misplaced (8.269%) 2024-09-11T07:49:23.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:22 smithi022 ceph-mon[37330]: 6.1a scrub starts 2024-09-11T07:49:23.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:22 smithi022 ceph-mon[37330]: 6.1a scrub ok 2024-09-11T07:49:23.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:22 smithi022 ceph-mon[37330]: pgmap v1060: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 1.1 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 965 B/s rd, 0 B/s wr, 1.15k op/s; 4329/52353 objects misplaced (8.269%) 2024-09-11T07:49:23.183 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:22 smithi104 ceph-mon[33507]: 6.1a scrub starts 2024-09-11T07:49:23.183 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:22 smithi104 ceph-mon[33507]: 6.1a scrub ok 2024-09-11T07:49:23.184 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:22 smithi104 ceph-mon[33507]: pgmap v1060: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 1.1 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 965 B/s rd, 0 B/s wr, 1.15k op/s; 4329/52353 objects misplaced (8.269%) 2024-09-11T07:49:23.257 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:23.298 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:23.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:23.299 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:23.400 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:23.439 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:23.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:23.440 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:23.541 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:23.569 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:23.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:23.570 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:23.672 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:23.710 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:23.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:23.711 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:23.812 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:23.838 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:23.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:23.839 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:23.940 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:23.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:23.969 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:23.969 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:24.070 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:24.094 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:24.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:24.095 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:24.196 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:24.219 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:23 smithi022 ceph-mon[29462]: 5.18 scrub starts 2024-09-11T07:49:24.219 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:23 smithi022 ceph-mon[29462]: 5.18 scrub ok 2024-09-11T07:49:24.219 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:49:24.220 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:23 smithi022 ceph-mon[37330]: 5.18 scrub starts 2024-09-11T07:49:24.220 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:23 smithi022 ceph-mon[37330]: 5.18 scrub ok 2024-09-11T07:49:24.220 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:49:24.226 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:24.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:24.227 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:24.279 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:49:24 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:49:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:49:24.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:23 smithi104 ceph-mon[33507]: 5.18 scrub starts 2024-09-11T07:49:24.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:23 smithi104 ceph-mon[33507]: 5.18 scrub ok 2024-09-11T07:49:24.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:49:24.327 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:24.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:24.356 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:24.356 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:24.457 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:24.500 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:24.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:24.501 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:24.602 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:24.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:24.648 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:24.648 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:24.720 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:49:24.748 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:24.774 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:24.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:24.775 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:24.876 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:24.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:24.924 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:24.925 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:25.025 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:25.053 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:25.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:25.055 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:25.126 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:24 smithi022 ceph-mon[37330]: pgmap v1061: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 1.0 GiB data, 7.8 GiB used, 529 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.05k op/s; 4329/51096 objects misplaced (8.472%) 2024-09-11T07:49:25.127 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:24 smithi022 ceph-mon[29462]: pgmap v1061: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 1.0 GiB data, 7.8 GiB used, 529 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.05k op/s; 4329/51096 objects misplaced (8.472%) 2024-09-11T07:49:25.155 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:25.182 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:25.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:25.183 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:25.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:24 smithi104 ceph-mon[33507]: pgmap v1061: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 1.0 GiB data, 7.8 GiB used, 529 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.05k op/s; 4329/51096 objects misplaced (8.472%) 2024-09-11T07:49:25.284 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:25.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:25.319 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:25.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:25 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-a[29458]: 2024-09-11T07:49:25.123+0000 7fde5d00f640 -1 mon.a@0(leader).osd e462 definitely_dead 0 2024-09-11T07:49:25.420 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:25.458 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:25.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:25.459 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:25.560 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:25.587 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:49:25.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:25.591 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:25.591 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:25.692 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:25.720 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:25.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:25.721 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:25.822 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:25.853 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:25.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:25.854 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:25.940 INFO:teuthology.orchestra.run.smithi022.stderr:marked out osd.7. 2024-09-11T07:49:25.955 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:25.979 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:25.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:25.980 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:26.081 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:26.104 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:26.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:26.105 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:26.206 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:26.228 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:26.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:26.229 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:26.264 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:25 smithi104 ceph-mon[33507]: 4.12 scrub starts 2024-09-11T07:49:26.264 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:25 smithi104 ceph-mon[33507]: 4.12 scrub ok 2024-09-11T07:49:26.264 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:25 smithi104 ceph-mon[33507]: 6.1e scrub starts 2024-09-11T07:49:26.264 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:25 smithi104 ceph-mon[33507]: 6.1e scrub ok 2024-09-11T07:49:26.264 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:25 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2128182864' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["7"]}]: dispatch 2024-09-11T07:49:26.264 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:25 smithi104 ceph-mon[33507]: Client client.admin marked osd.7 out, while it was still marked up 2024-09-11T07:49:26.330 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:26.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:26.353 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:26.353 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:26.371 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:49:26.371 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:49:26.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:25 smithi022 ceph-mon[29462]: 4.12 scrub starts 2024-09-11T07:49:26.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:25 smithi022 ceph-mon[29462]: 4.12 scrub ok 2024-09-11T07:49:26.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:25 smithi022 ceph-mon[29462]: 6.1e scrub starts 2024-09-11T07:49:26.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:25 smithi022 ceph-mon[29462]: 6.1e scrub ok 2024-09-11T07:49:26.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:25 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2128182864' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["7"]}]: dispatch 2024-09-11T07:49:26.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:25 smithi022 ceph-mon[29462]: Client client.admin marked osd.7 out, while it was still marked up 2024-09-11T07:49:26.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:25 smithi022 ceph-mon[37330]: 4.12 scrub starts 2024-09-11T07:49:26.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:25 smithi022 ceph-mon[37330]: 4.12 scrub ok 2024-09-11T07:49:26.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:25 smithi022 ceph-mon[37330]: 6.1e scrub starts 2024-09-11T07:49:26.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:25 smithi022 ceph-mon[37330]: 6.1e scrub ok 2024-09-11T07:49:26.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:25 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2128182864' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["7"]}]: dispatch 2024-09-11T07:49:26.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:25 smithi022 ceph-mon[37330]: Client client.admin marked osd.7 out, while it was still marked up 2024-09-11T07:49:26.454 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:26.478 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:26.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:26.479 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:26.580 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:26.605 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:26.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:26.606 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:26.707 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:26.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:26.730 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:26.730 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:26.831 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:26.858 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:26.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:26.859 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:26.960 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:26.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:26.988 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:26.988 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:27.049 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-11T07:49:27.089 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:27.114 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:27.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:27.115 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:27.216 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:27.228 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:26 smithi022 ceph-mon[37330]: 11.c scrub starts 2024-09-11T07:49:27.228 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:26 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2128182864' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["7"]}]': finished 2024-09-11T07:49:27.228 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:26 smithi022 ceph-mon[37330]: osdmap e463: 8 total, 8 up, 5 in 2024-09-11T07:49:27.228 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:26 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:49:27.228 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:26 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:49:27.228 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:26 smithi022 ceph-mon[37330]: pgmap v1063: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 685 MiB data, 6.0 GiB used, 441 GiB / 447 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s; 2725/33438 objects misplaced (8.149%) 2024-09-11T07:49:27.228 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:26 smithi022 ceph-mon[37330]: osdmap e464: 8 total, 8 up, 5 in 2024-09-11T07:49:27.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:26 smithi022 ceph-mon[29462]: 11.c scrub starts 2024-09-11T07:49:27.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:26 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2128182864' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["7"]}]': finished 2024-09-11T07:49:27.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:26 smithi022 ceph-mon[29462]: osdmap e463: 8 total, 8 up, 5 in 2024-09-11T07:49:27.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:26 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:49:27.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:26 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:49:27.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:26 smithi022 ceph-mon[29462]: pgmap v1063: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 685 MiB data, 6.0 GiB used, 441 GiB / 447 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s; 2725/33438 objects misplaced (8.149%) 2024-09-11T07:49:27.229 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:26 smithi022 ceph-mon[29462]: osdmap e464: 8 total, 8 up, 5 in 2024-09-11T07:49:27.240 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:27.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:27.241 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:27.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:26 smithi104 ceph-mon[33507]: 11.c scrub starts 2024-09-11T07:49:27.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:26 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2128182864' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["7"]}]': finished 2024-09-11T07:49:27.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:26 smithi104 ceph-mon[33507]: osdmap e463: 8 total, 8 up, 5 in 2024-09-11T07:49:27.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:26 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:49:27.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:26 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:49:27.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:26 smithi104 ceph-mon[33507]: pgmap v1063: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 685 MiB data, 6.0 GiB used, 441 GiB / 447 GiB avail; 1023 B/s rd, 0 B/s wr, 1.25k op/s; 2725/33438 objects misplaced (8.149%) 2024-09-11T07:49:27.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:26 smithi104 ceph-mon[33507]: osdmap e464: 8 total, 8 up, 5 in 2024-09-11T07:49:27.310 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:49:27.342 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:27.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:27.365 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:27.366 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:27.467 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:27.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:27.494 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:27.495 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:27.596 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:27.618 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:27.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:27.619 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:27.720 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:27.742 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:27.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:27.743 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:27.844 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:27.868 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:27.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:27.869 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:27.970 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:27.996 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:27.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:27.998 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:28.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:27 smithi104 ceph-mon[33507]: 11.4 continuing backfill to osd.1 from (445'9682,462'14643] MIN to 462'14643 2024-09-11T07:49:28.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:27 smithi104 ceph-mon[33507]: 11.4 starting backfill to osd.3 from (0'0,0'0] MAX to 462'14643 2024-09-11T07:49:28.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:27 smithi104 ceph-mon[33507]: 11.f continuing backfill to osd.0 from (444'9281,462'14221] MIN to 462'14221 2024-09-11T07:49:28.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:27 smithi104 ceph-mon[33507]: 11.f starting backfill to osd.3 from (0'0,0'0] MAX to 462'14221 2024-09-11T07:49:28.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:27 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:49:28.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:27 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:49:28.098 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:28.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:28.122 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:28.122 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:28.218 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:27 smithi022 ceph-mon[29462]: 11.4 continuing backfill to osd.1 from (445'9682,462'14643] MIN to 462'14643 2024-09-11T07:49:28.218 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:27 smithi022 ceph-mon[29462]: 11.4 starting backfill to osd.3 from (0'0,0'0] MAX to 462'14643 2024-09-11T07:49:28.218 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:27 smithi022 ceph-mon[29462]: 11.f continuing backfill to osd.0 from (444'9281,462'14221] MIN to 462'14221 2024-09-11T07:49:28.218 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:27 smithi022 ceph-mon[29462]: 11.f starting backfill to osd.3 from (0'0,0'0] MAX to 462'14221 2024-09-11T07:49:28.218 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:27 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:49:28.218 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:27 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:49:28.219 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:27 smithi022 ceph-mon[37330]: 11.4 continuing backfill to osd.1 from (445'9682,462'14643] MIN to 462'14643 2024-09-11T07:49:28.219 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:27 smithi022 ceph-mon[37330]: 11.4 starting backfill to osd.3 from (0'0,0'0] MAX to 462'14643 2024-09-11T07:49:28.219 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:27 smithi022 ceph-mon[37330]: 11.f continuing backfill to osd.0 from (444'9281,462'14221] MIN to 462'14221 2024-09-11T07:49:28.219 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:27 smithi022 ceph-mon[37330]: 11.f starting backfill to osd.3 from (0'0,0'0] MAX to 462'14221 2024-09-11T07:49:28.219 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:27 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:49:28.219 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:27 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:49:28.224 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:28.250 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:28.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:28.251 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:28.352 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:28.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:28.416 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:28.416 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:28.517 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:28.546 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:28.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:28.547 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:28.648 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:28.677 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:28.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:28.678 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:28.779 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:28.807 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:28.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:28.808 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:28.834 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:49:28.909 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:28.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:28.948 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:28.949 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:28.977 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is unset 2024-09-11T07:49:28.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:28 smithi022 ceph-mon[29462]: 11.0 starting backfill to osd.6 from (0'0,0'0] MAX to 462'14365 2024-09-11T07:49:28.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:28 smithi022 ceph-mon[29462]: 11.e starting backfill to osd.3 from (0'0,0'0] MAX to 462'14508 2024-09-11T07:49:28.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:28 smithi022 ceph-mon[29462]: osdmap e465: 8 total, 8 up, 5 in 2024-09-11T07:49:28.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:28 smithi022 ceph-mon[29462]: 11.2 starting backfill to osd.6 from (0'0,0'0] MAX to 462'14376 2024-09-11T07:49:28.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:28 smithi022 ceph-mon[29462]: 11.c starting backfill to osd.6 from (0'0,0'0] MAX to 462'14142 2024-09-11T07:49:28.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:28 smithi022 ceph-mon[29462]: 11.7 starting backfill to osd.6 from (0'0,0'0] MAX to 462'14253 2024-09-11T07:49:28.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:28 smithi022 ceph-mon[29462]: 2.1c scrub starts 2024-09-11T07:49:28.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:28 smithi022 ceph-mon[29462]: pgmap v1066: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 685 MiB data, 6.0 GiB used, 441 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 1.05k op/s; 2725/33438 objects misplaced (8.149%) 2024-09-11T07:49:28.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:28 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/660691365' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:49:28.998 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:28 smithi022 ceph-mon[37330]: 11.0 starting backfill to osd.6 from (0'0,0'0] MAX to 462'14365 2024-09-11T07:49:28.999 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:28 smithi022 ceph-mon[37330]: 11.e starting backfill to osd.3 from (0'0,0'0] MAX to 462'14508 2024-09-11T07:49:28.999 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:28 smithi022 ceph-mon[37330]: osdmap e465: 8 total, 8 up, 5 in 2024-09-11T07:49:28.999 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:28 smithi022 ceph-mon[37330]: 11.2 starting backfill to osd.6 from (0'0,0'0] MAX to 462'14376 2024-09-11T07:49:28.999 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:28 smithi022 ceph-mon[37330]: 11.c starting backfill to osd.6 from (0'0,0'0] MAX to 462'14142 2024-09-11T07:49:28.999 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:28 smithi022 ceph-mon[37330]: 11.7 starting backfill to osd.6 from (0'0,0'0] MAX to 462'14253 2024-09-11T07:49:28.999 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:28 smithi022 ceph-mon[37330]: 2.1c scrub starts 2024-09-11T07:49:28.999 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:28 smithi022 ceph-mon[37330]: pgmap v1066: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 685 MiB data, 6.0 GiB used, 441 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 1.05k op/s; 2725/33438 objects misplaced (8.149%) 2024-09-11T07:49:28.999 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:28 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/660691365' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:49:29.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:28 smithi104 ceph-mon[33507]: 11.0 starting backfill to osd.6 from (0'0,0'0] MAX to 462'14365 2024-09-11T07:49:29.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:28 smithi104 ceph-mon[33507]: 11.e starting backfill to osd.3 from (0'0,0'0] MAX to 462'14508 2024-09-11T07:49:29.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:28 smithi104 ceph-mon[33507]: osdmap e465: 8 total, 8 up, 5 in 2024-09-11T07:49:29.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:28 smithi104 ceph-mon[33507]: 11.2 starting backfill to osd.6 from (0'0,0'0] MAX to 462'14376 2024-09-11T07:49:29.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:28 smithi104 ceph-mon[33507]: 11.c starting backfill to osd.6 from (0'0,0'0] MAX to 462'14142 2024-09-11T07:49:29.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:28 smithi104 ceph-mon[33507]: 11.7 starting backfill to osd.6 from (0'0,0'0] MAX to 462'14253 2024-09-11T07:49:29.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:28 smithi104 ceph-mon[33507]: 2.1c scrub starts 2024-09-11T07:49:29.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:28 smithi104 ceph-mon[33507]: pgmap v1066: 158 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_toofull, 154 active+clean; 685 MiB data, 6.0 GiB used, 441 GiB / 447 GiB avail; 852 B/s rd, 0 B/s wr, 1.05k op/s; 2725/33438 objects misplaced (8.149%) 2024-09-11T07:49:29.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:28 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/660691365' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:49:29.049 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:29.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:29.099 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:29.200 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:29.242 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:29.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:29.243 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:29.343 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:29.389 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:29.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:29.390 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:29.491 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:29.518 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:29.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:29.519 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:29.620 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:29.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:29.656 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:29.656 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:29.671 INFO:teuthology.orchestra.run.smithi104.stdout:ERROR: (22) Invalid argument 2024-09-11T07:49:29.671 INFO:teuthology.orchestra.run.smithi104.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:49:29.757 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:29.795 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:29.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:29.796 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:29.897 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:29.941 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:29.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:29.942 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:30.043 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:30.081 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:30.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:30.082 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:30.183 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:30.212 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:30.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:30.213 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:30.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:29 smithi104 ceph-mon[33507]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:49:30.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:29 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/660691365' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:49:30.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:29 smithi104 ceph-mon[33507]: osdmap e466: 8 total, 8 up, 5 in 2024-09-11T07:49:30.315 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:30.352 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:30.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:30.353 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:30.374 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:29 smithi022 ceph-mon[29462]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:49:30.374 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:29 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/660691365' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:49:30.374 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:29 smithi022 ceph-mon[29462]: osdmap e466: 8 total, 8 up, 5 in 2024-09-11T07:49:30.374 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:29 smithi022 ceph-mon[37330]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:49:30.375 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:29 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/660691365' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:49:30.375 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:29 smithi022 ceph-mon[37330]: osdmap e466: 8 total, 8 up, 5 in 2024-09-11T07:49:30.413 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-11T07:49:30.454 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:30.500 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:30.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:30.501 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:30.603 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:30.638 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:30.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:30.639 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:30.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:49:30 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:49:30] "GET /metrics HTTP/1.1" 200 225170 "" "Prometheus/2.43.0" 2024-09-11T07:49:30.740 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:30.757 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:49:30.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:49:30 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:49:30.780 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:30.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:30.781 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:30.882 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:30.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:30.924 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:30.924 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:31.025 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:31.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:31.075 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:31.076 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:31.155 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:30 smithi104 ceph-mon[33507]: 2.1c scrub starts 2024-09-11T07:49:31.155 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:30 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:49:31.155 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:30 smithi104 ceph-mon[33507]: pgmap v1068: 158 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+clean+remapped, 3 active+remapped+backfill_toofull, 148 active+clean; 563 MiB data, 5.6 GiB used, 441 GiB / 447 GiB avail; 903 B/s rd, 0 B/s wr, 429 op/s; 4532/27612 objects misplaced (16.413%); 8.7 MiB/s, 146 objects/s recovering 2024-09-11T07:49:31.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:30 smithi022 ceph-mon[29462]: 2.1c scrub starts 2024-09-11T07:49:31.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:30 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:49:31.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:30 smithi022 ceph-mon[29462]: pgmap v1068: 158 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+clean+remapped, 3 active+remapped+backfill_toofull, 148 active+clean; 563 MiB data, 5.6 GiB used, 441 GiB / 447 GiB avail; 903 B/s rd, 0 B/s wr, 429 op/s; 4532/27612 objects misplaced (16.413%); 8.7 MiB/s, 146 objects/s recovering 2024-09-11T07:49:31.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:30 smithi022 ceph-mon[37330]: 2.1c scrub starts 2024-09-11T07:49:31.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:30 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:49:31.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:30 smithi022 ceph-mon[37330]: pgmap v1068: 158 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+clean+remapped, 3 active+remapped+backfill_toofull, 148 active+clean; 563 MiB data, 5.6 GiB used, 441 GiB / 447 GiB avail; 903 B/s rd, 0 B/s wr, 429 op/s; 4532/27612 objects misplaced (16.413%); 8.7 MiB/s, 146 objects/s recovering 2024-09-11T07:49:31.176 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:31.214 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:31.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:31.215 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:31.316 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:31.349 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:31.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:31.350 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:31.451 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:31.478 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:31.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:31.479 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:31.579 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:31.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:31.628 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:31.729 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:31.752 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:31.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:31.754 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:31.855 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:31.875 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:31.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:31.876 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:31.977 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:32.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:32.004 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:32.004 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:32.028 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:49:32.028 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:49:32.028 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:49:32.028 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:49:32.080 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2: {} 2024-09-11T07:49:32.081 INFO:teuthology.orchestra.run.smithi022.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T07:49:32.097 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:49:32.098 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:49:32.105 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:32.115 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:49:32.115 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:49:32.130 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:32.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:32.131 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:32.135 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:49:32.135 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:49:32.154 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:49:32.155 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:49:32.173 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:49:32.174 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:49:32.231 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:32.253 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:32.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:32.254 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:32.267 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:31 smithi104 ceph-mon[33507]: 2.1c scrub starts 2024-09-11T07:49:32.267 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:31 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/600333548' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:49:32.355 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:32.377 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:32.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:32.378 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:32.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:31 smithi022 ceph-mon[29462]: 2.1c scrub starts 2024-09-11T07:49:32.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:31 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/600333548' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:49:32.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:31 smithi022 ceph-mon[37330]: 2.1c scrub starts 2024-09-11T07:49:32.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:31 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/600333548' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:49:32.479 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:32.501 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:32.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:32.502 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:32.533 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:49:32.533 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:49:32.533 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 0 2024-09-11T07:49:32.534 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:49:32.538 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-11T07:49:32.603 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:32.625 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:32.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:32.626 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:32.727 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:32.749 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:32.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:32.750 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:32.851 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:32.872 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:32.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:32.873 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:32.974 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:32.998 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:32.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:32.999 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:33.100 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:33.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:33.131 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:33.131 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:33.232 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2 2024-09-11T07:49:33.234 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:32 smithi104 ceph-mon[33507]: 2.1c scrub starts 2024-09-11T07:49:33.235 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:32 smithi104 ceph-mon[33507]: osdmap e467: 8 total, 8 up, 5 in 2024-09-11T07:49:33.235 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:32 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:49:33.235 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:32 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:49:33.235 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:32 smithi104 ceph-mon[33507]: pgmap v1070: 158 pgs: 1 active+remapped+backfill_wait, 4 active+remapped+backfilling, 3 active+clean+remapped, 3 active+remapped+backfill_toofull, 147 active+clean; 504 MiB data, 5.6 GiB used, 441 GiB / 447 GiB avail; 4.3 KiB/s rd, 27 KiB/s wr, 526 op/s; 4725/24768 objects misplaced (19.077%); 11 MiB/s, 182 objects/s recovering 2024-09-11T07:49:33.265 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-11T07:49:33.266 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:33.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:33.268 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.2' 2024-09-11T07:49:33.369 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:33.391 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:33.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:33.392 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:33.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:32 smithi022 ceph-mon[29462]: 2.1c scrub starts 2024-09-11T07:49:33.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:32 smithi022 ceph-mon[29462]: osdmap e467: 8 total, 8 up, 5 in 2024-09-11T07:49:33.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:32 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:49:33.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:32 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:49:33.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:32 smithi022 ceph-mon[29462]: pgmap v1070: 158 pgs: 1 active+remapped+backfill_wait, 4 active+remapped+backfilling, 3 active+clean+remapped, 3 active+remapped+backfill_toofull, 147 active+clean; 504 MiB data, 5.6 GiB used, 441 GiB / 447 GiB avail; 4.3 KiB/s rd, 27 KiB/s wr, 526 op/s; 4725/24768 objects misplaced (19.077%); 11 MiB/s, 182 objects/s recovering 2024-09-11T07:49:33.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:32 smithi022 ceph-mon[37330]: 2.1c scrub starts 2024-09-11T07:49:33.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:32 smithi022 ceph-mon[37330]: osdmap e467: 8 total, 8 up, 5 in 2024-09-11T07:49:33.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:32 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:49:33.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:32 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:49:33.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:32 smithi022 ceph-mon[37330]: pgmap v1070: 158 pgs: 1 active+remapped+backfill_wait, 4 active+remapped+backfilling, 3 active+clean+remapped, 3 active+remapped+backfill_toofull, 147 active+clean; 504 MiB data, 5.6 GiB used, 441 GiB / 447 GiB avail; 4.3 KiB/s rd, 27 KiB/s wr, 526 op/s; 4725/24768 objects misplaced (19.077%); 11 MiB/s, 182 objects/s recovering 2024-09-11T07:49:33.493 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:33.515 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:33.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:33.516 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:33.522 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:49:33.617 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:33.638 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:33.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:33.639 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:33.740 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:33.762 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:33.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:33.763 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:33.784 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 5, 6] out_osds: [4, 2, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:49:33.784 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T07:49:33.784 INFO:tasks.thrashosds.thrasher:Killing osd 2, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-11T07:49:33.785 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-11T07:49:33.785 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.2 2024-09-11T07:49:33.863 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:33.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:33.885 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:33.885 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:33.986 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:33.996 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:33 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:49:33.996 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:33 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:49:33.996 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:33 smithi104 ceph-mon[33507]: osdmap e468: 8 total, 8 up, 5 in 2024-09-11T07:49:34.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:34.011 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:34.011 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:34.112 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:34.138 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:34.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:34.139 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:34.166 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:49:33 smithi022 systemd[1]: Stopping Ceph osd.2 for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:49:34.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:33 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:49:34.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:33 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:49:34.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:33 smithi022 ceph-mon[29462]: osdmap e468: 8 total, 8 up, 5 in 2024-09-11T07:49:34.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:33 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:49:34.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:33 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:49:34.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:33 smithi022 ceph-mon[37330]: osdmap e468: 8 total, 8 up, 5 in 2024-09-11T07:49:34.240 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:34.279 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:49:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:49:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:49:34.280 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:34.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:34.281 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:34.381 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:34.418 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:34.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:34.419 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:34.520 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:34.560 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:34.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:34.561 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:34.662 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:34.699 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:34.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:34.700 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:34.801 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:34.839 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:34.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:34.840 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:34.941 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:34.964 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:34.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:34.965 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:35.066 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:35.092 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:35.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:35.093 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:35.103 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:49:35.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:34 smithi022 ceph-mon[29462]: 5.19 scrub starts 2024-09-11T07:49:35.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:34 smithi022 ceph-mon[29462]: 5.19 scrub ok 2024-09-11T07:49:35.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:34 smithi022 ceph-mon[29462]: osdmap e469: 8 total, 8 up, 5 in 2024-09-11T07:49:35.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:34 smithi022 ceph-mon[29462]: 2.1c scrub starts 2024-09-11T07:49:35.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:34 smithi022 ceph-mon[29462]: 2.1c scrub ok 2024-09-11T07:49:35.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:34 smithi022 ceph-mon[29462]: pgmap v1073: 158 pgs: 4 active+remapped+backfilling, 3 active+clean+remapped, 3 active+remapped+backfill_toofull, 148 active+clean; 477 MiB data, 5.4 GiB used, 442 GiB / 447 GiB avail; 4.4 KiB/s rd, 27 KiB/s wr, 608 op/s; 4199/23475 objects misplaced (17.887%); 15 MiB/s, 244 objects/s recovering 2024-09-11T07:49:35.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:34 smithi022 ceph-mon[37330]: 5.19 scrub starts 2024-09-11T07:49:35.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:34 smithi022 ceph-mon[37330]: 5.19 scrub ok 2024-09-11T07:49:35.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:34 smithi022 ceph-mon[37330]: osdmap e469: 8 total, 8 up, 5 in 2024-09-11T07:49:35.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:34 smithi022 ceph-mon[37330]: 2.1c scrub starts 2024-09-11T07:49:35.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:34 smithi022 ceph-mon[37330]: 2.1c scrub ok 2024-09-11T07:49:35.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:34 smithi022 ceph-mon[37330]: pgmap v1073: 158 pgs: 4 active+remapped+backfilling, 3 active+clean+remapped, 3 active+remapped+backfill_toofull, 148 active+clean; 477 MiB data, 5.4 GiB used, 442 GiB / 447 GiB avail; 4.4 KiB/s rd, 27 KiB/s wr, 608 op/s; 4199/23475 objects misplaced (17.887%); 15 MiB/s, 244 objects/s recovering 2024-09-11T07:49:35.193 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:35.244 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:35.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:35.245 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:35.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:34 smithi104 ceph-mon[33507]: 5.19 scrub starts 2024-09-11T07:49:35.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:34 smithi104 ceph-mon[33507]: 5.19 scrub ok 2024-09-11T07:49:35.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:34 smithi104 ceph-mon[33507]: osdmap e469: 8 total, 8 up, 5 in 2024-09-11T07:49:35.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:34 smithi104 ceph-mon[33507]: 2.1c scrub starts 2024-09-11T07:49:35.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:34 smithi104 ceph-mon[33507]: 2.1c scrub ok 2024-09-11T07:49:35.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:34 smithi104 ceph-mon[33507]: pgmap v1073: 158 pgs: 4 active+remapped+backfilling, 3 active+clean+remapped, 3 active+remapped+backfill_toofull, 148 active+clean; 477 MiB data, 5.4 GiB used, 442 GiB / 447 GiB avail; 4.4 KiB/s rd, 27 KiB/s wr, 608 op/s; 4199/23475 objects misplaced (17.887%); 15 MiB/s, 244 objects/s recovering 2024-09-11T07:49:35.346 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:35.372 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:35.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:35.373 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:35.474 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:35.502 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:35.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:35.503 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:35.604 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:35.627 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:35.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:35.629 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:35.729 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:35.755 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:35.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:35.756 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:35.857 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:35.894 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:35.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:35.896 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:35.996 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:36.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:36.033 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:36.033 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:36.067 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:49:36.068 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:49:36.068 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2024-09-11T07:49:36.068 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2024-09-11T07:49:36.068 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:49:36.134 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:36.161 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:36.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:36.162 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:36.263 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:36.287 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:36.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:36.288 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:36.389 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:36.416 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:49:36 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2[53240]: 2024-09-11T07:49:36.026+0000 7fe7a8abf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:49:36.417 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:49:36 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2[53240]: 2024-09-11T07:49:36.026+0000 7fe7a8abf640 -1 osd.2 469 *** Got signal Terminated *** 2024-09-11T07:49:36.417 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:49:36 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2[53240]: 2024-09-11T07:49:36.026+0000 7fe7a8abf640 -1 osd.2 469 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:49:36.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:36.449 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:36.449 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:36.550 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:36.621 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:36.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:36.625 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:36.726 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:36.751 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:36.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:36.752 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:36.816 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-11T07:49:36.853 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:36.887 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:36.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:36.888 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:36.940 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T07:49:36.989 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:37.013 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:37.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:37.014 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:37.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:36 smithi104 ceph-mon[33507]: 11.c scrub starts 2024-09-11T07:49:37.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:36 smithi104 ceph-mon[33507]: 11.c scrub ok 2024-09-11T07:49:37.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:36 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:49:37.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:36 smithi104 ceph-mon[33507]: osd.2 marked itself down and dead 2024-09-11T07:49:37.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:36 smithi104 ceph-mon[33507]: pgmap v1074: 158 pgs: 3 active+clean+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 280 MiB data, 5.1 GiB used, 442 GiB / 447 GiB avail; 4.2 KiB/s rd, 25 KiB/s wr, 758 op/s; 1759/14022 objects misplaced (12.545%); 17 MiB/s, 282 objects/s recovering 2024-09-11T07:49:37.115 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:37.137 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:37.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:37.138 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:37.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:36 smithi022 ceph-mon[29462]: 11.c scrub starts 2024-09-11T07:49:37.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:36 smithi022 ceph-mon[29462]: 11.c scrub ok 2024-09-11T07:49:37.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:36 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:49:37.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:36 smithi022 ceph-mon[29462]: osd.2 marked itself down and dead 2024-09-11T07:49:37.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:36 smithi022 ceph-mon[29462]: pgmap v1074: 158 pgs: 3 active+clean+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 280 MiB data, 5.1 GiB used, 442 GiB / 447 GiB avail; 4.2 KiB/s rd, 25 KiB/s wr, 758 op/s; 1759/14022 objects misplaced (12.545%); 17 MiB/s, 282 objects/s recovering 2024-09-11T07:49:37.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:36 smithi022 ceph-mon[37330]: 11.c scrub starts 2024-09-11T07:49:37.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:36 smithi022 ceph-mon[37330]: 11.c scrub ok 2024-09-11T07:49:37.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:36 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:49:37.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:36 smithi022 ceph-mon[37330]: osd.2 marked itself down and dead 2024-09-11T07:49:37.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:36 smithi022 ceph-mon[37330]: pgmap v1074: 158 pgs: 3 active+clean+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 280 MiB data, 5.1 GiB used, 442 GiB / 447 GiB avail; 4.2 KiB/s rd, 25 KiB/s wr, 758 op/s; 1759/14022 objects misplaced (12.545%); 17 MiB/s, 282 objects/s recovering 2024-09-11T07:49:37.239 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:37.261 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:37.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:37.262 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:37.363 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:37.384 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:37.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:37.385 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:37.486 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:37.507 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:37.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:37.508 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:37.609 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:37.633 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:37.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:37.634 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:37.649 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:49:37.734 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:37.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:37.756 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:37.756 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:37.857 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:37.882 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:37.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:37.883 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:37.983 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:38.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:38.014 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:38.014 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:38.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:37 smithi104 ceph-mon[33507]: osdmap e470: 8 total, 7 up, 5 in 2024-09-11T07:49:38.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:37 smithi104 ceph-mon[33507]: 11.0 scrub starts 2024-09-11T07:49:38.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:37 smithi104 ceph-mon[33507]: 11.0 scrub ok 2024-09-11T07:49:38.114 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:38.137 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:38.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:38.138 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:38.166 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:49:38.066+0000 7f0b5d524640 -1 mgr get_metadata_python Requested missing service osd.2 2024-09-11T07:49:38.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:37 smithi022 ceph-mon[29462]: osdmap e470: 8 total, 7 up, 5 in 2024-09-11T07:49:38.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:37 smithi022 ceph-mon[29462]: 11.0 scrub starts 2024-09-11T07:49:38.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:37 smithi022 ceph-mon[29462]: 11.0 scrub ok 2024-09-11T07:49:38.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:37 smithi022 ceph-mon[37330]: osdmap e470: 8 total, 7 up, 5 in 2024-09-11T07:49:38.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:37 smithi022 ceph-mon[37330]: 11.0 scrub starts 2024-09-11T07:49:38.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:37 smithi022 ceph-mon[37330]: 11.0 scrub ok 2024-09-11T07:49:38.239 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:38.287 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:38.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:38.288 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:38.389 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:38.412 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:38.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:38.413 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:38.514 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:38.540 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:38.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:38.541 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:38.641 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:38.669 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:38.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:38.670 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:38.771 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:38.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:38.792 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:38.792 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:38.893 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:38.930 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:38.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:38.931 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:39.032 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:39.056 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:49:38 smithi022 podman[208335]: 2024-09-11 07:49:38.79452866 +0000 UTC m=+3.356137668 container died f993337d6c883138e24c444f3d19cbaafc74cf83fcd95b5867410a6f12e0f0c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-11T07:49:39.056 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-mon[29462]: 6.17 scrub starts 2024-09-11T07:49:39.056 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-mon[29462]: 6.17 scrub ok 2024-09-11T07:49:39.056 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-mon[29462]: 11.1 continuing backfill to osd.1 from (455'11995,469'14808] MIN to 469'14808 2024-09-11T07:49:39.056 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-mon[29462]: osdmap e471: 8 total, 7 up, 5 in 2024-09-11T07:49:39.056 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-mon[29462]: 11.4 continuing backfill to osd.1 from (445'9682,469'15107] MIN to 469'15107 2024-09-11T07:49:39.056 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-mon[29462]: 11.4 continuing backfill to osd.3 from (445'9682,469'15107] MIN to 469'15107 2024-09-11T07:49:39.057 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-mon[29462]: 11.f continuing backfill to osd.0 from (444'9281,469'14683] MIN to 469'14683 2024-09-11T07:49:39.057 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-mon[29462]: 11.f continuing backfill to osd.3 from (444'9281,469'14683] MIN to 469'14683 2024-09-11T07:49:39.057 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:49:39.057 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:49:39.057 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1738612552' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:49:39.057 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-mon[29462]: from='client.26147 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:49:39.057 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-mon[29462]: pgmap v1077: 158 pgs: 3 active+clean+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 280 MiB data, 5.1 GiB used, 442 GiB / 447 GiB avail; 932 B/s rd, 0 B/s wr, 653 op/s; 1759/14022 objects misplaced (12.545%); 15 MiB/s, 245 objects/s recovering 2024-09-11T07:49:39.057 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-mon[37330]: 6.17 scrub starts 2024-09-11T07:49:39.057 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-mon[37330]: 6.17 scrub ok 2024-09-11T07:49:39.057 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-mon[37330]: 11.1 continuing backfill to osd.1 from (455'11995,469'14808] MIN to 469'14808 2024-09-11T07:49:39.057 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-mon[37330]: osdmap e471: 8 total, 7 up, 5 in 2024-09-11T07:49:39.058 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-mon[37330]: 11.4 continuing backfill to osd.1 from (445'9682,469'15107] MIN to 469'15107 2024-09-11T07:49:39.058 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-mon[37330]: 11.4 continuing backfill to osd.3 from (445'9682,469'15107] MIN to 469'15107 2024-09-11T07:49:39.058 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-mon[37330]: 11.f continuing backfill to osd.0 from (444'9281,469'14683] MIN to 469'14683 2024-09-11T07:49:39.058 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-mon[37330]: 11.f continuing backfill to osd.3 from (444'9281,469'14683] MIN to 469'14683 2024-09-11T07:49:39.058 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:49:39.058 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:49:39.058 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1738612552' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:49:39.058 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-mon[37330]: from='client.26147 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:49:39.058 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:38 smithi022 ceph-mon[37330]: pgmap v1077: 158 pgs: 3 active+clean+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 280 MiB data, 5.1 GiB used, 442 GiB / 447 GiB avail; 932 B/s rd, 0 B/s wr, 653 op/s; 1759/14022 objects misplaced (12.545%); 15 MiB/s, 245 objects/s recovering 2024-09-11T07:49:39.070 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:39.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:39.071 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:39.080 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is set 2024-09-11T07:49:39.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:38 smithi104 ceph-mon[33507]: 6.17 scrub starts 2024-09-11T07:49:39.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:38 smithi104 ceph-mon[33507]: 6.17 scrub ok 2024-09-11T07:49:39.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:38 smithi104 ceph-mon[33507]: 11.1 continuing backfill to osd.1 from (455'11995,469'14808] MIN to 469'14808 2024-09-11T07:49:39.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:38 smithi104 ceph-mon[33507]: osdmap e471: 8 total, 7 up, 5 in 2024-09-11T07:49:39.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:38 smithi104 ceph-mon[33507]: 11.4 continuing backfill to osd.1 from (445'9682,469'15107] MIN to 469'15107 2024-09-11T07:49:39.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:38 smithi104 ceph-mon[33507]: 11.4 continuing backfill to osd.3 from (445'9682,469'15107] MIN to 469'15107 2024-09-11T07:49:39.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:38 smithi104 ceph-mon[33507]: 11.f continuing backfill to osd.0 from (444'9281,469'14683] MIN to 469'14683 2024-09-11T07:49:39.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:38 smithi104 ceph-mon[33507]: 11.f continuing backfill to osd.3 from (444'9281,469'14683] MIN to 469'14683 2024-09-11T07:49:39.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:49:39.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:49:39.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:38 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1738612552' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:49:39.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:38 smithi104 ceph-mon[33507]: from='client.26147 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-11T07:49:39.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:38 smithi104 ceph-mon[33507]: pgmap v1077: 158 pgs: 3 active+clean+remapped, 3 active+remapped+backfill_toofull, 152 active+clean; 280 MiB data, 5.1 GiB used, 442 GiB / 447 GiB avail; 932 B/s rd, 0 B/s wr, 653 op/s; 1759/14022 objects misplaced (12.545%); 15 MiB/s, 245 objects/s recovering 2024-09-11T07:49:39.172 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:39.195 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:39.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:39.196 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:39.297 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:39.317 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:39.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:39.318 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:39.419 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:39.441 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:39.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:39.442 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:39.528 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:49:39.529 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2024-09-11T07:49:39.529 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2024-09-11T07:49:39.529 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2024-09-11T07:49:39.529 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.529 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25706.0:254599 10.c 10:34692d4f:::benchmark_data_smithi022_163866_object125173:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e397)", 2024-09-11T07:49:39.529 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:45:30.138400+0000", 2024-09-11T07:49:39.529 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 249.374375128, 2024-09-11T07:49:39.529 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.65205146899999999, 2024-09-11T07:49:39.529 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:39.529 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:39.529 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:39.529 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25706", 2024-09-11T07:49:39.530 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:49:39.530 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 254599 2024-09-11T07:49:39.530 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.530 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:39.530 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.530 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:39.530 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.138400+0000", 2024-09-11T07:49:39.530 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.530 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.530 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.530 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:39.530 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.138400+0000", 2024-09-11T07:49:39.530 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.530 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.531 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.531 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:39.531 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.138399+0000", 2024-09-11T07:49:39.531 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:49:39.531 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.531 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.531 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:39.531 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.138405+0000", 2024-09-11T07:49:39.531 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8680000000000004e-06 2024-09-11T07:49:39.531 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.531 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.531 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:39.531 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.138413+0000", 2024-09-11T07:49:39.532 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6699999999999994e-06 2024-09-11T07:49:39.532 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.532 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.532 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:39.532 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.138416+0000", 2024-09-11T07:49:39.532 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5260000000000002e-06 2024-09-11T07:49:39.532 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.532 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.532 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.532 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.138453+0000", 2024-09-11T07:49:39.532 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6770000000000002e-05 2024-09-11T07:49:39.532 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.532 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.532 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:39.533 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.138458+0000", 2024-09-11T07:49:39.533 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6199999999999998e-06 2024-09-11T07:49:39.533 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.533 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.533 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.533 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.770416+0000", 2024-09-11T07:49:39.533 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.63195844899999998 2024-09-11T07:49:39.533 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.533 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.533 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:39.533 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.770487+0000", 2024-09-11T07:49:39.533 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0949000000000001e-05 2024-09-11T07:49:39.533 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.534 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.534 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:49:39.534 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.770553+0000", 2024-09-11T07:49:39.534 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6304999999999996e-05 2024-09-11T07:49:39.534 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.534 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.534 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:39.534 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.783705+0000", 2024-09-11T07:49:39.534 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.013151685 2024-09-11T07:49:39.534 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.534 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.534 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.534 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790322+0000", 2024-09-11T07:49:39.534 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0066173789999999996 2024-09-11T07:49:39.535 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.535 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.535 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.535 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790373+0000", 2024-09-11T07:49:39.535 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0693999999999999e-05 2024-09-11T07:49:39.535 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.535 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.535 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:39.535 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790417+0000", 2024-09-11T07:49:39.535 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3501000000000002e-05 2024-09-11T07:49:39.535 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.535 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.535 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:39.535 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790452+0000", 2024-09-11T07:49:39.536 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5377999999999999e-05 2024-09-11T07:49:39.536 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.536 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:39.536 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.536 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.536 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.536 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25706.0:254607 10.c 10:33e52c3b:::benchmark_data_smithi022_163866_object125181:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e397)", 2024-09-11T07:49:39.536 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:45:30.142279+0000", 2024-09-11T07:49:39.536 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 249.37049693200001, 2024-09-11T07:49:39.536 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.64834728900000005, 2024-09-11T07:49:39.536 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:39.536 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:39.536 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:39.536 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25706", 2024-09-11T07:49:39.537 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:49:39.537 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 254607 2024-09-11T07:49:39.537 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.537 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:39.537 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.537 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:39.537 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.142279+0000", 2024-09-11T07:49:39.537 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.537 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.537 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.537 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:39.537 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.142279+0000", 2024-09-11T07:49:39.537 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.537 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.538 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.538 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:39.538 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.142277+0000", 2024-09-11T07:49:39.538 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:49:39.538 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.538 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.538 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:39.538 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.142283+0000", 2024-09-11T07:49:39.538 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1020000000000002e-06 2024-09-11T07:49:39.538 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.538 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.538 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:39.538 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.142295+0000", 2024-09-11T07:49:39.538 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.222e-05 2024-09-11T07:49:39.538 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.539 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.539 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:39.539 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.142301+0000", 2024-09-11T07:49:39.539 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5509999999999999e-06 2024-09-11T07:49:39.539 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.539 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.539 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.539 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.142344+0000", 2024-09-11T07:49:39.539 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3316999999999999e-05 2024-09-11T07:49:39.539 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.539 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.539 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:39.539 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.142351+0000", 2024-09-11T07:49:39.540 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7750000000000004e-06 2024-09-11T07:49:39.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.540 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.540 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.773044+0000", 2024-09-11T07:49:39.540 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.63069285200000003 2024-09-11T07:49:39.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.540 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:39.540 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.773111+0000", 2024-09-11T07:49:39.540 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7602999999999996e-05 2024-09-11T07:49:39.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.540 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.540 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:49:39.541 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.773176+0000", 2024-09-11T07:49:39.541 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4419999999999996e-05 2024-09-11T07:49:39.541 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.541 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.541 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:39.541 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790332+0000", 2024-09-11T07:49:39.541 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.017156062 2024-09-11T07:49:39.541 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.541 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.541 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.541 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790512+0000", 2024-09-11T07:49:39.541 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018059900000000001 2024-09-11T07:49:39.541 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.541 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.542 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.542 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790551+0000", 2024-09-11T07:49:39.542 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8346000000000002e-05 2024-09-11T07:49:39.542 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.542 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.542 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:39.542 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790591+0000", 2024-09-11T07:49:39.542 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0272999999999999e-05 2024-09-11T07:49:39.542 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.542 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.542 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:39.542 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790626+0000", 2024-09-11T07:49:39.542 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4898999999999999e-05 2024-09-11T07:49:39.542 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.543 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:39.543 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.543 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.543 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.543 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:192480 11.4 11:2c053524:::benchmark_data_smithi022_188626_object72672:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e452)", 2024-09-11T07:49:39.543 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:48:46.105438+0000", 2024-09-11T07:49:39.543 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 53.407337325999997, 2024-09-11T07:49:39.543 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.2086161249999998, 2024-09-11T07:49:39.543 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:39.543 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:49:39.543 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:39.543 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:49:39.543 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:49:39.544 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 192480 2024-09-11T07:49:39.544 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.544 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:39.544 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.544 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:39.544 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.105438+0000", 2024-09-11T07:49:39.544 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.544 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.544 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.544 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:39.544 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.105438+0000", 2024-09-11T07:49:39.544 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.544 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.544 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.545 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:39.545 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.105437+0000", 2024-09-11T07:49:39.545 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:49:39.545 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.545 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:39.545 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.105443+0000", 2024-09-11T07:49:39.545 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9490000000000001e-06 2024-09-11T07:49:39.545 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.545 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.545 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:39.545 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.105453+0000", 2024-09-11T07:49:39.545 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8810000000000008e-06 2024-09-11T07:49:39.546 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.546 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.546 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:39.546 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.105457+0000", 2024-09-11T07:49:39.546 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8359999999999999e-06 2024-09-11T07:49:39.546 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.546 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.546 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.546 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.105491+0000", 2024-09-11T07:49:39.546 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3840999999999997e-05 2024-09-11T07:49:39.546 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.546 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.546 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:39.546 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.105571+0000", 2024-09-11T07:49:39.546 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9730999999999999e-05 2024-09-11T07:49:39.547 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.547 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.547 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:49:39.547 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.105639+0000", 2024-09-11T07:49:39.547 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.8310000000000002e-05 2024-09-11T07:49:39.547 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.547 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.547 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:39.547 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.108088+0000", 2024-09-11T07:49:39.547 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0024489350000000002 2024-09-11T07:49:39.547 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.547 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.547 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.547 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.108465+0000", 2024-09-11T07:49:39.547 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00037678100000000002 2024-09-11T07:49:39.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.548 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.548 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.137118+0000", 2024-09-11T07:49:39.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.028653151000000002 2024-09-11T07:49:39.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.548 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.548 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:39.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.137129+0000", 2024-09-11T07:49:39.548 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0584999999999999e-05 2024-09-11T07:49:39.548 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.548 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.548 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.548 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.134593+0000", 2024-09-11T07:49:39.549 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.99746406899999995 2024-09-11T07:49:39.549 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.549 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:39.549 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.134601+0000", 2024-09-11T07:49:39.549 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.585e-06 2024-09-11T07:49:39.549 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.549 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.549 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.313976+0000", 2024-09-11T07:49:39.549 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1793741639999999 2024-09-11T07:49:39.549 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.549 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.550 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:39.550 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.314054+0000", 2024-09-11T07:49:39.550 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.8885000000000001e-05 2024-09-11T07:49:39.550 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.550 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:39.550 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.550 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.550 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.550 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:192487 11.f 11:fd58c779:::benchmark_data_smithi022_188626_object72679:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e452)", 2024-09-11T07:49:39.550 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:48:46.113756+0000", 2024-09-11T07:49:39.550 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 53.399019131999999, 2024-09-11T07:49:39.550 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.2508924820000002, 2024-09-11T07:49:39.550 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:39.550 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:49:39.551 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:39.551 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:49:39.551 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:49:39.551 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 192487 2024-09-11T07:49:39.551 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.551 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:39.551 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.551 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:39.551 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.113756+0000", 2024-09-11T07:49:39.551 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.551 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.551 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.551 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:39.551 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.113756+0000", 2024-09-11T07:49:39.551 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.552 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.552 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.552 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:39.552 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.113755+0000", 2024-09-11T07:49:39.552 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:49:39.552 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.552 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.552 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:39.552 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.113761+0000", 2024-09-11T07:49:39.552 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0449999999999997e-06 2024-09-11T07:49:39.552 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.552 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.552 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:39.552 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.113770+0000", 2024-09-11T07:49:39.553 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5810000000000003e-06 2024-09-11T07:49:39.553 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.553 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.553 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:39.553 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.113777+0000", 2024-09-11T07:49:39.553 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4620000000000001e-06 2024-09-11T07:49:39.553 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.553 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.553 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.553 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.113811+0000", 2024-09-11T07:49:39.553 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4360000000000003e-05 2024-09-11T07:49:39.553 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.553 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.553 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:39.554 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.113918+0000", 2024-09-11T07:49:39.554 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010678799999999999 2024-09-11T07:49:39.554 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.554 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.554 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:49:39.554 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.114005+0000", 2024-09-11T07:49:39.554 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6760000000000003e-05 2024-09-11T07:49:39.554 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.554 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.554 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:39.554 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.116601+0000", 2024-09-11T07:49:39.554 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0025965659999999998 2024-09-11T07:49:39.554 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.554 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.555 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.555 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.126220+0000", 2024-09-11T07:49:39.555 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0096189129999999998 2024-09-11T07:49:39.555 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.555 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.555 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.555 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.137933+0000", 2024-09-11T07:49:39.555 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.011713146000000001 2024-09-11T07:49:39.555 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.555 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.555 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:39.555 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.137943+0000", 2024-09-11T07:49:39.555 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0097e-05 2024-09-11T07:49:39.555 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.555 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.556 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.556 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.133216+0000", 2024-09-11T07:49:39.556 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.995272508 2024-09-11T07:49:39.556 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.556 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.556 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:39.556 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.133226+0000", 2024-09-11T07:49:39.556 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0594999999999999e-05 2024-09-11T07:49:39.556 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.556 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.556 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.556 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.364575+0000", 2024-09-11T07:49:39.556 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2313488690000001 2024-09-11T07:49:39.557 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.557 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.557 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:39.557 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.364649+0000", 2024-09-11T07:49:39.557 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3706000000000002e-05 2024-09-11T07:49:39.557 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.557 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:39.557 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.557 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.557 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.557 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:192492 11.f 11:f5250b21:::benchmark_data_smithi022_188626_object72684:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e452)", 2024-09-11T07:49:39.557 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:48:46.117935+0000", 2024-09-11T07:49:39.557 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 53.394840369000001, 2024-09-11T07:49:39.557 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.246796008, 2024-09-11T07:49:39.558 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:39.558 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:49:39.558 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:39.558 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:49:39.558 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:49:39.558 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 192492 2024-09-11T07:49:39.558 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.558 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:39.558 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.558 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:39.558 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.117935+0000", 2024-09-11T07:49:39.558 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.558 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.558 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.559 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:39.559 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.117935+0000", 2024-09-11T07:49:39.559 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.559 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.559 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.559 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:39.559 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.117934+0000", 2024-09-11T07:49:39.559 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:49:39.559 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.559 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.559 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:39.559 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.117941+0000", 2024-09-11T07:49:39.559 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1600000000000001e-06 2024-09-11T07:49:39.559 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.560 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.560 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:39.560 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.117952+0000", 2024-09-11T07:49:39.560 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1224e-05 2024-09-11T07:49:39.560 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.560 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.560 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:39.560 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.117957+0000", 2024-09-11T07:49:39.560 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9989999999999999e-06 2024-09-11T07:49:39.560 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.560 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.560 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.560 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.117991+0000", 2024-09-11T07:49:39.560 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3668999999999999e-05 2024-09-11T07:49:39.561 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.561 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.561 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:39.561 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.118070+0000", 2024-09-11T07:49:39.561 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9060999999999994e-05 2024-09-11T07:49:39.561 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.561 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.561 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:49:39.561 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.118144+0000", 2024-09-11T07:49:39.561 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4585999999999994e-05 2024-09-11T07:49:39.561 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.561 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.561 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:39.561 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.120427+0000", 2024-09-11T07:49:39.562 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0022825469999999998 2024-09-11T07:49:39.562 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.562 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.562 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.562 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.127292+0000", 2024-09-11T07:49:39.562 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0068656669999999998 2024-09-11T07:49:39.562 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.562 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.562 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.562 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.137970+0000", 2024-09-11T07:49:39.562 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.010677944 2024-09-11T07:49:39.562 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.562 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.563 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:39.563 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.137978+0000", 2024-09-11T07:49:39.563 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7870000000000002e-06 2024-09-11T07:49:39.563 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.563 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.563 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.563 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.133302+0000", 2024-09-11T07:49:39.563 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.99532337599999998 2024-09-11T07:49:39.563 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.563 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.563 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:39.563 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.133318+0000", 2024-09-11T07:49:39.563 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6620999999999999e-05 2024-09-11T07:49:39.563 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.564 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.564 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.564 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.364674+0000", 2024-09-11T07:49:39.564 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2313559709999999 2024-09-11T07:49:39.564 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.564 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.564 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:39.564 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.364731+0000", 2024-09-11T07:49:39.564 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7077999999999998e-05 2024-09-11T07:49:39.564 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.564 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:39.564 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.564 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.564 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.565 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24353.0:3077 5.d 5:b4812045:::notify.4:head [watch reconnect cookie 94395805755184 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e453)", 2024-09-11T07:49:39.565 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:48:46.124437+0000", 2024-09-11T07:49:39.565 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 53.388338746000002, 2024-09-11T07:49:39.565 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0675898260000001, 2024-09-11T07:49:39.565 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:39.565 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2024-09-11T07:49:39.565 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:39.565 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24353", 2024-09-11T07:49:39.565 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1860969011", 2024-09-11T07:49:39.565 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 3077 2024-09-11T07:49:39.565 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.565 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:39.565 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.565 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:39.566 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124437+0000", 2024-09-11T07:49:39.566 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.566 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.566 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.566 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:39.566 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124437+0000", 2024-09-11T07:49:39.566 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.566 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.566 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.566 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:39.566 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124436+0000", 2024-09-11T07:49:39.566 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:49:39.566 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.566 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.566 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:39.567 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124441+0000", 2024-09-11T07:49:39.567 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3419999999999998e-06 2024-09-11T07:49:39.567 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.567 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.567 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:39.567 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124450+0000", 2024-09-11T07:49:39.567 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6680000000000007e-06 2024-09-11T07:49:39.567 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.567 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.567 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:39.567 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124453+0000", 2024-09-11T07:49:39.567 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8410000000000002e-06 2024-09-11T07:49:39.567 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.567 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.567 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.568 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124497+0000", 2024-09-11T07:49:39.568 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3220999999999998e-05 2024-09-11T07:49:39.568 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.568 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.568 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op must wait for map", 2024-09-11T07:49:39.568 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124503+0000", 2024-09-11T07:49:39.568 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8340000000000004e-06 2024-09-11T07:49:39.568 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.568 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.568 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.568 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.133144+0000", 2024-09-11T07:49:39.568 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0086415260000000001 2024-09-11T07:49:39.568 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.568 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.569 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:39.569 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.133153+0000", 2024-09-11T07:49:39.569 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9309999999999997e-06 2024-09-11T07:49:39.569 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.569 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.569 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.569 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.171356+0000", 2024-09-11T07:49:39.569 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0382031789999999 2024-09-11T07:49:39.569 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.569 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.569 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for degraded object", 2024-09-11T07:49:39.569 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.171652+0000", 2024-09-11T07:49:39.569 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00029536999999999999 2024-09-11T07:49:39.570 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.570 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.570 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.570 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.179886+0000", 2024-09-11T07:49:39.570 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0082341789999999995 2024-09-11T07:49:39.570 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.570 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.570 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:39.570 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.179964+0000", 2024-09-11T07:49:39.570 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7897000000000003e-05 2024-09-11T07:49:39.570 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.570 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.571 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:39.571 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.192027+0000", 2024-09-11T07:49:39.571 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.012063084999999999 2024-09-11T07:49:39.571 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.571 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:39.571 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.571 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.571 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.571 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24353.0:3079 5.14 5:2b04a3e9:::notify.6:head [watch reconnect cookie 94395805760480 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e453)", 2024-09-11T07:49:39.571 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:48:46.124463+0000", 2024-09-11T07:49:39.571 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 53.388312753000001, 2024-09-11T07:49:39.571 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.070989915, 2024-09-11T07:49:39.571 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:39.572 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2024-09-11T07:49:39.572 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:39.572 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24353", 2024-09-11T07:49:39.572 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1860969011", 2024-09-11T07:49:39.572 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 3079 2024-09-11T07:49:39.572 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.572 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:39.572 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.572 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:39.572 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124463+0000", 2024-09-11T07:49:39.572 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.572 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.572 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.573 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:39.573 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124463+0000", 2024-09-11T07:49:39.573 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.573 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.573 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.573 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:39.573 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124462+0000", 2024-09-11T07:49:39.573 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:49:39.573 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.573 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.573 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:39.573 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124465+0000", 2024-09-11T07:49:39.573 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.8619999999999999e-06 2024-09-11T07:49:39.573 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.574 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.574 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:39.574 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124471+0000", 2024-09-11T07:49:39.574 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0499999999999997e-06 2024-09-11T07:49:39.574 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.574 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.574 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:39.574 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124476+0000", 2024-09-11T07:49:39.574 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.69e-06 2024-09-11T07:49:39.574 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.574 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.574 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.574 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124537+0000", 2024-09-11T07:49:39.574 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1155999999999995e-05 2024-09-11T07:49:39.575 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.575 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.575 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op must wait for map", 2024-09-11T07:49:39.575 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124544+0000", 2024-09-11T07:49:39.575 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.601e-06 2024-09-11T07:49:39.575 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.575 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.575 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.575 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.139252+0000", 2024-09-11T07:49:39.575 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.014708407 2024-09-11T07:49:39.575 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.575 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.575 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:39.575 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.139260+0000", 2024-09-11T07:49:39.576 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.8490000000000006e-06 2024-09-11T07:49:39.576 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.576 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.576 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.576 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.180500+0000", 2024-09-11T07:49:39.576 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0412405689999999 2024-09-11T07:49:39.576 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.576 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.576 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for degraded object", 2024-09-11T07:49:39.576 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.180809+0000", 2024-09-11T07:49:39.576 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00030839099999999998 2024-09-11T07:49:39.576 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.576 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.576 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.577 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.192259+0000", 2024-09-11T07:49:39.577 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.011450426 2024-09-11T07:49:39.577 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.577 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.577 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:39.577 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.192335+0000", 2024-09-11T07:49:39.577 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5426999999999995e-05 2024-09-11T07:49:39.577 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.577 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.577 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:39.577 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.195453+0000", 2024-09-11T07:49:39.577 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003118039 2024-09-11T07:49:39.577 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.578 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:39.578 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.578 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.578 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.578 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24353.0:3080 5.9 5:93e5b521:::notify.7:head [watch reconnect cookie 94395805742128 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e453)", 2024-09-11T07:49:39.578 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:48:46.124482+0000", 2024-09-11T07:49:39.578 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 53.388293093999998, 2024-09-11T07:49:39.578 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0809971739999999, 2024-09-11T07:49:39.578 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:39.578 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2024-09-11T07:49:39.578 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:39.578 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24353", 2024-09-11T07:49:39.578 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1860969011", 2024-09-11T07:49:39.578 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 3080 2024-09-11T07:49:39.578 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.579 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:39.579 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.579 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:39.579 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124482+0000", 2024-09-11T07:49:39.579 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.579 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.579 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.579 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:39.579 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124482+0000", 2024-09-11T07:49:39.579 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.579 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.579 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.579 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:39.580 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124482+0000", 2024-09-11T07:49:39.580 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:49:39.580 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.580 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.580 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:39.580 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124485+0000", 2024-09-11T07:49:39.580 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.6230000000000001e-06 2024-09-11T07:49:39.580 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.580 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.580 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:39.580 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124491+0000", 2024-09-11T07:49:39.580 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7120000000000003e-06 2024-09-11T07:49:39.580 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.580 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.581 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:39.581 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124495+0000", 2024-09-11T07:49:39.581 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5389999999999999e-06 2024-09-11T07:49:39.581 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.581 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.581 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.581 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124539+0000", 2024-09-11T07:49:39.581 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4539999999999997e-05 2024-09-11T07:49:39.581 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.581 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.581 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op must wait for map", 2024-09-11T07:49:39.581 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124545+0000", 2024-09-11T07:49:39.581 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6749999999999999e-06 2024-09-11T07:49:39.582 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.582 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.582 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.582 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.130844+0000", 2024-09-11T07:49:39.582 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0062988330000000002 2024-09-11T07:49:39.582 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.582 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.582 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:39.582 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.130853+0000", 2024-09-11T07:49:39.582 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8400000000000001e-06 2024-09-11T07:49:39.582 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.582 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.582 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.582 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.166368+0000", 2024-09-11T07:49:39.583 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0355154 2024-09-11T07:49:39.583 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.583 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.583 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for degraded object", 2024-09-11T07:49:39.583 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.166658+0000", 2024-09-11T07:49:39.583 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00028989 2024-09-11T07:49:39.583 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.583 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.583 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.583 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.192286+0000", 2024-09-11T07:49:39.583 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.025627937 2024-09-11T07:49:39.583 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.583 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.583 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:39.584 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.192359+0000", 2024-09-11T07:49:39.584 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3052999999999993e-05 2024-09-11T07:49:39.584 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.584 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.584 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:39.584 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.205480+0000", 2024-09-11T07:49:39.584 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.013120573999999999 2024-09-11T07:49:39.584 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.584 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:39.584 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.584 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.584 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.584 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:192486 11.c 11:3cae2f82:::benchmark_data_smithi022_188626_object72678:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e453)", 2024-09-11T07:49:39.584 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:48:46.138064+0000", 2024-09-11T07:49:39.585 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 53.374711875999999, 2024-09-11T07:49:39.585 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.089428566, 2024-09-11T07:49:39.585 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:39.585 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:49:39.585 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:39.585 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:49:39.585 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:49:39.585 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 192486 2024-09-11T07:49:39.585 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.585 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:39.585 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.585 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:39.585 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.138064+0000", 2024-09-11T07:49:39.586 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.586 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.586 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.586 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:39.586 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.138064+0000", 2024-09-11T07:49:39.586 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.586 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.586 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.586 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:39.586 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.138061+0000", 2024-09-11T07:49:39.586 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:49:39.586 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.586 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.586 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:39.587 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.138069+0000", 2024-09-11T07:49:39.587 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5039999999999996e-06 2024-09-11T07:49:39.587 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.587 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.587 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:39.587 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.138079+0000", 2024-09-11T07:49:39.587 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8190000000000003e-06 2024-09-11T07:49:39.587 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.587 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.587 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:39.587 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.138086+0000", 2024-09-11T07:49:39.587 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.8399999999999997e-06 2024-09-11T07:49:39.587 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.587 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.588 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.588 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.139630+0000", 2024-09-11T07:49:39.588 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0015441719999999999 2024-09-11T07:49:39.588 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.588 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.588 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:39.588 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.139637+0000", 2024-09-11T07:49:39.588 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6729999999999992e-06 2024-09-11T07:49:39.588 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.588 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.588 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.588 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.134574+0000", 2024-09-11T07:49:39.588 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.99493694799999999 2024-09-11T07:49:39.588 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.589 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.589 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:39.589 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.134583+0000", 2024-09-11T07:49:39.589 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5439999999999999e-06 2024-09-11T07:49:39.589 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.589 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.589 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.589 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.226403+0000", 2024-09-11T07:49:39.589 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0918195580000001 2024-09-11T07:49:39.589 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.589 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.589 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:39.589 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.227492+0000", 2024-09-11T07:49:39.590 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0010897319999999999 2024-09-11T07:49:39.590 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.590 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:39.590 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.590 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.590 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.590 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:192489 11.c 11:316284ac:::benchmark_data_smithi022_188626_object72681:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e453)", 2024-09-11T07:49:39.590 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:48:46.138094+0000", 2024-09-11T07:49:39.590 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 53.374681774000003, 2024-09-11T07:49:39.590 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0895146549999999, 2024-09-11T07:49:39.590 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:39.590 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:49:39.590 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:39.590 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:49:39.590 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:49:39.591 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 192489 2024-09-11T07:49:39.591 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.591 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:39.591 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.591 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:39.591 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.138094+0000", 2024-09-11T07:49:39.591 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.591 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.591 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.591 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:39.591 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.138094+0000", 2024-09-11T07:49:39.591 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.591 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.591 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.592 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:39.592 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.138093+0000", 2024-09-11T07:49:39.592 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:49:39.592 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.592 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.592 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:39.592 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.138096+0000", 2024-09-11T07:49:39.592 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.303e-06 2024-09-11T07:49:39.592 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.592 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.592 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:39.592 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.138102+0000", 2024-09-11T07:49:39.592 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4070000000000001e-06 2024-09-11T07:49:39.592 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.593 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.593 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:39.593 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.138105+0000", 2024-09-11T07:49:39.593 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5939999999999998e-06 2024-09-11T07:49:39.593 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.593 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.593 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.593 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.139658+0000", 2024-09-11T07:49:39.593 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00155288 2024-09-11T07:49:39.593 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.593 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.593 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:39.593 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.139665+0000", 2024-09-11T07:49:39.593 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5520000000000001e-06 2024-09-11T07:49:39.594 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.594 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.594 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.594 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.134602+0000", 2024-09-11T07:49:39.594 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.99493757800000004 2024-09-11T07:49:39.594 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.594 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.594 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:39.594 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.134609+0000", 2024-09-11T07:49:39.594 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6800000000000004e-06 2024-09-11T07:49:39.594 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.594 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.594 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.594 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.227534+0000", 2024-09-11T07:49:39.595 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.09292451 2024-09-11T07:49:39.595 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.595 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.595 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:39.595 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.227608+0000", 2024-09-11T07:49:39.595 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4814000000000006e-05 2024-09-11T07:49:39.595 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.595 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:39.595 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.595 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.595 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.595 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:192498 11.c 11:35807c2b:::benchmark_data_smithi022_188626_object72690:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e453)", 2024-09-11T07:49:39.595 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:48:46.169803+0000", 2024-09-11T07:49:39.595 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 53.342973028000003, 2024-09-11T07:49:39.596 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1126209729999998, 2024-09-11T07:49:39.596 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:39.596 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:39.596 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:39.596 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:49:39.596 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:49:39.596 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 192498 2024-09-11T07:49:39.596 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.596 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:39.596 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.596 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:39.596 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.169803+0000", 2024-09-11T07:49:39.596 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.596 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.597 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.597 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:39.597 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.169803+0000", 2024-09-11T07:49:39.597 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.597 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.597 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.597 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:39.597 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.169801+0000", 2024-09-11T07:49:39.597 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:39.599 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:49:39.599 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.599 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.599 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:39.599 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.169807+0000", 2024-09-11T07:49:39.599 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.494e-06 2024-09-11T07:49:39.599 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.599 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.599 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:39.600 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.169818+0000", 2024-09-11T07:49:39.600 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.049e-05 2024-09-11T07:49:39.600 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.600 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.600 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:39.600 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.169827+0000", 2024-09-11T07:49:39.600 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5300000000000002e-06 2024-09-11T07:49:39.600 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.600 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.600 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.600 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.169867+0000", 2024-09-11T07:49:39.600 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9987000000000002e-05 2024-09-11T07:49:39.600 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.601 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.601 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:39.601 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.169874+0000", 2024-09-11T07:49:39.601 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.55e-06 2024-09-11T07:49:39.601 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.601 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.601 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.601 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.134628+0000", 2024-09-11T07:49:39.601 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.96475460099999999 2024-09-11T07:49:39.601 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.601 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.601 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:39.601 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.134635+0000", 2024-09-11T07:49:39.602 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9009999999999997e-06 2024-09-11T07:49:39.602 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.602 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.602 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.602 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.227645+0000", 2024-09-11T07:49:39.602 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0930101059999999 2024-09-11T07:49:39.602 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.602 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.602 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:39.602 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.227867+0000", 2024-09-11T07:49:39.602 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00022187300000000001 2024-09-11T07:49:39.602 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.602 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.602 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2,5", 2024-09-11T07:49:39.603 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.227948+0000", 2024-09-11T07:49:39.603 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1142000000000003e-05 2024-09-11T07:49:39.603 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.603 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.603 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:39.603 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.232220+0000", 2024-09-11T07:49:39.603 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0042720479999999996 2024-09-11T07:49:39.603 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.603 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.603 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.603 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.232256+0000", 2024-09-11T07:49:39.603 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5828000000000003e-05 2024-09-11T07:49:39.603 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.603 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.604 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.604 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.237868+0000", 2024-09-11T07:49:39.604 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0056120880000000003 2024-09-11T07:49:39.604 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.604 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.604 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.604 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.282241+0000", 2024-09-11T07:49:39.604 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.044372822999999999 2024-09-11T07:49:39.604 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.604 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.604 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:39.604 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.282298+0000", 2024-09-11T07:49:39.604 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7343999999999998e-05 2024-09-11T07:49:39.605 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.605 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.605 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:39.605 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.282424+0000", 2024-09-11T07:49:39.605 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012505799999999999 2024-09-11T07:49:39.605 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.605 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:39.605 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.605 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.605 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.605 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:237421 11.f 11:f5176df0:::benchmark_data_smithi022_188626_object117613:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e469)", 2024-09-11T07:49:39.605 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:36.818636+0000", 2024-09-11T07:49:39.605 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 2.6941398400000001, 2024-09-11T07:49:39.606 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.056616448, 2024-09-11T07:49:39.606 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:39.606 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:49:39.606 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:39.606 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:49:39.606 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:49:39.606 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 237421 2024-09-11T07:49:39.606 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.606 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:39.606 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.606 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:39.606 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818636+0000", 2024-09-11T07:49:39.606 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.606 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.607 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.607 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:39.607 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818636+0000", 2024-09-11T07:49:39.607 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.607 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.607 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.607 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:39.607 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818634+0000", 2024-09-11T07:49:39.607 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:49:39.607 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.607 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.607 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:39.607 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818641+0000", 2024-09-11T07:49:39.607 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.097e-06 2024-09-11T07:49:39.608 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.608 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.608 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:39.608 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818651+0000", 2024-09-11T07:49:39.608 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0319000000000001e-05 2024-09-11T07:49:39.608 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.608 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.608 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:39.608 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818656+0000", 2024-09-11T07:49:39.608 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.724e-06 2024-09-11T07:49:39.608 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.608 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.609 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.609 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818692+0000", 2024-09-11T07:49:39.609 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6757000000000001e-05 2024-09-11T07:49:39.609 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.609 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.609 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:39.609 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818854+0000", 2024-09-11T07:49:39.609 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000161228 2024-09-11T07:49:39.609 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.609 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.609 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,1,2,3", 2024-09-11T07:49:39.609 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.819040+0000", 2024-09-11T07:49:39.609 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000185813 2024-09-11T07:49:39.609 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.610 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.610 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:39.610 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.822270+0000", 2024-09-11T07:49:39.610 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0032303480000000001 2024-09-11T07:49:39.610 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.610 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.610 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.610 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.825552+0000", 2024-09-11T07:49:39.610 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0032818320000000002 2024-09-11T07:49:39.610 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.610 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.610 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.610 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.826494+0000", 2024-09-11T07:49:39.610 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000942357 2024-09-11T07:49:39.611 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.611 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.611 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.611 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.837443+0000", 2024-09-11T07:49:39.611 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.010948652 2024-09-11T07:49:39.611 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.611 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.611 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.611 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.842188+0000", 2024-09-11T07:49:39.611 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0047453010000000004 2024-09-11T07:49:39.611 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.611 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.611 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:39.611 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.842202+0000", 2024-09-11T07:49:39.612 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.402e-05 2024-09-11T07:49:39.612 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.612 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.612 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.612 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.875208+0000", 2024-09-11T07:49:39.612 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0330058799999999 2024-09-11T07:49:39.612 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.612 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.612 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:39.612 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.875252+0000", 2024-09-11T07:49:39.612 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4295000000000002e-05 2024-09-11T07:49:39.612 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.612 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:39.612 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.613 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.613 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.613 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:237422 11.f 11:fec018d7:::benchmark_data_smithi022_188626_object117614:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e469)", 2024-09-11T07:49:39.613 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:36.818663+0000", 2024-09-11T07:49:39.613 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 2.6941121880000001, 2024-09-11T07:49:39.613 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0566943790000001, 2024-09-11T07:49:39.613 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:39.613 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:49:39.613 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:39.613 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:49:39.613 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:49:39.613 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 237422 2024-09-11T07:49:39.613 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.614 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:39.614 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.614 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:39.614 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818663+0000", 2024-09-11T07:49:39.614 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.614 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.614 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.614 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:39.614 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818663+0000", 2024-09-11T07:49:39.614 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.614 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.614 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.614 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:39.615 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818663+0000", 2024-09-11T07:49:39.615 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:49:39.615 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.615 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.615 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:39.615 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818669+0000", 2024-09-11T07:49:39.615 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6740000000000002e-06 2024-09-11T07:49:39.615 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.615 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.615 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:39.615 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818675+0000", 2024-09-11T07:49:39.615 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9920000000000004e-06 2024-09-11T07:49:39.615 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.615 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.615 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:39.616 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818679+0000", 2024-09-11T07:49:39.616 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0879999999999997e-06 2024-09-11T07:49:39.616 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.616 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.616 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.616 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.822311+0000", 2024-09-11T07:49:39.616 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0036328530000000001 2024-09-11T07:49:39.616 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.616 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.616 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:39.616 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.822471+0000", 2024-09-11T07:49:39.616 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000159865 2024-09-11T07:49:39.616 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.616 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.617 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,1,2,3", 2024-09-11T07:49:39.617 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.822603+0000", 2024-09-11T07:49:39.617 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000131934 2024-09-11T07:49:39.617 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.617 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.617 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:39.617 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.825788+0000", 2024-09-11T07:49:39.617 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003184821 2024-09-11T07:49:39.617 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.617 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.617 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.617 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.834464+0000", 2024-09-11T07:49:39.617 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0086756890000000003 2024-09-11T07:49:39.618 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.618 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.618 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.618 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.836517+0000", 2024-09-11T07:49:39.618 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002053685 2024-09-11T07:49:39.618 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.618 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.618 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.618 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.842225+0000", 2024-09-11T07:49:39.618 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.005707681 2024-09-11T07:49:39.618 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.618 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.618 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:39.619 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.842230+0000", 2024-09-11T07:49:39.619 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7779999999999998e-06 2024-09-11T07:49:39.619 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.619 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.619 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.619 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.875289+0000", 2024-09-11T07:49:39.619 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0330593189999999 2024-09-11T07:49:39.619 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.619 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.619 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:39.619 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.875358+0000", 2024-09-11T07:49:39.619 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.8532e-05 2024-09-11T07:49:39.619 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.619 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:39.620 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.620 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.620 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.620 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:237434 11.f 11:fba8c028:::benchmark_data_smithi022_188626_object117626:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e469)", 2024-09-11T07:49:39.620 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:36.830957+0000", 2024-09-11T07:49:39.620 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 2.681818517, 2024-09-11T07:49:39.620 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.044451427, 2024-09-11T07:49:39.620 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:39.620 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:49:39.620 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:39.620 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:49:39.620 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:49:39.620 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 237434 2024-09-11T07:49:39.621 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.621 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:39.621 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.621 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:39.621 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.830957+0000", 2024-09-11T07:49:39.621 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.621 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.621 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.621 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:39.621 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.830957+0000", 2024-09-11T07:49:39.621 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.621 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.621 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.622 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:39.622 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.830955+0000", 2024-09-11T07:49:39.622 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:49:39.622 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.622 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.622 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:39.622 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.830964+0000", 2024-09-11T07:49:39.622 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.9259999999999995e-06 2024-09-11T07:49:39.622 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.622 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.623 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:39.623 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.830981+0000", 2024-09-11T07:49:39.623 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6646999999999998e-05 2024-09-11T07:49:39.623 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.623 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.623 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:39.623 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.830989+0000", 2024-09-11T07:49:39.623 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4549999999999998e-06 2024-09-11T07:49:39.623 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.623 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.623 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.623 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.831051+0000", 2024-09-11T07:49:39.623 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2564999999999995e-05 2024-09-11T07:49:39.624 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.624 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.624 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:39.624 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.831217+0000", 2024-09-11T07:49:39.624 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000165661 2024-09-11T07:49:39.624 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.624 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.624 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,1,2,3", 2024-09-11T07:49:39.624 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.831378+0000", 2024-09-11T07:49:39.624 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016110500000000001 2024-09-11T07:49:39.624 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.624 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.624 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:39.625 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.834560+0000", 2024-09-11T07:49:39.625 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003182463 2024-09-11T07:49:39.625 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.625 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.625 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.625 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.836772+0000", 2024-09-11T07:49:39.625 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0022113889999999998 2024-09-11T07:49:39.625 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.625 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.625 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.625 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.842253+0000", 2024-09-11T07:49:39.625 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0054810680000000004 2024-09-11T07:49:39.625 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.625 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.626 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:39.626 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.842261+0000", 2024-09-11T07:49:39.626 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1850000000000004e-06 2024-09-11T07:49:39.626 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.626 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.626 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.626 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.875375+0000", 2024-09-11T07:49:39.626 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.033113741 2024-09-11T07:49:39.626 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.626 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.626 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:39.626 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.875409+0000", 2024-09-11T07:49:39.626 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3779000000000003e-05 2024-09-11T07:49:39.627 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.627 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:39.627 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.627 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.627 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.627 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:237441 11.4 11:27d1a457:::benchmark_data_smithi022_188626_object117633:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e469)", 2024-09-11T07:49:39.627 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:36.837200+0000", 2024-09-11T07:49:39.627 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 2.6755756609999999, 2024-09-11T07:49:39.627 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.044532706, 2024-09-11T07:49:39.627 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:39.627 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:49:39.627 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:39.627 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:49:39.627 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:49:39.628 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 237441 2024-09-11T07:49:39.628 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.628 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:39.628 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.628 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:39.628 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.837200+0000", 2024-09-11T07:49:39.628 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.628 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.628 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.628 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:39.628 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.837200+0000", 2024-09-11T07:49:39.628 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.628 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.629 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.629 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:39.629 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.837197+0000", 2024-09-11T07:49:39.629 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:49:39.629 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.629 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.629 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:39.629 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.837205+0000", 2024-09-11T07:49:39.629 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0979999999999994e-06 2024-09-11T07:49:39.629 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.629 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.629 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:39.629 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.837220+0000", 2024-09-11T07:49:39.629 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4389000000000001e-05 2024-09-11T07:49:39.630 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.630 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.630 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:39.630 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.837227+0000", 2024-09-11T07:49:39.630 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.8519999999999998e-06 2024-09-11T07:49:39.630 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.630 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.630 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.630 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.837290+0000", 2024-09-11T07:49:39.630 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3184999999999999e-05 2024-09-11T07:49:39.630 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.630 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.630 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:39.630 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.837438+0000", 2024-09-11T07:49:39.631 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000147782 2024-09-11T07:49:39.631 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.631 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.631 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2,3,6", 2024-09-11T07:49:39.631 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.837565+0000", 2024-09-11T07:49:39.631 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012728199999999999 2024-09-11T07:49:39.631 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.631 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.631 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.631 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.842560+0000", 2024-09-11T07:49:39.631 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0049948960000000004 2024-09-11T07:49:39.631 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.631 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.631 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:39.632 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.842566+0000", 2024-09-11T07:49:39.632 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4479999999999999e-06 2024-09-11T07:49:39.632 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.632 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.632 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.632 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.871264+0000", 2024-09-11T07:49:39.632 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.028698069 2024-09-11T07:49:39.632 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.632 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.632 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for missing object", 2024-09-11T07:49:39.632 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.871345+0000", 2024-09-11T07:49:39.632 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0586000000000004e-05 2024-09-11T07:49:39.632 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.633 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.633 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.633 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881683+0000", 2024-09-11T07:49:39.633 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.010338424000000001 2024-09-11T07:49:39.633 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.633 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.633 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:39.633 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881733+0000", 2024-09-11T07:49:39.633 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9259999999999999e-05 2024-09-11T07:49:39.633 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.633 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:39.633 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.633 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.633 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.634 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:237470 11.4 11:22333720:::benchmark_data_smithi022_188626_object117662:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e470)", 2024-09-11T07:49:39.634 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:36.872959+0000", 2024-09-11T07:49:39.634 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 2.639816588, 2024-09-11T07:49:39.634 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0085034479999999, 2024-09-11T07:49:39.634 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:39.634 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:39.634 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:39.634 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:49:39.634 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:49:39.634 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 237470 2024-09-11T07:49:39.634 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.634 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:39.634 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.634 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:39.635 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.872959+0000", 2024-09-11T07:49:39.635 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.635 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.635 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.635 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:39.635 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.872959+0000", 2024-09-11T07:49:39.635 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.635 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.635 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.635 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:39.635 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.872956+0000", 2024-09-11T07:49:39.635 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999967 2024-09-11T07:49:39.635 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.635 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.636 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:39.636 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.872968+0000", 2024-09-11T07:49:39.636 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2327000000000001e-05 2024-09-11T07:49:39.636 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.636 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.636 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:39.636 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.872987+0000", 2024-09-11T07:49:39.636 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8729e-05 2024-09-11T07:49:39.636 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.636 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.636 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:39.636 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.872995+0000", 2024-09-11T07:49:39.636 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4840000000000004e-06 2024-09-11T07:49:39.637 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.637 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.637 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.637 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.873041+0000", 2024-09-11T07:49:39.637 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5136000000000003e-05 2024-09-11T07:49:39.637 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.637 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.637 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:39.637 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.873048+0000", 2024-09-11T07:49:39.637 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.542e-06 2024-09-11T07:49:39.637 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.637 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.637 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.637 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.871371+0000", 2024-09-11T07:49:39.638 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.99832300100000004 2024-09-11T07:49:39.638 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.638 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.638 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:39.638 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.871453+0000", 2024-09-11T07:49:39.638 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2058999999999995e-05 2024-09-11T07:49:39.638 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.638 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.638 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3,6", 2024-09-11T07:49:39.638 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.871570+0000", 2024-09-11T07:49:39.638 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011685500000000001 2024-09-11T07:49:39.638 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.638 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.638 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:39.638 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.878964+0000", 2024-09-11T07:49:39.639 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0073936030000000003 2024-09-11T07:49:39.639 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.639 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.639 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.639 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881233+0000", 2024-09-11T07:49:39.639 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0022693180000000002 2024-09-11T07:49:39.639 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.639 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.639 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.639 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881265+0000", 2024-09-11T07:49:39.639 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1813000000000002e-05 2024-09-11T07:49:39.639 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.639 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.639 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.639 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881352+0000", 2024-09-11T07:49:39.640 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7348999999999993e-05 2024-09-11T07:49:39.640 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.640 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.640 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:39.640 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881387+0000", 2024-09-11T07:49:39.640 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5330999999999998e-05 2024-09-11T07:49:39.640 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.640 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.640 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:39.640 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881462+0000", 2024-09-11T07:49:39.640 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5037999999999995e-05 2024-09-11T07:49:39.640 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.640 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:39.641 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.641 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.641 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.641 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:237477 11.4 11:215731e4:::benchmark_data_smithi022_188626_object117669:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e470)", 2024-09-11T07:49:39.641 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:36.878069+0000", 2024-09-11T07:49:39.641 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 2.6347064590000002, 2024-09-11T07:49:39.641 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0040192189999999, 2024-09-11T07:49:39.641 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:39.641 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:39.641 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:39.641 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:49:39.641 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:49:39.641 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 237477 2024-09-11T07:49:39.641 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.642 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:39.642 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.642 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:39.642 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.878069+0000", 2024-09-11T07:49:39.642 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.642 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.642 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.642 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:39.642 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.878069+0000", 2024-09-11T07:49:39.642 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.642 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.642 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.642 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:39.642 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.878062+0000", 2024-09-11T07:49:39.643 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999928 2024-09-11T07:49:39.643 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.643 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.643 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:39.643 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.878079+0000", 2024-09-11T07:49:39.643 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7456999999999999e-05 2024-09-11T07:49:39.643 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.643 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.643 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:39.643 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.878100+0000", 2024-09-11T07:49:39.643 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0826e-05 2024-09-11T07:49:39.643 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.643 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.643 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:39.644 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.878111+0000", 2024-09-11T07:49:39.644 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1073000000000001e-05 2024-09-11T07:49:39.644 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.644 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.644 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.644 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.878173+0000", 2024-09-11T07:49:39.644 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1766999999999993e-05 2024-09-11T07:49:39.644 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.644 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.644 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:39.644 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.878184+0000", 2024-09-11T07:49:39.644 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1048e-05 2024-09-11T07:49:39.644 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.645 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.645 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.645 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.873786+0000", 2024-09-11T07:49:39.645 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.99560231399999999 2024-09-11T07:49:39.645 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.645 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.645 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:39.645 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.873893+0000", 2024-09-11T07:49:39.645 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010639100000000001 2024-09-11T07:49:39.645 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.645 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.645 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3,6", 2024-09-11T07:49:39.645 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.874016+0000", 2024-09-11T07:49:39.645 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000122998 2024-09-11T07:49:39.646 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.646 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.646 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:39.646 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881112+0000", 2024-09-11T07:49:39.646 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0070964009999999996 2024-09-11T07:49:39.646 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.646 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.646 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.646 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881292+0000", 2024-09-11T07:49:39.646 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017989899999999999 2024-09-11T07:49:39.646 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.646 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.646 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.646 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881324+0000", 2024-09-11T07:49:39.647 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1776000000000002e-05 2024-09-11T07:49:39.647 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.647 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.647 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.647 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881978+0000", 2024-09-11T07:49:39.647 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00065408700000000003 2024-09-11T07:49:39.647 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.647 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.647 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:39.647 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.882018+0000", 2024-09-11T07:49:39.647 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0346000000000003e-05 2024-09-11T07:49:39.647 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.647 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.648 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:39.648 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.882088+0000", 2024-09-11T07:49:39.648 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9967000000000003e-05 2024-09-11T07:49:39.648 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.648 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:39.648 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.648 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.648 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.648 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:237493 11.4 11:2e313620:::benchmark_data_smithi022_188626_object117685:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e470)", 2024-09-11T07:49:39.648 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:36.890350+0000", 2024-09-11T07:49:39.648 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 2.6224260230000001, 2024-09-11T07:49:39.648 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.99186289699999997, 2024-09-11T07:49:39.648 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:39.648 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:39.649 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:39.649 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:49:39.649 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:49:39.649 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 237493 2024-09-11T07:49:39.649 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.649 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:39.649 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.649 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:39.649 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.890350+0000", 2024-09-11T07:49:39.649 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.649 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.649 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.649 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:39.649 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.890350+0000", 2024-09-11T07:49:39.650 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.650 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.650 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.650 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:39.650 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.890346+0000", 2024-09-11T07:49:39.650 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999967 2024-09-11T07:49:39.650 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.650 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.650 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:39.650 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.890357+0000", 2024-09-11T07:49:39.650 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1063000000000001e-05 2024-09-11T07:49:39.650 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.650 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.650 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:39.651 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.890377+0000", 2024-09-11T07:49:39.651 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9568e-05 2024-09-11T07:49:39.651 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.651 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.651 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:39.651 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.890385+0000", 2024-09-11T07:49:39.651 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2989999999999997e-06 2024-09-11T07:49:39.651 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.651 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.651 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.651 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.890448+0000", 2024-09-11T07:49:39.651 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3238999999999995e-05 2024-09-11T07:49:39.651 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.651 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.651 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:39.652 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.890458+0000", 2024-09-11T07:49:39.652 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.9920000000000002e-06 2024-09-11T07:49:39.652 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.652 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.652 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.652 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.876601+0000", 2024-09-11T07:49:39.652 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.98614258200000005 2024-09-11T07:49:39.652 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.652 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.652 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:39.652 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.876694+0000", 2024-09-11T07:49:39.652 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3158999999999994e-05 2024-09-11T07:49:39.652 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.653 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.653 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3,6", 2024-09-11T07:49:39.653 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.876773+0000", 2024-09-11T07:49:39.653 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9018999999999997e-05 2024-09-11T07:49:39.653 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.653 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.653 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:39.653 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881238+0000", 2024-09-11T07:49:39.653 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0044648559999999997 2024-09-11T07:49:39.653 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.653 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.653 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.653 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881785+0000", 2024-09-11T07:49:39.653 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00054708000000000003 2024-09-11T07:49:39.653 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.654 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.654 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.654 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881864+0000", 2024-09-11T07:49:39.654 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9532999999999999e-05 2024-09-11T07:49:39.654 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.654 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.654 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.654 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.882109+0000", 2024-09-11T07:49:39.654 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00024501200000000002 2024-09-11T07:49:39.654 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.654 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.654 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:39.654 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.882138+0000", 2024-09-11T07:49:39.654 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.8235999999999999e-05 2024-09-11T07:49:39.655 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.655 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.655 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:39.655 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.882212+0000", 2024-09-11T07:49:39.655 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4734000000000006e-05 2024-09-11T07:49:39.655 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.655 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:39.655 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.655 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.655 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.655 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:237498 11.f 11:f14bae7d:::benchmark_data_smithi022_188626_object117690:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e470)", 2024-09-11T07:49:39.655 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:36.896808+0000", 2024-09-11T07:49:39.655 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 2.615967602, 2024-09-11T07:49:39.655 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0266028819999999, 2024-09-11T07:49:39.656 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:39.656 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:39.656 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:39.656 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:49:39.656 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:49:39.656 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 237498 2024-09-11T07:49:39.656 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.656 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:39.656 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.656 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:39.656 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.896808+0000", 2024-09-11T07:49:39.656 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.656 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.656 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.657 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:39.657 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.896808+0000", 2024-09-11T07:49:39.657 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.657 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.657 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.657 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:39.657 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.896805+0000", 2024-09-11T07:49:39.657 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999967 2024-09-11T07:49:39.657 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.657 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.657 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:39.657 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.896821+0000", 2024-09-11T07:49:39.657 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6093e-05 2024-09-11T07:49:39.657 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.658 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.658 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:39.658 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.896852+0000", 2024-09-11T07:49:39.658 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.0973999999999999e-05 2024-09-11T07:49:39.658 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.658 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.658 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:39.658 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.896859+0000", 2024-09-11T07:49:39.658 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.8720000000000002e-06 2024-09-11T07:49:39.658 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.658 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.658 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.658 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.896921+0000", 2024-09-11T07:49:39.658 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1914000000000004e-05 2024-09-11T07:49:39.659 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.659 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.659 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:39.659 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.896931+0000", 2024-09-11T07:49:39.659 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.6379999999999999e-06 2024-09-11T07:49:39.659 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.659 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.659 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.659 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.875435+0000", 2024-09-11T07:49:39.659 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.97850396699999997 2024-09-11T07:49:39.659 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.659 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.659 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:39.659 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.875548+0000", 2024-09-11T07:49:39.660 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000113455 2024-09-11T07:49:39.660 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.660 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.660 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,1,3", 2024-09-11T07:49:39.660 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.875675+0000", 2024-09-11T07:49:39.660 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012674600000000001 2024-09-11T07:49:39.660 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.660 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.660 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.660 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.878705+0000", 2024-09-11T07:49:39.660 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003029988 2024-09-11T07:49:39.660 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.660 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.661 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:39.661 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.878731+0000", 2024-09-11T07:49:39.661 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.5667999999999999e-05 2024-09-11T07:49:39.661 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.661 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.661 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.661 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.878860+0000", 2024-09-11T07:49:39.661 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012885300000000001 2024-09-11T07:49:39.661 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.661 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.661 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.661 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.923310+0000", 2024-09-11T07:49:39.661 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.044450305000000002 2024-09-11T07:49:39.661 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.662 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.662 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:39.662 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.923346+0000", 2024-09-11T07:49:39.662 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6426e-05 2024-09-11T07:49:39.662 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.662 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.662 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:39.662 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.923411+0000", 2024-09-11T07:49:39.662 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4450999999999997e-05 2024-09-11T07:49:39.662 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.662 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:39.662 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.662 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.662 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.663 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:237502 11.4 11:292f92e5:::benchmark_data_smithi022_188626_object117694:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e470)", 2024-09-11T07:49:39.663 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:36.901577+0000", 2024-09-11T07:49:39.663 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 2.6111982920000001, 2024-09-11T07:49:39.663 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.98077161899999998, 2024-09-11T07:49:39.663 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:49:39.663 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:49:39.663 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:49:39.663 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:49:39.663 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:49:39.663 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 237502 2024-09-11T07:49:39.663 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.663 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:49:39.663 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.663 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:49:39.664 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.901577+0000", 2024-09-11T07:49:39.664 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.664 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.664 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.664 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:49:39.664 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.901577+0000", 2024-09-11T07:49:39.664 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:49:39.664 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.664 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.664 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:49:39.664 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.901575+0000", 2024-09-11T07:49:39.664 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:49:39.664 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.664 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.665 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:49:39.665 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.901584+0000", 2024-09-11T07:49:39.665 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.679e-06 2024-09-11T07:49:39.665 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.665 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.665 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:49:39.665 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.901600+0000", 2024-09-11T07:49:39.665 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5688e-05 2024-09-11T07:49:39.665 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.665 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.665 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:49:39.665 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.901606+0000", 2024-09-11T07:49:39.665 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8200000000000002e-06 2024-09-11T07:49:39.665 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.666 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.666 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.666 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.901656+0000", 2024-09-11T07:49:39.666 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0343000000000002e-05 2024-09-11T07:49:39.666 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.666 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.666 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:49:39.666 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.901664+0000", 2024-09-11T07:49:39.666 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.661e-06 2024-09-11T07:49:39.666 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.666 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.666 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:49:39.666 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.878992+0000", 2024-09-11T07:49:39.666 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.97732787399999999 2024-09-11T07:49:39.667 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.667 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.667 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:49:39.667 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.879080+0000", 2024-09-11T07:49:39.667 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8219000000000004e-05 2024-09-11T07:49:39.667 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.667 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.667 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3,6", 2024-09-11T07:49:39.667 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.879160+0000", 2024-09-11T07:49:39.667 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9611999999999996e-05 2024-09-11T07:49:39.667 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.667 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.667 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:49:39.667 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881299+0000", 2024-09-11T07:49:39.668 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0021391800000000001 2024-09-11T07:49:39.668 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.668 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.668 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.668 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881818+0000", 2024-09-11T07:49:39.668 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00051918299999999999 2024-09-11T07:49:39.668 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.668 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.668 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.668 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881961+0000", 2024-09-11T07:49:39.668 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014311799999999999 2024-09-11T07:49:39.668 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.668 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.668 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:49:39.669 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.882254+0000", 2024-09-11T07:49:39.669 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00029237300000000001 2024-09-11T07:49:39.669 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.669 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.669 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:49:39.669 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.882277+0000", 2024-09-11T07:49:39.669 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3643e-05 2024-09-11T07:49:39.669 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:49:39.669 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:49:39.669 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:49:39.669 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.882349+0000", 2024-09-11T07:49:39.669 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1738999999999999e-05 2024-09-11T07:49:39.669 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.669 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:39.670 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.670 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:49:39.670 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:49:39.670 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:49:39.670 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:49:39 smithi022 podman[208335]: 2024-09-11 07:49:39.366740082 +0000 UTC m=+3.928349095 container cleanup f993337d6c883138e24c444f3d19cbaafc74cf83fcd95b5867410a6f12e0f0c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-11T07:49:39.670 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:49:39 smithi022 bash[208335]: ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2 2024-09-11T07:49:39.671 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:49:39 smithi022 podman[208663]: 2024-09-11 07:49:39.610079757 +0000 UTC m=+1.839545686 container remove f993337d6c883138e24c444f3d19cbaafc74cf83fcd95b5867410a6f12e0f0c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-09-11T07:49:39.691 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:39.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:39.692 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:39.793 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:39.814 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:39.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:39.815 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:39.916 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:39.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:39 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:49:39.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:39 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:49:39.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:39 smithi022 ceph-mon[29462]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:49:39.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:39 smithi022 ceph-mon[29462]: from='client.26147 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:49:39.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:39 smithi022 ceph-mon[29462]: osdmap e472: 8 total, 7 up, 5 in 2024-09-11T07:49:39.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:39 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:49:39.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:39 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:49:39.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:39 smithi022 ceph-mon[37330]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:49:39.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:39 smithi022 ceph-mon[37330]: from='client.26147 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:49:39.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:39 smithi022 ceph-mon[37330]: osdmap e472: 8 total, 7 up, 5 in 2024-09-11T07:49:39.938 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:39.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:39.939 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:40.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:39 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:49:40.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:39 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:49:40.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:39 smithi104 ceph-mon[33507]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:49:40.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:39 smithi104 ceph-mon[33507]: from='client.26147 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-11T07:49:40.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:39 smithi104 ceph-mon[33507]: osdmap e472: 8 total, 7 up, 5 in 2024-09-11T07:49:40.040 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:40.061 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:40.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:40.062 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:40.163 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:40.204 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:40.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:40.205 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:40.305 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:40.333 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:40.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:40.334 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:40.435 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:40.445 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-11T07:49:40.465 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:40.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:40.466 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:40.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:49:40 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:49:40.567 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:40.596 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:40.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:40.598 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:40.643 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:49:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:49:40] "GET /metrics HTTP/1.1" 200 209983 "" "Prometheus/2.43.0" 2024-09-11T07:49:40.691 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Removed 119805 objects 2024-09-11T07:49:40.691 INFO:tasks.radosbench.radosbench.0.smithi022.stdout:Clean up completed and total clean up time :118.561 2024-09-11T07:49:40.699 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:40.720 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:40.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:40.721 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:40.821 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:40.845 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:40.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:40.846 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:40.947 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:40.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:40.969 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:40.969 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:41.070 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:41.087 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:49:41.097 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:41.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:41.098 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:41.132 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:49:41.199 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:41.214 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:41 smithi022 ceph-mon[29462]: osdmap e473: 8 total, 7 up, 5 in 2024-09-11T07:49:41.214 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:41 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:49:41.214 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:41 smithi022 ceph-mon[29462]: pgmap v1080: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 2 peering, 2 active+clean+remapped, 1 active+undersized+degraded, 152 active+clean; 108 MiB data, 4.6 GiB used, 442 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 691 op/s; 89/5733 objects degraded (1.552%); 44/5733 objects misplaced (0.767%); 2.8 MiB/s, 44 objects/s recovering 2024-09-11T07:49:41.214 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:41 smithi022 ceph-mon[37330]: osdmap e473: 8 total, 7 up, 5 in 2024-09-11T07:49:41.214 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:41 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:49:41.214 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:41 smithi022 ceph-mon[37330]: pgmap v1080: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 2 peering, 2 active+clean+remapped, 1 active+undersized+degraded, 152 active+clean; 108 MiB data, 4.6 GiB used, 442 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 691 op/s; 89/5733 objects degraded (1.552%); 44/5733 objects misplaced (0.767%); 2.8 MiB/s, 44 objects/s recovering 2024-09-11T07:49:41.220 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:41.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:41.221 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:41.321 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:41.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:41.343 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:41.343 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:41.444 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:41.466 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:41.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:41.467 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:41.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:41 smithi104 ceph-mon[33507]: osdmap e473: 8 total, 7 up, 5 in 2024-09-11T07:49:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:41 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:49:41.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:41 smithi104 ceph-mon[33507]: pgmap v1080: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 2 peering, 2 active+clean+remapped, 1 active+undersized+degraded, 152 active+clean; 108 MiB data, 4.6 GiB used, 442 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 691 op/s; 89/5733 objects degraded (1.552%); 44/5733 objects misplaced (0.767%); 2.8 MiB/s, 44 objects/s recovering 2024-09-11T07:49:41.568 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:41.590 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:41.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:41.591 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:41.692 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:41.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:41.714 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:41.715 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:41.815 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:41.838 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:41.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:41.839 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:41.939 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:41.961 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:41.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:41.962 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:42.063 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:42.086 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:42.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:42.087 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:42.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:42 smithi022 ceph-mon[29462]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:49:42.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:42 smithi022 ceph-mon[29462]: Health check failed: Degraded data redundancy: 89/5733 objects degraded (1.552%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:49:42.166 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:42 smithi022 ceph-mon[37330]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:49:42.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:42 smithi022 ceph-mon[37330]: Health check failed: Degraded data redundancy: 89/5733 objects degraded (1.552%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:49:42.188 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:42.209 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:42.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:42.210 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:42.311 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:42.332 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:42.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:42.333 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:42.386 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-11T07:49:42.434 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:42.455 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:42.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:42.456 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:42.509 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:42 smithi104 ceph-mon[33507]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-11T07:49:42.509 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:42 smithi104 ceph-mon[33507]: Health check failed: Degraded data redundancy: 89/5733 objects degraded (1.552%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:49:42.557 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:42.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:42.580 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:42.580 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:42.681 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:42.702 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:42.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:42.703 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:42.804 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:42.826 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:42.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:42.827 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:42.928 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:42.947 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:49:42.947 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:49:42.949 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:42.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:42.950 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:42.963 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:49:42.963 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:49:42.963 INFO:teuthology.orchestra.run.smithi022.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-11T07:49:42.982 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:49:42.982 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:49:43.003 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:49:43.003 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:49:43.019 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:49:43.019 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:49:43.035 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:49:43.035 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:49:43.050 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:49:43.050 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:49:43.050 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:43.073 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:43.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:43.074 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:43.175 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:43.197 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:43.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:43.198 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:43.299 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:43.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:43.321 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:43.321 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:43.398 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:43 smithi104 ceph-mon[33507]: pgmap v1081: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 2 peering, 2 active+clean+remapped, 1 active+undersized+degraded, 152 active+clean; 37 MiB data, 4.5 GiB used, 443 GiB / 447 GiB avail; 907 B/s rd, 0 B/s wr, 690 op/s; 89/2346 objects degraded (3.794%); 44/2346 objects misplaced (1.876%); 2.0 MiB/s, 31 objects/s recovering 2024-09-11T07:49:43.399 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:43 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:49:43.399 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:43 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:49:43.399 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:43 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/982638450' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:49:43.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:43 smithi022 ceph-mon[29462]: pgmap v1081: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 2 peering, 2 active+clean+remapped, 1 active+undersized+degraded, 152 active+clean; 37 MiB data, 4.5 GiB used, 443 GiB / 447 GiB avail; 907 B/s rd, 0 B/s wr, 690 op/s; 89/2346 objects degraded (3.794%); 44/2346 objects misplaced (1.876%); 2.0 MiB/s, 31 objects/s recovering 2024-09-11T07:49:43.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:43 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:49:43.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:43 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:49:43.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:43 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/982638450' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:49:43.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:43 smithi022 ceph-mon[37330]: pgmap v1081: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 2 peering, 2 active+clean+remapped, 1 active+undersized+degraded, 152 active+clean; 37 MiB data, 4.5 GiB used, 443 GiB / 447 GiB avail; 907 B/s rd, 0 B/s wr, 690 op/s; 89/2346 objects degraded (3.794%); 44/2346 objects misplaced (1.876%); 2.0 MiB/s, 31 objects/s recovering 2024-09-11T07:49:43.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:43 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:49:43.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:43 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:49:43.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:43 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/982638450' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:49:43.422 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:43.442 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:43.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:43.443 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:43.544 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:43.567 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:43.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:43.567 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:43.668 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:43.690 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:43.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:43.691 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:43.792 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:43.813 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:43.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:43.814 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:43.915 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:43.939 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:43.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:43.941 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:44.041 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:44.062 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:44.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:44.063 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:44.164 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:44.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:44 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:49:44.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:44 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:49:44.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:44 smithi022 ceph-mon[29462]: osdmap e474: 8 total, 7 up, 5 in 2024-09-11T07:49:44.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:44 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:49:44.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:44 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:49:44.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:44 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:49:44.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:44 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:49:44.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:44 smithi022 ceph-mon[37330]: osdmap e474: 8 total, 7 up, 5 in 2024-09-11T07:49:44.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:44 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:49:44.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:44 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:49:44.186 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:44.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:44.187 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:44.288 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:44.308 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:44.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:44.309 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:44.410 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:44.432 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:44.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:44.432 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:44.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:49:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:49:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:49:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:44 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:49:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:44 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:49:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:44 smithi104 ceph-mon[33507]: osdmap e474: 8 total, 7 up, 5 in 2024-09-11T07:49:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:44 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:49:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:44 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:49:44.533 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:44.538 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T07:49:44.539 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T07:49:44.554 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:44.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:44.555 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:44.656 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:44.677 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:44.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:44.678 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:44.771 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:49:44.779 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:44.807 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:44.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:44.808 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:44.909 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:44.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:44.931 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:44.931 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:45.032 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:45.054 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:45.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:45.055 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:45.155 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:45.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:45.177 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:45.177 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:45.204 INFO:teuthology.orchestra.run.smithi022.stdout:ERROR: (22) Invalid argument 2024-09-11T07:49:45.204 INFO:teuthology.orchestra.run.smithi022.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T07:49:45.278 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:45.301 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:45.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:45.302 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:45.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:45 smithi022 ceph-mon[29462]: pgmap v1083: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 2 peering, 2 active+clean+remapped, 1 active+undersized+degraded, 152 active+clean; 31 MiB data, 4.4 GiB used, 443 GiB / 447 GiB avail; 853 B/s rd, 0 B/s wr, 664 op/s; 89/2067 objects degraded (4.306%); 44/2067 objects misplaced (2.129%); 1.9 MiB/s, 29 objects/s recovering 2024-09-11T07:49:45.399 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:45 smithi022 ceph-mon[37330]: pgmap v1083: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 2 peering, 2 active+clean+remapped, 1 active+undersized+degraded, 152 active+clean; 31 MiB data, 4.4 GiB used, 443 GiB / 447 GiB avail; 853 B/s rd, 0 B/s wr, 664 op/s; 89/2067 objects degraded (4.306%); 44/2067 objects misplaced (2.129%); 1.9 MiB/s, 29 objects/s recovering 2024-09-11T07:49:45.403 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:45.425 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:45.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:45.426 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:45.526 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:45 smithi104 ceph-mon[33507]: pgmap v1083: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 2 peering, 2 active+clean+remapped, 1 active+undersized+degraded, 152 active+clean; 31 MiB data, 4.4 GiB used, 443 GiB / 447 GiB avail; 853 B/s rd, 0 B/s wr, 664 op/s; 89/2067 objects degraded (4.306%); 44/2067 objects misplaced (2.129%); 1.9 MiB/s, 29 objects/s recovering 2024-09-11T07:49:45.549 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:45.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:45.550 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:45.599 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_4 2024-09-11T07:49:45.599 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd pool rm unique_pool_4 unique_pool_4 --yes-i-really-really-mean-it 2024-09-11T07:49:45.651 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:45.673 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:45.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:45.674 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:45.774 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:45.798 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:45.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:45.799 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:45.900 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:45.922 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:45.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:45.923 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:46.023 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:46.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:46.047 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:46.048 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:46.148 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:46.170 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:46.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:46.171 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:46.272 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:46.293 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:46.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:46.294 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:46.394 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:46.417 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:46.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:46.418 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:46.519 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:46.540 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T07:49:46.541 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:46.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:46.542 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:46.563 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:49:46 smithi022 podman[209512]: 2024-09-11 07:49:46.274292111 +0000 UTC m=+1.450588301 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:49:46.642 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:46.664 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:46.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:46.665 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:46.765 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:46.791 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:46.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:46.792 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:46.803 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-11T07:49:46.893 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:46.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:46.917 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:46.917 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:47.017 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:47.040 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:47.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:47.041 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:47.078 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:46 smithi022 ceph-mon[29462]: pgmap v1084: 158 pgs: 1 active+undersized, 1 active+undersized+remapped+backfill_toofull, 2 active+clean+remapped, 154 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 606 op/s; 1.5 MiB/s, 24 objects/s recovering 2024-09-11T07:49:47.078 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:49:46 smithi022 podman[209512]: 2024-09-11 07:49:46.948547413 +0000 UTC m=+2.124843587 container create 98cb419c90c0bbe8d532c0365df5dcce54eaff8ba8dda8a78f8af288ffdf719b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:49:47.078 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:46 smithi022 ceph-mon[37330]: pgmap v1084: 158 pgs: 1 active+undersized, 1 active+undersized+remapped+backfill_toofull, 2 active+clean+remapped, 154 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 606 op/s; 1.5 MiB/s, 24 objects/s recovering 2024-09-11T07:49:47.142 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:47.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:47.165 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:47.165 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:47.266 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:47.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:46 smithi104 ceph-mon[33507]: pgmap v1084: 158 pgs: 1 active+undersized, 1 active+undersized+remapped+backfill_toofull, 2 active+clean+remapped, 154 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 606 op/s; 1.5 MiB/s, 24 objects/s recovering 2024-09-11T07:49:47.291 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:47.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:47.293 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:47.393 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:47.418 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:47.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:47.420 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:47.520 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:47.521 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:49:47 smithi022 podman[209512]: 2024-09-11 07:49:47.419963623 +0000 UTC m=+2.596259808 container init 98cb419c90c0bbe8d532c0365df5dcce54eaff8ba8dda8a78f8af288ffdf719b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-11T07:49:47.522 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:49:47 smithi022 podman[209512]: 2024-09-11 07:49:47.424058687 +0000 UTC m=+2.600354863 container start 98cb419c90c0bbe8d532c0365df5dcce54eaff8ba8dda8a78f8af288ffdf719b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-11T07:49:47.522 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:49:47 smithi022 podman[209512]: 2024-09-11 07:49:47.520255389 +0000 UTC m=+2.696551641 container attach 98cb419c90c0bbe8d532c0365df5dcce54eaff8ba8dda8a78f8af288ffdf719b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T07:49:47.543 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:47.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:47.544 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:47.645 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:47.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:47.671 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:47.671 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:47.772 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:47.795 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:47.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:47.796 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:47.839 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:49:47.839 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:49:47.898 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:47.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:47.924 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:47.925 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:48.025 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:48.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:48.050 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:48.050 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:48.102 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:49:48.152 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:48.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:47 smithi022 ceph-mon[29462]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 89/2067 objects degraded (4.306%), 2 pgs degraded) 2024-09-11T07:49:48.166 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:47 smithi022 ceph-mon[37330]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 89/2067 objects degraded (4.306%), 2 pgs degraded) 2024-09-11T07:49:48.177 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:48.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:48.178 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:48.279 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:48.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:47 smithi104 ceph-mon[33507]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 89/2067 objects degraded (4.306%), 2 pgs degraded) 2024-09-11T07:49:48.304 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:48.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:48.305 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:48.406 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:48.430 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:48.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:48.431 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:48.531 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:48.556 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:48.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:48.557 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:48.658 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:48.683 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:48.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:48.684 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:48.785 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:48.808 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:48.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:48.809 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:48.838 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is set 2024-09-11T07:49:48.910 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:48.933 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:48.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:48.934 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:49.034 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:49.060 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:49.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:49.061 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:49.162 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:49.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:48 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1206647784' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:49:49.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:48 smithi022 ceph-mon[29462]: from='client.26159 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:49:49.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:48 smithi022 ceph-mon[29462]: pgmap v1085: 158 pgs: 1 active+undersized, 1 active+undersized+remapped+backfill_toofull, 2 active+clean+remapped, 154 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 609 B/s rd, 0 B/s wr, 204 op/s 2024-09-11T07:49:49.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:48 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1206647784' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:49:49.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:48 smithi022 ceph-mon[37330]: from='client.26159 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:49:49.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:48 smithi022 ceph-mon[37330]: pgmap v1085: 158 pgs: 1 active+undersized, 1 active+undersized+remapped+backfill_toofull, 2 active+clean+remapped, 154 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 609 B/s rd, 0 B/s wr, 204 op/s 2024-09-11T07:49:49.187 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:49.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:49.188 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:49.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:48 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1206647784' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:49:49.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:48 smithi104 ceph-mon[33507]: from='client.26159 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:49:49.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:48 smithi104 ceph-mon[33507]: pgmap v1085: 158 pgs: 1 active+undersized, 1 active+undersized+remapped+backfill_toofull, 2 active+clean+remapped, 154 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 609 B/s rd, 0 B/s wr, 204 op/s 2024-09-11T07:49:49.289 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:49.306 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.2.service' 2024-09-11T07:49:49.313 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:49.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:49.314 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:49.373 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:49:49.373 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-11T07:49:49.373 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.2 2024-09-11T07:49:49.373 INFO:tasks.thrashosds.thrasher:remote for osd 2 is ubuntu@smithi022.front.sepia.ceph.com 2024-09-11T07:49:49.374 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -n osd.2 --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --op list-pgs 2024-09-11T07:49:49.414 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:49.439 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:49.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:49.440 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:49.540 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:49.600 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:49.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:49.601 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:49.702 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:49.726 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:49.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:49.727 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:49.828 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:49.855 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:49.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:49.856 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:49.957 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:49.979 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:49.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:49.980 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:50.081 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:50.103 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:50.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:50.104 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:50.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:49 smithi022 ceph-mon[29462]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:49:50.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:49 smithi022 ceph-mon[29462]: from='client.26159 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:49:50.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:49 smithi022 ceph-mon[29462]: osdmap e475: 8 total, 7 up, 5 in 2024-09-11T07:49:50.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:49 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:49:50.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:49 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:49:50.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:49 smithi022 ceph-mon[37330]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:49:50.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:49 smithi022 ceph-mon[37330]: from='client.26159 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:49:50.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:49 smithi022 ceph-mon[37330]: osdmap e475: 8 total, 7 up, 5 in 2024-09-11T07:49:50.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:49 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:49:50.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:49 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:49:50.206 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:50.231 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:50.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:50.232 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:50.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:49 smithi104 ceph-mon[33507]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:49:50.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:49 smithi104 ceph-mon[33507]: from='client.26159 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-11T07:49:50.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:49 smithi104 ceph-mon[33507]: osdmap e475: 8 total, 7 up, 5 in 2024-09-11T07:49:50.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:49 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:49:50.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:49 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:49:50.334 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:50.359 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:50.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:50.360 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:50.460 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:50.484 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:50.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:50.485 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:50.502 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/osd.2/config 2024-09-11T07:49:50.585 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:50.608 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:50.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:50.609 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:50.655 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:49:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:49:50] "GET /metrics HTTP/1.1" 200 209983 "" "Prometheus/2.43.0" 2024-09-11T07:49:50.710 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:50.736 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:50.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:50.737 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:50.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:49:50 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:49:50.838 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:50.861 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:50.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:50.862 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:50.916 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:50 smithi022 ceph-mon[37330]: osdmap e476: 8 total, 7 up, 5 in 2024-09-11T07:49:50.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:50 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:49:50.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:50 smithi022 ceph-mon[37330]: pgmap v1088: 158 pgs: 1 active+undersized, 1 active+undersized+remapped+backfill_toofull, 2 active+clean+remapped, 154 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 80 op/s 2024-09-11T07:49:50.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:50 smithi022 ceph-mon[29462]: osdmap e476: 8 total, 7 up, 5 in 2024-09-11T07:49:50.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:50 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:49:50.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:50 smithi022 ceph-mon[29462]: pgmap v1088: 158 pgs: 1 active+undersized, 1 active+undersized+remapped+backfill_toofull, 2 active+clean+remapped, 154 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 80 op/s 2024-09-11T07:49:50.962 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:50.991 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:50.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:50.992 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:51.093 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:51.117 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:51.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:51.118 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:51.218 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:51.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:51.242 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:51.242 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:51.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:50 smithi104 ceph-mon[33507]: osdmap e476: 8 total, 7 up, 5 in 2024-09-11T07:49:51.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:50 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:49:51.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:50 smithi104 ceph-mon[33507]: pgmap v1088: 158 pgs: 1 active+undersized, 1 active+undersized+remapped+backfill_toofull, 2 active+clean+remapped, 154 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 80 op/s 2024-09-11T07:49:51.343 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:51.367 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:51.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:51.368 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:51.469 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:51.492 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:51.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:51.493 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:51.594 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:51.617 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:51.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:51.618 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:51.718 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:51.745 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:51.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:51.746 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:51.846 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:51.872 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:51.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:51.873 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:51.974 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:51.999 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:51.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:52.000 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:52.101 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:52.126 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:52.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:52.127 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:52.227 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:52.254 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:52.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:52.255 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:52.355 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:52.381 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:52.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:52.382 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:52.483 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:52.507 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:52.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:52.508 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:52.609 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:52.633 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:52.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:52.634 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:52.735 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:52.761 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:52.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:52.762 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:52.863 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:52.888 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:52.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:52.889 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:52.990 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:53.014 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:53.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:53.015 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:53.115 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:53.141 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:53.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:53.142 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:53.243 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:53.244 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:49:53 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:49:53.059+0000 7f0b5d524640 -1 mgr get_metadata_python Requested missing service osd.2 2024-09-11T07:49:53.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:53.269 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:53.269 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:53.370 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:53.393 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:53.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:53.394 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:53.495 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:53.516 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:53.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:53.517 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:53.618 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:53.645 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:53.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:53.646 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:53.747 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:53.770 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:53.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:53.771 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:53.851 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T07:49:53.872 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:53.874 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:53 smithi022 ceph-mon[29462]: pgmap v1089: 158 pgs: 1 active+undersized, 1 active+undersized+remapped+backfill_toofull, 2 active+clean+remapped, 154 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 62 op/s 2024-09-11T07:49:53.874 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:49:53.874 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:49:53.874 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:53 smithi022 ceph-mon[37330]: pgmap v1089: 158 pgs: 1 active+undersized, 1 active+undersized+remapped+backfill_toofull, 2 active+clean+remapped, 154 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 62 op/s 2024-09-11T07:49:53.875 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:49:53.875 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:49:53.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:53.899 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:53.899 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:54.000 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:54.022 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:54.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:54.023 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:54.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:53 smithi104 ceph-mon[33507]: pgmap v1089: 158 pgs: 1 active+undersized, 1 active+undersized+remapped+backfill_toofull, 2 active+clean+remapped, 154 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 62 op/s 2024-09-11T07:49:54.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:49:54.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:49:54.125 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1 2024-09-11T07:49:54.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:54.148 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:54.148 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.1' 2024-09-11T07:49:54.249 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:54.271 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:54.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:54.272 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:54.373 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:54.395 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:54.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:54.396 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:54.497 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:54.519 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:54.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:54.520 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:54.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:49:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:49:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:49:54.621 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:54.643 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:54.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:54.645 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:54.743 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0: {} 2024-09-11T07:49:54.743 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1: {} 2024-09-11T07:49:54.743 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3: {} 2024-09-11T07:49:54.743 INFO:teuthology.orchestra.run.smithi022.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T07:49:54.744 INFO:teuthology.orchestra.run.smithi022.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T07:49:54.744 INFO:teuthology.orchestra.run.smithi022.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-11T07:49:54.744 INFO:teuthology.orchestra.run.smithi022.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T07:49:54.745 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:54.749 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4: {} 2024-09-11T07:49:54.749 INFO:teuthology.orchestra.run.smithi022.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T07:49:54.766 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:54.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:54.767 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:54.769 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5: {} 2024-09-11T07:49:54.769 INFO:teuthology.orchestra.run.smithi022.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T07:49:54.785 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6: {} 2024-09-11T07:49:54.785 INFO:teuthology.orchestra.run.smithi022.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-11T07:49:54.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:54 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1292742129' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:49:54.786 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:54 smithi022 ceph-mon[29462]: from='client.26165 ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:49:54.786 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:54 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1292742129' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:49:54.786 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:54 smithi022 ceph-mon[37330]: from='client.26165 ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:49:54.801 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7: {} 2024-09-11T07:49:54.801 INFO:teuthology.orchestra.run.smithi022.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-11T07:49:54.808 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:54 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1292742129' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:49:54.808 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:54 smithi104 ceph-mon[33507]: from='client.26165 ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T07:49:54.867 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:54.889 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:54.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:54.890 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:54.977 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:49:54.977 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:49:54.977 INFO:teuthology.orchestra.run.smithi022.stdout: "complaint_time": 30, 2024-09-11T07:49:54.977 INFO:teuthology.orchestra.run.smithi022.stdout: "num_blocked_ops": 0 2024-09-11T07:49:54.977 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:49:54.991 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7 2024-09-11T07:49:55.012 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:55.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:55.013 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.7' 2024-09-11T07:49:55.087 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'unique_pool_4' removed 2024-09-11T07:49:55.114 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:55.136 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:55.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:55.137 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:55.237 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:55.258 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:55.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:55.259 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:55.360 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:55.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:55.382 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:55.382 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:55.483 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:55.504 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:55.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:55.505 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:55.605 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5 2024-09-11T07:49:55.633 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:55 smithi022 ceph-mon[29462]: pgmap v1090: 158 pgs: 1 active+undersized, 1 active+undersized+remapped+backfill_toofull, 2 active+clean+remapped, 154 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:49:55.633 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:55 smithi022 ceph-mon[29462]: from='client.26165 ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T07:49:55.633 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:55 smithi022 ceph-mon[29462]: osdmap e477: 8 total, 7 up, 5 in 2024-09-11T07:49:55.633 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:55 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3184308429' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:49:55.633 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:55 smithi022 ceph-mon[37330]: pgmap v1090: 158 pgs: 1 active+undersized, 1 active+undersized+remapped+backfill_toofull, 2 active+clean+remapped, 154 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:49:55.633 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:55 smithi022 ceph-mon[37330]: from='client.26165 ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T07:49:55.633 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:55 smithi022 ceph-mon[37330]: osdmap e477: 8 total, 7 up, 5 in 2024-09-11T07:49:55.633 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:55 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3184308429' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:49:55.634 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:55.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:55.635 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.5' 2024-09-11T07:49:55.735 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:55.756 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:55.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:55.757 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:55.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:55 smithi104 ceph-mon[33507]: pgmap v1090: 158 pgs: 1 active+undersized, 1 active+undersized+remapped+backfill_toofull, 2 active+clean+remapped, 154 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:49:55.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:55 smithi104 ceph-mon[33507]: from='client.26165 ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T07:49:55.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:55 smithi104 ceph-mon[33507]: osdmap e477: 8 total, 7 up, 5 in 2024-09-11T07:49:55.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:55 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3184308429' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-11T07:49:55.858 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:55.880 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:55.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:55.881 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:55.981 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3 2024-09-11T07:49:56.010 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:56.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:56.011 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.3' 2024-09-11T07:49:56.112 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4 2024-09-11T07:49:56.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:56.134 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:56.135 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.4' 2024-09-11T07:49:56.236 DEBUG:teuthology.orchestra.run.smithi022:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0 2024-09-11T07:49:56.264 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: docker: command not found 2024-09-11T07:49:56.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:56.265 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi022 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.0' 2024-09-11T07:49:56.301 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T07:49:56.301 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T07:49:56.366 DEBUG:teuthology.orchestra.run.smithi104:> sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6 2024-09-11T07:49:56.388 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: docker: command not found 2024-09-11T07:49:56.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:49:56.389 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi104 with status 1: 'sudo docker kill -s 1 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd.6' 2024-09-11T07:49:56.396 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-11T07:49:56.410 INFO:tasks.thrashosds:joining thrashosds 2024-09-11T07:49:57.117 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:56 smithi022 ceph-mon[29462]: pgmap v1092: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:49:57.117 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:56 smithi022 ceph-mon[37330]: pgmap v1092: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:49:57.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:56 smithi104 ceph-mon[33507]: pgmap v1092: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:49:58.131 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:57 smithi022 ceph-mon[29462]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-11T07:49:58.131 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:57 smithi022 ceph-mon[37330]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-11T07:49:58.162 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-11T07:49:58.193 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:49:58.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:57 smithi104 ceph-mon[33507]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-11T07:49:59.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:49:58 smithi022 ceph-mon[29462]: pgmap v1093: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 474 B/s rd, 0 op/s 2024-09-11T07:49:59.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:49:58 smithi022 ceph-mon[37330]: pgmap v1093: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 474 B/s rd, 0 op/s 2024-09-11T07:49:59.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:49:58 smithi104 ceph-mon[33507]: pgmap v1093: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 474 B/s rd, 0 op/s 2024-09-11T07:50:00.372 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:00 smithi022 ceph-mon[29462]: Health detail: HEALTH_WARN noscrub,nodeep-scrub flag(s) set 2024-09-11T07:50:00.372 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:00 smithi022 ceph-mon[29462]: [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set 2024-09-11T07:50:00.372 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:00 smithi022 ceph-mon[37330]: Health detail: HEALTH_WARN noscrub,nodeep-scrub flag(s) set 2024-09-11T07:50:00.373 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:00 smithi022 ceph-mon[37330]: [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set 2024-09-11T07:50:00.390 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:00 smithi104 ceph-mon[33507]: Health detail: HEALTH_WARN noscrub,nodeep-scrub flag(s) set 2024-09-11T07:50:00.390 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:00 smithi104 ceph-mon[33507]: [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set 2024-09-11T07:50:00.663 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:50:00 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:50:00] "GET /metrics HTTP/1.1" 200 209820 "" "Prometheus/2.43.0" 2024-09-11T07:50:00.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:50:00 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:50:01.481 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:01 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:50:01.481 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:01 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:50:01.482 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:01 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:50:01.482 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:01 smithi022 ceph-mon[29462]: pgmap v1094: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 921 B/s rd, 0 op/s 2024-09-11T07:50:01.482 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:01 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:50:01.482 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:01 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:50:01.482 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:01 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:50:01.482 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:01 smithi022 ceph-mon[37330]: pgmap v1094: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 921 B/s rd, 0 op/s 2024-09-11T07:50:01.495 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:50:01.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:01 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:50:01.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:01 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:50:01.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:01 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:50:01.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:01 smithi104 ceph-mon[33507]: pgmap v1094: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 921 B/s rd, 0 op/s 2024-09-11T07:50:02.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:02 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:50:02.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:02 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:50:02.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:02 smithi022 ceph-mon[29462]: pgmap v1095: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1004 B/s rd, 0 op/s 2024-09-11T07:50:02.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:02 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:50:02.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:02 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3749742002' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:50:02.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:02 smithi022 ceph-mon[29462]: from='client.26177 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:50:02.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:02 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:50:02.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:02 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:50:02.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:02 smithi022 ceph-mon[37330]: pgmap v1095: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1004 B/s rd, 0 op/s 2024-09-11T07:50:02.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:02 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:50:02.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:02 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3749742002' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:50:02.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:02 smithi022 ceph-mon[37330]: from='client.26177 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:50:02.659 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is unset 2024-09-11T07:50:02.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:02 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:50:02.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:02 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:50:02.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:02 smithi104 ceph-mon[33507]: pgmap v1095: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1004 B/s rd, 0 op/s 2024-09-11T07:50:02.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:02 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:50:02.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:02 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3749742002' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:50:02.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:02 smithi104 ceph-mon[33507]: from='client.26177 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:50:03.366 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:03 smithi022 ceph-mon[29462]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-11T07:50:03.367 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:03 smithi022 ceph-mon[29462]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:50:03.367 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:03 smithi022 ceph-mon[29462]: from='client.26177 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:50:03.367 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:03 smithi022 ceph-mon[29462]: osdmap e478: 8 total, 7 up, 5 in 2024-09-11T07:50:03.367 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:03 smithi022 ceph-mon[37330]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-11T07:50:03.367 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:03 smithi022 ceph-mon[37330]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:50:03.367 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:03 smithi022 ceph-mon[37330]: from='client.26177 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:50:03.367 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:03 smithi022 ceph-mon[37330]: osdmap e478: 8 total, 7 up, 5 in 2024-09-11T07:50:03.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:03 smithi104 ceph-mon[33507]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-11T07:50:03.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:03 smithi104 ceph-mon[33507]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:50:03.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:03 smithi104 ceph-mon[33507]: from='client.26177 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:50:03.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:03 smithi104 ceph-mon[33507]: osdmap e478: 8 total, 7 up, 5 in 2024-09-11T07:50:03.985 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:50:03.985 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 20, 2024-09-11T07:50:03.986 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 600, 2024-09-11T07:50:03.986 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:50:03.986 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.986 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:192470 11.1 11:8a8c1514:::benchmark_data_smithi022_188626_object72662:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e452)", 2024-09-11T07:50:03.986 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:48:46.099384+0000", 2024-09-11T07:50:03.986 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 77.361157419999998, 2024-09-11T07:50:03.986 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.264914584, 2024-09-11T07:50:03.986 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:03.986 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:03.986 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:03.986 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:03.986 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:03.986 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 192470 2024-09-11T07:50:03.987 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.987 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:03.987 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.987 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:03.987 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.099384+0000", 2024-09-11T07:50:03.987 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:03.987 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.987 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.987 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:03.987 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.099384+0000", 2024-09-11T07:50:03.987 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:03.987 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.988 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.988 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:03.988 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.099383+0000", 2024-09-11T07:50:03.988 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:03.988 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.988 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.988 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:03.988 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.099388+0000", 2024-09-11T07:50:03.988 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.5369999999999997e-06 2024-09-11T07:50:03.988 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.988 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.988 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:03.989 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.099398+0000", 2024-09-11T07:50:03.989 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0212e-05 2024-09-11T07:50:03.989 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.989 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.989 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:03.989 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.099404+0000", 2024-09-11T07:50:03.989 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2089999999999996e-06 2024-09-11T07:50:03.989 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.989 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.990 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:03.990 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.213284+0000", 2024-09-11T07:50:03.990 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.1138807 2024-09-11T07:50:03.990 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.990 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.990 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:03.990 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.213293+0000", 2024-09-11T07:50:03.990 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.7029999999999995e-06 2024-09-11T07:50:03.990 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.990 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.990 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:03.990 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.161847+0000", 2024-09-11T07:50:03.990 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.948553541 2024-09-11T07:50:03.990 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.991 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.991 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:03.991 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.161857+0000", 2024-09-11T07:50:03.991 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0747000000000001e-05 2024-09-11T07:50:03.991 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.991 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.991 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:03.991 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.318400+0000", 2024-09-11T07:50:03.991 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.156542594 2024-09-11T07:50:03.991 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.991 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.991 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:03.992 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.318519+0000", 2024-09-11T07:50:03.992 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000119348 2024-09-11T07:50:03.992 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.992 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.992 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,2,6", 2024-09-11T07:50:03.992 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.318940+0000", 2024-09-11T07:50:03.992 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00042059299999999998 2024-09-11T07:50:03.992 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.992 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.992 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:03.992 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.334291+0000", 2024-09-11T07:50:03.992 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.015351340999999999 2024-09-11T07:50:03.992 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.993 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.993 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:03.993 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.334387+0000", 2024-09-11T07:50:03.993 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.5329999999999997e-05 2024-09-11T07:50:03.993 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.993 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.993 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:03.993 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.343666+0000", 2024-09-11T07:50:03.993 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0092792150000000004 2024-09-11T07:50:03.993 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.993 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.993 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:03.993 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.360689+0000", 2024-09-11T07:50:03.993 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.017023134999999998 2024-09-11T07:50:03.994 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.994 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.994 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:03.994 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.364144+0000", 2024-09-11T07:50:03.994 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.003454974 2024-09-11T07:50:03.994 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.994 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.994 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:03.994 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.364299+0000", 2024-09-11T07:50:03.994 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000154716 2024-09-11T07:50:03.994 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:03.994 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:03.994 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:03.995 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.995 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.995 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:192473 11.1 11:8888764e:::benchmark_data_smithi022_188626_object72665:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e452)", 2024-09-11T07:50:03.995 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:48:46.101497+0000", 2024-09-11T07:50:03.995 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 77.359044061000006, 2024-09-11T07:50:03.995 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2629777190000002, 2024-09-11T07:50:03.995 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:03.995 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:03.995 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:03.995 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:03.995 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:03.995 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 192473 2024-09-11T07:50:03.995 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.995 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:03.995 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.996 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:03.996 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.101497+0000", 2024-09-11T07:50:03.996 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:03.996 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.996 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.996 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:03.996 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.101497+0000", 2024-09-11T07:50:03.996 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:03.996 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.996 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.996 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:03.996 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.101495+0000", 2024-09-11T07:50:03.996 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:50:03.997 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.997 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.997 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:03.997 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.101502+0000", 2024-09-11T07:50:03.997 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9369999999999998e-06 2024-09-11T07:50:03.997 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.997 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.997 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:03.997 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.101518+0000", 2024-09-11T07:50:03.997 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5446000000000001e-05 2024-09-11T07:50:03.997 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.997 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.997 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:03.998 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.101524+0000", 2024-09-11T07:50:03.998 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1539999999999999e-06 2024-09-11T07:50:03.998 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.998 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.998 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:03.998 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.214283+0000", 2024-09-11T07:50:03.998 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.112759375 2024-09-11T07:50:03.998 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.998 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.998 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:03.998 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.214288+0000", 2024-09-11T07:50:03.998 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1189999999999996e-06 2024-09-11T07:50:03.998 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.999 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.999 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:03.999 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.163244+0000", 2024-09-11T07:50:03.999 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.948956087 2024-09-11T07:50:03.999 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.999 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.999 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:03.999 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.163253+0000", 2024-09-11T07:50:03.999 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.4600000000000003e-06 2024-09-11T07:50:03.999 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:03.999 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:03.999 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:03.999 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.334411+0000", 2024-09-11T07:50:03.999 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1711577529999999 2024-09-11T07:50:04.000 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.000 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.000 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:04.000 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.334520+0000", 2024-09-11T07:50:04.000 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000109824 2024-09-11T07:50:04.000 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.000 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.000 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,2,6", 2024-09-11T07:50:04.000 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.334637+0000", 2024-09-11T07:50:04.000 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000117003 2024-09-11T07:50:04.000 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.000 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.001 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.001 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.343698+0000", 2024-09-11T07:50:04.001 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0090602809999999999 2024-09-11T07:50:04.001 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.001 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.001 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.001 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.343735+0000", 2024-09-11T07:50:04.001 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7129000000000003e-05 2024-09-11T07:50:04.001 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.001 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.001 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.001 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.343766+0000", 2024-09-11T07:50:04.001 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.1562e-05 2024-09-11T07:50:04.001 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.002 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.002 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:04.002 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.364308+0000", 2024-09-11T07:50:04.002 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.020541441000000001 2024-09-11T07:50:04.002 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.002 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.002 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:04.002 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.364376+0000", 2024-09-11T07:50:04.002 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.8034000000000006e-05 2024-09-11T07:50:04.002 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.002 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.002 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:04.002 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.364475+0000", 2024-09-11T07:50:04.003 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.9207000000000001e-05 2024-09-11T07:50:04.003 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.003 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:04.003 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.003 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.003 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.003 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:192485 11.8 11:187d983e:::benchmark_data_smithi022_188626_object72677:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e452)", 2024-09-11T07:50:04.003 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:48:46.112039+0000", 2024-09-11T07:50:04.003 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 77.348502144999998, 2024-09-11T07:50:04.003 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2148457459999999, 2024-09-11T07:50:04.003 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:04.003 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:50:04.003 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:04.004 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:04.004 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:04.004 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 192485 2024-09-11T07:50:04.004 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.004 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:04.004 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.004 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:04.004 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.112039+0000", 2024-09-11T07:50:04.004 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.004 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.004 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.004 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:04.004 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.112039+0000", 2024-09-11T07:50:04.005 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.005 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.005 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.005 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:04.005 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.112037+0000", 2024-09-11T07:50:04.005 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:50:04.005 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.005 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.005 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:04.005 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.112045+0000", 2024-09-11T07:50:04.005 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5320000000000001e-06 2024-09-11T07:50:04.005 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.005 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.006 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:04.006 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.112060+0000", 2024-09-11T07:50:04.006 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5653000000000001e-05 2024-09-11T07:50:04.006 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.006 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.006 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:04.006 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.112066+0000", 2024-09-11T07:50:04.006 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0469999999999999e-06 2024-09-11T07:50:04.006 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.006 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.006 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.006 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.112131+0000", 2024-09-11T07:50:04.006 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4508e-05 2024-09-11T07:50:04.007 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.007 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.007 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:04.007 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.112237+0000", 2024-09-11T07:50:04.007 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010589 2024-09-11T07:50:04.007 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.007 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.007 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:50:04.007 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.112342+0000", 2024-09-11T07:50:04.007 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010501300000000001 2024-09-11T07:50:04.007 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.007 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.007 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.008 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.118411+0000", 2024-09-11T07:50:04.008 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0060687329999999998 2024-09-11T07:50:04.008 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.008 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.008 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:04.008 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.122716+0000", 2024-09-11T07:50:04.008 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0043050290000000001 2024-09-11T07:50:04.008 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.008 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.008 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.008 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.172822+0000", 2024-09-11T07:50:04.008 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.050106612000000002 2024-09-11T07:50:04.008 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.008 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.009 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:04.009 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.172831+0000", 2024-09-11T07:50:04.009 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.1379999999999996e-06 2024-09-11T07:50:04.009 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.009 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.009 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.009 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.146698+0000", 2024-09-11T07:50:04.009 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.97386635899999996 2024-09-11T07:50:04.009 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.009 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.009 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:04.009 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.146710+0000", 2024-09-11T07:50:04.009 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2061e-05 2024-09-11T07:50:04.010 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.010 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.010 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.010 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.323175+0000", 2024-09-11T07:50:04.010 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.176464763 2024-09-11T07:50:04.010 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.010 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.010 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:04.010 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.326885+0000", 2024-09-11T07:50:04.010 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0037105200000000001 2024-09-11T07:50:04.010 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.010 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:04.010 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.010 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.011 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.011 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:192488 11.5 11:a8be7972:::benchmark_data_smithi022_188626_object72680:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e452)", 2024-09-11T07:50:04.011 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:48:46.113837+0000", 2024-09-11T07:50:04.011 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 77.346704740000007, 2024-09-11T07:50:04.011 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.085193919, 2024-09-11T07:50:04.011 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:04.011 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:50:04.011 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:04.011 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:04.011 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:04.011 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 192488 2024-09-11T07:50:04.011 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.011 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:04.011 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.012 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:04.012 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.113837+0000", 2024-09-11T07:50:04.012 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.012 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.012 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.012 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:04.012 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.113837+0000", 2024-09-11T07:50:04.012 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.012 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.012 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.012 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:04.012 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.113835+0000", 2024-09-11T07:50:04.012 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:50:04.013 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.013 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.013 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:04.013 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.113842+0000", 2024-09-11T07:50:04.013 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7009999999999999e-06 2024-09-11T07:50:04.013 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.013 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.013 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:04.013 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.113852+0000", 2024-09-11T07:50:04.014 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0584999999999999e-05 2024-09-11T07:50:04.014 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.014 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.014 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:04.014 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.113858+0000", 2024-09-11T07:50:04.014 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.463e-06 2024-09-11T07:50:04.014 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.014 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.014 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.014 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.113900+0000", 2024-09-11T07:50:04.014 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2327999999999999e-05 2024-09-11T07:50:04.014 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.014 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.014 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:04.015 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.114004+0000", 2024-09-11T07:50:04.015 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000104252 2024-09-11T07:50:04.015 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.015 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.015 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:50:04.015 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.114249+0000", 2024-09-11T07:50:04.015 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00024434700000000001 2024-09-11T07:50:04.015 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.015 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.015 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.016 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.120276+0000", 2024-09-11T07:50:04.016 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0060276089999999997 2024-09-11T07:50:04.016 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.016 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.016 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:04.016 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.122692+0000", 2024-09-11T07:50:04.016 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0024154300000000001 2024-09-11T07:50:04.016 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.016 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.016 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.016 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.175467+0000", 2024-09-11T07:50:04.017 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.052774988000000002 2024-09-11T07:50:04.017 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.017 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.017 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:04.017 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.175477+0000", 2024-09-11T07:50:04.017 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.9729999999999992e-06 2024-09-11T07:50:04.017 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.017 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.017 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.017 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.150714+0000", 2024-09-11T07:50:04.017 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.97523706899999996 2024-09-11T07:50:04.017 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.017 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.018 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:04.018 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.150734+0000", 2024-09-11T07:50:04.018 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.9894e-05 2024-09-11T07:50:04.018 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.018 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.018 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.018 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.198971+0000", 2024-09-11T07:50:04.018 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0482375500000001 2024-09-11T07:50:04.018 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.018 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.018 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:04.018 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.199031+0000", 2024-09-11T07:50:04.018 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9435999999999998e-05 2024-09-11T07:50:04.019 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.019 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:04.019 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.019 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.019 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.019 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:192493 11.1 11:8eb5b5fb:::benchmark_data_smithi022_188626_object72685:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e452)", 2024-09-11T07:50:04.019 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:48:46.124558+0000", 2024-09-11T07:50:04.019 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 77.335983632999998, 2024-09-11T07:50:04.019 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.240947641, 2024-09-11T07:50:04.019 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:04.019 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:04.019 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:04.019 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:04.020 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:04.020 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 192493 2024-09-11T07:50:04.020 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.020 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:04.020 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.020 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:04.020 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.124558+0000", 2024-09-11T07:50:04.020 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.020 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.020 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.020 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:04.020 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.124558+0000", 2024-09-11T07:50:04.020 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.021 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.021 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.021 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:04.021 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.124555+0000", 2024-09-11T07:50:04.021 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:50:04.021 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.021 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.021 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:04.021 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.124564+0000", 2024-09-11T07:50:04.021 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.4959999999999996e-06 2024-09-11T07:50:04.021 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.021 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.021 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:04.021 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.124581+0000", 2024-09-11T07:50:04.022 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.6867999999999998e-05 2024-09-11T07:50:04.022 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.022 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.022 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:04.022 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.124588+0000", 2024-09-11T07:50:04.022 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.6750000000000002e-06 2024-09-11T07:50:04.022 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.022 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.022 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.022 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.214302+0000", 2024-09-11T07:50:04.022 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.089713487999999994 2024-09-11T07:50:04.022 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.022 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.023 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:04.023 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.214305+0000", 2024-09-11T07:50:04.023 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3409999999999999e-06 2024-09-11T07:50:04.023 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.023 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.023 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.023 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.163633+0000", 2024-09-11T07:50:04.023 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.94932752399999998 2024-09-11T07:50:04.023 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.023 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.023 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:04.023 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.163642+0000", 2024-09-11T07:50:04.023 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.4839999999999993e-06 2024-09-11T07:50:04.023 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.024 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.024 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.024 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.337866+0000", 2024-09-11T07:50:04.024 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.174223655 2024-09-11T07:50:04.024 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.024 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.024 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:04.024 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.340443+0000", 2024-09-11T07:50:04.024 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0025771489999999999 2024-09-11T07:50:04.024 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.024 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.024 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,2,6", 2024-09-11T07:50:04.024 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.340580+0000", 2024-09-11T07:50:04.025 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000136427 2024-09-11T07:50:04.025 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.025 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.025 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.025 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.343796+0000", 2024-09-11T07:50:04.025 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0032165969999999999 2024-09-11T07:50:04.025 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.025 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.025 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.025 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.343826+0000", 2024-09-11T07:50:04.025 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.9943000000000001e-05 2024-09-11T07:50:04.025 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.025 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.025 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.026 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.348476+0000", 2024-09-11T07:50:04.026 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0046496380000000002 2024-09-11T07:50:04.026 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.026 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.026 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:04.026 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.364690+0000", 2024-09-11T07:50:04.026 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.016214480999999999 2024-09-11T07:50:04.026 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.026 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.026 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:04.026 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.365407+0000", 2024-09-11T07:50:04.026 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00071681100000000003 2024-09-11T07:50:04.026 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.026 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.027 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:04.027 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.365505+0000", 2024-09-11T07:50:04.027 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.8482999999999999e-05 2024-09-11T07:50:04.027 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.027 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:04.027 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.027 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.027 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.027 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:192495 11.6 11:6538b30a:::benchmark_data_smithi022_188626_object72687:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e452)", 2024-09-11T07:50:04.027 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:48:46.124595+0000", 2024-09-11T07:50:04.027 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 77.335946347999993, 2024-09-11T07:50:04.027 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2375358099999998, 2024-09-11T07:50:04.027 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:04.028 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:50:04.028 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:04.028 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:04.028 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:04.028 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 192495 2024-09-11T07:50:04.028 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.028 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:04.028 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.028 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:04.028 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.124595+0000", 2024-09-11T07:50:04.028 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.028 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.028 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.029 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:04.029 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.124595+0000", 2024-09-11T07:50:04.029 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.029 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.029 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.029 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:04.029 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.124594+0000", 2024-09-11T07:50:04.029 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:50:04.029 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.029 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.029 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:04.029 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.124598+0000", 2024-09-11T07:50:04.029 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8999999999999999e-06 2024-09-11T07:50:04.030 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.030 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.030 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:04.030 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.124604+0000", 2024-09-11T07:50:04.030 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3879999999999999e-06 2024-09-11T07:50:04.030 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.030 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.030 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:04.030 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.124608+0000", 2024-09-11T07:50:04.030 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.963e-06 2024-09-11T07:50:04.030 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.030 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.030 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.031 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.127053+0000", 2024-09-11T07:50:04.031 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0024452940000000002 2024-09-11T07:50:04.031 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.031 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.031 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:04.031 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.127175+0000", 2024-09-11T07:50:04.031 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012184999999999999 2024-09-11T07:50:04.031 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.031 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.031 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:50:04.031 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.127287+0000", 2024-09-11T07:50:04.031 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000112011 2024-09-11T07:50:04.031 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.032 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.032 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:04.032 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.158693+0000", 2024-09-11T07:50:04.032 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.031406296 2024-09-11T07:50:04.032 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.032 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.032 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.032 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.169482+0000", 2024-09-11T07:50:04.032 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.010788895999999999 2024-09-11T07:50:04.032 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.032 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.032 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:04.032 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.169494+0000", 2024-09-11T07:50:04.033 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2417e-05 2024-09-11T07:50:04.033 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.033 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.033 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.033 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.149405+0000", 2024-09-11T07:50:04.033 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.97991054700000002 2024-09-11T07:50:04.033 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.033 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.033 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:04.033 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.149415+0000", 2024-09-11T07:50:04.033 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0142e-05 2024-09-11T07:50:04.033 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.033 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.033 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.034 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.343994+0000", 2024-09-11T07:50:04.034 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1945791240000001 2024-09-11T07:50:04.034 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.034 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.034 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for missing object", 2024-09-11T07:50:04.034 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.344106+0000", 2024-09-11T07:50:04.034 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000111877 2024-09-11T07:50:04.034 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.034 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.034 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.034 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.362026+0000", 2024-09-11T07:50:04.034 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.017920078999999998 2024-09-11T07:50:04.034 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.035 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.035 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:04.035 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.362131+0000", 2024-09-11T07:50:04.035 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010478499999999999 2024-09-11T07:50:04.035 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.035 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:04.035 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.035 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.035 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.035 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24353.0:3075 5.e 5:7759931f:::notify.3:head [watch reconnect cookie 94395805751360 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e453)", 2024-09-11T07:50:04.035 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:48:46.126904+0000", 2024-09-11T07:50:04.035 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 77.333637553000003, 2024-09-11T07:50:04.035 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.062417462, 2024-09-11T07:50:04.036 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:04.036 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:50:04.036 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:04.036 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24353", 2024-09-11T07:50:04.036 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1860969011", 2024-09-11T07:50:04.036 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 3075 2024-09-11T07:50:04.036 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.036 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:04.036 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.036 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:04.036 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.126904+0000", 2024-09-11T07:50:04.036 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.036 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.037 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.037 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:04.037 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.126904+0000", 2024-09-11T07:50:04.037 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.037 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.037 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.037 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:04.037 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.126901+0000", 2024-09-11T07:50:04.037 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:50:04.037 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.037 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.037 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:04.037 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.126911+0000", 2024-09-11T07:50:04.037 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.3519999999999996e-06 2024-09-11T07:50:04.038 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.038 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.038 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:04.038 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.126926+0000", 2024-09-11T07:50:04.038 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5155e-05 2024-09-11T07:50:04.038 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.038 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.038 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:04.038 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.126933+0000", 2024-09-11T07:50:04.038 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.4490000000000002e-06 2024-09-11T07:50:04.038 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.038 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.038 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.039 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.129797+0000", 2024-09-11T07:50:04.039 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0028636379999999999 2024-09-11T07:50:04.039 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.039 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.039 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op must wait for map", 2024-09-11T07:50:04.039 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.129803+0000", 2024-09-11T07:50:04.039 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.348e-06 2024-09-11T07:50:04.039 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.039 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.039 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.039 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.172870+0000", 2024-09-11T07:50:04.039 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.043066711000000001 2024-09-11T07:50:04.039 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.039 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.040 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:04.040 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.172879+0000", 2024-09-11T07:50:04.040 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.0890000000000006e-06 2024-09-11T07:50:04.040 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.040 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.040 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.040 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.175653+0000", 2024-09-11T07:50:04.040 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0027742749999999 2024-09-11T07:50:04.040 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.040 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.040 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for degraded object", 2024-09-11T07:50:04.040 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.175940+0000", 2024-09-11T07:50:04.040 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00028707400000000001 2024-09-11T07:50:04.041 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.041 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.041 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.041 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.184256+0000", 2024-09-11T07:50:04.041 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0083157920000000007 2024-09-11T07:50:04.041 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.041 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.041 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:04.041 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.184335+0000", 2024-09-11T07:50:04.041 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.8978000000000002e-05 2024-09-11T07:50:04.041 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.041 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.041 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:04.041 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.189321+0000", 2024-09-11T07:50:04.042 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0049861819999999996 2024-09-11T07:50:04.042 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.042 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:04.042 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.042 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.042 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.042 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:192496 11.6 11:6da4c73b:::benchmark_data_smithi022_188626_object72688:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e452)", 2024-09-11T07:50:04.042 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:48:46.130633+0000", 2024-09-11T07:50:04.042 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 77.329908568999997, 2024-09-11T07:50:04.042 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2449181610000002, 2024-09-11T07:50:04.042 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:04.042 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:50:04.042 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:04.042 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:04.042 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:04.043 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 192496 2024-09-11T07:50:04.043 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.043 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:04.043 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.043 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:04.043 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.130633+0000", 2024-09-11T07:50:04.043 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.043 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.043 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.043 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:04.043 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.130633+0000", 2024-09-11T07:50:04.043 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.043 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.044 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.044 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:04.044 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.130631+0000", 2024-09-11T07:50:04.044 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:50:04.044 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.044 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.044 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:04.044 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.130639+0000", 2024-09-11T07:50:04.044 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.593e-06 2024-09-11T07:50:04.044 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.044 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.044 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:04.044 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.130649+0000", 2024-09-11T07:50:04.045 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0559e-05 2024-09-11T07:50:04.045 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.045 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.045 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:04.045 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.130655+0000", 2024-09-11T07:50:04.045 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7119999999999997e-06 2024-09-11T07:50:04.045 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.045 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.045 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.045 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.130740+0000", 2024-09-11T07:50:04.045 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.4926000000000006e-05 2024-09-11T07:50:04.045 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.045 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.045 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:04.046 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.130866+0000", 2024-09-11T07:50:04.046 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000126372 2024-09-11T07:50:04.046 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.046 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.046 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:50:04.046 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.130980+0000", 2024-09-11T07:50:04.046 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000113577 2024-09-11T07:50:04.046 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.046 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.046 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:04.046 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.158751+0000", 2024-09-11T07:50:04.046 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.027771104000000001 2024-09-11T07:50:04.046 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.046 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.047 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.047 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.169526+0000", 2024-09-11T07:50:04.047 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.010775355 2024-09-11T07:50:04.047 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.047 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.047 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:04.047 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.169536+0000", 2024-09-11T07:50:04.047 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.8889999999999997e-06 2024-09-11T07:50:04.047 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.047 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.047 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.047 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.149453+0000", 2024-09-11T07:50:04.047 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.97991705200000001 2024-09-11T07:50:04.048 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.048 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.048 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:04.048 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.149460+0000", 2024-09-11T07:50:04.048 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7950000000000002e-06 2024-09-11T07:50:04.048 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.048 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.048 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.048 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.344125+0000", 2024-09-11T07:50:04.048 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1946647399999999 2024-09-11T07:50:04.048 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.048 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.048 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for missing object", 2024-09-11T07:50:04.049 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.344205+0000", 2024-09-11T07:50:04.049 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.0296999999999994e-05 2024-09-11T07:50:04.049 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.049 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.049 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.049 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.375476+0000", 2024-09-11T07:50:04.049 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.031270583999999997 2024-09-11T07:50:04.049 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.049 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.049 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:04.049 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.375551+0000", 2024-09-11T07:50:04.049 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5500999999999994e-05 2024-09-11T07:50:04.049 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.050 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:04.050 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.050 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.050 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.050 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:192482 11.2 11:4a52ff94:::benchmark_data_smithi022_188626_object72674:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e453)", 2024-09-11T07:50:04.050 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:48:46.148994+0000", 2024-09-11T07:50:04.050 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 77.311547469999994, 2024-09-11T07:50:04.050 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.195384856, 2024-09-11T07:50:04.050 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:04.050 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:50:04.050 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:04.050 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:04.050 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:04.050 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 192482 2024-09-11T07:50:04.051 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.051 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:04.051 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.051 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:04.051 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.148994+0000", 2024-09-11T07:50:04.051 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.051 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.051 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.051 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:04.051 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.148994+0000", 2024-09-11T07:50:04.051 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.051 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.051 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.052 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:04.052 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.148992+0000", 2024-09-11T07:50:04.052 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:04.052 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.052 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.052 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:04.052 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.148998+0000", 2024-09-11T07:50:04.052 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3360000000000002e-06 2024-09-11T07:50:04.052 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.052 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.052 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:04.052 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.149006+0000", 2024-09-11T07:50:04.052 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.6170000000000007e-06 2024-09-11T07:50:04.053 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.053 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.053 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:04.053 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.149011+0000", 2024-09-11T07:50:04.053 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2540000000000003e-06 2024-09-11T07:50:04.053 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.053 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.053 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.053 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.173480+0000", 2024-09-11T07:50:04.053 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.024469597999999999 2024-09-11T07:50:04.053 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.053 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.053 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:04.054 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.173488+0000", 2024-09-11T07:50:04.054 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.2640000000000008e-06 2024-09-11T07:50:04.054 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.054 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.054 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.054 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.150362+0000", 2024-09-11T07:50:04.054 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.97687383999999999 2024-09-11T07:50:04.054 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.054 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.054 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:04.054 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.150371+0000", 2024-09-11T07:50:04.054 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.8710000000000003e-06 2024-09-11T07:50:04.054 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.055 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.055 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.055 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.341605+0000", 2024-09-11T07:50:04.055 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1912335860000001 2024-09-11T07:50:04.055 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.055 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.055 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:04.055 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.344379+0000", 2024-09-11T07:50:04.055 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0027740709999999999 2024-09-11T07:50:04.055 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.055 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:04.055 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.055 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.055 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.056 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:192497 11.2 11:4a836668:::benchmark_data_smithi022_188626_object72689:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e453)", 2024-09-11T07:50:04.056 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:48:46.149014+0000", 2024-09-11T07:50:04.056 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 77.311527081999998, 2024-09-11T07:50:04.056 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.2308739360000001, 2024-09-11T07:50:04.056 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:04.056 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:04.056 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:04.056 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:04.056 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:04.056 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 192497 2024-09-11T07:50:04.056 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.056 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:04.056 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.057 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:04.057 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.149014+0000", 2024-09-11T07:50:04.057 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.057 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.057 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.057 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:04.057 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.149014+0000", 2024-09-11T07:50:04.057 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.057 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.057 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.057 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:04.057 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.149014+0000", 2024-09-11T07:50:04.057 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:50:04.058 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.058 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.058 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:04.058 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.149016+0000", 2024-09-11T07:50:04.058 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.351e-06 2024-09-11T07:50:04.058 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.058 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.058 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:04.058 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.149020+0000", 2024-09-11T07:50:04.058 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1779999999999996e-06 2024-09-11T07:50:04.058 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.058 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.058 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:04.059 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.149023+0000", 2024-09-11T07:50:04.059 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8159999999999998e-06 2024-09-11T07:50:04.059 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.059 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.059 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.059 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.173507+0000", 2024-09-11T07:50:04.059 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.024483773 2024-09-11T07:50:04.059 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.059 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.059 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:04.059 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.173512+0000", 2024-09-11T07:50:04.059 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.0629999999999996e-06 2024-09-11T07:50:04.059 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.059 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.060 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.060 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.150392+0000", 2024-09-11T07:50:04.060 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.97687973500000003 2024-09-11T07:50:04.060 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.060 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.060 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:04.060 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.150399+0000", 2024-09-11T07:50:04.060 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.8580000000000002e-06 2024-09-11T07:50:04.060 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.060 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.060 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.060 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.344379+0000", 2024-09-11T07:50:04.060 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1939801349999999 2024-09-11T07:50:04.061 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.061 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.061 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:04.061 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.345613+0000", 2024-09-11T07:50:04.061 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0012345699999999999 2024-09-11T07:50:04.061 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.061 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.061 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,2,7", 2024-09-11T07:50:04.061 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.345950+0000", 2024-09-11T07:50:04.061 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00033651500000000001 2024-09-11T07:50:04.061 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.061 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.061 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.062 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.360680+0000", 2024-09-11T07:50:04.062 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.014730243000000001 2024-09-11T07:50:04.062 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.062 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.062 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.062 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.361092+0000", 2024-09-11T07:50:04.062 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00041172799999999999 2024-09-11T07:50:04.062 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.062 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.062 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:04.062 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.363446+0000", 2024-09-11T07:50:04.062 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0023536659999999999 2024-09-11T07:50:04.062 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.063 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.063 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.063 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.379710+0000", 2024-09-11T07:50:04.063 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.016264010999999998 2024-09-11T07:50:04.063 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.063 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.063 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:04.063 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.379772+0000", 2024-09-11T07:50:04.063 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2524e-05 2024-09-11T07:50:04.063 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.063 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.063 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:04.063 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.379888+0000", 2024-09-11T07:50:04.063 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000116178 2024-09-11T07:50:04.064 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.064 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:04.064 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.064 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.064 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.064 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:192500 11.6 11:6d4000d0:::benchmark_data_smithi022_188626_object72692:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e453)", 2024-09-11T07:50:04.064 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:48:46.204813+0000", 2024-09-11T07:50:04.064 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 77.255728629999993, 2024-09-11T07:50:04.065 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1703854890000001, 2024-09-11T07:50:04.065 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:04.065 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:04.065 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:04.065 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:04.065 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:04.065 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 192500 2024-09-11T07:50:04.065 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.065 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:04.065 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.066 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:04.066 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.204813+0000", 2024-09-11T07:50:04.066 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.066 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.066 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.066 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:04.066 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.204813+0000", 2024-09-11T07:50:04.066 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.066 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.066 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.066 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:04.066 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.204811+0000", 2024-09-11T07:50:04.066 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:50:04.067 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.067 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.067 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:04.067 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.204818+0000", 2024-09-11T07:50:04.067 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.3390000000000004e-06 2024-09-11T07:50:04.067 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.067 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.067 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:04.067 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.204829+0000", 2024-09-11T07:50:04.067 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0977e-05 2024-09-11T07:50:04.067 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.067 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.067 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:04.068 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.204834+0000", 2024-09-11T07:50:04.068 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1989999999999997e-06 2024-09-11T07:50:04.068 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.068 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.068 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.068 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.204873+0000", 2024-09-11T07:50:04.068 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8928999999999999e-05 2024-09-11T07:50:04.068 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.068 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.068 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:04.068 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.204879+0000", 2024-09-11T07:50:04.068 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1619999999999996e-06 2024-09-11T07:50:04.069 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.069 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.069 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.069 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.150258+0000", 2024-09-11T07:50:04.069 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.94537839700000004 2024-09-11T07:50:04.069 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.069 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.069 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:04.069 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.150267+0000", 2024-09-11T07:50:04.069 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.5829999999999996e-06 2024-09-11T07:50:04.070 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.070 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.070 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.070 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.344232+0000", 2024-09-11T07:50:04.070 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1939647899999999 2024-09-11T07:50:04.070 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.070 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.070 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:04.070 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.344738+0000", 2024-09-11T07:50:04.070 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00050619700000000001 2024-09-11T07:50:04.070 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.070 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.071 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,3,6", 2024-09-11T07:50:04.071 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.345133+0000", 2024-09-11T07:50:04.071 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00039456799999999998 2024-09-11T07:50:04.071 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.071 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.071 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.071 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.361559+0000", 2024-09-11T07:50:04.071 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.016425967999999999 2024-09-11T07:50:04.071 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.071 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.071 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.072 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.362163+0000", 2024-09-11T07:50:04.072 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00060447799999999998 2024-09-11T07:50:04.072 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.072 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.072 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:04.072 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.363772+0000", 2024-09-11T07:50:04.072 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001608586 2024-09-11T07:50:04.072 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.072 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.072 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.072 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.375025+0000", 2024-09-11T07:50:04.073 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.011253302999999999 2024-09-11T07:50:04.073 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.073 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.073 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:04.073 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.375079+0000", 2024-09-11T07:50:04.073 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3955000000000001e-05 2024-09-11T07:50:04.073 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.073 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.073 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:04.073 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:48.375198+0000", 2024-09-11T07:50:04.073 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000119325 2024-09-11T07:50:04.073 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.074 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:04.074 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.074 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.074 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.074 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:208502 11.2 11:4bb9bd15:::benchmark_data_smithi022_188626_object88694:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e458)", 2024-09-11T07:50:04.074 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:49:06.806288+0000", 2024-09-11T07:50:04.074 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 56.654253093000001, 2024-09-11T07:50:04.074 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.035536792, 2024-09-11T07:50:04.074 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:04.074 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:50:04.075 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:04.075 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:04.075 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:04.075 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 208502 2024-09-11T07:50:04.075 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.075 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:04.075 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.075 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:04.075 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.806288+0000", 2024-09-11T07:50:04.075 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.075 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.076 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.076 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:04.076 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.806288+0000", 2024-09-11T07:50:04.076 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.076 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.076 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.076 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:04.076 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.806287+0000", 2024-09-11T07:50:04.076 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:04.076 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.076 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.076 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:04.077 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.806293+0000", 2024-09-11T07:50:04.077 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1020000000000002e-06 2024-09-11T07:50:04.077 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.077 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.077 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:04.077 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.806307+0000", 2024-09-11T07:50:04.077 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4311000000000001e-05 2024-09-11T07:50:04.077 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.077 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.077 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:04.077 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.806314+0000", 2024-09-11T07:50:04.078 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3369999999999996e-06 2024-09-11T07:50:04.078 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.078 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.078 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.078 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.806358+0000", 2024-09-11T07:50:04.078 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4008999999999998e-05 2024-09-11T07:50:04.078 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.078 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.078 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:04.078 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.806471+0000", 2024-09-11T07:50:04.078 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000113069 2024-09-11T07:50:04.079 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.079 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.079 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,2,7", 2024-09-11T07:50:04.079 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.806604+0000", 2024-09-11T07:50:04.079 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013320899999999999 2024-09-11T07:50:04.079 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.079 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.079 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.079 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.811527+0000", 2024-09-11T07:50:04.079 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0049232980000000004 2024-09-11T07:50:04.079 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.079 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.080 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:04.080 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.811536+0000", 2024-09-11T07:50:04.080 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.2599999999999994e-06 2024-09-11T07:50:04.080 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.080 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.080 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.080 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.812683+0000", 2024-09-11T07:50:04.080 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001146708 2024-09-11T07:50:04.080 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.080 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.080 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.081 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.832188+0000", 2024-09-11T07:50:04.081 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.019504904999999999 2024-09-11T07:50:04.081 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.081 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.081 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:04.081 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.832197+0000", 2024-09-11T07:50:04.081 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.0839999999999998e-06 2024-09-11T07:50:04.081 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.081 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.082 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.082 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:07.841743+0000", 2024-09-11T07:50:04.082 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.009545957 2024-09-11T07:50:04.082 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.082 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.082 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:04.082 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:07.841825+0000", 2024-09-11T07:50:04.082 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.2071999999999996e-05 2024-09-11T07:50:04.082 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.082 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:04.082 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.082 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.083 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.083 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:208505 11.2 11:4b23711e:::benchmark_data_smithi022_188626_object88697:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e458)", 2024-09-11T07:50:04.083 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:49:06.807602+0000", 2024-09-11T07:50:04.083 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 56.652939316999998, 2024-09-11T07:50:04.083 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0347198719999999, 2024-09-11T07:50:04.083 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:04.083 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:50:04.083 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:04.083 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:04.083 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:04.084 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 208505 2024-09-11T07:50:04.084 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.084 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:04.084 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.084 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:04.084 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.807602+0000", 2024-09-11T07:50:04.084 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.084 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.084 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.084 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:04.084 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.807602+0000", 2024-09-11T07:50:04.084 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.085 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.085 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.085 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:04.085 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.807601+0000", 2024-09-11T07:50:04.085 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:04.085 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.085 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.085 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:04.085 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.807607+0000", 2024-09-11T07:50:04.085 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0120000000000002e-06 2024-09-11T07:50:04.085 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.085 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.085 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:04.086 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.807618+0000", 2024-09-11T07:50:04.086 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1335000000000001e-05 2024-09-11T07:50:04.086 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.086 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.086 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:04.086 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.807624+0000", 2024-09-11T07:50:04.086 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.541e-06 2024-09-11T07:50:04.086 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.086 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.086 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.086 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.808893+0000", 2024-09-11T07:50:04.086 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00126954 2024-09-11T07:50:04.086 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.086 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.087 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:04.087 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.809000+0000", 2024-09-11T07:50:04.087 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010662699999999999 2024-09-11T07:50:04.087 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.087 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.087 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,2,7", 2024-09-11T07:50:04.087 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.809111+0000", 2024-09-11T07:50:04.087 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011137 2024-09-11T07:50:04.087 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.087 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.087 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:04.087 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.811549+0000", 2024-09-11T07:50:04.087 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0024372899999999999 2024-09-11T07:50:04.088 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.088 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.088 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.088 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.812966+0000", 2024-09-11T07:50:04.088 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0014171889999999999 2024-09-11T07:50:04.088 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.088 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.088 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.088 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.813550+0000", 2024-09-11T07:50:04.088 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00058379700000000005 2024-09-11T07:50:04.088 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.088 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.089 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.089 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.832423+0000", 2024-09-11T07:50:04.089 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.018873576 2024-09-11T07:50:04.089 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.089 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.089 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:04.089 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:06.832429+0000", 2024-09-11T07:50:04.089 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9830000000000001e-06 2024-09-11T07:50:04.089 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.089 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.089 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.089 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:07.842263+0000", 2024-09-11T07:50:04.090 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.009833612 2024-09-11T07:50:04.090 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.090 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.090 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:04.090 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:07.842322+0000", 2024-09-11T07:50:04.090 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9338e-05 2024-09-11T07:50:04.090 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.090 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:04.090 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.090 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.090 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.090 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:237427 11.5 11:a666a877:::benchmark_data_smithi022_188626_object117619:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e469)", 2024-09-11T07:50:04.090 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:49:36.819549+0000", 2024-09-11T07:50:04.091 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 26.640992421, 2024-09-11T07:50:04.091 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.090490953, 2024-09-11T07:50:04.091 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:04.091 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:50:04.091 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:04.091 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:04.091 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:04.091 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 237427 2024-09-11T07:50:04.091 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.091 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:04.091 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.092 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:04.092 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.819549+0000", 2024-09-11T07:50:04.092 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.092 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.092 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.092 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:04.092 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.819549+0000", 2024-09-11T07:50:04.092 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.092 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.092 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.092 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:04.092 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.819548+0000", 2024-09-11T07:50:04.093 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:04.093 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.093 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.093 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:04.093 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.819553+0000", 2024-09-11T07:50:04.093 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8189999999999997e-06 2024-09-11T07:50:04.093 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.093 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.093 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:04.093 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.819568+0000", 2024-09-11T07:50:04.093 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4674e-05 2024-09-11T07:50:04.094 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.094 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.094 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:04.094 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.819574+0000", 2024-09-11T07:50:04.094 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3280000000000002e-06 2024-09-11T07:50:04.094 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.094 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.094 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.094 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.828490+0000", 2024-09-11T07:50:04.094 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0089159110000000003 2024-09-11T07:50:04.094 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.094 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.094 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:04.095 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.828622+0000", 2024-09-11T07:50:04.095 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013157300000000001 2024-09-11T07:50:04.095 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.095 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.095 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:50:04.095 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.828977+0000", 2024-09-11T07:50:04.095 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00035467700000000002 2024-09-11T07:50:04.095 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.095 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.095 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:04.095 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.831987+0000", 2024-09-11T07:50:04.096 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0030099580000000001 2024-09-11T07:50:04.096 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.096 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.096 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.096 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.835378+0000", 2024-09-11T07:50:04.096 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0033910450000000001 2024-09-11T07:50:04.096 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.096 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.096 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.096 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.853023+0000", 2024-09-11T07:50:04.096 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.017645395000000001 2024-09-11T07:50:04.096 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.097 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.097 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:04.097 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.853030+0000", 2024-09-11T07:50:04.097 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.2490000000000004e-06 2024-09-11T07:50:04.097 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.097 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.097 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.097 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:37.909786+0000", 2024-09-11T07:50:04.097 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.056756027 2024-09-11T07:50:04.097 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.097 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.097 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:04.098 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:37.910040+0000", 2024-09-11T07:50:04.098 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00025365799999999998 2024-09-11T07:50:04.098 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.098 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:04.098 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.098 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.098 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.098 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:237452 11.1 11:8eb73911:::benchmark_data_smithi022_188626_object117644:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e470)", 2024-09-11T07:50:04.098 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:49:36.852615+0000", 2024-09-11T07:50:04.098 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 26.607926547000002, 2024-09-11T07:50:04.098 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0553682630000001, 2024-09-11T07:50:04.098 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:04.099 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:04.099 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:04.099 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:04.099 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:04.099 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 237452 2024-09-11T07:50:04.099 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.099 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:04.099 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.099 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:04.099 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.852615+0000", 2024-09-11T07:50:04.099 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.099 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.100 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.100 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:04.100 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.852615+0000", 2024-09-11T07:50:04.100 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.100 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.100 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.100 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:04.100 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.852613+0000", 2024-09-11T07:50:04.100 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:50:04.100 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.100 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.100 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:04.101 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.852621+0000", 2024-09-11T07:50:04.101 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.5730000000000008e-06 2024-09-11T07:50:04.101 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.101 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.101 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:04.101 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.852633+0000", 2024-09-11T07:50:04.101 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1656e-05 2024-09-11T07:50:04.101 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.101 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.101 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:04.101 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.852639+0000", 2024-09-11T07:50:04.102 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3389999999999998e-06 2024-09-11T07:50:04.102 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.102 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.102 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.102 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.852689+0000", 2024-09-11T07:50:04.102 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.935e-05 2024-09-11T07:50:04.102 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.102 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.102 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:04.102 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.852698+0000", 2024-09-11T07:50:04.102 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.9970000000000004e-06 2024-09-11T07:50:04.102 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.103 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.103 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.103 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:37.855787+0000", 2024-09-11T07:50:04.103 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.00308906 2024-09-11T07:50:04.103 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.103 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.103 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:04.103 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:37.855856+0000", 2024-09-11T07:50:04.103 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9122999999999995e-05 2024-09-11T07:50:04.103 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.103 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.103 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,6", 2024-09-11T07:50:04.104 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:37.855937+0000", 2024-09-11T07:50:04.104 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.0949e-05 2024-09-11T07:50:04.104 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.104 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.104 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:04.104 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:37.907739+0000", 2024-09-11T07:50:04.104 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.051802463 2024-09-11T07:50:04.104 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.104 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.104 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.104 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:37.907782+0000", 2024-09-11T07:50:04.104 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2644e-05 2024-09-11T07:50:04.105 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.105 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.105 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.105 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:37.907847+0000", 2024-09-11T07:50:04.105 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5106999999999999e-05 2024-09-11T07:50:04.105 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.105 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.105 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:04.105 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:37.907891+0000", 2024-09-11T07:50:04.105 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4224999999999998e-05 2024-09-11T07:50:04.105 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.105 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.106 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:04.106 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:37.907983+0000", 2024-09-11T07:50:04.106 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.2095000000000005e-05 2024-09-11T07:50:04.106 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.106 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:04.106 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.106 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.106 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.106 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:237460 11.1 11:8ae960b2:::benchmark_data_smithi022_188626_object117652:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e470)", 2024-09-11T07:50:04.106 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:49:36.861549+0000", 2024-09-11T07:50:04.106 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 26.598992594999999, 2024-09-11T07:50:04.106 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0466209150000001, 2024-09-11T07:50:04.107 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:04.107 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:04.107 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:04.107 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:04.107 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:04.107 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 237460 2024-09-11T07:50:04.107 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.107 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:04.107 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.107 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:04.107 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.861549+0000", 2024-09-11T07:50:04.108 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.108 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.108 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.108 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:04.108 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.861549+0000", 2024-09-11T07:50:04.108 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.108 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.108 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.108 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:04.108 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.861546+0000", 2024-09-11T07:50:04.108 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:50:04.109 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.109 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.109 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:04.109 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.861555+0000", 2024-09-11T07:50:04.109 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.4030000000000007e-06 2024-09-11T07:50:04.109 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.109 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.109 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:04.109 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.861571+0000", 2024-09-11T07:50:04.109 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.6724000000000001e-05 2024-09-11T07:50:04.109 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.109 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.110 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:04.110 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.861578+0000", 2024-09-11T07:50:04.110 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0949999999999998e-06 2024-09-11T07:50:04.110 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.110 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.110 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.110 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.861642+0000", 2024-09-11T07:50:04.110 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3972000000000003e-05 2024-09-11T07:50:04.110 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.110 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.110 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:04.111 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.861651+0000", 2024-09-11T07:50:04.111 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.1470000000000001e-06 2024-09-11T07:50:04.111 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.111 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.111 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.111 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:37.857159+0000", 2024-09-11T07:50:04.111 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.99550866900000001 2024-09-11T07:50:04.111 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.111 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.111 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:04.111 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:37.857220+0000", 2024-09-11T07:50:04.111 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0757999999999997e-05 2024-09-11T07:50:04.112 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.112 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.112 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,6", 2024-09-11T07:50:04.112 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:37.857299+0000", 2024-09-11T07:50:04.112 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.9141999999999995e-05 2024-09-11T07:50:04.112 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.112 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.112 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:04.112 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:37.907745+0000", 2024-09-11T07:50:04.112 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.050446182999999999 2024-09-11T07:50:04.112 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.113 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.113 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.113 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:37.907818+0000", 2024-09-11T07:50:04.113 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.3015000000000005e-05 2024-09-11T07:50:04.113 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.113 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.113 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.113 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:37.908046+0000", 2024-09-11T07:50:04.113 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00022729799999999999 2024-09-11T07:50:04.113 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.114 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.114 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:04.114 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:37.908083+0000", 2024-09-11T07:50:04.114 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7245e-05 2024-09-11T07:50:04.114 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.114 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.114 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:04.114 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:37.908170+0000", 2024-09-11T07:50:04.114 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.6915000000000004e-05 2024-09-11T07:50:04.115 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.115 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:04.115 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.115 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.115 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.115 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:237465 11.5 11:a94ba7e5:::benchmark_data_smithi022_188626_object117657:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e470)", 2024-09-11T07:50:04.115 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:49:36.868316+0000", 2024-09-11T07:50:04.115 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 26.592225761000002, 2024-09-11T07:50:04.115 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0480967459999999, 2024-09-11T07:50:04.115 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:04.115 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:04.116 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:04.116 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:04.116 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:04.116 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 237465 2024-09-11T07:50:04.116 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.116 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:04.116 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.116 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:04.116 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.868316+0000", 2024-09-11T07:50:04.116 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.116 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.117 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.117 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:04.117 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.868316+0000", 2024-09-11T07:50:04.117 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.117 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.117 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.117 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:04.117 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.868314+0000", 2024-09-11T07:50:04.117 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:50:04.117 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.117 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.118 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:04.118 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.868322+0000", 2024-09-11T07:50:04.118 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5560000000000002e-06 2024-09-11T07:50:04.118 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.118 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.118 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:04.118 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.868333+0000", 2024-09-11T07:50:04.118 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.198e-05 2024-09-11T07:50:04.118 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.118 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.118 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:04.118 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.868340+0000", 2024-09-11T07:50:04.119 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.6810000000000001e-06 2024-09-11T07:50:04.119 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.119 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.119 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.119 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.868394+0000", 2024-09-11T07:50:04.119 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3563000000000002e-05 2024-09-11T07:50:04.119 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.119 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.119 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:04.119 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:36.868402+0000", 2024-09-11T07:50:04.119 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.8310000000000001e-06 2024-09-11T07:50:04.120 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.120 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.120 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.120 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:37.910593+0000", 2024-09-11T07:50:04.120 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.04219192 2024-09-11T07:50:04.120 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.120 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.120 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:04.120 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:37.910705+0000", 2024-09-11T07:50:04.120 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000111976 2024-09-11T07:50:04.120 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.120 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.121 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 6", 2024-09-11T07:50:04.121 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:37.910832+0000", 2024-09-11T07:50:04.121 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012690100000000001 2024-09-11T07:50:04.121 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.121 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.121 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:04.121 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:37.912695+0000", 2024-09-11T07:50:04.121 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0018630789999999999 2024-09-11T07:50:04.121 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.121 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.121 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.122 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:37.916288+0000", 2024-09-11T07:50:04.122 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0035922689999999999 2024-09-11T07:50:04.122 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.122 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.122 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:04.122 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:37.916334+0000", 2024-09-11T07:50:04.122 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6792999999999997e-05 2024-09-11T07:50:04.122 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.122 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.122 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:04.122 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:37.916412+0000", 2024-09-11T07:50:04.122 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.7948000000000007e-05 2024-09-11T07:50:04.123 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.123 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:04.123 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.123 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.123 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.123 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:238738 11.f 11:fd452009:::benchmark_data_smithi022_188626_object118930:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e472)", 2024-09-11T07:50:04.123 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:49:39.109538+0000", 2024-09-11T07:50:04.123 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 24.351003158000001, 2024-09-11T07:50:04.123 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0422208289999999, 2024-09-11T07:50:04.123 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:04.123 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:04.124 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:04.124 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:04.124 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:04.124 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 238738 2024-09-11T07:50:04.124 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.124 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:04.124 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.124 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:04.124 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:39.109538+0000", 2024-09-11T07:50:04.124 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.124 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.125 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.125 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:04.125 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:39.109538+0000", 2024-09-11T07:50:04.125 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.125 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.125 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.125 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:04.125 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:39.109536+0000", 2024-09-11T07:50:04.125 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:50:04.125 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.125 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.125 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:04.126 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:39.109543+0000", 2024-09-11T07:50:04.126 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.144e-06 2024-09-11T07:50:04.126 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.126 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.126 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:04.126 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:39.109591+0000", 2024-09-11T07:50:04.126 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8835999999999998e-05 2024-09-11T07:50:04.126 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.126 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.126 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:04.127 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:39.109628+0000", 2024-09-11T07:50:04.127 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6426e-05 2024-09-11T07:50:04.127 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.127 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.127 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.127 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:39.110077+0000", 2024-09-11T07:50:04.127 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00044940799999999999 2024-09-11T07:50:04.127 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.127 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.127 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:04.127 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:39.110085+0000", 2024-09-11T07:50:04.127 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.2180000000000002e-06 2024-09-11T07:50:04.128 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.128 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.128 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.128 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:40.114875+0000", 2024-09-11T07:50:04.128 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0047909859999999 2024-09-11T07:50:04.128 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.128 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.128 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:04.128 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:40.115003+0000", 2024-09-11T07:50:04.128 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012766899999999999 2024-09-11T07:50:04.128 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.129 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.129 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:50:04.129 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:40.115525+0000", 2024-09-11T07:50:04.129 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000522179 2024-09-11T07:50:04.129 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.129 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.129 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:04.129 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:40.144841+0000", 2024-09-11T07:50:04.129 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.029315758000000001 2024-09-11T07:50:04.129 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.129 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.130 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.130 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:40.151495+0000", 2024-09-11T07:50:04.130 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0066540779999999999 2024-09-11T07:50:04.130 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.130 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.130 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.130 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:40.151670+0000", 2024-09-11T07:50:04.130 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000174551 2024-09-11T07:50:04.130 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.130 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.130 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:04.130 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:40.151716+0000", 2024-09-11T07:50:04.131 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6397999999999999e-05 2024-09-11T07:50:04.131 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.131 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.131 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:04.131 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:40.151759+0000", 2024-09-11T07:50:04.131 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2997000000000002e-05 2024-09-11T07:50:04.131 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.131 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:04.131 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.131 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.131 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.132 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:238741 11.f 11:f03328f1:::benchmark_data_smithi022_188626_object118933:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e472)", 2024-09-11T07:50:04.132 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:49:39.109972+0000", 2024-09-11T07:50:04.132 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 24.350568962000001, 2024-09-11T07:50:04.132 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0419300300000001, 2024-09-11T07:50:04.132 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:04.132 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:04.132 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:04.132 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:04.132 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:04.132 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 238741 2024-09-11T07:50:04.132 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.132 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:04.133 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.133 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:04.133 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:39.109972+0000", 2024-09-11T07:50:04.133 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.133 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.133 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.133 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:04.133 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:39.109972+0000", 2024-09-11T07:50:04.133 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.133 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.133 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.133 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:04.134 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:39.109971+0000", 2024-09-11T07:50:04.134 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:04.134 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.134 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.134 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:04.134 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:39.109977+0000", 2024-09-11T07:50:04.134 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8780000000000003e-06 2024-09-11T07:50:04.134 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.134 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.134 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:04.134 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:39.109990+0000", 2024-09-11T07:50:04.134 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2638e-05 2024-09-11T07:50:04.134 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.135 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.135 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:04.135 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:39.109995+0000", 2024-09-11T07:50:04.135 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7019999999999998e-06 2024-09-11T07:50:04.135 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.135 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.135 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.135 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:39.110103+0000", 2024-09-11T07:50:04.135 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000107559 2024-09-11T07:50:04.135 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.135 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.135 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:04.135 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:39.110108+0000", 2024-09-11T07:50:04.136 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7679999999999999e-06 2024-09-11T07:50:04.136 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.136 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.136 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.136 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:40.136597+0000", 2024-09-11T07:50:04.136 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.026489378 2024-09-11T07:50:04.136 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.136 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.136 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:04.136 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:40.136752+0000", 2024-09-11T07:50:04.136 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00015493500000000001 2024-09-11T07:50:04.136 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.136 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.137 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:50:04.137 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:40.136840+0000", 2024-09-11T07:50:04.137 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.7946000000000001e-05 2024-09-11T07:50:04.137 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.137 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.137 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:04.137 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:40.144848+0000", 2024-09-11T07:50:04.137 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0080083080000000004 2024-09-11T07:50:04.137 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.137 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.137 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.137 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:40.151540+0000", 2024-09-11T07:50:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0066915350000000002 2024-09-11T07:50:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:40.151826+0000", 2024-09-11T07:50:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00028621700000000001 2024-09-11T07:50:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:40.151865+0000", 2024-09-11T07:50:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9415999999999997e-05 2024-09-11T07:50:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.138 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:04.139 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:40.151903+0000", 2024-09-11T07:50:04.139 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7146e-05 2024-09-11T07:50:04.139 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.139 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:04.139 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.139 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.139 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.139 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:238750 11.f 11:ff155fcd:::benchmark_data_smithi022_188626_object118942:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e472)", 2024-09-11T07:50:04.139 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:49:39.116559+0000", 2024-09-11T07:50:04.139 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 24.343982620999999, 2024-09-11T07:50:04.139 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.035467709, 2024-09-11T07:50:04.139 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:04.140 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:04.140 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:04.140 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:04.140 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:04.140 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 238750 2024-09-11T07:50:04.140 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.140 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:04.140 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.140 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:04.140 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:39.116559+0000", 2024-09-11T07:50:04.140 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.141 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.141 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.141 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:04.141 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:39.116559+0000", 2024-09-11T07:50:04.141 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:04.141 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.141 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.141 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:04.141 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:39.116558+0000", 2024-09-11T07:50:04.141 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:50:04.141 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.142 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.142 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:04.142 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:39.116563+0000", 2024-09-11T07:50:04.142 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2619999999999998e-06 2024-09-11T07:50:04.142 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.142 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.142 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:04.142 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:39.116570+0000", 2024-09-11T07:50:04.142 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0380000000000002e-06 2024-09-11T07:50:04.142 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.142 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.142 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:04.142 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:39.116575+0000", 2024-09-11T07:50:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6650000000000002e-06 2024-09-11T07:50:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:39.121864+0000", 2024-09-11T07:50:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0052891049999999997 2024-09-11T07:50:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:39.121871+0000", 2024-09-11T07:50:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.3509999999999996e-06 2024-09-11T07:50:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.143 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.144 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:04.144 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:40.137908+0000", 2024-09-11T07:50:04.144 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.016036986 2024-09-11T07:50:04.144 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.144 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.144 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:04.144 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:40.137989+0000", 2024-09-11T07:50:04.144 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.0840000000000005e-05 2024-09-11T07:50:04.144 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.144 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.144 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 1,3", 2024-09-11T07:50:04.144 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:40.138073+0000", 2024-09-11T07:50:04.145 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3938999999999997e-05 2024-09-11T07:50:04.145 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.145 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.145 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:04.145 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:40.144857+0000", 2024-09-11T07:50:04.145 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0067843620000000004 2024-09-11T07:50:04.145 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.145 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.145 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.145 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:40.151605+0000", 2024-09-11T07:50:04.145 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0067473380000000003 2024-09-11T07:50:04.145 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.146 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.146 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:04.146 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:40.151948+0000", 2024-09-11T07:50:04.146 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00034283300000000002 2024-09-11T07:50:04.146 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.146 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.146 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:04.146 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:40.151993+0000", 2024-09-11T07:50:04.146 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5593000000000002e-05 2024-09-11T07:50:04.146 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:04.146 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:04.146 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:04.147 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:40.152027+0000", 2024-09-11T07:50:04.147 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.3414999999999998e-05 2024-09-11T07:50:04.147 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.147 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:04.147 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.147 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:04.147 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:04.147 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:50:04.287 INFO:teuthology.orchestra.run.smithi022.stdout:11.d 2024-09-11T07:50:04.287 INFO:teuthology.orchestra.run.smithi022.stdout:11.c 2024-09-11T07:50:04.287 INFO:teuthology.orchestra.run.smithi022.stdout:11.8 2024-09-11T07:50:04.287 INFO:teuthology.orchestra.run.smithi022.stdout:11.6 2024-09-11T07:50:04.287 INFO:teuthology.orchestra.run.smithi022.stdout:11.5 2024-09-11T07:50:04.287 INFO:teuthology.orchestra.run.smithi022.stdout:11.4 2024-09-11T07:50:04.288 INFO:teuthology.orchestra.run.smithi022.stdout:11.3 2024-09-11T07:50:04.288 INFO:teuthology.orchestra.run.smithi022.stdout:11.f 2024-09-11T07:50:04.288 INFO:teuthology.orchestra.run.smithi022.stdout:11.2 2024-09-11T07:50:04.288 INFO:teuthology.orchestra.run.smithi022.stdout:11.1 2024-09-11T07:50:04.295 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:50:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:50:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:50:04.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:04 smithi022 ceph-mon[29462]: 2.a scrub starts 2024-09-11T07:50:04.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:04 smithi022 ceph-mon[29462]: 2.a scrub ok 2024-09-11T07:50:04.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:04 smithi022 ceph-mon[29462]: 3.0 scrub starts 2024-09-11T07:50:04.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:04 smithi022 ceph-mon[29462]: 3.0 scrub ok 2024-09-11T07:50:04.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:04 smithi022 ceph-mon[29462]: pgmap v1097: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:50:04.632 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:04 smithi022 ceph-mon[37330]: 2.a scrub starts 2024-09-11T07:50:04.632 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:04 smithi022 ceph-mon[37330]: 2.a scrub ok 2024-09-11T07:50:04.632 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:04 smithi022 ceph-mon[37330]: 3.0 scrub starts 2024-09-11T07:50:04.632 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:04 smithi022 ceph-mon[37330]: 3.0 scrub ok 2024-09-11T07:50:04.632 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:04 smithi022 ceph-mon[37330]: pgmap v1097: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:50:04.657 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-11T07:50:04.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:04 smithi104 ceph-mon[33507]: 2.a scrub starts 2024-09-11T07:50:04.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:04 smithi104 ceph-mon[33507]: 2.a scrub ok 2024-09-11T07:50:04.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:04 smithi104 ceph-mon[33507]: 3.0 scrub starts 2024-09-11T07:50:04.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:04 smithi104 ceph-mon[33507]: 3.0 scrub ok 2024-09-11T07:50:04.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:04 smithi104 ceph-mon[33507]: pgmap v1097: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:50:05.518 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:05 smithi022 ceph-mon[29462]: 4.1a scrub starts 2024-09-11T07:50:05.518 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:05 smithi022 ceph-mon[29462]: 4.1a scrub ok 2024-09-11T07:50:05.518 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:05 smithi022 ceph-mon[29462]: 5.4 scrub starts 2024-09-11T07:50:05.518 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:05 smithi022 ceph-mon[29462]: 5.4 scrub ok 2024-09-11T07:50:05.518 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:05 smithi022 ceph-mon[29462]: 2.11 scrub starts 2024-09-11T07:50:05.518 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:05 smithi022 ceph-mon[29462]: 2.11 scrub ok 2024-09-11T07:50:05.519 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:05 smithi022 ceph-mon[37330]: 4.1a scrub starts 2024-09-11T07:50:05.519 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:05 smithi022 ceph-mon[37330]: 4.1a scrub ok 2024-09-11T07:50:05.519 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:05 smithi022 ceph-mon[37330]: 5.4 scrub starts 2024-09-11T07:50:05.519 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:05 smithi022 ceph-mon[37330]: 5.4 scrub ok 2024-09-11T07:50:05.519 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:05 smithi022 ceph-mon[37330]: 2.11 scrub starts 2024-09-11T07:50:05.519 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:05 smithi022 ceph-mon[37330]: 2.11 scrub ok 2024-09-11T07:50:05.546 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -n osd.2 --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --op export --pgid 11.3 --file /var/log/ceph/exp.11.3.2 2024-09-11T07:50:05.672 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:50:05.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:05 smithi104 ceph-mon[33507]: 4.1a scrub starts 2024-09-11T07:50:05.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:05 smithi104 ceph-mon[33507]: 4.1a scrub ok 2024-09-11T07:50:05.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:05 smithi104 ceph-mon[33507]: 5.4 scrub starts 2024-09-11T07:50:05.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:05 smithi104 ceph-mon[33507]: 5.4 scrub ok 2024-09-11T07:50:05.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:05 smithi104 ceph-mon[33507]: 2.11 scrub starts 2024-09-11T07:50:05.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:05 smithi104 ceph-mon[33507]: 2.11 scrub ok 2024-09-11T07:50:05.837 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-11T07:50:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/osd.2/config 2024-09-11T07:50:06.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:06 smithi022 ceph-mon[29462]: pgmap v1098: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:50:06.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:06 smithi022 ceph-mon[37330]: pgmap v1098: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:50:06.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:06 smithi104 ceph-mon[33507]: pgmap v1098: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:50:07.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:07 smithi022 ceph-mon[29462]: 5.16 scrub starts 2024-09-11T07:50:07.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:07 smithi022 ceph-mon[29462]: 5.16 scrub ok 2024-09-11T07:50:07.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:07 smithi022 ceph-mon[37330]: 5.16 scrub starts 2024-09-11T07:50:07.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:07 smithi022 ceph-mon[37330]: 5.16 scrub ok 2024-09-11T07:50:07.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:07 smithi104 ceph-mon[33507]: 5.16 scrub starts 2024-09-11T07:50:07.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:07 smithi104 ceph-mon[33507]: 5.16 scrub ok 2024-09-11T07:50:08.374 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:50:08 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:50:08.061+0000 7f0b5d524640 -1 mgr get_metadata_python Requested missing service osd.2 2024-09-11T07:50:08.375 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:08 smithi022 ceph-mon[37330]: 5.12 scrub starts 2024-09-11T07:50:08.375 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:08 smithi022 ceph-mon[37330]: 5.12 scrub ok 2024-09-11T07:50:08.375 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:08 smithi022 ceph-mon[37330]: pgmap v1099: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:50:08.375 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:50:08.375 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:50:08.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:08 smithi022 ceph-mon[29462]: 5.12 scrub starts 2024-09-11T07:50:08.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:08 smithi022 ceph-mon[29462]: 5.12 scrub ok 2024-09-11T07:50:08.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:08 smithi022 ceph-mon[29462]: pgmap v1099: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:50:08.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:50:08.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:50:08.456 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:50:08.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:08 smithi104 ceph-mon[33507]: 5.12 scrub starts 2024-09-11T07:50:08.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:08 smithi104 ceph-mon[33507]: 5.12 scrub ok 2024-09-11T07:50:08.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:08 smithi104 ceph-mon[33507]: pgmap v1099: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:50:08.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:50:08.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:50:08.984 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:50:08.985 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:50:08.985 INFO:teuthology.orchestra.run.smithi022.stdout: "num_ops": 0 2024-09-11T07:50:08.985 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:50:09.600 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:09 smithi022 ceph-mon[29462]: 2.d scrub starts 2024-09-11T07:50:09.600 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:09 smithi022 ceph-mon[29462]: 2.d scrub ok 2024-09-11T07:50:09.600 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:09 smithi022 ceph-mon[29462]: 4.13 scrub starts 2024-09-11T07:50:09.600 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:09 smithi022 ceph-mon[29462]: 4.13 scrub ok 2024-09-11T07:50:09.600 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:09 smithi022 ceph-mon[37330]: 2.d scrub starts 2024-09-11T07:50:09.600 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:09 smithi022 ceph-mon[37330]: 2.d scrub ok 2024-09-11T07:50:09.600 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:09 smithi022 ceph-mon[37330]: 4.13 scrub starts 2024-09-11T07:50:09.600 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:09 smithi022 ceph-mon[37330]: 4.13 scrub ok 2024-09-11T07:50:09.744 INFO:teuthology.orchestra.run.smithi022.stderr:Exporting 11.3 info 11.3( v 459'12990 (441'8100,459'12990] lb MIN local-lis/les=454/455 n=1894 ec=412/412 lis/c=454/454 les/c/f=455/459/0 sis=460) 2024-09-11T07:50:09.771 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c6e585cf:::benchmark_data_smithi022_188626_object100889:head# 2024-09-11T07:50:09.771 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c6ed3315:::benchmark_data_smithi022_188626_object107189:head# 2024-09-11T07:50:09.777 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c6ee46d2:::benchmark_data_smithi022_188626_object107122:head# 2024-09-11T07:50:09.777 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c6f99e62:::benchmark_data_smithi022_188626_object111416:head# 2024-09-11T07:50:09.777 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c6fa471f:::benchmark_data_smithi022_188626_object103789:head# 2024-09-11T07:50:09.777 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c6fa6556:::benchmark_data_smithi022_188626_object96308:head# 2024-09-11T07:50:09.777 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c6fbe4ef:::benchmark_data_smithi022_188626_object93585:head# 2024-09-11T07:50:09.778 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c6fce5ce:::benchmark_data_smithi022_188626_object103584:head# 2024-09-11T07:50:09.778 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c6fdcdb2:::benchmark_data_smithi022_188626_object100807:head# 2024-09-11T07:50:09.778 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c6ff62e0:::benchmark_data_smithi022_188626_object109952:head# 2024-09-11T07:50:09.778 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c70063cf:::benchmark_data_smithi022_188626_object115177:head# 2024-09-11T07:50:09.778 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c701d295:::benchmark_data_smithi022_188626_object106986:head# 2024-09-11T07:50:09.778 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c705310c:::benchmark_data_smithi022_188626_object103796:head# 2024-09-11T07:50:09.778 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c70655fa:::benchmark_data_smithi022_188626_object114676:head# 2024-09-11T07:50:09.778 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c70b8233:::benchmark_data_smithi022_188626_object101352:head# 2024-09-11T07:50:09.778 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c714fe57:::benchmark_data_smithi022_188626_object100996:head# 2024-09-11T07:50:09.778 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c715a339:::benchmark_data_smithi022_188626_object109111:head# 2024-09-11T07:50:09.778 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c718dbc6:::benchmark_data_smithi022_188626_object94807:head# 2024-09-11T07:50:09.779 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c71b5cfe:::benchmark_data_smithi022_188626_object100054:head# 2024-09-11T07:50:09.779 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c71b9f61:::benchmark_data_smithi022_188626_object100771:head# 2024-09-11T07:50:09.779 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c71bb997:::benchmark_data_smithi022_188626_object110050:head# 2024-09-11T07:50:09.779 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c71c780e:::benchmark_data_smithi022_188626_object101050:head# 2024-09-11T07:50:09.779 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c71e5606:::benchmark_data_smithi022_188626_object115472:head# 2024-09-11T07:50:09.779 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c72267b5:::benchmark_data_smithi022_188626_object119346:head# 2024-09-11T07:50:09.779 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7228c14:::benchmark_data_smithi022_188626_object103308:head# 2024-09-11T07:50:09.779 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c72488c3:::benchmark_data_smithi022_188626_object112819:head# 2024-09-11T07:50:09.779 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7259c34:::benchmark_data_smithi022_188626_object99199:head# 2024-09-11T07:50:09.780 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7266e42:::benchmark_data_smithi022_188626_object114272:head# 2024-09-11T07:50:09.780 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c72698ab:::benchmark_data_smithi022_188626_object95790:head# 2024-09-11T07:50:09.780 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c727a822:::benchmark_data_smithi022_188626_object107691:head# 2024-09-11T07:50:09.780 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c72873d7:::benchmark_data_smithi022_188626_object107005:head# 2024-09-11T07:50:09.780 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7289d93:::benchmark_data_smithi022_188626_object119471:head# 2024-09-11T07:50:09.780 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c72d6f00:::benchmark_data_smithi022_188626_object99316:head# 2024-09-11T07:50:09.780 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c734648b:::benchmark_data_smithi022_188626_object90555:head# 2024-09-11T07:50:09.780 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c735e868:::benchmark_data_smithi022_188626_object113394:head# 2024-09-11T07:50:09.780 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7393da0:::benchmark_data_smithi022_188626_object100068:head# 2024-09-11T07:50:09.780 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c73c50c3:::benchmark_data_smithi022_188626_object108013:head# 2024-09-11T07:50:09.780 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c73cd79d:::benchmark_data_smithi022_188626_object107108:head# 2024-09-11T07:50:09.780 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c73df660:::benchmark_data_smithi022_188626_object94857:head# 2024-09-11T07:50:09.781 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c742fa6e:::benchmark_data_smithi022_188626_object106410:head# 2024-09-11T07:50:09.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:09 smithi104 ceph-mon[33507]: 2.d scrub starts 2024-09-11T07:50:09.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:09 smithi104 ceph-mon[33507]: 2.d scrub ok 2024-09-11T07:50:09.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:09 smithi104 ceph-mon[33507]: 4.13 scrub starts 2024-09-11T07:50:09.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:09 smithi104 ceph-mon[33507]: 4.13 scrub ok 2024-09-11T07:50:09.813 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c74361d5:::benchmark_data_smithi022_188626_object100491:head# 2024-09-11T07:50:09.814 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c746b74e:::benchmark_data_smithi022_188626_object117769:head# 2024-09-11T07:50:09.814 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c746cf8d:::benchmark_data_smithi022_188626_object113061:head# 2024-09-11T07:50:09.814 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c748609a:::benchmark_data_smithi022_188626_object99723:head# 2024-09-11T07:50:09.814 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c74ad3f6:::benchmark_data_smithi022_188626_object106198:head# 2024-09-11T07:50:09.814 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c74ae325:::benchmark_data_smithi022_188626_object108297:head# 2024-09-11T07:50:09.814 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c74d1940:::benchmark_data_smithi022_188626_object97535:head# 2024-09-11T07:50:09.814 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7508046:::benchmark_data_smithi022_188626_object111319:head# 2024-09-11T07:50:09.814 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c751fa2a:::benchmark_data_smithi022_188626_object90126:head# 2024-09-11T07:50:09.814 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c75341c3:::benchmark_data_smithi022_188626_object95918:head# 2024-09-11T07:50:09.814 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c754c3c7:::benchmark_data_smithi022_188626_object114388:head# 2024-09-11T07:50:09.814 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c755df0c:::benchmark_data_smithi022_188626_object99930:head# 2024-09-11T07:50:09.815 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c757afaf:::benchmark_data_smithi022_188626_object114423:head# 2024-09-11T07:50:09.815 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c75b6003:::benchmark_data_smithi022_188626_object102022:head# 2024-09-11T07:50:09.815 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c75de3d0:::benchmark_data_smithi022_188626_object91975:head# 2024-09-11T07:50:09.815 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c763cc0e:::benchmark_data_smithi022_188626_object100915:head# 2024-09-11T07:50:09.815 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c769e9ab:::benchmark_data_smithi022_188626_object118500:head# 2024-09-11T07:50:09.815 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c76b907c:::benchmark_data_smithi022_188626_object101300:head# 2024-09-11T07:50:09.815 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c76ca706:::benchmark_data_smithi022_188626_object101662:head# 2024-09-11T07:50:09.815 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c76eb59c:::benchmark_data_smithi022_188626_object89807:head# 2024-09-11T07:50:09.815 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c770a7a6:::benchmark_data_smithi022_188626_object96096:head# 2024-09-11T07:50:09.815 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7720b52:::benchmark_data_smithi022_188626_object114776:head# 2024-09-11T07:50:09.815 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7721b4f:::benchmark_data_smithi022_188626_object95658:head# 2024-09-11T07:50:09.815 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c774096e:::benchmark_data_smithi022_188626_object95956:head# 2024-09-11T07:50:09.816 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7744a52:::benchmark_data_smithi022_188626_object111871:head# 2024-09-11T07:50:09.816 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c77ab2c2:::benchmark_data_smithi022_188626_object92476:head# 2024-09-11T07:50:09.816 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c77dd1bd:::benchmark_data_smithi022_188626_object94177:head# 2024-09-11T07:50:09.816 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c77e9d72:::benchmark_data_smithi022_188626_object93506:head# 2024-09-11T07:50:09.816 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c780a3b2:::benchmark_data_smithi022_188626_object102374:head# 2024-09-11T07:50:09.816 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7848c59:::benchmark_data_smithi022_188626_object114000:head# 2024-09-11T07:50:09.816 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c785337c:::benchmark_data_smithi022_188626_object117131:head# 2024-09-11T07:50:09.816 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7877bef:::benchmark_data_smithi022_188626_object116380:head# 2024-09-11T07:50:09.816 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c78af598:::benchmark_data_smithi022_188626_object93382:head# 2024-09-11T07:50:09.816 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c78eab42:::benchmark_data_smithi022_188626_object104887:head# 2024-09-11T07:50:09.816 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7906b34:::benchmark_data_smithi022_188626_object97077:head# 2024-09-11T07:50:09.817 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7966ec1:::benchmark_data_smithi022_188626_object90322:head# 2024-09-11T07:50:09.817 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c798d9aa:::benchmark_data_smithi022_188626_object101991:head# 2024-09-11T07:50:09.817 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c799b198:::benchmark_data_smithi022_188626_object104835:head# 2024-09-11T07:50:09.817 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c79eb3ed:::benchmark_data_smithi022_188626_object111157:head# 2024-09-11T07:50:09.817 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7a0988c:::benchmark_data_smithi022_188626_object113037:head# 2024-09-11T07:50:09.817 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7a28da2:::benchmark_data_smithi022_188626_object113009:head# 2024-09-11T07:50:09.817 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7a5c345:::benchmark_data_smithi022_188626_object99476:head# 2024-09-11T07:50:09.817 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7a73afb:::benchmark_data_smithi022_188626_object92627:head# 2024-09-11T07:50:09.817 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7ad0559:::benchmark_data_smithi022_188626_object106141:head# 2024-09-11T07:50:09.817 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7ad8769:::benchmark_data_smithi022_188626_object117673:head# 2024-09-11T07:50:09.817 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7b2ae22:::benchmark_data_smithi022_188626_object107848:head# 2024-09-11T07:50:09.817 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7b38b65:::benchmark_data_smithi022_188626_object95385:head# 2024-09-11T07:50:09.817 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7b453e2:::benchmark_data_smithi022_188626_object110569:head# 2024-09-11T07:50:09.818 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7b544ee:::benchmark_data_smithi022_188626_object104032:head# 2024-09-11T07:50:09.818 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7b7c1bd:::benchmark_data_smithi022_188626_object104884:head# 2024-09-11T07:50:09.818 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7b9d165:::benchmark_data_smithi022_188626_object115864:head# 2024-09-11T07:50:09.818 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7b9d9b7:::benchmark_data_smithi022_188626_object90647:head# 2024-09-11T07:50:09.818 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7c0e6ae:::benchmark_data_smithi022_188626_object115420:head# 2024-09-11T07:50:09.818 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7c1970d:::benchmark_data_smithi022_188626_object117072:head# 2024-09-11T07:50:09.818 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7c8ad1c:::benchmark_data_smithi022_188626_object107201:head# 2024-09-11T07:50:09.818 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7ca24a2:::benchmark_data_smithi022_188626_object99729:head# 2024-09-11T07:50:09.818 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7cc6ed0:::benchmark_data_smithi022_188626_object118019:head# 2024-09-11T07:50:09.818 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7d212e9:::benchmark_data_smithi022_188626_object110825:head# 2024-09-11T07:50:09.818 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7d9e793:::benchmark_data_smithi022_188626_object102170:head# 2024-09-11T07:50:09.818 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7de2681:::benchmark_data_smithi022_188626_object112984:head# 2024-09-11T07:50:09.818 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7e31285:::benchmark_data_smithi022_188626_object100787:head# 2024-09-11T07:50:09.818 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7e56005:::benchmark_data_smithi022_188626_object115116:head# 2024-09-11T07:50:09.819 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7e5c864:::benchmark_data_smithi022_188626_object110331:head# 2024-09-11T07:50:09.819 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7f01319:::benchmark_data_smithi022_188626_object113893:head# 2024-09-11T07:50:09.819 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7f58003:::benchmark_data_smithi022_188626_object104466:head# 2024-09-11T07:50:09.819 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7f62822:::benchmark_data_smithi022_188626_object92327:head# 2024-09-11T07:50:09.819 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7f7d946:::benchmark_data_smithi022_188626_object110997:head# 2024-09-11T07:50:09.819 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7f83bd2:::benchmark_data_smithi022_188626_object100221:head# 2024-09-11T07:50:09.819 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7f8e052:::benchmark_data_smithi022_188626_object101473:head# 2024-09-11T07:50:09.819 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7f952ec:::benchmark_data_smithi022_188626_object112806:head# 2024-09-11T07:50:09.819 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7fa2248:::benchmark_data_smithi022_188626_object104723:head# 2024-09-11T07:50:09.819 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7fb1efd:::benchmark_data_smithi022_188626_object105759:head# 2024-09-11T07:50:09.819 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7fbced2:::benchmark_data_smithi022_188626_object92144:head# 2024-09-11T07:50:09.819 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7fd497b:::benchmark_data_smithi022_188626_object94607:head# 2024-09-11T07:50:09.819 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c7fe638b:::benchmark_data_smithi022_188626_object110061:head# 2024-09-11T07:50:09.819 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c802ad73:::benchmark_data_smithi022_188626_object109507:head# 2024-09-11T07:50:09.820 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c803a5ea:::benchmark_data_smithi022_188626_object113533:head# 2024-09-11T07:50:09.820 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8052860:::benchmark_data_smithi022_188626_object101175:head# 2024-09-11T07:50:09.820 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c80733f6:::benchmark_data_smithi022_188626_object105410:head# 2024-09-11T07:50:09.820 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c808041d:::benchmark_data_smithi022_188626_object107259:head# 2024-09-11T07:50:09.820 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8084916:::benchmark_data_smithi022_188626_object110943:head# 2024-09-11T07:50:09.820 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c80c64d6:::benchmark_data_smithi022_188626_object108863:head# 2024-09-11T07:50:09.820 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c80c81af:::benchmark_data_smithi022_188626_object118253:head# 2024-09-11T07:50:09.820 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c80ca994:::benchmark_data_smithi022_188626_object106474:head# 2024-09-11T07:50:09.820 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c80d2ea3:::benchmark_data_smithi022_188626_object105214:head# 2024-09-11T07:50:09.820 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c80ffbbe:::benchmark_data_smithi022_188626_object101958:head# 2024-09-11T07:50:09.820 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c811c808:::benchmark_data_smithi022_188626_object118549:head# 2024-09-11T07:50:09.820 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8137658:::benchmark_data_smithi022_188626_object95810:head# 2024-09-11T07:50:09.820 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8149990:::benchmark_data_smithi022_188626_object107075:head# 2024-09-11T07:50:09.821 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c817acf8:::benchmark_data_smithi022_188626_object105345:head# 2024-09-11T07:50:09.821 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c818e369:::benchmark_data_smithi022_188626_object104125:head# 2024-09-11T07:50:09.821 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c81f812d:::benchmark_data_smithi022_188626_object98297:head# 2024-09-11T07:50:09.821 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8248d4f:::benchmark_data_smithi022_188626_object117231:head# 2024-09-11T07:50:09.821 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c828abce:::benchmark_data_smithi022_188626_object101475:head# 2024-09-11T07:50:09.821 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c828c56a:::benchmark_data_smithi022_188626_object117395:head# 2024-09-11T07:50:09.821 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8290068:::benchmark_data_smithi022_188626_object113082:head# 2024-09-11T07:50:09.821 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c82a66b2:::benchmark_data_smithi022_188626_object117980:head# 2024-09-11T07:50:09.821 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c82cfaeb:::benchmark_data_smithi022_188626_object105471:head# 2024-09-11T07:50:09.821 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c82df47e:::benchmark_data_smithi022_188626_object114263:head# 2024-09-11T07:50:09.821 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8303010:::benchmark_data_smithi022_188626_object96980:head# 2024-09-11T07:50:09.821 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8334bf8:::benchmark_data_smithi022_188626_object106471:head# 2024-09-11T07:50:09.822 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c83d0e16:::benchmark_data_smithi022_188626_object104403:head# 2024-09-11T07:50:09.822 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c83f5dc8:::benchmark_data_smithi022_188626_object108554:head# 2024-09-11T07:50:09.822 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c845162b:::benchmark_data_smithi022_188626_object118759:head# 2024-09-11T07:50:09.822 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c84574b4:::benchmark_data_smithi022_188626_object94699:head# 2024-09-11T07:50:09.822 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c847a17f:::benchmark_data_smithi022_188626_object89374:head# 2024-09-11T07:50:09.822 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c848c1a0:::benchmark_data_smithi022_188626_object100914:head# 2024-09-11T07:50:09.822 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c84b326b:::benchmark_data_smithi022_188626_object113434:head# 2024-09-11T07:50:09.822 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c84da272:::benchmark_data_smithi022_188626_object102862:head# 2024-09-11T07:50:09.822 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c84de2c3:::benchmark_data_smithi022_188626_object115494:head# 2024-09-11T07:50:09.822 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8536fb6:::benchmark_data_smithi022_188626_object108551:head# 2024-09-11T07:50:09.822 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8561c26:::benchmark_data_smithi022_188626_object103713:head# 2024-09-11T07:50:09.822 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8564826:::benchmark_data_smithi022_188626_object91232:head# 2024-09-11T07:50:09.822 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8589949:::benchmark_data_smithi022_188626_object92269:head# 2024-09-11T07:50:09.823 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c85d4110:::benchmark_data_smithi022_188626_object110731:head# 2024-09-11T07:50:09.823 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c86292ba:::benchmark_data_smithi022_188626_object98300:head# 2024-09-11T07:50:09.823 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8646bc3:::benchmark_data_smithi022_188626_object112633:head# 2024-09-11T07:50:09.823 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8654e59:::benchmark_data_smithi022_188626_object119748:head# 2024-09-11T07:50:09.823 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c866c9aa:::benchmark_data_smithi022_188626_object119493:head# 2024-09-11T07:50:09.823 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c867aaf4:::benchmark_data_smithi022_188626_object102256:head# 2024-09-11T07:50:09.823 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c86a60cf:::benchmark_data_smithi022_188626_object117595:head# 2024-09-11T07:50:09.823 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c86ae905:::benchmark_data_smithi022_188626_object94463:head# 2024-09-11T07:50:09.823 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c86dbc36:::benchmark_data_smithi022_188626_object101158:head# 2024-09-11T07:50:09.823 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8700b47:::benchmark_data_smithi022_188626_object101613:head# 2024-09-11T07:50:09.823 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c87325ba:::benchmark_data_smithi022_188626_object111051:head# 2024-09-11T07:50:09.823 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c874581d:::benchmark_data_smithi022_188626_object92630:head# 2024-09-11T07:50:09.823 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c875554f:::benchmark_data_smithi022_188626_object90369:head# 2024-09-11T07:50:09.824 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c877cea2:::benchmark_data_smithi022_188626_object106124:head# 2024-09-11T07:50:09.824 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c87905fc:::benchmark_data_smithi022_188626_object114979:head# 2024-09-11T07:50:09.824 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c87f1a6a:::benchmark_data_smithi022_188626_object116983:head# 2024-09-11T07:50:09.824 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8813bc3:::benchmark_data_smithi022_188626_object95893:head# 2024-09-11T07:50:09.824 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c883f059:::benchmark_data_smithi022_188626_object101687:head# 2024-09-11T07:50:09.824 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8841cff:::benchmark_data_smithi022_188626_object99708:head# 2024-09-11T07:50:09.824 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c887ee11:::benchmark_data_smithi022_188626_object92972:head# 2024-09-11T07:50:09.824 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c88ed973:::benchmark_data_smithi022_188626_object117525:head# 2024-09-11T07:50:09.824 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8906acc:::benchmark_data_smithi022_188626_object94185:head# 2024-09-11T07:50:09.824 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8928a2d:::benchmark_data_smithi022_188626_object96879:head# 2024-09-11T07:50:09.824 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c89323d6:::benchmark_data_smithi022_188626_object99233:head# 2024-09-11T07:50:09.824 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c893a6c2:::benchmark_data_smithi022_188626_object89953:head# 2024-09-11T07:50:09.825 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8984d55:::benchmark_data_smithi022_188626_object99293:head# 2024-09-11T07:50:09.825 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c89a9b18:::benchmark_data_smithi022_188626_object112738:head# 2024-09-11T07:50:09.825 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c89d1842:::benchmark_data_smithi022_188626_object95545:head# 2024-09-11T07:50:09.825 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c89e7168:::benchmark_data_smithi022_188626_object93290:head# 2024-09-11T07:50:09.825 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c89e8bdd:::benchmark_data_smithi022_188626_object113545:head# 2024-09-11T07:50:09.825 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8a1d41e:::benchmark_data_smithi022_188626_object95853:head# 2024-09-11T07:50:09.825 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8a1e524:::benchmark_data_smithi022_188626_object107052:head# 2024-09-11T07:50:09.825 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8a39144:::benchmark_data_smithi022_188626_object88980:head# 2024-09-11T07:50:09.825 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8a4043d:::benchmark_data_smithi022_188626_object110899:head# 2024-09-11T07:50:09.825 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8a4f0f8:::benchmark_data_smithi022_188626_object116207:head# 2024-09-11T07:50:09.825 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8a50d13:::benchmark_data_smithi022_188626_object116644:head# 2024-09-11T07:50:09.825 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8aa5b6e:::benchmark_data_smithi022_188626_object119400:head# 2024-09-11T07:50:09.825 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8aa6472:::benchmark_data_smithi022_188626_object117290:head# 2024-09-11T07:50:09.826 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8acaa80:::benchmark_data_smithi022_188626_object110022:head# 2024-09-11T07:50:09.826 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8b16402:::benchmark_data_smithi022_188626_object111231:head# 2024-09-11T07:50:09.826 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8b4452c:::benchmark_data_smithi022_188626_object100719:head# 2024-09-11T07:50:09.826 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8b743a3:::benchmark_data_smithi022_188626_object101745:head# 2024-09-11T07:50:09.826 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8b8e0b0:::benchmark_data_smithi022_188626_object107354:head# 2024-09-11T07:50:09.826 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8b9b566:::benchmark_data_smithi022_188626_object104865:head# 2024-09-11T07:50:09.826 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8bb82e5:::benchmark_data_smithi022_188626_object96361:head# 2024-09-11T07:50:09.826 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8bc4751:::benchmark_data_smithi022_188626_object118109:head# 2024-09-11T07:50:09.826 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8bddf59:::benchmark_data_smithi022_188626_object118187:head# 2024-09-11T07:50:09.826 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8c30a0f:::benchmark_data_smithi022_188626_object92684:head# 2024-09-11T07:50:09.826 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8c44808:::benchmark_data_smithi022_188626_object119337:head# 2024-09-11T07:50:09.826 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8d02655:::benchmark_data_smithi022_188626_object114978:head# 2024-09-11T07:50:09.826 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8d194e3:::benchmark_data_smithi022_188626_object92981:head# 2024-09-11T07:50:09.827 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8d5991c:::benchmark_data_smithi022_188626_object103764:head# 2024-09-11T07:50:09.827 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8d66c07:::benchmark_data_smithi022_188626_object107476:head# 2024-09-11T07:50:09.827 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8d772b1:::benchmark_data_smithi022_188626_object108168:head# 2024-09-11T07:50:09.827 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8debf68:::benchmark_data_smithi022_188626_object102243:head# 2024-09-11T07:50:09.827 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8e0007e:::benchmark_data_smithi022_188626_object101787:head# 2024-09-11T07:50:09.827 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8e2bcc4:::benchmark_data_smithi022_188626_object112662:head# 2024-09-11T07:50:09.827 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8e301ed:::benchmark_data_smithi022_188626_object115121:head# 2024-09-11T07:50:09.827 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8e386d0:::benchmark_data_smithi022_188626_object100268:head# 2024-09-11T07:50:09.827 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8e4deb3:::benchmark_data_smithi022_188626_object107319:head# 2024-09-11T07:50:09.827 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8e9e944:::benchmark_data_smithi022_188626_object105529:head# 2024-09-11T07:50:09.827 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8e9ec14:::benchmark_data_smithi022_188626_object112208:head# 2024-09-11T07:50:09.827 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8ea8529:::benchmark_data_smithi022_188626_object102675:head# 2024-09-11T07:50:09.828 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8ecf365:::benchmark_data_smithi022_188626_object90019:head# 2024-09-11T07:50:09.828 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8ed9080:::benchmark_data_smithi022_188626_object103536:head# 2024-09-11T07:50:09.828 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8f062b7:::benchmark_data_smithi022_188626_object107013:head# 2024-09-11T07:50:09.828 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8f14a2e:::benchmark_data_smithi022_188626_object116894:head# 2024-09-11T07:50:09.828 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8f8951b:::benchmark_data_smithi022_188626_object113926:head# 2024-09-11T07:50:09.828 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8f8bffc:::benchmark_data_smithi022_188626_object94800:head# 2024-09-11T07:50:09.828 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c8faada1:::benchmark_data_smithi022_188626_object117817:head# 2024-09-11T07:50:09.828 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9002398:::benchmark_data_smithi022_188626_object92751:head# 2024-09-11T07:50:09.828 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9008837:::benchmark_data_smithi022_188626_object89074:head# 2024-09-11T07:50:09.828 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c90626d7:::benchmark_data_smithi022_188626_object101580:head# 2024-09-11T07:50:09.828 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9080419:::benchmark_data_smithi022_188626_object118369:head# 2024-09-11T07:50:09.887 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9094d67:::benchmark_data_smithi022_188626_object107465:head# 2024-09-11T07:50:09.887 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c90bd4f5:::benchmark_data_smithi022_188626_object96550:head# 2024-09-11T07:50:09.887 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c915797d:::benchmark_data_smithi022_188626_object94799:head# 2024-09-11T07:50:09.888 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c916192f:::benchmark_data_smithi022_188626_object111336:head# 2024-09-11T07:50:09.888 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9169a47:::benchmark_data_smithi022_188626_object97406:head# 2024-09-11T07:50:09.888 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c917cb3a:::benchmark_data_smithi022_188626_object109096:head# 2024-09-11T07:50:09.888 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c91a7394:::benchmark_data_smithi022_188626_object90438:head# 2024-09-11T07:50:09.888 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c91f5132:::benchmark_data_smithi022_188626_object118070:head# 2024-09-11T07:50:09.888 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c922d8c9:::benchmark_data_smithi022_188626_object114325:head# 2024-09-11T07:50:09.888 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c924b093:::benchmark_data_smithi022_188626_object116190:head# 2024-09-11T07:50:09.888 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c925ead5:::benchmark_data_smithi022_188626_object89943:head# 2024-09-11T07:50:09.888 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9266746:::benchmark_data_smithi022_188626_object98121:head# 2024-09-11T07:50:09.888 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c92d5f30:::benchmark_data_smithi022_188626_object95206:head# 2024-09-11T07:50:09.888 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c934c4fa:::benchmark_data_smithi022_188626_object92295:head# 2024-09-11T07:50:09.889 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9384379:::benchmark_data_smithi022_188626_object117867:head# 2024-09-11T07:50:09.889 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c93ccfe4:::benchmark_data_smithi022_188626_object105824:head# 2024-09-11T07:50:09.889 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c93db533:::benchmark_data_smithi022_188626_object111037:head# 2024-09-11T07:50:09.889 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c93ff77d:::benchmark_data_smithi022_188626_object117663:head# 2024-09-11T07:50:09.889 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c941c1a8:::benchmark_data_smithi022_188626_object101002:head# 2024-09-11T07:50:09.889 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9431344:::benchmark_data_smithi022_188626_object117955:head# 2024-09-11T07:50:09.889 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c94831cc:::benchmark_data_smithi022_188626_object91392:head# 2024-09-11T07:50:09.889 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c949ecfc:::benchmark_data_smithi022_188626_object106609:head# 2024-09-11T07:50:09.889 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c94ac7ab:::benchmark_data_smithi022_188626_object115367:head# 2024-09-11T07:50:09.889 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c94d1c17:::benchmark_data_smithi022_188626_object103426:head# 2024-09-11T07:50:09.889 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c94eac16:::benchmark_data_smithi022_188626_object119011:head# 2024-09-11T07:50:09.890 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c94f58de:::benchmark_data_smithi022_188626_object91142:head# 2024-09-11T07:50:09.890 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c95026bd:::benchmark_data_smithi022_188626_object112176:head# 2024-09-11T07:50:09.890 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c951784c:::benchmark_data_smithi022_188626_object94473:head# 2024-09-11T07:50:09.890 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c95a448a:::benchmark_data_smithi022_188626_object99076:head# 2024-09-11T07:50:09.890 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c95a81ab:::benchmark_data_smithi022_188626_object89617:head# 2024-09-11T07:50:09.890 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c95b1bd4:::benchmark_data_smithi022_188626_object108272:head# 2024-09-11T07:50:09.890 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c95f885f:::benchmark_data_smithi022_188626_object93075:head# 2024-09-11T07:50:09.890 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c961302f:::benchmark_data_smithi022_188626_object107270:head# 2024-09-11T07:50:09.890 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c963abe7:::benchmark_data_smithi022_188626_object93288:head# 2024-09-11T07:50:09.890 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9659ada:::benchmark_data_smithi022_188626_object101734:head# 2024-09-11T07:50:09.890 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c965c8dc:::benchmark_data_smithi022_188626_object91233:head# 2024-09-11T07:50:09.891 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c965df3d:::benchmark_data_smithi022_188626_object106766:head# 2024-09-11T07:50:09.891 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c966c7aa:::benchmark_data_smithi022_188626_object89779:head# 2024-09-11T07:50:09.891 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c96735c3:::benchmark_data_smithi022_188626_object95138:head# 2024-09-11T07:50:09.891 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c970f576:::benchmark_data_smithi022_188626_object89092:head# 2024-09-11T07:50:09.891 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c972420b:::benchmark_data_smithi022_188626_object90075:head# 2024-09-11T07:50:09.891 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c972cc1c:::benchmark_data_smithi022_188626_object112363:head# 2024-09-11T07:50:09.891 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c97ee370:::benchmark_data_smithi022_188626_object102785:head# 2024-09-11T07:50:09.891 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c983fe16:::benchmark_data_smithi022_188626_object94230:head# 2024-09-11T07:50:09.891 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c987531b:::benchmark_data_smithi022_188626_object111766:head# 2024-09-11T07:50:09.891 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9876891:::benchmark_data_smithi022_188626_object104009:head# 2024-09-11T07:50:09.891 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9883449:::benchmark_data_smithi022_188626_object117890:head# 2024-09-11T07:50:09.891 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c98bbb50:::benchmark_data_smithi022_188626_object111697:head# 2024-09-11T07:50:09.892 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c98fa529:::benchmark_data_smithi022_188626_object116763:head# 2024-09-11T07:50:09.892 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9907b80:::benchmark_data_smithi022_188626_object118113:head# 2024-09-11T07:50:09.892 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9911809:::benchmark_data_smithi022_188626_object119341:head# 2024-09-11T07:50:09.892 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c994680f:::benchmark_data_smithi022_188626_object104580:head# 2024-09-11T07:50:09.892 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c99614a9:::benchmark_data_smithi022_188626_object94823:head# 2024-09-11T07:50:09.892 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c99903d0:::benchmark_data_smithi022_188626_object116965:head# 2024-09-11T07:50:09.892 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c99f4292:::benchmark_data_smithi022_188626_object105919:head# 2024-09-11T07:50:09.892 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9a0e882:::benchmark_data_smithi022_188626_object103307:head# 2024-09-11T07:50:09.892 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9a1580f:::benchmark_data_smithi022_188626_object118956:head# 2024-09-11T07:50:09.893 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9a26d24:::benchmark_data_smithi022_188626_object95835:head# 2024-09-11T07:50:09.893 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9a36dee:::benchmark_data_smithi022_188626_object105224:head# 2024-09-11T07:50:09.893 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9a976b4:::benchmark_data_smithi022_188626_object109737:head# 2024-09-11T07:50:09.893 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9ab74fd:::benchmark_data_smithi022_188626_object99247:head# 2024-09-11T07:50:09.893 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9ad6662:::benchmark_data_smithi022_188626_object110224:head# 2024-09-11T07:50:09.893 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9b1fa83:::benchmark_data_smithi022_188626_object104179:head# 2024-09-11T07:50:09.893 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9b781b5:::benchmark_data_smithi022_188626_object110033:head# 2024-09-11T07:50:09.893 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9b922bc:::benchmark_data_smithi022_188626_object113316:head# 2024-09-11T07:50:09.893 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9b9dbfb:::benchmark_data_smithi022_188626_object113609:head# 2024-09-11T07:50:09.893 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9c27de5:::benchmark_data_smithi022_188626_object98775:head# 2024-09-11T07:50:09.893 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9c330b1:::benchmark_data_smithi022_188626_object92970:head# 2024-09-11T07:50:09.893 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9c4beda:::benchmark_data_smithi022_188626_object103917:head# 2024-09-11T07:50:09.894 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9c9123f:::benchmark_data_smithi022_188626_object113033:head# 2024-09-11T07:50:09.894 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9cac8c3:::benchmark_data_smithi022_188626_object114764:head# 2024-09-11T07:50:09.894 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9cb8e16:::benchmark_data_smithi022_188626_object112817:head# 2024-09-11T07:50:09.894 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9cba5e8:::benchmark_data_smithi022_188626_object99454:head# 2024-09-11T07:50:09.894 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9cef085:::benchmark_data_smithi022_188626_object110837:head# 2024-09-11T07:50:09.894 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9cf99d2:::benchmark_data_smithi022_188626_object97360:head# 2024-09-11T07:50:09.894 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9d2a0ff:::benchmark_data_smithi022_188626_object107077:head# 2024-09-11T07:50:09.894 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9d2e9a1:::benchmark_data_smithi022_188626_object100105:head# 2024-09-11T07:50:09.894 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9d305b2:::benchmark_data_smithi022_188626_object118410:head# 2024-09-11T07:50:09.894 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9d3cc32:::benchmark_data_smithi022_188626_object112987:head# 2024-09-11T07:50:09.894 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9d4bbb3:::benchmark_data_smithi022_188626_object96200:head# 2024-09-11T07:50:09.895 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9d53a91:::benchmark_data_smithi022_188626_object118975:head# 2024-09-11T07:50:09.895 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9d5c2aa:::benchmark_data_smithi022_188626_object107771:head# 2024-09-11T07:50:09.895 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9d6e44f:::benchmark_data_smithi022_188626_object112065:head# 2024-09-11T07:50:09.895 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9d76e7a:::benchmark_data_smithi022_188626_object99395:head# 2024-09-11T07:50:09.895 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9d7eb08:::benchmark_data_smithi022_188626_object89482:head# 2024-09-11T07:50:09.895 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9d9e358:::benchmark_data_smithi022_188626_object91916:head# 2024-09-11T07:50:09.895 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9df02b8:::benchmark_data_smithi022_188626_object119215:head# 2024-09-11T07:50:09.895 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9df6ce3:::benchmark_data_smithi022_188626_object111053:head# 2024-09-11T07:50:09.895 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9eb8e2e:::benchmark_data_smithi022_188626_object100616:head# 2024-09-11T07:50:09.895 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9ec3a5c:::benchmark_data_smithi022_188626_object118915:head# 2024-09-11T07:50:09.895 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9ec9b7a:::benchmark_data_smithi022_188626_object91838:head# 2024-09-11T07:50:09.895 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9ee324a:::benchmark_data_smithi022_188626_object105062:head# 2024-09-11T07:50:09.896 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9f08015:::benchmark_data_smithi022_188626_object106156:head# 2024-09-11T07:50:09.896 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9f77bb6:::benchmark_data_smithi022_188626_object118808:head# 2024-09-11T07:50:09.896 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:c9f8ab26:::benchmark_data_smithi022_188626_object106859:head# 2024-09-11T07:50:09.896 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca00b480:::benchmark_data_smithi022_188626_object106562:head# 2024-09-11T07:50:09.896 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca01e022:::benchmark_data_smithi022_188626_object98497:head# 2024-09-11T07:50:09.896 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca034c03:::benchmark_data_smithi022_188626_object106123:head# 2024-09-11T07:50:09.896 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca046803:::benchmark_data_smithi022_188626_object100615:head# 2024-09-11T07:50:09.896 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca07e582:::benchmark_data_smithi022_188626_object91268:head# 2024-09-11T07:50:09.896 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca0b9375:::benchmark_data_smithi022_188626_object115135:head# 2024-09-11T07:50:09.896 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca0bea72:::benchmark_data_smithi022_188626_object105767:head# 2024-09-11T07:50:09.896 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca1005c5:::benchmark_data_smithi022_188626_object94405:head# 2024-09-11T07:50:09.896 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca162ef0:::benchmark_data_smithi022_188626_object114528:head# 2024-09-11T07:50:09.897 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca171841:::benchmark_data_smithi022_188626_object89894:head# 2024-09-11T07:50:09.897 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca1a757b:::benchmark_data_smithi022_188626_object117800:head# 2024-09-11T07:50:09.897 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca1be4a4:::benchmark_data_smithi022_188626_object109387:head# 2024-09-11T07:50:09.897 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca1d3693:::benchmark_data_smithi022_188626_object96110:head# 2024-09-11T07:50:09.897 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca1d50a0:::benchmark_data_smithi022_188626_object98478:head# 2024-09-11T07:50:09.897 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca1fdd5b:::benchmark_data_smithi022_188626_object96495:head# 2024-09-11T07:50:09.897 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca206ce2:::benchmark_data_smithi022_188626_object95234:head# 2024-09-11T07:50:09.897 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca254576:::benchmark_data_smithi022_188626_object115852:head# 2024-09-11T07:50:09.897 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca25e1f8:::benchmark_data_smithi022_188626_object101898:head# 2024-09-11T07:50:09.897 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca2940f1:::benchmark_data_smithi022_188626_object107463:head# 2024-09-11T07:50:09.897 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca29de74:::benchmark_data_smithi022_188626_object94900:head# 2024-09-11T07:50:09.897 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca2b7206:::benchmark_data_smithi022_188626_object97878:head# 2024-09-11T07:50:09.897 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca2f235c:::benchmark_data_smithi022_188626_object92272:head# 2024-09-11T07:50:09.898 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca305e68:::benchmark_data_smithi022_188626_object100175:head# 2024-09-11T07:50:09.898 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca33b317:::benchmark_data_smithi022_188626_object115460:head# 2024-09-11T07:50:09.898 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca342de6:::benchmark_data_smithi022_188626_object101701:head# 2024-09-11T07:50:09.898 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca36cc71:::benchmark_data_smithi022_188626_object110989:head# 2024-09-11T07:50:09.898 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca39510e:::benchmark_data_smithi022_188626_object104698:head# 2024-09-11T07:50:09.898 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca3a29d8:::benchmark_data_smithi022_188626_object104331:head# 2024-09-11T07:50:09.898 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca3b8c76:::benchmark_data_smithi022_188626_object101523:head# 2024-09-11T07:50:09.898 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca420902:::benchmark_data_smithi022_188626_object96390:head# 2024-09-11T07:50:09.898 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca4d4834:::benchmark_data_smithi022_188626_object90433:head# 2024-09-11T07:50:09.898 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca4ec4ae:::benchmark_data_smithi022_188626_object112580:head# 2024-09-11T07:50:09.898 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca4fadd7:::benchmark_data_smithi022_188626_object113118:head# 2024-09-11T07:50:09.898 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca542884:::benchmark_data_smithi022_188626_object95134:head# 2024-09-11T07:50:09.898 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca548356:::benchmark_data_smithi022_188626_object113078:head# 2024-09-11T07:50:09.898 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca563e5c:::benchmark_data_smithi022_188626_object95429:head# 2024-09-11T07:50:09.898 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca5a5d1f:::benchmark_data_smithi022_188626_object97488:head# 2024-09-11T07:50:09.899 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca5ce47f:::benchmark_data_smithi022_188626_object118362:head# 2024-09-11T07:50:09.899 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca5d99be:::benchmark_data_smithi022_188626_object106431:head# 2024-09-11T07:50:09.899 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca5df4c1:::benchmark_data_smithi022_188626_object94197:head# 2024-09-11T07:50:09.899 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca5e003a:::benchmark_data_smithi022_188626_object90511:head# 2024-09-11T07:50:09.899 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca608123:::benchmark_data_smithi022_188626_object101451:head# 2024-09-11T07:50:09.899 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca61b672:::benchmark_data_smithi022_188626_object114684:head# 2024-09-11T07:50:09.899 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca61eaba:::benchmark_data_smithi022_188626_object107726:head# 2024-09-11T07:50:09.899 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca634796:::benchmark_data_smithi022_188626_object95250:head# 2024-09-11T07:50:09.899 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca63815a:::benchmark_data_smithi022_188626_object110737:head# 2024-09-11T07:50:09.899 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca6687e6:::benchmark_data_smithi022_188626_object115342:head# 2024-09-11T07:50:09.899 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca669bce:::benchmark_data_smithi022_188626_object90628:head# 2024-09-11T07:50:09.899 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca669fb4:::benchmark_data_smithi022_188626_object98703:head# 2024-09-11T07:50:09.899 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca67850a:::benchmark_data_smithi022_188626_object98133:head# 2024-09-11T07:50:09.899 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca6b1db7:::benchmark_data_smithi022_188626_object94505:head# 2024-09-11T07:50:09.900 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca6c31cf:::benchmark_data_smithi022_188626_object115574:head# 2024-09-11T07:50:09.900 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca70e34e:::benchmark_data_smithi022_188626_object107432:head# 2024-09-11T07:50:09.900 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca7387dd:::benchmark_data_smithi022_188626_object92457:head# 2024-09-11T07:50:09.900 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca7417b6:::benchmark_data_smithi022_188626_object105339:head# 2024-09-11T07:50:09.900 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca758863:::benchmark_data_smithi022_188626_object103473:head# 2024-09-11T07:50:09.900 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca76021d:::benchmark_data_smithi022_188626_object105987:head# 2024-09-11T07:50:09.900 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca7ad43e:::benchmark_data_smithi022_188626_object108313:head# 2024-09-11T07:50:09.900 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca7c97f7:::benchmark_data_smithi022_188626_object98955:head# 2024-09-11T07:50:09.900 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca7d8e85:::benchmark_data_smithi022_188626_object100022:head# 2024-09-11T07:50:09.900 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca7e8992:::benchmark_data_smithi022_188626_object89483:head# 2024-09-11T07:50:09.900 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca7ed9ff:::benchmark_data_smithi022_188626_object106890:head# 2024-09-11T07:50:09.900 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca81f981:::benchmark_data_smithi022_188626_object104490:head# 2024-09-11T07:50:09.900 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca82e41c:::benchmark_data_smithi022_188626_object95053:head# 2024-09-11T07:50:09.900 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca85e9e2:::benchmark_data_smithi022_188626_object93485:head# 2024-09-11T07:50:09.901 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca86a73a:::benchmark_data_smithi022_188626_object101554:head# 2024-09-11T07:50:09.901 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca8f6787:::benchmark_data_smithi022_188626_object114626:head# 2024-09-11T07:50:09.901 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca9b9215:::benchmark_data_smithi022_188626_object91301:head# 2024-09-11T07:50:09.901 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ca9f17cb:::benchmark_data_smithi022_188626_object117752:head# 2024-09-11T07:50:09.901 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:caa283e1:::benchmark_data_smithi022_188626_object94922:head# 2024-09-11T07:50:09.901 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:caa2e53f:::benchmark_data_smithi022_188626_object110872:head# 2024-09-11T07:50:09.901 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:caa4954f:::benchmark_data_smithi022_188626_object99313:head# 2024-09-11T07:50:09.901 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:caa7361f:::benchmark_data_smithi022_188626_object90462:head# 2024-09-11T07:50:09.901 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:caa9ae76:::benchmark_data_smithi022_188626_object115296:head# 2024-09-11T07:50:09.901 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:caa9e9a0:::benchmark_data_smithi022_188626_object98273:head# 2024-09-11T07:50:09.901 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:caaa6527:::benchmark_data_smithi022_188626_object119416:head# 2024-09-11T07:50:09.901 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:caaa8497:::benchmark_data_smithi022_188626_object97925:head# 2024-09-11T07:50:09.901 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:caabd119:::benchmark_data_smithi022_188626_object109030:head# 2024-09-11T07:50:09.901 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cab0657e:::benchmark_data_smithi022_188626_object118000:head# 2024-09-11T07:50:09.902 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cab31c05:::benchmark_data_smithi022_188626_object95631:head# 2024-09-11T07:50:09.902 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cab96ef1:::benchmark_data_smithi022_188626_object118496:head# 2024-09-11T07:50:09.902 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:caba4148:::benchmark_data_smithi022_188626_object97529:head# 2024-09-11T07:50:09.902 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cabacf7a:::benchmark_data_smithi022_188626_object103830:head# 2024-09-11T07:50:09.902 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cabf697d:::benchmark_data_smithi022_188626_object97277:head# 2024-09-11T07:50:09.902 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cac01146:::benchmark_data_smithi022_188626_object107869:head# 2024-09-11T07:50:09.902 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cac19641:::benchmark_data_smithi022_188626_object101607:head# 2024-09-11T07:50:09.902 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cac43cf3:::benchmark_data_smithi022_188626_object118401:head# 2024-09-11T07:50:09.902 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cac53fd4:::benchmark_data_smithi022_188626_object113658:head# 2024-09-11T07:50:09.902 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cac62927:::benchmark_data_smithi022_188626_object92961:head# 2024-09-11T07:50:09.902 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:caca8a15:::benchmark_data_smithi022_188626_object108423:head# 2024-09-11T07:50:09.902 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cacaa2b6:::benchmark_data_smithi022_188626_object94872:head# 2024-09-11T07:50:09.902 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cacb0ef3:::benchmark_data_smithi022_188626_object115230:head# 2024-09-11T07:50:09.902 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cacbd2e5:::benchmark_data_smithi022_188626_object116472:head# 2024-09-11T07:50:09.903 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cacdde5c:::benchmark_data_smithi022_188626_object99311:head# 2024-09-11T07:50:09.903 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cace2df3:::benchmark_data_smithi022_188626_object105496:head# 2024-09-11T07:50:09.903 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cacf4b45:::benchmark_data_smithi022_188626_object105236:head# 2024-09-11T07:50:09.903 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cad2abeb:::benchmark_data_smithi022_188626_object100445:head# 2024-09-11T07:50:09.903 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cad3ab41:::benchmark_data_smithi022_188626_object112414:head# 2024-09-11T07:50:09.903 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cad6484f:::benchmark_data_smithi022_188626_object109055:head# 2024-09-11T07:50:09.903 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cad81d7c:::benchmark_data_smithi022_188626_object98135:head# 2024-09-11T07:50:09.903 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cad890e2:::benchmark_data_smithi022_188626_object99828:head# 2024-09-11T07:50:09.903 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cadc65c4:::benchmark_data_smithi022_188626_object108281:head# 2024-09-11T07:50:09.903 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cadf82af:::benchmark_data_smithi022_188626_object118588:head# 2024-09-11T07:50:09.903 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cae08003:::benchmark_data_smithi022_188626_object95719:head# 2024-09-11T07:50:09.903 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cae16a9a:::benchmark_data_smithi022_188626_object90471:head# 2024-09-11T07:50:09.903 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cae34ddf:::benchmark_data_smithi022_188626_object108351:head# 2024-09-11T07:50:09.903 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cae35d50:::benchmark_data_smithi022_188626_object109181:head# 2024-09-11T07:50:09.903 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cae5f1fb:::benchmark_data_smithi022_188626_object90230:head# 2024-09-11T07:50:09.904 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cae623e2:::benchmark_data_smithi022_188626_object111333:head# 2024-09-11T07:50:09.904 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cae8da82:::benchmark_data_smithi022_188626_object113691:head# 2024-09-11T07:50:09.904 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cae9b492:::benchmark_data_smithi022_188626_object102860:head# 2024-09-11T07:50:09.904 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cae9d3f7:::benchmark_data_smithi022_188626_object90759:head# 2024-09-11T07:50:09.904 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:caeb0923:::benchmark_data_smithi022_188626_object93138:head# 2024-09-11T07:50:09.904 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:caeb8855:::benchmark_data_smithi022_188626_object104157:head# 2024-09-11T07:50:09.904 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:caefa077:::benchmark_data_smithi022_188626_object92155:head# 2024-09-11T07:50:09.904 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:caf0bd61:::benchmark_data_smithi022_188626_object98884:head# 2024-09-11T07:50:09.904 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:caf2b7f6:::benchmark_data_smithi022_188626_object115672:head# 2024-09-11T07:50:09.904 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:caf70a35:::benchmark_data_smithi022_188626_object103477:head# 2024-09-11T07:50:09.904 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:caf7565e:::benchmark_data_smithi022_188626_object99523:head# 2024-09-11T07:50:09.904 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:caf82fe8:::benchmark_data_smithi022_188626_object103587:head# 2024-09-11T07:50:09.904 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:caf860b6:::benchmark_data_smithi022_188626_object104920:head# 2024-09-11T07:50:09.904 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:caf861c3:::benchmark_data_smithi022_188626_object105877:head# 2024-09-11T07:50:09.904 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:caf89de4:::benchmark_data_smithi022_188626_object107618:head# 2024-09-11T07:50:09.905 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cafbbe4e:::benchmark_data_smithi022_188626_object114241:head# 2024-09-11T07:50:09.905 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cafcbfc8:::benchmark_data_smithi022_188626_object91270:head# 2024-09-11T07:50:09.905 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cafe1d16:::benchmark_data_smithi022_188626_object106408:head# 2024-09-11T07:50:09.905 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:caffb15f:::benchmark_data_smithi022_188626_object99103:head# 2024-09-11T07:50:09.905 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb03bb6a:::benchmark_data_smithi022_188626_object100500:head# 2024-09-11T07:50:09.905 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb062c0d:::benchmark_data_smithi022_188626_object115911:head# 2024-09-11T07:50:09.905 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb07c379:::benchmark_data_smithi022_188626_object101317:head# 2024-09-11T07:50:09.905 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb0b73a0:::benchmark_data_smithi022_188626_object109491:head# 2024-09-11T07:50:09.905 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb0f5e0f:::benchmark_data_smithi022_188626_object90998:head# 2024-09-11T07:50:09.905 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb1052c2:::benchmark_data_smithi022_188626_object91219:head# 2024-09-11T07:50:09.905 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb128008:::benchmark_data_smithi022_188626_object117146:head# 2024-09-11T07:50:09.905 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb133c82:::benchmark_data_smithi022_188626_object96887:head# 2024-09-11T07:50:09.905 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb133fba:::benchmark_data_smithi022_188626_object108688:head# 2024-09-11T07:50:09.905 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb13cc35:::benchmark_data_smithi022_188626_object95077:head# 2024-09-11T07:50:09.905 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb1982e4:::benchmark_data_smithi022_188626_object96180:head# 2024-09-11T07:50:09.906 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb1af136:::benchmark_data_smithi022_188626_object91915:head# 2024-09-11T07:50:09.906 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb1d8805:::benchmark_data_smithi022_188626_object104616:head# 2024-09-11T07:50:09.906 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb210c25:::benchmark_data_smithi022_188626_object101710:head# 2024-09-11T07:50:09.906 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb251c92:::benchmark_data_smithi022_188626_object110204:head# 2024-09-11T07:50:09.906 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb25de02:::benchmark_data_smithi022_188626_object93195:head# 2024-09-11T07:50:09.906 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb276767:::benchmark_data_smithi022_188626_object111141:head# 2024-09-11T07:50:09.906 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb28817c:::benchmark_data_smithi022_188626_object89857:head# 2024-09-11T07:50:09.906 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb29c893:::benchmark_data_smithi022_188626_object91598:head# 2024-09-11T07:50:09.906 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb2a53cc:::benchmark_data_smithi022_188626_object114711:head# 2024-09-11T07:50:09.906 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb2b4a71:::benchmark_data_smithi022_188626_object116661:head# 2024-09-11T07:50:09.906 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb2dbdc9:::benchmark_data_smithi022_188626_object112162:head# 2024-09-11T07:50:09.906 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb2dec85:::benchmark_data_smithi022_188626_object115811:head# 2024-09-11T07:50:09.906 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb2f4c37:::benchmark_data_smithi022_188626_object101845:head# 2024-09-11T07:50:09.907 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb2fe4e8:::benchmark_data_smithi022_188626_object109539:head# 2024-09-11T07:50:09.907 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb31868a:::benchmark_data_smithi022_188626_object103175:head# 2024-09-11T07:50:09.907 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb328d97:::benchmark_data_smithi022_188626_object113569:head# 2024-09-11T07:50:09.907 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb33914b:::benchmark_data_smithi022_188626_object99906:head# 2024-09-11T07:50:09.907 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb379a83:::benchmark_data_smithi022_188626_object90982:head# 2024-09-11T07:50:09.907 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb37a82a:::benchmark_data_smithi022_188626_object92982:head# 2024-09-11T07:50:09.907 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb387d45:::benchmark_data_smithi022_188626_object103978:head# 2024-09-11T07:50:09.907 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb390636:::benchmark_data_smithi022_188626_object110982:head# 2024-09-11T07:50:09.907 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb3922fd:::benchmark_data_smithi022_188626_object100630:head# 2024-09-11T07:50:09.907 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb393549:::benchmark_data_smithi022_188626_object110059:head# 2024-09-11T07:50:09.907 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb395683:::benchmark_data_smithi022_188626_object117306:head# 2024-09-11T07:50:09.907 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb39bca7:::benchmark_data_smithi022_188626_object105538:head# 2024-09-11T07:50:09.908 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb3ae216:::benchmark_data_smithi022_188626_object91483:head# 2024-09-11T07:50:09.908 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb3b97cd:::benchmark_data_smithi022_188626_object100561:head# 2024-09-11T07:50:09.908 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb3c34b8:::benchmark_data_smithi022_188626_object105605:head# 2024-09-11T07:50:09.908 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb3dfe47:::benchmark_data_smithi022_188626_object102173:head# 2024-09-11T07:50:09.908 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb3eb04b:::benchmark_data_smithi022_188626_object96472:head# 2024-09-11T07:50:09.908 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb3ff1e7:::benchmark_data_smithi022_188626_object98307:head# 2024-09-11T07:50:09.908 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb4fbc7c:::benchmark_data_smithi022_188626_object110192:head# 2024-09-11T07:50:09.908 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb514286:::benchmark_data_smithi022_188626_object99470:head# 2024-09-11T07:50:09.908 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb52ecb3:::benchmark_data_smithi022_188626_object118442:head# 2024-09-11T07:50:09.908 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb540dcf:::benchmark_data_smithi022_188626_object104341:head# 2024-09-11T07:50:09.908 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb54ea47:::benchmark_data_smithi022_188626_object91048:head# 2024-09-11T07:50:09.909 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb5a5ee6:::benchmark_data_smithi022_188626_object103167:head# 2024-09-11T07:50:09.909 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb5b27be:::benchmark_data_smithi022_188626_object109957:head# 2024-09-11T07:50:09.909 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb5bf50d:::benchmark_data_smithi022_188626_object113335:head# 2024-09-11T07:50:09.909 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb5c27ca:::benchmark_data_smithi022_188626_object109462:head# 2024-09-11T07:50:09.909 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb5da31a:::benchmark_data_smithi022_188626_object92670:head# 2024-09-11T07:50:09.909 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb5f3b48:::benchmark_data_smithi022_188626_object108971:head# 2024-09-11T07:50:09.909 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb62d5c0:::benchmark_data_smithi022_188626_object114154:head# 2024-09-11T07:50:09.909 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb65c403:::benchmark_data_smithi022_188626_object104495:head# 2024-09-11T07:50:09.909 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb6766eb:::benchmark_data_smithi022_188626_object118485:head# 2024-09-11T07:50:09.909 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb6acf44:::benchmark_data_smithi022_188626_object113396:head# 2024-09-11T07:50:09.909 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb6ee18f:::benchmark_data_smithi022_188626_object112945:head# 2024-09-11T07:50:09.909 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb70fdc2:::benchmark_data_smithi022_188626_object98369:head# 2024-09-11T07:50:09.910 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb71fcde:::benchmark_data_smithi022_188626_object105829:head# 2024-09-11T07:50:09.910 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb732052:::benchmark_data_smithi022_188626_object90014:head# 2024-09-11T07:50:09.910 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb751875:::benchmark_data_smithi022_188626_object110310:head# 2024-09-11T07:50:09.910 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb75488a:::benchmark_data_smithi022_188626_object93739:head# 2024-09-11T07:50:09.910 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb7558e4:::benchmark_data_smithi022_188626_object107536:head# 2024-09-11T07:50:09.910 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb767fff:::benchmark_data_smithi022_188626_object101714:head# 2024-09-11T07:50:09.910 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb77f073:::benchmark_data_smithi022_188626_object111428:head# 2024-09-11T07:50:09.910 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb79d1bc:::benchmark_data_smithi022_188626_object111970:head# 2024-09-11T07:50:09.910 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb7a40cd:::benchmark_data_smithi022_188626_object107712:head# 2024-09-11T07:50:09.910 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb7bc195:::benchmark_data_smithi022_188626_object95062:head# 2024-09-11T07:50:09.910 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb7bfca6:::benchmark_data_smithi022_188626_object113414:head# 2024-09-11T07:50:09.911 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb7dfdaf:::benchmark_data_smithi022_188626_object103614:head# 2024-09-11T07:50:09.911 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb7fd822:::benchmark_data_smithi022_188626_object108152:head# 2024-09-11T07:50:09.911 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb86000e:::benchmark_data_smithi022_188626_object89394:head# 2024-09-11T07:50:09.911 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb86191a:::benchmark_data_smithi022_188626_object95046:head# 2024-09-11T07:50:09.911 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb883c03:::benchmark_data_smithi022_188626_object118324:head# 2024-09-11T07:50:09.911 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb89024f:::benchmark_data_smithi022_188626_object109347:head# 2024-09-11T07:50:09.911 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb896d0a:::benchmark_data_smithi022_188626_object104177:head# 2024-09-11T07:50:09.911 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb8a18e5:::benchmark_data_smithi022_188626_object89656:head# 2024-09-11T07:50:09.911 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb8b1a65:::benchmark_data_smithi022_188626_object111412:head# 2024-09-11T07:50:09.911 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb8d05d7:::benchmark_data_smithi022_188626_object111171:head# 2024-09-11T07:50:09.911 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb8d0c77:::benchmark_data_smithi022_188626_object118488:head# 2024-09-11T07:50:09.911 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb8e89dd:::benchmark_data_smithi022_188626_object119151:head# 2024-09-11T07:50:09.911 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb8f7f0f:::benchmark_data_smithi022_188626_object115913:head# 2024-09-11T07:50:09.912 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb90973e:::benchmark_data_smithi022_188626_object108698:head# 2024-09-11T07:50:09.912 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb917907:::benchmark_data_smithi022_188626_object94277:head# 2024-09-11T07:50:09.912 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb99f559:::benchmark_data_smithi022_188626_object94251:head# 2024-09-11T07:50:09.912 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb9b38af:::benchmark_data_smithi022_188626_object96423:head# 2024-09-11T07:50:09.912 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb9cb22f:::benchmark_data_smithi022_188626_object91299:head# 2024-09-11T07:50:09.912 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cb9ef932:::benchmark_data_smithi022_188626_object95660:head# 2024-09-11T07:50:09.912 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cba01f2b:::benchmark_data_smithi022_188626_object92067:head# 2024-09-11T07:50:09.912 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cba031f9:::benchmark_data_smithi022_188626_object108561:head# 2024-09-11T07:50:09.912 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cba22a60:::benchmark_data_smithi022_188626_object114235:head# 2024-09-11T07:50:09.912 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cba55d67:::benchmark_data_smithi022_188626_object110999:head# 2024-09-11T07:50:09.912 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbab3486:::benchmark_data_smithi022_188626_object91366:head# 2024-09-11T07:50:09.912 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbac1479:::benchmark_data_smithi022_188626_object93003:head# 2024-09-11T07:50:09.912 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbaf80f2:::benchmark_data_smithi022_188626_object101323:head# 2024-09-11T07:50:09.912 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbb3c31b:::benchmark_data_smithi022_188626_object89573:head# 2024-09-11T07:50:09.913 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbb9b175:::benchmark_data_smithi022_188626_object104665:head# 2024-09-11T07:50:09.913 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbbbce35:::benchmark_data_smithi022_188626_object95753:head# 2024-09-11T07:50:09.913 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbbce796:::benchmark_data_smithi022_188626_object105732:head# 2024-09-11T07:50:09.913 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbbd9991:::benchmark_data_smithi022_188626_object101792:head# 2024-09-11T07:50:09.913 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbbda5bc:::benchmark_data_smithi022_188626_object107074:head# 2024-09-11T07:50:09.913 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbbde72a:::benchmark_data_smithi022_188626_object112670:head# 2024-09-11T07:50:09.913 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbbe8182:::benchmark_data_smithi022_188626_object112245:head# 2024-09-11T07:50:09.913 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbbecbf2:::benchmark_data_smithi022_188626_object113931:head# 2024-09-11T07:50:09.913 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbbf070a:::benchmark_data_smithi022_188626_object98343:head# 2024-09-11T07:50:09.913 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbc33df0:::benchmark_data_smithi022_188626_object116241:head# 2024-09-11T07:50:09.913 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbc56350:::benchmark_data_smithi022_188626_object117106:head# 2024-09-11T07:50:09.913 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbc75fa8:::benchmark_data_smithi022_188626_object91266:head# 2024-09-11T07:50:09.913 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbcb240d:::benchmark_data_smithi022_188626_object91007:head# 2024-09-11T07:50:09.913 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbcf5bd9:::benchmark_data_smithi022_188626_object102276:head# 2024-09-11T07:50:09.914 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbd04f99:::benchmark_data_smithi022_188626_object99234:head# 2024-09-11T07:50:09.914 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbd088f0:::benchmark_data_smithi022_188626_object109761:head# 2024-09-11T07:50:09.914 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbd51236:::benchmark_data_smithi022_188626_object100777:head# 2024-09-11T07:50:09.914 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbd6a458:::benchmark_data_smithi022_188626_object108071:head# 2024-09-11T07:50:09.914 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbda0762:::benchmark_data_smithi022_188626_object106524:head# 2024-09-11T07:50:09.914 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbdc9ab3:::benchmark_data_smithi022_188626_object99429:head# 2024-09-11T07:50:09.914 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbde3298:::benchmark_data_smithi022_188626_object102263:head# 2024-09-11T07:50:09.914 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbe2cb67:::benchmark_data_smithi022_188626_object114727:head# 2024-09-11T07:50:09.914 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbe7ed6a:::benchmark_data_smithi022_188626_object107940:head# 2024-09-11T07:50:09.914 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbe9bbe3:::benchmark_data_smithi022_188626_object98811:head# 2024-09-11T07:50:09.914 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbea2975:::benchmark_data_smithi022_188626_object106900:head# 2024-09-11T07:50:09.914 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbee0819:::benchmark_data_smithi022_188626_object119272:head# 2024-09-11T07:50:09.914 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbee353a:::benchmark_data_smithi022_188626_object101798:head# 2024-09-11T07:50:09.914 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbef7660:::benchmark_data_smithi022_188626_object93948:head# 2024-09-11T07:50:09.915 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbf0cca0:::benchmark_data_smithi022_188626_object108098:head# 2024-09-11T07:50:09.915 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbf62f6b:::benchmark_data_smithi022_188626_object91342:head# 2024-09-11T07:50:09.915 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbf66771:::benchmark_data_smithi022_188626_object110671:head# 2024-09-11T07:50:09.915 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cbf66d8b:::benchmark_data_smithi022_188626_object91332:head# 2024-09-11T07:50:09.915 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc024109:::benchmark_data_smithi022_188626_object107034:head# 2024-09-11T07:50:09.915 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc032262:::benchmark_data_smithi022_188626_object118539:head# 2024-09-11T07:50:09.915 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc06412a:::benchmark_data_smithi022_188626_object99219:head# 2024-09-11T07:50:09.915 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc06915f:::benchmark_data_smithi022_188626_object89754:head# 2024-09-11T07:50:09.915 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc0ec549:::benchmark_data_smithi022_188626_object111535:head# 2024-09-11T07:50:09.915 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc1392aa:::benchmark_data_smithi022_188626_object111621:head# 2024-09-11T07:50:09.915 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc159cdf:::benchmark_data_smithi022_188626_object101402:head# 2024-09-11T07:50:09.915 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc17609e:::benchmark_data_smithi022_188626_object100483:head# 2024-09-11T07:50:09.915 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc1a8740:::benchmark_data_smithi022_188626_object117905:head# 2024-09-11T07:50:09.915 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc1d37e2:::benchmark_data_smithi022_188626_object96346:head# 2024-09-11T07:50:09.916 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc1e21a2:::benchmark_data_smithi022_188626_object97267:head# 2024-09-11T07:50:09.916 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc2000d8:::benchmark_data_smithi022_188626_object108755:head# 2024-09-11T07:50:09.916 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc28b392:::benchmark_data_smithi022_188626_object106496:head# 2024-09-11T07:50:09.916 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc298004:::benchmark_data_smithi022_188626_object91961:head# 2024-09-11T07:50:09.916 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc2d910e:::benchmark_data_smithi022_188626_object114846:head# 2024-09-11T07:50:09.916 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc2e62f5:::benchmark_data_smithi022_188626_object98016:head# 2024-09-11T07:50:09.916 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc312135:::benchmark_data_smithi022_188626_object95928:head# 2024-09-11T07:50:09.916 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc34e8a8:::benchmark_data_smithi022_188626_object102232:head# 2024-09-11T07:50:09.916 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc34f563:::benchmark_data_smithi022_188626_object109496:head# 2024-09-11T07:50:09.916 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc374d69:::benchmark_data_smithi022_188626_object106564:head# 2024-09-11T07:50:09.916 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc380b4b:::benchmark_data_smithi022_188626_object118246:head# 2024-09-11T07:50:09.916 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc3f37fd:::benchmark_data_smithi022_188626_object109884:head# 2024-09-11T07:50:09.917 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc4259cc:::benchmark_data_smithi022_188626_object98894:head# 2024-09-11T07:50:09.917 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc4384f2:::benchmark_data_smithi022_188626_object115186:head# 2024-09-11T07:50:09.917 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc43c5bb:::benchmark_data_smithi022_188626_object105808:head# 2024-09-11T07:50:09.917 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc43ec60:::benchmark_data_smithi022_188626_object108068:head# 2024-09-11T07:50:09.917 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc45a740:::benchmark_data_smithi022_188626_object118668:head# 2024-09-11T07:50:09.917 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc46c1b6:::benchmark_data_smithi022_188626_object100625:head# 2024-09-11T07:50:09.917 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc4b0b68:::benchmark_data_smithi022_188626_object119235:head# 2024-09-11T07:50:09.917 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc50a295:::benchmark_data_smithi022_188626_object103856:head# 2024-09-11T07:50:09.917 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc550ca6:::benchmark_data_smithi022_188626_object109318:head# 2024-09-11T07:50:09.917 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc56bc52:::benchmark_data_smithi022_188626_object98156:head# 2024-09-11T07:50:09.917 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc5a7cff:::benchmark_data_smithi022_188626_object116489:head# 2024-09-11T07:50:09.917 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc5b173f:::benchmark_data_smithi022_188626_object117939:head# 2024-09-11T07:50:09.989 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc5bb1b5:::benchmark_data_smithi022_188626_object114081:head# 2024-09-11T07:50:09.989 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc628209:::benchmark_data_smithi022_188626_object106378:head# 2024-09-11T07:50:09.989 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc64d1be:::benchmark_data_smithi022_188626_object96868:head# 2024-09-11T07:50:09.989 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc682812:::benchmark_data_smithi022_188626_object106218:head# 2024-09-11T07:50:09.989 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc69f0e2:::benchmark_data_smithi022_188626_object107349:head# 2024-09-11T07:50:09.989 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc6a2c32:::benchmark_data_smithi022_188626_object99276:head# 2024-09-11T07:50:09.989 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc6a95bc:::benchmark_data_smithi022_188626_object95847:head# 2024-09-11T07:50:09.990 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc6e7579:::benchmark_data_smithi022_188626_object107602:head# 2024-09-11T07:50:09.990 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc6f3ab3:::benchmark_data_smithi022_188626_object95249:head# 2024-09-11T07:50:09.990 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc71a456:::benchmark_data_smithi022_188626_object105729:head# 2024-09-11T07:50:09.990 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc73d37d:::benchmark_data_smithi022_188626_object119455:head# 2024-09-11T07:50:09.990 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc763f36:::benchmark_data_smithi022_188626_object107792:head# 2024-09-11T07:50:09.990 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc78c607:::benchmark_data_smithi022_188626_object93682:head# 2024-09-11T07:50:09.990 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc78c6e5:::benchmark_data_smithi022_188626_object104504:head# 2024-09-11T07:50:09.990 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc7ce4fc:::benchmark_data_smithi022_188626_object116006:head# 2024-09-11T07:50:09.990 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc811976:::benchmark_data_smithi022_188626_object118453:head# 2024-09-11T07:50:09.990 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc821c1f:::benchmark_data_smithi022_188626_object89932:head# 2024-09-11T07:50:09.990 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc82708e:::benchmark_data_smithi022_188626_object107550:head# 2024-09-11T07:50:09.990 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc83eade:::benchmark_data_smithi022_188626_object89884:head# 2024-09-11T07:50:09.990 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc863888:::benchmark_data_smithi022_188626_object108368:head# 2024-09-11T07:50:09.991 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc891a45:::benchmark_data_smithi022_188626_object93950:head# 2024-09-11T07:50:09.991 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc892385:::benchmark_data_smithi022_188626_object119670:head# 2024-09-11T07:50:09.991 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc89c70b:::benchmark_data_smithi022_188626_object90669:head# 2024-09-11T07:50:09.991 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc8c6147:::benchmark_data_smithi022_188626_object105620:head# 2024-09-11T07:50:09.991 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc8cc406:::benchmark_data_smithi022_188626_object109908:head# 2024-09-11T07:50:09.991 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc8e133a:::benchmark_data_smithi022_188626_object99307:head# 2024-09-11T07:50:09.991 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc90c49d:::benchmark_data_smithi022_188626_object103722:head# 2024-09-11T07:50:09.991 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc929c03:::benchmark_data_smithi022_188626_object117179:head# 2024-09-11T07:50:09.991 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc960dc5:::benchmark_data_smithi022_188626_object109441:head# 2024-09-11T07:50:09.991 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc98641d:::benchmark_data_smithi022_188626_object106438:head# 2024-09-11T07:50:09.991 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc9a62f5:::benchmark_data_smithi022_188626_object115473:head# 2024-09-11T07:50:09.991 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cc9b5639:::benchmark_data_smithi022_188626_object106866:head# 2024-09-11T07:50:09.992 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cca09110:::benchmark_data_smithi022_188626_object104286:head# 2024-09-11T07:50:09.992 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cca1bdda:::benchmark_data_smithi022_188626_object90615:head# 2024-09-11T07:50:09.992 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cca3fa2d:::benchmark_data_smithi022_188626_object97549:head# 2024-09-11T07:50:09.992 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cca50eb3:::benchmark_data_smithi022_188626_object109778:head# 2024-09-11T07:50:09.992 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cca789b4:::benchmark_data_smithi022_188626_object115948:head# 2024-09-11T07:50:09.992 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cca7be26:::benchmark_data_smithi022_188626_object112331:head# 2024-09-11T07:50:09.992 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cca8515f:::benchmark_data_smithi022_188626_object100730:head# 2024-09-11T07:50:09.992 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cca9aafc:::benchmark_data_smithi022_188626_object102916:head# 2024-09-11T07:50:09.992 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccab8d69:::benchmark_data_smithi022_188626_object116265:head# 2024-09-11T07:50:09.992 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccac3eae:::benchmark_data_smithi022_188626_object97422:head# 2024-09-11T07:50:09.992 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccadbaab:::benchmark_data_smithi022_188626_object93363:head# 2024-09-11T07:50:09.992 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccaeb1b7:::benchmark_data_smithi022_188626_object100118:head# 2024-09-11T07:50:09.992 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccaf2066:::benchmark_data_smithi022_188626_object111821:head# 2024-09-11T07:50:09.993 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccafd843:::benchmark_data_smithi022_188626_object99283:head# 2024-09-11T07:50:09.993 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccafe208:::benchmark_data_smithi022_188626_object119759:head# 2024-09-11T07:50:09.993 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccb012b4:::benchmark_data_smithi022_188626_object106180:head# 2024-09-11T07:50:09.993 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccb7a48b:::benchmark_data_smithi022_188626_object100008:head# 2024-09-11T07:50:09.993 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccb8aa0e:::benchmark_data_smithi022_188626_object105988:head# 2024-09-11T07:50:09.993 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccb9f7b4:::benchmark_data_smithi022_188626_object90099:head# 2024-09-11T07:50:09.993 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccbabe64:::benchmark_data_smithi022_188626_object116871:head# 2024-09-11T07:50:09.993 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccbfcd2a:::benchmark_data_smithi022_188626_object100159:head# 2024-09-11T07:50:09.993 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccc08417:::benchmark_data_smithi022_188626_object112579:head# 2024-09-11T07:50:09.993 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccc30ca7:::benchmark_data_smithi022_188626_object96432:head# 2024-09-11T07:50:09.993 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccc42563:::benchmark_data_smithi022_188626_object90875:head# 2024-09-11T07:50:09.993 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccc59cd8:::benchmark_data_smithi022_188626_object112035:head# 2024-09-11T07:50:09.993 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccc61dde:::benchmark_data_smithi022_188626_object91859:head# 2024-09-11T07:50:09.993 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccc87a10:::benchmark_data_smithi022_188626_object115481:head# 2024-09-11T07:50:09.994 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cccbbfc3:::benchmark_data_smithi022_188626_object102561:head# 2024-09-11T07:50:09.994 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cccd39a7:::benchmark_data_smithi022_188626_object116417:head# 2024-09-11T07:50:09.994 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccce2cde:::benchmark_data_smithi022_188626_object90999:head# 2024-09-11T07:50:09.994 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccce4f66:::benchmark_data_smithi022_188626_object101184:head# 2024-09-11T07:50:09.994 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccd41ddd:::benchmark_data_smithi022_188626_object106864:head# 2024-09-11T07:50:09.994 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccd6afe1:::benchmark_data_smithi022_188626_object116399:head# 2024-09-11T07:50:09.994 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccdb6608:::benchmark_data_smithi022_188626_object106647:head# 2024-09-11T07:50:09.994 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccdc5447:::benchmark_data_smithi022_188626_object94564:head# 2024-09-11T07:50:09.994 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cce4f895:::benchmark_data_smithi022_188626_object113976:head# 2024-09-11T07:50:09.994 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cce6d5af:::benchmark_data_smithi022_188626_object109168:head# 2024-09-11T07:50:09.994 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cce94722:::benchmark_data_smithi022_188626_object98870:head# 2024-09-11T07:50:09.994 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cceb1174:::benchmark_data_smithi022_188626_object116552:head# 2024-09-11T07:50:09.994 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccf07b65:::benchmark_data_smithi022_188626_object93538:head# 2024-09-11T07:50:09.994 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccf3950b:::benchmark_data_smithi022_188626_object93087:head# 2024-09-11T07:50:09.995 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccf599db:::benchmark_data_smithi022_188626_object95659:head# 2024-09-11T07:50:09.995 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccfb4b70:::benchmark_data_smithi022_188626_object93103:head# 2024-09-11T07:50:09.995 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ccfec9f3:::benchmark_data_smithi022_188626_object89612:head# 2024-09-11T07:50:09.995 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd0111fc:::benchmark_data_smithi022_188626_object94457:head# 2024-09-11T07:50:09.995 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd0304e4:::benchmark_data_smithi022_188626_object102060:head# 2024-09-11T07:50:09.995 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd04c5d4:::benchmark_data_smithi022_188626_object109302:head# 2024-09-11T07:50:09.995 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd06daba:::benchmark_data_smithi022_188626_object90972:head# 2024-09-11T07:50:09.995 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd0860ba:::benchmark_data_smithi022_188626_object102512:head# 2024-09-11T07:50:09.995 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd09dc08:::benchmark_data_smithi022_188626_object118287:head# 2024-09-11T07:50:09.995 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd100ed7:::benchmark_data_smithi022_188626_object107367:head# 2024-09-11T07:50:09.995 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd125288:::benchmark_data_smithi022_188626_object113487:head# 2024-09-11T07:50:09.995 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd12b24b:::benchmark_data_smithi022_188626_object91866:head# 2024-09-11T07:50:09.995 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd1352a7:::benchmark_data_smithi022_188626_object90186:head# 2024-09-11T07:50:09.995 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd1440d9:::benchmark_data_smithi022_188626_object93695:head# 2024-09-11T07:50:09.995 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd1538ec:::benchmark_data_smithi022_188626_object91685:head# 2024-09-11T07:50:09.996 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd174f33:::benchmark_data_smithi022_188626_object115463:head# 2024-09-11T07:50:09.996 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd1a9c91:::benchmark_data_smithi022_188626_object94640:head# 2024-09-11T07:50:09.996 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd1d4e51:::benchmark_data_smithi022_188626_object91063:head# 2024-09-11T07:50:09.996 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd1df515:::benchmark_data_smithi022_188626_object90806:head# 2024-09-11T07:50:09.996 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd229341:::benchmark_data_smithi022_188626_object116138:head# 2024-09-11T07:50:09.996 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd22f30b:::benchmark_data_smithi022_188626_object94415:head# 2024-09-11T07:50:09.996 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd230e31:::benchmark_data_smithi022_188626_object93348:head# 2024-09-11T07:50:09.996 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd2703e4:::benchmark_data_smithi022_188626_object111164:head# 2024-09-11T07:50:09.996 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd27513e:::benchmark_data_smithi022_188626_object93191:head# 2024-09-11T07:50:09.996 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd2a3e92:::benchmark_data_smithi022_188626_object112114:head# 2024-09-11T07:50:09.996 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd2b3ea2:::benchmark_data_smithi022_188626_object95955:head# 2024-09-11T07:50:09.996 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd2bcf2f:::benchmark_data_smithi022_188626_object114362:head# 2024-09-11T07:50:09.996 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd2f403a:::benchmark_data_smithi022_188626_object112784:head# 2024-09-11T07:50:09.996 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd2fb493:::benchmark_data_smithi022_188626_object114327:head# 2024-09-11T07:50:09.997 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd30834c:::benchmark_data_smithi022_188626_object96264:head# 2024-09-11T07:50:09.997 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd324037:::benchmark_data_smithi022_188626_object105071:head# 2024-09-11T07:50:09.997 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd34cff5:::benchmark_data_smithi022_188626_object96407:head# 2024-09-11T07:50:09.997 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd379b1f:::benchmark_data_smithi022_188626_object111096:head# 2024-09-11T07:50:09.997 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd3889f3:::benchmark_data_smithi022_188626_object108502:head# 2024-09-11T07:50:09.997 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd3bbed7:::benchmark_data_smithi022_188626_object113927:head# 2024-09-11T07:50:09.997 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd3c3a4f:::benchmark_data_smithi022_188626_object117688:head# 2024-09-11T07:50:09.997 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd3feef5:::benchmark_data_smithi022_188626_object119136:head# 2024-09-11T07:50:09.997 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd4077e9:::benchmark_data_smithi022_188626_object116996:head# 2024-09-11T07:50:09.997 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd4108a0:::benchmark_data_smithi022_188626_object97213:head# 2024-09-11T07:50:09.997 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd449777:::benchmark_data_smithi022_188626_object118537:head# 2024-09-11T07:50:09.997 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd45dd82:::benchmark_data_smithi022_188626_object105762:head# 2024-09-11T07:50:09.997 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd4668d1:::benchmark_data_smithi022_188626_object115954:head# 2024-09-11T07:50:09.997 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd467937:::benchmark_data_smithi022_188626_object112288:head# 2024-09-11T07:50:09.998 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd46d9a7:::benchmark_data_smithi022_188626_object96119:head# 2024-09-11T07:50:09.998 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd476994:::benchmark_data_smithi022_188626_object117458:head# 2024-09-11T07:50:09.998 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd4845a4:::benchmark_data_smithi022_188626_object116783:head# 2024-09-11T07:50:09.998 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd497372:::benchmark_data_smithi022_188626_object103116:head# 2024-09-11T07:50:09.998 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd4ae9ca:::benchmark_data_smithi022_188626_object110057:head# 2024-09-11T07:50:09.998 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd4aed42:::benchmark_data_smithi022_188626_object103382:head# 2024-09-11T07:50:09.998 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd4ba7b7:::benchmark_data_smithi022_188626_object90536:head# 2024-09-11T07:50:09.998 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd4d620b:::benchmark_data_smithi022_188626_object90096:head# 2024-09-11T07:50:09.998 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd4d8653:::benchmark_data_smithi022_188626_object99949:head# 2024-09-11T07:50:09.998 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd4f7c5c:::benchmark_data_smithi022_188626_object106073:head# 2024-09-11T07:50:09.998 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd50bbd1:::benchmark_data_smithi022_188626_object109325:head# 2024-09-11T07:50:09.998 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd514a53:::benchmark_data_smithi022_188626_object118954:head# 2024-09-11T07:50:09.998 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd51700d:::benchmark_data_smithi022_188626_object107591:head# 2024-09-11T07:50:09.998 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd5175b1:::benchmark_data_smithi022_188626_object101586:head# 2024-09-11T07:50:09.999 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd5276e1:::benchmark_data_smithi022_188626_object108772:head# 2024-09-11T07:50:09.999 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd53d024:::benchmark_data_smithi022_188626_object109490:head# 2024-09-11T07:50:09.999 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd5400a6:::benchmark_data_smithi022_188626_object102091:head# 2024-09-11T07:50:09.999 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd59263d:::benchmark_data_smithi022_188626_object96289:head# 2024-09-11T07:50:09.999 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd597a9e:::benchmark_data_smithi022_188626_object96610:head# 2024-09-11T07:50:09.999 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd5b7047:::benchmark_data_smithi022_188626_object110964:head# 2024-09-11T07:50:09.999 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd5e0cbf:::benchmark_data_smithi022_188626_object94748:head# 2024-09-11T07:50:09.999 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd65629b:::benchmark_data_smithi022_188626_object93550:head# 2024-09-11T07:50:09.999 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd65f4ec:::benchmark_data_smithi022_188626_object90858:head# 2024-09-11T07:50:09.999 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd6651c0:::benchmark_data_smithi022_188626_object111616:head# 2024-09-11T07:50:09.999 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd680355:::benchmark_data_smithi022_188626_object102494:head# 2024-09-11T07:50:09.999 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd68697a:::benchmark_data_smithi022_188626_object106281:head# 2024-09-11T07:50:09.999 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd69121c:::benchmark_data_smithi022_188626_object106002:head# 2024-09-11T07:50:09.999 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd6b1408:::benchmark_data_smithi022_188626_object101370:head# 2024-09-11T07:50:09.999 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd6b7d69:::benchmark_data_smithi022_188626_object104484:head# 2024-09-11T07:50:10.000 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd6bec73:::benchmark_data_smithi022_188626_object90906:head# 2024-09-11T07:50:10.000 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd6d681e:::benchmark_data_smithi022_188626_object100292:head# 2024-09-11T07:50:10.000 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd6dc679:::benchmark_data_smithi022_188626_object88985:head# 2024-09-11T07:50:10.000 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd6fd920:::benchmark_data_smithi022_188626_object93321:head# 2024-09-11T07:50:10.000 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd77eab2:::benchmark_data_smithi022_188626_object97768:head# 2024-09-11T07:50:10.000 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd7ce8c8:::benchmark_data_smithi022_188626_object96354:head# 2024-09-11T07:50:10.000 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd7db560:::benchmark_data_smithi022_188626_object93688:head# 2024-09-11T07:50:10.000 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd832573:::benchmark_data_smithi022_188626_object106816:head# 2024-09-11T07:50:10.000 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd8592db:::benchmark_data_smithi022_188626_object98455:head# 2024-09-11T07:50:10.000 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd8c2ebe:::benchmark_data_smithi022_188626_object105545:head# 2024-09-11T07:50:10.000 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd8f2521:::benchmark_data_smithi022_188626_object100493:head# 2024-09-11T07:50:10.000 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd911670:::benchmark_data_smithi022_188626_object99918:head# 2024-09-11T07:50:10.000 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd9277f0:::benchmark_data_smithi022_188626_object104570:head# 2024-09-11T07:50:10.001 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd9353f0:::benchmark_data_smithi022_188626_object117641:head# 2024-09-11T07:50:10.001 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd93f2f2:::benchmark_data_smithi022_188626_object91010:head# 2024-09-11T07:50:10.001 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd9959b7:::benchmark_data_smithi022_188626_object99147:head# 2024-09-11T07:50:10.001 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd9a472e:::benchmark_data_smithi022_188626_object111448:head# 2024-09-11T07:50:10.001 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd9df8bb:::benchmark_data_smithi022_188626_object113561:head# 2024-09-11T07:50:10.001 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd9e77bf:::benchmark_data_smithi022_188626_object112908:head# 2024-09-11T07:50:10.001 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd9ea3ce:::benchmark_data_smithi022_188626_object97389:head# 2024-09-11T07:50:10.001 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cd9f41f9:::benchmark_data_smithi022_188626_object93986:head# 2024-09-11T07:50:10.001 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cda08fe2:::benchmark_data_smithi022_188626_object100588:head# 2024-09-11T07:50:10.001 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cda4adc0:::benchmark_data_smithi022_188626_object104108:head# 2024-09-11T07:50:10.001 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cdac8206:::benchmark_data_smithi022_188626_object111373:head# 2024-09-11T07:50:10.001 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cdb500ff:::benchmark_data_smithi022_188626_object100035:head# 2024-09-11T07:50:10.001 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cdbbffe8:::benchmark_data_smithi022_188626_object100144:head# 2024-09-11T07:50:10.001 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cdbdc80f:::benchmark_data_smithi022_188626_object92902:head# 2024-09-11T07:50:10.002 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cdc7dfd6:::benchmark_data_smithi022_188626_object94803:head# 2024-09-11T07:50:10.002 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cdc81fc7:::benchmark_data_smithi022_188626_object99210:head# 2024-09-11T07:50:10.002 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cdc8bd13:::benchmark_data_smithi022_188626_object102991:head# 2024-09-11T07:50:10.002 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cdc93869:::benchmark_data_smithi022_188626_object95850:head# 2024-09-11T07:50:10.002 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cdccb943:::benchmark_data_smithi022_188626_object89200:head# 2024-09-11T07:50:10.002 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cdcd673a:::benchmark_data_smithi022_188626_object104267:head# 2024-09-11T07:50:10.002 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cdcf1ade:::benchmark_data_smithi022_188626_object112490:head# 2024-09-11T07:50:10.002 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cdcf20dc:::benchmark_data_smithi022_188626_object106391:head# 2024-09-11T07:50:10.002 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cdd0d553:::benchmark_data_smithi022_188626_object113934:head# 2024-09-11T07:50:10.002 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cdd17320:::benchmark_data_smithi022_188626_object112232:head# 2024-09-11T07:50:10.002 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cdd99f0a:::benchmark_data_smithi022_188626_object105262:head# 2024-09-11T07:50:10.002 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cddaa316:::benchmark_data_smithi022_188626_object113008:head# 2024-09-11T07:50:10.002 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cddb4486:::benchmark_data_smithi022_188626_object94860:head# 2024-09-11T07:50:10.002 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cde13f17:::benchmark_data_smithi022_188626_object108270:head# 2024-09-11T07:50:10.002 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cde950e0:::benchmark_data_smithi022_188626_object95301:head# 2024-09-11T07:50:10.003 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cdec741c:::benchmark_data_smithi022_188626_object119212:head# 2024-09-11T07:50:10.003 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cdf02929:::benchmark_data_smithi022_188626_object108139:head# 2024-09-11T07:50:10.003 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cdf796ef:::benchmark_data_smithi022_188626_object100639:head# 2024-09-11T07:50:10.003 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cdf90929:::benchmark_data_smithi022_188626_object111753:head# 2024-09-11T07:50:10.003 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cdfa829f:::benchmark_data_smithi022_188626_object89749:head# 2024-09-11T07:50:10.003 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cdfccf03:::benchmark_data_smithi022_188626_object99334:head# 2024-09-11T07:50:10.003 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cdfe25b0:::benchmark_data_smithi022_188626_object115733:head# 2024-09-11T07:50:10.003 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce010e76:::benchmark_data_smithi022_188626_object105776:head# 2024-09-11T07:50:10.003 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce055c94:::benchmark_data_smithi022_188626_object92079:head# 2024-09-11T07:50:10.003 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce0663e2:::benchmark_data_smithi022_188626_object94683:head# 2024-09-11T07:50:10.003 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce0994da:::benchmark_data_smithi022_188626_object89225:head# 2024-09-11T07:50:10.003 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce0bb028:::benchmark_data_smithi022_188626_object113605:head# 2024-09-11T07:50:10.003 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce0ccb9b:::benchmark_data_smithi022_188626_object104699:head# 2024-09-11T07:50:10.004 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce0cdd0a:::benchmark_data_smithi022_188626_object118418:head# 2024-09-11T07:50:10.004 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce0d71ea:::benchmark_data_smithi022_188626_object110141:head# 2024-09-11T07:50:10.004 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce0ddde7:::benchmark_data_smithi022_188626_object90653:head# 2024-09-11T07:50:10.004 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce10247c:::benchmark_data_smithi022_188626_object105246:head# 2024-09-11T07:50:10.004 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce102d27:::benchmark_data_smithi022_188626_object113947:head# 2024-09-11T07:50:10.004 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce137647:::benchmark_data_smithi022_188626_object95538:head# 2024-09-11T07:50:10.004 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce14acf5:::benchmark_data_smithi022_188626_object117743:head# 2024-09-11T07:50:10.004 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce1cc205:::benchmark_data_smithi022_188626_object113563:head# 2024-09-11T07:50:10.004 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce2095e2:::benchmark_data_smithi022_188626_object112497:head# 2024-09-11T07:50:10.004 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce23ae50:::benchmark_data_smithi022_188626_object104446:head# 2024-09-11T07:50:10.004 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce26bc46:::benchmark_data_smithi022_188626_object104520:head# 2024-09-11T07:50:10.004 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce2b7b07:::benchmark_data_smithi022_188626_object105537:head# 2024-09-11T07:50:10.004 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce2ce8e2:::benchmark_data_smithi022_188626_object117648:head# 2024-09-11T07:50:10.004 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce2d9043:::benchmark_data_smithi022_188626_object107965:head# 2024-09-11T07:50:10.005 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce322e53:::benchmark_data_smithi022_188626_object112628:head# 2024-09-11T07:50:10.005 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce36a3b9:::benchmark_data_smithi022_188626_object96893:head# 2024-09-11T07:50:10.005 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce387bbd:::benchmark_data_smithi022_188626_object95353:head# 2024-09-11T07:50:10.005 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce39d800:::benchmark_data_smithi022_188626_object89883:head# 2024-09-11T07:50:10.005 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce3cc271:::benchmark_data_smithi022_188626_object94570:head# 2024-09-11T07:50:10.005 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce3e3dad:::benchmark_data_smithi022_188626_object96285:head# 2024-09-11T07:50:10.005 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce412191:::benchmark_data_smithi022_188626_object116377:head# 2024-09-11T07:50:10.005 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce43d750:::benchmark_data_smithi022_188626_object95756:head# 2024-09-11T07:50:10.005 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce46884a:::benchmark_data_smithi022_188626_object113983:head# 2024-09-11T07:50:10.005 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce46f2c7:::benchmark_data_smithi022_188626_object111227:head# 2024-09-11T07:50:10.005 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce49dd6c:::benchmark_data_smithi022_188626_object118153:head# 2024-09-11T07:50:10.005 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce4b6e01:::benchmark_data_smithi022_188626_object113279:head# 2024-09-11T07:50:10.005 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce4df6e8:::benchmark_data_smithi022_188626_object97191:head# 2024-09-11T07:50:10.005 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce4e1cff:::benchmark_data_smithi022_188626_object100499:head# 2024-09-11T07:50:10.006 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce51c2a4:::benchmark_data_smithi022_188626_object98255:head# 2024-09-11T07:50:10.006 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce52dc9d:::benchmark_data_smithi022_188626_object97524:head# 2024-09-11T07:50:10.006 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce52f17a:::benchmark_data_smithi022_188626_object92890:head# 2024-09-11T07:50:10.006 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce53b075:::benchmark_data_smithi022_188626_object118428:head# 2024-09-11T07:50:10.006 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce57f267:::benchmark_data_smithi022_188626_object106873:head# 2024-09-11T07:50:10.006 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce58338a:::benchmark_data_smithi022_188626_object89812:head# 2024-09-11T07:50:10.006 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce58b3b5:::benchmark_data_smithi022_188626_object90873:head# 2024-09-11T07:50:10.006 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce5986ef:::benchmark_data_smithi022_188626_object93842:head# 2024-09-11T07:50:10.006 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce59e7ca:::benchmark_data_smithi022_188626_object94881:head# 2024-09-11T07:50:10.006 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce59f864:::benchmark_data_smithi022_188626_object89147:head# 2024-09-11T07:50:10.006 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce5e7fd6:::benchmark_data_smithi022_188626_object104751:head# 2024-09-11T07:50:10.006 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce5e9cd5:::benchmark_data_smithi022_188626_object108336:head# 2024-09-11T07:50:10.006 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce618c9b:::benchmark_data_smithi022_188626_object112435:head# 2024-09-11T07:50:10.006 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce645d45:::benchmark_data_smithi022_188626_object106507:head# 2024-09-11T07:50:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce65c6d0:::benchmark_data_smithi022_188626_object104896:head# 2024-09-11T07:50:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce70a368:::benchmark_data_smithi022_188626_object111827:head# 2024-09-11T07:50:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce71b1d6:::benchmark_data_smithi022_188626_object110563:head# 2024-09-11T07:50:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce73750e:::benchmark_data_smithi022_188626_object95337:head# 2024-09-11T07:50:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce746f17:::benchmark_data_smithi022_188626_object91606:head# 2024-09-11T07:50:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce756e6c:::benchmark_data_smithi022_188626_object105663:head# 2024-09-11T07:50:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce75e90a:::benchmark_data_smithi022_188626_object96383:head# 2024-09-11T07:50:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce770869:::benchmark_data_smithi022_188626_object92238:head# 2024-09-11T07:50:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce7d7b00:::benchmark_data_smithi022_188626_object98067:head# 2024-09-11T07:50:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce7ee984:::benchmark_data_smithi022_188626_object98310:head# 2024-09-11T07:50:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce80fefd:::benchmark_data_smithi022_188626_object108713:head# 2024-09-11T07:50:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce815992:::benchmark_data_smithi022_188626_object118756:head# 2024-09-11T07:50:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce82c05b:::benchmark_data_smithi022_188626_object103805:head# 2024-09-11T07:50:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce8374f0:::benchmark_data_smithi022_188626_object105160:head# 2024-09-11T07:50:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce872482:::benchmark_data_smithi022_188626_object116189:head# 2024-09-11T07:50:10.008 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce8d74da:::benchmark_data_smithi022_188626_object96352:head# 2024-09-11T07:50:10.008 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce8e2d14:::benchmark_data_smithi022_188626_object118814:head# 2024-09-11T07:50:10.008 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce8ec3b4:::benchmark_data_smithi022_188626_object119798:head# 2024-09-11T07:50:10.008 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce9142bf:::benchmark_data_smithi022_188626_object117446:head# 2024-09-11T07:50:10.008 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce95629b:::benchmark_data_smithi022_188626_object114516:head# 2024-09-11T07:50:10.008 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce968306:::benchmark_data_smithi022_188626_object105173:head# 2024-09-11T07:50:10.008 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce975f03:::benchmark_data_smithi022_188626_object114245:head# 2024-09-11T07:50:10.008 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce99a6b4:::benchmark_data_smithi022_188626_object117379:head# 2024-09-11T07:50:10.008 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce9bc84d:::benchmark_data_smithi022_188626_object98288:head# 2024-09-11T07:50:10.008 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ce9dcb3d:::benchmark_data_smithi022_188626_object89607:head# 2024-09-11T07:50:10.008 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cea34f2f:::benchmark_data_smithi022_188626_object112822:head# 2024-09-11T07:50:10.008 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cea67676:::benchmark_data_smithi022_188626_object110509:head# 2024-09-11T07:50:10.008 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cea6f550:::benchmark_data_smithi022_188626_object114838:head# 2024-09-11T07:50:10.008 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cea72d1f:::benchmark_data_smithi022_188626_object101495:head# 2024-09-11T07:50:10.008 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cea7dabd:::benchmark_data_smithi022_188626_object100260:head# 2024-09-11T07:50:10.009 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ceaaaeb1:::benchmark_data_smithi022_188626_object96164:head# 2024-09-11T07:50:10.009 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ceac2790:::benchmark_data_smithi022_188626_object94767:head# 2024-09-11T07:50:10.009 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ceaff13a:::benchmark_data_smithi022_188626_object104762:head# 2024-09-11T07:50:10.009 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ceb08681:::benchmark_data_smithi022_188626_object101512:head# 2024-09-11T07:50:10.009 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ceb10dc4:::benchmark_data_smithi022_188626_object119263:head# 2024-09-11T07:50:10.009 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ceb2e4a2:::benchmark_data_smithi022_188626_object93626:head# 2024-09-11T07:50:10.009 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ceb743ad:::benchmark_data_smithi022_188626_object98737:head# 2024-09-11T07:50:10.009 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ceb92659:::benchmark_data_smithi022_188626_object112238:head# 2024-09-11T07:50:10.009 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ceba2ba7:::benchmark_data_smithi022_188626_object111159:head# 2024-09-11T07:50:10.009 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cebdc92c:::benchmark_data_smithi022_188626_object102412:head# 2024-09-11T07:50:10.009 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cebe40a3:::benchmark_data_smithi022_188626_object102221:head# 2024-09-11T07:50:10.009 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cec40296:::benchmark_data_smithi022_188626_object107340:head# 2024-09-11T07:50:10.009 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cec47a54:::benchmark_data_smithi022_188626_object107010:head# 2024-09-11T07:50:10.009 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cec7e9f8:::benchmark_data_smithi022_188626_object106505:head# 2024-09-11T07:50:10.010 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cec8851d:::benchmark_data_smithi022_188626_object100192:head# 2024-09-11T07:50:10.010 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cec8f119:::benchmark_data_smithi022_188626_object99253:head# 2024-09-11T07:50:10.010 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cec931c2:::benchmark_data_smithi022_188626_object104620:head# 2024-09-11T07:50:10.010 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cec961c1:::benchmark_data_smithi022_188626_object112611:head# 2024-09-11T07:50:10.010 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cececa25:::benchmark_data_smithi022_188626_object98604:head# 2024-09-11T07:50:10.010 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cecff4c7:::benchmark_data_smithi022_188626_object104911:head# 2024-09-11T07:50:10.010 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ced15506:::benchmark_data_smithi022_188626_object117313:head# 2024-09-11T07:50:10.010 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ced3b054:::benchmark_data_smithi022_188626_object102433:head# 2024-09-11T07:50:10.010 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ced4144e:::benchmark_data_smithi022_188626_object104026:head# 2024-09-11T07:50:10.010 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ced4f2ae:::benchmark_data_smithi022_188626_object109207:head# 2024-09-11T07:50:10.010 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ced6be38:::benchmark_data_smithi022_188626_object93452:head# 2024-09-11T07:50:10.010 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ced6e7ce:::benchmark_data_smithi022_188626_object98203:head# 2024-09-11T07:50:10.010 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:ced96dd0:::benchmark_data_smithi022_188626_object110374:head# 2024-09-11T07:50:10.010 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cedb0a1d:::benchmark_data_smithi022_188626_object106360:head# 2024-09-11T07:50:10.010 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cedb7f25:::benchmark_data_smithi022_188626_object99962:head# 2024-09-11T07:50:10.011 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cee1f939:::benchmark_data_smithi022_188626_object90691:head# 2024-09-11T07:50:10.011 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cee24603:::benchmark_data_smithi022_188626_object116995:head# 2024-09-11T07:50:10.011 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cee86ef5:::benchmark_data_smithi022_188626_object101908:head# 2024-09-11T07:50:10.011 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cee9f797:::benchmark_data_smithi022_188626_object100388:head# 2024-09-11T07:50:10.011 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cef5712c:::benchmark_data_smithi022_188626_object93461:head# 2024-09-11T07:50:10.011 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cef5bf0a:::benchmark_data_smithi022_188626_object118599:head# 2024-09-11T07:50:10.011 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cef8d1e9:::benchmark_data_smithi022_188626_object109613:head# 2024-09-11T07:50:10.011 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cefacc1b:::benchmark_data_smithi022_188626_object119125:head# 2024-09-11T07:50:10.011 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cefb1f8e:::benchmark_data_smithi022_188626_object89022:head# 2024-09-11T07:50:10.011 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cefe3114:::benchmark_data_smithi022_188626_object115470:head# 2024-09-11T07:50:10.011 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf05ed34:::benchmark_data_smithi022_188626_object97662:head# 2024-09-11T07:50:10.011 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf0b460f:::benchmark_data_smithi022_188626_object89242:head# 2024-09-11T07:50:10.011 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf0b6a00:::benchmark_data_smithi022_188626_object119316:head# 2024-09-11T07:50:10.011 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf0eb43e:::benchmark_data_smithi022_188626_object101109:head# 2024-09-11T07:50:10.012 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf0f09c3:::benchmark_data_smithi022_188626_object105357:head# 2024-09-11T07:50:10.012 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf124800:::benchmark_data_smithi022_188626_object107967:head# 2024-09-11T07:50:10.012 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf144a70:::benchmark_data_smithi022_188626_object111244:head# 2024-09-11T07:50:10.012 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf155394:::benchmark_data_smithi022_188626_object119274:head# 2024-09-11T07:50:10.012 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf190608:::benchmark_data_smithi022_188626_object112613:head# 2024-09-11T07:50:10.012 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf19b8d3:::benchmark_data_smithi022_188626_object93371:head# 2024-09-11T07:50:10.012 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf22c79d:::benchmark_data_smithi022_188626_object91298:head# 2024-09-11T07:50:10.012 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf235a4f:::benchmark_data_smithi022_188626_object89165:head# 2024-09-11T07:50:10.012 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf23bcdc:::benchmark_data_smithi022_188626_object112585:head# 2024-09-11T07:50:10.012 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf266676:::benchmark_data_smithi022_188626_object105215:head# 2024-09-11T07:50:10.012 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf2725a2:::benchmark_data_smithi022_188626_object117976:head# 2024-09-11T07:50:10.012 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf277639:::benchmark_data_smithi022_188626_object112759:head# 2024-09-11T07:50:10.012 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf2c3f06:::benchmark_data_smithi022_188626_object92601:head# 2024-09-11T07:50:10.013 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf2d48d4:::benchmark_data_smithi022_188626_object107333:head# 2024-09-11T07:50:10.013 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf2ea8c9:::benchmark_data_smithi022_188626_object105846:head# 2024-09-11T07:50:10.013 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf336efb:::benchmark_data_smithi022_188626_object117409:head# 2024-09-11T07:50:10.013 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf346651:::benchmark_data_smithi022_188626_object103761:head# 2024-09-11T07:50:10.013 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf36c389:::benchmark_data_smithi022_188626_object110845:head# 2024-09-11T07:50:10.013 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf381fbb:::benchmark_data_smithi022_188626_object118671:head# 2024-09-11T07:50:10.013 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf3ac558:::benchmark_data_smithi022_188626_object119790:head# 2024-09-11T07:50:10.013 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf3ada04:::benchmark_data_smithi022_188626_object103354:head# 2024-09-11T07:50:10.013 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf3c4b80:::benchmark_data_smithi022_188626_object108364:head# 2024-09-11T07:50:10.013 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf3e906c:::benchmark_data_smithi022_188626_object114343:head# 2024-09-11T07:50:10.013 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf40641a:::benchmark_data_smithi022_188626_object106055:head# 2024-09-11T07:50:10.013 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf408f72:::benchmark_data_smithi022_188626_object111504:head# 2024-09-11T07:50:10.013 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf46b90a:::benchmark_data_smithi022_188626_object93015:head# 2024-09-11T07:50:10.013 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf478341:::benchmark_data_smithi022_188626_object92104:head# 2024-09-11T07:50:10.014 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf48644b:::benchmark_data_smithi022_188626_object93434:head# 2024-09-11T07:50:10.014 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf4a4601:::benchmark_data_smithi022_188626_object112452:head# 2024-09-11T07:50:10.014 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf4a525b:::benchmark_data_smithi022_188626_object100189:head# 2024-09-11T07:50:10.014 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf4e8f16:::benchmark_data_smithi022_188626_object92829:head# 2024-09-11T07:50:10.014 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf503781:::benchmark_data_smithi022_188626_object97124:head# 2024-09-11T07:50:10.014 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf50fa46:::benchmark_data_smithi022_188626_object93153:head# 2024-09-11T07:50:10.014 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf535599:::benchmark_data_smithi022_188626_object101304:head# 2024-09-11T07:50:10.014 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf547a32:::benchmark_data_smithi022_188626_object111215:head# 2024-09-11T07:50:10.014 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf597932:::benchmark_data_smithi022_188626_object99885:head# 2024-09-11T07:50:10.014 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf59a206:::benchmark_data_smithi022_188626_object90446:head# 2024-09-11T07:50:10.014 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf5b0e94:::benchmark_data_smithi022_188626_object108352:head# 2024-09-11T07:50:10.014 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf641b08:::benchmark_data_smithi022_188626_object95200:head# 2024-09-11T07:50:10.014 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf667325:::benchmark_data_smithi022_188626_object99820:head# 2024-09-11T07:50:10.014 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf689b3b:::benchmark_data_smithi022_188626_object118159:head# 2024-09-11T07:50:10.014 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf68bbf8:::benchmark_data_smithi022_188626_object107941:head# 2024-09-11T07:50:10.015 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf6a96a2:::benchmark_data_smithi022_188626_object101997:head# 2024-09-11T07:50:10.015 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf6b596d:::benchmark_data_smithi022_188626_object103829:head# 2024-09-11T07:50:10.015 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf7362d1:::benchmark_data_smithi022_188626_object118279:head# 2024-09-11T07:50:10.015 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf73fc60:::benchmark_data_smithi022_188626_object112146:head# 2024-09-11T07:50:10.015 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf766035:::benchmark_data_smithi022_188626_object98800:head# 2024-09-11T07:50:10.015 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf771104:::benchmark_data_smithi022_188626_object106258:head# 2024-09-11T07:50:10.015 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf7844c3:::benchmark_data_smithi022_188626_object110931:head# 2024-09-11T07:50:10.015 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf7deac9:::benchmark_data_smithi022_188626_object116314:head# 2024-09-11T07:50:10.015 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf85791a:::benchmark_data_smithi022_188626_object98559:head# 2024-09-11T07:50:10.015 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf872e19:::benchmark_data_smithi022_188626_object91454:head# 2024-09-11T07:50:10.015 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf88fdcb:::benchmark_data_smithi022_188626_object109148:head# 2024-09-11T07:50:10.015 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf89d8dd:::benchmark_data_smithi022_188626_object96450:head# 2024-09-11T07:50:10.015 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf8a3f2b:::benchmark_data_smithi022_188626_object107184:head# 2024-09-11T07:50:10.016 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf8d39ce:::benchmark_data_smithi022_188626_object100626:head# 2024-09-11T07:50:10.016 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf90a39c:::benchmark_data_smithi022_188626_object110702:head# 2024-09-11T07:50:10.016 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf940aca:::benchmark_data_smithi022_188626_object112467:head# 2024-09-11T07:50:10.016 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf95bccd:::benchmark_data_smithi022_188626_object119702:head# 2024-09-11T07:50:10.016 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf991b0d:::benchmark_data_smithi022_188626_object106545:head# 2024-09-11T07:50:10.016 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf9c970b:::benchmark_data_smithi022_188626_object119361:head# 2024-09-11T07:50:10.016 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cf9ce539:::benchmark_data_smithi022_188626_object102112:head# 2024-09-11T07:50:10.016 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfa0fe6a:::benchmark_data_smithi022_188626_object103859:head# 2024-09-11T07:50:10.016 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfa13c59:::benchmark_data_smithi022_188626_object95925:head# 2024-09-11T07:50:10.016 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfa1944a:::benchmark_data_smithi022_188626_object112056:head# 2024-09-11T07:50:10.016 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfa233a6:::benchmark_data_smithi022_188626_object103223:head# 2024-09-11T07:50:10.016 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfa4460c:::benchmark_data_smithi022_188626_object91036:head# 2024-09-11T07:50:10.016 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfa5c9f6:::benchmark_data_smithi022_188626_object104330:head# 2024-09-11T07:50:10.016 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfaa132f:::benchmark_data_smithi022_188626_object104518:head# 2024-09-11T07:50:10.017 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfaa9cb8:::benchmark_data_smithi022_188626_object105794:head# 2024-09-11T07:50:10.017 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfac59e0:::benchmark_data_smithi022_188626_object90450:head# 2024-09-11T07:50:10.017 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfacf805:::benchmark_data_smithi022_188626_object112257:head# 2024-09-11T07:50:10.017 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfae5006:::benchmark_data_smithi022_188626_object115525:head# 2024-09-11T07:50:10.017 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfaee425:::benchmark_data_smithi022_188626_object109735:head# 2024-09-11T07:50:10.017 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfaf568f:::benchmark_data_smithi022_188626_object119797:head# 2024-09-11T07:50:10.017 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfb03d72:::benchmark_data_smithi022_188626_object95216:head# 2024-09-11T07:50:10.017 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfb04547:::benchmark_data_smithi022_188626_object96571:head# 2024-09-11T07:50:10.017 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfb06210:::benchmark_data_smithi022_188626_object117448:head# 2024-09-11T07:50:10.017 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfb31c60:::benchmark_data_smithi022_188626_object94957:head# 2024-09-11T07:50:10.017 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfb44e84:::benchmark_data_smithi022_188626_object110610:head# 2024-09-11T07:50:10.017 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfb461b7:::benchmark_data_smithi022_188626_object95038:head# 2024-09-11T07:50:10.017 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfb8c9a0:::benchmark_data_smithi022_188626_object106140:head# 2024-09-11T07:50:10.018 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfbd6316:::benchmark_data_smithi022_188626_object102659:head# 2024-09-11T07:50:10.018 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfbe78bc:::benchmark_data_smithi022_188626_object98712:head# 2024-09-11T07:50:10.018 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfc1811f:::benchmark_data_smithi022_188626_object99172:head# 2024-09-11T07:50:10.018 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfc1ff93:::benchmark_data_smithi022_188626_object102415:head# 2024-09-11T07:50:10.018 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfc3b20f:::benchmark_data_smithi022_188626_object90711:head# 2024-09-11T07:50:10.018 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfc3f63e:::benchmark_data_smithi022_188626_object95267:head# 2024-09-11T07:50:10.018 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfc496e0:::benchmark_data_smithi022_188626_object92357:head# 2024-09-11T07:50:10.018 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfc4cad1:::benchmark_data_smithi022_188626_object110368:head# 2024-09-11T07:50:10.018 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfc9b765:::benchmark_data_smithi022_188626_object103980:head# 2024-09-11T07:50:10.018 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfcad8e4:::benchmark_data_smithi022_188626_object119264:head# 2024-09-11T07:50:10.018 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfcc82df:::benchmark_data_smithi022_188626_object106160:head# 2024-09-11T07:50:10.018 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfce6882:::benchmark_data_smithi022_188626_object109812:head# 2024-09-11T07:50:10.018 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfd0562c:::benchmark_data_smithi022_188626_object116747:head# 2024-09-11T07:50:10.018 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfd0b12c:::benchmark_data_smithi022_188626_object116045:head# 2024-09-11T07:50:10.019 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfd0bac7:::benchmark_data_smithi022_188626_object111695:head# 2024-09-11T07:50:10.019 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfd0ec5c:::benchmark_data_smithi022_188626_object93775:head# 2024-09-11T07:50:10.019 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfd3bcb4:::benchmark_data_smithi022_188626_object101643:head# 2024-09-11T07:50:10.019 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfd74ef4:::benchmark_data_smithi022_188626_object117630:head# 2024-09-11T07:50:10.019 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfd7741e:::benchmark_data_smithi022_188626_object118271:head# 2024-09-11T07:50:10.019 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfd7deb8:::benchmark_data_smithi022_188626_object108601:head# 2024-09-11T07:50:10.019 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfd8f484:::benchmark_data_smithi022_188626_object113108:head# 2024-09-11T07:50:10.019 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfda2048:::benchmark_data_smithi022_188626_object115437:head# 2024-09-11T07:50:10.019 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfdbf485:::benchmark_data_smithi022_188626_object106614:head# 2024-09-11T07:50:10.019 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfdcbc57:::benchmark_data_smithi022_188626_object92362:head# 2024-09-11T07:50:10.019 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfdf2ef1:::benchmark_data_smithi022_188626_object117579:head# 2024-09-11T07:50:10.019 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfdf7317:::benchmark_data_smithi022_188626_object112158:head# 2024-09-11T07:50:10.019 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfe1571f:::benchmark_data_smithi022_188626_object91795:head# 2024-09-11T07:50:10.019 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfe51d7d:::benchmark_data_smithi022_188626_object99557:head# 2024-09-11T07:50:10.019 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfe5b2ea:::benchmark_data_smithi022_188626_object100685:head# 2024-09-11T07:50:10.020 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfe7fcf6:::benchmark_data_smithi022_188626_object103779:head# 2024-09-11T07:50:10.020 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfe9a88b:::benchmark_data_smithi022_188626_object116412:head# 2024-09-11T07:50:10.020 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfeaa491:::benchmark_data_smithi022_188626_object93858:head# 2024-09-11T07:50:10.020 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfeb6803:::benchmark_data_smithi022_188626_object96805:head# 2024-09-11T07:50:10.020 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfedbc43:::benchmark_data_smithi022_188626_object103355:head# 2024-09-11T07:50:10.020 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfedd98c:::benchmark_data_smithi022_188626_object99048:head# 2024-09-11T07:50:10.020 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfee3708:::benchmark_data_smithi022_188626_object110772:head# 2024-09-11T07:50:10.020 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfef88b2:::benchmark_data_smithi022_188626_object96832:head# 2024-09-11T07:50:10.020 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cff0fbab:::benchmark_data_smithi022_188626_object119463:head# 2024-09-11T07:50:10.020 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cff30ae6:::benchmark_data_smithi022_188626_object113433:head# 2024-09-11T07:50:10.020 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cff33c13:::benchmark_data_smithi022_188626_object112252:head# 2024-09-11T07:50:10.020 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cff37791:::benchmark_data_smithi022_188626_object96093:head# 2024-09-11T07:50:10.020 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cffb4fd3:::benchmark_data_smithi022_188626_object100521:head# 2024-09-11T07:50:10.021 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cffb6e62:::benchmark_data_smithi022_188626_object107440:head# 2024-09-11T07:50:10.021 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cffe624e:::benchmark_data_smithi022_188626_object99390:head# 2024-09-11T07:50:10.021 INFO:teuthology.orchestra.run.smithi022.stderr:Read #11:cfff0682:::benchmark_data_smithi022_188626_object93337:head# 2024-09-11T07:50:10.021 INFO:teuthology.orchestra.run.smithi022.stderr:Export successful 2024-09-11T07:50:10.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:10 smithi022 ceph-mon[29462]: pgmap v1100: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 558 B/s rd, 0 op/s 2024-09-11T07:50:10.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:50:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:50:10] "GET /metrics HTTP/1.1" 200 207420 "" "Prometheus/2.43.0" 2024-09-11T07:50:10.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:10 smithi022 ceph-mon[37330]: pgmap v1100: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 558 B/s rd, 0 op/s 2024-09-11T07:50:10.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:10 smithi104 ceph-mon[33507]: pgmap v1100: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 558 B/s rd, 0 op/s 2024-09-11T07:50:10.780 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:50:10 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:50:11.280 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-11T07:50:11.554 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:50:11.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:11 smithi022 ceph-mon[29462]: 4.8 scrub starts 2024-09-11T07:50:11.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:11 smithi022 ceph-mon[29462]: 4.8 scrub ok 2024-09-11T07:50:11.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:11 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:50:11.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:11 smithi022 ceph-mon[29462]: 4.7 scrub starts 2024-09-11T07:50:11.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:11 smithi022 ceph-mon[29462]: 4.7 scrub ok 2024-09-11T07:50:11.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:11 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.11", "id": [1, 5]}]: dispatch 2024-09-11T07:50:11.657 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:11 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.1c", "id": [5, 1]}]: dispatch 2024-09-11T07:50:11.657 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:11 smithi022 ceph-mon[37330]: 4.8 scrub starts 2024-09-11T07:50:11.657 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:11 smithi022 ceph-mon[37330]: 4.8 scrub ok 2024-09-11T07:50:11.657 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:11 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:50:11.657 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:11 smithi022 ceph-mon[37330]: 4.7 scrub starts 2024-09-11T07:50:11.657 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:11 smithi022 ceph-mon[37330]: 4.7 scrub ok 2024-09-11T07:50:11.657 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:11 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.11", "id": [1, 5]}]: dispatch 2024-09-11T07:50:11.657 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:11 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.1c", "id": [5, 1]}]: dispatch 2024-09-11T07:50:11.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:11 smithi104 ceph-mon[33507]: 4.8 scrub starts 2024-09-11T07:50:11.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:11 smithi104 ceph-mon[33507]: 4.8 scrub ok 2024-09-11T07:50:11.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:11 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:50:11.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:11 smithi104 ceph-mon[33507]: 4.7 scrub starts 2024-09-11T07:50:11.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:11 smithi104 ceph-mon[33507]: 4.7 scrub ok 2024-09-11T07:50:11.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:11 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.11", "id": [1, 5]}]: dispatch 2024-09-11T07:50:11.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:11 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.1c", "id": [5, 1]}]: dispatch 2024-09-11T07:50:11.976 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -n osd.2 --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --force --op remove --pgid 11.3 2024-09-11T07:50:12.253 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/osd.2/config 2024-09-11T07:50:12.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:12 smithi022 ceph-mon[29462]: 4.4 scrub starts 2024-09-11T07:50:12.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:12 smithi022 ceph-mon[29462]: 4.4 scrub ok 2024-09-11T07:50:12.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:12 smithi022 ceph-mon[29462]: 6.1d scrub starts 2024-09-11T07:50:12.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:12 smithi022 ceph-mon[29462]: 6.1d scrub ok 2024-09-11T07:50:12.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:12 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.11", "id": [1, 5]}]': finished 2024-09-11T07:50:12.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:12 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.1c", "id": [5, 1]}]': finished 2024-09-11T07:50:12.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:12 smithi022 ceph-mon[29462]: osdmap e479: 8 total, 7 up, 5 in 2024-09-11T07:50:12.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:12 smithi022 ceph-mon[29462]: pgmap v1102: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:50:12.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:12 smithi022 ceph-mon[29462]: 2.17 scrub starts 2024-09-11T07:50:12.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:12 smithi022 ceph-mon[29462]: 2.17 scrub ok 2024-09-11T07:50:12.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:12 smithi022 ceph-mon[37330]: 4.4 scrub starts 2024-09-11T07:50:12.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:12 smithi022 ceph-mon[37330]: 4.4 scrub ok 2024-09-11T07:50:12.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:12 smithi022 ceph-mon[37330]: 6.1d scrub starts 2024-09-11T07:50:12.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:12 smithi022 ceph-mon[37330]: 6.1d scrub ok 2024-09-11T07:50:12.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:12 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.11", "id": [1, 5]}]': finished 2024-09-11T07:50:12.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:12 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.1c", "id": [5, 1]}]': finished 2024-09-11T07:50:12.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:12 smithi022 ceph-mon[37330]: osdmap e479: 8 total, 7 up, 5 in 2024-09-11T07:50:12.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:12 smithi022 ceph-mon[37330]: pgmap v1102: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:50:12.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:12 smithi022 ceph-mon[37330]: 2.17 scrub starts 2024-09-11T07:50:12.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:12 smithi022 ceph-mon[37330]: 2.17 scrub ok 2024-09-11T07:50:12.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:12 smithi104 ceph-mon[33507]: 4.4 scrub starts 2024-09-11T07:50:12.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:12 smithi104 ceph-mon[33507]: 4.4 scrub ok 2024-09-11T07:50:12.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:12 smithi104 ceph-mon[33507]: 6.1d scrub starts 2024-09-11T07:50:12.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:12 smithi104 ceph-mon[33507]: 6.1d scrub ok 2024-09-11T07:50:12.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:12 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.11", "id": [1, 5]}]': finished 2024-09-11T07:50:12.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:12 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.1c", "id": [5, 1]}]': finished 2024-09-11T07:50:12.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:12 smithi104 ceph-mon[33507]: osdmap e479: 8 total, 7 up, 5 in 2024-09-11T07:50:12.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:12 smithi104 ceph-mon[33507]: pgmap v1102: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:50:12.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:12 smithi104 ceph-mon[33507]: 2.17 scrub starts 2024-09-11T07:50:12.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:12 smithi104 ceph-mon[33507]: 2.17 scrub ok 2024-09-11T07:50:13.334 INFO:teuthology.orchestra.run.smithi022.stderr:noscrub is unset 2024-09-11T07:50:13.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:13 smithi022 ceph-mon[29462]: 6.4 scrub starts 2024-09-11T07:50:13.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:13 smithi022 ceph-mon[29462]: 6.4 scrub ok 2024-09-11T07:50:13.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:13 smithi022 ceph-mon[29462]: osdmap e480: 8 total, 7 up, 5 in 2024-09-11T07:50:13.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:13 smithi022 ceph-mon[29462]: 5.11 scrub starts 2024-09-11T07:50:13.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:13 smithi022 ceph-mon[29462]: 5.11 scrub ok 2024-09-11T07:50:13.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:13 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/685985251' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:50:13.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:13 smithi022 ceph-mon[29462]: from='client.26183 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:50:13.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:13 smithi022 ceph-mon[37330]: 6.4 scrub starts 2024-09-11T07:50:13.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:13 smithi022 ceph-mon[37330]: 6.4 scrub ok 2024-09-11T07:50:13.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:13 smithi022 ceph-mon[37330]: osdmap e480: 8 total, 7 up, 5 in 2024-09-11T07:50:13.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:13 smithi022 ceph-mon[37330]: 5.11 scrub starts 2024-09-11T07:50:13.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:13 smithi022 ceph-mon[37330]: 5.11 scrub ok 2024-09-11T07:50:13.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:13 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/685985251' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:50:13.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:13 smithi022 ceph-mon[37330]: from='client.26183 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:50:13.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:13 smithi104 ceph-mon[33507]: 6.4 scrub starts 2024-09-11T07:50:13.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:13 smithi104 ceph-mon[33507]: 6.4 scrub ok 2024-09-11T07:50:13.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:13 smithi104 ceph-mon[33507]: osdmap e480: 8 total, 7 up, 5 in 2024-09-11T07:50:13.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:13 smithi104 ceph-mon[33507]: 5.11 scrub starts 2024-09-11T07:50:13.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:13 smithi104 ceph-mon[33507]: 5.11 scrub ok 2024-09-11T07:50:13.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:13 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/685985251' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:50:13.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:13 smithi104 ceph-mon[33507]: from='client.26183 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-11T07:50:14.334 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:50:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:50:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:50:14.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:14 smithi022 ceph-mon[29462]: from='client.26183 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:50:14.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:14 smithi022 ceph-mon[29462]: osdmap e481: 8 total, 7 up, 5 in 2024-09-11T07:50:14.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:14 smithi022 ceph-mon[29462]: pgmap v1105: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:50:14.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:14 smithi022 ceph-mon[37330]: from='client.26183 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:50:14.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:14 smithi022 ceph-mon[37330]: osdmap e481: 8 total, 7 up, 5 in 2024-09-11T07:50:14.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:14 smithi022 ceph-mon[37330]: pgmap v1105: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:50:14.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:14 smithi104 ceph-mon[33507]: from='client.26183 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-11T07:50:14.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:14 smithi104 ceph-mon[33507]: osdmap e481: 8 total, 7 up, 5 in 2024-09-11T07:50:14.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:14 smithi104 ceph-mon[33507]: pgmap v1105: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:50:15.427 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-11T07:50:15.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:15 smithi022 ceph-mon[29462]: 2.12 scrub starts 2024-09-11T07:50:15.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:15 smithi022 ceph-mon[29462]: 2.12 scrub ok 2024-09-11T07:50:15.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:15 smithi022 ceph-mon[37330]: 2.12 scrub starts 2024-09-11T07:50:15.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:15 smithi022 ceph-mon[37330]: 2.12 scrub ok 2024-09-11T07:50:15.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:15 smithi104 ceph-mon[33507]: 2.12 scrub starts 2024-09-11T07:50:15.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:15 smithi104 ceph-mon[33507]: 2.12 scrub ok 2024-09-11T07:50:16.542 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:16 smithi022 ceph-mon[29462]: 2.2 scrub starts 2024-09-11T07:50:16.542 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:16 smithi022 ceph-mon[29462]: 2.2 scrub ok 2024-09-11T07:50:16.542 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:16 smithi022 ceph-mon[29462]: pgmap v1106: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.7 KiB/s rd, 1 op/s; 61 B/s, 0 objects/s recovering 2024-09-11T07:50:16.542 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:16 smithi022 ceph-mon[29462]: 5.3 scrub starts 2024-09-11T07:50:16.542 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:16 smithi022 ceph-mon[29462]: 5.3 scrub ok 2024-09-11T07:50:16.543 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:16 smithi022 ceph-mon[37330]: 2.2 scrub starts 2024-09-11T07:50:16.543 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:16 smithi022 ceph-mon[37330]: 2.2 scrub ok 2024-09-11T07:50:16.543 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:16 smithi022 ceph-mon[37330]: pgmap v1106: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.7 KiB/s rd, 1 op/s; 61 B/s, 0 objects/s recovering 2024-09-11T07:50:16.543 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:16 smithi022 ceph-mon[37330]: 5.3 scrub starts 2024-09-11T07:50:16.543 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:16 smithi022 ceph-mon[37330]: 5.3 scrub ok 2024-09-11T07:50:16.544 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:50:16.544 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:50:16.544 INFO:teuthology.orchestra.run.smithi022.stdout: "complaint_time": 30, 2024-09-11T07:50:16.544 INFO:teuthology.orchestra.run.smithi022.stdout: "num_blocked_ops": 0 2024-09-11T07:50:16.544 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:50:16.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:16 smithi104 ceph-mon[33507]: 2.2 scrub starts 2024-09-11T07:50:16.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:16 smithi104 ceph-mon[33507]: 2.2 scrub ok 2024-09-11T07:50:16.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:16 smithi104 ceph-mon[33507]: pgmap v1106: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.7 KiB/s rd, 1 op/s; 61 B/s, 0 objects/s recovering 2024-09-11T07:50:16.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:16 smithi104 ceph-mon[33507]: 5.3 scrub starts 2024-09-11T07:50:16.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:16 smithi104 ceph-mon[33507]: 5.3 scrub ok 2024-09-11T07:50:17.251 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-11T07:50:17.516 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:17 smithi022 ceph-mon[29462]: 5.1d scrub starts 2024-09-11T07:50:17.516 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:17 smithi022 ceph-mon[29462]: 5.1d scrub ok 2024-09-11T07:50:17.516 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:17 smithi022 ceph-mon[37330]: 5.1d scrub starts 2024-09-11T07:50:17.517 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:17 smithi022 ceph-mon[37330]: 5.1d scrub ok 2024-09-11T07:50:17.558 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:50:17.719 INFO:teuthology.orchestra.run.smithi022.stdout: marking collection for removal 2024-09-11T07:50:17.720 INFO:teuthology.orchestra.run.smithi022.stdout:setting '_remove' omap key 2024-09-11T07:50:17.720 INFO:teuthology.orchestra.run.smithi022.stdout:finish_remove_pgs 11.3_head removing 11.3 2024-09-11T07:50:17.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:17 smithi104 ceph-mon[33507]: 5.1d scrub starts 2024-09-11T07:50:17.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:17 smithi104 ceph-mon[33507]: 5.1d scrub ok 2024-09-11T07:50:17.795 INFO:teuthology.orchestra.run.smithi022.stdout:Remove successful 2024-09-11T07:50:18.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:18 smithi022 ceph-mon[29462]: 2.1b scrub starts 2024-09-11T07:50:18.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:18 smithi022 ceph-mon[29462]: 2.1b scrub ok 2024-09-11T07:50:18.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:18 smithi022 ceph-mon[29462]: pgmap v1107: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 808 B/s rd, 0 op/s; 58 B/s, 0 objects/s recovering 2024-09-11T07:50:18.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:18 smithi022 ceph-mon[37330]: 2.1b scrub starts 2024-09-11T07:50:18.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:18 smithi022 ceph-mon[37330]: 2.1b scrub ok 2024-09-11T07:50:18.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:18 smithi022 ceph-mon[37330]: pgmap v1107: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 808 B/s rd, 0 op/s; 58 B/s, 0 objects/s recovering 2024-09-11T07:50:18.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:18 smithi104 ceph-mon[33507]: 2.1b scrub starts 2024-09-11T07:50:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:18 smithi104 ceph-mon[33507]: 2.1b scrub ok 2024-09-11T07:50:18.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:18 smithi104 ceph-mon[33507]: pgmap v1107: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 808 B/s rd, 0 op/s; 58 B/s, 0 objects/s recovering 2024-09-11T07:50:19.120 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -n osd.2 --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.11.3.2 2024-09-11T07:50:19.131 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:50:19.405 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/osd.2/config 2024-09-11T07:50:19.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:19 smithi022 ceph-mon[29462]: 4.e scrub starts 2024-09-11T07:50:19.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:19 smithi022 ceph-mon[29462]: 4.e scrub ok 2024-09-11T07:50:19.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:19 smithi022 ceph-mon[37330]: 4.e scrub starts 2024-09-11T07:50:19.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:19 smithi022 ceph-mon[37330]: 4.e scrub ok 2024-09-11T07:50:19.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:19 smithi104 ceph-mon[33507]: 4.e scrub starts 2024-09-11T07:50:19.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:19 smithi104 ceph-mon[33507]: 4.e scrub ok 2024-09-11T07:50:20.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:20 smithi022 ceph-mon[29462]: 4.2 scrub starts 2024-09-11T07:50:20.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:20 smithi022 ceph-mon[29462]: 4.2 scrub ok 2024-09-11T07:50:20.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:20 smithi022 ceph-mon[29462]: pgmap v1108: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 639 B/s rd, 0 op/s; 46 B/s, 0 objects/s recovering 2024-09-11T07:50:20.417 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:50:20 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:50:20] "GET /metrics HTTP/1.1" 200 207420 "" "Prometheus/2.43.0" 2024-09-11T07:50:20.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:20 smithi022 ceph-mon[37330]: 4.2 scrub starts 2024-09-11T07:50:20.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:20 smithi022 ceph-mon[37330]: 4.2 scrub ok 2024-09-11T07:50:20.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:20 smithi022 ceph-mon[37330]: pgmap v1108: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 639 B/s rd, 0 op/s; 46 B/s, 0 objects/s recovering 2024-09-11T07:50:20.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:50:20 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:50:20.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:20 smithi104 ceph-mon[33507]: 4.2 scrub starts 2024-09-11T07:50:20.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:20 smithi104 ceph-mon[33507]: 4.2 scrub ok 2024-09-11T07:50:20.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:20 smithi104 ceph-mon[33507]: pgmap v1108: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 639 B/s rd, 0 op/s; 46 B/s, 0 objects/s recovering 2024-09-11T07:50:21.376 INFO:teuthology.orchestra.run.smithi022.stderr:nodeep-scrub is unset 2024-09-11T07:50:21.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:21 smithi022 ceph-mon[37330]: 2.1e scrub starts 2024-09-11T07:50:21.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:21 smithi022 ceph-mon[37330]: 2.1e scrub ok 2024-09-11T07:50:21.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:21 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:50:21.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:21 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1793689142' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:50:21.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:21 smithi022 ceph-mon[37330]: from='client.26189 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:50:21.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:21 smithi022 ceph-mon[29462]: 2.1e scrub starts 2024-09-11T07:50:21.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:21 smithi022 ceph-mon[29462]: 2.1e scrub ok 2024-09-11T07:50:21.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:21 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:50:21.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:21 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1793689142' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:50:21.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:21 smithi022 ceph-mon[29462]: from='client.26189 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:50:21.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:21 smithi104 ceph-mon[33507]: 2.1e scrub starts 2024-09-11T07:50:21.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:21 smithi104 ceph-mon[33507]: 2.1e scrub ok 2024-09-11T07:50:21.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:21 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:50:21.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:21 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1793689142' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:50:21.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:21 smithi104 ceph-mon[33507]: from='client.26189 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-11T07:50:22.633 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:22 smithi022 ceph-mon[29462]: 4.15 scrub starts 2024-09-11T07:50:22.634 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:22 smithi022 ceph-mon[29462]: 4.15 scrub ok 2024-09-11T07:50:22.634 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:22 smithi022 ceph-mon[29462]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:50:22.634 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:22 smithi022 ceph-mon[29462]: from='client.26189 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:50:22.634 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:22 smithi022 ceph-mon[29462]: osdmap e482: 8 total, 7 up, 5 in 2024-09-11T07:50:22.634 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:22 smithi022 ceph-mon[29462]: 5.c scrub starts 2024-09-11T07:50:22.634 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:22 smithi022 ceph-mon[29462]: 5.c scrub ok 2024-09-11T07:50:22.634 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:22 smithi022 ceph-mon[29462]: pgmap v1110: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.2 KiB/s rd, 1 op/s; 44 B/s, 0 objects/s recovering 2024-09-11T07:50:22.634 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:22 smithi022 ceph-mon[37330]: 4.15 scrub starts 2024-09-11T07:50:22.634 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:22 smithi022 ceph-mon[37330]: 4.15 scrub ok 2024-09-11T07:50:22.634 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:22 smithi022 ceph-mon[37330]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:50:22.635 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:22 smithi022 ceph-mon[37330]: from='client.26189 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:50:22.635 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:22 smithi022 ceph-mon[37330]: osdmap e482: 8 total, 7 up, 5 in 2024-09-11T07:50:22.635 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:22 smithi022 ceph-mon[37330]: 5.c scrub starts 2024-09-11T07:50:22.635 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:22 smithi022 ceph-mon[37330]: 5.c scrub ok 2024-09-11T07:50:22.635 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:22 smithi022 ceph-mon[37330]: pgmap v1110: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.2 KiB/s rd, 1 op/s; 44 B/s, 0 objects/s recovering 2024-09-11T07:50:22.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:22 smithi104 ceph-mon[33507]: 4.15 scrub starts 2024-09-11T07:50:22.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:22 smithi104 ceph-mon[33507]: 4.15 scrub ok 2024-09-11T07:50:22.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:22 smithi104 ceph-mon[33507]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-11T07:50:22.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:22 smithi104 ceph-mon[33507]: from='client.26189 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-11T07:50:22.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:22 smithi104 ceph-mon[33507]: osdmap e482: 8 total, 7 up, 5 in 2024-09-11T07:50:22.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:22 smithi104 ceph-mon[33507]: 5.c scrub starts 2024-09-11T07:50:22.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:22 smithi104 ceph-mon[33507]: 5.c scrub ok 2024-09-11T07:50:22.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:22 smithi104 ceph-mon[33507]: pgmap v1110: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.2 KiB/s rd, 1 op/s; 44 B/s, 0 objects/s recovering 2024-09-11T07:50:23.383 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:50:23 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:50:23.072+0000 7f0b5d524640 -1 mgr get_metadata_python Requested missing service osd.2 2024-09-11T07:50:23.666 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:23 smithi022 ceph-mon[37330]: 5.13 scrub starts 2024-09-11T07:50:23.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:23 smithi022 ceph-mon[37330]: 5.13 scrub ok 2024-09-11T07:50:23.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:23 smithi022 ceph-mon[37330]: 5.1e deep-scrub starts 2024-09-11T07:50:23.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:23 smithi022 ceph-mon[37330]: 5.1e deep-scrub ok 2024-09-11T07:50:23.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:50:23.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:50:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:23 smithi022 ceph-mon[29462]: 5.13 scrub starts 2024-09-11T07:50:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:23 smithi022 ceph-mon[29462]: 5.13 scrub ok 2024-09-11T07:50:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:23 smithi022 ceph-mon[29462]: 5.1e deep-scrub starts 2024-09-11T07:50:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:23 smithi022 ceph-mon[29462]: 5.1e deep-scrub ok 2024-09-11T07:50:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:50:23.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:50:23.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:23 smithi104 ceph-mon[33507]: 5.13 scrub starts 2024-09-11T07:50:23.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:23 smithi104 ceph-mon[33507]: 5.13 scrub ok 2024-09-11T07:50:23.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:23 smithi104 ceph-mon[33507]: 5.1e deep-scrub starts 2024-09-11T07:50:23.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:23 smithi104 ceph-mon[33507]: 5.1e deep-scrub ok 2024-09-11T07:50:23.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:50:23.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:50:24.104 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:50:24.104 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 20, 2024-09-11T07:50:24.104 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 600, 2024-09-11T07:50:24.104 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:50:24.104 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.104 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:234463 9.2 9:4445ea71:::benchmark_data_smithi022_133943_object107872:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:50:24.104 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:59.743553+0000", 2024-09-11T07:50:24.105 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 564.35784023199994, 2024-09-11T07:50:24.105 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1848900979999999, 2024-09-11T07:50:24.105 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:24.105 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:24.105 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:24.105 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:24.105 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:24.105 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 234463 2024-09-11T07:50:24.105 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.105 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:24.105 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.106 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:24.106 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.743553+0000", 2024-09-11T07:50:24.106 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.106 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.106 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.106 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:24.106 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.743553+0000", 2024-09-11T07:50:24.106 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.106 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.106 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.106 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:24.106 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.743552+0000", 2024-09-11T07:50:24.106 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:24.106 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.106 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.107 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:24.107 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.743558+0000", 2024-09-11T07:50:24.107 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0440000000000001e-06 2024-09-11T07:50:24.107 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.107 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.107 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:24.107 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.743573+0000", 2024-09-11T07:50:24.107 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5206e-05 2024-09-11T07:50:24.107 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.107 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.107 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:24.107 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.743614+0000", 2024-09-11T07:50:24.107 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1532000000000001e-05 2024-09-11T07:50:24.108 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.108 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.108 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.108 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.270167+0000", 2024-09-11T07:50:24.108 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.52655248300000002 2024-09-11T07:50:24.108 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.108 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.108 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:24.108 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.270264+0000", 2024-09-11T07:50:24.108 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.7097999999999998e-05 2024-09-11T07:50:24.108 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.108 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.109 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,7", 2024-09-11T07:50:24.109 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.270365+0000", 2024-09-11T07:50:24.109 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000100635 2024-09-11T07:50:24.109 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.109 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.109 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:24.109 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274143+0000", 2024-09-11T07:50:24.109 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0037784020000000001 2024-09-11T07:50:24.109 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.109 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.109 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.109 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.277336+0000", 2024-09-11T07:50:24.109 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0031928590000000001 2024-09-11T07:50:24.110 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.110 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.110 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.110 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928350+0000", 2024-09-11T07:50:24.110 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.65101431899999995 2024-09-11T07:50:24.110 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.110 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.110 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:24.110 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928401+0000", 2024-09-11T07:50:24.110 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1303e-05 2024-09-11T07:50:24.110 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.110 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.110 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:24.111 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928443+0000", 2024-09-11T07:50:24.111 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1795999999999998e-05 2024-09-11T07:50:24.111 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.111 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:24.111 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.111 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.111 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.111 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:234477 9.a 9:5131a6fa:::benchmark_data_smithi022_133943_object107886:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:50:24.111 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:59.763716+0000", 2024-09-11T07:50:24.111 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 564.33767765799996, 2024-09-11T07:50:24.111 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1647252889999999, 2024-09-11T07:50:24.111 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:24.111 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:24.112 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:24.112 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:24.112 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:24.112 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 234477 2024-09-11T07:50:24.112 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.112 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:24.112 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.112 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:24.112 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.763716+0000", 2024-09-11T07:50:24.112 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.112 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.112 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.112 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:24.113 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.763716+0000", 2024-09-11T07:50:24.113 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.113 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.113 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.113 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:24.113 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.763714+0000", 2024-09-11T07:50:24.113 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:50:24.113 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.113 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.113 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:24.113 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.763729+0000", 2024-09-11T07:50:24.113 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5347999999999999e-05 2024-09-11T07:50:24.113 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.114 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.114 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:24.114 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.763740+0000", 2024-09-11T07:50:24.114 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1165000000000001e-05 2024-09-11T07:50:24.114 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.114 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.114 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:24.114 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.763745+0000", 2024-09-11T07:50:24.114 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1050000000000003e-06 2024-09-11T07:50:24.114 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.114 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.114 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.114 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.269894+0000", 2024-09-11T07:50:24.115 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.50614913299999997 2024-09-11T07:50:24.115 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.115 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.115 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:24.115 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.269999+0000", 2024-09-11T07:50:24.115 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000105189 2024-09-11T07:50:24.115 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.115 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.115 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:50:24.115 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.270104+0000", 2024-09-11T07:50:24.115 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010485899999999999 2024-09-11T07:50:24.115 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.115 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.115 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.116 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274132+0000", 2024-09-11T07:50:24.116 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0040275689999999999 2024-09-11T07:50:24.116 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.116 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.116 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:24.116 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274138+0000", 2024-09-11T07:50:24.116 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5729999999999996e-06 2024-09-11T07:50:24.116 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.116 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.116 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.116 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928373+0000", 2024-09-11T07:50:24.116 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.65423462899999996 2024-09-11T07:50:24.116 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.117 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.117 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:24.117 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928412+0000", 2024-09-11T07:50:24.117 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9181000000000003e-05 2024-09-11T07:50:24.117 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.117 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.117 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:24.117 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928441+0000", 2024-09-11T07:50:24.117 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8656000000000001e-05 2024-09-11T07:50:24.117 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.117 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:24.117 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.117 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.117 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.118 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:234480 9.2 9:45c5c2ce:::benchmark_data_smithi022_133943_object107889:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:50:24.118 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:59.767350+0000", 2024-09-11T07:50:24.118 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 564.33404343999996, 2024-09-11T07:50:24.118 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.161190985, 2024-09-11T07:50:24.118 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:24.118 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:24.118 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:24.118 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:24.118 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:24.118 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 234480 2024-09-11T07:50:24.118 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.118 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:24.118 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.119 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:24.119 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.767350+0000", 2024-09-11T07:50:24.119 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.119 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.120 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.120 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:24.120 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.767350+0000", 2024-09-11T07:50:24.120 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.120 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.120 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.120 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:24.120 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.767347+0000", 2024-09-11T07:50:24.120 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:50:24.120 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.120 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.120 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:24.121 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.767356+0000", 2024-09-11T07:50:24.121 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.6160000000000002e-06 2024-09-11T07:50:24.121 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.121 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.121 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:24.121 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.767367+0000", 2024-09-11T07:50:24.121 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1946999999999999e-05 2024-09-11T07:50:24.121 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.121 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.121 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:24.121 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.767373+0000", 2024-09-11T07:50:24.121 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9159999999999998e-06 2024-09-11T07:50:24.121 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.121 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.122 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.122 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274356+0000", 2024-09-11T07:50:24.122 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.50698302299999998 2024-09-11T07:50:24.122 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.122 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.122 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:24.122 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274462+0000", 2024-09-11T07:50:24.122 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000105349 2024-09-11T07:50:24.122 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.122 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.122 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,7", 2024-09-11T07:50:24.122 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274568+0000", 2024-09-11T07:50:24.123 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010647899999999999 2024-09-11T07:50:24.123 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.123 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.123 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:24.123 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.279627+0000", 2024-09-11T07:50:24.123 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0050591680000000002 2024-09-11T07:50:24.123 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.123 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.123 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.123 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.321730+0000", 2024-09-11T07:50:24.123 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.042102300000000002 2024-09-11T07:50:24.123 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.123 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.124 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.124 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928475+0000", 2024-09-11T07:50:24.124 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.60674521000000003 2024-09-11T07:50:24.124 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.124 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.124 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:24.124 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928510+0000", 2024-09-11T07:50:24.124 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5036999999999997e-05 2024-09-11T07:50:24.124 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.124 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.124 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:24.124 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928541+0000", 2024-09-11T07:50:24.125 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0879e-05 2024-09-11T07:50:24.125 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.125 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:24.125 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.125 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.125 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.125 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:234484 9.a 9:5db53976:::benchmark_data_smithi022_133943_object107893:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:50:24.125 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:59.776130+0000", 2024-09-11T07:50:24.125 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 564.32526339499998, 2024-09-11T07:50:24.125 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1524430130000001, 2024-09-11T07:50:24.125 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:24.125 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:24.125 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:24.126 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:24.126 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:24.126 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 234484 2024-09-11T07:50:24.126 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.126 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:24.126 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.126 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:24.126 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.776130+0000", 2024-09-11T07:50:24.126 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.126 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.126 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.126 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:24.126 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.776130+0000", 2024-09-11T07:50:24.127 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.127 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.127 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.127 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:24.127 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.776128+0000", 2024-09-11T07:50:24.127 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:50:24.127 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.127 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.127 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:24.127 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.776136+0000", 2024-09-11T07:50:24.127 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3410000000000003e-06 2024-09-11T07:50:24.127 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.127 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.127 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:24.128 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.776165+0000", 2024-09-11T07:50:24.128 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.9031e-05 2024-09-11T07:50:24.128 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.128 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.128 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:24.128 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.776171+0000", 2024-09-11T07:50:24.128 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1929999999999998e-06 2024-09-11T07:50:24.128 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.128 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.128 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.128 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274214+0000", 2024-09-11T07:50:24.128 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.49804272399999999 2024-09-11T07:50:24.128 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.129 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.129 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:24.129 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274342+0000", 2024-09-11T07:50:24.129 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012783099999999999 2024-09-11T07:50:24.129 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.129 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.129 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 3,7", 2024-09-11T07:50:24.129 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.274443+0000", 2024-09-11T07:50:24.129 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000101517 2024-09-11T07:50:24.129 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.129 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.129 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.129 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.277315+0000", 2024-09-11T07:50:24.129 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002871958 2024-09-11T07:50:24.130 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.130 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.130 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:24.130 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.277321+0000", 2024-09-11T07:50:24.130 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9569999999999999e-06 2024-09-11T07:50:24.130 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.130 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.130 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.130 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928494+0000", 2024-09-11T07:50:24.130 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.651173212 2024-09-11T07:50:24.130 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.130 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.130 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:24.130 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928537+0000", 2024-09-11T07:50:24.131 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2274999999999998e-05 2024-09-11T07:50:24.131 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.131 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.131 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:24.131 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928573+0000", 2024-09-11T07:50:24.131 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6310000000000003e-05 2024-09-11T07:50:24.131 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.131 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:24.131 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.131 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.131 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.131 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:234493 9.2 9:43b51550:::benchmark_data_smithi022_133943_object107902:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:50:24.131 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:59.812761+0000", 2024-09-11T07:50:24.132 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 564.28863240800001, 2024-09-11T07:50:24.132 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1158854009999999, 2024-09-11T07:50:24.132 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:24.132 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:24.132 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:24.132 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:24.132 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:24.132 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 234493 2024-09-11T07:50:24.132 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.132 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:24.132 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.132 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:24.132 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.812761+0000", 2024-09-11T07:50:24.132 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.133 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.133 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.133 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:24.133 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.812761+0000", 2024-09-11T07:50:24.133 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.133 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.133 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.133 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:24.133 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.812758+0000", 2024-09-11T07:50:24.133 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:50:24.133 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.133 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.133 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:24.134 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.812767+0000", 2024-09-11T07:50:24.134 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.5229999999999995e-06 2024-09-11T07:50:24.134 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.134 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.134 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:24.134 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.812780+0000", 2024-09-11T07:50:24.134 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2869e-05 2024-09-11T07:50:24.134 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.134 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.134 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:24.134 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:59.812786+0000", 2024-09-11T07:50:24.134 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5479999999999998e-06 2024-09-11T07:50:24.134 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.135 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.135 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.135 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.277362+0000", 2024-09-11T07:50:24.135 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.46457584499999999 2024-09-11T07:50:24.135 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.135 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.135 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:24.135 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.277474+0000", 2024-09-11T07:50:24.135 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011171 2024-09-11T07:50:24.135 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.135 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.135 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,7", 2024-09-11T07:50:24.135 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.277592+0000", 2024-09-11T07:50:24.135 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011786900000000001 2024-09-11T07:50:24.136 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.136 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.136 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:24.136 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.321737+0000", 2024-09-11T07:50:24.136 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.044145292000000003 2024-09-11T07:50:24.136 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.136 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.136 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.136 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.321772+0000", 2024-09-11T07:50:24.136 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4910000000000003e-05 2024-09-11T07:50:24.136 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.136 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.136 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.137 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928574+0000", 2024-09-11T07:50:24.137 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.60680186700000005 2024-09-11T07:50:24.137 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.137 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.137 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:24.137 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928615+0000", 2024-09-11T07:50:24.137 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0728e-05 2024-09-11T07:50:24.137 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.137 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.137 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:24.137 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:00.928646+0000", 2024-09-11T07:50:24.137 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.1696999999999998e-05 2024-09-11T07:50:24.137 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.137 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:24.138 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.138 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.138 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.138 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235135 9.12 9:4bac11d3:::benchmark_data_smithi022_133943_object108544:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e321)", 2024-09-11T07:50:24.138 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.671922+0000", 2024-09-11T07:50:24.138 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 562.42947090200005, 2024-09-11T07:50:24.138 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.096036827, 2024-09-11T07:50:24.138 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:24.138 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:50:24.138 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:24.138 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:24.138 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:24.138 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235135 2024-09-11T07:50:24.139 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.139 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:24.139 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.139 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:24.139 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.671922+0000", 2024-09-11T07:50:24.139 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.139 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.139 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.139 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:24.139 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.671922+0000", 2024-09-11T07:50:24.139 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.139 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.139 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.140 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:24.140 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.671921+0000", 2024-09-11T07:50:24.140 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:24.140 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.140 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.140 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:24.140 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.671926+0000", 2024-09-11T07:50:24.140 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1359999999999996e-06 2024-09-11T07:50:24.140 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.140 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.140 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:24.140 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.671937+0000", 2024-09-11T07:50:24.140 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0676000000000001e-05 2024-09-11T07:50:24.141 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.141 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.141 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:24.141 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.671942+0000", 2024-09-11T07:50:24.141 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.062e-06 2024-09-11T07:50:24.141 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.141 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.141 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.141 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.684807+0000", 2024-09-11T07:50:24.141 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.012864855 2024-09-11T07:50:24.141 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.141 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.141 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:24.142 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.684939+0000", 2024-09-11T07:50:24.142 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013172800000000001 2024-09-11T07:50:24.142 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.142 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.142 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2,5", 2024-09-11T07:50:24.142 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.685060+0000", 2024-09-11T07:50:24.142 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000121413 2024-09-11T07:50:24.142 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.142 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.142 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.142 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.700013+0000", 2024-09-11T07:50:24.142 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.014952995 2024-09-11T07:50:24.142 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.143 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.143 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:24.143 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.700021+0000", 2024-09-11T07:50:24.143 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.6729999999999992e-06 2024-09-11T07:50:24.143 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.143 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.143 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.143 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.722409+0000", 2024-09-11T07:50:24.143 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.022387961 2024-09-11T07:50:24.143 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.143 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.143 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for missing object", 2024-09-11T07:50:24.143 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.722505+0000", 2024-09-11T07:50:24.143 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.6838999999999999e-05 2024-09-11T07:50:24.143 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.144 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.144 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.144 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.767903+0000", 2024-09-11T07:50:24.144 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.045397237 2024-09-11T07:50:24.144 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.144 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.144 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:24.144 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.767959+0000", 2024-09-11T07:50:24.144 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6588000000000001e-05 2024-09-11T07:50:24.144 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.144 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:24.144 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.144 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.144 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.144 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235140 9.12 9:4a4ad0d1:::benchmark_data_smithi022_133943_object108549:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-11T07:50:24.145 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.685741+0000", 2024-09-11T07:50:24.145 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 562.41565226299997, 2024-09-11T07:50:24.145 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.082396656, 2024-09-11T07:50:24.145 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:24.145 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:24.145 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:24.145 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:24.145 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:24.145 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235140 2024-09-11T07:50:24.145 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.145 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:24.145 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.145 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:24.145 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.685741+0000", 2024-09-11T07:50:24.146 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.146 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.146 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.146 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:24.146 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.685741+0000", 2024-09-11T07:50:24.146 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.146 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.146 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.146 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:24.146 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.685739+0000", 2024-09-11T07:50:24.146 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:50:24.146 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.146 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.146 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:24.147 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.685746+0000", 2024-09-11T07:50:24.147 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0310000000000001e-06 2024-09-11T07:50:24.147 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.147 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.147 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:24.147 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.685757+0000", 2024-09-11T07:50:24.147 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0773e-05 2024-09-11T07:50:24.147 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.147 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.147 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:24.147 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.685763+0000", 2024-09-11T07:50:24.147 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2400000000000004e-06 2024-09-11T07:50:24.147 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.147 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.147 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.148 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.702365+0000", 2024-09-11T07:50:24.148 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.016602519 2024-09-11T07:50:24.148 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.148 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.148 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:24.148 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.702372+0000", 2024-09-11T07:50:24.148 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2299999999999996e-06 2024-09-11T07:50:24.148 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.148 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.148 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.148 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.722543+0000", 2024-09-11T07:50:24.148 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0201711920000001 2024-09-11T07:50:24.148 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.148 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.149 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:24.149 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.722638+0000", 2024-09-11T07:50:24.149 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.4757000000000001e-05 2024-09-11T07:50:24.149 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.149 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.149 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2", 2024-09-11T07:50:24.149 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.722761+0000", 2024-09-11T07:50:24.149 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012308800000000001 2024-09-11T07:50:24.149 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.149 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.149 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:24.149 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.733739+0000", 2024-09-11T07:50:24.150 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.01097806 2024-09-11T07:50:24.150 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.150 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.150 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.150 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.767979+0000", 2024-09-11T07:50:24.150 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.034240697 2024-09-11T07:50:24.150 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.150 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.150 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.150 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.767996+0000", 2024-09-11T07:50:24.150 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.6189e-05 2024-09-11T07:50:24.150 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.150 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.150 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:24.150 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768036+0000", 2024-09-11T07:50:24.151 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0198999999999999e-05 2024-09-11T07:50:24.151 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.151 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.151 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:24.151 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768138+0000", 2024-09-11T07:50:24.151 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000101866 2024-09-11T07:50:24.151 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.151 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:24.151 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.151 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.151 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.151 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235178 9.12 9:4ef9439e:::benchmark_data_smithi022_133943_object108587:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-11T07:50:24.151 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.722941+0000", 2024-09-11T07:50:24.151 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 562.37845228499998, 2024-09-11T07:50:24.152 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.045418086, 2024-09-11T07:50:24.152 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:24.152 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:24.152 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:24.152 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:24.152 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:24.152 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235178 2024-09-11T07:50:24.152 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.152 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:24.152 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.152 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:24.152 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.722941+0000", 2024-09-11T07:50:24.152 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.153 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.153 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.153 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:24.153 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.722941+0000", 2024-09-11T07:50:24.153 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.153 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.153 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.153 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:24.153 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.722940+0000", 2024-09-11T07:50:24.153 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:50:24.153 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.153 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.153 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:24.153 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.722943+0000", 2024-09-11T07:50:24.154 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8650000000000001e-06 2024-09-11T07:50:24.154 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.154 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.154 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:24.154 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.722948+0000", 2024-09-11T07:50:24.154 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3390000000000003e-06 2024-09-11T07:50:24.154 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.154 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.154 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:24.154 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.722950+0000", 2024-09-11T07:50:24.154 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7609999999999999e-06 2024-09-11T07:50:24.154 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.154 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.154 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.155 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.733542+0000", 2024-09-11T07:50:24.155 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.010591313 2024-09-11T07:50:24.155 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.155 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.155 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:24.155 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.733548+0000", 2024-09-11T07:50:24.155 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7719999999999997e-06 2024-09-11T07:50:24.155 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.155 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.155 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.155 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726822+0000", 2024-09-11T07:50:24.155 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.99327332599999996 2024-09-11T07:50:24.155 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.155 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.156 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:24.156 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.726931+0000", 2024-09-11T07:50:24.156 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010914 2024-09-11T07:50:24.156 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.156 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.156 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2", 2024-09-11T07:50:24.156 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.727048+0000", 2024-09-11T07:50:24.156 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011745599999999999 2024-09-11T07:50:24.156 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.156 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.156 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:24.156 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.743419+0000", 2024-09-11T07:50:24.156 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.016370268 2024-09-11T07:50:24.156 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.157 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.157 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.157 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768173+0000", 2024-09-11T07:50:24.157 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.024754438 2024-09-11T07:50:24.157 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.157 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.157 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.157 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768201+0000", 2024-09-11T07:50:24.157 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7946e-05 2024-09-11T07:50:24.157 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.157 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.157 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:24.157 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768246+0000", 2024-09-11T07:50:24.158 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4567999999999997e-05 2024-09-11T07:50:24.158 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.158 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.158 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:24.158 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768359+0000", 2024-09-11T07:50:24.158 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011356 2024-09-11T07:50:24.158 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.158 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:24.158 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.158 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.158 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.158 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235201 9.12 9:4fabe53d:::benchmark_data_smithi022_133943_object108610:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-11T07:50:24.158 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.747159+0000", 2024-09-11T07:50:24.158 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 562.35423428299998, 2024-09-11T07:50:24.159 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.021527321, 2024-09-11T07:50:24.159 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:24.159 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:24.159 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:24.159 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:24.159 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:24.159 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235201 2024-09-11T07:50:24.159 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.159 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:24.159 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.159 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:24.159 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.747159+0000", 2024-09-11T07:50:24.159 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.159 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.160 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.160 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:24.160 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.747159+0000", 2024-09-11T07:50:24.160 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.160 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.160 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.160 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:24.160 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.747157+0000", 2024-09-11T07:50:24.160 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:50:24.160 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.160 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.160 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:24.160 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.747164+0000", 2024-09-11T07:50:24.160 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7360000000000004e-06 2024-09-11T07:50:24.161 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.161 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.161 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:24.161 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.747192+0000", 2024-09-11T07:50:24.161 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8422e-05 2024-09-11T07:50:24.161 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.161 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.161 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:24.161 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.747201+0000", 2024-09-11T07:50:24.161 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.5290000000000008e-06 2024-09-11T07:50:24.161 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.161 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.161 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.162 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.747514+0000", 2024-09-11T07:50:24.162 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00031351600000000001 2024-09-11T07:50:24.162 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.162 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.162 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:24.162 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.747524+0000", 2024-09-11T07:50:24.162 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.533e-06 2024-09-11T07:50:24.162 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.162 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.162 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.162 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.730920+0000", 2024-09-11T07:50:24.162 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.98339653500000002 2024-09-11T07:50:24.162 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.163 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.163 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:24.163 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.731036+0000", 2024-09-11T07:50:24.163 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000115571 2024-09-11T07:50:24.163 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.163 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.163 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2", 2024-09-11T07:50:24.163 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.731157+0000", 2024-09-11T07:50:24.163 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000121109 2024-09-11T07:50:24.163 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.163 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.163 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:24.163 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.743425+0000", 2024-09-11T07:50:24.164 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.012267554999999999 2024-09-11T07:50:24.164 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.164 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.164 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.164 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768397+0000", 2024-09-11T07:50:24.164 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.024972765000000001 2024-09-11T07:50:24.164 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.164 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.164 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.164 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768428+0000", 2024-09-11T07:50:24.164 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0167000000000001e-05 2024-09-11T07:50:24.164 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.164 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.164 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:24.165 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768615+0000", 2024-09-11T07:50:24.165 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00018698900000000001 2024-09-11T07:50:24.165 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.165 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.165 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:24.165 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768686+0000", 2024-09-11T07:50:24.165 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1790000000000002e-05 2024-09-11T07:50:24.165 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.165 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:24.165 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.165 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.165 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.165 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235224 9.12 9:4fcd73f7:::benchmark_data_smithi022_133943_object108633:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-11T07:50:24.166 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.769489+0000", 2024-09-11T07:50:24.166 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 562.33190456499995, 2024-09-11T07:50:24.166 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.99944138900000001, 2024-09-11T07:50:24.166 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:24.166 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:24.166 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:24.166 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:24.166 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:24.166 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235224 2024-09-11T07:50:24.166 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.166 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:24.166 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.166 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:24.166 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.769489+0000", 2024-09-11T07:50:24.167 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.167 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.167 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.167 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:24.167 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.769489+0000", 2024-09-11T07:50:24.167 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.167 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.167 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.167 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:24.167 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.769487+0000", 2024-09-11T07:50:24.167 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:50:24.167 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.168 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.168 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:24.168 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.769494+0000", 2024-09-11T07:50:24.168 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.3649999999999998e-06 2024-09-11T07:50:24.168 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.168 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.168 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:24.168 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.769509+0000", 2024-09-11T07:50:24.168 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5122e-05 2024-09-11T07:50:24.168 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.168 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.168 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:24.168 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.769515+0000", 2024-09-11T07:50:24.168 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1330000000000004e-06 2024-09-11T07:50:24.169 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.169 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.169 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.169 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.771837+0000", 2024-09-11T07:50:24.169 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0023218840000000002 2024-09-11T07:50:24.169 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.169 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.169 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:24.169 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.771845+0000", 2024-09-11T07:50:24.169 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.2239999999999995e-06 2024-09-11T07:50:24.169 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.169 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.169 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.170 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.733765+0000", 2024-09-11T07:50:24.170 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.96191987199999995 2024-09-11T07:50:24.170 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.170 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.170 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:24.170 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.733865+0000", 2024-09-11T07:50:24.170 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010025499999999999 2024-09-11T07:50:24.170 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.170 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.170 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2", 2024-09-11T07:50:24.170 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.734245+0000", 2024-09-11T07:50:24.170 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00037957000000000003 2024-09-11T07:50:24.170 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.170 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.170 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:24.171 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.746336+0000", 2024-09-11T07:50:24.171 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.012091497 2024-09-11T07:50:24.171 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.171 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.171 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.171 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768732+0000", 2024-09-11T07:50:24.171 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.022395854999999999 2024-09-11T07:50:24.171 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.171 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.171 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.171 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768797+0000", 2024-09-11T07:50:24.171 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4886000000000002e-05 2024-09-11T07:50:24.171 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.171 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.171 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:24.171 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768837+0000", 2024-09-11T07:50:24.172 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0203000000000001e-05 2024-09-11T07:50:24.172 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.172 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.172 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:24.172 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768930+0000", 2024-09-11T07:50:24.172 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.2763999999999995e-05 2024-09-11T07:50:24.172 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.172 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:24.172 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.172 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.172 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.172 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:235274 9.12 9:48c72601:::benchmark_data_smithi022_133943_object108683:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e322)", 2024-09-11T07:50:24.172 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:41:01.815001+0000", 2024-09-11T07:50:24.173 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 562.286391923, 2024-09-11T07:50:24.173 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.95419106899999995, 2024-09-11T07:50:24.173 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:24.173 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:24.173 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:24.173 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:24.173 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:24.173 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 235274 2024-09-11T07:50:24.173 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.173 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:24.173 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.173 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:24.173 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.815001+0000", 2024-09-11T07:50:24.173 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.174 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.174 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.174 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:24.174 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.815001+0000", 2024-09-11T07:50:24.174 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.174 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.174 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.174 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:24.174 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.815000+0000", 2024-09-11T07:50:24.174 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:50:24.174 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.174 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.174 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:24.175 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.815006+0000", 2024-09-11T07:50:24.175 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.55e-06 2024-09-11T07:50:24.175 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.175 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.175 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:24.175 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.815015+0000", 2024-09-11T07:50:24.175 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.4030000000000007e-06 2024-09-11T07:50:24.175 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.175 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.175 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:24.175 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.815019+0000", 2024-09-11T07:50:24.175 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1760000000000003e-06 2024-09-11T07:50:24.175 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.175 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.176 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.177 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.815066+0000", 2024-09-11T07:50:24.177 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7092000000000002e-05 2024-09-11T07:50:24.177 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.177 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.177 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:24.177 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:01.815074+0000", 2024-09-11T07:50:24.177 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.7069999999999992e-06 2024-09-11T07:50:24.177 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.177 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.177 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.177 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.738191+0000", 2024-09-11T07:50:24.177 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.92311757100000003 2024-09-11T07:50:24.178 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.178 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.178 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:24.178 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.738316+0000", 2024-09-11T07:50:24.178 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000124573 2024-09-11T07:50:24.178 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.178 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.178 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 0,2", 2024-09-11T07:50:24.178 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.738445+0000", 2024-09-11T07:50:24.178 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012939399999999999 2024-09-11T07:50:24.178 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.178 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.178 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:24.179 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.749519+0000", 2024-09-11T07:50:24.179 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.011073697 2024-09-11T07:50:24.179 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.179 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.179 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.179 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.768769+0000", 2024-09-11T07:50:24.179 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.019249720000000001 2024-09-11T07:50:24.179 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.179 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.179 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.179 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.769045+0000", 2024-09-11T07:50:24.179 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000276335 2024-09-11T07:50:24.179 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.180 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.180 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:24.180 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.769089+0000", 2024-09-11T07:50:24.180 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4314999999999998e-05 2024-09-11T07:50:24.180 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.180 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.180 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:24.180 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:41:02.769192+0000", 2024-09-11T07:50:24.180 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010329500000000001 2024-09-11T07:50:24.180 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.180 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:24.180 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.180 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.180 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.181 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:233677 11.c 11:3e7ea642:::benchmark_data_smithi022_188626_object113869:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e468)", 2024-09-11T07:50:24.181 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:49:33.014199+0000", 2024-09-11T07:50:24.181 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 51.087194605999997, 2024-09-11T07:50:24.181 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.035895035, 2024-09-11T07:50:24.181 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:24.181 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:24.181 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:24.181 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:24.181 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:24.181 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 233677 2024-09-11T07:50:24.181 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.181 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:24.181 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.182 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:24.182 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.014199+0000", 2024-09-11T07:50:24.182 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.182 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.182 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.182 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:24.182 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.014199+0000", 2024-09-11T07:50:24.182 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.182 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.182 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.182 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:24.182 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.014196+0000", 2024-09-11T07:50:24.182 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:50:24.183 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.183 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.183 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:24.183 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.014205+0000", 2024-09-11T07:50:24.183 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.3090000000000001e-06 2024-09-11T07:50:24.183 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.183 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.183 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:24.183 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.014308+0000", 2024-09-11T07:50:24.183 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000102614 2024-09-11T07:50:24.183 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.183 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.183 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:24.183 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.014317+0000", 2024-09-11T07:50:24.184 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.0100000000000001e-06 2024-09-11T07:50:24.184 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.184 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.184 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.184 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.014433+0000", 2024-09-11T07:50:24.184 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000116335 2024-09-11T07:50:24.184 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.184 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.184 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:24.184 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.014441+0000", 2024-09-11T07:50:24.184 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.6909999999999997e-06 2024-09-11T07:50:24.184 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.184 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.185 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.185 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.032150+0000", 2024-09-11T07:50:24.185 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0177085020000001 2024-09-11T07:50:24.185 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.185 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.185 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:24.185 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.032292+0000", 2024-09-11T07:50:24.186 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014209100000000001 2024-09-11T07:50:24.186 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.186 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.186 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:50:24.186 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.032432+0000", 2024-09-11T07:50:24.186 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000140619 2024-09-11T07:50:24.186 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.186 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.186 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:24.186 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.038122+0000", 2024-09-11T07:50:24.186 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0056898310000000002 2024-09-11T07:50:24.186 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.187 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.187 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.187 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.049961+0000", 2024-09-11T07:50:24.187 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.011838530999999999 2024-09-11T07:50:24.187 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.187 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.187 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.187 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.050018+0000", 2024-09-11T07:50:24.187 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7024999999999997e-05 2024-09-11T07:50:24.187 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.187 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.187 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:24.187 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.050063+0000", 2024-09-11T07:50:24.188 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5547999999999998e-05 2024-09-11T07:50:24.188 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.188 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.188 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:24.188 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.050094+0000", 2024-09-11T07:50:24.188 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0552999999999997e-05 2024-09-11T07:50:24.188 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.188 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:24.188 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.188 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.188 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.188 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:233682 11.c 11:3fb647af:::benchmark_data_smithi022_188626_object113874:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e468)", 2024-09-11T07:50:24.188 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:49:33.019517+0000", 2024-09-11T07:50:24.189 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 51.081876012000002, 2024-09-11T07:50:24.189 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.030736927, 2024-09-11T07:50:24.189 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:24.189 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:24.189 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:24.189 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:24.189 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:24.189 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 233682 2024-09-11T07:50:24.189 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.189 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:24.189 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.189 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:24.189 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.019517+0000", 2024-09-11T07:50:24.190 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.190 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.190 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.190 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:24.190 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.019517+0000", 2024-09-11T07:50:24.190 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.190 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.190 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.190 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:24.190 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.019515+0000", 2024-09-11T07:50:24.190 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:50:24.190 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.190 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.191 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:24.191 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.019524+0000", 2024-09-11T07:50:24.191 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.9479999999999997e-06 2024-09-11T07:50:24.191 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.191 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.191 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:24.191 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.019541+0000", 2024-09-11T07:50:24.191 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.6973000000000002e-05 2024-09-11T07:50:24.191 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.191 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.191 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:24.191 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.019547+0000", 2024-09-11T07:50:24.191 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.6680000000000004e-06 2024-09-11T07:50:24.191 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.192 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.192 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.192 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.019653+0000", 2024-09-11T07:50:24.192 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000105678 2024-09-11T07:50:24.192 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.192 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.192 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:24.192 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.019661+0000", 2024-09-11T07:50:24.192 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.7330000000000003e-06 2024-09-11T07:50:24.192 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.192 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.192 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.192 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.033931+0000", 2024-09-11T07:50:24.193 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.014269882 2024-09-11T07:50:24.193 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.193 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.193 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:24.193 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.033995+0000", 2024-09-11T07:50:24.193 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4535999999999993e-05 2024-09-11T07:50:24.193 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.193 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.193 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:50:24.193 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.034059+0000", 2024-09-11T07:50:24.193 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3959000000000002e-05 2024-09-11T07:50:24.193 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.194 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.194 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:24.194 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.038129+0000", 2024-09-11T07:50:24.194 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00406968 2024-09-11T07:50:24.194 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.194 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.194 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.194 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.049977+0000", 2024-09-11T07:50:24.194 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.011848455000000001 2024-09-11T07:50:24.194 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.194 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.194 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.194 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.050196+0000", 2024-09-11T07:50:24.194 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00021832500000000001 2024-09-11T07:50:24.195 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.195 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.195 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:24.195 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.050230+0000", 2024-09-11T07:50:24.195 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4700999999999999e-05 2024-09-11T07:50:24.195 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.195 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.195 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:24.195 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.050254+0000", 2024-09-11T07:50:24.195 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3887999999999999e-05 2024-09-11T07:50:24.195 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.195 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:24.195 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.195 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.196 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.196 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:233686 11.c 11:3aac20ee:::benchmark_data_smithi022_188626_object113878:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e468)", 2024-09-11T07:50:24.196 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:49:33.022831+0000", 2024-09-11T07:50:24.196 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 51.078562517000002, 2024-09-11T07:50:24.196 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0276436309999999, 2024-09-11T07:50:24.196 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:24.196 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:24.196 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:24.196 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:24.196 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:24.196 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 233686 2024-09-11T07:50:24.196 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.197 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:24.197 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.197 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:24.197 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.022831+0000", 2024-09-11T07:50:24.197 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.197 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.197 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.197 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:24.197 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.022831+0000", 2024-09-11T07:50:24.197 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.197 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.197 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.197 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:24.198 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.022828+0000", 2024-09-11T07:50:24.198 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:50:24.198 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.198 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.198 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:24.198 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.022837+0000", 2024-09-11T07:50:24.198 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.2930000000000006e-06 2024-09-11T07:50:24.198 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.198 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.198 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:24.198 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.022850+0000", 2024-09-11T07:50:24.198 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2612000000000001e-05 2024-09-11T07:50:24.198 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.199 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.199 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:24.199 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.022856+0000", 2024-09-11T07:50:24.199 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3779999999999998e-06 2024-09-11T07:50:24.199 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.199 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.199 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.199 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.022991+0000", 2024-09-11T07:50:24.199 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013465300000000001 2024-09-11T07:50:24.199 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.199 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.199 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:24.199 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.023008+0000", 2024-09-11T07:50:24.199 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.6878e-05 2024-09-11T07:50:24.200 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.200 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.200 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.200 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.034895+0000", 2024-09-11T07:50:24.200 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0118870719999999 2024-09-11T07:50:24.200 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.200 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.200 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:24.200 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.034961+0000", 2024-09-11T07:50:24.200 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.6215000000000002e-05 2024-09-11T07:50:24.200 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.200 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.200 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:50:24.200 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.035027+0000", 2024-09-11T07:50:24.201 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5309999999999996e-05 2024-09-11T07:50:24.201 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.201 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.201 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:24.201 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.038134+0000", 2024-09-11T07:50:24.201 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.003107121 2024-09-11T07:50:24.201 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.201 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.201 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.201 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.050155+0000", 2024-09-11T07:50:24.201 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.012021651 2024-09-11T07:50:24.201 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.201 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.202 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.202 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.050370+0000", 2024-09-11T07:50:24.202 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000214506 2024-09-11T07:50:24.202 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.202 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.202 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:24.202 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.050454+0000", 2024-09-11T07:50:24.202 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.4383999999999997e-05 2024-09-11T07:50:24.202 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.202 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.202 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:24.202 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.050474+0000", 2024-09-11T07:50:24.202 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0194999999999999e-05 2024-09-11T07:50:24.202 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.203 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:24.203 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.203 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.203 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.203 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:233710 11.c 11:33b1948c:::benchmark_data_smithi022_188626_object113902:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e468)", 2024-09-11T07:50:24.203 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:49:33.040696+0000", 2024-09-11T07:50:24.203 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 51.060697257999998, 2024-09-11T07:50:24.203 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.009930344, 2024-09-11T07:50:24.203 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:24.203 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:24.203 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:24.203 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:24.203 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:24.203 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 233710 2024-09-11T07:50:24.203 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.204 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:24.204 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.204 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:24.204 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.040696+0000", 2024-09-11T07:50:24.204 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.204 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.204 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.204 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:24.204 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.040696+0000", 2024-09-11T07:50:24.204 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.204 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.204 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.204 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:24.204 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.040694+0000", 2024-09-11T07:50:24.204 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:50:24.205 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.205 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.205 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:24.205 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.040702+0000", 2024-09-11T07:50:24.205 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.4540000000000007e-06 2024-09-11T07:50:24.205 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.205 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.205 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:24.205 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.040733+0000", 2024-09-11T07:50:24.205 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.1396000000000002e-05 2024-09-11T07:50:24.205 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.205 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.205 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:24.206 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.040740+0000", 2024-09-11T07:50:24.206 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.8170000000000001e-06 2024-09-11T07:50:24.206 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.206 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.206 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.206 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.040809+0000", 2024-09-11T07:50:24.206 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9054000000000007e-05 2024-09-11T07:50:24.206 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.206 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.206 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:24.206 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.040817+0000", 2024-09-11T07:50:24.206 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.3180000000000001e-06 2024-09-11T07:50:24.206 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.206 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.206 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.207 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.035884+0000", 2024-09-11T07:50:24.207 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.99506739200000005 2024-09-11T07:50:24.207 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.207 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.207 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:24.207 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.035990+0000", 2024-09-11T07:50:24.207 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000106389 2024-09-11T07:50:24.207 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.207 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.207 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:50:24.207 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.036070+0000", 2024-09-11T07:50:24.207 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.0129999999999993e-05 2024-09-11T07:50:24.207 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.207 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.208 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:24.208 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.041338+0000", 2024-09-11T07:50:24.208 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0052679800000000002 2024-09-11T07:50:24.208 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.208 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.208 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.208 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.050539+0000", 2024-09-11T07:50:24.208 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0092004819999999994 2024-09-11T07:50:24.208 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.208 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.208 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.208 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.050583+0000", 2024-09-11T07:50:24.208 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4038000000000001e-05 2024-09-11T07:50:24.209 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.209 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.209 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:24.209 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.050607+0000", 2024-09-11T07:50:24.209 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.4502999999999999e-05 2024-09-11T07:50:24.209 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.209 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.209 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:24.209 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.050626+0000", 2024-09-11T07:50:24.209 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8919e-05 2024-09-11T07:50:24.209 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.209 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:24.209 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.209 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.210 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.210 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:233740 11.c 11:383603f0:::benchmark_data_smithi022_188626_object113932:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e468)", 2024-09-11T07:50:24.210 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:49:33.061661+0000", 2024-09-11T07:50:24.210 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 51.039732045999997, 2024-09-11T07:50:24.210 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.98913375000000003, 2024-09-11T07:50:24.210 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:24.210 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:24.210 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:24.210 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:24.210 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:24.210 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 233740 2024-09-11T07:50:24.210 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.210 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:24.211 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.211 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:24.211 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.061661+0000", 2024-09-11T07:50:24.211 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.211 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.211 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.211 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:24.211 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.061661+0000", 2024-09-11T07:50:24.211 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.211 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.211 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.211 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:24.211 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.061659+0000", 2024-09-11T07:50:24.212 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:50:24.212 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.212 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.212 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:24.212 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.061667+0000", 2024-09-11T07:50:24.212 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3119999999999994e-06 2024-09-11T07:50:24.212 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.212 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.212 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:24.212 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.061682+0000", 2024-09-11T07:50:24.212 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4737e-05 2024-09-11T07:50:24.212 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.212 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.213 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:24.213 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.061689+0000", 2024-09-11T07:50:24.213 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5309999999999998e-06 2024-09-11T07:50:24.213 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.213 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.213 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.213 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.061762+0000", 2024-09-11T07:50:24.213 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.3700000000000002e-05 2024-09-11T07:50:24.213 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.213 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.213 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:24.213 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.061770+0000", 2024-09-11T07:50:24.214 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.328e-06 2024-09-11T07:50:24.214 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.214 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.214 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.214 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.037108+0000", 2024-09-11T07:50:24.214 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.97533852399999998 2024-09-11T07:50:24.214 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.214 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.214 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:24.214 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.037210+0000", 2024-09-11T07:50:24.214 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000101549 2024-09-11T07:50:24.214 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.214 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.215 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:50:24.215 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.037282+0000", 2024-09-11T07:50:24.215 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1988000000000003e-05 2024-09-11T07:50:24.215 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.215 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.215 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:24.215 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.042401+0000", 2024-09-11T07:50:24.215 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0051188529999999996 2024-09-11T07:50:24.215 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.215 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.215 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.215 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.050659+0000", 2024-09-11T07:50:24.215 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0082585939999999993 2024-09-11T07:50:24.215 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.216 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.216 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.216 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.050708+0000", 2024-09-11T07:50:24.216 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8822000000000001e-05 2024-09-11T07:50:24.216 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.216 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.216 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:24.216 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.050759+0000", 2024-09-11T07:50:24.216 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1121000000000001e-05 2024-09-11T07:50:24.216 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.216 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.216 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:24.216 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.050795+0000", 2024-09-11T07:50:24.216 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5939000000000003e-05 2024-09-11T07:50:24.217 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.217 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:24.217 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.217 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.217 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.217 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:233777 11.c 11:3bc420ab:::benchmark_data_smithi022_188626_object113969:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e468)", 2024-09-11T07:50:24.217 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:49:33.125929+0000", 2024-09-11T07:50:24.217 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 50.975464795000001, 2024-09-11T07:50:24.217 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.92521873200000004, 2024-09-11T07:50:24.217 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:24.217 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:24.217 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:24.217 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:24.218 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:24.218 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 233777 2024-09-11T07:50:24.218 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.218 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:24.218 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.218 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:24.218 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.125929+0000", 2024-09-11T07:50:24.218 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.218 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.218 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.218 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:24.218 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.125929+0000", 2024-09-11T07:50:24.218 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.219 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.219 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.219 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:24.219 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.125926+0000", 2024-09-11T07:50:24.219 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:50:24.219 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.219 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.219 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:24.219 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.125935+0000", 2024-09-11T07:50:24.219 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3650000000000004e-06 2024-09-11T07:50:24.219 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.219 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.219 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:24.219 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.125952+0000", 2024-09-11T07:50:24.220 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.698e-05 2024-09-11T07:50:24.220 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.220 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.220 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:24.220 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.125959+0000", 2024-09-11T07:50:24.220 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0430000000000002e-06 2024-09-11T07:50:24.220 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.220 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.220 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.220 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.126264+0000", 2024-09-11T07:50:24.220 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00030550199999999998 2024-09-11T07:50:24.220 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.220 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.220 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:24.220 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.126271+0000", 2024-09-11T07:50:24.221 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1690000000000003e-06 2024-09-11T07:50:24.221 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.221 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.221 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.221 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.038175+0000", 2024-09-11T07:50:24.221 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.91190388899999997 2024-09-11T07:50:24.221 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.221 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.221 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:24.221 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.038246+0000", 2024-09-11T07:50:24.221 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0574999999999994e-05 2024-09-11T07:50:24.221 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.221 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.221 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:50:24.222 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.038315+0000", 2024-09-11T07:50:24.222 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.8831999999999994e-05 2024-09-11T07:50:24.222 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.222 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.222 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:24.222 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.042406+0000", 2024-09-11T07:50:24.222 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0040913319999999996 2024-09-11T07:50:24.222 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.222 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.222 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.222 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.051017+0000", 2024-09-11T07:50:24.222 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0086114710000000008 2024-09-11T07:50:24.222 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.223 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.223 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.223 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.051062+0000", 2024-09-11T07:50:24.223 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4734999999999998e-05 2024-09-11T07:50:24.223 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.223 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.223 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:24.223 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.051111+0000", 2024-09-11T07:50:24.223 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8965000000000003e-05 2024-09-11T07:50:24.223 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.223 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.223 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:24.223 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.051147+0000", 2024-09-11T07:50:24.223 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6198999999999997e-05 2024-09-11T07:50:24.224 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.224 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:24.224 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.224 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.224 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.224 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:233798 11.c 11:3e0375a9:::benchmark_data_smithi022_188626_object113990:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e468)", 2024-09-11T07:50:24.224 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:49:33.143524+0000", 2024-09-11T07:50:24.224 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 50.957869125000002, 2024-09-11T07:50:24.224 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.90776996300000001, 2024-09-11T07:50:24.224 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:24.224 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:24.224 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:24.224 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:24.225 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:24.225 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 233798 2024-09-11T07:50:24.225 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.225 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:24.225 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.225 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:24.225 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.143524+0000", 2024-09-11T07:50:24.225 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.225 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.225 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.225 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:24.225 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.143524+0000", 2024-09-11T07:50:24.225 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.225 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.225 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.226 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:24.226 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.143522+0000", 2024-09-11T07:50:24.226 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:50:24.226 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.226 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.226 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:24.226 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.143530+0000", 2024-09-11T07:50:24.226 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.2689999999999999e-06 2024-09-11T07:50:24.226 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.226 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.226 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:24.226 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.143541+0000", 2024-09-11T07:50:24.226 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1289e-05 2024-09-11T07:50:24.226 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.227 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.227 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:24.227 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.143547+0000", 2024-09-11T07:50:24.227 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6810000000000003e-06 2024-09-11T07:50:24.227 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.227 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.227 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.227 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.143591+0000", 2024-09-11T07:50:24.227 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4339999999999999e-05 2024-09-11T07:50:24.227 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.227 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.227 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:24.227 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.143599+0000", 2024-09-11T07:50:24.227 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.2579999999999998e-06 2024-09-11T07:50:24.227 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.227 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.228 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.228 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.039245+0000", 2024-09-11T07:50:24.228 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.89564585699999999 2024-09-11T07:50:24.228 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.228 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.228 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:24.228 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.039309+0000", 2024-09-11T07:50:24.228 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4556999999999999e-05 2024-09-11T07:50:24.228 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.228 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.228 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:50:24.228 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.039372+0000", 2024-09-11T07:50:24.228 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2433000000000003e-05 2024-09-11T07:50:24.228 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.228 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.229 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:24.229 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.042412+0000", 2024-09-11T07:50:24.229 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0030403959999999999 2024-09-11T07:50:24.229 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.229 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.229 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.229 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.051184+0000", 2024-09-11T07:50:24.229 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0087716749999999996 2024-09-11T07:50:24.229 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.229 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.229 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.229 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.051219+0000", 2024-09-11T07:50:24.229 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5722000000000001e-05 2024-09-11T07:50:24.230 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.230 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.230 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:24.230 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.051261+0000", 2024-09-11T07:50:24.230 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.1941999999999999e-05 2024-09-11T07:50:24.230 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.230 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.230 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:24.230 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.051294+0000", 2024-09-11T07:50:24.230 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2889000000000003e-05 2024-09-11T07:50:24.230 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.230 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:24.230 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.231 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.231 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.231 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:233800 11.c 11:31a96cc2:::benchmark_data_smithi022_188626_object113992:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e468)", 2024-09-11T07:50:24.231 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:49:33.144497+0000", 2024-09-11T07:50:24.231 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 50.956896454999999, 2024-09-11T07:50:24.231 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.90693970300000004, 2024-09-11T07:50:24.231 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:24.231 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:24.231 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:24.231 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:24.231 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:24.231 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 233800 2024-09-11T07:50:24.231 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.231 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:24.231 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.232 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:24.232 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.144497+0000", 2024-09-11T07:50:24.232 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.232 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.232 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.232 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:24.232 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.144497+0000", 2024-09-11T07:50:24.232 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.232 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.232 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.232 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:24.232 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.144495+0000", 2024-09-11T07:50:24.232 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:24.232 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.232 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.233 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:24.233 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.144501+0000", 2024-09-11T07:50:24.233 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1070000000000001e-06 2024-09-11T07:50:24.233 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.233 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.233 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:24.233 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.144516+0000", 2024-09-11T07:50:24.233 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4956e-05 2024-09-11T07:50:24.233 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.233 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.233 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:24.233 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.144522+0000", 2024-09-11T07:50:24.233 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6470000000000003e-06 2024-09-11T07:50:24.234 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.234 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.234 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.234 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.144578+0000", 2024-09-11T07:50:24.234 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6267999999999997e-05 2024-09-11T07:50:24.234 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.234 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.234 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:24.234 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.144584+0000", 2024-09-11T07:50:24.234 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0929999999999999e-06 2024-09-11T07:50:24.234 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.234 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.234 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.234 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.040222+0000", 2024-09-11T07:50:24.234 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.89563800400000004 2024-09-11T07:50:24.235 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.235 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.235 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:24.235 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.040342+0000", 2024-09-11T07:50:24.235 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000119713 2024-09-11T07:50:24.235 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.235 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.235 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:50:24.235 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.040418+0000", 2024-09-11T07:50:24.235 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5656999999999997e-05 2024-09-11T07:50:24.235 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.235 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.235 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:24.235 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.045571+0000", 2024-09-11T07:50:24.236 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0051527750000000001 2024-09-11T07:50:24.236 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.236 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.236 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.236 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.051329+0000", 2024-09-11T07:50:24.236 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0057586290000000004 2024-09-11T07:50:24.236 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.236 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.236 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.236 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.051358+0000", 2024-09-11T07:50:24.236 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.8510999999999999e-05 2024-09-11T07:50:24.236 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.236 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.236 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:24.236 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.051398+0000", 2024-09-11T07:50:24.237 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0068999999999999e-05 2024-09-11T07:50:24.237 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.237 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.237 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:24.237 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.051437+0000", 2024-09-11T07:50:24.237 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8822000000000002e-05 2024-09-11T07:50:24.237 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.237 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:24.237 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.237 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.237 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.237 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:233872 11.c 11:3a5360a3:::benchmark_data_smithi022_188626_object114064:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e468)", 2024-09-11T07:50:24.237 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:49:33.199625+0000", 2024-09-11T07:50:24.237 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 50.901768367999999, 2024-09-11T07:50:24.238 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.85196864100000003, 2024-09-11T07:50:24.238 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:24.238 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:24.238 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:24.238 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:24.238 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:24.238 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 233872 2024-09-11T07:50:24.238 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.238 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:24.238 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.238 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:24.238 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.199625+0000", 2024-09-11T07:50:24.238 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.238 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.238 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.238 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:24.239 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.199625+0000", 2024-09-11T07:50:24.239 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:24.239 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.239 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.239 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:24.239 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.199623+0000", 2024-09-11T07:50:24.239 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:50:24.239 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.239 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.239 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:24.239 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.199630+0000", 2024-09-11T07:50:24.239 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1459999999999999e-06 2024-09-11T07:50:24.239 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.240 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.240 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:24.240 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.199644+0000", 2024-09-11T07:50:24.240 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4323e-05 2024-09-11T07:50:24.240 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.240 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.240 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:24.240 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.199651+0000", 2024-09-11T07:50:24.240 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9179999999999997e-06 2024-09-11T07:50:24.240 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.240 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.240 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.240 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.199753+0000", 2024-09-11T07:50:24.240 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000101236 2024-09-11T07:50:24.240 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.241 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.241 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:24.241 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:33.199770+0000", 2024-09-11T07:50:24.241 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.7671000000000001e-05 2024-09-11T07:50:24.241 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.241 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.241 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:24.241 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.041369+0000", 2024-09-11T07:50:24.241 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.841598438 2024-09-11T07:50:24.241 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.241 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.241 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:24.241 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.041478+0000", 2024-09-11T07:50:24.241 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000109503 2024-09-11T07:50:24.241 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.242 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.242 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:50:24.242 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.041547+0000", 2024-09-11T07:50:24.242 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.8664000000000005e-05 2024-09-11T07:50:24.242 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.242 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.242 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:24.242 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.046597+0000", 2024-09-11T07:50:24.242 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0050505960000000001 2024-09-11T07:50:24.242 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.242 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.242 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.242 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.051476+0000", 2024-09-11T07:50:24.242 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0048785199999999999 2024-09-11T07:50:24.242 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.242 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.243 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:24.243 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.051513+0000", 2024-09-11T07:50:24.243 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6554000000000003e-05 2024-09-11T07:50:24.243 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.243 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.243 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:24.243 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.051558+0000", 2024-09-11T07:50:24.243 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5195999999999999e-05 2024-09-11T07:50:24.243 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:24.243 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:24.243 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:24.243 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:34.051594+0000", 2024-09-11T07:50:24.243 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5846000000000002e-05 2024-09-11T07:50:24.244 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.244 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:24.244 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.244 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:24.244 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:24.244 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:50:24.388 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:50:24 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:50:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:50:24.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:24 smithi022 ceph-mon[29462]: pgmap v1111: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s; 36 B/s, 0 objects/s recovering 2024-09-11T07:50:24.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:24 smithi022 ceph-mon[37330]: pgmap v1111: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s; 36 B/s, 0 objects/s recovering 2024-09-11T07:50:24.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:24 smithi104 ceph-mon[33507]: pgmap v1111: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s; 36 B/s, 0 objects/s recovering 2024-09-11T07:50:25.443 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-11T07:50:25.620 INFO:teuthology.orchestra.run.smithi022.stdout:get_pg_num_history pg_num_history pg_num_history(e469 pg_nums {1={388=11},2={56=32},3={56=3},4={58=32},5={60=32},6={62=32},11={412=16}} deleted_pools ) 2024-09-11T07:50:25.620 INFO:teuthology.orchestra.run.smithi022.stdout:Importing pgid 11.3 2024-09-11T07:50:25.620 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c6e585cf:::benchmark_data_smithi022_188626_object100889:head# 2024-09-11T07:50:25.620 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.621 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c6ed3315:::benchmark_data_smithi022_188626_object107189:head# 2024-09-11T07:50:25.621 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.621 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c6ee46d2:::benchmark_data_smithi022_188626_object107122:head# 2024-09-11T07:50:25.621 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.621 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c6f99e62:::benchmark_data_smithi022_188626_object111416:head# 2024-09-11T07:50:25.621 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.621 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c6fa471f:::benchmark_data_smithi022_188626_object103789:head# 2024-09-11T07:50:25.621 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.621 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c6fa6556:::benchmark_data_smithi022_188626_object96308:head# 2024-09-11T07:50:25.621 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.621 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c6fbe4ef:::benchmark_data_smithi022_188626_object93585:head# 2024-09-11T07:50:25.621 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.622 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c6fce5ce:::benchmark_data_smithi022_188626_object103584:head# 2024-09-11T07:50:25.622 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.622 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c6fdcdb2:::benchmark_data_smithi022_188626_object100807:head# 2024-09-11T07:50:25.622 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.622 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c6ff62e0:::benchmark_data_smithi022_188626_object109952:head# 2024-09-11T07:50:25.622 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.622 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c70063cf:::benchmark_data_smithi022_188626_object115177:head# 2024-09-11T07:50:25.622 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.622 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c701d295:::benchmark_data_smithi022_188626_object106986:head# 2024-09-11T07:50:25.622 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.622 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c705310c:::benchmark_data_smithi022_188626_object103796:head# 2024-09-11T07:50:25.622 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.623 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c70655fa:::benchmark_data_smithi022_188626_object114676:head# 2024-09-11T07:50:25.623 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.623 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c70b8233:::benchmark_data_smithi022_188626_object101352:head# 2024-09-11T07:50:25.623 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.623 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c714fe57:::benchmark_data_smithi022_188626_object100996:head# 2024-09-11T07:50:25.623 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.623 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c715a339:::benchmark_data_smithi022_188626_object109111:head# 2024-09-11T07:50:25.623 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.623 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c718dbc6:::benchmark_data_smithi022_188626_object94807:head# 2024-09-11T07:50:25.623 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.623 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c71b5cfe:::benchmark_data_smithi022_188626_object100054:head# 2024-09-11T07:50:25.623 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.624 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c71b9f61:::benchmark_data_smithi022_188626_object100771:head# 2024-09-11T07:50:25.624 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.624 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c71bb997:::benchmark_data_smithi022_188626_object110050:head# 2024-09-11T07:50:25.624 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.624 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c71c780e:::benchmark_data_smithi022_188626_object101050:head# 2024-09-11T07:50:25.624 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.625 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c71e5606:::benchmark_data_smithi022_188626_object115472:head# 2024-09-11T07:50:25.625 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.625 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c72267b5:::benchmark_data_smithi022_188626_object119346:head# 2024-09-11T07:50:25.625 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.625 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7228c14:::benchmark_data_smithi022_188626_object103308:head# 2024-09-11T07:50:25.625 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.626 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c72488c3:::benchmark_data_smithi022_188626_object112819:head# 2024-09-11T07:50:25.626 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.626 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7259c34:::benchmark_data_smithi022_188626_object99199:head# 2024-09-11T07:50:25.626 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.626 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7266e42:::benchmark_data_smithi022_188626_object114272:head# 2024-09-11T07:50:25.626 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.627 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c72698ab:::benchmark_data_smithi022_188626_object95790:head# 2024-09-11T07:50:25.627 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.627 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c727a822:::benchmark_data_smithi022_188626_object107691:head# 2024-09-11T07:50:25.627 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.627 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c72873d7:::benchmark_data_smithi022_188626_object107005:head# 2024-09-11T07:50:25.627 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.627 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7289d93:::benchmark_data_smithi022_188626_object119471:head# 2024-09-11T07:50:25.628 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.628 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c72d6f00:::benchmark_data_smithi022_188626_object99316:head# 2024-09-11T07:50:25.628 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.628 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c734648b:::benchmark_data_smithi022_188626_object90555:head# 2024-09-11T07:50:25.628 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.628 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c735e868:::benchmark_data_smithi022_188626_object113394:head# 2024-09-11T07:50:25.628 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.629 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7393da0:::benchmark_data_smithi022_188626_object100068:head# 2024-09-11T07:50:25.629 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.629 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c73c50c3:::benchmark_data_smithi022_188626_object108013:head# 2024-09-11T07:50:25.629 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.629 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c73cd79d:::benchmark_data_smithi022_188626_object107108:head# 2024-09-11T07:50:25.629 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.630 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c73df660:::benchmark_data_smithi022_188626_object94857:head# 2024-09-11T07:50:25.630 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.630 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c742fa6e:::benchmark_data_smithi022_188626_object106410:head# 2024-09-11T07:50:25.630 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.630 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c74361d5:::benchmark_data_smithi022_188626_object100491:head# 2024-09-11T07:50:25.630 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.631 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c746b74e:::benchmark_data_smithi022_188626_object117769:head# 2024-09-11T07:50:25.631 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.631 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c746cf8d:::benchmark_data_smithi022_188626_object113061:head# 2024-09-11T07:50:25.631 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.631 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c748609a:::benchmark_data_smithi022_188626_object99723:head# 2024-09-11T07:50:25.631 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.632 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c74ad3f6:::benchmark_data_smithi022_188626_object106198:head# 2024-09-11T07:50:25.632 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.632 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c74ae325:::benchmark_data_smithi022_188626_object108297:head# 2024-09-11T07:50:25.632 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.632 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c74d1940:::benchmark_data_smithi022_188626_object97535:head# 2024-09-11T07:50:25.632 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.632 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7508046:::benchmark_data_smithi022_188626_object111319:head# 2024-09-11T07:50:25.633 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.633 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c751fa2a:::benchmark_data_smithi022_188626_object90126:head# 2024-09-11T07:50:25.633 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.633 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c75341c3:::benchmark_data_smithi022_188626_object95918:head# 2024-09-11T07:50:25.633 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.633 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c754c3c7:::benchmark_data_smithi022_188626_object114388:head# 2024-09-11T07:50:25.633 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.633 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c755df0c:::benchmark_data_smithi022_188626_object99930:head# 2024-09-11T07:50:25.633 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.633 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c757afaf:::benchmark_data_smithi022_188626_object114423:head# 2024-09-11T07:50:25.633 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.633 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c75b6003:::benchmark_data_smithi022_188626_object102022:head# 2024-09-11T07:50:25.634 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.634 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c75de3d0:::benchmark_data_smithi022_188626_object91975:head# 2024-09-11T07:50:25.634 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.634 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c763cc0e:::benchmark_data_smithi022_188626_object100915:head# 2024-09-11T07:50:25.634 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.634 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c769e9ab:::benchmark_data_smithi022_188626_object118500:head# 2024-09-11T07:50:25.634 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.634 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c76b907c:::benchmark_data_smithi022_188626_object101300:head# 2024-09-11T07:50:25.634 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.634 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c76ca706:::benchmark_data_smithi022_188626_object101662:head# 2024-09-11T07:50:25.634 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.634 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c76eb59c:::benchmark_data_smithi022_188626_object89807:head# 2024-09-11T07:50:25.634 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.634 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c770a7a6:::benchmark_data_smithi022_188626_object96096:head# 2024-09-11T07:50:25.635 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.635 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7720b52:::benchmark_data_smithi022_188626_object114776:head# 2024-09-11T07:50:25.635 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.635 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7721b4f:::benchmark_data_smithi022_188626_object95658:head# 2024-09-11T07:50:25.635 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.635 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c774096e:::benchmark_data_smithi022_188626_object95956:head# 2024-09-11T07:50:25.635 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.635 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7744a52:::benchmark_data_smithi022_188626_object111871:head# 2024-09-11T07:50:25.635 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.635 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c77ab2c2:::benchmark_data_smithi022_188626_object92476:head# 2024-09-11T07:50:25.635 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.636 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c77dd1bd:::benchmark_data_smithi022_188626_object94177:head# 2024-09-11T07:50:25.636 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.636 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c77e9d72:::benchmark_data_smithi022_188626_object93506:head# 2024-09-11T07:50:25.636 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.636 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c780a3b2:::benchmark_data_smithi022_188626_object102374:head# 2024-09-11T07:50:25.636 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.636 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7848c59:::benchmark_data_smithi022_188626_object114000:head# 2024-09-11T07:50:25.636 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.636 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c785337c:::benchmark_data_smithi022_188626_object117131:head# 2024-09-11T07:50:25.636 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.636 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7877bef:::benchmark_data_smithi022_188626_object116380:head# 2024-09-11T07:50:25.636 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.636 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c78af598:::benchmark_data_smithi022_188626_object93382:head# 2024-09-11T07:50:25.636 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.636 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c78eab42:::benchmark_data_smithi022_188626_object104887:head# 2024-09-11T07:50:25.637 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.637 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7906b34:::benchmark_data_smithi022_188626_object97077:head# 2024-09-11T07:50:25.637 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.637 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7966ec1:::benchmark_data_smithi022_188626_object90322:head# 2024-09-11T07:50:25.637 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.637 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c798d9aa:::benchmark_data_smithi022_188626_object101991:head# 2024-09-11T07:50:25.637 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.637 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c799b198:::benchmark_data_smithi022_188626_object104835:head# 2024-09-11T07:50:25.637 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.637 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c79eb3ed:::benchmark_data_smithi022_188626_object111157:head# 2024-09-11T07:50:25.637 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.637 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7a0988c:::benchmark_data_smithi022_188626_object113037:head# 2024-09-11T07:50:25.637 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.638 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7a28da2:::benchmark_data_smithi022_188626_object113009:head# 2024-09-11T07:50:25.638 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.638 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7a5c345:::benchmark_data_smithi022_188626_object99476:head# 2024-09-11T07:50:25.638 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.638 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7a73afb:::benchmark_data_smithi022_188626_object92627:head# 2024-09-11T07:50:25.638 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.638 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7ad0559:::benchmark_data_smithi022_188626_object106141:head# 2024-09-11T07:50:25.638 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.638 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7ad8769:::benchmark_data_smithi022_188626_object117673:head# 2024-09-11T07:50:25.638 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.638 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7b2ae22:::benchmark_data_smithi022_188626_object107848:head# 2024-09-11T07:50:25.638 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.638 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7b38b65:::benchmark_data_smithi022_188626_object95385:head# 2024-09-11T07:50:25.638 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.639 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7b453e2:::benchmark_data_smithi022_188626_object110569:head# 2024-09-11T07:50:25.639 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.639 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7b544ee:::benchmark_data_smithi022_188626_object104032:head# 2024-09-11T07:50:25.639 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.639 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7b7c1bd:::benchmark_data_smithi022_188626_object104884:head# 2024-09-11T07:50:25.639 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.639 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7b9d165:::benchmark_data_smithi022_188626_object115864:head# 2024-09-11T07:50:25.639 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.639 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7b9d9b7:::benchmark_data_smithi022_188626_object90647:head# 2024-09-11T07:50:25.639 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.639 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7c0e6ae:::benchmark_data_smithi022_188626_object115420:head# 2024-09-11T07:50:25.639 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.639 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7c1970d:::benchmark_data_smithi022_188626_object117072:head# 2024-09-11T07:50:25.640 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.640 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7c8ad1c:::benchmark_data_smithi022_188626_object107201:head# 2024-09-11T07:50:25.640 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.640 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7ca24a2:::benchmark_data_smithi022_188626_object99729:head# 2024-09-11T07:50:25.640 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.640 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7cc6ed0:::benchmark_data_smithi022_188626_object118019:head# 2024-09-11T07:50:25.640 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.640 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7d212e9:::benchmark_data_smithi022_188626_object110825:head# 2024-09-11T07:50:25.640 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.640 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7d9e793:::benchmark_data_smithi022_188626_object102170:head# 2024-09-11T07:50:25.640 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.641 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7de2681:::benchmark_data_smithi022_188626_object112984:head# 2024-09-11T07:50:25.641 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.641 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7e31285:::benchmark_data_smithi022_188626_object100787:head# 2024-09-11T07:50:25.641 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.641 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7e56005:::benchmark_data_smithi022_188626_object115116:head# 2024-09-11T07:50:25.641 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.641 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7e5c864:::benchmark_data_smithi022_188626_object110331:head# 2024-09-11T07:50:25.641 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.641 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7f01319:::benchmark_data_smithi022_188626_object113893:head# 2024-09-11T07:50:25.641 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.641 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7f58003:::benchmark_data_smithi022_188626_object104466:head# 2024-09-11T07:50:25.641 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.642 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7f62822:::benchmark_data_smithi022_188626_object92327:head# 2024-09-11T07:50:25.642 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.642 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7f7d946:::benchmark_data_smithi022_188626_object110997:head# 2024-09-11T07:50:25.642 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.642 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7f83bd2:::benchmark_data_smithi022_188626_object100221:head# 2024-09-11T07:50:25.642 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.642 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7f8e052:::benchmark_data_smithi022_188626_object101473:head# 2024-09-11T07:50:25.642 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.642 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7f952ec:::benchmark_data_smithi022_188626_object112806:head# 2024-09-11T07:50:25.642 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.642 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7fa2248:::benchmark_data_smithi022_188626_object104723:head# 2024-09-11T07:50:25.642 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.643 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7fb1efd:::benchmark_data_smithi022_188626_object105759:head# 2024-09-11T07:50:25.643 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.643 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7fbced2:::benchmark_data_smithi022_188626_object92144:head# 2024-09-11T07:50:25.643 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.643 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7fd497b:::benchmark_data_smithi022_188626_object94607:head# 2024-09-11T07:50:25.643 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.643 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c7fe638b:::benchmark_data_smithi022_188626_object110061:head# 2024-09-11T07:50:25.643 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.643 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c802ad73:::benchmark_data_smithi022_188626_object109507:head# 2024-09-11T07:50:25.643 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.643 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c803a5ea:::benchmark_data_smithi022_188626_object113533:head# 2024-09-11T07:50:25.644 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.644 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8052860:::benchmark_data_smithi022_188626_object101175:head# 2024-09-11T07:50:25.644 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.644 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c80733f6:::benchmark_data_smithi022_188626_object105410:head# 2024-09-11T07:50:25.644 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.644 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c808041d:::benchmark_data_smithi022_188626_object107259:head# 2024-09-11T07:50:25.644 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.644 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8084916:::benchmark_data_smithi022_188626_object110943:head# 2024-09-11T07:50:25.644 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.644 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c80c64d6:::benchmark_data_smithi022_188626_object108863:head# 2024-09-11T07:50:25.644 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.644 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c80c81af:::benchmark_data_smithi022_188626_object118253:head# 2024-09-11T07:50:25.644 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.645 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c80ca994:::benchmark_data_smithi022_188626_object106474:head# 2024-09-11T07:50:25.645 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.645 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c80d2ea3:::benchmark_data_smithi022_188626_object105214:head# 2024-09-11T07:50:25.645 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.645 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c80ffbbe:::benchmark_data_smithi022_188626_object101958:head# 2024-09-11T07:50:25.645 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.645 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c811c808:::benchmark_data_smithi022_188626_object118549:head# 2024-09-11T07:50:25.645 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.645 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8137658:::benchmark_data_smithi022_188626_object95810:head# 2024-09-11T07:50:25.645 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.645 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8149990:::benchmark_data_smithi022_188626_object107075:head# 2024-09-11T07:50:25.645 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.645 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c817acf8:::benchmark_data_smithi022_188626_object105345:head# 2024-09-11T07:50:25.646 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.646 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c818e369:::benchmark_data_smithi022_188626_object104125:head# 2024-09-11T07:50:25.646 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.646 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c81f812d:::benchmark_data_smithi022_188626_object98297:head# 2024-09-11T07:50:25.646 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.646 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8248d4f:::benchmark_data_smithi022_188626_object117231:head# 2024-09-11T07:50:25.646 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.646 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c828abce:::benchmark_data_smithi022_188626_object101475:head# 2024-09-11T07:50:25.646 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.646 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c828c56a:::benchmark_data_smithi022_188626_object117395:head# 2024-09-11T07:50:25.646 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.646 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8290068:::benchmark_data_smithi022_188626_object113082:head# 2024-09-11T07:50:25.646 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.646 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c82a66b2:::benchmark_data_smithi022_188626_object117980:head# 2024-09-11T07:50:25.647 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.647 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c82cfaeb:::benchmark_data_smithi022_188626_object105471:head# 2024-09-11T07:50:25.647 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.647 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c82df47e:::benchmark_data_smithi022_188626_object114263:head# 2024-09-11T07:50:25.647 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.647 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8303010:::benchmark_data_smithi022_188626_object96980:head# 2024-09-11T07:50:25.647 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.647 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8334bf8:::benchmark_data_smithi022_188626_object106471:head# 2024-09-11T07:50:25.647 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.647 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c83d0e16:::benchmark_data_smithi022_188626_object104403:head# 2024-09-11T07:50:25.647 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.647 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c83f5dc8:::benchmark_data_smithi022_188626_object108554:head# 2024-09-11T07:50:25.647 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.647 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c845162b:::benchmark_data_smithi022_188626_object118759:head# 2024-09-11T07:50:25.648 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.648 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c84574b4:::benchmark_data_smithi022_188626_object94699:head# 2024-09-11T07:50:25.648 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.648 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c847a17f:::benchmark_data_smithi022_188626_object89374:head# 2024-09-11T07:50:25.648 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.648 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c848c1a0:::benchmark_data_smithi022_188626_object100914:head# 2024-09-11T07:50:25.648 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.648 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c84b326b:::benchmark_data_smithi022_188626_object113434:head# 2024-09-11T07:50:25.648 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.648 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c84da272:::benchmark_data_smithi022_188626_object102862:head# 2024-09-11T07:50:25.648 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.648 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c84de2c3:::benchmark_data_smithi022_188626_object115494:head# 2024-09-11T07:50:25.648 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.648 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8536fb6:::benchmark_data_smithi022_188626_object108551:head# 2024-09-11T07:50:25.649 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.649 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8561c26:::benchmark_data_smithi022_188626_object103713:head# 2024-09-11T07:50:25.649 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.649 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8564826:::benchmark_data_smithi022_188626_object91232:head# 2024-09-11T07:50:25.649 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.649 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8589949:::benchmark_data_smithi022_188626_object92269:head# 2024-09-11T07:50:25.649 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.649 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c85d4110:::benchmark_data_smithi022_188626_object110731:head# 2024-09-11T07:50:25.649 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.649 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c86292ba:::benchmark_data_smithi022_188626_object98300:head# 2024-09-11T07:50:25.649 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.649 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8646bc3:::benchmark_data_smithi022_188626_object112633:head# 2024-09-11T07:50:25.649 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.650 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8654e59:::benchmark_data_smithi022_188626_object119748:head# 2024-09-11T07:50:25.650 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.650 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c866c9aa:::benchmark_data_smithi022_188626_object119493:head# 2024-09-11T07:50:25.650 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.650 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c867aaf4:::benchmark_data_smithi022_188626_object102256:head# 2024-09-11T07:50:25.650 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.650 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c86a60cf:::benchmark_data_smithi022_188626_object117595:head# 2024-09-11T07:50:25.650 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.650 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c86ae905:::benchmark_data_smithi022_188626_object94463:head# 2024-09-11T07:50:25.650 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.650 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c86dbc36:::benchmark_data_smithi022_188626_object101158:head# 2024-09-11T07:50:25.650 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.650 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8700b47:::benchmark_data_smithi022_188626_object101613:head# 2024-09-11T07:50:25.650 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.651 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c87325ba:::benchmark_data_smithi022_188626_object111051:head# 2024-09-11T07:50:25.651 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.651 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c874581d:::benchmark_data_smithi022_188626_object92630:head# 2024-09-11T07:50:25.651 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.651 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c875554f:::benchmark_data_smithi022_188626_object90369:head# 2024-09-11T07:50:25.651 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.651 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c877cea2:::benchmark_data_smithi022_188626_object106124:head# 2024-09-11T07:50:25.651 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.651 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c87905fc:::benchmark_data_smithi022_188626_object114979:head# 2024-09-11T07:50:25.651 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.651 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c87f1a6a:::benchmark_data_smithi022_188626_object116983:head# 2024-09-11T07:50:25.651 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.651 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8813bc3:::benchmark_data_smithi022_188626_object95893:head# 2024-09-11T07:50:25.651 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.652 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c883f059:::benchmark_data_smithi022_188626_object101687:head# 2024-09-11T07:50:25.652 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.652 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8841cff:::benchmark_data_smithi022_188626_object99708:head# 2024-09-11T07:50:25.652 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.652 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c887ee11:::benchmark_data_smithi022_188626_object92972:head# 2024-09-11T07:50:25.652 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.652 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c88ed973:::benchmark_data_smithi022_188626_object117525:head# 2024-09-11T07:50:25.652 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.652 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8906acc:::benchmark_data_smithi022_188626_object94185:head# 2024-09-11T07:50:25.652 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.652 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8928a2d:::benchmark_data_smithi022_188626_object96879:head# 2024-09-11T07:50:25.652 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.652 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c89323d6:::benchmark_data_smithi022_188626_object99233:head# 2024-09-11T07:50:25.653 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.653 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c893a6c2:::benchmark_data_smithi022_188626_object89953:head# 2024-09-11T07:50:25.653 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.653 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8984d55:::benchmark_data_smithi022_188626_object99293:head# 2024-09-11T07:50:25.653 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.653 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c89a9b18:::benchmark_data_smithi022_188626_object112738:head# 2024-09-11T07:50:25.653 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.653 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c89d1842:::benchmark_data_smithi022_188626_object95545:head# 2024-09-11T07:50:25.653 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.654 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c89e7168:::benchmark_data_smithi022_188626_object93290:head# 2024-09-11T07:50:25.654 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.654 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c89e8bdd:::benchmark_data_smithi022_188626_object113545:head# 2024-09-11T07:50:25.654 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.654 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8a1d41e:::benchmark_data_smithi022_188626_object95853:head# 2024-09-11T07:50:25.654 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.654 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8a1e524:::benchmark_data_smithi022_188626_object107052:head# 2024-09-11T07:50:25.654 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.654 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8a39144:::benchmark_data_smithi022_188626_object88980:head# 2024-09-11T07:50:25.654 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.654 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8a4043d:::benchmark_data_smithi022_188626_object110899:head# 2024-09-11T07:50:25.654 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.654 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8a4f0f8:::benchmark_data_smithi022_188626_object116207:head# 2024-09-11T07:50:25.655 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.655 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8a50d13:::benchmark_data_smithi022_188626_object116644:head# 2024-09-11T07:50:25.655 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.655 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8aa5b6e:::benchmark_data_smithi022_188626_object119400:head# 2024-09-11T07:50:25.655 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.655 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8aa6472:::benchmark_data_smithi022_188626_object117290:head# 2024-09-11T07:50:25.655 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.655 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8acaa80:::benchmark_data_smithi022_188626_object110022:head# 2024-09-11T07:50:25.655 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.655 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8b16402:::benchmark_data_smithi022_188626_object111231:head# 2024-09-11T07:50:25.655 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.655 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8b4452c:::benchmark_data_smithi022_188626_object100719:head# 2024-09-11T07:50:25.655 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.655 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8b743a3:::benchmark_data_smithi022_188626_object101745:head# 2024-09-11T07:50:25.656 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.656 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8b8e0b0:::benchmark_data_smithi022_188626_object107354:head# 2024-09-11T07:50:25.656 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.656 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8b9b566:::benchmark_data_smithi022_188626_object104865:head# 2024-09-11T07:50:25.656 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.656 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8bb82e5:::benchmark_data_smithi022_188626_object96361:head# 2024-09-11T07:50:25.656 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.656 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8bc4751:::benchmark_data_smithi022_188626_object118109:head# 2024-09-11T07:50:25.656 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.656 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8bddf59:::benchmark_data_smithi022_188626_object118187:head# 2024-09-11T07:50:25.656 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.656 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8c30a0f:::benchmark_data_smithi022_188626_object92684:head# 2024-09-11T07:50:25.656 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.657 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8c44808:::benchmark_data_smithi022_188626_object119337:head# 2024-09-11T07:50:25.657 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.657 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8d02655:::benchmark_data_smithi022_188626_object114978:head# 2024-09-11T07:50:25.657 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.657 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8d194e3:::benchmark_data_smithi022_188626_object92981:head# 2024-09-11T07:50:25.657 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.657 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8d5991c:::benchmark_data_smithi022_188626_object103764:head# 2024-09-11T07:50:25.657 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.657 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8d66c07:::benchmark_data_smithi022_188626_object107476:head# 2024-09-11T07:50:25.657 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.657 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8d772b1:::benchmark_data_smithi022_188626_object108168:head# 2024-09-11T07:50:25.657 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.657 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8debf68:::benchmark_data_smithi022_188626_object102243:head# 2024-09-11T07:50:25.657 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.658 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8e0007e:::benchmark_data_smithi022_188626_object101787:head# 2024-09-11T07:50:25.658 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.658 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8e2bcc4:::benchmark_data_smithi022_188626_object112662:head# 2024-09-11T07:50:25.658 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.658 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8e301ed:::benchmark_data_smithi022_188626_object115121:head# 2024-09-11T07:50:25.658 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.658 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8e386d0:::benchmark_data_smithi022_188626_object100268:head# 2024-09-11T07:50:25.658 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.658 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8e4deb3:::benchmark_data_smithi022_188626_object107319:head# 2024-09-11T07:50:25.658 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.658 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8e9e944:::benchmark_data_smithi022_188626_object105529:head# 2024-09-11T07:50:25.658 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.658 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8e9ec14:::benchmark_data_smithi022_188626_object112208:head# 2024-09-11T07:50:25.658 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.659 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8ea8529:::benchmark_data_smithi022_188626_object102675:head# 2024-09-11T07:50:25.659 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.659 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8ecf365:::benchmark_data_smithi022_188626_object90019:head# 2024-09-11T07:50:25.659 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.659 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8ed9080:::benchmark_data_smithi022_188626_object103536:head# 2024-09-11T07:50:25.659 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.659 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8f062b7:::benchmark_data_smithi022_188626_object107013:head# 2024-09-11T07:50:25.659 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.659 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8f14a2e:::benchmark_data_smithi022_188626_object116894:head# 2024-09-11T07:50:25.659 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.659 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8f8951b:::benchmark_data_smithi022_188626_object113926:head# 2024-09-11T07:50:25.659 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.659 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8f8bffc:::benchmark_data_smithi022_188626_object94800:head# 2024-09-11T07:50:25.660 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.660 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c8faada1:::benchmark_data_smithi022_188626_object117817:head# 2024-09-11T07:50:25.660 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.660 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9002398:::benchmark_data_smithi022_188626_object92751:head# 2024-09-11T07:50:25.660 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.660 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9008837:::benchmark_data_smithi022_188626_object89074:head# 2024-09-11T07:50:25.660 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.660 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c90626d7:::benchmark_data_smithi022_188626_object101580:head# 2024-09-11T07:50:25.660 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.660 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9080419:::benchmark_data_smithi022_188626_object118369:head# 2024-09-11T07:50:25.660 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.660 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9094d67:::benchmark_data_smithi022_188626_object107465:head# 2024-09-11T07:50:25.660 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.661 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c90bd4f5:::benchmark_data_smithi022_188626_object96550:head# 2024-09-11T07:50:25.661 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.661 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c915797d:::benchmark_data_smithi022_188626_object94799:head# 2024-09-11T07:50:25.661 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.661 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c916192f:::benchmark_data_smithi022_188626_object111336:head# 2024-09-11T07:50:25.661 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.661 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9169a47:::benchmark_data_smithi022_188626_object97406:head# 2024-09-11T07:50:25.661 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.661 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c917cb3a:::benchmark_data_smithi022_188626_object109096:head# 2024-09-11T07:50:25.661 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.661 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c91a7394:::benchmark_data_smithi022_188626_object90438:head# 2024-09-11T07:50:25.661 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.661 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c91f5132:::benchmark_data_smithi022_188626_object118070:head# 2024-09-11T07:50:25.662 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.662 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c922d8c9:::benchmark_data_smithi022_188626_object114325:head# 2024-09-11T07:50:25.662 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.662 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c924b093:::benchmark_data_smithi022_188626_object116190:head# 2024-09-11T07:50:25.662 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.662 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c925ead5:::benchmark_data_smithi022_188626_object89943:head# 2024-09-11T07:50:25.662 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.662 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9266746:::benchmark_data_smithi022_188626_object98121:head# 2024-09-11T07:50:25.662 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.662 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c92d5f30:::benchmark_data_smithi022_188626_object95206:head# 2024-09-11T07:50:25.662 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.662 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c934c4fa:::benchmark_data_smithi022_188626_object92295:head# 2024-09-11T07:50:25.662 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.662 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9384379:::benchmark_data_smithi022_188626_object117867:head# 2024-09-11T07:50:25.663 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.663 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c93ccfe4:::benchmark_data_smithi022_188626_object105824:head# 2024-09-11T07:50:25.663 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.663 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c93db533:::benchmark_data_smithi022_188626_object111037:head# 2024-09-11T07:50:25.663 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.663 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c93ff77d:::benchmark_data_smithi022_188626_object117663:head# 2024-09-11T07:50:25.663 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.663 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c941c1a8:::benchmark_data_smithi022_188626_object101002:head# 2024-09-11T07:50:25.663 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.663 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9431344:::benchmark_data_smithi022_188626_object117955:head# 2024-09-11T07:50:25.663 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.663 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c94831cc:::benchmark_data_smithi022_188626_object91392:head# 2024-09-11T07:50:25.664 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.664 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c949ecfc:::benchmark_data_smithi022_188626_object106609:head# 2024-09-11T07:50:25.664 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.664 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c94ac7ab:::benchmark_data_smithi022_188626_object115367:head# 2024-09-11T07:50:25.664 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.664 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c94d1c17:::benchmark_data_smithi022_188626_object103426:head# 2024-09-11T07:50:25.664 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.664 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c94eac16:::benchmark_data_smithi022_188626_object119011:head# 2024-09-11T07:50:25.664 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.664 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c94f58de:::benchmark_data_smithi022_188626_object91142:head# 2024-09-11T07:50:25.664 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.664 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c95026bd:::benchmark_data_smithi022_188626_object112176:head# 2024-09-11T07:50:25.664 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.665 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c951784c:::benchmark_data_smithi022_188626_object94473:head# 2024-09-11T07:50:25.665 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.665 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c95a448a:::benchmark_data_smithi022_188626_object99076:head# 2024-09-11T07:50:25.665 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.665 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c95a81ab:::benchmark_data_smithi022_188626_object89617:head# 2024-09-11T07:50:25.665 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.665 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c95b1bd4:::benchmark_data_smithi022_188626_object108272:head# 2024-09-11T07:50:25.665 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.665 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c95f885f:::benchmark_data_smithi022_188626_object93075:head# 2024-09-11T07:50:25.665 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.665 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c961302f:::benchmark_data_smithi022_188626_object107270:head# 2024-09-11T07:50:25.666 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.666 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c963abe7:::benchmark_data_smithi022_188626_object93288:head# 2024-09-11T07:50:25.666 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.666 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9659ada:::benchmark_data_smithi022_188626_object101734:head# 2024-09-11T07:50:25.666 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.666 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c965c8dc:::benchmark_data_smithi022_188626_object91233:head# 2024-09-11T07:50:25.666 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.666 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c965df3d:::benchmark_data_smithi022_188626_object106766:head# 2024-09-11T07:50:25.666 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.666 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c966c7aa:::benchmark_data_smithi022_188626_object89779:head# 2024-09-11T07:50:25.667 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.667 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c96735c3:::benchmark_data_smithi022_188626_object95138:head# 2024-09-11T07:50:25.667 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.667 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c970f576:::benchmark_data_smithi022_188626_object89092:head# 2024-09-11T07:50:25.667 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.667 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c972420b:::benchmark_data_smithi022_188626_object90075:head# 2024-09-11T07:50:25.667 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.667 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c972cc1c:::benchmark_data_smithi022_188626_object112363:head# 2024-09-11T07:50:25.667 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.667 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c97ee370:::benchmark_data_smithi022_188626_object102785:head# 2024-09-11T07:50:25.667 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.667 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c983fe16:::benchmark_data_smithi022_188626_object94230:head# 2024-09-11T07:50:25.667 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.668 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c987531b:::benchmark_data_smithi022_188626_object111766:head# 2024-09-11T07:50:25.668 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.668 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9876891:::benchmark_data_smithi022_188626_object104009:head# 2024-09-11T07:50:25.668 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.668 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9883449:::benchmark_data_smithi022_188626_object117890:head# 2024-09-11T07:50:25.668 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.668 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c98bbb50:::benchmark_data_smithi022_188626_object111697:head# 2024-09-11T07:50:25.668 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.668 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c98fa529:::benchmark_data_smithi022_188626_object116763:head# 2024-09-11T07:50:25.668 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.668 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9907b80:::benchmark_data_smithi022_188626_object118113:head# 2024-09-11T07:50:25.668 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.668 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9911809:::benchmark_data_smithi022_188626_object119341:head# 2024-09-11T07:50:25.668 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.669 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c994680f:::benchmark_data_smithi022_188626_object104580:head# 2024-09-11T07:50:25.669 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.669 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c99614a9:::benchmark_data_smithi022_188626_object94823:head# 2024-09-11T07:50:25.669 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.669 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c99903d0:::benchmark_data_smithi022_188626_object116965:head# 2024-09-11T07:50:25.669 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.669 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c99f4292:::benchmark_data_smithi022_188626_object105919:head# 2024-09-11T07:50:25.669 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.669 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9a0e882:::benchmark_data_smithi022_188626_object103307:head# 2024-09-11T07:50:25.669 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.669 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9a1580f:::benchmark_data_smithi022_188626_object118956:head# 2024-09-11T07:50:25.669 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.669 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9a26d24:::benchmark_data_smithi022_188626_object95835:head# 2024-09-11T07:50:25.669 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.670 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9a36dee:::benchmark_data_smithi022_188626_object105224:head# 2024-09-11T07:50:25.670 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.670 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9a976b4:::benchmark_data_smithi022_188626_object109737:head# 2024-09-11T07:50:25.670 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.670 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9ab74fd:::benchmark_data_smithi022_188626_object99247:head# 2024-09-11T07:50:25.670 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.670 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9ad6662:::benchmark_data_smithi022_188626_object110224:head# 2024-09-11T07:50:25.670 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.670 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9b1fa83:::benchmark_data_smithi022_188626_object104179:head# 2024-09-11T07:50:25.670 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.670 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9b781b5:::benchmark_data_smithi022_188626_object110033:head# 2024-09-11T07:50:25.670 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.670 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9b922bc:::benchmark_data_smithi022_188626_object113316:head# 2024-09-11T07:50:25.670 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.671 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9b9dbfb:::benchmark_data_smithi022_188626_object113609:head# 2024-09-11T07:50:25.671 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.671 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9c27de5:::benchmark_data_smithi022_188626_object98775:head# 2024-09-11T07:50:25.671 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.671 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9c330b1:::benchmark_data_smithi022_188626_object92970:head# 2024-09-11T07:50:25.671 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.671 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9c4beda:::benchmark_data_smithi022_188626_object103917:head# 2024-09-11T07:50:25.671 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.671 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9c9123f:::benchmark_data_smithi022_188626_object113033:head# 2024-09-11T07:50:25.671 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.671 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9cac8c3:::benchmark_data_smithi022_188626_object114764:head# 2024-09-11T07:50:25.671 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.671 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9cb8e16:::benchmark_data_smithi022_188626_object112817:head# 2024-09-11T07:50:25.671 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.671 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9cba5e8:::benchmark_data_smithi022_188626_object99454:head# 2024-09-11T07:50:25.672 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.672 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9cef085:::benchmark_data_smithi022_188626_object110837:head# 2024-09-11T07:50:25.672 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.672 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9cf99d2:::benchmark_data_smithi022_188626_object97360:head# 2024-09-11T07:50:25.672 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.672 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9d2a0ff:::benchmark_data_smithi022_188626_object107077:head# 2024-09-11T07:50:25.672 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.672 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9d2e9a1:::benchmark_data_smithi022_188626_object100105:head# 2024-09-11T07:50:25.672 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.672 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9d305b2:::benchmark_data_smithi022_188626_object118410:head# 2024-09-11T07:50:25.672 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.672 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9d3cc32:::benchmark_data_smithi022_188626_object112987:head# 2024-09-11T07:50:25.672 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.673 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9d4bbb3:::benchmark_data_smithi022_188626_object96200:head# 2024-09-11T07:50:25.673 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.673 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9d53a91:::benchmark_data_smithi022_188626_object118975:head# 2024-09-11T07:50:25.673 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.673 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9d5c2aa:::benchmark_data_smithi022_188626_object107771:head# 2024-09-11T07:50:25.673 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.673 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9d6e44f:::benchmark_data_smithi022_188626_object112065:head# 2024-09-11T07:50:25.673 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.673 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9d76e7a:::benchmark_data_smithi022_188626_object99395:head# 2024-09-11T07:50:25.673 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.673 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9d7eb08:::benchmark_data_smithi022_188626_object89482:head# 2024-09-11T07:50:25.674 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.674 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9d9e358:::benchmark_data_smithi022_188626_object91916:head# 2024-09-11T07:50:25.674 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.674 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9df02b8:::benchmark_data_smithi022_188626_object119215:head# 2024-09-11T07:50:25.674 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.674 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9df6ce3:::benchmark_data_smithi022_188626_object111053:head# 2024-09-11T07:50:25.674 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.674 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9eb8e2e:::benchmark_data_smithi022_188626_object100616:head# 2024-09-11T07:50:25.674 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.675 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9ec3a5c:::benchmark_data_smithi022_188626_object118915:head# 2024-09-11T07:50:25.675 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.675 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9ec9b7a:::benchmark_data_smithi022_188626_object91838:head# 2024-09-11T07:50:25.675 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.675 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9ee324a:::benchmark_data_smithi022_188626_object105062:head# 2024-09-11T07:50:25.675 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.675 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9f08015:::benchmark_data_smithi022_188626_object106156:head# 2024-09-11T07:50:25.675 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.675 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9f77bb6:::benchmark_data_smithi022_188626_object118808:head# 2024-09-11T07:50:25.675 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.675 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:c9f8ab26:::benchmark_data_smithi022_188626_object106859:head# 2024-09-11T07:50:25.675 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.675 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca00b480:::benchmark_data_smithi022_188626_object106562:head# 2024-09-11T07:50:25.675 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.676 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca01e022:::benchmark_data_smithi022_188626_object98497:head# 2024-09-11T07:50:25.676 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.676 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca034c03:::benchmark_data_smithi022_188626_object106123:head# 2024-09-11T07:50:25.676 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.676 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca046803:::benchmark_data_smithi022_188626_object100615:head# 2024-09-11T07:50:25.676 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.676 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca07e582:::benchmark_data_smithi022_188626_object91268:head# 2024-09-11T07:50:25.676 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.676 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca0b9375:::benchmark_data_smithi022_188626_object115135:head# 2024-09-11T07:50:25.676 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.676 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca0bea72:::benchmark_data_smithi022_188626_object105767:head# 2024-09-11T07:50:25.676 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.676 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca1005c5:::benchmark_data_smithi022_188626_object94405:head# 2024-09-11T07:50:25.677 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.677 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca162ef0:::benchmark_data_smithi022_188626_object114528:head# 2024-09-11T07:50:25.677 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.677 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca171841:::benchmark_data_smithi022_188626_object89894:head# 2024-09-11T07:50:25.677 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.677 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca1a757b:::benchmark_data_smithi022_188626_object117800:head# 2024-09-11T07:50:25.677 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.677 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca1be4a4:::benchmark_data_smithi022_188626_object109387:head# 2024-09-11T07:50:25.677 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.677 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca1d3693:::benchmark_data_smithi022_188626_object96110:head# 2024-09-11T07:50:25.677 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.677 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca1d50a0:::benchmark_data_smithi022_188626_object98478:head# 2024-09-11T07:50:25.677 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.678 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca1fdd5b:::benchmark_data_smithi022_188626_object96495:head# 2024-09-11T07:50:25.678 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.678 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca206ce2:::benchmark_data_smithi022_188626_object95234:head# 2024-09-11T07:50:25.678 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.678 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca254576:::benchmark_data_smithi022_188626_object115852:head# 2024-09-11T07:50:25.678 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.678 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca25e1f8:::benchmark_data_smithi022_188626_object101898:head# 2024-09-11T07:50:25.678 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.678 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca2940f1:::benchmark_data_smithi022_188626_object107463:head# 2024-09-11T07:50:25.678 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.678 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca29de74:::benchmark_data_smithi022_188626_object94900:head# 2024-09-11T07:50:25.678 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.678 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca2b7206:::benchmark_data_smithi022_188626_object97878:head# 2024-09-11T07:50:25.679 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.679 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca2f235c:::benchmark_data_smithi022_188626_object92272:head# 2024-09-11T07:50:25.679 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.679 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca305e68:::benchmark_data_smithi022_188626_object100175:head# 2024-09-11T07:50:25.679 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.679 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca33b317:::benchmark_data_smithi022_188626_object115460:head# 2024-09-11T07:50:25.679 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.679 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca342de6:::benchmark_data_smithi022_188626_object101701:head# 2024-09-11T07:50:25.679 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.679 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca36cc71:::benchmark_data_smithi022_188626_object110989:head# 2024-09-11T07:50:25.679 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.679 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca39510e:::benchmark_data_smithi022_188626_object104698:head# 2024-09-11T07:50:25.679 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.679 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca3a29d8:::benchmark_data_smithi022_188626_object104331:head# 2024-09-11T07:50:25.680 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.680 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca3b8c76:::benchmark_data_smithi022_188626_object101523:head# 2024-09-11T07:50:25.680 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.680 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca420902:::benchmark_data_smithi022_188626_object96390:head# 2024-09-11T07:50:25.680 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.680 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca4d4834:::benchmark_data_smithi022_188626_object90433:head# 2024-09-11T07:50:25.680 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.680 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca4ec4ae:::benchmark_data_smithi022_188626_object112580:head# 2024-09-11T07:50:25.680 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.680 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca4fadd7:::benchmark_data_smithi022_188626_object113118:head# 2024-09-11T07:50:25.680 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.680 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca542884:::benchmark_data_smithi022_188626_object95134:head# 2024-09-11T07:50:25.680 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.681 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca548356:::benchmark_data_smithi022_188626_object113078:head# 2024-09-11T07:50:25.681 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.681 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca563e5c:::benchmark_data_smithi022_188626_object95429:head# 2024-09-11T07:50:25.681 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.681 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca5a5d1f:::benchmark_data_smithi022_188626_object97488:head# 2024-09-11T07:50:25.681 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.681 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca5ce47f:::benchmark_data_smithi022_188626_object118362:head# 2024-09-11T07:50:25.681 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.681 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca5d99be:::benchmark_data_smithi022_188626_object106431:head# 2024-09-11T07:50:25.681 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.681 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca5df4c1:::benchmark_data_smithi022_188626_object94197:head# 2024-09-11T07:50:25.681 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.682 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca5e003a:::benchmark_data_smithi022_188626_object90511:head# 2024-09-11T07:50:25.682 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.682 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca608123:::benchmark_data_smithi022_188626_object101451:head# 2024-09-11T07:50:25.682 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.682 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca61b672:::benchmark_data_smithi022_188626_object114684:head# 2024-09-11T07:50:25.682 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.682 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca61eaba:::benchmark_data_smithi022_188626_object107726:head# 2024-09-11T07:50:25.682 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.682 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca634796:::benchmark_data_smithi022_188626_object95250:head# 2024-09-11T07:50:25.682 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.682 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca63815a:::benchmark_data_smithi022_188626_object110737:head# 2024-09-11T07:50:25.682 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.682 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca6687e6:::benchmark_data_smithi022_188626_object115342:head# 2024-09-11T07:50:25.683 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.683 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca669bce:::benchmark_data_smithi022_188626_object90628:head# 2024-09-11T07:50:25.683 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.683 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca669fb4:::benchmark_data_smithi022_188626_object98703:head# 2024-09-11T07:50:25.683 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.683 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca67850a:::benchmark_data_smithi022_188626_object98133:head# 2024-09-11T07:50:25.683 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.683 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca6b1db7:::benchmark_data_smithi022_188626_object94505:head# 2024-09-11T07:50:25.683 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.683 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca6c31cf:::benchmark_data_smithi022_188626_object115574:head# 2024-09-11T07:50:25.683 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.683 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca70e34e:::benchmark_data_smithi022_188626_object107432:head# 2024-09-11T07:50:25.684 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.684 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca7387dd:::benchmark_data_smithi022_188626_object92457:head# 2024-09-11T07:50:25.684 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.684 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca7417b6:::benchmark_data_smithi022_188626_object105339:head# 2024-09-11T07:50:25.684 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.684 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca758863:::benchmark_data_smithi022_188626_object103473:head# 2024-09-11T07:50:25.684 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.684 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca76021d:::benchmark_data_smithi022_188626_object105987:head# 2024-09-11T07:50:25.684 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.684 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca7ad43e:::benchmark_data_smithi022_188626_object108313:head# 2024-09-11T07:50:25.684 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.685 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca7c97f7:::benchmark_data_smithi022_188626_object98955:head# 2024-09-11T07:50:25.685 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.685 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca7d8e85:::benchmark_data_smithi022_188626_object100022:head# 2024-09-11T07:50:25.685 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.685 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca7e8992:::benchmark_data_smithi022_188626_object89483:head# 2024-09-11T07:50:25.685 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.685 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca7ed9ff:::benchmark_data_smithi022_188626_object106890:head# 2024-09-11T07:50:25.685 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.685 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca81f981:::benchmark_data_smithi022_188626_object104490:head# 2024-09-11T07:50:25.685 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.685 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca82e41c:::benchmark_data_smithi022_188626_object95053:head# 2024-09-11T07:50:25.685 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.686 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca85e9e2:::benchmark_data_smithi022_188626_object93485:head# 2024-09-11T07:50:25.686 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.686 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca86a73a:::benchmark_data_smithi022_188626_object101554:head# 2024-09-11T07:50:25.686 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.686 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca8f6787:::benchmark_data_smithi022_188626_object114626:head# 2024-09-11T07:50:25.686 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.686 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca9b9215:::benchmark_data_smithi022_188626_object91301:head# 2024-09-11T07:50:25.686 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.686 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ca9f17cb:::benchmark_data_smithi022_188626_object117752:head# 2024-09-11T07:50:25.686 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.687 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:caa283e1:::benchmark_data_smithi022_188626_object94922:head# 2024-09-11T07:50:25.687 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.687 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:caa2e53f:::benchmark_data_smithi022_188626_object110872:head# 2024-09-11T07:50:25.687 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.687 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:caa4954f:::benchmark_data_smithi022_188626_object99313:head# 2024-09-11T07:50:25.687 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.687 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:caa7361f:::benchmark_data_smithi022_188626_object90462:head# 2024-09-11T07:50:25.687 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.687 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:caa9ae76:::benchmark_data_smithi022_188626_object115296:head# 2024-09-11T07:50:25.687 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.687 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:caa9e9a0:::benchmark_data_smithi022_188626_object98273:head# 2024-09-11T07:50:25.687 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.687 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:caaa6527:::benchmark_data_smithi022_188626_object119416:head# 2024-09-11T07:50:25.688 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.688 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:caaa8497:::benchmark_data_smithi022_188626_object97925:head# 2024-09-11T07:50:25.688 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.688 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:caabd119:::benchmark_data_smithi022_188626_object109030:head# 2024-09-11T07:50:25.688 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.688 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cab0657e:::benchmark_data_smithi022_188626_object118000:head# 2024-09-11T07:50:25.688 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.688 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cab31c05:::benchmark_data_smithi022_188626_object95631:head# 2024-09-11T07:50:25.688 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.688 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cab96ef1:::benchmark_data_smithi022_188626_object118496:head# 2024-09-11T07:50:25.688 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.688 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:caba4148:::benchmark_data_smithi022_188626_object97529:head# 2024-09-11T07:50:25.689 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.689 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cabacf7a:::benchmark_data_smithi022_188626_object103830:head# 2024-09-11T07:50:25.689 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.689 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cabf697d:::benchmark_data_smithi022_188626_object97277:head# 2024-09-11T07:50:25.689 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.689 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cac01146:::benchmark_data_smithi022_188626_object107869:head# 2024-09-11T07:50:25.689 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.689 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cac19641:::benchmark_data_smithi022_188626_object101607:head# 2024-09-11T07:50:25.689 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.689 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cac43cf3:::benchmark_data_smithi022_188626_object118401:head# 2024-09-11T07:50:25.689 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.689 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cac53fd4:::benchmark_data_smithi022_188626_object113658:head# 2024-09-11T07:50:25.690 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.690 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cac62927:::benchmark_data_smithi022_188626_object92961:head# 2024-09-11T07:50:25.690 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.690 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:caca8a15:::benchmark_data_smithi022_188626_object108423:head# 2024-09-11T07:50:25.690 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.690 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cacaa2b6:::benchmark_data_smithi022_188626_object94872:head# 2024-09-11T07:50:25.690 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.690 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cacb0ef3:::benchmark_data_smithi022_188626_object115230:head# 2024-09-11T07:50:25.690 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.690 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cacbd2e5:::benchmark_data_smithi022_188626_object116472:head# 2024-09-11T07:50:25.690 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.690 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cacdde5c:::benchmark_data_smithi022_188626_object99311:head# 2024-09-11T07:50:25.690 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.690 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cace2df3:::benchmark_data_smithi022_188626_object105496:head# 2024-09-11T07:50:25.691 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.691 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cacf4b45:::benchmark_data_smithi022_188626_object105236:head# 2024-09-11T07:50:25.691 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.691 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cad2abeb:::benchmark_data_smithi022_188626_object100445:head# 2024-09-11T07:50:25.691 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.691 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cad3ab41:::benchmark_data_smithi022_188626_object112414:head# 2024-09-11T07:50:25.691 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.691 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cad6484f:::benchmark_data_smithi022_188626_object109055:head# 2024-09-11T07:50:25.691 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.692 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cad81d7c:::benchmark_data_smithi022_188626_object98135:head# 2024-09-11T07:50:25.692 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.693 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cad890e2:::benchmark_data_smithi022_188626_object99828:head# 2024-09-11T07:50:25.693 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.693 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cadc65c4:::benchmark_data_smithi022_188626_object108281:head# 2024-09-11T07:50:25.693 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.693 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cadf82af:::benchmark_data_smithi022_188626_object118588:head# 2024-09-11T07:50:25.693 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.693 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cae08003:::benchmark_data_smithi022_188626_object95719:head# 2024-09-11T07:50:25.693 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.693 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cae16a9a:::benchmark_data_smithi022_188626_object90471:head# 2024-09-11T07:50:25.693 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.693 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cae34ddf:::benchmark_data_smithi022_188626_object108351:head# 2024-09-11T07:50:25.693 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.694 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cae35d50:::benchmark_data_smithi022_188626_object109181:head# 2024-09-11T07:50:25.694 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.694 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cae5f1fb:::benchmark_data_smithi022_188626_object90230:head# 2024-09-11T07:50:25.694 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.694 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cae623e2:::benchmark_data_smithi022_188626_object111333:head# 2024-09-11T07:50:25.694 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.694 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cae8da82:::benchmark_data_smithi022_188626_object113691:head# 2024-09-11T07:50:25.694 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.694 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cae9b492:::benchmark_data_smithi022_188626_object102860:head# 2024-09-11T07:50:25.694 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.694 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cae9d3f7:::benchmark_data_smithi022_188626_object90759:head# 2024-09-11T07:50:25.694 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.695 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:caeb0923:::benchmark_data_smithi022_188626_object93138:head# 2024-09-11T07:50:25.695 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.695 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:caeb8855:::benchmark_data_smithi022_188626_object104157:head# 2024-09-11T07:50:25.695 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.695 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:caefa077:::benchmark_data_smithi022_188626_object92155:head# 2024-09-11T07:50:25.695 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.695 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:caf0bd61:::benchmark_data_smithi022_188626_object98884:head# 2024-09-11T07:50:25.695 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.695 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:caf2b7f6:::benchmark_data_smithi022_188626_object115672:head# 2024-09-11T07:50:25.695 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.695 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:caf70a35:::benchmark_data_smithi022_188626_object103477:head# 2024-09-11T07:50:25.695 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.695 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:caf7565e:::benchmark_data_smithi022_188626_object99523:head# 2024-09-11T07:50:25.696 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.696 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:caf82fe8:::benchmark_data_smithi022_188626_object103587:head# 2024-09-11T07:50:25.696 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.696 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:caf860b6:::benchmark_data_smithi022_188626_object104920:head# 2024-09-11T07:50:25.696 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.696 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:caf861c3:::benchmark_data_smithi022_188626_object105877:head# 2024-09-11T07:50:25.696 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.696 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:caf89de4:::benchmark_data_smithi022_188626_object107618:head# 2024-09-11T07:50:25.696 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.696 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cafbbe4e:::benchmark_data_smithi022_188626_object114241:head# 2024-09-11T07:50:25.696 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.697 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cafcbfc8:::benchmark_data_smithi022_188626_object91270:head# 2024-09-11T07:50:25.697 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.697 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cafe1d16:::benchmark_data_smithi022_188626_object106408:head# 2024-09-11T07:50:25.697 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.697 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:caffb15f:::benchmark_data_smithi022_188626_object99103:head# 2024-09-11T07:50:25.697 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.697 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb03bb6a:::benchmark_data_smithi022_188626_object100500:head# 2024-09-11T07:50:25.697 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.697 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb062c0d:::benchmark_data_smithi022_188626_object115911:head# 2024-09-11T07:50:25.697 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.697 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb07c379:::benchmark_data_smithi022_188626_object101317:head# 2024-09-11T07:50:25.697 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.697 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb0b73a0:::benchmark_data_smithi022_188626_object109491:head# 2024-09-11T07:50:25.698 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.698 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb0f5e0f:::benchmark_data_smithi022_188626_object90998:head# 2024-09-11T07:50:25.698 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.698 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb1052c2:::benchmark_data_smithi022_188626_object91219:head# 2024-09-11T07:50:25.698 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.698 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb128008:::benchmark_data_smithi022_188626_object117146:head# 2024-09-11T07:50:25.698 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.698 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb133c82:::benchmark_data_smithi022_188626_object96887:head# 2024-09-11T07:50:25.698 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.698 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb133fba:::benchmark_data_smithi022_188626_object108688:head# 2024-09-11T07:50:25.698 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.698 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb13cc35:::benchmark_data_smithi022_188626_object95077:head# 2024-09-11T07:50:25.698 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.698 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb1982e4:::benchmark_data_smithi022_188626_object96180:head# 2024-09-11T07:50:25.699 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.699 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb1af136:::benchmark_data_smithi022_188626_object91915:head# 2024-09-11T07:50:25.699 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.699 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb1d8805:::benchmark_data_smithi022_188626_object104616:head# 2024-09-11T07:50:25.699 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.699 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb210c25:::benchmark_data_smithi022_188626_object101710:head# 2024-09-11T07:50:25.699 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.699 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb251c92:::benchmark_data_smithi022_188626_object110204:head# 2024-09-11T07:50:25.699 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.699 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb25de02:::benchmark_data_smithi022_188626_object93195:head# 2024-09-11T07:50:25.699 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.699 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb276767:::benchmark_data_smithi022_188626_object111141:head# 2024-09-11T07:50:25.699 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.699 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb28817c:::benchmark_data_smithi022_188626_object89857:head# 2024-09-11T07:50:25.700 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.700 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb29c893:::benchmark_data_smithi022_188626_object91598:head# 2024-09-11T07:50:25.700 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.700 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb2a53cc:::benchmark_data_smithi022_188626_object114711:head# 2024-09-11T07:50:25.700 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.700 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb2b4a71:::benchmark_data_smithi022_188626_object116661:head# 2024-09-11T07:50:25.700 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.700 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb2dbdc9:::benchmark_data_smithi022_188626_object112162:head# 2024-09-11T07:50:25.700 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.700 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb2dec85:::benchmark_data_smithi022_188626_object115811:head# 2024-09-11T07:50:25.700 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.700 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb2f4c37:::benchmark_data_smithi022_188626_object101845:head# 2024-09-11T07:50:25.700 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.700 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb2fe4e8:::benchmark_data_smithi022_188626_object109539:head# 2024-09-11T07:50:25.701 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.701 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb31868a:::benchmark_data_smithi022_188626_object103175:head# 2024-09-11T07:50:25.701 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.701 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb328d97:::benchmark_data_smithi022_188626_object113569:head# 2024-09-11T07:50:25.701 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.701 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb33914b:::benchmark_data_smithi022_188626_object99906:head# 2024-09-11T07:50:25.701 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.701 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb379a83:::benchmark_data_smithi022_188626_object90982:head# 2024-09-11T07:50:25.701 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.701 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb37a82a:::benchmark_data_smithi022_188626_object92982:head# 2024-09-11T07:50:25.701 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.701 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb387d45:::benchmark_data_smithi022_188626_object103978:head# 2024-09-11T07:50:25.701 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.702 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb390636:::benchmark_data_smithi022_188626_object110982:head# 2024-09-11T07:50:25.702 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.702 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb3922fd:::benchmark_data_smithi022_188626_object100630:head# 2024-09-11T07:50:25.702 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.702 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb393549:::benchmark_data_smithi022_188626_object110059:head# 2024-09-11T07:50:25.702 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.702 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb395683:::benchmark_data_smithi022_188626_object117306:head# 2024-09-11T07:50:25.702 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.702 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb39bca7:::benchmark_data_smithi022_188626_object105538:head# 2024-09-11T07:50:25.702 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.702 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb3ae216:::benchmark_data_smithi022_188626_object91483:head# 2024-09-11T07:50:25.702 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.702 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb3b97cd:::benchmark_data_smithi022_188626_object100561:head# 2024-09-11T07:50:25.702 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.702 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb3c34b8:::benchmark_data_smithi022_188626_object105605:head# 2024-09-11T07:50:25.703 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.703 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb3dfe47:::benchmark_data_smithi022_188626_object102173:head# 2024-09-11T07:50:25.703 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.703 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb3eb04b:::benchmark_data_smithi022_188626_object96472:head# 2024-09-11T07:50:25.703 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.703 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb3ff1e7:::benchmark_data_smithi022_188626_object98307:head# 2024-09-11T07:50:25.703 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.703 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb4fbc7c:::benchmark_data_smithi022_188626_object110192:head# 2024-09-11T07:50:25.703 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.703 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb514286:::benchmark_data_smithi022_188626_object99470:head# 2024-09-11T07:50:25.703 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.703 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb52ecb3:::benchmark_data_smithi022_188626_object118442:head# 2024-09-11T07:50:25.703 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.703 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb540dcf:::benchmark_data_smithi022_188626_object104341:head# 2024-09-11T07:50:25.703 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.703 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb54ea47:::benchmark_data_smithi022_188626_object91048:head# 2024-09-11T07:50:25.704 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.704 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb5a5ee6:::benchmark_data_smithi022_188626_object103167:head# 2024-09-11T07:50:25.704 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.704 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb5b27be:::benchmark_data_smithi022_188626_object109957:head# 2024-09-11T07:50:25.704 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.704 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb5bf50d:::benchmark_data_smithi022_188626_object113335:head# 2024-09-11T07:50:25.704 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.704 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb5c27ca:::benchmark_data_smithi022_188626_object109462:head# 2024-09-11T07:50:25.704 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.704 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb5da31a:::benchmark_data_smithi022_188626_object92670:head# 2024-09-11T07:50:25.704 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.704 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb5f3b48:::benchmark_data_smithi022_188626_object108971:head# 2024-09-11T07:50:25.705 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.705 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb62d5c0:::benchmark_data_smithi022_188626_object114154:head# 2024-09-11T07:50:25.705 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.705 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb65c403:::benchmark_data_smithi022_188626_object104495:head# 2024-09-11T07:50:25.705 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.705 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb6766eb:::benchmark_data_smithi022_188626_object118485:head# 2024-09-11T07:50:25.705 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.705 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb6acf44:::benchmark_data_smithi022_188626_object113396:head# 2024-09-11T07:50:25.705 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.705 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb6ee18f:::benchmark_data_smithi022_188626_object112945:head# 2024-09-11T07:50:25.705 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.705 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb70fdc2:::benchmark_data_smithi022_188626_object98369:head# 2024-09-11T07:50:25.705 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.706 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb71fcde:::benchmark_data_smithi022_188626_object105829:head# 2024-09-11T07:50:25.706 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.706 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb732052:::benchmark_data_smithi022_188626_object90014:head# 2024-09-11T07:50:25.706 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.706 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb751875:::benchmark_data_smithi022_188626_object110310:head# 2024-09-11T07:50:25.706 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.706 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb75488a:::benchmark_data_smithi022_188626_object93739:head# 2024-09-11T07:50:25.706 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.706 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb7558e4:::benchmark_data_smithi022_188626_object107536:head# 2024-09-11T07:50:25.706 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.706 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb767fff:::benchmark_data_smithi022_188626_object101714:head# 2024-09-11T07:50:25.706 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.706 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb77f073:::benchmark_data_smithi022_188626_object111428:head# 2024-09-11T07:50:25.707 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.707 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb79d1bc:::benchmark_data_smithi022_188626_object111970:head# 2024-09-11T07:50:25.707 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.707 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb7a40cd:::benchmark_data_smithi022_188626_object107712:head# 2024-09-11T07:50:25.707 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.707 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb7bc195:::benchmark_data_smithi022_188626_object95062:head# 2024-09-11T07:50:25.707 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.707 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb7bfca6:::benchmark_data_smithi022_188626_object113414:head# 2024-09-11T07:50:25.707 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.707 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb7dfdaf:::benchmark_data_smithi022_188626_object103614:head# 2024-09-11T07:50:25.707 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.708 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb7fd822:::benchmark_data_smithi022_188626_object108152:head# 2024-09-11T07:50:25.708 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.708 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb86000e:::benchmark_data_smithi022_188626_object89394:head# 2024-09-11T07:50:25.708 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.708 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb86191a:::benchmark_data_smithi022_188626_object95046:head# 2024-09-11T07:50:25.708 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.708 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb883c03:::benchmark_data_smithi022_188626_object118324:head# 2024-09-11T07:50:25.708 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.708 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb89024f:::benchmark_data_smithi022_188626_object109347:head# 2024-09-11T07:50:25.708 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.708 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb896d0a:::benchmark_data_smithi022_188626_object104177:head# 2024-09-11T07:50:25.708 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.709 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb8a18e5:::benchmark_data_smithi022_188626_object89656:head# 2024-09-11T07:50:25.709 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.709 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb8b1a65:::benchmark_data_smithi022_188626_object111412:head# 2024-09-11T07:50:25.709 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.709 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb8d05d7:::benchmark_data_smithi022_188626_object111171:head# 2024-09-11T07:50:25.709 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.709 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb8d0c77:::benchmark_data_smithi022_188626_object118488:head# 2024-09-11T07:50:25.709 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.709 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb8e89dd:::benchmark_data_smithi022_188626_object119151:head# 2024-09-11T07:50:25.709 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.709 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb8f7f0f:::benchmark_data_smithi022_188626_object115913:head# 2024-09-11T07:50:25.709 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.709 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb90973e:::benchmark_data_smithi022_188626_object108698:head# 2024-09-11T07:50:25.710 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.710 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb917907:::benchmark_data_smithi022_188626_object94277:head# 2024-09-11T07:50:25.710 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.710 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb99f559:::benchmark_data_smithi022_188626_object94251:head# 2024-09-11T07:50:25.710 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.710 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb9b38af:::benchmark_data_smithi022_188626_object96423:head# 2024-09-11T07:50:25.710 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.710 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb9cb22f:::benchmark_data_smithi022_188626_object91299:head# 2024-09-11T07:50:25.710 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.710 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cb9ef932:::benchmark_data_smithi022_188626_object95660:head# 2024-09-11T07:50:25.710 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.710 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cba01f2b:::benchmark_data_smithi022_188626_object92067:head# 2024-09-11T07:50:25.710 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.711 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cba031f9:::benchmark_data_smithi022_188626_object108561:head# 2024-09-11T07:50:25.711 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.711 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cba22a60:::benchmark_data_smithi022_188626_object114235:head# 2024-09-11T07:50:25.711 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.711 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cba55d67:::benchmark_data_smithi022_188626_object110999:head# 2024-09-11T07:50:25.711 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.711 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbab3486:::benchmark_data_smithi022_188626_object91366:head# 2024-09-11T07:50:25.711 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.711 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbac1479:::benchmark_data_smithi022_188626_object93003:head# 2024-09-11T07:50:25.711 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.711 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbaf80f2:::benchmark_data_smithi022_188626_object101323:head# 2024-09-11T07:50:25.711 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.711 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbb3c31b:::benchmark_data_smithi022_188626_object89573:head# 2024-09-11T07:50:25.712 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.712 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbb9b175:::benchmark_data_smithi022_188626_object104665:head# 2024-09-11T07:50:25.712 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.712 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbbbce35:::benchmark_data_smithi022_188626_object95753:head# 2024-09-11T07:50:25.712 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.712 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbbce796:::benchmark_data_smithi022_188626_object105732:head# 2024-09-11T07:50:25.712 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.712 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbbd9991:::benchmark_data_smithi022_188626_object101792:head# 2024-09-11T07:50:25.712 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.712 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbbda5bc:::benchmark_data_smithi022_188626_object107074:head# 2024-09-11T07:50:25.712 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.712 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbbde72a:::benchmark_data_smithi022_188626_object112670:head# 2024-09-11T07:50:25.713 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.713 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbbe8182:::benchmark_data_smithi022_188626_object112245:head# 2024-09-11T07:50:25.713 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.713 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbbecbf2:::benchmark_data_smithi022_188626_object113931:head# 2024-09-11T07:50:25.713 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.713 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbbf070a:::benchmark_data_smithi022_188626_object98343:head# 2024-09-11T07:50:25.713 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.713 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbc33df0:::benchmark_data_smithi022_188626_object116241:head# 2024-09-11T07:50:25.713 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.713 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbc56350:::benchmark_data_smithi022_188626_object117106:head# 2024-09-11T07:50:25.713 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.713 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbc75fa8:::benchmark_data_smithi022_188626_object91266:head# 2024-09-11T07:50:25.713 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.714 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbcb240d:::benchmark_data_smithi022_188626_object91007:head# 2024-09-11T07:50:25.714 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.714 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbcf5bd9:::benchmark_data_smithi022_188626_object102276:head# 2024-09-11T07:50:25.714 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.714 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbd04f99:::benchmark_data_smithi022_188626_object99234:head# 2024-09-11T07:50:25.714 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.714 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbd088f0:::benchmark_data_smithi022_188626_object109761:head# 2024-09-11T07:50:25.714 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.714 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbd51236:::benchmark_data_smithi022_188626_object100777:head# 2024-09-11T07:50:25.714 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.714 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbd6a458:::benchmark_data_smithi022_188626_object108071:head# 2024-09-11T07:50:25.714 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.714 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbda0762:::benchmark_data_smithi022_188626_object106524:head# 2024-09-11T07:50:25.714 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.715 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbdc9ab3:::benchmark_data_smithi022_188626_object99429:head# 2024-09-11T07:50:25.715 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.715 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbde3298:::benchmark_data_smithi022_188626_object102263:head# 2024-09-11T07:50:25.715 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.715 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbe2cb67:::benchmark_data_smithi022_188626_object114727:head# 2024-09-11T07:50:25.715 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.715 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbe7ed6a:::benchmark_data_smithi022_188626_object107940:head# 2024-09-11T07:50:25.715 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.715 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbe9bbe3:::benchmark_data_smithi022_188626_object98811:head# 2024-09-11T07:50:25.715 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.715 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbea2975:::benchmark_data_smithi022_188626_object106900:head# 2024-09-11T07:50:25.716 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.716 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbee0819:::benchmark_data_smithi022_188626_object119272:head# 2024-09-11T07:50:25.716 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.716 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbee353a:::benchmark_data_smithi022_188626_object101798:head# 2024-09-11T07:50:25.716 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.716 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbef7660:::benchmark_data_smithi022_188626_object93948:head# 2024-09-11T07:50:25.716 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.716 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbf0cca0:::benchmark_data_smithi022_188626_object108098:head# 2024-09-11T07:50:25.716 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.716 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbf62f6b:::benchmark_data_smithi022_188626_object91342:head# 2024-09-11T07:50:25.716 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.716 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbf66771:::benchmark_data_smithi022_188626_object110671:head# 2024-09-11T07:50:25.716 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.717 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cbf66d8b:::benchmark_data_smithi022_188626_object91332:head# 2024-09-11T07:50:25.717 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.717 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc024109:::benchmark_data_smithi022_188626_object107034:head# 2024-09-11T07:50:25.717 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.717 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc032262:::benchmark_data_smithi022_188626_object118539:head# 2024-09-11T07:50:25.717 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.717 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc06412a:::benchmark_data_smithi022_188626_object99219:head# 2024-09-11T07:50:25.717 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.717 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc06915f:::benchmark_data_smithi022_188626_object89754:head# 2024-09-11T07:50:25.717 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.717 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc0ec549:::benchmark_data_smithi022_188626_object111535:head# 2024-09-11T07:50:25.717 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.717 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc1392aa:::benchmark_data_smithi022_188626_object111621:head# 2024-09-11T07:50:25.718 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.718 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc159cdf:::benchmark_data_smithi022_188626_object101402:head# 2024-09-11T07:50:25.718 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.718 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc17609e:::benchmark_data_smithi022_188626_object100483:head# 2024-09-11T07:50:25.718 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.718 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc1a8740:::benchmark_data_smithi022_188626_object117905:head# 2024-09-11T07:50:25.718 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.718 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc1d37e2:::benchmark_data_smithi022_188626_object96346:head# 2024-09-11T07:50:25.718 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.718 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc1e21a2:::benchmark_data_smithi022_188626_object97267:head# 2024-09-11T07:50:25.718 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.718 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc2000d8:::benchmark_data_smithi022_188626_object108755:head# 2024-09-11T07:50:25.718 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.719 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc28b392:::benchmark_data_smithi022_188626_object106496:head# 2024-09-11T07:50:25.719 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.719 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc298004:::benchmark_data_smithi022_188626_object91961:head# 2024-09-11T07:50:25.719 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.719 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc2d910e:::benchmark_data_smithi022_188626_object114846:head# 2024-09-11T07:50:25.719 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.719 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc2e62f5:::benchmark_data_smithi022_188626_object98016:head# 2024-09-11T07:50:25.719 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.719 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc312135:::benchmark_data_smithi022_188626_object95928:head# 2024-09-11T07:50:25.719 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.719 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc34e8a8:::benchmark_data_smithi022_188626_object102232:head# 2024-09-11T07:50:25.719 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.719 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc34f563:::benchmark_data_smithi022_188626_object109496:head# 2024-09-11T07:50:25.720 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.720 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc374d69:::benchmark_data_smithi022_188626_object106564:head# 2024-09-11T07:50:25.720 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.720 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc380b4b:::benchmark_data_smithi022_188626_object118246:head# 2024-09-11T07:50:25.720 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.720 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc3f37fd:::benchmark_data_smithi022_188626_object109884:head# 2024-09-11T07:50:25.720 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.720 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc4259cc:::benchmark_data_smithi022_188626_object98894:head# 2024-09-11T07:50:25.720 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.720 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc4384f2:::benchmark_data_smithi022_188626_object115186:head# 2024-09-11T07:50:25.720 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.720 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc43c5bb:::benchmark_data_smithi022_188626_object105808:head# 2024-09-11T07:50:25.720 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.721 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc43ec60:::benchmark_data_smithi022_188626_object108068:head# 2024-09-11T07:50:25.721 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.721 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc45a740:::benchmark_data_smithi022_188626_object118668:head# 2024-09-11T07:50:25.721 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.721 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc46c1b6:::benchmark_data_smithi022_188626_object100625:head# 2024-09-11T07:50:25.721 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.721 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc4b0b68:::benchmark_data_smithi022_188626_object119235:head# 2024-09-11T07:50:25.721 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.721 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc50a295:::benchmark_data_smithi022_188626_object103856:head# 2024-09-11T07:50:25.721 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.721 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc550ca6:::benchmark_data_smithi022_188626_object109318:head# 2024-09-11T07:50:25.721 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.721 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc56bc52:::benchmark_data_smithi022_188626_object98156:head# 2024-09-11T07:50:25.722 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.722 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc5a7cff:::benchmark_data_smithi022_188626_object116489:head# 2024-09-11T07:50:25.722 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.722 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc5b173f:::benchmark_data_smithi022_188626_object117939:head# 2024-09-11T07:50:25.722 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.722 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc5bb1b5:::benchmark_data_smithi022_188626_object114081:head# 2024-09-11T07:50:25.722 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.722 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc628209:::benchmark_data_smithi022_188626_object106378:head# 2024-09-11T07:50:25.722 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.722 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc64d1be:::benchmark_data_smithi022_188626_object96868:head# 2024-09-11T07:50:25.722 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.722 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc682812:::benchmark_data_smithi022_188626_object106218:head# 2024-09-11T07:50:25.722 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.722 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc69f0e2:::benchmark_data_smithi022_188626_object107349:head# 2024-09-11T07:50:25.723 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.723 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc6a2c32:::benchmark_data_smithi022_188626_object99276:head# 2024-09-11T07:50:25.723 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.723 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc6a95bc:::benchmark_data_smithi022_188626_object95847:head# 2024-09-11T07:50:25.723 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.723 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc6e7579:::benchmark_data_smithi022_188626_object107602:head# 2024-09-11T07:50:25.723 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.723 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc6f3ab3:::benchmark_data_smithi022_188626_object95249:head# 2024-09-11T07:50:25.723 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.723 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc71a456:::benchmark_data_smithi022_188626_object105729:head# 2024-09-11T07:50:25.723 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.723 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc73d37d:::benchmark_data_smithi022_188626_object119455:head# 2024-09-11T07:50:25.723 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.723 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc763f36:::benchmark_data_smithi022_188626_object107792:head# 2024-09-11T07:50:25.723 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.724 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc78c607:::benchmark_data_smithi022_188626_object93682:head# 2024-09-11T07:50:25.724 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.724 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc78c6e5:::benchmark_data_smithi022_188626_object104504:head# 2024-09-11T07:50:25.724 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.724 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc7ce4fc:::benchmark_data_smithi022_188626_object116006:head# 2024-09-11T07:50:25.724 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.724 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc811976:::benchmark_data_smithi022_188626_object118453:head# 2024-09-11T07:50:25.724 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.724 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc821c1f:::benchmark_data_smithi022_188626_object89932:head# 2024-09-11T07:50:25.724 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.724 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc82708e:::benchmark_data_smithi022_188626_object107550:head# 2024-09-11T07:50:25.724 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.725 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc83eade:::benchmark_data_smithi022_188626_object89884:head# 2024-09-11T07:50:25.725 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.725 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc863888:::benchmark_data_smithi022_188626_object108368:head# 2024-09-11T07:50:25.725 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.725 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc891a45:::benchmark_data_smithi022_188626_object93950:head# 2024-09-11T07:50:25.725 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.725 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc892385:::benchmark_data_smithi022_188626_object119670:head# 2024-09-11T07:50:25.725 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.725 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc89c70b:::benchmark_data_smithi022_188626_object90669:head# 2024-09-11T07:50:25.725 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.726 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc8c6147:::benchmark_data_smithi022_188626_object105620:head# 2024-09-11T07:50:25.726 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.726 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc8cc406:::benchmark_data_smithi022_188626_object109908:head# 2024-09-11T07:50:25.726 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.726 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc8e133a:::benchmark_data_smithi022_188626_object99307:head# 2024-09-11T07:50:25.726 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.726 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc90c49d:::benchmark_data_smithi022_188626_object103722:head# 2024-09-11T07:50:25.726 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.726 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc929c03:::benchmark_data_smithi022_188626_object117179:head# 2024-09-11T07:50:25.726 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.726 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc960dc5:::benchmark_data_smithi022_188626_object109441:head# 2024-09-11T07:50:25.726 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.726 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc98641d:::benchmark_data_smithi022_188626_object106438:head# 2024-09-11T07:50:25.727 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.727 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc9a62f5:::benchmark_data_smithi022_188626_object115473:head# 2024-09-11T07:50:25.727 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.727 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cc9b5639:::benchmark_data_smithi022_188626_object106866:head# 2024-09-11T07:50:25.727 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.727 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cca09110:::benchmark_data_smithi022_188626_object104286:head# 2024-09-11T07:50:25.727 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.727 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cca1bdda:::benchmark_data_smithi022_188626_object90615:head# 2024-09-11T07:50:25.727 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.727 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cca3fa2d:::benchmark_data_smithi022_188626_object97549:head# 2024-09-11T07:50:25.727 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.727 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cca50eb3:::benchmark_data_smithi022_188626_object109778:head# 2024-09-11T07:50:25.727 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.728 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cca789b4:::benchmark_data_smithi022_188626_object115948:head# 2024-09-11T07:50:25.728 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.728 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cca7be26:::benchmark_data_smithi022_188626_object112331:head# 2024-09-11T07:50:25.728 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.728 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cca8515f:::benchmark_data_smithi022_188626_object100730:head# 2024-09-11T07:50:25.728 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.728 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cca9aafc:::benchmark_data_smithi022_188626_object102916:head# 2024-09-11T07:50:25.728 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.728 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccab8d69:::benchmark_data_smithi022_188626_object116265:head# 2024-09-11T07:50:25.728 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.728 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccac3eae:::benchmark_data_smithi022_188626_object97422:head# 2024-09-11T07:50:25.728 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.728 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccadbaab:::benchmark_data_smithi022_188626_object93363:head# 2024-09-11T07:50:25.728 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.729 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccaeb1b7:::benchmark_data_smithi022_188626_object100118:head# 2024-09-11T07:50:25.729 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.729 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccaf2066:::benchmark_data_smithi022_188626_object111821:head# 2024-09-11T07:50:25.729 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.729 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccafd843:::benchmark_data_smithi022_188626_object99283:head# 2024-09-11T07:50:25.729 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.729 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccafe208:::benchmark_data_smithi022_188626_object119759:head# 2024-09-11T07:50:25.729 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.729 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccb012b4:::benchmark_data_smithi022_188626_object106180:head# 2024-09-11T07:50:25.729 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.729 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccb7a48b:::benchmark_data_smithi022_188626_object100008:head# 2024-09-11T07:50:25.729 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.729 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccb8aa0e:::benchmark_data_smithi022_188626_object105988:head# 2024-09-11T07:50:25.730 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.730 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccb9f7b4:::benchmark_data_smithi022_188626_object90099:head# 2024-09-11T07:50:25.730 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.730 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccbabe64:::benchmark_data_smithi022_188626_object116871:head# 2024-09-11T07:50:25.730 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.730 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccbfcd2a:::benchmark_data_smithi022_188626_object100159:head# 2024-09-11T07:50:25.730 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.730 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccc08417:::benchmark_data_smithi022_188626_object112579:head# 2024-09-11T07:50:25.730 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.730 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccc30ca7:::benchmark_data_smithi022_188626_object96432:head# 2024-09-11T07:50:25.730 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.730 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccc42563:::benchmark_data_smithi022_188626_object90875:head# 2024-09-11T07:50:25.730 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.731 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccc59cd8:::benchmark_data_smithi022_188626_object112035:head# 2024-09-11T07:50:25.731 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.731 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccc61dde:::benchmark_data_smithi022_188626_object91859:head# 2024-09-11T07:50:25.731 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.731 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccc87a10:::benchmark_data_smithi022_188626_object115481:head# 2024-09-11T07:50:25.731 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.731 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cccbbfc3:::benchmark_data_smithi022_188626_object102561:head# 2024-09-11T07:50:25.731 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.731 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cccd39a7:::benchmark_data_smithi022_188626_object116417:head# 2024-09-11T07:50:25.731 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.731 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccce2cde:::benchmark_data_smithi022_188626_object90999:head# 2024-09-11T07:50:25.731 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.731 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccce4f66:::benchmark_data_smithi022_188626_object101184:head# 2024-09-11T07:50:25.731 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.732 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccd41ddd:::benchmark_data_smithi022_188626_object106864:head# 2024-09-11T07:50:25.732 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.732 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccd6afe1:::benchmark_data_smithi022_188626_object116399:head# 2024-09-11T07:50:25.732 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.732 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccdb6608:::benchmark_data_smithi022_188626_object106647:head# 2024-09-11T07:50:25.732 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.732 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccdc5447:::benchmark_data_smithi022_188626_object94564:head# 2024-09-11T07:50:25.732 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.732 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cce4f895:::benchmark_data_smithi022_188626_object113976:head# 2024-09-11T07:50:25.732 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.733 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cce6d5af:::benchmark_data_smithi022_188626_object109168:head# 2024-09-11T07:50:25.733 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.733 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cce94722:::benchmark_data_smithi022_188626_object98870:head# 2024-09-11T07:50:25.733 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.733 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cceb1174:::benchmark_data_smithi022_188626_object116552:head# 2024-09-11T07:50:25.733 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.733 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccf07b65:::benchmark_data_smithi022_188626_object93538:head# 2024-09-11T07:50:25.733 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.733 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccf3950b:::benchmark_data_smithi022_188626_object93087:head# 2024-09-11T07:50:25.733 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.733 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccf599db:::benchmark_data_smithi022_188626_object95659:head# 2024-09-11T07:50:25.733 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.733 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccfb4b70:::benchmark_data_smithi022_188626_object93103:head# 2024-09-11T07:50:25.734 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.734 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ccfec9f3:::benchmark_data_smithi022_188626_object89612:head# 2024-09-11T07:50:25.734 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.734 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd0111fc:::benchmark_data_smithi022_188626_object94457:head# 2024-09-11T07:50:25.734 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.734 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd0304e4:::benchmark_data_smithi022_188626_object102060:head# 2024-09-11T07:50:25.734 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.734 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd04c5d4:::benchmark_data_smithi022_188626_object109302:head# 2024-09-11T07:50:25.734 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.734 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd06daba:::benchmark_data_smithi022_188626_object90972:head# 2024-09-11T07:50:25.734 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.734 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd0860ba:::benchmark_data_smithi022_188626_object102512:head# 2024-09-11T07:50:25.734 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.735 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd09dc08:::benchmark_data_smithi022_188626_object118287:head# 2024-09-11T07:50:25.735 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.735 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd100ed7:::benchmark_data_smithi022_188626_object107367:head# 2024-09-11T07:50:25.735 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.735 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd125288:::benchmark_data_smithi022_188626_object113487:head# 2024-09-11T07:50:25.735 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.735 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd12b24b:::benchmark_data_smithi022_188626_object91866:head# 2024-09-11T07:50:25.735 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.735 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd1352a7:::benchmark_data_smithi022_188626_object90186:head# 2024-09-11T07:50:25.735 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.735 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd1440d9:::benchmark_data_smithi022_188626_object93695:head# 2024-09-11T07:50:25.736 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.736 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd1538ec:::benchmark_data_smithi022_188626_object91685:head# 2024-09-11T07:50:25.736 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.736 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd174f33:::benchmark_data_smithi022_188626_object115463:head# 2024-09-11T07:50:25.736 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.736 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:25 smithi022 ceph-mon[37330]: 1.0 scrub starts 2024-09-11T07:50:25.736 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:25 smithi022 ceph-mon[37330]: 1.0 scrub ok 2024-09-11T07:50:25.736 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:25 smithi022 ceph-mon[37330]: 4.a scrub starts 2024-09-11T07:50:25.737 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:25 smithi022 ceph-mon[37330]: 4.a scrub ok 2024-09-11T07:50:25.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:25 smithi022 ceph-mon[29462]: 1.0 scrub starts 2024-09-11T07:50:25.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:25 smithi022 ceph-mon[29462]: 1.0 scrub ok 2024-09-11T07:50:25.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:25 smithi022 ceph-mon[29462]: 4.a scrub starts 2024-09-11T07:50:25.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:25 smithi022 ceph-mon[29462]: 4.a scrub ok 2024-09-11T07:50:25.738 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd1a9c91:::benchmark_data_smithi022_188626_object94640:head# 2024-09-11T07:50:25.738 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.738 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd1d4e51:::benchmark_data_smithi022_188626_object91063:head# 2024-09-11T07:50:25.738 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.738 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd1df515:::benchmark_data_smithi022_188626_object90806:head# 2024-09-11T07:50:25.738 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.738 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd229341:::benchmark_data_smithi022_188626_object116138:head# 2024-09-11T07:50:25.738 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.739 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd22f30b:::benchmark_data_smithi022_188626_object94415:head# 2024-09-11T07:50:25.739 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.739 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd230e31:::benchmark_data_smithi022_188626_object93348:head# 2024-09-11T07:50:25.739 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.739 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd2703e4:::benchmark_data_smithi022_188626_object111164:head# 2024-09-11T07:50:25.739 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.739 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd27513e:::benchmark_data_smithi022_188626_object93191:head# 2024-09-11T07:50:25.739 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.739 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd2a3e92:::benchmark_data_smithi022_188626_object112114:head# 2024-09-11T07:50:25.739 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.739 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd2b3ea2:::benchmark_data_smithi022_188626_object95955:head# 2024-09-11T07:50:25.739 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.739 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd2bcf2f:::benchmark_data_smithi022_188626_object114362:head# 2024-09-11T07:50:25.739 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.739 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd2f403a:::benchmark_data_smithi022_188626_object112784:head# 2024-09-11T07:50:25.740 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.740 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd2fb493:::benchmark_data_smithi022_188626_object114327:head# 2024-09-11T07:50:25.740 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.740 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd30834c:::benchmark_data_smithi022_188626_object96264:head# 2024-09-11T07:50:25.740 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.740 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd324037:::benchmark_data_smithi022_188626_object105071:head# 2024-09-11T07:50:25.740 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.740 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd34cff5:::benchmark_data_smithi022_188626_object96407:head# 2024-09-11T07:50:25.740 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.740 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd379b1f:::benchmark_data_smithi022_188626_object111096:head# 2024-09-11T07:50:25.740 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.740 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd3889f3:::benchmark_data_smithi022_188626_object108502:head# 2024-09-11T07:50:25.740 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.741 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd3bbed7:::benchmark_data_smithi022_188626_object113927:head# 2024-09-11T07:50:25.741 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.741 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd3c3a4f:::benchmark_data_smithi022_188626_object117688:head# 2024-09-11T07:50:25.741 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.741 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd3feef5:::benchmark_data_smithi022_188626_object119136:head# 2024-09-11T07:50:25.741 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.741 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd4077e9:::benchmark_data_smithi022_188626_object116996:head# 2024-09-11T07:50:25.741 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.741 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd4108a0:::benchmark_data_smithi022_188626_object97213:head# 2024-09-11T07:50:25.741 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.741 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd449777:::benchmark_data_smithi022_188626_object118537:head# 2024-09-11T07:50:25.741 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.742 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd45dd82:::benchmark_data_smithi022_188626_object105762:head# 2024-09-11T07:50:25.742 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.742 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd4668d1:::benchmark_data_smithi022_188626_object115954:head# 2024-09-11T07:50:25.742 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.742 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd467937:::benchmark_data_smithi022_188626_object112288:head# 2024-09-11T07:50:25.742 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.742 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd46d9a7:::benchmark_data_smithi022_188626_object96119:head# 2024-09-11T07:50:25.742 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.742 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd476994:::benchmark_data_smithi022_188626_object117458:head# 2024-09-11T07:50:25.742 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.742 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd4845a4:::benchmark_data_smithi022_188626_object116783:head# 2024-09-11T07:50:25.742 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.742 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd497372:::benchmark_data_smithi022_188626_object103116:head# 2024-09-11T07:50:25.743 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.743 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd4ae9ca:::benchmark_data_smithi022_188626_object110057:head# 2024-09-11T07:50:25.743 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.743 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd4aed42:::benchmark_data_smithi022_188626_object103382:head# 2024-09-11T07:50:25.743 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.743 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd4ba7b7:::benchmark_data_smithi022_188626_object90536:head# 2024-09-11T07:50:25.743 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.743 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd4d620b:::benchmark_data_smithi022_188626_object90096:head# 2024-09-11T07:50:25.743 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.743 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd4d8653:::benchmark_data_smithi022_188626_object99949:head# 2024-09-11T07:50:25.743 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.743 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd4f7c5c:::benchmark_data_smithi022_188626_object106073:head# 2024-09-11T07:50:25.743 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.744 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd50bbd1:::benchmark_data_smithi022_188626_object109325:head# 2024-09-11T07:50:25.744 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.744 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd514a53:::benchmark_data_smithi022_188626_object118954:head# 2024-09-11T07:50:25.744 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.744 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd51700d:::benchmark_data_smithi022_188626_object107591:head# 2024-09-11T07:50:25.744 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.744 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd5175b1:::benchmark_data_smithi022_188626_object101586:head# 2024-09-11T07:50:25.744 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.744 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd5276e1:::benchmark_data_smithi022_188626_object108772:head# 2024-09-11T07:50:25.744 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.744 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd53d024:::benchmark_data_smithi022_188626_object109490:head# 2024-09-11T07:50:25.744 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.744 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd5400a6:::benchmark_data_smithi022_188626_object102091:head# 2024-09-11T07:50:25.744 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.745 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd59263d:::benchmark_data_smithi022_188626_object96289:head# 2024-09-11T07:50:25.745 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.745 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd597a9e:::benchmark_data_smithi022_188626_object96610:head# 2024-09-11T07:50:25.745 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.745 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd5b7047:::benchmark_data_smithi022_188626_object110964:head# 2024-09-11T07:50:25.745 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.745 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd5e0cbf:::benchmark_data_smithi022_188626_object94748:head# 2024-09-11T07:50:25.745 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.745 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd65629b:::benchmark_data_smithi022_188626_object93550:head# 2024-09-11T07:50:25.745 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.745 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd65f4ec:::benchmark_data_smithi022_188626_object90858:head# 2024-09-11T07:50:25.745 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.745 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd6651c0:::benchmark_data_smithi022_188626_object111616:head# 2024-09-11T07:50:25.745 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.746 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd680355:::benchmark_data_smithi022_188626_object102494:head# 2024-09-11T07:50:25.746 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.746 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd68697a:::benchmark_data_smithi022_188626_object106281:head# 2024-09-11T07:50:25.746 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.746 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd69121c:::benchmark_data_smithi022_188626_object106002:head# 2024-09-11T07:50:25.746 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.746 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd6b1408:::benchmark_data_smithi022_188626_object101370:head# 2024-09-11T07:50:25.746 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.746 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd6b7d69:::benchmark_data_smithi022_188626_object104484:head# 2024-09-11T07:50:25.746 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.746 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd6bec73:::benchmark_data_smithi022_188626_object90906:head# 2024-09-11T07:50:25.746 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.746 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd6d681e:::benchmark_data_smithi022_188626_object100292:head# 2024-09-11T07:50:25.747 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.747 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd6dc679:::benchmark_data_smithi022_188626_object88985:head# 2024-09-11T07:50:25.747 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.747 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd6fd920:::benchmark_data_smithi022_188626_object93321:head# 2024-09-11T07:50:25.747 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.747 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd77eab2:::benchmark_data_smithi022_188626_object97768:head# 2024-09-11T07:50:25.747 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.747 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd7ce8c8:::benchmark_data_smithi022_188626_object96354:head# 2024-09-11T07:50:25.747 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.747 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd7db560:::benchmark_data_smithi022_188626_object93688:head# 2024-09-11T07:50:25.747 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.747 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd832573:::benchmark_data_smithi022_188626_object106816:head# 2024-09-11T07:50:25.747 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.748 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd8592db:::benchmark_data_smithi022_188626_object98455:head# 2024-09-11T07:50:25.748 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.748 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd8c2ebe:::benchmark_data_smithi022_188626_object105545:head# 2024-09-11T07:50:25.748 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.748 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd8f2521:::benchmark_data_smithi022_188626_object100493:head# 2024-09-11T07:50:25.748 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.748 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd911670:::benchmark_data_smithi022_188626_object99918:head# 2024-09-11T07:50:25.748 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.748 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd9277f0:::benchmark_data_smithi022_188626_object104570:head# 2024-09-11T07:50:25.748 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.748 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd9353f0:::benchmark_data_smithi022_188626_object117641:head# 2024-09-11T07:50:25.748 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.748 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd93f2f2:::benchmark_data_smithi022_188626_object91010:head# 2024-09-11T07:50:25.749 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.749 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd9959b7:::benchmark_data_smithi022_188626_object99147:head# 2024-09-11T07:50:25.749 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.749 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd9a472e:::benchmark_data_smithi022_188626_object111448:head# 2024-09-11T07:50:25.749 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.749 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd9df8bb:::benchmark_data_smithi022_188626_object113561:head# 2024-09-11T07:50:25.749 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.749 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd9e77bf:::benchmark_data_smithi022_188626_object112908:head# 2024-09-11T07:50:25.749 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.749 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd9ea3ce:::benchmark_data_smithi022_188626_object97389:head# 2024-09-11T07:50:25.749 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.749 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cd9f41f9:::benchmark_data_smithi022_188626_object93986:head# 2024-09-11T07:50:25.750 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.750 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cda08fe2:::benchmark_data_smithi022_188626_object100588:head# 2024-09-11T07:50:25.750 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.750 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cda4adc0:::benchmark_data_smithi022_188626_object104108:head# 2024-09-11T07:50:25.750 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.750 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cdac8206:::benchmark_data_smithi022_188626_object111373:head# 2024-09-11T07:50:25.750 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.750 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cdb500ff:::benchmark_data_smithi022_188626_object100035:head# 2024-09-11T07:50:25.750 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.750 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cdbbffe8:::benchmark_data_smithi022_188626_object100144:head# 2024-09-11T07:50:25.750 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.750 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cdbdc80f:::benchmark_data_smithi022_188626_object92902:head# 2024-09-11T07:50:25.750 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.751 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cdc7dfd6:::benchmark_data_smithi022_188626_object94803:head# 2024-09-11T07:50:25.751 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.751 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cdc81fc7:::benchmark_data_smithi022_188626_object99210:head# 2024-09-11T07:50:25.751 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.751 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cdc8bd13:::benchmark_data_smithi022_188626_object102991:head# 2024-09-11T07:50:25.751 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.751 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cdc93869:::benchmark_data_smithi022_188626_object95850:head# 2024-09-11T07:50:25.751 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.751 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cdccb943:::benchmark_data_smithi022_188626_object89200:head# 2024-09-11T07:50:25.751 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.751 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cdcd673a:::benchmark_data_smithi022_188626_object104267:head# 2024-09-11T07:50:25.751 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.751 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cdcf1ade:::benchmark_data_smithi022_188626_object112490:head# 2024-09-11T07:50:25.751 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.752 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cdcf20dc:::benchmark_data_smithi022_188626_object106391:head# 2024-09-11T07:50:25.752 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.752 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cdd0d553:::benchmark_data_smithi022_188626_object113934:head# 2024-09-11T07:50:25.752 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.752 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cdd17320:::benchmark_data_smithi022_188626_object112232:head# 2024-09-11T07:50:25.752 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.752 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cdd99f0a:::benchmark_data_smithi022_188626_object105262:head# 2024-09-11T07:50:25.752 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.752 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cddaa316:::benchmark_data_smithi022_188626_object113008:head# 2024-09-11T07:50:25.752 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.752 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cddb4486:::benchmark_data_smithi022_188626_object94860:head# 2024-09-11T07:50:25.752 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.753 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cde13f17:::benchmark_data_smithi022_188626_object108270:head# 2024-09-11T07:50:25.753 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.753 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cde950e0:::benchmark_data_smithi022_188626_object95301:head# 2024-09-11T07:50:25.753 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.753 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cdec741c:::benchmark_data_smithi022_188626_object119212:head# 2024-09-11T07:50:25.753 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.753 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cdf02929:::benchmark_data_smithi022_188626_object108139:head# 2024-09-11T07:50:25.753 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.753 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cdf796ef:::benchmark_data_smithi022_188626_object100639:head# 2024-09-11T07:50:25.753 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.753 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cdf90929:::benchmark_data_smithi022_188626_object111753:head# 2024-09-11T07:50:25.753 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.754 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cdfa829f:::benchmark_data_smithi022_188626_object89749:head# 2024-09-11T07:50:25.754 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.754 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cdfccf03:::benchmark_data_smithi022_188626_object99334:head# 2024-09-11T07:50:25.754 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.754 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cdfe25b0:::benchmark_data_smithi022_188626_object115733:head# 2024-09-11T07:50:25.754 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.754 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce010e76:::benchmark_data_smithi022_188626_object105776:head# 2024-09-11T07:50:25.754 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.754 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce055c94:::benchmark_data_smithi022_188626_object92079:head# 2024-09-11T07:50:25.754 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.754 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce0663e2:::benchmark_data_smithi022_188626_object94683:head# 2024-09-11T07:50:25.754 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.754 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce0994da:::benchmark_data_smithi022_188626_object89225:head# 2024-09-11T07:50:25.754 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.755 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce0bb028:::benchmark_data_smithi022_188626_object113605:head# 2024-09-11T07:50:25.755 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.755 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce0ccb9b:::benchmark_data_smithi022_188626_object104699:head# 2024-09-11T07:50:25.755 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.755 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce0cdd0a:::benchmark_data_smithi022_188626_object118418:head# 2024-09-11T07:50:25.755 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.755 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce0d71ea:::benchmark_data_smithi022_188626_object110141:head# 2024-09-11T07:50:25.755 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.755 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce0ddde7:::benchmark_data_smithi022_188626_object90653:head# 2024-09-11T07:50:25.755 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.755 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce10247c:::benchmark_data_smithi022_188626_object105246:head# 2024-09-11T07:50:25.755 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.755 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce102d27:::benchmark_data_smithi022_188626_object113947:head# 2024-09-11T07:50:25.756 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.756 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce137647:::benchmark_data_smithi022_188626_object95538:head# 2024-09-11T07:50:25.756 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.756 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce14acf5:::benchmark_data_smithi022_188626_object117743:head# 2024-09-11T07:50:25.756 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.756 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce1cc205:::benchmark_data_smithi022_188626_object113563:head# 2024-09-11T07:50:25.756 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.756 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce2095e2:::benchmark_data_smithi022_188626_object112497:head# 2024-09-11T07:50:25.756 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.756 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce23ae50:::benchmark_data_smithi022_188626_object104446:head# 2024-09-11T07:50:25.756 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.756 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce26bc46:::benchmark_data_smithi022_188626_object104520:head# 2024-09-11T07:50:25.756 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.756 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce2b7b07:::benchmark_data_smithi022_188626_object105537:head# 2024-09-11T07:50:25.757 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.757 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce2ce8e2:::benchmark_data_smithi022_188626_object117648:head# 2024-09-11T07:50:25.757 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.757 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce2d9043:::benchmark_data_smithi022_188626_object107965:head# 2024-09-11T07:50:25.757 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.757 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce322e53:::benchmark_data_smithi022_188626_object112628:head# 2024-09-11T07:50:25.757 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.757 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce36a3b9:::benchmark_data_smithi022_188626_object96893:head# 2024-09-11T07:50:25.757 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.757 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce387bbd:::benchmark_data_smithi022_188626_object95353:head# 2024-09-11T07:50:25.757 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.757 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce39d800:::benchmark_data_smithi022_188626_object89883:head# 2024-09-11T07:50:25.757 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.758 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce3cc271:::benchmark_data_smithi022_188626_object94570:head# 2024-09-11T07:50:25.758 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.758 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce3e3dad:::benchmark_data_smithi022_188626_object96285:head# 2024-09-11T07:50:25.758 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.758 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce412191:::benchmark_data_smithi022_188626_object116377:head# 2024-09-11T07:50:25.758 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.758 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce43d750:::benchmark_data_smithi022_188626_object95756:head# 2024-09-11T07:50:25.758 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.758 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce46884a:::benchmark_data_smithi022_188626_object113983:head# 2024-09-11T07:50:25.758 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.758 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce46f2c7:::benchmark_data_smithi022_188626_object111227:head# 2024-09-11T07:50:25.758 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.758 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce49dd6c:::benchmark_data_smithi022_188626_object118153:head# 2024-09-11T07:50:25.759 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.759 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce4b6e01:::benchmark_data_smithi022_188626_object113279:head# 2024-09-11T07:50:25.759 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.759 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce4df6e8:::benchmark_data_smithi022_188626_object97191:head# 2024-09-11T07:50:25.759 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.759 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce4e1cff:::benchmark_data_smithi022_188626_object100499:head# 2024-09-11T07:50:25.759 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.759 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce51c2a4:::benchmark_data_smithi022_188626_object98255:head# 2024-09-11T07:50:25.759 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.759 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce52dc9d:::benchmark_data_smithi022_188626_object97524:head# 2024-09-11T07:50:25.759 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.759 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce52f17a:::benchmark_data_smithi022_188626_object92890:head# 2024-09-11T07:50:25.759 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.759 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce53b075:::benchmark_data_smithi022_188626_object118428:head# 2024-09-11T07:50:25.760 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.760 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce57f267:::benchmark_data_smithi022_188626_object106873:head# 2024-09-11T07:50:25.760 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.760 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce58338a:::benchmark_data_smithi022_188626_object89812:head# 2024-09-11T07:50:25.760 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.760 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce58b3b5:::benchmark_data_smithi022_188626_object90873:head# 2024-09-11T07:50:25.760 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.760 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce5986ef:::benchmark_data_smithi022_188626_object93842:head# 2024-09-11T07:50:25.760 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.760 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce59e7ca:::benchmark_data_smithi022_188626_object94881:head# 2024-09-11T07:50:25.760 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.760 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce59f864:::benchmark_data_smithi022_188626_object89147:head# 2024-09-11T07:50:25.760 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.761 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce5e7fd6:::benchmark_data_smithi022_188626_object104751:head# 2024-09-11T07:50:25.761 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.761 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce5e9cd5:::benchmark_data_smithi022_188626_object108336:head# 2024-09-11T07:50:25.761 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.761 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce618c9b:::benchmark_data_smithi022_188626_object112435:head# 2024-09-11T07:50:25.761 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.761 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce645d45:::benchmark_data_smithi022_188626_object106507:head# 2024-09-11T07:50:25.761 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.761 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce65c6d0:::benchmark_data_smithi022_188626_object104896:head# 2024-09-11T07:50:25.761 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.761 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce70a368:::benchmark_data_smithi022_188626_object111827:head# 2024-09-11T07:50:25.761 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.762 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce71b1d6:::benchmark_data_smithi022_188626_object110563:head# 2024-09-11T07:50:25.762 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.762 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce73750e:::benchmark_data_smithi022_188626_object95337:head# 2024-09-11T07:50:25.762 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.762 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce746f17:::benchmark_data_smithi022_188626_object91606:head# 2024-09-11T07:50:25.762 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.762 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce756e6c:::benchmark_data_smithi022_188626_object105663:head# 2024-09-11T07:50:25.762 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.762 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce75e90a:::benchmark_data_smithi022_188626_object96383:head# 2024-09-11T07:50:25.762 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.762 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce770869:::benchmark_data_smithi022_188626_object92238:head# 2024-09-11T07:50:25.762 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.762 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce7d7b00:::benchmark_data_smithi022_188626_object98067:head# 2024-09-11T07:50:25.762 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.763 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce7ee984:::benchmark_data_smithi022_188626_object98310:head# 2024-09-11T07:50:25.763 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.763 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce80fefd:::benchmark_data_smithi022_188626_object108713:head# 2024-09-11T07:50:25.763 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.763 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce815992:::benchmark_data_smithi022_188626_object118756:head# 2024-09-11T07:50:25.763 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.763 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce82c05b:::benchmark_data_smithi022_188626_object103805:head# 2024-09-11T07:50:25.763 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.763 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce8374f0:::benchmark_data_smithi022_188626_object105160:head# 2024-09-11T07:50:25.763 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.763 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce872482:::benchmark_data_smithi022_188626_object116189:head# 2024-09-11T07:50:25.763 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.763 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce8d74da:::benchmark_data_smithi022_188626_object96352:head# 2024-09-11T07:50:25.764 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.764 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce8e2d14:::benchmark_data_smithi022_188626_object118814:head# 2024-09-11T07:50:25.764 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.764 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce8ec3b4:::benchmark_data_smithi022_188626_object119798:head# 2024-09-11T07:50:25.764 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.764 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce9142bf:::benchmark_data_smithi022_188626_object117446:head# 2024-09-11T07:50:25.764 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.764 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce95629b:::benchmark_data_smithi022_188626_object114516:head# 2024-09-11T07:50:25.764 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.764 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce968306:::benchmark_data_smithi022_188626_object105173:head# 2024-09-11T07:50:25.764 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.764 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce975f03:::benchmark_data_smithi022_188626_object114245:head# 2024-09-11T07:50:25.765 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.765 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce99a6b4:::benchmark_data_smithi022_188626_object117379:head# 2024-09-11T07:50:25.765 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.765 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce9bc84d:::benchmark_data_smithi022_188626_object98288:head# 2024-09-11T07:50:25.765 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.765 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ce9dcb3d:::benchmark_data_smithi022_188626_object89607:head# 2024-09-11T07:50:25.765 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.765 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cea34f2f:::benchmark_data_smithi022_188626_object112822:head# 2024-09-11T07:50:25.765 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.765 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cea67676:::benchmark_data_smithi022_188626_object110509:head# 2024-09-11T07:50:25.765 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.765 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cea6f550:::benchmark_data_smithi022_188626_object114838:head# 2024-09-11T07:50:25.765 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.765 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cea72d1f:::benchmark_data_smithi022_188626_object101495:head# 2024-09-11T07:50:25.766 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.766 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cea7dabd:::benchmark_data_smithi022_188626_object100260:head# 2024-09-11T07:50:25.766 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.766 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ceaaaeb1:::benchmark_data_smithi022_188626_object96164:head# 2024-09-11T07:50:25.766 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.766 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ceac2790:::benchmark_data_smithi022_188626_object94767:head# 2024-09-11T07:50:25.766 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.766 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ceaff13a:::benchmark_data_smithi022_188626_object104762:head# 2024-09-11T07:50:25.766 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.766 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ceb08681:::benchmark_data_smithi022_188626_object101512:head# 2024-09-11T07:50:25.766 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.766 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ceb10dc4:::benchmark_data_smithi022_188626_object119263:head# 2024-09-11T07:50:25.766 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.766 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ceb2e4a2:::benchmark_data_smithi022_188626_object93626:head# 2024-09-11T07:50:25.767 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.767 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ceb743ad:::benchmark_data_smithi022_188626_object98737:head# 2024-09-11T07:50:25.767 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.767 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ceb92659:::benchmark_data_smithi022_188626_object112238:head# 2024-09-11T07:50:25.767 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.767 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ceba2ba7:::benchmark_data_smithi022_188626_object111159:head# 2024-09-11T07:50:25.767 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.767 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cebdc92c:::benchmark_data_smithi022_188626_object102412:head# 2024-09-11T07:50:25.767 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.767 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cebe40a3:::benchmark_data_smithi022_188626_object102221:head# 2024-09-11T07:50:25.767 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.767 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cec40296:::benchmark_data_smithi022_188626_object107340:head# 2024-09-11T07:50:25.767 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.768 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cec47a54:::benchmark_data_smithi022_188626_object107010:head# 2024-09-11T07:50:25.768 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.768 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cec7e9f8:::benchmark_data_smithi022_188626_object106505:head# 2024-09-11T07:50:25.768 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.768 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cec8851d:::benchmark_data_smithi022_188626_object100192:head# 2024-09-11T07:50:25.768 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.768 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cec8f119:::benchmark_data_smithi022_188626_object99253:head# 2024-09-11T07:50:25.768 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.768 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cec931c2:::benchmark_data_smithi022_188626_object104620:head# 2024-09-11T07:50:25.768 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.768 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cec961c1:::benchmark_data_smithi022_188626_object112611:head# 2024-09-11T07:50:25.768 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.768 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cececa25:::benchmark_data_smithi022_188626_object98604:head# 2024-09-11T07:50:25.768 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.769 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cecff4c7:::benchmark_data_smithi022_188626_object104911:head# 2024-09-11T07:50:25.769 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.769 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ced15506:::benchmark_data_smithi022_188626_object117313:head# 2024-09-11T07:50:25.769 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.769 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ced3b054:::benchmark_data_smithi022_188626_object102433:head# 2024-09-11T07:50:25.769 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.769 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ced4144e:::benchmark_data_smithi022_188626_object104026:head# 2024-09-11T07:50:25.769 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.769 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ced4f2ae:::benchmark_data_smithi022_188626_object109207:head# 2024-09-11T07:50:25.769 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.769 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ced6be38:::benchmark_data_smithi022_188626_object93452:head# 2024-09-11T07:50:25.769 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.769 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ced6e7ce:::benchmark_data_smithi022_188626_object98203:head# 2024-09-11T07:50:25.770 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.770 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:ced96dd0:::benchmark_data_smithi022_188626_object110374:head# 2024-09-11T07:50:25.770 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.770 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cedb0a1d:::benchmark_data_smithi022_188626_object106360:head# 2024-09-11T07:50:25.770 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.770 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cedb7f25:::benchmark_data_smithi022_188626_object99962:head# 2024-09-11T07:50:25.770 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.770 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cee1f939:::benchmark_data_smithi022_188626_object90691:head# 2024-09-11T07:50:25.770 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.770 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cee24603:::benchmark_data_smithi022_188626_object116995:head# 2024-09-11T07:50:25.770 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.770 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cee86ef5:::benchmark_data_smithi022_188626_object101908:head# 2024-09-11T07:50:25.770 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.770 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cee9f797:::benchmark_data_smithi022_188626_object100388:head# 2024-09-11T07:50:25.771 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.771 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cef5712c:::benchmark_data_smithi022_188626_object93461:head# 2024-09-11T07:50:25.771 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.771 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cef5bf0a:::benchmark_data_smithi022_188626_object118599:head# 2024-09-11T07:50:25.771 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.771 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cef8d1e9:::benchmark_data_smithi022_188626_object109613:head# 2024-09-11T07:50:25.772 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.772 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cefacc1b:::benchmark_data_smithi022_188626_object119125:head# 2024-09-11T07:50:25.772 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.772 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cefb1f8e:::benchmark_data_smithi022_188626_object89022:head# 2024-09-11T07:50:25.772 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.772 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cefe3114:::benchmark_data_smithi022_188626_object115470:head# 2024-09-11T07:50:25.773 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.773 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf05ed34:::benchmark_data_smithi022_188626_object97662:head# 2024-09-11T07:50:25.773 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.773 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf0b460f:::benchmark_data_smithi022_188626_object89242:head# 2024-09-11T07:50:25.773 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.773 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf0b6a00:::benchmark_data_smithi022_188626_object119316:head# 2024-09-11T07:50:25.773 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.774 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf0eb43e:::benchmark_data_smithi022_188626_object101109:head# 2024-09-11T07:50:25.774 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.774 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf0f09c3:::benchmark_data_smithi022_188626_object105357:head# 2024-09-11T07:50:25.774 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.774 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf124800:::benchmark_data_smithi022_188626_object107967:head# 2024-09-11T07:50:25.774 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.775 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf144a70:::benchmark_data_smithi022_188626_object111244:head# 2024-09-11T07:50:25.775 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.775 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf155394:::benchmark_data_smithi022_188626_object119274:head# 2024-09-11T07:50:25.775 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.775 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf190608:::benchmark_data_smithi022_188626_object112613:head# 2024-09-11T07:50:25.775 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.775 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf19b8d3:::benchmark_data_smithi022_188626_object93371:head# 2024-09-11T07:50:25.775 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.776 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf22c79d:::benchmark_data_smithi022_188626_object91298:head# 2024-09-11T07:50:25.776 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.776 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf235a4f:::benchmark_data_smithi022_188626_object89165:head# 2024-09-11T07:50:25.776 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.776 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf23bcdc:::benchmark_data_smithi022_188626_object112585:head# 2024-09-11T07:50:25.776 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.777 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf266676:::benchmark_data_smithi022_188626_object105215:head# 2024-09-11T07:50:25.777 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.777 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf2725a2:::benchmark_data_smithi022_188626_object117976:head# 2024-09-11T07:50:25.777 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.777 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf277639:::benchmark_data_smithi022_188626_object112759:head# 2024-09-11T07:50:25.777 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.777 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf2c3f06:::benchmark_data_smithi022_188626_object92601:head# 2024-09-11T07:50:25.778 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.778 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf2d48d4:::benchmark_data_smithi022_188626_object107333:head# 2024-09-11T07:50:25.778 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.778 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf2ea8c9:::benchmark_data_smithi022_188626_object105846:head# 2024-09-11T07:50:25.778 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.779 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf336efb:::benchmark_data_smithi022_188626_object117409:head# 2024-09-11T07:50:25.779 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.779 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf346651:::benchmark_data_smithi022_188626_object103761:head# 2024-09-11T07:50:25.779 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.779 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf36c389:::benchmark_data_smithi022_188626_object110845:head# 2024-09-11T07:50:25.779 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.779 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf381fbb:::benchmark_data_smithi022_188626_object118671:head# 2024-09-11T07:50:25.779 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.779 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf3ac558:::benchmark_data_smithi022_188626_object119790:head# 2024-09-11T07:50:25.779 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.780 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf3ada04:::benchmark_data_smithi022_188626_object103354:head# 2024-09-11T07:50:25.780 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.780 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf3c4b80:::benchmark_data_smithi022_188626_object108364:head# 2024-09-11T07:50:25.780 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.780 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf3e906c:::benchmark_data_smithi022_188626_object114343:head# 2024-09-11T07:50:25.780 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.780 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf40641a:::benchmark_data_smithi022_188626_object106055:head# 2024-09-11T07:50:25.780 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.780 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf408f72:::benchmark_data_smithi022_188626_object111504:head# 2024-09-11T07:50:25.780 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.780 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf46b90a:::benchmark_data_smithi022_188626_object93015:head# 2024-09-11T07:50:25.780 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.780 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf478341:::benchmark_data_smithi022_188626_object92104:head# 2024-09-11T07:50:25.780 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.781 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf48644b:::benchmark_data_smithi022_188626_object93434:head# 2024-09-11T07:50:25.781 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.781 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf4a4601:::benchmark_data_smithi022_188626_object112452:head# 2024-09-11T07:50:25.781 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.781 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf4a525b:::benchmark_data_smithi022_188626_object100189:head# 2024-09-11T07:50:25.781 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.781 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf4e8f16:::benchmark_data_smithi022_188626_object92829:head# 2024-09-11T07:50:25.781 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.781 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf503781:::benchmark_data_smithi022_188626_object97124:head# 2024-09-11T07:50:25.781 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.781 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf50fa46:::benchmark_data_smithi022_188626_object93153:head# 2024-09-11T07:50:25.781 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.782 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf535599:::benchmark_data_smithi022_188626_object101304:head# 2024-09-11T07:50:25.782 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.782 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf547a32:::benchmark_data_smithi022_188626_object111215:head# 2024-09-11T07:50:25.782 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.782 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf597932:::benchmark_data_smithi022_188626_object99885:head# 2024-09-11T07:50:25.782 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.782 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf59a206:::benchmark_data_smithi022_188626_object90446:head# 2024-09-11T07:50:25.782 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.782 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf5b0e94:::benchmark_data_smithi022_188626_object108352:head# 2024-09-11T07:50:25.782 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.782 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf641b08:::benchmark_data_smithi022_188626_object95200:head# 2024-09-11T07:50:25.782 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.782 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf667325:::benchmark_data_smithi022_188626_object99820:head# 2024-09-11T07:50:25.782 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.783 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf689b3b:::benchmark_data_smithi022_188626_object118159:head# 2024-09-11T07:50:25.783 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.783 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf68bbf8:::benchmark_data_smithi022_188626_object107941:head# 2024-09-11T07:50:25.783 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.783 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf6a96a2:::benchmark_data_smithi022_188626_object101997:head# 2024-09-11T07:50:25.783 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.783 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf6b596d:::benchmark_data_smithi022_188626_object103829:head# 2024-09-11T07:50:25.783 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.783 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf7362d1:::benchmark_data_smithi022_188626_object118279:head# 2024-09-11T07:50:25.783 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.783 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf73fc60:::benchmark_data_smithi022_188626_object112146:head# 2024-09-11T07:50:25.783 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.783 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf766035:::benchmark_data_smithi022_188626_object98800:head# 2024-09-11T07:50:25.783 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.784 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf771104:::benchmark_data_smithi022_188626_object106258:head# 2024-09-11T07:50:25.784 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.784 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf7844c3:::benchmark_data_smithi022_188626_object110931:head# 2024-09-11T07:50:25.784 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.784 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf7deac9:::benchmark_data_smithi022_188626_object116314:head# 2024-09-11T07:50:25.784 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.784 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf85791a:::benchmark_data_smithi022_188626_object98559:head# 2024-09-11T07:50:25.784 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.784 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf872e19:::benchmark_data_smithi022_188626_object91454:head# 2024-09-11T07:50:25.784 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.784 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf88fdcb:::benchmark_data_smithi022_188626_object109148:head# 2024-09-11T07:50:25.784 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.784 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf89d8dd:::benchmark_data_smithi022_188626_object96450:head# 2024-09-11T07:50:25.785 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.785 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf8a3f2b:::benchmark_data_smithi022_188626_object107184:head# 2024-09-11T07:50:25.785 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.785 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf8d39ce:::benchmark_data_smithi022_188626_object100626:head# 2024-09-11T07:50:25.785 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.785 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf90a39c:::benchmark_data_smithi022_188626_object110702:head# 2024-09-11T07:50:25.785 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.785 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf940aca:::benchmark_data_smithi022_188626_object112467:head# 2024-09-11T07:50:25.785 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.785 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf95bccd:::benchmark_data_smithi022_188626_object119702:head# 2024-09-11T07:50:25.785 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.785 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf991b0d:::benchmark_data_smithi022_188626_object106545:head# 2024-09-11T07:50:25.786 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.786 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf9c970b:::benchmark_data_smithi022_188626_object119361:head# 2024-09-11T07:50:25.786 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.786 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cf9ce539:::benchmark_data_smithi022_188626_object102112:head# 2024-09-11T07:50:25.786 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.786 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfa0fe6a:::benchmark_data_smithi022_188626_object103859:head# 2024-09-11T07:50:25.786 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.786 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfa13c59:::benchmark_data_smithi022_188626_object95925:head# 2024-09-11T07:50:25.786 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.786 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfa1944a:::benchmark_data_smithi022_188626_object112056:head# 2024-09-11T07:50:25.786 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.786 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfa233a6:::benchmark_data_smithi022_188626_object103223:head# 2024-09-11T07:50:25.787 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.787 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfa4460c:::benchmark_data_smithi022_188626_object91036:head# 2024-09-11T07:50:25.787 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.787 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfa5c9f6:::benchmark_data_smithi022_188626_object104330:head# 2024-09-11T07:50:25.787 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.787 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfaa132f:::benchmark_data_smithi022_188626_object104518:head# 2024-09-11T07:50:25.787 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.787 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfaa9cb8:::benchmark_data_smithi022_188626_object105794:head# 2024-09-11T07:50:25.787 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.787 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfac59e0:::benchmark_data_smithi022_188626_object90450:head# 2024-09-11T07:50:25.787 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.787 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfacf805:::benchmark_data_smithi022_188626_object112257:head# 2024-09-11T07:50:25.787 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.788 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfae5006:::benchmark_data_smithi022_188626_object115525:head# 2024-09-11T07:50:25.788 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.788 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfaee425:::benchmark_data_smithi022_188626_object109735:head# 2024-09-11T07:50:25.788 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.788 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfaf568f:::benchmark_data_smithi022_188626_object119797:head# 2024-09-11T07:50:25.788 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.788 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfb03d72:::benchmark_data_smithi022_188626_object95216:head# 2024-09-11T07:50:25.788 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.788 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfb04547:::benchmark_data_smithi022_188626_object96571:head# 2024-09-11T07:50:25.788 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.788 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfb06210:::benchmark_data_smithi022_188626_object117448:head# 2024-09-11T07:50:25.788 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.788 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfb31c60:::benchmark_data_smithi022_188626_object94957:head# 2024-09-11T07:50:25.789 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.789 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfb44e84:::benchmark_data_smithi022_188626_object110610:head# 2024-09-11T07:50:25.789 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.789 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfb461b7:::benchmark_data_smithi022_188626_object95038:head# 2024-09-11T07:50:25.789 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.789 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfb8c9a0:::benchmark_data_smithi022_188626_object106140:head# 2024-09-11T07:50:25.789 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.789 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfbd6316:::benchmark_data_smithi022_188626_object102659:head# 2024-09-11T07:50:25.789 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.789 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfbe78bc:::benchmark_data_smithi022_188626_object98712:head# 2024-09-11T07:50:25.789 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.789 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfc1811f:::benchmark_data_smithi022_188626_object99172:head# 2024-09-11T07:50:25.789 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.790 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfc1ff93:::benchmark_data_smithi022_188626_object102415:head# 2024-09-11T07:50:25.790 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.790 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfc3b20f:::benchmark_data_smithi022_188626_object90711:head# 2024-09-11T07:50:25.790 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.790 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfc3f63e:::benchmark_data_smithi022_188626_object95267:head# 2024-09-11T07:50:25.790 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.790 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfc496e0:::benchmark_data_smithi022_188626_object92357:head# 2024-09-11T07:50:25.790 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.790 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfc4cad1:::benchmark_data_smithi022_188626_object110368:head# 2024-09-11T07:50:25.790 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.790 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfc9b765:::benchmark_data_smithi022_188626_object103980:head# 2024-09-11T07:50:25.790 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.790 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfcad8e4:::benchmark_data_smithi022_188626_object119264:head# 2024-09-11T07:50:25.790 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.791 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfcc82df:::benchmark_data_smithi022_188626_object106160:head# 2024-09-11T07:50:25.791 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.791 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfce6882:::benchmark_data_smithi022_188626_object109812:head# 2024-09-11T07:50:25.791 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.791 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfd0562c:::benchmark_data_smithi022_188626_object116747:head# 2024-09-11T07:50:25.791 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.791 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfd0b12c:::benchmark_data_smithi022_188626_object116045:head# 2024-09-11T07:50:25.791 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.791 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfd0bac7:::benchmark_data_smithi022_188626_object111695:head# 2024-09-11T07:50:25.791 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.791 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfd0ec5c:::benchmark_data_smithi022_188626_object93775:head# 2024-09-11T07:50:25.792 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.792 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfd3bcb4:::benchmark_data_smithi022_188626_object101643:head# 2024-09-11T07:50:25.792 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.792 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfd74ef4:::benchmark_data_smithi022_188626_object117630:head# 2024-09-11T07:50:25.792 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.792 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfd7741e:::benchmark_data_smithi022_188626_object118271:head# 2024-09-11T07:50:25.792 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.792 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfd7deb8:::benchmark_data_smithi022_188626_object108601:head# 2024-09-11T07:50:25.792 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.792 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfd8f484:::benchmark_data_smithi022_188626_object113108:head# 2024-09-11T07:50:25.792 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.792 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfda2048:::benchmark_data_smithi022_188626_object115437:head# 2024-09-11T07:50:25.792 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.792 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfdbf485:::benchmark_data_smithi022_188626_object106614:head# 2024-09-11T07:50:25.793 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.793 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfdcbc57:::benchmark_data_smithi022_188626_object92362:head# 2024-09-11T07:50:25.793 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.793 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfdf2ef1:::benchmark_data_smithi022_188626_object117579:head# 2024-09-11T07:50:25.793 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.793 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfdf7317:::benchmark_data_smithi022_188626_object112158:head# 2024-09-11T07:50:25.793 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.793 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfe1571f:::benchmark_data_smithi022_188626_object91795:head# 2024-09-11T07:50:25.793 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.793 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfe51d7d:::benchmark_data_smithi022_188626_object99557:head# 2024-09-11T07:50:25.793 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.793 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfe5b2ea:::benchmark_data_smithi022_188626_object100685:head# 2024-09-11T07:50:25.793 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.794 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfe7fcf6:::benchmark_data_smithi022_188626_object103779:head# 2024-09-11T07:50:25.794 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.794 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfe9a88b:::benchmark_data_smithi022_188626_object116412:head# 2024-09-11T07:50:25.794 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.794 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfeaa491:::benchmark_data_smithi022_188626_object93858:head# 2024-09-11T07:50:25.794 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.794 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfeb6803:::benchmark_data_smithi022_188626_object96805:head# 2024-09-11T07:50:25.794 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.794 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfedbc43:::benchmark_data_smithi022_188626_object103355:head# 2024-09-11T07:50:25.794 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.794 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfedd98c:::benchmark_data_smithi022_188626_object99048:head# 2024-09-11T07:50:25.794 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.794 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfee3708:::benchmark_data_smithi022_188626_object110772:head# 2024-09-11T07:50:25.794 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.795 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfef88b2:::benchmark_data_smithi022_188626_object96832:head# 2024-09-11T07:50:25.795 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.795 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cff0fbab:::benchmark_data_smithi022_188626_object119463:head# 2024-09-11T07:50:25.795 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.795 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cff30ae6:::benchmark_data_smithi022_188626_object113433:head# 2024-09-11T07:50:25.795 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.795 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cff33c13:::benchmark_data_smithi022_188626_object112252:head# 2024-09-11T07:50:25.795 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.795 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cff37791:::benchmark_data_smithi022_188626_object96093:head# 2024-09-11T07:50:25.795 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.795 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cffb4fd3:::benchmark_data_smithi022_188626_object100521:head# 2024-09-11T07:50:25.795 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.795 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cffb6e62:::benchmark_data_smithi022_188626_object107440:head# 2024-09-11T07:50:25.795 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.796 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cffe624e:::benchmark_data_smithi022_188626_object99390:head# 2024-09-11T07:50:25.796 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.796 INFO:teuthology.orchestra.run.smithi022.stdout:Write #11:cfff0682:::benchmark_data_smithi022_188626_object93337:head# 2024-09-11T07:50:25.796 INFO:teuthology.orchestra.run.smithi022.stdout:snapset 0=[]:{} 2024-09-11T07:50:25.796 INFO:teuthology.orchestra.run.smithi022.stdout:write_pg epoch 461 info 11.3( v 459'12990 (441'8100,459'12990] lb MIN local-lis/les=454/455 n=1894 ec=412/412 lis/c=454/454 les/c/f=455/459/0 sis=460) 2024-09-11T07:50:25.796 INFO:teuthology.orchestra.run.smithi022.stdout:Import successful 2024-09-11T07:50:25.796 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:50:25.797 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:25 smithi104 ceph-mon[33507]: 1.0 scrub starts 2024-09-11T07:50:25.797 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:25 smithi104 ceph-mon[33507]: 1.0 scrub ok 2024-09-11T07:50:25.797 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:25 smithi104 ceph-mon[33507]: 4.a scrub starts 2024-09-11T07:50:25.797 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:25 smithi104 ceph-mon[33507]: 4.a scrub ok 2024-09-11T07:50:26.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:26 smithi022 ceph-mon[29462]: 4.9 scrub starts 2024-09-11T07:50:26.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:26 smithi022 ceph-mon[29462]: 4.9 scrub ok 2024-09-11T07:50:26.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:26 smithi022 ceph-mon[29462]: pgmap v1112: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:50:26.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:26 smithi022 ceph-mon[37330]: 4.9 scrub starts 2024-09-11T07:50:26.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:26 smithi022 ceph-mon[37330]: 4.9 scrub ok 2024-09-11T07:50:26.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:26 smithi022 ceph-mon[37330]: pgmap v1112: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:50:26.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:26 smithi104 ceph-mon[33507]: 4.9 scrub starts 2024-09-11T07:50:26.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:26 smithi104 ceph-mon[33507]: 4.9 scrub ok 2024-09-11T07:50:26.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:26 smithi104 ceph-mon[33507]: pgmap v1112: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:50:27.066 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/exp.11.3.2 2024-09-11T07:50:27.661 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:27 smithi022 ceph-mon[29462]: 6.10 scrub starts 2024-09-11T07:50:27.661 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:27 smithi022 ceph-mon[29462]: 6.10 scrub ok 2024-09-11T07:50:27.661 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:27 smithi022 ceph-mon[37330]: 6.10 scrub starts 2024-09-11T07:50:27.661 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:27 smithi022 ceph-mon[37330]: 6.10 scrub ok 2024-09-11T07:50:27.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:27 smithi104 ceph-mon[33507]: 6.10 scrub starts 2024-09-11T07:50:27.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:27 smithi104 ceph-mon[33507]: 6.10 scrub ok 2024-09-11T07:50:28.025 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:50:28.025 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:50:28.025 INFO:teuthology.orchestra.run.smithi022.stdout: "num_ops": 0 2024-09-11T07:50:28.025 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:50:28.666 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:28 smithi022 ceph-mon[37330]: pgmap v1113: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:50:28.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:28 smithi022 ceph-mon[29462]: pgmap v1113: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:50:28.753 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-11T07:50:28.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:28 smithi104 ceph-mon[33507]: pgmap v1113: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:50:29.010 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:50:29.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:29 smithi022 ceph-mon[29462]: 4.1 deep-scrub starts 2024-09-11T07:50:29.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:29 smithi022 ceph-mon[29462]: 4.1 deep-scrub ok 2024-09-11T07:50:29.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:29 smithi022 ceph-mon[37330]: 4.1 deep-scrub starts 2024-09-11T07:50:29.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:29 smithi022 ceph-mon[37330]: 4.1 deep-scrub ok 2024-09-11T07:50:29.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:29 smithi104 ceph-mon[33507]: 4.1 deep-scrub starts 2024-09-11T07:50:29.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:29 smithi104 ceph-mon[33507]: 4.1 deep-scrub ok 2024-09-11T07:50:30.611 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:50:30 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:50:30] "GET /metrics HTTP/1.1" 200 207461 "" "Prometheus/2.43.0" 2024-09-11T07:50:30.612 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:30 smithi022 ceph-mon[29462]: 6.0 deep-scrub starts 2024-09-11T07:50:30.612 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:30 smithi022 ceph-mon[29462]: 6.0 deep-scrub ok 2024-09-11T07:50:30.612 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:30 smithi022 ceph-mon[29462]: pgmap v1114: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:50:30.612 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:30 smithi022 ceph-mon[37330]: 6.0 deep-scrub starts 2024-09-11T07:50:30.612 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:30 smithi022 ceph-mon[37330]: 6.0 deep-scrub ok 2024-09-11T07:50:30.613 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:30 smithi022 ceph-mon[37330]: pgmap v1114: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:50:30.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:50:30 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:50:30.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:30 smithi104 ceph-mon[33507]: 6.0 deep-scrub starts 2024-09-11T07:50:30.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:30 smithi104 ceph-mon[33507]: 6.0 deep-scrub ok 2024-09-11T07:50:30.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:30 smithi104 ceph-mon[33507]: pgmap v1114: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:50:30.946 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:50:30.946 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:50:30.946 INFO:teuthology.orchestra.run.smithi022.stdout: "complaint_time": 30, 2024-09-11T07:50:30.946 INFO:teuthology.orchestra.run.smithi022.stdout: "num_blocked_ops": 0 2024-09-11T07:50:30.946 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:50:31.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:31 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:50:31.666 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:31 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:50:31.711 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-11T07:50:31.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:31 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:50:31.968 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:50:32.110 INFO:tasks.thrashosds.thrasher:reviving osd 2024-09-11T07:50:32.110 INFO:tasks.thrashosds.thrasher:Reviving osd 2 2024-09-11T07:50:32.110 INFO:tasks.cephadm.osd.2:Restarting osd.2 (starting--it wasn't running)... 2024-09-11T07:50:32.111 DEBUG:teuthology.orchestra.run.smithi022:osd.2> sudo journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.2.service 2024-09-11T07:50:32.113 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl start ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.2 2024-09-11T07:50:32.346 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:32 smithi022 systemd[1]: Starting Ceph osd.2 for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:50:32.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:32 smithi022 ceph-mon[29462]: 2.7 scrub starts 2024-09-11T07:50:32.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:32 smithi022 ceph-mon[29462]: 2.7 scrub ok 2024-09-11T07:50:32.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:32 smithi022 ceph-mon[29462]: 6.18 scrub starts 2024-09-11T07:50:32.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:32 smithi022 ceph-mon[29462]: 6.18 scrub ok 2024-09-11T07:50:32.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:32 smithi022 ceph-mon[29462]: pgmap v1115: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 995 B/s rd, 0 op/s 2024-09-11T07:50:32.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:32 smithi022 ceph-mon[37330]: 2.7 scrub starts 2024-09-11T07:50:32.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:32 smithi022 ceph-mon[37330]: 2.7 scrub ok 2024-09-11T07:50:32.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:32 smithi022 ceph-mon[37330]: 6.18 scrub starts 2024-09-11T07:50:32.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:32 smithi022 ceph-mon[37330]: 6.18 scrub ok 2024-09-11T07:50:32.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:32 smithi022 ceph-mon[37330]: pgmap v1115: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 995 B/s rd, 0 op/s 2024-09-11T07:50:32.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:32 smithi104 ceph-mon[33507]: 2.7 scrub starts 2024-09-11T07:50:32.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:32 smithi104 ceph-mon[33507]: 2.7 scrub ok 2024-09-11T07:50:32.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:32 smithi104 ceph-mon[33507]: 6.18 scrub starts 2024-09-11T07:50:32.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:32 smithi104 ceph-mon[33507]: 6.18 scrub ok 2024-09-11T07:50:32.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:32 smithi104 ceph-mon[33507]: pgmap v1115: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 995 B/s rd, 0 op/s 2024-09-11T07:50:33.569 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:50:33.569 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 20, 2024-09-11T07:50:33.569 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 600, 2024-09-11T07:50:33.569 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:50:33.569 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.569 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:213692 9.18 9:1bf9b530:::benchmark_data_smithi022_133943_object87101:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:50:33.569 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:38.520588+0000", 2024-09-11T07:50:33.569 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 595.03491139300002, 2024-09-11T07:50:33.569 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1203804499999999, 2024-09-11T07:50:33.569 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:33.570 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:50:33.570 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:33.570 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:33.570 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:33.570 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213692 2024-09-11T07:50:33.570 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.570 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:33.570 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.570 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:33.570 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.520588+0000", 2024-09-11T07:50:33.570 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.570 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.570 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.570 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:33.571 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.520588+0000", 2024-09-11T07:50:33.571 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.571 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.571 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.571 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:33.571 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.520586+0000", 2024-09-11T07:50:33.571 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:50:33.571 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.571 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.571 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:33.571 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.520594+0000", 2024-09-11T07:50:33.571 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.9899999999999997e-06 2024-09-11T07:50:33.571 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.571 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.571 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:33.572 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.520605+0000", 2024-09-11T07:50:33.572 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0292000000000001e-05 2024-09-11T07:50:33.572 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.572 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.572 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:33.572 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.520610+0000", 2024-09-11T07:50:33.572 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.3700000000000003e-06 2024-09-11T07:50:33.572 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.572 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.572 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.572 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.523617+0000", 2024-09-11T07:50:33.572 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0030071709999999999 2024-09-11T07:50:33.572 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.572 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.573 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op must wait for map", 2024-09-11T07:50:33.573 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.523623+0000", 2024-09-11T07:50:33.573 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3690000000000003e-06 2024-09-11T07:50:33.573 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.573 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.573 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.573 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.526958+0000", 2024-09-11T07:50:33.573 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0033348179999999998 2024-09-11T07:50:33.573 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.573 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.573 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.573 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.526967+0000", 2024-09-11T07:50:33.573 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3000000000000002e-06 2024-09-11T07:50:33.574 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.574 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.574 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.574 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.515377+0000", 2024-09-11T07:50:33.574 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.98841051400000002 2024-09-11T07:50:33.574 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.574 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.574 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.574 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.515389+0000", 2024-09-11T07:50:33.574 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1469e-05 2024-09-11T07:50:33.574 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.574 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.574 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.574 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.640883+0000", 2024-09-11T07:50:33.574 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.125494005 2024-09-11T07:50:33.575 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.575 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.575 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:33.575 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.640969+0000", 2024-09-11T07:50:33.575 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.6353999999999993e-05 2024-09-11T07:50:33.575 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.575 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:33.575 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.575 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.575 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.575 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:213706 9.18 9:1b47dd8a:::benchmark_data_smithi022_133943_object87115:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:50:33.575 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:38.520616+0000", 2024-09-11T07:50:33.575 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 595.03488398499996, 2024-09-11T07:50:33.576 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.120655272, 2024-09-11T07:50:33.576 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:33.576 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:50:33.576 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:33.576 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:33.576 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:33.576 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213706 2024-09-11T07:50:33.576 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.576 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:33.576 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.576 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:33.576 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.520616+0000", 2024-09-11T07:50:33.576 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.576 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.577 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.577 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:33.577 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.520616+0000", 2024-09-11T07:50:33.577 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.577 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.577 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.577 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:33.577 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.520615+0000", 2024-09-11T07:50:33.577 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:50:33.577 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.577 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.577 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:33.577 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.520619+0000", 2024-09-11T07:50:33.577 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6320000000000001e-06 2024-09-11T07:50:33.578 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.578 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.578 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:33.578 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.520626+0000", 2024-09-11T07:50:33.578 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9670000000000004e-06 2024-09-11T07:50:33.578 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.578 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.578 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:33.578 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.520630+0000", 2024-09-11T07:50:33.578 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9530000000000001e-06 2024-09-11T07:50:33.578 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.578 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.578 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.578 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.523648+0000", 2024-09-11T07:50:33.578 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0030183860000000001 2024-09-11T07:50:33.579 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.579 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.579 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting_for_map not empty", 2024-09-11T07:50:33.579 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.523653+0000", 2024-09-11T07:50:33.579 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.6650000000000002e-06 2024-09-11T07:50:33.579 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.579 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.579 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.579 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.527883+0000", 2024-09-11T07:50:33.579 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0042298759999999996 2024-09-11T07:50:33.579 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.579 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.579 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.579 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.527891+0000", 2024-09-11T07:50:33.580 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.1370000000000002e-06 2024-09-11T07:50:33.580 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.580 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.580 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.580 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.515440+0000", 2024-09-11T07:50:33.580 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.98754916299999995 2024-09-11T07:50:33.580 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.580 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.580 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.580 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.515447+0000", 2024-09-11T07:50:33.580 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5020000000000003e-06 2024-09-11T07:50:33.580 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.580 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.580 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.581 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.641221+0000", 2024-09-11T07:50:33.581 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.12577382 2024-09-11T07:50:33.581 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.581 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.581 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:33.581 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.641271+0000", 2024-09-11T07:50:33.581 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9789000000000001e-05 2024-09-11T07:50:33.581 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.581 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:33.581 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.581 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.581 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.581 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:213711 9.18 9:1bd0a1c2:::benchmark_data_smithi022_133943_object87120:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:50:33.582 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:38.521665+0000", 2024-09-11T07:50:33.582 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 595.03383462199997, 2024-09-11T07:50:33.582 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.130706253, 2024-09-11T07:50:33.582 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:33.582 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:33.582 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:33.582 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:33.582 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:33.582 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213711 2024-09-11T07:50:33.582 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.582 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:33.582 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.582 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:33.582 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.521665+0000", 2024-09-11T07:50:33.582 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.583 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.583 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.583 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:33.583 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.521665+0000", 2024-09-11T07:50:33.583 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.583 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.583 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.583 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:33.583 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.521665+0000", 2024-09-11T07:50:33.583 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:50:33.583 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.583 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.583 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:33.583 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.521668+0000", 2024-09-11T07:50:33.583 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2629999999999999e-06 2024-09-11T07:50:33.584 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.584 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.584 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:33.584 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.521673+0000", 2024-09-11T07:50:33.584 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1939999999999997e-06 2024-09-11T07:50:33.584 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.584 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.584 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:33.584 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.521677+0000", 2024-09-11T07:50:33.584 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4939999999999999e-06 2024-09-11T07:50:33.584 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.584 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.584 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.584 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.523673+0000", 2024-09-11T07:50:33.585 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0019969559999999998 2024-09-11T07:50:33.585 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.585 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.585 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting_for_map not empty", 2024-09-11T07:50:33.585 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.523678+0000", 2024-09-11T07:50:33.585 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5430000000000004e-06 2024-09-11T07:50:33.585 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.585 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.585 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.585 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.527916+0000", 2024-09-11T07:50:33.585 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.004237663 2024-09-11T07:50:33.585 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.586 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.586 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.586 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.527922+0000", 2024-09-11T07:50:33.586 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7329999999999997e-06 2024-09-11T07:50:33.586 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.586 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.586 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.586 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.515500+0000", 2024-09-11T07:50:33.586 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.98757713400000002 2024-09-11T07:50:33.586 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.586 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.586 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.586 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.515509+0000", 2024-09-11T07:50:33.586 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.8910000000000007e-06 2024-09-11T07:50:33.587 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.587 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.587 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.587 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.641303+0000", 2024-09-11T07:50:33.587 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.125793807 2024-09-11T07:50:33.587 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.587 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.587 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:33.587 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.641401+0000", 2024-09-11T07:50:33.587 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.7643e-05 2024-09-11T07:50:33.587 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.587 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.587 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:50:33.587 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.641682+0000", 2024-09-11T07:50:33.588 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00028099700000000001 2024-09-11T07:50:33.588 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.588 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.588 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:33.588 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.649986+0000", 2024-09-11T07:50:33.588 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0083042949999999997 2024-09-11T07:50:33.588 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.588 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.588 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.588 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.650042+0000", 2024-09-11T07:50:33.588 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6295999999999998e-05 2024-09-11T07:50:33.588 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.588 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.588 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.589 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652201+0000", 2024-09-11T07:50:33.589 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0021582160000000001 2024-09-11T07:50:33.589 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.589 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.589 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.589 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652272+0000", 2024-09-11T07:50:33.589 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1070999999999998e-05 2024-09-11T07:50:33.589 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.589 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.589 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:33.589 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652304+0000", 2024-09-11T07:50:33.589 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2710999999999999e-05 2024-09-11T07:50:33.589 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.590 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.590 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:33.590 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652371+0000", 2024-09-11T07:50:33.590 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.6966999999999998e-05 2024-09-11T07:50:33.590 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.590 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:33.590 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.590 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.590 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.590 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:213773 9.18 9:19914c12:::benchmark_data_smithi022_133943_object87182:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:50:33.590 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:38.576545+0000", 2024-09-11T07:50:33.590 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 594.97895502300003, 2024-09-11T07:50:33.590 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0760134770000001, 2024-09-11T07:50:33.591 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:33.591 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:33.591 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:33.591 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:33.591 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:33.591 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213773 2024-09-11T07:50:33.591 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.591 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:33.591 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.591 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:33.591 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.576545+0000", 2024-09-11T07:50:33.591 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.591 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.591 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.592 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:33.592 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.576545+0000", 2024-09-11T07:50:33.592 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.592 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.592 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.592 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:33.592 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.576542+0000", 2024-09-11T07:50:33.592 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999967 2024-09-11T07:50:33.592 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.592 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.592 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:33.592 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.576552+0000", 2024-09-11T07:50:33.592 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0701e-05 2024-09-11T07:50:33.593 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.593 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.593 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:33.593 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.576573+0000", 2024-09-11T07:50:33.593 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0358999999999999e-05 2024-09-11T07:50:33.593 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.593 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.593 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:33.593 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.576580+0000", 2024-09-11T07:50:33.593 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.8070000000000002e-06 2024-09-11T07:50:33.593 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.593 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.593 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.594 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.576639+0000", 2024-09-11T07:50:33.594 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9517e-05 2024-09-11T07:50:33.594 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.594 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.594 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.594 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.576650+0000", 2024-09-11T07:50:33.594 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0750999999999999e-05 2024-09-11T07:50:33.594 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.594 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.594 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.594 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516367+0000", 2024-09-11T07:50:33.594 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.93971757 2024-09-11T07:50:33.594 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.594 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.594 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.595 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516376+0000", 2024-09-11T07:50:33.595 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3100000000000001e-06 2024-09-11T07:50:33.595 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.595 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.595 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.595 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.648174+0000", 2024-09-11T07:50:33.595 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1317987030000001 2024-09-11T07:50:33.595 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.595 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.595 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:33.595 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.648313+0000", 2024-09-11T07:50:33.595 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000138932 2024-09-11T07:50:33.595 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.595 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.596 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:50:33.596 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.648596+0000", 2024-09-11T07:50:33.596 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00028308700000000001 2024-09-11T07:50:33.596 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.596 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.596 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:33.596 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.651981+0000", 2024-09-11T07:50:33.596 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.003384827 2024-09-11T07:50:33.596 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.596 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.596 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.596 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652038+0000", 2024-09-11T07:50:33.596 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6925000000000001e-05 2024-09-11T07:50:33.597 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.597 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.597 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.597 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652418+0000", 2024-09-11T07:50:33.597 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00038032300000000001 2024-09-11T07:50:33.597 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.597 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.597 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.597 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652463+0000", 2024-09-11T07:50:33.597 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4298000000000002e-05 2024-09-11T07:50:33.597 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.597 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.598 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:33.598 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652489+0000", 2024-09-11T07:50:33.598 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.6219000000000001e-05 2024-09-11T07:50:33.598 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.598 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.598 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:33.598 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652558+0000", 2024-09-11T07:50:33.598 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9294000000000004e-05 2024-09-11T07:50:33.598 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.598 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:33.598 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.598 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.598 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.598 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:213812 9.18 9:190c7e59:::benchmark_data_smithi022_133943_object87221:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:50:33.598 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:38.613884+0000", 2024-09-11T07:50:33.599 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 594.94161623399998, 2024-09-11T07:50:33.599 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0389224910000001, 2024-09-11T07:50:33.599 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:33.599 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:33.599 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:33.599 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:33.599 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:33.599 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213812 2024-09-11T07:50:33.599 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.599 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:33.599 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.599 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:33.599 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.613884+0000", 2024-09-11T07:50:33.599 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.600 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.600 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.600 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:33.600 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.613884+0000", 2024-09-11T07:50:33.600 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.600 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.600 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.600 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:33.600 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.613881+0000", 2024-09-11T07:50:33.600 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:50:33.600 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.600 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.600 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:33.601 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.613891+0000", 2024-09-11T07:50:33.601 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.8889999999999997e-06 2024-09-11T07:50:33.601 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.601 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.601 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:33.601 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.613908+0000", 2024-09-11T07:50:33.601 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.7595e-05 2024-09-11T07:50:33.601 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.601 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.601 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:33.601 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.613916+0000", 2024-09-11T07:50:33.601 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.824e-06 2024-09-11T07:50:33.601 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.602 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.602 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.602 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.613975+0000", 2024-09-11T07:50:33.602 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9441999999999998e-05 2024-09-11T07:50:33.602 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.602 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.602 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.602 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.613986+0000", 2024-09-11T07:50:33.602 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0107e-05 2024-09-11T07:50:33.602 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.602 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.602 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.602 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516401+0000", 2024-09-11T07:50:33.602 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.90241526999999999 2024-09-11T07:50:33.603 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.603 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.603 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.603 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516407+0000", 2024-09-11T07:50:33.603 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5930000000000002e-06 2024-09-11T07:50:33.603 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.603 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.603 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.603 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.650082+0000", 2024-09-11T07:50:33.603 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1336741699999999 2024-09-11T07:50:33.603 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.603 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.603 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:33.603 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.650188+0000", 2024-09-11T07:50:33.603 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000106847 2024-09-11T07:50:33.604 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.604 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.604 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:50:33.604 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.650406+0000", 2024-09-11T07:50:33.604 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000217395 2024-09-11T07:50:33.604 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.604 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.604 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:33.604 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652381+0000", 2024-09-11T07:50:33.604 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0019755599999999999 2024-09-11T07:50:33.604 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.604 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.604 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.604 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652594+0000", 2024-09-11T07:50:33.605 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000212568 2024-09-11T07:50:33.605 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.605 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.605 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.605 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652626+0000", 2024-09-11T07:50:33.605 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.2514000000000001e-05 2024-09-11T07:50:33.605 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.605 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.605 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.605 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652672+0000", 2024-09-11T07:50:33.605 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.5481e-05 2024-09-11T07:50:33.605 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.605 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.605 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:33.606 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652700+0000", 2024-09-11T07:50:33.606 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.7827e-05 2024-09-11T07:50:33.606 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.606 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.606 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:33.606 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652806+0000", 2024-09-11T07:50:33.606 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000106327 2024-09-11T07:50:33.606 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.606 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:33.606 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.606 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.606 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.606 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:213830 9.18 9:1d562e34:::benchmark_data_smithi022_133943_object87239:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:50:33.606 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:38.628512+0000", 2024-09-11T07:50:33.607 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 594.92698796900004, 2024-09-11T07:50:33.607 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0332906839999998, 2024-09-11T07:50:33.607 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:33.607 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:33.607 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:33.607 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:33.607 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:33.607 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213830 2024-09-11T07:50:33.607 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.607 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:33.607 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.607 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:33.607 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.628512+0000", 2024-09-11T07:50:33.607 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.608 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.608 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.608 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:33.608 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.628512+0000", 2024-09-11T07:50:33.608 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.608 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.608 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.608 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:33.608 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.628510+0000", 2024-09-11T07:50:33.608 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:50:33.608 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.608 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.608 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:33.608 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.628517+0000", 2024-09-11T07:50:33.609 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.588e-06 2024-09-11T07:50:33.609 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.609 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.609 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:33.609 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.628529+0000", 2024-09-11T07:50:33.609 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1737e-05 2024-09-11T07:50:33.609 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.609 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.609 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:33.609 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.628535+0000", 2024-09-11T07:50:33.609 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.5860000000000004e-06 2024-09-11T07:50:33.609 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.609 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.609 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.609 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.628585+0000", 2024-09-11T07:50:33.610 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.0519000000000002e-05 2024-09-11T07:50:33.610 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.610 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.610 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.610 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.628594+0000", 2024-09-11T07:50:33.610 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.7199999999999995e-06 2024-09-11T07:50:33.610 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.610 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.610 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.610 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516430+0000", 2024-09-11T07:50:33.610 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.88783586599999997 2024-09-11T07:50:33.610 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.610 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.610 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.611 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516437+0000", 2024-09-11T07:50:33.611 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9280000000000004e-06 2024-09-11T07:50:33.611 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.611 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.611 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.611 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652844+0000", 2024-09-11T07:50:33.611 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.136407495 2024-09-11T07:50:33.611 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.611 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.611 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:33.611 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.652953+0000", 2024-09-11T07:50:33.611 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010859 2024-09-11T07:50:33.611 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.611 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.611 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:50:33.612 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.653209+0000", 2024-09-11T07:50:33.612 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00025661 2024-09-11T07:50:33.612 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.612 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.612 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.612 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.657770+0000", 2024-09-11T07:50:33.612 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0045608970000000004 2024-09-11T07:50:33.612 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.612 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.612 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:33.612 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.657841+0000", 2024-09-11T07:50:33.612 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0417e-05 2024-09-11T07:50:33.612 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.612 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.612 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.613 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.657898+0000", 2024-09-11T07:50:33.613 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.7083000000000002e-05 2024-09-11T07:50:33.613 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.613 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.613 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.613 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.661631+0000", 2024-09-11T07:50:33.613 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0037329630000000002 2024-09-11T07:50:33.613 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.613 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.613 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:33.613 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.661665+0000", 2024-09-11T07:50:33.613 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4257000000000001e-05 2024-09-11T07:50:33.613 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.613 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.613 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:33.614 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.661803+0000", 2024-09-11T07:50:33.614 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000137637 2024-09-11T07:50:33.614 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.614 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:33.614 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.614 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.614 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.614 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:213855 9.18 9:1c5e2bfb:::benchmark_data_smithi022_133943_object87264:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:50:33.614 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:38.650487+0000", 2024-09-11T07:50:33.614 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 594.90501292700003, 2024-09-11T07:50:33.614 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0237156920000001, 2024-09-11T07:50:33.614 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:33.614 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:33.615 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:33.615 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:33.615 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:33.615 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213855 2024-09-11T07:50:33.615 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.615 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:33.615 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.615 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:33.615 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.650487+0000", 2024-09-11T07:50:33.615 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.615 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.615 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.615 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:33.615 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.650487+0000", 2024-09-11T07:50:33.616 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.616 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.616 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.616 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:33.616 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.650485+0000", 2024-09-11T07:50:33.616 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:50:33.616 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.616 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.616 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:33.616 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.650492+0000", 2024-09-11T07:50:33.616 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9630000000000001e-06 2024-09-11T07:50:33.616 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.616 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.616 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:33.617 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.650505+0000", 2024-09-11T07:50:33.617 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3114e-05 2024-09-11T07:50:33.617 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.617 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.617 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:33.617 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.650511+0000", 2024-09-11T07:50:33.617 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1190000000000002e-06 2024-09-11T07:50:33.617 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.617 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.617 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.617 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.650568+0000", 2024-09-11T07:50:33.617 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6511999999999997e-05 2024-09-11T07:50:33.617 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.617 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.617 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.618 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.650578+0000", 2024-09-11T07:50:33.618 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0631e-05 2024-09-11T07:50:33.618 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.618 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.618 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.618 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516460+0000", 2024-09-11T07:50:33.618 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.86588189000000004 2024-09-11T07:50:33.618 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.618 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.618 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.618 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516467+0000", 2024-09-11T07:50:33.618 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.421e-06 2024-09-11T07:50:33.618 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.618 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.618 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.618 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.654928+0000", 2024-09-11T07:50:33.619 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1384612590000001 2024-09-11T07:50:33.619 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.619 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.619 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:33.619 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.655059+0000", 2024-09-11T07:50:33.619 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013165200000000001 2024-09-11T07:50:33.619 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.619 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.619 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:50:33.619 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.655322+0000", 2024-09-11T07:50:33.619 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00026282699999999998 2024-09-11T07:50:33.619 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.619 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.619 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.619 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.657828+0000", 2024-09-11T07:50:33.620 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0025059990000000001 2024-09-11T07:50:33.620 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.620 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.620 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:33.620 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.657850+0000", 2024-09-11T07:50:33.620 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1801e-05 2024-09-11T07:50:33.620 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.620 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.620 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.620 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.665730+0000", 2024-09-11T07:50:33.620 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0078795240000000006 2024-09-11T07:50:33.620 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.620 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.620 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.621 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.673993+0000", 2024-09-11T07:50:33.621 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0082633740000000004 2024-09-11T07:50:33.621 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.621 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.621 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:33.621 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.674059+0000", 2024-09-11T07:50:33.621 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.6138999999999998e-05 2024-09-11T07:50:33.621 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.621 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.621 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:33.621 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.674203+0000", 2024-09-11T07:50:33.621 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014348299999999999 2024-09-11T07:50:33.621 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.621 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:33.622 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.622 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.622 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.622 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:213989 9.18 9:19156234:::benchmark_data_smithi022_133943_object87398:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:50:33.622 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:38.779547+0000", 2024-09-11T07:50:33.622 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 594.77595318800002, 2024-09-11T07:50:33.622 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.894937614, 2024-09-11T07:50:33.622 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:33.622 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:33.623 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:33.623 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:33.623 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:33.623 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 213989 2024-09-11T07:50:33.623 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.623 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:33.623 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.623 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:33.623 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.779547+0000", 2024-09-11T07:50:33.623 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.623 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.623 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.624 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:33.624 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.779547+0000", 2024-09-11T07:50:33.624 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.624 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.624 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.624 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:33.624 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.779545+0000", 2024-09-11T07:50:33.624 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:50:33.624 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.625 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.625 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:33.625 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.779552+0000", 2024-09-11T07:50:33.625 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.6969999999999993e-06 2024-09-11T07:50:33.625 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.625 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.625 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:33.625 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.779596+0000", 2024-09-11T07:50:33.625 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.3779000000000002e-05 2024-09-11T07:50:33.625 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.625 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.625 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:33.625 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.779603+0000", 2024-09-11T07:50:33.626 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9800000000000001e-06 2024-09-11T07:50:33.626 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.626 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.626 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.626 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.779656+0000", 2024-09-11T07:50:33.626 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2692000000000003e-05 2024-09-11T07:50:33.626 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.626 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.626 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.626 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.779665+0000", 2024-09-11T07:50:33.626 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.0180000000000007e-06 2024-09-11T07:50:33.626 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.626 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.626 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.626 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516490+0000", 2024-09-11T07:50:33.627 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.736824967 2024-09-11T07:50:33.627 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.627 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.627 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.627 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516496+0000", 2024-09-11T07:50:33.627 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5050000000000004e-06 2024-09-11T07:50:33.627 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.627 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.627 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.627 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.656449+0000", 2024-09-11T07:50:33.627 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1399522580000001 2024-09-11T07:50:33.627 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.627 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.627 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:33.628 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.656542+0000", 2024-09-11T07:50:33.628 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.3465e-05 2024-09-11T07:50:33.628 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.628 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.628 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:50:33.628 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.656765+0000", 2024-09-11T07:50:33.628 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00022321900000000001 2024-09-11T07:50:33.628 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.628 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.628 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:33.628 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.659429+0000", 2024-09-11T07:50:33.628 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0026633440000000002 2024-09-11T07:50:33.628 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.628 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.628 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.629 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.661547+0000", 2024-09-11T07:50:33.629 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0021182459999999998 2024-09-11T07:50:33.629 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.629 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.629 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.629 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.665762+0000", 2024-09-11T07:50:33.629 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0042147069999999998 2024-09-11T07:50:33.629 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.629 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.629 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.629 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.674298+0000", 2024-09-11T07:50:33.629 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0085361289999999999 2024-09-11T07:50:33.629 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.629 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.629 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:33.630 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.674337+0000", 2024-09-11T07:50:33.630 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9708e-05 2024-09-11T07:50:33.630 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.630 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.630 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:33.630 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.674484+0000", 2024-09-11T07:50:33.630 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00014686500000000001 2024-09-11T07:50:33.630 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.630 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:33.630 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.630 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.630 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.630 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:214018 9.18 9:1a17d8cd:::benchmark_data_smithi022_133943_object87427:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:50:33.631 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:38.807290+0000", 2024-09-11T07:50:33.631 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 594.74820989800003, 2024-09-11T07:50:33.631 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.868875066, 2024-09-11T07:50:33.631 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:33.631 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:33.631 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:33.631 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:33.631 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:33.631 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 214018 2024-09-11T07:50:33.631 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.631 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:33.631 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.631 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:33.631 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.807290+0000", 2024-09-11T07:50:33.631 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.632 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.632 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.632 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:33.632 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.807290+0000", 2024-09-11T07:50:33.632 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.632 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.632 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.632 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:33.632 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.807288+0000", 2024-09-11T07:50:33.632 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:50:33.632 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.632 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.632 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:33.632 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.807296+0000", 2024-09-11T07:50:33.633 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.5150000000000006e-06 2024-09-11T07:50:33.633 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.633 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.633 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:33.633 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.807312+0000", 2024-09-11T07:50:33.633 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5974999999999999e-05 2024-09-11T07:50:33.633 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.633 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.633 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:33.633 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.807318+0000", 2024-09-11T07:50:33.633 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.1210000000000003e-06 2024-09-11T07:50:33.633 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.633 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.633 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.633 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.807371+0000", 2024-09-11T07:50:33.633 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2958999999999999e-05 2024-09-11T07:50:33.634 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.634 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.634 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.634 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.807380+0000", 2024-09-11T07:50:33.634 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.7059999999999993e-06 2024-09-11T07:50:33.634 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.634 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.634 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.634 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516519+0000", 2024-09-11T07:50:33.634 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.709139301 2024-09-11T07:50:33.634 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.634 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.634 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.634 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516526+0000", 2024-09-11T07:50:33.634 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4130000000000003e-06 2024-09-11T07:50:33.635 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.635 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.635 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.635 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.657921+0000", 2024-09-11T07:50:33.635 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.141395846 2024-09-11T07:50:33.635 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.635 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.635 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:33.635 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.657991+0000", 2024-09-11T07:50:33.635 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9283999999999996e-05 2024-09-11T07:50:33.635 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.635 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.635 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:50:33.635 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.658156+0000", 2024-09-11T07:50:33.635 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000165234 2024-09-11T07:50:33.636 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.636 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.636 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.636 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.661591+0000", 2024-09-11T07:50:33.636 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0034355660000000001 2024-09-11T07:50:33.636 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.636 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.636 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:33.636 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.661812+0000", 2024-09-11T07:50:33.636 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000220662 2024-09-11T07:50:33.636 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.636 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.636 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.636 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.666688+0000", 2024-09-11T07:50:33.637 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0048756729999999996 2024-09-11T07:50:33.637 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.637 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.637 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.637 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.675122+0000", 2024-09-11T07:50:33.637 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:33 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:50:33.637 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:33 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:50:33.637 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:33 smithi022 ceph-mon[29462]: 2.19 scrub starts 2024-09-11T07:50:33.637 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:33 smithi022 ceph-mon[29462]: 2.19 scrub ok 2024-09-11T07:50:33.637 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:33 smithi022 ceph-mon[29462]: osdmap e483: 8 total, 7 up, 5 in 2024-09-11T07:50:33.637 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:33 smithi022 ceph-mon[29462]: 1.1 scrub starts 2024-09-11T07:50:33.638 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:33 smithi022 ceph-mon[29462]: 1.1 scrub ok 2024-09-11T07:50:33.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:33 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:50:33.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:33 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:50:33.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:33 smithi022 ceph-mon[37330]: 2.19 scrub starts 2024-09-11T07:50:33.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:33 smithi022 ceph-mon[37330]: 2.19 scrub ok 2024-09-11T07:50:33.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:33 smithi022 ceph-mon[37330]: osdmap e483: 8 total, 7 up, 5 in 2024-09-11T07:50:33.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:33 smithi022 ceph-mon[37330]: 1.1 scrub starts 2024-09-11T07:50:33.638 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:33 smithi022 ceph-mon[37330]: 1.1 scrub ok 2024-09-11T07:50:33.639 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0084342429999999993 2024-09-11T07:50:33.639 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.639 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.640 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:33.640 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.675921+0000", 2024-09-11T07:50:33.640 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00079866599999999996 2024-09-11T07:50:33.640 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.640 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.640 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:33.640 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676165+0000", 2024-09-11T07:50:33.640 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00024431099999999999 2024-09-11T07:50:33.640 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.640 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:33.640 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.640 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.640 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.640 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:214026 9.18 9:1d1707cd:::benchmark_data_smithi022_133943_object87435:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:50:33.641 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:38.815450+0000", 2024-09-11T07:50:33.641 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 594.74004990399999, 2024-09-11T07:50:33.641 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8610872550000002, 2024-09-11T07:50:33.641 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:33.641 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:33.641 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:33.641 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:33.641 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:33.641 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 214026 2024-09-11T07:50:33.641 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.641 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:33.641 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.641 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:33.641 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.815450+0000", 2024-09-11T07:50:33.642 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.642 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.642 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.642 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:33.642 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.815450+0000", 2024-09-11T07:50:33.642 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.642 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.642 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.642 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:33.642 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.815448+0000", 2024-09-11T07:50:33.642 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:33.642 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.642 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.642 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:33.642 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.815455+0000", 2024-09-11T07:50:33.643 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4690000000000002e-06 2024-09-11T07:50:33.643 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.643 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.643 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:33.643 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.815462+0000", 2024-09-11T07:50:33.643 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0419999999999997e-06 2024-09-11T07:50:33.643 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.643 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.643 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:33.643 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.815466+0000", 2024-09-11T07:50:33.643 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8039999999999999e-06 2024-09-11T07:50:33.643 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.644 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.644 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.644 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.815510+0000", 2024-09-11T07:50:33.644 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4657999999999998e-05 2024-09-11T07:50:33.644 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.644 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.644 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.644 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.815519+0000", 2024-09-11T07:50:33.644 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.704e-06 2024-09-11T07:50:33.644 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.644 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.644 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.644 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516550+0000", 2024-09-11T07:50:33.644 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.70103092099999997 2024-09-11T07:50:33.645 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.645 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.645 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.645 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516556+0000", 2024-09-11T07:50:33.645 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4470000000000003e-06 2024-09-11T07:50:33.645 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.645 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.645 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.645 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.659474+0000", 2024-09-11T07:50:33.645 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1429177239999999 2024-09-11T07:50:33.645 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.645 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.645 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:33.645 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.659598+0000", 2024-09-11T07:50:33.646 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012443 2024-09-11T07:50:33.646 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.646 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.646 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:50:33.646 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.659846+0000", 2024-09-11T07:50:33.646 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000247259 2024-09-11T07:50:33.646 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.646 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.646 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.646 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.661864+0000", 2024-09-11T07:50:33.647 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002018527 2024-09-11T07:50:33.647 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.647 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.647 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:33.647 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.663437+0000", 2024-09-11T07:50:33.647 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0015723569999999999 2024-09-11T07:50:33.647 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.647 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.647 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.647 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.669061+0000", 2024-09-11T07:50:33.647 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.005624352 2024-09-11T07:50:33.647 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.647 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.647 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.648 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676315+0000", 2024-09-11T07:50:33.648 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0072540979999999996 2024-09-11T07:50:33.648 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.648 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.648 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:33.648 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676369+0000", 2024-09-11T07:50:33.648 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.4094000000000001e-05 2024-09-11T07:50:33.648 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.648 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.648 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:33.648 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676537+0000", 2024-09-11T07:50:33.648 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000167994 2024-09-11T07:50:33.648 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.648 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:33.649 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.649 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.649 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.649 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:214088 9.18 9:1b56ff78:::benchmark_data_smithi022_133943_object87497:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:50:33.649 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:38.875033+0000", 2024-09-11T07:50:33.649 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 594.68046662500001, 2024-09-11T07:50:33.649 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.8017833599999999, 2024-09-11T07:50:33.649 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:33.649 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:33.649 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:33.649 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:33.649 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:33.649 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 214088 2024-09-11T07:50:33.649 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.649 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:33.650 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.650 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:33.650 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.875033+0000", 2024-09-11T07:50:33.650 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.650 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.650 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.650 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:33.650 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.875033+0000", 2024-09-11T07:50:33.650 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.650 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.650 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.650 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:33.650 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.875031+0000", 2024-09-11T07:50:33.650 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:50:33.651 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.651 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.651 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:33.651 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.875039+0000", 2024-09-11T07:50:33.651 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1790000000000002e-06 2024-09-11T07:50:33.651 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.651 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.651 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:33.651 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.875051+0000", 2024-09-11T07:50:33.651 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2592000000000001e-05 2024-09-11T07:50:33.651 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.651 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.651 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:33.651 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.875058+0000", 2024-09-11T07:50:33.651 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7700000000000004e-06 2024-09-11T07:50:33.652 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.652 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.652 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.652 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.875107+0000", 2024-09-11T07:50:33.652 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9285e-05 2024-09-11T07:50:33.652 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.652 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.652 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.652 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.875114+0000", 2024-09-11T07:50:33.652 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2840000000000003e-06 2024-09-11T07:50:33.652 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.652 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.652 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.652 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516578+0000", 2024-09-11T07:50:33.653 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.64146455499999999 2024-09-11T07:50:33.653 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.653 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.653 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.653 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516585+0000", 2024-09-11T07:50:33.653 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7290000000000003e-06 2024-09-11T07:50:33.653 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.653 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.653 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.653 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.661910+0000", 2024-09-11T07:50:33.653 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1453256519999999 2024-09-11T07:50:33.653 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.653 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.653 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:33.654 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.662040+0000", 2024-09-11T07:50:33.654 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012906899999999999 2024-09-11T07:50:33.654 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.654 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.654 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:50:33.654 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.662277+0000", 2024-09-11T07:50:33.654 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00023776400000000001 2024-09-11T07:50:33.654 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.654 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.654 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.654 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.665805+0000", 2024-09-11T07:50:33.654 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0035281079999999999 2024-09-11T07:50:33.654 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.654 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.655 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:33.655 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.669022+0000", 2024-09-11T07:50:33.655 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0032164060000000002 2024-09-11T07:50:33.655 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.655 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.655 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.655 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.673957+0000", 2024-09-11T07:50:33.655 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0049350879999999998 2024-09-11T07:50:33.655 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.655 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.655 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.655 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676648+0000", 2024-09-11T07:50:33.655 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0026909709999999999 2024-09-11T07:50:33.655 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.656 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.656 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:33.656 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676696+0000", 2024-09-11T07:50:33.656 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8321e-05 2024-09-11T07:50:33.656 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.656 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.656 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:33.656 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676817+0000", 2024-09-11T07:50:33.656 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000120359 2024-09-11T07:50:33.656 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.656 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:33.656 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.656 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.656 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.657 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:214109 9.18 9:1f0f4b12:::benchmark_data_smithi022_133943_object87518:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:50:33.657 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:38.901408+0000", 2024-09-11T07:50:33.657 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 594.65409144299997, 2024-09-11T07:50:33.657 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.775759847, 2024-09-11T07:50:33.657 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:33.657 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:33.657 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:33.657 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:33.657 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:33.657 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 214109 2024-09-11T07:50:33.657 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.657 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:33.657 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.658 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:33.658 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.901408+0000", 2024-09-11T07:50:33.658 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.658 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.658 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.658 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:33.658 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.901408+0000", 2024-09-11T07:50:33.658 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.658 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.658 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.658 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:33.658 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.901406+0000", 2024-09-11T07:50:33.658 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:50:33.658 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.659 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.659 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:33.659 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.901414+0000", 2024-09-11T07:50:33.659 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.6860000000000006e-06 2024-09-11T07:50:33.659 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.659 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.659 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:33.659 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.901429+0000", 2024-09-11T07:50:33.659 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4842e-05 2024-09-11T07:50:33.659 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.659 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.659 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:33.659 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.901436+0000", 2024-09-11T07:50:33.659 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7789999999999998e-06 2024-09-11T07:50:33.660 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.660 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.660 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.660 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.901484+0000", 2024-09-11T07:50:33.660 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7979999999999998e-05 2024-09-11T07:50:33.660 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.660 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.660 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.660 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:38.901490+0000", 2024-09-11T07:50:33.660 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.139e-06 2024-09-11T07:50:33.660 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.660 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.660 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.661 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516608+0000", 2024-09-11T07:50:33.661 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.61511833400000004 2024-09-11T07:50:33.661 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.661 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.661 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.661 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516616+0000", 2024-09-11T07:50:33.661 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.5490000000000001e-06 2024-09-11T07:50:33.661 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.661 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.661 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.661 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.663475+0000", 2024-09-11T07:50:33.661 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1468589119999999 2024-09-11T07:50:33.661 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.661 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.662 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:33.662 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.663568+0000", 2024-09-11T07:50:33.662 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.3740999999999996e-05 2024-09-11T07:50:33.662 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.662 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.662 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:50:33.662 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.663812+0000", 2024-09-11T07:50:33.662 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00024389 2024-09-11T07:50:33.662 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.662 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.662 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:33.662 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.669028+0000", 2024-09-11T07:50:33.662 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0052160720000000004 2024-09-11T07:50:33.662 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.662 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.663 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.663 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.674240+0000", 2024-09-11T07:50:33.663 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0052113469999999999 2024-09-11T07:50:33.663 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.663 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.663 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.663 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.674270+0000", 2024-09-11T07:50:33.663 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.0156000000000001e-05 2024-09-11T07:50:33.663 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.663 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.663 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.663 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676998+0000", 2024-09-11T07:50:33.663 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0027284900000000001 2024-09-11T07:50:33.663 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.664 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.664 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:33.664 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.677045+0000", 2024-09-11T07:50:33.664 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7055999999999998e-05 2024-09-11T07:50:33.664 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.664 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.664 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:33.664 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.677168+0000", 2024-09-11T07:50:33.664 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012286400000000001 2024-09-11T07:50:33.664 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.664 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:33.664 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.664 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.664 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.665 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:214203 9.18 9:1e3a93c6:::benchmark_data_smithi022_133943_object87612:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:50:33.665 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:39.007710+0000", 2024-09-11T07:50:33.665 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 594.547789663, 2024-09-11T07:50:33.665 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.681745552, 2024-09-11T07:50:33.665 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:33.665 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:33.665 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:33.665 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:33.665 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:33.665 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 214203 2024-09-11T07:50:33.665 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.665 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:33.665 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.666 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:33.666 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.007710+0000", 2024-09-11T07:50:33.666 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.666 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.666 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.666 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:33.666 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.007710+0000", 2024-09-11T07:50:33.666 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.666 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.666 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.666 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:33.666 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.007708+0000", 2024-09-11T07:50:33.666 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:50:33.666 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.667 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.667 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:33.667 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.007715+0000", 2024-09-11T07:50:33.667 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1860000000000003e-06 2024-09-11T07:50:33.667 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.667 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.667 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:33.667 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.007739+0000", 2024-09-11T07:50:33.667 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.3646999999999999e-05 2024-09-11T07:50:33.667 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.667 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.667 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:33.667 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.007745+0000", 2024-09-11T07:50:33.667 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5270000000000004e-06 2024-09-11T07:50:33.668 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.668 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.668 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.668 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.007805+0000", 2024-09-11T07:50:33.668 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9833000000000001e-05 2024-09-11T07:50:33.668 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.668 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.668 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.668 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.007812+0000", 2024-09-11T07:50:33.668 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.5080000000000002e-06 2024-09-11T07:50:33.668 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.668 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.668 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.668 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516640+0000", 2024-09-11T07:50:33.669 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.50882864999999999 2024-09-11T07:50:33.669 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.669 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.669 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.669 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516647+0000", 2024-09-11T07:50:33.669 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3459999999999999e-06 2024-09-11T07:50:33.669 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.669 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.669 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.669 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.666712+0000", 2024-09-11T07:50:33.669 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.150065023 2024-09-11T07:50:33.669 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.669 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.669 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:33.670 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.666829+0000", 2024-09-11T07:50:33.670 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000117748 2024-09-11T07:50:33.670 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.670 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.670 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:50:33.670 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.667169+0000", 2024-09-11T07:50:33.670 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00033908499999999998 2024-09-11T07:50:33.670 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.670 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.670 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:33.670 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.673874+0000", 2024-09-11T07:50:33.670 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0067059199999999998 2024-09-11T07:50:33.670 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.670 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.671 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.671 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.675054+0000", 2024-09-11T07:50:33.671 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001179865 2024-09-11T07:50:33.671 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.671 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.671 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.671 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.675092+0000", 2024-09-11T07:50:33.671 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7827000000000002e-05 2024-09-11T07:50:33.671 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.671 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.671 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.671 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.689279+0000", 2024-09-11T07:50:33.671 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.014186772 2024-09-11T07:50:33.672 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.672 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.672 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:33.672 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.689331+0000", 2024-09-11T07:50:33.672 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2410999999999997e-05 2024-09-11T07:50:33.672 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.672 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.672 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:33.672 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.689456+0000", 2024-09-11T07:50:33.672 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00012438999999999999 2024-09-11T07:50:33.672 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.672 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:33.672 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.672 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.673 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.673 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:214212 9.18 9:1a025e7e:::benchmark_data_smithi022_133943_object87621:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:50:33.673 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:39.021035+0000", 2024-09-11T07:50:33.673 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 594.53446435299998, 2024-09-11T07:50:33.673 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.6686958949999999, 2024-09-11T07:50:33.673 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:33.673 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:33.673 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:33.673 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:33.673 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:33.673 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 214212 2024-09-11T07:50:33.673 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.673 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:33.674 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.674 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:33.674 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.021035+0000", 2024-09-11T07:50:33.674 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.674 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.674 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.674 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:33.674 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.021035+0000", 2024-09-11T07:50:33.676 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.676 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.676 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.676 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:33.676 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.021032+0000", 2024-09-11T07:50:33.676 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999967 2024-09-11T07:50:33.677 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.677 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.677 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:33.677 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.021043+0000", 2024-09-11T07:50:33.677 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0825999999999999e-05 2024-09-11T07:50:33.677 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.677 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.677 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:33.677 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.021064+0000", 2024-09-11T07:50:33.677 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0899000000000001e-05 2024-09-11T07:50:33.677 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.677 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.677 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:33.677 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.021070+0000", 2024-09-11T07:50:33.678 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2559999999999999e-06 2024-09-11T07:50:33.678 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.678 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.678 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.678 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.021141+0000", 2024-09-11T07:50:33.678 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0778000000000006e-05 2024-09-11T07:50:33.678 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.678 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.678 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.678 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.021154+0000", 2024-09-11T07:50:33.678 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3176000000000001e-05 2024-09-11T07:50:33.678 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.678 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.679 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.679 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516671+0000", 2024-09-11T07:50:33.679 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.49551719500000002 2024-09-11T07:50:33.679 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.679 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.679 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.679 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516678+0000", 2024-09-11T07:50:33.679 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.0740000000000004e-06 2024-09-11T07:50:33.679 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.679 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.679 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.679 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.669082+0000", 2024-09-11T07:50:33.679 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.152403734 2024-09-11T07:50:33.679 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.680 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.680 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:33.680 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.669197+0000", 2024-09-11T07:50:33.680 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000115133 2024-09-11T07:50:33.680 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.680 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.680 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:50:33.680 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.669451+0000", 2024-09-11T07:50:33.680 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00025389500000000002 2024-09-11T07:50:33.680 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.680 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.680 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.680 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676214+0000", 2024-09-11T07:50:33.680 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0067624369999999996 2024-09-11T07:50:33.681 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.681 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.681 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.681 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676265+0000", 2024-09-11T07:50:33.681 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1180999999999997e-05 2024-09-11T07:50:33.681 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.681 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.681 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:33.681 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676274+0000", 2024-09-11T07:50:33.681 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.5489999999999995e-06 2024-09-11T07:50:33.681 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.681 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.681 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.681 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.689519+0000", 2024-09-11T07:50:33.682 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.013244211000000001 2024-09-11T07:50:33.682 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.682 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.682 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:33.682 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.689599+0000", 2024-09-11T07:50:33.682 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.0069999999999997e-05 2024-09-11T07:50:33.682 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.682 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.682 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:33.682 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.689731+0000", 2024-09-11T07:50:33.682 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013274399999999999 2024-09-11T07:50:33.682 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.682 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:33.683 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.683 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.683 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.683 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:214216 9.18 9:182c3ab1:::benchmark_data_smithi022_133943_object87625:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:50:33.683 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:39.025544+0000", 2024-09-11T07:50:33.683 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 594.52995584999996, 2024-09-11T07:50:33.683 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.664438707, 2024-09-11T07:50:33.683 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:33.683 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:33.683 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:33.683 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:33.683 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:33.683 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 214216 2024-09-11T07:50:33.683 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.684 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:33.684 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.684 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:33.684 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.025544+0000", 2024-09-11T07:50:33.684 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.684 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.684 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.684 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:33.684 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.025544+0000", 2024-09-11T07:50:33.684 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.684 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.684 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.684 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:33.685 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.025542+0000", 2024-09-11T07:50:33.685 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:50:33.685 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.685 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.685 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:33.685 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.025551+0000", 2024-09-11T07:50:33.685 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.0429999999999996e-06 2024-09-11T07:50:33.685 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.685 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.685 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:33.685 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.025563+0000", 2024-09-11T07:50:33.685 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1949e-05 2024-09-11T07:50:33.685 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.686 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.686 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:33.686 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.025570+0000", 2024-09-11T07:50:33.686 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.7419999999999997e-06 2024-09-11T07:50:33.686 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.686 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.686 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.686 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.025627+0000", 2024-09-11T07:50:33.686 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6320999999999998e-05 2024-09-11T07:50:33.686 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.686 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.686 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.686 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.025639+0000", 2024-09-11T07:50:33.687 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2086000000000001e-05 2024-09-11T07:50:33.687 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.687 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.687 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.687 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516706+0000", 2024-09-11T07:50:33.687 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.49106744200000002 2024-09-11T07:50:33.687 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.687 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.687 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.687 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516713+0000", 2024-09-11T07:50:33.687 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.686e-06 2024-09-11T07:50:33.687 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.687 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.688 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.688 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.671438+0000", 2024-09-11T07:50:33.688 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1547249019999999 2024-09-11T07:50:33.688 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.688 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.688 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:33.688 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.671509+0000", 2024-09-11T07:50:33.688 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1224999999999997e-05 2024-09-11T07:50:33.688 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.688 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.688 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:50:33.688 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.671839+0000", 2024-09-11T07:50:33.688 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00032957899999999998 2024-09-11T07:50:33.688 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.689 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.689 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:33.689 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676280+0000", 2024-09-11T07:50:33.689 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0044416059999999999 2024-09-11T07:50:33.689 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.689 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.689 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.689 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676581+0000", 2024-09-11T07:50:33.689 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00030059999999999999 2024-09-11T07:50:33.689 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.689 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.689 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.689 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.676617+0000", 2024-09-11T07:50:33.689 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.6337000000000001e-05 2024-09-11T07:50:33.690 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.690 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.690 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.690 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.689767+0000", 2024-09-11T07:50:33.690 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.013149793999999999 2024-09-11T07:50:33.690 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.690 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.691 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:33.691 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.689851+0000", 2024-09-11T07:50:33.691 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3857999999999996e-05 2024-09-11T07:50:33.691 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.692 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.692 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:33.692 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.689983+0000", 2024-09-11T07:50:33.692 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013197799999999999 2024-09-11T07:50:33.692 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.692 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:33.692 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.692 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.692 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.692 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.15504.0:214246 9.18 9:1b25dba9:::benchmark_data_smithi022_133943_object87655:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e310)", 2024-09-11T07:50:33.692 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:40:39.125384+0000", 2024-09-11T07:50:33.692 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 594.43011611400004, 2024-09-11T07:50:33.692 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.5648045229999998, 2024-09-11T07:50:33.693 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:33.693 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:33.693 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:33.693 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.15504", 2024-09-11T07:50:33.693 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:33.693 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 214246 2024-09-11T07:50:33.693 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.693 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:33.693 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.693 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:33.693 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.125384+0000", 2024-09-11T07:50:33.693 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.693 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.694 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.694 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:33.694 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.125384+0000", 2024-09-11T07:50:33.694 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.694 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.694 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.694 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:33.694 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.125382+0000", 2024-09-11T07:50:33.694 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:50:33.694 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.694 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.694 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:33.694 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.125389+0000", 2024-09-11T07:50:33.695 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.2760000000000003e-06 2024-09-11T07:50:33.695 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.695 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.695 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:33.695 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.125401+0000", 2024-09-11T07:50:33.695 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2174000000000001e-05 2024-09-11T07:50:33.695 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.695 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.695 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:33.695 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.125406+0000", 2024-09-11T07:50:33.695 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8550000000000001e-06 2024-09-11T07:50:33.695 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.695 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.696 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.696 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.125456+0000", 2024-09-11T07:50:33.696 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9475000000000003e-05 2024-09-11T07:50:33.696 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.696 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.696 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.696 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.125463+0000", 2024-09-11T07:50:33.696 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.8129999999999996e-06 2024-09-11T07:50:33.696 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.696 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.696 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.696 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516885+0000", 2024-09-11T07:50:33.696 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.39142164699999998 2024-09-11T07:50:33.697 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.697 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.697 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.697 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:39.516894+0000", 2024-09-11T07:50:33.697 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.3929999999999997e-06 2024-09-11T07:50:33.697 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.697 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.697 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.697 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.677196+0000", 2024-09-11T07:50:33.697 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.160301123 2024-09-11T07:50:33.697 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.697 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.697 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:33.698 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.677304+0000", 2024-09-11T07:50:33.698 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00010870399999999999 2024-09-11T07:50:33.698 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.698 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.698 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 2,6,7", 2024-09-11T07:50:33.698 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.677637+0000", 2024-09-11T07:50:33.698 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00033315200000000002 2024-09-11T07:50:33.698 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.698 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.698 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.698 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.689127+0000", 2024-09-11T07:50:33.698 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.011489432000000001 2024-09-11T07:50:33.698 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.699 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.699 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:33.699 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.689182+0000", 2024-09-11T07:50:33.699 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.4849000000000003e-05 2024-09-11T07:50:33.699 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.699 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.699 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.699 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.689250+0000", 2024-09-11T07:50:33.699 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.8209999999999999e-05 2024-09-11T07:50:33.699 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.699 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.699 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.699 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.690018+0000", 2024-09-11T07:50:33.700 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00076781699999999996 2024-09-11T07:50:33.700 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.700 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.700 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:33.700 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.690069+0000", 2024-09-11T07:50:33.700 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.1017999999999998e-05 2024-09-11T07:50:33.700 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.700 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.700 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:33.700 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:40:40.690188+0000", 2024-09-11T07:50:33.700 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000119518 2024-09-11T07:50:33.700 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.700 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:33.700 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.701 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.701 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.701 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24353.0:3076 5.1f 5:f95f44c2:::notify.0:head [watch reconnect cookie 94395805741024 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e453)", 2024-09-11T07:50:33.701 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:48:46.124572+0000", 2024-09-11T07:50:33.701 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 107.430927819, 2024-09-11T07:50:33.701 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0651036270000001, 2024-09-11T07:50:33.701 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:33.701 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:50:33.701 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:33.701 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24353", 2024-09-11T07:50:33.701 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1860969011", 2024-09-11T07:50:33.701 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 3076 2024-09-11T07:50:33.701 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.701 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:33.702 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.702 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:33.702 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.124572+0000", 2024-09-11T07:50:33.702 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.702 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.702 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.702 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:33.702 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.124572+0000", 2024-09-11T07:50:33.702 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.702 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.702 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.702 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:33.702 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.124570+0000", 2024-09-11T07:50:33.702 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:50:33.703 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.703 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.703 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:33.703 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.124577+0000", 2024-09-11T07:50:33.703 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.8539999999999999e-06 2024-09-11T07:50:33.703 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.703 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.703 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:33.703 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.124589+0000", 2024-09-11T07:50:33.703 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2374e-05 2024-09-11T07:50:33.703 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.703 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.703 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:33.704 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.124595+0000", 2024-09-11T07:50:33.704 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.0310000000000004e-06 2024-09-11T07:50:33.704 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.704 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.704 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.704 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.124685+0000", 2024-09-11T07:50:33.704 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.9400999999999993e-05 2024-09-11T07:50:33.704 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.704 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.704 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op must wait for map", 2024-09-11T07:50:33.704 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.124692+0000", 2024-09-11T07:50:33.704 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.8240000000000002e-06 2024-09-11T07:50:33.704 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.705 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.705 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.705 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.145359+0000", 2024-09-11T07:50:33.705 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.020666956 2024-09-11T07:50:33.705 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.705 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.705 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.705 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.145368+0000", 2024-09-11T07:50:33.705 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.0580000000000004e-06 2024-09-11T07:50:33.705 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.705 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.705 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.705 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.167278+0000", 2024-09-11T07:50:33.706 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.021910377 2024-09-11T07:50:33.706 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.706 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.706 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for degraded object", 2024-09-11T07:50:33.706 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.167581+0000", 2024-09-11T07:50:33.706 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00030301599999999997 2024-09-11T07:50:33.706 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.706 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.706 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.706 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.186881+0000", 2024-09-11T07:50:33.706 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.019300320999999999 2024-09-11T07:50:33.706 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.706 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.706 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:33.707 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.186945+0000", 2024-09-11T07:50:33.707 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3377e-05 2024-09-11T07:50:33.707 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.707 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.707 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:33.707 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.189676+0000", 2024-09-11T07:50:33.707 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0027309729999999998 2024-09-11T07:50:33.707 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.707 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:33.707 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.707 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.707 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.707 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24353.0:3078 4.4 4:22d26bf9:::data_loggenerations_metadata:head [watch reconnect cookie 94395805765568 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e453)", 2024-09-11T07:50:33.707 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:48:46.124611+0000", 2024-09-11T07:50:33.708 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 107.430888577, 2024-09-11T07:50:33.708 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0443299589999999, 2024-09-11T07:50:33.708 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:33.708 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:50:33.708 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:33.708 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24353", 2024-09-11T07:50:33.708 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1860969011", 2024-09-11T07:50:33.708 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 3078 2024-09-11T07:50:33.708 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.708 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:33.708 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.708 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:33.708 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.124611+0000", 2024-09-11T07:50:33.709 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.709 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.709 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.709 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:33.709 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.124611+0000", 2024-09-11T07:50:33.709 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.709 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.709 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.709 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:33.709 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.124611+0000", 2024-09-11T07:50:33.709 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:50:33.709 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.709 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.710 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:33.710 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.124614+0000", 2024-09-11T07:50:33.710 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4170000000000001e-06 2024-09-11T07:50:33.710 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.710 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.710 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:33.710 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.124622+0000", 2024-09-11T07:50:33.710 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.8199999999999997e-06 2024-09-11T07:50:33.710 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.710 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.710 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:33.710 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.124627+0000", 2024-09-11T07:50:33.710 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8439999999999997e-06 2024-09-11T07:50:33.710 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.710 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.711 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.711 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.125287+0000", 2024-09-11T07:50:33.711 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00066072899999999998 2024-09-11T07:50:33.711 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.711 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.711 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op must wait for map", 2024-09-11T07:50:33.711 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.125298+0000", 2024-09-11T07:50:33.711 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0672999999999999e-05 2024-09-11T07:50:33.711 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.711 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.711 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.711 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.135463+0000", 2024-09-11T07:50:33.711 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.010165454000000001 2024-09-11T07:50:33.711 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.712 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.712 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.712 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:46.135474+0000", 2024-09-11T07:50:33.712 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0268999999999999e-05 2024-09-11T07:50:33.712 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.712 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.712 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.712 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.154095+0000", 2024-09-11T07:50:33.712 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0186210449999999 2024-09-11T07:50:33.712 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.712 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.712 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for degraded object", 2024-09-11T07:50:33.712 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.154523+0000", 2024-09-11T07:50:33.712 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000427742 2024-09-11T07:50:33.713 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.713 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.713 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.713 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.160021+0000", 2024-09-11T07:50:33.713 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0054983219999999999 2024-09-11T07:50:33.713 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.713 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.713 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:33.713 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.160130+0000", 2024-09-11T07:50:33.713 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000109382 2024-09-11T07:50:33.713 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.713 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.713 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:33.713 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:48:47.168941+0000", 2024-09-11T07:50:33.714 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0088109699999999996 2024-09-11T07:50:33.714 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.714 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:33.714 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.714 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.714 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.714 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:208787 11.3 11:c12457a2:::benchmark_data_smithi022_188626_object88979:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e460)", 2024-09-11T07:50:33.714 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:49:07.914125+0000", 2024-09-11T07:50:33.714 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 85.641374612000007, 2024-09-11T07:50:33.714 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.96805258599999999, 2024-09-11T07:50:33.714 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:33.714 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:33.714 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:33.714 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:33.714 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:33.715 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 208787 2024-09-11T07:50:33.715 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.715 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:33.715 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.715 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:33.715 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:07.914125+0000", 2024-09-11T07:50:33.715 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.715 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.715 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.715 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:33.715 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:07.914125+0000", 2024-09-11T07:50:33.715 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.715 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.715 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.715 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:33.716 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:07.914123+0000", 2024-09-11T07:50:33.716 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:50:33.716 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.716 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.716 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:33.716 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:07.914130+0000", 2024-09-11T07:50:33.716 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1430000000000001e-06 2024-09-11T07:50:33.716 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.716 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.716 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:33.716 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:07.914144+0000", 2024-09-11T07:50:33.716 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3037000000000001e-05 2024-09-11T07:50:33.716 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.716 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.717 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:33.717 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:07.914150+0000", 2024-09-11T07:50:33.717 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.4989999999999999e-06 2024-09-11T07:50:33.717 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.717 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.717 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.717 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:07.914200+0000", 2024-09-11T07:50:33.717 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.9746000000000001e-05 2024-09-11T07:50:33.717 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.717 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.717 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.717 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:07.914207+0000", 2024-09-11T07:50:33.717 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.7160000000000003e-06 2024-09-11T07:50:33.718 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.718 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.718 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.718 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:08.867350+0000", 2024-09-11T07:50:33.718 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.95314255000000003 2024-09-11T07:50:33.718 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.718 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.718 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:33.718 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:08.867450+0000", 2024-09-11T07:50:33.718 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000100238 2024-09-11T07:50:33.718 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.718 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.718 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:50:33.718 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:08.867843+0000", 2024-09-11T07:50:33.718 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00039308700000000003 2024-09-11T07:50:33.719 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.719 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.719 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:33.719 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:08.881982+0000", 2024-09-11T07:50:33.719 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.014138629999999999 2024-09-11T07:50:33.719 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.719 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.719 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.719 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:08.882066+0000", 2024-09-11T07:50:33.719 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.4167999999999998e-05 2024-09-11T07:50:33.719 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.719 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.719 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.720 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:08.882104+0000", 2024-09-11T07:50:33.720 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7534999999999999e-05 2024-09-11T07:50:33.720 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.720 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.720 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:33.720 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:08.882142+0000", 2024-09-11T07:50:33.720 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8735000000000001e-05 2024-09-11T07:50:33.720 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.720 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.720 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:33.720 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:08.882178+0000", 2024-09-11T07:50:33.720 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.5379000000000001e-05 2024-09-11T07:50:33.720 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.720 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:33.720 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.721 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.721 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.721 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.25990.0:208788 11.3 11:c8a39144:::benchmark_data_smithi022_188626_object88980:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e460)", 2024-09-11T07:50:33.721 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:49:07.914713+0000", 2024-09-11T07:50:33.721 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 85.640786997000006, 2024-09-11T07:50:33.721 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.97036324699999998, 2024-09-11T07:50:33.721 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:50:33.721 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:33.721 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:50:33.721 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.25990", 2024-09-11T07:50:33.721 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:33.721 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 208788 2024-09-11T07:50:33.721 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.721 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:50:33.722 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.722 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:50:33.722 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:07.914713+0000", 2024-09-11T07:50:33.722 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.722 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.722 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.722 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:50:33.722 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:07.914713+0000", 2024-09-11T07:50:33.722 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:50:33.723 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.723 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.723 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:50:33.723 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:07.914711+0000", 2024-09-11T07:50:33.723 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:50:33.723 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.723 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.723 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:50:33.723 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:07.914729+0000", 2024-09-11T07:50:33.723 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.7691000000000001e-05 2024-09-11T07:50:33.723 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.723 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.723 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:50:33.723 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:07.914742+0000", 2024-09-11T07:50:33.724 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3764999999999999e-05 2024-09-11T07:50:33.724 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.724 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.724 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:50:33.724 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:07.914749+0000", 2024-09-11T07:50:33.724 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9659999999999999e-06 2024-09-11T07:50:33.724 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.724 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.724 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.724 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:07.914791+0000", 2024-09-11T07:50:33.724 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2097e-05 2024-09-11T07:50:33.724 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.724 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.725 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:50:33.725 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:07.914800+0000", 2024-09-11T07:50:33.725 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.8720000000000008e-06 2024-09-11T07:50:33.725 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.725 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.725 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:50:33.725 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:08.874007+0000", 2024-09-11T07:50:33.725 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.959206844 2024-09-11T07:50:33.725 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.725 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.725 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:50:33.725 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:08.874112+0000", 2024-09-11T07:50:33.725 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000104769 2024-09-11T07:50:33.725 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.726 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.726 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for subops from 5,6", 2024-09-11T07:50:33.726 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:08.874223+0000", 2024-09-11T07:50:33.726 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000111482 2024-09-11T07:50:33.726 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.726 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.726 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "op_commit", 2024-09-11T07:50:33.726 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:08.881989+0000", 2024-09-11T07:50:33.726 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0077657220000000001 2024-09-11T07:50:33.726 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.726 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.726 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.726 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:08.882246+0000", 2024-09-11T07:50:33.727 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00025691499999999998 2024-09-11T07:50:33.727 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.727 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.727 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:33.727 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:08.884960+0000", 2024-09-11T07:50:33.727 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.002713774 2024-09-11T07:50:33.727 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.727 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.727 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:50:33.727 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:08.885038+0000", 2024-09-11T07:50:33.727 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.8108000000000005e-05 2024-09-11T07:50:33.727 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:50:33.727 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:50:33.728 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:50:33.728 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:49:08.885076+0000", 2024-09-11T07:50:33.728 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8124000000000003e-05 2024-09-11T07:50:33.728 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.728 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:33.728 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.728 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:50:33.728 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:50:33.728 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:50:33.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:33 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:50:33.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:33 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:50:33.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:33 smithi104 ceph-mon[33507]: 2.19 scrub starts 2024-09-11T07:50:33.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:33 smithi104 ceph-mon[33507]: 2.19 scrub ok 2024-09-11T07:50:33.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:33 smithi104 ceph-mon[33507]: osdmap e483: 8 total, 7 up, 5 in 2024-09-11T07:50:33.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:33 smithi104 ceph-mon[33507]: 1.1 scrub starts 2024-09-11T07:50:33.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:33 smithi104 ceph-mon[33507]: 1.1 scrub ok 2024-09-11T07:50:33.917 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:33 smithi022 podman[214603]: 2024-09-11 07:50:33.596707669 +0000 UTC m=+0.249975906 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:50:33.917 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:33 smithi022 podman[214603]: 2024-09-11 07:50:33.794581835 +0000 UTC m=+0.447850052 container create afc229d0d38740bbfec3670aa72de225520ea9f1522031a445d4108204e04466 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2-activate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-11T07:50:34.420 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:50:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:50:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:50:34.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:34 smithi022 ceph-mon[29462]: pgmap v1117: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:50:34.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:34 smithi022 ceph-mon[37330]: pgmap v1117: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:50:34.668 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:34 smithi022 podman[214603]: 2024-09-11 07:50:34.450375014 +0000 UTC m=+1.103643240 container init afc229d0d38740bbfec3670aa72de225520ea9f1522031a445d4108204e04466 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2-activate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-11T07:50:34.668 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:34 smithi022 podman[214603]: 2024-09-11 07:50:34.453975947 +0000 UTC m=+1.107244162 container start afc229d0d38740bbfec3670aa72de225520ea9f1522031a445d4108204e04466 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2-activate, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-11T07:50:34.668 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:34 smithi022 podman[214603]: 2024-09-11 07:50:34.567013178 +0000 UTC m=+1.220281392 container attach afc229d0d38740bbfec3670aa72de225520ea9f1522031a445d4108204e04466 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2-activate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-11T07:50:34.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:34 smithi104 ceph-mon[33507]: pgmap v1117: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:50:35.030 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:34 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2-activate[214635]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-11T07:50:35.030 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:34 smithi022 bash[214603]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-11T07:50:35.030 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:34 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2-activate[214635]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-11T07:50:35.030 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:34 smithi022 bash[214603]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-11T07:50:35.030 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:34 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2-activate[214635]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-11T07:50:35.031 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:34 smithi022 bash[214603]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-11T07:50:35.031 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:34 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2-activate[214635]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-11T07:50:35.031 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:34 smithi022 bash[214603]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-11T07:50:35.031 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:34 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2-activate[214635]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-09-11T07:50:35.031 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:34 smithi022 bash[214603]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-09-11T07:50:35.031 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:34 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2-activate[214635]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-11T07:50:35.031 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:34 smithi022 bash[214603]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-11T07:50:35.031 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:34 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2-activate[214635]: --> ceph-volume raw activate successful for osd ID: 2 2024-09-11T07:50:35.031 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:34 smithi022 bash[214603]: --> ceph-volume raw activate successful for osd ID: 2 2024-09-11T07:50:35.031 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:35 smithi022 podman[214603]: 2024-09-11 07:50:35.028456233 +0000 UTC m=+1.681724453 container died afc229d0d38740bbfec3670aa72de225520ea9f1522031a445d4108204e04466 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2-activate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-09-11T07:50:35.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:35 smithi022 ceph-mon[29462]: 1.4 scrub starts 2024-09-11T07:50:35.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:35 smithi022 ceph-mon[29462]: 1.4 scrub ok 2024-09-11T07:50:35.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:35 smithi022 ceph-mon[29462]: 2.10 scrub starts 2024-09-11T07:50:35.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:35 smithi022 ceph-mon[29462]: 2.10 scrub ok 2024-09-11T07:50:35.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:35 smithi022 ceph-mon[37330]: 1.4 scrub starts 2024-09-11T07:50:35.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:35 smithi022 ceph-mon[37330]: 1.4 scrub ok 2024-09-11T07:50:35.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:35 smithi022 ceph-mon[37330]: 2.10 scrub starts 2024-09-11T07:50:35.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:35 smithi022 ceph-mon[37330]: 2.10 scrub ok 2024-09-11T07:50:35.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:35 smithi104 ceph-mon[33507]: 1.4 scrub starts 2024-09-11T07:50:35.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:35 smithi104 ceph-mon[33507]: 1.4 scrub ok 2024-09-11T07:50:35.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:35 smithi104 ceph-mon[33507]: 2.10 scrub starts 2024-09-11T07:50:35.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:35 smithi104 ceph-mon[33507]: 2.10 scrub ok 2024-09-11T07:50:35.881 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-11T07:50:35.997 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:35 smithi022 podman[214766]: 2024-09-11 07:50:35.690931996 +0000 UTC m=+0.885401831 container remove afc229d0d38740bbfec3670aa72de225520ea9f1522031a445d4108204e04466 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2-activate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-11T07:50:36.135 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:50:36.295 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:35 smithi022 podman[214825]: 2024-09-11 07:50:35.899976984 +0000 UTC m=+0.017101916 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:50:36.295 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:36 smithi022 podman[214825]: 2024-09-11 07:50:36.012579735 +0000 UTC m=+0.129704654 container create 5fe27ec66de3ae28573a834f75c0335ba7d83d318860622206c24ab94f8b3c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-11T07:50:36.295 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:36 smithi022 podman[214825]: 2024-09-11 07:50:36.294739755 +0000 UTC m=+0.411864675 container init 5fe27ec66de3ae28573a834f75c0335ba7d83d318860622206c24ab94f8b3c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-11T07:50:36.417 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-11T07:50:36.523 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:36 smithi104 ceph-mon[33507]: 4.f scrub starts 2024-09-11T07:50:36.523 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:36 smithi104 ceph-mon[33507]: 4.f scrub ok 2024-09-11T07:50:36.523 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:36 smithi104 ceph-mon[33507]: pgmap v1118: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:50:36.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:36 smithi022 ceph-mon[29462]: 4.f scrub starts 2024-09-11T07:50:36.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:36 smithi022 ceph-mon[29462]: 4.f scrub ok 2024-09-11T07:50:36.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:36 smithi022 ceph-mon[29462]: pgmap v1118: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:50:36.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:36 smithi022 ceph-mon[37330]: 4.f scrub starts 2024-09-11T07:50:36.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:36 smithi022 ceph-mon[37330]: 4.f scrub ok 2024-09-11T07:50:36.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:36 smithi022 ceph-mon[37330]: pgmap v1118: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:50:36.667 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:36 smithi022 podman[214825]: 2024-09-11 07:50:36.298144958 +0000 UTC m=+0.415269879 container start 5fe27ec66de3ae28573a834f75c0335ba7d83d318860622206c24ab94f8b3c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:50:36.667 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:36 smithi022 bash[214825]: 5fe27ec66de3ae28573a834f75c0335ba7d83d318860622206c24ab94f8b3c31 2024-09-11T07:50:36.667 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:36 smithi022 systemd[1]: Started Ceph osd.2 for 12a285d8-700e-11ef-bcea-c7b262605968. 2024-09-11T07:50:36.683 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:50:37.065 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:36 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2[214840]: 2024-09-11T07:50:36.884+0000 7fa7df6bf440 -1 Falling back to public interface 2024-09-11T07:50:37.577 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:37 smithi104 ceph-mon[33507]: 5.18 scrub starts 2024-09-11T07:50:37.578 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:37 smithi104 ceph-mon[33507]: 5.18 scrub ok 2024-09-11T07:50:37.878 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:37 smithi022 ceph-mon[29462]: 5.18 scrub starts 2024-09-11T07:50:37.878 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:37 smithi022 ceph-mon[29462]: 5.18 scrub ok 2024-09-11T07:50:37.878 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:37 smithi022 ceph-mon[37330]: 5.18 scrub starts 2024-09-11T07:50:37.879 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:37 smithi022 ceph-mon[37330]: 5.18 scrub ok 2024-09-11T07:50:37.947 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:50:37.947 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:50:37.947 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 0 2024-09-11T07:50:37.947 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:50:38.166 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:50:38 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:50:38.057+0000 7f0b5d524640 -1 mgr get_metadata_python Requested missing service osd.2 2024-09-11T07:50:38.405 INFO:teuthology.orchestra.run.smithi022.stderr:no valid command found; 10 closest matches: 2024-09-11T07:50:38.405 INFO:teuthology.orchestra.run.smithi022.stderr:0 2024-09-11T07:50:38.405 INFO:teuthology.orchestra.run.smithi022.stderr:1 2024-09-11T07:50:38.406 INFO:teuthology.orchestra.run.smithi022.stderr:2 2024-09-11T07:50:38.406 INFO:teuthology.orchestra.run.smithi022.stderr:abort 2024-09-11T07:50:38.406 INFO:teuthology.orchestra.run.smithi022.stderr:assert 2024-09-11T07:50:38.406 INFO:teuthology.orchestra.run.smithi022.stderr:config diff 2024-09-11T07:50:38.406 INFO:teuthology.orchestra.run.smithi022.stderr:config diff get 2024-09-11T07:50:38.406 INFO:teuthology.orchestra.run.smithi022.stderr:config get 2024-09-11T07:50:38.406 INFO:teuthology.orchestra.run.smithi022.stderr:config help [] 2024-09-11T07:50:38.406 INFO:teuthology.orchestra.run.smithi022.stderr:config set ... 2024-09-11T07:50:38.406 INFO:teuthology.orchestra.run.smithi022.stderr:admin_socket: invalid command 2024-09-11T07:50:38.541 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-11T07:50:38.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:38 smithi104 ceph-mon[33507]: pgmap v1119: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:50:38.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:38 smithi104 ceph-mon[33507]: 5.e scrub starts 2024-09-11T07:50:38.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:38 smithi104 ceph-mon[33507]: 5.e scrub ok 2024-09-11T07:50:38.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:50:38.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:38 smithi022 ceph-mon[29462]: pgmap v1119: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:50:38.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:38 smithi022 ceph-mon[29462]: 5.e scrub starts 2024-09-11T07:50:38.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:38 smithi022 ceph-mon[29462]: 5.e scrub ok 2024-09-11T07:50:38.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:50:38.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:38 smithi022 ceph-mon[37330]: pgmap v1119: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:50:38.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:38 smithi022 ceph-mon[37330]: 5.e scrub starts 2024-09-11T07:50:38.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:38 smithi022 ceph-mon[37330]: 5.e scrub ok 2024-09-11T07:50:38.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:50:38.796 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:50:39.396 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T07:50:39.397 INFO:tasks.cephadm.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-11T07:50:39.941 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:39 smithi104 ceph-mon[33507]: pgmap v1120: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:50:40.135 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:39 smithi022 ceph-mon[29462]: pgmap v1120: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:50:40.136 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:39 smithi022 ceph-mon[37330]: pgmap v1120: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:50:40.416 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2[214840]: 2024-09-11T07:50:40.134+0000 7fa7df6bf440 -1 osd.2 469 log_to_monitors true 2024-09-11T07:50:40.417 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:50:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:50:40] "GET /metrics HTTP/1.1" 200 207456 "" "Prometheus/2.43.0" 2024-09-11T07:50:40.495 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:50:40.495 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:50:40.495 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2024-09-11T07:50:40.495 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2024-09-11T07:50:40.495 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:50:40.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:50:40 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:50:41.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:40 smithi104 ceph-mon[33507]: from='osd.2 v1:172.21.15.22:6809/574977173' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:50:41.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:40 smithi104 ceph-mon[33507]: 6.f deep-scrub starts 2024-09-11T07:50:41.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:40 smithi104 ceph-mon[33507]: 6.f deep-scrub ok 2024-09-11T07:50:41.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:40 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:50:41.163 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-11T07:50:41.166 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:50:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2[214840]: 2024-09-11T07:50:40.767+0000 7fa7d5c68640 -1 osd.2 469 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:50:41.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:40 smithi022 ceph-mon[29462]: from='osd.2 v1:172.21.15.22:6809/574977173' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:50:41.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:40 smithi022 ceph-mon[29462]: 6.f deep-scrub starts 2024-09-11T07:50:41.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:40 smithi022 ceph-mon[29462]: 6.f deep-scrub ok 2024-09-11T07:50:41.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:40 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:50:41.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:40 smithi022 ceph-mon[37330]: from='osd.2 v1:172.21.15.22:6809/574977173' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:50:41.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:40 smithi022 ceph-mon[37330]: 6.f deep-scrub starts 2024-09-11T07:50:41.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:40 smithi022 ceph-mon[37330]: 6.f deep-scrub ok 2024-09-11T07:50:41.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:40 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:50:41.451 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:50:41.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:41 smithi104 ceph-mon[33507]: 3.1 scrub starts 2024-09-11T07:50:41.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:41 smithi104 ceph-mon[33507]: 3.1 scrub ok 2024-09-11T07:50:41.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:41 smithi104 ceph-mon[33507]: from='osd.2 v1:172.21.15.22:6809/574977173' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T07:50:41.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:41 smithi104 ceph-mon[33507]: osdmap e484: 8 total, 7 up, 5 in 2024-09-11T07:50:41.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:41 smithi104 ceph-mon[33507]: from='osd.2 v1:172.21.15.22:6809/574977173' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-11T07:50:41.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:41 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:50:41.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:41 smithi104 ceph-mon[33507]: pgmap v1122: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:50:42.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:41 smithi022 ceph-mon[29462]: 3.1 scrub starts 2024-09-11T07:50:42.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:41 smithi022 ceph-mon[29462]: 3.1 scrub ok 2024-09-11T07:50:42.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:41 smithi022 ceph-mon[29462]: from='osd.2 v1:172.21.15.22:6809/574977173' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T07:50:42.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:41 smithi022 ceph-mon[29462]: osdmap e484: 8 total, 7 up, 5 in 2024-09-11T07:50:42.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:41 smithi022 ceph-mon[29462]: from='osd.2 v1:172.21.15.22:6809/574977173' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-11T07:50:42.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:41 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:50:42.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:41 smithi022 ceph-mon[29462]: pgmap v1122: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:50:42.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:41 smithi022 ceph-mon[37330]: 3.1 scrub starts 2024-09-11T07:50:42.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:41 smithi022 ceph-mon[37330]: 3.1 scrub ok 2024-09-11T07:50:42.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:41 smithi022 ceph-mon[37330]: from='osd.2 v1:172.21.15.22:6809/574977173' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T07:50:42.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:41 smithi022 ceph-mon[37330]: osdmap e484: 8 total, 7 up, 5 in 2024-09-11T07:50:42.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:41 smithi022 ceph-mon[37330]: from='osd.2 v1:172.21.15.22:6809/574977173' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-09-11T07:50:42.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:41 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:50:42.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:41 smithi022 ceph-mon[37330]: pgmap v1122: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:50:42.830 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:42 smithi104 ceph-mon[33507]: 2.16 scrub starts 2024-09-11T07:50:42.830 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:42 smithi104 ceph-mon[33507]: 2.16 scrub ok 2024-09-11T07:50:42.830 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:42 smithi104 ceph-mon[33507]: osd.2 v1:172.21.15.22:6809/574977173 boot 2024-09-11T07:50:42.830 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:42 smithi104 ceph-mon[33507]: osdmap e485: 8 total, 8 up, 5 in 2024-09-11T07:50:42.830 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:42 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:50:42.830 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:42 smithi104 ceph-mon[33507]: 6.8 scrub starts 2024-09-11T07:50:42.830 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:42 smithi104 ceph-mon[33507]: 6.8 scrub ok 2024-09-11T07:50:43.059 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:50:43.059 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2024-09-11T07:50:43.059 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2024-09-11T07:50:43.059 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [] 2024-09-11T07:50:43.059 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:50:43.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:42 smithi022 ceph-mon[29462]: 2.16 scrub starts 2024-09-11T07:50:43.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:42 smithi022 ceph-mon[29462]: 2.16 scrub ok 2024-09-11T07:50:43.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:42 smithi022 ceph-mon[29462]: osd.2 v1:172.21.15.22:6809/574977173 boot 2024-09-11T07:50:43.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:42 smithi022 ceph-mon[29462]: osdmap e485: 8 total, 8 up, 5 in 2024-09-11T07:50:43.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:42 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:50:43.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:42 smithi022 ceph-mon[29462]: 6.8 scrub starts 2024-09-11T07:50:43.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:42 smithi022 ceph-mon[29462]: 6.8 scrub ok 2024-09-11T07:50:43.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:42 smithi022 ceph-mon[37330]: 2.16 scrub starts 2024-09-11T07:50:43.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:42 smithi022 ceph-mon[37330]: 2.16 scrub ok 2024-09-11T07:50:43.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:42 smithi022 ceph-mon[37330]: osd.2 v1:172.21.15.22:6809/574977173 boot 2024-09-11T07:50:43.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:42 smithi022 ceph-mon[37330]: osdmap e485: 8 total, 8 up, 5 in 2024-09-11T07:50:43.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:42 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:50:43.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:42 smithi022 ceph-mon[37330]: 6.8 scrub starts 2024-09-11T07:50:43.169 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:42 smithi022 ceph-mon[37330]: 6.8 scrub ok 2024-09-11T07:50:43.719 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-11T07:50:43.964 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:50:44.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:43 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:50:44.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:43 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:50:44.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:43 smithi104 ceph-mon[33507]: osdmap e486: 8 total, 8 up, 5 in 2024-09-11T07:50:44.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:43 smithi104 ceph-mon[33507]: pgmap v1125: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:50:44.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:43 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:50:44.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:43 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:50:44.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:43 smithi022 ceph-mon[29462]: osdmap e486: 8 total, 8 up, 5 in 2024-09-11T07:50:44.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:43 smithi022 ceph-mon[29462]: pgmap v1125: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:50:44.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:43 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:50:44.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:43 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:50:44.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:43 smithi022 ceph-mon[37330]: osdmap e486: 8 total, 8 up, 5 in 2024-09-11T07:50:44.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:43 smithi022 ceph-mon[37330]: pgmap v1125: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:50:44.306 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:50:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:50:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:50:44.398 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-11T07:50:44.669 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:50:45.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:44 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:50:45.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:44 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:50:45.106 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:44 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:50:45.106 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:44 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:50:45.106 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:44 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:50:45.106 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:44 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:50:45.606 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:50:45.606 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:50:45.606 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 0 2024-09-11T07:50:45.606 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:50:46.165 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:45 smithi022 ceph-mon[29462]: osdmap e487: 8 total, 8 up, 5 in 2024-09-11T07:50:46.165 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:45 smithi022 ceph-mon[29462]: 4.10 scrub starts 2024-09-11T07:50:46.165 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:45 smithi022 ceph-mon[29462]: 4.10 scrub ok 2024-09-11T07:50:46.165 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:45 smithi022 ceph-mon[29462]: pgmap v1127: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T07:50:46.165 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:45 smithi022 ceph-mon[37330]: osdmap e487: 8 total, 8 up, 5 in 2024-09-11T07:50:46.165 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:45 smithi022 ceph-mon[37330]: 4.10 scrub starts 2024-09-11T07:50:46.166 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:45 smithi022 ceph-mon[37330]: 4.10 scrub ok 2024-09-11T07:50:46.166 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:45 smithi022 ceph-mon[37330]: pgmap v1127: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T07:50:46.183 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:45 smithi104 ceph-mon[33507]: osdmap e487: 8 total, 8 up, 5 in 2024-09-11T07:50:46.183 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:45 smithi104 ceph-mon[33507]: 4.10 scrub starts 2024-09-11T07:50:46.183 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:45 smithi104 ceph-mon[33507]: 4.10 scrub ok 2024-09-11T07:50:46.183 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:45 smithi104 ceph-mon[33507]: pgmap v1127: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T07:50:46.208 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-11T07:50:46.397 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:50:46.398 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:50:46.398 INFO:teuthology.orchestra.run.smithi022.stdout: "num_ops": 0 2024-09-11T07:50:46.398 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:50:46.451 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:50:47.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:46 smithi022 ceph-mon[29462]: 1.2 scrub starts 2024-09-11T07:50:47.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:46 smithi022 ceph-mon[29462]: 1.2 scrub ok 2024-09-11T07:50:47.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:46 smithi022 ceph-mon[29462]: 4.19 scrub starts 2024-09-11T07:50:47.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:46 smithi022 ceph-mon[29462]: 4.19 scrub ok 2024-09-11T07:50:47.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:46 smithi022 ceph-mon[37330]: 1.2 scrub starts 2024-09-11T07:50:47.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:46 smithi022 ceph-mon[37330]: 1.2 scrub ok 2024-09-11T07:50:47.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:46 smithi022 ceph-mon[37330]: 4.19 scrub starts 2024-09-11T07:50:47.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:46 smithi022 ceph-mon[37330]: 4.19 scrub ok 2024-09-11T07:50:47.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:46 smithi104 ceph-mon[33507]: 1.2 scrub starts 2024-09-11T07:50:47.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:46 smithi104 ceph-mon[33507]: 1.2 scrub ok 2024-09-11T07:50:47.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:46 smithi104 ceph-mon[33507]: 4.19 scrub starts 2024-09-11T07:50:47.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:46 smithi104 ceph-mon[33507]: 4.19 scrub ok 2024-09-11T07:50:47.321 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-11T07:50:47.321 INFO:tasks.thrashosds.thrasher:Adding osd 4 2024-09-11T07:50:47.321 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd in 4 2024-09-11T07:50:47.580 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:50:47.980 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:47 smithi022 ceph-mon[29462]: 4.0 scrub starts 2024-09-11T07:50:47.980 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:47 smithi022 ceph-mon[29462]: 4.0 scrub ok 2024-09-11T07:50:47.980 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:47 smithi022 ceph-mon[29462]: pgmap v1128: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:50:47.981 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:47 smithi022 ceph-mon[37330]: 4.0 scrub starts 2024-09-11T07:50:47.981 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:47 smithi022 ceph-mon[37330]: 4.0 scrub ok 2024-09-11T07:50:47.981 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:47 smithi022 ceph-mon[37330]: pgmap v1128: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:50:48.105 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:50:48.105 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:50:48.105 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2024-09-11T07:50:48.105 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2024-09-11T07:50:48.105 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:50:48.113 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:47 smithi104 ceph-mon[33507]: 4.0 scrub starts 2024-09-11T07:50:48.113 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:47 smithi104 ceph-mon[33507]: 4.0 scrub ok 2024-09-11T07:50:48.113 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:47 smithi104 ceph-mon[33507]: pgmap v1128: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:50:48.750 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-11T07:50:48.997 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:50:49.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:48 smithi104 ceph-mon[33507]: 2.1c scrub starts 2024-09-11T07:50:49.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:48 smithi104 ceph-mon[33507]: 2.1c scrub ok 2024-09-11T07:50:49.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:48 smithi104 ceph-mon[33507]: 2.9 scrub starts 2024-09-11T07:50:49.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:48 smithi104 ceph-mon[33507]: 2.9 scrub ok 2024-09-11T07:50:49.074 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:48 smithi022 ceph-mon[29462]: 2.1c scrub starts 2024-09-11T07:50:49.075 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:48 smithi022 ceph-mon[29462]: 2.1c scrub ok 2024-09-11T07:50:49.075 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:48 smithi022 ceph-mon[29462]: 2.9 scrub starts 2024-09-11T07:50:49.075 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:48 smithi022 ceph-mon[29462]: 2.9 scrub ok 2024-09-11T07:50:49.075 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:48 smithi022 ceph-mon[37330]: 2.1c scrub starts 2024-09-11T07:50:49.075 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:48 smithi022 ceph-mon[37330]: 2.1c scrub ok 2024-09-11T07:50:49.075 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:48 smithi022 ceph-mon[37330]: 2.9 scrub starts 2024-09-11T07:50:49.075 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:48 smithi022 ceph-mon[37330]: 2.9 scrub ok 2024-09-11T07:50:49.870 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:49 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-a[29458]: 2024-09-11T07:50:49.479+0000 7fde5d00f640 -1 mon.a@0(leader).osd e488 definitely_dead 0 2024-09-11T07:50:49.874 INFO:teuthology.orchestra.run.smithi022.stderr:marked in osd.4. 2024-09-11T07:50:50.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:49 smithi104 ceph-mon[33507]: 6.e scrub starts 2024-09-11T07:50:50.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:49 smithi104 ceph-mon[33507]: 6.e scrub ok 2024-09-11T07:50:50.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:49 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:50:50.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:49 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:50:50.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:49 smithi104 ceph-mon[33507]: osdmap e488: 8 total, 8 up, 5 in 2024-09-11T07:50:50.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:49 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3286370286' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["4"]}]: dispatch 2024-09-11T07:50:50.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:49 smithi104 ceph-mon[33507]: pgmap v1130: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 749 B/s rd, 0 op/s 2024-09-11T07:50:50.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:49 smithi022 ceph-mon[29462]: 6.e scrub starts 2024-09-11T07:50:50.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:49 smithi022 ceph-mon[29462]: 6.e scrub ok 2024-09-11T07:50:50.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:49 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:50:50.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:49 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:50:50.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:49 smithi022 ceph-mon[29462]: osdmap e488: 8 total, 8 up, 5 in 2024-09-11T07:50:50.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:49 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3286370286' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["4"]}]: dispatch 2024-09-11T07:50:50.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:49 smithi022 ceph-mon[29462]: pgmap v1130: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 749 B/s rd, 0 op/s 2024-09-11T07:50:50.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:49 smithi022 ceph-mon[37330]: 6.e scrub starts 2024-09-11T07:50:50.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:49 smithi022 ceph-mon[37330]: 6.e scrub ok 2024-09-11T07:50:50.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:49 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:50:50.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:49 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:50:50.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:49 smithi022 ceph-mon[37330]: osdmap e488: 8 total, 8 up, 5 in 2024-09-11T07:50:50.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:49 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3286370286' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["4"]}]: dispatch 2024-09-11T07:50:50.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:49 smithi022 ceph-mon[37330]: pgmap v1130: 142 pgs: 2 active+clean+remapped, 140 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 749 B/s rd, 0 op/s 2024-09-11T07:50:50.571 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:50:50.571 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2024-09-11T07:50:50.571 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2024-09-11T07:50:50.571 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2024-09-11T07:50:50.572 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.572 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:231567 9.6 9:62bf3667:::benchmark_data_smithi022_133943_object104976:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:50:50.572 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:56.613955+0000", 2024-09-11T07:50:50.572 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 593.95455475799997, 2024-09-11T07:50:50.572 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.14854477399999999, 2024-09-11T07:50:50.572 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:50.572 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:50.572 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:50:50.572 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:50:50.572 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:50.572 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 231567 2024-09-11T07:50:50.573 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.573 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:50.573 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.573 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:50.573 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.613955+0000", 2024-09-11T07:50:50.573 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.573 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.573 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.573 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:50.573 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.613955+0000", 2024-09-11T07:50:50.573 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.573 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.573 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.573 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:50.574 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.613953+0000", 2024-09-11T07:50:50.574 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:50:50.574 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.574 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.574 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:50.574 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.613960+0000", 2024-09-11T07:50:50.574 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3200000000000002e-06 2024-09-11T07:50:50.574 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.574 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.574 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:50.574 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.613988+0000", 2024-09-11T07:50:50.574 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.745e-05 2024-09-11T07:50:50.574 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.575 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.575 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:50.575 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.613994+0000", 2024-09-11T07:50:50.575 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0270000000000001e-06 2024-09-11T07:50:50.575 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.575 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.575 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:50.575 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.614040+0000", 2024-09-11T07:50:50.575 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6403999999999998e-05 2024-09-11T07:50:50.575 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.575 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.575 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:50.575 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.614209+0000", 2024-09-11T07:50:50.575 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016869000000000001 2024-09-11T07:50:50.576 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.576 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.576 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:50:50.576 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.614696+0000", 2024-09-11T07:50:50.576 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00048757100000000001 2024-09-11T07:50:50.576 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.576 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.576 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:50:50.576 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.617583+0000", 2024-09-11T07:50:50.576 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0028868129999999998 2024-09-11T07:50:50.576 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.576 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.576 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:50.576 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.618363+0000", 2024-09-11T07:50:50.577 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00078008600000000004 2024-09-11T07:50:50.577 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.577 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.577 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:50.577 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.762435+0000", 2024-09-11T07:50:50.577 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.144071592 2024-09-11T07:50:50.577 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.577 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.577 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:50.577 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.762464+0000", 2024-09-11T07:50:50.577 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9068999999999999e-05 2024-09-11T07:50:50.577 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.577 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.578 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:50.578 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:56.762500+0000", 2024-09-11T07:50:50.578 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5751999999999999e-05 2024-09-11T07:50:50.578 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.578 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:50.578 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.578 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.578 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.578 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:232544 9.13 9:cb4ef5ab:::benchmark_data_smithi022_133943_object105953:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e319)", 2024-09-11T07:50:50.578 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:40:57.689887+0000", 2024-09-11T07:50:50.578 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 592.87862264800003, 2024-09-11T07:50:50.578 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.186799876, 2024-09-11T07:50:50.578 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:50.578 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:50.579 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:50:50.579 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:50:50.579 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:50.579 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 232544 2024-09-11T07:50:50.579 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.579 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:50.579 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.579 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:50.579 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689887+0000", 2024-09-11T07:50:50.579 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.579 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.579 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.579 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:50.580 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689887+0000", 2024-09-11T07:50:50.580 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.580 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.580 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.580 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:50.580 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689885+0000", 2024-09-11T07:50:50.580 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:50:50.580 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.580 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.580 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:50.580 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689891+0000", 2024-09-11T07:50:50.580 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0310000000000004e-06 2024-09-11T07:50:50.580 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.581 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.581 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:50.581 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689903+0000", 2024-09-11T07:50:50.581 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2167000000000001e-05 2024-09-11T07:50:50.581 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.581 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.581 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:50.581 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689909+0000", 2024-09-11T07:50:50.581 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7620000000000001e-06 2024-09-11T07:50:50.581 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.582 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.582 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:50.582 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.689947+0000", 2024-09-11T07:50:50.582 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7960999999999998e-05 2024-09-11T07:50:50.582 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.582 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.582 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:50.582 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.690025+0000", 2024-09-11T07:50:50.582 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7885000000000004e-05 2024-09-11T07:50:50.582 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.582 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.582 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:50:50.582 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.690473+0000", 2024-09-11T07:50:50.583 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00044783300000000002 2024-09-11T07:50:50.583 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.583 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.583 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:50:50.583 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.692445+0000", 2024-09-11T07:50:50.583 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001972461 2024-09-11T07:50:50.583 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.583 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.583 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:50.583 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.692900+0000", 2024-09-11T07:50:50.583 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00045435300000000001 2024-09-11T07:50:50.583 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.583 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.584 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:50.584 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.876606+0000", 2024-09-11T07:50:50.584 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.18370584700000001 2024-09-11T07:50:50.584 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.584 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.584 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:50.584 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.876647+0000", 2024-09-11T07:50:50.584 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1140999999999998e-05 2024-09-11T07:50:50.584 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.584 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.584 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:50.584 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:40:57.876687+0000", 2024-09-11T07:50:50.584 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0153e-05 2024-09-11T07:50:50.585 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.585 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:50.585 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.585 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.585 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.585 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.15504.0:234546 9.6 9:65687430:::benchmark_data_smithi022_133943_object107955:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e320)", 2024-09-11T07:50:50.585 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:41:00.355245+0000", 2024-09-11T07:50:50.585 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 590.21326441500003, 2024-09-11T07:50:50.585 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.17097563599999999, 2024-09-11T07:50:50.586 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:50.586 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:50.586 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:50:50.586 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.15504", 2024-09-11T07:50:50.586 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1933311807", 2024-09-11T07:50:50.586 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 234546 2024-09-11T07:50:50.586 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.586 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:50.586 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.586 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:50.586 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355245+0000", 2024-09-11T07:50:50.586 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.587 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.587 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.587 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:50.587 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355245+0000", 2024-09-11T07:50:50.587 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.587 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.587 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.587 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:50.587 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355244+0000", 2024-09-11T07:50:50.587 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:50:50.587 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.587 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.587 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:50.588 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355248+0000", 2024-09-11T07:50:50.588 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.529e-06 2024-09-11T07:50:50.588 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.588 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.588 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:50.588 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355255+0000", 2024-09-11T07:50:50.588 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1289999999999998e-06 2024-09-11T07:50:50.588 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.588 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.588 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:50.588 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355259+0000", 2024-09-11T07:50:50.588 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7149999999999997e-06 2024-09-11T07:50:50.588 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.589 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.589 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:50.589 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355306+0000", 2024-09-11T07:50:50.589 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6631999999999997e-05 2024-09-11T07:50:50.589 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.589 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.589 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:50.589 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355383+0000", 2024-09-11T07:50:50.589 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6509999999999998e-05 2024-09-11T07:50:50.589 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.589 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.589 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:50:50.590 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.355796+0000", 2024-09-11T07:50:50.590 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00041318000000000003 2024-09-11T07:50:50.590 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.590 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.590 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:50:50.590 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.357852+0000", 2024-09-11T07:50:50.590 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0020561059999999998 2024-09-11T07:50:50.590 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.590 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.590 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:50.590 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.359012+0000", 2024-09-11T07:50:50.590 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001160295 2024-09-11T07:50:50.591 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.591 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.591 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:50.591 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.526128+0000", 2024-09-11T07:50:50.591 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.167115923 2024-09-11T07:50:50.591 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.591 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.591 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:50.591 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.526176+0000", 2024-09-11T07:50:50.591 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8310000000000003e-05 2024-09-11T07:50:50.591 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.591 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.591 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:50.592 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:41:00.526221+0000", 2024-09-11T07:50:50.592 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4447000000000003e-05 2024-09-11T07:50:50.592 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.592 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:50.592 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.592 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.592 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.592 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:70288 10.6 e356/346 10:6714b47c:::benchmark_data_smithi022_163866_object70287:head v 356'4440, mlcod=356'4437)", 2024-09-11T07:50:50.592 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:53.786739+0000", 2024-09-11T07:50:50.592 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 476.78177097499997, 2024-09-11T07:50:50.592 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.164540297, 2024-09-11T07:50:50.592 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:50.592 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:50.593 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:50.593 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.593 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:50.593 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786739+0000", 2024-09-11T07:50:50.593 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.593 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.593 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.593 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:50.593 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786739+0000", 2024-09-11T07:50:50.593 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.593 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.593 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.593 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:50.594 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786737+0000", 2024-09-11T07:50:50.594 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:50:50.594 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.594 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.594 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:50.594 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786793+0000", 2024-09-11T07:50:50.594 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.516e-05 2024-09-11T07:50:50.594 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.594 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.594 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:50.594 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786802+0000", 2024-09-11T07:50:50.594 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0149999999999992e-06 2024-09-11T07:50:50.594 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.594 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.595 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:50.595 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786807+0000", 2024-09-11T07:50:50.595 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3900000000000001e-06 2024-09-11T07:50:50.595 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.595 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.595 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:50.595 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786852+0000", 2024-09-11T07:50:50.595 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5426000000000002e-05 2024-09-11T07:50:50.595 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.595 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.595 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:50.595 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.786869+0000", 2024-09-11T07:50:50.595 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6477000000000002e-05 2024-09-11T07:50:50.595 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.596 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.596 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:50.596 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951240+0000", 2024-09-11T07:50:50.596 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16437090500000001 2024-09-11T07:50:50.596 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.596 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.596 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:50.596 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951279+0000", 2024-09-11T07:50:50.596 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9072999999999997e-05 2024-09-11T07:50:50.596 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.596 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:50.596 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.596 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.597 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.597 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:70292 10.a e356/341 10:547c94c8:::benchmark_data_smithi022_163866_object70291:head v 356'4331, mlcod=356'4329)", 2024-09-11T07:50:50.597 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:53.788575+0000", 2024-09-11T07:50:50.597 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 476.779934266, 2024-09-11T07:50:50.597 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16920853899999999, 2024-09-11T07:50:50.597 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:50.597 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:50.600 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:50.600 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.600 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:50.600 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.788575+0000", 2024-09-11T07:50:50.600 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.600 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.600 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.600 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:50.600 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.788575+0000", 2024-09-11T07:50:50.600 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.600 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.600 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.601 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:50.601 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.788574+0000", 2024-09-11T07:50:50.601 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:50.601 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.601 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.601 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:50.601 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.788630+0000", 2024-09-11T07:50:50.601 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6097999999999997e-05 2024-09-11T07:50:50.601 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.601 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.601 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:50.601 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951213+0000", 2024-09-11T07:50:50.601 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16258271199999999 2024-09-11T07:50:50.601 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.602 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.602 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:50.602 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951228+0000", 2024-09-11T07:50:50.602 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4562999999999999e-05 2024-09-11T07:50:50.602 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.602 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.602 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:50.602 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951332+0000", 2024-09-11T07:50:50.602 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000104002 2024-09-11T07:50:50.602 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.602 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.602 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:50.602 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951352+0000", 2024-09-11T07:50:50.602 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0638000000000001e-05 2024-09-11T07:50:50.603 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.603 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.603 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:50.603 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.957755+0000", 2024-09-11T07:50:50.603 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.006402451 2024-09-11T07:50:50.603 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.603 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.603 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:50.603 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.957784+0000", 2024-09-11T07:50:50.603 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9269e-05 2024-09-11T07:50:50.603 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.603 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:50.603 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.603 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.604 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.607 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:70291 10.3 e356/341 10:cdbcfc7c:::benchmark_data_smithi022_163866_object70290:head v 356'4299, mlcod=356'4297)", 2024-09-11T07:50:50.607 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:53.789090+0000", 2024-09-11T07:50:50.607 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 476.779419426, 2024-09-11T07:50:50.607 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16609167599999999, 2024-09-11T07:50:50.607 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:50.608 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:50.608 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:50.608 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.608 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:50.608 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.789090+0000", 2024-09-11T07:50:50.608 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.608 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.608 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.608 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:50.608 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.789090+0000", 2024-09-11T07:50:50.608 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.608 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.608 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.609 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:50.609 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.789089+0000", 2024-09-11T07:50:50.609 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:50.609 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.609 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.609 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:50.609 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.789132+0000", 2024-09-11T07:50:50.609 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2901000000000001e-05 2024-09-11T07:50:50.609 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.609 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.609 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:50.609 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951213+0000", 2024-09-11T07:50:50.609 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.16208165599999999 2024-09-11T07:50:50.610 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.610 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.610 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:50.610 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951228+0000", 2024-09-11T07:50:50.610 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4430000000000001e-05 2024-09-11T07:50:50.610 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.610 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.610 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:50.610 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951350+0000", 2024-09-11T07:50:50.610 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000122213 2024-09-11T07:50:50.610 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.610 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.610 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:50.610 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.951369+0000", 2024-09-11T07:50:50.611 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8819000000000001e-05 2024-09-11T07:50:50.611 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.611 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.611 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:50.611 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.955146+0000", 2024-09-11T07:50:50.611 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003776916 2024-09-11T07:50:50.611 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.611 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.611 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:50.611 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:53.955182+0000", 2024-09-11T07:50:50.611 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6123e-05 2024-09-11T07:50:50.612 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.612 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:50.612 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.612 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.612 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.612 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25990.0:203844 11.a e458/412 11:5dde9a9c:::benchmark_data_smithi022_188626_object84036:head v 458'12896, mlcod=458'12894)", 2024-09-11T07:50:50.612 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:02.188444+0000", 2024-09-11T07:50:50.612 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 108.380065496, 2024-09-11T07:50:50.612 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0082231930000000002, 2024-09-11T07:50:50.612 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:50.612 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:50.612 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:50.612 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.612 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:50.613 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:02.188444+0000", 2024-09-11T07:50:50.613 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.613 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.613 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.613 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:50.613 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:02.188444+0000", 2024-09-11T07:50:50.613 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.613 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.613 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.613 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:50.613 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:02.188443+0000", 2024-09-11T07:50:50.613 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:50.613 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.614 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.614 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:50.614 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:02.188453+0000", 2024-09-11T07:50:50.614 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8800000000000003e-06 2024-09-11T07:50:50.614 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.614 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.614 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:50.614 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:02.188462+0000", 2024-09-11T07:50:50.614 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.2699999999999993e-06 2024-09-11T07:50:50.614 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.614 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.614 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:50.614 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:02.188466+0000", 2024-09-11T07:50:50.615 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8829999999999999e-06 2024-09-11T07:50:50.615 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.615 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.615 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:50.615 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:02.188508+0000", 2024-09-11T07:50:50.615 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1868e-05 2024-09-11T07:50:50.615 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.615 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.615 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:50.615 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:02.188526+0000", 2024-09-11T07:50:50.615 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7830999999999999e-05 2024-09-11T07:50:50.615 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.615 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.615 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:50.616 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:02.196632+0000", 2024-09-11T07:50:50.616 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0081067489999999999 2024-09-11T07:50:50.616 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.616 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.616 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:50.616 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:02.196667+0000", 2024-09-11T07:50:50.616 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4925999999999997e-05 2024-09-11T07:50:50.616 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.616 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:50.616 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.616 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.616 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.617 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25990.0:209545 11.a e461/412 11:5e0df399:::benchmark_data_smithi022_188626_object89737:head v 461'13257, mlcod=461'13254)", 2024-09-11T07:50:50.617 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:09.412707+0000", 2024-09-11T07:50:50.617 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 101.155802858, 2024-09-11T07:50:50.617 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0093356299999999993, 2024-09-11T07:50:50.617 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:50.617 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:50.617 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:50.617 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.617 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:50.617 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.412707+0000", 2024-09-11T07:50:50.617 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.617 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.617 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.618 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:50.618 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.412707+0000", 2024-09-11T07:50:50.618 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.618 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.618 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.618 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:50.618 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.412706+0000", 2024-09-11T07:50:50.618 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:50:50.618 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.618 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.618 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:50.618 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.412715+0000", 2024-09-11T07:50:50.618 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8700000000000004e-06 2024-09-11T07:50:50.618 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.619 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.619 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:50.619 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.412726+0000", 2024-09-11T07:50:50.619 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0971999999999999e-05 2024-09-11T07:50:50.619 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.619 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.619 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:50.619 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.412732+0000", 2024-09-11T07:50:50.619 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.665e-06 2024-09-11T07:50:50.619 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.619 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.619 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:50.619 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.414962+0000", 2024-09-11T07:50:50.620 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002230311 2024-09-11T07:50:50.620 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.620 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.620 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:50.620 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.414982+0000", 2024-09-11T07:50:50.620 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0077000000000001e-05 2024-09-11T07:50:50.620 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.620 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.620 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:50.620 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.422007+0000", 2024-09-11T07:50:50.620 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0070242799999999999 2024-09-11T07:50:50.620 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.620 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.620 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:50.621 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:09.422042+0000", 2024-09-11T07:50:50.621 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5645999999999997e-05 2024-09-11T07:50:50.621 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.621 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:50.621 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.621 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.621 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.621 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25990.0:211016 11.3 e461/460 11:c248ee08:::benchmark_data_smithi022_188626_object91208:head v 461'13123, mlcod=461'13120)", 2024-09-11T07:50:50.621 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:10.505712+0000", 2024-09-11T07:50:50.621 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 100.06279809, 2024-09-11T07:50:50.621 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.018404352999999998, 2024-09-11T07:50:50.621 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:50.621 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:50.622 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:50.622 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.622 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:50.622 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:10.505712+0000", 2024-09-11T07:50:50.622 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.622 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.622 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.622 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:50.622 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:10.505712+0000", 2024-09-11T07:50:50.622 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.622 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.622 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.622 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:50.623 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:10.505711+0000", 2024-09-11T07:50:50.623 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:50:50.623 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.623 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.623 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:50.623 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:10.517805+0000", 2024-09-11T07:50:50.623 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.012093497999999999 2024-09-11T07:50:50.623 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.623 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.623 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:50.623 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:10.517849+0000", 2024-09-11T07:50:50.623 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4833000000000003e-05 2024-09-11T07:50:50.623 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.623 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.624 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:50.624 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:10.517866+0000", 2024-09-11T07:50:50.624 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6209999999999999e-05 2024-09-11T07:50:50.624 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.624 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.624 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:50.624 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:10.518049+0000", 2024-09-11T07:50:50.624 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018378000000000001 2024-09-11T07:50:50.624 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.624 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.624 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:50.624 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:10.518068+0000", 2024-09-11T07:50:50.624 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.861e-05 2024-09-11T07:50:50.625 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.625 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.625 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:50.625 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:10.524086+0000", 2024-09-11T07:50:50.625 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0060181619999999996 2024-09-11T07:50:50.625 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.625 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.625 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:50.625 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:10.524116+0000", 2024-09-11T07:50:50.625 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.9668000000000001e-05 2024-09-11T07:50:50.625 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.625 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:50.625 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.626 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.626 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.626 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25990.0:213326 11.a e462/412 11:5a597884:::benchmark_data_smithi022_188626_object93518:head v 462'13505, mlcod=462'13503)", 2024-09-11T07:50:50.626 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:12.339170+0000", 2024-09-11T07:50:50.626 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 98.229339960000004, 2024-09-11T07:50:50.626 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0089150870000000004, 2024-09-11T07:50:50.626 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:50.626 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:50.626 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:50.626 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.626 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:50.626 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.339170+0000", 2024-09-11T07:50:50.626 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.626 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.626 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.627 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:50.627 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.339170+0000", 2024-09-11T07:50:50.627 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.627 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.627 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.627 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:50.627 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.339168+0000", 2024-09-11T07:50:50.627 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:50.627 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.627 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.627 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:50.627 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.339177+0000", 2024-09-11T07:50:50.627 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2300000000000008e-06 2024-09-11T07:50:50.627 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.628 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.628 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:50.628 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.339191+0000", 2024-09-11T07:50:50.628 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.454e-05 2024-09-11T07:50:50.628 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.628 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.628 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:50.628 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.339197+0000", 2024-09-11T07:50:50.628 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8030000000000002e-06 2024-09-11T07:50:50.628 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.628 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.628 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:50.628 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.339246+0000", 2024-09-11T07:50:50.628 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8819999999999997e-05 2024-09-11T07:50:50.629 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.629 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.629 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:50.629 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.339264+0000", 2024-09-11T07:50:50.629 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8020999999999999e-05 2024-09-11T07:50:50.629 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.629 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.629 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:50.629 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.348018+0000", 2024-09-11T07:50:50.629 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.008754207 2024-09-11T07:50:50.629 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.629 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.629 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:50.629 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.348085+0000", 2024-09-11T07:50:50.630 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6837999999999999e-05 2024-09-11T07:50:50.630 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.630 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:50.630 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.630 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.630 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.630 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25990.0:213753 11.a e462/412 11:53b426d3:::benchmark_data_smithi022_188626_object93945:head v 462'13541, mlcod=462'13538)", 2024-09-11T07:50:50.630 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:12.644596+0000", 2024-09-11T07:50:50.630 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 97.923913764000005, 2024-09-11T07:50:50.630 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0086852509999999997, 2024-09-11T07:50:50.630 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:50.630 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:50.630 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:50.630 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.631 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:50.631 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.644596+0000", 2024-09-11T07:50:50.631 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.631 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.631 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.631 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:50.631 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.644596+0000", 2024-09-11T07:50:50.631 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.631 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.631 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.631 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:50.631 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.644595+0000", 2024-09-11T07:50:50.631 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:50:50.631 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.632 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.632 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:50.632 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.644603+0000", 2024-09-11T07:50:50.632 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2269999999999993e-06 2024-09-11T07:50:50.632 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.632 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.632 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:50.632 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.644611+0000", 2024-09-11T07:50:50.632 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1559999999999995e-06 2024-09-11T07:50:50.632 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.632 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.632 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:50.632 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.644616+0000", 2024-09-11T07:50:50.632 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.293e-06 2024-09-11T07:50:50.633 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.633 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.633 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:50.633 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.646603+0000", 2024-09-11T07:50:50.633 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0019864520000000001 2024-09-11T07:50:50.633 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.633 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.633 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:50.633 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.646622+0000", 2024-09-11T07:50:50.633 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9154e-05 2024-09-11T07:50:50.633 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.633 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.633 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:50.633 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.653215+0000", 2024-09-11T07:50:50.634 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0065931460000000002 2024-09-11T07:50:50.634 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.634 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.634 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:50.634 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:12.653281+0000", 2024-09-11T07:50:50.634 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5884999999999997e-05 2024-09-11T07:50:50.634 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.634 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:50.634 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.634 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.634 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.634 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25990.0:222221 11.c e462/456 11:36408cd8:::benchmark_data_smithi022_188626_object102413:head v 462'13701, mlcod=462'13699)", 2024-09-11T07:50:50.634 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:19.775145+0000", 2024-09-11T07:50:50.635 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 90.793364722000007, 2024-09-11T07:50:50.635 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0082431870000000008, 2024-09-11T07:50:50.635 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:50.635 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:50.635 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:50.635 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.635 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:50.635 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:19.775145+0000", 2024-09-11T07:50:50.635 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.635 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.635 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.635 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:50.635 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:19.775145+0000", 2024-09-11T07:50:50.635 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.636 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.636 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.636 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:50.636 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:19.775143+0000", 2024-09-11T07:50:50.636 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:50.636 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.636 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.636 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:50.636 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:19.775151+0000", 2024-09-11T07:50:50.636 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9119999999999998e-06 2024-09-11T07:50:50.636 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.636 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.636 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:50.637 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:19.775160+0000", 2024-09-11T07:50:50.637 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9369999999999993e-06 2024-09-11T07:50:50.637 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.637 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.637 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:50.637 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:19.775173+0000", 2024-09-11T07:50:50.637 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2979000000000001e-05 2024-09-11T07:50:50.637 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.637 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.637 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:50.637 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:19.776879+0000", 2024-09-11T07:50:50.637 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001706134 2024-09-11T07:50:50.637 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.637 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.638 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:50.638 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:19.776900+0000", 2024-09-11T07:50:50.638 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1026000000000002e-05 2024-09-11T07:50:50.638 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.638 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.638 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:50.638 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:19.783292+0000", 2024-09-11T07:50:50.638 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0063911280000000003 2024-09-11T07:50:50.638 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.638 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.639 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:50.639 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:19.783388+0000", 2024-09-11T07:50:50.639 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.6551000000000004e-05 2024-09-11T07:50:50.639 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.639 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:50.639 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.639 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.639 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.639 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25990.0:222267 11.a e462/412 11:506ef449:::benchmark_data_smithi022_188626_object102459:head v 462'14093, mlcod=462'14090)", 2024-09-11T07:50:50.639 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:19.800771+0000", 2024-09-11T07:50:50.639 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 90.767738171000005, 2024-09-11T07:50:50.639 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0095660250000000006, 2024-09-11T07:50:50.639 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:50.639 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:50.640 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:50.640 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.640 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:50.640 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:19.800771+0000", 2024-09-11T07:50:50.640 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.640 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.640 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.640 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:50.640 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:19.800771+0000", 2024-09-11T07:50:50.640 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.640 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.640 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.640 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:50.641 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:19.800770+0000", 2024-09-11T07:50:50.641 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:50.641 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.641 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.641 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:50.641 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:19.800781+0000", 2024-09-11T07:50:50.641 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0875e-05 2024-09-11T07:50:50.641 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.641 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.641 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:50.641 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:19.800793+0000", 2024-09-11T07:50:50.641 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2248e-05 2024-09-11T07:50:50.641 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.641 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.642 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:50.642 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:19.800799+0000", 2024-09-11T07:50:50.642 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8370000000000002e-06 2024-09-11T07:50:50.642 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.642 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.642 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:50.642 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:19.800860+0000", 2024-09-11T07:50:50.642 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0801999999999998e-05 2024-09-11T07:50:50.642 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.642 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.642 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:50.642 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:19.800879+0000", 2024-09-11T07:50:50.642 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8851e-05 2024-09-11T07:50:50.642 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.643 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.643 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:50.643 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:19.810145+0000", 2024-09-11T07:50:50.643 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0092665290000000008 2024-09-11T07:50:50.643 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.643 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.643 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:50.643 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:19.810337+0000", 2024-09-11T07:50:50.643 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019239800000000001 2024-09-11T07:50:50.643 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.643 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:50.643 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.643 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.643 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.644 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25990.0:222703 11.a e462/412 11:54f38b7e:::benchmark_data_smithi022_188626_object102895:head v 462'14117, mlcod=462'14115)", 2024-09-11T07:50:50.644 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:20.122408+0000", 2024-09-11T07:50:50.644 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 90.446101351999999, 2024-09-11T07:50:50.644 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0088307439999999997, 2024-09-11T07:50:50.644 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:50.644 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:50.644 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:50.644 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.644 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:50.644 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.122408+0000", 2024-09-11T07:50:50.644 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.644 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.644 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.644 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:50.645 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.122408+0000", 2024-09-11T07:50:50.645 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.645 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.645 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.645 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:50.645 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.122407+0000", 2024-09-11T07:50:50.645 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:50.645 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.645 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.645 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:50.645 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.122416+0000", 2024-09-11T07:50:50.645 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.1800000000000002e-06 2024-09-11T07:50:50.645 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.645 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.646 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:50.646 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.122429+0000", 2024-09-11T07:50:50.646 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2561999999999999e-05 2024-09-11T07:50:50.646 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.646 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.646 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:50.646 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.122435+0000", 2024-09-11T07:50:50.646 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2539999999999998e-06 2024-09-11T07:50:50.646 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.646 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.646 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:50.646 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.122477+0000", 2024-09-11T07:50:50.646 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2423999999999999e-05 2024-09-11T07:50:50.646 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.647 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.647 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:50.647 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.122499+0000", 2024-09-11T07:50:50.647 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1951999999999999e-05 2024-09-11T07:50:50.647 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.647 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.647 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:50.647 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.131121+0000", 2024-09-11T07:50:50.647 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0086217229999999995 2024-09-11T07:50:50.647 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.647 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.647 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:50.647 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.131239+0000", 2024-09-11T07:50:50.647 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000118136 2024-09-11T07:50:50.648 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.648 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:50.648 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.648 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.648 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.648 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25990.0:222705 11.d e462/456 11:b99626ae:::benchmark_data_smithi022_188626_object102897:head v 462'13844, mlcod=462'13841)", 2024-09-11T07:50:50.648 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:20.123188+0000", 2024-09-11T07:50:50.648 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 90.445321587999999, 2024-09-11T07:50:50.648 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.010153669000000001, 2024-09-11T07:50:50.648 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:50.648 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:50.648 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:50.648 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.648 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:50.649 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.123188+0000", 2024-09-11T07:50:50.649 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.649 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.649 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.649 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:50.649 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.123188+0000", 2024-09-11T07:50:50.649 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.649 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.649 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.649 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:50.649 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.123187+0000", 2024-09-11T07:50:50.649 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:50.649 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.649 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.650 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:50.650 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.123195+0000", 2024-09-11T07:50:50.650 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1510000000000004e-06 2024-09-11T07:50:50.650 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.650 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.650 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:50.650 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.123209+0000", 2024-09-11T07:50:50.650 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3777999999999999e-05 2024-09-11T07:50:50.650 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.650 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.650 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:50.650 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.123214+0000", 2024-09-11T07:50:50.650 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6420000000000004e-06 2024-09-11T07:50:50.651 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.651 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.651 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:50.651 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.123267+0000", 2024-09-11T07:50:50.651 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2349999999999999e-05 2024-09-11T07:50:50.651 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.651 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.651 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:50.651 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.123284+0000", 2024-09-11T07:50:50.651 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7439999999999999e-05 2024-09-11T07:50:50.651 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.651 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.651 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:50.652 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.133236+0000", 2024-09-11T07:50:50.652 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0099520570000000003 2024-09-11T07:50:50.652 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.652 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.652 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:50.652 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.133342+0000", 2024-09-11T07:50:50.652 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010565000000000001 2024-09-11T07:50:50.652 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.652 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:50.652 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.652 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.652 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.652 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(osd.0.0:308201 4.1 e482/424 4:80b6fe23:reshard::reshard.0000000011:head v 482'44, mlcod=426'41)", 2024-09-11T07:50:50.652 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:50:28.590011+0000", 2024-09-11T07:50:50.653 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 21.978498352999999, 2024-09-11T07:50:50.653 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001330159, 2024-09-11T07:50:50.653 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:50.653 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:50.653 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:50.653 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.653 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:50.653 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590011+0000", 2024-09-11T07:50:50.653 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.653 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.653 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.653 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:50.653 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590011+0000", 2024-09-11T07:50:50.653 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.654 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.654 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.654 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:50.654 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590010+0000", 2024-09-11T07:50:50.654 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:50.654 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.654 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.654 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:50.654 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590022+0000", 2024-09-11T07:50:50.654 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.171e-05 2024-09-11T07:50:50.654 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.654 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.654 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:50.655 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590031+0000", 2024-09-11T07:50:50.655 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.1379999999999996e-06 2024-09-11T07:50:50.655 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.655 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.655 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:50.655 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590036+0000", 2024-09-11T07:50:50.655 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3290000000000001e-06 2024-09-11T07:50:50.655 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.655 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.655 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:50.655 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590140+0000", 2024-09-11T07:50:50.655 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000104073 2024-09-11T07:50:50.655 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.655 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.656 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:50.656 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590155+0000", 2024-09-11T07:50:50.656 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.501e-05 2024-09-11T07:50:50.656 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.656 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.656 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:50.656 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.591308+0000", 2024-09-11T07:50:50.656 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0011529859999999999 2024-09-11T07:50:50.656 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.656 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.656 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:50.656 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.591341+0000", 2024-09-11T07:50:50.656 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3300999999999998e-05 2024-09-11T07:50:50.656 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.657 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:50.657 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.657 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.657 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.657 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(osd.0.0:308202 4.1 e482/424 4:80d23011:::obj_delete_at_hint.0000000085:head v 482'45, mlcod=426'41)", 2024-09-11T07:50:50.657 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:50:28.590200+0000", 2024-09-11T07:50:50.657 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 21.978309460999998, 2024-09-11T07:50:50.657 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001165506, 2024-09-11T07:50:50.657 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:50.657 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:50.657 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:50.657 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.657 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:50.657 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590200+0000", 2024-09-11T07:50:50.658 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.658 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.658 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.658 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:50.658 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590200+0000", 2024-09-11T07:50:50.658 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.658 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.658 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.658 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:50.658 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590199+0000", 2024-09-11T07:50:50.658 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:50.658 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.658 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.658 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:50.659 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590208+0000", 2024-09-11T07:50:50.659 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5990000000000008e-06 2024-09-11T07:50:50.659 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.659 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.659 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:50.659 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590218+0000", 2024-09-11T07:50:50.659 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.4930000000000004e-06 2024-09-11T07:50:50.659 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.659 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.659 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:50.659 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590224+0000", 2024-09-11T07:50:50.659 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1260000000000003e-06 2024-09-11T07:50:50.659 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.659 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.660 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:50.660 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590310+0000", 2024-09-11T07:50:50.660 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5772999999999993e-05 2024-09-11T07:50:50.660 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.660 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.660 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:50.660 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590328+0000", 2024-09-11T07:50:50.660 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8377e-05 2024-09-11T07:50:50.660 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.660 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.660 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:50.660 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.591345+0000", 2024-09-11T07:50:50.660 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0010165899999999999 2024-09-11T07:50:50.660 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.661 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.661 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:50.661 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.591366+0000", 2024-09-11T07:50:50.661 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1058999999999999e-05 2024-09-11T07:50:50.661 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.661 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:50.661 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.661 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.661 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.661 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(osd.0.0:308203 4.1 e482/424 4:8411e274:::obj_delete_at_hint.0000000073:head v 482'46, mlcod=426'41)", 2024-09-11T07:50:50.661 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:50:28.590360+0000", 2024-09-11T07:50:50.661 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 21.978149288000001, 2024-09-11T07:50:50.661 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001056335, 2024-09-11T07:50:50.661 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:50.662 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:50.662 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:50.662 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.662 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:50.662 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590360+0000", 2024-09-11T07:50:50.662 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.662 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.662 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.662 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:50.662 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590360+0000", 2024-09-11T07:50:50.662 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.662 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.662 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.662 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:50.663 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590358+0000", 2024-09-11T07:50:50.663 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:50:50.663 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.663 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.663 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:50.663 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590370+0000", 2024-09-11T07:50:50.663 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1263999999999999e-05 2024-09-11T07:50:50.663 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.663 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.663 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:50.663 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590381+0000", 2024-09-11T07:50:50.663 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1744e-05 2024-09-11T07:50:50.663 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.663 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.664 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:50.664 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590389+0000", 2024-09-11T07:50:50.664 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3560000000000004e-06 2024-09-11T07:50:50.664 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.664 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.664 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:50.664 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590589+0000", 2024-09-11T07:50:50.664 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00019978699999999999 2024-09-11T07:50:50.664 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.664 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.664 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:50.664 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590630+0000", 2024-09-11T07:50:50.664 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.1131000000000003e-05 2024-09-11T07:50:50.665 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.665 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.665 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:50.665 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.591384+0000", 2024-09-11T07:50:50.665 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000754688 2024-09-11T07:50:50.665 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.665 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.665 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:50.665 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.591417+0000", 2024-09-11T07:50:50.665 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2254e-05 2024-09-11T07:50:50.665 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.665 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:50.665 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.665 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.666 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.666 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(osd.0.0:308204 4.1 e482/424 4:84d19476:::obj_delete_at_hint.0000000038:head v 482'47, mlcod=426'41)", 2024-09-11T07:50:50.666 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:50:28.590544+0000", 2024-09-11T07:50:50.666 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 21.977965899000001, 2024-09-11T07:50:50.666 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00089424600000000002, 2024-09-11T07:50:50.666 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:50.666 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:50.666 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:50.666 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.666 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:50.666 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590544+0000", 2024-09-11T07:50:50.666 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.666 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.666 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.667 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:50.667 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590544+0000", 2024-09-11T07:50:50.667 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.667 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.667 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.667 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:50.667 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590542+0000", 2024-09-11T07:50:50.667 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:50.667 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.667 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.667 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:50.667 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590551+0000", 2024-09-11T07:50:50.667 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9150000000000002e-06 2024-09-11T07:50:50.667 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.668 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.668 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:50.668 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590576+0000", 2024-09-11T07:50:50.668 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.5154999999999999e-05 2024-09-11T07:50:50.668 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.668 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.668 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:50.668 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590584+0000", 2024-09-11T07:50:50.668 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.8650000000000001e-06 2024-09-11T07:50:50.668 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.668 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.668 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:50.668 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590814+0000", 2024-09-11T07:50:50.668 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00022997100000000001 2024-09-11T07:50:50.669 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.669 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.669 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:50.669 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.590842+0000", 2024-09-11T07:50:50.669 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.7520000000000001e-05 2024-09-11T07:50:50.669 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.669 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.669 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:50.669 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.591419+0000", 2024-09-11T07:50:50.669 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000577376 2024-09-11T07:50:50.669 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.669 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.669 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:50.669 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:28.591438+0000", 2024-09-11T07:50:50.670 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8958999999999999e-05 2024-09-11T07:50:50.670 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.670 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:50.670 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.670 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.670 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.670 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "MOSDScrubReserve(1.1 REQUEST e483)", 2024-09-11T07:50:50.670 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:50:32.610617+0000", 2024-09-11T07:50:50.670 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 17.957892319999999, 2024-09-11T07:50:50.670 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002888051, 2024-09-11T07:50:50.670 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:50.670 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2024-09-11T07:50:50.670 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:50.670 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.671 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:50.671 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:32.610617+0000", 2024-09-11T07:50:50.671 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.671 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.671 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.671 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:50.671 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:32.610617+0000", 2024-09-11T07:50:50.671 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:50.671 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.671 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.671 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:50.671 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:32.610617+0000", 2024-09-11T07:50:50.671 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:50:50.671 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.672 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.672 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:50.672 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:32.610621+0000", 2024-09-11T07:50:50.672 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0210000000000001e-06 2024-09-11T07:50:50.672 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.672 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.672 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:50.672 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:32.610638+0000", 2024-09-11T07:50:50.672 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7574000000000001e-05 2024-09-11T07:50:50.672 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.672 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.672 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:50.672 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:32.610645+0000", 2024-09-11T07:50:50.672 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9090000000000003e-06 2024-09-11T07:50:50.673 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.673 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.673 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:50.673 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:32.610720+0000", 2024-09-11T07:50:50.673 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4288000000000004e-05 2024-09-11T07:50:50.673 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.673 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.673 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op must wait for map", 2024-09-11T07:50:50.673 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:32.610729+0000", 2024-09-11T07:50:50.673 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.0990000000000005e-06 2024-09-11T07:50:50.673 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.673 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.673 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:50.674 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:32.613451+0000", 2024-09-11T07:50:50.674 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0027224139999999998 2024-09-11T07:50:50.674 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.674 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.674 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:50.674 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:32.613472+0000", 2024-09-11T07:50:50.674 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0530000000000002e-05 2024-09-11T07:50:50.674 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:50.674 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:50.674 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:50.674 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:50:32.613505+0000", 2024-09-11T07:50:50.674 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3658999999999997e-05 2024-09-11T07:50:50.674 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.674 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:50.675 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.675 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:50.675 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:50.675 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:50:50.675 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:50:50 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:50:50.676 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:50:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:50:50] "GET /metrics HTTP/1.1" 200 207456 "" "Prometheus/2.43.0" 2024-09-11T07:50:50.676 INFO:tasks.thrashosds.thrasher:Added osd 4 2024-09-11T07:50:50.676 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-11T07:50:50.677 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-11T07:50:50.677 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd in 2 2024-09-11T07:50:50.968 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:50:51.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:50 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3286370286' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["4"]}]': finished 2024-09-11T07:50:51.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:50 smithi022 ceph-mon[29462]: osdmap e489: 8 total, 8 up, 6 in 2024-09-11T07:50:51.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:50 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:50:51.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:50 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:50:51.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:50 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:50:51.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:50 smithi022 ceph-mon[29462]: osdmap e490: 8 total, 8 up, 6 in 2024-09-11T07:50:51.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:50 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3286370286' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["4"]}]': finished 2024-09-11T07:50:51.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:50 smithi022 ceph-mon[37330]: osdmap e489: 8 total, 8 up, 6 in 2024-09-11T07:50:51.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:50 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:50:51.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:50 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:50:51.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:50 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:50:51.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:50 smithi022 ceph-mon[37330]: osdmap e490: 8 total, 8 up, 6 in 2024-09-11T07:50:51.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:50 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3286370286' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["4"]}]': finished 2024-09-11T07:50:51.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:50 smithi104 ceph-mon[33507]: osdmap e489: 8 total, 8 up, 6 in 2024-09-11T07:50:51.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:50 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:50:51.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:50 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:50:51.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:50 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:50:51.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:50 smithi104 ceph-mon[33507]: osdmap e490: 8 total, 8 up, 6 in 2024-09-11T07:50:51.354 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-11T07:50:51.603 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:50:52.025 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:51 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:50:52.026 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:51 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:50:52.026 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:51 smithi104 ceph-mon[33507]: pgmap v1133: 142 pgs: 13 peering, 2 active+clean+remapped, 127 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:50:52.026 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:51 smithi104 ceph-mon[33507]: osdmap e491: 8 total, 8 up, 6 in 2024-09-11T07:50:52.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:51 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:50:52.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:51 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:50:52.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:51 smithi022 ceph-mon[29462]: pgmap v1133: 142 pgs: 13 peering, 2 active+clean+remapped, 127 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:50:52.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:51 smithi022 ceph-mon[29462]: osdmap e491: 8 total, 8 up, 6 in 2024-09-11T07:50:52.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:51 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:50:52.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:51 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:50:52.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:51 smithi022 ceph-mon[37330]: pgmap v1133: 142 pgs: 13 peering, 2 active+clean+remapped, 127 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:50:52.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:51 smithi022 ceph-mon[37330]: osdmap e491: 8 total, 8 up, 6 in 2024-09-11T07:50:52.885 INFO:teuthology.orchestra.run.smithi022.stderr:marked in osd.2. 2024-09-11T07:50:53.106 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:52 smithi104 ceph-mon[33507]: 6.1b scrub starts 2024-09-11T07:50:53.107 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:52 smithi104 ceph-mon[33507]: 6.1b scrub ok 2024-09-11T07:50:53.107 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:52 smithi104 ceph-mon[33507]: 5.0 deep-scrub starts 2024-09-11T07:50:53.107 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:52 smithi104 ceph-mon[33507]: 5.0 deep-scrub ok 2024-09-11T07:50:53.107 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:52 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2339629392' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["2"]}]: dispatch 2024-09-11T07:50:53.166 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:52 smithi022 ceph-mon[37330]: 6.1b scrub starts 2024-09-11T07:50:53.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:52 smithi022 ceph-mon[37330]: 6.1b scrub ok 2024-09-11T07:50:53.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:52 smithi022 ceph-mon[37330]: 5.0 deep-scrub starts 2024-09-11T07:50:53.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:52 smithi022 ceph-mon[37330]: 5.0 deep-scrub ok 2024-09-11T07:50:53.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:52 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2339629392' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["2"]}]: dispatch 2024-09-11T07:50:53.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:52 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-a[29458]: 2024-09-11T07:50:52.777+0000 7fde5d00f640 -1 mon.a@0(leader).osd e491 definitely_dead 0 2024-09-11T07:50:53.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:52 smithi022 ceph-mon[29462]: 6.1b scrub starts 2024-09-11T07:50:53.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:52 smithi022 ceph-mon[29462]: 6.1b scrub ok 2024-09-11T07:50:53.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:52 smithi022 ceph-mon[29462]: 5.0 deep-scrub starts 2024-09-11T07:50:53.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:52 smithi022 ceph-mon[29462]: 5.0 deep-scrub ok 2024-09-11T07:50:53.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:52 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2339629392' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["2"]}]: dispatch 2024-09-11T07:50:53.336 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:50:53.336 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:50:53.336 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 0 2024-09-11T07:50:53.336 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:50:53.528 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-11T07:50:53.528 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-11T07:50:53.528 INFO:tasks.thrashosds.thrasher:Adding osd 7 2024-09-11T07:50:53.528 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd in 7 2024-09-11T07:50:53.784 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:50:54.012 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-11T07:50:54.163 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:53 smithi022 ceph-mon[29462]: 2.5 scrub starts 2024-09-11T07:50:54.163 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:53 smithi022 ceph-mon[29462]: 2.5 scrub ok 2024-09-11T07:50:54.163 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:53 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2339629392' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["2"]}]': finished 2024-09-11T07:50:54.164 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:53 smithi022 ceph-mon[29462]: osdmap e492: 8 total, 8 up, 7 in 2024-09-11T07:50:54.164 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:50:54.164 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:53 smithi022 ceph-mon[29462]: 6.13 scrub starts 2024-09-11T07:50:54.164 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:53 smithi022 ceph-mon[29462]: 6.13 scrub ok 2024-09-11T07:50:54.164 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:53 smithi022 ceph-mon[29462]: pgmap v1136: 142 pgs: 13 peering, 2 active+clean+remapped, 127 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:50:54.164 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:53 smithi022 ceph-mon[37330]: 2.5 scrub starts 2024-09-11T07:50:54.164 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:53 smithi022 ceph-mon[37330]: 2.5 scrub ok 2024-09-11T07:50:54.164 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:53 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2339629392' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["2"]}]': finished 2024-09-11T07:50:54.164 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:53 smithi022 ceph-mon[37330]: osdmap e492: 8 total, 8 up, 7 in 2024-09-11T07:50:54.164 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:50:54.164 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:53 smithi022 ceph-mon[37330]: 6.13 scrub starts 2024-09-11T07:50:54.165 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:53 smithi022 ceph-mon[37330]: 6.13 scrub ok 2024-09-11T07:50:54.165 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:53 smithi022 ceph-mon[37330]: pgmap v1136: 142 pgs: 13 peering, 2 active+clean+remapped, 127 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:50:54.256 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:50:54.279 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:50:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:50:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:50:54.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:53 smithi104 ceph-mon[33507]: 2.5 scrub starts 2024-09-11T07:50:54.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:53 smithi104 ceph-mon[33507]: 2.5 scrub ok 2024-09-11T07:50:54.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:53 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2339629392' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["2"]}]': finished 2024-09-11T07:50:54.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:53 smithi104 ceph-mon[33507]: osdmap e492: 8 total, 8 up, 7 in 2024-09-11T07:50:54.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:50:54.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:53 smithi104 ceph-mon[33507]: 6.13 scrub starts 2024-09-11T07:50:54.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:53 smithi104 ceph-mon[33507]: 6.13 scrub ok 2024-09-11T07:50:54.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:53 smithi104 ceph-mon[33507]: pgmap v1136: 142 pgs: 13 peering, 2 active+clean+remapped, 127 active+clean; 483 KiB data, 4.3 GiB used, 443 GiB / 447 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:50:54.884 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:54 smithi104 ceph-mon[33507]: osdmap e493: 8 total, 8 up, 7 in 2024-09-11T07:50:55.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:54 smithi022 ceph-mon[29462]: osdmap e493: 8 total, 8 up, 7 in 2024-09-11T07:50:55.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:54 smithi022 ceph-mon[37330]: osdmap e493: 8 total, 8 up, 7 in 2024-09-11T07:50:55.894 INFO:teuthology.orchestra.run.smithi022.stderr:marked in osd.7. 2024-09-11T07:50:55.993 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:55 smithi104 ceph-mon[33507]: pgmap v1138: 142 pgs: 4 peering, 138 active+clean; 483 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 67 KiB/s, 26 objects/s recovering 2024-09-11T07:50:55.993 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:55 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/863263502' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["7"]}]: dispatch 2024-09-11T07:50:56.009 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:50:56.010 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:50:56.010 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2024-09-11T07:50:56.010 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2024-09-11T07:50:56.010 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:50:56.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:55 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-a[29458]: 2024-09-11T07:50:55.675+0000 7fde5d00f640 -1 mon.a@0(leader).osd e493 definitely_dead 0 2024-09-11T07:50:56.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:55 smithi022 ceph-mon[29462]: pgmap v1138: 142 pgs: 4 peering, 138 active+clean; 483 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 67 KiB/s, 26 objects/s recovering 2024-09-11T07:50:56.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:55 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/863263502' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["7"]}]: dispatch 2024-09-11T07:50:56.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:55 smithi022 ceph-mon[37330]: pgmap v1138: 142 pgs: 4 peering, 138 active+clean; 483 KiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 67 KiB/s, 26 objects/s recovering 2024-09-11T07:50:56.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:55 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/863263502' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["7"]}]: dispatch 2024-09-11T07:50:56.675 INFO:tasks.thrashosds.thrasher:Added osd 7 2024-09-11T07:50:56.675 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-11T07:50:56.726 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-11T07:50:56.902 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:56 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/863263502' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["7"]}]': finished 2024-09-11T07:50:56.902 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:56 smithi022 ceph-mon[29462]: osdmap e494: 8 total, 8 up, 8 in 2024-09-11T07:50:56.903 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:56 smithi022 ceph-mon[29462]: 6.1f scrub starts 2024-09-11T07:50:56.903 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:56 smithi022 ceph-mon[29462]: 6.1f scrub ok 2024-09-11T07:50:56.962 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:50:56.991 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:50:57.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:56 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/863263502' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["7"]}]': finished 2024-09-11T07:50:57.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:56 smithi104 ceph-mon[33507]: osdmap e494: 8 total, 8 up, 8 in 2024-09-11T07:50:57.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:56 smithi104 ceph-mon[33507]: 6.1f scrub starts 2024-09-11T07:50:57.030 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:56 smithi104 ceph-mon[33507]: 6.1f scrub ok 2024-09-11T07:50:57.152 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:56 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/863263502' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["7"]}]': finished 2024-09-11T07:50:57.153 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:56 smithi022 ceph-mon[37330]: osdmap e494: 8 total, 8 up, 8 in 2024-09-11T07:50:57.153 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:56 smithi022 ceph-mon[37330]: 6.1f scrub starts 2024-09-11T07:50:57.153 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:56 smithi022 ceph-mon[37330]: 6.1f scrub ok 2024-09-11T07:50:58.154 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:57 smithi022 ceph-mon[29462]: 1.a scrub starts 2024-09-11T07:50:58.154 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:57 smithi022 ceph-mon[29462]: 1.a scrub ok 2024-09-11T07:50:58.154 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:57 smithi022 ceph-mon[29462]: 5.2 scrub starts 2024-09-11T07:50:58.155 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:57 smithi022 ceph-mon[29462]: 5.2 scrub ok 2024-09-11T07:50:58.155 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:57 smithi022 ceph-mon[29462]: osdmap e495: 8 total, 8 up, 8 in 2024-09-11T07:50:58.155 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:57 smithi022 ceph-mon[29462]: 1.3 scrub starts 2024-09-11T07:50:58.155 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:57 smithi022 ceph-mon[29462]: 1.3 scrub ok 2024-09-11T07:50:58.155 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:57 smithi022 ceph-mon[29462]: 4.11 scrub starts 2024-09-11T07:50:58.155 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:57 smithi022 ceph-mon[29462]: 4.11 scrub ok 2024-09-11T07:50:58.155 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:57 smithi022 ceph-mon[29462]: 5.14 scrub starts 2024-09-11T07:50:58.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:57 smithi022 ceph-mon[29462]: 5.14 scrub ok 2024-09-11T07:50:58.156 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:57 smithi022 ceph-mon[29462]: pgmap v1141: 142 pgs: 6 peering, 136 active+clean; 483 KiB data, 5.8 GiB used, 620 GiB / 626 GiB avail; 213 B/s rd, 0 op/s; 175 KiB/s, 31 objects/s recovering 2024-09-11T07:50:58.156 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:57 smithi022 ceph-mon[37330]: 1.a scrub starts 2024-09-11T07:50:58.156 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:57 smithi022 ceph-mon[37330]: 1.a scrub ok 2024-09-11T07:50:58.157 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:57 smithi022 ceph-mon[37330]: 5.2 scrub starts 2024-09-11T07:50:58.157 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:57 smithi022 ceph-mon[37330]: 5.2 scrub ok 2024-09-11T07:50:58.157 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:57 smithi022 ceph-mon[37330]: osdmap e495: 8 total, 8 up, 8 in 2024-09-11T07:50:58.157 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:57 smithi022 ceph-mon[37330]: 1.3 scrub starts 2024-09-11T07:50:58.157 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:57 smithi022 ceph-mon[37330]: 1.3 scrub ok 2024-09-11T07:50:58.157 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:57 smithi022 ceph-mon[37330]: 4.11 scrub starts 2024-09-11T07:50:58.158 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:57 smithi022 ceph-mon[37330]: 4.11 scrub ok 2024-09-11T07:50:58.158 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:57 smithi022 ceph-mon[37330]: 5.14 scrub starts 2024-09-11T07:50:58.158 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:57 smithi022 ceph-mon[37330]: 5.14 scrub ok 2024-09-11T07:50:58.158 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:57 smithi022 ceph-mon[37330]: pgmap v1141: 142 pgs: 6 peering, 136 active+clean; 483 KiB data, 5.8 GiB used, 620 GiB / 626 GiB avail; 213 B/s rd, 0 op/s; 175 KiB/s, 31 objects/s recovering 2024-09-11T07:50:58.259 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:57 smithi104 ceph-mon[33507]: 1.a scrub starts 2024-09-11T07:50:58.259 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:57 smithi104 ceph-mon[33507]: 1.a scrub ok 2024-09-11T07:50:58.259 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:57 smithi104 ceph-mon[33507]: 5.2 scrub starts 2024-09-11T07:50:58.260 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:57 smithi104 ceph-mon[33507]: 5.2 scrub ok 2024-09-11T07:50:58.260 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:57 smithi104 ceph-mon[33507]: osdmap e495: 8 total, 8 up, 8 in 2024-09-11T07:50:58.260 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:57 smithi104 ceph-mon[33507]: 1.3 scrub starts 2024-09-11T07:50:58.260 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:57 smithi104 ceph-mon[33507]: 1.3 scrub ok 2024-09-11T07:50:58.260 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:57 smithi104 ceph-mon[33507]: 4.11 scrub starts 2024-09-11T07:50:58.260 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:57 smithi104 ceph-mon[33507]: 4.11 scrub ok 2024-09-11T07:50:58.260 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:57 smithi104 ceph-mon[33507]: 5.14 scrub starts 2024-09-11T07:50:58.260 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:57 smithi104 ceph-mon[33507]: 5.14 scrub ok 2024-09-11T07:50:58.260 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:57 smithi104 ceph-mon[33507]: pgmap v1141: 142 pgs: 6 peering, 136 active+clean; 483 KiB data, 5.8 GiB used, 620 GiB / 626 GiB avail; 213 B/s rd, 0 op/s; 175 KiB/s, 31 objects/s recovering 2024-09-11T07:50:58.654 INFO:teuthology.orchestra.run.smithi022.stdout:mon.a: {} 2024-09-11T07:50:58.655 INFO:teuthology.orchestra.run.smithi022.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T07:50:58.655 INFO:teuthology.orchestra.run.smithi022.stdout:mon.b: {} 2024-09-11T07:50:58.655 INFO:teuthology.orchestra.run.smithi022.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T07:50:58.664 INFO:teuthology.orchestra.run.smithi022.stdout:mon.c: {} 2024-09-11T07:50:58.664 INFO:teuthology.orchestra.run.smithi022.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T07:50:58.858 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:50:58.858 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2024-09-11T07:50:58.858 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2024-09-11T07:50:58.858 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2024-09-11T07:50:58.858 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.858 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:13784 10.7 e352/341 10:eca93056:::benchmark_data_smithi022_163866_object13783:head v 352'849, mlcod=352'846)", 2024-09-11T07:50:58.859 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:20.170173+0000", 2024-09-11T07:50:58.859 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 518.68532992600001, 2024-09-11T07:50:58.859 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.135244226, 2024-09-11T07:50:58.859 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:58.859 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:58.859 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:58.859 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.859 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:58.859 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.170173+0000", 2024-09-11T07:50:58.859 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.859 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.859 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.860 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:58.860 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.170173+0000", 2024-09-11T07:50:58.860 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.860 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.860 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.860 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:58.860 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.170172+0000", 2024-09-11T07:50:58.860 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:58.860 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.860 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.860 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:58.861 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.170271+0000", 2024-09-11T07:50:58.861 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.9666000000000004e-05 2024-09-11T07:50:58.861 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.861 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.861 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:58.861 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.170282+0000", 2024-09-11T07:50:58.861 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0937e-05 2024-09-11T07:50:58.861 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.861 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.861 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:58.861 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.170288+0000", 2024-09-11T07:50:58.861 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9649999999999996e-06 2024-09-11T07:50:58.862 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.862 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.862 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:58.862 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.170325+0000", 2024-09-11T07:50:58.862 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6823000000000003e-05 2024-09-11T07:50:58.862 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.862 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.862 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:58.862 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.170344+0000", 2024-09-11T07:50:58.862 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9123999999999999e-05 2024-09-11T07:50:58.862 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.862 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.863 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:58.863 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.305378+0000", 2024-09-11T07:50:58.863 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13503420499999999 2024-09-11T07:50:58.863 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.863 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.863 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:58.863 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.305417+0000", 2024-09-11T07:50:58.863 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8887999999999998e-05 2024-09-11T07:50:58.863 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.863 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:58.863 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.863 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.863 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.864 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:13783 10.a e352/341 10:526721f4:::benchmark_data_smithi022_163866_object13782:head v 352'847, mlcod=352'845)", 2024-09-11T07:50:58.864 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:20.170236+0000", 2024-09-11T07:50:58.864 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 518.68526661999999, 2024-09-11T07:50:58.864 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13539943099999999, 2024-09-11T07:50:58.864 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:58.864 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:58.864 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:58.864 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.864 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:58.864 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.170236+0000", 2024-09-11T07:50:58.864 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.865 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.865 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.865 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:58.865 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.170236+0000", 2024-09-11T07:50:58.865 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.865 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.865 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.865 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:58.865 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.170235+0000", 2024-09-11T07:50:58.865 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:50:58.865 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.865 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.865 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:58.866 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.170287+0000", 2024-09-11T07:50:58.866 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1264000000000002e-05 2024-09-11T07:50:58.866 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.866 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.866 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:58.866 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.170296+0000", 2024-09-11T07:50:58.866 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.6530000000000006e-06 2024-09-11T07:50:58.866 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.866 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.866 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:58.866 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.170305+0000", 2024-09-11T07:50:58.866 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8219999999999996e-06 2024-09-11T07:50:58.867 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.867 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.867 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:58.867 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.170346+0000", 2024-09-11T07:50:58.867 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0957000000000001e-05 2024-09-11T07:50:58.867 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.867 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.867 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:58.867 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.170364+0000", 2024-09-11T07:50:58.867 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7543000000000001e-05 2024-09-11T07:50:58.867 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.867 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.867 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:58.867 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.305592+0000", 2024-09-11T07:50:58.868 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.135228716 2024-09-11T07:50:58.868 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.868 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.868 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:58.868 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.305636+0000", 2024-09-11T07:50:58.868 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3631999999999998e-05 2024-09-11T07:50:58.868 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.868 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:58.868 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.868 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.868 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.868 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:13789 10.b e352/341 10:d9364373:::benchmark_data_smithi022_163866_object13788:head v 352'821, mlcod=352'819)", 2024-09-11T07:50:58.868 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:20.171532+0000", 2024-09-11T07:50:58.869 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 518.68397150800001, 2024-09-11T07:50:58.869 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13276058099999999, 2024-09-11T07:50:58.869 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:58.869 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:58.869 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:58.869 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.869 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:58.869 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.171532+0000", 2024-09-11T07:50:58.869 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.869 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.869 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.869 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:58.869 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.171532+0000", 2024-09-11T07:50:58.869 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.870 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.870 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.870 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:58.870 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.171530+0000", 2024-09-11T07:50:58.870 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:58.870 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.870 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.870 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:58.870 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.171582+0000", 2024-09-11T07:50:58.870 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1635999999999998e-05 2024-09-11T07:50:58.870 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.870 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.870 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:58.870 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.303199+0000", 2024-09-11T07:50:58.870 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13161684300000001 2024-09-11T07:50:58.871 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.871 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.871 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:58.871 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.303211+0000", 2024-09-11T07:50:58.871 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2595000000000001e-05 2024-09-11T07:50:58.871 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.871 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.871 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:58.871 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.303413+0000", 2024-09-11T07:50:58.871 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000201483 2024-09-11T07:50:58.871 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.871 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.871 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:58.871 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.303436+0000", 2024-09-11T07:50:58.872 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3578999999999999e-05 2024-09-11T07:50:58.872 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.872 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.872 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:58.872 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.304267+0000", 2024-09-11T07:50:58.872 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00083087799999999996 2024-09-11T07:50:58.872 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.872 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.872 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:58.872 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.304292+0000", 2024-09-11T07:50:58.872 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.4888e-05 2024-09-11T07:50:58.872 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.872 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:58.872 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.872 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.873 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.873 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:13793 10.2 e352/341 10:4541faa0:::benchmark_data_smithi022_163866_object13792:head v 352'875, mlcod=352'873)", 2024-09-11T07:50:58.873 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:20.171669+0000", 2024-09-11T07:50:58.873 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 518.68383454499997, 2024-09-11T07:50:58.873 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.133992955, 2024-09-11T07:50:58.873 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:58.873 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:58.873 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:58.873 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.873 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:58.873 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.171669+0000", 2024-09-11T07:50:58.873 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.873 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.874 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.874 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:58.874 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.171669+0000", 2024-09-11T07:50:58.874 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.874 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.874 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.874 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:58.874 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.171667+0000", 2024-09-11T07:50:58.874 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:50:58.874 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.874 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.874 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:58.874 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.171690+0000", 2024-09-11T07:50:58.875 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.2524e-05 2024-09-11T07:50:58.875 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.875 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.875 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:58.875 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.303279+0000", 2024-09-11T07:50:58.875 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.13158864200000001 2024-09-11T07:50:58.875 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.875 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.875 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:58.875 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.303286+0000", 2024-09-11T07:50:58.875 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5429999999999996e-06 2024-09-11T07:50:58.875 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.875 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.875 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:58.876 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.303563+0000", 2024-09-11T07:50:58.876 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00027712599999999999 2024-09-11T07:50:58.876 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.876 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.876 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:58.876 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.303592+0000", 2024-09-11T07:50:58.876 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.8634999999999999e-05 2024-09-11T07:50:58.876 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.876 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.876 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:58.876 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.305642+0000", 2024-09-11T07:50:58.876 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0020500100000000001 2024-09-11T07:50:58.876 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.877 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.877 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:58.877 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.305661+0000", 2024-09-11T07:50:58.877 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9514e-05 2024-09-11T07:50:58.877 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.877 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:58.877 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.877 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.877 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.877 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:13799 10.f e352/341 10:f7c07202:::benchmark_data_smithi022_163866_object13798:head v 352'859, mlcod=352'857)", 2024-09-11T07:50:58.877 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:20.176441+0000", 2024-09-11T07:50:58.877 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 518.67906181000001, 2024-09-11T07:50:58.877 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.12934169400000001, 2024-09-11T07:50:58.878 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:58.878 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:58.878 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:58.878 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.878 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:58.878 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.176441+0000", 2024-09-11T07:50:58.878 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.878 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.878 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.878 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:58.878 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.176441+0000", 2024-09-11T07:50:58.878 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.878 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.878 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.879 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:58.879 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.176439+0000", 2024-09-11T07:50:58.879 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:50:58.879 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.879 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.879 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:58.879 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.176544+0000", 2024-09-11T07:50:58.879 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010466700000000001 2024-09-11T07:50:58.879 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.879 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.879 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:58.879 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.303195+0000", 2024-09-11T07:50:58.879 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.12665132400000001 2024-09-11T07:50:58.879 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.880 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.880 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:58.880 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.303212+0000", 2024-09-11T07:50:58.880 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6707000000000001e-05 2024-09-11T07:50:58.880 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.880 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.880 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:58.880 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.304382+0000", 2024-09-11T07:50:58.880 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0011703970000000001 2024-09-11T07:50:58.880 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.880 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.880 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:58.880 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.304400+0000", 2024-09-11T07:50:58.881 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7700999999999999e-05 2024-09-11T07:50:58.881 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.881 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.881 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:58.881 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.305760+0000", 2024-09-11T07:50:58.881 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.001360535 2024-09-11T07:50:58.881 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.881 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.881 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:58.881 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:20.305783+0000", 2024-09-11T07:50:58.881 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.2480000000000002e-05 2024-09-11T07:50:58.881 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.882 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:58.882 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.882 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.882 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.882 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:19868 10.2 e353/341 10:4b68d96e:::benchmark_data_smithi022_163866_object19867:head v 353'1242, mlcod=353'1239)", 2024-09-11T07:50:58.882 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:22.737367+0000", 2024-09-11T07:50:58.882 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 516.11813649299995, 2024-09-11T07:50:58.882 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.083302861000000006, 2024-09-11T07:50:58.882 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:58.882 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:58.882 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:58.882 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.882 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:58.882 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.737367+0000", 2024-09-11T07:50:58.883 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.883 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.883 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.883 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:58.883 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.737367+0000", 2024-09-11T07:50:58.883 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.883 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.883 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.883 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:58.883 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.737366+0000", 2024-09-11T07:50:58.883 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:50:58.883 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.883 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.884 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:58.884 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.737391+0000", 2024-09-11T07:50:58.884 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.5604000000000001e-05 2024-09-11T07:50:58.884 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.884 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.884 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:58.884 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.737401+0000", 2024-09-11T07:50:58.884 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.6660000000000003e-06 2024-09-11T07:50:58.884 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.884 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.884 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:58.884 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.737407+0000", 2024-09-11T07:50:58.884 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5620000000000003e-06 2024-09-11T07:50:58.885 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.885 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.885 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:58.885 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.737454+0000", 2024-09-11T07:50:58.885 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7800000000000003e-05 2024-09-11T07:50:58.885 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.885 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.885 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:58.885 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.737472+0000", 2024-09-11T07:50:58.885 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7280999999999999e-05 2024-09-11T07:50:58.885 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.885 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.885 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:58.886 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.820634+0000", 2024-09-11T07:50:58.886 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.083162050000000001 2024-09-11T07:50:58.886 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.886 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.886 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:58.886 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.820669+0000", 2024-09-11T07:50:58.886 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5799000000000001e-05 2024-09-11T07:50:58.886 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.886 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:58.886 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.886 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.886 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.886 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:19869 10.2 e353/341 10:4ec5be85:::benchmark_data_smithi022_163866_object19868:head v 353'1243, mlcod=353'1239)", 2024-09-11T07:50:58.886 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:22.737854+0000", 2024-09-11T07:50:58.886 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 516.11764919799998, 2024-09-11T07:50:58.887 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.084258564999999994, 2024-09-11T07:50:58.887 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:58.887 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:58.887 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:58.887 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.887 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:58.887 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.737854+0000", 2024-09-11T07:50:58.887 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.887 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.887 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.887 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:58.887 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.737854+0000", 2024-09-11T07:50:58.889 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.889 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.889 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.889 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:58.889 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.737853+0000", 2024-09-11T07:50:58.889 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:50:58.889 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.889 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.889 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:58.889 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.737872+0000", 2024-09-11T07:50:58.890 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9037000000000001e-05 2024-09-11T07:50:58.890 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.890 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.890 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:58.890 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.737878+0000", 2024-09-11T07:50:58.890 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7960000000000001e-06 2024-09-11T07:50:58.890 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.890 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.890 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:58.890 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.737882+0000", 2024-09-11T07:50:58.890 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3529999999999999e-06 2024-09-11T07:50:58.890 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.890 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.890 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:58.891 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.819155+0000", 2024-09-11T07:50:58.891 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.081272901999999994 2024-09-11T07:50:58.891 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.891 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.891 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:58.891 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.819175+0000", 2024-09-11T07:50:58.891 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0445000000000002e-05 2024-09-11T07:50:58.891 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.891 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.891 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:58.891 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.822078+0000", 2024-09-11T07:50:58.891 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0029033969999999998 2024-09-11T07:50:58.891 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.892 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.892 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:58.892 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.822112+0000", 2024-09-11T07:50:58.892 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4091999999999998e-05 2024-09-11T07:50:58.892 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.892 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:58.892 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.892 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.892 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.892 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:19871 10.d e353/341 10:bbd679fd:::benchmark_data_smithi022_163866_object19870:head v 353'1216, mlcod=353'1212)", 2024-09-11T07:50:58.892 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:22.738024+0000", 2024-09-11T07:50:58.892 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 516.11747884800002, 2024-09-11T07:50:58.892 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.080969418000000001, 2024-09-11T07:50:58.892 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:58.893 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:58.893 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:58.893 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.893 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:58.893 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.738024+0000", 2024-09-11T07:50:58.893 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.893 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.893 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.893 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:58.893 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.738024+0000", 2024-09-11T07:50:58.893 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.893 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.893 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.894 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:58.894 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.738023+0000", 2024-09-11T07:50:58.894 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:50:58.894 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.894 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.894 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:58.894 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.738080+0000", 2024-09-11T07:50:58.894 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7417999999999998e-05 2024-09-11T07:50:58.894 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.894 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.894 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:58.894 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.738098+0000", 2024-09-11T07:50:58.894 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7101000000000001e-05 2024-09-11T07:50:58.894 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.895 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.895 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:58.895 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.738102+0000", 2024-09-11T07:50:58.895 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7260000000000002e-06 2024-09-11T07:50:58.895 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.895 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.895 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:58.895 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.738138+0000", 2024-09-11T07:50:58.895 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5816999999999999e-05 2024-09-11T07:50:58.895 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.895 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.895 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:58.895 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.738155+0000", 2024-09-11T07:50:58.895 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6875999999999999e-05 2024-09-11T07:50:58.895 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.896 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.896 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:58.896 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.738950+0000", 2024-09-11T07:50:58.896 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00079533200000000005 2024-09-11T07:50:58.896 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.896 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.896 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:58.896 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.818994+0000", 2024-09-11T07:50:58.896 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.080043305999999995 2024-09-11T07:50:58.896 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.896 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:58.896 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.896 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.896 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.896 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:19873 10.b e353/341 10:d4a898fb:::benchmark_data_smithi022_163866_object19872:head v 353'1189, mlcod=353'1186)", 2024-09-11T07:50:58.897 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:22.738086+0000", 2024-09-11T07:50:58.897 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 516.11741682900004, 2024-09-11T07:50:58.897 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.080980202000000001, 2024-09-11T07:50:58.897 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:58.897 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:58.897 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:58.897 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.897 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:58.897 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.738086+0000", 2024-09-11T07:50:58.897 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.897 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.897 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.897 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:58.897 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.738086+0000", 2024-09-11T07:50:58.897 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.898 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.898 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.898 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:58.898 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.738085+0000", 2024-09-11T07:50:58.898 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:50:58.898 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.898 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.898 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:58.898 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.738148+0000", 2024-09-11T07:50:58.898 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3040000000000006e-05 2024-09-11T07:50:58.898 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.898 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.898 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:58.898 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.738156+0000", 2024-09-11T07:50:58.899 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2609999999999993e-06 2024-09-11T07:50:58.899 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.899 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.899 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:58.899 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.738160+0000", 2024-09-11T07:50:58.899 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.709e-06 2024-09-11T07:50:58.899 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.899 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.899 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:58.899 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.738205+0000", 2024-09-11T07:50:58.899 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5065000000000003e-05 2024-09-11T07:50:58.899 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.899 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.899 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:58.900 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.738223+0000", 2024-09-11T07:50:58.900 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7927e-05 2024-09-11T07:50:58.900 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.900 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.900 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:58.900 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.819005+0000", 2024-09-11T07:50:58.900 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.080781758999999995 2024-09-11T07:50:58.900 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.900 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.900 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:58.900 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.819066+0000", 2024-09-11T07:50:58.900 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1595999999999998e-05 2024-09-11T07:50:58.900 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.901 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:58.901 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.901 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.901 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.901 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:19874 10.f e353/341 10:f54346e0:::benchmark_data_smithi022_163866_object19873:head v 353'1244, mlcod=353'1242)", 2024-09-11T07:50:58.901 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:22.738219+0000", 2024-09-11T07:50:58.901 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 516.11728416100004, 2024-09-11T07:50:58.901 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.081764044999999994, 2024-09-11T07:50:58.901 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:58.901 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:58.901 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:58.901 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.901 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:58.901 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.738219+0000", 2024-09-11T07:50:58.901 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.902 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.902 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.902 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:58.902 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.738219+0000", 2024-09-11T07:50:58.902 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.902 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.902 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.902 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:58.902 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.738218+0000", 2024-09-11T07:50:58.902 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:50:58.902 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.902 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.902 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:58.902 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.738298+0000", 2024-09-11T07:50:58.902 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9745000000000003e-05 2024-09-11T07:50:58.903 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.903 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.903 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:58.903 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.738306+0000", 2024-09-11T07:50:58.903 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1839999999999999e-06 2024-09-11T07:50:58.903 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.903 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.903 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:58.903 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.738310+0000", 2024-09-11T07:50:58.903 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4580000000000003e-06 2024-09-11T07:50:58.903 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.903 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.903 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:58.903 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.738355+0000", 2024-09-11T07:50:58.904 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5161999999999999e-05 2024-09-11T07:50:58.904 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.904 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.904 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:58.904 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.738372+0000", 2024-09-11T07:50:58.904 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6884e-05 2024-09-11T07:50:58.904 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.904 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.904 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:58.904 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.819940+0000", 2024-09-11T07:50:58.904 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.081568059999999998 2024-09-11T07:50:58.904 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.904 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.904 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:58.904 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.819983+0000", 2024-09-11T07:50:58.905 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2639999999999998e-05 2024-09-11T07:50:58.905 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.905 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:58.905 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.905 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.905 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.905 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:19875 10.b e353/341 10:d467c87a:::benchmark_data_smithi022_163866_object19874:head v 353'1190, mlcod=353'1186)", 2024-09-11T07:50:58.905 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:22.739302+0000", 2024-09-11T07:50:58.905 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 516.11620093399995, 2024-09-11T07:50:58.905 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.081659120000000002, 2024-09-11T07:50:58.905 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:58.905 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:58.905 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:58.905 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.905 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:58.905 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.739302+0000", 2024-09-11T07:50:58.906 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.906 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.906 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.906 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:58.906 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.739302+0000", 2024-09-11T07:50:58.906 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.906 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.906 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.906 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:58.906 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.739301+0000", 2024-09-11T07:50:58.906 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:58.906 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.906 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.906 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:58.906 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.739360+0000", 2024-09-11T07:50:58.907 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.872e-05 2024-09-11T07:50:58.907 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.907 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.907 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:58.907 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.818976+0000", 2024-09-11T07:50:58.907 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.079616452000000004 2024-09-11T07:50:58.907 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.907 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.907 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:58.907 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.818999+0000", 2024-09-11T07:50:58.907 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3366999999999998e-05 2024-09-11T07:50:58.907 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.907 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.907 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:58.907 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.819088+0000", 2024-09-11T07:50:58.907 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8502000000000001e-05 2024-09-11T07:50:58.908 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.908 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.908 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:58.908 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.819102+0000", 2024-09-11T07:50:58.908 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4422e-05 2024-09-11T07:50:58.908 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.908 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.908 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:58.908 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.820916+0000", 2024-09-11T07:50:58.908 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0018136719999999999 2024-09-11T07:50:58.908 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.908 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.908 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:58.908 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.820961+0000", 2024-09-11T07:50:58.909 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5206000000000001e-05 2024-09-11T07:50:58.909 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.909 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:58.909 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.909 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.909 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.909 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:19882 10.a e353/341 10:56f294dc:::benchmark_data_smithi022_163866_object19881:head v 353'1225, mlcod=353'1223)", 2024-09-11T07:50:58.909 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:22.741832+0000", 2024-09-11T07:50:58.909 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 516.11367153699996, 2024-09-11T07:50:58.909 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.080312070999999999, 2024-09-11T07:50:58.909 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:58.909 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:58.909 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:58.909 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.910 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:58.910 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.741832+0000", 2024-09-11T07:50:58.910 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.910 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.910 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.910 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:58.910 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.741832+0000", 2024-09-11T07:50:58.910 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.910 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.910 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.910 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:58.910 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.741830+0000", 2024-09-11T07:50:58.910 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:58.911 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.911 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.911 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:58.911 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.741881+0000", 2024-09-11T07:50:58.911 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.1496999999999998e-05 2024-09-11T07:50:58.911 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.911 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.911 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:58.911 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.819027+0000", 2024-09-11T07:50:58.911 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.077145477000000004 2024-09-11T07:50:58.911 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.911 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.912 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:58.912 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.819032+0000", 2024-09-11T07:50:58.912 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3909999999999997e-06 2024-09-11T07:50:58.912 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.912 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.912 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:58.912 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.819957+0000", 2024-09-11T07:50:58.912 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00092476099999999996 2024-09-11T07:50:58.912 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.912 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.912 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:58.912 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.819975+0000", 2024-09-11T07:50:58.912 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8199999999999999e-05 2024-09-11T07:50:58.912 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.913 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.913 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:58.913 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.822116+0000", 2024-09-11T07:50:58.913 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002141109 2024-09-11T07:50:58.913 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.913 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.913 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:58.913 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.822144+0000", 2024-09-11T07:50:58.913 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.7265000000000001e-05 2024-09-11T07:50:58.913 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.913 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:58.913 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.913 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.913 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.914 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:19886 10.f e353/341 10:ff12a744:::benchmark_data_smithi022_163866_object19885:head v 353'1245, mlcod=353'1242)", 2024-09-11T07:50:58.914 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:42:22.743375+0000", 2024-09-11T07:50:58.914 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 516.11212807499999, 2024-09-11T07:50:58.914 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.077004778999999995, 2024-09-11T07:50:58.914 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:58.914 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:58.914 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:58.914 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.914 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:58.914 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.743375+0000", 2024-09-11T07:50:58.914 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.914 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.914 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.914 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:58.915 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.743375+0000", 2024-09-11T07:50:58.915 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.915 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.915 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.915 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:58.915 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.743373+0000", 2024-09-11T07:50:58.915 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:58.915 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.915 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.915 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:58.915 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.743446+0000", 2024-09-11T07:50:58.915 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.2288999999999999e-05 2024-09-11T07:50:58.915 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.916 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.916 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:58.916 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.818966+0000", 2024-09-11T07:50:58.916 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.075519992999999994 2024-09-11T07:50:58.916 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.916 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.916 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:58.916 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.818995+0000", 2024-09-11T07:50:58.916 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.8833999999999999e-05 2024-09-11T07:50:58.916 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.916 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.916 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:58.916 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.819082+0000", 2024-09-11T07:50:58.917 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7040999999999996e-05 2024-09-11T07:50:58.917 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.917 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.917 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:58.917 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.819105+0000", 2024-09-11T07:50:58.917 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3788999999999999e-05 2024-09-11T07:50:58.917 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.917 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.917 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:58.917 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.820352+0000", 2024-09-11T07:50:58.917 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0012469029999999999 2024-09-11T07:50:58.917 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.917 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.918 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:58.918 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:42:22.820380+0000", 2024-09-11T07:50:58.918 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.7477000000000001e-05 2024-09-11T07:50:58.918 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.918 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:58.918 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.918 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.918 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.918 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:118520 10.b e368/341 10:db7f28c1:::benchmark_data_smithi022_163866_object118519:head v 368'7480, mlcod=368'7477)", 2024-09-11T07:50:58.918 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:43:36.292731+0000", 2024-09-11T07:50:58.918 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 442.56277167600001, 2024-09-11T07:50:58.918 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.051984559, 2024-09-11T07:50:58.918 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:58.918 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:58.919 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:58.919 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.919 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:58.919 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:36.292731+0000", 2024-09-11T07:50:58.919 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.919 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.919 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.919 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:58.919 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:36.292731+0000", 2024-09-11T07:50:58.919 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.919 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.919 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.919 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:58.920 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:36.292730+0000", 2024-09-11T07:50:58.920 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:58.920 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.920 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.920 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:58.920 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:36.331905+0000", 2024-09-11T07:50:58.920 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.039174845 2024-09-11T07:50:58.920 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.920 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.920 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:58.920 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:36.331924+0000", 2024-09-11T07:50:58.920 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9419000000000002e-05 2024-09-11T07:50:58.921 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.921 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.921 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:58.921 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:36.331931+0000", 2024-09-11T07:50:58.921 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.37e-06 2024-09-11T07:50:58.921 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.921 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.921 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:58.921 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:36.332019+0000", 2024-09-11T07:50:58.921 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8065999999999993e-05 2024-09-11T07:50:58.921 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.921 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.921 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:58.922 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:36.332041+0000", 2024-09-11T07:50:58.922 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.2572e-05 2024-09-11T07:50:58.922 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.922 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.922 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:58.922 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:36.344582+0000", 2024-09-11T07:50:58.922 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.012540968 2024-09-11T07:50:58.922 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.922 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.922 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:58.922 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:43:36.344716+0000", 2024-09-11T07:50:58.922 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013363700000000001 2024-09-11T07:50:58.922 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.922 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:58.923 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.923 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.923 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.923 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25706.0:231066 10.7 e392/341 10:eec60872:::benchmark_data_smithi022_163866_object101640:head v 392'14457, mlcod=392'14452)", 2024-09-11T07:50:58.923 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:45:09.000035+0000", 2024-09-11T07:50:58.923 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 349.85546816599998, 2024-09-11T07:50:58.923 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.035319853999999998, 2024-09-11T07:50:58.923 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:58.923 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:58.923 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:58.923 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.923 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:58.923 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:09.000035+0000", 2024-09-11T07:50:58.924 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.924 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.924 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.924 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:58.924 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:09.000035+0000", 2024-09-11T07:50:58.924 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.924 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.924 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.924 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:58.924 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:09.000033+0000", 2024-09-11T07:50:58.924 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:50:58.924 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.924 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.924 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:58.925 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:09.000041+0000", 2024-09-11T07:50:58.925 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5050000000000001e-06 2024-09-11T07:50:58.925 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.925 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.925 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:58.925 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:09.000051+0000", 2024-09-11T07:50:58.925 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.009e-05 2024-09-11T07:50:58.925 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.925 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.925 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:58.925 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:09.000057+0000", 2024-09-11T07:50:58.925 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7989999999999999e-06 2024-09-11T07:50:58.925 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.925 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.925 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:58.926 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:09.009521+0000", 2024-09-11T07:50:58.926 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0094641740000000005 2024-09-11T07:50:58.926 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.926 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.926 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:58.926 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:09.009569+0000", 2024-09-11T07:50:58.926 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7806000000000003e-05 2024-09-11T07:50:58.926 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.926 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.926 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:58.926 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:09.035288+0000", 2024-09-11T07:50:58.926 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.025719006999999999 2024-09-11T07:50:58.926 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.926 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.926 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:58.927 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:09.035355+0000", 2024-09-11T07:50:58.927 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6880999999999999e-05 2024-09-11T07:50:58.927 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.927 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:58.927 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.927 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.927 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.927 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25990.0:29326 11.4 e422/412 11:21c2fcbb:::benchmark_data_smithi022_188626_object29325:head v 422'1841, mlcod=422'1839)", 2024-09-11T07:50:58.927 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:46:24.998337+0000", 2024-09-11T07:50:58.927 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 273.857165882, 2024-09-11T07:50:58.927 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.035371946000000001, 2024-09-11T07:50:58.927 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:58.927 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:58.927 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:58.927 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.928 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:58.928 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:24.998337+0000", 2024-09-11T07:50:58.928 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.928 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.928 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.928 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:58.928 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:24.998337+0000", 2024-09-11T07:50:58.928 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.928 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.928 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.928 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:58.928 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:24.998336+0000", 2024-09-11T07:50:58.928 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:50:58.928 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.928 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.929 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:58.929 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:24.998358+0000", 2024-09-11T07:50:58.929 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1960999999999999e-05 2024-09-11T07:50:58.929 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.929 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.929 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:58.929 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:24.998368+0000", 2024-09-11T07:50:58.929 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0172e-05 2024-09-11T07:50:58.929 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.929 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.929 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:58.929 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:24.998373+0000", 2024-09-11T07:50:58.929 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2939999999999996e-06 2024-09-11T07:50:58.929 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.930 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.930 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:58.930 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:24.998420+0000", 2024-09-11T07:50:58.930 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6854000000000002e-05 2024-09-11T07:50:58.930 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.930 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.930 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:58.930 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:24.998437+0000", 2024-09-11T07:50:58.930 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6875999999999999e-05 2024-09-11T07:50:58.930 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.930 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.930 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:58.930 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:25.033682+0000", 2024-09-11T07:50:58.930 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.035245130999999999 2024-09-11T07:50:58.930 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.931 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.931 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:58.931 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:25.033709+0000", 2024-09-11T07:50:58.931 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.6787999999999998e-05 2024-09-11T07:50:58.931 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.931 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:58.931 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.931 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.931 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.931 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_repop(client.25990.0:29325 11.a e422/412 11:5d942199:::benchmark_data_smithi022_188626_object29324:head v 422'1867, mlcod=422'1865)", 2024-09-11T07:50:58.931 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:46:24.998516+0000", 2024-09-11T07:50:58.931 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 273.85698738299999, 2024-09-11T07:50:58.931 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.035751990999999997, 2024-09-11T07:50:58.932 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:58.932 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:58.932 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:58.932 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.932 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:58.932 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:24.998516+0000", 2024-09-11T07:50:58.932 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.932 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.932 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.932 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:58.932 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:24.998516+0000", 2024-09-11T07:50:58.932 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.932 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.932 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.932 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:58.933 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:24.998515+0000", 2024-09-11T07:50:58.933 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.999999 2024-09-11T07:50:58.933 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.933 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.933 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:58.933 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:24.998567+0000", 2024-09-11T07:50:58.933 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.2896999999999999e-05 2024-09-11T07:50:58.933 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.933 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.933 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:58.933 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:25.032444+0000", 2024-09-11T07:50:58.933 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.033876787999999998 2024-09-11T07:50:58.933 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.933 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.933 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:58.934 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:25.032452+0000", 2024-09-11T07:50:58.934 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0069999999999997e-06 2024-09-11T07:50:58.934 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.934 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.934 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:58.934 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:25.032495+0000", 2024-09-11T07:50:58.934 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3231e-05 2024-09-11T07:50:58.934 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.934 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.934 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:58.935 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:25.032515+0000", 2024-09-11T07:50:58.935 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9245e-05 2024-09-11T07:50:58.935 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.935 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.935 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:58.935 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:25.034237+0000", 2024-09-11T07:50:58.935 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0017227049999999999 2024-09-11T07:50:58.935 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.935 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.935 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:58.935 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:46:25.034268+0000", 2024-09-11T07:50:58.935 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.027e-05 2024-09-11T07:50:58.936 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.936 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:58.936 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.936 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.936 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.936 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:222955 11.d 11:b200bdc8:::benchmark_data_smithi022_188626_object103147:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e462)", 2024-09-11T07:50:58.936 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:20.362716+0000", 2024-09-11T07:50:58.936 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 98.492787454999998, 2024-09-11T07:50:58.936 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.076297867000000005, 2024-09-11T07:50:58.936 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:58.936 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:58.936 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:50:58.936 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:50:58.936 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:58.936 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 222955 2024-09-11T07:50:58.937 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.937 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:58.937 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.937 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:58.937 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.362716+0000", 2024-09-11T07:50:58.937 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.937 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.937 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.937 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:58.937 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.362716+0000", 2024-09-11T07:50:58.937 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.937 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.937 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.937 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:58.937 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.362714+0000", 2024-09-11T07:50:58.938 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:50:58.938 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.938 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.938 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:58.938 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.362720+0000", 2024-09-11T07:50:58.938 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.404e-06 2024-09-11T07:50:58.938 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.938 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.938 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:58.938 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.362729+0000", 2024-09-11T07:50:58.938 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9400000000000008e-06 2024-09-11T07:50:58.938 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.938 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.938 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:58.939 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.362734+0000", 2024-09-11T07:50:58.939 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6800000000000001e-06 2024-09-11T07:50:58.939 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.939 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.939 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:58.939 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.362772+0000", 2024-09-11T07:50:58.939 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8148999999999997e-05 2024-09-11T07:50:58.939 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.939 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.939 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:58.939 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.362854+0000", 2024-09-11T07:50:58.939 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2398999999999995e-05 2024-09-11T07:50:58.939 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.939 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.940 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,5", 2024-09-11T07:50:58.940 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.362915+0000", 2024-09-11T07:50:58.940 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1171000000000001e-05 2024-09-11T07:50:58.940 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.940 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.940 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:58.940 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.364749+0000", 2024-09-11T07:50:58.940 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0018340800000000001 2024-09-11T07:50:58.940 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.940 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.940 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:50:58.940 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.364867+0000", 2024-09-11T07:50:58.940 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000117551 2024-09-11T07:50:58.940 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.941 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.941 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:58.941 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.438930+0000", 2024-09-11T07:50:58.941 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.074063367000000005 2024-09-11T07:50:58.941 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.941 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.941 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:58.941 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.438976+0000", 2024-09-11T07:50:58.941 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5481e-05 2024-09-11T07:50:58.941 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.941 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.941 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:58.941 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.439013+0000", 2024-09-11T07:50:58.942 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.7626999999999997e-05 2024-09-11T07:50:58.942 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.942 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:58.942 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.942 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.942 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.942 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:222963 11.d 11:b39fa229:::benchmark_data_smithi022_188626_object103155:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e462)", 2024-09-11T07:50:58.942 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:20.367291+0000", 2024-09-11T07:50:58.942 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 98.488212317000006, 2024-09-11T07:50:58.942 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.072005267999999997, 2024-09-11T07:50:58.942 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:58.942 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:58.942 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:50:58.942 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:50:58.943 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:58.943 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 222963 2024-09-11T07:50:58.943 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.943 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:58.943 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.943 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:58.943 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.367291+0000", 2024-09-11T07:50:58.943 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.943 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.943 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.943 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:58.943 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.367291+0000", 2024-09-11T07:50:58.943 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.943 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.944 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.944 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:58.944 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.367289+0000", 2024-09-11T07:50:58.944 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:50:58.944 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.944 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.944 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:58.944 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.367297+0000", 2024-09-11T07:50:58.944 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9559999999999997e-06 2024-09-11T07:50:58.944 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.944 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.944 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:58.944 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.367309+0000", 2024-09-11T07:50:58.945 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2843000000000001e-05 2024-09-11T07:50:58.945 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.945 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.945 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:58.945 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.367314+0000", 2024-09-11T07:50:58.945 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5460000000000002e-06 2024-09-11T07:50:58.945 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.945 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.945 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:58.945 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.367373+0000", 2024-09-11T07:50:58.945 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9086999999999999e-05 2024-09-11T07:50:58.945 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.945 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.945 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:58.946 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.367509+0000", 2024-09-11T07:50:58.946 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00013586300000000001 2024-09-11T07:50:58.946 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.946 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.946 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,5", 2024-09-11T07:50:58.946 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.367634+0000", 2024-09-11T07:50:58.946 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000125272 2024-09-11T07:50:58.946 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.946 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.946 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:58.946 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.369324+0000", 2024-09-11T07:50:58.946 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00168979 2024-09-11T07:50:58.946 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.947 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.947 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:50:58.947 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.369330+0000", 2024-09-11T07:50:58.947 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.66e-06 2024-09-11T07:50:58.947 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.947 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.947 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:58.947 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.439221+0000", 2024-09-11T07:50:58.947 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.069891324000000005 2024-09-11T07:50:58.947 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.947 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.947 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:58.947 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.439263+0000", 2024-09-11T07:50:58.948 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.2157999999999999e-05 2024-09-11T07:50:58.948 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.948 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.948 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:58.948 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:20.439296+0000", 2024-09-11T07:50:58.948 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.2959000000000001e-05 2024-09-11T07:50:58.948 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.948 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:58.948 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.948 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.948 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.948 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:238063 11.d 11:b2d703c4:::benchmark_data_smithi022_188626_object118255:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e471)", 2024-09-11T07:50:58.948 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:38.528129+0000", 2024-09-11T07:50:58.949 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 80.32737358, 2024-09-11T07:50:58.949 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.072514225000000002, 2024-09-11T07:50:58.949 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:50:58.949 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:50:58.949 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:50:58.949 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:50:58.949 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:50:58.949 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 238063 2024-09-11T07:50:58.949 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.949 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:50:58.949 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.949 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:50:58.949 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:38.528129+0000", 2024-09-11T07:50:58.950 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.950 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.950 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.950 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:50:58.950 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:38.528129+0000", 2024-09-11T07:50:58.950 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:50:58.950 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.950 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.950 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:50:58.950 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:38.528127+0000", 2024-09-11T07:50:58.950 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:50:58.950 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.950 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.951 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:50:58.951 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:38.528136+0000", 2024-09-11T07:50:58.951 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9649999999999997e-06 2024-09-11T07:50:58.951 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.951 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.951 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:50:58.951 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:38.528150+0000", 2024-09-11T07:50:58.951 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.3613e-05 2024-09-11T07:50:58.951 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.951 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.951 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:50:58.951 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:38.528156+0000", 2024-09-11T07:50:58.951 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5590000000000002e-06 2024-09-11T07:50:58.952 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.952 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.952 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:50:58.952 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:38.528197+0000", 2024-09-11T07:50:58.952 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0958999999999999e-05 2024-09-11T07:50:58.952 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.952 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.952 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:50:58.952 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:38.528372+0000", 2024-09-11T07:50:58.952 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017498899999999999 2024-09-11T07:50:58.952 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.952 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.953 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,5", 2024-09-11T07:50:58.953 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:38.528650+0000", 2024-09-11T07:50:58.953 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00027809500000000001 2024-09-11T07:50:58.953 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.953 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.953 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:50:58.953 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:38.530734+0000", 2024-09-11T07:50:58.953 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0020835509999999999 2024-09-11T07:50:58.953 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.953 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.953 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:58.953 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:38.530918+0000", 2024-09-11T07:50:58.953 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018410799999999999 2024-09-11T07:50:58.954 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.954 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.954 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:50:58.954 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:38.600583+0000", 2024-09-11T07:50:58.954 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.069664873000000002 2024-09-11T07:50:58.954 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.954 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.954 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:50:58.954 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:38.600618+0000", 2024-09-11T07:50:58.954 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5617000000000001e-05 2024-09-11T07:50:58.954 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:50:58.954 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:50:58.954 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:50:58.955 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:38.600644+0000", 2024-09-11T07:50:58.955 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.5412999999999999e-05 2024-09-11T07:50:58.955 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.955 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:58.955 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.955 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:50:58.955 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:50:58.955 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:50:59.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:58 smithi022 ceph-mon[29462]: 1.6 scrub starts 2024-09-11T07:50:59.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:58 smithi022 ceph-mon[29462]: 1.6 scrub ok 2024-09-11T07:50:59.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:58 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:50:59.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:58 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:50:59.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:58 smithi022 ceph-mon[29462]: 1.5 deep-scrub starts 2024-09-11T07:50:59.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:58 smithi022 ceph-mon[29462]: 1.5 deep-scrub ok 2024-09-11T07:50:59.158 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:58 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2418834372' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:50:59.158 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:58 smithi022 ceph-mon[37330]: 1.6 scrub starts 2024-09-11T07:50:59.158 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:58 smithi022 ceph-mon[37330]: 1.6 scrub ok 2024-09-11T07:50:59.158 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:58 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:50:59.158 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:58 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:50:59.158 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:58 smithi022 ceph-mon[37330]: 1.5 deep-scrub starts 2024-09-11T07:50:59.158 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:58 smithi022 ceph-mon[37330]: 1.5 deep-scrub ok 2024-09-11T07:50:59.158 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:58 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2418834372' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:50:59.266 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:58 smithi104 ceph-mon[33507]: 1.6 scrub starts 2024-09-11T07:50:59.266 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:58 smithi104 ceph-mon[33507]: 1.6 scrub ok 2024-09-11T07:50:59.267 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:58 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:50:59.267 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:58 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:50:59.267 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:58 smithi104 ceph-mon[33507]: 1.5 deep-scrub starts 2024-09-11T07:50:59.267 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:58 smithi104 ceph-mon[33507]: 1.5 deep-scrub ok 2024-09-11T07:50:59.267 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:58 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2418834372' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:50:59.461 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight 0 1 2024-09-11T07:50:59.718 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:50:59.868 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-11T07:51:00.117 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:51:00.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:59 smithi022 ceph-mon[29462]: 5.15 scrub starts 2024-09-11T07:51:00.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:59 smithi022 ceph-mon[29462]: 5.15 scrub ok 2024-09-11T07:51:00.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:59 smithi022 ceph-mon[29462]: osdmap e496: 8 total, 8 up, 8 in 2024-09-11T07:51:00.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:50:59 smithi022 ceph-mon[29462]: pgmap v1143: 142 pgs: 5 peering, 137 active+clean; 483 KiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 177 B/s rd, 0 op/s; 145 KiB/s, 27 objects/s recovering 2024-09-11T07:51:00.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:59 smithi022 ceph-mon[37330]: 5.15 scrub starts 2024-09-11T07:51:00.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:59 smithi022 ceph-mon[37330]: 5.15 scrub ok 2024-09-11T07:51:00.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:59 smithi022 ceph-mon[37330]: osdmap e496: 8 total, 8 up, 8 in 2024-09-11T07:51:00.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:50:59 smithi022 ceph-mon[37330]: pgmap v1143: 142 pgs: 5 peering, 137 active+clean; 483 KiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 177 B/s rd, 0 op/s; 145 KiB/s, 27 objects/s recovering 2024-09-11T07:51:00.238 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:59 smithi104 ceph-mon[33507]: 5.15 scrub starts 2024-09-11T07:51:00.238 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:59 smithi104 ceph-mon[33507]: 5.15 scrub ok 2024-09-11T07:51:00.238 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:59 smithi104 ceph-mon[33507]: osdmap e496: 8 total, 8 up, 8 in 2024-09-11T07:51:00.238 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:50:59 smithi104 ceph-mon[33507]: pgmap v1143: 142 pgs: 5 peering, 137 active+clean; 483 KiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 177 B/s rd, 0 op/s; 145 KiB/s, 27 objects/s recovering 2024-09-11T07:51:00.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:51:00 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:51:00.534 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:51:00 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:51:00] "GET /metrics HTTP/1.1" 200 222348 "" "Prometheus/2.43.0" 2024-09-11T07:51:01.123 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:00 smithi022 ceph-mon[29462]: 4.18 scrub starts 2024-09-11T07:51:01.124 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:00 smithi022 ceph-mon[29462]: 4.18 scrub ok 2024-09-11T07:51:01.124 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:00 smithi022 ceph-mon[29462]: 5.b deep-scrub starts 2024-09-11T07:51:01.124 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:00 smithi022 ceph-mon[29462]: 5.b deep-scrub ok 2024-09-11T07:51:01.124 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:00 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:51:01.124 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:00 smithi022 ceph-mon[37330]: 4.18 scrub starts 2024-09-11T07:51:01.124 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:00 smithi022 ceph-mon[37330]: 4.18 scrub ok 2024-09-11T07:51:01.124 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:00 smithi022 ceph-mon[37330]: 5.b deep-scrub starts 2024-09-11T07:51:01.124 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:00 smithi022 ceph-mon[37330]: 5.b deep-scrub ok 2024-09-11T07:51:01.124 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:00 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:51:01.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:00 smithi104 ceph-mon[33507]: 4.18 scrub starts 2024-09-11T07:51:01.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:00 smithi104 ceph-mon[33507]: 4.18 scrub ok 2024-09-11T07:51:01.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:00 smithi104 ceph-mon[33507]: 5.b deep-scrub starts 2024-09-11T07:51:01.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:00 smithi104 ceph-mon[33507]: 5.b deep-scrub ok 2024-09-11T07:51:01.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:00 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:51:01.920 INFO:teuthology.orchestra.run.smithi022.stderr:reweighted osd.0 to 1 (10000) 2024-09-11T07:51:02.137 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:51:02.137 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:51:02.137 INFO:teuthology.orchestra.run.smithi104.stdout: "num_ops": 0 2024-09-11T07:51:02.137 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:51:02.138 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:01 smithi104 ceph-mon[33507]: 4.d deep-scrub starts 2024-09-11T07:51:02.138 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:01 smithi104 ceph-mon[33507]: 4.d deep-scrub ok 2024-09-11T07:51:02.138 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:01 smithi104 ceph-mon[33507]: 6.19 scrub starts 2024-09-11T07:51:02.138 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:01 smithi104 ceph-mon[33507]: 6.19 scrub ok 2024-09-11T07:51:02.138 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:01 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3872993308' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T07:51:02.138 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:01 smithi104 ceph-mon[33507]: from='client.26234 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T07:51:02.138 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:01 smithi104 ceph-mon[33507]: pgmap v1144: 142 pgs: 5 peering, 137 active+clean; 483 KiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 86 KiB/s, 7 objects/s recovering 2024-09-11T07:51:02.138 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:01 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:51:02.218 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:01 smithi022 ceph-mon[29462]: 4.d deep-scrub starts 2024-09-11T07:51:02.218 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:01 smithi022 ceph-mon[29462]: 4.d deep-scrub ok 2024-09-11T07:51:02.218 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:01 smithi022 ceph-mon[29462]: 6.19 scrub starts 2024-09-11T07:51:02.218 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:01 smithi022 ceph-mon[29462]: 6.19 scrub ok 2024-09-11T07:51:02.218 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:01 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3872993308' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T07:51:02.218 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:01 smithi022 ceph-mon[29462]: from='client.26234 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T07:51:02.218 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:01 smithi022 ceph-mon[29462]: pgmap v1144: 142 pgs: 5 peering, 137 active+clean; 483 KiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 86 KiB/s, 7 objects/s recovering 2024-09-11T07:51:02.218 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:01 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:51:02.218 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:01 smithi022 ceph-mon[37330]: 4.d deep-scrub starts 2024-09-11T07:51:02.219 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:01 smithi022 ceph-mon[37330]: 4.d deep-scrub ok 2024-09-11T07:51:02.219 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:01 smithi022 ceph-mon[37330]: 6.19 scrub starts 2024-09-11T07:51:02.219 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:01 smithi022 ceph-mon[37330]: 6.19 scrub ok 2024-09-11T07:51:02.219 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:01 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3872993308' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T07:51:02.219 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:01 smithi022 ceph-mon[37330]: from='client.26234 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T07:51:02.219 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:01 smithi022 ceph-mon[37330]: pgmap v1144: 142 pgs: 5 peering, 137 active+clean; 483 KiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 86 KiB/s, 7 objects/s recovering 2024-09-11T07:51:02.219 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:01 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:51:02.768 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-11T07:51:02.808 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 0 1 2024-09-11T07:51:03.023 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:51:03.080 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:51:03.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:02 smithi022 ceph-mon[29462]: from='client.26234 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-11T07:51:03.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:02 smithi022 ceph-mon[29462]: osdmap e497: 8 total, 8 up, 8 in 2024-09-11T07:51:03.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:02 smithi022 ceph-mon[37330]: from='client.26234 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-11T07:51:03.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:02 smithi022 ceph-mon[37330]: osdmap e497: 8 total, 8 up, 8 in 2024-09-11T07:51:03.243 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:02 smithi104 ceph-mon[33507]: from='client.26234 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-11T07:51:03.243 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:02 smithi104 ceph-mon[33507]: osdmap e497: 8 total, 8 up, 8 in 2024-09-11T07:51:04.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:03 smithi022 ceph-mon[29462]: 6.16 scrub starts 2024-09-11T07:51:04.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:03 smithi022 ceph-mon[29462]: 6.16 scrub ok 2024-09-11T07:51:04.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:03 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:51:04.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:03 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:51:04.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:03 smithi022 ceph-mon[29462]: pgmap v1146: 142 pgs: 142 active+clean; 483 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 842 B/s rd, 0 op/s; 156 B/s, 3 objects/s recovering 2024-09-11T07:51:04.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:03 smithi022 ceph-mon[29462]: pgmap v1147: 142 pgs: 142 active+clean; 483 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 965 B/s rd, 0 op/s; 179 B/s, 4 objects/s recovering 2024-09-11T07:51:04.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:03 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:51:04.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:03 smithi022 ceph-mon[29462]: 6.5 scrub starts 2024-09-11T07:51:04.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:03 smithi022 ceph-mon[29462]: 6.5 scrub ok 2024-09-11T07:51:04.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:03 smithi022 ceph-mon[29462]: 3.2 deep-scrub starts 2024-09-11T07:51:04.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:03 smithi022 ceph-mon[29462]: 3.2 deep-scrub ok 2024-09-11T07:51:04.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:03 smithi022 ceph-mon[37330]: 6.16 scrub starts 2024-09-11T07:51:04.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:03 smithi022 ceph-mon[37330]: 6.16 scrub ok 2024-09-11T07:51:04.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:03 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:51:04.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:03 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:51:04.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:03 smithi022 ceph-mon[37330]: pgmap v1146: 142 pgs: 142 active+clean; 483 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 842 B/s rd, 0 op/s; 156 B/s, 3 objects/s recovering 2024-09-11T07:51:04.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:03 smithi022 ceph-mon[37330]: pgmap v1147: 142 pgs: 142 active+clean; 483 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 965 B/s rd, 0 op/s; 179 B/s, 4 objects/s recovering 2024-09-11T07:51:04.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:03 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:51:04.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:03 smithi022 ceph-mon[37330]: 6.5 scrub starts 2024-09-11T07:51:04.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:03 smithi022 ceph-mon[37330]: 6.5 scrub ok 2024-09-11T07:51:04.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:03 smithi022 ceph-mon[37330]: 3.2 deep-scrub starts 2024-09-11T07:51:04.168 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:03 smithi022 ceph-mon[37330]: 3.2 deep-scrub ok 2024-09-11T07:51:04.279 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:51:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:51:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:51:04.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:03 smithi104 ceph-mon[33507]: 6.16 scrub starts 2024-09-11T07:51:04.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:03 smithi104 ceph-mon[33507]: 6.16 scrub ok 2024-09-11T07:51:04.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:03 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:51:04.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:03 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:51:04.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:03 smithi104 ceph-mon[33507]: pgmap v1146: 142 pgs: 142 active+clean; 483 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 842 B/s rd, 0 op/s; 156 B/s, 3 objects/s recovering 2024-09-11T07:51:04.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:03 smithi104 ceph-mon[33507]: pgmap v1147: 142 pgs: 142 active+clean; 483 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 965 B/s rd, 0 op/s; 179 B/s, 4 objects/s recovering 2024-09-11T07:51:04.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:03 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:51:04.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:03 smithi104 ceph-mon[33507]: 6.5 scrub starts 2024-09-11T07:51:04.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:03 smithi104 ceph-mon[33507]: 6.5 scrub ok 2024-09-11T07:51:04.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:03 smithi104 ceph-mon[33507]: 3.2 deep-scrub starts 2024-09-11T07:51:04.280 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:03 smithi104 ceph-mon[33507]: 3.2 deep-scrub ok 2024-09-11T07:51:04.813 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:51:04.813 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [], 2024-09-11T07:51:04.813 INFO:teuthology.orchestra.run.smithi104.stdout: "complaint_time": 30, 2024-09-11T07:51:04.813 INFO:teuthology.orchestra.run.smithi104.stdout: "num_blocked_ops": 0 2024-09-11T07:51:04.813 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:51:04.931 INFO:teuthology.orchestra.run.smithi022.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-11T07:51:05.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:04 smithi104 ceph-mon[33507]: 5.6 scrub starts 2024-09-11T07:51:05.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:04 smithi104 ceph-mon[33507]: 5.6 scrub ok 2024-09-11T07:51:05.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:04 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1897966818' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T07:51:05.355 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:04 smithi022 ceph-mon[29462]: 5.6 scrub starts 2024-09-11T07:51:05.355 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:04 smithi022 ceph-mon[29462]: 5.6 scrub ok 2024-09-11T07:51:05.355 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:04 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1897966818' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T07:51:05.356 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:04 smithi022 ceph-mon[37330]: 5.6 scrub starts 2024-09-11T07:51:05.356 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:04 smithi022 ceph-mon[37330]: 5.6 scrub ok 2024-09-11T07:51:05.356 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:04 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1897966818' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-09-11T07:51:05.638 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-11T07:51:05.639 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight 1 1 2024-09-11T07:51:05.886 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:51:05.905 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:51:06.251 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:05 smithi104 ceph-mon[33507]: 5.d scrub starts 2024-09-11T07:51:06.251 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:05 smithi104 ceph-mon[33507]: 5.d scrub ok 2024-09-11T07:51:06.251 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:05 smithi104 ceph-mon[33507]: 2.1f scrub starts 2024-09-11T07:51:06.251 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:05 smithi104 ceph-mon[33507]: 2.1f scrub ok 2024-09-11T07:51:06.251 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:05 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1897966818' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-11T07:51:06.251 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:05 smithi104 ceph-mon[33507]: osdmap e498: 8 total, 8 up, 8 in 2024-09-11T07:51:06.251 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:05 smithi104 ceph-mon[33507]: pgmap v1149: 142 pgs: 142 active+clean; 483 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 965 B/s rd, 0 op/s; 133 B/s, 3 objects/s recovering 2024-09-11T07:51:06.251 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:05 smithi104 ceph-mon[33507]: 5.1c scrub starts 2024-09-11T07:51:06.251 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:05 smithi104 ceph-mon[33507]: 5.1c scrub ok 2024-09-11T07:51:06.251 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:05 smithi104 ceph-mon[33507]: 6.15 scrub starts 2024-09-11T07:51:06.251 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:05 smithi104 ceph-mon[33507]: 6.15 scrub ok 2024-09-11T07:51:06.252 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:05 smithi104 ceph-mon[33507]: 4.14 scrub starts 2024-09-11T07:51:06.252 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:05 smithi104 ceph-mon[33507]: 4.14 scrub ok 2024-09-11T07:51:06.288 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:05 smithi022 ceph-mon[29462]: 5.d scrub starts 2024-09-11T07:51:06.288 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:05 smithi022 ceph-mon[29462]: 5.d scrub ok 2024-09-11T07:51:06.288 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:05 smithi022 ceph-mon[29462]: 2.1f scrub starts 2024-09-11T07:51:06.288 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:05 smithi022 ceph-mon[29462]: 2.1f scrub ok 2024-09-11T07:51:06.289 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:05 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1897966818' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-11T07:51:06.289 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:05 smithi022 ceph-mon[29462]: osdmap e498: 8 total, 8 up, 8 in 2024-09-11T07:51:06.289 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:05 smithi022 ceph-mon[29462]: pgmap v1149: 142 pgs: 142 active+clean; 483 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 965 B/s rd, 0 op/s; 133 B/s, 3 objects/s recovering 2024-09-11T07:51:06.289 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:05 smithi022 ceph-mon[29462]: 5.1c scrub starts 2024-09-11T07:51:06.289 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:05 smithi022 ceph-mon[29462]: 5.1c scrub ok 2024-09-11T07:51:06.289 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:05 smithi022 ceph-mon[29462]: 6.15 scrub starts 2024-09-11T07:51:06.289 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:05 smithi022 ceph-mon[29462]: 6.15 scrub ok 2024-09-11T07:51:06.289 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:05 smithi022 ceph-mon[29462]: 4.14 scrub starts 2024-09-11T07:51:06.289 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:05 smithi022 ceph-mon[29462]: 4.14 scrub ok 2024-09-11T07:51:06.289 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:05 smithi022 ceph-mon[37330]: 5.d scrub starts 2024-09-11T07:51:06.289 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:05 smithi022 ceph-mon[37330]: 5.d scrub ok 2024-09-11T07:51:06.290 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:05 smithi022 ceph-mon[37330]: 2.1f scrub starts 2024-09-11T07:51:06.290 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:05 smithi022 ceph-mon[37330]: 2.1f scrub ok 2024-09-11T07:51:06.290 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:05 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1897966818' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-11T07:51:06.290 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:05 smithi022 ceph-mon[37330]: osdmap e498: 8 total, 8 up, 8 in 2024-09-11T07:51:06.290 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:05 smithi022 ceph-mon[37330]: pgmap v1149: 142 pgs: 142 active+clean; 483 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 965 B/s rd, 0 op/s; 133 B/s, 3 objects/s recovering 2024-09-11T07:51:06.290 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:05 smithi022 ceph-mon[37330]: 5.1c scrub starts 2024-09-11T07:51:06.290 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:05 smithi022 ceph-mon[37330]: 5.1c scrub ok 2024-09-11T07:51:06.290 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:05 smithi022 ceph-mon[37330]: 6.15 scrub starts 2024-09-11T07:51:06.290 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:05 smithi022 ceph-mon[37330]: 6.15 scrub ok 2024-09-11T07:51:06.290 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:05 smithi022 ceph-mon[37330]: 4.14 scrub starts 2024-09-11T07:51:06.290 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:05 smithi022 ceph-mon[37330]: 4.14 scrub ok 2024-09-11T07:51:07.594 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-11T07:51:07.594 INFO:teuthology.orchestra.run.smithi104.stdout: "size": 20, 2024-09-11T07:51:07.594 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 600, 2024-09-11T07:51:07.594 INFO:teuthology.orchestra.run.smithi104.stdout: "ops": [ 2024-09-11T07:51:07.594 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.594 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25706.0:254599 10.c 10:34692d4f:::benchmark_data_smithi022_163866_object125173:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e397)", 2024-09-11T07:51:07.594 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:45:30.138400+0000", 2024-09-11T07:51:07.594 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 337.45291367900001, 2024-09-11T07:51:07.594 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.65205146899999999, 2024-09-11T07:51:07.595 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:51:07.595 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:51:07.595 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:51:07.595 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25706", 2024-09-11T07:51:07.595 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:51:07.595 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 254599 2024-09-11T07:51:07.595 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.595 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:51:07.595 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.595 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:51:07.595 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.138400+0000", 2024-09-11T07:51:07.595 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.595 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.595 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.595 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:51:07.596 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.138400+0000", 2024-09-11T07:51:07.596 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.596 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.596 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.596 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:51:07.596 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.138399+0000", 2024-09-11T07:51:07.596 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:51:07.596 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.596 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.596 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:51:07.596 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.138405+0000", 2024-09-11T07:51:07.596 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8680000000000004e-06 2024-09-11T07:51:07.596 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.597 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.597 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:51:07.597 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.138413+0000", 2024-09-11T07:51:07.597 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6699999999999994e-06 2024-09-11T07:51:07.597 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.597 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.597 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:51:07.597 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.138416+0000", 2024-09-11T07:51:07.597 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5260000000000002e-06 2024-09-11T07:51:07.598 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.598 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.598 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.598 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.138453+0000", 2024-09-11T07:51:07.598 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6770000000000002e-05 2024-09-11T07:51:07.598 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.598 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.598 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:51:07.598 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.138458+0000", 2024-09-11T07:51:07.598 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.6199999999999998e-06 2024-09-11T07:51:07.599 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.599 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.599 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.599 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.770416+0000", 2024-09-11T07:51:07.599 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.63195844899999998 2024-09-11T07:51:07.599 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.599 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.599 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:51:07.599 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.770487+0000", 2024-09-11T07:51:07.599 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.0949000000000001e-05 2024-09-11T07:51:07.599 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.599 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.599 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:51:07.600 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.770553+0000", 2024-09-11T07:51:07.600 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6304999999999996e-05 2024-09-11T07:51:07.600 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.600 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.600 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:51:07.600 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.783705+0000", 2024-09-11T07:51:07.600 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.013151685 2024-09-11T07:51:07.600 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.600 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.600 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.600 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790322+0000", 2024-09-11T07:51:07.600 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0066173789999999996 2024-09-11T07:51:07.600 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.600 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.601 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.601 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790373+0000", 2024-09-11T07:51:07.601 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0693999999999999e-05 2024-09-11T07:51:07.601 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.601 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.601 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:51:07.601 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790417+0000", 2024-09-11T07:51:07.601 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3501000000000002e-05 2024-09-11T07:51:07.602 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.602 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.602 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:51:07.602 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790452+0000", 2024-09-11T07:51:07.602 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5377999999999999e-05 2024-09-11T07:51:07.602 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.602 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:51:07.602 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.602 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.603 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.603 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25706.0:254607 10.c 10:33e52c3b:::benchmark_data_smithi022_163866_object125181:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e397)", 2024-09-11T07:51:07.603 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:45:30.142279+0000", 2024-09-11T07:51:07.603 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 337.44903548299999, 2024-09-11T07:51:07.603 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.64834728900000005, 2024-09-11T07:51:07.603 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:51:07.603 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:51:07.603 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:51:07.604 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25706", 2024-09-11T07:51:07.604 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/2214009588", 2024-09-11T07:51:07.604 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 254607 2024-09-11T07:51:07.604 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.604 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:51:07.604 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.604 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:51:07.604 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.142279+0000", 2024-09-11T07:51:07.605 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.605 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.605 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.605 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:51:07.605 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.142279+0000", 2024-09-11T07:51:07.605 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.605 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.605 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.606 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:51:07.606 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.142277+0000", 2024-09-11T07:51:07.606 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:51:07.606 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.606 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.606 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:51:07.606 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.142283+0000", 2024-09-11T07:51:07.606 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1020000000000002e-06 2024-09-11T07:51:07.607 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.607 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.607 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:51:07.607 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.142295+0000", 2024-09-11T07:51:07.607 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.222e-05 2024-09-11T07:51:07.607 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.607 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.607 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:51:07.607 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.142301+0000", 2024-09-11T07:51:07.608 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.5509999999999999e-06 2024-09-11T07:51:07.608 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.608 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.608 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.608 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.142344+0000", 2024-09-11T07:51:07.608 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3316999999999999e-05 2024-09-11T07:51:07.608 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.608 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.609 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:51:07.609 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.142351+0000", 2024-09-11T07:51:07.609 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7750000000000004e-06 2024-09-11T07:51:07.609 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.609 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.609 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.609 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.773044+0000", 2024-09-11T07:51:07.609 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.63069285200000003 2024-09-11T07:51:07.609 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.610 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.610 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:51:07.610 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.773111+0000", 2024-09-11T07:51:07.610 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7602999999999996e-05 2024-09-11T07:51:07.610 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.610 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.610 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,6", 2024-09-11T07:51:07.610 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.773176+0000", 2024-09-11T07:51:07.611 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4419999999999996e-05 2024-09-11T07:51:07.611 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.611 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.611 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:51:07.611 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790332+0000", 2024-09-11T07:51:07.611 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.017156062 2024-09-11T07:51:07.611 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.611 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.612 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.612 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790512+0000", 2024-09-11T07:51:07.612 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00018059900000000001 2024-09-11T07:51:07.612 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.612 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.612 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.612 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790551+0000", 2024-09-11T07:51:07.612 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8346000000000002e-05 2024-09-11T07:51:07.612 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.613 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.613 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:51:07.613 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790591+0000", 2024-09-11T07:51:07.613 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0272999999999999e-05 2024-09-11T07:51:07.613 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.613 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.615 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:51:07.615 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:45:30.790626+0000", 2024-09-11T07:51:07.615 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4898999999999999e-05 2024-09-11T07:51:07.615 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.615 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:51:07.615 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.615 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.615 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.615 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:192480 11.4 11:2c053524:::benchmark_data_smithi022_188626_object72672:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e452)", 2024-09-11T07:51:07.616 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:48:46.105438+0000", 2024-09-11T07:51:07.616 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 141.48587587700001, 2024-09-11T07:51:07.616 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.2086161249999998, 2024-09-11T07:51:07.616 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:51:07.616 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:51:07.616 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:51:07.616 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:51:07.616 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:51:07.616 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 192480 2024-09-11T07:51:07.616 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.617 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:51:07.617 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.617 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:51:07.617 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.105438+0000", 2024-09-11T07:51:07.617 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.617 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.617 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.617 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:51:07.617 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.105438+0000", 2024-09-11T07:51:07.617 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.617 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.617 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.617 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:51:07.618 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.105437+0000", 2024-09-11T07:51:07.618 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:51:07.618 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.618 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.618 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:51:07.618 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.105443+0000", 2024-09-11T07:51:07.618 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9490000000000001e-06 2024-09-11T07:51:07.618 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.618 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.618 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:51:07.618 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.105453+0000", 2024-09-11T07:51:07.618 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8810000000000008e-06 2024-09-11T07:51:07.618 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.619 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.619 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:51:07.619 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.105457+0000", 2024-09-11T07:51:07.619 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.8359999999999999e-06 2024-09-11T07:51:07.619 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.619 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.619 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.619 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.105491+0000", 2024-09-11T07:51:07.619 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3840999999999997e-05 2024-09-11T07:51:07.619 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.619 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.619 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:51:07.619 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.105571+0000", 2024-09-11T07:51:07.619 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9730999999999999e-05 2024-09-11T07:51:07.620 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.620 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.620 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 2,6", 2024-09-11T07:51:07.620 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.105639+0000", 2024-09-11T07:51:07.620 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.8310000000000002e-05 2024-09-11T07:51:07.620 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.620 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.620 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:51:07.620 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.108088+0000", 2024-09-11T07:51:07.620 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0024489350000000002 2024-09-11T07:51:07.620 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.620 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.620 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.621 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.108465+0000", 2024-09-11T07:51:07.621 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00037678100000000002 2024-09-11T07:51:07.621 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.621 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.621 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.621 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.137118+0000", 2024-09-11T07:51:07.621 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.028653151000000002 2024-09-11T07:51:07.621 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.621 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.621 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:51:07.621 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.137129+0000", 2024-09-11T07:51:07.622 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0584999999999999e-05 2024-09-11T07:51:07.622 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.622 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.622 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.622 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.134593+0000", 2024-09-11T07:51:07.622 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.99746406899999995 2024-09-11T07:51:07.622 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.622 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.622 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:51:07.622 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.134601+0000", 2024-09-11T07:51:07.622 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.585e-06 2024-09-11T07:51:07.622 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.622 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.622 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.623 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.313976+0000", 2024-09-11T07:51:07.623 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1793741639999999 2024-09-11T07:51:07.623 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.623 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.623 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:51:07.623 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.314054+0000", 2024-09-11T07:51:07.623 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.8885000000000001e-05 2024-09-11T07:51:07.623 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.623 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:51:07.623 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.623 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.623 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.624 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:192487 11.f 11:fd58c779:::benchmark_data_smithi022_188626_object72679:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e452)", 2024-09-11T07:51:07.624 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:48:46.113756+0000", 2024-09-11T07:51:07.624 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 141.47755768299999, 2024-09-11T07:51:07.624 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.2508924820000002, 2024-09-11T07:51:07.624 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:51:07.624 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:51:07.624 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:51:07.624 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:51:07.624 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:51:07.624 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 192487 2024-09-11T07:51:07.624 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.624 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:51:07.624 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.625 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:51:07.625 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.113756+0000", 2024-09-11T07:51:07.625 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.625 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.625 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.625 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:51:07.625 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.113756+0000", 2024-09-11T07:51:07.625 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.625 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.625 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.625 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:51:07.625 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.113755+0000", 2024-09-11T07:51:07.625 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:51:07.626 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.626 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.626 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:51:07.626 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.113761+0000", 2024-09-11T07:51:07.626 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0449999999999997e-06 2024-09-11T07:51:07.626 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.626 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.626 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:51:07.626 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.113770+0000", 2024-09-11T07:51:07.626 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5810000000000003e-06 2024-09-11T07:51:07.626 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.626 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.626 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:51:07.626 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.113777+0000", 2024-09-11T07:51:07.627 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4620000000000001e-06 2024-09-11T07:51:07.627 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.627 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.627 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.627 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.113811+0000", 2024-09-11T07:51:07.627 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.4360000000000003e-05 2024-09-11T07:51:07.627 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.627 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.627 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:51:07.627 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.113918+0000", 2024-09-11T07:51:07.627 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010678799999999999 2024-09-11T07:51:07.627 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.627 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.627 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:51:07.628 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.114005+0000", 2024-09-11T07:51:07.628 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6760000000000003e-05 2024-09-11T07:51:07.628 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.628 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.628 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:51:07.628 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.116601+0000", 2024-09-11T07:51:07.628 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0025965659999999998 2024-09-11T07:51:07.628 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.628 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.628 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.628 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.126220+0000", 2024-09-11T07:51:07.628 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0096189129999999998 2024-09-11T07:51:07.628 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.629 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.629 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.629 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.137933+0000", 2024-09-11T07:51:07.629 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.011713146000000001 2024-09-11T07:51:07.629 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.629 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.629 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:51:07.629 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.137943+0000", 2024-09-11T07:51:07.629 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0097e-05 2024-09-11T07:51:07.630 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.630 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.630 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.630 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.133216+0000", 2024-09-11T07:51:07.630 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.995272508 2024-09-11T07:51:07.630 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.630 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.630 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:51:07.630 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.133226+0000", 2024-09-11T07:51:07.630 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0594999999999999e-05 2024-09-11T07:51:07.630 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.630 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.630 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.631 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.364575+0000", 2024-09-11T07:51:07.631 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2313488690000001 2024-09-11T07:51:07.631 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.631 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.631 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:51:07.631 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.364649+0000", 2024-09-11T07:51:07.631 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3706000000000002e-05 2024-09-11T07:51:07.631 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.631 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:51:07.631 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.631 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.631 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.631 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:192492 11.f 11:f5250b21:::benchmark_data_smithi022_188626_object72684:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e452)", 2024-09-11T07:51:07.631 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:48:46.117935+0000", 2024-09-11T07:51:07.632 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 141.47337891999999, 2024-09-11T07:51:07.632 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.246796008, 2024-09-11T07:51:07.632 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:51:07.632 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:51:07.632 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:51:07.632 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:51:07.632 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:51:07.632 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 192492 2024-09-11T07:51:07.632 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.632 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:51:07.632 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.633 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:51:07.633 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.117935+0000", 2024-09-11T07:51:07.633 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.633 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.633 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.633 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:51:07.633 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.117935+0000", 2024-09-11T07:51:07.633 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.634 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.634 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.634 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:51:07.634 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.117934+0000", 2024-09-11T07:51:07.634 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:51:07.634 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.634 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.634 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:51:07.634 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.117941+0000", 2024-09-11T07:51:07.634 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1600000000000001e-06 2024-09-11T07:51:07.635 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.635 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.635 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:51:07.635 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.117952+0000", 2024-09-11T07:51:07.635 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1224e-05 2024-09-11T07:51:07.635 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.635 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.635 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:51:07.635 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.117957+0000", 2024-09-11T07:51:07.635 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9989999999999999e-06 2024-09-11T07:51:07.635 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.635 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.636 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.636 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.117991+0000", 2024-09-11T07:51:07.636 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3668999999999999e-05 2024-09-11T07:51:07.636 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.636 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.636 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:51:07.636 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.118070+0000", 2024-09-11T07:51:07.636 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9060999999999994e-05 2024-09-11T07:51:07.636 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.636 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.636 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2", 2024-09-11T07:51:07.636 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.118144+0000", 2024-09-11T07:51:07.637 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4585999999999994e-05 2024-09-11T07:51:07.637 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.637 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.637 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:51:07.637 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.120427+0000", 2024-09-11T07:51:07.637 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0022825469999999998 2024-09-11T07:51:07.637 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.637 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.637 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.637 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.127292+0000", 2024-09-11T07:51:07.637 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0068656669999999998 2024-09-11T07:51:07.637 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.637 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.637 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.638 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.137970+0000", 2024-09-11T07:51:07.638 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.010677944 2024-09-11T07:51:07.638 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.638 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.638 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:51:07.638 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.137978+0000", 2024-09-11T07:51:07.638 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7870000000000002e-06 2024-09-11T07:51:07.638 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.638 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.638 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.638 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.133302+0000", 2024-09-11T07:51:07.639 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.99532337599999998 2024-09-11T07:51:07.639 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.639 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.639 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:51:07.639 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.133318+0000", 2024-09-11T07:51:07.639 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6620999999999999e-05 2024-09-11T07:51:07.639 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.639 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.639 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.639 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.364674+0000", 2024-09-11T07:51:07.640 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2313559709999999 2024-09-11T07:51:07.640 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.640 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.640 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:51:07.640 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.364731+0000", 2024-09-11T07:51:07.640 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7077999999999998e-05 2024-09-11T07:51:07.640 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.640 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:51:07.640 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.640 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.640 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.640 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24353.0:3077 5.d 5:b4812045:::notify.4:head [watch reconnect cookie 94395805755184 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e453)", 2024-09-11T07:51:07.641 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:48:46.124437+0000", 2024-09-11T07:51:07.641 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 141.466877297, 2024-09-11T07:51:07.641 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0675898260000001, 2024-09-11T07:51:07.641 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:51:07.641 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2024-09-11T07:51:07.641 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:51:07.641 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24353", 2024-09-11T07:51:07.641 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1860969011", 2024-09-11T07:51:07.641 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 3077 2024-09-11T07:51:07.641 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.641 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:51:07.641 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.641 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:51:07.641 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124437+0000", 2024-09-11T07:51:07.642 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.642 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.642 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.642 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:51:07.642 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124437+0000", 2024-09-11T07:51:07.642 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.642 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.642 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.642 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:51:07.642 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124436+0000", 2024-09-11T07:51:07.642 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999986 2024-09-11T07:51:07.642 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.642 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.643 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:51:07.643 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124441+0000", 2024-09-11T07:51:07.643 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.3419999999999998e-06 2024-09-11T07:51:07.643 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.643 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.643 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:51:07.643 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124450+0000", 2024-09-11T07:51:07.643 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.6680000000000007e-06 2024-09-11T07:51:07.643 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.643 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.643 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:51:07.643 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124453+0000", 2024-09-11T07:51:07.643 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.8410000000000002e-06 2024-09-11T07:51:07.643 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.644 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.644 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.644 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124497+0000", 2024-09-11T07:51:07.644 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.3220999999999998e-05 2024-09-11T07:51:07.644 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.644 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.644 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op must wait for map", 2024-09-11T07:51:07.644 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124503+0000", 2024-09-11T07:51:07.644 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8340000000000004e-06 2024-09-11T07:51:07.644 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.644 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.645 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.645 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.133144+0000", 2024-09-11T07:51:07.645 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0086415260000000001 2024-09-11T07:51:07.645 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.645 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.645 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:51:07.645 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.133153+0000", 2024-09-11T07:51:07.645 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.9309999999999997e-06 2024-09-11T07:51:07.645 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.645 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.645 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.645 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.171356+0000", 2024-09-11T07:51:07.645 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0382031789999999 2024-09-11T07:51:07.646 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.646 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.646 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for degraded object", 2024-09-11T07:51:07.646 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.171652+0000", 2024-09-11T07:51:07.646 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00029536999999999999 2024-09-11T07:51:07.646 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.646 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.646 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.646 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.179886+0000", 2024-09-11T07:51:07.646 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0082341789999999995 2024-09-11T07:51:07.646 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.646 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.647 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:51:07.647 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.179964+0000", 2024-09-11T07:51:07.647 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.7897000000000003e-05 2024-09-11T07:51:07.647 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.647 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.647 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:51:07.647 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.192027+0000", 2024-09-11T07:51:07.647 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.012063084999999999 2024-09-11T07:51:07.647 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.647 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:51:07.647 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.647 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.648 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.648 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24353.0:3079 5.14 5:2b04a3e9:::notify.6:head [watch reconnect cookie 94395805760480 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e453)", 2024-09-11T07:51:07.648 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:48:46.124463+0000", 2024-09-11T07:51:07.648 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 141.46685130399999, 2024-09-11T07:51:07.648 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.070989915, 2024-09-11T07:51:07.648 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:51:07.648 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2024-09-11T07:51:07.648 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:51:07.648 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24353", 2024-09-11T07:51:07.648 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1860969011", 2024-09-11T07:51:07.648 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 3079 2024-09-11T07:51:07.648 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.648 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:51:07.649 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.649 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:51:07.649 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124463+0000", 2024-09-11T07:51:07.649 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.649 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.649 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.649 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:51:07.649 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124463+0000", 2024-09-11T07:51:07.649 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.649 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.649 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.649 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:51:07.649 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124462+0000", 2024-09-11T07:51:07.649 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:51:07.650 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.650 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.650 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:51:07.650 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124465+0000", 2024-09-11T07:51:07.650 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.8619999999999999e-06 2024-09-11T07:51:07.650 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.650 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.650 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:51:07.650 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124471+0000", 2024-09-11T07:51:07.650 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.0499999999999997e-06 2024-09-11T07:51:07.650 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.651 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.651 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:51:07.651 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124476+0000", 2024-09-11T07:51:07.651 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.69e-06 2024-09-11T07:51:07.651 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.651 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.651 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.651 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124537+0000", 2024-09-11T07:51:07.651 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1155999999999995e-05 2024-09-11T07:51:07.651 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.652 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.652 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op must wait for map", 2024-09-11T07:51:07.652 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124544+0000", 2024-09-11T07:51:07.652 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.601e-06 2024-09-11T07:51:07.652 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.652 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.652 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.652 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.139252+0000", 2024-09-11T07:51:07.652 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.014708407 2024-09-11T07:51:07.652 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.652 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.652 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:51:07.652 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.139260+0000", 2024-09-11T07:51:07.653 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.8490000000000006e-06 2024-09-11T07:51:07.653 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.653 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.653 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.653 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.180500+0000", 2024-09-11T07:51:07.653 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0412405689999999 2024-09-11T07:51:07.653 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.653 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.653 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for degraded object", 2024-09-11T07:51:07.653 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.180809+0000", 2024-09-11T07:51:07.653 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00030839099999999998 2024-09-11T07:51:07.653 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.653 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.654 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.654 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.192259+0000", 2024-09-11T07:51:07.654 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.011450426 2024-09-11T07:51:07.654 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.654 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.654 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:51:07.654 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.192335+0000", 2024-09-11T07:51:07.654 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5426999999999995e-05 2024-09-11T07:51:07.654 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.654 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.654 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:51:07.654 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.195453+0000", 2024-09-11T07:51:07.654 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003118039 2024-09-11T07:51:07.654 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.655 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:51:07.655 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.655 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.655 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.655 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.24353.0:3080 5.9 5:93e5b521:::notify.7:head [watch reconnect cookie 94395805742128 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e453)", 2024-09-11T07:51:07.655 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:48:46.124482+0000", 2024-09-11T07:51:07.655 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 141.46683164500001, 2024-09-11T07:51:07.655 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0809971739999999, 2024-09-11T07:51:07.655 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:51:07.655 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "started", 2024-09-11T07:51:07.655 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:51:07.655 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.24353", 2024-09-11T07:51:07.655 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1860969011", 2024-09-11T07:51:07.655 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 3080 2024-09-11T07:51:07.656 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.656 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:51:07.656 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.656 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:51:07.656 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124482+0000", 2024-09-11T07:51:07.656 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.656 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.656 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.656 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:51:07.656 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124482+0000", 2024-09-11T07:51:07.656 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.656 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.656 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.657 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:51:07.657 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124482+0000", 2024-09-11T07:51:07.657 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:51:07.657 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.657 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.657 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:51:07.657 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124485+0000", 2024-09-11T07:51:07.657 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.6230000000000001e-06 2024-09-11T07:51:07.657 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.657 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.657 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:51:07.657 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124491+0000", 2024-09-11T07:51:07.657 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.7120000000000003e-06 2024-09-11T07:51:07.657 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.658 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.658 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:51:07.658 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124495+0000", 2024-09-11T07:51:07.658 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5389999999999999e-06 2024-09-11T07:51:07.658 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.658 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.658 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.658 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124539+0000", 2024-09-11T07:51:07.658 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4539999999999997e-05 2024-09-11T07:51:07.658 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.658 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.658 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op must wait for map", 2024-09-11T07:51:07.658 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.124545+0000", 2024-09-11T07:51:07.659 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6749999999999999e-06 2024-09-11T07:51:07.659 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.659 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.659 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.659 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.130844+0000", 2024-09-11T07:51:07.659 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0062988330000000002 2024-09-11T07:51:07.659 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.659 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.659 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:51:07.659 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.130853+0000", 2024-09-11T07:51:07.659 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8400000000000001e-06 2024-09-11T07:51:07.659 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.659 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.659 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.659 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.166368+0000", 2024-09-11T07:51:07.660 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0355154 2024-09-11T07:51:07.660 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.660 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.660 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for degraded object", 2024-09-11T07:51:07.660 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.166658+0000", 2024-09-11T07:51:07.660 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00028989 2024-09-11T07:51:07.660 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.660 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.660 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.660 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.192286+0000", 2024-09-11T07:51:07.660 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.025627937 2024-09-11T07:51:07.660 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.660 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.661 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:51:07.661 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.192359+0000", 2024-09-11T07:51:07.661 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.3052999999999993e-05 2024-09-11T07:51:07.661 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.661 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.661 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:51:07.661 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.205480+0000", 2024-09-11T07:51:07.661 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.013120573999999999 2024-09-11T07:51:07.661 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.661 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:51:07.661 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.661 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.662 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.662 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:192486 11.c 11:3cae2f82:::benchmark_data_smithi022_188626_object72678:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e453)", 2024-09-11T07:51:07.662 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:48:46.138064+0000", 2024-09-11T07:51:07.662 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 141.453250427, 2024-09-11T07:51:07.662 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.089428566, 2024-09-11T07:51:07.662 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:51:07.662 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:51:07.662 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:51:07.662 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:51:07.662 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:51:07.662 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 192486 2024-09-11T07:51:07.662 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.662 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:51:07.662 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.663 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:51:07.663 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.138064+0000", 2024-09-11T07:51:07.663 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.663 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.663 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.663 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:51:07.663 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.138064+0000", 2024-09-11T07:51:07.663 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.663 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.663 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.663 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:51:07.663 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.138061+0000", 2024-09-11T07:51:07.663 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:51:07.664 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.664 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.664 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:51:07.664 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.138069+0000", 2024-09-11T07:51:07.664 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5039999999999996e-06 2024-09-11T07:51:07.664 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.664 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.664 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:51:07.664 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.138079+0000", 2024-09-11T07:51:07.664 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.8190000000000003e-06 2024-09-11T07:51:07.664 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.665 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.665 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:51:07.665 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.138086+0000", 2024-09-11T07:51:07.665 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.8399999999999997e-06 2024-09-11T07:51:07.665 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.665 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.665 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.665 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.139630+0000", 2024-09-11T07:51:07.665 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0015441719999999999 2024-09-11T07:51:07.665 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.665 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.665 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:51:07.665 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.139637+0000", 2024-09-11T07:51:07.666 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.6729999999999992e-06 2024-09-11T07:51:07.666 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.666 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.666 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.666 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.134574+0000", 2024-09-11T07:51:07.666 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.99493694799999999 2024-09-11T07:51:07.666 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.666 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.666 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:51:07.666 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.134583+0000", 2024-09-11T07:51:07.667 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.5439999999999999e-06 2024-09-11T07:51:07.667 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.667 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.667 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.667 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.226403+0000", 2024-09-11T07:51:07.667 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0918195580000001 2024-09-11T07:51:07.667 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.667 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.667 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:51:07.667 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.227492+0000", 2024-09-11T07:51:07.667 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0010897319999999999 2024-09-11T07:51:07.667 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.667 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:51:07.668 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.668 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.668 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.668 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:192489 11.c 11:316284ac:::benchmark_data_smithi022_188626_object72681:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e453)", 2024-09-11T07:51:07.668 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:48:46.138094+0000", 2024-09-11T07:51:07.668 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 141.45322032499999, 2024-09-11T07:51:07.668 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0895146549999999, 2024-09-11T07:51:07.668 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:51:07.668 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:51:07.668 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:51:07.668 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:51:07.668 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:51:07.669 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 192489 2024-09-11T07:51:07.669 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.669 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:51:07.669 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.669 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:51:07.669 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.138094+0000", 2024-09-11T07:51:07.669 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.669 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.669 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.669 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:51:07.669 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.138094+0000", 2024-09-11T07:51:07.669 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.669 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.670 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.670 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:51:07.670 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.138093+0000", 2024-09-11T07:51:07.670 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:51:07.670 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.670 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.670 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:51:07.670 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.138096+0000", 2024-09-11T07:51:07.670 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.303e-06 2024-09-11T07:51:07.670 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.670 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.670 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:51:07.670 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.138102+0000", 2024-09-11T07:51:07.670 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.4070000000000001e-06 2024-09-11T07:51:07.671 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.671 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.671 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:51:07.671 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.138105+0000", 2024-09-11T07:51:07.671 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5939999999999998e-06 2024-09-11T07:51:07.671 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.671 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.671 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.671 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.139658+0000", 2024-09-11T07:51:07.671 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00155288 2024-09-11T07:51:07.671 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.671 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.671 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:51:07.671 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.139665+0000", 2024-09-11T07:51:07.672 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.5520000000000001e-06 2024-09-11T07:51:07.672 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.672 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.672 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.672 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.134602+0000", 2024-09-11T07:51:07.672 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.99493757800000004 2024-09-11T07:51:07.672 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.672 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.672 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:51:07.672 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.134609+0000", 2024-09-11T07:51:07.672 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.6800000000000004e-06 2024-09-11T07:51:07.672 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.672 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.673 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.673 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.227534+0000", 2024-09-11T07:51:07.673 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.09292451 2024-09-11T07:51:07.673 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.673 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.673 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:51:07.673 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.227608+0000", 2024-09-11T07:51:07.673 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4814000000000006e-05 2024-09-11T07:51:07.673 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.673 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:51:07.673 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.673 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.673 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.673 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:192498 11.c 11:35807c2b:::benchmark_data_smithi022_188626_object72690:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e453)", 2024-09-11T07:51:07.674 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:48:46.169803+0000", 2024-09-11T07:51:07.674 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 141.421511579, 2024-09-11T07:51:07.674 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.1126209729999998, 2024-09-11T07:51:07.674 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:51:07.674 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:51:07.674 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:51:07.674 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:51:07.674 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:51:07.675 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 192498 2024-09-11T07:51:07.676 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.676 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:51:07.676 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.676 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:51:07.676 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.169803+0000", 2024-09-11T07:51:07.676 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.676 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.676 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.676 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:51:07.676 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.169803+0000", 2024-09-11T07:51:07.676 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.676 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.676 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.676 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:51:07.677 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.169801+0000", 2024-09-11T07:51:07.678 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999981 2024-09-11T07:51:07.678 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.678 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.678 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:51:07.678 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.169807+0000", 2024-09-11T07:51:07.678 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.494e-06 2024-09-11T07:51:07.678 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.678 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.678 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:51:07.679 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.169818+0000", 2024-09-11T07:51:07.679 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.049e-05 2024-09-11T07:51:07.679 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.679 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.679 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:51:07.679 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.169827+0000", 2024-09-11T07:51:07.679 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.5300000000000002e-06 2024-09-11T07:51:07.679 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.679 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.679 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.679 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.169867+0000", 2024-09-11T07:51:07.679 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.9987000000000002e-05 2024-09-11T07:51:07.679 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.679 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.680 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:51:07.680 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:46.169874+0000", 2024-09-11T07:51:07.680 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.55e-06 2024-09-11T07:51:07.680 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.680 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.680 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.680 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.134628+0000", 2024-09-11T07:51:07.680 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.96475460099999999 2024-09-11T07:51:07.680 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.680 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.680 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:51:07.680 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:47.134635+0000", 2024-09-11T07:51:07.680 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9009999999999997e-06 2024-09-11T07:51:07.681 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.681 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.681 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.681 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.227645+0000", 2024-09-11T07:51:07.681 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0930101059999999 2024-09-11T07:51:07.681 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.681 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.681 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:51:07.681 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.227867+0000", 2024-09-11T07:51:07.681 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00022187300000000001 2024-09-11T07:51:07.681 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.681 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.681 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2,5", 2024-09-11T07:51:07.681 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.227948+0000", 2024-09-11T07:51:07.681 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1142000000000003e-05 2024-09-11T07:51:07.682 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.682 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.682 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:51:07.682 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.232220+0000", 2024-09-11T07:51:07.682 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0042720479999999996 2024-09-11T07:51:07.682 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.682 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.682 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.682 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.232256+0000", 2024-09-11T07:51:07.682 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5828000000000003e-05 2024-09-11T07:51:07.682 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.682 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.682 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.683 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.237868+0000", 2024-09-11T07:51:07.683 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0056120880000000003 2024-09-11T07:51:07.683 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.683 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.683 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.683 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.282241+0000", 2024-09-11T07:51:07.683 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.044372822999999999 2024-09-11T07:51:07.683 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.683 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.683 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:51:07.683 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.282298+0000", 2024-09-11T07:51:07.683 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.7343999999999998e-05 2024-09-11T07:51:07.683 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.683 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.684 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:51:07.684 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:48:48.282424+0000", 2024-09-11T07:51:07.684 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012505799999999999 2024-09-11T07:51:07.684 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.684 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:51:07.684 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.684 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.684 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.684 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:237421 11.f 11:f5176df0:::benchmark_data_smithi022_188626_object117613:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e469)", 2024-09-11T07:51:07.684 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:36.818636+0000", 2024-09-11T07:51:07.684 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 90.772678390999999, 2024-09-11T07:51:07.684 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.056616448, 2024-09-11T07:51:07.684 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:51:07.685 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:51:07.685 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:51:07.685 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:51:07.685 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:51:07.685 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 237421 2024-09-11T07:51:07.685 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.685 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:51:07.685 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.685 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:51:07.685 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818636+0000", 2024-09-11T07:51:07.685 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.685 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.685 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.686 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:51:07.686 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818636+0000", 2024-09-11T07:51:07.686 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.686 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.686 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.686 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:51:07.686 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818634+0000", 2024-09-11T07:51:07.686 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:51:07.686 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.686 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.686 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:51:07.686 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818641+0000", 2024-09-11T07:51:07.686 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.097e-06 2024-09-11T07:51:07.686 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.687 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.687 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:51:07.687 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818651+0000", 2024-09-11T07:51:07.687 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0319000000000001e-05 2024-09-11T07:51:07.687 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.687 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.687 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:51:07.687 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818656+0000", 2024-09-11T07:51:07.687 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.724e-06 2024-09-11T07:51:07.687 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.687 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.687 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.687 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818692+0000", 2024-09-11T07:51:07.687 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6757000000000001e-05 2024-09-11T07:51:07.688 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.688 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.688 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:51:07.688 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818854+0000", 2024-09-11T07:51:07.688 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000161228 2024-09-11T07:51:07.688 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.688 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.688 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,1,2,3", 2024-09-11T07:51:07.688 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.819040+0000", 2024-09-11T07:51:07.688 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000185813 2024-09-11T07:51:07.688 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.688 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.688 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:51:07.688 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.822270+0000", 2024-09-11T07:51:07.689 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0032303480000000001 2024-09-11T07:51:07.689 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.689 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.689 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.689 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.825552+0000", 2024-09-11T07:51:07.689 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0032818320000000002 2024-09-11T07:51:07.689 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.689 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.689 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.689 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.826494+0000", 2024-09-11T07:51:07.689 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000942357 2024-09-11T07:51:07.690 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.690 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.690 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.690 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.837443+0000", 2024-09-11T07:51:07.690 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.010948652 2024-09-11T07:51:07.690 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.691 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.691 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.691 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.842188+0000", 2024-09-11T07:51:07.691 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0047453010000000004 2024-09-11T07:51:07.691 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.691 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.692 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:51:07.692 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.842202+0000", 2024-09-11T07:51:07.692 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.402e-05 2024-09-11T07:51:07.692 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.692 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.692 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.692 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.875208+0000", 2024-09-11T07:51:07.693 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0330058799999999 2024-09-11T07:51:07.693 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.693 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.693 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:51:07.693 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.875252+0000", 2024-09-11T07:51:07.693 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.4295000000000002e-05 2024-09-11T07:51:07.694 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.694 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:51:07.694 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.694 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.694 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.694 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:237422 11.f 11:fec018d7:::benchmark_data_smithi022_188626_object117614:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e469)", 2024-09-11T07:51:07.694 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:36.818663+0000", 2024-09-11T07:51:07.694 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 90.772650738999999, 2024-09-11T07:51:07.695 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0566943790000001, 2024-09-11T07:51:07.695 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:51:07.695 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:51:07.695 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:51:07.695 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:51:07.695 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:51:07.696 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 237422 2024-09-11T07:51:07.696 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.696 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:51:07.696 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.696 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:51:07.696 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818663+0000", 2024-09-11T07:51:07.697 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.697 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.697 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.697 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:51:07.697 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818663+0000", 2024-09-11T07:51:07.697 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.698 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.698 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.698 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:51:07.698 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818663+0000", 2024-09-11T07:51:07.698 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999995 2024-09-11T07:51:07.698 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.698 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.698 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:51:07.699 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818669+0000", 2024-09-11T07:51:07.699 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.6740000000000002e-06 2024-09-11T07:51:07.699 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.699 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.699 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:51:07.699 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818675+0000", 2024-09-11T07:51:07.699 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.9920000000000004e-06 2024-09-11T07:51:07.699 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.699 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.699 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:51:07.699 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.818679+0000", 2024-09-11T07:51:07.700 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0879999999999997e-06 2024-09-11T07:51:07.700 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.700 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.700 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.700 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.822311+0000", 2024-09-11T07:51:07.700 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0036328530000000001 2024-09-11T07:51:07.700 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.700 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.700 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:51:07.700 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.822471+0000", 2024-09-11T07:51:07.701 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000159865 2024-09-11T07:51:07.701 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.701 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.701 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,1,2,3", 2024-09-11T07:51:07.701 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.822603+0000", 2024-09-11T07:51:07.701 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000131934 2024-09-11T07:51:07.701 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.701 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.701 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:51:07.701 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.825788+0000", 2024-09-11T07:51:07.701 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003184821 2024-09-11T07:51:07.701 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.701 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.701 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.702 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.834464+0000", 2024-09-11T07:51:07.702 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0086756890000000003 2024-09-11T07:51:07.702 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.702 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.702 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.702 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.836517+0000", 2024-09-11T07:51:07.702 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.002053685 2024-09-11T07:51:07.702 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.702 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.702 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.702 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.842225+0000", 2024-09-11T07:51:07.702 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.005707681 2024-09-11T07:51:07.702 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.702 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.703 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:51:07.703 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.842230+0000", 2024-09-11T07:51:07.703 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.7779999999999998e-06 2024-09-11T07:51:07.703 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.703 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.703 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.703 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.875289+0000", 2024-09-11T07:51:07.703 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0330593189999999 2024-09-11T07:51:07.703 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.703 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.703 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:51:07.703 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.875358+0000", 2024-09-11T07:51:07.704 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.8532e-05 2024-09-11T07:51:07.704 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.704 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:51:07.704 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.704 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.704 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.704 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:237434 11.f 11:fba8c028:::benchmark_data_smithi022_188626_object117626:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e469)", 2024-09-11T07:51:07.704 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:36.830957+0000", 2024-09-11T07:51:07.704 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 90.760357068000005, 2024-09-11T07:51:07.704 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.044451427, 2024-09-11T07:51:07.705 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:51:07.705 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:51:07.705 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:51:07.705 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:51:07.705 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:51:07.705 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 237434 2024-09-11T07:51:07.705 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.705 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:51:07.705 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.705 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:51:07.705 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.830957+0000", 2024-09-11T07:51:07.705 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.705 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.706 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.706 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:51:07.706 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.830957+0000", 2024-09-11T07:51:07.706 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.706 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.706 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.706 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:51:07.706 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.830955+0000", 2024-09-11T07:51:07.706 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:51:07.706 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.706 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.706 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:51:07.706 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.830964+0000", 2024-09-11T07:51:07.706 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.9259999999999995e-06 2024-09-11T07:51:07.707 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.707 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.707 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:51:07.707 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.830981+0000", 2024-09-11T07:51:07.707 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6646999999999998e-05 2024-09-11T07:51:07.707 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.707 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.707 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:51:07.707 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.830989+0000", 2024-09-11T07:51:07.707 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4549999999999998e-06 2024-09-11T07:51:07.707 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.707 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.707 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.707 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.831051+0000", 2024-09-11T07:51:07.707 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.2564999999999995e-05 2024-09-11T07:51:07.708 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.708 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.708 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:51:07.708 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.831217+0000", 2024-09-11T07:51:07.708 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000165661 2024-09-11T07:51:07.708 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.708 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.708 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,1,2,3", 2024-09-11T07:51:07.708 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.831378+0000", 2024-09-11T07:51:07.708 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00016110500000000001 2024-09-11T07:51:07.708 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.708 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.708 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:51:07.708 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.834560+0000", 2024-09-11T07:51:07.708 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003182463 2024-09-11T07:51:07.709 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.709 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.709 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.709 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.836772+0000", 2024-09-11T07:51:07.709 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0022113889999999998 2024-09-11T07:51:07.709 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.709 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.709 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.709 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.842253+0000", 2024-09-11T07:51:07.709 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0054810680000000004 2024-09-11T07:51:07.709 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.709 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.709 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:51:07.710 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.842261+0000", 2024-09-11T07:51:07.710 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.1850000000000004e-06 2024-09-11T07:51:07.710 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.710 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.710 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.710 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.875375+0000", 2024-09-11T07:51:07.710 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.033113741 2024-09-11T07:51:07.710 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.710 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.710 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:51:07.710 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.875409+0000", 2024-09-11T07:51:07.710 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.3779000000000003e-05 2024-09-11T07:51:07.710 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.710 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:51:07.710 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.711 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.711 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.711 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:237441 11.4 11:27d1a457:::benchmark_data_smithi022_188626_object117633:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e469)", 2024-09-11T07:51:07.711 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:36.837200+0000", 2024-09-11T07:51:07.711 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 90.754114212000005, 2024-09-11T07:51:07.711 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.044532706, 2024-09-11T07:51:07.711 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:51:07.711 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "reached pg", 2024-09-11T07:51:07.711 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:51:07.711 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:51:07.711 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:51:07.711 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 237441 2024-09-11T07:51:07.711 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.712 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:51:07.712 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.712 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:51:07.712 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.837200+0000", 2024-09-11T07:51:07.712 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.712 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.712 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.712 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:51:07.712 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.837200+0000", 2024-09-11T07:51:07.712 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.712 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.712 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.713 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:51:07.713 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.837197+0000", 2024-09-11T07:51:07.713 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:51:07.713 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.713 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.713 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:51:07.713 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.837205+0000", 2024-09-11T07:51:07.713 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0979999999999994e-06 2024-09-11T07:51:07.713 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.713 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.713 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:51:07.713 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.837220+0000", 2024-09-11T07:51:07.713 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.4389000000000001e-05 2024-09-11T07:51:07.714 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.714 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.714 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:51:07.714 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.837227+0000", 2024-09-11T07:51:07.714 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.8519999999999998e-06 2024-09-11T07:51:07.714 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.714 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.714 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.714 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.837290+0000", 2024-09-11T07:51:07.714 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3184999999999999e-05 2024-09-11T07:51:07.714 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.714 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.714 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:51:07.714 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.837438+0000", 2024-09-11T07:51:07.715 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000147782 2024-09-11T07:51:07.715 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.715 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.715 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,2,3,6", 2024-09-11T07:51:07.715 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.837565+0000", 2024-09-11T07:51:07.715 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012728199999999999 2024-09-11T07:51:07.715 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.715 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.715 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.715 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.842560+0000", 2024-09-11T07:51:07.715 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0049948960000000004 2024-09-11T07:51:07.715 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.715 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.715 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:51:07.716 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.842566+0000", 2024-09-11T07:51:07.716 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4479999999999999e-06 2024-09-11T07:51:07.716 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.716 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.716 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.716 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.871264+0000", 2024-09-11T07:51:07.716 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.028698069 2024-09-11T07:51:07.716 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.716 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.716 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for missing object", 2024-09-11T07:51:07.716 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.871345+0000", 2024-09-11T07:51:07.716 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.0586000000000004e-05 2024-09-11T07:51:07.716 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.717 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.717 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.717 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881683+0000", 2024-09-11T07:51:07.717 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.010338424000000001 2024-09-11T07:51:07.717 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.717 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.717 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:51:07.717 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881733+0000", 2024-09-11T07:51:07.717 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.9259999999999999e-05 2024-09-11T07:51:07.717 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.717 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:51:07.718 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.718 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.718 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.718 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:237470 11.4 11:22333720:::benchmark_data_smithi022_188626_object117662:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e470)", 2024-09-11T07:51:07.718 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:36.872959+0000", 2024-09-11T07:51:07.718 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 90.718355138999996, 2024-09-11T07:51:07.718 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0085034479999999, 2024-09-11T07:51:07.718 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:51:07.718 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:51:07.718 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:51:07.718 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:51:07.718 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:51:07.718 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 237470 2024-09-11T07:51:07.718 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.719 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:51:07.719 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.719 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:51:07.719 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.872959+0000", 2024-09-11T07:51:07.719 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.719 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.719 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.719 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:51:07.719 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.872959+0000", 2024-09-11T07:51:07.719 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.719 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.719 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.719 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:51:07.719 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.872956+0000", 2024-09-11T07:51:07.720 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999967 2024-09-11T07:51:07.720 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.720 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.720 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:51:07.720 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.872968+0000", 2024-09-11T07:51:07.720 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.2327000000000001e-05 2024-09-11T07:51:07.720 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.720 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.720 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:51:07.720 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.872987+0000", 2024-09-11T07:51:07.720 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.8729e-05 2024-09-11T07:51:07.720 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.720 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.721 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:51:07.721 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.872995+0000", 2024-09-11T07:51:07.721 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.4840000000000004e-06 2024-09-11T07:51:07.721 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.721 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.721 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.721 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.873041+0000", 2024-09-11T07:51:07.721 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.5136000000000003e-05 2024-09-11T07:51:07.721 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.721 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.721 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:51:07.722 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.873048+0000", 2024-09-11T07:51:07.722 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.542e-06 2024-09-11T07:51:07.722 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.722 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.722 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.722 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.871371+0000", 2024-09-11T07:51:07.722 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.99832300100000004 2024-09-11T07:51:07.722 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.722 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.722 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:51:07.722 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.871453+0000", 2024-09-11T07:51:07.722 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2058999999999995e-05 2024-09-11T07:51:07.722 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.723 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.723 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3,6", 2024-09-11T07:51:07.723 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.871570+0000", 2024-09-11T07:51:07.723 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00011685500000000001 2024-09-11T07:51:07.723 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.723 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.723 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:51:07.723 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.878964+0000", 2024-09-11T07:51:07.723 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0073936030000000003 2024-09-11T07:51:07.723 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.723 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.723 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.723 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881233+0000", 2024-09-11T07:51:07.723 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0022693180000000002 2024-09-11T07:51:07.723 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.724 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.724 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.724 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881265+0000", 2024-09-11T07:51:07.724 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1813000000000002e-05 2024-09-11T07:51:07.724 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.724 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.724 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.724 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881352+0000", 2024-09-11T07:51:07.724 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.7348999999999993e-05 2024-09-11T07:51:07.725 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.725 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.725 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:51:07.725 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881387+0000", 2024-09-11T07:51:07.725 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.5330999999999998e-05 2024-09-11T07:51:07.725 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.725 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.725 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:51:07.725 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881462+0000", 2024-09-11T07:51:07.725 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.5037999999999995e-05 2024-09-11T07:51:07.725 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.725 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:51:07.725 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.726 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.726 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.726 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:237477 11.4 11:215731e4:::benchmark_data_smithi022_188626_object117669:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e470)", 2024-09-11T07:51:07.726 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:36.878069+0000", 2024-09-11T07:51:07.726 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 90.713245009999994, 2024-09-11T07:51:07.726 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0040192189999999, 2024-09-11T07:51:07.726 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:51:07.726 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:51:07.726 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:51:07.726 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:51:07.726 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:51:07.726 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 237477 2024-09-11T07:51:07.726 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.727 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:51:07.727 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.727 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:51:07.727 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.878069+0000", 2024-09-11T07:51:07.727 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.727 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.727 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.727 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:51:07.727 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.878069+0000", 2024-09-11T07:51:07.727 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.727 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.727 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.727 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:51:07.728 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.878062+0000", 2024-09-11T07:51:07.728 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999928 2024-09-11T07:51:07.728 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.728 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.728 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:51:07.728 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.878079+0000", 2024-09-11T07:51:07.728 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.7456999999999999e-05 2024-09-11T07:51:07.728 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.728 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.728 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:51:07.728 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.878100+0000", 2024-09-11T07:51:07.728 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.0826e-05 2024-09-11T07:51:07.728 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.728 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.729 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:51:07.729 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.878111+0000", 2024-09-11T07:51:07.729 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1073000000000001e-05 2024-09-11T07:51:07.729 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.729 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.729 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.729 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.878173+0000", 2024-09-11T07:51:07.729 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1766999999999993e-05 2024-09-11T07:51:07.729 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.729 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.729 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:51:07.729 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.878184+0000", 2024-09-11T07:51:07.730 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1048e-05 2024-09-11T07:51:07.730 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.730 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.730 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.730 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.873786+0000", 2024-09-11T07:51:07.730 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.99560231399999999 2024-09-11T07:51:07.730 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.730 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.730 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:51:07.730 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.873893+0000", 2024-09-11T07:51:07.730 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00010639100000000001 2024-09-11T07:51:07.730 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.730 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.731 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3,6", 2024-09-11T07:51:07.731 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.874016+0000", 2024-09-11T07:51:07.731 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000122998 2024-09-11T07:51:07.731 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.731 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.731 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:51:07.731 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881112+0000", 2024-09-11T07:51:07.731 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0070964009999999996 2024-09-11T07:51:07.731 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.731 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.731 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.731 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881292+0000", 2024-09-11T07:51:07.731 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00017989899999999999 2024-09-11T07:51:07.731 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.732 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.732 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.732 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881324+0000", 2024-09-11T07:51:07.732 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.1776000000000002e-05 2024-09-11T07:51:07.732 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.732 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.732 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.732 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881978+0000", 2024-09-11T07:51:07.732 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00065408700000000003 2024-09-11T07:51:07.732 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.732 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.732 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:51:07.732 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.882018+0000", 2024-09-11T07:51:07.733 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4.0346000000000003e-05 2024-09-11T07:51:07.733 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.733 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.733 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:51:07.733 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.882088+0000", 2024-09-11T07:51:07.733 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.9967000000000003e-05 2024-09-11T07:51:07.733 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.733 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:51:07.733 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.733 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.733 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.733 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:237493 11.4 11:2e313620:::benchmark_data_smithi022_188626_object117685:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e470)", 2024-09-11T07:51:07.733 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:36.890350+0000", 2024-09-11T07:51:07.733 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 90.700964573999997, 2024-09-11T07:51:07.734 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.99186289699999997, 2024-09-11T07:51:07.734 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:51:07.734 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:51:07.734 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:51:07.734 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:51:07.734 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:51:07.734 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 237493 2024-09-11T07:51:07.734 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.734 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:51:07.734 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.734 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:51:07.734 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.890350+0000", 2024-09-11T07:51:07.734 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.734 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.735 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.735 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:51:07.735 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.890350+0000", 2024-09-11T07:51:07.735 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.735 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.735 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.735 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:51:07.735 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.890346+0000", 2024-09-11T07:51:07.735 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999967 2024-09-11T07:51:07.735 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.735 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.735 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:51:07.735 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.890357+0000", 2024-09-11T07:51:07.735 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.1063000000000001e-05 2024-09-11T07:51:07.736 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.736 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.736 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:51:07.736 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.890377+0000", 2024-09-11T07:51:07.736 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.9568e-05 2024-09-11T07:51:07.736 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.736 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.736 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:51:07.736 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.890385+0000", 2024-09-11T07:51:07.736 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.2989999999999997e-06 2024-09-11T07:51:07.736 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.736 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.737 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.737 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.890448+0000", 2024-09-11T07:51:07.737 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.3238999999999995e-05 2024-09-11T07:51:07.737 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.737 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.737 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:51:07.737 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.890458+0000", 2024-09-11T07:51:07.737 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.9920000000000002e-06 2024-09-11T07:51:07.737 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.737 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.737 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.737 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.876601+0000", 2024-09-11T07:51:07.737 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.98614258200000005 2024-09-11T07:51:07.737 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.738 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.738 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:51:07.738 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.876694+0000", 2024-09-11T07:51:07.738 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.3158999999999994e-05 2024-09-11T07:51:07.738 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.738 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.738 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3,6", 2024-09-11T07:51:07.738 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.876773+0000", 2024-09-11T07:51:07.738 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9018999999999997e-05 2024-09-11T07:51:07.738 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.738 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.738 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:51:07.738 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881238+0000", 2024-09-11T07:51:07.738 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0044648559999999997 2024-09-11T07:51:07.739 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.739 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.739 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.739 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881785+0000", 2024-09-11T07:51:07.739 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00054708000000000003 2024-09-11T07:51:07.739 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.739 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.739 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.739 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881864+0000", 2024-09-11T07:51:07.739 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9532999999999999e-05 2024-09-11T07:51:07.739 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.740 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.740 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.740 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.882109+0000", 2024-09-11T07:51:07.740 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00024501200000000002 2024-09-11T07:51:07.740 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.740 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.740 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:51:07.740 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.882138+0000", 2024-09-11T07:51:07.740 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.8235999999999999e-05 2024-09-11T07:51:07.740 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.741 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.741 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:51:07.741 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.882212+0000", 2024-09-11T07:51:07.741 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.4734000000000006e-05 2024-09-11T07:51:07.741 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.741 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:51:07.741 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.741 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.741 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.741 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:237498 11.f 11:f14bae7d:::benchmark_data_smithi022_188626_object117690:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e470)", 2024-09-11T07:51:07.741 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:36.896808+0000", 2024-09-11T07:51:07.741 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 90.694506153000006, 2024-09-11T07:51:07.741 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.0266028819999999, 2024-09-11T07:51:07.742 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:51:07.742 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:51:07.742 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:51:07.742 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:51:07.742 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:51:07.742 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 237498 2024-09-11T07:51:07.742 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.742 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:51:07.742 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.742 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:51:07.742 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.896808+0000", 2024-09-11T07:51:07.742 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.742 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.743 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.743 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:51:07.743 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.896808+0000", 2024-09-11T07:51:07.743 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.743 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.743 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.743 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:51:07.743 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.896805+0000", 2024-09-11T07:51:07.743 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999967 2024-09-11T07:51:07.743 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.743 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.743 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:51:07.743 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.896821+0000", 2024-09-11T07:51:07.743 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.6093e-05 2024-09-11T07:51:07.744 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.744 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.744 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:51:07.744 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.896852+0000", 2024-09-11T07:51:07.744 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.0973999999999999e-05 2024-09-11T07:51:07.744 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.744 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.744 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:51:07.744 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.896859+0000", 2024-09-11T07:51:07.744 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.8720000000000002e-06 2024-09-11T07:51:07.744 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.744 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.745 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.745 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.896921+0000", 2024-09-11T07:51:07.745 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.1914000000000004e-05 2024-09-11T07:51:07.745 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.745 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.745 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:51:07.745 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.896931+0000", 2024-09-11T07:51:07.745 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.6379999999999999e-06 2024-09-11T07:51:07.745 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.745 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.746 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.746 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.875435+0000", 2024-09-11T07:51:07.746 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.97850396699999997 2024-09-11T07:51:07.746 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.746 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.746 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:51:07.746 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.875548+0000", 2024-09-11T07:51:07.746 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.000113455 2024-09-11T07:51:07.746 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.746 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.746 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 0,1,3", 2024-09-11T07:51:07.746 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.875675+0000", 2024-09-11T07:51:07.746 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012674600000000001 2024-09-11T07:51:07.747 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.747 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.747 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.747 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.878705+0000", 2024-09-11T07:51:07.747 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.003029988 2024-09-11T07:51:07.747 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.747 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.747 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:51:07.747 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.878731+0000", 2024-09-11T07:51:07.747 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.5667999999999999e-05 2024-09-11T07:51:07.747 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.747 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.747 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.747 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.878860+0000", 2024-09-11T07:51:07.748 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00012885300000000001 2024-09-11T07:51:07.748 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.748 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.748 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.748 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.923310+0000", 2024-09-11T07:51:07.748 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.044450305000000002 2024-09-11T07:51:07.748 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.748 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.748 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:51:07.748 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.923346+0000", 2024-09-11T07:51:07.748 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 3.6426e-05 2024-09-11T07:51:07.748 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.748 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.749 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:51:07.749 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.923411+0000", 2024-09-11T07:51:07.749 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 6.4450999999999997e-05 2024-09-11T07:51:07.749 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.749 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:51:07.749 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.749 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.749 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.749 INFO:teuthology.orchestra.run.smithi104.stdout: "description": "osd_op(client.25990.0:237502 11.4 11:292f92e5:::benchmark_data_smithi022_188626_object117694:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e470)", 2024-09-11T07:51:07.749 INFO:teuthology.orchestra.run.smithi104.stdout: "initiated_at": "2024-09-11T07:49:36.901577+0000", 2024-09-11T07:51:07.749 INFO:teuthology.orchestra.run.smithi104.stdout: "age": 90.689736843000006, 2024-09-11T07:51:07.749 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.98077161899999998, 2024-09-11T07:51:07.749 INFO:teuthology.orchestra.run.smithi104.stdout: "type_data": { 2024-09-11T07:51:07.749 INFO:teuthology.orchestra.run.smithi104.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:51:07.750 INFO:teuthology.orchestra.run.smithi104.stdout: "client_info": { 2024-09-11T07:51:07.750 INFO:teuthology.orchestra.run.smithi104.stdout: "client": "client.25990", 2024-09-11T07:51:07.750 INFO:teuthology.orchestra.run.smithi104.stdout: "client_addr": "v1:172.21.15.22:0/1051161942", 2024-09-11T07:51:07.750 INFO:teuthology.orchestra.run.smithi104.stdout: "tid": 237502 2024-09-11T07:51:07.750 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.750 INFO:teuthology.orchestra.run.smithi104.stdout: "events": [ 2024-09-11T07:51:07.750 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.750 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "initiated", 2024-09-11T07:51:07.750 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.901577+0000", 2024-09-11T07:51:07.750 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.750 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.750 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.750 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "throttled", 2024-09-11T07:51:07.750 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.901577+0000", 2024-09-11T07:51:07.751 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0 2024-09-11T07:51:07.751 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.751 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.751 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "header_read", 2024-09-11T07:51:07.751 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.901575+0000", 2024-09-11T07:51:07.751 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 4294967295.9999976 2024-09-11T07:51:07.751 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.751 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.751 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "all_read", 2024-09-11T07:51:07.751 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.901584+0000", 2024-09-11T07:51:07.751 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 9.679e-06 2024-09-11T07:51:07.751 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.751 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.752 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "dispatched", 2024-09-11T07:51:07.752 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.901600+0000", 2024-09-11T07:51:07.752 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 1.5688e-05 2024-09-11T07:51:07.752 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.752 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.752 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "queued_for_pg", 2024-09-11T07:51:07.752 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.901606+0000", 2024-09-11T07:51:07.752 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.8200000000000002e-06 2024-09-11T07:51:07.752 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.752 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.752 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.752 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.901656+0000", 2024-09-11T07:51:07.753 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 5.0343000000000002e-05 2024-09-11T07:51:07.753 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.753 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.753 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for peered", 2024-09-11T07:51:07.753 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:36.901664+0000", 2024-09-11T07:51:07.753 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.661e-06 2024-09-11T07:51:07.753 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.753 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.753 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "reached_pg", 2024-09-11T07:51:07.753 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.878992+0000", 2024-09-11T07:51:07.753 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.97732787399999999 2024-09-11T07:51:07.754 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.754 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.754 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "started", 2024-09-11T07:51:07.754 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.879080+0000", 2024-09-11T07:51:07.754 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 8.8219000000000004e-05 2024-09-11T07:51:07.754 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.754 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.754 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "waiting for subops from 1,3,6", 2024-09-11T07:51:07.754 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.879160+0000", 2024-09-11T07:51:07.754 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.9611999999999996e-05 2024-09-11T07:51:07.754 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.754 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.754 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "op_commit", 2024-09-11T07:51:07.754 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881299+0000", 2024-09-11T07:51:07.755 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.0021391800000000001 2024-09-11T07:51:07.755 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.755 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.755 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.755 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881818+0000", 2024-09-11T07:51:07.755 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00051918299999999999 2024-09-11T07:51:07.755 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.755 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.755 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.755 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.881961+0000", 2024-09-11T07:51:07.755 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00014311799999999999 2024-09-11T07:51:07.755 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.756 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.756 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "sub_op_commit_rec", 2024-09-11T07:51:07.756 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.882254+0000", 2024-09-11T07:51:07.756 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 0.00029237300000000001 2024-09-11T07:51:07.756 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.756 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.756 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "commit_sent", 2024-09-11T07:51:07.756 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.882277+0000", 2024-09-11T07:51:07.756 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 2.3643e-05 2024-09-11T07:51:07.756 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-11T07:51:07.756 INFO:teuthology.orchestra.run.smithi104.stdout: { 2024-09-11T07:51:07.756 INFO:teuthology.orchestra.run.smithi104.stdout: "event": "done", 2024-09-11T07:51:07.756 INFO:teuthology.orchestra.run.smithi104.stdout: "time": "2024-09-11T07:49:37.882349+0000", 2024-09-11T07:51:07.757 INFO:teuthology.orchestra.run.smithi104.stdout: "duration": 7.1738999999999999e-05 2024-09-11T07:51:07.757 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.757 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:51:07.757 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.757 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-11T07:51:07.757 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-11T07:51:07.757 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-11T07:51:08.029 INFO:teuthology.orchestra.run.smithi022.stderr:reweighted osd.1 to 1 (10000) 2024-09-11T07:51:08.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:08 smithi022 ceph-mon[29462]: pgmap v1150: 142 pgs: 142 active+clean; 483 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.9 KiB/s rd, 1 op/s; 2 B/s, 0 objects/s recovering 2024-09-11T07:51:08.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:08 smithi022 ceph-mon[29462]: 2.e deep-scrub starts 2024-09-11T07:51:08.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:08 smithi022 ceph-mon[29462]: 2.e deep-scrub ok 2024-09-11T07:51:08.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:08 smithi022 ceph-mon[29462]: 6.b scrub starts 2024-09-11T07:51:08.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:08 smithi022 ceph-mon[29462]: 6.b scrub ok 2024-09-11T07:51:08.375 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:08 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2670571775' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T07:51:08.375 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:08 smithi022 ceph-mon[37330]: pgmap v1150: 142 pgs: 142 active+clean; 483 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.9 KiB/s rd, 1 op/s; 2 B/s, 0 objects/s recovering 2024-09-11T07:51:08.376 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:08 smithi022 ceph-mon[37330]: 2.e deep-scrub starts 2024-09-11T07:51:08.376 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:08 smithi022 ceph-mon[37330]: 2.e deep-scrub ok 2024-09-11T07:51:08.376 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:08 smithi022 ceph-mon[37330]: 6.b scrub starts 2024-09-11T07:51:08.376 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:08 smithi022 ceph-mon[37330]: 6.b scrub ok 2024-09-11T07:51:08.376 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:08 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2670571775' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T07:51:08.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:08 smithi104 ceph-mon[33507]: pgmap v1150: 142 pgs: 142 active+clean; 483 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.9 KiB/s rd, 1 op/s; 2 B/s, 0 objects/s recovering 2024-09-11T07:51:08.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:08 smithi104 ceph-mon[33507]: 2.e deep-scrub starts 2024-09-11T07:51:08.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:08 smithi104 ceph-mon[33507]: 2.e deep-scrub ok 2024-09-11T07:51:08.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:08 smithi104 ceph-mon[33507]: 6.b scrub starts 2024-09-11T07:51:08.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:08 smithi104 ceph-mon[33507]: 6.b scrub ok 2024-09-11T07:51:08.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:08 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2670571775' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T07:51:08.622 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-11T07:51:08.793 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 1 1 2024-09-11T07:51:08.885 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:51:09.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:09 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2670571775' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-11T07:51:09.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:09 smithi022 ceph-mon[29462]: osdmap e499: 8 total, 8 up, 8 in 2024-09-11T07:51:09.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:09 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:51:09.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:09 smithi022 ceph-mon[29462]: 5.a scrub starts 2024-09-11T07:51:09.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:09 smithi022 ceph-mon[29462]: 5.a scrub ok 2024-09-11T07:51:09.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:09 smithi022 ceph-mon[29462]: 6.7 scrub starts 2024-09-11T07:51:09.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:09 smithi022 ceph-mon[29462]: 6.7 scrub ok 2024-09-11T07:51:09.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:09 smithi022 ceph-mon[29462]: 6.3 scrub starts 2024-09-11T07:51:09.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:09 smithi022 ceph-mon[29462]: 6.3 scrub ok 2024-09-11T07:51:09.322 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:09 smithi022 ceph-mon[29462]: osdmap e500: 8 total, 8 up, 8 in 2024-09-11T07:51:09.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:09 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2670571775' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-11T07:51:09.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:09 smithi022 ceph-mon[37330]: osdmap e499: 8 total, 8 up, 8 in 2024-09-11T07:51:09.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:09 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:51:09.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:09 smithi022 ceph-mon[37330]: 5.a scrub starts 2024-09-11T07:51:09.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:09 smithi022 ceph-mon[37330]: 5.a scrub ok 2024-09-11T07:51:09.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:09 smithi022 ceph-mon[37330]: 6.7 scrub starts 2024-09-11T07:51:09.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:09 smithi022 ceph-mon[37330]: 6.7 scrub ok 2024-09-11T07:51:09.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:09 smithi022 ceph-mon[37330]: 6.3 scrub starts 2024-09-11T07:51:09.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:09 smithi022 ceph-mon[37330]: 6.3 scrub ok 2024-09-11T07:51:09.323 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:09 smithi022 ceph-mon[37330]: osdmap e500: 8 total, 8 up, 8 in 2024-09-11T07:51:09.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:09 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2670571775' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-11T07:51:09.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:09 smithi104 ceph-mon[33507]: osdmap e499: 8 total, 8 up, 8 in 2024-09-11T07:51:09.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:09 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:51:09.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:09 smithi104 ceph-mon[33507]: 5.a scrub starts 2024-09-11T07:51:09.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:09 smithi104 ceph-mon[33507]: 5.a scrub ok 2024-09-11T07:51:09.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:09 smithi104 ceph-mon[33507]: 6.7 scrub starts 2024-09-11T07:51:09.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:09 smithi104 ceph-mon[33507]: 6.7 scrub ok 2024-09-11T07:51:09.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:09 smithi104 ceph-mon[33507]: 6.3 scrub starts 2024-09-11T07:51:09.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:09 smithi104 ceph-mon[33507]: 6.3 scrub ok 2024-09-11T07:51:09.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:09 smithi104 ceph-mon[33507]: osdmap e500: 8 total, 8 up, 8 in 2024-09-11T07:51:09.544 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:51:10.372 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:10 smithi022 ceph-mon[29462]: 4.6 scrub starts 2024-09-11T07:51:10.373 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:10 smithi022 ceph-mon[29462]: 4.6 scrub ok 2024-09-11T07:51:10.373 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:10 smithi022 ceph-mon[29462]: pgmap v1152: 142 pgs: 142 active+clean; 483 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:51:10.373 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:10 smithi022 ceph-mon[37330]: 4.6 scrub starts 2024-09-11T07:51:10.373 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:10 smithi022 ceph-mon[37330]: 4.6 scrub ok 2024-09-11T07:51:10.374 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:10 smithi022 ceph-mon[37330]: pgmap v1152: 142 pgs: 142 active+clean; 483 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:51:10.458 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:10 smithi104 ceph-mon[33507]: 4.6 scrub starts 2024-09-11T07:51:10.459 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:10 smithi104 ceph-mon[33507]: 4.6 scrub ok 2024-09-11T07:51:10.459 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:10 smithi104 ceph-mon[33507]: pgmap v1152: 142 pgs: 142 active+clean; 483 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:51:10.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:51:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:51:10] "GET /metrics HTTP/1.1" 200 222464 "" "Prometheus/2.43.0" 2024-09-11T07:51:10.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:51:10 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:51:11.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:11 smithi022 ceph-mon[29462]: 5.8 scrub starts 2024-09-11T07:51:11.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:11 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:51:11.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:11 smithi022 ceph-mon[29462]: 6.12 scrub starts 2024-09-11T07:51:11.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:11 smithi022 ceph-mon[29462]: 6.12 scrub ok 2024-09-11T07:51:11.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:11 smithi022 ceph-mon[37330]: 5.8 scrub starts 2024-09-11T07:51:11.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:11 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:51:11.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:11 smithi022 ceph-mon[37330]: 6.12 scrub starts 2024-09-11T07:51:11.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:11 smithi022 ceph-mon[37330]: 6.12 scrub ok 2024-09-11T07:51:11.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:11 smithi104 ceph-mon[33507]: 5.8 scrub starts 2024-09-11T07:51:11.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:11 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:51:11.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:11 smithi104 ceph-mon[33507]: 6.12 scrub starts 2024-09-11T07:51:11.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:11 smithi104 ceph-mon[33507]: 6.12 scrub ok 2024-09-11T07:51:12.258 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:12 smithi022 ceph-mon[37330]: 5.8 scrub ok 2024-09-11T07:51:12.259 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:12 smithi022 ceph-mon[37330]: 5.1b scrub starts 2024-09-11T07:51:12.259 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:12 smithi022 ceph-mon[37330]: 5.1b scrub ok 2024-09-11T07:51:12.259 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:12 smithi022 ceph-mon[37330]: pgmap v1154: 142 pgs: 142 active+clean; 483 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 847 B/s rd, 0 op/s; 91 B/s, 3 objects/s recovering 2024-09-11T07:51:12.259 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:12 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-09-11T07:51:12.259 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:12 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2024-09-11T07:51:12.259 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:12 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [3, 7]}]: dispatch 2024-09-11T07:51:12.259 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:12 smithi022 ceph-mon[37330]: 5.10 scrub starts 2024-09-11T07:51:12.259 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:12 smithi022 ceph-mon[37330]: 5.10 scrub ok 2024-09-11T07:51:12.259 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:12 smithi022 ceph-mon[29462]: 5.8 scrub ok 2024-09-11T07:51:12.259 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:12 smithi022 ceph-mon[29462]: 5.1b scrub starts 2024-09-11T07:51:12.259 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:12 smithi022 ceph-mon[29462]: 5.1b scrub ok 2024-09-11T07:51:12.260 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:12 smithi022 ceph-mon[29462]: pgmap v1154: 142 pgs: 142 active+clean; 483 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 847 B/s rd, 0 op/s; 91 B/s, 3 objects/s recovering 2024-09-11T07:51:12.260 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:12 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-09-11T07:51:12.260 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:12 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2024-09-11T07:51:12.260 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:12 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [3, 7]}]: dispatch 2024-09-11T07:51:12.260 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:12 smithi022 ceph-mon[29462]: 5.10 scrub starts 2024-09-11T07:51:12.260 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:12 smithi022 ceph-mon[29462]: 5.10 scrub ok 2024-09-11T07:51:12.375 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:51:12.375 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:51:12.375 INFO:teuthology.orchestra.run.smithi022.stdout: "num_ops": 0 2024-09-11T07:51:12.375 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:51:12.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:12 smithi104 ceph-mon[33507]: 5.8 scrub ok 2024-09-11T07:51:12.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:12 smithi104 ceph-mon[33507]: 5.1b scrub starts 2024-09-11T07:51:12.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:12 smithi104 ceph-mon[33507]: 5.1b scrub ok 2024-09-11T07:51:12.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:12 smithi104 ceph-mon[33507]: pgmap v1154: 142 pgs: 142 active+clean; 483 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 847 B/s rd, 0 op/s; 91 B/s, 3 objects/s recovering 2024-09-11T07:51:12.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:12 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-09-11T07:51:12.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:12 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2024-09-11T07:51:12.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:12 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [3, 7]}]: dispatch 2024-09-11T07:51:12.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:12 smithi104 ceph-mon[33507]: 5.10 scrub starts 2024-09-11T07:51:12.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:12 smithi104 ceph-mon[33507]: 5.10 scrub ok 2024-09-11T07:51:13.048 INFO:teuthology.orchestra.run.smithi022.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-11T07:51:13.182 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-11T07:51:13.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[29462]: 4.5 scrub starts 2024-09-11T07:51:13.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[29462]: 4.5 scrub ok 2024-09-11T07:51:13.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[29462]: 4.16 scrub starts 2024-09-11T07:51:13.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[29462]: 4.16 scrub ok 2024-09-11T07:51:13.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:51:13.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:51:13.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-09-11T07:51:13.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]': finished 2024-09-11T07:51:13.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [3, 7]}]': finished 2024-09-11T07:51:13.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[29462]: osdmap e501: 8 total, 8 up, 8 in 2024-09-11T07:51:13.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/399062180' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T07:51:13.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[29462]: 2.14 scrub starts 2024-09-11T07:51:13.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[29462]: 2.14 scrub ok 2024-09-11T07:51:13.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[29462]: 5.17 scrub starts 2024-09-11T07:51:13.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[29462]: 5.17 scrub ok 2024-09-11T07:51:13.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/399062180' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-11T07:51:13.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[29462]: osdmap e502: 8 total, 8 up, 8 in 2024-09-11T07:51:13.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[37330]: 4.5 scrub starts 2024-09-11T07:51:13.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[37330]: 4.5 scrub ok 2024-09-11T07:51:13.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[37330]: 4.16 scrub starts 2024-09-11T07:51:13.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[37330]: 4.16 scrub ok 2024-09-11T07:51:13.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:51:13.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:51:13.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-09-11T07:51:13.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]': finished 2024-09-11T07:51:13.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [3, 7]}]': finished 2024-09-11T07:51:13.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[37330]: osdmap e501: 8 total, 8 up, 8 in 2024-09-11T07:51:13.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/399062180' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T07:51:13.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[37330]: 2.14 scrub starts 2024-09-11T07:51:13.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[37330]: 2.14 scrub ok 2024-09-11T07:51:13.420 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[37330]: 5.17 scrub starts 2024-09-11T07:51:13.420 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[37330]: 5.17 scrub ok 2024-09-11T07:51:13.420 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/399062180' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-11T07:51:13.420 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:13 smithi022 ceph-mon[37330]: osdmap e502: 8 total, 8 up, 8 in 2024-09-11T07:51:13.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:13 smithi104 ceph-mon[33507]: 4.5 scrub starts 2024-09-11T07:51:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:13 smithi104 ceph-mon[33507]: 4.5 scrub ok 2024-09-11T07:51:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:13 smithi104 ceph-mon[33507]: 4.16 scrub starts 2024-09-11T07:51:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:13 smithi104 ceph-mon[33507]: 4.16 scrub ok 2024-09-11T07:51:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:13 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:51:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:13 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:51:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:13 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-09-11T07:51:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:13 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]': finished 2024-09-11T07:51:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:13 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [3, 7]}]': finished 2024-09-11T07:51:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:13 smithi104 ceph-mon[33507]: osdmap e501: 8 total, 8 up, 8 in 2024-09-11T07:51:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:13 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/399062180' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-09-11T07:51:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:13 smithi104 ceph-mon[33507]: 2.14 scrub starts 2024-09-11T07:51:13.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:13 smithi104 ceph-mon[33507]: 2.14 scrub ok 2024-09-11T07:51:13.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:13 smithi104 ceph-mon[33507]: 5.17 scrub starts 2024-09-11T07:51:13.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:13 smithi104 ceph-mon[33507]: 5.17 scrub ok 2024-09-11T07:51:13.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:13 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/399062180' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-11T07:51:13.531 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:13 smithi104 ceph-mon[33507]: osdmap e502: 8 total, 8 up, 8 in 2024-09-11T07:51:13.958 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:51:13.983 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight 3 1 2024-09-11T07:51:14.231 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:14 smithi022 ceph-mon[37330]: 5.5 scrub starts 2024-09-11T07:51:14.232 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:14 smithi022 ceph-mon[37330]: 5.5 scrub ok 2024-09-11T07:51:14.232 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:14 smithi022 ceph-mon[37330]: pgmap v1156: 142 pgs: 142 active+clean; 483 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 169 B/s, 5 objects/s recovering 2024-09-11T07:51:14.232 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:14 smithi022 ceph-mon[37330]: 4.1f scrub starts 2024-09-11T07:51:14.232 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:14 smithi022 ceph-mon[37330]: 4.1f scrub ok 2024-09-11T07:51:14.232 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:14 smithi022 ceph-mon[37330]: osdmap e503: 8 total, 8 up, 8 in 2024-09-11T07:51:14.232 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:14 smithi022 ceph-mon[29462]: 5.5 scrub starts 2024-09-11T07:51:14.232 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:14 smithi022 ceph-mon[29462]: 5.5 scrub ok 2024-09-11T07:51:14.232 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:14 smithi022 ceph-mon[29462]: pgmap v1156: 142 pgs: 142 active+clean; 483 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 169 B/s, 5 objects/s recovering 2024-09-11T07:51:14.232 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:14 smithi022 ceph-mon[29462]: 4.1f scrub starts 2024-09-11T07:51:14.233 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:14 smithi022 ceph-mon[29462]: 4.1f scrub ok 2024-09-11T07:51:14.233 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:14 smithi022 ceph-mon[29462]: osdmap e503: 8 total, 8 up, 8 in 2024-09-11T07:51:14.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:51:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:51:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:51:14.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:14 smithi104 ceph-mon[33507]: 5.5 scrub starts 2024-09-11T07:51:14.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:14 smithi104 ceph-mon[33507]: 5.5 scrub ok 2024-09-11T07:51:14.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:14 smithi104 ceph-mon[33507]: pgmap v1156: 142 pgs: 142 active+clean; 483 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 169 B/s, 5 objects/s recovering 2024-09-11T07:51:14.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:14 smithi104 ceph-mon[33507]: 4.1f scrub starts 2024-09-11T07:51:14.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:14 smithi104 ceph-mon[33507]: 4.1f scrub ok 2024-09-11T07:51:14.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:14 smithi104 ceph-mon[33507]: osdmap e503: 8 total, 8 up, 8 in 2024-09-11T07:51:14.721 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:51:15.358 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:15 smithi022 ceph-mon[29462]: 2.18 scrub starts 2024-09-11T07:51:15.358 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:15 smithi022 ceph-mon[29462]: 2.18 scrub ok 2024-09-11T07:51:15.358 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:15 smithi022 ceph-mon[37330]: 2.18 scrub starts 2024-09-11T07:51:15.358 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:15 smithi022 ceph-mon[37330]: 2.18 scrub ok 2024-09-11T07:51:15.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:15 smithi104 ceph-mon[33507]: 2.18 scrub starts 2024-09-11T07:51:15.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:15 smithi104 ceph-mon[33507]: 2.18 scrub ok 2024-09-11T07:51:16.388 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:16 smithi022 ceph-mon[29462]: 2.f scrub starts 2024-09-11T07:51:16.388 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:16 smithi022 ceph-mon[29462]: 2.f scrub ok 2024-09-11T07:51:16.388 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:16 smithi022 ceph-mon[29462]: pgmap v1159: 142 pgs: 142 active+clean; 535 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 25 KiB/s wr, 1 op/s; 8.5 KiB/s, 0 objects/s recovering 2024-09-11T07:51:16.388 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:16 smithi022 ceph-mon[37330]: 2.f scrub starts 2024-09-11T07:51:16.389 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:16 smithi022 ceph-mon[37330]: 2.f scrub ok 2024-09-11T07:51:16.389 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:16 smithi022 ceph-mon[37330]: pgmap v1159: 142 pgs: 142 active+clean; 535 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 25 KiB/s wr, 1 op/s; 8.5 KiB/s, 0 objects/s recovering 2024-09-11T07:51:16.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:16 smithi104 ceph-mon[33507]: 2.f scrub starts 2024-09-11T07:51:16.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:16 smithi104 ceph-mon[33507]: 2.f scrub ok 2024-09-11T07:51:16.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:16 smithi104 ceph-mon[33507]: pgmap v1159: 142 pgs: 142 active+clean; 535 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 25 KiB/s wr, 1 op/s; 8.5 KiB/s, 0 objects/s recovering 2024-09-11T07:51:16.953 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:51:16.953 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [], 2024-09-11T07:51:16.953 INFO:teuthology.orchestra.run.smithi022.stdout: "complaint_time": 30, 2024-09-11T07:51:16.953 INFO:teuthology.orchestra.run.smithi022.stdout: "num_blocked_ops": 0 2024-09-11T07:51:16.953 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:51:17.388 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:17 smithi022 ceph-mon[29462]: 6.11 deep-scrub starts 2024-09-11T07:51:17.388 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:17 smithi022 ceph-mon[29462]: 6.11 deep-scrub ok 2024-09-11T07:51:17.388 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:17 smithi022 ceph-mon[29462]: 2.4 scrub starts 2024-09-11T07:51:17.389 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:17 smithi022 ceph-mon[29462]: 2.4 scrub ok 2024-09-11T07:51:17.389 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:17 smithi022 ceph-mon[29462]: 5.1a scrub starts 2024-09-11T07:51:17.389 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:17 smithi022 ceph-mon[29462]: 5.1a scrub ok 2024-09-11T07:51:17.389 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:17 smithi022 ceph-mon[37330]: 6.11 deep-scrub starts 2024-09-11T07:51:17.389 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:17 smithi022 ceph-mon[37330]: 6.11 deep-scrub ok 2024-09-11T07:51:17.389 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:17 smithi022 ceph-mon[37330]: 2.4 scrub starts 2024-09-11T07:51:17.389 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:17 smithi022 ceph-mon[37330]: 2.4 scrub ok 2024-09-11T07:51:17.389 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:17 smithi022 ceph-mon[37330]: 5.1a scrub starts 2024-09-11T07:51:17.389 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:17 smithi022 ceph-mon[37330]: 5.1a scrub ok 2024-09-11T07:51:17.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:17 smithi104 ceph-mon[33507]: 6.11 deep-scrub starts 2024-09-11T07:51:17.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:17 smithi104 ceph-mon[33507]: 6.11 deep-scrub ok 2024-09-11T07:51:17.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:17 smithi104 ceph-mon[33507]: 2.4 scrub starts 2024-09-11T07:51:17.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:17 smithi104 ceph-mon[33507]: 2.4 scrub ok 2024-09-11T07:51:17.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:17 smithi104 ceph-mon[33507]: 5.1a scrub starts 2024-09-11T07:51:17.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:17 smithi104 ceph-mon[33507]: 5.1a scrub ok 2024-09-11T07:51:17.872 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-11T07:51:18.084 INFO:teuthology.orchestra.run.smithi022.stderr:reweighted osd.3 to 1 (10000) 2024-09-11T07:51:18.102 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:18 smithi022 ceph-mon[29462]: pgmap v1160: 142 pgs: 142 active+clean; 535 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 25 KiB/s wr, 1 op/s; 8.3 KiB/s, 0 objects/s recovering 2024-09-11T07:51:18.102 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:18 smithi022 ceph-mon[29462]: 2.1 scrub starts 2024-09-11T07:51:18.103 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:18 smithi022 ceph-mon[29462]: 2.1 scrub ok 2024-09-11T07:51:18.103 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:18 smithi022 ceph-mon[29462]: 6.c scrub starts 2024-09-11T07:51:18.103 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:18 smithi022 ceph-mon[29462]: 6.c scrub ok 2024-09-11T07:51:18.103 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:18 smithi022 ceph-mon[29462]: 2.b scrub starts 2024-09-11T07:51:18.103 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:18 smithi022 ceph-mon[29462]: 2.b scrub ok 2024-09-11T07:51:18.103 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:18 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2998218911' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T07:51:18.103 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:18 smithi022 ceph-mon[37330]: pgmap v1160: 142 pgs: 142 active+clean; 535 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 25 KiB/s wr, 1 op/s; 8.3 KiB/s, 0 objects/s recovering 2024-09-11T07:51:18.103 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:18 smithi022 ceph-mon[37330]: 2.1 scrub starts 2024-09-11T07:51:18.103 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:18 smithi022 ceph-mon[37330]: 2.1 scrub ok 2024-09-11T07:51:18.103 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:18 smithi022 ceph-mon[37330]: 6.c scrub starts 2024-09-11T07:51:18.103 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:18 smithi022 ceph-mon[37330]: 6.c scrub ok 2024-09-11T07:51:18.104 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:18 smithi022 ceph-mon[37330]: 2.b scrub starts 2024-09-11T07:51:18.104 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:18 smithi022 ceph-mon[37330]: 2.b scrub ok 2024-09-11T07:51:18.104 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:18 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2998218911' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T07:51:18.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:18 smithi104 ceph-mon[33507]: pgmap v1160: 142 pgs: 142 active+clean; 535 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 25 KiB/s wr, 1 op/s; 8.3 KiB/s, 0 objects/s recovering 2024-09-11T07:51:18.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:18 smithi104 ceph-mon[33507]: 2.1 scrub starts 2024-09-11T07:51:18.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:18 smithi104 ceph-mon[33507]: 2.1 scrub ok 2024-09-11T07:51:18.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:18 smithi104 ceph-mon[33507]: 6.c scrub starts 2024-09-11T07:51:18.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:18 smithi104 ceph-mon[33507]: 6.c scrub ok 2024-09-11T07:51:18.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:18 smithi104 ceph-mon[33507]: 2.b scrub starts 2024-09-11T07:51:18.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:18 smithi104 ceph-mon[33507]: 2.b scrub ok 2024-09-11T07:51:18.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:18 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2998218911' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T07:51:18.859 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 3 1 2024-09-11T07:51:18.880 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:51:19.092 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:19 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2998218911' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-11T07:51:19.093 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:19 smithi022 ceph-mon[37330]: osdmap e504: 8 total, 8 up, 8 in 2024-09-11T07:51:19.395 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:19 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2998218911' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-11T07:51:19.395 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:19 smithi022 ceph-mon[29462]: osdmap e504: 8 total, 8 up, 8 in 2024-09-11T07:51:19.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:19 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2998218911' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-11T07:51:19.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:19 smithi104 ceph-mon[33507]: osdmap e504: 8 total, 8 up, 8 in 2024-09-11T07:51:20.355 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:20 smithi022 ceph-mon[29462]: pgmap v1162: 142 pgs: 142 active+clean; 535 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 25 KiB/s wr, 1 op/s; 8.2 KiB/s, 0 objects/s recovering 2024-09-11T07:51:20.356 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:20 smithi022 ceph-mon[29462]: 2.3 scrub starts 2024-09-11T07:51:20.356 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:20 smithi022 ceph-mon[29462]: 2.3 scrub ok 2024-09-11T07:51:20.356 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:20 smithi022 ceph-mon[29462]: osdmap e505: 8 total, 8 up, 8 in 2024-09-11T07:51:20.356 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:20 smithi022 ceph-mon[29462]: 6.6 deep-scrub starts 2024-09-11T07:51:20.356 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:20 smithi022 ceph-mon[29462]: 6.6 deep-scrub ok 2024-09-11T07:51:20.356 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:20 smithi022 ceph-mon[37330]: pgmap v1162: 142 pgs: 142 active+clean; 535 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 25 KiB/s wr, 1 op/s; 8.2 KiB/s, 0 objects/s recovering 2024-09-11T07:51:20.356 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:20 smithi022 ceph-mon[37330]: 2.3 scrub starts 2024-09-11T07:51:20.356 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:20 smithi022 ceph-mon[37330]: 2.3 scrub ok 2024-09-11T07:51:20.356 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:20 smithi022 ceph-mon[37330]: osdmap e505: 8 total, 8 up, 8 in 2024-09-11T07:51:20.356 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:20 smithi022 ceph-mon[37330]: 6.6 deep-scrub starts 2024-09-11T07:51:20.357 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:20 smithi022 ceph-mon[37330]: 6.6 deep-scrub ok 2024-09-11T07:51:20.468 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:20 smithi104 ceph-mon[33507]: pgmap v1162: 142 pgs: 142 active+clean; 535 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 25 KiB/s wr, 1 op/s; 8.2 KiB/s, 0 objects/s recovering 2024-09-11T07:51:20.469 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:20 smithi104 ceph-mon[33507]: 2.3 scrub starts 2024-09-11T07:51:20.469 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:20 smithi104 ceph-mon[33507]: 2.3 scrub ok 2024-09-11T07:51:20.469 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:20 smithi104 ceph-mon[33507]: osdmap e505: 8 total, 8 up, 8 in 2024-09-11T07:51:20.469 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:20 smithi104 ceph-mon[33507]: 6.6 deep-scrub starts 2024-09-11T07:51:20.469 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:20 smithi104 ceph-mon[33507]: 6.6 deep-scrub ok 2024-09-11T07:51:20.575 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-11T07:51:20.575 INFO:teuthology.orchestra.run.smithi022.stdout: "size": 20, 2024-09-11T07:51:20.575 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 600, 2024-09-11T07:51:20.575 INFO:teuthology.orchestra.run.smithi022.stdout: "ops": [ 2024-09-11T07:51:20.575 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.575 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24463.0:1444 3.0 3:353ec8f9:::gateway.conf:head [getxattr epoch (1) in=5b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e492)", 2024-09-11T07:51:20.575 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:50:53.519578+0000", 2024-09-11T07:51:20.575 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 27.053061574000001, 2024-09-11T07:51:20.576 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.411809113, 2024-09-11T07:51:20.576 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:51:20.576 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:51:20.576 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:51:20.576 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24463", 2024-09-11T07:51:20.576 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.104:0/620185589", 2024-09-11T07:51:20.576 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 1444 2024-09-11T07:51:20.576 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.576 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:51:20.576 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.576 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:51:20.576 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.519578+0000", 2024-09-11T07:51:20.576 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.577 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.577 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.577 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:51:20.577 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.519578+0000", 2024-09-11T07:51:20.577 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.577 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.577 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.577 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:51:20.577 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.519575+0000", 2024-09-11T07:51:20.577 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999976 2024-09-11T07:51:20.577 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.577 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.577 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:51:20.578 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.519584+0000", 2024-09-11T07:51:20.578 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3040000000000005e-06 2024-09-11T07:51:20.578 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.578 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.578 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:51:20.578 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.519594+0000", 2024-09-11T07:51:20.578 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0614e-05 2024-09-11T07:51:20.578 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.578 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.578 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:51:20.578 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.519602+0000", 2024-09-11T07:51:20.578 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.157e-06 2024-09-11T07:51:20.578 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.579 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.579 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:51:20.579 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.519658+0000", 2024-09-11T07:51:20.579 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.5682e-05 2024-09-11T07:51:20.579 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.579 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.579 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for peered", 2024-09-11T07:51:20.579 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.519664+0000", 2024-09-11T07:51:20.579 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.2949999999999999e-06 2024-09-11T07:51:20.579 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.579 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.579 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:51:20.579 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.923205+0000", 2024-09-11T07:51:20.580 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.40354064499999998 2024-09-11T07:51:20.580 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.580 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.580 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "waiting for missing object", 2024-09-11T07:51:20.580 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.923449+0000", 2024-09-11T07:51:20.580 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00024409799999999999 2024-09-11T07:51:20.580 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.580 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.580 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:51:20.580 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.930929+0000", 2024-09-11T07:51:20.580 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0074802640000000004 2024-09-11T07:51:20.580 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.581 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.581 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:51:20.581 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.931015+0000", 2024-09-11T07:51:20.581 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.5204000000000006e-05 2024-09-11T07:51:20.581 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.581 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.581 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:51:20.581 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.931387+0000", 2024-09-11T07:51:20.581 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00037208100000000002 2024-09-11T07:51:20.581 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.581 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:51:20.581 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.581 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.582 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.582 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "MOSDPGPush(4.1c 493/492 [PushOp(4:3e426232:gc::gc.9:head, version: 64'3, data_included: [0~110], data_size: 110, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(4:3e426232:gc::gc.9:head@64'3, size: 110, copy_subset: [0~110], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:110, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-11T07:51:20.582 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:50:53.910497+0000", 2024-09-11T07:51:20.582 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 26.662142522, 2024-09-11T07:51:20.582 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00070669600000000004, 2024-09-11T07:51:20.582 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:51:20.582 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:51:20.582 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:51:20.582 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.582 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:51:20.582 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.910497+0000", 2024-09-11T07:51:20.582 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.582 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.582 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.583 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:51:20.583 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.910497+0000", 2024-09-11T07:51:20.583 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.583 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.583 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.583 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:51:20.583 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.910496+0000", 2024-09-11T07:51:20.583 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:51:20.583 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.583 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.583 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:51:20.583 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.910508+0000", 2024-09-11T07:51:20.584 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1800000000000001e-05 2024-09-11T07:51:20.584 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.584 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.584 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:51:20.584 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.910520+0000", 2024-09-11T07:51:20.584 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1868000000000001e-05 2024-09-11T07:51:20.584 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.584 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.584 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:51:20.584 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.910528+0000", 2024-09-11T07:51:20.584 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.6889999999999993e-06 2024-09-11T07:51:20.584 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.584 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.584 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:51:20.585 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.910771+0000", 2024-09-11T07:51:20.585 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00024236400000000001 2024-09-11T07:51:20.585 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.585 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.585 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:51:20.585 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.910778+0000", 2024-09-11T07:51:20.585 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.7149999999999998e-06 2024-09-11T07:51:20.585 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.585 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.585 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:51:20.585 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.911203+0000", 2024-09-11T07:51:20.585 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00042493000000000001 2024-09-11T07:51:20.585 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.585 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:51:20.586 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.586 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.586 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.586 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "MOSDPGPush(4.19 493/492 [PushOp(4:9fecf831:::obj_delete_at_hint.0000000057:head, version: 426'38, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(4:9fecf831:::obj_delete_at_hint.0000000057:head@426'38, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:0, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-11T07:51:20.586 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:50:53.913758+0000", 2024-09-11T07:51:20.586 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 26.658880962000001, 2024-09-11T07:51:20.586 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00091908799999999998, 2024-09-11T07:51:20.586 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:51:20.586 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:51:20.586 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:51:20.586 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.586 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:51:20.586 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.913758+0000", 2024-09-11T07:51:20.586 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.587 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.587 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.587 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:51:20.587 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.913758+0000", 2024-09-11T07:51:20.587 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.587 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.587 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.587 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:51:20.587 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.913757+0000", 2024-09-11T07:51:20.587 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:51:20.587 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.587 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.587 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:51:20.587 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.913779+0000", 2024-09-11T07:51:20.588 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1633000000000001e-05 2024-09-11T07:51:20.588 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.588 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.588 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:51:20.588 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.914458+0000", 2024-09-11T07:51:20.588 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00067949999999999998 2024-09-11T07:51:20.588 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.588 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.588 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:51:20.588 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.914469+0000", 2024-09-11T07:51:20.588 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.039e-05 2024-09-11T07:51:20.588 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.588 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.589 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:51:20.589 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.914533+0000", 2024-09-11T07:51:20.589 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.3791e-05 2024-09-11T07:51:20.589 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.589 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.589 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:51:20.589 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.914539+0000", 2024-09-11T07:51:20.589 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.6259999999999997e-06 2024-09-11T07:51:20.589 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.589 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.589 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:51:20.589 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.914677+0000", 2024-09-11T07:51:20.589 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00013810100000000001 2024-09-11T07:51:20.589 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.589 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:51:20.590 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.590 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.590 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.590 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "MOSDPGPush(4.14 493/492 [PushOp(4:2dbd613a:::obj_delete_at_hint.0000000090:head, version: 426'33, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(4:2dbd613a:::obj_delete_at_hint.0000000090:head@426'33, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:0, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-11T07:51:20.590 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:50:53.920255+0000", 2024-09-11T07:51:20.590 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 26.652384602000001, 2024-09-11T07:51:20.590 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001486383, 2024-09-11T07:51:20.590 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:51:20.590 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:51:20.590 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:51:20.590 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.590 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:51:20.590 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.920255+0000", 2024-09-11T07:51:20.591 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.591 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.591 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.591 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:51:20.591 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.920255+0000", 2024-09-11T07:51:20.591 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.591 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.591 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.591 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:51:20.591 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.920254+0000", 2024-09-11T07:51:20.591 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:51:20.591 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.591 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.591 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:51:20.592 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.920265+0000", 2024-09-11T07:51:20.592 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1355999999999999e-05 2024-09-11T07:51:20.592 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.592 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.592 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:51:20.592 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.920278+0000", 2024-09-11T07:51:20.592 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3101999999999999e-05 2024-09-11T07:51:20.592 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.592 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.592 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:51:20.592 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.920287+0000", 2024-09-11T07:51:20.592 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.3680000000000002e-06 2024-09-11T07:51:20.592 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.592 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.593 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:51:20.593 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.920511+0000", 2024-09-11T07:51:20.593 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00022414200000000001 2024-09-11T07:51:20.593 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.593 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.593 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:51:20.593 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.920518+0000", 2024-09-11T07:51:20.593 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9199999999999998e-06 2024-09-11T07:51:20.593 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.593 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.593 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:51:20.593 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.921741+0000", 2024-09-11T07:51:20.593 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0012233159999999999 2024-09-11T07:51:20.593 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.594 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:51:20.594 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.594 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.594 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.594 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "MOSDPGPush(4.1b 493/492 [PushOp(4:db9d130d:gc::gc.24:head, version: 82'6, data_included: [0~110], data_size: 110, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(4:db9d130d:gc::gc.24:head@82'6, size: 110, copy_subset: [0~110], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:110, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-11T07:51:20.594 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:50:53.923446+0000", 2024-09-11T07:51:20.594 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 26.649192958, 2024-09-11T07:51:20.594 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0042096199999999999, 2024-09-11T07:51:20.594 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:51:20.594 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:51:20.594 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:51:20.594 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.594 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:51:20.594 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.923446+0000", 2024-09-11T07:51:20.595 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.595 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.595 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.595 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:51:20.595 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.923446+0000", 2024-09-11T07:51:20.595 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.595 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.595 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.595 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:51:20.595 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.923445+0000", 2024-09-11T07:51:20.595 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:51:20.595 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.595 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.595 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:51:20.596 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.923458+0000", 2024-09-11T07:51:20.596 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2993000000000001e-05 2024-09-11T07:51:20.596 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.596 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.596 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:51:20.596 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.923471+0000", 2024-09-11T07:51:20.596 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2508e-05 2024-09-11T07:51:20.596 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.596 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.596 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:51:20.596 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.923481+0000", 2024-09-11T07:51:20.596 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0047000000000001e-05 2024-09-11T07:51:20.596 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.596 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.597 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:51:20.597 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.927369+0000", 2024-09-11T07:51:20.597 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0038878889999999998 2024-09-11T07:51:20.597 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.597 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.597 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:51:20.597 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.927376+0000", 2024-09-11T07:51:20.597 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.8329999999999996e-06 2024-09-11T07:51:20.597 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.597 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.597 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:51:20.597 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.927656+0000", 2024-09-11T07:51:20.597 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00028006100000000001 2024-09-11T07:51:20.597 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.598 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:51:20.598 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.598 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.598 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.598 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "MOSDPGPush(4.14 493/492 [PushOp(4:2ece19cb:::obj_delete_at_hint.0000000110:head, version: 426'35, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(4:2ece19cb:::obj_delete_at_hint.0000000110:head@426'35, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:0, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-11T07:51:20.598 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:50:53.928787+0000", 2024-09-11T07:51:20.598 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 26.64385219, 2024-09-11T07:51:20.598 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00097424000000000002, 2024-09-11T07:51:20.598 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:51:20.598 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:51:20.598 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:51:20.598 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.598 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:51:20.598 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.928787+0000", 2024-09-11T07:51:20.599 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.599 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.599 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.599 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:51:20.599 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.928787+0000", 2024-09-11T07:51:20.599 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.599 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.599 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.599 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:51:20.599 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.928786+0000", 2024-09-11T07:51:20.599 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:51:20.599 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.599 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.599 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:51:20.599 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.928798+0000", 2024-09-11T07:51:20.600 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2452999999999999e-05 2024-09-11T07:51:20.600 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.600 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.600 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:51:20.600 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.928812+0000", 2024-09-11T07:51:20.600 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3816e-05 2024-09-11T07:51:20.600 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.600 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.600 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:51:20.600 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.928823+0000", 2024-09-11T07:51:20.600 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0444e-05 2024-09-11T07:51:20.600 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.600 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.601 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:51:20.601 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.929524+0000", 2024-09-11T07:51:20.601 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00070164999999999995 2024-09-11T07:51:20.601 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.601 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.601 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:51:20.601 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.929531+0000", 2024-09-11T07:51:20.601 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.883e-06 2024-09-11T07:51:20.601 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.601 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.601 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:51:20.601 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.929761+0000", 2024-09-11T07:51:20.601 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00023012299999999999 2024-09-11T07:51:20.602 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.602 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:51:20.602 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.602 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.602 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.602 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "MOSDPGPush(1.8 493/492 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 87'80, data_included: [0~262688], data_size: 262688, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@87'80, size: 262688, copy_subset: [0~262688], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:262688, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-11T07:51:20.602 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:50:53.932091+0000", 2024-09-11T07:51:20.602 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 26.640547725000001, 2024-09-11T07:51:20.602 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0011787830000000001, 2024-09-11T07:51:20.602 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:51:20.602 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:51:20.602 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:51:20.602 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.602 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:51:20.602 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.932091+0000", 2024-09-11T07:51:20.603 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.603 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.603 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.603 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:51:20.603 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.932091+0000", 2024-09-11T07:51:20.603 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.603 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.603 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.603 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:51:20.603 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.932090+0000", 2024-09-11T07:51:20.603 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:51:20.603 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.603 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.603 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:51:20.604 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.932389+0000", 2024-09-11T07:51:20.604 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00029832099999999999 2024-09-11T07:51:20.604 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.604 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.604 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:51:20.604 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.932404+0000", 2024-09-11T07:51:20.604 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4808e-05 2024-09-11T07:51:20.604 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.604 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.604 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:51:20.604 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.932417+0000", 2024-09-11T07:51:20.604 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2944e-05 2024-09-11T07:51:20.604 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.604 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.604 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:51:20.605 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.932605+0000", 2024-09-11T07:51:20.605 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00018803400000000001 2024-09-11T07:51:20.605 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.605 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.605 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:51:20.605 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.932612+0000", 2024-09-11T07:51:20.605 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.9379999999999992e-06 2024-09-11T07:51:20.605 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.605 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.605 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:51:20.605 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.933270+0000", 2024-09-11T07:51:20.605 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00065771500000000001 2024-09-11T07:51:20.605 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.605 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:51:20.606 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.606 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.606 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.606 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "MOSDPGPush(4.4 493/492 [PushOp(4:2166d8e1:gc::gc.3:head, version: 64'12, data_included: [0~110], data_size: 110, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(4:2166d8e1:gc::gc.3:head@64'12, size: 110, copy_subset: [0~110], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:110, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-11T07:51:20.606 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:50:53.940803+0000", 2024-09-11T07:51:20.606 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 26.631836585999999, 2024-09-11T07:51:20.606 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00069722600000000001, 2024-09-11T07:51:20.606 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:51:20.606 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:51:20.606 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:51:20.606 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.606 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:51:20.606 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.940803+0000", 2024-09-11T07:51:20.606 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.607 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.607 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.607 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:51:20.607 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.940803+0000", 2024-09-11T07:51:20.607 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.607 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.607 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.607 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:51:20.607 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.940802+0000", 2024-09-11T07:51:20.607 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:51:20.607 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.607 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.607 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:51:20.607 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.940812+0000", 2024-09-11T07:51:20.608 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0703000000000001e-05 2024-09-11T07:51:20.608 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.608 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.608 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:51:20.608 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.940824+0000", 2024-09-11T07:51:20.608 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1965e-05 2024-09-11T07:51:20.608 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.608 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.608 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:51:20.608 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.940831+0000", 2024-09-11T07:51:20.608 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.9600000000000003e-06 2024-09-11T07:51:20.608 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.608 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.608 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:51:20.609 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.940903+0000", 2024-09-11T07:51:20.609 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1947999999999996e-05 2024-09-11T07:51:20.609 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.609 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.609 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:51:20.609 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.940912+0000", 2024-09-11T07:51:20.609 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.4460000000000001e-06 2024-09-11T07:51:20.609 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.609 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.609 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:51:20.609 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:53.941500+0000", 2024-09-11T07:51:20.609 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00058823299999999996 2024-09-11T07:51:20.609 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.609 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:51:20.610 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.610 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.610 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.610 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24353.0:3319 5.d 5:b4812045:::notify.4:head [watch reconnect cookie 94395805755184 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e493)", 2024-09-11T07:51:20.610 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:50:54.805431+0000", 2024-09-11T07:51:20.610 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 25.767208191999998, 2024-09-11T07:51:20.610 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0019939509999999999, 2024-09-11T07:51:20.610 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:51:20.610 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:51:20.610 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:51:20.610 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24353", 2024-09-11T07:51:20.610 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1860969011", 2024-09-11T07:51:20.610 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 3319 2024-09-11T07:51:20.610 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.610 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:51:20.611 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.611 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:51:20.611 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805431+0000", 2024-09-11T07:51:20.611 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.611 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.611 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.611 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:51:20.611 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805431+0000", 2024-09-11T07:51:20.611 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.611 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.611 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.612 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:51:20.612 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805428+0000", 2024-09-11T07:51:20.612 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999971 2024-09-11T07:51:20.612 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.612 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.612 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:51:20.613 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805438+0000", 2024-09-11T07:51:20.613 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0118999999999999e-05 2024-09-11T07:51:20.613 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.613 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.613 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:51:20.613 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805452+0000", 2024-09-11T07:51:20.613 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.3555e-05 2024-09-11T07:51:20.613 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.613 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.613 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:51:20.613 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805461+0000", 2024-09-11T07:51:20.613 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.1430000000000004e-06 2024-09-11T07:51:20.613 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.614 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.614 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:51:20.614 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805520+0000", 2024-09-11T07:51:20.614 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.9633000000000003e-05 2024-09-11T07:51:20.614 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.614 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.614 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:51:20.614 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805614+0000", 2024-09-11T07:51:20.614 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.3708999999999993e-05 2024-09-11T07:51:20.614 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.614 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.614 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:51:20.614 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.807425+0000", 2024-09-11T07:51:20.614 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0018108519999999999 2024-09-11T07:51:20.614 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.615 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:51:20.615 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.615 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.615 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.615 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24353.0:3320 5.14 5:2b04a3e9:::notify.6:head [watch reconnect cookie 94395805760480 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e493)", 2024-09-11T07:51:20.615 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:50:54.805476+0000", 2024-09-11T07:51:20.615 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 25.767163319000002, 2024-09-11T07:51:20.615 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001943794, 2024-09-11T07:51:20.615 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:51:20.615 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:51:20.615 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:51:20.615 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24353", 2024-09-11T07:51:20.615 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1860969011", 2024-09-11T07:51:20.615 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 3320 2024-09-11T07:51:20.616 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.616 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:51:20.616 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.616 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:51:20.616 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805476+0000", 2024-09-11T07:51:20.616 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.616 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.616 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.616 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:51:20.616 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805476+0000", 2024-09-11T07:51:20.616 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.616 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.616 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.616 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:51:20.617 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805475+0000", 2024-09-11T07:51:20.617 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:51:20.617 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.617 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.617 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:51:20.617 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805479+0000", 2024-09-11T07:51:20.617 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.4009999999999999e-06 2024-09-11T07:51:20.617 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.617 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.617 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:51:20.617 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805488+0000", 2024-09-11T07:51:20.617 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.6600000000000001e-06 2024-09-11T07:51:20.617 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.617 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.618 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:51:20.618 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805493+0000", 2024-09-11T07:51:20.618 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.8300000000000003e-06 2024-09-11T07:51:20.618 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.618 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.618 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:51:20.618 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805543+0000", 2024-09-11T07:51:20.618 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.0170000000000002e-05 2024-09-11T07:51:20.618 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.618 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.618 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:51:20.618 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805619+0000", 2024-09-11T07:51:20.618 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.6303000000000005e-05 2024-09-11T07:51:20.618 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.618 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.619 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:51:20.619 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.807420+0000", 2024-09-11T07:51:20.619 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0018004519999999999 2024-09-11T07:51:20.619 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.619 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:51:20.619 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.619 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.619 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.619 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24353.0:3324 5.d 5:b4812045:::notify.4:head [watch reconnect cookie 94395805755184 gen 7] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e493)", 2024-09-11T07:51:20.619 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:50:54.805507+0000", 2024-09-11T07:51:20.619 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 25.767131694, 2024-09-11T07:51:20.619 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0020210319999999999, 2024-09-11T07:51:20.619 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:51:20.619 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:51:20.619 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:51:20.620 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24353", 2024-09-11T07:51:20.620 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1860969011", 2024-09-11T07:51:20.620 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 3324 2024-09-11T07:51:20.620 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.620 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:51:20.620 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.620 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:51:20.620 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805507+0000", 2024-09-11T07:51:20.620 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.620 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.620 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.620 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:51:20.620 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805507+0000", 2024-09-11T07:51:20.620 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.621 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.621 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.621 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:51:20.621 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805507+0000", 2024-09-11T07:51:20.621 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:51:20.621 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.621 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.621 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:51:20.621 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805511+0000", 2024-09-11T07:51:20.621 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.9559999999999999e-06 2024-09-11T07:51:20.621 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.621 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.621 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:51:20.621 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805520+0000", 2024-09-11T07:51:20.622 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.414e-06 2024-09-11T07:51:20.622 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.622 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.622 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:51:20.622 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805525+0000", 2024-09-11T07:51:20.622 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.0690000000000001e-06 2024-09-11T07:51:20.622 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.622 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.622 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:51:20.622 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805928+0000", 2024-09-11T07:51:20.622 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00040258999999999998 2024-09-11T07:51:20.622 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.622 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.623 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:51:20.623 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.806016+0000", 2024-09-11T07:51:20.623 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.8877999999999998e-05 2024-09-11T07:51:20.623 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.623 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.623 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:51:20.623 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.807528+0000", 2024-09-11T07:51:20.623 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0015119930000000001 2024-09-11T07:51:20.623 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.623 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:51:20.623 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.623 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.623 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.623 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24353.0:3325 4.4 4:22d26bf9:::data_loggenerations_metadata:head [watch reconnect cookie 94395805765568 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e493)", 2024-09-11T07:51:20.624 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:50:54.805530+0000", 2024-09-11T07:51:20.624 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 25.767108989, 2024-09-11T07:51:20.624 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0022988549999999998, 2024-09-11T07:51:20.624 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:51:20.624 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:51:20.624 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:51:20.624 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24353", 2024-09-11T07:51:20.624 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1860969011", 2024-09-11T07:51:20.624 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 3325 2024-09-11T07:51:20.624 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.624 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:51:20.624 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.624 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:51:20.624 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805530+0000", 2024-09-11T07:51:20.625 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.625 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.625 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.625 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:51:20.625 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805530+0000", 2024-09-11T07:51:20.625 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.625 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.625 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.625 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:51:20.625 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805529+0000", 2024-09-11T07:51:20.625 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:51:20.625 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.625 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.626 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:51:20.626 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805534+0000", 2024-09-11T07:51:20.626 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.0810000000000004e-06 2024-09-11T07:51:20.626 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.626 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.626 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:51:20.626 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805539+0000", 2024-09-11T07:51:20.626 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8560000000000003e-06 2024-09-11T07:51:20.626 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.626 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.626 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:51:20.626 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805543+0000", 2024-09-11T07:51:20.626 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.8340000000000001e-06 2024-09-11T07:51:20.627 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.627 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.627 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:51:20.627 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805618+0000", 2024-09-11T07:51:20.627 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.4548000000000005e-05 2024-09-11T07:51:20.627 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.627 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.627 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:51:20.627 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805832+0000", 2024-09-11T07:51:20.627 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00021440000000000001 2024-09-11T07:51:20.627 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.627 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.628 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:51:20.628 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.807829+0000", 2024-09-11T07:51:20.628 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0019968199999999998 2024-09-11T07:51:20.628 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.628 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:51:20.628 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.628 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.628 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.628 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24353.0:3327 5.14 5:2b04a3e9:::notify.6:head [watch reconnect cookie 94395805760480 gen 7] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e493)", 2024-09-11T07:51:20.628 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:50:54.805548+0000", 2024-09-11T07:51:20.628 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 25.767091306000001, 2024-09-11T07:51:20.628 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0020511800000000001, 2024-09-11T07:51:20.628 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:51:20.629 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:51:20.629 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:51:20.629 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24353", 2024-09-11T07:51:20.629 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1860969011", 2024-09-11T07:51:20.629 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 3327 2024-09-11T07:51:20.629 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.629 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:51:20.629 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.629 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:51:20.629 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805548+0000", 2024-09-11T07:51:20.629 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.629 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.629 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.629 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:51:20.630 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805548+0000", 2024-09-11T07:51:20.630 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.630 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.630 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.630 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:51:20.630 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805547+0000", 2024-09-11T07:51:20.630 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:51:20.630 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.630 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.630 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:51:20.630 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805550+0000", 2024-09-11T07:51:20.630 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.9100000000000001e-06 2024-09-11T07:51:20.630 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.630 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.630 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:51:20.631 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805555+0000", 2024-09-11T07:51:20.631 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.0590000000000002e-06 2024-09-11T07:51:20.631 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.631 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.631 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:51:20.631 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805559+0000", 2024-09-11T07:51:20.631 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.4829999999999999e-06 2024-09-11T07:51:20.631 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.631 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.631 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:51:20.631 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805966+0000", 2024-09-11T07:51:20.631 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00040749500000000001 2024-09-11T07:51:20.631 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.631 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.632 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:51:20.632 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.806041+0000", 2024-09-11T07:51:20.632 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.4622000000000005e-05 2024-09-11T07:51:20.632 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.632 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.632 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:51:20.632 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.807599+0000", 2024-09-11T07:51:20.632 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001558359 2024-09-11T07:51:20.632 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.632 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:51:20.632 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.632 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.632 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.632 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "pg_update_log_missing(5.1f epoch 493/492 rep_tid 114448 entries 493'15 (0'0) error 5:f95f44c2:::notify.0:head by client.24353.0:3322 0.000000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2024-09-11T07:51:20.633 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:50:54.805752+0000", 2024-09-11T07:51:20.633 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 25.766886738, 2024-09-11T07:51:20.633 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001299206, 2024-09-11T07:51:20.633 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:51:20.633 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:51:20.633 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:51:20.633 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.633 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:51:20.633 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805752+0000", 2024-09-11T07:51:20.633 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.633 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.633 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.633 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:51:20.633 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805752+0000", 2024-09-11T07:51:20.634 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.634 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.634 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.634 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:51:20.634 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805751+0000", 2024-09-11T07:51:20.634 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999981 2024-09-11T07:51:20.634 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.634 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.634 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:51:20.634 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805762+0000", 2024-09-11T07:51:20.634 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1567e-05 2024-09-11T07:51:20.634 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.634 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.634 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:51:20.634 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805777+0000", 2024-09-11T07:51:20.634 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.4473e-05 2024-09-11T07:51:20.635 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.635 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.635 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:51:20.635 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805785+0000", 2024-09-11T07:51:20.635 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.6670000000000002e-06 2024-09-11T07:51:20.635 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.635 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.635 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:51:20.635 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805843+0000", 2024-09-11T07:51:20.635 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.8116999999999999e-05 2024-09-11T07:51:20.635 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.635 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.635 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:51:20.635 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.807052+0000", 2024-09-11T07:51:20.635 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0012081559999999999 2024-09-11T07:51:20.636 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.636 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:51:20.636 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.636 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.636 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.636 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "pg_update_log_missing(5.9 epoch 493/492 rep_tid 114449 entries 493'14 (0'0) error 5:93e5b521:::notify.7:head by client.24353.0:3321 0.000000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2024-09-11T07:51:20.636 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:50:54.805804+0000", 2024-09-11T07:51:20.636 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 25.766834748000001, 2024-09-11T07:51:20.636 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0012233770000000001, 2024-09-11T07:51:20.636 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:51:20.636 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:51:20.636 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:51:20.636 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.636 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:51:20.636 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805804+0000", 2024-09-11T07:51:20.637 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.637 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.637 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.637 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:51:20.637 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805804+0000", 2024-09-11T07:51:20.637 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.637 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.637 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.637 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:51:20.637 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805804+0000", 2024-09-11T07:51:20.637 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999995 2024-09-11T07:51:20.637 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.637 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.637 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:51:20.638 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805810+0000", 2024-09-11T07:51:20.638 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7549999999999997e-06 2024-09-11T07:51:20.638 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.638 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.638 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:51:20.638 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805820+0000", 2024-09-11T07:51:20.638 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 9.6120000000000005e-06 2024-09-11T07:51:20.638 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.638 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.638 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:51:20.638 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805828+0000", 2024-09-11T07:51:20.639 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.0180000000000001e-06 2024-09-11T07:51:20.639 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.639 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.639 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:51:20.639 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.805885+0000", 2024-09-11T07:51:20.639 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6530000000000003e-05 2024-09-11T07:51:20.639 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.639 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.639 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:51:20.639 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.807028+0000", 2024-09-11T07:51:20.639 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.0011431569999999999 2024-09-11T07:51:20.639 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.639 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:51:20.639 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.639 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.640 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.640 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "pg_update_log_missing(5.1f epoch 493/492 rep_tid 114450 entries 493'16 (0'0) error 5:f95f44c2:::notify.0:head by client.24353.0:3326 0.000000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2024-09-11T07:51:20.640 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:50:54.806192+0000", 2024-09-11T07:51:20.640 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 25.766447068000002, 2024-09-11T07:51:20.640 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00086481600000000002, 2024-09-11T07:51:20.640 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:51:20.640 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:51:20.640 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:51:20.640 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.640 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:51:20.640 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.806192+0000", 2024-09-11T07:51:20.640 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.640 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.641 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.641 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:51:20.641 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.806192+0000", 2024-09-11T07:51:20.641 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.641 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.641 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.641 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:51:20.641 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.806191+0000", 2024-09-11T07:51:20.641 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:51:20.641 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.641 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.642 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:51:20.642 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.806201+0000", 2024-09-11T07:51:20.642 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0642000000000001e-05 2024-09-11T07:51:20.642 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.642 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.642 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:51:20.642 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.806214+0000", 2024-09-11T07:51:20.642 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2824e-05 2024-09-11T07:51:20.642 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.642 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.642 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:51:20.642 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.806227+0000", 2024-09-11T07:51:20.642 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2743e-05 2024-09-11T07:51:20.642 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.642 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.643 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:51:20.643 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.806314+0000", 2024-09-11T07:51:20.643 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.6830999999999996e-05 2024-09-11T07:51:20.643 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.643 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.643 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:51:20.643 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.807057+0000", 2024-09-11T07:51:20.643 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00074316699999999998 2024-09-11T07:51:20.643 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.643 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:51:20.643 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.643 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.643 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.643 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "pg_update_log_missing(5.9 epoch 493/492 rep_tid 114451 entries 493'15 (0'0) error 5:93e5b521:::notify.7:head by client.24353.0:3328 0.000000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2024-09-11T07:51:20.644 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:50:54.806330+0000", 2024-09-11T07:51:20.644 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 25.766309064000001, 2024-09-11T07:51:20.644 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00070685699999999997, 2024-09-11T07:51:20.644 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:51:20.644 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "reached pg", 2024-09-11T07:51:20.644 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:51:20.644 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.644 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:51:20.644 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.806330+0000", 2024-09-11T07:51:20.644 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.644 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.644 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.644 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:51:20.644 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.806330+0000", 2024-09-11T07:51:20.645 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.645 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.645 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.645 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:51:20.645 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.806329+0000", 2024-09-11T07:51:20.645 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.999999 2024-09-11T07:51:20.645 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.645 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.645 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:51:20.645 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.806338+0000", 2024-09-11T07:51:20.645 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.6000000000000007e-06 2024-09-11T07:51:20.645 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.645 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.645 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:51:20.646 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.806349+0000", 2024-09-11T07:51:20.646 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1751e-05 2024-09-11T07:51:20.646 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.646 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.646 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:51:20.646 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.806357+0000", 2024-09-11T07:51:20.646 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 7.1620000000000002e-06 2024-09-11T07:51:20.646 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.646 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.646 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:51:20.646 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.806474+0000", 2024-09-11T07:51:20.646 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00011736200000000001 2024-09-11T07:51:20.646 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.646 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.646 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:51:20.646 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:54.807037+0000", 2024-09-11T07:51:20.647 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00056293300000000005 2024-09-11T07:51:20.647 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.647 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:51:20.647 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.647 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.647 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.647 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24353.0:3338 5.d 5:b4812045:::notify.4:head [watch reconnect cookie 94395805755184 gen 8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e496)", 2024-09-11T07:51:20.647 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:50:59.806386+0000", 2024-09-11T07:51:20.647 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 20.766252884, 2024-09-11T07:51:20.647 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.001080398, 2024-09-11T07:51:20.647 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:51:20.647 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:51:20.647 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:51:20.647 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24353", 2024-09-11T07:51:20.648 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1860969011", 2024-09-11T07:51:20.648 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 3338 2024-09-11T07:51:20.648 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.648 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:51:20.648 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.648 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:51:20.648 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:59.806386+0000", 2024-09-11T07:51:20.648 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.648 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.648 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.648 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:51:20.648 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:59.806386+0000", 2024-09-11T07:51:20.648 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.648 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.648 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.648 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:51:20.649 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:59.806385+0000", 2024-09-11T07:51:20.649 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:51:20.649 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.649 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.649 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:51:20.649 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:59.806391+0000", 2024-09-11T07:51:20.649 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.5659999999999997e-06 2024-09-11T07:51:20.649 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.649 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.649 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:51:20.649 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:59.806401+0000", 2024-09-11T07:51:20.649 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.0825999999999999e-05 2024-09-11T07:51:20.649 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.650 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.650 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:51:20.650 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:59.806407+0000", 2024-09-11T07:51:20.650 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.6319999999999996e-06 2024-09-11T07:51:20.650 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.650 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.650 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:51:20.650 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:59.806450+0000", 2024-09-11T07:51:20.650 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.2588e-05 2024-09-11T07:51:20.650 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.650 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.650 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:51:20.650 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:59.806536+0000", 2024-09-11T07:51:20.650 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 8.6199999999999995e-05 2024-09-11T07:51:20.651 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.651 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.651 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:51:20.651 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:50:59.807467+0000", 2024-09-11T07:51:20.651 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00093091300000000001 2024-09-11T07:51:20.651 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.651 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:51:20.651 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.651 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.651 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.651 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_repop(osd.4.0:114455 4.d e496/492 4:b3052b9f:reshard::reshard.0000000013:head v 496'49, mlcod=0'0)", 2024-09-11T07:51:20.651 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:51:00.996282+0000", 2024-09-11T07:51:20.651 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 19.576357539, 2024-09-11T07:51:20.651 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00074850000000000003, 2024-09-11T07:51:20.651 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:51:20.652 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T07:51:20.652 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:51:20.652 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.652 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:51:20.652 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:51:00.996282+0000", 2024-09-11T07:51:20.652 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.652 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.652 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.652 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:51:20.652 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:51:00.996282+0000", 2024-09-11T07:51:20.652 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.652 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.652 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.652 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:51:20.653 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:51:00.996280+0000", 2024-09-11T07:51:20.653 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:51:20.653 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.653 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.653 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:51:20.653 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:51:00.996300+0000", 2024-09-11T07:51:20.653 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.0024e-05 2024-09-11T07:51:20.653 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.653 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.653 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:51:20.653 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:51:00.996312+0000", 2024-09-11T07:51:20.653 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.1941e-05 2024-09-11T07:51:20.653 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.653 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.653 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:51:20.654 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:51:00.996318+0000", 2024-09-11T07:51:20.654 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.5520000000000004e-06 2024-09-11T07:51:20.654 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.654 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.654 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:51:20.654 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:51:00.996458+0000", 2024-09-11T07:51:20.654 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.000140741 2024-09-11T07:51:20.654 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.654 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.654 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:51:20.654 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:51:00.996480+0000", 2024-09-11T07:51:20.654 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 2.1613000000000001e-05 2024-09-11T07:51:20.654 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.654 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.655 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "commit_sent", 2024-09-11T07:51:20.655 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:51:00.996993+0000", 2024-09-11T07:51:20.655 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00051300399999999996 2024-09-11T07:51:20.655 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.655 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.655 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:51:20.655 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:51:00.997030+0000", 2024-09-11T07:51:20.655 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 3.7091000000000001e-05 2024-09-11T07:51:20.655 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.655 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:51:20.655 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.655 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.655 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.655 INFO:teuthology.orchestra.run.smithi022.stdout: "description": "osd_op(client.24353.0:3359 4.4 4:22d26bf9:::data_loggenerations_metadata:head [watch reconnect cookie 94395805765568 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e500)", 2024-09-11T07:51:20.656 INFO:teuthology.orchestra.run.smithi022.stdout: "initiated_at": "2024-09-11T07:51:09.808340+0000", 2024-09-11T07:51:20.656 INFO:teuthology.orchestra.run.smithi022.stdout: "age": 10.764299358000001, 2024-09-11T07:51:20.656 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00092500799999999995, 2024-09-11T07:51:20.656 INFO:teuthology.orchestra.run.smithi022.stdout: "type_data": { 2024-09-11T07:51:20.656 INFO:teuthology.orchestra.run.smithi022.stdout: "flag_point": "started", 2024-09-11T07:51:20.656 INFO:teuthology.orchestra.run.smithi022.stdout: "client_info": { 2024-09-11T07:51:20.656 INFO:teuthology.orchestra.run.smithi022.stdout: "client": "client.24353", 2024-09-11T07:51:20.656 INFO:teuthology.orchestra.run.smithi022.stdout: "client_addr": "v1:172.21.15.22:0/1860969011", 2024-09-11T07:51:20.656 INFO:teuthology.orchestra.run.smithi022.stdout: "tid": 3359 2024-09-11T07:51:20.656 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.656 INFO:teuthology.orchestra.run.smithi022.stdout: "events": [ 2024-09-11T07:51:20.656 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.656 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "initiated", 2024-09-11T07:51:20.656 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:51:09.808340+0000", 2024-09-11T07:51:20.657 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.657 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.657 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.657 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "throttled", 2024-09-11T07:51:20.657 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:51:09.808340+0000", 2024-09-11T07:51:20.657 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0 2024-09-11T07:51:20.657 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.657 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.657 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "header_read", 2024-09-11T07:51:20.657 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:51:09.808339+0000", 2024-09-11T07:51:20.657 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4294967295.9999986 2024-09-11T07:51:20.657 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.657 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.658 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "all_read", 2024-09-11T07:51:20.658 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:51:09.808344+0000", 2024-09-11T07:51:20.658 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.2000000000000002e-06 2024-09-11T07:51:20.658 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.658 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.658 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "dispatched", 2024-09-11T07:51:20.658 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:51:09.808356+0000", 2024-09-11T07:51:20.658 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 1.2651e-05 2024-09-11T07:51:20.658 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.658 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.658 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "queued_for_pg", 2024-09-11T07:51:20.658 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:51:09.808362+0000", 2024-09-11T07:51:20.658 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 5.852e-06 2024-09-11T07:51:20.658 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.658 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.659 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "reached_pg", 2024-09-11T07:51:20.659 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:51:09.808410+0000", 2024-09-11T07:51:20.659 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 4.7595000000000001e-05 2024-09-11T07:51:20.659 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.659 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.659 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "started", 2024-09-11T07:51:20.659 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:51:09.808478+0000", 2024-09-11T07:51:20.659 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 6.7662000000000004e-05 2024-09-11T07:51:20.659 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-11T07:51:20.659 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-11T07:51:20.659 INFO:teuthology.orchestra.run.smithi022.stdout: "event": "done", 2024-09-11T07:51:20.659 INFO:teuthology.orchestra.run.smithi022.stdout: "time": "2024-09-11T07:51:09.809265+0000", 2024-09-11T07:51:20.659 INFO:teuthology.orchestra.run.smithi022.stdout: "duration": 0.00078728000000000001 2024-09-11T07:51:20.659 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.660 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:51:20.660 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.660 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-11T07:51:20.660 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-11T07:51:20.660 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-11T07:51:20.660 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:51:20 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:51:20] "GET /metrics HTTP/1.1" 200 222464 "" "Prometheus/2.43.0" 2024-09-11T07:51:20.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:51:20 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:51:20.936 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:51:21.316 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:21 smithi022 ceph-mon[37330]: 6.14 scrub starts 2024-09-11T07:51:21.316 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:21 smithi022 ceph-mon[37330]: 6.14 scrub ok 2024-09-11T07:51:21.316 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:21 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:51:21.316 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:21 smithi022 ceph-mon[37330]: 2.13 scrub starts 2024-09-11T07:51:21.316 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:21 smithi022 ceph-mon[37330]: 2.13 scrub ok 2024-09-11T07:51:21.316 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:21 smithi022 ceph-mon[29462]: 6.14 scrub starts 2024-09-11T07:51:21.316 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:21 smithi022 ceph-mon[29462]: 6.14 scrub ok 2024-09-11T07:51:21.317 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:21 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:51:21.317 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:21 smithi022 ceph-mon[29462]: 2.13 scrub starts 2024-09-11T07:51:21.317 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:21 smithi022 ceph-mon[29462]: 2.13 scrub ok 2024-09-11T07:51:21.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:21 smithi104 ceph-mon[33507]: 6.14 scrub starts 2024-09-11T07:51:21.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:21 smithi104 ceph-mon[33507]: 6.14 scrub ok 2024-09-11T07:51:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:21 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:51:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:21 smithi104 ceph-mon[33507]: 2.13 scrub starts 2024-09-11T07:51:21.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:21 smithi104 ceph-mon[33507]: 2.13 scrub ok 2024-09-11T07:51:22.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:22 smithi022 ceph-mon[29462]: 5.9 scrub starts 2024-09-11T07:51:22.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:22 smithi022 ceph-mon[29462]: 5.9 scrub ok 2024-09-11T07:51:22.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:22 smithi022 ceph-mon[29462]: pgmap v1164: 142 pgs: 142 active+clean; 535 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 21 KiB/s wr, 1 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-09-11T07:51:22.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:22 smithi022 ceph-mon[29462]: 2.6 deep-scrub starts 2024-09-11T07:51:22.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:22 smithi022 ceph-mon[29462]: 2.6 deep-scrub ok 2024-09-11T07:51:22.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:22 smithi022 ceph-mon[37330]: 5.9 scrub starts 2024-09-11T07:51:22.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:22 smithi022 ceph-mon[37330]: 5.9 scrub ok 2024-09-11T07:51:22.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:22 smithi022 ceph-mon[37330]: pgmap v1164: 142 pgs: 142 active+clean; 535 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 21 KiB/s wr, 1 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-09-11T07:51:22.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:22 smithi022 ceph-mon[37330]: 2.6 deep-scrub starts 2024-09-11T07:51:22.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:22 smithi022 ceph-mon[37330]: 2.6 deep-scrub ok 2024-09-11T07:51:22.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:22 smithi104 ceph-mon[33507]: 5.9 scrub starts 2024-09-11T07:51:22.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:22 smithi104 ceph-mon[33507]: 5.9 scrub ok 2024-09-11T07:51:22.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:22 smithi104 ceph-mon[33507]: pgmap v1164: 142 pgs: 142 active+clean; 535 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 21 KiB/s wr, 1 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-09-11T07:51:22.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:22 smithi104 ceph-mon[33507]: 2.6 deep-scrub starts 2024-09-11T07:51:22.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:22 smithi104 ceph-mon[33507]: 2.6 deep-scrub ok 2024-09-11T07:51:23.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:51:23.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:51:23.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:51:24.105 INFO:teuthology.orchestra.run.smithi022.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-11T07:51:24.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:24 smithi022 ceph-mon[29462]: 2.1a scrub starts 2024-09-11T07:51:24.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:24 smithi022 ceph-mon[29462]: 2.1a scrub ok 2024-09-11T07:51:24.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:24 smithi022 ceph-mon[29462]: pgmap v1165: 142 pgs: 142 active+clean; 535 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 68 B/s, 3 objects/s recovering 2024-09-11T07:51:24.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:24 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2357730790' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T07:51:24.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:24 smithi022 ceph-mon[37330]: 2.1a scrub starts 2024-09-11T07:51:24.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:24 smithi022 ceph-mon[37330]: 2.1a scrub ok 2024-09-11T07:51:24.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:24 smithi022 ceph-mon[37330]: pgmap v1165: 142 pgs: 142 active+clean; 535 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 68 B/s, 3 objects/s recovering 2024-09-11T07:51:24.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:24 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2357730790' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T07:51:24.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:51:24 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:51:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:51:24.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:24 smithi104 ceph-mon[33507]: 2.1a scrub starts 2024-09-11T07:51:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:24 smithi104 ceph-mon[33507]: 2.1a scrub ok 2024-09-11T07:51:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:24 smithi104 ceph-mon[33507]: pgmap v1165: 142 pgs: 142 active+clean; 535 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 68 B/s, 3 objects/s recovering 2024-09-11T07:51:24.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:24 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2357730790' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-11T07:51:24.788 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight 4 1 2024-09-11T07:51:25.053 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:51:25.400 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:25 smithi022 ceph-mon[37330]: 2.c scrub starts 2024-09-11T07:51:25.400 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:25 smithi022 ceph-mon[37330]: 2.c scrub ok 2024-09-11T07:51:25.400 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:25 smithi022 ceph-mon[37330]: 2.8 scrub starts 2024-09-11T07:51:25.400 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:25 smithi022 ceph-mon[37330]: 2.8 scrub ok 2024-09-11T07:51:25.401 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:25 smithi022 ceph-mon[37330]: 6.d scrub starts 2024-09-11T07:51:25.401 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:25 smithi022 ceph-mon[37330]: 6.d scrub ok 2024-09-11T07:51:25.401 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:25 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2357730790' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-11T07:51:25.401 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:25 smithi022 ceph-mon[37330]: osdmap e506: 8 total, 8 up, 8 in 2024-09-11T07:51:25.401 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:25 smithi022 ceph-mon[37330]: 4.1d scrub starts 2024-09-11T07:51:25.401 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:25 smithi022 ceph-mon[37330]: 4.1d scrub ok 2024-09-11T07:51:25.401 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:25 smithi022 ceph-mon[37330]: 4.3 scrub starts 2024-09-11T07:51:25.401 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:25 smithi022 ceph-mon[37330]: 4.3 scrub ok 2024-09-11T07:51:25.401 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:25 smithi022 ceph-mon[29462]: 2.c scrub starts 2024-09-11T07:51:25.401 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:25 smithi022 ceph-mon[29462]: 2.c scrub ok 2024-09-11T07:51:25.402 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:25 smithi022 ceph-mon[29462]: 2.8 scrub starts 2024-09-11T07:51:25.402 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:25 smithi022 ceph-mon[29462]: 2.8 scrub ok 2024-09-11T07:51:25.402 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:25 smithi022 ceph-mon[29462]: 6.d scrub starts 2024-09-11T07:51:25.402 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:25 smithi022 ceph-mon[29462]: 6.d scrub ok 2024-09-11T07:51:25.402 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:25 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2357730790' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-11T07:51:25.402 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:25 smithi022 ceph-mon[29462]: osdmap e506: 8 total, 8 up, 8 in 2024-09-11T07:51:25.402 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:25 smithi022 ceph-mon[29462]: 4.1d scrub starts 2024-09-11T07:51:25.402 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:25 smithi022 ceph-mon[29462]: 4.1d scrub ok 2024-09-11T07:51:25.402 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:25 smithi022 ceph-mon[29462]: 4.3 scrub starts 2024-09-11T07:51:25.402 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:25 smithi022 ceph-mon[29462]: 4.3 scrub ok 2024-09-11T07:51:25.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:25 smithi104 ceph-mon[33507]: 2.c scrub starts 2024-09-11T07:51:25.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:25 smithi104 ceph-mon[33507]: 2.c scrub ok 2024-09-11T07:51:25.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:25 smithi104 ceph-mon[33507]: 2.8 scrub starts 2024-09-11T07:51:25.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:25 smithi104 ceph-mon[33507]: 2.8 scrub ok 2024-09-11T07:51:25.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:25 smithi104 ceph-mon[33507]: 6.d scrub starts 2024-09-11T07:51:25.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:25 smithi104 ceph-mon[33507]: 6.d scrub ok 2024-09-11T07:51:25.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:25 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2357730790' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-11T07:51:25.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:25 smithi104 ceph-mon[33507]: osdmap e506: 8 total, 8 up, 8 in 2024-09-11T07:51:25.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:25 smithi104 ceph-mon[33507]: 4.1d scrub starts 2024-09-11T07:51:25.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:25 smithi104 ceph-mon[33507]: 4.1d scrub ok 2024-09-11T07:51:25.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:25 smithi104 ceph-mon[33507]: 4.3 scrub starts 2024-09-11T07:51:25.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:25 smithi104 ceph-mon[33507]: 4.3 scrub ok 2024-09-11T07:51:26.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:26 smithi022 ceph-mon[29462]: pgmap v1167: 142 pgs: 142 active+clean; 535 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 63 B/s, 2 objects/s recovering 2024-09-11T07:51:26.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:26 smithi022 ceph-mon[37330]: pgmap v1167: 142 pgs: 142 active+clean; 535 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 63 B/s, 2 objects/s recovering 2024-09-11T07:51:26.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:26 smithi104 ceph-mon[33507]: pgmap v1167: 142 pgs: 142 active+clean; 535 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 63 B/s, 2 objects/s recovering 2024-09-11T07:51:27.117 INFO:teuthology.orchestra.run.smithi022.stderr:reweighted osd.4 to 1 (10000) 2024-09-11T07:51:27.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:27 smithi022 ceph-mon[29462]: 2.1d scrub starts 2024-09-11T07:51:27.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:27 smithi022 ceph-mon[29462]: 2.1d scrub ok 2024-09-11T07:51:27.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:27 smithi022 ceph-mon[29462]: 4.1c scrub starts 2024-09-11T07:51:27.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:27 smithi022 ceph-mon[29462]: 4.1c scrub ok 2024-09-11T07:51:27.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:27 smithi022 ceph-mon[29462]: 4.17 scrub starts 2024-09-11T07:51:27.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:27 smithi022 ceph-mon[29462]: 4.17 scrub ok 2024-09-11T07:51:27.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:27 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2492590310' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T07:51:27.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:27 smithi022 ceph-mon[29462]: from='client.26270 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T07:51:27.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:27 smithi022 ceph-mon[37330]: 2.1d scrub starts 2024-09-11T07:51:27.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:27 smithi022 ceph-mon[37330]: 2.1d scrub ok 2024-09-11T07:51:27.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:27 smithi022 ceph-mon[37330]: 4.1c scrub starts 2024-09-11T07:51:27.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:27 smithi022 ceph-mon[37330]: 4.1c scrub ok 2024-09-11T07:51:27.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:27 smithi022 ceph-mon[37330]: 4.17 scrub starts 2024-09-11T07:51:27.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:27 smithi022 ceph-mon[37330]: 4.17 scrub ok 2024-09-11T07:51:27.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:27 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2492590310' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T07:51:27.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:27 smithi022 ceph-mon[37330]: from='client.26270 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T07:51:27.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:27 smithi104 ceph-mon[33507]: 2.1d scrub starts 2024-09-11T07:51:27.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:27 smithi104 ceph-mon[33507]: 2.1d scrub ok 2024-09-11T07:51:27.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:27 smithi104 ceph-mon[33507]: 4.1c scrub starts 2024-09-11T07:51:27.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:27 smithi104 ceph-mon[33507]: 4.1c scrub ok 2024-09-11T07:51:27.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:27 smithi104 ceph-mon[33507]: 4.17 scrub starts 2024-09-11T07:51:27.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:27 smithi104 ceph-mon[33507]: 4.17 scrub ok 2024-09-11T07:51:27.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:27 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2492590310' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T07:51:27.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:27 smithi104 ceph-mon[33507]: from='client.26270 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T07:51:27.892 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 4 1 2024-09-11T07:51:28.155 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:51:28.397 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:28 smithi022 ceph-mon[37330]: 4.1e scrub starts 2024-09-11T07:51:28.398 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:28 smithi022 ceph-mon[37330]: 4.1e scrub ok 2024-09-11T07:51:28.398 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:28 smithi022 ceph-mon[37330]: pgmap v1168: 142 pgs: 142 active+clean; 535 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 54 B/s, 2 objects/s recovering 2024-09-11T07:51:28.398 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:28 smithi022 ceph-mon[37330]: from='client.26270 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-11T07:51:28.398 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:28 smithi022 ceph-mon[37330]: osdmap e507: 8 total, 8 up, 8 in 2024-09-11T07:51:28.398 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:28 smithi022 ceph-mon[37330]: 1.7 scrub starts 2024-09-11T07:51:28.398 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:28 smithi022 ceph-mon[37330]: 1.7 scrub ok 2024-09-11T07:51:28.398 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:28 smithi022 ceph-mon[37330]: 4.b scrub starts 2024-09-11T07:51:28.398 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:28 smithi022 ceph-mon[37330]: 4.b scrub ok 2024-09-11T07:51:28.398 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:28 smithi022 ceph-mon[29462]: 4.1e scrub starts 2024-09-11T07:51:28.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:28 smithi022 ceph-mon[29462]: 4.1e scrub ok 2024-09-11T07:51:28.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:28 smithi022 ceph-mon[29462]: pgmap v1168: 142 pgs: 142 active+clean; 535 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 54 B/s, 2 objects/s recovering 2024-09-11T07:51:28.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:28 smithi022 ceph-mon[29462]: from='client.26270 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-11T07:51:28.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:28 smithi022 ceph-mon[29462]: osdmap e507: 8 total, 8 up, 8 in 2024-09-11T07:51:28.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:28 smithi022 ceph-mon[29462]: 1.7 scrub starts 2024-09-11T07:51:28.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:28 smithi022 ceph-mon[29462]: 1.7 scrub ok 2024-09-11T07:51:28.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:28 smithi022 ceph-mon[29462]: 4.b scrub starts 2024-09-11T07:51:28.399 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:28 smithi022 ceph-mon[29462]: 4.b scrub ok 2024-09-11T07:51:28.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:28 smithi104 ceph-mon[33507]: 4.1e scrub starts 2024-09-11T07:51:28.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:28 smithi104 ceph-mon[33507]: 4.1e scrub ok 2024-09-11T07:51:28.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:28 smithi104 ceph-mon[33507]: pgmap v1168: 142 pgs: 142 active+clean; 535 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 54 B/s, 2 objects/s recovering 2024-09-11T07:51:28.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:28 smithi104 ceph-mon[33507]: from='client.26270 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-11T07:51:28.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:28 smithi104 ceph-mon[33507]: osdmap e507: 8 total, 8 up, 8 in 2024-09-11T07:51:28.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:28 smithi104 ceph-mon[33507]: 1.7 scrub starts 2024-09-11T07:51:28.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:28 smithi104 ceph-mon[33507]: 1.7 scrub ok 2024-09-11T07:51:28.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:28 smithi104 ceph-mon[33507]: 4.b scrub starts 2024-09-11T07:51:28.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:28 smithi104 ceph-mon[33507]: 4.b scrub ok 2024-09-11T07:51:29.378 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:29 smithi022 ceph-mon[29462]: 4.1b scrub starts 2024-09-11T07:51:29.378 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:29 smithi022 ceph-mon[29462]: 4.1b scrub ok 2024-09-11T07:51:29.378 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:29 smithi022 ceph-mon[29462]: 2.1e scrub starts 2024-09-11T07:51:29.378 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:29 smithi022 ceph-mon[29462]: 2.1e scrub ok 2024-09-11T07:51:29.378 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:29 smithi022 ceph-mon[37330]: 4.1b scrub starts 2024-09-11T07:51:29.378 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:29 smithi022 ceph-mon[37330]: 4.1b scrub ok 2024-09-11T07:51:29.378 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:29 smithi022 ceph-mon[37330]: 2.1e scrub starts 2024-09-11T07:51:29.378 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:29 smithi022 ceph-mon[37330]: 2.1e scrub ok 2024-09-11T07:51:29.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:29 smithi104 ceph-mon[33507]: 4.1b scrub starts 2024-09-11T07:51:29.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:29 smithi104 ceph-mon[33507]: 4.1b scrub ok 2024-09-11T07:51:29.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:29 smithi104 ceph-mon[33507]: 2.1e scrub starts 2024-09-11T07:51:29.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:29 smithi104 ceph-mon[33507]: 2.1e scrub ok 2024-09-11T07:51:30.130 INFO:teuthology.orchestra.run.smithi022.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-11T07:51:30.416 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:51:30 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:51:30] "GET /metrics HTTP/1.1" 200 222499 "" "Prometheus/2.43.0" 2024-09-11T07:51:30.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:30 smithi022 ceph-mon[29462]: pgmap v1170: 142 pgs: 142 active+clean; 535 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:51:30.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:30 smithi022 ceph-mon[29462]: 6.9 deep-scrub starts 2024-09-11T07:51:30.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:30 smithi022 ceph-mon[29462]: 6.9 deep-scrub ok 2024-09-11T07:51:30.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:30 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2298706888' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T07:51:30.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:30 smithi022 ceph-mon[29462]: from='client.26276 ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T07:51:30.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:30 smithi022 ceph-mon[37330]: pgmap v1170: 142 pgs: 142 active+clean; 535 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:51:30.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:30 smithi022 ceph-mon[37330]: 6.9 deep-scrub starts 2024-09-11T07:51:30.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:30 smithi022 ceph-mon[37330]: 6.9 deep-scrub ok 2024-09-11T07:51:30.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:30 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2298706888' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T07:51:30.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:30 smithi022 ceph-mon[37330]: from='client.26276 ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T07:51:30.470 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:30 smithi104 ceph-mon[33507]: pgmap v1170: 142 pgs: 142 active+clean; 535 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:51:30.471 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:30 smithi104 ceph-mon[33507]: 6.9 deep-scrub starts 2024-09-11T07:51:30.471 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:30 smithi104 ceph-mon[33507]: 6.9 deep-scrub ok 2024-09-11T07:51:30.471 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:30 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2298706888' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T07:51:30.471 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:30 smithi104 ceph-mon[33507]: from='client.26276 ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-09-11T07:51:30.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:51:30 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:51:30.903 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight 5 1 2024-09-11T07:51:31.161 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:51:31.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:31 smithi022 ceph-mon[29462]: 2.15 scrub starts 2024-09-11T07:51:31.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:31 smithi022 ceph-mon[29462]: 2.15 scrub ok 2024-09-11T07:51:31.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:31 smithi022 ceph-mon[29462]: from='client.26276 ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-11T07:51:31.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:31 smithi022 ceph-mon[29462]: osdmap e508: 8 total, 8 up, 8 in 2024-09-11T07:51:31.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:31 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:51:31.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:31 smithi022 ceph-mon[29462]: 6.1c scrub starts 2024-09-11T07:51:31.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:31 smithi022 ceph-mon[29462]: 6.1c scrub ok 2024-09-11T07:51:31.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:31 smithi022 ceph-mon[37330]: 2.15 scrub starts 2024-09-11T07:51:31.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:31 smithi022 ceph-mon[37330]: 2.15 scrub ok 2024-09-11T07:51:31.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:31 smithi022 ceph-mon[37330]: from='client.26276 ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-11T07:51:31.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:31 smithi022 ceph-mon[37330]: osdmap e508: 8 total, 8 up, 8 in 2024-09-11T07:51:31.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:31 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:51:31.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:31 smithi022 ceph-mon[37330]: 6.1c scrub starts 2024-09-11T07:51:31.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:31 smithi022 ceph-mon[37330]: 6.1c scrub ok 2024-09-11T07:51:31.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:31 smithi104 ceph-mon[33507]: 2.15 scrub starts 2024-09-11T07:51:31.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:31 smithi104 ceph-mon[33507]: 2.15 scrub ok 2024-09-11T07:51:31.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:31 smithi104 ceph-mon[33507]: from='client.26276 ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-11T07:51:31.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:31 smithi104 ceph-mon[33507]: osdmap e508: 8 total, 8 up, 8 in 2024-09-11T07:51:31.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:31 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:51:31.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:31 smithi104 ceph-mon[33507]: 6.1c scrub starts 2024-09-11T07:51:31.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:31 smithi104 ceph-mon[33507]: 6.1c scrub ok 2024-09-11T07:51:32.368 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:32 smithi022 ceph-mon[29462]: 5.1 scrub starts 2024-09-11T07:51:32.368 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:32 smithi022 ceph-mon[29462]: 5.1 scrub ok 2024-09-11T07:51:32.368 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:32 smithi022 ceph-mon[29462]: 6.1 scrub starts 2024-09-11T07:51:32.368 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:32 smithi022 ceph-mon[29462]: 6.1 scrub ok 2024-09-11T07:51:32.369 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:32 smithi022 ceph-mon[29462]: pgmap v1172: 142 pgs: 1 active+clean+scrubbing+deep, 141 active+clean; 483 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-11T07:51:32.369 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:32 smithi022 ceph-mon[29462]: 2.0 deep-scrub starts 2024-09-11T07:51:32.369 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:32 smithi022 ceph-mon[29462]: 2.0 deep-scrub ok 2024-09-11T07:51:32.369 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:32 smithi022 ceph-mon[37330]: 5.1 scrub starts 2024-09-11T07:51:32.369 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:32 smithi022 ceph-mon[37330]: 5.1 scrub ok 2024-09-11T07:51:32.369 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:32 smithi022 ceph-mon[37330]: 6.1 scrub starts 2024-09-11T07:51:32.369 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:32 smithi022 ceph-mon[37330]: 6.1 scrub ok 2024-09-11T07:51:32.369 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:32 smithi022 ceph-mon[37330]: pgmap v1172: 142 pgs: 1 active+clean+scrubbing+deep, 141 active+clean; 483 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-11T07:51:32.369 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:32 smithi022 ceph-mon[37330]: 2.0 deep-scrub starts 2024-09-11T07:51:32.369 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:32 smithi022 ceph-mon[37330]: 2.0 deep-scrub ok 2024-09-11T07:51:32.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:32 smithi104 ceph-mon[33507]: 5.1 scrub starts 2024-09-11T07:51:32.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:32 smithi104 ceph-mon[33507]: 5.1 scrub ok 2024-09-11T07:51:32.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:32 smithi104 ceph-mon[33507]: 6.1 scrub starts 2024-09-11T07:51:32.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:32 smithi104 ceph-mon[33507]: 6.1 scrub ok 2024-09-11T07:51:32.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:32 smithi104 ceph-mon[33507]: pgmap v1172: 142 pgs: 1 active+clean+scrubbing+deep, 141 active+clean; 483 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-11T07:51:32.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:32 smithi104 ceph-mon[33507]: 2.0 deep-scrub starts 2024-09-11T07:51:32.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:32 smithi104 ceph-mon[33507]: 2.0 deep-scrub ok 2024-09-11T07:51:33.140 INFO:teuthology.orchestra.run.smithi022.stderr:reweighted osd.5 to 1 (10000) 2024-09-11T07:51:33.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:33 smithi022 ceph-mon[29462]: 5.f scrub starts 2024-09-11T07:51:33.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:33 smithi022 ceph-mon[29462]: 5.f scrub ok 2024-09-11T07:51:33.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:33 smithi022 ceph-mon[29462]: 1.8 scrub starts 2024-09-11T07:51:33.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:33 smithi022 ceph-mon[29462]: 1.8 scrub ok 2024-09-11T07:51:33.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:33 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2163946863' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T07:51:33.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:33 smithi022 ceph-mon[37330]: 5.f scrub starts 2024-09-11T07:51:33.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:33 smithi022 ceph-mon[37330]: 5.f scrub ok 2024-09-11T07:51:33.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:33 smithi022 ceph-mon[37330]: 1.8 scrub starts 2024-09-11T07:51:33.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:33 smithi022 ceph-mon[37330]: 1.8 scrub ok 2024-09-11T07:51:33.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:33 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2163946863' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T07:51:33.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:33 smithi104 ceph-mon[33507]: 5.f scrub starts 2024-09-11T07:51:33.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:33 smithi104 ceph-mon[33507]: 5.f scrub ok 2024-09-11T07:51:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:33 smithi104 ceph-mon[33507]: 1.8 scrub starts 2024-09-11T07:51:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:33 smithi104 ceph-mon[33507]: 1.8 scrub ok 2024-09-11T07:51:33.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:33 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2163946863' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T07:51:33.930 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 5 1 2024-09-11T07:51:34.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:34 smithi022 ceph-mon[29462]: pgmap v1173: 142 pgs: 1 active+clean+scrubbing+deep, 141 active+clean; 483 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-11T07:51:34.166 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:34 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2163946863' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-11T07:51:34.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:34 smithi022 ceph-mon[29462]: osdmap e509: 8 total, 8 up, 8 in 2024-09-11T07:51:34.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:34 smithi022 ceph-mon[29462]: 2.1b scrub starts 2024-09-11T07:51:34.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:34 smithi022 ceph-mon[29462]: 2.1b scrub ok 2024-09-11T07:51:34.167 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:34 smithi022 ceph-mon[29462]: osdmap e510: 8 total, 8 up, 8 in 2024-09-11T07:51:34.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:34 smithi022 ceph-mon[37330]: pgmap v1173: 142 pgs: 1 active+clean+scrubbing+deep, 141 active+clean; 483 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-11T07:51:34.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:34 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2163946863' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-11T07:51:34.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:34 smithi022 ceph-mon[37330]: osdmap e509: 8 total, 8 up, 8 in 2024-09-11T07:51:34.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:34 smithi022 ceph-mon[37330]: 2.1b scrub starts 2024-09-11T07:51:34.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:34 smithi022 ceph-mon[37330]: 2.1b scrub ok 2024-09-11T07:51:34.167 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:34 smithi022 ceph-mon[37330]: osdmap e510: 8 total, 8 up, 8 in 2024-09-11T07:51:34.197 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:51:34.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:51:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:51:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:51:34.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:34 smithi104 ceph-mon[33507]: pgmap v1173: 142 pgs: 1 active+clean+scrubbing+deep, 141 active+clean; 483 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-11T07:51:34.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:34 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2163946863' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-11T07:51:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:34 smithi104 ceph-mon[33507]: osdmap e509: 8 total, 8 up, 8 in 2024-09-11T07:51:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:34 smithi104 ceph-mon[33507]: 2.1b scrub starts 2024-09-11T07:51:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:34 smithi104 ceph-mon[33507]: 2.1b scrub ok 2024-09-11T07:51:34.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:34 smithi104 ceph-mon[33507]: osdmap e510: 8 total, 8 up, 8 in 2024-09-11T07:51:35.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:35 smithi022 ceph-mon[29462]: 5.1f scrub starts 2024-09-11T07:51:35.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:35 smithi022 ceph-mon[29462]: 5.1f scrub ok 2024-09-11T07:51:35.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:35 smithi022 ceph-mon[29462]: 5.16 scrub starts 2024-09-11T07:51:35.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:35 smithi022 ceph-mon[29462]: 5.16 scrub ok 2024-09-11T07:51:35.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:35 smithi022 ceph-mon[29462]: 1.9 scrub starts 2024-09-11T07:51:35.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:35 smithi022 ceph-mon[29462]: 1.9 scrub ok 2024-09-11T07:51:35.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:35 smithi022 ceph-mon[37330]: 5.1f scrub starts 2024-09-11T07:51:35.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:35 smithi022 ceph-mon[37330]: 5.1f scrub ok 2024-09-11T07:51:35.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:35 smithi022 ceph-mon[37330]: 5.16 scrub starts 2024-09-11T07:51:35.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:35 smithi022 ceph-mon[37330]: 5.16 scrub ok 2024-09-11T07:51:35.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:35 smithi022 ceph-mon[37330]: 1.9 scrub starts 2024-09-11T07:51:35.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:35 smithi022 ceph-mon[37330]: 1.9 scrub ok 2024-09-11T07:51:35.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:35 smithi104 ceph-mon[33507]: 5.1f scrub starts 2024-09-11T07:51:35.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:35 smithi104 ceph-mon[33507]: 5.1f scrub ok 2024-09-11T07:51:35.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:35 smithi104 ceph-mon[33507]: 5.16 scrub starts 2024-09-11T07:51:35.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:35 smithi104 ceph-mon[33507]: 5.16 scrub ok 2024-09-11T07:51:35.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:35 smithi104 ceph-mon[33507]: 1.9 scrub starts 2024-09-11T07:51:35.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:35 smithi104 ceph-mon[33507]: 1.9 scrub ok 2024-09-11T07:51:36.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:36 smithi022 ceph-mon[29462]: pgmap v1176: 142 pgs: 2 activating, 2 activating+degraded, 138 active+clean; 431 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 12/585 objects degraded (2.051%); 0 B/s, 0 objects/s recovering 2024-09-11T07:51:36.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:36 smithi022 ceph-mon[29462]: Health check failed: Degraded data redundancy: 12/585 objects degraded (2.051%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:51:36.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:36 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3051303405' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T07:51:36.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:36 smithi022 ceph-mon[29462]: from='client.26288 ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T07:51:36.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:36 smithi022 ceph-mon[37330]: pgmap v1176: 142 pgs: 2 activating, 2 activating+degraded, 138 active+clean; 431 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 12/585 objects degraded (2.051%); 0 B/s, 0 objects/s recovering 2024-09-11T07:51:36.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:36 smithi022 ceph-mon[37330]: Health check failed: Degraded data redundancy: 12/585 objects degraded (2.051%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:51:36.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:36 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3051303405' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T07:51:36.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:36 smithi022 ceph-mon[37330]: from='client.26288 ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T07:51:36.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:36 smithi104 ceph-mon[33507]: pgmap v1176: 142 pgs: 2 activating, 2 activating+degraded, 138 active+clean; 431 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 12/585 objects degraded (2.051%); 0 B/s, 0 objects/s recovering 2024-09-11T07:51:36.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:36 smithi104 ceph-mon[33507]: Health check failed: Degraded data redundancy: 12/585 objects degraded (2.051%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:51:36.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:36 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3051303405' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T07:51:36.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:36 smithi104 ceph-mon[33507]: from='client.26288 ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-11T07:51:36.851 INFO:teuthology.orchestra.run.smithi022.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-11T07:51:37.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:37 smithi022 ceph-mon[37330]: 6.1e scrub starts 2024-09-11T07:51:37.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:37 smithi022 ceph-mon[37330]: 6.1e scrub ok 2024-09-11T07:51:37.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:37 smithi022 ceph-mon[37330]: 6.a scrub starts 2024-09-11T07:51:37.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:37 smithi022 ceph-mon[37330]: 6.a scrub ok 2024-09-11T07:51:37.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:37 smithi022 ceph-mon[37330]: from='client.26288 ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-11T07:51:37.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:37 smithi022 ceph-mon[37330]: osdmap e511: 8 total, 8 up, 8 in 2024-09-11T07:51:37.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:37 smithi022 ceph-mon[29462]: 6.1e scrub starts 2024-09-11T07:51:37.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:37 smithi022 ceph-mon[29462]: 6.1e scrub ok 2024-09-11T07:51:37.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:37 smithi022 ceph-mon[29462]: 6.a scrub starts 2024-09-11T07:51:37.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:37 smithi022 ceph-mon[29462]: 6.a scrub ok 2024-09-11T07:51:37.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:37 smithi022 ceph-mon[29462]: from='client.26288 ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-11T07:51:37.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:37 smithi022 ceph-mon[29462]: osdmap e511: 8 total, 8 up, 8 in 2024-09-11T07:51:37.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:37 smithi104 ceph-mon[33507]: 6.1e scrub starts 2024-09-11T07:51:37.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:37 smithi104 ceph-mon[33507]: 6.1e scrub ok 2024-09-11T07:51:37.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:37 smithi104 ceph-mon[33507]: 6.a scrub starts 2024-09-11T07:51:37.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:37 smithi104 ceph-mon[33507]: 6.a scrub ok 2024-09-11T07:51:37.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:37 smithi104 ceph-mon[33507]: from='client.26288 ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-11T07:51:37.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:37 smithi104 ceph-mon[33507]: osdmap e511: 8 total, 8 up, 8 in 2024-09-11T07:51:37.691 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight 6 1 2024-09-11T07:51:37.960 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:51:38.332 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:38 smithi022 ceph-mon[29462]: 5.19 scrub starts 2024-09-11T07:51:38.332 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:38 smithi022 ceph-mon[29462]: 5.19 scrub ok 2024-09-11T07:51:38.332 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:38 smithi022 ceph-mon[29462]: pgmap v1178: 142 pgs: 2 activating, 2 activating+degraded, 138 active+clean; 635 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 12/588 objects degraded (2.041%); 0 B/s, 0 objects/s recovering 2024-09-11T07:51:38.332 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:38 smithi022 ceph-mon[29462]: 5.4 scrub starts 2024-09-11T07:51:38.332 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:38 smithi022 ceph-mon[29462]: 5.4 scrub ok 2024-09-11T07:51:38.332 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:38 smithi022 ceph-mon[29462]: osdmap e512: 8 total, 8 up, 8 in 2024-09-11T07:51:38.332 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:51:38.332 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:51:38.333 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:38 smithi022 ceph-mon[37330]: 5.19 scrub starts 2024-09-11T07:51:38.333 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:38 smithi022 ceph-mon[37330]: 5.19 scrub ok 2024-09-11T07:51:38.333 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:38 smithi022 ceph-mon[37330]: pgmap v1178: 142 pgs: 2 activating, 2 activating+degraded, 138 active+clean; 635 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 12/588 objects degraded (2.041%); 0 B/s, 0 objects/s recovering 2024-09-11T07:51:38.333 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:38 smithi022 ceph-mon[37330]: 5.4 scrub starts 2024-09-11T07:51:38.333 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:38 smithi022 ceph-mon[37330]: 5.4 scrub ok 2024-09-11T07:51:38.333 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:38 smithi022 ceph-mon[37330]: osdmap e512: 8 total, 8 up, 8 in 2024-09-11T07:51:38.333 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:51:38.333 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:51:38.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:38 smithi104 ceph-mon[33507]: 5.19 scrub starts 2024-09-11T07:51:38.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:38 smithi104 ceph-mon[33507]: 5.19 scrub ok 2024-09-11T07:51:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:38 smithi104 ceph-mon[33507]: pgmap v1178: 142 pgs: 2 activating, 2 activating+degraded, 138 active+clean; 635 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 12/588 objects degraded (2.041%); 0 B/s, 0 objects/s recovering 2024-09-11T07:51:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:38 smithi104 ceph-mon[33507]: 5.4 scrub starts 2024-09-11T07:51:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:38 smithi104 ceph-mon[33507]: 5.4 scrub ok 2024-09-11T07:51:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:38 smithi104 ceph-mon[33507]: osdmap e512: 8 total, 8 up, 8 in 2024-09-11T07:51:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:51:38.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:51:39.355 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:39 smithi022 ceph-mon[29462]: 6.10 scrub starts 2024-09-11T07:51:39.356 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:39 smithi022 ceph-mon[29462]: 6.10 scrub ok 2024-09-11T07:51:39.356 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:39 smithi022 ceph-mon[37330]: 6.10 scrub starts 2024-09-11T07:51:39.356 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:39 smithi022 ceph-mon[37330]: 6.10 scrub ok 2024-09-11T07:51:39.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:39 smithi104 ceph-mon[33507]: 6.10 scrub starts 2024-09-11T07:51:39.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:39 smithi104 ceph-mon[33507]: 6.10 scrub ok 2024-09-11T07:51:40.163 INFO:teuthology.orchestra.run.smithi022.stderr:reweighted osd.6 to 1 (10000) 2024-09-11T07:51:40.416 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:51:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:51:40] "GET /metrics HTTP/1.1" 200 222474 "" "Prometheus/2.43.0" 2024-09-11T07:51:40.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:40 smithi022 ceph-mon[29462]: 4.12 scrub starts 2024-09-11T07:51:40.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:40 smithi022 ceph-mon[29462]: 4.12 scrub ok 2024-09-11T07:51:40.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:40 smithi022 ceph-mon[29462]: pgmap v1180: 142 pgs: 2 activating, 2 activating+degraded, 138 active+clean; 635 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 12/588 objects degraded (2.041%); 0 B/s, 1 objects/s recovering 2024-09-11T07:51:40.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:40 smithi022 ceph-mon[29462]: 6.1a scrub starts 2024-09-11T07:51:40.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:40 smithi022 ceph-mon[29462]: 6.1a scrub ok 2024-09-11T07:51:40.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:40 smithi022 ceph-mon[29462]: 5.3 scrub starts 2024-09-11T07:51:40.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:40 smithi022 ceph-mon[29462]: 5.3 scrub ok 2024-09-11T07:51:40.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:40 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3138225511' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T07:51:40.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:40 smithi022 ceph-mon[29462]: from='client.26294 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T07:51:40.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:40 smithi022 ceph-mon[29462]: 2.7 scrub starts 2024-09-11T07:51:40.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:40 smithi022 ceph-mon[29462]: 2.7 scrub ok 2024-09-11T07:51:40.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:40 smithi022 ceph-mon[37330]: 4.12 scrub starts 2024-09-11T07:51:40.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:40 smithi022 ceph-mon[37330]: 4.12 scrub ok 2024-09-11T07:51:40.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:40 smithi022 ceph-mon[37330]: pgmap v1180: 142 pgs: 2 activating, 2 activating+degraded, 138 active+clean; 635 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 12/588 objects degraded (2.041%); 0 B/s, 1 objects/s recovering 2024-09-11T07:51:40.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:40 smithi022 ceph-mon[37330]: 6.1a scrub starts 2024-09-11T07:51:40.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:40 smithi022 ceph-mon[37330]: 6.1a scrub ok 2024-09-11T07:51:40.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:40 smithi022 ceph-mon[37330]: 5.3 scrub starts 2024-09-11T07:51:40.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:40 smithi022 ceph-mon[37330]: 5.3 scrub ok 2024-09-11T07:51:40.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:40 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3138225511' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T07:51:40.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:40 smithi022 ceph-mon[37330]: from='client.26294 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T07:51:40.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:40 smithi022 ceph-mon[37330]: 2.7 scrub starts 2024-09-11T07:51:40.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:40 smithi022 ceph-mon[37330]: 2.7 scrub ok 2024-09-11T07:51:40.474 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:40 smithi104 ceph-mon[33507]: 4.12 scrub starts 2024-09-11T07:51:40.474 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:40 smithi104 ceph-mon[33507]: 4.12 scrub ok 2024-09-11T07:51:40.474 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:40 smithi104 ceph-mon[33507]: pgmap v1180: 142 pgs: 2 activating, 2 activating+degraded, 138 active+clean; 635 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 12/588 objects degraded (2.041%); 0 B/s, 1 objects/s recovering 2024-09-11T07:51:40.474 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:40 smithi104 ceph-mon[33507]: 6.1a scrub starts 2024-09-11T07:51:40.474 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:40 smithi104 ceph-mon[33507]: 6.1a scrub ok 2024-09-11T07:51:40.475 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:40 smithi104 ceph-mon[33507]: 5.3 scrub starts 2024-09-11T07:51:40.475 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:40 smithi104 ceph-mon[33507]: 5.3 scrub ok 2024-09-11T07:51:40.475 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:40 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3138225511' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T07:51:40.475 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:40 smithi104 ceph-mon[33507]: from='client.26294 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T07:51:40.475 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:40 smithi104 ceph-mon[33507]: 2.7 scrub starts 2024-09-11T07:51:40.475 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:40 smithi104 ceph-mon[33507]: 2.7 scrub ok 2024-09-11T07:51:40.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:51:40 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:51:41.050 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 6 1 2024-09-11T07:51:41.313 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:51:41.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:41 smithi022 ceph-mon[29462]: from='client.26294 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-11T07:51:41.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:41 smithi022 ceph-mon[29462]: osdmap e513: 8 total, 8 up, 8 in 2024-09-11T07:51:41.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:41 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:51:41.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:41 smithi022 ceph-mon[29462]: Health check update: Degraded data redundancy: 12/588 objects degraded (2.041%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:51:41.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:41 smithi022 ceph-mon[37330]: from='client.26294 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-11T07:51:41.420 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:41 smithi022 ceph-mon[37330]: osdmap e513: 8 total, 8 up, 8 in 2024-09-11T07:51:41.420 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:41 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:51:41.420 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:41 smithi022 ceph-mon[37330]: Health check update: Degraded data redundancy: 12/588 objects degraded (2.041%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:51:41.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:41 smithi104 ceph-mon[33507]: from='client.26294 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-11T07:51:41.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:41 smithi104 ceph-mon[33507]: osdmap e513: 8 total, 8 up, 8 in 2024-09-11T07:51:41.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:41 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:51:41.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:41 smithi104 ceph-mon[33507]: Health check update: Degraded data redundancy: 12/588 objects degraded (2.041%), 2 pgs degraded (PG_DEGRADED) 2024-09-11T07:51:42.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:42 smithi022 ceph-mon[29462]: pgmap v1182: 142 pgs: 142 active+clean; 739 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 4.3 KiB/s rd, 25 KiB/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:51:42.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:42 smithi022 ceph-mon[29462]: 4.c scrub starts 2024-09-11T07:51:42.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:42 smithi022 ceph-mon[29462]: 4.c scrub ok 2024-09-11T07:51:42.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:42 smithi022 ceph-mon[29462]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/588 objects degraded (2.041%), 2 pgs degraded) 2024-09-11T07:51:42.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:42 smithi022 ceph-mon[37330]: pgmap v1182: 142 pgs: 142 active+clean; 739 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 4.3 KiB/s rd, 25 KiB/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:51:42.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:42 smithi022 ceph-mon[37330]: 4.c scrub starts 2024-09-11T07:51:42.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:42 smithi022 ceph-mon[37330]: 4.c scrub ok 2024-09-11T07:51:42.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:42 smithi022 ceph-mon[37330]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/588 objects degraded (2.041%), 2 pgs degraded) 2024-09-11T07:51:42.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:42 smithi104 ceph-mon[33507]: pgmap v1182: 142 pgs: 142 active+clean; 739 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 4.3 KiB/s rd, 25 KiB/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2024-09-11T07:51:42.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:42 smithi104 ceph-mon[33507]: 4.c scrub starts 2024-09-11T07:51:42.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:42 smithi104 ceph-mon[33507]: 4.c scrub ok 2024-09-11T07:51:42.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:42 smithi104 ceph-mon[33507]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/588 objects degraded (2.041%), 2 pgs degraded) 2024-09-11T07:51:44.173 INFO:teuthology.orchestra.run.smithi022.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-11T07:51:44.494 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:44 smithi022 ceph-mon[29462]: 5.7 scrub starts 2024-09-11T07:51:44.494 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:44 smithi022 ceph-mon[29462]: 5.7 scrub ok 2024-09-11T07:51:44.494 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:44 smithi022 ceph-mon[29462]: 2.a scrub starts 2024-09-11T07:51:44.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:44 smithi022 ceph-mon[29462]: 2.a scrub ok 2024-09-11T07:51:44.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:44 smithi022 ceph-mon[29462]: pgmap v1183: 142 pgs: 142 active+clean; 739 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 7.7 KiB/s rd, 48 KiB/s wr, 7 op/s 2024-09-11T07:51:44.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:44 smithi022 ceph-mon[29462]: 4.a scrub starts 2024-09-11T07:51:44.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:44 smithi022 ceph-mon[29462]: 4.a scrub ok 2024-09-11T07:51:44.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:44 smithi022 ceph-mon[29462]: 6.2 scrub starts 2024-09-11T07:51:44.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:44 smithi022 ceph-mon[29462]: 6.2 scrub ok 2024-09-11T07:51:44.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:44 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2602103499' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T07:51:44.495 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:44 smithi022 ceph-mon[29462]: from='client.26300 ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T07:51:44.495 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:44 smithi022 ceph-mon[37330]: 5.7 scrub starts 2024-09-11T07:51:44.495 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:44 smithi022 ceph-mon[37330]: 5.7 scrub ok 2024-09-11T07:51:44.495 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:44 smithi022 ceph-mon[37330]: 2.a scrub starts 2024-09-11T07:51:44.496 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:44 smithi022 ceph-mon[37330]: 2.a scrub ok 2024-09-11T07:51:44.496 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:44 smithi022 ceph-mon[37330]: pgmap v1183: 142 pgs: 142 active+clean; 739 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 7.7 KiB/s rd, 48 KiB/s wr, 7 op/s 2024-09-11T07:51:44.496 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:44 smithi022 ceph-mon[37330]: 4.a scrub starts 2024-09-11T07:51:44.496 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:44 smithi022 ceph-mon[37330]: 4.a scrub ok 2024-09-11T07:51:44.496 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:44 smithi022 ceph-mon[37330]: 6.2 scrub starts 2024-09-11T07:51:44.496 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:44 smithi022 ceph-mon[37330]: 6.2 scrub ok 2024-09-11T07:51:44.496 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:44 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2602103499' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T07:51:44.496 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:44 smithi022 ceph-mon[37330]: from='client.26300 ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T07:51:44.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:51:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:51:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:51:44.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:44 smithi104 ceph-mon[33507]: 5.7 scrub starts 2024-09-11T07:51:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:44 smithi104 ceph-mon[33507]: 5.7 scrub ok 2024-09-11T07:51:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:44 smithi104 ceph-mon[33507]: 2.a scrub starts 2024-09-11T07:51:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:44 smithi104 ceph-mon[33507]: 2.a scrub ok 2024-09-11T07:51:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:44 smithi104 ceph-mon[33507]: pgmap v1183: 142 pgs: 142 active+clean; 739 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 7.7 KiB/s rd, 48 KiB/s wr, 7 op/s 2024-09-11T07:51:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:44 smithi104 ceph-mon[33507]: 4.a scrub starts 2024-09-11T07:51:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:44 smithi104 ceph-mon[33507]: 4.a scrub ok 2024-09-11T07:51:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:44 smithi104 ceph-mon[33507]: 6.2 scrub starts 2024-09-11T07:51:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:44 smithi104 ceph-mon[33507]: 6.2 scrub ok 2024-09-11T07:51:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:44 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2602103499' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T07:51:44.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:44 smithi104 ceph-mon[33507]: from='client.26300 ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-09-11T07:51:44.801 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight 7 1 2024-09-11T07:51:45.068 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:51:45.448 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:45 smithi022 ceph-mon[29462]: from='client.26300 ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-11T07:51:45.448 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:45 smithi022 ceph-mon[29462]: osdmap e514: 8 total, 8 up, 8 in 2024-09-11T07:51:45.448 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:45 smithi022 ceph-mon[29462]: 2.12 deep-scrub starts 2024-09-11T07:51:45.448 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:45 smithi022 ceph-mon[29462]: 2.12 deep-scrub ok 2024-09-11T07:51:45.448 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:45 smithi022 ceph-mon[37330]: from='client.26300 ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-11T07:51:45.448 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:45 smithi022 ceph-mon[37330]: osdmap e514: 8 total, 8 up, 8 in 2024-09-11T07:51:45.448 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:45 smithi022 ceph-mon[37330]: 2.12 deep-scrub starts 2024-09-11T07:51:45.448 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:45 smithi022 ceph-mon[37330]: 2.12 deep-scrub ok 2024-09-11T07:51:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:45 smithi104 ceph-mon[33507]: from='client.26300 ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-11T07:51:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:45 smithi104 ceph-mon[33507]: osdmap e514: 8 total, 8 up, 8 in 2024-09-11T07:51:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:45 smithi104 ceph-mon[33507]: 2.12 deep-scrub starts 2024-09-11T07:51:45.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:45 smithi104 ceph-mon[33507]: 2.12 deep-scrub ok 2024-09-11T07:51:46.471 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:46 smithi022 ceph-mon[29462]: pgmap v1185: 142 pgs: 12 peering, 130 active+clean; 844 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 13 KiB/s rd, 83 KiB/s wr, 13 op/s; 7.9 KiB/s, 0 objects/s recovering 2024-09-11T07:51:46.471 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:46 smithi022 ceph-mon[29462]: Health check failed: Reduced data availability: 3 pgs inactive, 7 pgs peering (PG_AVAILABILITY) 2024-09-11T07:51:46.471 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:46 smithi022 ceph-mon[29462]: osdmap e515: 8 total, 8 up, 8 in 2024-09-11T07:51:46.471 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:46 smithi022 ceph-mon[29462]: 2.2 deep-scrub starts 2024-09-11T07:51:46.471 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:46 smithi022 ceph-mon[29462]: 2.2 deep-scrub ok 2024-09-11T07:51:46.471 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:46 smithi022 ceph-mon[29462]: 5.11 scrub starts 2024-09-11T07:51:46.471 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:46 smithi022 ceph-mon[29462]: 5.11 scrub ok 2024-09-11T07:51:46.471 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:46 smithi022 ceph-mon[37330]: pgmap v1185: 142 pgs: 12 peering, 130 active+clean; 844 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 13 KiB/s rd, 83 KiB/s wr, 13 op/s; 7.9 KiB/s, 0 objects/s recovering 2024-09-11T07:51:46.472 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:46 smithi022 ceph-mon[37330]: Health check failed: Reduced data availability: 3 pgs inactive, 7 pgs peering (PG_AVAILABILITY) 2024-09-11T07:51:46.472 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:46 smithi022 ceph-mon[37330]: osdmap e515: 8 total, 8 up, 8 in 2024-09-11T07:51:46.472 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:46 smithi022 ceph-mon[37330]: 2.2 deep-scrub starts 2024-09-11T07:51:46.472 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:46 smithi022 ceph-mon[37330]: 2.2 deep-scrub ok 2024-09-11T07:51:46.472 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:46 smithi022 ceph-mon[37330]: 5.11 scrub starts 2024-09-11T07:51:46.472 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:46 smithi022 ceph-mon[37330]: 5.11 scrub ok 2024-09-11T07:51:46.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:46 smithi104 ceph-mon[33507]: pgmap v1185: 142 pgs: 12 peering, 130 active+clean; 844 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 13 KiB/s rd, 83 KiB/s wr, 13 op/s; 7.9 KiB/s, 0 objects/s recovering 2024-09-11T07:51:46.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:46 smithi104 ceph-mon[33507]: Health check failed: Reduced data availability: 3 pgs inactive, 7 pgs peering (PG_AVAILABILITY) 2024-09-11T07:51:46.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:46 smithi104 ceph-mon[33507]: osdmap e515: 8 total, 8 up, 8 in 2024-09-11T07:51:46.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:46 smithi104 ceph-mon[33507]: 2.2 deep-scrub starts 2024-09-11T07:51:46.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:46 smithi104 ceph-mon[33507]: 2.2 deep-scrub ok 2024-09-11T07:51:46.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:46 smithi104 ceph-mon[33507]: 5.11 scrub starts 2024-09-11T07:51:46.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:46 smithi104 ceph-mon[33507]: 5.11 scrub ok 2024-09-11T07:51:47.194 INFO:teuthology.orchestra.run.smithi022.stderr:reweighted osd.7 to 1 (10000) 2024-09-11T07:51:47.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:47 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:51:47.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:47 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:51:47.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:47 smithi104 ceph-mon[33507]: 4.8 scrub starts 2024-09-11T07:51:47.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:47 smithi104 ceph-mon[33507]: 4.8 scrub ok 2024-09-11T07:51:47.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:47 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1183607668' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T07:51:47.649 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:47 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:51:47.649 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:47 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:51:47.649 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:47 smithi022 ceph-mon[37330]: 4.8 scrub starts 2024-09-11T07:51:47.649 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:47 smithi022 ceph-mon[37330]: 4.8 scrub ok 2024-09-11T07:51:47.650 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:47 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1183607668' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T07:51:47.650 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:47 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:51:47.650 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:47 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:51:47.650 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:47 smithi022 ceph-mon[29462]: 4.8 scrub starts 2024-09-11T07:51:47.650 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:47 smithi022 ceph-mon[29462]: 4.8 scrub ok 2024-09-11T07:51:47.650 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:47 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1183607668' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T07:51:47.807 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 7 1 2024-09-11T07:51:48.070 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:51:48.473 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:48 smithi022 ceph-mon[29462]: pgmap v1187: 142 pgs: 12 peering, 130 active+clean; 844 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 7.0 KiB/s rd, 43 KiB/s wr, 7 op/s; 15 KiB/s, 0 objects/s recovering 2024-09-11T07:51:48.473 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:48 smithi022 ceph-mon[29462]: 6.17 scrub starts 2024-09-11T07:51:48.473 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:48 smithi022 ceph-mon[29462]: 6.17 scrub ok 2024-09-11T07:51:48.473 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:48 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1183607668' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-11T07:51:48.473 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:48 smithi022 ceph-mon[29462]: osdmap e516: 8 total, 8 up, 8 in 2024-09-11T07:51:48.473 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:48 smithi022 ceph-mon[29462]: 1.1 scrub starts 2024-09-11T07:51:48.473 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:48 smithi022 ceph-mon[29462]: 1.1 scrub ok 2024-09-11T07:51:48.473 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:48 smithi022 ceph-mon[29462]: 4.13 scrub starts 2024-09-11T07:51:48.474 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:48 smithi022 ceph-mon[29462]: 4.13 scrub ok 2024-09-11T07:51:48.474 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:48 smithi022 ceph-mon[37330]: pgmap v1187: 142 pgs: 12 peering, 130 active+clean; 844 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 7.0 KiB/s rd, 43 KiB/s wr, 7 op/s; 15 KiB/s, 0 objects/s recovering 2024-09-11T07:51:48.474 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:48 smithi022 ceph-mon[37330]: 6.17 scrub starts 2024-09-11T07:51:48.474 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:48 smithi022 ceph-mon[37330]: 6.17 scrub ok 2024-09-11T07:51:48.474 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:48 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1183607668' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-11T07:51:48.474 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:48 smithi022 ceph-mon[37330]: osdmap e516: 8 total, 8 up, 8 in 2024-09-11T07:51:48.474 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:48 smithi022 ceph-mon[37330]: 1.1 scrub starts 2024-09-11T07:51:48.474 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:48 smithi022 ceph-mon[37330]: 1.1 scrub ok 2024-09-11T07:51:48.474 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:48 smithi022 ceph-mon[37330]: 4.13 scrub starts 2024-09-11T07:51:48.474 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:48 smithi022 ceph-mon[37330]: 4.13 scrub ok 2024-09-11T07:51:48.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:48 smithi104 ceph-mon[33507]: pgmap v1187: 142 pgs: 12 peering, 130 active+clean; 844 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 7.0 KiB/s rd, 43 KiB/s wr, 7 op/s; 15 KiB/s, 0 objects/s recovering 2024-09-11T07:51:48.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:48 smithi104 ceph-mon[33507]: 6.17 scrub starts 2024-09-11T07:51:48.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:48 smithi104 ceph-mon[33507]: 6.17 scrub ok 2024-09-11T07:51:48.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:48 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1183607668' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-11T07:51:48.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:48 smithi104 ceph-mon[33507]: osdmap e516: 8 total, 8 up, 8 in 2024-09-11T07:51:48.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:48 smithi104 ceph-mon[33507]: 1.1 scrub starts 2024-09-11T07:51:48.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:48 smithi104 ceph-mon[33507]: 1.1 scrub ok 2024-09-11T07:51:48.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:48 smithi104 ceph-mon[33507]: 4.13 scrub starts 2024-09-11T07:51:48.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:48 smithi104 ceph-mon[33507]: 4.13 scrub ok 2024-09-11T07:51:49.456 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:49 smithi022 ceph-mon[29462]: 5.13 scrub starts 2024-09-11T07:51:49.456 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:49 smithi022 ceph-mon[29462]: 5.13 scrub ok 2024-09-11T07:51:49.456 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:49 smithi022 ceph-mon[29462]: osdmap e517: 8 total, 8 up, 8 in 2024-09-11T07:51:49.456 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:49 smithi022 ceph-mon[29462]: 6.f scrub starts 2024-09-11T07:51:49.456 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:49 smithi022 ceph-mon[29462]: 6.f scrub ok 2024-09-11T07:51:49.456 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:49 smithi022 ceph-mon[29462]: 5.1e scrub starts 2024-09-11T07:51:49.456 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:49 smithi022 ceph-mon[29462]: 5.1e scrub ok 2024-09-11T07:51:49.456 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:49 smithi022 ceph-mon[29462]: 2.11 scrub starts 2024-09-11T07:51:49.457 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:49 smithi022 ceph-mon[29462]: 2.11 scrub ok 2024-09-11T07:51:49.457 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:49 smithi022 ceph-mon[29462]: 4.1a scrub starts 2024-09-11T07:51:49.457 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:49 smithi022 ceph-mon[29462]: 4.1a scrub ok 2024-09-11T07:51:49.457 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:49 smithi022 ceph-mon[37330]: 5.13 scrub starts 2024-09-11T07:51:49.457 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:49 smithi022 ceph-mon[37330]: 5.13 scrub ok 2024-09-11T07:51:49.457 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:49 smithi022 ceph-mon[37330]: osdmap e517: 8 total, 8 up, 8 in 2024-09-11T07:51:49.457 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:49 smithi022 ceph-mon[37330]: 6.f scrub starts 2024-09-11T07:51:49.457 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:49 smithi022 ceph-mon[37330]: 6.f scrub ok 2024-09-11T07:51:49.457 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:49 smithi022 ceph-mon[37330]: 5.1e scrub starts 2024-09-11T07:51:49.457 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:49 smithi022 ceph-mon[37330]: 5.1e scrub ok 2024-09-11T07:51:49.457 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:49 smithi022 ceph-mon[37330]: 2.11 scrub starts 2024-09-11T07:51:49.458 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:49 smithi022 ceph-mon[37330]: 2.11 scrub ok 2024-09-11T07:51:49.458 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:49 smithi022 ceph-mon[37330]: 4.1a scrub starts 2024-09-11T07:51:49.458 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:49 smithi022 ceph-mon[37330]: 4.1a scrub ok 2024-09-11T07:51:49.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:49 smithi104 ceph-mon[33507]: 5.13 scrub starts 2024-09-11T07:51:49.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:49 smithi104 ceph-mon[33507]: 5.13 scrub ok 2024-09-11T07:51:49.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:49 smithi104 ceph-mon[33507]: osdmap e517: 8 total, 8 up, 8 in 2024-09-11T07:51:49.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:49 smithi104 ceph-mon[33507]: 6.f scrub starts 2024-09-11T07:51:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:49 smithi104 ceph-mon[33507]: 6.f scrub ok 2024-09-11T07:51:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:49 smithi104 ceph-mon[33507]: 5.1e scrub starts 2024-09-11T07:51:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:49 smithi104 ceph-mon[33507]: 5.1e scrub ok 2024-09-11T07:51:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:49 smithi104 ceph-mon[33507]: 2.11 scrub starts 2024-09-11T07:51:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:49 smithi104 ceph-mon[33507]: 2.11 scrub ok 2024-09-11T07:51:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:49 smithi104 ceph-mon[33507]: 4.1a scrub starts 2024-09-11T07:51:49.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:49 smithi104 ceph-mon[33507]: 4.1a scrub ok 2024-09-11T07:51:50.204 INFO:teuthology.orchestra.run.smithi022.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-11T07:51:50.485 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:50 smithi104 ceph-mon[33507]: pgmap v1190: 142 pgs: 12 peering, 130 active+clean; 844 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 10 KiB/s rd, 62 KiB/s wr, 9 op/s; 22 KiB/s, 0 objects/s recovering 2024-09-11T07:51:50.485 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:50 smithi104 ceph-mon[33507]: 2.17 scrub starts 2024-09-11T07:51:50.485 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:50 smithi104 ceph-mon[33507]: 2.17 scrub ok 2024-09-11T07:51:50.485 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:50 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3492449797' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T07:51:50.486 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:50 smithi104 ceph-mon[33507]: from='client.26312 ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T07:51:50.486 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:50 smithi104 ceph-mon[33507]: 4.e scrub starts 2024-09-11T07:51:50.486 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:50 smithi104 ceph-mon[33507]: 4.e scrub ok 2024-09-11T07:51:50.486 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:51:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:51:50] "GET /metrics HTTP/1.1" 200 222474 "" "Prometheus/2.43.0" 2024-09-11T07:51:50.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:50 smithi022 ceph-mon[29462]: pgmap v1190: 142 pgs: 12 peering, 130 active+clean; 844 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 10 KiB/s rd, 62 KiB/s wr, 9 op/s; 22 KiB/s, 0 objects/s recovering 2024-09-11T07:51:50.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:50 smithi022 ceph-mon[29462]: 2.17 scrub starts 2024-09-11T07:51:50.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:50 smithi022 ceph-mon[29462]: 2.17 scrub ok 2024-09-11T07:51:50.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:50 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3492449797' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T07:51:50.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:50 smithi022 ceph-mon[29462]: from='client.26312 ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T07:51:50.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:50 smithi022 ceph-mon[29462]: 4.e scrub starts 2024-09-11T07:51:50.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:50 smithi022 ceph-mon[29462]: 4.e scrub ok 2024-09-11T07:51:50.487 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:50 smithi022 ceph-mon[37330]: pgmap v1190: 142 pgs: 12 peering, 130 active+clean; 844 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 10 KiB/s rd, 62 KiB/s wr, 9 op/s; 22 KiB/s, 0 objects/s recovering 2024-09-11T07:51:50.487 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:50 smithi022 ceph-mon[37330]: 2.17 scrub starts 2024-09-11T07:51:50.487 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:50 smithi022 ceph-mon[37330]: 2.17 scrub ok 2024-09-11T07:51:50.487 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:50 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3492449797' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T07:51:50.487 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:50 smithi022 ceph-mon[37330]: from='client.26312 ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-11T07:51:50.487 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:50 smithi022 ceph-mon[37330]: 4.e scrub starts 2024-09-11T07:51:50.487 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:50 smithi022 ceph-mon[37330]: 4.e scrub ok 2024-09-11T07:51:50.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:51:50 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:51:50.793 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd reweight 2 1 2024-09-11T07:51:51.046 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:51:51.447 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:51 smithi022 ceph-mon[29462]: 4.0 scrub starts 2024-09-11T07:51:51.448 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:51 smithi022 ceph-mon[29462]: 4.0 scrub ok 2024-09-11T07:51:51.448 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:51 smithi022 ceph-mon[29462]: from='client.26312 ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-11T07:51:51.448 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:51 smithi022 ceph-mon[29462]: osdmap e518: 8 total, 8 up, 8 in 2024-09-11T07:51:51.448 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:51 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:51:51.448 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:51 smithi022 ceph-mon[29462]: 1.6 scrub starts 2024-09-11T07:51:51.448 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:51 smithi022 ceph-mon[29462]: 1.6 scrub ok 2024-09-11T07:51:51.448 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:51 smithi022 ceph-mon[29462]: Health check update: Reduced data availability: 3 pgs inactive, 8 pgs peering (PG_AVAILABILITY) 2024-09-11T07:51:51.448 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:51 smithi022 ceph-mon[37330]: 4.0 scrub starts 2024-09-11T07:51:51.449 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:51 smithi022 ceph-mon[37330]: 4.0 scrub ok 2024-09-11T07:51:51.449 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:51 smithi022 ceph-mon[37330]: from='client.26312 ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-11T07:51:51.449 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:51 smithi022 ceph-mon[37330]: osdmap e518: 8 total, 8 up, 8 in 2024-09-11T07:51:51.449 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:51 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:51:51.449 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:51 smithi022 ceph-mon[37330]: 1.6 scrub starts 2024-09-11T07:51:51.449 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:51 smithi022 ceph-mon[37330]: 1.6 scrub ok 2024-09-11T07:51:51.449 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:51 smithi022 ceph-mon[37330]: Health check update: Reduced data availability: 3 pgs inactive, 8 pgs peering (PG_AVAILABILITY) 2024-09-11T07:51:51.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:51 smithi104 ceph-mon[33507]: 4.0 scrub starts 2024-09-11T07:51:51.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:51 smithi104 ceph-mon[33507]: 4.0 scrub ok 2024-09-11T07:51:51.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:51 smithi104 ceph-mon[33507]: from='client.26312 ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-11T07:51:51.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:51 smithi104 ceph-mon[33507]: osdmap e518: 8 total, 8 up, 8 in 2024-09-11T07:51:51.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:51 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:51:51.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:51 smithi104 ceph-mon[33507]: 1.6 scrub starts 2024-09-11T07:51:51.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:51 smithi104 ceph-mon[33507]: 1.6 scrub ok 2024-09-11T07:51:51.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:51 smithi104 ceph-mon[33507]: Health check update: Reduced data availability: 3 pgs inactive, 8 pgs peering (PG_AVAILABILITY) 2024-09-11T07:51:52.518 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:52 smithi022 ceph-mon[37330]: pgmap v1192: 142 pgs: 142 active+clean; 792 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 37 KiB/s, 6 objects/s recovering 2024-09-11T07:51:52.519 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:52 smithi022 ceph-mon[37330]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs inactive, 8 pgs peering) 2024-09-11T07:51:52.519 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:52 smithi022 ceph-mon[37330]: 1.a scrub starts 2024-09-11T07:51:52.519 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:52 smithi022 ceph-mon[37330]: 1.a scrub ok 2024-09-11T07:51:52.519 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:52 smithi022 ceph-mon[37330]: 1.7 scrub starts 2024-09-11T07:51:52.519 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:52 smithi022 ceph-mon[37330]: 1.7 scrub ok 2024-09-11T07:51:52.519 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:52 smithi022 ceph-mon[29462]: pgmap v1192: 142 pgs: 142 active+clean; 792 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 37 KiB/s, 6 objects/s recovering 2024-09-11T07:51:52.519 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:52 smithi022 ceph-mon[29462]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs inactive, 8 pgs peering) 2024-09-11T07:51:52.519 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:52 smithi022 ceph-mon[29462]: 1.a scrub starts 2024-09-11T07:51:52.519 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:52 smithi022 ceph-mon[29462]: 1.a scrub ok 2024-09-11T07:51:52.519 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:52 smithi022 ceph-mon[29462]: 1.7 scrub starts 2024-09-11T07:51:52.520 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:52 smithi022 ceph-mon[29462]: 1.7 scrub ok 2024-09-11T07:51:52.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:52 smithi104 ceph-mon[33507]: pgmap v1192: 142 pgs: 142 active+clean; 792 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 37 KiB/s, 6 objects/s recovering 2024-09-11T07:51:52.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:52 smithi104 ceph-mon[33507]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs inactive, 8 pgs peering) 2024-09-11T07:51:52.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:52 smithi104 ceph-mon[33507]: 1.a scrub starts 2024-09-11T07:51:52.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:52 smithi104 ceph-mon[33507]: 1.a scrub ok 2024-09-11T07:51:52.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:52 smithi104 ceph-mon[33507]: 1.7 scrub starts 2024-09-11T07:51:52.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:52 smithi104 ceph-mon[33507]: 1.7 scrub ok 2024-09-11T07:51:53.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:53 smithi104 ceph-mon[33507]: 2.19 scrub starts 2024-09-11T07:51:53.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:53 smithi104 ceph-mon[33507]: 2.19 scrub ok 2024-09-11T07:51:53.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:51:53.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:53 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:51:53.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:53 smithi022 ceph-mon[29462]: 2.19 scrub starts 2024-09-11T07:51:53.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:53 smithi022 ceph-mon[29462]: 2.19 scrub ok 2024-09-11T07:51:53.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:51:53.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:53 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:51:53.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:53 smithi022 ceph-mon[37330]: 2.19 scrub starts 2024-09-11T07:51:53.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:53 smithi022 ceph-mon[37330]: 2.19 scrub ok 2024-09-11T07:51:53.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:51:53.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:53 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:51:54.230 INFO:teuthology.orchestra.run.smithi022.stderr:reweighted osd.2 to 1 (10000) 2024-09-11T07:51:54.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:51:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:51:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:51:54.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:54 smithi104 ceph-mon[33507]: pgmap v1193: 142 pgs: 142 active+clean; 792 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 36 KiB/s, 6 objects/s recovering 2024-09-11T07:51:54.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:54 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2985658665' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T07:51:54.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:54 smithi104 ceph-mon[33507]: from='client.26318 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T07:51:54.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:54 smithi104 ceph-mon[33507]: 4.15 scrub starts 2024-09-11T07:51:54.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:54 smithi104 ceph-mon[33507]: 4.15 scrub ok 2024-09-11T07:51:54.576 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:54 smithi022 ceph-mon[37330]: pgmap v1193: 142 pgs: 142 active+clean; 792 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 36 KiB/s, 6 objects/s recovering 2024-09-11T07:51:54.576 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:54 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2985658665' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T07:51:54.576 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:54 smithi022 ceph-mon[37330]: from='client.26318 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T07:51:54.576 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:54 smithi022 ceph-mon[37330]: 4.15 scrub starts 2024-09-11T07:51:54.576 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:54 smithi022 ceph-mon[37330]: 4.15 scrub ok 2024-09-11T07:51:54.576 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:54 smithi022 ceph-mon[29462]: pgmap v1193: 142 pgs: 142 active+clean; 792 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 36 KiB/s, 6 objects/s recovering 2024-09-11T07:51:54.576 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:54 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2985658665' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T07:51:54.576 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:54 smithi022 ceph-mon[29462]: from='client.26318 ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T07:51:54.577 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:54 smithi022 ceph-mon[29462]: 4.15 scrub starts 2024-09-11T07:51:54.577 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:54 smithi022 ceph-mon[29462]: 4.15 scrub ok 2024-09-11T07:51:54.993 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 2 1 2024-09-11T07:51:55.254 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:51:55.486 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:55 smithi022 ceph-mon[37330]: from='client.26318 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-11T07:51:55.486 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:55 smithi022 ceph-mon[37330]: osdmap e519: 8 total, 8 up, 8 in 2024-09-11T07:51:55.487 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:55 smithi022 ceph-mon[37330]: 1.0 scrub starts 2024-09-11T07:51:55.487 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:55 smithi022 ceph-mon[37330]: 1.0 scrub ok 2024-09-11T07:51:55.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:55 smithi022 ceph-mon[29462]: from='client.26318 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-11T07:51:55.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:55 smithi022 ceph-mon[29462]: osdmap e519: 8 total, 8 up, 8 in 2024-09-11T07:51:55.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:55 smithi022 ceph-mon[29462]: 1.0 scrub starts 2024-09-11T07:51:55.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:55 smithi022 ceph-mon[29462]: 1.0 scrub ok 2024-09-11T07:51:55.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:55 smithi104 ceph-mon[33507]: from='client.26318 ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-11T07:51:55.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:55 smithi104 ceph-mon[33507]: osdmap e519: 8 total, 8 up, 8 in 2024-09-11T07:51:55.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:55 smithi104 ceph-mon[33507]: 1.0 scrub starts 2024-09-11T07:51:55.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:55 smithi104 ceph-mon[33507]: 1.0 scrub ok 2024-09-11T07:51:56.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:56 smithi022 ceph-mon[29462]: pgmap v1195: 142 pgs: 142 active+clean; 635 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.0 KiB/s, 5 objects/s recovering 2024-09-11T07:51:56.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:56 smithi022 ceph-mon[29462]: 5.18 scrub starts 2024-09-11T07:51:56.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:56 smithi022 ceph-mon[29462]: 5.18 scrub ok 2024-09-11T07:51:56.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:56 smithi022 ceph-mon[29462]: 2.10 scrub starts 2024-09-11T07:51:56.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:56 smithi022 ceph-mon[29462]: 2.10 scrub ok 2024-09-11T07:51:56.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:56 smithi022 ceph-mon[37330]: pgmap v1195: 142 pgs: 142 active+clean; 635 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.0 KiB/s, 5 objects/s recovering 2024-09-11T07:51:56.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:56 smithi022 ceph-mon[37330]: 5.18 scrub starts 2024-09-11T07:51:56.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:56 smithi022 ceph-mon[37330]: 5.18 scrub ok 2024-09-11T07:51:56.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:56 smithi022 ceph-mon[37330]: 2.10 scrub starts 2024-09-11T07:51:56.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:56 smithi022 ceph-mon[37330]: 2.10 scrub ok 2024-09-11T07:51:56.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:56 smithi104 ceph-mon[33507]: pgmap v1195: 142 pgs: 142 active+clean; 635 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.0 KiB/s, 5 objects/s recovering 2024-09-11T07:51:56.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:56 smithi104 ceph-mon[33507]: 5.18 scrub starts 2024-09-11T07:51:56.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:56 smithi104 ceph-mon[33507]: 5.18 scrub ok 2024-09-11T07:51:56.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:56 smithi104 ceph-mon[33507]: 2.10 scrub starts 2024-09-11T07:51:56.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:56 smithi104 ceph-mon[33507]: 2.10 scrub ok 2024-09-11T07:51:57.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:57 smithi104 ceph-mon[33507]: 2.5 scrub starts 2024-09-11T07:51:57.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:57 smithi104 ceph-mon[33507]: 2.5 scrub ok 2024-09-11T07:51:57.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:57 smithi022 ceph-mon[29462]: 2.5 scrub starts 2024-09-11T07:51:57.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:57 smithi022 ceph-mon[29462]: 2.5 scrub ok 2024-09-11T07:51:57.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:57 smithi022 ceph-mon[37330]: 2.5 scrub starts 2024-09-11T07:51:57.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:57 smithi022 ceph-mon[37330]: 2.5 scrub ok 2024-09-11T07:51:58.246 INFO:teuthology.orchestra.run.smithi022.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-11T07:51:58.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:58 smithi104 ceph-mon[33507]: pgmap v1196: 142 pgs: 142 active+clean; 635 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 899 B/s, 4 objects/s recovering 2024-09-11T07:51:58.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:58 smithi104 ceph-mon[33507]: 4.9 scrub starts 2024-09-11T07:51:58.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:58 smithi104 ceph-mon[33507]: 4.9 scrub ok 2024-09-11T07:51:58.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:58 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2457404754' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T07:51:58.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:58 smithi104 ceph-mon[33507]: from='client.26324 ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T07:51:58.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:58 smithi104 ceph-mon[33507]: 1.4 scrub starts 2024-09-11T07:51:58.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:58 smithi104 ceph-mon[33507]: 1.4 scrub ok 2024-09-11T07:51:58.569 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:58 smithi022 ceph-mon[29462]: pgmap v1196: 142 pgs: 142 active+clean; 635 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 899 B/s, 4 objects/s recovering 2024-09-11T07:51:58.569 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:58 smithi022 ceph-mon[29462]: 4.9 scrub starts 2024-09-11T07:51:58.569 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:58 smithi022 ceph-mon[29462]: 4.9 scrub ok 2024-09-11T07:51:58.569 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:58 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2457404754' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T07:51:58.569 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:58 smithi022 ceph-mon[29462]: from='client.26324 ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T07:51:58.569 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:58 smithi022 ceph-mon[29462]: 1.4 scrub starts 2024-09-11T07:51:58.570 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:58 smithi022 ceph-mon[29462]: 1.4 scrub ok 2024-09-11T07:51:58.570 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:58 smithi022 ceph-mon[37330]: pgmap v1196: 142 pgs: 142 active+clean; 635 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 899 B/s, 4 objects/s recovering 2024-09-11T07:51:58.570 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:58 smithi022 ceph-mon[37330]: 4.9 scrub starts 2024-09-11T07:51:58.570 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:58 smithi022 ceph-mon[37330]: 4.9 scrub ok 2024-09-11T07:51:58.570 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:58 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2457404754' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T07:51:58.570 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:58 smithi022 ceph-mon[37330]: from='client.26324 ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-09-11T07:51:58.570 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:58 smithi022 ceph-mon[37330]: 1.4 scrub starts 2024-09-11T07:51:58.570 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:58 smithi022 ceph-mon[37330]: 1.4 scrub ok 2024-09-11T07:51:59.011 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-11T07:51:59.011 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-11T07:51:59.011 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-11T07:51:59.011 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-11T07:51:59.011 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-11T07:51:59.011 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T07:51:59.276 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:51:59.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:59 smithi104 ceph-mon[33507]: 1.3 scrub starts 2024-09-11T07:51:59.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:59 smithi104 ceph-mon[33507]: 1.3 scrub ok 2024-09-11T07:51:59.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:59 smithi104 ceph-mon[33507]: from='client.26324 ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-11T07:51:59.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:59 smithi104 ceph-mon[33507]: osdmap e520: 8 total, 8 up, 8 in 2024-09-11T07:51:59.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:59 smithi104 ceph-mon[33507]: 3.1 deep-scrub starts 2024-09-11T07:51:59.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:51:59 smithi104 ceph-mon[33507]: 3.1 deep-scrub ok 2024-09-11T07:51:59.535 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:59 smithi022 ceph-mon[29462]: 1.3 scrub starts 2024-09-11T07:51:59.535 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:59 smithi022 ceph-mon[29462]: 1.3 scrub ok 2024-09-11T07:51:59.535 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:59 smithi022 ceph-mon[29462]: from='client.26324 ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-11T07:51:59.536 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:59 smithi022 ceph-mon[29462]: osdmap e520: 8 total, 8 up, 8 in 2024-09-11T07:51:59.536 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:59 smithi022 ceph-mon[29462]: 3.1 deep-scrub starts 2024-09-11T07:51:59.536 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:51:59 smithi022 ceph-mon[29462]: 3.1 deep-scrub ok 2024-09-11T07:51:59.536 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:59 smithi022 ceph-mon[37330]: 1.3 scrub starts 2024-09-11T07:51:59.536 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:59 smithi022 ceph-mon[37330]: 1.3 scrub ok 2024-09-11T07:51:59.536 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:59 smithi022 ceph-mon[37330]: from='client.26324 ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-11T07:51:59.536 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:59 smithi022 ceph-mon[37330]: osdmap e520: 8 total, 8 up, 8 in 2024-09-11T07:51:59.536 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:59 smithi022 ceph-mon[37330]: 3.1 deep-scrub starts 2024-09-11T07:51:59.536 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:51:59 smithi022 ceph-mon[37330]: 3.1 deep-scrub ok 2024-09-11T07:52:00.416 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:00 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:52:00] "GET /metrics HTTP/1.1" 200 222510 "" "Prometheus/2.43.0" 2024-09-11T07:52:00.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:00 smithi022 ceph-mon[29462]: 2.d scrub starts 2024-09-11T07:52:00.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:00 smithi022 ceph-mon[29462]: 2.d scrub ok 2024-09-11T07:52:00.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:00 smithi022 ceph-mon[29462]: pgmap v1198: 142 pgs: 142 active+clean; 635 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-11T07:52:00.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:00 smithi022 ceph-mon[29462]: 4.1 scrub starts 2024-09-11T07:52:00.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:00 smithi022 ceph-mon[29462]: 4.1 scrub ok 2024-09-11T07:52:00.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:00 smithi022 ceph-mon[37330]: 2.d scrub starts 2024-09-11T07:52:00.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:00 smithi022 ceph-mon[37330]: 2.d scrub ok 2024-09-11T07:52:00.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:00 smithi022 ceph-mon[37330]: pgmap v1198: 142 pgs: 142 active+clean; 635 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-11T07:52:00.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:00 smithi022 ceph-mon[37330]: 4.1 scrub starts 2024-09-11T07:52:00.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:00 smithi022 ceph-mon[37330]: 4.1 scrub ok 2024-09-11T07:52:00.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:52:00 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:52:00.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:00 smithi104 ceph-mon[33507]: 2.d scrub starts 2024-09-11T07:52:00.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:00 smithi104 ceph-mon[33507]: 2.d scrub ok 2024-09-11T07:52:00.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:00 smithi104 ceph-mon[33507]: pgmap v1198: 142 pgs: 142 active+clean; 635 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-11T07:52:00.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:00 smithi104 ceph-mon[33507]: 4.1 scrub starts 2024-09-11T07:52:00.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:00 smithi104 ceph-mon[33507]: 4.1 scrub ok 2024-09-11T07:52:01.259 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:52:01.260 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":521,"fsid":"12a285d8-700e-11ef-bcea-c7b262605968","created":"2024-09-11T07:21:18.695332+0000","modified":"2024-09-11T07:52:01.247637+0000","last_up_change":"2024-09-11T07:50:41.757788+0000","last_in_change":"2024-09-11T07:50:55.883749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T07:24:21.772426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":11,"pg_placement_num":11,"pg_placement_num_target":11,"pg_num_target":11,"pg_num_pending":11,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"450","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"388","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-11T07:26:29.941354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-11T07:26:30.396859+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-11T07:26:31.922926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-11T07:26:34.007746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-11T07:26:36.077215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"09075f16-32f4-4f88-a116-c3d7f977a0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":516,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6801","nonce":2821493591}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6802","nonce":2821493591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6804","nonce":2821493591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6803","nonce":2821493591}]},"public_addr":"172.21.15.22:6801/2821493591","cluster_addr":"172.21.15.22:6802/2821493591","heartbeat_back_addr":"172.21.15.22:6804/2821493591","heartbeat_front_addr":"172.21.15.22:6803/2821493591","state":["exists","up"]},{"osd":1,"uuid":"52cc1e06-9506-48c0-9f7b-d4270dff4b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":509,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6805","nonce":4037911317}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6806","nonce":4037911317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6808","nonce":4037911317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6807","nonce":4037911317}]},"public_addr":"172.21.15.22:6805/4037911317","cluster_addr":"172.21.15.22:6806/4037911317","heartbeat_back_addr":"172.21.15.22:6808/4037911317","heartbeat_front_addr":"172.21.15.22:6807/4037911317","state":["exists","up"]},{"osd":2,"uuid":"fa2b2730-2c1d-45d0-be8a-494a60dc6b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":18,"last_clean_end":469,"up_from":485,"up_thru":516,"down_at":470,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6809","nonce":574977173}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6810","nonce":574977173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6812","nonce":574977173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6811","nonce":574977173}]},"public_addr":"172.21.15.22:6809/574977173","cluster_addr":"172.21.15.22:6810/574977173","heartbeat_back_addr":"172.21.15.22:6812/574977173","heartbeat_front_addr":"172.21.15.22:6811/574977173","state":["exists","up"]},{"osd":3,"uuid":"8d53d3fc-e401-47df-b70b-4b4b1f4539df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":516,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6813","nonce":1501962752}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6814","nonce":1501962752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6816","nonce":1501962752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6815","nonce":1501962752}]},"public_addr":"172.21.15.22:6813/1501962752","cluster_addr":"172.21.15.22:6814/1501962752","heartbeat_back_addr":"172.21.15.22:6816/1501962752","heartbeat_front_addr":"172.21.15.22:6815/1501962752","state":["exists","up"]},{"osd":4,"uuid":"d3207fe0-8b97-4067-a755-ba6d449c53c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":516,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6800","nonce":1210229443}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6801","nonce":1210229443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6803","nonce":1210229443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6802","nonce":1210229443}]},"public_addr":"172.21.15.104:6800/1210229443","cluster_addr":"172.21.15.104:6801/1210229443","heartbeat_back_addr":"172.21.15.104:6803/1210229443","heartbeat_front_addr":"172.21.15.104:6802/1210229443","state":["exists","up"]},{"osd":5,"uuid":"89c9352b-30b2-48c6-869e-047d07a23a43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":516,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6804","nonce":2735328732}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6805","nonce":2735328732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6807","nonce":2735328732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6806","nonce":2735328732}]},"public_addr":"172.21.15.104:6804/2735328732","cluster_addr":"172.21.15.104:6805/2735328732","heartbeat_back_addr":"172.21.15.104:6807/2735328732","heartbeat_front_addr":"172.21.15.104:6806/2735328732","state":["exists","up"]},{"osd":6,"uuid":"704563de-d88f-480a-b469-720135704d42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":516,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6808","nonce":721493010}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6809","nonce":721493010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6811","nonce":721493010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6810","nonce":721493010}]},"public_addr":"172.21.15.104:6808/721493010","cluster_addr":"172.21.15.104:6809/721493010","heartbeat_back_addr":"172.21.15.104:6811/721493010","heartbeat_front_addr":"172.21.15.104:6810/721493010","state":["exists","up"]},{"osd":7,"uuid":"18199e47-0052-49f0-b45f-0fb201e0d0f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":52,"last_clean_end":133,"up_from":147,"up_thru":516,"down_at":134,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6812","nonce":2218863853}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6813","nonce":2218863853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6815","nonce":2218863853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6814","nonce":2218863853}]},"public_addr":"172.21.15.104:6812/2218863853","cluster_addr":"172.21.15.104:6813/2218863853","heartbeat_back_addr":"172.21.15.104:6815/2218863853","heartbeat_front_addr":"172.21.15.104:6814/2218863853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:50:50.572843+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:50:48.782924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:52:00.939955+0000","dead_epoch":469},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:51:11.808837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:50:57.976130+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:50:50.038018+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:50:43.904502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:51:46.231870+0000","dead_epoch":133}],"pg_upmap":[{"pgid":"2.1c","osds":[5,2,7]},{"pgid":"4.1e","osds":[4,5,3]},{"pgid":"4.1f","osds":[3,5,6]}],"pg_upmap_items":[{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.15","mappings":[{"from":3,"to":0}]},{"pgid":"4.16","mappings":[{"from":3,"to":7}]},{"pgid":"5.16","mappings":[{"from":1,"to":6}]},{"pgid":"6.1c","mappings":[{"from":5,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/240999923":"2024-09-12T07:27:07.919998+0000","172.21.15.22:6800/4016369460":"2024-09-12T07:21:48.438371+0000","172.21.15.22:0/585929238":"2024-09-12T07:22:11.238618+0000","172.21.15.22:0/544031024":"2024-09-12T07:27:07.919998+0000","172.21.15.22:6800/2504196132":"2024-09-12T07:27:07.919998+0000","172.21.15.22:0/1451388349":"2024-09-12T07:27:07.919998+0000","172.21.15.22:0/639336081":"2024-09-12T07:21:48.438371+0000","172.21.15.22:0/2104069730":"2024-09-12T07:27:07.919998+0000","172.21.15.22:0/4236450731":"2024-09-12T07:21:48.438371+0000","172.21.15.22:6800/3573193776":"2024-09-12T07:22:11.238618+0000","172.21.15.22:0/2661764422":"2024-09-12T07:21:48.438371+0000","172.21.15.22:0/2197488530":"2024-09-12T07:22:11.238618+0000","172.21.15.22:0/2968034970":"2024-09-12T07:22:11.238618+0000","172.21.15.22:0/599318175":"2024-09-12T07:27:07.919998+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T07:52:01.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:01 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:52:01.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:01 smithi104 ceph-mon[33507]: 1.2 scrub starts 2024-09-11T07:52:01.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:01 smithi104 ceph-mon[33507]: 1.2 scrub ok 2024-09-11T07:52:01.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:01 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:52:01.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:01 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:52:01.666 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:01 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:52:01.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:01 smithi022 ceph-mon[37330]: 1.2 scrub starts 2024-09-11T07:52:01.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:01 smithi022 ceph-mon[37330]: 1.2 scrub ok 2024-09-11T07:52:01.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:01 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:52:01.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:01 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:52:01.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:01 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:52:01.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:01 smithi022 ceph-mon[29462]: 1.2 scrub starts 2024-09-11T07:52:01.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:01 smithi022 ceph-mon[29462]: 1.2 scrub ok 2024-09-11T07:52:01.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:01 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:52:01.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:01 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:52:01.868 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-11T07:52:01.868 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T07:52:02.133 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:52:02.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:02 smithi104 ceph-mon[33507]: pgmap v1199: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-11T07:52:02.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:02 smithi104 ceph-mon[33507]: osdmap e521: 8 total, 8 up, 8 in 2024-09-11T07:52:02.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:02 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/810129692' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:52:02.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:02 smithi104 ceph-mon[33507]: 5.2 scrub starts 2024-09-11T07:52:02.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:02 smithi104 ceph-mon[33507]: 5.2 scrub ok 2024-09-11T07:52:02.556 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:02 smithi022 ceph-mon[37330]: pgmap v1199: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-11T07:52:02.556 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:02 smithi022 ceph-mon[37330]: osdmap e521: 8 total, 8 up, 8 in 2024-09-11T07:52:02.557 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:02 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/810129692' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:52:02.557 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:02 smithi022 ceph-mon[37330]: 5.2 scrub starts 2024-09-11T07:52:02.557 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:02 smithi022 ceph-mon[37330]: 5.2 scrub ok 2024-09-11T07:52:02.557 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:02 smithi022 ceph-mon[29462]: pgmap v1199: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-11T07:52:02.557 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:02 smithi022 ceph-mon[29462]: osdmap e521: 8 total, 8 up, 8 in 2024-09-11T07:52:02.557 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:02 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/810129692' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:52:02.557 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:02 smithi022 ceph-mon[29462]: 5.2 scrub starts 2024-09-11T07:52:02.557 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:02 smithi022 ceph-mon[29462]: 5.2 scrub ok 2024-09-11T07:52:03.408 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:03 smithi022 ceph-mon[37330]: 5.12 deep-scrub starts 2024-09-11T07:52:03.409 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:03 smithi022 ceph-mon[37330]: 5.12 deep-scrub ok 2024-09-11T07:52:03.409 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:03 smithi022 ceph-mon[37330]: 4.10 scrub starts 2024-09-11T07:52:03.409 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:03 smithi022 ceph-mon[37330]: 4.10 scrub ok 2024-09-11T07:52:03.409 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:03 smithi022 ceph-mon[37330]: 6.0 scrub starts 2024-09-11T07:52:03.409 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:03 smithi022 ceph-mon[37330]: 6.0 scrub ok 2024-09-11T07:52:03.409 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:03 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:52:03.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:03 smithi022 ceph-mon[29462]: 5.12 deep-scrub starts 2024-09-11T07:52:03.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:03 smithi022 ceph-mon[29462]: 5.12 deep-scrub ok 2024-09-11T07:52:03.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:03 smithi022 ceph-mon[29462]: 4.10 scrub starts 2024-09-11T07:52:03.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:03 smithi022 ceph-mon[29462]: 4.10 scrub ok 2024-09-11T07:52:03.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:03 smithi022 ceph-mon[29462]: 6.0 scrub starts 2024-09-11T07:52:03.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:03 smithi022 ceph-mon[29462]: 6.0 scrub ok 2024-09-11T07:52:03.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:03 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:52:03.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:03 smithi104 ceph-mon[33507]: 5.12 deep-scrub starts 2024-09-11T07:52:03.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:03 smithi104 ceph-mon[33507]: 5.12 deep-scrub ok 2024-09-11T07:52:03.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:03 smithi104 ceph-mon[33507]: 4.10 scrub starts 2024-09-11T07:52:03.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:03 smithi104 ceph-mon[33507]: 4.10 scrub ok 2024-09-11T07:52:03.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:03 smithi104 ceph-mon[33507]: 6.0 scrub starts 2024-09-11T07:52:03.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:03 smithi104 ceph-mon[33507]: 6.0 scrub ok 2024-09-11T07:52:03.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:03 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:52:04.150 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:52:04.150 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":521,"fsid":"12a285d8-700e-11ef-bcea-c7b262605968","created":"2024-09-11T07:21:18.695332+0000","modified":"2024-09-11T07:52:01.247637+0000","last_up_change":"2024-09-11T07:50:41.757788+0000","last_in_change":"2024-09-11T07:50:55.883749+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T07:24:21.772426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":11,"pg_placement_num":11,"pg_placement_num_target":11,"pg_num_target":11,"pg_num_pending":11,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"450","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"388","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-11T07:26:29.941354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-11T07:26:30.396859+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-11T07:26:31.922926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-11T07:26:34.007746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-11T07:26:36.077215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"09075f16-32f4-4f88-a116-c3d7f977a0e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":516,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6801","nonce":2821493591}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6802","nonce":2821493591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6804","nonce":2821493591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6803","nonce":2821493591}]},"public_addr":"172.21.15.22:6801/2821493591","cluster_addr":"172.21.15.22:6802/2821493591","heartbeat_back_addr":"172.21.15.22:6804/2821493591","heartbeat_front_addr":"172.21.15.22:6803/2821493591","state":["exists","up"]},{"osd":1,"uuid":"52cc1e06-9506-48c0-9f7b-d4270dff4b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":509,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6805","nonce":4037911317}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6806","nonce":4037911317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6808","nonce":4037911317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6807","nonce":4037911317}]},"public_addr":"172.21.15.22:6805/4037911317","cluster_addr":"172.21.15.22:6806/4037911317","heartbeat_back_addr":"172.21.15.22:6808/4037911317","heartbeat_front_addr":"172.21.15.22:6807/4037911317","state":["exists","up"]},{"osd":2,"uuid":"fa2b2730-2c1d-45d0-be8a-494a60dc6b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":18,"last_clean_end":469,"up_from":485,"up_thru":516,"down_at":470,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6809","nonce":574977173}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6810","nonce":574977173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6812","nonce":574977173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6811","nonce":574977173}]},"public_addr":"172.21.15.22:6809/574977173","cluster_addr":"172.21.15.22:6810/574977173","heartbeat_back_addr":"172.21.15.22:6812/574977173","heartbeat_front_addr":"172.21.15.22:6811/574977173","state":["exists","up"]},{"osd":3,"uuid":"8d53d3fc-e401-47df-b70b-4b4b1f4539df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":516,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6813","nonce":1501962752}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6814","nonce":1501962752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6816","nonce":1501962752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.22:6815","nonce":1501962752}]},"public_addr":"172.21.15.22:6813/1501962752","cluster_addr":"172.21.15.22:6814/1501962752","heartbeat_back_addr":"172.21.15.22:6816/1501962752","heartbeat_front_addr":"172.21.15.22:6815/1501962752","state":["exists","up"]},{"osd":4,"uuid":"d3207fe0-8b97-4067-a755-ba6d449c53c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":516,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6800","nonce":1210229443}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6801","nonce":1210229443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6803","nonce":1210229443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6802","nonce":1210229443}]},"public_addr":"172.21.15.104:6800/1210229443","cluster_addr":"172.21.15.104:6801/1210229443","heartbeat_back_addr":"172.21.15.104:6803/1210229443","heartbeat_front_addr":"172.21.15.104:6802/1210229443","state":["exists","up"]},{"osd":5,"uuid":"89c9352b-30b2-48c6-869e-047d07a23a43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":516,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6804","nonce":2735328732}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6805","nonce":2735328732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6807","nonce":2735328732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6806","nonce":2735328732}]},"public_addr":"172.21.15.104:6804/2735328732","cluster_addr":"172.21.15.104:6805/2735328732","heartbeat_back_addr":"172.21.15.104:6807/2735328732","heartbeat_front_addr":"172.21.15.104:6806/2735328732","state":["exists","up"]},{"osd":6,"uuid":"704563de-d88f-480a-b469-720135704d42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":516,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6808","nonce":721493010}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6809","nonce":721493010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6811","nonce":721493010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6810","nonce":721493010}]},"public_addr":"172.21.15.104:6808/721493010","cluster_addr":"172.21.15.104:6809/721493010","heartbeat_back_addr":"172.21.15.104:6811/721493010","heartbeat_front_addr":"172.21.15.104:6810/721493010","state":["exists","up"]},{"osd":7,"uuid":"18199e47-0052-49f0-b45f-0fb201e0d0f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":52,"last_clean_end":133,"up_from":147,"up_thru":516,"down_at":134,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6812","nonce":2218863853}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6813","nonce":2218863853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6815","nonce":2218863853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.104:6814","nonce":2218863853}]},"public_addr":"172.21.15.104:6812/2218863853","cluster_addr":"172.21.15.104:6813/2218863853","heartbeat_back_addr":"172.21.15.104:6815/2218863853","heartbeat_front_addr":"172.21.15.104:6814/2218863853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:50:50.572843+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:50:48.782924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:52:00.939955+0000","dead_epoch":469},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:51:11.808837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:50:57.976130+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:50:50.038018+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:50:43.904502+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:51:46.231870+0000","dead_epoch":133}],"pg_upmap":[{"pgid":"2.1c","osds":[5,2,7]},{"pgid":"4.1e","osds":[4,5,3]},{"pgid":"4.1f","osds":[3,5,6]}],"pg_upmap_items":[{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.15","mappings":[{"from":3,"to":0}]},{"pgid":"4.16","mappings":[{"from":3,"to":7}]},{"pgid":"5.16","mappings":[{"from":1,"to":6}]},{"pgid":"6.1c","mappings":[{"from":5,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/240999923":"2024-09-12T07:27:07.919998+0000","172.21.15.22:6800/4016369460":"2024-09-12T07:21:48.438371+0000","172.21.15.22:0/585929238":"2024-09-12T07:22:11.238618+0000","172.21.15.22:0/544031024":"2024-09-12T07:27:07.919998+0000","172.21.15.22:6800/2504196132":"2024-09-12T07:27:07.919998+0000","172.21.15.22:0/1451388349":"2024-09-12T07:27:07.919998+0000","172.21.15.22:0/639336081":"2024-09-12T07:21:48.438371+0000","172.21.15.22:0/2104069730":"2024-09-12T07:27:07.919998+0000","172.21.15.22:0/4236450731":"2024-09-12T07:21:48.438371+0000","172.21.15.22:6800/3573193776":"2024-09-12T07:22:11.238618+0000","172.21.15.22:0/2661764422":"2024-09-12T07:21:48.438371+0000","172.21.15.22:0/2197488530":"2024-09-12T07:22:11.238618+0000","172.21.15.22:0/2968034970":"2024-09-12T07:22:11.238618+0000","172.21.15.22:0/599318175":"2024-09-12T07:27:07.919998+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T07:52:04.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:04 smithi022 ceph-mon[29462]: pgmap v1201: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-11T07:52:04.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:04 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:52:04.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:04 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:52:04.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:04 smithi022 ceph-mon[29462]: pgmap v1202: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-11T07:52:04.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:04 smithi022 ceph-mon[29462]: pgmap v1203: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-11T07:52:04.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:04 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:52:04.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:04 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1819705328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:52:04.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:04 smithi022 ceph-mon[37330]: pgmap v1201: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-11T07:52:04.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:04 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:52:04.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:04 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:52:04.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:04 smithi022 ceph-mon[37330]: pgmap v1202: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-11T07:52:04.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:04 smithi022 ceph-mon[37330]: pgmap v1203: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-11T07:52:04.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:04 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:52:04.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:04 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1819705328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:52:04.529 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:52:04 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:52:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:52:04.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:04 smithi104 ceph-mon[33507]: pgmap v1201: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-11T07:52:04.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:04 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:52:04.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:04 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:52:04.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:04 smithi104 ceph-mon[33507]: pgmap v1202: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-11T07:52:04.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:04 smithi104 ceph-mon[33507]: pgmap v1203: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-11T07:52:04.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:04 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:52:04.530 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:04 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1819705328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:52:04.778 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-11T07:52:04.778 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-11T07:52:04.779 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-11T07:52:04.779 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-11T07:52:04.780 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-11T07:52:04.780 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-11T07:52:04.781 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-11T07:52:04.781 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-11T07:52:05.256 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:52:05.283 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:52:05.331 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:52:05.331 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:52:05.335 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:52:05.335 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:52:05.337 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:52:05.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:05 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:52:05.529 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:05 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:52:05.635 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:05 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:52:05.636 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:05 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:52:05.636 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:05 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:52:05.636 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:05 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:52:06.492 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:06 smithi022 ceph-mon[37330]: osdmap e522: 8 total, 8 up, 8 in 2024-09-11T07:52:06.492 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:06 smithi022 ceph-mon[37330]: pgmap v1205: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:52:06.492 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:06 smithi022 ceph-mon[29462]: osdmap e522: 8 total, 8 up, 8 in 2024-09-11T07:52:06.492 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:06 smithi022 ceph-mon[29462]: pgmap v1205: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:52:06.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:06 smithi104 ceph-mon[33507]: osdmap e522: 8 total, 8 up, 8 in 2024-09-11T07:52:06.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:06 smithi104 ceph-mon[33507]: pgmap v1205: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:52:07.560 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:07 smithi022 ceph-mon[29462]: 5.c deep-scrub starts 2024-09-11T07:52:07.560 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:07 smithi022 ceph-mon[29462]: 5.c deep-scrub ok 2024-09-11T07:52:07.561 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:07 smithi022 ceph-mon[29462]: 5.b scrub starts 2024-09-11T07:52:07.561 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:07 smithi022 ceph-mon[29462]: 5.b scrub ok 2024-09-11T07:52:07.561 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:07 smithi022 ceph-mon[37330]: 5.c deep-scrub starts 2024-09-11T07:52:07.561 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:07 smithi022 ceph-mon[37330]: 5.c deep-scrub ok 2024-09-11T07:52:07.561 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:07 smithi022 ceph-mon[37330]: 5.b scrub starts 2024-09-11T07:52:07.561 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:07 smithi022 ceph-mon[37330]: 5.b scrub ok 2024-09-11T07:52:07.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:07 smithi104 ceph-mon[33507]: 5.c deep-scrub starts 2024-09-11T07:52:07.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:07 smithi104 ceph-mon[33507]: 5.c deep-scrub ok 2024-09-11T07:52:07.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:07 smithi104 ceph-mon[33507]: 5.b scrub starts 2024-09-11T07:52:07.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:07 smithi104 ceph-mon[33507]: 5.b scrub ok 2024-09-11T07:52:08.557 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:08 smithi022 ceph-mon[29462]: 6.4 scrub starts 2024-09-11T07:52:08.558 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:08 smithi022 ceph-mon[29462]: 6.4 scrub ok 2024-09-11T07:52:08.558 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:08 smithi022 ceph-mon[29462]: 6.19 scrub starts 2024-09-11T07:52:08.558 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:08 smithi022 ceph-mon[29462]: 6.19 scrub ok 2024-09-11T07:52:08.558 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:08 smithi022 ceph-mon[29462]: pgmap v1206: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 811 B/s rd, 0 op/s 2024-09-11T07:52:08.558 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:08 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:52:08.558 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:08 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:52:08.558 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:08 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:52:08.558 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:08 smithi022 ceph-mon[29462]: 5.6 deep-scrub starts 2024-09-11T07:52:08.558 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:08 smithi022 ceph-mon[29462]: 5.6 deep-scrub ok 2024-09-11T07:52:08.558 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:08 smithi022 ceph-mon[37330]: 6.4 scrub starts 2024-09-11T07:52:08.558 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:08 smithi022 ceph-mon[37330]: 6.4 scrub ok 2024-09-11T07:52:08.559 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:08 smithi022 ceph-mon[37330]: 6.19 scrub starts 2024-09-11T07:52:08.559 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:08 smithi022 ceph-mon[37330]: 6.19 scrub ok 2024-09-11T07:52:08.559 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:08 smithi022 ceph-mon[37330]: pgmap v1206: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 811 B/s rd, 0 op/s 2024-09-11T07:52:08.559 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:08 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:52:08.559 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:08 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:52:08.559 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:08 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:52:08.559 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:08 smithi022 ceph-mon[37330]: 5.6 deep-scrub starts 2024-09-11T07:52:08.559 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:08 smithi022 ceph-mon[37330]: 5.6 deep-scrub ok 2024-09-11T07:52:08.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:08 smithi104 ceph-mon[33507]: 6.4 scrub starts 2024-09-11T07:52:08.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:08 smithi104 ceph-mon[33507]: 6.4 scrub ok 2024-09-11T07:52:08.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:08 smithi104 ceph-mon[33507]: 6.19 scrub starts 2024-09-11T07:52:08.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:08 smithi104 ceph-mon[33507]: 6.19 scrub ok 2024-09-11T07:52:08.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:08 smithi104 ceph-mon[33507]: pgmap v1206: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 811 B/s rd, 0 op/s 2024-09-11T07:52:08.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:08 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:52:08.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:08 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:52:08.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:08 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:52:08.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:08 smithi104 ceph-mon[33507]: 5.6 deep-scrub starts 2024-09-11T07:52:08.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:08 smithi104 ceph-mon[33507]: 5.6 deep-scrub ok 2024-09-11T07:52:09.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:09 smithi022 ceph-mon[37330]: 4.4 deep-scrub starts 2024-09-11T07:52:09.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:09 smithi022 ceph-mon[37330]: 4.4 deep-scrub ok 2024-09-11T07:52:09.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:09 smithi022 ceph-mon[37330]: osdmap e523: 8 total, 8 up, 8 in 2024-09-11T07:52:09.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:09 smithi022 ceph-mon[37330]: 6.18 scrub starts 2024-09-11T07:52:09.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:09 smithi022 ceph-mon[37330]: 6.18 scrub ok 2024-09-11T07:52:09.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:09 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:52:09.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:09 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:52:09.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:09 smithi022 ceph-mon[29462]: 4.4 deep-scrub starts 2024-09-11T07:52:09.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:09 smithi022 ceph-mon[29462]: 4.4 deep-scrub ok 2024-09-11T07:52:09.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:09 smithi022 ceph-mon[29462]: osdmap e523: 8 total, 8 up, 8 in 2024-09-11T07:52:09.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:09 smithi022 ceph-mon[29462]: 6.18 scrub starts 2024-09-11T07:52:09.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:09 smithi022 ceph-mon[29462]: 6.18 scrub ok 2024-09-11T07:52:09.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:09 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:52:09.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:09 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:52:09.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:09 smithi104 ceph-mon[33507]: 4.4 deep-scrub starts 2024-09-11T07:52:09.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:09 smithi104 ceph-mon[33507]: 4.4 deep-scrub ok 2024-09-11T07:52:09.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:09 smithi104 ceph-mon[33507]: osdmap e523: 8 total, 8 up, 8 in 2024-09-11T07:52:09.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:09 smithi104 ceph-mon[33507]: 6.18 scrub starts 2024-09-11T07:52:09.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:09 smithi104 ceph-mon[33507]: 6.18 scrub ok 2024-09-11T07:52:09.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:09 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:52:09.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:09 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:52:10.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:10 smithi022 ceph-mon[29462]: osdmap e524: 8 total, 8 up, 8 in 2024-09-11T07:52:10.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:10 smithi022 ceph-mon[29462]: pgmap v1209: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T07:52:10.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:10 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:52:10] "GET /metrics HTTP/1.1" 200 222460 "" "Prometheus/2.43.0" 2024-09-11T07:52:10.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:10 smithi022 ceph-mon[37330]: osdmap e524: 8 total, 8 up, 8 in 2024-09-11T07:52:10.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:10 smithi022 ceph-mon[37330]: pgmap v1209: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T07:52:10.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:52:10 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:52:10.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:10 smithi104 ceph-mon[33507]: osdmap e524: 8 total, 8 up, 8 in 2024-09-11T07:52:10.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:10 smithi104 ceph-mon[33507]: pgmap v1209: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T07:52:11.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:11 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:52:11.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:11 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:52:11.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:11 smithi022 ceph-mon[29462]: 5.e scrub starts 2024-09-11T07:52:11.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:11 smithi022 ceph-mon[29462]: 5.e scrub ok 2024-09-11T07:52:11.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:11 smithi022 ceph-mon[29462]: osdmap e525: 8 total, 8 up, 8 in 2024-09-11T07:52:11.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:11 smithi022 ceph-mon[29462]: 6.8 deep-scrub starts 2024-09-11T07:52:11.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:11 smithi022 ceph-mon[29462]: 6.8 deep-scrub ok 2024-09-11T07:52:11.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:11 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:52:11.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:11 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:52:11.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:11 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:52:11.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:11 smithi022 ceph-mon[37330]: 5.e scrub starts 2024-09-11T07:52:11.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:11 smithi022 ceph-mon[37330]: 5.e scrub ok 2024-09-11T07:52:11.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:11 smithi022 ceph-mon[37330]: osdmap e525: 8 total, 8 up, 8 in 2024-09-11T07:52:11.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:11 smithi022 ceph-mon[37330]: 6.8 deep-scrub starts 2024-09-11T07:52:11.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:11 smithi022 ceph-mon[37330]: 6.8 deep-scrub ok 2024-09-11T07:52:11.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:11 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:52:11.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:11 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:52:11.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:11 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:52:11.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:11 smithi104 ceph-mon[33507]: 5.e scrub starts 2024-09-11T07:52:11.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:11 smithi104 ceph-mon[33507]: 5.e scrub ok 2024-09-11T07:52:11.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:11 smithi104 ceph-mon[33507]: osdmap e525: 8 total, 8 up, 8 in 2024-09-11T07:52:11.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:11 smithi104 ceph-mon[33507]: 6.8 deep-scrub starts 2024-09-11T07:52:11.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:11 smithi104 ceph-mon[33507]: 6.8 deep-scrub ok 2024-09-11T07:52:11.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:11 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:52:12.577 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:12 smithi022 ceph-mon[29462]: pgmap v1211: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:52:12.577 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:12 smithi022 ceph-mon[29462]: 2.16 scrub starts 2024-09-11T07:52:12.577 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:12 smithi022 ceph-mon[29462]: 2.16 scrub ok 2024-09-11T07:52:12.577 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:12 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2024-09-11T07:52:12.578 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:12 smithi022 ceph-mon[37330]: pgmap v1211: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:52:12.578 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:12 smithi022 ceph-mon[37330]: 2.16 scrub starts 2024-09-11T07:52:12.578 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:12 smithi022 ceph-mon[37330]: 2.16 scrub ok 2024-09-11T07:52:12.578 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:12 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2024-09-11T07:52:12.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:12 smithi104 ceph-mon[33507]: pgmap v1211: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:52:12.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:12 smithi104 ceph-mon[33507]: 2.16 scrub starts 2024-09-11T07:52:12.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:12 smithi104 ceph-mon[33507]: 2.16 scrub ok 2024-09-11T07:52:12.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:12 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2024-09-11T07:52:13.214 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:52:13.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:13 smithi022 ceph-mon[29462]: 6.1d deep-scrub starts 2024-09-11T07:52:13.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:13 smithi022 ceph-mon[29462]: 6.1d deep-scrub ok 2024-09-11T07:52:13.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:13 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]': finished 2024-09-11T07:52:13.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:13 smithi022 ceph-mon[29462]: osdmap e526: 8 total, 8 up, 8 in 2024-09-11T07:52:13.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:13 smithi022 ceph-mon[29462]: 6.1f scrub starts 2024-09-11T07:52:13.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:13 smithi022 ceph-mon[29462]: 6.1f scrub ok 2024-09-11T07:52:13.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:13 smithi022 ceph-mon[37330]: 6.1d deep-scrub starts 2024-09-11T07:52:13.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:13 smithi022 ceph-mon[37330]: 6.1d deep-scrub ok 2024-09-11T07:52:13.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:13 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]': finished 2024-09-11T07:52:13.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:13 smithi022 ceph-mon[37330]: osdmap e526: 8 total, 8 up, 8 in 2024-09-11T07:52:13.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:13 smithi022 ceph-mon[37330]: 6.1f scrub starts 2024-09-11T07:52:13.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:13 smithi022 ceph-mon[37330]: 6.1f scrub ok 2024-09-11T07:52:13.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:13 smithi104 ceph-mon[33507]: 6.1d deep-scrub starts 2024-09-11T07:52:13.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:13 smithi104 ceph-mon[33507]: 6.1d deep-scrub ok 2024-09-11T07:52:13.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:13 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]': finished 2024-09-11T07:52:13.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:13 smithi104 ceph-mon[33507]: osdmap e526: 8 total, 8 up, 8 in 2024-09-11T07:52:13.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:13 smithi104 ceph-mon[33507]: 6.1f scrub starts 2024-09-11T07:52:13.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:13 smithi104 ceph-mon[33507]: 6.1f scrub ok 2024-09-11T07:52:14.312 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:52:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:52:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:52:14.514 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:14 smithi022 ceph-mon[29462]: osdmap e527: 8 total, 8 up, 8 in 2024-09-11T07:52:14.515 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:14 smithi022 ceph-mon[29462]: 4.19 scrub starts 2024-09-11T07:52:14.515 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:14 smithi022 ceph-mon[29462]: 4.19 scrub ok 2024-09-11T07:52:14.515 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:14 smithi022 ceph-mon[29462]: pgmap v1214: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-11T07:52:14.515 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:14 smithi022 ceph-mon[37330]: osdmap e527: 8 total, 8 up, 8 in 2024-09-11T07:52:14.515 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:14 smithi022 ceph-mon[37330]: 4.19 scrub starts 2024-09-11T07:52:14.515 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:14 smithi022 ceph-mon[37330]: 4.19 scrub ok 2024-09-11T07:52:14.515 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:14 smithi022 ceph-mon[37330]: pgmap v1214: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-11T07:52:14.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:14 smithi104 ceph-mon[33507]: osdmap e527: 8 total, 8 up, 8 in 2024-09-11T07:52:14.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:14 smithi104 ceph-mon[33507]: 4.19 scrub starts 2024-09-11T07:52:14.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:14 smithi104 ceph-mon[33507]: 4.19 scrub ok 2024-09-11T07:52:14.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:14 smithi104 ceph-mon[33507]: pgmap v1214: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-11T07:52:15.511 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:15 smithi022 ceph-mon[29462]: 5.15 scrub starts 2024-09-11T07:52:15.511 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:15 smithi022 ceph-mon[29462]: 5.15 scrub ok 2024-09-11T07:52:15.512 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:15 smithi022 ceph-mon[37330]: 5.15 scrub starts 2024-09-11T07:52:15.512 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:15 smithi022 ceph-mon[37330]: 5.15 scrub ok 2024-09-11T07:52:15.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:15 smithi104 ceph-mon[33507]: 5.15 scrub starts 2024-09-11T07:52:15.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:15 smithi104 ceph-mon[33507]: 5.15 scrub ok 2024-09-11T07:52:16.329 INFO:teuthology.orchestra.run.smithi022.stdout:34359738714 2024-09-11T07:52:16.330 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-11T07:52:16.630 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:16 smithi022 ceph-mon[37330]: pgmap v1215: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:52:16.630 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:16 smithi022 ceph-mon[37330]: 2.1c scrub starts 2024-09-11T07:52:16.631 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:16 smithi022 ceph-mon[37330]: 2.1c scrub ok 2024-09-11T07:52:16.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:16 smithi022 ceph-mon[29462]: pgmap v1215: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:52:16.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:16 smithi022 ceph-mon[29462]: 2.1c scrub starts 2024-09-11T07:52:16.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:16 smithi022 ceph-mon[29462]: 2.1c scrub ok 2024-09-11T07:52:16.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:16 smithi104 ceph-mon[33507]: pgmap v1215: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:52:16.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:16 smithi104 ceph-mon[33507]: 2.1c scrub starts 2024-09-11T07:52:16.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:16 smithi104 ceph-mon[33507]: 2.1c scrub ok 2024-09-11T07:52:17.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:17 smithi022 ceph-mon[29462]: 6.e scrub starts 2024-09-11T07:52:17.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:17 smithi022 ceph-mon[29462]: 6.e scrub ok 2024-09-11T07:52:17.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:17 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:52:17.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:17 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:52:17.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:17 smithi022 ceph-mon[37330]: 6.e scrub starts 2024-09-11T07:52:17.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:17 smithi022 ceph-mon[37330]: 6.e scrub ok 2024-09-11T07:52:17.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:17 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:52:17.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:17 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:52:17.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:17 smithi104 ceph-mon[33507]: 6.e scrub starts 2024-09-11T07:52:17.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:17 smithi104 ceph-mon[33507]: 6.e scrub ok 2024-09-11T07:52:17.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:17 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:52:17.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:17 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:52:18.142 INFO:teuthology.orchestra.run.smithi022.stdout:107374182732 2024-09-11T07:52:18.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-11T07:52:18.377 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:18 smithi022 ceph-mon[29462]: osdmap e528: 8 total, 8 up, 8 in 2024-09-11T07:52:18.377 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:18 smithi022 ceph-mon[29462]: pgmap v1217: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:52:18.377 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:18 smithi022 ceph-mon[29462]: 4.d scrub starts 2024-09-11T07:52:18.378 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:18 smithi022 ceph-mon[37330]: osdmap e528: 8 total, 8 up, 8 in 2024-09-11T07:52:18.378 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:18 smithi022 ceph-mon[37330]: pgmap v1217: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:52:18.378 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:18 smithi022 ceph-mon[37330]: 4.d scrub starts 2024-09-11T07:52:18.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:18 smithi104 ceph-mon[33507]: osdmap e528: 8 total, 8 up, 8 in 2024-09-11T07:52:18.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:18 smithi104 ceph-mon[33507]: pgmap v1217: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:52:18.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:18 smithi104 ceph-mon[33507]: 4.d scrub starts 2024-09-11T07:52:19.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:19 smithi022 ceph-mon[29462]: 4.d scrub ok 2024-09-11T07:52:19.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:19 smithi022 ceph-mon[29462]: 5.d deep-scrub starts 2024-09-11T07:52:19.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:19 smithi022 ceph-mon[29462]: 5.d deep-scrub ok 2024-09-11T07:52:19.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:19 smithi022 ceph-mon[37330]: 4.d scrub ok 2024-09-11T07:52:19.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:19 smithi022 ceph-mon[37330]: 5.d deep-scrub starts 2024-09-11T07:52:19.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:19 smithi022 ceph-mon[37330]: 5.d deep-scrub ok 2024-09-11T07:52:19.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:19 smithi104 ceph-mon[33507]: 4.d scrub ok 2024-09-11T07:52:19.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:19 smithi104 ceph-mon[33507]: 5.d deep-scrub starts 2024-09-11T07:52:19.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:19 smithi104 ceph-mon[33507]: 5.d deep-scrub ok 2024-09-11T07:52:20.450 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:20 smithi022 ceph-mon[29462]: pgmap v1218: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T07:52:20.450 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:20 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:52:20] "GET /metrics HTTP/1.1" 200 222460 "" "Prometheus/2.43.0" 2024-09-11T07:52:20.450 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:20 smithi022 ceph-mon[37330]: pgmap v1218: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T07:52:20.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:20 smithi104 ceph-mon[33507]: pgmap v1218: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T07:52:20.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:52:20 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:52:20.875 INFO:teuthology.orchestra.run.smithi022.stdout:2083059138582 2024-09-11T07:52:20.876 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-11T07:52:21.396 INFO:teuthology.orchestra.run.smithi022.stdout:158913790276 2024-09-11T07:52:21.397 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-11T07:52:21.641 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:21 smithi022 ceph-mon[29462]: 6.1b scrub starts 2024-09-11T07:52:21.641 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:21 smithi022 ceph-mon[29462]: 6.1b scrub ok 2024-09-11T07:52:21.641 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:21 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:52:21.641 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:21 smithi022 ceph-mon[29462]: osdmap e529: 8 total, 8 up, 8 in 2024-09-11T07:52:21.641 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:21 smithi022 ceph-mon[37330]: 6.1b scrub starts 2024-09-11T07:52:21.641 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:21 smithi022 ceph-mon[37330]: 6.1b scrub ok 2024-09-11T07:52:21.641 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:21 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:52:21.642 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:21 smithi022 ceph-mon[37330]: osdmap e529: 8 total, 8 up, 8 in 2024-09-11T07:52:21.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:21 smithi104 ceph-mon[33507]: 6.1b scrub starts 2024-09-11T07:52:21.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:21 smithi104 ceph-mon[33507]: 6.1b scrub ok 2024-09-11T07:52:21.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:21 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:52:21.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:21 smithi104 ceph-mon[33507]: osdmap e529: 8 total, 8 up, 8 in 2024-09-11T07:52:22.339 INFO:teuthology.orchestra.run.smithi022.stdout:55834575189 2024-09-11T07:52:22.339 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-11T07:52:22.647 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:22 smithi022 ceph-mon[29462]: 4.18 scrub starts 2024-09-11T07:52:22.647 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:22 smithi022 ceph-mon[29462]: 4.18 scrub ok 2024-09-11T07:52:22.647 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:22 smithi022 ceph-mon[29462]: 6.13 scrub starts 2024-09-11T07:52:22.648 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:22 smithi022 ceph-mon[29462]: 6.13 scrub ok 2024-09-11T07:52:22.648 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:22 smithi022 ceph-mon[29462]: pgmap v1220: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:52:22.648 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:22 smithi022 ceph-mon[29462]: 2.1f scrub starts 2024-09-11T07:52:22.648 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:22 smithi022 ceph-mon[29462]: 2.1f scrub ok 2024-09-11T07:52:22.648 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:22 smithi022 ceph-mon[37330]: 4.18 scrub starts 2024-09-11T07:52:22.648 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:22 smithi022 ceph-mon[37330]: 4.18 scrub ok 2024-09-11T07:52:22.648 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:22 smithi022 ceph-mon[37330]: 6.13 scrub starts 2024-09-11T07:52:22.648 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:22 smithi022 ceph-mon[37330]: 6.13 scrub ok 2024-09-11T07:52:22.648 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:22 smithi022 ceph-mon[37330]: pgmap v1220: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:52:22.649 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:22 smithi022 ceph-mon[37330]: 2.1f scrub starts 2024-09-11T07:52:22.649 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:22 smithi022 ceph-mon[37330]: 2.1f scrub ok 2024-09-11T07:52:22.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:22 smithi104 ceph-mon[33507]: 4.18 scrub starts 2024-09-11T07:52:22.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:22 smithi104 ceph-mon[33507]: 4.18 scrub ok 2024-09-11T07:52:22.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:22 smithi104 ceph-mon[33507]: 6.13 scrub starts 2024-09-11T07:52:22.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:22 smithi104 ceph-mon[33507]: 6.13 scrub ok 2024-09-11T07:52:22.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:22 smithi104 ceph-mon[33507]: pgmap v1220: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:52:22.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:22 smithi104 ceph-mon[33507]: 2.1f scrub starts 2024-09-11T07:52:22.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:22 smithi104 ceph-mon[33507]: 2.1f scrub ok 2024-09-11T07:52:22.939 INFO:teuthology.orchestra.run.smithi022.stdout:631360192758 2024-09-11T07:52:22.940 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-11T07:52:23.391 INFO:teuthology.orchestra.run.smithi022.stdout:133143986504 2024-09-11T07:52:23.392 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-11T07:52:23.666 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:23 smithi022 ceph-mon[37330]: purged_snaps scrub starts 2024-09-11T07:52:23.666 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:23 smithi022 ceph-mon[37330]: purged_snaps scrub ok 2024-09-11T07:52:23.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:23 smithi022 ceph-mon[37330]: osdmap e530: 8 total, 8 up, 8 in 2024-09-11T07:52:23.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:23 smithi022 ceph-mon[37330]: 5.8 scrub starts 2024-09-11T07:52:23.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:23 smithi022 ceph-mon[37330]: 5.8 scrub ok 2024-09-11T07:52:23.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:23 smithi022 ceph-mon[37330]: 5.14 scrub starts 2024-09-11T07:52:23.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:23 smithi022 ceph-mon[37330]: 5.14 scrub ok 2024-09-11T07:52:23.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:23 smithi022 ceph-mon[37330]: 4.6 deep-scrub starts 2024-09-11T07:52:23.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:23 smithi022 ceph-mon[37330]: 4.6 deep-scrub ok 2024-09-11T07:52:23.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:23 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:52:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:23 smithi022 ceph-mon[29462]: purged_snaps scrub starts 2024-09-11T07:52:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:23 smithi022 ceph-mon[29462]: purged_snaps scrub ok 2024-09-11T07:52:23.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:23 smithi022 ceph-mon[29462]: osdmap e530: 8 total, 8 up, 8 in 2024-09-11T07:52:23.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:23 smithi022 ceph-mon[29462]: 5.8 scrub starts 2024-09-11T07:52:23.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:23 smithi022 ceph-mon[29462]: 5.8 scrub ok 2024-09-11T07:52:23.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:23 smithi022 ceph-mon[29462]: 5.14 scrub starts 2024-09-11T07:52:23.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:23 smithi022 ceph-mon[29462]: 5.14 scrub ok 2024-09-11T07:52:23.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:23 smithi022 ceph-mon[29462]: 4.6 deep-scrub starts 2024-09-11T07:52:23.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:23 smithi022 ceph-mon[29462]: 4.6 deep-scrub ok 2024-09-11T07:52:23.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:23 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:52:23.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:23 smithi104 ceph-mon[33507]: purged_snaps scrub starts 2024-09-11T07:52:23.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:23 smithi104 ceph-mon[33507]: purged_snaps scrub ok 2024-09-11T07:52:23.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:23 smithi104 ceph-mon[33507]: osdmap e530: 8 total, 8 up, 8 in 2024-09-11T07:52:23.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:23 smithi104 ceph-mon[33507]: 5.8 scrub starts 2024-09-11T07:52:23.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:23 smithi104 ceph-mon[33507]: 5.8 scrub ok 2024-09-11T07:52:23.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:23 smithi104 ceph-mon[33507]: 5.14 scrub starts 2024-09-11T07:52:23.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:23 smithi104 ceph-mon[33507]: 5.14 scrub ok 2024-09-11T07:52:23.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:23 smithi104 ceph-mon[33507]: 4.6 deep-scrub starts 2024-09-11T07:52:23.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:23 smithi104 ceph-mon[33507]: 4.6 deep-scrub ok 2024-09-11T07:52:23.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:23 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:52:23.838 INFO:teuthology.orchestra.run.smithi022.stdout:188978561342 2024-09-11T07:52:23.838 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-11T07:52:23.872 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:52:23.921 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:52:23.923 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:52:23.940 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:52:23.947 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:52:24.180 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:52:24.346 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:52:24 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:52:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:52:24.433 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:24 smithi022 ceph-mon[29462]: pgmap v1222: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 819 B/s rd, 0 op/s 2024-09-11T07:52:24.434 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:24 smithi022 ceph-mon[37330]: pgmap v1222: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 819 B/s rd, 0 op/s 2024-09-11T07:52:24.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:24 smithi104 ceph-mon[33507]: pgmap v1222: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 819 B/s rd, 0 op/s 2024-09-11T07:52:25.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:25 smithi022 ceph-mon[29462]: 5.1a deep-scrub starts 2024-09-11T07:52:25.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:25 smithi022 ceph-mon[29462]: 5.1a deep-scrub ok 2024-09-11T07:52:25.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:25 smithi022 ceph-mon[29462]: 6.b scrub starts 2024-09-11T07:52:25.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:25 smithi022 ceph-mon[29462]: 6.b scrub ok 2024-09-11T07:52:25.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:25 smithi022 ceph-mon[29462]: 2.14 scrub starts 2024-09-11T07:52:25.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:25 smithi022 ceph-mon[29462]: 2.14 scrub ok 2024-09-11T07:52:25.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:25 smithi022 ceph-mon[37330]: 5.1a deep-scrub starts 2024-09-11T07:52:25.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:25 smithi022 ceph-mon[37330]: 5.1a deep-scrub ok 2024-09-11T07:52:25.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:25 smithi022 ceph-mon[37330]: 6.b scrub starts 2024-09-11T07:52:25.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:25 smithi022 ceph-mon[37330]: 6.b scrub ok 2024-09-11T07:52:25.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:25 smithi022 ceph-mon[37330]: 2.14 scrub starts 2024-09-11T07:52:25.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:25 smithi022 ceph-mon[37330]: 2.14 scrub ok 2024-09-11T07:52:25.682 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:52:25.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:25 smithi104 ceph-mon[33507]: 5.1a deep-scrub starts 2024-09-11T07:52:25.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:25 smithi104 ceph-mon[33507]: 5.1a deep-scrub ok 2024-09-11T07:52:25.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:25 smithi104 ceph-mon[33507]: 6.b scrub starts 2024-09-11T07:52:25.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:25 smithi104 ceph-mon[33507]: 6.b scrub ok 2024-09-11T07:52:25.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:25 smithi104 ceph-mon[33507]: 2.14 scrub starts 2024-09-11T07:52:25.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:25 smithi104 ceph-mon[33507]: 2.14 scrub ok 2024-09-11T07:52:26.353 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:26 smithi022 ceph-mon[29462]: 4.7 scrub starts 2024-09-11T07:52:26.353 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:26 smithi022 ceph-mon[29462]: 4.7 scrub ok 2024-09-11T07:52:26.353 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:26 smithi022 ceph-mon[29462]: 2.f scrub starts 2024-09-11T07:52:26.354 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:26 smithi022 ceph-mon[29462]: 2.f scrub ok 2024-09-11T07:52:26.354 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:26 smithi022 ceph-mon[29462]: 5.0 scrub starts 2024-09-11T07:52:26.354 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:26 smithi022 ceph-mon[29462]: 5.0 scrub ok 2024-09-11T07:52:26.354 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:26 smithi022 ceph-mon[29462]: pgmap v1223: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:52:26.354 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:26 smithi022 ceph-mon[29462]: osdmap e531: 8 total, 8 up, 8 in 2024-09-11T07:52:26.354 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:26 smithi022 ceph-mon[37330]: 4.7 scrub starts 2024-09-11T07:52:26.354 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:26 smithi022 ceph-mon[37330]: 4.7 scrub ok 2024-09-11T07:52:26.354 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:26 smithi022 ceph-mon[37330]: 2.f scrub starts 2024-09-11T07:52:26.354 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:26 smithi022 ceph-mon[37330]: 2.f scrub ok 2024-09-11T07:52:26.354 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:26 smithi022 ceph-mon[37330]: 5.0 scrub starts 2024-09-11T07:52:26.354 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:26 smithi022 ceph-mon[37330]: 5.0 scrub ok 2024-09-11T07:52:26.355 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:26 smithi022 ceph-mon[37330]: pgmap v1223: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:52:26.355 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:26 smithi022 ceph-mon[37330]: osdmap e531: 8 total, 8 up, 8 in 2024-09-11T07:52:26.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:26 smithi104 ceph-mon[33507]: 4.7 scrub starts 2024-09-11T07:52:26.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:26 smithi104 ceph-mon[33507]: 4.7 scrub ok 2024-09-11T07:52:26.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:26 smithi104 ceph-mon[33507]: 2.f scrub starts 2024-09-11T07:52:26.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:26 smithi104 ceph-mon[33507]: 2.f scrub ok 2024-09-11T07:52:26.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:26 smithi104 ceph-mon[33507]: 5.0 scrub starts 2024-09-11T07:52:26.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:26 smithi104 ceph-mon[33507]: 5.0 scrub ok 2024-09-11T07:52:26.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:26 smithi104 ceph-mon[33507]: pgmap v1223: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:52:26.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:26 smithi104 ceph-mon[33507]: osdmap e531: 8 total, 8 up, 8 in 2024-09-11T07:52:27.396 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:27 smithi022 ceph-mon[37330]: 5.9 scrub starts 2024-09-11T07:52:27.397 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:27 smithi022 ceph-mon[37330]: 5.9 scrub ok 2024-09-11T07:52:27.397 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:27 smithi022 ceph-mon[29462]: 5.9 scrub starts 2024-09-11T07:52:27.397 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:27 smithi022 ceph-mon[29462]: 5.9 scrub ok 2024-09-11T07:52:27.501 INFO:teuthology.orchestra.run.smithi022.stdout:55834575191 2024-09-11T07:52:27.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:27 smithi104 ceph-mon[33507]: 5.9 scrub starts 2024-09-11T07:52:27.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:27 smithi104 ceph-mon[33507]: 5.9 scrub ok 2024-09-11T07:52:28.450 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834575189 got 55834575191 for osd.1 2024-09-11T07:52:28.450 DEBUG:teuthology.parallel:result is None 2024-09-11T07:52:28.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:28 smithi022 ceph-mon[29462]: 1.5 scrub starts 2024-09-11T07:52:28.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:28 smithi022 ceph-mon[29462]: 1.5 scrub ok 2024-09-11T07:52:28.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:28 smithi022 ceph-mon[29462]: 3.2 scrub starts 2024-09-11T07:52:28.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:28 smithi022 ceph-mon[29462]: 3.2 scrub ok 2024-09-11T07:52:28.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:28 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/88037589' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T07:52:28.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:28 smithi022 ceph-mon[29462]: pgmap v1225: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 762 B/s rd, 0 op/s 2024-09-11T07:52:28.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:28 smithi022 ceph-mon[29462]: 6.7 scrub starts 2024-09-11T07:52:28.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:28 smithi022 ceph-mon[29462]: 6.7 scrub ok 2024-09-11T07:52:28.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:28 smithi022 ceph-mon[29462]: 4.5 scrub starts 2024-09-11T07:52:28.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:28 smithi022 ceph-mon[29462]: 4.5 scrub ok 2024-09-11T07:52:28.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:28 smithi022 ceph-mon[37330]: 1.5 scrub starts 2024-09-11T07:52:28.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:28 smithi022 ceph-mon[37330]: 1.5 scrub ok 2024-09-11T07:52:28.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:28 smithi022 ceph-mon[37330]: 3.2 scrub starts 2024-09-11T07:52:28.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:28 smithi022 ceph-mon[37330]: 3.2 scrub ok 2024-09-11T07:52:28.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:28 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/88037589' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T07:52:28.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:28 smithi022 ceph-mon[37330]: pgmap v1225: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 762 B/s rd, 0 op/s 2024-09-11T07:52:28.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:28 smithi022 ceph-mon[37330]: 6.7 scrub starts 2024-09-11T07:52:28.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:28 smithi022 ceph-mon[37330]: 6.7 scrub ok 2024-09-11T07:52:28.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:28 smithi022 ceph-mon[37330]: 4.5 scrub starts 2024-09-11T07:52:28.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:28 smithi022 ceph-mon[37330]: 4.5 scrub ok 2024-09-11T07:52:28.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:28 smithi104 ceph-mon[33507]: 1.5 scrub starts 2024-09-11T07:52:28.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:28 smithi104 ceph-mon[33507]: 1.5 scrub ok 2024-09-11T07:52:28.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:28 smithi104 ceph-mon[33507]: 3.2 scrub starts 2024-09-11T07:52:28.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:28 smithi104 ceph-mon[33507]: 3.2 scrub ok 2024-09-11T07:52:28.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:28 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/88037589' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T07:52:28.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:28 smithi104 ceph-mon[33507]: pgmap v1225: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 762 B/s rd, 0 op/s 2024-09-11T07:52:28.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:28 smithi104 ceph-mon[33507]: 6.7 scrub starts 2024-09-11T07:52:28.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:28 smithi104 ceph-mon[33507]: 6.7 scrub ok 2024-09-11T07:52:28.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:28 smithi104 ceph-mon[33507]: 4.5 scrub starts 2024-09-11T07:52:28.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:28 smithi104 ceph-mon[33507]: 4.5 scrub ok 2024-09-11T07:52:29.639 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:29 smithi022 ceph-mon[29462]: 2.8 scrub starts 2024-09-11T07:52:29.639 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:29 smithi022 ceph-mon[29462]: 2.8 scrub ok 2024-09-11T07:52:29.639 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:29 smithi022 ceph-mon[29462]: 3.0 scrub starts 2024-09-11T07:52:29.639 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:29 smithi022 ceph-mon[29462]: 3.0 scrub ok 2024-09-11T07:52:29.639 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:29 smithi022 ceph-mon[29462]: 5.1c scrub starts 2024-09-11T07:52:29.640 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:29 smithi022 ceph-mon[29462]: 5.1c scrub ok 2024-09-11T07:52:29.640 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:29 smithi022 ceph-mon[29462]: 5.a scrub starts 2024-09-11T07:52:29.640 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:29 smithi022 ceph-mon[29462]: 5.a scrub ok 2024-09-11T07:52:29.640 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:29 smithi022 ceph-mon[29462]: 2.13 scrub starts 2024-09-11T07:52:29.640 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:29 smithi022 ceph-mon[29462]: 2.13 scrub ok 2024-09-11T07:52:29.640 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:29 smithi022 ceph-mon[37330]: 2.8 scrub starts 2024-09-11T07:52:29.640 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:29 smithi022 ceph-mon[37330]: 2.8 scrub ok 2024-09-11T07:52:29.640 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:29 smithi022 ceph-mon[37330]: 3.0 scrub starts 2024-09-11T07:52:29.640 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:29 smithi022 ceph-mon[37330]: 3.0 scrub ok 2024-09-11T07:52:29.640 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:29 smithi022 ceph-mon[37330]: 5.1c scrub starts 2024-09-11T07:52:29.640 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:29 smithi022 ceph-mon[37330]: 5.1c scrub ok 2024-09-11T07:52:29.641 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:29 smithi022 ceph-mon[37330]: 5.a scrub starts 2024-09-11T07:52:29.641 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:29 smithi022 ceph-mon[37330]: 5.a scrub ok 2024-09-11T07:52:29.641 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:29 smithi022 ceph-mon[37330]: 2.13 scrub starts 2024-09-11T07:52:29.641 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:29 smithi022 ceph-mon[37330]: 2.13 scrub ok 2024-09-11T07:52:29.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:29 smithi104 ceph-mon[33507]: 2.8 scrub starts 2024-09-11T07:52:29.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:29 smithi104 ceph-mon[33507]: 2.8 scrub ok 2024-09-11T07:52:29.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:29 smithi104 ceph-mon[33507]: 3.0 scrub starts 2024-09-11T07:52:29.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:29 smithi104 ceph-mon[33507]: 3.0 scrub ok 2024-09-11T07:52:29.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:29 smithi104 ceph-mon[33507]: 5.1c scrub starts 2024-09-11T07:52:29.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:29 smithi104 ceph-mon[33507]: 5.1c scrub ok 2024-09-11T07:52:29.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:29 smithi104 ceph-mon[33507]: 5.a scrub starts 2024-09-11T07:52:29.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:29 smithi104 ceph-mon[33507]: 5.a scrub ok 2024-09-11T07:52:29.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:29 smithi104 ceph-mon[33507]: 2.13 scrub starts 2024-09-11T07:52:29.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:29 smithi104 ceph-mon[33507]: 2.13 scrub ok 2024-09-11T07:52:30.666 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:30 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:52:30] "GET /metrics HTTP/1.1" 200 222442 "" "Prometheus/2.43.0" 2024-09-11T07:52:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:30 smithi022 ceph-mon[29462]: 6.3 deep-scrub starts 2024-09-11T07:52:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:30 smithi022 ceph-mon[29462]: 6.3 deep-scrub ok 2024-09-11T07:52:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:30 smithi022 ceph-mon[29462]: 4.1b scrub starts 2024-09-11T07:52:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:30 smithi022 ceph-mon[29462]: 4.1b scrub ok 2024-09-11T07:52:30.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:30 smithi022 ceph-mon[29462]: pgmap v1226: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:52:30.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:30 smithi022 ceph-mon[37330]: 6.3 deep-scrub starts 2024-09-11T07:52:30.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:30 smithi022 ceph-mon[37330]: 6.3 deep-scrub ok 2024-09-11T07:52:30.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:30 smithi022 ceph-mon[37330]: 4.1b scrub starts 2024-09-11T07:52:30.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:30 smithi022 ceph-mon[37330]: 4.1b scrub ok 2024-09-11T07:52:30.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:30 smithi022 ceph-mon[37330]: pgmap v1226: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:52:30.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:52:30 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:52:30.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:30 smithi104 ceph-mon[33507]: 6.3 deep-scrub starts 2024-09-11T07:52:30.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:30 smithi104 ceph-mon[33507]: 6.3 deep-scrub ok 2024-09-11T07:52:30.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:30 smithi104 ceph-mon[33507]: 4.1b scrub starts 2024-09-11T07:52:30.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:30 smithi104 ceph-mon[33507]: 4.1b scrub ok 2024-09-11T07:52:30.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:30 smithi104 ceph-mon[33507]: pgmap v1226: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:52:31.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:31 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:52:31.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:31 smithi022 ceph-mon[29462]: osdmap e532: 8 total, 8 up, 8 in 2024-09-11T07:52:31.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:31 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:52:31.417 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:31 smithi022 ceph-mon[37330]: osdmap e532: 8 total, 8 up, 8 in 2024-09-11T07:52:31.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:31 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:52:31.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:31 smithi104 ceph-mon[33507]: osdmap e532: 8 total, 8 up, 8 in 2024-09-11T07:52:32.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:32 smithi022 ceph-mon[29462]: 6.12 scrub starts 2024-09-11T07:52:32.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:32 smithi022 ceph-mon[29462]: 6.12 scrub ok 2024-09-11T07:52:32.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:32 smithi022 ceph-mon[29462]: 2.e scrub starts 2024-09-11T07:52:32.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:32 smithi022 ceph-mon[29462]: 2.e scrub ok 2024-09-11T07:52:32.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:32 smithi022 ceph-mon[29462]: pgmap v1228: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:52:32.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:32 smithi022 ceph-mon[29462]: 4.14 scrub starts 2024-09-11T07:52:32.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:32 smithi022 ceph-mon[29462]: 4.14 scrub ok 2024-09-11T07:52:32.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:32 smithi022 ceph-mon[29462]: 6.16 scrub starts 2024-09-11T07:52:32.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:32 smithi022 ceph-mon[29462]: 6.16 scrub ok 2024-09-11T07:52:32.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:32 smithi022 ceph-mon[29462]: 4.2 scrub starts 2024-09-11T07:52:32.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:32 smithi022 ceph-mon[29462]: 4.2 scrub ok 2024-09-11T07:52:32.500 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:32 smithi022 ceph-mon[37330]: 6.12 scrub starts 2024-09-11T07:52:32.500 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:32 smithi022 ceph-mon[37330]: 6.12 scrub ok 2024-09-11T07:52:32.500 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:32 smithi022 ceph-mon[37330]: 2.e scrub starts 2024-09-11T07:52:32.500 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:32 smithi022 ceph-mon[37330]: 2.e scrub ok 2024-09-11T07:52:32.500 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:32 smithi022 ceph-mon[37330]: pgmap v1228: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:52:32.500 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:32 smithi022 ceph-mon[37330]: 4.14 scrub starts 2024-09-11T07:52:32.500 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:32 smithi022 ceph-mon[37330]: 4.14 scrub ok 2024-09-11T07:52:32.500 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:32 smithi022 ceph-mon[37330]: 6.16 scrub starts 2024-09-11T07:52:32.500 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:32 smithi022 ceph-mon[37330]: 6.16 scrub ok 2024-09-11T07:52:32.500 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:32 smithi022 ceph-mon[37330]: 4.2 scrub starts 2024-09-11T07:52:32.500 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:32 smithi022 ceph-mon[37330]: 4.2 scrub ok 2024-09-11T07:52:32.757 INFO:teuthology.orchestra.run.smithi022.stdout:2083059138584 2024-09-11T07:52:32.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:32 smithi104 ceph-mon[33507]: 6.12 scrub starts 2024-09-11T07:52:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:32 smithi104 ceph-mon[33507]: 6.12 scrub ok 2024-09-11T07:52:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:32 smithi104 ceph-mon[33507]: 2.e scrub starts 2024-09-11T07:52:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:32 smithi104 ceph-mon[33507]: 2.e scrub ok 2024-09-11T07:52:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:32 smithi104 ceph-mon[33507]: pgmap v1228: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:52:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:32 smithi104 ceph-mon[33507]: 4.14 scrub starts 2024-09-11T07:52:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:32 smithi104 ceph-mon[33507]: 4.14 scrub ok 2024-09-11T07:52:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:32 smithi104 ceph-mon[33507]: 6.16 scrub starts 2024-09-11T07:52:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:32 smithi104 ceph-mon[33507]: 6.16 scrub ok 2024-09-11T07:52:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:32 smithi104 ceph-mon[33507]: 4.2 scrub starts 2024-09-11T07:52:32.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:32 smithi104 ceph-mon[33507]: 4.2 scrub ok 2024-09-11T07:52:33.387 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:52:33.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:33 smithi022 ceph-mon[29462]: 6.5 scrub starts 2024-09-11T07:52:33.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:33 smithi022 ceph-mon[29462]: 6.5 scrub ok 2024-09-11T07:52:33.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:33 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/600947279' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T07:52:33.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:33 smithi022 ceph-mon[37330]: 6.5 scrub starts 2024-09-11T07:52:33.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:33 smithi022 ceph-mon[37330]: 6.5 scrub ok 2024-09-11T07:52:33.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:33 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/600947279' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T07:52:33.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:33 smithi104 ceph-mon[33507]: 6.5 scrub starts 2024-09-11T07:52:33.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:33 smithi104 ceph-mon[33507]: 6.5 scrub ok 2024-09-11T07:52:33.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:33 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/600947279' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T07:52:34.225 INFO:teuthology.orchestra.run.smithi022.stdout:34359738718 2024-09-11T07:52:34.388 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:52:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:52:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:52:34.646 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:34 smithi022 ceph-mon[29462]: pgmap v1229: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:52:34.647 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:34 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/378995773' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T07:52:34.647 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:34 smithi022 ceph-mon[37330]: pgmap v1229: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:52:34.647 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:34 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/378995773' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T07:52:34.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:34 smithi104 ceph-mon[33507]: pgmap v1229: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:52:34.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:34 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/378995773' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T07:52:35.220 INFO:tasks.cephadm.ceph_manager.ceph:need seq 2083059138582 got 2083059138584 for osd.2 2024-09-11T07:52:35.220 DEBUG:teuthology.parallel:result is None 2024-09-11T07:52:35.541 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:35 smithi022 ceph-mon[29462]: 4.3 scrub starts 2024-09-11T07:52:35.541 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:35 smithi022 ceph-mon[29462]: 4.3 scrub ok 2024-09-11T07:52:35.541 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:35 smithi022 ceph-mon[37330]: 4.3 scrub starts 2024-09-11T07:52:35.541 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:35 smithi022 ceph-mon[37330]: 4.3 scrub ok 2024-09-11T07:52:35.585 INFO:teuthology.orchestra.run.smithi022.stdout:631360192760 2024-09-11T07:52:35.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:35 smithi104 ceph-mon[33507]: 4.3 scrub starts 2024-09-11T07:52:35.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:35 smithi104 ceph-mon[33507]: 4.3 scrub ok 2024-09-11T07:52:36.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:36 smithi022 ceph-mon[29462]: pgmap v1230: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T07:52:36.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:36 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/103740423' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T07:52:36.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:36 smithi022 ceph-mon[29462]: 6.d scrub starts 2024-09-11T07:52:36.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:36 smithi022 ceph-mon[29462]: 6.d scrub ok 2024-09-11T07:52:36.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:36 smithi022 ceph-mon[29462]: 5.1d scrub starts 2024-09-11T07:52:36.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:36 smithi022 ceph-mon[29462]: 5.1d scrub ok 2024-09-11T07:52:36.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:36 smithi022 ceph-mon[29462]: osdmap e533: 8 total, 8 up, 8 in 2024-09-11T07:52:36.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:36 smithi022 ceph-mon[37330]: pgmap v1230: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T07:52:36.418 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:36 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/103740423' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T07:52:36.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:36 smithi022 ceph-mon[37330]: 6.d scrub starts 2024-09-11T07:52:36.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:36 smithi022 ceph-mon[37330]: 6.d scrub ok 2024-09-11T07:52:36.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:36 smithi022 ceph-mon[37330]: 5.1d scrub starts 2024-09-11T07:52:36.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:36 smithi022 ceph-mon[37330]: 5.1d scrub ok 2024-09-11T07:52:36.419 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:36 smithi022 ceph-mon[37330]: osdmap e533: 8 total, 8 up, 8 in 2024-09-11T07:52:36.548 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738714 got 34359738718 for osd.0 2024-09-11T07:52:36.548 DEBUG:teuthology.parallel:result is None 2024-09-11T07:52:36.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:36 smithi104 ceph-mon[33507]: pgmap v1230: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T07:52:36.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:36 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/103740423' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T07:52:36.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:36 smithi104 ceph-mon[33507]: 6.d scrub starts 2024-09-11T07:52:36.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:36 smithi104 ceph-mon[33507]: 6.d scrub ok 2024-09-11T07:52:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:36 smithi104 ceph-mon[33507]: 5.1d scrub starts 2024-09-11T07:52:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:36 smithi104 ceph-mon[33507]: 5.1d scrub ok 2024-09-11T07:52:36.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:36 smithi104 ceph-mon[33507]: osdmap e533: 8 total, 8 up, 8 in 2024-09-11T07:52:37.128 INFO:tasks.cephadm.ceph_manager.ceph:need seq 631360192758 got 631360192760 for osd.7 2024-09-11T07:52:37.129 DEBUG:teuthology.parallel:result is None 2024-09-11T07:52:37.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:37 smithi022 ceph-mon[29462]: 2.1a scrub starts 2024-09-11T07:52:37.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:37 smithi022 ceph-mon[29462]: 2.1a scrub ok 2024-09-11T07:52:37.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:37 smithi022 ceph-mon[37330]: 2.1a scrub starts 2024-09-11T07:52:37.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:37 smithi022 ceph-mon[37330]: 2.1a scrub ok 2024-09-11T07:52:37.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:37 smithi104 ceph-mon[33507]: 2.1a scrub starts 2024-09-11T07:52:37.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:37 smithi104 ceph-mon[33507]: 2.1a scrub ok 2024-09-11T07:52:38.025 INFO:teuthology.orchestra.run.smithi022.stdout:107374182736 2024-09-11T07:52:38.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:38 smithi022 ceph-mon[29462]: 6.11 scrub starts 2024-09-11T07:52:38.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:38 smithi022 ceph-mon[29462]: 6.11 scrub ok 2024-09-11T07:52:38.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:38 smithi022 ceph-mon[29462]: pgmap v1232: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:52:38.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:38 smithi022 ceph-mon[29462]: 2.9 scrub starts 2024-09-11T07:52:38.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:38 smithi022 ceph-mon[29462]: 2.9 scrub ok 2024-09-11T07:52:38.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:38 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3254198882' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T07:52:38.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:38 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:52:38.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:38 smithi022 ceph-mon[37330]: 6.11 scrub starts 2024-09-11T07:52:38.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:38 smithi022 ceph-mon[37330]: 6.11 scrub ok 2024-09-11T07:52:38.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:38 smithi022 ceph-mon[37330]: pgmap v1232: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:52:38.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:38 smithi022 ceph-mon[37330]: 2.9 scrub starts 2024-09-11T07:52:38.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:38 smithi022 ceph-mon[37330]: 2.9 scrub ok 2024-09-11T07:52:38.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:38 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3254198882' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T07:52:38.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:38 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:52:38.695 INFO:teuthology.orchestra.run.smithi022.stdout:158913790280 2024-09-11T07:52:38.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:38 smithi104 ceph-mon[33507]: 6.11 scrub starts 2024-09-11T07:52:38.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:38 smithi104 ceph-mon[33507]: 6.11 scrub ok 2024-09-11T07:52:38.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:38 smithi104 ceph-mon[33507]: pgmap v1232: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:52:38.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:38 smithi104 ceph-mon[33507]: 2.9 scrub starts 2024-09-11T07:52:38.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:38 smithi104 ceph-mon[33507]: 2.9 scrub ok 2024-09-11T07:52:38.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:38 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3254198882' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T07:52:38.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:38 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:52:39.319 INFO:teuthology.orchestra.run.smithi022.stdout:133143986508 2024-09-11T07:52:39.623 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:39 smithi022 ceph-mon[29462]: 6.15 deep-scrub starts 2024-09-11T07:52:39.623 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:39 smithi022 ceph-mon[29462]: 6.15 deep-scrub ok 2024-09-11T07:52:39.623 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:39 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/3966314078' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T07:52:39.623 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:39 smithi022 ceph-mon[29462]: 5.5 deep-scrub starts 2024-09-11T07:52:39.623 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:39 smithi022 ceph-mon[29462]: 5.5 deep-scrub ok 2024-09-11T07:52:39.623 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:39 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/1041220619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T07:52:39.623 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:39 smithi022 ceph-mon[37330]: 6.15 deep-scrub starts 2024-09-11T07:52:39.623 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:39 smithi022 ceph-mon[37330]: 6.15 deep-scrub ok 2024-09-11T07:52:39.623 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:39 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/3966314078' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T07:52:39.623 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:39 smithi022 ceph-mon[37330]: 5.5 deep-scrub starts 2024-09-11T07:52:39.624 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:39 smithi022 ceph-mon[37330]: 5.5 deep-scrub ok 2024-09-11T07:52:39.624 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:39 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/1041220619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T07:52:39.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:39 smithi104 ceph-mon[33507]: 6.15 deep-scrub starts 2024-09-11T07:52:39.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:39 smithi104 ceph-mon[33507]: 6.15 deep-scrub ok 2024-09-11T07:52:39.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:39 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/3966314078' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T07:52:39.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:39 smithi104 ceph-mon[33507]: 5.5 deep-scrub starts 2024-09-11T07:52:39.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:39 smithi104 ceph-mon[33507]: 5.5 deep-scrub ok 2024-09-11T07:52:39.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:39 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/1041220619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T07:52:40.046 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986504 got 133143986508 for osd.4 2024-09-11T07:52:40.046 DEBUG:teuthology.parallel:result is None 2024-09-11T07:52:40.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:40 smithi022 ceph-mon[29462]: 4.11 deep-scrub starts 2024-09-11T07:52:40.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:40 smithi022 ceph-mon[29462]: 4.11 deep-scrub ok 2024-09-11T07:52:40.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:40 smithi022 ceph-mon[29462]: pgmap v1233: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:52:40.667 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:40 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:52:40] "GET /metrics HTTP/1.1" 200 222452 "" "Prometheus/2.43.0" 2024-09-11T07:52:40.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:40 smithi022 ceph-mon[37330]: 4.11 deep-scrub starts 2024-09-11T07:52:40.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:40 smithi022 ceph-mon[37330]: 4.11 deep-scrub ok 2024-09-11T07:52:40.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:40 smithi022 ceph-mon[37330]: pgmap v1233: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:52:40.768 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182732 got 107374182736 for osd.3 2024-09-11T07:52:40.769 DEBUG:teuthology.parallel:result is None 2024-09-11T07:52:40.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:52:40 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:52:40.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:40 smithi104 ceph-mon[33507]: 4.11 deep-scrub starts 2024-09-11T07:52:40.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:40 smithi104 ceph-mon[33507]: 4.11 deep-scrub ok 2024-09-11T07:52:40.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:40 smithi104 ceph-mon[33507]: pgmap v1233: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:52:41.147 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790276 got 158913790280 for osd.5 2024-09-11T07:52:41.147 DEBUG:teuthology.parallel:result is None 2024-09-11T07:52:41.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:41 smithi022 ceph-mon[29462]: 2.1d scrub starts 2024-09-11T07:52:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:41 smithi022 ceph-mon[29462]: 2.1d scrub ok 2024-09-11T07:52:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:41 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:52:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:41 smithi022 ceph-mon[29462]: 4.f deep-scrub starts 2024-09-11T07:52:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:41 smithi022 ceph-mon[29462]: 4.f deep-scrub ok 2024-09-11T07:52:41.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:41 smithi022 ceph-mon[29462]: osdmap e534: 8 total, 8 up, 8 in 2024-09-11T07:52:41.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:41 smithi022 ceph-mon[37330]: 2.1d scrub starts 2024-09-11T07:52:41.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:41 smithi022 ceph-mon[37330]: 2.1d scrub ok 2024-09-11T07:52:41.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:41 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:52:41.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:41 smithi022 ceph-mon[37330]: 4.f deep-scrub starts 2024-09-11T07:52:41.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:41 smithi022 ceph-mon[37330]: 4.f deep-scrub ok 2024-09-11T07:52:41.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:41 smithi022 ceph-mon[37330]: osdmap e534: 8 total, 8 up, 8 in 2024-09-11T07:52:41.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:41 smithi104 ceph-mon[33507]: 2.1d scrub starts 2024-09-11T07:52:41.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:41 smithi104 ceph-mon[33507]: 2.1d scrub ok 2024-09-11T07:52:41.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:41 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:52:41.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:41 smithi104 ceph-mon[33507]: 4.f deep-scrub starts 2024-09-11T07:52:41.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:41 smithi104 ceph-mon[33507]: 4.f deep-scrub ok 2024-09-11T07:52:41.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:41 smithi104 ceph-mon[33507]: osdmap e534: 8 total, 8 up, 8 in 2024-09-11T07:52:41.928 INFO:teuthology.orchestra.run.smithi022.stdout:188978561347 2024-09-11T07:52:42.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:42 smithi022 ceph-mon[29462]: 5.17 deep-scrub starts 2024-09-11T07:52:42.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:42 smithi022 ceph-mon[29462]: 5.17 deep-scrub ok 2024-09-11T07:52:42.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:42 smithi022 ceph-mon[29462]: pgmap v1235: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:52:42.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:42 smithi022 ceph-mon[29462]: 2.6 scrub starts 2024-09-11T07:52:42.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:42 smithi022 ceph-mon[29462]: 2.6 scrub ok 2024-09-11T07:52:42.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:42 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/2703902095' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T07:52:42.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:42 smithi022 ceph-mon[37330]: 5.17 deep-scrub starts 2024-09-11T07:52:42.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:42 smithi022 ceph-mon[37330]: 5.17 deep-scrub ok 2024-09-11T07:52:42.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:42 smithi022 ceph-mon[37330]: pgmap v1235: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:52:42.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:42 smithi022 ceph-mon[37330]: 2.6 scrub starts 2024-09-11T07:52:42.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:42 smithi022 ceph-mon[37330]: 2.6 scrub ok 2024-09-11T07:52:42.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:42 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/2703902095' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T07:52:42.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:42 smithi104 ceph-mon[33507]: 5.17 deep-scrub starts 2024-09-11T07:52:42.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:42 smithi104 ceph-mon[33507]: 5.17 deep-scrub ok 2024-09-11T07:52:42.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:42 smithi104 ceph-mon[33507]: pgmap v1235: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:52:42.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:42 smithi104 ceph-mon[33507]: 2.6 scrub starts 2024-09-11T07:52:42.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:42 smithi104 ceph-mon[33507]: 2.6 scrub ok 2024-09-11T07:52:42.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:42 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/2703902095' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T07:52:42.714 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561342 got 188978561347 for osd.6 2024-09-11T07:52:42.714 DEBUG:teuthology.parallel:result is None 2024-09-11T07:52:42.714 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-11T07:52:42.714 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:52:42.982 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:52:43.485 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:43 smithi022 ceph-mon[37330]: 2.18 scrub starts 2024-09-11T07:52:43.485 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:43 smithi022 ceph-mon[37330]: 2.18 scrub ok 2024-09-11T07:52:43.485 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:43 smithi022 ceph-mon[37330]: 5.19 deep-scrub starts 2024-09-11T07:52:43.486 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:43 smithi022 ceph-mon[37330]: 5.19 deep-scrub ok 2024-09-11T07:52:43.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:43 smithi022 ceph-mon[29462]: 2.18 scrub starts 2024-09-11T07:52:43.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:43 smithi022 ceph-mon[29462]: 2.18 scrub ok 2024-09-11T07:52:43.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:43 smithi022 ceph-mon[29462]: 5.19 deep-scrub starts 2024-09-11T07:52:43.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:43 smithi022 ceph-mon[29462]: 5.19 deep-scrub ok 2024-09-11T07:52:43.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:43 smithi104 ceph-mon[33507]: 2.18 scrub starts 2024-09-11T07:52:43.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:43 smithi104 ceph-mon[33507]: 2.18 scrub ok 2024-09-11T07:52:43.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:43 smithi104 ceph-mon[33507]: 5.19 deep-scrub starts 2024-09-11T07:52:43.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:43 smithi104 ceph-mon[33507]: 5.19 deep-scrub ok 2024-09-11T07:52:44.279 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:52:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:52:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:52:44.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:44 smithi022 ceph-mon[29462]: pgmap v1236: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:52:44.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:44 smithi022 ceph-mon[29462]: 5.1b scrub starts 2024-09-11T07:52:44.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:44 smithi022 ceph-mon[29462]: 5.1b scrub ok 2024-09-11T07:52:44.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:44 smithi022 ceph-mon[29462]: 2.4 scrub starts 2024-09-11T07:52:44.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:44 smithi022 ceph-mon[29462]: 2.4 scrub ok 2024-09-11T07:52:44.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:44 smithi022 ceph-mon[37330]: pgmap v1236: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:52:44.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:44 smithi022 ceph-mon[37330]: 5.1b scrub starts 2024-09-11T07:52:44.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:44 smithi022 ceph-mon[37330]: 5.1b scrub ok 2024-09-11T07:52:44.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:44 smithi022 ceph-mon[37330]: 2.4 scrub starts 2024-09-11T07:52:44.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:44 smithi022 ceph-mon[37330]: 2.4 scrub ok 2024-09-11T07:52:44.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:44 smithi104 ceph-mon[33507]: pgmap v1236: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:52:44.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:44 smithi104 ceph-mon[33507]: 5.1b scrub starts 2024-09-11T07:52:44.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:44 smithi104 ceph-mon[33507]: 5.1b scrub ok 2024-09-11T07:52:44.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:44 smithi104 ceph-mon[33507]: 2.4 scrub starts 2024-09-11T07:52:44.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:44 smithi104 ceph-mon[33507]: 2.4 scrub ok 2024-09-11T07:52:44.830 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:52:44.833 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:52:45.592 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":1236,"stamp":"2024-09-11T07:52:43.570942+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":2679,"num_read_kb":2431,"num_write":1002,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":539,"num_bytes_recovered":996389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2124,"ondisk_log_size":2124,"up":426,"acting":426,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":426,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":6689764,"kb_used_data":9296,"kb_used_omap":65,"kb_used_meta":6680318,"kb_avail":743271452,"statfs":{"total":767960285184,"available":761109966848,"internally_reserved":0,"allocated":9519104,"data_stored":5736898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":66799,"internal_metadata":6840646417},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002268"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":58,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.368062+0000","last_change":"2024-09-11T07:52:20.420243+0000","last_active":"2024-09-11T07:52:31.368062+0000","last_peered":"2024-09-11T07:52:31.368062+0000","last_clean":"2024-09-11T07:52:31.368062+0000","last_became_active":"2024-09-11T07:51:48.206877+0000","last_became_peered":"2024-09-11T07:51:48.206877+0000","last_unstale":"2024-09-11T07:52:31.368062+0000","last_undegraded":"2024-09-11T07:52:31.368062+0000","last_fullsized":"2024-09-11T07:52:31.368062+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:20.420170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:46:28.323351+0000","last_clean_scrub_stamp":"2024-09-11T07:52:20.420170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:33.920201+0000","scrub_duration":0.0020486520000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":651,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:30.861878+0000","last_change":"2024-09-11T07:52:21.808023+0000","last_active":"2024-09-11T07:52:30.861878+0000","last_peered":"2024-09-11T07:52:30.861878+0000","last_clean":"2024-09-11T07:52:30.861878+0000","last_became_active":"2024-09-11T07:50:56.906118+0000","last_became_peered":"2024-09-11T07:50:56.906118+0000","last_unstale":"2024-09-11T07:52:30.861878+0000","last_undegraded":"2024-09-11T07:52:30.861878+0000","last_fullsized":"2024-09-11T07:52:30.861878+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:21.807966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:47:24.201007+0000","last_clean_scrub_stamp":"2024-09-11T07:52:21.807966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:26.501637+0000","scrub_duration":0.001658262,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"426'46","reported_seq":705,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.368730+0000","last_change":"2024-09-11T07:52:13.488090+0000","last_active":"2024-09-11T07:52:31.368730+0000","last_peered":"2024-09-11T07:52:31.368730+0000","last_clean":"2024-09-11T07:52:31.368730+0000","last_became_active":"2024-09-11T07:50:53.904640+0000","last_became_peered":"2024-09-11T07:50:53.904640+0000","last_unstale":"2024-09-11T07:52:31.368730+0000","last_undegraded":"2024-09-11T07:52:31.368730+0000","last_fullsized":"2024-09-11T07:52:31.368730+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'46","last_scrub_stamp":"2024-09-11T07:52:13.487992+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:52:13.487992+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:34.823509+0000","scrub_duration":0.002954557,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":63,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.252715+0000","last_change":"2024-09-11T07:51:55.154414+0000","last_active":"2024-09-11T07:52:31.252715+0000","last_peered":"2024-09-11T07:52:31.252715+0000","last_clean":"2024-09-11T07:52:31.252715+0000","last_became_active":"2024-09-11T07:50:50.862411+0000","last_became_peered":"2024-09-11T07:50:50.862411+0000","last_unstale":"2024-09-11T07:52:31.252715+0000","last_undegraded":"2024-09-11T07:52:31.252715+0000","last_fullsized":"2024-09-11T07:52:31.252715+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:55.154345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:51:55.154345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:15.076843+0000","scrub_duration":0.0013346720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":660,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.369261+0000","last_change":"2024-09-11T07:51:34.321543+0000","last_active":"2024-09-11T07:52:31.369261+0000","last_peered":"2024-09-11T07:52:31.369261+0000","last_clean":"2024-09-11T07:52:31.369261+0000","last_became_active":"2024-09-11T07:51:34.320960+0000","last_became_peered":"2024-09-11T07:51:34.320960+0000","last_unstale":"2024-09-11T07:52:31.369261+0000","last_undegraded":"2024-09-11T07:52:31.369261+0000","last_fullsized":"2024-09-11T07:52:31.369261+0000","mapping_epoch":509,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":510,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:28.443214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:19.655401+0000","last_clean_scrub_stamp":"2024-09-11T07:51:28.443214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:50.940231+0000","scrub_duration":0.001291483,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"426'25","reported_seq":83,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.252708+0000","last_change":"2024-09-11T07:52:21.165183+0000","last_active":"2024-09-11T07:52:31.252708+0000","last_peered":"2024-09-11T07:52:31.252708+0000","last_clean":"2024-09-11T07:52:31.252708+0000","last_became_active":"2024-09-11T07:50:50.861997+0000","last_became_peered":"2024-09-11T07:50:50.861997+0000","last_unstale":"2024-09-11T07:52:31.252708+0000","last_undegraded":"2024-09-11T07:52:31.252708+0000","last_fullsized":"2024-09-11T07:52:31.252708+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'25","last_scrub_stamp":"2024-09-11T07:52:21.165111+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:36:42.708941+0000","last_clean_scrub_stamp":"2024-09-11T07:52:21.165111+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:34.072089+0000","scrub_duration":0.0036606170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":515,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383153+0000","last_change":"2024-09-11T07:51:36.922257+0000","last_active":"2024-09-11T07:52:31.383153+0000","last_peered":"2024-09-11T07:52:31.383153+0000","last_clean":"2024-09-11T07:52:31.383153+0000","last_became_active":"2024-09-11T07:51:14.064108+0000","last_became_peered":"2024-09-11T07:51:14.064108+0000","last_unstale":"2024-09-11T07:52:31.383153+0000","last_undegraded":"2024-09-11T07:52:31.383153+0000","last_fullsized":"2024-09-11T07:52:31.383153+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:36.922194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:50.648420+0000","last_clean_scrub_stamp":"2024-09-11T07:51:36.922194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:37.023835+0000","scrub_duration":0.0016947450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":73,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.253008+0000","last_change":"2024-09-11T07:51:39.194332+0000","last_active":"2024-09-11T07:52:31.253008+0000","last_peered":"2024-09-11T07:52:31.253008+0000","last_clean":"2024-09-11T07:52:31.253008+0000","last_became_active":"2024-09-11T07:51:09.045103+0000","last_became_peered":"2024-09-11T07:51:09.045103+0000","last_unstale":"2024-09-11T07:52:31.253008+0000","last_undegraded":"2024-09-11T07:52:31.253008+0000","last_fullsized":"2024-09-11T07:52:31.253008+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:39.194263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:51:39.194263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:03.055189+0000","scrub_duration":0.0014768450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":58,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:30.862483+0000","last_change":"2024-09-11T07:51:25.593237+0000","last_active":"2024-09-11T07:52:30.862483+0000","last_peered":"2024-09-11T07:52:30.862483+0000","last_clean":"2024-09-11T07:52:30.862483+0000","last_became_active":"2024-09-11T07:50:56.912957+0000","last_became_peered":"2024-09-11T07:50:56.912957+0000","last_unstale":"2024-09-11T07:52:30.862483+0000","last_undegraded":"2024-09-11T07:52:30.862483+0000","last_fullsized":"2024-09-11T07:52:30.862483+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:25.593167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:48:37.724091+0000","last_clean_scrub_stamp":"2024-09-11T07:51:25.593167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015057460000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"426'11","reported_seq":111,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.253123+0000","last_change":"2024-09-11T07:52:29.226069+0000","last_active":"2024-09-11T07:52:31.253123+0000","last_peered":"2024-09-11T07:52:31.253123+0000","last_clean":"2024-09-11T07:52:31.253123+0000","last_became_active":"2024-09-11T07:51:09.045188+0000","last_became_peered":"2024-09-11T07:51:09.045188+0000","last_unstale":"2024-09-11T07:52:31.253123+0000","last_undegraded":"2024-09-11T07:52:31.253123+0000","last_fullsized":"2024-09-11T07:52:31.253123+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'11","last_scrub_stamp":"2024-09-11T07:52:29.226003+0000","last_deep_scrub":"234'9","last_deep_scrub_stamp":"2024-09-11T07:45:32.375156+0000","last_clean_scrub_stamp":"2024-09-11T07:52:29.226003+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:35.352619+0000","scrub_duration":0.001949267,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":13,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":68,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:30.861997+0000","last_change":"2024-09-11T07:52:23.685295+0000","last_active":"2024-09-11T07:52:30.861997+0000","last_peered":"2024-09-11T07:52:30.861997+0000","last_clean":"2024-09-11T07:52:30.861997+0000","last_became_active":"2024-09-11T07:50:56.910585+0000","last_became_peered":"2024-09-11T07:50:56.910585+0000","last_unstale":"2024-09-11T07:52:30.861997+0000","last_undegraded":"2024-09-11T07:52:30.861997+0000","last_fullsized":"2024-09-11T07:52:30.861997+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:23.685224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:52:23.685224+0000","last_clean_scrub_stamp":"2024-09-11T07:52:23.685224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:37.473468+0000","scrub_duration":0.001870053,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":206,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383451+0000","last_change":"2024-09-11T07:52:07.526749+0000","last_active":"2024-09-11T07:52:31.383451+0000","last_peered":"2024-09-11T07:52:31.383451+0000","last_clean":"2024-09-11T07:52:31.383451+0000","last_became_active":"2024-09-11T07:51:37.859052+0000","last_became_peered":"2024-09-11T07:51:37.859052+0000","last_unstale":"2024-09-11T07:52:31.383451+0000","last_undegraded":"2024-09-11T07:52:31.383451+0000","last_fullsized":"2024-09-11T07:52:31.383451+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:07.526676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:45.149916+0000","last_clean_scrub_stamp":"2024-09-11T07:52:07.526676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:12.818897+0000","scrub_duration":0.0020718569999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":511,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.384274+0000","last_change":"2024-09-11T07:52:15.610486+0000","last_active":"2024-09-11T07:52:31.384274+0000","last_peered":"2024-09-11T07:52:31.384274+0000","last_clean":"2024-09-11T07:52:31.384274+0000","last_became_active":"2024-09-11T07:51:37.859870+0000","last_became_peered":"2024-09-11T07:51:37.859870+0000","last_unstale":"2024-09-11T07:52:31.384274+0000","last_undegraded":"2024-09-11T07:52:31.384274+0000","last_fullsized":"2024-09-11T07:52:31.384274+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:15.610418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:47:19.164579+0000","last_clean_scrub_stamp":"2024-09-11T07:52:15.610418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:43.312842+0000","scrub_duration":0.0017306789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"426'25","reported_seq":73,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.252912+0000","last_change":"2024-09-11T07:51:49.117720+0000","last_active":"2024-09-11T07:52:31.252912+0000","last_peered":"2024-09-11T07:52:31.252912+0000","last_clean":"2024-09-11T07:52:31.252912+0000","last_became_active":"2024-09-11T07:50:50.870738+0000","last_became_peered":"2024-09-11T07:50:50.870738+0000","last_unstale":"2024-09-11T07:52:31.252912+0000","last_undegraded":"2024-09-11T07:52:31.252912+0000","last_fullsized":"2024-09-11T07:52:31.252912+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'25","last_scrub_stamp":"2024-09-11T07:51:49.117649+0000","last_deep_scrub":"136'13","last_deep_scrub_stamp":"2024-09-11T07:31:08.042198+0000","last_clean_scrub_stamp":"2024-09-11T07:51:49.117649+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:06.774558+0000","scrub_duration":0.0023084400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":638,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383605+0000","last_change":"2024-09-11T07:51:48.206055+0000","last_active":"2024-09-11T07:52:31.383605+0000","last_peered":"2024-09-11T07:52:31.383605+0000","last_clean":"2024-09-11T07:52:31.383605+0000","last_became_active":"2024-09-11T07:51:48.205734+0000","last_became_peered":"2024-09-11T07:51:48.205734+0000","last_unstale":"2024-09-11T07:52:31.383605+0000","last_undegraded":"2024-09-11T07:52:31.383605+0000","last_fullsized":"2024-09-11T07:52:31.383605+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:10.665641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:51:10.665641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:40.541936+0000","scrub_duration":0.0020442619999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":645,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:30.859867+0000","last_change":"2024-09-11T07:52:08.761674+0000","last_active":"2024-09-11T07:52:30.859867+0000","last_peered":"2024-09-11T07:52:30.859867+0000","last_clean":"2024-09-11T07:52:30.859867+0000","last_became_active":"2024-09-11T07:50:56.906207+0000","last_became_peered":"2024-09-11T07:50:56.906207+0000","last_unstale":"2024-09-11T07:52:30.859867+0000","last_undegraded":"2024-09-11T07:52:30.859867+0000","last_fullsized":"2024-09-11T07:52:30.859867+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:08.761607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:52:08.761607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:28.344892+0000","scrub_duration":0.001579762,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.8","version":"87'81","reported_seq":76,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383451+0000","last_change":"2024-09-11T07:51:37.859022+0000","last_active":"2024-09-11T07:52:31.383451+0000","last_peered":"2024-09-11T07:52:31.383451+0000","last_clean":"2024-09-11T07:52:31.383451+0000","last_became_active":"2024-09-11T07:51:37.858651+0000","last_became_peered":"2024-09-11T07:51:37.858651+0000","last_unstale":"2024-09-11T07:52:31.383451+0000","last_undegraded":"2024-09-11T07:52:31.383451+0000","last_fullsized":"2024-09-11T07:52:31.383451+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:51:32.046031+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:51:32.046031+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:48.720658+0000","scrub_duration":0.0023556440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"496'53","reported_seq":127,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.253342+0000","last_change":"2024-09-11T07:52:18.155170+0000","last_active":"2024-09-11T07:52:31.253342+0000","last_peered":"2024-09-11T07:52:31.253342+0000","last_clean":"2024-09-11T07:52:31.253342+0000","last_became_active":"2024-09-11T07:50:53.916590+0000","last_became_peered":"2024-09-11T07:50:53.916590+0000","last_unstale":"2024-09-11T07:52:31.253342+0000","last_undegraded":"2024-09-11T07:52:31.253342+0000","last_fullsized":"2024-09-11T07:52:31.253342+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":0,"last_scrub":"496'53","last_scrub_stamp":"2024-09-11T07:52:18.155106+0000","last_deep_scrub":"496'53","last_deep_scrub_stamp":"2024-09-11T07:51:00.998091+0000","last_clean_scrub_stamp":"2024-09-11T07:52:18.155106+0000","objects_scrubbed":8,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:29.317629+0000","scrub_duration":0.0023213600000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":69,"num_read_kb":59,"num_write":46,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":58,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:30.862148+0000","last_change":"2024-09-11T07:51:17.562232+0000","last_active":"2024-09-11T07:52:30.862148+0000","last_peered":"2024-09-11T07:52:30.862148+0000","last_clean":"2024-09-11T07:52:30.862148+0000","last_became_active":"2024-09-11T07:50:56.910651+0000","last_became_peered":"2024-09-11T07:50:56.910651+0000","last_unstale":"2024-09-11T07:52:30.862148+0000","last_undegraded":"2024-09-11T07:52:30.862148+0000","last_fullsized":"2024-09-11T07:52:30.862148+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:17.562160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:51:17.562160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001615266,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":65,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383792+0000","last_change":"2024-09-11T07:52:05.823115+0000","last_active":"2024-09-11T07:52:31.383792+0000","last_peered":"2024-09-11T07:52:31.383792+0000","last_clean":"2024-09-11T07:52:31.383792+0000","last_became_active":"2024-09-11T07:51:14.061875+0000","last_became_peered":"2024-09-11T07:51:14.061875+0000","last_unstale":"2024-09-11T07:52:31.383792+0000","last_undegraded":"2024-09-11T07:52:31.383792+0000","last_fullsized":"2024-09-11T07:52:31.383792+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:05.823058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:52:05.823058+0000","last_clean_scrub_stamp":"2024-09-11T07:52:05.823058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:09.724551+0000","scrub_duration":0.0016080650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":61,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642521+0000","last_change":"2024-09-11T07:51:48.250235+0000","last_active":"2024-09-11T07:52:36.642521+0000","last_peered":"2024-09-11T07:52:36.642521+0000","last_clean":"2024-09-11T07:52:36.642521+0000","last_became_active":"2024-09-11T07:50:53.906064+0000","last_became_peered":"2024-09-11T07:50:53.906064+0000","last_unstale":"2024-09-11T07:52:36.642521+0000","last_undegraded":"2024-09-11T07:52:36.642521+0000","last_fullsized":"2024-09-11T07:52:36.642521+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:48.250179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:50:40.403125+0000","last_clean_scrub_stamp":"2024-09-11T07:51:48.250179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:10.988456+0000","scrub_duration":0.001584043,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"87'81","reported_seq":58,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:30.861847+0000","last_change":"2024-09-11T07:51:34.528144+0000","last_active":"2024-09-11T07:52:30.861847+0000","last_peered":"2024-09-11T07:52:30.861847+0000","last_clean":"2024-09-11T07:52:30.861847+0000","last_became_active":"2024-09-11T07:50:56.908493+0000","last_became_peered":"2024-09-11T07:50:56.908493+0000","last_unstale":"2024-09-11T07:52:30.861847+0000","last_undegraded":"2024-09-11T07:52:30.861847+0000","last_fullsized":"2024-09-11T07:52:30.861847+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:51:34.528009+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:51:34.528009+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020188860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"426'35","reported_seq":75,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.253119+0000","last_change":"2024-09-11T07:51:41.210125+0000","last_active":"2024-09-11T07:52:31.253119+0000","last_peered":"2024-09-11T07:52:31.253119+0000","last_clean":"2024-09-11T07:52:31.253119+0000","last_became_active":"2024-09-11T07:50:50.866011+0000","last_became_peered":"2024-09-11T07:50:50.866011+0000","last_unstale":"2024-09-11T07:52:31.253119+0000","last_undegraded":"2024-09-11T07:52:31.253119+0000","last_fullsized":"2024-09-11T07:52:31.253119+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'35","last_scrub_stamp":"2024-09-11T07:51:41.210061+0000","last_deep_scrub":"112'15","last_deep_scrub_stamp":"2024-09-11T07:31:10.005896+0000","last_clean_scrub_stamp":"2024-09-11T07:51:41.210061+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:53.242433+0000","scrub_duration":0.0037170250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":38,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":469,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383495+0000","last_change":"2024-09-11T07:51:42.925663+0000","last_active":"2024-09-11T07:52:31.383495+0000","last_peered":"2024-09-11T07:52:31.383495+0000","last_clean":"2024-09-11T07:52:31.383495+0000","last_became_active":"2024-09-11T07:51:14.062395+0000","last_became_peered":"2024-09-11T07:51:14.062395+0000","last_unstale":"2024-09-11T07:52:31.383495+0000","last_undegraded":"2024-09-11T07:52:31.383495+0000","last_fullsized":"2024-09-11T07:52:31.383495+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:42.925597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:51:42.925597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:43.536725+0000","scrub_duration":0.0014936229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"528'18","reported_seq":127,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:39.825250+0000","last_change":"2024-09-11T07:52:18.341124+0000","last_active":"2024-09-11T07:52:39.825250+0000","last_peered":"2024-09-11T07:52:39.825250+0000","last_clean":"2024-09-11T07:52:39.825250+0000","last_became_active":"2024-09-11T07:51:34.324704+0000","last_became_peered":"2024-09-11T07:51:34.324704+0000","last_unstale":"2024-09-11T07:52:39.825250+0000","last_undegraded":"2024-09-11T07:52:39.825250+0000","last_fullsized":"2024-09-11T07:52:39.825250+0000","mapping_epoch":509,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":510,"parent":"0.0","parent_split_bits":0,"last_scrub":"528'18","last_scrub_stamp":"2024-09-11T07:52:18.341070+0000","last_deep_scrub":"528'18","last_deep_scrub_stamp":"2024-09-11T07:52:18.341070+0000","last_clean_scrub_stamp":"2024-09-11T07:52:18.341070+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:30.963958+0000","scrub_duration":0.002956728,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":92,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.253394+0000","last_change":"2024-09-11T07:52:16.224361+0000","last_active":"2024-09-11T07:52:31.253394+0000","last_peered":"2024-09-11T07:52:31.253394+0000","last_clean":"2024-09-11T07:52:31.253394+0000","last_became_active":"2024-09-11T07:51:09.045883+0000","last_became_peered":"2024-09-11T07:51:09.045883+0000","last_unstale":"2024-09-11T07:52:31.253394+0000","last_undegraded":"2024-09-11T07:52:31.253394+0000","last_fullsized":"2024-09-11T07:52:31.253394+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:16.224291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:52:16.224291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:22.931969+0000","scrub_duration":0.0017219640000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"87'81","reported_seq":45,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.384111+0000","last_change":"2024-09-11T07:51:51.553544+0000","last_active":"2024-09-11T07:52:31.384111+0000","last_peered":"2024-09-11T07:52:31.384111+0000","last_clean":"2024-09-11T07:52:31.384111+0000","last_became_active":"2024-09-11T07:51:37.864802+0000","last_became_peered":"2024-09-11T07:51:37.864802+0000","last_unstale":"2024-09-11T07:52:31.384111+0000","last_undegraded":"2024-09-11T07:52:31.384111+0000","last_fullsized":"2024-09-11T07:52:31.384111+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:51:51.553432+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:51:51.553432+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:14.493316+0000","scrub_duration":0.0017339759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"426'46","reported_seq":414,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383977+0000","last_change":"2024-09-11T07:51:14.061112+0000","last_active":"2024-09-11T07:52:31.383977+0000","last_peered":"2024-09-11T07:52:31.383977+0000","last_clean":"2024-09-11T07:52:31.383977+0000","last_became_active":"2024-09-11T07:51:14.060758+0000","last_became_peered":"2024-09-11T07:51:14.060758+0000","last_unstale":"2024-09-11T07:52:31.383977+0000","last_undegraded":"2024-09-11T07:52:31.383977+0000","last_fullsized":"2024-09-11T07:52:31.383977+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'46","last_scrub_stamp":"2024-09-11T07:50:35.400695+0000","last_deep_scrub":"83'22","last_deep_scrub_stamp":"2024-09-11T07:34:04.375457+0000","last_clean_scrub_stamp":"2024-09-11T07:50:35.400695+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086024310000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":60,"num_read_kb":51,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":480,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:37.758811+0000","last_change":"2024-09-11T07:52:37.758749+0000","last_active":"2024-09-11T07:52:37.758811+0000","last_peered":"2024-09-11T07:52:37.758811+0000","last_clean":"2024-09-11T07:52:37.758811+0000","last_became_active":"2024-09-11T07:52:13.489096+0000","last_became_peered":"2024-09-11T07:52:13.489096+0000","last_unstale":"2024-09-11T07:52:37.758811+0000","last_undegraded":"2024-09-11T07:52:37.758811+0000","last_fullsized":"2024-09-11T07:52:37.758811+0000","mapping_epoch":526,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":527,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:37.758609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:27.416207+0000","last_clean_scrub_stamp":"2024-09-11T07:52:37.758609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:49.480063+0000","scrub_duration":0.0017490310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"493'12","reported_seq":88,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:34.824363+0000","last_change":"2024-09-11T07:52:10.276621+0000","last_active":"2024-09-11T07:52:34.824363+0000","last_peered":"2024-09-11T07:52:34.824363+0000","last_clean":"2024-09-11T07:52:34.824363+0000","last_became_active":"2024-09-11T07:50:50.874198+0000","last_became_peered":"2024-09-11T07:50:50.874198+0000","last_unstale":"2024-09-11T07:52:34.824363+0000","last_undegraded":"2024-09-11T07:52:34.824363+0000","last_fullsized":"2024-09-11T07:52:34.824363+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"493'12","last_scrub_stamp":"2024-09-11T07:52:10.276547+0000","last_deep_scrub":"366'9","last_deep_scrub_stamp":"2024-09-11T07:43:26.334501+0000","last_clean_scrub_stamp":"2024-09-11T07:52:10.276547+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:12.125028+0000","scrub_duration":0.0019136960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":484,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.698831+0000","last_change":"2024-09-11T07:52:35.698778+0000","last_active":"2024-09-11T07:52:35.698831+0000","last_peered":"2024-09-11T07:52:35.698831+0000","last_clean":"2024-09-11T07:52:35.698831+0000","last_became_active":"2024-09-11T07:51:37.859466+0000","last_became_peered":"2024-09-11T07:51:37.859466+0000","last_unstale":"2024-09-11T07:52:35.698831+0000","last_undegraded":"2024-09-11T07:52:35.698831+0000","last_fullsized":"2024-09-11T07:52:35.698831+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:35.698708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:31.200685+0000","last_clean_scrub_stamp":"2024-09-11T07:52:35.698708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:48.789509+0000","scrub_duration":0.001681475,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"426'32","reported_seq":97,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.253586+0000","last_change":"2024-09-11T07:51:50.090512+0000","last_active":"2024-09-11T07:52:31.253586+0000","last_peered":"2024-09-11T07:52:31.253586+0000","last_clean":"2024-09-11T07:52:31.253586+0000","last_became_active":"2024-09-11T07:51:09.046471+0000","last_became_peered":"2024-09-11T07:51:09.046471+0000","last_unstale":"2024-09-11T07:52:31.253586+0000","last_undegraded":"2024-09-11T07:52:31.253586+0000","last_fullsized":"2024-09-11T07:52:31.253586+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'32","last_scrub_stamp":"2024-09-11T07:51:50.090443+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-09-11T07:29:38.112111+0000","last_clean_scrub_stamp":"2024-09-11T07:51:50.090443+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:05.891441+0000","scrub_duration":0.0022821479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":68,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:30.861964+0000","last_change":"2024-09-11T07:52:27.676312+0000","last_active":"2024-09-11T07:52:30.861964+0000","last_peered":"2024-09-11T07:52:30.861964+0000","last_clean":"2024-09-11T07:52:30.861964+0000","last_became_active":"2024-09-11T07:50:56.904614+0000","last_became_peered":"2024-09-11T07:50:56.904614+0000","last_unstale":"2024-09-11T07:52:30.861964+0000","last_undegraded":"2024-09-11T07:52:30.861964+0000","last_fullsized":"2024-09-11T07:52:30.861964+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:27.676249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:43:24.643676+0000","last_clean_scrub_stamp":"2024-09-11T07:52:27.676249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:57.320431+0000","scrub_duration":0.0018964139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":385,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.384621+0000","last_change":"2024-09-11T07:51:37.864978+0000","last_active":"2024-09-11T07:52:31.384621+0000","last_peered":"2024-09-11T07:52:31.384621+0000","last_clean":"2024-09-11T07:52:31.384621+0000","last_became_active":"2024-09-11T07:51:37.863388+0000","last_became_peered":"2024-09-11T07:51:37.863388+0000","last_unstale":"2024-09-11T07:52:31.384621+0000","last_undegraded":"2024-09-11T07:52:31.384621+0000","last_fullsized":"2024-09-11T07:52:31.384621+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:31.195206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:19.714568+0000","last_clean_scrub_stamp":"2024-09-11T07:51:31.195206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:51.515704+0000","scrub_duration":0.0078526010000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":578,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.368923+0000","last_change":"2024-09-11T07:51:17.351548+0000","last_active":"2024-09-11T07:52:31.368923+0000","last_peered":"2024-09-11T07:52:31.368923+0000","last_clean":"2024-09-11T07:52:31.368923+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:52:31.368923+0000","last_undegraded":"2024-09-11T07:52:31.368923+0000","last_fullsized":"2024-09-11T07:52:31.368923+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:17.351485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:48:31.314305+0000","last_clean_scrub_stamp":"2024-09-11T07:51:17.351485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:37.321825+0000","scrub_duration":0.0019907700000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"87'81","reported_seq":243,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.368065+0000","last_change":"2024-09-11T07:51:57.398645+0000","last_active":"2024-09-11T07:52:31.368065+0000","last_peered":"2024-09-11T07:52:31.368065+0000","last_clean":"2024-09-11T07:52:31.368065+0000","last_became_active":"2024-09-11T07:51:48.201002+0000","last_became_peered":"2024-09-11T07:51:48.201002+0000","last_unstale":"2024-09-11T07:52:31.368065+0000","last_undegraded":"2024-09-11T07:52:31.368065+0000","last_fullsized":"2024-09-11T07:52:31.368065+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:51:57.398587+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:51:57.398587+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:16.586047+0000","scrub_duration":0.00197993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":983040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"482'49","reported_seq":79,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.252836+0000","last_change":"2024-09-11T07:51:59.199379+0000","last_active":"2024-09-11T07:52:31.252836+0000","last_peered":"2024-09-11T07:52:31.252836+0000","last_clean":"2024-09-11T07:52:31.252836+0000","last_became_active":"2024-09-11T07:50:50.859433+0000","last_became_peered":"2024-09-11T07:50:50.859433+0000","last_unstale":"2024-09-11T07:52:31.252836+0000","last_undegraded":"2024-09-11T07:52:31.252836+0000","last_fullsized":"2024-09-11T07:52:31.252836+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"482'49","last_scrub_stamp":"2024-09-11T07:51:59.199312+0000","last_deep_scrub":"482'49","last_deep_scrub_stamp":"2024-09-11T07:50:28.592061+0000","last_clean_scrub_stamp":"2024-09-11T07:51:59.199312+0000","objects_scrubbed":7,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:02.976284+0000","scrub_duration":0.0020065500000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":55,"num_write":42,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":275,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383810+0000","last_change":"2024-09-11T07:51:48.207708+0000","last_active":"2024-09-11T07:52:31.383810+0000","last_peered":"2024-09-11T07:52:31.383810+0000","last_clean":"2024-09-11T07:52:31.383810+0000","last_became_active":"2024-09-11T07:51:48.206731+0000","last_became_peered":"2024-09-11T07:51:48.206731+0000","last_unstale":"2024-09-11T07:52:31.383810+0000","last_undegraded":"2024-09-11T07:52:31.383810+0000","last_fullsized":"2024-09-11T07:52:31.383810+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:40.135512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:51:40.135512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:48.257364+0000","scrub_duration":0.0015432390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"493'11","reported_seq":932,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.544181+0000","last_change":"2024-09-11T07:52:25.442066+0000","last_active":"2024-09-11T07:52:36.544181+0000","last_peered":"2024-09-11T07:52:36.544181+0000","last_clean":"2024-09-11T07:52:36.544181+0000","last_became_active":"2024-09-11T07:50:50.854861+0000","last_became_peered":"2024-09-11T07:50:50.854861+0000","last_unstale":"2024-09-11T07:52:36.544181+0000","last_undegraded":"2024-09-11T07:52:36.544181+0000","last_fullsized":"2024-09-11T07:52:36.544181+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"493'11","last_scrub_stamp":"2024-09-11T07:52:25.442006+0000","last_deep_scrub":"395'10","last_deep_scrub_stamp":"2024-09-11T07:50:52.366462+0000","last_clean_scrub_stamp":"2024-09-11T07:52:25.442006+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:26.785729+0000","scrub_duration":0.0018460810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":68,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:30.862161+0000","last_change":"2024-09-11T07:52:28.629545+0000","last_active":"2024-09-11T07:52:30.862161+0000","last_peered":"2024-09-11T07:52:30.862161+0000","last_clean":"2024-09-11T07:52:30.862161+0000","last_became_active":"2024-09-11T07:50:56.908521+0000","last_became_peered":"2024-09-11T07:50:56.908521+0000","last_unstale":"2024-09-11T07:52:30.862161+0000","last_undegraded":"2024-09-11T07:52:30.862161+0000","last_fullsized":"2024-09-11T07:52:30.862161+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:28.629478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:52:28.629478+0000","last_clean_scrub_stamp":"2024-09-11T07:52:28.629478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:50.098037+0000","scrub_duration":0.001653182,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"87'81","reported_seq":83,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.253395+0000","last_change":"2024-09-11T07:52:27.262912+0000","last_active":"2024-09-11T07:52:31.253395+0000","last_peered":"2024-09-11T07:52:31.253395+0000","last_clean":"2024-09-11T07:52:31.253395+0000","last_became_active":"2024-09-11T07:50:53.916379+0000","last_became_peered":"2024-09-11T07:50:53.916379+0000","last_unstale":"2024-09-11T07:52:31.253395+0000","last_undegraded":"2024-09-11T07:52:31.253395+0000","last_fullsized":"2024-09-11T07:52:31.253395+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:52:27.262822+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:50:58.031293+0000","last_clean_scrub_stamp":"2024-09-11T07:52:27.262822+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:52.165620+0000","scrub_duration":0.0015215350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"426'50","reported_seq":94,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.367760+0000","last_change":"2024-09-11T07:51:49.603296+0000","last_active":"2024-09-11T07:52:31.367760+0000","last_peered":"2024-09-11T07:52:31.367760+0000","last_clean":"2024-09-11T07:52:31.367760+0000","last_became_active":"2024-09-11T07:51:48.199106+0000","last_became_peered":"2024-09-11T07:51:48.199106+0000","last_unstale":"2024-09-11T07:52:31.367760+0000","last_undegraded":"2024-09-11T07:52:31.367760+0000","last_fullsized":"2024-09-11T07:52:31.367760+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'50","last_scrub_stamp":"2024-09-11T07:51:49.603219+0000","last_deep_scrub":"234'38","last_deep_scrub_stamp":"2024-09-11T07:46:26.243126+0000","last_clean_scrub_stamp":"2024-09-11T07:51:49.603219+0000","objects_scrubbed":8,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:57.505566+0000","scrub_duration":0.002432479,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":69,"num_read_kb":57,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1100,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":74,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383702+0000","last_change":"2024-09-11T07:51:21.084707+0000","last_active":"2024-09-11T07:52:31.383702+0000","last_peered":"2024-09-11T07:52:31.383702+0000","last_clean":"2024-09-11T07:52:31.383702+0000","last_became_active":"2024-09-11T07:51:14.059556+0000","last_became_peered":"2024-09-11T07:51:14.059556+0000","last_unstale":"2024-09-11T07:52:31.383702+0000","last_undegraded":"2024-09-11T07:52:31.383702+0000","last_fullsized":"2024-09-11T07:52:31.383702+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:21.084654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:51:21.084654+0000","last_clean_scrub_stamp":"2024-09-11T07:51:21.084654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016110300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":73,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.252910+0000","last_change":"2024-09-11T07:51:48.196635+0000","last_active":"2024-09-11T07:52:31.252910+0000","last_peered":"2024-09-11T07:52:31.252910+0000","last_clean":"2024-09-11T07:52:31.252910+0000","last_became_active":"2024-09-11T07:51:48.196080+0000","last_became_peered":"2024-09-11T07:51:48.196080+0000","last_unstale":"2024-09-11T07:52:31.252910+0000","last_undegraded":"2024-09-11T07:52:31.252910+0000","last_fullsized":"2024-09-11T07:52:31.252910+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:30.238175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:27.968579+0000","last_clean_scrub_stamp":"2024-09-11T07:51:30.238175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:44.121833+0000","scrub_duration":0.0028010420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":83,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.252558+0000","last_change":"2024-09-11T07:51:43.204972+0000","last_active":"2024-09-11T07:52:31.252558+0000","last_peered":"2024-09-11T07:52:31.252558+0000","last_clean":"2024-09-11T07:52:31.252558+0000","last_became_active":"2024-09-11T07:51:19.095139+0000","last_became_peered":"2024-09-11T07:51:19.095139+0000","last_unstale":"2024-09-11T07:52:31.252558+0000","last_undegraded":"2024-09-11T07:52:31.252558+0000","last_fullsized":"2024-09-11T07:52:31.252558+0000","mapping_epoch":504,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":505,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:43.204907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:29.978798+0000","last_clean_scrub_stamp":"2024-09-11T07:51:43.204907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:46.701601+0000","scrub_duration":0.0015091830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"87'81","reported_seq":129,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.384043+0000","last_change":"2024-09-11T07:51:50.745497+0000","last_active":"2024-09-11T07:52:31.384043+0000","last_peered":"2024-09-11T07:52:31.384043+0000","last_clean":"2024-09-11T07:52:31.384043+0000","last_became_active":"2024-09-11T07:51:45.187794+0000","last_became_peered":"2024-09-11T07:51:45.187794+0000","last_unstale":"2024-09-11T07:52:31.384043+0000","last_undegraded":"2024-09-11T07:52:31.384043+0000","last_fullsized":"2024-09-11T07:52:31.384043+0000","mapping_epoch":514,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":515,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:51:50.745359+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:51:50.745359+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:59.502445+0000","scrub_duration":0.0017247079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"426'42","reported_seq":734,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:34.766575+0000","last_change":"2024-09-11T07:52:34.766519+0000","last_active":"2024-09-11T07:52:34.766575+0000","last_peered":"2024-09-11T07:52:34.766575+0000","last_clean":"2024-09-11T07:52:34.766575+0000","last_became_active":"2024-09-11T07:50:56.906069+0000","last_became_peered":"2024-09-11T07:50:56.906069+0000","last_unstale":"2024-09-11T07:52:34.766575+0000","last_undegraded":"2024-09-11T07:52:34.766575+0000","last_fullsized":"2024-09-11T07:52:34.766575+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'42","last_scrub_stamp":"2024-09-11T07:52:34.766460+0000","last_deep_scrub":"236'32","last_deep_scrub_stamp":"2024-09-11T07:36:50.023909+0000","last_clean_scrub_stamp":"2024-09-11T07:52:34.766460+0000","objects_scrubbed":8,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:55.771951+0000","scrub_duration":0.002274206,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":69,"num_read_kb":55,"num_write":42,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":58,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:30.862474+0000","last_change":"2024-09-11T07:51:56.568463+0000","last_active":"2024-09-11T07:52:30.862474+0000","last_peered":"2024-09-11T07:52:30.862474+0000","last_clean":"2024-09-11T07:52:30.862474+0000","last_became_active":"2024-09-11T07:50:56.911468+0000","last_became_peered":"2024-09-11T07:50:56.911468+0000","last_unstale":"2024-09-11T07:52:30.862474+0000","last_undegraded":"2024-09-11T07:52:30.862474+0000","last_fullsized":"2024-09-11T07:52:30.862474+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:56.568389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:24.470602+0000","last_clean_scrub_stamp":"2024-09-11T07:51:56.568389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:02.603574+0000","scrub_duration":0.0017941400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":261,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383720+0000","last_change":"2024-09-11T07:52:01.856460+0000","last_active":"2024-09-11T07:52:31.383720+0000","last_peered":"2024-09-11T07:52:31.383720+0000","last_clean":"2024-09-11T07:52:31.383720+0000","last_became_active":"2024-09-11T07:51:45.190268+0000","last_became_peered":"2024-09-11T07:51:45.190268+0000","last_unstale":"2024-09-11T07:52:31.383720+0000","last_undegraded":"2024-09-11T07:52:31.383720+0000","last_fullsized":"2024-09-11T07:52:31.383720+0000","mapping_epoch":514,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":515,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:01.856392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:22.285763+0000","last_clean_scrub_stamp":"2024-09-11T07:52:01.856392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:16.324863+0000","scrub_duration":0.003168737,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":447,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383086+0000","last_change":"2024-09-11T07:51:30.893785+0000","last_active":"2024-09-11T07:52:31.383086+0000","last_peered":"2024-09-11T07:52:31.383086+0000","last_clean":"2024-09-11T07:52:31.383086+0000","last_became_active":"2024-09-11T07:51:14.057166+0000","last_became_peered":"2024-09-11T07:51:14.057166+0000","last_unstale":"2024-09-11T07:52:31.383086+0000","last_undegraded":"2024-09-11T07:52:31.383086+0000","last_fullsized":"2024-09-11T07:52:31.383086+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:30.893701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:39.723598+0000","last_clean_scrub_stamp":"2024-09-11T07:51:30.893701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:45.207509+0000","scrub_duration":0.0016288920000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"87'81","reported_seq":139,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.384344+0000","last_change":"2024-09-11T07:51:51.734366+0000","last_active":"2024-09-11T07:52:31.384344+0000","last_peered":"2024-09-11T07:52:31.384344+0000","last_clean":"2024-09-11T07:52:31.384344+0000","last_became_active":"2024-09-11T07:51:48.207499+0000","last_became_peered":"2024-09-11T07:51:48.207499+0000","last_unstale":"2024-09-11T07:52:31.384344+0000","last_undegraded":"2024-09-11T07:52:31.384344+0000","last_fullsized":"2024-09-11T07:52:31.384344+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:51:51.734264+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:51:51.734264+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:00.126696+0000","scrub_duration":0.0019810140000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"426'30","reported_seq":130,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.868394+0000","last_change":"2024-09-11T07:52:31.868326+0000","last_active":"2024-09-11T07:52:31.868394+0000","last_peered":"2024-09-11T07:52:31.868394+0000","last_clean":"2024-09-11T07:52:31.868394+0000","last_became_active":"2024-09-11T07:51:34.322813+0000","last_became_peered":"2024-09-11T07:51:34.322813+0000","last_unstale":"2024-09-11T07:52:31.868394+0000","last_undegraded":"2024-09-11T07:52:31.868394+0000","last_fullsized":"2024-09-11T07:52:31.868394+0000","mapping_epoch":509,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":510,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'30","last_scrub_stamp":"2024-09-11T07:52:31.868262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:52:31.868262+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:36.068726+0000","scrub_duration":0.0023282620000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":477,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383907+0000","last_change":"2024-09-11T07:51:16.228370+0000","last_active":"2024-09-11T07:52:31.383907+0000","last_peered":"2024-09-11T07:52:31.383907+0000","last_clean":"2024-09-11T07:52:31.383907+0000","last_became_active":"2024-09-11T07:51:14.063313+0000","last_became_peered":"2024-09-11T07:51:14.063313+0000","last_unstale":"2024-09-11T07:52:31.383907+0000","last_undegraded":"2024-09-11T07:52:31.383907+0000","last_fullsized":"2024-09-11T07:52:31.383907+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:51:16.228313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:51:16.228313+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:42.567074+0000","scrub_duration":0.001784147,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"510'10","reported_seq":939,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:34.824059+0000","last_change":"2024-09-11T07:51:39.648636+0000","last_active":"2024-09-11T07:52:34.824059+0000","last_peered":"2024-09-11T07:52:34.824059+0000","last_clean":"2024-09-11T07:52:34.824059+0000","last_became_active":"2024-09-11T07:51:34.323354+0000","last_became_peered":"2024-09-11T07:51:34.323354+0000","last_unstale":"2024-09-11T07:52:34.824059+0000","last_undegraded":"2024-09-11T07:52:34.824059+0000","last_fullsized":"2024-09-11T07:52:34.824059+0000","mapping_epoch":509,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":510,"parent":"0.0","parent_split_bits":0,"last_scrub":"510'10","last_scrub_stamp":"2024-09-11T07:51:39.648577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:51:39.648577+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:46.481278+0000","scrub_duration":0.0018664269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":618,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:30.859998+0000","last_change":"2024-09-11T07:52:02.771507+0000","last_active":"2024-09-11T07:52:30.859998+0000","last_peered":"2024-09-11T07:52:30.859998+0000","last_clean":"2024-09-11T07:52:30.859998+0000","last_became_active":"2024-09-11T07:50:53.914995+0000","last_became_peered":"2024-09-11T07:50:53.914995+0000","last_unstale":"2024-09-11T07:52:30.859998+0000","last_undegraded":"2024-09-11T07:52:30.859998+0000","last_fullsized":"2024-09-11T07:52:30.859998+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:02.771444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:50:29.549901+0000","last_clean_scrub_stamp":"2024-09-11T07:52:02.771444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:07.848581+0000","scrub_duration":0.001304195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"87'81","reported_seq":313,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383318+0000","last_change":"2024-09-11T07:51:47.611204+0000","last_active":"2024-09-11T07:52:31.383318+0000","last_peered":"2024-09-11T07:52:31.383318+0000","last_clean":"2024-09-11T07:52:31.383318+0000","last_became_active":"2024-09-11T07:51:37.856067+0000","last_became_peered":"2024-09-11T07:51:37.856067+0000","last_unstale":"2024-09-11T07:52:31.383318+0000","last_undegraded":"2024-09-11T07:52:31.383318+0000","last_fullsized":"2024-09-11T07:52:31.383318+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:51:47.611126+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:51:47.611126+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:06.463793+0000","scrub_duration":0.0016262830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"522'68","reported_seq":123,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:34.824212+0000","last_change":"2024-09-11T07:52:07.750614+0000","last_active":"2024-09-11T07:52:34.824212+0000","last_peered":"2024-09-11T07:52:34.824212+0000","last_clean":"2024-09-11T07:52:34.824212+0000","last_became_active":"2024-09-11T07:51:14.061014+0000","last_became_peered":"2024-09-11T07:51:14.061014+0000","last_unstale":"2024-09-11T07:52:34.824212+0000","last_undegraded":"2024-09-11T07:52:34.824212+0000","last_fullsized":"2024-09-11T07:52:34.824212+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"522'68","last_scrub_stamp":"2024-09-11T07:52:07.750543+0000","last_deep_scrub":"522'68","last_deep_scrub_stamp":"2024-09-11T07:52:07.750543+0000","last_clean_scrub_stamp":"2024-09-11T07:52:07.750543+0000","objects_scrubbed":10,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:29.280111+0000","scrub_duration":0.0082493559999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":82,"num_read_kb":67,"num_write":50,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":582,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383815+0000","last_change":"2024-09-11T07:51:45.558592+0000","last_active":"2024-09-11T07:52:31.383815+0000","last_peered":"2024-09-11T07:52:31.383815+0000","last_clean":"2024-09-11T07:52:31.383815+0000","last_became_active":"2024-09-11T07:51:37.864361+0000","last_became_peered":"2024-09-11T07:51:37.864361+0000","last_unstale":"2024-09-11T07:52:31.383815+0000","last_undegraded":"2024-09-11T07:52:31.383815+0000","last_fullsized":"2024-09-11T07:52:31.383815+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:45.558527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:51:45.558527+0000","last_clean_scrub_stamp":"2024-09-11T07:51:45.558527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:08.808308+0000","scrub_duration":0.00170009,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":610,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:38.818422+0000","last_change":"2024-09-11T07:52:38.818396+0000","last_active":"2024-09-11T07:52:38.818422+0000","last_peered":"2024-09-11T07:52:38.818422+0000","last_clean":"2024-09-11T07:52:38.818422+0000","last_became_active":"2024-09-11T07:50:50.861632+0000","last_became_peered":"2024-09-11T07:50:50.861632+0000","last_unstale":"2024-09-11T07:52:38.818422+0000","last_undegraded":"2024-09-11T07:52:38.818422+0000","last_fullsized":"2024-09-11T07:52:38.818422+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:38.818340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:52:38.818340+0000","last_clean_scrub_stamp":"2024-09-11T07:52:38.818340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:45.067717+0000","scrub_duration":0.0013967789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":635,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.369313+0000","last_change":"2024-09-11T07:51:19.302258+0000","last_active":"2024-09-11T07:52:31.369313+0000","last_peered":"2024-09-11T07:52:31.369313+0000","last_clean":"2024-09-11T07:52:31.369313+0000","last_became_active":"2024-09-11T07:50:56.906352+0000","last_became_peered":"2024-09-11T07:50:56.906352+0000","last_unstale":"2024-09-11T07:52:31.369313+0000","last_undegraded":"2024-09-11T07:52:31.369313+0000","last_fullsized":"2024-09-11T07:52:31.369313+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:19.302192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:51:19.302192+0000","last_clean_scrub_stamp":"2024-09-11T07:51:19.302192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:39.917906+0000","scrub_duration":0.001603826,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"87'81","reported_seq":36,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:30.861796+0000","last_change":"2024-09-11T07:52:00.554953+0000","last_active":"2024-09-11T07:52:30.861796+0000","last_peered":"2024-09-11T07:52:30.861796+0000","last_clean":"2024-09-11T07:52:30.861796+0000","last_became_active":"2024-09-11T07:51:48.197429+0000","last_became_peered":"2024-09-11T07:51:48.197429+0000","last_unstale":"2024-09-11T07:52:30.861796+0000","last_undegraded":"2024-09-11T07:52:30.861796+0000","last_fullsized":"2024-09-11T07:52:30.861796+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:52:00.554885+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:52:00.554885+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:02.952594+0000","scrub_duration":0.0016275630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"426'26","reported_seq":455,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383828+0000","last_change":"2024-09-11T07:52:24.781578+0000","last_active":"2024-09-11T07:52:31.383828+0000","last_peered":"2024-09-11T07:52:31.383828+0000","last_clean":"2024-09-11T07:52:31.383828+0000","last_became_active":"2024-09-11T07:51:14.064155+0000","last_became_peered":"2024-09-11T07:51:14.064155+0000","last_unstale":"2024-09-11T07:52:31.383828+0000","last_undegraded":"2024-09-11T07:52:31.383828+0000","last_fullsized":"2024-09-11T07:52:31.383828+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'26","last_scrub_stamp":"2024-09-11T07:52:24.781521+0000","last_deep_scrub":"144'18","last_deep_scrub_stamp":"2024-09-11T07:31:51.603378+0000","last_clean_scrub_stamp":"2024-09-11T07:52:24.781521+0000","objects_scrubbed":5,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:38.502079+0000","scrub_duration":0.0021468749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":41,"num_read_kb":30,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":1615,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:38.204740+0000","last_change":"2024-09-11T07:52:27.900661+0000","last_active":"2024-09-11T07:52:38.204740+0000","last_peered":"2024-09-11T07:52:38.204740+0000","last_clean":"2024-09-11T07:52:38.204740+0000","last_became_active":"2024-09-11T07:51:14.056679+0000","last_became_peered":"2024-09-11T07:51:14.056679+0000","last_unstale":"2024-09-11T07:52:38.204740+0000","last_undegraded":"2024-09-11T07:52:38.204740+0000","last_fullsized":"2024-09-11T07:52:38.204740+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'5","last_scrub_stamp":"2024-09-11T07:52:27.900603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:52:27.900603+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:42.347823+0000","scrub_duration":0.0017524960000000001,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1143,"num_read_kb":1138,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":61,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642146+0000","last_change":"2024-09-11T07:51:17.078520+0000","last_active":"2024-09-11T07:52:36.642146+0000","last_peered":"2024-09-11T07:52:36.642146+0000","last_clean":"2024-09-11T07:52:36.642146+0000","last_became_active":"2024-09-11T07:50:53.928925+0000","last_became_peered":"2024-09-11T07:50:53.928925+0000","last_unstale":"2024-09-11T07:52:36.642146+0000","last_undegraded":"2024-09-11T07:52:36.642146+0000","last_fullsized":"2024-09-11T07:52:36.642146+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:17.078461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:07.149367+0000","last_clean_scrub_stamp":"2024-09-11T07:51:17.078461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:43.916710+0000","scrub_duration":0.0012560749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":81,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642555+0000","last_change":"2024-09-11T07:52:08.118907+0000","last_active":"2024-09-11T07:52:36.642555+0000","last_peered":"2024-09-11T07:52:36.642555+0000","last_clean":"2024-09-11T07:52:36.642555+0000","last_became_active":"2024-09-11T07:50:56.914441+0000","last_became_peered":"2024-09-11T07:50:56.914441+0000","last_unstale":"2024-09-11T07:52:36.642555+0000","last_undegraded":"2024-09-11T07:52:36.642555+0000","last_fullsized":"2024-09-11T07:52:36.642555+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:08.118841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:52:08.118841+0000","last_clean_scrub_stamp":"2024-09-11T07:52:08.118841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:31.008978+0000","scrub_duration":0.0021706490000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":68,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.656979+0000","last_change":"2024-09-11T07:52:31.656911+0000","last_active":"2024-09-11T07:52:31.656979+0000","last_peered":"2024-09-11T07:52:31.656979+0000","last_clean":"2024-09-11T07:52:31.656979+0000","last_became_active":"2024-09-11T07:50:56.908047+0000","last_became_peered":"2024-09-11T07:50:56.908047+0000","last_unstale":"2024-09-11T07:52:31.656979+0000","last_undegraded":"2024-09-11T07:52:31.656979+0000","last_fullsized":"2024-09-11T07:52:31.656979+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:31.656792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:25.382533+0000","last_clean_scrub_stamp":"2024-09-11T07:52:31.656792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:51.434152+0000","scrub_duration":0.0017941669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"87'81","reported_seq":165,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383559+0000","last_change":"2024-09-11T07:51:57.833896+0000","last_active":"2024-09-11T07:52:31.383559+0000","last_peered":"2024-09-11T07:52:31.383559+0000","last_clean":"2024-09-11T07:52:31.383559+0000","last_became_active":"2024-09-11T07:51:14.059851+0000","last_became_peered":"2024-09-11T07:51:14.059851+0000","last_unstale":"2024-09-11T07:52:31.383559+0000","last_undegraded":"2024-09-11T07:52:31.383559+0000","last_fullsized":"2024-09-11T07:52:31.383559+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:51:57.833831+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:51:57.833831+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:13.353364+0000","scrub_duration":0.0018593209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"426'42","reported_seq":806,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:30.861309+0000","last_change":"2024-09-11T07:52:22.817089+0000","last_active":"2024-09-11T07:52:30.861309+0000","last_peered":"2024-09-11T07:52:30.861309+0000","last_clean":"2024-09-11T07:52:30.861309+0000","last_became_active":"2024-09-11T07:50:53.923727+0000","last_became_peered":"2024-09-11T07:50:53.923727+0000","last_unstale":"2024-09-11T07:52:30.861309+0000","last_undegraded":"2024-09-11T07:52:30.861309+0000","last_fullsized":"2024-09-11T07:52:30.861309+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'42","last_scrub_stamp":"2024-09-11T07:52:22.817028+0000","last_deep_scrub":"426'42","last_deep_scrub_stamp":"2024-09-11T07:52:22.817028+0000","last_clean_scrub_stamp":"2024-09-11T07:52:22.817028+0000","objects_scrubbed":6,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:47.108592+0000","scrub_duration":0.00193462,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":54,"num_read_kb":48,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"422'2","reported_seq":671,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:30.860325+0000","last_change":"2024-09-11T07:51:58.665327+0000","last_active":"2024-09-11T07:52:30.860325+0000","last_peered":"2024-09-11T07:52:30.860325+0000","last_clean":"2024-09-11T07:52:30.860325+0000","last_became_active":"2024-09-11T07:51:19.098192+0000","last_became_peered":"2024-09-11T07:51:19.098192+0000","last_unstale":"2024-09-11T07:52:30.860325+0000","last_undegraded":"2024-09-11T07:52:30.860325+0000","last_fullsized":"2024-09-11T07:52:30.860325+0000","mapping_epoch":504,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":505,"parent":"0.0","parent_split_bits":0,"last_scrub":"422'2","last_scrub_stamp":"2024-09-11T07:51:58.665247+0000","last_deep_scrub":"422'2","last_deep_scrub_stamp":"2024-09-11T07:51:58.665247+0000","last_clean_scrub_stamp":"2024-09-11T07:51:58.665247+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:25.958559+0000","scrub_duration":0.0032743450000000001,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:30.862069+0000","last_change":"2024-09-11T07:51:31.522195+0000","last_active":"2024-09-11T07:52:30.862069+0000","last_peered":"2024-09-11T07:52:30.862069+0000","last_clean":"2024-09-11T07:52:30.862069+0000","last_became_active":"2024-09-11T07:51:09.045724+0000","last_became_peered":"2024-09-11T07:51:09.045724+0000","last_unstale":"2024-09-11T07:52:30.862069+0000","last_undegraded":"2024-09-11T07:52:30.862069+0000","last_fullsized":"2024-09-11T07:52:30.862069+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:31.522132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:51:31.522132+0000","last_clean_scrub_stamp":"2024-09-11T07:51:31.522132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:57.981776+0000","scrub_duration":0.0016075250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":174,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.384542+0000","last_change":"2024-09-11T07:51:42.567022+0000","last_active":"2024-09-11T07:52:31.384542+0000","last_peered":"2024-09-11T07:52:31.384542+0000","last_clean":"2024-09-11T07:52:31.384542+0000","last_became_active":"2024-09-11T07:51:37.859118+0000","last_became_peered":"2024-09-11T07:51:37.859118+0000","last_unstale":"2024-09-11T07:52:31.384542+0000","last_undegraded":"2024-09-11T07:52:31.384542+0000","last_fullsized":"2024-09-11T07:52:31.384542+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:42.566952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:51.187127+0000","last_clean_scrub_stamp":"2024-09-11T07:51:42.566952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:47.068464+0000","scrub_duration":0.001607027,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":156,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383482+0000","last_change":"2024-09-11T07:52:06.786372+0000","last_active":"2024-09-11T07:52:31.383482+0000","last_peered":"2024-09-11T07:52:31.383482+0000","last_clean":"2024-09-11T07:52:31.383482+0000","last_became_active":"2024-09-11T07:51:14.066002+0000","last_became_peered":"2024-09-11T07:51:14.066002+0000","last_unstale":"2024-09-11T07:52:31.383482+0000","last_undegraded":"2024-09-11T07:52:31.383482+0000","last_fullsized":"2024-09-11T07:52:31.383482+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:06.786303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:52:06.786303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:33.770498+0000","scrub_duration":0.001625041,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"426'45","reported_seq":206,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.384128+0000","last_change":"2024-09-11T07:52:27.809409+0000","last_active":"2024-09-11T07:52:31.384128+0000","last_peered":"2024-09-11T07:52:31.384128+0000","last_clean":"2024-09-11T07:52:31.384128+0000","last_became_active":"2024-09-11T07:51:45.190817+0000","last_became_peered":"2024-09-11T07:51:45.190817+0000","last_unstale":"2024-09-11T07:52:31.384128+0000","last_undegraded":"2024-09-11T07:52:31.384128+0000","last_fullsized":"2024-09-11T07:52:31.384128+0000","mapping_epoch":514,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":515,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'45","last_scrub_stamp":"2024-09-11T07:52:27.809340+0000","last_deep_scrub":"399'33","last_deep_scrub_stamp":"2024-09-11T07:45:32.442160+0000","last_clean_scrub_stamp":"2024-09-11T07:52:27.809340+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:52.210963+0000","scrub_duration":0.0021207800000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":58,"num_read_kb":48,"num_write":37,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'3","reported_seq":658,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.368423+0000","last_change":"2024-09-11T07:52:27.449877+0000","last_active":"2024-09-11T07:52:31.368423+0000","last_peered":"2024-09-11T07:52:31.368423+0000","last_clean":"2024-09-11T07:52:31.368423+0000","last_became_active":"2024-09-11T07:51:34.321017+0000","last_became_peered":"2024-09-11T07:51:34.321017+0000","last_unstale":"2024-09-11T07:52:31.368423+0000","last_undegraded":"2024-09-11T07:52:31.368423+0000","last_fullsized":"2024-09-11T07:52:31.368423+0000","mapping_epoch":509,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":510,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-09-11T07:52:27.449814+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-09-11T07:51:03.338306+0000","last_clean_scrub_stamp":"2024-09-11T07:52:27.449814+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:37.698089+0000","scrub_duration":0.0020273040000000002,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":527,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383953+0000","last_change":"2024-09-11T07:51:48.206709+0000","last_active":"2024-09-11T07:52:31.383953+0000","last_peered":"2024-09-11T07:52:31.383953+0000","last_clean":"2024-09-11T07:52:31.383953+0000","last_became_active":"2024-09-11T07:51:48.206358+0000","last_became_peered":"2024-09-11T07:51:48.206358+0000","last_unstale":"2024-09-11T07:52:31.383953+0000","last_undegraded":"2024-09-11T07:52:31.383953+0000","last_fullsized":"2024-09-11T07:52:31.383953+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:19.076548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:51:19.076548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:46.529907+0000","scrub_duration":0.0073747420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"87'81","reported_seq":36,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:30.861550+0000","last_change":"2024-09-11T07:51:54.557199+0000","last_active":"2024-09-11T07:52:30.861550+0000","last_peered":"2024-09-11T07:52:30.861550+0000","last_clean":"2024-09-11T07:52:30.861550+0000","last_became_active":"2024-09-11T07:51:48.199620+0000","last_became_peered":"2024-09-11T07:51:48.199620+0000","last_unstale":"2024-09-11T07:52:30.861550+0000","last_undegraded":"2024-09-11T07:52:30.861550+0000","last_fullsized":"2024-09-11T07:52:30.861550+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:51:54.557128+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:51:54.557128+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:20.283442+0000","scrub_duration":0.0019394849999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":26,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:30.862058+0000","last_change":"2024-09-11T07:51:48.202466+0000","last_active":"2024-09-11T07:52:30.862058+0000","last_peered":"2024-09-11T07:52:30.862058+0000","last_clean":"2024-09-11T07:52:30.862058+0000","last_became_active":"2024-09-11T07:51:48.202154+0000","last_became_peered":"2024-09-11T07:51:48.202154+0000","last_unstale":"2024-09-11T07:52:30.862058+0000","last_undegraded":"2024-09-11T07:52:30.862058+0000","last_fullsized":"2024-09-11T07:52:30.862058+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:37.210850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:51:37.210850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:57.124781+0000","scrub_duration":0.001335276,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":567,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.384661+0000","last_change":"2024-09-11T07:52:27.719342+0000","last_active":"2024-09-11T07:52:31.384661+0000","last_peered":"2024-09-11T07:52:31.384661+0000","last_clean":"2024-09-11T07:52:31.384661+0000","last_became_active":"2024-09-11T07:51:37.859778+0000","last_became_peered":"2024-09-11T07:51:37.859778+0000","last_unstale":"2024-09-11T07:52:31.384661+0000","last_undegraded":"2024-09-11T07:52:31.384661+0000","last_fullsized":"2024-09-11T07:52:31.384661+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:27.719279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:52:27.719279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:29.032317+0000","scrub_duration":0.0017284430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"426'51","reported_seq":299,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383509+0000","last_change":"2024-09-11T07:51:48.225263+0000","last_active":"2024-09-11T07:52:31.383509+0000","last_peered":"2024-09-11T07:52:31.383509+0000","last_clean":"2024-09-11T07:52:31.383509+0000","last_became_active":"2024-09-11T07:51:48.206795+0000","last_became_peered":"2024-09-11T07:51:48.206795+0000","last_unstale":"2024-09-11T07:52:31.383509+0000","last_undegraded":"2024-09-11T07:52:31.383509+0000","last_fullsized":"2024-09-11T07:52:31.383509+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'51","last_scrub_stamp":"2024-09-11T07:51:43.167159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:51:43.167159+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:46.886452+0000","scrub_duration":0.0022568250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":78,"num_read_kb":67,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,7],"acting":[6,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":61,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642431+0000","last_change":"2024-09-11T07:51:23.166604+0000","last_active":"2024-09-11T07:52:36.642431+0000","last_peered":"2024-09-11T07:52:36.642431+0000","last_clean":"2024-09-11T07:52:36.642431+0000","last_became_active":"2024-09-11T07:50:53.929557+0000","last_became_peered":"2024-09-11T07:50:53.929557+0000","last_unstale":"2024-09-11T07:52:36.642431+0000","last_undegraded":"2024-09-11T07:52:36.642431+0000","last_fullsized":"2024-09-11T07:52:36.642431+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:23.166543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:51:23.166543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:50.976076+0000","scrub_duration":0.0019013649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":91,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642153+0000","last_change":"2024-09-11T07:52:07.082580+0000","last_active":"2024-09-11T07:52:36.642153+0000","last_peered":"2024-09-11T07:52:36.642153+0000","last_clean":"2024-09-11T07:52:36.642153+0000","last_became_active":"2024-09-11T07:51:34.324125+0000","last_became_peered":"2024-09-11T07:51:34.324125+0000","last_unstale":"2024-09-11T07:52:36.642153+0000","last_undegraded":"2024-09-11T07:52:36.642153+0000","last_fullsized":"2024-09-11T07:52:36.642153+0000","mapping_epoch":509,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":510,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:07.082507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:51:00.116883+0000","last_clean_scrub_stamp":"2024-09-11T07:52:07.082507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:30.795023+0000","scrub_duration":0.0019556529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":36,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:30.861555+0000","last_change":"2024-09-11T07:52:10.480698+0000","last_active":"2024-09-11T07:52:30.861555+0000","last_peered":"2024-09-11T07:52:30.861555+0000","last_clean":"2024-09-11T07:52:30.861555+0000","last_became_active":"2024-09-11T07:51:48.202773+0000","last_became_peered":"2024-09-11T07:51:48.202773+0000","last_unstale":"2024-09-11T07:52:30.861555+0000","last_undegraded":"2024-09-11T07:52:30.861555+0000","last_fullsized":"2024-09-11T07:52:30.861555+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:10.480638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:52:10.480638+0000","last_clean_scrub_stamp":"2024-09-11T07:52:10.480638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:38.526861+0000","scrub_duration":0.0031260929999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"426'21","reported_seq":656,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:30.860665+0000","last_change":"2024-09-11T07:51:27.765216+0000","last_active":"2024-09-11T07:52:30.860665+0000","last_peered":"2024-09-11T07:52:30.860665+0000","last_clean":"2024-09-11T07:52:30.860665+0000","last_became_active":"2024-09-11T07:50:50.861614+0000","last_became_peered":"2024-09-11T07:50:50.861614+0000","last_unstale":"2024-09-11T07:52:30.860665+0000","last_undegraded":"2024-09-11T07:52:30.860665+0000","last_fullsized":"2024-09-11T07:52:30.860665+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'21","last_scrub_stamp":"2024-09-11T07:51:27.765140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:51:27.765140+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:47.142359+0000","scrub_duration":0.002239245,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":460,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383994+0000","last_change":"2024-09-11T07:51:58.882854+0000","last_active":"2024-09-11T07:52:31.383994+0000","last_peered":"2024-09-11T07:52:31.383994+0000","last_clean":"2024-09-11T07:52:31.383994+0000","last_became_active":"2024-09-11T07:51:14.060150+0000","last_became_peered":"2024-09-11T07:51:14.060150+0000","last_unstale":"2024-09-11T07:52:31.383994+0000","last_undegraded":"2024-09-11T07:52:31.383994+0000","last_fullsized":"2024-09-11T07:52:31.383994+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:58.882784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:51:58.882784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:05.245534+0000","scrub_duration":0.0016724590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":71,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642193+0000","last_change":"2024-09-11T07:52:28.328123+0000","last_active":"2024-09-11T07:52:36.642193+0000","last_peered":"2024-09-11T07:52:36.642193+0000","last_clean":"2024-09-11T07:52:36.642193+0000","last_became_active":"2024-09-11T07:50:53.919253+0000","last_became_peered":"2024-09-11T07:50:53.919253+0000","last_unstale":"2024-09-11T07:52:36.642193+0000","last_undegraded":"2024-09-11T07:52:36.642193+0000","last_fullsized":"2024-09-11T07:52:36.642193+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:28.328056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:52:28.328056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:58.262347+0000","scrub_duration":0.001878443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":635,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:30.859625+0000","last_change":"2024-09-11T07:51:29.724652+0000","last_active":"2024-09-11T07:52:30.859625+0000","last_peered":"2024-09-11T07:52:30.859625+0000","last_clean":"2024-09-11T07:52:30.859625+0000","last_became_active":"2024-09-11T07:50:56.913071+0000","last_became_peered":"2024-09-11T07:50:56.913071+0000","last_unstale":"2024-09-11T07:52:30.859625+0000","last_undegraded":"2024-09-11T07:52:30.859625+0000","last_fullsized":"2024-09-11T07:52:30.859625+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:29.724592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:51:29.724592+0000","last_clean_scrub_stamp":"2024-09-11T07:51:29.724592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:37.620521+0000","scrub_duration":0.0020246410000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"426'46","reported_seq":117,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383296+0000","last_change":"2024-09-11T07:51:48.220627+0000","last_active":"2024-09-11T07:52:31.383296+0000","last_peered":"2024-09-11T07:52:31.383296+0000","last_clean":"2024-09-11T07:52:31.383296+0000","last_became_active":"2024-09-11T07:51:48.206628+0000","last_became_peered":"2024-09-11T07:51:48.206628+0000","last_unstale":"2024-09-11T07:52:31.383296+0000","last_undegraded":"2024-09-11T07:52:31.383296+0000","last_fullsized":"2024-09-11T07:52:31.383296+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'46","last_scrub_stamp":"2024-09-11T07:51:46.603659+0000","last_deep_scrub":"421'34","last_deep_scrub_stamp":"2024-09-11T07:46:14.474174+0000","last_clean_scrub_stamp":"2024-09-11T07:51:46.603659+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:48.799771+0000","scrub_duration":0.0020717209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":91,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642498+0000","last_change":"2024-09-11T07:52:31.370748+0000","last_active":"2024-09-11T07:52:36.642498+0000","last_peered":"2024-09-11T07:52:36.642498+0000","last_clean":"2024-09-11T07:52:36.642498+0000","last_became_active":"2024-09-11T07:51:48.201810+0000","last_became_peered":"2024-09-11T07:51:48.201810+0000","last_unstale":"2024-09-11T07:52:36.642498+0000","last_undegraded":"2024-09-11T07:52:36.642498+0000","last_fullsized":"2024-09-11T07:52:36.642498+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:31.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:51:07.108424+0000","last_clean_scrub_stamp":"2024-09-11T07:52:31.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:53.993217+0000","scrub_duration":0.0044957570000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"496'16","reported_seq":92,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:34.824296+0000","last_change":"2024-09-11T07:52:25.691754+0000","last_active":"2024-09-11T07:52:34.824296+0000","last_peered":"2024-09-11T07:52:34.824296+0000","last_clean":"2024-09-11T07:52:34.824296+0000","last_became_active":"2024-09-11T07:50:56.909842+0000","last_became_peered":"2024-09-11T07:50:56.909842+0000","last_unstale":"2024-09-11T07:52:34.824296+0000","last_undegraded":"2024-09-11T07:52:34.824296+0000","last_fullsized":"2024-09-11T07:52:34.824296+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"496'16","last_scrub_stamp":"2024-09-11T07:52:25.691683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:52:25.691683+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:53.473430+0000","scrub_duration":0.001732578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":386,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.384011+0000","last_change":"2024-09-11T07:51:37.865997+0000","last_active":"2024-09-11T07:52:31.384011+0000","last_peered":"2024-09-11T07:52:31.384011+0000","last_clean":"2024-09-11T07:52:31.384011+0000","last_became_active":"2024-09-11T07:51:37.865205+0000","last_became_peered":"2024-09-11T07:51:37.865205+0000","last_unstale":"2024-09-11T07:52:31.384011+0000","last_undegraded":"2024-09-11T07:52:31.384011+0000","last_fullsized":"2024-09-11T07:52:31.384011+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:36.572101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:48:43.848057+0000","last_clean_scrub_stamp":"2024-09-11T07:51:36.572101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:48.841195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"426'29","reported_seq":97,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.252742+0000","last_change":"2024-09-11T07:51:57.164629+0000","last_active":"2024-09-11T07:52:31.252742+0000","last_peered":"2024-09-11T07:52:31.252742+0000","last_clean":"2024-09-11T07:52:31.252742+0000","last_became_active":"2024-09-11T07:51:19.095126+0000","last_became_peered":"2024-09-11T07:51:19.095126+0000","last_unstale":"2024-09-11T07:52:31.252742+0000","last_undegraded":"2024-09-11T07:52:31.252742+0000","last_fullsized":"2024-09-11T07:52:31.252742+0000","mapping_epoch":504,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":505,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'29","last_scrub_stamp":"2024-09-11T07:51:57.164526+0000","last_deep_scrub":"234'23","last_deep_scrub_stamp":"2024-09-11T07:37:34.613280+0000","last_clean_scrub_stamp":"2024-09-11T07:51:57.164526+0000","objects_scrubbed":5,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:05.705415+0000","scrub_duration":0.0022382869999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":95,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.253705+0000","last_change":"2024-09-11T07:52:25.270117+0000","last_active":"2024-09-11T07:52:31.253705+0000","last_peered":"2024-09-11T07:52:31.253705+0000","last_clean":"2024-09-11T07:52:31.253705+0000","last_became_active":"2024-09-11T07:50:56.913432+0000","last_became_peered":"2024-09-11T07:50:56.913432+0000","last_unstale":"2024-09-11T07:52:31.253705+0000","last_undegraded":"2024-09-11T07:52:31.253705+0000","last_fullsized":"2024-09-11T07:52:31.253705+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:52:25.270048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:52:25.270048+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:25.575579+0000","scrub_duration":0.0020442350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":71,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.641907+0000","last_change":"2024-09-11T07:52:22.354266+0000","last_active":"2024-09-11T07:52:36.641907+0000","last_peered":"2024-09-11T07:52:36.641907+0000","last_clean":"2024-09-11T07:52:36.641907+0000","last_became_active":"2024-09-11T07:50:53.912926+0000","last_became_peered":"2024-09-11T07:50:53.912926+0000","last_unstale":"2024-09-11T07:52:36.641907+0000","last_undegraded":"2024-09-11T07:52:36.641907+0000","last_fullsized":"2024-09-11T07:52:36.641907+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:22.354174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:48:30.290136+0000","last_clean_scrub_stamp":"2024-09-11T07:52:22.354174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:28.702842+0000","scrub_duration":0.002140145,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":625,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.368311+0000","last_change":"2024-09-11T07:52:24.474104+0000","last_active":"2024-09-11T07:52:31.368311+0000","last_peered":"2024-09-11T07:52:31.368311+0000","last_clean":"2024-09-11T07:52:31.368311+0000","last_became_active":"2024-09-11T07:51:48.202842+0000","last_became_peered":"2024-09-11T07:51:48.202842+0000","last_unstale":"2024-09-11T07:52:31.368311+0000","last_undegraded":"2024-09-11T07:52:31.368311+0000","last_fullsized":"2024-09-11T07:52:31.368311+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:24.474046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:27.626761+0000","last_clean_scrub_stamp":"2024-09-11T07:52:24.474046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:49.199583+0000","scrub_duration":0.0017009080000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"426'25","reported_seq":737,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:30.861410+0000","last_change":"2024-09-11T07:51:48.210843+0000","last_active":"2024-09-11T07:52:30.861410+0000","last_peered":"2024-09-11T07:52:30.861410+0000","last_clean":"2024-09-11T07:52:30.861410+0000","last_became_active":"2024-09-11T07:51:48.197552+0000","last_became_peered":"2024-09-11T07:51:48.197552+0000","last_unstale":"2024-09-11T07:52:30.861410+0000","last_undegraded":"2024-09-11T07:52:30.861410+0000","last_fullsized":"2024-09-11T07:52:30.861410+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'25","last_scrub_stamp":"2024-09-11T07:51:11.679256+0000","last_deep_scrub":"426'25","last_deep_scrub_stamp":"2024-09-11T07:47:22.197145+0000","last_clean_scrub_stamp":"2024-09-11T07:51:11.679256+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:38.771876+0000","scrub_duration":0.0020626199999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":61,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.641984+0000","last_change":"2024-09-11T07:51:56.057998+0000","last_active":"2024-09-11T07:52:36.641984+0000","last_peered":"2024-09-11T07:52:36.641984+0000","last_clean":"2024-09-11T07:52:36.641984+0000","last_became_active":"2024-09-11T07:50:53.913208+0000","last_became_peered":"2024-09-11T07:50:53.913208+0000","last_unstale":"2024-09-11T07:52:36.641984+0000","last_undegraded":"2024-09-11T07:52:36.641984+0000","last_fullsized":"2024-09-11T07:52:36.641984+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:56.057935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:43:26.338227+0000","last_clean_scrub_stamp":"2024-09-11T07:51:56.057935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:10.881674+0000","scrub_duration":0.001460434,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":38,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.367652+0000","last_change":"2024-09-11T07:51:19.098675+0000","last_active":"2024-09-11T07:52:31.367652+0000","last_peered":"2024-09-11T07:52:31.367652+0000","last_clean":"2024-09-11T07:52:31.367652+0000","last_became_active":"2024-09-11T07:51:19.098400+0000","last_became_peered":"2024-09-11T07:51:19.098400+0000","last_unstale":"2024-09-11T07:52:31.367652+0000","last_undegraded":"2024-09-11T07:52:31.367652+0000","last_fullsized":"2024-09-11T07:52:31.367652+0000","mapping_epoch":504,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":505,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:12.504549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:35.392360+0000","last_clean_scrub_stamp":"2024-09-11T07:51:12.504549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0084664969999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":79,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642226+0000","last_change":"2024-09-11T07:51:20.118837+0000","last_active":"2024-09-11T07:52:36.642226+0000","last_peered":"2024-09-11T07:52:36.642226+0000","last_clean":"2024-09-11T07:52:36.642226+0000","last_became_active":"2024-09-11T07:50:56.915847+0000","last_became_peered":"2024-09-11T07:50:56.915847+0000","last_unstale":"2024-09-11T07:52:36.642226+0000","last_undegraded":"2024-09-11T07:52:36.642226+0000","last_fullsized":"2024-09-11T07:52:36.642226+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:51:20.118777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:51:20.118777+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:43.943149+0000","scrub_duration":0.002056127,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":39,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":410,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383208+0000","last_change":"2024-09-11T07:51:48.704390+0000","last_active":"2024-09-11T07:52:31.383208+0000","last_peered":"2024-09-11T07:52:31.383208+0000","last_clean":"2024-09-11T07:52:31.383208+0000","last_became_active":"2024-09-11T07:51:45.420950+0000","last_became_peered":"2024-09-11T07:51:45.420950+0000","last_unstale":"2024-09-11T07:52:31.383208+0000","last_undegraded":"2024-09-11T07:52:31.383208+0000","last_fullsized":"2024-09-11T07:52:31.383208+0000","mapping_epoch":514,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":515,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:48.704324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:51:48.704324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:17.400227+0000","scrub_duration":0.001362526,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"426'21","reported_seq":74,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.369133+0000","last_change":"2024-09-11T07:51:48.212742+0000","last_active":"2024-09-11T07:52:31.369133+0000","last_peered":"2024-09-11T07:52:31.369133+0000","last_clean":"2024-09-11T07:52:31.369133+0000","last_became_active":"2024-09-11T07:51:48.199537+0000","last_became_peered":"2024-09-11T07:51:48.199537+0000","last_unstale":"2024-09-11T07:52:31.369133+0000","last_undegraded":"2024-09-11T07:52:31.369133+0000","last_fullsized":"2024-09-11T07:52:31.369133+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'21","last_scrub_stamp":"2024-09-11T07:51:26.373238+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:51:26.373238+0000","objects_scrubbed":3,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:51.233199+0000","scrub_duration":0.001756051,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":25,"num_read_kb":22,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":513,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.384633+0000","last_change":"2024-09-11T07:51:37.864171+0000","last_active":"2024-09-11T07:52:31.384633+0000","last_peered":"2024-09-11T07:52:31.384633+0000","last_clean":"2024-09-11T07:52:31.384633+0000","last_became_active":"2024-09-11T07:51:37.863762+0000","last_became_peered":"2024-09-11T07:51:37.863762+0000","last_unstale":"2024-09-11T07:52:31.384633+0000","last_undegraded":"2024-09-11T07:52:31.384633+0000","last_fullsized":"2024-09-11T07:52:31.384633+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:34.438728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:51:34.438728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:43.984479+0000","scrub_duration":0.0069828010000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":68,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:37.626169+0000","last_change":"2024-09-11T07:52:37.626132+0000","last_active":"2024-09-11T07:52:37.626169+0000","last_peered":"2024-09-11T07:52:37.626169+0000","last_clean":"2024-09-11T07:52:37.626169+0000","last_became_active":"2024-09-11T07:50:56.912212+0000","last_became_peered":"2024-09-11T07:50:56.912212+0000","last_unstale":"2024-09-11T07:52:37.626169+0000","last_undegraded":"2024-09-11T07:52:37.626169+0000","last_fullsized":"2024-09-11T07:52:37.626169+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:37.626067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:52:37.626067+0000","last_clean_scrub_stamp":"2024-09-11T07:52:37.626067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:44.277193+0000","scrub_duration":0.001561406,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"426'35","reported_seq":791,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.572573+0000","last_change":"2024-09-11T07:52:31.572517+0000","last_active":"2024-09-11T07:52:31.572573+0000","last_peered":"2024-09-11T07:52:31.572573+0000","last_clean":"2024-09-11T07:52:31.572573+0000","last_became_active":"2024-09-11T07:51:09.045315+0000","last_became_peered":"2024-09-11T07:51:09.045315+0000","last_unstale":"2024-09-11T07:52:31.572573+0000","last_undegraded":"2024-09-11T07:52:31.572573+0000","last_fullsized":"2024-09-11T07:52:31.572573+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'35","last_scrub_stamp":"2024-09-11T07:52:31.572440+0000","last_deep_scrub":"234'25","last_deep_scrub_stamp":"2024-09-11T07:44:29.610392+0000","last_clean_scrub_stamp":"2024-09-11T07:52:31.572440+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:56.638332+0000","scrub_duration":0.002231402,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":537,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383813+0000","last_change":"2024-09-11T07:51:48.203783+0000","last_active":"2024-09-11T07:52:31.383813+0000","last_peered":"2024-09-11T07:52:31.383813+0000","last_clean":"2024-09-11T07:52:31.383813+0000","last_became_active":"2024-09-11T07:51:48.203478+0000","last_became_peered":"2024-09-11T07:51:48.203478+0000","last_unstale":"2024-09-11T07:52:31.383813+0000","last_undegraded":"2024-09-11T07:52:31.383813+0000","last_fullsized":"2024-09-11T07:52:31.383813+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:44.578359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:51:44.578359+0000","last_clean_scrub_stamp":"2024-09-11T07:51:44.578359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:04.793452+0000","scrub_duration":0.0018232980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"512'11","reported_seq":753,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:34.824365+0000","last_change":"2024-09-11T07:52:14.603918+0000","last_active":"2024-09-11T07:52:34.824365+0000","last_peered":"2024-09-11T07:52:34.824365+0000","last_clean":"2024-09-11T07:52:34.824365+0000","last_became_active":"2024-09-11T07:51:37.860270+0000","last_became_peered":"2024-09-11T07:51:37.860270+0000","last_unstale":"2024-09-11T07:52:34.824365+0000","last_undegraded":"2024-09-11T07:52:34.824365+0000","last_fullsized":"2024-09-11T07:52:34.824365+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":0,"last_scrub":"512'11","last_scrub_stamp":"2024-09-11T07:52:14.603845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:52:14.603845+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:26.791335+0000","scrub_duration":0.001773997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":655,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.764258+0000","last_change":"2024-09-11T07:52:31.764190+0000","last_active":"2024-09-11T07:52:31.764258+0000","last_peered":"2024-09-11T07:52:31.764258+0000","last_clean":"2024-09-11T07:52:31.764258+0000","last_became_active":"2024-09-11T07:51:48.196907+0000","last_became_peered":"2024-09-11T07:51:48.196907+0000","last_unstale":"2024-09-11T07:52:31.764258+0000","last_undegraded":"2024-09-11T07:52:31.764258+0000","last_fullsized":"2024-09-11T07:52:31.764258+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:31.764120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:52:31.764120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:50.660752+0000","scrub_duration":0.001894632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"426'25","reported_seq":80,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.384140+0000","last_change":"2024-09-11T07:51:53.584531+0000","last_active":"2024-09-11T07:52:31.384140+0000","last_peered":"2024-09-11T07:52:31.384140+0000","last_clean":"2024-09-11T07:52:31.384140+0000","last_became_active":"2024-09-11T07:51:37.860023+0000","last_became_peered":"2024-09-11T07:51:37.860023+0000","last_unstale":"2024-09-11T07:52:31.384140+0000","last_undegraded":"2024-09-11T07:52:31.384140+0000","last_fullsized":"2024-09-11T07:52:31.384140+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'25","last_scrub_stamp":"2024-09-11T07:51:53.584457+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:37:41.338036+0000","last_clean_scrub_stamp":"2024-09-11T07:51:53.584457+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:04.168037+0000","scrub_duration":0.0022322000000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":617,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:30.861159+0000","last_change":"2024-09-11T07:52:28.806350+0000","last_active":"2024-09-11T07:52:30.861159+0000","last_peered":"2024-09-11T07:52:30.861159+0000","last_clean":"2024-09-11T07:52:30.861159+0000","last_became_active":"2024-09-11T07:50:50.863004+0000","last_became_peered":"2024-09-11T07:50:50.863004+0000","last_unstale":"2024-09-11T07:52:30.861159+0000","last_undegraded":"2024-09-11T07:52:30.861159+0000","last_fullsized":"2024-09-11T07:52:30.861159+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:28.806276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:52:28.806276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:44.445398+0000","scrub_duration":0.001691641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"505'17","reported_seq":69,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.544680+0000","last_change":"2024-09-11T07:52:22.442125+0000","last_active":"2024-09-11T07:52:36.544680+0000","last_peered":"2024-09-11T07:52:36.544680+0000","last_clean":"2024-09-11T07:52:36.544680+0000","last_became_active":"2024-09-11T07:51:19.099800+0000","last_became_peered":"2024-09-11T07:51:19.099800+0000","last_unstale":"2024-09-11T07:52:36.544680+0000","last_undegraded":"2024-09-11T07:52:36.544680+0000","last_fullsized":"2024-09-11T07:52:36.544680+0000","mapping_epoch":504,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":505,"parent":"0.0","parent_split_bits":0,"last_scrub":"505'17","last_scrub_stamp":"2024-09-11T07:52:22.442069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:52:22.442069+0000","objects_scrubbed":1,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:34.832077+0000","scrub_duration":0.0026387569999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":73,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.253611+0000","last_change":"2024-09-11T07:51:47.176810+0000","last_active":"2024-09-11T07:52:31.253611+0000","last_peered":"2024-09-11T07:52:31.253611+0000","last_clean":"2024-09-11T07:52:31.253611+0000","last_became_active":"2024-09-11T07:50:53.913522+0000","last_became_peered":"2024-09-11T07:50:53.913522+0000","last_unstale":"2024-09-11T07:52:31.253611+0000","last_undegraded":"2024-09-11T07:52:31.253611+0000","last_fullsized":"2024-09-11T07:52:31.253611+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:47.176746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:51:47.176746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:10.590927+0000","scrub_duration":0.001565575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"426'22","reported_seq":81,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383159+0000","last_change":"2024-09-11T07:51:38.914292+0000","last_active":"2024-09-11T07:52:31.383159+0000","last_peered":"2024-09-11T07:52:31.383159+0000","last_clean":"2024-09-11T07:52:31.383159+0000","last_became_active":"2024-09-11T07:51:14.059568+0000","last_became_peered":"2024-09-11T07:51:14.059568+0000","last_unstale":"2024-09-11T07:52:31.383159+0000","last_undegraded":"2024-09-11T07:52:31.383159+0000","last_fullsized":"2024-09-11T07:52:31.383159+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'22","last_scrub_stamp":"2024-09-11T07:51:38.914229+0000","last_deep_scrub":"236'16","last_deep_scrub_stamp":"2024-09-11T07:36:49.809484+0000","last_clean_scrub_stamp":"2024-09-11T07:51:38.914229+0000","objects_scrubbed":4,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:04.524583+0000","scrub_duration":0.0022210229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":439,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383352+0000","last_change":"2024-09-11T07:52:24.847969+0000","last_active":"2024-09-11T07:52:31.383352+0000","last_peered":"2024-09-11T07:52:31.383352+0000","last_clean":"2024-09-11T07:52:31.383352+0000","last_became_active":"2024-09-11T07:51:45.190308+0000","last_became_peered":"2024-09-11T07:51:45.190308+0000","last_unstale":"2024-09-11T07:52:31.383352+0000","last_undegraded":"2024-09-11T07:52:31.383352+0000","last_fullsized":"2024-09-11T07:52:31.383352+0000","mapping_epoch":514,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":515,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:24.847906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:38.721118+0000","last_clean_scrub_stamp":"2024-09-11T07:52:24.847906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:48.339953+0000","scrub_duration":0.001516547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":586,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.368447+0000","last_change":"2024-09-11T07:51:47.656510+0000","last_active":"2024-09-11T07:52:31.368447+0000","last_peered":"2024-09-11T07:52:31.368447+0000","last_clean":"2024-09-11T07:52:31.368447+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:52:31.368447+0000","last_undegraded":"2024-09-11T07:52:31.368447+0000","last_fullsized":"2024-09-11T07:52:31.368447+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:47.656446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:51:47.656446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:49.332839+0000","scrub_duration":0.0013035880000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":608,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:30.860083+0000","last_change":"2024-09-11T07:51:38.644591+0000","last_active":"2024-09-11T07:52:30.860083+0000","last_peered":"2024-09-11T07:52:30.860083+0000","last_clean":"2024-09-11T07:52:30.860083+0000","last_became_active":"2024-09-11T07:51:34.323298+0000","last_became_peered":"2024-09-11T07:51:34.323298+0000","last_unstale":"2024-09-11T07:52:30.860083+0000","last_undegraded":"2024-09-11T07:52:30.860083+0000","last_fullsized":"2024-09-11T07:52:30.860083+0000","mapping_epoch":509,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":510,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:38.644532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:51:38.644532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:58.564573+0000","scrub_duration":0.001609573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"426'32","reported_seq":75,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.253188+0000","last_change":"2024-09-11T07:51:48.135202+0000","last_active":"2024-09-11T07:52:31.253188+0000","last_peered":"2024-09-11T07:52:31.253188+0000","last_clean":"2024-09-11T07:52:31.253188+0000","last_became_active":"2024-09-11T07:50:50.862648+0000","last_became_peered":"2024-09-11T07:50:50.862648+0000","last_unstale":"2024-09-11T07:52:31.253188+0000","last_undegraded":"2024-09-11T07:52:31.253188+0000","last_fullsized":"2024-09-11T07:52:31.253188+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'32","last_scrub_stamp":"2024-09-11T07:51:48.135133+0000","last_deep_scrub":"234'24","last_deep_scrub_stamp":"2024-09-11T07:41:32.694510+0000","last_clean_scrub_stamp":"2024-09-11T07:51:48.135133+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:51.716765+0000","scrub_duration":0.00246175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":453,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383937+0000","last_change":"2024-09-11T07:51:29.942644+0000","last_active":"2024-09-11T07:52:31.383937+0000","last_peered":"2024-09-11T07:52:31.383937+0000","last_clean":"2024-09-11T07:52:31.383937+0000","last_became_active":"2024-09-11T07:51:14.064911+0000","last_became_peered":"2024-09-11T07:51:14.064911+0000","last_unstale":"2024-09-11T07:52:31.383937+0000","last_undegraded":"2024-09-11T07:52:31.383937+0000","last_fullsized":"2024-09-11T07:52:31.383937+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:29.942588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:51:29.942588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:45.079513+0000","scrub_duration":0.001484642,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":566,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383454+0000","last_change":"2024-09-11T07:52:01.907696+0000","last_active":"2024-09-11T07:52:31.383454+0000","last_peered":"2024-09-11T07:52:31.383454+0000","last_clean":"2024-09-11T07:52:31.383454+0000","last_became_active":"2024-09-11T07:51:19.095374+0000","last_became_peered":"2024-09-11T07:51:19.095374+0000","last_unstale":"2024-09-11T07:52:31.383454+0000","last_undegraded":"2024-09-11T07:52:31.383454+0000","last_fullsized":"2024-09-11T07:52:31.383454+0000","mapping_epoch":504,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":505,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:01.907636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:52:01.907636+0000","last_clean_scrub_stamp":"2024-09-11T07:52:01.907636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:04.072559+0000","scrub_duration":0.0032001429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":58,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:37.524414+0000","last_change":"2024-09-11T07:52:37.524381+0000","last_active":"2024-09-11T07:52:37.524414+0000","last_peered":"2024-09-11T07:52:37.524414+0000","last_clean":"2024-09-11T07:52:37.524414+0000","last_became_active":"2024-09-11T07:51:34.321822+0000","last_became_peered":"2024-09-11T07:51:34.321822+0000","last_unstale":"2024-09-11T07:52:37.524414+0000","last_undegraded":"2024-09-11T07:52:37.524414+0000","last_fullsized":"2024-09-11T07:52:37.524414+0000","mapping_epoch":509,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":510,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:37.524308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:51:15.671653+0000","last_clean_scrub_stamp":"2024-09-11T07:52:37.524308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:49.945387+0000","scrub_duration":0.0019592149999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"426'14","reported_seq":628,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.367945+0000","last_change":"2024-09-11T07:52:02.431975+0000","last_active":"2024-09-11T07:52:31.367945+0000","last_peered":"2024-09-11T07:52:31.367945+0000","last_clean":"2024-09-11T07:52:31.367945+0000","last_became_active":"2024-09-11T07:51:13.062932+0000","last_became_peered":"2024-09-11T07:51:13.062932+0000","last_unstale":"2024-09-11T07:52:31.367945+0000","last_undegraded":"2024-09-11T07:52:31.367945+0000","last_fullsized":"2024-09-11T07:52:31.367945+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'14","last_scrub_stamp":"2024-09-11T07:52:02.431914+0000","last_deep_scrub":"234'10","last_deep_scrub_stamp":"2024-09-11T07:41:29.585284+0000","last_clean_scrub_stamp":"2024-09-11T07:52:02.431914+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:06.057909+0000","scrub_duration":0.001901727,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":16,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":576,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.384652+0000","last_change":"2024-09-11T07:52:11.613845+0000","last_active":"2024-09-11T07:52:31.384652+0000","last_peered":"2024-09-11T07:52:31.384652+0000","last_clean":"2024-09-11T07:52:31.384652+0000","last_became_active":"2024-09-11T07:51:37.863531+0000","last_became_peered":"2024-09-11T07:51:37.863531+0000","last_unstale":"2024-09-11T07:52:31.384652+0000","last_undegraded":"2024-09-11T07:52:31.384652+0000","last_fullsized":"2024-09-11T07:52:31.384652+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:11.613764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_clean_scrub_stamp":"2024-09-11T07:52:11.613764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:30.554453+0000","scrub_duration":0.001556357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":390,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.384027+0000","last_change":"2024-09-11T07:51:48.204460+0000","last_active":"2024-09-11T07:52:31.384027+0000","last_peered":"2024-09-11T07:52:31.384027+0000","last_clean":"2024-09-11T07:52:31.384027+0000","last_became_active":"2024-09-11T07:51:48.203815+0000","last_became_peered":"2024-09-11T07:51:48.203815+0000","last_unstale":"2024-09-11T07:52:31.384027+0000","last_undegraded":"2024-09-11T07:52:31.384027+0000","last_fullsized":"2024-09-11T07:52:31.384027+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:45.775439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:31.477552+0000","last_clean_scrub_stamp":"2024-09-11T07:51:45.775439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:49.810374+0000","scrub_duration":0.001281325,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":68,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:30.862093+0000","last_change":"2024-09-11T07:52:30.608369+0000","last_active":"2024-09-11T07:52:30.862093+0000","last_peered":"2024-09-11T07:52:30.862093+0000","last_clean":"2024-09-11T07:52:30.862093+0000","last_became_active":"2024-09-11T07:50:56.911204+0000","last_became_peered":"2024-09-11T07:50:56.911204+0000","last_unstale":"2024-09-11T07:52:30.862093+0000","last_undegraded":"2024-09-11T07:52:30.862093+0000","last_fullsized":"2024-09-11T07:52:30.862093+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:30.608304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:52:30.608304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:59.901358+0000","scrub_duration":0.0017707720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"533'32","reported_seq":681,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:39.461816+0000","last_change":"2024-09-11T07:52:39.461787+0000","last_active":"2024-09-11T07:52:39.461816+0000","last_peered":"2024-09-11T07:52:39.461816+0000","last_clean":"2024-09-11T07:52:39.461816+0000","last_became_active":"2024-09-11T07:51:48.202896+0000","last_became_peered":"2024-09-11T07:51:48.202896+0000","last_unstale":"2024-09-11T07:52:39.461816+0000","last_undegraded":"2024-09-11T07:52:39.461816+0000","last_fullsized":"2024-09-11T07:52:39.461816+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"533'32","last_scrub_stamp":"2024-09-11T07:52:39.461710+0000","last_deep_scrub":"533'32","last_deep_scrub_stamp":"2024-09-11T07:52:39.461710+0000","last_clean_scrub_stamp":"2024-09-11T07:52:39.461710+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:45.067466+0000","scrub_duration":0.0045958320000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":334,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383912+0000","last_change":"2024-09-11T07:51:49.725554+0000","last_active":"2024-09-11T07:52:31.383912+0000","last_peered":"2024-09-11T07:52:31.383912+0000","last_clean":"2024-09-11T07:52:31.383912+0000","last_became_active":"2024-09-11T07:51:45.192691+0000","last_became_peered":"2024-09-11T07:51:45.192691+0000","last_unstale":"2024-09-11T07:52:31.383912+0000","last_undegraded":"2024-09-11T07:52:31.383912+0000","last_fullsized":"2024-09-11T07:52:31.383912+0000","mapping_epoch":514,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":515,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:49.725487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:48.366931+0000","last_clean_scrub_stamp":"2024-09-11T07:51:49.725487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:08.814392+0000","scrub_duration":0.001478095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":58,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:30.861818+0000","last_change":"2024-09-11T07:51:11.530813+0000","last_active":"2024-09-11T07:52:30.861818+0000","last_peered":"2024-09-11T07:52:30.861818+0000","last_clean":"2024-09-11T07:52:30.861818+0000","last_became_active":"2024-09-11T07:50:56.911495+0000","last_became_peered":"2024-09-11T07:50:56.911495+0000","last_unstale":"2024-09-11T07:52:30.861818+0000","last_undegraded":"2024-09-11T07:52:30.861818+0000","last_fullsized":"2024-09-11T07:52:30.861818+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:11.530745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:39.021876+0000","last_clean_scrub_stamp":"2024-09-11T07:51:11.530745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014437320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":581,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.368230+0000","last_change":"2024-09-11T07:52:21.416292+0000","last_active":"2024-09-11T07:52:31.368230+0000","last_peered":"2024-09-11T07:52:31.368230+0000","last_clean":"2024-09-11T07:52:31.368230+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:52:31.368230+0000","last_undegraded":"2024-09-11T07:52:31.368230+0000","last_fullsized":"2024-09-11T07:52:31.368230+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:21.416240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:32.904006+0000","last_clean_scrub_stamp":"2024-09-11T07:52:21.416240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:28.815980+0000","scrub_duration":0.0021936759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"450'2","reported_seq":62,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:30.862384+0000","last_change":"2024-09-11T07:51:30.503164+0000","last_active":"2024-09-11T07:52:30.862384+0000","last_peered":"2024-09-11T07:52:30.862384+0000","last_clean":"2024-09-11T07:52:30.862384+0000","last_became_active":"2024-09-11T07:50:56.909284+0000","last_became_peered":"2024-09-11T07:50:56.909284+0000","last_unstale":"2024-09-11T07:52:30.862384+0000","last_undegraded":"2024-09-11T07:52:30.862384+0000","last_fullsized":"2024-09-11T07:52:30.862384+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"450'2","last_scrub_stamp":"2024-09-11T07:51:30.503098+0000","last_deep_scrub":"450'2","last_deep_scrub_stamp":"2024-09-11T07:48:29.800545+0000","last_clean_scrub_stamp":"2024-09-11T07:51:30.503098+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002092244,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":551,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383540+0000","last_change":"2024-09-11T07:51:48.212311+0000","last_active":"2024-09-11T07:52:31.383540+0000","last_peered":"2024-09-11T07:52:31.383540+0000","last_clean":"2024-09-11T07:52:31.383540+0000","last_became_active":"2024-09-11T07:51:48.202367+0000","last_became_peered":"2024-09-11T07:51:48.202367+0000","last_unstale":"2024-09-11T07:52:31.383540+0000","last_undegraded":"2024-09-11T07:52:31.383540+0000","last_fullsized":"2024-09-11T07:52:31.383540+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:51:13.175220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:51:13.175220+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:39.570954+0000","scrub_duration":0.0068790429999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":2622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"426'30","reported_seq":80,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.253501+0000","last_change":"2024-09-11T07:51:26.260317+0000","last_active":"2024-09-11T07:52:31.253501+0000","last_peered":"2024-09-11T07:52:31.253501+0000","last_clean":"2024-09-11T07:52:31.253501+0000","last_became_active":"2024-09-11T07:50:50.868078+0000","last_became_peered":"2024-09-11T07:50:50.868078+0000","last_unstale":"2024-09-11T07:52:31.253501+0000","last_undegraded":"2024-09-11T07:52:31.253501+0000","last_fullsized":"2024-09-11T07:52:31.253501+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'30","last_scrub_stamp":"2024-09-11T07:51:26.260244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:51:26.260244+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:48.632277+0000","scrub_duration":0.0019868400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1f","version":"517'18","reported_seq":413,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:34.824352+0000","last_change":"2024-09-11T07:51:48.216012+0000","last_active":"2024-09-11T07:52:34.824352+0000","last_peered":"2024-09-11T07:52:34.824352+0000","last_clean":"2024-09-11T07:52:34.824352+0000","last_became_active":"2024-09-11T07:51:48.206146+0000","last_became_peered":"2024-09-11T07:51:48.206146+0000","last_unstale":"2024-09-11T07:52:34.824352+0000","last_undegraded":"2024-09-11T07:52:34.824352+0000","last_fullsized":"2024-09-11T07:52:34.824352+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"493'16","last_scrub_stamp":"2024-09-11T07:51:34.147125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:51:34.147125+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:56.229859+0000","scrub_duration":0.0037941960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":500,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.383557+0000","last_change":"2024-09-11T07:52:11.677900+0000","last_active":"2024-09-11T07:52:31.383557+0000","last_peered":"2024-09-11T07:52:31.383557+0000","last_clean":"2024-09-11T07:52:31.383557+0000","last_became_active":"2024-09-11T07:51:14.065311+0000","last_became_peered":"2024-09-11T07:51:14.065311+0000","last_unstale":"2024-09-11T07:52:31.383557+0000","last_undegraded":"2024-09-11T07:52:31.383557+0000","last_fullsized":"2024-09-11T07:52:31.383557+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:11.677846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:52:11.677846+0000","last_clean_scrub_stamp":"2024-09-11T07:52:11.677846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:30.305406+0000","scrub_duration":0.014118188,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":597,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.368898+0000","last_change":"2024-09-11T07:51:52.526031+0000","last_active":"2024-09-11T07:52:31.368898+0000","last_peered":"2024-09-11T07:52:31.368898+0000","last_clean":"2024-09-11T07:52:31.368898+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:52:31.368898+0000","last_undegraded":"2024-09-11T07:52:31.368898+0000","last_fullsized":"2024-09-11T07:52:31.368898+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:52.525949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:51:52.525949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:01.957606+0000","scrub_duration":0.001859044,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"426'27","reported_seq":166,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.369201+0000","last_change":"2024-09-11T07:51:13.238482+0000","last_active":"2024-09-11T07:52:31.369201+0000","last_peered":"2024-09-11T07:52:31.369201+0000","last_clean":"2024-09-11T07:52:31.369201+0000","last_became_active":"2024-09-11T07:46:13.790166+0000","last_became_peered":"2024-09-11T07:46:13.790166+0000","last_unstale":"2024-09-11T07:52:31.369201+0000","last_undegraded":"2024-09-11T07:52:31.369201+0000","last_fullsized":"2024-09-11T07:52:31.369201+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'27","last_scrub_stamp":"2024-09-11T07:51:13.238422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:51:13.238422+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021108820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":647,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:30.861396+0000","last_change":"2024-09-11T07:51:48.651760+0000","last_active":"2024-09-11T07:52:30.861396+0000","last_peered":"2024-09-11T07:52:30.861396+0000","last_clean":"2024-09-11T07:52:30.861396+0000","last_became_active":"2024-09-11T07:50:56.906837+0000","last_became_peered":"2024-09-11T07:50:56.906837+0000","last_unstale":"2024-09-11T07:52:30.861396+0000","last_undegraded":"2024-09-11T07:52:30.861396+0000","last_fullsized":"2024-09-11T07:52:30.861396+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:48.651694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:50:22.592981+0000","last_clean_scrub_stamp":"2024-09-11T07:51:48.651694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:02.597517+0000","scrub_duration":0.001630065,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":63,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.253603+0000","last_change":"2024-09-11T07:51:35.175475+0000","last_active":"2024-09-11T07:52:31.253603+0000","last_peered":"2024-09-11T07:52:31.253603+0000","last_clean":"2024-09-11T07:52:31.253603+0000","last_became_active":"2024-09-11T07:50:50.860452+0000","last_became_peered":"2024-09-11T07:50:50.860452+0000","last_unstale":"2024-09-11T07:52:31.253603+0000","last_undegraded":"2024-09-11T07:52:31.253603+0000","last_fullsized":"2024-09-11T07:52:31.253603+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:35.175412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:51:35.175412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:46.700844+0000","scrub_duration":0.001368794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":285,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.971850+0000","last_change":"2024-09-11T07:52:36.971807+0000","last_active":"2024-09-11T07:52:36.971850+0000","last_peered":"2024-09-11T07:52:36.971850+0000","last_clean":"2024-09-11T07:52:36.971850+0000","last_became_active":"2024-09-11T07:51:45.190906+0000","last_became_peered":"2024-09-11T07:51:45.190906+0000","last_unstale":"2024-09-11T07:52:36.971850+0000","last_undegraded":"2024-09-11T07:52:36.971850+0000","last_fullsized":"2024-09-11T07:52:36.971850+0000","mapping_epoch":514,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":515,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:36.971740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:52:36.971740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:51.940992+0000","scrub_duration":0.0015586409999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"426'39","reported_seq":77,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642264+0000","last_change":"2024-09-11T07:51:26.109917+0000","last_active":"2024-09-11T07:52:36.642264+0000","last_peered":"2024-09-11T07:52:36.642264+0000","last_clean":"2024-09-11T07:52:36.642264+0000","last_became_active":"2024-09-11T07:50:53.903030+0000","last_became_peered":"2024-09-11T07:50:53.903030+0000","last_unstale":"2024-09-11T07:52:36.642264+0000","last_undegraded":"2024-09-11T07:52:36.642264+0000","last_fullsized":"2024-09-11T07:52:36.642264+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'39","last_scrub_stamp":"2024-09-11T07:51:26.109850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:51:26.109850+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:42.668917+0000","scrub_duration":0.0023462560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":449,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.790088+0000","last_change":"2024-09-11T07:52:35.790021+0000","last_active":"2024-09-11T07:52:35.790088+0000","last_peered":"2024-09-11T07:52:35.790088+0000","last_clean":"2024-09-11T07:52:35.790088+0000","last_became_active":"2024-09-11T07:51:14.061993+0000","last_became_peered":"2024-09-11T07:51:14.061993+0000","last_unstale":"2024-09-11T07:52:35.790088+0000","last_undegraded":"2024-09-11T07:52:35.790088+0000","last_fullsized":"2024-09-11T07:52:35.790088+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:35.789963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:32.456852+0000","last_clean_scrub_stamp":"2024-09-11T07:52:35.789963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:53.408426+0000","scrub_duration":0.0015193349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":628,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.369318+0000","last_change":"2024-09-11T07:52:12.445048+0000","last_active":"2024-09-11T07:52:31.369318+0000","last_peered":"2024-09-11T07:52:31.369318+0000","last_clean":"2024-09-11T07:52:31.369318+0000","last_became_active":"2024-09-11T07:51:34.321779+0000","last_became_peered":"2024-09-11T07:51:34.321779+0000","last_unstale":"2024-09-11T07:52:31.369318+0000","last_undegraded":"2024-09-11T07:52:31.369318+0000","last_fullsized":"2024-09-11T07:52:31.369318+0000","mapping_epoch":509,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":510,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:12.444990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:52:12.444990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:23.945312+0000","scrub_duration":0.001808046,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":641,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.368540+0000","last_change":"2024-09-11T07:51:48.209238+0000","last_active":"2024-09-11T07:52:31.368540+0000","last_peered":"2024-09-11T07:52:31.368540+0000","last_clean":"2024-09-11T07:52:31.368540+0000","last_became_active":"2024-09-11T07:51:48.208985+0000","last_became_peered":"2024-09-11T07:51:48.208985+0000","last_unstale":"2024-09-11T07:52:31.368540+0000","last_undegraded":"2024-09-11T07:52:31.368540+0000","last_fullsized":"2024-09-11T07:52:31.368540+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:33.422146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:51:33.422146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:03.422139+0000","scrub_duration":0.0017024309999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"426'29","reported_seq":672,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.368996+0000","last_change":"2024-09-11T07:51:24.392905+0000","last_active":"2024-09-11T07:52:31.368996+0000","last_peered":"2024-09-11T07:52:31.368996+0000","last_clean":"2024-09-11T07:52:31.368996+0000","last_became_active":"2024-09-11T07:50:50.857201+0000","last_became_peered":"2024-09-11T07:50:50.857201+0000","last_unstale":"2024-09-11T07:52:31.368996+0000","last_undegraded":"2024-09-11T07:52:31.368996+0000","last_fullsized":"2024-09-11T07:52:31.368996+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'29","last_scrub_stamp":"2024-09-11T07:51:24.392838+0000","last_deep_scrub":"234'23","last_deep_scrub_stamp":"2024-09-11T07:43:25.530826+0000","last_clean_scrub_stamp":"2024-09-11T07:51:24.392838+0000","objects_scrubbed":5,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029751859999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":83,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-09-11T07:52:31.253096+0000","last_change":"2024-09-11T07:52:28.242947+0000","last_active":"2024-09-11T07:52:31.253096+0000","last_peered":"2024-09-11T07:52:31.253096+0000","last_clean":"2024-09-11T07:52:31.253096+0000","last_became_active":"2024-09-11T07:50:53.915112+0000","last_became_peered":"2024-09-11T07:50:53.915112+0000","last_unstale":"2024-09-11T07:52:31.253096+0000","last_undegraded":"2024-09-11T07:52:31.253096+0000","last_fullsized":"2024-09-11T07:52:31.253096+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:28.242881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:52:28.242881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:55.145036+0000","scrub_duration":0.0014834550000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":777,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1522,"num_read_kb":1279,"num_write":989,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":493,"num_bytes_recovered":9506,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1103,"ondisk_log_size":1103,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1143,"num_read_kb":1138,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":983040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":891,"ondisk_log_size":891,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":147,"seq":631360192762,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":924796,"kb_used_data":1268,"kb_used_omap":4,"kb_used_meta":923515,"kb_avail":92820356,"statfs":{"total":95995035648,"available":95048044544,"internally_reserved":0,"allocated":1298432,"data_stored":823349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":945679878},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:51:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":1.5980000000000001,"15min":1.593},"min":{"1min":0.29199999999999998,"5min":0.224,"15min":0.191},"max":{"1min":0.70099999999999996,"5min":43.406999999999996,"15min":75.677000000000007},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":1.6619999999999999,"15min":1.6890000000000001},"min":{"1min":0.34100000000000003,"5min":0.22,"15min":0.214},"max":{"1min":0.79700000000000004,"5min":43.845999999999997,"15min":75.638999999999996},"last":0.625}]},{"osd":1,"last update":"Wed Sep 11 07:51:51 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":3.5270000000000001,"15min":3.2250000000000001},"min":{"1min":0.31,"5min":0.247,"15min":0.22500000000000001},"max":{"1min":0.89800000000000002,"5min":161.63900000000001,"15min":327.06099999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":3.5870000000000002,"15min":3.1949999999999998},"min":{"1min":0.32500000000000001,"5min":0.23400000000000001,"15min":0.22800000000000001},"max":{"1min":0.69999999999999996,"5min":161.65199999999999,"15min":327.07499999999999},"last":0.57799999999999996}]},{"osd":2,"last update":"Wed Sep 11 07:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":14.401,"5min":14.401,"15min":14.401},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.60399999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:51:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":1.9490000000000001,"15min":1.29},"min":{"1min":0.34799999999999998,"5min":0.27000000000000002,"15min":0.252},"max":{"1min":0.75900000000000001,"5min":84.013999999999996,"15min":84.013999999999996},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":1.847,"15min":1.3089999999999999},"min":{"1min":0.45800000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.876,"5min":84.167000000000002,"15min":84.167000000000002},"last":0.58999999999999997}]},{"osd":4,"last update":"Wed Sep 11 07:51:51 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48199999999999998,"15min":0.68000000000000005},"min":{"1min":0.32200000000000001,"5min":0.27800000000000002,"15min":0.23999999999999999},"max":{"1min":1.01,"5min":2.5310000000000001,"15min":6.0220000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.48599999999999999,"15min":0.69499999999999995},"min":{"1min":0.36099999999999999,"5min":0.27700000000000002,"15min":0.22800000000000001},"max":{"1min":0.71999999999999997,"5min":1.359,"15min":6.9000000000000004},"last":0.55600000000000005}]},{"osd":5,"last update":"Wed Sep 11 07:51:51 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.496,"15min":0.745},"min":{"1min":0.378,"5min":0.25700000000000001,"15min":0.25},"max":{"1min":0.97299999999999998,"5min":2.839,"15min":9.9339999999999993},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.55000000000000004,"15min":0.69999999999999996},"min":{"1min":0.33500000000000002,"5min":0.27700000000000002,"15min":0.26700000000000002},"max":{"1min":0.72299999999999998,"5min":4.4470000000000001,"15min":10.183999999999999},"last":0.47899999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:51:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56599999999999995,"15min":0.64000000000000001},"min":{"1min":0.318,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.84899999999999998,"5min":3.0449999999999999,"15min":5.1909999999999998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.57899999999999996,"15min":0.71199999999999997},"min":{"1min":0.36399999999999999,"5min":0.32600000000000001,"15min":0.27300000000000002},"max":{"1min":0.91800000000000004,"5min":2.8069999999999999,"15min":4.2030000000000003},"last":0.54100000000000004}]}]},{"osd":6,"up_from":44,"seq":188978561347,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":898216,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":897278,"kb_avail":92846936,"statfs":{"total":95995035648,"available":95075262464,"internally_reserved":0,"allocated":946176,"data_stored":494147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":918813016},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":1.77,"15min":2.0099999999999998},"min":{"1min":0.314,"5min":0.221,"15min":0.219},"max":{"1min":0.67400000000000004,"5min":75.728999999999999,"15min":130.62299999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":1.593,"15min":2.0219999999999998},"min":{"1min":0.38100000000000001,"5min":0.216,"15min":0.20499999999999999},"max":{"1min":0.71999999999999997,"5min":75.536000000000001,"15min":130.86199999999999},"last":0.38800000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.99199999999999999,"15min":1.9910000000000001},"min":{"1min":0.308,"5min":0.27100000000000002,"15min":0.23599999999999999},"max":{"1min":0.72999999999999998,"5min":17.029,"15min":131.42500000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":1.1910000000000001,"15min":1.9990000000000001},"min":{"1min":0.33100000000000002,"5min":0.22800000000000001,"15min":0.22},"max":{"1min":0.75600000000000001,"5min":16.992999999999999,"15min":132.39599999999999},"last":0.45700000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:51:42 2024","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.54,"15min":1.54},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":18.166,"5min":18.166,"15min":18.166},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.495}]},{"osd":3,"last update":"Wed Sep 11 07:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":1.4450000000000001,"15min":1.609},"min":{"1min":0.377,"5min":0.28899999999999998,"15min":0.246},"max":{"1min":0.78600000000000003,"5min":36.414000000000001,"15min":49.109000000000002},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.50900000000000001,"5min":1.498,"15min":1.6859999999999999},"min":{"1min":0.32000000000000001,"5min":0.28299999999999997,"15min":0.246},"max":{"1min":0.70199999999999996,"5min":39.390000000000001,"15min":48.963999999999999},"last":0.47799999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.46700000000000003,"15min":1.7190000000000001},"min":{"1min":0.29099999999999998,"5min":0.27600000000000002,"15min":0.223},"max":{"1min":0.64200000000000002,"5min":2.8029999999999999,"15min":181.78800000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.504,"15min":1.175},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.21099999999999999},"max":{"1min":0.65000000000000002,"5min":4.3220000000000001,"15min":128.59299999999999},"last":0.57799999999999996}]},{"osd":5,"last update":"Wed Sep 11 07:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.45800000000000002,"15min":0.90800000000000003},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.23499999999999999},"max":{"1min":0.55900000000000005,"5min":2.9220000000000002,"15min":57.436},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.49399999999999999,"15min":0.94599999999999995},"min":{"1min":0.34200000000000003,"5min":0.30299999999999999,"15min":0.253},"max":{"1min":0.74099999999999999,"5min":1.49,"15min":56.822000000000003},"last":0.65100000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.56200000000000006,"15min":1.048},"min":{"1min":0.36399999999999999,"5min":0.34899999999999998,"15min":0.27800000000000002},"max":{"1min":0.76900000000000002,"5min":2.8340000000000001,"15min":60.279000000000003},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.54900000000000004,"15min":0.98099999999999998},"min":{"1min":0.32300000000000001,"5min":0.29899999999999999,"15min":0.215},"max":{"1min":0.69099999999999995,"5min":2.9550000000000001,"15min":60.767000000000003},"last":0.64100000000000001}]}]},{"osd":5,"up_from":37,"seq":158913790281,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":897708,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":896190,"kb_avail":92847444,"statfs":{"total":95995035648,"available":95075782656,"internally_reserved":0,"allocated":1540096,"data_stored":1084956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":917698904},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:52:21 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.242,"15min":1.403},"min":{"1min":0.36799999999999999,"5min":0.20799999999999999,"15min":0.20200000000000001},"max":{"1min":17.818000000000001,"5min":24.353000000000002,"15min":40.313000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":1.089,"15min":1.363},"min":{"1min":0.318,"5min":0.223,"15min":0.20200000000000001},"max":{"1min":0.96699999999999997,"5min":25.890000000000001,"15min":40.235999999999997},"last":0.46500000000000002}]},{"osd":1,"last update":"Wed Sep 11 07:52:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.86699999999999999,"15min":1.4590000000000001},"min":{"1min":0.46300000000000002,"5min":0.21199999999999999,"15min":0.20799999999999999},"max":{"1min":0.76000000000000001,"5min":6.5960000000000001,"15min":122.738},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.85999999999999999,"15min":1.4299999999999999},"min":{"1min":0.35899999999999999,"5min":0.254,"15min":0.24099999999999999},"max":{"1min":0.77600000000000002,"5min":6.7069999999999999,"15min":122.66200000000001},"last":0.56499999999999995}]},{"osd":2,"last update":"Wed Sep 11 07:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.496},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.59599999999999997}]},{"osd":3,"last update":"Wed Sep 11 07:52:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":1.865,"15min":2.5249999999999999},"min":{"1min":0.34399999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.72999999999999998,"5min":82.995000000000005,"15min":132.715},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":1.7609999999999999,"15min":2.5470000000000002},"min":{"1min":0.48199999999999998,"5min":0.28399999999999997,"15min":0.28100000000000003},"max":{"1min":0.752,"5min":82.959000000000003,"15min":133.01599999999999},"last":0.53400000000000003}]},{"osd":4,"last update":"Wed Sep 11 07:52:21 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.46899999999999997,"15min":0.73099999999999998},"min":{"1min":0.33100000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.66700000000000004,"5min":2.3969999999999998,"15min":11.103},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.46100000000000002,"15min":0.72699999999999998},"min":{"1min":0.38200000000000001,"5min":0.25900000000000001,"15min":0.22500000000000001},"max":{"1min":0.77400000000000002,"5min":2.516,"15min":10.305},"last":0.60899999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:52:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.88800000000000001,"15min":0.79000000000000004},"min":{"1min":0.39100000000000001,"5min":0.26600000000000001,"15min":0.248},"max":{"1min":0.68899999999999995,"5min":28.893000000000001,"15min":28.893000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.90900000000000003,"15min":0.82399999999999995},"min":{"1min":0.40100000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.70799999999999996,"5min":29.033999999999999,"15min":29.033999999999999},"last":0.621}]},{"osd":7,"last update":"Wed Sep 11 07:52:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.505,"15min":1.4690000000000001},"min":{"1min":0.42999999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68799999999999994,"5min":1.6699999999999999,"15min":115.44499999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.51200000000000001,"15min":1.421},"min":{"1min":0.42599999999999999,"5min":0.314,"15min":0.26800000000000002},"max":{"1min":0.751,"5min":2.5470000000000002,"15min":115.071},"last":0.65200000000000002}]}]},{"osd":4,"up_from":31,"seq":133143986509,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":902336,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":901374,"kb_avail":92842816,"statfs":{"total":95995035648,"available":95071043584,"internally_reserved":0,"allocated":970752,"data_stored":495158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":923007320},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:52:02 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":1.069,"15min":1.367},"min":{"1min":0.27700000000000002,"5min":0.219,"15min":0.20499999999999999},"max":{"1min":0.73299999999999998,"5min":18.327999999999999,"15min":87.623000000000005},"last":13.653},{"interface":"front","average":{"1min":0.53700000000000003,"5min":1.0980000000000001,"15min":1.45},"min":{"1min":0.34799999999999998,"5min":0.253,"15min":0.222},"max":{"1min":0.77000000000000002,"5min":18.431999999999999,"15min":87.682000000000002},"last":13.723000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:52:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.751,"15min":1.72},"min":{"1min":0.31900000000000001,"5min":0.23200000000000001,"15min":0.22900000000000001},"max":{"1min":0.67400000000000004,"5min":4.9320000000000004,"15min":141.78399999999999},"last":0.438},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.77200000000000002,"15min":1.8009999999999999},"min":{"1min":0.36699999999999999,"5min":0.26300000000000001,"15min":0.224},"max":{"1min":0.85299999999999998,"5min":7.383,"15min":145.65899999999999},"last":13.689}]},{"osd":2,"last update":"Wed Sep 11 07:51:43 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":13.597}]},{"osd":3,"last update":"Wed Sep 11 07:52:02 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":2.6440000000000001,"15min":1.589},"min":{"1min":0.34599999999999997,"5min":0.26700000000000002,"15min":0.25700000000000001},"max":{"1min":0.92300000000000004,"5min":66.855999999999995,"15min":66.855999999999995},"last":13.702999999999999},{"interface":"front","average":{"1min":1.0389999999999999,"5min":2.8460000000000001,"15min":1.6379999999999999},"min":{"1min":0.41299999999999998,"5min":0.28799999999999998,"15min":0.26100000000000001},"max":{"1min":9.4610000000000003,"5min":66.140000000000001,"15min":66.140000000000001},"last":0.40000000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:52:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.43099999999999999,"15min":0.64900000000000002},"min":{"1min":0.375,"5min":0.24399999999999999,"15min":0.224},"max":{"1min":0.65300000000000002,"5min":0.88100000000000001,"15min":5.6470000000000002},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.41199999999999998,"15min":0.67600000000000005},"min":{"1min":0.34699999999999998,"5min":0.25,"15min":0.23000000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":5.7450000000000001},"last":0.41899999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:51:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.83099999999999996,"15min":1.0329999999999999},"min":{"1min":0.308,"5min":0.26400000000000001,"15min":0.23599999999999999},"max":{"1min":0.66500000000000004,"5min":38.334000000000003,"15min":38.334000000000003},"last":0.499},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.81799999999999995,"15min":0.99299999999999999},"min":{"1min":0.36399999999999999,"5min":0.28299999999999997,"15min":0.27500000000000002},"max":{"1min":0.61399999999999999,"5min":38.466999999999999,"15min":38.466999999999999},"last":0.48299999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.49199999999999999,"15min":1.147},"min":{"1min":0.372,"5min":0.313,"15min":0.29399999999999998},"max":{"1min":0.69899999999999995,"5min":1.089,"15min":102.258},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.48799999999999999,"15min":1.0529999999999999},"min":{"1min":0.38400000000000001,"5min":0.315,"15min":0.29799999999999999},"max":{"1min":0.69799999999999995,"5min":1.0149999999999999,"15min":101.886},"last":0.46800000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182738,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":900424,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":898878,"kb_avail":92844728,"statfs":{"total":95995035648,"available":95073001472,"internally_reserved":0,"allocated":1572864,"data_stored":1084990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":920451416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":1.1659999999999999,"15min":1.3520000000000001},"min":{"1min":0.22800000000000001,"5min":0.184,"15min":0.16600000000000001},"max":{"1min":0.60199999999999998,"5min":9.5399999999999991,"15min":27.655999999999999},"last":0.498},{"interface":"front","average":{"1min":0.39200000000000002,"5min":1.276,"15min":1.468},"min":{"1min":0.27800000000000002,"5min":0.21299999999999999,"15min":0.183},"max":{"1min":0.57899999999999996,"5min":8.0969999999999995,"15min":26.759},"last":0.41199999999999998}]},{"osd":1,"last update":"Wed Sep 11 07:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":1.1220000000000001,"15min":2.5089999999999999},"min":{"1min":0.28899999999999998,"5min":0.23599999999999999,"15min":0.19500000000000001},"max":{"1min":0.76900000000000002,"5min":16.971,"15min":135.41999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":1.1519999999999999,"15min":2.2679999999999998},"min":{"1min":0.224,"5min":0.221,"15min":0.221},"max":{"1min":0.47899999999999998,"5min":18.277000000000001,"15min":136.126},"last":0.46100000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:51:44 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.55400000000000005}]},{"osd":4,"last update":"Wed Sep 11 07:52:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.82499999999999996,"15min":1.6539999999999999},"min":{"1min":0.36699999999999999,"5min":0.32000000000000001,"15min":0.27500000000000002},"max":{"1min":0.70799999999999996,"5min":7.5659999999999998,"15min":132.76400000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":1.0580000000000001,"15min":1.774},"min":{"1min":0.40999999999999998,"5min":0.34599999999999997,"15min":0.255},"max":{"1min":0.89400000000000002,"5min":18.957000000000001,"15min":132.154},"last":0.58799999999999997}]},{"osd":5,"last update":"Wed Sep 11 07:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.87,"15min":1.363},"min":{"1min":0.44500000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.79000000000000004,"5min":7.5270000000000001,"15min":137.50299999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":1.252,"15min":1.5449999999999999},"min":{"1min":0.439,"5min":0.31,"15min":0.29199999999999998},"max":{"1min":0.93500000000000005,"5min":19.266999999999999,"15min":137.45599999999999},"last":0.64800000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.873,"15min":1.861},"min":{"1min":0.40600000000000003,"5min":0.316,"15min":0.313},"max":{"1min":0.748,"5min":6.3609999999999998,"15min":233.98500000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.97199999999999998,"15min":1.897},"min":{"1min":0.41399999999999998,"5min":0.32500000000000001,"15min":0.30399999999999999},"max":{"1min":0.77400000000000002,"5min":7.4500000000000002,"15min":234.05199999999999},"last":0.71899999999999997}]},{"osd":7,"last update":"Wed Sep 11 07:52:30 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.80700000000000005,"15min":2.6070000000000002},"min":{"1min":0.51000000000000001,"5min":0.376,"15min":0.35699999999999998},"max":{"1min":0.97099999999999997,"5min":4.952,"15min":162.42699999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.873,"15min":2.5390000000000001},"min":{"1min":0.42699999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77300000000000002,"5min":6.3860000000000001,"15min":162.39099999999999},"last":0.56899999999999995}]}]},{"osd":2,"up_from":485,"seq":2083059138587,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":358332,"kb_used_data":1200,"kb_used_omap":50,"kb_used_meta":357069,"kb_avail":93386820,"statfs":{"total":95995035648,"available":95628103680,"internally_reserved":0,"allocated":1228800,"data_stored":763766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51973,"internal_metadata":365638907},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":0.307},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":0.32800000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.253},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.36699999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":0.47399999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.79300000000000004}]},{"osd":5,"last update":"Wed Sep 11 07:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.81299999999999994}]},{"osd":6,"last update":"Wed Sep 11 07:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.72599999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.77600000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575194,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":913620,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":912638,"kb_avail":92831532,"statfs":{"total":95995035648,"available":95059488768,"internally_reserved":0,"allocated":995328,"data_stored":495943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":934541656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:51:51 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":2.7650000000000001,"15min":1.893},"min":{"1min":0.221,"5min":0.20499999999999999,"15min":0.17299999999999999},"max":{"1min":0.48599999999999999,"5min":88.762,"15min":88.762},"last":0.376},{"interface":"front","average":{"1min":0.32200000000000001,"5min":2.7029999999999998,"15min":1.9119999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.186},"max":{"1min":0.41799999999999998,"5min":84.926000000000002,"15min":84.926000000000002},"last":0.438}]},{"osd":2,"last update":"Wed Sep 11 07:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.45200000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:51:39 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":5.7939999999999996,"15min":3.4529999999999998},"min":{"1min":0.27300000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.60899999999999999,"5min":276.24900000000002,"15min":276.24900000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.378,"5min":5.5149999999999997,"15min":3.2909999999999999},"min":{"1min":0.27000000000000002,"5min":0.22900000000000001,"15min":0.19800000000000001},"max":{"1min":0.55000000000000004,"5min":274.98000000000002,"15min":274.98000000000002},"last":0.46600000000000003}]},{"osd":4,"last update":"Wed Sep 11 07:51:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.81999999999999995,"15min":1.2490000000000001},"min":{"1min":0.42899999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.70399999999999996,"5min":8.1219999999999999,"15min":13.82},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.98099999999999998,"15min":1.2410000000000001},"min":{"1min":0.36899999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.75,"5min":4.7149999999999999,"15min":13.801},"last":0.52100000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.82199999999999995,"15min":1.1040000000000001},"min":{"1min":0.42999999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.873,"5min":8.0939999999999994,"15min":8.3680000000000003},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.86199999999999999,"15min":1.1619999999999999},"min":{"1min":0.35999999999999999,"5min":0.31900000000000001,"15min":0.27200000000000002},"max":{"1min":0.81299999999999994,"5min":4.7569999999999997,"15min":8.6809999999999992},"last":0.71599999999999997}]},{"osd":6,"last update":"Wed Sep 11 07:51:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":1.04,"15min":1.1870000000000001},"min":{"1min":0.38900000000000001,"5min":0.318,"15min":0.30599999999999999},"max":{"1min":0.752,"5min":4.79,"15min":13.981999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.70499999999999996,"15min":1.036},"min":{"1min":0.45000000000000001,"5min":0.372,"15min":0.318},"max":{"1min":0.754,"5min":2.9039999999999999,"15min":13.928000000000001},"last":0.63500000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:51:39 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.71499999999999997,"15min":1.0860000000000001},"min":{"1min":0.39900000000000002,"5min":0.35599999999999998,"15min":0.32200000000000001},"max":{"1min":0.80200000000000005,"5min":2.875,"15min":9.3629999999999995},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.92700000000000005,"15min":1.3620000000000001},"min":{"1min":0.48299999999999998,"5min":0.34000000000000002,"15min":0.33400000000000002},"max":{"1min":0.76800000000000002,"5min":8.0549999999999997,"15min":13.715},"last":0.70199999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738720,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":894332,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":893374,"kb_avail":92850820,"statfs":{"total":95995035648,"available":95079239680,"internally_reserved":0,"allocated":966656,"data_stored":494589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":914815320},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:51:47 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":1.4419999999999999,"15min":1.5069999999999999},"min":{"1min":0.20599999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.47799999999999998,"5min":6.79,"15min":72.418000000000006},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":1.577,"15min":1.5720000000000001},"min":{"1min":0.26300000000000001,"5min":0.187,"15min":0.17599999999999999},"max":{"1min":0.47499999999999998,"5min":10.061,"15min":71.703999999999994},"last":0.34799999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.40799999999999997}]},{"osd":3,"last update":"Wed Sep 11 07:51:47 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":1.944,"15min":1.671},"min":{"1min":0.30199999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.70799999999999996,"5min":62.667000000000002,"15min":62.667000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":2.2869999999999999,"15min":1.8169999999999999},"min":{"1min":0.27500000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.69199999999999995,"5min":61.691000000000003,"15min":61.691000000000003},"last":0.376}]},{"osd":4,"last update":"Wed Sep 11 07:52:08 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.35099999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.75800000000000001,"5min":9.6310000000000002,"15min":30.315000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.95699999999999996,"15min":1.2410000000000001},"min":{"1min":0.39400000000000002,"5min":0.30099999999999999,"15min":0.28599999999999998},"max":{"1min":0.79000000000000004,"5min":4.5960000000000001,"15min":30.363},"last":0.65400000000000003}]},{"osd":5,"last update":"Wed Sep 11 07:52:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":1.2869999999999999,"15min":1.1830000000000001},"min":{"1min":0.38300000000000001,"5min":0.29999999999999999,"15min":0.26800000000000002},"max":{"1min":0.89400000000000002,"5min":9.5839999999999996,"15min":9.5839999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.91800000000000004,"15min":1.1379999999999999},"min":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81499999999999995,"5min":4.6529999999999996,"15min":7.202},"last":0.80800000000000005}]},{"osd":6,"last update":"Wed Sep 11 07:51:47 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":1.008,"15min":1.1659999999999999},"min":{"1min":0.38400000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.1120000000000001,"5min":5.9249999999999998,"15min":6.5190000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":1.252,"15min":1.1990000000000001},"min":{"1min":0.436,"5min":0.36699999999999999,"15min":0.30199999999999999},"max":{"1min":0.89500000000000002,"5min":5.891,"15min":15.404},"last":0.75}]},{"osd":7,"last update":"Wed Sep 11 07:51:47 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":1.0009999999999999,"15min":1.429},"min":{"1min":0.42499999999999999,"5min":0.35699999999999998,"15min":0.34399999999999997},"max":{"1min":1.032,"5min":4.6260000000000003,"15min":34.704999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.629,"5min":1.2529999999999999,"15min":1.526},"min":{"1min":0.48299999999999998,"5min":0.41299999999999998,"15min":0.35999999999999999},"max":{"1min":1.0920000000000001,"5min":5.9139999999999997,"15min":34.930999999999997},"last":0.82099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:52:45.600 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:52:45.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:45 smithi104 ceph-mon[33507]: 5.1 scrub starts 2024-09-11T07:52:45.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:45 smithi104 ceph-mon[33507]: 5.1 scrub ok 2024-09-11T07:52:45.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:45 smithi104 ceph-mon[33507]: 4.1c scrub starts 2024-09-11T07:52:45.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:45 smithi104 ceph-mon[33507]: 4.1c scrub ok 2024-09-11T07:52:45.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:45 smithi104 ceph-mon[33507]: 6.1c scrub starts 2024-09-11T07:52:45.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:45 smithi104 ceph-mon[33507]: 6.1c scrub ok 2024-09-11T07:52:45.857 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:52:45.916 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:45 smithi022 ceph-mon[37330]: 5.1 scrub starts 2024-09-11T07:52:45.916 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:45 smithi022 ceph-mon[37330]: 5.1 scrub ok 2024-09-11T07:52:45.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:45 smithi022 ceph-mon[37330]: 4.1c scrub starts 2024-09-11T07:52:45.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:45 smithi022 ceph-mon[37330]: 4.1c scrub ok 2024-09-11T07:52:45.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:45 smithi022 ceph-mon[37330]: 6.1c scrub starts 2024-09-11T07:52:45.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:45 smithi022 ceph-mon[37330]: 6.1c scrub ok 2024-09-11T07:52:45.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:45 smithi022 ceph-mon[29462]: 5.1 scrub starts 2024-09-11T07:52:45.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:45 smithi022 ceph-mon[29462]: 5.1 scrub ok 2024-09-11T07:52:45.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:45 smithi022 ceph-mon[29462]: 4.1c scrub starts 2024-09-11T07:52:45.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:45 smithi022 ceph-mon[29462]: 4.1c scrub ok 2024-09-11T07:52:45.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:45 smithi022 ceph-mon[29462]: 6.1c scrub starts 2024-09-11T07:52:45.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:45 smithi022 ceph-mon[29462]: 6.1c scrub ok 2024-09-11T07:52:46.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:46 smithi022 ceph-mon[29462]: from='client.16569 v1:172.21.15.22:0/1829851548' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:52:46.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:46 smithi022 ceph-mon[29462]: 2.1 scrub starts 2024-09-11T07:52:46.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:46 smithi022 ceph-mon[29462]: 2.1 scrub ok 2024-09-11T07:52:46.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:46 smithi022 ceph-mon[29462]: pgmap v1237: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T07:52:46.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:46 smithi022 ceph-mon[29462]: 5.16 scrub starts 2024-09-11T07:52:46.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:46 smithi022 ceph-mon[29462]: 5.16 scrub ok 2024-09-11T07:52:46.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:46 smithi022 ceph-mon[29462]: 6.9 scrub starts 2024-09-11T07:52:46.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:46 smithi022 ceph-mon[29462]: 6.9 scrub ok 2024-09-11T07:52:46.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:46 smithi022 ceph-mon[37330]: from='client.16569 v1:172.21.15.22:0/1829851548' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:52:46.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:46 smithi022 ceph-mon[37330]: 2.1 scrub starts 2024-09-11T07:52:46.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:46 smithi022 ceph-mon[37330]: 2.1 scrub ok 2024-09-11T07:52:46.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:46 smithi022 ceph-mon[37330]: pgmap v1237: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T07:52:46.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:46 smithi022 ceph-mon[37330]: 5.16 scrub starts 2024-09-11T07:52:46.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:46 smithi022 ceph-mon[37330]: 5.16 scrub ok 2024-09-11T07:52:46.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:46 smithi022 ceph-mon[37330]: 6.9 scrub starts 2024-09-11T07:52:46.668 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:46 smithi022 ceph-mon[37330]: 6.9 scrub ok 2024-09-11T07:52:46.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:46 smithi104 ceph-mon[33507]: from='client.16569 v1:172.21.15.22:0/1829851548' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:52:46.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:46 smithi104 ceph-mon[33507]: 2.1 scrub starts 2024-09-11T07:52:46.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:46 smithi104 ceph-mon[33507]: 2.1 scrub ok 2024-09-11T07:52:46.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:46 smithi104 ceph-mon[33507]: pgmap v1237: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T07:52:46.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:46 smithi104 ceph-mon[33507]: 5.16 scrub starts 2024-09-11T07:52:46.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:46 smithi104 ceph-mon[33507]: 5.16 scrub ok 2024-09-11T07:52:46.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:46 smithi104 ceph-mon[33507]: 6.9 scrub starts 2024-09-11T07:52:46.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:46 smithi104 ceph-mon[33507]: 6.9 scrub ok 2024-09-11T07:52:47.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:47 smithi022 ceph-mon[29462]: 2.b scrub starts 2024-09-11T07:52:47.666 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:47 smithi022 ceph-mon[29462]: 2.b scrub ok 2024-09-11T07:52:47.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:47 smithi022 ceph-mon[37330]: 2.b scrub starts 2024-09-11T07:52:47.667 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:47 smithi022 ceph-mon[37330]: 2.b scrub ok 2024-09-11T07:52:47.776 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-11T07:52:47.778 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-11T07:52:47.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:47 smithi104 ceph-mon[33507]: 2.b scrub starts 2024-09-11T07:52:47.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:47 smithi104 ceph-mon[33507]: 2.b scrub ok 2024-09-11T07:52:48.492 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":1238,"stamp":"2024-09-11T07:52:47.572057+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":2684,"num_read_kb":2436,"num_write":1002,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":539,"num_bytes_recovered":996389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2124,"ondisk_log_size":2124,"up":426,"acting":426,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":426,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":6689840,"kb_used_data":9380,"kb_used_omap":65,"kb_used_meta":6680318,"kb_avail":743271376,"statfs":{"total":767960285184,"available":761109889024,"internally_reserved":0,"allocated":9605120,"data_stored":5783756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":66799,"internal_metadata":6840646417},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002703"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":59,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.544495+0000","last_change":"2024-09-11T07:52:20.420243+0000","last_active":"2024-09-11T07:52:36.544495+0000","last_peered":"2024-09-11T07:52:36.544495+0000","last_clean":"2024-09-11T07:52:36.544495+0000","last_became_active":"2024-09-11T07:51:48.206877+0000","last_became_peered":"2024-09-11T07:51:48.206877+0000","last_unstale":"2024-09-11T07:52:36.544495+0000","last_undegraded":"2024-09-11T07:52:36.544495+0000","last_fullsized":"2024-09-11T07:52:36.544495+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:20.420170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:46:28.323351+0000","last_clean_scrub_stamp":"2024-09-11T07:52:20.420170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:33.920201+0000","scrub_duration":0.0020486520000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":652,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861291+0000","last_change":"2024-09-11T07:52:21.808023+0000","last_active":"2024-09-11T07:52:35.861291+0000","last_peered":"2024-09-11T07:52:35.861291+0000","last_clean":"2024-09-11T07:52:35.861291+0000","last_became_active":"2024-09-11T07:50:56.906118+0000","last_became_peered":"2024-09-11T07:50:56.906118+0000","last_unstale":"2024-09-11T07:52:35.861291+0000","last_undegraded":"2024-09-11T07:52:35.861291+0000","last_fullsized":"2024-09-11T07:52:35.861291+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:21.807966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:47:24.201007+0000","last_clean_scrub_stamp":"2024-09-11T07:52:21.807966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:26.501637+0000","scrub_duration":0.001658262,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"426'46","reported_seq":706,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.544756+0000","last_change":"2024-09-11T07:52:13.488090+0000","last_active":"2024-09-11T07:52:36.544756+0000","last_peered":"2024-09-11T07:52:36.544756+0000","last_clean":"2024-09-11T07:52:36.544756+0000","last_became_active":"2024-09-11T07:50:53.904640+0000","last_became_peered":"2024-09-11T07:50:53.904640+0000","last_unstale":"2024-09-11T07:52:36.544756+0000","last_undegraded":"2024-09-11T07:52:36.544756+0000","last_fullsized":"2024-09-11T07:52:36.544756+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'46","last_scrub_stamp":"2024-09-11T07:52:13.487992+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-09-11T07:27:50.756707+0000","last_clean_scrub_stamp":"2024-09-11T07:52:13.487992+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:34.823509+0000","scrub_duration":0.002954557,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":64,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861184+0000","last_change":"2024-09-11T07:51:55.154414+0000","last_active":"2024-09-11T07:52:35.861184+0000","last_peered":"2024-09-11T07:52:35.861184+0000","last_clean":"2024-09-11T07:52:35.861184+0000","last_became_active":"2024-09-11T07:50:50.862411+0000","last_became_peered":"2024-09-11T07:50:50.862411+0000","last_unstale":"2024-09-11T07:52:35.861184+0000","last_undegraded":"2024-09-11T07:52:35.861184+0000","last_fullsized":"2024-09-11T07:52:35.861184+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:55.154345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:51:55.154345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:15.076843+0000","scrub_duration":0.0013346720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":661,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.545126+0000","last_change":"2024-09-11T07:51:34.321543+0000","last_active":"2024-09-11T07:52:36.545126+0000","last_peered":"2024-09-11T07:52:36.545126+0000","last_clean":"2024-09-11T07:52:36.545126+0000","last_became_active":"2024-09-11T07:51:34.320960+0000","last_became_peered":"2024-09-11T07:51:34.320960+0000","last_unstale":"2024-09-11T07:52:36.545126+0000","last_undegraded":"2024-09-11T07:52:36.545126+0000","last_fullsized":"2024-09-11T07:52:36.545126+0000","mapping_epoch":509,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":510,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:28.443214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:19.655401+0000","last_clean_scrub_stamp":"2024-09-11T07:51:28.443214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:50.940231+0000","scrub_duration":0.001291483,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"426'25","reported_seq":84,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861178+0000","last_change":"2024-09-11T07:52:21.165183+0000","last_active":"2024-09-11T07:52:35.861178+0000","last_peered":"2024-09-11T07:52:35.861178+0000","last_clean":"2024-09-11T07:52:35.861178+0000","last_became_active":"2024-09-11T07:50:50.861997+0000","last_became_peered":"2024-09-11T07:50:50.861997+0000","last_unstale":"2024-09-11T07:52:35.861178+0000","last_undegraded":"2024-09-11T07:52:35.861178+0000","last_fullsized":"2024-09-11T07:52:35.861178+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'25","last_scrub_stamp":"2024-09-11T07:52:21.165111+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:36:42.708941+0000","last_clean_scrub_stamp":"2024-09-11T07:52:21.165111+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:34.072089+0000","scrub_duration":0.0036606170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":526,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-11T07:52:42.835180+0000","last_change":"2024-09-11T07:52:42.835155+0000","last_active":"2024-09-11T07:52:42.835180+0000","last_peered":"2024-09-11T07:52:42.835180+0000","last_clean":"2024-09-11T07:52:42.835180+0000","last_became_active":"2024-09-11T07:51:14.064108+0000","last_became_peered":"2024-09-11T07:51:14.064108+0000","last_unstale":"2024-09-11T07:52:42.835180+0000","last_undegraded":"2024-09-11T07:52:42.835180+0000","last_fullsized":"2024-09-11T07:52:42.835180+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:42.835099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:52:42.835099+0000","last_clean_scrub_stamp":"2024-09-11T07:52:42.835099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:58.673027+0000","scrub_duration":0.0014183049999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":74,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861707+0000","last_change":"2024-09-11T07:51:39.194332+0000","last_active":"2024-09-11T07:52:35.861707+0000","last_peered":"2024-09-11T07:52:35.861707+0000","last_clean":"2024-09-11T07:52:35.861707+0000","last_became_active":"2024-09-11T07:51:09.045103+0000","last_became_peered":"2024-09-11T07:51:09.045103+0000","last_unstale":"2024-09-11T07:52:35.861707+0000","last_undegraded":"2024-09-11T07:52:35.861707+0000","last_fullsized":"2024-09-11T07:52:35.861707+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:39.194263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:51:39.194263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:03.055189+0000","scrub_duration":0.0014768450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":69,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:40.523898+0000","last_change":"2024-09-11T07:52:40.523852+0000","last_active":"2024-09-11T07:52:40.523898+0000","last_peered":"2024-09-11T07:52:40.523898+0000","last_clean":"2024-09-11T07:52:40.523898+0000","last_became_active":"2024-09-11T07:50:56.912957+0000","last_became_peered":"2024-09-11T07:50:56.912957+0000","last_unstale":"2024-09-11T07:52:40.523898+0000","last_undegraded":"2024-09-11T07:52:40.523898+0000","last_fullsized":"2024-09-11T07:52:40.523898+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:40.523771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:48:37.724091+0000","last_clean_scrub_stamp":"2024-09-11T07:52:40.523771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:03.482981+0000","scrub_duration":0.0016190499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"426'11","reported_seq":112,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861525+0000","last_change":"2024-09-11T07:52:29.226069+0000","last_active":"2024-09-11T07:52:35.861525+0000","last_peered":"2024-09-11T07:52:35.861525+0000","last_clean":"2024-09-11T07:52:35.861525+0000","last_became_active":"2024-09-11T07:51:09.045188+0000","last_became_peered":"2024-09-11T07:51:09.045188+0000","last_unstale":"2024-09-11T07:52:35.861525+0000","last_undegraded":"2024-09-11T07:52:35.861525+0000","last_fullsized":"2024-09-11T07:52:35.861525+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'11","last_scrub_stamp":"2024-09-11T07:52:29.226003+0000","last_deep_scrub":"234'9","last_deep_scrub_stamp":"2024-09-11T07:45:32.375156+0000","last_clean_scrub_stamp":"2024-09-11T07:52:29.226003+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:35.352619+0000","scrub_duration":0.001949267,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":13,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":69,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.599124+0000","last_change":"2024-09-11T07:52:23.685295+0000","last_active":"2024-09-11T07:52:36.599124+0000","last_peered":"2024-09-11T07:52:36.599124+0000","last_clean":"2024-09-11T07:52:36.599124+0000","last_became_active":"2024-09-11T07:50:56.910585+0000","last_became_peered":"2024-09-11T07:50:56.910585+0000","last_unstale":"2024-09-11T07:52:36.599124+0000","last_undegraded":"2024-09-11T07:52:36.599124+0000","last_fullsized":"2024-09-11T07:52:36.599124+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:23.685224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:52:23.685224+0000","last_clean_scrub_stamp":"2024-09-11T07:52:23.685224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:37.473468+0000","scrub_duration":0.001870053,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":207,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642376+0000","last_change":"2024-09-11T07:52:07.526749+0000","last_active":"2024-09-11T07:52:36.642376+0000","last_peered":"2024-09-11T07:52:36.642376+0000","last_clean":"2024-09-11T07:52:36.642376+0000","last_became_active":"2024-09-11T07:51:37.859052+0000","last_became_peered":"2024-09-11T07:51:37.859052+0000","last_unstale":"2024-09-11T07:52:36.642376+0000","last_undegraded":"2024-09-11T07:52:36.642376+0000","last_fullsized":"2024-09-11T07:52:36.642376+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:07.526676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:45.149916+0000","last_clean_scrub_stamp":"2024-09-11T07:52:07.526676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:12.818897+0000","scrub_duration":0.0020718569999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":512,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642758+0000","last_change":"2024-09-11T07:52:15.610486+0000","last_active":"2024-09-11T07:52:36.642758+0000","last_peered":"2024-09-11T07:52:36.642758+0000","last_clean":"2024-09-11T07:52:36.642758+0000","last_became_active":"2024-09-11T07:51:37.859870+0000","last_became_peered":"2024-09-11T07:51:37.859870+0000","last_unstale":"2024-09-11T07:52:36.642758+0000","last_undegraded":"2024-09-11T07:52:36.642758+0000","last_fullsized":"2024-09-11T07:52:36.642758+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:15.610418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:47:19.164579+0000","last_clean_scrub_stamp":"2024-09-11T07:52:15.610418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:43.312842+0000","scrub_duration":0.0017306789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"426'25","reported_seq":74,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861415+0000","last_change":"2024-09-11T07:51:49.117720+0000","last_active":"2024-09-11T07:52:35.861415+0000","last_peered":"2024-09-11T07:52:35.861415+0000","last_clean":"2024-09-11T07:52:35.861415+0000","last_became_active":"2024-09-11T07:50:50.870738+0000","last_became_peered":"2024-09-11T07:50:50.870738+0000","last_unstale":"2024-09-11T07:52:35.861415+0000","last_undegraded":"2024-09-11T07:52:35.861415+0000","last_fullsized":"2024-09-11T07:52:35.861415+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'25","last_scrub_stamp":"2024-09-11T07:51:49.117649+0000","last_deep_scrub":"136'13","last_deep_scrub_stamp":"2024-09-11T07:31:08.042198+0000","last_clean_scrub_stamp":"2024-09-11T07:51:49.117649+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:06.774558+0000","scrub_duration":0.0023084400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":649,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-11T07:52:43.749692+0000","last_change":"2024-09-11T07:52:43.749669+0000","last_active":"2024-09-11T07:52:43.749692+0000","last_peered":"2024-09-11T07:52:43.749692+0000","last_clean":"2024-09-11T07:52:43.749692+0000","last_became_active":"2024-09-11T07:51:48.205734+0000","last_became_peered":"2024-09-11T07:51:48.205734+0000","last_unstale":"2024-09-11T07:52:43.749692+0000","last_undegraded":"2024-09-11T07:52:43.749692+0000","last_fullsized":"2024-09-11T07:52:43.749692+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:43.749615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:52:43.749615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:04.752905+0000","scrub_duration":0.0015164589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":646,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.860342+0000","last_change":"2024-09-11T07:52:08.761674+0000","last_active":"2024-09-11T07:52:35.860342+0000","last_peered":"2024-09-11T07:52:35.860342+0000","last_clean":"2024-09-11T07:52:35.860342+0000","last_became_active":"2024-09-11T07:50:56.906207+0000","last_became_peered":"2024-09-11T07:50:56.906207+0000","last_unstale":"2024-09-11T07:52:35.860342+0000","last_undegraded":"2024-09-11T07:52:35.860342+0000","last_fullsized":"2024-09-11T07:52:35.860342+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:08.761607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:52:08.761607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:28.344892+0000","scrub_duration":0.001579762,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.8","version":"87'81","reported_seq":77,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642323+0000","last_change":"2024-09-11T07:51:37.859022+0000","last_active":"2024-09-11T07:52:36.642323+0000","last_peered":"2024-09-11T07:52:36.642323+0000","last_clean":"2024-09-11T07:52:36.642323+0000","last_became_active":"2024-09-11T07:51:37.858651+0000","last_became_peered":"2024-09-11T07:51:37.858651+0000","last_unstale":"2024-09-11T07:52:36.642323+0000","last_undegraded":"2024-09-11T07:52:36.642323+0000","last_fullsized":"2024-09-11T07:52:36.642323+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:51:32.046031+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:51:32.046031+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:48.720658+0000","scrub_duration":0.0023556440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"496'53","reported_seq":128,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861746+0000","last_change":"2024-09-11T07:52:18.155170+0000","last_active":"2024-09-11T07:52:35.861746+0000","last_peered":"2024-09-11T07:52:35.861746+0000","last_clean":"2024-09-11T07:52:35.861746+0000","last_became_active":"2024-09-11T07:50:53.916590+0000","last_became_peered":"2024-09-11T07:50:53.916590+0000","last_unstale":"2024-09-11T07:52:35.861746+0000","last_undegraded":"2024-09-11T07:52:35.861746+0000","last_fullsized":"2024-09-11T07:52:35.861746+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":0,"last_scrub":"496'53","last_scrub_stamp":"2024-09-11T07:52:18.155106+0000","last_deep_scrub":"496'53","last_deep_scrub_stamp":"2024-09-11T07:51:00.998091+0000","last_clean_scrub_stamp":"2024-09-11T07:52:18.155106+0000","objects_scrubbed":8,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:29.317629+0000","scrub_duration":0.0023213600000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":69,"num_read_kb":59,"num_write":46,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":59,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.599374+0000","last_change":"2024-09-11T07:51:17.562232+0000","last_active":"2024-09-11T07:52:36.599374+0000","last_peered":"2024-09-11T07:52:36.599374+0000","last_clean":"2024-09-11T07:52:36.599374+0000","last_became_active":"2024-09-11T07:50:56.910651+0000","last_became_peered":"2024-09-11T07:50:56.910651+0000","last_unstale":"2024-09-11T07:52:36.599374+0000","last_undegraded":"2024-09-11T07:52:36.599374+0000","last_fullsized":"2024-09-11T07:52:36.599374+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:17.562160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:51:17.562160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001615266,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":66,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861657+0000","last_change":"2024-09-11T07:52:05.823115+0000","last_active":"2024-09-11T07:52:35.861657+0000","last_peered":"2024-09-11T07:52:35.861657+0000","last_clean":"2024-09-11T07:52:35.861657+0000","last_became_active":"2024-09-11T07:51:14.061875+0000","last_became_peered":"2024-09-11T07:51:14.061875+0000","last_unstale":"2024-09-11T07:52:35.861657+0000","last_undegraded":"2024-09-11T07:52:35.861657+0000","last_fullsized":"2024-09-11T07:52:35.861657+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:05.823058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:52:05.823058+0000","last_clean_scrub_stamp":"2024-09-11T07:52:05.823058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:09.724551+0000","scrub_duration":0.0016080650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":61,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642521+0000","last_change":"2024-09-11T07:51:48.250235+0000","last_active":"2024-09-11T07:52:36.642521+0000","last_peered":"2024-09-11T07:52:36.642521+0000","last_clean":"2024-09-11T07:52:36.642521+0000","last_became_active":"2024-09-11T07:50:53.906064+0000","last_became_peered":"2024-09-11T07:50:53.906064+0000","last_unstale":"2024-09-11T07:52:36.642521+0000","last_undegraded":"2024-09-11T07:52:36.642521+0000","last_fullsized":"2024-09-11T07:52:36.642521+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:48.250179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:50:40.403125+0000","last_clean_scrub_stamp":"2024-09-11T07:51:48.250179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:10.988456+0000","scrub_duration":0.001584043,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"87'81","reported_seq":59,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.598855+0000","last_change":"2024-09-11T07:51:34.528144+0000","last_active":"2024-09-11T07:52:36.598855+0000","last_peered":"2024-09-11T07:52:36.598855+0000","last_clean":"2024-09-11T07:52:36.598855+0000","last_became_active":"2024-09-11T07:50:56.908493+0000","last_became_peered":"2024-09-11T07:50:56.908493+0000","last_unstale":"2024-09-11T07:52:36.598855+0000","last_undegraded":"2024-09-11T07:52:36.598855+0000","last_fullsized":"2024-09-11T07:52:36.598855+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:51:34.528009+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:51:34.528009+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020188860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"426'35","reported_seq":76,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861542+0000","last_change":"2024-09-11T07:51:41.210125+0000","last_active":"2024-09-11T07:52:35.861542+0000","last_peered":"2024-09-11T07:52:35.861542+0000","last_clean":"2024-09-11T07:52:35.861542+0000","last_became_active":"2024-09-11T07:50:50.866011+0000","last_became_peered":"2024-09-11T07:50:50.866011+0000","last_unstale":"2024-09-11T07:52:35.861542+0000","last_undegraded":"2024-09-11T07:52:35.861542+0000","last_fullsized":"2024-09-11T07:52:35.861542+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'35","last_scrub_stamp":"2024-09-11T07:51:41.210061+0000","last_deep_scrub":"112'15","last_deep_scrub_stamp":"2024-09-11T07:31:10.005896+0000","last_clean_scrub_stamp":"2024-09-11T07:51:41.210061+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:53.242433+0000","scrub_duration":0.0037170250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":38,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":470,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861165+0000","last_change":"2024-09-11T07:51:42.925663+0000","last_active":"2024-09-11T07:52:35.861165+0000","last_peered":"2024-09-11T07:52:35.861165+0000","last_clean":"2024-09-11T07:52:35.861165+0000","last_became_active":"2024-09-11T07:51:14.062395+0000","last_became_peered":"2024-09-11T07:51:14.062395+0000","last_unstale":"2024-09-11T07:52:35.861165+0000","last_undegraded":"2024-09-11T07:52:35.861165+0000","last_fullsized":"2024-09-11T07:52:35.861165+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:42.925597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:51:42.925597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:43.536725+0000","scrub_duration":0.0014936229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"528'18","reported_seq":128,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-11T07:52:40.861144+0000","last_change":"2024-09-11T07:52:18.341124+0000","last_active":"2024-09-11T07:52:40.861144+0000","last_peered":"2024-09-11T07:52:40.861144+0000","last_clean":"2024-09-11T07:52:40.861144+0000","last_became_active":"2024-09-11T07:51:34.324704+0000","last_became_peered":"2024-09-11T07:51:34.324704+0000","last_unstale":"2024-09-11T07:52:40.861144+0000","last_undegraded":"2024-09-11T07:52:40.861144+0000","last_fullsized":"2024-09-11T07:52:40.861144+0000","mapping_epoch":509,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":510,"parent":"0.0","parent_split_bits":0,"last_scrub":"528'18","last_scrub_stamp":"2024-09-11T07:52:18.341070+0000","last_deep_scrub":"528'18","last_deep_scrub_stamp":"2024-09-11T07:52:18.341070+0000","last_clean_scrub_stamp":"2024-09-11T07:52:18.341070+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:30.963958+0000","scrub_duration":0.002956728,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":93,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861770+0000","last_change":"2024-09-11T07:52:16.224361+0000","last_active":"2024-09-11T07:52:35.861770+0000","last_peered":"2024-09-11T07:52:35.861770+0000","last_clean":"2024-09-11T07:52:35.861770+0000","last_became_active":"2024-09-11T07:51:09.045883+0000","last_became_peered":"2024-09-11T07:51:09.045883+0000","last_unstale":"2024-09-11T07:52:35.861770+0000","last_undegraded":"2024-09-11T07:52:35.861770+0000","last_fullsized":"2024-09-11T07:52:35.861770+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:16.224291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:52:16.224291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:22.931969+0000","scrub_duration":0.0017219640000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"87'81","reported_seq":46,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642802+0000","last_change":"2024-09-11T07:51:51.553544+0000","last_active":"2024-09-11T07:52:36.642802+0000","last_peered":"2024-09-11T07:52:36.642802+0000","last_clean":"2024-09-11T07:52:36.642802+0000","last_became_active":"2024-09-11T07:51:37.864802+0000","last_became_peered":"2024-09-11T07:51:37.864802+0000","last_unstale":"2024-09-11T07:52:36.642802+0000","last_undegraded":"2024-09-11T07:52:36.642802+0000","last_fullsized":"2024-09-11T07:52:36.642802+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:51:51.553432+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:51:51.553432+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:14.493316+0000","scrub_duration":0.0017339759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"426'46","reported_seq":425,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:40.786951+0000","last_change":"2024-09-11T07:52:40.786870+0000","last_active":"2024-09-11T07:52:40.786951+0000","last_peered":"2024-09-11T07:52:40.786951+0000","last_clean":"2024-09-11T07:52:40.786951+0000","last_became_active":"2024-09-11T07:51:14.060758+0000","last_became_peered":"2024-09-11T07:51:14.060758+0000","last_unstale":"2024-09-11T07:52:40.786951+0000","last_undegraded":"2024-09-11T07:52:40.786951+0000","last_fullsized":"2024-09-11T07:52:40.786951+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'46","last_scrub_stamp":"2024-09-11T07:52:40.786800+0000","last_deep_scrub":"426'46","last_deep_scrub_stamp":"2024-09-11T07:52:40.786800+0000","last_clean_scrub_stamp":"2024-09-11T07:52:40.786800+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:04.122981+0000","scrub_duration":0.0028706370000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":60,"num_read_kb":51,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":481,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:37.758832+0000","last_change":"2024-09-11T07:52:37.758749+0000","last_active":"2024-09-11T07:52:37.758832+0000","last_peered":"2024-09-11T07:52:37.758832+0000","last_clean":"2024-09-11T07:52:37.758832+0000","last_became_active":"2024-09-11T07:52:13.489096+0000","last_became_peered":"2024-09-11T07:52:13.489096+0000","last_unstale":"2024-09-11T07:52:37.758832+0000","last_undegraded":"2024-09-11T07:52:37.758832+0000","last_fullsized":"2024-09-11T07:52:37.758832+0000","mapping_epoch":526,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":527,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:37.758609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:27.416207+0000","last_clean_scrub_stamp":"2024-09-11T07:52:37.758609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:49.480063+0000","scrub_duration":0.0017490310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"493'12","reported_seq":90,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:39.825289+0000","last_change":"2024-09-11T07:52:10.276621+0000","last_active":"2024-09-11T07:52:39.825289+0000","last_peered":"2024-09-11T07:52:39.825289+0000","last_clean":"2024-09-11T07:52:39.825289+0000","last_became_active":"2024-09-11T07:50:50.874198+0000","last_became_peered":"2024-09-11T07:50:50.874198+0000","last_unstale":"2024-09-11T07:52:39.825289+0000","last_undegraded":"2024-09-11T07:52:39.825289+0000","last_fullsized":"2024-09-11T07:52:39.825289+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"493'12","last_scrub_stamp":"2024-09-11T07:52:10.276547+0000","last_deep_scrub":"366'9","last_deep_scrub_stamp":"2024-09-11T07:43:26.334501+0000","last_clean_scrub_stamp":"2024-09-11T07:52:10.276547+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:12.125028+0000","scrub_duration":0.0019136960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":485,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642855+0000","last_change":"2024-09-11T07:52:35.698778+0000","last_active":"2024-09-11T07:52:36.642855+0000","last_peered":"2024-09-11T07:52:36.642855+0000","last_clean":"2024-09-11T07:52:36.642855+0000","last_became_active":"2024-09-11T07:51:37.859466+0000","last_became_peered":"2024-09-11T07:51:37.859466+0000","last_unstale":"2024-09-11T07:52:36.642855+0000","last_undegraded":"2024-09-11T07:52:36.642855+0000","last_fullsized":"2024-09-11T07:52:36.642855+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:35.698708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:30:31.200685+0000","last_clean_scrub_stamp":"2024-09-11T07:52:35.698708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:48.789509+0000","scrub_duration":0.001681475,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"426'32","reported_seq":98,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.862062+0000","last_change":"2024-09-11T07:51:50.090512+0000","last_active":"2024-09-11T07:52:35.862062+0000","last_peered":"2024-09-11T07:52:35.862062+0000","last_clean":"2024-09-11T07:52:35.862062+0000","last_became_active":"2024-09-11T07:51:09.046471+0000","last_became_peered":"2024-09-11T07:51:09.046471+0000","last_unstale":"2024-09-11T07:52:35.862062+0000","last_undegraded":"2024-09-11T07:52:35.862062+0000","last_fullsized":"2024-09-11T07:52:35.862062+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'32","last_scrub_stamp":"2024-09-11T07:51:50.090443+0000","last_deep_scrub":"107'16","last_deep_scrub_stamp":"2024-09-11T07:29:38.112111+0000","last_clean_scrub_stamp":"2024-09-11T07:51:50.090443+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:05.891441+0000","scrub_duration":0.0022821479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":69,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.598879+0000","last_change":"2024-09-11T07:52:27.676312+0000","last_active":"2024-09-11T07:52:36.598879+0000","last_peered":"2024-09-11T07:52:36.598879+0000","last_clean":"2024-09-11T07:52:36.598879+0000","last_became_active":"2024-09-11T07:50:56.904614+0000","last_became_peered":"2024-09-11T07:50:56.904614+0000","last_unstale":"2024-09-11T07:52:36.598879+0000","last_undegraded":"2024-09-11T07:52:36.598879+0000","last_fullsized":"2024-09-11T07:52:36.598879+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:27.676249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:43:24.643676+0000","last_clean_scrub_stamp":"2024-09-11T07:52:27.676249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:57.320431+0000","scrub_duration":0.0018964139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":386,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.643049+0000","last_change":"2024-09-11T07:51:37.864978+0000","last_active":"2024-09-11T07:52:36.643049+0000","last_peered":"2024-09-11T07:52:36.643049+0000","last_clean":"2024-09-11T07:52:36.643049+0000","last_became_active":"2024-09-11T07:51:37.863388+0000","last_became_peered":"2024-09-11T07:51:37.863388+0000","last_unstale":"2024-09-11T07:52:36.643049+0000","last_undegraded":"2024-09-11T07:52:36.643049+0000","last_fullsized":"2024-09-11T07:52:36.643049+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:31.195206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:19.714568+0000","last_clean_scrub_stamp":"2024-09-11T07:51:31.195206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:51.515704+0000","scrub_duration":0.0078526010000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":579,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.544802+0000","last_change":"2024-09-11T07:51:17.351548+0000","last_active":"2024-09-11T07:52:36.544802+0000","last_peered":"2024-09-11T07:52:36.544802+0000","last_clean":"2024-09-11T07:52:36.544802+0000","last_became_active":"2024-09-11T07:26:37.861370+0000","last_became_peered":"2024-09-11T07:26:37.861370+0000","last_unstale":"2024-09-11T07:52:36.544802+0000","last_undegraded":"2024-09-11T07:52:36.544802+0000","last_fullsized":"2024-09-11T07:52:36.544802+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:17.351485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:48:31.314305+0000","last_clean_scrub_stamp":"2024-09-11T07:51:17.351485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019907700000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"87'81","reported_seq":244,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.544633+0000","last_change":"2024-09-11T07:51:57.398645+0000","last_active":"2024-09-11T07:52:36.544633+0000","last_peered":"2024-09-11T07:52:36.544633+0000","last_clean":"2024-09-11T07:52:36.544633+0000","last_became_active":"2024-09-11T07:51:48.201002+0000","last_became_peered":"2024-09-11T07:51:48.201002+0000","last_unstale":"2024-09-11T07:52:36.544633+0000","last_undegraded":"2024-09-11T07:52:36.544633+0000","last_fullsized":"2024-09-11T07:52:36.544633+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:51:57.398587+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:51:57.398587+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:16.586047+0000","scrub_duration":0.00197993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":983040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"482'49","reported_seq":80,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861367+0000","last_change":"2024-09-11T07:51:59.199379+0000","last_active":"2024-09-11T07:52:35.861367+0000","last_peered":"2024-09-11T07:52:35.861367+0000","last_clean":"2024-09-11T07:52:35.861367+0000","last_became_active":"2024-09-11T07:50:50.859433+0000","last_became_peered":"2024-09-11T07:50:50.859433+0000","last_unstale":"2024-09-11T07:52:35.861367+0000","last_undegraded":"2024-09-11T07:52:35.861367+0000","last_fullsized":"2024-09-11T07:52:35.861367+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"482'49","last_scrub_stamp":"2024-09-11T07:51:59.199312+0000","last_deep_scrub":"482'49","last_deep_scrub_stamp":"2024-09-11T07:50:28.592061+0000","last_clean_scrub_stamp":"2024-09-11T07:51:59.199312+0000","objects_scrubbed":7,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:02.976284+0000","scrub_duration":0.0020065500000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":55,"num_write":42,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":276,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.628891+0000","last_change":"2024-09-11T07:51:48.207708+0000","last_active":"2024-09-11T07:52:36.628891+0000","last_peered":"2024-09-11T07:52:36.628891+0000","last_clean":"2024-09-11T07:52:36.628891+0000","last_became_active":"2024-09-11T07:51:48.206731+0000","last_became_peered":"2024-09-11T07:51:48.206731+0000","last_unstale":"2024-09-11T07:52:36.628891+0000","last_undegraded":"2024-09-11T07:52:36.628891+0000","last_fullsized":"2024-09-11T07:52:36.628891+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:40.135512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:51:40.135512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:48.257364+0000","scrub_duration":0.0015432390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"493'11","reported_seq":934,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-11T07:52:41.255406+0000","last_change":"2024-09-11T07:52:25.442066+0000","last_active":"2024-09-11T07:52:41.255406+0000","last_peered":"2024-09-11T07:52:41.255406+0000","last_clean":"2024-09-11T07:52:41.255406+0000","last_became_active":"2024-09-11T07:50:50.854861+0000","last_became_peered":"2024-09-11T07:50:50.854861+0000","last_unstale":"2024-09-11T07:52:41.255406+0000","last_undegraded":"2024-09-11T07:52:41.255406+0000","last_fullsized":"2024-09-11T07:52:41.255406+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"493'11","last_scrub_stamp":"2024-09-11T07:52:25.442006+0000","last_deep_scrub":"395'10","last_deep_scrub_stamp":"2024-09-11T07:50:52.366462+0000","last_clean_scrub_stamp":"2024-09-11T07:52:25.442006+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:26.785729+0000","scrub_duration":0.0018460810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":69,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.599398+0000","last_change":"2024-09-11T07:52:28.629545+0000","last_active":"2024-09-11T07:52:36.599398+0000","last_peered":"2024-09-11T07:52:36.599398+0000","last_clean":"2024-09-11T07:52:36.599398+0000","last_became_active":"2024-09-11T07:50:56.908521+0000","last_became_peered":"2024-09-11T07:50:56.908521+0000","last_unstale":"2024-09-11T07:52:36.599398+0000","last_undegraded":"2024-09-11T07:52:36.599398+0000","last_fullsized":"2024-09-11T07:52:36.599398+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:28.629478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:52:28.629478+0000","last_clean_scrub_stamp":"2024-09-11T07:52:28.629478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:50.098037+0000","scrub_duration":0.001653182,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"87'81","reported_seq":84,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861785+0000","last_change":"2024-09-11T07:52:27.262912+0000","last_active":"2024-09-11T07:52:35.861785+0000","last_peered":"2024-09-11T07:52:35.861785+0000","last_clean":"2024-09-11T07:52:35.861785+0000","last_became_active":"2024-09-11T07:50:53.916379+0000","last_became_peered":"2024-09-11T07:50:53.916379+0000","last_unstale":"2024-09-11T07:52:35.861785+0000","last_undegraded":"2024-09-11T07:52:35.861785+0000","last_fullsized":"2024-09-11T07:52:35.861785+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:52:27.262822+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:50:58.031293+0000","last_clean_scrub_stamp":"2024-09-11T07:52:27.262822+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:52.165620+0000","scrub_duration":0.0015215350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"426'50","reported_seq":95,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.543867+0000","last_change":"2024-09-11T07:51:49.603296+0000","last_active":"2024-09-11T07:52:36.543867+0000","last_peered":"2024-09-11T07:52:36.543867+0000","last_clean":"2024-09-11T07:52:36.543867+0000","last_became_active":"2024-09-11T07:51:48.199106+0000","last_became_peered":"2024-09-11T07:51:48.199106+0000","last_unstale":"2024-09-11T07:52:36.543867+0000","last_undegraded":"2024-09-11T07:52:36.543867+0000","last_fullsized":"2024-09-11T07:52:36.543867+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'50","last_scrub_stamp":"2024-09-11T07:51:49.603219+0000","last_deep_scrub":"234'38","last_deep_scrub_stamp":"2024-09-11T07:46:26.243126+0000","last_clean_scrub_stamp":"2024-09-11T07:51:49.603219+0000","objects_scrubbed":8,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:57.505566+0000","scrub_duration":0.002432479,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":69,"num_read_kb":57,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1100,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":85,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-11T07:52:41.787428+0000","last_change":"2024-09-11T07:52:41.787403+0000","last_active":"2024-09-11T07:52:41.787428+0000","last_peered":"2024-09-11T07:52:41.787428+0000","last_clean":"2024-09-11T07:52:41.787428+0000","last_became_active":"2024-09-11T07:51:14.059556+0000","last_became_peered":"2024-09-11T07:51:14.059556+0000","last_unstale":"2024-09-11T07:52:41.787428+0000","last_undegraded":"2024-09-11T07:52:41.787428+0000","last_fullsized":"2024-09-11T07:52:41.787428+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:41.787347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:51:21.084654+0000","last_clean_scrub_stamp":"2024-09-11T07:52:41.787347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:01.100290+0000","scrub_duration":0.0018192200000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":74,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861417+0000","last_change":"2024-09-11T07:51:48.196635+0000","last_active":"2024-09-11T07:52:35.861417+0000","last_peered":"2024-09-11T07:52:35.861417+0000","last_clean":"2024-09-11T07:52:35.861417+0000","last_became_active":"2024-09-11T07:51:48.196080+0000","last_became_peered":"2024-09-11T07:51:48.196080+0000","last_unstale":"2024-09-11T07:52:35.861417+0000","last_undegraded":"2024-09-11T07:52:35.861417+0000","last_fullsized":"2024-09-11T07:52:35.861417+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:30.238175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:27.968579+0000","last_clean_scrub_stamp":"2024-09-11T07:51:30.238175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:44.121833+0000","scrub_duration":0.0028010420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":84,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861259+0000","last_change":"2024-09-11T07:51:43.204972+0000","last_active":"2024-09-11T07:52:35.861259+0000","last_peered":"2024-09-11T07:52:35.861259+0000","last_clean":"2024-09-11T07:52:35.861259+0000","last_became_active":"2024-09-11T07:51:19.095139+0000","last_became_peered":"2024-09-11T07:51:19.095139+0000","last_unstale":"2024-09-11T07:52:35.861259+0000","last_undegraded":"2024-09-11T07:52:35.861259+0000","last_fullsized":"2024-09-11T07:52:35.861259+0000","mapping_epoch":504,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":505,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:43.204907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:29.978798+0000","last_clean_scrub_stamp":"2024-09-11T07:51:43.204907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:46.701601+0000","scrub_duration":0.0015091830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"87'81","reported_seq":130,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.628914+0000","last_change":"2024-09-11T07:51:50.745497+0000","last_active":"2024-09-11T07:52:36.628914+0000","last_peered":"2024-09-11T07:52:36.628914+0000","last_clean":"2024-09-11T07:52:36.628914+0000","last_became_active":"2024-09-11T07:51:45.187794+0000","last_became_peered":"2024-09-11T07:51:45.187794+0000","last_unstale":"2024-09-11T07:52:36.628914+0000","last_undegraded":"2024-09-11T07:52:36.628914+0000","last_fullsized":"2024-09-11T07:52:36.628914+0000","mapping_epoch":514,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":515,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:51:50.745359+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:51:50.745359+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:59.502445+0000","scrub_duration":0.0017247079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"426'42","reported_seq":735,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.860668+0000","last_change":"2024-09-11T07:52:34.766519+0000","last_active":"2024-09-11T07:52:35.860668+0000","last_peered":"2024-09-11T07:52:35.860668+0000","last_clean":"2024-09-11T07:52:35.860668+0000","last_became_active":"2024-09-11T07:50:56.906069+0000","last_became_peered":"2024-09-11T07:50:56.906069+0000","last_unstale":"2024-09-11T07:52:35.860668+0000","last_undegraded":"2024-09-11T07:52:35.860668+0000","last_fullsized":"2024-09-11T07:52:35.860668+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'42","last_scrub_stamp":"2024-09-11T07:52:34.766460+0000","last_deep_scrub":"236'32","last_deep_scrub_stamp":"2024-09-11T07:36:50.023909+0000","last_clean_scrub_stamp":"2024-09-11T07:52:34.766460+0000","objects_scrubbed":8,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:55.771951+0000","scrub_duration":0.002274206,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":69,"num_read_kb":55,"num_write":42,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":59,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.599528+0000","last_change":"2024-09-11T07:51:56.568463+0000","last_active":"2024-09-11T07:52:36.599528+0000","last_peered":"2024-09-11T07:52:36.599528+0000","last_clean":"2024-09-11T07:52:36.599528+0000","last_became_active":"2024-09-11T07:50:56.911468+0000","last_became_peered":"2024-09-11T07:50:56.911468+0000","last_unstale":"2024-09-11T07:52:36.599528+0000","last_undegraded":"2024-09-11T07:52:36.599528+0000","last_fullsized":"2024-09-11T07:52:36.599528+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:56.568389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:24.470602+0000","last_clean_scrub_stamp":"2024-09-11T07:51:56.568389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:02.603574+0000","scrub_duration":0.0017941400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":262,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.628575+0000","last_change":"2024-09-11T07:52:01.856460+0000","last_active":"2024-09-11T07:52:36.628575+0000","last_peered":"2024-09-11T07:52:36.628575+0000","last_clean":"2024-09-11T07:52:36.628575+0000","last_became_active":"2024-09-11T07:51:45.190268+0000","last_became_peered":"2024-09-11T07:51:45.190268+0000","last_unstale":"2024-09-11T07:52:36.628575+0000","last_undegraded":"2024-09-11T07:52:36.628575+0000","last_fullsized":"2024-09-11T07:52:36.628575+0000","mapping_epoch":514,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":515,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:01.856392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:22.285763+0000","last_clean_scrub_stamp":"2024-09-11T07:52:01.856392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:16.324863+0000","scrub_duration":0.003168737,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":448,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.859930+0000","last_change":"2024-09-11T07:51:30.893785+0000","last_active":"2024-09-11T07:52:35.859930+0000","last_peered":"2024-09-11T07:52:35.859930+0000","last_clean":"2024-09-11T07:52:35.859930+0000","last_became_active":"2024-09-11T07:51:14.057166+0000","last_became_peered":"2024-09-11T07:51:14.057166+0000","last_unstale":"2024-09-11T07:52:35.859930+0000","last_undegraded":"2024-09-11T07:52:35.859930+0000","last_fullsized":"2024-09-11T07:52:35.859930+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:30.893701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:28:39.723598+0000","last_clean_scrub_stamp":"2024-09-11T07:51:30.893701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:45.207509+0000","scrub_duration":0.0016288920000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"87'81","reported_seq":140,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.629064+0000","last_change":"2024-09-11T07:51:51.734366+0000","last_active":"2024-09-11T07:52:36.629064+0000","last_peered":"2024-09-11T07:52:36.629064+0000","last_clean":"2024-09-11T07:52:36.629064+0000","last_became_active":"2024-09-11T07:51:48.207499+0000","last_became_peered":"2024-09-11T07:51:48.207499+0000","last_unstale":"2024-09-11T07:52:36.629064+0000","last_undegraded":"2024-09-11T07:52:36.629064+0000","last_fullsized":"2024-09-11T07:52:36.629064+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:51:51.734264+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:51:51.734264+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:00.126696+0000","scrub_duration":0.0019810140000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"426'30","reported_seq":131,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.860838+0000","last_change":"2024-09-11T07:52:31.868326+0000","last_active":"2024-09-11T07:52:35.860838+0000","last_peered":"2024-09-11T07:52:35.860838+0000","last_clean":"2024-09-11T07:52:35.860838+0000","last_became_active":"2024-09-11T07:51:34.322813+0000","last_became_peered":"2024-09-11T07:51:34.322813+0000","last_unstale":"2024-09-11T07:52:35.860838+0000","last_undegraded":"2024-09-11T07:52:35.860838+0000","last_fullsized":"2024-09-11T07:52:35.860838+0000","mapping_epoch":509,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":510,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'30","last_scrub_stamp":"2024-09-11T07:52:31.868262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:52:31.868262+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:36.068726+0000","scrub_duration":0.0023282620000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"57'1","reported_seq":488,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-11T07:52:43.874500+0000","last_change":"2024-09-11T07:52:43.874468+0000","last_active":"2024-09-11T07:52:43.874500+0000","last_peered":"2024-09-11T07:52:43.874500+0000","last_clean":"2024-09-11T07:52:43.874500+0000","last_became_active":"2024-09-11T07:51:14.063313+0000","last_became_peered":"2024-09-11T07:51:14.063313+0000","last_unstale":"2024-09-11T07:52:43.874500+0000","last_undegraded":"2024-09-11T07:52:43.874500+0000","last_fullsized":"2024-09-11T07:52:43.874500+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:52:43.874399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:52:43.874399+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:50.967073+0000","scrub_duration":0.001640842,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"510'10","reported_seq":941,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:39.825242+0000","last_change":"2024-09-11T07:51:39.648636+0000","last_active":"2024-09-11T07:52:39.825242+0000","last_peered":"2024-09-11T07:52:39.825242+0000","last_clean":"2024-09-11T07:52:39.825242+0000","last_became_active":"2024-09-11T07:51:34.323354+0000","last_became_peered":"2024-09-11T07:51:34.323354+0000","last_unstale":"2024-09-11T07:52:39.825242+0000","last_undegraded":"2024-09-11T07:52:39.825242+0000","last_fullsized":"2024-09-11T07:52:39.825242+0000","mapping_epoch":509,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":510,"parent":"0.0","parent_split_bits":0,"last_scrub":"510'10","last_scrub_stamp":"2024-09-11T07:51:39.648577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:51:39.648577+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:46.481278+0000","scrub_duration":0.0018664269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":619,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.860482+0000","last_change":"2024-09-11T07:52:02.771507+0000","last_active":"2024-09-11T07:52:35.860482+0000","last_peered":"2024-09-11T07:52:35.860482+0000","last_clean":"2024-09-11T07:52:35.860482+0000","last_became_active":"2024-09-11T07:50:53.914995+0000","last_became_peered":"2024-09-11T07:50:53.914995+0000","last_unstale":"2024-09-11T07:52:35.860482+0000","last_undegraded":"2024-09-11T07:52:35.860482+0000","last_fullsized":"2024-09-11T07:52:35.860482+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:02.771444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:50:29.549901+0000","last_clean_scrub_stamp":"2024-09-11T07:52:02.771444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:07.848581+0000","scrub_duration":0.001304195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"87'81","reported_seq":314,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642264+0000","last_change":"2024-09-11T07:51:47.611204+0000","last_active":"2024-09-11T07:52:36.642264+0000","last_peered":"2024-09-11T07:52:36.642264+0000","last_clean":"2024-09-11T07:52:36.642264+0000","last_became_active":"2024-09-11T07:51:37.856067+0000","last_became_peered":"2024-09-11T07:51:37.856067+0000","last_unstale":"2024-09-11T07:52:36.642264+0000","last_undegraded":"2024-09-11T07:52:36.642264+0000","last_fullsized":"2024-09-11T07:52:36.642264+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:51:47.611126+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:51:47.611126+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:06.463793+0000","scrub_duration":0.0016262830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"522'68","reported_seq":125,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:39.825685+0000","last_change":"2024-09-11T07:52:07.750614+0000","last_active":"2024-09-11T07:52:39.825685+0000","last_peered":"2024-09-11T07:52:39.825685+0000","last_clean":"2024-09-11T07:52:39.825685+0000","last_became_active":"2024-09-11T07:51:14.061014+0000","last_became_peered":"2024-09-11T07:51:14.061014+0000","last_unstale":"2024-09-11T07:52:39.825685+0000","last_undegraded":"2024-09-11T07:52:39.825685+0000","last_fullsized":"2024-09-11T07:52:39.825685+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"522'68","last_scrub_stamp":"2024-09-11T07:52:07.750543+0000","last_deep_scrub":"522'68","last_deep_scrub_stamp":"2024-09-11T07:52:07.750543+0000","last_clean_scrub_stamp":"2024-09-11T07:52:07.750543+0000","objects_scrubbed":10,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:29.280111+0000","scrub_duration":0.0082493559999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":82,"num_read_kb":67,"num_write":50,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":583,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642378+0000","last_change":"2024-09-11T07:51:45.558592+0000","last_active":"2024-09-11T07:52:36.642378+0000","last_peered":"2024-09-11T07:52:36.642378+0000","last_clean":"2024-09-11T07:52:36.642378+0000","last_became_active":"2024-09-11T07:51:37.864361+0000","last_became_peered":"2024-09-11T07:51:37.864361+0000","last_unstale":"2024-09-11T07:52:36.642378+0000","last_undegraded":"2024-09-11T07:52:36.642378+0000","last_fullsized":"2024-09-11T07:52:36.642378+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:45.558527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:51:45.558527+0000","last_clean_scrub_stamp":"2024-09-11T07:51:45.558527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:08.808308+0000","scrub_duration":0.00170009,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":611,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:38.818431+0000","last_change":"2024-09-11T07:52:38.818396+0000","last_active":"2024-09-11T07:52:38.818431+0000","last_peered":"2024-09-11T07:52:38.818431+0000","last_clean":"2024-09-11T07:52:38.818431+0000","last_became_active":"2024-09-11T07:50:50.861632+0000","last_became_peered":"2024-09-11T07:50:50.861632+0000","last_unstale":"2024-09-11T07:52:38.818431+0000","last_undegraded":"2024-09-11T07:52:38.818431+0000","last_fullsized":"2024-09-11T07:52:38.818431+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:38.818340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:52:38.818340+0000","last_clean_scrub_stamp":"2024-09-11T07:52:38.818340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:45.067717+0000","scrub_duration":0.0013967789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":636,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.545103+0000","last_change":"2024-09-11T07:51:19.302258+0000","last_active":"2024-09-11T07:52:36.545103+0000","last_peered":"2024-09-11T07:52:36.545103+0000","last_clean":"2024-09-11T07:52:36.545103+0000","last_became_active":"2024-09-11T07:50:56.906352+0000","last_became_peered":"2024-09-11T07:50:56.906352+0000","last_unstale":"2024-09-11T07:52:36.545103+0000","last_undegraded":"2024-09-11T07:52:36.545103+0000","last_fullsized":"2024-09-11T07:52:36.545103+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:19.302192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:51:19.302192+0000","last_clean_scrub_stamp":"2024-09-11T07:51:19.302192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001603826,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"87'81","reported_seq":37,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.598461+0000","last_change":"2024-09-11T07:52:00.554953+0000","last_active":"2024-09-11T07:52:36.598461+0000","last_peered":"2024-09-11T07:52:36.598461+0000","last_clean":"2024-09-11T07:52:36.598461+0000","last_became_active":"2024-09-11T07:51:48.197429+0000","last_became_peered":"2024-09-11T07:51:48.197429+0000","last_unstale":"2024-09-11T07:52:36.598461+0000","last_undegraded":"2024-09-11T07:52:36.598461+0000","last_fullsized":"2024-09-11T07:52:36.598461+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":4,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:52:00.554885+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:52:00.554885+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:02.952594+0000","scrub_duration":0.0016275630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"426'26","reported_seq":456,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.862201+0000","last_change":"2024-09-11T07:52:24.781578+0000","last_active":"2024-09-11T07:52:35.862201+0000","last_peered":"2024-09-11T07:52:35.862201+0000","last_clean":"2024-09-11T07:52:35.862201+0000","last_became_active":"2024-09-11T07:51:14.064155+0000","last_became_peered":"2024-09-11T07:51:14.064155+0000","last_unstale":"2024-09-11T07:52:35.862201+0000","last_undegraded":"2024-09-11T07:52:35.862201+0000","last_fullsized":"2024-09-11T07:52:35.862201+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'26","last_scrub_stamp":"2024-09-11T07:52:24.781521+0000","last_deep_scrub":"144'18","last_deep_scrub_stamp":"2024-09-11T07:31:51.603378+0000","last_clean_scrub_stamp":"2024-09-11T07:52:24.781521+0000","objects_scrubbed":5,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:38.502079+0000","scrub_duration":0.0021468749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":41,"num_read_kb":30,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":1621,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-11T07:52:43.213583+0000","last_change":"2024-09-11T07:52:27.900661+0000","last_active":"2024-09-11T07:52:43.213583+0000","last_peered":"2024-09-11T07:52:43.213583+0000","last_clean":"2024-09-11T07:52:43.213583+0000","last_became_active":"2024-09-11T07:51:14.056679+0000","last_became_peered":"2024-09-11T07:51:14.056679+0000","last_unstale":"2024-09-11T07:52:43.213583+0000","last_undegraded":"2024-09-11T07:52:43.213583+0000","last_fullsized":"2024-09-11T07:52:43.213583+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'5","last_scrub_stamp":"2024-09-11T07:52:27.900603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:52:27.900603+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:42.347823+0000","scrub_duration":0.0017524960000000001,"objects_trimmed":0,"snaptrim_duration":0.00083522999999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1148,"num_read_kb":1143,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":71,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-11T07:52:45.341193+0000","last_change":"2024-09-11T07:52:45.341162+0000","last_active":"2024-09-11T07:52:45.341193+0000","last_peered":"2024-09-11T07:52:45.341193+0000","last_clean":"2024-09-11T07:52:45.341193+0000","last_became_active":"2024-09-11T07:50:53.928925+0000","last_became_peered":"2024-09-11T07:50:53.928925+0000","last_unstale":"2024-09-11T07:52:45.341193+0000","last_undegraded":"2024-09-11T07:52:45.341193+0000","last_fullsized":"2024-09-11T07:52:45.341193+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:45.341104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:34:07.149367+0000","last_clean_scrub_stamp":"2024-09-11T07:52:45.341104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:00.779598+0000","scrub_duration":0.00127366,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":81,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642555+0000","last_change":"2024-09-11T07:52:08.118907+0000","last_active":"2024-09-11T07:52:36.642555+0000","last_peered":"2024-09-11T07:52:36.642555+0000","last_clean":"2024-09-11T07:52:36.642555+0000","last_became_active":"2024-09-11T07:50:56.914441+0000","last_became_peered":"2024-09-11T07:50:56.914441+0000","last_unstale":"2024-09-11T07:52:36.642555+0000","last_undegraded":"2024-09-11T07:52:36.642555+0000","last_fullsized":"2024-09-11T07:52:36.642555+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:08.118841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:52:08.118841+0000","last_clean_scrub_stamp":"2024-09-11T07:52:08.118841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:31.008978+0000","scrub_duration":0.0021706490000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":69,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.599380+0000","last_change":"2024-09-11T07:52:31.656911+0000","last_active":"2024-09-11T07:52:36.599380+0000","last_peered":"2024-09-11T07:52:36.599380+0000","last_clean":"2024-09-11T07:52:36.599380+0000","last_became_active":"2024-09-11T07:50:56.908047+0000","last_became_peered":"2024-09-11T07:50:56.908047+0000","last_unstale":"2024-09-11T07:52:36.599380+0000","last_undegraded":"2024-09-11T07:52:36.599380+0000","last_fullsized":"2024-09-11T07:52:36.599380+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:31.656792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:25.382533+0000","last_clean_scrub_stamp":"2024-09-11T07:52:31.656792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:51.434152+0000","scrub_duration":0.0017941669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"87'81","reported_seq":166,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.860534+0000","last_change":"2024-09-11T07:51:57.833896+0000","last_active":"2024-09-11T07:52:35.860534+0000","last_peered":"2024-09-11T07:52:35.860534+0000","last_clean":"2024-09-11T07:52:35.860534+0000","last_became_active":"2024-09-11T07:51:14.059851+0000","last_became_peered":"2024-09-11T07:51:14.059851+0000","last_unstale":"2024-09-11T07:52:35.860534+0000","last_undegraded":"2024-09-11T07:52:35.860534+0000","last_fullsized":"2024-09-11T07:52:35.860534+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":388,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":3,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:51:57.833831+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:51:57.833831+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:13.353364+0000","scrub_duration":0.0018593209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"426'42","reported_seq":807,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861442+0000","last_change":"2024-09-11T07:52:22.817089+0000","last_active":"2024-09-11T07:52:35.861442+0000","last_peered":"2024-09-11T07:52:35.861442+0000","last_clean":"2024-09-11T07:52:35.861442+0000","last_became_active":"2024-09-11T07:50:53.923727+0000","last_became_peered":"2024-09-11T07:50:53.923727+0000","last_unstale":"2024-09-11T07:52:35.861442+0000","last_undegraded":"2024-09-11T07:52:35.861442+0000","last_fullsized":"2024-09-11T07:52:35.861442+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'42","last_scrub_stamp":"2024-09-11T07:52:22.817028+0000","last_deep_scrub":"426'42","last_deep_scrub_stamp":"2024-09-11T07:52:22.817028+0000","last_clean_scrub_stamp":"2024-09-11T07:52:22.817028+0000","objects_scrubbed":6,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:47.108592+0000","scrub_duration":0.00193462,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":54,"num_read_kb":48,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"422'2","reported_seq":672,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.860657+0000","last_change":"2024-09-11T07:51:58.665327+0000","last_active":"2024-09-11T07:52:35.860657+0000","last_peered":"2024-09-11T07:52:35.860657+0000","last_clean":"2024-09-11T07:52:35.860657+0000","last_became_active":"2024-09-11T07:51:19.098192+0000","last_became_peered":"2024-09-11T07:51:19.098192+0000","last_unstale":"2024-09-11T07:52:35.860657+0000","last_undegraded":"2024-09-11T07:52:35.860657+0000","last_fullsized":"2024-09-11T07:52:35.860657+0000","mapping_epoch":504,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":505,"parent":"0.0","parent_split_bits":0,"last_scrub":"422'2","last_scrub_stamp":"2024-09-11T07:51:58.665247+0000","last_deep_scrub":"422'2","last_deep_scrub_stamp":"2024-09-11T07:51:58.665247+0000","last_clean_scrub_stamp":"2024-09-11T07:51:58.665247+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:25.958559+0000","scrub_duration":0.0032743450000000001,"objects_trimmed":0,"snaptrim_duration":0.001077742,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":69,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.598946+0000","last_change":"2024-09-11T07:51:31.522195+0000","last_active":"2024-09-11T07:52:36.598946+0000","last_peered":"2024-09-11T07:52:36.598946+0000","last_clean":"2024-09-11T07:52:36.598946+0000","last_became_active":"2024-09-11T07:51:09.045724+0000","last_became_peered":"2024-09-11T07:51:09.045724+0000","last_unstale":"2024-09-11T07:52:36.598946+0000","last_undegraded":"2024-09-11T07:52:36.598946+0000","last_fullsized":"2024-09-11T07:52:36.598946+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:31.522132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:51:31.522132+0000","last_clean_scrub_stamp":"2024-09-11T07:51:31.522132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:57.981776+0000","scrub_duration":0.0016075250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":175,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642955+0000","last_change":"2024-09-11T07:51:42.567022+0000","last_active":"2024-09-11T07:52:36.642955+0000","last_peered":"2024-09-11T07:52:36.642955+0000","last_clean":"2024-09-11T07:52:36.642955+0000","last_became_active":"2024-09-11T07:51:37.859118+0000","last_became_peered":"2024-09-11T07:51:37.859118+0000","last_unstale":"2024-09-11T07:52:36.642955+0000","last_undegraded":"2024-09-11T07:52:36.642955+0000","last_fullsized":"2024-09-11T07:52:36.642955+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:42.566952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:51.187127+0000","last_clean_scrub_stamp":"2024-09-11T07:51:42.566952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:47.068464+0000","scrub_duration":0.001607027,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":157,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861374+0000","last_change":"2024-09-11T07:52:06.786372+0000","last_active":"2024-09-11T07:52:35.861374+0000","last_peered":"2024-09-11T07:52:35.861374+0000","last_clean":"2024-09-11T07:52:35.861374+0000","last_became_active":"2024-09-11T07:51:14.066002+0000","last_became_peered":"2024-09-11T07:51:14.066002+0000","last_unstale":"2024-09-11T07:52:35.861374+0000","last_undegraded":"2024-09-11T07:52:35.861374+0000","last_fullsized":"2024-09-11T07:52:35.861374+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:06.786303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:52:06.786303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:33.770498+0000","scrub_duration":0.001625041,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"426'45","reported_seq":207,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.628666+0000","last_change":"2024-09-11T07:52:27.809409+0000","last_active":"2024-09-11T07:52:36.628666+0000","last_peered":"2024-09-11T07:52:36.628666+0000","last_clean":"2024-09-11T07:52:36.628666+0000","last_became_active":"2024-09-11T07:51:45.190817+0000","last_became_peered":"2024-09-11T07:51:45.190817+0000","last_unstale":"2024-09-11T07:52:36.628666+0000","last_undegraded":"2024-09-11T07:52:36.628666+0000","last_fullsized":"2024-09-11T07:52:36.628666+0000","mapping_epoch":514,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":515,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'45","last_scrub_stamp":"2024-09-11T07:52:27.809340+0000","last_deep_scrub":"399'33","last_deep_scrub_stamp":"2024-09-11T07:45:32.442160+0000","last_clean_scrub_stamp":"2024-09-11T07:52:27.809340+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:52.210963+0000","scrub_duration":0.0021207800000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":58,"num_read_kb":48,"num_write":37,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'3","reported_seq":659,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.544675+0000","last_change":"2024-09-11T07:52:27.449877+0000","last_active":"2024-09-11T07:52:36.544675+0000","last_peered":"2024-09-11T07:52:36.544675+0000","last_clean":"2024-09-11T07:52:36.544675+0000","last_became_active":"2024-09-11T07:51:34.321017+0000","last_became_peered":"2024-09-11T07:51:34.321017+0000","last_unstale":"2024-09-11T07:52:36.544675+0000","last_undegraded":"2024-09-11T07:52:36.544675+0000","last_fullsized":"2024-09-11T07:52:36.544675+0000","mapping_epoch":509,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":510,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-09-11T07:52:27.449814+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-09-11T07:51:03.338306+0000","last_clean_scrub_stamp":"2024-09-11T07:52:27.449814+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:37.698089+0000","scrub_duration":0.0020273040000000002,"objects_trimmed":0,"snaptrim_duration":0.003800767,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":528,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642652+0000","last_change":"2024-09-11T07:51:48.206709+0000","last_active":"2024-09-11T07:52:36.642652+0000","last_peered":"2024-09-11T07:52:36.642652+0000","last_clean":"2024-09-11T07:52:36.642652+0000","last_became_active":"2024-09-11T07:51:48.206358+0000","last_became_peered":"2024-09-11T07:51:48.206358+0000","last_unstale":"2024-09-11T07:52:36.642652+0000","last_undegraded":"2024-09-11T07:52:36.642652+0000","last_fullsized":"2024-09-11T07:52:36.642652+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:19.076548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:51:19.076548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:46.529907+0000","scrub_duration":0.0073747420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"87'81","reported_seq":37,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.598447+0000","last_change":"2024-09-11T07:51:54.557199+0000","last_active":"2024-09-11T07:52:36.598447+0000","last_peered":"2024-09-11T07:52:36.598447+0000","last_clean":"2024-09-11T07:52:36.598447+0000","last_became_active":"2024-09-11T07:51:48.199620+0000","last_became_peered":"2024-09-11T07:51:48.199620+0000","last_unstale":"2024-09-11T07:52:36.598447+0000","last_undegraded":"2024-09-11T07:52:36.598447+0000","last_fullsized":"2024-09-11T07:52:36.598447+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'81","last_scrub_stamp":"2024-09-11T07:51:54.557128+0000","last_deep_scrub":"87'81","last_deep_scrub_stamp":"2024-09-11T07:33:10.189376+0000","last_clean_scrub_stamp":"2024-09-11T07:51:54.557128+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:20.283442+0000","scrub_duration":0.0019394849999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":27,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.599175+0000","last_change":"2024-09-11T07:51:48.202466+0000","last_active":"2024-09-11T07:52:36.599175+0000","last_peered":"2024-09-11T07:52:36.599175+0000","last_clean":"2024-09-11T07:52:36.599175+0000","last_became_active":"2024-09-11T07:51:48.202154+0000","last_became_peered":"2024-09-11T07:51:48.202154+0000","last_unstale":"2024-09-11T07:52:36.599175+0000","last_undegraded":"2024-09-11T07:52:36.599175+0000","last_fullsized":"2024-09-11T07:52:36.599175+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:37.210850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:51:37.210850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:57.124781+0000","scrub_duration":0.001335276,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":568,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.643055+0000","last_change":"2024-09-11T07:52:27.719342+0000","last_active":"2024-09-11T07:52:36.643055+0000","last_peered":"2024-09-11T07:52:36.643055+0000","last_clean":"2024-09-11T07:52:36.643055+0000","last_became_active":"2024-09-11T07:51:37.859778+0000","last_became_peered":"2024-09-11T07:51:37.859778+0000","last_unstale":"2024-09-11T07:52:36.643055+0000","last_undegraded":"2024-09-11T07:52:36.643055+0000","last_fullsized":"2024-09-11T07:52:36.643055+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:27.719279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:52:27.719279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:29.032317+0000","scrub_duration":0.0017284430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"426'51","reported_seq":300,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.628416+0000","last_change":"2024-09-11T07:51:48.225263+0000","last_active":"2024-09-11T07:52:36.628416+0000","last_peered":"2024-09-11T07:52:36.628416+0000","last_clean":"2024-09-11T07:52:36.628416+0000","last_became_active":"2024-09-11T07:51:48.206795+0000","last_became_peered":"2024-09-11T07:51:48.206795+0000","last_unstale":"2024-09-11T07:52:36.628416+0000","last_undegraded":"2024-09-11T07:52:36.628416+0000","last_fullsized":"2024-09-11T07:52:36.628416+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'51","last_scrub_stamp":"2024-09-11T07:51:43.167159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:51:43.167159+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:46.886452+0000","scrub_duration":0.0022568250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":78,"num_read_kb":67,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,7],"acting":[6,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":61,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642431+0000","last_change":"2024-09-11T07:51:23.166604+0000","last_active":"2024-09-11T07:52:36.642431+0000","last_peered":"2024-09-11T07:52:36.642431+0000","last_clean":"2024-09-11T07:52:36.642431+0000","last_became_active":"2024-09-11T07:50:53.929557+0000","last_became_peered":"2024-09-11T07:50:53.929557+0000","last_unstale":"2024-09-11T07:52:36.642431+0000","last_undegraded":"2024-09-11T07:52:36.642431+0000","last_fullsized":"2024-09-11T07:52:36.642431+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:23.166543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:51:23.166543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:50.976076+0000","scrub_duration":0.0019013649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":91,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642153+0000","last_change":"2024-09-11T07:52:07.082580+0000","last_active":"2024-09-11T07:52:36.642153+0000","last_peered":"2024-09-11T07:52:36.642153+0000","last_clean":"2024-09-11T07:52:36.642153+0000","last_became_active":"2024-09-11T07:51:34.324125+0000","last_became_peered":"2024-09-11T07:51:34.324125+0000","last_unstale":"2024-09-11T07:52:36.642153+0000","last_undegraded":"2024-09-11T07:52:36.642153+0000","last_fullsized":"2024-09-11T07:52:36.642153+0000","mapping_epoch":509,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":510,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:07.082507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:51:00.116883+0000","last_clean_scrub_stamp":"2024-09-11T07:52:07.082507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:30.795023+0000","scrub_duration":0.0019556529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":37,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.598656+0000","last_change":"2024-09-11T07:52:10.480698+0000","last_active":"2024-09-11T07:52:36.598656+0000","last_peered":"2024-09-11T07:52:36.598656+0000","last_clean":"2024-09-11T07:52:36.598656+0000","last_became_active":"2024-09-11T07:51:48.202773+0000","last_became_peered":"2024-09-11T07:51:48.202773+0000","last_unstale":"2024-09-11T07:52:36.598656+0000","last_undegraded":"2024-09-11T07:52:36.598656+0000","last_fullsized":"2024-09-11T07:52:36.598656+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:10.480638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:52:10.480638+0000","last_clean_scrub_stamp":"2024-09-11T07:52:10.480638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:38.526861+0000","scrub_duration":0.0031260929999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"426'21","reported_seq":657,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.860583+0000","last_change":"2024-09-11T07:51:27.765216+0000","last_active":"2024-09-11T07:52:35.860583+0000","last_peered":"2024-09-11T07:52:35.860583+0000","last_clean":"2024-09-11T07:52:35.860583+0000","last_became_active":"2024-09-11T07:50:50.861614+0000","last_became_peered":"2024-09-11T07:50:50.861614+0000","last_unstale":"2024-09-11T07:52:35.860583+0000","last_undegraded":"2024-09-11T07:52:35.860583+0000","last_fullsized":"2024-09-11T07:52:35.860583+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'21","last_scrub_stamp":"2024-09-11T07:51:27.765140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:51:27.765140+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:47.142359+0000","scrub_duration":0.002239245,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":461,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.862092+0000","last_change":"2024-09-11T07:51:58.882854+0000","last_active":"2024-09-11T07:52:35.862092+0000","last_peered":"2024-09-11T07:52:35.862092+0000","last_clean":"2024-09-11T07:52:35.862092+0000","last_became_active":"2024-09-11T07:51:14.060150+0000","last_became_peered":"2024-09-11T07:51:14.060150+0000","last_unstale":"2024-09-11T07:52:35.862092+0000","last_undegraded":"2024-09-11T07:52:35.862092+0000","last_fullsized":"2024-09-11T07:52:35.862092+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:58.882784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:51:58.882784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:05.245534+0000","scrub_duration":0.0016724590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":71,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642193+0000","last_change":"2024-09-11T07:52:28.328123+0000","last_active":"2024-09-11T07:52:36.642193+0000","last_peered":"2024-09-11T07:52:36.642193+0000","last_clean":"2024-09-11T07:52:36.642193+0000","last_became_active":"2024-09-11T07:50:53.919253+0000","last_became_peered":"2024-09-11T07:50:53.919253+0000","last_unstale":"2024-09-11T07:52:36.642193+0000","last_undegraded":"2024-09-11T07:52:36.642193+0000","last_fullsized":"2024-09-11T07:52:36.642193+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:28.328056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:52:28.328056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:58.262347+0000","scrub_duration":0.001878443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":636,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.860228+0000","last_change":"2024-09-11T07:51:29.724652+0000","last_active":"2024-09-11T07:52:35.860228+0000","last_peered":"2024-09-11T07:52:35.860228+0000","last_clean":"2024-09-11T07:52:35.860228+0000","last_became_active":"2024-09-11T07:50:56.913071+0000","last_became_peered":"2024-09-11T07:50:56.913071+0000","last_unstale":"2024-09-11T07:52:35.860228+0000","last_undegraded":"2024-09-11T07:52:35.860228+0000","last_fullsized":"2024-09-11T07:52:35.860228+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:29.724592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:51:29.724592+0000","last_clean_scrub_stamp":"2024-09-11T07:51:29.724592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020246410000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"426'46","reported_seq":118,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642190+0000","last_change":"2024-09-11T07:51:48.220627+0000","last_active":"2024-09-11T07:52:36.642190+0000","last_peered":"2024-09-11T07:52:36.642190+0000","last_clean":"2024-09-11T07:52:36.642190+0000","last_became_active":"2024-09-11T07:51:48.206628+0000","last_became_peered":"2024-09-11T07:51:48.206628+0000","last_unstale":"2024-09-11T07:52:36.642190+0000","last_undegraded":"2024-09-11T07:52:36.642190+0000","last_fullsized":"2024-09-11T07:52:36.642190+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'46","last_scrub_stamp":"2024-09-11T07:51:46.603659+0000","last_deep_scrub":"421'34","last_deep_scrub_stamp":"2024-09-11T07:46:14.474174+0000","last_clean_scrub_stamp":"2024-09-11T07:51:46.603659+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:48.799771+0000","scrub_duration":0.0020717209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":91,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642498+0000","last_change":"2024-09-11T07:52:31.370748+0000","last_active":"2024-09-11T07:52:36.642498+0000","last_peered":"2024-09-11T07:52:36.642498+0000","last_clean":"2024-09-11T07:52:36.642498+0000","last_became_active":"2024-09-11T07:51:48.201810+0000","last_became_peered":"2024-09-11T07:51:48.201810+0000","last_unstale":"2024-09-11T07:52:36.642498+0000","last_undegraded":"2024-09-11T07:52:36.642498+0000","last_fullsized":"2024-09-11T07:52:36.642498+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:31.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:51:07.108424+0000","last_clean_scrub_stamp":"2024-09-11T07:52:31.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:53.993217+0000","scrub_duration":0.0044957570000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"496'16","reported_seq":94,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:39.825273+0000","last_change":"2024-09-11T07:52:25.691754+0000","last_active":"2024-09-11T07:52:39.825273+0000","last_peered":"2024-09-11T07:52:39.825273+0000","last_clean":"2024-09-11T07:52:39.825273+0000","last_became_active":"2024-09-11T07:50:56.909842+0000","last_became_peered":"2024-09-11T07:50:56.909842+0000","last_unstale":"2024-09-11T07:52:39.825273+0000","last_undegraded":"2024-09-11T07:52:39.825273+0000","last_fullsized":"2024-09-11T07:52:39.825273+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"496'16","last_scrub_stamp":"2024-09-11T07:52:25.691683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:52:25.691683+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:53.473430+0000","scrub_duration":0.001732578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":387,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642654+0000","last_change":"2024-09-11T07:51:37.865997+0000","last_active":"2024-09-11T07:52:36.642654+0000","last_peered":"2024-09-11T07:52:36.642654+0000","last_clean":"2024-09-11T07:52:36.642654+0000","last_became_active":"2024-09-11T07:51:37.865205+0000","last_became_peered":"2024-09-11T07:51:37.865205+0000","last_unstale":"2024-09-11T07:52:36.642654+0000","last_undegraded":"2024-09-11T07:52:36.642654+0000","last_fullsized":"2024-09-11T07:52:36.642654+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:36.572101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:48:43.848057+0000","last_clean_scrub_stamp":"2024-09-11T07:51:36.572101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:48.841195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"426'29","reported_seq":98,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861161+0000","last_change":"2024-09-11T07:51:57.164629+0000","last_active":"2024-09-11T07:52:35.861161+0000","last_peered":"2024-09-11T07:52:35.861161+0000","last_clean":"2024-09-11T07:52:35.861161+0000","last_became_active":"2024-09-11T07:51:19.095126+0000","last_became_peered":"2024-09-11T07:51:19.095126+0000","last_unstale":"2024-09-11T07:52:35.861161+0000","last_undegraded":"2024-09-11T07:52:35.861161+0000","last_fullsized":"2024-09-11T07:52:35.861161+0000","mapping_epoch":504,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":505,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'29","last_scrub_stamp":"2024-09-11T07:51:57.164526+0000","last_deep_scrub":"234'23","last_deep_scrub_stamp":"2024-09-11T07:37:34.613280+0000","last_clean_scrub_stamp":"2024-09-11T07:51:57.164526+0000","objects_scrubbed":5,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:05.705415+0000","scrub_duration":0.0022382869999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":96,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.862138+0000","last_change":"2024-09-11T07:52:25.270117+0000","last_active":"2024-09-11T07:52:35.862138+0000","last_peered":"2024-09-11T07:52:35.862138+0000","last_clean":"2024-09-11T07:52:35.862138+0000","last_became_active":"2024-09-11T07:50:56.913432+0000","last_became_peered":"2024-09-11T07:50:56.913432+0000","last_unstale":"2024-09-11T07:52:35.862138+0000","last_undegraded":"2024-09-11T07:52:35.862138+0000","last_fullsized":"2024-09-11T07:52:35.862138+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-09-11T07:52:25.270048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:52:25.270048+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:25.575579+0000","scrub_duration":0.0020442350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":71,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.641907+0000","last_change":"2024-09-11T07:52:22.354266+0000","last_active":"2024-09-11T07:52:36.641907+0000","last_peered":"2024-09-11T07:52:36.641907+0000","last_clean":"2024-09-11T07:52:36.641907+0000","last_became_active":"2024-09-11T07:50:53.912926+0000","last_became_peered":"2024-09-11T07:50:53.912926+0000","last_unstale":"2024-09-11T07:52:36.641907+0000","last_undegraded":"2024-09-11T07:52:36.641907+0000","last_fullsized":"2024-09-11T07:52:36.641907+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:22.354174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:48:30.290136+0000","last_clean_scrub_stamp":"2024-09-11T07:52:22.354174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:28.702842+0000","scrub_duration":0.002140145,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":626,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.544019+0000","last_change":"2024-09-11T07:52:24.474104+0000","last_active":"2024-09-11T07:52:36.544019+0000","last_peered":"2024-09-11T07:52:36.544019+0000","last_clean":"2024-09-11T07:52:36.544019+0000","last_became_active":"2024-09-11T07:51:48.202842+0000","last_became_peered":"2024-09-11T07:51:48.202842+0000","last_unstale":"2024-09-11T07:52:36.544019+0000","last_undegraded":"2024-09-11T07:52:36.544019+0000","last_fullsized":"2024-09-11T07:52:36.544019+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:24.474046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:44:27.626761+0000","last_clean_scrub_stamp":"2024-09-11T07:52:24.474046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:49.199583+0000","scrub_duration":0.0017009080000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"426'25","reported_seq":738,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861519+0000","last_change":"2024-09-11T07:51:48.210843+0000","last_active":"2024-09-11T07:52:35.861519+0000","last_peered":"2024-09-11T07:52:35.861519+0000","last_clean":"2024-09-11T07:52:35.861519+0000","last_became_active":"2024-09-11T07:51:48.197552+0000","last_became_peered":"2024-09-11T07:51:48.197552+0000","last_unstale":"2024-09-11T07:52:35.861519+0000","last_undegraded":"2024-09-11T07:52:35.861519+0000","last_fullsized":"2024-09-11T07:52:35.861519+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'25","last_scrub_stamp":"2024-09-11T07:51:11.679256+0000","last_deep_scrub":"426'25","last_deep_scrub_stamp":"2024-09-11T07:47:22.197145+0000","last_clean_scrub_stamp":"2024-09-11T07:51:11.679256+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020626199999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":61,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.641984+0000","last_change":"2024-09-11T07:51:56.057998+0000","last_active":"2024-09-11T07:52:36.641984+0000","last_peered":"2024-09-11T07:52:36.641984+0000","last_clean":"2024-09-11T07:52:36.641984+0000","last_became_active":"2024-09-11T07:50:53.913208+0000","last_became_peered":"2024-09-11T07:50:53.913208+0000","last_unstale":"2024-09-11T07:52:36.641984+0000","last_undegraded":"2024-09-11T07:52:36.641984+0000","last_fullsized":"2024-09-11T07:52:36.641984+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:56.057935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:43:26.338227+0000","last_clean_scrub_stamp":"2024-09-11T07:51:56.057935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:10.881674+0000","scrub_duration":0.001460434,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":49,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-11T07:52:41.457093+0000","last_change":"2024-09-11T07:52:41.457063+0000","last_active":"2024-09-11T07:52:41.457093+0000","last_peered":"2024-09-11T07:52:41.457093+0000","last_clean":"2024-09-11T07:52:41.457093+0000","last_became_active":"2024-09-11T07:51:19.098400+0000","last_became_peered":"2024-09-11T07:51:19.098400+0000","last_unstale":"2024-09-11T07:52:41.457093+0000","last_undegraded":"2024-09-11T07:52:41.457093+0000","last_fullsized":"2024-09-11T07:52:41.457093+0000","mapping_epoch":504,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":505,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:41.457007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:52:41.457007+0000","last_clean_scrub_stamp":"2024-09-11T07:52:41.457007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:59.092058+0000","scrub_duration":0.0020169340000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"64'1","reported_seq":79,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642226+0000","last_change":"2024-09-11T07:51:20.118837+0000","last_active":"2024-09-11T07:52:36.642226+0000","last_peered":"2024-09-11T07:52:36.642226+0000","last_clean":"2024-09-11T07:52:36.642226+0000","last_became_active":"2024-09-11T07:50:56.915847+0000","last_became_peered":"2024-09-11T07:50:56.915847+0000","last_unstale":"2024-09-11T07:52:36.642226+0000","last_undegraded":"2024-09-11T07:52:36.642226+0000","last_fullsized":"2024-09-11T07:52:36.642226+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-11T07:51:20.118777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:51:20.118777+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:43.943149+0000","scrub_duration":0.002056127,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":39,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":411,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.628176+0000","last_change":"2024-09-11T07:51:48.704390+0000","last_active":"2024-09-11T07:52:36.628176+0000","last_peered":"2024-09-11T07:52:36.628176+0000","last_clean":"2024-09-11T07:52:36.628176+0000","last_became_active":"2024-09-11T07:51:45.420950+0000","last_became_peered":"2024-09-11T07:51:45.420950+0000","last_unstale":"2024-09-11T07:52:36.628176+0000","last_undegraded":"2024-09-11T07:52:36.628176+0000","last_fullsized":"2024-09-11T07:52:36.628176+0000","mapping_epoch":514,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":515,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:48.704324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:51:48.704324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:17.400227+0000","scrub_duration":0.001362526,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"426'21","reported_seq":75,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.543982+0000","last_change":"2024-09-11T07:51:48.212742+0000","last_active":"2024-09-11T07:52:36.543982+0000","last_peered":"2024-09-11T07:52:36.543982+0000","last_clean":"2024-09-11T07:52:36.543982+0000","last_became_active":"2024-09-11T07:51:48.199537+0000","last_became_peered":"2024-09-11T07:51:48.199537+0000","last_unstale":"2024-09-11T07:52:36.543982+0000","last_undegraded":"2024-09-11T07:52:36.543982+0000","last_fullsized":"2024-09-11T07:52:36.543982+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'21","last_scrub_stamp":"2024-09-11T07:51:26.373238+0000","last_deep_scrub":"82'9","last_deep_scrub_stamp":"2024-09-11T07:27:48.761397+0000","last_clean_scrub_stamp":"2024-09-11T07:51:26.373238+0000","objects_scrubbed":3,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:51.233199+0000","scrub_duration":0.001756051,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":25,"num_read_kb":22,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":514,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.643026+0000","last_change":"2024-09-11T07:51:37.864171+0000","last_active":"2024-09-11T07:52:36.643026+0000","last_peered":"2024-09-11T07:52:36.643026+0000","last_clean":"2024-09-11T07:52:36.643026+0000","last_became_active":"2024-09-11T07:51:37.863762+0000","last_became_peered":"2024-09-11T07:51:37.863762+0000","last_unstale":"2024-09-11T07:52:36.643026+0000","last_undegraded":"2024-09-11T07:52:36.643026+0000","last_fullsized":"2024-09-11T07:52:36.643026+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:34.438728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:42.917680+0000","last_clean_scrub_stamp":"2024-09-11T07:51:34.438728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:43.984479+0000","scrub_duration":0.0069828010000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":69,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:37.626181+0000","last_change":"2024-09-11T07:52:37.626132+0000","last_active":"2024-09-11T07:52:37.626181+0000","last_peered":"2024-09-11T07:52:37.626181+0000","last_clean":"2024-09-11T07:52:37.626181+0000","last_became_active":"2024-09-11T07:50:56.912212+0000","last_became_peered":"2024-09-11T07:50:56.912212+0000","last_unstale":"2024-09-11T07:52:37.626181+0000","last_undegraded":"2024-09-11T07:52:37.626181+0000","last_fullsized":"2024-09-11T07:52:37.626181+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:37.626067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:52:37.626067+0000","last_clean_scrub_stamp":"2024-09-11T07:52:37.626067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:44.277193+0000","scrub_duration":0.001561406,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"426'35","reported_seq":792,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.544968+0000","last_change":"2024-09-11T07:52:31.572517+0000","last_active":"2024-09-11T07:52:36.544968+0000","last_peered":"2024-09-11T07:52:36.544968+0000","last_clean":"2024-09-11T07:52:36.544968+0000","last_became_active":"2024-09-11T07:51:09.045315+0000","last_became_peered":"2024-09-11T07:51:09.045315+0000","last_unstale":"2024-09-11T07:52:36.544968+0000","last_undegraded":"2024-09-11T07:52:36.544968+0000","last_fullsized":"2024-09-11T07:52:36.544968+0000","mapping_epoch":499,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'35","last_scrub_stamp":"2024-09-11T07:52:31.572440+0000","last_deep_scrub":"234'25","last_deep_scrub_stamp":"2024-09-11T07:44:29.610392+0000","last_clean_scrub_stamp":"2024-09-11T07:52:31.572440+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:56.638332+0000","scrub_duration":0.002231402,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":538,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642520+0000","last_change":"2024-09-11T07:51:48.203783+0000","last_active":"2024-09-11T07:52:36.642520+0000","last_peered":"2024-09-11T07:52:36.642520+0000","last_clean":"2024-09-11T07:52:36.642520+0000","last_became_active":"2024-09-11T07:51:48.203478+0000","last_became_peered":"2024-09-11T07:51:48.203478+0000","last_unstale":"2024-09-11T07:52:36.642520+0000","last_undegraded":"2024-09-11T07:52:36.642520+0000","last_fullsized":"2024-09-11T07:52:36.642520+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:44.578359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:51:44.578359+0000","last_clean_scrub_stamp":"2024-09-11T07:51:44.578359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:04.793452+0000","scrub_duration":0.0018232980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"512'11","reported_seq":755,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:39.825272+0000","last_change":"2024-09-11T07:52:14.603918+0000","last_active":"2024-09-11T07:52:39.825272+0000","last_peered":"2024-09-11T07:52:39.825272+0000","last_clean":"2024-09-11T07:52:39.825272+0000","last_became_active":"2024-09-11T07:51:37.860270+0000","last_became_peered":"2024-09-11T07:51:37.860270+0000","last_unstale":"2024-09-11T07:52:39.825272+0000","last_undegraded":"2024-09-11T07:52:39.825272+0000","last_fullsized":"2024-09-11T07:52:39.825272+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":0,"last_scrub":"512'11","last_scrub_stamp":"2024-09-11T07:52:14.603845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:52:14.603845+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:26.791335+0000","scrub_duration":0.001773997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":656,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861710+0000","last_change":"2024-09-11T07:52:31.764190+0000","last_active":"2024-09-11T07:52:35.861710+0000","last_peered":"2024-09-11T07:52:35.861710+0000","last_clean":"2024-09-11T07:52:35.861710+0000","last_became_active":"2024-09-11T07:51:48.196907+0000","last_became_peered":"2024-09-11T07:51:48.196907+0000","last_unstale":"2024-09-11T07:52:35.861710+0000","last_undegraded":"2024-09-11T07:52:35.861710+0000","last_fullsized":"2024-09-11T07:52:35.861710+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:31.764120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:52:31.764120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:50.660752+0000","scrub_duration":0.001894632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"426'25","reported_seq":81,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.642745+0000","last_change":"2024-09-11T07:51:53.584531+0000","last_active":"2024-09-11T07:52:36.642745+0000","last_peered":"2024-09-11T07:52:36.642745+0000","last_clean":"2024-09-11T07:52:36.642745+0000","last_became_active":"2024-09-11T07:51:37.860023+0000","last_became_peered":"2024-09-11T07:51:37.860023+0000","last_unstale":"2024-09-11T07:52:36.642745+0000","last_undegraded":"2024-09-11T07:52:36.642745+0000","last_fullsized":"2024-09-11T07:52:36.642745+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'25","last_scrub_stamp":"2024-09-11T07:51:53.584457+0000","last_deep_scrub":"234'19","last_deep_scrub_stamp":"2024-09-11T07:37:41.338036+0000","last_clean_scrub_stamp":"2024-09-11T07:51:53.584457+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:04.168037+0000","scrub_duration":0.0022322000000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":618,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.860808+0000","last_change":"2024-09-11T07:52:28.806350+0000","last_active":"2024-09-11T07:52:35.860808+0000","last_peered":"2024-09-11T07:52:35.860808+0000","last_clean":"2024-09-11T07:52:35.860808+0000","last_became_active":"2024-09-11T07:50:50.863004+0000","last_became_peered":"2024-09-11T07:50:50.863004+0000","last_unstale":"2024-09-11T07:52:35.860808+0000","last_undegraded":"2024-09-11T07:52:35.860808+0000","last_fullsized":"2024-09-11T07:52:35.860808+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:28.806276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:52:28.806276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:44.445398+0000","scrub_duration":0.001691641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"505'17","reported_seq":71,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-11T07:52:41.255945+0000","last_change":"2024-09-11T07:52:22.442125+0000","last_active":"2024-09-11T07:52:41.255945+0000","last_peered":"2024-09-11T07:52:41.255945+0000","last_clean":"2024-09-11T07:52:41.255945+0000","last_became_active":"2024-09-11T07:51:19.099800+0000","last_became_peered":"2024-09-11T07:51:19.099800+0000","last_unstale":"2024-09-11T07:52:41.255945+0000","last_undegraded":"2024-09-11T07:52:41.255945+0000","last_fullsized":"2024-09-11T07:52:41.255945+0000","mapping_epoch":504,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":505,"parent":"0.0","parent_split_bits":0,"last_scrub":"505'17","last_scrub_stamp":"2024-09-11T07:52:22.442069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:52:22.442069+0000","objects_scrubbed":1,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:34.832077+0000","scrub_duration":0.0026387569999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":74,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.862015+0000","last_change":"2024-09-11T07:51:47.176810+0000","last_active":"2024-09-11T07:52:35.862015+0000","last_peered":"2024-09-11T07:52:35.862015+0000","last_clean":"2024-09-11T07:52:35.862015+0000","last_became_active":"2024-09-11T07:50:53.913522+0000","last_became_peered":"2024-09-11T07:50:53.913522+0000","last_unstale":"2024-09-11T07:52:35.862015+0000","last_undegraded":"2024-09-11T07:52:35.862015+0000","last_fullsized":"2024-09-11T07:52:35.862015+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:47.176746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:51:47.176746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:10.590927+0000","scrub_duration":0.001565575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"426'22","reported_seq":82,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861306+0000","last_change":"2024-09-11T07:51:38.914292+0000","last_active":"2024-09-11T07:52:35.861306+0000","last_peered":"2024-09-11T07:52:35.861306+0000","last_clean":"2024-09-11T07:52:35.861306+0000","last_became_active":"2024-09-11T07:51:14.059568+0000","last_became_peered":"2024-09-11T07:51:14.059568+0000","last_unstale":"2024-09-11T07:52:35.861306+0000","last_undegraded":"2024-09-11T07:52:35.861306+0000","last_fullsized":"2024-09-11T07:52:35.861306+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'22","last_scrub_stamp":"2024-09-11T07:51:38.914229+0000","last_deep_scrub":"236'16","last_deep_scrub_stamp":"2024-09-11T07:36:49.809484+0000","last_clean_scrub_stamp":"2024-09-11T07:51:38.914229+0000","objects_scrubbed":4,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:04.524583+0000","scrub_duration":0.0022210229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":440,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.628448+0000","last_change":"2024-09-11T07:52:24.847969+0000","last_active":"2024-09-11T07:52:36.628448+0000","last_peered":"2024-09-11T07:52:36.628448+0000","last_clean":"2024-09-11T07:52:36.628448+0000","last_became_active":"2024-09-11T07:51:45.190308+0000","last_became_peered":"2024-09-11T07:51:45.190308+0000","last_unstale":"2024-09-11T07:52:36.628448+0000","last_undegraded":"2024-09-11T07:52:36.628448+0000","last_fullsized":"2024-09-11T07:52:36.628448+0000","mapping_epoch":514,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":515,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:24.847906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:41:38.721118+0000","last_clean_scrub_stamp":"2024-09-11T07:52:24.847906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:48.339953+0000","scrub_duration":0.001516547,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":587,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.543891+0000","last_change":"2024-09-11T07:51:47.656510+0000","last_active":"2024-09-11T07:52:36.543891+0000","last_peered":"2024-09-11T07:52:36.543891+0000","last_clean":"2024-09-11T07:52:36.543891+0000","last_became_active":"2024-09-11T07:26:35.562341+0000","last_became_peered":"2024-09-11T07:26:35.562341+0000","last_unstale":"2024-09-11T07:52:36.543891+0000","last_undegraded":"2024-09-11T07:52:36.543891+0000","last_fullsized":"2024-09-11T07:52:36.543891+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:47.656446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:51:47.656446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:49.332839+0000","scrub_duration":0.0013035880000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":609,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.860560+0000","last_change":"2024-09-11T07:51:38.644591+0000","last_active":"2024-09-11T07:52:35.860560+0000","last_peered":"2024-09-11T07:52:35.860560+0000","last_clean":"2024-09-11T07:52:35.860560+0000","last_became_active":"2024-09-11T07:51:34.323298+0000","last_became_peered":"2024-09-11T07:51:34.323298+0000","last_unstale":"2024-09-11T07:52:35.860560+0000","last_undegraded":"2024-09-11T07:52:35.860560+0000","last_fullsized":"2024-09-11T07:52:35.860560+0000","mapping_epoch":509,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":510,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:38.644532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:51:38.644532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:58.564573+0000","scrub_duration":0.001609573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"426'32","reported_seq":76,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861645+0000","last_change":"2024-09-11T07:51:48.135202+0000","last_active":"2024-09-11T07:52:35.861645+0000","last_peered":"2024-09-11T07:52:35.861645+0000","last_clean":"2024-09-11T07:52:35.861645+0000","last_became_active":"2024-09-11T07:50:50.862648+0000","last_became_peered":"2024-09-11T07:50:50.862648+0000","last_unstale":"2024-09-11T07:52:35.861645+0000","last_undegraded":"2024-09-11T07:52:35.861645+0000","last_fullsized":"2024-09-11T07:52:35.861645+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'32","last_scrub_stamp":"2024-09-11T07:51:48.135133+0000","last_deep_scrub":"234'24","last_deep_scrub_stamp":"2024-09-11T07:41:32.694510+0000","last_clean_scrub_stamp":"2024-09-11T07:51:48.135133+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:51.716765+0000","scrub_duration":0.00246175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":454,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.862008+0000","last_change":"2024-09-11T07:51:29.942644+0000","last_active":"2024-09-11T07:52:35.862008+0000","last_peered":"2024-09-11T07:52:35.862008+0000","last_clean":"2024-09-11T07:52:35.862008+0000","last_became_active":"2024-09-11T07:51:14.064911+0000","last_became_peered":"2024-09-11T07:51:14.064911+0000","last_unstale":"2024-09-11T07:52:35.862008+0000","last_undegraded":"2024-09-11T07:52:35.862008+0000","last_fullsized":"2024-09-11T07:52:35.862008+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:29.942588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:49.677003+0000","last_clean_scrub_stamp":"2024-09-11T07:51:29.942588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:45.079513+0000","scrub_duration":0.001484642,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":567,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861037+0000","last_change":"2024-09-11T07:52:01.907696+0000","last_active":"2024-09-11T07:52:35.861037+0000","last_peered":"2024-09-11T07:52:35.861037+0000","last_clean":"2024-09-11T07:52:35.861037+0000","last_became_active":"2024-09-11T07:51:19.095374+0000","last_became_peered":"2024-09-11T07:51:19.095374+0000","last_unstale":"2024-09-11T07:52:35.861037+0000","last_undegraded":"2024-09-11T07:52:35.861037+0000","last_fullsized":"2024-09-11T07:52:35.861037+0000","mapping_epoch":504,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":505,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:01.907636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:52:01.907636+0000","last_clean_scrub_stamp":"2024-09-11T07:52:01.907636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:04.072559+0000","scrub_duration":0.0032001429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":59,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:37.524426+0000","last_change":"2024-09-11T07:52:37.524381+0000","last_active":"2024-09-11T07:52:37.524426+0000","last_peered":"2024-09-11T07:52:37.524426+0000","last_clean":"2024-09-11T07:52:37.524426+0000","last_became_active":"2024-09-11T07:51:34.321822+0000","last_became_peered":"2024-09-11T07:51:34.321822+0000","last_unstale":"2024-09-11T07:52:37.524426+0000","last_undegraded":"2024-09-11T07:52:37.524426+0000","last_fullsized":"2024-09-11T07:52:37.524426+0000","mapping_epoch":509,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":510,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:37.524308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:51:15.671653+0000","last_clean_scrub_stamp":"2024-09-11T07:52:37.524308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:49.945387+0000","scrub_duration":0.0019592149999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"426'14","reported_seq":629,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.544178+0000","last_change":"2024-09-11T07:52:02.431975+0000","last_active":"2024-09-11T07:52:36.544178+0000","last_peered":"2024-09-11T07:52:36.544178+0000","last_clean":"2024-09-11T07:52:36.544178+0000","last_became_active":"2024-09-11T07:51:13.062932+0000","last_became_peered":"2024-09-11T07:51:13.062932+0000","last_unstale":"2024-09-11T07:52:36.544178+0000","last_undegraded":"2024-09-11T07:52:36.544178+0000","last_fullsized":"2024-09-11T07:52:36.544178+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'14","last_scrub_stamp":"2024-09-11T07:52:02.431914+0000","last_deep_scrub":"234'10","last_deep_scrub_stamp":"2024-09-11T07:41:29.585284+0000","last_clean_scrub_stamp":"2024-09-11T07:52:02.431914+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:06.057909+0000","scrub_duration":0.001901727,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":16,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":577,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.643060+0000","last_change":"2024-09-11T07:52:11.613845+0000","last_active":"2024-09-11T07:52:36.643060+0000","last_peered":"2024-09-11T07:52:36.643060+0000","last_clean":"2024-09-11T07:52:36.643060+0000","last_became_active":"2024-09-11T07:51:37.863531+0000","last_became_peered":"2024-09-11T07:51:37.863531+0000","last_unstale":"2024-09-11T07:52:36.643060+0000","last_undegraded":"2024-09-11T07:52:36.643060+0000","last_fullsized":"2024-09-11T07:52:36.643060+0000","mapping_epoch":511,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":512,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:11.613764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:27:58.043009+0000","last_clean_scrub_stamp":"2024-09-11T07:52:11.613764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:30.554453+0000","scrub_duration":0.001556357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":391,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.628008+0000","last_change":"2024-09-11T07:51:48.204460+0000","last_active":"2024-09-11T07:52:36.628008+0000","last_peered":"2024-09-11T07:52:36.628008+0000","last_clean":"2024-09-11T07:52:36.628008+0000","last_became_active":"2024-09-11T07:51:48.203815+0000","last_became_peered":"2024-09-11T07:51:48.203815+0000","last_unstale":"2024-09-11T07:52:36.628008+0000","last_undegraded":"2024-09-11T07:52:36.628008+0000","last_fullsized":"2024-09-11T07:52:36.628008+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:45.775439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:45:31.477552+0000","last_clean_scrub_stamp":"2024-09-11T07:51:45.775439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:49.810374+0000","scrub_duration":0.001281325,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":69,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.599048+0000","last_change":"2024-09-11T07:52:30.608369+0000","last_active":"2024-09-11T07:52:36.599048+0000","last_peered":"2024-09-11T07:52:36.599048+0000","last_clean":"2024-09-11T07:52:36.599048+0000","last_became_active":"2024-09-11T07:50:56.911204+0000","last_became_peered":"2024-09-11T07:50:56.911204+0000","last_unstale":"2024-09-11T07:52:36.599048+0000","last_undegraded":"2024-09-11T07:52:36.599048+0000","last_fullsized":"2024-09-11T07:52:36.599048+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:30.608304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:52:30.608304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:59.901358+0000","scrub_duration":0.0017707720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"533'32","reported_seq":682,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:39.461834+0000","last_change":"2024-09-11T07:52:39.461787+0000","last_active":"2024-09-11T07:52:39.461834+0000","last_peered":"2024-09-11T07:52:39.461834+0000","last_clean":"2024-09-11T07:52:39.461834+0000","last_became_active":"2024-09-11T07:51:48.202896+0000","last_became_peered":"2024-09-11T07:51:48.202896+0000","last_unstale":"2024-09-11T07:52:39.461834+0000","last_undegraded":"2024-09-11T07:52:39.461834+0000","last_fullsized":"2024-09-11T07:52:39.461834+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"533'32","last_scrub_stamp":"2024-09-11T07:52:39.461710+0000","last_deep_scrub":"533'32","last_deep_scrub_stamp":"2024-09-11T07:52:39.461710+0000","last_clean_scrub_stamp":"2024-09-11T07:52:39.461710+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:45.067466+0000","scrub_duration":0.0045958320000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":335,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.628947+0000","last_change":"2024-09-11T07:51:49.725554+0000","last_active":"2024-09-11T07:52:36.628947+0000","last_peered":"2024-09-11T07:52:36.628947+0000","last_clean":"2024-09-11T07:52:36.628947+0000","last_became_active":"2024-09-11T07:51:45.192691+0000","last_became_peered":"2024-09-11T07:51:45.192691+0000","last_unstale":"2024-09-11T07:52:36.628947+0000","last_undegraded":"2024-09-11T07:52:36.628947+0000","last_fullsized":"2024-09-11T07:52:36.628947+0000","mapping_epoch":514,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":515,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:49.725487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:35:48.366931+0000","last_clean_scrub_stamp":"2024-09-11T07:51:49.725487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:08.814392+0000","scrub_duration":0.001478095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":59,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.598767+0000","last_change":"2024-09-11T07:51:11.530813+0000","last_active":"2024-09-11T07:52:36.598767+0000","last_peered":"2024-09-11T07:52:36.598767+0000","last_clean":"2024-09-11T07:52:36.598767+0000","last_became_active":"2024-09-11T07:50:56.911495+0000","last_became_peered":"2024-09-11T07:50:56.911495+0000","last_unstale":"2024-09-11T07:52:36.598767+0000","last_undegraded":"2024-09-11T07:52:36.598767+0000","last_fullsized":"2024-09-11T07:52:36.598767+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:11.530745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:29:39.021876+0000","last_clean_scrub_stamp":"2024-09-11T07:51:11.530745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014437320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":582,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.544621+0000","last_change":"2024-09-11T07:52:21.416292+0000","last_active":"2024-09-11T07:52:36.544621+0000","last_peered":"2024-09-11T07:52:36.544621+0000","last_clean":"2024-09-11T07:52:36.544621+0000","last_became_active":"2024-09-11T07:26:37.862215+0000","last_became_peered":"2024-09-11T07:26:37.862215+0000","last_unstale":"2024-09-11T07:52:36.544621+0000","last_undegraded":"2024-09-11T07:52:36.544621+0000","last_fullsized":"2024-09-11T07:52:36.544621+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:21.416240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:32.904006+0000","last_clean_scrub_stamp":"2024-09-11T07:52:21.416240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:28.815980+0000","scrub_duration":0.0021936759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"450'2","reported_seq":73,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-11T07:52:44.576158+0000","last_change":"2024-09-11T07:52:44.576126+0000","last_active":"2024-09-11T07:52:44.576158+0000","last_peered":"2024-09-11T07:52:44.576158+0000","last_clean":"2024-09-11T07:52:44.576158+0000","last_became_active":"2024-09-11T07:50:56.909284+0000","last_became_peered":"2024-09-11T07:50:56.909284+0000","last_unstale":"2024-09-11T07:52:44.576158+0000","last_undegraded":"2024-09-11T07:52:44.576158+0000","last_fullsized":"2024-09-11T07:52:44.576158+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"450'2","last_scrub_stamp":"2024-09-11T07:52:44.576038+0000","last_deep_scrub":"450'2","last_deep_scrub_stamp":"2024-09-11T07:48:29.800545+0000","last_clean_scrub_stamp":"2024-09-11T07:52:44.576038+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:11.588848+0000","scrub_duration":0.0016215019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"57'1","reported_seq":562,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-11T07:52:42.720550+0000","last_change":"2024-09-11T07:52:42.720498+0000","last_active":"2024-09-11T07:52:42.720550+0000","last_peered":"2024-09-11T07:52:42.720550+0000","last_clean":"2024-09-11T07:52:42.720550+0000","last_became_active":"2024-09-11T07:51:48.202367+0000","last_became_peered":"2024-09-11T07:51:48.202367+0000","last_unstale":"2024-09-11T07:52:42.720550+0000","last_undegraded":"2024-09-11T07:52:42.720550+0000","last_fullsized":"2024-09-11T07:52:42.720550+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-09-11T07:52:42.720425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:52:42.720425+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:48.558787+0000","scrub_duration":0.0017103579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":2622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"426'30","reported_seq":81,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861876+0000","last_change":"2024-09-11T07:51:26.260317+0000","last_active":"2024-09-11T07:52:35.861876+0000","last_peered":"2024-09-11T07:52:35.861876+0000","last_clean":"2024-09-11T07:52:35.861876+0000","last_became_active":"2024-09-11T07:50:50.868078+0000","last_became_peered":"2024-09-11T07:50:50.868078+0000","last_unstale":"2024-09-11T07:52:35.861876+0000","last_undegraded":"2024-09-11T07:52:35.861876+0000","last_fullsized":"2024-09-11T07:52:35.861876+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'30","last_scrub_stamp":"2024-09-11T07:51:26.260244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:51:26.260244+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:48.632277+0000","scrub_duration":0.0019868400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1f","version":"517'18","reported_seq":415,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:39.825425+0000","last_change":"2024-09-11T07:51:48.216012+0000","last_active":"2024-09-11T07:52:39.825425+0000","last_peered":"2024-09-11T07:52:39.825425+0000","last_clean":"2024-09-11T07:52:39.825425+0000","last_became_active":"2024-09-11T07:51:48.206146+0000","last_became_peered":"2024-09-11T07:51:48.206146+0000","last_unstale":"2024-09-11T07:52:39.825425+0000","last_undegraded":"2024-09-11T07:52:39.825425+0000","last_fullsized":"2024-09-11T07:52:39.825425+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"493'16","last_scrub_stamp":"2024-09-11T07:51:34.147125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:51:34.147125+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:56.229859+0000","scrub_duration":0.0037941960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":501,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861570+0000","last_change":"2024-09-11T07:52:11.677900+0000","last_active":"2024-09-11T07:52:35.861570+0000","last_peered":"2024-09-11T07:52:35.861570+0000","last_clean":"2024-09-11T07:52:35.861570+0000","last_became_active":"2024-09-11T07:51:14.065311+0000","last_became_peered":"2024-09-11T07:51:14.065311+0000","last_unstale":"2024-09-11T07:52:35.861570+0000","last_undegraded":"2024-09-11T07:52:35.861570+0000","last_fullsized":"2024-09-11T07:52:35.861570+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:11.677846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:52:11.677846+0000","last_clean_scrub_stamp":"2024-09-11T07:52:11.677846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:30.305406+0000","scrub_duration":0.014118188,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":598,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.544687+0000","last_change":"2024-09-11T07:51:52.526031+0000","last_active":"2024-09-11T07:52:36.544687+0000","last_peered":"2024-09-11T07:52:36.544687+0000","last_clean":"2024-09-11T07:52:36.544687+0000","last_became_active":"2024-09-11T07:26:31.419189+0000","last_became_peered":"2024-09-11T07:26:31.419189+0000","last_unstale":"2024-09-11T07:52:36.544687+0000","last_undegraded":"2024-09-11T07:52:36.544687+0000","last_fullsized":"2024-09-11T07:52:36.544687+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:52.525949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:51:52.525949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:01.957606+0000","scrub_duration":0.001859044,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"426'27","reported_seq":167,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.545127+0000","last_change":"2024-09-11T07:51:13.238482+0000","last_active":"2024-09-11T07:52:36.545127+0000","last_peered":"2024-09-11T07:52:36.545127+0000","last_clean":"2024-09-11T07:52:36.545127+0000","last_became_active":"2024-09-11T07:46:13.790166+0000","last_became_peered":"2024-09-11T07:46:13.790166+0000","last_unstale":"2024-09-11T07:52:36.545127+0000","last_undegraded":"2024-09-11T07:52:36.545127+0000","last_fullsized":"2024-09-11T07:52:36.545127+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'27","last_scrub_stamp":"2024-09-11T07:51:13.238422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:51:13.238422+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021108820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":648,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861591+0000","last_change":"2024-09-11T07:51:48.651760+0000","last_active":"2024-09-11T07:52:35.861591+0000","last_peered":"2024-09-11T07:52:35.861591+0000","last_clean":"2024-09-11T07:52:35.861591+0000","last_became_active":"2024-09-11T07:50:56.906837+0000","last_became_peered":"2024-09-11T07:50:56.906837+0000","last_unstale":"2024-09-11T07:52:35.861591+0000","last_undegraded":"2024-09-11T07:52:35.861591+0000","last_fullsized":"2024-09-11T07:52:35.861591+0000","mapping_epoch":494,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":495,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:48.651694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:50:22.592981+0000","last_clean_scrub_stamp":"2024-09-11T07:51:48.651694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:02.597517+0000","scrub_duration":0.001630065,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":64,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.862108+0000","last_change":"2024-09-11T07:51:35.175475+0000","last_active":"2024-09-11T07:52:35.862108+0000","last_peered":"2024-09-11T07:52:35.862108+0000","last_clean":"2024-09-11T07:52:35.862108+0000","last_became_active":"2024-09-11T07:50:50.860452+0000","last_became_peered":"2024-09-11T07:50:50.860452+0000","last_unstale":"2024-09-11T07:52:35.862108+0000","last_undegraded":"2024-09-11T07:52:35.862108+0000","last_fullsized":"2024-09-11T07:52:35.862108+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:35.175412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:51:35.175412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:52:46.700844+0000","scrub_duration":0.001368794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":286,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.971864+0000","last_change":"2024-09-11T07:52:36.971807+0000","last_active":"2024-09-11T07:52:36.971864+0000","last_peered":"2024-09-11T07:52:36.971864+0000","last_clean":"2024-09-11T07:52:36.971864+0000","last_became_active":"2024-09-11T07:51:45.190906+0000","last_became_peered":"2024-09-11T07:51:45.190906+0000","last_unstale":"2024-09-11T07:52:36.971864+0000","last_undegraded":"2024-09-11T07:52:36.971864+0000","last_fullsized":"2024-09-11T07:52:36.971864+0000","mapping_epoch":514,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":515,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:36.971740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:52:36.971740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:51.940992+0000","scrub_duration":0.0015586409999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"426'39","reported_seq":87,"reported_epoch":534,"state":"active+clean","last_fresh":"2024-09-11T07:52:44.382819+0000","last_change":"2024-09-11T07:52:44.382781+0000","last_active":"2024-09-11T07:52:44.382819+0000","last_peered":"2024-09-11T07:52:44.382819+0000","last_clean":"2024-09-11T07:52:44.382819+0000","last_became_active":"2024-09-11T07:50:53.903030+0000","last_became_peered":"2024-09-11T07:50:53.903030+0000","last_unstale":"2024-09-11T07:52:44.382819+0000","last_undegraded":"2024-09-11T07:52:44.382819+0000","last_fullsized":"2024-09-11T07:52:44.382819+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'39","last_scrub_stamp":"2024-09-11T07:52:44.382697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:32.521323+0000","last_clean_scrub_stamp":"2024-09-11T07:52:44.382697+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:10.310569+0000","scrub_duration":0.0023195859999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":450,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861852+0000","last_change":"2024-09-11T07:52:35.790021+0000","last_active":"2024-09-11T07:52:35.861852+0000","last_peered":"2024-09-11T07:52:35.861852+0000","last_clean":"2024-09-11T07:52:35.861852+0000","last_became_active":"2024-09-11T07:51:14.061993+0000","last_became_peered":"2024-09-11T07:51:14.061993+0000","last_unstale":"2024-09-11T07:52:35.861852+0000","last_undegraded":"2024-09-11T07:52:35.861852+0000","last_fullsized":"2024-09-11T07:52:35.861852+0000","mapping_epoch":502,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":503,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:35.789963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:42:32.456852+0000","last_clean_scrub_stamp":"2024-09-11T07:52:35.789963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:53.408426+0000","scrub_duration":0.0015193349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":629,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.545229+0000","last_change":"2024-09-11T07:52:12.445048+0000","last_active":"2024-09-11T07:52:36.545229+0000","last_peered":"2024-09-11T07:52:36.545229+0000","last_clean":"2024-09-11T07:52:36.545229+0000","last_became_active":"2024-09-11T07:51:34.321779+0000","last_became_peered":"2024-09-11T07:51:34.321779+0000","last_unstale":"2024-09-11T07:52:36.545229+0000","last_undegraded":"2024-09-11T07:52:36.545229+0000","last_fullsized":"2024-09-11T07:52:36.545229+0000","mapping_epoch":509,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":510,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:12.444990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:36.546273+0000","last_clean_scrub_stamp":"2024-09-11T07:52:12.444990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:23.945312+0000","scrub_duration":0.001808046,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":642,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.544855+0000","last_change":"2024-09-11T07:51:48.209238+0000","last_active":"2024-09-11T07:52:36.544855+0000","last_peered":"2024-09-11T07:52:36.544855+0000","last_clean":"2024-09-11T07:52:36.544855+0000","last_became_active":"2024-09-11T07:51:48.208985+0000","last_became_peered":"2024-09-11T07:51:48.208985+0000","last_unstale":"2024-09-11T07:52:36.544855+0000","last_undegraded":"2024-09-11T07:52:36.544855+0000","last_fullsized":"2024-09-11T07:52:36.544855+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:51:33.422146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:30.397002+0000","last_clean_scrub_stamp":"2024-09-11T07:51:33.422146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:03.422139+0000","scrub_duration":0.0017024309999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"426'29","reported_seq":673,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:36.544763+0000","last_change":"2024-09-11T07:51:24.392905+0000","last_active":"2024-09-11T07:52:36.544763+0000","last_peered":"2024-09-11T07:52:36.544763+0000","last_clean":"2024-09-11T07:52:36.544763+0000","last_became_active":"2024-09-11T07:50:50.857201+0000","last_became_peered":"2024-09-11T07:50:50.857201+0000","last_unstale":"2024-09-11T07:52:36.544763+0000","last_undegraded":"2024-09-11T07:52:36.544763+0000","last_fullsized":"2024-09-11T07:52:36.544763+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"426'29","last_scrub_stamp":"2024-09-11T07:51:24.392838+0000","last_deep_scrub":"234'23","last_deep_scrub_stamp":"2024-09-11T07:43:25.530826+0000","last_clean_scrub_stamp":"2024-09-11T07:51:24.392838+0000","objects_scrubbed":5,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029751859999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":84,"reported_epoch":533,"state":"active+clean","last_fresh":"2024-09-11T07:52:35.861375+0000","last_change":"2024-09-11T07:52:28.242947+0000","last_active":"2024-09-11T07:52:35.861375+0000","last_peered":"2024-09-11T07:52:35.861375+0000","last_clean":"2024-09-11T07:52:35.861375+0000","last_became_active":"2024-09-11T07:50:53.915112+0000","last_became_peered":"2024-09-11T07:50:53.915112+0000","last_unstale":"2024-09-11T07:52:35.861375+0000","last_undegraded":"2024-09-11T07:52:35.861375+0000","last_fullsized":"2024-09-11T07:52:35.861375+0000","mapping_epoch":492,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":493,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:52:28.242881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:26:34.535883+0000","last_clean_scrub_stamp":"2024-09-11T07:52:28.242881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:53:55.145036+0000","scrub_duration":0.0014834550000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":777,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1522,"num_read_kb":1279,"num_write":989,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":493,"num_bytes_recovered":9506,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1103,"ondisk_log_size":1103,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1148,"num_read_kb":1143,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":983040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":891,"ondisk_log_size":891,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":147,"seq":631360192763,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":924808,"kb_used_data":1280,"kb_used_omap":4,"kb_used_meta":923515,"kb_avail":92820344,"statfs":{"total":95995035648,"available":95048032256,"internally_reserved":0,"allocated":1310720,"data_stored":830043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":945679878},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:51:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":1.5980000000000001,"15min":1.593},"min":{"1min":0.29199999999999998,"5min":0.224,"15min":0.191},"max":{"1min":0.70099999999999996,"5min":43.406999999999996,"15min":75.677000000000007},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":1.6619999999999999,"15min":1.6890000000000001},"min":{"1min":0.34100000000000003,"5min":0.22,"15min":0.214},"max":{"1min":0.79700000000000004,"5min":43.845999999999997,"15min":75.638999999999996},"last":0.498}]},{"osd":1,"last update":"Wed Sep 11 07:51:51 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":3.5270000000000001,"15min":3.2250000000000001},"min":{"1min":0.31,"5min":0.247,"15min":0.22500000000000001},"max":{"1min":0.89800000000000002,"5min":161.63900000000001,"15min":327.06099999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.48199999999999998,"5min":3.5870000000000002,"15min":3.1949999999999998},"min":{"1min":0.32500000000000001,"5min":0.23400000000000001,"15min":0.22800000000000001},"max":{"1min":0.69999999999999996,"5min":161.65199999999999,"15min":327.07499999999999},"last":0.48799999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":14.401,"5min":14.401,"15min":14.401},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.48299999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:51:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":1.9490000000000001,"15min":1.29},"min":{"1min":0.34799999999999998,"5min":0.27000000000000002,"15min":0.252},"max":{"1min":0.75900000000000001,"5min":84.013999999999996,"15min":84.013999999999996},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":1.847,"15min":1.3089999999999999},"min":{"1min":0.45800000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.876,"5min":84.167000000000002,"15min":84.167000000000002},"last":0.56200000000000006}]},{"osd":4,"last update":"Wed Sep 11 07:51:51 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48199999999999998,"15min":0.68000000000000005},"min":{"1min":0.32200000000000001,"5min":0.27800000000000002,"15min":0.23999999999999999},"max":{"1min":1.01,"5min":2.5310000000000001,"15min":6.0220000000000002},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.48599999999999999,"15min":0.69499999999999995},"min":{"1min":0.36099999999999999,"5min":0.27700000000000002,"15min":0.22800000000000001},"max":{"1min":0.71999999999999997,"5min":1.359,"15min":6.9000000000000004},"last":0.36299999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:51:51 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.496,"15min":0.745},"min":{"1min":0.378,"5min":0.25700000000000001,"15min":0.25},"max":{"1min":0.97299999999999998,"5min":2.839,"15min":9.9339999999999993},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.55000000000000004,"15min":0.69999999999999996},"min":{"1min":0.33500000000000002,"5min":0.27700000000000002,"15min":0.26700000000000002},"max":{"1min":0.72299999999999998,"5min":4.4470000000000001,"15min":10.183999999999999},"last":0.44}]},{"osd":6,"last update":"Wed Sep 11 07:51:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56599999999999995,"15min":0.64000000000000001},"min":{"1min":0.318,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.84899999999999998,"5min":3.0449999999999999,"15min":5.1909999999999998},"last":0.375},{"interface":"front","average":{"1min":0.62,"5min":0.57899999999999996,"15min":0.71199999999999997},"min":{"1min":0.36399999999999999,"5min":0.32600000000000001,"15min":0.27300000000000002},"max":{"1min":0.91800000000000004,"5min":2.8069999999999999,"15min":4.2030000000000003},"last":0.40000000000000002}]}]},{"osd":6,"up_from":44,"seq":188978561348,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":898228,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":897278,"kb_avail":92846924,"statfs":{"total":95995035648,"available":95075250176,"internally_reserved":0,"allocated":958464,"data_stored":500841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":918813016},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:52:43 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":1.546,"15min":1.919},"min":{"1min":0.36699999999999999,"5min":0.221,"15min":0.219},"max":{"1min":0.78700000000000003,"5min":75.728999999999999,"15min":130.62299999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":1.4610000000000001,"15min":1.9299999999999999},"min":{"1min":0.38800000000000001,"5min":0.23799999999999999,"15min":0.20499999999999999},"max":{"1min":0.70999999999999996,"5min":75.536000000000001,"15min":130.86199999999999},"last":0.42999999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:52:43 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.89700000000000002,"15min":1.833},"min":{"1min":0.42499999999999999,"5min":0.30199999999999999,"15min":0.23599999999999999},"max":{"1min":0.79900000000000004,"5min":17.029,"15min":131.42500000000001},"last":0.441},{"interface":"front","average":{"1min":0.58199999999999996,"5min":1.0840000000000001,"15min":1.8320000000000001},"min":{"1min":0.41899999999999998,"5min":0.22800000000000001,"15min":0.22},"max":{"1min":0.76700000000000002,"5min":16.992999999999999,"15min":132.39599999999999},"last":0.41899999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:52:43 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":1.349,"15min":1.476},"min":{"1min":0.42999999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84199999999999997,"5min":18.166,"15min":18.166},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.44400000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.46100000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:52:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":1.3080000000000001,"15min":1.496},"min":{"1min":0.46300000000000002,"5min":0.28899999999999998,"15min":0.246},"max":{"1min":0.74199999999999999,"5min":36.414000000000001,"15min":49.109000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":1.369,"15min":1.585},"min":{"1min":0.38100000000000001,"5min":0.29199999999999998,"15min":0.246},"max":{"1min":0.77200000000000002,"5min":39.390000000000001,"15min":48.963999999999999},"last":0.38100000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:52:43 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48099999999999998,"15min":1.7190000000000001},"min":{"1min":0.34399999999999997,"5min":0.29099999999999998,"15min":0.23899999999999999},"max":{"1min":0.67000000000000004,"5min":2.8029999999999999,"15min":181.78800000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.502,"5min":0.52600000000000002,"15min":1.171},"min":{"1min":0.34499999999999997,"5min":0.27900000000000003,"15min":0.21099999999999999},"max":{"1min":0.74399999999999999,"5min":4.3220000000000001,"15min":128.59299999999999},"last":0.35999999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:52:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48299999999999998,"15min":0.69899999999999995},"min":{"1min":0.32700000000000001,"5min":0.25600000000000001,"15min":0.23499999999999999},"max":{"1min":0.72599999999999998,"5min":2.9220000000000002,"15min":10.935},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.51600000000000001,"15min":0.75900000000000001},"min":{"1min":0.40699999999999997,"5min":0.30499999999999999,"15min":0.253},"max":{"1min":0.73499999999999999,"5min":1.49,"15min":11.15},"last":0.40699999999999997}]},{"osd":7,"last update":"Wed Sep 11 07:52:43 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57899999999999996,"15min":1.056},"min":{"1min":0.46500000000000002,"5min":0.34899999999999998,"15min":0.27800000000000002},"max":{"1min":0.70499999999999996,"5min":2.8340000000000001,"15min":60.279000000000003},"last":0.501},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57599999999999996,"15min":0.97099999999999997},"min":{"1min":0.39800000000000002,"5min":0.29899999999999999,"15min":0.215},"max":{"1min":0.69199999999999995,"5min":2.9550000000000001,"15min":60.767000000000003},"last":0.45300000000000001}]}]},{"osd":5,"up_from":37,"seq":158913790282,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":897716,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":896190,"kb_avail":92847436,"statfs":{"total":95995035648,"available":95075774464,"internally_reserved":0,"allocated":1552384,"data_stored":1091650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":917698904},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:52:21 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.242,"15min":1.403},"min":{"1min":0.36799999999999999,"5min":0.20799999999999999,"15min":0.20200000000000001},"max":{"1min":17.818000000000001,"5min":24.353000000000002,"15min":40.313000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":1.089,"15min":1.363},"min":{"1min":0.318,"5min":0.223,"15min":0.20200000000000001},"max":{"1min":0.96699999999999997,"5min":25.890000000000001,"15min":40.235999999999997},"last":0.38900000000000001}]},{"osd":1,"last update":"Wed Sep 11 07:52:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.86699999999999999,"15min":1.4590000000000001},"min":{"1min":0.46300000000000002,"5min":0.21199999999999999,"15min":0.20799999999999999},"max":{"1min":0.76000000000000001,"5min":6.5960000000000001,"15min":122.738},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.85999999999999999,"15min":1.4299999999999999},"min":{"1min":0.35899999999999999,"5min":0.254,"15min":0.24099999999999999},"max":{"1min":0.77600000000000002,"5min":6.7069999999999999,"15min":122.66200000000001},"last":0.56999999999999995}]},{"osd":2,"last update":"Wed Sep 11 07:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.441}]},{"osd":3,"last update":"Wed Sep 11 07:52:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":1.865,"15min":2.5249999999999999},"min":{"1min":0.34399999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.72999999999999998,"5min":82.995000000000005,"15min":132.715},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":1.7609999999999999,"15min":2.5470000000000002},"min":{"1min":0.48199999999999998,"5min":0.28399999999999997,"15min":0.28100000000000003},"max":{"1min":0.752,"5min":82.959000000000003,"15min":133.01599999999999},"last":0.60599999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:52:21 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.46899999999999997,"15min":0.73099999999999998},"min":{"1min":0.33100000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.66700000000000004,"5min":2.3969999999999998,"15min":11.103},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.46100000000000002,"15min":0.72699999999999998},"min":{"1min":0.38200000000000001,"5min":0.25900000000000001,"15min":0.22500000000000001},"max":{"1min":0.77400000000000002,"5min":2.516,"15min":10.305},"last":0.51400000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:52:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.88800000000000001,"15min":0.79000000000000004},"min":{"1min":0.39100000000000001,"5min":0.26600000000000001,"15min":0.248},"max":{"1min":0.68899999999999995,"5min":28.893000000000001,"15min":28.893000000000001},"last":0.621},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.90900000000000003,"15min":0.82399999999999995},"min":{"1min":0.40100000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.70799999999999996,"5min":29.033999999999999,"15min":29.033999999999999},"last":0.627}]},{"osd":7,"last update":"Wed Sep 11 07:52:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.505,"15min":1.4690000000000001},"min":{"1min":0.42999999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68799999999999994,"5min":1.6699999999999999,"15min":115.44499999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.51200000000000001,"15min":1.421},"min":{"1min":0.42599999999999999,"5min":0.314,"15min":0.26800000000000002},"max":{"1min":0.751,"5min":2.5470000000000002,"15min":115.071},"last":0.59599999999999997}]}]},{"osd":4,"up_from":31,"seq":133143986510,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":902348,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":901374,"kb_avail":92842804,"statfs":{"total":95995035648,"available":95071031296,"internally_reserved":0,"allocated":983040,"data_stored":501852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":923007320},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:52:02 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":1.069,"15min":1.367},"min":{"1min":0.27700000000000002,"5min":0.219,"15min":0.20499999999999999},"max":{"1min":0.73299999999999998,"5min":18.327999999999999,"15min":87.623000000000005},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":1.0980000000000001,"15min":1.45},"min":{"1min":0.34799999999999998,"5min":0.253,"15min":0.222},"max":{"1min":0.77000000000000002,"5min":18.431999999999999,"15min":87.682000000000002},"last":0.34499999999999997}]},{"osd":1,"last update":"Wed Sep 11 07:52:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.751,"15min":1.72},"min":{"1min":0.31900000000000001,"5min":0.23200000000000001,"15min":0.22900000000000001},"max":{"1min":0.67400000000000004,"5min":4.9320000000000004,"15min":141.78399999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.77200000000000002,"15min":1.8009999999999999},"min":{"1min":0.36699999999999999,"5min":0.26300000000000001,"15min":0.224},"max":{"1min":0.85299999999999998,"5min":7.383,"15min":145.65899999999999},"last":0.375}]},{"osd":2,"last update":"Wed Sep 11 07:51:43 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.39200000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:52:02 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":2.6440000000000001,"15min":1.589},"min":{"1min":0.34599999999999997,"5min":0.26700000000000002,"15min":0.25700000000000001},"max":{"1min":0.92300000000000004,"5min":66.855999999999995,"15min":66.855999999999995},"last":0.315},{"interface":"front","average":{"1min":1.0389999999999999,"5min":2.8460000000000001,"15min":1.6379999999999999},"min":{"1min":0.41299999999999998,"5min":0.28799999999999998,"15min":0.26100000000000001},"max":{"1min":9.4610000000000003,"5min":66.140000000000001,"15min":66.140000000000001},"last":0.46300000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:52:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.43099999999999999,"15min":0.64900000000000002},"min":{"1min":0.375,"5min":0.24399999999999999,"15min":0.224},"max":{"1min":0.65300000000000002,"5min":0.88100000000000001,"15min":5.6470000000000002},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.41199999999999998,"15min":0.67600000000000005},"min":{"1min":0.34699999999999998,"5min":0.25,"15min":0.23000000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":5.7450000000000001},"last":0.434}]},{"osd":6,"last update":"Wed Sep 11 07:51:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.83099999999999996,"15min":1.0329999999999999},"min":{"1min":0.308,"5min":0.26400000000000001,"15min":0.23599999999999999},"max":{"1min":0.66500000000000004,"5min":38.334000000000003,"15min":38.334000000000003},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.81799999999999995,"15min":0.99299999999999999},"min":{"1min":0.36399999999999999,"5min":0.28299999999999997,"15min":0.27500000000000002},"max":{"1min":0.61399999999999999,"5min":38.466999999999999,"15min":38.466999999999999},"last":0.41199999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.49199999999999999,"15min":1.147},"min":{"1min":0.372,"5min":0.313,"15min":0.29399999999999998},"max":{"1min":0.69899999999999995,"5min":1.089,"15min":102.258},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.48799999999999999,"15min":1.0529999999999999},"min":{"1min":0.38400000000000001,"5min":0.315,"15min":0.29799999999999999},"max":{"1min":0.69799999999999995,"5min":1.0149999999999999,"15min":101.886},"last":0.40200000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182739,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":900436,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":898878,"kb_avail":92844716,"statfs":{"total":95995035648,"available":95072989184,"internally_reserved":0,"allocated":1585152,"data_stored":1091684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":920451416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":1.1659999999999999,"15min":1.3520000000000001},"min":{"1min":0.22800000000000001,"5min":0.184,"15min":0.16600000000000001},"max":{"1min":0.60199999999999998,"5min":9.5399999999999991,"15min":27.655999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":1.276,"15min":1.468},"min":{"1min":0.27800000000000002,"5min":0.21299999999999999,"15min":0.183},"max":{"1min":0.57899999999999996,"5min":8.0969999999999995,"15min":26.759},"last":0.48199999999999998}]},{"osd":1,"last update":"Wed Sep 11 07:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":1.1220000000000001,"15min":2.5089999999999999},"min":{"1min":0.28899999999999998,"5min":0.23599999999999999,"15min":0.19500000000000001},"max":{"1min":0.76900000000000002,"5min":16.971,"15min":135.41999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.35299999999999998,"5min":1.1519999999999999,"15min":2.2679999999999998},"min":{"1min":0.224,"5min":0.221,"15min":0.221},"max":{"1min":0.47899999999999998,"5min":18.277000000000001,"15min":136.126},"last":0.46500000000000002}]},{"osd":2,"last update":"Wed Sep 11 07:51:44 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.42199999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:52:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.82499999999999996,"15min":1.6539999999999999},"min":{"1min":0.36699999999999999,"5min":0.32000000000000001,"15min":0.27500000000000002},"max":{"1min":0.70799999999999996,"5min":7.5659999999999998,"15min":132.76400000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":1.0580000000000001,"15min":1.774},"min":{"1min":0.40999999999999998,"5min":0.34599999999999997,"15min":0.255},"max":{"1min":0.89400000000000002,"5min":18.957000000000001,"15min":132.154},"last":0.57199999999999995}]},{"osd":5,"last update":"Wed Sep 11 07:52:18 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.87,"15min":1.363},"min":{"1min":0.44500000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.79000000000000004,"5min":7.5270000000000001,"15min":137.50299999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":1.252,"15min":1.5449999999999999},"min":{"1min":0.439,"5min":0.31,"15min":0.29199999999999998},"max":{"1min":0.93500000000000005,"5min":19.266999999999999,"15min":137.45599999999999},"last":0.60599999999999998}]},{"osd":6,"last update":"Wed Sep 11 07:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.873,"15min":1.861},"min":{"1min":0.40600000000000003,"5min":0.316,"15min":0.313},"max":{"1min":0.748,"5min":6.3609999999999998,"15min":233.98500000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.97199999999999998,"15min":1.897},"min":{"1min":0.41399999999999998,"5min":0.32500000000000001,"15min":0.30399999999999999},"max":{"1min":0.77400000000000002,"5min":7.4500000000000002,"15min":234.05199999999999},"last":0.59199999999999997}]},{"osd":7,"last update":"Wed Sep 11 07:52:30 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.80700000000000005,"15min":2.6070000000000002},"min":{"1min":0.51000000000000001,"5min":0.376,"15min":0.35699999999999998},"max":{"1min":0.97099999999999997,"5min":4.952,"15min":162.42699999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.873,"15min":2.5390000000000001},"min":{"1min":0.42699999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77300000000000002,"5min":6.3860000000000001,"15min":162.39099999999999},"last":0.63700000000000001}]}]},{"osd":2,"up_from":485,"seq":2083059138588,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":358328,"kb_used_data":1200,"kb_used_omap":50,"kb_used_meta":357069,"kb_avail":93386824,"statfs":{"total":95995035648,"available":95628107776,"internally_reserved":0,"allocated":1228800,"data_stored":763766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51973,"internal_metadata":365638907},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:52:45 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.44500000000000001,"15min":0.45200000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.72999999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":0.375},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.45100000000000001,"15min":0.46300000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.64800000000000002,"5min":1.948,"15min":1.948},"last":0.28100000000000003}]},{"osd":1,"last update":"Wed Sep 11 07:52:45 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.39400000000000002,"15min":0.40200000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.61199999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.437,"15min":0.44},"min":{"1min":0.27700000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.34899999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:52:45 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.442,"15min":0.44600000000000001},"min":{"1min":0.30199999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63300000000000001,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.47899999999999998,"15min":0.48699999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.59799999999999998,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":0.26100000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:52:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.58399999999999996,"15min":0.58999999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.74299999999999999,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.61199999999999999,"15min":0.622},"min":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.79300000000000004,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.46800000000000003}]},{"osd":5,"last update":"Wed Sep 11 07:52:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.62,"15min":0.628},"min":{"1min":0.45600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.70199999999999996,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.63100000000000001,"15min":0.63500000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81299999999999994,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.45500000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:52:45 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66900000000000004,"15min":0.67700000000000005},"min":{"1min":0.42299999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80600000000000005,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.65700000000000003,"15min":0.66100000000000003},"min":{"1min":0.47399999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.755,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.57799999999999996}]},{"osd":7,"last update":"Wed Sep 11 07:52:45 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.66200000000000003,"15min":0.66900000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83099999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.67000000000000004,"15min":0.67600000000000005},"min":{"1min":0.49099999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.78600000000000003,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.49099999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575195,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":913636,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":912638,"kb_avail":92831516,"statfs":{"total":95995035648,"available":95059472384,"internally_reserved":0,"allocated":1007616,"data_stored":502637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":934541656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:51:51 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":2.7650000000000001,"15min":1.893},"min":{"1min":0.221,"5min":0.20499999999999999,"15min":0.17299999999999999},"max":{"1min":0.48599999999999999,"5min":88.762,"15min":88.762},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.32200000000000001,"5min":2.7029999999999998,"15min":1.9119999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.186},"max":{"1min":0.41799999999999998,"5min":84.926000000000002,"15min":84.926000000000002},"last":0.29099999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.34699999999999998}]},{"osd":3,"last update":"Wed Sep 11 07:52:43 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":1.2629999999999999,"15min":3.3799999999999999},"min":{"1min":0.27900000000000003,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.52900000000000003,"5min":34.646000000000001,"15min":276.24900000000002},"last":0.313},{"interface":"front","average":{"1min":0.35499999999999998,"5min":1.159,"15min":3.2200000000000002},"min":{"1min":0.245,"5min":0.22900000000000001,"15min":0.19800000000000001},"max":{"1min":0.48499999999999999,"5min":34.518999999999998,"15min":274.98000000000002},"last":0.36199999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:51:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.81999999999999995,"15min":1.2490000000000001},"min":{"1min":0.42899999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.70399999999999996,"5min":8.1219999999999999,"15min":13.82},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.98099999999999998,"15min":1.2410000000000001},"min":{"1min":0.36899999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.75,"5min":4.7149999999999999,"15min":13.801},"last":0.49299999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:52:23 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.82199999999999995,"15min":1.1040000000000001},"min":{"1min":0.42999999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.873,"5min":8.0939999999999994,"15min":8.3680000000000003},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.86199999999999999,"15min":1.1619999999999999},"min":{"1min":0.35999999999999999,"5min":0.31900000000000001,"15min":0.27200000000000002},"max":{"1min":0.81299999999999994,"5min":4.7569999999999997,"15min":8.6809999999999992},"last":0.54600000000000004}]},{"osd":6,"last update":"Wed Sep 11 07:52:43 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.82499999999999996,"15min":1.159},"min":{"1min":0.39100000000000001,"5min":0.32300000000000001,"15min":0.30599999999999999},"max":{"1min":0.85199999999999998,"5min":4.79,"15min":13.981999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.68999999999999995,"15min":0.98999999999999999},"min":{"1min":0.42299999999999999,"5min":0.372,"15min":0.318},"max":{"1min":0.84999999999999998,"5min":2.9039999999999999,"15min":13.928000000000001},"last":0.47599999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:52:43 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.70099999999999996,"15min":1.034},"min":{"1min":0.47199999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88,"5min":2.875,"15min":9.3629999999999995},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.85999999999999999,"15min":1.2869999999999999},"min":{"1min":0.503,"5min":0.34000000000000002,"15min":0.33400000000000002},"max":{"1min":0.83399999999999996,"5min":8.0549999999999997,"15min":13.715},"last":0.51300000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738721,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":894340,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":893374,"kb_avail":92850812,"statfs":{"total":95995035648,"available":95079231488,"internally_reserved":0,"allocated":978944,"data_stored":501283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":914815320},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:51:47 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":1.4419999999999999,"15min":1.5069999999999999},"min":{"1min":0.20599999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.47799999999999998,"5min":6.79,"15min":72.418000000000006},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":1.577,"15min":1.5720000000000001},"min":{"1min":0.26300000000000001,"5min":0.187,"15min":0.17599999999999999},"max":{"1min":0.47499999999999998,"5min":10.061,"15min":71.703999999999994},"last":0.44800000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:52:44 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.35699999999999998,"15min":0.35499999999999998},"min":{"1min":0.26200000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.378,"15min":0.375},"min":{"1min":0.27300000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.39700000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:51:47 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":1.944,"15min":1.671},"min":{"1min":0.30199999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.70799999999999996,"5min":62.667000000000002,"15min":62.667000000000002},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":2.2869999999999999,"15min":1.8169999999999999},"min":{"1min":0.27500000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.69199999999999995,"5min":61.691000000000003,"15min":61.691000000000003},"last":0.50700000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:52:08 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.35099999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.75800000000000001,"5min":9.6310000000000002,"15min":30.315000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.95699999999999996,"15min":1.2410000000000001},"min":{"1min":0.39400000000000002,"5min":0.30099999999999999,"15min":0.28599999999999998},"max":{"1min":0.79000000000000004,"5min":4.5960000000000001,"15min":30.363},"last":0.64300000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:52:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":1.2869999999999999,"15min":1.1830000000000001},"min":{"1min":0.38300000000000001,"5min":0.29999999999999999,"15min":0.26800000000000002},"max":{"1min":0.89400000000000002,"5min":9.5839999999999996,"15min":9.5839999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.91800000000000004,"15min":1.1379999999999999},"min":{"1min":0.33800000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81499999999999995,"5min":4.6529999999999996,"15min":7.202},"last":0.75800000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:51:47 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":1.008,"15min":1.1659999999999999},"min":{"1min":0.38400000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.1120000000000001,"5min":5.9249999999999998,"15min":6.5190000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":1.252,"15min":1.1990000000000001},"min":{"1min":0.436,"5min":0.36699999999999999,"15min":0.30199999999999999},"max":{"1min":0.89500000000000002,"5min":5.891,"15min":15.404},"last":0.79100000000000004}]},{"osd":7,"last update":"Wed Sep 11 07:51:47 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":1.0009999999999999,"15min":1.429},"min":{"1min":0.42499999999999999,"5min":0.35699999999999998,"15min":0.34399999999999997},"max":{"1min":1.032,"5min":4.6260000000000003,"15min":34.704999999999998},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.629,"5min":1.2529999999999999,"15min":1.526},"min":{"1min":0.48299999999999998,"5min":0.41299999999999998,"15min":0.35999999999999999},"max":{"1min":1.0920000000000001,"5min":5.9139999999999997,"15min":34.930999999999997},"last":0.74099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:52:48.499 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-11T07:52:48.499 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 12a285d8-700e-11ef-bcea-c7b262605968 -- ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-11T07:52:48.756 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:52:48.787 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:48 smithi104 ceph-mon[33507]: pgmap v1238: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:52:48.788 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:48 smithi104 ceph-mon[33507]: 2.3 scrub starts 2024-09-11T07:52:48.788 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:48 smithi104 ceph-mon[33507]: 2.3 scrub ok 2024-09-11T07:52:48.788 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:48 smithi104 ceph-mon[33507]: 4.16 scrub starts 2024-09-11T07:52:48.788 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:48 smithi104 ceph-mon[33507]: 4.16 scrub ok 2024-09-11T07:52:48.878 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:48 smithi022 ceph-mon[37330]: pgmap v1238: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:52:48.878 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:48 smithi022 ceph-mon[37330]: 2.3 scrub starts 2024-09-11T07:52:48.878 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:48 smithi022 ceph-mon[37330]: 2.3 scrub ok 2024-09-11T07:52:48.878 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:48 smithi022 ceph-mon[37330]: 4.16 scrub starts 2024-09-11T07:52:48.878 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:48 smithi022 ceph-mon[37330]: 4.16 scrub ok 2024-09-11T07:52:48.879 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:48 smithi022 ceph-mon[29462]: pgmap v1238: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:52:48.879 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:48 smithi022 ceph-mon[29462]: 2.3 scrub starts 2024-09-11T07:52:48.879 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:48 smithi022 ceph-mon[29462]: 2.3 scrub ok 2024-09-11T07:52:48.879 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:48 smithi022 ceph-mon[29462]: 4.16 scrub starts 2024-09-11T07:52:48.879 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:48 smithi022 ceph-mon[29462]: 4.16 scrub ok 2024-09-11T07:52:49.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:49 smithi104 ceph-mon[33507]: from='client.16575 v1:172.21.15.22:0/2126580544' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:52:49.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:49 smithi104 ceph-mon[33507]: 6.1e scrub starts 2024-09-11T07:52:49.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:49 smithi104 ceph-mon[33507]: 6.1e scrub ok 2024-09-11T07:52:49.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:49 smithi104 ceph-mon[33507]: 4.1d deep-scrub starts 2024-09-11T07:52:49.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:49 smithi104 ceph-mon[33507]: 4.1d deep-scrub ok 2024-09-11T07:52:49.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:49 smithi104 ceph-mon[33507]: 2.a scrub starts 2024-09-11T07:52:49.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:49 smithi104 ceph-mon[33507]: 2.a scrub ok 2024-09-11T07:52:49.916 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:49 smithi022 ceph-mon[29462]: from='client.16575 v1:172.21.15.22:0/2126580544' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:52:49.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:49 smithi022 ceph-mon[29462]: 6.1e scrub starts 2024-09-11T07:52:49.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:49 smithi022 ceph-mon[29462]: 6.1e scrub ok 2024-09-11T07:52:49.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:49 smithi022 ceph-mon[29462]: 4.1d deep-scrub starts 2024-09-11T07:52:49.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:49 smithi022 ceph-mon[29462]: 4.1d deep-scrub ok 2024-09-11T07:52:49.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:49 smithi022 ceph-mon[29462]: 2.a scrub starts 2024-09-11T07:52:49.917 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:49 smithi022 ceph-mon[29462]: 2.a scrub ok 2024-09-11T07:52:49.917 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:49 smithi022 ceph-mon[37330]: from='client.16575 v1:172.21.15.22:0/2126580544' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:52:49.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:49 smithi022 ceph-mon[37330]: 6.1e scrub starts 2024-09-11T07:52:49.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:49 smithi022 ceph-mon[37330]: 6.1e scrub ok 2024-09-11T07:52:49.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:49 smithi022 ceph-mon[37330]: 4.1d deep-scrub starts 2024-09-11T07:52:49.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:49 smithi022 ceph-mon[37330]: 4.1d deep-scrub ok 2024-09-11T07:52:49.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:49 smithi022 ceph-mon[37330]: 2.a scrub starts 2024-09-11T07:52:49.918 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:49 smithi022 ceph-mon[37330]: 2.a scrub ok 2024-09-11T07:52:50.631 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:52:50] "GET /metrics HTTP/1.1" 200 222452 "" "Prometheus/2.43.0" 2024-09-11T07:52:50.632 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:50 smithi022 ceph-mon[29462]: 6.2 scrub starts 2024-09-11T07:52:50.632 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:50 smithi022 ceph-mon[29462]: 6.2 scrub ok 2024-09-11T07:52:50.632 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:50 smithi022 ceph-mon[29462]: pgmap v1239: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:52:50.632 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:50 smithi022 ceph-mon[37330]: 6.2 scrub starts 2024-09-11T07:52:50.632 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:50 smithi022 ceph-mon[37330]: 6.2 scrub ok 2024-09-11T07:52:50.632 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:50 smithi022 ceph-mon[37330]: pgmap v1239: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:52:50.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:52:50 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug there is no tcmu-runner data available 2024-09-11T07:52:50.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:50 smithi104 ceph-mon[33507]: 6.2 scrub starts 2024-09-11T07:52:50.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:50 smithi104 ceph-mon[33507]: 6.2 scrub ok 2024-09-11T07:52:50.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:50 smithi104 ceph-mon[33507]: pgmap v1239: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:52:50.916 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:52:50] ENGINE Bus STOPPING 2024-09-11T07:52:50.916 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:52:50] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T07:52:50.916 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:52:50] ENGINE Bus STOPPED 2024-09-11T07:52:50.916 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:52:50] ENGINE Bus STARTING 2024-09-11T07:52:51.297 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:52:50] ENGINE Serving on http://:::9283 2024-09-11T07:52:51.298 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:50 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:52:50] ENGINE Bus STARTED 2024-09-11T07:52:51.323 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-11T07:52:51.332 INFO:tasks.cephadm:Teardown begin 2024-09-11T07:52:51.333 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:52:51.361 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:52:51.410 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-11T07:52:51.410 DEBUG:teuthology.orchestra.run.smithi022:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T07:52:51.426 DEBUG:teuthology.orchestra.run.smithi104:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T07:52:51.444 INFO:tasks.cephadm:Stopping all daemons... 2024-09-11T07:52:51.444 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-11T07:52:51.444 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-12a285d8-700e-11ef-bcea-c7b262605968@mon.a 2024-09-11T07:52:51.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[29462]: 5.10 deep-scrub starts 2024-09-11T07:52:51.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[29462]: 5.10 deep-scrub ok 2024-09-11T07:52:51.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[29462]: 6.14 scrub starts 2024-09-11T07:52:51.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[29462]: 6.14 scrub ok 2024-09-11T07:52:51.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[29462]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:52:51.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[29462]: from='client.? v1:172.21.15.22:0/773564965' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-09-11T07:52:51.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[29462]: from='client.26450 ' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-09-11T07:52:51.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[29462]: from='client.26450 ' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-11T07:52:51.667 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:52:51.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:52:51.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:52:51.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[29462]: pgmap v1240: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T07:52:51.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[29462]: pgmap v1241: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:52:51.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[29462]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:52:51.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[29462]: 2.15 scrub starts 2024-09-11T07:52:51.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[29462]: 2.15 scrub ok 2024-09-11T07:52:51.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[29462]: 5.3 scrub starts 2024-09-11T07:52:51.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[29462]: 5.3 scrub ok 2024-09-11T07:52:51.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[29462]: 4.a scrub starts 2024-09-11T07:52:51.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[29462]: 4.a scrub ok 2024-09-11T07:52:51.668 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:51 smithi022 systemd[1]: Stopping Ceph mon.a for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:52:51.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[37330]: 5.10 deep-scrub starts 2024-09-11T07:52:51.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[37330]: 5.10 deep-scrub ok 2024-09-11T07:52:51.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[37330]: 6.14 scrub starts 2024-09-11T07:52:51.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[37330]: 6.14 scrub ok 2024-09-11T07:52:51.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[37330]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:52:51.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[37330]: from='client.? v1:172.21.15.22:0/773564965' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-09-11T07:52:51.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[37330]: from='client.26450 ' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-09-11T07:52:51.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[37330]: from='client.26450 ' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-11T07:52:51.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:52:51.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:52:51.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:52:51.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[37330]: pgmap v1240: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T07:52:51.669 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[37330]: pgmap v1241: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:52:51.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[37330]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:52:51.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[37330]: 2.15 scrub starts 2024-09-11T07:52:51.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[37330]: 2.15 scrub ok 2024-09-11T07:52:51.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[37330]: 5.3 scrub starts 2024-09-11T07:52:51.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[37330]: 5.3 scrub ok 2024-09-11T07:52:51.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[37330]: 4.a scrub starts 2024-09-11T07:52:51.670 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-mon[37330]: 4.a scrub ok 2024-09-11T07:52:51.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:51 smithi104 ceph-mon[33507]: 5.10 deep-scrub starts 2024-09-11T07:52:51.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:51 smithi104 ceph-mon[33507]: 5.10 deep-scrub ok 2024-09-11T07:52:51.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:51 smithi104 ceph-mon[33507]: 6.14 scrub starts 2024-09-11T07:52:51.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:51 smithi104 ceph-mon[33507]: 6.14 scrub ok 2024-09-11T07:52:51.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:51 smithi104 ceph-mon[33507]: from='client.14547 v1:172.21.15.104:0/2548539384' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:52:51.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:51 smithi104 ceph-mon[33507]: from='client.? v1:172.21.15.22:0/773564965' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-09-11T07:52:51.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:51 smithi104 ceph-mon[33507]: from='client.26450 ' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-09-11T07:52:51.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:51 smithi104 ceph-mon[33507]: from='client.26450 ' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-11T07:52:51.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:51 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:52:51.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:51 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:52:51.780 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:51 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:52:51.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:51 smithi104 ceph-mon[33507]: pgmap v1240: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T07:52:51.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:51 smithi104 ceph-mon[33507]: pgmap v1241: 142 pgs: 142 active+clean; 582 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:52:51.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:51 smithi104 ceph-mon[33507]: from='mgr.24494 v1:172.21.15.22:0/69925258' entity='mgr.y' 2024-09-11T07:52:51.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:51 smithi104 ceph-mon[33507]: 2.15 scrub starts 2024-09-11T07:52:51.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:51 smithi104 ceph-mon[33507]: 2.15 scrub ok 2024-09-11T07:52:51.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:51 smithi104 ceph-mon[33507]: 5.3 scrub starts 2024-09-11T07:52:51.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:51 smithi104 ceph-mon[33507]: 5.3 scrub ok 2024-09-11T07:52:51.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:51 smithi104 ceph-mon[33507]: 4.a scrub starts 2024-09-11T07:52:51.781 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:51 smithi104 ceph-mon[33507]: 4.a scrub ok 2024-09-11T07:52:52.087 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-a[29458]: 2024-09-11T07:52:51.783+0000 7fde6281a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:52:52.087 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-a[29458]: 2024-09-11T07:52:51.783+0000 7fde6281a640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-11T07:52:52.087 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:51 smithi022 podman[222747]: 2024-09-11 07:52:51.906700831 +0000 UTC m=+0.244125523 container died e6840d670ba28df616e91626938892c2ae2ab42e6b832943fa4315a6ba29d612 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-11T07:52:52.088 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:52:51] ENGINE Bus STOPPING 2024-09-11T07:52:52.088 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:52:51] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T07:52:52.088 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:52:51] ENGINE Bus STOPPED 2024-09-11T07:52:52.088 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:51 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:52:51] ENGINE Bus STARTING 2024-09-11T07:52:52.088 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:52 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:52:52] ENGINE Serving on http://:::9283 2024-09-11T07:52:52.416 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:52 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: [11/Sep/2024:07:52:52] ENGINE Bus STARTED 2024-09-11T07:52:52.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:52 smithi022 podman[222747]: 2024-09-11 07:52:52.283807637 +0000 UTC m=+0.621232321 container cleanup e6840d670ba28df616e91626938892c2ae2ab42e6b832943fa4315a6ba29d612 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-11T07:52:52.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 11 07:52:52 smithi022 bash[222747]: ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-a 2024-09-11T07:52:52.710 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@mon.a.service' 2024-09-11T07:52:52.749 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:52:52.749 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-11T07:52:52.749 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-11T07:52:52.749 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-12a285d8-700e-11ef-bcea-c7b262605968@mon.c 2024-09-11T07:52:52.916 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:52 smithi022 systemd[1]: Stopping Ceph mon.c for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:52:53.249 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:53 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-c[37326]: 2024-09-11T07:52:53.105+0000 7f0a5de94640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:52:53.250 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:53 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-c[37326]: 2024-09-11T07:52:53.105+0000 7f0a5de94640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-11T07:52:53.538 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:53 smithi022 podman[222880]: 2024-09-11 07:52:53.249051227 +0000 UTC m=+0.272886682 container died 33c9f56728148f766a4dd99e964cff711ea6a149627631940ebe2f2a17955fcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-c, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:52:53.882 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:53 smithi022 podman[222880]: 2024-09-11 07:52:53.639029953 +0000 UTC m=+0.662865406 container cleanup 33c9f56728148f766a4dd99e964cff711ea6a149627631940ebe2f2a17955fcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-c, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-11T07:52:53.882 INFO:journalctl@ceph.mon.c.smithi022.stdout:Sep 11 07:52:53 smithi022 bash[222880]: ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-c 2024-09-11T07:52:54.030 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@mon.c.service' 2024-09-11T07:52:54.067 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:52:54.067 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-11T07:52:54.067 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-11T07:52:54.067 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-12a285d8-700e-11ef-bcea-c7b262605968@mon.b 2024-09-11T07:52:54.279 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:52:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x[35341]: ::ffff:172.21.15.104 - - [11/Sep/2024:07:52:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:52:54.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:54 smithi104 systemd[1]: Stopping Ceph mon.b for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:52:54.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-b[33490]: 2024-09-11T07:52:54.393+0000 7f423ae35640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:52:54.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-b[33490]: 2024-09-11T07:52:54.393+0000 7f423ae35640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-11T07:52:54.779 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:54 smithi104 podman[166344]: 2024-09-11 07:52:54.608762578 +0000 UTC m=+0.328284079 container died a00a636a58c32139f1377c348694d6be406705ca60f2450007b611ee1ad485de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-11T07:52:55.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:55 smithi104 podman[166344]: 2024-09-11 07:52:55.018622815 +0000 UTC m=+0.738144357 container cleanup a00a636a58c32139f1377c348694d6be406705ca60f2450007b611ee1ad485de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-11T07:52:55.279 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 11 07:52:55 smithi104 bash[166344]: ceph-12a285d8-700e-11ef-bcea-c7b262605968-mon-b 2024-09-11T07:52:55.413 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@mon.b.service' 2024-09-11T07:52:55.460 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:52:55.460 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-11T07:52:55.460 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-11T07:52:55.460 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-12a285d8-700e-11ef-bcea-c7b262605968@mgr.y 2024-09-11T07:52:55.811 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:55 smithi022 systemd[1]: Stopping Ceph mgr.y for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:52:56.166 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:55 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:52:55.809+0000 7f0b95143640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:52:56.167 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:55 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y[29688]: 2024-09-11T07:52:55.809+0000 7f0b95143640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-11T07:52:56.167 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:55 smithi022 podman[223001]: 2024-09-11 07:52:55.842094725 +0000 UTC m=+0.174563414 container died 022b1767529aa35be84838926d54af37199fc6839648ee9a70ef725f8231d76e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:52:56.621 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:56 smithi022 podman[223001]: 2024-09-11 07:52:56.36925174 +0000 UTC m=+0.701720470 container cleanup 022b1767529aa35be84838926d54af37199fc6839648ee9a70ef725f8231d76e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-11T07:52:56.622 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:56 smithi022 bash[223001]: ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y 2024-09-11T07:52:56.861 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-11T07:52:56.884 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:56 smithi022 podman[223013]: 2024-09-11 07:52:56.621078377 +0000 UTC m=+0.768448252 container remove 022b1767529aa35be84838926d54af37199fc6839648ee9a70ef725f8231d76e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-y, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:52:56.884 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:56 smithi022 systemd[1]: ceph-12a285d8-700e-11ef-bcea-c7b262605968@mgr.y.service: Deactivated successfully. 2024-09-11T07:52:56.884 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:56 smithi022 systemd[1]: Stopped Ceph mgr.y for 12a285d8-700e-11ef-bcea-c7b262605968. 2024-09-11T07:52:56.884 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Sep 11 07:52:56 smithi022 systemd[1]: ceph-12a285d8-700e-11ef-bcea-c7b262605968@mgr.y.service: Consumed 54.839s CPU time. 2024-09-11T07:52:56.898 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:52:56.898 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-11T07:52:56.898 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-11T07:52:56.898 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-12a285d8-700e-11ef-bcea-c7b262605968@mgr.x 2024-09-11T07:52:57.228 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:52:56 smithi104 systemd[1]: Stopping Ceph mgr.x for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:52:57.525 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:52:57 smithi104 podman[166464]: 2024-09-11 07:52:57.227661359 +0000 UTC m=+0.130867302 container died 5d978c819564a188fc0c090d77ec6ce442946e7b689637082f046e4e5986fb08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-11T07:52:57.779 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:52:57 smithi104 podman[166464]: 2024-09-11 07:52:57.615057228 +0000 UTC m=+0.518263181 container cleanup 5d978c819564a188fc0c090d77ec6ce442946e7b689637082f046e4e5986fb08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-11T07:52:57.779 INFO:journalctl@ceph.mgr.x.smithi104.stdout:Sep 11 07:52:57 smithi104 bash[166464]: ceph-12a285d8-700e-11ef-bcea-c7b262605968-mgr-x 2024-09-11T07:52:58.043 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-11T07:52:58.078 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:52:58.079 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-11T07:52:58.079 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-11T07:52:58.079 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.0 2024-09-11T07:52:58.383 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 11 07:52:58 smithi022 systemd[1]: Stopping Ceph osd.0 for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:52:58.666 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 11 07:52:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-0[41188]: 2024-09-11T07:52:58.381+0000 7f948ba6e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:52:58.666 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 11 07:52:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-0[41188]: 2024-09-11T07:52:58.381+0000 7f948ba6e640 -1 osd.0 534 *** Got signal Terminated *** 2024-09-11T07:52:58.666 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 11 07:52:58 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-0[41188]: 2024-09-11T07:52:58.381+0000 7f948ba6e640 -1 osd.0 534 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:53:04.242 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 11 07:53:03 smithi022 podman[223121]: 2024-09-11 07:53:03.979615293 +0000 UTC m=+5.707841353 container died 670fe8e880882bb3fbc52f1cc4d15d8e677046fee30e19d1f86c04c55f91032a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-11T07:53:04.666 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 11 07:53:04 smithi022 podman[223121]: 2024-09-11 07:53:04.448601819 +0000 UTC m=+6.176827920 container cleanup 670fe8e880882bb3fbc52f1cc4d15d8e677046fee30e19d1f86c04c55f91032a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-11T07:53:04.666 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 11 07:53:04 smithi022 bash[223121]: ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-0 2024-09-11T07:53:04.667 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:53:04 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:53:04.241Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi022.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.22:8443: connect: connection refused" 2024-09-11T07:53:04.667 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:53:04 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:53:04.242Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi104.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.104:8443: connect: connection refused" 2024-09-11T07:53:05.071 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 11 07:53:04 smithi022 podman[223135]: 2024-09-11 07:53:04.716862336 +0000 UTC m=+0.732803990 container remove 670fe8e880882bb3fbc52f1cc4d15d8e677046fee30e19d1f86c04c55f91032a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-11T07:53:05.387 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 11 07:53:05 smithi022 podman[223222]: 2024-09-11 07:53:04.973485932 +0000 UTC m=+0.016808488 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:53:05.387 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 11 07:53:05 smithi022 podman[223222]: 2024-09-11 07:53:05.075870079 +0000 UTC m=+0.119192624 container create 8df7502ae827a98532687befa9abe43a614d320ae5fdaaf4fda52e55fe80bcb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-11T07:53:05.387 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 11 07:53:05 smithi022 podman[223222]: 2024-09-11 07:53:05.303436005 +0000 UTC m=+0.346758551 container init 8df7502ae827a98532687befa9abe43a614d320ae5fdaaf4fda52e55fe80bcb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-11T07:53:05.387 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 11 07:53:05 smithi022 podman[223222]: 2024-09-11 07:53:05.306952856 +0000 UTC m=+0.350275402 container start 8df7502ae827a98532687befa9abe43a614d320ae5fdaaf4fda52e55fe80bcb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-11T07:53:05.388 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 11 07:53:05 smithi022 podman[223222]: 2024-09-11 07:53:05.386691022 +0000 UTC m=+0.430013570 container attach 8df7502ae827a98532687befa9abe43a614d320ae5fdaaf4fda52e55fe80bcb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-11T07:53:05.666 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 11 07:53:05 smithi022 podman[223222]: 2024-09-11 07:53:05.49187611 +0000 UTC m=+0.535198665 container died 8df7502ae827a98532687befa9abe43a614d320ae5fdaaf4fda52e55fe80bcb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T07:53:06.125 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.0.service' 2024-09-11T07:53:06.167 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:53:06.168 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-11T07:53:06.168 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-11T07:53:06.168 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.1 2024-09-11T07:53:06.416 INFO:journalctl@ceph.osd.1.smithi022.stdout:Sep 11 07:53:06 smithi022 systemd[1]: Stopping Ceph osd.1 for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:53:06.916 INFO:journalctl@ceph.osd.1.smithi022.stdout:Sep 11 07:53:06 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-1[47211]: 2024-09-11T07:53:06.527+0000 7fe3da97c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:53:06.916 INFO:journalctl@ceph.osd.1.smithi022.stdout:Sep 11 07:53:06 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-1[47211]: 2024-09-11T07:53:06.527+0000 7fe3da97c640 -1 osd.1 534 *** Got signal Terminated *** 2024-09-11T07:53:06.917 INFO:journalctl@ceph.osd.1.smithi022.stdout:Sep 11 07:53:06 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-1[47211]: 2024-09-11T07:53:06.527+0000 7fe3da97c640 -1 osd.1 534 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:53:12.496 INFO:journalctl@ceph.osd.1.smithi022.stdout:Sep 11 07:53:12 smithi022 podman[223325]: 2024-09-11 07:53:12.177312483 +0000 UTC m=+5.770786932 container died c0c5acfc36cb4cf992a2e1cac93657118a8cf726fa1c524029350e66e00780e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:53:12.838 INFO:journalctl@ceph.osd.1.smithi022.stdout:Sep 11 07:53:12 smithi022 podman[223325]: 2024-09-11 07:53:12.694162132 +0000 UTC m=+6.287636571 container cleanup c0c5acfc36cb4cf992a2e1cac93657118a8cf726fa1c524029350e66e00780e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-11T07:53:12.838 INFO:journalctl@ceph.osd.1.smithi022.stdout:Sep 11 07:53:12 smithi022 bash[223325]: ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-1 2024-09-11T07:53:13.166 INFO:journalctl@ceph.osd.1.smithi022.stdout:Sep 11 07:53:12 smithi022 podman[223342]: 2024-09-11 07:53:12.92903129 +0000 UTC m=+0.741847103 container remove c0c5acfc36cb4cf992a2e1cac93657118a8cf726fa1c524029350e66e00780e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-1, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux ) 2024-09-11T07:53:13.666 INFO:journalctl@ceph.osd.1.smithi022.stdout:Sep 11 07:53:13 smithi022 podman[223443]: 2024-09-11 07:53:13.305106164 +0000 UTC m=+0.023501201 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:53:13.666 INFO:journalctl@ceph.osd.1.smithi022.stdout:Sep 11 07:53:13 smithi022 podman[223443]: 2024-09-11 07:53:13.413238211 +0000 UTC m=+0.131633234 container create f08618cbe85f653c5ed597b50abc9f7ee9c60c8f936fd63bca052edce05eedd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-09-11T07:53:14.167 INFO:journalctl@ceph.osd.1.smithi022.stdout:Sep 11 07:53:13 smithi022 podman[223443]: 2024-09-11 07:53:13.684384129 +0000 UTC m=+0.402779160 container init f08618cbe85f653c5ed597b50abc9f7ee9c60c8f936fd63bca052edce05eedd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-11T07:53:14.167 INFO:journalctl@ceph.osd.1.smithi022.stdout:Sep 11 07:53:13 smithi022 podman[223443]: 2024-09-11 07:53:13.688298627 +0000 UTC m=+0.406693650 container start f08618cbe85f653c5ed597b50abc9f7ee9c60c8f936fd63bca052edce05eedd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-11T07:53:14.167 INFO:journalctl@ceph.osd.1.smithi022.stdout:Sep 11 07:53:13 smithi022 podman[223443]: 2024-09-11 07:53:13.774520397 +0000 UTC m=+0.492915429 container attach f08618cbe85f653c5ed597b50abc9f7ee9c60c8f936fd63bca052edce05eedd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:53:14.167 INFO:journalctl@ceph.osd.1.smithi022.stdout:Sep 11 07:53:13 smithi022 podman[223443]: 2024-09-11 07:53:13.884886776 +0000 UTC m=+0.603281805 container died f08618cbe85f653c5ed597b50abc9f7ee9c60c8f936fd63bca052edce05eedd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-11T07:53:14.570 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:53:14 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:53:14.242Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi022.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.22:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi104.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.104:8443: connect: connection refused" 2024-09-11T07:53:14.571 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:53:14 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:53:14.242Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi022.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.22:8443: connect: connection refused" 2024-09-11T07:53:14.571 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:53:14 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:53:14.243Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi104.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.104:8443: connect: connection refused" 2024-09-11T07:53:14.600 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.1.service' 2024-09-11T07:53:14.642 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:53:14.642 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-11T07:53:14.642 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-11T07:53:14.642 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.2 2024-09-11T07:53:14.916 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:53:14 smithi022 systemd[1]: Stopping Ceph osd.2 for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:53:15.416 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:53:14 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2[214840]: 2024-09-11T07:53:14.981+0000 7fa7dbe55640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:53:15.417 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:53:14 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2[214840]: 2024-09-11T07:53:14.981+0000 7fa7dbe55640 -1 osd.2 534 *** Got signal Terminated *** 2024-09-11T07:53:15.417 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:53:14 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2[214840]: 2024-09-11T07:53:14.981+0000 7fa7dbe55640 -1 osd.2 534 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:53:20.624 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:53:20 smithi022 podman[223546]: 2024-09-11 07:53:20.331252612 +0000 UTC m=+5.463626978 container died 5fe27ec66de3ae28573a834f75c0335ba7d83d318860622206c24ab94f8b3c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-09-11T07:53:20.916 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:53:20 smithi022 podman[223546]: 2024-09-11 07:53:20.816462858 +0000 UTC m=+5.948837219 container cleanup 5fe27ec66de3ae28573a834f75c0335ba7d83d318860622206c24ab94f8b3c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:53:20.917 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:53:20 smithi022 bash[223546]: ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2 2024-09-11T07:53:21.416 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:53:21 smithi022 podman[223563]: 2024-09-11 07:53:21.00961625 +0000 UTC m=+0.680593378 container remove 5fe27ec66de3ae28573a834f75c0335ba7d83d318860622206c24ab94f8b3c31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:53:21.713 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:53:21 smithi022 podman[223653]: 2024-09-11 07:53:21.34778772 +0000 UTC m=+0.016207071 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:53:21.713 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:53:21 smithi022 podman[223653]: 2024-09-11 07:53:21.460428884 +0000 UTC m=+0.128848262 container create bb57c99ef1a1371a861915f32f597634fb6246c5ff16215ea59a3583329815f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-11T07:53:21.713 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:53:21 smithi022 podman[223653]: 2024-09-11 07:53:21.712773019 +0000 UTC m=+0.381192358 container init bb57c99ef1a1371a861915f32f597634fb6246c5ff16215ea59a3583329815f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-11T07:53:22.166 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:53:21 smithi022 podman[223653]: 2024-09-11 07:53:21.71619857 +0000 UTC m=+0.384617909 container start bb57c99ef1a1371a861915f32f597634fb6246c5ff16215ea59a3583329815f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS) 2024-09-11T07:53:22.166 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:53:21 smithi022 podman[223653]: 2024-09-11 07:53:21.837592916 +0000 UTC m=+0.506012262 container attach bb57c99ef1a1371a861915f32f597634fb6246c5ff16215ea59a3583329815f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:53:22.167 INFO:journalctl@ceph.osd.2.smithi022.stdout:Sep 11 07:53:21 smithi022 podman[223653]: 2024-09-11 07:53:21.894317212 +0000 UTC m=+0.562736561 container died bb57c99ef1a1371a861915f32f597634fb6246c5ff16215ea59a3583329815f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:53:22.601 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.2.service' 2024-09-11T07:53:22.644 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:53:22.645 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-11T07:53:22.645 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-11T07:53:22.645 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.3 2024-09-11T07:53:22.917 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 11 07:53:22 smithi022 systemd[1]: Stopping Ceph osd.3 for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:53:23.416 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 11 07:53:23 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-3[58935]: 2024-09-11T07:53:23.002+0000 7eff35115640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:53:23.416 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 11 07:53:23 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-3[58935]: 2024-09-11T07:53:23.003+0000 7eff35115640 -1 osd.3 534 *** Got signal Terminated *** 2024-09-11T07:53:23.417 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 11 07:53:23 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-3[58935]: 2024-09-11T07:53:23.003+0000 7eff35115640 -1 osd.3 534 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:53:24.666 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:53:24 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:53:24.242Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi022.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.22:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi104.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.104:8443: connect: connection refused" 2024-09-11T07:53:24.667 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:53:24 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:53:24.243Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi022.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.22:8443: connect: connection refused" 2024-09-11T07:53:24.667 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:53:24 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:53:24.243Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi104.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.104:8443: connect: connection refused" 2024-09-11T07:53:26.029 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:25 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:25.666+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.060142+0000 front 2024-09-11T07:53:01.060249+0000 (oldest deadline 2024-09-11T07:53:25.160079+0000) 2024-09-11T07:53:27.029 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:26 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:26.702+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.060142+0000 front 2024-09-11T07:53:01.060249+0000 (oldest deadline 2024-09-11T07:53:25.160079+0000) 2024-09-11T07:53:27.030 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:26 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:26.636+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.531177+0000 front 2024-09-11T07:53:01.531220+0000 (oldest deadline 2024-09-11T07:53:26.231172+0000) 2024-09-11T07:53:27.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:26 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:26.841+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:27.916 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 11 07:53:27 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-3[58935]: 2024-09-11T07:53:27.587+0000 7eff2ef28640 -1 osd.3 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.186507+0000 front 2024-09-11T07:53:02.186547+0000 (oldest deadline 2024-09-11T07:53:27.486668+0000) 2024-09-11T07:53:28.029 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:27 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:27.666+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.060142+0000 front 2024-09-11T07:53:01.060249+0000 (oldest deadline 2024-09-11T07:53:25.160079+0000) 2024-09-11T07:53:28.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:27 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:27.794+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:28.030 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:27 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:27.627+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.531177+0000 front 2024-09-11T07:53:01.531220+0000 (oldest deadline 2024-09-11T07:53:26.231172+0000) 2024-09-11T07:53:28.529 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 11 07:53:28 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-4[38402]: 2024-09-11T07:53:28.383+0000 7fadc3104640 -1 osd.4 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.193873+0000 front 2024-09-11T07:53:02.193796+0000 (oldest deadline 2024-09-11T07:53:28.093765+0000) 2024-09-11T07:53:28.746 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 11 07:53:28 smithi022 podman[223756]: 2024-09-11 07:53:28.443294969 +0000 UTC m=+5.576485453 container died 14b6e45c6e3687e5a8e2f0d366eb7e55eee9f1e4ecdb03fb16b9f25a036f5c4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-3, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-11T07:53:29.029 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:28 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:28.704+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.060142+0000 front 2024-09-11T07:53:01.060249+0000 (oldest deadline 2024-09-11T07:53:25.160079+0000) 2024-09-11T07:53:29.029 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:28 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:28.644+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.531177+0000 front 2024-09-11T07:53:01.531220+0000 (oldest deadline 2024-09-11T07:53:26.231172+0000) 2024-09-11T07:53:29.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:28 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:28.754+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:29.166 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 11 07:53:28 smithi022 podman[223756]: 2024-09-11 07:53:28.927182535 +0000 UTC m=+6.060373008 container cleanup 14b6e45c6e3687e5a8e2f0d366eb7e55eee9f1e4ecdb03fb16b9f25a036f5c4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:53:29.166 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 11 07:53:28 smithi022 bash[223756]: ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-3 2024-09-11T07:53:29.631 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 11 07:53:29 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-4[38402]: 2024-09-11T07:53:29.335+0000 7fadc3104640 -1 osd.4 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.193873+0000 front 2024-09-11T07:53:02.193796+0000 (oldest deadline 2024-09-11T07:53:28.093765+0000) 2024-09-11T07:53:29.666 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 11 07:53:29 smithi022 podman[223772]: 2024-09-11 07:53:29.287248193 +0000 UTC m=+0.830133750 container remove 14b6e45c6e3687e5a8e2f0d366eb7e55eee9f1e4ecdb03fb16b9f25a036f5c4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T07:53:30.024 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 11 07:53:29 smithi022 podman[223862]: 2024-09-11 07:53:29.704705813 +0000 UTC m=+0.112068744 container create c0ffbb3b8c65dad6d5172e96db99782d609dc22ed3a40c98b3883ea46bb278c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:53:30.024 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 11 07:53:29 smithi022 podman[223862]: 2024-09-11 07:53:29.60875804 +0000 UTC m=+0.016120978 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:53:30.025 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 11 07:53:29 smithi022 podman[223862]: 2024-09-11 07:53:29.940273939 +0000 UTC m=+0.347636869 container init c0ffbb3b8c65dad6d5172e96db99782d609dc22ed3a40c98b3883ea46bb278c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-11T07:53:30.025 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 11 07:53:29 smithi022 podman[223862]: 2024-09-11 07:53:29.943643236 +0000 UTC m=+0.351006170 container start c0ffbb3b8c65dad6d5172e96db99782d609dc22ed3a40c98b3883ea46bb278c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-11T07:53:30.025 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 11 07:53:30 smithi022 podman[223862]: 2024-09-11 07:53:30.023584893 +0000 UTC m=+0.430947823 container attach c0ffbb3b8c65dad6d5172e96db99782d609dc22ed3a40c98b3883ea46bb278c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-11T07:53:30.029 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:29 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:29.732+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.060142+0000 front 2024-09-11T07:53:01.060249+0000 (oldest deadline 2024-09-11T07:53:25.160079+0000) 2024-09-11T07:53:30.029 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:29 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:29.630+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.531177+0000 front 2024-09-11T07:53:01.531220+0000 (oldest deadline 2024-09-11T07:53:26.231172+0000) 2024-09-11T07:53:30.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:29 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:29.764+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:30.410 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 11 07:53:30 smithi022 podman[223862]: 2024-09-11 07:53:30.122127244 +0000 UTC m=+0.529490176 container died c0ffbb3b8c65dad6d5172e96db99782d609dc22ed3a40c98b3883ea46bb278c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-11T07:53:30.618 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 11 07:53:30 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-4[38402]: 2024-09-11T07:53:30.363+0000 7fadc3104640 -1 osd.4 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.193873+0000 front 2024-09-11T07:53:02.193796+0000 (oldest deadline 2024-09-11T07:53:28.093765+0000) 2024-09-11T07:53:30.706 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.3.service' 2024-09-11T07:53:30.751 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:53:30.751 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-11T07:53:30.751 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-11T07:53:30.752 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.4 2024-09-11T07:53:31.029 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 11 07:53:30 smithi104 systemd[1]: Stopping Ceph osd.4 for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:53:31.030 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:30 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:30.684+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.060142+0000 front 2024-09-11T07:53:01.060249+0000 (oldest deadline 2024-09-11T07:53:25.160079+0000) 2024-09-11T07:53:31.030 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:30 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:30.616+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.531177+0000 front 2024-09-11T07:53:01.531220+0000 (oldest deadline 2024-09-11T07:53:26.231172+0000) 2024-09-11T07:53:31.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:30 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:30.749+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:31.331 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 11 07:53:31 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-4[38402]: 2024-09-11T07:53:31.072+0000 7fadc8af0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:53:31.331 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 11 07:53:31 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-4[38402]: 2024-09-11T07:53:31.072+0000 7fadc8af0640 -1 osd.4 534 *** Got signal Terminated *** 2024-09-11T07:53:31.331 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 11 07:53:31 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-4[38402]: 2024-09-11T07:53:31.072+0000 7fadc8af0640 -1 osd.4 534 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:53:31.331 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 11 07:53:31 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-4[38402]: 2024-09-11T07:53:31.330+0000 7fadc3104640 -1 osd.4 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.193873+0000 front 2024-09-11T07:53:02.193796+0000 (oldest deadline 2024-09-11T07:53:28.093765+0000) 2024-09-11T07:53:32.029 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:31 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:31.662+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.531177+0000 front 2024-09-11T07:53:01.531220+0000 (oldest deadline 2024-09-11T07:53:26.231172+0000) 2024-09-11T07:53:32.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:31 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:31.752+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:32.030 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:31 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:31.724+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.060142+0000 front 2024-09-11T07:53:01.060249+0000 (oldest deadline 2024-09-11T07:53:25.160079+0000) 2024-09-11T07:53:32.706 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 11 07:53:32 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-4[38402]: 2024-09-11T07:53:32.292+0000 7fadc3104640 -1 osd.4 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.193873+0000 front 2024-09-11T07:53:02.193796+0000 (oldest deadline 2024-09-11T07:53:28.093765+0000) 2024-09-11T07:53:33.029 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:32 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:32.763+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.060142+0000 front 2024-09-11T07:53:01.060249+0000 (oldest deadline 2024-09-11T07:53:25.160079+0000) 2024-09-11T07:53:33.029 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:32 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:32.705+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.531177+0000 front 2024-09-11T07:53:01.531220+0000 (oldest deadline 2024-09-11T07:53:26.231172+0000) 2024-09-11T07:53:33.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:32 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:32.744+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:33.701 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 11 07:53:33 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-4[38402]: 2024-09-11T07:53:33.298+0000 7fadc3104640 -1 osd.4 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.193873+0000 front 2024-09-11T07:53:02.193796+0000 (oldest deadline 2024-09-11T07:53:28.093765+0000) 2024-09-11T07:53:34.029 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:33 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:33.751+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.060142+0000 front 2024-09-11T07:53:01.060249+0000 (oldest deadline 2024-09-11T07:53:25.160079+0000) 2024-09-11T07:53:34.029 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:33 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:33.700+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.531177+0000 front 2024-09-11T07:53:01.531220+0000 (oldest deadline 2024-09-11T07:53:26.231172+0000) 2024-09-11T07:53:34.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:33 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:33.767+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:34.667 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:53:34 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:53:34.243Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi022.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.22:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi104.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.104:8443: connect: connection refused" 2024-09-11T07:53:34.667 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:53:34 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:53:34.244Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi022.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.22:8443: connect: connection refused" 2024-09-11T07:53:34.667 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:53:34 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:53:34.244Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi104.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.104:8443: connect: connection refused" 2024-09-11T07:53:34.706 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 11 07:53:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-4[38402]: 2024-09-11T07:53:34.343+0000 7fadc3104640 -1 osd.4 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.193873+0000 front 2024-09-11T07:53:02.193796+0000 (oldest deadline 2024-09-11T07:53:28.093765+0000) 2024-09-11T07:53:34.706 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 11 07:53:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-4[38402]: 2024-09-11T07:53:34.343+0000 7fadc3104640 -1 osd.4 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:08.094281+0000 front 2024-09-11T07:53:08.094403+0000 (oldest deadline 2024-09-11T07:53:33.394243+0000) 2024-09-11T07:53:35.029 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:34.726+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.060142+0000 front 2024-09-11T07:53:01.060249+0000 (oldest deadline 2024-09-11T07:53:25.160079+0000) 2024-09-11T07:53:35.029 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:34.703+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.531177+0000 front 2024-09-11T07:53:01.531220+0000 (oldest deadline 2024-09-11T07:53:26.231172+0000) 2024-09-11T07:53:35.029 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:34.703+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.432503+0000 front 2024-09-11T07:53:11.432433+0000 (oldest deadline 2024-09-11T07:53:34.332360+0000) 2024-09-11T07:53:35.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:34 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:34.727+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:35.678 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 11 07:53:35 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-4[38402]: 2024-09-11T07:53:35.316+0000 7fadc3104640 -1 osd.4 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.193873+0000 front 2024-09-11T07:53:02.193796+0000 (oldest deadline 2024-09-11T07:53:28.093765+0000) 2024-09-11T07:53:35.678 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 11 07:53:35 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-4[38402]: 2024-09-11T07:53:35.316+0000 7fadc3104640 -1 osd.4 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:08.094281+0000 front 2024-09-11T07:53:08.094403+0000 (oldest deadline 2024-09-11T07:53:33.394243+0000) 2024-09-11T07:53:36.029 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:35 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:35.676+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.060142+0000 front 2024-09-11T07:53:01.060249+0000 (oldest deadline 2024-09-11T07:53:25.160079+0000) 2024-09-11T07:53:36.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:35 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:35.688+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:36.029 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:35 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:35.732+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.531177+0000 front 2024-09-11T07:53:01.531220+0000 (oldest deadline 2024-09-11T07:53:26.231172+0000) 2024-09-11T07:53:36.030 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:35 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:35.732+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.432503+0000 front 2024-09-11T07:53:11.432433+0000 (oldest deadline 2024-09-11T07:53:34.332360+0000) 2024-09-11T07:53:36.984 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 11 07:53:36 smithi104 podman[166599]: 2024-09-11 07:53:36.692186238 +0000 UTC m=+5.743624489 container died 7f38cbbc482ec764bdd59bc99d2bf9d41d10ffbeb92ec7c81d37f4edbe74db59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-4, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:53:36.985 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:36 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:36.704+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.060142+0000 front 2024-09-11T07:53:01.060249+0000 (oldest deadline 2024-09-11T07:53:25.160079+0000) 2024-09-11T07:53:36.985 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:36 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:36.704+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:10.961358+0000 front 2024-09-11T07:53:10.961577+0000 (oldest deadline 2024-09-11T07:53:36.261419+0000) 2024-09-11T07:53:36.985 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:36 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:36.779+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.531177+0000 front 2024-09-11T07:53:01.531220+0000 (oldest deadline 2024-09-11T07:53:26.231172+0000) 2024-09-11T07:53:36.986 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:36 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:36.779+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.432503+0000 front 2024-09-11T07:53:11.432433+0000 (oldest deadline 2024-09-11T07:53:34.332360+0000) 2024-09-11T07:53:36.986 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:36 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:36.721+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:36.986 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:36 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:36.721+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.266482+0000 front 2024-09-11T07:53:11.266496+0000 (oldest deadline 2024-09-11T07:53:35.966394+0000) 2024-09-11T07:53:37.279 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 11 07:53:37 smithi104 podman[166599]: 2024-09-11 07:53:37.063999738 +0000 UTC m=+6.115437992 container cleanup 7f38cbbc482ec764bdd59bc99d2bf9d41d10ffbeb92ec7c81d37f4edbe74db59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-11T07:53:37.279 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 11 07:53:37 smithi104 bash[166599]: ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-4 2024-09-11T07:53:37.661 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 11 07:53:37 smithi104 podman[166615]: 2024-09-11 07:53:37.374011512 +0000 UTC m=+0.672737076 container remove 7f38cbbc482ec764bdd59bc99d2bf9d41d10ffbeb92ec7c81d37f4edbe74db59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-11T07:53:38.027 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:37 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:37.659+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.060142+0000 front 2024-09-11T07:53:01.060249+0000 (oldest deadline 2024-09-11T07:53:25.160079+0000) 2024-09-11T07:53:38.028 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:37 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:37.659+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:10.961358+0000 front 2024-09-11T07:53:10.961577+0000 (oldest deadline 2024-09-11T07:53:36.261419+0000) 2024-09-11T07:53:38.028 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:37 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:37.813+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.531177+0000 front 2024-09-11T07:53:01.531220+0000 (oldest deadline 2024-09-11T07:53:26.231172+0000) 2024-09-11T07:53:38.028 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:37 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:37.813+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.432503+0000 front 2024-09-11T07:53:11.432433+0000 (oldest deadline 2024-09-11T07:53:34.332360+0000) 2024-09-11T07:53:38.028 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:37 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:37.680+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:38.028 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:37 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:37.680+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.266482+0000 front 2024-09-11T07:53:11.266496+0000 (oldest deadline 2024-09-11T07:53:35.966394+0000) 2024-09-11T07:53:38.029 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 11 07:53:37 smithi104 podman[166704]: 2024-09-11 07:53:37.626856174 +0000 UTC m=+0.016202506 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:53:38.029 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 11 07:53:37 smithi104 podman[166704]: 2024-09-11 07:53:37.758202296 +0000 UTC m=+0.147548623 container create abed557207ab165283754708042b6b487b509cf24b7b5fa84eb45cff7941dd5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-11T07:53:38.279 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 11 07:53:38 smithi104 podman[166704]: 2024-09-11 07:53:38.027185712 +0000 UTC m=+0.416532042 container init abed557207ab165283754708042b6b487b509cf24b7b5fa84eb45cff7941dd5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:53:38.279 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 11 07:53:38 smithi104 podman[166704]: 2024-09-11 07:53:38.030796823 +0000 UTC m=+0.420143160 container start abed557207ab165283754708042b6b487b509cf24b7b5fa84eb45cff7941dd5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-11T07:53:38.279 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 11 07:53:38 smithi104 podman[166704]: 2024-09-11 07:53:38.118845622 +0000 UTC m=+0.508191948 container attach abed557207ab165283754708042b6b487b509cf24b7b5fa84eb45cff7941dd5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-4-deactivate, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-11T07:53:38.279 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 11 07:53:38 smithi104 podman[166704]: 2024-09-11 07:53:38.255031054 +0000 UTC m=+0.644377382 container died abed557207ab165283754708042b6b487b509cf24b7b5fa84eb45cff7941dd5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-11T07:53:38.856 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:38 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:38.628+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.060142+0000 front 2024-09-11T07:53:01.060249+0000 (oldest deadline 2024-09-11T07:53:25.160079+0000) 2024-09-11T07:53:38.856 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:38 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:38.628+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:10.961358+0000 front 2024-09-11T07:53:10.961577+0000 (oldest deadline 2024-09-11T07:53:36.261419+0000) 2024-09-11T07:53:38.857 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:38 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:38.634+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:38.857 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:38 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:38.634+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.266482+0000 front 2024-09-11T07:53:11.266496+0000 (oldest deadline 2024-09-11T07:53:35.966394+0000) 2024-09-11T07:53:38.857 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:38 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:38.807+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.531177+0000 front 2024-09-11T07:53:01.531220+0000 (oldest deadline 2024-09-11T07:53:26.231172+0000) 2024-09-11T07:53:38.857 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:38 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:38.807+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.432503+0000 front 2024-09-11T07:53:11.432433+0000 (oldest deadline 2024-09-11T07:53:34.332360+0000) 2024-09-11T07:53:38.876 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.4.service' 2024-09-11T07:53:38.914 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:53:38.915 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-11T07:53:38.915 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-11T07:53:38.915 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.5 2024-09-11T07:53:39.243 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:38 smithi104 systemd[1]: Stopping Ceph osd.5 for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:53:39.529 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:39 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:39.242+0000 7fafea195640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:53:39.529 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:39 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:39.242+0000 7fafea195640 -1 osd.5 534 *** Got signal Terminated *** 2024-09-11T07:53:39.529 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:39 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:39.242+0000 7fafea195640 -1 osd.5 534 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:53:40.029 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:39 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:39.626+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.060142+0000 front 2024-09-11T07:53:01.060249+0000 (oldest deadline 2024-09-11T07:53:25.160079+0000) 2024-09-11T07:53:40.029 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:39 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:39.626+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:10.961358+0000 front 2024-09-11T07:53:10.961577+0000 (oldest deadline 2024-09-11T07:53:36.261419+0000) 2024-09-11T07:53:40.030 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:39 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:39.773+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.531177+0000 front 2024-09-11T07:53:01.531220+0000 (oldest deadline 2024-09-11T07:53:26.231172+0000) 2024-09-11T07:53:40.030 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:39 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:39.773+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.432503+0000 front 2024-09-11T07:53:11.432433+0000 (oldest deadline 2024-09-11T07:53:34.332360+0000) 2024-09-11T07:53:40.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:39 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:39.665+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:40.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:39 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:39.665+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.266482+0000 front 2024-09-11T07:53:11.266496+0000 (oldest deadline 2024-09-11T07:53:35.966394+0000) 2024-09-11T07:53:41.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:40 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:40.629+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:41.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:40 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:40.629+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.266482+0000 front 2024-09-11T07:53:11.266496+0000 (oldest deadline 2024-09-11T07:53:35.966394+0000) 2024-09-11T07:53:41.030 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:40 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:40.583+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.060142+0000 front 2024-09-11T07:53:01.060249+0000 (oldest deadline 2024-09-11T07:53:25.160079+0000) 2024-09-11T07:53:41.030 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:40 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:40.583+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:10.961358+0000 front 2024-09-11T07:53:10.961577+0000 (oldest deadline 2024-09-11T07:53:36.261419+0000) 2024-09-11T07:53:41.030 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:40 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:40.789+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.531177+0000 front 2024-09-11T07:53:01.531220+0000 (oldest deadline 2024-09-11T07:53:26.231172+0000) 2024-09-11T07:53:41.030 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:40 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:40.789+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.432503+0000 front 2024-09-11T07:53:11.432433+0000 (oldest deadline 2024-09-11T07:53:34.332360+0000) 2024-09-11T07:53:42.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:41 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:41.616+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:42.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:41 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:41.616+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.266482+0000 front 2024-09-11T07:53:11.266496+0000 (oldest deadline 2024-09-11T07:53:35.966394+0000) 2024-09-11T07:53:42.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:41 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:41.616+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:15.966995+0000 front 2024-09-11T07:53:15.966926+0000 (oldest deadline 2024-09-11T07:53:41.266809+0000) 2024-09-11T07:53:42.030 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:41 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:41.761+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.531177+0000 front 2024-09-11T07:53:01.531220+0000 (oldest deadline 2024-09-11T07:53:26.231172+0000) 2024-09-11T07:53:42.030 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:41 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:41.761+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.432503+0000 front 2024-09-11T07:53:11.432433+0000 (oldest deadline 2024-09-11T07:53:34.332360+0000) 2024-09-11T07:53:42.030 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:41 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:41.608+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.060142+0000 front 2024-09-11T07:53:01.060249+0000 (oldest deadline 2024-09-11T07:53:25.160079+0000) 2024-09-11T07:53:42.030 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:41 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:41.608+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:10.961358+0000 front 2024-09-11T07:53:10.961577+0000 (oldest deadline 2024-09-11T07:53:36.261419+0000) 2024-09-11T07:53:42.030 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:41 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:41.608+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:17.962288+0000 front 2024-09-11T07:53:17.962406+0000 (oldest deadline 2024-09-11T07:53:41.462271+0000) 2024-09-11T07:53:43.029 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:42 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:42.803+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.531177+0000 front 2024-09-11T07:53:01.531220+0000 (oldest deadline 2024-09-11T07:53:26.231172+0000) 2024-09-11T07:53:43.029 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:42 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:42.803+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.432503+0000 front 2024-09-11T07:53:11.432433+0000 (oldest deadline 2024-09-11T07:53:34.332360+0000) 2024-09-11T07:53:43.030 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:42 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:42.585+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.060142+0000 front 2024-09-11T07:53:01.060249+0000 (oldest deadline 2024-09-11T07:53:25.160079+0000) 2024-09-11T07:53:43.030 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:42 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:42.585+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:10.961358+0000 front 2024-09-11T07:53:10.961577+0000 (oldest deadline 2024-09-11T07:53:36.261419+0000) 2024-09-11T07:53:43.030 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:42 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:42.585+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:17.962288+0000 front 2024-09-11T07:53:17.962406+0000 (oldest deadline 2024-09-11T07:53:41.462271+0000) 2024-09-11T07:53:43.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:42 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:42.659+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:43.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:42 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:42.659+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.266482+0000 front 2024-09-11T07:53:11.266496+0000 (oldest deadline 2024-09-11T07:53:35.966394+0000) 2024-09-11T07:53:43.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:42 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:42.659+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:15.966995+0000 front 2024-09-11T07:53:15.966926+0000 (oldest deadline 2024-09-11T07:53:41.266809+0000) 2024-09-11T07:53:44.029 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:43 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:43.762+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.531177+0000 front 2024-09-11T07:53:01.531220+0000 (oldest deadline 2024-09-11T07:53:26.231172+0000) 2024-09-11T07:53:44.029 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:43 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:43.762+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.432503+0000 front 2024-09-11T07:53:11.432433+0000 (oldest deadline 2024-09-11T07:53:34.332360+0000) 2024-09-11T07:53:44.030 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:43 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:43.575+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.060142+0000 front 2024-09-11T07:53:01.060249+0000 (oldest deadline 2024-09-11T07:53:25.160079+0000) 2024-09-11T07:53:44.030 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:43 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:43.575+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:10.961358+0000 front 2024-09-11T07:53:10.961577+0000 (oldest deadline 2024-09-11T07:53:36.261419+0000) 2024-09-11T07:53:44.030 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:43 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5[44304]: 2024-09-11T07:53:43.576+0000 7fafe47a9640 -1 osd.5 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:17.962288+0000 front 2024-09-11T07:53:17.962406+0000 (oldest deadline 2024-09-11T07:53:41.462271+0000) 2024-09-11T07:53:44.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:43 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:43.673+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:44.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:43 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:43.673+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.266482+0000 front 2024-09-11T07:53:11.266496+0000 (oldest deadline 2024-09-11T07:53:35.966394+0000) 2024-09-11T07:53:44.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:43 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:43.673+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:15.966995+0000 front 2024-09-11T07:53:15.966926+0000 (oldest deadline 2024-09-11T07:53:41.266809+0000) 2024-09-11T07:53:44.578 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:53:44 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:53:44.244Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi104.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.104:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi022.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.22:8443: connect: connection refused" 2024-09-11T07:53:44.578 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:53:44 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:53:44.245Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi022.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.22:8443: connect: connection refused" 2024-09-11T07:53:44.578 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:53:44 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:53:44.245Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi104.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.104:8443: connect: connection refused" 2024-09-11T07:53:45.029 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:44 smithi104 podman[166808]: 2024-09-11 07:53:44.822978612 +0000 UTC m=+5.686533450 container died 26baefbcce3ae8b6470a85dac914a447f74365c54f8254e3cc48ad9dc5397c2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:53:45.162 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:44.756+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.531177+0000 front 2024-09-11T07:53:01.531220+0000 (oldest deadline 2024-09-11T07:53:26.231172+0000) 2024-09-11T07:53:45.163 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:44.756+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.432503+0000 front 2024-09-11T07:53:11.432433+0000 (oldest deadline 2024-09-11T07:53:34.332360+0000) 2024-09-11T07:53:45.163 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:44.664+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:45.163 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:44.664+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.266482+0000 front 2024-09-11T07:53:11.266496+0000 (oldest deadline 2024-09-11T07:53:35.966394+0000) 2024-09-11T07:53:45.163 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:44 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:44.664+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:15.966995+0000 front 2024-09-11T07:53:15.966926+0000 (oldest deadline 2024-09-11T07:53:41.266809+0000) 2024-09-11T07:53:45.493 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:45 smithi104 podman[166808]: 2024-09-11 07:53:45.199773475 +0000 UTC m=+6.063328355 container cleanup 26baefbcce3ae8b6470a85dac914a447f74365c54f8254e3cc48ad9dc5397c2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:53:45.493 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:45 smithi104 bash[166808]: ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5 2024-09-11T07:53:45.779 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:45 smithi104 podman[166824]: 2024-09-11 07:53:45.492541003 +0000 UTC m=+0.658566128 container remove 26baefbcce3ae8b6470a85dac914a447f74365c54f8254e3cc48ad9dc5397c2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-11T07:53:45.779 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:45 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:45.736+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.531177+0000 front 2024-09-11T07:53:01.531220+0000 (oldest deadline 2024-09-11T07:53:26.231172+0000) 2024-09-11T07:53:45.780 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:45 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:45.736+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.432503+0000 front 2024-09-11T07:53:11.432433+0000 (oldest deadline 2024-09-11T07:53:34.332360+0000) 2024-09-11T07:53:45.780 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:45 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:45.653+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:45.780 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:45 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:45.653+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.266482+0000 front 2024-09-11T07:53:11.266496+0000 (oldest deadline 2024-09-11T07:53:35.966394+0000) 2024-09-11T07:53:45.780 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:45 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:45.653+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:15.966995+0000 front 2024-09-11T07:53:15.966926+0000 (oldest deadline 2024-09-11T07:53:41.266809+0000) 2024-09-11T07:53:46.063 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:45 smithi104 podman[166912]: 2024-09-11 07:53:45.701413417 +0000 UTC m=+0.015673915 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:53:46.063 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:45 smithi104 podman[166912]: 2024-09-11 07:53:45.835622016 +0000 UTC m=+0.149882512 container create d66b05dbac8743dcfef96702d70f4797e606e186669f8a44c2028204bb709b5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T07:53:46.529 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:46 smithi104 podman[166912]: 2024-09-11 07:53:46.062942647 +0000 UTC m=+0.377203145 container init d66b05dbac8743dcfef96702d70f4797e606e186669f8a44c2028204bb709b5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-11T07:53:46.529 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:46 smithi104 podman[166912]: 2024-09-11 07:53:46.066251352 +0000 UTC m=+0.380511857 container start d66b05dbac8743dcfef96702d70f4797e606e186669f8a44c2028204bb709b5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-11T07:53:46.529 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:46 smithi104 podman[166912]: 2024-09-11 07:53:46.154468696 +0000 UTC m=+0.468729215 container attach d66b05dbac8743dcfef96702d70f4797e606e186669f8a44c2028204bb709b5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-11T07:53:46.529 INFO:journalctl@ceph.osd.5.smithi104.stdout:Sep 11 07:53:46 smithi104 podman[166912]: 2024-09-11 07:53:46.250359903 +0000 UTC m=+0.564620401 container died d66b05dbac8743dcfef96702d70f4797e606e186669f8a44c2028204bb709b5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-5-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-11T07:53:46.800 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:46 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:46.723+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.531177+0000 front 2024-09-11T07:53:01.531220+0000 (oldest deadline 2024-09-11T07:53:26.231172+0000) 2024-09-11T07:53:46.800 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:46 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:46.723+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.432503+0000 front 2024-09-11T07:53:11.432433+0000 (oldest deadline 2024-09-11T07:53:34.332360+0000) 2024-09-11T07:53:46.800 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:46 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:46.723+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:15.433501+0000 front 2024-09-11T07:53:15.433561+0000 (oldest deadline 2024-09-11T07:53:46.033961+0000) 2024-09-11T07:53:46.801 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:46 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:46.698+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:46.801 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:46 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:46.698+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.266482+0000 front 2024-09-11T07:53:11.266496+0000 (oldest deadline 2024-09-11T07:53:35.966394+0000) 2024-09-11T07:53:46.801 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:46 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:46.698+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:15.966995+0000 front 2024-09-11T07:53:15.966926+0000 (oldest deadline 2024-09-11T07:53:41.266809+0000) 2024-09-11T07:53:46.825 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.5.service' 2024-09-11T07:53:46.865 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:53:46.865 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-11T07:53:46.865 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-11T07:53:46.865 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.6 2024-09-11T07:53:47.187 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:46 smithi104 systemd[1]: Stopping Ceph osd.6 for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:53:47.187 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:47 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:47.185+0000 7f6564c51640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:53:47.529 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:47 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:47.185+0000 7f6564c51640 -1 osd.6 534 *** Got signal Terminated *** 2024-09-11T07:53:47.529 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:47 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:47.185+0000 7f6564c51640 -1 osd.6 534 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:53:48.029 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:47 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:47.772+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.531177+0000 front 2024-09-11T07:53:01.531220+0000 (oldest deadline 2024-09-11T07:53:26.231172+0000) 2024-09-11T07:53:48.029 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:47 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:47.772+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.432503+0000 front 2024-09-11T07:53:11.432433+0000 (oldest deadline 2024-09-11T07:53:34.332360+0000) 2024-09-11T07:53:48.029 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:47 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:47.772+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:15.433501+0000 front 2024-09-11T07:53:15.433561+0000 (oldest deadline 2024-09-11T07:53:46.033961+0000) 2024-09-11T07:53:48.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:47 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:47.665+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:48.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:47 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:47.666+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.266482+0000 front 2024-09-11T07:53:11.266496+0000 (oldest deadline 2024-09-11T07:53:35.966394+0000) 2024-09-11T07:53:48.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:47 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:47.666+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:15.966995+0000 front 2024-09-11T07:53:15.966926+0000 (oldest deadline 2024-09-11T07:53:41.266809+0000) 2024-09-11T07:53:49.029 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:48 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:48.803+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.531177+0000 front 2024-09-11T07:53:01.531220+0000 (oldest deadline 2024-09-11T07:53:26.231172+0000) 2024-09-11T07:53:49.029 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:48 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:48.803+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.432503+0000 front 2024-09-11T07:53:11.432433+0000 (oldest deadline 2024-09-11T07:53:34.332360+0000) 2024-09-11T07:53:49.029 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:48 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:48.803+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:15.433501+0000 front 2024-09-11T07:53:15.433561+0000 (oldest deadline 2024-09-11T07:53:46.033961+0000) 2024-09-11T07:53:49.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:48 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:48.685+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:49.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:48 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:48.685+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.266482+0000 front 2024-09-11T07:53:11.266496+0000 (oldest deadline 2024-09-11T07:53:35.966394+0000) 2024-09-11T07:53:49.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:48 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:48.685+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:15.966995+0000 front 2024-09-11T07:53:15.966926+0000 (oldest deadline 2024-09-11T07:53:41.266809+0000) 2024-09-11T07:53:50.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:49 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:49.726+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:50.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:49 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:49.726+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.266482+0000 front 2024-09-11T07:53:11.266496+0000 (oldest deadline 2024-09-11T07:53:35.966394+0000) 2024-09-11T07:53:50.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:49 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:49.726+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:15.966995+0000 front 2024-09-11T07:53:15.966926+0000 (oldest deadline 2024-09-11T07:53:41.266809+0000) 2024-09-11T07:53:50.030 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:49 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:49.850+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.531177+0000 front 2024-09-11T07:53:01.531220+0000 (oldest deadline 2024-09-11T07:53:26.231172+0000) 2024-09-11T07:53:50.030 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:49 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:49.850+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.432503+0000 front 2024-09-11T07:53:11.432433+0000 (oldest deadline 2024-09-11T07:53:34.332360+0000) 2024-09-11T07:53:50.030 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:49 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:49.850+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:15.433501+0000 front 2024-09-11T07:53:15.433561+0000 (oldest deadline 2024-09-11T07:53:46.033961+0000) 2024-09-11T07:53:50.030 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:49 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:49.850+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6815 osd.3 since back 2024-09-11T07:53:26.034375+0000 front 2024-09-11T07:53:26.034415+0000 (oldest deadline 2024-09-11T07:53:49.534333+0000) 2024-09-11T07:53:51.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:50 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:50.733+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:51.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:50 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:50.733+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.266482+0000 front 2024-09-11T07:53:11.266496+0000 (oldest deadline 2024-09-11T07:53:35.966394+0000) 2024-09-11T07:53:51.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:50 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:50.733+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:15.966995+0000 front 2024-09-11T07:53:15.966926+0000 (oldest deadline 2024-09-11T07:53:41.266809+0000) 2024-09-11T07:53:51.030 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:50 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:50.819+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.531177+0000 front 2024-09-11T07:53:01.531220+0000 (oldest deadline 2024-09-11T07:53:26.231172+0000) 2024-09-11T07:53:51.030 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:50 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:50.819+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.432503+0000 front 2024-09-11T07:53:11.432433+0000 (oldest deadline 2024-09-11T07:53:34.332360+0000) 2024-09-11T07:53:51.030 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:50 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:50.819+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:15.433501+0000 front 2024-09-11T07:53:15.433561+0000 (oldest deadline 2024-09-11T07:53:46.033961+0000) 2024-09-11T07:53:51.030 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:50 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:50.819+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6815 osd.3 since back 2024-09-11T07:53:26.034375+0000 front 2024-09-11T07:53:26.034415+0000 (oldest deadline 2024-09-11T07:53:49.534333+0000) 2024-09-11T07:53:52.029 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:51 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:51.859+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:01.531177+0000 front 2024-09-11T07:53:01.531220+0000 (oldest deadline 2024-09-11T07:53:26.231172+0000) 2024-09-11T07:53:52.029 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:51 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:51.859+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.432503+0000 front 2024-09-11T07:53:11.432433+0000 (oldest deadline 2024-09-11T07:53:34.332360+0000) 2024-09-11T07:53:52.030 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:51 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:51.859+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:15.433501+0000 front 2024-09-11T07:53:15.433561+0000 (oldest deadline 2024-09-11T07:53:46.033961+0000) 2024-09-11T07:53:52.030 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:51 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6[49942]: 2024-09-11T07:53:51.859+0000 7f655e263640 -1 osd.6 534 heartbeat_check: no reply from 172.21.15.22:6815 osd.3 since back 2024-09-11T07:53:26.034375+0000 front 2024-09-11T07:53:26.034415+0000 (oldest deadline 2024-09-11T07:53:49.534333+0000) 2024-09-11T07:53:52.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:51 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:51.693+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:52.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:51 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:51.693+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.266482+0000 front 2024-09-11T07:53:11.266496+0000 (oldest deadline 2024-09-11T07:53:35.966394+0000) 2024-09-11T07:53:52.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:51 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:51.693+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:15.966995+0000 front 2024-09-11T07:53:15.966926+0000 (oldest deadline 2024-09-11T07:53:41.266809+0000) 2024-09-11T07:53:52.030 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:51 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:51.693+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6815 osd.3 since back 2024-09-11T07:53:27.568827+0000 front 2024-09-11T07:53:27.568876+0000 (oldest deadline 2024-09-11T07:53:51.068795+0000) 2024-09-11T07:53:52.979 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:52 smithi104 podman[167015]: 2024-09-11 07:53:52.691131209 +0000 UTC m=+5.609129147 container died 2fe7f82ab136d67b3fdd20854e0e8eed04e8c27570ac111a184a6961688b6d8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True) 2024-09-11T07:53:52.979 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:52 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:52.685+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:52.979 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:52 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:52.685+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.266482+0000 front 2024-09-11T07:53:11.266496+0000 (oldest deadline 2024-09-11T07:53:35.966394+0000) 2024-09-11T07:53:52.979 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:52 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:52.685+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:15.966995+0000 front 2024-09-11T07:53:15.966926+0000 (oldest deadline 2024-09-11T07:53:41.266809+0000) 2024-09-11T07:53:52.979 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:52 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:52.685+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6815 osd.3 since back 2024-09-11T07:53:27.568827+0000 front 2024-09-11T07:53:27.568876+0000 (oldest deadline 2024-09-11T07:53:51.068795+0000) 2024-09-11T07:53:53.245 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:53 smithi104 podman[167015]: 2024-09-11 07:53:53.043000956 +0000 UTC m=+5.960998904 container cleanup 2fe7f82ab136d67b3fdd20854e0e8eed04e8c27570ac111a184a6961688b6d8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-11T07:53:53.245 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:53 smithi104 bash[167015]: ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6 2024-09-11T07:53:53.529 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:53 smithi104 podman[167033]: 2024-09-11 07:53:53.244197675 +0000 UTC m=+0.538270204 container remove 2fe7f82ab136d67b3fdd20854e0e8eed04e8c27570ac111a184a6961688b6d8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:53:53.908 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:53 smithi104 podman[167121]: 2024-09-11 07:53:53.557079963 +0000 UTC m=+0.015372864 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:53:53.908 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:53 smithi104 podman[167121]: 2024-09-11 07:53:53.670158158 +0000 UTC m=+0.128451049 container create e372e9b73898ee15872a78534614965c99d66325f1901e22a0ee1eb1f7c4d515 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True) 2024-09-11T07:53:53.908 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:53.669+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:53.908 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:53.669+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.266482+0000 front 2024-09-11T07:53:11.266496+0000 (oldest deadline 2024-09-11T07:53:35.966394+0000) 2024-09-11T07:53:53.909 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:53.669+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:15.966995+0000 front 2024-09-11T07:53:15.966926+0000 (oldest deadline 2024-09-11T07:53:41.266809+0000) 2024-09-11T07:53:53.909 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:53 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:53.669+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6815 osd.3 since back 2024-09-11T07:53:27.568827+0000 front 2024-09-11T07:53:27.568876+0000 (oldest deadline 2024-09-11T07:53:51.068795+0000) 2024-09-11T07:53:54.279 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:53 smithi104 podman[167121]: 2024-09-11 07:53:53.905737466 +0000 UTC m=+0.364030358 container init e372e9b73898ee15872a78534614965c99d66325f1901e22a0ee1eb1f7c4d515 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-11T07:53:54.279 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:53 smithi104 podman[167121]: 2024-09-11 07:53:53.909190727 +0000 UTC m=+0.367483633 container start e372e9b73898ee15872a78534614965c99d66325f1901e22a0ee1eb1f7c4d515 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-11T07:53:54.279 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:54 smithi104 podman[167121]: 2024-09-11 07:53:54.005681639 +0000 UTC m=+0.463974534 container attach e372e9b73898ee15872a78534614965c99d66325f1901e22a0ee1eb1f7c4d515 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T07:53:54.280 INFO:journalctl@ceph.osd.6.smithi104.stdout:Sep 11 07:53:54 smithi104 podman[167121]: 2024-09-11 07:53:54.091561713 +0000 UTC m=+0.549854616 container died e372e9b73898ee15872a78534614965c99d66325f1901e22a0ee1eb1f7c4d515 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-11T07:53:54.667 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:53:54 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:53:54.245Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi104.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.104:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi022.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.22:8443: connect: connection refused" 2024-09-11T07:53:54.667 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:53:54 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:53:54.246Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi022.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.22:8443: connect: connection refused" 2024-09-11T07:53:54.667 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:53:54 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:53:54.246Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi104.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.104:8443: connect: connection refused" 2024-09-11T07:53:54.677 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.6.service' 2024-09-11T07:53:54.716 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:53:54.716 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-11T07:53:54.716 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-11T07:53:54.716 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.7 2024-09-11T07:53:55.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:54.686+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:55.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:54.686+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.266482+0000 front 2024-09-11T07:53:11.266496+0000 (oldest deadline 2024-09-11T07:53:35.966394+0000) 2024-09-11T07:53:55.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:54.686+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:15.966995+0000 front 2024-09-11T07:53:15.966926+0000 (oldest deadline 2024-09-11T07:53:41.266809+0000) 2024-09-11T07:53:55.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:54 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:54.686+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6815 osd.3 since back 2024-09-11T07:53:27.568827+0000 front 2024-09-11T07:53:27.568876+0000 (oldest deadline 2024-09-11T07:53:51.068795+0000) 2024-09-11T07:53:55.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:54 smithi104 systemd[1]: Stopping Ceph osd.7 for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:53:55.529 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:55 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:55.070+0000 7f18b2e17640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:53:55.529 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:55 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:55.070+0000 7f18b2e17640 -1 osd.7 534 *** Got signal Terminated *** 2024-09-11T07:53:55.529 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:55 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:55.070+0000 7f18b2e17640 -1 osd.7 534 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:53:56.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:55 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:55.690+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:56.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:55 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:55.690+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.266482+0000 front 2024-09-11T07:53:11.266496+0000 (oldest deadline 2024-09-11T07:53:35.966394+0000) 2024-09-11T07:53:56.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:55 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:55.690+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:15.966995+0000 front 2024-09-11T07:53:15.966926+0000 (oldest deadline 2024-09-11T07:53:41.266809+0000) 2024-09-11T07:53:56.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:55 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:55.690+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6815 osd.3 since back 2024-09-11T07:53:27.568827+0000 front 2024-09-11T07:53:27.568876+0000 (oldest deadline 2024-09-11T07:53:51.068795+0000) 2024-09-11T07:53:57.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:56 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:56.697+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:57.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:56 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:56.698+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.266482+0000 front 2024-09-11T07:53:11.266496+0000 (oldest deadline 2024-09-11T07:53:35.966394+0000) 2024-09-11T07:53:57.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:56 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:56.698+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:15.966995+0000 front 2024-09-11T07:53:15.966926+0000 (oldest deadline 2024-09-11T07:53:41.266809+0000) 2024-09-11T07:53:57.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:56 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:56.698+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6815 osd.3 since back 2024-09-11T07:53:27.568827+0000 front 2024-09-11T07:53:27.568876+0000 (oldest deadline 2024-09-11T07:53:51.068795+0000) 2024-09-11T07:53:58.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:57 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:57.681+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:58.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:57 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:57.681+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.266482+0000 front 2024-09-11T07:53:11.266496+0000 (oldest deadline 2024-09-11T07:53:35.966394+0000) 2024-09-11T07:53:58.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:57 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:57.681+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:15.966995+0000 front 2024-09-11T07:53:15.966926+0000 (oldest deadline 2024-09-11T07:53:41.266809+0000) 2024-09-11T07:53:58.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:57 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:57.681+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6815 osd.3 since back 2024-09-11T07:53:27.568827+0000 front 2024-09-11T07:53:27.568876+0000 (oldest deadline 2024-09-11T07:53:51.068795+0000) 2024-09-11T07:53:59.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:58 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:58.700+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:53:59.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:58 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:58.700+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.266482+0000 front 2024-09-11T07:53:11.266496+0000 (oldest deadline 2024-09-11T07:53:35.966394+0000) 2024-09-11T07:53:59.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:58 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:58.700+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:15.966995+0000 front 2024-09-11T07:53:15.966926+0000 (oldest deadline 2024-09-11T07:53:41.266809+0000) 2024-09-11T07:53:59.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:58 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:58.700+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6815 osd.3 since back 2024-09-11T07:53:27.568827+0000 front 2024-09-11T07:53:27.568876+0000 (oldest deadline 2024-09-11T07:53:51.068795+0000) 2024-09-11T07:53:59.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:58 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:58.700+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.104:6802 osd.4 since back 2024-09-11T07:53:33.369714+0000 front 2024-09-11T07:53:33.369624+0000 (oldest deadline 2024-09-11T07:53:58.069565+0000) 2024-09-11T07:54:00.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:59 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:59.737+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6803 osd.0 since back 2024-09-11T07:53:02.065145+0000 front 2024-09-11T07:53:02.065049+0000 (oldest deadline 2024-09-11T07:53:26.665277+0000) 2024-09-11T07:54:00.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:59 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:59.737+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6807 osd.1 since back 2024-09-11T07:53:11.266482+0000 front 2024-09-11T07:53:11.266496+0000 (oldest deadline 2024-09-11T07:53:35.966394+0000) 2024-09-11T07:54:00.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:59 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:59.737+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6811 osd.2 since back 2024-09-11T07:53:15.966995+0000 front 2024-09-11T07:53:15.966926+0000 (oldest deadline 2024-09-11T07:53:41.266809+0000) 2024-09-11T07:54:00.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:59 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:59.737+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.22:6815 osd.3 since back 2024-09-11T07:53:27.568827+0000 front 2024-09-11T07:53:27.568876+0000 (oldest deadline 2024-09-11T07:53:51.068795+0000) 2024-09-11T07:54:00.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:53:59 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7[81445]: 2024-09-11T07:53:59.737+0000 7f18ae42d640 -1 osd.7 534 heartbeat_check: no reply from 172.21.15.104:6802 osd.4 since back 2024-09-11T07:53:33.369714+0000 front 2024-09-11T07:53:33.369624+0000 (oldest deadline 2024-09-11T07:53:58.069565+0000) 2024-09-11T07:54:00.779 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:54:00 smithi104 podman[167224]: 2024-09-11 07:54:00.516546323 +0000 UTC m=+5.575234579 container died 1bfaac5696d838d343422ce2b9c4763abacc2bb3c2f3033e67ad5eb285348862 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-11T07:54:01.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:54:00 smithi104 podman[167224]: 2024-09-11 07:54:00.961387752 +0000 UTC m=+6.020076074 container cleanup 1bfaac5696d838d343422ce2b9c4763abacc2bb3c2f3033e67ad5eb285348862 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:54:01.029 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:54:00 smithi104 bash[167224]: ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7 2024-09-11T07:54:01.725 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:54:01 smithi104 podman[167253]: 2024-09-11 07:54:01.354596036 +0000 UTC m=+0.838354696 container remove 1bfaac5696d838d343422ce2b9c4763abacc2bb3c2f3033e67ad5eb285348862 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-11T07:54:01.978 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:54:01 smithi104 podman[167341]: 2024-09-11 07:54:01.62735945 +0000 UTC m=+0.015619003 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:54:01.978 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:54:01 smithi104 podman[167341]: 2024-09-11 07:54:01.738689074 +0000 UTC m=+0.126948619 container create 93179faa1b3fd988be418ad7369fda89b0be57172287c7ed93894b17bdd8a57d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-11T07:54:01.978 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:54:01 smithi104 podman[167341]: 2024-09-11 07:54:01.974063393 +0000 UTC m=+0.362322938 container init 93179faa1b3fd988be418ad7369fda89b0be57172287c7ed93894b17bdd8a57d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:54:02.279 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:54:01 smithi104 podman[167341]: 2024-09-11 07:54:01.977361061 +0000 UTC m=+0.365620615 container start 93179faa1b3fd988be418ad7369fda89b0be57172287c7ed93894b17bdd8a57d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=) 2024-09-11T07:54:02.279 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:54:02 smithi104 podman[167341]: 2024-09-11 07:54:02.091160329 +0000 UTC m=+0.479419887 container attach 93179faa1b3fd988be418ad7369fda89b0be57172287c7ed93894b17bdd8a57d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-11T07:54:02.279 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 11 07:54:02 smithi104 podman[167341]: 2024-09-11 07:54:02.152895514 +0000 UTC m=+0.541155064 container died 93179faa1b3fd988be418ad7369fda89b0be57172287c7ed93894b17bdd8a57d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-09-11T07:54:02.760 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@osd.7.service' 2024-09-11T07:54:02.799 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:54:02.799 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-11T07:54:02.799 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-11T07:54:02.799 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-12a285d8-700e-11ef-bcea-c7b262605968@rgw.foo.a 2024-09-11T07:54:03.119 INFO:journalctl@ceph.rgw.foo.a.smithi022.stdout:Sep 11 07:54:02 smithi022 systemd[1]: Stopping Ceph rgw.foo.a for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:54:03.416 INFO:journalctl@ceph.rgw.foo.a.smithi022.stdout:Sep 11 07:54:03 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-rgw-foo-a[64022]: 2024-09-11T07:54:03.117+0000 7fdf73fff640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:54:03.416 INFO:journalctl@ceph.rgw.foo.a.smithi022.stdout:Sep 11 07:54:03 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-rgw-foo-a[64022]: 2024-09-11T07:54:03.117+0000 7fdf84c8b7c0 -1 shutting down 2024-09-11T07:54:04.666 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:54:04 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:54:04.246Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi022.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.22:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi104.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.104:8443: connect: connection refused" 2024-09-11T07:54:04.666 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:54:04 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:54:04.246Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi022.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.22:8443: connect: connection refused" 2024-09-11T07:54:04.666 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:54:04 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:54:04.247Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi104.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.104:8443: connect: connection refused" 2024-09-11T07:54:11.916 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:54:11 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:54:11.625Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi022.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.22:8443: connect: connection refused" 2024-09-11T07:54:11.917 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:54:11 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:54:11.626Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi104.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.104:8443: connect: connection refused" 2024-09-11T07:54:13.416 INFO:journalctl@ceph.rgw.foo.a.smithi022.stdout:Sep 11 07:54:13 smithi022 bash[223975]: time="2024-09-11T07:54:13Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-12a285d8-700e-11ef-bcea-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-11T07:54:13.416 INFO:journalctl@ceph.rgw.foo.a.smithi022.stdout:Sep 11 07:54:13 smithi022 podman[223975]: 2024-09-11 07:54:13.1322936 +0000 UTC m=+10.129800262 container died 83a6fd1a9ef37607602a7cbdfabd556dfca953882a6284c99f83cd2fab799e60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-rgw-foo-a, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2) 2024-09-11T07:54:13.916 INFO:journalctl@ceph.rgw.foo.a.smithi022.stdout:Sep 11 07:54:13 smithi022 podman[223975]: 2024-09-11 07:54:13.715854405 +0000 UTC m=+10.713361069 container cleanup 83a6fd1a9ef37607602a7cbdfabd556dfca953882a6284c99f83cd2fab799e60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-rgw-foo-a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-11T07:54:13.917 INFO:journalctl@ceph.rgw.foo.a.smithi022.stdout:Sep 11 07:54:13 smithi022 bash[223975]: ceph-12a285d8-700e-11ef-bcea-c7b262605968-rgw-foo-a 2024-09-11T07:54:14.162 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@rgw.foo.a.service' 2024-09-11T07:54:14.199 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:54:14.199 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-11T07:54:14.200 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-11T07:54:14.200 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-12a285d8-700e-11ef-bcea-c7b262605968@prometheus.a 2024-09-11T07:54:14.416 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:54:14 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:54:14.246Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi022.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.22:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi104.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.104:8443: connect: connection refused" 2024-09-11T07:54:14.416 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:54:14 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:54:14.247Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi022.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.22:8443: connect: connection refused" 2024-09-11T07:54:14.416 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:54:14 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:54:14.247Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi104.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.104:8443: connect: connection refused" 2024-09-11T07:54:14.488 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:54:14 smithi104 systemd[1]: Stopping Ceph prometheus.a for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:54:14.779 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:54:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:54:14.487Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-11T07:54:14.779 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:54:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:54:14.487Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-11T07:54:14.779 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:54:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:54:14.487Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-11T07:54:14.779 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:54:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:54:14.487Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-11T07:54:14.780 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:54:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:54:14.487Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-11T07:54:14.780 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:54:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:54:14.487Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-11T07:54:14.780 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:54:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:54:14.487Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-11T07:54:14.780 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:54:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:54:14.487Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-11T07:54:14.780 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:54:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:54:14.493Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-11T07:54:14.780 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:54:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:54:14.494Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-11T07:54:14.780 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:54:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:54:14.494Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-11T07:54:14.780 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:54:14 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a[64903]: ts=2024-09-11T07:54:14.494Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-11T07:54:14.780 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:54:14 smithi104 podman[167446]: 2024-09-11 07:54:14.497707735 +0000 UTC m=+0.105545799 container died 79e4627df0f1e0410710e4a310cd7df609aff1aefe0ab327c43d6651fd5c8107 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:54:15.197 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:54:14 smithi104 podman[167446]: 2024-09-11 07:54:14.961815412 +0000 UTC m=+0.569653477 container cleanup 79e4627df0f1e0410710e4a310cd7df609aff1aefe0ab327c43d6651fd5c8107 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:54:15.197 INFO:journalctl@ceph.prometheus.a.smithi104.stdout:Sep 11 07:54:14 smithi104 bash[167446]: ceph-12a285d8-700e-11ef-bcea-c7b262605968-prometheus-a 2024-09-11T07:54:15.412 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-12a285d8-700e-11ef-bcea-c7b262605968@prometheus.a.service' 2024-09-11T07:54:15.447 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:54:15.447 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-11T07:54:15.447 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 12a285d8-700e-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T07:54:17.480 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:54:17 smithi022 systemd[1]: Stopping Ceph alertmanager.a for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:54:17.480 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:54:17 smithi022 ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a[72526]: ts=2024-09-11T07:54:17.438Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-11T07:54:17.764 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:54:17 smithi022 podman[224416]: 2024-09-11 07:54:17.479211756 +0000 UTC m=+0.134142201 container died 18e7fd2d2738f439ee3ee99ca3783c563e95bc06fe17cff00ce3afeea03a502d (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:54:18.122 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:54:17 smithi022 podman[224416]: 2024-09-11 07:54:17.871134582 +0000 UTC m=+0.526065028 container cleanup 18e7fd2d2738f439ee3ee99ca3783c563e95bc06fe17cff00ce3afeea03a502d (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:54:18.123 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:54:17 smithi022 bash[224416]: ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a 2024-09-11T07:54:18.416 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:54:18 smithi022 podman[224428]: 2024-09-11 07:54:18.122114302 +0000 UTC m=+0.642515224 container remove 18e7fd2d2738f439ee3ee99ca3783c563e95bc06fe17cff00ce3afeea03a502d (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:54:18.417 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:54:18 smithi022 podman[224428]: 2024-09-11 07:54:18.206617644 +0000 UTC m=+0.727018569 volume remove b51e41168b703fb8c54a4e0ff2710f1fd1b4124dafdffe7fe8acd2b0e7c40df9 2024-09-11T07:54:18.417 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:54:18 smithi022 systemd[1]: ceph-12a285d8-700e-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-11T07:54:18.417 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:54:18 smithi022 systemd[1]: Stopped Ceph alertmanager.a for 12a285d8-700e-11ef-bcea-c7b262605968. 2024-09-11T07:54:18.417 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Sep 11 07:54:18 smithi022 systemd[1]: ceph-12a285d8-700e-11ef-bcea-c7b262605968@alertmanager.a.service: Consumed 1.938s CPU time. 2024-09-11T07:54:18.916 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:54:18 smithi022 systemd[1]: Stopping Ceph node-exporter.a for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:54:18.917 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:54:18 smithi022 podman[224542]: 2024-09-11 07:54:18.786019157 +0000 UTC m=+0.102289960 container died 494ea4af76a7526556334b0baa2251f388c5ae977eda9bb11f3793b0cd33b5b5 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T07:54:19.393 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:54:19 smithi022 podman[224542]: 2024-09-11 07:54:19.150432737 +0000 UTC m=+0.466703533 container cleanup 494ea4af76a7526556334b0baa2251f388c5ae977eda9bb11f3793b0cd33b5b5 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T07:54:19.394 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:54:19 smithi022 bash[224542]: ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a 2024-09-11T07:54:19.666 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:54:19 smithi022 podman[224554]: 2024-09-11 07:54:19.393221053 +0000 UTC m=+0.610772000 container remove 494ea4af76a7526556334b0baa2251f388c5ae977eda9bb11f3793b0cd33b5b5 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T07:54:19.666 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:54:19 smithi022 systemd[1]: ceph-12a285d8-700e-11ef-bcea-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-11T07:54:19.666 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:54:19 smithi022 systemd[1]: ceph-12a285d8-700e-11ef-bcea-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-11T07:54:19.666 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:54:19 smithi022 systemd[1]: Stopped Ceph node-exporter.a for 12a285d8-700e-11ef-bcea-c7b262605968. 2024-09-11T07:54:19.667 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Sep 11 07:54:19 smithi022 systemd[1]: ceph-12a285d8-700e-11ef-bcea-c7b262605968@node-exporter.a.service: Consumed 5.428s CPU time. 2024-09-11T07:54:20.232 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 12a285d8-700e-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T07:54:22.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:54:21 smithi104 systemd[1]: Stopping Ceph iscsi.iscsi.a for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:54:22.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:54:21 smithi104 ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a[60182]: debug Shutdown received 2024-09-11T07:54:32.117 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:54:31 smithi104 bash[167819]: time="2024-09-11T07:54:31Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-11T07:54:32.117 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:54:31 smithi104 podman[167819]: 2024-09-11 07:54:31.852382882 +0000 UTC m=+10.118228823 container died ec3dd728952b91d81ae74d4ea4aa9a079f9f33d0d07d32f7f0d27d0f8bd01a3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T07:54:32.515 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:54:32 smithi104 podman[167819]: 2024-09-11 07:54:32.280731063 +0000 UTC m=+10.546576994 container cleanup ec3dd728952b91d81ae74d4ea4aa9a079f9f33d0d07d32f7f0d27d0f8bd01a3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:54:32.516 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:54:32 smithi104 bash[167819]: ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a 2024-09-11T07:54:32.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:54:32 smithi104 podman[167832]: 2024-09-11 07:54:32.531975647 +0000 UTC m=+0.677237536 container remove ec3dd728952b91d81ae74d4ea4aa9a079f9f33d0d07d32f7f0d27d0f8bd01a3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-11T07:54:32.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:54:32 smithi104 systemd[1]: ceph-12a285d8-700e-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-11T07:54:33.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:54:32 smithi104 podman[167885]: 2024-09-11 07:54:32.878690813 +0000 UTC m=+0.139466014 container died c4542248f90409c787020ee6a50263409a03e9ca67971a345c810b86c3f02df4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:54:33.460 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:54:33 smithi104 podman[167885]: 2024-09-11 07:54:33.24265269 +0000 UTC m=+0.503427879 container cleanup c4542248f90409c787020ee6a50263409a03e9ca67971a345c810b86c3f02df4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-11T07:54:33.461 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:54:33 smithi104 bash[167885]: ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu 2024-09-11T07:54:33.461 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:54:33 smithi104 systemd[1]: ceph-12a285d8-700e-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-11T07:54:33.461 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:54:33 smithi104 systemd[1]: ceph-12a285d8-700e-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 60038 (conmon) remains running after unit stopped. 2024-09-11T07:54:33.461 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:54:33 smithi104 systemd[1]: ceph-12a285d8-700e-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 167902 (podman) remains running after unit stopped. 2024-09-11T07:54:33.461 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:54:33 smithi104 systemd[1]: Stopped Ceph iscsi.iscsi.a for 12a285d8-700e-11ef-bcea-c7b262605968. 2024-09-11T07:54:33.461 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:54:33 smithi104 systemd[1]: ceph-12a285d8-700e-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Consumed 3.167s CPU time. 2024-09-11T07:54:33.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi104.stdout:Sep 11 07:54:33 smithi104 podman[167902]: 2024-09-11 07:54:33.459092577 +0000 UTC m=+0.572223044 container remove c4542248f90409c787020ee6a50263409a03e9ca67971a345c810b86c3f02df4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-11T07:54:33.779 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:54:33 smithi104 systemd[1]: Stopping Ceph grafana.a for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:54:34.279 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:54:33 smithi104 podman[168023]: 2024-09-11 07:54:33.891351652 +0000 UTC m=+0.114118709 container died 5ed6f8ca9d7d4edc988ea0ee7d1bff9e1b8d521317573da52ef56357c7e75cb0 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a, name=ubi8, release=239.1651231664, vcs-type=git, io.openshift.tags=base rhel8, description=Ceph Grafana Container, maintainer=Paul Cuzner , vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.display-name=Red Hat Universal Base Image 8, distribution-scope=public, summary=Grafana Container configured for Ceph mgr/dashboard integration, version=8.5, io.buildah.version=1.32.0, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vendor=Red Hat, Inc., architecture=x86_64, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.component=ubi8-container, build-date=2022-04-29T11:48:11.055897, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.expose-services=) 2024-09-11T07:54:34.637 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:54:34 smithi104 podman[168023]: 2024-09-11 07:54:34.29689487 +0000 UTC m=+0.519661927 container cleanup 5ed6f8ca9d7d4edc988ea0ee7d1bff9e1b8d521317573da52ef56357c7e75cb0 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a, build-date=2022-04-29T11:48:11.055897, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, architecture=x86_64, release=239.1651231664, com.redhat.component=ubi8-container, io.buildah.version=1.32.0, io.openshift.tags=base rhel8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, summary=Grafana Container configured for Ceph mgr/dashboard integration, vendor=Red Hat, Inc., version=8.5, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., distribution-scope=public, name=ubi8, vcs-type=git, description=Ceph Grafana Container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, maintainer=Paul Cuzner , io.openshift.expose-services=) 2024-09-11T07:54:34.637 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:54:34 smithi104 bash[168023]: ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a 2024-09-11T07:54:34.637 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:54:34 smithi104 podman[168035]: 2024-09-11 07:54:34.531347103 +0000 UTC m=+0.640933200 container remove 5ed6f8ca9d7d4edc988ea0ee7d1bff9e1b8d521317573da52ef56357c7e75cb0 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-grafana-a, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.buildah.version=1.32.0, distribution-scope=public, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, build-date=2022-04-29T11:48:11.055897, release=239.1651231664, com.redhat.component=ubi8-container, description=Ceph Grafana Container, io.k8s.display-name=Red Hat Universal Base Image 8, vendor=Red Hat, Inc., io.openshift.expose-services=, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vcs-type=git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., name=ubi8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, version=8.5, io.openshift.tags=base rhel8, maintainer=Paul Cuzner , com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, architecture=x86_64) 2024-09-11T07:54:34.637 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:54:34 smithi104 systemd[1]: ceph-12a285d8-700e-11ef-bcea-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-11T07:54:35.029 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:54:34 smithi104 systemd[1]: Stopping Ceph node-exporter.b for 12a285d8-700e-11ef-bcea-c7b262605968... 2024-09-11T07:54:35.029 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:54:34 smithi104 systemd[1]: ceph-12a285d8-700e-11ef-bcea-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-11T07:54:35.029 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:54:34 smithi104 systemd[1]: Stopped Ceph grafana.a for 12a285d8-700e-11ef-bcea-c7b262605968. 2024-09-11T07:54:35.030 INFO:journalctl@ceph.grafana.a.smithi104.stdout:Sep 11 07:54:34 smithi104 systemd[1]: ceph-12a285d8-700e-11ef-bcea-c7b262605968@grafana.a.service: Consumed 14.160s CPU time. 2024-09-11T07:54:35.418 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:54:35 smithi104 podman[168143]: 2024-09-11 07:54:35.086784191 +0000 UTC m=+0.120169490 container died 5b33e3e5e5c0d6312f21e2248fdfa6d6280818a7e77e0488574cd75147e9bed3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T07:54:35.418 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:54:35 smithi104 podman[168143]: 2024-09-11 07:54:35.417457843 +0000 UTC m=+0.450843141 container cleanup 5b33e3e5e5c0d6312f21e2248fdfa6d6280818a7e77e0488574cd75147e9bed3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T07:54:35.779 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:54:35 smithi104 bash[168143]: ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b 2024-09-11T07:54:35.779 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:54:35 smithi104 podman[168157]: 2024-09-11 07:54:35.602121123 +0000 UTC m=+0.514763579 container remove 5b33e3e5e5c0d6312f21e2248fdfa6d6280818a7e77e0488574cd75147e9bed3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-12a285d8-700e-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T07:54:35.779 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:54:35 smithi104 systemd[1]: ceph-12a285d8-700e-11ef-bcea-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-11T07:54:36.250 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:54:35 smithi104 systemd[1]: ceph-12a285d8-700e-11ef-bcea-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-11T07:54:36.250 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:54:35 smithi104 systemd[1]: Stopped Ceph node-exporter.b for 12a285d8-700e-11ef-bcea-c7b262605968. 2024-09-11T07:54:36.250 INFO:journalctl@ceph.node-exporter.b.smithi104.stdout:Sep 11 07:54:35 smithi104 systemd[1]: ceph-12a285d8-700e-11ef-bcea-c7b262605968@node-exporter.b.service: Consumed 4.498s CPU time. 2024-09-11T07:54:36.542 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:54:36.578 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:54:36.606 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-11T07:54:36.608 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901210/remote/smithi022/crash 2024-09-11T07:54:36.609 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/crash -- . 2024-09-11T07:54:36.663 INFO:teuthology.orchestra.run.smithi022.stderr:tar: /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T07:54:36.664 INFO:teuthology.orchestra.run.smithi022.stderr:tar: Error is not recoverable: exiting now 2024-09-11T07:54:36.666 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901210/remote/smithi104/crash 2024-09-11T07:54:36.666 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/crash -- . 2024-09-11T07:54:36.700 INFO:teuthology.orchestra.run.smithi104.stderr:tar: /var/lib/ceph/12a285d8-700e-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T07:54:36.700 INFO:teuthology.orchestra.run.smithi104.stderr:tar: Error is not recoverable: exiting now 2024-09-11T07:54:36.701 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-11T07:54:36.702 DEBUG:teuthology.orchestra.run.smithi022:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2024-09-11T07:54:36.732 INFO:teuthology.orchestra.run.smithi022.stderr:grep: /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph.log: No such file or directory 2024-09-11T07:54:36.736 INFO:tasks.cephadm:Compressing logs... 2024-09-11T07:54:36.737 DEBUG:teuthology.orchestra.run.smithi022:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T07:54:36.780 DEBUG:teuthology.orchestra.run.smithi104:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T07:54:36.846 INFO:teuthology.orchestra.run.smithi022.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-11T07:54:36.846 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-11T07:54:36.846 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-11T07:54:36.846 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/objectstore_tool.2.log 2024-09-11T07:54:36.849 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T07:54:36.849 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-osd.1.log: /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/objectstore_tool.2.log: gzip -5 --verbose -- /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-11T07:54:36.849 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-11T07:54:36.849 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-11T07:54:36.849 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-11T07:54:36.849 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log 2024-09-11T07:54:36.849 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-11T07:54:36.849 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T07:54:36.849 INFO:teuthology.orchestra.run.smithi022.stderr: 58.6% -- replaced with /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-09-11T07:54:36.856 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.9% -- replaced with /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/objectstore_tool.2.log.gz 2024-09-11T07:54:36.857 INFO:teuthology.orchestra.run.smithi104.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-11T07:54:36.857 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/tcmu-runner.log 2024-09-11T07:54:36.857 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/objectstore_tool.2.log 2024-09-11T07:54:36.858 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/tcmu-runner.log: gzip -5 --verbose -- /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-11T07:54:36.858 INFO:teuthology.orchestra.run.smithi104.stderr: 63.5% -- replaced with /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/tcmu-runner.log.gz 2024-09-11T07:54:36.859 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T07:54:36.859 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/objectstore_tool.2.log: /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-11T07:54:36.860 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-11T07:54:36.860 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-11T07:54:36.861 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-11T07:54:36.861 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-11T07:54:36.861 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T07:54:36.927 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-mon.b.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-11T07:54:37.000 INFO:teuthology.orchestra.run.smithi104.stderr: 91.0% -- replaced with /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T07:54:37.031 INFO:teuthology.orchestra.run.smithi022.stderr: 95.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T07:54:37.075 INFO:teuthology.orchestra.run.smithi104.stderr: 95.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T07:54:37.174 INFO:teuthology.orchestra.run.smithi104.stderr: 95.1% -- replaced with /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/objectstore_tool.2.log.gz 2024-09-11T07:54:37.341 INFO:teuthology.orchestra.run.smithi022.stderr: 91.1% -- replaced with /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T07:54:37.762 INFO:teuthology.orchestra.run.smithi022.stderr: 89.2% -- replaced with /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-11T07:54:40.331 INFO:teuthology.orchestra.run.smithi104.stderr: 90.8% -- replaced with /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-11T07:54:44.774 INFO:teuthology.orchestra.run.smithi022.stderr: 90.9% -- replaced with /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-11T07:55:03.377 INFO:teuthology.orchestra.run.smithi022.stderr: 90.1% -- replaced with /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-11T07:57:47.548 INFO:teuthology.orchestra.run.smithi104.stderr: 93.3% -- replaced with /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-11T07:58:14.063 INFO:teuthology.orchestra.run.smithi104.stderr: 93.3% -- replaced with /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-11T07:58:38.024 INFO:teuthology.orchestra.run.smithi104.stderr: 93.1% -- replaced with /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-11T07:58:44.542 INFO:teuthology.orchestra.run.smithi104.stderr: 92.7% -- replaced with /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-11T07:58:44.545 INFO:teuthology.orchestra.run.smithi104.stderr: 2024-09-11T07:58:44.545 INFO:teuthology.orchestra.run.smithi104.stderr:real 4m7.749s 2024-09-11T07:58:44.545 INFO:teuthology.orchestra.run.smithi104.stderr:user 4m19.678s 2024-09-11T07:58:44.545 INFO:teuthology.orchestra.run.smithi104.stderr:sys 0m10.707s 2024-09-11T08:05:16.075 INFO:teuthology.orchestra.run.smithi022.stderr: 93.0% -- replaced with /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-11T08:07:52.970 INFO:teuthology.orchestra.run.smithi022.stderr: 93.1% -- replaced with /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-11T08:08:00.908 INFO:teuthology.orchestra.run.smithi022.stderr: 93.4% -- replaced with /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-11T08:09:58.122 INFO:teuthology.orchestra.run.smithi022.stderr: 92.3% -- replaced with /var/log/ceph/12a285d8-700e-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-11T08:09:58.123 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-09-11T08:09:58.123 INFO:teuthology.orchestra.run.smithi022.stderr:real 15m21.330s 2024-09-11T08:09:58.123 INFO:teuthology.orchestra.run.smithi022.stderr:user 7m12.898s 2024-09-11T08:09:58.123 INFO:teuthology.orchestra.run.smithi022.stderr:sys 0m19.726s 2024-09-11T08:09:58.125 INFO:tasks.cephadm:Archiving logs... 2024-09-11T08:09:58.437 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901210/remote/smithi022/log 2024-09-11T08:09:58.700 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T08:11:01.616 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901210/remote/smithi104/log 2024-09-11T08:11:01.617 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T08:11:43.383 INFO:tasks.cephadm:Removing cluster... 2024-09-11T08:11:43.384 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 12a285d8-700e-11ef-bcea-c7b262605968 --force 2024-09-11T08:11:44.483 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 12a285d8-700e-11ef-bcea-c7b262605968 --force 2024-09-11T08:11:45.356 INFO:tasks.cephadm:Removing cephadm ... 2024-09-11T08:11:45.356 DEBUG:teuthology.orchestra.run.smithi022:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T08:11:45.374 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T08:11:45.389 INFO:tasks.cephadm:Teardown complete 2024-09-11T08:11:45.390 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-11T08:11:45.399 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-11T08:11:45.400 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T08:11:45.417 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T08:11:45.523 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-11T08:11:45.536 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-11T08:11:45.536 DEBUG:teuthology.orchestra.run.smithi022:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T08:11:45.539 DEBUG:teuthology.orchestra.run.smithi104:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T08:11:45.552 INFO:teuthology.orchestra.run.smithi022.stderr:bash: line 1: ntpq: command not found 2024-09-11T08:11:45.553 INFO:teuthology.orchestra.run.smithi104.stderr:bash: line 1: ntpq: command not found 2024-09-11T08:11:45.569 INFO:teuthology.orchestra.run.smithi104.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T08:11:45.569 INFO:teuthology.orchestra.run.smithi104.stdout:=============================================================================== 2024-09-11T08:11:45.569 INFO:teuthology.orchestra.run.smithi104.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 24 -1203us[-1209us] +/- 70ms 2024-09-11T08:11:45.569 INFO:teuthology.orchestra.run.smithi104.stdout:^* hv02.front.sepia.ceph.com 3 6 377 21 +738us[ +732us] +/- 62ms 2024-09-11T08:11:45.570 INFO:teuthology.orchestra.run.smithi104.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 +335us[ +329us] +/- 61ms 2024-09-11T08:11:45.570 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:11:45.588 INFO:teuthology.orchestra.run.smithi022.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T08:11:45.588 INFO:teuthology.orchestra.run.smithi022.stdout:=============================================================================== 2024-09-11T08:11:45.588 INFO:teuthology.orchestra.run.smithi022.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 20 -1196us[-1196us] +/- 70ms 2024-09-11T08:11:45.588 INFO:teuthology.orchestra.run.smithi022.stdout:^* hv02.front.sepia.ceph.com 3 6 377 21 +719us[ +732us] +/- 63ms 2024-09-11T08:11:45.588 INFO:teuthology.orchestra.run.smithi022.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 +311us[ +311us] +/- 61ms 2024-09-11T08:11:45.588 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:11:45.589 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-11T08:11:45.600 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-11T08:11:45.600 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-11T08:11:45.660 DEBUG:teuthology.orchestra.run.smithi022:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T08:11:45.750 DEBUG:teuthology.orchestra.run.smithi104:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T08:11:45.821 DEBUG:teuthology.orchestra.run.smithi022:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T08:11:45.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:11:45.850 DEBUG:teuthology.orchestra.run.smithi104:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T08:11:45.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:11:45.877 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-11T08:11:45.886 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-11T08:11:45.896 INFO:teuthology.task.internal:Duration was 3549.054683 seconds 2024-09-11T08:11:45.897 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-11T08:11:45.906 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-11T08:11:45.906 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T08:11:45.908 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T08:11:45.984 INFO:teuthology.orchestra.run.smithi022.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T08:11:45.993 INFO:teuthology.orchestra.run.smithi104.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T08:11:46.338 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-11T08:11:46.338 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi022.front.sepia.ceph.com 2024-09-11T08:11:46.338 DEBUG:teuthology.orchestra.run.smithi022:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T08:11:46.370 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi104.front.sepia.ceph.com 2024-09-11T08:11:46.371 DEBUG:teuthology.orchestra.run.smithi104:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T08:11:46.395 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-11T08:11:46.395 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T08:11:46.413 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T08:11:46.465 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-11T08:11:46.465 DEBUG:teuthology.orchestra.run.smithi022:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T08:11:46.488 DEBUG:teuthology.orchestra.run.smithi104:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T08:11:48.559 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-11T08:11:48.569 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-11T08:11:48.569 DEBUG:teuthology.orchestra.run.smithi022:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T08:11:48.596 DEBUG:teuthology.orchestra.run.smithi104:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T08:11:48.628 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-11T08:11:48.638 DEBUG:teuthology.orchestra.run.smithi022:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:11:48.640 DEBUG:teuthology.orchestra.run.smithi104:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:11:48.687 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = core 2024-09-11T08:11:48.706 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = core 2024-09-11T08:11:48.756 DEBUG:teuthology.orchestra.run.smithi022:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:11:48.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:11:48.771 DEBUG:teuthology.orchestra.run.smithi104:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:11:48.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:11:48.814 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-11T08:11:48.826 INFO:teuthology.task.internal:Transferring archived files... 2024-09-11T08:11:48.826 DEBUG:teuthology.misc:Transferring archived files from smithi022:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901210/remote/smithi022 2024-09-11T08:11:48.827 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T08:11:48.908 DEBUG:teuthology.misc:Transferring archived files from smithi104:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901210/remote/smithi104 2024-09-11T08:11:48.908 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T08:11:48.952 INFO:teuthology.task.internal:Removing archive directory... 2024-09-11T08:11:48.952 DEBUG:teuthology.orchestra.run.smithi022:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T08:11:48.955 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T08:11:49.002 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-11T08:11:49.013 INFO:teuthology.task.internal:Not uploading archives. 2024-09-11T08:11:49.013 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-11T08:11:49.024 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-11T08:11:49.024 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T08:11:49.026 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T08:11:49.040 INFO:teuthology.orchestra.run.smithi022.stdout: 265811 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 08:11 /home/ubuntu/cephtest 2024-09-11T08:11:49.059 INFO:teuthology.orchestra.run.smithi104.stdout: 265972 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 08:11 /home/ubuntu/cephtest 2024-09-11T08:11:49.060 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-11T08:11:49.071 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-11T08:11:49.094 INFO:teuthology.run:Summary data: description: orch:cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} duration: 3549.054683446884 flavor: default owner: scheduled_gabrioux@teuthology success: true 2024-09-11T08:11:49.094 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T08:11:49.176 INFO:teuthology.run:pass